diff --git a/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.i18n.yaml b/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.i18n.yaml index efdb14ea94..ddb215be9f 100644 --- a/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.md -2026-06-17-filesystem-capability-seam.md: fee0161e5e8397ac1d1c0e2850efad840c65d971 -2026-06-17-filesystem-capability-seam.zh.md: 46e3ad22c530319d0d1b6ba23a8aba8ffc2fdb8c +2026-06-17-filesystem-capability-seam.md: 7436d2a9402c76f17c7d6571eb489a86e550ebfb +2026-06-17-filesystem-capability-seam.zh.md: fdeafba387b562352b377321327c526a67669ef6 diff --git a/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.md b/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.md index fee0161e5e..7436d2a940 100644 --- a/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.md +++ b/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.md @@ -62,8 +62,9 @@ The root `tool-fs` plugin registers the full filesystem tool suite (`read`, `wri The interface covers these semantic operations: - Resolve a model/plugin-supplied path into a backend-defined target. +- Convert a resolved target to the canonical process path or `file:` URI for the same execution world, and test containment without parsing its opaque key. - Stat target metadata without reading file contents. -- Read a bounded UTF-8 text page from a target. +- Read complete or streamed UTF-8 text; consumers apply their own view and retention limits. - Create or replace a UTF-8 text file. - Edit an existing UTF-8 text file by literal replacement. @@ -83,9 +84,11 @@ Resolved targets must expose at least three concepts: - An opaque `targetKey`, used for stale guards and file-state lookup. The local backend might use a realpath-like key; a remote backend might use a workspace URI or file id. Consumers must not parse or assume this is a local absolute path. - A `displayPath`, used for model/UI-facing output. It may be a local absolute path, workspace-relative path, or remote URI depending on the backend. +`targetKey` remains opaque even when another capability shares the provider's execution world. Such consumers ask the provider for `processPath(target)`, `fileUrl(target)`, or `contains(parent, child)`; the [portable execution-world decision](2026-07-28-portable-execution-world-consumers.md) owns why these facts sit on the filesystem seam. + Read and mutation results must include an opaque file `version`. The local backend derives its token from bigint stat metadata (`dev`, `ino`, `size`, `mtimeNs`, and `ctimeNs`) so same-size rewrites and inode replacement invalidate consumers reliably; a remote backend can use a revision id or hash-like token. The `dsh-fs-policy` plugin records versions for stale checks; consumers may display related metadata but must not interpret the version token. -The provider hands back decoded text: `readText` returns a whole regular text file, `streamText` streams the same text semantics for large files. Both own regular-file checks, bounded line/output handling is NOT theirs — line windowing, numbered-line rendering, and total-line accounting live in the executor (`dsh-tool-fs`), which reads through `ctx.fs` and renders the model-facing window. The provider owns UTF-8 decoding and binary/NUL rejection; it does not know about line windows or views. +The provider hands back decoded text: `readText` returns a whole regular text file and `streamText` streams the same text semantics for large files or consumer-owned retention limits. Line windowing, byte ceilings, numbered-line rendering, and total-line accounting live in consumers such as `dsh-tool-fs` and `dsh-lsp-local`. The provider owns regular-file checks, UTF-8 decoding, and binary/NUL rejection; it does not know about line windows, protocol limits, or views. Observed-state recording is not on `ctx.fs`: after a successful read the executor emits `fs/observed`, and the `dsh-fs-policy` plugin records `{ version }` for the deriving owner. There is no `full`/`partial` view — a read at any window records the version, and freshness (not view completeness) authorizes a later write/edit. diff --git a/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.zh.md b/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.zh.md index 46e3ad22c5..fdeafba387 100644 --- a/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.zh.md +++ b/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.zh.md @@ -16,7 +16,7 @@ harness 已有一个具体的 `bash` 能力 seam（`dsh-bash` / `dsh-bash-local` 如果没有 `ctx.fs` 接口，将本地文件系统访问替换为沙箱或远程后端时，即使面向模型的契约应当保持稳定，工具 schema、演示和提示词引导也会被迫变动。这还使权限/沙箱边界更难推理：一个 `cwd` 选项看起来像沙箱，但除非有显式的后端或 `tools/execute` 策略强制隔离，否则它只是一个基础路径。 -我们需要文件系统工具在成为公开包接口之前，以与 bash 相同的能力 seam 形态落地。 +我们需要文件系统工具在成为公开包（package）接口之前，以与 bash 相同的能力 seam 形态落地。 ## 决策 @@ -51,7 +51,7 @@ harness 已有一个具体的 `bash` 能力 seam（`dsh-bash` / `dsh-bash-local` `@deepseek-ai/dsh-fs-local` 依赖 `@deepseek-ai/dsh-fs` 和 `cordis`。它继承 `FileSystem`，将自身注册为 `ctx.fs`，拥有本地后端配置（如基目录），并包含所有直接的 `node:fs` / `node:path` 访问。它不持有观测状态存储——新鲜度是后端铸造、策略插件记录的版本令牌。 -`@deepseek-ai/dsh-tool-fs` 依赖 `@deepseek-ai/dsh-fs`、`@deepseek-ai/dsh-tools`、`@deepseek-ai/dsh-system-prompt` 和 `cordis`。它注册面向模型的工具和提示词段落。它禁止导入 `node:fs`、`node:path` 或 `@deepseek-ai/dsh-fs-local`；文件系统执行始终通过 `ctx.fs`。如果实现需要具体的 agent（智能体）或会话辅助类型，这些依赖属于 `tool-fs`；它们禁止回漏到 `dsh-fs` 中。 +`@deepseek-ai/dsh-tool-fs` 依赖 `@deepseek-ai/dsh-fs`、`@deepseek-ai/dsh-tools`、`@deepseek-ai/dsh-system-prompt` 和 `cordis`。它注册面向模型的工具和提示词段落。它禁止导入 `node:fs`、`node:path` 或 `@deepseek-ai/dsh-fs-local`；文件系统执行始终通过 `ctx.fs`。如果实现需要具体的 agent 或会话辅助类型，这些依赖属于 `tool-fs`；它们禁止回漏到 `dsh-fs` 中。 根 `tool-fs` 插件通过组合各工具的注册辅助函数来注册完整的文件系统工具套件（`read`、`write` 和 `edit`）。它注入 `fs`，从不导入实现包。 @@ -62,8 +62,9 @@ harness 已有一个具体的 `bash` 能力 seam（`dsh-bash` / `dsh-bash-local` 该接口涵盖以下语义操作： - 将模型/插件提供的路径解析为后端定义的目标。 +- 将解析后的目标转换为同一执行环境的规范进程路径或 `file:` URI，并在不解析其不透明键的情况下检查包含关系。 - 获取目标元数据而不读取文件内容。 -- 从目标读取有界的 UTF-8 文本页。 +- 读取完整或流式 UTF-8 文本；消费方执行各自的视图与保留上限。 - 创建或替换一个 UTF-8 文本文件。 - 通过字面替换编辑一个已有的 UTF-8 文本文件。 @@ -83,13 +84,15 @@ harness 已有一个具体的 `bash` 能力 seam（`dsh-bash` / `dsh-bash-local` - 不透明的 `targetKey`，用于陈旧守护和文件状态查找。本地后端可能使用类似 realpath 的键；远程后端可能使用工作区 URI 或文件 id。消费方禁止解析或假设它是本地绝对路径。 - `displayPath`，用于面向模型/UI 的输出。根据后端不同，它可能是本地绝对路径、工作区相对路径或远程 URI。 -读取和变更结果必须包含不透明的文件 `version`。本地后端从 bigint stat 元数据（`dev`、`ino`、`size`、`mtimeNs` 和 `ctimeNs`）派生令牌，因此同大小重写和 inode 替换都会可靠地使消费方持有的版本失效；远程后端可以使用 revision id 或类似 hash 的令牌。`dsh-fs-policy` 插件记录版本用于陈旧检查；消费方可以展示相关元数据但禁止解释版本令牌。 +即使另一项能力共享提供方的执行环境，`targetKey` 仍保持不透明。这类消费方通过提供方的 `processPath(target)`、`fileUrl(target)` 或 `contains(parent, child)` 获取所需事实；[可移植执行环境决策](2026-07-28-portable-execution-world-consumers.md)说明这些事实为何属于文件系统 seam。 -提供方返回已解码的文本：`readText` 返回整个常规文本文件，`streamText` 为大文件流式输出相同的文本语义。两者负责常规文件检查；有界的行/输出处理不是它们的职责——行窗口化、带行号渲染和总行数统计位于执行器（`dsh-tool-fs`）中，执行器通过 `ctx.fs` 读取并渲染面向模型的窗口。提供方负责 UTF-8 解码和二进制/NUL 拒绝；它不知道行窗口或视图。 +读取和变更结果必须包含不透明的文件 `version`。本地后端从 bigint stat 元数据（`dev`、`ino`、`size`、`mtimeNs` 和 `ctimeNs`）派生令牌，因此同大小重写和 inode 替换都会可靠地使消费方失效；远程后端可以使用 revision id 或类似 hash 的令牌。`dsh-fs-policy` 插件记录版本用于陈旧检查；消费方可以展示相关元数据但禁止解释版本令牌。 + +提供方返回已解码的文本：`readText` 返回整个普通文本文件，`streamText` 为大文件或消费方自有的保留上限流式传输相同的文本语义。行窗口化、字节上限、带行号渲染和总行数统计归 `dsh-tool-fs`、`dsh-lsp-local` 等消费方所有。提供方负责普通文件检查、UTF-8 解码和二进制／NUL 拒绝；它不知道行窗口、协议上限或视图。 观测状态记录不在 `ctx.fs` 上：成功读取后，执行器发出 `fs/observed`，`dsh-fs-policy` 插件为推导出的 owner 记录 `{ version }`。没有 `full`/`partial` 视图——任何窗口的读取都记录版本，新鲜度（而非视图完整性）授权后续的写入/编辑。 -全文件写入创建或替换 UTF-8 文本文件。后端在支持且有文档说明时可以创建父目录。已有的非常规目标被拒绝。`writeText` 接受一个可选期望：`createIfAbsent` 创建缺失的目标并拒绝已存在的（报 `FS_NOT_OBSERVED`，这是策略处理未观测 owner 时采用的分支）；`replaceIfVersion` 仅在目标处于观测版本时替换，否则报 `FS_STALE_VERSION`；省略期望则为无条件的裸提供方创建或覆盖。策略插件根据 owner 的观测状态选择提供哪个期望。 +全文件写入创建或替换 UTF-8 文本文件。后端在支持且有文档说明时可以创建父目录。已有的非常规目标被拒绝。`writeText` 接受一个可选期望：`createIfAbsent` 创建缺失的目标并拒绝已存在的（报 `FS_NOT_OBSERVED`，这是策略为未观测 owner 使用的路径）；`replaceIfVersion` 仅在目标处于观测版本时替换，否则报 `FS_STALE_VERSION`；省略期望则为无条件的裸提供方创建或覆盖。策略插件根据 owner 的观测状态选择提供哪个期望。 字面编辑是提供方原语（`editText`），而非在 `tool-fs` 中由读取加写入组合而成。字面匹配、重复匹配拒绝、CRLF 保留、二进制拒绝、可选的陈旧版本检查和原子读-改-写必须一起留在后端的变更临界区内。`editText` 接受相同的可选版本期望；陈旧检查在字面匹配之前运行，因此基于旧读取的编辑会报 `FS_STALE_VERSION`。远程后端可以将编辑实现为原生的 compare-and-edit 操作；消费方不强制本地风格的组合。 @@ -124,11 +127,11 @@ harness 已有一个具体的 `bash` 能力 seam（`dsh-bash` / `dsh-bash-local` ## 测试 -测试遵循包边界，而不仅是用户可见的工具：`dsh-fs` 中的服务 seam；`dsh-fs-local` 中通过 `ctx.fs` 接口测试的真实文件系统行为（解析、符号链接、流式输出、二进制/UTF-8 拒绝、无条件和版本守护的写入、字面编辑语义、行尾保留、结构化 `FsError` 错误码）；`dsh-tool-fs` 中基于真实本地提供方的消费方接口（只 mock 模型/时钟，从不 mock 协作者）；以及通过 `ctx.tools.execute()` 在有和没有 `dsh-fs-policy` 的情况下进行集成测试，通过从磁盘回读文件来验证世界状态，既不信任规范值，也不信任渲染内容。观测状态/owner 推导策略在 `dsh-fs-policy` 中测试，不在此处。 +测试遵循包边界，而不仅是用户可见的工具：`dsh-fs` 中的服务 seam；`dsh-fs-local` 中通过 `ctx.fs` 接口测试的真实文件系统行为（解析、符号链接、流式传输、二进制/UTF-8 拒绝、无条件和版本守护的写入、字面编辑语义、行尾保留、结构化 `FsError` 错误码）；`dsh-tool-fs` 中基于真实本地提供方的消费方接口（只 mock 模型/时钟，从不 mock 协作者）；以及通过 `ctx.tools.execute()` 在有和没有 `dsh-fs-policy` 的情况下进行集成测试，通过从磁盘回读文件来验证世界状态，既不信任规范值，也不信任渲染内容。观测状态/owner 推导策略在 `dsh-fs-policy` 中测试，不在此处。 本仓库曾踩过的防御性模式类别被直接固定： -- **原子写入临时文件安全。** 写入/编辑通过目标旁边一个私有随机 `0700` 目录中独占且仅所有者可访问（`'wx'`、`0o600`）的临时文件暂存，失败时清理，最后原子 rename——与 bash spill 文件规则一致，因为可预测的全局可读临时路径会招致符号链接竞争和信息泄露。测试断言权限，并断言已存在的临时路径不会被覆盖；此原语是 seam 的常设要求。 +- **原子写入临时文件安全。** 写入/编辑通过目标旁边一个私有随机 `0700` 目录中的独占 owner-only（`'wx'`、`0o600`）临时文件暂存，失败时清理，最后原子 rename——与 bash 溢出文件规则一致，因为可预测的 world-readable 临时路径招致符号链接竞争和信息泄露。测试断言权限，并断言已存在的临时路径不会被覆盖；此原语是 seam 的常设要求。 - **通过符号链接的 `targetKey` 同一性。** 两个输入路径解析到同一 realpath 时共享一个观测状态条目：通过路径 A 的 `read` 满足通过符号链接路径 B 的 `edit` 的读后编辑守护，通过一个路径的陈旧写入可通过另一个路径检测到。 - **并发/陈旧竞争。** 对同一目标的两个并发写入/编辑操作确定性地收敛——一个成功，另一个被 `FS_STALE_VERSION` 拒绝——成功的编辑刷新记录状态，使同一 owner 的下一次编辑可以继续。 - **HMR（热模块替换）安全与 dispose（资源释放）。** dispose 后端的 fiber 会撤回 `ctx.fs` 提供方；后续的提供方以无继承状态启动。 diff --git a/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.i18n.yaml index 11073946ea..aac1c86e11 100644 --- a/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md -2026-07-15-lsp-capability-seam.md: d96b3a9c5139c1455a51f4fff793293d7b5a11c0 -2026-07-15-lsp-capability-seam.zh.md: 256b293f213acb06588f3ef6c655242b1c8fd5b9 +2026-07-15-lsp-capability-seam.md: 63cec0a4e349ffc5be27a84e955b15e9168f898b +2026-07-15-lsp-capability-seam.zh.md: cdb1289f6956d8a4e46ed92847ddd8a58e588afa diff --git a/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md b/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md index d96b3a9c51..63cec0a4e3 100644 --- a/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md +++ b/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md @@ -62,7 +62,7 @@ interface LspProviderQuery extends LspQueryRequest { } type LspQueryResult = - | { readonly kind: 'locations'; readonly locations: readonly { readonly uri: string; readonly range: LspRange }[]; readonly resolvedWorkspaceRoot: string } + | { readonly kind: 'locations'; readonly locations: readonly { readonly uri: string; readonly range: LspRange }[]; readonly resolvedWorkspaceUri: string } | { readonly kind: 'hover'; readonly hover: { readonly contents: string; readonly range?: LspRange } | null } interface LspProvider { @@ -77,9 +77,9 @@ interface LspService { } ``` -Mapping keys normalize to lowercase, leading-dot extensions selected from `filePath`'s final extension; language ids only synchronize documents. Seam positions and ranges are zero-based UTF-16. `findReferences` always includes declarations: providers enforce this internally, the local mapping sets `context.includeDeclaration: true`, and callers get no flag. Closed result unions normalize navigation to locations and hover to content or `null`; navigation results carry the provider's resolved workspace root so consumers relativize file URIs in the same canonical namespace. The seam exposes no protocol types, process or document controls, or generic request escape hatch. +Mapping keys normalize to lowercase, leading-dot extensions selected from `filePath`'s final extension; language ids only synchronize documents. Seam positions and ranges are zero-based UTF-16. `findReferences` always includes declarations: providers enforce this internally, the local mapping sets `context.includeDeclaration: true`, and callers get no flag. Closed result unions normalize navigation to locations and hover to content or `null`; navigation results carry the provider's canonical workspace URI so consumers relativize file URIs in the execution world's namespace. The seam exposes no protocol types, process or document controls, or generic request escape hatch. -`dsh-lsp-local` owns host files, server configuration, JSON-RPC, process and transient-document state, and protocol translation; it depends on `dsh-lsp` and Node APIs, not `dsh-fs`. The server-table key is its provider id. The plugin resolves every server-local setting before registration, rolls back earlier registrations if a later mapping is invalid or conflicts, and retains an independent process pool per provider. `dsh-tool-lsp` runtime-injects only `tools`, `lsp`, and `systemPrompt`, obtains the workspace from `exec.agent?.session.header.cwd` through a package-local `sessionCwd(exec)` helper matching the filesystem tools' lookup, and imports no provider. +`dsh-lsp-local` owns server configuration, JSON-RPC, process and transient-document state, and protocol translation. It reads through `ctx.fs` and launches through `ctx.subprocess`, depending on their interface packages rather than concrete providers; the [portable execution-world decision](2026-07-28-portable-execution-world-consumers.md) owns that pairing. The server-table key is its provider id. The plugin resolves every server-local setting before registration, rolls back earlier registrations if a later mapping is invalid or conflicts, and retains an independent process pool per provider. `dsh-tool-lsp` runtime-injects only `tools`, `lsp`, and `systemPrompt`, obtains the workspace from `exec.agent?.session.header.cwd` through a package-local `sessionCwd(exec)` helper matching the filesystem tools' lookup, and imports no provider. ## Model-facing contract @@ -98,7 +98,7 @@ interface LspToolInput { The tool requires `workspaceRoot` from session `header.cwd`, with no fallback; absence fails as `LSP_WORKSPACE_REQUIRED` before querying or startup. The local provider resolves relative paths against that root and accepts absolute paths directly; both forms are canonicalized and rejected before startup when the target is outside the canonical workspace. -Locations render as stable, file-grouped `path:line:character` entries. A `file:` URI accepted by Node `fileURLToPath()` becomes a relative path inside the workspace or an absolute path outside it; other URIs remain verbatim. `maxLocations` defaults to `100` and reports omitted items; `maxResultChars` defaults to `16_000` and bounds every complete rendered result, including its truncation metadata. Empty locations and `null` hover are successful no-result responses; missing or malformed server payloads fail with structured `LSP_MALFORMED_RESPONSE` errors. +Locations render as stable, file-grouped `path:line:character` entries without applying harness-host path rules. A valid `file:` URI becomes a relative path inside the provider's canonical workspace URI or a URI-derived absolute path outside it; malformed and non-`file:` URIs remain verbatim. `maxLocations` defaults to `100` and reports omitted items; `maxResultChars` defaults to `16_000` and bounds every complete rendered result, including its truncation metadata. Empty locations and `null` hover are successful no-result responses; missing or malformed server payloads fail with structured `LSP_MALFORMED_RESPONSE` errors. The transport-neutral presenter uses `{ card: 'generic', kind: 'search', title, locations: [{ path: file_path, line }] }` with an args-derived operation/cursor `title`. Because `FileLocation` has no character, follow-along focuses the input line while the title preserves the cursor; presentation remains pure. @@ -112,26 +112,26 @@ Provider disposal occurs outside tool execution, so `dsh-lsp-local` keeps `shutd ## Workspace, filesystem, and document synchronization -`dsh-lsp-local` canonicalizes and reads through Node APIs in the subprocess's host namespace. It rejects missing, non-regular, non-UTF-8, oversized, or canonical out-of-workspace sources and keeps one `O_NOFOLLOW | O_NONBLOCK` handle through validation and reading, so a FIFO with no writer cannot block before the regular-file check. It observes caller cancellation around each filesystem operation. It does not consume `ctx.fs` or emit `fs/observed`: only the LSP result is model-visible, so the query does not satisfy read-before-write policy. +`dsh-lsp-local` canonicalizes and reads through `ctx.fs` in the language server's execution world. It requires the workspace target to be a directory, rejects out-of-workspace sources through provider-owned containment, consumes `streamText`, and enforces `maxDocumentBytes` as chunks arrive; the provider retains regular-file validation and UTF-8 decoding while the protocol consumer owns its document limit. It fuses caller cancellation with provider disposal across each filesystem operation, tracks workspace lookups before they enter a queue, and awaits those lookups during disposal. It does not emit `fs/observed`: only the LSP result is model-visible, so the query does not satisfy read-before-write policy. The `read` tool is unsuitable source because its output is windowed, numbered, transcript-visible, and observed. Reading in `tool-lsp` would also assign provider-specific synchronization to the consumer and preclude non-local providers. The local provider uses a compatibility-first transient-open sequence for every query. It accepts legacy `textDocumentSync` `Full` or `Incremental`, or options with `openClose: true`; omitted, `None`, or explicitly incompatible synchronization fails as unsupported before `didOpen`. -1. Canonicalize and validate the host path, then read the current source with Node filesystem APIs. +1. Resolve and contain the source through `ctx.fs`, then stream its current text through the same provider while enforcing the document byte limit. 2. Send `textDocument/didOpen` with version `1`, full text, and the configured language id. Its write remains abortable; failure or cancellation invalidates the instance and awaits bounded process termination before the pool can reuse it. 3. Send the requested `textDocument/definition`, `textDocument/references`, `textDocument/implementation`, or `textDocument/hover` request. 4. If `didOpen` succeeded, attempt `textDocument/didClose` in `finally` after the request settles or aborts. A close-write failure does not replace the settled result or error, but invalidates the instance and awaits bounded process termination. Documents close after each call, so the first version needs no `didChange`, `didSave`, content cache, mutation listener, or document LRU. One abortable per-workspace provider queue serializes source-read/open/query/close lifecycles, so a waiting query reads current bytes only when its turn starts; the instance also keeps protocol lifecycles serialized. Distinct workspaces may run in parallel. The server's workspace index remains responsible for closed files reached from the source. -The canonical workspace `realpath` must be a directory and supplies process cwd, `rootUri`, the sole `workspaceFolders` entry, and pool identity; symlink aliases therefore share an instance. Result locations may be external, but an external path cannot become a query source. Remote, virtual, or independently sandboxed filesystems require another provider. +The canonical workspace target must be a directory. Its target key supplies pool identity, its process path supplies cwd, and its provider-owned `file:` URI supplies `rootUri` and the sole `workspaceFolders` entry; aliases share an instance when the filesystem provider resolves them to one key. Result locations may be external, but an external path cannot become a query source. A filesystem that cannot share paths with the mounted subprocess provider is a composition error, not a reason for another LSP package. ## Local server lifecycle and protocol behavior -`dsh-lsp-local` lazily single-flights one server per `(provider id, canonical workspace realpath)`. At load it resolves the executable after credential scrubbing and environment overrides, failing before registration if unavailable; server process launch stays lazy (first query spawns it) and uses no shell. `maxMessageBytes` defaults to `16_000_000`, `maxStderrBytes` to `1_000_000`, and `maxDocumentBytes` to `4_000_000`. A crash fails the active query without replay; a later query may replace the process. Each query starts at most one process, so the MVP has no cross-request restart counter. +`dsh-lsp-local` lazily single-flights one server per `(provider id, canonical workspace target)`. At load it calls `ctx.subprocess.resolveExecutable()` with the configured environment, failing before registration if unavailable; first query launches through raw protocol pipes with no shell and a bounded collected stderr tail. `maxMessageBytes` defaults to `16_000_000`, `maxStderrBytes` to `1_000_000`, and `maxDocumentBytes` to `4_000_000`. A crash fails the active query without replay; a later query may replace the process. Each query starts at most one process, so the MVP has no cross-request restart counter. -Initialization advertises `general.positionEncodings: ['utf-16']`, `workspace: { workspaceFolders: true, configuration: true }`, `textDocument.hover.contentFormat: ['markdown', 'plaintext']`, and `linkSupport: true` for definition and implementation, with no dynamic registration. Returned operation and synchronization capabilities are authoritative. An omitted server `positionEncoding` defaults to `utf-16`; any other value is a protocol error. Configuration may supply initialization options and `workspace/configuration` responses, but the client rejects `workspace/applyEdit` and never executes commands or edits. +Initialization uses `processId: null` because the client and server may inhabit different process namespaces. It advertises `general.positionEncodings: ['utf-16']`, `workspace: { workspaceFolders: true, configuration: true }`, `textDocument.hover.contentFormat: ['markdown', 'plaintext']`, and `linkSupport: true` for definition and implementation, with no dynamic registration. Returned operation and synchronization capabilities are authoritative. An omitted server `positionEncoding` defaults to `utf-16`; any other value is a protocol error. Configuration may supply initialization options and `workspace/configuration` responses, but the client rejects `workspace/applyEdit` and never executes commands or edits. Navigation maps `Location` directly and `LocationLink` from `targetUri` plus `targetSelectionRange`. Positions must be nonnegative integers. Hover normalization accepts only valid `MarkupContent` and `MarkedString` shapes, preserves string values, renders language-tagged values as fenced code, and joins arrays with one blank line. The model-facing tool applies `maxResultChars` after rendering. @@ -143,7 +143,7 @@ Symbols are deferred because they need different schemas and overlap read/search Diagnostics need separate freshness, accumulation, and transcript rules. Mutations such as rename, code actions, and formatting require separate tools with preview, permission, and write-policy integration. -The local provider trusts its configured server and claims no sandbox confinement. Supporting untrusted binaries requires a later process/filesystem contract for workspace reads plus private cache and temporary writes; restricted, remote, or virtual workspaces require another provider. +The provider trusts its configured server. Its filesystem visibility and process confinement are exactly those of the mounted execution world; LSP adds no independent sandbox policy. ## Alternatives considered @@ -157,7 +157,7 @@ The local provider trusts its configured server and claims no sandbox confinemen **Wrap the signal in a per-seam execution-context object.** Web passes a bare `AbortSignal`; wrapping this single field would add unexplained asymmetry. `query()` gains a context object only when another field requires it. -**Read through `ctx.fs` or the `read` tool.** This could mix the document with a server index from another filesystem namespace; tool output is also windowed, numbered, and observed. The host-local provider reads unobserved full text beside its subprocess. +**Read through the model-facing `read` tool.** Rejected because tool output is windowed, numbered, transcript-visible, and observed. The provider consumes streamed full text directly through the same `ctx.fs` execution world used by its subprocess. **Keep documents open.** Mirroring edits requires version ownership, all-path `didChange`, HMR recovery, eviction, and stale-state rules. Transient opens avoid that MVP state machine. @@ -180,7 +180,7 @@ The local provider trusts its configured server and claims no sandbox confinemen - Synchronization tests pin UTF-16 negotiation and conversion, supported and rejected `textDocumentSync` forms, blocked and failed open writes, balanced transient open/close, close-write failure, and malformed-response rejection. - Timeout tests pin one `TOOL_TIMEOUT` budget, unclassified upstream cancellation, no hidden seam deadline, and bounded awaited teardown. - Lifecycle tests pin startup single-flight, complete-lifecycle serialization with fresh queued source reads, cross-workspace parallelism, abortable queues, crash replacement without replay, failed-stdin teardown, and quiescent disposal. -- Host-filesystem tests pin session-cwd requirements, relative and absolute source containment through symlinks, document validation, file/non-file URI rendering, unformatted source, and no `fs/observed` event. +- Filesystem-host tests pin session-cwd requirements, provider-owned containment and URI rendering, bounded document reads, unformatted source, and no `fs/observed` event. - A keyless pinned TypeScript real-server e2e exercises all four operations; runnable configuration uses the same explicit provider mapping. - Snapshots cover model-visible schema, prompt, results, and omissions; a built-artifact smoke test covers framing and cleanup. - Package and architecture docs cover configuration, security boundaries, and search/read guidance; the new `packages/lsp/` group is added to the AGENTS.md repository-layout block, the packages/README.md group table, and architecture.md in the same change. @@ -195,4 +195,4 @@ Extension ownership is exclusive within one runtime. Two providers cannot both c UTF-16 cursor columns are exact for the protocol but difficult for a model to count around non-BMP characters. Invalid or off-symbol positions may produce empty results, so error text and prompt examples must explain the coordinate convention without encouraging broad LSP use. -Direct Node access aligns the query snapshot with the server index but bypasses `ctx.fs` and its policy. Canonical containment rejects source files outside the workspace; a trusted server may still read the workspace and use caches. The first implementation therefore requires trusted host-local deployment and provides no sandbox guarantee. +The paired filesystem/subprocess providers align the query snapshot with the server index but do not make a trusted language server safe. Canonical containment rejects query sources outside the workspace at resolution time, but stream opening does not add stable-handle identity across a concurrent path replacement; the server itself receives the execution world's configured authority and may read other paths or use caches. diff --git a/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.zh.md b/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.zh.md index 256b293f21..cdb1289f69 100644 --- a/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.zh.md @@ -1,4 +1,4 @@ -# Agent Note: LSP 能力 seam 与面向模型的查询工具 +# Agent Note: LSP 能力服务边界与面向模型的查询工具 Status: implemented @@ -14,7 +14,7 @@ harness 已具备文本搜索与文件读取能力，但二者都无法识别程 ## 决策 -将 LSP 建成由三个包组成的能力 seam，其中包含一个只读模型工具和一个通用本地提供方实现： +将 LSP 建成由三个包（package）组成的能力服务边界，其中包含一个只读模型工具和一个通用本地提供方实现： 1. `packages/lsp/lsp` 下的 `@deepseek-ai/dsh-lsp` 负责 `ctx.lsp`、提供方注册与选择、标准化请求与结果、执行控制，以及结构化 LSP 错误。 2. `packages/lsp/lsp-local` 下的 `@deepseek-ai/dsh-lsp-local` 将配置的 stdio 语言服务器适配到该服务边界。一个插件实例接收具名服务器表，并为每组命令及扩展名到语言 id 的映射注册一个隔离的提供方。 @@ -62,7 +62,7 @@ interface LspProviderQuery extends LspQueryRequest { } type LspQueryResult = - | { readonly kind: 'locations'; readonly locations: readonly { readonly uri: string; readonly range: LspRange }[]; readonly resolvedWorkspaceRoot: string } + | { readonly kind: 'locations'; readonly locations: readonly { readonly uri: string; readonly range: LspRange }[]; readonly resolvedWorkspaceUri: string } | { readonly kind: 'hover'; readonly hover: { readonly contents: string; readonly range?: LspRange } | null } interface LspProvider { @@ -77,9 +77,9 @@ interface LspService { } ``` -映射键规范化为带前导点的小写扩展名，并按 `filePath` 的最后一个扩展名选择；语言 id 仅用于文档同步。服务边界中的位置和范围从零开始按 UTF-16 计数。`findReferences` 始终包含声明：提供方在内部执行该约束，本地映射设置 `context.includeDeclaration: true`，调用方不能配置。封闭结果联合将导航统一为位置，将 `hover` 统一为内容或 `null`；导航结果携带提供方解析后的工作区根目录，使消费方依据同一规范化根目录相对化文件 URI。服务边界不公开协议类型、进程或文档控制，也不提供通用请求逃生口。 +映射键规范化为带前导点的小写扩展名，并按 `filePath` 的最后一个扩展名选择；语言 id 仅用于文档同步。服务边界中的位置和范围从零开始按 UTF-16 计数。`findReferences` 始终包含声明：提供方在内部执行该约束，本地映射设置 `context.includeDeclaration: true`，调用方不能配置。封闭结果联合将导航统一为位置，将 `hover` 统一为内容或 `null`；导航结果携带提供方的规范工作区 URI，使消费方在执行世界的命名空间内相对化文件 URI。服务边界不公开协议类型、进程或文档控制，也不提供通用请求逃生口。 -`dsh-lsp-local` 负责主机文件、服务器配置、JSON-RPC、进程与临时文档状态和协议转换；它依赖 `dsh-lsp` 与 Node API，不依赖 `dsh-fs`。服务器表的键是提供方 id。插件在注册前解析每个服务器的本地设置；如果后续映射无效或发生冲突，插件会撤销此前的注册，并为每个提供方保留独立进程池。`dsh-tool-lsp` 在运行时只注入 `tools`、`lsp` 和 `systemPrompt`，通过包内的 `sessionCwd(exec)` 辅助函数从 `exec.agent?.session.header.cwd` 取得工作区，其取值方式与文件系统工具一致，也不导入提供方。 +`dsh-lsp-local` 负责服务器配置、JSON-RPC、进程与临时文档状态和协议转换。它通过 `ctx.fs` 读取，通过 `ctx.subprocess` 启动，只依赖二者的接口包而非具体提供方；[可移植执行环境决策](2026-07-28-portable-execution-world-consumers.md)负责定义这种配对。服务器表的键是提供方 id。插件在注册前解析每个服务器的本地设置；如果后续映射无效或发生冲突，插件会撤销此前的注册，并为每个提供方保留独立进程池。`dsh-tool-lsp` 在运行时只注入 `tools`、`lsp` 和 `systemPrompt`，通过包内的 `sessionCwd(exec)` 辅助函数从 `exec.agent?.session.header.cwd` 取得工作区，其取值方式与文件系统工具一致，也不导入提供方。 ## 面向模型的契约 @@ -98,9 +98,9 @@ interface LspToolInput { 工具必须从会话 `header.cwd` 取得 `workspaceRoot`，没有后备值；缺失时在查询或启动前以 `LSP_WORKSPACE_REQUIRED` 失败。本地提供方基于根目录解析相对路径并直接接受绝对路径；两种路径都会进行规范化，如果目标位于规范工作区外，则在启动前拒绝。 -位置按文件稳定分组并渲染为 `path:line:character`。Node `fileURLToPath()` 可接受的 `file:` URI 在工作区内转换为相对路径，在工作区外转换为绝对路径；其他 URI 保持原样。`maxLocations` 默认值为 `100`，并报告省略的条目；`maxResultChars` 默认值为 `16_000`，并将每个完整渲染结果（包括截断元数据）限制在该字符数内。空位置与 `null` hover 是成功的无结果响应；服务器载荷缺失或格式错误时，以结构化 `LSP_MALFORMED_RESPONSE` 错误失败。 +位置在不应用 harness 宿主路径规则的情况下按文件稳定分组并渲染为 `path:line:character`。有效的 `file:` URI 落在提供方的规范工作区 URI 内时转换为相对路径，位于其外时转换为从 URI 派生的绝对路径；格式错误的 URI 与非 `file:` URI 保持原样。`maxLocations` 默认值为 `100`，并报告省略的条目；`maxResultChars` 默认值为 `16_000`，并限制每个完整渲染结果，其中包括截断元数据。空位置与 `null` hover 是成功的无结果响应；服务器载荷缺失或格式错误时，以结构化 `LSP_MALFORMED_RESPONSE` 错误失败。 -与传输方式无关的展示器使用 `{ card: 'generic', kind: 'search', title, locations: [{ path: file_path, line }] }`，`title` 由参数推导并标明操作与光标。由于 `FileLocation` 没有 character，跟随位置聚焦输入行，标题保留完整光标；展示器仍为纯函数。 +与传输方式无关的展示器使用 `{ card: 'generic', kind: 'search', title, locations: [{ path: file_path, line }] }`，`title` 由参数推导并标明操作与光标。由于 `FileLocation` 没有 character，跟随位置聚焦输入行，标题保留完整光标；展示保持纯函数。 ## 超时归属 @@ -112,26 +112,26 @@ interface LspToolInput { ## 工作区、文件系统与文档同步 -`dsh-lsp-local` 通过 Node API 在子进程所在的主机命名空间中规范化并读取文件。它拒绝缺失、非普通、非 UTF-8、超大或规范路径越出工作区的源文件，并在校验与读取期间保持同一个 `O_NOFOLLOW | O_NONBLOCK` 句柄，因此没有写入方的 FIFO 不会在普通文件校验前造成阻塞。它在每项文件系统操作前后检查调用方是否取消。它不使用 `ctx.fs` 或发送 `fs/observed`：只有 LSP 结果对模型可见，因此查询不满足写前读取策略。 +`dsh-lsp-local` 在语言服务器的执行环境中通过 `ctx.fs` 规范化并读取文件。它要求工作区目标是目录，使用提供方自有的 containment 拒绝工作区外的源文件，消费 `streamText`，并在分片到达时执行 `maxDocumentBytes` 上限；普通文件校验和 UTF-8 解码仍由提供方负责，文档上限则由协议消费方负责。它会针对每项文件系统操作合并调用方取消与提供方资源释放，跟踪尚未进入队列的工作区查找，并在资源释放期间等待这些查找结算。它不发送 `fs/observed`：只有 LSP 结果对模型可见，因此查询不满足写前读取策略。 `read` 工具的输出带窗口与行号，进入 transcript（文本记录）且已被观察，不适合作为源文件。在 `tool-lsp` 内读取还会把提供方专用同步职责交给消费方，并排除非本地提供方。 本地提供方对每次查询都采用兼容优先的临时打开流程。它接受旧式 `textDocumentSync` 的 `Full` 或 `Incremental`，也接受设置了 `openClose: true` 的选项；同步能力缺失、为 `None` 或明确不兼容时，在 `didOpen` 前以不支持错误失败。 -1. 规范化并校验主机路径，再使用 Node 文件系统 API 读取当前源文件。 +1. 通过 `ctx.fs` 解析源文件并检查其位于工作区内，再通过同一提供方流式读取当前文本，同时执行文档字节上限。 2. 发送 `textDocument/didOpen`，其中包含版本 `1`、完整文本和配置的语言 id。该写入仍可取消；写入失败或遭取消会使实例失效，并等待有界进程终止完成，池才能复用它。 3. 发送所请求的 `textDocument/definition`、`textDocument/references`、`textDocument/implementation` 或 `textDocument/hover` 请求。 4. 如果 `didOpen` 成功，则在请求完成或取消后于 `finally` 中尝试发送 `textDocument/didClose`。关闭写入失败不会覆盖已经确定的结果或错误，但会使实例失效，并等待有界进程终止完成。 每次调用后都关闭文档，因此第一版不需要 `didChange`、`didSave`、内容缓存、变更监听器或文档 LRU。每个工作区的提供方队列可取消，并串行执行源文件读取、打开、查询和关闭的完整生命周期，因此等待中的查询只在轮到它时才读取当前字节；实例也会串行执行协议生命周期。不同工作区可以并行。服务器工作区索引仍负责从源文件跳转到的已关闭文件。 -规范工作区 `realpath` 必须是目录，并用于进程 cwd、`rootUri`、唯一的 `workspaceFolders` 条目和进程池 identity；符号链接别名因此共享实例。结果位置可以在工作区外，但外部路径不能成为查询源。远程、虚拟或独立沙箱化文件系统需要另一种提供方。 +规范工作区目标必须是目录。其目标键提供进程池 identity，进程路径提供 cwd，归提供方所有的 `file:` URI 则提供 `rootUri` 和唯一的 `workspaceFolders` 条目；文件系统提供方将别名解析为同一键时，它们共享实例。结果位置可以在工作区外，但外部路径不能成为查询源。无法与挂载的子进程提供方共享路径的文件系统属于组合错误，不是另建 LSP 包的理由。 ## 本地服务器生命周期与协议行为 -`dsh-lsp-local` 按 `(provider id, canonical workspace realpath)` 懒启动一个服务器，并通过 single-flight 合并启动。插件加载时，它在清除凭据并应用环境变量覆盖后解析可执行文件；命令不可用时在注册前失败。服务器进程的启动保持懒执行（首次查询时才拉起），且不经过 shell。`maxMessageBytes` 默认值为 `16_000_000`，`maxStderrBytes` 默认值为 `1_000_000`，`maxDocumentBytes` 默认值为 `4_000_000`。崩溃使当前查询失败且不重放；后续查询可以替换进程。每次查询最多启动一个进程，因此 MVP 不设置跨请求重启计数器。 +`dsh-lsp-local` 按 `(provider id, canonical workspace target)` 懒启动一个服务器，并通过 single-flight 合并启动。插件加载时，它使用已配置的环境调用 `ctx.subprocess.resolveExecutable()`；命令不可用时在注册前失败。首次查询通过原始协议管道启动服务器，不经过 shell，并收集有界的 stderr 尾部。`maxMessageBytes` 默认值为 `16_000_000`，`maxStderrBytes` 默认值为 `1_000_000`，`maxDocumentBytes` 默认值为 `4_000_000`。崩溃使当前查询失败且不重放；后续查询可以替换进程。每次查询最多启动一个进程，因此 MVP 不设置跨请求重启计数器。 -初始化声明 `general.positionEncodings: ['utf-16']`、`workspace: { workspaceFolders: true, configuration: true }`、`textDocument.hover.contentFormat: ['markdown', 'plaintext']`，以及 definition 与 implementation 的 `linkSupport: true`，但不支持动态注册。服务器返回的操作能力与同步能力均为真源。服务器省略 `positionEncoding` 时默认为 `utf-16`；其他值均属于协议错误。配置可以提供初始化选项和 `workspace/configuration` 响应，但客户端拒绝 `workspace/applyEdit`，绝不执行命令或编辑。 +初始化使用 `processId: null`，因为客户端与服务器可能位于不同的进程命名空间。它声明 `general.positionEncodings: ['utf-16']`、`workspace: { workspaceFolders: true, configuration: true }`、`textDocument.hover.contentFormat: ['markdown', 'plaintext']`，以及 definition 与 implementation 的 `linkSupport: true`，但不支持动态注册。服务器返回的操作与同步能力均为真源。服务器省略 `positionEncoding` 时默认为 `utf-16`；其他值均属于协议错误。配置可以提供初始化选项和 `workspace/configuration` 响应，但客户端拒绝 `workspace/applyEdit`，绝不执行命令或编辑。 导航结果直接映射 `Location`，并将 `LocationLink` 的 `targetUri` 与 `targetSelectionRange` 映射为统一位置。位置必须是非负整数。`hover` 归一化只接受有效的 `MarkupContent` 和 `MarkedString` 结构，保留字符串值，把带语言标签的值渲染为围栏代码块，并以一个空行连接数组。面向模型的工具在渲染后应用 `maxResultChars`。 @@ -143,7 +143,7 @@ interface LspToolInput { 诊断需要独立的新鲜度、累积与 transcript 规则。重命名、代码操作和格式化等变更能力需要单独工具，并集成预览、权限和写入策略。 -本地提供方信任配置的服务器，不声称具备沙箱隔离。支持不受信任的二进制文件需要后续补充允许读取工作区，并执行私有缓存写入与临时写入的进程／文件系统契约；受限、远程或虚拟工作区需要另一种提供方。 +提供方信任配置的服务器。其文件系统可见性与进程隔离完全取决于挂载的执行环境；LSP 不增加独立的沙箱策略。 ## 备选方案 @@ -155,9 +155,9 @@ interface LspToolInput { **公开 `resolve(request)` / `query(spec)`。** 没有需要填充默认值的字段时，resolve 只会暴露提供方选择，而公开 spec 可能活过提供方释放或替换。单一操作让选择与调用共用注册生命周期。 -**将信号包装为服务边界专用的执行上下文对象。** Web 传递裸 `AbortSignal`；仅包装这一个字段会造成无谓的不对称。只有另一个字段确有需要时，`query()` 才引入上下文对象。 +**将信号包装为每服务边界的执行上下文对象。** Web 传递裸 `AbortSignal`；仅包装这一个字段会造成无谓的不对称。只有另一个字段确有需要时，`query()` 才引入上下文对象。 -**通过 `ctx.fs` 或 `read` 工具读取。** 这可能把文档与另一文件系统命名空间中的服务器索引混合；工具输出还带窗口、行号且已被观察。host-local 提供方在子进程旁读取未观察的完整文本。 +**通过面向模型的 `read` 工具读取。**拒绝，因为工具输出带窗口与行号，会进入 transcript 且已被观察。提供方直接通过子进程所用的同一 `ctx.fs` 执行环境消费流式传输的完整文本。 **保持文档打开。** 镜像编辑需要版本归属、覆盖所有路径的 `didChange`、HMR 恢复、淘汰和陈旧状态规则。临时打开避免在 MVP 引入这套状态机。 @@ -180,14 +180,14 @@ interface LspToolInput { - 同步测试固定 UTF-16 协商与转换、受支持和被拒绝的 `textDocumentSync` 形式、打开写入阻塞与失败、配对的临时打开/关闭、关闭写入失败和错误响应拒绝。 - 超时测试固定一个 `TOOL_TIMEOUT` 预算、不对上游取消错误分类、服务边界无隐藏截止时间，以及受限且等待完成的清理。 - 生命周期测试固定启动 single-flight、完整生命周期串行化及排队查询读取最新源文件、跨工作区并行、可取消队列、崩溃后不重放的替换、stdin 失败后的进程拆除，以及释放后完全停稳。 -- 主机文件系统测试固定 session cwd 要求、符号链接下相对与绝对源路径的规范 containment、文档校验、file/non-file URI 渲染、无格式源文本和不发送 `fs/observed`。 +- 文件系统宿主测试固定 session cwd 要求、提供方自有的 containment 与 URI 渲染、有界文档读取、无格式源文本和不发送 `fs/observed`。 - 无密钥且固定版本的 TypeScript 真实服务器 e2e 覆盖四种操作；可运行配置使用同一项显式提供方映射。 - 快照覆盖模型可见 schema、提示词、结果和省略提示；构建产物冒烟测试覆盖分帧与清理。 - 包与架构文档覆盖配置、安全边界和搜索/读取指导；同一改动中，新的 `packages/lsp/` 包组要加入 AGENTS.md 的仓库布局块、packages/README.md 的分组表和 architecture.md。 ## 影响 -各语言服务器对方法支持、能力解释和索引就绪时机的处理不同；LSP 没有统一的「索引完成」信号。不具备兼容临时打开同步能力的服务器不受支持，即使它能查询已关闭文档。受支持的服务器仍可能返回空结果或不完整结果，因此工具不承诺跨服务器完整性。固定的 TypeScript e2e 只建立一条兼容性基线，不代表跨语言承诺。 +各语言服务器对方法支持、能力解释和索引就绪时机的处理不同；LSP 没有统一的“索引完成”信号。无法声明兼容临时打开同步能力的服务器不受支持，即使它能查询已关闭文档。受支持的服务器仍可能返回空结果或不完整结果，因此工具不承诺跨服务器完整性。固定的 TypeScript e2e 只建立一条兼容性基线，不代表跨语言承诺。 临时打开会重复解析并产生通知。实例内串行会增加并发 agent 的延迟，长期运行的工作区进程则持续占用内存直到释放。 @@ -195,4 +195,4 @@ interface LspToolInput { UTF-16 光标列与协议完全一致，但模型难以在包含非 BMP 字符的文本中准确计数。无效位置或不在符号上的位置可能返回空结果，因此错误文本和提示词示例必须说明坐标约定，同时避免鼓励模型广泛使用 LSP。 -直接访问 Node 文件系统会对齐查询快照与服务器索引，但绕过 `ctx.fs` 及其策略。规范路径 containment 会拒绝工作区外的源文件；受信任的服务器仍可读取工作区并使用缓存。因此，第一版要求受信任的 host-local 部署，不提供沙箱保证。 +配对的文件系统／子进程提供方会对齐查询快照与服务器索引，但不会因此使受信任的语言服务器变得安全。规范 containment 会在解析时拒绝工作区外的查询源，但打开流不会在路径并发替换期间额外保证稳定句柄身份；服务器本身获得执行环境所配置的权限，仍可读取其他路径或使用缓存。 diff --git a/.agents/notes/implemented/architecture/2026-07-20-canonical-tool-output-contract.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-20-canonical-tool-output-contract.i18n.yaml index f8ce3ee5f5..680d2dd94a 100644 --- a/.agents/notes/implemented/architecture/2026-07-20-canonical-tool-output-contract.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-20-canonical-tool-output-contract.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-20-canonical-tool-output-contract.md -2026-07-20-canonical-tool-output-contract.md: b2de9480d2659153dfb8a76ee07438d12e5b07c3 -2026-07-20-canonical-tool-output-contract.zh.md: 1852ae849aec27ef3af28895e902d840af5de14d +2026-07-20-canonical-tool-output-contract.md: 2429e1c141ad8c8ee932c6d654a6ba74bc4f7618 +2026-07-20-canonical-tool-output-contract.zh.md: 26653ea769b8a642b70c4d3dd2f5ab907a70bd85 diff --git a/.agents/notes/implemented/architecture/2026-07-20-canonical-tool-output-contract.md b/.agents/notes/implemented/architecture/2026-07-20-canonical-tool-output-contract.md index b2de9480d2..2429e1c141 100644 --- a/.agents/notes/implemented/architecture/2026-07-20-canonical-tool-output-contract.md +++ b/.agents/notes/implemented/architecture/2026-07-20-canonical-tool-output-contract.md @@ -46,7 +46,7 @@ The first-party tools preserve their existing Native text while returning domain | `glob` | `{ paths: string[] }` | | `grep` | `{ matches: [{ path, lineNumber, line }] }` | | `web_search` / `web_fetch` | The normalized `WebSearchResult` / `WebFetchResult` | -| `lsp` | `{ kind: "locations", locations, resolvedWorkspaceRoot }` or `{ kind: "hover", hover }` | +| `lsp` | `{ kind: "locations", locations, resolvedWorkspaceUri }` or `{ kind: "hover", hover }` | | `bash` | `{ kind: "background", taskId }` or `{ kind: "foreground" } & BashRunResult` | | `terminal_open` / `terminal_list` / `terminal_send` / `terminal_read` / `terminal_signal` / `terminal_close` | Public session snapshots, bounded read/send DTOs, signal/close outcomes, or a background task handle | | `task_output` / `task_list` / `task_kill` | Public task snapshots without owner or notification bookkeeping | diff --git a/.agents/notes/implemented/architecture/2026-07-20-canonical-tool-output-contract.zh.md b/.agents/notes/implemented/architecture/2026-07-20-canonical-tool-output-contract.zh.md index 1852ae849a..26653ea769 100644 --- a/.agents/notes/implemented/architecture/2026-07-20-canonical-tool-output-contract.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-20-canonical-tool-output-contract.zh.md @@ -46,7 +46,7 @@ type ToolExecutionResult = | `glob` | `{ paths: string[] }` | | `grep` | `{ matches: [{ path, lineNumber, line }] }` | | `web_search` ／ `web_fetch` | 归一化后的 `WebSearchResult` ／ `WebFetchResult` | -| `lsp` | `{ kind: "locations", locations, resolvedWorkspaceRoot }` 或 `{ kind: "hover", hover }` | +| `lsp` | `{ kind: "locations", locations, resolvedWorkspaceUri }` 或 `{ kind: "hover", hover }` | | `bash` | `{ kind: "background", taskId }` 或 `{ kind: "foreground" } & BashRunResult` | | `terminal_open` ／ `terminal_list` ／ `terminal_send` ／ `terminal_read` ／ `terminal_signal` ／ `terminal_close` | 公开会话快照、有界的读取／发送 DTO、信号／关闭操作结果，或后台任务句柄 | | `task_output` ／ `task_list` ／ `task_kill` | 不含所有者或通知管理信息的公开任务快照 | diff --git a/.agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.md b/.agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.md deleted file mode 100644 index 477dffc227..0000000000 --- a/.agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.md +++ /dev/null @@ -1,38 +0,0 @@ -# Agent Note: The subprocess seam goes Node-shaped and every eligible spawner rides it - -Status: implemented - -English | [中文](2026-07-26-subprocess-consumer-migration.zh.md) - -## Problem - -The [subprocess seam](2026-07-26-subprocess-seam.md) shipped shaped for exactly one consumer family: batch-collected stdout/stderr, batch stdin, a single escalating `kill()`. That was deliberate scope control, and its own note records "migrate the other spawn sites" as rejected-for-now. Review on the introducing PR reversed that deferral: the stacked follow-up should reshape the interface toward Node's API and move the remaining process-running places onto the service. The remaining spawners each carried a private copy of some slice of the same mechanics — lsp-local had its own detached-tree signalling (POSIX group + Windows taskkill + liveness polling), subagent-subprocess had the dispose ladder and its own scrub, and mcp-client, pty-local, the SDK helper, and the TUI Git probe each carried another credential scrub — and none of it was swappable or centrally testable. - -## Decision - -The seam's vocabulary is now Node-shaped, and every spawner that can ride the service does: - -- **Per-stream stdio dispositions** on `SubprocessSpawnSpec`: `'pipe'` (the raw `Readable`/`Writable`, for consumer-owned protocol framing), `'inherit'` (diagnostics to the parent's stream), and collect mode `{ maxBytes, spill? }` — the original bounded tail-keep shape, with the spill file now optional so a diagnostic tail (a language server's stderr) buffers without touching disk. stdin is `'ignore'`, `'pipe'`, or `{ data }` (write-and-close batch). -- **`SubprocessOutcome` carries exit facts only** (Node's close-event vocabulary); collected output stays readable through `handle.collected` after settlement (spill fds seal at the settle boundary), so batch and streaming callers share one access path and nothing is copied into the outcome. -- **Tree-scoped termination behind one verb**: `terminate()` owns the SIGTERM→grace→SIGKILL escalation (serves the spec's abort signal too, and is a no-op once the tree is gone) — the handle exposes no single-signal `kill(signal?)`, so a consumer cannot skip the grace window; `waitForExit()` polls tree liveness (POSIX group probe; direct-child boundary on Windows); Windows tree termination (`taskkill /T`, injectable) moved in from lsp-local, so tree semantics are platform-correct for every consumer. (The stdin-EOF-first dispose ladder initially absorbed from `subagent-subprocess` later moved back out to its one consumer — see the [ladder-ownership Agent Note](2026-07-27-dispose-ladder-to-consumer.md).) -- **One scrub definition**: `scrubbedParentEnv()`/`SENSITIVE_ENV_PATTERN` live on the seam, and credential-shaped names contain `KEY`, `PASSWORD`, `SECRET`, or `TOKEN`, case-insensitively. Spawners that cannot route the spawn itself through the service — pty-local (node-pty owns the fork), mcp-client (the MCP SDK owns the transport spawn), and the TUI's synchronous Git branch probe — import the function, so environment policy is single-sourced even where process ownership is not. The SDK helper's `scrubEnvironment()` defaults through the function and applies the exported pattern when its caller supplies an explicit environment; the pattern remains public for this production consumer. - -Migrations landed with the reshape: **bash-local/bash-sandbox** (collect modes + batch stdin; the bash `kill()` maps to `terminate()` so `task_kill` keeps escalation semantics), **lsp-local** (piped protocol streams + a no-spill collected stderr tail; `LspConnection` takes the seam's spawn function; its private tree-op helpers deleted), **subagent-acp** (piped ndjson streams + inherited stderr; spawn failure surfaces through `done` rejection into the same startup race; disposal is the backend-owned `disposeAcpChild` ladder over the seam's verbs, with the plugin's configured graces). **`dsh-subagent-subprocess` is deleted** — the dispose ladder and scrub are the seam's; the unused isolated-config-dir helper died with it (no consumer existed). - -Compositions mounting lsp-local or subagent-acp now load `dsh-subprocess-local` (the plugins inject `'subprocess'`); the acp/lsp test fixtures gained the row. - -## Alternatives considered - -**Keep the batch-only seam and let stream consumers stay bespoke.** The introducing note's position, rejected by review: it leaves three private copies of tree signalling and six of the scrub, and any future runner (containerized executor, remote process host) would have to pick which private copy to fork. The Node-shaped dispositions cover all three observed stream shapes without widening the outcome type or buffering piped streams. - -**A single `stdio: 'pipe' | 'inherit' | 'collect'` mode for all three streams at once.** Rejected: real consumers mix modes per stream (lsp: pipe/pipe/collect; acp: pipe/pipe/inherit; bash: data/collect/collect). Per-stream dispositions are exactly Node's shape and avoid a second spawn call for the mixed cases. - -**Migrate pty-local and mcp-client spawns too.** Rejected on ownership grounds, not scope: node-pty's `fork()` allocates the terminal itself, and the MCP SDK's `StdioClientTransport` spawns internally — neither call site is ours to route. They adopt the shared scrub (the part that is policy), and their READMEs say why the spawn stays put. - -**Migrate the test-support launchers (acp-snapshot, loader-smoke), the SDK package-manager runner, and the TUI Git probe.** Rejected: the support packages are deliberately dependency-light test infrastructure that must not depend on product seams; the SDK wizard's `stdio: 'inherit'`-with-redirect semantics plus its out-of-composition lifecycle (no cordis context at all) make the service a poor fit; and the TUI probe is synchronous. The production callers share the scrub instead. - -## Consequences - -Bought: one implementation of tree signalling, escalation, bounded collection, and the scrub, tested once in `dsh-subprocess-local`'s suites (including injected-platform Windows coverage that lsp-local's private copy never had); lsp-local and subagent-acp shed their process plumbing and their children now survive plugin reloads and die with composition teardown like bash's; a whole package (`dsh-subagent-subprocess`) is gone. The seam README's "one consumer family" limitation is retired. - -Cost: the seam is wider — three stdio modes and the terminate/waitForExit/dispose lifecycle surface instead of one mode and one verb — so a future backend implements more surface; the compositions for lsp-local/subagent-acp each carry the subprocess row now; and `SubprocessOutcome` no longer carries output, a breaking shape change inside the still-unreleased stack (the PR2 layer was updated in place rather than shimmed, per the pre-release stance). pty-local/mcp-client/SDK/TUI/test-support spawns remain outside the service by ownership or execution shape, with the scrub as the shared floor and its pattern intentionally exported for explicit-environment consumers. diff --git a/.agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.zh.md b/.agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.zh.md deleted file mode 100644 index 8e0e377fc3..0000000000 --- a/.agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.zh.md +++ /dev/null @@ -1,38 +0,0 @@ -# Agent Note: 子进程 seam 转向 Node 形状，所有具备条件的 spawn 调用点一并迁入 - -Status: implemented - -[English](2026-07-26-subprocess-consumer-migration.md) | 中文 - -## 问题 - -[子进程 seam](2026-07-26-subprocess-seam.md) 交付时恰好只为一个消费方家族塑形：批量收集的 stdout/stderr、批量 stdin、单一的升级式 `kill()`。那是有意的范围控制，其自身的 Agent Note 也把「迁移其余 spawn 调用点」记为暂缓否决项。引入该 seam 的 PR（Pull Request）上的评审推翻了这一暂缓决定：堆叠其上的后续变更应当把接口向 Node 的 API 方向重塑，并把其余运行进程之处迁到该服务上。其余各 spawn 调用点此前各自持有同一套机制中某个切片的私有副本——lsp-local 自带 detached 进程树信号发送（POSIX 进程组 + Windows taskkill + 存活轮询），subagent-subprocess 自带 dispose（资源释放）阶梯和自己的凭据清除，mcp-client、pty-local、SDK helper 与 TUI Git 探测则各自持有另一份凭据清除——而这一切既不可替换，也无法集中测试。 - -## 决策 - -这道 seam 的词汇如今已是 Node 形状，凡能接入该服务的 spawn 调用点均已迁入： - -- **按流划分的 stdio 处置方式（disposition）**，位于 `SubprocessSpawnSpec` 上：`'pipe'`（原始的 `Readable`/`Writable`，供消费方自有的协议分帧使用）、`'inherit'`（诊断输出直通父进程的流），以及收集模式（collect）`{ maxBytes, spill? }`——即最初的有界尾部保留形状，只是 spill 文件改为可选，使诊断尾部（例如语言服务器的 stderr）无需落盘即可缓冲。stdin 则为 `'ignore'`、`'pipe'` 或 `{ data }`（写完即关闭的批量形式）。 -- **`SubprocessOutcome` 只承载退出事实**（Node close 事件的词汇）；收集到的输出在结算后仍可经 `handle.collected` 读取（spill 文件描述符在结算边界封存），因此批量与流式调用方共用一条访问路径，也没有任何内容被复制进这份结果。 -- **以进程树为范围的终止，集中在一个动词后面**：`terminate()` 拥有 SIGTERM→宽限期→SIGKILL 升级（也承接 spec 的 abort 信号，进程树消亡后为空操作）——句柄不暴露单信号的 `kill(signal?)`，因此消费方无法跳过宽限窗口；`waitForExit()` 轮询进程树存活状态（POSIX 进程组探测；Windows 上以直接子进程为界）。Windows 进程树终止（`taskkill /T`，可注入）自 lsp-local 迁入，因此每个消费方拿到的进程树语义在各平台上都正确。（最初从 `subagent-subprocess` 吸收的以 stdin EOF 打头的 dispose 阶梯，后来又移回其唯一消费方——见[阶梯归属 Agent Note](2026-07-27-dispose-ladder-to-consumer.md)。） -- **凭据清除只有一份定义**：`scrubbedParentEnv()`/`SENSITIVE_ENV_PATTERN` 定义在 seam 上，且凭据形状的名称不区分大小写地包含 `KEY`、`PASSWORD`、`SECRET` 或 `TOKEN`。无法把 spawn 本身路由到该服务的调用点——pty-local（node-pty 拥有 fork）、mcp-client（MCP SDK 拥有传输层的 spawn）与 TUI 的同步 Git 分支探测——改为导入该函数，因此即便进程所有权无法统一，环境策略仍是单一来源。SDK helper 的 `scrubEnvironment()` 默认委托给该函数，并在调用方显式传入环境时应用导出的正则；该正则因此生产消费方而保持公开。 - -各项迁移随这次重塑一并落地：**bash-local/bash-sandbox**（收集模式 + 批量 stdin；bash 的 `kill()` 映射到 `terminate()`，因此 `task_kill` 保有升级语义），**lsp-local**（管道化的协议流 + 无 spill 的 stderr 收集尾部；`LspConnection` 改为接收 seam 的 spawn 函数；其私有的进程树操作辅助函数已删除），**subagent-acp**（管道化的 ndjson 流 + inherit 的 stderr；spawn 失败经 `done` 的 reject 汇入同一个启动竞态；dispose 是后端自有的 `disposeAcpChild` 阶梯，经由 seam 的动词运行，携带插件所配置的宽限期）。**`dsh-subagent-subprocess` 已删除**——dispose 阶梯与凭据清除归 seam 所有；无人使用的隔离配置目录辅助函数随之消亡（其消费方本就不存在）。 - -挂载 lsp-local 或 subagent-acp 的组合如今都加载 `dsh-subprocess-local`（这两个插件注入 `'subprocess'`）；acp/lsp 测试 fixture（测试前置数据）补上了这一行组合配置。 - -## 曾考虑的替代方案 - -**保持只支持批量的 seam，让流式消费方继续各自为政。**这正是引入该 seam 的 Agent Note 当初的立场，评审将其否决：这样会留下三份进程树信号发送的私有副本和六份凭据清除的私有副本，而未来任何运行器（容器化执行器、远程进程宿主）都得挑选去 fork 哪一份私有副本。Node 形状的处置方式覆盖已观察到的全部三种流形状，既不拓宽结果类型，也不缓冲管道化的流。 - -**用单个 `stdio: 'pipe' | 'inherit' | 'collect'` 模式一次性统辖全部三条流。**否决：真实消费方按流混用模式（lsp：pipe/pipe/collect；acp：pipe/pipe/inherit；bash：data/collect/collect）。按流划分的处置方式恰好就是 Node 的形状，也免去了混用场景的第二个 spawn 调用。 - -**把 pty-local 与 mcp-client 的 spawn 也一并迁移。**基于所有权而非范围否决：node-pty 的 `fork()` 自行分配终端，MCP SDK 的 `StdioClientTransport` 在内部完成 spawn——这两处调用点都不归我们路由。它们采纳共享的凭据清除（那正是属于策略的部分），并在各自的 README 中说明 spawn 为何留在原地。 - -**迁移 test-support 启动器（acp-snapshot、loader-smoke）、SDK package-manager 运行器与 TUI Git 探测。**否决：support 各包是刻意保持轻依赖的测试基础设施，不得依赖产品 seam；SDK 向导那套附带重定向的 `stdio: 'inherit'` 语义，加上其完全脱离组合的生命周期（根本没有 cordis 上下文），使该服务并不合用；TUI 探测则是同步调用。这些生产调用点改为共享凭据清除。 - -## 后果 - -换来的是：进程树信号发送、升级、有界收集与凭据清除各自只剩一份实现，且只在 `dsh-subprocess-local` 的测试套件中测试一次（其中包括 lsp-local 的私有副本从未有过的、以注入平台方式实现的 Windows 覆盖）；lsp-local 与 subagent-acp 卸下了自己的进程管道，其子进程如今像 bash 的一样，在插件重载后存活、随组合拆除而终止；一个完整的包（`dsh-subagent-subprocess`）就此消失。seam README 中「只有一个消费方家族」的限制说明也随之退役。 - -代价是：这道 seam 变宽了（stdio 模式从一种变为三种、生命周期接口面从一个动词扩展为 terminate/waitForExit/dispose 这一组），未来的后端因此要实现更宽的接口面；lsp-local/subagent-acp 的各组合如今都多出 subprocess 这一行组合配置；`SubprocessOutcome` 也不再承载输出，这是仍未发布的堆叠变更内部的一次破坏性形状变更（依照预发布立场，PR2 那一层被就地更新，而非加 shim）。pty-local/mcp-client/SDK/TUI/test-support 的 spawn 因所有权归属或执行形状留在该服务之外，以凭据清除作为共底线，且为显式环境的生产消费方有意保持该正则导出。 diff --git a/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.i18n.yaml index 1a918fc62c..f805e7cde3 100644 --- a/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md -2026-07-26-subprocess-seam.md: ad2f8522be51ba16b0df155aeb334a88f493890f -2026-07-26-subprocess-seam.zh.md: 575c02e346531824ef409ddc6155aa2b59ce4e63 +2026-07-26-subprocess-seam.md: 8797102c65ebab663bcf72fced5791364fe2c6ae +2026-07-26-subprocess-seam.zh.md: 8a7ff14079374d6d74a1ec729dd02c8961fa23e6 diff --git a/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md b/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md index ad2f8522be..8797102c65 100644 --- a/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md +++ b/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md @@ -12,8 +12,8 @@ English | [中文](2026-07-26-subprocess-seam.zh.md) A new `subprocess/` capability family owns "run and manage a process"; the bash family keeps "run a bash command" and consumes it: -- **`@deepseek-ai/dsh-subprocess` (interface)** — the abstract `SubprocessService` owning `ctx.subprocess` with one method, `spawn(spec): SubprocessHandle`, and the shared vocabulary: the fully-explicit `SubprocessSpawnSpec` (argv, cwd, per-stream stdio dispositions, grace — no defaults; deployment-varying knobs stay with the calling seam's config, per the `dsh-bash` request/spec template and the no-hidden-defaults rule), `SubprocessHandle` with non-consuming offset-based readers, `SubprocessOutcome` with deliberately no timeout/cancel classification, and the shared scrub plus `DSH_ENV_PREFIX`/`DshEnvironment`/`CollectedOutput` types. `argv` is never shell-interpreted. (The [consumer-migration Agent Note](2026-07-26-subprocess-consumer-migration.md) later widened the stdio and termination vocabulary Node-ward.) -- **`@deepseek-ai/dsh-subprocess-local` (implementation)** — `LocalSubprocessService` over the former `run.ts` plumbing (`spawn.ts`): detached groups, tail-keep truncation with private bounded spill files, credential scrub with the explicit-env merge after it, group kill escalation, and disposal that kills and joins every still-running managed process. It has no config; every limit arrives on the spec. The terminal `ENV_OVERRIDES` (`TERM=dumb` etc.) did NOT move — that is bash-tool presentation policy and stays in `dsh-bash-local`, merged through the ordinary env channel. +- **`@deepseek-ai/dsh-subprocess` (interface)** — the abstract `SubprocessService` owning `ctx.subprocess`: executable lookup, fully explicit ordinary spawns, and the terminal primitive added by the [portable execution-world decision](2026-07-28-portable-execution-world-consumers.md). Each stdio stream independently selects `'pipe'`, `'inherit'`, or bounded collection `{ maxBytes, spill? }`; stdin selects `'ignore'`, `'pipe'`, or `{ data }`. `SubprocessOutcome` carries exit facts with deliberately no timeout/cancel classification, while collected output remains on the handle after settlement. The seam also owns process and terminal handles, the shared scrub, and `DSH_ENV_PREFIX`/`DshEnvironment`/`CollectedOutput`; `argv` is never shell-interpreted. +- **`@deepseek-ai/dsh-subprocess-local` (implementation)** — `LocalSubprocessService` over the former `run.ts` plumbing (`spawn.ts`) plus `node-pty`: detached groups, bounded collection and private spill files, executable lookup, foreground/session inspection, and disposal that terminates and joins every managed process. `terminate()` owns TERM→grace→KILL for the tree, `waitForExit()` observes tree liveness, and injected `taskkill /T` covers Windows. Ordinary and terminal spawns apply the seam's case-insensitive `KEY`/`PASSWORD`/`SECRET`/`TOKEN` scrub before explicit env. The implementation has no config; every limit arrives on the spec, while Bash and PTY presentation environment overrides stay in their consumers. - **`dsh-bash-local` (consumer)** — `inject: ['subprocess']`; maps each resolved `BashExecSpec` onto a `SubprocessSpawnSpec` (`['bash', '-c', command]`), keeps its config, `resolve()` defaulting, fused-deadline `timedOut`/`aborted` classification, the `[stderr]`-marked background read merge with its consuming cursor, and the `onProcessDone` subclass hook. `dsh-bash-sandbox` is unchanged apart from redeclaring the inherited inject; it still wraps at the command-string level and re-enters the inherited spawn path. - **`dsh-bash` (seam)** — re-exports the moved vocabulary from `dsh-subprocess`, so no bash consumer changes an import; `BashExecRequest`/`BashExecSpec`/`BashProcess` and the sandbox facts remain bash-owned. @@ -21,11 +21,17 @@ Every composition that loads a bash executor now also loads `@deepseek-ai/dsh-su Background-process lifetime moved from the executor to the subprocess service: the executor no longer retains a live-process set, so an executor reload leaves background work running and readable, and composition teardown (the service's disposal) remains the kill-and-join boundary. One behavioral seam shifted with it: a background spawn failure can no longer be buffered as fake stderr inside the plumbing (the service rejects `done` and buffers nothing for a process that never ran), so the executor injects the `spawn failed: …` note into exactly one `readOutput()` delta. +Observed stream and lifecycle needs then moved the eligible process consumers onto the seam: LSP uses piped protocol streams plus a collected stderr tail; the ACP backend uses piped ndjson, inherited stderr, and a consumer-owned stdin-EOF disposal ladder; PTY uses `spawnTerminal()` while keeping readiness and terminal policy. `dsh-subagent-subprocess` and the private LSP tree helpers were deleted. MCP transport spawning, the SDK package-manager runner, synchronous TUI Git probing, and dependency-light test-support launchers remain outside by ownership or execution shape; their production callers share the scrub where applicable. + ## Alternatives considered **Leave the process plumbing inside `dsh-bash-local` (status quo).** Rejected for the same reason the [task registry split](2026-07-26-task-registry-seam.md) landed: the boundary is stable and already documented in-code (`run.ts`'s module doc said "this layer reacts to an abort signal; the executor owns deadlines and classifies causes"), and keeping it private makes every future non-shell runner either fork the mechanics or depend on a bash-named package for non-bash work. The user-visible driver for this stack was exactly this split. -**Migrate the repo's other spawn sites (lsp-local, pty-local, subagent-subprocess, sdk package-manager, test-support launchers) onto `ctx.subprocess` in the same change.** Rejected as scope creep with real design risk at this PR's scale: those sites have materially different stream and lifecycle needs — node-pty ownership (pty), LSP framing over long-lived stdio with tree-kill fallbacks (lsp), stdin-EOF-first disposal ladders and no output buffering (subagent transports) — and forcing them under a handle shaped for bounded batch output would either bloat the seam or misfit the consumers. The seam shipped proven against its one real consumer family, per the shape-interfaces-around-current-consumers rule. Review then asked for exactly that follow-up as a stacked PR; the [consumer-migration Agent Note](2026-07-26-subprocess-consumer-migration.md) records the Node-ward reshape and which sites moved (and which stayed, by ownership). +**Keep the original batch-only interface and leave stream consumers bespoke.** Rejected after the observed LSP, ACP, and PTY shapes showed that private process-tree signalling and environment scrubs would otherwise remain duplicated. The Node-shaped dispositions cover those consumers without buffering piped streams. + +**Use one `stdio: 'pipe' | 'inherit' | 'collect'` mode for all streams.** Rejected because real consumers mix modes per stream: LSP uses pipe/pipe/collect, ACP uses pipe/pipe/inherit, and Bash uses data/collect/collect. + +**Route every process launch through `ctx.subprocess`.** Rejected because the MCP SDK owns its transport spawn, the SDK wizard has no Cordis context and needs inherited redirection, the TUI probe is synchronous, and support launchers deliberately stay independent of product seams. PTY allocation did move behind `spawnTerminal()` because the provider, not the consumer, owns that substrate-specific primitive. **Put `run_in_background`/task semantics into the process seam instead.** Rejected: that boundary already exists — `ctx.tasks` owns ids, ownership, and notices, and the bash tool adapts a `BashProcess` into task hooks. The process seam sits *below* the bash executor, not beside the task registry. @@ -33,6 +39,6 @@ Background-process lifetime moved from the executor to the subprocess service: t ## Consequences -Bought: "run and manage a process" is a swappable capability with the standard three-package shape (consumer count starts at two: `bash-local`, `bash-sandbox`); a containerized or remote process backend slots in without touching bash semantics; the shared `DSH_*`/output vocabulary has a non-shell home; and background processes survive executor reloads, matching the task registry's lifetime model. The spawn plumbing suite moved wholesale to `dsh-subprocess-local` (argv-based, plus argv-validation and service lifecycle/disposal suites); the executor suite now pins the bash-owned layers (classification, merge, spawn-failure note, service-owned lifetime) against the real service. +Bought: "run and manage a process" is a swappable capability used by Bash, LSP, PTY, and ACP consumers; a containerized or remote process backend slots in without changing their domain semantics; tree signalling, escalation, bounded collection, terminal mechanics, and credential scrubbing each have one implementation; and background processes survive executor reloads, matching the task registry's lifetime model. Process and terminal plumbing is tested through `dsh-subprocess-local`; consumer suites pin only their owned behavior against the real service. -Cost: one more package pair and one more composition row everywhere a bash executor loads — a boot that loads an executor without the subprocess service leaves `ctx.bash` pending on `ctx.subprocess` (standard missing-service behavior). The moved-vocabulary re-exports keep `dsh-bash` imports working but mean two packages now name the same types; the subprocess seam is the owner and the bash seam documents the re-export. The spawn-failure note became single-delivery through the read path where the old plumbing retained it in the stderr buffer for repeated `readFrom(0)` reads — acceptable because the bash background read path was already a consuming cursor, and the note reaches the one reader that exists. +Cost: one more package pair and one more composition row wherever a consumer loads; a missing subprocess provider leaves the consumer pending by standard service-injection behavior. Every backend implements executable lookup, three stdio modes, tree lifecycle, and one terminal primitive. The moved-vocabulary re-exports keep `dsh-bash` imports working but mean two packages name the same types; the subprocess seam is the owner. The spawn-failure note became single-delivery through Bash's consuming read cursor instead of repeatable stderr-buffer content. diff --git a/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.zh.md b/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.zh.md index 575c02e346..8a7ff14079 100644 --- a/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.zh.md @@ -1,4 +1,4 @@ -# Agent Note: 子进程服务是 bash 执行器之下的独立 seam（`dsh-subprocess` / `dsh-subprocess-local`） +# Agent Note: 进程管理器是 bash 执行器之下的独立 seam（`dsh-subprocess` / `dsh-subprocess-local`） Status: implemented @@ -6,33 +6,39 @@ Status: implemented ## 问题 -`dsh-bash-local` 原先把两项因不同原因而变化的能力捆绑在一起：*运行一条 bash 命令*（命令默认值补全、超时分类、对模型友好的终端环境、bash 工具所渲染的 stdout/stderr 合并）与*运行并管理一个子进程*（detached 进程组、附带 spill 文件的有界尾部保留输出、凭据清除与 `DSH_*` 合并次序、SIGTERM→宽限期→SIGKILL 升级、先终止再等待退出的 dispose（资源释放））。进程这一半（`run.ts`）约占整个包的一半，却没有属于自己的 seam：未来的非 shell 运行器（直接执行 argv 的执行器、worker supervisor）将不得不重新实现这套机制，或者探入 bash 内部；而共享的 `DSH_*`/`CollectedOutput` 词汇则存放在一个名字承诺 shell 语义的包里。这种捆绑还把后台进程的存续期系在执行器的 fiber 上：重载 bash 执行器会杀死每一个存活的后台进程。这一点不同于同级的[任务注册表](2026-07-26-task-registry-seam.md)：后者的注册存续期刻意长于生产方 fiber。 +`dsh-bash-local` 原先把两项因不同原因而变化的能力捆绑在一起：*运行一条 bash 命令*（命令默认值补全、超时分类、对模型友好的终端环境、bash 工具所渲染的 stdout/stderr 合并）与*运行并管理一个子进程*（detached 进程组、附带 spill 文件的有界尾部保留输出、凭据清除与 `DSH_*` 合并次序、SIGTERM→宽限期→SIGKILL 升级、先终止再等待退出的 dispose（资源释放））。进程这一半（`run.ts`）约占整个包（package）的一半，却没有属于自己的 seam：未来的非 shell 运行器（直接执行 argv 的执行器、worker supervisor）将不得不重新实现这套机制，或者探入 bash 内部；而共享的 `DSH_*`/`CollectedOutput` 词汇则存放在一个名字承诺 shell 语义的包里。这种捆绑还把后台进程的存续期系在执行器的 fiber 上：重载 bash 执行器会杀死每一个存活的后台进程。这一点不同于兄弟的[任务注册表](2026-07-26-task-registry-seam.md)：后者的注册存续期刻意长于生产方 fiber。 ## 决策 新的 `subprocess/` 能力家族拥有「运行并管理一个进程」；bash 家族保留「运行一条 bash 命令」，并成为前者的消费方： -- **`@deepseek-ai/dsh-subprocess`（接口）**——拥有 `ctx.subprocess` 的抽象 `SubprocessService`（仅一个方法：`spawn(spec): SubprocessHandle`），以及共享词汇：完全显式的 `SubprocessSpawnSpec`（argv、cwd、按流划分的 stdio 处置方式（disposition）、宽限期，一律不设默认值；随部署变化的旋钮依照 `dsh-bash` 的 request/spec 模板与无隐藏默认值规则，留在调用方 seam 的配置里）、携带基于偏移量的非消费式读取器的 `SubprocessHandle`、刻意不含超时/取消分类的 `SubprocessOutcome`，以及共享的凭据清除与 `DSH_ENV_PREFIX`/`DshEnvironment`/`CollectedOutput` 类型。`argv` 绝不经过 shell 解释。（[消费方迁移 Agent Note](2026-07-26-subprocess-consumer-migration.md) 其后将 stdio 与终止词汇拓宽为 Node 形状。） -- **`@deepseek-ai/dsh-subprocess-local`（实现）**——`LocalSubprocessService`，构建在原 `run.ts` 管道（现为 `spawn.ts`）之上：detached 进程组、带私有有界 spill 文件的尾部保留截断、清除之后合并显式 env 的凭据清除、进程组 kill 升级，以及会终止每个仍在运行的受管进程并等待其退出的 dispose。该实现没有任何配置；每项限制都随 spec 到达。终端相关的 `ENV_OVERRIDES`（`TERM=dumb` 等）并未迁移：那是 bash 工具的呈现策略，留在 `dsh-bash-local` 里，经普通 env 通道合并。 +- **`@deepseek-ai/dsh-subprocess`（接口）**——拥有 `ctx.subprocess` 的抽象 `SubprocessService`：可执行文件查找、完全显式的普通 spawn，以及[可移植执行环境决策](2026-07-28-portable-execution-world-consumers.md)新增的终端原语。每条 stdio 流独立选择 `'pipe'`、`'inherit'` 或有界收集 `{ maxBytes, spill? }`；stdin 选择 `'ignore'`、`'pipe'` 或 `{ data }`。`SubprocessOutcome` 只承载刻意不含超时／取消分类的退出事实，收集输出在结算后仍留在句柄上。该 seam 还拥有进程与终端句柄、共享凭据清除，以及 `DSH_ENV_PREFIX`/`DshEnvironment`/`CollectedOutput`；`argv` 绝不经过 shell 解释。 +- **`@deepseek-ai/dsh-subprocess-local`（实现）**——`LocalSubprocessService` 构建在原 `run.ts` 管道（现为 `spawn.ts`）与 `node-pty` 之上：detached 进程组、有界收集与私有 spill 文件、可执行文件查找、前台／会话检查，以及终止每个受管进程并等待其退出的 dispose。`terminate()` 拥有面向进程树的 TERM→宽限→KILL，`waitForExit()` 观察进程树存活性，可注入的 `taskkill /T` 覆盖 Windows。普通与终端 spawn 都先应用 seam 对 `KEY`/`PASSWORD`/`SECRET`/`TOKEN` 不区分大小写的清除，再合并显式 env。该实现没有配置；每项限制都随 spec 到达，Bash 与 PTY 的呈现环境覆盖仍归各自消费方所有。 - **`dsh-bash-local`（消费方）**——`inject: ['subprocess']`；把每个解析后的 `BashExecSpec` 映射为一个 `SubprocessSpawnSpec`（`['bash', '-c', command]`），并保留自身配置、`resolve()` 默认值补全、基于融合 deadline 的 `timedOut`/`aborted` 分类、带 `[stderr]` 标记的后台读取合并及其消费游标，以及 `onProcessDone` 子类钩子。`dsh-bash-sandbox` 除了重新声明继承来的 inject 之外没有变化；它仍在命令字符串层面做包装，并重新进入继承的 spawn 路径。 - **`dsh-bash`（seam）**——把迁走的词汇从 `dsh-subprocess` 重导出，因此没有任何 bash 消费方需要改动导入；`BashExecRequest`/`BashExecSpec`/`BashProcess` 与沙箱事实仍归 bash 所有。 如今，每个加载 bash 执行器的组合都同时加载 `@deepseek-ai/dsh-subprocess-local`：CLI（命令行界面）、各示例、Python 捆绑运行时、create-sdk 的 bash 功能资源，以及各内联测试配置。 -后台进程的存续期从执行器移到了子进程服务：执行器不再保有存活进程集合，于是重载执行器后，后台工作会继续运行且仍可读取，而组合拆除（服务的 dispose）仍是先终止再等待退出的边界。一条行为 seam 随之挪动：后台 spawn 失败不再能在管道内部被缓冲成伪造的 stderr（对一个从未真正运行的进程，服务会 reject `done`，且不缓冲任何内容），因此执行器把 `spawn failed: …` 提示注入恰好一个 `readOutput()` 增量。 +后台进程的存续期从执行器移到了管理器：执行器不再保有存活进程集合，于是重载执行器后，后台工作会继续运行且仍可读取，而组合拆除（管理器的 dispose）仍是先终止再等待退出的边界。一条行为 seam 随之挪动：后台 spawn 失败不再能在管道内部被缓冲成伪造的 stderr（对一个从未真正运行的进程，管理器会 reject `done`，且不缓冲任何内容），因此执行器把 `spawn failed: …` 提示注入恰好一个 `readOutput()` 增量。 + +基于已观察到的流与生命周期需求，具备条件的进程消费方随后迁到该 seam：LSP 使用管道化协议流加收集式 stderr 尾部；ACP（Agent Client Protocol）后端使用管道化 ndjson、继承式 stderr 和消费方拥有的 stdin-EOF dispose 阶梯；PTY 使用 `spawnTerminal()`，同时保留就绪与终端策略。`dsh-subagent-subprocess` 与 LSP 私有进程树辅助函数均被删除。MCP 传输 spawn、SDK 包管理器运行器、同步 TUI Git 探测和刻意保持轻依赖的 test-support 启动器因所有权或执行形状仍留在外部；适用的生产调用方共享凭据清除。 ## 曾考虑的替代方案 **把进程管道留在 `dsh-bash-local` 里（维持现状）。**否决的理由与[任务注册表拆分](2026-07-26-task-registry-seam.md)得以落地的理由相同：这条边界既稳定，也早已记录在代码里（`run.ts` 的模块文档曾写明「this layer reacts to an abort signal; the executor owns deadlines and classifies causes」），而若继续将它保持私有，未来每个非 shell 运行器就只能要么 fork 这套机制，要么为非 bash 工作去依赖一个以 bash 命名的包。这组堆叠变更对用户可见的动因正是这一拆分。 -**在同一变更中把仓库其余 spawn 调用点（lsp-local、pty-local、subagent-subprocess、sdk package-manager、test-support 各启动器）迁到 `ctx.subprocess` 上。**在本 PR（Pull Request）的规模下，作为带有真实设计风险的范围蔓延否决。这些调用点在流与生命周期上的需求存在实质差异：node-pty 所有权（pty）、长生命周期 stdio 上的 LSP 分帧加进程树终止回退（lsp）、以 stdin EOF 打头的 dispose 阶梯和完全不缓冲输出（subagent 传输层）。把它们强行纳入一个按有界批量输出塑形的句柄之下，要么会让这道 seam 膨胀，要么会让句柄与消费方错配。依照「接口围绕当前消费方塑形」的规则，该 seam 当时在其唯一真实的消费方家族上得到验证后交付。评审随后恰恰要求以堆叠 PR 的形式完成这项后续工作；[消费方迁移 Agent Note](2026-07-26-subprocess-consumer-migration.md) 记录了向 Node 形状的重塑，以及哪些调用点迁入（哪些因所有权归属而留在原地）。 +**保留最初只支持批量的接口，让流式消费方继续各自实现。**否决：已观察到的 LSP、ACP 与 PTY 形状表明，这会继续保留重复的私有进程树信号与环境清除。Node 形状的处置方式覆盖这些消费方，又不缓冲管道化流。 + +**用单个 `stdio: 'pipe' | 'inherit' | 'collect'` 模式统一全部流。**否决：真实消费方按流混用模式——LSP 使用 pipe/pipe/collect，ACP 使用 pipe/pipe/inherit，Bash 使用 data/collect/collect。 + +**把每一次进程启动都路由到 `ctx.subprocess`。**否决：MCP SDK 拥有其传输 spawn，SDK 向导没有 Cordis 上下文且需要继承式重定向，TUI 探测是同步的，support 启动器则刻意独立于产品 seam。PTY 分配迁到 `spawnTerminal()`，因为这项底层专用原语归提供方而非消费方所有。 **改把 `run_in_background`/任务语义放进进程 seam。**否决：那条边界已经存在。`ctx.tasks` 拥有 id、所有权与通知，bash 工具则把 `BashProcess` 适配成任务钩子。进程 seam 位于 bash 执行器*之下*，而不是与任务注册表并列。 -**把 `ENV_OVERRIDES`（TERM=dumb、PAGER=cat 等）移入子进程服务。**否决：通用子进程服务不得把终端呈现策略强加给非终端消费方；对环境中凭据形态名称与 `DSH_*` 名称的清除是安全与身份不变式，予以保留，但终端友好性是 bash 工具自己的选择，经 spec 的显式 env 表达，而调用方自己的条目依旧优先。 +**把 `ENV_OVERRIDES`（TERM=dumb、PAGER=cat 等）移入管理器。**否决：通用进程管理器不得把终端呈现策略强加给非终端消费方；对环境中凭据形态名称与 `DSH_*` 名称的清除是安全与身份不变式，予以保留，但终端友好性是 bash 工具自己的选择，经 spec 的显式 env 表达，而调用方自己的条目依旧优先。 ## 后果 -换来的是：「运行并管理一个进程」成为一项具备标准三包形态的可替换能力（消费方起步就有两个：`bash-local`、`bash-sandbox`）；容器化或远程进程后端可以直接接入，而不触碰 bash 语义；共享的 `DSH_*`/输出词汇有了一个不带 shell 含义的归属；后台进程也能在执行器重载后存活，与任务注册表的存续期模型一致。spawn 管道测试套件整体迁至 `dsh-subprocess-local`（现以 argv 为基础，外加 argv 校验与服务生命周期/dispose 套件）；执行器测试套件如今以真实服务为基准，固定 bash 自有的各层行为（分类、合并、spawn 失败提示、归服务所有的存续期）。 +换来的是：「运行并管理一个进程」成为 Bash、LSP、PTY 与 ACP 消费方共用的可替换能力；容器化或远程进程后端可以直接接入，而无需改变各领域语义；进程树信号、升级终止、有界收集、终端机制与凭据清除各自只剩一份实现；后台进程也能在执行器重载后存活，与任务注册表的存续期模型一致。进程与终端管道通过 `dsh-subprocess-local` 测试；消费方测试套件只需针对真实服务固定各自拥有的行为。 -代价是：多出一对包，而且凡加载 bash 执行器之处都多一行组合配置。若某次启动加载了执行器却没有加载子进程服务，`ctx.bash` 会因等待 `ctx.subprocess` 而保持挂起（标准的服务缺失行为）。迁移词汇的重导出让 `dsh-bash` 的导入继续可用，但也意味着两个包如今命名同一批类型；进程 seam 是所有者，bash seam 则记录这层重导出。spawn 失败提示经由读取路径变为单次交付，而旧管道曾把它保留在 stderr 缓冲区里，供重复的 `readFrom(0)` 读取；这一点可以接受，因为 bash 的后台读取路径本就是消费游标，该提示能到达唯一存在的那个读取方。 +代价是：多出一对包，而且凡加载消费方之处都多一行组合配置；缺少 subprocess 提供方时，消费方会按标准服务注入行为保持挂起。每个后端都要实现可执行文件查找、三种 stdio 模式、进程树生命周期和一个终端原语。迁移词汇的重导出让 `dsh-bash` 的导入继续可用，但也意味着两个包命名同一批类型；进程 seam 是所有者。spawn 失败提示经由 Bash 的消费式读取游标变为单次交付，不再是可重复读取的 stderr 缓冲内容。 diff --git a/.agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml similarity index 54% rename from .agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.i18n.yaml rename to .agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml index daa727ccb0..8cd6ad419a 100644 --- a/.agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml @@ -1,6 +1,6 @@ # Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: -# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.md -2026-07-26-subprocess-consumer-migration.md: 477dffc2271db08b8986b34645067b247ad7ace7 -2026-07-26-subprocess-consumer-migration.zh.md: 8e0e377fc3fe00ff452803fe7fe5f4115003f937 +# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md +2026-07-28-portable-execution-world-consumers.md: 2cfdf08ac369048994ebc64d498caaa0847b77de +2026-07-28-portable-execution-world-consumers.zh.md: ce6650a284e7d64fb1277c758bb77c2394e942c8 diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md new file mode 100644 index 0000000000..2cfdf08ac3 --- /dev/null +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md @@ -0,0 +1,73 @@ +# Agent Note: Portable consumers over filesystem and subprocess execution worlds + +Status: implemented + +English | [中文](2026-07-28-portable-execution-world-consumers.zh.md) + +## Problem + +The filesystem and subprocess seams made file and ordinary process access replaceable, but PTY and LSP still reached host Node APIs directly. A remote execution provider therefore appeared to need separate PTY and LSP packages even though their domain behavior did not change. Those packages would be shallow adapters: each would duplicate an existing consumer merely to replace its file and process operations. + +A remote coding world is useful only when file operations, commands, terminals, and language servers share one sandbox identity. Moving the complete harness into that sandbox would also entangle provider experimentation with plugin loading, credentials, model transport, session durability, supervision, and deployment. + +Ordinary pipes do not cover one requirement. A persistent terminal needs PTY allocation, foreground-process-group inspection and signalling, and cleanup of the complete terminal session. Pretending those operations can be rebuilt in `dsh-pty-local` from an ordinary `spawn()` handle would either leak provider internals or weaken its lifecycle contract. + +## Decision + +`ctx.fs` and `ctx.subprocess` together define one execution world. Providers mounted together must describe the same path namespace, executables, processes, and terminal sessions; higher capabilities consume those two interfaces rather than name the provider. + +The filesystem interface owns the path facts that another capability needs without exposing its opaque target identity: a canonical process path, canonical `file:` URI, and containment. Existing whole and streaming text operations remain filesystem-owned; protocol consumers enforce their own retention limits while consuming the stream. + +The subprocess interface owns executable lookup and process primitives: ordinary raw or collected process spawning and `spawnTerminal()`. The terminal operation is one deep primitive whose handle owns text I/O, foreground groups, signalling, and one awaited TERM-to-KILL operation that settles in-flight handle calls and reaches quiescence for every session member the provider can still observe. Its signal cancels allocation only; the published handle owns its lifetime. Prompt detection, idle inference, scrollback, sandbox policy, and owner lifecycle remain in the PTY consumer. + +Generic consumers use that execution world: + +- `dsh-bash-local` continues to map Bash semantics onto ordinary `ctx.subprocess.spawn()`. +- `dsh-lsp-local` reads and contains source through `ctx.fs`, resolves and launches language servers through `ctx.subprocess`, and carries provider-owned file URIs through initialization and result rendering. One provider-lifetime signal aborts filesystem and protocol work during disposal, including workspace lookup before queue ownership; its JSON-RPC, pooling, synchronization, and normalization stay unchanged. +- `dsh-pty-local` maps persistent-shell semantics onto `ctx.subprocess.spawnTerminal()`. The local `node-pty` and process-inspection implementation moves into `dsh-subprocess-local`; another subprocess provider supplies the same primitive. `danger-full-access` needs no `ctx.sandbox`; a confined mode requires a same-world sandbox provider and fails before spawn when none is mounted. Prompt and silence evidence collected during asynchronous pre-write inspection is discarded when the provider write begins. Cancellation retains the send reservation while an in-flight write settles and then signals the foreground group, so late bytes or the signal cannot target a successor; an in-flight readiness poll cannot release that reservation, and a rejected write sends no signal. The absolute deadline remains armed throughout cancellation. A signal failure becomes terminal transport failure. Completion of a stale inspection resumes polling for the current send. Startup cancellation begins terminal rollback without waiting for a stalled readiness or signalling call. Close rejects new public signals and delegates provider-observable session quiescence to the handle's awaited termination operation. + +## E2B POC boundary + +The opt-in E2B realization has exactly three provider-specific packages under `packages/e2b/`: `dsh-e2b` creates one sandbox and deletes it on timeout or disposal, `dsh-fs-e2b` implements `ctx.fs`, and `dsh-subprocess-e2b` implements `ctx.subprocess` over E2B Commands, PTYs, and remote Linux process groups. The two adapters obtain the sole SDK handle from the owner and never create private sandboxes. + +E2B owns the mutable filesystem, managed command and Bash processes, terminal allocation and terminal-session groups, language-server processes and source reads, and adapter-private files under `.dsh-e2b`. The host owns Cordis and plugin objects, the agent loop, agent/session/goal state, session logs and persistence, LLM calls, prompts and tools, authority, skills, subagent orchestration, PTY buffers and readiness, LSP protocol state, and E2B SDK/network buffers. The overlay neither uploads nor synchronizes the host workspace. + +The adapters retain only substrate mechanics. Filesystem canonicalization crosses the SDK's decoded command transport as strict base64-encoded NUL framing; streamed reads leave byte ceilings with consumers. Subprocess command output and environment snapshots use ASCII/base64 where SDK chunk decoding would otherwise lose bytes, while private control shells isolate profiles and later launches blank discovered credential-shaped names. Process and terminal cleanup uses remote groups and proves quiescence before settlement. + +Sandbox state is deliberately ephemeral: timeout and disposal delete the remote files and unmanaged state. The POC adds no reconnect or pause/leave retention, session-persistence backend, template builder, volume, snapshot, network-policy layer, sandbox catalog, workspace synchronization, durable remote handles, or whole-harness execution. + +## Verification + +Focused package suites pin sandbox lifecycle, canonical path framing, filesystem metadata and atomic versions, subprocess publication/rollback, terminal text I/O and session cleanup, output limits, cancellation, disposal, and invariant registration. A credential-gated Loader composition exercises the same three-package provider through source imports and built exports, including FS/Bash visibility, hostile login profiles, byte-split UTF-8 output, process and terminal cleanup, LSP queries, host-workspace isolation, and final sandbox deletion. + +## Alternatives considered + +**Keep one PTY and LSP package per remote provider.** Rejected because provider mechanics would be repeated above the existing seams. The deletion test exposes the problem: deleting those adapters should not scatter domain behavior into the remote provider; the generic consumers already own it. + +**Create a separate sandbox per capability or tool.** Rejected because file and process operations would not share identity or state, defeating the coding use case and multiplying lifecycle owners. + +**Model a terminal as an ordinary piped subprocess.** Rejected because pipes cannot allocate a controlling terminal, resolve the current foreground process group, or prove complete terminal-session cleanup. One terminal primitive is smaller and more honest than exposing substrate-specific escape hatches. + +**Move PTY readiness and session policy into the subprocess service.** Rejected because those are persistent-terminal consumer semantics, not OS process mechanics. A subprocess provider owns what only its substrate can do; `dsh-pty-local` owns what a Harness terminal means. + +**Expose separate terminal termination and quiescence operations plus a shared lifecycle controller.** Rejected because every terminal consumer needs the same single cleanup outcome. Separate operations export provider bookkeeping, bounded-observer, and retry semantics without a production consumer; one awaited provider operation is a deeper interface. + +**Add a stable bounded-read primitive to the filesystem seam.** Rejected because only LSP needs a complete-document byte ceiling, which it can enforce while consuming the existing text stream. A second primitive forces every provider to implement stable-handle and no-follow mechanics, including a remote helper protocol, without an observed concurrent-replacement defect. + +**Run the whole harness inside the remote environment.** Rejected as a different deployment model. Making execution capabilities portable does not move model calls, session state, plugin state, or the agent loop. + +**Put every provider operation in one shared owner package.** Rejected because sandbox identity and lifecycle are the owner's only concerns. Filesystem and subprocess retain distinct contracts, tests, and consumers without turning the owner into a capability grab bag. + +**Implement remote filesystem operations only through shell commands.** Rejected because that discards structured filesystem identity, errors, streaming, version guards, and atomic mutation semantics already consumed by the file tools. + +**Add a generic distributed-runtime abstraction or reconnect live handles.** Rejected because the existing capability seams carry the demonstrated contracts, while remote identity alone cannot reconstruct callbacks, pending promises, authority, protocol state, or output cursors. A new layer would speculate about persistence and synchronization beyond the POC. + +## Consequences + +A remote execution provider implements only its shared sandbox owner plus filesystem and subprocess adapters. Bash, PTY, and LSP compose above them, so fixes to those capabilities remain provider-neutral. + +The fundamental interfaces are wider, and a filesystem/subprocess pair must agree on one execution world. The added operations are limited to facts and lifecycle mechanics that current generic consumers require; model schemas, protocol framing, readiness policy, and presentation do not leak into the providers. + +The local implementation absorbs `node-pty` and platform process inspection because it owns local terminal mechanics. This moves code without weakening terminal teardown: disposal sweeps descendants before and after terminating the top-level shell, waits for exact PID-identity-fenced descendants retained during foreground inspection, and retains Linux session members that survive top-level exit. macOS cannot enumerate a POSIX session after its leader exits, so a child that reparents between inspection snapshots remains an explicit local-provider limitation rather than a reason to move process mechanics back into the PTY consumer. + +The E2B composition demonstrates that a shared sandbox owner plus filesystem and subprocess adapters are sufficient to move the mutable coding world off-host while leaving higher capabilities provider-neutral. Its POC limits remain explicit: the SDK retains complete command transport in host memory, remote startup cannot publish a PID synchronously, exact terminal stdin-wait and independent signal facts are unavailable, numeric PID/PGID operations are not identity-fenced, the initial environment probe cannot hide unknown sandbox-default secrets from already-running same-UID processes, and adapter artifacts remain until sandbox deletion. These are provider constraints, not justification for compatibility shims or more E2B packages. diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md new file mode 100644 index 0000000000..ce6650a284 --- /dev/null +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md @@ -0,0 +1,73 @@ +# Agent Note: 基于文件系统与进程管理执行世界的可移植消费方 + +Status: implemented + +[English](2026-07-28-portable-execution-world-consumers.md) | 中文 + +## 问题 + +文件系统与进程管理 seam 使文件访问和普通进程访问具备可替换性，但 PTY 和 LSP 仍直接调用宿主 Node API。因此，即使领域行为没有变化，远程执行提供方看起来仍需要独立的 PTY 与 LSP 包（package）。这些包只会成为浅层适配器：每个包都仅为替换文件与进程操作而复制一个现有消费方。 + +只有文件操作、命令、终端和语言服务器共享同一个沙箱身份时，远程编码世界才有用。若把完整 harness 移入该沙箱，还会把提供方实验与插件加载、凭据、模型传输、会话持久性、监督和部署纠缠在一起。 + +普通管道无法满足其中一项要求。持久终端需要分配 PTY、检查前台进程组并发送信号，以及清理完整的终端会话。如果假设可以在 `dsh-pty-local` 中基于普通 `spawn()` 句柄重建这些操作，最终不是泄漏提供方内部细节，就是削弱其生命周期契约。 + +## 决策 + +`ctx.fs` 与 `ctx.subprocess` 共同定义一个执行世界。共同挂载的提供方必须描述相同的路径命名空间、可执行文件、进程和终端会话；上层能力消费这两个接口，而不引用具体提供方。 + +文件系统接口负责其他能力需要的路径事实，同时不公开其不透明目标身份：规范化进程路径、规范化 `file:` URI 和包含关系。现有完整文本与流式文本操作仍归文件系统负责；协议消费方在消费流时执行各自的保留上限。 + +进程管理接口负责可执行文件查找与进程原语：以原始或收集模式 spawn 普通进程，以及 `spawnTerminal()`。终端操作是一项深层原语，其句柄负责文本 I/O、前台进程组、信号发送，以及一项须等待的 TERM→KILL 操作；该操作会结算所有在途句柄调用，并使提供方仍可观察到的每个会话成员完全停稳。其信号只取消分配；句柄一经发布，便负责自身生命周期。提示符检测、空闲推断、scrollback、沙箱策略和所有者生命周期仍由 PTY 消费方负责。 + +通用消费方使用该执行世界： + +- `dsh-bash-local` 继续把 Bash 语义映射到普通的 `ctx.subprocess.spawn()`。 +- `dsh-lsp-local` 通过 `ctx.fs` 读取源文件并验证包含关系，通过 `ctx.subprocess` 解析和启动语言服务器，并让由提供方负责的文件 URI 贯穿初始化与结果渲染。一个提供方生命周期信号会在资源释放期间中止文件系统与协议操作，包括取得队列所有权之前的工作区查找；其 JSON-RPC、池化、同步和规范化保持不变。 +- `dsh-pty-local` 把持久 shell 语义映射到 `ctx.subprocess.spawnTerminal()`。本地 `node-pty` 与进程检查实现移入 `dsh-subprocess-local`；其他进程管理提供方则提供相同原语。`danger-full-access` 不需要 `ctx.sandbox`；受限模式要求同一执行世界中存在沙箱提供方，未挂载时会在 spawn 前失败。提供方开始写入时，系统会丢弃异步写入前检查期间收集的提示符与静默证据。取消会在在途写入结算期间保留发送预留，随后向前台进程组发送信号，因此延迟字节和该信号都无法落到后续发送；在途就绪检查无法释放该预留，写入被拒绝时也不会发送信号。绝对截止时间会在整个取消期间保持启用。信号发送失败会成为终结性传输失败。陈旧检查完成后，会针对当前发送恢复轮询。启动取消会立即开始终端回滚，而不等待停滞的就绪检查或信号发送调用。关闭操作会拒绝新的公开信号，并把提供方可观察会话成员的完全停稳委托给句柄上须等待的终止操作。 + +## E2B POC 边界 + +可选启用的 E2B 实现在 `packages/e2b/` 下恰好只有三个提供方专用包：`dsh-e2b` 创建一个沙箱，并在超时或资源释放时将其删除；`dsh-fs-e2b` 实现 `ctx.fs`；`dsh-subprocess-e2b` 基于 E2B Commands、PTY 和远程 Linux 进程组实现 `ctx.subprocess`。两个适配器都从所有者取得唯一的 SDK 句柄，绝不创建私有沙箱。 + +E2B 负责可变文件系统、受管命令与 Bash 进程、终端分配与终端会话组、语言服务器进程与源文件读取，以及 `.dsh-e2b` 下的适配器私有文件。宿主负责 Cordis 与插件对象、agent loop（智能体循环）、agent（智能体）状态、会话状态与目标状态、会话日志与持久化、LLM（大语言模型）调用、提示词与工具、权限、skill（技能）、subagent 编排、PTY 缓冲区与就绪状态、LSP 协议状态，以及 E2B SDK／网络缓冲区。该叠加层既不上传，也不同步宿主工作区。 + +适配器只保留执行基底机制。文件系统规范化以严格的 base64 加 NUL 分帧穿过 SDK 已解码的命令传输；流式读取把字节上限留给消费方执行。进程管理命令输出与环境快照采用 ASCII/base64，避免 SDK 分片解码丢失字节；私有控制 shell 隔离 profile，后续启动会把已发现且名称呈凭据特征的环境变量置空。进程与终端清理使用远程进程组，并在结算前证明完全停稳。 + +沙箱状态有意保持短暂：超时与资源释放会删除远程文件和非托管状态。该 POC 不提供重新连接、pause/leave 保留、会话持久化后端、模板构建器、卷、快照、网络策略层、沙箱目录、工作区同步、持久远程句柄，也不会在其中运行整个 harness。 + +## 验证 + +聚焦的包测试套件锁定了沙箱生命周期、规范化路径分帧、文件系统元数据与原子版本、进程管理发布／回滚、终端文本 I/O 与会话清理、输出上限、取消、资源释放和不变式注册。一项受凭据门控的 Loader 组合通过源代码导入与构建后导出运行同一套三包提供方组合，其中包括 FS/Bash 可见性、恶意登录 profile、跨字节边界拆分的 UTF-8 输出、进程与终端清理、LSP 查询、宿主工作区隔离，以及最终沙箱删除。 + +## 考虑过的替代方案 + +**为每个远程提供方分别保留 PTY 与 LSP 包。** 不予采纳，因为这会在现有 seam 之上重复实现提供方机制。删除检验揭示了这一问题：删除这些适配器不应使领域行为散落到远程提供方中；通用消费方本已负责这些行为。 + +**为每项能力或工具创建独立沙箱。** 不予采纳，因为文件与进程操作将无法共享身份或状态，从而破坏编码用例，并增加生命周期所有者的数量。 + +**把终端建模为普通的管道子进程。** 不予采纳，因为管道无法分配控制终端、确定当前前台进程组或证明完整终端会话已清理。一项终端原语比公开特定于执行基底的逃生口更小，也更能如实表达契约。 + +**把 PTY 就绪判断与会话策略移入进程管理服务。** 不予采纳，因为这些属于持久终端消费方的语义，而非 OS 进程机制。进程管理提供方负责只有其执行基底才能完成的操作；`dsh-pty-local` 负责 Harness 终端的语义。 + +**分别公开终端终止与完全停稳操作，并提供共享生命周期控制器。** 不予采纳，因为每个终端消费方都需要相同的单一清理结果。拆分操作会把提供方簿记、有界观察者和重试语义暴露出来，却没有生产消费方；由提供方提供一个须等待的操作，接口更深。 + +**在文件系统 seam 中新增稳定的有界读取原语。** 不予采纳，因为只有 LSP 需要完整文档字节上限，而它可以在消费现有文本流时执行该上限。第二项原语会迫使每个提供方实现稳定句柄和不跟随符号链接的机制，远程提供方甚至需要辅助协议，却没有已观察到的并发替换缺陷。 + +**在远程环境中运行整个 harness。** 不予采纳，因为这是另一种部署模型。让执行能力可移植，并不意味着移动模型调用、会话状态、插件状态或 agent loop。 + +**把所有提供方操作都放进一个共享所有者包。** 不予采纳，因为沙箱身份与生命周期是所有者唯一的关注点。文件系统与进程管理保留各自独立的契约、测试和消费方，同时避免把所有者变成无边界的能力集合。 + +**只通过 shell 命令实现远程文件系统操作。** 不予采纳，因为这会丢弃现有文件工具已消费的结构化文件系统身份、错误、流式输出、版本保护和原子变更语义。 + +**新增通用分布式运行时抽象，或重新连接活跃句柄。** 不予采纳，因为现有能力 seam 已承载经证实的契约，而仅凭远程身份无法重建回调、待处理 promise、权限、协议状态或输出游标。新增一层只会推测 POC 边界之外的持久化与同步问题。 + +## 后果 + +远程执行提供方只需实现共享沙箱所有者，以及文件系统与进程管理适配器。Bash、PTY 与 LSP 组合在这些适配器之上，因此对这些能力的修复仍与提供方无关。 + +基础接口更宽，一对文件系统／进程管理提供方必须在同一个执行世界上保持一致。新增操作仅限当前通用消费方所需的事实与生命周期机制；模型 schema、协议分帧、就绪策略和呈现不会渗入提供方。 + +本地实现承接 `node-pty` 和平台进程检查，因为它负责本地终端机制。这种代码迁移不会削弱终端拆卸：dispose（资源释放）会在终止顶层 shell 前后清理后代进程，等待前台检查期间保留下来且受精确 PID 身份围栏保护的后代进程，并继续追踪在顶层进程退出后仍存活的 Linux 会话成员。macOS 无法在 POSIX 会话 leader 退出后枚举该会话，因此在两次检查快照之间重新设定父进程的子进程仍是明确的本地提供方限制，而不是把进程机制移回 PTY 消费方的理由。 + +E2B 组合证明，共享沙箱所有者加上文件系统与进程管理适配器，就足以在保持上层能力与提供方无关的同时，把可变编码世界移出宿主。其 POC 限制仍明确在案：SDK 会把完整命令传输内容保留在宿主内存中；远程启动无法同步发布 PID；无法获得精确的终端 stdin 等待状态与独立信号事实；基于数值 PID/PGID 的操作没有身份围栏；初始环境探测无法向已在运行的同 UID 进程隐藏未知的沙箱默认 secret；适配器产物会一直保留到沙箱删除。这些是提供方限制，不是引入兼容性 shim 或更多 E2B 包的理由。 diff --git a/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.i18n.yaml b/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.i18n.yaml index 451a4ea0d3..72944d7433 100644 --- a/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.i18n.yaml +++ b/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md -2026-07-16-persistent-pty-sessions.md: d7d06dc8517780a37889e4f94dd0b99475dc5432 -2026-07-16-persistent-pty-sessions.zh.md: 84ef8988ba657a87137f904df38e45808d32b483 +2026-07-16-persistent-pty-sessions.md: 88296a9318b2c398fead382153bd2c652dbd9e68 +2026-07-16-persistent-pty-sessions.zh.md: 40f26c67fe046ad292f05919501600bd2e146c62 diff --git a/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md b/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md index d7d06dc851..88296a9318 100644 --- a/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md +++ b/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md @@ -23,10 +23,10 @@ The implementation supports interactive shells and line-oriented REPLs on Linux | Package | Role | ctx key | |---|---|---| | `dsh-pty` | `PtyService`, branded `PtySessionId`, backend registry, owner-scoped session contract, and result types | `ctx.pty` | -| `dsh-pty-local` | [`node-pty`](https://github.com/microsoft/node-pty)-based local backend, platform process inspection, bounded terminal buffer, sandbox resolution, and process-tree supervision | registers a backend on `ctx.pty` | +| `dsh-pty-local` | Persistent-shell backend over `ctx.subprocess.spawnTerminal()`: readiness, bounded terminal buffers, sandbox resolution, and owner-aware session lifecycle | registers a backend on `ctx.pty` | | `dsh-tool-pty` | Six model-facing tools, task-runtime integration for background sends, guidance, and UI render intents | registers on `ctx.tools` | -Idle detection is backend behavior, not a second public seam. A remote or container backend may have authoritative readiness signals that do not resemble local `/proc` inspection; every `PtyBackend` therefore returns the common send result while owning its detection mechanism internally. +Readiness remains PTY-backend behavior, not a second public seam. The terminal-process provider supplies only substrate facts such as the foreground process group and whether it can prove that group is waiting on input; `dsh-pty-local` combines those facts with prompt and silence evidence into the common send result. ### Agent ownership and identity @@ -40,12 +40,12 @@ Agent-scope disposal closes registrations first, then awaits quiescent teardown A registered `shell` backend constrains how a terminal starts; it does not constrain commands typed after startup. `dsh-pty-local` therefore applies two protections before spawning: -- It builds a scrubbed child environment using the same credential-shaped-name policy as `bash-local`, removing ambient `*KEY*`, `*PASSWORD*`, `*SECRET*`, `*TOKEN*`, and harness-managed variables unless an explicit trusted mapping supplies them. -- It requires `ctx.sandbox` and the shared `ctx.sandboxPolicy`. At spawn, the backend resolves the owner's effective session mode over the deployment default and wraps the shell argv once; that mode and workspace root remain the process boundary for the PTY lifetime. A write that would change the effective `sandbox/mode` is rejected before commit while the owner has any open PTY or unpublished spawn, with an instruction to wait for creation to settle and close those sessions first; same-effective-mode writes remain valid. The pending reservation spans backend setup through publication, so there is no race in which a wider terminal appears after a downgrade. `danger-full-access` is the existing explicit unconfined choice rather than a PTY-specific bypass. +- It supplies only terminal-specific environment overrides; the mounted subprocess provider applies the shared credential-shaped-name scrub before merging them. +- It requires the shared `ctx.sandboxPolicy`. At spawn, the backend resolves the owner's effective session mode over the deployment default; `danger-full-access` starts the shell directly, while confined modes require a same-world `ctx.sandbox` provider and wrap the shell argv once. That mode and workspace root remain the process boundary for the PTY lifetime. A write that would change the effective `sandbox/mode` is rejected before commit while the owner has any open PTY or unpublished spawn, with an instruction to wait for creation to settle and close those sessions first; same-effective-mode writes remain valid. The pending reservation spans backend setup through publication, so there is no race in which a wider terminal appears after a downgrade. `danger-full-access` is the existing explicit unconfined choice rather than a PTY-specific bypass. Sandboxing confines local process effects but does not make arbitrary shell input safe: network calls and other external side effects remain governed by deployment policy. Tool descriptions state that PTY sessions are less auditable than one-shot tools and should be used only when persistence or interactive stdin is necessary. -The implementation uses only public `node-pty` capabilities: child PID, `data` and `exit` notifications, `write`, `resize`, and `kill`. It does not assume access to the native master fd or call `waitpid` from TypeScript. Platform process inspectors derive foreground process groups and parent/child identity from `/proc` on Linux and `ps` on macOS. +The local subprocess terminal primitive uses only public `node-pty` capabilities: child PID, `data` and `exit` notifications, `write`, and `kill`. It does not assume access to the native master fd or call `waitpid` from TypeScript. Platform process inspectors below that primitive derive foreground process groups and parent/child identity from `/proc` on Linux and `ps` on macOS. The [portable execution-world decision](../architecture/2026-07-28-portable-execution-world-consumers.md) owns this process/consumer split. ### Six model-facing tools @@ -60,7 +60,7 @@ The implementation uses only public `node-pty` capabilities: child PID, `data` a The UI render contract is exact and location-free. `terminal_send` uses terminal call/result cards only for foreground sends; its background form is generic `execute`. `terminal_open`, `terminal_read`, `terminal_signal`, `terminal_close`, and `terminal_list` use generic `execute`, `read`, `execute`, `delete`, and `read` cards respectively. No PTY tool emits `locations`. -`terminal_send({ sessionId, text, submit?, run_in_background? })` treats `text` as UTF-8 bytes and resolves `submit` to `true` in the tool implementation. When `submit` is true it writes the platform Enter sequence after the text; when false it writes only the text, allowing control characters and REPL fragments without hidden content heuristics. `enableRunInBackground` defaults to true; false removes `run_in_background` from the schema and rejects the same undeclared argument if a caller forces it through execution. +`terminal_send({ sessionId, text, submit?, run_in_background? })` treats `text` as UTF-8 bytes and resolves `submit` to `true` in the tool implementation. When `submit` is true it writes the platform Enter sequence after the text; when false it writes only the text, allowing control characters and REPL fragments without hidden content heuristics. Cancellation marks queued input before signaling the real foreground group, so input cannot execute if an asynchronous pre-write inspection settles afterward. The canceled send retains its reservation until asynchronous foreground signalling settles, so a successor cannot become that signal's target. `enableRunInBackground` defaults to true; false removes `run_in_background` from the schema and rejects the same undeclared argument if a caller forces it through execution. Foreground sends return a bounded rendered delta and two independent facts: `waitReason` (`stdin_read | inferred_idle | timeout | session_exit`) and `sessionStatus` (`running` or `exited` with exit code or signal). `session_exit` refers to the PTY's top-level shell process, not an arbitrary foreground command whose status the shell consumes. A timeout never implies process exit. `dsh-tool-pty.maxResultBytes` defaults to 262144, rejects values below 64 so creation acknowledgements retain registry-issued ids, and caps each single-text UTF-8 result after normalized tool or pipeline errors, wait, session, pagination, truncation, generic task-status wrappers, policy denials or short-circuits, and post-execute replacements or blocks; the terminal definitions' last-mile `finalizeContent` callback leaves deliberately structured multi-block policy content unchanged. The renderer reserves suffix space and preserves code-point boundaries instead of treating the backend payload cap as the final model bound. @@ -72,7 +72,7 @@ With `run_in_background: true`, `dsh-tool-pty` registers the in-flight send on ` ### Local readiness detection -The local backend first recognizes a private OSC prompt marker emitted by its controlled bash startup, then requires printable prompt text after that marker before declaring prompt readiness and runs three bounded fallback tiers. Carrying that state across data callbacks covers macOS delivery where the OSC marker and `PS1` arrive separately; the marker alone can no longer publish an empty MOTD. The marker is removed before output reaches the model and avoids a fixed silence delay for ordinary shell commands on both platforms. Unpublished startup does not accept zero-output silence as readiness; timeout rejects the spawn. If caller cancellation wins during startup, the backend closes the private session and propagates the exact `AbortSignal.reason`; a foreground PGID that is not observable yet cannot replace cancellation with a lookup error. All timings are validated config fields: `pollIntervalMs`, `exactProbeAfterMs`, `idleSilenceMs`, `handoffGraceMs`, and `timeoutMs`. +The local backend first recognizes a private OSC prompt marker emitted by its controlled bash startup, then requires the printable tail after the latest marker to exactly equal the controlled `PS1` before declaring prompt readiness and runs three bounded fallback tiers. Carrying that tail across data callbacks covers delivery where the marker and prompt arrive separately; requiring the exact tail rejects a delayed earlier prompt once echoed input or output follows it, so it cannot settle the current send. The marker is removed before output reaches the model and avoids a fixed silence delay for ordinary shell commands on both platforms. Unpublished startup does not accept zero-output silence as readiness; timeout rejects the spawn. If caller cancellation wins during startup, the backend closes the private session and propagates the exact `AbortSignal.reason`; a foreground PGID that is not observable yet cannot replace cancellation with a lookup error. All timings are validated config fields: `pollIntervalMs`, `exactProbeAfterMs`, `idleSilenceMs`, `handoffGraceMs`, and `timeoutMs`. On Linux, the inspector reads the shell's terminal foreground PGID from `/proc//stat`, enumerates every process and thread in that process group, and probes their current syscalls. A positive Tier 1 result requires an observed stdin wait: direct `read(0)`, a permitted read of a `select`/`pselect6` or `poll`/`ppoll` argument containing fd 0, or an epoll interest list containing fd 0. A wait already present before terminal input is not post-write readiness: the same PGID must be observed outside that wait before re-entering it, while a changed foreground PGID is new evidence. Unreadable process memory and unrecognized syscalls are misses, never positive guesses. Architecture tables contain only syscall numbers defined by the corresponding Linux UAPI; unsupported architectures skip Tier 1. @@ -92,9 +92,9 @@ Background sends use the existing task completion notice and `task_output` resul ### Process-tree teardown -The top-level `node-pty` child is the ownership anchor. On close, the backend stops callbacks, snapshots its transitive descendants by parent PID in children-first order, sends `SIGTERM`, waits, rescans for children forked during shutdown, sends `SIGKILL` to the remaining descendant tree, and verifies that every non-zombie descendant left the process table while the shell is still alive. A matching Linux zombie has no executable work and therefore counts as quiescent, allowing shell shutdown to reap or reparent it. Only then does the backend stop the shell with its own TERM/grace/KILL sequence. Every captured PID includes process-start identity so reuse cannot redirect escalation. +The subprocess terminal handle owns the top-level terminal process and its session. On close it snapshots transitive descendants by parent PID in children-first order, sends `SIGTERM`, waits, rescans for children forked during shutdown, sends `SIGKILL` to the union, and verifies every non-zombie descendant left the process table before stopping the top-level process. A matching Linux zombie has no executable work and therefore counts as quiescent. Every captured PID includes process-start identity so reuse cannot redirect escalation. -Teardown reports root exit and survivor cleanup independently. It does not claim success merely because the shell exited; disposal resolves only after no captured non-quiescent tree member remains or returns a cleanup failure naming the survivors. A failed close is not cached forever: the registry and local session each clear the fence only when it still names that failed attempt, so a later explicit or lifecycle close retries after the external survivor condition changes without disturbing a newer concurrent attempt. Service disposal still clears its backend, reservation, and owner-detacher registries when a close fails. It never broadens ownership to every member of the root PID's POSIX session. +Teardown reports top-level exit and survivor cleanup independently. The PTY session does not claim success merely because the shell exited: it calls `SubprocessTerminalHandle.terminate()` and awaits whole-session quiescence, propagating a cleanup failure that names survivors. A failed close is not cached forever: the registry and local session clear the fence only when it still names that failed attempt, so a later explicit or lifecycle close retries without disturbing a newer concurrent attempt. Service disposal still clears its backend, reservation, and owner-detacher registries when a close fails. ### Composition and rollout @@ -108,6 +108,7 @@ plugins: mode: workspace-write workspaceRoot: . '@deepseek-ai/dsh-pty': + '@deepseek-ai/dsh-subprocess-local': '@deepseek-ai/dsh-pty-local': config: scrollbackLines: 10000 @@ -141,11 +142,11 @@ The package ships concise tool guidance explaining persistent state, owner isola **Add persistent mode to `bash`.** Rejected. Returning on readiness rather than process exit, retaining a process tree across calls, and exposing interactive stdin create a different ownership and failure contract. -**Require native master-fd access from `node-pty`.** Rejected. Its public API exposes no master fd. The local backend instead derives foreground groups and descendants from supported OS process metadata and treats unreadable metadata as a detector miss. +**Require native master-fd access from `node-pty`.** Rejected. Its public API exposes no master fd. The local subprocess terminal adapter derives foreground groups and descendants from supported OS process metadata and treats unreadable metadata as a detector miss. **Signal every member of the root PID's POSIX session.** Rejected. `node-pty` may expose a helper PID whose session belongs to the launcher, so SID-wide teardown can signal unrelated harness or desktop processes. A PID-identity-fenced descendant tree is narrower and safe by construction. -**Publish `PtyIdleDetector` as a replaceable registry.** Rejected. Only the local backend needs these platform probes, while remote backends may receive readiness over their own protocol. Backend replacement already provides the necessary extension point. +**Publish `PtyIdleDetector` as a replaceable registry.** Rejected. Substrate-specific foreground facts come from the mounted terminal-process primitive, while prompt/silence readiness remains one private policy in `dsh-pty-local`. The filesystem/subprocess execution-world replacement is the necessary extension point. **Add a PTY-specific `sleep` tool.** Rejected. `ctx.tasks` already owns bounded waiting, cancellation, completion notices, and model-facing collection. A second general wake mechanism would cross the agent-loop boundary and duplicate that contract. @@ -155,9 +156,9 @@ The package ships concise tool guidance explaining persistent state, owner isola ## Verification -- Per-file coverage pins owner fencing, concurrent reservations, unpublished-spawn cancellation and awaited teardown, sandbox-mode change rejection, retriable lifecycle cleanup, readiness tiers, rejection of pre-write stdin waits, the configured handoff grace holding the idle fallback past one poll and its rejection below `pollIntervalMs`, sanitizer carry state, complete UTF-8 bounds, task integration, schemas, and exact render intents. -- Linux process fixtures cover non-leader and non-main-thread stdin waits, zombie quiescence, unreadable process state, supported syscall tables, unsupported architectures, and false-positive rejection; macOS inspector logic is injected into the same unit suite. -- Real `node-pty` tests exercise shell state, shared sandbox policy, environment scrubbing, raw-mode foreground `SIGINT` after deliberately delayed child readiness under scenario-owned timing bounds, a TERM-ignoring descendant, and immediate post-disposal quiescence on supported hosts. +- Per-file coverage pins owner fencing, concurrent reservations, cancellation during pre-write inspection, unpublished-spawn cancellation and awaited teardown, sandbox-mode change rejection, retriable lifecycle cleanup, readiness tiers, rejection of pre-write stdin waits and delayed earlier prompts, the configured handoff grace holding the idle fallback past one poll and its rejection below `pollIntervalMs`, sanitizer carry state, complete UTF-8 bounds, task integration, schemas, and exact render intents. +- Subprocess process fixtures cover non-leader and non-main-thread stdin waits, zombie quiescence, unreadable process state, supported syscall tables, unsupported architectures, and false-positive rejection; macOS inspector logic is injected into the same unit suite. +- Real `node-pty` and PTY-consumer tests jointly exercise shell state, shared sandbox policy, environment scrubbing, raw-mode foreground `SIGINT`, a TERM-ignoring descendant, and immediate post-disposal quiescence on supported hosts. - A Loader-driven `cordis.yml` test mounts the real three-package composition. ACP and headless snapshots pin the six schemas, bounded results, and errors through opt-in overlays; TUI snapshots pin terminal and generic card presentation. - Package contracts, the architecture map, core data structures, generated catalogs, and the website API describe the same shipped surface. - The repository CI-equivalent sequence owns type, lint, coverage, snapshot, documentation, build, hygiene, demo, and built-entry verification. @@ -172,10 +173,10 @@ The package ships concise tool guidance explaining persistent state, owner isola **Persistent state can drift from the model's belief.** The model may forget its cwd or active REPL. Session summaries and retained output help recovery, but no prompt can make state persistence deterministic. -**A daemonized descendant can leave the captured tree.** A process that reparents before teardown is no longer discoverable from the `node-pty` root. The implementation accepts that cleanup gap instead of risking SID-wide signals to unrelated processes. +**A daemonized descendant can leave the local provider's captured tree.** A process that reparents before teardown is no longer discoverable from the `node-pty` root. The local terminal primitive accepts that cleanup gap instead of risking SID-wide signals to unrelated processes. **A shell can cause external side effects.** Session sandboxing and environment scrubbing reduce local exposure but do not undo pushes, API calls, or messages. Deployments that cannot tolerate those effects must omit PTY or add network policy. **Process loss destroys terminal state.** In-process sessions do not survive a harness crash or restart, and raw scrollback is not durable. Important work must be committed to files or another durable system. -**`node-pty` is a native dependency.** Installation, supported Node versions, prebuild availability, and platform behavior require built-artifact smokes on every supported OS. +**`node-pty` is a native dependency of `dsh-subprocess-local`.** Installation, supported Node versions, prebuild availability, and platform behavior require built-artifact smokes on every supported OS. diff --git a/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.zh.md b/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.zh.md index 84ef8988ba..40f26c67fe 100644 --- a/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.zh.md +++ b/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.zh.md @@ -14,7 +14,7 @@ harness 可以运行前台与后台命令、编辑文件和委派工作，但无 ## 决策 -可选的 `packages/pty/` 能力家族提供由 agent 拥有、持久化且面向行式交互的 PTY 会话。它遵循仓库的 [capability pattern](../../implemented/architecture/2026-06-13-capability-seams.md)，与现有命令和文件系统工具并存，并且不修改 `agent-loop`。 +可选的 `packages/pty/` 功能家族提供由 agent 拥有、持久化且面向行式交互的 PTY 会话。它遵循仓库的 [capability pattern](../../implemented/architecture/2026-06-13-capability-seams.md)，与现有命令和文件系统工具并存，并且不修改 `agent-loop`。 当前实现在 Linux 和 macOS 上支持交互式 shell 与行式 REPL。全屏终端应用、按键序列、BEL 触发的控制流、进程丢失后的会话恢复以及跨 agent 共享会话都明确推迟。 @@ -23,10 +23,10 @@ harness 可以运行前台与后台命令、编辑文件和委派工作，但无 | 包 | 角色 | ctx key | |---|---|---| | `dsh-pty` | `PtyService`、branded `PtySessionId`、后端注册表、按 owner 隔离的会话契约和结果类型 | `ctx.pty` | -| `dsh-pty-local` | 基于 [`node-pty`](https://github.com/microsoft/node-pty) 的本地后端、平台进程检查、有界终端缓冲、沙箱解析和进程树监管 | 在 `ctx.pty` 上注册后端 | +| `dsh-pty-local` | 基于 `ctx.subprocess.spawnTerminal()` 的持久 shell 后端：就绪状态、有界终端缓冲、沙箱解析和感知 owner 的会话生命周期 | 在 `ctx.pty` 上注册后端 | | `dsh-tool-pty` | 6 个面向模型的工具、后台发送的 task 运行时集成、使用指引和 UI 渲染意图 | 注册到 `ctx.tools` | -idle 检测属于后端行为，不是第二条公共 seam。远程或容器后端可能拥有完全不同于本地 `/proc` 检查的权威就绪信号；因此每个 `PtyBackend` 都返回统一的发送结果，同时在内部拥有自己的检测机制。 +就绪判定仍属于 PTY 后端行为，不是第二条公共 seam。终端进程提供方只提供基底事实，例如前台进程组，以及能否证明该组正在等待输入；`dsh-pty-local` 将这些事实与提示符和静默证据组合成统一的发送结果。 ### agent 所有权与身份 @@ -34,18 +34,18 @@ idle 检测属于后端行为，不是第二条公共 seam。远程或容器后 实现不提供插件加载期 auto-start 会话。`terminal_open` 只在 agent 工具调用期间创建会话，此时所有权和所属的事件溯源会话都已确定。未来的声明式启动功能必须通过尚未发布的 agent setup 组合，而不能创建全局共享终端。 -agent scope dispose（资源释放）时先关闭注册，再等待全部所属 PTY 完全停稳。未发布的后端 setup 同样是受追踪的生命周期操作：owner 或服务 dispose 会中止服务自有的 signal，等待后端结算与回滚完成后才返回。即使后端 reject，或返回的会话在回滚 close 时失败，调用方取消仍会原样保留其 `AbortSignal.reason`；该清理失败不会替换调用方原因，而会继续受追踪，留待后续 owner 或服务 dispose 处理。由 lifecycle dispose 触发的回滚 close 失败会使 spawn 与该 lifecycle dispose 都 reject，而 `PtyBackendCleanupError` 让后端在不替换调用方取消的前提下，为该 lifecycle dispose 保留自身的启动清理失败。若调用方取消已完成结算，而 dispose 尚未发生，该清理失败会继续作为受追踪的 owner activity 保留，直到后续 owner 或服务 dispose 消费并报告它，因此沙箱模式策略不会把清理失败误判为完全停稳。后端或工具插件 reload 不会遗留会话：所有权持续存放在 `PtyService` 中，直到 agent 结束，与 [`ctx.tasks`](../../../../packages/tasks/tasks/README.md) 的服务持有记录模式一致。服务会先同步把会话预留给一次活跃发送，再返回该操作；后台发送同样会在 task id 对外可见前完成预留。第二次发送会以 `SEND_ACTIVE` 失败，因此输出与取消无法跨越操作所有权。 +agent scope dispose 时先关闭注册，再等待全部所属 PTY 静默退出。未发布的后端 setup 同样是受追踪的生命周期操作：owner 或服务 dispose 会中止服务自有的 signal，等待后端结算与回滚完成后才返回。即使后端 reject，或返回的会话在回滚 close 时失败，调用方取消仍会原样保留其 `AbortSignal.reason`；该清理失败不会替换调用方原因，而会继续受追踪，留待后续 owner 或服务 dispose 处理。由 lifecycle dispose 触发的回滚 close 失败会使 spawn 与该 lifecycle dispose 都 reject，而 `PtyBackendCleanupError` 让后端在不替换调用方取消的前提下，为该 lifecycle dispose 保留自身的启动清理失败。若调用方取消先于 dispose 完成结算，该清理失败会继续作为受追踪的 owner activity 保留，直到后续 owner 或服务 dispose 消费并报告它，因此沙箱模式策略不会把清理失败误判为静默。后端或工具插件 reload 不会遗留会话：所有权持续存放在 `PtyService` 中，直到 agent 结束，与 [`ctx.tasks`](../../../../packages/tasks/tasks/README.md) 的服务持有记录模式一致。服务会先同步把会话预留给一次活跃发送，再返回该操作；后台发送同样会在 task id 对外可见前完成预留。第二次发送会以 `SEND_ACTIVE` 失败，因此输出与取消无法跨越操作所有权。 ### 安全与进程边界 注册的 `shell` 后端只约束终端如何启动，不约束启动后输入的命令。因此 `dsh-pty-local` 在 spawn 前应用两层保护： -- 它使用与 `bash-local` 相同的凭证形态名称策略构建清洗后的子进程环境，移除环境中的 `*KEY*`、`*PASSWORD*`、`*SECRET*`、`*TOKEN*` 和 harness 管理的变量，除非显式的可信映射提供这些值。 -- 它要求 `ctx.sandbox` 和共享的 `ctx.sandboxPolicy`。后端在 spawn 时，以部署默认值为底折叠 owner 的有效 session mode，并只包装一次 shell argv；该 mode 与 workspace root 在 PTY 的整个生命周期中充当进程边界。只要 owner 有任何已打开的 PTY 或尚未发布的 spawn，任何会改变生效 `sandbox/mode` 的写入都会在提交前被拒绝，并提示先等待创建操作结算，再关闭这些会话；不会改变生效模式的写入仍然有效。这项进行中的预留从后端 setup 持续到发布完成，因此不存在降级后又出现权限更宽的终端这一竞态。`danger-full-access` 是现有的显式无约束选择，不另设 PTY 私有 bypass。 +- 它只提供终端专用的环境覆盖；挂载的子进程提供方先清除名称形似凭据的环境变量，再合并这些覆盖。 +- 它要求共享的 `ctx.sandboxPolicy`。后端在 spawn 时，以部署默认值为底折叠 owner 的有效 session mode；`danger-full-access` 会直接启动 shell，受限模式则要求同一执行世界中存在 `ctx.sandbox` 提供方，并只包装一次 shell argv。该 mode 与 workspace root 在 PTY 的整个生命周期中充当进程边界。只要 owner 有任何已打开的 PTY 或尚未发布的 spawn，任何会改变生效 `sandbox/mode` 的写入都会在提交前被拒绝，并提示先等待创建操作结算，再关闭这些会话；不会改变生效模式的写入仍然有效。这项进行中的预留从后端 setup 持续到发布完成，因此不存在降级后又出现权限更宽的终端这一竞态。`danger-full-access` 是现有的显式无约束选择，不另设 PTY 私有 bypass。 沙箱限制本地进程副作用，但不会让任意 shell 输入自动安全：网络调用和其他外部副作用仍由部署策略治理。工具描述会说明 PTY 会话比一次性工具更难审计，只应在确实需要持久状态或交互式 stdin 时使用。 -实现只使用 `node-pty` 的公共功能：子进程 PID、`data` 与 `exit` 通知、`write`、`resize` 和 `kill`。它不假设能访问原生 master fd，也不从 TypeScript 调用 `waitpid`。平台进程检查器在 Linux 上通过 `/proc`、在 macOS 上通过 `ps` 推导前台进程组和父子进程身份。 +本地子进程终端原语只使用 `node-pty` 的公共功能：子进程 PID、`data` 与 `exit` 通知、`write` 和 `kill`。它不假设能访问原生 master fd，也不从 TypeScript 调用 `waitpid`。该原语下的平台进程检查器在 Linux 上通过 `/proc`、在 macOS 上通过 `ps` 推导前台进程组和父子进程身份。[可移植执行环境决策](../architecture/2026-07-28-portable-execution-world-consumers.md)负责定义这种进程／消费方拆分。 ### 6 个面向模型的工具 @@ -55,28 +55,28 @@ agent scope dispose（资源释放）时先关闭注册，再等待全部所属 | `terminal_send` | 发送文本、可选提交 Enter，并等待就绪或注册一个后台任务 | 有界 viewport、等待状态和会话状态；后台模式还返回 `taskId` | | `terminal_read` | 从保留的 scrollback 读取一个有界页 | `{ text, totalLines, lineBegin, lineEnd, truncated }` | | `terminal_signal` | 向当前前台进程组发送一种允许的信号 | `{ delivered, targetPgid }` | -| `terminal_close` | 关闭一个会话并等待进程树完全停稳 | `{ killed }` | +| `terminal_close` | 关闭一个会话并等待进程树静默退出 | `{ killed }` | | `terminal_list` | 列出调用方的活会话 | 按 owner 隔离的会话摘要 | UI 渲染契约精确且不携带位置信息。`terminal_send` 只为前台发送使用 terminal 调用卡片和结果卡片；后台形式使用通用 `execute` 卡片。`terminal_open`、`terminal_read`、`terminal_signal`、`terminal_close` 和 `terminal_list` 分别使用通用 `execute`、`read`、`execute`、`delete` 和 `read` 卡片。所有 PTY 工具都不发出 `locations`。 -`terminal_send({ sessionId, text, submit?, run_in_background? })` 将 `text` 视为 UTF-8 字节，并由工具实现在解析阶段把 `submit` 默认成 `true`。`submit` 为 true 时先写入文本，再写入平台 Enter 序列；为 false 时只写文本，使控制字符和 REPL 片段无需隐藏的内容启发式即可发送。`enableRunInBackground` 默认为 true；设为 false 时，schema 中会移除 `run_in_background`，调用方即使强行把这个未声明参数传入执行流程，也会被拒绝。 +`terminal_send({ sessionId, text, submit?, run_in_background? })` 将 `text` 视为 UTF-8 字节，并由工具实现在解析阶段把 `submit` 默认成 `true`。`submit` 为 true 时先写入文本，再写入平台 Enter 序列；为 false 时只写文本，使控制字符和 REPL 片段无需隐藏的内容启发式即可发送。取消会在向真实前台进程组发送信号前将排队输入标记为已取消，因此即使异步的写入前检查随后才结算，该输入也无法执行。被取消的发送会保留其预留，直至异步前台信号发送结算，因此后续发送不会成为该信号的目标。`enableRunInBackground` 默认为 true；设为 false 时，schema 中会移除 `run_in_background`，调用方即使强行把这个未声明参数传入执行流程，也会被拒绝。 前台发送返回有界的渲染增量和两个独立事实：`waitReason`（`stdin_read | inferred_idle | timeout | session_exit`）与 `sessionStatus`（`running`，或携带退出码或信号的 `exited`）。`session_exit` 指 PTY 顶层 shell 进程退出，不指由 shell 消费状态的任意前台命令。timeout 从不意味着进程已经退出。`dsh-tool-pty.maxResultBytes` 默认为 262144；低于 64 的值会被拒绝，以确保创建确认保留 registry 签发的 id；每个单文本 UTF-8 结果在加入规范化的工具或流水线错误、等待、会话、分页、截断、通用 task 状态包装、策略拒绝或短路以及 post-execute 替换或阻断后，仍受该值限制；终端定义自有的末端 `finalizeContent` callback 会原样保留策略刻意返回的结构化多 block 内容。渲染器会为后缀预留空间并保持代码点边界，而不会把后端载荷上限当作面向模型结果的最终上限。 当 `run_in_background: true` 时，`dsh-tool-pty` 在 `ctx.tasks` 上注册进行中的发送，并立即返回 `taskId`。生产方把 `maxResultBytes` 写入 task 快照，使 `task_output`、kill 返回的终态状态和完成通知在加上通用元数据后，仍对完整结果执行同一上限。`task_output(wait: true)` 负责等待、读取增量输出并记录最终结果；`task_kill` 会解析当前前台 PGID 并发送真正的 `SIGINT`，即使应用已禁用终端 `ISIG` 也同样如此，且后续升级仍只通过 PTY 后端拥有的 teardown 路径进行。若 task 对外接口不存在，后台模式必须在写入输入前失败。设计不新增 PTY 专用的 `sleep` 工具或通用唤醒 seam。 -`terminal_read` 从最新保留行起，朝更早的行分页。后端同时对保留的 scrollback 和返回页载荷执行行数与 UTF-8 字节上限，因此单个超长行无法绕过后端上限；工具随后再限制包含分页与截断元数据的完整渲染页。`truncated` 用于区分保留数据丢失与普通 viewport 增量。 +`terminal_read` 从最新保留行向后分页。后端同时对保留的 scrollback 和返回页载荷执行行数与 UTF-8 字节上限，因此单个超长行无法绕过后端上限；工具随后再限制包含分页与截断元数据的完整渲染页。`truncated` 用于区分保留数据丢失与普通 viewport 增量。 `terminal_signal` 接受闭合集 `SIGINT | SIGTERM | SIGKILL | SIGTSTP | SIGHUP`。后端在执行时解析终端前台进程组。当目标组是顶层 shell 时拒绝 `SIGKILL`，并指引调用方使用 `terminal_close`；进程组解析失败时操作直接失败，而不是向猜测的 PID 发送信号。 ### 本地就绪检测 -本地后端先识别受控 bash 启动时发出的私有 OSC prompt marker，并且只有在该 marker 后出现可打印的 prompt 文本时才据此声明 prompt 就绪；除此之外，它还运行 3 个有界 fallback 层级。在 data callback 之间保留这项状态，可以适配 macOS 分开交付 OSC marker 与 `PS1` 的情况；单独的 marker 不会发布空 MOTD。marker 在输出到达模型前被移除，使两个平台上的普通 shell 命令都无需固定等待静默阈值。尚未发布的 startup 不会把零输出静默视为就绪；timeout 会拒绝 spawn。若调用方取消在 startup 期间胜出，后端会关闭私有会话并原样抛出 `AbortSignal.reason`；尚不可观察的前台 PGID 不会再用查找错误覆盖取消原因。所有时间参数都是经校验的配置字段：`pollIntervalMs`、`exactProbeAfterMs`、`idleSilenceMs`、`handoffGraceMs` 和 `timeoutMs`。 +本地后端先识别受控 bash 启动时发出的私有 OSC prompt marker，并且只有在最近一个 marker 后的可打印尾部与受控 `PS1` 完全相等时才声明 prompt 就绪；除此之外，它还运行 3 个有界 fallback 层级。在 data callback 之间保留该尾部，可以适配 marker 与 prompt 被分开交付的情况；如果回显的输入或输出跟在延迟到达的先前 prompt 之后，要求尾部完全相等会拒绝该 prompt，使其无法完成当前 send。marker 在输出到达模型前被移除，使两个平台上的普通 shell 命令都无需固定等待静默阈值。尚未发布的 startup 不会把零输出静默视为就绪；timeout 会拒绝 spawn。若调用方取消在 startup 期间胜出，后端会关闭私有会话并原样抛出 `AbortSignal.reason`；尚不可观察的前台 PGID 不会再用查找错误覆盖取消原因。所有时间参数都是经校验的配置字段：`pollIntervalMs`、`exactProbeAfterMs`、`idleSilenceMs`、`handoffGraceMs` 和 `timeoutMs`。 在 Linux 上，检查器从 `/proc//stat` 读取 shell 的终端前台 PGID，枚举该进程组中的每个进程与线程，并检查它们当前的 syscall。Tier 1 只有观察到 stdin 等待才返回正结果：直接 `read(0)`、获准读取且含 fd 0 的 `select`/`pselect6` 或 `poll`/`ppoll` 参数，或者含 fd 0 的 epoll interest list。终端输入前就已存在的等待并不代表写入后就绪：必须先观察到同一 PGID 脱离该等待，之后再次进入等待才能使该次 send 完成；前台 PGID 发生变化则构成新的证据。无法读取的进程内存和未识别的 syscall 都是 miss，绝不作为正向猜测。架构表只包含对应 Linux UAPI 定义的 syscall number；不支持的架构跳过 Tier 1。 -macOS 没有精确 syscall 层。任何前台进程组输出静默都会返回 `inferred_idle`，包括 Python 和 `gdb`；从 `ps` 推导的终端 PGID 只用于发送信号，不作为「只有 shell 才能 idle」的证明。纯进程检查逻辑可注入，并在 Linux 上进行单元测试，同时由 macOS CI job 驱动真实 PTY 和进程表路径。 +macOS 没有精确 syscall 层。任何前台进程组输出静默都会返回 `inferred_idle`，包括 Python 和 `gdb`；从 `ps` 推导的终端 PGID 只用于发送信号，不作为「只有 shell 才能 idle」的证明。纯进程检查逻辑可注入并在 Linux 上完成 unit 覆盖率，同时由 macOS CI job 驱动真实 PTY 和进程表路径。 Tier 2 在持续 `idleSilenceMs` 没有输出后返回 `inferred_idle`，因此 sleep 或网络阻塞的命令可能看似 ready。如果此前已经见过 prompt marker，Tier 2 会再等待 `handoffGraceMs`，使恰好落在静默边界上的 bash 前台交接仍然以精确的 `stdin_read` 归因结束，而不是退到较弱的推断；该宽限是由部署方拥有的配置字段，并被校验为至少覆盖一个 `pollIntervalMs`——短于轮询周期的宽限装不下一次就绪轮询，因此不可能改变任何结果。它只约束见过 marker 的 send，代价是这一种情况的交互返回延迟，而不是每一次 send。Tier 3 在 `timeoutMs` 后返回 `timeout`，避免前台工具调用无限占住 agent。结果保留这些区别；调用方可以通过 `ctx.tasks` 等待、向前台组发信号，或从另一个会话排查。 @@ -88,13 +88,13 @@ Tier 2 在持续 `idleSilenceMs` 没有输出后返回 `inferred_idle`，因此 现有持久化 `tool/call` 与 `tool/result` 事件是模型发送文本和返回给模型的渲染输出的真源。`terminal_open` 通过已记录的工具结果返回 MOTD；前台 `send`/`read`/`list`/`signal`/`close` 结果走同一路径记录。PTY 包不会把原始字节流重复写入自定义会话事件。 -后台发送复用现有后台任务完成通知和 `task_output` 结果路径，因此进入后续模型请求的任何输出同样持久化。原始终端字节只作为有界的进程内状态存在，既不持久化也不可恢复。未来的 opt-in transcript（文本记录）sink 必须拥有独立的保留、凭证和隐私契约。 +后台发送复用现有后台任务完成通知和 `task_output` 结果路径，因此进入后续模型请求的任何输出同样持久化。原始终端字节只作为有界的进程内状态存在，既不持久化也不可恢复。未来的 opt-in transcript sink 必须拥有独立的保留、凭证和隐私契约。 ### 进程树 teardown -顶层 `node-pty` 子进程是所有权锚点。关闭时，后端先停止 callback，再按父 PID 以子进程优先顺序捕获其传递子进程、发送 `SIGTERM` 并等待，然后重新扫描关停期间 fork 出的子进程，向剩余子孙进程树发送 `SIGKILL`，并在 shell 仍存活时验证每个非僵尸子孙进程都已离开进程表。身份匹配的 Linux 僵尸进程已无可执行工作，因此视为完全停稳；shell 关闭时会回收它或将其重新挂接给负责回收的父进程。完成这些步骤后，后端才用 shell 自身的 TERM、宽限等待、KILL 序列停止 shell。每个捕获的 PID 都包含进程启动身份，避免 PID 复用把升级信号发给无关进程。 +子进程终端句柄拥有顶层终端进程及其会话。关闭时，它按父 PID 以子进程优先顺序捕获传递后代、发送 `SIGTERM` 并等待，然后重新扫描关停期间 fork 出的子进程，向二者并集发送 `SIGKILL`，并在停止顶层进程前验证每个非僵尸后代都已离开进程表。身份匹配的 Linux 僵尸进程已无可执行工作，因此视为完全停稳。每个捕获的 PID 都包含进程启动身份，避免 PID 复用把升级信号发给无关进程。 -teardown 独立报告根进程退出与存活进程清理。它不会只因 shell 退出就声称成功；dispose 只有在已捕获的进程树中不再存在尚未完全停稳的成员后才完成，否则返回清理失败并列出存活者。失败的 close 不会永久缓存：注册表与本地会话各自仅在关闭围栏仍指向该次失败尝试时才将其清除，因此外部存活进程状态改变后，后续的显式 close 或生命周期 close 会重试，且不会干扰较新的并发尝试。即使某个 close 失败，服务 dispose 仍会清空其后端、预留与 owner detacher 注册表。所有权绝不会扩大到根 PID 所属 POSIX 会话的全部成员。 +teardown 独立报告顶层进程退出与存活进程清理。PTY 会话不会只因 shell 退出就声称成功：它会调用 `SubprocessTerminalHandle.terminate()` 并等待整个会话完全停稳，若清理失败则向外传播并列出存活者。失败的 close 不会永久缓存：注册表与本地会话各自仅在关闭围栏仍指向该次失败尝试时才将其清除，因此后续的显式 close 或生命周期 close 会重试，且不会干扰较新的并发尝试。即使某个 close 失败，服务 dispose 仍会清空其后端、预留与 owner detacher 注册表。 ### 组合与推行 @@ -108,6 +108,7 @@ plugins: mode: workspace-write workspaceRoot: . '@deepseek-ai/dsh-pty': + '@deepseek-ai/dsh-subprocess-local': '@deepseek-ai/dsh-pty-local': config: scrollbackLines: 10000 @@ -141,11 +142,11 @@ plugins: **给 `bash` 增加持久模式。**拒绝。按就绪而不是进程退出返回、跨调用保留进程树、暴露交互式 stdin 会形成不同的所有权和失败契约。 -**要求从 `node-pty` 获取原生 master fd。**拒绝。它的公共 API 不暴露 master fd。本地后端改为从受支持的 OS 进程元数据推导前台组与子孙进程，并把不可读元数据视为 detector miss。 +**要求从 `node-pty` 获取原生 master fd。**拒绝。它的公共 API 不暴露 master fd。本地子进程终端适配器改为从受支持的 OS 进程元数据推导前台组与子孙进程，并把不可读元数据视为 detector miss。 **向根 PID 所属 POSIX 会话的全部成员发送信号。**拒绝。`node-pty` 可能暴露属于启动器会话的 helper PID，因此按 SID 清理可能向无关的 harness 或桌面进程发送信号。带 PID 启动身份校验的子孙进程树范围更窄，其安全边界由结构保证。 -**发布可替换注册表 `PtyIdleDetector`。**拒绝。只有本地后端需要这些平台 probe，远程后端可能通过自己的协议接收就绪状态。替换后端已经提供所需扩展点。 +**发布可替换注册表 `PtyIdleDetector`。**拒绝。基底专用的前台事实来自挂载的终端进程原语，提示符／静默就绪判定则仍是 `dsh-pty-local` 内部的一项私有策略。替换文件系统／子进程执行环境就是所需扩展点。 **新增 PTY 专用 `sleep` 工具。**拒绝。`ctx.tasks` 已经拥有有界等待、取消、完成通知和面向模型的收集。第二套通用唤醒机制会跨越 agent loop（智能体循环）边界并重复该契约。 @@ -155,9 +156,9 @@ plugins: ## 验证 -- 逐文件测试覆盖并固定 owner 隔离、并发预留、未发布 spawn 的取消与等待式 teardown、沙箱模式变更拒绝、可重试的生命周期清理、就绪层级、对写入前 stdin 等待的拒绝、配置化交接宽限把 idle fallback 顶过一次轮询以及低于 `pollIntervalMs` 时的拒绝、sanitizer carry state、完整 UTF-8 结果上限、task 集成、schema 和精确 render intent。 -- Linux 进程 fixture（测试前置数据）覆盖非 leader 与非主线程的 stdin 等待、僵尸进程的完全停稳、不可读进程状态、受支持的 syscall 表、不支持的架构和误报拒绝；同一单元测试套件通过注入覆盖 macOS 检查器逻辑。 -- 真实 `node-pty` 测试在受支持宿主上覆盖 shell 状态、共享沙箱策略、环境清洗、在由场景掌控的时间界限内先有意延迟子进程就绪，再对 raw mode 前台进程发送 `SIGINT`、忽略 `SIGTERM` 的子进程，以及 dispose 返回后立即完全停稳。 +- 每文件覆盖率固定 owner 隔离、并发预留、写入前检查期间的取消、未发布 spawn 的取消与等待式 teardown、沙箱模式变更拒绝、可重试的生命周期清理、就绪层级、对写入前 stdin 等待与延迟到达的先前 prompt 的拒绝、配置化交接宽限把 idle fallback 顶过一次轮询以及低于 `pollIntervalMs` 时的拒绝、sanitizer carry state、完整 UTF-8 结果上限、task 集成、schema 和精确 render intent。 +- 子进程 fixture 覆盖非 leader 与非主线程的 stdin 等待、僵尸进程完全停稳、不可读进程状态、受支持的 syscall 表、不支持的架构和误报拒绝；同一单元测试套件通过注入覆盖 macOS 检查器逻辑。 +- 真实 `node-pty` 与 PTY 消费方测试共同在受支持宿主上覆盖 shell 状态、共享沙箱策略、环境清洗、raw mode 前台 `SIGINT`、忽略 `SIGTERM` 的后代进程，以及 dispose 返回后立即完全停稳。 - Loader 驱动的 `cordis.yml` 测试挂载真实三包组合。ACP 与 headless 快照通过 opt-in overlay 固定 6 个 schema、有界结果和错误；TUI 快照固定 terminal 与 generic 卡片展示。 - 包契约、架构图、核心数据结构、生成目录和 website API 描述同一个已发布接口。 - 仓库 CI 等价序列负责类型、lint、覆盖率、快照、文档、构建、hygiene、demo 和 built-entry 验证。 @@ -172,10 +173,10 @@ plugins: **持久状态可能偏离模型认知。**模型可能忘记 cwd 或活跃 REPL。会话摘要和保留输出有助恢复，但任何 prompt 都无法让状态持久化变成确定行为。 -**daemonized 子进程可能离开捕获树。**在 teardown 前 reparent 的进程无法再从 `node-pty` 根进程发现。实现接受这个清理缺口，不冒险按 SID 向无关进程发送信号。 +**daemonized 后代进程可能离开本地提供方捕获的进程树。**在 teardown 前 reparent 的进程无法再从 `node-pty` 根进程发现。本地终端原语接受这个清理缺口，不冒险按 SID 向无关进程发送信号。 **Shell 可以造成外部副作用。**会话沙箱和环境清洗降低本地暴露，但无法撤销 push、API 调用或消息发送。无法容忍这些副作用的部署必须省略 PTY 或增加网络策略。 **进程丢失会销毁终端状态。**进程内会话无法跨 harness crash 或 restart 存活，原始 scrollback 也不持久化。重要工作必须提交到文件或其他持久系统。 -**`node-pty` 是原生依赖。**安装、支持的 Node 版本、prebuild 可用性和平台行为都需要在每个支持 OS 上运行构建产物冒烟测试。 +**`node-pty` 是 `dsh-subprocess-local` 的原生依赖。**安装、支持的 Node 版本、prebuild 可用性和平台行为都需要在每个支持 OS 上运行 built-artifact smoke。 diff --git a/.agents/notes/implemented/process/2026-07-25-semantic-pr-label-taxonomy.md b/.agents/notes/implemented/process/2026-07-25-semantic-pr-label-taxonomy.md deleted file mode 100644 index 3217b405e9..0000000000 --- a/.agents/notes/implemented/process/2026-07-25-semantic-pr-label-taxonomy.md +++ /dev/null @@ -1,71 +0,0 @@ -# Agent Note: Semantic pull request label taxonomy - -Status: implemented - -English | [中文](2026-07-25-semantic-pr-label-taxonomy.zh.md) - -## Problem - -Pull requests need two different signals: what kind of change they make and which repository domains they affect. A flat or broadly named label set conflates those questions, hides work in distinct areas such as `session` and `llm`, and gives reviewers and automation weak inputs. - -The repository also gains new domains over time. Treating today's area labels as a closed set would force future work into inaccurate labels or a generic catch-all. - -## Decision - -Every open or merged pull request carries exactly one kind and every materially affected area. Closed pull requests that were never merged are outside the maintained historical corpus. Other operational labels may coexist, but they do not satisfy either dimension. - -### Kinds - -| Kind | Meaning | -|---|---| -| `feature` | Adds or intentionally changes behavior. | -| `bug-fix` | Corrects incorrect behavior. | -| `doc` | Makes documentation the dominant intent. | -| `testing` | Changes tests or testing infrastructure without changing product behavior. | -| `cleanup` | Preserves behavior while maintaining or simplifying the implementation or repository process. | - -The kind records the change's dominant intent: accompanying tests and documentation do not turn a feature or bug fix into a testing or documentation change. - -Areas record semantic repository domains rather than temporary initiatives, ownership, or every path touched incidentally. Area labels are not a hierarchy: a pull request may carry several when it changes distinct contracts, but an umbrella and a narrower label do not both describe the same work. - -### Current areas - -The 46 current areas are listed below. The group names organize the list for readability; they are not labels or another taxonomy level. - -| Group | Areas | -|---|---| -| Agent and model | `agent`, `agent-loop`, `session`, `llm`, `model-context`, `compaction`, `tools`, `persistence` | -| Orchestration | `subagent`, `workflow`, `planning`, `tasks`, `schedule`, `telemetry`, `storage`, `workspace` | -| Capabilities | `bash`, `subprocess`, `pty`, `filesystem`, `lsp`, `skills`, `web-search`, `code-mode`, `artifact`, `attachment`, `sandbox`, `mcp`, `hooks`, `cordis` | -| Interfaces | `ui`, `gui`, `tui`, `acp`, `json-rpc`, `cli`, `python-sdk`, `vscode`, `website` | -| Repository and release | `dev-infra`, `ci`, `build`, `dependencies`, `platform`, `i18n`, `release` | - -`gui` covers browser and Electron graphical applications, including standalone graphical developer tools; `vscode` remains the editor extension integration. `ui` covers shared cross-interface commands, approval interaction, presentation, and app boot; it coexists with `gui`, `tui`, or a protocol area only when the pull request also changes that shared contract. - -`tasks` owns background work tied to a running process, while `schedule` owns durable time-triggered jobs. `tools` owns generic registry, schema, and execution contracts; a concrete capability receives `tools` only when it changes one of those contracts. `attachment` owns durable media references and multimodal input delivery, while `artifact` owns model-declared deliverable identity and preview lifecycle; neither borrows `tools` or `ui` for its implementation parts. - -Names follow semantic ownership rather than lexical resemblance. `hooks` means the Claude Code and Codex agent bridges, not local Git hooks; `platform` means product portability, not CI runner selection; and `build` means compilation, bundling, and built package artifacts, not documentation generators. - -### Extensibility - -The area set is intentionally extensible. Add an area when a recurring, meaningful repository domain is missing; do not add a label for one pull request, a temporary project, a status, or a person or team. Rename, split, or retire an area when the domain model changes, and update this list and the affected open and merged pull requests together. - -The kind set stays narrow because kinds are mutually exclusive. A new kind requires a distinct change intent that cannot be represented by the current five; it is not a substitute for an area. - -## Alternatives considered - -- **One undifferentiated label set.** Rejected because kind and area answer different questions; mixing them makes the presence of one label say nothing about whether the other dimension was considered. -- **A fixed, closed area set.** Rejected because repository domains evolve. A closed set would preserve spelling at the cost of semantic accuracy. -- **One broad `core` area or package-derived labels.** Rejected because domains such as `session`, `llm`, and `agent` remain independently meaningful across package boundaries, while incidental file paths are not the scope reviewers or automation need. -- **Separate browser and desktop areas.** Rejected because browser delivery and Electron packaging expose one graphical client domain; splitting them classifies the delivery shell rather than the semantic work. -- **Broad implementation areas in place of a domain.** Rejected because a durable scheduled job is not a background task, an attachment is not merely its source interface or filesystem implementation, and an artifact is not merely its declaring tool or preview interface. -- **Umbrella and leaf areas for the same contract.** Rejected because duplicate labels inflate scope without adding information. Multiple areas remain correct when a pull request changes genuinely distinct contracts. -- **Exactly one area per pull request.** Rejected because coherent changes can legitimately span several domains, and dropping secondary areas hides affected contracts. - -## Consequences - -- Reviewers and automation receive one stable intent signal plus a complete semantic scope. -- `gui` queries cover browser and desktop delivery together, while `ui` queries retain only shared cross-interface contracts. -- `schedule`, `attachment`, and `artifact` queries identify those domains directly instead of approximating them through implementation dependencies. -- Selecting labels remains a judgment call: paths and title prefixes can suggest areas, but they cannot replace reading the change. -- Taxonomy changes carry maintenance work. Area additions, renames, splits, and removals update this decision record and backfill open and merged pull requests so historical queries keep their meaning. diff --git a/.agents/notes/implemented/process/2026-07-25-semantic-pr-label-taxonomy.zh.md b/.agents/notes/implemented/process/2026-07-25-semantic-pr-label-taxonomy.zh.md deleted file mode 100644 index 978f11af94..0000000000 --- a/.agents/notes/implemented/process/2026-07-25-semantic-pr-label-taxonomy.zh.md +++ /dev/null @@ -1,71 +0,0 @@ -# Agent Note: 语义化 PR 标签分类体系 - -Status: implemented - -[English](2026-07-25-semantic-pr-label-taxonomy.md) | 中文 - -## 问题 - -PR（Pull Request）需要传达两个不同的信号：它带来哪一类变更，以及会影响仓库中的哪些领域。一套扁平或命名宽泛的标签会混淆这两个问题，掩盖 `session`、`llm` 等不同领域的工作，也让评审人和自动化流程得到的输入缺乏有效信息。 - -仓库还会随时间发展出新的领域。如果把当前的领域标签视为封闭集合，未来的工作就只能归入不准确的标签或通用兜底标签。 - -## 决策 - -每项开放或已合并的 PR 都带有恰好一个类型标签，以及所有受到实质影响的领域标签。未合并即关闭的 PR 不属于持续维护的历史记录集合。其他管理用途的标签可以并存，但都不能满足这两个维度中的任一个。 - -### 类型 - -| 类型 | 含义 | -|---|---| -| `feature` | 新增行为或有意改变行为。 | -| `bug-fix` | 修正错误行为。 | -| `doc` | 以文档变更为主要意图。 | -| `testing` | 修改测试或测试基础设施，但不改变产品行为。 | -| `cleanup` | 在保持行为不变的前提下，维护或简化实现或仓库流程。 | - -类型记录变更的主要意图：配套测试与文档并不会把一项功能或缺陷修复变成测试或文档变更。 - -领域记录仓库中的语义领域，而不是临时项目、归属关系或偶然触及的每条路径。领域标签不构成层级：一项 PR 修改不同契约时可以带有多个领域标签，但不能用一个总括标签和一个较窄标签重复描述同一项工作。 - -### 当前领域 - -当前的 46 个领域如下。分组名称仅用于提高列表的可读性；它们既不是标签，也不是分类体系中的另一个层级。 - -| 分组 | 领域 | -|---|---| -| agent（智能体）与模型 | `agent`, `agent-loop`, `session`, `llm`, `model-context`, `compaction`, `tools`, `persistence` | -| 编排 | `subagent`, `workflow`, `planning`, `tasks`, `schedule`, `telemetry`, `storage`, `workspace` | -| 能力 | `bash`, `subprocess`, `pty`, `filesystem`, `lsp`, `skills`, `web-search`, `code-mode`, `artifact`, `attachment`, `sandbox`, `mcp`, `hooks`, `cordis` | -| 接口 | `ui`, `gui`, `tui`, `acp`, `json-rpc`, `cli`, `python-sdk`, `vscode`, `website` | -| 仓库与发布 | `dev-infra`, `ci`, `build`, `dependencies`, `platform`, `i18n`, `release` | - -`gui` 涵盖浏览器和 Electron 图形应用，包括独立的图形化开发者工具；`vscode` 仍表示编辑器扩展集成。`ui` 涵盖共享的跨接口命令、审批交互、呈现和应用启动；只有当 PR 还修改这项共享契约时，它才与 `gui`、`tui` 或某个协议领域并用。 - -`tasks` 负责与运行中进程绑定的后台工作，`schedule` 则负责持久化的定时作业。`tools` 负责通用的注册表契约、schema 契约和执行契约；具体能力只有在修改其中一项契约时才带有 `tools`。`attachment` 负责持久化的媒体引用和多模态输入传递，`artifact` 则负责模型声明的交付物标识和预览生命周期；二者都不会因实现包含工具或界面部分而借用 `tools` 或 `ui`。 - -标签名称以语义归属为准，而不是词面相似性。`hooks` 指 Claude Code 和 Codex 的 agent 桥接，而不是本地 Git 钩子；`platform` 指产品可移植性，而不是 CI 运行器选择；`build` 指编译、打包和已构建的包产物，而不是文档生成器。 - -### 可扩展性 - -领域集合有意保持可扩展。当分类体系缺少一个会反复涉及且具有实际意义的仓库领域时，就新增领域；不要仅为一项 PR、临时项目、状态、个人或团队新增标签。当领域模型发生变化时，重命名、拆分或退役相应领域，同时更新本列表以及所有受影响的开放和已合并 PR。 - -类型集合保持精简，因为各类型互斥。新增类型的前提是存在一种当前五类无法表达的独立变更意图；类型不能用来替代领域。 - -## 曾考虑的替代方案 - -- **一套不区分维度的标签。** 不予采纳，因为类型与领域回答的是不同问题；两者混在一起时，存在一个维度的标签并不表示另一个维度也经过了考虑。 -- **一套固定、封闭的领域集合。** 不予采纳，因为仓库领域会持续演变。封闭集合会以牺牲语义准确性为代价来维持拼写不变。 -- **一个宽泛的 `core` 领域，或从包结构派生的标签。** 不予采纳，因为 `session`、`llm` 和 `agent` 等领域在跨越包边界时仍各自具有意义，而偶然涉及的文件路径并不是评审人或自动化流程所需的范围信息。 -- **为浏览器和桌面端分别设置领域。** 不予采纳，因为浏览器交付和 Electron 打包共同呈现同一个图形客户端领域；拆开二者将按交付形态而非工作的语义进行分类。 -- **以宽泛的实现领域替代语义领域。** 不予采纳，因为持久化的定时作业不是后台任务，附件不只是其来源接口或文件系统实现，产物也不只是声明它的工具或预览接口。 -- **同一项契约同时使用总括领域与细分领域。** 不予采纳，因为重复标签只会虚增范围，不会增加信息。一项 PR 确实修改不同契约时，多个领域标签仍然合理。 -- **每项 PR 恰好一个领域。** 不予采纳，因为一项内聚的变更可以合理地跨越多个领域；省略次要领域会隐藏受影响的契约。 - -## 后果 - -- 评审人和自动化流程获得一个稳定的意图信号，以及完整的语义范围。 -- `gui` 查询会同时覆盖浏览器与桌面端交付，`ui` 查询则只涵盖共享的跨接口契约。 -- `schedule`、`attachment` 与 `artifact` 查询直接对应各自领域，无需通过实现依赖近似归类。 -- 选择标签仍然需要判断：路径和标题前缀可以提示领域，但不能替代阅读变更内容。 -- 变更分类体系会产生维护工作。新增、重命名、拆分或移除领域时，需要更新本决策记录，并回填开放和已合并的 PR，使历史查询保持原有含义。 diff --git a/.agents/notes/implemented/process/2026-07-27-dependabot-version-updates.i18n.yaml b/.agents/notes/implemented/process/2026-07-27-dependabot-version-updates.i18n.yaml index 316c31771e..dea250de53 100644 --- a/.agents/notes/implemented/process/2026-07-27-dependabot-version-updates.i18n.yaml +++ b/.agents/notes/implemented/process/2026-07-27-dependabot-version-updates.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/process/2026-07-27-dependabot-version-updates.md -2026-07-27-dependabot-version-updates.md: 5d42563788d9f1e72da65c8e9750d6b1ecba06a5 -2026-07-27-dependabot-version-updates.zh.md: 4847059944e7e35de5719a6cbfd3d5b133467ccb +2026-07-27-dependabot-version-updates.md: bba83c9e720cf87f91638131d7f9580422ea7f76 +2026-07-27-dependabot-version-updates.zh.md: 74b399778fdd06cbbe38234f7bddc5c28c4fd600 diff --git a/.agents/notes/implemented/process/2026-07-27-dependabot-version-updates.md b/.agents/notes/implemented/process/2026-07-27-dependabot-version-updates.md index 5d42563788..bba83c9e72 100644 --- a/.agents/notes/implemented/process/2026-07-27-dependabot-version-updates.md +++ b/.agents/notes/implemented/process/2026-07-27-dependabot-version-updates.md @@ -12,7 +12,7 @@ Maintained registry and GitHub Actions dependencies need a regular update path. The default branch carries [`.github/dependabot.yml`](../../../../.github/dependabot.yml) with weekly version-update checks for the root pnpm workspace, including `native/landlock-run`, the `python/sdk` uv project, and GitHub Actions. Every entry sets `cooldown.default-days` to `30`, so a version release becomes eligible only after it is at least 30 days old and is proposed on the next weekly check. The [in-repository Landlock release decision](2026-08-06-in-repository-landlock-release.md) owns the shared-workspace boundary. -The root pnpm version-update scan excludes `vendor/**`, whose source and manifests move only through the [vendoring procedure](../../../../vendor/README.md). GitHub applies `exclude-paths` only to version updates; a security pull request that touches a vendored manifest is replaced through the vendoring procedure instead of being merged as generated. Dependabot pull requests receive the repository's `cleanup` kind and `area/infra` area labels, run the normal pull-request checks, and remain subject to maintainer review; this automation does not merge them. +The root pnpm version-update scan excludes `vendor/**`, whose source and manifests move only through the [vendoring procedure](../../../../vendor/README.md). GitHub applies `exclude-paths` only to version updates; a security pull request that touches a vendored manifest is replaced through the vendoring procedure instead of being merged as generated. Dependabot pull requests receive the repository's `kind/dependency` kind and `area/infra` area labels, run the normal pull-request checks, and remain subject to maintainer review; this automation does not merge them. Repository settings enable dependency vulnerability alerts and Dependabot security updates. GitHub does not apply version-update cooldowns to those security updates, so security fixes remain eligible immediately. A generated pnpm security pull request can still fail the repository's lockfile release-age verification when dependency resolution selects unrelated fresh transitive versions; that pull request waits or is narrowed instead of weakening the policy. The repository's coordinated fresh-release exceptions are not copied into Dependabot's cooldown exclusions: automated version updates use the uniform 30-day wait, while an explicitly reviewed manual update can still follow its owning release procedure. diff --git a/.agents/notes/implemented/process/2026-07-27-dependabot-version-updates.zh.md b/.agents/notes/implemented/process/2026-07-27-dependabot-version-updates.zh.md index 4847059944..74b399778f 100644 --- a/.agents/notes/implemented/process/2026-07-27-dependabot-version-updates.zh.md +++ b/.agents/notes/implemented/process/2026-07-27-dependabot-version-updates.zh.md @@ -12,7 +12,7 @@ Status: implemented 默认分支包含 [`.github/dependabot.yml`](../../../../.github/dependabot.yml)，其中为包含 `native/landlock-run` 的根 pnpm 工作区、`python/sdk` uv 项目和 GitHub Actions 配置了每周一次的版本更新检查。每个更新项都将 `cooldown.default-days` 设为 `30`，因此某个版本只有在发布至少 30 天后才符合更新条件，并会在下一次每周检查时生成更新提案。[仓库内 Landlock 发布决策](2026-08-06-in-repository-landlock-release.md)负责共享工作区边界。 -根 pnpm 工作区的版本更新扫描排除 `vendor/**`，其中的源码和 manifest（元数据清单）只能通过 [vendoring 流程](../../../../vendor/README.md)变更。GitHub 仅将 `exclude-paths` 用于版本更新；如果安全更新 PR（Pull Request）涉及随源码纳入仓库的 manifest，则改由 vendoring 流程处理，以替代自动生成的 PR，而不会将其原样合并。Dependabot PR 会获得仓库的 `cleanup` 类型标签和 `area/infra` 区域标签，运行常规 PR 检查，并且仍须由维护者评审；该自动化不会合并这些 PR。 +根 pnpm 工作区的版本更新扫描排除 `vendor/**`，其中的源码和 manifest（元数据清单）只能通过 [vendoring 流程](../../../../vendor/README.md)变更。GitHub 仅将 `exclude-paths` 用于版本更新；如果安全更新 PR（Pull Request）涉及随源码纳入仓库的 manifest，则改由 vendoring 流程处理，以替代自动生成的 PR，而不会将其原样合并。Dependabot PR 会获得仓库的 `kind/dependency` 类型标签和 `area/infra` 区域标签，运行常规 PR 检查，并且仍须由维护者评审；该自动化不会合并这些 PR。 仓库设置已启用依赖项漏洞警报和 Dependabot 安全更新。GitHub 不会对这些安全更新应用版本更新冷却期，因此安全修复仍可立即进入更新流程。如果依赖解析还选中了其他刚发布的传递依赖，pnpm 安全更新 PR 仍可能无法通过仓库的锁文件发布时长校验；此类 PR 应等待隔离期结束或缩小更新范围，不得因此放宽政策。仓库为协调刚发布版本而设置的例外，不会纳入 Dependabot 的冷却期排除项：自动版本更新统一等待 30 天；经过明确评审的手动更新仍可遵循相应的发布流程。 diff --git a/.agents/notes/implemented/process/2026-07-25-semantic-pr-label-taxonomy.i18n.yaml b/.agents/notes/implemented/process/2026-08-08-unified-github-label-taxonomy.i18n.yaml similarity index 56% rename from .agents/notes/implemented/process/2026-07-25-semantic-pr-label-taxonomy.i18n.yaml rename to .agents/notes/implemented/process/2026-08-08-unified-github-label-taxonomy.i18n.yaml index 1aaa9f992b..5ee57b914f 100644 --- a/.agents/notes/implemented/process/2026-07-25-semantic-pr-label-taxonomy.i18n.yaml +++ b/.agents/notes/implemented/process/2026-08-08-unified-github-label-taxonomy.i18n.yaml @@ -1,6 +1,6 @@ # Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: -# pnpm run verify-translation-pairing --write .agents/notes/implemented/process/2026-07-25-semantic-pr-label-taxonomy.md -2026-07-25-semantic-pr-label-taxonomy.md: 3217b405e968d4d2c1eba1f1a5a08008b18ba514 -2026-07-25-semantic-pr-label-taxonomy.zh.md: 978f11af9402f248679e2087ff7fb513321b69b9 +# pnpm run verify-translation-pairing --write .agents/notes/implemented/process/2026-08-08-unified-github-label-taxonomy.md +2026-08-08-unified-github-label-taxonomy.md: fddda6c99053e55d3e31a2f6e5c55add09772117 +2026-08-08-unified-github-label-taxonomy.zh.md: c8d727c9a56d6c7c25f73eacde42e0bef1c779b7 diff --git a/.agents/notes/implemented/process/2026-08-08-unified-github-label-taxonomy.md b/.agents/notes/implemented/process/2026-08-08-unified-github-label-taxonomy.md new file mode 100644 index 0000000000..fddda6c990 --- /dev/null +++ b/.agents/notes/implemented/process/2026-08-08-unified-github-label-taxonomy.md @@ -0,0 +1,72 @@ +# Agent Note: Unified GitHub label taxonomy + +Status: implemented + +English | [中文](2026-08-08-unified-github-label-taxonomy.zh.md) + +## Problem + +Pull request labels answer two independent questions: what kind of change the work makes and which durable repository domains it materially affects. Mixing those dimensions or retaining synonymous plain and namespaced labels makes queries ambiguous, while a closed area inventory forces new domains into inaccurate categories. + +Issues already have a native Type and a separate source taxonomy. Reusing pull request kind or source labels across both object types duplicates metadata and weakens the meaning of each family. + +## Decision + +Every open or merged pull request carries exactly one canonical `kind/*` label and at least one materially affected `area/*` label. Closed pull requests that were never merged retain migrated historical assignments but do not receive invented missing classification. Operational labels may coexist without satisfying either dimension. + +### Kinds + +The kind set is closed and mutually exclusive: + +| Kind | Meaning | +|---|---| +| `kind/feature` | Adds or intentionally changes behavior. | +| `kind/bug-fix` | Corrects incorrect behavior. | +| `kind/doc` | Makes documentation the dominant intent. | +| `kind/testing` | Changes tests or testing infrastructure without changing product behavior. | +| `kind/cleanup` | Preserves behavior while maintaining or simplifying implementation or repository process. | +| `kind/dependency` | Updates dependencies without another dominant intent. | + +The kind records the dominant intent. Accompanying tests, documentation, cleanup, or dependency movement do not override a feature or bug fix. A new kind changes this classification contract and requires an explicit taxonomy and policy change. + +Repository policy rejects unsupported `kind/*` values and reserves every alias removed by the unification: `kind/bug`, `kind/documentation`, `feature`, `bug-fix`, `doc`, `cleanup`, `testing`, `dependencies`, `ci`, `cli`, `llm`, and `web-search`. Reserving the exact migrated set prevents an obsolete synonym from being recreated as an apparently unrelated operational label. + +### Areas + +Areas name durable semantic domains rather than temporary initiatives, ownership, or every path touched incidentally. A pull request carries multiple areas when it changes distinct contracts, but it does not combine an umbrella and a narrower label for the same contract. GitHub's live `area/*` names and descriptions own the current inventory; this record owns the selection rule and the non-obvious boundaries that cannot fit reliably in short label descriptions. + +- `area/web` covers browser and Electron graphical interfaces, `area/vscode` covers the editor extension, and `area/api` covers cross-interface protocols and language SDKs. +- `area/planning` covers goals, plans, todos, and scheduling, while `area/workflow` covers executable workflows and background task runtimes. +- `area/artifact` deliberately combines artifacts, attachments, and multimodal delivery. Split labels become justified only when those concerns again need independent review or queries. +- `area/tools` applies to generic registry, schema, and execution contracts. A concrete capability uses its own area unless it also changes that generic contract. +- `area/hooks` means the Claude Code and Codex bridges, `area/infra` covers build, release, CI, repository gates, generators, dependencies, and developer tooling, and `area/windows` covers native Windows product support rather than CI runner selection. + +The area set is intentionally extensible. When no existing description honestly covers a durable and reusable domain, an agent may create a concise `area/` label without separate approval. It must not create an area for one pull request, an incidental path, a temporary project, a status, or a person or team, and it reports the new label and rationale to the requester after applying it. Reusing an inaccurate area merely to avoid a justified addition is not acceptable. + +### Issues and migrations + +Issues use native Issue Type instead of `kind/*`; their `area/*` labels remain optional. `source/*` labels record Issue provenance and do not apply to pull requests. Priority, GitHub defaults, and workflow triggers remain independent operational metadata. + +Label migrations preserve meaning before removing aliases: add the canonical replacement, verify the labelable, then remove the obsolete assignment. A label is deleted only after no pull request or Issue still uses it, and unrelated labels are never replaced as a set. + +## Alternatives considered + +**Unprefixed labels.** Plain names reduce visual noise, but they do not identify whether a label classifies intent, domain, source, priority, or automation. Retaining both plain and namespaced synonyms also makes queries and policy enforcement ambiguous. + +**One undifferentiated label set.** A label's presence would not prove that both intent and semantic scope were considered. + +**A fixed area allowlist in repository policy.** Durable repository domains evolve. The `area/*` namespace remains mechanically recognizable while live descriptions carry the extensible inventory. + +**Package- or path-derived areas.** Areas describe semantic impact across package boundaries, while changed paths include incidental tests, documentation, and support files. + +**Separate labels for every delivery shell or media lifecycle.** Browser and Electron delivery share one graphical domain, and artifact, attachment, and multimodal delivery currently share one review/query domain. A split belongs in a later taxonomy change only when it restores useful independent classification. + +**Broad implementation labels in place of semantic domains.** A concrete capability is not merely its tool, interface, filesystem, or process implementation. Generic implementation areas apply only when their own contracts change. + +**Kinds on Issues.** Native Issue Type already owns that classification; duplicating it as a label creates drift. + +**Exactly one area per pull request.** Coherent changes can materially affect several independent contracts, and dropping secondary areas hides affected scope. + +## Consequences + +Reviewers and automation can query intent, semantic scope, provenance, priority, and operational triggers independently. Maintainers must read the change and the live label descriptions instead of inferring classification from title prefixes or paths. The live catalog, this rationale, and policy enforcement must move together when a kind or a non-obvious area boundary changes, and taxonomy migrations carry an explicit historical backfill and verification cost. diff --git a/.agents/notes/implemented/process/2026-08-08-unified-github-label-taxonomy.zh.md b/.agents/notes/implemented/process/2026-08-08-unified-github-label-taxonomy.zh.md new file mode 100644 index 0000000000..c8d727c9a5 --- /dev/null +++ b/.agents/notes/implemented/process/2026-08-08-unified-github-label-taxonomy.zh.md @@ -0,0 +1,72 @@ +# Agent Note: 统一 GitHub 标签分类体系 + +Status: implemented + +[English](2026-08-08-unified-github-label-taxonomy.md) | 中文 + +## 问题 + +PR（Pull Request）标签回答两个相互独立的问题：工作带来哪一类变更，以及会对哪些持久的仓库领域产生实质影响。混用这两个维度，或同时保留同义的无前缀标签与带命名空间的标签，都会使查询含义模糊；封闭的领域清单则会迫使新领域归入不准确的类别。 + +Issue 已有原生 Issue Type 和独立的来源分类体系。在这两类对象上复用 PR 类型或来源标签会产生重复元数据，并削弱每个标签族的含义。 + +## 决策 + +每项开放或已合并的 PR 都带有恰好一个规范的 `kind/*` 标签，以及至少一个表示实质受影响领域的 `area/*` 标签。未合并即关闭的 PR 保留经迁移的历史标签关系，但不会凭空补充缺失分类。管理用途的标签可以并存，但不能满足这两个维度中的任一个。 + +### 变更类型 + +类型集合封闭且互斥： + +| 变更类型 | 含义 | +|---|---| +| `kind/feature` | 新增行为或有意改变行为。 | +| `kind/bug-fix` | 修正错误行为。 | +| `kind/doc` | 以文档变更为主导意图。 | +| `kind/testing` | 在不改变产品行为的前提下修改测试或测试基础设施。 | +| `kind/cleanup` | 在保持行为不变的前提下，维护或简化实现或仓库流程。 | +| `kind/dependency` | 在没有其他主导意图时更新依赖。 | + +类型记录主导意图。配套测试、文档、清理或依赖调整不会盖过功能变更或缺陷修复这一主导意图。新增类型会改变这项分类契约，因此必须明确修改分类体系和政策。 + +仓库政策会拒绝不支持的 `kind/*` 值，并将统一过程中移除的所有别名列为保留名称：`kind/bug`、`kind/documentation`、`feature`、`bug-fix`、`doc`、`cleanup`、`testing`、`dependencies`、`ci`、`cli`、`llm` 和 `web-search`。精确保留这组已迁移的名称，可以防止过时的同义名称被重新创建成看似无关的管理用途标签。 + +### 领域 + +领域表示持久的语义领域，而不是临时专项、归属关系或偶然触及的每条路径。一项 PR 修改不同契约时带有多个领域标签，但不会用一个总括标签和一个较窄标签重复描述同一项契约。GitHub 上现行的 `area/*` 名称和说明定义当前清单；本记录定义选择规则，以及简短标签说明无法可靠容纳的非显然边界。 + +- `area/web` 覆盖浏览器与 Electron 图形界面，`area/vscode` 覆盖编辑器扩展，`area/api` 覆盖跨界面协议与各语言 SDK。 +- `area/planning` 覆盖目标、计划、待办和调度，`area/workflow` 则覆盖可执行工作流与后台任务运行时。 +- `area/artifact` 有意合并产物、附件与多模态交付。只有当这些关注点再次需要独立评审或查询时，才有理由拆分标签。 +- `area/tools` 适用于通用注册表、schema 与执行契约。具体能力使用自身的领域标签，除非它还修改了这项通用契约。 +- `area/hooks` 表示 Claude Code 与 Codex 桥接，`area/infra` 覆盖构建、发布、CI、仓库门禁、生成器、依赖与开发者工具，`area/windows` 覆盖原生 Windows 产品支持，而不是 CI runner 的选型。 + +领域集合有意保持可扩展。当现有说明都无法如实涵盖一个持久且可复用的领域时，agent（智能体）无需另行批准，即可创建一个简洁的 `area/` 标签。agent 不得为单个 PR、偶然涉及的路径、临时项目、状态、个人或团队创建领域，并且必须在应用新标签后向请求者报告该标签及理由。仅为避免新增一个确有必要的领域标签而复用不准确的领域，不可接受。 + +### Issue 与迁移 + +Issue 使用原生 Issue Type，而不是 `kind/*`；其 `area/*` 标签仍然可选。`source/*` 标签记录 Issue 来源，不适用于 PR。优先级、GitHub 默认标签和工作流触发器仍是相互独立的管理元数据。 + +迁移标签时，须先保留语义，再移除别名：先添加规范替代标签，核验可加标签对象，再移除废弃的标签关系。只有在所有 PR 和 Issue 都不再使用某个标签后才能将其删除，且绝不整组替换无关标签。 + +## 考虑过的替代方案 + +**无前缀标签。** 无前缀名称可以减少视觉噪声，但无法表明标签分类的是意图、领域、来源、优先级还是自动化用途。同时保留无前缀和带命名空间的同义标签，也会使查询和政策执行含义模糊。 + +**不区分维度的单一标签集合。** 某个标签存在，并不能证明意图和语义范围都经过了考虑。 + +**仓库政策中的固定领域允许清单。** 持久的仓库领域会演进。`area/*` 命名空间仍可机械识别，而现行说明承载可扩展清单。 + +**按包或路径派生的领域。** 领域描述跨越包边界的语义影响，而变更路径会包含偶然涉及的测试、文档和支持文件。 + +**为每种交付载体或媒体生命周期单设标签。** 浏览器与 Electron 交付共用一个图形界面领域，产物、附件与多模态交付目前也共用一个评审／查询领域。只有当拆分能恢复有用的独立分类时，才应在后续分类体系变更中进行。 + +**用宽泛的实现标签取代语义领域。** 一项具体能力并不只是其工具、接口、文件系统或进程实现。通用实现领域只在其自身契约变化时适用。 + +**在 Issue 上使用类型标签。** 原生 Issue Type 已负责这项分类；再用标签复制会造成漂移。 + +**每个 PR 恰好一个领域。** 内聚的变更可能对多个独立契约产生实质影响，丢弃次要领域会隐藏受影响范围。 + +## 后果 + +评审人和自动化流程可以分别查询意图、语义范围、来源、优先级和工作流触发条件。维护者必须阅读变更内容和现行标签说明，而不能根据标题前缀或路径推断分类。当某种类型或某条非显然的领域边界发生变化时，现行标签清单、本记录中的决策依据和政策执行必须同步更新；分类体系迁移还会产生明确的历史回填和验证成本。 diff --git a/.github/dependabot.yml b/.github/dependabot.yml index 524d7912e2..dd91db8bf1 100644 --- a/.github/dependabot.yml +++ b/.github/dependabot.yml @@ -13,7 +13,7 @@ updates: cooldown: default-days: 30 labels: - - "cleanup" + - "kind/dependency" - "area/infra" - package-ecosystem: "uv" @@ -25,7 +25,7 @@ updates: cooldown: default-days: 30 labels: - - "cleanup" + - "kind/dependency" - "area/infra" - package-ecosystem: "github-actions" @@ -37,5 +37,5 @@ updates: cooldown: default-days: 30 labels: - - "cleanup" + - "kind/dependency" - "area/infra" diff --git a/.github/issue-management/policy.mjs b/.github/issue-management/policy.mjs index 608291c4f8..09e1bbe7fd 100644 --- a/.github/issue-management/policy.mjs +++ b/.github/issue-management/policy.mjs @@ -12,6 +12,29 @@ const AUDIT_MARKER = '' const OWNER_LINE = /^Owner: @([A-Za-z0-9](?:[A-Za-z0-9-]{0,37}[A-Za-z0-9])?)$/ const TYPES = new Set(['Idea', 'Feature', 'Bug', 'Research', 'Task']) const PRIORITIES = ['p0', 'p1', 'p2', 'p3'] +const PR_KINDS = new Set([ + 'kind/feature', + 'kind/bug-fix', + 'kind/doc', + 'kind/testing', + 'kind/cleanup', + 'kind/dependency', +]) +// Aliases removed by the unified taxonomy migration remain reserved so they cannot be recreated. +const LEGACY_LABELS = new Set([ + 'kind/bug', + 'kind/documentation', + 'feature', + 'bug-fix', + 'doc', + 'cleanup', + 'testing', + 'dependencies', + 'ci', + 'cli', + 'llm', + 'web-search', +]) const TERMINAL_STATUSES = new Set(['Done', 'No action']) const ACTIVE_STATUS_ORDER = config.statuses.filter((status) => !TERMINAL_STATUSES.has(status)) @@ -224,8 +247,14 @@ export function retainIssueReferences(references, issues) { export function validateIssue(issue) { const errors = validateBody(issue) const status = issue.status + const invalidLabels = issue.labels.filter( + (label) => label.startsWith('kind/') || LEGACY_LABELS.has(label), + ) if (!/\p{Script=Han}/u.test(issue.title)) errors.push('Issue 标题必须包含中文') + if (invalidLabels.length > 0) { + errors.push(`Issue 不得使用 PR kind 或旧版标签：${invalidLabels.join(', ')}`) + } if ( /^\s*(?:\[(?:Idea|Feature|Bug|Research|Task|P[0-3]|Inbox|Backlog|Ready|In progress|In review|Done|No action|Owner|area\/[^\]]+)[^\]]*\]|(?:Idea|Feature|Bug|Research|Task|P[0-3]|Inbox|Backlog|Ready|In progress|In review|Done|No action|Owner|area\/[^:： ]+)\s*[:：-])/iu.test( issue.title, @@ -261,12 +290,24 @@ export function validateIssue(issue) { export function validatePullRequest(input) { if (!requiresPullRequestPolicy(input)) return [] const errors = [] - const kinds = input.labels.filter((label) => label.startsWith('kind/')) + const kinds = input.labels.filter((label) => PR_KINDS.has(label)) + const unknownKinds = input.labels.filter( + (label) => label.startsWith('kind/') && !PR_KINDS.has(label) && !LEGACY_LABELS.has(label), + ) + const legacyLabels = input.labels.filter((label) => LEGACY_LABELS.has(label)) + const sourceLabels = input.labels.filter((label) => label.startsWith('source/')) const priorities = input.labels.filter((label) => PRIORITIES.includes(label)) const areas = input.labels.filter((label) => label.startsWith('area/')) if (input.references.all.length === 0) errors.push('PR 正文必须引用至少一个同仓库 Issue') - if (kinds.length !== 1) errors.push(`PR 必须恰好有一个 kind/*，当前为 ${kinds.length}`) + if (kinds.length !== 1) { + errors.push(`PR 必须恰好有一个允许的 kind/*，当前为 ${kinds.length}`) + } + if (unknownKinds.length > 0) { + errors.push(`PR 含不支持的 kind/*：${unknownKinds.join(', ')}`) + } + if (legacyLabels.length > 0) errors.push(`PR 含旧版标签：${legacyLabels.join(', ')}`) + if (sourceLabels.length > 0) errors.push(`source/* 仅用于 Issue：${sourceLabels.join(', ')}`) if (priorities.length > 1) errors.push(`PR 最多有一个 p0–p3，当前为 ${priorities.length}`) if (areas.length === 0) errors.push('PR 必须至少有一个 area/*') for (const number of input.references.all) { diff --git a/.github/issue-management/policy.test.mjs b/.github/issue-management/policy.test.mjs index 86750127a7..8a9b0f91e6 100644 --- a/.github/issue-management/policy.test.mjs +++ b/.github/issue-management/policy.test.mjs @@ -27,6 +27,41 @@ const legalIssue = { stateReason: null, } +const canonicalKinds = [ + 'kind/feature', + 'kind/bug-fix', + 'kind/doc', + 'kind/testing', + 'kind/cleanup', + 'kind/dependency', +] + +// Keep an independent oracle rather than importing the implementation's reserved set. +const legacyLabels = [ + 'kind/bug', + 'kind/documentation', + 'feature', + 'bug-fix', + 'doc', + 'cleanup', + 'testing', + 'dependencies', + 'ci', + 'cli', + 'llm', + 'web-search', +] + +const reviewedPull = (labels) => ({ + isDraft: false, + authorType: 'User', + reviewRequestCount: 1, + reviewCount: 0, + labels, + references: { all: [2], resolving: [], related: [2] }, + issues: new Map([[2, { priority: null }]]), +}) + test('counts only text outside details', () => { assert.deepEqual(countVisibleUnits('支持 GitHub Project。
隐藏文字
'), { units: 4, @@ -92,6 +127,22 @@ test('rejects metadata prefixes in an Issue title', () => { assert.ok(errors.includes('Issue 标题不得带 Type、Priority、Status、area 或 Owner 前缀')) }) +test('reserves PR kind and legacy labels for pull requests', () => { + for (const label of [ + ...canonicalKinds, + 'kind/experimental', + ...legacyLabels, + ]) { + assert.ok( + validateIssue({ ...legalIssue, labels: [label] }).some((error) => + error.startsWith('Issue 不得使用 PR kind 或旧版标签：'), + ), + label, + ) + } + assert.deepEqual(validateIssue({ ...legalIssue, labels: ['area/web', 'source/member'] }), []) +}) + test('keeps terminal Status aligned with the native close reason', () => { assert.deepEqual( validateIssue({ ...legalIssue, status: 'Done', state: 'closed', stateReason: 'completed' }), @@ -260,22 +311,39 @@ test('requires repository PR labels in the enforcement scope', () => { references: { all: [2], resolving: [], related: [2] }, issues: new Map([[2, { priority: null }]]), }) - assert.ok(errors.includes('PR 必须恰好有一个 kind/*，当前为 0')) + assert.ok(errors.includes('PR 必须恰好有一个允许的 kind/*，当前为 0')) assert.ok(errors.includes('PR 必须至少有一个 area/*')) }) -test('accepts repository-extensible kind labels', () => { - assert.deepEqual( - validatePullRequest({ - isDraft: false, - authorType: 'User', - reviewRequestCount: 1, - reviewCount: 0, - labels: ['kind/dependency', 'area/infra'], - references: { all: [2], resolving: [], related: [2] }, - issues: new Map([[2, { priority: null }]]), - }), - [], +test('accepts exactly the canonical kinds with extensible areas', () => { + for (const kind of canonicalKinds) { + assert.deepEqual(validatePullRequest(reviewedPull([kind, 'area/future-domain'])), [], kind) + } +}) + +test('rejects multiple, unknown, legacy, and Issue-source PR labels', () => { + assert.ok( + validatePullRequest( + reviewedPull(['kind/feature', 'kind/doc', 'area/web']), + ).includes('PR 必须恰好有一个允许的 kind/*，当前为 2'), + ) + assert.ok( + validatePullRequest(reviewedPull(['kind/experimental', 'area/web'])).includes( + 'PR 含不支持的 kind/*：kind/experimental', + ), + ) + for (const label of legacyLabels) { + assert.ok( + validatePullRequest(reviewedPull(['kind/feature', 'area/web', label])).some((error) => + error.startsWith('PR 含旧版标签：'), + ), + label, + ) + } + assert.ok( + validatePullRequest( + reviewedPull(['kind/feature', 'area/web', 'source/internal-pr']), + ).includes('source/* 仅用于 Issue：source/internal-pr'), ) }) diff --git a/AGENTS.md b/AGENTS.md index 64f2e944a8..47a90f0065 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -15,11 +15,12 @@ packages/ @deepseek-ai/dsh- workspaces at packages/// api/ Remote BFF assembly and TypeRT RPC gateway typert/ type graph generator, loader, and runtime registry llm/ LLM seam + DeepSeek adapters (direct-fetch + pi-ai design twin) + e2b/ E2B POC: sandbox + FS/subprocess adapters bash/ bash executor seam + local/pwsh impls + model-facing shell tools subprocess/ subprocess seam + local process-tree impl pty/ persistent PTY seam/backend/tools - fs/ filesystem seam + local impl + policy gate + read/write/edit tools - lsp/ language-server seam + local stdio provider + model-facing lsp tool + fs/ filesystem seam/backends/policy/tools + lsp/ language-server seam/local backend/tool skill/ skill provider registry + local impl + catalog/loader tool web/ web seam + search/fetch providers + model-facing web tools compact/ compaction seam + basic backend @@ -118,7 +119,7 @@ Real-API tests and demos read `DEEPSEEK_API_KEY`, optional `DEEPSEEK_BASE_URL`, - **A tool's UI render intent is part of its design**, decided up front (`generic`/`terminal`/`diff`, `locations`); presentation methods are pure functions of `args` ([cookbook](docs/cookbook/adding-a-tool.md)). - **Plan unit, e2e, and snapshot coverage** for new seams, lifecycle shapes, and transcript surfaces; missing snapshot-harness support is part of the implementation, not deferred follow-up. - **Choose PR history deliberately.** Split independent changes; fix the introducing PR before propagation. Standalone PRs and official stacks may merge-forward or rebase after review. Rewrites use `--force-with-lease`, abort on remote movement, never raw `--force`; an in-progress merge-forward preserves its checkpoint before taking a newer base ([rationale](.agents/notes/implemented/process/2026-08-02-native-github-stacks-and-optional-rebases.md)). -- **Label PRs:** one kind (`feature`/`bug-fix`/`doc`/`testing`/`cleanup`), each matching area; the [taxonomy](.agents/notes/implemented/process/2026-07-25-semantic-pr-label-taxonomy.md) is extensible. +- **Labels:** one PR `kind/*`, all material `area/*`, and native Issue Type ([taxonomy](.agents/notes/implemented/process/2026-08-08-unified-github-label-taxonomy.md)). - TODO markers: `FIXME`/`TODO`/`XXX` by urgency ([semantics](docs/development.md)). - Files end with exactly one trailing newline; `git diff --cached --check` (pre-commit) gates it. diff --git a/THIRD_PARTY_NOTICES.md b/THIRD_PARTY_NOTICES.md index 1d169819b4..48f04c9799 100644 --- a/THIRD_PARTY_NOTICES.md +++ b/THIRD_PARTY_NOTICES.md @@ -57,6 +57,7 @@ External packages that a workspace package resolves at runtime. `scripts/install | [`clsx`](https://github.com/lukeed/clsx) | MIT | | [`commander`](https://github.com/tj/commander.js) | MIT | | [`diff`](https://github.com/kpdecker/jsdiff) | BSD-3-Clause | +| [`e2b`](https://github.com/e2b-dev/e2b) | MIT | | [`eventsource-parser`](https://github.com/rexxars/eventsource-parser) | MIT | | [`handlebars`](https://github.com/handlebars-lang/handlebars.js) | MIT | | [`immer`](https://github.com/immerjs/immer) | MIT | diff --git a/docs/architecture.i18n.yaml b/docs/architecture.i18n.yaml index de15631197..e0ddc02423 100644 --- a/docs/architecture.i18n.yaml +++ b/docs/architecture.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write docs/architecture.md -architecture.md: ea78faa62773a6b8ac98e5baab6e181ad6a3b7f0 -architecture.zh.md: 5a46dfaf84276de5f4dc0352a529bdd4e5d8c267 +architecture.md: e88bdac118e382d91c41c471d03182952d8a4508 +architecture.zh.md: 9314a3fbfc87d467a0f902fa1dae9e8f07e2eb28 diff --git a/docs/architecture.md b/docs/architecture.md index ea78faa627..e88bdac118 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -26,12 +26,12 @@ Harnesses are [Cordis](cordis-primer.md) contexts; packages contribute services, | `ctx.llm` | [`llm/`](../packages/llm/README.md) | adapter registry, streaming model calls | | `ctx.tokenMeter` | [`llm/token-meter`](../packages/llm/token-meter/README.md) | replay-aware request and surface pressure | | `ctx.bash` | [`bash/`](../packages/bash/README.md) | foreground/background command execution | -| `ctx.subprocess` | [`subprocess/`](../packages/subprocess/README.md) | managed child-process trees for bash, LSP, and ACP subagent backends | +| `ctx.subprocess` | [`subprocess/`](../packages/subprocess/README.md) | executable lookup, managed trees, terminals | | `ctx.pty` | [`pty/`](../packages/pty/README.md) | owner-scoped persistent terminal sessions | | `ctx.sandbox` | [`sandbox/`](../packages/sandbox/README.md) | same-world process confinement through argv wrapping and per-call policy | | `ctx.sandboxPolicy` | [`sandbox/`](../packages/sandbox/README.md) | shared sandbox policy home | | `ctx.codeRuntime` | [`code-runtime/`](../packages/code-runtime/README.md) | model-written program execution | -| `ctx.fs` | [`fs/`](../packages/fs/README.md) | filesystem provider primitives and policy events | +| `ctx.fs` | [`fs/`](../packages/fs/README.md) | execution-world paths, bounded IO, and policy events | | `ctx.lsp` | [`lsp/`](../packages/lsp/README.md) | semantic navigation registry | | `ctx.skills` | [`skill/`](../packages/skill/README.md) | skill provider registry, progressive disclosure | | `ctx.web` | [`web/`](../packages/web/README.md) | search/fetch provider registries | @@ -155,7 +155,7 @@ Streaming uses raw chunks and `BlockAssembler`. Each `LlmAdapter.stream()` is on ### Capability Pattern -A swappable capability usually has **interface / implementation / consumer** layers: service/events, backend, and model-facing tools/prompts. Bash is the reference; the [capability graph](capability-seams.md) maps each family. +Capabilities separate **interface / implementation / consumer** layers. Filesystem and subprocess providers define one execution world; Bash, PTY, and LSP run there without provider forks. See the [capability graph](capability-seams.md). Exceptions combine LLM interface/consumer, filesystem policy, web registries, and named skill/subagent providers. Subagents spawn fresh, fork a completed-turn prefix, use ACP children, or delegate one self-contained turn to a real product provider such as Codex ([subagent.md](core-data-structures/subagent.md)). diff --git a/docs/architecture.zh.md b/docs/architecture.zh.md index 5a46dfaf84..9314a3fbfc 100644 --- a/docs/architecture.zh.md +++ b/docs/architecture.zh.md @@ -26,12 +26,12 @@ | `ctx.llm` | [`llm/`](../packages/llm/README.md) | 适配器注册表和模型流式调用 | | `ctx.tokenMeter` | [`llm/token-meter`](../packages/llm/token-meter/README.md) | 感知回放的单实例请求压力与表面压力 | | `ctx.bash` | [`bash/`](../packages/bash/README.md) | 前台和后台命令执行 | -| `ctx.subprocess` | [`subprocess/`](../packages/subprocess/README.md) | 供 bash、LSP 与 ACP subagent 后端使用的受管子进程树 | +| `ctx.subprocess` | [`subprocess/`](../packages/subprocess/README.md) | 可执行文件查找、受管进程树、终端 | | `ctx.pty` | [`pty/`](../packages/pty/README.md) | 按 owner 隔离的持久化终端会话 | | `ctx.sandbox` | [`sandbox/`](../packages/sandbox/README.md) | 通过 argv 包装和逐调用策略限制同一执行环境内的进程 | | `ctx.sandboxPolicy` | [`sandbox/`](../packages/sandbox/README.md) | 共享沙箱策略归属点 | | `ctx.codeRuntime` | [`code-runtime/`](../packages/code-runtime/README.md) | 执行模型编写的程序 | -| `ctx.fs` | [`fs/`](../packages/fs/README.md) | 文件系统提供方原语和策略事件 | +| `ctx.fs` | [`fs/`](../packages/fs/README.md) | 执行世界路径、有界 I/O 和策略事件 | | `ctx.lsp` | [`lsp/`](../packages/lsp/README.md) | 语义导航注册表 | | `ctx.skills` | [`skill/`](../packages/skill/README.md) | skill（技能）提供方注册表和渐进式披露 | | `ctx.web` | [`web/`](../packages/web/README.md) | 搜索与抓取提供方注册表 | @@ -155,7 +155,7 @@ idle inject: ### 功能模式 -可替换功能通常具有**接口／实现／消费方**三层：服务和事件、后端、面向模型的工具和提示词。Bash 是参考实现；[功能图](capability-seams.md)映射了每个包族。 +能力分为**接口／实现／消费方**三层。文件系统与进程管理提供方共同定义一个执行世界；Bash、PTY 和 LSP 都在其中运行，无需提供方专用 fork。参见[功能图](capability-seams.md)。 例外情况包括 LLM（大语言模型）合并接口和消费方、文件系统整合策略、web 使用注册表、skill 和 subagent 使用具名提供方。subagent 可以通过 spawn 创建全新实例、fork 一个已完成轮次的前缀、使用 ACP（Agent Client Protocol）子 agent，或将一个独立完整的轮次委派给 Codex 等真实产品提供方（[subagent.md](core-data-structures/subagent.md)）。 diff --git a/docs/capability-seams.md b/docs/capability-seams.md index 48381f9249..6fbe858330 100644 --- a/docs/capability-seams.md +++ b/docs/capability-seams.md @@ -100,11 +100,16 @@ flowchart LR pkg_agent_spine_demo["agent-spine-demo"] pkg_goal["goal"] svc_goals["ctx.goals
Same-session goal domain"] + pkg_e2b["e2b"] + svc_e2b["ctx.e2b
E2B sandbox lifecycle owner"] + pkg_fs_e2b["fs-e2b"] + pkg_subprocess_e2b["subprocess-e2b"] pkg_subprocess["subprocess"] svc_subprocess["ctx.subprocess
Subprocess seam"] pkg_subprocess_local["subprocess-local"] pkg_bash_local["bash-local"] pkg_bash_sandbox["bash-sandbox"] + pkg_pty_local["pty-local"] pkg_lsp_local["lsp-local"] pkg_subagent_acp["subagent-acp"] pkg_subagent_codex["subagent-codex"] @@ -117,7 +122,6 @@ flowchart LR svc_bashEnv["ctx.bashEnv
Managed bash environment registry"] pkg_pty["pty"] svc_pty["ctx.pty
Persistent PTY session registry"] - pkg_pty_local["pty-local"] pkg_sandbox["sandbox"] svc_sandbox["ctx.sandbox
Process-sandbox seam"] pkg_sandbox_local["sandbox-local"] @@ -193,7 +197,9 @@ flowchart LR pkg_directory_picker --> svc_directoryPicker pkg_directory_picker_browse --> svc_directoryPicker pkg_directory_picker_native --> svc_directoryPicker + pkg_e2b --> svc_e2b pkg_fs --> svc_fs + pkg_fs_e2b --> svc_fs pkg_fs_local --> svc_fs pkg_fs_sandbox --> svc_fs pkg_goal --> svc_goals @@ -244,6 +250,7 @@ flowchart LR pkg_subagent_fork --> svc_subagents pkg_subagent_spawn --> svc_subagents pkg_subprocess --> svc_subprocess + pkg_subprocess_e2b --> svc_subprocess pkg_subprocess_local --> svc_subprocess pkg_system_prompt --> svc_systemPrompt pkg_tasks --> svc_tasks @@ -280,6 +287,8 @@ flowchart LR svc_credentials --> pkg_llm_deepseek svc_credentials --> pkg_llm_pi_ai svc_directoryPicker --> pkg_apiproxy + svc_e2b --> pkg_fs_e2b + svc_e2b --> pkg_subprocess_e2b svc_fs --> pkg_tool_fs svc_httpServer --> pkg_connection svc_httpServer --> pkg_hmr @@ -328,6 +337,7 @@ flowchart LR svc_subprocess --> pkg_bash_local svc_subprocess --> pkg_bash_sandbox svc_subprocess --> pkg_lsp_local + svc_subprocess --> pkg_pty_local svc_subprocess --> pkg_subagent_acp svc_subprocess --> pkg_subagent_claude_code svc_subprocess --> pkg_subagent_codex @@ -393,7 +403,8 @@ flowchart LR | `ctx.agents` | `core` | [`agent`](../packages/core/agent) | - | [`agent-loop`](../packages/core/agent-loop), [`acp`](../packages/acp/acp), [`subagent-inprocess`](../packages/subagent/subagent-inprocess) | - | Owns live Agent handles, the create/resume factory seam, and process-local initiator propagation. | | `ctx.agentLoop` | `bundle` | [`agent-loop`](../packages/core/agent-loop) | - | [`agent-spine-demo`](../packages/examples/agent-spine-demo) | - | The one concrete loop plugin; extension packages depend on dsh-agent events and services, not on this package. | | `ctx.goals` | `core` | [`goal`](../packages/goal/goal) | - | - | - | Folds revisioned objective state from the session log and keeps live continuation activation process-local. | -| `ctx.subprocess` | `seam` | [`subprocess`](../packages/subprocess/subprocess) | [`subprocess-local`](../packages/subprocess/subprocess-local) | [`bash-local`](../packages/bash/bash-local), [`bash-sandbox`](../packages/bash/bash-sandbox), [`lsp-local`](../packages/lsp/lsp-local), [`subagent-acp`](../packages/subagent/subagent-acp), [`subagent-codex`](../packages/subagent/subagent-codex), [`subagent-claude-code`](../packages/subagent/subagent-claude-code) | - | The bash executors, the LSP host, and the out-of-process ACP, Codex, and Claude Code subagent backends spawn their children through ctx.subprocess; the service owns tree lifetime, stdio dispositions (pipes, inherit, bounded spill-backed collection), and kill escalation. | +| `ctx.e2b` | `core` | [`e2b`](../packages/e2b/e2b) | - | [`fs-e2b`](../packages/e2b/fs-e2b), [`subprocess-e2b`](../packages/e2b/subprocess-e2b) | - | Owns one shared E2B SDK handle, remote working directory, and final sandbox disposition so both fundamental E2B providers inhabit the same Linux runtime. | +| `ctx.subprocess` | `seam` | [`subprocess`](../packages/subprocess/subprocess) | [`subprocess-local`](../packages/subprocess/subprocess-local), [`subprocess-e2b`](../packages/e2b/subprocess-e2b) | [`bash-local`](../packages/bash/bash-local), [`bash-sandbox`](../packages/bash/bash-sandbox), [`pty-local`](../packages/pty/pty-local), [`lsp-local`](../packages/lsp/lsp-local), [`subagent-acp`](../packages/subagent/subagent-acp), [`subagent-codex`](../packages/subagent/subagent-codex), [`subagent-claude-code`](../packages/subagent/subagent-claude-code) | - | The bash executors, the PTY shell backend, the LSP host, and the out-of-process ACP, Codex, and Claude Code subagent backends spawn through ctx.subprocess; the service owns process coordinates, tree/session lifetime, stdio dispositions, terminal mechanics, and kill escalation. | | `ctx.bash` | `seam` | [`bash`](../packages/bash/bash) | [`bash-local`](../packages/bash/bash-local), [`bash-sandbox`](../packages/bash/bash-sandbox), [`pwsh-local`](../packages/bash/pwsh-local) | [`tool-bash`](../packages/bash/tool-bash), [`tool-pwsh`](../packages/bash/tool-pwsh), [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex) | - | The model-facing shell tools and hook bridges consume this seam; sandboxed, remote, or PowerShell executors replace bash-local without touching them. | | `ctx.bashEnv` | `core` | [`bash-env`](../packages/bash/bash-env) | - | [`tool-bash`](../packages/bash/tool-bash), [`tool-pwsh`](../packages/bash/tool-pwsh) | - | Plugins declare effect-scoped DSH_* facts; each shell tool collects one trusted snapshot per execution and its executor rebuilds the namespace. | | `ctx.pty` | `seam` | [`pty`](../packages/pty/pty) | [`pty-local`](../packages/pty/pty-local) | [`tool-pty`](../packages/pty/tool-pty) | - | The registry owns exact-Agent session identity and cleanup; backends own terminal mechanics, while tool-pty exposes the owner-scoped model surface. | @@ -402,7 +413,7 @@ flowchart LR | `ctx.approval` | `seam` | `approval` | [`acp`](../packages/acp/acp) | [`tools`](../packages/core/tools), [`tool-bash`](../packages/bash/tool-bash) | - | One-shot permission decisions dispatched over the `approval/request` waterfall; answerers are listeners (the ACP bridge for its own agents), absence fails closed to `unavailable`. | | `ctx.permission` | `core` | [`permission`](../packages/ui/permission) | - | - | - | User-facing preset table (`workspace-write`/`danger-full-access`) bundling the sandbox-mode and approval-policy knobs; a switch writes one `permission/preset` event through to both knob events. | | `ctx.codeRuntime` | `seam` | [`code-runtime`](../packages/code-runtime/code-runtime) | [`code-runtime-worker`](../packages/code-runtime/code-runtime-worker) | [`tools`](../packages/core/tools) | - | Runs one model-written program against host-provided async bindings; backends differ by substrate and language (the tool registry consumes it for Code Mode). | -| `ctx.fs` | `seam` | [`fs`](../packages/fs/fs) | [`fs-local`](../packages/fs/fs-local), [`fs-sandbox`](../packages/fs/fs-sandbox) | [`tool-fs`](../packages/fs/tool-fs) | [`fs-policy`](../packages/fs/fs-policy) | tool-fs executes read/write/edit through ctx.fs; fs-sandbox fences mutations by the shared sandbox mode; fs-policy contributes observed-state checks through the fs/* event gate. | +| `ctx.fs` | `seam` | [`fs`](../packages/fs/fs) | [`fs-local`](../packages/fs/fs-local), [`fs-sandbox`](../packages/fs/fs-sandbox), [`fs-e2b`](../packages/e2b/fs-e2b) | [`tool-fs`](../packages/fs/tool-fs) | [`fs-policy`](../packages/fs/fs-policy) | tool-fs executes read/write/edit through ctx.fs; fs-sandbox fences mutations by the shared sandbox mode; fs-policy contributes observed-state checks through the fs/* event gate. | | `ctx.compact` | `seam` | [`compact`](../packages/compact/compact) | [`compact-basic`](../packages/compact/compact-basic) | [`compact-basic`](../packages/compact/compact-basic) | - | The basic backend consumes post-step pressure and request-error recovery events; a model-facing compact tool remains deferred. | | `ctx.subagents` | `seam` | [`subagent`](../packages/subagent/subagent) | [`subagent-spawn`](../packages/subagent/subagent-spawn), [`subagent-fork`](../packages/subagent/subagent-fork), [`subagent-acp`](../packages/subagent/subagent-acp), [`subagent-codex`](../packages/subagent/subagent-codex), [`subagent-claude-code`](../packages/subagent/subagent-claude-code), [`subagent-dsh-sdk`](../packages/subagent/subagent-dsh-sdk) | [`tool-subagent`](../packages/subagent/tool-subagent), [`tool-subagent-control`](../packages/subagent/tool-subagent-control), [`tool-ralph`](../packages/workflow/tool-ralph) | - | Providers implement transports; the service also owns optional Activation-based continuation orchestration, tool-subagent selects one-shot or continuable delegation, tool-subagent-control delivers follow-ups, and tool-ralph requires one fresh structured-output route. | | `ctx.tasks` | `seam` | [`tasks`](../packages/tasks/tasks) | [`tasks-local`](../packages/tasks/tasks-local) | [`tool-bash`](../packages/bash/tool-bash), [`tool-pty`](../packages/pty/tool-pty), [`tool-subagent`](../packages/subagent/tool-subagent), [`tool-tasks`](../packages/tasks/tool-tasks) | - | Producers (background bash, PTY sends, and subagent delegations) register running work; tool-tasks is the model-facing control surface that reads, lists, and kills it; tasks-local is the process-local registry. | diff --git a/docs/config-catalog.md b/docs/config-catalog.md index 61665e3897..c0ca554fea 100644 --- a/docs/config-catalog.md +++ b/docs/config-catalog.md @@ -430,6 +430,22 @@ export interface Config { Source: [`packages/credentials/credentials-local/src/index.ts:54`](../packages/credentials/credentials-local/src/index.ts) +## `@deepseek-ai/dsh-e2b` + +```ts config-catalog +/** Configuration for the shared E2B sandbox owner. */ +export interface Config { + /** API key; omission reads `E2B_API_KEY`. It is never forwarded into the sandbox. */ + apiKey?: string + /** Shared remote working directory, created before adapters receive the sandbox. */ + cwd?: string + /** E2B sandbox lifetime in milliseconds; expiry always deletes the sandbox. */ + timeoutMs?: number +} +``` + +Source: [`packages/e2b/e2b/src/index.ts:43`](../packages/e2b/e2b/src/index.ts) + ## `@deepseek-ai/dsh-frontend-static` Requires: `httpServer` @@ -454,7 +470,7 @@ export interface Config { } ``` -Source: [`packages/fs/fs-local/src/index.ts:38`](../packages/fs/fs-local/src/index.ts) +Source: [`packages/fs/fs-local/src/index.ts:39`](../packages/fs/fs-local/src/index.ts) ## `@deepseek-ai/dsh-fs-sandbox` @@ -937,7 +953,7 @@ Source: [`packages/llm/llm-retry/src/index.ts:46`](../packages/llm/llm-retry/src ## `@deepseek-ai/dsh-lsp-local` -Requires: `lsp` · `subprocess` +Requires: `fs` · `lsp` · `subprocess` ```ts config-catalog /** Plugin configuration: provider id → local language-server configuration. */ @@ -973,7 +989,7 @@ export interface LspLocalServerConfig { } ``` -Source: [`packages/lsp/lsp-local/src/index.ts:87`](../packages/lsp/lsp-local/src/index.ts) +Source: [`packages/lsp/lsp-local/src/index.ts:82`](../packages/lsp/lsp-local/src/index.ts) ## `@deepseek-ai/dsh-mcp-client` @@ -1097,7 +1113,7 @@ Source: [`packages/plan/plan-mode/src/index.ts:69`](../packages/plan/plan-mode/s ## `@deepseek-ai/dsh-pty-local` -Requires: `pty` · `sandbox` · `sandboxPolicy` +Requires: `pty` · `sandboxPolicy` · `subprocess` ```ts config-catalog /** Public plugin configuration. */ @@ -1862,6 +1878,20 @@ export interface Config { Source: [`packages/subagent/subagent-spawn/src/index.ts:25`](../packages/subagent/subagent-spawn/src/index.ts) +## `@deepseek-ai/dsh-subprocess-e2b` + +Requires: `e2b` + +```ts config-catalog +/** Configuration for the E2B subprocess adapter. */ +export interface Config { + /** Remote status/liveness poll cadence in milliseconds; each tick is one control-plane request. */ + pollMs?: number +} +``` + +Source: [`packages/e2b/subprocess-e2b/src/index.ts:25`](../packages/e2b/subprocess-e2b/src/index.ts) + ## `@deepseek-ai/dsh-system-prompt` ```ts config-catalog @@ -2622,6 +2652,7 @@ These load from a `cordis.yml` entry with no `config:` block; they declare no co - `@deepseek-ai/dsh-command-feedback` — requires `commands` ([`packages/feedback/command-feedback/src/index.ts`](../packages/feedback/command-feedback/src/index.ts)) - `@deepseek-ai/dsh-command-goal` — requires `commands` · `goals` ([`packages/goal/command-goal/src/index.ts`](../packages/goal/command-goal/src/index.ts)) - `@deepseek-ai/dsh-commands` ([`packages/ui/commands/src/index.ts`](../packages/ui/commands/src/index.ts)) +- `@deepseek-ai/dsh-fs-e2b` — requires `e2b` ([`packages/e2b/fs-e2b/src/index.ts`](../packages/e2b/fs-e2b/src/index.ts)) - `@deepseek-ai/dsh-fs-policy` ([`packages/fs/fs-policy/src/index.ts`](../packages/fs/fs-policy/src/index.ts)) - `@deepseek-ai/dsh-goal-session` — requires `agents` · `goals` · `sessions` ([`packages/goal/goal-session/src/index.ts`](../packages/goal/goal-session/src/index.ts)) - `@deepseek-ai/dsh-host-directory-picker-auto` — requires `httpServer` · `loader` ([`packages/host/directory-picker-auto/src/index.ts`](../packages/host/directory-picker-auto/src/index.ts)) diff --git a/docs/cordis-catalog/events.md b/docs/cordis-catalog/events.md index edfd046430..7015d39bc8 100644 --- a/docs/cordis-catalog/events.md +++ b/docs/cordis-catalog/events.md @@ -410,7 +410,7 @@ Single-slot decision for the next FileSystem.editText. Calling `next()` yields a Types: [FsTarget](../core-data-structures/filesystem.md) · [FsVersion](../core-data-structures/filesystem.md) -Source: [`packages/fs/fs/src/index.ts:62`](../../packages/fs/fs/src/index.ts) +Source: [`packages/fs/fs/src/index.ts:64`](../../packages/fs/fs/src/index.ts) ### `fs/observed` — emit @@ -430,7 +430,7 @@ Record a successful observation. Listeners must be synchronous recorders: throws Types: [FsTarget](../core-data-structures/filesystem.md) · [FsVersion](../core-data-structures/filesystem.md) -Source: [`packages/fs/fs/src/index.ts:71`](../../packages/fs/fs/src/index.ts) +Source: [`packages/fs/fs/src/index.ts:73`](../../packages/fs/fs/src/index.ts) ### `fs/write-intent` — waterfall @@ -450,7 +450,7 @@ Single-slot decision for the next FileSystem.writeText. Calling `next()` yields Types: [FsTarget](../core-data-structures/filesystem.md) · [FsWriteIntent](../core-data-structures/filesystem.md) -Source: [`packages/fs/fs/src/index.ts:54`](../../packages/fs/fs/src/index.ts) +Source: [`packages/fs/fs/src/index.ts:56`](../../packages/fs/fs/src/index.ts) ## `goal/*` diff --git a/docs/cordis-catalog/services.md b/docs/cordis-catalog/services.md index 80a421cc37..7581d2fbf6 100644 --- a/docs/cordis-catalog/services.md +++ b/docs/cordis-catalog/services.md @@ -649,6 +649,21 @@ abstract capability(): DirectoryPickerCapability Source: [`packages/host/directory-picker/src/index.ts:131`](../../packages/host/directory-picker/src/index.ts) +## `ctx.e2b` — `E2BSandboxService` + +Creates one lazily consumable E2B SDK handle and deletes the sandbox at timeout or disposal. Creation begins at plugin construction; adapters await getSandbox before their first operation. + +```ts cordis-catalog +/** + * Return the shared live SDK handle. + * @returns the created sandbox after the configured cwd exists. + * @throws when E2B rejects creation or the service is disposing. + */ +async getSandbox(): Promise +``` + +Source: [`packages/e2b/e2b/src/index.ts:74`](../../packages/e2b/e2b/src/index.ts) + ## `ctx.fs` — `FileSystem` (abstract seam) Abstract filesystem provider. Targets must preserve identity across aliases; reads expose regular UTF-8 text or typed errors, listings are stable and content-free, and mutations are atomic. Optional guards add stale protection without changing the unguarded provider contract. @@ -665,6 +680,34 @@ Abstract filesystem provider. Targets must preserve identity across aliases; rea */ abstract resolve(path: string, opts?: { cwd?: string; signal?: AbortSignal }): Promise +/** + * Return the canonical absolute path a subprocess in this filesystem's + * execution world can open. The path is deliberately separate from + * {@link FsTarget.targetKey}: consumers may pass this value to another OS + * capability, but must continue treating the target key as opaque. + * @param target - the resolved target whose process path is required. + * @returns an absolute path in the backend's execution world. + */ +abstract processPath(target: FsTarget): string + +/** + * Return the canonical `file:` URI for a target in this filesystem's + * execution world. Backends own URI encoding because the host platform may + * differ from the execution platform. + * @param target - the resolved target to encode. + * @returns the target's canonical file URI. + */ +abstract fileUrl(target: FsTarget): string + +/** + * Test canonical containment without exposing or parsing backend target + * keys. Both targets must come from this provider. + * @param parent - canonical directory target. + * @param child - canonical candidate target. + * @returns true when `child` is `parent` or a descendant of it. + */ +abstract contains(parent: FsTarget, child: FsTarget): boolean + /** * Return target metadata, or `undefined` when the target does not exist. * @param target - the resolved target to stat. @@ -749,7 +792,7 @@ abstract editText( target: FsTarget, edit: FsEditRequest, expected?: { version: Types: [FsDirEntry](../core-data-structures/filesystem.md) · [FsEditOutcome](../core-data-structures/filesystem.md) · [FsEditRequest](../core-data-structures/filesystem.md) · [FsInfo](../core-data-structures/filesystem.md) · [FsPathInfo](../core-data-structures/filesystem.md) · [FsTarget](../core-data-structures/filesystem.md) · [FsVersion](../core-data-structures/filesystem.md) · [FsWriteIntent](../core-data-structures/filesystem.md) · [FsWriteOutcome](../core-data-structures/filesystem.md) · [SandboxExecutionPolicy](../core-data-structures/sandbox.md) -Source: [`packages/fs/fs/src/index.ts:81`](../../packages/fs/fs/src/index.ts) +Source: [`packages/fs/fs/src/index.ts:83`](../../packages/fs/fs/src/index.ts) ## `ctx.goals` — `GoalService` @@ -2286,12 +2329,27 @@ Abstract subprocess service. Subclass, implement spawn, and load the subclass as Implementations must honor these semantics: +- Executable paths belong to one execution world shared with the mounted filesystem provider. - spawn returns immediately with a live handle; `done` resolves at process close with exit facts and rejects only for spawn-level failures. - Collect-mode readers are offset-based and non-consuming, so independent readers never consume one another's output; lossy reads report truncation and the spill file holding the complete stream when one exists. Piped streams are handed to the caller raw and never buffered here. - SubprocessHandle.terminate (and the spec's abort signal) escalates SIGTERM→grace→SIGKILL — the only termination verb — tree-scoped on every platform. SubprocessHandle.waitForExit observes whole-tree liveness, so a consumer-owned teardown ladder can hold each tier on real quiescence. - Disposal of the service terminates all still-running managed processes and awaits their exit. +- spawnTerminal owns terminal allocation, text transport, foreground groups, signalling, and whole-session quiescence behind one awaited termination method; readiness and persistent-shell policy stay in the PTY consumer. Its output stream ends after queued terminal output when the top-level process exits. ```ts cordis-catalog +/** + * Resolve one configured executable in this provider's execution world. + * Absolute paths are verified; bare names use the provider's scrubbed PATH + * plus explicit environment overrides. Relative paths containing separators + * are rejected: no current consumer defines which directory they would + * resolve against, so providers fail loud instead of guessing. + * @param command - absolute executable path or bare PATH name. + * @param env - explicit environment entries used for lookup. + * @param signal - aborts remote or local lookup. + * @returns a canonical executable path. + */ +abstract resolveExecutable( command: string, env?: Readonly>, signal?: AbortSignal, ): Promise + /** * Start one managed child process from a fully-specified spec; this seam * applies no defaults. @@ -2299,11 +2357,20 @@ Implementations must honor these semantics: * @returns the live process handle (streams/readers, signalling, outcome promise). */ abstract spawn(spec: SubprocessSpawnSpec): SubprocessHandle + +/** + * Allocate a real terminal and start one owned process session. This is the + * only non-pipe process primitive: implementations own terminal byte I/O, + * foreground groups, signals, and complete session-tree cleanup. + * @param spec - fully specified argv, cwd, environment, dimensions, grace, and allocation cancellation. + * @returns the live terminal handle after allocation succeeds. + */ +abstract spawnTerminal(spec: SubprocessTerminalSpawnSpec): Promise ``` -Types: [SubprocessHandle](../core-data-structures/subprocess.md) · [SubprocessSpawnSpec](../core-data-structures/subprocess.md) +Types: [SubprocessHandle](../core-data-structures/subprocess.md) · [SubprocessSpawnSpec](../core-data-structures/subprocess.md) · [SubprocessTerminalHandle](../core-data-structures/subprocess.md) · [SubprocessTerminalSpawnSpec](../core-data-structures/subprocess.md) -Source: [`packages/subprocess/subprocess/src/index.ts:91`](../../packages/subprocess/subprocess/src/index.ts) +Source: [`packages/subprocess/subprocess/src/index.ts:102`](../../packages/subprocess/subprocess/src/index.ts) ## `ctx.systemPrompt` — `SystemPrompt` diff --git a/docs/core-data-structures/filesystem.i18n.yaml b/docs/core-data-structures/filesystem.i18n.yaml index 91ac0ed81d..2f9478a7ba 100644 --- a/docs/core-data-structures/filesystem.i18n.yaml +++ b/docs/core-data-structures/filesystem.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write docs/core-data-structures/filesystem.md -filesystem.md: 110c1fd428b15c5094f9dcc94050cad61c324373 -filesystem.zh.md: 010ec22a5d4a29555425c3deede08b317cba7004 +filesystem.md: addded9f673ed435e95109d4fb772967514c0b87 +filesystem.zh.md: 1e378928ed570b97dbec73b836a7e6ff18726ff8 diff --git a/docs/core-data-structures/filesystem.md b/docs/core-data-structures/filesystem.md index 110c1fd428..addded9f67 100644 --- a/docs/core-data-structures/filesystem.md +++ b/docs/core-data-structures/filesystem.md @@ -12,6 +12,8 @@ Provider source: [`packages/fs/fs/src/types.ts`](../../packages/fs/fs/src/types. Every operation resolves a user-supplied path to an opaque backend target first. Consumers may display `displayPath`, but must not parse `targetKey` (a branded opaque id) or assume it is a local absolute path. +Consumers that share the filesystem's execution world obtain cross-capability coordinates through the provider instead of interpreting that identity: `processPath(target)` returns the canonical absolute path a subprocess can open, `fileUrl(target)` returns its provider-platform `file:` URI, and `contains(parent, child)` tests canonical identity or descendant containment. + ```ts type-equiv /** * A path resolved by a backend into a stable identity. `resolve()` produces @@ -50,7 +52,7 @@ type FsTargetKey = Branded<'FsTargetKey'> type FsVersion = Branded<'FsVersion'> ``` -`stat` returns metadata (never content), or `undefined` when the target is absent. `type` lets the tool reject directories/special files before reading, and `size` lets it choose `readText` vs `streamText` without probing by failure. +`stat` returns metadata (never content), or `undefined` when the target is absent. `type` lets the tool reject directories/special files before reading, and `size` lets it choose `readText` vs `streamText` without probing by failure. A protocol consumer that needs a byte ceiling applies it while consuming `streamText`, so the filesystem seam needs no consumer-specific bounded-read primitive. ```ts type-equiv /** @@ -256,4 +258,4 @@ type FsErrorCode = ## The service and the plugin -`FileSystem` (`ctx.fs`, abstract) owns the provider primitives: `resolve`, `stat`, `lstat`, `readText`, `streamText`, `listDir`, `writeText`, and `editText`. `dsh-fs-policy` registers **no service** — it is a plugin that adds policy through the `fs/*` event gate: it decides the write/edit intent waterfalls (supplying `createIfAbsent`/`replaceIfVersion`/`{ version }` or throwing `FS_NOT_OBSERVED`) and records on `fs/observed`. The executor is `dsh-tool-fs`: it reads/writes/edits through `ctx.fs`, dispatches the waterfalls, and emits the recording event. The generated wiring catalog shows the exact `ctx.fs` signatures on [services.md](../cordis-catalog/services.md#ctxfs--filesystem-abstract-seam). +`FileSystem` (`ctx.fs`, abstract) owns the provider primitives: `resolve`, `processPath`, `fileUrl`, `contains`, `stat`, `lstat`, `readText`, `streamText`, `listDir`, `writeText`, and `editText`. `dsh-fs-policy` registers **no service** — it is a plugin that adds policy through the `fs/*` event gate: it decides the write/edit intent waterfalls (supplying `createIfAbsent`/`replaceIfVersion`/`{ version }` or throwing `FS_NOT_OBSERVED`) and records on `fs/observed`. The executor is `dsh-tool-fs`: it reads/writes/edits through `ctx.fs`, dispatches the waterfalls, and emits the recording event. The generated wiring catalog shows the exact `ctx.fs` signatures on [services.md](../cordis-catalog/services.md#ctxfs--filesystem-abstract-seam). diff --git a/docs/core-data-structures/filesystem.zh.md b/docs/core-data-structures/filesystem.zh.md index 010ec22a5d..1e378928ed 100644 --- a/docs/core-data-structures/filesystem.zh.md +++ b/docs/core-data-structures/filesystem.zh.md @@ -12,6 +12,8 @@ 每个操作首先将用户提供的路径解析为不透明的后端目标。消费方可以显示 `displayPath`，但禁止解析 `targetKey`（一个品牌化的不透明 id），也不得假设它是本地绝对路径。 +与文件系统共享执行世界的消费方通过提供方获取跨能力坐标，而不是解释该身份：`processPath(target)` 返回子进程可以打开的规范化绝对路径，`fileUrl(target)` 返回采用提供方平台语法的 `file:` URI，`contains(parent, child)` 则检查规范化身份相等或后代包含关系。 + ```ts type-equiv /** * A path resolved by a backend into a stable identity. `resolve()` produces @@ -50,7 +52,7 @@ type FsTargetKey = Branded<'FsTargetKey'> type FsVersion = Branded<'FsVersion'> ``` -`stat` 返回元数据（从不返回内容），目标不存在时返回 `undefined`。`type` 让工具在读取前拒绝目录或特殊文件；`size` 让工具无需通过失败探测即可选择 `readText` 还是 `streamText`。 +`stat` 返回元数据（从不返回内容），目标不存在时返回 `undefined`。`type` 让工具在读取前拒绝目录或特殊文件；`size` 让工具无需通过失败探测即可选择 `readText` 还是 `streamText`。需要字节上限的协议消费方在消费 `streamText` 时执行该上限，因此文件系统 seam 无需消费方专用的有界读取原语。 ```ts type-equiv /** @@ -256,4 +258,4 @@ type FsErrorCode = ## 服务与插件 -`FileSystem`（`ctx.fs`，abstract）拥有提供方原语：`resolve`、`stat`、`lstat`、`readText`、`streamText`、`listDir`、`writeText` 与 `editText`。`dsh-fs-policy` **不注册服务**——它是一个通过 `fs/*` 事件门禁添加策略的插件：对写入/编辑意图 waterfall 作出决策（提供 `createIfAbsent`/`replaceIfVersion`/`{ version }`，或抛出 `FS_NOT_OBSERVED`），并在 `fs/observed` 上记录。执行器是 `dsh-tool-fs`：它通过 `ctx.fs` 读取/写入/编辑，分发 waterfall，并 emit 记录事件。生成的 wiring 目录在 [services.md](../cordis-catalog/services.md#ctxfs--filesystem-abstract-seam) 中展示确切的 `ctx.fs` 签名。 +`FileSystem`（`ctx.fs`，abstract）拥有提供方原语：`resolve`、`processPath`、`fileUrl`、`contains`、`stat`、`lstat`、`readText`、`streamText`、`listDir`、`writeText` 与 `editText`。`dsh-fs-policy` **不注册服务**——它是一个通过 `fs/*` 事件门禁添加策略的插件：对写入/编辑意图 waterfall 作出决策（提供 `createIfAbsent`/`replaceIfVersion`/`{ version }`，或抛出 `FS_NOT_OBSERVED`），并在 `fs/observed` 上记录。执行器是 `dsh-tool-fs`：它通过 `ctx.fs` 读取/写入/编辑，分发 waterfall，并 emit 记录事件。生成的 wiring 目录在 [services.md](../cordis-catalog/services.md#ctxfs--filesystem-abstract-seam) 中展示确切的 `ctx.fs` 签名。 diff --git a/docs/core-data-structures/lsp.i18n.yaml b/docs/core-data-structures/lsp.i18n.yaml index 5bed4a1a23..25b54df775 100644 --- a/docs/core-data-structures/lsp.i18n.yaml +++ b/docs/core-data-structures/lsp.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write docs/core-data-structures/lsp.md -lsp.md: 62b133cbfdf521e067c56355664d7514a613397f -lsp.zh.md: 51a19a51a8ad92e744cb920a51f3214f68ae0036 +lsp.md: 03d0ce2dbe246aadb6f6c9a702fa50e3e792eb09 +lsp.zh.md: d428f2b5f9568b231b0b50a13bd4a41f7ea81346 diff --git a/docs/core-data-structures/lsp.md b/docs/core-data-structures/lsp.md index 62b133cbfd..03d0ce2dbe 100644 --- a/docs/core-data-structures/lsp.md +++ b/docs/core-data-structures/lsp.md @@ -73,7 +73,7 @@ interface LspProviderQuery extends LspQueryRequest { ## Result -A CLOSED discriminated union: navigation operations normalize to `locations`, `hover` to content or `null`. Consumers `switch` on `kind` to exhaustiveness so a new arm breaks compilation until handled. `findReferences` always includes declarations — the provider enforces this internally, so callers get no flag. The `locations` variant carries `resolvedWorkspaceRoot`: the provider's canonical form of the request's `workspaceRoot` and the root its `file:` URIs are relative to, so a caller relativizing display paths uses it rather than the possibly-symlinked request root. +A CLOSED discriminated union: navigation operations normalize to `locations`, `hover` to content or `null`. Consumers `switch` on `kind` to exhaustiveness so a new arm breaks compilation until handled. `findReferences` always includes declarations — the provider enforces this internally, so callers get no flag. The `locations` variant carries `resolvedWorkspaceUri`, the provider's canonical workspace `file:` URI. A caller relativizing location URIs uses that coordinate rather than applying host-platform path rules to the possibly-symlinked request root. ```ts type-equiv /** One resolved location: a document URI and the range within it. */ @@ -101,13 +101,13 @@ interface LspHover { * `goToImplementation`) normalize to `locations`; `hover` normalizes to content or `null`. * Consumers `switch` on `kind` to exhaustiveness so a new arm breaks compilation until handled. * - * The `locations` variant carries `resolvedWorkspaceRoot`: the provider's canonical form of the - * request's `workspaceRoot`, and the root its `file:` location URIs are relative to. A caller that - * relativizes display paths MUST use this, not the request's (possibly symlinked) `workspaceRoot`; - * otherwise a symlinked workspace misclassifies in-workspace results as external. + * The `locations` variant carries `resolvedWorkspaceUri`: the provider's canonical `file:` URI for + * the request's workspace root. A caller that relativizes location URIs MUST use this, not parse the + * request's possibly symlinked process path with host-platform rules; the execution platform may + * differ from the caller's. */ type LspQueryResult = - | { readonly kind: 'locations'; readonly locations: readonly LspLocation[]; readonly resolvedWorkspaceRoot: string } + | { readonly kind: 'locations'; readonly locations: readonly LspLocation[]; readonly resolvedWorkspaceUri: string } | { readonly kind: 'hover'; readonly hover: LspHover | null } ``` diff --git a/docs/core-data-structures/lsp.zh.md b/docs/core-data-structures/lsp.zh.md index 51a19a51a8..d428f2b5f9 100644 --- a/docs/core-data-structures/lsp.zh.md +++ b/docs/core-data-structures/lsp.zh.md @@ -73,7 +73,7 @@ interface LspProviderQuery extends LspQueryRequest { ## 结果 -这是一个闭合的可辨识联合：导航操作规范化为 `locations`，`hover` 规范化为内容或 `null`。消费方使用 `switch` 对 `kind` 做穷尽处理，因此新增分支会使编译失败，直到完成处理。`findReferences` 始终包含声明；提供方在内部强制保证这一点，因此调用方没有对应 flag。`locations` 变体携带 `resolvedWorkspaceRoot`，即提供方对请求中 `workspaceRoot` 的规范形式，也是其 `file:` URI 所相对的根目录；调用方在相对化显示路径时应使用它，而不是可能经过符号链接的请求根目录。 +这是一个闭合的可辨识联合：导航操作规范化为 `locations`，`hover` 规范化为内容或 `null`。消费方使用 `switch` 对 `kind` 做穷尽处理，因此新增分支会使编译失败，直到完成处理。`findReferences` 始终包含声明；提供方在内部强制保证这一点，因此调用方没有对应 flag。`locations` 变体携带 `resolvedWorkspaceUri`，即提供方的规范工作区 `file:` URI。调用方相对化位置 URI 时应使用这一坐标，而不是对可能经过符号链接的请求根目录应用宿主平台路径规则。 ```ts type-equiv /** One resolved location: a document URI and the range within it. */ @@ -101,13 +101,13 @@ interface LspHover { * `goToImplementation`) normalize to `locations`; `hover` normalizes to content or `null`. * Consumers `switch` on `kind` to exhaustiveness so a new arm breaks compilation until handled. * - * The `locations` variant carries `resolvedWorkspaceRoot`: the provider's canonical form of the - * request's `workspaceRoot`, and the root its `file:` location URIs are relative to. A caller that - * relativizes display paths MUST use this, not the request's (possibly symlinked) `workspaceRoot`; - * otherwise a symlinked workspace misclassifies in-workspace results as external. + * The `locations` variant carries `resolvedWorkspaceUri`: the provider's canonical `file:` URI for + * the request's workspace root. A caller that relativizes location URIs MUST use this, not parse the + * request's possibly symlinked process path with host-platform rules; the execution platform may + * differ from the caller's. */ type LspQueryResult = - | { readonly kind: 'locations'; readonly locations: readonly LspLocation[]; readonly resolvedWorkspaceRoot: string } + | { readonly kind: 'locations'; readonly locations: readonly LspLocation[]; readonly resolvedWorkspaceUri: string } | { readonly kind: 'hover'; readonly hover: LspHover | null } ``` diff --git a/docs/core-data-structures/subprocess.i18n.yaml b/docs/core-data-structures/subprocess.i18n.yaml index 5d59f6fd16..b2d38854b2 100644 --- a/docs/core-data-structures/subprocess.i18n.yaml +++ b/docs/core-data-structures/subprocess.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write docs/core-data-structures/subprocess.md -subprocess.md: 8189795c5acf2fc1882d9e5bc1c006f49c327e2f -subprocess.zh.md: 370a1a05807eefbe09a896ea8bcca1e4ab0a6b83 +subprocess.md: 023b122218ad1caa2b8e16c26b0bc8b0d4183c28 +subprocess.zh.md: 5ee9c782c0ce73ca9a2e694450ed38b8d548dc09 diff --git a/docs/core-data-structures/subprocess.md b/docs/core-data-structures/subprocess.md index 8189795c5a..023b122218 100644 --- a/docs/core-data-structures/subprocess.md +++ b/docs/core-data-structures/subprocess.md @@ -2,9 +2,13 @@ English | [中文](subprocess.zh.md) -The subprocess seam is split across interface ([dsh-subprocess](../../packages/subprocess/subprocess), `ctx.subprocess`) and implementation ([dsh-subprocess-local](../../packages/subprocess/subprocess-local)); its consumers are other capability seams and out-of-process backends — the [bash executor family](bash.md) (collect-mode batch output), the LSP host (piped protocol streams + a collected stderr tail), and the ACP subagent backend (piped protocol streams + inherited stderr). This seam owns the managed `DSH_*` environment namespace, the shared credential scrub (`scrubbedParentEnv`), and the `CollectedOutput` shape; [dsh-bash](../../packages/bash/bash) re-exports the vocabulary so bash consumers keep one import root. +The subprocess seam is split across interface ([dsh-subprocess](../../packages/subprocess/subprocess), `ctx.subprocess`) and implementation ([dsh-subprocess-local](../../packages/subprocess/subprocess-local)); its consumers are other capability seams and out-of-process backends: the [bash executor family](bash.md) uses collected batch output, LSP uses raw protocol pipes, the PTY backend uses the terminal primitive, and the ACP subagent backend uses piped ndjson plus inherited stderr. This seam owns the managed `DSH_*` environment namespace, the shared credential scrub (`scrubbedParentEnv`), and the `CollectedOutput` shape; [dsh-bash](../../packages/bash/bash) re-exports the vocabulary so bash consumers keep one import root. -Source: [`packages/subprocess/subprocess/src/types.ts`](../../packages/subprocess/subprocess/src/types.ts) +Source: [`packages/subprocess/subprocess/src/types.ts`](../../packages/subprocess/subprocess/src/types.ts) and [`packages/subprocess/subprocess/src/index.ts`](../../packages/subprocess/subprocess/src/index.ts) + +## Executable lookup + +One provider's spawn working directories, executable paths, ordinary processes, and terminal sessions inhabit the same path and process namespace as the mounted filesystem provider. `resolveExecutable(command, env?, signal?)` verifies absolute executable paths or resolves bare names through the provider's scrubbed `PATH` plus deliberate overrides. ## Managed environment namespace and captured output @@ -234,6 +238,12 @@ interface SubprocessOutcome { } ``` +## Terminal-process primitive + +`spawnTerminal(spec)` is the non-pipe process primitive. The provider allocates the controlling terminal and owns UTF-8 text transport, foreground-process-group inspection and signalling, and one awaited TERM-to-KILL operation that reaches quiescence for every session member the provider can still observe; providers document substrate-specific observability limits. The PTY backend remains responsible for prompt detection, readiness inference, scrollback, sandbox policy, and persistent-session ownership; ordinary `spawn()` cannot reconstruct controlling-terminal semantics. + +The terminal spec fully specifies argv, cwd, environment overrides, dimensions, cleanup grace, and optional allocation cancellation. Its handle exposes `pid`, ordered output, `done`, `write`, `inspectForeground`, `signalForeground`, and awaited `terminate`; the exact public shapes are generated into the [`ctx.subprocess` service catalog](../cordis-catalog/services.md#ctxsubprocess--subprocessservice-abstract-seam). + ## Service behavior -The abstract [`SubprocessService`](../../packages/subprocess/subprocess/src/index.ts) seam defines `spawn` only; [`LocalSubprocessService`](../../packages/subprocess/subprocess-local/src/index.ts) is the local implementation (detached trees, per-disposition wiring, credential scrub, terminate-and-join disposal). See [`dsh-subprocess`](../../packages/subprocess/subprocess/README.md) for the seam contract and [`dsh-subprocess-local`](../../packages/subprocess/subprocess-local/README.md) for the mechanics. +The abstract [`SubprocessService`](../../packages/subprocess/subprocess/src/index.ts) seam defines execution-world coordinates, executable lookup, ordinary `spawn`, and `spawnTerminal`. [`LocalSubprocessService`](../../packages/subprocess/subprocess-local/src/index.ts) implements them with detached process trees, per-disposition wiring, credential scrubbing, `node-pty`, platform process inspection, and terminate-and-join disposal. See [`dsh-subprocess`](../../packages/subprocess/subprocess/README.md) for the interface contract and [`dsh-subprocess-local`](../../packages/subprocess/subprocess-local/README.md) for local mechanics. diff --git a/docs/core-data-structures/subprocess.zh.md b/docs/core-data-structures/subprocess.zh.md index 370a1a0580..5ee9c782c0 100644 --- a/docs/core-data-structures/subprocess.zh.md +++ b/docs/core-data-structures/subprocess.zh.md @@ -2,9 +2,13 @@ [English](subprocess.md) | 中文 -子进程 seam 分为接口（[dsh-subprocess](../../packages/subprocess/subprocess)，`ctx.subprocess`）与实现（[dsh-subprocess-local](../../packages/subprocess/subprocess-local)）；它的消费方是其他能力 seam 与进程外后端：[bash 执行器家族](bash.md)使用收集模式（collect）的批量输出，LSP 主机使用管道化的协议流 + 收集的 stderr 尾部，ACP（Agent Client Protocol）subagent 后端则使用管道化的协议流 + inherit 的 stderr。该 seam 拥有受管的 `DSH_*` 环境命名空间、共享的凭据清除（`scrubbedParentEnv`）与 `CollectedOutput` 形状；[dsh-bash](../../packages/bash/bash) 重导出这套词汇，使 bash 消费方保持单一导入入口。 +进程管理器 seam 分为接口（[dsh-subprocess](../../packages/subprocess/subprocess)，`ctx.subprocess`）与实现（[dsh-subprocess-local](../../packages/subprocess/subprocess-local)）；它的消费方是其他能力 seam 与进程外后端：[bash 执行器家族](bash.md)使用收集模式的批量输出，LSP 使用原始协议管道，PTY 后端使用终端原语，ACP（Agent Client Protocol）subagent 后端则使用管道化 ndjson 加 inherit 的 stderr。该 seam 拥有受管的 `DSH_*` 环境命名空间、共享的凭据清除（`scrubbedParentEnv`）与 `CollectedOutput` 形状；[dsh-bash](../../packages/bash/bash) 重导出这套词汇，使 bash 消费方保持单一导入入口。 -源码：[`packages/subprocess/subprocess/src/types.ts`](../../packages/subprocess/subprocess/src/types.ts) +源码：[`packages/subprocess/subprocess/src/types.ts`](../../packages/subprocess/subprocess/src/types.ts) 与 [`packages/subprocess/subprocess/src/index.ts`](../../packages/subprocess/subprocess/src/index.ts) + +## 可执行文件查找 + +一个提供方的 spawn 工作目录、可执行文件路径、普通进程与终端会话，和挂载的文件系统提供方处于同一路径与进程命名空间。`resolveExecutable(command, env?, signal?)` 验证绝对可执行文件路径，或通过提供方清理后的 `PATH` 加有意覆盖来解析裸名称。 ## 受管环境命名空间与捕获的输出 @@ -234,6 +238,12 @@ interface SubprocessOutcome { } ``` +## 终端进程原语 + +`spawnTerminal(spec)` 是非管道进程原语。提供方分配控制终端，并负责 UTF-8 文本传输、前台进程组检查与信号发送，以及一项须等待的 TERM→KILL 操作；该操作会使提供方仍可观察到的每个会话成员完全停稳，提供方则会记录执行基底特有的可观察性限制。PTY 后端仍负责提示符检测、就绪推断、scrollback、沙箱策略和持久会话所有权；普通 `spawn()` 无法重建控制终端语义。 + +终端 spec 完全指定 argv、cwd、环境覆盖、尺寸、清理宽限期与可选的分配取消。其句柄公开 `pid`、有序输出、`done`、`write`、`inspectForeground`、`signalForeground` 和须等待的 `terminate`；确切的公共形状生成到 [`ctx.subprocess` 服务目录](../cordis-catalog/services.md#ctxsubprocess--subprocessservice-abstract-seam)中。 + ## 服务行为 -抽象的 [`SubprocessService`](../../packages/subprocess/subprocess/src/index.ts) seam 只定义 `spawn`；[`LocalSubprocessService`](../../packages/subprocess/subprocess-local/src/index.ts) 是本地实现（detached 进程树、按处置方式接线的流、凭据清除、先终止再等待退出的 dispose（资源释放））。seam 契约见 [`dsh-subprocess`](../../packages/subprocess/subprocess/README.md)，具体机制见 [`dsh-subprocess-local`](../../packages/subprocess/subprocess-local/README.md)。 +抽象的 [`SubprocessService`](../../packages/subprocess/subprocess/src/index.ts) seam 定义执行世界坐标、可执行文件查找、普通 `spawn` 与 `spawnTerminal`。[`LocalSubprocessService`](../../packages/subprocess/subprocess-local/src/index.ts) 以 detached 进程树、按处置方式接线、凭据清除、`node-pty`、平台进程检查，以及先终止再等待退出的资源释放实现这些能力。接口契约见 [`dsh-subprocess`](../../packages/subprocess/subprocess/README.md)，本地机制见 [`dsh-subprocess-local`](../../packages/subprocess/subprocess-local/README.md)。 diff --git a/docs/event-producer-consumer.md b/docs/event-producer-consumer.md index d8141078a1..ea2371972f 100644 --- a/docs/event-producer-consumer.md +++ b/docs/event-producer-consumer.md @@ -24,9 +24,9 @@ This matrix shows which packages dispatch each harness-owned event and which pac | `commands/change` | `emit` | [`packages/ui/commands/src/index.ts:172`](../packages/ui/commands/src/index.ts) | [`commands`](../packages/ui/commands) (`events.dispatch`) | `apiproxy` | | `credentials/updated` | `emit` | [`packages/credentials/credentials/src/index.ts:67`](../packages/credentials/credentials/src/index.ts) | [`credentials`](../packages/credentials/credentials) (`events.dispatch`) | `apiproxy`, [`credentials`](../packages/credentials/credentials) | | `domain/changed` | `emit` | [`packages/storage/storage-domain/src/events.ts:46`](../packages/storage/storage-domain/src/events.ts) | [`storage-domain`](../packages/storage/storage-domain) (`emit`) | `apiproxy`, [`storage-domain`](../packages/storage/storage-domain), [`workspace`](../packages/workspace/workspace) | -| `fs/edit-intent` | `waterfall` | [`packages/fs/fs/src/index.ts:62`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`waterfall`), [`tool-str-replace-editor`](../packages/fs/tool-str-replace-editor) (`waterfall`) | [`fs-policy`](../packages/fs/fs-policy) | -| `fs/observed` | `emit` | [`packages/fs/fs/src/index.ts:71`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`emit`), [`tool-str-replace-editor`](../packages/fs/tool-str-replace-editor) (`emit`) | [`fs-policy`](../packages/fs/fs-policy), [`skill-local`](../packages/skill/skill-local) | -| `fs/write-intent` | `waterfall` | [`packages/fs/fs/src/index.ts:54`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`waterfall`), [`tool-str-replace-editor`](../packages/fs/tool-str-replace-editor) (`waterfall`) | [`fs-policy`](../packages/fs/fs-policy) | +| `fs/edit-intent` | `waterfall` | [`packages/fs/fs/src/index.ts:64`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`waterfall`), [`tool-str-replace-editor`](../packages/fs/tool-str-replace-editor) (`waterfall`) | [`fs-policy`](../packages/fs/fs-policy) | +| `fs/observed` | `emit` | [`packages/fs/fs/src/index.ts:73`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`emit`), [`tool-str-replace-editor`](../packages/fs/tool-str-replace-editor) (`emit`) | [`fs-policy`](../packages/fs/fs-policy), [`skill-local`](../packages/skill/skill-local) | +| `fs/write-intent` | `waterfall` | [`packages/fs/fs/src/index.ts:56`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`waterfall`), [`tool-str-replace-editor`](../packages/fs/tool-str-replace-editor) (`waterfall`) | [`fs-policy`](../packages/fs/fs-policy) | | `goal/changed` | `emit` | [`packages/goal/goal/src/domain.ts:114`](../packages/goal/goal/src/domain.ts) | [`goal`](../packages/goal/goal) (`emit`) | [`goal-session`](../packages/goal/goal-session) | | `llm/adapters-updated` | `emit` | [`packages/llm/llm/src/index.ts:73`](../packages/llm/llm/src/index.ts) | [`llm`](../packages/llm/llm) (`events.dispatch`) | `apiproxy`, [`llm`](../packages/llm/llm) | | `llm/stream` | `waterfall` | [`packages/llm/llm/src/index.ts:62`](../packages/llm/llm/src/index.ts) | [`llm`](../packages/llm/llm) (`waterfall`) | [`agent-loop`](../packages/core/agent-loop), [`llm`](../packages/llm/llm), [`llm-replay`](../packages/support/llm-replay), [`session-checkpoint-policy`](../packages/session-persistence/session-checkpoint-policy), [`session-title`](../packages/session-title/session-title) | @@ -65,7 +65,7 @@ This matrix shows which packages dispatch each harness-owned event and which pac | `connection/reset` | `runtime` (`emit`) | `ui-command`, `ui-models`, `ui-permission`, `ui-settings-general` | | `credentials/changed` | `runtime` (`emit`) | `ui-models` | | `internal/dispatch` | - | [`commands`](../packages/ui/commands), [`compact`](../packages/compact/compact), [`fs`](../packages/fs/fs), [`goal`](../packages/goal/goal), [`goal-session`](../packages/goal/goal-session), [`hook-protocol`](../packages/hooks/hook-protocol), [`llm-retry`](../packages/llm/llm-retry), [`permission`](../packages/ui/permission), [`plan-mode`](../packages/plan/plan-mode), [`pty-local`](../packages/pty/pty-local), `runtime`, [`sandbox-policy`](../packages/sandbox/sandbox-policy), [`scope`](../packages/core/scope), [`session`](../packages/core/session), [`session-title`](../packages/session-title/session-title), [`subagent`](../packages/subagent/subagent), [`time-context`](../packages/context/time-context), [`tool-todo`](../packages/todo/tool-todo), [`tools`](../packages/core/tools), [`user-approval`](../packages/ui/user-approval), [`workflow`](../packages/workflow/workflow) | -| `internal/plugin` | - | `hmr`, `loader`, `modules`, `webserver` | +| `internal/plugin` | - | `hmr`, `loader`, [`lsp-local`](../packages/lsp/lsp-local), `modules`, `webserver` | | `internal/service` | - | [`agent-presets`](../packages/preset/agent-presets), `gateway` | | `internal/status` | - | [`agent`](../packages/core/agent) | | `locale/change` | `locale` (`emit`) | `locale` | diff --git a/docs/module-graph.md b/docs/module-graph.md index e1bba183ba..5920ff0dd0 100644 --- a/docs/module-graph.md +++ b/docs/module-graph.md @@ -204,6 +204,11 @@ flowchart TD pkg_credentials["credentials"] pkg_credentials_local["credentials-local"] end + subgraph group_e2b["packages/e2b"] + pkg_e2b["e2b"] + pkg_fs_e2b["fs-e2b"] + pkg_subprocess_e2b["subprocess-e2b"] + end subgraph group_examples["packages/examples"] pkg_acp_demo["acp-demo"] pkg_agent_spine_demo["agent-spine-demo"] @@ -312,6 +317,7 @@ flowchart TD pkg_client_web --> pkg_invariants pkg_client_web_react --> pkg_invariants pkg_code_runtime --> pkg_invariants + pkg_e2b --> pkg_invariants pkg_jsonrpc_demo --> pkg_invariants pkg_host_apiproxy --> pkg_invariants pkg_host_directory_picker --> pkg_invariants @@ -334,6 +340,10 @@ flowchart TD pkg_client_runtime --> pkg_typert_registry pkg_credentials --> pkg_brand pkg_credentials --> pkg_invariants + pkg_subprocess_e2b --> pkg_e2b + pkg_subprocess_e2b --> pkg_invariants + pkg_subprocess_e2b --> pkg_subprocess + pkg_subprocess_e2b --> pkg_timeout pkg_frontend_static --> pkg_host_webserver pkg_frontend_static --> pkg_invariants pkg_agent_presets --> pkg_invariants @@ -497,12 +507,6 @@ flowchart TD pkg_code_runtime_worker --> pkg_invariants pkg_code_runtime_worker --> pkg_session pkg_code_runtime_worker --> pkg_timeout - pkg_lsp_local --> pkg_brand - pkg_lsp_local --> pkg_invariants - pkg_lsp_local --> pkg_llm - pkg_lsp_local --> pkg_lsp - pkg_lsp_local --> pkg_subprocess - pkg_lsp_local --> pkg_timeout pkg_sandbox_local --> pkg_invariants pkg_sandbox_local --> pkg_llm pkg_sandbox_local --> pkg_sandbox @@ -603,6 +607,9 @@ flowchart TD pkg_tmux_context --> pkg_bash pkg_tmux_context --> pkg_invariants pkg_tmux_context --> pkg_session + pkg_fs_e2b --> pkg_e2b + pkg_fs_e2b --> pkg_fs + pkg_fs_e2b --> pkg_invariants pkg_host_directory_picker_browse --> pkg_client_locale pkg_host_directory_picker_browse --> pkg_client_runtime pkg_host_directory_picker_browse --> pkg_client_ui_primitives @@ -613,6 +620,13 @@ flowchart TD pkg_host_directory_picker_native --> pkg_client_ui_slots pkg_host_directory_picker_native --> pkg_client_ui_workspace pkg_host_directory_picker_native --> pkg_invariants + pkg_lsp_local --> pkg_brand + pkg_lsp_local --> pkg_fs + pkg_lsp_local --> pkg_invariants + pkg_lsp_local --> pkg_llm + pkg_lsp_local --> pkg_lsp + pkg_lsp_local --> pkg_subprocess + pkg_lsp_local --> pkg_timeout pkg_pty --> pkg_agent pkg_pty --> pkg_brand pkg_pty --> pkg_invariants @@ -1186,6 +1200,7 @@ flowchart TD | [`client-web`](../packages/client/web) | `client` | [`invariants`](../packages/support/invariants) | | [`client-web-react`](../packages/client/web-react) | `client` | [`invariants`](../packages/support/invariants) | | [`code-runtime`](../packages/code-runtime/code-runtime) | `code-runtime` | [`invariants`](../packages/support/invariants) | +| [`e2b`](../packages/e2b/e2b) | `e2b` | [`invariants`](../packages/support/invariants) | | [`jsonrpc-demo`](../packages/examples/jsonrpc-demo) | `examples` | [`invariants`](../packages/support/invariants) | | [`host-apiproxy`](../packages/host/apiproxy) | `host` | [`invariants`](../packages/support/invariants) | | [`host-directory-picker`](../packages/host/directory-picker) | `host` | [`invariants`](../packages/support/invariants) | @@ -1200,6 +1215,7 @@ flowchart TD | [`client-hmr`](../packages/client/hmr) | `client` | [`client-modules`](../packages/client/modules), [`host-webserver`](../packages/host/webserver), [`invariants`](../packages/support/invariants) | | [`client-runtime`](../packages/client/runtime) | `client` | [`invariants`](../packages/support/invariants), [`type-meta`](../packages/typert/type-meta), [`typert-registry`](../packages/typert/registry) | | [`credentials`](../packages/credentials/credentials) | `credentials` | [`brand`](../packages/util/brand), [`invariants`](../packages/support/invariants) | +| [`subprocess-e2b`](../packages/e2b/subprocess-e2b) | `e2b` | [`e2b`](../packages/e2b/e2b), [`invariants`](../packages/support/invariants), [`subprocess`](../packages/subprocess/subprocess), [`timeout`](../packages/util/timeout) | | [`frontend-static`](../packages/host/frontend-static) | `host` | [`host-webserver`](../packages/host/webserver), [`invariants`](../packages/support/invariants) | | [`agent-presets`](../packages/preset/agent-presets) | `preset` | [`invariants`](../packages/support/invariants), [`paths`](../packages/util/paths), [`scope`](../packages/core/scope) | | [`helper`](../packages/sdk/helper) | `sdk` | [`brand`](../packages/util/brand), [`invariants`](../packages/support/invariants), [`subprocess`](../packages/subprocess/subprocess) | @@ -1245,7 +1261,6 @@ flowchart TD | [`client-ui-theme`](../packages/client/ui-theme) | `client` | [`client-locale`](../packages/client/locale), [`client-runtime`](../packages/client/runtime), [`client-ui-primitives`](../packages/client/ui-primitives), [`client-ui-slots`](../packages/client/ui-slots), [`invariants`](../packages/support/invariants) | | [`client-ui-workspace`](../packages/client/ui-workspace) | `client` | [`client-locale`](../packages/client/locale), [`client-runtime`](../packages/client/runtime), [`client-ui-primitives`](../packages/client/ui-primitives), [`client-ui-slots`](../packages/client/ui-slots), [`invariants`](../packages/support/invariants) | | [`code-runtime-worker`](../packages/code-runtime/code-runtime-worker) | `code-runtime` | [`code-runtime`](../packages/code-runtime/code-runtime), [`invariants`](../packages/support/invariants), [`session`](../packages/core/session), [`timeout`](../packages/util/timeout) | -| [`lsp-local`](../packages/lsp/lsp-local) | `lsp` | [`brand`](../packages/util/brand), [`invariants`](../packages/support/invariants), [`llm`](../packages/llm/llm), [`lsp`](../packages/lsp/lsp), [`subprocess`](../packages/subprocess/subprocess), [`timeout`](../packages/util/timeout) | | [`sandbox-local`](../packages/sandbox/sandbox-local) | `sandbox` | [`invariants`](../packages/support/invariants), [`llm`](../packages/llm/llm), [`sandbox`](../packages/sandbox/sandbox) | | [`session-projection`](../packages/session-projection/session-projection) | `session-projection` | [`invariants`](../packages/support/invariants), [`session`](../packages/core/session) | | [`llm-retry`](../packages/llm/llm-retry) | `llm` | [`agent`](../packages/core/agent), [`invariants`](../packages/support/invariants), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`timeout`](../packages/util/timeout) | @@ -1271,8 +1286,10 @@ flowchart TD | [`client-ui-layout`](../packages/client/ui-layout) | `client` | [`client-runtime`](../packages/client/runtime), [`client-ui-slots`](../packages/client/ui-slots), [`client-ui-theme`](../packages/client/ui-theme), [`invariants`](../packages/support/invariants) | | [`time-context`](../packages/context/time-context) | `context` | [`agent`](../packages/core/agent), [`invariants`](../packages/support/invariants), [`session`](../packages/core/session) | | [`tmux-context`](../packages/context/tmux-context) | `context` | [`agent`](../packages/core/agent), [`bash`](../packages/bash/bash), [`invariants`](../packages/support/invariants), [`session`](../packages/core/session) | +| [`fs-e2b`](../packages/e2b/fs-e2b) | `e2b` | [`e2b`](../packages/e2b/e2b), [`fs`](../packages/fs/fs), [`invariants`](../packages/support/invariants) | | [`host-directory-picker-browse`](../packages/host/directory-picker-browse) | `host` | [`client-locale`](../packages/client/locale), [`client-runtime`](../packages/client/runtime), [`client-ui-primitives`](../packages/client/ui-primitives), [`client-ui-slots`](../packages/client/ui-slots), [`client-ui-workspace`](../packages/client/ui-workspace), [`invariants`](../packages/support/invariants) | | [`host-directory-picker-native`](../packages/host/directory-picker-native) | `host` | [`client-runtime`](../packages/client/runtime), [`client-ui-slots`](../packages/client/ui-slots), [`client-ui-workspace`](../packages/client/ui-workspace), [`invariants`](../packages/support/invariants) | +| [`lsp-local`](../packages/lsp/lsp-local) | `lsp` | [`brand`](../packages/util/brand), [`fs`](../packages/fs/fs), [`invariants`](../packages/support/invariants), [`llm`](../packages/llm/llm), [`lsp`](../packages/lsp/lsp), [`subprocess`](../packages/subprocess/subprocess), [`timeout`](../packages/util/timeout) | | [`pty`](../packages/pty/pty) | `pty` | [`agent`](../packages/core/agent), [`brand`](../packages/util/brand), [`invariants`](../packages/support/invariants) | | [`sandbox-policy`](../packages/sandbox/sandbox-policy) | `sandbox` | [`agent`](../packages/core/agent), [`invariants`](../packages/support/invariants), [`sandbox`](../packages/sandbox/sandbox), [`session`](../packages/core/session), [`system-prompt`](../packages/core/system-prompt) | | [`scripts`](../packages/sdk/scripts) | `sdk` | [`app-boot`](../packages/ui/app-boot), [`invariants`](../packages/support/invariants) | diff --git a/examples/headless-agent/README.i18n.yaml b/examples/headless-agent/README.i18n.yaml index 7b6139f28b..4a71194e5f 100644 --- a/examples/headless-agent/README.i18n.yaml +++ b/examples/headless-agent/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write examples/headless-agent/README.md -README.md: e00f3d2d4fd21a860239f3d3a3e5eb2d7520f14a -README.zh.md: 6cd845783b1c112ba73676474b176ec28a4d0b78 +README.md: 6e80e56dec70c2be341ae5dfbad70e13a5109715 +README.zh.md: ea8c41b9ae75f0cee3edd78e59408f37c19182d3 diff --git a/examples/headless-agent/README.md b/examples/headless-agent/README.md index e00f3d2d4f..6e80e56dec 100644 --- a/examples/headless-agent/README.md +++ b/examples/headless-agent/README.md @@ -17,6 +17,16 @@ The product command is [`dsh run`](../../apps/cli/README.md): it accepts one non Snapshot suites run this directory's configuration through [`tests/fixtures/headless-driver.ts`](tests/fixtures/headless-driver.ts), an unexported test-only process that emits canonical session events as JSONL before its result record. That stream is test infrastructure, not a supported CLI output format. Child sessions surface only through parent tool events and results. +## E2B POC overlay + +[`e2b.cordis.yml`](e2b.cordis.yml) replaces the local filesystem and subprocess providers with one shared E2B sandbox while retaining `dsh-bash-local` and the same model-facing tools. Put `E2B_API_KEY` beside `DEEPSEEK_API_KEY` in the gitignored root `.env`, then run the credential-gated live composition, which drives FS, Bash, PTY, and LSP in one sandbox and proves final deletion: + +```sh +pnpm exec vitest run --config vitest.e2e.config.ts packages/e2b/e2b/tests/composition.e2e.ts +``` + +The overlay creates the same absolute cwd inside the sandbox, but it does not upload or mount the host workspace. File and Bash mutations exist only in E2B; Cordis, model calls, agent/session state, session logs, skills, and SDK buffers remain on the host. The composition kills its sandbox on timeout and disposal. It is a provider-composition POC, not a whole-harness migration or a workspace-sync feature. + ## Advanced configuration [`advanced.cordis.yml`](advanced.cordis.yml) adds Code Mode and the Cordis tools to the test composition. diff --git a/examples/headless-agent/README.zh.md b/examples/headless-agent/README.zh.md index 6cd845783b..ea8c41b9ae 100644 --- a/examples/headless-agent/README.zh.md +++ b/examples/headless-agent/README.zh.md @@ -17,6 +17,16 @@ pnpm run dsh run "fix the failing test in this workspace" 快照套件通过 [`tests/fixtures/headless-driver.ts`](tests/fixtures/headless-driver.ts) 运行本目录的配置。这个未导出且仅供测试使用的进程会在结果记录之前，以 JSONL 发出规范会话事件。该事件流属于测试基础设施，不是受支持的 CLI（命令行界面）输出格式。子会话只通过父会话的工具事件和结果对外显示。 +## E2B POC overlay + +[`e2b.cordis.yml`](e2b.cordis.yml) 使用一个共享 E2B 沙箱替换本地文件系统与进程管理提供方，同时保留 `dsh-bash-local` 和相同的面向模型工具。请在 git 忽略的根目录 `.env` 中，将 `E2B_API_KEY` 与 `DEEPSEEK_API_KEY` 放在一起，然后运行凭据门控的实机组合测试；它在同一个沙箱中驱动 FS、Bash、PTY 和 LSP，并证明沙箱最终被删除： + +```sh +pnpm exec vitest run --config vitest.e2e.config.ts packages/e2b/e2b/tests/composition.e2e.ts +``` + +该 overlay 会在沙箱中创建拼写相同的绝对 cwd，但不会上传或挂载宿主工作区。文件与 Bash 变更只存在于 E2B；Cordis、模型调用、agent／会话状态、会话日志、skill（技能）和 SDK 缓冲仍在宿主上。该组合会在超时和资源释放时终止其沙箱。它是提供方组合 POC，而不是完整 harness 迁移或工作区同步功能。 + ## 高级配置 [`advanced.cordis.yml`](advanced.cordis.yml) 在测试组装中添加 Code Mode 和 Cordis 工具。 diff --git a/examples/headless-agent/e2b.cordis.yml b/examples/headless-agent/e2b.cordis.yml new file mode 100644 index 0000000000..61bdbc8aa7 --- /dev/null +++ b/examples/headless-agent/e2b.cordis.yml @@ -0,0 +1,57 @@ +# POC overlay: keep the advanced headless agent and model-facing tools, but +# place its filesystem and process substrate in one short-lived E2B sandbox; +# the generic Bash, PTY, and LSP consumers compose above them. +# +# One-world invariant: e2b.cwd, sandbox-policy.workspaceRoot, and bash-local's +# default workdir (implicit host process.cwd()) must all name the same remote +# directory. Only e2b.cwd is created at sandbox open; dropping its !!js line +# falls back to /home/user/workspace while Bash and PTY keep targeting the +# host path, so every tool call fails with a remote spawn error. +- id: base + name: '@cordisjs/plugin-include' + config: + path: ./advanced.cordis.yml + patches: + - id: subprocess + name: '@deepseek-ai/dsh-subprocess-local' + disabled: true + - id: fs-local + name: '@deepseek-ai/dsh-fs-local' + disabled: true + - insert: + - id: e2b + name: '@deepseek-ai/dsh-e2b' + config: + cwd: !!js process.cwd() + timeoutMs: 300000 + - id: subprocess-e2b + name: '@deepseek-ai/dsh-subprocess-e2b' + - id: fs-e2b + name: '@deepseek-ai/dsh-fs-e2b' + - id: sandbox-policy + name: '@deepseek-ai/dsh-sandbox-policy' + config: + mode: danger-full-access + workspaceRoot: !!js process.cwd() + - id: pty + name: '@deepseek-ai/dsh-pty' + - id: pty-local + name: '@deepseek-ai/dsh-pty-local' + - id: tool-pty + name: '@deepseek-ai/dsh-tool-pty' + - id: lsp + name: '@deepseek-ai/dsh-lsp' + - id: lsp-local + name: '@deepseek-ai/dsh-lsp-local' + config: + servers: + typescript: + command: npx + args: [--yes, typescript-language-server@5.0.0, --stdio] + extensionToLanguage: + .ts: typescript + .tsx: typescriptreact + .js: javascript + .jsx: javascriptreact + - id: tool-lsp + name: '@deepseek-ai/dsh-tool-lsp' diff --git a/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts b/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts new file mode 100644 index 0000000000..7a787c735b --- /dev/null +++ b/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts @@ -0,0 +1,202 @@ +import { readFile } from 'node:fs/promises' +import { resolve } from 'node:path' +import { boot } from '@deepseek-ai/dsh-app-boot' +import { Inbox } from '@deepseek-ai/dsh-agent' +import type { Agent } from '@deepseek-ai/dsh-agent' +import { Session, SessionId } from '@deepseek-ai/dsh-session' +import type {} from '@deepseek-ai/dsh-fs-e2b' +import type {} from '@deepseek-ai/dsh-bash-local' +import type {} from '@deepseek-ai/dsh-lsp-local' +import type {} from '@deepseek-ai/dsh-pty-local' + +const configPath = process.argv[2] +if (configPath === undefined) throw new Error('usage: bin.ts ') + +const ctx = await boot('e2b-composition', resolve(configPath)) +const ownerFiber = ctx.plugin(() => {}) +const ownerId = SessionId('e2b-live-owner') +const session = Session.create(ownerId) +const owner: Agent = { + id: ownerId, + options: {}, + session, + inbox: new Inbox(session, { inserted: () => {}, discarded: () => {}, claimed: () => {} }), + status: 'idle', + ctx: ownerFiber.ctx, + send() {}, + followup() {}, + steer() {}, + inject() {}, + cancel() {}, + runMaintenance: task => task(new AbortController().signal), + whenIdle: () => Promise.resolve(), +} +const unregisterOwner = ctx.agents.register(owner) +let terminalId: Awaited>['sessionId'] | undefined +try { + const sandbox = await ctx.e2b.getSandbox() + const fromFs = await ctx.fs.resolve('from-fs.txt') + const written = await ctx.fs.writeText(fromFs, 'written-by-fs\n', { kind: 'createIfAbsent' }) + const observed = await ctx.fs.stat(fromFs) + if (observed?.version !== written.version) { + throw new Error(`E2B rename did not preserve version metadata: ${JSON.stringify({ written, observed })}`) + } + await ctx.fs.editText( + fromFs, + { oldString: 'written-by-fs', newString: 'versioned-by-fs', replaceAll: false }, + { version: observed.version }, + ) + const bashRead = await ctx.bash.run(ctx.bash.resolve({ command: 'cat from-fs.txt' })) + if (bashRead.exitCode !== 0 || bashRead.stdout.text !== 'versioned-by-fs\n') { + throw new Error(`E2B Bash could not read the FS write: ${JSON.stringify(bashRead)}`) + } + + const bashWrite = await ctx.bash.run(ctx.bash.resolve({ command: "printf 'written-by-bash\\n' > from-bash.txt" })) + if (bashWrite.exitCode !== 0) { + throw new Error(`E2B Bash could not write the shared filesystem: ${JSON.stringify(bashWrite)}`) + } + const fromBash = await ctx.fs.resolve('from-bash.txt') + const fsRead = await ctx.fs.readText(fromBash) + + const environmentHandle = ctx.subprocess.spawn({ + argv: ['env'], + cwd: process.cwd(), + stdio: { stdin: 'ignore', stdout: { maxBytes: 65_536 }, stderr: { maxBytes: 4_096 } }, + graceMs: 500, + env: { + 'FOO-BAR': 'hyphen-value', + DSH_EXPLICIT: 'managed-value', + TOKEN_EXPLICIT: 'credential-value', + }, + }) + const environmentOutcome = await environmentHandle.done + const environmentText = environmentHandle.collected.stdout?.readFrom(0).text + if (environmentOutcome.exitCode !== 0 || environmentText === undefined) { + throw new Error(`E2B subprocess environment probe failed: ${JSON.stringify(environmentOutcome)}`) + } + const environmentLines = new Set(environmentText.trimEnd().split('\n')) + const explicitEnvironment = [ + 'FOO-BAR=hyphen-value', + 'DSH_EXPLICIT=managed-value', + 'TOKEN_EXPLICIT=credential-value', + ].every(entry => environmentLines.has(entry)) + if (!explicitEnvironment) throw new Error(`E2B subprocess dropped an explicit environment entry: ${environmentText}`) + + const splitUtf8Handle = ctx.subprocess.spawn({ + argv: ['bash', '-c', "printf '\\344'; sleep 0.05; printf '\\275'; sleep 0.05; printf '\\240'; sleep 0.05; printf '\\345'; sleep 0.05; printf '\\245'; sleep 0.05; printf '\\275'"], + cwd: process.cwd(), + stdio: { stdin: 'ignore', stdout: { maxBytes: 32 }, stderr: { maxBytes: 4_096 } }, + graceMs: 500, + env: {}, + }) + const splitUtf8Outcome = await splitUtf8Handle.done + const splitUtf8Output = splitUtf8Handle.collected.stdout?.readFrom(0).text + if (splitUtf8Outcome.exitCode !== 0 || splitUtf8Output !== '你好') { + throw new Error(`E2B subprocess corrupted split UTF-8 output: ${JSON.stringify({ splitUtf8Outcome, splitUtf8Output })}`) + } + + const outputDrainStarted = Date.now() + const outputDrainHandle = ctx.subprocess.spawn({ + argv: ['bash', '-c', "bash -c 'exec -a dsh-output-drain-descendant sleep 30' & printf 'leader-done\\n'"], + cwd: process.cwd(), + stdio: { stdin: 'ignore', stdout: { maxBytes: 64 }, stderr: { maxBytes: 4_096 } }, + graceMs: 250, + env: {}, + }) + const outputDrainOutcome = await outputDrainHandle.done + const outputDrainText = outputDrainHandle.collected.stdout?.readFrom(0).text + const outputDrainElapsedMs = Date.now() - outputDrainStarted + outputDrainHandle.terminate() + const outputDrainExited = await outputDrainHandle.waitForExit(AbortSignal.timeout(5_000)) + const outputDrainProcesses = await sandbox.commands.list() + const outputDrainClean = !outputDrainProcesses.some(processInfo => + JSON.stringify([processInfo.cmd, processInfo.args]).includes('dsh-output-drain-descendant'), + ) + if (outputDrainOutcome.exitCode !== 0 || outputDrainText !== 'leader-done\n' + || outputDrainElapsedMs >= 10_000 || !outputDrainExited || !outputDrainClean) { + throw new Error(`E2B subprocess did not bound descendant-held output: ${JSON.stringify({ + outputDrainOutcome, outputDrainText, outputDrainElapsedMs, outputDrainExited, outputDrainClean, + })}`) + } + + const lspFixture = await readFile(new URL('./fixture-lsp.mjs', import.meta.url), 'utf8') + const remoteLspFixture = await ctx.fs.resolve('fixture-lsp.mjs') + await ctx.fs.writeText(remoteLspFixture, lspFixture, { kind: 'createIfAbsent' }) + const remoteSource = await ctx.fs.resolve('multibyte # file.ts') + await ctx.fs.writeText(remoteSource, 'const café = "你好"\nconsole.log(café)\n', { kind: 'createIfAbsent' }) + const hover = await ctx.lsp.query({ + operation: 'hover', + filePath: 'multibyte # file.ts', + position: { line: 0, character: 7 }, + workspaceRoot: process.cwd(), + }) + const definition = await ctx.lsp.query({ + operation: 'goToDefinition', + filePath: 'multibyte # file.ts', + position: { line: 0, character: 7 }, + workspaceRoot: process.cwd(), + }) + + const terminal = await ctx.pty.spawn(owner, { type: 'shell' }) + terminalId = terminal.sessionId + const terminalEcho = await ctx.pty.startSend(owner, terminal.sessionId, { + text: "printf 'PTY-你好\\n'", + submit: true, + }).done + const sleeping = ctx.pty.startSend(owner, terminal.sessionId, { + text: "printf 'DSH_SLEEP_%s\\n' READY; sleep 30", + submit: true, + }) + let sleepReadyOutput = '' + const sleepReadyDeadline = Date.now() + 5_000 + while (!sleepReadyOutput.includes('DSH_SLEEP_READY\n')) { + sleepReadyOutput += sleeping.readOutput().delta + if (sleepReadyOutput.includes('DSH_SLEEP_READY\n')) break + const settled = await Promise.race([ + sleeping.done.then(result => ({ result })), + new Promise(resolveDelay => setTimeout(() => { resolveDelay(undefined) }, 25)), + ]) + if (settled !== undefined) { + throw new Error(`E2B PTY successor settled before executing: ${JSON.stringify(settled.result)}`) + } + if (Date.now() >= sleepReadyDeadline) throw new Error(`E2B PTY successor did not execute: ${sleepReadyOutput}`) + } + const terminalSignal = await ctx.pty.signal(owner, terminal.sessionId, 'SIGINT') + const interrupted = await sleeping.done + const stubborn = await ctx.pty.startSend(owner, terminal.sessionId, { + text: "bash -c 'trap \"\" TERM; exec sleep 30' & printf 'DSH_STUBBORN_PID=%s\\n' \"$!\"", + submit: true, + }).done + const stubbornMatch = /DSH_STUBBORN_PID=([1-9][0-9]*)/.exec(stubborn.viewport) + if (stubbornMatch?.[1] === undefined) throw new Error(`E2B PTY did not report its stubborn child: ${stubborn.viewport}`) + const stubbornPid = Number(stubbornMatch[1]) + const terminalScrollback = ctx.pty.read(owner, terminal.sessionId, { count: 50 }) + await ctx.pty.kill(owner, terminal.sessionId, 'live E2B composition complete') + terminalId = undefined + const stubbornProbe = await sandbox.commands.run(`if kill -0 ${stubbornPid} 2>/dev/null; then printf alive; else printf gone; fi`) + const terminalTreeCleanup = stubbornProbe.stdout === 'gone' + if (!terminalTreeCleanup) throw new Error(`E2B PTY left process ${stubbornPid} alive after close`) + + process.stdout.write(`${JSON.stringify({ + sandboxId: (await ctx.e2b.getSandbox()).sandboxId, + bashRead: bashRead.stdout.text, + fsRead, + explicitEnvironment, + splitUtf8Output, + hover, + definition, + terminal: { + motd: terminal.motd, + echo: terminalEcho, + signal: terminalSignal, + interrupted, + treeCleanup: terminalTreeCleanup, + scrollback: terminalScrollback.text, + }, + })}\n`) +} finally { + if (terminalId !== undefined) await ctx.pty.kill(owner, terminalId, 'fixture cleanup').catch(() => false) + unregisterOwner() + await ownerFiber.dispose() + await ctx.fiber.dispose() +} diff --git a/examples/headless-agent/tests/fixtures/e2b/e2b/cordis.yml b/examples/headless-agent/tests/fixtures/e2b/e2b/cordis.yml new file mode 100644 index 0000000000..e6fd968779 --- /dev/null +++ b/examples/headless-agent/tests/fixtures/e2b/e2b/cordis.yml @@ -0,0 +1,57 @@ +# One-world invariant (same pairing as examples/headless-agent/e2b.cordis.yml): +# e2b.cwd and sandbox-policy.workspaceRoot must name the same remote directory, +# which is also bash-local's implicit default workdir. +- id: e2b + name: '@deepseek-ai/dsh-e2b' + config: + cwd: !!js process.cwd() + timeoutMs: 180000 + +- id: subprocess-e2b + name: '@deepseek-ai/dsh-subprocess-e2b' + +- id: bash + name: '@deepseek-ai/dsh-bash-local' + config: + timeoutMs: 30000 + +- id: fs-e2b + name: '@deepseek-ai/dsh-fs-e2b' + +- id: agents + name: '@deepseek-ai/dsh-agent' + +- id: sandbox-policy + name: '@deepseek-ai/dsh-sandbox-policy' + config: + mode: danger-full-access + workspaceRoot: !!js process.cwd() + +- id: pty + name: '@deepseek-ai/dsh-pty' + +- id: pty-local + name: '@deepseek-ai/dsh-pty-local' + config: + pollIntervalMs: 25 + exactProbeAfterMs: 150 + idleSilenceMs: 2000 + handoffGraceMs: 500 + timeoutMs: 5000 + disposeGraceMs: 1000 + +- id: lsp + name: '@deepseek-ai/dsh-lsp' + +- id: lsp-local + name: '@deepseek-ai/dsh-lsp-local' + config: + servers: + fixture: + command: node + args: + - !!js process.cwd() + '/fixture-lsp.mjs' + extensionToLanguage: + .ts: typescript + shutdownTimeoutMs: 1000 + killGraceMs: 500 diff --git a/examples/headless-agent/tests/fixtures/e2b/e2b/fixture-lsp.mjs b/examples/headless-agent/tests/fixtures/e2b/e2b/fixture-lsp.mjs new file mode 100644 index 0000000000..dc4f7bb219 --- /dev/null +++ b/examples/headless-agent/tests/fixtures/e2b/e2b/fixture-lsp.mjs @@ -0,0 +1,85 @@ +import { Buffer } from 'node:buffer' + +let pending = Buffer.alloc(0) +let source = '' +let sourceUri = '' + +function send(message) { + const body = Buffer.from(JSON.stringify(message)) + process.stdout.write(`Content-Length: ${body.length}\r\n\r\n`) + process.stdout.write(body) +} + +function respond(id, result) { + send({ jsonrpc: '2.0', id, result }) +} + +function dispatch(message) { + switch (message.method) { + case 'initialize': + respond(message.id, { + capabilities: { + positionEncoding: 'utf-16', + textDocumentSync: { openClose: true, change: 1 }, + definitionProvider: true, + referencesProvider: true, + implementationProvider: true, + hoverProvider: true, + }, + }) + return + case 'textDocument/didOpen': + source = message.params.textDocument.text + sourceUri = message.params.textDocument.uri + return + case 'textDocument/didClose': + source = '' + sourceUri = '' + return + case 'textDocument/hover': + if (!source.includes('const café = "你好"')) { + send({ jsonrpc: '2.0', id: message.id, error: { code: -32000, message: 'multibyte source was corrupted' } }) + return + } + respond(message.id, { + contents: { kind: 'markdown', value: '**remote hover** 你好 café' }, + range: { start: { line: 0, character: 6 }, end: { line: 0, character: 10 } }, + }) + return + case 'textDocument/definition': + case 'textDocument/references': + case 'textDocument/implementation': + respond(message.id, [{ + uri: sourceUri, + range: { start: { line: 0, character: 6 }, end: { line: 0, character: 10 } }, + }]) + return + case 'shutdown': + respond(message.id, null) + return + case 'exit': + process.exit(0) + return + } +} + +function drain() { + for (;;) { + const headerEnd = pending.indexOf('\r\n\r\n') + if (headerEnd < 0) return + const header = pending.subarray(0, headerEnd).toString('ascii') + const match = /(?:^|\r\n)Content-Length: ([0-9]+)(?:\r\n|$)/i.exec(header) + if (!match) throw new Error('missing Content-Length') + const length = Number(match[1]) + const bodyStart = headerEnd + 4 + if (pending.length < bodyStart + length) return + const body = pending.subarray(bodyStart, bodyStart + length) + pending = pending.subarray(bodyStart + length) + dispatch(JSON.parse(body.toString('utf8'))) + } +} + +process.stdin.on('data', chunk => { + pending = Buffer.concat([pending, chunk]) + drain() +}) diff --git a/examples/jsonrpc-agent/persistent-tools.cordis.yml b/examples/jsonrpc-agent/persistent-tools.cordis.yml index 6f42441ca7..ebe0a00e61 100644 --- a/examples/jsonrpc-agent/persistent-tools.cordis.yml +++ b/examples/jsonrpc-agent/persistent-tools.cordis.yml @@ -18,6 +18,9 @@ mode: danger-full-access workspaceRoot: !!js process.env.DSH_CWD ?? process.cwd() +- id: subprocess + name: '@deepseek-ai/dsh-subprocess-local' + - id: pty name: '@deepseek-ai/dsh-pty' diff --git a/examples/package.json b/examples/package.json index 1f6e99d240..48ac39f5ca 100644 --- a/examples/package.json +++ b/examples/package.json @@ -26,7 +26,9 @@ "@deepseek-ai/dsh-compact-basic": "workspace:*", "@deepseek-ai/dsh-compact-tool-result-prune": "workspace:*", "@deepseek-ai/dsh-credentials-local": "workspace:*", + "@deepseek-ai/dsh-e2b": "workspace:*", "@deepseek-ai/dsh-fs-local": "workspace:*", + "@deepseek-ai/dsh-fs-e2b": "workspace:*", "@deepseek-ai/dsh-fs-policy": "workspace:*", "@deepseek-ai/dsh-fs-sandbox": "workspace:^", "@deepseek-ai/dsh-goal": "workspace:*", @@ -76,6 +78,7 @@ "@deepseek-ai/dsh-subagent-fork": "workspace:*", "@deepseek-ai/dsh-subagent-spawn": "workspace:*", "@deepseek-ai/dsh-subprocess-local": "workspace:*", + "@deepseek-ai/dsh-subprocess-e2b": "workspace:*", "@deepseek-ai/dsh-system-prompt": "workspace:*", "@deepseek-ai/dsh-tasks-local": "workspace:*", "@deepseek-ai/dsh-time-context": "workspace:*", diff --git a/knip.json b/knip.json index ede8b2c738..d154cd646b 100644 --- a/knip.json +++ b/knip.json @@ -45,6 +45,7 @@ "headless-agent/tests/fixtures/time-context-mock-llm.ts", "headless-agent/tests/fixtures/telemetry-otel-driver.ts", "headless-agent/tests/fixtures/telemetry-redact-rule.ts", + "headless-agent/tests/fixtures/e2b/e2b/bin.ts", "acp-agent/tests/snapshots/lsp-definition/workspace/subject.ts", "acp-agent/tests/fixtures/child-question-tripwire.ts", "acp-agent/tests/fixtures/partial-landlock-sandbox.ts", @@ -229,6 +230,16 @@ "tests/**/*.ts" ] }, + "packages/e2b/e2b": { + "entry": [ + "tests/**/*.spec.ts", + "tests/**/*.e2e.ts" + ], + "project": [ + "src/**/*.ts", + "tests/**/*.ts" + ] + }, "packages/context/time-context": { "entry": [ "tests/**/*.spec.ts", diff --git a/packages/README.i18n.yaml b/packages/README.i18n.yaml index db14ad97d5..9d1ef06532 100644 --- a/packages/README.i18n.yaml +++ b/packages/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/README.md -README.md: fcbe4859954efa20f41a5a7e26bc0f1406cbe7f8 -README.zh.md: 304b95c4fa47ced455411b03529b77956cfb2953 +README.md: d1e579633ea597c46003af97ab9a47e5e616ca73 +README.zh.md: aa6577413899627a947b446ef8f6545492c880cb diff --git a/packages/README.md b/packages/README.md index fcbe485995..d1e579633e 100644 --- a/packages/README.md +++ b/packages/README.md @@ -6,7 +6,7 @@ Packages use the `@deepseek-ai/dsh-*` scope. Cordis `Service` subclasses and fun ## Hierarchy -Packages live at `packages///`; groups are containers, while names remain `@deepseek-ai/dsh-`. **Each group README is the canonical package/ctx-key map.** +Groups hold `packages///`; names stay `@deepseek-ai/dsh-`. **Group READMEs own package/ctx-key maps.** | Group | Role | Release expectation | |---|---|---| @@ -16,6 +16,7 @@ Packages live at `packages///`; groups are containers, while names r | [`goal/`](goal/README.md) | Same-session goal persistence and lifecycle | Product — stable surface | | [`feedback/`](feedback/README.md) | Human feedback | Product — stable surface | | [`llm/`](llm/README.md) | LLM capability family: the abstract service + provider adapters | Product — stable surface | +| [`e2b/`](e2b/README.md) | E2B providers | POC | | [`subprocess/`](subprocess/README.md) | Subprocess capability family: spawn seam + local process-tree implementation | Product — stable surface | | [`bash/`](bash/README.md) | Bash capability family: executor seam, local impl, model-facing tool | Product — stable surface | | [`pty/`](pty/README.md) | Persistent PTY capability family: owner-scoped sessions, local implementation, and model-facing tools | Product — stable surface | diff --git a/packages/README.zh.md b/packages/README.zh.md index 304b95c4fa..aa65774138 100644 --- a/packages/README.zh.md +++ b/packages/README.zh.md @@ -6,7 +6,7 @@ ## 层级结构 -包位于 `packages///`；组是容器，包名仍为 `@deepseek-ai/dsh-`。**每个组 README 是规范的包／ctx 键映射。** +包按组置于 `packages///`；包名仍为 `@deepseek-ai/dsh-`。**组 README 负责包／ctx 键映射。** | 组 | 职责 | 发布预期 | |---|---|---| @@ -16,6 +16,7 @@ | [`goal/`](goal/README.md) | 同会话 goal 的持久化与生命周期 | 产品：稳定表面 | | [`feedback/`](feedback/README.md) | 人类反馈 | 产品：稳定表面 | | [`llm/`](llm/README.md) | LLM（大语言模型）能力系列：抽象服务 + 提供方适配器 | 产品：稳定表面 | +| [`e2b/`](e2b/README.md) | E2B 提供方 | POC | | [`subprocess/`](subprocess/README.md) | 进程管理能力系列：spawn seam + 本地进程树实现 | 产品：稳定表面 | | [`bash/`](bash/README.md) | Bash 能力系列：执行器 seam、本地实现、面向模型的工具 | 产品：稳定表面 | | [`pty/`](pty/README.md) | 持久 PTY 能力系列：按所有者隔离的会话、本地实现和面向模型的工具 | 产品：稳定表面 | diff --git a/packages/bash/pwsh-local/tests/executor.spec.ts b/packages/bash/pwsh-local/tests/executor.spec.ts index c7da16c44c..350ea4de89 100644 --- a/packages/bash/pwsh-local/tests/executor.spec.ts +++ b/packages/bash/pwsh-local/tests/executor.spec.ts @@ -119,6 +119,8 @@ describe('spawn construction (pure, every platform)', () => { /** A subprocess service that records spawn specs and settles instantly. */ class CapturingSubprocessService extends SubprocessService { specs: SubprocessSpawnSpec[] = [] + override async resolveExecutable(command: string): Promise { return command } + override spawnTerminal(): Promise { throw new Error('pwsh spawns pipes, never terminals') } private readonly reader: SubprocessOutputReader = { readFrom: () => ({ text: '', lossy: false, nextOffset: 0 }), } diff --git a/packages/context/workspace-context/tests/workspace-context.spec.ts b/packages/context/workspace-context/tests/workspace-context.spec.ts index 0af12596cc..fcdcd7c422 100644 --- a/packages/context/workspace-context/tests/workspace-context.spec.ts +++ b/packages/context/workspace-context/tests/workspace-context.spec.ts @@ -75,6 +75,14 @@ class RecordingFileSystem extends FileSystem { return { targetKey: FsTargetKey(absolute), displayPath: absolute } } + override processPath(target: FsTarget): string { return String(target.targetKey) } + + override fileUrl(target: FsTarget): string { return `file://${target.targetKey}` } + + override contains(parent: FsTarget, child: FsTarget): boolean { + return child.targetKey === parent.targetKey || String(child.targetKey).startsWith(`${parent.targetKey}/`) + } + override async stat(target: FsTarget, signal?: AbortSignal): Promise { if (signal !== undefined) this.signals.push(signal) signal?.throwIfAborted() diff --git a/packages/cordis/tool-cordis/src/api-catalog.ts b/packages/cordis/tool-cordis/src/api-catalog.ts index 8ea7b3b98c..a9140d6396 100644 --- a/packages/cordis/tool-cordis/src/api-catalog.ts +++ b/packages/cordis/tool-cordis/src/api-catalog.ts @@ -330,6 +330,16 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [ }, ], }, + { + key: 'e2b', + summary: 'Creates one lazily consumable E2B SDK handle and deletes the sandbox at timeout or disposal.', + methods: [ + { + signature: 'async getSandbox(): Promise', + jsDoc: '/**\n * Return the shared live SDK handle.\n * @returns the created sandbox after the configured cwd exists.\n * @throws when E2B rejects creation or the service is disposing.\n */', + }, + ], + }, { key: 'fs', summary: 'Abstract filesystem provider.', @@ -338,6 +348,18 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [ signature: 'abstract resolve(path: string, opts?: { cwd?: string; signal?: AbortSignal }): Promise', jsDoc: '/**\n * Resolve a model/plugin-supplied path into a stable {@link FsTarget}. May perform I/O (a\n * remote/sandboxed backend may need a round-trip to map a path to a stable identity), hence\n * async even though the local backend only normalizes + realpaths.\n *\n * @param path - the path to resolve; relative paths resolve against `opts.cwd`.\n * @param opts - optional cwd override and cancellation signal.\n * @returns the stable target; the same file yields the same `targetKey`.\n */', }, + { + signature: 'abstract processPath(target: FsTarget): string', + jsDoc: '/**\n * Return the canonical absolute path a subprocess in this filesystem\'s\n * execution world can open. The path is deliberately separate from\n * {@link FsTarget.targetKey}: consumers may pass this value to another OS\n * capability, but must continue treating the target key as opaque.\n * @param target - the resolved target whose process path is required.\n * @returns an absolute path in the backend\'s execution world.\n */', + }, + { + signature: 'abstract fileUrl(target: FsTarget): string', + jsDoc: '/**\n * Return the canonical `file:` URI for a target in this filesystem\'s\n * execution world. Backends own URI encoding because the host platform may\n * differ from the execution platform.\n * @param target - the resolved target to encode.\n * @returns the target\'s canonical file URI.\n */', + }, + { + signature: 'abstract contains(parent: FsTarget, child: FsTarget): boolean', + jsDoc: '/**\n * Test canonical containment without exposing or parsing backend target\n * keys. Both targets must come from this provider.\n * @param parent - canonical directory target.\n * @param child - canonical candidate target.\n * @returns true when `child` is `parent` or a descendant of it.\n */', + }, { signature: 'abstract stat(target: FsTarget, signal?: AbortSignal): Promise', jsDoc: '/**\n * Return target metadata, or `undefined` when the target does not exist.\n * @param target - the resolved target to stat.\n * @param signal - aborts the metadata round-trip.\n * @returns metadata only, never content; undefined for an absent target.\n */', @@ -1000,10 +1022,18 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [ key: 'subprocess', summary: 'Abstract subprocess service.', methods: [ + { + signature: 'abstract resolveExecutable( command: string, env?: Readonly>, signal?: AbortSignal, ): Promise', + jsDoc: '/**\n * Resolve one configured executable in this provider\'s execution world.\n * Absolute paths are verified; bare names use the provider\'s scrubbed PATH\n * plus explicit environment overrides. Relative paths containing separators\n * are rejected: no current consumer defines which directory they would\n * resolve against, so providers fail loud instead of guessing.\n * @param command - absolute executable path or bare PATH name.\n * @param env - explicit environment entries used for lookup.\n * @param signal - aborts remote or local lookup.\n * @returns a canonical executable path.\n */', + }, { signature: 'abstract spawn(spec: SubprocessSpawnSpec): SubprocessHandle', jsDoc: '/**\n * Start one managed child process from a fully-specified spec; this seam\n * applies no defaults.\n * @param spec - argv, directory, stdio dispositions, grace, cancellation, and environment.\n * @returns the live process handle (streams/readers, signalling, outcome promise).\n */', }, + { + signature: 'abstract spawnTerminal(spec: SubprocessTerminalSpawnSpec): Promise', + jsDoc: '/**\n * Allocate a real terminal and start one owned process session. This is the\n * only non-pipe process primitive: implementations own terminal byte I/O,\n * foreground groups, signals, and complete session-tree cleanup.\n * @param spec - fully specified argv, cwd, environment, dimensions, grace, and allocation cancellation.\n * @returns the live terminal handle after allocation succeeds.\n */', + }, ], }, { @@ -2913,6 +2943,22 @@ export const TYPE_API: readonly TypeApiEntry[] = [ name: 'SubprocessStdio', declaration: 'export interface SubprocessStdio {\n stdin: SubprocessStdinMode;\n stdout: SubprocessOutputMode;\n stderr: SubprocessOutputMode;\n}', }, + { + name: 'SubprocessTerminalForeground', + declaration: 'export interface SubprocessTerminalForeground {\n processGroupId: number;\n inputWaiting: boolean;\n}', + }, + { + name: 'SubprocessTerminalHandle', + declaration: 'export interface SubprocessTerminalHandle {\n readonly pid: number;\n readonly output: Readable;\n readonly done: Promise;\n write(data: string): Promise;\n inspectForeground(): Promise;\n signalForeground(signal: SubprocessTerminalSignal): Promise;\n terminate(): Promise;\n}', + }, + { + name: 'SubprocessTerminalSignal', + declaration: 'export type SubprocessTerminalSignal = \'SIGINT\' | \'SIGTERM\' | \'SIGKILL\' | \'SIGTSTP\' | \'SIGHUP\';', + }, + { + name: 'SubprocessTerminalSpawnSpec', + declaration: 'export interface SubprocessTerminalSpawnSpec {\n argv: readonly string[];\n cwd: string;\n env?: Record | undefined;\n rows: number;\n cols: number;\n graceMs: number;\n signal?: AbortSignal | undefined;\n}', + }, { name: 'SurfaceEvent', declaration: 'export type SurfaceEvent = SessionEvent & {\n surfaceOp: SurfaceOp;\n};', diff --git a/packages/e2b/README.i18n.yaml b/packages/e2b/README.i18n.yaml new file mode 100644 index 0000000000..4125487253 --- /dev/null +++ b/packages/e2b/README.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write packages/e2b/README.md +README.md: f9758e2748aaa2acffb0e928752b2b0fb70cd0a4 +README.zh.md: de068c16d4309499f5dcaa2d08cd9b6cc3023d94 diff --git a/packages/e2b/README.md b/packages/e2b/README.md new file mode 100644 index 0000000000..f9758e2748 --- /dev/null +++ b/packages/e2b/README.md @@ -0,0 +1,15 @@ +# e2b/ — E2B remote runtime family + +English | [中文](README.zh.md) + +An experimental provider-composition POC that places one filesystem/process execution world in an E2B Linux sandbox. E2B supplies only sandbox lifecycle and the two fundamental OS adapters; provider-neutral consumers build higher capabilities above them. + +| Package | ctx key | Role | +|---|---|---| +| [`e2b`](e2b/README.md) (`@deepseek-ai/dsh-e2b`) | `ctx.e2b` | Create one sandbox, prepare its working/runtime directories, expose the shared SDK handle, and delete it on timeout or disposal | +| [`fs-e2b`](fs-e2b/README.md) (`@deepseek-ai/dsh-fs-e2b`) | `ctx.fs` | Implement the filesystem seam over E2B Filesystem APIs | +| [`subprocess-e2b`](subprocess-e2b/README.md) (`@deepseek-ai/dsh-subprocess-e2b`) | `ctx.subprocess` | Implement executable lookup, managed process groups and stdio, remote spill files, and terminal sessions over E2B Commands and PTY APIs | + +The existing [`dsh-bash-local`](../bash/bash-local/README.md), [`dsh-pty-local`](../pty/pty-local/README.md), and [`dsh-lsp-local`](../lsp/lsp-local/README.md) need no E2B-specific forks. They delegate every execution-world operation to `ctx.fs` and `ctx.subprocess`, so mounting the two E2B adapters places their mutable work in the same sandbox. + +This boundary does not move the harness process, Cordis objects, model calls, agent/session state, session persistence, skills, higher-level protocol state, or E2B SDK buffers. The [portable execution-world decision](../../.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md) owns both the generic composition and this POC boundary. diff --git a/packages/e2b/README.zh.md b/packages/e2b/README.zh.md new file mode 100644 index 0000000000..de068c16d4 --- /dev/null +++ b/packages/e2b/README.zh.md @@ -0,0 +1,15 @@ +# e2b/ — E2B 远程运行时家族 + +[English](README.md) | 中文 + +这是一个实验性提供方组合 POC，把一个文件系统／进程执行环境放进 E2B Linux 沙箱。E2B 只提供沙箱生命周期与两个基础 OS 适配器；提供方无关的消费方在其上构建更高层能力。 + +| 包（package） | ctx 键 | 职责 | +|---|---|---| +| [`e2b`](e2b/README.md)（`@deepseek-ai/dsh-e2b`） | `ctx.e2b` | 创建一个沙箱，准备其工作目录与运行时目录，公开共享 SDK 句柄，并在超时或资源释放时将其删除 | +| [`fs-e2b`](fs-e2b/README.md)（`@deepseek-ai/dsh-fs-e2b`） | `ctx.fs` | 通过 E2B Filesystem API 实现文件系统 seam | +| [`subprocess-e2b`](subprocess-e2b/README.md)（`@deepseek-ai/dsh-subprocess-e2b`） | `ctx.subprocess` | 通过 E2B Commands 与 PTY API 实现可执行文件查找、受管进程组与 stdio、远程 spill 文件及终端会话 | + +现有的 [`dsh-bash-local`](../bash/bash-local/README.md)、[`dsh-pty-local`](../pty/pty-local/README.md) 和 [`dsh-lsp-local`](../lsp/lsp-local/README.md) 无需 E2B 专用 fork。它们把执行环境中的所有操作委托给 `ctx.fs` 和 `ctx.subprocess`，因此挂载这两个 E2B 适配器后，它们执行的可变操作都发生在同一个沙箱内。 + +该边界不会迁移 harness 进程、Cordis 对象、模型调用、agent（智能体）／会话状态、会话持久化、skill（技能）、更高层协议状态或 E2B SDK 缓冲。[可移植执行世界决策](../../.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md)同时界定通用组合和此 POC 边界。 diff --git a/packages/e2b/e2b/README.i18n.yaml b/packages/e2b/e2b/README.i18n.yaml new file mode 100644 index 0000000000..3587067d31 --- /dev/null +++ b/packages/e2b/e2b/README.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write packages/e2b/e2b/README.md +README.md: 7ade7c3d6522d8fa6d54d7011766238451b17c9a +README.zh.md: b683f33d2211106cf422e780b2b37904b0c640ad diff --git a/packages/e2b/e2b/README.md b/packages/e2b/e2b/README.md new file mode 100644 index 0000000000..7ade7c3d65 --- /dev/null +++ b/packages/e2b/e2b/README.md @@ -0,0 +1,44 @@ +# @deepseek-ai/dsh-e2b + +English | [中文](README.zh.md) + +Shared lifecycle owner for one E2B sandbox. The filesystem and subprocess adapters inject `ctx.e2b`, await its single SDK handle, and therefore inhabit the same remote Linux working tree and process world. The package pins `e2b@2.29.1`; the [family map](../README.md) lists the opt-in composition. + +## Configuration + +```yaml +- id: e2b + name: '@deepseek-ai/dsh-e2b' + config: + cwd: /home/user/workspace + timeoutMs: 300000 + +- id: subprocess-e2b + name: '@deepseek-ai/dsh-subprocess-e2b' + +- id: fs-e2b + name: '@deepseek-ai/dsh-fs-e2b' +``` + +`apiKey` is optional and otherwise reads `E2B_API_KEY`; the key configures the host SDK connection and is never installed in the sandbox. `cwd` defaults to `/home/user/workspace` and must be an absolute POSIX path. `timeoutMs` defaults to five minutes and controls the sandbox lifetime; expiry deletes the sandbox. + +## Lifecycle and ownership + +Construction starts one sandbox creation. Before resolving `getSandbox()`, the service creates `cwd` and the private `cwd/.dsh-e2b` adapter-state directory, verifies that the reserved path is a real directory rather than a symlink or another file type, then sets it to mode `0700`. Each adapter-internal E2B command shell receives a fresh randomized root-level `HOME`, so the SDK's fixed login shell does not resolve profile files from the mutable user home before the control command. + +Disposal first prevents new handle acquisition, then awaits setup and deletes the sandbox. A `SandboxNotFoundError` means expiry or another owner already deleted it and is accepted as quiescence. Initial directory setup failure makes one deletion attempt; the configured E2B timeout bounds a second failure. Provider plugins must load after this owner and dispose before it. + +## Model Experience + +None, as this shared runtime owner registers no model-visible context; provider adapters and their consumers own any rendered effects. + +#### KV Cache effect + +No direct invalidation; this package does not contribute request tokens. + +## Known Limitations and Deferred Work + +- **This is not a whole-harness runtime** — Cordis services, agent/session state, session logs, LLM requests, skills, and SDK-side buffers stay in the host process. +- **Sandbox state is ephemeral** — disposal and timeout delete the sandbox; reconnect, pause/leave retention, templates, volumes, and snapshots are outside this POC. +- **No deployment platform is configured** — network policy, host-workspace synchronization, and sandbox discovery are outside this POC. +- **`cwd` is a resolution convention, not containment** — adapters and commands can address other sandbox paths; E2B network access retains the base image's policy. diff --git a/packages/e2b/e2b/README.zh.md b/packages/e2b/e2b/README.zh.md new file mode 100644 index 0000000000..b683f33d22 --- /dev/null +++ b/packages/e2b/e2b/README.zh.md @@ -0,0 +1,44 @@ +# @deepseek-ai/dsh-e2b + +[English](README.md) | 中文 + +一个 E2B 沙箱的共享生命周期所有者。文件系统与进程管理适配器注入 `ctx.e2b`，等待其唯一的 SDK 句柄，因此处于同一个远程 Linux 工作树与进程环境中。本包固定使用 `e2b@2.29.1`；可选组合见[包族索引](../README.md)。 + +## 配置 + +```yaml +- id: e2b + name: '@deepseek-ai/dsh-e2b' + config: + cwd: /home/user/workspace + timeoutMs: 300000 + +- id: subprocess-e2b + name: '@deepseek-ai/dsh-subprocess-e2b' + +- id: fs-e2b + name: '@deepseek-ai/dsh-fs-e2b' +``` + +`apiKey` 可省略；省略时读取 `E2B_API_KEY`。该密钥只配置宿主 SDK 连接，绝不会安装进沙箱。`cwd` 默认为 `/home/user/workspace`，并且必须是绝对 POSIX 路径。`timeoutMs` 默认为 5 分钟并控制沙箱生命周期；超时会删除沙箱。 + +## 生命周期与所有权 + +构造阶段会启动一次沙箱创建。服务在 `getSandbox()` 结算前创建 `cwd` 和私有的 `cwd/.dsh-e2b` 适配器状态目录，验证该预留路径是真实目录而非符号链接或其他文件类型，再把该目录的 mode 设为 `0700`。每个适配器内部的 E2B 命令 shell 都会获得一个位于根目录下、全新随机生成的 `HOME`，因此 SDK 固定使用的登录 shell 不会在控制命令之前解析可变用户主目录中的配置文件。 + +资源释放会先阻止继续获取新句柄，再等待初始化完成，然后删除沙箱。`SandboxNotFoundError` 表示沙箱已因超时或被另一个所有者删除，因此可视为完全停稳。初始目录设置失败时会尝试删除一次；若该尝试也失败，则由已配置的 E2B 超时约束沙箱的存活时间。提供方插件必须在该所有者之后加载，并在其之前 dispose（资源释放）。 + +## 模型体验 + +无。本共享运行时所有者不注册模型可见上下文；提供方适配器及其消费方拥有所有渲染效果。 + +#### KV Cache 影响 + +不会直接失效；本包不会贡献请求 token。 + +## 已知限制与延后工作 + +- **这不是完整的 harness 运行时**：Cordis 服务、agent（智能体）／会话状态、会话日志、LLM（大语言模型）请求、skill（技能）和 SDK 侧缓冲仍留在宿主进程中。 +- **沙箱状态是短暂的**：资源释放和超时都会删除沙箱；重新连接、pause/leave 保留、模板、卷和快照均不在本 POC 范围内。 +- **没有配置部署平台**：网络策略、宿主工作区同步和沙箱发现均不在本 POC 范围内。 +- **`cwd` 是解析约定，而不是包含边界**：适配器和命令可以访问沙箱中的其他路径；E2B 网络访问也继续采用基础镜像的策略。 diff --git a/packages/e2b/e2b/package.json b/packages/e2b/e2b/package.json new file mode 100644 index 0000000000..a77e3d9464 --- /dev/null +++ b/packages/e2b/e2b/package.json @@ -0,0 +1,40 @@ +{ + "name": "@deepseek-ai/dsh-e2b", + "description": "Shared E2B sandbox lifecycle for DeepSeek Harness provider adapters", + "version": "0.0.1", + "private": true, + "type": "module", + "main": "lib/index.js", + "types": "lib/types/index.d.ts", + "exports": { + ".": { + "types": "./lib/types/index.d.ts", + "default": "./lib/index.js" + }, + "./invariant": { + "types": "./lib/types/invariant.d.ts", + "default": "./lib/invariant.js" + }, + "./src/*": "./src/*", + "./package.json": "./package.json" + }, + "files": [ + "lib/index.js", + "lib/invariant.js", + "lib/types/**/*.d.ts" + ], + "license": "BSD-3-Clause", + "peerDependencies": { + "@deepseek-ai/dsh-invariants": "^0.0.1", + "cordis": "^4.0.0-rc.7" + }, + "dependencies": { + "e2b": "2.29.1", + "schemastery": "^3.18.0" + }, + "devDependencies": { + "@deepseek-ai/dsh-invariants": "workspace:^", + "@deepseek-ai/dsh-loader-smoke": "workspace:^", + "cordis": "^4.0.0-rc.7" + } +} diff --git a/packages/e2b/e2b/src/index.ts b/packages/e2b/e2b/src/index.ts new file mode 100644 index 0000000000..906941d421 --- /dev/null +++ b/packages/e2b/e2b/src/index.ts @@ -0,0 +1,182 @@ +/** + * Shared ownership of one E2B sandbox. Capability adapters await the same SDK + * handle, so filesystem and process operations inhabit one remote Linux world. + * @module @deepseek-ai/dsh-e2b + */ + +import { randomUUID } from 'node:crypto' +import { posix } from 'node:path' +import { Context, Service } from 'cordis' +import z from 'schemastery' +import { FileType, Sandbox, SandboxNotFoundError } from 'e2b' + +export { + CommandExitError, + FileNotFoundError, + FileType, + Sandbox, + SandboxNotFoundError, +} from 'e2b' +export type { CommandHandle, CommandResult, EntryInfo } from 'e2b' + +/** + * Quote one opaque argument for the SDK's unavoidable `/bin/bash -l -c` layer. + * @param value - Exact argument value to preserve. + * @returns A single shell word with no interpolation. + */ +export function quoteE2BShellArg(value: string): string { + return `'${value.replaceAll('\'', "'\"'\"'")}'` +} + +/** + * Isolate E2B's hard-coded login shell behind a fresh randomized home path. + * @param overrides - Additional environment entries for the internal command. + * @returns A fresh mutable map that the E2B SDK may extend. + */ +export function e2bControlEnvs( + overrides: Readonly> = {}, +): Record { + return { ...overrides, HOME: `/.dsh-e2b-control-${randomUUID()}` } +} + +/** Configuration for the shared E2B sandbox owner. */ +export interface Config { + /** API key; omission reads `E2B_API_KEY`. It is never forwarded into the sandbox. */ + apiKey?: string + /** Shared remote working directory, created before adapters receive the sandbox. */ + cwd?: string + /** E2B sandbox lifetime in milliseconds; expiry always deletes the sandbox. */ + timeoutMs?: number +} + +interface ResolvedConfig { + apiKey: string + cwd: string + timeoutMs: number +} + +interface SchemaResolvedConfig extends Config { + cwd: string + timeoutMs: number +} + +declare module 'cordis' { + interface Context { + e2b: E2BSandboxService + } +} + +/** + * Creates one lazily consumable E2B SDK handle and deletes the sandbox at + * timeout or disposal. Creation begins at plugin construction; adapters await + * {@link getSandbox} before their first operation. + */ +export class E2BSandboxService extends Service { + static Config: z = z.object({ + apiKey: z.string(), + cwd: z.string().default('/home/user/workspace'), + timeoutMs: z.number().default(300_000), + }) + + /** Validated remote working directory shared by provider adapters. */ + readonly cwd: string + /** Remote directory reserved for adapter-owned process and terminal state. */ + readonly runtimeRoot: string + + private readonly config: ResolvedConfig + private readonly ready: Promise + private disposed = false + + constructor(ctx: Context, config: Config) { + super(ctx, 'e2b') + // Schemastery fills these fields before construction; the type does not encode that step. + const resolved = config as SchemaResolvedConfig + const apiKey = config.apiKey ?? process.env.E2B_API_KEY + this.config = { + apiKey: apiKey ?? '', + cwd: resolved.cwd, + timeoutMs: resolved.timeoutMs, + } + this.validate() + this.cwd = this.config.cwd + this.runtimeRoot = posix.join(this.cwd, '.dsh-e2b') + this.ready = this.open() + // A deployment may load the owner before any adapter uses it. Keep a + // failed eager connection observed; getSandbox() still returns the error. + void this.ready.catch(() => {}) + + ctx.effect(() => async () => { + this.disposed = true + let sandbox: Sandbox + try { + sandbox = await this.ready + } catch (_sandboxSetupFailure) { + // open() either acquired no sandbox or already made the POC's one rollback attempt. + return + } + try { + await sandbox.kill() + } catch (error: unknown) { + if (!(error instanceof SandboxNotFoundError)) throw error + } + }, 'e2b sandbox teardown') + } + + /** + * Return the shared live SDK handle. + * @returns the created sandbox after the configured cwd exists. + * @throws when E2B rejects creation or the service is disposing. + */ + async getSandbox(): Promise { + if (this.disposed) throw new Error('E2B sandbox service is disposing') + const sandbox = await this.ready + // Disposal can race the awaited sandbox readiness despite the synchronous precheck. + // oxlint-disable-next-line typescript/no-unnecessary-condition -- Awaiting readiness yields to disposal. + if (this.disposed) throw new Error('E2B sandbox service is disposing') + return sandbox + } + + private validate(): void { + if (this.config.apiKey.length === 0) { + throw new Error('dsh-e2b: configure apiKey or set E2B_API_KEY') + } + if (!posix.isAbsolute(this.config.cwd)) { + throw new Error(`dsh-e2b: cwd must be an absolute Linux path: ${this.config.cwd}`) + } + if (!Number.isFinite(this.config.timeoutMs) || this.config.timeoutMs <= 0) { + throw new Error('dsh-e2b: timeoutMs must be a positive finite number') + } + } + + private async open(): Promise { + const sandbox = await Sandbox.create({ + apiKey: this.config.apiKey, + timeoutMs: this.config.timeoutMs, + secure: true, + lifecycle: { onTimeout: 'kill' }, + }) + try { + await sandbox.files.makeDir(this.cwd) + await sandbox.files.makeDir(this.runtimeRoot) + const runtimeRoot = await sandbox.files.getInfo(this.runtimeRoot) + if (runtimeRoot.type !== FileType.DIR || runtimeRoot.symlinkTarget !== undefined) { + throw new Error(`dsh-e2b: runtime root must be a real directory: ${this.runtimeRoot}`) + } + await sandbox.commands.run( + `chmod 700 -- ${quoteE2BShellArg(this.runtimeRoot)}`, + { envs: e2bControlEnvs() }, + ) + return sandbox + } catch (error: unknown) { + try { + await sandbox.kill() + } catch (_sandboxSetupRollbackFailure) { + // TODO(e2b-setup-rollback): Add retry state only if a real double failure + // outlives E2B's configured sandbox timeout. + } + throw error + } + } +} + +export default E2BSandboxService diff --git a/packages/e2b/e2b/src/invariant.ts b/packages/e2b/e2b/src/invariant.ts new file mode 100644 index 0000000000..891cabb2db --- /dev/null +++ b/packages/e2b/e2b/src/invariant.ts @@ -0,0 +1,30 @@ +/** + * Package-owned invariant companion for `@deepseek-ai/dsh-e2b`. + * @module @deepseek-ai/dsh-e2b/invariant + */ + +/* jscpd:ignore-start */ +import type { Context } from 'cordis' +import type { InvariantInstaller } from '@deepseek-ai/dsh-invariants' + +const PACKAGE_NAME = '@deepseek-ai/dsh-e2b' + +/** Cordis companion plugin name. */ +export const name = 'e2b-invariant' +/** Service required before the companion can reserve package ownership. */ +export const inject = ['invariants'] + +/** + * No runtime invariant: sandbox creation and teardown have one SDK promise and + * no independent event or mutable-data relationship to cross-check. + */ +const install: InvariantInstaller = () => {} + +/** + * Register this package's invariant companion. + * @param ctx - Cordis context carrying the invariant service. + * @returns the installed registration's disposer after setup succeeds. + */ +export const apply = (ctx: Context): Promise<() => void> => + Promise.resolve(ctx.invariants.register(PACKAGE_NAME, install)) +/* jscpd:ignore-end */ diff --git a/packages/e2b/e2b/tests/composition.e2e.ts b/packages/e2b/e2b/tests/composition.e2e.ts new file mode 100644 index 0000000000..01316aeb2d --- /dev/null +++ b/packages/e2b/e2b/tests/composition.e2e.ts @@ -0,0 +1,182 @@ +import { access } from 'node:fs/promises' +import { join, posix } from 'node:path' +import { fileURLToPath } from 'node:url' +import { Context } from 'cordis' +import { describe, expect, it } from 'vitest' +import { Inbox } from '@deepseek-ai/dsh-agent' +import type { Agent } from '@deepseek-ai/dsh-agent' +import { runLoaderSmoke } from '@deepseek-ai/dsh-loader-smoke' +import { + FileNotFoundError, + Sandbox, + SandboxNotFoundError, +} from '@deepseek-ai/dsh-e2b' +import PtyService, { PtySessionId } from '@deepseek-ai/dsh-pty' +import { LocalPtyBackend } from '@deepseek-ai/dsh-pty-local' +import { Session, SessionId } from '@deepseek-ai/dsh-session' +import E2BSubprocessService from '@deepseek-ai/dsh-subprocess-e2b' + +const fixtureRoot = fileURLToPath(new URL('../../../../examples/headless-agent/tests/fixtures/e2b/e2b/', import.meta.url)) +const binScript = join(fixtureRoot, 'bin.ts') +const configPath = join(fixtureRoot, 'cordis.yml') +const tsconfigPath = fileURLToPath(new URL('../../../../tsconfig.json', import.meta.url)) + +describe.skipIf(!process.env.E2B_API_KEY)('E2B live Loader composition', () => { + it('scrubs credentials before actual E2B command and PTY login shells', async () => { + const apiKey = process.env.E2B_API_KEY + if (apiKey === undefined) throw new Error('E2B_API_KEY disappeared before the PTY environment test') + const sandbox = await Sandbox.create({ + apiKey, + envs: { NPM_TOKEN: 'sentinel-secret', DSH_STALE: 'sentinel-stale', KEEP: 'visible' }, + timeoutMs: 60_000, + secure: true, + lifecycle: { onTimeout: 'kill' }, + }) + try { + const profileLeakPath = '/home/user/dsh-e2b-bootstrap-profile-leak' + const hostileProfile = [ + 'if [[ "${NPM_TOKEN-}" == "sentinel-secret" ]]; then', + ` printf leaked > ${profileLeakPath}`, + 'fi', + '', + ].join('\n') + await sandbox.files.write([ + { path: '/home/user/.bash_profile', data: hostileProfile }, + { path: '/home/user/.profile', data: hostileProfile }, + { path: '/home/user/.bashrc', data: hostileProfile }, + ]) + const ctx = new Context() + ctx.provide('e2b', { + cwd: '/home/user', + runtimeRoot: '/home/user/.dsh-e2b', + getSandbox: async () => sandbox, + } as never) + ctx.provide('sandboxPolicy', { + defaultMode: 'danger-full-access', + workspaceRoot: '/home/user', + } as never) + const ptyFiber = await ctx.plugin(PtyService) + const subprocessFiber = await ctx.plugin(E2BSubprocessService) + const node = await ctx.subprocess.resolveExecutable('node') + const relativeNodePath = posix.relative(ctx.e2b.cwd, posix.dirname(node)) || '.' + await expect(ctx.subprocess.resolveExecutable('node', { PATH: relativeNodePath })).resolves.toBe(node) + await expect(sandbox.files.read(profileLeakPath)).rejects.toBeInstanceOf(FileNotFoundError) + const environmentProbe = ctx.subprocess.spawn({ + argv: ['/bin/bash', '-c', [ + 'dsh_leak=0', + 'for dsh_pid in "$PPID" $(ps -o pid= --ppid "$PPID"); do', + ' [[ "$dsh_pid" == "$$" ]] && continue', + ' if tr "\\0" "\\n" < "/proc/$dsh_pid/environ" 2>/dev/null | grep -Fqx "NPM_TOKEN=sentinel-secret"; then dsh_leak=1; fi', + 'done', + 'printf "DIRECT=<%s> LEAK=<%s>\\n" "${NPM_TOKEN-}" "$dsh_leak"', + ].join('\n')], + cwd: '/home/user', + stdio: { stdin: 'ignore', stdout: { maxBytes: 1_024 }, stderr: { maxBytes: 1_024 } }, + graceMs: 500, + env: {}, + }) + await expect(environmentProbe.done).resolves.toEqual({ exitCode: 0, signal: null }) + expect(environmentProbe.collected.stdout?.readFrom(0).text).toBe('DIRECT=<> LEAK=<0>\n') + await expect(sandbox.files.read(profileLeakPath)).rejects.toBeInstanceOf(FileNotFoundError) + const ownerId = SessionId('e2b-pty-env-owner') + const ownerSession = Session.create(ownerId) + const owner: Agent = { + id: ownerId, + options: {}, + session: ownerSession, + inbox: new Inbox(ownerSession, { inserted: () => {}, discarded: () => {}, claimed: () => {} }), + status: 'idle', + ctx, + send() {}, + followup() {}, + steer() {}, + inject() {}, + cancel() {}, + runMaintenance: task => task(new AbortController().signal), + whenIdle: () => Promise.resolve(), + } + const backend = new LocalPtyBackend(ctx, { + backendType: 'shell', shellPath: '/bin/bash', shellArgs: ['--noprofile', '--norc', '-i'], + rows: 24, cols: 80, + scrollbackLines: 100, scrollbackMaxBytes: 65_536, maxReadBytes: 16_384, + pollIntervalMs: 25, exactProbeAfterMs: 150, idleSilenceMs: 1_000, + handoffGraceMs: 500, timeoutMs: 5_000, disposeGraceMs: 1_000, + }) + const session = await backend.spawn({ sessionId: PtySessionId('env'), owner, type: 'shell' }) + const result = await session.startSend({ + text: "printf 'NPM=<%s> DSH=<%s> KEEP=<%s>\\n' \"$NPM_TOKEN\" \"$DSH_STALE\" \"$KEEP\"", + submit: true, + }).done + expect(result.viewport).toContain('NPM=<> DSH=<> KEEP=') + expect(result.viewport).not.toContain('sentinel-secret') + expect(result.viewport).not.toContain('sentinel-stale') + await expect(sandbox.files.read(profileLeakPath)).rejects.toBeInstanceOf(FileNotFoundError) + await session.close('environment test complete') + await subprocessFiber.dispose() + await ptyFiber.dispose() + + } finally { + await sandbox.kill().catch(() => false) + } + }, 70_000) + + it('runs FS, Bash, PTY, and LSP in one sandbox and deletes it', async () => { + const { stdout, stderr } = await runLoaderSmoke({ + label: 'E2B composition', + tempDirPrefix: 'dsh-e2b-composition-', + binScript, + libBinScript: binScript, + configPath, + tsconfigPath, + env: { + NODE_OPTIONS: [process.env.NODE_OPTIONS, '--disable-warning=ExperimentalWarning'].filter(Boolean).join(' '), + }, + processTimeoutMs: 180_000, + inspect: async (cwd) => { + for (const name of ['from-fs.txt', 'from-bash.txt', 'multibyte # file.ts', 'fixture-lsp.mjs']) { + await expect(access(join(cwd, name))).rejects.toMatchObject({ code: 'ENOENT' }) + } + }, + }) + + expect(stderr).toBe('') + const output = JSON.parse(stdout) as Record + expect(output).toMatchObject({ + bashRead: 'versioned-by-fs\n', + fsRead: 'written-by-bash\n', + explicitEnvironment: true, + splitUtf8Output: '你好', + hover: { + kind: 'hover', + hover: { contents: '**remote hover** 你好 café' }, + }, + definition: { + kind: 'locations', + locations: [{ range: { start: { line: 0, character: 6 }, end: { line: 0, character: 10 } } }], + }, + terminal: { + echo: { waitReason: 'stdin_read', sessionStatus: { kind: 'running' } }, + signal: { delivered: true }, + interrupted: { sessionStatus: { kind: 'running' } }, + treeCleanup: true, + }, + }) + const terminalMotd = (output.terminal as { motd: string }).motd + expect(terminalMotd.length).toBeGreaterThan(0) + expect(terminalMotd).not.toContain('exec /bin/bash') + expect(terminalMotd).not.toContain('.dsh-e2b/terminals/') + expect((output.terminal as { echo: { viewport: string } }).echo.viewport).toContain('PTY-你好') + expect((output.terminal as { scrollback: string }).scrollback).toContain('PTY-你好') + expect((output.terminal as { signal: { targetPgid: number } }).signal.targetPgid).toBeGreaterThan(0) + expect(['stdin_read', 'inferred_idle']).toContain( + (output.terminal as { interrupted: { waitReason: string } }).interrupted.waitReason, + ) + const apiKey = process.env.E2B_API_KEY + if (apiKey === undefined) throw new Error('E2B_API_KEY disappeared during the live composition test') + await expect(Sandbox.getInfo(String(output.sandboxId), { apiKey })).rejects.toBeInstanceOf(SandboxNotFoundError) + await expect.poll(async () => { + const sandboxes = await Sandbox.list({ apiKey }).nextItems() + return sandboxes.some(sandbox => sandbox.sandboxId === output.sandboxId) + }, { interval: 250, timeout: 5_000 }).toBe(false) + }, 195_000) +}) diff --git a/packages/e2b/e2b/tests/e2b.spec.ts b/packages/e2b/e2b/tests/e2b.spec.ts new file mode 100644 index 0000000000..b108bc68b0 --- /dev/null +++ b/packages/e2b/e2b/tests/e2b.spec.ts @@ -0,0 +1,247 @@ +import { beforeEach, describe, expect, it, vi } from 'vitest' +import type { Mock } from 'vitest' +import { Context } from 'cordis' +import type { Sandbox as SandboxType } from 'e2b' +import E2BSandboxService, { + e2bControlEnvs, + FileType, + SandboxNotFoundError, + quoteE2BShellArg, +} from '@deepseek-ai/dsh-e2b' +import * as E2BInvariant from '../src/invariant.ts' +import InvariantService from '@deepseek-ai/dsh-invariants' + +const sdk = vi.hoisted(() => ({ + create: vi.fn(), +})) + +vi.mock('e2b', async (importOriginal) => { + const actual = await importOriginal() + // The mock replaces only the SDK's static factory surface and is never constructed. + // oxlint-disable-next-line typescript/no-extraneous-class -- The SDK contract is a class with a static factory. + class FakeSandbox { + static create(...args: unknown[]): unknown { + return sdk.create(...args) + } + } + return { ...actual, Sandbox: FakeSandbox } +}) + +interface SandboxFixture { + sandbox: SandboxType + makeDir: ReturnType + getInfo: ReturnType + run: Mock + kill: ReturnType +} + +type RunCommand = ( + command: string, + options?: { envs?: Record }, +) => Promise<{ exitCode: number; stdout: string; stderr: string }> + +function fakeSandbox(id = 'sandbox-1'): SandboxFixture { + const makeDir = vi.fn().mockResolvedValue(true) + const getInfo = vi.fn().mockResolvedValue({ type: FileType.DIR }) + const run = vi.fn().mockResolvedValue({ exitCode: 0, stdout: '', stderr: '' }) + const kill = vi.fn().mockResolvedValue(undefined) + const sandbox = { + sandboxId: id, + files: { makeDir, getInfo }, + commands: { run }, + kill, + } as unknown as SandboxType + return { sandbox, makeDir, getInfo, run, kill } +} + +beforeEach(() => { + sdk.create.mockReset() + vi.unstubAllEnvs() +}) + +describe('E2BSandboxService', () => { + it('gives each SDK login shell a fresh non-overridable control home', () => { + const first = e2bControlEnvs({ HOME: '/hostile', NPM_TOKEN: '' }) + const second = e2bControlEnvs() + + expect(first.HOME).toMatch(/^\/\.dsh-e2b-control-/) + expect(first).toEqual({ HOME: first.HOME, NPM_TOKEN: '' }) + expect(first.HOME).not.toBe(second.HOME) + }) + + it('creates one protected shared sandbox and kills it on default disposal', async () => { + const fixture = fakeSandbox() + sdk.create.mockResolvedValue(fixture.sandbox) + const ctx = new Context() + const fiber = await ctx.plugin(E2BSandboxService, { apiKey: 'test-key' }) + + const service = ctx.e2b + await expect(service.getSandbox()).resolves.toBe(fixture.sandbox) + expect(service.cwd).toBe('/home/user/workspace') + expect(service.runtimeRoot).toBe('/home/user/workspace/.dsh-e2b') + expect(sdk.create).toHaveBeenCalledWith({ + apiKey: 'test-key', + timeoutMs: 300_000, + secure: true, + lifecycle: { onTimeout: 'kill' }, + }) + expect(fixture.makeDir).toHaveBeenNthCalledWith(1, '/home/user/workspace') + expect(fixture.makeDir).toHaveBeenNthCalledWith(2, '/home/user/workspace/.dsh-e2b') + expect(fixture.getInfo).toHaveBeenCalledWith('/home/user/workspace/.dsh-e2b') + const runOptions = fixture.run.mock.calls[0]?.[1] + expect(runOptions?.envs?.HOME).toMatch(/^\/\.dsh-e2b-control-/) + expect(fixture.run).toHaveBeenCalledWith( + "chmod 700 -- '/home/user/workspace/.dsh-e2b'", + { envs: { HOME: runOptions?.envs?.HOME } }, + ) + + await fiber.dispose() + expect(fixture.kill).toHaveBeenCalledOnce() + await expect(service.getSandbox()).rejects.toThrow(/disposing/) + }) + + it('rejects handle acquisition when disposal starts during setup', async () => { + const fixture = fakeSandbox() + const opening = Promise.withResolvers() + sdk.create.mockReturnValue(opening.promise) + const ctx = new Context() + const fiber = await ctx.plugin(E2BSandboxService, { apiKey: 'test-key' }) + + const acquisition = ctx.e2b.getSandbox() + const disposing = fiber.dispose() + opening.resolve(fixture.sandbox) + + await expect(acquisition).rejects.toThrow(/disposing/) + await expect(disposing).resolves.toBeUndefined() + expect(fixture.kill).toHaveBeenCalledOnce() + }) + + it('reads the key from the environment and honors the configured cwd and lifetime', async () => { + vi.stubEnv('E2B_API_KEY', 'environment-key') + const fixture = fakeSandbox('configured-sandbox') + sdk.create.mockResolvedValue(fixture.sandbox) + const ctx = new Context() + const fiber = await ctx.plugin(E2BSandboxService, { + cwd: '/workspace/project', + timeoutMs: 60_000, + }) + await ctx.e2b.getSandbox() + + expect(sdk.create).toHaveBeenCalledWith({ + apiKey: 'environment-key', + timeoutMs: 60_000, + secure: true, + lifecycle: { onTimeout: 'kill' }, + }) + expect(ctx.e2b.cwd).toBe('/workspace/project') + await fiber.dispose() + expect(fixture.kill).toHaveBeenCalledOnce() + }) + + it('accepts a missing sandbox when disposal itself requests deletion', async () => { + const fixture = fakeSandbox() + fixture.kill.mockRejectedValue(new SandboxNotFoundError('already deleted')) + sdk.create.mockResolvedValue(fixture.sandbox) + const ctx = new Context() + const errors: unknown[] = [] + ctx.logger.error = ((error: unknown) => { errors.push(error) }) as typeof ctx.logger.error + const fiber = await ctx.plugin(E2BSandboxService, { apiKey: 'test-key' }) + await ctx.e2b.getSandbox() + + await fiber.dispose() + expect(fixture.kill).toHaveBeenCalledOnce() + expect(errors).toEqual([]) + }) + + it('does not classify other disposal failures as an already-gone sandbox', async () => { + const fixture = fakeSandbox() + const failure = new Error('disposition unknown') + fixture.kill.mockRejectedValue(failure) + sdk.create.mockResolvedValue(fixture.sandbox) + const ctx = new Context() + const errors: unknown[] = [] + ctx.logger.error = ((error: unknown) => { errors.push(error) }) as typeof ctx.logger.error + const fiber = await ctx.plugin(E2BSandboxService, { apiKey: 'test-key' }) + await ctx.e2b.getSandbox() + await expect(fiber.dispose()).resolves.toBeUndefined() + expect(fixture.kill).toHaveBeenCalledOnce() + expect(errors).toContain(failure) + }) + + it('kills a newly created sandbox when remote directory setup fails', async () => { + const fixture = fakeSandbox() + fixture.makeDir.mockRejectedValueOnce(new Error('setup failed')) + sdk.create.mockResolvedValue(fixture.sandbox) + const ctx = new Context() + const fiber = await ctx.plugin(E2BSandboxService, { apiKey: 'test-key' }) + + await expect(ctx.e2b.getSandbox()).rejects.toThrow('setup failed') + expect(fixture.kill).toHaveBeenCalledOnce() + await fiber.dispose() + }) + + it('preserves the setup failure after its one rollback attempt fails', async () => { + const fixture = fakeSandbox() + fixture.run.mockRejectedValueOnce(new Error('chmod failed')) + fixture.kill.mockRejectedValueOnce(new Error('cleanup failed')) + sdk.create.mockResolvedValue(fixture.sandbox) + const ctx = new Context() + const fiber = await ctx.plugin(E2BSandboxService, { apiKey: 'test-key' }) + await expect(ctx.e2b.getSandbox()).rejects.toThrow('chmod failed') + expect(fixture.kill).toHaveBeenCalledOnce() + + await fiber.dispose() + expect(fixture.kill).toHaveBeenCalledOnce() + }) + + it.each([ + ['symbolic link', { type: FileType.DIR, symlinkTarget: '/tmp/redirected' }], + ['regular file', { type: FileType.FILE }], + ])('rejects a reserved runtime root that is a %s', async (_label, info) => { + const fixture = fakeSandbox() + fixture.getInfo.mockResolvedValueOnce(info) + sdk.create.mockResolvedValue(fixture.sandbox) + const ctx = new Context() + await ctx.plugin(E2BSandboxService, { apiKey: 'test-key' }) + + await expect(ctx.e2b.getSandbox()).rejects.toThrow('runtime root must be a real directory') + expect(fixture.run).not.toHaveBeenCalled() + expect(fixture.kill).toHaveBeenCalledOnce() + }) + + it.each([ + [{ apiKey: '' }, /configure apiKey/], + [{ apiKey: 'x', cwd: 'relative' }, /absolute Linux path/], + [{ apiKey: 'x', timeoutMs: 0 }, /positive finite/], + ] as const)('fails self-contained configuration before opening E2B: %j', async (config, message) => { + vi.stubEnv('E2B_API_KEY', '') + const ctx = new Context() + await expect(ctx.plugin(E2BSandboxService, config)).rejects.toThrow(message) + expect(sdk.create).not.toHaveBeenCalled() + }) + + it('requires a key when both config and the environment omit it', async () => { + const original = process.env.E2B_API_KEY + delete process.env.E2B_API_KEY + try { + const ctx = new Context() + await expect(ctx.plugin(E2BSandboxService, {})).rejects.toThrow(/configure apiKey/) + } finally { + if (original === undefined) delete process.env.E2B_API_KEY + else process.env.E2B_API_KEY = original + } + }) +}) + +describe('E2B helpers and invariant companion', () => { + it('quotes opaque shell arguments without interpolation', () => { + expect(quoteE2BShellArg("a'b $HOME")).toBe("'a'\"'\"'b $HOME'") + }) + + it('registers the package-owned empty invariant installer', async () => { + const ctx = new Context() + await ctx.plugin(InvariantService, { enabled: true }) + const fiber = await ctx.plugin(E2BInvariant).await() + await fiber.dispose() + }) +}) diff --git a/packages/e2b/e2b/tsconfig.json b/packages/e2b/e2b/tsconfig.json new file mode 100644 index 0000000000..5890a41d66 --- /dev/null +++ b/packages/e2b/e2b/tsconfig.json @@ -0,0 +1,25 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": ["src"], + "references": [ + { + "path": "../../../vendor/cosmokit" + }, + { + "path": "../../../vendor/cordis" + }, + { + "path": "../../../vendor/schemastery" + }, + { + "path": "../../util/brand" + }, + { + "path": "../../support/invariants" + } + ] +} diff --git a/packages/e2b/fs-e2b/README.i18n.yaml b/packages/e2b/fs-e2b/README.i18n.yaml new file mode 100644 index 0000000000..a8edcb34ab --- /dev/null +++ b/packages/e2b/fs-e2b/README.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write packages/e2b/fs-e2b/README.md +README.md: cd170bcbe2831b0856b51791a2045382d93c1148 +README.zh.md: f97790cf1d4ef90f042df8ed564723e186327f00 diff --git a/packages/e2b/fs-e2b/README.md b/packages/e2b/fs-e2b/README.md new file mode 100644 index 0000000000..cd170bcbe2 --- /dev/null +++ b/packages/e2b/fs-e2b/README.md @@ -0,0 +1,31 @@ +# @deepseek-ai/dsh-fs-e2b + +English | [中文](README.zh.md) + +E2B implementation of the [`@deepseek-ai/dsh-fs`](../../fs/fs/README.md) provider seam. It has no config: load [`@deepseek-ai/dsh-e2b`](../e2b/README.md) first, then this service in place of `dsh-fs-local`. The provider uses the owner's remote cwd and SDK handle, so file tools observe the same world as E2B-backed Bash processes. + +## Behavior + +- **Remote identity and metadata** — relative paths resolve as POSIX paths against the caller cwd or `ctx.e2b.cwd`; GNU `realpath -mz` supplies canonical target identity without requiring the final file to exist, and ASCII/base64 plus strict NUL framing preserves newline and multibyte paths across the decoded SDK transport. `stat`, no-follow `lstat`, and stable one-level directory listings project E2B metadata into the filesystem seam; listings reuse returned metadata and resolve symbolic-link entries sequentially. Versions are opaque hashes of E2B metadata plus a per-write extended attribute. +- **Execution-world paths** — canonical targets expose absolute POSIX process paths, percent-encoded `file:` URIs, and provider-owned containment checks, so generic subprocess consumers never parse E2B target ids or apply host path rules. +- **UTF-8 reads** — whole reads and streamed reads preserve cross-chunk decoding, reject invalid UTF-8, and use the seam's 8192-byte NUL sample for binary detection. The model-facing tool still owns size selection and line windowing. +- **Atomic mutations** — writes create a random sibling staging directory, change it to mode `0700` before uploading content, preserve an existing file's POSIX mode, and publish the staged file through E2B's same-filesystem atomic rename. The rename response supplies the committed version, so no fallible metadata request follows the commit point. E2B creates missing parent directories. Literal edits LF-normalize for matching, restore dominant CRLF storage, and serialize mutations per canonical target within the host process. Optional create/version guards keep the base seam's observed-state semantics. +- **Failures and cancellation** — E2B not-found, permission, abort, and other controller failures map to the existing `FsError` vocabulary. Cancellation is best-effort at earlier SDK request boundaries and checked immediately before rename. The signal is not forwarded into the rename RPC, so cancellation cannot interrupt the atomic commit; a successful rename is the commit point. + +The provider does not copy, mount, or reconcile the host workspace. Giving it a host path as `cwd` creates a remote directory with the same spelling only. + +## Model Experience + +Indirectly, through [`dsh-tool-fs`](../../fs/tool-fs/README.md), which renders remote UTF-8 content, directory results, mutation acknowledgements, and provider errors while E2B identity and transport remain internal. + +#### KV Cache effect + +No direct invalidation; the named consumer owns any request-prefix changes. + +## Known Limitations and Deferred Work + +- **No host synchronization** — an empty E2B cwd stays empty until a tool, command, or external process populates it; local files are neither uploaded nor reflected back. +- **Mutation coordination is host-process-local** — another harness connection or remote command can race the adapter; version guards detect only metadata changes represented by E2B. +- **Reads reopen canonical targets by path** — a concurrent remote path replacement between resolution and stream opening is not fenced by a stable file handle; no observed product defect justifies a provider-specific bounded-read protocol in this POC. +- **Whole-file mutation costs remain** — overwrite diffs and literal edits read complete files into host memory, and every operation incurs E2B controller latency. +- **The POC targets E2B's default Linux image** — it relies on GNU `realpath`/`base64`/`chmod`, same-filesystem rename, streaming reads, and metadata extended attributes; custom templates are outside this POC. diff --git a/packages/e2b/fs-e2b/README.zh.md b/packages/e2b/fs-e2b/README.zh.md new file mode 100644 index 0000000000..f97790cf1d --- /dev/null +++ b/packages/e2b/fs-e2b/README.zh.md @@ -0,0 +1,31 @@ +# @deepseek-ai/dsh-fs-e2b + +[English](README.md) | 中文 + +[`@deepseek-ai/dsh-fs`](../../fs/fs/README.md) 提供方 seam 的 E2B 实现。它没有配置：先加载 [`@deepseek-ai/dsh-e2b`](../e2b/README.md)，再用本服务取代 `dsh-fs-local`。该提供方使用所有者的远程 cwd 和 SDK 句柄，因此文件工具观察到的环境与 E2B 后端 Bash 进程相同。 + +## 行为 + +- **远程身份与元数据**：相对路径以调用方 cwd 或 `ctx.e2b.cwd` 为基准，按照 POSIX 路径解析；GNU `realpath -mz` 提供规范化目标身份，且不要求最终文件存在；ASCII/base64 加严格 NUL 分帧会在已解码的 SDK 传输中保留含换行符和多字节字符的路径。`stat`、不跟随链接的 `lstat` 和稳定的单层目录列表会把 E2B 元数据投影到文件系统 seam；目录列表会复用已返回的元数据，并依次解析符号链接条目。版本是 E2B 元数据与每次写入设置的扩展属性所组成的不透明哈希。 +- **执行世界路径**：规范化目标公开绝对 POSIX 进程路径、百分号编码的 `file:` URI，以及由提供方负责的包含关系检查，因此通用进程管理消费方无需解析 E2B 目标 ID，也不会套用宿主路径规则。 +- **UTF-8 读取**：完整读取和流式读取会保留跨分片解码、拒绝无效 UTF-8，并使用 seam 的 8192 字节 NUL 样本检测二进制内容。面向模型的工具仍负责选择大小和行窗口。 +- **原子变更**：写入会创建随机的同级暂存目录，在上传内容前将其 mode 改为 `0700`，保留现有文件的 POSIX mode，并通过 E2B 的同一文件系统原子重命名发布暂存文件。重命名响应会提供已提交的版本，因此提交点之后不会再进行可能失败的元数据请求。E2B 会创建缺失的父目录。字面量编辑匹配时会规范化为 LF，存储时恢复占主导的 CRLF，并在宿主进程内按规范化目标串行执行变更。可选的创建／版本防护会保留基础 seam 的已观察状态语义。 +- **失败与取消**：E2B 的未找到、权限、中止及其他控制器故障会映射到现有 `FsError` 词汇。取消在更早的 SDK 请求边界上采用尽力而为语义，并在 rename 前立即检查。信号不会传入 rename RPC，因此取消无法中断原子提交；成功 rename 是提交点。 + +该提供方不会复制、挂载或协调宿主工作区。把宿主路径用作 `cwd`，只会在远程创建一个拼写相同的目录。 + +## 模型体验 + +通过 [`dsh-tool-fs`](../../fs/tool-fs/README.md) 间接影响模型；该工具会渲染远程 UTF-8 内容、目录结果、变更确认和提供方错误，而 E2B 身份及传输保持内部实现。 + +#### KV Cache 影响 + +不会直接失效；请求前缀变更由具名消费方负责。 + +## 已知限制与延后工作 + +- **不提供宿主同步**：空的 E2B cwd 会一直为空，直到工具、命令或外部进程填充它；本地文件既不会上传，也不会同步回本地。 +- **变更协调仅限宿主进程内**：另一个 harness 连接或远程命令可能与适配器发生竞态；版本防护只能检测 E2B 元数据所体现的变更。 +- **读取会按路径重新打开规范化目标**：在解析与打开流之间若并发替换远程路径，该操作没有稳定文件句柄提供围栏；在该 POC 中，没有已观察到的产品缺陷能够证明提供方专用的有界读取协议值得引入。 +- **仍需承担完整文件变更成本**：覆盖差异和字面量编辑会把完整文件读入宿主内存，每项操作也都会产生 E2B 控制器延迟。 +- **该 POC 面向 E2B 默认 Linux 镜像**：它依赖 GNU `realpath`／`base64`／`chmod`、同一文件系统内的 rename、流式读取和元数据扩展属性；自定义模板不在该 POC 范围内。 diff --git a/packages/e2b/fs-e2b/package.json b/packages/e2b/fs-e2b/package.json new file mode 100644 index 0000000000..02925ee7c4 --- /dev/null +++ b/packages/e2b/fs-e2b/package.json @@ -0,0 +1,39 @@ +{ + "name": "@deepseek-ai/dsh-fs-e2b", + "description": "E2B filesystem implementation for DeepSeek Harness", + "version": "0.0.1", + "private": true, + "type": "module", + "main": "lib/index.js", + "types": "lib/types/index.d.ts", + "exports": { + ".": { + "types": "./lib/types/index.d.ts", + "default": "./lib/index.js" + }, + "./invariant": { + "types": "./lib/types/invariant.d.ts", + "default": "./lib/invariant.js" + }, + "./src/*": "./src/*", + "./package.json": "./package.json" + }, + "files": [ + "lib/index.js", + "lib/invariant.js", + "lib/types/**/*.d.ts" + ], + "license": "BSD-3-Clause", + "peerDependencies": { + "@deepseek-ai/dsh-e2b": "^0.0.1", + "@deepseek-ai/dsh-fs": "^0.0.1", + "@deepseek-ai/dsh-invariants": "^0.0.1", + "cordis": "^4.0.0-rc.7" + }, + "devDependencies": { + "@deepseek-ai/dsh-e2b": "workspace:^", + "@deepseek-ai/dsh-fs": "workspace:^", + "@deepseek-ai/dsh-invariants": "workspace:^", + "cordis": "^4.0.0-rc.7" + } +} diff --git a/packages/e2b/fs-e2b/src/index.ts b/packages/e2b/fs-e2b/src/index.ts new file mode 100644 index 0000000000..84c9ac7868 --- /dev/null +++ b/packages/e2b/fs-e2b/src/index.ts @@ -0,0 +1,499 @@ +/** + * E2B implementation of the filesystem provider seam. Paths, contents, and + * atomic staging files remain inside the shared remote sandbox. + * @module @deepseek-ai/dsh-fs-e2b + */ + +import { createHash, randomUUID } from 'node:crypto' +import { Buffer } from 'node:buffer' +import { posix } from 'node:path' +import { FileSystem, FsError, FsTargetKey, FsVersion } from '@deepseek-ai/dsh-fs' +import type { + FsDirEntry, + FsEditOutcome, + FsEditRequest, + FsInfo, + FsPathInfo, + FsTarget, + FsWriteIntent, + FsWriteOutcome, +} from '@deepseek-ai/dsh-fs' +import { + CommandExitError, + e2bControlEnvs, + FileNotFoundError, + FileType, + quoteE2BShellArg, +} from '@deepseek-ai/dsh-e2b' +import type { EntryInfo, Sandbox } from '@deepseek-ai/dsh-e2b' + +const VERSION_METADATA_KEY = 'dsh-version' +const BINARY_SAMPLE_BYTES = 8192 +const BASE64 = /^(?:[A-Za-z0-9+/]{4})*(?:[A-Za-z0-9+/]{2}==|[A-Za-z0-9+/]{3}=)?$/ + +function assertNotAborted(signal: AbortSignal | undefined, operation: string): void { + if (signal?.aborted === true) throw new FsError(`${operation} aborted`, 'FS_ABORTED') +} + +function normalizeLineEndings(value: string): string { + return value.replaceAll('\r\n', '\n') +} + +function detectsCrlf(value: string): boolean { + const sample = value.slice(0, 4096) + const crlf = sample.split('\r\n').length - 1 + const lf = sample.split('\n').length - 1 - crlf + return crlf > lf +} + +function restoreLineEndings(value: string, crlf: boolean): string { + return crlf ? normalizeLineEndings(value).replaceAll('\n', '\r\n') : value +} + +function decodeText(bytes: Uint8Array, displayPath: string, binarySampleBytes: number): string { + if (bytes.subarray(0, binarySampleBytes).includes(0)) { + throw new FsError(`cannot read "${displayPath}": binary file`, 'FS_NOT_TEXT') + } + try { + return new TextDecoder('utf-8', { fatal: true }).decode(bytes) + } catch (error: unknown) { + throw new FsError(`cannot read "${displayPath}": invalid UTF-8 text`, 'FS_NOT_TEXT', { cause: error }) + } +} + +function decodeCanonicalPath(encoded: string): string { + if (encoded.length === 0 || !BASE64.test(encoded)) { + throw new Error('fs-e2b: canonical path transport returned invalid base64') + } + const framed = Buffer.from(encoded, 'base64') + if (framed.toString('base64') !== encoded + || framed.length < 2 + || framed.at(-1) !== 0 + || framed.subarray(0, -1).includes(0)) { + throw new Error('fs-e2b: canonical path transport returned invalid NUL framing') + } + let path: string + try { + path = new TextDecoder('utf-8', { fatal: true }).decode(framed.subarray(0, -1)) + } catch (error: unknown) { + throw new Error('fs-e2b: canonical path is not valid UTF-8', { cause: error }) + } + if (!posix.isAbsolute(path)) throw new Error('fs-e2b: canonical path is not absolute') + return path +} + +function signalOpts(signal: AbortSignal | undefined): { signal?: AbortSignal } { + return signal === undefined ? {} : { signal } +} + +function commandOpts(signal: AbortSignal | undefined): { envs: Record; signal?: AbortSignal } { + return { envs: e2bControlEnvs(), ...signalOpts(signal) } +} + +function entryType(entry: EntryInfo): FsInfo['type'] { + switch (entry.type) { + case FileType.FILE: + return 'file' + case FileType.DIR: + return 'directory' + default: + return 'other' + } +} + +function entryVersion(entry: EntryInfo): ReturnType { + const facts = JSON.stringify([ + entry.metadata?.[VERSION_METADATA_KEY], + entry.path, + entry.type, + entry.size, + entry.mode, + entry.modifiedTime?.toISOString(), + entry.symlinkTarget, + ]) + return FsVersion(`e2b:${createHash('sha256').update(facts).digest('hex')}`) +} + +function mapError(error: unknown, operation: string, displayPath: string, signal?: AbortSignal): FsError { + if (error instanceof FsError) return error + if (signal?.aborted === true || (error instanceof DOMException && error.name === 'AbortError')) { + return new FsError(`${operation} aborted`, 'FS_ABORTED', { cause: error }) + } + if (error instanceof FileNotFoundError) { + return new FsError(`cannot ${operation} "${displayPath}": not found`, 'FS_NOT_FOUND', { cause: error }) + } + if (/permission denied|operation not permitted/i.test(String(error))) { + return new FsError(`cannot ${operation} "${displayPath}": permission denied`, 'FS_PERMISSION_DENIED', { cause: error }) + } + return new FsError(`cannot ${operation} "${displayPath}": ${String(error)}`, 'FS_IO_ERROR', { cause: error }) +} + +function literalEdit(content: string, request: FsEditRequest, displayPath: string): string { + const oldString = normalizeLineEndings(request.oldString) + const newString = normalizeLineEndings(request.newString) + if (oldString.length === 0) { + throw new FsError(`cannot edit "${displayPath}": old_string must be non-empty`, 'FS_EDIT_NOT_FOUND') + } + let matches = 0 + let offset = 0 + while (true) { + const found = content.indexOf(oldString, offset) + if (found < 0) break + matches += 1 + offset = found + oldString.length + } + if (matches === 0) throw new FsError(`cannot edit "${displayPath}": old_string was not found`, 'FS_EDIT_NOT_FOUND') + if (!request.replaceAll && matches !== 1) { + throw new FsError(`cannot edit "${displayPath}": old_string matched ${matches} times`, 'FS_AMBIGUOUS_EDIT') + } + return request.replaceAll ? content.split(oldString).join(newString) : content.replace(oldString, newString) +} + +/** Remote filesystem backend sharing the sandbox owned by `ctx.e2b`. */ +export class E2BFileSystem extends FileSystem { + static inject = ['e2b'] + + private readonly locks = new Map>() + + override async resolve(path: string, opts?: { cwd?: string; signal?: AbortSignal }): Promise { + assertNotAborted(opts?.signal, 'resolve') + if (path.trim().length === 0) throw new FsError('file_path must be a non-empty string', 'FS_NOT_FOUND') + const displayPath = posix.resolve(opts?.cwd ?? this.ctx.e2b.cwd, path) + try { + const sandbox = await this.ctx.e2b.getSandbox() + const targetKey = await this.canonicalPath(sandbox, displayPath, opts?.signal) + assertNotAborted(opts?.signal, 'resolve') + return { targetKey: FsTargetKey(targetKey), displayPath } + } catch (error: unknown) { + throw mapError(error, 'resolve', displayPath, opts?.signal) + } + } + + override processPath(target: FsTarget): string { + return String(target.targetKey) + } + + override fileUrl(target: FsTarget): string { + const path = this.processPath(target) + if (!posix.isAbsolute(path)) throw new Error(`fs-e2b: expected an absolute process path: ${JSON.stringify(path)}`) + return `file://${path.split('/').map(segment => encodeURIComponent(segment)).join('/')}` + } + + override contains(parent: FsTarget, child: FsTarget): boolean { + const relative = posix.relative(this.processPath(parent), this.processPath(child)) + return relative === '' || (relative !== '..' && !relative.startsWith('../') && !posix.isAbsolute(relative)) + } + + override async stat(target: FsTarget, signal?: AbortSignal): Promise { + assertNotAborted(signal, 'stat') + const entry = await this.probe(String(target.targetKey), target.displayPath, signal) + if (entry === undefined) return undefined + return { + version: entryVersion(entry), + type: entryType(entry), + ...(entry.type === FileType.FILE ? { size: entry.size } : {}), + } + } + + override async lstat(path: string, opts?: { cwd?: string }, signal?: AbortSignal): Promise { + assertNotAborted(signal, 'lstat') + if (path.trim().length === 0) throw new FsError('file_path must be a non-empty string', 'FS_NOT_FOUND') + const displayPath = posix.resolve(opts?.cwd ?? this.ctx.e2b.cwd, path) + const entry = await this.probe(displayPath, displayPath, signal) + if (entry === undefined) return undefined + const type = entry.symlinkTarget !== undefined + ? 'symlink' as const + : entry.type === FileType.FILE + ? 'file' as const + : entry.type === FileType.DIR + ? 'directory' as const + : 'other' as const + return { + version: entryVersion(entry), + type, + ...(entry.type === FileType.FILE ? { size: entry.size } : {}), + } + } + + override async readText(target: FsTarget, signal?: AbortSignal): Promise { + const sandbox = await this.ctx.e2b.getSandbox() + await this.requireRegular(target, signal) + try { + const bytes = await sandbox.files.read(String(target.targetKey), { format: 'bytes', ...signalOpts(signal) }) + assertNotAborted(signal, 'read') + return decodeText(bytes, target.displayPath, BINARY_SAMPLE_BYTES) + } catch (error: unknown) { + throw mapError(error, 'read', target.displayPath, signal) + } + } + + override async streamText(target: FsTarget, signal?: AbortSignal): Promise> { + const sandbox = await this.ctx.e2b.getSandbox() + await this.requireRegular(target, signal) + let stream: ReadableStream + try { + // The pinned SDK's stream overload lies for empty files: content-length 0 + // returns '' instead of a ReadableStream. + const read = await sandbox.files.read(String(target.targetKey), { format: 'stream', ...signalOpts(signal) }) as + ReadableStream | string + stream = typeof read === 'string' + ? new ReadableStream({ start(controller) { controller.close() } }) + : read + } catch (error: unknown) { + throw mapError(error, 'read', target.displayPath, signal) + } + const displayPath = target.displayPath + return { + async *[Symbol.asyncIterator](): AsyncGenerator { + const reader = stream.getReader() + const decoder = new TextDecoder('utf-8', { fatal: true }) + let sampledBytes = 0 + let completed = false + try { + while (true) { + assertNotAborted(signal, 'read') + const next = await reader.read() + if (next.done) break + if (sampledBytes < BINARY_SAMPLE_BYTES) { + const sample = next.value.subarray(0, BINARY_SAMPLE_BYTES - sampledBytes) + if (sample.includes(0)) throw new FsError(`cannot read "${displayPath}": binary file`, 'FS_NOT_TEXT') + sampledBytes += sample.length + } + let text: string + try { + text = decoder.decode(next.value, { stream: true }) + } catch (error: unknown) { + throw new FsError(`cannot read "${displayPath}": invalid UTF-8 text`, 'FS_NOT_TEXT', { cause: error }) + } + if (text.length > 0) yield text + } + try { + decoder.decode() + } catch (error: unknown) { + throw new FsError(`cannot read "${displayPath}": invalid UTF-8 text`, 'FS_NOT_TEXT', { cause: error }) + } + completed = true + } catch (error: unknown) { + throw mapError(error, 'read', displayPath, signal) + } finally { + if (!completed) { + try { + await reader.cancel() + } catch (_streamCancellationFailure) { + // The primary read outcome owns the result; cancellation is best-effort after early stop. + } + } + reader.releaseLock() + } + }, + } + } + + override async listDir(target: FsTarget, signal?: AbortSignal): Promise { + const info = await this.stat(target, signal) + if (info === undefined) throw new FsError(`cannot list "${target.displayPath}": not found`, 'FS_NOT_FOUND') + if (info.type !== 'directory') throw new FsError(`cannot list "${target.displayPath}": not a directory`, 'FS_NOT_DIRECTORY') + try { + const sandbox = await this.ctx.e2b.getSandbox() + const listed = await sandbox.files.list(String(target.targetKey), { depth: 1, ...signalOpts(signal) }) + const entries: FsDirEntry[] = [] + for (const entry of listed) { + const displayPath = posix.join(target.displayPath, entry.name) + const canonical = entry.symlinkTarget === undefined + ? entry.path + : await this.canonicalPath(sandbox, entry.path, signal) + const resolved = entry.symlinkTarget === undefined + ? entry + : await this.probe(canonical, displayPath, signal) + entries.push({ + name: entry.name, + type: resolved === undefined ? 'other' : entryType(resolved), + target: { targetKey: FsTargetKey(canonical), displayPath }, + ...(resolved !== undefined ? { version: entryVersion(resolved) } : {}), + ...(resolved?.type === FileType.FILE ? { size: resolved.size } : {}), + }) + } + return entries.sort((left, right) => left.name.localeCompare(right.name)) + } catch (error: unknown) { + throw mapError(error, 'list', target.displayPath, signal) + } + } + + override async writeText( + target: FsTarget, + content: string, + expected?: FsWriteIntent, + signal?: AbortSignal, + ): Promise { + return this.withLock(String(target.targetKey), async () => { + const existing = await this.probe(String(target.targetKey), target.displayPath, signal) + if (existing !== undefined && entryType(existing) !== 'file') { + throw new FsError(`cannot write "${target.displayPath}": not a regular file`, 'FS_NOT_REGULAR_FILE') + } + this.checkWriteIntent(existing, expected, target) + const before = existing === undefined ? null : await this.readForDiff(target, signal) + const version = await this.writeAtomic(target, content, existing, signal) + return { + operation: existing === undefined ? 'create' : 'update', + version, + before, + after: normalizeLineEndings(content), + } + }) + } + + override async editText( + target: FsTarget, + edit: FsEditRequest, + expected?: { version: ReturnType }, + signal?: AbortSignal, + ): Promise { + return this.withLock(String(target.targetKey), async () => { + const existing = await this.probe(String(target.targetKey), target.displayPath, signal) + if (existing === undefined) { + throw new FsError(`cannot edit "${target.displayPath}": file changed since it was read`, 'FS_STALE_VERSION') + } + if (entryType(existing) !== 'file') { + throw new FsError(`cannot edit "${target.displayPath}": not a regular file`, 'FS_NOT_REGULAR_FILE') + } + if (expected !== undefined && entryVersion(existing) !== expected.version) { + throw new FsError(`cannot edit "${target.displayPath}": file changed since it was read`, 'FS_STALE_VERSION') + } + const raw = await this.readForEdit(target, signal) + const before = normalizeLineEndings(raw) + const after = literalEdit(before, edit, target.displayPath) + const storage = restoreLineEndings(after, detectsCrlf(raw)) + const version = await this.writeAtomic(target, storage, existing, signal) + return { version, before, after } + }) + } + + private async withLock(targetKey: string, operation: () => Promise): Promise { + const prior = this.locks.get(targetKey) ?? Promise.resolve() + const run = prior.then(operation, operation) + const tail = run.then(() => undefined, () => undefined) + this.locks.set(targetKey, tail) + try { + return await run + } finally { + if (this.locks.get(targetKey) === tail) this.locks.delete(targetKey) + } + } + + private async canonicalPath(sandbox: Sandbox, path: string, signal?: AbortSignal): Promise { + try { + const result = await sandbox.commands.run( + `set -o pipefail; realpath -mz -- ${quoteE2BShellArg(path)} | base64 -w0`, + commandOpts(signal), + ) + return decodeCanonicalPath(result.stdout) + } catch (error: unknown) { + if (error instanceof CommandExitError) throw new Error(error.stderr || error.message, { cause: error }) + throw error + } + } + + private async probe(path: string, displayPath: string, signal?: AbortSignal): Promise { + assertNotAborted(signal, 'stat') + try { + const sandbox = await this.ctx.e2b.getSandbox() + const entry = await sandbox.files.getInfo(path, signalOpts(signal)) + assertNotAborted(signal, 'stat') + return entry + } catch (error: unknown) { + if (error instanceof FileNotFoundError) return undefined + throw mapError(error, 'stat', displayPath, signal) + } + } + + private async requireRegular(target: FsTarget, signal?: AbortSignal): Promise { + const info = await this.stat(target, signal) + if (info === undefined) throw new FsError(`cannot read "${target.displayPath}": not found`, 'FS_NOT_FOUND') + if (info.type !== 'file') throw new FsError(`cannot read "${target.displayPath}": not a regular file`, 'FS_NOT_REGULAR_FILE') + } + + private checkWriteIntent(existing: EntryInfo | undefined, expected: FsWriteIntent | undefined, target: FsTarget): void { + if (expected?.kind === 'createIfAbsent' && existing !== undefined) { + throw new FsError(`cannot overwrite existing "${target.displayPath}" without reading it first`, 'FS_NOT_OBSERVED') + } + if (expected?.kind === 'replaceIfVersion') { + if (existing === undefined || entryVersion(existing) !== expected.version) { + throw new FsError(`cannot write "${target.displayPath}": file changed since it was read`, 'FS_STALE_VERSION') + } + } + } + + private async readForDiff(target: FsTarget, signal?: AbortSignal): Promise { + try { + const sandbox = await this.ctx.e2b.getSandbox() + const bytes = await sandbox.files.read(String(target.targetKey), { format: 'bytes', ...signalOpts(signal) }) + assertNotAborted(signal, 'read') + return normalizeLineEndings(decodeText(bytes, target.displayPath, bytes.length)) + } catch (error: unknown) { + if (error instanceof FsError && error.code === 'FS_NOT_TEXT') return null + throw mapError(error, 'read', target.displayPath, signal) + } + } + + private async readForEdit(target: FsTarget, signal?: AbortSignal): Promise { + try { + const sandbox = await this.ctx.e2b.getSandbox() + const bytes = await sandbox.files.read(String(target.targetKey), { format: 'bytes', ...signalOpts(signal) }) + assertNotAborted(signal, 'edit') + return decodeText(bytes, target.displayPath, bytes.length) + } catch (error: unknown) { + throw mapError(error, 'edit', target.displayPath, signal) + } + } + + private async writeAtomic( + target: FsTarget, + content: string, + existing: EntryInfo | undefined, + signal?: AbortSignal, + ): Promise> { + assertNotAborted(signal, 'write') + const sandbox = await this.ctx.e2b.getSandbox() + const targetPath = String(target.targetKey) + const versionId = randomUUID() + const stagingDirectory = posix.join(posix.dirname(targetPath), `.dsh-${randomUUID()}.tmp`) + const temporary = posix.join(stagingDirectory, 'content') + let stagingDirectoryCreated = false + try { + const created = await sandbox.files.makeDir(stagingDirectory, signalOpts(signal)) + if (!created) throw new Error('private staging directory already exists') + stagingDirectoryCreated = true + await sandbox.commands.run(`chmod 700 -- ${quoteE2BShellArg(stagingDirectory)}`, commandOpts(signal)) + assertNotAborted(signal, 'write') + await sandbox.files.write(temporary, content, { + metadata: { [VERSION_METADATA_KEY]: versionId }, + ...signalOpts(signal), + }) + assertNotAborted(signal, 'write') + const mode = existing === undefined ? 0o600 : existing.mode & 0o777 + await sandbox.commands.run( + `chmod ${mode.toString(8)} -- ${quoteE2BShellArg(temporary)}`, + commandOpts(signal), + ) + assertNotAborted(signal, 'write') + const committed = await sandbox.files.rename(temporary, targetPath) + try { + await sandbox.files.remove(stagingDirectory) + } catch (_committedStagingCleanupFailure) { + // The target is already committed; an empty private directory cannot turn that write into a failure. + } + return entryVersion(committed) + } catch (error: unknown) { + if (stagingDirectoryCreated) { + try { + await sandbox.files.remove(stagingDirectory) + } catch (_stagingDirectoryAlreadyAbsentOrCleanupFailed) { + // Only the private staging directory is swallowed; the original failure owns the operation. + } + } + throw mapError(error, 'write', target.displayPath, signal) + } + } +} + +export default E2BFileSystem diff --git a/packages/e2b/fs-e2b/src/invariant.ts b/packages/e2b/fs-e2b/src/invariant.ts new file mode 100644 index 0000000000..9f14bb37a6 --- /dev/null +++ b/packages/e2b/fs-e2b/src/invariant.ts @@ -0,0 +1,30 @@ +/** + * Package-owned invariant companion for `@deepseek-ai/dsh-fs-e2b`. + * @module @deepseek-ai/dsh-fs-e2b/invariant + */ + +/* jscpd:ignore-start */ +import type { Context } from 'cordis' +import type { InvariantInstaller } from '@deepseek-ai/dsh-invariants' + +const PACKAGE_NAME = '@deepseek-ai/dsh-fs-e2b' + +/** Cordis companion plugin name. */ +export const name = 'fs-e2b-invariant' +/** Service required before reserving package ownership. */ +export const inject = ['invariants'] + +/** + * No runtime invariant: each operation returns the E2B controller's committed + * result directly, with no independent event or cache to cross-check. + */ +const install: InvariantInstaller = () => {} + +/** + * Register this package's invariant companion. + * @param ctx - Cordis context carrying the invariant service. + * @returns the installed registration's disposer after setup succeeds. + */ +export const apply = (ctx: Context): Promise<() => void> => + Promise.resolve(ctx.invariants.register(PACKAGE_NAME, install)) +/* jscpd:ignore-end */ diff --git a/packages/e2b/fs-e2b/tests/filesystem.spec.ts b/packages/e2b/fs-e2b/tests/filesystem.spec.ts new file mode 100644 index 0000000000..9cad00b6eb --- /dev/null +++ b/packages/e2b/fs-e2b/tests/filesystem.spec.ts @@ -0,0 +1,682 @@ +import { Buffer } from 'node:buffer' +import { dirname, posix } from 'node:path' +import { Context } from 'cordis' +import { + CommandExitError, + FileNotFoundError, + FileType, + type EntryInfo, + type Sandbox, +} from '@deepseek-ai/dsh-e2b' +import type E2BSandboxService from '@deepseek-ai/dsh-e2b' +import { FsTargetKey, FsVersion } from '@deepseek-ai/dsh-fs' +import E2BFileSystem from '@deepseek-ai/dsh-fs-e2b' +import * as E2BFsInvariant from '../src/invariant.ts' +import InvariantService from '@deepseek-ai/dsh-invariants' +import { describe, expect, it, vi } from 'vitest' + +interface RemoteNode { + type: FileType + data: Uint8Array + mode: number + modified: number + metadata?: Record + symlinkTarget?: string +} + +function bytes(value: string | readonly number[]): Uint8Array { + return typeof value === 'string' ? new TextEncoder().encode(value) : Uint8Array.from(value) +} + +function commandError(exitCode: number, stderr = ''): CommandExitError { + return new CommandExitError({ exitCode, stdout: '', stderr, error: stderr }) +} + +class FakeRemote { + readonly nodes = new Map() + readonly writes: Array<{ path: string; data: string; metadata?: Record }> = [] + readonly writeParentModes: number[] = [] + readonly renames: Array<{ from: string; to: string }> = [] + readonly removals: string[] = [] + readonly commands: string[] = [] + streamChunks: Uint8Array[] | undefined + streamKeepOpen = false + readonly streamCancel = vi.fn() + nextCommandError: unknown + nextMakeDirResult: boolean | undefined + nextInfoError: unknown + nextListError: unknown + nextReadError: unknown + nextRenameError: unknown + nextRemoveError: unknown + canonicalOutput: string | undefined + abortAfterRename: AbortController | undefined + disappearOnInfo = new Set() + private clock = 1 + + constructor() { + this.dir('/') + this.dir('/workspace') + } + + dir(path: string): void { + this.nodes.set(path, { type: FileType.DIR, data: bytes(''), mode: 0o755, modified: this.clock++ }) + } + + file(path: string, data: string | readonly number[], mode = 0o644): void { + this.nodes.set(path, { type: FileType.FILE, data: bytes(data), mode, modified: this.clock++ }) + } + + other(path: string): void { + this.nodes.set(path, { type: 'other' as FileType, data: bytes(''), mode: 0o600, modified: this.clock++ }) + } + + symlink(path: string, target: string): void { + this.nodes.set(path, { + type: FileType.FILE, + data: bytes(''), + mode: 0o777, + modified: this.clock++, + symlinkTarget: target, + }) + } + + mutate(path: string, data: string): void { + const node = this.required(path) + node.data = bytes(data) + node.modified = this.clock++ + } + + private required(path: string): RemoteNode { + const node = this.nodes.get(path) + if (node === undefined) throw new FileNotFoundError(`missing: ${path}`) + return node + } + + private followed(path: string): { path: string; node: RemoteNode; link?: RemoteNode } { + const node = this.required(path) + if (node.symlinkTarget === undefined) return { path, node } + return { path: node.symlinkTarget, node: this.required(node.symlinkTarget), link: node } + } + + private info(path: string): EntryInfo { + if (this.disappearOnInfo.delete(path)) throw new FileNotFoundError(`missing: ${path}`) + return this.rawInfo(path) + } + + private rawInfo(path: string): EntryInfo { + const followed = this.followed(path) + const node = followed.node + return { + name: posix.basename(path), + path, + type: node.type, + size: node.data.byteLength, + mode: node.mode, + permissions: 'rw-------', + owner: 'user', + group: 'user', + modifiedTime: new Date(node.modified), + ...(node.metadata !== undefined ? { metadata: { ...node.metadata } } : {}), + ...(followed.link?.symlinkTarget !== undefined ? { symlinkTarget: followed.link.symlinkTarget } : {}), + } + } + + private checkAbort(options: { signal?: AbortSignal } | undefined): void { + if (options?.signal?.aborted === true) throw new DOMException('aborted', 'AbortError') + } + + readonly sandbox = { + sandboxId: 'fake', + files: { + makeDir: async (path: string, options?: { signal?: AbortSignal }): Promise => { + this.checkAbort(options) + if (this.nextMakeDirResult !== undefined) { + const result = this.nextMakeDirResult + this.nextMakeDirResult = undefined + return result + } + if (this.nodes.has(path)) return false + this.dir(path) + return true + }, + getInfo: async (path: string, options?: { signal?: AbortSignal }): Promise => { + this.checkAbort(options) + if (this.nextInfoError !== undefined) { + const error = this.nextInfoError + this.nextInfoError = undefined + throw error + } + return this.info(path) + }, + read: async (path: string, options: { format: 'bytes' | 'stream'; signal?: AbortSignal }): Promise | string> => { + this.checkAbort(options) + if (this.nextReadError !== undefined) { + const error = this.nextReadError + this.nextReadError = undefined + throw error + } + const data = this.followed(path).node.data + if (options.format === 'bytes') return data.slice() + // Pinned-SDK fidelity: a content-length-0 response returns '' even in stream format. + if (data.length === 0 && this.streamChunks === undefined) return '' + const chunks = this.streamChunks ?? [data.slice()] + return new ReadableStream({ + start: (controller) => { + for (const chunk of chunks) controller.enqueue(chunk) + if (!this.streamKeepOpen) controller.close() + }, + cancel: () => { this.streamCancel() }, + }) + }, + list: async (path: string, options?: { depth?: number; signal?: AbortSignal }): Promise => { + this.checkAbort(options) + if (this.nextListError !== undefined) { + const error = this.nextListError + this.nextListError = undefined + throw error + } + this.required(path) + return [...this.nodes.keys()] + .filter(candidate => candidate !== path && dirname(candidate) === path) + .map(candidate => this.rawInfo(candidate)) + }, + write: async (path: string, data: string, options?: { metadata?: Record; signal?: AbortSignal }): Promise => { + this.checkAbort(options) + const parent = dirname(path) + if (!this.nodes.has(parent)) this.dir(parent) + this.writeParentModes.push(this.required(parent).mode) + this.nodes.set(path, { + type: FileType.FILE, + data: bytes(data), + mode: 0o644, + modified: this.clock++, + ...(options?.metadata !== undefined ? { metadata: { ...options.metadata } } : {}), + }) + this.writes.push({ path, data, ...(options?.metadata !== undefined ? { metadata: options.metadata } : {}) }) + return {} + }, + rename: async (from: string, to: string, options?: { signal?: AbortSignal }): Promise => { + this.checkAbort(options) + if (this.nextRenameError !== undefined) { + const error = this.nextRenameError + this.nextRenameError = undefined + throw error + } + const node = this.required(from) + this.nodes.delete(from) + this.nodes.set(to, node) + this.renames.push({ from, to }) + this.abortAfterRename?.abort('after commit') + this.checkAbort(options) + return this.info(to) + }, + remove: async (path: string): Promise => { + this.removals.push(path) + if (this.nextRemoveError !== undefined) { + const error = this.nextRemoveError + this.nextRemoveError = undefined + throw error + } + for (const candidate of this.nodes.keys()) { + if (candidate === path || candidate.startsWith(`${path}/`)) this.nodes.delete(candidate) + } + }, + }, + commands: { + run: async ( + command: string, + options?: { envs?: Record; signal?: AbortSignal }, + ): Promise<{ exitCode: number; stdout: string; stderr: string }> => { + this.checkAbort(options) + const home = options?.envs?.HOME + expect(home).toMatch(/^\/\.dsh-e2b-control-/) + expect(options?.envs).toEqual({ HOME: home }) + this.commands.push(command) + if (this.nextCommandError !== undefined) { + const error = this.nextCommandError + this.nextCommandError = undefined + throw error + } + const realpathPrefix = 'set -o pipefail; realpath -mz -- ' + const realpathSuffix = ' | base64 -w0' + if (command.startsWith(realpathPrefix) && command.endsWith(realpathSuffix)) { + const quoted = command.slice(realpathPrefix.length, -realpathSuffix.length) + const input = quoted.slice(1, -1).replaceAll(String.raw`'"'"'`, '\'') + const node = this.nodes.get(input) + const canonical = `${node?.symlinkTarget ?? input}\0` + return { + exitCode: 0, + stdout: this.canonicalOutput ?? Buffer.from(canonical).toString('base64'), + stderr: '', + } + } + const chmod = /^chmod ([0-7]+) -- '([^']+)'$/.exec(command) + if (chmod !== null) this.required(chmod[2]!).mode = Number.parseInt(chmod[1]!, 8) + const move = /^mv -f -- '([^']+)' '([^']+)'$/.exec(command) + if (move !== null) { + if (this.nextRenameError !== undefined) { + const error = this.nextRenameError + this.nextRenameError = undefined + throw error + } + const node = this.required(move[1]!) + this.nodes.delete(move[1]!) + this.nodes.set(move[2]!, node) + this.renames.push({ from: move[1]!, to: move[2]! }) + this.abortAfterRename?.abort('after commit') + } + return { exitCode: 0, stdout: '', stderr: '' } + }, + }, + } as unknown as Sandbox +} + +async function setup(remote = new FakeRemote()): Promise<{ ctx: Context; fs: E2BFileSystem; remote: FakeRemote }> { + const ctx = new Context() + const runtime = { + cwd: '/workspace', + runtimeRoot: '/workspace/.dsh-e2b', + getSandbox: async () => remote.sandbox, + } as unknown as E2BSandboxService + ctx.provide('e2b', runtime) + await ctx.plugin(E2BFileSystem) + return { ctx, fs: ctx.fs as E2BFileSystem, remote } +} + +async function expectCode(promise: Promise, code: string): Promise { + await expect(promise).rejects.toMatchObject({ code }) +} + +describe('E2BFileSystem identity, metadata, and reads', () => { + it('resolves remote paths, reports symlinks, and lists direct children in stable order', async () => { + const remote = new FakeRemote() + remote.file('/workspace/z.txt', 'z') + remote.file('/workspace/a.txt', 'a') + remote.dir('/workspace/dir') + remote.other('/workspace/special') + remote.file('/workspace/dir/nested.txt', 'nested') + remote.symlink('/workspace/link.txt', '/workspace/a.txt') + const { fs } = await setup(remote) + + const link = await fs.resolve('link.txt') + expect(link).toEqual({ targetKey: '/workspace/a.txt', displayPath: '/workspace/link.txt' }) + await expect(fs.lstat('link.txt')).resolves.toMatchObject({ type: 'symlink', size: 1 }) + await expect(fs.lstat('a.txt')).resolves.toMatchObject({ type: 'file', size: 1 }) + await expect(fs.lstat('dir')).resolves.toEqual(expect.objectContaining({ type: 'directory' })) + await expect(fs.lstat('special')).resolves.toEqual(expect.objectContaining({ type: 'other' })) + await expect(fs.lstat('missing')).resolves.toBeUndefined() + await expect(fs.stat(link)).resolves.toMatchObject({ type: 'file', size: 1 }) + const directory = await fs.resolve('.') + const listed = await fs.listDir(directory) + expect(listed.map(entry => entry.name)).toEqual(['a.txt', 'dir', 'link.txt', 'special', 'z.txt']) + expect(listed.find(entry => entry.name === 'dir')).toMatchObject({ type: 'directory' }) + expect(listed.find(entry => entry.name === 'link.txt')).toMatchObject({ + type: 'file', + target: { targetKey: '/workspace/a.txt', displayPath: '/workspace/link.txt' }, + }) + expect(listed.some(entry => entry.name === 'nested.txt')).toBe(false) + }) + + it('projects canonical process paths, file URLs, and containment', async () => { + const remote = new FakeRemote() + remote.dir('/workspace/nested') + remote.file('/workspace/nested/multibyte # file.ts', 'text') + remote.file('/outside.ts', 'outside') + const { fs } = await setup(remote) + const workspace = await fs.resolve('/workspace') + const nested = await fs.resolve('/workspace/nested/multibyte # file.ts') + const outside = await fs.resolve('/outside.ts') + + expect(fs.processPath(nested)).toBe('/workspace/nested/multibyte # file.ts') + expect(fs.fileUrl(nested)).toBe('file:///workspace/nested/multibyte%20%23%20file.ts') + expect(fs.contains(workspace, workspace)).toBe(true) + expect(fs.contains(workspace, nested)).toBe(true) + expect(fs.contains(nested, workspace)).toBe(false) + expect(fs.contains(workspace, outside)).toBe(false) + expect(() => fs.fileUrl({ targetKey: FsTargetKey('relative'), displayPath: 'relative' })) + .toThrow('expected an absolute process path') + }) + + it('preserves newline and multibyte canonical paths through strict ASCII framing', async () => { + const remote = new FakeRemote() + const path = '/workspace/你好\nfile.ts' + remote.file(path, 'text') + const { fs } = await setup(remote) + + await expect(fs.resolve(path)).resolves.toEqual({ targetKey: path, displayPath: path }) + }) + + it.each([ + ['invalid base64', '!!!!'], + ['missing terminator', Buffer.from('/workspace/file').toString('base64')], + ['multiple records', Buffer.from('/workspace/file\0/other\0').toString('base64')], + ['invalid UTF-8', Buffer.from([47, 0xff, 0]).toString('base64')], + ['relative path', Buffer.from('workspace/file\0').toString('base64')], + ])('rejects %s from canonical path transport', async (_label, output) => { + const remote = new FakeRemote() + remote.canonicalOutput = output + const { fs } = await setup(remote) + await expectCode(fs.resolve('file'), 'FS_IO_ERROR') + }) + + it('reads whole and streamed UTF-8 across chunk boundaries', async () => { + const remote = new FakeRemote() + remote.file('/workspace/text.txt', 'A€B') + remote.streamChunks = [bytes([65, 0xe2]), bytes([0x82, 0xac, 66])] + const { fs } = await setup(remote) + const target = await fs.resolve('text.txt') + await expect(fs.readText(target)).resolves.toBe('A€B') + let streamed = '' + for await (const chunk of await fs.streamText(target)) streamed += chunk + expect(streamed).toBe('A€B') + + remote.streamChunks = [bytes([0xe2]), bytes([0x82, 0xac])] + let initiallyBuffered = '' + for await (const chunk of await fs.streamText(target)) initiallyBuffered += chunk + expect(initiallyBuffered).toBe('€') + }) + + it('streams an empty file even though the pinned SDK returns a non-stream value', async () => { + const remote = new FakeRemote() + remote.file('/workspace/empty.txt', '') + const { fs } = await setup(remote) + let streamed = '' + for await (const chunk of await fs.streamText(await fs.resolve('empty.txt'))) streamed += chunk + expect(streamed).toBe('') + }) + + it('cancels a remote stream when its consumer stops early', async () => { + const remote = new FakeRemote() + remote.file('/workspace/text.txt', 'ab') + remote.streamChunks = [bytes('a'), bytes('b')] + remote.streamKeepOpen = true + const { fs } = await setup(remote) + const stream = await fs.streamText(await fs.resolve('text.txt')) + + for await (const chunk of stream) { + expect(chunk).toBe('a') + break + } + + expect(remote.streamCancel).toHaveBeenCalledOnce() + }) + + it('matches local binary sampling while edits still reject any NUL byte', async () => { + const remote = new FakeRemote() + remote.file('/workspace/late-nul.txt', `${'a'.repeat(8192)}\0tail`) + const { fs } = await setup(remote) + const target = await fs.resolve('late-nul.txt') + await expect(fs.readText(target)).resolves.toContain('\0tail') + remote.streamChunks = [bytes('a'.repeat(8192)), bytes([0, 116])] + let streamed = '' + for await (const chunk of await fs.streamText(target)) streamed += chunk + expect(streamed).toBe(`${'a'.repeat(8192)}\0t`) + await expectCode(fs.editText(target, { oldString: 'tail', newString: 'end', replaceAll: false }), 'FS_NOT_TEXT') + }) + + it('maps binary, invalid UTF-8, missing, and non-regular read failures', async () => { + const remote = new FakeRemote() + remote.file('/workspace/binary', [0, 1]) + remote.file('/workspace/invalid', [0xff]) + remote.dir('/workspace/directory') + const { fs } = await setup(remote) + await expectCode(fs.readText(await fs.resolve('binary')), 'FS_NOT_TEXT') + await expectCode(fs.readText(await fs.resolve('invalid')), 'FS_NOT_TEXT') + await expectCode(fs.readText(await fs.resolve('missing')), 'FS_NOT_FOUND') + await expectCode(fs.readText(await fs.resolve('directory')), 'FS_NOT_REGULAR_FILE') + + remote.streamChunks = [bytes([0xff])] + const invalid = await fs.streamText(await fs.resolve('invalid')) + await expect((async () => { for await (const _chunk of invalid) void _chunk })()).rejects.toMatchObject({ code: 'FS_NOT_TEXT' }) + remote.streamChunks = [bytes([0])] + const binary = await fs.streamText(await fs.resolve('binary')) + await expect((async () => { for await (const _chunk of binary) void _chunk })()).rejects.toMatchObject({ code: 'FS_NOT_TEXT' }) + + remote.streamChunks = [bytes([0xe2])] + const incomplete = await fs.streamText(await fs.resolve('invalid')) + await expect((async () => { for await (const _chunk of incomplete) void _chunk })()).rejects.toMatchObject({ code: 'FS_NOT_TEXT' }) + + const raced = await fs.resolve('invalid') + remote.nextReadError = new FileNotFoundError('gone after stat') + await expectCode(fs.streamText(raced), 'FS_NOT_FOUND') + }) + + it('honors aborts before and during remote reads', async () => { + const remote = new FakeRemote() + remote.file('/workspace/a', 'a') + const { fs } = await setup(remote) + await expectCode(fs.resolve('a', { signal: AbortSignal.abort() }), 'FS_ABORTED') + await expectCode(fs.lstat('a', undefined, AbortSignal.abort()), 'FS_ABORTED') + await expectCode(fs.stat(await fs.resolve('a'), AbortSignal.abort()), 'FS_ABORTED') + remote.nextReadError = new DOMException('aborted', 'AbortError') + await expectCode(fs.readText(await fs.resolve('a')), 'FS_ABORTED') + }) + + it('rejects empty paths and directory-listing type errors', async () => { + const remote = new FakeRemote() + remote.file('/workspace/file', 'x') + const { fs } = await setup(remote) + await expectCode(fs.resolve(' '), 'FS_NOT_FOUND') + await expectCode(fs.lstat(''), 'FS_NOT_FOUND') + await expectCode(fs.listDir(await fs.resolve('missing')), 'FS_NOT_FOUND') + await expectCode(fs.listDir(await fs.resolve('/workspace/file')), 'FS_NOT_DIRECTORY') + remote.nextListError = new Error('listing transport failed') + await expectCode(fs.listDir(await fs.resolve('/workspace')), 'FS_IO_ERROR') + }) +}) + +describe('E2BFileSystem atomic writes and edits', () => { + it('creates owner-only files and returns metadata after the committed move', async () => { + const { fs, remote } = await setup() + const target = await fs.resolve('new.txt') + const outcome = await fs.writeText(target, 'one\r\ntwo\rthree', { kind: 'createIfAbsent' }) + expect(outcome).toMatchObject({ operation: 'create', before: null, after: 'one\ntwo\rthree' }) + expect(remote.nodes.get('/workspace/new.txt')?.mode).toBe(0o600) + expect(remote.nodes.get('/workspace/new.txt')?.metadata?.['dsh-version']).toBeDefined() + expect(remote.writeParentModes).toEqual([0o700]) + const stagingDirectory = posix.dirname(remote.writes[0]!.path) + expect(posix.dirname(stagingDirectory)).toBe('/workspace') + expect(remote.removals).toContain(stagingDirectory) + await expect(fs.stat(target)).resolves.toMatchObject({ version: outcome.version, size: 14 }) + }) + + it('preserves replacement mode, normalizes only CRLF for diffs, and changes version on external writes', async () => { + const remote = new FakeRemote() + remote.file('/workspace/file.txt', 'old\r\nline\rlone', 0o640) + const { fs } = await setup(remote) + const target = await fs.resolve('file.txt') + const before = (await fs.stat(target))!.version + const outcome = await fs.writeText(target, 'new', { kind: 'replaceIfVersion', version: before }) + expect(outcome).toMatchObject({ operation: 'update', before: 'old\nline\rlone', after: 'new' }) + expect(remote.nodes.get('/workspace/file.txt')?.mode).toBe(0o640) + const committed = outcome.version + remote.mutate('/workspace/file.txt', 'external') + expect((await fs.stat(target))!.version).not.toBe(committed) + }) + + it('returns null as the overwrite diff basis for binary or invalid prior content', async () => { + const remote = new FakeRemote() + remote.file('/workspace/file.txt', [0xff]) + const { fs } = await setup(remote) + const target = await fs.resolve('file.txt') + await expect(fs.writeText(target, 'valid')).resolves.toMatchObject({ before: null, after: 'valid' }) + }) + + it('fails an overwrite when reading its text diff basis fails for another reason', async () => { + const remote = new FakeRemote() + remote.file('/workspace/file.txt', 'prior') + const { fs } = await setup(remote) + const target = await fs.resolve('file.txt') + remote.nextReadError = new Error('read transport failed') + await expectCode(fs.writeText(target, 'replacement'), 'FS_IO_ERROR') + expect(new TextDecoder().decode(remote.nodes.get('/workspace/file.txt')?.data)).toBe('prior') + }) + + it('enforces create and version intents before publication', async () => { + const remote = new FakeRemote() + remote.file('/workspace/file.txt', 'v1') + const { fs } = await setup(remote) + const target = await fs.resolve('file.txt') + const version = (await fs.stat(target))!.version + await expectCode(fs.writeText(target, 'blind', { kind: 'createIfAbsent' }), 'FS_NOT_OBSERVED') + remote.mutate('/workspace/file.txt', 'v2') + await expectCode(fs.writeText(target, 'stale', { kind: 'replaceIfVersion', version }), 'FS_STALE_VERSION') + await expectCode(fs.writeText(await fs.resolve('missing'), 'stale', { kind: 'replaceIfVersion', version }), 'FS_STALE_VERSION') + remote.dir('/workspace/dir') + await expectCode(fs.writeText(await fs.resolve('dir'), 'x'), 'FS_NOT_REGULAR_FILE') + }) + + it('does not turn an abort observed after a successful move into a failed write', async () => { + const remote = new FakeRemote() + const controller = new AbortController() + remote.abortAfterRename = controller + const { fs } = await setup(remote) + await expect(fs.writeText(await fs.resolve('committed'), 'yes', undefined, controller.signal)) + .resolves.toMatchObject({ operation: 'create' }) + expect(controller.signal.aborted).toBe(true) + }) + + it('does not turn post-commit staging cleanup failure into a failed write', async () => { + const remote = new FakeRemote() + remote.nextRemoveError = new Error('empty staging cleanup failed') + const { fs } = await setup(remote) + await expect(fs.writeText(await fs.resolve('committed'), 'yes')) + .resolves.toMatchObject({ operation: 'create' }) + expect(new TextDecoder().decode(remote.nodes.get('/workspace/committed')?.data)).toBe('yes') + }) + + it('returns committed rename metadata without a fallible post-commit lookup', async () => { + const remote = new FakeRemote() + const getInfo = vi.spyOn(remote.sandbox.files, 'getInfo') + const { fs } = await setup(remote) + + await expect(fs.writeText(await fs.resolve('committed'), 'yes')) + .resolves.toMatchObject({ operation: 'create' }) + expect(getInfo).toHaveBeenCalledTimes(1) + expect(remote.renames).toHaveLength(1) + }) + + it('cleans staging files and maps command, permission, and abort failures', async () => { + const remote = new FakeRemote() + const { fs } = await setup(remote) + const commandTarget = await fs.resolve('command') + remote.nextCommandError = commandError(1, 'chmod failed') + await expectCode(fs.writeText(commandTarget, 'x'), 'FS_IO_ERROR') + expect(remote.removals).toHaveLength(1) + + remote.nextRenameError = new Error('permission denied') + await expectCode(fs.writeText(await fs.resolve('permission'), 'x'), 'FS_PERMISSION_DENIED') + remote.nextRemoveError = new Error('cleanup also failed') + remote.nextRenameError = new DOMException('aborted', 'AbortError') + await expectCode(fs.writeText(await fs.resolve('abort'), 'x'), 'FS_ABORTED') + + const removalsBeforeCollision = remote.removals.length + remote.nextMakeDirResult = false + await expectCode(fs.writeText(await fs.resolve('collision'), 'x'), 'FS_IO_ERROR') + expect(remote.removals).toHaveLength(removalsBeforeCollision) + }) + + it('applies literal edits atomically and restores the detected CRLF style', async () => { + const remote = new FakeRemote() + remote.file('/workspace/file.txt', 'one\r\ntwo\r\nthree\n') + const { fs } = await setup(remote) + const target = await fs.resolve('file.txt') + const version = (await fs.stat(target))!.version + const outcome = await fs.editText( + target, + { oldString: 'two\r\n', newString: 'TWO\r\n', replaceAll: false }, + { version }, + ) + expect(outcome).toMatchObject({ before: 'one\ntwo\nthree\n', after: 'one\nTWO\nthree\n' }) + expect(new TextDecoder().decode(remote.nodes.get('/workspace/file.txt')?.data)).toBe('one\r\nTWO\r\nthree\r\n') + }) + + it('reports stale and literal-match failures with stable codes', async () => { + const remote = new FakeRemote() + remote.file('/workspace/file.txt', 'a a') + remote.dir('/workspace/dir') + const { fs } = await setup(remote) + const target = await fs.resolve('file.txt') + await expectCode(fs.editText(target, { oldString: '', newString: 'x', replaceAll: false }), 'FS_EDIT_NOT_FOUND') + await expectCode(fs.editText(target, { oldString: 'z', newString: 'x', replaceAll: false }), 'FS_EDIT_NOT_FOUND') + await expectCode(fs.editText(target, { oldString: 'a', newString: 'x', replaceAll: false }), 'FS_AMBIGUOUS_EDIT') + await expect(fs.editText(target, { oldString: 'a', newString: 'x', replaceAll: true })) + .resolves.toMatchObject({ after: 'x x' }) + await expectCode(fs.editText(target, { oldString: 'x', newString: 'y', replaceAll: false }, { version: FsVersion('stale') }), 'FS_STALE_VERSION') + await expectCode(fs.editText(await fs.resolve('missing'), { oldString: 'x', newString: 'y', replaceAll: false }), 'FS_STALE_VERSION') + await expectCode(fs.editText(await fs.resolve('dir'), { oldString: 'x', newString: 'y', replaceAll: false }), 'FS_NOT_REGULAR_FILE') + }) + + it('serializes guarded mutations so only one stale version can win', async () => { + const remote = new FakeRemote() + remote.file('/workspace/file.txt', 'base') + const { fs } = await setup(remote) + const target = await fs.resolve('file.txt') + const version = (await fs.stat(target))!.version + const results = await Promise.allSettled([ + fs.writeText(target, 'one', { kind: 'replaceIfVersion', version }), + fs.editText(target, { oldString: 'base', newString: 'two', replaceAll: false }, { version }), + ]) + expect(results.filter(result => result.status === 'fulfilled')).toHaveLength(1) + expect(results.filter(result => result.status === 'rejected')).toHaveLength(1) + }) +}) + +describe('E2B filesystem adapter integration edges', () => { + it('maps canonicalization, permission, and generic provider failures', async () => { + const remote = new FakeRemote() + const { fs } = await setup(remote) + remote.nextCommandError = commandError(1, 'not a directory') + await expectCode(fs.resolve('bad'), 'FS_IO_ERROR') + remote.nextCommandError = commandError(1) + await expectCode(fs.resolve('bad-again'), 'FS_IO_ERROR') + remote.nextCommandError = new Error('canonical transport failed') + await expectCode(fs.resolve('bad-transport'), 'FS_IO_ERROR') + remote.file('/workspace/a', 'a') + const target = await fs.resolve('a') + remote.nextInfoError = new Error('metadata transport failed') + await expectCode(fs.stat(target), 'FS_IO_ERROR') + remote.nextReadError = new Error('operation not permitted') + await expectCode(fs.readText(target), 'FS_PERMISSION_DENIED') + remote.nextReadError = 'transport vanished' + await expectCode(fs.readText(target), 'FS_IO_ERROR') + }) + + it('uses listing metadata directly and canonicalizes only symbolic links', async () => { + const remote = new FakeRemote() + remote.file('/workspace/a', 'a') + remote.file('/workspace/target', 'target') + remote.file('/workspace/gone', 'gone') + remote.symlink('/workspace/link', '/workspace/target') + remote.symlink('/workspace/vanished-link', '/workspace/gone') + remote.disappearOnInfo.add('/workspace/gone') + const { fs } = await setup(remote) + const directory = await fs.resolve('/workspace') + const commandsBefore = remote.commands.length + const getInfo = vi.spyOn(remote.sandbox.files, 'getInfo') + + const listed = await fs.listDir(directory) + + expect(listed.find(entry => entry.name === 'a')).toMatchObject({ + type: 'file', target: { targetKey: '/workspace/a' }, size: 1, + }) + expect(listed.find(entry => entry.name === 'link')).toMatchObject({ + type: 'file', target: { targetKey: '/workspace/target' }, size: 6, + }) + expect(listed.find(entry => entry.name === 'vanished-link')).toEqual({ + name: 'vanished-link', + type: 'other', + target: { targetKey: '/workspace/gone', displayPath: '/workspace/vanished-link' }, + }) + expect(remote.commands.slice(commandsBefore)).toHaveLength(2) + expect(getInfo).toHaveBeenCalledTimes(3) + }) + + it('registers the package-owned empty invariant installer', async () => { + const ctx = new Context() + await ctx.plugin(InvariantService, { enabled: true }) + const fiber = await ctx.plugin(E2BFsInvariant).await() + await fiber.dispose() + }) +}) diff --git a/packages/e2b/fs-e2b/tsconfig.json b/packages/e2b/fs-e2b/tsconfig.json new file mode 100644 index 0000000000..bcb54a88d9 --- /dev/null +++ b/packages/e2b/fs-e2b/tsconfig.json @@ -0,0 +1,25 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": ["src"], + "references": [ + { + "path": "../../../vendor/cosmokit" + }, + { + "path": "../../../vendor/cordis" + }, + { + "path": "../e2b" + }, + { + "path": "../../fs/fs" + }, + { + "path": "../../support/invariants" + } + ] +} diff --git a/packages/e2b/subprocess-e2b/README.i18n.yaml b/packages/e2b/subprocess-e2b/README.i18n.yaml new file mode 100644 index 0000000000..d0ce148746 --- /dev/null +++ b/packages/e2b/subprocess-e2b/README.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write packages/e2b/subprocess-e2b/README.md +README.md: 926d4f22f8e96daa103c236121d35e461290221a +README.zh.md: d1634b6d1a701f7f4815135b4ec6bb86ad1a8c87 diff --git a/packages/e2b/subprocess-e2b/README.md b/packages/e2b/subprocess-e2b/README.md new file mode 100644 index 0000000000..926d4f22f8 --- /dev/null +++ b/packages/e2b/subprocess-e2b/README.md @@ -0,0 +1,43 @@ +# @deepseek-ai/dsh-subprocess-e2b + +English | [中文](README.zh.md) + +E2B implementation of the [`@deepseek-ai/dsh-subprocess`](../../subprocess/subprocess/README.md) seam. Load [`@deepseek-ai/dsh-e2b`](../e2b/README.md) first, then this service in place of `dsh-subprocess-local`. Existing Bash, PTY, and LSP consumers then execute in the shared remote sandbox without E2B-specific capability packages. + +## Configuration + +| Key | Default | Meaning | +| --- | --- | --- | +| `pollMs` | `20` | Remote status/liveness poll cadence in milliseconds; each tick is one control-plane request, so a larger value trades exit-observation latency for fewer requests. | + +## Behavior + +- **Asynchronous remote start** — the synchronous seam returns a handle immediately while `Sandbox.commands.run(..., { background: true })` starts remotely. `pid` is `-1` until the wrapper publishes and the adapter validates its process-group id; stdin and ordinary observation wait for that publication. An owned startup signal aborts environment and private-state preparation before allocation; once allocation begins, cancellation waits for a provisional SDK handle it can clean. +- **Execution-world coordinates** — `cwd` and private `runtimeRoot` come from the shared owner; executable lookup verifies absolute paths or resolves a bare name against the sandbox PATH plus explicit overrides, and rejects relative paths containing separators like every subprocess provider. +- **Linux process groups** — a quoted wrapper starts each argv under `exec setsid --wait` and records its actual process-group id plus private status files beneath `ctx.e2b.runtimeRoot/processes`. The handle waits for that file instead of treating the SDK command PID as its published identity. Termination signals the negative recorded id with `SIGTERM`, waits the caller's `graceMs`, then escalates to `SIGKILL` and the SDK kill fallback; TERM delivery or probe failures also force that escalation. Process-table probes treat groups containing only zombie or dead entries as quiescent. Force cleanup succeeds only after a bounded probe finds the group empty; otherwise `waitForExit()` exposes a retryable failure, while proven quiescence makes later termination a no-op. Publication and monitoring failures apply the same cleanup transaction before rejecting. Service disposal rejects new starts, terminates and joins every retained process group, then awaits SDK settlement and private cleanup before the sandbox owner disposes. +- **Environment boundary** — one trusted control-shell probe resolves the sandbox user's login home from its passwd entry and transports the sandbox environment as base64 ASCII for one strict UTF-8 decode; the wrapper then removes ambient `DSH_*` and credential-shaped (`*KEY*`, `*SECRET*`, `*TOKEN*`) names and restores every valid `spec.env` entry as an explicit caller opt-in. Empty names, `=`, and NUL framing violations reject before launch. Subsequent E2B command and PTY login shells receive a fresh randomized root-level `HOME` plus empty overrides for every scrubbed ambient name before user profiles can run; the requested argv receives the serialized environment afterward without changing the sandbox user's umask. Host ambient variables never enter the sandbox implicitly. Private environment files are removed after consumption, and failed command or terminal setup removes its private state before rejecting. +- **Stdio projection** — the remote wrapper branches raw bytes into optional bounded spill files, frames each live chunk as newline-delimited base64 ASCII, and the host incrementally restores bytes across arbitrary SDK callback boundaries. Pipe mode writes those bytes to host Node streams; inherit mode writes them to the harness process streams; collect mode retains a bounded host tail with offset reads. The wrapper publishes the direct command status before waiting for inherited writers. For collect or inherit output, the adapter disconnects an incomplete SDK stream after `graceMs`, withholds its partial spill, and returns that status while retaining the remote group for `waitForExit()` and termination. Natural raw-pipe completion instead awaits lossless transport and preserves backpressure; explicit termination destroys the host pipes and releases blocked output before remote cleanup. Batch and streaming stdin use the SDK handle. +- **Terminal sessions** — `spawnTerminal()` uses E2B's byte PTY API, installs the exact argv and scrubbed environment through private mode-`0600` files, reports the foreground process group, sends real signals, and tears down every live group in the remote terminal session through one retryable awaited `terminate()`; termination rejects new handle operations, aborts and joins in-flight writes, inspections, and signals, and treats zombie-only groups as quiescent. A private random output boundary discards the E2B bootstrap shell's prompt and echoed runner command while preserving every requested-process byte, including its first prompt. Terminal output is pushed to the handle's stream without awaiting host backpressure: a flowing consumer (the PTY backend attaches one at construction) folds bytes into its own bounded state, while a paused consumer buffers in host memory. PTY allocation is awaited through handle publication before cancellation is observed, so owned rollback can clean the published handle. Setup and teardown own the private state transaction, abort pending setup during service disposal, and fence publication; sandbox disposal or timeout bounds a setup rollback that also fails. Prompt detection, scrollback, readiness, and owner policy remain in `dsh-pty-local`. +- **Sandbox disappearance** — `SandboxNotFoundError` during process or terminal liveness, termination, rollback, or disconnect proves the remote execution world cannot retain work, so cleanup treats it as quiescent; unrelated failures remain observable. + +The default E2B base image supplies the runtime and Bash/GNU utilities this adapter invokes: `node`, `bash`, `setsid`, `ps`, `awk`, `tr`, `env`, `base64`, `chmod`, `tee`, `head`, `rm`, `kill`, `id`, and `getent`. + +## Model Experience + +Indirectly, through consumer seams such as the Bash executor behind `dsh-tool-bash`, which render remote output, exit facts, background deltas, and spill paths. + +#### KV Cache effect + +No direct invalidation; the named consumers own any request-prefix changes. + +## Known Limitations and Deferred Work + +- **The SDK still retains complete command output in host memory** — E2B `CommandHandle.stdout` and `.stderr` accumulate the base64 transport even when this adapter exposes bounded raw-byte tails, so the subprocess seam's normal host-memory bound is not achieved and transport retention is larger than the source stream. +- **Synchronous-PID consumers are unsupported** — `pid` remains `-1` during remote startup; consumers that require a positive PID immediately, including the ACP child backend, cannot use this provider unchanged. +- **Private state lives for the sandbox lifetime** — process directories and valid spill files remain under `.dsh-e2b` until the owner deletes the sandbox; this POC supplies no in-sandbox sweep. +- **Control state shares the sandbox user's UID** — E2B runs every command as the same default user, so `0700`/`0600` modes cannot isolate `.dsh-e2b` control files from concurrently running sandbox processes. A background process could rewrite `pid`/`exit-code` or read a not-yet-consumed `environment` file. The adapter validates published values and refuses group ids whose negative form is unsafe to signal (`<= 1`), but real isolation needs an E2B per-command user or an out-of-band control channel. +- **Numeric process identities are not reuse-fenced** — E2B exposes numeric PID/PGID PTY input, signalling, and cleanup operations but no atomic identity-bound alternative. The adapter minimizes host round trips and live coverage exercises the reproducible stale-interrupt overlap; replacement is deferred until E2B adds an identity primitive or a failure demonstrates a narrower protocol. +- **The initial environment probe inherits sandbox defaults** — E2B merges command overrides with default environment entries, so the probe cannot blank unknown credential-shaped names before enumerating them. A same-UID untrusted process already in the sandbox could inspect that short-lived control shell; this POC therefore does not support secrets in sandbox-default environment variables and requires an E2B replacement-environment primitive to close the gap. +- **E2B exposes no signal fact** — an adapter-requested `SIGTERM` or `SIGKILL` is reported only when no wrapper-published direct exit code wins; every unrequested SDK exit remains an exit code, including values shaped like `128 + signal`. +- **Exact terminal stdin-wait inspection is unavailable** — E2B exposes the foreground process group but not the syscall evidence needed to prove it is waiting on fd 0, so the generic PTY backend falls back to controlled prompt markers and bounded silence. +- **Linux utility and E2B transport semantics are assumed** — there is no Windows, escaped-session recovery, or network-partition fidelity layer. diff --git a/packages/e2b/subprocess-e2b/README.zh.md b/packages/e2b/subprocess-e2b/README.zh.md new file mode 100644 index 0000000000..d1634b6d1a --- /dev/null +++ b/packages/e2b/subprocess-e2b/README.zh.md @@ -0,0 +1,43 @@ +# @deepseek-ai/dsh-subprocess-e2b + +[English](README.md) | 中文 + +[`@deepseek-ai/dsh-subprocess`](../../subprocess/subprocess/README.md) seam 的 E2B 实现。先加载 [`@deepseek-ai/dsh-e2b`](../e2b/README.md)，再用本服务取代 `dsh-subprocess-local`。现有的 Bash、PTY 和 LSP 消费方随后会在共享远程沙箱中执行，无需 E2B 专用的功能包（package）。 + +## 配置 + +| 键 | 默认值 | 含义 | +| --- | --- | --- | +| `pollMs` | `20` | 远程状态／存活轮询节奏（毫秒）；每个 tick 是一次控制面请求，调大该值以牺牲退出观察延迟换取更少的请求。 | + +## 行为 + +- **异步远程启动**：同步 seam 会立即返回一个句柄，同时由 `Sandbox.commands.run(..., { background: true })` 在远程启动进程。包装层发布进程组 ID 并由适配器完成验证之前，`pid` 为 `-1`；stdin 和常规观察会等待该发布。自有启动信号会在分配前中止环境和私有状态准备；分配开始后，取消会等待可清理的临时 SDK 句柄。 +- **执行世界坐标**：`cwd` 和私有 `runtimeRoot` 来自共享所有者；可执行文件查找会验证绝对路径，或根据沙箱 PATH 加显式覆盖来解析裸名称，并与所有 subprocess 提供方一致地拒绝含分隔符的相对路径。 +- **Linux 进程组**：带引号保护的包装层会在 `exec setsid --wait` 下启动每组 argv，并在 `ctx.e2b.runtimeRoot/processes` 下记录实际进程组 ID 和私有状态文件。句柄会等待该文件，而不会把 SDK 命令 PID 当作已发布的身份。终止操作以记录的负数 ID 发送 `SIGTERM`，等待调用方的 `graceMs`，再升级到 `SIGKILL` 和 SDK kill 回退；TERM 信号发送或探测失败也会强制触发该升级。进程表探测会把仅含僵尸或已死亡条目的进程组视为完全停稳。强制清理只有在有界探测发现进程组为空后才算成功；否则 `waitForExit()` 会公开可重试的失败，而已证明的完全停稳会让后续终止操作不再执行任何动作。发布失败与监控失败都会在拒绝前执行同一清理事务。服务 dispose（资源释放）会拒绝新的启动请求、终止并等待每个保留进程组退出，再等待 SDK 结算和私有清理完成，之后沙箱所有者才会释放。 +- **环境边界**：一次受信任的控制 shell 探测会从 passwd 条目解析沙箱用户的登录主目录，以 base64 ASCII 传输沙箱环境，再进行一次严格 UTF-8 解码；随后包装层移除环境中的 `DSH_*` 和形似凭据的名称（`*KEY*`、`*SECRET*`、`*TOKEN*`），并把每个有效的 `spec.env` 条目恢复为调用方显式选择。空名称、`=` 和违反 NUL 分帧规则的条目会在启动前被拒绝。在用户 profile 脚本运行前，此后的 E2B 命令 shell 与 PTY 登录 shell 会获得位于根目录下、全新随机生成的 `HOME`，并为每个被清理的环境变量名设置空值覆盖；之后，请求的 argv 会在不改变沙箱用户 umask 的前提下接收序列化环境。宿主环境变量绝不会隐式进入沙箱。私有环境文件在使用后会被删除；命令或终端设置失败时，会先删除其私有状态再拒绝。 +- **stdio 投影**：远程包装层先把原始字节分流到可选的有界 spill 文件，再把每个实时分片编码为换行分隔的 base64 ASCII 帧；宿主会跨任意 SDK 回调边界增量恢复字节。pipe 模式把这些字节写入宿主 Node 流；inherit 模式把字节写入 harness 进程流；collect 模式保留有界的宿主尾部，并支持基于偏移量读取。包装层会在等待继承管道的写入方之前发布直接命令状态。对于 collect 或 inherit 输出，超过 `graceMs` 后，适配器会断开未完成的 SDK 流，不公开其中不完整的 spill，并返回该状态，同时保留远程进程组供 `waitForExit()` 和终止操作使用。原始 pipe 自然完成时，会等待无损传输完成并保留背压；显式终止则会销毁宿主 pipe，并在远程清理前释放受阻的输出写入。批量 stdin 和流式 stdin 都使用 SDK 句柄。 +- **终端会话**：`spawnTerminal()` 使用 E2B 的字节 PTY API，以 mode 为 `0600` 的私有文件传入原样 argv 与清理后的环境，报告前台进程组，发送真实信号，并通过一项可重试且须等待的 `terminate()` 清理远程终端会话中仍存活的每个进程组；终止会拒绝新的句柄操作，中止并等待在途写入、检查和信号操作结算，并把仅含僵尸进程的进程组视为已经完全停稳。私有随机输出边界会丢弃 E2B 引导 shell 的提示符和回显的 runner 命令，同时保留请求进程的每个字节，包括其第一个提示符。终端输出推入句柄流时不等待宿主背压：流动的消费方（PTY 后端在构造时就挂上一个）把字节折叠进自身的有界状态，而暂停的消费方会在宿主内存中缓冲。PTY 分配会一直等待到句柄发布后才观察取消，以便由承担清理责任的回滚清理已发布句柄。setup 与 teardown 负责私有状态事务，在服务 dispose 期间中止待处理的 setup 并阻止发布；若 setup 回滚也失败，则由沙箱 dispose 或超时约束其存活时间。提示符检测、scrollback、就绪状态与所有者策略仍归 `dsh-pty-local` 所有。 +- **沙箱消失**：在进程或终端的存活探测、终止、回滚或断开连接期间出现 `SandboxNotFoundError`，证明远程执行环境无法保留工作，因此清理会将其视为完全停稳；其他故障仍可观察。 + +E2B 默认基础镜像提供该适配器调用的运行时和 Bash/GNU 工具：`node`、`bash`、`setsid`、`ps`、`awk`、`tr`、`env`、`base64`、`chmod`、`tee`、`head`、`rm`、`kill`、`id` 和 `getent`。 + +## 模型体验 + +通过消费方 seam 间接影响模型，例如 `dsh-tool-bash` 背后的 Bash 执行器；这些消费方会渲染远程输出、退出事实、后台增量和 spill 路径。 + +#### KV Cache 影响 + +不会直接失效；请求前缀变更由具名消费方负责。 + +## 已知限制与延后工作 + +- **SDK 仍会在宿主内存中保留完整命令输出**：即使本适配器公开的是有界原始字节尾部，E2B `CommandHandle.stdout` 和 `.stderr` 仍会累积 base64 传输内容，因此无法达到进程管理 seam 通常提供的宿主内存边界，而且传输保留量大于源数据流。 +- **不支持需要同步 PID 的消费方**：远程启动期间，`pid` 保持为 `-1`；包括 ACP 子进程后端在内，要求立即获得正 PID 的消费方无法原样使用本提供方。 +- **私有状态随沙箱生命周期存在**：进程目录和有效的 spill 文件会留在 `.dsh-e2b` 下，直到所有者删除沙箱；本 POC 不提供沙箱内清理。 +- **控制状态与沙箱用户同 UID**：E2B 以同一默认用户运行每条命令，因此 `0700`/`0600` 权限无法把 `.dsh-e2b` 控制文件与并发运行的沙箱进程隔离开。后台进程可以改写 `pid`/`exit-code`，或读取尚未被消费的 `environment` 文件。适配器会验证已发布的值，并拒绝取负后不安全的进程组 ID（`<= 1`），但真正的隔离需要 E2B 提供按命令用户或带外控制通道。 +- **数值进程身份没有复用围栏**：E2B 公开基于数值 PID/PGID 的 PTY 输入、信号发送和清理操作，却没有与身份原子绑定的替代方案。适配器会尽量减少宿主往返，真实环境测试会覆盖可复现的陈旧中断重叠；在 E2B 新增身份原语，或实际故障证明需要更窄的协议之前，替代方案会继续延后。 +- **初始环境探测会继承沙箱默认值**：E2B 会把命令覆盖与默认环境条目合并，因此探测无法在枚举未知且形似凭据的名称之前将它们置空。一个已在沙箱内运行的同 UID 不可信进程可以检查该短时存在的控制 shell；因此，该 POC 不支持把 secret 放入沙箱默认环境变量，需要 E2B 的替换环境原语才能弥合该缺口。 +- **E2B 不公开信号事实**：适配器请求的 `SIGTERM` 或 `SIGKILL` 只有在包装层发布的直接退出码没有胜出时才报告为信号；其他未请求的 SDK 退出始终保留为退出码，包括形似 `128 + signal` 的值。 +- **无法精确检查终端 stdin 等待状态**：E2B 会公开前台进程组，但不提供证明其正在等待 fd 0 所需的 syscall 证据，因此通用 PTY 后端会回退到受控提示符标记与有界静默机制。 +- **依赖 Linux 工具与 E2B 传输语义**：没有 Windows、逃逸会话恢复或网络分区的保真层。 diff --git a/packages/e2b/subprocess-e2b/package.json b/packages/e2b/subprocess-e2b/package.json new file mode 100644 index 0000000000..007b88f1c9 --- /dev/null +++ b/packages/e2b/subprocess-e2b/package.json @@ -0,0 +1,44 @@ +{ + "name": "@deepseek-ai/dsh-subprocess-e2b", + "description": "E2B subprocess implementation for DeepSeek Harness", + "version": "0.0.1", + "private": true, + "type": "module", + "main": "lib/index.js", + "types": "lib/types/index.d.ts", + "exports": { + ".": { + "types": "./lib/types/index.d.ts", + "default": "./lib/index.js" + }, + "./invariant": { + "types": "./lib/types/invariant.d.ts", + "default": "./lib/invariant.js" + }, + "./src/*": "./src/*", + "./package.json": "./package.json" + }, + "files": [ + "lib/index.js", + "lib/invariant.js", + "lib/types/**/*.d.ts" + ], + "license": "BSD-3-Clause", + "peerDependencies": { + "@deepseek-ai/dsh-e2b": "^0.0.1", + "@deepseek-ai/dsh-invariants": "^0.0.1", + "@deepseek-ai/dsh-subprocess": "^0.0.1", + "@deepseek-ai/dsh-timeout": "^0.0.1", + "cordis": "^4.0.0-rc.7" + }, + "dependencies": { + "schemastery": "^3.18.0" + }, + "devDependencies": { + "@deepseek-ai/dsh-e2b": "workspace:^", + "@deepseek-ai/dsh-invariants": "workspace:^", + "@deepseek-ai/dsh-subprocess": "workspace:^", + "@deepseek-ai/dsh-timeout": "workspace:^", + "cordis": "^4.0.0-rc.7" + } +} diff --git a/packages/e2b/subprocess-e2b/src/environment.ts b/packages/e2b/subprocess-e2b/src/environment.ts new file mode 100644 index 0000000000..7cdb1b7fbf --- /dev/null +++ b/packages/e2b/subprocess-e2b/src/environment.ts @@ -0,0 +1,104 @@ +/** Shared remote-environment scrubbing for E2B process and terminal launchers. */ + +import { Buffer } from 'node:buffer' +import { posix } from 'node:path' +import { e2bControlEnvs } from '@deepseek-ai/dsh-e2b' +import type { Sandbox } from '@deepseek-ai/dsh-e2b' +import { SENSITIVE_ENV_PATTERN } from '@deepseek-ai/dsh-subprocess' + +const BASE64 = /^(?:[A-Za-z0-9+/]{4})*(?:[A-Za-z0-9+/]{2}==|[A-Za-z0-9+/]{3}=)?$/ + +function remoteEnvironmentEntries(raw: string): Array { + const entries: Array = [] + for (const entry of raw.split('\0')) { + if (entry.length === 0) continue + const separator = entry.indexOf('=') + if (separator <= 0) continue + entries.push([entry.slice(0, separator), entry.slice(separator + 1)]) + } + return entries +} + +/** + * Read the remote environment through ASCII base64 so SDK callback chunking cannot corrupt UTF-8. + * @param sandbox - shared E2B execution world. + * @param signal - optional cancellation for the control-plane request. + * @returns the complete NUL-delimited UTF-8 environment. + */ +export async function readRemoteEnvironment(sandbox: Sandbox, signal?: AbortSignal): Promise { + // TODO(e2b-replace-environment): Remove this ambient probe when E2B can start + // a command with a replacement environment instead of merged overrides. + const result = await sandbox.commands.run( + 'set -o pipefail; dsh_e2b_passwd="$(getent passwd "$(id -u)")"; IFS=: read -r _ _ _ _ _ dsh_e2b_home _ <<<"$dsh_e2b_passwd"; test -n "$dsh_e2b_home" -a -d "$dsh_e2b_home"; printf \'%s\' "$dsh_e2b_home" | base64 -w 0; printf \'\\n\'; env -0 | base64 -w 0', + { envs: e2bControlEnvs(), ...(signal === undefined ? {} : { signal }) }, + ) + const lines = result.stdout.trim().split('\n') + if (lines.length !== 2 || !lines.every(line => BASE64.test(line))) { + throw new Error('subprocess-e2b: remote environment transport returned invalid base64') + } + const [encodedHome, encodedEnvironment] = lines as [string, string] + let home: string + let raw: string + try { + const decoder = new TextDecoder('utf-8', { fatal: true }) + home = decoder.decode(Buffer.from(encodedHome, 'base64')) + raw = decoder.decode(Buffer.from(encodedEnvironment, 'base64')) + } catch (error: unknown) { + throw new Error('subprocess-e2b: remote environment is not valid UTF-8', { cause: error }) + } + if (!posix.isAbsolute(home) || home.includes('\0')) { + throw new Error(`subprocess-e2b: remote login home is invalid: ${JSON.stringify(home)}`) + } + const environment = new Map(remoteEnvironmentEntries(raw)) + environment.set('HOME', home) + return [...environment].map(([name, value]) => `${name}=${value}\0`).join('') +} + +/** + * Parse an E2B NUL-delimited environment while removing harness-private and credential-shaped names. + * @param raw - The complete NUL-delimited remote environment. + * @returns Mutable retained entries for the caller to overlay and serialize. + */ +export function scrubRemoteEnvironment(raw: string): Map { + const environment = new Map() + for (const [name, value] of remoteEnvironmentEntries(raw)) { + if (name.startsWith('DSH_') || SENSITIVE_ENV_PATTERN.test(name)) continue + environment.set(name, value) + } + return environment +} + +/** + * Isolate E2B's fixed login-shell bootstrap from user profiles and ambient credentials. + * @param raw - The complete NUL-delimited remote environment. + * @returns Explicit E2B command or PTY overrides for bootstrap-shell startup. + */ +export function bootstrapEnvironment(raw: string): Record { + const environment: Record = { TERM: 'dumb' } + for (const [name] of remoteEnvironmentEntries(raw)) { + if (name.startsWith('DSH_') || SENSITIVE_ENV_PATTERN.test(name)) environment[name] = '' + } + return environment +} + +/** + * Overlay explicit entries and serialize one validated E2B environment. + * @param raw - The complete NUL-delimited remote environment. + * @param explicit - Deliberate caller overrides applied after ambient scrubbing; an `undefined` tombstone removes an ambient entry. + * @returns NUL-delimited `name=value` entries accepted by `env -i`. + */ +export function serializeRemoteEnvironment( + raw: string, + explicit: Readonly | undefined, +): string { + const environment = scrubRemoteEnvironment(raw) + for (const [name, value] of Object.entries(explicit ?? {})) { + if (name.length === 0 || name.includes('=') || name.includes('\0') || value?.includes('\0') === true) { + throw new Error('subprocess-e2b: environment entries require non-empty NUL-free names without = and NUL-free values') + } + // An explicit undefined is the seam's tombstone: remove the ambient entry. + if (value === undefined) environment.delete(name) + else environment.set(name, value) + } + return [...environment].map(([name, value]) => `${name}=${value}\0`).join('') +} diff --git a/packages/e2b/subprocess-e2b/src/index.ts b/packages/e2b/subprocess-e2b/src/index.ts new file mode 100644 index 0000000000..c6d2dbd4eb --- /dev/null +++ b/packages/e2b/subprocess-e2b/src/index.ts @@ -0,0 +1,208 @@ +/** + * E2B implementation of the subprocess seam. Each handle starts through the + * shared sandbox and retains command output/status paths in that remote world. + * @module @deepseek-ai/dsh-subprocess-e2b + */ + +import { randomUUID } from 'node:crypto' +import { posix } from 'node:path' +import { Context } from 'cordis' +import z from 'schemastery' +import { SubprocessService } from '@deepseek-ai/dsh-subprocess' +import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout' +import type { + SubprocessHandle, + SubprocessSpawnSpec, + SubprocessTerminalHandle, + SubprocessTerminalSpawnSpec, +} from '@deepseek-ai/dsh-subprocess' +import { e2bControlEnvs, quoteE2BShellArg } from '@deepseek-ai/dsh-e2b' +import { E2BSubprocessHandle } from './process.ts' +import { asError, signalOpts } from './remote.ts' +import { spawnE2BTerminal } from './terminal.ts' + +/** Configuration for the E2B subprocess adapter. */ +export interface Config { + /** Remote status/liveness poll cadence in milliseconds; each tick is one control-plane request. */ + pollMs?: number +} + +interface SchemaResolvedConfig extends Config { + pollMs: number +} + +interface TerminalSetup { + done: Promise + controller: AbortController +} + +/** + * Enforce the seam's documented grace bound (positive, finite, one Node timer), + * matching subprocess-local's spawn-time check; an unbounded grace would make + * the remote force-escalation deadline unreachable. + * @param graceMs - The spec's cleanup grace in milliseconds. + */ +function requireRepresentableGrace(graceMs: number): void { + if (!Number.isFinite(graceMs) || graceMs <= 0 || graceMs > MAX_TIMER_DELAY_MS) { + throw new Error(`subprocess graceMs must be a positive finite number no greater than ${MAX_TIMER_DELAY_MS}`) + } +} + +/** E2B command manager registered as `ctx.subprocess`. */ +export class E2BSubprocessService extends SubprocessService { + static inject = ['e2b'] + + static Config: z = z.object({ + pollMs: z.number().default(20), + }) + + private readonly live = new Set() + private readonly terminals = new Set() + private readonly terminalSetups = new Set() + private readonly pollMs: number + private disposing = false + + /** Create the E2B subprocess service and bind its disposal policy. */ + constructor(ctx: Context, config: Config) { + super(ctx) + // Schemastery fills pollMs before construction; the type does not encode that step. + const { pollMs } = config as SchemaResolvedConfig + if (!Number.isSafeInteger(pollMs) || pollMs <= 0) { + throw new Error('subprocess-e2b: pollMs must be a positive safe integer') + } + this.pollMs = pollMs + ctx.effect(() => async () => { + this.disposing = true + for (const setup of this.terminalSetups) { + setup.controller.abort(new Error('subprocess-e2b: service disposed during terminal setup')) + } + await Promise.all([...this.terminalSetups].map(setup => setup.done)) + const handles = [...this.live] + const terminals = [...this.terminals] + const pending: Promise[] = [] + for (const handle of handles) { + handle.terminate() + pending.push(handle.waitForExit().then(async () => { + await handle.done.catch(() => undefined) + this.live.delete(handle) + })) + } + for (const terminal of terminals) { + pending.push(terminal.terminate().then(() => { this.terminals.delete(terminal) })) + } + const outcomes = await Promise.allSettled(pending) + const failures = outcomes.flatMap(outcome => outcome.status === 'rejected' + ? [outcome.reason as unknown] + : []) + if (failures.length === 1) throw asError(failures[0]) + if (failures.length > 1) throw new AggregateError(failures, 'subprocess-e2b: teardown failed') + }, 'e2b subprocess teardown') + } + + /** @inheritdoc */ + async resolveExecutable( + command: string, + env?: Readonly>, + signal?: AbortSignal, + ): Promise { + if (command.length === 0) throw new Error('subprocess-e2b: executable name must be non-empty') + signal?.throwIfAborted() + const sandbox = await this.ctx.e2b.getSandbox() + if (posix.isAbsolute(command)) { + await sandbox.commands.run( + `test -f ${quoteE2BShellArg(command)} -a -x ${quoteE2BShellArg(command)}`, + { envs: e2bControlEnvs(), ...signalOpts(signal) }, + ) + signal?.throwIfAborted() + return command + } + if (command.includes('/')) { + throw new Error( + `subprocess-e2b: command ${JSON.stringify(command)} is a relative path; use an absolute path or a bare PATH name`, + ) + } + const path = env?.PATH + const prefix = path === undefined ? '' : `PATH=${quoteE2BShellArg(path)} ` + const result = await sandbox.commands.run( + `${prefix}command -v -- ${quoteE2BShellArg(command)}`, + { cwd: this.ctx.e2b.cwd, envs: e2bControlEnvs(), ...signalOpts(signal) }, + ) + signal?.throwIfAborted() + const executable = result.stdout.trim() + if (executable.includes('\n') || (!posix.isAbsolute(executable) && !executable.includes('/'))) { + throw new Error(`subprocess-e2b: executable ${JSON.stringify(command)} did not resolve to one absolute path`) + } + // A relative result comes from a relative PATH entry; the lookup ran with the shared cwd. + return posix.resolve(this.ctx.e2b.cwd, executable) + } + + /** @inheritdoc */ + spawn(spec: SubprocessSpawnSpec): SubprocessHandle { + if (this.disposing) throw new Error('subprocess-e2b: service is disposing') + const program = spec.argv[0] + if (program === undefined || program.length === 0) { + throw new Error('invalid argv: expected a non-empty program name at argv[0]') + } + requireRepresentableGrace(spec.graceMs) + if (spec.signal?.aborted === true) { + throw new Error(`aborted before spawn: ${String(spec.signal.reason)}`) + } + const stateDir = posix.join(this.ctx.e2b.runtimeRoot, 'processes', randomUUID()) + const handle = new E2BSubprocessHandle(this.ctx.e2b, spec, stateDir, this.pollMs) + this.live.add(handle) + const release = async (): Promise => { + await handle.waitForExit() + this.live.delete(handle) + } + void handle.done.then(release, release).catch((_automaticReleaseFailure: unknown) => { + // Retain the handle so service disposal can retry its cleanup transaction. + }) + return handle + } + + /** @inheritdoc */ + async spawnTerminal(spec: SubprocessTerminalSpawnSpec): Promise { + if (this.disposing) throw new Error('subprocess-e2b: service is disposing') + const program = spec.argv[0] + if (program === undefined || program.length === 0) { + throw new Error('subprocess-e2b: terminal argv must contain a program') + } + requireRepresentableGrace(spec.graceMs) + spec.signal?.throwIfAborted() + const stateDir = posix.join(this.ctx.e2b.runtimeRoot, 'terminals', randomUUID()) + const done = Promise.withResolvers() + const setup: TerminalSetup = { done: done.promise, controller: new AbortController() } + const setupSignal = spec.signal === undefined + ? setup.controller.signal + : AbortSignal.any([spec.signal, setup.controller.signal]) + this.terminalSetups.add(setup) + try { + const terminal = await spawnE2BTerminal( + this.ctx.e2b, + { ...spec, signal: setupSignal }, + stateDir, + this.pollMs, + ) + this.terminals.add(terminal) + // oxlint-disable-next-line typescript/no-unnecessary-condition -- Remote allocation yields to disposal. + if (this.disposing) { + await terminal.terminate() + this.terminals.delete(terminal) + throw new Error('subprocess-e2b: service disposed during terminal setup') + } + const release = async (): Promise => { + await terminal.terminate() + this.terminals.delete(terminal) + } + void terminal.done.then(release, release).catch((_automaticReleaseFailure: unknown) => { + // Retain the terminal so service disposal can retry its cleanup transaction. + }) + return terminal + } finally { + this.terminalSetups.delete(setup) + done.resolve() + } + } +} + +export default E2BSubprocessService diff --git a/packages/e2b/subprocess-e2b/src/invariant.ts b/packages/e2b/subprocess-e2b/src/invariant.ts new file mode 100644 index 0000000000..9f8b8fb739 --- /dev/null +++ b/packages/e2b/subprocess-e2b/src/invariant.ts @@ -0,0 +1,30 @@ +/** + * Package-owned invariant companion for `@deepseek-ai/dsh-subprocess-e2b`. + * @module @deepseek-ai/dsh-subprocess-e2b/invariant + */ + +/* jscpd:ignore-start */ +import type { Context } from 'cordis' +import type { InvariantInstaller } from '@deepseek-ai/dsh-invariants' + +const PACKAGE_NAME = '@deepseek-ai/dsh-subprocess-e2b' + +/** Cordis companion plugin name. */ +export const name = 'subprocess-e2b-invariant' +/** Service required before reserving package ownership. */ +export const inject = ['invariants'] + +/** + * No runtime invariant: live remote handles are private teardown ownership, + * and the E2B command event stream is the sole outcome authority. + */ +const install: InvariantInstaller = () => {} + +/** + * Register this package's invariant companion. + * @param ctx - Cordis context carrying the invariant service. + * @returns the installed registration's disposer after setup succeeds. + */ +export const apply = (ctx: Context): Promise<() => void> => + Promise.resolve(ctx.invariants.register(PACKAGE_NAME, install)) +/* jscpd:ignore-end */ diff --git a/packages/e2b/subprocess-e2b/src/output.ts b/packages/e2b/subprocess-e2b/src/output.ts new file mode 100644 index 0000000000..ba1dc1a716 --- /dev/null +++ b/packages/e2b/subprocess-e2b/src/output.ts @@ -0,0 +1,131 @@ +/** Bounded host-side projection of a complete output file retained in E2B. */ + +import { Buffer } from 'node:buffer' +import type { SubprocessOutputRead, SubprocessOutputReader } from '@deepseek-ai/dsh-subprocess' + +const BASE64_TEXT = /^[A-Za-z0-9+/]+={0,2}$/u + +/** Reserved non-base64 frame proving that one remote encoder reached clean EOF. */ +export const E2B_OUTPUT_COMPLETE_FRAME = '!dsh-e2b-output-complete!' + +/** Incrementally decode newline-delimited base64 frames emitted by one remote encoder. */ +export class E2BBase64Decoder { + private pending = '' + private complete = false + + /** + * Decode every complete newline-delimited frame in one arbitrarily split SDK callback. + * @param text - ASCII base64 frames from E2B's decoded callback. + * @returns the complete raw bytes made available by this callback. + */ + push(text: string): Buffer { + if (text.length === 0) return Buffer.alloc(0) + this.pending += text + const decoded: Buffer[] = [] + for (;;) { + const boundary = this.pending.indexOf('\n') + if (boundary < 0) break + const frame = this.pending.slice(0, boundary) + this.pending = this.pending.slice(boundary + 1) + if (frame === E2B_OUTPUT_COMPLETE_FRAME) { + if (this.complete) throw new Error('subprocess-e2b: duplicate output transport completion') + this.complete = true + continue + } + if (this.complete) throw new Error('subprocess-e2b: output transport continued after completion') + if (!BASE64_TEXT.test(frame)) { + throw new Error('subprocess-e2b: invalid base64 output transport') + } + const bytes = Buffer.from(frame, 'base64') + if (bytes.toString('base64') !== frame) { + throw new Error('subprocess-e2b: invalid base64 output transport') + } + decoded.push(bytes) + } + return Buffer.concat(decoded) + } + + /** + * Validate clean encoder completion, or discard an interrupted trailing frame after requested termination. + * @param requireComplete - Whether natural completion requires the reserved EOF frame. + */ + finish(requireComplete = true): void { + if (!requireComplete) { + this.pending = '' + return + } + if (this.pending.length > 0) { + throw new Error('subprocess-e2b: truncated base64 output transport') + } + if (!this.complete) throw new Error('subprocess-e2b: incomplete output transport') + } +} + +/** Offset reader used for one collect-mode E2B stream. */ +export class E2BOutputReader implements SubprocessOutputReader { + private chunks: Buffer[] = [] + private retainedBytes = 0 + private totalBytes = 0 + private spillValid = true + + /** + * Create a bounded reader over one remote spill path. + * @param maxBytes - In-memory tail cap. + * @param maxSpillBytes - Maximum complete remote file size the caller accepts. + * @param spillPath - Remote full-output path. + */ + constructor( + private readonly maxBytes: number, + private readonly maxSpillBytes: number | undefined, + private readonly spillPath: string, + ) {} + + /** Total bytes observed from the SDK stream. */ + get size(): number { + return this.totalBytes + } + + /** Stop advertising a remote spill whose writer did not reach clean EOF. */ + invalidateSpill(): void { + this.spillValid = false + } + + /** + * Append one byte-faithful decoded transport event. + * @param bytes - Raw command bytes recovered from the ASCII SDK transport. + */ + push(bytes: Uint8Array): void { + if (bytes.length === 0) return + const chunk = Buffer.from(bytes) + this.totalBytes += chunk.length + this.chunks.push(chunk) + this.retainedBytes += chunk.length + while (this.retainedBytes > this.maxBytes) { + const head = this.chunks[0] as Buffer + const excess = this.retainedBytes - this.maxBytes + if (head.length <= excess) { + this.chunks.shift() + this.retainedBytes -= head.length + } else { + this.chunks[0] = head.subarray(excess) + this.retainedBytes -= excess + } + } + } + + /** @inheritdoc */ + readFrom(fromByte: number): SubprocessOutputRead { + const retained = Buffer.concat(this.chunks, this.retainedBytes) + const firstRetained = this.totalBytes - this.retainedBytes + const lossy = fromByte < firstRetained + const start = lossy ? 0 : Math.min(retained.length, Math.max(0, fromByte - firstRetained)) + return { + text: retained.subarray(start).toString('utf8'), + nextOffset: this.totalBytes, + lossy, + ...(lossy && this.spillValid && this.maxSpillBytes !== undefined && this.totalBytes <= this.maxSpillBytes + ? { spillPath: this.spillPath } + : {}), + } + } +} diff --git a/packages/e2b/subprocess-e2b/src/process.ts b/packages/e2b/subprocess-e2b/src/process.ts new file mode 100644 index 0000000000..c80a2f5bd9 --- /dev/null +++ b/packages/e2b/subprocess-e2b/src/process.ts @@ -0,0 +1,698 @@ +/** One asynchronously-started E2B command projected onto the subprocess seam. */ + +import { Buffer } from 'node:buffer' +import { PassThrough, Writable } from 'node:stream' +import { posix } from 'node:path' +import { + CommandExitError, + e2bControlEnvs, + FileNotFoundError, + SandboxNotFoundError, + quoteE2BShellArg, +} from '@deepseek-ai/dsh-e2b' +import type { CommandHandle, CommandResult, Sandbox } from '@deepseek-ai/dsh-e2b' +import type { + SubprocessCollect, + SubprocessHandle, + SubprocessOutcome, + SubprocessOutputMode, + SubprocessSpawnSpec, +} from '@deepseek-ai/dsh-subprocess' +import type E2BSandboxService from '@deepseek-ai/dsh-e2b' +import { bootstrapEnvironment, readRemoteEnvironment, serializeRemoteEnvironment } from './environment.ts' +import { E2BBase64Decoder, E2B_OUTPUT_COMPLETE_FRAME, E2BOutputReader } from './output.ts' +import { asError, commandOpts, signalRemoteGroups, waitTick } from './remote.ts' + +const OUTPUT_ENCODER_SOURCE = [ + '(async () => {', + ' for await (const chunk of process.stdin) {', + " if (!process.stdout.write(chunk.toString('base64') + '\\n')) {", + " await new Promise(resolve => process.stdout.once('drain', resolve))", + ' }', + ' }', + ` if (!process.stdout.write(${JSON.stringify(E2B_OUTPUT_COMPLETE_FRAME)} + '\\n')) {`, + " await new Promise(resolve => process.stdout.once('drain', resolve))", + ' }', + '})().catch(() => { process.exitCode = 1 })', +].join('\n') + +function isCollect(mode: SubprocessOutputMode): mode is SubprocessCollect { + return mode !== 'pipe' && mode !== 'inherit' +} + +function hasSpill(mode: SubprocessOutputMode): mode is SubprocessCollect & { spill: { maxBytes: number } } { + return isCollect(mode) && mode.spill !== undefined +} + +function isValidProcessId(value: number): boolean { + return Number.isSafeInteger(value) && value > 0 +} + +class DeferredStdin extends Writable { + constructor(private readonly ready: Promise) { + super({ decodeStrings: false }) + } + + override _write(chunk: string | Buffer, _encoding: BufferEncoding, callback: (error?: Error | null) => void): void { + void this.ready.then(handle => handle.sendStdin(chunk)).then( + () => { callback() }, + (error: unknown) => { callback(asError(error)) }, + ) + } + + override _final(callback: (error?: Error | null) => void): void { + void this.ready.then(handle => handle.closeStdin()).then( + () => { callback() }, + (error: unknown) => { callback(asError(error)) }, + ) + } +} + +interface RemotePaths { + pid: string + status: string + environment: string + stdout: string + stderr: string +} + +type CommandSettlement = + | { kind: 'result'; result: CommandResult } + | { kind: 'error'; error: unknown } + +function withinMs(settlement: Promise, timeoutMs: number): Promise { + return new Promise((resolve) => { + const timer = setTimeout(() => { resolve(undefined) }, timeoutMs) + void settlement.then((value) => { + clearTimeout(timer) + resolve(value) + }) + }) +} + +function commandText(spec: SubprocessSpawnSpec, paths: RemotePaths): string { + const encoder = `"$dsh_e2b_env_bin" -i "$dsh_e2b_node" -e ${quoteE2BShellArg(OUTPUT_ENCODER_SOURCE)}` + const stdoutRedirect = hasSpill(spec.stdio.stdout) + ? `> >("$dsh_e2b_tee" --output-error=warn-nopipe >("$dsh_e2b_head" -c ${spec.stdio.stdout.spill.maxBytes} > ${quoteE2BShellArg(paths.stdout)}) | ${encoder} 2>/dev/null)` + : `> >(${encoder} 2>/dev/null)` + const stderrRedirect = hasSpill(spec.stdio.stderr) + ? `2> >("$dsh_e2b_tee" --output-error=warn-nopipe >("$dsh_e2b_head" -c ${spec.stdio.stderr.spill.maxBytes} > ${quoteE2BShellArg(paths.stderr)}) | ${encoder} >&2 2>/dev/null)` + : `2> >(${encoder} >&2 2>/dev/null)` + const inner = [ + 'set +e', + 'dsh_e2b_env_bin=$1', + 'dsh_e2b_node=$2', + 'dsh_e2b_ps=$3', + 'dsh_e2b_tr=$4', + 'dsh_e2b_tee=$5', + 'dsh_e2b_head=$6', + 'dsh_e2b_rm=$7', + 'shift 7', + 'dsh_e2b_pgid="$("$dsh_e2b_ps" -o pgid= -p "$$" | "$dsh_e2b_tr" -d " ")"', + `printf '%s\\n' "$dsh_e2b_pgid" > ${quoteE2BShellArg(paths.pid)}`, + `mapfile -d '' -t dsh_e2b_env < ${quoteE2BShellArg(paths.environment)}`, + `"$dsh_e2b_rm" -f -- ${quoteE2BShellArg(paths.environment)}`, + `"$dsh_e2b_env_bin" -i -- "\${dsh_e2b_env[@]}" "$@" ${stdoutRedirect} ${stderrRedirect}`.trimEnd(), + 'dsh_e2b_status=$?', + `printf '%s\\n' "$dsh_e2b_status" > ${quoteE2BShellArg(paths.status)}`, + 'wait', + 'exit "$dsh_e2b_status"', + ].join('\n') + const argv = spec.argv.map(quoteE2BShellArg).join(' ') + const bootstrap = [ + `mapfile -d '' -t dsh_e2b_env < ${quoteE2BShellArg(paths.environment)}`, + 'dsh_e2b_env_bin="$(command -v env)"', + 'dsh_e2b_setsid="$(command -v setsid)"', + 'dsh_e2b_bash="$(command -v bash)"', + 'dsh_e2b_node="$(command -v node)"', + 'dsh_e2b_ps="$(command -v ps)"', + 'dsh_e2b_tr="$(command -v tr)"', + 'dsh_e2b_tee="$(command -v tee)"', + 'dsh_e2b_head="$(command -v head)"', + 'dsh_e2b_rm="$(command -v rm)"', + 'for dsh_e2b_tool in "$dsh_e2b_env_bin" "$dsh_e2b_setsid" "$dsh_e2b_bash" "$dsh_e2b_node" "$dsh_e2b_ps" "$dsh_e2b_tr" "$dsh_e2b_tee" "$dsh_e2b_head" "$dsh_e2b_rm"; do', + ' [[ "$dsh_e2b_tool" == /* && -x "$dsh_e2b_tool" ]] || exit 125', + 'done', + `exec "$dsh_e2b_env_bin" -i -- "\${dsh_e2b_env[@]}" "$dsh_e2b_setsid" --wait -- "$dsh_e2b_bash" -c ${quoteE2BShellArg(inner)} dsh-e2b "$dsh_e2b_env_bin" "$dsh_e2b_node" "$dsh_e2b_ps" "$dsh_e2b_tr" "$dsh_e2b_tee" "$dsh_e2b_head" "$dsh_e2b_rm" ${argv}`, + ].join('\n') + return bootstrap +} + +const WAIT_ABORTED = Symbol('wait aborted') + +function waitWithSignal(promise: Promise, signal: AbortSignal | undefined): Promise { + if (signal === undefined) return promise + if (signal.aborted) return Promise.resolve(WAIT_ABORTED) + return new Promise((resolve) => { + const onAbort = (): void => { cleanup(); resolve(WAIT_ABORTED) } + const cleanup = (): void => { signal.removeEventListener('abort', onAbort) } + signal.addEventListener('abort', onAbort, { once: true }) + if (signal.aborted) { + onAbort() + return + } + void promise.then((value) => { cleanup(); resolve(value) }) + }) +} + +/** E2B-backed subprocess handle with deferred remote PID acquisition. */ +export class E2BSubprocessHandle implements SubprocessHandle { + readonly stdin: Writable | undefined + readonly stdout: PassThrough | undefined + readonly stderr: PassThrough | undefined + readonly collected: SubprocessHandle['collected'] + readonly done: Promise + + private readonly commandState = Promise.withResolvers() + private readonly readyState = Promise.withResolvers() + private readonly stdoutDecoder = new E2BBase64Decoder() + private readonly stderrDecoder = new E2BBase64Decoder() + private readonly terminationController = new AbortController() + /** Releases output waits that survive the command outcome, so blocked SDK callbacks settle. */ + private readonly outputReleased = new AbortController() + private readonly stdoutReader: E2BOutputReader | undefined + private readonly stderrReader: E2BOutputReader | undefined + private readonly paths: RemotePaths + private controlEnvs: Record = {} + private remotePid = -1 + private outputTransportError: Error | undefined + private outputDrainExpired = false + private stateDirectoryCreated = false + private quiescenceProven = false + private terminationAttempt: Promise | undefined + private terminationFailure: Error | undefined + private terminationSignal: NodeJS.Signals | null = null + + /** + * Begin an E2B command without blocking the synchronous subprocess spawn seam. + * @param runtime - Shared E2B sandbox owner. + * @param spec - Fully resolved subprocess request. + * @param stateDir - Remote directory retaining process identity, status, and valid spills. + * @param pollMs - Remote status/liveness poll cadence. + */ + constructor( + private readonly runtime: E2BSandboxService, + private readonly spec: SubprocessSpawnSpec, + readonly stateDir: string, + private readonly pollMs: number, + ) { + this.paths = { + pid: posix.join(stateDir, 'pid'), + status: posix.join(stateDir, 'exit-code'), + environment: posix.join(stateDir, 'environment'), + stdout: posix.join(stateDir, 'stdout.log'), + stderr: posix.join(stateDir, 'stderr.log'), + } + const outMode = spec.stdio.stdout + const errMode = spec.stdio.stderr + this.stdout = outMode === 'pipe' ? new PassThrough() : undefined + this.stderr = errMode === 'pipe' ? new PassThrough() : undefined + this.stdoutReader = isCollect(outMode) + ? new E2BOutputReader(outMode.maxBytes, outMode.spill?.maxBytes, this.paths.stdout) + : undefined + this.stderrReader = isCollect(errMode) + ? new E2BOutputReader(errMode.maxBytes, errMode.spill?.maxBytes, this.paths.stderr) + : undefined + this.collected = { + ...(this.stdoutReader !== undefined ? { stdout: this.stdoutReader } : {}), + ...(this.stderrReader !== undefined ? { stderr: this.stderrReader } : {}), + } + this.stdin = spec.stdio.stdin === 'pipe' ? new DeferredStdin(this.readyState.promise) : undefined + void this.readyState.promise.catch(() => {}) + spec.signal?.addEventListener('abort', this.onAbort, { once: true }) + this.done = this.run() + void this.done.catch(() => {}) + if (spec.signal?.aborted === true) this.terminate() + } + + /** Remote process id after start; `-1` while E2B startup is pending or after it fails. */ + get pid(): number { + return this.remotePid + } + + /** @inheritdoc */ + terminate(): void { + if (this.quiescenceProven || this.terminationAttempt !== undefined) return + this.terminationController.abort(new Error('subprocess-e2b: command terminated')) + this.stdout?.destroy() + this.stderr?.destroy() + this.terminationFailure = undefined + const attempt = this.terminateRemote() + this.terminationAttempt = attempt + void attempt.then( + () => { this.terminationAttempt = undefined }, + (error: unknown) => { + if (!this.quiescenceProven) this.terminationFailure = asError(error) + this.terminationAttempt = undefined + }, + ) + } + + /** @inheritdoc */ + async waitForExit(signal?: AbortSignal): Promise { + if (this.quiescenceProven) return true + let handle: CommandHandle | undefined + if (this.terminationController.signal.aborted) { + const observed = await waitWithSignal(this.commandState.promise, signal) + if (observed === WAIT_ABORTED) return false + handle = observed + if (handle === undefined) { + this.markQuiescent() + return true + } + if (this.remotePid <= 0) { + const attempt = this.terminationAttempt + if (attempt !== undefined && await waitWithSignal(attempt.catch(() => undefined), signal) === WAIT_ABORTED) { + return false + } + this.throwTerminationFailure() + // Successful pre-publication termination records quiescence; its only other outcome is the failure above. + return true + } + } else { + const observed = await waitWithSignal( + this.readyState.promise.catch(() => this.commandState.promise), + signal, + ) + if (observed === WAIT_ABORTED) return false + handle = observed + if (handle === undefined) { + this.markQuiescent() + return true + } + } + this.throwTerminationFailure() + let sandbox: Sandbox + try { + sandbox = await this.runtime.getSandbox() + } catch (error: unknown) { + if (signal?.aborted === true) return false + if (error instanceof SandboxNotFoundError) { + this.markQuiescent() + return true + } + throw error + } + const processGroupId = this.remotePid > 0 ? this.remotePid : handle.pid + while (await this.groupAlive(sandbox, processGroupId, signal)) { + this.throwTerminationFailure() + if (!await waitTick(this.pollMs, signal)) return false + } + this.throwTerminationFailure() + if (signal?.aborted === true) return false + this.markQuiescent() + return true + } + + private readonly onAbort = (): void => { this.terminate() } + + private markQuiescent(): void { + this.quiescenceProven = true + this.terminationFailure = undefined + } + + private async run(): Promise { + let sandbox: Sandbox | undefined + let preparing = true + try { + sandbox = await this.runtime.getSandbox() + await this.prepareState(sandbox) + preparing = false + const handle = await sandbox.commands.run( + commandText(this.spec, this.paths), + { + background: true, + cwd: this.spec.cwd, + envs: e2bControlEnvs(this.controlEnvs), + stdin: this.spec.stdio.stdin !== 'ignore', + timeoutMs: 0, + onStdout: async (data) => { await this.dispatchOutput('stdout', data) }, + onStderr: async (data) => { await this.dispatchOutput('stderr', data) }, + }, + ) + const completion = handle.wait() + void completion.catch(() => {}) + if (!isValidProcessId(handle.pid)) { + const invalidPid = new Error(`subprocess-e2b: E2B returned invalid command pid ${handle.pid}`) + try { + await handle.kill() + this.markQuiescent() + } catch (cleanupError: unknown) { + this.terminationFailure = asError(cleanupError) + this.commandState.resolve(handle) + throw new AggregateError( + [invalidPid, cleanupError], + 'subprocess-e2b: invalid command pid rollback did not reach quiescence', + ) + } + throw invalidPid + } + this.commandState.resolve(handle) + try { + this.remotePid = await this.waitForProcessGroupId(sandbox, completion) + } catch (error: unknown) { + try { + await this.rollbackUnpublishedGroup(sandbox, handle) + } catch (cleanupError: unknown) { + throw new AggregateError( + [error, cleanupError], + 'subprocess-e2b: process-group publication failed and rollback did not reach quiescence', + ) + } + throw error + } + this.readyState.resolve(handle) + await this.writeBatchStdin(handle) + const outcome = await this.waitForCommand(sandbox, handle, completion) + if (this.outputTransportError !== undefined) throw this.outputTransportError + const requireCompleteOutput = this.terminationSignal === null && !this.outputDrainExpired + this.stdoutDecoder.finish(requireCompleteOutput) + this.stderrDecoder.finish(requireCompleteOutput) + await this.finalizeSpills(sandbox) + return outcome + } catch (error: unknown) { + const canceledPreparation = preparing && this.terminationController.signal.aborted + let failure = await this.rollbackPublishedFailure(error) + if (sandbox !== undefined && this.stateDirectoryCreated) { + try { + await this.removeFailedState(sandbox) + } catch (cleanupError: unknown) { + failure = new AggregateError( + [failure, cleanupError], + 'subprocess-e2b: command failed and private state cleanup failed', + ) + } + } + this.commandState.resolve(undefined) + this.readyState.reject(failure) + if (canceledPreparation && failure === error) return { exitCode: null, signal: 'SIGTERM' } + throw failure + } finally { + this.spec.signal?.removeEventListener('abort', this.onAbort) + this.stdout?.end() + this.stderr?.end() + } + } + + private async prepareState(sandbox: Sandbox): Promise { + const signal = this.terminationController.signal + const ambient = await readRemoteEnvironment(sandbox, signal) + this.controlEnvs = bootstrapEnvironment(ambient) + // Own the directory before the request: a cancellation racing a committed + // creation must still enter cleanup (removal tolerates an absent path). + this.stateDirectoryCreated = true + await sandbox.files.makeDir(this.stateDir, { signal }) + await sandbox.commands.run( + `chmod 700 -- ${quoteE2BShellArg(this.stateDir)}`, + commandOpts(this.controlEnvs, signal), + ) + const files = [ + { path: this.paths.pid, data: '' }, + { path: this.paths.status, data: '' }, + { path: this.paths.environment, data: serializeRemoteEnvironment(ambient, this.spec.env) }, + ...(hasSpill(this.spec.stdio.stdout) ? [{ path: this.paths.stdout, data: '' }] : []), + ...(hasSpill(this.spec.stdio.stderr) ? [{ path: this.paths.stderr, data: '' }] : []), + ] + await sandbox.files.write(files, { signal }) + await sandbox.commands.run( + `chmod 600 -- ${files.map(file => quoteE2BShellArg(file.path)).join(' ')}`, + commandOpts(this.controlEnvs, signal), + ) + signal.throwIfAborted() + } + + private async writeBatchStdin(handle: CommandHandle): Promise { + if (typeof this.spec.stdio.stdin !== 'object') return + try { + await handle.sendStdin(this.spec.stdio.stdin.data) + await handle.closeStdin() + } catch (_processClosedItsInput) { + // Like the local adapter, batch stdin is best-effort; exit and output remain authoritative. + } + } + + private async dispatchOutput(stream: 'stdout' | 'stderr', data: string): Promise { + let bytes: Buffer + try { + bytes = stream === 'stdout' ? this.stdoutDecoder.push(data) : this.stderrDecoder.push(data) + } catch (error: unknown) { + this.outputTransportError ??= asError(error) + const target = stream === 'stdout' ? this.stdout : this.stderr + target?.destroy(this.outputTransportError) + return + } + try { + if (stream === 'stdout') { + this.stdoutReader?.push(bytes) + await this.writeOutput(this.stdout, this.spec.stdio.stdout === 'inherit' ? process.stdout : undefined, bytes) + return + } + this.stderrReader?.push(bytes) + await this.writeOutput(this.stderr, this.spec.stdio.stderr === 'inherit' ? process.stderr : undefined, bytes) + } catch (error: unknown) { + const target = stream === 'stdout' ? this.stdout : this.stderr + target?.destroy(asError(error)) + } + } + + private async writeOutput(pipe: PassThrough | undefined, inherited: NodeJS.WriteStream | undefined, data: Uint8Array): Promise { + const target = pipe ?? inherited + if (target === undefined || data.length === 0 || this.terminationController.signal.aborted) return + if (target.destroyed) throw new Error('subprocess output stream is closed') + if (target.write(data)) return + await new Promise((resolve, reject) => { + const onDrain = (): void => { cleanup(); resolve() } + const onClose = (): void => { cleanup(); resolve() } + const onRelease = (): void => { cleanup(); resolve() } + const onError = (error: Error): void => { cleanup(); reject(error) } + const cleanup = (): void => { + target.removeListener('drain', onDrain) + target.removeListener('close', onClose) + target.removeListener('error', onError) + this.terminationController.signal.removeEventListener('abort', onRelease) + this.outputReleased.signal.removeEventListener('abort', onRelease) + } + target.once('drain', onDrain) + target.once('close', onClose) + target.once('error', onError) + this.terminationController.signal.addEventListener('abort', onRelease, { once: true }) + this.outputReleased.signal.addEventListener('abort', onRelease, { once: true }) + if (this.terminationController.signal.aborted || this.outputReleased.signal.aborted) onRelease() + }) + } + + private async waitForProcessGroupId(sandbox: Sandbox, completion: Promise): Promise { + const commandSettled = completion.then( + () => true, + () => true, + ) + while (true) { + // TODO(e2b-publication-cancel): Join cancellation to the existing + // termination transaction before aborting an in-flight SDK file read. + const raw = await sandbox.files.read(this.paths.pid) + const value = raw.trim() + if (value.length > 0) { + const pid = Number(value) + if (!/^[1-9][0-9]*$/.test(value) || !Number.isSafeInteger(pid)) { + throw new Error(`subprocess-e2b: remote wrapper published invalid process-group id ${JSON.stringify(value)}`) + } + // A same-UID sandbox process can rewrite this file; refuse ids whose + // negative form addresses every process (`kill -- -1`) or init's group. + if (pid <= 1) { + throw new Error(`subprocess-e2b: unsafe published process-group id ${pid}`) + } + return pid + } + const settled = await Promise.race([commandSettled, waitTick(this.pollMs).then(() => false)]) + if (settled) throw new Error('subprocess-e2b: remote command exited before publishing its process-group id') + } + } + + private async waitForCommand( + sandbox: Sandbox, + handle: CommandHandle, + completion: Promise, + ): Promise { + const settlement = completion.then( + result => ({ kind: 'result', result }), + (error: unknown) => ({ kind: 'error', error }), + ) + const hasPipeOutput = this.spec.stdio.stdout === 'pipe' || this.spec.stdio.stderr === 'pipe' + let completed = hasPipeOutput ? await settlement : undefined + while (true) { + const rawStatus = (await sandbox.files.read(this.paths.status)).trim() + if (rawStatus.length > 0) { + const exitCode = Number(rawStatus) + if (!/^(?:0|[1-9][0-9]*)$/.test(rawStatus) || !Number.isSafeInteger(exitCode) || exitCode > 255) { + throw new Error(`subprocess-e2b: remote wrapper published invalid exit code ${JSON.stringify(rawStatus)}`) + } + if (completed !== undefined) return this.commandOutcome(completed, exitCode) + const drained = await withinMs(settlement, this.spec.graceMs) + if (drained !== undefined) return this.commandOutcome(drained, exitCode) + this.outputDrainExpired = true + this.stdoutReader?.invalidateSpill() + this.stderrReader?.invalidateSpill() + // Release inherited-output waits so a callback blocked on host + // backpressure cannot keep the disconnected SDK settlement pending. + this.outputReleased.abort(new Error('subprocess-e2b: output drain grace expired')) + await handle.disconnect() + return { exitCode, signal: null } + } + if (completed !== undefined) return this.commandOutcome(completed) + // TODO(e2b-status-watch): Replace collect/inherit control-plane polling + // when E2B can observe direct-command exit independently of descendant-held output. + completed = await Promise.race([settlement, waitTick(this.pollMs).then(() => undefined)]) + } + } + + private commandOutcome(settlement: CommandSettlement, publishedExitCode?: number): SubprocessOutcome { + if (settlement.kind === 'result') { + return { exitCode: publishedExitCode ?? settlement.result.exitCode, signal: null } + } + if (settlement.error instanceof CommandExitError) { + if (publishedExitCode !== undefined) return { exitCode: publishedExitCode, signal: null } + return this.terminationSignal === null + ? { exitCode: settlement.error.exitCode, signal: null } + : { exitCode: null, signal: this.terminationSignal } + } + throw settlement.error + } + + private async rollbackPublishedFailure(error: unknown): Promise { + if (this.remotePid <= 0 || this.quiescenceProven) return error + this.terminate() + try { + await this.waitForExit() + return error + } catch (cleanupError: unknown) { + return new AggregateError( + [asError(error), asError(cleanupError)], + 'subprocess-e2b: command monitoring failed and process-group rollback did not reach quiescence', + ) + } + } + + private async rollbackUnpublishedGroup(sandbox: Sandbox, handle: CommandHandle): Promise { + // The bootstrap ends in an exec chain through the scrubbed environment and + // `setsid`, so E2B's command PID is the provisional group id even before the + // private publication file can be trusted. Kill that group before the SDK-PID + // fallback, then prove no group member survived before rejecting startup. + await this.forceKillGroup(sandbox, handle, handle.pid) + this.markQuiescent() + } + + private async terminateRemote(): Promise { + try { + await this.terminateRemoteInSandbox() + } catch (error: unknown) { + if (error instanceof SandboxNotFoundError) { + this.markQuiescent() + return + } + throw error + } + } + + private async terminateRemoteInSandbox(): Promise { + const handle = await this.commandState.promise + if (handle === undefined) { + this.markQuiescent() + return + } + if (!isValidProcessId(handle.pid) && this.remotePid <= 0) { + await handle.kill() + this.markQuiescent() + return + } + const sandbox = await this.runtime.getSandbox() + const processGroupId = this.remotePid > 0 ? this.remotePid : handle.pid + await this.terminateGroup(sandbox, handle, processGroupId) + } + + private async terminateGroup(sandbox: Sandbox, handle: CommandHandle, processGroupId: number): Promise { + this.terminationSignal = 'SIGTERM' + try { + await signalRemoteGroups(sandbox, this.controlEnvs, [processGroupId], 'TERM') + if (await this.waitForGroupExit(sandbox, processGroupId)) { + this.markQuiescent() + return + } + } catch (_gracefulTerminationFailure) { + // Failed TERM delivery or observation cannot prove exit; force cleanup still owns the group. + } + this.terminationSignal = 'SIGKILL' + await this.forceKillGroup(sandbox, handle, processGroupId) + this.markQuiescent() + } + + private async forceKillGroup(sandbox: Sandbox, handle: CommandHandle, processGroupId: number): Promise { + try { + await signalRemoteGroups(sandbox, this.controlEnvs, [processGroupId], 'KILL') + } catch (_processGroupKillFailure) { + // SDK kill and the final liveness probe remain independent cleanup paths. + } + try { + await handle.kill() + } catch (_sdkKillFailure) { + // The final liveness probe, not either transport's self-report, proves cleanup. + } + if (await this.waitForGroupExit(sandbox, processGroupId)) return + throw new Error(`subprocess-e2b: remote process group ${processGroupId} remained live after force termination`) + } + + private async waitForGroupExit(sandbox: Sandbox, processGroupId: number): Promise { + const deadline = Date.now() + this.spec.graceMs + while (await this.groupAlive(sandbox, processGroupId)) { + if (Date.now() >= deadline) return false + await waitTick(this.pollMs) + } + return true + } + + private throwTerminationFailure(): void { + if (this.terminationFailure !== undefined) throw this.terminationFailure + } + + private async groupAlive(sandbox: Sandbox, pid: number, signal?: AbortSignal): Promise { + const result = await sandbox.commands.run( + `set -o pipefail; ps -eo pgid=,stat= | awk '$1 == ${pid} && $2 !~ /^[ZXx]/ { live=1 } END { if (live) print "live" }'`, + commandOpts(this.controlEnvs, signal), + ).catch((error: unknown) => { + if (signal?.aborted === true) return undefined + if (error instanceof SandboxNotFoundError) return { exitCode: 0, stdout: '', stderr: '' } + throw error + }) + return result?.stdout.trim() === 'live' + } + + private async finalizeSpills(sandbox: Sandbox): Promise { + const removals: Promise[] = [] + const collect = (mode: SubprocessOutputMode, reader: E2BOutputReader | undefined, path: string): void => { + if (!hasSpill(mode)) return + // A spill mode is a collect mode, so construction always created its reader. + const size = (reader as E2BOutputReader).size + if (this.outputDrainExpired || size <= mode.maxBytes || size > mode.spill.maxBytes) { + removals.push(sandbox.files.remove(path).catch((_adapterPrivateSpillRemovalFailure: unknown) => { + // The command outcome is authoritative; owner teardown bounds private residue. + })) + } + } + collect(this.spec.stdio.stdout, this.stdoutReader, this.paths.stdout) + collect(this.spec.stdio.stderr, this.stderrReader, this.paths.stderr) + await Promise.all(removals) + } + + private async removeFailedState(sandbox: Sandbox): Promise { + const failures: Error[] = [] + for (const path of [this.paths.environment, this.stateDir]) { + try { + await sandbox.files.remove(path) + } catch (error: unknown) { + if (!(error instanceof FileNotFoundError)) failures.push(asError(error)) + } + } + if (failures.length > 0) { + throw new AggregateError(failures, 'subprocess-e2b: failed to remove private command state') + } + } +} diff --git a/packages/e2b/subprocess-e2b/src/remote.ts b/packages/e2b/subprocess-e2b/src/remote.ts new file mode 100644 index 0000000000..1937f54bf0 --- /dev/null +++ b/packages/e2b/subprocess-e2b/src/remote.ts @@ -0,0 +1,97 @@ +/** + * Shared remote-control helpers for the E2B subprocess adapter: SDK option + * shaping, poll ticks, and the one tolerant process-group signal used by both + * the ordinary-process and terminal teardown ladders. + */ + +import { CommandExitError, e2bControlEnvs, SandboxNotFoundError } from '@deepseek-ai/dsh-e2b' +import type { Sandbox } from '@deepseek-ai/dsh-e2b' + +/** + * Normalize an unknown rejection into an Error. + * @param error - Any thrown or rejected value. + * @returns The value itself when already an Error, else a stringified wrapper. + */ +export function asError(error: unknown): Error { + return error instanceof Error ? error : new Error(String(error)) +} + +/** + * Shape the optional-signal SDK options object. + * @param signal - Optional cancellation for one SDK request. + * @returns An options fragment that omits an undefined signal. + */ +export function signalOpts(signal: AbortSignal | undefined): { signal?: AbortSignal } { + return signal === undefined ? {} : { signal } +} + +/** + * Shape control-shell command options with the isolated HOME override. + * @param envs - Explicit environment entries for the control command. + * @param signal - Optional cancellation for the SDK request. + * @returns Options for `sandbox.commands.run` control invocations. + */ +export function commandOpts( + envs: Record, + signal?: AbortSignal, +): { envs: Record; signal?: AbortSignal } { + return { envs: e2bControlEnvs(envs), ...signalOpts(signal) } +} + +/** + * Resolve after one duration. + * @param ms - Milliseconds to wait. + * @returns Settles after the timeout. + */ +export function delay(ms: number): Promise { + return new Promise(resolve => setTimeout(resolve, ms)) +} + +/** + * Wait one poll interval or until the signal aborts. + * @param pollMs - Poll cadence in milliseconds. + * @param signal - Optional abort that ends the wait early. + * @returns `true` after a full tick, `false` when aborted first. + */ +export function waitTick(pollMs: number, signal?: AbortSignal): Promise { + if (signal?.aborted === true) return Promise.resolve(false) + return new Promise((resolve) => { + const timer = setTimeout(() => { + signal?.removeEventListener('abort', onAbort) + resolve(true) + }, pollMs) + const onAbort = (): void => { + clearTimeout(timer) + resolve(false) + } + signal?.addEventListener('abort', onAbort, { once: true }) + }) +} + +/** + * Signal remote process groups, tolerating the shared teardown outcomes: a + * nonzero `kill` (groups already gone) and a disappeared sandbox. Both the + * pgid-keyed process ladder and the sid-keyed terminal ladder deliver signals + * through this single tolerance so they cannot drift apart. + * @param sandbox - Live SDK handle. + * @param envs - Control-shell environment entries. + * @param groups - Positive process-group ids to signal. + * @param signal - `TERM` or `KILL`. + */ +export async function signalRemoteGroups( + sandbox: Sandbox, + envs: Record, + groups: readonly number[], + signal: 'TERM' | 'KILL', +): Promise { + // TODO(e2b-pgid-identity): Prefer an atomic identity-bound group signal if E2B adds one; + // a userspace identity precheck cannot close the numeric-PGID reuse race. + try { + await sandbox.commands.run( + `kill -${signal} -- ${groups.map(group => `-${group}`).join(' ')}`, + commandOpts(envs), + ) + } catch (error: unknown) { + if (!(error instanceof CommandExitError) && !(error instanceof SandboxNotFoundError)) throw error + } +} diff --git a/packages/e2b/subprocess-e2b/src/terminal.ts b/packages/e2b/subprocess-e2b/src/terminal.ts new file mode 100644 index 0000000000..c8f3bb57b7 --- /dev/null +++ b/packages/e2b/subprocess-e2b/src/terminal.ts @@ -0,0 +1,567 @@ +/** E2B PTY allocation and process-session ownership for the subprocess seam. */ + +import { Buffer } from 'node:buffer' +import { randomUUID } from 'node:crypto' +import { PassThrough } from 'node:stream' +import { posix } from 'node:path' +import { + CommandExitError, + e2bControlEnvs, + FileNotFoundError, + SandboxNotFoundError, + quoteE2BShellArg, +} from '@deepseek-ai/dsh-e2b' +import type { CommandHandle, CommandResult, Sandbox } from '@deepseek-ai/dsh-e2b' +import type { + SubprocessOutcome, + SubprocessTerminalForeground, + SubprocessTerminalHandle, + SubprocessTerminalSignal, + SubprocessTerminalSpawnSpec, +} from '@deepseek-ai/dsh-subprocess' +import type E2BSandboxService from '@deepseek-ai/dsh-e2b' +import { + bootstrapEnvironment, + readRemoteEnvironment, + serializeRemoteEnvironment, +} from './environment.ts' +import { asError, commandOpts, delay, signalOpts, signalRemoteGroups } from './remote.ts' + +const TERMINAL_RUNNER_SOURCE = [ + '#!/bin/bash', + 'set -euo pipefail', + 'dsh_state=$1', + 'mapfile -d \'\' -t dsh_env < "$dsh_state/environment"', + 'mapfile -d \'\' -t dsh_argv < "$dsh_state/argv"', + 'dsh_output_marker=$(<"$dsh_state/output-marker")', + 'rm -f -- "$dsh_state/environment" "$dsh_state/argv" "$dsh_state/output-marker" "$dsh_state/runner.bash"', + 'if (( ${#dsh_argv[@]} == 0 )); then', + " printf 'terminal runner received empty argv\\n' >&2", + ' exit 125', + 'fi', + 'printf \'%s\' "$dsh_output_marker"', + 'exec env -i -- "${dsh_env[@]}" "${dsh_argv[@]}"', + '', +].join('\n') + +interface TerminalPaths { + runner: string + environment: string + argv: string + outputMarker: string +} + +class BootstrapOutputFilter { + readonly ready: Promise + + private readonly readyState = Promise.withResolvers() + private pending = Buffer.alloc(0) + private published = false + + constructor( + private readonly marker: Buffer, + private readonly output: PassThrough, + ) { + this.ready = this.readyState.promise + } + + push(data: Uint8Array): void { + if (this.published) { + this.write(data) + return + } + const combined = Buffer.concat([this.pending, Buffer.from(data)]) + const markerOffset = combined.indexOf(this.marker) + if (markerOffset < 0) { + const retained = Math.min(combined.length, this.marker.length - 1) + this.pending = Buffer.from(combined.subarray(combined.length - retained)) + return + } + this.published = true + this.pending = Buffer.alloc(0) + this.readyState.resolve() + this.write(combined.subarray(markerOffset + this.marker.length)) + } + + private write(data: Uint8Array): void { + if (data.length > 0 && !this.output.destroyed) this.output.write(data) + } +} + +async function waitForBootstrapOutput( + ready: Promise, + completion: Promise, + signal?: AbortSignal, +): Promise { + signal?.throwIfAborted() + await new Promise((resolve, reject) => { + let settled = false + let removeAbort: (() => void) | undefined + const finish = (complete: () => void): void => { + if (settled) return + settled = true + removeAbort?.() + complete() + } + const onExit = (): void => { + finish(() => { reject(new Error('subprocess-e2b: terminal exited before publishing its output boundary')) }) + } + if (signal !== undefined) { + const onAbort = (): void => { + finish(() => { reject(asError(signal.reason)) }) + } + signal.addEventListener('abort', onAbort, { once: true }) + removeAbort = () => { signal.removeEventListener('abort', onAbort) } + } + void ready.then(() => { finish(resolve) }) + void completion.then(onExit, onExit) + }) +} + +function parsePositiveId(value: string, message: string): number { + const raw = value.trim() + const id = Number(raw) + if (!/^[1-9][0-9]*$/.test(raw) || !Number.isSafeInteger(id)) throw new Error(message) + return id +} + +function serializeValues(values: readonly string[], kind: string): string { + for (const value of values) { + if (value.includes('\0')) throw new Error(`subprocess-e2b: terminal ${kind} must not contain NUL bytes`) + } + return values.map(value => `${value}\0`).join('') +} + +async function terminalSessionId( + sandbox: Sandbox, + pid: number, + envs: Record, + signal?: AbortSignal, +): Promise { + const result = await sandbox.commands.run(`ps -o sid= -p ${pid}`, commandOpts(envs, signal)) + signal?.throwIfAborted() + return parsePositiveId(result.stdout, `subprocess-e2b: cannot resolve process session for terminal ${pid}`) +} + +async function sessionProcessGroups( + sandbox: Sandbox, + sessionId: number, + envs: Record, +): Promise { + let result: CommandResult + try { + result = await sandbox.commands.run( + `set -o pipefail; ps -eo sid=,pgid=,stat= | awk '$1 == ${sessionId} && $3 !~ /^[ZXx]/ { print $2 }'`, + commandOpts(envs), + ) + } catch (error: unknown) { + if (error instanceof SandboxNotFoundError) return [] + throw error + } + const groups = new Set() + for (const raw of result.stdout.trim().split(/\s+/)) { + if (raw.length === 0) continue + const group = parsePositiveId( + raw, + `subprocess-e2b: invalid process group ${JSON.stringify(raw)} in terminal session ${sessionId}`, + ) + if (group <= 1) { + throw new Error(`subprocess-e2b: unsafe process group ${group} in terminal session ${sessionId}`) + } + groups.add(group) + } + return [...groups] +} + +async function awaitSessionEmpty( + sandbox: Sandbox, + sessionId: number, + envs: Record, + graceMs: number, + pollMs: number, + kill = false, +): Promise { + const deadline = Date.now() + graceMs + for (;;) { + const groups = await sessionProcessGroups(sandbox, sessionId, envs) + if (groups.length === 0) return groups + if (kill) { + await signalRemoteGroups(sandbox, envs, groups, 'KILL') + if (Date.now() >= deadline) return await sessionProcessGroups(sandbox, sessionId, envs) + } else if (Date.now() >= deadline) { + return groups + } + await delay(Math.min(pollMs, Math.max(1, deadline - Date.now()))) + } +} + +async function rollbackUnpublishedTerminal( + sandbox: Sandbox, + handle: CommandHandle, + completion: Promise, + envs: Record, + graceMs: number, + pollMs: number, +): Promise { + let topLevelExited = false + void completion.then( + () => { topLevelExited = true }, + () => { topLevelExited = true }, + ) + const validPid = Number.isSafeInteger(handle.pid) && handle.pid > 1 + const attemptFailures: Error[] = [] + let sessionId: number | undefined + if (validPid) { + sessionId = handle.pid + try { + sessionId = await terminalSessionId(sandbox, handle.pid, envs) + } catch (_sessionLookupFailure) { + // E2B's PTY leader is also the provisional POSIX session leader, so its + // PID remains usable after the setup lookup itself fails or is canceled. + } + try { + let groups = await sessionProcessGroups(sandbox, sessionId, envs) + if (groups.length > 0) { + await signalRemoteGroups(sandbox, envs, groups, 'TERM') + groups = await awaitSessionEmpty(sandbox, sessionId, envs, graceMs, pollMs) + } + if (groups.length > 0) { + await awaitSessionEmpty(sandbox, sessionId, envs, graceMs, pollMs, true) + } + } catch (error: unknown) { + attemptFailures.push(asError(error)) + } + } + // Completion can settle while any awaited provider cleanup above is running. + // oxlint-disable-next-line typescript/no-unnecessary-condition -- Provider cleanup yields to completion. + if (!topLevelExited) { + try { + await handle.kill() + } catch (error: unknown) { + if (error instanceof SandboxNotFoundError) return + attemptFailures.push(asError(error)) + } + await Promise.race([completion.catch(() => undefined), delay(graceMs)]) + } + const proofFailures: Error[] = [] + if (sessionId !== undefined) { + try { + const groups = await awaitSessionEmpty(sandbox, sessionId, envs, graceMs, pollMs, true) + if (groups.length > 0) { + proofFailures.push(new Error( + `subprocess-e2b: terminal setup rollback failed; surviving process groups: ${groups.join(', ')}`, + )) + } + } catch (error: unknown) { + proofFailures.push(asError(error)) + } + } + // The bounded completion race above updates this callback-owned state. + // oxlint-disable-next-line typescript/no-unnecessary-condition -- The callback mutates this after a race. + if (!topLevelExited) { + proofFailures.push(new Error(`subprocess-e2b: terminal setup rollback failed; surviving pid: ${handle.pid}`)) + } + if (proofFailures.length > 0) { + throw new AggregateError( + [...attemptFailures, ...proofFailures], + 'subprocess-e2b: terminal setup rollback did not reach quiescence', + ) + } + try { + await handle.disconnect() + } catch (error: unknown) { + if (!(error instanceof SandboxNotFoundError)) throw error + } +} + +/** One E2B PTY and all process groups in its remote process session. */ +export class E2BTerminalHandle implements SubprocessTerminalHandle { + readonly pid: number + readonly done: Promise + + private topLevelExited = false + private cleanup: Promise | undefined + private readonly operationController = new AbortController() + private readonly operations = new Set>() + private terminationSignal: NodeJS.Signals | null = null + + constructor( + private readonly sandbox: Sandbox, + private readonly handle: CommandHandle, + readonly output: PassThrough, + private readonly completion: Promise, + private readonly sessionId: number, + private readonly controlEnvs: Record, + private readonly stateDir: string, + private readonly graceMs: number, + private readonly pollMs: number, + ) { + this.pid = handle.pid + this.done = this.waitForCommand() + } + + // TODO(e2b-pgid-identity): Replace retained numeric PTY/session ids when E2B + // exposes identity-bound input, foreground-signal, and cleanup operations. + /** @inheritdoc */ + write(data: string): Promise { + return this.trackOperation(async (signal) => { + if (this.topLevelExited) throw new Error('terminal process has exited') + await this.sandbox.pty.sendInput(this.pid, Buffer.from(data, 'utf8'), { signal }) + }) + } + + /** @inheritdoc */ + inspectForeground(): Promise { + return this.trackOperation(signal => this.inspectForegroundOnce(signal)) + } + + /** @inheritdoc */ + signalForeground(signal: SubprocessTerminalSignal): Promise { + return this.trackOperation(async (operationSignal) => { + const foreground = await this.inspectForegroundOnce(operationSignal) + if (foreground === undefined) { + throw new Error(`subprocess-e2b: cannot resolve foreground process group for terminal ${this.pid}`) + } + if (signal === 'SIGKILL' && foreground.processGroupId === this.pid) { + throw new Error('refusing to SIGKILL the terminal shell; terminate the terminal session instead') + } + await this.sandbox.commands.run( + `kill -${signal.slice(3)} -- -${foreground.processGroupId}`, + commandOpts(this.controlEnvs, operationSignal), + ) + return foreground.processGroupId + }) + } + + /** @inheritdoc */ + terminate(): Promise { + if (this.cleanup !== undefined) return this.cleanup + this.operationController.abort(new Error('subprocess-e2b: terminal is terminating')) + const cleanup = this.closeAfterOperations() + this.cleanup = cleanup + void cleanup.catch((_cleanupFailure: unknown) => { + this.cleanup = undefined + }) + return cleanup + } + + private async inspectForegroundOnce( + signal: AbortSignal, + ): Promise { + try { + const result = await this.sandbox.commands.run( + `ps -o tpgid= -p ${this.pid}`, + commandOpts(this.controlEnvs, signal), + ) + return { + processGroupId: parsePositiveId( + result.stdout, + `subprocess-e2b: cannot resolve foreground process group for terminal ${this.pid}`, + ), + // E2B exposes process-table commands but not the /proc memory access + // needed to prove a specific syscall is waiting on fd 0. + inputWaiting: false, + } + } catch (error: unknown) { + if (error instanceof CommandExitError && (error.exitCode === 1 || this.topLevelExited)) return undefined + throw error + } + } + + private trackOperation(operation: (signal: AbortSignal) => Promise): Promise { + if (this.operationController.signal.aborted) { + return Promise.reject(new Error('subprocess-e2b: terminal is terminating')) + } + const pending = operation(this.operationController.signal) + this.operations.add(pending) + void pending.then( + () => { this.operations.delete(pending) }, + () => { this.operations.delete(pending) }, + ) + return pending + } + + private async closeAfterOperations(): Promise { + await Promise.allSettled(this.operations) + await this.closeOnce() + } + + private async waitForCommand(): Promise { + try { + const result = await this.completion + return { exitCode: result.exitCode, signal: null } + } catch (error: unknown) { + if (error instanceof CommandExitError) { + return this.terminationSignal === null + ? { exitCode: error.exitCode, signal: null } + : { exitCode: null, signal: this.terminationSignal } + } + this.output.destroy(error instanceof Error ? error : new Error(String(error))) + throw error + } finally { + this.topLevelExited = true + if (!this.output.destroyed) this.output.end() + } + } + + private async closeOnce(): Promise { + let groups = await sessionProcessGroups(this.sandbox, this.sessionId, this.controlEnvs) + if (groups.length > 0) { + this.terminationSignal = 'SIGTERM' + await signalRemoteGroups(this.sandbox, this.controlEnvs, groups, 'TERM') + groups = await awaitSessionEmpty(this.sandbox, this.sessionId, this.controlEnvs, this.graceMs, this.pollMs) + } + if (groups.length === 0 && !this.topLevelExited) { + await Promise.race([this.done.catch(() => undefined), delay(this.graceMs)]) + } + if (groups.length > 0 || !this.topLevelExited) { + this.terminationSignal = 'SIGKILL' + if (!this.topLevelExited) { + try { + await this.handle.kill() + } catch (error: unknown) { + if (error instanceof SandboxNotFoundError) return + throw error + } + } + groups = await awaitSessionEmpty(this.sandbox, this.sessionId, this.controlEnvs, this.graceMs, this.pollMs, true) + if (!this.topLevelExited) await Promise.race([this.done.catch(() => undefined), delay(this.graceMs)]) + } + if (groups.length > 0) { + throw new Error(`subprocess-e2b: terminal cleanup failed; surviving process groups: ${groups.join(', ')}`) + } + if (!this.topLevelExited) { + throw new Error(`subprocess-e2b: terminal cleanup failed; surviving pid: ${this.pid}`) + } + try { + await this.handle.disconnect() + } catch (error: unknown) { + if (!(error instanceof SandboxNotFoundError)) throw error + } + try { + await this.sandbox.files.remove(this.stateDir) + } catch (_adapterPrivateStateRemovalFailure) { + // The terminal is quiescent; owner teardown bounds private residue. + } + } +} + +/** + * Allocate an E2B PTY, replace its bootstrap shell with the requested argv, + * and return only after the private runner has published readiness. + * @param runtime - Shared E2B sandbox owner. + * @param spec - Fully specified terminal-process request. + * @param stateDir - Private remote directory for one startup transaction. + * @param pollMs - Remote session liveness poll cadence. + * @returns The live subprocess terminal handle. + */ +export async function spawnE2BTerminal( + runtime: E2BSandboxService, + spec: SubprocessTerminalSpawnSpec, + stateDir: string, + pollMs: number, +): Promise { + const sandbox = await runtime.getSandbox() + spec.signal?.throwIfAborted() + const paths: TerminalPaths = { + runner: posix.join(stateDir, 'runner.bash'), + environment: posix.join(stateDir, 'environment'), + argv: posix.join(stateDir, 'argv'), + outputMarker: posix.join(stateDir, 'output-marker'), + } + const outputMarker = Buffer.from(`dsh-e2b-bootstrap:${randomUUID()}`) + const output = new PassThrough() + const outputFilter = new BootstrapOutputFilter(outputMarker, output) + let handle: CommandHandle | undefined + let completion: Promise | undefined + let stateDirectoryCreated = false + let controlEnvs: Record = {} + try { + const ambient = await readRemoteEnvironment(sandbox, spec.signal) + controlEnvs = bootstrapEnvironment(ambient) + const environment = serializeRemoteEnvironment(ambient, spec.env) + const argv = serializeValues(spec.argv, 'argv') + stateDirectoryCreated = true + await sandbox.files.makeDir(stateDir, signalOpts(spec.signal)) + await sandbox.commands.run( + `chmod 700 -- ${quoteE2BShellArg(stateDir)}`, + commandOpts(controlEnvs, spec.signal), + ) + await sandbox.files.write([ + { path: paths.runner, data: TERMINAL_RUNNER_SOURCE }, + { path: paths.environment, data: environment }, + { path: paths.argv, data: argv }, + { path: paths.outputMarker, data: outputMarker.toString('utf8') }, + ], signalOpts(spec.signal)) + await sandbox.commands.run( + `chmod 600 -- ${quoteE2BShellArg(paths.runner)} ${quoteE2BShellArg(paths.environment)} ${quoteE2BShellArg(paths.argv)} ${quoteE2BShellArg(paths.outputMarker)}`, + commandOpts(controlEnvs, spec.signal), + ) + handle = await sandbox.pty.create({ + rows: spec.rows, + cols: spec.cols, + cwd: spec.cwd, + envs: e2bControlEnvs(controlEnvs), + timeoutMs: 0, + onData: (data) => { outputFilter.push(data) }, + }) + completion = handle.wait() + void completion.catch(() => {}) + spec.signal?.throwIfAborted() + if (!Number.isSafeInteger(handle.pid) || handle.pid <= 0) { + throw new Error(`subprocess-e2b: E2B returned invalid terminal pid ${handle.pid}`) + } + const command = `exec /bin/bash ${quoteE2BShellArg(paths.runner)} ${quoteE2BShellArg(stateDir)}\r` + await sandbox.pty.sendInput(handle.pid, Buffer.from(command), signalOpts(spec.signal)) + await waitForBootstrapOutput(outputFilter.ready, completion, spec.signal) + const sessionId = await terminalSessionId(sandbox, handle.pid, controlEnvs, spec.signal) + return new E2BTerminalHandle( + sandbox, + handle, + output, + completion, + sessionId, + controlEnvs, + stateDir, + spec.graceMs, + pollMs, + ) + } catch (error: unknown) { + output.destroy() + let terminalQuiescent = handle === undefined + let stateRemoved = !stateDirectoryCreated + const cleanup = async (): Promise => { + const failures: Error[] = [] + if (!terminalQuiescent && handle !== undefined) { + try { + if (completion === undefined) await handle.kill() + else await rollbackUnpublishedTerminal(sandbox, handle, completion, controlEnvs, spec.graceMs, pollMs) + terminalQuiescent = true + } catch (cleanupError: unknown) { + if (cleanupError instanceof SandboxNotFoundError) terminalQuiescent = true + else failures.push(asError(cleanupError)) + } + } + if (!stateRemoved) { + try { + await sandbox.files.remove(stateDir) + stateRemoved = true + } catch (stateError: unknown) { + if (stateError instanceof FileNotFoundError || stateError instanceof SandboxNotFoundError) stateRemoved = true + else failures.push(asError(stateError)) + } + } + if (failures.length > 0) { + throw new AggregateError(failures, 'subprocess-e2b: terminal setup cleanup did not complete') + } + } + try { + await cleanup() + } catch (cleanupError: unknown) { + // TODO(e2b-terminal-setup-rollback): Retain retry state only if a real + // double failure must be recovered before sandbox disposal or timeout. + throw new AggregateError([asError(error), asError(cleanupError)], asError(error).message) + } + throw error + } +} diff --git a/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts b/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts new file mode 100644 index 0000000000..153b144ccd --- /dev/null +++ b/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts @@ -0,0 +1,1792 @@ +import { once } from 'node:events' +import { Context } from 'cordis' +import { + CommandExitError, + FileNotFoundError, + SandboxNotFoundError, + type CommandHandle, + type CommandResult, + type Sandbox, +} from '@deepseek-ai/dsh-e2b' +import type E2BSandboxService from '@deepseek-ai/dsh-e2b' +import type { SubprocessSpawnSpec } from '@deepseek-ai/dsh-subprocess' +import E2BSubprocessService from '@deepseek-ai/dsh-subprocess-e2b' +import * as E2BSubprocessInvariant from '../src/invariant.ts' +import { E2BBase64Decoder, E2B_OUTPUT_COMPLETE_FRAME, E2BOutputReader } from '../src/output.ts' +import { E2BSubprocessHandle } from '../src/process.ts' +import InvariantService from '@deepseek-ai/dsh-invariants' +import { describe, expect, it, vi } from 'vitest' + +function commandError(exitCode: number): CommandExitError { + return new CommandExitError({ exitCode, stdout: '', stderr: '', error: `exit ${exitCode}` }) +} + +interface StartOptions { + background: true + cwd: string + stdin: boolean + timeoutMs: number + signal?: AbortSignal + envs?: Record + onStdout?: (data: string) => void | Promise + onStderr?: (data: string) => void | Promise +} + +class FakeCommandHandle { + pid = 4242 + readonly sent: Array = [] + closes = 0 + kills = 0 + disconnects = 0 + killError: unknown + killResult = true + disconnectError: unknown + private readonly result = Promise.withResolvers() + private settled = false + + constructor(private readonly onKill: () => void = () => {}) {} + + wait(): Promise { + return this.result.promise + } + + async sendStdin(data: string | Uint8Array): Promise { + this.sent.push(data) + } + + async closeStdin(): Promise { + this.closes += 1 + } + + async kill(): Promise { + this.kills += 1 + if (this.killError !== undefined) throw this.killError + this.onKill() + return this.killResult + } + + async disconnect(): Promise { + this.disconnects += 1 + if (this.disconnectError !== undefined) throw this.disconnectError + } + + succeed(exitCode = 0): void { + if (this.settled) return + this.settled = true + this.result.resolve({ exitCode, stdout: '', stderr: '' }) + } + + fail(exitCode: number): void { + if (this.settled) return + this.settled = true + this.result.reject(commandError(exitCode)) + } + + crash(error: unknown): void { + if (this.settled) return + this.settled = true + this.result.reject(error) + } +} + +class FakeSandbox { + readonly handle: FakeCommandHandle + readonly commandsSeen: string[] = [] + readonly writtenFiles: string[][] = [] + readonly writtenFileData = new Map() + readonly removed: string[] = [] + readonly directories: string[] = [] + startOptions: StartOptions | undefined + backgroundError: unknown + envError: unknown + statusError: unknown + nextRemoveError: unknown + probeError: unknown + signalError: unknown + readonly signalErrors: unknown[] = [] + trapsTerm = false + delaysKill = false + delaysKillCompletion = false + sdkKillStops = true + alive = true + zombieOnly = false + ambient = 'PATH=/ambient/bin\0KEEP=safe\0UNICODE=你好\0NPM_TOKEN=secret\0DSH_STALE=old\0BROKEN\0=bad\0' + environmentHome = '/home/user' + environmentWire: string | undefined + environmentRequest: ((signal: AbortSignal | undefined) => Promise) | undefined + processGroupId = '4242\n' + exitStatus = '' + statusReads = 0 + readonly processGroupReads: string[] = [] + afterStatusRead: (() => void) | undefined + beforeProbe: (() => void) | undefined + afterProbe: (() => void) | undefined + private startGate: Promise | undefined + private openStart: (() => void) | undefined + private processGroupReadGate: Promise | undefined + private openProcessGroupRead: (() => void) | undefined + private signalGate: Promise | undefined + private openSignal: (() => void) | undefined + + constructor() { + this.handle = new FakeCommandHandle(() => { + if (this.sdkKillStops) { + this.alive = false + this.handle.fail(137) + } + }) + } + + deferStart(): void { + const gate = Promise.withResolvers() + this.startGate = gate.promise + this.openStart = () => { gate.resolve(undefined) } + } + + releaseStart(): void { + this.openStart?.() + } + + deferProcessGroupRead(): void { + const gate = Promise.withResolvers() + this.processGroupReadGate = gate.promise + this.openProcessGroupRead = () => { gate.resolve(undefined) } + } + + releaseProcessGroupRead(): void { + this.openProcessGroupRead?.() + } + + deferSignals(): void { + const gate = Promise.withResolvers() + this.signalGate = gate.promise + this.openSignal = () => { gate.resolve(undefined) } + } + + releaseSignals(): void { + this.openSignal?.() + } + + finish(exitCode = 0): void { + this.alive = false + void this.completeOutput().then( + () => { + if (exitCode === 0) this.handle.succeed(0) + else this.handle.fail(exitCode) + }, + (error: unknown) => { this.handle.crash(error) }, + ) + } + + async completeOutput(): Promise { + await Promise.all([ + this.stdoutWire(`${E2B_OUTPUT_COMPLETE_FRAME}\n`), + this.stderrWire(`${E2B_OUTPUT_COMPLETE_FRAME}\n`), + ]) + } + + async stdout(data: string): Promise { + await this.stdoutWire(data.length === 0 ? '' : `${Buffer.from(data).toString('base64')}\n`) + } + + async stderr(data: string): Promise { + await this.stderrWire(data.length === 0 ? '' : `${Buffer.from(data).toString('base64')}\n`) + } + + async stdoutWire(data: string): Promise { + await this.startOptions?.onStdout?.(data) + } + + async stderrWire(data: string): Promise { + await this.startOptions?.onStderr?.(data) + } + + readonly sandbox = { + sandboxId: 'fake', + files: { + makeDir: async (path: string): Promise => { + this.directories.push(path) + return true + }, + write: async (files: Array<{ path: string; data: string }>): Promise => { + this.writtenFiles.push(files.map(file => file.path)) + for (const file of files) this.writtenFileData.set(file.path, file.data) + return files.map(() => ({})) + }, + read: async (path: string): Promise => { + if (!path.endsWith('/exit-code')) { + await this.processGroupReadGate + return this.processGroupReads.shift() ?? this.processGroupId + } + if (this.statusError !== undefined) { + const error = this.statusError + this.statusError = undefined + throw error + } + this.statusReads += 1 + this.afterStatusRead?.() + return this.exitStatus + }, + remove: async (path: string): Promise => { + this.removed.push(path) + if (this.nextRemoveError !== undefined) { + const error = this.nextRemoveError + this.nextRemoveError = undefined + throw error + } + }, + }, + commands: { + run: async (command: string, options?: StartOptions | { signal?: AbortSignal }): Promise => { + this.commandsSeen.push(command) + if (command.includes('env -0 | base64')) { + await this.environmentRequest?.(options?.signal) + if (this.envError !== undefined) throw this.envError + return { + exitCode: 0, + stdout: this.environmentWire ?? [this.environmentHome, this.ambient] + .map(value => Buffer.from(value).toString('base64')) + .join('\n'), + stderr: '', + } + } + if (command.startsWith('set -o pipefail; ps -eo pgid=,stat=')) { + this.beforeProbe?.() + if (options?.signal?.aborted === true) throw new DOMException('aborted', 'AbortError') + if (this.probeError !== undefined) { + const error = this.probeError + this.probeError = undefined + throw error + } + const stdout = this.alive && !this.zombieOnly ? 'live\n' : '' + this.afterProbe?.() + return { exitCode: 0, stdout, stderr: '' } + } + if (command.startsWith('kill -TERM ')) { + await this.signalGate + const error = this.signalErrors.shift() ?? this.signalError + if (error !== undefined) { + if (this.signalErrors.length === 0) this.signalError = undefined + throw error + } + if (!this.trapsTerm) { + this.alive = false + this.handle.fail(143) + } + return { exitCode: 0, stdout: '', stderr: '' } + } + if (command.startsWith('kill -KILL ')) { + await this.signalGate + const error = this.signalErrors.shift() ?? this.signalError + if (error !== undefined) { + if (this.signalErrors.length === 0) this.signalError = undefined + throw error + } + if (!this.delaysKill) this.alive = false + if (!this.delaysKillCompletion) this.handle.fail(137) + return { exitCode: 0, stdout: '', stderr: '' } + } + if ((options as StartOptions | undefined)?.background === true) { + this.startOptions = options as StartOptions + await this.startGate + if (this.backgroundError !== undefined) throw this.backgroundError + return this.handle as unknown as CommandHandle + } + return { exitCode: 0, stdout: '', stderr: '' } + }, + }, + } as unknown as Sandbox +} + +function spec(overrides: Partial = {}): SubprocessSpawnSpec { + return { + argv: ['bash', '-c', 'printf ok'], + cwd: '/workspace', + stdio: { + stdin: 'ignore', + stdout: { maxBytes: 4, spill: { maxBytes: 16 } }, + stderr: { maxBytes: 4 }, + }, + graceMs: 5, + ...overrides, + } +} + +function runtime(fake: FakeSandbox, getSandbox: () => Promise = async () => fake.sandbox): E2BSandboxService { + return { + cwd: '/workspace', + runtimeRoot: '/workspace/.dsh-e2b', + getSandbox, + } as unknown as E2BSandboxService +} + +async function flush(): Promise { + await new Promise(resolve => setTimeout(resolve, 0)) +} + +/** Construct the handle under test with the config default the service would pass. */ +function testHandle( + runtime: ConstructorParameters[0], + spec: ConstructorParameters[1], + stateDir: string, + pollMs = 20, +): E2BSubprocessHandle { + return new E2BSubprocessHandle(runtime, spec, stateDir, pollMs) +} + +describe('E2BOutputReader', () => { + it('decodes base64 across arbitrary callback boundaries and rejects malformed framing', () => { + const decoder = new E2BBase64Decoder() + expect(decoder.push('')).toEqual(Buffer.alloc(0)) + expect(decoder.push('5')).toEqual(Buffer.alloc(0)) + expect(decoder.push('L2')).toEqual(Buffer.alloc(0)) + expect(decoder.push('g\n').toString()).toBe('你') + expect(decoder.push('YQ==\nYg==\n').toString()).toBe('ab') + expect(decoder.push(`${Buffer.from([0, 255]).toString('base64')}\n`)).toEqual(Buffer.from([0, 255])) + expect(decoder.push(`${E2B_OUTPUT_COMPLETE_FRAME}\n`)).toEqual(Buffer.alloc(0)) + decoder.finish() + + expect(() => new E2BBase64Decoder().push('%\n')).toThrow('invalid base64') + expect(() => new E2BBase64Decoder().push('AB==\n')).toThrow('invalid base64') + expect(() => decoder.push(`${E2B_OUTPUT_COMPLETE_FRAME}\n`)).toThrow('duplicate output transport completion') + expect(() => decoder.push('YQ==\n')).toThrow('continued after completion') + const truncated = new E2BBase64Decoder() + truncated.push('YQ') + expect(() => { truncated.finish() }).toThrow('truncated base64') + expect(() => { new E2BBase64Decoder().finish() }).toThrow('incomplete output transport') + const interrupted = new E2BBase64Decoder() + interrupted.push('YQ') + expect(() => { interrupted.finish(false) }).not.toThrow() + }) + + it('keeps a byte-exact tail with independent whole-stream cursors', () => { + const reader = new E2BOutputReader(4, 10, '/remote/spill') + reader.push(Buffer.alloc(0)) + reader.push(Buffer.from('ab')) + reader.push(Buffer.from('cdef')) + expect(reader.size).toBe(6) + expect(reader.readFrom(0)).toEqual({ text: 'cdef', nextOffset: 6, lossy: true, spillPath: '/remote/spill' }) + expect(reader.readFrom(2)).toEqual({ text: 'cdef', nextOffset: 6, lossy: false }) + expect(reader.readFrom(5)).toEqual({ text: 'f', nextOffset: 6, lossy: false }) + expect(reader.readFrom(99)).toEqual({ text: '', nextOffset: 6, lossy: false }) + reader.invalidateSpill() + expect(reader.readFrom(0)).toEqual({ text: 'cdef', nextOffset: 6, lossy: true }) + }) + + it('drops whole head chunks and withholds absent or over-cap spills', () => { + const withoutSpill = new E2BOutputReader(2, undefined, '/unused') + withoutSpill.push(Buffer.from('ab')) + withoutSpill.push(Buffer.from('cd')) + expect(withoutSpill.readFrom(0)).toEqual({ text: 'cd', nextOffset: 4, lossy: true }) + const overCap = new E2BOutputReader(2, 3, '/too-small') + overCap.push(Buffer.from('abcd')) + expect(overCap.readFrom(0)).toEqual({ text: 'cd', nextOffset: 4, lossy: true }) + }) +}) + +describe('E2BSubprocessHandle', () => { + it('starts asynchronously, keeps secrets out of the command, and supports deferred piped stdin/output', async () => { + const fake = new FakeSandbox() + fake.processGroupId = '4343\n' + fake.deferStart() + const handle = testHandle(runtime(fake), spec({ + argv: ['tool', 'argument with spaces'], + stdio: { stdin: 'pipe', stdout: 'pipe', stderr: { maxBytes: 8, spill: { maxBytes: 32 } } }, + env: { + PATH: '/bin', + 'FOO-BAR': 'hyphen-value', + '--split-string': 'literal-value', + DEEPSEEK_API_KEY: 'explicit-secret', + DSH_MODE: 'test', + // The seam's tombstone: an explicit undefined removes the ambient entry. + KEEP: undefined, + }, + }), '/workspace/.dsh-e2b/processes/one') + expect(handle.pid).toBe(-1) + handle.stdin!.write('hello') + handle.stdin!.end() + fake.releaseStart() + await flush() + expect(handle.pid).toBe(4343) + expect(fake.handle.sent.map(value => String(value))).toEqual(['hello']) + expect(fake.handle.closes).toBe(1) + const controlEnvs = fake.startOptions?.envs + expect(controlEnvs?.HOME).toMatch(/^\/\.dsh-e2b-control-/) + expect(controlEnvs).toEqual({ + TERM: 'dumb', + NPM_TOKEN: '', + DSH_STALE: '', + HOME: controlEnvs?.HOME, + }) + const command = fake.commandsSeen.find(value => value.includes('exec "$dsh_e2b_env_bin" -i'))! + expect(command).toContain('"$dsh_e2b_setsid" --wait -- "$dsh_e2b_bash" -c') + expect(command).not.toContain('DEEPSEEK_API_KEY') + expect(command).not.toContain('DSH_MODE') + expect(command).not.toContain('FOO-BAR') + expect(command).not.toContain('explicit-secret') + expect(command).not.toContain('hyphen-value') + expect(command).not.toContain('${!dsh_e2b_name}') + const environmentProbe = fake.commandsSeen.find(value => value.includes('env -0 | base64')) + expect(environmentProbe).toContain('getent passwd "$(id -u)"') + expect(environmentProbe).toContain('test -n "$dsh_e2b_home" -a -d "$dsh_e2b_home"') + expect(environmentProbe).not.toContain('"$PWD"') + expect(command).toContain('mapfile -d') + expect(command).toContain('dsh_e2b_node="$(command -v node)"') + expect(command).toContain('"$dsh_e2b_env_bin" -i "$dsh_e2b_node" -e') + expect(command).toContain('"$dsh_e2b_env_bin" -i -- "${dsh_e2b_env[@]}" "$@"') + expect(command).toContain('exec "$dsh_e2b_env_bin" -i -- "${dsh_e2b_env[@]}"') + expect(command).toContain('>&2 2>/dev/null') + expect(command).not.toContain('2>/dev/null >&2') + expect(command).toContain('base64') + expect(fake.writtenFiles[0]).toEqual([ + '/workspace/.dsh-e2b/processes/one/pid', + '/workspace/.dsh-e2b/processes/one/exit-code', + '/workspace/.dsh-e2b/processes/one/environment', + '/workspace/.dsh-e2b/processes/one/stderr.log', + ]) + expect(fake.writtenFileData.get('/workspace/.dsh-e2b/processes/one/environment')).toBe( + 'PATH=/bin\0UNICODE=你好\0HOME=/home/user\0FOO-BAR=hyphen-value\0--split-string=literal-value\0DEEPSEEK_API_KEY=explicit-secret\0DSH_MODE=test\0', + ) + + let piped = '' + handle.stdout!.on('data', (chunk) => { piped += String(chunk) }) + await fake.stdout('pipe-data') + await fake.stderr('err') + fake.finish() + await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) + expect(piped).toBe('pipe-data') + expect(handle.collected.stderr!.readFrom(0)).toMatchObject({ text: 'err', lossy: false }) + expect(fake.removed).toContain('/workspace/.dsh-e2b/processes/one/stderr.log') + await expect(handle.waitForExit()).resolves.toBe(true) + }) + + it('rejects an unrepresentable graceMs before any remote work', () => { + const ctx = new Context() + const service = Object.create(E2BSubprocessService.prototype) as E2BSubprocessService + Reflect.set(service, 'disposing', false) + Reflect.set(service, 'ctx', ctx) + for (const graceMs of [0, -1, Number.NaN, Number.POSITIVE_INFINITY]) { + expect(() => service.spawn(spec({ graceMs }))).toThrow('graceMs must be a positive finite number') + void expect(service.spawnTerminal({ + argv: ['bash'], cwd: '/w', rows: 24, cols: 80, graceMs, + })).rejects.toThrow('graceMs must be a positive finite number') + } + }) + + it('rejects malformed environment entries before command start', async () => { + for (const env of [{ 'BAD=NAME': 'x' }, { BAD: 'x\0INJECTED=1' }]) { + const fake = new FakeSandbox() + const handle = testHandle(runtime(fake), spec({ env }), '/runtime/invalid-environment') + await expect(handle.done).rejects.toThrow('environment entries') + expect(fake.startOptions).toBeUndefined() + expect(fake.removed).toContain('/runtime/invalid-environment') + } + }) + + it('preserves UTF-8 bytes when the ASCII transport is split across callbacks', async () => { + const fake = new FakeSandbox() + const handle = testHandle(runtime(fake), spec({ + stdio: { stdin: 'ignore', stdout: 'pipe', stderr: { maxBytes: 4 } }, + }), '/runtime/split-utf8') + await flush() + const chunks: Buffer[] = [] + handle.stdout!.on('data', (chunk: Buffer) => { chunks.push(chunk) }) + for (const character of `${Buffer.from('A你好B').toString('base64')}\n`) { + await fake.stdoutWire(character) + } + fake.finish() + await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) + expect(Buffer.concat(chunks).toString('utf8')).toBe('A你好B') + }) + + it('rejects malformed output transport without confusing it with a consumer sink failure', async () => { + const fake = new FakeSandbox() + const handle = testHandle(runtime(fake), spec(), '/runtime/malformed-output') + await flush() + await fake.stdoutWire('%\n') + fake.finish() + await expect(handle.done).rejects.toThrow('invalid base64 output transport') + + const stderrFake = new FakeSandbox() + const stderrHandle = testHandle(runtime(stderrFake), spec(), '/runtime/malformed-stderr') + await flush() + await stderrFake.stderrWire('%\n') + stderrFake.finish() + await expect(stderrHandle.done).rejects.toThrow('invalid base64 output transport') + }) + + it('rejects a naturally completed command whose encoder omits its completion frame', async () => { + const fake = new FakeSandbox() + const handle = testHandle(runtime(fake), spec(), '/runtime/incomplete-output') + await flush() + fake.alive = false + fake.handle.succeed(0) + await expect(handle.done).rejects.toThrow('incomplete output transport') + }) + + it('bounds descendant-held output draining and withholds the incomplete spill', async () => { + const fake = new FakeSandbox() + const handle = testHandle(runtime(fake), spec({ graceMs: 5 }), '/runtime/drain-bound') + await flush() + await fake.stdout('leader-output') + fake.exitStatus = '0\n' + + await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) + expect(fake.handle.disconnects).toBe(1) + expect(handle.collected.stdout?.readFrom(0)).toEqual({ + text: 'tput', + nextOffset: 13, + lossy: true, + }) + expect(fake.removed).toContain('/runtime/drain-bound/stdout.log') + + handle.terminate() + await expect(handle.waitForExit()).resolves.toBe(true) + }) + + it('releases an inherited-output callback blocked on host backpressure at drain expiry', async () => { + const fake = new FakeSandbox() + const written: string[] = [] + const stdoutWrite = vi.spyOn(process.stdout, 'write').mockImplementation(((chunk: Uint8Array) => { + written.push(Buffer.from(chunk).toString()) + return false + }) as typeof process.stdout.write) + try { + const handle = testHandle(runtime(fake), spec({ + graceMs: 5, + stdio: { stdin: 'ignore', stdout: 'inherit', stderr: { maxBytes: 4 } }, + }), '/runtime/inherit-backpressure') + await flush() + let callbackSettled = false + const blocked = fake.stdout('blocked bytes').then(() => { callbackSettled = true }) + await flush() + expect(callbackSettled).toBe(false) + fake.exitStatus = '0\n' + + await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) + await blocked + expect(callbackSettled).toBe(true) + expect(written.join('')).toBe('blocked bytes') + expect(fake.handle.disconnects).toBe(1) + + handle.terminate() + await expect(handle.waitForExit()).resolves.toBe(true) + } finally { + stdoutWrite.mockRestore() + } + }) + + it('waits for lossless raw-pipe output after the direct status is published', async () => { + const fake = new FakeSandbox() + const handle = testHandle(runtime(fake), spec({ + graceMs: 1, + stdio: { stdin: 'ignore', stdout: 'pipe', stderr: { maxBytes: 4 } }, + }), '/runtime/pipe-drain') + let output = '' + handle.stdout!.on('data', (chunk) => { output += String(chunk) }) + await flush() + fake.exitStatus = '0\n' + + let settled = false + void handle.done.then(() => { settled = true }) + await new Promise(resolve => setTimeout(resolve, 50)) + expect(settled).toBe(false) + expect(fake.handle.disconnects).toBe(0) + expect(fake.statusReads).toBe(0) + + await fake.stdout('complete protocol frame') + fake.finish() + await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) + expect(output).toBe('complete protocol frame') + expect(fake.statusReads).toBe(1) + }) + + it('accepts clean encoder completion inside the output-drain grace', async () => { + const fake = new FakeSandbox() + const handle = testHandle(runtime(fake), spec({ graceMs: 100 }), '/runtime/drain-complete') + await flush() + fake.exitStatus = '0\n' + fake.afterStatusRead = () => { + fake.afterStatusRead = undefined + setTimeout(() => { fake.finish() }, 0) + } + + await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) + expect(fake.handle.disconnects).toBe(0) + }) + + it('preserves a published exit code when requested termination outlives output draining', async () => { + const fake = new FakeSandbox() + fake.trapsTerm = true + fake.delaysKill = true + fake.delaysKillCompletion = true + fake.sdkKillStops = false + const handle = testHandle(runtime(fake), spec({ graceMs: 5 }), '/runtime/drain-signal') + await flush() + + handle.terminate() + await vi.waitFor(() => { expect(fake.commandsSeen).toContain('kill -KILL -- -4242') }) + fake.exitStatus = '0\n' + + await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) + expect(fake.handle.disconnects).toBe(1) + fake.alive = false + handle.terminate() + await expect(handle.waitForExit()).resolves.toBe(true) + }) + + it('preserves a published nonzero exit code when termination settles the SDK inside the drain grace', async () => { + const fake = new FakeSandbox() + const handle = testHandle(runtime(fake), spec({ graceMs: 100 }), '/runtime/drain-signal-settled') + await flush() + fake.exitStatus = '7\n' + fake.afterStatusRead = () => { + fake.afterStatusRead = undefined + handle.terminate() + } + + await expect(handle.done).resolves.toEqual({ exitCode: 7, signal: null }) + await expect(handle.waitForExit()).resolves.toBe(true) + }) + + it('rejects an invalid direct-command exit status', async () => { + const fake = new FakeSandbox() + const handle = testHandle(runtime(fake), spec(), '/runtime/invalid-status') + await flush() + fake.exitStatus = '999\n' + await expect(handle.done).rejects.toThrow('invalid exit code') + handle.terminate() + await expect(handle.waitForExit()).resolves.toBe(true) + }) + + it('rolls back a published process group before rejecting a monitoring failure', async () => { + const fake = new FakeSandbox() + fake.statusError = new Error('status transport failed') + const handle = testHandle(runtime(fake), spec(), '/runtime/status-failure') + + await expect(handle.done).rejects.toThrow('status transport failed') + expect(fake.commandsSeen).toContain('kill -TERM -- -4242') + expect(fake.alive).toBe(false) + await expect(handle.waitForExit()).resolves.toBe(true) + + const failed = new FakeSandbox() + failed.statusError = new Error('status transport failed') + failed.signalErrors.push(new Error('TERM transport failed'), new Error('KILL transport failed')) + failed.handle.killError = new Error('SDK kill failed') + const retained = testHandle(runtime(failed), spec({ graceMs: 1 }), '/runtime/status-cleanup-failure') + + await expect(retained.done).rejects.toThrow( + 'command monitoring failed and process-group rollback did not reach quiescence', + ) + expect(failed.alive).toBe(true) + failed.handle.killError = undefined + retained.terminate() + await expect(retained.waitForExit()).resolves.toBe(true) + + // A state-cleanup failure on top preserves the rollback failure instead of + // re-aggregating only the original monitoring error. + const triple = new FakeSandbox() + triple.statusError = new Error('status transport failed') + triple.signalErrors.push(new Error('TERM transport failed'), new Error('KILL transport failed')) + triple.handle.killError = new Error('SDK kill failed') + triple.nextRemoveError = new Error('state cleanup failed') + const tripleHandle = testHandle(runtime(triple), spec({ graceMs: 1 }), '/runtime/triple-failure') + const failure = await tripleHandle.done.catch((error: unknown) => error as AggregateError) + expect(failure).toBeInstanceOf(AggregateError) + expect((failure as AggregateError).message).toContain('private state cleanup failed') + const nested = (failure as AggregateError).errors[0] as AggregateError + expect(nested.message).toContain('rollback did not reach quiescence') + triple.handle.killError = undefined + tripleHandle.terminate() + await expect(tripleHandle.waitForExit()).resolves.toBe(true) + }) + + it('surfaces deferred piped-stdin write and close failures as stream errors', async () => { + const writeFake = new FakeSandbox() + writeFake.deferStart() + vi.spyOn(writeFake.handle, 'sendStdin').mockRejectedValueOnce('stdin rejected') + const writeHandle = testHandle(runtime(writeFake), spec({ + stdio: { stdin: 'pipe', stdout: { maxBytes: 4 }, stderr: { maxBytes: 4 } }, + }), '/runtime/stdin-write-error') + const writeError = once(writeHandle.stdin!, 'error') + writeHandle.stdin!.write('input') + writeFake.releaseStart() + await expect(writeError).resolves.toMatchObject([{ message: 'stdin rejected' }]) + writeFake.finish() + await writeHandle.done + + const closeFake = new FakeSandbox() + vi.spyOn(closeFake.handle, 'closeStdin').mockRejectedValueOnce(new Error('close rejected')) + const closeHandle = testHandle(runtime(closeFake), spec({ + stdio: { stdin: 'pipe', stdout: { maxBytes: 4 }, stderr: { maxBytes: 4 } }, + }), '/runtime/stdin-close-error') + await flush() + const closeError = once(closeHandle.stdin!, 'error') + closeHandle.stdin!.end() + await expect(closeError).resolves.toMatchObject([{ message: 'close rejected' }]) + closeFake.finish() + await closeHandle.done + }) + + it('collects bounded tails, retains valid spills, and maps natural nonzero exits', async () => { + const fake = new FakeSandbox() + const handle = testHandle(runtime(fake), spec({ + stdio: { + stdin: { data: 'batch' }, + stdout: { maxBytes: 4, spill: { maxBytes: 16 } }, + stderr: { maxBytes: 3 }, + }, + }), '/runtime/two') + await flush() + await fake.stdout('abcdef') + await fake.stderr('12345') + fake.finish(7) + await expect(handle.done).resolves.toEqual({ exitCode: 7, signal: null }) + expect(fake.handle.sent).toEqual(['batch']) + expect(fake.handle.closes).toBe(1) + expect(handle.collected.stdout!.readFrom(0)).toEqual({ + text: 'cdef', + nextOffset: 6, + lossy: true, + spillPath: '/runtime/two/stdout.log', + }) + expect(handle.collected.stderr!.readFrom(0)).toEqual({ text: '345', nextOffset: 5, lossy: true }) + expect(fake.removed).not.toContain('/runtime/two/stdout.log') + }) + + it('removes a spill once the complete stream exceeds its cap', async () => { + const fake = new FakeSandbox() + const handle = testHandle(runtime(fake), spec({ + stdio: { stdin: 'ignore', stdout: { maxBytes: 2, spill: { maxBytes: 3 } }, stderr: 'inherit' }, + }), '/runtime/oversize') + await flush() + await fake.stdout('abcd') + await fake.stderr('') + fake.finish() + await handle.done + expect(handle.collected.stdout!.readFrom(0)).toEqual({ text: 'cd', nextOffset: 4, lossy: true }) + expect(fake.removed).toContain('/runtime/oversize/stdout.log') + const command = fake.commandsSeen.find(value => value.includes('dsh_e2b_tee='))! + expect(command).toContain('"$dsh_e2b_head" -c 3') + expect(command).toContain('/runtime/oversize/stdout.log') + expect(command).toContain('"$dsh_e2b_tee" --output-error=warn-nopipe') + expect(command).not.toContain('tee -a') + }) + + it('contains remote spill-removal failures and routes empty inherited output', async () => { + const fake = new FakeSandbox() + fake.nextRemoveError = new Error('already removed') + const handle = testHandle(runtime(fake), spec({ + stdio: { stdin: 'ignore', stdout: 'inherit', stderr: { maxBytes: 4, spill: { maxBytes: 8 } } }, + }), '/runtime/remove-error') + await flush() + await fake.stdout('') + await fake.stderr('') + fake.finish() + await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) + expect(fake.removed).toContain('/runtime/remove-error/stderr.log') + }) + + it('terminates a process group with TERM and reports the signal outcome', async () => { + const fake = new FakeSandbox() + const handle = testHandle(runtime(fake), spec(), '/runtime/term') + await flush() + handle.terminate() + handle.terminate() + await expect(handle.done).resolves.toEqual({ exitCode: null, signal: 'SIGTERM' }) + await expect(handle.waitForExit()).resolves.toBe(true) + expect(fake.commandsSeen).toContain('kill -TERM -- -4242') + expect(fake.commandsSeen).not.toContain('kill -KILL -- -4242') + const signals = fake.commandsSeen.filter(command => command.startsWith('kill -')).length + fake.alive = true + handle.terminate() + await flush() + expect(fake.alive).toBe(true) + expect(fake.commandsSeen.filter(command => command.startsWith('kill -'))).toHaveLength(signals) + }) + + it('makes termination a permanent no-op after natural quiescence is observed', async () => { + const fake = new FakeSandbox() + const handle = testHandle(runtime(fake), spec(), '/runtime/natural-quiescence') + await flush() + fake.finish() + await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) + await expect(handle.waitForExit()).resolves.toBe(true) + + const signals = fake.commandsSeen.filter(command => command.startsWith('kill -')).length + fake.alive = true + handle.terminate() + await flush() + expect(fake.alive).toBe(true) + expect(fake.commandsSeen.filter(command => command.startsWith('kill -'))).toHaveLength(signals) + }) + + it('treats a zombie-only process group as quiescent', async () => { + const fake = new FakeSandbox() + fake.zombieOnly = true + const handle = testHandle(runtime(fake), spec(), '/runtime/zombie-quiescence') + await flush() + + await expect(handle.waitForExit()).resolves.toBe(true) + expect(fake.commandsSeen).toContain( + 'set -o pipefail; ps -eo pgid=,stat= | awk \'$1 == 4242 && $2 !~ /^[ZXx]/ { live=1 } END { if (live) print "live" }\'', + ) + fake.finish() + await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) + }) + + it('keeps proven quiescence after a concurrent termination transport fails', async () => { + const fake = new FakeSandbox() + fake.signalErrors.push(new Error('TERM transport failed'), new Error('KILL transport failed')) + fake.handle.killError = new Error('SDK kill failed') + fake.deferSignals() + const handle = testHandle(runtime(fake), spec(), '/runtime/quiescent-race') + await flush() + + handle.terminate() + await vi.waitFor(() => { expect(fake.commandsSeen).toContain('kill -TERM -- -4242') }) + fake.alive = false + await expect(handle.waitForExit()).resolves.toBe(true) + + fake.probeError = new Error('post-quiescence probe failed') + fake.releaseSignals() + await vi.waitFor(() => { expect(fake.handle.kills).toBe(1) }) + fake.finish() + await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) + const signals = fake.commandsSeen.filter(command => command.startsWith('kill -')).length + handle.terminate() + await expect(handle.waitForExit()).resolves.toBe(true) + expect(fake.commandsSeen.filter(command => command.startsWith('kill -'))).toHaveLength(signals) + }) + + it('escalates a TERM-trapping process group to KILL and uses the SDK kill as fallback', async () => { + const fake = new FakeSandbox() + fake.trapsTerm = true + fake.handle.killError = new Error('already gone') + const handle = testHandle(runtime(fake), spec({ graceMs: 1 }), '/runtime/kill') + await flush() + handle.terminate() + await expect(handle.done).resolves.toEqual({ exitCode: null, signal: 'SIGKILL' }) + await expect(handle.waitForExit()).resolves.toBe(true) + expect(fake.commandsSeen).toContain('kill -KILL -- -4242') + expect(fake.handle.kills).toBe(1) + }) + + it('keeps force cleanup retryable until quiescence is proven', async () => { + const fake = new FakeSandbox() + fake.trapsTerm = true + fake.delaysKill = true + fake.delaysKillCompletion = true + fake.sdkKillStops = false + const handle = testHandle(runtime(fake), spec({ graceMs: 1 }), '/runtime/termination-fence') + await flush() + handle.terminate() + await vi.waitFor(() => { expect(fake.handle.kills).toBe(1) }) + await expect(handle.waitForExit()).rejects.toThrow('remained live after force termination') + expect(fake.alive).toBe(true) + + fake.delaysKill = false + fake.delaysKillCompletion = false + fake.sdkKillStops = true + handle.terminate() + await expect(handle.waitForExit()).resolves.toBe(true) + await expect(handle.done).resolves.toEqual({ exitCode: null, signal: 'SIGKILL' }) + }) + + it('honors termination requested before asynchronous startup finishes', async () => { + const fake = new FakeSandbox() + fake.deferStart() + const handle = testHandle(runtime(fake), spec(), '/runtime/deferred-kill') + handle.terminate() + fake.releaseStart() + await expect(handle.done).resolves.toEqual({ exitCode: null, signal: 'SIGTERM' }) + }) + + it('aborts a stalled preparation request before reporting startup quiescence', async () => { + const fake = new FakeSandbox() + let preparationSignal: AbortSignal | undefined + fake.environmentRequest = async (signal) => { + preparationSignal = signal + await new Promise((_resolve, reject) => { + const rejectAbort = (): void => { + const reason: unknown = signal?.reason + reject(reason instanceof Error ? reason : new Error(String(reason))) + } + if (signal?.aborted === true) { + rejectAbort() + return + } + signal?.addEventListener('abort', rejectAbort, { once: true }) + }) + } + const handle = testHandle(runtime(fake), spec(), '/runtime/stalled-preparation') + await vi.waitFor(() => { expect(preparationSignal).toBeDefined() }) + + handle.terminate() + await expect(handle.done).resolves.toEqual({ exitCode: null, signal: 'SIGTERM' }) + await expect(handle.waitForExit()).resolves.toBe(true) + expect(preparationSignal?.aborted).toBe(true) + expect(fake.startOptions).toBeUndefined() + }) + + it('kills through the provisional SDK handle before process-group publication', async () => { + const fake = new FakeSandbox() + fake.deferProcessGroupRead() + fake.signalErrors.push(commandError(1), commandError(1)) + const handle = testHandle(runtime(fake), spec(), '/runtime/pre-publication-kill') + await vi.waitFor(() => { expect(fake.startOptions).toBeDefined() }) + + handle.terminate() + await vi.waitFor(() => { expect(fake.handle.kills).toBe(1) }) + expect(fake.alive).toBe(false) + await expect(handle.waitForExit()).resolves.toBe(true) + + fake.releaseProcessGroupRead() + await expect(handle.done).resolves.toEqual({ exitCode: null, signal: 'SIGKILL' }) + }) + + it('does not treat an unsuccessful SDK fallback as provisional group quiescence', async () => { + const fake = new FakeSandbox() + fake.deferProcessGroupRead() + fake.trapsTerm = true + fake.delaysKill = true + fake.delaysKillCompletion = true + fake.sdkKillStops = false + fake.handle.killResult = false + const handle = testHandle(runtime(fake), spec({ graceMs: 1 }), '/runtime/provisional-sdk-false') + await vi.waitFor(() => { expect(fake.startOptions).toBeDefined() }) + + handle.terminate() + await vi.waitFor(() => { expect(fake.handle.kills).toBe(1) }) + await expect(handle.waitForExit()).rejects.toThrow('remained live after force termination') + fake.alive = false + handle.terminate() + await expect(handle.waitForExit()).resolves.toBe(true) + fake.releaseProcessGroupRead() + fake.finish() + await handle.done + }) + + it('bounds a quiescence observer while provisional termination is awaiting the controller', async () => { + const fake = new FakeSandbox() + fake.deferProcessGroupRead() + const reconnect = Promise.withResolvers() + let calls = 0 + const delayedRuntime = runtime(fake, async () => { + calls += 1 + return calls === 1 ? fake.sandbox : await reconnect.promise + }) + const handle = testHandle(delayedRuntime, spec(), '/runtime/pre-publication-observer') + await vi.waitFor(() => { expect(fake.startOptions).toBeDefined() }) + handle.terminate() + + const controller = new AbortController() + const waiting = handle.waitForExit(controller.signal) + await flush() + controller.abort() + await expect(waiting).resolves.toBe(false) + + reconnect.resolve(fake.sandbox) + await expect(handle.waitForExit()).resolves.toBe(true) + fake.releaseProcessGroupRead() + await handle.done + }) + + it('proves a provisional group exit when the SDK kill fallback fails', async () => { + const fake = new FakeSandbox() + fake.deferProcessGroupRead() + fake.trapsTerm = true + fake.handle.killError = new Error('SDK kill unavailable') + const handle = testHandle(runtime(fake), spec({ graceMs: 1 }), '/runtime/pre-publication-group-kill') + await vi.waitFor(() => { expect(fake.startOptions).toBeDefined() }) + + handle.terminate() + await expect(handle.waitForExit()).resolves.toBe(true) + fake.releaseProcessGroupRead() + await expect(handle.done).resolves.toEqual({ exitCode: null, signal: 'SIGKILL' }) + }) + + it('reports failed provisional group and SDK force transports', async () => { + const fake = new FakeSandbox() + fake.deferProcessGroupRead() + fake.signalErrors.push(new Error('TERM transport failed'), new Error('KILL transport failed')) + fake.handle.killError = new Error('SDK kill failed') + const handle = testHandle(runtime(fake), spec({ graceMs: 1 }), '/runtime/pre-publication-failure') + await vi.waitFor(() => { expect(fake.startOptions).toBeDefined() }) + + handle.terminate() + await expect(handle.waitForExit()).rejects.toThrow('remained live after force termination') + + fake.handle.killError = undefined + handle.terminate() + await expect(handle.waitForExit()).resolves.toBe(true) + fake.releaseProcessGroupRead() + await handle.done + + const absentGroup = new FakeSandbox() + absentGroup.deferProcessGroupRead() + absentGroup.signalErrors.push(commandError(1), commandError(1)) + absentGroup.handle.killError = new Error('SDK kill failed without a provisional group') + const absentHandle = testHandle( + runtime(absentGroup), + spec({ graceMs: 1 }), + '/runtime/pre-publication-absent-group', + ) + await vi.waitFor(() => { expect(absentGroup.startOptions).toBeDefined() }) + absentHandle.terminate() + await expect(absentHandle.waitForExit()).rejects.toThrow('remained live after force termination') + absentGroup.handle.killError = undefined + absentHandle.terminate() + await expect(absentHandle.waitForExit()).resolves.toBe(true) + absentGroup.releaseProcessGroupRead() + await absentHandle.done + + const optimisticSdk = new FakeSandbox() + optimisticSdk.deferProcessGroupRead() + optimisticSdk.signalErrors.push(commandError(1), commandError(1)) + optimisticSdk.sdkKillStops = false + const optimisticHandle = testHandle( + runtime(optimisticSdk), + spec({ graceMs: 1 }), + '/runtime/pre-publication-optimistic-sdk', + ) + await vi.waitFor(() => { expect(optimisticSdk.startOptions).toBeDefined() }) + optimisticHandle.terminate() + await expect(optimisticHandle.waitForExit()).rejects.toThrow('remained live after force termination') + optimisticHandle.terminate() + await expect(optimisticHandle.waitForExit()).resolves.toBe(true) + optimisticSdk.releaseProcessGroupRead() + await optimisticHandle.done + }) + + it('honors an already-aborted signal when constructing the asynchronous handle directly', async () => { + const fake = new FakeSandbox() + const handle = testHandle(runtime(fake), spec({ signal: AbortSignal.abort('stop') }), '/runtime/pre-aborted') + await expect(handle.done).resolves.toEqual({ exitCode: null, signal: 'SIGTERM' }) + }) + + it('reacts to a signal that aborts after the remote command has started', async () => { + const fake = new FakeSandbox() + const controller = new AbortController() + const handle = testHandle(runtime(fake), spec({ signal: controller.signal }), '/runtime/live-abort') + await flush() + controller.abort('stop') + await expect(handle.done).resolves.toEqual({ exitCode: null, signal: 'SIGTERM' }) + }) + + it('can terminate a surviving process group after the command leader settles', async () => { + const fake = new FakeSandbox() + const handle = testHandle(runtime(fake), spec(), '/runtime/surviving-group') + await flush() + await fake.completeOutput() + fake.handle.succeed(0) + await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) + expect(fake.alive).toBe(true) + + handle.terminate() + await flush() + const signaled = fake.commandsSeen.includes('kill -TERM -- -4242') + if (!signaled) fake.finish() + await expect(handle.waitForExit()).resolves.toBe(true) + expect(signaled).toBe(true) + }) + + it('bounds waitForExit while startup or a live group is pending', async () => { + const fake = new FakeSandbox() + fake.deferStart() + const handle = testHandle(runtime(fake), spec(), '/runtime/wait') + const beforeStart = new AbortController() + const pending = handle.waitForExit(beforeStart.signal) + beforeStart.abort() + await expect(pending).resolves.toBe(false) + await expect(handle.waitForExit(AbortSignal.abort())).resolves.toBe(false) + fake.releaseStart() + await flush() + const live = new AbortController() + const liveWait = handle.waitForExit(live.signal) + live.abort() + await expect(liveWait).resolves.toBe(false) + fake.finish() + await handle.done + + const terminatingFake = new FakeSandbox() + terminatingFake.deferStart() + const terminating = testHandle(runtime(terminatingFake), spec(), '/runtime/wait-termination-start') + terminating.terminate() + const beforeHandle = new AbortController() + const handlePending = terminating.waitForExit(beforeHandle.signal) + beforeHandle.abort() + await expect(handlePending).resolves.toBe(false) + terminatingFake.releaseStart() + await terminating.done + }) + + it('bounds both sides of the liveness-poll abort race', async () => { + const fake = new FakeSandbox() + const handle = testHandle(runtime(fake), spec(), '/runtime/poll-abort') + await flush() + + const beforeTick = new AbortController() + fake.afterProbe = () => { beforeTick.abort(); fake.afterProbe = undefined } + await expect(handle.waitForExit(beforeTick.signal)).resolves.toBe(false) + + const duringTick = new AbortController() + fake.afterProbe = () => { + fake.afterProbe = undefined + setTimeout(() => { duringTick.abort() }, 0) + } + await expect(handle.waitForExit(duringTick.signal)).resolves.toBe(false) + + const duringProbe = new AbortController() + fake.beforeProbe = () => { duringProbe.abort(); fake.beforeProbe = undefined } + await expect(handle.waitForExit(duringProbe.signal)).resolves.toBe(false) + + let racedAbort = false + const raceSignal = { + get aborted() { return racedAbort }, + addEventListener: () => { racedAbort = true }, + removeEventListener: () => {}, + } as unknown as AbortSignal + await expect(handle.waitForExit(raceSignal)).resolves.toBe(false) + fake.finish() + await handle.done + }) + + it('observes a live group across one successful bounded poll', async () => { + const fake = new FakeSandbox() + const handle = testHandle(runtime(fake), spec(), '/runtime/poll-success') + await flush() + setTimeout(() => { fake.finish() }, 1) + await expect(handle.waitForExit(new AbortController().signal)).resolves.toBe(true) + await handle.done + }) + + it('treats startup failure as no live tree and contains readiness rejection', async () => { + const fake = new FakeSandbox() + fake.backgroundError = new Error('start failed') + const handle = testHandle(runtime(fake), spec(), '/runtime/fail') + await expect(handle.done).rejects.toThrow('start failed') + expect(handle.pid).toBe(-1) + expect(fake.removed).toContain('/runtime/fail/environment') + expect(fake.removed).toContain('/runtime/fail') + await expect(handle.waitForExit()).resolves.toBe(true) + handle.terminate() + + const unavailableHandle = testHandle( + runtime(new FakeSandbox(), async () => { throw new Error('sandbox unavailable') }), + spec(), + '/runtime/unavailable-start', + ) + await expect(unavailableHandle.done).rejects.toThrow('sandbox unavailable') + await expect(unavailableHandle.waitForExit()).resolves.toBe(true) + + const envFailure = new FakeSandbox() + envFailure.envError = new Error('ambient lookup failed') + const envHandle = testHandle(runtime(envFailure), spec(), '/runtime/env-failure') + await expect(envHandle.done).rejects.toThrow('ambient lookup failed') + expect(envFailure.removed).toEqual([]) + + const expectEnvironmentFailure = async (name: string, wire: string, message: string): Promise => { + const fake = new FakeSandbox() + fake.environmentWire = wire + const failed = testHandle(runtime(fake), spec(), `/runtime/${name}`) + await expect(failed.done).rejects.toThrow(message) + } + const encodedEnvironment = Buffer.from('PATH=/bin\0').toString('base64') + const encodedHome = Buffer.from('/home/user').toString('base64') + await expectEnvironmentFailure('malformed-frame', '%', 'invalid base64') + await expectEnvironmentFailure('malformed-base64', `${encodedHome}\n%`, 'invalid base64') + await expectEnvironmentFailure( + 'invalid-utf8-home', + `${Buffer.from([0xff]).toString('base64')}\n${encodedEnvironment}`, + 'not valid UTF-8', + ) + await expectEnvironmentFailure( + 'invalid-utf8-environment', + `${encodedHome}\n${Buffer.from([0xff]).toString('base64')}`, + 'not valid UTF-8', + ) + await expectEnvironmentFailure( + 'relative-home', + `${Buffer.from('home/user').toString('base64')}\n${encodedEnvironment}`, + 'remote login home is invalid', + ) + await expectEnvironmentFailure( + 'nul-home', + `${Buffer.from('/home/user\0tail').toString('base64')}\n${encodedEnvironment}`, + 'remote login home is invalid', + ) + + const cleanupFailure = new FakeSandbox() + cleanupFailure.backgroundError = new Error('start failed before credential consumption') + cleanupFailure.nextRemoveError = new Error('credential cleanup failed') + const cleanupHandle = testHandle(runtime(cleanupFailure), spec(), '/runtime/cleanup-failure') + await expect(cleanupHandle.done).rejects.toThrow('command failed and private state cleanup failed') + + const absentState = new FakeSandbox() + absentState.backgroundError = new Error('start failed after external cleanup') + absentState.nextRemoveError = new FileNotFoundError('already removed') + const absentHandle = testHandle(runtime(absentState), spec(), '/runtime/absent-state') + await expect(absentHandle.done).rejects.toThrow('start failed after external cleanup') + }) + + it('bounds a readiness rejection with a still-live caller signal', async () => { + const fake = new FakeSandbox() + fake.deferStart() + fake.backgroundError = new Error('start failed') + const handle = testHandle(runtime(fake), spec(), '/runtime/fail-with-signal') + const waiting = handle.waitForExit(new AbortController().signal) + fake.releaseStart() + await expect(handle.done).rejects.toThrow('start failed') + await expect(waiting).resolves.toBe(true) + }) + + it('propagates an unavailable sandbox unless the caller aborts the wait', async () => { + const fake = new FakeSandbox() + let calls = 0 + const unavailable = runtime(fake, async () => { + calls += 1 + if (calls === 1) return fake.sandbox + throw new Error('connection unavailable') + }) + const handle = testHandle(unavailable, spec(), '/runtime/unavailable') + await flush() + await expect(handle.waitForExit()).rejects.toThrow('connection unavailable') + fake.finish() + await handle.done + }) + + it('returns false when the caller aborts while reconnecting for liveness', async () => { + const fake = new FakeSandbox() + const reconnect = Promise.withResolvers() + let calls = 0 + const unavailable = runtime(fake, async () => { + calls += 1 + return calls === 1 ? fake.sandbox : await reconnect.promise + }) + const handle = testHandle(unavailable, spec(), '/runtime/reconnect-abort') + await flush() + const controller = new AbortController() + const waiting = handle.waitForExit(controller.signal) + await flush() + controller.abort() + reconnect.reject(new Error('connection unavailable')) + await expect(waiting).resolves.toBe(false) + fake.finish() + await handle.done + }) + + it('returns false when a liveness request itself is aborted and surfaces other probe failures', async () => { + const fake = new FakeSandbox() + const handle = testHandle(runtime(fake), spec(), '/runtime/probe') + await flush() + const controller = new AbortController() + controller.abort() + await expect(handle.waitForExit(controller.signal)).resolves.toBe(false) + fake.probeError = new Error('probe failed') + await expect(handle.waitForExit()).rejects.toThrow('probe failed') + fake.finish() + await handle.done + }) + + it('treats a timeout-killed sandbox as quiescent during liveness probing', async () => { + const fake = new FakeSandbox() + const handle = testHandle(runtime(fake), spec(), '/runtime/expired-sandbox') + await flush() + fake.finish() + await handle.done + fake.probeError = new SandboxNotFoundError('sandbox expired') + + await expect(handle.waitForExit()).resolves.toBe(true) + }) + + it('treats a missing sandbox handle as quiescent during liveness acquisition', async () => { + const fake = new FakeSandbox() + let calls = 0 + const handle = testHandle(runtime(fake, async () => { + calls += 1 + if (calls === 1) return fake.sandbox + throw new SandboxNotFoundError('sandbox expired') + }), spec(), '/runtime/expired-acquisition') + await flush() + + await expect(handle.waitForExit()).resolves.toBe(true) + await fake.completeOutput() + fake.alive = false + fake.handle.succeed(0) + await handle.done + }) + + it('treats sandbox loss during termination as quiescent', async () => { + const fake = new FakeSandbox() + let calls = 0 + const handle = testHandle(runtime(fake, async () => { + calls += 1 + if (calls === 1) return fake.sandbox + throw new SandboxNotFoundError('sandbox expired') + }), spec(), '/runtime/expired-termination') + await flush() + await fake.completeOutput() + + handle.terminate() + await expect(handle.waitForExit()).resolves.toBe(true) + fake.alive = false + fake.handle.succeed(0) + await handle.done + }) + + it('makes batch stdin close failures best-effort', async () => { + const fake = new FakeSandbox() + vi.spyOn(fake.handle, 'sendStdin').mockRejectedValueOnce(new Error('closed')) + const handle = testHandle(runtime(fake), spec({ + stdio: { stdin: { data: 'ignored' }, stdout: { maxBytes: 4 }, stderr: { maxBytes: 4 } }, + }), '/runtime/stdin-closed') + await flush() + fake.finish() + await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) + }) + + it('rejects malformed SDK process ids and non-command settlement failures', async () => { + const invalidPid = new FakeSandbox() + invalidPid.handle.pid = 0 + const invalid = testHandle(runtime(invalidPid), spec(), '/runtime/invalid-pid') + await expect(invalid.done).rejects.toThrow(/invalid command pid 0/) + expect(invalidPid.handle.kills).toBe(1) + expect(invalidPid.removed).toContain('/runtime/invalid-pid/environment') + await expect(invalid.waitForExit()).resolves.toBe(true) + + const failedRollback = new FakeSandbox() + failedRollback.handle.pid = 0 + failedRollback.handle.killError = new Error('invalid handle kill failed') + const retained = testHandle(runtime(failedRollback), spec(), '/runtime/invalid-pid-retained') + await expect(retained.done).rejects.toThrow('invalid command pid rollback did not reach quiescence') + await expect(retained.waitForExit()).rejects.toThrow('invalid handle kill failed') + failedRollback.handle.killError = undefined + retained.terminate() + await expect(retained.waitForExit()).resolves.toBe(true) + + const crashedFake = new FakeSandbox() + const crashed = testHandle(runtime(crashedFake), spec(), '/runtime/crashed') + await flush() + crashedFake.alive = false + crashedFake.handle.crash(new Error('command transport failed')) + await expect(crashed.done).rejects.toThrow('command transport failed') + }) + + it('rejects invalid or absent process-group publication', async () => { + const invalidGroup = new FakeSandbox() + invalidGroup.processGroupId = 'not-a-pid\n' + invalidGroup.delaysKill = true + invalidGroup.sdkKillStops = false + invalidGroup.afterProbe = () => { invalidGroup.alive = false } + const invalid = testHandle(runtime(invalidGroup), spec(), '/runtime/invalid-group') + await expect(invalid.done).rejects.toThrow(/invalid process-group id/) + expect(invalidGroup.handle.kills).toBe(1) + expect(invalidGroup.commandsSeen).toContain('kill -KILL -- -4242') + await expect(invalid.waitForExit()).resolves.toBe(true) + + // A rewritten pid file must not aim the kill at every process (`-- -1`). + const unsafeGroup = new FakeSandbox() + unsafeGroup.processGroupId = '1\n' + unsafeGroup.delaysKill = true + unsafeGroup.sdkKillStops = false + unsafeGroup.afterProbe = () => { unsafeGroup.alive = false } + const unsafe = testHandle(runtime(unsafeGroup), spec(), '/runtime/unsafe-group') + await expect(unsafe.done).rejects.toThrow(/unsafe published process-group id 1/) + expect(unsafeGroup.commandsSeen).not.toContain('kill -KILL -- -1') + await expect(unsafe.waitForExit()).resolves.toBe(true) + + const absentGroup = new FakeSandbox() + absentGroup.processGroupId = '' + const absent = testHandle(runtime(absentGroup), spec(), '/runtime/absent-group') + await flush() + absentGroup.finish() + await expect(absent.done).rejects.toThrow(/exited before publishing/) + expect(absentGroup.handle.kills).toBe(1) + expect(absentGroup.commandsSeen).toContain('kill -KILL -- -4242') + await expect(absent.waitForExit()).resolves.toBe(true) + }) + + it('preserves publication failure and reports cleanup that cannot be verified', async () => { + const fake = new FakeSandbox() + fake.processGroupId = 'not-a-pid\n' + fake.signalError = new Error('rollback signal failed') + fake.handle.killError = new Error('SDK kill failed') + const handle = testHandle(runtime(fake), spec(), '/runtime/failed-rollback') + + let failure: unknown + try { + await handle.done + } catch (error: unknown) { + failure = error + } + expect(failure).toBeInstanceOf(AggregateError) + if (!(failure instanceof AggregateError)) throw new Error('expected AggregateError') + expect(failure.message).toBe('subprocess-e2b: process-group publication failed and rollback did not reach quiescence') + const failures = Array.from(failure.errors as Iterable) + expect(failures).toHaveLength(2) + expect(failures[0]).toBeInstanceOf(Error) + expect(failures[1]).toBeInstanceOf(Error) + if (!(failures[0] instanceof Error) || !(failures[1] instanceof Error)) throw new Error('expected nested errors') + expect(failures[0].message).toContain('invalid process-group id') + expect(failures[1].message).toContain('remained live after force termination') + expect(fake.handle.kills).toBe(1) + const bounded = new AbortController() + const waiting = handle.waitForExit(bounded.signal) + bounded.abort() + await expect(waiting).resolves.toBe(false) + handle.terminate() + await expect(handle.waitForExit()).resolves.toBe(true) + expect(fake.commandsSeen).toContain('kill -TERM -- -4242') + + const naturallyGone = new FakeSandbox() + naturallyGone.processGroupId = 'not-a-pid\n' + naturallyGone.signalError = new Error('rollback signal failed') + naturallyGone.handle.killError = new Error('SDK kill failed') + const observed = testHandle(runtime(naturallyGone), spec(), '/runtime/failed-rollback-observed') + await expect(observed.done).rejects.toThrow('process-group publication failed') + naturallyGone.alive = false + await expect(observed.waitForExit()).resolves.toBe(true) + }) + + it('waits for delayed process-group publication', async () => { + const fake = new FakeSandbox() + fake.processGroupReads.push('', '4242\n') + const handle = testHandle(runtime(fake), spec(), '/runtime/delayed-group') + await vi.waitFor(() => { expect(handle.pid).toBe(4242) }) + fake.finish() + await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) + }) + + it('handles output backpressure and contains a stderr sink failure', async () => { + const fake = new FakeSandbox() + const handle = testHandle(runtime(fake), spec({ + stdio: { stdin: 'ignore', stdout: 'pipe', stderr: 'pipe' }, + }), '/runtime/backpressure') + await flush() + + handle.stdout!.on('error', () => {}) + const stdoutWrite = vi.spyOn(handle.stdout!, 'write').mockReturnValueOnce(false) + const stdoutPending = fake.stdout('blocked') + queueMicrotask(() => { handle.stdout!.emit('drain') }) + await stdoutPending + stdoutWrite.mockRestore() + + handle.stderr!.on('error', () => {}) + const stderrWrite = vi.spyOn(handle.stderr!, 'write').mockReturnValueOnce(false) + const stderrPending = fake.stderr('broken') + queueMicrotask(() => { handle.stderr!.emit('error', new Error('sink failed')) }) + await stderrPending + stderrWrite.mockRestore() + + fake.finish() + await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) + }) + + it('settles output backpressure when the consumer closes the pipe', async () => { + const fake = new FakeSandbox() + const handle = testHandle(runtime(fake), spec({ + stdio: { stdin: 'ignore', stdout: 'pipe', stderr: { maxBytes: 4 } }, + }), '/runtime/backpressure-close') + await flush() + + const stdoutWrite = vi.spyOn(handle.stdout!, 'write').mockReturnValueOnce(false) + const pending = fake.stdout('discarded') + queueMicrotask(() => { handle.stdout!.destroy() }) + await pending + stdoutWrite.mockRestore() + + fake.finish() + await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) + }) + + it('breaks output backpressure when termination owns the command', async () => { + const fake = new FakeSandbox() + const handle = testHandle(runtime(fake), spec({ + stdio: { stdin: 'ignore', stdout: 'pipe', stderr: { maxBytes: 4 } }, + }), '/runtime/backpressure-termination') + await flush() + + const stdoutWrite = vi.spyOn(handle.stdout!, 'write').mockReturnValueOnce(false) + let released = false + const pending = fake.stdout('blocked').then(() => { released = true }) + await Promise.resolve() + handle.terminate() + await flush() + const releasedByTermination = released + if (!released) handle.stdout!.emit('drain') + await pending + stdoutWrite.mockRestore() + await handle.done + + expect(releasedByTermination).toBe(true) + }) + + it('settles backpressure when a synchronous pipe write starts termination', async () => { + const fake = new FakeSandbox() + const handle = testHandle(runtime(fake), spec({ + stdio: { stdin: 'ignore', stdout: 'pipe', stderr: { maxBytes: 4 } }, + }), '/runtime/backpressure-synchronous-termination') + await flush() + + const stdoutWrite = vi.spyOn(handle.stdout!, 'write').mockImplementationOnce(() => { + handle.terminate() + return false + }) + await expect(fake.stdout('blocked')).resolves.toBeUndefined() + stdoutWrite.mockRestore() + await handle.done + }) + + it('contains a pipe callback failure instead of rejecting command settlement', async () => { + const fake = new FakeSandbox() + const handle = testHandle(runtime(fake), spec({ + stdio: { stdin: 'ignore', stdout: 'pipe', stderr: { maxBytes: 4 } }, + }), '/runtime/pipe-error') + await flush() + const emitted = once(handle.stdout!, 'error') + handle.stdout!.destroy(new Error('consumer failed')) + await emitted + await fake.stdout('late output') + fake.finish() + await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) + }) + + it('contains an already-gone group signal and escalates after a TERM transport failure', async () => { + const gone = new FakeSandbox() + gone.trapsTerm = true + gone.signalError = commandError(1) + const goneHandle = testHandle(runtime(gone), spec({ graceMs: 1 }), '/runtime/gone-signal') + await flush() + goneHandle.terminate() + await expect(goneHandle.done).resolves.toEqual({ exitCode: null, signal: 'SIGKILL' }) + + const failed = new FakeSandbox() + failed.signalError = new Error('signal transport failed') + const failedHandle = testHandle(runtime(failed), spec(), '/runtime/failed-signal') + await flush() + failedHandle.terminate() + await expect(failedHandle.done).resolves.toEqual({ exitCode: null, signal: 'SIGKILL' }) + expect(failed.commandsSeen).toContain('kill -KILL -- -4242') + }) + + it('allows termination retry after both force transports fail', async () => { + const fake = new FakeSandbox() + fake.signalErrors.push(new Error('TERM transport failed'), new Error('KILL transport failed')) + fake.handle.killError = new Error('SDK kill failed') + const handle = testHandle(runtime(fake), spec({ graceMs: 1 }), '/runtime/retry-signal') + await flush() + + handle.terminate() + await expect(handle.waitForExit()).rejects.toThrow('remained live after force termination') + fake.handle.killError = undefined + handle.terminate() + await expect(handle.done).resolves.toEqual({ exitCode: null, signal: 'SIGTERM' }) + expect(fake.commandsSeen.filter(command => command.startsWith('kill -TERM '))).toHaveLength(2) + + const missingGroup = new FakeSandbox() + missingGroup.trapsTerm = true + missingGroup.signalErrors.push(undefined, commandError(1)) + missingGroup.handle.killError = new Error('SDK kill failed after group exit race') + const raced = testHandle(runtime(missingGroup), spec({ graceMs: 1 }), '/runtime/group-exit-race') + await flush() + raced.terminate() + await expect(raced.waitForExit()).rejects.toThrow('remained live after force termination') + missingGroup.handle.killError = undefined + raced.terminate() + await expect(raced.waitForExit()).resolves.toBe(true) + }) + + it('rejects an optimistic SDK kill while descendants survive a failed group KILL', async () => { + const fake = new FakeSandbox() + fake.trapsTerm = true + fake.sdkKillStops = false + fake.signalErrors.push(undefined, new Error('KILL transport failed')) + const handle = testHandle(runtime(fake), spec({ graceMs: 1 }), '/runtime/optimistic-sdk-kill') + await flush() + + handle.terminate() + await expect(handle.waitForExit()).rejects.toThrow('remained live after force termination') + expect(fake.alive).toBe(true) + + fake.sdkKillStops = true + handle.terminate() + await expect(handle.waitForExit()).resolves.toBe(true) + await expect(handle.done).resolves.toEqual({ exitCode: null, signal: 'SIGKILL' }) + }) +}) + +describe('E2BSubprocessService', () => { + async function service( + fake = new FakeSandbox(), + providedRuntime: E2BSandboxService = runtime(fake), + ): Promise<{ ctx: Context; fiber: Awaited> }> { + const ctx = new Context() + ctx.provide('e2b', providedRuntime) + const fiber = await ctx.plugin(E2BSubprocessService) + return { ctx, fiber } + } + + it('registers handles and disposal terminates and joins live remote groups regardless of sandbox policy', async () => { + const fake = new FakeSandbox() + fake.trapsTerm = true + const { ctx, fiber } = await service(fake) + const handle = ctx.subprocess.spawn(spec({ graceMs: 1 })) + await flush() + await fiber.dispose() + await expect(handle.done).resolves.toEqual({ exitCode: null, signal: 'SIGKILL' }) + expect(fake.alive).toBe(false) + }) + + it('awaits SDK settlement after the remote process group becomes quiescent', async () => { + const fake = new FakeSandbox() + fake.trapsTerm = true + const { ctx, fiber } = await service(fake) + const handle = ctx.subprocess.spawn(spec()) + await flush() + fake.alive = false + + let disposed = false + const disposing = fiber.dispose().then(() => { disposed = true }) + await flush() + expect(disposed).toBe(false) + + fake.finish() + await disposing + await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) + }) + + it('reports a failed termination transaction from disposal instead of waiting on done', async () => { + const fake = new FakeSandbox() + fake.signalErrors.push(new Error('TERM transport failed'), new Error('KILL transport failed')) + fake.handle.killError = new Error('SDK kill failed') + const { ctx, fiber } = await service(fake) + const handle = ctx.subprocess.spawn(spec({ graceMs: 1 })) + await flush() + + await expect(fiber.dispose()).resolves.toBeUndefined() + await expect(handle.waitForExit()).rejects.toThrow('remained live after force termination') + + fake.handle.killError = undefined + handle.terminate() + await expect(handle.waitForExit()).resolves.toBe(true) + await expect(handle.done).resolves.toEqual({ exitCode: null, signal: 'SIGTERM' }) + }) + + it('aggregates sibling cleanup failures instead of reporting only the first', async () => { + const { ctx, fiber } = await service() + const disposalErrors: unknown[] = [] + ctx.logger.error = ((error: unknown) => { disposalErrors.push(error) }) as typeof ctx.logger.error + const first = { + terminate: vi.fn(), + waitForExit: vi.fn(async () => { throw new Error('first cleanup failed') }), + done: Promise.resolve({ exitCode: 0, signal: null }), + } as unknown as E2BSubprocessHandle + const second = { + terminate: vi.fn(), + waitForExit: vi.fn(async () => { throw new Error('second cleanup failed') }), + done: Promise.resolve({ exitCode: 0, signal: null }), + } as unknown as E2BSubprocessHandle + const live = (ctx.subprocess as unknown as { live: Set }).live + live.add(first) + live.add(second) + + await fiber.dispose() + const failure = disposalErrors[0] + expect(failure).toBeInstanceOf(AggregateError) + if (!(failure instanceof AggregateError)) throw new Error('expected AggregateError') + expect(failure.errors.map(error => (error as Error).message).sort()).toEqual([ + 'first cleanup failed', + 'second cleanup failed', + ]) + }) + + it('waits for every owned cleanup before reporting a disposal failure', async () => { + const { ctx, fiber } = await service() + const failed = { + terminate: vi.fn(), + waitForExit: vi.fn(async () => { throw new Error('cleanup failed') }), + done: Promise.resolve({ exitCode: 0, signal: null }), + } as unknown as E2BSubprocessHandle + let finishCleanup!: () => void + const cleanup = new Promise((resolve) => { + finishCleanup = () => { resolve(true) } + }) + const draining = { + terminate: vi.fn(), + waitForExit: vi.fn(() => cleanup), + done: Promise.resolve({ exitCode: 0, signal: null }), + } as unknown as E2BSubprocessHandle + const live = (ctx.subprocess as unknown as { live: Set }).live + live.add(failed) + live.add(draining) + + let disposed = false + const disposing = fiber.dispose().then(() => { disposed = true }) + await flush() + expect(disposed).toBe(false) + finishCleanup() + await disposing + expect(live).toEqual(new Set([failed])) + }) + + it('releases naturally settled handles before later service disposal', async () => { + const fake = new FakeSandbox() + const { ctx, fiber } = await service(fake) + const handle = ctx.subprocess.spawn(spec()) + await flush() + fake.finish() + await handle.done + await flush() + const signalsBefore = fake.commandsSeen.filter(command => command.startsWith('kill -')).length + await fiber.dispose() + expect(fake.commandsSeen.filter(command => command.startsWith('kill -')).length).toBe(signalsBefore) + }) + + it('contains a release liveness failure and retries quiescence during disposal', async () => { + const fake = new FakeSandbox() + let calls = 0 + const reconnecting = runtime(fake, async () => { + calls += 1 + if (calls === 2) throw new Error('transient liveness failure') + return fake.sandbox + }) + const { ctx, fiber } = await service(fake, reconnecting) + const handle = ctx.subprocess.spawn(spec()) + await flush() + fake.finish() + await handle.done + await flush() + await fiber.dispose() + expect(calls).toBeGreaterThanOrEqual(3) + }) + + it('contains spawn rejection while disposal is joining the pending handle', async () => { + const fake = new FakeSandbox() + fake.deferStart() + fake.backgroundError = new Error('start failed during disposal') + const { ctx, fiber } = await service(fake) + const subprocess = ctx.subprocess + const handle = subprocess.spawn(spec()) + await vi.waitFor(() => { expect(fake.startOptions).toBeDefined() }) + const disposing = fiber.dispose() + await flush() + expect(() => subprocess.spawn(spec())).toThrow('service is disposing') + fake.releaseStart() + await expect(disposing).resolves.toBeUndefined() + await expect(handle.done).rejects.toThrow('start failed during disposal') + }) + + it('validates synchronous spawn preconditions', async () => { + const { ctx } = await service() + expect(() => ctx.subprocess.spawn(spec({ argv: [] }))).toThrow(/non-empty program/) + expect(() => ctx.subprocess.spawn(spec({ signal: AbortSignal.abort('stop') }))).toThrow(/aborted before spawn/) + }) + + it('registers the package-owned empty invariant installer', async () => { + const ctx = new Context() + await ctx.plugin(InvariantService, { enabled: true }) + const fiber = await ctx.plugin(E2BSubprocessInvariant).await() + await fiber.dispose() + }) +}) diff --git a/packages/e2b/subprocess-e2b/tests/terminal.spec.ts b/packages/e2b/subprocess-e2b/tests/terminal.spec.ts new file mode 100644 index 0000000000..5472b2cfa7 --- /dev/null +++ b/packages/e2b/subprocess-e2b/tests/terminal.spec.ts @@ -0,0 +1,926 @@ +import { Buffer } from 'node:buffer' +import { once } from 'node:events' +import { Context } from 'cordis' +import { describe, expect, it, vi } from 'vitest' +import { + CommandExitError, + FileNotFoundError, + SandboxNotFoundError, + type CommandHandle, + type CommandResult, + type Sandbox, +} from '@deepseek-ai/dsh-e2b' +import type E2BSandboxService from '@deepseek-ai/dsh-e2b' +import type { SubprocessTerminalSpawnSpec } from '@deepseek-ai/dsh-subprocess' +import E2BSubprocessService from '@deepseek-ai/dsh-subprocess-e2b' +import { spawnE2BTerminal } from '../src/terminal.ts' + +function commandError(exitCode: number): CommandExitError { + return new CommandExitError({ exitCode, stdout: '', stderr: '', error: `exit ${exitCode}` }) +} + +interface CommandOptions { + signal?: AbortSignal + cwd?: string + envs?: Record +} + +class FakeTerminalCommandHandle { + pid = 123 + disconnects = 0 + sdkKills = 0 + disconnectError: unknown + sdkKillError: unknown + waitError: unknown + settleOnSdkKill = true + private readonly result = Promise.withResolvers() + private settled = false + + wait(): Promise { + if (this.waitError !== undefined) throw this.waitError + return this.result.promise + } + + async disconnect(): Promise { + this.disconnects += 1 + if (this.disconnectError !== undefined) throw this.disconnectError + } + + async kill(): Promise { + this.sdkKills += 1 + if (this.sdkKillError !== undefined) { + const error = this.sdkKillError + if (this.settleOnSdkKill) this.fail(137) + throw error + } + if (this.settleOnSdkKill) this.fail(137) + return true + } + + succeed(exitCode = 0): void { + if (this.settled) return + this.settled = true + this.result.resolve({ exitCode, stdout: '', stderr: '' }) + } + + fail(exitCode: number): void { + if (this.settled) return + this.settled = true + this.result.reject(commandError(exitCode)) + } + + crash(error: unknown): void { + if (this.settled) return + this.settled = true + this.result.reject(error) + } + + asHandle(): CommandHandle { + return this as unknown as CommandHandle + } +} + +class FakeTerminalSandbox { + readonly handle = new FakeTerminalCommandHandle() + readonly commands: string[] = [] + readonly commandOptions: CommandOptions[] = [] + readonly inputs: Array<{ pid: number; data: Buffer }> = [] + readonly removed: string[] = [] + readonly directories: string[] = [] + readonly writes = new Map() + createOptions: Parameters[0] | undefined + ambient = 'KEEP=visible\0UNICODE=你好\0NPM_TOKEN=secret\0DSH_STALE=old\0BROKEN\0=bad\0' + sessionId = '123\n' + foreground = '456\n' + groups = [123] + zombieGroups: number[] = [] + createError: unknown + writeError: unknown + sendError: unknown + commandFailure: unknown + makeDirRequest: ((signal: AbortSignal | undefined) => Promise) | undefined + sendInputRequest: ((signal: AbortSignal | undefined) => Promise) | undefined + foregroundRequest: ((signal: AbortSignal | undefined) => Promise) | undefined + signalRequest: ((signal: AbortSignal | undefined) => Promise) | undefined + sessionGroupsFailure: unknown + foregroundFailure: unknown + termFailure: unknown + removeError: unknown + clearOnTerm = true + clearOnKill = true + resolvedExecutable = '/usr/bin/node\n' + requestedOutput = 'requested-shell$ ' + emitOutputMarker = true + afterSessionLookup: (() => void) | undefined + private createGate: Promise | undefined + private releaseCreateGate: (() => void) | undefined + + deferCreate(): void { + const gate = Promise.withResolvers() + this.createGate = gate.promise + this.releaseCreateGate = () => { gate.resolve(undefined) } + } + + releaseCreate(): void { + this.releaseCreateGate?.() + } + + readonly sandbox = { + files: { + makeDir: async (path: string, options?: CommandOptions): Promise => { + this.directories.push(path) + await this.makeDirRequest?.(options?.signal) + options?.signal?.throwIfAborted() + return true + }, + write: async (files: Array<{ path: string; data: string }>): Promise => { + for (const file of files) this.writes.set(file.path, file.data) + if (this.writeError !== undefined) throw this.writeError + return files.map(() => ({})) + }, + remove: async (path: string): Promise => { + this.removed.push(path) + if (this.removeError !== undefined) throw this.removeError + }, + }, + commands: { + run: async (command: string, options?: CommandOptions): Promise => { + this.commands.push(command) + if (options !== undefined) this.commandOptions.push(options) + options?.signal?.throwIfAborted() + if (this.commandFailure !== undefined) { + const error = this.commandFailure + this.commandFailure = undefined + throw error + } + if (command.includes('env -0 | base64')) { + return { + exitCode: 0, + stdout: ['/home/user', this.ambient].map(value => Buffer.from(value).toString('base64')).join('\n'), + stderr: '', + } + } + if (command.includes('command -v -- ')) { + return { exitCode: 0, stdout: this.resolvedExecutable, stderr: '' } + } + if (command.startsWith('ps -o sid=')) { + this.afterSessionLookup?.() + return { exitCode: 0, stdout: this.sessionId, stderr: '' } + } + if (command.startsWith('ps -o tpgid=')) { + await this.foregroundRequest?.(options?.signal) + options?.signal?.throwIfAborted() + if (this.foregroundFailure !== undefined) throw this.foregroundFailure + return { exitCode: 0, stdout: this.foreground, stderr: '' } + } + if (command.startsWith('set -o pipefail; ps -eo sid=')) { + if (this.sessionGroupsFailure !== undefined) throw this.sessionGroupsFailure + const groups = command.includes('stat=') && command.includes('$3 !~ /^[ZXx]/') + ? this.groups + : [...this.groups, ...this.zombieGroups] + return { exitCode: 0, stdout: groups.map(group => `${group}\n`).join(''), stderr: '' } + } + if (command.startsWith('kill -TERM -- ')) { + if (this.termFailure !== undefined) throw this.termFailure + if (this.clearOnTerm) { + this.groups = [] + this.handle.fail(143) + } + } + if (command.startsWith('kill -INT -- ')) { + await this.signalRequest?.(options?.signal) + options?.signal?.throwIfAborted() + } + if (command.startsWith('kill -KILL -- ') && this.clearOnKill) this.groups = [] + return { exitCode: 0, stdout: '', stderr: '' } + }, + }, + pty: { + create: async (options: Parameters[0]): Promise => { + this.createOptions = options + if (this.createError !== undefined) throw this.createError + await this.createGate + options.signal?.throwIfAborted() + await options.onData(Buffer.from('buffered banner\n')) + return this.handle.asHandle() + }, + sendInput: async (pid: number, data: Uint8Array, options?: { signal?: AbortSignal }): Promise => { + options?.signal?.throwIfAborted() + await this.sendInputRequest?.(options?.signal) + options?.signal?.throwIfAborted() + this.inputs.push({ pid, data: Buffer.from(data) }) + if (this.sendError !== undefined) throw this.sendError + if (this.emitOutputMarker && Buffer.from(data).includes(Buffer.from('runner.bash'))) { + const marker = [...this.writes].find(([path]) => path.endsWith('/output-marker'))?.[1] + const onData = this.createOptions?.onData + if (marker !== undefined && onData !== undefined) { + await onData(Buffer.from(Buffer.from(data).toString().replace(/\r$/, '\r\n'))) + const split = Math.floor(marker.length / 2) + await onData(Buffer.from(marker.slice(0, split))) + await onData(Buffer.from(marker.slice(split))) + await onData(Buffer.from(this.requestedOutput)) + } + } + }, + }, + } as unknown as Sandbox +} + +function runtime(fake: FakeTerminalSandbox): E2BSandboxService { + return { + cwd: '/workspace', + runtimeRoot: '/workspace/.dsh-e2b', + getSandbox: async () => fake.sandbox, + } as unknown as E2BSandboxService +} + +function spec(overrides: Partial = {}): SubprocessTerminalSpawnSpec { + return { + argv: ['/bin/bash', '--noprofile', '--norc'], + cwd: '/workspace', + rows: 24, + cols: 80, + graceMs: 5, + env: { TERM: 'dumb', DSH_SESSION_ID: 'owner', TOKEN_EXPLICIT: 'kept' }, + ...overrides, + } +} + +function holdRequestUntilAbort(started: PromiseWithResolvers) { + return async (signal: AbortSignal | undefined): Promise => { + if (signal === undefined) throw new Error('expected an operation signal') + signal.throwIfAborted() + started.resolve(signal) + await new Promise((_resolve, reject) => { + signal.addEventListener('abort', () => { + reject(signal.reason instanceof Error ? signal.reason : new Error(String(signal.reason))) + }, { once: true }) + }) + } +} + +/** Spawn the terminal under test with the config default the service would pass. */ +function testSpawn( + runtime: Parameters[0], + spec: Parameters[1], + stateDir: string, + pollMs = 20, +): ReturnType { + return spawnE2BTerminal(runtime, spec, stateDir, pollMs) +} + +describe('E2B terminal allocation', () => { + it('hides bootstrap-shell bytes and preserves requested-shell bytes across the output boundary', async () => { + const fake = new FakeTerminalSandbox() + const terminal = await testSpawn(runtime(fake), spec(), '/runtime/terminal-one') + let output = '' + terminal.output.on('data', (chunk) => { output += String(chunk) }) + await new Promise(resolve => setTimeout(resolve, 0)) + + expect(output).toBe('requested-shell$ ') + expect(output).not.toContain('buffered banner') + expect(output).not.toContain('runner.bash') + expect(fake.createOptions).toMatchObject({ rows: 24, cols: 80, cwd: '/workspace', timeoutMs: 0 }) + const controlEnvs = fake.createOptions?.envs + expect(controlEnvs?.HOME).toMatch(/^\/\.dsh-e2b-control-/) + expect(controlEnvs).toEqual({ + TERM: 'dumb', + NPM_TOKEN: '', + DSH_STALE: '', + HOME: controlEnvs?.HOME, + }) + expect(fake.inputs[0]?.data.toString()).toContain("exec /bin/bash '/runtime/terminal-one/runner.bash'") + expect(fake.writes.get('/runtime/terminal-one/environment')).toContain('KEEP=visible\0') + expect(fake.writes.get('/runtime/terminal-one/environment')).toContain('UNICODE=你好\0') + expect(fake.writes.get('/runtime/terminal-one/environment')).toContain('TOKEN_EXPLICIT=kept\0') + expect(fake.writes.get('/runtime/terminal-one/environment')).not.toContain('secret') + expect(fake.writes.get('/runtime/terminal-one/environment')).not.toContain('DSH_STALE') + expect(fake.writes.get('/runtime/terminal-one/argv')).toBe('/bin/bash\0--noprofile\0--norc\0') + const marker = fake.writes.get('/runtime/terminal-one/output-marker') ?? '' + expect(marker).toMatch(/^dsh-e2b-bootstrap:/) + expect(fake.inputs[0]?.data.toString()).not.toContain(marker) + const runner = fake.writes.get('/runtime/terminal-one/runner.bash') ?? '' + expect(runner).toContain('if (( ${#dsh_argv[@]} == 0 )); then') + expect(runner).toContain('printf \'%s\' "$dsh_output_marker"') + expect(runner).toContain('exec env -i -- "${dsh_env[@]}" "${dsh_argv[@]}"') + expect(runner).not.toContain('\u007f') + terminal.output.destroy() + await fake.createOptions?.onData(Buffer.from('late bootstrap callback')) + expect(output).toBe('requested-shell$ ') + + await terminal.write('echo ok\r') + expect(fake.inputs.at(-1)?.data.toString()).toBe('echo ok\r') + await expect(terminal.inspectForeground()).resolves.toEqual({ processGroupId: 456, inputWaiting: false }) + await expect(terminal.signalForeground('SIGINT')).resolves.toBe(456) + expect(fake.commands).toContain('kill -INT -- -456') + + const terminated = terminal.terminate() + await expect(terminal.done).resolves.toEqual({ exitCode: null, signal: 'SIGTERM' }) + await terminated + expect(fake.handle.disconnects).toBe(1) + expect(fake.removed).toContain('/runtime/terminal-one') + }) + + it('inherits only safe ambient values and limits the allocation signal to setup', async () => { + const fake = new FakeTerminalSandbox() + const controller = new AbortController() + const terminal = await testSpawn( + runtime(fake), + spec({ env: undefined, signal: controller.signal }), + '/runtime/abort-live', + ) + const environment = fake.writes.get('/runtime/abort-live/environment') ?? '' + expect(environment).toContain('KEEP=visible\0') + expect(environment).not.toContain('secret') + expect(environment).not.toContain('DSH_STALE') + + controller.abort(new Error('stop')) + await terminal.write('still live\r') + expect(fake.inputs.at(-1)?.data.toString()).toBe('still live\r') + await terminal.terminate() + await expect(terminal.done).resolves.toEqual({ exitCode: null, signal: 'SIGTERM' }) + }) + + it('publishes the PTY handle before honoring allocation cancellation', async () => { + const fake = new FakeTerminalSandbox() + fake.deferCreate() + const controller = new AbortController() + const spawning = testSpawn( + runtime(fake), + spec({ signal: controller.signal }), + '/runtime/allocation-cancel', + ) + await vi.waitFor(() => { expect(fake.createOptions).toBeDefined() }) + + controller.abort(new Error('allocation cancelled')) + fake.releaseCreate() + await expect(spawning).rejects.toThrow('allocation cancelled') + expect(fake.createOptions?.signal).toBeUndefined() + expect(fake.groups).toEqual([]) + expect(fake.handle.disconnects).toBe(1) + }) + + it('rejects malformed environment and argv values before PTY allocation', async () => { + const invalidName = new FakeTerminalSandbox() + await expect(testSpawn(runtime(invalidName), spec({ env: { 'BAD=NAME': 'x' } }), '/runtime/name')) + .rejects.toThrow('environment entries') + expect(invalidName.createOptions).toBeUndefined() + + const invalidValue = new FakeTerminalSandbox() + await expect(testSpawn(runtime(invalidValue), spec({ env: { BAD: 'x\0y' } }), '/runtime/value')) + .rejects.toThrow('environment entries') + + const invalidArg = new FakeTerminalSandbox() + await expect(testSpawn(runtime(invalidArg), spec({ argv: ['/bin/bash', 'x\0y'] }), '/runtime/argv')) + .rejects.toThrow('argv must not contain NUL') + }) + + it('cleans malformed handles, bootstrap failures, and readiness failures', async () => { + const failedState = new FakeTerminalSandbox() + failedState.writeError = new Error('state write failed') + await expect(testSpawn(runtime(failedState), spec(), '/runtime/state-write')) + .rejects.toThrow('state write failed') + expect(failedState.writes.get('/runtime/state-write/environment')).toContain('KEEP=visible\0') + expect(failedState.removed).toContain('/runtime/state-write') + expect(failedState.createOptions).toBeUndefined() + + const stateAlreadyGone = new FakeTerminalSandbox() + stateAlreadyGone.writeError = new Error('state write failed after external cleanup') + stateAlreadyGone.removeError = new FileNotFoundError('state already gone') + await expect(testSpawn(runtime(stateAlreadyGone), spec(), '/runtime/state-gone')) + .rejects.toThrow('state write failed after external cleanup') + + const invalidPid = new FakeTerminalSandbox() + invalidPid.handle.pid = 0 + await expect(testSpawn(runtime(invalidPid), spec(), '/runtime/invalid-pid')) + .rejects.toThrow('invalid terminal pid 0') + expect(invalidPid.handle.sdkKills).toBe(1) + expect(invalidPid.removed).toContain('/runtime/invalid-pid') + + const failedInput = new FakeTerminalSandbox() + failedInput.sendError = new Error('bootstrap failed') + await expect(testSpawn(runtime(failedInput), spec(), '/runtime/input')) + .rejects.toThrow('bootstrap failed') + expect(failedInput.commands).toContain('kill -TERM -- -123') + expect(failedInput.groups).toEqual([]) + + const invalidSession = new FakeTerminalSandbox() + invalidSession.sessionId = 'not-a-session\n' + invalidSession.clearOnTerm = false + await expect(testSpawn(runtime(invalidSession), spec(), '/runtime/session')) + .rejects.toThrow('cannot resolve process session') + expect(invalidSession.commands).toContain('kill -TERM -- -123') + expect(invalidSession.commands).toContain('kill -KILL -- -123') + expect(invalidSession.groups).toEqual([]) + expect(invalidSession.handle.sdkKills).toBe(1) + const lateData = invalidSession.createOptions?.onData + if (lateData === undefined) throw new Error('missing captured terminal callback') + expect(lateData(Buffer.from('late bytes'))).toBeUndefined() + + const termFailed = new FakeTerminalSandbox() + termFailed.sendError = new Error('bootstrap failed') + termFailed.termFailure = new Error('TERM transport failed') + await expect(testSpawn(runtime(termFailed), spec(), '/runtime/term-failed')) + .rejects.toThrow('bootstrap failed') + expect(termFailed.commands).toContain('kill -KILL -- -123') + expect(termFailed.handle.sdkKills).toBe(1) + + const uninspectable = new FakeTerminalSandbox() + uninspectable.sendError = new Error('bootstrap failed') + uninspectable.sessionGroupsFailure = 'session enumeration failed' + uninspectable.handle.sdkKillError = new Error('PTY kill failed') + let uninspectableFailure: unknown + try { + await testSpawn(runtime(uninspectable), spec(), '/runtime/uninspectable') + } catch (error: unknown) { + uninspectableFailure = error + } + expect(uninspectableFailure).toBeInstanceOf(AggregateError) + expect(uninspectable.handle.sdkKills).toBe(1) + + const survivingGroups = new FakeTerminalSandbox() + survivingGroups.sendError = new Error('bootstrap failed') + survivingGroups.clearOnTerm = false + survivingGroups.clearOnKill = false + await expect(testSpawn(runtime(survivingGroups), spec({ graceMs: 1 }), '/runtime/surviving-groups')) + .rejects.toThrow('bootstrap failed') + + const survivingPid = new FakeTerminalSandbox() + survivingPid.sendError = new Error('bootstrap failed') + survivingPid.groups = [] + survivingPid.handle.settleOnSdkKill = false + await expect(testSpawn(runtime(survivingPid), spec({ graceMs: 1 }), '/runtime/surviving-pid')) + .rejects.toThrow('bootstrap failed') + + const waitFailed = new FakeTerminalSandbox() + waitFailed.handle.waitError = new Error('wait failed') + waitFailed.handle.settleOnSdkKill = false + waitFailed.handle.sdkKillError = new Error('kill failed') + await expect(testSpawn(runtime(waitFailed), spec(), '/runtime/wait-failed')) + .rejects.toThrow('wait failed') + expect(waitFailed.handle.sdkKills).toBe(1) + + const cleanupFailed = new FakeTerminalSandbox() + cleanupFailed.handle.pid = 0 + cleanupFailed.handle.sdkKillError = new Error('kill transport failed') + cleanupFailed.removeError = new Error('remove transport failed') + await expect(testSpawn(runtime(cleanupFailed), spec(), '/runtime/cleanup-failed')) + .rejects.toThrow('invalid terminal pid 0') + + const expiredDuringRollback = new FakeTerminalSandbox() + expiredDuringRollback.sendError = new Error('bootstrap failed before timeout') + expiredDuringRollback.groups = [] + expiredDuringRollback.handle.settleOnSdkKill = false + expiredDuringRollback.handle.sdkKillError = new SandboxNotFoundError('sandbox expired') + expiredDuringRollback.removeError = new SandboxNotFoundError('sandbox expired') + await expect(testSpawn(runtime(expiredDuringRollback), spec(), '/runtime/expired-rollback')) + .rejects.toThrow('bootstrap failed before timeout') + expect(expiredDuringRollback.handle.sdkKills).toBe(1) + + const expiredBeforeSdkRollback = new FakeTerminalSandbox() + expiredBeforeSdkRollback.handle.waitError = new Error('wait failed after timeout') + expiredBeforeSdkRollback.handle.sdkKillError = new SandboxNotFoundError('sandbox expired') + expiredBeforeSdkRollback.handle.settleOnSdkKill = false + await expect(testSpawn(runtime(expiredBeforeSdkRollback), spec(), '/runtime/expired-sdk-rollback')) + .rejects.toThrow('wait failed after timeout') + + const missingDuringDisconnect = new FakeTerminalSandbox() + missingDuringDisconnect.sendError = new Error('bootstrap failed before disconnect') + missingDuringDisconnect.handle.disconnectError = new SandboxNotFoundError('sandbox expired') + await expect(testSpawn(runtime(missingDuringDisconnect), spec(), '/runtime/missing-disconnect')) + .rejects.toThrow('bootstrap failed before disconnect') + + const failedDisconnect = new FakeTerminalSandbox() + failedDisconnect.sendError = new Error('bootstrap failed with disconnect failure') + failedDisconnect.handle.disconnectError = new Error('disconnect transport failed') + await expect(testSpawn(runtime(failedDisconnect), spec(), '/runtime/failed-disconnect')) + .rejects.toThrow('bootstrap failed with disconnect failure') + }) + + it('propagates setup cancellation and provider failures', async () => { + const aborted = new FakeTerminalSandbox() + await expect(testSpawn(runtime(aborted), spec({ signal: AbortSignal.abort(new Error('stop')) }), '/runtime/abort')) + .rejects.toThrow('stop') + + const createFailed = new FakeTerminalSandbox() + createFailed.createError = new Error('create failed') + await expect(testSpawn(runtime(createFailed), spec(), '/runtime/create')) + .rejects.toThrow('create failed') + + }) + + it('bounds a missing bootstrap-output boundary by process exit or cancellation', async () => { + const exited = new FakeTerminalSandbox() + exited.emitOutputMarker = false + const exiting = testSpawn(runtime(exited), spec(), '/runtime/missing-output-boundary') + await vi.waitFor(() => { expect(exited.inputs).toHaveLength(1) }) + exited.handle.succeed(0) + await expect(exiting).rejects.toThrow('terminal exited before publishing its output boundary') + + const cancelled = new FakeTerminalSandbox() + cancelled.emitOutputMarker = false + const controller = new AbortController() + const cancelling = testSpawn( + runtime(cancelled), + spec({ signal: controller.signal }), + '/runtime/cancel-output-boundary', + ) + await vi.waitFor(() => { expect(cancelled.inputs).toHaveLength(1) }) + await new Promise(resolve => setTimeout(resolve, 0)) + controller.abort(new Error('cancel output boundary')) + await expect(cancelling).rejects.toThrow('cancel output boundary') + }) +}) + +describe('E2B terminal lifecycle', () => { + it('aborts and joins in-flight terminal operations before cleanup', async () => { + const fake = new FakeTerminalSandbox() + const terminal = await testSpawn(runtime(fake), spec(), '/runtime/in-flight-operations') + const writeStarted = Promise.withResolvers() + const inspectStarted = Promise.withResolvers() + const signalStarted = Promise.withResolvers() + fake.sendInputRequest = holdRequestUntilAbort(writeStarted) + let foregroundRequests = 0 + fake.foregroundRequest = async (signal) => { + foregroundRequests += 1 + if (foregroundRequests === 1) await holdRequestUntilAbort(inspectStarted)(signal) + } + let signalCompleted = false + fake.signalRequest = async (operationSignal) => { + await holdRequestUntilAbort(signalStarted)(operationSignal) + signalCompleted = true + } + const write = terminal.write('late input') + const inspect = terminal.inspectForeground() + await Promise.all([writeStarted.promise, inspectStarted.promise]) + const signal = terminal.signalForeground('SIGINT') + await signalStarted.promise + + const terminating = terminal.terminate() + await expect(write).rejects.toThrow('terminal is terminating') + await expect(inspect).rejects.toThrow('terminal is terminating') + await expect(signal).rejects.toThrow('terminal is terminating') + await terminating + expect(signalCompleted).toBe(false) + expect(fake.inputs).toHaveLength(1) + const commandCount = fake.commands.length + await expect(terminal.write('after termination')).rejects.toThrow('terminal is terminating') + await expect(terminal.inspectForeground()).rejects.toThrow('terminal is terminating') + await expect(terminal.signalForeground('SIGINT')).rejects.toThrow('terminal is terminating') + expect(fake.commands).toHaveLength(commandCount) + }) + + it('maps ordinary exits, closes output, and reports an absent foreground after exit', async () => { + const fake = new FakeTerminalSandbox() + fake.groups = [] + const terminal = await testSpawn(runtime(fake), spec(), '/runtime/natural') + terminal.output.resume() + const ended = once(terminal.output, 'end') + fake.handle.succeed(7) + await expect(terminal.done).resolves.toEqual({ exitCode: 7, signal: null }) + await ended + await expect(terminal.write('late')).rejects.toThrow('exited') + fake.foregroundFailure = commandError(1) + await expect(terminal.inspectForeground()).resolves.toBeUndefined() + await expect(terminal.signalForeground('SIGINT')).rejects.toThrow('cannot resolve foreground process group') + await terminal.terminate() + }) + + it.each([ + [7, { exitCode: 7, signal: null }], + [143, { exitCode: 143, signal: null }], + [255, { exitCode: 255, signal: null }], + ] as const)('classifies an unrequested command exit %i', async (exitCode, expected) => { + const fake = new FakeTerminalSandbox() + fake.groups = [] + const terminal = await testSpawn(runtime(fake), spec(), `/runtime/exit-${exitCode}`) + fake.handle.fail(exitCode) + await expect(terminal.done).resolves.toEqual(expected) + await terminal.terminate() + }) + + it('treats a terminal session containing only zombies as quiescent', async () => { + const fake = new FakeTerminalSandbox() + fake.groups = [] + fake.zombieGroups = [123] + const terminal = await testSpawn(runtime(fake), spec(), '/runtime/zombie-session') + + fake.handle.succeed(0) + await expect(terminal.done).resolves.toEqual({ exitCode: 0, signal: null }) + await terminal.terminate() + expect(fake.commands).toContain( + "set -o pipefail; ps -eo sid=,pgid=,stat= | awk '$1 == 123 && $3 !~ /^[ZXx]/ { print $2 }'", + ) + }) + + it('treats a timeout-killed sandbox as quiescent during terminal cleanup', async () => { + const fake = new FakeTerminalSandbox() + const terminal = await testSpawn(runtime(fake), spec(), '/runtime/expired-sandbox') + fake.sessionGroupsFailure = new SandboxNotFoundError('sandbox expired') + fake.handle.succeed(0) + + await expect(terminal.done).resolves.toEqual({ exitCode: 0, signal: null }) + await terminal.terminate() + }) + + it('treats sandbox disappearance during PTY kill as quiescent', async () => { + const fake = new FakeTerminalSandbox() + fake.groups = [] + fake.handle.settleOnSdkKill = false + fake.handle.sdkKillError = new SandboxNotFoundError('sandbox expired') + const terminal = await testSpawn(runtime(fake), spec({ graceMs: 1 }), '/runtime/expired-pty-kill') + + await terminal.terminate() + expect(fake.handle.sdkKills).toBe(1) + }) + + it('propagates a non-missing PTY kill failure', async () => { + const fake = new FakeTerminalSandbox() + fake.groups = [] + fake.handle.settleOnSdkKill = false + fake.handle.sdkKillError = new Error('PTY kill transport failed') + const terminal = await testSpawn(runtime(fake), spec({ graceMs: 1 }), '/runtime/failed-pty-kill') + + await expect(terminal.terminate()).rejects.toThrow('PTY kill transport failed') + fake.handle.sdkKillError = undefined + fake.handle.succeed(0) + await terminal.done + await terminal.terminate() + }) + + it.each([ + ['accepts sandbox loss', new SandboxNotFoundError('sandbox expired'), true], + ['propagates another failure', new Error('disconnect failed'), false], + ] as const)('%s while disconnecting a settled terminal', async (_label, failure, accepted) => { + const fake = new FakeTerminalSandbox() + const terminal = await testSpawn(runtime(fake), spec(), `/runtime/disconnect-${accepted}`) + fake.handle.disconnectError = failure + fake.groups = [] + fake.handle.succeed(0) + + if (accepted) await expect(terminal.terminate()).resolves.toBeUndefined() + else await expect(terminal.terminate()).rejects.toThrow('disconnect failed') + }) + + it('rejects killing the terminal shell and propagates live foreground failures', async () => { + const fake = new FakeTerminalSandbox() + fake.foreground = '123\n' + const terminal = await testSpawn(runtime(fake), spec(), '/runtime/signal') + await expect(terminal.signalForeground('SIGKILL')).rejects.toThrow('refusing to SIGKILL') + fake.foreground = 'invalid\n' + await expect(terminal.inspectForeground()).rejects.toThrow('cannot resolve foreground') + fake.foregroundFailure = commandError(1) + await expect(terminal.inspectForeground()).resolves.toBeUndefined() + fake.foregroundFailure = commandError(2) + await expect(terminal.inspectForeground()).rejects.toBeInstanceOf(CommandExitError) + fake.clearOnTerm = true + await terminal.terminate() + }) + + it('sends KILL before checking an expired force-cleanup deadline', async () => { + const fake = new FakeTerminalSandbox() + fake.groups = [123, 456] + fake.clearOnTerm = false + const terminal = await testSpawn(runtime(fake), spec({ graceMs: 0 }), '/runtime/escalate') + const terminating = terminal.terminate() + await expect(terminal.done).resolves.toEqual({ exitCode: null, signal: 'SIGKILL' }) + await terminating + expect(fake.commands).toContain('kill -TERM -- -123 -456') + expect(fake.commands).toContain('kill -KILL -- -123 -456') + }) + + it('surfaces cleanup failures and allows a later retry', async () => { + const fake = new FakeTerminalSandbox() + fake.groups = [1] + const terminal = await testSpawn(runtime(fake), spec({ graceMs: 1 }), '/runtime/retry') + await expect(terminal.terminate()).rejects.toThrow('unsafe process group 1') + + fake.groups = [] + fake.handle.succeed(0) + await terminal.done + await terminal.terminate() + }) + + it('propagates a process-group signalling transport failure before retry', async () => { + const fake = new FakeTerminalSandbox() + fake.termFailure = new Error('signal transport failed') + const terminal = await testSpawn(runtime(fake), spec({ graceMs: 1 }), '/runtime/signal-failure') + await expect(terminal.terminate()).rejects.toThrow('signal transport failed') + + fake.groups = [] + fake.handle.succeed(0) + await terminal.done + await terminal.terminate() + + const alreadyExited = new FakeTerminalSandbox() + alreadyExited.termFailure = commandError(1) + const tolerant = await testSpawn(runtime(alreadyExited), spec({ graceMs: 1 }), '/runtime/group-exited') + const tolerantTermination = tolerant.terminate() + await expect(tolerant.done).resolves.toEqual({ exitCode: null, signal: 'SIGKILL' }) + await tolerantTermination + }) + + it('keeps command rejection authoritative while cleanup is already waiting', async () => { + const fake = new FakeTerminalSandbox() + fake.groups = [] + fake.removeError = new Error('private state already gone') + const terminal = await testSpawn(runtime(fake), spec(), '/runtime/reject-during-cleanup') + terminal.output.on('error', () => {}) + const cleanup = terminal.terminate() + await Promise.resolve() + fake.handle.crash(new Error('command transport failed')) + await expect(terminal.done).rejects.toThrow('command transport failed') + await cleanup + }) + + it('keeps a late command rejection authoritative after PTY kill', async () => { + const fake = new FakeTerminalSandbox() + fake.groups = [] + fake.handle.settleOnSdkKill = false + const terminal = await testSpawn(runtime(fake), spec({ graceMs: 1 }), '/runtime/reject-after-kill') + terminal.output.on('error', () => {}) + const cleanup = terminal.terminate() + while (fake.handle.sdkKills === 0) await new Promise(resolve => setTimeout(resolve, 0)) + await Promise.resolve() + fake.handle.crash(new Error('late command transport failed')) + await expect(terminal.done).rejects.toThrow('late command transport failed') + await cleanup + }) + + it('reports surviving groups, a surviving top-level pid, and transport failure', async () => { + const survivor = new FakeTerminalSandbox() + survivor.clearOnTerm = false + survivor.clearOnKill = false + const terminal = await testSpawn(runtime(survivor), spec({ graceMs: 1 }), '/runtime/survivor') + await expect(terminal.terminate()).rejects.toThrow('surviving process groups: 123') + + const livePid = new FakeTerminalSandbox() + livePid.groups = [] + livePid.handle.settleOnSdkKill = false + const live = await testSpawn(runtime(livePid), spec({ graceMs: 1 }), '/runtime/live-pid') + await expect(live.terminate()).rejects.toThrow('surviving pid: 123') + livePid.handle.succeed(0) + await live.done + + const crashed = new FakeTerminalSandbox() + crashed.groups = [] + const failed = await testSpawn(runtime(crashed), spec(), '/runtime/crashed') + const outputError = once(failed.output, 'error') + crashed.handle.crash('transport gone') + await expect(failed.done).rejects.toEqual('transport gone') + await expect(outputError).resolves.toMatchObject([{ message: 'transport gone' }]) + await failed.terminate() + }) +}) + +describe('E2B subprocess terminal service', () => { + async function service(fake = new FakeTerminalSandbox()): Promise<{ + ctx: Context + fiber: Awaited> + fake: FakeTerminalSandbox + }> { + const ctx = new Context() + ctx.provide('e2b', runtime(fake)) + const fiber = await ctx.plugin(E2BSubprocessService) + return { ctx, fiber, fake } + } + + it('resolves remote executables', async () => { + const { ctx, fake } = await service() + await expect(ctx.subprocess.resolveExecutable('/bin/bash')).resolves.toBe('/bin/bash') + await expect(ctx.subprocess.resolveExecutable('node', { PATH: '/custom/bin' }, new AbortController().signal)) + .resolves.toBe('/usr/bin/node') + fake.resolvedExecutable = 'tools/bin/node\n' + await expect(ctx.subprocess.resolveExecutable('node', { PATH: 'tools/bin' })) + .resolves.toBe('/workspace/tools/bin/node') + const commandOptions = fake.commandOptions.at(-1) + expect(commandOptions).toMatchObject({ cwd: '/workspace' }) + expect(commandOptions?.envs?.HOME).toMatch(/^\/\.dsh-e2b-control-/) + expect(commandOptions?.envs).toEqual({ HOME: commandOptions?.envs?.HOME }) + expect((ctx.e2b)).toBeDefined() + }) + + it('rejects invalid executable lookup inputs and results', async () => { + const { ctx, fake } = await service() + await expect(ctx.subprocess.resolveExecutable('')).rejects.toThrow('non-empty') + await expect(ctx.subprocess.resolveExecutable('./bin/server')).rejects.toThrow('is a relative path') + await expect(ctx.subprocess.resolveExecutable('node_modules/.bin/server')).rejects.toThrow('is a relative path') + await expect(ctx.subprocess.resolveExecutable('node', undefined, AbortSignal.abort(new Error('stop')))) + .rejects.toThrow('stop') + fake.resolvedExecutable = 'node\n' + await expect(ctx.subprocess.resolveExecutable('node')).rejects.toThrow('did not resolve') + fake.resolvedExecutable = '/one\n/two\n' + await expect(ctx.subprocess.resolveExecutable('node')).rejects.toThrow('did not resolve') + }) + + it('rejects a non-positive poll cadence at load', async () => { + const ctx = new Context() + ctx.provide('e2b', runtime(new FakeTerminalSandbox())) + await expect(ctx.plugin(E2BSubprocessService, { pollMs: 0 })) + .rejects.toThrow('pollMs must be a positive safe integer') + const explicit = await ctx.plugin(E2BSubprocessService, { pollMs: 5 }) + await explicit.dispose() + }) + + it('owns live terminals through service disposal', async () => { + const { ctx, fiber, fake } = await service() + const terminal = await ctx.subprocess.spawnTerminal(spec({ signal: new AbortController().signal })) + await fiber.dispose() + await expect(terminal.done).resolves.toEqual({ exitCode: null, signal: 'SIGTERM' }) + expect(fake.handle.disconnects).toBe(1) + }) + + it('joins and rejects terminal setup that completes during service disposal', async () => { + const fake = new FakeTerminalSandbox() + const { ctx, fiber } = await service(fake) + let disposing: Promise | undefined + fake.afterSessionLookup = () => { + fake.afterSessionLookup = undefined + queueMicrotask(() => { + queueMicrotask(() => { disposing = fiber.dispose() }) + }) + } + const subprocess = ctx.subprocess + const spawning = ctx.subprocess.spawnTerminal(spec()) + const rejected = expect(spawning).rejects.toThrow('service disposed during terminal setup') + await vi.waitFor(() => { expect(disposing).toBeDefined() }) + await expect(subprocess.spawnTerminal(spec())).rejects.toThrow('service is disposing') + + await rejected + await disposing + expect(fake.groups).toEqual([]) + expect(fake.handle.disconnects).toBe(1) + expect(fake.removed.some(path => path.includes('/terminals/'))).toBe(true) + }) + + it('aborts and rolls back terminal setup that cannot publish its output boundary during disposal', async () => { + const fake = new FakeTerminalSandbox() + fake.emitOutputMarker = false + const { ctx, fiber } = await service(fake) + const spawning = ctx.subprocess.spawnTerminal(spec()) + const rejected = expect(spawning).rejects.toThrow('service disposed during terminal setup') + await vi.waitFor(() => { expect(fake.inputs).toHaveLength(1) }) + + await fiber.dispose() + await rejected + expect(fake.groups).toEqual([]) + expect(fake.handle.disconnects).toBe(1) + }) + + it('owns and cancels terminal state-directory creation during disposal', async () => { + const fake = new FakeTerminalSandbox() + fake.makeDirRequest = async (signal) => { + await new Promise((_resolve, reject) => { + const onAbort = (): void => { + const reason: unknown = signal?.reason + reject(reason instanceof Error ? reason : new Error(String(reason))) + } + signal?.addEventListener('abort', onAbort, { once: true }) + if (signal?.aborted === true) onAbort() + }) + } + const { ctx, fiber } = await service(fake) + const spawning = ctx.subprocess.spawnTerminal(spec()) + const rejected = expect(spawning).rejects.toThrow('service disposed during terminal setup') + await vi.waitFor(() => { expect(fake.directories.some(path => path.includes('/terminals/'))).toBe(true) }) + + await fiber.dispose() + await rejected + expect(fake.removed.some(path => path.includes('/terminals/'))).toBe(true) + expect(fake.createOptions).toBeUndefined() + }) + + it('releases naturally settled terminals and validates terminal requests', async () => { + const { ctx, fiber, fake } = await service() + for (const request of [ + spec({ argv: [] }), + spec({ signal: AbortSignal.abort(new Error('cancelled')) }), + ]) { + await expect(ctx.subprocess.spawnTerminal(request)).rejects.toThrow() + } + + fake.groups = [] + const terminal = await ctx.subprocess.spawnTerminal(spec()) + fake.handle.succeed(0) + await terminal.done + await terminal.terminate() + const signals = fake.commands.filter(command => command.startsWith('kill -')).length + await fiber.dispose() + expect(fake.commands.filter(command => command.startsWith('kill -'))).toHaveLength(signals) + }) + + it('contains a failed automatic terminal release until service disposal retries it', async () => { + const { fiber, fake } = await service() + fake.clearOnTerm = false + fake.clearOnKill = false + const terminal = await (fiber.ctx).subprocess.spawnTerminal(spec({ graceMs: 1 })) + fake.handle.succeed(0) + await terminal.done + await new Promise(resolve => setTimeout(resolve, 10)) + expect(fake.commands).toContain('kill -KILL -- -123') + + fake.groups = [] + await fiber.dispose() + await expect(terminal.terminate()).resolves.toBeUndefined() + }) +}) diff --git a/packages/e2b/subprocess-e2b/tsconfig.json b/packages/e2b/subprocess-e2b/tsconfig.json new file mode 100644 index 0000000000..1bae0ef012 --- /dev/null +++ b/packages/e2b/subprocess-e2b/tsconfig.json @@ -0,0 +1,30 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": [ + "src" + ], + "references": [ + { + "path": "../../../vendor/cordis" + }, + { + "path": "../../../vendor/cosmokit" + }, + { + "path": "../../subprocess/subprocess" + }, + { + "path": "../../support/invariants" + }, + { + "path": "../../util/timeout" + }, + { + "path": "../e2b" + } + ] +} diff --git a/packages/fs/README.i18n.yaml b/packages/fs/README.i18n.yaml index e435f166ad..f4719447aa 100644 --- a/packages/fs/README.i18n.yaml +++ b/packages/fs/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/fs/README.md -README.md: 108d7d862a1dc6307268e2a93fa00789c952e440 -README.zh.md: 8b037cc3192bf6ceb0f0671d62911a8e035db24c +README.md: b15012e882b60847e1ad22edf08d1202ba64fe5b +README.zh.md: 628f6c74894bc67559d49f7cf5d1378d0ece2382 diff --git a/packages/fs/README.md b/packages/fs/README.md index 108d7d862a..b15012e882 100644 --- a/packages/fs/README.md +++ b/packages/fs/README.md @@ -2,16 +2,20 @@ English | [中文](README.zh.md) -The filesystem capability family: provider seam, interchangeable backends, policy, and model-facing tools. All **product** packages. +The filesystem stack: a provider seam (execution-world paths, bounded text IO, and atomic mutation with an optional version guard), a local implementation, a policy gate plugin (observed-state + read-before-edit + version-guarded write/edit), the model-facing file tools + executor, and the bash-backed discovery tools. All **product** packages. | Package | Role | ctx key | |---|---|---| -| [`fs/`](fs/README.md) | Filesystem provider seam and policy-event vocabulary | `ctx.fs` | -| [`fs-local/`](fs-local/README.md) | Local-filesystem backend | registers `ctx.fs` | -| [`fs-sandbox/`](fs-sandbox/README.md) | Sandbox-enforcing backend | registers `ctx.fs` | -| [`fs-policy/`](fs-policy/README.md) | Observed-state and mutation policy | `fs/*` listeners | -| [`tool-fs/`](tool-fs/README.md) | Model-facing file tools | registers on `ctx.tools` | -| [`tool-fs-search/`](tool-fs-search/README.md) | Process-backed discovery tools | registers on `ctx.tools` | -| [`tool-str-replace-editor/`](tool-str-replace-editor/README.md) | Model-facing string-replacement editor | registers on `ctx.tools` | +| `fs/` | Provider seam: canonical process paths/file URIs/containment, text IO, and atomic mutation primitives; owns the `fs/*` policy events | `ctx.fs` | +| `fs-local/` | Local-filesystem `FileSystem` implementation | (registers `ctx.fs`) | +| [`e2b/fs-e2b`](../e2b/fs-e2b/README.md) | E2B-backed `FileSystem` implementation sharing the remote runtime owned by `ctx.e2b` | (registers `ctx.fs`) | +| `fs-sandbox/` | Sandbox-enforcing `FileSystem`: extends `fs-local` and fences write/edit by the per-call mode + workspace root policy (read-only denies, workspace-write contains to the session workspace + temp roots), reads pass through | (registers `ctx.fs`) | +| `fs-policy/` | Policy gate plugin: observed-state + read-before-edit + version-guarded write/edit, via the `fs/*` event gate | (no service — `fs/*` listeners) | +| `tool-fs/` | Model-facing `read`/`write`/`edit` tools AND the executor (reads via `ctx.fs`, owns read windowing, dispatches `fs/*`); preserves filesystem semantics for session-cwd-relative paths and advertises sandbox escalation fields when the mounted `ctx.fs` confines | (registers on `ctx.tools`) | +| `tool-fs-search/` | Model-facing `glob`/`grep` discovery tools when `rg` is available on the bash executor `PATH`, backed by fixed ripgrep commands through `ctx.bash`, NOT by `ctx.fs` provider methods | (registers on `ctx.tools`) | -Backends replace one another behind `ctx.fs`; policy and tools consume the seam independently. Discovery remains process-backed instead of expanding the provider contract. Child READMEs own containment, mutation, schema, and timeout details. +The interface lives at `fs/fs/`. A sandboxed, remote, or project-scoped filesystem backend can replace `fs-local` without touching the seam, the policy gate, or the model-facing tool schemas: `fs-sandbox` provides an in-process path fence over the shared sandbox mode ([decision](../../.agents/notes/implemented/feature/2026-07-14-cross-family-fs-sandbox.md)), while `fs-e2b` places file state in the remote execution world shared with the E2B subprocess provider ([decision](../../.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md)). The policy (`fs-policy/`) is a plugin that participates only through the `fs/*` event gate, not a service the tool injects — so dropping it gracefully loses the policy and leaves the unconstrained bare provider rather than breaking the tool. A deployment that loads `tool-fs/` is expected to also load it. The mode fence and the read-before-edit gate are orthogonal and compose. Discovery (`tool-fs-search/`) deliberately does NOT extend the provider seam: search is a process-backed `rg` workflow on the bash executor, so filesystem backends stay free of a universal search contract; its tools register only when that executor can find `rg`, and its results are follow-up-readable when the bash workdir and the `read` root are the same workspace (the co-located deployment its README documents). + +## No timeouts on file IO + +`read`/`write`/`edit` take **no** `timeoutMs`, and the provider seam arms no deadline — unlike bash and web (which consume [`@deepseek-ai/dsh-timeout`](../util/timeout/README.md)) and the bash-backed `glob`/`grep` (whose declared `timeoutMs` is enforced by `@deepseek-ai/dsh-timeout-policy`): those are process-backed, where a deadline can really kill the work. A local syscall is best-effort-abortable at most: a timeout could not force an in-progress `fsync`/`rename` to stop, so a deadline here would be a knob that cannot deliver on its promise. Adding one would also be an implicit default in the exact place explicit-over-implicit forbids. Both reference agents (Claude Code, Codex) leave file IO untimed for the same reason; cancellation still propagates through the tool-execution signal for best-effort abort at syscall boundaries. diff --git a/packages/fs/README.zh.md b/packages/fs/README.zh.md index 8b037cc319..628f6c7489 100644 --- a/packages/fs/README.zh.md +++ b/packages/fs/README.zh.md @@ -1,17 +1,21 @@ -# fs/ - 文件系统能力家族 +# fs/：文件系统能力族 [English](README.md) | 中文 -文件系统能力家族：提供方 seam、可互换后端、策略和面向模型工具。这些全是**产品**包。 +文件系统栈包括：提供方 seam（执行世界路径、有界文本 I/O 与带可选版本防护的原子变更）、本地实现、政策门禁插件（已观察状态、编辑前读取、版本防护的写入/编辑）、面向模型的文件工具与执行器，以及基于 bash 的发现工具。全部都是**产品** 包。 -| 包 | 职责 | ctx key | +| 包 | 角色 | ctx 键 | |---|---|---| -| [`fs/`](fs/README.md) | 文件系统提供方 seam 和策略事件词汇 | `ctx.fs` | -| [`fs-local/`](fs-local/README.md) | 本地文件系统后端 | 注册 `ctx.fs` | -| [`fs-sandbox/`](fs-sandbox/README.md) | 强制执行沙箱的后端 | 注册 `ctx.fs` | -| [`fs-policy/`](fs-policy/README.md) | 已观察状态和修改策略 | `fs/*` 监听器 | -| [`tool-fs/`](tool-fs/README.md) | 面向模型的文件工具 | 注册到 `ctx.tools` | -| [`tool-fs-search/`](tool-fs-search/README.md) | 基于进程的发现工具 | 注册到 `ctx.tools` | -| [`tool-str-replace-editor/`](tool-str-replace-editor/README.md) | 面向模型的字符串替换编辑器 | 注册到 `ctx.tools` | +| `fs/` | 提供方 seam：规范化进程路径、文件 URI 与包含关系、文本 I/O 和原子变更原语；拥有 `fs/*` 政策事件 | `ctx.fs` | +| `fs-local/` | 本地文件系统 `FileSystem` 实现 | （注册 `ctx.fs`） | +| [`e2b/fs-e2b`](../e2b/fs-e2b/README.md) | 以 E2B 为后端的 `FileSystem` 实现，共享由 `ctx.e2b` 拥有的远程运行时 | （注册 `ctx.fs`） | +| `fs-sandbox/` | 强制沙箱的 `FileSystem`：扩展 `fs-local`，并按每次调用的模式与工作区根政策约束写入/编辑（只读模式拒绝，工作区写入模式限制在会话工作区与临时根目录内）；读取直接通过 | （注册 `ctx.fs`） | +| `fs-policy/` | 政策门禁插件：通过 `fs/*` 事件门禁提供已观察状态、编辑前读取和版本防护的写入/编辑 | （无服务，仅有 `fs/*` 监听器） | +| `tool-fs/` | 面向模型的 `read`/`write`/`edit` 工具以及执行器（通过 `ctx.fs` 读取，拥有读取窗口逻辑，分派 `fs/*`）；为会话 cwd 相对路径保留文件系统语义，并在已挂载的 `ctx.fs` 实施约束时声明沙箱升级字段 | （注册到 `ctx.tools`） | +| `tool-fs-search/` | 面向模型的 `glob`/`grep` 发现工具；当 `rg` 位于 bash 执行器 `PATH` 上时注册，通过 `ctx.bash` 运行固定 ripgrep 命令，而不是使用 `ctx.fs` 提供方方法 | （注册到 `ctx.tools`） | -后端可在 `ctx.fs` 后互相替换；策略和工具独立消费该 seam。发现功能仍由进程提供，不扩展提供方契约。子 README 负责围堵、修改、schema 和超时细节。 +接口位于 `fs/fs/`。沙箱化、远程或限定项目作用域的文件系统后端可以替换 `fs-local`，而无需更改 seam、政策门禁或面向模型的工具 schema：`fs-sandbox` 基于共享沙箱模式提供进程内路径围栏（[决策](../../.agents/notes/implemented/feature/2026-07-14-cross-family-fs-sandbox.md)），而 `fs-e2b` 则把文件状态置于与 E2B 进程管理提供方共享的远程执行世界中（[决策](../../.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md)）。政策（`fs-policy/`）是一个只通过 `fs/*` 事件门禁参与的插件，不是工具注入的服务；因此移除它会平稳失去政策，留下不受约束的裸提供方，而不会破坏工具。加载 `tool-fs/` 的部署也应加载该插件。模式围栏与编辑前读取门禁彼此正交，可以组合。发现（`tool-fs-search/`）有意不扩展提供方 seam：搜索是在 bash 执行器上运行 `rg`、由进程支持的工作流，因此文件系统后端无需承担通用搜索契约；只有当执行器能找到 `rg` 时，其工具才会注册。如果 bash 工作目录与 `read` 根目录是同一工作区，结果就能继续读取，这也是其 README 所述的共置部署。 + +## 文件 I/O 不设超时 + +`read`/`write`/`edit` **不** 接受 `timeoutMs`，提供方 seam 也不启动 deadline。这与 bash 和 web（两者使用 [`@deepseek-ai/dsh-timeout`](../util/timeout/README.md)）及基于 bash 的 `glob`/`grep` 不同（它们声明的 `timeoutMs` 由 `@deepseek-ai/dsh-timeout-policy` 强制执行）：这些工作由进程支持，deadline 可以实际终止工作。本地系统调用至多只能尽力中止：超时无法强制正在进行的 `fsync`/`rename` 停止，因此这里的 deadline 会成为无法兑现承诺的配置项。在此添加 deadline 还会在「显式优于隐式」明确禁止的地方引入隐式默认值。两个参考 agent（Claude Code、Codex）出于同一原因都不为文件 I/O 计时；取消仍通过工具执行信号传播，在系统调用边界尽力中止。 diff --git a/packages/fs/fs-local/README.i18n.yaml b/packages/fs/fs-local/README.i18n.yaml index cc14be584f..250b94dacf 100644 --- a/packages/fs/fs-local/README.i18n.yaml +++ b/packages/fs/fs-local/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/fs/fs-local/README.md -README.md: 6d344fa3fef7f6bda6c0daa50184661156a925a7 -README.zh.md: 4c94de64561d805684f91f02d5b0dfc375f0d04f +README.md: 2e934298ceff75440357b0742770010c8b1c3904 +README.zh.md: 14f4867ce0f9eaae2a1dea98dbd7d401c98d4535 diff --git a/packages/fs/fs-local/README.md b/packages/fs/fs-local/README.md index 6d344fa3fe..2e934298ce 100644 --- a/packages/fs/fs-local/README.md +++ b/packages/fs/fs-local/README.md @@ -2,7 +2,7 @@ English | [中文](README.zh.md) -The **local-filesystem implementation** of the `ctx.fs` provider seam ([`@deepseek-ai/dsh-fs`](../fs)). Backs the eight `FileSystem` primitives with the host filesystem; loading it as a plugin populates `ctx.fs`. +The **local-filesystem implementation** of the `ctx.fs` provider seam ([`@deepseek-ai/dsh-fs`](../fs)). Backs the eleven `FileSystem` primitives with the host filesystem; loading it as a plugin populates `ctx.fs`. ```ts ignore-check import { LocalFileSystem } from '@deepseek-ai/dsh-fs-local' @@ -15,8 +15,9 @@ await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) ## Behavior - **`resolve(path, opts?)`** — a relative `path` resolves against `opts.cwd` when the caller supplies one (the model-facing tools pass the calling agent's session cwd — see [the per-session cwd Agent Note](../../../.agents/notes/implemented/architecture/2026-07-02-fs-per-session-cwd.md)), else `config.cwd` (default `process.cwd()`); an absolute `path` ignores both. `opts.signal` is checked before and after local resolution, while a remote sibling backend may use it to abort its round-trip. The `targetKey` is the file's `realpath`, so two input paths reaching the same file through symlinks share one identity, and writes/edits land on the link target (preserving the link). A not-yet-existing path uses the realpathed parent directory plus basename when the parent exists; only an unresolvable parent falls back to the absolute path. `displayPath` is the absolute (un-resolved) path. +- **Execution-world coordinates** — `processPath` exposes the target's canonical host path, `fileUrl` encodes that path through Node's platform-aware URL conversion, and `contains` uses platform path semantics to test identity or descendant containment without consumers parsing `targetKey`. - **`stat` / `lstat`** — return target metadata or `undefined` when absent. `stat` reports `FsInfo` for an already resolved target (`version` = an opaque token derived from bigint `dev:ino:size:mtimeNs:ctimeNs`, `type` of `file`/`directory`/`other`, byte `size`); path-shaped `lstat` reports `FsPathInfo` without following the final symlink and can therefore return `symlink`. Both check cancellation before and after their asynchronous metadata probe, so an abort that lands in flight reports `FS_ABORTED` rather than stale absence. -- **`readText` / `streamText`** — UTF-8 only. `readText` reads the whole file; `streamText` streams it in chunks (cross-chunk decoding) so a huge file never has to be held whole in memory. Both reject invalid UTF-8 and NUL-byte binary samples (`FS_NOT_TEXT`) and non-regular targets. The `read` tool (`@deepseek-ai/dsh-tool-fs`) decides which to call by size and owns the line windowing. +- **`readText` / `streamText`** — UTF-8 only. `readText` reads the whole file; `streamText` decodes chunks so a huge file need not be held whole in memory and consumers can enforce their own retention bounds. Both reject invalid UTF-8 and NUL-byte binary samples (`FS_NOT_TEXT`) and non-regular targets. The `read` tool (`@deepseek-ai/dsh-tool-fs`) owns line windowing. - **`listDir`** — lists one directory level in stable `name.localeCompare()` order. Each entry carries the child basename, type, resolved child target (`displayPath` under the listed directory, `targetKey` as the realpath identity), and cheap stat metadata (`version`, plus `size` for regular files). It never opens or decodes file contents. Missing targets report `FS_NOT_FOUND`, file/special-file targets report `FS_NOT_DIRECTORY`, aborted calls report `FS_ABORTED`, permission failures report `FS_PERMISSION_DENIED`, and other listing or child metadata I/O failures report `FS_IO_ERROR`. Broken/disappeared children are returned as `other` without metadata, but permission/IO failures while resolving a child fail the whole listing with a structured `FsError`. - **`writeText`** — atomic: writes to a temp file opened exclusively (`wx`, `0o600`) inside a randomly-named private staging dir (`0o700`) next to the target, fsyncs, then renames over the target. An existing file's mode is preserved, while new files default to `0o600`; on Windows a new file inherits the destination directory's DACL, while replacement copies the target DACL onto the empty temp before writing and publishes through `ReplaceFileW` so the original access policy survives ([Windows DACL preservation Agent Note](../../../.agents/notes/implemented/bug-fix/2026-07-19-windows-atomic-write-dacl-preservation.md)). The `expected` guard is OPTIONAL: omitting it unconditionally creates-or-overwrites; `createIfAbsent` creates a missing target and rejects an existing one (`FS_NOT_OBSERVED`); `replaceIfVersion` replaces only at the observed version (a missing target or mismatch is `FS_STALE_VERSION`). - **`editText`** — atomic literal read-modify-write over the same primitive, serialized per target by a mutation lock. The `expected` guard is OPTIONAL: when supplied it verifies the version BEFORE literal matching (a stale edit reports `FS_STALE_VERSION`, never `FS_EDIT_NOT_FOUND`/`FS_AMBIGUOUS_EDIT` against newer content); omitting it edits the current content unconditionally. A missing target reports `FS_STALE_VERSION` either way. LF-normalizes for matching, restores the file's dominant CRLF/LF style, and rejects empty `oldString` / zero matches (`FS_EDIT_NOT_FOUND`) or ambiguous multi-matches without `replace_all` (`FS_AMBIGUOUS_EDIT`). @@ -35,7 +36,7 @@ No direct invalidation; the named consumer owns any request-prefix changes. - **`config.cwd` is not a sandbox** — it is a resolution default, not containment: absolute paths and `..` escape it. Enforce containment with a stricter `ctx.fs` backend or a permission plugin on the `tools/execute` waterfall ([capability-seam Agent Note](../../../.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.md#consequences)). - **An overwrite reads the whole prior file into memory** — solely as the UI diff basis; bounding that pre-read above a size threshold is deferred (`TODO(overwrite-diff-bound)`). -- **Version tokens are `mtimeMs:size`** — an external change that preserves both within the filesystem's timestamp granularity defeats the stale guard. +- **Version tokens depend on filesystem metadata** — they combine device, inode, size, nanosecond mtime, and nanosecond ctime; a storage layer that cannot update any of those facts for a rewrite can still defeat the stale guard. - **`editText` holds the whole file (plus the edited copy) in memory** — streaming exists only on the read path. - **Binary detection is asymmetric** — reads NUL-sample only the first 8192 bytes while edits scan the whole buffer, so a file with a late NUL reads fine but rejects edits. - **The per-target mutation lock is in-process only** — a writer in another process is caught only by the optional version guard, never serialized. diff --git a/packages/fs/fs-local/README.zh.md b/packages/fs/fs-local/README.zh.md index 4c94de6456..14f4867ce0 100644 --- a/packages/fs/fs-local/README.zh.md +++ b/packages/fs/fs-local/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -`ctx.fs` 提供方 seam（[`@deepseek-ai/dsh-fs`](../fs)）的**本地文件系统实现**。它使用宿主文件系统支持八个 `FileSystem` 原语；将其作为插件加载会填充 `ctx.fs`。 +`ctx.fs` 提供方 seam（[`@deepseek-ai/dsh-fs`](../fs)）的**本地文件系统实现**。它使用宿主文件系统支持十一个 `FileSystem` 原语；将其作为插件加载会填充 `ctx.fs`。 ```ts ignore-check import { LocalFileSystem } from '@deepseek-ai/dsh-fs-local' @@ -15,27 +15,28 @@ await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) ## 行为 - **`resolve(path, opts?)`**：相对 `path` 在调用方提供 `opts.cwd` 时以该值为基准解析（面向模型的工具会传入调用 agent（智能体）的会话 cwd；见[每会话 cwd Agent Note](../../../.agents/notes/implemented/architecture/2026-07-02-fs-per-session-cwd.md)），否则以 `config.cwd` 为基准（默认 `process.cwd()`）；绝对 `path` 会忽略两者。`opts.signal` 会在本地解析前后检查，远程同级后端则可以用它中止往返。`targetKey` 是文件的 `realpath`，因此经符号链接到达同一文件的两个输入路径会共享一个身份，写入/编辑落在链接目标上，同时保留链接。尚不存在的路径在父目录存在时使用 realpath 后的父目录加 basename；只有父目录无法解析时才回退到绝对路径。`displayPath` 是绝对但未经解析的路径。 -- **`stat` / `lstat`**：返回目标元数据；目标不存在时返回 `undefined`。`stat` 为已解析目标报告 `FsInfo`（`version` 是由 bigint `dev:ino:size:mtimeNs:ctimeNs` 派生的不透明 token，`type` 为 `file`/`directory`/`other`，`size` 以字节计）；路径形态的 `lstat` 不跟随最后一个符号链接，报告 `FsPathInfo`，因此可以返回 `symlink`。两者都会在异步元数据探测前后检查取消，因此异步探测进行期间发生的中止会报告 `FS_ABORTED`，而非已失效的「不存在」结果。 -- **`readText` / `streamText`**：只支持 UTF-8。`readText` 读取整个文件；`streamText` 按分片流式读取（跨分片解码），因此超大文件无需整体保存在内存中。两者都会拒绝无效 UTF-8、包含 NUL 字节的二进制样本（`FS_NOT_TEXT`）以及非普通文件目标。`read` 工具（`@deepseek-ai/dsh-tool-fs`）按大小决定调用哪个方法，并负责行窗口逻辑。 +- **执行世界坐标**：`processPath` 公开目标的规范化宿主路径，`fileUrl` 通过 Node 的平台感知 URL 转换对该路径编码，`contains` 则使用平台路径语义检查身份相等或后代包含关系，消费方无需解析 `targetKey`。 +- **`stat` / `lstat`**：返回目标元数据；目标不存在时返回 `undefined`。`stat` 为已解析目标报告 `FsInfo`（`version` 是由 bigint `dev:ino:size:mtimeNs:ctimeNs` 派生的不透明 token，`type` 为 `file`/`directory`/`other`，`size` 以字节计）；路径形态的 `lstat` 不跟随最后一个符号链接，报告 `FsPathInfo`，因此可以返回 `symlink`。两者都会在异步元数据探测前后检查取消，因此飞行中的中止会报告 `FS_ABORTED`，而非陈旧的不存在结果。 +- **`readText` / `streamText`**：只支持 UTF-8。`readText` 读取整个文件；`streamText` 按分片解码，因此超大文件无需整体保存在内存中，消费方也可以执行各自的保留上限。两者都会拒绝无效 UTF-8、包含 NUL 字节的二进制样本（`FS_NOT_TEXT`）以及非普通文件目标。`read` 工具（`@deepseek-ai/dsh-tool-fs`）拥有行窗口逻辑。 - **`listDir`**：按稳定的 `name.localeCompare()` 顺序列出一层目录。每个条目携带子项 basename、类型、解析后的子目标（`displayPath` 位于所列目录下，`targetKey` 是 realpath 身份）和低成本 stat 元数据（`version`，普通文件另有 `size`）。它绝不会打开或解码文件内容。缺失目标报告 `FS_NOT_FOUND`，文件/特殊文件目标报告 `FS_NOT_DIRECTORY`，已中止调用报告 `FS_ABORTED`，权限失败报告 `FS_PERMISSION_DENIED`，其他列出或子项元数据 I/O 失败报告 `FS_IO_ERROR`。损坏/消失的子项以无元数据的 `other` 返回，但解析子项时出现权限/I/O 失败会让整个列表以结构化 `FsError` 失败。 -- **`writeText`**：原子写入。它会向排他打开的临时文件（`wx`、`0o600`）写入；该文件位于目标旁随机命名的私有暂存目录（`0o700`）内。完成写入和 fsync 后，以 rename 覆盖目标。现有文件的 mode 会保留，新文件默认为 `0o600`；Windows 上的新文件继承目标目录的 DACL，而替换会在写入前把目标 DACL 复制到空临时文件，并通过 `ReplaceFileW` 发布，使原访问策略得以保留（见 [Windows DACL 保留 Agent Note](../../../.agents/notes/implemented/bug-fix/2026-07-19-windows-atomic-write-dacl-preservation.md)）。`expected` 防护是可选（OPTIONAL）的：省略时无条件创建或覆盖；`createIfAbsent` 创建缺失目标并拒绝现有目标（`FS_NOT_OBSERVED`）；`replaceIfVersion` 只在观察到的版本上替换（目标缺失或版本不匹配均为 `FS_STALE_VERSION`）。 -- **`editText`**：在同一原语之上执行原子式的字面量读取-修改-写入，并通过变更锁按目标串行化。`expected` 防护是可选（OPTIONAL）的：提供时，会在字面量匹配之前校验版本（陈旧编辑报告 `FS_STALE_VERSION`，绝不会针对较新内容报告 `FS_EDIT_NOT_FOUND`/`FS_AMBIGUOUS_EDIT`）；省略时，无条件编辑当前内容。无论哪种情况，目标缺失都报告 `FS_STALE_VERSION`。匹配时规范化为 LF，随后恢复文件主要的 CRLF/LF 风格；空 `oldString` / 零匹配报告 `FS_EDIT_NOT_FOUND`，未设置 `replace_all` 的多个匹配则报告 `FS_AMBIGUOUS_EDIT`。 +- **`writeText`**：原子写入。它会向排他打开的临时文件（`wx`、`0o600`）写入；该文件位于目标旁随机命名的私有暂存目录（`0o700`）内。完成写入和 fsync 后，以 rename 覆盖目标。现有文件的 mode 会保留，新文件默认为 `0o600`；Windows 上的新文件继承目标目录的 DACL，而替换会在写入前把目标 DACL 复制到空临时文件，并通过 `ReplaceFileW` 发布，使原访问政策得以保留（见 [Windows DACL 保留 Agent Note](../../../.agents/notes/implemented/bug-fix/2026-07-19-windows-atomic-write-dacl-preservation.md)）。`expected` 防护是可选的：省略时无条件创建或覆盖；`createIfAbsent` 创建缺失目标并拒绝现有目标（`FS_NOT_OBSERVED`）；`replaceIfVersion` 只在观察到的版本上替换（目标缺失或版本不匹配均为 `FS_STALE_VERSION`）。 +- **`editText`**：在同一原语之上依次执行原子的字面量读取、修改和写入，并通过变更锁按目标串行化。`expected` 防护是可选的：提供时，会在字面量匹配之前校验版本（陈旧编辑报告 `FS_STALE_VERSION`，绝不会针对较新内容报告 `FS_EDIT_NOT_FOUND`/`FS_AMBIGUOUS_EDIT`）；省略时，无条件编辑当前内容。无论哪种情况，目标缺失都报告 `FS_STALE_VERSION`。匹配时规范化为 LF，随后恢复文件主要的 CRLF/LF 风格；空 `oldString` / 零匹配报告 `FS_EDIT_NOT_FOUND`，未设置 `replace_all` 的多个匹配则报告 `FS_AMBIGUOUS_EDIT`。 -包根目录的 SDK 接口包含默认/具名 `LocalFileSystem` 类和 `Config`。原始 I/O 位于 `src/fsio.ts`（不依赖 Cordis，单独进行单元测试）；`src/index.ts` 是轻量服务接线。 +包根 SDK 接口包含默认/具名 `LocalFileSystem` 类和 `Config`。原始 I/O 位于 `src/fsio.ts`（不依赖 Cordis，单独进行单元测试）；`src/index.ts` 是轻量服务接线。 ## 模型体验 -通过 [`dsh-tool-fs`](../tool-fs/README.md) 间接产生影响；该消费方在有上限的保留结果中渲染本提供方带行窗口的 UTF-8 内容、变更确认和精确提供方消息，而版本、原子写入机制和目录元数据仍属内部实现。 +通过 [`dsh-tool-fs`](../tool-fs/README.md) 间接产生影响；该消费方把本提供方带行窗口的 UTF-8 内容、变更确认和精确提供方消息渲染为有上限且保留的结果，而版本、原子写入机制和目录元数据保持内部可见。 #### KV Cache 影响 不会直接使缓存失效；具名消费方负责请求前缀的任何变化。 -## 已知限制与暂缓事项 +## 已知限制与延期工作 - **`config.cwd` 不是沙箱**：它是解析默认值，而非约束；绝对路径和 `..` 可以逃逸。请使用更严格的 `ctx.fs` 后端或 `tools/execute` waterfall（瀑布式事件）上的权限插件实施约束（见[能力 seam Agent Note](../../../.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.md#consequences)）。 - **覆盖会把整个旧文件读入内存**：只用于 UI diff；在大小阈值之上限制这次预读取的工作延期处理（`TODO(overwrite-diff-bound)`）。 -- **版本 token 是 `mtimeMs:size`**：如果外部变更在文件系统时间戳粒度内保持两者不变，就能绕过陈旧防护。 +- **版本 token 依赖文件系统元数据**：它们组合设备、inode、大小、纳秒级 mtime 和纳秒级 ctime；如果存储层在重写时无法更新其中任何一项事实，仍可能绕过陈旧防护。 - **`editText` 会把整个文件及编辑后的副本保存在内存中**：只有读取路径支持流式处理。 - **二进制检测不对称**：读取只对前 8192 字节执行 NUL 采样，编辑则扫描整个 buffer，因此 NUL 出现在后部的文件可以读取，但编辑会被拒绝。 - **每目标变更锁仅限进程内**：其他进程中的写入方只会被可选版本防护发现，绝不会被串行化。 diff --git a/packages/fs/fs-local/src/index.ts b/packages/fs/fs-local/src/index.ts index 18433f3f7c..f6049b1795 100644 --- a/packages/fs/fs-local/src/index.ts +++ b/packages/fs/fs-local/src/index.ts @@ -5,7 +5,8 @@ */ import { Context } from 'cordis' -import { resolve } from 'node:path' +import { isAbsolute, relative, resolve, sep } from 'node:path' +import { pathToFileURL } from 'node:url' import z from 'schemastery' import { FileSystem, FsError, FsVersion } from '@deepseek-ai/dsh-fs' import type { @@ -90,6 +91,19 @@ export class LocalFileSystem extends FileSystem { return { targetKey: local.targetKey, displayPath: local.displayPath } } + override processPath(target: FsTarget): string { + return String(target.targetKey) + } + + override fileUrl(target: FsTarget): string { + return pathToFileURL(this.processPath(target)).href + } + + override contains(parent: FsTarget, child: FsTarget): boolean { + const path = relative(this.processPath(parent), this.processPath(child)) + return path === '' || (path !== '..' && !path.startsWith(`..${sep}`) && !isAbsolute(path)) + } + override async stat(target: FsTarget, signal?: AbortSignal): Promise { if (signal?.aborted) throw new FsError('stat aborted', 'FS_ABORTED') const info = await probe(target.targetKey) diff --git a/packages/fs/fs-local/tests/filesystem.spec.ts b/packages/fs/fs-local/tests/filesystem.spec.ts index 61e2c0e999..4906bb0adc 100644 --- a/packages/fs/fs-local/tests/filesystem.spec.ts +++ b/packages/fs/fs-local/tests/filesystem.spec.ts @@ -10,6 +10,7 @@ import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' import { mkdir, mkdtemp, readFile, realpath, rm, stat, symlink, unlink, utimes, writeFile } from 'node:fs/promises' import { tmpdir } from 'node:os' import { join } from 'node:path' +import { pathToFileURL } from 'node:url' import { Context } from 'cordis' import { LocalFileSystem } from '@deepseek-ai/dsh-fs-local' import { FsVersion } from '@deepseek-ai/dsh-fs' @@ -85,6 +86,20 @@ describe('resolve', () => { await expect(pending).rejects.toMatchObject({ code: 'FS_ABORTED' }) }) + + it('projects process paths, file URLs, and canonical containment', async () => { + await mkdir(join(dir, 'nested')) + await writeFile(join(dir, 'nested', 'file.txt'), 'text') + const root = await fs.resolve('.') + const child = await fs.resolve('nested/file.txt') + const outside = await fs.resolve('..') + + expect(fs.processPath(child)).toBe(await realpath(join(dir, 'nested', 'file.txt'))) + expect(fs.fileUrl(child)).toBe(pathToFileURL(await realpath(join(dir, 'nested', 'file.txt'))).href) + expect(fs.contains(root, root)).toBe(true) + expect(fs.contains(root, child)).toBe(true) + expect(fs.contains(root, outside)).toBe(false) + }) }) describe('stat', () => { diff --git a/packages/fs/fs/README.i18n.yaml b/packages/fs/fs/README.i18n.yaml index ddf57a98d0..d9b5f0ebc0 100644 --- a/packages/fs/fs/README.i18n.yaml +++ b/packages/fs/fs/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/fs/fs/README.md -README.md: 6c80cc22f6f28e792c3458df3390b241b51d8202 -README.zh.md: 4772d799221efbfff9667cbc8b9e1df6af07dcff +README.md: bf1dd1c1eb65146258cd64e450749845522e7057 +README.zh.md: f3fcc0c3794b972233dc418e93bdd80b1cc8570a diff --git a/packages/fs/fs/README.md b/packages/fs/fs/README.md index 6c80cc22f6..bf1dd1c1eb 100644 --- a/packages/fs/fs/README.md +++ b/packages/fs/fs/README.md @@ -2,21 +2,33 @@ English | [中文](README.zh.md) -The **filesystem provider seam**: an abstract `FileSystem` service (`ctx.fs`) defining the storage primitives a backend provides — resolve a path, stat metadata, no-follow path metadata, read/stream text, list directories, write atomically, and apply a literal edit — without saying HOW. Both mutations take their version guard **optionally**, so `ctx.fs` on its own is a complete, unconstrained text-storage seam. This package also owns the `fs/*` policy event vocabulary the tool dispatches and the policy plugin listens for. +The **filesystem provider seam**: an abstract `FileSystem` service (`ctx.fs`) defining the storage primitives in one execution world — resolve paths, expose canonical process paths and file URIs, test containment, read whole or streaming text, inspect/list metadata, write atomically, and apply a literal edit — without saying HOW. Both mutations take their version guard **optionally**, so `ctx.fs` on its own is a complete, unconstrained text-storage seam. This package also owns the `fs/*` policy event vocabulary the tool dispatches and the policy plugin listens for. -This package is the provider-seam layer of the [filesystem family](../README.md). The [tool](../tool-fs/README.md), [policy](../fs-policy/README.md), and [local](../fs-local/README.md) and [sandboxed](../fs-sandbox/README.md) backends remain separate consumers and implementations; the capability-seam decisions own the split ([foundation](../../../.agents/notes/implemented/architecture/2026-06-13-capability-seams.md), [filesystem seam](../../../.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.md), [provider split](../../../.agents/notes/implemented/simplification/2026-06-26-fsspec-style-fs-seam.md), [event gate](../../../.agents/notes/implemented/architecture/2026-06-26-file-context-as-event-gate.md)). +This package is the provider-seam layer of the four-layer filesystem stack, split so each concern can evolve (and be swapped) independently (see [the capability-seam Agent Note](../../../.agents/notes/implemented/architecture/2026-06-13-capability-seams.md), [the filesystem capability-seam Agent Note](../../../.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.md), [the split-the-filesystem-seam Agent Note](../../../.agents/notes/implemented/simplification/2026-06-26-fsspec-style-fs-seam.md), and [the file-context event-gate Agent Note](../../../.agents/notes/implemented/architecture/2026-06-26-file-context-as-event-gate.md)): + +| Layer | Package | Role | +|---|---|---| +| tool / executor | `@deepseek-ai/dsh-tool-fs` | model-facing `read`/`write`/`edit` schemas + read windowing + text rendering; reads/writes/edits via `ctx.fs`, dispatches the `fs/*` events | +| policy | `@deepseek-ai/dsh-fs-policy` | observed-state + read-before-edit + version-guarded write/edit, contributed through the `fs/*` event gate (no service) | +| provider seam | `@deepseek-ai/dsh-fs` (this) | `ctx.fs`: execution-world paths, text IO, and atomic mutation primitives (optional version guard); owns the `fs/*` event vocabulary | +| provider | `@deepseek-ai/dsh-fs-local` | the host-filesystem implementation | + +A future sandboxed, virtual, or remote backend implements this interface and the policy/tool layers don't change. ## Service API (`ctx.fs`) -A backend subclasses `FileSystem` and implements eight primitives. +A backend subclasses `FileSystem` and implements eleven primitives. | Member | Semantics | |---|---| | `resolve(path, opts?)` | Resolve a path into a stable `FsTarget` (opaque `targetKey`, `displayPath`). `opts.cwd` is the base a relative `path` resolves against (a caller supplies its session workspace; absolute paths ignore it; omitted ⇒ the backend default), while `opts.signal` aborts a backend round-trip. Async — a remote backend may need I/O. The same file via different paths must yield the same `targetKey`. | +| `processPath(target)` | Return the canonical absolute path that a subprocess in this provider's execution world can open. This is intentionally distinct from opaque `targetKey`. | +| `fileUrl(target)` | Return the canonical `file:` URI in the execution world's platform syntax. The backend, not the host process, owns encoding. | +| `contains(parent, child)` | Test canonical identity/descendant containment without exposing or parsing target keys. Both targets come from this provider. | | `stat(target, signal?)` | Return `FsInfo` metadata (`version`, `type`, optional `size`), or `undefined` when the target is absent. Never content. | | `lstat(path, opts?, signal?)` | Return `FsPathInfo` metadata without following the final path component when it is a symlink. This is path-shaped so consumers can reject repository-owned symlinks before `resolve` follows them into a target. | | `readText(target, signal?)` | Read the whole regular text file as one decoded string. Owns regular-file checks, UTF-8 decoding, binary/NUL rejection (`FS_NOT_TEXT`). | -| `streamText(target, signal?)` | Stream the same text as decoded chunks for large files (cross-chunk UTF-8 decoding stays here). | +| `streamText(target, signal?)` | Stream the same text as decoded chunks for large files (cross-chunk UTF-8 decoding stays here); consumers that need a byte ceiling enforce it while consuming the stream. | | `listDir(target, signal?)` | List direct directory children in stable name order. Returns entry names, entry types, resolved child targets, and cheap metadata (`version`/file `size` when available); never reads file contents. Missing targets throw `FS_NOT_FOUND`, non-directories throw `FS_NOT_DIRECTORY`, permission failures throw `FS_PERMISSION_DENIED`, and other backend I/O failures throw `FS_IO_ERROR`. Broken/disappeared children may be returned as `other` without metadata; child permission/IO failures fail the whole listing with the same structured codes. | | `writeText(target, content, expected?, signal?)` | Atomic create/replace. `expected` is OPTIONAL: omit ⇒ unconditional create-or-overwrite; supply an `FsWriteIntent` (`createIfAbsent`/`replaceIfVersion`) to guard. | | `editText(target, edit, expected?, signal?)` | Literal edit. `expected` is OPTIONAL: omit ⇒ unconditional edit of the current content; supply `{ version }` to guard (verified BEFORE matching). A missing target reports `FS_STALE_VERSION` either way. Applies and writes atomically — one mutation critical section. | @@ -37,10 +49,6 @@ This package declares three events (see the generated [events catalog](../../../ `FsTargetKey` / `FsVersion` are branded opaque ids ([the branded-ids Agent Note](../../../.agents/notes/implemented/architecture/2026-06-20-branded-ids.md)) — consumers must not parse `targetKey` or interpret `version`; only `displayPath` is for model/UI output. `FsWriteIntent` is the explicit GUARDED write intent (`createIfAbsent` creates a missing target and rejects an existing one with `FS_NOT_OBSERVED`; `replaceIfVersion` replaces only at the observed version, else `FS_STALE_VERSION`); omitting it from `writeText` is the third, unconditional state. `FsPathInfo` is the no-follow metadata shape that can report `symlink`, unlike target-level `FsInfo`. Failures throw `FsError` (extends `HarnessError`, [the structured error taxonomy Agent Note](../../../.agents/notes/implemented/architecture/2026-06-11-structured-error-taxonomy.md)) carrying a stable `FsErrorCode` (`FS_NOT_FOUND`, `FS_NOT_DIRECTORY`, `FS_NOT_TEXT`, `FS_NOT_REGULAR_FILE`, `FS_PERMISSION_DENIED`, `FS_IO_ERROR`, `FS_STALE_VERSION`, `FS_NOT_OBSERVED`, `FS_AMBIGUOUS_EDIT`, `FS_EDIT_NOT_FOUND`, `FS_ABORTED`); the tool registry surfaces `{ name, code }` on `isError` results. See `src/types.ts` for the full contracts. -## No IO deadline - -Filesystem primitives accept an optional `AbortSignal` but arm no deadline. Local IO is only best-effort abortable: a timeout cannot force an in-progress `fsync` or `rename` to stop, so a fixed deadline would promise control the backend cannot provide. Process-backed discovery owns its separate timeout contract. - ## Model Experience Indirectly, through `dsh-tool-fs`, which renders provider text and errors as bounded, retained filesystem tool results. @@ -52,6 +60,6 @@ No direct invalidation; the named consumer owns any request-prefix changes. ## Known Limitations and Deferred Work - **Text-only by contract** — backends reject binary/non-UTF-8 content with `FS_NOT_TEXT`; binary-safe operations are a deliberate deferral of [the tool-schemas Agent Note](../../../.agents/notes/implemented/feature/2026-06-17-filesystem-tool-schemas.md). -- **Eight primitives only** — no delete, rename/move, copy, or watch; `listDir` is single-level, with recursion, globbing, pagination, and search out of scope per [the directory-listing Agent Note](../../../.agents/notes/archived/architecture/2026-07-03-filesystem-directory-listing-seam.md). -- **No IO deadline** — cancellation is best-effort at primitive boundaries. +- **Eleven primitives only** — no delete, rename/move, copy, or watch; `listDir` is single-level, with recursion, globbing, pagination, and search out of scope per [the directory-listing Agent Note](../../../.agents/notes/archived/architecture/2026-07-03-filesystem-directory-listing-seam.md). +- **No IO deadline** — the seam arms no timeout; cancellation is a best-effort optional `AbortSignal` per primitive (the deliberate [fs-family stance](../README.md)). - **Resolve-then-operate costs a remote backend two round-trips per tool call** — folding or caching resolution is left to such a backend. diff --git a/packages/fs/fs/README.zh.md b/packages/fs/fs/README.zh.md index 4772d79922..f3fcc0c379 100644 --- a/packages/fs/fs/README.zh.md +++ b/packages/fs/fs/README.zh.md @@ -2,56 +2,64 @@ [English](README.md) | 中文 -**文件系统提供方 seam**：抽象 `FileSystem` 服务（`ctx.fs`），定义后端提供的存储原语，包括路径解析、stat 元数据、不跟随链接的路径元数据、读取/流式读取文本、列出目录、原子写入和应用字面量编辑，但不规定实现方式。两个变更操作都**可选**接收版本防护，因此 `ctx.fs` 本身就是完整且不受约束的文本存储 seam。本包还拥有由工具分派、策略插件监听的 `fs/*` 策略事件词汇。 +**文件系统提供方 seam**：抽象 `FileSystem` 服务（`ctx.fs`），定义同一个执行世界中的存储原语，包括解析路径、公开规范化进程路径与文件 URI、检查包含关系、完整或流式读取文本、检查／列出元数据、原子写入和应用字面量编辑，但不规定实现方式。两个变更操作都**可选** 接收版本防护，因此 `ctx.fs` 本身就是完整且不受约束的文本存储 seam。本包还拥有由工具分派、政策插件监听的 `fs/*` 政策事件词汇。 -本包是[文件系统家族](../README.md)中的提供方 seam 层。[工具](../tool-fs/README.md)、[策略](../fs-policy/README.md)、[本地](../fs-local/README.md)与[沙箱化](../fs-sandbox/README.md)后端分别作为消费方与实现保持独立；能力 seam 决策负责该拆分（[基础](../../../.agents/notes/implemented/architecture/2026-06-13-capability-seams.md)、[文件系统 seam](../../../.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.md)、[提供方拆分](../../../.agents/notes/implemented/simplification/2026-06-26-fsspec-style-fs-seam.md)、[事件门禁](../../../.agents/notes/implemented/architecture/2026-06-26-file-context-as-event-gate.md)）。 +本包是四层文件系统栈中的提供方 seam 层；该拆分使每个关注点可以独立演进和替换（见[能力 seam Agent Note](../../../.agents/notes/implemented/architecture/2026-06-13-capability-seams.md)、[文件系统能力 seam Agent Note](../../../.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.md)、[拆分文件系统 seam Agent Note](../../../.agents/notes/implemented/simplification/2026-06-26-fsspec-style-fs-seam.md)和[文件上下文事件门禁 Agent Note](../../../.agents/notes/implemented/architecture/2026-06-26-file-context-as-event-gate.md)）： + +| 层 | 包 | 角色 | +|---|---|---| +| 工具/执行器 | `@deepseek-ai/dsh-tool-fs` | 面向模型的 `read`/`write`/`edit` schema、读取窗口和文本渲染；通过 `ctx.fs` 读取/写入/编辑，并分派 `fs/*` 事件 | +| 政策 | `@deepseek-ai/dsh-fs-policy` | 已观察状态、编辑前读取和版本防护的写入/编辑，通过 `fs/*` 事件门禁贡献（无服务） | +| 提供方 seam | `@deepseek-ai/dsh-fs`（本包） | `ctx.fs`：执行世界路径、文本 I/O 与原子变更原语（可选版本防护）；拥有 `fs/*` 事件词汇 | +| 提供方 | `@deepseek-ai/dsh-fs-local` | 宿主文件系统实现 | + +未来的沙箱化、虚拟或远程后端只需实现该接口，政策层和工具层无需改变。 ## 服务 API（`ctx.fs`） -后端继承 `FileSystem` 并实现八个原语。 +后端继承 `FileSystem` 并实现十一个原语。 | 成员 | 语义 | |---|---| | `resolve(path, opts?)` | 把路径解析为稳定的 `FsTarget`（不透明 `targetKey`、`displayPath`）。`opts.cwd` 是相对 `path` 解析所依据的基准（调用方提供其会话工作区；绝对路径忽略该值；省略时使用后端默认值），`opts.signal` 则中止后端往返。该方法是异步的，因为远程后端可能需要 I/O。经不同路径到达的同一文件必须产生相同 `targetKey`。 | +| `processPath(target)` | 返回该提供方执行世界中的子进程可以打开的规范化绝对路径。该路径有意与不透明的 `targetKey` 分离。 | +| `fileUrl(target)` | 返回采用执行世界平台语法的规范化 `file:` URI。编码由后端而非宿主进程负责。 | +| `contains(parent, child)` | 在不公开或解析目标 key 的情况下，检查规范化身份相等或后代包含关系。两个目标都来自该提供方。 | | `stat(target, signal?)` | 返回 `FsInfo` 元数据（`version`、`type`、可选 `size`）；目标不存在时返回 `undefined`。绝不返回内容。 | -| `lstat(path, opts?, signal?)` | 当最后一个路径组件是符号链接时，不跟随该组件，返回 `FsPathInfo` 元数据。该方法采用路径形态，使消费方能在 `resolve` 跟随仓库自有的符号链接进入目标前拒绝它。 | +| `lstat(path, opts?, signal?)` | 当最后一个路径组件是符号链接时，不跟随该组件，返回 `FsPathInfo` 元数据。该方法采用路径形态，使消费方能在 `resolve` 跟随仓库所有的符号链接进入目标前拒绝它。 | | `readText(target, signal?)` | 把整个普通文本文件读取为一个解码后的字符串。负责普通文件检查、UTF-8 解码和二进制/NUL 拒绝（`FS_NOT_TEXT`）。 | -| `streamText(target, signal?)` | 为大文件按解码后的分片流式读取相同文本（跨分片 UTF-8 解码仍由此处负责）。 | -| `listDir(target, signal?)` | 按稳定名称顺序列出直接子项。返回条目名称、条目类型、解析后的子目标和低成本元数据（若可用则包括 `version`/文件 `size`）；绝不读取文件内容。缺失目标抛出 `FS_NOT_FOUND`，非目录抛出 `FS_NOT_DIRECTORY`，权限失败抛出 `FS_PERMISSION_DENIED`，其他后端 I/O 失败抛出 `FS_IO_ERROR`。损坏/消失的子项可以作为无元数据的 `other` 返回；子项权限/I/O 失败会使用相同结构化代码使整个列出操作失败。 | +| `streamText(target, signal?)` | 为大文件按解码后的分片流式读取相同文本（跨分片 UTF-8 解码仍由此处负责）；需要字节上限的消费方在消费流时执行该上限。 | +| `listDir(target, signal?)` | 按稳定名称顺序列出直接子项。返回条目名称、条目类型、解析后的子目标和低成本元数据（若可用则包括 `version`/文件 `size`）；绝不读取文件内容。缺失目标抛出 `FS_NOT_FOUND`，非目录抛出 `FS_NOT_DIRECTORY`，权限失败抛出 `FS_PERMISSION_DENIED`，其他后端 I/O 失败抛出 `FS_IO_ERROR`。损坏/消失的子项可以作为无元数据的 `other` 返回；子项权限/I/O 失败会使用相同结构化代码使整个列表失败。 | | `writeText(target, content, expected?, signal?)` | 原子创建/替换。`expected` 是可选的：省略 ⇒ 无条件创建或覆盖；提供 `FsWriteIntent`（`createIfAbsent`/`replaceIfVersion`）⇒ 添加防护。 | | `editText(target, edit, expected?, signal?)` | 字面量编辑。`expected` 是可选的：省略 ⇒ 无条件编辑当前内容；提供 `{ version }` ⇒ 添加防护，并在匹配之前校验。无论哪种情况，目标缺失都报告 `FS_STALE_VERSION`。应用和写入以原子方式完成，使用同一个变更临界区。 | 无论是否有版本防护，变更都在后端的每目标锁内运行，因此无条件写入/编辑仍是原子的；「无条件」只移除*版本*前置条件，不移除原子性。 -## `fs/*` 策略事件 +## `fs/*` 政策事件 -本包声明三个事件（见已生成的[事件目录](../../../docs/cordis-catalog/events.md)），使发出方（`@deepseek-ai/dsh-tool-fs`）和策略监听器（`@deepseek-ai/dsh-fs-policy`）共享词汇，而无需让发出方依赖策略插件。`fs/write-intent` 和 `fs/edit-intent` 是单槽决策 waterfall（瀑布式事件）（监听器完整决策，绝不调用 `next()`）；`fs/observed` 是发后即忘的记录事件。它们只携带 `dsh-fs` 词汇和一个不透明 `object` 参与者，不含面向模型的概念或 agent（智能体）/会话所有者结构。 +本包声明三个事件（见已生成的[事件目录](../../../docs/cordis-catalog/events.md)），使发出方（`@deepseek-ai/dsh-tool-fs`）和政策监听器（`@deepseek-ai/dsh-fs-policy`）共享词汇，而无需让发出方依赖政策插件。`fs/write-intent` 和 `fs/edit-intent` 是单槽决策 waterfall（监听器完整决策，绝不调用 `next()`）；`fs/observed` 是发后即忘的记录事件。它们只携带 `dsh-fs` 词汇和一个不透明 `object` 参与者，不含面向模型的概念或 agent（智能体）/会话所有者结构。 -## 提供方 seam，不是策略层 +## 提供方 seam，不是政策层 -`ctx.fs` 有意接近 fsspec 风格的存储原语，比字节级 `cat`/`open` 高半层，因为它会解码文本并拒绝二进制，使策略层绝不接触原始字节。它负责 UTF-8 解码、二进制拒绝、原子写入和字面量编辑临界区。它**不**负责行窗口、编号行、渲染 footer 或已观察状态。已观察状态、编辑前读取和版本防护的写入/编辑属于插件（`@deepseek-ai/dsh-fs-policy`）通过提供可选防护而添加的策略，并非提供方行为，因此沙箱化/远程后端不会继承任何面向模型的观察策略。 +`ctx.fs` 有意接近 fsspec 风格的存储原语，比字节级 `cat`/`open` 高半层，因为它会解码文本并拒绝二进制，使政策层绝不接触原始字节。它负责 UTF-8 解码、二进制拒绝、原子写入和字面量编辑临界区。它**不** 负责行窗口、编号行、渲染 footer 或已观察状态。已观察状态、编辑前读取和版本防护的写入/编辑属于插件（`@deepseek-ai/dsh-fs-policy`）通过提供可选防护而添加的政策，并非提供方行为，因此沙箱化/远程后端不会继承任何面向模型的观察政策。 -`editText` 留在该 seam 上，不由策略层通过读取加写入组合，因为版本防护、字面量匹配和原子重写必须处于同一临界区内，才能正确归因错误并实现一方胜出/一方陈旧的并发；远程后端也可以将其实现为原生比较并编辑操作。 +`editText` 留在该 seam 上，不由政策层通过读取加写入组合，因为版本防护、字面量匹配和原子重写必须处于同一临界区内，才能正确归因错误并实现一方胜出/一方陈旧的并发；远程后端也可以将其实现为原生比较并编辑操作。 ## 词汇 `FsTargetKey` / `FsVersion` 是带品牌的不透明 id（见[品牌 id Agent Note](../../../.agents/notes/implemented/architecture/2026-06-20-branded-ids.md)）；消费方不得解析 `targetKey` 或解释 `version`，只有 `displayPath` 用于模型/UI 输出。`FsWriteIntent` 是显式的防护写入意图（`createIfAbsent` 创建缺失目标，并以 `FS_NOT_OBSERVED` 拒绝现有目标；`replaceIfVersion` 只在观察版本上替换，否则为 `FS_STALE_VERSION`）；从 `writeText` 中省略该值就是第三种无条件状态。`FsPathInfo` 是可报告 `symlink` 的不跟随链接元数据形态，区别于目标级 `FsInfo`。失败会抛出 `FsError`（继承 `HarnessError`；见[结构化错误分类 Agent Note](../../../.agents/notes/implemented/architecture/2026-06-11-structured-error-taxonomy.md)），并携带稳定的 `FsErrorCode`（`FS_NOT_FOUND`、`FS_NOT_DIRECTORY`、`FS_NOT_TEXT`、`FS_NOT_REGULAR_FILE`、`FS_PERMISSION_DENIED`、`FS_IO_ERROR`、`FS_STALE_VERSION`、`FS_NOT_OBSERVED`、`FS_AMBIGUOUS_EDIT`、`FS_EDIT_NOT_FOUND`、`FS_ABORTED`）；工具注册表公开 `{ name, code }`，并将其附在 `isError` 结果上。完整契约见 `src/types.ts`。 -## 无 I/O deadline - -文件系统原语接受可选 `AbortSignal`，但不会启动 deadline。本地 I/O 只能尽力取消：超时无法强制进行中的 `fsync` 或 `rename` 停止，因此固定 deadline 会承诺后端无法提供的控制能力。基于进程的发现功能拥有独立的超时契约。 - ## 模型体验 通过 `dsh-tool-fs` 间接产生影响；该消费方把提供方文本和错误渲染为有界且保留的文件系统工具结果。 #### KV Cache 影响 -不会直接使缓存失效；上述消费方负责请求前缀的任何变化。 +不会直接使缓存失效；具名消费方负责请求前缀的任何变化。 -## 已知限制与暂缓事项 +## 已知限制与延期工作 - **契约只支持文本**：后端以 `FS_NOT_TEXT` 拒绝二进制/非 UTF-8 内容；二进制安全操作是[工具 schema Agent Note](../../../.agents/notes/implemented/feature/2026-06-17-filesystem-tool-schemas.md)有意延期的工作。 -- **只有八个原语**：没有删除、重命名/移动、复制或监视；`listDir` 只支持一层，递归、glob、分页和搜索不在范围内，见[目录列出 Agent Note](../../../.agents/notes/archived/architecture/2026-07-03-filesystem-directory-listing-seam.md)。 -- **没有 I/O deadline**：取消只能在原语边界尽力执行。 +- **只有十一个原语**：没有删除、重命名/移动、复制或监视；`listDir` 只支持一层，递归、glob、分页和搜索不在范围内，见[目录列出 Agent Note](../../../.agents/notes/archived/architecture/2026-07-03-filesystem-directory-listing-seam.md)。 +- **没有 I/O deadline**：该 seam 不启动超时；取消只是每个原语上尽力而为的可选 `AbortSignal`（见有意采用的 [fs 能力族立场](../README.md)）。 - **先解析后操作使远程后端每次工具调用需要两次往返**：折叠或缓存解析由这种后端自行决定。 diff --git a/packages/fs/fs/src/index.ts b/packages/fs/fs/src/index.ts index b43fa48c4b..2d46868a57 100644 --- a/packages/fs/fs/src/index.ts +++ b/packages/fs/fs/src/index.ts @@ -1,8 +1,10 @@ /** - * Filesystem text-storage provider seam. Backends own stable target identity, - * text decoding, binary rejection, and atomic mutations. Read windows and - * observed-state policy stay in consumer and policy plugins; `editText` remains - * here so version check, literal match, and rewrite share one critical section. + * Filesystem provider seam for one execution world. Backends own stable target + * identity, process paths and file URIs, containment, text reads, decoding, + * binary rejection, and atomic mutations. Read windows and + * observed-state policy stay in consumer and policy plugins; `editText` + * remains here so version check, literal match, and rewrite share one critical + * section. * @module @deepseek-ai/dsh-fs */ @@ -83,7 +85,6 @@ export abstract class FileSystem extends Service { super(ctx, 'fs') } - /** /** * The sandbox mode this backend enforces on mutations BY DEFAULT, or * `undefined` when it does not confine at all — the capability fact the tool @@ -111,6 +112,34 @@ export abstract class FileSystem extends Service { */ abstract resolve(path: string, opts?: { cwd?: string; signal?: AbortSignal }): Promise + /** + * Return the canonical absolute path a subprocess in this filesystem's + * execution world can open. The path is deliberately separate from + * {@link FsTarget.targetKey}: consumers may pass this value to another OS + * capability, but must continue treating the target key as opaque. + * @param target - the resolved target whose process path is required. + * @returns an absolute path in the backend's execution world. + */ + abstract processPath(target: FsTarget): string + + /** + * Return the canonical `file:` URI for a target in this filesystem's + * execution world. Backends own URI encoding because the host platform may + * differ from the execution platform. + * @param target - the resolved target to encode. + * @returns the target's canonical file URI. + */ + abstract fileUrl(target: FsTarget): string + + /** + * Test canonical containment without exposing or parsing backend target + * keys. Both targets must come from this provider. + * @param parent - canonical directory target. + * @param child - canonical candidate target. + * @returns true when `child` is `parent` or a descendant of it. + */ + abstract contains(parent: FsTarget, child: FsTarget): boolean + /** * Return target metadata, or `undefined` when the target does not exist. * @param target - the resolved target to stat. diff --git a/packages/fs/fs/tests/service.spec.ts b/packages/fs/fs/tests/service.spec.ts index 19ee033cce..f225e51cc0 100644 --- a/packages/fs/fs/tests/service.spec.ts +++ b/packages/fs/fs/tests/service.spec.ts @@ -19,13 +19,18 @@ import type { FsWriteOutcome, } from '@deepseek-ai/dsh-fs' -/** A minimal in-memory fake implementing the eight provider primitives. */ +/** A minimal in-memory fake implementing the provider primitives. */ class FakeFileSystem extends FileSystem { files = new Map() override async resolve(path: string): Promise { return { targetKey: FsTargetKey(path), displayPath: path } } + override processPath(target: FsTarget): string { return String(target.targetKey) } + override fileUrl(target: FsTarget): string { return `file:///${encodeURIComponent(String(target.targetKey))}` } + override contains(parent: FsTarget, child: FsTarget): boolean { + return child.targetKey === parent.targetKey || String(child.targetKey).startsWith(`${parent.targetKey}/`) + } override async stat(target: FsTarget): Promise { const content = this.files.get(target.targetKey) if (content === undefined) return undefined @@ -75,6 +80,7 @@ describe('FileSystem provider seam', () => { const ctx = new Context() await ctx.plugin(FakeFileSystem) const fs = ctx.fs as FakeFileSystem + expect(fs.sandboxMode).toBeUndefined() fs.files.set('a.txt', 'hi') const target = await fs.resolve('a.txt') expect((await fs.stat(target))?.type).toBe('file') diff --git a/packages/fs/tool-fs-search/tests/tools.spec.ts b/packages/fs/tool-fs-search/tests/tools.spec.ts index e9ae5588ee..c02e4f93db 100644 --- a/packages/fs/tool-fs-search/tests/tools.spec.ts +++ b/packages/fs/tool-fs-search/tests/tools.spec.ts @@ -148,6 +148,8 @@ class FakeHandle implements SubprocessHandle { */ class FakeSubprocess extends SubprocessService { spawns: SubprocessSpawnSpec[] = [] + override async resolveExecutable(command: string): Promise { return command } + override spawnTerminal(): Promise { throw new Error('search tools spawn pipes, never terminals') } handles: FakeHandle[] = [] /** Arms the per-spawn script; a `{ reject }` return scripts a spawn-level failure. */ handler: (spec: SubprocessSpawnSpec) => ScriptedRun | { reject: Error } = () => runResult('') diff --git a/packages/fs/tool-fs/tests/tools.spec.ts b/packages/fs/tool-fs/tests/tools.spec.ts index ad01237c2b..b217260972 100644 --- a/packages/fs/tool-fs/tests/tools.spec.ts +++ b/packages/fs/tool-fs/tests/tools.spec.ts @@ -48,6 +48,11 @@ class FakeFs extends FileSystem { override async resolve(path: string): Promise { return { targetKey: FsTargetKey(`key:${path}`), displayPath: `/abs/${path}` } } + override processPath(target: FsTarget): string { return String(target.targetKey) } + override fileUrl(target: FsTarget): string { return `file://${target.targetKey}` } + override contains(parent: FsTarget, child: FsTarget): boolean { + return child.targetKey === parent.targetKey || String(child.targetKey).startsWith(`${parent.targetKey}/`) + } override async stat(target: FsTarget): Promise { this.throwIfArmed() const content = this.files.get(target.targetKey) diff --git a/packages/lsp/README.i18n.yaml b/packages/lsp/README.i18n.yaml index a41b05f148..9919cdaba5 100644 --- a/packages/lsp/README.i18n.yaml +++ b/packages/lsp/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/lsp/README.md -README.md: 4964d78f1096d1a4bc78fa80c6b5febaf24a5661 -README.zh.md: 93b872002cf1f53cdbb96ff42402bfeb9557575f +README.md: 7fbdf071735673fb0158f6fa66148be1c644a433 +README.zh.md: e059dbd80b7e38c0e447e54178162316dfd127c7 diff --git a/packages/lsp/README.md b/packages/lsp/README.md index 4964d78f10..7fbdf07173 100644 --- a/packages/lsp/README.md +++ b/packages/lsp/README.md @@ -6,8 +6,10 @@ The language-server capability seam: an abstract LSP interface, a generic stdio | Package | Role | ctx key | |---|---|---| -| [`lsp/`](lsp/README.md) | LSP provider seam and shared vocabulary | `ctx.lsp` | -| [`lsp-local/`](lsp-local/README.md) | Local stdio language-server backend | registers providers on `ctx.lsp` | -| [`tool-lsp/`](tool-lsp/README.md) | Model-facing semantic-navigation tool | registers on `ctx.tools` | +| `lsp/` | Abstract LSP seam (provider registry by branded id + extension mapping, per-query selection, vocabulary, `LspError`) | `ctx.lsp` | +| `lsp-local/` | Generic multi-server stdio backend over `ctx.fs` and `ctx.subprocess` (JSON-RPC, transient-open queries) | (registers providers on `ctx.lsp`) | +| `tool-lsp/` | Model-facing `lsp` tool (four operations, one-based UTF-16 cursor coordinates) | (registers on `ctx.tools`) | -Providers register semantic capabilities; the tool owns the model-facing contract. The child READMEs document operation, protocol, and presentation details, while the [LSP capability-seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md) owns the rationale. +The interface lives at `lsp/lsp/`. The seam exposes exactly four semantic operations — `goToDefinition`, `findReferences`, `goToImplementation`, `hover` — and no generic JSON-RPC escape hatch, so a provider swap does not change how the model asks for navigation and no protocol payload or unreviewed mutation reaches the model contract. Providers register **capabilities**, not tools; `tool-lsp` is the only owner of the model-facing name, schema, prompt guidance, and presentation. + +See the [LSP capability seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md) for the design rationale, including why documents open transiently per query, why the stdio host consumes the shared filesystem/subprocess execution world, and why extension ownership is exclusive within one runtime. diff --git a/packages/lsp/README.zh.md b/packages/lsp/README.zh.md index 93b872002c..e059dbd80b 100644 --- a/packages/lsp/README.zh.md +++ b/packages/lsp/README.zh.md @@ -2,12 +2,14 @@ [English](README.md) | 中文 -语言服务器能力 seam：抽象 LSP 接口、通用 stdio 提供方和面向模型的 `lsp` 工具。这些全是**产品**包。 +语言服务器能力 seam：抽象 LSP 接口、通用 stdio 提供方，以及面向模型的 `lsp` 工具。这些全是**产品** 包。 | 包 | 职责 | ctx key | |---|---|---| -| [`lsp/`](lsp/README.md) | LSP 提供方 seam 和共享词汇 | `ctx.lsp` | -| [`lsp-local/`](lsp-local/README.md) | 本地 stdio 语言服务器后端 | 在 `ctx.lsp` 上注册提供方 | -| [`tool-lsp/`](tool-lsp/README.md) | 面向模型的语义导航工具 | 注册到 `ctx.tools` | +| `lsp/` | 抽象 LSP seam（按品牌化 id + 扩展名映射组织的提供方注册表、逐查询选择、词汇、`LspError`） | `ctx.lsp` | +| `lsp-local/` | 基于 `ctx.fs` 与 `ctx.subprocess` 的通用多服务器 stdio 后端（JSON-RPC、临时打开查询） | （在 `ctx.lsp` 上注册提供方） | +| `tool-lsp/` | 面向模型的 `lsp` 工具（四种操作、从 1 开始的 UTF-16 光标坐标） | （注册到 `ctx.tools`） | -提供方注册语义能力；工具负责面向模型的契约。子 README 记录操作、协议和呈现细节，[LSP 能力 seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md)负责设计原理。 +接口位于 `lsp/lsp/`。该 seam 恰好公开四种语义操作：`goToDefinition`、`findReferences`、`goToImplementation`、`hover`，且不提供通用 JSON-RPC 逃生口；因此，替换提供方不会改变模型请求导航的方式，也不会让协议载荷或未经评审的修改进入模型契约。提供方注册的是**能力** 而非工具；`tool-lsp` 是面向模型名称、schema、提示词指引和呈现的唯一 owner。 + +设计原理见 [LSP 能力 seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md)，其中也解释了文档为何在每次查询时临时打开、stdio 主机为何使用共享的文件系统／子进程执行环境，以及扩展名归属为何在同一运行时内互斥。 diff --git a/packages/lsp/lsp-local/README.i18n.yaml b/packages/lsp/lsp-local/README.i18n.yaml index b786ec1a60..03487d5719 100644 --- a/packages/lsp/lsp-local/README.i18n.yaml +++ b/packages/lsp/lsp-local/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/lsp/lsp-local/README.md -README.md: 37676a82fb5d45b40ca86507259aca9509d25a43 -README.zh.md: 9e8b7f4f4395985bdbc29c1d911520b3559d7e0c +README.md: 661c27d3326adfc3408b33550a63fe3ebe183a39 +README.zh.md: 83f0eaa1fb4caccc381a1623b791355b2f5c5b65 diff --git a/packages/lsp/lsp-local/README.md b/packages/lsp/lsp-local/README.md index 37676a82fb..661c27d332 100644 --- a/packages/lsp/lsp-local/README.md +++ b/packages/lsp/lsp-local/README.md @@ -2,18 +2,19 @@ English | [中文](README.zh.md) -A **generic local stdio language-server backend** for `ctx.lsp`. One plugin instance accepts a named server table and registers one isolated provider per entry. This is a generic host, not a language-server catalog or installer — deployments configure commands and mappings explicitly; presets belong in `cordis.yml` overlays. +A **generic stdio language-server backend** for `ctx.lsp`. One plugin instance accepts a named server table and registers one isolated provider per entry. It reads through `ctx.fs` and launches through `ctx.subprocess`, so the server and source always inhabit the mounted execution world. This is a generic host, not a language-server catalog or installer — deployments configure commands and mappings explicitly; presets belong in `cordis.yml` overlays. Namespace plugin (`name` / `inject` / `Config` / `apply`, no default export). ## What it does - Resolves every server-local setting before registration; an invalid mapping or registration conflict rolls back earlier entries, so a failed load leaves no provider routes. -- Lazily single-flights one server process per `(server id, canonical workspace realpath)`. A live server error is not replayed; if the selected pooled transport fails before or during a read-only query, the provider awaits its disposal and retries that query once on a fresh process. -- Uses a compatibility-first **transient-open** sequence per query: canonicalize and read the source with Node APIs, `textDocument/didOpen` (version 1, full text), the requested request, then `textDocument/didClose` in `finally`. A failed or canceled `didOpen` write terminates the instance before the pool can reuse it. Documents close after each call, so the first version needs no `didChange`, content cache, or document LRU. -- Serializes each source-read/open/query/close lifecycle through one abortable per-workspace queue so queued calls read current source only when their turn starts; distinct workspaces run in parallel. +- Lazily single-flights one server process per `(server id, canonical workspace target)`. A live server error is not replayed; if the selected pooled transport fails before or during a read-only query, the provider awaits its disposal and retries that query once on a fresh process. +- Uses a compatibility-first **transient-open** sequence per query: resolve and byte-bound the source while streaming it through `ctx.fs`, `textDocument/didOpen` (version 1, full text), the requested request, then `textDocument/didClose` in `finally`. A failed or canceled `didOpen` write terminates the instance before the pool can reuse it. Documents close after each call, so the first version needs no `didChange`, content cache, or document LRU. +- Serializes each source-read/open/query/close lifecycle through one abortable per-workspace queue so queued calls read current source only when their turn starts; distinct workspaces run in parallel. Provider disposal aborts filesystem and protocol work, awaits workspace lookups that have not entered a queue, then drains every queue and server. - After protocol shutdown fails, terminates the server's descendant tree through the subprocess seam (POSIX process-group signaling; Windows `taskkill /T /F`). Tree-kill delivery is contained like every group signal — it races server exit — and quiescence is confirmed by the handle's tree-liveness wait rather than by the kill's own outcome. -- Reads sources through Node filesystem APIs in the subprocess's host namespace — NOT `ctx.fs`, and emits no `fs/observed`: only the LSP result is model-visible, so a query does not satisfy read-before-write policy. +- Resolves the server executable, cwd, process, and protocol streams through `ctx.subprocess`; `initialize.processId` is `null` because another machine or PID namespace must not monitor the harness process. +- Uses `ctx.fs` canonical containment, file URIs, and streamed text validation, but emits no `fs/observed`: only the LSP result is model-visible, so a query does not satisfy read-before-write policy. ## Configuration @@ -41,7 +42,7 @@ Initialization advertises `general.positionEncodings: ['utf-16']`, `workspace: { ## Security boundary -The provider trusts its configured server and claims no sandbox confinement. It canonicalizes and reads source through Node APIs, rejecting a source that is missing, non-regular, non-UTF-8, oversized, or whose canonical path resolves outside the canonical workspace (symlink aliases share one instance). Result locations may be external, but an external path cannot become a query source. The first implementation therefore requires trusted host-local deployment; restricted, remote, or virtual workspaces require another provider. +The provider trusts its configured server and claims no sandbox confinement. It delegates canonical identity, containment, regular-file streaming, UTF-8 validation, and file-URI encoding to `ctx.fs`; it rejects missing, non-regular, non-UTF-8, oversized, or canonically out-of-workspace query sources before server startup. Containment is evaluated before the stream opens and does not promise stable-handle identity across concurrent path replacement. Result locations may be external, but an external path cannot become a query source. A deployment must mount filesystem and subprocess providers for the same execution world; split-world composition is invalid. ## Model Experience @@ -53,6 +54,7 @@ No direct invalidation; `dsh-tool-lsp` owns request-prefix changes. ## Known Limitations and Deferred Work -- **Trusted host-local only** — no sandbox confinement, no private cache/temp write contract; supporting untrusted binaries or restricted/remote/virtual workspaces requires a later process/filesystem contract and a different provider ([seam Agent Note](../../../.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md)). Containment resolves `realpath`, then opens the source through one handle with `O_NOFOLLOW | O_NONBLOCK` (final-component symlink guard plus nonblocking rejection of FIFOs) and a bounded read; a concurrent mutator that swaps an *ancestor* directory for a symlink between the resolve and the open is an accepted residual TOCTOU under this trusted-deployment model, not closed with non-portable `openat` segment walks. -- **Transient-open compatibility floor** — servers whose synchronization omits open/close (or advertise `None`) are unsupported even if closed-document queries would work; compatibility with one TypeScript server does not imply cross-language support. +- **No confinement policy** — this package trusts the configured server and does not sandbox its process; a restricted deployment must supply appropriate process/filesystem providers or a same-world sandbox wrapper. +- **Transient-open compatibility floor** — servers whose synchronization omits open/close (or advertise `None`) are unsupported even if closed-document queries would work; the pinned TypeScript e2e establishes one compatibility floor, not a cross-language claim. - **Per-server/workspace serialization latency** — parallel agents sharing one server and workspace queue behind one process; long-lived workspace processes consume memory until disposal. +- **A hard-killed harness orphans language servers** — `initialize.processId: null` removes server-side client-PID monitoring, so servers are cleaned only by graceful service disposal; a SIGKILL'd harness leaves them running until they exit on their own. diff --git a/packages/lsp/lsp-local/README.zh.md b/packages/lsp/lsp-local/README.zh.md index 9e8b7f4f43..83f0eaa1fb 100644 --- a/packages/lsp/lsp-local/README.zh.md +++ b/packages/lsp/lsp-local/README.zh.md @@ -2,18 +2,19 @@ [English](README.md) | 中文 -`ctx.lsp` 的**通用本地 stdio 语言服务器后端**。一个插件实例接受一张命名服务器表，并逐配置项注册一个隔离的提供方。这是通用主机，而不是语言服务器目录或安装器：部署需要显式配置命令与映射，预设应放在 `cordis.yml` overlay 中。 +`ctx.lsp` 的**通用 stdio 语言服务器后端**。一个插件实例接受一张命名服务器表，并逐配置项注册一个隔离的提供方。它通过 `ctx.fs` 读取，并通过 `ctx.subprocess` 启动，因此服务器与源文件始终位于已挂载的执行世界中。这是通用主机，而不是语言服务器目录或安装器：部署需要显式配置命令与映射，预设应放在 `cordis.yml` overlay 中。 Namespace 插件（`name`／`inject`／`Config`／`apply`，无默认导出）。 ## 功能 - 在注册前解析每项服务器局部设置；无效映射或注册冲突会回滚较早配置项，因此加载失败不会留下提供方路由。 -- 每个 `(server id, canonical workspace realpath)` 惰性 single-flight 一个服务器进程。存活服务器错误不会回放；如果选中的池化传输在只读查询之前或期间失败，提供方会等待其 dispose（资源释放）完成，并在新进程上重试该查询一次。 -- 每次查询都使用兼容性优先的**临时打开**序列：通过 Node API 规范化并读取源文件、`textDocument/didOpen`（版本 1、完整文本）、所请求操作，然后执行 `textDocument/didClose`，该操作位于 `finally` 中。写入 `didOpen` 失败或取消时，会在池复用该实例前将其终止。文档在每次调用后关闭，因此第一版不需要 `didChange`、内容 cache 或文档 LRU。 -- 通过一条逐 Workspace、可中止的队列，串行执行每个源读取／打开／查询／关闭生命周期，因此排队调用只会在轮到自身时读取当前源；不同 Workspace 并行运行。 +- 每个 `(server id, canonical workspace target)` 惰性 single-flight 一个服务器进程。存活服务器错误不会回放；如果选中的池化传输在只读查询之前或期间失败，提供方会等待其 dispose（资源释放）完成，并在新进程上重试该查询一次。 +- 每次查询都使用兼容性优先的**临时打开**序列：通过 `ctx.fs` 流式读取源文件，同时解析并限制其字节数；随后执行 `textDocument/didOpen`（版本 1、完整文本）、所请求操作，再执行位于 `finally` 中的 `textDocument/didClose`。写入 `didOpen` 失败或取消时，会在池复用该实例前将其终止。文档在每次调用后关闭，因此第一版不需要 `didChange`、内容 cache 或文档 LRU。 +- 通过一条逐 Workspace、可中止的队列，串行执行每个源读取／打开／查询／关闭生命周期，因此排队调用只会在轮到自身时读取当前源；不同 Workspace 并行运行。提供方 dispose 会中止文件系统与协议工作，等待尚未进入队列的 Workspace 查找完成，随后排空每条队列与每个服务器。 - 协议 shutdown 失败后，经由子进程 seam 终止服务器后代树（POSIX 进程组信号；Windows `taskkill /T /F`）。树终止的投递结果与所有进程组信号一样被就地吸收，不向外抛出（投递与服务器退出存在竞态）；服务器是否完全停稳，由句柄的进程树存活等待确认，而非由这次终止自身的结果确认。 -- 通过子进程 host namespace 中的 Node 文件系统 API 读取源文件，绝不使用 `ctx.fs`，也不发出 `fs/observed`：只有 LSP 结果对模型可见，因此查询不满足先读后写策略。 +- 通过 `ctx.subprocess` 解析服务器可执行文件、cwd、进程和协议流；`initialize.processId` 为 `null`，因为另一台机器或 PID namespace 不得监视 harness 进程。 +- 使用 `ctx.fs` 提供的规范化包含关系、文件 URI 与流式文本验证，但不发出 `fs/observed`：只有 LSP 结果对模型可见，因此查询不满足先读后写策略。 ## 配置 @@ -41,7 +42,7 @@ Namespace 插件（`name`／`inject`／`Config`／`apply`，无默认导出） ## 安全边界 -提供方信任其配置的服务器，不提供任何沙箱隔离。它通过 Node API 规范化并读取源文件，拒绝缺失、非普通文件、非 UTF-8、过大，或规范路径位于规范 Workspace 外部的源文件（符号链接别名共享一个实例）。结果位置可以在外部，但外部路径不能成为查询源。因此，第一版要求可信的主机本地部署；受限、远程或虚拟 Workspace 需要另一个提供方。 +提供方信任其配置的服务器，不提供任何沙箱隔离。它把规范化身份、包含关系、普通文件流式读取、UTF-8 验证和文件 URI 编码委托给 `ctx.fs`；并在服务器启动前拒绝缺失、非普通文件、非 UTF-8、过大，或规范化后位于 Workspace 外部的查询源。包含关系在打开流之前评估，不承诺在并发路径替换期间保持稳定句柄身份。结果位置可以在外部，但外部路径不能成为查询源。部署必须挂载描述同一执行世界的文件系统与进程管理提供方；分裂世界组合无效。 ## 模型体验 @@ -53,6 +54,7 @@ Namespace 插件（`name`／`inject`／`Config`／`apply`，无默认导出） ## 已知限制与暂缓事项 -- **仅限可信主机本地环境**：没有沙箱隔离，也没有私有 cache／temp 写入契约；支持不受信任 binary 或受限／远程／虚拟 Workspace，需要后续的进程／文件系统契约及不同提供方（见 [seam Agent Note](../../../.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md)）。限制逻辑先解析 `realpath`，再通过一个带 `O_NOFOLLOW | O_NONBLOCK` 的 handle 打开源文件（最终组件符号链接防护，并以非阻塞方式拒绝 FIFO），同时进行有界读取；并发修改方如果在解析与打开之间把*祖先*目录替换为符号链接，会造成残余 TOCTOU。在该可信部署模型下接受此风险，不使用不可移植的 `openat` 逐 segment 遍历来封闭。 -- **临时打开兼容性下限**：同步能力省略打开／关闭（或声明 `None`）的服务器不受支持，即使关闭文档查询能够工作；与一个 TypeScript 服务器兼容，并不表示支持其他语言。 +- **不提供隔离策略**：本包（package）信任所配置的服务器，不对其进程实施沙箱；受限部署必须提供适当的进程／文件系统提供方，或使用同一执行世界的沙箱包装层。 +- **临时打开兼容性下限**：同步能力省略打开／关闭（或声明 `None`）的服务器不受支持，即使关闭文档查询能够工作；固定的 TypeScript e2e 只建立一项兼容性下限，不代表跨语言承诺。 - **逐服务器／Workspace 串行化延迟**：共享同一个服务器与 Workspace 的并行 agent（智能体）会在一个进程后排队；长生命周期 Workspace 进程会占用内存直到 dispose。 +- **被强制杀死的 harness 会遗留语言服务器**：`initialize.processId: null` 取消了服务器侧的客户端 PID 监视，因此服务器只能由服务的优雅 dispose 清理；被 SIGKILL 的 harness 会让它们继续运行，直到自行退出。 diff --git a/packages/lsp/lsp-local/package.json b/packages/lsp/lsp-local/package.json index b68745d617..a5411bab55 100644 --- a/packages/lsp/lsp-local/package.json +++ b/packages/lsp/lsp-local/package.json @@ -26,6 +26,7 @@ "license": "BSD-3-Clause", "peerDependencies": { "@deepseek-ai/dsh-brand": "^0.0.1", + "@deepseek-ai/dsh-fs": "^0.0.1", "@deepseek-ai/dsh-invariants": "^0.0.1", "@deepseek-ai/dsh-llm": "^0.0.1", "@deepseek-ai/dsh-lsp": "^0.0.1", @@ -38,6 +39,8 @@ }, "devDependencies": { "@deepseek-ai/dsh-brand": "workspace:^", + "@deepseek-ai/dsh-fs": "workspace:^", + "@deepseek-ai/dsh-fs-local": "workspace:^", "@deepseek-ai/dsh-invariants": "workspace:^", "@deepseek-ai/dsh-llm": "workspace:^", "@deepseek-ai/dsh-lsp": "workspace:^", diff --git a/packages/lsp/lsp-local/src/connection.ts b/packages/lsp/lsp-local/src/connection.ts index e19718eed6..4be75cbd33 100644 --- a/packages/lsp/lsp-local/src/connection.ts +++ b/packages/lsp/lsp-local/src/connection.ts @@ -22,7 +22,7 @@ export interface ConnectionSpec { readonly args: readonly string[] /** The child's working directory (the canonical workspace). */ readonly cwd: string - /** The child's environment (credential-scrubbed, with overrides applied). */ + /** Explicit child environment overrides; the subprocess provider owns its ambient scrub. */ readonly env: Record /** Largest single framed message accepted from the server. */ readonly maxMessageBytes: number @@ -98,9 +98,8 @@ export class LspConnection { stderr: { maxBytes: spec.maxStderrBytes }, }, graceMs: spec.killGraceMs, - // spec.env mixes the scrubbed base with explicit config entries; the - // seam merges the whole map after its own ambient scrub, so a - // configured DSH_* fact reaches the child. + // The seam merges explicit config entries after its ambient scrub, so a + // configured credential or DSH_* fact reaches the child deliberately. env: spec.env, }) /* v8 ignore start -- 'pipe' dispositions expose both streams by the seam contract; defensive. */ diff --git a/packages/lsp/lsp-local/src/host.ts b/packages/lsp/lsp-local/src/host.ts index 11996908ac..ef4b56809d 100644 --- a/packages/lsp/lsp-local/src/host.ts +++ b/packages/lsp/lsp-local/src/host.ts @@ -1,154 +1,124 @@ -/** - * Host-filesystem source access for the local provider, using Node APIs directly in the - * subprocess's namespace (never `ctx.fs`): only the LSP result is model-visible, so a query does not - * satisfy read-before-write policy and emits no `fs/observed`. Canonicalization derives target - * identity from `realpath`, so symlink aliases share a workspace; a source is rejected before server - * startup when it is missing, non-regular, non-UTF-8, oversized, or canonically outside the - * workspace. External result locations are allowed, but an external path can never become a query - * source. - * @module @deepseek-ai/dsh-lsp-local/host - */ +/** Filesystem-seam source access for the generic stdio LSP provider. */ -import { constants } from 'node:fs' -import { open, realpath, stat } from 'node:fs/promises' -import type { FileHandle } from 'node:fs/promises' -import { isAbsolute, resolve as resolvePath, sep } from 'node:path' +import { Buffer } from 'node:buffer' +import type { FileSystem, FsTarget } from '@deepseek-ai/dsh-fs' import { throwIfAborted } from './abort.ts' -/** A validated source: its canonical absolute path and current UTF-8 text. */ -export interface HostSource { - /** The canonical (realpath-resolved) absolute path, inside the canonical workspace. */ +/** A canonical workspace in the filesystem/subprocess execution world. */ +export interface HostWorkspace { + /** Stable filesystem identity used for provider pooling. */ + readonly target: FsTarget + /** Canonical absolute path accepted as a subprocess cwd. */ readonly canonicalPath: string - /** The file's current text, read as UTF-8. */ + /** Canonical file URI sent during LSP initialization. */ + readonly fileUrl: string +} + +/** A validated source and the exact URI sent to the language server. */ +export interface HostSource { + /** Canonical file URI in the execution world's platform syntax. */ + readonly fileUrl: string + /** Current complete UTF-8 text. */ readonly text: string } /** - * Canonicalize a workspace root: it must exist and be a directory. The returned realpath supplies - * process cwd, `rootUri`, the sole `workspaceFolders` entry, and pool identity, so symlinked roots - * collapse to one instance. - * @param workspaceRoot - the caller's workspace root (absolute). - * @param signal - optional cancellation observed around each filesystem operation. - * @returns the canonical directory path. - * @throws Error when the path is missing or not a directory. + * Resolve and validate one workspace through `ctx.fs`. + * @param fs - filesystem provider sharing the language server's execution world. + * @param workspaceRoot - caller-supplied workspace path. + * @param signal - optional cancellation around provider operations. + * @returns stable identity plus process path and file URI. */ -export async function canonicalizeWorkspace(workspaceRoot: string, signal?: AbortSignal): Promise { +export async function canonicalizeWorkspace( + fs: FileSystem, + workspaceRoot: string, + signal?: AbortSignal, +): Promise { throwIfAborted(signal) - let canonical: string + let target: FsTarget try { - canonical = await realpath(workspaceRoot) - } catch (error) { - throw new Error(`workspace root "${workspaceRoot}" cannot be resolved: ${messageOf(error)}`) + target = await fs.resolve(workspaceRoot, signal === undefined ? {} : { signal }) + } catch (error: unknown) { + throwIfAborted(signal) + throw new Error(`workspace root "${workspaceRoot}" cannot be resolved: ${messageOf(error)}`, { cause: error }) } throwIfAborted(signal) - const info = await stat(canonical) + const info = await fs.stat(target, signal).catch((error: unknown) => { + throwIfAborted(signal) + throw error + }) throwIfAborted(signal) - if (!info.isDirectory()) { + if (info?.type !== 'directory') { throw new Error(`workspace root "${workspaceRoot}" is not a directory`) } - return canonical + return { + target, + canonicalPath: fs.processPath(target), + fileUrl: fs.fileUrl(target), + } } /** - * Resolve, canonicalize, validate, and read a query source in one pass. A relative `filePath` - * resolves against `canonicalWorkspace`; an absolute one is taken directly. The canonical target - * must be a regular UTF-8 file no larger than `maxDocumentBytes`, and must lie inside the canonical - * workspace. - * @param filePath - the model-supplied source path (relative or absolute). - * @param canonicalWorkspace - the already-canonicalized workspace root. - * @param maxDocumentBytes - the largest source this host will open. - * @param signal - optional cancellation observed throughout resolution, validation, and reading. - * @returns the canonical path and current UTF-8 text. - * @throws Error when the source is missing, non-regular, oversized, non-UTF-8, or out of workspace. + * Resolve, contain, and read one byte-bounded query source through `ctx.fs`. + * This layer owns the LSP-specific complete-document cap while the filesystem + * provider owns streaming, regular-file checks, and UTF-8 validation. + * @param fs - filesystem provider sharing the server's execution world. + * @param filePath - absolute source path or path relative to `workspace`. + * @param workspace - already-canonical workspace. + * @param maxDocumentBytes - largest complete source accepted by this host. + * @param signal - optional cancellation. + * @returns canonical file URI and current text. */ export async function readHostSource( + fs: FileSystem, filePath: string, - canonicalWorkspace: string, + workspace: HostWorkspace, maxDocumentBytes: number, signal?: AbortSignal, ): Promise { throwIfAborted(signal) - const requested = isAbsolute(filePath) ? filePath : resolvePath(canonicalWorkspace, filePath) - let canonicalPath: string + let target: FsTarget try { - canonicalPath = await realpath(requested) - } catch (error) { - throw new Error(`source "${filePath}" cannot be resolved: ${messageOf(error)}`) + target = await fs.resolve(filePath, { + cwd: workspace.canonicalPath, + ...signal === undefined ? {} : { signal }, + }) + } catch (error: unknown) { + throwIfAborted(signal) + throw new Error(`source "${filePath}" cannot be resolved: ${messageOf(error)}`, { cause: error }) } throwIfAborted(signal) - if (!isInside(canonicalWorkspace, canonicalPath)) { + if (!fs.contains(workspace.target, target)) { throw new Error(`source "${filePath}" resolves outside the workspace`) } - // Open ONE handle after containment, then stat and read through it: a concurrent replace between - // realpath and read cannot swap the target, so the regular-file and size checks bind the bytes we - // actually read (no path-based TOCTOU). O_NOFOLLOW rejects the final component being swapped for a - // symlink between realpath and open (which would otherwise escape the workspace). - // O_NONBLOCK prevents a FIFO with no writer from hanging before fstat can reject it as nonregular. - const handle = await open(canonicalPath, constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK) + const chunks: string[] = [] + let bytes = 0 try { - throwIfAborted(signal) - const info = await handle.stat() - throwIfAborted(signal) - if (!info.isFile()) { - throw new Error(`source "${filePath}" is not a regular file`) + // XXX(lsp-source-replacement): Revisit stable-handle identity only if a real query observes + // replacement between canonical containment and the provider opening this stream. + const stream = await fs.streamText(target, signal) + for await (const chunk of stream) { + throwIfAborted(signal) + bytes += Buffer.byteLength(chunk) + if (bytes > maxDocumentBytes) break + chunks.push(chunk) } - if (info.size > maxDocumentBytes) { - throw new Error(`source "${filePath}" is ${info.size} bytes, over the ${maxDocumentBytes}-byte limit`) - } - // Bound the read to the cap even if the file grew after stat: read one extra byte and reject on - // overflow, so a concurrent grow cannot defeat the memory bound. - const buffer = await readCapped(handle, maxDocumentBytes, filePath, signal) - const text = decodeUtf8Strict(buffer, filePath) + } catch (error: unknown) { throwIfAborted(signal) - return { canonicalPath, text } - } finally { - await handle.close() + throw new Error(`source "${filePath}" could not be read: ${messageOf(error)}`, { cause: error }) + } + if (bytes > maxDocumentBytes) { + throw new Error( + `source "${filePath}" exceeds the ${maxDocumentBytes}-byte limit; reading stopped after ${bytes} bytes`, + ) + } + throwIfAborted(signal) + return { + fileUrl: fs.fileUrl(target), + text: chunks.join(''), } } -/** Read at most `maxBytes` from the handle, rejecting when the source overflows the cap. */ -async function readCapped( - handle: FileHandle, - maxBytes: number, - filePath: string, - signal?: AbortSignal, -): Promise { - const limit = maxBytes + 1 - const chunk = Buffer.allocUnsafe(limit) - let total = 0 - for (;;) { - throwIfAborted(signal) - const { bytesRead } = await handle.read(chunk, total, limit - total, total) - throwIfAborted(signal) - if (bytesRead === 0) break - total += bytesRead - /* v8 ignore next 3 -- overflow requires the file to grow past the cap between stat and read (a concurrent mutation); defensive. */ - if (total > maxBytes) { - throw new Error(`source "${filePath}" grew past the ${maxBytes}-byte limit while reading`) - } - } - return chunk.subarray(0, total) -} - -/** Whether `child` is the workspace itself or a descendant of it (both already canonical). */ -function isInside(workspace: string, child: string): boolean { - if (child === workspace) return true - /* v8 ignore next -- a canonical non-root workspace never ends with a separator; the guard covers the filesystem root. */ - const base = workspace.endsWith(sep) ? workspace : workspace + sep - return child.startsWith(base) -} - -/** Decode strictly as UTF-8: a fatal decoder rejects only malformed bytes, keeping a legitimate U+FFFD. */ -function decodeUtf8Strict(buffer: Buffer, filePath: string): string { - try { - return new TextDecoder('utf-8', { fatal: true }).decode(buffer) - } catch { - throw new Error(`source "${filePath}" is not valid UTF-8 text`) - } -} - -/** Extract a message from an unknown thrown value without leaking `any`. */ function messageOf(error: unknown): string { - /* v8 ignore next -- Node fs rejections are always Error instances; the String() fallback is defensive. */ return error instanceof Error ? error.message : String(error) } diff --git a/packages/lsp/lsp-local/src/index.ts b/packages/lsp/lsp-local/src/index.ts index d926ae9428..fc7e50787f 100644 --- a/packages/lsp/lsp-local/src/index.ts +++ b/packages/lsp/lsp-local/src/index.ts @@ -1,18 +1,16 @@ /** * Generic stdio language-server backend for `ctx.lsp`. One plugin instance configures a named table * of server commands and registers one isolated provider for each entry. Every provider lazily - * single-flights one server process per canonical workspace realpath, serves transient-open queries + * single-flights one server process per canonical workspace target, serves transient-open queries * through it, and replaces a selected transport that fails before or during the next read-only - * query. Providers read sources through Node APIs in the host namespace (not `ctx.fs`) - * and trust their configured servers — no sandbox confinement. + * query. Providers read sources through `ctx.fs` and launch servers through + * `ctx.subprocess`, so both local and remote implementations share one host. * * Namespace plugin (named exports, no default export). Lifecycle is effect-scoped: disposal * unregisters from `ctx.lsp` and tears down every live server. * @module @deepseek-ai/dsh-lsp-local */ -import { accessSync, constants, statSync } from 'node:fs' -import { delimiter, isAbsolute, join } from 'node:path' import type { Context } from 'cordis' import z from 'schemastery' import { LspError, LspProviderId } from '@deepseek-ai/dsh-lsp' @@ -24,9 +22,9 @@ import type { import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout' import { abortable, abortError } from './abort.ts' import { canonicalizeWorkspace, readHostSource } from './host.ts' +import type { HostWorkspace } from './host.ts' import { LspInstance } from './instance.ts' import type { ConnectionSpawner } from './connection.ts' -import { scrubbedParentEnv } from '@deepseek-ai/dsh-subprocess' import type { InstanceSpec } from './instance.ts' export { canonicalizeWorkspace, readHostSource } from './host.ts' @@ -46,10 +44,7 @@ export { LspConnection } from './connection.ts' export const name = 'lsp-local' /** Services required by this plugin. */ -export const inject = ['lsp', 'subprocess'] - -/** Credential-shaped ambient env vars are NOT forwarded to the child by default. */ - +export const inject = ['fs', 'lsp', 'subprocess'] const DEFAULT_MAX_MESSAGE_BYTES = 16_000_000 const DEFAULT_MAX_STDERR_BYTES = 1_000_000 @@ -91,6 +86,7 @@ export interface Config { /** One server config after schemastery fills every default. */ type ResolvedServerConfig = Required +type WorkspaceKey = HostWorkspace['target']['targetKey'] const LspLocalServerConfig: z = z.object({ command: z.string().required(), @@ -110,27 +106,67 @@ export const Config: z = z.object({ servers: z.dict(LspLocalServerConfig).required(), }) +/** Propagate teardown failures only after every sibling has settled. */ +function throwTeardownFailures(results: readonly PromiseSettledResult[], message: string): void { + const failures: unknown[] = [] + for (const result of results) { + if (result.status === 'rejected') failures.push(result.reason) + } + if (failures.length === 1) throw failures[0] + if (failures.length > 1) throw new AggregateError(failures, message) +} + /** * Register the configured stdio LSP providers. Resolves every executable at load (after credential * scrubbing) before publishing any provider; each process launches lazily on its first matching * query. - * @param ctx - the plugin context (must inject `lsp`). + * @param ctx - the plugin context carrying `fs`, `lsp`, and `subprocess`. * @param config - the resolved plugin configuration (schemastery has filled every default). */ -export function apply(ctx: Context, config: Config): void { +export async function apply(ctx: Context, config: Config): Promise { const entries = Object.entries(config.servers) if (entries.length === 0) throw new Error('lsp-local: servers must contain at least one server') + const setupAbort = new AbortController() + const stopSetupCancellation = ctx.on('internal/plugin', (fiber) => { + // An async plugin callback must observe its own disposal before Cordis can + // run effect cleanup, because unload otherwise waits for this callback. + if (fiber === ctx.fiber && fiber.uid === null) { + setupAbort.abort(new Error('lsp-local setup disposed')) + } + }) + // Resolve every server-local setting before registration so a bad later command or bound cannot // publish an earlier provider. Registry-level mapping conflicts are rolled back below. - const providers = entries.map(([providerId, rawConfig]) => { - if (providerId.trim() === '') throw new Error('lsp-local: server ids must be non-empty strings') - const resolved = rawConfig as ResolvedServerConfig - validateServerConfig(providerId, resolved) - const childEnv = buildChildEnv(resolved.env) - const executable = resolveExecutable(resolved.command, childEnv) - return new LocalLspProvider(providerId, resolved, childEnv, executable, spec => ctx.subprocess.spawn(spec)) - }) + const providers = await (async () => { + const lookups = entries.map(async ([providerId, rawConfig]) => { + if (providerId.trim() === '') throw new Error('lsp-local: server ids must be non-empty strings') + const resolved = rawConfig as ResolvedServerConfig + validateServerConfig(providerId, resolved) + const executable = await ctx.subprocess.resolveExecutable( + resolved.command, + resolved.env, + setupAbort.signal, + ) + setupAbort.signal.throwIfAborted() + return new LocalLspProvider( + providerId, + ctx.fs, + resolved, + executable, + spec => ctx.subprocess.spawn(spec), + ) + }) + try { + return await Promise.all(lookups) + } catch (error: unknown) { + setupAbort.abort(error) + await Promise.allSettled(lookups) + throw error + } finally { + stopSetupCancellation() + } + })() ctx.effect(() => { const disposers: Array<() => void> = [] @@ -143,7 +179,8 @@ export function apply(ctx: Context, config: Config): void { return async () => { // Remove every route before process teardown so no new query can enter a draining provider. for (const dispose of disposers.reverse()) dispose() - await Promise.all(providers.map(provider => provider.disposeAll())) + const results = await Promise.allSettled(providers.map(provider => provider.disposeAll())) + throwTeardownFailures(results, 'lsp-local provider teardown failed') } }, 'lsp-local.registerProviders') } @@ -180,16 +217,19 @@ function assertPositiveInteger(providerId: string, name: string, value: number): class LocalLspProvider implements LspProvider { readonly id: LspProviderId readonly extensionToLanguage: Readonly> - /** One live instance per canonical workspace realpath. */ - private readonly instances = new Map() + /** One live instance per stable canonical workspace identity. */ + private readonly instances = new Map() /** One complete source-read→open→query→close serialization tail per canonical workspace. */ - private readonly queues = new Map>() + private readonly queues = new Map>() + /** Workspace canonicalizations that have not entered a provider-owned queue yet. */ + private readonly workspaceLookups = new Set>() + private readonly lifetime = new AbortController() private disposed = false constructor( providerId: string, + private readonly fs: Context['fs'], private readonly config: ResolvedServerConfig, - private readonly childEnv: Record, private readonly executable: string, private readonly spawner: ConnectionSpawner, ) { @@ -210,43 +250,60 @@ class LocalLspProvider implements LspProvider { if (signal?.aborted) throw abortError(signal) } + /** Fuse caller cancellation with provider disposal for every filesystem and protocol await. */ + private querySignal(signal?: AbortSignal): AbortSignal { + return signal === undefined + ? this.lifetime.signal + : AbortSignal.any([signal, this.lifetime.signal]) + } + async query(request: LspProviderQuery, signal?: AbortSignal): Promise { - // Honor an already-aborted signal before host I/O so a canceled request never starts a server. + // Honor an already-aborted signal before provider I/O so a canceled request never starts a server. this.assertActive(signal) - const workspace = await canonicalizeWorkspace(request.workspaceRoot, signal) - this.assertActive(signal) - return this.enqueue(workspace, signal, async () => { - this.assertActive(signal) + const querySignal = this.querySignal(signal) + const workspaceResult = canonicalizeWorkspace(this.fs, request.workspaceRoot, querySignal) + const workspaceLookup = workspaceResult.then(() => undefined, () => undefined) + this.workspaceLookups.add(workspaceLookup) + let workspace: HostWorkspace + try { + workspace = await workspaceResult + } finally { + this.workspaceLookups.delete(workspaceLookup) + } + this.assertActive(querySignal) + const workspaceKey = workspace.target.targetKey + return this.enqueue(workspaceKey, querySignal, async () => { + this.assertActive(querySignal) // Read inside the workspace queue but before spawning: a queued query sees current bytes when // its turn starts, while an invalid source still cannot leave an idle process pooled. - const source = await readHostSource(request.filePath, workspace, this.config.maxDocumentBytes, signal) + const source = await readHostSource(this.fs, request.filePath, workspace, this.config.maxDocumentBytes, querySignal) // Disposal may have snapshotted the instance map while host I/O was pending. Re-check before a // synchronous get-or-create so every spawned process remains owned by teardown. - this.assertActive(signal) - let instance = this.instanceFor(workspace) + this.assertActive(querySignal) + let instance = this.instanceFor(workspaceKey, workspace) try { - return await instance.query(request, source, signal) + return await instance.query(request, source, querySignal) } catch (error) { // A selected child can have died while idle or fail during the next write. Queries are // read-only, so replace that transport once and retry transparently. if (!instance.isTransportFailure(error)) throw error await instance.dispose() - this.evictIfCurrent(workspace, instance) - this.assertActive(signal) - instance = this.instanceFor(workspace) - return await instance.query(request, source, signal) + this.evictIfCurrent(workspaceKey, instance) + this.assertActive(querySignal) + instance = this.instanceFor(workspaceKey, workspace) + return await instance.query(request, source, querySignal) } finally { // Reach quiescence before dropping a dead slot; a replacement must survive this ownership check. if (instance.dead) { await instance.dispose() - this.evictIfCurrent(workspace, instance) + this.evictIfCurrent(workspaceKey, instance) } } }) } /** Serialize one complete query lifecycle for a canonical workspace. */ - private enqueue(workspace: string, signal: AbortSignal | undefined, run: () => Promise): Promise { + private enqueue(workspace: WorkspaceKey, signal: AbortSignal | undefined, run: () => Promise): Promise { const previous = this.queues.get(workspace) ?? Promise.resolve() const result = abortable(previous, signal).then(run) // The tail follows the actual prior work even when this caller aborts its wait. It never rejects, @@ -260,27 +317,28 @@ class LocalLspProvider implements LspProvider { } /** Return or synchronously publish the one instance for a canonical workspace. */ - private instanceFor(workspace: string): LspInstance { + private instanceFor(workspaceKey: WorkspaceKey, workspace: HostWorkspace): LspInstance { this.assertActive() - const existing = this.instances.get(workspace) + const existing = this.instances.get(workspaceKey) if (existing !== undefined) return existing const created = this.createInstance(workspace) - this.instances.set(workspace, created) + this.instances.set(workspaceKey, created) return created } /** Drop the slot iff it still contains this instance. */ - private evictIfCurrent(workspace: string, instance: LspInstance): void { + private evictIfCurrent(workspace: WorkspaceKey, instance: LspInstance): void { /* v8 ignore next -- mismatch requires another query to replace the slot before this finally runs. */ if (this.instances.get(workspace) === instance) this.instances.delete(workspace) } - private createInstance(workspace: string): LspInstance { + private createInstance(workspace: HostWorkspace): LspInstance { const spec: InstanceSpec = { command: this.executable, args: this.config.args, - cwd: workspace, - env: this.childEnv, + cwd: workspace.canonicalPath, + workspaceUri: workspace.fileUrl, + env: this.config.env, configuration: this.config.configuration, initializationOptions: this.config.initializationOptions, maxMessageBytes: this.config.maxMessageBytes, @@ -294,51 +352,18 @@ class LocalLspProvider implements LspProvider { /** Dispose every live instance and block further queries. */ async disposeAll(): Promise { this.disposed = true + this.lifetime.abort(new LspError('lsp-local provider is disposed', 'LSP_DISPOSED')) const live = [...this.instances.values()] const draining = [...this.queues.values()] + const resolving = [...this.workspaceLookups] this.instances.clear() - await Promise.all([ + const results = await Promise.allSettled([ ...live.map(instance => instance.dispose()), ...draining, + ...resolving, ]) this.queues.clear() - } -} - -/** The seam's scrubbed parent env (credential-shaped and DSH_* names dropped), plus the config's explicit env. */ -function buildChildEnv(extra: Record): Record { - return { ...scrubbedParentEnv(), ...extra } -} - -/** - * Resolve the server executable to an absolute path: an absolute command is verified directly; a - * bare command is looked up on the child's PATH. Fails loudly when nothing is executable. - */ -function resolveExecutable(command: string, childEnv: Record): string { - if (isAbsolute(command)) { - // Verify an absolute command too, so an unavailable one fails at load, not on the first query. - if (!isExecutableFileSync(command)) { - throw new Error(`lsp-local: command "${command}" is not an executable file`) - } - return command - } - /* v8 ignore next -- buildChildEnv always sets PATH from the ambient env; the further fallbacks are defensive. */ - const pathValue = childEnv.PATH ?? process.env.PATH ?? '' - for (const dir of pathValue.split(delimiter)) { - if (dir === '') continue - const candidate = join(dir, command) - if (isExecutableFileSync(candidate)) return candidate - } - throw new Error(`lsp-local: command "${command}" was not found on PATH`) -} - -/** Synchronous regular-file and executable check used only at load-time resolution. */ -function isExecutableFileSync(path: string): boolean { - try { - if (!statSync(path).isFile()) return false - accessSync(path, constants.X_OK) - return true - } catch { - return false + this.workspaceLookups.clear() + throwTeardownFailures(results, 'lsp-local instance teardown failed') } } diff --git a/packages/lsp/lsp-local/src/instance.ts b/packages/lsp/lsp-local/src/instance.ts index c1f78eaa38..318704b9fc 100644 --- a/packages/lsp/lsp-local/src/instance.ts +++ b/packages/lsp/lsp-local/src/instance.ts @@ -7,7 +7,6 @@ * @module @deepseek-ai/dsh-lsp-local/instance */ -import { pathToFileURL } from 'node:url' import { LspError } from '@deepseek-ai/dsh-lsp' import type { LspOperation, @@ -31,6 +30,8 @@ import { /** Everything an instance needs beyond the connection spec. */ export interface InstanceSpec extends ConnectionSpec { + /** Canonical workspace file URI supplied by the filesystem provider. */ + readonly workspaceUri: string /** Static `initialize` options forwarded to the server. */ readonly initializationOptions: unknown /** Graceful `shutdown`/`exit` budget before escalation (ms). */ @@ -108,9 +109,11 @@ export class LspInstance { private async initialize(): Promise { const initializeResult = await this.connection.request('initialize', { - processId: process.pid, - rootUri: pathToFileURL(this.spec.cwd).href, - workspaceFolders: [{ uri: pathToFileURL(this.spec.cwd).href, name: 'workspace' }], + // A subprocess provider may run in another PID namespace or machine; + // the host PID would let the server monitor an unrelated process. + processId: null, + rootUri: this.spec.workspaceUri, + workspaceFolders: [{ uri: this.spec.workspaceUri, name: 'workspace' }], capabilities: CLIENT_CAPABILITIES, initializationOptions: this.spec.initializationOptions, }) as WireInitializeResult @@ -147,7 +150,7 @@ export class LspInstance { throw new LspError('server does not support the transient textDocument/didOpen this host requires', 'LSP_UNSUPPORTED_OPERATION') } - const uri = pathToFileURL(source.canonicalPath).href + const uri = source.fileUrl let opened = false try { /* v8 ignore next -- guards an abort landing between the ready wait and didOpen; not deterministically reproducible. */ @@ -241,10 +244,9 @@ export class LspInstance { if (operation === 'hover') { return { kind: 'hover', hover: normalizeHover(payload) } } - // `spec.cwd` is the canonical workspace realpath (the provider canonicalizes before spawning), - // and every `file:` location URI is relative to it — so it is the root a caller must relativize - // display paths against, not the request's possibly-symlinked workspaceRoot. - return { kind: 'locations', locations: normalizeLocations(payload), resolvedWorkspaceRoot: this.spec.cwd } + // The filesystem provider owns URI syntax for the execution platform, which may differ from the + // harness host. Preserve that coordinate through rendering instead of reparsing `spec.cwd` there. + return { kind: 'locations', locations: normalizeLocations(payload), resolvedWorkspaceUri: this.spec.workspaceUri } } private answerServerRequest(method: string, params: unknown): Promise { diff --git a/packages/lsp/lsp-local/tests/built-lib.e2e.ts b/packages/lsp/lsp-local/tests/built-lib.e2e.ts index bdd0ec1067..75f9f227c3 100644 --- a/packages/lsp/lsp-local/tests/built-lib.e2e.ts +++ b/packages/lsp/lsp-local/tests/built-lib.e2e.ts @@ -16,8 +16,9 @@ import { afterAll, beforeAll, describe, expect, it } from 'vitest' const pkgDir = fileURLToPath(new URL('..', import.meta.url)) const seamLib = join(pkgDir, '../lsp/lib/index.js') +const fsLib = join(pkgDir, '../../fs/fs-local/lib/index.js') const subprocessLib = join(pkgDir, '../../subprocess/subprocess-local/lib/index.js') -const built = existsSync(join(pkgDir, 'lib/index.js')) && existsSync(seamLib) && existsSync(subprocessLib) +const built = existsSync(join(pkgDir, 'lib/index.js')) && existsSync(seamLib) && existsSync(fsLib) && existsSync(subprocessLib) const fixtureServer = fileURLToPath(new URL('./fixture-server.ts', import.meta.url)) @@ -42,10 +43,12 @@ describe.skipIf(!built)('built lib real load path (plain node)', () => { const { Context } = await import('cordis') const { default: Lsp } = await import('@deepseek-ai/dsh-lsp') const LspLocal = await import('@deepseek-ai/dsh-lsp-local') + const { default: LocalFileSystem } = await import('@deepseek-ai/dsh-fs-local') const { default: LocalSubprocessService } = await import('@deepseek-ai/dsh-subprocess-local') const ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) await ctx.plugin(LspLocal, { servers: { fake: { diff --git a/packages/lsp/lsp-local/tests/host.spec.ts b/packages/lsp/lsp-local/tests/host.spec.ts index 26aacdc1f4..c06b6d9df8 100644 --- a/packages/lsp/lsp-local/tests/host.spec.ts +++ b/packages/lsp/lsp-local/tests/host.spec.ts @@ -4,7 +4,10 @@ import { tmpdir } from 'node:os' import { join } from 'node:path' import { realpath } from 'node:fs/promises' import { execFile } from 'node:child_process' +import { pathToFileURL } from 'node:url' import { promisify } from 'node:util' +import { Context } from 'cordis' +import LocalFileSystem from '@deepseek-ai/dsh-fs-local' import { deadline } from '@deepseek-ai/dsh-timeout' import { canonicalizeWorkspace, readHostSource } from '@deepseek-ai/dsh-lsp-local' @@ -12,84 +15,125 @@ const execFileAsync = promisify(execFile) let root: string let ws: string +let ctx: Context +let fs: LocalFileSystem beforeEach(async () => { root = await realpath(await mkdtemp(join(tmpdir(), 'lsp-host-'))) ws = join(root, 'ws') await mkdir(ws) + ctx = new Context() + await ctx.plugin(LocalFileSystem, { cwd: root }) + fs = ctx.fs as LocalFileSystem }) afterEach(async () => { + await ctx.fiber.dispose() await rm(root, { recursive: true, force: true }) }) const BIG = 1_000_000 +async function workspace() { + return await canonicalizeWorkspace(fs, ws) +} + +async function readSource(filePath: string, maxBytes = BIG, signal?: AbortSignal) { + return await readHostSource(fs, filePath, await workspace(), maxBytes, signal) +} + describe('canonicalizeWorkspace', () => { it('returns the realpath of a directory', async () => { - expect(await canonicalizeWorkspace(ws)).toBe(ws) + expect((await workspace()).canonicalPath).toBe(ws) }) it('resolves a symlinked workspace to its target so aliases share identity', async () => { const link = join(root, 'ws-link') await symlink(ws, link) - expect(await canonicalizeWorkspace(link)).toBe(ws) + expect((await canonicalizeWorkspace(fs, link)).canonicalPath).toBe(ws) }) it('rejects a missing workspace', async () => { - await expect(canonicalizeWorkspace(join(root, 'nope'))).rejects.toThrow(/cannot be resolved/) + await expect(canonicalizeWorkspace(fs, join(root, 'nope'))).rejects.toThrow(/not a directory/) + }) + + it('wraps a provider failure while resolving the workspace', async () => { + fs.resolve = async () => { throw 'raw workspace resolve failure' } + await expect(canonicalizeWorkspace(fs, ws)) + .rejects.toThrow(`workspace root "${ws}" cannot be resolved: raw workspace resolve failure`) }) it('rejects a non-directory workspace', async () => { const file = join(root, 'file.txt') await writeFile(file, 'x') - await expect(canonicalizeWorkspace(file)).rejects.toThrow(/not a directory/) + await expect(canonicalizeWorkspace(fs, file)).rejects.toThrow(/not a directory/) + }) + + it('normalizes workspace metadata cancellation and preserves other provider failures', async () => { + const providerFailure = new Error('workspace metadata failed') + fs.stat = async () => { throw providerFailure } + await expect(canonicalizeWorkspace(fs, ws)).rejects.toBe(providerFailure) + + const controller = new AbortController() + fs.stat = async () => { + controller.abort(new Error('workspace metadata cancelled')) + throw providerFailure + } + await expect(canonicalizeWorkspace(fs, ws, controller.signal)) + .rejects.toThrow('workspace metadata cancelled') }) }) describe('readHostSource', () => { it('reads a relative path against the workspace', async () => { await writeFile(join(ws, 'a.ts'), 'const x = 1\n') - const source = await readHostSource('a.ts', ws, BIG) - expect(source.canonicalPath).toBe(join(ws, 'a.ts')) + const source = await readSource('a.ts') + expect(source.fileUrl).toBe(pathToFileURL(join(ws, 'a.ts')).href) expect(source.text).toBe('const x = 1\n') }) it('reads an absolute path inside the workspace', async () => { const abs = join(ws, 'b.ts') await writeFile(abs, 'b') - const source = await readHostSource(abs, ws, BIG) - expect(source.canonicalPath).toBe(abs) + const source = await readSource(abs) + expect(source.fileUrl).toBe(pathToFileURL(abs).href) }) it('accepts a source reached through a symlink that stays inside the workspace', async () => { await mkdir(join(ws, 'real')) await writeFile(join(ws, 'real', 'c.ts'), 'c') await symlink(join(ws, 'real'), join(ws, 'linked')) - const source = await readHostSource('linked/c.ts', ws, BIG) - expect(source.canonicalPath).toBe(join(ws, 'real', 'c.ts')) + const source = await readSource('linked/c.ts') + expect(source.fileUrl).toBe(pathToFileURL(join(ws, 'real', 'c.ts')).href) }) it('rejects a source whose canonical path escapes the workspace via symlink', async () => { const outside = join(root, 'outside.ts') await writeFile(outside, 'secret') await symlink(outside, join(ws, 'escape.ts')) - await expect(readHostSource('escape.ts', ws, BIG)).rejects.toThrow(/outside the workspace/) + await expect(readSource('escape.ts')).rejects.toThrow(/outside the workspace/) }) it('rejects an absolute source outside the workspace', async () => { const outside = join(root, 'out.ts') await writeFile(outside, 'x') - await expect(readHostSource(outside, ws, BIG)).rejects.toThrow(/outside the workspace/) + await expect(readSource(outside)).rejects.toThrow(/outside the workspace/) }) it('rejects a missing source', async () => { - await expect(readHostSource('nope.ts', ws, BIG)).rejects.toThrow(/cannot be resolved/) + await expect(readSource('nope.ts')).rejects.toThrow(/not found/) + }) + + it('wraps a provider failure while resolving the source', async () => { + const canonical = await workspace() + fs.resolve = async () => { throw 'raw resolve failure' } + await expect(readHostSource(fs, 'broken.ts', canonical, BIG)) + .rejects.toThrow('source "broken.ts" cannot be resolved: raw resolve failure') }) it('rejects a non-regular source (directory)', async () => { await mkdir(join(ws, 'dir')) - await expect(readHostSource('dir', ws, BIG)).rejects.toThrow(/not a regular file/) + await expect(readSource('dir')).rejects.toThrow(/not a regular file/) }) // Windows has no filesystem FIFO; the directory case above pins non-regular rejection there. @@ -97,36 +141,44 @@ describe('readHostSource', () => { const fifo = join(ws, 'pipe.ts') await execFileAsync('mkfifo', [fifo]) using d = deadline(undefined, 1000, 'FIFO_READ_TIMEOUT') - await expect(readHostSource('pipe.ts', ws, BIG, d.signal)).rejects.toThrow(/not a regular file/) + await expect(readSource('pipe.ts', BIG, d.signal)).rejects.toThrow(/not a regular file/) }) it('honors a pre-aborted source read before filesystem work', async () => { const controller = new AbortController() controller.abort(new Error('source read cancelled')) - await expect(readHostSource('missing.ts', ws, BIG, controller.signal)).rejects.toThrow(/source read cancelled/) + await expect(readSource('missing.ts', BIG, controller.signal)).rejects.toThrow(/source read cancelled/) }) it('treats the workspace root itself as inside, then rejects it as non-regular', async () => { - // filePath '.' canonicalizes to the workspace dir: isInside's identity branch is taken, and the - // directory then fails the regular-file check. - await expect(readHostSource('.', ws, BIG)).rejects.toThrow(/not a regular file/) + // The filesystem containment primitive accepts the workspace itself; the + // bounded read then rejects the directory as non-regular. + await expect(readSource('.')).rejects.toThrow(/not a regular file/) }) - it('rejects an oversized source', async () => { + it('rejects an oversized source and reports the observed lower bound', async () => { await writeFile(join(ws, 'big.ts'), 'x'.repeat(100)) - await expect(readHostSource('big.ts', ws, 10)).rejects.toThrow(/over the 10-byte limit/) + await expect(readSource('big.ts', 10)).rejects.toMatchObject({ + message: 'source "big.ts" exceeds the 10-byte limit; reading stopped after 100 bytes', + }) + }) + + it('counts the complete UTF-8 byte length at the configured boundary', async () => { + await writeFile(join(ws, 'multibyte.ts'), '€abc') + await expect(readSource('multibyte.ts', 6)).resolves.toMatchObject({ text: '€abc' }) + await expect(readSource('multibyte.ts', 5)).rejects.toThrow(/5-byte limit/) }) it('rejects a non-UTF-8 source', async () => { await writeFile(join(ws, 'bin.ts'), Buffer.from([0xff, 0xfe, 0x00])) - await expect(readHostSource('bin.ts', ws, BIG)).rejects.toThrow(/not valid UTF-8/) + await expect(readSource('bin.ts')).rejects.toThrow(/invalid UTF-8|binary file/) }) it('keeps a valid U+FFFD replacement character in otherwise-valid UTF-8', async () => { // The literal replacement char is valid UTF-8; a fatal decoder must accept it (only malformed // byte sequences are rejected). await writeFile(join(ws, 'repl.ts'), 'const s = "�"\n') - const source = await readHostSource('repl.ts', ws, BIG) + const source = await readSource('repl.ts') expect(source.text).toBe('const s = "�"\n') }) }) diff --git a/packages/lsp/lsp-local/tests/instance.spec.ts b/packages/lsp/lsp-local/tests/instance.spec.ts index 75b9e541fc..08bad67ae2 100644 --- a/packages/lsp/lsp-local/tests/instance.spec.ts +++ b/packages/lsp/lsp-local/tests/instance.spec.ts @@ -1,8 +1,11 @@ import { afterEach, beforeEach, describe, expect, it } from 'vitest' +import { readFileSync } from 'node:fs' import { mkdtemp, mkdir, readFile, rm, writeFile, realpath } from 'node:fs/promises' import { tmpdir } from 'node:os' import { join } from 'node:path' import { pathToFileURL, fileURLToPath } from 'node:url' +import { Context } from 'cordis' +import LocalFileSystem from '@deepseek-ai/dsh-fs-local' import { LspInstance, readHostSource } from '@deepseek-ai/dsh-lsp-local' import { encodeMessage } from '@deepseek-ai/dsh-lsp-local' import type { ConnectionWriter } from '@deepseek-ai/dsh-lsp-local/src/connection.ts' @@ -15,6 +18,8 @@ const fixtureServer = fileURLToPath(new URL('./fixture-server.ts', import.meta.u let root: string let ws: string +let ctx: Context +let fs: LocalFileSystem let live: LspInstance[] = [] beforeEach(async () => { @@ -22,11 +27,15 @@ beforeEach(async () => { ws = join(root, 'ws') await mkdir(ws) await writeFile(join(ws, 'a.ts'), 'const x = 1\n') + ctx = new Context() + await ctx.plugin(LocalFileSystem, { cwd: root }) + fs = ctx.fs as LocalFileSystem }) afterEach(async () => { for (const instance of live) await instance.dispose() live = [] + await ctx.fiber.dispose() await rm(root, { recursive: true, force: true }) }) @@ -39,6 +48,7 @@ function makeInstance( command: process.execPath, args: [fixtureServer], cwd: ws, + workspaceUri: pathToFileURL(ws).href, env: { ...scrubbedParentEnv(), ...env }, configuration: { setting: 42 }, initializationOptions: { init: true }, @@ -58,7 +68,12 @@ function query(operation: LspProviderQuery['operation'] = 'goToDefinition'): Lsp /** Run a query against an instance, reading the source first the way the provider does. */ async function run(instance: LspInstance, operation: LspProviderQuery['operation'] = 'goToDefinition', signal?: AbortSignal): Promise { - const source = await readHostSource('a.ts', ws, 4_000_000) + const workspace = { + target: await fs.resolve(ws), + canonicalPath: ws, + fileUrl: pathToFileURL(ws).href, + } + const source = await readHostSource(fs, 'a.ts', workspace, 4_000_000) return instance.query(query(operation), source, signal) } @@ -68,6 +83,7 @@ function scriptInstance(script: string, overrides: Partial = {}): command: process.execPath, args: ['-e', script], cwd: ws, + workspaceUri: pathToFileURL(ws).href, env: scrubbedParentEnv(), configuration: null, initializationOptions: null, @@ -102,17 +118,17 @@ describe('LspInstance server-request handling', () => { it('accepts a lifecycle client/registerCapability request', async () => { const instance = makeInstance({ LSP_FAKE_ON_OPEN: 'lifecycle', LSP_FAKE_DEF: 'null' }) - await expect(run(instance, 'goToDefinition')).resolves.toEqual({ kind: 'locations', locations: [], resolvedWorkspaceRoot: ws }) + await expect(run(instance, 'goToDefinition')).resolves.toEqual({ kind: 'locations', locations: [], resolvedWorkspaceUri: pathToFileURL(ws).href }) }) it('rejects a workspace/applyEdit request but keeps serving', async () => { const instance = makeInstance({ LSP_FAKE_ON_OPEN: 'applyEdit', LSP_FAKE_DEF: 'null' }) - await expect(run(instance, 'goToDefinition')).resolves.toEqual({ kind: 'locations', locations: [], resolvedWorkspaceRoot: ws }) + await expect(run(instance, 'goToDefinition')).resolves.toEqual({ kind: 'locations', locations: [], resolvedWorkspaceUri: pathToFileURL(ws).href }) }) it('rejects an unknown server request but keeps serving', async () => { const instance = makeInstance({ LSP_FAKE_ON_OPEN: 'unknown', LSP_FAKE_DEF: 'null' }) - await expect(run(instance, 'goToDefinition')).resolves.toEqual({ kind: 'locations', locations: [], resolvedWorkspaceRoot: ws }) + await expect(run(instance, 'goToDefinition')).resolves.toEqual({ kind: 'locations', locations: [], resolvedWorkspaceUri: pathToFileURL(ws).href }) }) }) @@ -248,7 +264,7 @@ describe('LspInstance query and abort', () => { await expect(run(instance, 'goToDefinition')).resolves.toEqual({ kind: 'locations', locations: [], - resolvedWorkspaceRoot: ws, + resolvedWorkspaceUri: pathToFileURL(ws).href, }) expect(instance.dead).toBe(true) }) @@ -331,14 +347,22 @@ describe('LspInstance disposal', () => { function processAlive(pid: number): boolean { try { process.kill(pid, 0) - return true } catch (error) { if ((error as NodeJS.ErrnoException).code === 'ESRCH') return false throw error } + if (process.platform !== 'linux') return true + try { + const stat = readFileSync(`/proc/${pid}/stat`, 'utf8') + const state = stat.slice(stat.lastIndexOf(')') + 2).split(/\s+/, 1)[0] + return !/^[ZXx]$/.test(state ?? '') + } catch (error) { + if ((error as NodeJS.ErrnoException).code === 'ENOENT') return false + throw error + } } -/** Wait until a process id disappears so temporary-workspace cleanup cannot race handle release. */ +/** Wait until a process can no longer execute so temporary-workspace cleanup cannot race handle release. */ async function waitForProcessExit(pid: number, timeoutMs = 3_000): Promise { const started = Date.now() while (processAlive(pid)) { diff --git a/packages/lsp/lsp-local/tests/lifecycle.spec.ts b/packages/lsp/lsp-local/tests/lifecycle.spec.ts index ccd85a1e14..20c4bc7bf0 100644 --- a/packages/lsp/lsp-local/tests/lifecycle.spec.ts +++ b/packages/lsp/lsp-local/tests/lifecycle.spec.ts @@ -8,6 +8,7 @@ import { Context } from 'cordis' import Lsp, { type LspProvider, type LspQueryRequest, type LspQueryResult } from '@deepseek-ai/dsh-lsp' import { deadline } from '@deepseek-ai/dsh-timeout' import LocalSubprocessService from '@deepseek-ai/dsh-subprocess-local' +import LocalFileSystem from '@deepseek-ai/dsh-fs-local' import * as LspLocal from '@deepseek-ai/dsh-lsp-local' import type { LspLocalServerConfig } from '@deepseek-ai/dsh-lsp-local' @@ -47,6 +48,7 @@ async function mount( const ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) const register = ctx.lsp.registerProvider.bind(ctx.lsp) const registrationSpy = captureProvider === undefined ? undefined @@ -79,6 +81,7 @@ describe('lsp-local end to end over a fake server', () => { const ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) await ctx.plugin(LspLocal, { servers: { typescript: fakeServer({ LSP_FAKE_HOVER: JSON.stringify({ contents: 'ts' }) }), @@ -99,7 +102,7 @@ describe('lsp-local end to end over a fake server', () => { expect(result).toEqual({ kind: 'locations', locations: [{ uri: pathToFileURL(join(ws, 'a.ts')).href, range: { start: { line: 0, character: 0 }, end: { line: 0, character: 3 } } }], - resolvedWorkspaceRoot: ws, + resolvedWorkspaceUri: pathToFileURL(ws).href, }) await ctx.fiber.dispose() }) @@ -130,7 +133,7 @@ describe('lsp-local end to end over a fake server', () => { it('returns an empty locations result for a null definition', async () => { const ctx = await mount({ LSP_FAKE_DEF: 'null' }) - expect(await ctx.lsp.query(query('goToDefinition'))).toEqual({ kind: 'locations', locations: [], resolvedWorkspaceRoot: ws }) + expect(await ctx.lsp.query(query('goToDefinition'))).toEqual({ kind: 'locations', locations: [], resolvedWorkspaceUri: pathToFileURL(ws).href }) await ctx.fiber.dispose() }) @@ -170,7 +173,7 @@ describe('lsp-local end to end over a fake server', () => { it('accepts openClose options sync', async () => { const ctx = await mount({ LSP_FAKE_SYNC: JSON.stringify({ openClose: true, change: 2 }), LSP_FAKE_DEF: 'null' }) - expect(await ctx.lsp.query(query('goToDefinition'))).toEqual({ kind: 'locations', locations: [], resolvedWorkspaceRoot: ws }) + expect(await ctx.lsp.query(query('goToDefinition'))).toEqual({ kind: 'locations', locations: [], resolvedWorkspaceUri: pathToFileURL(ws).href }) await ctx.fiber.dispose() }) @@ -279,8 +282,9 @@ describe('lsp-local end to end over a fake server', () => { readonly instances: ReadonlyMap }).instances const instance = [...instances.values()][0] - if (instance === undefined) throw new Error('expected one pooled LSP instance') - await waitFor(async () => instance.dead) + // The query's finally may already have observed the exit and evicted the dead slot. When the + // slot remains, synchronize with its close before proving the next query replaces it. + if (instance !== undefined) await waitFor(async () => instance.dead) expect(await ctx.lsp.query(query('goToDefinition'))).toMatchObject({ kind: 'locations' }) await ctx.fiber.dispose() }) @@ -294,7 +298,128 @@ describe('lsp-local end to end over a fake server', () => { controller.abort(new Error('mid-read cancel')) await expect(pending).rejects.toThrow(/mid-read cancel/) // A subsequent live query still works, proving no half-created instance poisoned the pool. - expect(await ctx.lsp.query(query('goToDefinition'))).toEqual({ kind: 'locations', locations: [], resolvedWorkspaceRoot: ws }) + expect(await ctx.lsp.query(query('goToDefinition'))).toEqual({ kind: 'locations', locations: [], resolvedWorkspaceUri: pathToFileURL(ws).href }) + await ctx.fiber.dispose() + }) + + it('aborts and awaits a workspace lookup when the provider is disposed', async () => { + const ctx = await mount({ LSP_FAKE_DEF: 'null' }) + const fs = ctx.fs + const resolve = fs.resolve.bind(fs) + const started = Promise.withResolvers() + const release = Promise.withResolvers() + vi.spyOn(fs, 'resolve').mockImplementation(async (path, options) => { + if (path !== ws) return await resolve(path, options) + const signal = options?.signal + if (signal === undefined) throw new Error('workspace lookup missing provider lifetime signal') + started.resolve(signal) + return await rejectWhenAborted(signal, release.promise) + }) + + const pending = ctx.lsp.query(query('goToDefinition')) + const signal = await started.promise + let disposed = false + const disposing = ctx.fiber.dispose().then(() => { disposed = true }) + await new Promise(resolve => setImmediate(resolve)) + + expect(signal.aborted).toBe(true) + expect(disposed).toBe(false) + release.resolve(undefined) + await expect(pending).rejects.toThrow('provider is disposed') + await expect(disposing).resolves.toBeUndefined() + }) + + it('aborts a queued source stream when the provider is disposed', async () => { + const ctx = await mount({ LSP_FAKE_DEF: 'null' }) + const fs = ctx.fs + const started = Promise.withResolvers() + vi.spyOn(fs, 'streamText').mockImplementation(async (_target, signal) => { + if (signal === undefined) throw new Error('source read missing provider lifetime signal') + started.resolve(signal) + return (async function* () { + await rejectWhenAborted(signal) + yield '' + })() + }) + + const pending = ctx.lsp.query(query('goToDefinition')) + const signal = await started.promise + const disposing = ctx.fiber.dispose() + + await expect(pending).rejects.toThrow('provider is disposed') + await expect(disposing).resolves.toBeUndefined() + expect(signal.aborted).toBe(true) + }) + + it('waits for every owned teardown before aggregating instance failures', async () => { + let provider: LspProvider | undefined + const ctx = await mount({ LSP_FAKE_DEF: 'null' }, {}, (registered) => { provider = registered }) + if (provider === undefined) throw new Error('expected lsp-local to register a provider') + const internals = provider as unknown as { + readonly instances: Map }> + readonly queues: Map> + readonly workspaceLookups: Set> + disposeAll(): Promise + } + const firstFailure = new Error('first instance cleanup failed') + const secondFailure = new Error('second instance cleanup failed') + const release = Promise.withResolvers() + internals.instances.set('first', { dispose: async () => { throw firstFailure } }) + internals.instances.set('second', { dispose: async () => { throw secondFailure } }) + internals.queues.set('pending', release.promise) + internals.workspaceLookups.add(Promise.resolve()) + + let settled = false + const disposing = internals.disposeAll().finally(() => { settled = true }) + await new Promise(resolve => setImmediate(resolve)) + expect(settled).toBe(false) + release.resolve(undefined) + await expect(disposing).rejects.toMatchObject({ + errors: [firstFailure, secondFailure], + message: 'lsp-local instance teardown failed', + }) + expect(internals.instances.size).toBe(0) + expect(internals.queues.size).toBe(0) + expect(internals.workspaceLookups.size).toBe(0) + await ctx.fiber.dispose() + }) + + it('waits for every provider before reporting plugin teardown failure', async () => { + const ctx = new Context() + const disposalErrors: unknown[] = [] + ctx.logger.error = ((error: unknown) => { disposalErrors.push(error) }) as typeof ctx.logger.error + await ctx.plugin(Lsp) + await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) + const providers: LspProvider[] = [] + const register = ctx.lsp.registerProvider.bind(ctx.lsp) + const registrationSpy = vi.spyOn(ctx.lsp, 'registerProvider').mockImplementation((provider) => { + providers.push(provider) + return register(provider) + }) + const fiber = await ctx.plugin(LspLocal, { + servers: { + first: fakeServer(), + second: fakeServer({}, { extensionToLanguage: { '.js': 'javascript' } }), + }, + }) + registrationSpy.mockRestore() + expect(providers).toHaveLength(2) + const failure = new Error('provider cleanup failed') + const release = Promise.withResolvers() + const first = providers[0] as LspProvider & { disposeAll(): Promise } + const second = providers[1] as LspProvider & { disposeAll(): Promise } + first.disposeAll = async () => { throw failure } + second.disposeAll = async () => { await release.promise } + + let disposed = false + const disposing = fiber.dispose().then(() => { disposed = true }) + await new Promise(resolve => setImmediate(resolve)) + expect(disposed).toBe(false) + expect(disposalErrors).toEqual([]) + release.resolve(undefined) + await disposing + expect(disposalErrors).toEqual([failure]) await ctx.fiber.dispose() }) @@ -322,6 +447,7 @@ describe('lsp-local end to end over a fake server', () => { const ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) await expect(ctx.plugin(LspLocal, { servers: { missing: { @@ -354,3 +480,16 @@ async function waitFor(condition: () => Promise, timeoutMs = 3000): Pro await new Promise(resolve => setTimeout(resolve, 10)) } } + +/** Hold one fake provider operation until cancellation, optionally behind a cleanup gate. */ +function rejectWhenAborted(signal: AbortSignal, release: Promise = Promise.resolve()): Promise { + return new Promise((_resolve, reject) => { + const onAbort = (): void => { + void release.then(() => { + reject(signal.reason instanceof Error ? signal.reason : new Error(String(signal.reason))) + }) + } + signal.addEventListener('abort', onAbort, { once: true }) + if (signal.aborted) onAbort() + }) +} diff --git a/packages/lsp/lsp-local/tests/provider.spec.ts b/packages/lsp/lsp-local/tests/provider.spec.ts index 77fcbe7851..88b2b0144f 100644 --- a/packages/lsp/lsp-local/tests/provider.spec.ts +++ b/packages/lsp/lsp-local/tests/provider.spec.ts @@ -1,9 +1,10 @@ -import { afterEach, beforeEach, describe, expect, it } from 'vitest' +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' import { chmod, mkdtemp, mkdir, rm, writeFile, realpath } from 'node:fs/promises' import { tmpdir } from 'node:os' import { delimiter, join } from 'node:path' import { Context } from 'cordis' import LocalSubprocessService from '@deepseek-ai/dsh-subprocess-local' +import LocalFileSystem from '@deepseek-ai/dsh-fs-local' import Lsp, { type LspQueryRequest } from '@deepseek-ai/dsh-lsp' import * as LspLocal from '@deepseek-ai/dsh-lsp-local' import type { Config, LspLocalServerConfig } from '@deepseek-ai/dsh-lsp-local' @@ -44,6 +45,7 @@ describe('lsp-local provider resolution', () => { const ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) await expect(ctx.plugin(LspLocal, config('onpath', { command: 'fake-lsp', args: [], @@ -57,6 +59,7 @@ describe('lsp-local provider resolution', () => { const ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) await expect(ctx.plugin(LspLocal, config('nope', { command: 'fake-lsp', args: [], @@ -71,6 +74,7 @@ describe('lsp-local provider resolution', () => { const ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) // Grab the provider instance by registering, then dispose the whole plugin fiber. const lsp = ctx.lsp const fiber = await ctx.plugin(LspLocal, config('disp', { @@ -88,6 +92,7 @@ describe('lsp-local provider resolution', () => { const ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) await expect(ctx.plugin(LspLocal, config('bad-budget', { command: process.execPath, args: ['-e', ''], @@ -101,6 +106,7 @@ describe('lsp-local provider resolution', () => { const ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) await expect(ctx.plugin(LspLocal, config('bad-cap', { command: process.execPath, args: ['-e', ''], @@ -114,6 +120,7 @@ describe('lsp-local provider resolution', () => { const ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) await expect(ctx.plugin(LspLocal, config('bad-timer', { command: process.execPath, args: ['-e', ''], @@ -130,6 +137,7 @@ describe('lsp-local provider resolution', () => { const ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) await expect(ctx.plugin(LspLocal, config('abs-bad', { command: notExe, args: [], @@ -142,6 +150,7 @@ describe('lsp-local provider resolution', () => { const ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) await expect(ctx.plugin(LspLocal, config('abs-directory', { command: ws, args: [], @@ -154,6 +163,7 @@ describe('lsp-local provider resolution', () => { const ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) await expect(ctx.plugin(LspLocal, { servers: {} })).rejects.toThrow(/servers must contain at least one server/) await ctx.fiber.dispose() }) @@ -162,6 +172,7 @@ describe('lsp-local provider resolution', () => { const ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) await expect(ctx.plugin(LspLocal, config('', { command: process.execPath, extensionToLanguage: { '.ts': 'typescript' }, @@ -173,6 +184,7 @@ describe('lsp-local provider resolution', () => { const ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) await expect(ctx.plugin(LspLocal, { servers: { valid: { command: process.execPath, extensionToLanguage: { '.ts': 'typescript' } }, @@ -183,10 +195,90 @@ describe('lsp-local provider resolution', () => { await ctx.fiber.dispose() }) + it('waits for aborted sibling executable lookups before setup rejects', async () => { + const ctx = new Context() + await ctx.plugin(Lsp) + await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) + const slowStarted = Promise.withResolvers() + const slowAborted = Promise.withResolvers() + const releaseCleanup = Promise.withResolvers() + vi.spyOn(ctx.subprocess, 'resolveExecutable').mockImplementation(async (command, _env, signal) => { + if (signal === undefined) throw new Error('missing setup signal') + if (command === 'slow-lsp') { + return await new Promise((_resolve, reject) => { + const onAbort = (): void => { + slowAborted.resolve(undefined) + void releaseCleanup.promise.then(() => { + reject(signal.reason instanceof Error ? signal.reason : new Error(String(signal.reason))) + }) + } + signal.addEventListener('abort', onAbort, { once: true }) + slowStarted.resolve(undefined) + if (signal.aborted) onAbort() + }) + } + await slowStarted.promise + throw new Error('lookup failed') + }) + + const loading = ctx.plugin(LspLocal, { + servers: { + slow: { command: 'slow-lsp', extensionToLanguage: { '.ts': 'typescript' } }, + failing: { command: 'failing-lsp', extensionToLanguage: { '.js': 'javascript' } }, + }, + }) + await slowAborted.promise + let settled = false + void loading.then(() => { settled = true }, () => { settled = true }) + await new Promise((resolve) => { setImmediate(resolve) }) + expect(settled).toBe(false) + + releaseCleanup.resolve(undefined) + await expect(loading).rejects.toThrow('lookup failed') + await ctx.fiber.dispose() + }) + + it('aborts executable resolution when disposed during setup', async () => { + const ctx = new Context() + await ctx.plugin(Lsp) + await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) + const subprocess = ctx.subprocess + const lookupStarted = Promise.withResolvers() + vi.spyOn(subprocess, 'resolveExecutable').mockImplementation(async (_command, _env, signal) => { + if (signal === undefined) throw new Error('missing setup signal') + lookupStarted.resolve(signal) + return await new Promise((_resolve, reject) => { + const onAbort = (): void => { + reject(signal.reason instanceof Error ? signal.reason : new Error(String(signal.reason))) + } + signal.addEventListener('abort', onAbort, { once: true }) + if (signal.aborted) onAbort() + }) + }) + + const loading = ctx.plugin(LspLocal, config('pending', { + command: 'pending-lsp', + extensionToLanguage: { '.ts': 'typescript' }, + })) + const signal = await lookupStarted.promise + const unrelated = await ctx.plugin(() => {}) + await unrelated.dispose() + expect(signal.aborted).toBe(false) + const disposing = loading.dispose() + + await expect(loading).rejects.toThrow('lsp-local setup disposed') + await expect(disposing).resolves.toBeUndefined() + expect(signal.aborted).toBe(true) + await ctx.fiber.dispose() + }) + it('rolls back earlier registrations when a later server conflicts', async () => { const ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) await expect(ctx.plugin(LspLocal, { servers: { first: { command: process.execPath, extensionToLanguage: { '.ts': 'typescript' } }, diff --git a/packages/lsp/lsp-local/tests/typescript-server.e2e.ts b/packages/lsp/lsp-local/tests/typescript-server.e2e.ts index fe9230ce14..9bdb7d1fd4 100644 --- a/packages/lsp/lsp-local/tests/typescript-server.e2e.ts +++ b/packages/lsp/lsp-local/tests/typescript-server.e2e.ts @@ -11,6 +11,7 @@ import { tmpdir } from 'node:os' import { join } from 'node:path' import { Context } from 'cordis' import LocalSubprocessService from '@deepseek-ai/dsh-subprocess-local' +import LocalFileSystem from '@deepseek-ai/dsh-fs-local' import Lsp, { type LspQueryRequest, type LspQueryResult } from '@deepseek-ai/dsh-lsp' import * as LspLocal from '@deepseek-ai/dsh-lsp-local' @@ -54,6 +55,7 @@ beforeAll(async () => { ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) await ctx.plugin(LspLocal, { servers: { typescript: { diff --git a/packages/lsp/lsp-local/tsconfig.json b/packages/lsp/lsp-local/tsconfig.json index 2b106ddc81..04fdeb9a84 100644 --- a/packages/lsp/lsp-local/tsconfig.json +++ b/packages/lsp/lsp-local/tsconfig.json @@ -26,6 +26,9 @@ { "path": "../../llm/llm" }, + { + "path": "../../fs/fs" + }, { "path": "../lsp" }, diff --git a/packages/lsp/lsp/README.i18n.yaml b/packages/lsp/lsp/README.i18n.yaml index 3faf906868..02649a8d39 100644 --- a/packages/lsp/lsp/README.i18n.yaml +++ b/packages/lsp/lsp/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/lsp/lsp/README.md -README.md: f96fc67ec8cb95f423eff9b312b7b591ec9d3008 -README.zh.md: 9757147de692684747d9965efda6329b3bbe2638 +README.md: 5c1044be50368acf13d8c36a15d5b2bd99d02701 +README.zh.md: cc412333e63b9469319240d67269bf0192ad3858 diff --git a/packages/lsp/lsp/README.md b/packages/lsp/lsp/README.md index f96fc67ec8..5c1044be50 100644 --- a/packages/lsp/lsp/README.md +++ b/packages/lsp/lsp/README.md @@ -27,7 +27,7 @@ Providers register **capabilities**, not tools. `dsh-tool-lsp` is the only owner ## Vocabulary -`LspQueryRequest` (`operation`, `filePath`, `position`, `workspaceRoot`) — every field required, so no field needs implementation defaulting and there is no `resolve()` step. Positions and ranges are zero-based UTF-16, matching the protocol; the tool owns the one-based cursor convention. `findReferences` always includes declarations — providers enforce this internally, so callers get no flag. `LspQueryResult` is a CLOSED discriminated union: `{ kind: 'locations'; locations; resolvedWorkspaceRoot }` for navigation, `{ kind: 'hover'; hover }` for hover (content or `null`) — consumers `switch` to exhaustiveness so a new arm breaks compilation until handled. `resolvedWorkspaceRoot` is the provider's canonical form of the request's `workspaceRoot` and the root its `file:` URIs are relative to; a caller relativizing display paths uses it, not the (possibly symlinked) request root. See `src/types.ts` for the full contracts and `src/index.ts` for the `LspError` codes, including `LSP_DISPOSED` and `LSP_MALFORMED_RESPONSE`. +`LspQueryRequest` (`operation`, `filePath`, `position`, `workspaceRoot`) — every field required, so no field needs implementation defaulting and there is no `resolve()` step. Positions and ranges are zero-based UTF-16, matching the protocol; the tool owns the one-based cursor convention. `findReferences` always includes declarations — providers enforce this internally, so callers get no flag. `LspQueryResult` is a CLOSED discriminated union: `{ kind: 'locations'; locations; resolvedWorkspaceUri }` for navigation, `{ kind: 'hover'; hover }` for hover (content or `null`) — consumers `switch` to exhaustiveness so a new arm breaks compilation until handled. `resolvedWorkspaceUri` is the provider's canonical workspace `file:` URI; callers relativize location URIs against it instead of applying host-platform path rules to the possibly symlinked request root. See `src/types.ts` for the full contracts and `src/index.ts` for the `LspError` codes, including `LSP_DISPOSED` and `LSP_MALFORMED_RESPONSE`. ## Model Experience diff --git a/packages/lsp/lsp/README.zh.md b/packages/lsp/lsp/README.zh.md index 9757147de6..cc412333e6 100644 --- a/packages/lsp/lsp/README.zh.md +++ b/packages/lsp/lsp/README.zh.md @@ -8,7 +8,7 @@ | 包 | 职责 | |---|---| -| `@deepseek-ai/dsh-lsp`（本包） | 接口：服务、以带品牌类型的 id 和扩展名映射为键的提供方注册表、逐查询选择、请求／结果词汇、`LspError` 分类体系 | +| `@deepseek-ai/dsh-lsp`（本包） | 接口：服务、以品牌化 id + 扩展名映射为 key 的提供方注册表、逐查询选择、请求／结果词汇、`LspError` 分类体系 | | `@deepseek-ai/dsh-lsp-local` | 通用本地后端，注册已配置的 stdio 语言服务器提供方 | | `@deepseek-ai/dsh-tool-lsp` | 面向模型的 `lsp` 工具，基于 `ctx.lsp` | @@ -18,16 +18,16 @@ | 成员 | 语义 | |---|---| -| `registerProvider(provider)` | 注册后端，以原子方式保留其带品牌类型的 `id` 与每个规范化文件扩展名。任何无效输入或冲突都不会发布内容，并抛出 `LspError`（`LSP_INVALID_PROVIDER`／`LSP_CONFLICT`）。返回释放所有保留项的 disposer。随调用 fiber 一同 dispose（资源释放）。 | +| `registerProvider(provider)` | 注册后端，以原子方式保留其品牌化 `id` 与每个规范化文件扩展名。任何无效输入或冲突都不会发布内容，并抛出 `LspError`（`LSP_INVALID_PROVIDER`／`LSP_CONFLICT`）。返回释放所有保留项的 disposer。随调用 fiber 释放。 | | `query(request, signal?)` | 按文件最终扩展名选择提供方，从该提供方的映射派生 `languageId`，并运行一次查询。没有匹配项时抛出 `LspError` `LSP_UNAVAILABLE`。 | -选择逐查询进行且与顺序无关：一个提供方独占一组扩展名，因此注册和 HMR（热模块替换）顺序绝不会改变路由。扩展名 key 规范化为小写且以点开头；`languageId` 只用于同步临时文档，绝不参与选择。第一版没有 glob、language-id 或显式路由 selector。 +选择逐查询进行且与顺序无关：一个提供方独占一组扩展名，因此注册和 HMR 顺序绝不会改变路由。扩展名 key 规范化为小写且以点开头；`languageId` 只用于同步临时文档，绝不参与选择。第一版没有 glob、language-id 或显式路由 selector。 -提供方注册的是**能力**，而非工具。`dsh-tool-lsp` 是面向模型名称、描述、提示词指引、schema 和呈现的唯一 owner。 +提供方注册的是**能力** 而非工具。`dsh-tool-lsp` 是面向模型名称、描述、提示词指引、schema 和呈现的唯一 owner。 ## 词汇 -`LspQueryRequest`（`operation`、`filePath`、`position`、`workspaceRoot`）：每个字段都必填，因此没有字段需要实现默认值，也不存在 `resolve()` 步骤。位置与范围使用从零开始的 UTF-16，与协议一致；工具负责从 1 开始的光标约定。`findReferences` 始终包含声明，提供方在内部强制执行，因此调用方没有 flag。`LspQueryResult` 是封闭的判别联合：导航使用 `{ kind: 'locations'; locations; resolvedWorkspaceRoot }`，悬停使用 `{ kind: 'hover'; hover }`（内容或 `null`）；消费方通过 `switch` 实现穷尽检查，因此新增分支会使编译失败，直到完成处理。`resolvedWorkspaceRoot` 是提供方对请求 `workspaceRoot` 的规范形式，也是其 `file:` URI 所相对的根；调用方把显示路径相对化时使用该值，而非可能含符号链接的请求根。完整契约见 `src/types.ts`；`src/index.ts` 给出 `LspError` 代码，包括 `LSP_DISPOSED` 和 `LSP_MALFORMED_RESPONSE`。 +`LspQueryRequest`（`operation`、`filePath`、`position`、`workspaceRoot`）：每个字段都必填，因此没有字段需要实现默认值，也不存在 `resolve()` 步骤。位置与范围使用从零开始的 UTF-16，与协议一致；工具拥有从 1 开始的光标约定。`findReferences` 始终包含声明，提供方在内部强制执行，因此调用方没有 flag。`LspQueryResult` 是封闭的判别联合：导航使用 `{ kind: 'locations'; locations; resolvedWorkspaceUri }`，悬停使用 `{ kind: 'hover'; hover }`（内容或 `null`）；消费方通过 `switch` 实现穷尽检查，因此新增分支会使编译失败，直到完成处理。`resolvedWorkspaceUri` 是提供方的规范工作区 `file:` URI；调用方相对化位置 URI 时以它为基准，而不是对可能含符号链接的请求根应用宿主平台路径规则。完整契约见 `src/types.ts`；`src/index.ts` 给出 `LspError` code，包括 `LSP_DISPOSED` 和 `LSP_MALFORMED_RESPONSE`。 ## 模型体验 @@ -39,6 +39,6 @@ ## 已知限制与暂缓事项 -- **同一运行时内扩展名归属互斥**：两个提供方不能同时声明 `.ts`，即使语言 ID 不同；重叠会使注册失败。预期扩展是在注册之上增加部署配置的 selector；它可以放宽互斥保留，而无需把提供方选择加入模型输入（见 [seam Agent Note](../../../.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md)）。 +- **同一运行时内扩展名归属互斥**：两个提供方不能同时声明 `.ts`，即使 language id 不同；重叠会使注册失败。预期扩展是在注册之上增加部署配置的 selector；它可以放宽互斥保留，而无需把提供方选择加入模型输入（见 [seam Agent Note](../../../.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md)）。 - **仅四种操作**：symbol 与 call hierarchy 暂缓（它们需要不同 schema）；diagnostics 需要独立的新鲜度／累积规则；修改操作（rename、code action、formatting）需要独立工具，并集成预览、权限和写入策略。 -- **没有观测接口**：可用性只能通过运行 `query()` 并按抛出的 `LspError` 代码进行路由来观测；没有提供方变更事件或能力状态查询。 +- **没有观测表层**：可用性只能通过运行 `query()` 并按抛出的 `LspError` code 路由来观测；没有提供方变更事件或能力状态查询。 diff --git a/packages/lsp/lsp/src/types.ts b/packages/lsp/lsp/src/types.ts index d0c84f606d..d1bebe2a58 100644 --- a/packages/lsp/lsp/src/types.ts +++ b/packages/lsp/lsp/src/types.ts @@ -77,13 +77,13 @@ export interface LspHover { * `goToImplementation`) normalize to `locations`; `hover` normalizes to content or `null`. * Consumers `switch` on `kind` to exhaustiveness so a new arm breaks compilation until handled. * - * The `locations` variant carries `resolvedWorkspaceRoot`: the provider's canonical form of the - * request's `workspaceRoot`, and the root its `file:` location URIs are relative to. A caller that - * relativizes display paths MUST use this, not the request's (possibly symlinked) `workspaceRoot`; - * otherwise a symlinked workspace misclassifies in-workspace results as external. + * The `locations` variant carries `resolvedWorkspaceUri`: the provider's canonical `file:` URI for + * the request's workspace root. A caller that relativizes location URIs MUST use this, not parse the + * request's possibly symlinked process path with host-platform rules; the execution platform may + * differ from the caller's. */ export type LspQueryResult = - | { readonly kind: 'locations'; readonly locations: readonly LspLocation[]; readonly resolvedWorkspaceRoot: string } + | { readonly kind: 'locations'; readonly locations: readonly LspLocation[]; readonly resolvedWorkspaceUri: string } | { readonly kind: 'hover'; readonly hover: LspHover | null } /** diff --git a/packages/lsp/lsp/tests/lsp.spec.ts b/packages/lsp/lsp/tests/lsp.spec.ts index 77ea9687c7..86c9b0ac53 100644 --- a/packages/lsp/lsp/tests/lsp.spec.ts +++ b/packages/lsp/lsp/tests/lsp.spec.ts @@ -13,7 +13,7 @@ import Lsp, { function makeProvider( id: string, extensionToLanguage: Record, - result: LspQueryResult = { kind: 'locations', locations: [], resolvedWorkspaceRoot: '/ws' }, + result: LspQueryResult = { kind: 'locations', locations: [], resolvedWorkspaceUri: 'file:///ws' }, ): LspProvider & { seen: LspProviderQuery[]; seenSignals: (AbortSignal | undefined)[] } { const seen: LspProviderQuery[] = [] const seenSignals: (AbortSignal | undefined)[] = [] @@ -63,7 +63,7 @@ describe('Lsp registration', () => { const provider = makeProvider('ts', { '.ts': 'typescript' }) const dispose = lsp.registerProvider(provider) - await expect(lsp.query(query('a.ts'))).resolves.toEqual({ kind: 'locations', locations: [], resolvedWorkspaceRoot: '/ws' }) + await expect(lsp.query(query('a.ts'))).resolves.toEqual({ kind: 'locations', locations: [], resolvedWorkspaceUri: 'file:///ws' }) expect(provider.seen[0]).toMatchObject({ filePath: 'a.ts', languageId: 'typescript' }) dispose() @@ -148,7 +148,7 @@ describe('Lsp registration', () => { const py = makeProvider('py', { '.py': 'python' }) lsp.registerProvider(ts) lsp.registerProvider(py) - await expect(lsp.query(query('a.py'))).resolves.toEqual({ kind: 'locations', locations: [], resolvedWorkspaceRoot: '/ws' }) + await expect(lsp.query(query('a.py'))).resolves.toEqual({ kind: 'locations', locations: [], resolvedWorkspaceUri: 'file:///ws' }) await expect(lsp.query(query('a.ts', 'hover'))).resolves.toEqual(hover) }) @@ -172,7 +172,7 @@ describe('Lsp registration', () => { const fiber = await ctx.plugin(Object.assign((inner: Context) => { inner.lsp.registerProvider(makeProvider('ts', { '.ts': 'typescript' })) }, { inject: ['lsp'] })) - await expect(lsp.query(query('a.ts'))).resolves.toEqual({ kind: 'locations', locations: [], resolvedWorkspaceRoot: '/ws' }) + await expect(lsp.query(query('a.ts'))).resolves.toEqual({ kind: 'locations', locations: [], resolvedWorkspaceUri: 'file:///ws' }) await fiber.dispose() await expect(lsp.query(query('a.ts'))).rejects.toThrow(expect.objectContaining({ code: 'LSP_UNAVAILABLE' })) }) diff --git a/packages/lsp/tool-lsp/README.i18n.yaml b/packages/lsp/tool-lsp/README.i18n.yaml index e2e7c7517f..85a33f73af 100644 --- a/packages/lsp/tool-lsp/README.i18n.yaml +++ b/packages/lsp/tool-lsp/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/lsp/tool-lsp/README.md -README.md: 9b4130015ddf7e1cad6fa9a0e131be86f3bd4bcc -README.zh.md: a3f59fb6e39bb6c19cc2ef06d0bc256633e75386 +README.md: 1e89abf22e853735c094d17047b2c946af210905 +README.zh.md: 396b4d22ada8761b17c01d0c40e4bb4cfe80ca67 diff --git a/packages/lsp/tool-lsp/README.md b/packages/lsp/tool-lsp/README.md index 9b4130015d..1e89abf22e 100644 --- a/packages/lsp/tool-lsp/README.md +++ b/packages/lsp/tool-lsp/README.md @@ -10,7 +10,7 @@ Namespace plugin (`name` / `inject` / `Config` / `apply`, no default export). In `lsp` accepts `operation` (`goToDefinition` | `findReferences` | `goToImplementation` | `hover`), `file_path`, `line`, and `character`. `line` and `character` are positive, one-based UTF-16 cursor coordinates; the tool converts them to the seam's zero-based positions and converts rendered locations back. `findReferences` includes declarations so impact analysis does not omit the defining site. Provider, language id, workspace root, limits, timeout, initialization, and executable stay outside model input. -The tool requires the workspace root from the session `header.cwd`, with no fallback: absence fails as `LSP_WORKSPACE_REQUIRED` before querying. Its canonical result is the complete normalized seam union: `{ kind: "locations", locations, resolvedWorkspaceRoot }` or `{ kind: "hover", hover }`; Code Mode can inspect every acquired location and zero-based range directly. Native rendering then projects stable, file-grouped `path:line:character` entries relativized against the result's `resolvedWorkspaceRoot` (the provider's canonical root), not the session cwd — so a symlinked cwd still renders in-workspace results as workspace-relative paths; a `file:` URI becomes a workspace-relative path (inside) or absolute path (outside), and any other URI stays verbatim. Empty locations and `null` hover are successful no-result responses; malformed provider payloads remain structured errors. +The tool requires the workspace root from the session `header.cwd`, with no fallback: absence fails as `LSP_WORKSPACE_REQUIRED` before querying. Its canonical result is the complete normalized seam union: `{ kind: "locations", locations, resolvedWorkspaceUri }` or `{ kind: "hover", hover }`; Code Mode can inspect every acquired location and zero-based range directly. Native rendering projects stable, file-grouped `path:line:character` entries against the provider's canonical workspace URI rather than applying host-platform path rules to the session cwd. A `file:` URI becomes a workspace-relative path inside that URI or a URI-derived absolute path outside it; malformed and non-`file:` URIs stay verbatim. Empty locations and `null` hover are successful no-result responses; malformed provider payloads remain structured errors. ## Configuration diff --git a/packages/lsp/tool-lsp/README.zh.md b/packages/lsp/tool-lsp/README.zh.md index a3f59fb6e3..396b4d22ad 100644 --- a/packages/lsp/tool-lsp/README.zh.md +++ b/packages/lsp/tool-lsp/README.zh.md @@ -8,13 +8,13 @@ Namespace 插件（`name`／`inject`／`Config`／`apply`，无默认导出） ## 工具 -`lsp` 接受 `operation`（`goToDefinition` | `findReferences` | `goToImplementation` | `hover`）、`file_path`、`line` 和 `character`。`line` 与 `character` 是正的、从 1 开始的 UTF-16 光标坐标；工具将其转换为 seam 从零开始的位置，并把渲染位置转换回来。`findReferences` 包含声明，因此影响分析不会遗漏定义位置。提供方、语言 ID、工作区根目录、限制、超时、初始化和可执行文件均不进入模型输入。 +`lsp` 接受 `operation`（`goToDefinition` | `findReferences` | `goToImplementation` | `hover`）、`file_path`、`line` 和 `character`。`line` 与 `character` 是正的、从 1 开始的 UTF-16 光标坐标；工具将其转换为 seam 从零开始的位置，并把渲染位置转换回来。`findReferences` 包含声明，因此影响分析不会遗漏定义位置。提供方、language id、Workspace 根、限制、超时、初始化和可执行文件均不进入模型输入。 -该工具要求从会话 `header.cwd` 取得工作区根目录，没有回退值：缺失时会在查询前以 `LSP_WORKSPACE_REQUIRED` 失败。其规范结果是完整的已规范化 seam 联合：`{ kind: "locations", locations, resolvedWorkspaceRoot }` 或 `{ kind: "hover", hover }`；Code Mode 可以直接检查每个已取得的位置和从零开始的范围。Native 渲染随后投影出稳定的、按文件分组的 `path:line:character` 条目，并相对于结果的 `resolvedWorkspaceRoot`（提供方的规范根目录）而非会话 cwd；因此，即使 cwd 包含符号链接，工作区内的结果仍渲染为相对路径。`file:` URI 位于工作区内时成为工作区相对路径，位于工作区外时成为绝对路径，其他 URI 保持原样。空位置和 `null` hover 都是成功的无结果响应；格式错误的提供方载荷仍是结构化错误。 +该工具要求从会话 `header.cwd` 取得 Workspace 根，没有回退值：缺失时会在查询前以 `LSP_WORKSPACE_REQUIRED` 失败。其规范结果是完整的已规范化 seam 联合：`{ kind: "locations", locations, resolvedWorkspaceUri }` 或 `{ kind: "hover", hover }`；Code Mode 可以直接检查每个已取得的位置和从零开始的范围。原生渲染以提供方的规范工作区 URI 为基准，投影按文件稳定分组的 `path:line:character` 条目，而不对会话 cwd 应用宿主平台路径规则。`file:` URI 落在该工作区 URI 内时成为工作区相对路径，位于其外时成为从 URI 派生的绝对路径；格式错误的 URI 与非 `file:` URI 保持原样。空位置和 `null` hover 都是成功的无结果响应；格式错误的提供方载荷仍是结构化错误。 ## 配置 -| 配置键 | 默认值 | 含义 | +| Key | 默认值 | 含义 | |---|---|---| | `maxLocations` | `100` | 出现省略标记前可渲染位置的最大数量。 | | `maxResultChars` | `16000` | 完整渲染结果的最大长度，包括截断元数据。 | @@ -40,7 +40,7 @@ Use search/read for ordinary navigation. Use lsp when textual matches are ambigu #### KV Cache 影响 -只要插件作用域与指引文本不变，前缀就保持稳定；激活或 dispose（资源释放）可能使从该区段起的复用失效。 +只要插件 scope 与指引文本不变，前缀就保持稳定；激活或释放可能使从该区段起的复用失效。 ### 工具 schema @@ -54,13 +54,13 @@ Use search/read for ordinary navigation. Use lsp when textual matches are ambigu #### KV Cache 影响 -只要可见工具定义与顺序不变，前缀就保持稳定；注册生命周期或作用域限制可能使从第一个变化的 schema token 起的复用失效。 +只要可见工具定义与顺序不变，前缀就保持稳定；注册生命周期或 scope 限制可能使从第一个变化的 schema token 起的复用失效。 ### 结果 #### 模型看到的内容 -按文件分组的 `path:line:character` 位置行或规范化 hover 文本，先由 `maxLocations` 限制，再由 `maxResultChars` 限制；省略与截断标记计入完整字符上限。这些上限只影响 Native／模型呈现，不影响规范值。空结果使用不同的 `No results.`／`No hover information.` 行。 +按文件分组的 `path:line:character` 位置行或规范化 hover 文本，先由 `maxLocations` 限制，再由 `maxResultChars` 限制；省略与截断标记计入完整字符上限。这些上限只影响原生／模型呈现，不影响规范值。空结果使用不同的 `No results.`／`No hover information.` 行。 #### Token 影响 @@ -74,7 +74,7 @@ Use search/read for ordinary navigation. Use lsp when textual matches are ambigu #### 模型看到的内容 -无。客户端渲染通用搜索卡片：`{ card: 'generic', kind: 'search', title, locations: [{ path, line }] }`；从 args 派生的标题携带操作与从 1 开始的光标，编辑器跟随定位会聚焦所查询的行，标题则保留列号。 +无。客户端渲染通用搜索卡片：`{ card: 'generic', kind: 'search', title, locations: [{ path, line }] }`；从 args 派生的标题携带操作与从 1 开始的光标，跟随焦点对准查询行，标题则保留列号。 #### Token 影响 @@ -86,5 +86,5 @@ Use search/read for ordinary navigation. Use lsp when textual matches are ambigu ## 已知限制与暂缓事项 -- **UTF-16 光标坐标**：列坐标与协议精确一致，但模型难以在非 BMP 字符周围计数；不在符号上的位置可能返回空结果，因此提示词解释了该约定，但不会鼓励宽泛使用 LSP（见 [seam Agent Note](../../../.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md)）。 +- **UTF-16 光标坐标**：列坐标与协议精确一致，但模型难以在非 BMP 字符周围计数；非 symbol 位置可能返回空结果，因此提示词解释了该约定，但不会鼓励宽泛使用 LSP（见 [seam Agent Note](../../../.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md)）。 - **不承诺跨服务器完整性**：受支持的服务器仍可能根据索引就绪情况返回空或部分结果；该工具不承诺跨语言或服务器的完整性。 diff --git a/packages/lsp/tool-lsp/package.json b/packages/lsp/tool-lsp/package.json index 28b5a34bdb..f757df35fd 100644 --- a/packages/lsp/tool-lsp/package.json +++ b/packages/lsp/tool-lsp/package.json @@ -38,11 +38,13 @@ }, "devDependencies": { "@deepseek-ai/dsh-agent": "workspace:^", + "@deepseek-ai/dsh-fs-local": "workspace:^", "@deepseek-ai/dsh-invariants": "workspace:^", "@deepseek-ai/dsh-llm": "workspace:^", "@deepseek-ai/dsh-lsp": "workspace:^", "@deepseek-ai/dsh-lsp-local": "workspace:^", "@deepseek-ai/dsh-session": "workspace:^", + "@deepseek-ai/dsh-subprocess-local": "workspace:^", "@deepseek-ai/dsh-system-prompt": "workspace:^", "@deepseek-ai/dsh-timeout": "workspace:^", "@deepseek-ai/dsh-timeout-policy": "workspace:^", diff --git a/packages/lsp/tool-lsp/src/index.ts b/packages/lsp/tool-lsp/src/index.ts index e47dbc87db..688b48b369 100644 --- a/packages/lsp/tool-lsp/src/index.ts +++ b/packages/lsp/tool-lsp/src/index.ts @@ -138,7 +138,7 @@ export function apply(ctx: Context, config: Config): void { }, }, }, - resolvedWorkspaceRoot: { type: 'string', required: true }, + resolvedWorkspaceUri: { type: 'string', required: true }, }, }, { @@ -167,7 +167,7 @@ export function apply(ctx: Context, config: Config): void { render: (_args, value) => { switch (value.kind) { case 'locations': - return [{ type: 'text', text: formatLocations(value.locations, value.resolvedWorkspaceRoot, resolved.maxLocations, resolved.maxResultChars) }] + return [{ type: 'text', text: formatLocations(value.locations, value.resolvedWorkspaceUri, resolved.maxLocations, resolved.maxResultChars) }] case 'hover': return [{ type: 'text', text: formatHover(value.hover, resolved.maxResultChars) }] /* v8 ignore next -- exhaustive over the output schema's closed union; unreachable. */ @@ -200,7 +200,7 @@ export function apply(ctx: Context, config: Config): void { end: { line: location.range.end.line, character: location.range.end.character }, }, })), - resolvedWorkspaceRoot: result.resolvedWorkspaceRoot, + resolvedWorkspaceUri: result.resolvedWorkspaceUri, } case 'hover': return { diff --git a/packages/lsp/tool-lsp/src/render.ts b/packages/lsp/tool-lsp/src/render.ts index 215dfb60e6..ad77b2a6e2 100644 --- a/packages/lsp/tool-lsp/src/render.ts +++ b/packages/lsp/tool-lsp/src/render.ts @@ -6,10 +6,10 @@ * @module @deepseek-ai/dsh-tool-lsp/render */ -import { fileURLToPath } from 'node:url' -import { isAbsolute, relative, sep } from 'node:path' import type { GenericCallView } from '@deepseek-ai/dsh-tools' import type { LspHover, LspLocation, LspOperation, LspPosition } from '@deepseek-ai/dsh-lsp' +import { posix, win32 } from 'node:path' +import { fileURLToPath } from 'node:url' /** The four operations the tool exposes, as a runtime tuple for schema enum + validation. */ export const LSP_OPERATIONS: readonly LspOperation[] = ['goToDefinition', 'findReferences', 'goToImplementation', 'hover'] @@ -74,17 +74,17 @@ function oneBased(value: number, name: string): number { /** * Render a locations result grouped by file, converting each zero-based location back to a one-based * `path:line:character` entry. A `file:` URI inside the workspace becomes a workspace-relative path; - * outside it, an absolute path; a non-`file:` URI is kept verbatim. Applies `maxLocations` and + * outside it, a URI-derived absolute path; a non-`file:` URI is kept verbatim. Applies `maxLocations` and * appends an omission marker when it truncates by count, then applies the complete result cap. * @param locations - the seam's locations (possibly empty). - * @param workspaceRoot - the canonical workspace root for relativizing `file:` paths. + * @param workspaceUri - the provider's canonical workspace `file:` URI. * @param maxLocations - the cap before truncation. * @param maxResultChars - the complete rendered-text cap, including truncation metadata. * @returns the rendered text; a distinct no-result line when there are none. */ export function formatLocations( locations: readonly LspLocation[], - workspaceRoot: string, + workspaceUri: string, maxLocations: number, maxResultChars: number, ): string { @@ -93,7 +93,7 @@ export function formatLocations( const omitted = locations.length - shown.length const grouped = new Map() for (const location of shown) { - const path = renderUri(location.uri, workspaceRoot) + const path = renderUri(location.uri, workspaceUri) const line = location.range.start.line + 1 const character = location.range.start.character + 1 const entries = grouped.get(path) ?? [] @@ -128,27 +128,50 @@ function boundResult(text: string, maxChars: number, label: string): string { } /** - * Resolve a location URI to a display path. A `file:` URI accepted by Node becomes workspace-relative - * (inside) or absolute (outside); any other URI is returned verbatim. + * Resolve a location URI without applying the harness host's path rules. A valid `file:` URI becomes + * workspace-relative when it is under the provider's canonical workspace URI, or a URI-derived + * absolute path otherwise; malformed and non-`file:` URIs remain verbatim. * @param uri - the target URI from the seam. - * @param workspaceRoot - the canonical workspace root. + * @param workspaceUri - the provider's canonical workspace `file:` URI. * @returns the display path or the verbatim URI. */ -export function renderUri(uri: string, workspaceRoot: string): string { +export function renderUri(uri: string, workspaceUri: string): string { if (!uri.startsWith('file:')) return uri - let absolute: string + let target: URL + let workspace: URL try { - absolute = fileURLToPath(uri) + target = new URL(uri) + workspace = new URL(workspaceUri) } catch { - // A malformed file: URI is not a path we can resolve; show it verbatim. return uri } - const rel = relative(workspaceRoot, absolute) - if (rel === '') return '.' - // A leading `..` SEGMENT (or an absolute rel) means outside the workspace; guard against a false - // positive on an in-workspace path whose first component merely starts with dots (e.g. `..gen/x`). - const outside = rel === '..' || rel.startsWith(`..${sep}`) || isAbsolute(rel) - return outside ? absolute : rel.split(sep).join('/') + if (workspace.protocol !== 'file:') return uri + // A `file:` URI does not carry its world's OS, so a leading `/X:` segment is + // read as a Windows drive. A POSIX workspace literally rooted at `/c:/...` + // would mis-render (display only; edits and reads use the exact URI). + const drivePath = /^\/[a-z](?::|%3A)/iu + const windowsWorld = workspace.hostname.length > 0 || drivePath.test(workspace.pathname) + const targetWindowsWorld = windowsWorld && (target.hostname.length > 0 || drivePath.test(target.pathname)) + const workspacePath = filePath(workspace, windowsWorld) + const targetPath = filePath(target, targetWindowsWorld) + if (workspacePath === undefined || targetPath === undefined) return uri + if (windowsWorld !== targetWindowsWorld) return targetPath + const path = windowsWorld ? win32 : posix + const relative = path.relative(workspacePath, targetPath) + const outside = relative === '..' || relative.startsWith(`..${path.sep}`) || path.isAbsolute(relative) + const rendered = relative === '' ? '.' : outside ? targetPath : relative + return windowsWorld ? rendered.replaceAll('\\', '/') : rendered +} + +/** Decode a file URL for its execution world while containing malformed URL failures. */ +function filePath(url: URL, windows: boolean): string | undefined { + try { + const path = fileURLToPath(url, { windows }) + return path.includes('\0') ? undefined : path + } catch { + // `fileURLToPath` rejects malformed escapes, authorities, and encoded path separators. + return undefined + } } /** diff --git a/packages/lsp/tool-lsp/tests/integration.spec.ts b/packages/lsp/tool-lsp/tests/integration.spec.ts index 4d790cbd6b..8312a6e3ab 100644 --- a/packages/lsp/tool-lsp/tests/integration.spec.ts +++ b/packages/lsp/tool-lsp/tests/integration.spec.ts @@ -6,6 +6,7 @@ import { pathToFileURL } from 'node:url' import { Context } from 'cordis' import SystemPrompt from '@deepseek-ai/dsh-system-prompt' import ToolRegistry from '@deepseek-ai/dsh-tools' +import LocalFileSystem from '@deepseek-ai/dsh-fs-local' import LocalSubprocessService from '@deepseek-ai/dsh-subprocess-local' import Lsp from '@deepseek-ai/dsh-lsp' import * as LspLocal from '@deepseek-ai/dsh-lsp-local' @@ -50,6 +51,7 @@ async function mount(hang: boolean, timeoutMs?: number): Promise { await ctx.plugin(SystemPrompt) await ctx.plugin(ToolRegistry) await ctx.plugin(Lsp) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) await ctx.plugin(LocalSubprocessService) await ctx.plugin(LspLocal, { servers: { diff --git a/packages/lsp/tool-lsp/tests/render.spec.ts b/packages/lsp/tool-lsp/tests/render.spec.ts index 1fd0eeba51..2d181f25b3 100644 --- a/packages/lsp/tool-lsp/tests/render.spec.ts +++ b/packages/lsp/tool-lsp/tests/render.spec.ts @@ -14,6 +14,7 @@ import { import type { LspLocation } from '@deepseek-ai/dsh-lsp' const WS = resolve('/home/u/proj') +const WS_URI = pathToFileURL(WS).href function loc(uri: string, line: number, character = 0): LspLocation { return { uri, range: { start: { line, character }, end: { line, character: character + 1 } } } @@ -48,64 +49,92 @@ describe('parseLspArgs', () => { describe('renderUri', () => { it('relativizes a file: URI inside the workspace with forward slashes', () => { const uri = pathToFileURL(join(WS, 'src', 'a.ts')).href - expect(renderUri(uri, WS)).toBe('src/a.ts') + expect(renderUri(uri, WS_URI)).toBe('src/a.ts') }) it('returns an absolute path for a file: URI outside the workspace', () => { const outside = resolve(WS, '..', 'other', 'lib', 'b.ts') const uri = pathToFileURL(outside).href - expect(renderUri(uri, WS)).toBe(outside) + expect(renderUri(uri, WS_URI)).toBe(outside) }) it('renders the workspace root itself as "."', () => { - expect(renderUri(pathToFileURL(WS).href, WS)).toBe('.') + expect(renderUri(WS_URI, WS_URI)).toBe('.') }) it('keeps an in-workspace path whose first segment starts with dots relative', () => { // `..generated` is a real in-workspace dir, not a parent escape; only a `..` segment is external. const uri = pathToFileURL(join(WS, '..generated', 'a.ts')).href - expect(renderUri(uri, WS)).toBe('..generated/a.ts') + expect(renderUri(uri, WS_URI)).toBe('..generated/a.ts') + }) + + it('relativizes Windows execution-world URIs on a non-Windows host', () => { + expect(renderUri('file:///C:/WORKSPACE/src/a.ts', 'file:///c:/workspace')).toBe('src/a.ts') + expect(renderUri('file:///D:/lib/b.ts', 'file:///C:/workspace')).toBe('D:/lib/b.ts') + }) + + it('renders remote file authorities without host path conversion', () => { + expect(renderUri('file://server/share/workspace/a.ts', 'file://server/share/workspace')).toBe('a.ts') + expect(renderUri('file://SERVER/share/workspace/src/A.ts', 'file://server/Share/Workspace')).toBe('src/A.ts') + expect(renderUri('file://other/share/b.ts', 'file://server/share/workspace')).toBe('//other/share/b.ts') + expect(renderUri('file:///D:/lib/a.ts', 'file://server/share/workspace')).toBe('D:/lib/a.ts') + expect(renderUri('file:///a.ts', 'file://server/')).toBe('/a.ts') + expect(renderUri('file:///a.ts', 'file:///')).toBe('a.ts') + }) + + it('preserves backslashes as ordinary POSIX filename characters', () => { + expect(renderUri('file:///home/u/proj/dir%5Cname/a.ts', WS_URI)).toBe('dir\\name/a.ts') + }) + + it('keeps malformed or mismatched URI coordinates verbatim', () => { + expect(renderUri('file://[', WS_URI)).toBe('file://[') + expect(renderUri('file:///a.ts', 'https://example.com/workspace')).toBe('file:///a.ts') + expect(renderUri('file:///a.ts', 'file:///bad%ZZ')).toBe('file:///a.ts') + expect(renderUri('file:///C:/workspace/bad%5Cpath', 'file:///C:/workspace')).toBe('file:///C:/workspace/bad%5Cpath') + expect(renderUri('file:///short', 'file:///short/deeper')).toBe('/short') + expect(renderUri('file:///', 'file:///C:/workspace')).toBe('/') }) it('keeps a non-file URI verbatim', () => { - expect(renderUri('untitled:Untitled-1', WS)).toBe('untitled:Untitled-1') - expect(renderUri('jdt://contents/Foo.class', WS)).toBe('jdt://contents/Foo.class') + expect(renderUri('untitled:Untitled-1', WS_URI)).toBe('untitled:Untitled-1') + expect(renderUri('jdt://contents/Foo.class', WS_URI)).toBe('jdt://contents/Foo.class') }) it('keeps a malformed file: URI verbatim when it cannot be parsed to a path', () => { // An encoded path separator is invalid on every platform and must remain verbatim. - expect(renderUri('file:///bad%2Fpath', WS)).toBe('file:///bad%2Fpath') + expect(renderUri('file:///bad%2Fpath', WS_URI)).toBe('file:///bad%2Fpath') + expect(renderUri('file:///bad%00path', WS_URI)).toBe('file:///bad%00path') }) }) describe('formatLocations', () => { it('renders a no-result line for an empty list', () => { - expect(formatLocations([], WS, DEFAULT_MAX_LOCATIONS, DEFAULT_MAX_RESULT_CHARS)).toBe('No results.') + expect(formatLocations([], WS_URI, DEFAULT_MAX_LOCATIONS, DEFAULT_MAX_RESULT_CHARS)).toBe('No results.') }) it('renders one-based path:line:character grouped by file', () => { const a = pathToFileURL(join(WS, 'a.ts')).href - const text = formatLocations([loc(a, 0, 0), loc(a, 4, 2)], WS, DEFAULT_MAX_LOCATIONS, DEFAULT_MAX_RESULT_CHARS) + const text = formatLocations([loc(a, 0, 0), loc(a, 4, 2)], WS_URI, DEFAULT_MAX_LOCATIONS, DEFAULT_MAX_RESULT_CHARS) expect(text).toBe('a.ts:1:1\na.ts:5:3') }) it('caps at maxLocations and marks the omission', () => { const a = pathToFileURL(join(WS, 'a.ts')).href const many = Array.from({ length: 5 }, (_, i) => loc(a, i)) - const text = formatLocations(many, WS, 2, DEFAULT_MAX_RESULT_CHARS) + const text = formatLocations(many, WS_URI, 2, DEFAULT_MAX_RESULT_CHARS) expect(text).toContain('a.ts:1:1') expect(text).toContain('3 more locations omitted (limit 2).') }) it('uses the singular omission marker for exactly one extra', () => { const a = pathToFileURL(join(WS, 'a.ts')).href - const text = formatLocations([loc(a, 0), loc(a, 1)], WS, 1, DEFAULT_MAX_RESULT_CHARS) + const text = formatLocations([loc(a, 0), loc(a, 1)], WS_URI, 1, DEFAULT_MAX_RESULT_CHARS) expect(text).toContain('1 more location omitted (limit 1).') }) it('caps the complete location text even when one URI is enormous', () => { const maxResultChars = 80 - const text = formatLocations([loc(`custom:${'x'.repeat(1_000_000)}`, 0)], WS, 1, maxResultChars) + const text = formatLocations([loc(`custom:${'x'.repeat(1_000_000)}`, 0)], WS_URI, 1, maxResultChars) expect(text).toHaveLength(maxResultChars) expect(text).toContain('locations truncated') }) diff --git a/packages/lsp/tool-lsp/tests/tool-lsp.spec.ts b/packages/lsp/tool-lsp/tests/tool-lsp.spec.ts index de7a5319ef..846bd702ea 100644 --- a/packages/lsp/tool-lsp/tests/tool-lsp.spec.ts +++ b/packages/lsp/tool-lsp/tests/tool-lsp.spec.ts @@ -44,6 +44,7 @@ let seq = 0 const testToolSignal = new AbortController().signal const workspaceRoot = resolve('/virtual/workspace') const resolvedWorkspaceRoot = resolve('/virtual/real-workspace') +const resolvedWorkspaceUri = pathToFileURL(resolvedWorkspaceRoot).href const workspaceAlias = resolve('/virtual/workspace-alias') /** `cwd: null` means "no agent" (tests LSP_WORKSPACE_REQUIRED); a string is the session cwd. */ function call(ctx: Context, args: unknown, cwd: string | null = workspaceRoot) { @@ -59,7 +60,7 @@ function call(ctx: Context, args: unknown, cwd: string | null = workspaceRoot) { const okLocations: LspQueryResult = { kind: 'locations', locations: [{ uri: pathToFileURL(join(workspaceRoot, 'a.ts')).href, range: { start: { line: 0, character: 0 }, end: { line: 0, character: 1 } } }], - resolvedWorkspaceRoot: workspaceRoot, + resolvedWorkspaceUri: pathToFileURL(workspaceRoot).href, } describe('tool-lsp registration', () => { @@ -138,7 +139,7 @@ describe('tool-lsp execution', () => { const { ctx } = await mount(stubProvider(() => ({ kind: 'locations', locations, - resolvedWorkspaceRoot: cappedWorkspaceRoot, + resolvedWorkspaceUri: pathToFileURL(cappedWorkspaceRoot).href, })), { maxLocations: 1 }) const result = await call(ctx, { operation: 'findReferences', file_path: 'a.ts', line: 1, character: 1 }, cappedWorkspaceRoot) expect(result.content[0]).toEqual({ @@ -147,17 +148,17 @@ describe('tool-lsp execution', () => { }) expect(result).toMatchObject({ isError: false, - value: { kind: 'locations', locations, resolvedWorkspaceRoot: cappedWorkspaceRoot }, + value: { kind: 'locations', locations, resolvedWorkspaceUri: pathToFileURL(cappedWorkspaceRoot).href }, }) }) - it('relativizes against the provider resolvedWorkspaceRoot, not the session cwd', async () => { + it('relativizes against the provider resolvedWorkspaceUri, not the session cwd', async () => { // A symlinked session cwd resolves to the real path that contains the provider's location URIs. // Relativizing against the alias would misclassify the location as external. const provider = stubProvider(() => ({ kind: 'locations', locations: [{ uri: pathToFileURL(join(resolvedWorkspaceRoot, 'a.ts')).href, range: { start: { line: 0, character: 0 }, end: { line: 0, character: 1 } } }], - resolvedWorkspaceRoot, + resolvedWorkspaceUri, })) const { ctx } = await mount(provider) const result = await call(ctx, { operation: 'goToDefinition', file_path: 'a.ts', line: 1, character: 1 }, workspaceAlias) diff --git a/packages/pty/README.i18n.yaml b/packages/pty/README.i18n.yaml index 083a144b55..65d541a7c1 100644 --- a/packages/pty/README.i18n.yaml +++ b/packages/pty/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/pty/README.md -README.md: 9c8206464d45b1be1d6ee3861c57c128e77686c5 -README.zh.md: 70d081e60a7db61443ed616b64586a93c119a640 +README.md: a4f743056b4a524be9623b0f700f37e0534b463f +README.zh.md: c84ad3f1b59afcdbbd111f1b82c57c56aa24fdcf diff --git a/packages/pty/README.md b/packages/pty/README.md index 9c8206464d..a4f743056b 100644 --- a/packages/pty/README.md +++ b/packages/pty/README.md @@ -2,13 +2,12 @@ English | [中文](README.zh.md) -This family provides persistent, owner-scoped pseudo-terminal sessions for interactive or stateful terminal work. It complements one-shot bash execution. +`PTY` stands for **Pseudo-Terminal**（伪终端）. This capability provides persistent, owner-scoped terminal sessions for workflows that require state across tool calls or interactive stdin. PTY complements the one-shot bash and filesystem tools; it does not replace their stronger per-operation contracts. | Package | Role | ctx key | |---|---|---| -| [`pty/`](pty/README.md) | Defines the PTY service and session lifecycle | `ctx.pty` | -| [`pty-local/`](pty-local/README.md) | Provides local persistent terminal sessions | registers on `ctx.pty` | -| [`tool-pty/`](tool-pty/README.md) | Exposes PTY session operations to the model | registers on `ctx.tools` | -| [`tool-bash-persistent/`](tool-bash-persistent/README.md) | Exposes a reusable PTY-backed bash tool | registers on `ctx.tools` | +| [`pty`](pty/README.md) (`@deepseek-ai/dsh-pty`) | Backend registry, branded ids, exact-Agent ownership, session operations, and awaited cleanup | `ctx.pty` | +| `pty-local` (`@deepseek-ai/dsh-pty-local`) | Shell backend over `ctx.subprocess.spawnTerminal`: readiness detection, bounded terminal state, sandbox policy, and session operations | registers on `ctx.pty` | +| `tool-pty` (`@deepseek-ai/dsh-tool-pty`) | Six model-facing tools and generic task integration for background sends | registers on `ctx.tools` | -The [persistent PTY decision](../../.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md) records the family boundary. +The design and deferred boundaries live in the [persistent PTY Agent Note](../../.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md). diff --git a/packages/pty/README.zh.md b/packages/pty/README.zh.md index 70d081e60a..c84ad3f1b5 100644 --- a/packages/pty/README.zh.md +++ b/packages/pty/README.zh.md @@ -2,13 +2,12 @@ [English](README.md) | 中文 -本家族为交互式或有状态的终端工作提供持久且限定所有者范围的伪终端会话，是单次 bash 执行的补充。 +`PTY` 的全称是 **Pseudo-Terminal（伪终端）**。这项能力提供持久且限定所有者范围的终端会话，适用于需要跨工具调用保留状态或使用交互式 stdin 的工作流。PTY 是单次 bash 与文件系统工具的补充，不会取代后两者更严格的逐操作契约。 | 包 | 职责 | ctx 键 | |---|---|---| -| [`pty/`](pty/README.md) | 定义 PTY 服务和会话生命周期 | `ctx.pty` | -| [`pty-local/`](pty-local/README.md) | 提供本地持久终端会话 | 注册到 `ctx.pty` | -| [`tool-pty/`](tool-pty/README.md) | 向模型公开 PTY 会话操作 | 注册到 `ctx.tools` | -| [`tool-bash-persistent/`](tool-bash-persistent/README.md) | 公开可复用的 PTY 后端 bash 工具 | 注册到 `ctx.tools` | +| [`pty`](pty/README.md)（`@deepseek-ai/dsh-pty`） | 后端注册表、品牌化 id、精确的 Agent 所有权、会话操作与等待完成的清理 | `ctx.pty` | +| `pty-local`（`@deepseek-ai/dsh-pty-local`） | `ctx.subprocess.spawnTerminal` 之上的 shell 后端：就绪检测、有界终端状态、沙箱策略与会话操作 | 注册到 `ctx.pty` | +| `tool-pty`（`@deepseek-ai/dsh-tool-pty`） | 6 个面向模型的工具，并为后台发送集成通用任务 | 注册到 `ctx.tools` | -[持久 PTY 决策](../../.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md)记录了该家族的边界。 +设计与暂缓边界记录在[持久 PTY Agent Note](../../.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md) 中。 diff --git a/packages/pty/pty-local/README.i18n.yaml b/packages/pty/pty-local/README.i18n.yaml index 48ae675425..bd835a091d 100644 --- a/packages/pty/pty-local/README.i18n.yaml +++ b/packages/pty/pty-local/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/pty/pty-local/README.md -README.md: ba05495318127b63b3d2a6a60ec743e1ff1c5821 -README.zh.md: 81987ea0685d761507b535b7ed6eefa0888fbd54 +README.md: 5acc92853e6e8fcb8938c48e391559bf4a28fb75 +README.zh.md: 353c2a4bdac7e8402fc63071dfb6fb85dcff66d5 diff --git a/packages/pty/pty-local/README.md b/packages/pty/pty-local/README.md index ba05495318..5acc92853e 100644 --- a/packages/pty/pty-local/README.md +++ b/packages/pty/pty-local/README.md @@ -2,15 +2,15 @@ English | [中文](README.zh.md) -Local Linux/macOS `node-pty` backend for `ctx.pty`; loading it on another platform fails as unsupported. It starts an interactive shell under the shared `ctx.sandboxPolicy`, strips credential-shaped ambient environment variables, retains bounded line-oriented output, detects readiness, and tears down the captured process tree rooted at the `node-pty` child. +Persistent shell backend for `ctx.pty` over `ctx.subprocess.spawnTerminal`. It starts an interactive shell under the shared `ctx.sandboxPolicy`, retains bounded line-oriented output, and detects readiness while the subprocess provider owns PTY allocation, environment scrubbing, foreground process groups, signalling, and complete terminal-session cleanup. The same PTY backend therefore composes with local or remote execution-world providers. ## Plugin (`pty-local`) -The plugin injects `pty`, `sandbox`, and `sandboxPolicy`, then registers the configured backend type (`shell`). `danger-full-access` starts the shell directly; confined modes wrap the exact shell argv through `ctx.sandbox`. At spawn, one `ctx.sandboxPolicy.resolve({ session })` call supplies both the effective mode and the session workspace root; the same root is the default shell cwd when the caller omits one. A change to a different effective mode is rejected before its `sandbox/mode` event commits while that owner has an open PTY or a spawn in progress; the fence is attached to the exact owner and therefore outlives a local-provider reload that retains existing sessions. Wait for creation to settle and close the sessions before changing modes, so a terminal opened with wider access cannot survive a downgrade. +The plugin injects `pty`, `sandboxPolicy`, and `subprocess`, then registers the configured backend type (`shell`). `danger-full-access` starts the shell directly without requiring a sandbox provider; confined modes require a same-world `ctx.sandbox` and wrap the exact shell argv through it, failing before spawn when none is mounted. At spawn, one `ctx.sandboxPolicy.resolve({ session })` call supplies both the effective mode and the session workspace root; the same root is the default shell cwd when the caller omits one. A change to a different effective mode is rejected before its `sandbox/mode` event commits while that owner has an open PTY or a spawn in progress; the fence is attached to the exact owner and therefore outlives a provider reload that retains existing sessions. Wait for creation to settle and close the sessions before changing modes, so a terminal opened with wider access cannot survive a downgrade. -Linux readiness combines a foreground-verified private bash prompt marker, foreground-process-group syscall inspection, silence fallback, and absolute timeout. macOS uses the verified prompt marker plus silence/timeout because it has no `/proc` syscall surface. A marker is not ready until printable prompt text arrives, including when the OSC marker and `PS1` are split across data callbacks; when bash prints the marker before the kernel publishes its return to the foreground process group, polling retains the candidate for `handoffGraceMs` past the ordinary silence bound so a coincident handoff can win; that grace must cover at least one `pollIntervalMs` and is rejected at load otherwise. An interactive child that inherits `PROMPT_COMMAND` therefore cannot suppress inferred-idle readiness until the absolute timeout. Unrecognized or unreadable process state is never a positive exact-idle signal. A foreground group's stdin wait that already existed before a send is likewise not post-write readiness: the same group must be observed outside that wait before a later wait can settle the send, while a changed foreground group is new evidence. During unpublished startup, a fallback requires observed output; zero-output silence cannot publish an empty session, and timeout rejects the spawn. Cancellation closes the unpublished shell and rejects with the caller's exact abort reason even when its foreground process group is not observable yet; if that close fails, `PtyBackendCleanupError` separately preserves the cleanup failure for registry disposal. Incomplete terminal-control sequences are bounded by `maxReadBytes` and discarded through their terminator after crossing that limit; a trailing carriage return is carried across callbacks so split CRLF becomes one newline. +Readiness combines a foreground-verified private bash prompt marker, provider-reported foreground stdin-wait facts, silence fallback, and absolute timeout. A marker is not ready until the printable tail after the latest owned marker exactly equals the controlled `PS1`, including when the OSC marker and prompt are split across data callbacks; echoed input or output following an earlier prompt therefore cannot settle the current send. Prompt and silence evidence collected before the provider write, including while pre-write foreground inspection is pending, is discarded at the write boundary. When bash prints the marker before the terminal provider publishes its return to the foreground process group, polling retains the candidate for `handoffGraceMs` past the ordinary silence bound so a coincident handoff can win. An interactive child that inherits `PROMPT_COMMAND` therefore cannot suppress inferred-idle readiness until the absolute timeout. Unknown foreground state is never a positive exact-idle signal. A foreground group's stdin wait that existed before a send is likewise not post-write readiness: the same group must be observed outside that wait before a later wait can settle the send, while a changed foreground group is new evidence. During unpublished startup, a fallback requires observed output; zero-output silence cannot publish an empty session, and timeout rejects the spawn. Cancellation closes the unpublished shell and rejects with the caller's exact abort reason; `PtyBackendCleanupError` separately preserves a cleanup failure. The caller's signal is forwarded for terminal allocation and readiness initialization; after publication the handle owns its lifetime. Incomplete terminal-control sequences are bounded by `maxReadBytes` and discarded through their terminator after crossing that limit; malformed UTF-8 terminal output uses replacement characters, and a trailing carriage return is carried across callbacks so split CRLF becomes one newline. -Send cancellation resolves the current foreground process group and delivers a real `SIGINT`; it never emulates interruption by writing `\x03`, so raw-mode programs remain cancellable. Close sends `SIGTERM` to descendants, waits, then sends `SIGKILL` to the union of captured survivors and newly scanned descendants so reparenting cannot hide a process from teardown. It verifies that every retained identity is gone or, on Linux, a non-executing zombie before stopping the shell; zombie entries are quiescent and are reaped as the shell exits. A survivor failure does not cache a permanently rejected close; a later close retries the teardown. +Send cancellation marks queued input as canceled before asking the terminal handle to signal the current foreground process group with a real `SIGINT`; if asynchronous pre-write inspection later settles, it cannot execute that input. If a provider write is already in flight, signalling waits for it to settle; a rejected write sends no signal. The canceled send retains its slot until the write and foreground signalling settle, so a successor cannot receive either late bytes or that signal. A provider write or signal that never settles therefore retains the slot indefinitely; closing the session (`terminal_close`) is the recovery. The absolute deadline remains armed while cancellation waits. A signal failure is a terminal transport failure and rejects the active send. Cancellation never emulates interruption by writing `\x03`, so raw-mode programs remain cancellable. Close rejects new public signals, stops readiness polling, and awaits the handle's provider-owned complete-session termination before settling the active send as `session_exit`. ## Model Experience @@ -31,6 +31,6 @@ A standing-policy change appends an owner-rendered superseding runtime-context s ## Known Limitations and Deferred Work - Line-oriented output is normalized; full-screen alternate-buffer interaction is unsupported. -- Linux exact probes support x64 and arm64 UAPI tables; other architectures use prompt-marker and silence/timeout readiness. -- A descendant that daemonizes and reparents before teardown leaves the captured tree; cleanup never broadens to the launcher PID's POSIX session because that can include unrelated processes. +- Exact stdin-wait detection depends on the mounted subprocess provider; providers that cannot prove it use prompt-marker and silence/timeout readiness. +- Cleanup guarantees are those of `SubprocessTerminalHandle`; provider-specific gaps belong to that implementation's contract rather than this PTY consumer. - Sessions do not survive harness process exit. diff --git a/packages/pty/pty-local/README.zh.md b/packages/pty/pty-local/README.zh.md index 81987ea068..353c2a4bda 100644 --- a/packages/pty/pty-local/README.zh.md +++ b/packages/pty/pty-local/README.zh.md @@ -2,15 +2,15 @@ [English](README.md) | 中文 -这个本地 Linux／macOS `node-pty` 后端实现 `ctx.pty`；在其他平台加载时会以不支持为由失败。它在共享 `ctx.sandboxPolicy` 下启动交互式 shell，移除形似凭据的环境变量，保留有界的逐行输出，检测就绪状态，并清理以 `node-pty` 子进程为根的已捕获进程树。 +这是一个基于 `ctx.subprocess.spawnTerminal`、为 `ctx.pty` 提供的持久 shell 后端。它在共享 `ctx.sandboxPolicy` 下启动交互式 shell，保留有界的逐行输出并检测就绪状态；进程管理提供方则负责 PTY 分配、环境清理、前台进程组、信号发送和完整终端会话清理。因此，同一个 PTY 后端可以与本地或远程执行世界提供方组合。 ## 插件（`pty-local`） -该插件注入 `pty`、`sandbox` 和 `sandboxPolicy`，然后注册所配置的后端类型（`shell`）。`danger-full-access` 会直接启动 shell；受限模式则通过 `ctx.sandbox` 包装确切的 shell argv。Spawn 时，一次 `ctx.sandboxPolicy.resolve({ session })` 调用会同时给出实际模式与会话工作区根目录；调用方省略 cwd 时，同一根目录也是 shell 的默认 cwd。当某个所有者存在开放的 PTY 或正在进行 spawn 时，如果配置变更会得到不同的实际模式，系统会在对应 `sandbox/mode` 事件提交前拒绝该变更。该限制绑定到确切所有者，因此即使本地提供方重新加载并保留现有会话，它仍然有效。更改模式前，请等待创建完成并关闭会话，避免以更宽权限打开的终端在权限降级后继续存在。 +该插件注入 `pty`、`sandboxPolicy` 和 `subprocess`，然后注册所配置的后端类型（`shell`）。`danger-full-access` 无需沙箱提供方即可直接启动 shell；受限模式要求同一执行世界中存在 `ctx.sandbox`，并通过它包装确切的 shell argv，未挂载时会在 spawn 前失败。Spawn 时，一次 `ctx.sandboxPolicy.resolve({ session })` 调用会同时给出实际模式与会话工作区根目录；调用方省略 cwd 时，同一根目录也是 shell 的默认 cwd。当某个所有者存在开放的 PTY 或正在进行 spawn 时，如果配置变更会得到不同的实际模式，系统会在对应 `sandbox/mode` 事件提交前拒绝该变更。该限制绑定到确切所有者，因此即使提供方重新加载并保留现有会话，它仍然有效。更改模式前，请等待创建完成并关闭会话，避免以更宽权限打开的终端在权限降级后继续存在。 -Linux 的就绪检测结合以下机制：由前台状态验证的私有 bash 提示符标记、前台进程组 syscall 检查、静默回退和绝对超时。macOS 没有 `/proc` syscall 接口，因此使用经过验证的提示符标记以及静默／超时。当可打印的提示符文本尚未到达时，即使 OSC 标记和 `PS1` 被拆到多个数据回调中，系统也不会把标记视为就绪。如果 bash 在内核发布其重新取得前台进程组的状态前打印标记，轮询会在普通静默上限之后再保留该候选状态 `handoffGraceMs`，使恰好同时发生的前台交接有机会胜出；该宽限至少要覆盖一个 `pollIntervalMs`，否则加载时即被拒绝。因此，继承 `PROMPT_COMMAND` 的交互式子进程无法一直抑制推断空闲就绪直至绝对超时。无法识别或读取的进程状态绝不会作为精确空闲的正向信号。同样，一次 send 之前就已存在的前台进程组 stdin 等待并不代表写入后就绪：必须先观察到同一进程组脱离该等待，之后再次进入等待才能使该次 send 完成；前台进程组发生变化则构成新的证据。尚未发布的启动过程中，回退路径要求已经观察到输出；零输出静默不能发布空会话，超时则拒绝 spawn。取消操作会关闭尚未发布的 shell，并以调用方提供的确切中止原因拒绝，即使当时还无法观察其前台进程组。如果关闭失败，`PtyBackendCleanupError` 会单独保留清理失败，供注册表 dispose（资源释放）时处理。未完成的终端控制序列受 `maxReadBytes` 限制；超过上限后，系统会丢弃内容直到其终止符。末尾的回车会跨回调保留，使拆分的 CRLF 合并为一个换行。 +就绪检测结合以下机制：由前台状态验证的私有 bash 提示符标记、提供方报告的前台 stdin 等待事实、静默回退和绝对超时。只有最新自有标记之后的可打印尾部与受控 `PS1` 完全相等，标记才算就绪；即使 OSC 标记和提示符被拆到多个数据回调中也一样。因此，较早提示符之后的回显输入或输出无法使当前 send 完成。提供方写入前收集的提示符与静默证据，包括写入前前台检查仍在等待时收集的证据，都会在写入边界丢弃。如果 bash 在终端提供方发布其重新取得前台进程组的状态前打印标记，轮询会在普通静默上限之后再保留该候选状态 `handoffGraceMs`，使恰好同时发生的前台交接有机会胜出。因此，继承 `PROMPT_COMMAND` 的交互式子进程无法一直抑制推断空闲就绪直至绝对超时。未知的前台状态绝不会作为精确空闲的正向信号。同样，一次 send 之前就已存在的前台进程组 stdin 等待并不代表写入后就绪：必须先观察到同一进程组脱离该等待，之后再次进入等待才能使该次 send 完成；前台进程组发生变化则构成新的证据。尚未发布的启动过程中，回退路径要求已经观察到输出；零输出静默不能发布空会话，超时则拒绝 spawn。取消操作会关闭尚未发布的 shell，并以调用方提供的确切中止原因拒绝；`PtyBackendCleanupError` 会单独保留清理失败。调用方的 signal 会转发给终端分配与就绪初始化；发布后，句柄负责其生命周期。未完成的终端控制序列受 `maxReadBytes` 限制；超过上限后，系统会丢弃内容直到其终止符。格式错误的 UTF-8 终端输出使用替换字符；末尾的回车会跨回调保留，使拆分的 CRLF 合并为一个换行。 -取消发送时，系统会解析当前前台进程组并发送真正的 `SIGINT`；它绝不会通过写入 `\x03` 模拟中断，因此，即使程序运行在 raw 模式下，也仍可取消。关闭操作先向后代发送 `SIGTERM` 并等待，再向已捕获的存活进程与新扫描到的后代之并集发送 `SIGKILL`，防止进程通过重新设定父进程而逃避清理。在停止 shell 前，系统会确认每个保留的进程身份都已消失，或者在 Linux 上已成为不再执行的僵尸进程；僵尸进程条目视为完全停稳，并会随 shell 退出而回收。如果仍有进程存活，失败结果不会缓存成永久拒绝的关闭操作；后续关闭仍会重试清理。 +取消发送时，系统会先把排队输入标记为已取消，再要求终端句柄向当前前台进程组发送真正的 `SIGINT`；异步写入前检查即使随后结算，也无法执行该输入。如果提供方写入已在途，信号发送会等待其结算；写入被拒绝时不会发送信号。已取消的 send 会保留其位置，直到写入与前台信号发送都结算，因此后继 send 不会收到延迟字节或该信号。因此，永不结算的提供方写入或信号会无限期保留该位置；恢复手段是关闭会话（`terminal_close`）。取消等待期间，绝对 deadline 仍保持启用。信号发送失败是终端传输失败，会拒绝活跃 send。取消绝不会通过写入 `\x03` 模拟中断，因此，即使程序运行在 raw 模式下，也仍可取消。关闭操作会拒绝新的公开信号、停止就绪轮询，并等待由句柄提供方负责的完整会话终止，然后才把活跃 send 结算为 `session_exit`。 ## 模型体验 @@ -31,6 +31,6 @@ Linux 的就绪检测结合以下机制：由前台状态验证的私有 bash ## 已知限制与暂缓事项 - 输出按行规范化；不支持全屏备用缓冲区交互。 -- Linux 精确探针支持 x64 与 arm64 UAPI 表；其他架构使用提示符标记和静默／超时就绪机制。 -- 如果后代进程在清理前守护化并重新设定父进程，它会脱离已捕获的进程树；清理绝不会扩大到启动器 PID 所属的整个 POSIX 会话，因为其中可能包含无关进程。 +- 精确 stdin 等待检测取决于已挂载的进程管理提供方；无法证明该状态的提供方使用提示符标记和静默／超时就绪机制。 +- 清理保证以 `SubprocessTerminalHandle` 的保证为准；提供方特定的缺口属于该实现的契约，而非这个 PTY 消费方。 - harness 进程退出后，会话无法继续存在。 diff --git a/packages/pty/pty-local/package.json b/packages/pty/pty-local/package.json index a859f482fa..94f98c02d5 100644 --- a/packages/pty/pty-local/package.json +++ b/packages/pty/pty-local/package.json @@ -1,6 +1,6 @@ { "name": "@deepseek-ai/dsh-pty-local", - "description": "Local node-pty backend for persistent DeepSeek Harness PTY sessions", + "description": "Persistent shell PTY backend over the DeepSeek Harness subprocess terminal primitive", "version": "0.0.1", "private": true, "type": "module", @@ -21,12 +21,8 @@ "files": [ "lib/index.js", "lib/invariant.js", - "scripts/ensure-spawn-helper.mjs", "lib/types/**/*.d.ts" ], - "scripts": { - "postinstall": "node scripts/ensure-spawn-helper.mjs" - }, "license": "BSD-3-Clause", "peerDependencies": { "@deepseek-ai/dsh-agent": "^0.0.1", @@ -39,7 +35,6 @@ "cordis": "^4.0.0-rc.7" }, "dependencies": { - "node-pty": "^1.1.0", "schemastery": "^3.18.0" }, "devDependencies": { @@ -50,6 +45,7 @@ "@deepseek-ai/dsh-sandbox-policy": "workspace:^", "@deepseek-ai/dsh-session": "workspace:^", "@deepseek-ai/dsh-subprocess": "workspace:^", + "@deepseek-ai/dsh-subprocess-local": "workspace:^", "cordis": "^4.0.0-rc.7" } } diff --git a/packages/pty/pty-local/src/index.ts b/packages/pty/pty-local/src/index.ts index e59d53446a..fa2237c959 100644 --- a/packages/pty/pty-local/src/index.ts +++ b/packages/pty/pty-local/src/index.ts @@ -1,31 +1,28 @@ /** - * Local persistent PTY backend using public `node-pty` APIs, shared sandbox - * policy, bounded output, platform readiness probes, and process-session cleanup. + * Persistent shell PTY backend over the subprocess terminal primitive, shared + * sandbox policy, bounded output, and provider-owned session cleanup. * @module @deepseek-ai/dsh-pty-local */ import { Context } from 'cordis' -import * as nodePty from 'node-pty' -import type { IPtyForkOptions } from 'node-pty' import type { Agent } from '@deepseek-ai/dsh-agent' import type { Session, SessionEvent } from '@deepseek-ai/dsh-session' import { PtyBackendCleanupError } from '@deepseek-ai/dsh-pty' -import { scrubbedParentEnv } from '@deepseek-ai/dsh-subprocess' import type { PtyBackend, PtyBackendSpawnSpec } from '@deepseek-ai/dsh-pty' +import type { SubprocessTerminalHandle, SubprocessTerminalSpawnSpec } from '@deepseek-ai/dsh-subprocess' import type { SandboxExecutionPolicy } from '@deepseek-ai/dsh-sandbox' import { effectiveSandboxMode } from '@deepseek-ai/dsh-sandbox-policy' import { type Config, type ResolvedConfig, validateConfig } from './config.ts' -import { createProcessInspector } from './process-inspector.ts' -import type { ProcessInspector } from './process-inspector.ts' import { LocalPtySession } from './session.ts' +import { CONTROLLED_PROMPT } from './sanitize.ts' export { Config } from './config.ts' export type { Config as PtyLocalConfig } from './config.ts' /** Cordis plugin name. */ export const name = 'pty-local' -/** Required services: PTY registry plus the one shared confinement policy. */ -export const inject = ['pty', 'sandbox', 'sandboxPolicy'] +/** Required services: PTY registry, shared confinement policy, and process substrate. */ +export const inject = ['pty', 'sandboxPolicy', 'subprocess'] interface SandboxModeFenceState { pty: Context['pty'] @@ -55,14 +52,14 @@ function ensureSandboxModeFence(ctx: Context, owner: Agent): void { }, { global: true }) } -function childEnvironment(spec: PtyBackendSpawnSpec): NodeJS.ProcessEnv { - // node-pty owns the spawn; the base env shares the subprocess seam's scrub. +function childEnvironment(spec: PtyBackendSpawnSpec): Record { + // The subprocess provider supplies its own scrubbed ambient base; these are + // deliberate terminal-specific overrides layered after it. return { - ...scrubbedParentEnv(), TERM: 'dumb', PAGER: 'cat', GIT_PAGER: 'cat', - PS1: 'dsh> ', + PS1: CONTROLLED_PROMPT, PROMPT_COMMAND: 'printf "\\033]133;D;%s\\007" "$?"', BASH_SILENCE_DEPRECATION_WARNING: '1', DSH_SHELL: '1', @@ -74,8 +71,31 @@ function childEnvironment(spec: PtyBackendSpawnSpec): NodeJS.ProcessEnv { function spawnArgv(ctx: Context, config: ResolvedConfig, policy: SandboxExecutionPolicy): string[] { const argv = [config.shellPath, ...config.shellArgs] if (policy.mode === 'danger-full-access') return argv + const sandbox = ctx.get('sandbox') + if (sandbox === undefined) { + throw new Error(`pty-local: sandbox mode "${policy.mode}" requires a ctx.sandbox provider in the execution world`) + } // Re-state the discriminant because object spread does not preserve its narrowed type. - return ctx.sandbox.confine(argv, { ...policy, mode: policy.mode }).argv + return sandbox.confine(argv, { ...policy, mode: policy.mode }).argv +} + +// TODO(pty-initialize-race-home): Fold this outer abort race into +// LocalPtySession.initialize when the send-state consolidation lands; the +// session already owns the send lifecycle the race protects. +async function initializeSession(session: LocalPtySession, signal?: AbortSignal): Promise { + if (signal === undefined) { + await session.initialize(signal) + return + } + const aborted = Promise.withResolvers() + const onAbort = (): void => { aborted.reject(signal.reason) } + signal.addEventListener('abort', onAbort, { once: true }) + try { + signal.throwIfAborted() + await Promise.race([session.initialize(signal), aborted.promise]) + } finally { + signal.removeEventListener('abort', onAbort) + } } /** Local shell backend registered under the configured type. */ @@ -85,13 +105,13 @@ export class LocalPtyBackend implements PtyBackend { constructor( private readonly ctx: Context, private readonly config: ResolvedConfig, - private readonly inspector: ProcessInspector, - private readonly spawnTerminal: typeof nodePty.spawn = nodePty.spawn, + private readonly spawnTerminal: ( + spec: SubprocessTerminalSpawnSpec, + ) => Promise = spec => ctx.subprocess.spawnTerminal(spec), private readonly createSession: ( - terminal: ReturnType, - inspector: ProcessInspector, + terminal: SubprocessTerminalHandle, config: ResolvedConfig, - ) => LocalPtySession = (terminal, inspector, config) => new LocalPtySession(terminal, inspector, config), + ) => LocalPtySession = (terminal, config) => new LocalPtySession(terminal, config), ) { this.type = config.backendType } @@ -101,19 +121,19 @@ export class LocalPtyBackend implements PtyBackend { ensureSandboxModeFence(this.ctx, spec.owner) const policy = this.ctx.sandboxPolicy.resolve({ session: spec.owner.session }) const argv = spawnArgv(this.ctx, this.config, policy) - const file = argv[0] - if (file === undefined) throw new Error('pty-local: sandbox returned empty argv') - const options: IPtyForkOptions = { - name: 'dumb', - cols: this.config.cols, - rows: this.config.rows, + if (argv[0] === undefined) throw new Error('pty-local: sandbox returned empty argv') + const terminal = await this.spawnTerminal({ + argv, cwd: spec.cwd ?? policy.workspaceRoot, env: childEnvironment(spec), - } - const terminal = this.spawnTerminal(file, argv.slice(1), options) - const session = this.createSession(terminal, this.inspector, this.config) + rows: this.config.rows, + cols: this.config.cols, + graceMs: this.config.disposeGraceMs, + signal: spec.signal, + }) + const session = this.createSession(terminal, this.config) try { - await session.initialize(spec.signal) + await initializeSession(session, spec.signal) return session } catch (error) { try { @@ -129,6 +149,5 @@ export class LocalPtyBackend implements PtyBackend { /** Register the local PTY backend. */ export function apply(ctx: Context, config: Config): void { validateConfig(config) - const inspector = createProcessInspector() - ctx.pty.registerBackend(new LocalPtyBackend(ctx, config, inspector)) + ctx.pty.registerBackend(new LocalPtyBackend(ctx, config)) } diff --git a/packages/pty/pty-local/src/sanitize.ts b/packages/pty/pty-local/src/sanitize.ts index cc22c29c01..1f28315fb1 100644 --- a/packages/pty/pty-local/src/sanitize.ts +++ b/packages/pty/pty-local/src/sanitize.ts @@ -5,12 +5,15 @@ import { Buffer } from 'node:buffer' /** OSC marker emitted by the controlled bash before each prompt. */ export const PROMPT_MARKER_PREFIX = '133;D;' +/** Exact printable prompt emitted after the private marker. */ +export const CONTROLLED_PROMPT = 'dsh> ' + /** One sanitized chunk plus whether it contained the owned prompt marker. */ export interface SanitizedChunk { text: string prompt: boolean - /** Present when printable text followed the latest owned prompt marker. */ - promptText?: true + /** Printable text after the latest owned marker in this chunk. */ + promptTail?: string } /** @@ -23,7 +26,7 @@ export class TerminalSanitizer { private discardMode: 'osc' | 'csi' | undefined private discardOscEscape = false private trailingCarriageReturn = false - private awaitingPromptText = false + private trackingPromptTail = false constructor(private readonly maxPendingBytes: number) {} @@ -36,24 +39,21 @@ export class TerminalSanitizer { this.pending += this.discardPrefix(chunk) let text = '' let prompt = false - let promptText = false + let includePromptTail = this.trackingPromptTail + let promptTail = '' let index = 0 - const appendText = (value: string): boolean => { + const appendText = (value: string): void => { text += value - if (this.awaitingPromptText && value.replace(/[\r\n\x07]/g, '').length > 0) { - this.awaitingPromptText = false - return true - } - return false + if (this.trackingPromptTail) promptTail += value } while (index < this.pending.length) { const escape = this.pending.indexOf('\x1b', index) if (escape < 0) { - promptText = appendText(this.pending.slice(index)) || promptText + appendText(this.pending.slice(index)) index = this.pending.length break } - promptText = appendText(this.pending.slice(index, escape)) || promptText + appendText(this.pending.slice(index, escape)) if (escape + 1 >= this.pending.length) { index = escape break @@ -74,8 +74,9 @@ export class TerminalSanitizer { const content = this.pending.slice(escape + 2, end - terminatorBytes) if (content.startsWith(PROMPT_MARKER_PREFIX)) { prompt = true - promptText = false - this.awaitingPromptText = true + this.trackingPromptTail = true + includePromptTail = true + promptTail = '' } index = end continue @@ -99,7 +100,11 @@ export class TerminalSanitizer { } this.pending = this.pending.slice(index) this.enforcePendingBound() - return { text: this.normalizeText(text), prompt, ...promptText ? { promptText: true } : {} } + return { + text: this.normalizeText(text), + prompt, + ...includePromptTail ? { promptTail } : {}, + } } /** @@ -111,7 +116,7 @@ export class TerminalSanitizer { this.pending = '' this.discardMode = undefined this.discardOscEscape = false - this.awaitingPromptText = false + this.trackingPromptTail = false const normalized = this.normalizeText(text) if (!this.trailingCarriageReturn) return normalized this.trailingCarriageReturn = false diff --git a/packages/pty/pty-local/src/session.ts b/packages/pty/pty-local/src/session.ts index 46f3be2ca8..25ed8fe586 100644 --- a/packages/pty/pty-local/src/session.ts +++ b/packages/pty/pty-local/src/session.ts @@ -1,8 +1,12 @@ -/** Local `node-pty` session: bounded output, readiness, signals, and teardown. */ +/** Persistent PTY session over the subprocess seam's terminal primitive. */ -import { constants } from 'node:os' import { Buffer } from 'node:buffer' -import type { IDisposable, IPty } from 'node-pty' +import type { + SubprocessOutcome, + SubprocessTerminalForeground, + SubprocessTerminalHandle, +} from '@deepseek-ai/dsh-subprocess' +import { PtyError } from '@deepseek-ai/dsh-pty' import type { PtyBackendSession, PtyReadRequest, @@ -17,12 +21,7 @@ import type { PtyWaitReason, } from '@deepseek-ai/dsh-pty' import type { ResolvedConfig } from './config.ts' -import type { ProcessIdentity, ProcessInspector } from './process-inspector.ts' -import { TerminalSanitizer } from './sanitize.ts' - -function delay(ms: number): Promise { - return new Promise(resolve => setTimeout(resolve, ms)) -} +import { CONTROLLED_PROMPT, TerminalSanitizer } from './sanitize.ts' function utf8Tail(text: string, maxBytes: number): { text: string; truncated: boolean } { if (Buffer.byteLength(text) <= maxBytes) return { text, truncated: false } @@ -79,24 +78,32 @@ class LocalSendOperation implements PtySendOperation { private readonly output: BoundedTextBuffer private readonly promise: PromiseWithResolvers private finished = false + private cancellationRequested = false private initialForegroundLeftWait: boolean + private initialForegroundPgid: number | undefined constructor( maxBytes: number, readonly startedAt: number, - private readonly initialForegroundPgid: number | undefined, - initialForegroundWasWaiting: boolean, private readonly onCancel: () => void, ) { this.output = new BoundedTextBuffer(maxBytes) this.promise = Promise.withResolvers() - this.initialForegroundLeftWait = !initialForegroundWasWaiting + this.initialForegroundLeftWait = true } get done(): Promise { return this.promise.promise } + get settled(): boolean { + return this.finished + } + + get cancelRequested(): boolean { + return this.cancellationRequested + } + append(text: string): void { if (!this.finished) this.output.append(text) } @@ -123,6 +130,11 @@ class LocalSendOperation implements PtySendOperation { return this.output.consume() } + setInitialForeground(foreground: SubprocessTerminalForeground | undefined): void { + this.initialForegroundPgid = foreground?.processGroupId + this.initialForegroundLeftWait = foreground?.inputWaiting !== true + } + acceptsStdinWait(pgid: number, waiting: boolean): boolean { // The same group may still expose the wait that existed before terminal.write. // Observe every poll so a departure before the exact-settlement threshold @@ -134,56 +146,59 @@ class LocalSendOperation implements PtySendOperation { cancel(): boolean { if (this.finished) return false + this.cancellationRequested = true this.onCancel() return true } } -function signalName(number: number | undefined): NodeJS.Signals | null { - if (number === undefined || number === 0) return null - for (const [name, value] of Object.entries(constants.signals)) { - if (value === number) return name as NodeJS.Signals - } - return null -} - -/** Backend session wrapping one `node-pty` process and its captured process tree. */ +/** Backend session wrapping one provider-owned terminal process. */ export class LocalPtySession implements PtyBackendSession { motd = '' readonly pid: number + private readonly decoder = new TextDecoder() private readonly sanitizer: TerminalSanitizer private readonly scrollback: BoundedTextBuffer - private readonly exitPromise: PromiseWithResolvers = Promise.withResolvers() - private readonly dataDisposable: IDisposable - private readonly exitDisposable: IDisposable + private readonly outputEnded = Promise.withResolvers() + private readonly completion: Promise private statusValue: PtySessionStatus = { kind: 'running' } + // TODO(pty-send-state-consolidation): Fold the per-send fields below + // (active/activeTimer/activeDeadlineTimer/activeAbort/interrupting/ + // activeWrite/pollingReady/polling) into one send-lifecycle owner; the + // cancellation/readiness interplay now has enough pinned tests to carry + // that refactor safely. private active: LocalSendOperation | undefined private activeTimer: NodeJS.Timeout | undefined + private activeDeadlineTimer: NodeJS.Timeout | undefined private activeAbort: (() => void) | undefined + private interrupting: LocalSendOperation | undefined + private activeWrite: Promise | undefined + private pollingReady: LocalSendOperation | undefined + private polling = false private promptSeen = false private promptTextSeen = false + private promptTail = '' private shellPgid: number | undefined private initializing = false private lastOutputAt = Date.now() private closing = false private closePromise: Promise | undefined + private transportFailure: Error | undefined constructor( - private readonly terminal: IPty, - private readonly inspector: ProcessInspector, + private readonly terminal: SubprocessTerminalHandle, private readonly config: ResolvedConfig, ) { this.pid = terminal.pid this.sanitizer = new TerminalSanitizer(config.maxReadBytes) this.scrollback = new BoundedTextBuffer(config.scrollbackMaxBytes, config.scrollbackLines) - this.dataDisposable = terminal.onData((data) => { this.onData(data) }) - this.exitDisposable = terminal.onExit(({ exitCode, signal }) => { - const tail = this.sanitizer.flush() - this.appendOutput(tail) - this.statusValue = { kind: 'exited', exitCode, signal: signalName(signal) } - this.settleActive('session_exit') - this.exitPromise.resolve() - }) + terminal.output.on('data', this.onTerminalData) + terminal.output.once('end', this.onTerminalEnd) + terminal.output.once('error', this.onTerminalError) + this.completion = terminal.done.then( + outcome => this.onExit(outcome), + (error: unknown) => { this.onTransportFailure(error) }, + ) } /** @@ -210,43 +225,95 @@ export class LocalPtySession implements PtyBackendSession { startSend(request: PtySendRequest): PtySendOperation { if (this.closing) throw new Error('PTY session is closing') if (this.statusValue.kind === 'exited') throw new Error('PTY session has exited') - if (this.active !== undefined) throw new Error('PTY session already has an active send') + if (this.active !== undefined) { + const draining = this.activeWrite !== undefined + ? ' or draining provider write' + : this.interrupting !== undefined + ? ' or draining foreground interrupt' + : '' + throw new PtyError(`PTY session already has an active send${draining}`, 'SEND_ACTIVE') + } if (request.signal?.aborted === true) throw new Error('PTY send aborted before write') - const initialForegroundPgid = this.inspector.foregroundPgid(this.pid) - const initialForegroundWasWaiting = initialForegroundPgid !== undefined - && this.inspector.isStdinWaiting(initialForegroundPgid) const operation = new LocalSendOperation( this.config.maxReadBytes, Date.now(), - initialForegroundPgid, - initialForegroundWasWaiting, () => { this.interrupt(operation) }, ) this.active = operation - this.lastOutputAt = Date.now() - this.promptSeen = false - this.promptTextSeen = false + this.resetReadinessEvidence() if (request.signal !== undefined) { const onAbort = (): void => { operation.cancel() } request.signal.addEventListener('abort', onAbort, { once: true }) this.activeAbort = () => request.signal?.removeEventListener('abort', onAbort) } - - try { - if (request.text.length > 0) this.terminal.write(request.text) - if (request.submit) this.terminal.write('\r') - } catch (error: unknown) { - this.clearActive() - operation.fail(error) - return operation - } - - this.activeTimer = setInterval(() => { this.pollReadiness(operation) }, this.config.pollIntervalMs) + this.activeDeadlineTimer = setTimeout(() => { + if (this.active === operation) { + this.settleActive('timeout', this.activeWrite !== undefined || this.interrupting === operation) + } + }, this.config.timeoutMs) + void this.beginSend(operation, request) return operation } + private async beginSend(operation: LocalSendOperation, request: PtySendRequest): Promise { + let foreground: SubprocessTerminalForeground | undefined + try { + foreground = await this.terminal.inspectForeground() + } catch (error: unknown) { + // A pre-write inspection failure while cancellation owns the slot must not + // release it: interruptOnce's in-flight foreground signal could land on a + // successor's foreground group. The interrupt path's post-signal tail + // resumes polling, whose guarded catch propagates a persistent failure. + // A retained settled operation implies that same in-flight interrupt, so + // this guard admits only an unsettled active send. + if (this.active === operation && !this.closing && this.interrupting !== operation) { + this.failActive(error) + } + return + } + try { + if (this.active !== operation || this.closing || this.interrupting === operation) return + operation.setInitialForeground(foreground) + const input = `${request.text}${request.submit ? '\r' : ''}` + if (input.length > 0 && !operation.cancelRequested) { + this.resetReadinessEvidence() + const write = this.terminal.write(input) + this.activeWrite = write.then(() => true, () => false) + try { + await write + } finally { + this.activeWrite = undefined + } + } + // Cancellation owns post-write signalling and reservation release. + if (operation.cancelRequested) return + if (this.active === operation && operation.settled) { + this.clearActive() + return + } + // Closing can race the awaited provider write even though static analysis sees only local assignments. + // oxlint-disable-next-line typescript/no-unnecessary-condition -- awaited provider writes can close the session. + if (this.active === operation && !this.closing) { + this.pollingReady = operation + this.schedulePoll(operation) + } + } catch (error: unknown) { + if (this.active === operation && !this.closing) { + if (operation.settled) this.clearActive() + else this.failActive(error) + } + } + } + + private resetReadinessEvidence(): void { + this.lastOutputAt = Date.now() + this.promptSeen = false + this.promptTextSeen = false + this.promptTail = '' + } + read(request: PtyReadRequest): PtyReadResult { const snapshot = this.scrollback.snapshot() const lines = snapshot.text.split('\n') @@ -272,16 +339,10 @@ export class LocalPtySession implements PtyBackendSession { } } - signal(signal: PtySignal): Promise { - return Promise.resolve().then(() => { - const pgid = this.inspector.foregroundPgid(this.pid) - if (pgid === undefined) throw new Error(`cannot resolve foreground process group for PTY ${this.pid}`) - if (signal === 'SIGKILL' && pgid === this.pid) { - throw new Error('refusing to SIGKILL the PTY shell; use terminal_close') - } - this.inspector.signalGroup(pgid, signal) - return { delivered: true, targetPgid: pgid } - }) + async signal(signal: PtySignal): Promise { + if (this.closing) throw new Error('PTY session is closing') + const targetPgid = await this.terminal.signalForeground(signal) + return { delivered: true, targetPgid } } status(): PtySessionStatus { @@ -300,21 +361,56 @@ export class LocalPtySession implements PtyBackendSession { return closing } + private readonly onTerminalData = (chunk: Buffer | Uint8Array | string): void => { + const bytes = typeof chunk === 'string' ? Buffer.from(chunk, 'utf8') : chunk + this.onData(this.decoder.decode(bytes, { stream: true })) + } + + private readonly onTerminalEnd = (): void => { + this.onData(this.decoder.decode()) + this.appendOutput(this.sanitizer.flush()) + this.outputEnded.resolve() + } + + private readonly onTerminalError = (error: Error): void => { + this.onTransportFailure(error) + this.outputEnded.resolve() + } + private onData(data: string): void { const sanitized = this.sanitizer.push(data) this.appendOutput(sanitized.text) if (sanitized.prompt) { - const foregroundPgid = this.inspector.foregroundPgid(this.pid) - if (this.shellPgid === undefined) this.shellPgid = foregroundPgid + // TODO(pty-delayed-signal-prompt): With a reproducer, define a marker-generation boundary + // before attributing a signal-delayed prompt to a later send. // Bash can print PROMPT_COMMAND before the kernel publishes its return // to the foreground process group. Retain the marker; polling below is // the authority that accepts it only after bash owns the foreground. this.promptSeen = true - this.promptTextSeen = sanitized.promptText === true + this.promptTail = '' this.lastOutputAt = Date.now() - } else if (this.promptSeen && sanitized.promptText === true) { - this.promptTextSeen = true } + if (this.promptSeen && sanitized.promptTail !== undefined) { + const remaining = Math.max(0, CONTROLLED_PROMPT.length + 1 - this.promptTail.length) + this.promptTail += sanitized.promptTail.slice(0, remaining) + if (sanitized.promptTail.length > remaining) this.promptTail = `${CONTROLLED_PROMPT}\0` + this.promptTextSeen = this.promptTail === CONTROLLED_PROMPT + } + } + + private async onExit(outcome: SubprocessOutcome): Promise { + await this.outputEnded.promise + if (this.transportFailure !== undefined) return + this.statusValue = { kind: 'exited', exitCode: outcome.exitCode, signal: outcome.signal } + this.settleActive('session_exit') + } + + private onTransportFailure(error: unknown): void { + const failure = error instanceof Error ? error : new Error(String(error)) + this.transportFailure ??= failure + this.statusValue = { kind: 'exited', exitCode: null, signal: null } + this.failActive(failure) + void this.terminal.terminate().catch(() => {}) } private appendOutput(text: string): void { @@ -324,63 +420,94 @@ export class LocalPtySession implements PtyBackendSession { this.active?.append(text) } - private pollReadiness(operation: LocalSendOperation): void { - if (this.active !== operation) return - if (this.statusValue.kind === 'exited') { - this.settleActive('session_exit') - return - } - if (this.promptSeen && this.promptTextSeen && Date.now() - this.lastOutputAt >= this.config.pollIntervalMs) { - const pgid = this.inspector.foregroundPgid(this.pid) - if (this.shellPgid !== undefined && pgid === this.shellPgid) { + private schedulePoll(operation: LocalSendOperation, delayMs = this.config.pollIntervalMs): void { + if (this.active !== operation || this.interrupting === operation || this.polling) return + if (this.activeTimer !== undefined) clearTimeout(this.activeTimer) + this.activeTimer = setTimeout(() => { + this.activeTimer = undefined + void this.pollReadiness(operation) + }, delayMs) + } + + private async pollReadiness(operation: LocalSendOperation): Promise { + if (this.active !== operation || this.polling) return + this.polling = true + try { + if (this.statusValue.kind === 'exited') { + this.settleActive('session_exit') + return + } + const foreground = await this.terminal.inspectForeground() + if (this.active !== operation || this.closing || this.interrupting === operation) return + const idleFor = Date.now() - this.lastOutputAt + if (this.promptSeen && foreground !== undefined && this.shellPgid === undefined) { + this.shellPgid = foreground.processGroupId + } + if (this.promptSeen && this.promptTextSeen && idleFor >= this.config.pollIntervalMs + && foreground?.processGroupId === this.shellPgid) { this.settleActive('stdin_read') return } + const elapsed = Date.now() - operation.startedAt + const startupHasOutput = !this.initializing || this.scrollback.snapshot().text.length > 0 + const acceptsStdinWait = startupHasOutput && foreground !== undefined + && operation.acceptsStdinWait(foreground.processGroupId, foreground.inputWaiting) + if (elapsed >= this.config.exactProbeAfterMs && acceptsStdinWait) { + this.settleActive('stdin_read') + return + } + // A prompt candidate can race bash's foreground handoff, but an interactive + // child also inherits PROMPT_COMMAND. Silence therefore remains the bound + // on waiting for shell ownership instead of letting a child marker suppress + // readiness until the absolute timeout. + const handoffGrace = this.promptSeen ? this.config.handoffGraceMs : 0 + if (startupHasOutput && idleFor >= this.config.idleSilenceMs + handoffGrace) { + this.settleActive('inferred_idle') + } + } catch (error: unknown) { + if (this.active === operation && !this.closing && this.interrupting !== operation) this.failActive(error) + } finally { + this.polling = false + const active = this.active + // Awaited provider inspection can clear or replace the active send despite static analysis. + // oxlint-disable-next-line typescript/no-unnecessary-condition -- awaited inspection can replace the active send. + if (active !== undefined && this.pollingReady === active) this.schedulePoll(active) } - const elapsed = Date.now() - operation.startedAt - const startupHasOutput = !this.initializing || this.scrollback.snapshot().text.length > 0 - let acceptsStdinWait = false - if (startupHasOutput) { - const pgid = this.inspector.foregroundPgid(this.pid) - acceptsStdinWait = pgid !== undefined - && operation.acceptsStdinWait(pgid, this.inspector.isStdinWaiting(pgid)) - } - if (elapsed >= this.config.exactProbeAfterMs && acceptsStdinWait) { - this.settleActive('stdin_read') - return - } - // A prompt candidate can race bash's foreground handoff, but an interactive - // child also inherits PROMPT_COMMAND. Silence therefore remains the bound - // on waiting for shell ownership instead of letting a child marker suppress - // readiness until the absolute timeout. When a prompt marker was seen, the - // configured grace holds the fallback past the silence bound so polls in - // that window can observe the foreground handoff and settle as stdin_read. - const idleFor = Date.now() - this.lastOutputAt - const handoffGrace = this.promptSeen ? this.config.handoffGraceMs : 0 - if (startupHasOutput && idleFor >= this.config.idleSilenceMs + handoffGrace) { - this.settleActive('inferred_idle') - return - } - if (elapsed >= this.config.timeoutMs) this.settleActive('timeout') } - private settleActive(waitReason: PtyWaitReason): void { + private settleActive(waitReason: PtyWaitReason, retainOwnership = false): void { const operation = this.active if (operation === undefined) return const scrollbackTruncated = this.scrollback.snapshot().truncated - this.clearActive() + if (retainOwnership) { + this.stopPolling() + this.activeAbort?.() + this.activeAbort = undefined + } else { + this.clearActive() + } operation.settle(waitReason, this.statusValue, scrollbackTruncated) } private stopPolling(): void { - if (this.activeTimer !== undefined) clearInterval(this.activeTimer) + this.stopReadinessPolling() + if (this.activeDeadlineTimer !== undefined) clearTimeout(this.activeDeadlineTimer) + this.activeDeadlineTimer = undefined + } + + private stopReadinessPolling(): void { + if (this.activeTimer !== undefined) clearTimeout(this.activeTimer) this.activeTimer = undefined + this.pollingReady = undefined } private clearActive(): void { + const operation = this.active this.stopPolling() this.activeAbort?.() this.activeAbort = undefined + if (this.interrupting === operation) this.interrupting = undefined + this.pollingReady = undefined this.active = undefined } @@ -393,104 +520,46 @@ export class LocalPtySession implements PtyBackendSession { private interrupt(operation: LocalSendOperation): void { if (this.active !== operation) return + this.interrupting = operation + this.stopReadinessPolling() + void this.interruptOnce(operation) + } + + private async interruptOnce(operation: LocalSendOperation): Promise { try { - const pgid = this.inspector.foregroundPgid(this.pid) - if (pgid === undefined) throw new Error(`cannot resolve foreground process group for PTY ${this.pid}`) - this.inspector.signalGroup(pgid, 'SIGINT') + const activeWrite = this.activeWrite + if (activeWrite !== undefined && !await activeWrite) return + await this.terminal.signalForeground('SIGINT') } catch (error: unknown) { - this.failActive(error) + if (this.active === operation && !this.closing) this.onTransportFailure(error) + return + } finally { + if (this.interrupting === operation) this.interrupting = undefined } - } - - private survivors(members: ProcessIdentity[]): ProcessIdentity[] { - return members.filter(member => this.inspector.isAlive(member)) - } - - private descendants(): ProcessIdentity[] { - return this.inspector.processTree(this.pid).filter(member => member.pid !== this.pid) - } - - private async waitForExit(members: ProcessIdentity[]): Promise { - const deadline = Date.now() + this.config.disposeGraceMs - let survivors = this.survivors(members) - while (survivors.length > 0 && Date.now() < deadline) { - await delay(Math.min(25, Math.max(1, deadline - Date.now()))) - survivors = this.survivors(members) - } - return survivors - } - - private signalMembers(members: ProcessIdentity[], signal: 'SIGTERM' | 'SIGKILL'): void { - for (const member of members) { - try { - this.inspector.signalProcess(member, signal) - } catch (_alreadyExitedDuringSignal) { - // Identity is rechecked by the inspector; a same-tick exit is success. - } - } - } - - private unionMembers(...groups: ProcessIdentity[][]): ProcessIdentity[] { - const members: ProcessIdentity[] = [] - const seen = new Set() - for (const group of groups) { - for (const member of group) { - const key = JSON.stringify([member.pid, member.started]) - if (seen.has(key)) continue - seen.add(key) - members.push(member) - } - } - return members - } - - private async stopDescendants(): Promise { - const captured = this.descendants() - this.signalMembers(captured, 'SIGTERM') - const capturedSurvivors = await this.waitForExit(captured) - // A TERM-handling descendant may have forked while winding down. Rescan - // while the shell can still reap every member, then kill both the fresh - // tree and captured survivors that were reparented out of that tree. - const members = this.unionMembers(capturedSurvivors, this.descendants()) - this.signalMembers(members, 'SIGKILL') - const survivors = await this.waitForExit(members) - return this.survivors(this.unionMembers(survivors, this.descendants())) - } - - private async stopShell(): Promise { - try { - this.terminal.kill('SIGTERM') - } catch (_topLevelAlreadyExitedDuringTerm) { - // The exit notification remains authoritative. - } - if (this.statusValue.kind === 'running') { - await Promise.race([this.exitPromise.promise, delay(this.config.disposeGraceMs)]) - } - if (this.statusValue.kind === 'running') { - try { - this.terminal.kill('SIGKILL') - } catch (_topLevelAlreadyExitedDuringKill) { - // The exit notification remains authoritative. - } - await Promise.race([this.exitPromise.promise, delay(this.config.disposeGraceMs)]) - } - if (this.statusValue.kind === 'running') { - throw new Error(`PTY cleanup failed; surviving pids: ${this.pid}`) + if (this.active === operation && operation.settled) { + this.clearActive() + } else if (this.active === operation && !this.closing) { + this.pollingReady = operation + this.schedulePoll(operation, 0) } } private async closeOnce(reason: string): Promise { - this.dataDisposable.dispose() // Stop readiness polling but retain the active operation: teardown settles // it as session_exit below, so an in-flight send is never mis-settled as // stdin_read/inferred_idle/timeout during the grace period. this.stopPolling() - const survivors = await this.stopDescendants() - if (survivors.length > 0) { - throw new Error(`PTY cleanup failed (${reason}); surviving pids: ${survivors.map(member => member.pid).join(', ')}`) + try { + await this.terminal.terminate() + } catch (error: unknown) { + throw new Error(`PTY cleanup failed (${reason})`, { cause: error }) } - await this.stopShell() + // Quiescence is the active send's terminal outcome. this.settleActive('session_exit') - this.exitDisposable.dispose() + await this.completion + this.terminal.output.off('data', this.onTerminalData) + this.terminal.output.off('end', this.onTerminalEnd) + this.terminal.output.off('error', this.onTerminalError) + if (this.transportFailure !== undefined) throw this.transportFailure } } diff --git a/packages/pty/pty-local/tests/index.spec.ts b/packages/pty/pty-local/tests/index.spec.ts index 1ad29d7d01..d61eb58da5 100644 --- a/packages/pty/pty-local/tests/index.spec.ts +++ b/packages/pty/pty-local/tests/index.spec.ts @@ -1,5 +1,5 @@ import { describe, expect, it, vi } from 'vitest' -import type { IPty, IPtyForkOptions } from 'node-pty' +import { PassThrough } from 'node:stream' import { Context } from 'cordis' import Loader from '@cordisjs/plugin-loader' import SessionStore, { Session, SessionId } from '@deepseek-ai/dsh-session' @@ -11,8 +11,14 @@ import PtyService, { PtyBackendCleanupError, PtySessionId } from '@deepseek-ai/d import { LocalPtyBackend } from '@deepseek-ai/dsh-pty-local' import * as ptyLocal from '@deepseek-ai/dsh-pty-local' import type { ResolvedConfig } from '@deepseek-ai/dsh-pty-local/src/config.ts' -import type { ProcessInspector } from '@deepseek-ai/dsh-pty-local/src/process-inspector.ts' import type { LocalPtySession } from '@deepseek-ai/dsh-pty-local/src/session.ts' +import { SubprocessService } from '@deepseek-ai/dsh-subprocess' +import type { + SubprocessHandle, + SubprocessSpawnSpec, + SubprocessTerminalHandle, + SubprocessTerminalSpawnSpec, +} from '@deepseek-ai/dsh-subprocess' class EmptySandbox extends SandboxProvider { confine(_argv: readonly string[], _policy: SandboxPolicy): ConfinedArgv { @@ -52,14 +58,26 @@ function agent(ctx: Context, cwd?: string): Agent { } } -const inspector = { - foregroundPgid: () => undefined, - isStdinWaiting: () => false, - processTree: () => [], - isAlive: () => false, - signalGroup() {}, - signalProcess() {}, -} satisfies ProcessInspector +function terminalHandle(): SubprocessTerminalHandle { + const output = new PassThrough() + return { + pid: 123, + output, + done: Promise.resolve({ exitCode: 0, signal: null }), + write: async () => {}, + inspectForeground: async () => ({ processGroupId: 123, inputWaiting: true }), + signalForeground: async () => 123, + terminate: async () => { output.end() }, + } +} + +class StubSubprocessService extends SubprocessService { + async resolveExecutable(command: string): Promise { return command } + spawn(_spec: SubprocessSpawnSpec): SubprocessHandle { throw new Error('unused') } + async spawnTerminal(_spec: SubprocessTerminalSpawnSpec): Promise { + return terminalHandle() + } +} function spec(owner: Agent, signal?: AbortSignal) { return { @@ -81,12 +99,11 @@ function stubLocalSession(initialize: () => Promise = () => Promise.resolv } function registerStubLocalBackend(ctx: Context, createSession: () => LocalPtySession) { - return ctx.inject(['pty', 'sandbox', 'sandboxPolicy'], (providerCtx) => { + return ctx.inject(['pty', 'sandbox', 'sandboxPolicy', 'subprocess'], (providerCtx) => { providerCtx.pty.registerBackend(new LocalPtyBackend( providerCtx, { ...config(), backendType: 'stub' }, - inspector, - (() => ({})) as never, + async () => terminalHandle(), createSession, )) }) @@ -97,7 +114,7 @@ describe('LocalPtyBackend startup rollback', () => { const ctx = new Context() await ctx.plugin(EmptySandbox) await ctx.plugin(SandboxPolicyService, { mode: 'read-only', workspaceRoot: '/tmp' }) - const backend = new LocalPtyBackend(ctx, config(), inspector) + const backend = new LocalPtyBackend(ctx, config(), async () => terminalHandle()) const controller = new AbortController() const abortReason = new Error('spawn aborted') controller.abort(abortReason) @@ -107,13 +124,12 @@ describe('LocalPtyBackend startup rollback', () => { it('closes failed startup and aggregates cleanup failure', async () => { const ctx = new Context() - await ctx.plugin(EmptySandbox) await ctx.plugin(SandboxPolicyService, { mode: 'danger-full-access', workspaceRoot: '/tmp' }) - const spawnTerminal = (() => ({} as IPty)) as never + const spawnTerminal = async (): Promise => terminalHandle() const closed = vi.fn<() => Promise>().mockResolvedValue(undefined) const failed = { initialize: () => Promise.reject(new Error('startup failed')), close: closed } as unknown as LocalPtySession - const backend = new LocalPtyBackend(ctx, config(), inspector, spawnTerminal, () => failed) + const backend = new LocalPtyBackend(ctx, config(), spawnTerminal, () => failed) await expect(backend.spawn(spec(agent(ctx)))).rejects.toThrow('startup failed') expect(closed).toHaveBeenCalledWith('PTY startup failed') @@ -123,7 +139,7 @@ describe('LocalPtyBackend startup rollback', () => { initialize: () => Promise.reject(startupFailure), close: () => Promise.reject(cleanupFailure), } as unknown as LocalPtySession - const aggregate = new LocalPtyBackend(ctx, config(), inspector, spawnTerminal, () => doublyFailed) + const aggregate = new LocalPtyBackend(ctx, config(), spawnTerminal, () => doublyFailed) await expect(aggregate.spawn(spec(agent(ctx)))).rejects.toEqual(expect.objectContaining({ name: 'PtyBackendCleanupError', spawnError: startupFailure, @@ -131,79 +147,191 @@ describe('LocalPtyBackend startup rollback', () => { } satisfies Partial)) }) - it('resolves session mode and root together before wrapping the shell', async () => { + it('starts startup rollback when cancellation wins a stalled initialization', async () => { + const ctx = new Context() + await ctx.plugin(SandboxPolicyService, { mode: 'danger-full-access', workspaceRoot: '/tmp' }) + const initialization = Promise.withResolvers() + const initializationStarted = Promise.withResolvers() + const close = vi.fn<() => Promise>().mockResolvedValue(undefined) + const session = { + initialize: () => { + initializationStarted.resolve(undefined) + return initialization.promise + }, + close, + } as unknown as LocalPtySession + const backend = new LocalPtyBackend(ctx, config(), async () => terminalHandle(), () => session) + const controller = new AbortController() + const reason = new Error('cancel stalled startup') + + const spawning = backend.spawn(spec(agent(ctx), controller.signal)) + await initializationStarted.promise + controller.abort(reason) + + await expect(spawning).rejects.toBe(reason) + expect(close).toHaveBeenCalledWith('PTY startup failed') + initialization.resolve(undefined) + }) + + it('wraps confined argv, scrubs the environment, and returns initialized sessions', async () => { const ctx = new Context() await ctx.plugin(RecordingSandbox) - await ctx.plugin(SandboxPolicyService, { mode: 'read-only', workspaceRoot: '/deployment-fallback' }) - const terminal = {} as IPty - let spawned: { file: string; args: string[]; options: IPtyForkOptions } | undefined - const spawnTerminal = ((file: string, args: string[], options: IPtyForkOptions) => { - spawned = { file, args, options } + await ctx.plugin(SandboxPolicyService, { mode: 'workspace-write', workspaceRoot: '/workspace' }) + const terminal = terminalHandle() + let spawned: SubprocessTerminalSpawnSpec | undefined + const spawnTerminal = async (spec: SubprocessTerminalSpawnSpec): Promise => { + spawned = spec return terminal - }) as never + } const initialized = vi.fn<() => Promise>().mockResolvedValue(undefined) const session = { initialize: initialized } as unknown as LocalPtySession const backend = new LocalPtyBackend( ctx, { ...config(), shellArgs: ['-i'] }, - inspector, spawnTerminal, () => session, ) const previous = process.env.PTY_TEST_SECRET process.env.PTY_TEST_SECRET = 'must-not-leak' - const owner = agent(ctx, '/session-workspace') - setSandboxMode(owner.session, 'workspace-write') try { - expect(await backend.spawn(spec(owner))).toBe(session) + expect(await backend.spawn({ ...spec(agent(ctx)), cwd: '/work' })).toBe(session) } finally { if (previous === undefined) delete process.env.PTY_TEST_SECRET else process.env.PTY_TEST_SECRET = previous } expect(spawned).toMatchObject({ - file: '/sandbox', - args: ['--', '/bin/bash', '-i'], - options: { - name: 'dumb', cols: 80, rows: 24, cwd: '/session-workspace', - env: { - TERM: 'dumb', PAGER: 'cat', GIT_PAGER: 'cat', PS1: 'dsh> ', BASH_SILENCE_DEPRECATION_WARNING: '1', - DSH_SHELL: '1', DSH_SESSION_ID: 'agent', DSH_PTY_SESSION_ID: 'pty-1', - }, + argv: ['/sandbox', '--', '/bin/bash', '-i'], + cols: 80, + rows: 24, + cwd: '/work', + graceMs: 10, + env: { + TERM: 'dumb', PAGER: 'cat', GIT_PAGER: 'cat', PS1: 'dsh> ', BASH_SILENCE_DEPRECATION_WARNING: '1', + DSH_SHELL: '1', DSH_SESSION_ID: 'agent', DSH_PTY_SESSION_ID: 'pty-1', }, }) - expect(spawned?.options.env?.PTY_TEST_SECRET).toBeUndefined() + expect(spawned?.env?.PTY_TEST_SECRET).toBeUndefined() expect(initialized).toHaveBeenCalledWith(undefined) + expect((ctx.sandbox as RecordingSandbox).calls).toEqual([{ + argv: ['/bin/bash', '-i'], + policy: { mode: 'workspace-write', workspaceRoot: '/workspace' }, + }]) + }) + + it('resolves session mode and root together before wrapping the shell', async () => { + const ctx = new Context() + await ctx.plugin(RecordingSandbox) + await ctx.plugin(SandboxPolicyService, { mode: 'read-only', workspaceRoot: '/deployment-fallback' }) + const terminal = terminalHandle() + let spawned: SubprocessTerminalSpawnSpec | undefined + const spawnTerminal = async (spec: SubprocessTerminalSpawnSpec): Promise => { + spawned = spec + return terminal + } + const initialized = vi.fn<() => Promise>().mockResolvedValue(undefined) + const session = { initialize: initialized } as unknown as LocalPtySession + const backend = new LocalPtyBackend( + ctx, + { ...config(), shellArgs: ['-i'] }, + spawnTerminal, + () => session, + ) + const owner = agent(ctx, '/session-workspace') + setSandboxMode(owner.session, 'workspace-write') + expect(await backend.spawn(spec(owner))).toBe(session) + + expect(spawned).toMatchObject({ + argv: ['/sandbox', '--', '/bin/bash', '-i'], + cwd: '/session-workspace', + }) expect((ctx.sandbox as RecordingSandbox).calls).toEqual([{ argv: ['/bin/bash', '-i'], policy: { mode: 'workspace-write', workspaceRoot: '/session-workspace' }, }]) }) + it('rejects a confined spawn without a sandbox provider', async () => { + const confinedCtx = new Context() + await confinedCtx.plugin(SandboxPolicyService, { mode: 'workspace-write', workspaceRoot: '/workspace' }) + const confined = new LocalPtyBackend( + confinedCtx, + config(), + async () => { throw new Error('terminal spawn must not run') }, + () => stubLocalSession(), + ) + await expect(confined.spawn(spec(agent(confinedCtx)))).rejects.toThrow( + 'sandbox mode "workspace-write" requires a ctx.sandbox provider in the execution world', + ) + }) + + it('forwards terminal allocation cancellation directly', async () => { + const ctx = new Context() + await ctx.plugin(EmptySandbox) + await ctx.plugin(SandboxPolicyService, { mode: 'danger-full-access', workspaceRoot: '/tmp' }) + + const publishedController = new AbortController() + let publishedSignal: AbortSignal | undefined + const published = new LocalPtyBackend( + ctx, + config(), + async (spawnSpec) => { + publishedSignal = spawnSpec.signal + return terminalHandle() + }, + () => stubLocalSession(), + ) + await published.spawn(spec(agent(ctx), publishedController.signal)) + expect(publishedSignal).toBe(publishedController.signal) + publishedController.abort(new Error('originating turn ended')) + expect(publishedSignal?.aborted).toBe(true) + + const pendingController = new AbortController() + const seen = Promise.withResolvers() + const pending = new LocalPtyBackend( + ctx, + config(), + async spawnSpec => await new Promise((_resolve, reject) => { + const setupSignal = spawnSpec.signal as AbortSignal + seen.resolve(setupSignal) + const onAbort = (): void => { + reject(setupSignal.reason instanceof Error ? setupSignal.reason : new Error(String(setupSignal.reason))) + } + setupSignal.addEventListener('abort', onAbort, { once: true }) + }), + () => stubLocalSession(), + ) + const spawning = pending.spawn(spec(agent(ctx), pendingController.signal)) + const pendingSignal = await seen.promise + const reason = new Error('cancel pending allocation') + pendingController.abort(reason) + await expect(spawning).rejects.toBe(reason) + expect(pendingSignal.aborted).toBe(true) + }) + it('composes the default local session around a spawned terminal', async () => { const ctx = new Context() await ctx.plugin(EmptySandbox) await ctx.plugin(SandboxPolicyService, { mode: 'danger-full-access', workspaceRoot: '/workspace' }) - let exitListener: ((event: { exitCode: number; signal?: number }) => void) | undefined - const terminal = { - pid: 123, cols: 80, rows: 24, process: 'bash', handleFlowControl: false, - onData(listener: (data: string) => void) { - queueMicrotask(() => { listener('\x1b]133;D;0\x07dsh> ') }) - return { dispose() {} } + const output = new PassThrough() + const outcome = Promise.withResolvers<{ exitCode: number | null; signal: NodeJS.Signals | null }>() + const terminal: SubprocessTerminalHandle = { + pid: 123, + output, + done: outcome.promise, + write: async () => {}, + inspectForeground: async () => ({ processGroupId: 123, inputWaiting: true }), + signalForeground: async () => 123, + async terminate() { + output.end() + outcome.resolve({ exitCode: null, signal: 'SIGTERM' }) }, - onExit(listener: (event: { exitCode: number; signal?: number }) => void) { - exitListener = listener - return { dispose() {} } - }, - write() {}, - kill() { exitListener?.({ exitCode: 0, signal: 15 }) }, - resize() {}, clear() {}, pause() {}, resume() {}, - } as IPty + } + queueMicrotask(() => { output.write(Buffer.from('\x1b]133;D;0\x07dsh> ')) }) const backend = new LocalPtyBackend( ctx, config(), - { ...inspector, foregroundPgid: () => terminal.pid }, - () => terminal, + async () => terminal, ) const session = await backend.spawn(spec(agent(ctx))) expect(session.motd).toBe('dsh> ') @@ -217,7 +345,7 @@ describe('pty-local plugin shape', () => { const loader = Object.create(Loader.prototype) as Loader const unwrapped = loader.unwrapExports(ptyLocal) as Record expect(unwrapped.name).toBe('pty-local') - expect(unwrapped.inject).toEqual(['pty', 'sandbox', 'sandboxPolicy']) + expect(unwrapped.inject).toEqual(['pty', 'sandboxPolicy', 'subprocess']) expect(unwrapped.Config).toBeDefined() }) @@ -225,8 +353,8 @@ describe('pty-local plugin shape', () => { const ctx = new Context() await ctx.plugin(AgentRegistry) await ctx.plugin(PtyService) - await ctx.plugin(EmptySandbox) await ctx.plugin(SandboxPolicyService, { mode: 'danger-full-access', workspaceRoot: '/tmp' }) + await ctx.plugin(StubSubprocessService) const fiber = await ctx.plugin(ptyLocal, config()) expect(ctx.pty.listBackends()).toEqual(['shell']) await fiber.dispose() @@ -240,6 +368,7 @@ describe('pty-local plugin shape', () => { await ctx.plugin(PtyService) await ctx.plugin(EmptySandbox) await ctx.plugin(SandboxPolicyService, { mode: 'danger-full-access', workspaceRoot: '/tmp' }) + await ctx.plugin(StubSubprocessService) await ctx.plugin(ptyLocal, config()) const session = ctx.sessions.create(SessionId('unowned-mode')) @@ -256,6 +385,7 @@ describe('pty-local plugin shape', () => { await ctx.plugin(PtyService) await ctx.plugin(RecordingSandbox) await ctx.plugin(SandboxPolicyService, { mode: 'danger-full-access', workspaceRoot: '/tmp' }) + await ctx.plugin(StubSubprocessService) const session = ctx.sessions.create(SessionId('mode-owner')) const ownerFiber = await ctx.plugin(() => {}) @@ -304,6 +434,7 @@ describe('pty-local plugin shape', () => { await ctx.plugin(PtyService) await ctx.plugin(RecordingSandbox) await ctx.plugin(SandboxPolicyService, { mode: 'danger-full-access', workspaceRoot: '/tmp' }) + await ctx.plugin(StubSubprocessService) const session = ctx.sessions.create(SessionId('pending-mode-owner')) const ownerFiber = await ctx.plugin(() => {}) diff --git a/packages/pty/pty-local/tests/local.spec.ts b/packages/pty/pty-local/tests/local.spec.ts index c57b0ebb8e..1c0de1660a 100644 --- a/packages/pty/pty-local/tests/local.spec.ts +++ b/packages/pty/pty-local/tests/local.spec.ts @@ -1,4 +1,4 @@ -import { mkdtempSync, realpathSync, rmSync } from 'node:fs' +import { existsSync, mkdtempSync, readFileSync, realpathSync, rmSync } from 'node:fs' import { tmpdir } from 'node:os' import { join } from 'node:path' import { afterEach, describe, expect, it } from 'vitest' @@ -11,6 +11,7 @@ import type { PtySendOperation } from '@deepseek-ai/dsh-pty' import SandboxProvider from '@deepseek-ai/dsh-sandbox' import type { ConfinedArgv, SandboxPolicy } from '@deepseek-ai/dsh-sandbox' import SandboxPolicyService from '@deepseek-ai/dsh-sandbox-policy' +import LocalSubprocessService from '@deepseek-ai/dsh-subprocess-local' import * as ptyLocal from '@deepseek-ai/dsh-pty-local' const roots: string[] = [] @@ -57,6 +58,7 @@ async function harness( await ctx.plugin(PtyService) await ctx.plugin(PassthroughSandbox) await ctx.plugin(SandboxPolicyService, { mode, workspaceRoot: root }) + await ctx.plugin(LocalSubprocessService) const fiber = await ctx.plugin(ptyLocal, { pollIntervalMs: 10, exactProbeAfterMs: 20, @@ -96,6 +98,22 @@ function expectReadyForNextSend(waitReason: string): void { expect(['stdin_read', 'inferred_idle']).toContain(waitReason) } +function processIsRunning(pid: number): boolean { + try { + process.kill(pid, 0) + } catch (_missingProcess) { + return false + } + if (process.platform !== 'linux') return true + try { + const stat = readFileSync(`/proc/${pid}/stat`, 'utf8') + const state = stat.slice(stat.lastIndexOf(')') + 2).split(/\s+/, 1)[0] + return !/^[ZXx]$/.test(state ?? '') + } catch (_unreadableProcEntry) { + return false + } +} + describe('pty-local real shell', () => { it('persists cwd and environment across sends, scrubs secrets, and closes', async () => { const previous = process.env.DSH_TEST_SECRET @@ -154,6 +172,47 @@ describe('pty-local real shell', () => { expect(() => process.kill(pid, 0)).toThrow() }, 10_000) + it('quiesces a disowned same-session descendant after the shell exits naturally', async () => { + const { ctx, root, agent } = await harness('danger-full-access') + const created = await ctx.pty.spawn(agent, { type: 'shell' }) + const pidFile = join(root, 'disowned.pid') + let pid: number | undefined + try { + const background = ctx.pty.startSend(agent, created.sessionId, { + text: `sh -c 'trap "" TERM; printf "%s" "$$" > "$1"; sleep 60' dsh "${pidFile}" & disown`, + submit: true, + }) + await background.done + const pidDeadline = Date.now() + 2_000 + let childPid = 0 + while (childPid === 0 && Date.now() < pidDeadline) { + if (existsSync(pidFile)) childPid = Number(readFileSync(pidFile, 'utf8')) + if (childPid > 0) break + await new Promise(resolve => setTimeout(resolve, 10)) + } + expect(existsSync(pidFile), ctx.pty.read(agent, created.sessionId, { offset: 0, count: 100 }).text).toBe(true) + expect(childPid).toBeGreaterThan(0) + pid = childPid + expect(() => process.kill(childPid, 0)).not.toThrow() + await ctx.pty.startSend(agent, created.sessionId, { text: 'exit', submit: true }).done + const deadline = Date.now() + 2_000 + while (ctx.pty.list(agent)[0]?.status.kind !== 'exited' && Date.now() < deadline) { + await new Promise(resolve => setTimeout(resolve, 10)) + } + expect(ctx.pty.list(agent)[0]?.status.kind).toBe('exited') + await ctx.pty.kill(agent, created.sessionId) + expect(processIsRunning(childPid)).toBe(false) + } finally { + if (pid !== undefined) { + try { + process.kill(pid, 'SIGKILL') + } catch (_alreadyReaped) { + // Product cleanup is the expected path; this only contains a failed regression. + } + } + } + }, 10_000) + it('cancels a slow-starting raw-mode foreground process with a real SIGINT', async () => { const { ctx, agent } = await harness('danger-full-access', { idleSilenceMs: 10_000, diff --git a/packages/pty/pty-local/tests/sanitize.spec.ts b/packages/pty/pty-local/tests/sanitize.spec.ts index 4da4ab0d73..f6649b3c6f 100644 --- a/packages/pty/pty-local/tests/sanitize.spec.ts +++ b/packages/pty/pty-local/tests/sanitize.spec.ts @@ -7,7 +7,7 @@ describe('TerminalSanitizer', () => { expect(sanitizer.push('red\x1b[3')).toEqual({ text: 'red', prompt: false }) expect(sanitizer.push('1m text\x1b[0m\r\n')).toEqual({ text: ' text\n', prompt: false }) expect(sanitizer.push('\x1b]133;')).toEqual({ text: '', prompt: false }) - expect(sanitizer.push('D;0\x07dsh> ')).toEqual({ text: 'dsh> ', prompt: true, promptText: true }) + expect(sanitizer.push('D;0\x07dsh> ')).toEqual({ text: 'dsh> ', prompt: true, promptTail: 'dsh> ' }) }) it('drops unrelated OSC, short escapes, BEL, and incomplete trailing escape', () => { @@ -35,8 +35,8 @@ describe('TerminalSanitizer', () => { it('reports printable prompt text that follows a marker in a later chunk', () => { const sanitizer = new TerminalSanitizer(64) - expect(sanitizer.push('\x1b]133;D;0\x07')).toEqual({ text: '', prompt: true }) - expect(sanitizer.push('dsh> ')).toEqual({ text: 'dsh> ', prompt: false, promptText: true }) + expect(sanitizer.push('\x1b]133;D;0\x07')).toEqual({ text: '', prompt: true, promptTail: '' }) + expect(sanitizer.push('dsh> ')).toEqual({ text: 'dsh> ', prompt: false, promptTail: 'dsh> ' }) }) it('bounds and discards unterminated control sequences through their terminators', () => { diff --git a/packages/pty/pty-local/tests/session.spec.ts b/packages/pty/pty-local/tests/session.spec.ts index 53ce61eebc..0ca1835e53 100644 --- a/packages/pty/pty-local/tests/session.spec.ts +++ b/packages/pty/pty-local/tests/session.spec.ts @@ -1,62 +1,18 @@ import { afterEach, describe, expect, it, vi } from 'vitest' -import type { IDisposable, IPty } from 'node-pty' +import { PassThrough } from 'node:stream' import { LocalPtySession } from '@deepseek-ai/dsh-pty-local/src/session.ts' import type { ResolvedConfig } from '@deepseek-ai/dsh-pty-local/src/config.ts' -import type { ProcessIdentity, ProcessInspector } from '@deepseek-ai/dsh-pty-local/src/process-inspector.ts' import type { PtySendOperation, PtySessionStatus, PtySignal } from '@deepseek-ai/dsh-pty' - -class FakeTerminal { - pid = 123 - cols = 80 - rows = 24 - process = 'bash' - handleFlowControl = false - writes: string[] = [] - kills: string[] = [] - throwWrite = false - throwKill = false - autoExitOnKill = true - private dataListeners = new Set<(data: string) => void>() - private exitListeners = new Set<(event: { exitCode: number; signal?: number }) => void>() - - readonly onData = (listener: (data: string) => void): IDisposable => { - this.dataListeners.add(listener) - return { dispose: () => this.dataListeners.delete(listener) } - } - - readonly onExit = (listener: (event: { exitCode: number; signal?: number }) => void): IDisposable => { - this.exitListeners.add(listener) - return { dispose: () => this.exitListeners.delete(listener) } - } - - emitData(data: string): void { - for (const listener of this.dataListeners) listener(data) - } - - emitExit(exitCode = 0, signal?: number): void { - for (const listener of this.exitListeners) listener({ exitCode, ...signal === undefined ? {} : { signal } }) - } - - write(data: string): void { - if (this.throwWrite) throw new Error('write failed') - this.writes.push(data) - } - - kill(signal?: string): void { - if (this.throwKill) throw new Error('kill failed') - this.kills.push(signal ?? 'SIGHUP') - if (this.autoExitOnKill) this.emitExit(0, signal === 'SIGKILL' ? 9 : 15) - } - - resize() {} - clear() {} - pause() {} - resume() {} - - asPty(): IPty { - return this - } -} +import type { + SubprocessOutcome, + SubprocessTerminalHandle, + SubprocessTerminalSignal, +} from '@deepseek-ai/dsh-subprocess' +import { PtyError } from '@deepseek-ai/dsh-pty' +import type { + ProcessIdentity, + ProcessInspector, +} from '@deepseek-ai/dsh-subprocess-local/src/process-inspector.ts' class FakeInspector implements ProcessInspector { pgid: number | undefined = 456 @@ -72,6 +28,7 @@ class FakeInspector implements ProcessInspector { foregroundPgid() { return this.pgid } isStdinWaiting() { return this.waiting } processTree() { return this.members } + processSession() { return [] } isAlive(identity: ProcessIdentity) { return this.alive.has(identity.pid) } signalGroup(pgid: number, signal: PtySignal) { if (this.throwGroup) throw new Error('group failed') @@ -84,6 +41,93 @@ class FakeInspector implements ProcessInspector { } } +class FakeTerminal implements SubprocessTerminalHandle { + pid = 123 + readonly output = new PassThrough() + readonly writes: string[] = [] + readonly kills: string[] = [] + readonly outcome = Promise.withResolvers() + readonly done = this.outcome.promise + throwWrite = false + throwKill = false + autoExitOnKill = true + terminateError: Error | undefined + private cleanup: Promise | undefined + + constructor(public inspector = new FakeInspector()) {} + + emitData(data: string): void { + this.output.write(Buffer.from(data, 'utf8')) + } + + emitBytes(data: Uint8Array): void { + this.output.write(data) + } + + emitError(error: Error): void { + this.output.emit('error', error) + } + + emitFailure(error: unknown): void { + this.output.end() + this.outcome.reject(error) + } + + emitExit(exitCode = 0, signal?: number): void { + this.output.end() + this.outcome.resolve({ + exitCode: signal === undefined || signal === 0 ? exitCode : null, + signal: signal === 9 ? 'SIGKILL' : signal === 15 ? 'SIGTERM' : null, + }) + } + + async write(data: string): Promise { + if (this.throwWrite) throw new Error('write failed') + this.writes.push(data) + } + + async inspectForeground() { + const processGroupId = this.inspector.foregroundPgid() + return processGroupId === undefined + ? undefined + : { processGroupId, inputWaiting: this.inspector.isStdinWaiting() } + } + + async signalForeground(signal: SubprocessTerminalSignal): Promise { + const foreground = await this.inspectForeground() + if (foreground === undefined) throw new Error(`cannot resolve foreground process group for terminal ${this.pid}`) + if (signal === 'SIGKILL' && foreground.processGroupId === this.pid) { + throw new Error('refusing to SIGKILL the terminal shell; terminate the terminal session instead') + } + this.inspector.signalGroup(foreground.processGroupId, signal) + return foreground.processGroupId + } + + terminate(): Promise { + if (this.cleanup !== undefined) return this.cleanup + const cleanup = this.terminateOnce() + this.cleanup = cleanup + void cleanup.catch(() => { this.cleanup = undefined }) + return cleanup + } + + private async terminateOnce(): Promise { + if (this.terminateError !== undefined) throw this.terminateError + if (this.throwKill) throw new Error('kill failed') + this.kills.push('SIGTERM') + if (this.autoExitOnKill) this.emitExit(0, 15) + } +} + +function makeSession( + terminal: FakeTerminal, + inspector: FakeInspector, + resolved: ResolvedConfig, +): LocalPtySession { + terminal.inspector = inspector + return new LocalPtySession(terminal, resolved) +} + function config(overrides: Partial = {}): ResolvedConfig { return { backendType: 'shell', shellPath: '/bin/bash', shellArgs: [], rows: 24, cols: 80, @@ -104,17 +148,67 @@ async function initialize(session: LocalPtySession, terminal: FakeTerminal): Pro } describe('LocalPtySession readiness and output', () => { + it('lets queued terminal output run before the first post-write readiness poll', async () => { + vi.useFakeTimers() + const terminal = new FakeTerminal() + const inspector = new FakeInspector() + const session = makeSession(terminal, inspector, config()) + await initialize(session, terminal) + + const inspect = terminal.inspectForeground.bind(terminal) + let inspections = 0 + terminal.inspectForeground = async () => { + inspections += 1 + return await inspect() + } + const operation = session.startSend({ text: 'true', submit: true }) + await Promise.resolve() + await Promise.resolve() + expect(inspections).toBe(1) + + await vi.advanceTimersByTimeAsync(0) + expect(inspections).toBe(1) + terminal.emitData('\x1b]133;D;0\x07dsh> ') + await vi.advanceTimersByTimeAsync(10) + expect((await operation.done).waitReason).toBe('stdin_read') + }) + + it('discards prompt readiness observed during asynchronous pre-write inspection', async () => { + vi.useFakeTimers() + const terminal = new FakeTerminal() + const session = new LocalPtySession(terminal, config()) + await initialize(session, terminal) + + const inspection = Promise.withResolvers<{ processGroupId: number; inputWaiting: boolean }>() + terminal.inspectForeground = async () => await inspection.promise + const operation = session.startSend({ text: 'long-running-command', submit: true }) + let settled = false + void operation.done.then(() => { settled = true }) + + terminal.emitData('\x1b]133;D;0\x07dsh> ') + inspection.resolve({ processGroupId: 456, inputWaiting: true }) + await vi.advanceTimersByTimeAsync(20) + expect(terminal.writes).toEqual(['long-running-command\r']) + expect(settled).toBe(false) + + terminal.emitData('\x1b]133;D;0\x07dsh> ') + await vi.advanceTimersByTimeAsync(10) + expect((await operation.done).waitReason).toBe('stdin_read') + }) + it('captures prompt MOTD, writes submit explicitly, and settles exact stdin waits', async () => { vi.useFakeTimers() const terminal = new FakeTerminal() const inspector = new FakeInspector() - const session = new LocalPtySession(terminal.asPty(), inspector, config()) + const session = makeSession(terminal, inspector, config()) await initialize(session, terminal) expect(session.motd).toBe('dsh> ') inspector.waiting = true const operation = session.startSend({ text: 'python3', submit: true }) - expect(terminal.writes).toEqual(['python3', '\r']) + await Promise.resolve() + await Promise.resolve() + expect(terminal.writes).toEqual(['python3\r']) inspector.pgid = 789 terminal.emitData('Python\r\n>>> ') await vi.advanceTimersByTimeAsync(20) @@ -126,7 +220,7 @@ describe('LocalPtySession readiness and output', () => { vi.useFakeTimers() const terminal = new FakeTerminal() const inspector = new FakeInspector() - const session = new LocalPtySession(terminal.asPty(), inspector, config()) + const session = makeSession(terminal, inspector, config()) await initialize(session, terminal) inspector.waiting = true @@ -148,7 +242,7 @@ describe('LocalPtySession readiness and output', () => { vi.useFakeTimers() const terminal = new FakeTerminal() const inspector = new FakeInspector() - const session = new LocalPtySession(terminal.asPty(), inspector, config({ + const session = makeSession(terminal, inspector, config({ exactProbeAfterMs: 50, idleSilenceMs: 100, timeoutMs: 200, @@ -173,7 +267,7 @@ describe('LocalPtySession readiness and output', () => { vi.useFakeTimers() const terminal = new FakeTerminal() const inspector = new FakeInspector() - const session = new LocalPtySession(terminal.asPty(), inspector, config()) + const session = makeSession(terminal, inspector, config()) await initialize(session, terminal) inspector.pgid = undefined @@ -193,7 +287,7 @@ describe('LocalPtySession readiness and output', () => { const exiting = session.startSend({ text: 'exit', submit: true }) terminal.emitExit(7, 9) - expect(await exiting.done).toMatchObject({ waitReason: 'session_exit', sessionStatus: { kind: 'exited', exitCode: 7, signal: 'SIGKILL' } }) + expect(await exiting.done).toMatchObject({ waitReason: 'session_exit', sessionStatus: { kind: 'exited', exitCode: null, signal: 'SIGKILL' } }) expect(() => session.startSend({ text: '', submit: false })).toThrow('has exited') }) @@ -201,13 +295,15 @@ describe('LocalPtySession readiness and output', () => { vi.useFakeTimers() const terminal = new FakeTerminal() const inspector = new FakeInspector() - const session = new LocalPtySession(terminal.asPty(), inspector, config()) + const session = makeSession(terminal, inspector, config()) await initialize(session, terminal) const controller = new AbortController() const operation = session.startSend({ text: 'sleep', submit: true, signal: controller.signal }) expect(() => session.startSend({ text: 'again', submit: true })).toThrow('active send') controller.abort() + await Promise.resolve() + await Promise.resolve() expect(inspector.groups).toContainEqual([456, 'SIGINT']) expect(terminal.writes).not.toContain('\x03') terminal.emitData('\x1b]133;D;130\x07dsh> ') @@ -226,17 +322,391 @@ describe('LocalPtySession readiness and output', () => { failedInternal.fail(new Error('ignored')) }) + it('does not write a send canceled during asynchronous foreground inspection', async () => { + vi.useFakeTimers() + const terminal = new FakeTerminal() + const inspector = new FakeInspector() + const session = makeSession(terminal, inspector, config()) + await initialize(session, terminal) + + const inspection = Promise.withResolvers<{ processGroupId: number; inputWaiting: boolean }>() + terminal.inspectForeground = async () => await inspection.promise + const controller = new AbortController() + const operation = session.startSend({ text: 'must not execute', submit: true, signal: controller.signal }) + controller.abort() + inspection.resolve({ processGroupId: 456, inputWaiting: false }) + await Promise.resolve() + await Promise.resolve() + + expect(terminal.writes).toEqual([]) + expect(inspector.groups).toContainEqual([456, 'SIGINT']) + terminal.emitData('\x1b]133;D;130\x07dsh> ') + await vi.advanceTimersByTimeAsync(10) + await operation.done + }) + + it('retains a canceled send when the pre-write inspection rejects while its signal is in flight', async () => { + vi.useFakeTimers() + const terminal = new FakeTerminal() + const inspector = new FakeInspector() + const session = makeSession(terminal, inspector, config()) + await initialize(session, terminal) + + const failed = Promise.withResolvers<{ processGroupId: number; inputWaiting: boolean }>() + terminal.inspectForeground = async () => await failed.promise + const uncanceled = session.startSend({ text: 'plain failure', submit: true }) + failed.reject(new Error('inspect failed before write')) + await expect(uncanceled.done).rejects.toThrow('inspect failed before write') + + terminal.inspectForeground = FakeTerminal.prototype.inspectForeground.bind(terminal) + const inspection = Promise.withResolvers<{ processGroupId: number; inputWaiting: boolean }>() + terminal.inspectForeground = async () => await inspection.promise + const signalGate = Promise.withResolvers() + terminal.signalForeground = async (signal) => { + await signalGate.promise + inspector.signalGroup(456, signal) + return 456 + } + const controller = new AbortController() + const operation = session.startSend({ text: 'must stay owned', submit: true, signal: controller.signal }) + controller.abort() + inspection.reject(new Error('transient inspection failure')) + await Promise.resolve() + await Promise.resolve() + + // The slot stays reserved while the cancellation's foreground signal is in flight. + expect(() => session.startSend({ text: 'successor', submit: true })).toThrow('active send') + terminal.inspectForeground = async () => ({ processGroupId: 456, inputWaiting: false }) + signalGate.resolve(undefined) + await vi.advanceTimersByTimeAsync(0) + expect(inspector.groups).toContainEqual([456, 'SIGINT']) + + terminal.emitData('\x1b]133;D;130\x07dsh> ') + await vi.advanceTimersByTimeAsync(10) + await operation.done + }) + + it('retains a canceled send until asynchronous foreground signalling settles', async () => { + vi.useFakeTimers() + const terminal = new FakeTerminal() + const inspector = new FakeInspector() + const session = makeSession(terminal, inspector, config()) + await initialize(session, terminal) + + const signalGate = Promise.withResolvers() + terminal.signalForeground = async (signal) => { + await signalGate.promise + const foreground = await terminal.inspectForeground() + if (foreground === undefined) throw new Error('cannot resolve foreground') + inspector.signalGroup(foreground.processGroupId, signal) + return foreground.processGroupId + } + const operation = session.startSend({ text: 'first', submit: true }) + await Promise.resolve() + await Promise.resolve() + expect(operation.cancel()).toBe(true) + + terminal.emitData('\x1b]133;D;130\x07dsh> ') + await vi.advanceTimersByTimeAsync(100) + expect((await operation.done).waitReason).toBe('timeout') + expect(() => session.startSend({ text: 'successor', submit: true })).toThrow('active send') + signalGate.resolve(undefined) + await vi.advanceTimersByTimeAsync(0) + expect(inspector.groups).toContainEqual([456, 'SIGINT']) + expect(inspector.groups).not.toContainEqual([789, 'SIGINT']) + }) + + it('does not let an in-flight readiness inspection release a canceled send before signalling settles', async () => { + vi.useFakeTimers() + const terminal = new FakeTerminal() + const inspector = new FakeInspector() + const session = makeSession(terminal, inspector, config()) + await initialize(session, terminal) + + const readiness = Promise.withResolvers<{ processGroupId: number; inputWaiting: boolean }>() + let inspections = 0 + terminal.inspectForeground = async () => { + inspections += 1 + if (inspections === 1) return { processGroupId: 456, inputWaiting: false } + if (inspections === 2) return await readiness.promise + return { processGroupId: 456, inputWaiting: true } + } + const signalling = Promise.withResolvers() + const signalled = Promise.withResolvers() + terminal.signalForeground = async (signal) => { + await signalling.promise + const foreground = await terminal.inspectForeground() + if (foreground === undefined) throw new Error('cannot resolve foreground') + inspector.signalGroup(foreground.processGroupId, signal) + signalled.resolve(foreground.processGroupId) + return foreground.processGroupId + } + + const operation = session.startSend({ text: 'first', submit: true }) + await Promise.resolve() + await Promise.resolve() + await vi.advanceTimersByTimeAsync(10) + expect(inspections).toBe(2) + await vi.advanceTimersByTimeAsync(50) + expect(operation.cancel()).toBe(true) + let settled = false + void operation.done.then(() => { settled = true }) + + readiness.resolve({ processGroupId: 456, inputWaiting: true }) + await Promise.resolve() + await Promise.resolve() + expect(settled).toBe(false) + expect(() => session.startSend({ text: 'successor', submit: true })).toThrow(PtyError) + + signalling.resolve(undefined) + expect(await signalled.promise).toBe(456) + expect(inspector.groups).toContainEqual([456, 'SIGINT']) + await session.close('test complete') + expect((await operation.done).waitReason).toBe('session_exit') + }) + + it('does not let an in-flight readiness failure release a canceled send before signalling settles', async () => { + vi.useFakeTimers() + const terminal = new FakeTerminal() + const inspector = new FakeInspector() + const session = makeSession(terminal, inspector, config()) + await initialize(session, terminal) + + const readiness = Promise.withResolvers() + let inspections = 0 + terminal.inspectForeground = async () => { + inspections += 1 + if (inspections === 1) return { processGroupId: 456, inputWaiting: false } + if (inspections === 2) return await readiness.promise + return { processGroupId: 456, inputWaiting: true } + } + const signalling = Promise.withResolvers() + const signalled = Promise.withResolvers() + terminal.signalForeground = async (signal) => { + await signalling.promise + const foreground = await terminal.inspectForeground() + if (foreground === undefined) throw new Error('cannot resolve foreground') + inspector.signalGroup(foreground.processGroupId, signal) + signalled.resolve(foreground.processGroupId) + return foreground.processGroupId + } + + const operation = session.startSend({ text: 'first', submit: true }) + await Promise.resolve() + await Promise.resolve() + await vi.advanceTimersByTimeAsync(10) + expect(inspections).toBe(2) + expect(operation.cancel()).toBe(true) + let settled = false + void operation.done.then(() => { settled = true }) + + readiness.reject(new Error('inspection failed during cancellation')) + await Promise.resolve() + await Promise.resolve() + expect(settled).toBe(false) + expect(() => session.startSend({ text: 'successor', submit: true })).toThrow(PtyError) + + signalling.resolve(undefined) + expect(await signalled.promise).toBe(456) + expect(inspector.groups).toContainEqual([456, 'SIGINT']) + await session.close('test complete') + expect((await operation.done).waitReason).toBe('session_exit') + }) + + it('signals only after an in-flight provider write lands', async () => { + vi.useFakeTimers() + const terminal = new FakeTerminal() + const inspector = new FakeInspector() + const session = makeSession(terminal, inspector, config()) + await initialize(session, terminal) + + const writeGate = Promise.withResolvers() + terminal.write = async () => { await writeGate.promise } + const operation = session.startSend({ text: 'must be interrupted', submit: true }) + await Promise.resolve() + await Promise.resolve() + expect(operation.cancel()).toBe(true) + await vi.advanceTimersByTimeAsync(20) + expect(inspector.groups).toEqual([]) + + writeGate.resolve(undefined) + await vi.advanceTimersByTimeAsync(0) + expect(inspector.groups).toContainEqual([456, 'SIGINT']) + terminal.emitData('\x1b]133;D;130\x07dsh> ') + await vi.advanceTimersByTimeAsync(10) + await operation.done + }) + + it('does not signal when a cancelled provider write rejects', async () => { + vi.useFakeTimers() + const terminal = new FakeTerminal() + const inspector = new FakeInspector() + const session = makeSession(terminal, inspector, config()) + await initialize(session, terminal) + + const writeGate = Promise.withResolvers() + terminal.write = async () => { await writeGate.promise } + const operation = session.startSend({ text: 'rejected write', submit: true }) + await Promise.resolve() + await Promise.resolve() + expect(operation.cancel()).toBe(true) + + const rejected = expect(operation.done).rejects.toThrow('write failed after cancellation') + writeGate.reject(new Error('write failed after cancellation')) + await rejected + expect(inspector.groups).toEqual([]) + + const next = session.startSend({ text: '', submit: false }) + await vi.advanceTimersByTimeAsync(100) + expect((await next.done).waitReason).toBe('inferred_idle') + }) + + it('releases a timed-out cancellation after the provider write and signal settle', async () => { + vi.useFakeTimers() + const terminal = new FakeTerminal() + const inspector = new FakeInspector() + const session = makeSession(terminal, inspector, config()) + await initialize(session, terminal) + + const writeGate = Promise.withResolvers() + terminal.write = async () => { await writeGate.promise } + const operation = session.startSend({ text: 'slow cancelled write', submit: true }) + await Promise.resolve() + await Promise.resolve() + expect(operation.cancel()).toBe(true) + await vi.advanceTimersByTimeAsync(100) + + expect((await operation.done).waitReason).toBe('timeout') + expect(() => session.startSend({ text: 'must wait', submit: true })).toThrow(expect.objectContaining({ + code: 'SEND_ACTIVE', + })) + + writeGate.resolve(undefined) + await vi.advanceTimersByTimeAsync(0) + expect(inspector.groups).toContainEqual([456, 'SIGINT']) + + const next = session.startSend({ text: '', submit: false }) + await vi.advanceTimersByTimeAsync(100) + expect((await next.done).waitReason).toBe('inferred_idle') + }) + + it('retains the absolute timeout after cancellation while output stays active', async () => { + vi.useFakeTimers() + const terminal = new FakeTerminal() + const inspector = new FakeInspector() + const session = makeSession(terminal, inspector, config()) + await initialize(session, terminal) + + const operation = session.startSend({ text: 'ignore-sigint-and-write', submit: true }) + await Promise.resolve() + await Promise.resolve() + expect(operation.cancel()).toBe(true) + for (let elapsed = 20; elapsed <= 100; elapsed += 20) { + terminal.emitData('.') + await vi.advanceTimersByTimeAsync(20) + } + + expect(await operation.done).toMatchObject({ waitReason: 'timeout' }) + }) + + it('does not resume cancellation polling after the terminal exits during signalling', async () => { + vi.useFakeTimers() + const terminal = new FakeTerminal() + const inspector = new FakeInspector() + const session = makeSession(terminal, inspector, config()) + await initialize(session, terminal) + + const signalGate = Promise.withResolvers() + terminal.signalForeground = async () => { + await signalGate.promise + return 456 + } + const operation = session.startSend({ text: 'first', submit: true }) + await Promise.resolve() + await Promise.resolve() + expect(operation.cancel()).toBe(true) + terminal.emitExit(0) + await expect(operation.done).resolves.toMatchObject({ waitReason: 'session_exit' }) + + signalGate.resolve(undefined) + await vi.advanceTimersByTimeAsync(10) + expect(session.status()).toEqual({ kind: 'exited', exitCode: 0, signal: null }) + }) + + it('retains send ownership after timeout until an asynchronous provider write settles', async () => { + vi.useFakeTimers() + const terminal = new FakeTerminal() + const inspector = new FakeInspector() + const session = makeSession(terminal, inspector, config()) + await initialize(session, terminal) + + const writeGate = Promise.withResolvers() + terminal.write = async () => { await writeGate.promise } + const operation = session.startSend({ text: 'slow write', submit: true }) + await Promise.resolve() + await Promise.resolve() + await vi.advanceTimersByTimeAsync(100) + + expect((await operation.done).waitReason).toBe('timeout') + expect(() => session.startSend({ text: 'must wait', submit: true })).toThrow('active send') + + writeGate.resolve(undefined) + await Promise.resolve() + await Promise.resolve() + const rejectedWrite = Promise.withResolvers() + terminal.write = async () => { await rejectedWrite.promise } + const rejected = session.startSend({ text: 'late rejection', submit: true }) + await vi.advanceTimersByTimeAsync(100) + expect((await rejected.done).waitReason).toBe('timeout') + rejectedWrite.reject(new Error('write failed after timeout')) + await Promise.resolve() + await Promise.resolve() + + const next = session.startSend({ text: '', submit: false }) + await vi.advanceTimersByTimeAsync(100) + expect((await next.done).waitReason).toBe('inferred_idle') + }) + + it('retains send ownership when cancellation signalling fails during an asynchronous write', async () => { + vi.useFakeTimers() + const terminal = new FakeTerminal() + const inspector = new FakeInspector() + const session = makeSession(terminal, inspector, config()) + await initialize(session, terminal) + + const writeGate = Promise.withResolvers() + terminal.write = async () => { await writeGate.promise } + let signalCalls = 0 + terminal.signalForeground = async () => { + signalCalls += 1 + throw new Error('interrupt failed') + } + const operation = session.startSend({ text: 'slow write', submit: true }) + await Promise.resolve() + await Promise.resolve() + expect(operation.cancel()).toBe(true) + await vi.advanceTimersByTimeAsync(20) + expect(signalCalls).toBe(0) + expect(() => session.startSend({ text: 'must wait', submit: true })).toThrow('active send') + + writeGate.resolve(undefined) + await expect(operation.done).rejects.toThrow('interrupt failed') + expect(signalCalls).toBe(1) + expect(session.status()).toEqual({ kind: 'exited', exitCode: null, signal: null }) + expect(() => session.startSend({ text: '', submit: false })).toThrow('has exited') + }) + it('handles startup exit, unknown exit signals, cancel-write failure, and stale polls', async () => { vi.useFakeTimers() const startupTerminal = new FakeTerminal() - const startup = new LocalPtySession(startupTerminal.asPty(), new FakeInspector(), config()) + const startup = new LocalPtySession(startupTerminal, config()) const initializing = startup.initialize(new AbortController().signal) startupTerminal.emitExit(1) await expect(initializing).rejects.toThrow('exited during startup') expect(startup.status()).toEqual({ kind: 'exited', exitCode: 1, signal: null }) const terminal = new FakeTerminal() - const session = new LocalPtySession(terminal.asPty(), new FakeInspector(), config()) + const session = new LocalPtySession(terminal, config()) await initialize(session, terminal) const operation = session.startSend({ text: '', submit: false }) const operationInternal = operation as unknown as { @@ -247,11 +717,17 @@ describe('LocalPtySession readiness and output', () => { const sessionInternal = session as unknown as { pollReadiness(operation: PtySendOperation): void interrupt(operation: PtySendOperation): void + schedulePoll(operation: PtySendOperation): void + polling: boolean statusValue: PtySessionStatus appendOutput(text: string): void } sessionInternal.appendOutput('') sessionInternal.pollReadiness({} as PtySendOperation) + sessionInternal.schedulePoll({} as PtySendOperation) + sessionInternal.polling = true + sessionInternal.schedulePoll(operation) + sessionInternal.polling = false sessionInternal.interrupt({} as PtySendOperation) sessionInternal.statusValue = { kind: 'exited', exitCode: 2, signal: null } sessionInternal.pollReadiness(operation) @@ -259,13 +735,15 @@ describe('LocalPtySession readiness and output', () => { operationInternal.settle('timeout', { kind: 'running' }, false) const unknownTerminal = new FakeTerminal() - const unknown = new LocalPtySession(unknownTerminal.asPty(), new FakeInspector(), config()) + const unknown = new LocalPtySession(unknownTerminal, config()) unknownTerminal.emitExit(1, 999) - expect(unknown.status()).toEqual({ kind: 'exited', exitCode: 1, signal: null }) + await vi.waitFor(() => { + expect(unknown.status()).toEqual({ kind: 'exited', exitCode: null, signal: null }) + }) const cancelTerminal = new FakeTerminal() const cancelInspector = new FakeInspector() - const cancel = new LocalPtySession(cancelTerminal.asPty(), cancelInspector, config()) + const cancel = makeSession(cancelTerminal, cancelInspector, config()) await initialize(cancel, cancelTerminal) const cancellable = cancel.startSend({ text: '', submit: false }) cancelInspector.throwGroup = true @@ -275,7 +753,7 @@ describe('LocalPtySession readiness and output', () => { const missingGroupTerminal = new FakeTerminal() const missingGroupInspector = new FakeInspector() - const missingGroup = new LocalPtySession(missingGroupTerminal.asPty(), missingGroupInspector, config()) + const missingGroup = makeSession(missingGroupTerminal, missingGroupInspector, config()) await initialize(missingGroup, missingGroupTerminal) missingGroupInspector.pgid = undefined const unresolved = missingGroup.startSend({ text: '', submit: false }) @@ -286,7 +764,7 @@ describe('LocalPtySession readiness and output', () => { it('does not treat zero-output startup silence as readiness and fails on startup timeout', async () => { vi.useFakeTimers() const terminal = new FakeTerminal() - const session = new LocalPtySession(terminal.asPty(), new FakeInspector(), config()) + const session = new LocalPtySession(terminal, config()) let settled = false const initializing = session.initialize().then(() => { settled = true }) await vi.advanceTimersByTimeAsync(60) @@ -296,7 +774,7 @@ describe('LocalPtySession readiness and output', () => { await initializing const timeoutTerminal = new FakeTerminal() - const timeout = new LocalPtySession(timeoutTerminal.asPty(), new FakeInspector(), config()) + const timeout = new LocalPtySession(timeoutTerminal, config()) const timedOut = expect(timeout.initialize()).rejects.toThrow('startup timeout') await vi.advanceTimersByTimeAsync(100) await timedOut @@ -306,7 +784,7 @@ describe('LocalPtySession readiness and output', () => { const terminal = new FakeTerminal() const inspector = new FakeInspector() inspector.pgid = undefined - const session = new LocalPtySession(terminal.asPty(), inspector, config()) + const session = makeSession(terminal, inspector, config()) const controller = new AbortController() const reason = new Error('startup cancelled') @@ -320,7 +798,7 @@ describe('LocalPtySession readiness and output', () => { it('waits for printable prompt text when the startup marker is split from PS1', async () => { vi.useFakeTimers() const terminal = new FakeTerminal() - const session = new LocalPtySession(terminal.asPty(), new FakeInspector(), config()) + const session = new LocalPtySession(terminal, config()) let settled = false const initializing = session.initialize().then(() => { settled = true }) @@ -334,16 +812,39 @@ describe('LocalPtySession readiness and output', () => { expect(session.motd).toBe('dsh> ') }) + it('does not attribute a delayed prior prompt to the current send', async () => { + vi.useFakeTimers() + const terminal = new FakeTerminal() + const session = new LocalPtySession(terminal, config({ idleSilenceMs: 100, timeoutMs: 200 })) + await initialize(session, terminal) + + const operation = session.startSend({ text: "printf 'PID=%s\\n' \"$!\"", submit: true }) + let settled = false + void operation.done.then(() => { settled = true }) + await Promise.resolve() + await Promise.resolve() + + terminal.emitData('\x1b]133;D;0\x07dsh> printf \'PID=%s\\n\' "$!"\r\n') + await vi.advanceTimersByTimeAsync(20) + expect(settled).toBe(false) + + terminal.emitData('PID=123\r\n\x1b]133;D;0\x07dsh> ') + await vi.advanceTimersByTimeAsync(10) + expect(await operation.done).toMatchObject({ waitReason: 'stdin_read' }) + }) + it('retains a prompt marker until the startup shell regains the foreground group', async () => { vi.useFakeTimers() const terminal = new FakeTerminal() const inspector = new FakeInspector() - const session = new LocalPtySession(terminal.asPty(), inspector, config()) + const session = makeSession(terminal, inspector, config()) await initialize(session, terminal) const operation = session.startSend({ text: 'run', submit: true }) let settled = false void operation.done.then(() => { settled = true }) + await Promise.resolve() + await Promise.resolve() inspector.pgid = 789 terminal.emitData('\x1b]133;D;0\x07dsh> ') await vi.advanceTimersByTimeAsync(50) @@ -359,12 +860,14 @@ describe('LocalPtySession readiness and output', () => { vi.useFakeTimers() const terminal = new FakeTerminal() const inspector = new FakeInspector() - const session = new LocalPtySession(terminal.asPty(), inspector, config({ handoffGraceMs: 40 })) + const session = makeSession(terminal, inspector, config({ handoffGraceMs: 40 })) await initialize(session, terminal) const operation = session.startSend({ text: 'run', submit: true }) let settled = false void operation.done.then(() => { settled = true }) + await Promise.resolve() + await Promise.resolve() inspector.pgid = 789 terminal.emitData('\x1b]133;D;0\x07dsh> ') // One poll past the silence bound would already have settled inferred_idle. @@ -380,7 +883,7 @@ describe('LocalPtySession readiness and output', () => { vi.useFakeTimers() const terminal = new FakeTerminal() const inspector = new FakeInspector() - const session = new LocalPtySession(terminal.asPty(), inspector, config()) + const session = makeSession(terminal, inspector, config()) await initialize(session, terminal) const operation = session.startSend({ text: 'bash -i', submit: true }) @@ -390,6 +893,238 @@ describe('LocalPtySession readiness and output', () => { expect((await operation.done).waitReason).toBe('inferred_idle') }) + + it('contains terminal transport failures and preserves the first failure', async () => { + const terminal = new FakeTerminal() + terminal.terminateError = new Error('cleanup after transport failure') + const session = new LocalPtySession(terminal, config()) + const operation = session.startSend({ text: '', submit: false }) + terminal.output.emit('data', 'plain text') + terminal.emitError(new Error('output transport failed')) + ;(session as unknown as { onTransportFailure(error: unknown): void }) + .onTransportFailure(new Error('later failure')) + await expect(operation.done).rejects.toThrow('output transport failed') + expect(session.status()).toEqual({ kind: 'exited', exitCode: null, signal: null }) + terminal.terminateError = undefined + await expect(session.close('transport')).rejects.toThrow('output transport failed') + + const rejectedTerminal = new FakeTerminal() + const rejected = new LocalPtySession(rejectedTerminal, config()) + const rejectedOperation = rejected.startSend({ text: '', submit: false }) + rejectedTerminal.emitFailure('raw transport failure') + await expect(rejectedOperation.done).rejects.toThrow('raw transport failure') + }) + + it('replaces invalid UTF-8 terminal output', async () => { + const chunkTerminal = new FakeTerminal() + const chunkSession = new LocalPtySession(chunkTerminal, config()) + const chunkOperation = chunkSession.startSend({ text: '', submit: false }) + chunkTerminal.emitBytes(Uint8Array.from([0xff])) + expect(chunkOperation.readOutput()).toEqual({ delta: '�', truncated: false }) + chunkTerminal.emitExit() + await chunkOperation.done + + const endTerminal = new FakeTerminal() + const endSession = new LocalPtySession(endTerminal, config()) + const endOperation = endSession.startSend({ text: '', submit: false }) + endTerminal.emitBytes(Uint8Array.from([0xe2])) + endTerminal.emitExit() + expect((await endOperation.done).viewport).toBe('�') + }) + + it('contains readiness inspection failure and a stale inspection result', async () => { + vi.useFakeTimers() + const failedTerminal = new FakeTerminal() + const failedSession = new LocalPtySession(failedTerminal, config()) + const failedOperation = failedSession.startSend({ text: '', submit: false }) + await Promise.resolve() + await Promise.resolve() + failedTerminal.inspectForeground = async () => { throw new Error('inspect failed') } + const failedInternal = failedSession as unknown as { + pollReadiness(operation: PtySendOperation): Promise + } + await failedInternal.pollReadiness(failedOperation) + await expect(failedOperation.done).rejects.toThrow('inspect failed') + + const staleTerminal = new FakeTerminal() + const staleSession = new LocalPtySession(staleTerminal, config()) + const staleOperation = staleSession.startSend({ text: '', submit: false }) + await Promise.resolve() + await Promise.resolve() + const gate = Promise.withResolvers extends Promise ? T : never>() + staleTerminal.inspectForeground = async () => await gate.promise + const staleInternal = staleSession as unknown as { + active: PtySendOperation | undefined + pollReadiness(operation: PtySendOperation): Promise + } + const polling = staleInternal.pollReadiness(staleOperation) + staleInternal.active = undefined + gate.resolve({ processGroupId: 456, inputWaiting: false }) + await polling + ;(staleOperation as unknown as { + settle(reason: 'timeout', status: PtySessionStatus, inherited: boolean): void + }).settle('timeout', { kind: 'running' }, false) + }) + + it('reschedules readiness for a new send after a stale remote inspection releases the poll slot', async () => { + vi.useFakeTimers() + const terminal = new FakeTerminal() + const inspector = new FakeInspector() + const session = makeSession(terminal, inspector, config()) + await initialize(session, terminal) + + const old = session.startSend({ text: '', submit: false }) + await Promise.resolve() + await Promise.resolve() + const inspection = Promise.withResolvers<{ processGroupId: number; inputWaiting: boolean }>() + let block = true + terminal.inspectForeground = async () => block + ? await inspection.promise + : { processGroupId: 456, inputWaiting: false } + const internals = session as unknown as { + pollReadiness(operation: PtySendOperation): Promise + settleActive(reason: 'timeout'): void + } + const stalePoll = internals.pollReadiness(old) + internals.settleActive('timeout') + await old.done + + block = false + const current = session.startSend({ text: '', submit: false }) + terminal.emitData('\x1b]133;D;0\x07dsh> ') + await Promise.resolve() + await Promise.resolve() + inspection.resolve({ processGroupId: 456, inputWaiting: false }) + await stalePoll + await vi.advanceTimersByTimeAsync(10) + + expect((await current.done).waitReason).toBe('stdin_read') + }) + + it('does not poll a successor before its own pre-write inspection and write complete', async () => { + vi.useFakeTimers() + const terminal = new FakeTerminal() + const inspector = new FakeInspector() + const session = makeSession(terminal, inspector, config()) + await initialize(session, terminal) + + const old = session.startSend({ text: '', submit: false }) + await Promise.resolve() + await Promise.resolve() + const staleInspection = Promise.withResolvers<{ processGroupId: number; inputWaiting: boolean }>() + const successorInspection = Promise.withResolvers<{ processGroupId: number; inputWaiting: boolean }>() + let inspectCalls = 0 + terminal.inspectForeground = async () => { + inspectCalls += 1 + return inspectCalls === 1 ? await staleInspection.promise : await successorInspection.promise + } + const internals = session as unknown as { + pollReadiness(operation: PtySendOperation): Promise + settleActive(reason: 'timeout'): void + } + const stalePoll = internals.pollReadiness(old) + internals.settleActive('timeout') + await old.done + + const current = session.startSend({ text: 'successor', submit: true }) + await Promise.resolve() + await Promise.resolve() + staleInspection.resolve({ processGroupId: 456, inputWaiting: false }) + await stalePoll + await vi.advanceTimersByTimeAsync(10) + expect(inspectCalls).toBe(2) + expect(terminal.writes).toEqual([]) + + successorInspection.resolve({ processGroupId: 456, inputWaiting: false }) + await Promise.resolve() + await Promise.resolve() + terminal.emitData('\x1b]133;D;0\x07dsh> ') + await vi.advanceTimersByTimeAsync(10) + expect(terminal.writes).toEqual(['successor\r']) + expect((await current.done).waitReason).toBe('stdin_read') + }) + + it('contains stale timer, write, inspection, and interrupt continuations', async () => { + vi.useFakeTimers() + const settle = (operation: PtySendOperation): void => { + ;(operation as unknown as { + settle(reason: 'timeout', status: PtySessionStatus, inherited: boolean): void + }).settle('timeout', { kind: 'running' }, false) + } + + const deadlineTerminal = new FakeTerminal() + const deadlineSession = new LocalPtySession(deadlineTerminal, config()) + const deadlineOperation = deadlineSession.startSend({ text: '', submit: false }) + ;(deadlineSession as unknown as { active: PtySendOperation | undefined }).active = undefined + await vi.advanceTimersByTimeAsync(100) + settle(deadlineOperation) + + const writeTerminal = new FakeTerminal() + const writeGate = Promise.withResolvers() + writeTerminal.write = async () => { await writeGate.promise } + const writeSession = new LocalPtySession(writeTerminal, config()) + const writeOperation = writeSession.startSend({ text: 'x', submit: false }) + await Promise.resolve() + await Promise.resolve() + ;(writeSession as unknown as { closing: boolean }).closing = true + writeGate.resolve(undefined) + await Promise.resolve() + await Promise.resolve() + settle(writeOperation) + + const beginTerminal = new FakeTerminal() + const beginGate = Promise.withResolvers() + beginTerminal.inspectForeground = async () => await beginGate.promise + const beginSession = new LocalPtySession(beginTerminal, config()) + const beginOperation = beginSession.startSend({ text: '', submit: false }) + ;(beginSession as unknown as { active: PtySendOperation | undefined }).active = undefined + beginGate.reject(new Error('stale begin failure')) + await Promise.resolve() + await Promise.resolve() + settle(beginOperation) + + const scheduledTerminal = new FakeTerminal() + const scheduledSession = new LocalPtySession(scheduledTerminal, config()) + const scheduledOperation = scheduledSession.startSend({ text: '', submit: false }) + await Promise.resolve() + await Promise.resolve() + const scheduledInternal = scheduledSession as unknown as { + schedulePoll(operation: PtySendOperation, delayMs?: number): void + settleActive(reason: 'timeout'): void + } + scheduledInternal.schedulePoll(scheduledOperation, 5) + scheduledInternal.settleActive('timeout') + await scheduledOperation.done + + const pollTerminal = new FakeTerminal() + const pollSession = new LocalPtySession(pollTerminal, config()) + const pollOperation = pollSession.startSend({ text: '', submit: false }) + await Promise.resolve() + await Promise.resolve() + const pollGate = Promise.withResolvers() + pollTerminal.inspectForeground = async () => await pollGate.promise + const pollInternal = pollSession as unknown as { + active: PtySendOperation | undefined + pollReadiness(operation: PtySendOperation): Promise + } + const stalePoll = pollInternal.pollReadiness(pollOperation) + pollInternal.active = undefined + pollGate.reject(new Error('stale poll failure')) + await stalePoll + settle(pollOperation) + + const interruptTerminal = new FakeTerminal() + const interruptGate = Promise.withResolvers() + interruptTerminal.signalForeground = async () => await interruptGate.promise + const interruptSession = new LocalPtySession(interruptTerminal, config()) + const interruptOperation = interruptSession.startSend({ text: '', submit: false }) + expect(interruptOperation.cancel()).toBe(true) + ;(interruptSession as unknown as { active: PtySendOperation | undefined }).active = undefined + interruptGate.reject(new Error('stale interrupt failure')) + await Promise.resolve() + await Promise.resolve() + settle(interruptOperation) + }) }) describe('LocalPtySession bounds, signals, and teardown', () => { @@ -397,8 +1132,7 @@ describe('LocalPtySession bounds, signals, and teardown', () => { vi.useFakeTimers() const terminal = new FakeTerminal() const session = new LocalPtySession( - terminal.asPty(), - new FakeInspector(), + terminal, config({ scrollbackLines: 3, scrollbackMaxBytes: 12, maxReadBytes: 6 }), ) expect(session.read({})).toMatchObject({ text: '' }) @@ -415,7 +1149,7 @@ describe('LocalPtySession bounds, signals, and teardown', () => { expect(() => session.read({ count: 0 })).toThrow('count') const tinyTerminal = new FakeTerminal() - const tiny = new LocalPtySession(tinyTerminal.asPty(), new FakeInspector(), config({ maxReadBytes: 1 })) + const tiny = new LocalPtySession(tinyTerminal, config({ maxReadBytes: 1 })) await initialize(tiny, tinyTerminal) const tinyOperation = tiny.startSend({ text: '', submit: false }) tinyTerminal.emitData('一') @@ -427,32 +1161,47 @@ describe('LocalPtySession bounds, signals, and teardown', () => { it('signals verified groups and refuses unresolved or shell-targeted hard kills', async () => { const terminal = new FakeTerminal() const inspector = new FakeInspector() - const session = new LocalPtySession(terminal.asPty(), inspector, config()) + const session = makeSession(terminal, inspector, config()) expect(await session.signal('SIGINT')).toEqual({ delivered: true, targetPgid: 456 }) inspector.pgid = terminal.pid - await expect(session.signal('SIGKILL')).rejects.toThrow('use terminal_close') + await expect(session.signal('SIGKILL')).rejects.toThrow('terminate the terminal session') inspector.pgid = undefined await expect(session.signal('SIGTERM')).rejects.toThrow('cannot resolve') }) - it('closes idempotently, contains signal races, and reports survivors', async () => { + it('closes idempotently and rejects new signals', async () => { const terminal = new FakeTerminal() - const inspector = new FakeInspector() - inspector.members = [{ pid: 123, started: 'a' }] - inspector.alive.add(123) - inspector.throwProcess = true terminal.throwKill = true - const session = new LocalPtySession(terminal.asPty(), inspector, config({ disposeGraceMs: 1 })) + const session = new LocalPtySession(terminal, config({ disposeGraceMs: 1 })) const closing = session.close('test') expect(session.close('other')).toBe(closing) - await expect(closing).rejects.toThrow('surviving pids: 123') + await expect(closing).rejects.toThrow('PTY cleanup failed (test)') expect(() => session.startSend({ text: '', submit: false })).toThrow('closing') + await expect(session.signal('SIGTERM')).rejects.toThrow('closing') + }) + + it('reports cleanup failure without waiting for top-level exit and permits retry', async () => { + const terminal = new FakeTerminal() + terminal.autoExitOnKill = false + terminal.terminateError = new Error('terminal cleanup failed; surviving pids: 456') + const session = new LocalPtySession(terminal, config()) + + await expect(session.close('survivor')).rejects.toMatchObject({ + message: 'PTY cleanup failed (survivor)', + cause: terminal.terminateError, + }) + expect(terminal.kills).toEqual([]) + + terminal.terminateError = undefined + terminal.autoExitOnKill = true + await expect(session.close('retry')).resolves.toBeUndefined() + expect(terminal.kills).toEqual(['SIGTERM']) }) it('settles an active send as session_exit when closed mid-operation', async () => { vi.useFakeTimers() const terminal = new FakeTerminal() - const session = new LocalPtySession(terminal.asPty(), new FakeInspector(), config({ disposeGraceMs: 50 })) + const session = new LocalPtySession(terminal, config({ disposeGraceMs: 50 })) await initialize(session, terminal) const operation = session.startSend({ text: 'run', submit: true }) // The shell returns to its prompt while the send is active; a running @@ -467,94 +1216,80 @@ describe('LocalPtySession bounds, signals, and teardown', () => { await closing }) - it('keeps the shell alive until SIGKILL recipients leave the process table', async () => { + it('settles a closing send when provider termination cancels inspection', async () => { vi.useFakeTimers() const terminal = new FakeTerminal() - const inspector = new FakeInspector() - inspector.members = [{ pid: 124, started: 'child' }] - inspector.alive.add(124) - inspector.removeOnSignal = false - const session = new LocalPtySession(terminal.asPty(), inspector, config({ disposeGraceMs: 20 })) + const session = new LocalPtySession(terminal, config()) + await initialize(session, terminal) + const write = Promise.withResolvers() + terminal.write = async () => { await write.promise } + const writeOperation = session.startSend({ text: 'pending write', submit: true }) + await Promise.resolve() + await Promise.resolve() + await session.close('pending write') + expect((await writeOperation.done).waitReason).toBe('session_exit') + // The rejection lands after close released the send; it must stay contained. + write.reject(new Error('write rejected during close')) + await Promise.resolve() + await Promise.resolve() + }) + it('settles a closing send when provider termination cancels a pending inspection', async () => { + vi.useFakeTimers() + const terminal = new FakeTerminal() + const session = new LocalPtySession(terminal, config()) + await initialize(session, terminal) + const inspection = Promise.withResolvers<{ processGroupId: number; inputWaiting: boolean }>() + terminal.inspectForeground = async () => await inspection.promise + const terminate = terminal.terminate.bind(terminal) + terminal.terminate = async () => { + inspection.reject(new Error('terminal terminated')) + await terminate() + } + const operation = session.startSend({ text: 'pending inspection', submit: true }) + await Promise.resolve() + + await session.close('pending inspection') + + expect((await operation.done).waitReason).toBe('session_exit') + expect(terminal.writes).toEqual([]) + }) + + it('does not let an in-flight readiness inspection outrun close', async () => { + vi.useFakeTimers() + const terminal = new FakeTerminal() + const session = new LocalPtySession(terminal, config()) + await initialize(session, terminal) + const inspection = Promise.withResolvers<{ processGroupId: number; inputWaiting: boolean }>() + const originalInspect = terminal.inspectForeground.bind(terminal) + let inspections = 0 + terminal.inspectForeground = async () => { + inspections += 1 + return inspections === 1 ? await originalInspect() : await inspection.promise + } + const operation = session.startSend({ text: 'pending readiness', submit: true }) + await Promise.resolve() + await Promise.resolve() + terminal.emitData('\x1b]133;D;0\x07dsh> ') + await vi.advanceTimersByTimeAsync(10) + expect(inspections).toBe(2) + + const termination = Promise.withResolvers() + terminal.terminate = async () => { + await termination.promise + terminal.emitExit(0, 15) + } + const closing = session.close('in-flight readiness') let settled = false - const closing = session.close('test').then(() => { settled = true }) - await vi.advanceTimersByTimeAsync(20) - expect(inspector.processes).toContainEqual([124, 'SIGKILL']) - expect(terminal.kills).toEqual([]) + void operation.done.then(() => { settled = true }) + inspection.resolve({ processGroupId: 456, inputWaiting: true }) + await Promise.resolve() + await Promise.resolve() expect(settled).toBe(false) - inspector.alive.delete(124) - await vi.advanceTimersByTimeAsync(20) + termination.resolve(undefined) await closing - expect(terminal.kills).toEqual(['SIGTERM']) - expect(settled).toBe(true) + expect((await operation.done).waitReason).toBe('session_exit') }) - it('rescans for descendants forked during TERM before stopping the shell', async () => { - const terminal = new FakeTerminal() - const inspector = new FakeInspector() - let reads = 0 - inspector.processTree = () => { - reads += 1 - if (reads === 1) { - inspector.alive.add(124) - return [{ pid: 124, started: 'first' }] - } - if (reads === 2) { - inspector.alive.add(125) - return [{ pid: 125, started: 'late' }] - } - return [] - } - const session = new LocalPtySession(terminal.asPty(), inspector, config()) - - await session.close('test') - - expect(inspector.processes).toEqual([[124, 'SIGTERM'], [125, 'SIGKILL']]) - expect(terminal.kills).toEqual(['SIGTERM']) - }) - - it('retains captured survivors that are reparented out of the teardown rescan', async () => { - vi.useFakeTimers() - const terminal = new FakeTerminal() - const inspector = new FakeInspector() - const captured = { pid: 124, started: 'captured' } - let reads = 0 - inspector.alive.add(captured.pid) - inspector.processTree = () => reads++ === 0 ? [captured] : [] - inspector.signalProcess = (identity, signal) => { - inspector.processes.push([identity.pid, signal]) - if (signal === 'SIGKILL') inspector.alive.delete(identity.pid) - } - const session = new LocalPtySession(terminal.asPty(), inspector, config({ disposeGraceMs: 20 })) - - const closing = session.close('test') - await vi.advanceTimersByTimeAsync(25) - await closing - - expect(inspector.processes).toEqual([[124, 'SIGTERM'], [124, 'SIGKILL']]) - expect(terminal.kills).toEqual(['SIGTERM']) - }) - - it('allows teardown to retry after a descendant-survivor failure', async () => { - vi.useFakeTimers() - const terminal = new FakeTerminal() - const inspector = new FakeInspector() - inspector.members = [{ pid: 124, started: 'child' }] - inspector.alive.add(124) - inspector.removeOnSignal = false - const session = new LocalPtySession(terminal.asPty(), inspector, config({ disposeGraceMs: 10 })) - - const first = session.close('first') - const rejected = expect(first).rejects.toThrow('surviving pids: 124') - await vi.advanceTimersByTimeAsync(25) - await rejected - expect(terminal.kills).toEqual([]) - - inspector.alive.delete(124) - const second = session.close('retry') - expect(second).not.toBe(first) - await second - expect(terminal.kills).toEqual(['SIGTERM']) - }) }) diff --git a/packages/pty/pty/src/types.ts b/packages/pty/pty/src/types.ts index a4985ead93..0317bb9d43 100644 --- a/packages/pty/pty/src/types.ts +++ b/packages/pty/pty/src/types.ts @@ -28,7 +28,11 @@ export class PtyBackendCleanupError extends AggregateError { /** Why one interactive send returned control to its caller. */ export type PtyWaitReason = 'stdin_read' | 'inferred_idle' | 'timeout' | 'session_exit' -/** Signals the model-facing PTY surface permits for foreground process groups. */ +/** + * Signals the model-facing PTY surface permits for foreground process groups. + * Kept member-identical to `SubprocessTerminalSignal` in + * `@deepseek-ai/dsh-subprocess` without a cross-seam dependency; change both together. + */ export type PtySignal = 'SIGINT' | 'SIGTERM' | 'SIGKILL' | 'SIGTSTP' | 'SIGHUP' /** Top-level PTY process status, independent of a send's wait reason. */ diff --git a/packages/pty/tool-bash-persistent/package.json b/packages/pty/tool-bash-persistent/package.json index b5cede3920..23aad9f32e 100644 --- a/packages/pty/tool-bash-persistent/package.json +++ b/packages/pty/tool-bash-persistent/package.json @@ -45,6 +45,7 @@ "@deepseek-ai/dsh-sandbox": "workspace:^", "@deepseek-ai/dsh-sandbox-policy": "workspace:^", "@deepseek-ai/dsh-session": "workspace:^", + "@deepseek-ai/dsh-subprocess-local": "workspace:^", "@deepseek-ai/dsh-system-prompt": "workspace:^", "@deepseek-ai/dsh-timeout": "workspace:^", "@deepseek-ai/dsh-tools": "workspace:^", diff --git a/packages/pty/tool-bash-persistent/tests/loader-composition.spec.ts b/packages/pty/tool-bash-persistent/tests/loader-composition.spec.ts index 16c4a7fea5..a69048e8d6 100644 --- a/packages/pty/tool-bash-persistent/tests/loader-composition.spec.ts +++ b/packages/pty/tool-bash-persistent/tests/loader-composition.spec.ts @@ -15,6 +15,7 @@ import * as PtyLocal from '@deepseek-ai/dsh-pty-local' import SandboxProvider from '@deepseek-ai/dsh-sandbox' import type { ConfinedArgv, SandboxPolicy } from '@deepseek-ai/dsh-sandbox' import SandboxPolicyService from '@deepseek-ai/dsh-sandbox-policy' +import LocalSubprocessService from '@deepseek-ai/dsh-subprocess-local' import SystemPrompt from '@deepseek-ai/dsh-system-prompt' import ToolRegistry from '@deepseek-ai/dsh-tools' import * as ToolBashPersistent from '@deepseek-ai/dsh-tool-bash-persistent' @@ -78,6 +79,7 @@ suite('persistent Bash through a real cordis.yml Loader composition', () => { ' config:', ' mode: danger-full-access', ` workspaceRoot: ${JSON.stringify(root)}`, + "- name: '@deepseek-ai/dsh-subprocess-local'", "- name: '@deepseek-ai/dsh-pty-local'", ' config:', ' pollIntervalMs: 10', @@ -104,6 +106,7 @@ suite('persistent Bash through a real cordis.yml Loader composition', () => { ['@deepseek-ai/dsh-pty', PtyService], ['@deepseek-ai/dsh-test-sandbox', PassthroughSandbox], ['@deepseek-ai/dsh-sandbox-policy', SandboxPolicyService], + ['@deepseek-ai/dsh-subprocess-local', LocalSubprocessService], ['@deepseek-ai/dsh-pty-local', PtyLocal], ['@deepseek-ai/dsh-tool-bash-persistent', ToolBashPersistent], ]) diff --git a/packages/pty/tool-pty/package.json b/packages/pty/tool-pty/package.json index 5512daba92..f443bbbdf4 100644 --- a/packages/pty/tool-pty/package.json +++ b/packages/pty/tool-pty/package.json @@ -50,6 +50,7 @@ "@deepseek-ai/dsh-sandbox": "workspace:^", "@deepseek-ai/dsh-sandbox-policy": "workspace:^", "@deepseek-ai/dsh-session": "workspace:^", + "@deepseek-ai/dsh-subprocess-local": "workspace:^", "@deepseek-ai/dsh-system-prompt": "workspace:^", "@deepseek-ai/dsh-tasks": "workspace:^", "@deepseek-ai/dsh-tasks-local": "workspace:^", diff --git a/packages/pty/tool-pty/tests/loader-composition.spec.ts b/packages/pty/tool-pty/tests/loader-composition.spec.ts index 835df86abc..35d0ea5d0a 100644 --- a/packages/pty/tool-pty/tests/loader-composition.spec.ts +++ b/packages/pty/tool-pty/tests/loader-composition.spec.ts @@ -16,6 +16,7 @@ import PtyService from '@deepseek-ai/dsh-pty' import SandboxProvider from '@deepseek-ai/dsh-sandbox' import type { ConfinedArgv, SandboxPolicy } from '@deepseek-ai/dsh-sandbox' import SandboxPolicyService from '@deepseek-ai/dsh-sandbox-policy' +import LocalSubprocessService from '@deepseek-ai/dsh-subprocess-local' import * as PtyLocal from '@deepseek-ai/dsh-pty-local' import * as ToolPty from '@deepseek-ai/dsh-tool-pty' @@ -72,6 +73,7 @@ suite('terminal real Loader composition through cordis.yml', () => { ' config:', ' mode: danger-full-access', ` workspaceRoot: ${JSON.stringify(root)}`, + "- name: '@deepseek-ai/dsh-subprocess-local'", "- name: '@deepseek-ai/dsh-pty-local'", ' config:', ' pollIntervalMs: 10', @@ -95,6 +97,7 @@ suite('terminal real Loader composition through cordis.yml', () => { ['@deepseek-ai/dsh-pty', PtyService], ['@deepseek-ai/dsh-test-sandbox', PassthroughSandbox], ['@deepseek-ai/dsh-sandbox-policy', SandboxPolicyService], + ['@deepseek-ai/dsh-subprocess-local', LocalSubprocessService], ['@deepseek-ai/dsh-pty-local', PtyLocal], ['@deepseek-ai/dsh-tool-pty', ToolPty], ]) diff --git a/packages/skill/skill-local/tests/skill-local.spec.ts b/packages/skill/skill-local/tests/skill-local.spec.ts index 0f92cac1d6..75f4a4ed65 100644 --- a/packages/skill/skill-local/tests/skill-local.spec.ts +++ b/packages/skill/skill-local/tests/skill-local.spec.ts @@ -42,6 +42,14 @@ class TestFileSystem extends FileSystem { return { targetKey: path as never, displayPath: path } } + override processPath(target: FsTarget): string { return String(target.targetKey) } + + override fileUrl(target: FsTarget): string { return `file://${target.targetKey}` } + + override contains(parent: FsTarget, child: FsTarget): boolean { + return child.targetKey === parent.targetKey || String(child.targetKey).startsWith(`${parent.targetKey}/`) + } + override async stat(target: FsTarget, signal?: AbortSignal): Promise { this.statSignals.push(signal) if (this.failStatPaths.has(target.displayPath)) throw new FsError('stat failed', 'FS_NOT_FOUND') diff --git a/packages/subprocess/README.i18n.yaml b/packages/subprocess/README.i18n.yaml index 9c0cb0c25d..0ebb5bd4af 100644 --- a/packages/subprocess/README.i18n.yaml +++ b/packages/subprocess/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/subprocess/README.md -README.md: 187ea5b778a4bc1f9c3c9121adb18bda11cc7b58 -README.zh.md: dd3a975daec014131877d7b1523810bd932619d8 +README.md: f2b19436da40feb14d067e2cfc706222625680b5 +README.zh.md: 938312448dd5c0a691ed07ddc9843cf2c4445637 diff --git a/packages/subprocess/README.md b/packages/subprocess/README.md index 187ea5b778..f2b19436da 100644 --- a/packages/subprocess/README.md +++ b/packages/subprocess/README.md @@ -2,11 +2,11 @@ English | [中文](README.zh.md) -This family runs host subprocesses behind an explicit process-lifecycle service. +The shared process substrate for one execution world: executable lookup, fully-specified managed child-process trees with raw or collected stdio, and one deep terminal-process primitive that owns PTY allocation, foreground groups, and provider-observable session cleanup. Command defaulting, shell semantics, deadlines, protocol framing, readiness, and presentation stay with consumers — the [bash executors](../bash/README.md), [LSP host](../lsp/README.md), [PTY shell backend](../pty/README.md), and [ACP subagent backend](../subagent/README.md). See the [subprocess seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md). -| Package | Role | ctx key | +| Package | ctx key | Role | |---|---|---| -| [`subprocess/`](subprocess/README.md) | Defines subprocess launch, stream, termination, and disposal contracts | `ctx.subprocess` | -| [`subprocess-local/`](subprocess-local/README.md) | Implements local process-tree execution | registers on `ctx.subprocess` | +| [`subprocess`](subprocess/README.md) (`@deepseek-ai/dsh-subprocess`) | `ctx.subprocess` | The seam: executable lookup, ordinary managed spawns, the terminal-process primitive, handle lifecycles, and shared environment/output vocabulary | +| [`subprocess-local`](subprocess-local/README.md) (`@deepseek-ai/dsh-subprocess-local`) | — | The local implementation: detached process trees, bounded collection/spill, `node-pty`, foreground/session inspection, tree signalling, and terminate-and-join disposal | -The service owns process lifetime; each consumer owns what the process does and which defaults apply. +The service owns process lifetime across consumer reloads; consumers own what a process means (a bash command, a future non-shell runner) and every default that shapes one. diff --git a/packages/subprocess/README.zh.md b/packages/subprocess/README.zh.md index dd3a975dae..938312448d 100644 --- a/packages/subprocess/README.zh.md +++ b/packages/subprocess/README.zh.md @@ -2,11 +2,11 @@ [English](README.md) | 中文 -本家族通过显式的进程生命周期服务运行宿主子进程。 +这里集中提供一个执行世界的共享进程基底：可执行文件查找、具有原始或收集式 stdio 的完整指定受管子进程树，以及一项深层终端进程原语，负责 PTY 分配、前台进程组和提供方仍可观察到的会话成员清理。命令默认值补全、shell 语义、时限、协议分帧、就绪状态与呈现留在消费方：[bash 执行器](../bash/README.md)、[LSP 主机](../lsp/README.md)、[PTY shell 后端](../pty/README.md)与 [ACP（Agent Client Protocol）subagent 后端](../subagent/README.md)。参见[subprocess seam Agent Note（agent 决策记录）](../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md)。 -| 包 | 职责 | ctx 键 | +| 包（package） | ctx 键 | 角色 | |---|---|---| -| [`subprocess/`](subprocess/README.md) | 定义子进程启动、流、终止和 dispose（资源释放）契约 | `ctx.subprocess` | -| [`subprocess-local/`](subprocess-local/README.md) | 实现本地进程树执行 | 注册到 `ctx.subprocess` | +| [`subprocess`](subprocess/README.md)（`@deepseek-ai/dsh-subprocess`） | `ctx.subprocess` | seam 本体：可执行文件查找、普通受管 spawn、终端进程原语、句柄生命周期，以及共享的环境／输出词汇 | +| [`subprocess-local`](subprocess-local/README.md)（`@deepseek-ai/dsh-subprocess-local`） | 无 | 本地实现：detached 进程树、有界收集／spill、`node-pty`、前台／会话检查、进程树信号发送，以及先终止再等待退出的资源释放 | -服务负责进程生命周期；每个消费方负责进程执行的工作以及所应用的默认值。 +即使消费方重载，进程生命周期仍由服务负责管理；消费方负责定义进程的含义（一条 bash 命令、未来的非 shell 运行器），以及决定塑造该进程的每一项默认值。 diff --git a/packages/subprocess/subprocess-local/README.i18n.yaml b/packages/subprocess/subprocess-local/README.i18n.yaml index 28ebc2c110..22d971b251 100644 --- a/packages/subprocess/subprocess-local/README.i18n.yaml +++ b/packages/subprocess/subprocess-local/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/subprocess/subprocess-local/README.md -README.md: af9f92db714398dd52c9b5e7aeb64d5af71021da -README.zh.md: da78cbcfbff174eb5fda5b8323fbdc84b50c9997 +README.md: 087ca24a3207cb8cb1568769a462fbbb010aaa35 +README.zh.md: 4d400906f71b653ce2be95a22751fd9893f447bd diff --git a/packages/subprocess/subprocess-local/README.md b/packages/subprocess/subprocess-local/README.md index af9f92db71..087ca24a32 100644 --- a/packages/subprocess/subprocess-local/README.md +++ b/packages/subprocess/subprocess-local/README.md @@ -2,7 +2,7 @@ English | [中文](README.zh.md) -Local implementation of the [`@deepseek-ai/dsh-subprocess`](../subprocess/README.md) seam: `LocalSubprocessService` spawns each spec's argv as a detached process tree, wires the spec's per-stream stdio dispositions (raw pipes, inherit, bounded tail-keep collection with optional spill files), and signals tree-scoped with SIGTERM→SIGKILL escalation. It has no config: every disposition, limit, and directory arrives on the spawn spec, so the deployment-varying knobs stay with the calling seams' configs ([`dsh-bash-local`](../../bash/bash-local/README.md), [`dsh-lsp-local`](../../lsp/lsp-local/README.md), [`dsh-subagent-acp`](../../subagent/subagent-acp/README.md)). +Local implementation of the [`@deepseek-ai/dsh-subprocess`](../subprocess/README.md) seam. `LocalSubprocessService` resolves local executables, spawns ordinary detached process trees with explicit stdio, and implements terminal processes through `node-pty` plus platform process inspection. It has no config: every disposition, limit, terminal dimension, grace, and directory arrives from the calling seams ([`dsh-bash-local`](../../bash/bash-local/README.md), [`dsh-lsp-local`](../../lsp/lsp-local/README.md), and [`dsh-pty-local`](../../pty/pty-local/README.md)). ## Behavior (and where it came from) @@ -10,6 +10,8 @@ Local implementation of the [`@deepseek-ai/dsh-subprocess`](../subprocess/README - **Per-stream dispositions** — `'pipe'` hands the raw stream to the caller untouched (protocol framing stays consumer-owned); `'inherit'` passes the parent descriptor through; collect mode keeps the in-memory TAIL beyond its cap (errors and results cluster at the end — pi/OpenCode rationale) while the FULL stream is appended to a private temp file when a spill cap is configured — omitting `spill` keeps only the tail, the diagnostic shape. A stream larger than the spill cap discards its now-incomplete spill and returns only the marked truncated tail; spill fds are sealed at settlement, and a failed final close withholds the path rather than advertising an incomplete file. Spill files are `0600` with random names under a lazily-created `0700` per-process directory. - **Credential scrub + explicit merge** — `process.env` minus credential-shaped vars (`*KEY*`/`*PASSWORD*`/`*SECRET*`/`*TOKEN*`) and all ambient `DSH_*` names; the spec's explicit `env` merges after that scrub with no namespace validation, so a deliberately supplied credential or current `DSH_*` fact wins while stale nested-harness identity cannot leak in ambiently. Supplied stdin is written and closed; otherwise fd 0 is `/dev/null`. See the [stdin/env Agent Note](../../../.agents/notes/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-surface.md) and [managed environment Agent Note](../../../.agents/notes/implemented/feature/2026-07-10-agent-session-identity-and-log-location.md). - **Offset-based reads** — collect-mode readers return deltas in whole-stream byte coordinates; the service never holds a cursor, so consumer-owned cursors (the bash background read path) and full-stream re-reads coexist, before and after settlement. +- **Executable lookup** — `resolveExecutable` checks absolute files or searches the scrubbed effective PATH with platform-aware executable extensions; relative paths containing separators are rejected at the seam, and relative PATH entries resolve from the host process cwd. +- **Terminal-process ownership** — `spawnTerminal` allocates `node-pty`, bridges UTF-8 terminal text, inspects and signals the current foreground process group, and exposes one awaited termination operation that sweeps descendants before and after terminating the top-level shell. Each foreground inspection retains exact identities from the rooted tree; Linux also enumerates the POSIX session after its leader exits. A previously observed macOS descendant and any same-session Linux member therefore remain fenced after reparenting, while pid/start identity prevents cleanup from following PID reuse. The higher PTY backend owns prompt readiness, buffers, and model-facing operations. - **Terminate-and-join disposal** — the service retains live handles only so its own disposal can escalate every running tree and await its exit; settled and spawn-failed handles leave the live set on settlement. ## Model Experience @@ -23,6 +25,8 @@ No direct invalidation; the named consumers own any request-prefix changes. ## Known Limitations and Deferred Work - **Windows tree support is best-effort** — termination routes through `taskkill /PID /T /F` with all outcomes contained (absent tree, races, missing binary), and liveness falls back to the direct-child boundary. +- **Terminal process inspection is Linux/macOS only** — the terminal primitive fails when its inspector has no supported platform implementation; Linux exact probes cover x64 and arm64, while macOS uses `ps` snapshots. +- **A daemonized terminal descendant can still escape the observable boundary** — on macOS, a child that reparents before any foreground-inspection snapshot is no longer discoverable from the `node-pty` root; on Linux, a child that calls `setsid` leaves both the tree and owned terminal session. The local provider does not add a continuous process-table monitor. - **The credential scrub is a name heuristic** — `*KEY*`/`*PASSWORD*`/`*SECRET*`/`*TOKEN*` only; differently-named secrets (e.g. `*PASSPHRASE*`) pass through, and a whitelist for over-scrubbed vars is noted future work. - **Completed spill files are not deleted** — bounded full-output recovery files (and the private per-process spill dir) accumulate under the OS tmpdir until something external cleans them; oversize incomplete spills are discarded and deletion is attempted immediately, but a cleanup failure can leave a bounded file behind. diff --git a/packages/subprocess/subprocess-local/README.zh.md b/packages/subprocess/subprocess-local/README.zh.md index da78cbcfbf..4d400906f7 100644 --- a/packages/subprocess/subprocess-local/README.zh.md +++ b/packages/subprocess/subprocess-local/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -[`@deepseek-ai/dsh-subprocess`](../subprocess/README.md) seam 的本地实现：`LocalSubprocessService` 将每个 spec 的 argv spawn 为 detached 进程树，依照 spec 中按流划分的 stdio 处置方式（disposition）完成接线（原始管道、inherit、附带可选 spill 文件的有界尾部保留收集），并以进程树为范围发送信号，按 SIGTERM→SIGKILL 逐级升级。该实现没有任何配置：每项处置方式、限制与目录都随 spawn spec 传入，因此随部署变化的可调参数留在各调用方 seam 的配置里（[`dsh-bash-local`](../../bash/bash-local/README.md)、[`dsh-lsp-local`](../../lsp/lsp-local/README.md)、[`dsh-subagent-acp`](../../subagent/subagent-acp/README.md)）。 +[`@deepseek-ai/dsh-subprocess`](../subprocess/README.md) seam 的本地实现。`LocalSubprocessService` 解析本地可执行文件，以显式 stdio spawn 普通 detached 进程树，并通过 `node-pty` 加平台进程检查实现终端进程。该实现没有任何配置：每项处置方式、限制、终端尺寸、宽限期与目录都来自调用方 seam（[`dsh-bash-local`](../../bash/bash-local/README.md)、[`dsh-lsp-local`](../../lsp/lsp-local/README.md) 和 [`dsh-pty-local`](../../pty/pty-local/README.md)）。 ## 行为（以及设计来源） @@ -10,6 +10,8 @@ - **按流划分的处置方式**：`'pipe'` 把原始流原样交给调用方（协议分帧仍归消费方所有）；`'inherit'` 直通父进程的描述符；收集模式（collect）在输出超过上限后于内存中保留尾部（错误与结果通常聚集在末尾，沿用 pi/OpenCode 的理由），并在配置了 spill 上限时把完整流追加到一个私有临时文件；省略 `spill` 则只保留用于诊断的尾部。某条流大于 spill 上限时，会丢弃已不完整的 spill，仅返回带截断标记的尾部；spill 文件描述符在结算时封存，最终关闭失败时则不公布路径，以免声称存在不完整的文件。spill 文件权限为 `0600`、名称随机，位于按需创建、权限为 `0700` 的每进程目录之下。 - **凭据清除 + 显式合并**：以 `process.env` 为基础，移除形似凭据的变量（`*KEY*`／`*PASSWORD*`／`*SECRET*`／`*TOKEN*`）和所有环境中已有的 `DSH_*` 名称；spec 的显式 `env` 在该清除之后合并且不做命名空间校验，因此有意提供的凭据或当前 `DSH_*` 事实会胜出，而陈旧的嵌套 harness 身份无法从环境中隐式漏入。提供的 stdin 会被写入后关闭；否则 fd 0 指向 `/dev/null`。参见 [stdin/env Agent Note](../../../.agents/notes/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-surface.md)与[受管环境 Agent Note](../../../.agents/notes/implemented/feature/2026-07-10-agent-session-identity-and-log-location.md)。 - **基于偏移量的读取**：收集模式的读取器按完整流的字节坐标返回增量；服务自身从不持有游标，因此消费方自有的游标（bash 的后台读取路径）与完整流重读可以共存，结算前后皆然。 +- **可执行文件查找**：`resolveExecutable` 检查绝对文件，或根据平台可执行文件扩展名在清理后的有效 PATH 中搜索；含分隔符的相对路径在接缝处被拒绝，相对 PATH 条目从宿主进程 cwd 解析。 +- **终端进程所有权**：`spawnTerminal` 分配 `node-pty`，桥接 UTF-8 终端文本，检查当前前台进程组并向其发送信号，还会公开一项须等待的终止操作，在终止顶层 shell 前后清理后代进程。每次前台检查都会保留根进程树中的精确身份；Linux 还会在 POSIX 会话 leader 退出后枚举该会话。因此，之前观察到的 macOS 后代以及同会话 Linux 成员在重新设定父进程后仍受围栏保护，pid/start 身份则防止清理跟随 PID 复用。上层 PTY 后端负责提示符就绪、缓冲区与面向模型的操作。 - **先终止再等待退出的 dispose（资源释放）**：服务保留存活句柄，只为让自身的 dispose 能对每个仍在运行的进程树执行升级并等待其退出；已结算与 spawn 失败的句柄在结算时即离开存活集合。 ## 模型体验 @@ -23,6 +25,8 @@ ## 已知限制与暂缓事项 - **Windows 进程树支持仅为尽力而为**：终止经由 `taskkill /PID /T /F` 完成，所有结果都被就地吸收，不向外抛出（进程树已不存在、竞态、二进制缺失），存活探测则回退到直接子进程边界。 +- **终端进程检查仅支持 Linux／macOS**：检查器没有受支持的平台实现时，终端原语会失败；Linux 精确探针覆盖 x64 与 arm64，macOS 则使用 `ps` 快照。 +- **守护化的终端后代仍可能逃出可观察边界**：在 macOS 上，子进程如果在任何前台检查快照之前重新设定父进程，将无法再从 `node-pty` 根进程发现；在 Linux 上，调用 `setsid` 的子进程会同时离开进程树与自有终端会话。本地提供方不会新增持续进程表监视器。 - **凭据清除依赖名称启发式规则**：只匹配 `*KEY*`／`*PASSWORD*`／`*SECRET*`／`*TOKEN*`；名称不同的 secret（例如 `*PASSPHRASE*`）会继续传递，对误删变量引入白名单属于已记录的后续工作。 - **不会删除已完成的 spill 文件**：有界的完整输出恢复文件（以及每个进程的私有 spill 目录）会在 OS tmpdir 下累积，直到外部机制进行清理；超大的不完整 spill 会被丢弃并立即尝试删除，但清理失败可能留下一个有界文件。 diff --git a/packages/subprocess/subprocess-local/package.json b/packages/subprocess/subprocess-local/package.json index e1429bffd0..7a68a9b91b 100644 --- a/packages/subprocess/subprocess-local/package.json +++ b/packages/subprocess/subprocess-local/package.json @@ -21,8 +21,12 @@ "files": [ "lib/index.js", "lib/invariant.js", + "scripts/ensure-spawn-helper.mjs", "lib/types/**/*.d.ts" ], + "scripts": { + "postinstall": "node scripts/ensure-spawn-helper.mjs" + }, "license": "BSD-3-Clause", "peerDependencies": { "@deepseek-ai/dsh-invariants": "^0.0.1", @@ -30,6 +34,9 @@ "@deepseek-ai/dsh-timeout": "^0.0.1", "cordis": "^4.0.0-rc.7" }, + "dependencies": { + "node-pty": "^1.1.0" + }, "devDependencies": { "@deepseek-ai/dsh-invariants": "workspace:^", "@deepseek-ai/dsh-subprocess": "workspace:^", diff --git a/packages/pty/pty-local/scripts/ensure-spawn-helper.mjs b/packages/subprocess/subprocess-local/scripts/ensure-spawn-helper.mjs similarity index 100% rename from packages/pty/pty-local/scripts/ensure-spawn-helper.mjs rename to packages/subprocess/subprocess-local/scripts/ensure-spawn-helper.mjs diff --git a/packages/subprocess/subprocess-local/src/index.ts b/packages/subprocess/subprocess-local/src/index.ts index eea1f1e739..e441944225 100644 --- a/packages/subprocess/subprocess-local/src/index.ts +++ b/packages/subprocess/subprocess-local/src/index.ts @@ -7,11 +7,24 @@ * @module @deepseek-ai/dsh-subprocess-local */ +import { constants } from 'node:fs' +import { access, stat } from 'node:fs/promises' +import { delimiter, extname, isAbsolute, resolve } from 'node:path' import { Context } from 'cordis' +import * as nodePty from 'node-pty' +import type { IPtyForkOptions } from 'node-pty' import { SubprocessService } from '@deepseek-ai/dsh-subprocess' -import type { SubprocessHandle, SubprocessSpawnSpec } from '@deepseek-ai/dsh-subprocess' -import { spawnSubprocess } from './spawn.ts' +import type { + SubprocessHandle, + SubprocessSpawnSpec, + SubprocessTerminalHandle, + SubprocessTerminalSpawnSpec, +} from '@deepseek-ai/dsh-subprocess' +import { childEnv, spawnSubprocess } from './spawn.ts' import type { SpawnInternals } from './spawn.ts' +import { createProcessInspector } from './process-inspector.ts' +import type { ProcessInspector } from './process-inspector.ts' +import { LocalTerminalHandle } from './terminal.ts' /** * Local subprocess service: detached process trees, Node-shaped stdio @@ -22,8 +35,12 @@ import type { SpawnInternals } from './spawn.ts' export class LocalSubprocessService extends SubprocessService { /** Live handles retained only so disposal can terminate and join them. */ private live = new Set() + /** Live terminal sessions retained through whole-session quiescence. */ + private terminals = new Set() /** Test seam: spill and platform knobs forwarded to spawnSubprocess. */ internals: SpawnInternals = {} + /** Test seam for platform process inspection; production resolves lazily on terminal spawn. */ + terminalInspector: ProcessInspector | undefined constructor(ctx: Context) { super(ctx) @@ -37,11 +54,62 @@ export class LocalSubprocessService extends SubprocessService { // Spawn-failure rejections already settled and left the live set. pending.push(handle.done.catch(() => {}).then(() => handle.waitForExit())) } + for (const terminal of this.terminals) { + pending.push(terminal.terminate()) + } this.live.clear() - await Promise.all(pending) + this.terminals.clear() + const outcomes = await Promise.allSettled(pending) + const failures = outcomes.flatMap(outcome => outcome.status === 'rejected' + ? [outcome.reason as unknown] + : []) + if (failures.length === 1) throw failures[0] + if (failures.length > 1) throw new AggregateError(failures, 'local subprocess teardown failed') }, 'local subprocess teardown') } + async resolveExecutable( + command: string, + env?: Readonly>, + signal?: AbortSignal, + ): Promise { + if (command.length === 0) throw new Error('subprocess-local: executable must be non-empty') + signal?.throwIfAborted() + const environment = childEnv(env) + const absolute = isAbsolute(command) + if (!absolute && (command.includes('/') || (process.platform === 'win32' && command.includes('\\')))) { + throw new Error( + `subprocess-local: command ${JSON.stringify(command)} is a relative path; use an absolute path or a bare PATH name`, + ) + } + const candidates = absolute ? [command] : this.executableCandidates(command, environment) + for (const candidate of candidates) { + signal?.throwIfAborted() + try { + const info = await stat(candidate) + if (!info.isFile()) continue + await access(candidate, constants.X_OK) + signal?.throwIfAborted() + return candidate + } catch { + // Try the next PATH candidate; the final miss receives one stable error. + } + } + signal?.throwIfAborted() + throw new Error(absolute + ? `subprocess-local: command ${JSON.stringify(command)} is not an executable file` + : `subprocess-local: command ${JSON.stringify(command)} was not found on PATH`) + } + + private executableCandidates(command: string, env: NodeJS.ProcessEnv): string[] { + const path = environmentValue(env, 'PATH') ?? '' + const extensions = process.platform === 'win32' && extname(command) === '' + ? (environmentValue(env, 'PATHEXT') ?? '.COM;.EXE;.BAT;.CMD').split(';') + : [''] + return path.split(delimiter).flatMap(directory => + extensions.map(extension => resolve(process.cwd(), directory, command + extension))) + } + spawn(spec: SubprocessSpawnSpec): SubprocessHandle { const handle = spawnSubprocess(spec, this.internals) this.live.add(handle) @@ -54,6 +122,41 @@ export class LocalSubprocessService extends SubprocessService { handle.done.then(release, release) return handle } + + // Local PTY allocation is synchronous, but the provider seam permits remote asynchronous allocation. + // oxlint-disable-next-line typescript/require-await -- Preserve promise rejection semantics at the async provider seam. + async spawnTerminal(spec: SubprocessTerminalSpawnSpec): Promise { + const file = spec.argv[0] + if (file === undefined || file.length === 0) { + throw new Error('subprocess-local: terminal argv must contain a program') + } + spec.signal?.throwIfAborted() + const options: IPtyForkOptions = { + name: 'dumb', + rows: spec.rows, + cols: spec.cols, + cwd: spec.cwd, + env: childEnv(spec.env), + } + const inspector = this.terminalInspector ?? createProcessInspector() + const terminal = nodePty.spawn(file, [...spec.argv.slice(1)], options) + const handle = new LocalTerminalHandle(terminal, inspector, spec.graceMs) + this.terminals.add(handle) + const release = async (): Promise => { + await handle.terminate() + this.terminals.delete(handle) + } + void handle.done.then(release, release).catch(() => {}) + return handle + } +} + +/** Read a Windows environment key using the platform's case-insensitive semantics. */ +function environmentValue(env: NodeJS.ProcessEnv, name: 'PATH' | 'PATHEXT'): string | undefined { + const exact = env[name] + if (exact !== undefined || process.platform !== 'win32') return exact + const normalized = name.toUpperCase() + return Object.entries(env).find(([key]) => key.toUpperCase() === normalized)?.[1] } export default LocalSubprocessService diff --git a/packages/pty/pty-local/src/process-inspector.ts b/packages/subprocess/subprocess-local/src/process-inspector.ts similarity index 85% rename from packages/pty/pty-local/src/process-inspector.ts rename to packages/subprocess/subprocess-local/src/process-inspector.ts index 5be25a224a..f31de010de 100644 --- a/packages/pty/pty-local/src/process-inspector.ts +++ b/packages/subprocess/subprocess-local/src/process-inspector.ts @@ -1,8 +1,8 @@ -/** Platform process-table inspection used for readiness, signals, and teardown. */ +/** Platform process-table inspection for terminal readiness, signals, and teardown. */ import { closeSync, openSync, readFileSync, readdirSync, readSync } from 'node:fs' import { execFileSync } from 'node:child_process' -import type { PtySignal } from '@deepseek-ai/dsh-pty' +import type { SubprocessTerminalSignal } from '@deepseek-ai/dsh-subprocess' /** PID plus start identity, preventing teardown escalation after PID reuse. */ export interface ProcessIdentity { @@ -16,9 +16,11 @@ export interface ProcessInspector { isStdinWaiting(pgid: number): boolean /** Return the root and its current transitive descendants, children first. */ processTree(rootPid: number): ProcessIdentity[] + /** Return current members of one POSIX process session when the platform exposes them. */ + processSession(sessionId: number): ProcessIdentity[] /** Return whether the exact identity remains a non-quiescent process. */ isAlive(identity: ProcessIdentity): boolean - signalGroup(pgid: number, signal: PtySignal): void + signalGroup(pgid: number, signal: SubprocessTerminalSignal): void signalProcess(identity: ProcessIdentity, signal: 'SIGTERM' | 'SIGKILL'): void } @@ -85,6 +87,35 @@ function readLinuxStat(internals: ProcessInspectorInternals, pid: number): ProcS } } +/** + * Report whether a Linux process group has an executing member. `false` + * means the group contains only zombie/dead entries; `undefined` means the + * process table could not prove either outcome. + * @param processGroupId - POSIX process-group id to inspect. + * @param internals - injectable process-table operations. + * @returns Live-member presence, or `undefined` when unavailable/absent. + */ +export function linuxProcessGroupHasLiveMembers( + processGroupId: number, + internals: ProcessInspectorInternals = DEFAULT_INTERNALS, +): boolean | undefined { + let entries: string[] + try { + entries = internals.readDir('/proc') + } catch (_unreadableProcDirectory) { + return undefined + } + let matched = false + for (const entry of entries) { + if (!/^\d+$/.test(entry)) continue + const stat = readLinuxStat(internals, Number(entry)) + if (stat?.pgrp !== processGroupId) continue + matched = true + if (!/^[ZXx]$/.test(stat.state)) return true + } + return matched ? false : undefined +} + function numericEntries(internals: ProcessInspectorInternals, path: string): number[] { try { return internals.readDir(path).filter(entry => /^\d+$/.test(entry)).map(Number) @@ -201,9 +232,10 @@ abstract class PosixProcessInspector implements ProcessInspector { abstract foregroundPgid(shellPid: number): number | undefined abstract isStdinWaiting(pgid: number): boolean abstract processTree(rootPid: number): ProcessIdentity[] + abstract processSession(sessionId: number): ProcessIdentity[] abstract isAlive(identity: ProcessIdentity): boolean - signalGroup(pgid: number, signal: PtySignal): void { + signalGroup(pgid: number, signal: SubprocessTerminalSignal): void { this.internals.kill(-pgid, signal) } @@ -272,6 +304,13 @@ class LinuxProcessInspector extends PosixProcessInspector { return processTree(entries, rootPid) } + processSession(sessionId: number): ProcessIdentity[] { + return numericEntries(this.internals, '/proc').flatMap((pid) => { + const stat = readLinuxStat(this.internals, pid) + return stat?.session === sessionId ? [{ pid, started: stat.started }] : [] + }) + } + isAlive(identity: ProcessIdentity): boolean { const stat = readLinuxStat(this.internals, identity.pid) return stat?.started === identity.started && !/^[ZXx]$/.test(stat.state) @@ -307,6 +346,10 @@ class MacProcessInspector extends PosixProcessInspector { return processTree(macProcessTable(this.internals), rootPid) } + processSession(_sessionId: number): ProcessIdentity[] { + return [] + } + isAlive(identity: ProcessIdentity): boolean { return macProcessTable(this.internals).some(entry => entry.pid === identity.pid && entry.started === identity.started) } @@ -327,5 +370,5 @@ export function createProcessInspector( ): ProcessInspector { if (platform === 'linux') return new LinuxProcessInspector(arch, internals) if (platform === 'darwin') return new MacProcessInspector(internals) - throw new Error(`pty-local: unsupported platform ${platform}`) + throw new Error(`subprocess-local: terminal inspection is unsupported on platform ${platform}`) } diff --git a/packages/subprocess/subprocess-local/src/spawn.ts b/packages/subprocess/subprocess-local/src/spawn.ts index 462da41382..5b977cacc1 100644 --- a/packages/subprocess/subprocess-local/src/spawn.ts +++ b/packages/subprocess/subprocess-local/src/spawn.ts @@ -24,16 +24,26 @@ import type { SubprocessOutputMode, SubprocessSpawnSpec, } from '@deepseek-ai/dsh-subprocess' +import { linuxProcessGroupHasLiveMembers } from './process-inspector.ts' /** - * Build a child environment: explicit caller entries merge after the scrubbed - * parent base. A string deliberately restores or overrides an entry; an - * explicit `undefined` tombstone removes an ordinary ambient entry. + * Build a child environment: explicit caller entries override the scrubbed + * parent base using the target platform's environment-key semantics. A string + * deliberately restores or overrides an entry; an explicit `undefined` + * tombstone removes an ordinary ambient entry. * @param extra - explicit caller entries and tombstones, merged after the scrub. * @returns the environment to hand to `spawn` for the child process. */ export function childEnv(extra?: Readonly): NodeJS.ProcessEnv { - return { ...scrubbedParentEnv(), ...extra } + const env = scrubbedParentEnv() + if (process.platform !== 'win32') return { ...env, ...extra } + let entries: [string, string | undefined][] = Object.entries(env) + for (const [key, value] of Object.entries(extra ?? {})) { + const normalized = key.toUpperCase() + entries = entries.filter(([inherited]) => inherited.toUpperCase() !== normalized) + entries.push([key, value]) + } + return Object.fromEntries(entries) } /** Injectable knobs so tests can exercise spill and platform behavior deterministically. */ @@ -44,6 +54,8 @@ export interface SpawnInternals { taskkill?: (pid: number) => void /** Host platform override for signalling decisions. */ platform?: NodeJS.Platform + /** Linux process-group member probe (defaults to `/proc` inspection). */ + linuxProcessGroupHasLiveMembers?: (processGroupId: number) => boolean | undefined } /** @@ -308,6 +320,7 @@ export function spawnSubprocess(spec: SubprocessSpawnSpec, internals: SpawnInter const spillDir = internals.spillDir ?? privateSpillDir() const platform = internals.platform ?? process.platform const taskkill = internals.taskkill ?? taskkillProcessTree + const linuxGroupHasLiveMembers = internals.linuxProcessGroupHasLiveMembers ?? linuxProcessGroupHasLiveMembers if (spec.signal?.aborted) { throw new Error(`aborted before spawn: ${String(spec.signal.reason ?? 'aborted')}`) @@ -367,6 +380,11 @@ export function spawnSubprocess(spec: SubprocessSpawnSpec, internals: SpawnInter } try { process.kill(-pid, 0) + // A group containing only unreaped zombies still answers kill(0), but + // it can execute no work and cannot be signalled into quiescence. Only + // inspect after direct-child settlement so live-process polls remain a + // syscall rather than repeated process-table scans. + if (settled && platform === 'linux' && linuxGroupHasLiveMembers(pid) === false) return false return true } catch (error) { const code = (error as NodeJS.ErrnoException).code diff --git a/packages/subprocess/subprocess-local/src/terminal.ts b/packages/subprocess/subprocess-local/src/terminal.ts new file mode 100644 index 0000000000..2a763948bb --- /dev/null +++ b/packages/subprocess/subprocess-local/src/terminal.ts @@ -0,0 +1,212 @@ +/** Local node-pty terminal-process implementation for the subprocess seam. */ + +import { Buffer } from 'node:buffer' +import { constants } from 'node:os' +import { PassThrough } from 'node:stream' +import type { IDisposable, IPty } from 'node-pty' +import type { + SubprocessOutcome, + SubprocessTerminalForeground, + SubprocessTerminalHandle, + SubprocessTerminalSignal, +} from '@deepseek-ai/dsh-subprocess' +import type { ProcessIdentity, ProcessInspector } from './process-inspector.ts' + +function delay(ms: number): Promise { + return new Promise(resolve => setTimeout(resolve, ms)) +} + +function signalName(number: number | undefined): NodeJS.Signals | null { + if (number === undefined || number === 0) return null + for (const [name, value] of Object.entries(constants.signals)) { + if (value === number) return name as NodeJS.Signals + } + return null +} + +/** + * A local terminal whose process-session ownership stays below the PTY backend. + * The seam's terminate() promise — no write, inspection, or signal in flight + * after settlement — holds here without operation tracking only because every + * handle call completes synchronously under the hood (node-pty write, ps-based + * inspection). A first genuinely asynchronous step in any handle call must add + * the tracking a remote provider needs. + */ +export class LocalTerminalHandle implements SubprocessTerminalHandle { + readonly pid: number + readonly output = new PassThrough() + readonly done: Promise + + private readonly outcome = Promise.withResolvers() + private readonly dataDisposable: IDisposable + private readonly exitDisposable: IDisposable + private cleanup: Promise | undefined + private exited = false + private trackedDescendants: ProcessIdentity[] = [] + /** The spawned shell's start identity; scans stop adopting members once the root pid no longer carries it. */ + private readonly rootIdentity: ProcessIdentity | undefined + + /** + * @param terminal - allocated node-pty process. + * @param inspector - platform process/session operations. + * @param graceMs - TERM-to-KILL and exit-wait grace. + */ + constructor( + private readonly terminal: IPty, + private readonly inspector: ProcessInspector, + private readonly graceMs: number, + ) { + this.pid = terminal.pid + this.rootIdentity = inspector.processTree(this.pid).find(member => member.pid === this.pid) + this.done = this.outcome.promise + this.dataDisposable = terminal.onData((data) => { this.output.write(Buffer.from(data, 'utf8')) }) + this.exitDisposable = terminal.onExit(({ exitCode, signal: exitSignal }) => { + if (this.exited) return + this.exited = true + this.output.end() + this.outcome.resolve({ + exitCode: exitSignal === undefined || exitSignal === 0 ? exitCode : null, + signal: signalName(exitSignal), + }) + }) + } + + // node-pty writes synchronously; the seam returns a promise for remote transports. + // oxlint-disable-next-line typescript/require-await -- Preserve promise rejection semantics at the async provider seam. + async write(data: string): Promise { + if (this.exited) throw new Error('terminal process has exited') + this.terminal.write(data) + } + + // Local inspection is synchronous; the seam returns a promise for remote transports. + // oxlint-disable-next-line typescript/require-await -- Preserve promise rejection semantics at the async provider seam. + async inspectForeground(): Promise { + this.descendants() + const processGroupId = this.inspector.foregroundPgid(this.pid) + if (processGroupId === undefined) return undefined + return { + processGroupId, + inputWaiting: this.inspector.isStdinWaiting(processGroupId), + } + } + + async signalForeground(signal: SubprocessTerminalSignal): Promise { + const foreground = await this.inspectForeground() + if (foreground === undefined) { + throw new Error(`cannot resolve foreground process group for terminal ${this.pid}`) + } + if (signal === 'SIGKILL' && foreground.processGroupId === this.pid) { + throw new Error('refusing to SIGKILL the terminal shell; terminate the terminal session instead') + } + this.inspector.signalGroup(foreground.processGroupId, signal) + return foreground.processGroupId + } + + terminate(): Promise { + if (this.cleanup !== undefined) return this.cleanup + const cleanup = this.closeOnce() + this.cleanup = cleanup + void cleanup.catch(() => { this.cleanup = undefined }) + return cleanup + } + + private survivors(members: ProcessIdentity[]): ProcessIdentity[] { + return members.filter(member => this.inspector.isAlive(member)) + } + + private descendants(): ProcessIdentity[] { + // Adopt newly scanned members only while the numeric root pid provably + // still carries the spawned shell's start identity: after the shell dies, + // a recycled pid's tree and session must not donate an unrelated + // process's children to this session's signalling. Already-adopted + // members keep their own start identities, which every signal rechecks. + const tree = this.inspector.processTree(this.pid) + const root = tree.find(member => member.pid === this.pid) + const rootVerified = this.rootIdentity !== undefined + && root !== undefined + && root.started === this.rootIdentity.started + this.trackedDescendants = this.survivors(this.unionMembers( + this.trackedDescendants, + ...rootVerified ? [tree, this.inspector.processSession(this.pid)] : [], + ).filter(member => member.pid !== this.pid)) + return this.trackedDescendants + } + + private async waitForMembers(members: ProcessIdentity[]): Promise { + const until = Date.now() + this.graceMs + let survivors = this.survivors(members) + while (survivors.length > 0 && Date.now() < until) { + await delay(Math.min(25, Math.max(1, until - Date.now()))) + survivors = this.survivors(members) + } + return survivors + } + + private signalMembers(members: ProcessIdentity[], signal: 'SIGTERM' | 'SIGKILL'): void { + for (const member of members) { + try { + this.inspector.signalProcess(member, signal) + } catch (_alreadyExitedDuringSignal) { + // The exact process identity is rechecked; a same-tick exit is success. + } + } + } + + private unionMembers(...groups: ProcessIdentity[][]): ProcessIdentity[] { + const members: ProcessIdentity[] = [] + const seen = new Set() + for (const group of groups) { + for (const member of group) { + const key = `${member.pid}:${member.started}` + if (seen.has(key)) continue + seen.add(key) + members.push(member) + } + } + return members + } + + private async stopDescendants(): Promise { + const captured = this.descendants() + this.signalMembers(captured, 'SIGTERM') + const capturedSurvivors = await this.waitForMembers(captured) + const members = this.unionMembers(capturedSurvivors, this.descendants()) + this.signalMembers(members, 'SIGKILL') + const survivors = await this.waitForMembers(members) + return this.survivors(this.unionMembers(survivors, this.descendants())) + } + + private async stopShell(): Promise { + if (!this.exited) { + try { + this.terminal.kill('SIGTERM') + } catch (_topLevelAlreadyExitedDuringTerm) { + // The exit callback is authoritative. + } + await Promise.race([this.done.then(() => undefined), delay(this.graceMs)]) + } + if (!this.exited) { + try { + this.terminal.kill('SIGKILL') + } catch (_topLevelAlreadyExitedDuringKill) { + // The exit callback is authoritative. + } + await Promise.race([this.done.then(() => undefined), delay(this.graceMs)]) + } + if (!this.exited) throw new Error(`terminal cleanup failed; surviving pid: ${this.pid}`) + } + + private async closeOnce(): Promise { + let survivors = await this.stopDescendants() + if (survivors.length > 0) { + throw new Error(`terminal cleanup failed; surviving pids: ${survivors.map(member => member.pid).join(', ')}`) + } + await this.stopShell() + survivors = await this.stopDescendants() + if (survivors.length > 0) { + throw new Error(`terminal cleanup failed; surviving pids: ${survivors.map(member => member.pid).join(', ')}`) + } + this.dataDisposable.dispose() + this.exitDisposable.dispose() + } +} diff --git a/packages/subprocess/subprocess-local/tests/local.spec.ts b/packages/subprocess/subprocess-local/tests/local.spec.ts index ca723fdb87..c4e3f91522 100644 --- a/packages/subprocess/subprocess-local/tests/local.spec.ts +++ b/packages/subprocess/subprocess-local/tests/local.spec.ts @@ -1,7 +1,10 @@ -import { describe, expect, it } from 'vitest' +import { PassThrough } from 'node:stream' +import { describe, expect, it, vi } from 'vitest' +import { basename, dirname, relative, resolve } from 'node:path' import { Context } from 'cordis' import LocalSubprocessService from '@deepseek-ai/dsh-subprocess-local' -import type { SubprocessSpawnSpec } from '@deepseek-ai/dsh-subprocess' +import type { SubprocessSpawnSpec, SubprocessTerminalHandle, SubprocessTerminalSpawnSpec } from '@deepseek-ai/dsh-subprocess' +import { childEnv } from '../src/spawn.ts' function spec(command: string, overrides: Partial = {}): SubprocessSpawnSpec { return { @@ -18,6 +21,255 @@ function spec(command: string, overrides: Partial = {}): Su } describe('LocalSubprocessService', () => { + it('resolves absolute and PATH executables and honors lookup cancellation', async () => { + const ctx = new Context() + const fiber = await ctx.plugin(LocalSubprocessService) + expect(await ctx.subprocess.resolveExecutable(process.execPath)).toBe(process.execPath) + expect(await ctx.subprocess.resolveExecutable(basename(process.execPath), { + PATH: dirname(process.execPath), + })).toBe(process.execPath) + expect(await ctx.subprocess.resolveExecutable(basename(process.execPath), { + PATH: relative(process.cwd(), dirname(process.execPath)) || '.', + })).toBe(process.execPath) + await expect(ctx.subprocess.resolveExecutable('')).rejects.toThrow('must be non-empty') + await expect(ctx.subprocess.resolveExecutable('./bin/tsserver')) + .rejects.toThrow('is a relative path') + await expect(ctx.subprocess.resolveExecutable('node_modules/.bin/server')) + .rejects.toThrow('is a relative path') + await expect(ctx.subprocess.resolveExecutable('dsh-command-that-does-not-exist', { PATH: '' })) + .rejects.toThrow('was not found on PATH') + await expect(ctx.subprocess.resolveExecutable('/dsh-absolute-command-that-does-not-exist')) + .rejects.toThrow('is not an executable file') + await expect(ctx.subprocess.resolveExecutable(process.cwd())) + .rejects.toThrow('is not an executable file') + await expect(ctx.subprocess.resolveExecutable(process.execPath, {}, AbortSignal.abort('stop'))) + .rejects.toBe('stop') + await fiber.dispose() + }) + + it('builds Windows executable candidates with case-insensitive overrides', async () => { + const ctx = new Context() + const fiber = await ctx.plugin(LocalSubprocessService) + const service = ctx.subprocess as LocalSubprocessService + const candidates = (service as unknown as { + executableCandidates(command: string, env: NodeJS.ProcessEnv): string[] + }).executableCandidates.bind(service) + const platform = vi.spyOn(process, 'platform', 'get').mockReturnValue('win32') + try { + expect(Object.keys(childEnv()).filter(key => key.toUpperCase() === 'PATH')).toHaveLength(1) + const explicit = childEnv({ Path: '/bin', PathExt: '.EXE;.CMD' }) + expect(Object.keys(explicit).filter(key => key.toUpperCase() === 'PATH')).toEqual(['Path']) + expect(Object.keys(explicit).filter(key => key.toUpperCase() === 'PATHEXT')).toEqual(['PathExt']) + expect(candidates('tool', explicit)).toEqual(['/bin/tool.EXE', '/bin/tool.CMD']) + expect(candidates('tool', { Path: '/ambient', PATH: '/explicit', PATHEXT: '.EXE' })) + .toEqual(['/explicit/tool.EXE']) + expect(candidates('tool.exe', {})).toEqual([resolve(process.cwd(), 'tool.exe')]) + expect(candidates('tool', { PATH: '/bin' })).toHaveLength(4) + await expect(ctx.subprocess.resolveExecutable(String.raw`bin\server.exe`)) + .rejects.toThrow('is a relative path') + } finally { + platform.mockRestore() + await fiber.dispose() + } + }) + + it('validates terminal allocation inputs before allocating a PTY', async () => { + const ctx = new Context() + const fiber = await ctx.plugin(LocalSubprocessService) + const base: SubprocessTerminalSpawnSpec = { + argv: ['bash'], cwd: process.cwd(), rows: 24, cols: 80, graceMs: 10, + } + await expect(ctx.subprocess.spawnTerminal({ ...base, argv: [] })).rejects.toThrow('must contain a program') + await expect(ctx.subprocess.spawnTerminal({ ...base, argv: [''] })).rejects.toThrow('must contain a program') + await expect(ctx.subprocess.spawnTerminal({ ...base, signal: AbortSignal.abort('stop') })).rejects.toBe('stop') + await fiber.dispose() + }) + + it('terminates and joins an owned terminal during disposal', async () => { + const ctx = new Context() + const fiber = await ctx.plugin(LocalSubprocessService) + const terminate = vi.fn(async () => {}) + const terminal: SubprocessTerminalHandle = { + pid: 1, + output: new PassThrough(), + done: Promise.resolve({ exitCode: 0, signal: null }), + write: async () => {}, + inspectForeground: async () => undefined, + signalForeground: async () => 1, + terminate, + } + const terminals = (ctx.subprocess as unknown as { terminals: Set }).terminals + terminals.add(terminal) + await fiber.dispose() + expect(terminate).toHaveBeenCalledOnce() + expect(terminals.size).toBe(0) + }) + + it('waits for every terminal cleanup and aggregates teardown failures', async () => { + const ctx = new Context() + const fiber = await ctx.plugin(LocalSubprocessService) + const service = ctx.subprocess + const firstFailure = new Error('first cleanup failure') + const secondFailure = new Error('second cleanup failure') + const disposalErrors: unknown[] = [] + ctx.logger.error = ((error: unknown) => { disposalErrors.push(error) }) as typeof ctx.logger.error + const failedTerminal: SubprocessTerminalHandle = { + pid: 1, + output: new PassThrough(), + done: Promise.resolve({ exitCode: 0, signal: null }), + write: async () => {}, + inspectForeground: async () => undefined, + signalForeground: async () => 1, + terminate: vi.fn(async () => { throw firstFailure }), + } + const secondFailedTerminal: SubprocessTerminalHandle = { + ...failedTerminal, + terminate: vi.fn(async () => { throw secondFailure }), + } + let finishCleanup!: () => void + const cleanup = new Promise((resolve) => { + finishCleanup = resolve + }) + const drainingTerminal: SubprocessTerminalHandle = { + ...failedTerminal, + terminate: vi.fn(() => cleanup), + } + const terminals = (service as unknown as { terminals: Set }).terminals + terminals.add(failedTerminal) + terminals.add(secondFailedTerminal) + terminals.add(drainingTerminal) + + let disposed = false + const disposing = fiber.dispose().then(() => { disposed = true }) + await new Promise(resolve => setImmediate(resolve)) + expect(disposed).toBe(false) + finishCleanup() + await disposing + expect(terminals.size).toBe(0) + expect(disposalErrors).toHaveLength(1) + expect(disposalErrors[0]).toMatchObject({ + errors: [firstFailure, secondFailure], + message: 'local subprocess teardown failed', + }) + }) + + it('reports one cleanup failure without wrapping it', async () => { + const ctx = new Context() + const failure = new Error('single cleanup failure') + const disposalErrors: unknown[] = [] + ctx.logger.error = ((error: unknown) => { disposalErrors.push(error) }) as typeof ctx.logger.error + const fiber = await ctx.plugin(LocalSubprocessService) + const service = ctx.subprocess + const terminal: SubprocessTerminalHandle = { + pid: 1, + output: new PassThrough(), + done: Promise.resolve({ exitCode: 0, signal: null }), + write: async () => {}, + inspectForeground: async () => undefined, + signalForeground: async () => 1, + terminate: vi.fn(async () => { throw failure }), + } + const terminals = (service as unknown as { terminals: Set }).terminals + terminals.add(terminal) + + await fiber.dispose() + + expect(disposalErrors).toEqual([failure]) + }) + + it('releases a terminal after top-level exit reaches quiescence', async () => { + let exitListener: ((event: { exitCode: number; signal?: number }) => void) | undefined + const inspector = { + foregroundPgid: () => undefined, + isStdinWaiting: () => false, + processTree: () => [], + processSession: () => [], + isAlive: () => false, + signalGroup: () => {}, + signalProcess: () => {}, + } + const terminal = { + pid: 123, + onData: () => ({ dispose: () => {} }), + onExit: (listener: (event: { exitCode: number; signal?: number }) => void) => { + exitListener = listener + return { dispose: () => {} } + }, + write: () => {}, + kill: () => {}, + } + vi.resetModules() + vi.doMock('node-pty', () => ({ spawn: () => terminal })) + vi.doMock('../src/process-inspector.ts', async importOriginal => ({ + ...await importOriginal(), + createProcessInspector: () => inspector, + })) + try { + const { default: IsolatedLocalSubprocessService } = await import('../src/index.ts') + const ctx = new Context() + const fiber = await ctx.plugin(IsolatedLocalSubprocessService) + const service = ctx.subprocess as InstanceType + const handle = await ctx.subprocess.spawnTerminal({ + argv: ['shell'], cwd: process.cwd(), rows: 24, cols: 80, graceMs: 1, + }) + expect((service as unknown as { terminals: Set }).terminals.size).toBe(1) + exitListener?.({ exitCode: 0 }) + await handle.done + await new Promise(resolve => setImmediate(resolve)) + expect((service as unknown as { terminals: Set }).terminals.size).toBe(0) + await fiber.dispose() + } finally { + vi.doUnmock('node-pty') + vi.doUnmock('../src/process-inspector.ts') + vi.resetModules() + } + }) + + it('retains a terminal whose automatic cleanup fails', async () => { + let exitListener: ((event: { exitCode: number; signal?: number }) => void) | undefined + const terminal = { + pid: 123, + onData: () => ({ dispose: () => {} }), + onExit: (listener: (event: { exitCode: number; signal?: number }) => void) => { + exitListener = listener + return { dispose: () => {} } + }, + write: () => {}, + kill: () => {}, + } + vi.resetModules() + vi.doMock('node-pty', () => ({ spawn: () => terminal })) + try { + const { default: IsolatedLocalSubprocessService } = await import('../src/index.ts') + const ctx = new Context() + const disposalErrors: unknown[] = [] + ctx.logger.error = ((error: unknown) => { disposalErrors.push(error) }) as typeof ctx.logger.error + const fiber = await ctx.plugin(IsolatedLocalSubprocessService) + const alive = new Set([124]) + ;(ctx.subprocess as InstanceType).terminalInspector = { + foregroundPgid: () => 123, + isStdinWaiting: () => false, + processTree: () => [{ pid: 123, started: 'shell' }, { pid: 124, started: 'child' }], + processSession: () => [], + isAlive: identity => alive.has(identity.pid), + signalGroup: () => {}, + signalProcess: () => {}, + } + const handle = await ctx.subprocess.spawnTerminal({ + argv: ['shell'], cwd: process.cwd(), rows: 24, cols: 80, graceMs: 1, + }) + exitListener?.({ exitCode: 0 }) + await handle.done + await new Promise(resolve => setTimeout(resolve, 10)) + expect((ctx.subprocess as unknown as { terminals: Set }).terminals.size).toBe(1) + await fiber.dispose() + expect(disposalErrors).toHaveLength(1) + } finally { + vi.doUnmock('node-pty') + vi.resetModules() + } + }) + it('registers as ctx.subprocess and spawns managed handles', async () => { const ctx = new Context() const fiber = await ctx.plugin(LocalSubprocessService) diff --git a/packages/pty/pty-local/tests/process-inspector.spec.ts b/packages/subprocess/subprocess-local/tests/process-inspector.spec.ts similarity index 88% rename from packages/pty/pty-local/tests/process-inspector.spec.ts rename to packages/subprocess/subprocess-local/tests/process-inspector.spec.ts index 218a3d77fe..c90a7b3490 100644 --- a/packages/pty/pty-local/tests/process-inspector.spec.ts +++ b/packages/subprocess/subprocess-local/tests/process-inspector.spec.ts @@ -1,6 +1,10 @@ import { describe, expect, it } from 'vitest' -import { createProcessInspector, parseProcStat } from '@deepseek-ai/dsh-pty-local/src/process-inspector.ts' -import type { ProcessInspectorInternals } from '@deepseek-ai/dsh-pty-local/src/process-inspector.ts' +import { + createProcessInspector, + linuxProcessGroupHasLiveMembers, + parseProcStat, +} from '@deepseek-ai/dsh-subprocess-local/src/process-inspector.ts' +import type { ProcessInspectorInternals } from '@deepseek-ai/dsh-subprocess-local/src/process-inspector.ts' function stat(pid: number, pgrp: number, session: number, tpgid: number, started: string, parentPid = 1, state = 'S'): string { const rest = [state, String(parentPid), String(pgrp), String(session), '99', String(tpgid)] @@ -63,6 +67,21 @@ function fakeInternals() { } describe('Linux process inspector', () => { + it('treats zombie-only process groups as quiescent and fails closed when unobservable', () => { + const fake = fakeInternals() + expect(linuxProcessGroupHasLiveMembers(77, fake.internals)).toBeUndefined() + + fake.dirs.set('/proc', ['self', '10', '11', '12']) + fake.files.set('/proc/10/stat', stat(10, 77, 10, -1, '500', 1, 'Z')) + fake.files.set('/proc/11/stat', stat(11, 77, 10, -1, '501', 1, 'X')) + fake.files.set('/proc/12/stat', stat(12, 88, 12, -1, '502')) + expect(linuxProcessGroupHasLiveMembers(77, fake.internals)).toBe(false) + expect(linuxProcessGroupHasLiveMembers(99, fake.internals)).toBeUndefined() + + fake.files.set('/proc/11/stat', stat(11, 77, 10, -1, '501')) + expect(linuxProcessGroupHasLiveMembers(77, fake.internals)).toBe(true) + }) + it('parses stat safely, captures only the rooted process tree, and signals identities', () => { expect(parseProcStat('bad')).toBeUndefined() expect(parseProcStat('1 () ')).toBeUndefined() @@ -86,6 +105,13 @@ describe('Linux process inspector', () => { { pid: 10, started: '500' }, ]) expect(inspector.processTree(99)).toEqual([]) + expect(inspector.processSession(30)).toEqual([ + { pid: 10, started: '500' }, + { pid: 11, started: '501' }, + { pid: 12, started: '502' }, + { pid: 13, started: '503' }, + ]) + expect(inspector.processSession(99)).toEqual([]) expect(inspector.isAlive({ pid: 10, started: '500' })).toBe(true) expect(inspector.isAlive({ pid: 10, started: 'old' })).toBe(false) inspector.signalGroup(40, 'SIGINT') @@ -197,6 +223,7 @@ describe('macOS process inspector', () => { { pid: 10, started: 'Mon Jul 21 10:00:00 2026' }, ]) expect(inspector.processTree(99)).toEqual([]) + expect(inspector.processSession(10)).toEqual([]) expect(inspector.isAlive({ pid: 11, started: 'Mon Jul 21 10:00:01 2026' })).toBe(true) inspector.signalGroup(55, 'SIGTSTP') inspector.signalProcess({ pid: 11, started: 'Mon Jul 21 10:00:01 2026' }, 'SIGKILL') @@ -216,6 +243,6 @@ describe('macOS process inspector', () => { expect(createProcessInspector('darwin', 'arm64', fake.internals).foregroundPgid(1)).toBeUndefined() fake.internals.exec = () => { throw new Error('gone') } expect(createProcessInspector('darwin', 'arm64', fake.internals).foregroundPgid(1)).toBeUndefined() - expect(() => createProcessInspector('win32', 'x64', fake.internals)).toThrow('unsupported platform win32') + expect(() => createProcessInspector('win32', 'x64', fake.internals)).toThrow('unsupported on platform win32') }) }) diff --git a/packages/subprocess/subprocess-local/tests/spawn.spec.ts b/packages/subprocess/subprocess-local/tests/spawn.spec.ts index 08b6b1dbf8..6bac46ac16 100644 --- a/packages/subprocess/subprocess-local/tests/spawn.spec.ts +++ b/packages/subprocess/subprocess-local/tests/spawn.spec.ts @@ -60,7 +60,7 @@ function spec(command: string, overrides: SpecOverrides = {}) { } } -/** Poll until a pid no longer exists (kill(pid, 0) throws ESRCH). */ +/** Poll until a pid no longer exists, or is only a zombie on Linux. */ async function waitGone(pid: number, timeoutMs = 5_000): Promise { const deadline = Date.now() + timeoutMs while (Date.now() < deadline) { @@ -69,6 +69,16 @@ async function waitGone(pid: number, timeoutMs = 5_000): Promise { } catch { return } + if (process.platform === 'linux') { + try { + const stat = readFileSync(`/proc/${pid}/stat`, 'utf8') + const state = stat.slice(stat.lastIndexOf(')') + 2, stat.lastIndexOf(')') + 3) + if (state === 'Z' || state === 'X') return + } catch (error: unknown) { + if ((error as NodeJS.ErrnoException).code === 'ENOENT') return + throw error + } + } await new Promise(resolve => setTimeout(resolve, 20)) } throw new Error(`pid ${pid} still alive after ${timeoutMs}ms`) @@ -268,6 +278,24 @@ describe('spawnSubprocess', () => { expect(result.signal).toBe('SIGTERM') }) + it('does not wait for a Linux group that has only zombie members', async () => { + const pidFile = join(spillDir, `zombie-group-${Date.now()}.pid`) + const running = spawnSubprocess(spec(`sleep 60 & echo $! > ${pidFile}; echo leader-done`, { graceMs: 100 }), { + platform: 'linux', + linuxProcessGroupHasLiveMembers: () => false, + }) + const descendant = await waitForPidFile(pidFile) + try { + await running.done + await expect(running.waitForExit()).resolves.toBe(true) + } finally { + // The confirmed-absent verdict is a permanent no-more-signals boundary, + // so terminate() must stay inert here; reap the live survivor directly. + process.kill(descendant, 'SIGKILL') + await waitGone(descendant) + } + }) + it('bounds inherited-pipe draining after the shell exits', async () => { const pidFile = join(spillDir, `pipe-holder-${Date.now()}.pid`) const started = Date.now() @@ -637,7 +665,7 @@ describe('tree-survivor escalation (terminate and bounded waits reach helpers th clearTimeout(timer) running.terminate() await expect(running.waitForExit()).resolves.toBe(true) - expect(() => process.kill(helper, 0)).toThrow() + await expect(waitGone(helper)).resolves.toBeUndefined() }) it('service teardown awaits tree survivors, not just handle settlement', async () => { @@ -654,8 +682,8 @@ describe('tree-survivor escalation (terminate and bounded waits reach helpers th const helper = await waitForPidFile(pidFile) await running.done await fiber.dispose() - // Teardown itself waited for the survivor to die. - expect(() => process.kill(helper, 0)).toThrow() + // Teardown itself waited for the survivor to become quiescent. + await expect(waitGone(helper)).resolves.toBeUndefined() }) }) diff --git a/packages/subprocess/subprocess-local/tests/terminal.spec.ts b/packages/subprocess/subprocess-local/tests/terminal.spec.ts new file mode 100644 index 0000000000..79501c7dc4 --- /dev/null +++ b/packages/subprocess/subprocess-local/tests/terminal.spec.ts @@ -0,0 +1,332 @@ +import { afterEach, describe, expect, it, vi } from 'vitest' +import type { IDisposable, IPty } from 'node-pty' +import { LocalTerminalHandle } from '@deepseek-ai/dsh-subprocess-local/src/terminal.ts' +import type { + ProcessIdentity, + ProcessInspector, +} from '@deepseek-ai/dsh-subprocess-local/src/process-inspector.ts' +import type { SubprocessTerminalSignal } from '@deepseek-ai/dsh-subprocess' + +class FakePty { + pid = 123 + readonly writes: string[] = [] + readonly kills: string[] = [] + autoExitOnKill = true + throwKill = false + onKill?: () => void + private readonly dataListeners = new Set<(data: string) => void>() + private readonly exitListeners = new Set<(event: { exitCode: number; signal?: number }) => void>() + + readonly onData = (listener: (data: string) => void): IDisposable => { + this.dataListeners.add(listener) + return { dispose: () => { this.dataListeners.delete(listener) } } + } + + readonly onExit = (listener: (event: { exitCode: number; signal?: number }) => void): IDisposable => { + this.exitListeners.add(listener) + return { dispose: () => { this.exitListeners.delete(listener) } } + } + + emitData(data: string): void { + for (const listener of this.dataListeners) listener(data) + } + + emitExit(exitCode = 0, signal?: number): void { + for (const listener of this.exitListeners) listener({ exitCode, ...signal === undefined ? {} : { signal } }) + } + + write(data: string): void { this.writes.push(data) } + + kill(signal?: string): void { + if (this.throwKill) throw new Error('process raced') + this.kills.push(signal ?? 'SIGHUP') + this.onKill?.() + if (this.autoExitOnKill) this.emitExit(0, signal === 'SIGKILL' ? 9 : 15) + } + + asPty(): IPty { + return this as unknown as IPty + } +} + +class FakeInspector implements ProcessInspector { + pgid: number | undefined = 456 + waiting = false + /** The shell's own row, present like the real /proc- and ps-backed scans; tests recycle or drop it. */ + root: ProcessIdentity | undefined = { pid: 123, started: 'shell' } + members: ProcessIdentity[] = [] + sessionMembers: ProcessIdentity[] = [] + readonly alive = new Set() + readonly groups: Array<[number, SubprocessTerminalSignal]> = [] + readonly processes: Array<[number, 'SIGTERM' | 'SIGKILL']> = [] + throwGroup = false + throwProcess = false + removeOnSignal = true + + foregroundPgid() { return this.pgid } + isStdinWaiting() { return this.waiting } + processTree() { return this.root === undefined ? this.members : [this.root, ...this.members] } + processSession() { return this.sessionMembers } + isAlive(identity: ProcessIdentity) { return this.alive.has(identity.pid) } + signalGroup(pgid: number, signal: SubprocessTerminalSignal) { + if (this.throwGroup) throw new Error('group failed') + this.groups.push([pgid, signal]) + } + signalProcess(identity: ProcessIdentity, signal: 'SIGTERM' | 'SIGKILL') { + if (this.throwProcess) throw new Error('process raced') + this.processes.push([identity.pid, signal]) + if (this.removeOnSignal) this.alive.delete(identity.pid) + } +} + +afterEach(() => { vi.useRealTimers() }) + +describe('LocalTerminalHandle', () => { + it('bridges terminal bytes, foreground control, and signalled exit facts', async () => { + const pty = new FakePty() + const inspector = new FakeInspector() + inspector.waiting = true + const handle = new LocalTerminalHandle(pty.asPty(), inspector, 10) + const chunks: Buffer[] = [] + handle.output.on('data', (chunk: Buffer) => { chunks.push(chunk) }) + + pty.emitData('hello €') + await handle.write('input\r') + expect(pty.writes).toEqual(['input\r']) + expect(await handle.inspectForeground()).toEqual({ processGroupId: 456, inputWaiting: true }) + expect(await handle.signalForeground('SIGINT')).toBe(456) + expect(inspector.groups).toEqual([[456, 'SIGINT']]) + + pty.emitExit(7, 9) + pty.emitExit(0) + expect(await handle.done).toEqual({ exitCode: null, signal: 'SIGKILL' }) + await handle.terminate() + expect(Buffer.concat(chunks).toString('utf8')).toBe('hello €') + }) + + it('rejects unsafe foreground signals and writes after exit', async () => { + const pty = new FakePty() + const inspector = new FakeInspector() + const handle = new LocalTerminalHandle(pty.asPty(), inspector, 10) + inspector.pgid = handle.pid + await expect(handle.signalForeground('SIGKILL')).rejects.toThrow('terminate the terminal session') + inspector.pgid = undefined + expect(await handle.inspectForeground()).toBeUndefined() + await expect(handle.signalForeground('SIGTERM')).rejects.toThrow('cannot resolve') + + pty.emitExit(3) + expect(await handle.done).toEqual({ exitCode: 3, signal: null }) + await handle.terminate() + await expect(handle.write('late')).rejects.toThrow('has exited') + }) + + it('keeps the shell alive until forced descendants leave', async () => { + vi.useFakeTimers() + const pty = new FakePty() + const inspector = new FakeInspector() + inspector.members = [{ pid: 124, started: 'child' }] + inspector.alive.add(124) + inspector.removeOnSignal = false + const handle = new LocalTerminalHandle(pty.asPty(), inspector, 20) + + const quiescent = handle.terminate() + expect(handle.terminate()).toBe(quiescent) + await vi.advanceTimersByTimeAsync(20) + expect(inspector.processes).toContainEqual([124, 'SIGKILL']) + expect(pty.kills).toEqual([]) + + inspector.alive.delete(124) + await vi.advanceTimersByTimeAsync(20) + await quiescent + expect(pty.kills).toEqual(['SIGTERM']) + }) + + it('keeps an early exit wait pending through descendant cleanup', async () => { + vi.useFakeTimers() + const pty = new FakePty() + const inspector = new FakeInspector() + inspector.members = [{ pid: 124, started: 'child' }] + inspector.alive.add(124) + inspector.removeOnSignal = false + const handle = new LocalTerminalHandle(pty.asPty(), inspector, 20) + pty.emitExit() + const waiting = handle.terminate() + let settled = false + void waiting.then(() => { settled = true }) + await vi.advanceTimersByTimeAsync(10) + expect(settled).toBe(false) + + inspector.alive.delete(124) + await vi.advanceTimersByTimeAsync(20) + await waiting + }) + + it('cleans a same-session descendant after the top-level shell exits naturally', async () => { + const pty = new FakePty() + const inspector = new FakeInspector() + const disowned = { pid: 124, started: 'disowned' } + inspector.processSession = () => inspector.alive.has(disowned.pid) ? [disowned] : [] + inspector.alive.add(124) + const handle = new LocalTerminalHandle(pty.asPty(), inspector, 20) + + pty.emitExit() + + await handle.terminate() + expect(inspector.processes).toEqual([[124, 'SIGTERM']]) + }) + + it('retains an inspected descendant after it reparents away from the shell', async () => { + const pty = new FakePty() + const inspector = new FakeInspector() + const descendant = { pid: 124, started: 'observed' } + inspector.members = [descendant] + inspector.alive.add(descendant.pid) + const handle = new LocalTerminalHandle(pty.asPty(), inspector, 20) + + await handle.inspectForeground() + inspector.members = [] + pty.emitExit() + + await handle.terminate() + expect(inspector.processes).toEqual([[124, 'SIGTERM']]) + }) + + it('does not adopt the children of a recycled shell pid', async () => { + const pty = new FakePty() + const inspector = new FakeInspector() + const handle = new LocalTerminalHandle(pty.asPty(), inspector, 10) + + pty.emitExit() + const imposterChild = { pid: 999, started: 'imposter-child' } + inspector.root = { pid: 123, started: 'imposter' } + inspector.members = [imposterChild] + inspector.alive.add(imposterChild.pid) + + await handle.terminate() + expect(inspector.processes).toEqual([]) + }) + + it('adopts nothing when the shell identity was never observable', async () => { + const pty = new FakePty() + const inspector = new FakeInspector() + inspector.root = undefined + const orphan = { pid: 321, started: 'unverifiable' } + inspector.members = [orphan] + inspector.alive.add(orphan.pid) + const handle = new LocalTerminalHandle(pty.asPty(), inspector, 10) + + await handle.terminate() + expect(inspector.processes).toEqual([]) + expect(pty.kills).toEqual(['SIGTERM']) + }) + + it('rescans for descendants forked during TERM', async () => { + const pty = new FakePty() + const inspector = new FakeInspector() + const root = { pid: 123, started: 'shell' } + let reads = 0 + inspector.processTree = () => { + reads += 1 + if (reads === 1) return [root] + if (reads === 2) { + inspector.alive.add(124) + return [root, { pid: 124, started: 'first' }] + } + if (reads === 3) { + inspector.alive.add(125) + return [root, { pid: 125, started: 'late' }] + } + return [] + } + const handle = new LocalTerminalHandle(pty.asPty(), inspector, 10) + await handle.terminate() + expect(inspector.processes).toEqual([[124, 'SIGTERM'], [125, 'SIGKILL']]) + expect(pty.kills).toEqual(['SIGTERM']) + }) + + it('sweeps a same-session descendant forked while the shell handles TERM', async () => { + const pty = new FakePty() + const inspector = new FakeInspector() + const late = { pid: 124, started: 'shell-term-trap' } + pty.onKill = () => { + inspector.sessionMembers = [late] + inspector.alive.add(late.pid) + } + const handle = new LocalTerminalHandle(pty.asPty(), inspector, 10) + + await handle.terminate() + + expect(inspector.processes).toEqual([[late.pid, 'SIGTERM']]) + expect(pty.kills).toEqual(['SIGTERM']) + }) + + it('retries failed cleanup after a surviving descendant leaves', async () => { + vi.useFakeTimers() + const pty = new FakePty() + const inspector = new FakeInspector() + const late = { pid: 124, started: 'shell-term-survivor' } + inspector.removeOnSignal = false + pty.onKill = () => { + inspector.sessionMembers = [late] + inspector.alive.add(late.pid) + } + const handle = new LocalTerminalHandle(pty.asPty(), inspector, 10) + + const first = handle.terminate() + const failed = expect(first).rejects.toThrow('surviving pids: 124') + await vi.advanceTimersByTimeAsync(25) + await failed + + inspector.alive.delete(late.pid) + const retry = handle.terminate() + expect(retry).not.toBe(first) + await retry + expect(inspector.processes).toEqual([[late.pid, 'SIGTERM'], [late.pid, 'SIGKILL']]) + }) + + it('retains captured descendants after reparenting', async () => { + vi.useFakeTimers() + const pty = new FakePty() + const inspector = new FakeInspector() + const captured = { pid: 124, started: 'captured' } + const root = { pid: 123, started: 'shell' } + let reads = 0 + inspector.alive.add(captured.pid) + inspector.processTree = () => { reads += 1; return reads === 1 ? [root] : reads === 2 ? [root, captured] : [] } + inspector.signalProcess = (identity, signal) => { + inspector.processes.push([identity.pid, signal]) + if (signal === 'SIGKILL') inspector.alive.delete(identity.pid) + } + const handle = new LocalTerminalHandle(pty.asPty(), inspector, 20) + const quiescent = handle.terminate() + await vi.advanceTimersByTimeAsync(25) + await quiescent + expect(inspector.processes).toEqual([[124, 'SIGTERM'], [124, 'SIGKILL']]) + }) + + it('reports a top-level process that ignores escalation', async () => { + vi.useFakeTimers() + const pty = new FakePty() + pty.autoExitOnKill = false + const handle = new LocalTerminalHandle(pty.asPty(), new FakeInspector(), 10) + const failed = expect(handle.terminate()).rejects.toThrow('surviving pid: 123') + await vi.advanceTimersByTimeAsync(25) + await failed + expect(pty.kills).toEqual(['SIGTERM', 'SIGKILL']) + + pty.emitExit(0, 999) + expect(await handle.done).toEqual({ exitCode: null, signal: null }) + await handle.terminate() + }) + + it('contains process races while reporting surviving descendants', async () => { + const pty = new FakePty() + pty.throwKill = true + const inspector = new FakeInspector() + inspector.members = [{ pid: 124, started: 'child' }] + inspector.alive.add(124) + inspector.throwProcess = true + const handle = new LocalTerminalHandle(pty.asPty(), inspector, 1) + await expect(handle.terminate()).rejects.toThrow('surviving pids: 124') + }) +}) diff --git a/packages/subprocess/subprocess/README.i18n.yaml b/packages/subprocess/subprocess/README.i18n.yaml index b178daafb2..e67bf1a87f 100644 --- a/packages/subprocess/subprocess/README.i18n.yaml +++ b/packages/subprocess/subprocess/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/subprocess/subprocess/README.md -README.md: e59dd96df036826f36bd0286c977438d2d87d1cf -README.zh.md: e8fb89dfd1f8c41a0caefc96469c13d9ae7d415d +README.md: ec4a4e3328a5a600441d2e7983b3844f4ccc5e91 +README.zh.md: 3da79995fad9a2c8f2a6020ae18152a0e4459c71 diff --git a/packages/subprocess/subprocess/README.md b/packages/subprocess/subprocess/README.md index e59dd96df0..ec4a4e3328 100644 --- a/packages/subprocess/subprocess/README.md +++ b/packages/subprocess/subprocess/README.md @@ -2,15 +2,17 @@ English | [中文](README.zh.md) -The subprocess seam (`ctx.subprocess`). The abstract `SubprocessService` exposes one method — `spawn(spec): SubprocessHandle` — plus the vocabulary shared by every consumer: the fully-explicit `SubprocessSpawnSpec`, `SubprocessHandle` with its non-consuming offset-based output readers, `SubprocessOutcome`, `CollectedOutput`, and the managed `DSH_*` environment namespace (`DSH_ENV_PREFIX`, `DshEnvironment`). The local implementation lives in [`dsh-subprocess-local`](../subprocess-local/README.md). +The subprocess seam (`ctx.subprocess`) is the process half of one execution world. The abstract `SubprocessService` exposes executable lookup, ordinary managed `spawn`, and one terminal-process primitive; its vocabulary covers raw/collected stdio, process and terminal handles, exit facts, tree/session cleanup, and the managed `DSH_*` environment namespace. The local implementation lives in [`dsh-subprocess-local`](../subprocess-local/README.md). ## Contract - `spawn(spec)` returns immediately with a live handle; `done` resolves at process close with exit facts (`SubprocessOutcome` carries no output and no cause classification) and rejects only for spawn-level failures. -- The spec is fully explicit — argv, cwd, per-stream stdio dispositions, grace — because deployment-varying defaults belong to the calling seam's config, not to a hidden subprocess-service default (the `dsh-bash` request/spec split is the owning template). Grace must be positive, finite, and no greater than [`MAX_TIMER_DELAY_MS`](../../util/timeout/README.md), so the implementation can represent it with one Node timer instead of accepting a value that Node collapses to one millisecond. `argv` is never shell-interpreted; a consumer that wants a shell passes `['bash', '-c', command]` itself. +- Spawn working directories and executable paths belong to the provider's execution world. `resolveExecutable(command, env?, signal?)` verifies absolute commands or resolves bare names against that world's scrubbed PATH plus explicit overrides. +- The spec is fully explicit — argv, cwd, per-stream stdio dispositions, grace — because deployment-varying defaults belong to the calling seam's config, not to a hidden subprocess-service default (the `dsh-bash` request/spec split is the owning template). `argv` is never shell-interpreted; a consumer that wants a shell passes `['bash', '-c', command]` itself. - Stdio is Node-shaped per stream: `'pipe'` hands the caller the raw stream for its own protocol framing (LSP JSON-RPC, ACP ndjson), `'inherit'` passes the parent descriptor through for diagnostics, and collect mode (`{ maxBytes, spill? }`) buffers a bounded tail with an optional full-stream spill file. Collect readers take whole-stream byte offsets and never consume, so independent readers cannot steal one another's deltas; a read whose offset slid out of the in-memory tail is `lossy` and points at the spill file when one exists. Collected output stays readable after settlement. - Termination is tree-scoped on every platform (POSIX detached groups with direct-child fallback; Windows `taskkill /T`): `terminate()` — the only termination verb — escalates SIGTERM→grace→SIGKILL (idempotent, driven by the spec's abort signal too, a no-op once the tree is gone), and `waitForExit(signal?)` observes whole-tree liveness so a consumer-owned teardown ladder holds each tier on real quiescence — the manager reacts but never classifies why (callers own deadlines, teardown ladders, and cause classification). -- `scrubbedParentEnv()` / `SENSITIVE_ENV_PATTERN` are the one shared scrub definition: ambient credential-shaped and `DSH_*` names are dropped, and the spec's explicit `env` merges after the scrub with no namespace validation — a string deliberately forwards or overrides a value, while an `undefined` tombstone removes an ordinary ambient entry. Spawners that cannot route through the service (node-pty backends, SDK-managed transports) import the scrub. +- `spawnTerminal(spec)` is the only non-pipe primitive. Its handle owns a real PTY, UTF-8 text I/O, foreground-process-group inspection/signalling, and one awaited `terminate()` operation that reaches quiescence for every session member the provider can still observe and settles in-flight handle calls; providers document substrate-specific observability limits. The spec signal cancels allocation only; the published handle owns its lifetime. The output stream ends after queued output when the top-level process exits, and a live transport failure rejects `done`. These operations remain one substrate primitive because ordinary pipes cannot allocate a controlling terminal or clean terminal-session members; readiness, scrollback, and owner policy remain in the PTY consumer. +- `scrubbedParentEnv()` / `SENSITIVE_ENV_PATTERN` are the one shared scrub definition: ambient credential-shaped and `DSH_*` names are dropped, and explicit `env` merges after the scrub. The local ordinary and terminal spawns both apply it; SDK-managed transports that own their spawn may import it directly. - Disposal of the service terminates all still-running managed processes and awaits their exit. See the [subprocess data-structure catalog](../../../docs/core-data-structures/subprocess.md) and the [seam Agent Note](../../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md). @@ -25,5 +27,5 @@ No direct invalidation; the named consumers own any request-prefix changes. ## Known Limitations and Deferred Work -- **node-pty and SDK-managed spawns share only the scrub** — the PTY backend's terminal fork and the MCP SDK's own stdio transport cannot route their spawns through this seam (the library owns the fork/spawn call); they import `scrubbedParentEnv` so the environment policy stays single-sourced. +- **SDK-managed spawns remain outside** — an SDK transport that owns its internal spawn cannot route that call through this service; it can still import `scrubbedParentEnv` so environment policy stays single-sourced. - **Teardown ladders are consumer-owned** — the seam ships signalling verbs and the tree-liveness wait, not a canned quiesce sequence; each out-of-process consumer encodes its child's cooperation shape itself (the ACP backend's stdin-EOF-first ladder is the in-repo template). diff --git a/packages/subprocess/subprocess/README.zh.md b/packages/subprocess/subprocess/README.zh.md index e8fb89dfd1..3da79995fa 100644 --- a/packages/subprocess/subprocess/README.zh.md +++ b/packages/subprocess/subprocess/README.zh.md @@ -2,18 +2,20 @@ [English](README.md) | 中文 -子进程 seam（`ctx.subprocess`）。抽象的 `SubprocessService` 只暴露一个方法：`spawn(spec): SubprocessHandle`，外加所有消费方共享的词汇：完全显式的 `SubprocessSpawnSpec`、携带基于偏移量的非消费式输出读取器的 `SubprocessHandle`、`SubprocessOutcome`、`CollectedOutput`，以及受管的 `DSH_*` 环境命名空间（`DSH_ENV_PREFIX`、`DshEnvironment`）。本地实现位于 [`dsh-subprocess-local`](../subprocess-local/README.md)。 +子进程 seam（`ctx.subprocess`）是一个执行世界的进程部分。抽象的 `SubprocessService` 公开可执行文件查找、普通受管 `spawn` 和一项终端进程原语；其词汇涵盖原始／收集式 stdio、进程与终端句柄、退出事实、进程树／会话清理，以及受管的 `DSH_*` 环境命名空间。本地实现位于 [`dsh-subprocess-local`](../subprocess-local/README.md)。 ## 契约 - `spawn(spec)` 立即返回一个活动句柄；`done` 在进程关闭时以退出事实 resolve（`SubprocessOutcome` 不携带输出，也不携带原因分类），仅在 spawn 层面失败时 reject。 -- spec 完全显式（argv、cwd、按流划分的 stdio 处置方式（disposition）、宽限期），因为随部署变化的默认值属于调用方 seam 的配置，而不属于某个隐藏的子进程默认值（`dsh-bash` 的 request/spec 拆分是这条规则的所属模板）。宽限期须为正有限值，且不得大于 [`MAX_TIMER_DELAY_MS`](../../util/timeout/README.md)，这样实现便可用一个 Node 定时器表示它，而不会接受会被 Node 折叠为 1 毫秒的值。`argv` 绝不经过 shell 解释；需要 shell 的消费方自行传入 `['bash', '-c', command]`。 +- spawn 工作目录和可执行文件路径属于提供方的执行世界。`resolveExecutable(command, env?, signal?)` 验证绝对命令，或根据该执行世界清理后的 PATH 加显式覆盖来解析裸名称。 +- spec 完全显式（argv、cwd、按流划分的 stdio 处置方式（disposition）、宽限期），因为随部署变化的默认值属于调用方 seam 的配置，而不属于某个隐藏的子进程默认值（`dsh-bash` 的 request/spec 拆分是这条规则的所属模板）。`argv` 绝不经过 shell 解释；需要 shell 的消费方自行传入 `['bash', '-c', command]`。 - stdio 按流采用 Node 风格：`'pipe'` 把原始流交给调用方做自己的协议分帧（LSP 的 JSON-RPC、ACP（Agent Client Protocol）的 ndjson），`'inherit'` 直通父进程描述符以承载诊断输出，收集模式（collect）`{ maxBytes, spill? }` 则缓冲一段有界尾部，外加可选的完整流 spill 文件。收集模式的读取器接受全流字节偏移量且从不消费，因此独立的读取器不会抢走彼此的增量；偏移量滑出内存尾部窗口的读取标记为 `lossy`，并在 spill 文件存在时指向它。收集到的输出在结算后仍可读取。 - 终止在每个平台上都以进程树为范围（POSIX 用 detached 进程组并以直接子进程回退；Windows 用 `taskkill /T`）：`terminate()`（唯一的终止动词）执行 SIGTERM→宽限期→SIGKILL 升级（幂等，也由 spec 的 abort 信号驱动，进程树消亡后为空操作）；`waitForExit(signal?)` 观察整棵进程树的存活状态，使消费方自有的拆卸阶梯能在真正完全停稳后才进入下一层。管理器只响应中止，但绝不判定原因（deadline、拆卸阶梯与原因分类归调用方所有）。 -- `scrubbedParentEnv()` / `SENSITIVE_ENV_PATTERN` 是唯一一份共享的环境清理定义：环境中形似凭据的名称与 `DSH_*` 名称都会被丢弃，spec 的显式 `env` 在清理后合并且不做命名空间校验——字符串会有意转发或覆盖某个值，而 `undefined` tombstone 则会删除普通的环境条目。无法把 spawn 路由到该服务的进程启动方（node-pty 后端、由 SDK 管理的传输层）会导入该环境清理定义。 +- `spawnTerminal(spec)` 是唯一的非管道原语。其句柄负责真实 PTY、UTF-8 文本 I/O、前台进程组检查／信号发送，以及一项须等待的 `terminate()` 操作；该操作会使提供方仍可观察到的每个会话成员完全停稳，并结算在途句柄调用；提供方会记录执行基底特有的可观察性限制。spec 信号只取消分配；句柄一经发布，便负责自身生命周期。顶层进程退出时，输出流在已排队输出之后结束；存活传输失败会拒绝 `done`。这些操作保留为一项执行基底原语，因为普通管道无法分配控制终端或清理终端会话成员；就绪状态、scrollback 和所有者策略仍归 PTY 消费方所有。 +- `scrubbedParentEnv()` / `SENSITIVE_ENV_PATTERN` 是唯一一份共享的环境清理定义：环境中形似凭据的名称与 `DSH_*` 名称都会被丢弃，显式 `env` 在清除之后合并。本地的普通 spawn 与终端 spawn 都应用该定义；拥有自身 spawn 的 SDK 管理传输可直接导入它。 - 服务自身的 dispose（资源释放）会终止所有仍在运行的受管进程并等待其退出。 -参见[子进程数据结构目录](../../../docs/core-data-structures/subprocess.md)与[seam Agent Note](../../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md)。 +参见[子进程数据结构目录](../../../docs/core-data-structures/subprocess.md)与[seam Agent Note（agent 决策记录）](../../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md)。 ## 模型体验 @@ -25,5 +27,5 @@ ## 已知限制与暂缓事项 -- **node-pty 与由 SDK 管理的 spawn 只共享环境清理**：PTY 后端的终端 fork 与 MCP SDK 自己的 stdio 传输层无法把 spawn 路由到这道 seam（fork/spawn 调用归库所有）；它们改为导入 `scrubbedParentEnv`，使环境策略保持单一来源。 +- **由 SDK 管理的 spawn 仍在服务之外**：拥有内部 spawn 的 SDK 传输无法把该调用路由到本服务；它仍可导入 `scrubbedParentEnv`，使环境策略保持单一来源。 - **拆卸阶梯归消费方所有**：该 seam 只提供信号动词与进程树存活等待，不提供现成的停稳序列；每个进程外消费方自行编码其子进程的配合方式（ACP 后端以 stdin EOF 打头的阶梯是仓库内模板）。 diff --git a/packages/subprocess/subprocess/src/index.ts b/packages/subprocess/subprocess/src/index.ts index d484faa24d..6b9904078f 100644 --- a/packages/subprocess/subprocess/src/index.ts +++ b/packages/subprocess/subprocess/src/index.ts @@ -1,10 +1,9 @@ /** - * The subprocess seam (`ctx.subprocess`): spawn fully-specified commands into - * managed process trees with Node-shaped stdio dispositions — raw pipes for - * protocol streams, inherit for diagnostics, bounded spill-backed collection - * for batch output — plus tree-scoped signalling. Command defaulting, shell - * semantics, deadlines, teardown ladders, framing, and presentation belong to - * consumers; the bash executor seam is the owning template. The local implementation lives in + * The subprocess seam (`ctx.subprocess`): execution-world executable lookup, + * fully specified managed process trees with raw or + * collected stdio, and one terminal-process primitive. Command defaulting, + * shell semantics, deadlines, protocol framing, terminal readiness, and + * presentation belong to consumers. The local implementation lives in * `@deepseek-ai/dsh-subprocess-local`. * @module @deepseek-ai/dsh-subprocess */ @@ -12,6 +11,7 @@ import { Context, Service } from 'cordis' import { DSH_ENV_PREFIX } from './types.ts' import type { SubprocessHandle, SubprocessSpawnSpec } from './types.ts' +import type { SubprocessTerminalHandle, SubprocessTerminalSpawnSpec } from './types.ts' export { DSH_ENV_PREFIX } from './types.ts' export type { @@ -28,6 +28,10 @@ export type { SubprocessSpawnSpec, SubprocessStdinMode, SubprocessStdio, + SubprocessTerminalForeground, + SubprocessTerminalHandle, + SubprocessTerminalSignal, + SubprocessTerminalSpawnSpec, } from './types.ts' /** @@ -74,6 +78,8 @@ declare module 'cordis' { * duplicate-service behavior). * * Implementations must honor these semantics: + * - Executable paths belong to one execution world shared with the mounted + * filesystem provider. * - {@link spawn} returns immediately with a live handle; `done` resolves at * process close with exit facts and rejects only for spawn-level failures. * - Collect-mode readers are offset-based and non-consuming, so independent @@ -87,12 +93,34 @@ declare module 'cordis' { * quiescence. * - Disposal of the service terminates all still-running managed processes * and awaits their exit. + * - {@link spawnTerminal} owns terminal allocation, text transport, + * foreground groups, signalling, and whole-session quiescence behind one + * awaited termination method; readiness and persistent-shell policy stay + * in the PTY consumer. Its output stream ends after queued terminal output + * when the top-level process exits. */ export abstract class SubprocessService extends Service { constructor(ctx: Context) { super(ctx, 'subprocess') } + /** + * Resolve one configured executable in this provider's execution world. + * Absolute paths are verified; bare names use the provider's scrubbed PATH + * plus explicit environment overrides. Relative paths containing separators + * are rejected: no current consumer defines which directory they would + * resolve against, so providers fail loud instead of guessing. + * @param command - absolute executable path or bare PATH name. + * @param env - explicit environment entries used for lookup. + * @param signal - aborts remote or local lookup. + * @returns a canonical executable path. + */ + abstract resolveExecutable( + command: string, + env?: Readonly>, + signal?: AbortSignal, + ): Promise + /** * Start one managed child process from a fully-specified spec; this seam * applies no defaults. @@ -100,6 +128,15 @@ export abstract class SubprocessService extends Service { * @returns the live process handle (streams/readers, signalling, outcome promise). */ abstract spawn(spec: SubprocessSpawnSpec): SubprocessHandle + + /** + * Allocate a real terminal and start one owned process session. This is the + * only non-pipe process primitive: implementations own terminal byte I/O, + * foreground groups, signals, and complete session-tree cleanup. + * @param spec - fully specified argv, cwd, environment, dimensions, grace, and allocation cancellation. + * @returns the live terminal handle after allocation succeeds. + */ + abstract spawnTerminal(spec: SubprocessTerminalSpawnSpec): Promise } export default SubprocessService diff --git a/packages/subprocess/subprocess/src/types.ts b/packages/subprocess/subprocess/src/types.ts index 6084cc8c8e..2aa2de0c91 100644 --- a/packages/subprocess/subprocess/src/types.ts +++ b/packages/subprocess/subprocess/src/types.ts @@ -192,3 +192,73 @@ export interface SubprocessHandle { */ waitForExit(signal?: AbortSignal): Promise } + +/** + * Signals supported by the terminal-process primitive. Kept member-identical + * to `PtySignal` in `@deepseek-ai/dsh-pty` without a cross-seam dependency; + * change both together. + */ +export type SubprocessTerminalSignal = 'SIGINT' | 'SIGTERM' | 'SIGKILL' | 'SIGTSTP' | 'SIGHUP' + +/** A fully specified terminal-process spawn. */ +export interface SubprocessTerminalSpawnSpec { + /** Executable and arguments; `argv[0]` is the program. */ + argv: readonly string[] + /** Working directory in this subprocess provider's execution world. */ + cwd: string + /** Explicit environment layered after the provider's ambient scrub. */ + env?: Record | undefined + /** Initial terminal row count. */ + rows: number + /** Initial terminal column count. */ + cols: number + /** TERM-to-KILL cleanup grace for the complete terminal session. */ + graceMs: number + /** Cancellation of terminal allocation; a published handle owns its later lifetime. */ + signal?: AbortSignal | undefined +} + +/** Current foreground process-group facts for one terminal. */ +export interface SubprocessTerminalForeground { + /** Foreground process-group id published by the terminal driver. */ + processGroupId: number + /** Whether the provider can currently prove that group is waiting on terminal input. */ + inputWaiting: boolean +} + +/** + * One live terminal process and its owned OS session. Terminal allocation, + * foreground-group inspection/signalling, and session-tree cleanup are one + * deep subprocess primitive because none can be reconstructed from ordinary + * piped stdio without substrate-specific process control. + */ +export interface SubprocessTerminalHandle { + /** Top-level terminal process id. */ + readonly pid: number + /** UTF-8 terminal output bytes in delivery order; ends after queued output when the terminal exits. */ + readonly output: Readable + /** Resolves when the top-level process exits; rejects only for a live transport failure. */ + readonly done: Promise + /** + * Write text to the terminal input. + * @param data - text to deliver without implicit newline conversion. + */ + write(data: string): Promise + /** + * Inspect the current foreground process group. + * @returns its id and input-wait fact, or undefined when no foreground group can be resolved. + */ + inspectForeground(): Promise + /** + * Deliver a signal to the current foreground process group. + * @param signal - permitted terminal signal. + * @returns the exact group id that received it. + */ + signalForeground(signal: SubprocessTerminalSignal): Promise + /** + * Idempotently terminate every terminal-session member the provider can still observe and await quiescence. + * After settlement, no write, inspection, or signal call remains in flight. + * Providers document substrate-specific observability limits. + */ + terminate(): Promise +} diff --git a/packages/subprocess/subprocess/tests/service.spec.ts b/packages/subprocess/subprocess/tests/service.spec.ts index d0ef5c9fd6..e4f770a9a3 100644 --- a/packages/subprocess/subprocess/tests/service.spec.ts +++ b/packages/subprocess/subprocess/tests/service.spec.ts @@ -1,7 +1,14 @@ import { describe, expect, it } from 'vitest' +import { PassThrough } from 'node:stream' import { Context } from 'cordis' import { scrubbedParentEnv, SubprocessService } from '@deepseek-ai/dsh-subprocess' -import type { SubprocessHandle, SubprocessOutputRead, SubprocessSpawnSpec } from '@deepseek-ai/dsh-subprocess' +import type { + SubprocessHandle, + SubprocessOutputRead, + SubprocessSpawnSpec, + SubprocessTerminalHandle, + SubprocessTerminalSpawnSpec, +} from '@deepseek-ai/dsh-subprocess' /** * Minimal concrete service: a hand-built handle. The seam is spawn-only — @@ -9,6 +16,10 @@ import type { SubprocessHandle, SubprocessOutputRead, SubprocessSpawnSpec } from * is all an implementation owes the abstract class. */ class StubSubprocessService extends SubprocessService { + async resolveExecutable(command: string): Promise { + return `/bin/${command}` + } + spawn(spec: SubprocessSpawnSpec): SubprocessHandle { const read: SubprocessOutputRead = { text: '', nextOffset: 0, lossy: false } const collected = spec.stdio.stdout !== 'pipe' && spec.stdio.stdout !== 'inherit' @@ -25,6 +36,18 @@ class StubSubprocessService extends SubprocessService { waitForExit: () => Promise.resolve(true), } } + + async spawnTerminal(spec: SubprocessTerminalSpawnSpec): Promise { + return { + pid: spec.argv.length, + output: new PassThrough(), + done: Promise.resolve({ exitCode: 0, signal: null }), + write: async () => {}, + inspectForeground: async () => ({ processGroupId: 1, inputWaiting: true }), + signalForeground: async () => 1, + terminate: async () => {}, + } + } } describe('SubprocessService seam', () => { diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index a39c8731b7..d9f811ee3c 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -433,6 +433,12 @@ importers: '@deepseek-ai/dsh-credentials-local': specifier: workspace:* version: link:../packages/credentials/credentials-local + '@deepseek-ai/dsh-e2b': + specifier: workspace:* + version: link:../packages/e2b/e2b + '@deepseek-ai/dsh-fs-e2b': + specifier: workspace:* + version: link:../packages/e2b/fs-e2b '@deepseek-ai/dsh-fs-local': specifier: workspace:* version: link:../packages/fs/fs-local @@ -580,6 +586,9 @@ importers: '@deepseek-ai/dsh-subagent-spawn': specifier: workspace:* version: link:../packages/subagent/subagent-spawn + '@deepseek-ai/dsh-subprocess-e2b': + specifier: workspace:* + version: link:../packages/e2b/subprocess-e2b '@deepseek-ai/dsh-subprocess-local': specifier: workspace:* version: link:../packages/subprocess/subprocess-local @@ -3219,6 +3228,62 @@ importers: specifier: ^4.0.0-rc.7 version: link:../../../vendor/cordis + packages/e2b/e2b: + dependencies: + e2b: + specifier: 2.29.1 + version: 2.29.1 + schemastery: + specifier: ^3.18.0 + version: link:../../../vendor/schemastery + devDependencies: + '@deepseek-ai/dsh-invariants': + specifier: workspace:^ + version: link:../../support/invariants + '@deepseek-ai/dsh-loader-smoke': + specifier: workspace:^ + version: link:../../support/loader-smoke + cordis: + specifier: ^4.0.0-rc.7 + version: link:../../../vendor/cordis + + packages/e2b/fs-e2b: + devDependencies: + '@deepseek-ai/dsh-e2b': + specifier: workspace:^ + version: link:../e2b + '@deepseek-ai/dsh-fs': + specifier: workspace:^ + version: link:../../fs/fs + '@deepseek-ai/dsh-invariants': + specifier: workspace:^ + version: link:../../support/invariants + cordis: + specifier: ^4.0.0-rc.7 + version: link:../../../vendor/cordis + + packages/e2b/subprocess-e2b: + dependencies: + schemastery: + specifier: ^3.18.0 + version: link:../../../vendor/schemastery + devDependencies: + '@deepseek-ai/dsh-e2b': + specifier: workspace:^ + version: link:../e2b + '@deepseek-ai/dsh-invariants': + specifier: workspace:^ + version: link:../../support/invariants + '@deepseek-ai/dsh-subprocess': + specifier: workspace:^ + version: link:../../subprocess/subprocess + '@deepseek-ai/dsh-timeout': + specifier: workspace:^ + version: link:../../util/timeout + cordis: + specifier: ^4.0.0-rc.7 + version: link:../../../vendor/cordis + packages/examples/acp-demo: devDependencies: '@cordisjs/plugin-include': @@ -4365,6 +4430,12 @@ importers: '@deepseek-ai/dsh-brand': specifier: workspace:^ version: link:../../util/brand + '@deepseek-ai/dsh-fs': + specifier: workspace:^ + version: link:../../fs/fs + '@deepseek-ai/dsh-fs-local': + specifier: workspace:^ + version: link:../../fs/fs-local '@deepseek-ai/dsh-invariants': specifier: workspace:^ version: link:../../support/invariants @@ -4402,6 +4473,9 @@ importers: '@deepseek-ai/dsh-agent': specifier: workspace:^ version: link:../../core/agent + '@deepseek-ai/dsh-fs-local': + specifier: workspace:^ + version: link:../../fs/fs-local '@deepseek-ai/dsh-invariants': specifier: workspace:^ version: link:../../support/invariants @@ -4417,6 +4491,9 @@ importers: '@deepseek-ai/dsh-session': specifier: workspace:^ version: link:../../core/session + '@deepseek-ai/dsh-subprocess-local': + specifier: workspace:^ + version: link:../../subprocess/subprocess-local '@deepseek-ai/dsh-system-prompt': specifier: workspace:^ version: link:../../core/system-prompt @@ -4598,9 +4675,6 @@ importers: packages/pty/pty-local: dependencies: - node-pty: - specifier: ^1.1.0 - version: 1.1.0(patch_hash=7a0c04f1f49d798a9ffe2f7f414c01064a44ca2489772d0c3e1235ab336755e6) schemastery: specifier: ^3.18.0 version: link:../../../vendor/schemastery @@ -4626,6 +4700,9 @@ importers: '@deepseek-ai/dsh-subprocess': specifier: workspace:^ version: link:../../subprocess/subprocess + '@deepseek-ai/dsh-subprocess-local': + specifier: workspace:^ + version: link:../../subprocess/subprocess-local cordis: specifier: ^4.0.0-rc.7 version: link:../../../vendor/cordis @@ -4666,6 +4743,9 @@ importers: '@deepseek-ai/dsh-session': specifier: workspace:^ version: link:../../core/session + '@deepseek-ai/dsh-subprocess-local': + specifier: workspace:^ + version: link:../../subprocess/subprocess-local '@deepseek-ai/dsh-system-prompt': specifier: workspace:^ version: link:../../core/system-prompt @@ -4718,6 +4798,9 @@ importers: '@deepseek-ai/dsh-session': specifier: workspace:^ version: link:../../core/session + '@deepseek-ai/dsh-subprocess-local': + specifier: workspace:^ + version: link:../../subprocess/subprocess-local '@deepseek-ai/dsh-system-prompt': specifier: workspace:^ version: link:../../core/system-prompt @@ -6095,6 +6178,10 @@ importers: version: link:../../../vendor/cordis packages/subprocess/subprocess-local: + dependencies: + node-pty: + specifier: ^1.1.0 + version: 1.1.0(patch_hash=7a0c04f1f49d798a9ffe2f7f414c01064a44ca2489772d0c3e1235ab336755e6) devDependencies: '@deepseek-ai/dsh-invariants': specifier: workspace:^ @@ -7990,6 +8077,9 @@ packages: resolution: {integrity: sha512-ctxtJ/eA+t+6q2++vj5j7FYX3nRu311q1wfYH3xjlLOsczhlhxAg2FWNUXhpGvAw3BWo1xBcvOV6/YLc2r5FJw==} hasBin: true + '@bufbuild/protobuf@2.13.0': + resolution: {integrity: sha512-acq7c49vxfm1ggJ95P70TX7ABDM0vxr1SYD3BB0o0jnBLB4OAqeHyKuN+cD3w80gXEDQ2zxHpR6CUeA+O/aU9g==} + '@chevrotain/types@11.1.2': resolution: {integrity: sha512-U+HFai5+zmJCkK86QsaJtoITlboZHBqrVketcO2ROv865xfCMSFpELQoz1GkX5GzME8pTa+3kbKrZHQtI0gdbw==} @@ -8001,6 +8091,17 @@ packages: resolution: {integrity: sha512-y7/yvZ2TPAnR9+jnc00klvNNLkJiXFFrQA/hlLCcxA9a2A4zQIOimyFQ9XfwYKiGD1fb5GY8vbKIIgO8d5Tb2A==} engines: {node: '>= 20.12.0'} + '@connectrpc/connect-web@2.0.0-rc.3': + resolution: {integrity: sha512-w88P8Lsn5CCsA7MFRl2e6oLY4J/5toiNtJns/YJrlyQaWOy3RO8pDgkz+iIkG98RPMhj2thuBvsd3Cn4DKKCkw==} + peerDependencies: + '@bufbuild/protobuf': ^2.2.0 + '@connectrpc/connect': 2.0.0-rc.3 + + '@connectrpc/connect@2.0.0-rc.3': + resolution: {integrity: sha512-ARBt64yEyKbanyRETTjcjJuHr2YXorzQo0etyS5+P6oSeW8xEuzajA9g+zDnMcj1hlX2dQE93foIWQGfpru7gQ==} + peerDependencies: + '@bufbuild/protobuf': ^2.2.0 + '@csstools/color-helpers@6.1.0': resolution: {integrity: sha512-064IFJdjTfUqnjpCVpMOdbr8FLQBhinbZj6yRv2An2E41O/pLEXqfFRWqGq/SxlE5PEUYTlvWsG2r8MswAVvkg==} engines: {node: '>=20.19.0'} @@ -8620,6 +8721,14 @@ packages: resolution: {integrity: sha512-O8jcjabXaleOG9DQ0+ARXWZBTfnP4WNAqzuiJK7ll44AmxGKv/J2M4TPjxjY3znBCfvBXFzucm1twdyFybFqEA==} engines: {node: '>=12'} + '@isaacs/cliui@9.0.0': + resolution: {integrity: sha512-AokJm4tuBHillT+FpMtxQ60n8ObyXBatq7jD2/JA9dxbDDokKQm8KMht5ibGzLVU9IJDIKK4TPKgMHEYMn3lMg==} + engines: {node: '>=18'} + + '@isaacs/fs-minipass@4.0.1': + resolution: {integrity: sha512-wgm9Ehl2jpeqP3zw/7mo3kRHFp5MEDhqAdwy1fTGkHAwnkGOVsgpvQhL8B5n1qlb01jV3n/bI0ZfZp5lWA1k4w==} + engines: {node: '>=18.0.0'} + '@joplin/turndown-plugin-gfm@1.0.67': resolution: {integrity: sha512-FZfW5EZfidhzd1IaY1uxHnIZPTVOxAdleMZ4/1U6Nt5b7+Qj5JThDnaIomuJtetnUBzuRNbe9FWMuqD4B3dlWA==} @@ -10405,6 +10514,10 @@ packages: resolution: {integrity: sha512-NUPRluOfOiTKBKvWPtSD4PhFvWCqOi0BGStNWs57X9js7XGTprSmFoz5F0tWhR4WPjNeR9jXqdC7/UpSJTnlRg==} engines: {node: '>=18'} + chalk@5.6.2: + resolution: {integrity: sha512-7NzBL0rN6fMUW+f7A6Io4h40qQlG+xGmtMxfbnH/K7TAtt8JQWVQK+6g0UXKMeVJoyV5EkkNsErQ8pVD3bLHbA==} + engines: {node: ^12.17.0 || ^14.13 || >=16.0.0} + character-entities-html4@2.1.0: resolution: {integrity: sha512-1v7fgQRj6hnSwFpq1Eu0ynr/CDEw0rXo2B61qXrLNdHZmPKgb7fqS1a2JwF0rISo9q77jDI8VMEHoApn8qDoZA==} @@ -10422,6 +10535,10 @@ packages: resolution: {integrity: sha512-TQMmc3w+5AxjpL8iIiwebF73dRDF4fBIieAqGn9RGCWaEVwQ6Fb2cGe31Yns0RRIzii5goJ1Y7xbMwo1TxMplw==} engines: {node: '>= 20.19.0'} + chownr@3.0.0: + resolution: {integrity: sha512-+IxzY9BZOQd/XuYPRmrvEVjF/nqj5kgT4kEq7VofrDoM1MxoRjEWkrCC3EtLi59TVawxTAn+orJwFQcrqEN1+g==} + engines: {node: '>=18'} + clsx@2.1.1: resolution: {integrity: sha512-eYm0QWBtUrBWZWG0d386OGAw16Z995PiOVo2B7bjWSbHedGl5e0ZWaq65kOGgUSNesEIDkB9ISbTg/JK9dhCZA==} engines: {node: '>=6'} @@ -10448,6 +10565,9 @@ packages: resolution: {integrity: sha512-OkTL9umf+He2DZkUq8f8J9of7yL6RJKI24dVITBmNfZBmri9zYZQrKkuXiKhyfPSu8tUhnVBB1iKXevvnlR4Ww==} engines: {node: '>= 12'} + compare-versions@6.1.1: + resolution: {integrity: sha512-4hm4VPpIecmlg59CHXnRDnqGplJFrbLG4aFEl5vl6cK1u76ws3LLvX7ikFnTDl5vo39sjWD6AaDPYodJp/NNHg==} + content-disposition@1.1.0: resolution: {integrity: sha512-5jRCH9Z/+DRP7rkvY83B+yGIGX96OYdJmzngqnw2SBSxqCFPd0w2km3s5iawpGX8krnwSGmF0FW5Nhr0Hfai3g==} engines: {node: '>=18'} @@ -10713,6 +10833,9 @@ packages: resolution: {integrity: sha512-svtcdpS8CgJyqAjEQIXdb3OjhFVVYjzGAPO8WGCmRbrml64SPw/jJD4GoE98aR7r25A0XcgrK3F02yw9R/vhQw==} engines: {node: '>=0.3.1'} + dockerfile-ast@0.7.1: + resolution: {integrity: sha512-oX/A4I0EhSkGqrFv0YuvPkBUSYp1XiY8O8zAKc8Djglx8ocz+JfOr8gP0ryRMC2myqvDLagmnZaU9ot1vG2ijw==} + dom-accessibility-api@0.5.16: resolution: {integrity: sha512-X7BJ2yElsnOJ30pZF4uIIDfBEVgF4XEBxL9Bxhy6dnrm5hkzqmsWHGTiHqRiITNhMyFLyAiWndIJP7Z1NTteDg==} @@ -10732,6 +10855,10 @@ packages: resolution: {integrity: sha512-KIN/nDJBQRcXw0MLVhZE9iQHmG68qAVIBg9CqmUYjmQIhgij9U5MFvrqkUL5FbtyyzZuOeOt0zdeRe4UY7ct+A==} engines: {node: '>= 0.4'} + e2b@2.29.1: + resolution: {integrity: sha512-n4aGNwRKTj2oct7BrOWfR4T+xGO834vbsrzfSlWUNJrhz615Lp+ad9hc8KtRaaHULKr/W/14Z6v4c7fqk3y0pg==} + engines: {node: '>=20.18.1'} + eastasianwidth@0.2.0: resolution: {integrity: sha512-I88TYZWc9XiYHRQ4/3c5rjjfgkjhLyW2luGIheGERbNQ6OY7yTybanSpDXZa8y7VUP9YmDcYa+eyq4ca7iLqWA==} @@ -11062,6 +11189,12 @@ packages: deprecated: Old versions of glob are not supported, and contain widely publicized security vulnerabilities, which have been fixed in the current version. Please update. Support for old versions may be purchased (at exorbitant rates) by contacting i@izs.me hasBin: true + glob@11.1.0: + resolution: {integrity: sha512-vuNwKSaKiqm7g0THUBu2x7ckSs3XJLXE+2ssL7/MfTGPLLcrJQ/4Uq1CjPTtO5cCIiRxqvN6Twy1qOwhL0Xjcw==} + engines: {node: 20 || >=22} + deprecated: Old versions of glob are not supported, and contain widely publicized security vulnerabilities, which have been fixed in the current version. Please update. Support for old versions may be purchased (at exorbitant rates) by contacting i@izs.me + hasBin: true + globals@17.7.0: resolution: {integrity: sha512-Czmyns5dUsq4seFBR/Kdydhmo8y9kC79hiSkPn0YcGtNnYWnrgt0vjrSjx9tspoDGWm2CMarffRuLjM4xUz8xg==} engines: {node: '>=18'} @@ -11245,6 +11378,10 @@ packages: jackspeak@3.4.3: resolution: {integrity: sha512-OGlZQpz2yfahA/Rd1Y8Cd9SIEsqvXkLVoSw/cgwhnhFMDbsQFeZYoJJ7bIZBS9BcamUW96asq/npPWugM+RQBw==} + jackspeak@4.2.3: + resolution: {integrity: sha512-ykkVRwrYvFm1nb2AJfKKYPr0emF6IiXDYUaFx4Zn9ZuIH7MrzEZ3sD5RlqGXNRpHtvUHJyOnCEFxOlNDtGo7wg==} + engines: {node: 20 || >=22} + jiti@2.7.0: resolution: {integrity: sha512-AC/7JofJvZGrrneWNaEnJeOLUx+JlGt7tNa0wZiRPT4MY1wmfKjt2+6O2p2uz2+skll8OZZmJMNqeke7kKbNgQ==} hasBin: true @@ -11741,6 +11878,10 @@ packages: minisearch@7.2.0: resolution: {integrity: sha512-dqT2XBYUOZOiC5t2HRnwADjhNS2cecp9u+TJRiJ1Qp/f5qjkeT5APcGPjHw+bz89Ms8Jp+cG4AlE+QZ/QnDglg==} + minizlib@3.1.0: + resolution: {integrity: sha512-KZxYo1BUkWD2TVFLr0MQoM8vUUigWD3LlD83a/75BqC+4qE0Hb1Vo5v1FgcfaNXvfXzr+5EhQ6ing/CaBijTlw==} + engines: {node: '>= 18'} + mitt@3.0.1: resolution: {integrity: sha512-vKivATfr97l2/QBCYAkXYDbrIWPM2IIKEl7YPhjCvKlG3kE2gm+uBo6nEXK3M5/Ffh/FLpKExzOQ3JJoJGFKBw==} @@ -11884,6 +12025,12 @@ packages: zod: optional: true + openapi-fetch@0.14.1: + resolution: {integrity: sha512-l7RarRHxlEZYjMLd/PR0slfMVse2/vvIAGm75/F7J6MlQ8/b9uUQmUF2kCPrQhJqMXSxmYWObVgeYXbFYzZR+A==} + + openapi-typescript-helpers@0.0.15: + resolution: {integrity: sha512-opyTPaunsklCBpTK8JGef6mfPhLSnyy5a0IN9vKtx3+4aExf+KxEqYwIy3hqkedXIB97u357uLMJsOnm3GVjsw==} + optionator@0.9.4: resolution: {integrity: sha512-6IpQ7mKUxRcZNLIObR0hz7lxsapSSIYNZJwXPGeF0mTVqGKFIXj1DQcMoT22S3ROcLyY/rz0PWaWZ9ayWmad9g==} engines: {node: '>= 0.8.0'} @@ -11970,6 +12117,10 @@ packages: resolution: {integrity: sha512-Xa4Nw17FS9ApQFJ9umLiJS4orGjm7ZzwUrwamcGQuHSzDyth9boKDaycYdDcZDuqYATXw4HFXgaqWTctW/v1HA==} engines: {node: '>=16 || 14 >=14.18'} + path-scurry@2.0.2: + resolution: {integrity: sha512-3O/iVVsJAPsOnpwWIeD+d6z/7PmqApyQePUtCndjatj/9I5LylHvt5qluFaBT3I5h3r1ejfR056c+FCv+NnNXg==} + engines: {node: 18 || 20 || >=22} + path-to-regexp@8.4.2: resolution: {integrity: sha512-qRcuIdP69NPm4qbACK+aDogI5CBDMi1jKe0ry5rSQJz8JVLsC7jV8XpiJjGRLLol3N+R5ihGYcrPLTno6pAdBA==} @@ -11990,6 +12141,9 @@ packages: resolution: {integrity: sha512-wQ0b/W4Fr01qtpHlqSqspcj3EhBvimsdh0KlHhH8HRZnMsEa0ea2fTULOXOS9ccQr3om+GcGRk4e+isrZWV8qQ==} engines: {node: '>=16.20.0'} + platform@1.3.6: + resolution: {integrity: sha512-fnWVljUchTro6RiCFvCXBbNhJc2NijN7oIQxbwsyL0buWJPG85v81ehlHI9fXrJsMNgTofEoWIQeClKpgxFLrg==} + playwright-core@1.61.1: resolution: {integrity: sha512-h7Qlt6m4REp25qvIdvbDtVmD4LqVXfpRxhORv9L0jzETM05p4fuPJ3dKyuSXQxDSbXnmS79HAgi9589lGSpLkg==} engines: {node: '>=18'} @@ -12368,6 +12522,10 @@ packages: tabbable@6.5.0: resolution: {integrity: sha512-wieBHXygIm7OyQOu5hQlkk62/WyCFYGlWg7L6/ZCUZwx0o398Zkn4pVmMyfYhfMG8kGrj/Krt8eIk6UKC6VzwA==} + tar@7.5.22: + resolution: {integrity: sha512-MFO/QzvtAOmJbkhOaCTvbGcFN9L9b+JunIsDwaKljSOdcLMea3NJ1k9Usz/rjdfSXTq4dfzfeS7W4p4YOAAHeA==} + engines: {node: '>=18'} + tinybench@2.9.0: resolution: {integrity: sha512-0+DUvqWMValLmha6lr4kD8iAMK1HzV0/aKnCtWb9v9641TnP/MFb7Pc2bxoxQjTXAErryXVgUOfv2YqNllqGeg==} @@ -12771,6 +12929,9 @@ packages: vscode-languageserver-protocol@3.18.2: resolution: {integrity: sha512-XRyDbT0Pp3sSNti3JmxVEUMySWCSi1hhM+/KUlCy1hV1zmrqpM1OwO12EAki8blhmLuIMpaJrYbo0OzGVfK2Qg==} + vscode-languageserver-textdocument@1.0.12: + resolution: {integrity: sha512-cxWNPesCnQCcMPeenjKKsOCKQZ/L6Tv19DTRIGuLWe32lyzWhihGVJ/rcckZXJxfdKCFvRLS3fpBIsV/ZGX4zA==} + vscode-languageserver-types@3.18.0: resolution: {integrity: sha512-8TsGPNMIMiiBdkORgRSvLjuiEIiAFtO+KssmYWxQ+uSVvlf7RjK8YKCOjPzZ+YA04jXEV7+7LvkSmHkhpNS99g==} @@ -12865,6 +13026,10 @@ packages: yallist@3.1.1: resolution: {integrity: sha512-a4UGQaWPH59mOXUYnAG2ewncQS4i4F43Tv3JoAM+s2VDAmS9NsK8GpDMLrCHPksFT7h3K6TOoUNn2pb7RoXx4g==} + yallist@5.0.0: + resolution: {integrity: sha512-YgvUTfwqyc7UXVMrB+SImsVYSmTS8X/tSrtdNZMImM+n7+QTriRXyXim0mBrTXNeqzVF0KWGgHPeiyViFFrNDw==} + engines: {node: '>=18'} + yaml@2.9.0: resolution: {integrity: sha512-2AvhNX3mb8zd6Zy7INTtSpl1F15HW6Wnqj0srWlkKLcpYl/gMIMJiyuGq2KeI2YFxUPjdlB+3Lc10seMLtL4cA==} engines: {node: '>= 14.6'} @@ -13468,6 +13633,8 @@ snapshots: dependencies: css-tree: 3.2.1 + '@bufbuild/protobuf@2.13.0': {} + '@chevrotain/types@11.1.2': {} '@clack/core@1.4.3': @@ -13482,6 +13649,15 @@ snapshots: fast-wrap-ansi: 0.2.2 sisteransi: 1.0.5 + '@connectrpc/connect-web@2.0.0-rc.3(@bufbuild/protobuf@2.13.0)(@connectrpc/connect@2.0.0-rc.3(@bufbuild/protobuf@2.13.0))': + dependencies: + '@bufbuild/protobuf': 2.13.0 + '@connectrpc/connect': 2.0.0-rc.3(@bufbuild/protobuf@2.13.0) + + '@connectrpc/connect@2.0.0-rc.3(@bufbuild/protobuf@2.13.0)': + dependencies: + '@bufbuild/protobuf': 2.13.0 + '@csstools/color-helpers@6.1.0': {} '@csstools/css-calc@3.2.1(@csstools/css-parser-algorithms@4.0.0(@csstools/css-tokenizer@4.0.0))(@csstools/css-tokenizer@4.0.0)': @@ -13898,6 +14074,12 @@ snapshots: wrap-ansi: 8.1.0 wrap-ansi-cjs: wrap-ansi@7.0.0 + '@isaacs/cliui@9.0.0': {} + + '@isaacs/fs-minipass@4.0.1': + dependencies: + minipass: 7.1.3 + '@joplin/turndown-plugin-gfm@1.0.67': {} '@jridgewell/gen-mapping@0.3.13': @@ -15452,6 +15634,8 @@ snapshots: chai@6.2.2: {} + chalk@5.6.2: {} + character-entities-html4@2.1.0: {} character-entities-legacy@3.0.0: {} @@ -15466,6 +15650,8 @@ snapshots: dependencies: readdirp: 5.0.0 + chownr@3.0.0: {} + clsx@2.1.1: {} color-convert@2.0.1: @@ -15482,6 +15668,8 @@ snapshots: commander@8.3.0: {} + compare-versions@6.1.1: {} + content-disposition@1.1.0: {} content-type@1.0.5: {} @@ -15753,6 +15941,11 @@ snapshots: diff@9.0.0: {} + dockerfile-ast@0.7.1: + dependencies: + vscode-languageserver-textdocument: 1.0.12 + vscode-languageserver-types: 3.18.0 + dom-accessibility-api@0.5.16: {} dompurify@3.4.11: @@ -15769,6 +15962,20 @@ snapshots: es-errors: 1.3.0 gopd: 1.2.0 + e2b@2.29.1: + dependencies: + '@bufbuild/protobuf': 2.13.0 + '@connectrpc/connect': 2.0.0-rc.3(@bufbuild/protobuf@2.13.0) + '@connectrpc/connect-web': 2.0.0-rc.3(@bufbuild/protobuf@2.13.0)(@connectrpc/connect@2.0.0-rc.3(@bufbuild/protobuf@2.13.0)) + chalk: 5.6.2 + compare-versions: 6.1.1 + dockerfile-ast: 0.7.1 + glob: 11.1.0 + openapi-fetch: 0.14.1 + platform: 1.3.6 + tar: 7.5.22 + undici: 7.28.0 + eastasianwidth@0.2.0: {} ecdsa-sig-formatter@1.0.11: @@ -16231,6 +16438,15 @@ snapshots: package-json-from-dist: 1.0.1 path-scurry: 1.11.1 + glob@11.1.0: + dependencies: + foreground-child: 3.3.1 + jackspeak: 4.2.3 + minimatch: 10.2.5 + minipass: 7.1.3 + package-json-from-dist: 1.0.1 + path-scurry: 2.0.2 + globals@17.7.0: {} globrex@0.1.2: {} @@ -16400,6 +16616,10 @@ snapshots: optionalDependencies: '@pkgjs/parseargs': 0.11.0 + jackspeak@4.2.3: + dependencies: + '@isaacs/cliui': 9.0.0 + jiti@2.7.0: {} jose@6.2.3: {} @@ -17084,6 +17304,10 @@ snapshots: minisearch@7.2.0: {} + minizlib@3.1.0: + dependencies: + minipass: 7.1.3 + mitt@3.0.1: {} mri@1.2.0: {} @@ -17204,6 +17428,12 @@ snapshots: ws: 8.21.0 zod: 4.4.3 + openapi-fetch@0.14.1: + dependencies: + openapi-typescript-helpers: 0.0.15 + + openapi-typescript-helpers@0.0.15: {} + optionator@0.9.4: dependencies: deep-is: 0.1.4 @@ -17336,6 +17566,11 @@ snapshots: lru-cache: 10.4.3 minipass: 7.1.3 + path-scurry@2.0.2: + dependencies: + lru-cache: 11.5.1 + minipass: 7.1.3 + path-to-regexp@8.4.2: {} pathe@2.0.3: {} @@ -17348,6 +17583,8 @@ snapshots: pkce-challenge@5.0.1: {} + platform@1.3.6: {} + playwright-core@1.61.1: {} playwright@1.61.1: @@ -17801,6 +18038,14 @@ snapshots: tabbable@6.5.0: {} + tar@7.5.22: + dependencies: + '@isaacs/fs-minipass': 4.0.1 + chownr: 3.0.0 + minipass: 7.1.3 + minizlib: 3.1.0 + yallist: 5.0.0 + tinybench@2.9.0: {} tinyexec@1.2.4: {} @@ -18234,6 +18479,8 @@ snapshots: vscode-jsonrpc: 9.0.1 vscode-languageserver-types: 3.18.0 + vscode-languageserver-textdocument@1.0.12: {} + vscode-languageserver-types@3.18.0: {} vue@3.5.39(typescript@6.0.3): @@ -18305,6 +18552,8 @@ snapshots: yallist@3.1.1: {} + yallist@5.0.0: {} + yaml@2.9.0: {} yocto-queue@0.1.0: {} diff --git a/scripts/build-exe-for-python-sdk.ts b/scripts/build-exe-for-python-sdk.ts index db72f4c52e..536342dc89 100644 --- a/scripts/build-exe-for-python-sdk.ts +++ b/scripts/build-exe-for-python-sdk.ts @@ -326,7 +326,7 @@ class SingleExeBuild { if (this.cli.dryRun) console.log(`build-exe-for-python-sdk: [dry-run] rm -rf ${stagedBuild}`) else await rm(stagedBuild, { recursive: true, force: true }) if (target.platform !== 'linux') return - const source = join(root, 'packages', 'pty', 'pty-local', 'node_modules', 'node-pty', 'build', 'Release', 'pty.node') + const source = join(root, 'packages', 'subprocess', 'subprocess-local', 'node_modules', 'node-pty', 'build', 'Release', 'pty.node') const destination = join(stagedBuild, 'Release', 'pty.node') if (this.cli.dryRun) { console.log(`build-exe-for-python-sdk: [dry-run] cp ${source} ${destination}`) diff --git a/scripts/check-workspace-constraints.ts b/scripts/check-workspace-constraints.ts index 355b817e68..1c7bedc696 100644 --- a/scripts/check-workspace-constraints.ts +++ b/scripts/check-workspace-constraints.ts @@ -125,8 +125,8 @@ const packageFileExtras: Readonly> = { '@deepseek-ai/dsh-headless': ['cordis.patch.yml'], '@deepseek-ai/dsh-client-ui-theme': ['lib/styles'], '@deepseek-ai/dsh-helper': ['lib/assets'], - '@deepseek-ai/dsh-pty-local': ['scripts/ensure-spawn-helper.mjs'], '@deepseek-ai/dsh-skill-badge': ['assets'], + '@deepseek-ai/dsh-subprocess-local': ['scripts/ensure-spawn-helper.mjs'], '@deepseek-ai/dsh-scripts': [ 'lib/dev/tsdown-config.js', 'lib/local-plugin-loader-hooks.js', diff --git a/scripts/gen-cordis-catalog.ts b/scripts/gen-cordis-catalog.ts index 47fd89c8b8..46f8514153 100644 --- a/scripts/gen-cordis-catalog.ts +++ b/scripts/gen-cordis-catalog.ts @@ -73,6 +73,8 @@ export const LINK_MAP: Readonly> = { SubprocessOutputRead: 'subprocess.md', SubprocessOutputReader: 'subprocess.md', SubprocessSpawnSpec: 'subprocess.md', + SubprocessTerminalHandle: 'subprocess.md', + SubprocessTerminalSpawnSpec: 'subprocess.md', CodeRunRequest: 'code-runtime.md', CodeRunResult: 'code-runtime.md', CompactionResult: 'compaction.md', @@ -242,6 +244,7 @@ export const FOUNDATION_TYPE_NAMES: ReadonlySet = new Set([ 'Partial', 'Pick', 'Promise', + 'Record', 'Readonly', ]) @@ -299,6 +302,7 @@ export const TYPE_LINK_EXEMPTIONS: Readonly> = { PermissionSelect: 'permissions projection payload is owned by packages/ui/permission/src/types.ts', PromptAssembly: 'assembly result is owned by packages/core/system-prompt/README.md', ResumeAgentOptions: 'agent resume contract is owned by packages/core/agent/README.md', + Sandbox: 'external E2B SDK handle is owned by packages/e2b/e2b/README.md', SessionForkSource: 'service-local fork input is owned by packages/core/session/src/index.ts', SubagentRunEndInfo: 'event payload contract is owned by packages/subagent/subagent/src/types.ts', SubagentRunInfo: 'event payload contract is owned by packages/subagent/subagent/src/types.ts', diff --git a/scripts/gen-doc-graphs.ts b/scripts/gen-doc-graphs.ts index acecf2c47d..ddd5435363 100644 --- a/scripts/gen-doc-graphs.ts +++ b/scripts/gen-doc-graphs.ts @@ -70,6 +70,7 @@ const GROUP_ORDER = [ 'bash', 'pty', 'sandbox', + 'e2b', 'fs', 'skill', 'compact', @@ -328,14 +329,22 @@ const SERVICE_ROLES: ServiceRole[] = [ mode: 'core', note: 'Folds revisioned objective state from the session log and keeps live continuation activation process-local.', }, + { + key: 'e2b', + pkg: 'e2b', + title: 'E2B sandbox lifecycle owner', + mode: 'core', + consumers: ['fs-e2b', 'subprocess-e2b'], + note: 'Owns one shared E2B SDK handle, remote working directory, and final sandbox disposition so both fundamental E2B providers inhabit the same Linux runtime.', + }, { key: 'subprocess', pkg: 'subprocess', title: 'Subprocess seam', mode: 'seam', - implementations: ['subprocess-local'], - consumers: ['bash-local', 'bash-sandbox', 'lsp-local', 'subagent-acp', 'subagent-codex', 'subagent-claude-code'], - note: 'The bash executors, the LSP host, and the out-of-process ACP, Codex, and Claude Code subagent backends spawn their children through ctx.subprocess; the service owns tree lifetime, stdio dispositions (pipes, inherit, bounded spill-backed collection), and kill escalation.', + implementations: ['subprocess-local', 'subprocess-e2b'], + consumers: ['bash-local', 'bash-sandbox', 'pty-local', 'lsp-local', 'subagent-acp', 'subagent-codex', 'subagent-claude-code'], + note: 'The bash executors, the PTY shell backend, the LSP host, and the out-of-process ACP, Codex, and Claude Code subagent backends spawn through ctx.subprocess; the service owns process coordinates, tree/session lifetime, stdio dispositions, terminal mechanics, and kill escalation.', }, { key: 'bash', @@ -412,7 +421,7 @@ const SERVICE_ROLES: ServiceRole[] = [ pkg: 'fs', title: 'Filesystem provider seam', mode: 'seam', - implementations: ['fs-local', 'fs-sandbox'], + implementations: ['fs-local', 'fs-sandbox', 'fs-e2b'], consumers: ['tool-fs'], companions: ['fs-policy'], note: 'tool-fs executes read/write/edit through ctx.fs; fs-sandbox fences mutations by the shared sandbox mode; fs-policy contributes observed-state checks through the fs/* event gate.', diff --git a/scripts/run-gates.ts b/scripts/run-gates.ts index 77432e7dbe..74a09a874e 100644 --- a/scripts/run-gates.ts +++ b/scripts/run-gates.ts @@ -623,11 +623,12 @@ function builtBinSmokeGate(needs: string[] = ['build']): Gate { 'packages/subagent/subagent-codex/tests/loader-composition.e2e.ts', 'packages/subagent/subagent-claude-code/tests/loader-composition.e2e.ts', 'packages/api/remotes/tests/built-lib.e2e.ts', - // The worker-entry packages' built bundles: the only automated proof - // that lib/index.js resolves its sibling lib/worker.cjs under plain node - // (the e2e lane runs unbuilt, so these files self-skip there). + // Built execution consumers: the only automated proof that package-name + // imports reach their lib/ entrypoints under plain Node. The e2e lane runs + // unbuilt, so these files self-skip there. 'packages/workflow/workflow-workerthread/tests/built-worker.e2e.ts', 'packages/code-runtime/code-runtime-worker/tests/built-lib.e2e.ts', + 'packages/lsp/lsp-local/tests/built-lib.e2e.ts', ], { label: 'built-bin smoke', needs, diff --git a/scripts/verify-package-readme-model-experience.ts b/scripts/verify-package-readme-model-experience.ts index afb1095c2d..986c6ae2c4 100644 --- a/scripts/verify-package-readme-model-experience.ts +++ b/scripts/verify-package-readme-model-experience.ts @@ -51,6 +51,7 @@ const SENTENCE_MODEL_EXPERIENCE: Readonly> = { 'packages/preset/agent-presets': { kind: 'indirect', reason: 'The mount installs a preset\'s own plugins, which own every model-facing registration it makes visible.' }, 'packages/typert/registry': { kind: 'none', reason: 'Runtime type registry; consumers (cordis_inspect, wire faces, gates) own any model-visible projection of registry contents.' }, 'packages/typert/loader': { kind: 'none', reason: 'Loader integration only registers generated artifacts; consumers own any model-visible projection.' }, + 'packages/e2b/e2b': { kind: 'none', reason: 'The shared remote-runtime owner registers no model context; provider adapters and consumers own rendered effects.' }, 'packages/client/hmr': { kind: 'none', reason: 'Browser-side UI plugin layer; registers no model surface.' }, 'packages/client/modules': { kind: 'none', reason: 'Browser-side module-loading kernel machinery; registers no model surface.' }, 'packages/client/test-runtime': { kind: 'none', reason: 'Browser-side test infrastructure (jsdom bench); registers no model surface.' }, @@ -83,6 +84,7 @@ const SENTENCE_MODEL_EXPERIENCE: Readonly> = { 'packages/client/web': { kind: 'none', reason: 'Browser-side UI plugin layer; registers no model surface.' }, 'packages/examples/agent-spine-demo': { kind: 'indirect', reason: 'The bundle only mounts model-facing child plugins.' }, 'packages/fs/fs': { kind: 'indirect', reason: 'The service interface delegates model rendering to dsh-tool-fs.' }, + 'packages/e2b/fs-e2b': { kind: 'indirect', reason: 'The provider backend delegates model rendering to dsh-tool-fs.' }, 'packages/fs/fs-local': { kind: 'indirect', reason: 'The provider backend delegates model rendering to dsh-tool-fs.' }, 'packages/hooks/hook-protocol': { kind: 'indirect', reason: 'Only the hook bridge plugins render decoded hook output to a model.' }, 'packages/host/apiproxy': { kind: 'none', reason: 'The wire contract and fetch carriers move already-composed messages and register no model surface.' }, @@ -99,6 +101,7 @@ const SENTENCE_MODEL_EXPERIENCE: Readonly> = { 'packages/lsp/lsp': { kind: 'indirect', reason: 'The provider registry delegates model rendering to dsh-tool-lsp.' }, 'packages/lsp/lsp-local': { kind: 'indirect', reason: 'The provider backend delegates model rendering to dsh-tool-lsp.' }, 'packages/subprocess/subprocess': { kind: 'indirect', reason: 'The seam delegates all model rendering to consumer seams such as the bash executor family.' }, + 'packages/e2b/subprocess-e2b': { kind: 'indirect', reason: 'The remote spawn backend delegates model rendering to consumer seams such as the bash executor family.' }, 'packages/subprocess/subprocess-local': { kind: 'indirect', reason: 'The spawn backend delegates model rendering to consumer seams such as the bash executor family.' }, 'packages/sandbox/sandbox-local': { kind: 'indirect', reason: 'The provider backend delegates model rendering to dsh-bash-sandbox and dsh-tool-bash.' }, 'packages/sdk/create-sdk': { kind: 'indirect', reason: 'The initializer only writes project files; selected runtime plugins provide the generated project model surface.' }, diff --git a/tsconfig.base.json b/tsconfig.base.json index 99ca16bf87..4b50d929b0 100644 --- a/tsconfig.base.json +++ b/tsconfig.base.json @@ -85,6 +85,7 @@ "./packages/llm/*/src/invariant.ts", "./packages/bash/*/src/invariant.ts", "./packages/subprocess/*/src/invariant.ts", + "./packages/e2b/*/src/invariant.ts", "./packages/code-runtime/*/src/invariant.ts", "./packages/fs/*/src/invariant.ts", "./packages/skill/*/src/invariant.ts", @@ -189,6 +190,7 @@ "./packages/bash/*/src", "./packages/pty/*/src", "./packages/subprocess/*/src", + "./packages/e2b/*/src", "./packages/code-runtime/*/src", "./packages/fs/*/src", "./packages/lsp/*/src", diff --git a/tsconfig.host.json b/tsconfig.host.json index 3eddad29b1..582efd8953 100644 --- a/tsconfig.host.json +++ b/tsconfig.host.json @@ -162,6 +162,8 @@ { "path": "./packages/examples/agent-spine-demo" }, { "path": "./packages/subprocess/subprocess" }, { "path": "./packages/subprocess/subprocess-local" }, + { "path": "./packages/e2b/e2b" }, + { "path": "./packages/e2b/subprocess-e2b" }, { "path": "./packages/bash/bash" }, { "path": "./packages/pty/pty" }, { "path": "./packages/pty/pty-local" }, @@ -183,6 +185,7 @@ { "path": "./packages/bash/tool-bash" }, { "path": "./packages/fs/fs" }, { "path": "./packages/fs/fs-local" }, + { "path": "./packages/e2b/fs-e2b" }, { "path": "./packages/fs/fs-policy" }, { "path": "./packages/fs/fs-sandbox" }, { "path": "./packages/fs/tool-fs" },