diff --git a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.i18n.yaml b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.i18n.yaml new file mode 100644 index 0000000000..13d081c28f --- /dev/null +++ b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md +2026-07-27-e2b-remote-runtime-poc.md: 307a08d6ca77c83bf836bc8d50b071fbf33ac49a +2026-07-27-e2b-remote-runtime-poc.zh.md: bb4e657d999760f000949d509228bbb65f22773a diff --git a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md new file mode 100644 index 0000000000..307a08d6ca --- /dev/null +++ b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md @@ -0,0 +1,46 @@ +# Agent Note: Shared E2B remote runtime POC + +Status: implemented + +English | [中文](2026-07-27-e2b-remote-runtime-poc.zh.md) + +## Problem + +A remote coding-agent backend is useful only when file operations and commands observe one coherent world. Attaching E2B independently at individual tools would allow a Bash command and a filesystem edit to address different sandboxes, while moving the complete harness into a remote VM would couple provider experimentation to agent, session, model, persistence, and deployment changes. + +## Decision + +The E2B integration is a provider-composition POC with one shared lifecycle owner and two capability implementations: + +- `@deepseek-ai/dsh-e2b` creates or reconnects one secure E2B sandbox, creates its working and private runtime directories, and owns kill/pause/leave disposal. +- `@deepseek-ai/dsh-fs-e2b` implements `ctx.fs` over that sandbox's Filesystem API. +- `@deepseek-ai/dsh-subprocess-e2b` implements `ctx.subprocess` over E2B Commands and remote Linux process groups. +- The existing `@deepseek-ai/dsh-bash-local` remains the Bash implementation because it delegates all process mechanics to `ctx.subprocess`. + +The owner is the sole source of sandbox identity. Providers inject it and never create private sandboxes. The composition therefore gives filesystem tools and Bash one remote cwd, process namespace, and spill/state directory while preserving the existing capability interfaces and model-facing tools. + +## POC boundary + +Only filesystem state, command processes during the provider lifetime, and adapter-owned remote files move into E2B. The host retains Cordis and plugin objects, the agent loop, agent/session state, session logs and persistence, model requests, skills, subagent orchestration, and E2B SDK buffers. The overlay does not upload or mount the host workspace; identical cwd strings name independent host and remote directories. Managed process groups still terminate and join when the subprocess service disposes, including before a retained-sandbox pause or leave disposition. + +The POC has no PTY adapter, LSP-specific integration, session-persistence backend, code-runtime backend, template builder, volume, snapshot, network-policy layer, sandbox catalog, or workspace synchronization. Retained sandbox reconnect proves lifecycle continuity only; it does not reconstruct host process handles, output cursors, or locks. + +## Verification + +Package tests pin lifecycle cleanup, filesystem seam semantics, subprocess group/stdio/abort behavior, and the package-owned invariant registrations. A credential-gated real Loader composition creates one sandbox, proves FS-write→Bash-read and Bash-write→FS-read in the same remote cwd, proves neither file appears in the host cwd, disposes the composition, and confirms the sandbox id is gone. + +## Alternatives considered + +**A separate E2B sandbox per capability or tool** — rejected because file and command operations would not share identity or state, defeating the coding-agent use case and multiplying lifecycle ownership. + +**Run the entire harness process inside E2B** — rejected because it changes deployment, credential flow, model transport, session durability, plugin loading, and supervision at once. Those questions are independent of proving the provider seams. + +**Add E2B-specific Bash, PTY, LSP, persistence, and synchronization packages together** — rejected because Bash already has the required subprocess seam and the other capabilities need separate consumer evidence and lifecycle designs. Their absence is an explicit fidelity boundary, not an incomplete hidden plan. + +**Implement filesystem operations through shell commands only** — rejected because that bypasses `ctx.fs` identity, structured errors, version guards, streaming reads, and atomic mutation semantics already consumed by the file tools. + +## Consequences + +The small composition demonstrates that existing capability seams can move an agent's mutable coding world off-host without changing the loop or model-facing tool packages. `sandboxId` plus pause/leave permits manual state retention for experiments, while kill remains the demo's cleanup policy. + +The provider is not interchangeable with the local subprocess backend for every consumer: remote startup cannot synchronously expose a PID, E2B retains complete command output in SDK memory, callback output is not byte-faithful, signal attribution is inferred, and reconnect cannot restore handles. Remote process/spill artifacts accumulate in a retained sandbox. These gaps remain documented POC constraints rather than compatibility shims or new cross-cutting abstractions. diff --git a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.zh.md b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.zh.md new file mode 100644 index 0000000000..bb4e657d99 --- /dev/null +++ b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.zh.md @@ -0,0 +1,46 @@ +# Agent Note: 共享 E2B 远程运行时 POC + +Status: implemented + +[English](2026-07-27-e2b-remote-runtime-poc.md) | 中文 + +## 问题 + +远程 coding agent（智能体）后端只有在文件操作与命令观察到同一个一致环境时才有用。若在各工具上分别接入 E2B，Bash 命令和文件系统编辑可能访问不同的沙箱；若把完整 harness 迁入远程 VM，则会把提供方实验与 agent、会话、模型、持久化及部署变更耦合在一起。 + +## 决策 + +E2B 集成是一个提供方组合 POC，由一个共享生命周期所有者和两个功能实现组成： + +- `@deepseek-ai/dsh-e2b` 创建或重新连接一个安全的 E2B 沙箱，创建其工作目录与私有运行时目录，并拥有 kill/pause/leave 资源释放操作。 +- `@deepseek-ai/dsh-fs-e2b` 在该沙箱的 Filesystem API 之上实现 `ctx.fs`。 +- `@deepseek-ai/dsh-subprocess-e2b` 在 E2B Commands 和远程 Linux 进程组之上实现 `ctx.subprocess`。 +- 现有的 `@deepseek-ai/dsh-bash-local` 继续作为 Bash 实现，因为它把所有进程机制委托给 `ctx.subprocess`。 + +该所有者是沙箱身份的唯一真源。提供方会注入该所有者，绝不创建私有沙箱。因此，该组合让文件系统工具与 Bash 共享一个远程 cwd、进程命名空间和 spill／状态目录，同时保留现有功能接口与面向模型的工具。 + +## POC 边界 + +只有文件系统状态、提供方存续期内的命令进程，以及适配器拥有的远程文件会迁入 E2B。宿主仍保留 Cordis 和插件对象、agent loop（智能体循环）、agent／会话状态、会话日志及其持久化、模型请求、skill（技能）、subagent 编排和 E2B SDK 缓冲。该 overlay 不会上传或挂载宿主工作区；拼写相同的 cwd 字符串分别指向彼此独立的宿主与远程目录。受管进程组仍会在进程管理服务 dispose（资源释放）时终止并等待退出，包括保留沙箱采用 `pause` 或 `leave` 处置方式之前。 + +本 POC 没有 PTY 适配器、LSP 专用集成、会话持久化后端、代码运行时后端、模板构建器、卷、快照、网络策略层、沙箱目录或工作区同步。保留沙箱后重新连接只能证明生命周期连续性；它不会重建宿主进程句柄、输出游标或锁。 + +## 验证 + +包测试固定生命周期清理、文件系统 seam 语义、进程管理的进程组／stdio／中止行为，以及包自有不变式注册。凭据门控的真实 Loader 组合会创建一个沙箱，证明同一远程 cwd 中 FS-write→Bash-read 和 Bash-write→FS-read 双向可见，证明两个文件均未出现在宿主 cwd 中，释放组合，并确认该沙箱 id 已不存在。 + +## 曾考虑的替代方案 + +**每项功能或每个工具使用独立的 E2B 沙箱。** 不予采纳，因为文件操作和命令操作将无法共享身份或状态，既违背 coding agent 用例，也会增加生命周期所有者的数量。 + +**在 E2B 内运行完整 harness 进程。** 不予采纳，因为这会同时改变部署、凭据流、模型传输、会话持久性、插件加载和监管方式。要证明提供方 seam，并不需要同时回答这些彼此独立的问题。 + +**同时添加 E2B 专用 Bash、PTY、LSP、持久化和同步包。** 不予采纳，因为 Bash 已经具备所需的进程管理 seam，其他功能则需要各自的消费方证据和生命周期设计。缺少它们是显式保真边界，而不是尚未公开的不完整计划。 + +**仅通过 shell 命令实现文件系统操作。** 不予采纳，因为这会绕过文件工具已经使用的 `ctx.fs` 身份、结构化错误、版本防护、流式读取和原子变更语义。 + +## 后果 + +这个小型组合证明，现有功能 seam 可以把 agent 的可变 coding 环境移出宿主，而无需改变循环或面向模型的工具包。`sandboxId` 与 `pause`／`leave` 允许实验手动保留状态，演示仍以 `kill` 作为清理策略。 + +该提供方并不能对所有消费方与本地进程管理后端互换：远程启动无法同步公开 PID，E2B 会在 SDK 内存中保留完整命令输出，回调输出并非字节保真，信号归因依靠推断，重新连接也无法恢复句柄。保留沙箱后会累积远程进程／spill 产物。这些缺口作为 POC 约束明确记录，而不会引入兼容垫片或新的跨领域抽象。 diff --git a/docs/architecture.zh.md b/docs/architecture.zh.md index 9314a3fbfc..2d1a0b43d5 100644 --- a/docs/architecture.zh.md +++ b/docs/architecture.zh.md @@ -25,6 +25,7 @@ |---|---|---| | `ctx.llm` | [`llm/`](../packages/llm/README.md) | 适配器注册表和模型流式调用 | | `ctx.tokenMeter` | [`llm/token-meter`](../packages/llm/token-meter/README.md) | 感知回放的单实例请求压力与表面压力 | +| `ctx.e2b` | [`e2b/`](../packages/e2b/README.md) | 共享 E2B 沙箱 | | `ctx.bash` | [`bash/`](../packages/bash/README.md) | 前台和后台命令执行 | | `ctx.subprocess` | [`subprocess/`](../packages/subprocess/README.md) | 可执行文件查找、受管进程树、终端 | | `ctx.pty` | [`pty/`](../packages/pty/README.md) | 按 owner 隔离的持久化终端会话 | diff --git a/docs/cordis-catalog/services.md b/docs/cordis-catalog/services.md index 7e125cd02c..7ce45c6817 100644 --- a/docs/cordis-catalog/services.md +++ b/docs/cordis-catalog/services.md @@ -578,6 +578,21 @@ abstract capability(): DirectoryPickerCapability Source: [`packages/host/directory-picker/src/index.ts:131`](../../packages/host/directory-picker/src/index.ts) +## `ctx.e2b` — `E2BSandboxService` + +Owns one lazily consumable E2B SDK handle and its final kill/pause/leave decision. The connection begins at plugin construction; adapters await getSandbox before their first operation. + +```ts cordis-catalog +/** + * Return the shared live SDK handle. + * @returns the created or reconnected sandbox after the configured cwd exists. + * @throws when E2B rejects creation/reconnection or the service is disposing. + */ +async getSandbox(): Promise +``` + +Source: [`packages/e2b/e2b/src/index.ts:97`](../../packages/e2b/e2b/src/index.ts) + ## `ctx.fs` — `FileSystem` (abstract seam) Abstract filesystem provider. Targets must preserve identity across aliases; reads expose regular UTF-8 text or typed errors, listings are stable and content-free, and mutations are atomic. Optional guards add stale protection without changing the unguarded provider contract. diff --git a/examples/headless-agent/README.i18n.yaml b/examples/headless-agent/README.i18n.yaml index 7b6139f28b..56c02d3e0a 100644 --- a/examples/headless-agent/README.i18n.yaml +++ b/examples/headless-agent/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write examples/headless-agent/README.md -README.md: e00f3d2d4fd21a860239f3d3a3e5eb2d7520f14a -README.zh.md: 6cd845783b1c112ba73676474b176ec28a4d0b78 +README.md: 16c45d304eb0f06a2b062518b053db683cd402c0 +README.zh.md: 72d2813d174e74a3bf93c363b268499a44f1cd4f diff --git a/examples/headless-agent/README.md b/examples/headless-agent/README.md index e00f3d2d4f..16c45d304e 100644 --- a/examples/headless-agent/README.md +++ b/examples/headless-agent/README.md @@ -2,7 +2,7 @@ English | [中文](README.zh.md) -This directory owns the replay and real-model test composition for a headless coding agent: DeepSeek V4 + local bash and filesystem tools + subagent delegation + workflows and fresh-agent Ralph iteration + `todo_write` + JSONL persistence. It explicitly mounts the shared agent spine, one root agent, persistence, and checkpoint policy; it is not a second product front door. +Headless one-shot agent wiring: DeepSeek V4 + local bash and filesystem tools + subagent delegation + workflows and fresh-agent Ralph iteration + `todo_write` + JSONL persistence, with [`@deepseek-ai/dsh-cli-demo`](../../packages/examples/cli-demo) as the app front door. ## Run it @@ -10,13 +10,27 @@ This directory owns the replay and real-model test composition for a headless co # repo root .env (gitignored) or exported env: # DEEPSEEK_API_KEY=sk-… # DEEPSEEK_BASE_URL=https://… # optional; defaults to the public API -pnpm run dsh run "fix the failing test in this workspace" +pnpm run demo:headless "fix the failing test in this workspace" +pnpm run demo:headless --output-format json -- "summarize the implementation" +pnpm run demo:headless --output-format stream-json -- "run the focused tests" ``` -The product command is [`dsh run`](../../apps/cli/README.md): it accepts one nonblank task, creates and persists a fresh session, prints the final assistant text, and exits. The root `demo:headless` script is only an alias of that command. +Exactly one nonblank positional task is required; quote tasks containing spaces. There is no `-p` flag. `text` prints the last text-bearing assistant message, `json` prints one DSH-native result record, and `stream-json` emits the top-level session's canonical task-turn events before that record. Child sessions surface only through parent tool events and results. -Snapshot suites run this directory's configuration through [`tests/fixtures/headless-driver.ts`](tests/fixtures/headless-driver.ts), an unexported test-only process that emits canonical session events as JSONL before its result record. That stream is test infrastructure, not a supported CLI output format. Child sessions surface only through parent tool events and results. +Each invocation creates and persists a fresh session, runs all model and tool steps in one turn, flushes, disposes, and exits. This is non-interactive automation: there is no prompt, approval, resume, second turn, or stdin context. The configured tools can mutate the launch workspace, run commands, spawn child agents, and consume provider tokens. -## Advanced configuration +## E2B POC overlay -[`advanced.cordis.yml`](advanced.cordis.yml) adds Code Mode and the Cordis tools to the test composition. +[`e2b.cordis.yml`](e2b.cordis.yml) replaces the local filesystem and subprocess providers with one shared E2B sandbox while retaining `dsh-bash-local` and the same model-facing tools. Put `E2B_API_KEY` beside `DEEPSEEK_API_KEY` in the gitignored root `.env`, then run: + +```sh +pnpm run demo:e2b "create hello.txt, read it back, and run pwd" +``` + +The overlay creates the same absolute cwd inside the sandbox, but it does not upload or mount the host workspace. File and Bash mutations exist only in E2B; Cordis, model calls, agent/session state, session logs, skills, and SDK buffers remain on the host. The demo kills its sandbox on timeout and disposal. It is a provider-composition POC, not a whole-harness migration or a workspace-sync feature. + +## Advanced and snapshot wiring + +[`advanced.cordis.yml`](advanced.cordis.yml) adds Code Mode and the Cordis tools to the shipped leaf. [`advanced.cordis.snapshot.yml`](advanced.cordis.snapshot.yml) replaces only the live LLM with replay. The tests under [`tests/`](tests/) own the keyless real-Loader smoke, key-gated world-verified smoke, and the `stream-json` replay snapshot with its parent and child session fixtures. + +The package-level [CLI contract](../../packages/examples/cli-demo/README.md) documents output records, exit status, cancellation, persistence, and model/token effects. diff --git a/examples/headless-agent/README.zh.md b/examples/headless-agent/README.zh.md index 6cd845783b..72d2813d17 100644 --- a/examples/headless-agent/README.zh.md +++ b/examples/headless-agent/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -本目录负责 headless coding agent（智能体）的回放和真实模型测试组装：DeepSeek V4 + 本地 bash 与文件系统工具 + subagent 委托 + 工作流与全新 agent Ralph 迭代 + `todo_write` + JSONL 持久化。本目录显式挂载共享 agent 主干、一个根 agent、持久化和检查点策略；它不是第二个产品入口。 +无头单次 agent（智能体）接线：DeepSeek V4 + 本地 bash 与文件系统工具 + subagent 委托 + 工作流与新 agent Ralph 迭代 + `todo_write` + JSONL 持久化，并以 [`@deepseek-ai/dsh-cli-demo`](../../packages/examples/cli-demo) 作为应用入口。 ## 运行 @@ -10,13 +10,27 @@ # repo root .env (gitignored) or exported env: # DEEPSEEK_API_KEY=sk-… # DEEPSEEK_BASE_URL=https://… # optional; defaults to the public API -pnpm run dsh run "fix the failing test in this workspace" +pnpm run demo:headless "fix the failing test in this workspace" +pnpm run demo:headless --output-format json -- "summarize the implementation" +pnpm run demo:headless --output-format stream-json -- "run the focused tests" ``` -产品命令是 [`dsh run`](../../apps/cli/README.md)：它接受一项非空任务，创建并持久化新会话，打印最终 assistant 文本，然后退出。根目录的 `demo:headless` 脚本只是该命令的别名。 +必须提供且只能提供一个非空位置任务；含空格的任务需要加引号。没有 `-p` 标志。`text` 打印最后一条包含文本的 assistant 消息，`json` 打印一条 DSH 原生结果记录，`stream-json` 则在该记录之前发出顶层会话的规范任务轮次事件。子会话只通过父工具事件和结果对外显示。 -快照套件通过 [`tests/fixtures/headless-driver.ts`](tests/fixtures/headless-driver.ts) 运行本目录的配置。这个未导出且仅供测试使用的进程会在结果记录之前，以 JSONL 发出规范会话事件。该事件流属于测试基础设施，不是受支持的 CLI（命令行界面）输出格式。子会话只通过父会话的工具事件和结果对外显示。 +每次调用都会创建并持久化新会话，在一个轮次中运行所有模型和工具步骤，然后刷新、释放并退出。这是非交互式自动化：没有提示符、批准、恢复、第二轮次或 stdin 上下文。已配置工具可以修改启动 workspace、运行命令、spawn 子 agent，并消耗提供方 token。 -## 高级配置 +## E2B POC overlay -[`advanced.cordis.yml`](advanced.cordis.yml) 在测试组装中添加 Code Mode 和 Cordis 工具。 +[`e2b.cordis.yml`](e2b.cordis.yml) 使用一个共享 E2B 沙箱替换本地文件系统与进程管理提供方，同时保留 `dsh-bash-local` 和相同的面向模型工具。请在 git 忽略的根目录 `.env` 中，将 `E2B_API_KEY` 与 `DEEPSEEK_API_KEY` 放在一起，然后运行： + +```sh +pnpm run demo:e2b "create hello.txt, read it back, and run pwd" +``` + +该 overlay 会在沙箱中创建拼写相同的绝对 cwd，但不会上传或挂载宿主工作区。文件与 Bash 变更只存在于 E2B；Cordis、模型调用、agent／会话状态、会话日志、skill（技能）和 SDK 缓冲仍在宿主上。演示会在超时和资源释放时终止其沙箱。它是提供方组合 POC，而不是完整 harness 迁移或工作区同步功能。 + +## 高级与快照接线 + +[`advanced.cordis.yml`](advanced.cordis.yml) 在已交付叶节点上添加 Code Mode 和 Cordis 工具。[`advanced.cordis.snapshot.yml`](advanced.cordis.snapshot.yml) 只将实时 LLM（大语言模型）替换为回放。[`tests/`](tests/) 下的测试拥有无密钥真实 Loader 冒烟测试、密钥门控的外部状态验证冒烟测试，以及带父子会话 fixture（测试前置数据）的 `stream-json` 回放快照。 + +包级 [CLI 契约](../../packages/examples/cli-demo/README.md)记录输出记录、退出状态、取消、持久化以及模型／token 影响。 diff --git a/examples/headless-agent/e2b.cordis.yml b/examples/headless-agent/e2b.cordis.yml new file mode 100644 index 0000000000..4f961100b4 --- /dev/null +++ b/examples/headless-agent/e2b.cordis.yml @@ -0,0 +1,30 @@ +# POC overlay: keep the headless agent and model-facing tools, but place its +# filesystem and managed Bash process world in one short-lived E2B sandbox. +- id: base + name: '@cordisjs/plugin-include' + config: + path: ./cordis.yml + patches: + - id: subprocess + name: '@deepseek-ai/dsh-subprocess-local' + disabled: true + - id: fs-local + name: '@deepseek-ai/dsh-fs-local' + disabled: true + - id: bash + name: '@deepseek-ai/dsh-bash-local' + config: + cwd: !!js process.cwd() + timeoutMs: 60000 + - insert: + - id: e2b + name: '@deepseek-ai/dsh-e2b' + config: + cwd: !!js process.cwd() + timeoutMs: 300000 + onTimeout: kill + onDispose: kill + - id: subprocess-e2b + name: '@deepseek-ai/dsh-subprocess-e2b' + - id: fs-e2b + name: '@deepseek-ai/dsh-fs-e2b' diff --git a/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts b/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts new file mode 100644 index 0000000000..3b2cef609b --- /dev/null +++ b/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts @@ -0,0 +1,32 @@ +import { resolve } from 'node:path' +import { boot } from '@deepseek-ai/dsh-app-boot' +import type {} from '@deepseek-ai/dsh-e2b' +import type {} from '@deepseek-ai/dsh-fs-e2b' +import type {} from '@deepseek-ai/dsh-bash-local' + +const configPath = process.argv[2] +if (configPath === undefined) throw new Error('usage: bin.ts ') + +const ctx = await boot('e2b-composition', resolve(configPath)) +try { + const fromFs = await ctx.fs.resolve('from-fs.txt') + await ctx.fs.writeText(fromFs, 'written-by-fs\n', { kind: 'createIfAbsent' }) + const bashRead = await ctx.bash.run(ctx.bash.resolve({ command: 'cat from-fs.txt' })) + if (bashRead.exitCode !== 0 || bashRead.stdout.text !== 'written-by-fs\n') { + throw new Error(`E2B Bash could not read the FS write: ${JSON.stringify(bashRead)}`) + } + + const bashWrite = await ctx.bash.run(ctx.bash.resolve({ command: "printf 'written-by-bash\\n' > from-bash.txt" })) + if (bashWrite.exitCode !== 0) { + throw new Error(`E2B Bash could not write the shared filesystem: ${JSON.stringify(bashWrite)}`) + } + const fromBash = await ctx.fs.resolve('from-bash.txt') + const fsRead = await ctx.fs.readText(fromBash) + process.stdout.write(`${JSON.stringify({ + sandboxId: await ctx.e2b.sandboxId, + bashRead: bashRead.stdout.text, + fsRead, + })}\n`) +} finally { + await ctx.fiber.dispose() +} diff --git a/examples/headless-agent/tests/fixtures/e2b/e2b/cordis.yml b/examples/headless-agent/tests/fixtures/e2b/e2b/cordis.yml new file mode 100644 index 0000000000..08a7e7225c --- /dev/null +++ b/examples/headless-agent/tests/fixtures/e2b/e2b/cordis.yml @@ -0,0 +1,19 @@ +- id: e2b + name: '@deepseek-ai/dsh-e2b' + config: + cwd: !!js process.cwd() + timeoutMs: 60000 + onTimeout: kill + onDispose: kill + +- id: subprocess-e2b + name: '@deepseek-ai/dsh-subprocess-e2b' + +- id: bash + name: '@deepseek-ai/dsh-bash-local' + config: + cwd: !!js process.cwd() + timeoutMs: 30000 + +- id: fs-e2b + name: '@deepseek-ai/dsh-fs-e2b' diff --git a/examples/package.json b/examples/package.json index 1f6e99d240..48ac39f5ca 100644 --- a/examples/package.json +++ b/examples/package.json @@ -26,7 +26,9 @@ "@deepseek-ai/dsh-compact-basic": "workspace:*", "@deepseek-ai/dsh-compact-tool-result-prune": "workspace:*", "@deepseek-ai/dsh-credentials-local": "workspace:*", + "@deepseek-ai/dsh-e2b": "workspace:*", "@deepseek-ai/dsh-fs-local": "workspace:*", + "@deepseek-ai/dsh-fs-e2b": "workspace:*", "@deepseek-ai/dsh-fs-policy": "workspace:*", "@deepseek-ai/dsh-fs-sandbox": "workspace:^", "@deepseek-ai/dsh-goal": "workspace:*", @@ -76,6 +78,7 @@ "@deepseek-ai/dsh-subagent-fork": "workspace:*", "@deepseek-ai/dsh-subagent-spawn": "workspace:*", "@deepseek-ai/dsh-subprocess-local": "workspace:*", + "@deepseek-ai/dsh-subprocess-e2b": "workspace:*", "@deepseek-ai/dsh-system-prompt": "workspace:*", "@deepseek-ai/dsh-tasks-local": "workspace:*", "@deepseek-ai/dsh-time-context": "workspace:*", diff --git a/knip.json b/knip.json index ede8b2c738..d154cd646b 100644 --- a/knip.json +++ b/knip.json @@ -45,6 +45,7 @@ "headless-agent/tests/fixtures/time-context-mock-llm.ts", "headless-agent/tests/fixtures/telemetry-otel-driver.ts", "headless-agent/tests/fixtures/telemetry-redact-rule.ts", + "headless-agent/tests/fixtures/e2b/e2b/bin.ts", "acp-agent/tests/snapshots/lsp-definition/workspace/subject.ts", "acp-agent/tests/fixtures/child-question-tripwire.ts", "acp-agent/tests/fixtures/partial-landlock-sandbox.ts", @@ -229,6 +230,16 @@ "tests/**/*.ts" ] }, + "packages/e2b/e2b": { + "entry": [ + "tests/**/*.spec.ts", + "tests/**/*.e2e.ts" + ], + "project": [ + "src/**/*.ts", + "tests/**/*.ts" + ] + }, "packages/context/time-context": { "entry": [ "tests/**/*.spec.ts", diff --git a/packages/README.i18n.yaml b/packages/README.i18n.yaml index 8aa9b92b91..3eee08f35b 100644 --- a/packages/README.i18n.yaml +++ b/packages/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/README.md -README.md: 229feae568ba6e40a9c633696097eff46fd5bc95 -README.zh.md: b84aef020a7e3edf305df709d399fbc7b093b6a3 +README.md: 1b2a1737bcac33829a65fa937668ca9496035330 +README.zh.md: 2ef61e9c38750f4420ce99abc9943af1f3308b56 diff --git a/packages/README.md b/packages/README.md index 229feae568..54a5779016 100644 --- a/packages/README.md +++ b/packages/README.md @@ -16,6 +16,7 @@ Packages live at `packages///`; groups are containers, while names r | [`goal/`](goal/README.md) | Same-session goal persistence and lifecycle | Product — stable surface | | [`feedback/`](feedback/README.md) | Human feedback | Product — stable surface | | [`llm/`](llm/README.md) | LLM capability family: the abstract service + provider adapters | Product — stable surface | +| [`e2b/`](e2b/README.md) | E2B remote filesystem/process providers | POC | | [`subprocess/`](subprocess/README.md) | Subprocess capability family: spawn seam + local process-tree implementation | Product — stable surface | | [`bash/`](bash/README.md) | Bash capability family: executor seam, local impl, model-facing tool | Product — stable surface | | [`pty/`](pty/README.md) | Persistent PTY capability family: owner-scoped sessions, local implementation, and model-facing tools | Product — stable surface | diff --git a/packages/README.zh.md b/packages/README.zh.md index b84aef020a..46b112ec8f 100644 --- a/packages/README.zh.md +++ b/packages/README.zh.md @@ -16,6 +16,7 @@ | [`goal/`](goal/README.md) | 同会话 goal 的持久化与生命周期 | 产品：稳定表面 | | [`feedback/`](feedback/README.md) | 人类反馈 | 产品：稳定表面 | | [`llm/`](llm/README.md) | LLM（大语言模型）能力系列：抽象服务 + 提供方适配器 | 产品：稳定表面 | +| [`e2b/`](e2b/README.md) | E2B 远程文件系统／进程管理提供方 | POC | | [`subprocess/`](subprocess/README.md) | 进程管理能力系列：spawn seam + 本地进程树实现 | 产品：稳定表面 | | [`bash/`](bash/README.md) | Bash 能力系列：执行器 seam、本地实现、面向模型的工具 | 产品：稳定表面 | | [`pty/`](pty/README.md) | 持久 PTY 能力系列：按所有者隔离的会话、本地实现和面向模型的工具 | 产品：稳定表面 | diff --git a/packages/e2b/README.i18n.yaml b/packages/e2b/README.i18n.yaml new file mode 100644 index 0000000000..1bc9761930 --- /dev/null +++ b/packages/e2b/README.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write packages/e2b/README.md +README.md: 4b28ec7c0452189a579a6d7e0cd22a0b8561784d +README.zh.md: ab254eecad82c05cd6a6021e7f8bc4eda6cf3076 diff --git a/packages/e2b/README.md b/packages/e2b/README.md new file mode 100644 index 0000000000..4b28ec7c04 --- /dev/null +++ b/packages/e2b/README.md @@ -0,0 +1,13 @@ +# e2b/ — E2B remote runtime family + +English | [中文](README.zh.md) + +An experimental provider-composition POC that places the filesystem and managed subprocess world in one E2B Linux sandbox. The shared owner is separate from the capability adapters so every remote provider awaits the same sandbox identity and lifecycle. + +| Package | ctx key | Role | +|---|---|---| +| [`e2b`](e2b/README.md) (`@deepseek-ai/dsh-e2b`) | `ctx.e2b` | Create or reconnect one sandbox, create its working/runtime directories, expose the shared SDK handle, and apply the configured kill/pause/leave disposition | +| [`fs-e2b`](../fs/fs-e2b/README.md) (`@deepseek-ai/dsh-fs-e2b`) | `ctx.fs` | Implement the filesystem seam over E2B Filesystem APIs | +| [`subprocess-e2b`](../subprocess/subprocess-e2b/README.md) (`@deepseek-ai/dsh-subprocess-e2b`) | `ctx.subprocess` | Implement managed process groups, stdio projection, and remote spill files over E2B Commands | + +The existing [`dsh-bash-local`](../bash/bash-local/README.md) needs no E2B-specific fork: it already delegates process mechanics to `ctx.subprocess`, so replacing that provider places Bash in the same remote world as `ctx.fs`. This boundary does not move the harness process, Cordis objects, model calls, agent/session state, session persistence, skills, or E2B SDK buffers. The [decision record](../../.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md) owns the POC boundary and rejected expansion. diff --git a/packages/e2b/README.zh.md b/packages/e2b/README.zh.md new file mode 100644 index 0000000000..ab254eecad --- /dev/null +++ b/packages/e2b/README.zh.md @@ -0,0 +1,13 @@ +# e2b/ — E2B 远程运行时家族 + +[English](README.md) | 中文 + +这是一个实验性提供方组合 POC，把文件系统和受管子进程环境放进同一个 E2B Linux 沙箱。共享所有者与功能适配器彼此分离，使每个远程提供方都等待同一个沙箱身份和生命周期。 + +| 包（package） | ctx 键 | 职责 | +|---|---|---| +| [`e2b`](e2b/README.md)（`@deepseek-ai/dsh-e2b`） | `ctx.e2b` | 创建或重新连接一个沙箱，创建其工作目录与运行时目录，公开共享 SDK 句柄，并应用配置的 kill/pause/leave 处置方式 | +| [`fs-e2b`](../fs/fs-e2b/README.md)（`@deepseek-ai/dsh-fs-e2b`） | `ctx.fs` | 通过 E2B Filesystem API 实现文件系统 seam | +| [`subprocess-e2b`](../subprocess/subprocess-e2b/README.md)（`@deepseek-ai/dsh-subprocess-e2b`） | `ctx.subprocess` | 通过 E2B Commands 实现受管进程组、stdio 投影与远程 spill 文件 | + +现有的 [`dsh-bash-local`](../bash/bash-local/README.md) 无需 E2B 专用 fork：它已经把进程机制委托给 `ctx.subprocess`，因此替换该提供方即可让 Bash 与 `ctx.fs` 进入同一个远程环境。该边界不会迁移 harness 进程、Cordis 对象、模型调用、agent（智能体）／会话状态、会话持久化、skill（技能）或 E2B SDK 缓冲。[决策记录](../../.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md)负责说明 POC 边界及未采纳的扩展方案。 diff --git a/packages/e2b/e2b/README.i18n.yaml b/packages/e2b/e2b/README.i18n.yaml new file mode 100644 index 0000000000..9255c21577 --- /dev/null +++ b/packages/e2b/e2b/README.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write packages/e2b/e2b/README.md +README.md: bf62cb7d4811ca92f263bd3d337fff9fe41ce223 +README.zh.md: 8c7603aa501a2481aba6dab1f22750056e43ad26 diff --git a/packages/e2b/e2b/README.md b/packages/e2b/e2b/README.md new file mode 100644 index 0000000000..bf62cb7d48 --- /dev/null +++ b/packages/e2b/e2b/README.md @@ -0,0 +1,50 @@ +# @deepseek-ai/dsh-e2b + +English | [中文](README.zh.md) + +Shared lifecycle owner for one E2B sandbox. Filesystem and subprocess adapters inject `ctx.e2b`, await its single SDK handle, and therefore inhabit the same remote Linux working tree and process world. The package pins `e2b@2.29.1`. + +## Configuration + +```yaml +- id: e2b + name: '@deepseek-ai/dsh-e2b' + config: + cwd: /home/user/workspace + timeoutMs: 300000 + onTimeout: pause + onDispose: kill + +- id: subprocess-e2b + name: '@deepseek-ai/dsh-subprocess-e2b' + +- id: fs-e2b + name: '@deepseek-ai/dsh-fs-e2b' +``` + +`apiKey` is optional and otherwise reads `E2B_API_KEY`; the key configures the host SDK connection and is never installed in the sandbox. `cwd` defaults to `/home/user/workspace` and must be an absolute POSIX path. `timeoutMs` defaults to five minutes. `onTimeout` is `pause` by default and accepts `pause | kill`; it applies only when this service creates a sandbox. Pause-on-timeout enables E2B auto-resume so the shared SDK handle wakes on its next operation. `onDispose` defaults to `kill` and accepts `kill | pause | leave`. + +Set `sandboxId` to reconnect a running or paused sandbox instead of creating one. E2B resumes a paused sandbox during connect; `template` is creation-only and cannot accompany `sandboxId`. Omitting `template` uses E2B's default base template. + +## Lifecycle and ownership + +Construction starts one create/connect operation. Before resolving `getSandbox()`, the service creates `cwd` and the private `cwd/.dsh-e2b` adapter-state directory, then sets that directory to mode `0700`. `sandboxId` resolves to a branded `E2BSandboxId` after setup. + +Disposal first prevents new handle acquisition, then awaits setup and applies exactly one configured disposition. A newly created sandbox is killed when initial directory setup fails; a reconnected sandbox is not killed on setup failure because the service did not create it. Provider plugins must load after this owner and dispose before it. + +`pause` and `leave` retain remote filesystem and adapter artifacts for a later `sandboxId` connection, but a later harness process receives only a new SDK handle. The subprocess service still fulfills its seam contract by terminating managed groups before owner disposal; neither disposition recovers prior process objects, output cursors, or in-memory adapter locks. + +## Model Experience + +None, as this shared runtime owner registers no model-visible context; provider adapters and their consumers own any rendered effects. + +#### KV Cache effect + +No direct invalidation; this package does not contribute request tokens. + +## Known Limitations and Deferred Work + +- **This is not a whole-harness runtime** — Cordis services, agent/session state, session logs, LLM requests, skills, and SDK-side buffers stay in the host process. +- **Retained sandboxes do not restore host handles** — reconnect preserves remote files and adapter artifacts, but cannot reconstruct subprocess handles, stream cursors, or mutation locks; managed subprocesses terminate when their service disposes. +- **No deployment platform is configured** — templates, volumes, snapshots, network policy, host-workspace synchronization, and sandbox discovery are outside this POC. +- **`cwd` is a resolution convention, not containment** — adapters and commands can address other sandbox paths; E2B network access also retains the template's policy. diff --git a/packages/e2b/e2b/README.zh.md b/packages/e2b/e2b/README.zh.md new file mode 100644 index 0000000000..8c7603aa50 --- /dev/null +++ b/packages/e2b/e2b/README.zh.md @@ -0,0 +1,50 @@ +# @deepseek-ai/dsh-e2b + +[English](README.md) | 中文 + +一个 E2B 沙箱的共享生命周期所有者。文件系统与进程管理适配器注入 `ctx.e2b`，等待其唯一的 SDK 句柄，因此处于同一个远程 Linux 工作树与进程环境中。本包固定使用 `e2b@2.29.1`。 + +## 配置 + +```yaml +- id: e2b + name: '@deepseek-ai/dsh-e2b' + config: + cwd: /home/user/workspace + timeoutMs: 300000 + onTimeout: pause + onDispose: kill + +- id: subprocess-e2b + name: '@deepseek-ai/dsh-subprocess-e2b' + +- id: fs-e2b + name: '@deepseek-ai/dsh-fs-e2b' +``` + +`apiKey` 可省略；省略时读取 `E2B_API_KEY`。该密钥只配置宿主 SDK 连接，绝不会安装进沙箱。`cwd` 默认为 `/home/user/workspace`，并且必须是绝对 POSIX 路径。`timeoutMs` 默认为 5 分钟。`onTimeout` 默认为 `pause`，接受 `pause | kill`；它只在本服务创建沙箱时生效。超时时 pause 会启用 E2B 自动恢复，使共享 SDK 句柄在下一次操作时唤醒。`onDispose` 默认为 `kill`，接受 `kill | pause | leave`。 + +设置 `sandboxId` 可重新连接正在运行或已经暂停的沙箱，而不是创建新沙箱。连接时，E2B 会恢复已经暂停的沙箱；`template` 仅用于创建，不能与 `sandboxId` 同时使用。省略 `template` 时使用 E2B 的默认基础模板。 + +## 生命周期与所有权 + +构造阶段会启动一次 create/connect 操作。服务在 `getSandbox()` 结算前创建 `cwd` 和私有的 `cwd/.dsh-e2b` 适配器状态目录，再把该目录的 mode 设为 `0700`。初始化完成后，`sandboxId` 会结算为品牌类型 `E2BSandboxId`。 + +资源释放会先阻止继续获取新句柄，再等待初始化完成，并且只应用一种已配置的处置方式。新建沙箱的初始目录设置失败时，服务会终止该沙箱；重新连接的沙箱设置失败时不会被终止，因为它不是由本服务创建的。提供方插件必须在该所有者之后加载，并在其之前 dispose（资源释放）。 + +`pause` 和 `leave` 会保留远程文件系统及适配器产物，供稍后的 `sandboxId` 连接使用，但后续 harness 进程只会获得新的 SDK 句柄。进程管理服务仍会履行其 seam 契约，在所有者释放前终止受管进程组；这两种处置方式都不会恢复先前的进程对象、输出游标或内存中的适配器锁。 + +## 模型体验 + +无。本共享运行时所有者不注册模型可见上下文；提供方适配器及其消费方拥有所有渲染效果。 + +#### KV Cache 影响 + +不会直接失效；本包不会贡献请求 token。 + +## 已知限制与延后工作 + +- **这不是完整的 harness 运行时**：Cordis 服务、agent（智能体）／会话状态、会话日志、LLM（大语言模型）请求、skill（技能）和 SDK 侧缓冲仍留在宿主进程中。 +- **保留的沙箱不会恢复宿主句柄**：重新连接会保留远程文件和适配器产物，但无法重建进程管理句柄、流游标或变更锁；进程管理服务 dispose 时会终止受管子进程。 +- **没有配置部署平台**：模板、卷、快照、网络策略、宿主工作区同步和沙箱发现均不在本 POC 范围内。 +- **`cwd` 是解析约定，而不是包含边界**：适配器和命令可以访问沙箱中的其他路径；E2B 网络访问也继续采用模板的策略。 diff --git a/packages/e2b/e2b/package.json b/packages/e2b/e2b/package.json new file mode 100644 index 0000000000..7ecc5fabed --- /dev/null +++ b/packages/e2b/e2b/package.json @@ -0,0 +1,44 @@ +{ + "name": "@deepseek-ai/dsh-e2b", + "description": "Shared E2B sandbox lifecycle for DeepSeek Harness provider adapters", + "version": "0.0.1", + "private": true, + "type": "module", + "main": "lib/index.js", + "types": "lib/types/index.d.ts", + "exports": { + ".": { + "types": "./lib/types/index.d.ts", + "default": "./lib/index.js" + }, + "./invariant": { + "types": "./lib/types/invariant.d.ts", + "default": "./lib/invariant.js" + }, + "./src/*": "./src/*", + "./package.json": "./package.json" + }, + "files": [ + "lib/index.js", + "lib/invariant.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", + "src" + ], + "license": "BSD-3-Clause", + "peerDependencies": { + "@deepseek-ai/dsh-brand": "^0.0.1", + "@deepseek-ai/dsh-invariants": "^0.0.1", + "cordis": "^4.0.0-rc.7" + }, + "dependencies": { + "e2b": "2.29.1", + "schemastery": "^3.18.0" + }, + "devDependencies": { + "@deepseek-ai/dsh-brand": "workspace:^", + "@deepseek-ai/dsh-invariants": "workspace:^", + "@deepseek-ai/dsh-loader-smoke": "workspace:^", + "cordis": "^4.0.0-rc.7" + } +} diff --git a/packages/e2b/e2b/src/index.ts b/packages/e2b/e2b/src/index.ts new file mode 100644 index 0000000000..29cbf1dc53 --- /dev/null +++ b/packages/e2b/e2b/src/index.ts @@ -0,0 +1,240 @@ +/** + * Shared ownership of one E2B sandbox. Capability adapters await the same SDK + * handle, so filesystem and process operations inhabit one remote Linux world. + * @module @deepseek-ai/dsh-e2b + */ + +import { posix } from 'node:path' +import { Context, Service } from 'cordis' +import z from 'schemastery' +import { Sandbox } from 'e2b' +import type { Branded } from '@deepseek-ai/dsh-brand' + +export { + CommandExitError, + FileNotFoundError, + FileType, + Sandbox, + SandboxError, + SandboxNotFoundError, + TimeoutError, +} from 'e2b' +export type { CommandHandle, CommandResult, EntryInfo } from 'e2b' + +/** Opaque E2B sandbox identity used for reconnecting a later harness process. */ +export type E2BSandboxId = Branded<'E2BSandboxId'> + +/** + * Brand an SDK sandbox id after E2B has created or resolved it. + * @param value - E2B's opaque sandbox id. + * @returns the same string with the harness brand. + */ +export function E2BSandboxId(value: string): E2BSandboxId { + return value as E2BSandboxId +} + +/** + * Quote one opaque argument for the SDK's unavoidable `/bin/bash -l -c` layer. + * @param value - Exact argument value to preserve. + * @returns A single shell word with no interpolation. + */ +export function quoteE2BShellArg(value: string): string { + return `'${value.replaceAll('\'', "'\"'\"'")}'` +} + +/** Action taken on the owned sandbox when the Cordis service is disposed. */ +export type E2BDisposeMode = 'kill' | 'pause' | 'leave' + +/** Action E2B takes when a newly created sandbox reaches its lifetime. */ +export type E2BTimeoutMode = 'kill' | 'pause' + +/** Configuration for the shared E2B sandbox owner. */ +export interface Config { + /** API key; omission reads `E2B_API_KEY`. It is never forwarded into the sandbox. */ + apiKey?: string + /** Existing sandbox to reconnect instead of creating a new one. */ + sandboxId?: string + /** Template name or id for a newly created sandbox. */ + template?: string + /** Shared remote working directory, created before adapters receive the sandbox. */ + cwd?: string + /** E2B sandbox lifetime in milliseconds. */ + timeoutMs?: number + /** E2B action when a newly created sandbox reaches `timeoutMs`. */ + onTimeout?: E2BTimeoutMode + /** Disposal policy; `pause` and `leave` retain remote state for reconnect. */ + onDispose?: E2BDisposeMode +} + +interface ResolvedConfig { + apiKey: string + cwd: string + timeoutMs: number + onTimeout: E2BTimeoutMode + onDispose: E2BDisposeMode + sandboxId?: string + template?: string +} + +interface SchemaResolvedConfig extends Config { + cwd: string + timeoutMs: number + onTimeout: E2BTimeoutMode + onDispose: E2BDisposeMode +} + +declare module 'cordis' { + interface Context { + e2b: E2BSandboxService + } +} + +/** + * Owns one lazily consumable E2B SDK handle and its final kill/pause/leave + * decision. The connection begins at plugin construction; adapters await + * {@link getSandbox} before their first operation. + */ +export class E2BSandboxService extends Service { + static Config: z = z.object({ + apiKey: z.string(), + sandboxId: z.string(), + template: z.string(), + cwd: z.string().default('/home/user/workspace'), + timeoutMs: z.number().default(300_000), + onTimeout: z.union(['kill', 'pause'] as const).default('pause'), + onDispose: z.union(['kill', 'pause', 'leave'] as const).default('kill'), + }) + + /** Validated remote working directory shared by provider adapters. */ + readonly cwd: string + /** Remote directory reserved for adapter-owned process and terminal state. */ + readonly runtimeRoot: string + /** Whether this service creates a sandbox rather than reconnecting one. */ + readonly created: boolean + /** Configured action when a newly created sandbox reaches its lifetime. */ + readonly timeoutMode: E2BTimeoutMode + /** Configured final sandbox disposition. */ + readonly disposeMode: E2BDisposeMode + /** Sandbox id once E2B has created or resolved the remote runtime. */ + readonly sandboxId: Promise + + private readonly config: ResolvedConfig + private readonly ready: Promise + private disposed = false + + constructor(ctx: Context, config: Config) { + super(ctx, 'e2b') + // Schemastery fills these fields before construction; the type does not encode that step. + const resolved = config as SchemaResolvedConfig + const apiKey = config.apiKey ?? process.env.E2B_API_KEY + this.config = { + apiKey: apiKey ?? '', + cwd: resolved.cwd, + timeoutMs: resolved.timeoutMs, + onTimeout: resolved.onTimeout, + onDispose: resolved.onDispose, + ...(config.sandboxId !== undefined ? { sandboxId: config.sandboxId } : {}), + ...(config.template !== undefined ? { template: config.template } : {}), + } + this.validate() + this.cwd = this.config.cwd + this.runtimeRoot = posix.join(this.cwd, '.dsh-e2b') + this.created = this.config.sandboxId === undefined + this.timeoutMode = this.config.onTimeout + this.disposeMode = this.config.onDispose + this.ready = this.open() + // A deployment may load the owner before any adapter uses it. Keep a + // failed eager connection observed; getSandbox() still returns the error. + void this.ready.catch(() => {}) + this.sandboxId = this.ready.then(sandbox => E2BSandboxId(sandbox.sandboxId)) + void this.sandboxId.catch(() => {}) + + ctx.effect(() => async () => { + this.disposed = true + let sandbox: Sandbox + try { + sandbox = await this.ready + } catch { + // Connection creation already failed and is exposed by getSandbox(); + // there is no remote resource for teardown to own. + return + } + switch (this.config.onDispose) { + case 'kill': + await sandbox.kill() + return + case 'pause': { + await sandbox.pause() + return + } + case 'leave': + return + } + }, 'e2b sandbox teardown') + } + + /** + * Return the shared live SDK handle. + * @returns the created or reconnected sandbox after the configured cwd exists. + * @throws when E2B rejects creation/reconnection or the service is disposing. + */ + async getSandbox(): Promise { + if (this.disposed) throw new Error('E2B sandbox service is disposing') + return await this.ready + } + + private validate(): void { + if (this.config.apiKey.length === 0) { + throw new Error('dsh-e2b: configure apiKey or set E2B_API_KEY') + } + if (!posix.isAbsolute(this.config.cwd)) { + throw new Error(`dsh-e2b: cwd must be an absolute Linux path: ${this.config.cwd}`) + } + if (!Number.isFinite(this.config.timeoutMs) || this.config.timeoutMs <= 0) { + throw new Error('dsh-e2b: timeoutMs must be a positive finite number') + } + if (this.config.sandboxId !== undefined && this.config.sandboxId.length === 0) { + throw new Error('dsh-e2b: sandboxId must be non-empty when provided') + } + if (this.config.sandboxId !== undefined && this.config.template !== undefined) { + throw new Error('dsh-e2b: template applies only when creating; omit it when sandboxId reconnects') + } + } + + private async open(): Promise { + const connection = { + apiKey: this.config.apiKey, + timeoutMs: this.config.timeoutMs, + } + const sandbox = this.config.sandboxId === undefined + ? this.config.template === undefined + ? await Sandbox.create({ + ...connection, + secure: true, + lifecycle: { onTimeout: this.config.onTimeout, autoResume: this.config.onTimeout === 'pause' }, + }) + : await Sandbox.create(this.config.template, { + ...connection, + secure: true, + lifecycle: { onTimeout: this.config.onTimeout, autoResume: this.config.onTimeout === 'pause' }, + }) + : await Sandbox.connect(this.config.sandboxId, connection) + try { + await sandbox.files.makeDir(this.cwd) + await sandbox.files.makeDir(this.runtimeRoot) + await sandbox.commands.run(`chmod 700 -- ${quoteE2BShellArg(this.runtimeRoot)}`) + return sandbox + } catch (error: unknown) { + if (this.created) { + try { + await sandbox.kill() + } catch (_cleanupFailure) { + // The setup failure remains authoritative; E2B will still apply the configured lifetime. + } + } + throw error + } + } +} + +export default E2BSandboxService diff --git a/packages/e2b/e2b/src/invariant.ts b/packages/e2b/e2b/src/invariant.ts new file mode 100644 index 0000000000..891cabb2db --- /dev/null +++ b/packages/e2b/e2b/src/invariant.ts @@ -0,0 +1,30 @@ +/** + * Package-owned invariant companion for `@deepseek-ai/dsh-e2b`. + * @module @deepseek-ai/dsh-e2b/invariant + */ + +/* jscpd:ignore-start */ +import type { Context } from 'cordis' +import type { InvariantInstaller } from '@deepseek-ai/dsh-invariants' + +const PACKAGE_NAME = '@deepseek-ai/dsh-e2b' + +/** Cordis companion plugin name. */ +export const name = 'e2b-invariant' +/** Service required before the companion can reserve package ownership. */ +export const inject = ['invariants'] + +/** + * No runtime invariant: sandbox creation and teardown have one SDK promise and + * no independent event or mutable-data relationship to cross-check. + */ +const install: InvariantInstaller = () => {} + +/** + * Register this package's invariant companion. + * @param ctx - Cordis context carrying the invariant service. + * @returns the installed registration's disposer after setup succeeds. + */ +export const apply = (ctx: Context): Promise<() => void> => + Promise.resolve(ctx.invariants.register(PACKAGE_NAME, install)) +/* jscpd:ignore-end */ diff --git a/packages/e2b/e2b/tests/composition.e2e.ts b/packages/e2b/e2b/tests/composition.e2e.ts new file mode 100644 index 0000000000..8c01fae7b4 --- /dev/null +++ b/packages/e2b/e2b/tests/composition.e2e.ts @@ -0,0 +1,39 @@ +import { access } from 'node:fs/promises' +import { join } from 'node:path' +import { fileURLToPath } from 'node:url' +import { describe, expect, it } from 'vitest' +import { runLoaderSmoke } from '@deepseek-ai/dsh-loader-smoke' +import { Sandbox, SandboxNotFoundError } from '@deepseek-ai/dsh-e2b' + +const fixtureRoot = fileURLToPath(new URL('../../../../examples/headless-agent/tests/fixtures/e2b/e2b/', import.meta.url)) +const binScript = join(fixtureRoot, 'bin.ts') +const configPath = join(fixtureRoot, 'cordis.yml') +const tsconfigPath = fileURLToPath(new URL('../../../../tsconfig.json', import.meta.url)) + +describe.skipIf(!process.env.E2B_API_KEY)('E2B live Loader composition', () => { + it('shares remote state across FS and Bash without creating host workspace files', async () => { + const { stdout, stderr } = await runLoaderSmoke({ + label: 'E2B composition', + tempDirPrefix: 'dsh-e2b-composition-', + binScript, + libBinScript: binScript, + configPath, + tsconfigPath, + processTimeoutMs: 90_000, + inspect: async (cwd) => { + await expect(access(join(cwd, 'from-fs.txt'))).rejects.toMatchObject({ code: 'ENOENT' }) + await expect(access(join(cwd, 'from-bash.txt'))).rejects.toMatchObject({ code: 'ENOENT' }) + }, + }) + + expect(stderr).toBe('') + const output = JSON.parse(stdout) as Record + expect(output).toMatchObject({ + bashRead: 'written-by-fs\n', + fsRead: 'written-by-bash\n', + }) + const apiKey = process.env.E2B_API_KEY + if (apiKey === undefined) throw new Error('E2B_API_KEY disappeared during the live composition test') + await expect(Sandbox.getInfo(String(output.sandboxId), { apiKey })).rejects.toBeInstanceOf(SandboxNotFoundError) + }, 105_000) +}) diff --git a/packages/e2b/e2b/tests/e2b.spec.ts b/packages/e2b/e2b/tests/e2b.spec.ts new file mode 100644 index 0000000000..c842731f5c --- /dev/null +++ b/packages/e2b/e2b/tests/e2b.spec.ts @@ -0,0 +1,218 @@ +import { beforeEach, describe, expect, it, vi } from 'vitest' +import { Context } from 'cordis' +import type { Sandbox as SandboxType } from 'e2b' +import E2BSandboxService, { + E2BSandboxId, + quoteE2BShellArg, +} from '@deepseek-ai/dsh-e2b' +import * as E2BInvariant from '../src/invariant.ts' +import InvariantService from '@deepseek-ai/dsh-invariants' + +const sdk = vi.hoisted(() => ({ + create: vi.fn(), + connect: vi.fn(), +})) + +vi.mock('e2b', async (importOriginal) => { + const actual = await importOriginal() + // The mock replaces only the SDK's static factory surface and is never constructed. + // eslint-disable-next-line @typescript-eslint/no-extraneous-class + class FakeSandbox { + static create(...args: unknown[]): unknown { + return sdk.create(...args) + } + + static connect(...args: unknown[]): unknown { + return sdk.connect(...args) + } + } + return { ...actual, Sandbox: FakeSandbox } +}) + +interface SandboxFixture { + sandbox: SandboxType + makeDir: ReturnType + run: ReturnType + kill: ReturnType + pause: ReturnType +} + +function fakeSandbox(id = 'sandbox-1'): SandboxFixture { + const makeDir = vi.fn().mockResolvedValue(true) + const run = vi.fn().mockResolvedValue({ exitCode: 0, stdout: '', stderr: '' }) + const kill = vi.fn().mockResolvedValue(undefined) + const pause = vi.fn().mockResolvedValue(true) + const sandbox = { + sandboxId: id, + files: { makeDir }, + commands: { run }, + kill, + pause, + } as unknown as SandboxType + return { sandbox, makeDir, run, kill, pause } +} + +beforeEach(() => { + sdk.create.mockReset() + sdk.connect.mockReset() + vi.unstubAllEnvs() +}) + +describe('E2BSandboxService', () => { + it('creates one protected shared sandbox and kills it on default disposal', async () => { + const fixture = fakeSandbox() + sdk.create.mockResolvedValue(fixture.sandbox) + const ctx = new Context() + const fiber = await ctx.plugin(E2BSandboxService, { apiKey: 'test-key' }) + + const service = ctx.e2b + await expect(service.getSandbox()).resolves.toBe(fixture.sandbox) + await expect(service.sandboxId).resolves.toBe(E2BSandboxId('sandbox-1')) + expect(service.cwd).toBe('/home/user/workspace') + expect(service.runtimeRoot).toBe('/home/user/workspace/.dsh-e2b') + expect(service.created).toBe(true) + expect(service.timeoutMode).toBe('pause') + expect(service.disposeMode).toBe('kill') + expect(sdk.create).toHaveBeenCalledWith({ + apiKey: 'test-key', + timeoutMs: 300_000, + secure: true, + lifecycle: { onTimeout: 'pause', autoResume: true }, + }) + expect(fixture.makeDir).toHaveBeenNthCalledWith(1, '/home/user/workspace') + expect(fixture.makeDir).toHaveBeenNthCalledWith(2, '/home/user/workspace/.dsh-e2b') + expect(fixture.run).toHaveBeenCalledWith("chmod 700 -- '/home/user/workspace/.dsh-e2b'") + + await fiber.dispose() + expect(fixture.kill).toHaveBeenCalledOnce() + await expect(service.getSandbox()).rejects.toThrow(/disposing/) + }) + + it('creates from a template, honors timeout and pause policies, and reads the key from the environment', async () => { + vi.stubEnv('E2B_API_KEY', 'environment-key') + const fixture = fakeSandbox('template-sandbox') + sdk.create.mockResolvedValue(fixture.sandbox) + const ctx = new Context() + const fiber = await ctx.plugin(E2BSandboxService, { + template: 'agent-template', + cwd: '/workspace/project', + timeoutMs: 60_000, + onTimeout: 'kill', + onDispose: 'pause', + }) + await ctx.e2b.getSandbox() + + expect(sdk.create).toHaveBeenCalledWith('agent-template', { + apiKey: 'environment-key', + timeoutMs: 60_000, + secure: true, + lifecycle: { onTimeout: 'kill', autoResume: false }, + }) + await fiber.dispose() + expect(fixture.pause).toHaveBeenCalledOnce() + expect(fixture.kill).not.toHaveBeenCalled() + }) + + it('accepts an already-paused result during configured pause disposal', async () => { + const fixture = fakeSandbox() + fixture.pause.mockResolvedValue(false) + sdk.create.mockResolvedValue(fixture.sandbox) + const ctx = new Context() + const fiber = await ctx.plugin(E2BSandboxService, { apiKey: 'test-key', onDispose: 'pause' }) + await ctx.e2b.getSandbox() + await fiber.dispose() + expect(fixture.pause).toHaveBeenCalledOnce() + }) + + it('reconnects without applying creation lifecycle options and can leave state running', async () => { + const fixture = fakeSandbox('existing') + sdk.connect.mockResolvedValue(fixture.sandbox) + const ctx = new Context() + const fiber = await ctx.plugin(E2BSandboxService, { + apiKey: 'test-key', + sandboxId: 'existing', + timeoutMs: 90_000, + onDispose: 'leave', + }) + await ctx.e2b.getSandbox() + + expect(ctx.e2b.created).toBe(false) + expect(sdk.connect).toHaveBeenCalledWith('existing', { apiKey: 'test-key', timeoutMs: 90_000 }) + expect(sdk.create).not.toHaveBeenCalled() + await fiber.dispose() + expect(fixture.kill).not.toHaveBeenCalled() + expect(fixture.pause).not.toHaveBeenCalled() + }) + + it('kills a newly created sandbox when remote directory setup fails', async () => { + const fixture = fakeSandbox() + fixture.makeDir.mockRejectedValueOnce(new Error('setup failed')) + sdk.create.mockResolvedValue(fixture.sandbox) + const ctx = new Context() + const fiber = await ctx.plugin(E2BSandboxService, { apiKey: 'test-key' }) + + await expect(ctx.e2b.getSandbox()).rejects.toThrow('setup failed') + await expect(ctx.e2b.sandboxId).rejects.toThrow('setup failed') + expect(fixture.kill).toHaveBeenCalledOnce() + await fiber.dispose() + }) + + it('preserves the setup failure even when cleanup also fails', async () => { + const fixture = fakeSandbox() + fixture.run.mockRejectedValueOnce(new Error('chmod failed')) + fixture.kill.mockRejectedValueOnce(new Error('cleanup failed')) + sdk.create.mockResolvedValue(fixture.sandbox) + const ctx = new Context() + await ctx.plugin(E2BSandboxService, { apiKey: 'test-key' }) + await expect(ctx.e2b.getSandbox()).rejects.toThrow('chmod failed') + }) + + it('does not kill a reconnected sandbox when setup fails', async () => { + const fixture = fakeSandbox() + fixture.makeDir.mockRejectedValueOnce(new Error('setup failed')) + sdk.connect.mockResolvedValue(fixture.sandbox) + const ctx = new Context() + await ctx.plugin(E2BSandboxService, { apiKey: 'test-key', sandboxId: 'existing' }) + await expect(ctx.e2b.getSandbox()).rejects.toThrow('setup failed') + expect(fixture.kill).not.toHaveBeenCalled() + }) + + it.each([ + [{ apiKey: '' }, /configure apiKey/], + [{ apiKey: 'x', cwd: 'relative' }, /absolute Linux path/], + [{ apiKey: 'x', timeoutMs: 0 }, /positive finite/], + [{ apiKey: 'x', sandboxId: '' }, /sandboxId must be non-empty/], + [{ apiKey: 'x', sandboxId: 'one', template: 'two' }, /template applies only/], + ] as const)('fails self-contained configuration before opening E2B: %j', async (config, message) => { + vi.stubEnv('E2B_API_KEY', '') + const ctx = new Context() + await expect(ctx.plugin(E2BSandboxService, config)).rejects.toThrow(message) + expect(sdk.create).not.toHaveBeenCalled() + expect(sdk.connect).not.toHaveBeenCalled() + }) + + it('requires a key when both config and the environment omit it', async () => { + const original = process.env.E2B_API_KEY + delete process.env.E2B_API_KEY + try { + const ctx = new Context() + await expect(ctx.plugin(E2BSandboxService, {})).rejects.toThrow(/configure apiKey/) + } finally { + if (original === undefined) delete process.env.E2B_API_KEY + else process.env.E2B_API_KEY = original + } + }) +}) + +describe('E2B helpers and invariant companion', () => { + it('quotes opaque shell arguments without interpolation', () => { + expect(quoteE2BShellArg("a'b $HOME")).toBe("'a'\"'\"'b $HOME'") + }) + + it('registers the package-owned empty invariant installer', async () => { + const ctx = new Context() + await ctx.plugin(InvariantService, { enabled: true }) + const fiber = await ctx.plugin(E2BInvariant).await() + await fiber.dispose() + }) +}) diff --git a/packages/e2b/e2b/tsconfig.json b/packages/e2b/e2b/tsconfig.json new file mode 100644 index 0000000000..5890a41d66 --- /dev/null +++ b/packages/e2b/e2b/tsconfig.json @@ -0,0 +1,25 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": ["src"], + "references": [ + { + "path": "../../../vendor/cosmokit" + }, + { + "path": "../../../vendor/cordis" + }, + { + "path": "../../../vendor/schemastery" + }, + { + "path": "../../util/brand" + }, + { + "path": "../../support/invariants" + } + ] +} diff --git a/packages/fs/README.i18n.yaml b/packages/fs/README.i18n.yaml index c1a381cb81..eabb6a602e 100644 --- a/packages/fs/README.i18n.yaml +++ b/packages/fs/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/fs/README.md -README.md: b6adabd5744cb2b3dcee78b71815f8e95ba780f1 -README.zh.md: 0841f538932452921d2b0d7d9534f023672f241d +README.md: 8ac8807cc9ca4b5cc31b686cef9796a8e42b7f81 +README.zh.md: 49705790cc0cdf1f7199251e09af0af01f704efb diff --git a/packages/fs/README.md b/packages/fs/README.md index b6adabd574..460af7c53b 100644 --- a/packages/fs/README.md +++ b/packages/fs/README.md @@ -8,12 +8,13 @@ The filesystem stack: a provider seam (execution-world paths, bounded text IO, a |---|---|---| | `fs/` | Provider seam: canonical process paths/file URIs/containment, text IO, and atomic mutation primitives; owns the `fs/*` policy events | `ctx.fs` | | `fs-local/` | Local-filesystem `FileSystem` implementation | (registers `ctx.fs`) | +| `fs-e2b/` | E2B-backed `FileSystem` implementation sharing the remote runtime owned by `ctx.e2b` | (registers `ctx.fs`) | | `fs-sandbox/` | Sandbox-enforcing `FileSystem`: extends `fs-local` and fences write/edit by the per-call mode + workspace root policy (read-only denies, workspace-write contains to the session workspace + temp roots), reads pass through | (registers `ctx.fs`) | | `fs-policy/` | Policy gate plugin: observed-state + read-before-edit + version-guarded write/edit, via the `fs/*` event gate | (no service — `fs/*` listeners) | | `tool-fs/` | Model-facing `read`/`write`/`edit` tools AND the executor (reads via `ctx.fs`, owns read windowing, dispatches `fs/*`); preserves filesystem semantics for session-cwd-relative paths and advertises sandbox escalation fields when the mounted `ctx.fs` confines | (registers on `ctx.tools`) | | `tool-fs-search/` | Model-facing `glob`/`grep` discovery tools when `rg` is available on the bash executor `PATH`, backed by fixed ripgrep commands through `ctx.bash`, NOT by `ctx.fs` provider methods | (registers on `ctx.tools`) | -The interface lives at `fs/fs/`. A sandboxed, remote, or project-scoped filesystem backend can replace `fs-local` without touching the seam, the policy gate, or the model-facing tool schemas — `fs-sandbox` is the first such replacement (an in-process path fence over the shared sandbox mode; see [the cross-family fs sandbox Agent Note](../../.agents/notes/implemented/feature/2026-07-14-cross-family-fs-sandbox.md)). The policy (`fs-policy/`) is a plugin that participates only through the `fs/*` event gate, not a service the tool injects — so dropping it gracefully loses the policy and leaves the unconstrained bare provider rather than breaking the tool. A deployment that loads `tool-fs/` is expected to also load it. The mode fence and the read-before-edit gate are orthogonal and compose. Discovery (`tool-fs-search/`) deliberately does NOT extend the provider seam: search is a process-backed `rg` workflow on the bash executor, so filesystem backends stay free of a universal search contract; its tools register only when that executor can find `rg`, and its results are follow-up-readable when the bash workdir and the `read` root are the same workspace (the co-located deployment its README documents). +The interface lives at `fs/fs/`. A sandboxed, remote, or project-scoped filesystem backend can replace `fs-local` without touching the seam, the policy gate, or the model-facing tool schemas: `fs-sandbox` provides an in-process path fence over the shared sandbox mode ([decision](../../.agents/notes/implemented/feature/2026-07-14-cross-family-fs-sandbox.md)), while `fs-e2b` places file state in the remote runtime shared with the E2B subprocess provider ([POC decision](../../.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md)). The policy (`fs-policy/`) is a plugin that participates only through the `fs/*` event gate, not a service the tool injects — so dropping it gracefully loses the policy and leaves the unconstrained bare provider rather than breaking the tool. A deployment that loads `tool-fs/` is expected to also load it. The mode fence and the read-before-edit gate are orthogonal and compose. Discovery (`tool-fs-search/`) deliberately does NOT extend the provider seam: search is a process-backed `rg` workflow on the bash executor, so filesystem backends stay free of a universal search contract; its tools register only when that executor can find `rg`, and its results are follow-up-readable when the bash workdir and the `read` root are the same workspace (the co-located deployment its README documents). ## No timeouts on file IO diff --git a/packages/fs/README.zh.md b/packages/fs/README.zh.md index 0841f53893..5f70e3cb50 100644 --- a/packages/fs/README.zh.md +++ b/packages/fs/README.zh.md @@ -8,12 +8,13 @@ |---|---|---| | `fs/` | 提供方 seam：规范化进程路径、文件 URI 与包含关系、文本 I/O 和原子变更原语；拥有 `fs/*` 政策事件 | `ctx.fs` | | `fs-local/` | 本地文件系统 `FileSystem` 实现 | （注册 `ctx.fs`） | +| `fs-e2b/` | 以 E2B 为后端的 `FileSystem` 实现，共享由 `ctx.e2b` 拥有的远程运行时 | （注册 `ctx.fs`） | | `fs-sandbox/` | 强制沙箱的 `FileSystem`：扩展 `fs-local`，并按每次调用的模式与工作区根政策约束写入/编辑（只读模式拒绝，工作区写入模式限制在会话工作区与临时根目录内）；读取直接通过 | （注册 `ctx.fs`） | | `fs-policy/` | 政策门禁插件：通过 `fs/*` 事件门禁提供已观察状态、编辑前读取和版本防护的写入/编辑 | （无服务，仅有 `fs/*` 监听器） | | `tool-fs/` | 面向模型的 `read`/`write`/`edit` 工具以及执行器（通过 `ctx.fs` 读取，拥有读取窗口逻辑，分派 `fs/*`）；为会话 cwd 相对路径保留文件系统语义，并在已挂载的 `ctx.fs` 实施约束时声明沙箱升级字段 | （注册到 `ctx.tools`） | | `tool-fs-search/` | 面向模型的 `glob`/`grep` 发现工具；当 `rg` 位于 bash 执行器 `PATH` 上时注册，通过 `ctx.bash` 运行固定 ripgrep 命令，而不是使用 `ctx.fs` 提供方方法 | （注册到 `ctx.tools`） | -接口位于 `fs/fs/`。沙箱化、远程或限定项目作用域的文件系统后端可以替换 `fs-local`，而无需更改 seam、政策门禁或面向模型的工具 schema；`fs-sandbox` 是第一个这样的替代实现（基于共享沙箱模式的进程内路径围栏；见[跨能力族 fs 沙箱 Agent Note](../../.agents/notes/implemented/feature/2026-07-14-cross-family-fs-sandbox.md)）。政策（`fs-policy/`）是一个只通过 `fs/*` 事件门禁参与的插件，不是工具注入的服务；因此移除它会平稳失去政策，留下不受约束的裸提供方，而不会破坏工具。加载 `tool-fs/` 的部署也应加载该插件。模式围栏与编辑前读取门禁彼此正交，可以组合。发现（`tool-fs-search/`）有意不扩展提供方 seam：搜索是在 bash 执行器上运行 `rg`、由进程支持的工作流，因此文件系统后端无需承担通用搜索契约；只有当执行器能找到 `rg` 时，其工具才会注册。如果 bash 工作目录与 `read` 根目录是同一工作区，结果就能继续读取，这也是其 README 所述的共置部署。 +接口位于 `fs/fs/`。沙箱化、远程或限定项目作用域的文件系统后端可以替换 `fs-local`，而无需更改 seam、政策门禁或面向模型的工具 schema：`fs-sandbox` 基于共享沙箱模式提供进程内路径围栏（[决策](../../.agents/notes/implemented/feature/2026-07-14-cross-family-fs-sandbox.md)），而 `fs-e2b` 则把文件状态置于与 E2B 进程管理提供方共享的远程运行时中（[POC 决策](../../.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md)）。政策（`fs-policy/`）是一个只通过 `fs/*` 事件门禁参与的插件，不是工具注入的服务；因此移除它会平稳失去政策，留下不受约束的裸提供方，而不会破坏工具。加载 `tool-fs/` 的部署也应加载该插件。模式围栏与编辑前读取门禁彼此正交，可以组合。发现（`tool-fs-search/`）有意不扩展提供方 seam：搜索是在 bash 执行器上运行 `rg`、由进程支持的工作流，因此文件系统后端无需承担通用搜索契约；只有当执行器能找到 `rg` 时，其工具才会注册。如果 bash 工作目录与 `read` 根目录是同一工作区，结果就能继续读取，这也是其 README 所述的共置部署。 ## 文件 I/O 不设超时 diff --git a/packages/fs/fs-e2b/README.i18n.yaml b/packages/fs/fs-e2b/README.i18n.yaml new file mode 100644 index 0000000000..d3ea2786b2 --- /dev/null +++ b/packages/fs/fs-e2b/README.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write packages/fs/fs-e2b/README.md +README.md: 2b23884e345e7c39b0559465afa0bc588699b70a +README.zh.md: 1a9a174ac90c59b68b02887303f58ee2a99dc72e diff --git a/packages/fs/fs-e2b/README.md b/packages/fs/fs-e2b/README.md new file mode 100644 index 0000000000..2b23884e34 --- /dev/null +++ b/packages/fs/fs-e2b/README.md @@ -0,0 +1,29 @@ +# @deepseek-ai/dsh-fs-e2b + +English | [中文](README.zh.md) + +E2B implementation of the [`@deepseek-ai/dsh-fs`](../fs/README.md) provider seam. It has no config: load [`@deepseek-ai/dsh-e2b`](../../e2b/e2b/README.md) first, then this service in place of `dsh-fs-local`. The provider uses the owner's remote cwd and SDK handle, so file tools observe the same world as E2B-backed Bash processes. + +## Behavior + +- **Remote identity and metadata** — relative paths resolve as POSIX paths against the caller cwd or `ctx.e2b.cwd`; `realpath -m` supplies canonical target identity without requiring the final file to exist. `stat`, no-follow `lstat`, and stable one-level directory listings project E2B metadata into the filesystem seam. Versions are opaque hashes of E2B metadata plus a per-write extended attribute. +- **UTF-8 reads** — whole reads and streamed reads preserve cross-chunk decoding, reject invalid UTF-8, and use the seam's 8192-byte NUL sample for binary detection. The model-facing tool still owns size selection and line windowing. +- **Atomic mutations** — writes upload a mode-`0600` temporary sibling, preserve an existing file's POSIX mode, and publish through same-directory Linux `mv -f`. E2B creates missing parent directories. Literal edits LF-normalize for matching, restore dominant CRLF storage, and serialize mutations per canonical target within the host process. Optional create/version guards keep the base seam's observed-state semantics. +- **Failures and cancellation** — E2B not-found, permission, abort, and other controller failures map to the existing `FsError` vocabulary. Cancellation is best-effort at SDK request boundaries; a successful rename is the commit point. + +The provider does not copy, mount, or reconcile the host workspace. Giving it a host path as `cwd` creates a remote directory with the same spelling only. + +## Model Experience + +Indirectly, through [`dsh-tool-fs`](../tool-fs/README.md), which renders remote UTF-8 content, directory results, mutation acknowledgements, and provider errors while E2B identity and transport remain internal. + +#### KV Cache effect + +No direct invalidation; the named consumer owns any request-prefix changes. + +## Known Limitations and Deferred Work + +- **No host synchronization** — an empty E2B cwd stays empty until a tool, command, template, or external process populates it; local files are neither uploaded nor reflected back. +- **Mutation coordination is host-process-local** — another harness connection or remote command can race the adapter; version guards detect only metadata changes represented by E2B. +- **Whole-file mutation costs remain** — overwrite diffs and literal edits read complete files into host memory, and every operation incurs E2B controller latency. +- **Custom templates must support the used Linux and envd features** — `realpath`, `chmod`, `mv`, same-filesystem POSIX rename, streaming reads, and file metadata extended attributes are required; unsupported templates fail rather than degrade silently. diff --git a/packages/fs/fs-e2b/README.zh.md b/packages/fs/fs-e2b/README.zh.md new file mode 100644 index 0000000000..1a9a174ac9 --- /dev/null +++ b/packages/fs/fs-e2b/README.zh.md @@ -0,0 +1,29 @@ +# @deepseek-ai/dsh-fs-e2b + +[English](README.md) | 中文 + +[`@deepseek-ai/dsh-fs`](../fs/README.md) 提供方 seam 的 E2B 实现。它没有配置：先加载 [`@deepseek-ai/dsh-e2b`](../../e2b/e2b/README.md)，再用本服务取代 `dsh-fs-local`。该提供方使用所有者的远程 cwd 和 SDK 句柄，因此文件工具观察到的环境与 E2B 后端 Bash 进程相同。 + +## 行为 + +- **远程身份与元数据**：相对路径以调用方 cwd 或 `ctx.e2b.cwd` 为基准，按照 POSIX 路径解析；`realpath -m` 提供规范化目标身份，且不要求最终文件存在。`stat`、不跟随链接的 `lstat` 和稳定的单层目录列表会把 E2B 元数据投影到文件系统 seam。版本是 E2B 元数据与每次写入设置的扩展属性所组成的不透明哈希。 +- **UTF-8 读取**：完整读取和流式读取会保留跨分片解码、拒绝无效 UTF-8，并使用 seam 的 8192 字节 NUL 样本检测二进制内容。面向模型的工具仍负责选择大小和行窗口。 +- **原子变更**：写入会上传 mode 为 `0600` 的同级临时文件，保留现有文件的 POSIX mode，并通过同目录 Linux `mv -f` 发布。E2B 会创建缺失的父目录。字面量编辑匹配时会规范化为 LF，存储时恢复占主导的 CRLF，并在宿主进程内按规范化目标串行执行变更。可选的创建／版本防护会保留基础 seam 的已观察状态语义。 +- **失败与取消**：E2B 的未找到、权限、中止及其他控制器故障会映射到现有 `FsError` 词汇。取消在 SDK 请求边界上采用尽力而为语义；成功 rename 是提交点。 + +该提供方不会复制、挂载或协调宿主工作区。把宿主路径用作 `cwd`，只会在远程创建一个拼写相同的目录。 + +## 模型体验 + +通过 [`dsh-tool-fs`](../tool-fs/README.md) 间接影响模型；该工具会渲染远程 UTF-8 内容、目录结果、变更确认和提供方错误，而 E2B 身份及传输保持内部实现。 + +#### KV Cache 影响 + +不会直接失效；请求前缀变更由具名消费方负责。 + +## 已知限制与延后工作 + +- **不提供宿主同步**：空的 E2B cwd 会一直为空，直到工具、命令、模板或外部进程填充它；本地文件既不会上传，也不会同步回本地。 +- **变更协调仅限宿主进程内**：另一个 harness 连接或远程命令可能与适配器发生竞态；版本防护只能检测 E2B 元数据所体现的变更。 +- **仍需承担完整文件变更成本**：覆盖差异和字面量编辑会把完整文件读入宿主内存，每项操作也都会产生 E2B 控制器延迟。 +- **自定义模板必须支持所用的 Linux 与 envd 功能**：必须支持 `realpath`、`chmod`、`mv`、同一文件系统内的 POSIX rename、流式读取和文件元数据扩展属性；不支持的模板会失败，而不会静默降级。 diff --git a/packages/fs/fs-e2b/package.json b/packages/fs/fs-e2b/package.json new file mode 100644 index 0000000000..cc96cba6c9 --- /dev/null +++ b/packages/fs/fs-e2b/package.json @@ -0,0 +1,41 @@ +{ + "name": "@deepseek-ai/dsh-fs-e2b", + "description": "E2B filesystem implementation for DeepSeek Harness", + "version": "0.0.1", + "private": true, + "type": "module", + "main": "lib/index.js", + "types": "lib/types/index.d.ts", + "exports": { + ".": { + "types": "./lib/types/index.d.ts", + "default": "./lib/index.js" + }, + "./invariant": { + "types": "./lib/types/invariant.d.ts", + "default": "./lib/invariant.js" + }, + "./src/*": "./src/*", + "./package.json": "./package.json" + }, + "files": [ + "lib/index.js", + "lib/invariant.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", + "src" + ], + "license": "BSD-3-Clause", + "peerDependencies": { + "@deepseek-ai/dsh-e2b": "^0.0.1", + "@deepseek-ai/dsh-fs": "^0.0.1", + "@deepseek-ai/dsh-invariants": "^0.0.1", + "cordis": "^4.0.0-rc.7" + }, + "devDependencies": { + "@deepseek-ai/dsh-e2b": "workspace:^", + "@deepseek-ai/dsh-fs": "workspace:^", + "@deepseek-ai/dsh-invariants": "workspace:^", + "cordis": "^4.0.0-rc.7" + } +} diff --git a/packages/fs/fs-e2b/src/index.ts b/packages/fs/fs-e2b/src/index.ts new file mode 100644 index 0000000000..020b0f4857 --- /dev/null +++ b/packages/fs/fs-e2b/src/index.ts @@ -0,0 +1,423 @@ +/** + * E2B implementation of the filesystem provider seam. Paths, contents, and + * atomic staging files remain inside the shared remote sandbox. + * @module @deepseek-ai/dsh-fs-e2b + */ + +import { createHash, randomUUID } from 'node:crypto' +import { posix } from 'node:path' +import { FileSystem, FsError, FsTargetKey, FsVersion } from '@deepseek-ai/dsh-fs' +import type { + FsDirEntry, + FsEditOutcome, + FsEditRequest, + FsInfo, + FsPathInfo, + FsTarget, + FsWriteIntent, + FsWriteOutcome, +} from '@deepseek-ai/dsh-fs' +import { + CommandExitError, + FileNotFoundError, + FileType, + quoteE2BShellArg, +} from '@deepseek-ai/dsh-e2b' +import type { EntryInfo, Sandbox } from '@deepseek-ai/dsh-e2b' + +const VERSION_METADATA_KEY = 'dsh-version' +const BINARY_SAMPLE_BYTES = 8192 + +function assertNotAborted(signal: AbortSignal | undefined, operation: string): void { + if (signal?.aborted === true) throw new FsError(`${operation} aborted`, 'FS_ABORTED') +} + +function normalizeLineEndings(value: string): string { + return value.replaceAll('\r\n', '\n') +} + +function detectsCrlf(value: string): boolean { + const sample = value.slice(0, 4096) + const crlf = sample.split('\r\n').length - 1 + const lf = sample.split('\n').length - 1 - crlf + return crlf > lf +} + +function restoreLineEndings(value: string, crlf: boolean): string { + return crlf ? normalizeLineEndings(value).replaceAll('\n', '\r\n') : value +} + +function decodeText(bytes: Uint8Array, displayPath: string, binarySampleBytes: number): string { + if (bytes.subarray(0, binarySampleBytes).includes(0)) { + throw new FsError(`cannot read "${displayPath}": binary file`, 'FS_NOT_TEXT') + } + try { + return new TextDecoder('utf-8', { fatal: true }).decode(bytes) + } catch (error: unknown) { + throw new FsError(`cannot read "${displayPath}": invalid UTF-8 text`, 'FS_NOT_TEXT', { cause: error }) + } +} + +function signalOpts(signal: AbortSignal | undefined): { signal?: AbortSignal } { + return signal === undefined ? {} : { signal } +} + +function entryType(entry: EntryInfo): FsInfo['type'] { + switch (entry.type) { + case FileType.FILE: + return 'file' + case FileType.DIR: + return 'directory' + default: + return 'other' + } +} + +function entryVersion(entry: EntryInfo): ReturnType { + const facts = JSON.stringify([ + entry.metadata?.[VERSION_METADATA_KEY], + entry.path, + entry.type, + entry.size, + entry.mode, + entry.modifiedTime?.toISOString(), + entry.symlinkTarget, + ]) + return FsVersion(`e2b:${createHash('sha256').update(facts).digest('hex')}`) +} + +function mapError(error: unknown, operation: string, displayPath: string, signal?: AbortSignal): FsError { + if (error instanceof FsError) return error + if (signal?.aborted === true || (error instanceof DOMException && error.name === 'AbortError')) { + return new FsError(`${operation} aborted`, 'FS_ABORTED', { cause: error }) + } + if (error instanceof FileNotFoundError) { + return new FsError(`cannot ${operation} "${displayPath}": not found`, 'FS_NOT_FOUND', { cause: error }) + } + if (/permission denied|operation not permitted/i.test(String(error))) { + return new FsError(`cannot ${operation} "${displayPath}": permission denied`, 'FS_PERMISSION_DENIED', { cause: error }) + } + return new FsError(`cannot ${operation} "${displayPath}": ${String(error)}`, 'FS_IO_ERROR', { cause: error }) +} + +function literalEdit(content: string, request: FsEditRequest, displayPath: string): string { + const oldString = normalizeLineEndings(request.oldString) + const newString = normalizeLineEndings(request.newString) + if (oldString.length === 0) { + throw new FsError(`cannot edit "${displayPath}": old_string must be non-empty`, 'FS_EDIT_NOT_FOUND') + } + let matches = 0 + let offset = 0 + while (true) { + const found = content.indexOf(oldString, offset) + if (found < 0) break + matches += 1 + offset = found + oldString.length + } + if (matches === 0) throw new FsError(`cannot edit "${displayPath}": old_string was not found`, 'FS_EDIT_NOT_FOUND') + if (!request.replaceAll && matches !== 1) { + throw new FsError(`cannot edit "${displayPath}": old_string matched ${matches} times`, 'FS_AMBIGUOUS_EDIT') + } + return request.replaceAll ? content.split(oldString).join(newString) : content.replace(oldString, newString) +} + +/** Remote filesystem backend sharing the sandbox owned by `ctx.e2b`. */ +export class E2BFileSystem extends FileSystem { + static inject = ['e2b'] + + private readonly locks = new Map>() + + override async resolve(path: string, opts?: { cwd?: string; signal?: AbortSignal }): Promise { + assertNotAborted(opts?.signal, 'resolve') + if (path.trim().length === 0) throw new FsError('file_path must be a non-empty string', 'FS_NOT_FOUND') + const displayPath = posix.resolve(opts?.cwd ?? this.ctx.e2b.cwd, path) + try { + const sandbox = await this.ctx.e2b.getSandbox() + const targetKey = await this.canonicalPath(sandbox, displayPath, opts?.signal) + assertNotAborted(opts?.signal, 'resolve') + return { targetKey: FsTargetKey(targetKey), displayPath } + } catch (error: unknown) { + throw mapError(error, 'resolve', displayPath, opts?.signal) + } + } + + override async stat(target: FsTarget, signal?: AbortSignal): Promise { + assertNotAborted(signal, 'stat') + const entry = await this.probe(String(target.targetKey), target.displayPath, signal) + if (entry === undefined) return undefined + return { + version: entryVersion(entry), + type: entryType(entry), + ...(entry.type === FileType.FILE ? { size: entry.size } : {}), + } + } + + override async lstat(path: string, opts?: { cwd?: string }, signal?: AbortSignal): Promise { + assertNotAborted(signal, 'lstat') + if (path.trim().length === 0) throw new FsError('file_path must be a non-empty string', 'FS_NOT_FOUND') + const displayPath = posix.resolve(opts?.cwd ?? this.ctx.e2b.cwd, path) + const entry = await this.probe(displayPath, displayPath, signal) + if (entry === undefined) return undefined + const type = entry.symlinkTarget !== undefined + ? 'symlink' as const + : entry.type === FileType.FILE + ? 'file' as const + : entry.type === FileType.DIR + ? 'directory' as const + : 'other' as const + return { + version: entryVersion(entry), + type, + ...(entry.type === FileType.FILE ? { size: entry.size } : {}), + } + } + + override async readText(target: FsTarget, signal?: AbortSignal): Promise { + const sandbox = await this.ctx.e2b.getSandbox() + await this.requireRegular(target, signal) + try { + const bytes = await sandbox.files.read(String(target.targetKey), { format: 'bytes', ...signalOpts(signal) }) + assertNotAborted(signal, 'read') + return decodeText(bytes, target.displayPath, BINARY_SAMPLE_BYTES) + } catch (error: unknown) { + throw mapError(error, 'read', target.displayPath, signal) + } + } + + override async streamText(target: FsTarget, signal?: AbortSignal): Promise> { + const sandbox = await this.ctx.e2b.getSandbox() + await this.requireRegular(target, signal) + let stream: ReadableStream + try { + stream = await sandbox.files.read(String(target.targetKey), { format: 'stream', ...signalOpts(signal) }) + } catch (error: unknown) { + throw mapError(error, 'read', target.displayPath, signal) + } + const displayPath = target.displayPath + return { + async *[Symbol.asyncIterator](): AsyncGenerator { + const reader = stream.getReader() + const decoder = new TextDecoder('utf-8', { fatal: true }) + let sampledBytes = 0 + try { + while (true) { + assertNotAborted(signal, 'read') + const next = await reader.read() + if (next.done) break + if (sampledBytes < BINARY_SAMPLE_BYTES) { + const sample = next.value.subarray(0, BINARY_SAMPLE_BYTES - sampledBytes) + if (sample.includes(0)) throw new FsError(`cannot read "${displayPath}": binary file`, 'FS_NOT_TEXT') + sampledBytes += sample.length + } + let text: string + try { + text = decoder.decode(next.value, { stream: true }) + } catch (error: unknown) { + throw new FsError(`cannot read "${displayPath}": invalid UTF-8 text`, 'FS_NOT_TEXT', { cause: error }) + } + if (text.length > 0) yield text + } + try { + decoder.decode() + } catch (error: unknown) { + throw new FsError(`cannot read "${displayPath}": invalid UTF-8 text`, 'FS_NOT_TEXT', { cause: error }) + } + } catch (error: unknown) { + throw mapError(error, 'read', displayPath, signal) + } finally { + reader.releaseLock() + } + }, + } + } + + override async listDir(target: FsTarget, signal?: AbortSignal): Promise { + const info = await this.stat(target, signal) + if (info === undefined) throw new FsError(`cannot list "${target.displayPath}": not found`, 'FS_NOT_FOUND') + if (info.type !== 'directory') throw new FsError(`cannot list "${target.displayPath}": not a directory`, 'FS_NOT_DIRECTORY') + try { + const sandbox = await this.ctx.e2b.getSandbox() + const listed = await sandbox.files.list(String(target.targetKey), { depth: 1, ...signalOpts(signal) }) + const entries = await Promise.all(listed.map(async (entry): Promise => { + const displayPath = posix.join(target.displayPath, entry.name) + const canonical = await this.canonicalPath(sandbox, entry.path, signal) + const resolved = await this.probe(canonical, displayPath, signal) + return { + name: entry.name, + type: resolved === undefined ? 'other' : entryType(resolved), + target: { targetKey: FsTargetKey(canonical), displayPath }, + ...(resolved !== undefined ? { version: entryVersion(resolved) } : {}), + ...(resolved?.type === FileType.FILE ? { size: resolved.size } : {}), + } + })) + return entries.sort((left, right) => left.name.localeCompare(right.name)) + } catch (error: unknown) { + throw mapError(error, 'list', target.displayPath, signal) + } + } + + override async writeText( + target: FsTarget, + content: string, + expected?: FsWriteIntent, + signal?: AbortSignal, + ): Promise { + return this.withLock(String(target.targetKey), async () => { + const existing = await this.probe(String(target.targetKey), target.displayPath, signal) + if (existing !== undefined && entryType(existing) !== 'file') { + throw new FsError(`cannot write "${target.displayPath}": not a regular file`, 'FS_NOT_REGULAR_FILE') + } + this.checkWriteIntent(existing, expected, target) + const before = existing === undefined ? null : await this.readForDiff(target, signal) + const version = await this.writeAtomic(target, content, existing, signal) + return { + operation: existing === undefined ? 'create' : 'update', + version, + before, + after: normalizeLineEndings(content), + } + }) + } + + override async editText( + target: FsTarget, + edit: FsEditRequest, + expected?: { version: ReturnType }, + signal?: AbortSignal, + ): Promise { + return this.withLock(String(target.targetKey), async () => { + const existing = await this.probe(String(target.targetKey), target.displayPath, signal) + if (existing === undefined) { + throw new FsError(`cannot edit "${target.displayPath}": file changed since it was read`, 'FS_STALE_VERSION') + } + if (entryType(existing) !== 'file') { + throw new FsError(`cannot edit "${target.displayPath}": not a regular file`, 'FS_NOT_REGULAR_FILE') + } + if (expected !== undefined && entryVersion(existing) !== expected.version) { + throw new FsError(`cannot edit "${target.displayPath}": file changed since it was read`, 'FS_STALE_VERSION') + } + const raw = await this.readForEdit(target, signal) + const before = normalizeLineEndings(raw) + const after = literalEdit(before, edit, target.displayPath) + const storage = restoreLineEndings(after, detectsCrlf(raw)) + const version = await this.writeAtomic(target, storage, existing, signal) + return { version, before, after } + }) + } + + private async withLock(targetKey: string, operation: () => Promise): Promise { + const prior = this.locks.get(targetKey) ?? Promise.resolve() + const run = prior.then(operation, operation) + const tail = run.then(() => undefined, () => undefined) + this.locks.set(targetKey, tail) + try { + return await run + } finally { + if (this.locks.get(targetKey) === tail) this.locks.delete(targetKey) + } + } + + private async canonicalPath(sandbox: Sandbox, path: string, signal?: AbortSignal): Promise { + try { + const result = await sandbox.commands.run(`realpath -m -- ${quoteE2BShellArg(path)}`, signalOpts(signal)) + return result.stdout.replace(/\n$/, '') + } catch (error: unknown) { + if (error instanceof CommandExitError) throw new Error(error.stderr || error.message, { cause: error }) + throw error + } + } + + private async probe(path: string, displayPath: string, signal?: AbortSignal): Promise { + assertNotAborted(signal, 'stat') + try { + const sandbox = await this.ctx.e2b.getSandbox() + const entry = await sandbox.files.getInfo(path, signalOpts(signal)) + assertNotAborted(signal, 'stat') + return entry + } catch (error: unknown) { + if (error instanceof FileNotFoundError) return undefined + throw mapError(error, 'stat', displayPath, signal) + } + } + + private async requireRegular(target: FsTarget, signal?: AbortSignal): Promise { + const info = await this.stat(target, signal) + if (info === undefined) throw new FsError(`cannot read "${target.displayPath}": not found`, 'FS_NOT_FOUND') + if (info.type !== 'file') throw new FsError(`cannot read "${target.displayPath}": not a regular file`, 'FS_NOT_REGULAR_FILE') + } + + private checkWriteIntent(existing: EntryInfo | undefined, expected: FsWriteIntent | undefined, target: FsTarget): void { + if (expected?.kind === 'createIfAbsent' && existing !== undefined) { + throw new FsError(`cannot overwrite existing "${target.displayPath}" without reading it first`, 'FS_NOT_OBSERVED') + } + if (expected?.kind === 'replaceIfVersion') { + if (existing === undefined || entryVersion(existing) !== expected.version) { + throw new FsError(`cannot write "${target.displayPath}": file changed since it was read`, 'FS_STALE_VERSION') + } + } + } + + private async readForDiff(target: FsTarget, signal?: AbortSignal): Promise { + try { + const sandbox = await this.ctx.e2b.getSandbox() + const bytes = await sandbox.files.read(String(target.targetKey), { format: 'bytes', ...signalOpts(signal) }) + assertNotAborted(signal, 'read') + return normalizeLineEndings(decodeText(bytes, target.displayPath, bytes.length)) + } catch (error: unknown) { + if (error instanceof FsError && error.code === 'FS_NOT_TEXT') return null + throw mapError(error, 'read', target.displayPath, signal) + } + } + + private async readForEdit(target: FsTarget, signal?: AbortSignal): Promise { + try { + const sandbox = await this.ctx.e2b.getSandbox() + const bytes = await sandbox.files.read(String(target.targetKey), { format: 'bytes', ...signalOpts(signal) }) + assertNotAborted(signal, 'edit') + return decodeText(bytes, target.displayPath, bytes.length) + } catch (error: unknown) { + throw mapError(error, 'edit', target.displayPath, signal) + } + } + + private async writeAtomic( + target: FsTarget, + content: string, + existing: EntryInfo | undefined, + signal?: AbortSignal, + ): Promise> { + assertNotAborted(signal, 'write') + const sandbox = await this.ctx.e2b.getSandbox() + const targetPath = String(target.targetKey) + const versionId = randomUUID() + const temporary = posix.join(posix.dirname(targetPath), `.${posix.basename(targetPath)}.dsh-${randomUUID()}.tmp`) + try { + await sandbox.files.write(temporary, content, { + metadata: { [VERSION_METADATA_KEY]: versionId }, + ...signalOpts(signal), + }) + assertNotAborted(signal, 'write') + const mode = existing === undefined ? 0o600 : existing.mode & 0o777 + await sandbox.commands.run( + `chmod ${mode.toString(8)} -- ${quoteE2BShellArg(temporary)}`, + signalOpts(signal), + ) + assertNotAborted(signal, 'write') + await sandbox.commands.run( + `mv -f -- ${quoteE2BShellArg(temporary)} ${quoteE2BShellArg(targetPath)}`, + signalOpts(signal), + ) + const committed = await sandbox.files.getInfo(targetPath) + return entryVersion(committed) + } catch (error: unknown) { + try { + await sandbox.files.remove(temporary) + } catch (_temporaryAlreadyAbsent) { + // Only the private staging path is swallowed; the original failure owns the operation. + } + throw mapError(error, 'write', target.displayPath, signal) + } + } +} + +export default E2BFileSystem diff --git a/packages/fs/fs-e2b/src/invariant.ts b/packages/fs/fs-e2b/src/invariant.ts new file mode 100644 index 0000000000..6294d66ace --- /dev/null +++ b/packages/fs/fs-e2b/src/invariant.ts @@ -0,0 +1,27 @@ +/** Package-owned invariant companion for `@deepseek-ai/dsh-fs-e2b`. */ + +/* jscpd:ignore-start */ +import type { Context } from 'cordis' +import type { InvariantInstaller } from '@deepseek-ai/dsh-invariants' + +const PACKAGE_NAME = '@deepseek-ai/dsh-fs-e2b' + +/** Cordis companion plugin name. */ +export const name = 'fs-e2b-invariant' +/** Service required before reserving package ownership. */ +export const inject = ['invariants'] + +/** + * No runtime invariant: each operation returns the E2B controller's committed + * result directly, with no independent event or cache to cross-check. + */ +const install: InvariantInstaller = () => {} + +/** + * Register this package's invariant companion. + * @param ctx - Cordis context carrying the invariant service. + * @returns the installed registration's disposer after setup succeeds. + */ +export const apply = (ctx: Context): Promise<() => void> => + Promise.resolve(ctx.invariants.register(PACKAGE_NAME, install)) +/* jscpd:ignore-end */ diff --git a/packages/fs/fs-e2b/tests/filesystem.spec.ts b/packages/fs/fs-e2b/tests/filesystem.spec.ts new file mode 100644 index 0000000000..b8eb5447d6 --- /dev/null +++ b/packages/fs/fs-e2b/tests/filesystem.spec.ts @@ -0,0 +1,537 @@ +import { dirname, posix } from 'node:path' +import { Context } from 'cordis' +import { + CommandExitError, + FileNotFoundError, + FileType, + type EntryInfo, + type Sandbox, +} from '@deepseek-ai/dsh-e2b' +import type E2BSandboxService from '@deepseek-ai/dsh-e2b' +import { FsVersion } from '@deepseek-ai/dsh-fs' +import E2BFileSystem from '@deepseek-ai/dsh-fs-e2b' +import * as E2BFsInvariant from '../src/invariant.ts' +import InvariantService from '@deepseek-ai/dsh-invariants' +import { describe, expect, it } from 'vitest' + +interface RemoteNode { + type: FileType + data: Uint8Array + mode: number + modified: number + metadata?: Record + symlinkTarget?: string +} + +function bytes(value: string | readonly number[]): Uint8Array { + return typeof value === 'string' ? new TextEncoder().encode(value) : Uint8Array.from(value) +} + +function commandError(exitCode: number, stderr = ''): CommandExitError { + return new CommandExitError({ exitCode, stdout: '', stderr, error: stderr }) +} + +class FakeRemote { + readonly nodes = new Map() + readonly writes: Array<{ path: string; data: string; metadata?: Record }> = [] + readonly renames: Array<{ from: string; to: string }> = [] + readonly removals: string[] = [] + readonly commands: string[] = [] + streamChunks: Uint8Array[] | undefined + nextCommandError: unknown + nextInfoError: unknown + nextListError: unknown + nextReadError: unknown + nextRenameError: unknown + nextRemoveError: unknown + abortAfterRename: AbortController | undefined + disappearOnInfo = new Set() + private clock = 1 + + constructor() { + this.dir('/') + this.dir('/workspace') + } + + dir(path: string): void { + this.nodes.set(path, { type: FileType.DIR, data: bytes(''), mode: 0o755, modified: this.clock++ }) + } + + file(path: string, data: string | readonly number[], mode = 0o644): void { + this.nodes.set(path, { type: FileType.FILE, data: bytes(data), mode, modified: this.clock++ }) + } + + other(path: string): void { + this.nodes.set(path, { type: 'other' as FileType, data: bytes(''), mode: 0o600, modified: this.clock++ }) + } + + symlink(path: string, target: string): void { + this.nodes.set(path, { + type: FileType.FILE, + data: bytes(''), + mode: 0o777, + modified: this.clock++, + symlinkTarget: target, + }) + } + + mutate(path: string, data: string): void { + const node = this.required(path) + node.data = bytes(data) + node.modified = this.clock++ + } + + private required(path: string): RemoteNode { + const node = this.nodes.get(path) + if (node === undefined) throw new FileNotFoundError(`missing: ${path}`) + return node + } + + private followed(path: string): { path: string; node: RemoteNode; link?: RemoteNode } { + const node = this.required(path) + if (node.symlinkTarget === undefined) return { path, node } + return { path: node.symlinkTarget, node: this.required(node.symlinkTarget), link: node } + } + + private info(path: string): EntryInfo { + if (this.disappearOnInfo.delete(path)) throw new FileNotFoundError(`missing: ${path}`) + return this.rawInfo(path) + } + + private rawInfo(path: string): EntryInfo { + const followed = this.followed(path) + const node = followed.node + return { + name: posix.basename(path), + path, + type: node.type, + size: node.data.byteLength, + mode: node.mode, + permissions: 'rw-------', + owner: 'user', + group: 'user', + modifiedTime: new Date(node.modified), + ...(node.metadata !== undefined ? { metadata: { ...node.metadata } } : {}), + ...(followed.link?.symlinkTarget !== undefined ? { symlinkTarget: followed.link.symlinkTarget } : {}), + } + } + + private checkAbort(options: { signal?: AbortSignal } | undefined): void { + if (options?.signal?.aborted === true) throw new DOMException('aborted', 'AbortError') + } + + readonly sandbox = { + sandboxId: 'fake', + files: { + makeDir: async (path: string): Promise => { + if (this.nodes.has(path)) return false + this.dir(path) + return true + }, + getInfo: async (path: string, options?: { signal?: AbortSignal }): Promise => { + this.checkAbort(options) + if (this.nextInfoError !== undefined) { + const error = this.nextInfoError + this.nextInfoError = undefined + throw error + } + return this.info(path) + }, + read: async (path: string, options: { format: 'bytes' | 'stream'; signal?: AbortSignal }): Promise> => { + this.checkAbort(options) + if (this.nextReadError !== undefined) { + const error = this.nextReadError + this.nextReadError = undefined + throw error + } + const data = this.followed(path).node.data + if (options.format === 'bytes') return data.slice() + const chunks = this.streamChunks ?? [data.slice()] + return new ReadableStream({ + start(controller) { + for (const chunk of chunks) controller.enqueue(chunk) + controller.close() + }, + }) + }, + list: async (path: string, options?: { depth?: number; signal?: AbortSignal }): Promise => { + this.checkAbort(options) + if (this.nextListError !== undefined) { + const error = this.nextListError + this.nextListError = undefined + throw error + } + this.required(path) + return [...this.nodes.keys()] + .filter(candidate => candidate !== path && dirname(candidate) === path) + .map(candidate => this.rawInfo(candidate)) + }, + write: async (path: string, data: string, options?: { metadata?: Record; signal?: AbortSignal }): Promise => { + this.checkAbort(options) + const parent = dirname(path) + if (!this.nodes.has(parent)) this.dir(parent) + this.nodes.set(path, { + type: FileType.FILE, + data: bytes(data), + mode: 0o644, + modified: this.clock++, + ...(options?.metadata !== undefined ? { metadata: { ...options.metadata } } : {}), + }) + this.writes.push({ path, data, ...(options?.metadata !== undefined ? { metadata: options.metadata } : {}) }) + return {} + }, + rename: async (from: string, to: string, options?: { signal?: AbortSignal }): Promise => { + this.checkAbort(options) + if (this.nextRenameError !== undefined) { + const error = this.nextRenameError + this.nextRenameError = undefined + throw error + } + const node = this.required(from) + this.nodes.delete(from) + this.nodes.set(to, node) + this.renames.push({ from, to }) + this.abortAfterRename?.abort('after commit') + return this.info(to) + }, + remove: async (path: string): Promise => { + this.removals.push(path) + if (this.nextRemoveError !== undefined) { + const error = this.nextRemoveError + this.nextRemoveError = undefined + throw error + } + this.nodes.delete(path) + }, + }, + commands: { + run: async (command: string, options?: { signal?: AbortSignal }): Promise<{ exitCode: number; stdout: string; stderr: string }> => { + this.checkAbort(options) + this.commands.push(command) + if (this.nextCommandError !== undefined) { + const error = this.nextCommandError + this.nextCommandError = undefined + throw error + } + if (command.startsWith('realpath -m -- ')) { + const input = command.slice('realpath -m -- '.length).slice(1, -1) + const node = this.nodes.get(input) + return { exitCode: 0, stdout: `${node?.symlinkTarget ?? input}\n`, stderr: '' } + } + const chmod = /^chmod ([0-7]+) -- '([^']+)'$/.exec(command) + if (chmod !== null) this.required(chmod[2]!).mode = Number.parseInt(chmod[1]!, 8) + const move = /^mv -f -- '([^']+)' '([^']+)'$/.exec(command) + if (move !== null) { + if (this.nextRenameError !== undefined) { + const error = this.nextRenameError + this.nextRenameError = undefined + throw error + } + const node = this.required(move[1]!) + this.nodes.delete(move[1]!) + this.nodes.set(move[2]!, node) + this.renames.push({ from: move[1]!, to: move[2]! }) + this.abortAfterRename?.abort('after commit') + } + return { exitCode: 0, stdout: '', stderr: '' } + }, + }, + } as unknown as Sandbox +} + +async function setup(remote = new FakeRemote()): Promise<{ ctx: Context; fs: E2BFileSystem; remote: FakeRemote }> { + const ctx = new Context() + const runtime = { + cwd: '/workspace', + runtimeRoot: '/workspace/.dsh-e2b', + disposeMode: 'kill', + getSandbox: async () => remote.sandbox, + } as unknown as E2BSandboxService + ctx.provide('e2b', runtime) + await ctx.plugin(E2BFileSystem) + return { ctx, fs: ctx.fs as E2BFileSystem, remote } +} + +async function expectCode(promise: Promise, code: string): Promise { + await expect(promise).rejects.toMatchObject({ code }) +} + +describe('E2BFileSystem identity, metadata, and reads', () => { + it('resolves remote paths, reports symlinks, and lists direct children in stable order', async () => { + const remote = new FakeRemote() + remote.file('/workspace/z.txt', 'z') + remote.file('/workspace/a.txt', 'a') + remote.dir('/workspace/dir') + remote.other('/workspace/special') + remote.file('/workspace/dir/nested.txt', 'nested') + remote.symlink('/workspace/link.txt', '/workspace/a.txt') + const { fs } = await setup(remote) + + const link = await fs.resolve('link.txt') + expect(link).toEqual({ targetKey: '/workspace/a.txt', displayPath: '/workspace/link.txt' }) + await expect(fs.lstat('link.txt')).resolves.toMatchObject({ type: 'symlink', size: 1 }) + await expect(fs.lstat('a.txt')).resolves.toMatchObject({ type: 'file', size: 1 }) + await expect(fs.lstat('dir')).resolves.toEqual(expect.objectContaining({ type: 'directory' })) + await expect(fs.lstat('special')).resolves.toEqual(expect.objectContaining({ type: 'other' })) + await expect(fs.lstat('missing')).resolves.toBeUndefined() + await expect(fs.stat(link)).resolves.toMatchObject({ type: 'file', size: 1 }) + const directory = await fs.resolve('.') + const listed = await fs.listDir(directory) + expect(listed.map(entry => entry.name)).toEqual(['a.txt', 'dir', 'link.txt', 'special', 'z.txt']) + expect(listed.find(entry => entry.name === 'dir')).toMatchObject({ type: 'directory' }) + expect(listed.find(entry => entry.name === 'link.txt')).toMatchObject({ + type: 'file', + target: { targetKey: '/workspace/a.txt', displayPath: '/workspace/link.txt' }, + }) + expect(listed.some(entry => entry.name === 'nested.txt')).toBe(false) + }) + + it('reads whole and streamed UTF-8 across chunk boundaries', async () => { + const remote = new FakeRemote() + remote.file('/workspace/text.txt', 'A€B') + remote.streamChunks = [bytes([65, 0xe2]), bytes([0x82, 0xac, 66])] + const { fs } = await setup(remote) + const target = await fs.resolve('text.txt') + await expect(fs.readText(target)).resolves.toBe('A€B') + let streamed = '' + for await (const chunk of await fs.streamText(target)) streamed += chunk + expect(streamed).toBe('A€B') + + remote.streamChunks = [bytes([0xe2]), bytes([0x82, 0xac])] + let initiallyBuffered = '' + for await (const chunk of await fs.streamText(target)) initiallyBuffered += chunk + expect(initiallyBuffered).toBe('€') + }) + + it('matches local binary sampling while edits still reject any NUL byte', async () => { + const remote = new FakeRemote() + remote.file('/workspace/late-nul.txt', `${'a'.repeat(8192)}\0tail`) + const { fs } = await setup(remote) + const target = await fs.resolve('late-nul.txt') + await expect(fs.readText(target)).resolves.toContain('\0tail') + remote.streamChunks = [bytes('a'.repeat(8192)), bytes([0, 116])] + let streamed = '' + for await (const chunk of await fs.streamText(target)) streamed += chunk + expect(streamed).toBe(`${'a'.repeat(8192)}\0t`) + await expectCode(fs.editText(target, { oldString: 'tail', newString: 'end', replaceAll: false }), 'FS_NOT_TEXT') + }) + + it('maps binary, invalid UTF-8, missing, and non-regular read failures', async () => { + const remote = new FakeRemote() + remote.file('/workspace/binary', [0, 1]) + remote.file('/workspace/invalid', [0xff]) + remote.dir('/workspace/directory') + const { fs } = await setup(remote) + await expectCode(fs.readText(await fs.resolve('binary')), 'FS_NOT_TEXT') + await expectCode(fs.readText(await fs.resolve('invalid')), 'FS_NOT_TEXT') + await expectCode(fs.readText(await fs.resolve('missing')), 'FS_NOT_FOUND') + await expectCode(fs.readText(await fs.resolve('directory')), 'FS_NOT_REGULAR_FILE') + + remote.streamChunks = [bytes([0xff])] + const invalid = await fs.streamText(await fs.resolve('invalid')) + await expect((async () => { for await (const _chunk of invalid) void _chunk })()).rejects.toMatchObject({ code: 'FS_NOT_TEXT' }) + remote.streamChunks = [bytes([0])] + const binary = await fs.streamText(await fs.resolve('binary')) + await expect((async () => { for await (const _chunk of binary) void _chunk })()).rejects.toMatchObject({ code: 'FS_NOT_TEXT' }) + + remote.streamChunks = [bytes([0xe2])] + const incomplete = await fs.streamText(await fs.resolve('invalid')) + await expect((async () => { for await (const _chunk of incomplete) void _chunk })()).rejects.toMatchObject({ code: 'FS_NOT_TEXT' }) + + const raced = await fs.resolve('invalid') + remote.nextReadError = new FileNotFoundError('gone after stat') + await expectCode(fs.streamText(raced), 'FS_NOT_FOUND') + }) + + it('honors aborts before and during remote reads', async () => { + const remote = new FakeRemote() + remote.file('/workspace/a', 'a') + const { fs } = await setup(remote) + await expectCode(fs.resolve('a', { signal: AbortSignal.abort() }), 'FS_ABORTED') + await expectCode(fs.lstat('a', undefined, AbortSignal.abort()), 'FS_ABORTED') + await expectCode(fs.stat(await fs.resolve('a'), AbortSignal.abort()), 'FS_ABORTED') + remote.nextReadError = new DOMException('aborted', 'AbortError') + await expectCode(fs.readText(await fs.resolve('a')), 'FS_ABORTED') + }) + + it('rejects empty paths and directory-listing type errors', async () => { + const remote = new FakeRemote() + remote.file('/workspace/file', 'x') + const { fs } = await setup(remote) + await expectCode(fs.resolve(' '), 'FS_NOT_FOUND') + await expectCode(fs.lstat(''), 'FS_NOT_FOUND') + await expectCode(fs.listDir(await fs.resolve('missing')), 'FS_NOT_FOUND') + await expectCode(fs.listDir(await fs.resolve('/workspace/file')), 'FS_NOT_DIRECTORY') + remote.nextListError = new Error('listing transport failed') + await expectCode(fs.listDir(await fs.resolve('/workspace')), 'FS_IO_ERROR') + }) +}) + +describe('E2BFileSystem atomic writes and edits', () => { + it('creates owner-only files and returns metadata after the committed move', async () => { + const { fs, remote } = await setup() + const target = await fs.resolve('new.txt') + const outcome = await fs.writeText(target, 'one\r\ntwo\rthree', { kind: 'createIfAbsent' }) + expect(outcome).toMatchObject({ operation: 'create', before: null, after: 'one\ntwo\rthree' }) + expect(remote.nodes.get('/workspace/new.txt')?.mode).toBe(0o600) + expect(remote.nodes.get('/workspace/new.txt')?.metadata?.['dsh-version']).toBeDefined() + await expect(fs.stat(target)).resolves.toMatchObject({ version: outcome.version, size: 14 }) + }) + + it('preserves replacement mode, normalizes only CRLF for diffs, and changes version on external writes', async () => { + const remote = new FakeRemote() + remote.file('/workspace/file.txt', 'old\r\nline\rlone', 0o640) + const { fs } = await setup(remote) + const target = await fs.resolve('file.txt') + const before = (await fs.stat(target))!.version + const outcome = await fs.writeText(target, 'new', { kind: 'replaceIfVersion', version: before }) + expect(outcome).toMatchObject({ operation: 'update', before: 'old\nline\rlone', after: 'new' }) + expect(remote.nodes.get('/workspace/file.txt')?.mode).toBe(0o640) + const committed = outcome.version + remote.mutate('/workspace/file.txt', 'external') + expect((await fs.stat(target))!.version).not.toBe(committed) + }) + + it('returns null as the overwrite diff basis for binary or invalid prior content', async () => { + const remote = new FakeRemote() + remote.file('/workspace/file.txt', [0xff]) + const { fs } = await setup(remote) + const target = await fs.resolve('file.txt') + await expect(fs.writeText(target, 'valid')).resolves.toMatchObject({ before: null, after: 'valid' }) + }) + + it('fails an overwrite when reading its text diff basis fails for another reason', async () => { + const remote = new FakeRemote() + remote.file('/workspace/file.txt', 'prior') + const { fs } = await setup(remote) + const target = await fs.resolve('file.txt') + remote.nextReadError = new Error('read transport failed') + await expectCode(fs.writeText(target, 'replacement'), 'FS_IO_ERROR') + expect(new TextDecoder().decode(remote.nodes.get('/workspace/file.txt')?.data)).toBe('prior') + }) + + it('enforces create and version intents before publication', async () => { + const remote = new FakeRemote() + remote.file('/workspace/file.txt', 'v1') + const { fs } = await setup(remote) + const target = await fs.resolve('file.txt') + const version = (await fs.stat(target))!.version + await expectCode(fs.writeText(target, 'blind', { kind: 'createIfAbsent' }), 'FS_NOT_OBSERVED') + remote.mutate('/workspace/file.txt', 'v2') + await expectCode(fs.writeText(target, 'stale', { kind: 'replaceIfVersion', version }), 'FS_STALE_VERSION') + await expectCode(fs.writeText(await fs.resolve('missing'), 'stale', { kind: 'replaceIfVersion', version }), 'FS_STALE_VERSION') + remote.dir('/workspace/dir') + await expectCode(fs.writeText(await fs.resolve('dir'), 'x'), 'FS_NOT_REGULAR_FILE') + }) + + it('does not turn an abort observed after a successful move into a failed write', async () => { + const remote = new FakeRemote() + const controller = new AbortController() + remote.abortAfterRename = controller + const { fs } = await setup(remote) + await expect(fs.writeText(await fs.resolve('committed'), 'yes', undefined, controller.signal)) + .resolves.toMatchObject({ operation: 'create' }) + expect(controller.signal.aborted).toBe(true) + }) + + it('cleans staging files and maps command, permission, and abort failures', async () => { + const remote = new FakeRemote() + const { fs } = await setup(remote) + const commandTarget = await fs.resolve('command') + remote.nextCommandError = commandError(1, 'chmod failed') + await expectCode(fs.writeText(commandTarget, 'x'), 'FS_IO_ERROR') + expect(remote.removals).toHaveLength(1) + + remote.nextRenameError = new Error('permission denied') + await expectCode(fs.writeText(await fs.resolve('permission'), 'x'), 'FS_PERMISSION_DENIED') + remote.nextRemoveError = new Error('cleanup also failed') + remote.nextRenameError = new DOMException('aborted', 'AbortError') + await expectCode(fs.writeText(await fs.resolve('abort'), 'x'), 'FS_ABORTED') + }) + + it('applies literal edits atomically and restores the detected CRLF style', async () => { + const remote = new FakeRemote() + remote.file('/workspace/file.txt', 'one\r\ntwo\r\nthree\n') + const { fs } = await setup(remote) + const target = await fs.resolve('file.txt') + const version = (await fs.stat(target))!.version + const outcome = await fs.editText( + target, + { oldString: 'two\r\n', newString: 'TWO\r\n', replaceAll: false }, + { version }, + ) + expect(outcome).toMatchObject({ before: 'one\ntwo\nthree\n', after: 'one\nTWO\nthree\n' }) + expect(new TextDecoder().decode(remote.nodes.get('/workspace/file.txt')?.data)).toBe('one\r\nTWO\r\nthree\r\n') + }) + + it('reports stale and literal-match failures with stable codes', async () => { + const remote = new FakeRemote() + remote.file('/workspace/file.txt', 'a a') + remote.dir('/workspace/dir') + const { fs } = await setup(remote) + const target = await fs.resolve('file.txt') + await expectCode(fs.editText(target, { oldString: '', newString: 'x', replaceAll: false }), 'FS_EDIT_NOT_FOUND') + await expectCode(fs.editText(target, { oldString: 'z', newString: 'x', replaceAll: false }), 'FS_EDIT_NOT_FOUND') + await expectCode(fs.editText(target, { oldString: 'a', newString: 'x', replaceAll: false }), 'FS_AMBIGUOUS_EDIT') + await expect(fs.editText(target, { oldString: 'a', newString: 'x', replaceAll: true })) + .resolves.toMatchObject({ after: 'x x' }) + await expectCode(fs.editText(target, { oldString: 'x', newString: 'y', replaceAll: false }, { version: FsVersion('stale') }), 'FS_STALE_VERSION') + await expectCode(fs.editText(await fs.resolve('missing'), { oldString: 'x', newString: 'y', replaceAll: false }), 'FS_STALE_VERSION') + await expectCode(fs.editText(await fs.resolve('dir'), { oldString: 'x', newString: 'y', replaceAll: false }), 'FS_NOT_REGULAR_FILE') + }) + + it('serializes guarded mutations so only one stale version can win', async () => { + const remote = new FakeRemote() + remote.file('/workspace/file.txt', 'base') + const { fs } = await setup(remote) + const target = await fs.resolve('file.txt') + const version = (await fs.stat(target))!.version + const results = await Promise.allSettled([ + fs.writeText(target, 'one', { kind: 'replaceIfVersion', version }), + fs.editText(target, { oldString: 'base', newString: 'two', replaceAll: false }, { version }), + ]) + expect(results.filter(result => result.status === 'fulfilled')).toHaveLength(1) + expect(results.filter(result => result.status === 'rejected')).toHaveLength(1) + }) +}) + +describe('E2B filesystem adapter integration edges', () => { + it('maps canonicalization, permission, and generic provider failures', async () => { + const remote = new FakeRemote() + const { fs } = await setup(remote) + remote.nextCommandError = commandError(1, 'not a directory') + await expectCode(fs.resolve('bad'), 'FS_IO_ERROR') + remote.nextCommandError = commandError(1) + await expectCode(fs.resolve('bad-again'), 'FS_IO_ERROR') + remote.nextCommandError = new Error('canonical transport failed') + await expectCode(fs.resolve('bad-transport'), 'FS_IO_ERROR') + remote.file('/workspace/a', 'a') + const target = await fs.resolve('a') + remote.nextInfoError = new Error('metadata transport failed') + await expectCode(fs.stat(target), 'FS_IO_ERROR') + remote.nextReadError = new Error('operation not permitted') + await expectCode(fs.readText(target), 'FS_PERMISSION_DENIED') + remote.nextReadError = 'transport vanished' + await expectCode(fs.readText(target), 'FS_IO_ERROR') + }) + + it('keeps a listed child whose metadata disappears as an other entry', async () => { + const remote = new FakeRemote() + remote.file('/workspace/a', 'a') + remote.disappearOnInfo.add('/workspace/a') + const { fs } = await setup(remote) + const listed = await fs.listDir(await fs.resolve('/workspace')) + expect(listed).toEqual([{ + name: 'a', + type: 'other', + target: { targetKey: '/workspace/a', displayPath: '/workspace/a' }, + }]) + }) + + it('registers the package-owned empty invariant installer', async () => { + const ctx = new Context() + await ctx.plugin(InvariantService, { enabled: true }) + const fiber = await ctx.plugin(E2BFsInvariant).await() + await fiber.dispose() + }) +}) diff --git a/packages/fs/fs-e2b/tsconfig.json b/packages/fs/fs-e2b/tsconfig.json new file mode 100644 index 0000000000..c424efd2e5 --- /dev/null +++ b/packages/fs/fs-e2b/tsconfig.json @@ -0,0 +1,25 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": ["src"], + "references": [ + { + "path": "../../../vendor/cosmokit" + }, + { + "path": "../../../vendor/cordis" + }, + { + "path": "../../e2b/e2b" + }, + { + "path": "../fs" + }, + { + "path": "../../support/invariants" + } + ] +} diff --git a/packages/subprocess/README.i18n.yaml b/packages/subprocess/README.i18n.yaml index 0ebb5bd4af..0efaf7838e 100644 --- a/packages/subprocess/README.i18n.yaml +++ b/packages/subprocess/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/subprocess/README.md -README.md: f2b19436da40feb14d067e2cfc706222625680b5 -README.zh.md: 938312448dd5c0a691ed07ddc9843cf2c4445637 +README.md: ae18c55205edd6085a0ed8de1bb7f875c411c79f +README.zh.md: e27fd240e4c1e96d8b859b326574e441241e8508 diff --git a/packages/subprocess/README.md b/packages/subprocess/README.md index f2b19436da..ae18c55205 100644 --- a/packages/subprocess/README.md +++ b/packages/subprocess/README.md @@ -2,11 +2,12 @@ English | [中文](README.zh.md) -The shared process substrate for one execution world: executable lookup, fully-specified managed child-process trees with raw or collected stdio, and one deep terminal-process primitive that owns PTY allocation, foreground groups, and provider-observable session cleanup. Command defaulting, shell semantics, deadlines, protocol framing, readiness, and presentation stay with consumers — the [bash executors](../bash/README.md), [LSP host](../lsp/README.md), [PTY shell backend](../pty/README.md), and [ACP subagent backend](../subagent/README.md). See the [subprocess seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md). +The shared home for spawning managed child-process trees: fully-specified spawn specs with Node-shaped per-stream stdio dispositions (raw pipes, inherit, bounded tail-keep collection with spill files), the one credential scrub every harness spawner uses, offset-based incremental reads, tree-scoped signalling with SIGTERM→grace→SIGKILL escalation, and the cooperative dispose ladder. Command defaulting, shell semantics, deadlines, protocol framing, and presentation stay with consumers — the [bash executors](../bash/README.md), the [LSP host](../lsp/README.md), and the [ACP subagent backend](../subagent/README.md). See the [subprocess seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md). | Package | ctx key | Role | |---|---|---| -| [`subprocess`](subprocess/README.md) (`@deepseek-ai/dsh-subprocess`) | `ctx.subprocess` | The seam: executable lookup, ordinary managed spawns, the terminal-process primitive, handle lifecycles, and shared environment/output vocabulary | -| [`subprocess-local`](subprocess-local/README.md) (`@deepseek-ai/dsh-subprocess-local`) | — | The local implementation: detached process trees, bounded collection/spill, `node-pty`, foreground/session inspection, tree signalling, and terminate-and-join disposal | +| [`subprocess`](subprocess/README.md) (`@deepseek-ai/dsh-subprocess`) | `ctx.subprocess` | The seam: abstract `SubprocessService.spawn(spec)`, the fully-explicit `SubprocessSpawnSpec` with per-stream stdio dispositions, `SubprocessHandle` (streams, offset-based readers, terminate/waitForExit/dispose), and the shared scrub + `DSH_*`/`CollectedOutput` vocabulary | +| [`subprocess-local`](subprocess-local/README.md) (`@deepseek-ai/dsh-subprocess-local`) | — | The local implementation: detached process trees, per-disposition stream wiring, tail-keep truncation with bounded private spill files, the `DSH_*` merge order, tree signalling with escalation, the dispose ladder, and terminate-and-join disposal | +| [`subprocess-e2b`](subprocess-e2b/README.md) (`@deepseek-ai/dsh-subprocess-e2b`) | — | Experimental E2B implementation: remote Linux process groups and spill state in the shared `ctx.e2b` sandbox, with asynchronous PID acquisition and SDK buffering limitations | The service owns process lifetime across consumer reloads; consumers own what a process means (a bash command, a future non-shell runner) and every default that shapes one. diff --git a/packages/subprocess/README.zh.md b/packages/subprocess/README.zh.md index 938312448d..e27fd240e4 100644 --- a/packages/subprocess/README.zh.md +++ b/packages/subprocess/README.zh.md @@ -1,12 +1,13 @@ -# subprocess/：子进程能力家族 +# subprocess/：进程管理能力家族 [English](README.md) | 中文 -这里集中提供一个执行世界的共享进程基底：可执行文件查找、具有原始或收集式 stdio 的完整指定受管子进程树，以及一项深层终端进程原语，负责 PTY 分配、前台进程组和提供方仍可观察到的会话成员清理。命令默认值补全、shell 语义、时限、协议分帧、就绪状态与呈现留在消费方：[bash 执行器](../bash/README.md)、[LSP 主机](../lsp/README.md)、[PTY shell 后端](../pty/README.md)与 [ACP（Agent Client Protocol）subagent 后端](../subagent/README.md)。参见[subprocess seam Agent Note（agent 决策记录）](../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md)。 +spawn 受管子进程树的共用归属位置：完全显式的 spawn spec，其 stdio 处置方式（disposition）为 Node 形状、按流划分（原始管道、inherit、附带 spill 文件的有界尾部保留收集）；harness 中所有 spawn 调用方共用的那一份凭据清除；基于偏移量的增量读取；以进程树为范围、带 SIGTERM→宽限期→SIGKILL 升级的信号发送；以及协作式 dispose（资源释放）阶梯。命令默认值补全、shell 语义、deadline、协议分帧与呈现留在消费方：[bash 执行器](../bash/README.md)、[LSP 主机](../lsp/README.md)与 [ACP（Agent Client Protocol）subagent 后端](../subagent/README.md)。参见[进程管理器 seam Agent Note（agent 决策记录）](../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md)。 | 包（package） | ctx 键 | 角色 | |---|---|---| -| [`subprocess`](subprocess/README.md)（`@deepseek-ai/dsh-subprocess`） | `ctx.subprocess` | seam 本体：可执行文件查找、普通受管 spawn、终端进程原语、句柄生命周期，以及共享的环境／输出词汇 | -| [`subprocess-local`](subprocess-local/README.md)（`@deepseek-ai/dsh-subprocess-local`） | 无 | 本地实现：detached 进程树、有界收集／spill、`node-pty`、前台／会话检查、进程树信号发送，以及先终止再等待退出的资源释放 | +| [`subprocess`](subprocess/README.md)（`@deepseek-ai/dsh-subprocess`） | `ctx.subprocess` | seam 本体：抽象的 `SubprocessService.spawn(spec)`、完全显式且带按流划分 stdio 处置方式的 `SubprocessSpawnSpec`、`SubprocessHandle`（流、基于偏移量的读取器、terminate/waitForExit/dispose），以及共享的凭据清除 + `DSH_*`/`CollectedOutput` 词汇 | +| [`subprocess-local`](subprocess-local/README.md)（`@deepseek-ai/dsh-subprocess-local`） | 无 | 本地实现：detached 进程树、按处置方式接线的流、附带有界私有 spill 文件的尾部保留截断、`DSH_*` 合并次序、带升级的进程树信号发送、dispose 阶梯，以及先终止再等待退出的 dispose | +| [`subprocess-e2b`](subprocess-e2b/README.md)（`@deepseek-ai/dsh-subprocess-e2b`） | 无 | 实验性 E2B 实现：远程 Linux 进程组和共享 `ctx.e2b` 沙箱中的 spill 状态，但 PID 异步获取，且受 SDK 缓冲限制 | -即使消费方重载，进程生命周期仍由服务负责管理；消费方负责定义进程的含义（一条 bash 命令、未来的非 shell 运行器），以及决定塑造该进程的每一项默认值。 +服务拥有跨消费方重载的进程存续期；消费方拥有一个进程的含义（一条 bash 命令、未来的非 shell 运行器）以及塑造它的每一项默认值。 diff --git a/packages/subprocess/subprocess-e2b/README.i18n.yaml b/packages/subprocess/subprocess-e2b/README.i18n.yaml new file mode 100644 index 0000000000..dfbe97e8b9 --- /dev/null +++ b/packages/subprocess/subprocess-e2b/README.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write packages/subprocess/subprocess-e2b/README.md +README.md: 9ec1518103413f33a52023c17722bf5a5fe275ce +README.zh.md: f50ff2f3d85a5ddd9bd352e95d0eb0a23e756404 diff --git a/packages/subprocess/subprocess-e2b/README.md b/packages/subprocess/subprocess-e2b/README.md new file mode 100644 index 0000000000..9ec1518103 --- /dev/null +++ b/packages/subprocess/subprocess-e2b/README.md @@ -0,0 +1,32 @@ +# @deepseek-ai/dsh-subprocess-e2b + +English | [中文](README.zh.md) + +E2B implementation of the [`@deepseek-ai/dsh-subprocess`](../subprocess/README.md) seam. It has no config: load [`@deepseek-ai/dsh-e2b`](../../e2b/e2b/README.md) first, then this service in place of `dsh-subprocess-local`. Existing consumers such as [`dsh-bash-local`](../../bash/bash-local/README.md) then execute in the shared remote sandbox without an E2B-specific Bash adapter. + +## Behavior + +- **Asynchronous remote start** — the synchronous seam returns a handle immediately while `Sandbox.commands.run(..., { background: true })` starts remotely. `pid` is `-1` until the SDK returns the command PID; `done`, stdin, termination, and `waitForExit()` wait for readiness internally. +- **Linux process groups** — a quoted wrapper starts each argv under `setsid --wait` and records its actual process-group id plus private status files beneath `ctx.e2b.runtimeRoot/processes`. The handle waits for that file instead of assuming the SDK command PID is the group id. Termination signals the negative recorded id with `SIGTERM`, waits the caller's `graceMs`, then escalates to `SIGKILL` and the SDK kill fallback. Service disposal terminates and joins every retained handle before the sandbox owner disposes. +- **Environment boundary** — the wrapper starts from the sandbox command environment, removes ambient `DSH_*` and credential-shaped (`*KEY*`, `*SECRET*`, `*TOKEN*`) names, then restores every `spec.env` entry as an explicit caller opt-in. Host ambient variables never enter the sandbox implicitly. +- **Stdio projection** — pipe mode forwards E2B callbacks into host Node streams; inherit mode forwards them to the harness process streams; collect mode retains a bounded host tail with offset reads. Optional complete spill files are written remotely and advertised only while within their cap. Batch and streaming stdin use the SDK handle. + +The base E2B image supplies the Bash/GNU utilities this adapter invokes: `bash`, `setsid`, `ps`, `tr`, `env`, `chmod`, `tee`, and `kill`. A custom template must retain compatible commands. + +## Model Experience + +Indirectly, through consumer seams such as the Bash executor behind `dsh-tool-bash`, which render remote output, exit facts, background deltas, and spill paths. + +#### KV Cache effect + +No direct invalidation; the named consumers own any request-prefix changes. + +## Known Limitations and Deferred Work + +- **The SDK still retains complete command output in host memory** — E2B `CommandHandle.stdout` and `.stderr` accumulate even when this adapter exposes bounded tails, so the subprocess seam's normal host-memory bound is not achieved. +- **Pipe output is not byte-faithful** — E2B delivers separately decoded strings rather than raw bytes, so split multibyte sequences and arbitrary binary protocols can be corrupted; LSP and other framed byte-stream consumers are unsupported. +- **Synchronous-PID consumers are unsupported** — `pid` remains `-1` during remote startup; consumers that require a positive PID immediately, including the ACP child backend, cannot use this provider unchanged. +- **Reconnect does not reconstruct handles** — remote PID/status/spill files survive a retained sandbox, but a new harness process does not rebuild live `SubprocessHandle` objects or output cursors from them. +- **Remote state accumulates when retained** — process directories and valid spill files remain under `.dsh-e2b`; this POC supplies no retention sweep. +- **Signal attribution is inferred** — when termination was requested and E2B reports a nonzero exit code, the adapter reports the last requested signal because the SDK result does not identify the terminating signal. +- **Linux utility and E2B transport semantics are assumed** — there is no PTY, Windows, arbitrary-template, or network-partition fidelity layer. diff --git a/packages/subprocess/subprocess-e2b/README.zh.md b/packages/subprocess/subprocess-e2b/README.zh.md new file mode 100644 index 0000000000..f50ff2f3d8 --- /dev/null +++ b/packages/subprocess/subprocess-e2b/README.zh.md @@ -0,0 +1,32 @@ +# @deepseek-ai/dsh-subprocess-e2b + +[English](README.md) | 中文 + +[`@deepseek-ai/dsh-subprocess`](../subprocess/README.md) seam 的 E2B 实现。它没有配置：先加载 [`@deepseek-ai/dsh-e2b`](../../e2b/e2b/README.md)，再用本服务取代 `dsh-subprocess-local`。随后，[`dsh-bash-local`](../../bash/bash-local/README.md) 等现有消费方会在共享远程沙箱中执行，无需 E2B 专用 Bash 适配器。 + +## 行为 + +- **异步远程启动**：同步 seam 会立即返回一个句柄，同时由 `Sandbox.commands.run(..., { background: true })` 在远程启动进程。SDK 返回命令 PID 之前，`pid` 为 `-1`；`done`、stdin、终止和 `waitForExit()` 会在内部等待就绪。 +- **Linux 进程组**：带引号保护的包装层会在 `setsid --wait` 下启动每组 argv，并在 `ctx.e2b.runtimeRoot/processes` 下记录实际进程组 ID 和私有状态文件。句柄会等待该文件，而不会假设 SDK 命令 PID 就是进程组 ID。终止操作以记录的负数 ID 发送 `SIGTERM`，等待调用方的 `graceMs`，再升级到 `SIGKILL` 和 SDK kill 回退。服务 dispose（资源释放）会在沙箱所有者释放前终止并等待每个保留句柄退出。 +- **环境边界**：包装层从沙箱命令环境开始，移除环境中的 `DSH_*` 和形似凭据的名称（`*KEY*`、`*SECRET*`、`*TOKEN*`），再把每个 `spec.env` 条目恢复为调用方显式选择。宿主环境变量绝不会隐式进入沙箱。 +- **stdio 投影**：pipe 模式把 E2B 回调转发到宿主 Node 流；inherit 模式把回调转发到 harness 进程流；collect 模式保留有界的宿主尾部，并支持基于偏移量读取。可选的完整 spill 文件写在远程，并且只有未超过其上限时才会对外公布。批量 stdin 和流式 stdin 都使用 SDK 句柄。 + +基础 E2B 镜像提供该适配器调用的 Bash/GNU 工具：`bash`、`setsid`、`ps`、`tr`、`env`、`chmod`、`tee` 和 `kill`。自定义模板必须保留兼容的命令。 + +## 模型体验 + +通过消费方 seam 间接影响模型，例如 `dsh-tool-bash` 背后的 Bash 执行器；这些消费方会渲染远程输出、退出事实、后台增量和 spill 路径。 + +#### KV Cache 影响 + +不会直接失效；请求前缀变更由具名消费方负责。 + +## 已知限制与延后工作 + +- **SDK 仍会在宿主内存中保留完整命令输出**：即使本适配器公开的是有界尾部，E2B `CommandHandle.stdout` 和 `.stderr` 仍会持续累积，因此无法达到进程管理 seam 通常提供的宿主内存边界。 +- **Pipe 输出并非字节保真**：E2B 交付的是分别解码后的字符串，而不是原始字节，因此拆分的多字节序列和任意二进制协议可能损坏；不支持 LSP 及其他带帧字节流消费方。 +- **不支持需要同步 PID 的消费方**：远程启动期间，`pid` 保持为 `-1`；包括 ACP 子进程后端在内，要求立即获得正 PID 的消费方无法原样使用本提供方。 +- **重新连接不会重建句柄**：保留沙箱后，远程 PID／状态／spill 文件仍然存在，但新的 harness 进程不会据此重建实时 `SubprocessHandle` 对象或输出游标。 +- **保留沙箱时会累积远程状态**：进程目录和有效的 spill 文件会留在 `.dsh-e2b` 下；本 POC 不提供保留清理。 +- **信号归因依靠推断**：如果已经请求终止，而 E2B 报告非零退出码，适配器会报告最后请求的信号，因为 SDK 结果不标识终止信号。 +- **依赖 Linux 工具与 E2B 传输语义**：没有 PTY、Windows、任意模板或网络分区的保真层。 diff --git a/packages/subprocess/subprocess-e2b/package.json b/packages/subprocess/subprocess-e2b/package.json new file mode 100644 index 0000000000..d73607d1f3 --- /dev/null +++ b/packages/subprocess/subprocess-e2b/package.json @@ -0,0 +1,41 @@ +{ + "name": "@deepseek-ai/dsh-subprocess-e2b", + "description": "E2B subprocess implementation for DeepSeek Harness", + "version": "0.0.1", + "private": true, + "type": "module", + "main": "lib/index.js", + "types": "lib/types/index.d.ts", + "exports": { + ".": { + "types": "./lib/types/index.d.ts", + "default": "./lib/index.js" + }, + "./invariant": { + "types": "./lib/types/invariant.d.ts", + "default": "./lib/invariant.js" + }, + "./src/*": "./src/*", + "./package.json": "./package.json" + }, + "files": [ + "lib/index.js", + "lib/invariant.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", + "src" + ], + "license": "BSD-3-Clause", + "peerDependencies": { + "@deepseek-ai/dsh-e2b": "^0.0.1", + "@deepseek-ai/dsh-invariants": "^0.0.1", + "@deepseek-ai/dsh-subprocess": "^0.0.1", + "cordis": "^4.0.0-rc.7" + }, + "devDependencies": { + "@deepseek-ai/dsh-e2b": "workspace:^", + "@deepseek-ai/dsh-invariants": "workspace:^", + "@deepseek-ai/dsh-subprocess": "workspace:^", + "cordis": "^4.0.0-rc.7" + } +} diff --git a/packages/subprocess/subprocess-e2b/src/index.ts b/packages/subprocess/subprocess-e2b/src/index.ts new file mode 100644 index 0000000000..2856df67bd --- /dev/null +++ b/packages/subprocess/subprocess-e2b/src/index.ts @@ -0,0 +1,58 @@ +/** + * E2B implementation of the subprocess seam. Each handle starts through the + * shared sandbox and retains command output/status paths in that remote world. + * @module @deepseek-ai/dsh-subprocess-e2b + */ + +import { randomUUID } from 'node:crypto' +import { posix } from 'node:path' +import { Context } from 'cordis' +import { SubprocessService } from '@deepseek-ai/dsh-subprocess' +import type { SubprocessHandle, SubprocessSpawnSpec } from '@deepseek-ai/dsh-subprocess' +import { E2BSubprocessHandle } from './process.ts' + +/** E2B command manager registered as `ctx.subprocess`. */ +export class E2BSubprocessService extends SubprocessService { + static inject = ['e2b'] + + private readonly live = new Set() + + /** Create the E2B subprocess service and bind its disposal policy. */ + constructor(ctx: Context) { + super(ctx) + ctx.effect(() => async () => { + const handles = [...this.live] + for (const handle of handles) handle.terminate() + await Promise.all(handles.map(async (handle) => { + await handle.done.catch(() => {}) + await handle.waitForExit() + })) + this.live.clear() + }, 'e2b subprocess teardown') + } + + /** @inheritdoc */ + spawn(spec: SubprocessSpawnSpec): SubprocessHandle { + const program = spec.argv[0] + if (program === undefined || program.length === 0) { + throw new Error('invalid argv: expected a non-empty program name at argv[0]') + } + if (!Number.isFinite(spec.graceMs) || spec.graceMs <= 0) { + throw new Error('subprocess-e2b: graceMs must be a positive finite number') + } + if (spec.signal?.aborted === true) { + throw new Error(`aborted before spawn: ${String(spec.signal.reason ?? 'aborted')}`) + } + const stateDir = posix.join(this.ctx.e2b.runtimeRoot, 'processes', randomUUID()) + const handle = new E2BSubprocessHandle(this.ctx.e2b, spec, stateDir) + this.live.add(handle) + const release = async (): Promise => { + await handle.waitForExit() + this.live.delete(handle) + } + void handle.done.then(release, release).catch(() => {}) + return handle + } +} + +export default E2BSubprocessService diff --git a/packages/subprocess/subprocess-e2b/src/invariant.ts b/packages/subprocess/subprocess-e2b/src/invariant.ts new file mode 100644 index 0000000000..4416175b1b --- /dev/null +++ b/packages/subprocess/subprocess-e2b/src/invariant.ts @@ -0,0 +1,27 @@ +/** Package-owned invariant companion for `@deepseek-ai/dsh-subprocess-e2b`. */ + +/* jscpd:ignore-start */ +import type { Context } from 'cordis' +import type { InvariantInstaller } from '@deepseek-ai/dsh-invariants' + +const PACKAGE_NAME = '@deepseek-ai/dsh-subprocess-e2b' + +/** Cordis companion plugin name. */ +export const name = 'subprocess-e2b-invariant' +/** Service required before reserving package ownership. */ +export const inject = ['invariants'] + +/** + * No runtime invariant: live remote handles are private teardown ownership, + * and the E2B command event stream is the sole outcome authority. + */ +const install: InvariantInstaller = () => {} + +/** + * Register this package's invariant companion. + * @param ctx - Cordis context carrying the invariant service. + * @returns the installed registration's disposer after setup succeeds. + */ +export const apply = (ctx: Context): Promise<() => void> => + Promise.resolve(ctx.invariants.register(PACKAGE_NAME, install)) +/* jscpd:ignore-end */ diff --git a/packages/subprocess/subprocess-e2b/src/output.ts b/packages/subprocess/subprocess-e2b/src/output.ts new file mode 100644 index 0000000000..b6551983a0 --- /dev/null +++ b/packages/subprocess/subprocess-e2b/src/output.ts @@ -0,0 +1,70 @@ +/** Bounded host-side projection of a complete output file retained in E2B. */ + +import { Buffer } from 'node:buffer' +import type { SubprocessOutputRead, SubprocessOutputReader } from '@deepseek-ai/dsh-subprocess' + +/** Offset reader used for one collect-mode E2B stream. */ +export class E2BOutputReader implements SubprocessOutputReader { + private chunks: Buffer[] = [] + private retainedBytes = 0 + private totalBytes = 0 + + /** + * Create a bounded reader over one remote spill path. + * @param maxBytes - In-memory tail cap. + * @param maxSpillBytes - Maximum complete remote file size the caller accepts. + * @param spillPath - Remote full-output path. + */ + constructor( + private readonly maxBytes: number, + private readonly maxSpillBytes: number | undefined, + private readonly spillPath: string, + ) {} + + /** Total bytes observed from the SDK stream. */ + get size(): number { + return this.totalBytes + } + + /** + * Append one decoded SDK output event. + * @param text - Event text delivered by E2B. + */ + push(text: string): void { + if (text.length === 0) return + const chunk = Buffer.from(text) + this.totalBytes += chunk.length + this.chunks.push(chunk) + this.retainedBytes += chunk.length + while (this.retainedBytes > this.maxBytes) { + const head = this.chunks[0] as Buffer + const excess = this.retainedBytes - this.maxBytes + if (head.length <= excess) { + this.chunks.shift() + this.retainedBytes -= head.length + } else { + this.chunks[0] = head.subarray(excess) + this.retainedBytes -= excess + } + } + } + + /** @inheritdoc */ + readFrom(fromByte: number): SubprocessOutputRead { + if (!Number.isSafeInteger(fromByte) || fromByte < 0) { + throw new Error('subprocess output offset must be a non-negative safe integer') + } + const retained = Buffer.concat(this.chunks, this.retainedBytes) + const firstRetained = this.totalBytes - this.retainedBytes + const lossy = fromByte < firstRetained + const start = lossy ? 0 : Math.min(retained.length, Math.max(0, fromByte - firstRetained)) + return { + text: retained.subarray(start).toString('utf8'), + nextOffset: this.totalBytes, + lossy, + ...(lossy && this.maxSpillBytes !== undefined && this.totalBytes <= this.maxSpillBytes + ? { spillPath: this.spillPath } + : {}), + } + } +} diff --git a/packages/subprocess/subprocess-e2b/src/process.ts b/packages/subprocess/subprocess-e2b/src/process.ts new file mode 100644 index 0000000000..507ea700c4 --- /dev/null +++ b/packages/subprocess/subprocess-e2b/src/process.ts @@ -0,0 +1,417 @@ +/** One asynchronously-started E2B command projected onto the subprocess seam. */ + +import { Buffer } from 'node:buffer' +import { PassThrough, Writable } from 'node:stream' +import { posix } from 'node:path' +import { + CommandExitError, + quoteE2BShellArg, +} from '@deepseek-ai/dsh-e2b' +import type { CommandHandle, CommandResult, Sandbox } from '@deepseek-ai/dsh-e2b' +import type { + SubprocessCollect, + SubprocessHandle, + SubprocessOutcome, + SubprocessOutputMode, + SubprocessSpawnSpec, +} from '@deepseek-ai/dsh-subprocess' +import type E2BSandboxService from '@deepseek-ai/dsh-e2b' +import { E2BOutputReader } from './output.ts' + +const GROUP_POLL_MS = 20 + +function isCollect(mode: SubprocessOutputMode): mode is SubprocessCollect { + return mode !== 'pipe' && mode !== 'inherit' +} + +function hasSpill(mode: SubprocessOutputMode): mode is SubprocessCollect & { spill: { maxBytes: number } } { + return isCollect(mode) && mode.spill !== undefined +} + +function asError(error: unknown): Error { + return error instanceof Error ? error : new Error(String(error)) +} + +class DeferredStdin extends Writable { + constructor(private readonly ready: Promise) { + super({ decodeStrings: false }) + } + + override _write(chunk: string | Buffer, _encoding: BufferEncoding, callback: (error?: Error | null) => void): void { + void this.ready.then(handle => handle.sendStdin(chunk)).then( + () => { callback() }, + (error: unknown) => { callback(asError(error)) }, + ) + } + + override _final(callback: (error?: Error | null) => void): void { + void this.ready.then(handle => handle.closeStdin()).then( + () => { callback() }, + (error: unknown) => { callback(asError(error)) }, + ) + } +} + +interface RemotePaths { + pid: string + status: string + stdout: string + stderr: string +} + +function explicitEnvironmentNames(env: Readonly> | undefined): string { + return Object.keys(env ?? {}) + .map(quoteE2BShellArg) + .join(' ') +} + +function commandText(spec: SubprocessSpawnSpec, paths: RemotePaths): string { + const stdoutRedirect = hasSpill(spec.stdio.stdout) + ? `> >(tee -a -- ${quoteE2BShellArg(paths.stdout)})` + : '' + const stderrRedirect = hasSpill(spec.stdio.stderr) + ? `2> >(tee -a -- ${quoteE2BShellArg(paths.stderr)} >&2)` + : '' + const environmentNames = explicitEnvironmentNames(spec.env) + const inner = [ + 'set +e', + 'umask 077', + 'dsh_e2b_pgid="$(ps -o pgid= -p "$$" | tr -d " ")"', + `printf '%s\\n' "$dsh_e2b_pgid" > ${quoteE2BShellArg(paths.pid)}`, + 'dsh_e2b_env=()', + `dsh_e2b_explicit=(${environmentNames})`, + 'while IFS= read -r dsh_e2b_name; do', + ' case "${dsh_e2b_name^^}" in DSH_*|*KEY*|*SECRET*|*TOKEN*) continue ;; esac', + ' dsh_e2b_env+=("$dsh_e2b_name=${!dsh_e2b_name}")', + 'done < <(compgen -e)', + 'for dsh_e2b_name in "${dsh_e2b_explicit[@]}"; do dsh_e2b_env+=("$dsh_e2b_name=${!dsh_e2b_name}"); done', + `env -i "\${dsh_e2b_env[@]}" "$@" ${stdoutRedirect} ${stderrRedirect}`.trimEnd(), + 'dsh_e2b_status=$?', + 'wait', + `printf '%s\\n' "$dsh_e2b_status" > ${quoteE2BShellArg(paths.status)}`, + 'exit "$dsh_e2b_status"', + ].join('\n') + const argv = spec.argv.map(quoteE2BShellArg).join(' ') + return `exec setsid --wait -- bash -c ${quoteE2BShellArg(inner)} dsh-e2b ${argv}` +} + +function signalOpts(signal: AbortSignal | undefined): { signal?: AbortSignal } { + return signal === undefined ? {} : { signal } +} + +function isAborted(signal: AbortSignal | undefined): boolean { + return signal?.aborted === true +} + +function waitTick(signal?: AbortSignal): Promise { + if (signal?.aborted === true) return Promise.resolve(false) + return new Promise((resolve) => { + const timer = setTimeout(() => { + signal?.removeEventListener('abort', onAbort) + resolve(true) + }, GROUP_POLL_MS) + const onAbort = (): void => { + clearTimeout(timer) + resolve(false) + } + signal?.addEventListener('abort', onAbort, { once: true }) + }) +} + +/** E2B-backed subprocess handle with deferred remote PID acquisition. */ +export class E2BSubprocessHandle implements SubprocessHandle { + readonly stdin: Writable | undefined + readonly stdout: PassThrough | undefined + readonly stderr: PassThrough | undefined + readonly collected: SubprocessHandle['collected'] + readonly done: Promise + + private readonly readyState = Promise.withResolvers() + private readonly stdoutReader: E2BOutputReader | undefined + private readonly stderrReader: E2BOutputReader | undefined + private readonly paths: RemotePaths + private remotePid = -1 + private settled = false + private terminationRequested = false + private terminationSignal: NodeJS.Signals | null = null + private termination: Promise | undefined + + /** + * Begin an E2B command without blocking the synchronous subprocess spawn seam. + * @param runtime - Shared E2B sandbox owner. + * @param spec - Fully resolved subprocess request. + * @param stateDir - Remote directory retaining process identity, status, and valid spills. + */ + constructor( + private readonly runtime: E2BSandboxService, + private readonly spec: SubprocessSpawnSpec, + readonly stateDir: string, + ) { + this.paths = { + pid: posix.join(stateDir, 'pid'), + status: posix.join(stateDir, 'exit-code'), + stdout: posix.join(stateDir, 'stdout.log'), + stderr: posix.join(stateDir, 'stderr.log'), + } + const outMode = spec.stdio.stdout + const errMode = spec.stdio.stderr + this.stdout = outMode === 'pipe' ? new PassThrough() : undefined + this.stderr = errMode === 'pipe' ? new PassThrough() : undefined + this.stdoutReader = isCollect(outMode) + ? new E2BOutputReader(outMode.maxBytes, outMode.spill?.maxBytes, this.paths.stdout) + : undefined + this.stderrReader = isCollect(errMode) + ? new E2BOutputReader(errMode.maxBytes, errMode.spill?.maxBytes, this.paths.stderr) + : undefined + this.collected = { + ...(this.stdoutReader !== undefined ? { stdout: this.stdoutReader } : {}), + ...(this.stderrReader !== undefined ? { stderr: this.stderrReader } : {}), + } + this.stdin = spec.stdio.stdin === 'pipe' ? new DeferredStdin(this.readyState.promise) : undefined + void this.readyState.promise.catch(() => {}) + spec.signal?.addEventListener('abort', this.onAbort, { once: true }) + this.done = this.run() + void this.done.catch(() => {}) + if (spec.signal?.aborted === true) this.terminate() + } + + /** Remote process id after start; `-1` while E2B startup is pending or after it fails. */ + get pid(): number { + return this.remotePid + } + + /** @inheritdoc */ + terminate(): void { + if (this.terminationRequested || this.settled) return + this.terminationRequested = true + this.termination = this.terminateRemote() + void this.termination.catch(() => {}) + } + + /** @inheritdoc */ + async waitForExit(signal?: AbortSignal): Promise { + let handle: CommandHandle | undefined + try { + handle = await this.readyForWait(signal) + } catch { + return true + } + if (handle === undefined) return false + let sandbox: Sandbox + try { + sandbox = await this.runtime.getSandbox() + } catch (error: unknown) { + if (isAborted(signal)) return false + throw error + } + while (await this.groupAlive(sandbox, this.remotePid, signal)) { + if (!await waitTick(signal)) return false + } + return !isAborted(signal) + } + + private readyForWait(signal: AbortSignal | undefined): Promise { + if (signal === undefined) return this.readyState.promise + return new Promise((resolve, reject) => { + const onAbort = (): void => { cleanup(); resolve(undefined) } + const cleanup = (): void => { signal.removeEventListener('abort', onAbort) } + signal.addEventListener('abort', onAbort, { once: true }) + if (signal.aborted) { + onAbort() + return + } + void this.readyState.promise.then( + (handle) => { cleanup(); resolve(handle) }, + (error: unknown) => { cleanup(); reject(asError(error)) }, + ) + }) + } + + private readonly onAbort = (): void => { this.terminate() } + + private async run(): Promise { + try { + const sandbox = await this.runtime.getSandbox() + await this.prepareState(sandbox) + const handle = await sandbox.commands.run( + commandText(this.spec, this.paths), + { + background: true, + cwd: this.spec.cwd, + stdin: this.spec.stdio.stdin !== 'ignore', + timeoutMs: 0, + ...(this.spec.env !== undefined ? { envs: this.spec.env } : {}), + onStdout: async (data) => { await this.dispatchOutput('stdout', data) }, + onStderr: async (data) => { await this.dispatchOutput('stderr', data) }, + }, + ) + if (!Number.isSafeInteger(handle.pid) || handle.pid <= 0) { + throw new Error(`subprocess-e2b: E2B returned invalid command pid ${handle.pid}`) + } + const completion = handle.wait() + void completion.catch(() => {}) + this.remotePid = await this.waitForProcessGroupId(sandbox, completion) + this.readyState.resolve(handle) + await this.writeBatchStdin(handle) + const outcome = await this.waitForCommand(completion) + await this.finalizeSpills(sandbox) + return outcome + } catch (error: unknown) { + this.readyState.reject(error) + throw error + } finally { + this.settled = true + this.spec.signal?.removeEventListener('abort', this.onAbort) + this.stdout?.end() + this.stderr?.end() + } + } + + private async prepareState(sandbox: Sandbox): Promise { + await sandbox.files.makeDir(this.stateDir) + const files = [ + { path: this.paths.pid, data: '' }, + { path: this.paths.status, data: '' }, + ...(hasSpill(this.spec.stdio.stdout) ? [{ path: this.paths.stdout, data: '' }] : []), + ...(hasSpill(this.spec.stdio.stderr) ? [{ path: this.paths.stderr, data: '' }] : []), + ] + await sandbox.files.write(files) + await sandbox.commands.run([ + `chmod 700 -- ${quoteE2BShellArg(this.stateDir)}`, + `chmod 600 -- ${files.map(file => quoteE2BShellArg(file.path)).join(' ')}`, + ].join('\n')) + } + + private async writeBatchStdin(handle: CommandHandle): Promise { + if (typeof this.spec.stdio.stdin !== 'object') return + try { + await handle.sendStdin(this.spec.stdio.stdin.data) + await handle.closeStdin() + } catch (_processClosedItsInput) { + // Like the local adapter, batch stdin is best-effort; exit and output remain authoritative. + } + } + + private async dispatchOutput(stream: 'stdout' | 'stderr', data: string): Promise { + try { + if (stream === 'stdout') { + this.stdoutReader?.push(data) + await this.writeOutput(this.stdout, this.spec.stdio.stdout === 'inherit' ? process.stdout : undefined, data) + return + } + this.stderrReader?.push(data) + await this.writeOutput(this.stderr, this.spec.stdio.stderr === 'inherit' ? process.stderr : undefined, data) + } catch (error: unknown) { + const target = stream === 'stdout' ? this.stdout : this.stderr + target?.destroy(asError(error)) + } + } + + private async writeOutput(pipe: PassThrough | undefined, inherited: NodeJS.WriteStream | undefined, data: string): Promise { + const target = pipe ?? inherited + if (target === undefined || data.length === 0) return + if (target.destroyed) throw new Error('subprocess output stream is closed') + if (target.write(Buffer.from(data))) return + await new Promise((resolve, reject) => { + const onDrain = (): void => { cleanup(); resolve() } + const onError = (error: Error): void => { cleanup(); reject(error) } + const cleanup = (): void => { + target.removeListener('drain', onDrain) + target.removeListener('error', onError) + } + target.once('drain', onDrain) + target.once('error', onError) + }) + } + + private async waitForProcessGroupId(sandbox: Sandbox, completion: Promise): Promise { + const commandSettled = completion.then( + () => true, + () => true, + ) + while (true) { + const raw = await sandbox.files.read(this.paths.pid) + const value = raw.trim() + if (value.length > 0) { + const pid = Number(value) + if (!/^[1-9][0-9]*$/.test(value) || !Number.isSafeInteger(pid)) { + throw new Error(`subprocess-e2b: remote wrapper published invalid process-group id ${JSON.stringify(value)}`) + } + return pid + } + const settled = await Promise.race([commandSettled, waitTick().then(() => false)]) + if (settled) throw new Error('subprocess-e2b: remote command exited before publishing its process-group id') + } + } + + private async waitForCommand(completion: Promise): Promise { + try { + const result = await completion + return { exitCode: result.exitCode, signal: null } + } catch (error: unknown) { + if (error instanceof CommandExitError) { + return this.terminationSignal === null + ? { exitCode: error.exitCode, signal: null } + : { exitCode: null, signal: this.terminationSignal } + } + throw error + } + } + + private async terminateRemote(): Promise { + let handle: CommandHandle + try { + handle = await this.readyState.promise + } catch { + return + } + const sandbox = await this.runtime.getSandbox() + this.terminationSignal = 'SIGTERM' + await this.signalGroup(sandbox, this.remotePid, 'TERM') + const deadline = Date.now() + this.spec.graceMs + while (Date.now() < deadline && await this.groupAlive(sandbox, this.remotePid)) { + await waitTick() + } + if (!await this.groupAlive(sandbox, this.remotePid)) return + this.terminationSignal = 'SIGKILL' + try { + await this.signalGroup(sandbox, this.remotePid, 'KILL') + } finally { + await handle.kill().catch(() => false) + } + } + + private async signalGroup(sandbox: Sandbox, pid: number, signal: 'TERM' | 'KILL'): Promise { + try { + await sandbox.commands.run(`kill -${signal} -- -${pid}`) + } catch (error: unknown) { + if (!(error instanceof CommandExitError)) throw error + } + } + + private async groupAlive(sandbox: Sandbox, pid: number, signal?: AbortSignal): Promise { + try { + await sandbox.commands.run(`kill -0 -- -${pid}`, signalOpts(signal)) + return true + } catch (error: unknown) { + if (signal?.aborted === true) return false + if (error instanceof CommandExitError) return false + throw error + } + } + + private async finalizeSpills(sandbox: Sandbox): Promise { + const removals: Promise[] = [] + const collect = (mode: SubprocessOutputMode, reader: E2BOutputReader | undefined, path: string): void => { + if (!hasSpill(mode)) return + // A spill mode is a collect mode, so construction always created its reader. + const size = (reader as E2BOutputReader).size + if (size <= mode.maxBytes || size > mode.spill.maxBytes) { + removals.push(sandbox.files.remove(path).catch(() => {})) + } + } + collect(this.spec.stdio.stdout, this.stdoutReader, this.paths.stdout) + collect(this.spec.stdio.stderr, this.stderrReader, this.paths.stderr) + await Promise.all(removals) + } +} diff --git a/packages/subprocess/subprocess-e2b/tests/subprocess.spec.ts b/packages/subprocess/subprocess-e2b/tests/subprocess.spec.ts new file mode 100644 index 0000000000..ff9d48073c --- /dev/null +++ b/packages/subprocess/subprocess-e2b/tests/subprocess.spec.ts @@ -0,0 +1,725 @@ +import { once } from 'node:events' +import { Context } from 'cordis' +import { + CommandExitError, + type CommandHandle, + type CommandResult, + type Sandbox, +} from '@deepseek-ai/dsh-e2b' +import type E2BSandboxService from '@deepseek-ai/dsh-e2b' +import type { SubprocessSpawnSpec } from '@deepseek-ai/dsh-subprocess' +import E2BSubprocessService from '@deepseek-ai/dsh-subprocess-e2b' +import * as E2BSubprocessInvariant from '../src/invariant.ts' +import { E2BOutputReader } from '../src/output.ts' +import { E2BSubprocessHandle } from '../src/process.ts' +import InvariantService from '@deepseek-ai/dsh-invariants' +import { describe, expect, it, vi } from 'vitest' + +function commandError(exitCode: number): CommandExitError { + return new CommandExitError({ exitCode, stdout: '', stderr: '', error: `exit ${exitCode}` }) +} + +interface StartOptions { + background: true + cwd: string + stdin: boolean + timeoutMs: number + signal?: AbortSignal + envs?: Record + onStdout?: (data: string) => void | Promise + onStderr?: (data: string) => void | Promise +} + +class FakeCommandHandle { + pid = 4242 + readonly sent: Array = [] + closes = 0 + kills = 0 + killError: unknown + private readonly result = Promise.withResolvers() + private settled = false + + wait(): Promise { + return this.result.promise + } + + async sendStdin(data: string | Uint8Array): Promise { + this.sent.push(data) + } + + async closeStdin(): Promise { + this.closes += 1 + } + + async kill(): Promise { + this.kills += 1 + if (this.killError !== undefined) throw this.killError + return true + } + + succeed(exitCode = 0): void { + if (this.settled) return + this.settled = true + this.result.resolve({ exitCode, stdout: '', stderr: '' }) + } + + fail(exitCode: number): void { + if (this.settled) return + this.settled = true + this.result.reject(commandError(exitCode)) + } + + crash(error: unknown): void { + if (this.settled) return + this.settled = true + this.result.reject(error) + } +} + +class FakeSandbox { + readonly handle = new FakeCommandHandle() + readonly commandsSeen: string[] = [] + readonly writtenFiles: string[][] = [] + readonly removed: string[] = [] + readonly directories: string[] = [] + startOptions: StartOptions | undefined + backgroundError: unknown + nextRemoveError: unknown + probeError: unknown + signalError: unknown + trapsTerm = false + alive = true + processGroupId = '4242\n' + readonly processGroupReads: string[] = [] + beforeProbe: (() => void) | undefined + afterProbe: (() => void) | undefined + private startGate: Promise | undefined + private openStart: (() => void) | undefined + + deferStart(): void { + const gate = Promise.withResolvers() + this.startGate = gate.promise + this.openStart = () => { gate.resolve(undefined) } + } + + releaseStart(): void { + this.openStart?.() + } + + finish(exitCode = 0): void { + this.alive = false + if (exitCode === 0) this.handle.succeed(0) + else this.handle.fail(exitCode) + } + + async stdout(data: string): Promise { + await this.startOptions?.onStdout?.(data) + } + + async stderr(data: string): Promise { + await this.startOptions?.onStderr?.(data) + } + + readonly sandbox = { + sandboxId: 'fake', + files: { + makeDir: async (path: string): Promise => { + this.directories.push(path) + return true + }, + write: async (files: Array<{ path: string; data: string }>): Promise => { + this.writtenFiles.push(files.map(file => file.path)) + return files.map(() => ({})) + }, + read: async (): Promise => this.processGroupReads.shift() ?? this.processGroupId, + remove: async (path: string): Promise => { + this.removed.push(path) + if (this.nextRemoveError !== undefined) { + const error = this.nextRemoveError + this.nextRemoveError = undefined + throw error + } + }, + }, + commands: { + run: async (command: string, options?: StartOptions | { signal?: AbortSignal }): Promise => { + this.commandsSeen.push(command) + if (command.startsWith('kill -0 ')) { + this.beforeProbe?.() + if (options?.signal?.aborted === true) throw new DOMException('aborted', 'AbortError') + if (this.probeError !== undefined) { + const error = this.probeError + this.probeError = undefined + throw error + } + if (!this.alive) throw commandError(1) + this.afterProbe?.() + return { exitCode: 0, stdout: '', stderr: '' } + } + if (command.startsWith('kill -TERM ')) { + if (this.signalError !== undefined) { + const error = this.signalError + this.signalError = undefined + throw error + } + if (!this.trapsTerm) { + this.alive = false + this.handle.fail(143) + } + return { exitCode: 0, stdout: '', stderr: '' } + } + if (command.startsWith('kill -KILL ')) { + if (this.signalError !== undefined) { + const error = this.signalError + this.signalError = undefined + throw error + } + this.alive = false + this.handle.fail(137) + return { exitCode: 0, stdout: '', stderr: '' } + } + if ((options as StartOptions | undefined)?.background === true) { + this.startOptions = options as StartOptions + await this.startGate + if (this.backgroundError !== undefined) throw this.backgroundError + return this.handle as unknown as CommandHandle + } + return { exitCode: 0, stdout: '', stderr: '' } + }, + }, + } as unknown as Sandbox +} + +function spec(overrides: Partial = {}): SubprocessSpawnSpec { + return { + argv: ['bash', '-c', 'printf ok'], + cwd: '/workspace', + stdio: { + stdin: 'ignore', + stdout: { maxBytes: 4, spill: { maxBytes: 16 } }, + stderr: { maxBytes: 4 }, + }, + graceMs: 5, + ...overrides, + } +} + +function runtime(fake: FakeSandbox, getSandbox: () => Promise = async () => fake.sandbox): E2BSandboxService { + return { + cwd: '/workspace', + runtimeRoot: '/workspace/.dsh-e2b', + disposeMode: 'kill', + getSandbox, + } as unknown as E2BSandboxService +} + +async function flush(): Promise { + await new Promise(resolve => setTimeout(resolve, 0)) +} + +describe('E2BOutputReader', () => { + it('keeps a byte-exact tail with independent whole-stream cursors', () => { + const reader = new E2BOutputReader(4, 10, '/remote/spill') + reader.push('') + reader.push('ab') + reader.push('cdef') + expect(reader.size).toBe(6) + expect(reader.readFrom(0)).toEqual({ text: 'cdef', nextOffset: 6, lossy: true, spillPath: '/remote/spill' }) + expect(reader.readFrom(2)).toEqual({ text: 'cdef', nextOffset: 6, lossy: false }) + expect(reader.readFrom(5)).toEqual({ text: 'f', nextOffset: 6, lossy: false }) + expect(reader.readFrom(99)).toEqual({ text: '', nextOffset: 6, lossy: false }) + }) + + it('drops whole head chunks and withholds absent or over-cap spills', () => { + const withoutSpill = new E2BOutputReader(2, undefined, '/unused') + withoutSpill.push('ab') + withoutSpill.push('cd') + expect(withoutSpill.readFrom(0)).toEqual({ text: 'cd', nextOffset: 4, lossy: true }) + const overCap = new E2BOutputReader(2, 3, '/too-small') + overCap.push('abcd') + expect(overCap.readFrom(0)).toEqual({ text: 'cd', nextOffset: 4, lossy: true }) + expect(() => overCap.readFrom(-1)).toThrow(/non-negative safe integer/) + expect(() => overCap.readFrom(1.5)).toThrow(/non-negative safe integer/) + }) +}) + +describe('E2BSubprocessHandle', () => { + it('starts asynchronously, keeps secrets out of the command, and supports deferred piped stdin/output', async () => { + const fake = new FakeSandbox() + fake.processGroupId = '4343\n' + fake.deferStart() + const handle = new E2BSubprocessHandle(runtime(fake), spec({ + argv: ['tool', 'argument with spaces'], + stdio: { stdin: 'pipe', stdout: 'pipe', stderr: { maxBytes: 8, spill: { maxBytes: 32 } } }, + env: { PATH: '/bin', DEEPSEEK_API_KEY: 'explicit-secret', DSH_MODE: 'test' }, + }), '/workspace/.dsh-e2b/processes/one') + expect(handle.pid).toBe(-1) + handle.stdin!.write('hello') + handle.stdin!.end() + fake.releaseStart() + await flush() + expect(handle.pid).toBe(4343) + expect(fake.handle.sent.map(value => String(value))).toEqual(['hello']) + expect(fake.handle.closes).toBe(1) + expect(fake.startOptions?.envs).toEqual({ PATH: '/bin', DEEPSEEK_API_KEY: 'explicit-secret', DSH_MODE: 'test' }) + const command = fake.commandsSeen.find(value => value.startsWith('exec setsid'))! + expect(command).toContain('exec setsid --wait -- bash -c') + expect(command).toContain('DEEPSEEK_API_KEY') + expect(command).toContain('DSH_MODE') + expect(command).not.toContain('explicit-secret') + expect(fake.writtenFiles[0]).toEqual([ + '/workspace/.dsh-e2b/processes/one/pid', + '/workspace/.dsh-e2b/processes/one/exit-code', + '/workspace/.dsh-e2b/processes/one/stderr.log', + ]) + + let piped = '' + handle.stdout!.on('data', (chunk) => { piped += String(chunk) }) + await fake.stdout('pipe-data') + await fake.stderr('err') + fake.finish() + await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) + expect(piped).toBe('pipe-data') + expect(handle.collected.stderr!.readFrom(0)).toMatchObject({ text: 'err', lossy: false }) + expect(fake.removed).toContain('/workspace/.dsh-e2b/processes/one/stderr.log') + await expect(handle.waitForExit()).resolves.toBe(true) + }) + + it('surfaces deferred piped-stdin write and close failures as stream errors', async () => { + const writeFake = new FakeSandbox() + writeFake.deferStart() + vi.spyOn(writeFake.handle, 'sendStdin').mockRejectedValueOnce('stdin rejected') + const writeHandle = new E2BSubprocessHandle(runtime(writeFake), spec({ + stdio: { stdin: 'pipe', stdout: { maxBytes: 4 }, stderr: { maxBytes: 4 } }, + }), '/runtime/stdin-write-error') + const writeError = once(writeHandle.stdin!, 'error') + writeHandle.stdin!.write('input') + writeFake.releaseStart() + await expect(writeError).resolves.toMatchObject([{ message: 'stdin rejected' }]) + writeFake.finish() + await writeHandle.done + + const closeFake = new FakeSandbox() + vi.spyOn(closeFake.handle, 'closeStdin').mockRejectedValueOnce(new Error('close rejected')) + const closeHandle = new E2BSubprocessHandle(runtime(closeFake), spec({ + stdio: { stdin: 'pipe', stdout: { maxBytes: 4 }, stderr: { maxBytes: 4 } }, + }), '/runtime/stdin-close-error') + await flush() + const closeError = once(closeHandle.stdin!, 'error') + closeHandle.stdin!.end() + await expect(closeError).resolves.toMatchObject([{ message: 'close rejected' }]) + closeFake.finish() + await closeHandle.done + }) + + it('collects bounded tails, retains valid spills, and maps natural nonzero exits', async () => { + const fake = new FakeSandbox() + const handle = new E2BSubprocessHandle(runtime(fake), spec({ + stdio: { + stdin: { data: 'batch' }, + stdout: { maxBytes: 4, spill: { maxBytes: 16 } }, + stderr: { maxBytes: 3 }, + }, + }), '/runtime/two') + await flush() + await fake.stdout('abcdef') + await fake.stderr('12345') + fake.finish(7) + await expect(handle.done).resolves.toEqual({ exitCode: 7, signal: null }) + expect(fake.handle.sent).toEqual(['batch']) + expect(fake.handle.closes).toBe(1) + expect(handle.collected.stdout!.readFrom(0)).toEqual({ + text: 'cdef', + nextOffset: 6, + lossy: true, + spillPath: '/runtime/two/stdout.log', + }) + expect(handle.collected.stderr!.readFrom(0)).toEqual({ text: '345', nextOffset: 5, lossy: true }) + expect(fake.removed).not.toContain('/runtime/two/stdout.log') + }) + + it('removes a spill once the complete stream exceeds its cap', async () => { + const fake = new FakeSandbox() + const handle = new E2BSubprocessHandle(runtime(fake), spec({ + stdio: { stdin: 'ignore', stdout: { maxBytes: 2, spill: { maxBytes: 3 } }, stderr: 'inherit' }, + }), '/runtime/oversize') + await flush() + await fake.stdout('abcd') + await fake.stderr('') + fake.finish() + await handle.done + expect(handle.collected.stdout!.readFrom(0)).toEqual({ text: 'cd', nextOffset: 4, lossy: true }) + expect(fake.removed).toContain('/runtime/oversize/stdout.log') + }) + + it('contains remote spill-removal failures and routes empty inherited output', async () => { + const fake = new FakeSandbox() + fake.nextRemoveError = new Error('already removed') + const handle = new E2BSubprocessHandle(runtime(fake), spec({ + stdio: { stdin: 'ignore', stdout: 'inherit', stderr: { maxBytes: 4, spill: { maxBytes: 8 } } }, + }), '/runtime/remove-error') + await flush() + await fake.stdout('') + await fake.stderr('') + fake.finish() + await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) + expect(fake.removed).toContain('/runtime/remove-error/stderr.log') + }) + + it('terminates a process group with TERM and reports the signal outcome', async () => { + const fake = new FakeSandbox() + const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/term') + await flush() + handle.terminate() + handle.terminate() + await expect(handle.done).resolves.toEqual({ exitCode: null, signal: 'SIGTERM' }) + await expect(handle.waitForExit()).resolves.toBe(true) + expect(fake.commandsSeen).toContain('kill -TERM -- -4242') + expect(fake.commandsSeen).not.toContain('kill -KILL -- -4242') + }) + + it('escalates a TERM-trapping process group to KILL and uses the SDK kill as fallback', async () => { + const fake = new FakeSandbox() + fake.trapsTerm = true + fake.handle.killError = new Error('already gone') + const handle = new E2BSubprocessHandle(runtime(fake), spec({ graceMs: 1 }), '/runtime/kill') + await flush() + handle.terminate() + await expect(handle.done).resolves.toEqual({ exitCode: null, signal: 'SIGKILL' }) + await expect(handle.waitForExit()).resolves.toBe(true) + expect(fake.commandsSeen).toContain('kill -KILL -- -4242') + expect(fake.handle.kills).toBe(1) + }) + + it('honors termination requested before asynchronous startup finishes', async () => { + const fake = new FakeSandbox() + fake.deferStart() + const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/deferred-kill') + handle.terminate() + fake.releaseStart() + await expect(handle.done).resolves.toEqual({ exitCode: null, signal: 'SIGTERM' }) + }) + + it('honors an already-aborted signal when constructing the asynchronous handle directly', async () => { + const fake = new FakeSandbox() + const handle = new E2BSubprocessHandle(runtime(fake), spec({ signal: AbortSignal.abort('stop') }), '/runtime/pre-aborted') + await expect(handle.done).resolves.toEqual({ exitCode: null, signal: 'SIGTERM' }) + }) + + it('reacts to a signal that aborts after the remote command has started', async () => { + const fake = new FakeSandbox() + const controller = new AbortController() + const handle = new E2BSubprocessHandle(runtime(fake), spec({ signal: controller.signal }), '/runtime/live-abort') + await flush() + controller.abort('stop') + await expect(handle.done).resolves.toEqual({ exitCode: null, signal: 'SIGTERM' }) + }) + + it('bounds waitForExit while startup or a live group is pending', async () => { + const fake = new FakeSandbox() + fake.deferStart() + const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/wait') + const beforeStart = new AbortController() + const pending = handle.waitForExit(beforeStart.signal) + beforeStart.abort() + await expect(pending).resolves.toBe(false) + await expect(handle.waitForExit(AbortSignal.abort())).resolves.toBe(false) + fake.releaseStart() + await flush() + const live = new AbortController() + const liveWait = handle.waitForExit(live.signal) + live.abort() + await expect(liveWait).resolves.toBe(false) + fake.finish() + await handle.done + }) + + it('bounds both sides of the liveness-poll abort race', async () => { + const fake = new FakeSandbox() + const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/poll-abort') + await flush() + + const beforeTick = new AbortController() + fake.afterProbe = () => { beforeTick.abort(); fake.afterProbe = undefined } + await expect(handle.waitForExit(beforeTick.signal)).resolves.toBe(false) + + const duringTick = new AbortController() + fake.afterProbe = () => { + fake.afterProbe = undefined + setTimeout(() => { duringTick.abort() }, 0) + } + await expect(handle.waitForExit(duringTick.signal)).resolves.toBe(false) + + const duringProbe = new AbortController() + fake.beforeProbe = () => { duringProbe.abort(); fake.beforeProbe = undefined } + await expect(handle.waitForExit(duringProbe.signal)).resolves.toBe(false) + fake.finish() + await handle.done + }) + + it('observes a live group across one successful bounded poll', async () => { + const fake = new FakeSandbox() + const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/poll-success') + await flush() + setTimeout(() => { fake.finish() }, 1) + await expect(handle.waitForExit(new AbortController().signal)).resolves.toBe(true) + await handle.done + }) + + it('treats startup failure as no live tree and contains readiness rejection', async () => { + const fake = new FakeSandbox() + fake.backgroundError = new Error('start failed') + const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/fail') + await expect(handle.done).rejects.toThrow('start failed') + expect(handle.pid).toBe(-1) + await expect(handle.waitForExit()).resolves.toBe(true) + handle.terminate() + }) + + it('bounds a readiness rejection with a still-live caller signal', async () => { + const fake = new FakeSandbox() + fake.deferStart() + fake.backgroundError = new Error('start failed') + const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/fail-with-signal') + const waiting = handle.waitForExit(new AbortController().signal) + fake.releaseStart() + await expect(handle.done).rejects.toThrow('start failed') + await expect(waiting).resolves.toBe(true) + }) + + it('propagates an unavailable sandbox unless the caller aborts the wait', async () => { + const fake = new FakeSandbox() + let calls = 0 + const unavailable = runtime(fake, async () => { + calls += 1 + if (calls === 1) return fake.sandbox + throw new Error('connection unavailable') + }) + const handle = new E2BSubprocessHandle(unavailable, spec(), '/runtime/unavailable') + await flush() + await expect(handle.waitForExit()).rejects.toThrow('connection unavailable') + fake.finish() + await handle.done + }) + + it('returns false when the caller aborts while reconnecting for liveness', async () => { + const fake = new FakeSandbox() + const reconnect = Promise.withResolvers() + let calls = 0 + const unavailable = runtime(fake, async () => { + calls += 1 + return calls === 1 ? fake.sandbox : await reconnect.promise + }) + const handle = new E2BSubprocessHandle(unavailable, spec(), '/runtime/reconnect-abort') + await flush() + const controller = new AbortController() + const waiting = handle.waitForExit(controller.signal) + await flush() + controller.abort() + reconnect.reject(new Error('connection unavailable')) + await expect(waiting).resolves.toBe(false) + fake.finish() + await handle.done + }) + + it('returns false when a liveness request itself is aborted and surfaces other probe failures', async () => { + const fake = new FakeSandbox() + const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/probe') + await flush() + const controller = new AbortController() + controller.abort() + await expect(handle.waitForExit(controller.signal)).resolves.toBe(false) + fake.probeError = new Error('probe failed') + await expect(handle.waitForExit()).rejects.toThrow('probe failed') + fake.finish() + await handle.done + }) + + it('makes batch stdin close failures best-effort', async () => { + const fake = new FakeSandbox() + vi.spyOn(fake.handle, 'sendStdin').mockRejectedValueOnce(new Error('closed')) + const handle = new E2BSubprocessHandle(runtime(fake), spec({ + stdio: { stdin: { data: 'ignored' }, stdout: { maxBytes: 4 }, stderr: { maxBytes: 4 } }, + }), '/runtime/stdin-closed') + await flush() + fake.finish() + await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) + }) + + it('rejects malformed SDK process ids and non-command settlement failures', async () => { + const invalidPid = new FakeSandbox() + invalidPid.handle.pid = 0 + const invalid = new E2BSubprocessHandle(runtime(invalidPid), spec(), '/runtime/invalid-pid') + await expect(invalid.done).rejects.toThrow(/invalid command pid 0/) + await expect(invalid.waitForExit()).resolves.toBe(true) + + const crashedFake = new FakeSandbox() + const crashed = new E2BSubprocessHandle(runtime(crashedFake), spec(), '/runtime/crashed') + await flush() + crashedFake.alive = false + crashedFake.handle.crash(new Error('command transport failed')) + await expect(crashed.done).rejects.toThrow('command transport failed') + }) + + it('rejects invalid or absent process-group publication', async () => { + const invalidGroup = new FakeSandbox() + invalidGroup.processGroupId = 'not-a-pid\n' + const invalid = new E2BSubprocessHandle(runtime(invalidGroup), spec(), '/runtime/invalid-group') + await expect(invalid.done).rejects.toThrow(/invalid process-group id/) + + const absentGroup = new FakeSandbox() + absentGroup.processGroupId = '' + const absent = new E2BSubprocessHandle(runtime(absentGroup), spec(), '/runtime/absent-group') + await flush() + absentGroup.finish() + await expect(absent.done).rejects.toThrow(/exited before publishing/) + }) + + it('waits for delayed process-group publication', async () => { + const fake = new FakeSandbox() + fake.processGroupReads.push('', '4242\n') + const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/delayed-group') + await vi.waitFor(() => { expect(handle.pid).toBe(4242) }) + fake.finish() + await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) + }) + + it('handles output backpressure and contains a stderr sink failure', async () => { + const fake = new FakeSandbox() + const handle = new E2BSubprocessHandle(runtime(fake), spec({ + stdio: { stdin: 'ignore', stdout: 'pipe', stderr: 'pipe' }, + }), '/runtime/backpressure') + await flush() + + handle.stdout!.on('error', () => {}) + const stdoutWrite = vi.spyOn(handle.stdout!, 'write').mockReturnValueOnce(false) + const stdoutPending = fake.stdout('blocked') + queueMicrotask(() => { handle.stdout!.emit('drain') }) + await stdoutPending + stdoutWrite.mockRestore() + + handle.stderr!.on('error', () => {}) + const stderrWrite = vi.spyOn(handle.stderr!, 'write').mockReturnValueOnce(false) + const stderrPending = fake.stderr('broken') + queueMicrotask(() => { handle.stderr!.emit('error', new Error('sink failed')) }) + await stderrPending + stderrWrite.mockRestore() + + fake.finish() + await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) + }) + + it('contains a pipe callback failure instead of rejecting command settlement', async () => { + const fake = new FakeSandbox() + const handle = new E2BSubprocessHandle(runtime(fake), spec({ + stdio: { stdin: 'ignore', stdout: 'pipe', stderr: { maxBytes: 4 } }, + }), '/runtime/pipe-error') + await flush() + const emitted = once(handle.stdout!, 'error') + handle.stdout!.destroy(new Error('consumer failed')) + await emitted + await fake.stdout('late output') + fake.finish() + await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) + }) + + it('contains an already-gone group signal and observes non-command signal failures', async () => { + const gone = new FakeSandbox() + gone.trapsTerm = true + gone.signalError = commandError(1) + const goneHandle = new E2BSubprocessHandle(runtime(gone), spec({ graceMs: 1 }), '/runtime/gone-signal') + await flush() + goneHandle.terminate() + await expect(goneHandle.done).resolves.toEqual({ exitCode: null, signal: 'SIGKILL' }) + + const failed = new FakeSandbox() + failed.signalError = new Error('signal transport failed') + const failedHandle = new E2BSubprocessHandle(runtime(failed), spec(), '/runtime/failed-signal') + await flush() + failedHandle.terminate() + await flush() + failed.finish() + await expect(failedHandle.done).resolves.toEqual({ exitCode: 0, signal: null }) + }) +}) + +describe('E2BSubprocessService', () => { + async function service( + fake = new FakeSandbox(), + providedRuntime: E2BSandboxService = runtime(fake), + ): Promise<{ ctx: Context; fiber: Awaited> }> { + const ctx = new Context() + ctx.provide('e2b', providedRuntime) + const fiber = await ctx.plugin(E2BSubprocessService) + return { ctx, fiber } + } + + it('registers handles and disposal terminates and joins live remote groups regardless of sandbox policy', async () => { + const fake = new FakeSandbox() + fake.trapsTerm = true + const { ctx, fiber } = await service(fake) + const handle = ctx.subprocess.spawn(spec({ graceMs: 1 })) + await flush() + await fiber.dispose() + await expect(handle.done).resolves.toEqual({ exitCode: null, signal: 'SIGKILL' }) + expect(fake.alive).toBe(false) + }) + + it('releases naturally settled handles before later service disposal', async () => { + const fake = new FakeSandbox() + const { ctx, fiber } = await service(fake) + const handle = ctx.subprocess.spawn(spec()) + await flush() + fake.finish() + await handle.done + await flush() + const signalsBefore = fake.commandsSeen.filter(command => command.startsWith('kill -')).length + await fiber.dispose() + expect(fake.commandsSeen.filter(command => command.startsWith('kill -')).length).toBe(signalsBefore) + }) + + it('contains a release liveness failure and retries quiescence during disposal', async () => { + const fake = new FakeSandbox() + let calls = 0 + const reconnecting = runtime(fake, async () => { + calls += 1 + if (calls === 2) throw new Error('transient liveness failure') + return fake.sandbox + }) + const { ctx, fiber } = await service(fake, reconnecting) + const handle = ctx.subprocess.spawn(spec()) + await flush() + fake.finish() + await handle.done + await flush() + await fiber.dispose() + expect(calls).toBeGreaterThanOrEqual(3) + }) + + it('contains spawn rejection while disposal is joining the pending handle', async () => { + const fake = new FakeSandbox() + fake.deferStart() + fake.backgroundError = new Error('start failed during disposal') + const { ctx, fiber } = await service(fake) + const handle = ctx.subprocess.spawn(spec()) + const disposing = fiber.dispose() + fake.releaseStart() + await expect(disposing).resolves.toBeUndefined() + await expect(handle.done).rejects.toThrow('start failed during disposal') + }) + + it('validates synchronous spawn preconditions', async () => { + const { ctx } = await service() + expect(() => ctx.subprocess.spawn(spec({ argv: [] }))).toThrow(/non-empty program/) + expect(() => ctx.subprocess.spawn(spec({ graceMs: 0 }))).toThrow(/positive finite/) + expect(() => ctx.subprocess.spawn(spec({ signal: AbortSignal.abort('stop') }))).toThrow(/aborted before spawn/) + expect(() => ctx.subprocess.spawn(spec({ signal: { aborted: true, reason: undefined } as AbortSignal }))).toThrow(/aborted$/) + }) + + it('registers the package-owned empty invariant installer', async () => { + const ctx = new Context() + await ctx.plugin(InvariantService, { enabled: true }) + const fiber = await ctx.plugin(E2BSubprocessInvariant).await() + await fiber.dispose() + }) +}) diff --git a/packages/subprocess/subprocess-e2b/tsconfig.json b/packages/subprocess/subprocess-e2b/tsconfig.json new file mode 100644 index 0000000000..f3bb8c2a26 --- /dev/null +++ b/packages/subprocess/subprocess-e2b/tsconfig.json @@ -0,0 +1,25 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": ["src"], + "references": [ + { + "path": "../../../vendor/cosmokit" + }, + { + "path": "../../../vendor/cordis" + }, + { + "path": "../../e2b/e2b" + }, + { + "path": "../subprocess" + }, + { + "path": "../../support/invariants" + } + ] +} diff --git a/scripts/gen-cordis-catalog.ts b/scripts/gen-cordis-catalog.ts index 256ae90caa..b17cc5b4f8 100644 --- a/scripts/gen-cordis-catalog.ts +++ b/scripts/gen-cordis-catalog.ts @@ -301,6 +301,7 @@ export const TYPE_LINK_EXEMPTIONS: Readonly> = { PermissionSelect: 'permissions projection payload is owned by packages/ui/permission/src/types.ts', PromptAssembly: 'assembly result is owned by packages/core/system-prompt/README.md', ResumeAgentOptions: 'agent resume contract is owned by packages/core/agent/README.md', + Sandbox: 'external E2B SDK handle is owned by packages/e2b/e2b/README.md', SessionForkSource: 'service-local fork input is owned by packages/core/session/src/index.ts', SubagentRunEndInfo: 'event payload contract is owned by packages/subagent/subagent/src/types.ts', SubagentRunInfo: 'event payload contract is owned by packages/subagent/subagent/src/types.ts', diff --git a/scripts/verify-package-readme-model-experience.ts b/scripts/verify-package-readme-model-experience.ts index 3ca0317cf4..90aa22e322 100644 --- a/scripts/verify-package-readme-model-experience.ts +++ b/scripts/verify-package-readme-model-experience.ts @@ -50,6 +50,7 @@ const SENTENCE_MODEL_EXPERIENCE: Readonly> = { 'packages/code-runtime/code-runtime-worker': { kind: 'indirect', reason: 'The worker backend delegates model rendering to Code Mode in dsh-tools.' }, 'packages/typert/registry': { kind: 'none', reason: 'Runtime type registry; consumers (cordis_inspect, wire faces, gates) own any model-visible projection of registry contents.' }, 'packages/typert/loader': { kind: 'none', reason: 'Loader integration only registers generated artifacts; consumers own any model-visible projection.' }, + 'packages/e2b/e2b': { kind: 'none', reason: 'The shared remote-runtime owner registers no model context; provider adapters and consumers own rendered effects.' }, 'packages/client/hmr': { kind: 'none', reason: 'Browser-side UI plugin layer; registers no model surface.' }, 'packages/client/modules': { kind: 'none', reason: 'Browser-side module-loading kernel machinery; registers no model surface.' }, 'packages/client/test-runtime': { kind: 'none', reason: 'Browser-side test infrastructure (jsdom bench); registers no model surface.' }, @@ -82,6 +83,7 @@ const SENTENCE_MODEL_EXPERIENCE: Readonly> = { 'packages/client/web': { kind: 'none', reason: 'Browser-side UI plugin layer; registers no model surface.' }, 'packages/examples/agent-spine-demo': { kind: 'indirect', reason: 'The bundle only mounts model-facing child plugins.' }, 'packages/fs/fs': { kind: 'indirect', reason: 'The service interface delegates model rendering to dsh-tool-fs.' }, + 'packages/fs/fs-e2b': { kind: 'indirect', reason: 'The provider backend delegates model rendering to dsh-tool-fs.' }, 'packages/fs/fs-local': { kind: 'indirect', reason: 'The provider backend delegates model rendering to dsh-tool-fs.' }, 'packages/hooks/hook-protocol': { kind: 'indirect', reason: 'Only the hook bridge plugins render decoded hook output to a model.' }, 'packages/host/apiproxy': { kind: 'none', reason: 'The wire contract and fetch carriers move already-composed messages and register no model surface.' }, @@ -98,6 +100,7 @@ const SENTENCE_MODEL_EXPERIENCE: Readonly> = { 'packages/lsp/lsp': { kind: 'indirect', reason: 'The provider registry delegates model rendering to dsh-tool-lsp.' }, 'packages/lsp/lsp-local': { kind: 'indirect', reason: 'The provider backend delegates model rendering to dsh-tool-lsp.' }, 'packages/subprocess/subprocess': { kind: 'indirect', reason: 'The seam delegates all model rendering to consumer seams such as the bash executor family.' }, + 'packages/subprocess/subprocess-e2b': { kind: 'indirect', reason: 'The remote spawn backend delegates model rendering to consumer seams such as the bash executor family.' }, 'packages/subprocess/subprocess-local': { kind: 'indirect', reason: 'The spawn backend delegates model rendering to consumer seams such as the bash executor family.' }, 'packages/sandbox/sandbox-local': { kind: 'indirect', reason: 'The provider backend delegates model rendering to dsh-bash-sandbox and dsh-tool-bash.' }, 'packages/sdk/create-sdk': { kind: 'indirect', reason: 'The initializer only writes project files; selected runtime plugins provide the generated project model surface.' }, diff --git a/tsconfig.base.json b/tsconfig.base.json index de9b63459c..8380182e83 100644 --- a/tsconfig.base.json +++ b/tsconfig.base.json @@ -188,6 +188,7 @@ "./packages/bash/*/src", "./packages/pty/*/src", "./packages/subprocess/*/src", + "./packages/e2b/*/src", "./packages/code-runtime/*/src", "./packages/fs/*/src", "./packages/lsp/*/src", diff --git a/tsconfig.host.json b/tsconfig.host.json index 1a8ef6508a..03b1bcd2dd 100644 --- a/tsconfig.host.json +++ b/tsconfig.host.json @@ -162,6 +162,8 @@ { "path": "./packages/examples/agent-spine-demo" }, { "path": "./packages/subprocess/subprocess" }, { "path": "./packages/subprocess/subprocess-local" }, + { "path": "./packages/e2b/e2b" }, + { "path": "./packages/subprocess/subprocess-e2b" }, { "path": "./packages/bash/bash" }, { "path": "./packages/pty/pty" }, { "path": "./packages/pty/pty-local" }, @@ -183,6 +185,7 @@ { "path": "./packages/bash/tool-bash" }, { "path": "./packages/fs/fs" }, { "path": "./packages/fs/fs-local" }, + { "path": "./packages/fs/fs-e2b" }, { "path": "./packages/fs/fs-policy" }, { "path": "./packages/fs/fs-sandbox" }, { "path": "./packages/fs/tool-fs" },