From b0eb7f8f35e385ad32b0a8af6e407de8f859c65f Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Sat, 8 Aug 2026 01:23:26 +0800 Subject: [PATCH 001/108] docs: define unified GitHub label taxonomy --- ...07-25-semantic-pr-label-taxonomy.i18n.yaml | 4 +- .../2026-07-25-semantic-pr-label-taxonomy.md | 65 +++++++------------ ...026-07-25-semantic-pr-label-taxonomy.zh.md | 65 +++++++------------ AGENTS.md | 2 +- 4 files changed, 53 insertions(+), 83 deletions(-) diff --git a/.agents/notes/implemented/process/2026-07-25-semantic-pr-label-taxonomy.i18n.yaml b/.agents/notes/implemented/process/2026-07-25-semantic-pr-label-taxonomy.i18n.yaml index 1aaa9f992b..a83236521c 100644 --- a/.agents/notes/implemented/process/2026-07-25-semantic-pr-label-taxonomy.i18n.yaml +++ b/.agents/notes/implemented/process/2026-07-25-semantic-pr-label-taxonomy.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/process/2026-07-25-semantic-pr-label-taxonomy.md -2026-07-25-semantic-pr-label-taxonomy.md: 3217b405e968d4d2c1eba1f1a5a08008b18ba514 -2026-07-25-semantic-pr-label-taxonomy.zh.md: 978f11af9402f248679e2087ff7fb513321b69b9 +2026-07-25-semantic-pr-label-taxonomy.md: 6aef5ecbb80005d1fcd90b4ee673bbbebb92cba6 +2026-07-25-semantic-pr-label-taxonomy.zh.md: 0bd3fbcaeca37fa894e92ce5a4e78c4125056ac9 diff --git a/.agents/notes/implemented/process/2026-07-25-semantic-pr-label-taxonomy.md b/.agents/notes/implemented/process/2026-07-25-semantic-pr-label-taxonomy.md index 3217b405e9..6aef5ecbb8 100644 --- a/.agents/notes/implemented/process/2026-07-25-semantic-pr-label-taxonomy.md +++ b/.agents/notes/implemented/process/2026-07-25-semantic-pr-label-taxonomy.md @@ -6,66 +6,51 @@ English | [中文](2026-07-25-semantic-pr-label-taxonomy.zh.md) ## Problem -Pull requests need two different signals: what kind of change they make and which repository domains they affect. A flat or broadly named label set conflates those questions, hides work in distinct areas such as `session` and `llm`, and gives reviewers and automation weak inputs. +Pull request labels answer two independent questions: what kind of change the work makes and which durable repository domains it materially affects. Mixing those dimensions or keeping synonymous labels makes queries ambiguous, while a closed area inventory forces new domains into inaccurate categories. -The repository also gains new domains over time. Treating today's area labels as a closed set would force future work into inaccurate labels or a generic catch-all. +Issues already have a native Type and a separate source taxonomy. Reusing pull request kind or source labels across both object types duplicates metadata and weakens the meaning of each label family. ## Decision -Every open or merged pull request carries exactly one kind and every materially affected area. Closed pull requests that were never merged are outside the maintained historical corpus. Other operational labels may coexist, but they do not satisfy either dimension. +Every open or merged pull request carries exactly one canonical `kind/*` label and at least one materially affected `area/*` label. Closed pull requests that were never merged retain migrated historical assignments but do not receive invented missing classification. Operational labels may coexist without satisfying either dimension. ### Kinds +The kind set is closed and mutually exclusive: + | Kind | Meaning | |---|---| -| `feature` | Adds or intentionally changes behavior. | -| `bug-fix` | Corrects incorrect behavior. | -| `doc` | Makes documentation the dominant intent. | -| `testing` | Changes tests or testing infrastructure without changing product behavior. | -| `cleanup` | Preserves behavior while maintaining or simplifying the implementation or repository process. | +| `kind/feature` | Adds or intentionally changes behavior. | +| `kind/bug-fix` | Corrects incorrect behavior. | +| `kind/doc` | Makes documentation the dominant intent. | +| `kind/testing` | Changes tests or testing infrastructure without changing product behavior. | +| `kind/cleanup` | Preserves behavior while maintaining or simplifying implementation or repository process. | +| `kind/dependency` | Updates dependencies without another dominant intent. | -The kind records the change's dominant intent: accompanying tests and documentation do not turn a feature or bug fix into a testing or documentation change. +The kind records the dominant intent. Accompanying tests, documentation, cleanup, or dependency movement do not override a feature or bug fix. -Areas record semantic repository domains rather than temporary initiatives, ownership, or every path touched incidentally. Area labels are not a hierarchy: a pull request may carry several when it changes distinct contracts, but an umbrella and a narrower label do not both describe the same work. +### Areas -### Current areas +Areas name durable semantic domains rather than temporary initiatives, ownership, or every path touched incidentally. A pull request carries multiple areas when it changes distinct contracts, but it does not combine an umbrella and a narrower label for the same contract. GitHub's live `area/*` names and descriptions own the current inventory. -The 46 current areas are listed below. The group names organize the list for readability; they are not labels or another taxonomy level. +The area set is intentionally extensible. When no existing description honestly covers a durable and reusable repository domain, an agent is empowered to create a concise `area/` label without separate approval. The agent must not create an area for one pull request, an incidental path, a temporary project, a status, or a person or team, and must report the new label after applying it. Reusing an inaccurate area merely to avoid a justified addition is not acceptable. -| Group | Areas | -|---|---| -| Agent and model | `agent`, `agent-loop`, `session`, `llm`, `model-context`, `compaction`, `tools`, `persistence` | -| Orchestration | `subagent`, `workflow`, `planning`, `tasks`, `schedule`, `telemetry`, `storage`, `workspace` | -| Capabilities | `bash`, `subprocess`, `pty`, `filesystem`, `lsp`, `skills`, `web-search`, `code-mode`, `artifact`, `attachment`, `sandbox`, `mcp`, `hooks`, `cordis` | -| Interfaces | `ui`, `gui`, `tui`, `acp`, `json-rpc`, `cli`, `python-sdk`, `vscode`, `website` | -| Repository and release | `dev-infra`, `ci`, `build`, `dependencies`, `platform`, `i18n`, `release` | +Kinds are not extended this way. A new kind changes the mutually exclusive classification contract and requires an explicit taxonomy change with corresponding policy enforcement. -`gui` covers browser and Electron graphical applications, including standalone graphical developer tools; `vscode` remains the editor extension integration. `ui` covers shared cross-interface commands, approval interaction, presentation, and app boot; it coexists with `gui`, `tui`, or a protocol area only when the pull request also changes that shared contract. +### Issues and operational labels -`tasks` owns background work tied to a running process, while `schedule` owns durable time-triggered jobs. `tools` owns generic registry, schema, and execution contracts; a concrete capability receives `tools` only when it changes one of those contracts. `attachment` owns durable media references and multimodal input delivery, while `artifact` owns model-declared deliverable identity and preview lifecycle; neither borrows `tools` or `ui` for its implementation parts. +Issues use native Issue Type instead of `kind/*`; their `area/*` labels remain optional. `source/*` labels record Issue provenance and do not apply to pull requests. Priority, GitHub defaults, and workflow triggers remain independent operational metadata. -Names follow semantic ownership rather than lexical resemblance. `hooks` means the Claude Code and Codex agent bridges, not local Git hooks; `platform` means product portability, not CI runner selection; and `build` means compilation, bundling, and built package artifacts, not documentation generators. - -### Extensibility - -The area set is intentionally extensible. Add an area when a recurring, meaningful repository domain is missing; do not add a label for one pull request, a temporary project, a status, or a person or team. Rename, split, or retire an area when the domain model changes, and update this list and the affected open and merged pull requests together. - -The kind set stays narrow because kinds are mutually exclusive. A new kind requires a distinct change intent that cannot be represented by the current five; it is not a substitute for an area. +Label migrations preserve meaning before removing aliases: add the canonical replacement, verify the labelable, then remove the obsolete assignment. A label is deleted only after no pull request or Issue still uses it, and unrelated labels are never replaced as a set. ## Alternatives considered -- **One undifferentiated label set.** Rejected because kind and area answer different questions; mixing them makes the presence of one label say nothing about whether the other dimension was considered. -- **A fixed, closed area set.** Rejected because repository domains evolve. A closed set would preserve spelling at the cost of semantic accuracy. -- **One broad `core` area or package-derived labels.** Rejected because domains such as `session`, `llm`, and `agent` remain independently meaningful across package boundaries, while incidental file paths are not the scope reviewers or automation need. -- **Separate browser and desktop areas.** Rejected because browser delivery and Electron packaging expose one graphical client domain; splitting them classifies the delivery shell rather than the semantic work. -- **Broad implementation areas in place of a domain.** Rejected because a durable scheduled job is not a background task, an attachment is not merely its source interface or filesystem implementation, and an artifact is not merely its declaring tool or preview interface. -- **Umbrella and leaf areas for the same contract.** Rejected because duplicate labels inflate scope without adding information. Multiple areas remain correct when a pull request changes genuinely distinct contracts. -- **Exactly one area per pull request.** Rejected because coherent changes can legitimately span several domains, and dropping secondary areas hides affected contracts. +- **Unprefixed labels.** Rejected because a flat name does not identify whether it classifies intent, domain, source, priority, or automation, and synonymous plain and prefixed labels caused ambiguous queries. +- **A fixed area allowlist in repository policy.** Rejected because durable repository domains evolve. The `area/*` namespace remains mechanically recognizable while live descriptions carry the extensible inventory. +- **Kinds on Issues.** Rejected because native Issue Type already owns that classification; duplicating it as a label creates drift. +- **Automatic areas from paths.** Rejected because areas describe semantic impact across package boundaries, while changed paths include incidental tests, documentation, and support files. +- **Exactly one area per pull request.** Rejected because coherent changes can materially affect several independent contracts. ## Consequences -- Reviewers and automation receive one stable intent signal plus a complete semantic scope. -- `gui` queries cover browser and desktop delivery together, while `ui` queries retain only shared cross-interface contracts. -- `schedule`, `attachment`, and `artifact` queries identify those domains directly instead of approximating them through implementation dependencies. -- Selecting labels remains a judgment call: paths and title prefixes can suggest areas, but they cannot replace reading the change. -- Taxonomy changes carry maintenance work. Area additions, renames, splits, and removals update this decision record and backfill open and merged pull requests so historical queries keep their meaning. +Reviewers and automation can query intent, semantic scope, provenance, priority, and operational triggers independently. Maintainers must read the change and live label descriptions rather than infer classification from title prefixes or paths, and taxonomy migrations carry an explicit historical backfill and verification cost. diff --git a/.agents/notes/implemented/process/2026-07-25-semantic-pr-label-taxonomy.zh.md b/.agents/notes/implemented/process/2026-07-25-semantic-pr-label-taxonomy.zh.md index 978f11af94..0bd3fbcaec 100644 --- a/.agents/notes/implemented/process/2026-07-25-semantic-pr-label-taxonomy.zh.md +++ b/.agents/notes/implemented/process/2026-07-25-semantic-pr-label-taxonomy.zh.md @@ -6,66 +6,51 @@ Status: implemented ## 问题 -PR(Pull Request)需要传达两个不同的信号:它带来哪一类变更,以及会影响仓库中的哪些领域。一套扁平或命名宽泛的标签会混淆这两个问题,掩盖 `session`、`llm` 等不同领域的工作,也让评审人和自动化流程得到的输入缺乏有效信息。 +PR(Pull Request)标签回答两个相互独立的问题:工作带来哪一类变更,以及会对哪些持久的仓库领域产生实质影响。混用这两个维度或保留同义标签会使查询含义模糊,而封闭的领域清单会迫使新领域被归入不准确的类别。 -仓库还会随时间发展出新的领域。如果把当前的领域标签视为封闭集合,未来的工作就只能归入不准确的标签或通用兜底标签。 +Issue 已有原生 Issue Type 和独立的来源分类体系。在这两类对象上复用 PR 类型或来源标签会产生重复元数据,并削弱每个标签族的含义。 ## 决策 -每项开放或已合并的 PR 都带有恰好一个类型标签,以及所有受到实质影响的领域标签。未合并即关闭的 PR 不属于持续维护的历史记录集合。其他管理用途的标签可以并存,但都不能满足这两个维度中的任一个。 +每项开放或已合并的 PR 都带有恰好一个规范的 `kind/*` 标签,以及至少一个受到实质影响的 `area/*` 标签。未合并即关闭的 PR 保留经迁移的历史标签关系,但不会凭空补充缺失分类。管理用途的标签可以并存,但不能满足这两个维度中的任一个。 ### 类型 +类型集合封闭且互斥: + | 类型 | 含义 | |---|---| -| `feature` | 新增行为或有意改变行为。 | -| `bug-fix` | 修正错误行为。 | -| `doc` | 以文档变更为主要意图。 | -| `testing` | 修改测试或测试基础设施,但不改变产品行为。 | -| `cleanup` | 在保持行为不变的前提下,维护或简化实现或仓库流程。 | +| `kind/feature` | 新增行为或有意改变行为。 | +| `kind/bug-fix` | 修正错误行为。 | +| `kind/doc` | 以文档变更为主要意图。 | +| `kind/testing` | 修改测试或测试基础设施,但不改变产品行为。 | +| `kind/cleanup` | 在保持行为不变的前提下,维护或简化实现或仓库流程。 | +| `kind/dependency` | 在没有其他主要意图时更新依赖。 | -类型记录变更的主要意图:配套测试与文档并不会把一项功能或缺陷修复变成测试或文档变更。 +类型记录主要意图。配套测试、文档、清理或依赖调整不会覆盖一项功能或缺陷修复的主要意图。 -领域记录仓库中的语义领域,而不是临时项目、归属关系或偶然触及的每条路径。领域标签不构成层级:一项 PR 修改不同契约时可以带有多个领域标签,但不能用一个总括标签和一个较窄标签重复描述同一项工作。 +### 领域 -### 当前领域 +领域表示持久的语义领域,而不是临时项目、归属关系或偶然触及的每条路径。一项 PR 修改不同契约时带有多个领域标签,但不会用一个总括标签和一个较窄标签重复描述同一项契约。当前分类清单以 GitHub 上实时的 `area/*` 名称和描述为准。 -当前的 46 个领域如下。分组名称仅用于提高列表的可读性;它们既不是标签,也不是分类体系中的另一个层级。 +领域集合有意保持可扩展。当现有描述都无法准确涵盖一个持久且可复用的仓库领域时,agent(智能体)有权无须另行批准,直接创建简洁的 `area/` 标签。agent 不得为单个 PR、偶然涉及的路径、临时项目、状态、个人或团队创建领域,并且必须在应用新标签后报告。仅为避免新增一个确有必要的标签而复用不准确的领域,不可接受。 -| 分组 | 领域 | -|---|---| -| agent(智能体)与模型 | `agent`, `agent-loop`, `session`, `llm`, `model-context`, `compaction`, `tools`, `persistence` | -| 编排 | `subagent`, `workflow`, `planning`, `tasks`, `schedule`, `telemetry`, `storage`, `workspace` | -| 能力 | `bash`, `subprocess`, `pty`, `filesystem`, `lsp`, `skills`, `web-search`, `code-mode`, `artifact`, `attachment`, `sandbox`, `mcp`, `hooks`, `cordis` | -| 接口 | `ui`, `gui`, `tui`, `acp`, `json-rpc`, `cli`, `python-sdk`, `vscode`, `website` | -| 仓库与发布 | `dev-infra`, `ci`, `build`, `dependencies`, `platform`, `i18n`, `release` | +类型不以这种方式扩展。新增类型会改变互斥的分类契约,必须明确修改分类体系并同步调整策略强制校验。 -`gui` 涵盖浏览器和 Electron 图形应用,包括独立的图形化开发者工具;`vscode` 仍表示编辑器扩展集成。`ui` 涵盖共享的跨接口命令、审批交互、呈现和应用启动;只有当 PR 还修改这项共享契约时,它才与 `gui`、`tui` 或某个协议领域并用。 +### Issue 与管理用途标签 -`tasks` 负责与运行中进程绑定的后台工作,`schedule` 则负责持久化的定时作业。`tools` 负责通用的注册表契约、schema 契约和执行契约;具体能力只有在修改其中一项契约时才带有 `tools`。`attachment` 负责持久化的媒体引用和多模态输入传递,`artifact` 则负责模型声明的交付物标识和预览生命周期;二者都不会因实现包含工具或界面部分而借用 `tools` 或 `ui`。 +Issue 使用原生 Issue Type,而不是 `kind/*`;其 `area/*` 标签仍为可选。`source/*` 标签记录 Issue 来源,不适用于 PR。优先级、GitHub 默认标签和工作流触发器仍是相互独立的管理元数据。 -标签名称以语义归属为准,而不是词面相似性。`hooks` 指 Claude Code 和 Codex 的 agent 桥接,而不是本地 Git 钩子;`platform` 指产品可移植性,而不是 CI 运行器选择;`build` 指编译、打包和已构建的包产物,而不是文档生成器。 - -### 可扩展性 - -领域集合有意保持可扩展。当分类体系缺少一个会反复涉及且具有实际意义的仓库领域时,就新增领域;不要仅为一项 PR、临时项目、状态、个人或团队新增标签。当领域模型发生变化时,重命名、拆分或退役相应领域,同时更新本列表以及所有受影响的开放和已合并 PR。 - -类型集合保持精简,因为各类型互斥。新增类型的前提是存在一种当前五类无法表达的独立变更意图;类型不能用来替代领域。 +迁移标签时,须先保留语义,再移除别名:先添加规范替代标签并验证对象,再移除废弃的标签关系。只有在所有 PR 和 Issue 都不再使用某个标签后才能将其删除,且绝不整组替换无关标签。 ## 曾考虑的替代方案 -- **一套不区分维度的标签。** 不予采纳,因为类型与领域回答的是不同问题;两者混在一起时,存在一个维度的标签并不表示另一个维度也经过了考虑。 -- **一套固定、封闭的领域集合。** 不予采纳,因为仓库领域会持续演变。封闭集合会以牺牲语义准确性为代价来维持拼写不变。 -- **一个宽泛的 `core` 领域,或从包结构派生的标签。** 不予采纳,因为 `session`、`llm` 和 `agent` 等领域在跨越包边界时仍各自具有意义,而偶然涉及的文件路径并不是评审人或自动化流程所需的范围信息。 -- **为浏览器和桌面端分别设置领域。** 不予采纳,因为浏览器交付和 Electron 打包共同呈现同一个图形客户端领域;拆开二者将按交付形态而非工作的语义进行分类。 -- **以宽泛的实现领域替代语义领域。** 不予采纳,因为持久化的定时作业不是后台任务,附件不只是其来源接口或文件系统实现,产物也不只是声明它的工具或预览接口。 -- **同一项契约同时使用总括领域与细分领域。** 不予采纳,因为重复标签只会虚增范围,不会增加信息。一项 PR 确实修改不同契约时,多个领域标签仍然合理。 -- **每项 PR 恰好一个领域。** 不予采纳,因为一项内聚的变更可以合理地跨越多个领域;省略次要领域会隐藏受影响的契约。 +- **无前缀标签。** 不予采纳,因为扁平名称无法表明其表示的是意图、领域、来源、优先级还是自动化用途,而同义的无前缀和带前缀标签会使查询含义模糊。 +- **在仓库策略中使用固定的领域允许列表。** 不予采纳,因为持久的仓库领域会持续演变。`area/*` 命名空间仍可由机制识别,而实时描述负责承载可扩展的分类清单。 +- **在 Issue 上使用类型标签。** 不予采纳,因为原生 Issue Type 已负责这项分类;再用标签重复表示会产生漂移。 +- **从路径自动生成领域。** 不予采纳,因为领域描述跨越包边界的语义影响,而变更路径还会包含偶然涉及的测试、文档和支持文件。 +- **每项 PR 恰好一个领域。** 不予采纳,因为一项内聚的变更可以对多个独立契约产生实质影响。 ## 后果 -- 评审人和自动化流程获得一个稳定的意图信号,以及完整的语义范围。 -- `gui` 查询会同时覆盖浏览器与桌面端交付,`ui` 查询则只涵盖共享的跨接口契约。 -- `schedule`、`attachment` 与 `artifact` 查询直接对应各自领域,无需通过实现依赖近似归类。 -- 选择标签仍然需要判断:路径和标题前缀可以提示领域,但不能替代阅读变更内容。 -- 变更分类体系会产生维护工作。新增、重命名、拆分或移除领域时,需要更新本决策记录,并回填开放和已合并的 PR,使历史查询保持原有含义。 +评审人和自动化流程可以分别查询意图、语义范围、来源、优先级和工作流触发器。维护者必须阅读变更内容和实时标签描述,而不能根据标题前缀或路径推断分类;迁移分类体系还会产生明确的历史回填与验证成本。 diff --git a/AGENTS.md b/AGENTS.md index 64f2e944a8..90b2ab4a67 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -118,7 +118,7 @@ Real-API tests and demos read `DEEPSEEK_API_KEY`, optional `DEEPSEEK_BASE_URL`, - **A tool's UI render intent is part of its design**, decided up front (`generic`/`terminal`/`diff`, `locations`); presentation methods are pure functions of `args` ([cookbook](docs/cookbook/adding-a-tool.md)). - **Plan unit, e2e, and snapshot coverage** for new seams, lifecycle shapes, and transcript surfaces; missing snapshot-harness support is part of the implementation, not deferred follow-up. - **Choose PR history deliberately.** Split independent changes; fix the introducing PR before propagation. Standalone PRs and official stacks may merge-forward or rebase after review. Rewrites use `--force-with-lease`, abort on remote movement, never raw `--force`; an in-progress merge-forward preserves its checkpoint before taking a newer base ([rationale](.agents/notes/implemented/process/2026-08-02-native-github-stacks-and-optional-rebases.md)). -- **Label PRs:** one kind (`feature`/`bug-fix`/`doc`/`testing`/`cleanup`), each matching area; the [taxonomy](.agents/notes/implemented/process/2026-07-25-semantic-pr-label-taxonomy.md) is extensible. +- **Labels:** one PR `kind/*`, all material `area/*`, and native Issue Type ([workflow](.agents/skills/dsh-labeling/SKILL.md)). - TODO markers: `FIXME`/`TODO`/`XXX` by urgency ([semantics](docs/development.md)). - Files end with exactly one trailing newline; `git diff --cached --check` (pre-commit) gates it. From a9abb62195f438c61b163fa2e9b8133b1523e4a9 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Sat, 8 Aug 2026 01:23:36 +0800 Subject: [PATCH 002/108] chore: enforce unified GitHub labels --- .github/dependabot.yml | 6 +- .github/issue-management/policy.mjs | 43 +++++++++- .github/issue-management/policy.test.mjs | 100 ++++++++++++++++++++--- AGENTS.md | 2 +- 4 files changed, 132 insertions(+), 19 deletions(-) diff --git a/.github/dependabot.yml b/.github/dependabot.yml index 524d7912e2..dd91db8bf1 100644 --- a/.github/dependabot.yml +++ b/.github/dependabot.yml @@ -13,7 +13,7 @@ updates: cooldown: default-days: 30 labels: - - "cleanup" + - "kind/dependency" - "area/infra" - package-ecosystem: "uv" @@ -25,7 +25,7 @@ updates: cooldown: default-days: 30 labels: - - "cleanup" + - "kind/dependency" - "area/infra" - package-ecosystem: "github-actions" @@ -37,5 +37,5 @@ updates: cooldown: default-days: 30 labels: - - "cleanup" + - "kind/dependency" - "area/infra" diff --git a/.github/issue-management/policy.mjs b/.github/issue-management/policy.mjs index 608291c4f8..18a9b03628 100644 --- a/.github/issue-management/policy.mjs +++ b/.github/issue-management/policy.mjs @@ -12,6 +12,27 @@ const AUDIT_MARKER = '' const OWNER_LINE = /^Owner: @([A-Za-z0-9](?:[A-Za-z0-9-]{0,37}[A-Za-z0-9])?)$/ const TYPES = new Set(['Idea', 'Feature', 'Bug', 'Research', 'Task']) const PRIORITIES = ['p0', 'p1', 'p2', 'p3'] +const PR_KINDS = new Set([ + 'kind/feature', + 'kind/bug-fix', + 'kind/doc', + 'kind/testing', + 'kind/cleanup', + 'kind/dependency', +]) +const LEGACY_LABELS = new Set([ + 'kind/bug', + 'kind/documentation', + 'bug-fix', + 'doc', + 'cleanup', + 'testing', + 'dependencies', + 'ci', + 'cli', + 'llm', + 'web-search', +]) const TERMINAL_STATUSES = new Set(['Done', 'No action']) const ACTIVE_STATUS_ORDER = config.statuses.filter((status) => !TERMINAL_STATUSES.has(status)) @@ -224,8 +245,14 @@ export function retainIssueReferences(references, issues) { export function validateIssue(issue) { const errors = validateBody(issue) const status = issue.status + const invalidLabels = issue.labels.filter( + (label) => label.startsWith('kind/') || LEGACY_LABELS.has(label), + ) if (!/\p{Script=Han}/u.test(issue.title)) errors.push('Issue 标题必须包含中文') + if (invalidLabels.length > 0) { + errors.push(`Issue 不得使用 PR kind 或旧版标签:${invalidLabels.join(', ')}`) + } if ( /^\s*(?:\[(?:Idea|Feature|Bug|Research|Task|P[0-3]|Inbox|Backlog|Ready|In progress|In review|Done|No action|Owner|area\/[^\]]+)[^\]]*\]|(?:Idea|Feature|Bug|Research|Task|P[0-3]|Inbox|Backlog|Ready|In progress|In review|Done|No action|Owner|area\/[^:: ]+)\s*[::-])/iu.test( issue.title, @@ -261,12 +288,24 @@ export function validateIssue(issue) { export function validatePullRequest(input) { if (!requiresPullRequestPolicy(input)) return [] const errors = [] - const kinds = input.labels.filter((label) => label.startsWith('kind/')) + const kinds = input.labels.filter((label) => PR_KINDS.has(label)) + const unknownKinds = input.labels.filter( + (label) => label.startsWith('kind/') && !PR_KINDS.has(label) && !LEGACY_LABELS.has(label), + ) + const legacyLabels = input.labels.filter((label) => LEGACY_LABELS.has(label)) + const sourceLabels = input.labels.filter((label) => label.startsWith('source/')) const priorities = input.labels.filter((label) => PRIORITIES.includes(label)) const areas = input.labels.filter((label) => label.startsWith('area/')) if (input.references.all.length === 0) errors.push('PR 正文必须引用至少一个同仓库 Issue') - if (kinds.length !== 1) errors.push(`PR 必须恰好有一个 kind/*,当前为 ${kinds.length}`) + if (kinds.length !== 1) { + errors.push(`PR 必须恰好有一个允许的 kind/*,当前为 ${kinds.length}`) + } + if (unknownKinds.length > 0) { + errors.push(`PR 含不支持的 kind/*:${unknownKinds.join(', ')}`) + } + if (legacyLabels.length > 0) errors.push(`PR 含旧版标签:${legacyLabels.join(', ')}`) + if (sourceLabels.length > 0) errors.push(`source/* 仅用于 Issue:${sourceLabels.join(', ')}`) if (priorities.length > 1) errors.push(`PR 最多有一个 p0–p3,当前为 ${priorities.length}`) if (areas.length === 0) errors.push('PR 必须至少有一个 area/*') for (const number of input.references.all) { diff --git a/.github/issue-management/policy.test.mjs b/.github/issue-management/policy.test.mjs index 86750127a7..bcaf450afa 100644 --- a/.github/issue-management/policy.test.mjs +++ b/.github/issue-management/policy.test.mjs @@ -27,6 +27,25 @@ const legalIssue = { stateReason: null, } +const canonicalKinds = [ + 'kind/feature', + 'kind/bug-fix', + 'kind/doc', + 'kind/testing', + 'kind/cleanup', + 'kind/dependency', +] + +const reviewedPull = (labels) => ({ + isDraft: false, + authorType: 'User', + reviewRequestCount: 1, + reviewCount: 0, + labels, + references: { all: [2], resolving: [], related: [2] }, + issues: new Map([[2, { priority: null }]]), +}) + test('counts only text outside details', () => { assert.deepEqual(countVisibleUnits('支持 GitHub Project。
隐藏文字
'), { units: 4, @@ -92,6 +111,32 @@ test('rejects metadata prefixes in an Issue title', () => { assert.ok(errors.includes('Issue 标题不得带 Type、Priority、Status、area 或 Owner 前缀')) }) +test('reserves PR kind and legacy labels for pull requests', () => { + for (const label of [ + ...canonicalKinds, + 'kind/experimental', + 'kind/bug', + 'kind/documentation', + 'bug-fix', + 'doc', + 'cleanup', + 'testing', + 'dependencies', + 'ci', + 'cli', + 'llm', + 'web-search', + ]) { + assert.ok( + validateIssue({ ...legalIssue, labels: [label] }).some((error) => + error.startsWith('Issue 不得使用 PR kind 或旧版标签:'), + ), + label, + ) + } + assert.deepEqual(validateIssue({ ...legalIssue, labels: ['area/web', 'source/member'] }), []) +}) + test('keeps terminal Status aligned with the native close reason', () => { assert.deepEqual( validateIssue({ ...legalIssue, status: 'Done', state: 'closed', stateReason: 'completed' }), @@ -260,22 +305,51 @@ test('requires repository PR labels in the enforcement scope', () => { references: { all: [2], resolving: [], related: [2] }, issues: new Map([[2, { priority: null }]]), }) - assert.ok(errors.includes('PR 必须恰好有一个 kind/*,当前为 0')) + assert.ok(errors.includes('PR 必须恰好有一个允许的 kind/*,当前为 0')) assert.ok(errors.includes('PR 必须至少有一个 area/*')) }) -test('accepts repository-extensible kind labels', () => { - assert.deepEqual( - validatePullRequest({ - isDraft: false, - authorType: 'User', - reviewRequestCount: 1, - reviewCount: 0, - labels: ['kind/dependency', 'area/infra'], - references: { all: [2], resolving: [], related: [2] }, - issues: new Map([[2, { priority: null }]]), - }), - [], +test('accepts exactly the canonical kinds with extensible areas', () => { + for (const kind of canonicalKinds) { + assert.deepEqual(validatePullRequest(reviewedPull([kind, 'area/future-domain'])), [], kind) + } +}) + +test('rejects multiple, unknown, legacy, and Issue-source PR labels', () => { + assert.ok( + validatePullRequest( + reviewedPull(['kind/feature', 'kind/doc', 'area/web']), + ).includes('PR 必须恰好有一个允许的 kind/*,当前为 2'), + ) + assert.ok( + validatePullRequest(reviewedPull(['kind/experimental', 'area/web'])).includes( + 'PR 含不支持的 kind/*:kind/experimental', + ), + ) + for (const label of [ + 'kind/bug', + 'kind/documentation', + 'bug-fix', + 'doc', + 'cleanup', + 'testing', + 'dependencies', + 'ci', + 'cli', + 'llm', + 'web-search', + ]) { + assert.ok( + validatePullRequest(reviewedPull(['kind/feature', 'area/web', label])).some((error) => + error.startsWith('PR 含旧版标签:'), + ), + label, + ) + } + assert.ok( + validatePullRequest( + reviewedPull(['kind/feature', 'area/web', 'source/internal-pr']), + ).includes('source/* 仅用于 Issue:source/internal-pr'), ) }) diff --git a/AGENTS.md b/AGENTS.md index 90b2ab4a67..9344e2e6e2 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -118,7 +118,7 @@ Real-API tests and demos read `DEEPSEEK_API_KEY`, optional `DEEPSEEK_BASE_URL`, - **A tool's UI render intent is part of its design**, decided up front (`generic`/`terminal`/`diff`, `locations`); presentation methods are pure functions of `args` ([cookbook](docs/cookbook/adding-a-tool.md)). - **Plan unit, e2e, and snapshot coverage** for new seams, lifecycle shapes, and transcript surfaces; missing snapshot-harness support is part of the implementation, not deferred follow-up. - **Choose PR history deliberately.** Split independent changes; fix the introducing PR before propagation. Standalone PRs and official stacks may merge-forward or rebase after review. Rewrites use `--force-with-lease`, abort on remote movement, never raw `--force`; an in-progress merge-forward preserves its checkpoint before taking a newer base ([rationale](.agents/notes/implemented/process/2026-08-02-native-github-stacks-and-optional-rebases.md)). -- **Labels:** one PR `kind/*`, all material `area/*`, and native Issue Type ([workflow](.agents/skills/dsh-labeling/SKILL.md)). +- **Labels:** one PR `kind/*`, all material `area/*`, and native Issue Type ([taxonomy](.agents/notes/implemented/process/2026-07-25-semantic-pr-label-taxonomy.md)). - TODO markers: `FIXME`/`TODO`/`XXX` by urgency ([semantics](docs/development.md)). - Files end with exactly one trailing newline; `git diff --cached --check` (pre-commit) gates it. From 07ac5e6d4cbf06e5078e73844e38e97684788a75 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Sat, 8 Aug 2026 19:00:10 +0800 Subject: [PATCH 003/108] fix(labels): address taxonomy review findings --- .../2026-07-25-semantic-pr-label-taxonomy.md | 56 --------------- ...026-07-25-semantic-pr-label-taxonomy.zh.md | 56 --------------- ...07-27-dependabot-version-updates.i18n.yaml | 4 +- .../2026-07-27-dependabot-version-updates.md | 2 +- ...026-07-27-dependabot-version-updates.zh.md | 2 +- ...8-unified-github-label-taxonomy.i18n.yaml} | 6 +- ...026-08-08-unified-github-label-taxonomy.md | 72 +++++++++++++++++++ ...-08-08-unified-github-label-taxonomy.zh.md | 72 +++++++++++++++++++ .github/issue-management/policy.mjs | 2 + .github/issue-management/policy.test.mjs | 42 +++++------ AGENTS.md | 2 +- 11 files changed, 172 insertions(+), 144 deletions(-) delete mode 100644 .agents/notes/implemented/process/2026-07-25-semantic-pr-label-taxonomy.md delete mode 100644 .agents/notes/implemented/process/2026-07-25-semantic-pr-label-taxonomy.zh.md rename .agents/notes/implemented/process/{2026-07-25-semantic-pr-label-taxonomy.i18n.yaml => 2026-08-08-unified-github-label-taxonomy.i18n.yaml} (56%) create mode 100644 .agents/notes/implemented/process/2026-08-08-unified-github-label-taxonomy.md create mode 100644 .agents/notes/implemented/process/2026-08-08-unified-github-label-taxonomy.zh.md diff --git a/.agents/notes/implemented/process/2026-07-25-semantic-pr-label-taxonomy.md b/.agents/notes/implemented/process/2026-07-25-semantic-pr-label-taxonomy.md deleted file mode 100644 index 6aef5ecbb8..0000000000 --- a/.agents/notes/implemented/process/2026-07-25-semantic-pr-label-taxonomy.md +++ /dev/null @@ -1,56 +0,0 @@ -# Agent Note: Semantic pull request label taxonomy - -Status: implemented - -English | [中文](2026-07-25-semantic-pr-label-taxonomy.zh.md) - -## Problem - -Pull request labels answer two independent questions: what kind of change the work makes and which durable repository domains it materially affects. Mixing those dimensions or keeping synonymous labels makes queries ambiguous, while a closed area inventory forces new domains into inaccurate categories. - -Issues already have a native Type and a separate source taxonomy. Reusing pull request kind or source labels across both object types duplicates metadata and weakens the meaning of each label family. - -## Decision - -Every open or merged pull request carries exactly one canonical `kind/*` label and at least one materially affected `area/*` label. Closed pull requests that were never merged retain migrated historical assignments but do not receive invented missing classification. Operational labels may coexist without satisfying either dimension. - -### Kinds - -The kind set is closed and mutually exclusive: - -| Kind | Meaning | -|---|---| -| `kind/feature` | Adds or intentionally changes behavior. | -| `kind/bug-fix` | Corrects incorrect behavior. | -| `kind/doc` | Makes documentation the dominant intent. | -| `kind/testing` | Changes tests or testing infrastructure without changing product behavior. | -| `kind/cleanup` | Preserves behavior while maintaining or simplifying implementation or repository process. | -| `kind/dependency` | Updates dependencies without another dominant intent. | - -The kind records the dominant intent. Accompanying tests, documentation, cleanup, or dependency movement do not override a feature or bug fix. - -### Areas - -Areas name durable semantic domains rather than temporary initiatives, ownership, or every path touched incidentally. A pull request carries multiple areas when it changes distinct contracts, but it does not combine an umbrella and a narrower label for the same contract. GitHub's live `area/*` names and descriptions own the current inventory. - -The area set is intentionally extensible. When no existing description honestly covers a durable and reusable repository domain, an agent is empowered to create a concise `area/` label without separate approval. The agent must not create an area for one pull request, an incidental path, a temporary project, a status, or a person or team, and must report the new label after applying it. Reusing an inaccurate area merely to avoid a justified addition is not acceptable. - -Kinds are not extended this way. A new kind changes the mutually exclusive classification contract and requires an explicit taxonomy change with corresponding policy enforcement. - -### Issues and operational labels - -Issues use native Issue Type instead of `kind/*`; their `area/*` labels remain optional. `source/*` labels record Issue provenance and do not apply to pull requests. Priority, GitHub defaults, and workflow triggers remain independent operational metadata. - -Label migrations preserve meaning before removing aliases: add the canonical replacement, verify the labelable, then remove the obsolete assignment. A label is deleted only after no pull request or Issue still uses it, and unrelated labels are never replaced as a set. - -## Alternatives considered - -- **Unprefixed labels.** Rejected because a flat name does not identify whether it classifies intent, domain, source, priority, or automation, and synonymous plain and prefixed labels caused ambiguous queries. -- **A fixed area allowlist in repository policy.** Rejected because durable repository domains evolve. The `area/*` namespace remains mechanically recognizable while live descriptions carry the extensible inventory. -- **Kinds on Issues.** Rejected because native Issue Type already owns that classification; duplicating it as a label creates drift. -- **Automatic areas from paths.** Rejected because areas describe semantic impact across package boundaries, while changed paths include incidental tests, documentation, and support files. -- **Exactly one area per pull request.** Rejected because coherent changes can materially affect several independent contracts. - -## Consequences - -Reviewers and automation can query intent, semantic scope, provenance, priority, and operational triggers independently. Maintainers must read the change and live label descriptions rather than infer classification from title prefixes or paths, and taxonomy migrations carry an explicit historical backfill and verification cost. diff --git a/.agents/notes/implemented/process/2026-07-25-semantic-pr-label-taxonomy.zh.md b/.agents/notes/implemented/process/2026-07-25-semantic-pr-label-taxonomy.zh.md deleted file mode 100644 index 0bd3fbcaec..0000000000 --- a/.agents/notes/implemented/process/2026-07-25-semantic-pr-label-taxonomy.zh.md +++ /dev/null @@ -1,56 +0,0 @@ -# Agent Note: 语义化 PR 标签分类体系 - -Status: implemented - -[English](2026-07-25-semantic-pr-label-taxonomy.md) | 中文 - -## 问题 - -PR(Pull Request)标签回答两个相互独立的问题:工作带来哪一类变更,以及会对哪些持久的仓库领域产生实质影响。混用这两个维度或保留同义标签会使查询含义模糊,而封闭的领域清单会迫使新领域被归入不准确的类别。 - -Issue 已有原生 Issue Type 和独立的来源分类体系。在这两类对象上复用 PR 类型或来源标签会产生重复元数据,并削弱每个标签族的含义。 - -## 决策 - -每项开放或已合并的 PR 都带有恰好一个规范的 `kind/*` 标签,以及至少一个受到实质影响的 `area/*` 标签。未合并即关闭的 PR 保留经迁移的历史标签关系,但不会凭空补充缺失分类。管理用途的标签可以并存,但不能满足这两个维度中的任一个。 - -### 类型 - -类型集合封闭且互斥: - -| 类型 | 含义 | -|---|---| -| `kind/feature` | 新增行为或有意改变行为。 | -| `kind/bug-fix` | 修正错误行为。 | -| `kind/doc` | 以文档变更为主要意图。 | -| `kind/testing` | 修改测试或测试基础设施,但不改变产品行为。 | -| `kind/cleanup` | 在保持行为不变的前提下,维护或简化实现或仓库流程。 | -| `kind/dependency` | 在没有其他主要意图时更新依赖。 | - -类型记录主要意图。配套测试、文档、清理或依赖调整不会覆盖一项功能或缺陷修复的主要意图。 - -### 领域 - -领域表示持久的语义领域,而不是临时项目、归属关系或偶然触及的每条路径。一项 PR 修改不同契约时带有多个领域标签,但不会用一个总括标签和一个较窄标签重复描述同一项契约。当前分类清单以 GitHub 上实时的 `area/*` 名称和描述为准。 - -领域集合有意保持可扩展。当现有描述都无法准确涵盖一个持久且可复用的仓库领域时,agent(智能体)有权无须另行批准,直接创建简洁的 `area/` 标签。agent 不得为单个 PR、偶然涉及的路径、临时项目、状态、个人或团队创建领域,并且必须在应用新标签后报告。仅为避免新增一个确有必要的标签而复用不准确的领域,不可接受。 - -类型不以这种方式扩展。新增类型会改变互斥的分类契约,必须明确修改分类体系并同步调整策略强制校验。 - -### Issue 与管理用途标签 - -Issue 使用原生 Issue Type,而不是 `kind/*`;其 `area/*` 标签仍为可选。`source/*` 标签记录 Issue 来源,不适用于 PR。优先级、GitHub 默认标签和工作流触发器仍是相互独立的管理元数据。 - -迁移标签时,须先保留语义,再移除别名:先添加规范替代标签并验证对象,再移除废弃的标签关系。只有在所有 PR 和 Issue 都不再使用某个标签后才能将其删除,且绝不整组替换无关标签。 - -## 曾考虑的替代方案 - -- **无前缀标签。** 不予采纳,因为扁平名称无法表明其表示的是意图、领域、来源、优先级还是自动化用途,而同义的无前缀和带前缀标签会使查询含义模糊。 -- **在仓库策略中使用固定的领域允许列表。** 不予采纳,因为持久的仓库领域会持续演变。`area/*` 命名空间仍可由机制识别,而实时描述负责承载可扩展的分类清单。 -- **在 Issue 上使用类型标签。** 不予采纳,因为原生 Issue Type 已负责这项分类;再用标签重复表示会产生漂移。 -- **从路径自动生成领域。** 不予采纳,因为领域描述跨越包边界的语义影响,而变更路径还会包含偶然涉及的测试、文档和支持文件。 -- **每项 PR 恰好一个领域。** 不予采纳,因为一项内聚的变更可以对多个独立契约产生实质影响。 - -## 后果 - -评审人和自动化流程可以分别查询意图、语义范围、来源、优先级和工作流触发器。维护者必须阅读变更内容和实时标签描述,而不能根据标题前缀或路径推断分类;迁移分类体系还会产生明确的历史回填与验证成本。 diff --git a/.agents/notes/implemented/process/2026-07-27-dependabot-version-updates.i18n.yaml b/.agents/notes/implemented/process/2026-07-27-dependabot-version-updates.i18n.yaml index 316c31771e..dea250de53 100644 --- a/.agents/notes/implemented/process/2026-07-27-dependabot-version-updates.i18n.yaml +++ b/.agents/notes/implemented/process/2026-07-27-dependabot-version-updates.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/process/2026-07-27-dependabot-version-updates.md -2026-07-27-dependabot-version-updates.md: 5d42563788d9f1e72da65c8e9750d6b1ecba06a5 -2026-07-27-dependabot-version-updates.zh.md: 4847059944e7e35de5719a6cbfd3d5b133467ccb +2026-07-27-dependabot-version-updates.md: bba83c9e720cf87f91638131d7f9580422ea7f76 +2026-07-27-dependabot-version-updates.zh.md: 74b399778fdd06cbbe38234f7bddc5c28c4fd600 diff --git a/.agents/notes/implemented/process/2026-07-27-dependabot-version-updates.md b/.agents/notes/implemented/process/2026-07-27-dependabot-version-updates.md index 5d42563788..bba83c9e72 100644 --- a/.agents/notes/implemented/process/2026-07-27-dependabot-version-updates.md +++ b/.agents/notes/implemented/process/2026-07-27-dependabot-version-updates.md @@ -12,7 +12,7 @@ Maintained registry and GitHub Actions dependencies need a regular update path. The default branch carries [`.github/dependabot.yml`](../../../../.github/dependabot.yml) with weekly version-update checks for the root pnpm workspace, including `native/landlock-run`, the `python/sdk` uv project, and GitHub Actions. Every entry sets `cooldown.default-days` to `30`, so a version release becomes eligible only after it is at least 30 days old and is proposed on the next weekly check. The [in-repository Landlock release decision](2026-08-06-in-repository-landlock-release.md) owns the shared-workspace boundary. -The root pnpm version-update scan excludes `vendor/**`, whose source and manifests move only through the [vendoring procedure](../../../../vendor/README.md). GitHub applies `exclude-paths` only to version updates; a security pull request that touches a vendored manifest is replaced through the vendoring procedure instead of being merged as generated. Dependabot pull requests receive the repository's `cleanup` kind and `area/infra` area labels, run the normal pull-request checks, and remain subject to maintainer review; this automation does not merge them. +The root pnpm version-update scan excludes `vendor/**`, whose source and manifests move only through the [vendoring procedure](../../../../vendor/README.md). GitHub applies `exclude-paths` only to version updates; a security pull request that touches a vendored manifest is replaced through the vendoring procedure instead of being merged as generated. Dependabot pull requests receive the repository's `kind/dependency` kind and `area/infra` area labels, run the normal pull-request checks, and remain subject to maintainer review; this automation does not merge them. Repository settings enable dependency vulnerability alerts and Dependabot security updates. GitHub does not apply version-update cooldowns to those security updates, so security fixes remain eligible immediately. A generated pnpm security pull request can still fail the repository's lockfile release-age verification when dependency resolution selects unrelated fresh transitive versions; that pull request waits or is narrowed instead of weakening the policy. The repository's coordinated fresh-release exceptions are not copied into Dependabot's cooldown exclusions: automated version updates use the uniform 30-day wait, while an explicitly reviewed manual update can still follow its owning release procedure. diff --git a/.agents/notes/implemented/process/2026-07-27-dependabot-version-updates.zh.md b/.agents/notes/implemented/process/2026-07-27-dependabot-version-updates.zh.md index 4847059944..74b399778f 100644 --- a/.agents/notes/implemented/process/2026-07-27-dependabot-version-updates.zh.md +++ b/.agents/notes/implemented/process/2026-07-27-dependabot-version-updates.zh.md @@ -12,7 +12,7 @@ Status: implemented 默认分支包含 [`.github/dependabot.yml`](../../../../.github/dependabot.yml),其中为包含 `native/landlock-run` 的根 pnpm 工作区、`python/sdk` uv 项目和 GitHub Actions 配置了每周一次的版本更新检查。每个更新项都将 `cooldown.default-days` 设为 `30`,因此某个版本只有在发布至少 30 天后才符合更新条件,并会在下一次每周检查时生成更新提案。[仓库内 Landlock 发布决策](2026-08-06-in-repository-landlock-release.md)负责共享工作区边界。 -根 pnpm 工作区的版本更新扫描排除 `vendor/**`,其中的源码和 manifest(元数据清单)只能通过 [vendoring 流程](../../../../vendor/README.md)变更。GitHub 仅将 `exclude-paths` 用于版本更新;如果安全更新 PR(Pull Request)涉及随源码纳入仓库的 manifest,则改由 vendoring 流程处理,以替代自动生成的 PR,而不会将其原样合并。Dependabot PR 会获得仓库的 `cleanup` 类型标签和 `area/infra` 区域标签,运行常规 PR 检查,并且仍须由维护者评审;该自动化不会合并这些 PR。 +根 pnpm 工作区的版本更新扫描排除 `vendor/**`,其中的源码和 manifest(元数据清单)只能通过 [vendoring 流程](../../../../vendor/README.md)变更。GitHub 仅将 `exclude-paths` 用于版本更新;如果安全更新 PR(Pull Request)涉及随源码纳入仓库的 manifest,则改由 vendoring 流程处理,以替代自动生成的 PR,而不会将其原样合并。Dependabot PR 会获得仓库的 `kind/dependency` 类型标签和 `area/infra` 区域标签,运行常规 PR 检查,并且仍须由维护者评审;该自动化不会合并这些 PR。 仓库设置已启用依赖项漏洞警报和 Dependabot 安全更新。GitHub 不会对这些安全更新应用版本更新冷却期,因此安全修复仍可立即进入更新流程。如果依赖解析还选中了其他刚发布的传递依赖,pnpm 安全更新 PR 仍可能无法通过仓库的锁文件发布时长校验;此类 PR 应等待隔离期结束或缩小更新范围,不得因此放宽政策。仓库为协调刚发布版本而设置的例外,不会纳入 Dependabot 的冷却期排除项:自动版本更新统一等待 30 天;经过明确评审的手动更新仍可遵循相应的发布流程。 diff --git a/.agents/notes/implemented/process/2026-07-25-semantic-pr-label-taxonomy.i18n.yaml b/.agents/notes/implemented/process/2026-08-08-unified-github-label-taxonomy.i18n.yaml similarity index 56% rename from .agents/notes/implemented/process/2026-07-25-semantic-pr-label-taxonomy.i18n.yaml rename to .agents/notes/implemented/process/2026-08-08-unified-github-label-taxonomy.i18n.yaml index a83236521c..5ee57b914f 100644 --- a/.agents/notes/implemented/process/2026-07-25-semantic-pr-label-taxonomy.i18n.yaml +++ b/.agents/notes/implemented/process/2026-08-08-unified-github-label-taxonomy.i18n.yaml @@ -1,6 +1,6 @@ # Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: -# pnpm run verify-translation-pairing --write .agents/notes/implemented/process/2026-07-25-semantic-pr-label-taxonomy.md -2026-07-25-semantic-pr-label-taxonomy.md: 6aef5ecbb80005d1fcd90b4ee673bbbebb92cba6 -2026-07-25-semantic-pr-label-taxonomy.zh.md: 0bd3fbcaeca37fa894e92ce5a4e78c4125056ac9 +# pnpm run verify-translation-pairing --write .agents/notes/implemented/process/2026-08-08-unified-github-label-taxonomy.md +2026-08-08-unified-github-label-taxonomy.md: fddda6c99053e55d3e31a2f6e5c55add09772117 +2026-08-08-unified-github-label-taxonomy.zh.md: c8d727c9a56d6c7c25f73eacde42e0bef1c779b7 diff --git a/.agents/notes/implemented/process/2026-08-08-unified-github-label-taxonomy.md b/.agents/notes/implemented/process/2026-08-08-unified-github-label-taxonomy.md new file mode 100644 index 0000000000..fddda6c990 --- /dev/null +++ b/.agents/notes/implemented/process/2026-08-08-unified-github-label-taxonomy.md @@ -0,0 +1,72 @@ +# Agent Note: Unified GitHub label taxonomy + +Status: implemented + +English | [中文](2026-08-08-unified-github-label-taxonomy.zh.md) + +## Problem + +Pull request labels answer two independent questions: what kind of change the work makes and which durable repository domains it materially affects. Mixing those dimensions or retaining synonymous plain and namespaced labels makes queries ambiguous, while a closed area inventory forces new domains into inaccurate categories. + +Issues already have a native Type and a separate source taxonomy. Reusing pull request kind or source labels across both object types duplicates metadata and weakens the meaning of each family. + +## Decision + +Every open or merged pull request carries exactly one canonical `kind/*` label and at least one materially affected `area/*` label. Closed pull requests that were never merged retain migrated historical assignments but do not receive invented missing classification. Operational labels may coexist without satisfying either dimension. + +### Kinds + +The kind set is closed and mutually exclusive: + +| Kind | Meaning | +|---|---| +| `kind/feature` | Adds or intentionally changes behavior. | +| `kind/bug-fix` | Corrects incorrect behavior. | +| `kind/doc` | Makes documentation the dominant intent. | +| `kind/testing` | Changes tests or testing infrastructure without changing product behavior. | +| `kind/cleanup` | Preserves behavior while maintaining or simplifying implementation or repository process. | +| `kind/dependency` | Updates dependencies without another dominant intent. | + +The kind records the dominant intent. Accompanying tests, documentation, cleanup, or dependency movement do not override a feature or bug fix. A new kind changes this classification contract and requires an explicit taxonomy and policy change. + +Repository policy rejects unsupported `kind/*` values and reserves every alias removed by the unification: `kind/bug`, `kind/documentation`, `feature`, `bug-fix`, `doc`, `cleanup`, `testing`, `dependencies`, `ci`, `cli`, `llm`, and `web-search`. Reserving the exact migrated set prevents an obsolete synonym from being recreated as an apparently unrelated operational label. + +### Areas + +Areas name durable semantic domains rather than temporary initiatives, ownership, or every path touched incidentally. A pull request carries multiple areas when it changes distinct contracts, but it does not combine an umbrella and a narrower label for the same contract. GitHub's live `area/*` names and descriptions own the current inventory; this record owns the selection rule and the non-obvious boundaries that cannot fit reliably in short label descriptions. + +- `area/web` covers browser and Electron graphical interfaces, `area/vscode` covers the editor extension, and `area/api` covers cross-interface protocols and language SDKs. +- `area/planning` covers goals, plans, todos, and scheduling, while `area/workflow` covers executable workflows and background task runtimes. +- `area/artifact` deliberately combines artifacts, attachments, and multimodal delivery. Split labels become justified only when those concerns again need independent review or queries. +- `area/tools` applies to generic registry, schema, and execution contracts. A concrete capability uses its own area unless it also changes that generic contract. +- `area/hooks` means the Claude Code and Codex bridges, `area/infra` covers build, release, CI, repository gates, generators, dependencies, and developer tooling, and `area/windows` covers native Windows product support rather than CI runner selection. + +The area set is intentionally extensible. When no existing description honestly covers a durable and reusable domain, an agent may create a concise `area/` label without separate approval. It must not create an area for one pull request, an incidental path, a temporary project, a status, or a person or team, and it reports the new label and rationale to the requester after applying it. Reusing an inaccurate area merely to avoid a justified addition is not acceptable. + +### Issues and migrations + +Issues use native Issue Type instead of `kind/*`; their `area/*` labels remain optional. `source/*` labels record Issue provenance and do not apply to pull requests. Priority, GitHub defaults, and workflow triggers remain independent operational metadata. + +Label migrations preserve meaning before removing aliases: add the canonical replacement, verify the labelable, then remove the obsolete assignment. A label is deleted only after no pull request or Issue still uses it, and unrelated labels are never replaced as a set. + +## Alternatives considered + +**Unprefixed labels.** Plain names reduce visual noise, but they do not identify whether a label classifies intent, domain, source, priority, or automation. Retaining both plain and namespaced synonyms also makes queries and policy enforcement ambiguous. + +**One undifferentiated label set.** A label's presence would not prove that both intent and semantic scope were considered. + +**A fixed area allowlist in repository policy.** Durable repository domains evolve. The `area/*` namespace remains mechanically recognizable while live descriptions carry the extensible inventory. + +**Package- or path-derived areas.** Areas describe semantic impact across package boundaries, while changed paths include incidental tests, documentation, and support files. + +**Separate labels for every delivery shell or media lifecycle.** Browser and Electron delivery share one graphical domain, and artifact, attachment, and multimodal delivery currently share one review/query domain. A split belongs in a later taxonomy change only when it restores useful independent classification. + +**Broad implementation labels in place of semantic domains.** A concrete capability is not merely its tool, interface, filesystem, or process implementation. Generic implementation areas apply only when their own contracts change. + +**Kinds on Issues.** Native Issue Type already owns that classification; duplicating it as a label creates drift. + +**Exactly one area per pull request.** Coherent changes can materially affect several independent contracts, and dropping secondary areas hides affected scope. + +## Consequences + +Reviewers and automation can query intent, semantic scope, provenance, priority, and operational triggers independently. Maintainers must read the change and the live label descriptions instead of inferring classification from title prefixes or paths. The live catalog, this rationale, and policy enforcement must move together when a kind or a non-obvious area boundary changes, and taxonomy migrations carry an explicit historical backfill and verification cost. diff --git a/.agents/notes/implemented/process/2026-08-08-unified-github-label-taxonomy.zh.md b/.agents/notes/implemented/process/2026-08-08-unified-github-label-taxonomy.zh.md new file mode 100644 index 0000000000..c8d727c9a5 --- /dev/null +++ b/.agents/notes/implemented/process/2026-08-08-unified-github-label-taxonomy.zh.md @@ -0,0 +1,72 @@ +# Agent Note: 统一 GitHub 标签分类体系 + +Status: implemented + +[English](2026-08-08-unified-github-label-taxonomy.md) | 中文 + +## 问题 + +PR(Pull Request)标签回答两个相互独立的问题:工作带来哪一类变更,以及会对哪些持久的仓库领域产生实质影响。混用这两个维度,或同时保留同义的无前缀标签与带命名空间的标签,都会使查询含义模糊;封闭的领域清单则会迫使新领域归入不准确的类别。 + +Issue 已有原生 Issue Type 和独立的来源分类体系。在这两类对象上复用 PR 类型或来源标签会产生重复元数据,并削弱每个标签族的含义。 + +## 决策 + +每项开放或已合并的 PR 都带有恰好一个规范的 `kind/*` 标签,以及至少一个表示实质受影响领域的 `area/*` 标签。未合并即关闭的 PR 保留经迁移的历史标签关系,但不会凭空补充缺失分类。管理用途的标签可以并存,但不能满足这两个维度中的任一个。 + +### 变更类型 + +类型集合封闭且互斥: + +| 变更类型 | 含义 | +|---|---| +| `kind/feature` | 新增行为或有意改变行为。 | +| `kind/bug-fix` | 修正错误行为。 | +| `kind/doc` | 以文档变更为主导意图。 | +| `kind/testing` | 在不改变产品行为的前提下修改测试或测试基础设施。 | +| `kind/cleanup` | 在保持行为不变的前提下,维护或简化实现或仓库流程。 | +| `kind/dependency` | 在没有其他主导意图时更新依赖。 | + +类型记录主导意图。配套测试、文档、清理或依赖调整不会盖过功能变更或缺陷修复这一主导意图。新增类型会改变这项分类契约,因此必须明确修改分类体系和政策。 + +仓库政策会拒绝不支持的 `kind/*` 值,并将统一过程中移除的所有别名列为保留名称:`kind/bug`、`kind/documentation`、`feature`、`bug-fix`、`doc`、`cleanup`、`testing`、`dependencies`、`ci`、`cli`、`llm` 和 `web-search`。精确保留这组已迁移的名称,可以防止过时的同义名称被重新创建成看似无关的管理用途标签。 + +### 领域 + +领域表示持久的语义领域,而不是临时专项、归属关系或偶然触及的每条路径。一项 PR 修改不同契约时带有多个领域标签,但不会用一个总括标签和一个较窄标签重复描述同一项契约。GitHub 上现行的 `area/*` 名称和说明定义当前清单;本记录定义选择规则,以及简短标签说明无法可靠容纳的非显然边界。 + +- `area/web` 覆盖浏览器与 Electron 图形界面,`area/vscode` 覆盖编辑器扩展,`area/api` 覆盖跨界面协议与各语言 SDK。 +- `area/planning` 覆盖目标、计划、待办和调度,`area/workflow` 则覆盖可执行工作流与后台任务运行时。 +- `area/artifact` 有意合并产物、附件与多模态交付。只有当这些关注点再次需要独立评审或查询时,才有理由拆分标签。 +- `area/tools` 适用于通用注册表、schema 与执行契约。具体能力使用自身的领域标签,除非它还修改了这项通用契约。 +- `area/hooks` 表示 Claude Code 与 Codex 桥接,`area/infra` 覆盖构建、发布、CI、仓库门禁、生成器、依赖与开发者工具,`area/windows` 覆盖原生 Windows 产品支持,而不是 CI runner 的选型。 + +领域集合有意保持可扩展。当现有说明都无法如实涵盖一个持久且可复用的领域时,agent(智能体)无需另行批准,即可创建一个简洁的 `area/` 标签。agent 不得为单个 PR、偶然涉及的路径、临时项目、状态、个人或团队创建领域,并且必须在应用新标签后向请求者报告该标签及理由。仅为避免新增一个确有必要的领域标签而复用不准确的领域,不可接受。 + +### Issue 与迁移 + +Issue 使用原生 Issue Type,而不是 `kind/*`;其 `area/*` 标签仍然可选。`source/*` 标签记录 Issue 来源,不适用于 PR。优先级、GitHub 默认标签和工作流触发器仍是相互独立的管理元数据。 + +迁移标签时,须先保留语义,再移除别名:先添加规范替代标签,核验可加标签对象,再移除废弃的标签关系。只有在所有 PR 和 Issue 都不再使用某个标签后才能将其删除,且绝不整组替换无关标签。 + +## 考虑过的替代方案 + +**无前缀标签。** 无前缀名称可以减少视觉噪声,但无法表明标签分类的是意图、领域、来源、优先级还是自动化用途。同时保留无前缀和带命名空间的同义标签,也会使查询和政策执行含义模糊。 + +**不区分维度的单一标签集合。** 某个标签存在,并不能证明意图和语义范围都经过了考虑。 + +**仓库政策中的固定领域允许清单。** 持久的仓库领域会演进。`area/*` 命名空间仍可机械识别,而现行说明承载可扩展清单。 + +**按包或路径派生的领域。** 领域描述跨越包边界的语义影响,而变更路径会包含偶然涉及的测试、文档和支持文件。 + +**为每种交付载体或媒体生命周期单设标签。** 浏览器与 Electron 交付共用一个图形界面领域,产物、附件与多模态交付目前也共用一个评审/查询领域。只有当拆分能恢复有用的独立分类时,才应在后续分类体系变更中进行。 + +**用宽泛的实现标签取代语义领域。** 一项具体能力并不只是其工具、接口、文件系统或进程实现。通用实现领域只在其自身契约变化时适用。 + +**在 Issue 上使用类型标签。** 原生 Issue Type 已负责这项分类;再用标签复制会造成漂移。 + +**每个 PR 恰好一个领域。** 内聚的变更可能对多个独立契约产生实质影响,丢弃次要领域会隐藏受影响范围。 + +## 后果 + +评审人和自动化流程可以分别查询意图、语义范围、来源、优先级和工作流触发条件。维护者必须阅读变更内容和现行标签说明,而不能根据标题前缀或路径推断分类。当某种类型或某条非显然的领域边界发生变化时,现行标签清单、本记录中的决策依据和政策执行必须同步更新;分类体系迁移还会产生明确的历史回填和验证成本。 diff --git a/.github/issue-management/policy.mjs b/.github/issue-management/policy.mjs index 18a9b03628..09e1bbe7fd 100644 --- a/.github/issue-management/policy.mjs +++ b/.github/issue-management/policy.mjs @@ -20,9 +20,11 @@ const PR_KINDS = new Set([ 'kind/cleanup', 'kind/dependency', ]) +// Aliases removed by the unified taxonomy migration remain reserved so they cannot be recreated. const LEGACY_LABELS = new Set([ 'kind/bug', 'kind/documentation', + 'feature', 'bug-fix', 'doc', 'cleanup', diff --git a/.github/issue-management/policy.test.mjs b/.github/issue-management/policy.test.mjs index bcaf450afa..8a9b0f91e6 100644 --- a/.github/issue-management/policy.test.mjs +++ b/.github/issue-management/policy.test.mjs @@ -36,6 +36,22 @@ const canonicalKinds = [ 'kind/dependency', ] +// Keep an independent oracle rather than importing the implementation's reserved set. +const legacyLabels = [ + 'kind/bug', + 'kind/documentation', + 'feature', + 'bug-fix', + 'doc', + 'cleanup', + 'testing', + 'dependencies', + 'ci', + 'cli', + 'llm', + 'web-search', +] + const reviewedPull = (labels) => ({ isDraft: false, authorType: 'User', @@ -115,17 +131,7 @@ test('reserves PR kind and legacy labels for pull requests', () => { for (const label of [ ...canonicalKinds, 'kind/experimental', - 'kind/bug', - 'kind/documentation', - 'bug-fix', - 'doc', - 'cleanup', - 'testing', - 'dependencies', - 'ci', - 'cli', - 'llm', - 'web-search', + ...legacyLabels, ]) { assert.ok( validateIssue({ ...legalIssue, labels: [label] }).some((error) => @@ -326,19 +332,7 @@ test('rejects multiple, unknown, legacy, and Issue-source PR labels', () => { 'PR 含不支持的 kind/*:kind/experimental', ), ) - for (const label of [ - 'kind/bug', - 'kind/documentation', - 'bug-fix', - 'doc', - 'cleanup', - 'testing', - 'dependencies', - 'ci', - 'cli', - 'llm', - 'web-search', - ]) { + for (const label of legacyLabels) { assert.ok( validatePullRequest(reviewedPull(['kind/feature', 'area/web', label])).some((error) => error.startsWith('PR 含旧版标签:'), diff --git a/AGENTS.md b/AGENTS.md index 9344e2e6e2..928a118868 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -118,7 +118,7 @@ Real-API tests and demos read `DEEPSEEK_API_KEY`, optional `DEEPSEEK_BASE_URL`, - **A tool's UI render intent is part of its design**, decided up front (`generic`/`terminal`/`diff`, `locations`); presentation methods are pure functions of `args` ([cookbook](docs/cookbook/adding-a-tool.md)). - **Plan unit, e2e, and snapshot coverage** for new seams, lifecycle shapes, and transcript surfaces; missing snapshot-harness support is part of the implementation, not deferred follow-up. - **Choose PR history deliberately.** Split independent changes; fix the introducing PR before propagation. Standalone PRs and official stacks may merge-forward or rebase after review. Rewrites use `--force-with-lease`, abort on remote movement, never raw `--force`; an in-progress merge-forward preserves its checkpoint before taking a newer base ([rationale](.agents/notes/implemented/process/2026-08-02-native-github-stacks-and-optional-rebases.md)). -- **Labels:** one PR `kind/*`, all material `area/*`, and native Issue Type ([taxonomy](.agents/notes/implemented/process/2026-07-25-semantic-pr-label-taxonomy.md)). +- **Labels:** one PR `kind/*`, all material `area/*`, and native Issue Type ([taxonomy](.agents/notes/implemented/process/2026-08-08-unified-github-label-taxonomy.md)). - TODO markers: `FIXME`/`TODO`/`XXX` by urgency ([semantics](docs/development.md)). - Files end with exactly one trailing newline; `git diff --cached --check` (pre-commit) gates it. From 8917ff8ef4a6f8c79d9d0f66429ab10734ca3b23 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Tue, 28 Jul 2026 23:00:00 +0800 Subject: [PATCH 004/108] refactor(runtime): compose consumers over fs and subprocess --- ...06-17-filesystem-capability-seam.i18n.yaml | 4 +- .../2026-06-17-filesystem-capability-seam.md | 7 +- ...026-06-17-filesystem-capability-seam.zh.md | 19 +- .../2026-07-15-lsp-capability-seam.i18n.yaml | 4 +- .../2026-07-15-lsp-capability-seam.md | 20 +- .../2026-07-15-lsp-capability-seam.zh.md | 32 +- ...26-subprocess-consumer-migration.i18n.yaml | 4 +- ...026-07-26-subprocess-consumer-migration.md | 16 +- ...-07-26-subprocess-consumer-migration.zh.md | 18 +- .../2026-07-26-subprocess-seam.i18n.yaml | 4 +- .../2026-07-26-subprocess-seam.md | 8 +- .../2026-07-26-subprocess-seam.zh.md | 18 +- ...rtable-execution-world-consumers.i18n.yaml | 6 + ...7-28-portable-execution-world-consumers.md | 48 ++ ...8-portable-execution-world-consumers.zh.md | 48 ++ .../feature/2026-06-15-code-mode.i18n.yaml | 4 +- .../feature/2026-06-15-code-mode.md | 28 +- .../feature/2026-06-15-code-mode.zh.md | 48 +- ...26-07-16-persistent-pty-sessions.i18n.yaml | 4 +- .../2026-07-16-persistent-pty-sessions.md | 25 +- .../2026-07-16-persistent-pty-sessions.zh.md | 39 +- docs/config-catalog.md | 34 +- docs/cordis-catalog/events.md | 6 +- .../core-data-structures/filesystem.i18n.yaml | 4 +- docs/core-data-structures/filesystem.md | 6 +- docs/core-data-structures/filesystem.zh.md | 6 +- .../core-data-structures/subprocess.i18n.yaml | 4 +- docs/core-data-structures/subprocess.md | 16 +- docs/core-data-structures/subprocess.zh.md | 46 +- knip.json | 10 + packages/README.i18n.yaml | 4 +- packages/README.md | 2 +- packages/README.zh.md | 2 +- .../code-runtime-worker/README.md | 2 +- .../code-runtime-worker/src/runtime-host.ts | 222 +++++++++ .../code-runtime-worker/tsconfig.json | 3 - .../code-runtime-worker/tsdown.config.ts | 17 +- packages/code-runtime/code-runtime/README.md | 4 +- .../tests/workspace-context.spec.ts | 14 + .../cordis/tool-cordis/src/api-catalog.ts | 40 ++ packages/fs/README.i18n.yaml | 4 +- packages/fs/README.md | 21 +- packages/fs/README.zh.md | 25 +- packages/fs/fs-local/README.i18n.yaml | 4 +- packages/fs/fs-local/README.md | 7 +- packages/fs/fs-local/README.zh.md | 19 +- packages/fs/fs-local/src/fsio.ts | 73 ++- packages/fs/fs-local/src/index.ts | 21 +- packages/fs/fs-local/tests/filesystem.spec.ts | 24 + packages/fs/fs-local/tests/fsio.spec.ts | 63 ++- packages/fs/fs/README.i18n.yaml | 4 +- packages/fs/fs/README.md | 27 +- packages/fs/fs/README.zh.md | 45 +- packages/fs/fs/src/index.ts | 52 +- packages/fs/fs/tests/service.spec.ts | 13 +- packages/fs/tool-fs/tests/tools.spec.ts | 10 + packages/lsp/README.i18n.yaml | 4 +- packages/lsp/README.md | 10 +- packages/lsp/README.zh.md | 12 +- packages/lsp/lsp-local/README.i18n.yaml | 4 +- packages/lsp/lsp-local/README.md | 18 +- packages/lsp/lsp-local/README.zh.md | 26 +- packages/lsp/lsp-local/package.json | 3 + packages/lsp/lsp-local/src/connection.ts | 7 +- packages/lsp/lsp-local/src/host.ts | 182 +++---- packages/lsp/lsp-local/src/index.ts | 115 ++--- packages/lsp/lsp-local/src/instance.ts | 13 +- packages/lsp/lsp-local/tests/built-lib.e2e.ts | 5 +- packages/lsp/lsp-local/tests/host.spec.ts | 68 ++- packages/lsp/lsp-local/tests/instance.spec.ts | 17 +- .../lsp/lsp-local/tests/lifecycle.spec.ts | 4 + packages/lsp/lsp-local/tests/provider.spec.ts | 13 + .../lsp-local/tests/typescript-server.e2e.ts | 2 + packages/lsp/lsp-local/tsconfig.json | 3 + packages/lsp/tool-lsp/package.json | 2 + .../lsp/tool-lsp/tests/integration.spec.ts | 2 + packages/pty/README.i18n.yaml | 4 +- packages/pty/README.md | 11 +- packages/pty/README.zh.md | 11 +- packages/pty/pty-local/README.i18n.yaml | 4 +- packages/pty/pty-local/README.md | 20 +- packages/pty/pty-local/README.zh.md | 24 +- packages/pty/pty-local/package.json | 8 +- packages/pty/pty-local/src/index.ts | 69 ++- packages/pty/pty-local/src/session.ts | 341 ++++++------- packages/pty/pty-local/tests/index.spec.ts | 173 +++---- packages/pty/pty-local/tests/local.spec.ts | 2 + packages/pty/pty-local/tests/session.spec.ts | 471 +++++++++++------- packages/pty/tool-pty/package.json | 1 + .../tool-pty/tests/loader-composition.spec.ts | 3 + .../skill-local/tests/skill-local.spec.ts | 14 + packages/subprocess/README.i18n.yaml | 4 +- packages/subprocess/README.md | 10 +- packages/subprocess/README.zh.md | 12 +- .../subprocess-local/README.i18n.yaml | 4 +- .../subprocess/subprocess-local/README.md | 14 +- .../subprocess/subprocess-local/README.zh.md | 22 +- .../subprocess/subprocess-local/package.json | 7 + .../scripts/ensure-spawn-helper.mjs | 0 .../subprocess/subprocess-local/src/index.ts | 104 +++- .../src/process-inspector.ts | 10 +- .../subprocess-local/src/terminal.ts | 226 +++++++++ .../subprocess-local/tests/local.spec.ts | 134 ++++- .../tests/process-inspector.spec.ts | 6 +- .../subprocess-local/tests/terminal.spec.ts | 275 ++++++++++ .../subprocess/subprocess/README.i18n.yaml | 4 +- packages/subprocess/subprocess/README.md | 10 +- packages/subprocess/subprocess/README.zh.md | 22 +- packages/subprocess/subprocess/src/index.ts | 52 +- packages/subprocess/subprocess/src/types.ts | 68 +++ .../subprocess/tests/service.spec.ts | 29 +- scripts/check-workspace-constraints.ts | 1 + scripts/gen-cordis-catalog.ts | 3 + scripts/run-gates.ts | 7 +- .../verify-package-readme-model-experience.ts | 1 + tsconfig.host.json | 1 + 116 files changed, 2828 insertions(+), 1122 deletions(-) create mode 100644 .agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml create mode 100644 .agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md create mode 100644 .agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md create mode 100644 packages/code-runtime/code-runtime-worker/src/runtime-host.ts rename packages/{pty/pty-local => subprocess/subprocess-local}/scripts/ensure-spawn-helper.mjs (100%) rename packages/{pty/pty-local => subprocess/subprocess-local}/src/process-inspector.ts (96%) create mode 100644 packages/subprocess/subprocess-local/src/terminal.ts rename packages/{pty/pty-local => subprocess/subprocess-local}/tests/process-inspector.spec.ts (98%) create mode 100644 packages/subprocess/subprocess-local/tests/terminal.spec.ts diff --git a/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.i18n.yaml b/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.i18n.yaml index efdb14ea94..dc9b924f54 100644 --- a/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.md -2026-06-17-filesystem-capability-seam.md: fee0161e5e8397ac1d1c0e2850efad840c65d971 -2026-06-17-filesystem-capability-seam.zh.md: 46e3ad22c530319d0d1b6ba23a8aba8ffc2fdb8c +2026-06-17-filesystem-capability-seam.md: 2fb360e5ad3c972f4a8de50c602a79af158dfc22 +2026-06-17-filesystem-capability-seam.zh.md: 1f70113b1583d890047ee4a17161f76b6390d8e8 diff --git a/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.md b/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.md index fee0161e5e..2fb360e5ad 100644 --- a/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.md +++ b/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.md @@ -62,8 +62,9 @@ The root `tool-fs` plugin registers the full filesystem tool suite (`read`, `wri The interface covers these semantic operations: - Resolve a model/plugin-supplied path into a backend-defined target. +- Convert a resolved target to the canonical process path or `file:` URI for the same execution world, and test containment without parsing its opaque key. - Stat target metadata without reading file contents. -- Read a bounded UTF-8 text page from a target. +- Read complete or streamed UTF-8 text, including one stable-handle byte-bounded whole-file operation. - Create or replace a UTF-8 text file. - Edit an existing UTF-8 text file by literal replacement. @@ -83,9 +84,11 @@ Resolved targets must expose at least three concepts: - An opaque `targetKey`, used for stale guards and file-state lookup. The local backend might use a realpath-like key; a remote backend might use a workspace URI or file id. Consumers must not parse or assume this is a local absolute path. - A `displayPath`, used for model/UI-facing output. It may be a local absolute path, workspace-relative path, or remote URI depending on the backend. +`targetKey` remains opaque even when another capability shares the provider's execution world. Such consumers ask the provider for `processPath(target)`, `fileUrl(target)`, or `contains(parent, child)`; the [portable execution-world decision](2026-07-28-portable-execution-world-consumers.md) owns why these facts sit on the filesystem seam. + Read and mutation results must include an opaque file `version`. The local backend derives its token from bigint stat metadata (`dev`, `ino`, `size`, `mtimeNs`, and `ctimeNs`) so same-size rewrites and inode replacement invalidate consumers reliably; a remote backend can use a revision id or hash-like token. The `dsh-fs-policy` plugin records versions for stale checks; consumers may display related metadata but must not interpret the version token. -The provider hands back decoded text: `readText` returns a whole regular text file, `streamText` streams the same text semantics for large files. Both own regular-file checks, bounded line/output handling is NOT theirs — line windowing, numbered-line rendering, and total-line accounting live in the executor (`dsh-tool-fs`), which reads through `ctx.fs` and renders the model-facing window. The provider owns UTF-8 decoding and binary/NUL rejection; it does not know about line windows or views. +The provider hands back decoded text: `readText` returns a whole regular text file, `streamText` streams the same text semantics for large files, and `readTextBounded` holds one backend-owned stable handle while rejecting a complete file above its byte ceiling. Line windowing, numbered-line rendering, and total-line accounting live in the executor (`dsh-tool-fs`). The provider owns regular-file checks, UTF-8 decoding, binary/NUL rejection, and the bounded read's replacement/growth race; it does not know about line windows or views. Observed-state recording is not on `ctx.fs`: after a successful read the executor emits `fs/observed`, and the `dsh-fs-policy` plugin records `{ version }` for the deriving owner. There is no `full`/`partial` view — a read at any window records the version, and freshness (not view completeness) authorizes a later write/edit. diff --git a/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.zh.md b/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.zh.md index 46e3ad22c5..1f70113b15 100644 --- a/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.zh.md +++ b/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.zh.md @@ -16,7 +16,7 @@ harness 已有一个具体的 `bash` 能力 seam(`dsh-bash` / `dsh-bash-local` 如果没有 `ctx.fs` 接口,将本地文件系统访问替换为沙箱或远程后端时,即使面向模型的契约应当保持稳定,工具 schema、演示和提示词引导也会被迫变动。这还使权限/沙箱边界更难推理:一个 `cwd` 选项看起来像沙箱,但除非有显式的后端或 `tools/execute` 策略强制隔离,否则它只是一个基础路径。 -我们需要文件系统工具在成为公开包接口之前,以与 bash 相同的能力 seam 形态落地。 +我们需要文件系统工具在成为公开包(package)接口之前,以与 bash 相同的能力 seam 形态落地。 ## 决策 @@ -51,7 +51,7 @@ harness 已有一个具体的 `bash` 能力 seam(`dsh-bash` / `dsh-bash-local` `@deepseek-ai/dsh-fs-local` 依赖 `@deepseek-ai/dsh-fs` 和 `cordis`。它继承 `FileSystem`,将自身注册为 `ctx.fs`,拥有本地后端配置(如基目录),并包含所有直接的 `node:fs` / `node:path` 访问。它不持有观测状态存储——新鲜度是后端铸造、策略插件记录的版本令牌。 -`@deepseek-ai/dsh-tool-fs` 依赖 `@deepseek-ai/dsh-fs`、`@deepseek-ai/dsh-tools`、`@deepseek-ai/dsh-system-prompt` 和 `cordis`。它注册面向模型的工具和提示词段落。它禁止导入 `node:fs`、`node:path` 或 `@deepseek-ai/dsh-fs-local`;文件系统执行始终通过 `ctx.fs`。如果实现需要具体的 agent(智能体)或会话辅助类型,这些依赖属于 `tool-fs`;它们禁止回漏到 `dsh-fs` 中。 +`@deepseek-ai/dsh-tool-fs` 依赖 `@deepseek-ai/dsh-fs`、`@deepseek-ai/dsh-tools`、`@deepseek-ai/dsh-system-prompt` 和 `cordis`。它注册面向模型的工具和提示词段落。它禁止导入 `node:fs`、`node:path` 或 `@deepseek-ai/dsh-fs-local`;文件系统执行始终通过 `ctx.fs`。如果实现需要具体的 agent 或会话辅助类型,这些依赖属于 `tool-fs`;它们禁止回漏到 `dsh-fs` 中。 根 `tool-fs` 插件通过组合各工具的注册辅助函数来注册完整的文件系统工具套件(`read`、`write` 和 `edit`)。它注入 `fs`,从不导入实现包。 @@ -62,8 +62,9 @@ harness 已有一个具体的 `bash` 能力 seam(`dsh-bash` / `dsh-bash-local` 该接口涵盖以下语义操作: - 将模型/插件提供的路径解析为后端定义的目标。 +- 将解析后的目标转换为同一执行环境的规范进程路径或 `file:` URI,并在不解析其不透明键的情况下检查包含关系。 - 获取目标元数据而不读取文件内容。 -- 从目标读取有界的 UTF-8 文本页。 +- 读取完整或流式 UTF-8 文本,其中包括一项持有稳定句柄、以字节为上限的全文件读取操作。 - 创建或替换一个 UTF-8 文本文件。 - 通过字面替换编辑一个已有的 UTF-8 文本文件。 @@ -83,13 +84,15 @@ harness 已有一个具体的 `bash` 能力 seam(`dsh-bash` / `dsh-bash-local` - 不透明的 `targetKey`,用于陈旧守护和文件状态查找。本地后端可能使用类似 realpath 的键;远程后端可能使用工作区 URI 或文件 id。消费方禁止解析或假设它是本地绝对路径。 - `displayPath`,用于面向模型/UI 的输出。根据后端不同,它可能是本地绝对路径、工作区相对路径或远程 URI。 -读取和变更结果必须包含不透明的文件 `version`。本地后端从 bigint stat 元数据(`dev`、`ino`、`size`、`mtimeNs` 和 `ctimeNs`)派生令牌,因此同大小重写和 inode 替换都会可靠地使消费方持有的版本失效;远程后端可以使用 revision id 或类似 hash 的令牌。`dsh-fs-policy` 插件记录版本用于陈旧检查;消费方可以展示相关元数据但禁止解释版本令牌。 +即使另一项能力共享提供方的执行环境,`targetKey` 仍保持不透明。这类消费方通过提供方的 `processPath(target)`、`fileUrl(target)` 或 `contains(parent, child)` 获取所需事实;[可移植执行环境决策](2026-07-28-portable-execution-world-consumers.md)说明这些事实为何属于文件系统 seam。 -提供方返回已解码的文本:`readText` 返回整个常规文本文件,`streamText` 为大文件流式输出相同的文本语义。两者负责常规文件检查;有界的行/输出处理不是它们的职责——行窗口化、带行号渲染和总行数统计位于执行器(`dsh-tool-fs`)中,执行器通过 `ctx.fs` 读取并渲染面向模型的窗口。提供方负责 UTF-8 解码和二进制/NUL 拒绝;它不知道行窗口或视图。 +读取和变更结果必须包含不透明的文件 `version`。本地后端从 bigint stat 元数据(`dev`、`ino`、`size`、`mtimeNs` 和 `ctimeNs`)派生令牌,因此同大小重写和 inode 替换都会可靠地使消费方失效;远程后端可以使用 revision id 或类似 hash 的令牌。`dsh-fs-policy` 插件记录版本用于陈旧检查;消费方可以展示相关元数据但禁止解释版本令牌。 + +提供方返回已解码的文本:`readText` 返回整个常规文本文件,`streamText` 为大文件流式传输相同的文本语义,`readTextBounded` 则持有一个归后端所有的稳定句柄,并在完整文件超过字节上限时拒绝。行窗口化、带行号渲染和总行数统计位于执行器(`dsh-tool-fs`)中。提供方负责普通文件检查、UTF-8 解码、二进制/NUL 拒绝,以及有界读取期间的路径替换/增长竞态;它不知道行窗口或视图。 观测状态记录不在 `ctx.fs` 上:成功读取后,执行器发出 `fs/observed`,`dsh-fs-policy` 插件为推导出的 owner 记录 `{ version }`。没有 `full`/`partial` 视图——任何窗口的读取都记录版本,新鲜度(而非视图完整性)授权后续的写入/编辑。 -全文件写入创建或替换 UTF-8 文本文件。后端在支持且有文档说明时可以创建父目录。已有的非常规目标被拒绝。`writeText` 接受一个可选期望:`createIfAbsent` 创建缺失的目标并拒绝已存在的(报 `FS_NOT_OBSERVED`,这是策略处理未观测 owner 时采用的分支);`replaceIfVersion` 仅在目标处于观测版本时替换,否则报 `FS_STALE_VERSION`;省略期望则为无条件的裸提供方创建或覆盖。策略插件根据 owner 的观测状态选择提供哪个期望。 +全文件写入创建或替换 UTF-8 文本文件。后端在支持且有文档说明时可以创建父目录。已有的非常规目标被拒绝。`writeText` 接受一个可选期望:`createIfAbsent` 创建缺失的目标并拒绝已存在的(报 `FS_NOT_OBSERVED`,这是策略为未观测 owner 使用的路径);`replaceIfVersion` 仅在目标处于观测版本时替换,否则报 `FS_STALE_VERSION`;省略期望则为无条件的裸提供方创建或覆盖。策略插件根据 owner 的观测状态选择提供哪个期望。 字面编辑是提供方原语(`editText`),而非在 `tool-fs` 中由读取加写入组合而成。字面匹配、重复匹配拒绝、CRLF 保留、二进制拒绝、可选的陈旧版本检查和原子读-改-写必须一起留在后端的变更临界区内。`editText` 接受相同的可选版本期望;陈旧检查在字面匹配之前运行,因此基于旧读取的编辑会报 `FS_STALE_VERSION`。远程后端可以将编辑实现为原生的 compare-and-edit 操作;消费方不强制本地风格的组合。 @@ -124,11 +127,11 @@ harness 已有一个具体的 `bash` 能力 seam(`dsh-bash` / `dsh-bash-local` ## 测试 -测试遵循包边界,而不仅是用户可见的工具:`dsh-fs` 中的服务 seam;`dsh-fs-local` 中通过 `ctx.fs` 接口测试的真实文件系统行为(解析、符号链接、流式输出、二进制/UTF-8 拒绝、无条件和版本守护的写入、字面编辑语义、行尾保留、结构化 `FsError` 错误码);`dsh-tool-fs` 中基于真实本地提供方的消费方接口(只 mock 模型/时钟,从不 mock 协作者);以及通过 `ctx.tools.execute()` 在有和没有 `dsh-fs-policy` 的情况下进行集成测试,通过从磁盘回读文件来验证世界状态,既不信任规范值,也不信任渲染内容。观测状态/owner 推导策略在 `dsh-fs-policy` 中测试,不在此处。 +测试遵循包边界,而不仅是用户可见的工具:`dsh-fs` 中的服务 seam;`dsh-fs-local` 中通过 `ctx.fs` 接口测试的真实文件系统行为(解析、符号链接、流式传输、二进制/UTF-8 拒绝、无条件和版本守护的写入、字面编辑语义、行尾保留、结构化 `FsError` 错误码);`dsh-tool-fs` 中基于真实本地提供方的消费方接口(只 mock 模型/时钟,从不 mock 协作者);以及通过 `ctx.tools.execute()` 在有和没有 `dsh-fs-policy` 的情况下进行集成测试,通过从磁盘回读文件来验证世界状态,既不信任规范值,也不信任渲染内容。观测状态/owner 推导策略在 `dsh-fs-policy` 中测试,不在此处。 本仓库曾踩过的防御性模式类别被直接固定: -- **原子写入临时文件安全。** 写入/编辑通过目标旁边一个私有随机 `0700` 目录中独占且仅所有者可访问(`'wx'`、`0o600`)的临时文件暂存,失败时清理,最后原子 rename——与 bash spill 文件规则一致,因为可预测的全局可读临时路径会招致符号链接竞争和信息泄露。测试断言权限,并断言已存在的临时路径不会被覆盖;此原语是 seam 的常设要求。 +- **原子写入临时文件安全。** 写入/编辑通过目标旁边一个私有随机 `0700` 目录中的独占 owner-only(`'wx'`、`0o600`)临时文件暂存,失败时清理,最后原子 rename——与 bash 溢出文件规则一致,因为可预测的 world-readable 临时路径招致符号链接竞争和信息泄露。测试断言权限,并断言已存在的临时路径不会被覆盖;此原语是 seam 的常设要求。 - **通过符号链接的 `targetKey` 同一性。** 两个输入路径解析到同一 realpath 时共享一个观测状态条目:通过路径 A 的 `read` 满足通过符号链接路径 B 的 `edit` 的读后编辑守护,通过一个路径的陈旧写入可通过另一个路径检测到。 - **并发/陈旧竞争。** 对同一目标的两个并发写入/编辑操作确定性地收敛——一个成功,另一个被 `FS_STALE_VERSION` 拒绝——成功的编辑刷新记录状态,使同一 owner 的下一次编辑可以继续。 - **HMR(热模块替换)安全与 dispose(资源释放)。** dispose 后端的 fiber 会撤回 `ctx.fs` 提供方;后续的提供方以无继承状态启动。 diff --git a/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.i18n.yaml index 11073946ea..afb9251d33 100644 --- a/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md -2026-07-15-lsp-capability-seam.md: d96b3a9c5139c1455a51f4fff793293d7b5a11c0 -2026-07-15-lsp-capability-seam.zh.md: 256b293f213acb06588f3ef6c655242b1c8fd5b9 +2026-07-15-lsp-capability-seam.md: a9dab45ebf034f6d5218b25a0868a7e86803719b +2026-07-15-lsp-capability-seam.zh.md: b411a2456ea2a3838e56422f58954473da51ae23 diff --git a/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md b/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md index d96b3a9c51..a9dab45ebf 100644 --- a/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md +++ b/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md @@ -79,7 +79,7 @@ interface LspService { Mapping keys normalize to lowercase, leading-dot extensions selected from `filePath`'s final extension; language ids only synchronize documents. Seam positions and ranges are zero-based UTF-16. `findReferences` always includes declarations: providers enforce this internally, the local mapping sets `context.includeDeclaration: true`, and callers get no flag. Closed result unions normalize navigation to locations and hover to content or `null`; navigation results carry the provider's resolved workspace root so consumers relativize file URIs in the same canonical namespace. The seam exposes no protocol types, process or document controls, or generic request escape hatch. -`dsh-lsp-local` owns host files, server configuration, JSON-RPC, process and transient-document state, and protocol translation; it depends on `dsh-lsp` and Node APIs, not `dsh-fs`. The server-table key is its provider id. The plugin resolves every server-local setting before registration, rolls back earlier registrations if a later mapping is invalid or conflicts, and retains an independent process pool per provider. `dsh-tool-lsp` runtime-injects only `tools`, `lsp`, and `systemPrompt`, obtains the workspace from `exec.agent?.session.header.cwd` through a package-local `sessionCwd(exec)` helper matching the filesystem tools' lookup, and imports no provider. +`dsh-lsp-local` owns server configuration, JSON-RPC, process and transient-document state, and protocol translation. It reads through `ctx.fs` and launches through `ctx.subprocess`, depending on their interface packages rather than concrete providers; the [portable execution-world decision](2026-07-28-portable-execution-world-consumers.md) owns that pairing. The server-table key is its provider id. The plugin resolves every server-local setting before registration, rolls back earlier registrations if a later mapping is invalid or conflicts, and retains an independent process pool per provider. `dsh-tool-lsp` runtime-injects only `tools`, `lsp`, and `systemPrompt`, obtains the workspace from `exec.agent?.session.header.cwd` through a package-local `sessionCwd(exec)` helper matching the filesystem tools' lookup, and imports no provider. ## Model-facing contract @@ -112,26 +112,26 @@ Provider disposal occurs outside tool execution, so `dsh-lsp-local` keeps `shutd ## Workspace, filesystem, and document synchronization -`dsh-lsp-local` canonicalizes and reads through Node APIs in the subprocess's host namespace. It rejects missing, non-regular, non-UTF-8, oversized, or canonical out-of-workspace sources and keeps one `O_NOFOLLOW | O_NONBLOCK` handle through validation and reading, so a FIFO with no writer cannot block before the regular-file check. It observes caller cancellation around each filesystem operation. It does not consume `ctx.fs` or emit `fs/observed`: only the LSP result is model-visible, so the query does not satisfy read-before-write policy. +`dsh-lsp-local` canonicalizes and reads through `ctx.fs` in the language server's execution world. It requires the workspace target to be a directory, rejects out-of-workspace sources through provider-owned containment, and uses `readTextBounded` so regular-file validation, UTF-8 decoding, the byte ceiling, and path replacement/growth safety stay one filesystem operation. It observes caller cancellation around each provider operation. It does not emit `fs/observed`: only the LSP result is model-visible, so the query does not satisfy read-before-write policy. The `read` tool is unsuitable source because its output is windowed, numbered, transcript-visible, and observed. Reading in `tool-lsp` would also assign provider-specific synchronization to the consumer and preclude non-local providers. The local provider uses a compatibility-first transient-open sequence for every query. It accepts legacy `textDocumentSync` `Full` or `Incremental`, or options with `openClose: true`; omitted, `None`, or explicitly incompatible synchronization fails as unsupported before `didOpen`. -1. Canonicalize and validate the host path, then read the current source with Node filesystem APIs. +1. Resolve and contain the source through `ctx.fs`, then read its current bounded text through the same provider. 2. Send `textDocument/didOpen` with version `1`, full text, and the configured language id. Its write remains abortable; failure or cancellation invalidates the instance and awaits bounded process termination before the pool can reuse it. 3. Send the requested `textDocument/definition`, `textDocument/references`, `textDocument/implementation`, or `textDocument/hover` request. 4. If `didOpen` succeeded, attempt `textDocument/didClose` in `finally` after the request settles or aborts. A close-write failure does not replace the settled result or error, but invalidates the instance and awaits bounded process termination. Documents close after each call, so the first version needs no `didChange`, `didSave`, content cache, mutation listener, or document LRU. One abortable per-workspace provider queue serializes source-read/open/query/close lifecycles, so a waiting query reads current bytes only when its turn starts; the instance also keeps protocol lifecycles serialized. Distinct workspaces may run in parallel. The server's workspace index remains responsible for closed files reached from the source. -The canonical workspace `realpath` must be a directory and supplies process cwd, `rootUri`, the sole `workspaceFolders` entry, and pool identity; symlink aliases therefore share an instance. Result locations may be external, but an external path cannot become a query source. Remote, virtual, or independently sandboxed filesystems require another provider. +The canonical workspace target must be a directory. Its target key supplies pool identity, its process path supplies cwd, and its provider-owned `file:` URI supplies `rootUri` and the sole `workspaceFolders` entry; aliases share an instance when the filesystem provider resolves them to one key. Result locations may be external, but an external path cannot become a query source. A filesystem that cannot share paths with the mounted subprocess provider is a composition error, not a reason for another LSP package. ## Local server lifecycle and protocol behavior -`dsh-lsp-local` lazily single-flights one server per `(provider id, canonical workspace realpath)`. At load it resolves the executable after credential scrubbing and environment overrides, failing before registration if unavailable; server process launch stays lazy (first query spawns it) and uses no shell. `maxMessageBytes` defaults to `16_000_000`, `maxStderrBytes` to `1_000_000`, and `maxDocumentBytes` to `4_000_000`. A crash fails the active query without replay; a later query may replace the process. Each query starts at most one process, so the MVP has no cross-request restart counter. +`dsh-lsp-local` lazily single-flights one server per `(provider id, canonical workspace target)`. At load it calls `ctx.subprocess.resolveExecutable()` with the configured environment, failing before registration if unavailable; first query launches through raw protocol pipes with no shell and a bounded collected stderr tail. `maxMessageBytes` defaults to `16_000_000`, `maxStderrBytes` to `1_000_000`, and `maxDocumentBytes` to `4_000_000`. A crash fails the active query without replay; a later query may replace the process. Each query starts at most one process, so the MVP has no cross-request restart counter. -Initialization advertises `general.positionEncodings: ['utf-16']`, `workspace: { workspaceFolders: true, configuration: true }`, `textDocument.hover.contentFormat: ['markdown', 'plaintext']`, and `linkSupport: true` for definition and implementation, with no dynamic registration. Returned operation and synchronization capabilities are authoritative. An omitted server `positionEncoding` defaults to `utf-16`; any other value is a protocol error. Configuration may supply initialization options and `workspace/configuration` responses, but the client rejects `workspace/applyEdit` and never executes commands or edits. +Initialization uses `processId: null` because the client and server may inhabit different process namespaces. It advertises `general.positionEncodings: ['utf-16']`, `workspace: { workspaceFolders: true, configuration: true }`, `textDocument.hover.contentFormat: ['markdown', 'plaintext']`, and `linkSupport: true` for definition and implementation, with no dynamic registration. Returned operation and synchronization capabilities are authoritative. An omitted server `positionEncoding` defaults to `utf-16`; any other value is a protocol error. Configuration may supply initialization options and `workspace/configuration` responses, but the client rejects `workspace/applyEdit` and never executes commands or edits. Navigation maps `Location` directly and `LocationLink` from `targetUri` plus `targetSelectionRange`. Positions must be nonnegative integers. Hover normalization accepts only valid `MarkupContent` and `MarkedString` shapes, preserves string values, renders language-tagged values as fenced code, and joins arrays with one blank line. The model-facing tool applies `maxResultChars` after rendering. @@ -143,7 +143,7 @@ Symbols are deferred because they need different schemas and overlap read/search Diagnostics need separate freshness, accumulation, and transcript rules. Mutations such as rename, code actions, and formatting require separate tools with preview, permission, and write-policy integration. -The local provider trusts its configured server and claims no sandbox confinement. Supporting untrusted binaries requires a later process/filesystem contract for workspace reads plus private cache and temporary writes; restricted, remote, or virtual workspaces require another provider. +The provider trusts its configured server. Its filesystem visibility and process confinement are exactly those of the mounted execution world; LSP adds no independent sandbox policy. ## Alternatives considered @@ -157,7 +157,7 @@ The local provider trusts its configured server and claims no sandbox confinemen **Wrap the signal in a per-seam execution-context object.** Web passes a bare `AbortSignal`; wrapping this single field would add unexplained asymmetry. `query()` gains a context object only when another field requires it. -**Read through `ctx.fs` or the `read` tool.** This could mix the document with a server index from another filesystem namespace; tool output is also windowed, numbered, and observed. The host-local provider reads unobserved full text beside its subprocess. +**Read through the model-facing `read` tool.** Rejected because tool output is windowed, numbered, transcript-visible, and observed. The provider reads bounded full text directly through the same `ctx.fs` execution world used by its subprocess. **Keep documents open.** Mirroring edits requires version ownership, all-path `didChange`, HMR recovery, eviction, and stale-state rules. Transient opens avoid that MVP state machine. @@ -180,7 +180,7 @@ The local provider trusts its configured server and claims no sandbox confinemen - Synchronization tests pin UTF-16 negotiation and conversion, supported and rejected `textDocumentSync` forms, blocked and failed open writes, balanced transient open/close, close-write failure, and malformed-response rejection. - Timeout tests pin one `TOOL_TIMEOUT` budget, unclassified upstream cancellation, no hidden seam deadline, and bounded awaited teardown. - Lifecycle tests pin startup single-flight, complete-lifecycle serialization with fresh queued source reads, cross-workspace parallelism, abortable queues, crash replacement without replay, failed-stdin teardown, and quiescent disposal. -- Host-filesystem tests pin session-cwd requirements, relative and absolute source containment through symlinks, document validation, file/non-file URI rendering, unformatted source, and no `fs/observed` event. +- Filesystem-host tests pin session-cwd requirements, provider-owned containment and URI rendering, bounded document reads, unformatted source, and no `fs/observed` event. - A keyless pinned TypeScript real-server e2e exercises all four operations; runnable configuration uses the same explicit provider mapping. - Snapshots cover model-visible schema, prompt, results, and omissions; a built-artifact smoke test covers framing and cleanup. - Package and architecture docs cover configuration, security boundaries, and search/read guidance; the new `packages/lsp/` group is added to the AGENTS.md repository-layout block, the packages/README.md group table, and architecture.md in the same change. @@ -195,4 +195,4 @@ Extension ownership is exclusive within one runtime. Two providers cannot both c UTF-16 cursor columns are exact for the protocol but difficult for a model to count around non-BMP characters. Invalid or off-symbol positions may produce empty results, so error text and prompt examples must explain the coordinate convention without encouraging broad LSP use. -Direct Node access aligns the query snapshot with the server index but bypasses `ctx.fs` and its policy. Canonical containment rejects source files outside the workspace; a trusted server may still read the workspace and use caches. The first implementation therefore requires trusted host-local deployment and provides no sandbox guarantee. +The paired filesystem/subprocess providers align the query snapshot with the server index but do not make a trusted language server safe. Canonical containment rejects query sources outside the workspace; the server itself receives the execution world's configured authority and may read other paths or use caches. diff --git a/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.zh.md b/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.zh.md index 256b293f21..b411a2456e 100644 --- a/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.zh.md @@ -1,4 +1,4 @@ -# Agent Note: LSP 能力 seam 与面向模型的查询工具 +# Agent Note: LSP 能力服务边界与面向模型的查询工具 Status: implemented @@ -14,7 +14,7 @@ harness 已具备文本搜索与文件读取能力,但二者都无法识别程 ## 决策 -将 LSP 建成由三个包组成的能力 seam,其中包含一个只读模型工具和一个通用本地提供方实现: +将 LSP 建成由三个包(package)组成的能力服务边界,其中包含一个只读模型工具和一个通用本地提供方实现: 1. `packages/lsp/lsp` 下的 `@deepseek-ai/dsh-lsp` 负责 `ctx.lsp`、提供方注册与选择、标准化请求与结果、执行控制,以及结构化 LSP 错误。 2. `packages/lsp/lsp-local` 下的 `@deepseek-ai/dsh-lsp-local` 将配置的 stdio 语言服务器适配到该服务边界。一个插件实例接收具名服务器表,并为每组命令及扩展名到语言 id 的映射注册一个隔离的提供方。 @@ -79,7 +79,7 @@ interface LspService { 映射键规范化为带前导点的小写扩展名,并按 `filePath` 的最后一个扩展名选择;语言 id 仅用于文档同步。服务边界中的位置和范围从零开始按 UTF-16 计数。`findReferences` 始终包含声明:提供方在内部执行该约束,本地映射设置 `context.includeDeclaration: true`,调用方不能配置。封闭结果联合将导航统一为位置,将 `hover` 统一为内容或 `null`;导航结果携带提供方解析后的工作区根目录,使消费方依据同一规范化根目录相对化文件 URI。服务边界不公开协议类型、进程或文档控制,也不提供通用请求逃生口。 -`dsh-lsp-local` 负责主机文件、服务器配置、JSON-RPC、进程与临时文档状态和协议转换;它依赖 `dsh-lsp` 与 Node API,不依赖 `dsh-fs`。服务器表的键是提供方 id。插件在注册前解析每个服务器的本地设置;如果后续映射无效或发生冲突,插件会撤销此前的注册,并为每个提供方保留独立进程池。`dsh-tool-lsp` 在运行时只注入 `tools`、`lsp` 和 `systemPrompt`,通过包内的 `sessionCwd(exec)` 辅助函数从 `exec.agent?.session.header.cwd` 取得工作区,其取值方式与文件系统工具一致,也不导入提供方。 +`dsh-lsp-local` 负责服务器配置、JSON-RPC、进程与临时文档状态和协议转换。它通过 `ctx.fs` 读取,通过 `ctx.subprocess` 启动,只依赖二者的接口包而非具体提供方;[可移植执行环境决策](2026-07-28-portable-execution-world-consumers.md)负责定义这种配对。服务器表的键是提供方 id。插件在注册前解析每个服务器的本地设置;如果后续映射无效或发生冲突,插件会撤销此前的注册,并为每个提供方保留独立进程池。`dsh-tool-lsp` 在运行时只注入 `tools`、`lsp` 和 `systemPrompt`,通过包内的 `sessionCwd(exec)` 辅助函数从 `exec.agent?.session.header.cwd` 取得工作区,其取值方式与文件系统工具一致,也不导入提供方。 ## 面向模型的契约 @@ -98,9 +98,9 @@ interface LspToolInput { 工具必须从会话 `header.cwd` 取得 `workspaceRoot`,没有后备值;缺失时在查询或启动前以 `LSP_WORKSPACE_REQUIRED` 失败。本地提供方基于根目录解析相对路径并直接接受绝对路径;两种路径都会进行规范化,如果目标位于规范工作区外,则在启动前拒绝。 -位置按文件稳定分组并渲染为 `path:line:character`。Node `fileURLToPath()` 可接受的 `file:` URI 在工作区内转换为相对路径,在工作区外转换为绝对路径;其他 URI 保持原样。`maxLocations` 默认值为 `100`,并报告省略的条目;`maxResultChars` 默认值为 `16_000`,并将每个完整渲染结果(包括截断元数据)限制在该字符数内。空位置与 `null` hover 是成功的无结果响应;服务器载荷缺失或格式错误时,以结构化 `LSP_MALFORMED_RESPONSE` 错误失败。 +位置按文件稳定分组并渲染为 `path:line:character`。Node `fileURLToPath()` 可接受的 `file:` URI 在工作区内转换为相对路径,在工作区外转换为绝对路径;其他 URI 保持原样。`maxLocations` 默认值为 `100`,并报告省略的条目;`maxResultChars` 默认值为 `16_000`,并限制每个完整渲染结果,其中包括截断元数据。空位置与 `null` hover 是成功的无结果响应;服务器载荷缺失或格式错误时,以结构化 `LSP_MALFORMED_RESPONSE` 错误失败。 -与传输方式无关的展示器使用 `{ card: 'generic', kind: 'search', title, locations: [{ path: file_path, line }] }`,`title` 由参数推导并标明操作与光标。由于 `FileLocation` 没有 character,跟随位置聚焦输入行,标题保留完整光标;展示器仍为纯函数。 +与传输方式无关的展示器使用 `{ card: 'generic', kind: 'search', title, locations: [{ path: file_path, line }] }`,`title` 由参数推导并标明操作与光标。由于 `FileLocation` 没有 character,跟随位置聚焦输入行,标题保留完整光标;展示保持纯函数。 ## 超时归属 @@ -112,26 +112,26 @@ interface LspToolInput { ## 工作区、文件系统与文档同步 -`dsh-lsp-local` 通过 Node API 在子进程所在的主机命名空间中规范化并读取文件。它拒绝缺失、非普通、非 UTF-8、超大或规范路径越出工作区的源文件,并在校验与读取期间保持同一个 `O_NOFOLLOW | O_NONBLOCK` 句柄,因此没有写入方的 FIFO 不会在普通文件校验前造成阻塞。它在每项文件系统操作前后检查调用方是否取消。它不使用 `ctx.fs` 或发送 `fs/observed`:只有 LSP 结果对模型可见,因此查询不满足写前读取策略。 +`dsh-lsp-local` 在语言服务器的执行环境中通过 `ctx.fs` 规范化并读取文件。它要求工作区目标是目录,使用提供方自有的 containment 拒绝工作区外的源文件,并通过 `readTextBounded` 把普通文件校验、UTF-8 解码、字节上限和路径替换/增长安全性保留在同一项文件系统操作中。它在每项提供方操作前后检查调用方是否取消。它不发送 `fs/observed`:只有 LSP 结果对模型可见,因此查询不满足写前读取策略。 `read` 工具的输出带窗口与行号,进入 transcript(文本记录)且已被观察,不适合作为源文件。在 `tool-lsp` 内读取还会把提供方专用同步职责交给消费方,并排除非本地提供方。 本地提供方对每次查询都采用兼容优先的临时打开流程。它接受旧式 `textDocumentSync` 的 `Full` 或 `Incremental`,也接受设置了 `openClose: true` 的选项;同步能力缺失、为 `None` 或明确不兼容时,在 `didOpen` 前以不支持错误失败。 -1. 规范化并校验主机路径,再使用 Node 文件系统 API 读取当前源文件。 +1. 通过 `ctx.fs` 解析源文件并检查其位于工作区内,再通过同一提供方对当前文本进行有界读取。 2. 发送 `textDocument/didOpen`,其中包含版本 `1`、完整文本和配置的语言 id。该写入仍可取消;写入失败或遭取消会使实例失效,并等待有界进程终止完成,池才能复用它。 3. 发送所请求的 `textDocument/definition`、`textDocument/references`、`textDocument/implementation` 或 `textDocument/hover` 请求。 4. 如果 `didOpen` 成功,则在请求完成或取消后于 `finally` 中尝试发送 `textDocument/didClose`。关闭写入失败不会覆盖已经确定的结果或错误,但会使实例失效,并等待有界进程终止完成。 每次调用后都关闭文档,因此第一版不需要 `didChange`、`didSave`、内容缓存、变更监听器或文档 LRU。每个工作区的提供方队列可取消,并串行执行源文件读取、打开、查询和关闭的完整生命周期,因此等待中的查询只在轮到它时才读取当前字节;实例也会串行执行协议生命周期。不同工作区可以并行。服务器工作区索引仍负责从源文件跳转到的已关闭文件。 -规范工作区 `realpath` 必须是目录,并用于进程 cwd、`rootUri`、唯一的 `workspaceFolders` 条目和进程池 identity;符号链接别名因此共享实例。结果位置可以在工作区外,但外部路径不能成为查询源。远程、虚拟或独立沙箱化文件系统需要另一种提供方。 +规范工作区目标必须是目录。其目标键提供进程池 identity,进程路径提供 cwd,归提供方所有的 `file:` URI 则提供 `rootUri` 和唯一的 `workspaceFolders` 条目;文件系统提供方将别名解析为同一键时,它们共享实例。结果位置可以在工作区外,但外部路径不能成为查询源。无法与挂载的子进程提供方共享路径的文件系统属于组合错误,不是另建 LSP 包的理由。 ## 本地服务器生命周期与协议行为 -`dsh-lsp-local` 按 `(provider id, canonical workspace realpath)` 懒启动一个服务器,并通过 single-flight 合并启动。插件加载时,它在清除凭据并应用环境变量覆盖后解析可执行文件;命令不可用时在注册前失败。服务器进程的启动保持懒执行(首次查询时才拉起),且不经过 shell。`maxMessageBytes` 默认值为 `16_000_000`,`maxStderrBytes` 默认值为 `1_000_000`,`maxDocumentBytes` 默认值为 `4_000_000`。崩溃使当前查询失败且不重放;后续查询可以替换进程。每次查询最多启动一个进程,因此 MVP 不设置跨请求重启计数器。 +`dsh-lsp-local` 按 `(provider id, canonical workspace target)` 懒启动一个服务器,并通过 single-flight 合并启动。插件加载时,它使用已配置的环境调用 `ctx.subprocess.resolveExecutable()`;命令不可用时在注册前失败。首次查询通过原始协议管道启动服务器,不经过 shell,并收集有界的 stderr 尾部。`maxMessageBytes` 默认值为 `16_000_000`,`maxStderrBytes` 默认值为 `1_000_000`,`maxDocumentBytes` 默认值为 `4_000_000`。崩溃使当前查询失败且不重放;后续查询可以替换进程。每次查询最多启动一个进程,因此 MVP 不设置跨请求重启计数器。 -初始化声明 `general.positionEncodings: ['utf-16']`、`workspace: { workspaceFolders: true, configuration: true }`、`textDocument.hover.contentFormat: ['markdown', 'plaintext']`,以及 definition 与 implementation 的 `linkSupport: true`,但不支持动态注册。服务器返回的操作能力与同步能力均为真源。服务器省略 `positionEncoding` 时默认为 `utf-16`;其他值均属于协议错误。配置可以提供初始化选项和 `workspace/configuration` 响应,但客户端拒绝 `workspace/applyEdit`,绝不执行命令或编辑。 +初始化使用 `processId: null`,因为客户端与服务器可能位于不同的进程命名空间。它声明 `general.positionEncodings: ['utf-16']`、`workspace: { workspaceFolders: true, configuration: true }`、`textDocument.hover.contentFormat: ['markdown', 'plaintext']`,以及 definition 与 implementation 的 `linkSupport: true`,但不支持动态注册。服务器返回的操作与同步能力均为真源。服务器省略 `positionEncoding` 时默认为 `utf-16`;其他值均属于协议错误。配置可以提供初始化选项和 `workspace/configuration` 响应,但客户端拒绝 `workspace/applyEdit`,绝不执行命令或编辑。 导航结果直接映射 `Location`,并将 `LocationLink` 的 `targetUri` 与 `targetSelectionRange` 映射为统一位置。位置必须是非负整数。`hover` 归一化只接受有效的 `MarkupContent` 和 `MarkedString` 结构,保留字符串值,把带语言标签的值渲染为围栏代码块,并以一个空行连接数组。面向模型的工具在渲染后应用 `maxResultChars`。 @@ -143,7 +143,7 @@ interface LspToolInput { 诊断需要独立的新鲜度、累积与 transcript 规则。重命名、代码操作和格式化等变更能力需要单独工具,并集成预览、权限和写入策略。 -本地提供方信任配置的服务器,不声称具备沙箱隔离。支持不受信任的二进制文件需要后续补充允许读取工作区,并执行私有缓存写入与临时写入的进程/文件系统契约;受限、远程或虚拟工作区需要另一种提供方。 +提供方信任配置的服务器。其文件系统可见性与进程隔离完全取决于挂载的执行环境;LSP 不增加独立的沙箱策略。 ## 备选方案 @@ -155,9 +155,9 @@ interface LspToolInput { **公开 `resolve(request)` / `query(spec)`。** 没有需要填充默认值的字段时,resolve 只会暴露提供方选择,而公开 spec 可能活过提供方释放或替换。单一操作让选择与调用共用注册生命周期。 -**将信号包装为服务边界专用的执行上下文对象。** Web 传递裸 `AbortSignal`;仅包装这一个字段会造成无谓的不对称。只有另一个字段确有需要时,`query()` 才引入上下文对象。 +**将信号包装为每服务边界的执行上下文对象。** Web 传递裸 `AbortSignal`;仅包装这一个字段会造成无谓的不对称。只有另一个字段确有需要时,`query()` 才引入上下文对象。 -**通过 `ctx.fs` 或 `read` 工具读取。** 这可能把文档与另一文件系统命名空间中的服务器索引混合;工具输出还带窗口、行号且已被观察。host-local 提供方在子进程旁读取未观察的完整文本。 +**通过面向模型的 `read` 工具读取。**拒绝,因为工具输出带窗口与行号,会进入 transcript 且已被观察。提供方直接通过子进程所用的同一 `ctx.fs` 执行环境读取有界的完整文本。 **保持文档打开。** 镜像编辑需要版本归属、覆盖所有路径的 `didChange`、HMR 恢复、淘汰和陈旧状态规则。临时打开避免在 MVP 引入这套状态机。 @@ -180,14 +180,14 @@ interface LspToolInput { - 同步测试固定 UTF-16 协商与转换、受支持和被拒绝的 `textDocumentSync` 形式、打开写入阻塞与失败、配对的临时打开/关闭、关闭写入失败和错误响应拒绝。 - 超时测试固定一个 `TOOL_TIMEOUT` 预算、不对上游取消错误分类、服务边界无隐藏截止时间,以及受限且等待完成的清理。 - 生命周期测试固定启动 single-flight、完整生命周期串行化及排队查询读取最新源文件、跨工作区并行、可取消队列、崩溃后不重放的替换、stdin 失败后的进程拆除,以及释放后完全停稳。 -- 主机文件系统测试固定 session cwd 要求、符号链接下相对与绝对源路径的规范 containment、文档校验、file/non-file URI 渲染、无格式源文本和不发送 `fs/observed`。 +- 文件系统宿主测试固定 session cwd 要求、提供方自有的 containment 与 URI 渲染、有界文档读取、无格式源文本和不发送 `fs/observed`。 - 无密钥且固定版本的 TypeScript 真实服务器 e2e 覆盖四种操作;可运行配置使用同一项显式提供方映射。 - 快照覆盖模型可见 schema、提示词、结果和省略提示;构建产物冒烟测试覆盖分帧与清理。 - 包与架构文档覆盖配置、安全边界和搜索/读取指导;同一改动中,新的 `packages/lsp/` 包组要加入 AGENTS.md 的仓库布局块、packages/README.md 的分组表和 architecture.md。 ## 影响 -各语言服务器对方法支持、能力解释和索引就绪时机的处理不同;LSP 没有统一的「索引完成」信号。不具备兼容临时打开同步能力的服务器不受支持,即使它能查询已关闭文档。受支持的服务器仍可能返回空结果或不完整结果,因此工具不承诺跨服务器完整性。固定的 TypeScript e2e 只建立一条兼容性基线,不代表跨语言承诺。 +各语言服务器对方法支持、能力解释和索引就绪时机的处理不同;LSP 没有统一的“索引完成”信号。无法声明兼容临时打开同步能力的服务器不受支持,即使它能查询已关闭文档。受支持的服务器仍可能返回空结果或不完整结果,因此工具不承诺跨服务器完整性。固定的 TypeScript e2e 只建立一条兼容性基线,不代表跨语言承诺。 临时打开会重复解析并产生通知。实例内串行会增加并发 agent 的延迟,长期运行的工作区进程则持续占用内存直到释放。 @@ -195,4 +195,4 @@ interface LspToolInput { UTF-16 光标列与协议完全一致,但模型难以在包含非 BMP 字符的文本中准确计数。无效位置或不在符号上的位置可能返回空结果,因此错误文本和提示词示例必须说明坐标约定,同时避免鼓励模型广泛使用 LSP。 -直接访问 Node 文件系统会对齐查询快照与服务器索引,但绕过 `ctx.fs` 及其策略。规范路径 containment 会拒绝工作区外的源文件;受信任的服务器仍可读取工作区并使用缓存。因此,第一版要求受信任的 host-local 部署,不提供沙箱保证。 +配对的文件系统/子进程提供方会对齐查询快照与服务器索引,但不会因此使受信任的语言服务器变得安全。规范 containment 会拒绝工作区外的查询源;服务器本身获得执行环境所配置的权限,仍可读取其他路径或使用缓存。 diff --git a/.agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.i18n.yaml index daa727ccb0..469c3f9a24 100644 --- a/.agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.md -2026-07-26-subprocess-consumer-migration.md: 477dffc2271db08b8986b34645067b247ad7ace7 -2026-07-26-subprocess-consumer-migration.zh.md: 8e0e377fc3fe00ff452803fe7fe5f4115003f937 +2026-07-26-subprocess-consumer-migration.md: abc7c8504595641b2b821df5ddbd4645c3a15336 +2026-07-26-subprocess-consumer-migration.zh.md: 500dcfe36f63315156b1a3cc48670eaf1c204b20 diff --git a/.agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.md b/.agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.md index 477dffc227..abc7c85045 100644 --- a/.agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.md +++ b/.agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.md @@ -6,7 +6,7 @@ English | [中文](2026-07-26-subprocess-consumer-migration.zh.md) ## Problem -The [subprocess seam](2026-07-26-subprocess-seam.md) shipped shaped for exactly one consumer family: batch-collected stdout/stderr, batch stdin, a single escalating `kill()`. That was deliberate scope control, and its own note records "migrate the other spawn sites" as rejected-for-now. Review on the introducing PR reversed that deferral: the stacked follow-up should reshape the interface toward Node's API and move the remaining process-running places onto the service. The remaining spawners each carried a private copy of some slice of the same mechanics — lsp-local had its own detached-tree signalling (POSIX group + Windows taskkill + liveness polling), subagent-subprocess had the dispose ladder and its own scrub, and mcp-client, pty-local, the SDK helper, and the TUI Git probe each carried another credential scrub — and none of it was swappable or centrally testable. +The [subprocess seam](2026-07-26-subprocess-seam.md) shipped shaped for exactly one consumer family: batch-collected stdout/stderr, batch stdin, a single escalating `kill()`. That was deliberate scope control, and its own note records "migrate the other spawn sites" as rejected-for-now. Review on the introducing PR reversed that deferral: the stacked follow-up should reshape the interface toward Node's API and move the remaining process-running places onto the service. The remaining spawners each carried a private copy of some slice of the same mechanics — lsp-local had its own detached-tree signalling (POSIX group + Windows taskkill + liveness polling), subagent-subprocess had the dispose ladder and its own scrub, mcp-client and pty-local and the SDK helper each had a third/fourth/fifth copy of the credential scrub — and none of it was swappable or centrally testable. ## Decision @@ -15,24 +15,24 @@ The seam's vocabulary is now Node-shaped, and every spawner that can ride the se - **Per-stream stdio dispositions** on `SubprocessSpawnSpec`: `'pipe'` (the raw `Readable`/`Writable`, for consumer-owned protocol framing), `'inherit'` (diagnostics to the parent's stream), and collect mode `{ maxBytes, spill? }` — the original bounded tail-keep shape, with the spill file now optional so a diagnostic tail (a language server's stderr) buffers without touching disk. stdin is `'ignore'`, `'pipe'`, or `{ data }` (write-and-close batch). - **`SubprocessOutcome` carries exit facts only** (Node's close-event vocabulary); collected output stays readable through `handle.collected` after settlement (spill fds seal at the settle boundary), so batch and streaming callers share one access path and nothing is copied into the outcome. - **Tree-scoped termination behind one verb**: `terminate()` owns the SIGTERM→grace→SIGKILL escalation (serves the spec's abort signal too, and is a no-op once the tree is gone) — the handle exposes no single-signal `kill(signal?)`, so a consumer cannot skip the grace window; `waitForExit()` polls tree liveness (POSIX group probe; direct-child boundary on Windows); Windows tree termination (`taskkill /T`, injectable) moved in from lsp-local, so tree semantics are platform-correct for every consumer. (The stdin-EOF-first dispose ladder initially absorbed from `subagent-subprocess` later moved back out to its one consumer — see the [ladder-ownership Agent Note](2026-07-27-dispose-ladder-to-consumer.md).) -- **One scrub definition**: `scrubbedParentEnv()`/`SENSITIVE_ENV_PATTERN` live on the seam, and credential-shaped names contain `KEY`, `PASSWORD`, `SECRET`, or `TOKEN`, case-insensitively. Spawners that cannot route the spawn itself through the service — pty-local (node-pty owns the fork), mcp-client (the MCP SDK owns the transport spawn), and the TUI's synchronous Git branch probe — import the function, so environment policy is single-sourced even where process ownership is not. The SDK helper's `scrubEnvironment()` defaults through the function and applies the exported pattern when its caller supplies an explicit environment; the pattern remains public for this production consumer. +- **One scrub definition**: `scrubbedParentEnv()`/`SENSITIVE_ENV_PATTERN` live on the seam. Ordinary and terminal local spawns apply it inside `dsh-subprocess-local`; mcp-client still imports it because the MCP SDK owns that transport spawn, and the SDK helper's `scrubEnvironment()` defaults through it as well. -Migrations landed with the reshape: **bash-local/bash-sandbox** (collect modes + batch stdin; the bash `kill()` maps to `terminate()` so `task_kill` keeps escalation semantics), **lsp-local** (piped protocol streams + a no-spill collected stderr tail; `LspConnection` takes the seam's spawn function; its private tree-op helpers deleted), **subagent-acp** (piped ndjson streams + inherited stderr; spawn failure surfaces through `done` rejection into the same startup race; disposal is the backend-owned `disposeAcpChild` ladder over the seam's verbs, with the plugin's configured graces). **`dsh-subagent-subprocess` is deleted** — the dispose ladder and scrub are the seam's; the unused isolated-config-dir helper died with it (no consumer existed). +Migrations landed with the reshape: **bash-local/bash-sandbox** (collect modes + batch stdin; the bash `kill()` maps to `terminate()` so `task_kill` keeps escalation semantics), **lsp-local** (piped protocol streams + a no-spill collected stderr tail; `LspConnection` takes the seam's spawn function; its private tree-op helpers deleted), **subagent-acp** (piped ndjson streams + inherited stderr; spawn failure surfaces through `done` rejection into the same startup race; disposal is the backend-owned `disposeAcpChild` ladder over the seam's verbs, with the plugin's configured graces), and **pty-local** through the later [portable execution-world decision](2026-07-28-portable-execution-world-consumers.md) (`node-pty` allocation and process inspection sit behind `spawnTerminal()`, while readiness and terminal policy remain in the consumer). **`dsh-subagent-subprocess` is deleted** — the dispose ladder and scrub are the seam's; the unused isolated-config-dir helper died with it (no consumer existed). Compositions mounting lsp-local or subagent-acp now load `dsh-subprocess-local` (the plugins inject `'subprocess'`); the acp/lsp test fixtures gained the row. ## Alternatives considered -**Keep the batch-only seam and let stream consumers stay bespoke.** The introducing note's position, rejected by review: it leaves three private copies of tree signalling and six of the scrub, and any future runner (containerized executor, remote process host) would have to pick which private copy to fork. The Node-shaped dispositions cover all three observed stream shapes without widening the outcome type or buffering piped streams. +**Keep the batch-only seam and let stream consumers stay bespoke.** The introducing note's position, rejected by review: it leaves three private copies of tree signalling and five of the scrub, and any future runner (containerized executor, remote process host) would have to pick which private copy to fork. The Node-shaped dispositions cover all three observed stream shapes without widening the outcome type or buffering piped streams. **A single `stdio: 'pipe' | 'inherit' | 'collect'` mode for all three streams at once.** Rejected: real consumers mix modes per stream (lsp: pipe/pipe/collect; acp: pipe/pipe/inherit; bash: data/collect/collect). Per-stream dispositions are exactly Node's shape and avoid a second spawn call for the mixed cases. -**Migrate pty-local and mcp-client spawns too.** Rejected on ownership grounds, not scope: node-pty's `fork()` allocates the terminal itself, and the MCP SDK's `StdioClientTransport` spawns internally — neither call site is ours to route. They adopt the shared scrub (the part that is policy), and their READMEs say why the spawn stays put. +**Keep pty-local and mcp-client spawns outside the service.** The MCP SDK still owns its transport spawn. PTY allocation is different: the [portable execution-world decision](2026-07-28-portable-execution-world-consumers.md) moves `node-pty` behind one deep terminal primitive, resolving the ownership objection without pretending an ordinary piped spawn can provide terminal semantics. -**Migrate the test-support launchers (acp-snapshot, loader-smoke), the SDK package-manager runner, and the TUI Git probe.** Rejected: the support packages are deliberately dependency-light test infrastructure that must not depend on product seams; the SDK wizard's `stdio: 'inherit'`-with-redirect semantics plus its out-of-composition lifecycle (no cordis context at all) make the service a poor fit; and the TUI probe is synchronous. The production callers share the scrub instead. +**Migrate the test-support launchers (acp-snapshot, loader-smoke) and the SDK package-manager runner.** Rejected: the support packages are deliberately dependency-light test infrastructure that must not depend on product seams, and the SDK wizard's `stdio: 'inherit'`-with-redirect semantics plus its out-of-composition lifecycle (no cordis context at all) make the service a poor fit; it shares the scrub instead. ## Consequences -Bought: one implementation of tree signalling, escalation, bounded collection, and the scrub, tested once in `dsh-subprocess-local`'s suites (including injected-platform Windows coverage that lsp-local's private copy never had); lsp-local and subagent-acp shed their process plumbing and their children now survive plugin reloads and die with composition teardown like bash's; a whole package (`dsh-subagent-subprocess`) is gone. The seam README's "one consumer family" limitation is retired. +Bought: one implementation of tree signalling, escalation, bounded collection, terminal process mechanics, and the scrub, tested once in `dsh-subprocess-local`'s suites (including injected-platform Windows coverage that lsp-local's private copy never had); lsp-local, pty-local, and subagent-acp shed provider-specific process plumbing and their children die with composition teardown like bash's; a whole package (`dsh-subagent-subprocess`) is gone. The seam README's "one consumer family" limitation is retired. -Cost: the seam is wider — three stdio modes and the terminate/waitForExit/dispose lifecycle surface instead of one mode and one verb — so a future backend implements more surface; the compositions for lsp-local/subagent-acp each carry the subprocess row now; and `SubprocessOutcome` no longer carries output, a breaking shape change inside the still-unreleased stack (the PR2 layer was updated in place rather than shimmed, per the pre-release stance). pty-local/mcp-client/SDK/TUI/test-support spawns remain outside the service by ownership or execution shape, with the scrub as the shared floor and its pattern intentionally exported for explicit-environment consumers. +Cost: the seam is wider — execution-world coordinates, executable lookup, three stdio modes, process-tree lifecycle, and one terminal primitive — so a future backend implements more surface; the compositions for lsp-local, pty-local, and subagent-acp each carry the subprocess row; and `SubprocessOutcome` no longer carries output, a breaking shape change inside the still-unreleased stack. MCP/SDK/test-support spawns remain outside the service by ownership, with the scrub as the shared floor. diff --git a/.agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.zh.md b/.agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.zh.md index 8e0e377fc3..500dcfe36f 100644 --- a/.agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.zh.md @@ -1,4 +1,4 @@ -# Agent Note: 子进程 seam 转向 Node 形状,所有具备条件的 spawn 调用点一并迁入 +# Agent Note: 进程 seam 转向 Node 形状,所有具备条件的 spawn 调用点一并迁入 Status: implemented @@ -6,7 +6,7 @@ Status: implemented ## 问题 -[子进程 seam](2026-07-26-subprocess-seam.md) 交付时恰好只为一个消费方家族塑形:批量收集的 stdout/stderr、批量 stdin、单一的升级式 `kill()`。那是有意的范围控制,其自身的 Agent Note 也把「迁移其余 spawn 调用点」记为暂缓否决项。引入该 seam 的 PR(Pull Request)上的评审推翻了这一暂缓决定:堆叠其上的后续变更应当把接口向 Node 的 API 方向重塑,并把其余运行进程之处迁到该服务上。其余各 spawn 调用点此前各自持有同一套机制中某个切片的私有副本——lsp-local 自带 detached 进程树信号发送(POSIX 进程组 + Windows taskkill + 存活轮询),subagent-subprocess 自带 dispose(资源释放)阶梯和自己的凭据清除,mcp-client、pty-local、SDK helper 与 TUI Git 探测则各自持有另一份凭据清除——而这一切既不可替换,也无法集中测试。 +[进程 seam](2026-07-26-subprocess-seam.md) 交付时恰好只为一个消费方家族塑形:批量收集的 stdout/stderr、批量 stdin、单一的升级式 `kill()`。那是有意的范围控制,其自身的 Agent Note 也把「迁移其余 spawn 调用点」记为暂缓否决项。引入该 seam 的 PR(Pull Request)上的评审推翻了这一暂缓决定:堆叠其上的后续变更应当把接口向 Node 的 API 方向重塑,并把其余运行进程之处迁到该服务上。其余各 spawn 调用点此前各自持有同一套机制中某个切片的私有副本——lsp-local 自带 detached 进程树信号发送(POSIX 进程组 + Windows taskkill + 存活轮询),subagent-subprocess 自带 dispose(资源释放)阶梯和自己的凭据清除,mcp-client、pty-local 与 SDK helper 则各自持有凭据清除的第三、第四、第五份副本——而这一切既不可替换,也无法集中测试。 ## 决策 @@ -15,24 +15,24 @@ Status: implemented - **按流划分的 stdio 处置方式(disposition)**,位于 `SubprocessSpawnSpec` 上:`'pipe'`(原始的 `Readable`/`Writable`,供消费方自有的协议分帧使用)、`'inherit'`(诊断输出直通父进程的流),以及收集模式(collect)`{ maxBytes, spill? }`——即最初的有界尾部保留形状,只是 spill 文件改为可选,使诊断尾部(例如语言服务器的 stderr)无需落盘即可缓冲。stdin 则为 `'ignore'`、`'pipe'` 或 `{ data }`(写完即关闭的批量形式)。 - **`SubprocessOutcome` 只承载退出事实**(Node close 事件的词汇);收集到的输出在结算后仍可经 `handle.collected` 读取(spill 文件描述符在结算边界封存),因此批量与流式调用方共用一条访问路径,也没有任何内容被复制进这份结果。 - **以进程树为范围的终止,集中在一个动词后面**:`terminate()` 拥有 SIGTERM→宽限期→SIGKILL 升级(也承接 spec 的 abort 信号,进程树消亡后为空操作)——句柄不暴露单信号的 `kill(signal?)`,因此消费方无法跳过宽限窗口;`waitForExit()` 轮询进程树存活状态(POSIX 进程组探测;Windows 上以直接子进程为界)。Windows 进程树终止(`taskkill /T`,可注入)自 lsp-local 迁入,因此每个消费方拿到的进程树语义在各平台上都正确。(最初从 `subagent-subprocess` 吸收的以 stdin EOF 打头的 dispose 阶梯,后来又移回其唯一消费方——见[阶梯归属 Agent Note](2026-07-27-dispose-ladder-to-consumer.md)。) -- **凭据清除只有一份定义**:`scrubbedParentEnv()`/`SENSITIVE_ENV_PATTERN` 定义在 seam 上,且凭据形状的名称不区分大小写地包含 `KEY`、`PASSWORD`、`SECRET` 或 `TOKEN`。无法把 spawn 本身路由到该服务的调用点——pty-local(node-pty 拥有 fork)、mcp-client(MCP SDK 拥有传输层的 spawn)与 TUI 的同步 Git 分支探测——改为导入该函数,因此即便进程所有权无法统一,环境策略仍是单一来源。SDK helper 的 `scrubEnvironment()` 默认委托给该函数,并在调用方显式传入环境时应用导出的正则;该正则因此生产消费方而保持公开。 +- **凭据清除只有一份定义**:`scrubbedParentEnv()`/`SENSITIVE_ENV_PATTERN` 定义在 seam 上。普通本地 spawn 与终端本地 spawn 都在 `dsh-subprocess-local` 内部应用该定义;mcp-client 仍需导入它,因为 MCP SDK 拥有该传输层 spawn,SDK helper 的 `scrubEnvironment()` 默认同样委托给它。 -各项迁移随这次重塑一并落地:**bash-local/bash-sandbox**(收集模式 + 批量 stdin;bash 的 `kill()` 映射到 `terminate()`,因此 `task_kill` 保有升级语义),**lsp-local**(管道化的协议流 + 无 spill 的 stderr 收集尾部;`LspConnection` 改为接收 seam 的 spawn 函数;其私有的进程树操作辅助函数已删除),**subagent-acp**(管道化的 ndjson 流 + inherit 的 stderr;spawn 失败经 `done` 的 reject 汇入同一个启动竞态;dispose 是后端自有的 `disposeAcpChild` 阶梯,经由 seam 的动词运行,携带插件所配置的宽限期)。**`dsh-subagent-subprocess` 已删除**——dispose 阶梯与凭据清除归 seam 所有;无人使用的隔离配置目录辅助函数随之消亡(其消费方本就不存在)。 +各项迁移随这次重塑一并落地:**bash-local/bash-sandbox**(收集模式 + 批量 stdin;bash 的 `kill()` 映射到 `terminate()`,因此 `task_kill` 保有升级语义),**lsp-local**(管道化的协议流 + 无 spill 的 stderr 收集尾部;`LspConnection` 改为接收 seam 的 spawn 函数;其私有的进程树操作辅助函数已删除),**subagent-acp**(管道化的 ndjson 流 + inherit 的 stderr;spawn 失败经 `done` 的 reject 汇入同一个启动竞态;dispose 是后端自有的 `disposeAcpChild` 阶梯,经由 seam 的动词运行,携带插件所配置的宽限期),以及后来通过[可移植执行环境决策](2026-07-28-portable-execution-world-consumers.md)迁移的 **pty-local**(`node-pty` 分配与进程检查位于 `spawnTerminal()` 之后,就绪状态和终端策略仍归消费方所有)。**`dsh-subagent-subprocess` 已删除**——dispose 阶梯与凭据清除归 seam 所有;无人使用的隔离配置目录辅助函数随之消亡(其消费方本就不存在)。 挂载 lsp-local 或 subagent-acp 的组合如今都加载 `dsh-subprocess-local`(这两个插件注入 `'subprocess'`);acp/lsp 测试 fixture(测试前置数据)补上了这一行组合配置。 ## 曾考虑的替代方案 -**保持只支持批量的 seam,让流式消费方继续各自为政。**这正是引入该 seam 的 Agent Note 当初的立场,评审将其否决:这样会留下三份进程树信号发送的私有副本和六份凭据清除的私有副本,而未来任何运行器(容器化执行器、远程进程宿主)都得挑选去 fork 哪一份私有副本。Node 形状的处置方式覆盖已观察到的全部三种流形状,既不拓宽结果类型,也不缓冲管道化的流。 +**保持只支持批量的 seam,让流式消费方继续各自为政。**这正是引入该 seam 的 Agent Note 当初的立场,评审将其否决:这样会留下三份进程树信号发送的私有副本和五份凭据清除的私有副本,而未来任何运行器(容器化执行器、远程进程宿主)都得挑选去 fork 哪一份私有副本。Node 形状的处置方式覆盖已观察到的全部三种流形状,既不拓宽结果类型,也不缓冲管道化的流。 **用单个 `stdio: 'pipe' | 'inherit' | 'collect'` 模式一次性统辖全部三条流。**否决:真实消费方按流混用模式(lsp:pipe/pipe/collect;acp:pipe/pipe/inherit;bash:data/collect/collect)。按流划分的处置方式恰好就是 Node 的形状,也免去了混用场景的第二个 spawn 调用。 -**把 pty-local 与 mcp-client 的 spawn 也一并迁移。**基于所有权而非范围否决:node-pty 的 `fork()` 自行分配终端,MCP SDK 的 `StdioClientTransport` 在内部完成 spawn——这两处调用点都不归我们路由。它们采纳共享的凭据清除(那正是属于策略的部分),并在各自的 README 中说明 spawn 为何留在原地。 +**让 pty-local 与 mcp-client 的 spawn 留在服务之外。**MCP SDK 仍拥有其传输层 spawn。PTY 分配则不同:[可移植执行环境决策](2026-07-28-portable-execution-world-consumers.md)把 `node-pty` 放到一个深层终端原语之后,既解决所有权异议,也不假装普通管道化 spawn 能够提供终端语义。 -**迁移 test-support 启动器(acp-snapshot、loader-smoke)、SDK package-manager 运行器与 TUI Git 探测。**否决:support 各包是刻意保持轻依赖的测试基础设施,不得依赖产品 seam;SDK 向导那套附带重定向的 `stdio: 'inherit'` 语义,加上其完全脱离组合的生命周期(根本没有 cordis 上下文),使该服务并不合用;TUI 探测则是同步调用。这些生产调用点改为共享凭据清除。 +**迁移 test-support 启动器(acp-snapshot、loader-smoke)与 SDK package-manager 运行器。**否决:support 各包(package)是刻意保持轻依赖的测试基础设施,不得依赖产品 seam;而 SDK 向导那套附带重定向的 `stdio: 'inherit'` 语义,加上其完全脱离组合的生命周期(根本没有 cordis 上下文),使该服务并不合用;它改为共享凭据清除。 ## 后果 -换来的是:进程树信号发送、升级、有界收集与凭据清除各自只剩一份实现,且只在 `dsh-subprocess-local` 的测试套件中测试一次(其中包括 lsp-local 的私有副本从未有过的、以注入平台方式实现的 Windows 覆盖);lsp-local 与 subagent-acp 卸下了自己的进程管道,其子进程如今像 bash 的一样,在插件重载后存活、随组合拆除而终止;一个完整的包(`dsh-subagent-subprocess`)就此消失。seam README 中「只有一个消费方家族」的限制说明也随之退役。 +换来的是:进程树信号发送、升级、有界收集、终端进程机制与凭据清除各自只剩一份实现,且只在 `dsh-subprocess-local` 的测试套件中测试一次(其中包括 lsp-local 的私有副本从未有过的、以注入平台方式实现的 Windows 覆盖);lsp-local、pty-local 与 subagent-acp 卸下了提供方专用的进程管道,其子进程如今像 bash 的一样随组合拆除而终止;一个完整的包(`dsh-subagent-subprocess`)就此消失。seam README 中「只有一个消费方家族」的限制说明也随之退役。 -代价是:这道 seam 变宽了(stdio 模式从一种变为三种、生命周期接口面从一个动词扩展为 terminate/waitForExit/dispose 这一组),未来的后端因此要实现更宽的接口面;lsp-local/subagent-acp 的各组合如今都多出 subprocess 这一行组合配置;`SubprocessOutcome` 也不再承载输出,这是仍未发布的堆叠变更内部的一次破坏性形状变更(依照预发布立场,PR2 那一层被就地更新,而非加 shim)。pty-local/mcp-client/SDK/TUI/test-support 的 spawn 因所有权归属或执行形状留在该服务之外,以凭据清除作为共底线,且为显式环境的生产消费方有意保持该正则导出。 +代价是:这道 seam 变宽了,涵盖执行环境坐标、可执行文件查找、三种 stdio 模式、进程树生命周期和一个终端原语,未来的后端因此要实现更宽的表面;lsp-local、pty-local 与 subagent-acp 的各组合如今都多出 subprocess 这一行组合配置;`SubprocessOutcome` 也不再承载输出,这是仍未发布的堆叠变更内部的一次破坏性形状变更。MCP/SDK/test-support 的 spawn 因所有权归属留在该服务之外,以凭据清除作为共享底线。 diff --git a/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.i18n.yaml index 1a918fc62c..94711572d6 100644 --- a/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md -2026-07-26-subprocess-seam.md: ad2f8522be51ba16b0df155aeb334a88f493890f -2026-07-26-subprocess-seam.zh.md: 575c02e346531824ef409ddc6155aa2b59ce4e63 +2026-07-26-subprocess-seam.md: d5dee7361e48f18f3147444b2db74da3dd1c1161 +2026-07-26-subprocess-seam.zh.md: 884c572dc65c0fbf240f7e4e3b0387038e21f5ad diff --git a/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md b/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md index ad2f8522be..d5dee7361e 100644 --- a/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md +++ b/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md @@ -12,8 +12,8 @@ English | [中文](2026-07-26-subprocess-seam.zh.md) A new `subprocess/` capability family owns "run and manage a process"; the bash family keeps "run a bash command" and consumes it: -- **`@deepseek-ai/dsh-subprocess` (interface)** — the abstract `SubprocessService` owning `ctx.subprocess` with one method, `spawn(spec): SubprocessHandle`, and the shared vocabulary: the fully-explicit `SubprocessSpawnSpec` (argv, cwd, per-stream stdio dispositions, grace — no defaults; deployment-varying knobs stay with the calling seam's config, per the `dsh-bash` request/spec template and the no-hidden-defaults rule), `SubprocessHandle` with non-consuming offset-based readers, `SubprocessOutcome` with deliberately no timeout/cancel classification, and the shared scrub plus `DSH_ENV_PREFIX`/`DshEnvironment`/`CollectedOutput` types. `argv` is never shell-interpreted. (The [consumer-migration Agent Note](2026-07-26-subprocess-consumer-migration.md) later widened the stdio and termination vocabulary Node-ward.) -- **`@deepseek-ai/dsh-subprocess-local` (implementation)** — `LocalSubprocessService` over the former `run.ts` plumbing (`spawn.ts`): detached groups, tail-keep truncation with private bounded spill files, credential scrub with the explicit-env merge after it, group kill escalation, and disposal that kills and joins every still-running managed process. It has no config; every limit arrives on the spec. The terminal `ENV_OVERRIDES` (`TERM=dumb` etc.) did NOT move — that is bash-tool presentation policy and stays in `dsh-bash-local`, merged through the ordinary env channel. +- **`@deepseek-ai/dsh-subprocess` (interface)** — the abstract `SubprocessService` owning `ctx.subprocess`: execution-world cwd and runtime storage, executable lookup, fully explicit ordinary spawns, and the terminal primitive added by the [portable execution-world decision](2026-07-28-portable-execution-world-consumers.md). Its vocabulary includes per-stream stdio dispositions, process and terminal handles, exit facts with deliberately no timeout/cancel classification, and the shared scrub plus `DSH_ENV_PREFIX`/`DshEnvironment`/`CollectedOutput` types. `argv` is never shell-interpreted. +- **`@deepseek-ai/dsh-subprocess-local` (implementation)** — `LocalSubprocessService` over the former `run.ts` plumbing (`spawn.ts`) plus `node-pty`: detached groups, bounded collection and private spill files, executable lookup, private runtime storage, foreground/session inspection, credential scrub with explicit env merged after it, tree cleanup, and disposal that terminates and joins every managed process. It has no config; every limit arrives on the spec. Bash and PTY presentation environment overrides stay in their consumers. - **`dsh-bash-local` (consumer)** — `inject: ['subprocess']`; maps each resolved `BashExecSpec` onto a `SubprocessSpawnSpec` (`['bash', '-c', command]`), keeps its config, `resolve()` defaulting, fused-deadline `timedOut`/`aborted` classification, the `[stderr]`-marked background read merge with its consuming cursor, and the `onProcessDone` subclass hook. `dsh-bash-sandbox` is unchanged apart from redeclaring the inherited inject; it still wraps at the command-string level and re-enters the inherited spawn path. - **`dsh-bash` (seam)** — re-exports the moved vocabulary from `dsh-subprocess`, so no bash consumer changes an import; `BashExecRequest`/`BashExecSpec`/`BashProcess` and the sandbox facts remain bash-owned. @@ -25,7 +25,7 @@ Background-process lifetime moved from the executor to the subprocess service: t **Leave the process plumbing inside `dsh-bash-local` (status quo).** Rejected for the same reason the [task registry split](2026-07-26-task-registry-seam.md) landed: the boundary is stable and already documented in-code (`run.ts`'s module doc said "this layer reacts to an abort signal; the executor owns deadlines and classifies causes"), and keeping it private makes every future non-shell runner either fork the mechanics or depend on a bash-named package for non-bash work. The user-visible driver for this stack was exactly this split. -**Migrate the repo's other spawn sites (lsp-local, pty-local, subagent-subprocess, sdk package-manager, test-support launchers) onto `ctx.subprocess` in the same change.** Rejected as scope creep with real design risk at this PR's scale: those sites have materially different stream and lifecycle needs — node-pty ownership (pty), LSP framing over long-lived stdio with tree-kill fallbacks (lsp), stdin-EOF-first disposal ladders and no output buffering (subagent transports) — and forcing them under a handle shaped for bounded batch output would either bloat the seam or misfit the consumers. The seam shipped proven against its one real consumer family, per the shape-interfaces-around-current-consumers rule. Review then asked for exactly that follow-up as a stacked PR; the [consumer-migration Agent Note](2026-07-26-subprocess-consumer-migration.md) records the Node-ward reshape and which sites moved (and which stayed, by ownership). +**Migrate the repo's other spawn sites in the introducing change.** Rejected as scope creep with real design risk at that scale. The later [consumer-migration](2026-07-26-subprocess-consumer-migration.md) and [portable execution-world](2026-07-28-portable-execution-world-consumers.md) decisions reshape the interface around the observed LSP, subagent, PTY, and Code Runtime consumers; SDK and test-support launchers remain outside by ownership. **Put `run_in_background`/task semantics into the process seam instead.** Rejected: that boundary already exists — `ctx.tasks` owns ids, ownership, and notices, and the bash tool adapts a `BashProcess` into task hooks. The process seam sits *below* the bash executor, not beside the task registry. @@ -33,6 +33,6 @@ Background-process lifetime moved from the executor to the subprocess service: t ## Consequences -Bought: "run and manage a process" is a swappable capability with the standard three-package shape (consumer count starts at two: `bash-local`, `bash-sandbox`); a containerized or remote process backend slots in without touching bash semantics; the shared `DSH_*`/output vocabulary has a non-shell home; and background processes survive executor reloads, matching the task registry's lifetime model. The spawn plumbing suite moved wholesale to `dsh-subprocess-local` (argv-based, plus argv-validation and service lifecycle/disposal suites); the executor suite now pins the bash-owned layers (classification, merge, spawn-failure note, service-owned lifetime) against the real service. +Bought: "run and manage a process" is a swappable capability used by Bash, LSP, PTY, Code Runtime, and ACP consumers; a containerized or remote process backend slots in without changing their domain semantics; the shared `DSH_*`/output vocabulary has a non-shell home; and background processes survive executor reloads, matching the task registry's lifetime model. Process and terminal plumbing is tested through `dsh-subprocess-local`; consumer suites pin only their owned behavior against the real service. Cost: one more package pair and one more composition row everywhere a bash executor loads — a boot that loads an executor without the subprocess service leaves `ctx.bash` pending on `ctx.subprocess` (standard missing-service behavior). The moved-vocabulary re-exports keep `dsh-bash` imports working but mean two packages now name the same types; the subprocess seam is the owner and the bash seam documents the re-export. The spawn-failure note became single-delivery through the read path where the old plumbing retained it in the stderr buffer for repeated `readFrom(0)` reads — acceptable because the bash background read path was already a consuming cursor, and the note reaches the one reader that exists. diff --git a/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.zh.md b/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.zh.md index 575c02e346..884c572dc6 100644 --- a/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.zh.md @@ -1,4 +1,4 @@ -# Agent Note: 子进程服务是 bash 执行器之下的独立 seam(`dsh-subprocess` / `dsh-subprocess-local`) +# Agent Note: 进程管理器是 bash 执行器之下的独立 seam(`dsh-subprocess` / `dsh-subprocess-local`) Status: implemented @@ -6,33 +6,33 @@ Status: implemented ## 问题 -`dsh-bash-local` 原先把两项因不同原因而变化的能力捆绑在一起:*运行一条 bash 命令*(命令默认值补全、超时分类、对模型友好的终端环境、bash 工具所渲染的 stdout/stderr 合并)与*运行并管理一个子进程*(detached 进程组、附带 spill 文件的有界尾部保留输出、凭据清除与 `DSH_*` 合并次序、SIGTERM→宽限期→SIGKILL 升级、先终止再等待退出的 dispose(资源释放))。进程这一半(`run.ts`)约占整个包的一半,却没有属于自己的 seam:未来的非 shell 运行器(直接执行 argv 的执行器、worker supervisor)将不得不重新实现这套机制,或者探入 bash 内部;而共享的 `DSH_*`/`CollectedOutput` 词汇则存放在一个名字承诺 shell 语义的包里。这种捆绑还把后台进程的存续期系在执行器的 fiber 上:重载 bash 执行器会杀死每一个存活的后台进程。这一点不同于同级的[任务注册表](2026-07-26-task-registry-seam.md):后者的注册存续期刻意长于生产方 fiber。 +`dsh-bash-local` 原先把两项因不同原因而变化的能力捆绑在一起:*运行一条 bash 命令*(命令默认值补全、超时分类、对模型友好的终端环境、bash 工具所渲染的 stdout/stderr 合并)与*运行并管理一个子进程*(detached 进程组、附带 spill 文件的有界尾部保留输出、凭据清除与 `DSH_*` 合并次序、SIGTERM→宽限期→SIGKILL 升级、先终止再等待退出的 dispose(资源释放))。进程这一半(`run.ts`)约占整个包(package)的一半,却没有属于自己的 seam:未来的非 shell 运行器(直接执行 argv 的执行器、worker supervisor)将不得不重新实现这套机制,或者探入 bash 内部;而共享的 `DSH_*`/`CollectedOutput` 词汇则存放在一个名字承诺 shell 语义的包里。这种捆绑还把后台进程的存续期系在执行器的 fiber 上:重载 bash 执行器会杀死每一个存活的后台进程。这一点不同于兄弟的[任务注册表](2026-07-26-task-registry-seam.md):后者的注册存续期刻意长于生产方 fiber。 ## 决策 新的 `subprocess/` 能力家族拥有「运行并管理一个进程」;bash 家族保留「运行一条 bash 命令」,并成为前者的消费方: -- **`@deepseek-ai/dsh-subprocess`(接口)**——拥有 `ctx.subprocess` 的抽象 `SubprocessService`(仅一个方法:`spawn(spec): SubprocessHandle`),以及共享词汇:完全显式的 `SubprocessSpawnSpec`(argv、cwd、按流划分的 stdio 处置方式(disposition)、宽限期,一律不设默认值;随部署变化的旋钮依照 `dsh-bash` 的 request/spec 模板与无隐藏默认值规则,留在调用方 seam 的配置里)、携带基于偏移量的非消费式读取器的 `SubprocessHandle`、刻意不含超时/取消分类的 `SubprocessOutcome`,以及共享的凭据清除与 `DSH_ENV_PREFIX`/`DshEnvironment`/`CollectedOutput` 类型。`argv` 绝不经过 shell 解释。([消费方迁移 Agent Note](2026-07-26-subprocess-consumer-migration.md) 其后将 stdio 与终止词汇拓宽为 Node 形状。) -- **`@deepseek-ai/dsh-subprocess-local`(实现)**——`LocalSubprocessService`,构建在原 `run.ts` 管道(现为 `spawn.ts`)之上:detached 进程组、带私有有界 spill 文件的尾部保留截断、清除之后合并显式 env 的凭据清除、进程组 kill 升级,以及会终止每个仍在运行的受管进程并等待其退出的 dispose。该实现没有任何配置;每项限制都随 spec 到达。终端相关的 `ENV_OVERRIDES`(`TERM=dumb` 等)并未迁移:那是 bash 工具的呈现策略,留在 `dsh-bash-local` 里,经普通 env 通道合并。 +- **`@deepseek-ai/dsh-subprocess`(接口)**——拥有 `ctx.subprocess` 的抽象 `SubprocessService`:执行环境 cwd 与运行时存储、可执行文件查找、完全显式的普通 spawn,以及[可移植执行环境决策](2026-07-28-portable-execution-world-consumers.md)新增的终端原语。其词汇包括按流划分的 stdio 处置方式(disposition)、进程与终端句柄、刻意不含超时/取消分类的退出事实,以及共享的凭据清除与 `DSH_ENV_PREFIX`/`DshEnvironment`/`CollectedOutput` 类型。`argv` 绝不经过 shell 解释。 +- **`@deepseek-ai/dsh-subprocess-local`(实现)**——`LocalSubprocessService` 构建在原 `run.ts` 管道(现为 `spawn.ts`)与 `node-pty` 之上:detached 进程组、有界收集与私有 spill 文件、可执行文件查找、私有运行时存储、前台/会话检查、清除之后合并显式 env 的凭据清除、进程树清理,以及终止每个受管进程并等待其退出的 dispose。该实现没有任何配置;每项限制都随 spec 到达。Bash 与 PTY 的呈现环境覆盖仍归各自消费方所有。 - **`dsh-bash-local`(消费方)**——`inject: ['subprocess']`;把每个解析后的 `BashExecSpec` 映射为一个 `SubprocessSpawnSpec`(`['bash', '-c', command]`),并保留自身配置、`resolve()` 默认值补全、基于融合 deadline 的 `timedOut`/`aborted` 分类、带 `[stderr]` 标记的后台读取合并及其消费游标,以及 `onProcessDone` 子类钩子。`dsh-bash-sandbox` 除了重新声明继承来的 inject 之外没有变化;它仍在命令字符串层面做包装,并重新进入继承的 spawn 路径。 - **`dsh-bash`(seam)**——把迁走的词汇从 `dsh-subprocess` 重导出,因此没有任何 bash 消费方需要改动导入;`BashExecRequest`/`BashExecSpec`/`BashProcess` 与沙箱事实仍归 bash 所有。 如今,每个加载 bash 执行器的组合都同时加载 `@deepseek-ai/dsh-subprocess-local`:CLI(命令行界面)、各示例、Python 捆绑运行时、create-sdk 的 bash 功能资源,以及各内联测试配置。 -后台进程的存续期从执行器移到了子进程服务:执行器不再保有存活进程集合,于是重载执行器后,后台工作会继续运行且仍可读取,而组合拆除(服务的 dispose)仍是先终止再等待退出的边界。一条行为 seam 随之挪动:后台 spawn 失败不再能在管道内部被缓冲成伪造的 stderr(对一个从未真正运行的进程,服务会 reject `done`,且不缓冲任何内容),因此执行器把 `spawn failed: …` 提示注入恰好一个 `readOutput()` 增量。 +后台进程的存续期从执行器移到了管理器:执行器不再保有存活进程集合,于是重载执行器后,后台工作会继续运行且仍可读取,而组合拆除(管理器的 dispose)仍是先终止再等待退出的边界。一条行为 seam 随之挪动:后台 spawn 失败不再能在管道内部被缓冲成伪造的 stderr(对一个从未真正运行的进程,管理器会 reject `done`,且不缓冲任何内容),因此执行器把 `spawn failed: …` 提示注入恰好一个 `readOutput()` 增量。 ## 曾考虑的替代方案 **把进程管道留在 `dsh-bash-local` 里(维持现状)。**否决的理由与[任务注册表拆分](2026-07-26-task-registry-seam.md)得以落地的理由相同:这条边界既稳定,也早已记录在代码里(`run.ts` 的模块文档曾写明「this layer reacts to an abort signal; the executor owns deadlines and classifies causes」),而若继续将它保持私有,未来每个非 shell 运行器就只能要么 fork 这套机制,要么为非 bash 工作去依赖一个以 bash 命名的包。这组堆叠变更对用户可见的动因正是这一拆分。 -**在同一变更中把仓库其余 spawn 调用点(lsp-local、pty-local、subagent-subprocess、sdk package-manager、test-support 各启动器)迁到 `ctx.subprocess` 上。**在本 PR(Pull Request)的规模下,作为带有真实设计风险的范围蔓延否决。这些调用点在流与生命周期上的需求存在实质差异:node-pty 所有权(pty)、长生命周期 stdio 上的 LSP 分帧加进程树终止回退(lsp)、以 stdin EOF 打头的 dispose 阶梯和完全不缓冲输出(subagent 传输层)。把它们强行纳入一个按有界批量输出塑形的句柄之下,要么会让这道 seam 膨胀,要么会让句柄与消费方错配。依照「接口围绕当前消费方塑形」的规则,该 seam 当时在其唯一真实的消费方家族上得到验证后交付。评审随后恰恰要求以堆叠 PR 的形式完成这项后续工作;[消费方迁移 Agent Note](2026-07-26-subprocess-consumer-migration.md) 记录了向 Node 形状的重塑,以及哪些调用点迁入(哪些因所有权归属而留在原地)。 +**在引入 seam 的同一变更中迁移仓库其余 spawn 调用点。**否决,因为在当时的规模下属于带有真实设计风险的范围蔓延。后续的[消费方迁移](2026-07-26-subprocess-consumer-migration.md)与[可移植执行环境](2026-07-28-portable-execution-world-consumers.md)决策围绕已观察到的 LSP、subagent、PTY 与 Code Runtime 消费方重塑接口;SDK 与 test-support 启动器因所有权归属仍留在服务之外。 **改把 `run_in_background`/任务语义放进进程 seam。**否决:那条边界已经存在。`ctx.tasks` 拥有 id、所有权与通知,bash 工具则把 `BashProcess` 适配成任务钩子。进程 seam 位于 bash 执行器*之下*,而不是与任务注册表并列。 -**把 `ENV_OVERRIDES`(TERM=dumb、PAGER=cat 等)移入子进程服务。**否决:通用子进程服务不得把终端呈现策略强加给非终端消费方;对环境中凭据形态名称与 `DSH_*` 名称的清除是安全与身份不变式,予以保留,但终端友好性是 bash 工具自己的选择,经 spec 的显式 env 表达,而调用方自己的条目依旧优先。 +**把 `ENV_OVERRIDES`(TERM=dumb、PAGER=cat 等)移入管理器。**否决:通用进程管理器不得把终端呈现策略强加给非终端消费方;对环境中凭据形态名称与 `DSH_*` 名称的清除是安全与身份不变式,予以保留,但终端友好性是 bash 工具自己的选择,经 spec 的显式 env 表达,而调用方自己的条目依旧优先。 ## 后果 -换来的是:「运行并管理一个进程」成为一项具备标准三包形态的可替换能力(消费方起步就有两个:`bash-local`、`bash-sandbox`);容器化或远程进程后端可以直接接入,而不触碰 bash 语义;共享的 `DSH_*`/输出词汇有了一个不带 shell 含义的归属;后台进程也能在执行器重载后存活,与任务注册表的存续期模型一致。spawn 管道测试套件整体迁至 `dsh-subprocess-local`(现以 argv 为基础,外加 argv 校验与服务生命周期/dispose 套件);执行器测试套件如今以真实服务为基准,固定 bash 自有的各层行为(分类、合并、spawn 失败提示、归服务所有的存续期)。 +换来的是:「运行并管理一个进程」成为 Bash、LSP、PTY、Code Runtime 与 ACP(Agent Client Protocol)消费方共用的可替换能力;容器化或远程进程后端可以直接接入,而无需改变各领域语义;共享的 `DSH_*`/输出词汇有了一个不带 shell 含义的归属;后台进程也能在执行器重载后存活,与任务注册表的存续期模型一致。进程与终端管道通过 `dsh-subprocess-local` 测试;消费方测试套件只需针对真实服务固定各自拥有的行为。 -代价是:多出一对包,而且凡加载 bash 执行器之处都多一行组合配置。若某次启动加载了执行器却没有加载子进程服务,`ctx.bash` 会因等待 `ctx.subprocess` 而保持挂起(标准的服务缺失行为)。迁移词汇的重导出让 `dsh-bash` 的导入继续可用,但也意味着两个包如今命名同一批类型;进程 seam 是所有者,bash seam 则记录这层重导出。spawn 失败提示经由读取路径变为单次交付,而旧管道曾把它保留在 stderr 缓冲区里,供重复的 `readFrom(0)` 读取;这一点可以接受,因为 bash 的后台读取路径本就是消费游标,该提示能到达唯一存在的那个读取方。 +代价是:多出一对包,而且凡加载 bash 执行器之处都多一行组合配置。若某次启动加载了执行器却没有加载管理器,`ctx.bash` 会因等待 `ctx.subprocess` 而保持挂起(标准的服务缺失行为)。迁移词汇的重导出让 `dsh-bash` 的导入继续可用,但也意味着两个包如今命名同一批类型;进程 seam 是所有者,bash seam 则记录这层重导出。spawn 失败提示经由读取路径变为单次交付,而旧管道曾把它保留在 stderr 缓冲区里,供重复的 `readFrom(0)` 读取;这一点可以接受,因为 bash 的后台读取路径本就是消费游标,该提示能到达唯一存在的那个读取方。 diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml new file mode 100644 index 0000000000..8b4288edb4 --- /dev/null +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md +2026-07-28-portable-execution-world-consumers.md: 000fa1b98964eb85550d0c607281937f8d7932c6 +2026-07-28-portable-execution-world-consumers.zh.md: 94e6bfcf94e450fafea2b33e152384da400e4b63 diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md new file mode 100644 index 0000000000..000fa1b989 --- /dev/null +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md @@ -0,0 +1,48 @@ +# Agent Note: Portable consumers over filesystem and subprocess execution worlds + +Status: implemented + +English | [中文](2026-07-28-portable-execution-world-consumers.zh.md) + +## Problem + +The filesystem and subprocess seams made file and ordinary process access replaceable, but several higher capabilities still reached host Node APIs directly. A remote execution provider therefore appeared to need separate PTY, LSP, and Code Runtime packages even though their domain behavior did not change. Those packages would be shallow adapters: each would duplicate an existing consumer merely to replace its file and process operations. + +Ordinary pipes do not cover one requirement. A persistent terminal needs PTY allocation, foreground-process-group inspection and signalling, and cleanup of the complete terminal session. Pretending those operations can be rebuilt in `dsh-pty-local` from an ordinary `spawn()` handle would either leak provider internals or weaken its lifecycle contract. + +## Decision + +`ctx.fs` and `ctx.subprocess` together define one execution world. Providers mounted together must describe the same path namespace, executables, processes, and terminal sessions; higher capabilities consume those two interfaces rather than name the provider. + +The filesystem interface owns the path facts that another capability needs without exposing its opaque target identity: a canonical process path, canonical `file:` URI, containment, and a bounded stable-handle text read. The existing text and mutation operations remain filesystem-owned. + +The subprocess interface owns the process coordinates and primitives: canonical cwd, private runtime storage, executable lookup, ordinary raw or collected process spawning, and `spawnTerminal()`. The terminal operation is one deep primitive whose handle owns byte I/O, foreground groups, signalling, TERM-to-KILL session cleanup, and a quiescence wait. Prompt detection, idle inference, scrollback, sandbox policy, and owner lifecycle remain in the PTY consumer. + +Generic consumers use that execution world: + +- `dsh-bash-local` continues to map Bash semantics onto ordinary `ctx.subprocess.spawn()`. +- `dsh-lsp-local` reads and contains source through `ctx.fs`, resolves and launches language servers through `ctx.subprocess`, and sends provider-owned file URIs. Its JSON-RPC, pooling, synchronization, cancellation, and normalization stay unchanged. +- `dsh-pty-local` maps persistent-shell semantics onto `ctx.subprocess.spawnTerminal()`. The local `node-pty` and process-inspection implementation moves into `dsh-subprocess-local`; another subprocess provider supplies the same primitive. +- `dsh-code-runtime-subprocess` materializes a dependency-free runner through `ctx.fs` and launches it through `ctx.subprocess`, preserving the Code Runtime binding and output contract across local or remote worlds. It shares host-side worker mechanics through the non-plugin `dsh-code-runtime-worker/runtime-host` subpath instead of copying them. + +`dsh-code-runtime-worker` remains a separate implementation. It is the smaller in-process backend and works in single-file distributions that cannot assume an installed Node executable. Remote filesystem/process compositions select `dsh-code-runtime-subprocess`; they do not need a provider-specific Code Runtime package. + +## Alternatives considered + +**Keep one PTY, LSP, and Code Runtime package per remote provider.** Rejected because provider mechanics would be repeated above the existing seams. The deletion test exposes the problem: deleting those adapters should not scatter domain behavior into the remote provider; the generic consumers already own it. + +**Model a terminal as an ordinary piped subprocess.** Rejected because pipes cannot allocate a controlling terminal, resolve the current foreground process group, or prove complete terminal-session cleanup. One terminal primitive is smaller and more honest than exposing substrate-specific escape hatches. + +**Move PTY readiness and session policy into the subprocess service.** Rejected because those are persistent-terminal consumer semantics, not OS process mechanics. A subprocess provider owns what only its substrate can do; `dsh-pty-local` owns what a Harness terminal means. + +**Delete the worker-thread Code Runtime.** Rejected because portability does not erase its current deployment need. The subprocess backend requires a Node executable and filesystem materialization; the worker backend has neither requirement and remains the supported single-process path. + +**Run the whole harness inside the remote environment.** Rejected as a different deployment model. Making execution capabilities portable does not move model calls, session state, plugin state, or the agent loop. + +## Consequences + +A remote execution provider implements only its shared sandbox owner plus filesystem and subprocess adapters. Bash, PTY, LSP, and subprocess Code Runtime compose above them, so fixes to those capabilities remain provider-neutral. + +The fundamental interfaces are wider, and a filesystem/subprocess pair must agree on one execution world. The added operations are limited to facts and lifecycle mechanics that current generic consumers require; model schemas, protocol framing, readiness policy, and presentation do not leak into the providers. + +The local implementation absorbs `node-pty` and platform process inspection because it owns local terminal mechanics. This moves code without weakening terminal teardown: disposal still waits for exact PID-identity-fenced descendants and the top-level terminal process to reach quiescence. diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md new file mode 100644 index 0000000000..94e6bfcf94 --- /dev/null +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md @@ -0,0 +1,48 @@ +# Agent Note: 基于文件系统与进程管理执行世界的可移植消费方 + +Status: implemented + +[English](2026-07-28-portable-execution-world-consumers.md) | 中文 + +## 问题 + +文件系统与进程管理 seam 使文件访问和普通进程访问具备可替换性,但若干上层能力仍直接调用宿主 Node API。因此,即使领域行为没有变化,远程执行提供方看起来仍需要独立的 PTY、LSP 与代码运行时包(package)。这些包只会成为浅层适配器:每个包都仅为替换文件与进程操作而复制一个现有消费方。 + +普通管道无法满足其中一项要求。持久终端需要分配 PTY、检查前台进程组并发送信号,以及清理完整的终端会话。如果假设可以在 `dsh-pty-local` 中基于普通 `spawn()` 句柄重建这些操作,最终不是泄漏提供方内部细节,就是削弱其生命周期契约。 + +## 决策 + +`ctx.fs` 与 `ctx.subprocess` 共同定义一个执行世界。共同挂载的提供方必须描述相同的路径命名空间、可执行文件、进程和终端会话;上层能力消费这两个接口,而不引用具体提供方。 + +文件系统接口负责其他能力需要的路径事实,同时不公开其不透明目标身份:规范化进程路径、规范化 `file:` URI、包含关系,以及通过稳定句柄执行的有界文本读取。现有文本与变更操作仍归文件系统负责。 + +进程管理接口负责进程运行坐标与原语:规范化 cwd、私有运行时存储、可执行文件查找、以原始或收集模式 spawn 普通进程,以及 `spawnTerminal()`。终端操作是一项深层原语,其句柄负责字节 I/O、前台进程组管理、信号发送、TERM→KILL 会话清理以及等待完全停稳。提示符检测、空闲推断、scrollback、沙箱策略和所有者生命周期仍由 PTY 消费方负责。 + +通用消费方使用该执行世界: + +- `dsh-bash-local` 继续把 Bash 语义映射到普通的 `ctx.subprocess.spawn()`。 +- `dsh-lsp-local` 通过 `ctx.fs` 读取源文件并验证包含关系,通过 `ctx.subprocess` 解析和启动语言服务器,并发送由提供方负责的文件 URI。其 JSON-RPC、池化、同步、取消和规范化保持不变。 +- `dsh-pty-local` 把持久 shell 语义映射到 `ctx.subprocess.spawnTerminal()`。本地 `node-pty` 与进程检查实现移入 `dsh-subprocess-local`;其他进程管理提供方则提供相同原语。 +- `dsh-code-runtime-subprocess` 通过 `ctx.fs` 物化无依赖 runner,并通过 `ctx.subprocess` 启动它,从而在本地或远程执行世界中保留代码运行时的绑定与输出契约。它通过非插件子路径 `dsh-code-runtime-worker/runtime-host` 共享宿主侧 worker 机制,而不是复制这些机制。 + +`dsh-code-runtime-worker` 仍是独立实现。它是较小的进程内后端,可用于无法假定已安装 Node 可执行文件的单文件分发。远程文件系统/进程组合选择 `dsh-code-runtime-subprocess`;它们不需要提供方专用的代码运行时包。 + +## 考虑过的替代方案 + +**为每个远程提供方分别保留 PTY、LSP 和代码运行时包。** 不予采纳,因为这会在现有 seam 之上重复实现提供方机制。删除检验揭示了这一问题:删除这些适配器不应使领域行为散落到远程提供方中;通用消费方本已负责这些行为。 + +**把终端建模为普通的管道子进程。** 不予采纳,因为管道无法分配控制终端、确定当前前台进程组或证明完整终端会话已清理。一项终端原语比公开特定于执行基底的逃生口更小,也更能如实表达契约。 + +**把 PTY 就绪判断与会话策略移入进程管理服务。** 不予采纳,因为这些属于持久终端消费方的语义,而非 OS 进程机制。进程管理提供方负责只有其执行基底才能完成的操作;`dsh-pty-local` 负责 Harness 终端的语义。 + +**删除 worker 线程代码运行时。** 不予采纳,因为可移植性不会消除其当前部署需求。进程管理后端需要 Node 可执行文件和文件系统物化,而 worker 后端两者都不需要,并且仍是受支持的单进程路径。 + +**在远程环境中运行整个 harness。** 不予采纳,因为这是另一种部署模型。让执行能力可移植,并不意味着移动模型调用、会话状态、插件状态或 agent loop(智能体循环)。 + +## 后果 + +远程执行提供方只需实现共享沙箱所有者,以及文件系统与进程管理适配器。Bash、PTY、LSP 和基于进程管理的代码运行时组合在这些适配器之上,因此对这些能力的修复仍与提供方无关。 + +基础接口更宽,一对文件系统/进程管理提供方必须在同一个执行世界上保持一致。新增操作仅限当前通用消费方所需的事实与生命周期机制;模型 schema、协议分帧、就绪策略和呈现不会渗入提供方。 + +本地实现承接 `node-pty` 和平台进程检查,因为它负责本地终端机制。这种代码迁移不会削弱终端拆卸:dispose(资源释放)仍会等待受精确 PID 身份校验保护的后代进程和顶层终端进程完全停稳。 diff --git a/.agents/notes/implemented/feature/2026-06-15-code-mode.i18n.yaml b/.agents/notes/implemented/feature/2026-06-15-code-mode.i18n.yaml index bc05e497c5..a878942993 100644 --- a/.agents/notes/implemented/feature/2026-06-15-code-mode.i18n.yaml +++ b/.agents/notes/implemented/feature/2026-06-15-code-mode.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-06-15-code-mode.md -2026-06-15-code-mode.md: 99bbed3edab32512f88ece9694d6519a1f89c2dd -2026-06-15-code-mode.zh.md: ca1bbe9ed3e412186763d1ed4fca9ed06669d4c3 +2026-06-15-code-mode.md: 7e51b9fa726afe4c34b87457b562b4092ee6c93c +2026-06-15-code-mode.zh.md: 4ba098d87d8c944e88c4cbe11ff78ac4383d3ac5 diff --git a/.agents/notes/implemented/feature/2026-06-15-code-mode.md b/.agents/notes/implemented/feature/2026-06-15-code-mode.md index 99bbed3eda..7e51b9fa72 100644 --- a/.agents/notes/implemented/feature/2026-06-15-code-mode.md +++ b/.agents/notes/implemented/feature/2026-06-15-code-mode.md @@ -6,7 +6,7 @@ English | [中文](2026-06-15-code-mode.zh.md) ## Problem -In the registry's native presentation, the agent loop advertises every visible capability as a JSON-schema function definition. `ToolRegistry` contributes its schemas to the system-prompt assembly, the assembly's `tools` land on the wire (and in the logged request header), the model invokes one `tool-call` block per step, and at the time of this note the loop dispatched each call through `ctx.tools.execute()` **sequentially** (parallel tool execution was an open TODO then; bounded parallel dispatch has since shipped — the [parallel tool-call note](2026-07-10-parallel-tool-call-execution.md), the rolling pool in [docs/architecture.md](../../../../docs/architecture.md)) — with **every** intermediate `tool-result` re-entering the model's context on the next request. +In the registry's native presentation, the agent loop advertises every visible capability as a JSON-schema function definition. `ToolRegistry` contributes its schemas to the system-prompt assembly, the assembly's `tools` land on the wire (and in the logged request header), the model invokes one `tool-call` block per step, and the loop dispatches each call through `ctx.tools.execute()` **sequentially** (parallel tool execution is an explicit open TODO in `dsh-tools` and [docs/architecture.md](../../../../docs/architecture.md)), with **every** intermediate `tool-result` re-entering the model's context on the next request. For multi-step tool work this is token-heavy and serial. The model cannot compose tools — loop over a result set, branch on an intermediate value, fan out, post-process — without a full model round-trip per call, and each round-trip drags the entire intermediate result back into context whether the model needs it or not. @@ -20,7 +20,7 @@ Three decisions, each elaborated in its own section below: 1. **Code Mode is a first-class presentation mode of `ToolRegistry`** (`dsh-tools`), selected by a validated `mode` config: `'native'` (the default, contributing the visible capability schemas), `'code'` (the registry contributes only its reserved `run_code` transport plus a generated SDK `.d.ts` in the system prompt), or `'both'` (native schemas and the transport + SDK). The registry shapes its canonical contribution at the source; the cooperative prompt-assembly result remains authoritative, and the logged request header records exactly that returned presentation. 2. **Code execution is a capability seam** — `packages/code-runtime/` contains the interface package `@deepseek-ai/dsh-code-runtime`, which owns `ctx.codeRuntime` ([capability seams](../architecture/2026-06-13-capability-seams.md); consumer = `dsh-tools`, with core-consumes-a-seam precedent in `agent-loop` → `dsh-llm`). The runtime knows nothing about tools: it is handed a program and named async bindings, runs the program, and reports `{ value, logs, error? }`. Language and substrate are backend properties, so a future Python or container backend is another implementation package, not a redesign. -3. **The shipped implementation is `@deepseek-ai/dsh-code-runtime-worker`**: one fresh Node worker thread per run, executing the model's TypeScript after type-strip, with bindings bridged over the message port, an empty environment, configurable heap/output/time caps, and hard termination. Its trust posture is bash-equivalent by design — no unsafe-acknowledgement flags — because the harness already ships `dsh-bash-local`, which executes arbitrary model-written shell commands with strictly *more* ambient authority. +3. **Two implementations preserve one fresh-worker contract**: `@deepseek-ai/dsh-code-runtime-worker` runs the worker in the harness process, while `@deepseek-ai/dsh-code-runtime-subprocess` materializes a runner through `ctx.fs` and launches it through `ctx.subprocess` for another execution world. Both execute host-stripped TypeScript in a fresh Node worker with an empty environment, bridged bindings, configurable heap/output/time caps, and hard termination. Their trust posture is bash-equivalent by design; stronger isolation comes from the mounted execution world. This note owns Code Mode's presentation, composition, isolation, and settlement foundation. The later [typed tool-return Agent Note](2026-07-20-code-mode-typed-tool-returns.md) owns the generated output map, canonical binding values, `ToolCallError`, and the lossless outer-output boundary. @@ -32,7 +32,7 @@ This note owns Code Mode's presentation, composition, isolation, and settlement **Interaction with `toolOrder`, stated up front:** a configured `systemPrompt.toolOrder` naming native capabilities rejects every assembly under `mode: 'code'`, because those names are outside that mode's wire-validation universe. This is correct behavior, not a bug: a deployment using Code Mode updates its order config or drops it. -**SDK prompt section.** In `'code'` and `'both'`, the lazy `tools:sdk` section in the tool-guidance order band renders the loaded runtime's language declarations plus fixed usage instructions for the scope's visible capabilities (TypeScript by default; the [language-dispatch note](2026-07-31-code-mode-language-dispatch.md) added Python and the `ctx.codeRuntime.language` renderer table). It shares lookup and execution visibility, excludes `run_code`, and sorts tools lexicographically for byte-stable output. +**SDK prompt section.** In `'code'` and `'both'`, the lazy `tools:sdk` section in the tool-guidance order band renders TypeScript declarations plus fixed usage instructions for the scope's visible capabilities. It shares lookup and execution visibility, excludes `run_code`, and sorts tools lexicographically for byte-stable output. **Assembly ownership.** `run_code` and `tools:sdk` enter the trusted `system-prompt/assemble` waterfall as normal assembly inputs. A scoped `tools:sdk` section may shadow the global default before dispatch, and a listener may remove or replace either contribution. The waterfall's returned assembly is final, so whoever changes these inputs owns preserving a viable Code Mode protocol when the deployment expects Code Mode to remain usable; no restoration pass overrides deliberate composition. @@ -48,7 +48,7 @@ Under `'code'` and `'both'` the registry owns `run_code` as a reserved presentat **Sub-call contexts are deferred through the parent.** Injecting inside `run_code` would break parent call/result adjacency, so `ToolRunContext.deferContext()` collects every sub-result `additionalContexts` entry in dispatch order. The registry carries that array even when the program later throws, and the loop appends each entry only after the outer result and every sibling result in the step. An outer post-execute block discards tool-deferred entries and exposes only contexts explicitly attached by the blocking decision. -**Concurrency is bounded, not serialized.** Each run owns a dispatch queue that starts calls strictly in submission order and classifies each one through `registry.executionMode`, the same fail-closed `isConcurrencySafe` contract the native loop uses. Consecutive parallel-classified calls overlap up to `maxParallelSubCalls` (default 10; `1` restores serial dispatch); an exclusive call drains the pool and runs alone. Settlement abandons queued calls that have not started. This note shipped the serialized placeholder; the [live-parallel Agent Note](2026-07-26-code-mode-live-parallel-dispatch.md) owns the scheduler that replaced it. +**Concurrency is serialized.** Each run owns a dispatch queue, so even `Promise.all` executes tool calls in submission order. Settlement abandons queued calls that have not started. Parallelism requires per-tool concurrency-safety metadata. **Presentation.** `run_code`'s render intent is decided here per the [render-intent Agent Note](../architecture/2026-07-02-tool-render-intent-union.md): `presentCall` creates a `generic` card with `kind: 'execute'`, the program text as its title, and the same program text as `rawInput`; `run_code` intentionally declares no `presentResult`, so the TUI and host/client runtime (Web) complete that card through their generic raw-content fallback using the final durable `tool/result.content`, including captured logs plus the returned value, failure, or post-policy spill preview. This is not a `terminal` card: that card's semantics are "a shell command in a working directory", which a program is not. See the [result-card completeness note](../../archived/bug-fix/2026-07-20-code-mode-result-card-completeness.md). @@ -64,7 +64,7 @@ Each sub-dispatch appends a log-only `tool/code-dispatch-start` event at pool en - `CodeBindingNamespace = { global: string; functions: Record Promise>; errorClass?: { name: string; memberNameProperty: string } }` — the runtime exposes each namespace as a global object of async functions inside the program; the optional descriptor asks the runtime to inject a real program-visible rejection class without teaching the seam consumer-specific names. `CodeJsonValue` is this dependency-light seam's structural lossless-JSON type, so binding arguments and resolutions cross the implementation's serialization boundary whole. - `CodeRunResult = { value?: CodeJsonValue; logs: string[]; error?: CodeRunFailure }` — program execution outcomes resolve as the `error` field. `run()` may reject only for caller/seam misuse (for example a duplicate binding namespace); consumers still contain a non-conforming backend rejection at their own error boundary. - `CodeRunFailure = { kind: 'exception' | 'timeout' | 'abort' | 'worker-exit' | 'invalid-output' | 'output-limit'; message: string }` — orthogonal outcomes reported independently per [defensive patterns](../../../../docs/defensive-patterns.md); a timed-out run is not an exception, an abort is not a timeout, a lossy completion is not an overflow, and a substrate exit is none of them. -- Two readonly backend descriptors, informational not gating: `language` (what the program must be written in — `'typescript'` for the first backend; a Python backend says `'python'` and pairs with its own SDK generator on the presentation side) and `isolation` (`'worker-thread'` for the shipped backend; `'process'`, `'container'`, … for future ones). `dsh-tools` accepts any `language` with a registered SDK renderer and `run_code` flavor (TypeScript and Python ship; see the [language-dispatch note](2026-07-31-code-mode-language-dispatch.md)) and fails the assembly loudly otherwise, the same misconfiguration idiom as `toolOrder` violations (as when `mode` is non-native with no `ctx.codeRuntime` loaded at all). +- Two readonly backend descriptors, informational not gating: `language` (what the program must be written in — `'typescript'` for both shipped backends; a Python backend would pair with its own SDK generator) and `isolation` (`'worker-thread'` for both shipped backends; the subprocess provider may add a container boundary). `dsh-tools` requires `language === 'typescript'` in the MVP — its codegen emits TS — and fails the assembly loudly otherwise, the same misconfiguration idiom as `toolOrder` violations (as when `mode` is non-native with no `ctx.codeRuntime` loaded at all). Requests contain every runtime input; implementations own validated timeout and cap defaults. The registry looks up the optional runtime only when Code Mode is assembled, so native mode does not depend on one. Missing or language-incompatible runtimes fail loudly. Alternate substrates or languages can replace the implementation behind the same seam, paired with the appropriate SDK generator. @@ -79,21 +79,23 @@ Requests contain every runtime input; implementations own validated timeout and 5. **Enforce independent budgets.** `computeMs` meters worker busy time, allowing slow awaited tools without excusing a hot loop. `maxWallMs` bounds total elapsed time, including unresolved waits. `maxOutputBytes` bounds only the combined serialized outer logs, completion, or diagnostic; intermediate binding values have no byte cap. Expiry, cancellation, and completion terminate the worker, and heap exits or outer overflow are explicit failures. 6. **Dispose to quiescence**: the service's own disposal terminates in-flight workers and *awaits* their exits before resolving, per [defensive patterns](../../../../docs/defensive-patterns.md). +`@deepseek-ai/dsh-code-runtime-subprocess` preserves those program, binding, output, and worker-budget semantics across a filesystem/subprocess execution world. It writes a dependency-free runner below `ctx.subprocess.runtimeRoot`, resolves Node through the provider, and carries binding traffic over bounded base64 JSON frames on raw pipes. The [portable execution-world decision](../architecture/2026-07-28-portable-execution-world-consumers.md) owns why this generic backend replaces provider-specific Code Runtime packages; `dsh-code-runtime-worker` remains the single-process and single-file-distribution path. + ### Trust posture -The worker runtime provides containment, not a security boundary: model code can reach Node APIs and has authority comparable to the bash tool. `worker.terminate()` stops the thread but not OS processes it spawned. Code Mode uses the same `tools/pre-execute` policy gate as bash and adds an empty environment, heap limits, a separate isolate, and hard termination of the program itself. Deployments that need a hard multi-tenant boundary need a container-class backend for both code and bash; the runtime's isolation descriptor lets them distinguish that backend. +The runtimes provide containment, not an independent security boundary: model code can reach Node APIs and has the authority of its mounted execution world. Code Mode uses the same `tools/pre-execute` policy gate as bash and adds an empty worker environment, heap limits, a separate isolate, and hard termination of the program itself. Deployments that need a hard multi-tenant boundary mount container-class filesystem/subprocess providers for both code and bash. ### What the model sees -The SDK instructs the model to write an async body in the loaded runtime's language (an erasable-TypeScript body by default; a Python `async` body under a Python runtime — see the [language-dispatch note](2026-07-31-code-mode-language-dispatch.md)), call tools through `await tools.name(args)`, catch rejected tool calls when needed, and return or log only the output that should re-enter context. Both flavors state the same contract in their own primitive: independent read-only calls MAY overlap under `Promise.all` (TypeScript) or `asyncio.gather` (Python), mutating calls run alone in submission order, and dependent work sequences with `await`. The declaration prefix can be as large as native schemas, especially in `'both'`, but remains stable for provider caching. +The SDK instructs the model to write an async erasable-TypeScript body, call tools through `await tools.name(args)`, catch rejected tool calls when needed, and return or log only the output that should re-enter context. Calls remain sequential even under `Promise.all`. The declaration prefix can be as large as native schemas, especially in `'both'`, but remains stable for provider caching. ## Consequences -Deployments switching to `'code'` must update any native-only `toolOrder`. Assembly listeners own the integrity of any rewritten protocol surface. Sub-dispatch starts in submission order under a bounded overlap pool, while per-call contexts retain their source, envelope, and metadata through the outer result. +Deployments switching to `'code'` must update any native-only `toolOrder`. Assembly listeners own the integrity of any rewritten protocol surface. Sub-dispatch remains serialized, while per-call contexts retain their source, envelope, and metadata through the outer result. ## Testing -- **Worker runtime:** Real-worker tests cover typed binding values and failures, every lossless JSON completion root, invalid and over-limit output, exact combined ledger boundaries, compute and wall budgets, hostile binding traffic, empty environment, and disposal to quiescence. A built-package test runs the worker entry under plain Node. +- **Runtime implementations:** Real-worker suites cover typed binding values and failures, every lossless JSON completion root, invalid and over-limit output, exact combined ledger boundaries, compute and wall budgets, hostile binding traffic, empty environment, and disposal to quiescence. Built-package tests run both the direct worker entry and the filesystem/subprocess composition under plain Node; the latter also has a Loader-driven `cordis.yml` test. - **Registry integration:** Tests cover code generation, all presentation modes, reserved-name and restriction rules, scoped visibility, authoritative assembly rewrites, `toolOrder`, runtime compatibility failures, full-pipeline sub-dispatch, parent-token correlation, serialization, cancellation and queue drain, JSON normalization, error propagation, log events, ordered context deferral across successful and failed programs, outer-block suppression, and HMR cleanup. - **With-key e2e:** A real model composes two bash calls in one program; another discovers nested workspace instructions through a Code Mode fs dispatch. The tests verify collapsed request headers, correlated dispatch events, resulting files, deferred context, and model behavior. - **Snapshot:** The `code-mode-turn`, `both-mode-turn`, and `code-mode-workspace-context` fixtures pin SDK text, header tool lists, dispatch events, deferred context, and result cards. @@ -106,7 +108,7 @@ Deployments switching to `'code'` must update any native-only `toolOrder`. Assem **Result elision / summarization over native tool-calling.** Addresses only the context-bloat half of the problem: trimming old `tool-result`s is cheap to add as a logged surface replacement under reconstructable requests, but still pays one model round-trip per call and cannot express loops, branches, or joins. Complementary, not competing; it can layer under Code Mode for residual native calls. -**Parallel native dispatch in the loop.** The other answer to round-trip cost at decision time; it was blocked on concurrency-safety metadata and offers no composition either way — it parallelizes calls the model already decided on in one step. Code Mode's queue decision kept the two compatible, and that is how it played out: the metadata landed as `isConcurrencySafe` (the [parallel tool-call note](2026-07-10-parallel-tool-call-execution.md)), and native rolling-pool dispatch and per-tool binding parallelism unlocked on the same classifier. +**Parallel native dispatch in the loop.** The other answer to round-trip cost; still valid future work (the open TODO), still blocked on concurrency-safety metadata, and still no composition — it parallelizes calls the model already decided on in one step. Code Mode's serialized-queue decision keeps the two compatible: when the metadata lands, both native parallel dispatch and per-tool binding parallelism unlock together. **Always-exclusive (Cloudflare-faithful, no mode).** Rejected for this SDK's primary consumer: a coding agent's bread-and-butter single calls (`bash`, `read`, `edit`) are already ideal as native calls, and forcing every edit through a program taxes the common case. The mode config keeps the faithful form (`'code'`) one line away without imposing it. @@ -118,7 +120,7 @@ Deployments switching to `'code'` must update any native-only `toolOrder`. Assem ## Risks -**The worker is not a hard security boundary.** Deliberate and documented (§Trust posture): posture equals the existing bash tool, containment exceeds it, gating uses the same seams. Deployments needing more need a future `isolation: 'container'` backend — tracked as the seam's designed extension, not a TODO on this design. +**A worker is not a hard security boundary.** Deliberate and documented (§Trust posture): posture equals the existing bash tool and gating uses the same seams. The subprocess implementation can run inside a container-class execution world, but its worker descriptor does not itself claim that boundary. **`stripTypeScriptTypes` is marked experimental.** It is the same engine (amaro/swc) behind Node's own native `.ts` execution, exposed as an API across this repo's whole engines range. Mitigations: the runtime's unit suite pins the behaviors relied on (position preservation, erasable-only rejection message shape loosely), the call sits behind one private function, and `amaro`/`sucrase` are drop-in replacements if the API shifts. The erasable-only subset is a model-facing contract line, and the error path is a working feedback loop, not a dead end. @@ -126,8 +128,8 @@ Deployments switching to `'code'` must update any native-only `toolOrder`. Assem **Registry scope growth.** `dsh-tools` absorbs codegen, a tool, a bridge, and an event. Contained by module boundaries inside the package (`ts-types.ts`, `code-mode.ts` beside `schema.ts`/`json-schema.ts`/`presentation.ts`) and by the seam: everything substrate-shaped lives behind `ctx.codeRuntime`. -**Large lossless JSON values can exhaust memory.** Tool bindings snapshot lossless JSON before dispatch and return canonical JSON resolutions whole. The runtime validates both sides of the worker port and applies no per-binding byte cap; structured-clone cost and process or worker memory are the practical bounds. The combined outer-output ledger for logs, the completion value, and a failure diagnostic is the only byte-capped boundary. +**Large lossless JSON values can exhaust memory.** Tool bindings snapshot lossless JSON before dispatch and return canonical JSON resolutions whole. The direct worker runtime applies no per-binding byte cap; the subprocess runtime bounds each transport frame with `maxFrameBytes`, but repeated or concurrent calls can still consume process or worker memory. The combined outer-output ledger for logs, the completion value, and a failure diagnostic is separately bounded by `maxOutputBytes`. -**Sub-dispatch overlap is bounded by tool safety claims, not by the caller.** A program's `Promise.all` or `asyncio.gather` buys wall-clock parallelism only across calls the tool itself classifies concurrency-safe; a run of exclusive calls still costs its round-trips in sequence, and models may over-expect. Both flavors' SDK instructions state the real contract. This note shipped the serialized placeholder that made the risk absolute; the [live-parallel Agent Note](2026-07-26-code-mode-live-parallel-dispatch.md) owns the scheduler and its overlap cap. +**Serialized-only sub-dispatch.** `Promise.all` gains no wall-clock parallelism yet, only fewer round-trips; models may over-expect. The instructions state it; lifting it is tied to the same concurrency-safety metadata the native parallel-dispatch TODO needs. **Budget metering reads the event loop, not a flag.** Busy-time polling (`eventLoopUtilization()`) is coarser than an exact CPU meter — a budget expires up to one poll interval late — and its correctness claim ("a pending dispatch cannot pause it") is load-bearing against a hostile program. Both sides are unit-tested (hot loop with a pending decoy dispatch dies at `computeMs`; idle-on-slow-binding survives to `maxWallMs`), and the poll interval is an internal constant, not config — nothing a deployment could mis-tune into a bypass. `maxWallMs` is config, and it reaches `setTimeout`, which clamps a delay above `MAX_TIMER_DELAY_MS` (2^31-1 ms) to 1 ms; a positivity check alone therefore accepts a 25-day ceiling that expires on the first tick and times out every run. The worker runtime range-checks the field at load for that reason. `computeMs` needs no upper bound because it is compared against measured utilization instead of being handed to a timer. diff --git a/.agents/notes/implemented/feature/2026-06-15-code-mode.zh.md b/.agents/notes/implemented/feature/2026-06-15-code-mode.zh.md index ca1bbe9ed3..4ba098d87d 100644 --- a/.agents/notes/implemented/feature/2026-06-15-code-mode.zh.md +++ b/.agents/notes/implemented/feature/2026-06-15-code-mode.zh.md @@ -6,7 +6,7 @@ Status: implemented ## 问题 -在注册表的原生呈现方式下,agent loop(智能体循环)将每个可见能力以 JSON Schema 函数定义的形式通告给模型。`ToolRegistry` 将其 schema 贡献给系统提示词组装,组装结果中的 `tools` 落到协议格式(wire format)上(也记录在请求头日志中),模型每步调用一个 `tool-call` 块,而在本 note 写作时,循环通过 `ctx.tools.execute()` **逐个**分发每次调用(并行工具执行当时还是 open TODO;此后有界的并行分发已经交付——见[并行工具调用 note](2026-07-10-parallel-tool-call-execution.md),以及 [docs/architecture.md](../../../../docs/architecture.md) 中的 rolling pool)——且**每一个**中间 `tool-result` 都会在下一次请求时重新进入模型上下文。 +在注册表的原生呈现方式下,agent loop(智能体循环)将每个可见能力以 JSON Schema 函数定义的形式通告给模型。`ToolRegistry` 将其 schema 贡献给系统提示词组装,组装结果中的 `tools` 落到协议格式(wire format)上(也记录在请求头日志中),模型每步调用一个 `tool-call` 块,循环通过 `ctx.tools.execute()` **逐个**分发每次调用(并行工具执行是 `dsh-tools` 和 [docs/architecture.md](../../../../docs/architecture.md) 中明确标注的 open TODO),且**每一个**中间 `tool-result` 都会在下一次请求时重新进入模型上下文。 对于多步工具操作,这种方式 token 开销大且串行。模型无法组合工具——遍历结果集、根据中间值分支、扇出、后处理——每次调用都需要一次完整的模型往返,而每次往返都会把完整的中间结果拖回上下文,不管模型是否需要。 @@ -19,8 +19,8 @@ Cloudflare 的 [Code Mode](https://blog.cloudflare.com/code-mode/) 提出了一 三项决策,各自在下方独立小节中展开: 1. **Code Mode 是 `ToolRegistry`(`dsh-tools`)的一等呈现模式**,通过经校验的 `mode` 配置选择:`'native'`(默认,贡献可见能力 schema)、`'code'`(注册表仅贡献其保留的 `run_code` 传输通道加一份生成的 SDK `.d.ts` 到系统提示词中)或 `'both'`(原生 schema 加传输通道 + SDK)。注册表在源头塑造其权威贡献;协作式提示词组装的结果仍具权威性,记录在日志中的请求头精确反映该返回的呈现。 -2. **代码执行是一个能力 seam**——`packages/code-runtime/` 包含接口包`@deepseek-ai/dsh-code-runtime`,拥有 `ctx.codeRuntime`([能力 seam](../architecture/2026-06-13-capability-seams.md);消费方 = `dsh-tools`,core 消费 seam 的先例见 `agent-loop` → `dsh-llm`)。运行时对工具一无所知:它接收一段程序和命名的异步绑定,执行程序,报告 `{ value, logs, error? }`。语言和基底是后端属性,因此未来的 Python 或容器后端只是另一个实现包,而非重新设计。 -3. **交付的实现是 `@deepseek-ai/dsh-code-runtime-worker`**:每次运行 spawn 一个全新的 Node worker 线程,对模型的 TypeScript 进行 type-strip 后执行,绑定通过消息端口桥接,环境为空,堆/输出/时间上限可配置,并支持硬终止。其信任姿态在设计上等同于 bash——无需 unsafe-acknowledgement flag——因为 harness 已经交付了 `dsh-bash-local`,后者以严格*更高*的环境权限执行模型编写的任意 shell 命令。 +2. **代码执行是一个能力 seam**——`packages/code-runtime/` 包含接口包 `@deepseek-ai/dsh-code-runtime`,拥有 `ctx.codeRuntime`([能力 seam](../architecture/2026-06-13-capability-seams.md);消费方 = `dsh-tools`,core 消费 seam 的先例见 `agent-loop` → `dsh-llm`)。运行时对工具一无所知:它接收一段程序和命名的异步绑定,执行程序,报告 `{ value, logs, error? }`。语言和基底是后端属性,因此未来的 Python 或容器后端只是另一个实现包,而非重新设计。 +3. **两种实现保持同一份全新 worker 契约**:`@deepseek-ai/dsh-code-runtime-worker` 在 harness 进程内运行 worker,`@deepseek-ai/dsh-code-runtime-subprocess` 则通过 `ctx.fs` 物化 runner,再通过 `ctx.subprocess` 在另一执行环境中启动。二者都在具有空环境的全新 Node worker 内执行由宿主剥离类型的 TypeScript,并提供桥接绑定、可配置的堆/输出/时间上限和硬终止。其信任姿态在设计上等同于 bash;更强的隔离来自挂载的执行环境。 本说明负责定义 Code Mode 的呈现、组合、隔离与结算基础。后续的[类型化工具返回值 Agent Note](2026-07-20-code-mode-typed-tool-returns.md)负责定义生成的输出映射、规范绑定值、`ToolCallError` 和无损外层输出边界。 @@ -32,7 +32,7 @@ Cloudflare 的 [Code Mode](https://blog.cloudflare.com/code-mode/) 提出了一 **与 `toolOrder` 的交互,预先说明:** 如果配置的 `systemPrompt.toolOrder` 引用了原生能力名称,在 `mode: 'code'` 下会拒绝所有组装,因为那些名称不在该模式的协议校验范围内。这是正确行为而非 bug:使用 Code Mode 的部署需要更新其 order 配置或移除它。 -**SDK 提示词段。** 在 `'code'` 和 `'both'` 下,tool-guidance order band 中的惰性 `tools:sdk` 段为当前 scope 的可见能力渲染所加载运行时语言的声明加固定的使用说明(默认 TypeScript;[语言分发 note](2026-07-31-code-mode-language-dispatch.md) 加入了 Python 与按 `ctx.codeRuntime.language` 选择的渲染器表)。它共享查找和执行可见性,排除 `run_code`,并按字典序排列工具以获得字节稳定的输出。 +**SDK 提示词段。** 在 `'code'` 和 `'both'` 下,tool-guidance order band 中的惰性 `tools:sdk` 段为当前 scope 的可见能力渲染 TypeScript 声明加固定的使用说明。它共享查找和执行可见性,排除 `run_code`,并按字典序排列工具以获得字节稳定的输出。 **组装所有权。** `run_code` 和 `tools:sdk` 作为正常的组装输入进入受信任的 `system-prompt/assemble` waterfall。一个 scoped 的 `tools:sdk` 段可以在分发前遮蔽全局默认值,监听器也可以移除或替换任一贡献。waterfall 返回的组装结果是最终的,因此修改这些输入的人有责任在部署期望 Code Mode 可用时保持协议面的完整性;没有恢复 pass 会覆盖有意的组合。 @@ -40,7 +40,7 @@ Cloudflare 的 [Code Mode](https://blog.cloudflare.com/code-mode/) 提出了一 ### run_code 工具与分发桥 -在 `'code'` 和 `'both'` 下,注册表拥有 `run_code` 作为保留的呈现传输通道,带两个必需参数 `{ code: string; description: string }`(description 为 UI 标注该调用,沿用 bash 的先例)。它由一个正常的 `ToolDefinition` 表示以供分发,但位于可过滤的能力层之外,因此限制规则不会意外移除 Code Mode 的唯一入口。调用遍历完整的工具流水线——`tools/pre-execute` → 单调性守卫 → `tools/execute` 包裹分发 → `tools/post-execute` → 由定义拥有的可选 `finalizeContent` → 不可变的 `tools/result` 通知——与原生调用完全一致;权限插件可以在程序运行前检查程序文本,最终结果观察者看到的是规范化的外层结果。其 `execute(args, exec)`: +在 `'code'` 和 `'both'` 下,注册表拥有 `run_code` 作为保留的呈现传输通道,带两个必需参数 `{ code: string; description: string }`(description 为 UI 标注该调用,沿用 bash 的先例)。它由一个正常的 `ToolDefinition` 表示以供分发,但位于可过滤的能力层之外,因此限制规则不会意外移除 Code Mode 的唯一入口。调用遍历完整的工具流水线——`tools/pre-execute` → 单调守卫 → `tools/execute` 包裹分发 → `tools/post-execute` → 由定义拥有的可选 `finalizeContent` → 不可变的 `tools/result` 通知——与原生调用完全一致;权限插件可以在程序运行前检查程序文本,最终结果观察者看到的是规范化的外层结果。其 `execute(args, exec)`: 1. **构建绑定。** 一个 run 级别的 signal 跟随外层取消,并在 run 结算时被 abort。每个可见工具绑定都会对无损 JSON 参数创建快照,进入原生契约的分发池(调度设计由[实时并行 Agent Note](2026-07-26-code-mode-live-parallel-dispatch.md) 负责),以确定性的 call id 和外层 token 作为 `parent` 执行,通过外层 execution 延后返回的上下文,并记录 `tool/code-dispatch-start`/`tool/code-dispatch` 事件对,其中结算侧携带完整渲染后的结果内容。成功时返回工具最终的规范 JSON 值;失败则变为程序可见的 `ToolCallError`。每个子调用保留自己不可变的执行标识,并遍历完整的工具流水线。 2. **运行程序**:`ctx.codeRuntime.run({ program: args.code, bindings: [{ global: 'tools', functions }], signal: runController.signal })`。运行时接收的是 run 级别的 signal 而非仅调用方的外层 signal,因此外层 run 以任何方式结算都会同时 abort 运行时内部的工作。 @@ -48,7 +48,7 @@ Cloudflare 的 [Code Mode](https://blog.cloudflare.com/code-mode/) 提出了一 **子调用上下文通过父调用延后。** 在 `run_code` 内部注入会破坏父调用/结果的相邻性,因此 `ToolRunContext.deferContext()` 按分发顺序收集每个子结果的 `additionalContexts` 条目。即使程序后来抛出异常,注册表仍携带该数组;循环只在外层结果与步骤中所有兄弟结果之后追加每个条目。外层 post-execute 阻止会丢弃工具延后的条目,只暴露阻止 decision 显式附加的上下文。 -**并发是有界的,而非被序列化。** 每次 run 拥有一个分发队列,严格按提交顺序启动调用,并通过 `registry.executionMode` 对每个调用分类——与原生循环所用的 fail-closed `isConcurrencySafe` 契约相同。连续的 parallel 类调用最多重叠 `maxParallelSubCalls` 个(默认 10;设为 `1` 恢复串行分发);exclusive 类调用会排空池并单独运行。结算时放弃尚未开始的排队调用。本 note 交付的是被序列化的占位实现;取代它的调度器由[实时并行 Agent Note](2026-07-26-code-mode-live-parallel-dispatch.md) 负责。 +**并发被序列化。** 每次 run 拥有一个分发队列,因此即使 `Promise.all` 也按提交顺序执行工具调用。结算时放弃尚未开始的排队调用。并行化需要每个工具的并发安全元数据。 **呈现。** `run_code` 的 render intent 按[呈现意图 Agent Note](../architecture/2026-07-02-tool-render-intent-union.md)在此决定:`presentCall` 创建一个 `generic` 卡片,`kind: 'execute'`,以程序文本作为标题,并将同一程序文本作为 `rawInput`;`run_code` 有意不声明 `presentResult`,因此 TUI 和宿主/客户端运行时(Web)会通过通用原始内容回退机制,使用最终持久化的 `tool/result.content` 补全该卡片,其中包括捕获的日志,以及返回值、失败信息或 post-policy 输出落盘预览。这不是 `terminal` 卡片:该卡片的语义是「工作目录中的 shell 命令」,程序不是。参见[结果卡片完整性说明](../../archived/bug-fix/2026-07-20-code-mode-result-card-completeness.md)。 @@ -61,39 +61,41 @@ Cloudflare 的 [Code Mode](https://blog.cloudflare.com/code-mode/) 提出了一 `packages/code-runtime/code-runtime/`——`@deepseek-ai/dsh-code-runtime`,仅依赖 `cordis`。一个抽象的 `CodeRuntime extends Service`(`super(ctx, 'codeRuntime')`)加上词汇: - `CodeRunRequest = { program: string; bindings: CodeBindingNamespace[]; signal?: AbortSignal }` -- `CodeBindingNamespace = { global: string; functions: Record Promise>; errorClass?: { name: string; memberNameProperty: string } }`——运行时将每个命名空间作为程序内部的全局异步函数对象暴露;可选描述符要求运行时注入真正的、程序可见的 reject 类,而无需让 seam 获知消费方专用名称。`CodeJsonValue` 是这个低依赖 seam 的结构化无损 JSON 类型,因此绑定参数与返回值可以完整跨越实现的序列化边界。 +- `CodeBindingNamespace = { global: string; functions: Record Promise>; errorClass?: { name: string; memberNameProperty: string } }`——运行时将每个命名空间作为程序内部的全局异步函数对象暴露;可选描述符要求运行时注入真正的、程序可见的 reject 类,而无需让 seam 获知消费方专用名称。`CodeJsonValue` 是这个低依赖 seam 的结构化无损 JSON 类型,因此绑定参数与解析值可以完整跨越实现的序列化边界。 - `CodeRunResult = { value?: CodeJsonValue; logs: string[]; error?: CodeRunFailure }`——程序执行失败时,执行 promise 仍会 fulfill,并通过 `error` 字段返回失败结果。只有调用方/seam 误用(例如重复的绑定命名空间)时,`run()` 才会 reject;消费方仍在自己的错误边界处理不合规后端的拒绝。 - `CodeRunFailure = { kind: 'exception' | 'timeout' | 'abort' | 'worker-exit' | 'invalid-output' | 'output-limit'; message: string }`——按[防御性模式](../../../../docs/defensive-patterns.md)独立报告的正交结果;超时的 run 不是异常,abort 不是超时,有损完成值不是溢出,基底退出也与上述情况相互独立。 -- 两个只读的后端描述符,仅供信息参考而非门禁判定:`language`(程序必须使用的语言——首个后端为 `'typescript'`;Python 后端声明 `'python'`,并在呈现侧配对自己的 SDK 生成器)和 `isolation`(交付的后端为 `'worker-thread'`;未来可为 `'process'`、`'container'` 等)。`dsh-tools` 接受任何注册了 SDK 渲染器与 `run_code` flavor 的 `language`(TypeScript 与 Python 已交付;见[语言分发 note](2026-07-31-code-mode-language-dispatch.md)),否则组装会大声失败,与 `toolOrder` 违规时的配置错误惯用法相同(如 `mode` 为非 native 但根本没有加载 `ctx.codeRuntime`)。 +- 两个只读的后端描述符,仅供信息参考而非门禁判定:`language`(程序必须使用的语言——两种已交付后端均为 `'typescript'`;Python 后端会配对自己的 SDK 生成器)和 `isolation`(两种已交付后端均为 `'worker-thread'`;子进程提供方可以增加容器边界)。`dsh-tools` 在 MVP 中要求 `language === 'typescript'`——其代码生成输出 TS——否则组装会大声失败,与 `toolOrder` 违规时的配置错误惯用法相同(如 `mode` 为非 native 但根本没有加载 `ctx.codeRuntime`)。 请求包含所有运行时输入;实现方拥有经校验的超时和上限默认值。注册表仅在组装 Code Mode 时查找可选的运行时,因此 native 模式不依赖它。缺失或语言不兼容的运行时会大声失败。替代基底或语言可以在同一 seam 背后替换实现,配对相应的 SDK 生成器。 ### worker-thread 运行时 -`@deepseek-ai/dsh-code-runtime-worker`,`packages/code-runtime/` 组的第二个包。每次 `run()`: +`@deepseek-ai/dsh-code-runtime-worker`,`packages/code-runtime/` 组的第二个包(package)。每次 `run()`: -1. **宿主侧 type-strip**,使用 Node 内置的 `stripTypeScriptTypes`(`node:module`;在本仓库的整个引擎范围 `^22.19.0 || >=24.0.0` 内可用,且会保留源码位置,因此运行时错误行号与模型源码一致)。仅剥离模式拒绝不可擦除的语法(`enum`、namespaces)——该拒绝以 `error.kind: 'exception'` 加 Node 的消息返回,SDK 说明写明「仅限可擦除 TypeScript」,模型像处理其他程序错误一样自我修正。语法级失败不会 spawn worker。 +1. **宿主侧 type-strip**,使用 Node 内置的 `stripTypeScriptTypes`(`node:module`;在本仓库的整个引擎范围 `^22.19.0 || >=24.0.0` 内可用,且保持位置不变,因此运行时错误行号与模型源码一致)。仅剥离模式拒绝不可擦除的语法(`enum`、namespaces)——该拒绝以 `error.kind: 'exception'` 加 Node 的消息返回,SDK 说明写明「仅限可擦除 TypeScript」,模型像处理其他程序错误一样自我修正。语法级失败不会 spawn worker。 2. **每次 run spawn 一个全新 `Worker`**,来自包自身的 bootstrap 模块:`env: {}`(真正为空——比 spawn 命令的 scrubbed-env 规则更严格),`resourceLimits` 来自配置,`stdout`/`stderr` 捕获到 `logs` 而非继承。不做池化,不跨 run 保留状态:程序的世界随 worker 消亡,这使得 run 仅从日志即可重建,状态泄漏不可表达。 3. **在 bootstrap 中执行**:剥离后的程序成为一个 `AsyncFunction` 的函数体,其参数是绑定全局变量、消费方声明的 reject 类和一个捕获式 `console` shim,因此顶层 `await` 和 `return` 可用。Code Mode 声明 `ToolCallError`,成员属性为 `toolName`;运行时无需硬编码工具即可实体化真正的构造函数。无损 JSON 完成值会精确跨越边界;`undefined` 仍表示缺席,有损值产生 `invalid-output`,过大的外层结果产生 `output-limit`,而不会退化为检查格式化后的字符串替代品。 4. **通过消息端口桥接绑定**:worker 中的每个绑定函数发送 `{ id, global, name, args }` 并等待回复;宿主根据请求的绑定校验名称、调用、并回复 `{ id, ok, value }` 或 `{ id, ok: false, message }`(宿主侧绑定拒绝变为程序侧 rejection)。worker 侧的命名空间对象通过 `defineProperty` 构建为 null-prototype,因此名为 `__proto__`、`constructor` 或 `toString` 的绑定是普通自有属性,而非原型链碰撞。未知名称、重复 id 和结算后消息被拒绝或忽略——端口协议假设对端是恶意的,因为对端运行的是模型代码。 5. **强制独立预算。** `computeMs` 计量 worker 忙碌时间,允许慢速的 awaited 工具而不放过热循环。`maxWallMs` 约束总经过时间,包括未解析的等待。`maxOutputBytes` 只约束序列化后的外层日志、完成值或诊断的组合;中间绑定值没有字节数上限。到期、取消和完成都终止 worker,堆退出或外层溢出会作为显式失败报告。 -6. **dispose(资源释放)至完全停稳**:服务自身的 dispose 终止进行中的 worker 并*等待*其退出后再 resolve,遵循[防御性模式](../../../../docs/defensive-patterns.md)。 +6. **dispose 至完全停稳**:服务自身的 dispose(资源释放)终止进行中的 worker 并*等待*其退出后再 resolve,遵循[防御性模式](../../../../docs/defensive-patterns.md)。 + +`@deepseek-ai/dsh-code-runtime-subprocess` 在文件系统/子进程执行环境中保持相同的程序、绑定、输出与 worker 预算语义。它在 `ctx.subprocess.runtimeRoot` 下写入一个无依赖 runner,通过提供方解析 Node,并在原始管道上使用有界 base64 JSON 帧承载绑定通信。[可移植执行环境决策](../architecture/2026-07-28-portable-execution-world-consumers.md)说明为何这个通用后端会取代提供方专用的 Code Runtime 包;`dsh-code-runtime-worker` 仍用于单进程和单文件发行版。 ### 信任姿态 -worker 运行时只能约束程序的运行,而不构成安全边界:模型代码可以访问 Node API,权限与 bash 工具相当。`worker.terminate()` 停止线程但不停止它 spawn 的 OS 进程。Code Mode 使用与 bash 相同的 `tools/pre-execute` 策略门禁,并额外提供空环境、堆限制、独立 isolate 和对程序本身的硬终止。需要硬多租户边界的部署需要为代码和 bash 都使用容器级后端;运行时的 isolation 描述符让它们能区分该后端。 +这些运行时提供的是隔离,而非独立安全边界:模型代码可以访问 Node API,并拥有挂载的执行环境所授予的权限。Code Mode 使用与 bash 相同的 `tools/pre-execute` 策略门禁,并额外提供空 worker 环境、堆限制、独立 isolate 和对程序本身的硬终止。需要硬多租户边界的部署应当为代码和 bash 都挂载容器级文件系统/子进程提供方。 ### 模型看到的内容 -SDK 指示模型编写一个所加载运行时语言的异步函数体(默认可擦除 TypeScript;Python 运行时下为 Python `async` 函数体——见[语言分发 note](2026-07-31-code-mode-language-dispatch.md)),通过 `await tools.name(args)` 调用工具,在需要时捕获被拒绝的工具调用,并仅 return 或 log 应重新进入上下文的输出。两种 flavor 用各自的原语陈述同一契约:相互独立的只读调用可以在 `Promise.all`(TypeScript)或 `asyncio.gather`(Python)下重叠,有副作用的调用按提交顺序单独运行,有依赖的工作用 `await` 排序。声明前缀可能与原生 schema 一样大,尤其在 `'both'` 下,但对提供方缓存保持稳定。 +SDK 指示模型编写一个异步的可擦除 TypeScript 函数体,通过 `await tools.name(args)` 调用工具,在需要时捕获被拒绝的工具调用,并仅 return 或 log 应重新进入上下文的输出。即使在 `Promise.all` 下调用仍保持顺序。声明前缀可能与原生 schema 一样大,尤其在 `'both'` 下,但对提供方缓存保持稳定。 ## 后果 -切换到 `'code'` 的部署必须更新任何仅限 native 的 `toolOrder`。组装监听器有责任维护任何被重写的协议面的完整性。子分发在有界的重叠池下按提交顺序启动,而每次调用的上下文会通过外层结果保留其 source、信封与元数据。 +切换到 `'code'` 的部署必须更新任何仅限 native 的 `toolOrder`。组装监听器有责任维护任何被重写的协议面的完整性。子分发保持序列化,而每次调用的上下文会通过外层结果保留其 source、信封与元数据。 ## 测试 -- **Worker 运行时:** 真实 worker 测试覆盖类型化的绑定值与失败、每一种无损 JSON 根类型的完成值、无效和超限输出、精确的组合账本边界、compute 和 wall 预算、恶意绑定流量、空环境以及 dispose 至完全停稳。一个构建后包测试在纯 Node 下运行 worker 入口。 +- **运行时实现:** 真实 worker 测试套件覆盖类型化的绑定值与失败、每一种无损 JSON 根类型的完成值、无效和超限输出、精确的组合账本边界、compute 和 wall 预算、恶意绑定流量、空环境以及 dispose 至完全停稳。构建后包测试会在纯 Node 下分别运行直接 worker 入口与文件系统/子进程组合;后者另有一个由 Loader 驱动的 `cordis.yml` 测试。 - **注册表集成:** 测试覆盖代码生成、所有呈现模式、保留名称和限制规则、scoped 可见性、权威组装重写、`toolOrder`、运行时兼容性失败、完整流水线子分发、parent-token 关联、序列化、取消和队列排空、JSON 规范化、错误传播、日志事件、成功与失败程序中的有序上下文延后、外层阻止抑制以及 HMR(热模块替换)清理。 - **带密钥 e2e:** 真实模型在一个程序中组合两次 bash 调用;另一个模型通过 Code Mode fs 分发发现嵌套的工作区指令。测试验证折叠的请求头、关联的分发事件、结果文件、延后上下文和模型行为。 - **快照:** `code-mode-turn`、`both-mode-turn` 和 `code-mode-workspace-context` fixture(测试前置数据)固定 SDK 文本、请求头工具列表、分发事件、延后上下文和结果卡片。 @@ -104,9 +106,9 @@ SDK 指示模型编写一个所加载运行时语言的异步函数体(默认 **`node:vm` 作为参考运行时,加固推迟。** 否决:`node:vm` 不是隔离(原型链逃逸可达宿主 realm)且无法中断热循环。worker 线程提供独立 isolate、空环境、`resourceLimits` 和可靠的 `terminate()`,信任等级等同于 bash,因此参考实现和生产实现是同一个包,无需 unsafe-acknowledgement 仪式。 -**在原生工具调用上做结果省略/摘要。** 仅解决问题中上下文膨胀这一半:裁剪旧 `tool-result` 作为可重建请求下的日志化表面替换成本低,但仍需每次调用一次模型往返,且无法表达循环、分支或汇合。互补而非竞争;它可以在 Code Mode 下为残余的原生调用分层。 +**在原生工具调用上做结果省略/摘要。** 仅解决问题的上下文膨胀一半:裁剪旧 `tool-result` 作为可重建请求下的日志化表面替换成本低,但仍需每次调用一次模型往返,且无法表达循环、分支或汇合。互补而非竞争;它可以在 Code Mode 下为残余的原生调用分层。 -**循环中的并行原生分发。** 决策当时对往返成本的另一个答案;它被并发安全元数据阻塞,且无论如何都不提供组合能力——它并行化的是模型在一步中已经决定的调用。Code Mode 的队列决策保持了两者兼容,后续也正是这样落地的:元数据以 `isConcurrencySafe` 的形式就绪(见[并行工具调用 note](2026-07-10-parallel-tool-call-execution.md)),原生 rolling-pool 分发与每工具绑定并行化基于同一个分类器一起解锁。 +**循环中的并行原生分发。** 往返成本的另一个答案;仍是有效的未来工作(open TODO),仍被并发安全元数据阻塞,且仍无组合能力——它并行化的是模型在一步中已经决定的调用。Code Mode 的序列化队列决策保持两者兼容:当元数据就绪时,原生并行分发和每工具绑定并行化一起解锁。 **始终排他(忠于 Cloudflare,无模式)。** 否决,因为本 SDK 的主要消费方是编码 agent:其日常的单次调用(`bash`、`read`、`edit`)作为原生调用已经是最优的,强制每次编辑都通过程序会给常见场景增加负担。mode 配置让忠实形式(`'code'`)只需一行配置即可启用,而不强加于人。 @@ -114,20 +116,20 @@ SDK 指示模型编写一个所加载运行时语言的异步函数体(默认 **SDK 中的清洁化标识符别名**(`my-tool` → `my_tool`,Cloudflare 的做法)。否决:`declare const` 上的带引号键使每个名称可达,零别名碰撞逻辑;模型能正常处理 `tools["my-tool"](…)`。 -**REPL 风格的持久内核**(状态跨 `run_code` 调用存活)。在 MVP 中否决:跨调用状态对会话日志不可见,破坏了「每个请求是日志的纯函数」这一可重建性保证;每次 run 均使用全新实例则维持了这一保证。内核风格的后端在未来仍可通过同一 seam 表达,配合自己的日志方案。 +**REPL 风格的持久内核**(状态跨 `run_code` 调用存活)。在 MVP 中否决:跨调用状态对会话日志不可见,破坏了「每个请求是日志的纯函数」这一可重建性保证;每次 run 全新保持了这一点。内核风格的后端在未来仍可通过同一 seam 表达,配合自己的日志方案。 ## 风险 -**Worker 不是硬安全边界。** 有意为之且已文档化(§信任姿态):姿态等同于既有的 bash 工具,约束能力强于它,门禁使用相同的 seam。需要更强隔离的部署需要未来的 `isolation: 'container'` 后端——作为 seam 设计的扩展点跟踪,而非本设计的 TODO。 +**worker 不是硬安全边界。** 有意为之且已文档化(§信任姿态):姿态等同于既有的 bash 工具,门禁使用相同的 seam。子进程实现可以在容器级执行环境中运行,但其 worker 描述符本身不声称具备该边界。 **`stripTypeScriptTypes` 标记为 experimental。** 它与 Node 自身原生 `.ts` 执行背后的引擎(amaro/swc)相同,在本仓库的整个引擎范围内作为 API 暴露。缓解措施:运行时的单元测试套件固定了所依赖的行为(位置保持、可擦除限制的拒绝消息形状宽松匹配),调用位于一个私有函数之后,且 `amaro`/`sucrase` 是 API 变化时的直接替代品。仅可擦除子集是面向模型的契约线,错误路径是一个可工作的反馈循环,而非死胡同。 **SDK 的提示词成本,尤其在 `'both'` 下。** `.d.ts` 可能与它补充的原生 schema 体量相当;`'both'` 携带两种表示。前缀稳定性 + 提供方缓存摊销了每会话成本;mode 是每部署的;本 Agent Note 不做无条件节省的声明。何时优先使用哪种模式的量化指导明确属于上线后学习。 -**注册表 scope 增长。** `dsh-tools` 吸收了代码生成、一个工具、一个桥和一个事件。包内的模块边界(`ts-types.ts`、`code-mode.ts` 与 `schema.ts`/`json-schema.ts`/`presentation.ts` 并列)和 seam 共同约束了这一增长:所有基底相关的内容都在 `ctx.codeRuntime` 之后。 +**注册表 scope 增长。** `dsh-tools` 吸收了代码生成、一个工具、一个桥和一个事件。通过包内的模块边界(`ts-types.ts`、`code-mode.ts` 与 `schema.ts`/`json-schema.ts`/`presentation.ts` 并列)和 seam 约束:所有基底相关的内容都在 `ctx.codeRuntime` 之后。 -**大型无损 JSON 值可能耗尽内存。** 工具绑定会在分发前对无损 JSON 创建快照,并完整返回规范 JSON 返回值。运行时会校验 worker 端口两侧,但不对单次绑定设置字节数上限;结构化克隆成本以及进程或 worker 内存构成实际边界。只有包含日志、完成值和失败诊断的组合外层输出账本受字节数上限约束。 +**大型无损 JSON 值可能耗尽内存。** 工具绑定会在分发前对无损 JSON 创建快照,并完整返回规范 JSON 解析值。直接 worker 运行时不对单次绑定设置字节数上限;子进程运行时使用 `maxFrameBytes` 限制每个传输帧,但重复或并发调用仍可能消耗进程或 worker 内存。包含日志、完成值和失败诊断的组合外层输出账本另由 `maxOutputBytes` 限制。 -**子分发的重叠由工具自身的安全声明限定,而非由调用方决定。** 程序里的 `Promise.all` 或 `asyncio.gather` 只在工具自己分类为并发安全的调用之间换来挂钟并行性;一串 exclusive 调用仍要按顺序付出各自的往返开销,模型可能过度期望。两种 flavor 的 SDK 说明都陈述了真实契约。本 note 交付的是使该风险绝对化的序列化占位实现;调度器及其重叠上限由[实时并行 Agent Note](2026-07-26-code-mode-live-parallel-dispatch.md) 负责。 +**仅序列化的子分发。** `Promise.all` 尚未获得挂钟并行性,仅减少往返次数;模型可能过度期望。说明中已声明;解除此限制与原生并行分发 TODO 所需的并发安全元数据绑定。 -**预算计量读取事件循环,而非 flag。** 忙碌时间轮询(`eventLoopUtilization()`)比精确 CPU 计量更粗糙——预算到期最多延迟一个轮询间隔——且其正确性声明(「pending 的分发不能暂停它」)是抵御恶意程序的关键。两种情况均有单元测试(带 pending 诱饵分发的热循环会在耗尽 `computeMs` 预算时终止;等待慢速绑定的空闲程序则会持续运行至 `maxWallMs`),轮询间隔是内部常量而非配置——部署无法将其误调为绕过手段。`maxWallMs` 是配置项,且会传入 `setTimeout`,后者会把超过 `MAX_TIMER_DELAY_MS`(2^31-1 ms)的延迟夹到 1 ms;因此仅有正数校验会放行一个 25 天的上限,它在第一个 tick 就到期,使每次运行都超时。worker 运行时正因如此在加载时对该字段做范围校验。`computeMs` 不需要上界,因为它对照的是实测占用率,而不是交给定时器。 +**预算计量读取事件循环,而非 flag。** 忙碌时间轮询(`eventLoopUtilization()`)比精确 CPU 计量更粗糙——预算到期最多延迟一个轮询间隔——且其正确性声明(「pending 的分发不能暂停它」)对恶意程序是承重的。两侧都有单元测试(带 pending 诱饵分发的热循环在 `computeMs` 处死亡;在慢绑定上空闲的程序存活到 `maxWallMs`),轮询间隔是内部常量而非配置——部署无法将其误调为绕过手段。`maxWallMs` 是配置项,且会传入 `setTimeout`,后者会把超过 `MAX_TIMER_DELAY_MS`(2^31-1 ms)的延迟夹到 1 ms;因此仅有正数校验会放行一个 25 天的上限,它在第一个 tick 就到期,使每次运行都超时。worker 运行时正因如此在加载时对该字段做范围校验。`computeMs` 不需要上界,因为它对照的是实测占用率,而不是交给定时器。 diff --git a/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.i18n.yaml b/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.i18n.yaml index 451a4ea0d3..6a801df865 100644 --- a/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.i18n.yaml +++ b/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md -2026-07-16-persistent-pty-sessions.md: d7d06dc8517780a37889e4f94dd0b99475dc5432 -2026-07-16-persistent-pty-sessions.zh.md: 84ef8988ba657a87137f904df38e45808d32b483 +2026-07-16-persistent-pty-sessions.md: 65d265b83ab5ad89c2b919364043eb28e75977c7 +2026-07-16-persistent-pty-sessions.zh.md: 6ea92da9a1b2870f3773d84e7f5629bec220dd1c diff --git a/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md b/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md index d7d06dc851..65d265b83a 100644 --- a/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md +++ b/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md @@ -23,10 +23,10 @@ The implementation supports interactive shells and line-oriented REPLs on Linux | Package | Role | ctx key | |---|---|---| | `dsh-pty` | `PtyService`, branded `PtySessionId`, backend registry, owner-scoped session contract, and result types | `ctx.pty` | -| `dsh-pty-local` | [`node-pty`](https://github.com/microsoft/node-pty)-based local backend, platform process inspection, bounded terminal buffer, sandbox resolution, and process-tree supervision | registers a backend on `ctx.pty` | +| `dsh-pty-local` | Persistent-shell backend over `ctx.subprocess.spawnTerminal()`: readiness, bounded terminal buffers, sandbox resolution, and owner-aware session lifecycle | registers a backend on `ctx.pty` | | `dsh-tool-pty` | Six model-facing tools, task-runtime integration for background sends, guidance, and UI render intents | registers on `ctx.tools` | -Idle detection is backend behavior, not a second public seam. A remote or container backend may have authoritative readiness signals that do not resemble local `/proc` inspection; every `PtyBackend` therefore returns the common send result while owning its detection mechanism internally. +Readiness remains PTY-backend behavior, not a second public seam. The terminal-process provider supplies only substrate facts such as the foreground process group and whether it can prove that group is waiting on input; `dsh-pty-local` combines those facts with prompt and silence evidence into the common send result. ### Agent ownership and identity @@ -40,12 +40,12 @@ Agent-scope disposal closes registrations first, then awaits quiescent teardown A registered `shell` backend constrains how a terminal starts; it does not constrain commands typed after startup. `dsh-pty-local` therefore applies two protections before spawning: -- It builds a scrubbed child environment using the same credential-shaped-name policy as `bash-local`, removing ambient `*KEY*`, `*PASSWORD*`, `*SECRET*`, `*TOKEN*`, and harness-managed variables unless an explicit trusted mapping supplies them. +- It supplies only terminal-specific environment overrides; the mounted subprocess provider applies the shared credential-shaped-name scrub before merging them. - It requires `ctx.sandbox` and the shared `ctx.sandboxPolicy`. At spawn, the backend resolves the owner's effective session mode over the deployment default and wraps the shell argv once; that mode and workspace root remain the process boundary for the PTY lifetime. A write that would change the effective `sandbox/mode` is rejected before commit while the owner has any open PTY or unpublished spawn, with an instruction to wait for creation to settle and close those sessions first; same-effective-mode writes remain valid. The pending reservation spans backend setup through publication, so there is no race in which a wider terminal appears after a downgrade. `danger-full-access` is the existing explicit unconfined choice rather than a PTY-specific bypass. Sandboxing confines local process effects but does not make arbitrary shell input safe: network calls and other external side effects remain governed by deployment policy. Tool descriptions state that PTY sessions are less auditable than one-shot tools and should be used only when persistence or interactive stdin is necessary. -The implementation uses only public `node-pty` capabilities: child PID, `data` and `exit` notifications, `write`, `resize`, and `kill`. It does not assume access to the native master fd or call `waitpid` from TypeScript. Platform process inspectors derive foreground process groups and parent/child identity from `/proc` on Linux and `ps` on macOS. +The local subprocess terminal primitive uses only public `node-pty` capabilities: child PID, `data` and `exit` notifications, `write`, and `kill`. It does not assume access to the native master fd or call `waitpid` from TypeScript. Platform process inspectors below that primitive derive foreground process groups and parent/child identity from `/proc` on Linux and `ps` on macOS. The [portable execution-world decision](../architecture/2026-07-28-portable-execution-world-consumers.md) owns this process/consumer split. ### Six model-facing tools @@ -92,9 +92,9 @@ Background sends use the existing task completion notice and `task_output` resul ### Process-tree teardown -The top-level `node-pty` child is the ownership anchor. On close, the backend stops callbacks, snapshots its transitive descendants by parent PID in children-first order, sends `SIGTERM`, waits, rescans for children forked during shutdown, sends `SIGKILL` to the remaining descendant tree, and verifies that every non-zombie descendant left the process table while the shell is still alive. A matching Linux zombie has no executable work and therefore counts as quiescent, allowing shell shutdown to reap or reparent it. Only then does the backend stop the shell with its own TERM/grace/KILL sequence. Every captured PID includes process-start identity so reuse cannot redirect escalation. +The subprocess terminal handle owns the top-level terminal process and its session. On close it snapshots transitive descendants by parent PID in children-first order, sends `SIGTERM`, waits, rescans for children forked during shutdown, sends `SIGKILL` to the union, and verifies every non-zombie descendant left the process table before stopping the top-level process. A matching Linux zombie has no executable work and therefore counts as quiescent. Every captured PID includes process-start identity so reuse cannot redirect escalation. -Teardown reports root exit and survivor cleanup independently. It does not claim success merely because the shell exited; disposal resolves only after no captured non-quiescent tree member remains or returns a cleanup failure naming the survivors. A failed close is not cached forever: the registry and local session each clear the fence only when it still names that failed attempt, so a later explicit or lifecycle close retries after the external survivor condition changes without disturbing a newer concurrent attempt. Service disposal still clears its backend, reservation, and owner-detacher registries when a close fails. It never broadens ownership to every member of the root PID's POSIX session. +Teardown reports top-level exit and survivor cleanup independently. The PTY session does not claim success merely because the shell exited: it calls `SubprocessTerminalHandle.terminate()` and awaits whole-session quiescence, propagating a cleanup failure that names survivors. A failed close is not cached forever: the registry and local session clear the fence only when it still names that failed attempt, so a later explicit or lifecycle close retries without disturbing a newer concurrent attempt. Service disposal still clears its backend, reservation, and owner-detacher registries when a close fails. ### Composition and rollout @@ -108,6 +108,7 @@ plugins: mode: workspace-write workspaceRoot: . '@deepseek-ai/dsh-pty': + '@deepseek-ai/dsh-subprocess-local': '@deepseek-ai/dsh-pty-local': config: scrollbackLines: 10000 @@ -141,11 +142,11 @@ The package ships concise tool guidance explaining persistent state, owner isola **Add persistent mode to `bash`.** Rejected. Returning on readiness rather than process exit, retaining a process tree across calls, and exposing interactive stdin create a different ownership and failure contract. -**Require native master-fd access from `node-pty`.** Rejected. Its public API exposes no master fd. The local backend instead derives foreground groups and descendants from supported OS process metadata and treats unreadable metadata as a detector miss. +**Require native master-fd access from `node-pty`.** Rejected. Its public API exposes no master fd. The local subprocess terminal adapter derives foreground groups and descendants from supported OS process metadata and treats unreadable metadata as a detector miss. **Signal every member of the root PID's POSIX session.** Rejected. `node-pty` may expose a helper PID whose session belongs to the launcher, so SID-wide teardown can signal unrelated harness or desktop processes. A PID-identity-fenced descendant tree is narrower and safe by construction. -**Publish `PtyIdleDetector` as a replaceable registry.** Rejected. Only the local backend needs these platform probes, while remote backends may receive readiness over their own protocol. Backend replacement already provides the necessary extension point. +**Publish `PtyIdleDetector` as a replaceable registry.** Rejected. Substrate-specific foreground facts come from the mounted terminal-process primitive, while prompt/silence readiness remains one private policy in `dsh-pty-local`. The filesystem/subprocess execution-world replacement is the necessary extension point. **Add a PTY-specific `sleep` tool.** Rejected. `ctx.tasks` already owns bounded waiting, cancellation, completion notices, and model-facing collection. A second general wake mechanism would cross the agent-loop boundary and duplicate that contract. @@ -156,8 +157,8 @@ The package ships concise tool guidance explaining persistent state, owner isola ## Verification - Per-file coverage pins owner fencing, concurrent reservations, unpublished-spawn cancellation and awaited teardown, sandbox-mode change rejection, retriable lifecycle cleanup, readiness tiers, rejection of pre-write stdin waits, the configured handoff grace holding the idle fallback past one poll and its rejection below `pollIntervalMs`, sanitizer carry state, complete UTF-8 bounds, task integration, schemas, and exact render intents. -- Linux process fixtures cover non-leader and non-main-thread stdin waits, zombie quiescence, unreadable process state, supported syscall tables, unsupported architectures, and false-positive rejection; macOS inspector logic is injected into the same unit suite. -- Real `node-pty` tests exercise shell state, shared sandbox policy, environment scrubbing, raw-mode foreground `SIGINT` after deliberately delayed child readiness under scenario-owned timing bounds, a TERM-ignoring descendant, and immediate post-disposal quiescence on supported hosts. +- Subprocess process fixtures cover non-leader and non-main-thread stdin waits, zombie quiescence, unreadable process state, supported syscall tables, unsupported architectures, and false-positive rejection; macOS inspector logic is injected into the same unit suite. +- Real `node-pty` and PTY-consumer tests jointly exercise shell state, shared sandbox policy, environment scrubbing, raw-mode foreground `SIGINT`, a TERM-ignoring descendant, and immediate post-disposal quiescence on supported hosts. - A Loader-driven `cordis.yml` test mounts the real three-package composition. ACP and headless snapshots pin the six schemas, bounded results, and errors through opt-in overlays; TUI snapshots pin terminal and generic card presentation. - Package contracts, the architecture map, core data structures, generated catalogs, and the website API describe the same shipped surface. - The repository CI-equivalent sequence owns type, lint, coverage, snapshot, documentation, build, hygiene, demo, and built-entry verification. @@ -172,10 +173,10 @@ The package ships concise tool guidance explaining persistent state, owner isola **Persistent state can drift from the model's belief.** The model may forget its cwd or active REPL. Session summaries and retained output help recovery, but no prompt can make state persistence deterministic. -**A daemonized descendant can leave the captured tree.** A process that reparents before teardown is no longer discoverable from the `node-pty` root. The implementation accepts that cleanup gap instead of risking SID-wide signals to unrelated processes. +**A daemonized descendant can leave the local provider's captured tree.** A process that reparents before teardown is no longer discoverable from the `node-pty` root. The local terminal primitive accepts that cleanup gap instead of risking SID-wide signals to unrelated processes. **A shell can cause external side effects.** Session sandboxing and environment scrubbing reduce local exposure but do not undo pushes, API calls, or messages. Deployments that cannot tolerate those effects must omit PTY or add network policy. **Process loss destroys terminal state.** In-process sessions do not survive a harness crash or restart, and raw scrollback is not durable. Important work must be committed to files or another durable system. -**`node-pty` is a native dependency.** Installation, supported Node versions, prebuild availability, and platform behavior require built-artifact smokes on every supported OS. +**`node-pty` is a native dependency of `dsh-subprocess-local`.** Installation, supported Node versions, prebuild availability, and platform behavior require built-artifact smokes on every supported OS. diff --git a/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.zh.md b/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.zh.md index 84ef8988ba..6ea92da9a1 100644 --- a/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.zh.md +++ b/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.zh.md @@ -14,7 +14,7 @@ harness 可以运行前台与后台命令、编辑文件和委派工作,但无 ## 决策 -可选的 `packages/pty/` 能力家族提供由 agent 拥有、持久化且面向行式交互的 PTY 会话。它遵循仓库的 [capability pattern](../../implemented/architecture/2026-06-13-capability-seams.md),与现有命令和文件系统工具并存,并且不修改 `agent-loop`。 +可选的 `packages/pty/` 功能家族提供由 agent 拥有、持久化且面向行式交互的 PTY 会话。它遵循仓库的 [capability pattern](../../implemented/architecture/2026-06-13-capability-seams.md),与现有命令和文件系统工具并存,并且不修改 `agent-loop`。 当前实现在 Linux 和 macOS 上支持交互式 shell 与行式 REPL。全屏终端应用、按键序列、BEL 触发的控制流、进程丢失后的会话恢复以及跨 agent 共享会话都明确推迟。 @@ -23,10 +23,10 @@ harness 可以运行前台与后台命令、编辑文件和委派工作,但无 | 包 | 角色 | ctx key | |---|---|---| | `dsh-pty` | `PtyService`、branded `PtySessionId`、后端注册表、按 owner 隔离的会话契约和结果类型 | `ctx.pty` | -| `dsh-pty-local` | 基于 [`node-pty`](https://github.com/microsoft/node-pty) 的本地后端、平台进程检查、有界终端缓冲、沙箱解析和进程树监管 | 在 `ctx.pty` 上注册后端 | +| `dsh-pty-local` | 基于 `ctx.subprocess.spawnTerminal()` 的持久 shell 后端:就绪状态、有界终端缓冲、沙箱解析和感知 owner 的会话生命周期 | 在 `ctx.pty` 上注册后端 | | `dsh-tool-pty` | 6 个面向模型的工具、后台发送的 task 运行时集成、使用指引和 UI 渲染意图 | 注册到 `ctx.tools` | -idle 检测属于后端行为,不是第二条公共 seam。远程或容器后端可能拥有完全不同于本地 `/proc` 检查的权威就绪信号;因此每个 `PtyBackend` 都返回统一的发送结果,同时在内部拥有自己的检测机制。 +就绪判定仍属于 PTY 后端行为,不是第二条公共 seam。终端进程提供方只提供基底事实,例如前台进程组,以及能否证明该组正在等待输入;`dsh-pty-local` 将这些事实与提示符和静默证据组合成统一的发送结果。 ### agent 所有权与身份 @@ -34,18 +34,18 @@ idle 检测属于后端行为,不是第二条公共 seam。远程或容器后 实现不提供插件加载期 auto-start 会话。`terminal_open` 只在 agent 工具调用期间创建会话,此时所有权和所属的事件溯源会话都已确定。未来的声明式启动功能必须通过尚未发布的 agent setup 组合,而不能创建全局共享终端。 -agent scope dispose(资源释放)时先关闭注册,再等待全部所属 PTY 完全停稳。未发布的后端 setup 同样是受追踪的生命周期操作:owner 或服务 dispose 会中止服务自有的 signal,等待后端结算与回滚完成后才返回。即使后端 reject,或返回的会话在回滚 close 时失败,调用方取消仍会原样保留其 `AbortSignal.reason`;该清理失败不会替换调用方原因,而会继续受追踪,留待后续 owner 或服务 dispose 处理。由 lifecycle dispose 触发的回滚 close 失败会使 spawn 与该 lifecycle dispose 都 reject,而 `PtyBackendCleanupError` 让后端在不替换调用方取消的前提下,为该 lifecycle dispose 保留自身的启动清理失败。若调用方取消已完成结算,而 dispose 尚未发生,该清理失败会继续作为受追踪的 owner activity 保留,直到后续 owner 或服务 dispose 消费并报告它,因此沙箱模式策略不会把清理失败误判为完全停稳。后端或工具插件 reload 不会遗留会话:所有权持续存放在 `PtyService` 中,直到 agent 结束,与 [`ctx.tasks`](../../../../packages/tasks/tasks/README.md) 的服务持有记录模式一致。服务会先同步把会话预留给一次活跃发送,再返回该操作;后台发送同样会在 task id 对外可见前完成预留。第二次发送会以 `SEND_ACTIVE` 失败,因此输出与取消无法跨越操作所有权。 +agent scope dispose 时先关闭注册,再等待全部所属 PTY 静默退出。未发布的后端 setup 同样是受追踪的生命周期操作:owner 或服务 dispose 会中止服务自有的 signal,等待后端结算与回滚完成后才返回。即使后端 reject,或返回的会话在回滚 close 时失败,调用方取消仍会原样保留其 `AbortSignal.reason`;该清理失败不会替换调用方原因,而会继续受追踪,留待后续 owner 或服务 dispose 处理。由 lifecycle dispose 触发的回滚 close 失败会使 spawn 与该 lifecycle dispose 都 reject,而 `PtyBackendCleanupError` 让后端在不替换调用方取消的前提下,为该 lifecycle dispose 保留自身的启动清理失败。若调用方取消先于 dispose 完成结算,该清理失败会继续作为受追踪的 owner activity 保留,直到后续 owner 或服务 dispose 消费并报告它,因此沙箱模式策略不会把清理失败误判为静默。后端或工具插件 reload 不会遗留会话:所有权持续存放在 `PtyService` 中,直到 agent 结束,与 [`ctx.tasks`](../../../../packages/tasks/tasks/README.md) 的服务持有记录模式一致。服务会先同步把会话预留给一次活跃发送,再返回该操作;后台发送同样会在 task id 对外可见前完成预留。第二次发送会以 `SEND_ACTIVE` 失败,因此输出与取消无法跨越操作所有权。 ### 安全与进程边界 注册的 `shell` 后端只约束终端如何启动,不约束启动后输入的命令。因此 `dsh-pty-local` 在 spawn 前应用两层保护: -- 它使用与 `bash-local` 相同的凭证形态名称策略构建清洗后的子进程环境,移除环境中的 `*KEY*`、`*PASSWORD*`、`*SECRET*`、`*TOKEN*` 和 harness 管理的变量,除非显式的可信映射提供这些值。 +- 它只提供终端专用的环境覆盖;挂载的子进程提供方先清除名称形似凭据的环境变量,再合并这些覆盖。 - 它要求 `ctx.sandbox` 和共享的 `ctx.sandboxPolicy`。后端在 spawn 时,以部署默认值为底折叠 owner 的有效 session mode,并只包装一次 shell argv;该 mode 与 workspace root 在 PTY 的整个生命周期中充当进程边界。只要 owner 有任何已打开的 PTY 或尚未发布的 spawn,任何会改变生效 `sandbox/mode` 的写入都会在提交前被拒绝,并提示先等待创建操作结算,再关闭这些会话;不会改变生效模式的写入仍然有效。这项进行中的预留从后端 setup 持续到发布完成,因此不存在降级后又出现权限更宽的终端这一竞态。`danger-full-access` 是现有的显式无约束选择,不另设 PTY 私有 bypass。 沙箱限制本地进程副作用,但不会让任意 shell 输入自动安全:网络调用和其他外部副作用仍由部署策略治理。工具描述会说明 PTY 会话比一次性工具更难审计,只应在确实需要持久状态或交互式 stdin 时使用。 -实现只使用 `node-pty` 的公共功能:子进程 PID、`data` 与 `exit` 通知、`write`、`resize` 和 `kill`。它不假设能访问原生 master fd,也不从 TypeScript 调用 `waitpid`。平台进程检查器在 Linux 上通过 `/proc`、在 macOS 上通过 `ps` 推导前台进程组和父子进程身份。 +本地子进程终端原语只使用 `node-pty` 的公共功能:子进程 PID、`data` 与 `exit` 通知、`write` 和 `kill`。它不假设能访问原生 master fd,也不从 TypeScript 调用 `waitpid`。该原语下的平台进程检查器在 Linux 上通过 `/proc`、在 macOS 上通过 `ps` 推导前台进程组和父子进程身份。[可移植执行环境决策](../architecture/2026-07-28-portable-execution-world-consumers.md)负责定义这种进程/消费方拆分。 ### 6 个面向模型的工具 @@ -55,7 +55,7 @@ agent scope dispose(资源释放)时先关闭注册,再等待全部所属 | `terminal_send` | 发送文本、可选提交 Enter,并等待就绪或注册一个后台任务 | 有界 viewport、等待状态和会话状态;后台模式还返回 `taskId` | | `terminal_read` | 从保留的 scrollback 读取一个有界页 | `{ text, totalLines, lineBegin, lineEnd, truncated }` | | `terminal_signal` | 向当前前台进程组发送一种允许的信号 | `{ delivered, targetPgid }` | -| `terminal_close` | 关闭一个会话并等待进程树完全停稳 | `{ killed }` | +| `terminal_close` | 关闭一个会话并等待进程树静默退出 | `{ killed }` | | `terminal_list` | 列出调用方的活会话 | 按 owner 隔离的会话摘要 | UI 渲染契约精确且不携带位置信息。`terminal_send` 只为前台发送使用 terminal 调用卡片和结果卡片;后台形式使用通用 `execute` 卡片。`terminal_open`、`terminal_read`、`terminal_signal`、`terminal_close` 和 `terminal_list` 分别使用通用 `execute`、`read`、`execute`、`delete` 和 `read` 卡片。所有 PTY 工具都不发出 `locations`。 @@ -66,7 +66,7 @@ UI 渲染契约精确且不携带位置信息。`terminal_send` 只为前台发 当 `run_in_background: true` 时,`dsh-tool-pty` 在 `ctx.tasks` 上注册进行中的发送,并立即返回 `taskId`。生产方把 `maxResultBytes` 写入 task 快照,使 `task_output`、kill 返回的终态状态和完成通知在加上通用元数据后,仍对完整结果执行同一上限。`task_output(wait: true)` 负责等待、读取增量输出并记录最终结果;`task_kill` 会解析当前前台 PGID 并发送真正的 `SIGINT`,即使应用已禁用终端 `ISIG` 也同样如此,且后续升级仍只通过 PTY 后端拥有的 teardown 路径进行。若 task 对外接口不存在,后台模式必须在写入输入前失败。设计不新增 PTY 专用的 `sleep` 工具或通用唤醒 seam。 -`terminal_read` 从最新保留行起,朝更早的行分页。后端同时对保留的 scrollback 和返回页载荷执行行数与 UTF-8 字节上限,因此单个超长行无法绕过后端上限;工具随后再限制包含分页与截断元数据的完整渲染页。`truncated` 用于区分保留数据丢失与普通 viewport 增量。 +`terminal_read` 从最新保留行向后分页。后端同时对保留的 scrollback 和返回页载荷执行行数与 UTF-8 字节上限,因此单个超长行无法绕过后端上限;工具随后再限制包含分页与截断元数据的完整渲染页。`truncated` 用于区分保留数据丢失与普通 viewport 增量。 `terminal_signal` 接受闭合集 `SIGINT | SIGTERM | SIGKILL | SIGTSTP | SIGHUP`。后端在执行时解析终端前台进程组。当目标组是顶层 shell 时拒绝 `SIGKILL`,并指引调用方使用 `terminal_close`;进程组解析失败时操作直接失败,而不是向猜测的 PID 发送信号。 @@ -76,7 +76,7 @@ UI 渲染契约精确且不携带位置信息。`terminal_send` 只为前台发 在 Linux 上,检查器从 `/proc//stat` 读取 shell 的终端前台 PGID,枚举该进程组中的每个进程与线程,并检查它们当前的 syscall。Tier 1 只有观察到 stdin 等待才返回正结果:直接 `read(0)`、获准读取且含 fd 0 的 `select`/`pselect6` 或 `poll`/`ppoll` 参数,或者含 fd 0 的 epoll interest list。终端输入前就已存在的等待并不代表写入后就绪:必须先观察到同一 PGID 脱离该等待,之后再次进入等待才能使该次 send 完成;前台 PGID 发生变化则构成新的证据。无法读取的进程内存和未识别的 syscall 都是 miss,绝不作为正向猜测。架构表只包含对应 Linux UAPI 定义的 syscall number;不支持的架构跳过 Tier 1。 -macOS 没有精确 syscall 层。任何前台进程组输出静默都会返回 `inferred_idle`,包括 Python 和 `gdb`;从 `ps` 推导的终端 PGID 只用于发送信号,不作为「只有 shell 才能 idle」的证明。纯进程检查逻辑可注入,并在 Linux 上进行单元测试,同时由 macOS CI job 驱动真实 PTY 和进程表路径。 +macOS 没有精确 syscall 层。任何前台进程组输出静默都会返回 `inferred_idle`,包括 Python 和 `gdb`;从 `ps` 推导的终端 PGID 只用于发送信号,不作为「只有 shell 才能 idle」的证明。纯进程检查逻辑可注入并在 Linux 上完成 unit 覆盖率,同时由 macOS CI job 驱动真实 PTY 和进程表路径。 Tier 2 在持续 `idleSilenceMs` 没有输出后返回 `inferred_idle`,因此 sleep 或网络阻塞的命令可能看似 ready。如果此前已经见过 prompt marker,Tier 2 会再等待 `handoffGraceMs`,使恰好落在静默边界上的 bash 前台交接仍然以精确的 `stdin_read` 归因结束,而不是退到较弱的推断;该宽限是由部署方拥有的配置字段,并被校验为至少覆盖一个 `pollIntervalMs`——短于轮询周期的宽限装不下一次就绪轮询,因此不可能改变任何结果。它只约束见过 marker 的 send,代价是这一种情况的交互返回延迟,而不是每一次 send。Tier 3 在 `timeoutMs` 后返回 `timeout`,避免前台工具调用无限占住 agent。结果保留这些区别;调用方可以通过 `ctx.tasks` 等待、向前台组发信号,或从另一个会话排查。 @@ -88,13 +88,13 @@ Tier 2 在持续 `idleSilenceMs` 没有输出后返回 `inferred_idle`,因此 现有持久化 `tool/call` 与 `tool/result` 事件是模型发送文本和返回给模型的渲染输出的真源。`terminal_open` 通过已记录的工具结果返回 MOTD;前台 `send`/`read`/`list`/`signal`/`close` 结果走同一路径记录。PTY 包不会把原始字节流重复写入自定义会话事件。 -后台发送复用现有后台任务完成通知和 `task_output` 结果路径,因此进入后续模型请求的任何输出同样持久化。原始终端字节只作为有界的进程内状态存在,既不持久化也不可恢复。未来的 opt-in transcript(文本记录)sink 必须拥有独立的保留、凭证和隐私契约。 +后台发送复用现有后台任务完成通知和 `task_output` 结果路径,因此进入后续模型请求的任何输出同样持久化。原始终端字节只作为有界的进程内状态存在,既不持久化也不可恢复。未来的 opt-in transcript sink 必须拥有独立的保留、凭证和隐私契约。 ### 进程树 teardown -顶层 `node-pty` 子进程是所有权锚点。关闭时,后端先停止 callback,再按父 PID 以子进程优先顺序捕获其传递子进程、发送 `SIGTERM` 并等待,然后重新扫描关停期间 fork 出的子进程,向剩余子孙进程树发送 `SIGKILL`,并在 shell 仍存活时验证每个非僵尸子孙进程都已离开进程表。身份匹配的 Linux 僵尸进程已无可执行工作,因此视为完全停稳;shell 关闭时会回收它或将其重新挂接给负责回收的父进程。完成这些步骤后,后端才用 shell 自身的 TERM、宽限等待、KILL 序列停止 shell。每个捕获的 PID 都包含进程启动身份,避免 PID 复用把升级信号发给无关进程。 +子进程终端句柄拥有顶层终端进程及其会话。关闭时,它按父 PID 以子进程优先顺序捕获传递后代、发送 `SIGTERM` 并等待,然后重新扫描关停期间 fork 出的子进程,向二者并集发送 `SIGKILL`,并在停止顶层进程前验证每个非僵尸后代都已离开进程表。身份匹配的 Linux 僵尸进程已无可执行工作,因此视为完全停稳。每个捕获的 PID 都包含进程启动身份,避免 PID 复用把升级信号发给无关进程。 -teardown 独立报告根进程退出与存活进程清理。它不会只因 shell 退出就声称成功;dispose 只有在已捕获的进程树中不再存在尚未完全停稳的成员后才完成,否则返回清理失败并列出存活者。失败的 close 不会永久缓存:注册表与本地会话各自仅在关闭围栏仍指向该次失败尝试时才将其清除,因此外部存活进程状态改变后,后续的显式 close 或生命周期 close 会重试,且不会干扰较新的并发尝试。即使某个 close 失败,服务 dispose 仍会清空其后端、预留与 owner detacher 注册表。所有权绝不会扩大到根 PID 所属 POSIX 会话的全部成员。 +teardown 独立报告顶层进程退出与存活进程清理。PTY 会话不会只因 shell 退出就声称成功:它会调用 `SubprocessTerminalHandle.terminate()` 并等待整个会话完全停稳,若清理失败则向外传播并列出存活者。失败的 close 不会永久缓存:注册表与本地会话各自仅在关闭围栏仍指向该次失败尝试时才将其清除,因此后续的显式 close 或生命周期 close 会重试,且不会干扰较新的并发尝试。即使某个 close 失败,服务 dispose 仍会清空其后端、预留与 owner detacher 注册表。 ### 组合与推行 @@ -108,6 +108,7 @@ plugins: mode: workspace-write workspaceRoot: . '@deepseek-ai/dsh-pty': + '@deepseek-ai/dsh-subprocess-local': '@deepseek-ai/dsh-pty-local': config: scrollbackLines: 10000 @@ -141,11 +142,11 @@ plugins: **给 `bash` 增加持久模式。**拒绝。按就绪而不是进程退出返回、跨调用保留进程树、暴露交互式 stdin 会形成不同的所有权和失败契约。 -**要求从 `node-pty` 获取原生 master fd。**拒绝。它的公共 API 不暴露 master fd。本地后端改为从受支持的 OS 进程元数据推导前台组与子孙进程,并把不可读元数据视为 detector miss。 +**要求从 `node-pty` 获取原生 master fd。**拒绝。它的公共 API 不暴露 master fd。本地子进程终端适配器改为从受支持的 OS 进程元数据推导前台组与子孙进程,并把不可读元数据视为 detector miss。 **向根 PID 所属 POSIX 会话的全部成员发送信号。**拒绝。`node-pty` 可能暴露属于启动器会话的 helper PID,因此按 SID 清理可能向无关的 harness 或桌面进程发送信号。带 PID 启动身份校验的子孙进程树范围更窄,其安全边界由结构保证。 -**发布可替换注册表 `PtyIdleDetector`。**拒绝。只有本地后端需要这些平台 probe,远程后端可能通过自己的协议接收就绪状态。替换后端已经提供所需扩展点。 +**发布可替换注册表 `PtyIdleDetector`。**拒绝。基底专用的前台事实来自挂载的终端进程原语,提示符/静默就绪判定则仍是 `dsh-pty-local` 内部的一项私有策略。替换文件系统/子进程执行环境就是所需扩展点。 **新增 PTY 专用 `sleep` 工具。**拒绝。`ctx.tasks` 已经拥有有界等待、取消、完成通知和面向模型的收集。第二套通用唤醒机制会跨越 agent loop(智能体循环)边界并重复该契约。 @@ -155,9 +156,9 @@ plugins: ## 验证 -- 逐文件测试覆盖并固定 owner 隔离、并发预留、未发布 spawn 的取消与等待式 teardown、沙箱模式变更拒绝、可重试的生命周期清理、就绪层级、对写入前 stdin 等待的拒绝、配置化交接宽限把 idle fallback 顶过一次轮询以及低于 `pollIntervalMs` 时的拒绝、sanitizer carry state、完整 UTF-8 结果上限、task 集成、schema 和精确 render intent。 -- Linux 进程 fixture(测试前置数据)覆盖非 leader 与非主线程的 stdin 等待、僵尸进程的完全停稳、不可读进程状态、受支持的 syscall 表、不支持的架构和误报拒绝;同一单元测试套件通过注入覆盖 macOS 检查器逻辑。 -- 真实 `node-pty` 测试在受支持宿主上覆盖 shell 状态、共享沙箱策略、环境清洗、在由场景掌控的时间界限内先有意延迟子进程就绪,再对 raw mode 前台进程发送 `SIGINT`、忽略 `SIGTERM` 的子进程,以及 dispose 返回后立即完全停稳。 +- 每文件覆盖率固定 owner 隔离、并发预留、未发布 spawn 的取消与等待式 teardown、沙箱模式变更拒绝、可重试的生命周期清理、就绪层级、对写入前 stdin 等待的拒绝、配置化交接宽限把 idle fallback 顶过一次轮询以及低于 `pollIntervalMs` 时的拒绝、sanitizer carry state、完整 UTF-8 结果上限、task 集成、schema 和精确 render intent。 +- 子进程 fixture 覆盖非 leader 与非主线程的 stdin 等待、僵尸进程完全停稳、不可读进程状态、受支持的 syscall 表、不支持的架构和误报拒绝;同一单元测试套件通过注入覆盖 macOS 检查器逻辑。 +- 真实 `node-pty` 与 PTY 消费方测试共同在受支持宿主上覆盖 shell 状态、共享沙箱策略、环境清洗、raw mode 前台 `SIGINT`、忽略 `SIGTERM` 的后代进程,以及 dispose 返回后立即完全停稳。 - Loader 驱动的 `cordis.yml` 测试挂载真实三包组合。ACP 与 headless 快照通过 opt-in overlay 固定 6 个 schema、有界结果和错误;TUI 快照固定 terminal 与 generic 卡片展示。 - 包契约、架构图、核心数据结构、生成目录和 website API 描述同一个已发布接口。 - 仓库 CI 等价序列负责类型、lint、覆盖率、快照、文档、构建、hygiene、demo 和 built-entry 验证。 @@ -172,10 +173,10 @@ plugins: **持久状态可能偏离模型认知。**模型可能忘记 cwd 或活跃 REPL。会话摘要和保留输出有助恢复,但任何 prompt 都无法让状态持久化变成确定行为。 -**daemonized 子进程可能离开捕获树。**在 teardown 前 reparent 的进程无法再从 `node-pty` 根进程发现。实现接受这个清理缺口,不冒险按 SID 向无关进程发送信号。 +**daemonized 后代进程可能离开本地提供方捕获的进程树。**在 teardown 前 reparent 的进程无法再从 `node-pty` 根进程发现。本地终端原语接受这个清理缺口,不冒险按 SID 向无关进程发送信号。 **Shell 可以造成外部副作用。**会话沙箱和环境清洗降低本地暴露,但无法撤销 push、API 调用或消息发送。无法容忍这些副作用的部署必须省略 PTY 或增加网络策略。 **进程丢失会销毁终端状态。**进程内会话无法跨 harness crash 或 restart 存活,原始 scrollback 也不持久化。重要工作必须提交到文件或其他持久系统。 -**`node-pty` 是原生依赖。**安装、支持的 Node 版本、prebuild 可用性和平台行为都需要在每个支持 OS 上运行构建产物冒烟测试。 +**`node-pty` 是 `dsh-subprocess-local` 的原生依赖。**安装、支持的 Node 版本、prebuild 可用性和平台行为都需要在每个支持 OS 上运行 built-artifact smoke。 diff --git a/docs/config-catalog.md b/docs/config-catalog.md index 1f81974b21..c07e81089c 100644 --- a/docs/config-catalog.md +++ b/docs/config-catalog.md @@ -284,6 +284,30 @@ export interface Config { Source: [`packages/client/hmr/src/index.ts:29`](../packages/client/hmr/src/index.ts) +## `@deepseek-ai/dsh-code-runtime-subprocess` + +Requires: `fs` · `subprocess` + +```ts config-catalog +/** Runtime configuration; every execution and bridge bound is deployment-tunable. */ +export interface Config { + /** Worker measured event-loop busy-time budget. */ + computeMs?: number + /** Host-observed wall-clock ceiling. */ + maxWallMs?: number + /** Combined serialized outer logs/value/diagnostic cap. */ + maxOutputBytes?: number + /** Worker old-generation heap cap in MiB. */ + maxOldGenerationSizeMb?: number + /** Largest decoded bridge frame, including binding traffic. */ + maxFrameBytes?: number + /** Process-tree TERM-to-KILL grace. */ + killGraceMs?: number +} +``` + +Source: [`packages/code-runtime/code-runtime-subprocess/src/index.ts:28`](../packages/code-runtime/code-runtime-subprocess/src/index.ts) + ## `@deepseek-ai/dsh-code-runtime-worker` ```ts config-catalog @@ -317,7 +341,7 @@ export interface Config { } ``` -Source: [`packages/code-runtime/code-runtime-worker/src/index.ts:25`](../packages/code-runtime/code-runtime-worker/src/index.ts) +Source: [`packages/code-runtime/code-runtime-worker/src/index.ts:30`](../packages/code-runtime/code-runtime-worker/src/index.ts) ## `@deepseek-ai/dsh-compact-basic` @@ -423,7 +447,7 @@ export interface Config { } ``` -Source: [`packages/fs/fs-local/src/index.ts:38`](../packages/fs/fs-local/src/index.ts) +Source: [`packages/fs/fs-local/src/index.ts:40`](../packages/fs/fs-local/src/index.ts) ## `@deepseek-ai/dsh-fs-sandbox` @@ -906,7 +930,7 @@ Source: [`packages/llm/llm-retry/src/index.ts:46`](../packages/llm/llm-retry/src ## `@deepseek-ai/dsh-lsp-local` -Requires: `lsp` · `subprocess` +Requires: `fs` · `lsp` · `subprocess` ```ts config-catalog /** Plugin configuration: provider id → local language-server configuration. */ @@ -942,7 +966,7 @@ export interface LspLocalServerConfig { } ``` -Source: [`packages/lsp/lsp-local/src/index.ts:87`](../packages/lsp/lsp-local/src/index.ts) +Source: [`packages/lsp/lsp-local/src/index.ts:82`](../packages/lsp/lsp-local/src/index.ts) ## `@deepseek-ai/dsh-mcp-client` @@ -1048,7 +1072,7 @@ Source: [`packages/plan/plan-mode/src/index.ts:69`](../packages/plan/plan-mode/s ## `@deepseek-ai/dsh-pty-local` -Requires: `pty` · `sandbox` · `sandboxPolicy` +Requires: `pty` · `sandbox` · `sandboxPolicy` · `subprocess` ```ts config-catalog /** Public plugin configuration. */ diff --git a/docs/cordis-catalog/events.md b/docs/cordis-catalog/events.md index edfd046430..7015d39bc8 100644 --- a/docs/cordis-catalog/events.md +++ b/docs/cordis-catalog/events.md @@ -410,7 +410,7 @@ Single-slot decision for the next FileSystem.editText. Calling `next()` yields a Types: [FsTarget](../core-data-structures/filesystem.md) · [FsVersion](../core-data-structures/filesystem.md) -Source: [`packages/fs/fs/src/index.ts:62`](../../packages/fs/fs/src/index.ts) +Source: [`packages/fs/fs/src/index.ts:64`](../../packages/fs/fs/src/index.ts) ### `fs/observed` — emit @@ -430,7 +430,7 @@ Record a successful observation. Listeners must be synchronous recorders: throws Types: [FsTarget](../core-data-structures/filesystem.md) · [FsVersion](../core-data-structures/filesystem.md) -Source: [`packages/fs/fs/src/index.ts:71`](../../packages/fs/fs/src/index.ts) +Source: [`packages/fs/fs/src/index.ts:73`](../../packages/fs/fs/src/index.ts) ### `fs/write-intent` — waterfall @@ -450,7 +450,7 @@ Single-slot decision for the next FileSystem.writeText. Calling `next()` yields Types: [FsTarget](../core-data-structures/filesystem.md) · [FsWriteIntent](../core-data-structures/filesystem.md) -Source: [`packages/fs/fs/src/index.ts:54`](../../packages/fs/fs/src/index.ts) +Source: [`packages/fs/fs/src/index.ts:56`](../../packages/fs/fs/src/index.ts) ## `goal/*` diff --git a/docs/core-data-structures/filesystem.i18n.yaml b/docs/core-data-structures/filesystem.i18n.yaml index 91ac0ed81d..129c5569b9 100644 --- a/docs/core-data-structures/filesystem.i18n.yaml +++ b/docs/core-data-structures/filesystem.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write docs/core-data-structures/filesystem.md -filesystem.md: 110c1fd428b15c5094f9dcc94050cad61c324373 -filesystem.zh.md: 010ec22a5d4a29555425c3deede08b317cba7004 +filesystem.md: 50b936bdef7535769f19d6cc0a75eb11eea9a869 +filesystem.zh.md: 61a885203b0718b34b0fd936c318b03c65ce4770 diff --git a/docs/core-data-structures/filesystem.md b/docs/core-data-structures/filesystem.md index 110c1fd428..50b936bdef 100644 --- a/docs/core-data-structures/filesystem.md +++ b/docs/core-data-structures/filesystem.md @@ -12,6 +12,8 @@ Provider source: [`packages/fs/fs/src/types.ts`](../../packages/fs/fs/src/types. Every operation resolves a user-supplied path to an opaque backend target first. Consumers may display `displayPath`, but must not parse `targetKey` (a branded opaque id) or assume it is a local absolute path. +Consumers that share the filesystem's execution world obtain cross-capability coordinates through the provider instead of interpreting that identity: `processPath(target)` returns the canonical absolute path a subprocess can open, `fileUrl(target)` returns its provider-platform `file:` URI, and `contains(parent, child)` tests canonical identity or descendant containment. + ```ts type-equiv /** * A path resolved by a backend into a stable identity. `resolve()` produces @@ -50,7 +52,7 @@ type FsTargetKey = Branded<'FsTargetKey'> type FsVersion = Branded<'FsVersion'> ``` -`stat` returns metadata (never content), or `undefined` when the target is absent. `type` lets the tool reject directories/special files before reading, and `size` lets it choose `readText` vs `streamText` without probing by failure. +`stat` returns metadata (never content), or `undefined` when the target is absent. `type` lets the tool reject directories/special files before reading, and `size` lets it choose `readText` vs `streamText` without probing by failure. Protocol consumers use `readTextBounded(target, maxBytes)` when size validation and the complete UTF-8 read must remain one backend-owned stable operation; composing `stat` with `readText` would admit growth and replacement races. ```ts type-equiv /** @@ -256,4 +258,4 @@ type FsErrorCode = ## The service and the plugin -`FileSystem` (`ctx.fs`, abstract) owns the provider primitives: `resolve`, `stat`, `lstat`, `readText`, `streamText`, `listDir`, `writeText`, and `editText`. `dsh-fs-policy` registers **no service** — it is a plugin that adds policy through the `fs/*` event gate: it decides the write/edit intent waterfalls (supplying `createIfAbsent`/`replaceIfVersion`/`{ version }` or throwing `FS_NOT_OBSERVED`) and records on `fs/observed`. The executor is `dsh-tool-fs`: it reads/writes/edits through `ctx.fs`, dispatches the waterfalls, and emits the recording event. The generated wiring catalog shows the exact `ctx.fs` signatures on [services.md](../cordis-catalog/services.md#ctxfs--filesystem-abstract-seam). +`FileSystem` (`ctx.fs`, abstract) owns the provider primitives: `resolve`, `processPath`, `fileUrl`, `contains`, `stat`, `lstat`, `readText`, `readTextBounded`, `streamText`, `listDir`, `writeText`, and `editText`. `dsh-fs-policy` registers **no service** — it is a plugin that adds policy through the `fs/*` event gate: it decides the write/edit intent waterfalls (supplying `createIfAbsent`/`replaceIfVersion`/`{ version }` or throwing `FS_NOT_OBSERVED`) and records on `fs/observed`. The executor is `dsh-tool-fs`: it reads/writes/edits through `ctx.fs`, dispatches the waterfalls, and emits the recording event. The generated wiring catalog shows the exact `ctx.fs` signatures on [services.md](../cordis-catalog/services.md#ctxfs--filesystem-abstract-seam). diff --git a/docs/core-data-structures/filesystem.zh.md b/docs/core-data-structures/filesystem.zh.md index 010ec22a5d..6b7642b3f7 100644 --- a/docs/core-data-structures/filesystem.zh.md +++ b/docs/core-data-structures/filesystem.zh.md @@ -12,6 +12,8 @@ 每个操作首先将用户提供的路径解析为不透明的后端目标。消费方可以显示 `displayPath`,但禁止解析 `targetKey`(一个品牌化的不透明 id),也不得假设它是本地绝对路径。 +与文件系统共享执行世界的消费方通过提供方获取跨能力坐标,而不是解释该身份:`processPath(target)` 返回子进程可以打开的规范化绝对路径,`fileUrl(target)` 返回采用提供方平台语法的 `file:` URI,`contains(parent, child)` 则检查规范化身份相等或后代包含关系。 + ```ts type-equiv /** * A path resolved by a backend into a stable identity. `resolve()` produces @@ -50,7 +52,7 @@ type FsTargetKey = Branded<'FsTargetKey'> type FsVersion = Branded<'FsVersion'> ``` -`stat` 返回元数据(从不返回内容),目标不存在时返回 `undefined`。`type` 让工具在读取前拒绝目录或特殊文件;`size` 让工具无需通过失败探测即可选择 `readText` 还是 `streamText`。 +`stat` 返回元数据(从不返回内容),目标不存在时返回 `undefined`。`type` 让工具在读取前拒绝目录或特殊文件;`size` 让工具无需通过失败探测即可选择 `readText` 还是 `streamText`。当大小校验与完整 UTF-8 读取必须保持为一个由后端负责的稳定操作时,协议消费方使用 `readTextBounded(target, maxBytes)`;组合 `stat` 与 `readText` 会容许文件增长与替换竞态。 ```ts type-equiv /** @@ -256,4 +258,4 @@ type FsErrorCode = ## 服务与插件 -`FileSystem`(`ctx.fs`,abstract)拥有提供方原语:`resolve`、`stat`、`lstat`、`readText`、`streamText`、`listDir`、`writeText` 与 `editText`。`dsh-fs-policy` **不注册服务**——它是一个通过 `fs/*` 事件门禁添加策略的插件:对写入/编辑意图 waterfall 作出决策(提供 `createIfAbsent`/`replaceIfVersion`/`{ version }`,或抛出 `FS_NOT_OBSERVED`),并在 `fs/observed` 上记录。执行器是 `dsh-tool-fs`:它通过 `ctx.fs` 读取/写入/编辑,分发 waterfall,并 emit 记录事件。生成的 wiring 目录在 [services.md](../cordis-catalog/services.md#ctxfs--filesystem-abstract-seam) 中展示确切的 `ctx.fs` 签名。 +`FileSystem`(`ctx.fs`,abstract)拥有提供方原语:`resolve`、`processPath`、`fileUrl`、`contains`、`stat`、`lstat`、`readText`、`readTextBounded`、`streamText`、`listDir`、`writeText` 与 `editText`。`dsh-fs-policy` **不注册服务**——它是一个通过 `fs/*` 事件门禁添加策略的插件:对写入/编辑意图 waterfall 作出决策(提供 `createIfAbsent`/`replaceIfVersion`/`{ version }`,或抛出 `FS_NOT_OBSERVED`),并在 `fs/observed` 上记录。执行器是 `dsh-tool-fs`:它通过 `ctx.fs` 读取/写入/编辑,分发 waterfall,并 emit 记录事件。生成的 wiring 目录在 [services.md](../cordis-catalog/services.md#ctxfs--filesystem-abstract-seam) 中展示确切的 `ctx.fs` 签名。 diff --git a/docs/core-data-structures/subprocess.i18n.yaml b/docs/core-data-structures/subprocess.i18n.yaml index 5d59f6fd16..fca7947aae 100644 --- a/docs/core-data-structures/subprocess.i18n.yaml +++ b/docs/core-data-structures/subprocess.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write docs/core-data-structures/subprocess.md -subprocess.md: 8189795c5acf2fc1882d9e5bc1c006f49c327e2f -subprocess.zh.md: 370a1a05807eefbe09a896ea8bcca1e4ab0a6b83 +subprocess.md: eae497530ff54a7a259c23a2d0f665f3313488f5 +subprocess.zh.md: c3504ef7cd42448a15341f7f92d0f309a6216de1 diff --git a/docs/core-data-structures/subprocess.md b/docs/core-data-structures/subprocess.md index 8189795c5a..c866276b8e 100644 --- a/docs/core-data-structures/subprocess.md +++ b/docs/core-data-structures/subprocess.md @@ -2,9 +2,13 @@ English | [中文](subprocess.zh.md) -The subprocess seam is split across interface ([dsh-subprocess](../../packages/subprocess/subprocess), `ctx.subprocess`) and implementation ([dsh-subprocess-local](../../packages/subprocess/subprocess-local)); its consumers are other capability seams and out-of-process backends — the [bash executor family](bash.md) (collect-mode batch output), the LSP host (piped protocol streams + a collected stderr tail), and the ACP subagent backend (piped protocol streams + inherited stderr). This seam owns the managed `DSH_*` environment namespace, the shared credential scrub (`scrubbedParentEnv`), and the `CollectedOutput` shape; [dsh-bash](../../packages/bash/bash) re-exports the vocabulary so bash consumers keep one import root. +The subprocess seam is split across interface ([dsh-subprocess](../../packages/subprocess/subprocess), `ctx.subprocess`) and implementation ([dsh-subprocess-local](../../packages/subprocess/subprocess-local)); its consumers are other capability seams and out-of-process backends: the [bash executor family](bash.md) uses collected batch output, the LSP and Code Runtime hosts use raw protocol pipes, the PTY backend uses the terminal primitive, and the ACP subagent backend uses piped ndjson plus inherited stderr. This seam owns the managed `DSH_*` environment namespace, the shared credential scrub (`scrubbedParentEnv`), and the `CollectedOutput` shape; [dsh-bash](../../packages/bash/bash) re-exports the vocabulary so bash consumers keep one import root. -Source: [`packages/subprocess/subprocess/src/types.ts`](../../packages/subprocess/subprocess/src/types.ts) +Source: [`packages/subprocess/subprocess/src/types.ts`](../../packages/subprocess/subprocess/src/types.ts) and [`packages/subprocess/subprocess/src/index.ts`](../../packages/subprocess/subprocess/src/index.ts) + +## Execution-world coordinates + +One provider's `cwd`, `runtimeRoot`, executable paths, ordinary processes, and terminal sessions inhabit the same path and process namespace as the mounted filesystem provider. `resolveExecutable(command, env?, signal?)` verifies absolute executable paths or resolves bare names through the provider's scrubbed `PATH` plus deliberate overrides. Consumers use `runtimeRoot` for private materialized helpers and never assume a host path exists in that world. ## Managed environment namespace and captured output @@ -234,6 +238,12 @@ interface SubprocessOutcome { } ``` +## Terminal-process primitive + +`spawnTerminal(spec)` is the non-pipe process primitive. The provider allocates the controlling terminal and owns raw UTF-8 byte transport, foreground-process-group inspection and signalling, TERM-to-KILL cleanup, and whole-session quiescence. The PTY backend remains responsible for prompt detection, readiness inference, scrollback, sandbox policy, and persistent-session ownership; ordinary `spawn()` cannot reconstruct controlling-terminal semantics. + +The terminal spec fully specifies argv, cwd, environment overrides, dimensions, cleanup grace, and optional cancellation. Its handle exposes `pid`, ordered output, `done`, `write`, `inspectForeground`, `signalForeground`, `terminate`, and `waitForExit`; the exact public shapes are generated into the [`ctx.subprocess` service catalog](../cordis-catalog/services.md#ctxsubprocess--subprocessservice-abstract-seam). + ## Service behavior -The abstract [`SubprocessService`](../../packages/subprocess/subprocess/src/index.ts) seam defines `spawn` only; [`LocalSubprocessService`](../../packages/subprocess/subprocess-local/src/index.ts) is the local implementation (detached trees, per-disposition wiring, credential scrub, terminate-and-join disposal). See [`dsh-subprocess`](../../packages/subprocess/subprocess/README.md) for the seam contract and [`dsh-subprocess-local`](../../packages/subprocess/subprocess-local/README.md) for the mechanics. +The abstract [`SubprocessService`](../../packages/subprocess/subprocess/src/index.ts) seam defines execution-world coordinates, executable lookup, ordinary `spawn`, and `spawnTerminal`. [`LocalSubprocessService`](../../packages/subprocess/subprocess-local/src/index.ts) implements them with temporary runtime storage, detached process trees, per-disposition wiring, credential scrubbing, `node-pty`, platform process inspection, and terminate-and-join disposal. See [`dsh-subprocess`](../../packages/subprocess/subprocess/README.md) for the interface contract and [`dsh-subprocess-local`](../../packages/subprocess/subprocess-local/README.md) for local mechanics. diff --git a/docs/core-data-structures/subprocess.zh.md b/docs/core-data-structures/subprocess.zh.md index 370a1a0580..c3504ef7cd 100644 --- a/docs/core-data-structures/subprocess.zh.md +++ b/docs/core-data-structures/subprocess.zh.md @@ -1,14 +1,18 @@ -# 子进程 +# 进程管理器 [English](subprocess.md) | 中文 -子进程 seam 分为接口([dsh-subprocess](../../packages/subprocess/subprocess),`ctx.subprocess`)与实现([dsh-subprocess-local](../../packages/subprocess/subprocess-local));它的消费方是其他能力 seam 与进程外后端:[bash 执行器家族](bash.md)使用收集模式(collect)的批量输出,LSP 主机使用管道化的协议流 + 收集的 stderr 尾部,ACP(Agent Client Protocol)subagent 后端则使用管道化的协议流 + inherit 的 stderr。该 seam 拥有受管的 `DSH_*` 环境命名空间、共享的凭据清除(`scrubbedParentEnv`)与 `CollectedOutput` 形状;[dsh-bash](../../packages/bash/bash) 重导出这套词汇,使 bash 消费方保持单一导入入口。 +进程管理器 seam 分为接口([dsh-subprocess](../../packages/subprocess/subprocess),`ctx.subprocess`)与实现([dsh-subprocess-local](../../packages/subprocess/subprocess-local));它的消费方是其他能力 seam 与进程外后端:[bash 执行器家族](bash.md)使用收集模式的批量输出,LSP 与 Code Runtime 主机使用原始协议管道,PTY 后端使用终端原语,ACP(Agent Client Protocol)subagent 后端则使用管道化 ndjson 加 inherit 的 stderr。该 seam 拥有受管的 `DSH_*` 环境命名空间、共享的凭据清除(`scrubbedParentEnv`)与 `CollectedOutput` 形状;[dsh-bash](../../packages/bash/bash) 重导出这套词汇,使 bash 消费方保持单一导入入口。 -源码:[`packages/subprocess/subprocess/src/types.ts`](../../packages/subprocess/subprocess/src/types.ts) +源码:[`packages/subprocess/subprocess/src/types.ts`](../../packages/subprocess/subprocess/src/types.ts) 与 [`packages/subprocess/subprocess/src/index.ts`](../../packages/subprocess/subprocess/src/index.ts) + +## 执行世界坐标 + +一个提供方的 `cwd`、`runtimeRoot`、可执行文件路径、普通进程与终端会话,和挂载的文件系统提供方处于同一路径与进程命名空间。`resolveExecutable(command, env?, signal?)` 验证绝对可执行文件路径,或通过提供方清理后的 `PATH` 加有意覆盖来解析裸名称。消费方使用 `runtimeRoot` 存放私有物化辅助程序,绝不假设该执行世界中存在某条宿主路径。 ## 受管环境命名空间与捕获的输出 -`DSH_*` 变量是归 Harness 所有的子进程事实;实现会在合并调用方显式 `env` 之前丢弃环境中已有的 `DSH_*` 名称,因此当前事实只会以有意提供的字符串条目形式到达,而显式的 `undefined` tombstone 会删除普通环境中已有的值。每条被收集的流都通过 `CollectedOutput` 报告自身的截断与 spill 恢复状态。 +`DSH_*` 变量是归 Harness 所有的子进程事实;实现会在合并调用方显式 `env` 之前丢弃环境中已有的 `DSH_*` 名称,因此当前事实只会以有意提供的条目形式到达,每条被收集的流都通过 `CollectedOutput` 报告自身的截断与 spill 恢复状态。 ```ts type-equiv /** One environment key inside the managed {@link DSH_ENV_PREFIX} namespace. */ @@ -32,7 +36,7 @@ interface CollectedOutput { } ``` -## Node 风格的 stdio 处置方式(disposition) +## Node 形状的 stdio 处置方式(disposition) 每条流的处置方式都显式给出,由各消费方自行选择:原始管道用于协议分帧(LSP JSON-RPC、ACP ndjson),inherit 用于直通的诊断输出,收集模式用于有界的批量输出;其中 spill 文件是可选的,因此诊断尾部(语言服务器的 stderr)可以只在内存中缓冲,不留下任何文件。 @@ -84,7 +88,7 @@ interface SubprocessStdio { ## 完全显式的 spawn spec -该 seam 不应用任何默认值:每项处置方式、限制与目录都在 spec 上显式给出,因此由调用方自己的配置决定它们,而不是由某个隐藏的子进程服务默认值决定。`argv` 绝不经过 shell 解释。 +该 seam 不应用任何默认值:每项处置方式、限制与目录都在 spec 上显式给出,因此由调用方自己的配置决定它们,而不是由某个隐藏的进程管理器默认值决定。`argv` 绝不经过 shell 解释。 ```ts type-equiv /** @@ -101,11 +105,10 @@ interface SubprocessSpawnSpec { /** Per-stream stdio dispositions. */ stdio: SubprocessStdio /** - * Positive finite grace period in milliseconds, no greater than - * `MAX_TIMER_DELAY_MS`, for the {@link SubprocessHandle.terminate} escalation - * and for draining still-open collected pipes after the process exits (an - * inherited descriptor held by a surviving descendant cannot hold the - * outcome open indefinitely). + * Grace period in milliseconds for the {@link SubprocessHandle.terminate} + * escalation and for draining still-open collected pipes after the process + * exits (an inherited descriptor held by a surviving descendant cannot hold + * the outcome open indefinitely). */ graceMs: number /** @@ -116,18 +119,19 @@ interface SubprocessSpawnSpec { signal?: AbortSignal | undefined /** * Explicit environment entries merged onto the implementation's scrubbed - * parent base (see `scrubbedParentEnv`), with no namespace validation. A - * string is a deliberate caller opt-in, so a forwarded credential-shaped - * entry or current `DSH_*` fact survives the scrub; `undefined` is a - * tombstone that removes an ordinary ambient entry from the child. + * parent base (see `scrubbedParentEnv`), with no namespace validation: + * every entry is a deliberate caller opt-in, so a forwarded + * credential-shaped entry or a current `DSH_*` fact survives precisely + * because this layer merges after the scrub that drops its ambient + * namesake. */ - env?: NodeJS.ProcessEnv | undefined + env?: Record | undefined } ``` ## 句柄:流、读取器与以进程树为范围的终止 -spawn 会立即返回一个活动句柄。收集模式的读取器接受全流字节偏移量且从不消费,因此独立的读取器不会抢走彼此的增量;管道化的流归调用方所有。终止在每个平台上都以进程树为范围:`terminate()`(唯一的终止动词)执行 SIGTERM→宽限期→SIGKILL 升级,`waitForExit()` 观察整棵进程树。这足以让消费方构建自己的拆卸阶梯;ACP 后端的 `disposeAcpChild` 以 stdin EOF 开始,即为仓库内模板。 +spawn 会立即返回一个实时句柄。收集模式的读取器接受全流字节偏移量且从不消费,因此独立的读取器不会抢走彼此的增量;管道化的流归调用方所有。终止在每个平台上都以进程树为范围:`terminate()`(唯一的终止动词)执行 SIGTERM→宽限期→SIGKILL 升级,`waitForExit()` 观察整棵进程树——这足以让消费方构建自己的拆卸阶梯(ACP 后端以 stdin EOF 打头的 `disposeAcpChild` 即是模板)。 ```ts type-equiv /** @@ -234,6 +238,12 @@ interface SubprocessOutcome { } ``` +## 终端进程原语 + +`spawnTerminal(spec)` 是非管道进程原语。提供方分配控制终端,并负责原始 UTF-8 字节传输、前台进程组检查与信号发送、TERM→KILL 清理,以及整个会话的完全停稳。PTY 后端仍负责提示符检测、就绪推断、scrollback、沙箱策略和持久会话所有权;普通 `spawn()` 无法重建控制终端语义。 + +终端 spec 完全指定 argv、cwd、环境覆盖、尺寸、清理宽限期与可选取消。其句柄公开 `pid`、有序输出、`done`、`write`、`inspectForeground`、`signalForeground`、`terminate` 和 `waitForExit`;确切的公共形状生成到 [`ctx.subprocess` 服务目录](../cordis-catalog/services.md#ctxsubprocess--subprocessservice-abstract-seam)中。 + ## 服务行为 -抽象的 [`SubprocessService`](../../packages/subprocess/subprocess/src/index.ts) seam 只定义 `spawn`;[`LocalSubprocessService`](../../packages/subprocess/subprocess-local/src/index.ts) 是本地实现(detached 进程树、按处置方式接线的流、凭据清除、先终止再等待退出的 dispose(资源释放))。seam 契约见 [`dsh-subprocess`](../../packages/subprocess/subprocess/README.md),具体机制见 [`dsh-subprocess-local`](../../packages/subprocess/subprocess-local/README.md)。 +抽象的 [`SubprocessService`](../../packages/subprocess/subprocess/src/index.ts) seam 定义执行世界坐标、可执行文件查找、普通 `spawn` 与 `spawnTerminal`。[`LocalSubprocessService`](../../packages/subprocess/subprocess-local/src/index.ts) 以临时运行时存储、detached 进程树、按处置方式接线、凭据清除、`node-pty`、平台进程检查,以及先终止再等待退出的资源释放实现这些能力。接口契约见 [`dsh-subprocess`](../../packages/subprocess/subprocess/README.md),本地机制见 [`dsh-subprocess-local`](../../packages/subprocess/subprocess-local/README.md)。 diff --git a/knip.json b/knip.json index ede8b2c738..20bcb5ff2d 100644 --- a/knip.json +++ b/knip.json @@ -383,6 +383,16 @@ "tests/**/*.ts" ] }, + "packages/code-runtime/code-runtime-subprocess": { + "entry": [ + "tests/**/*.spec.ts", + "tests/**/*.e2e.ts" + ], + "project": [ + "src/**/*.ts", + "tests/**/*.ts" + ] + }, "packages/llm/llm-deepseek": { "entry": [ "tests/**/*.spec.ts", diff --git a/packages/README.i18n.yaml b/packages/README.i18n.yaml index 8aa9b92b91..c5be7e67d6 100644 --- a/packages/README.i18n.yaml +++ b/packages/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/README.md -README.md: 229feae568ba6e40a9c633696097eff46fd5bc95 -README.zh.md: b84aef020a7e3edf305df709d399fbc7b093b6a3 +README.md: 7fcd98bd19ca3291f0472af57841f2f763bb346f +README.zh.md: 7c4aed9b343ec57001883e094ea3dd0d73e2a920 diff --git a/packages/README.md b/packages/README.md index 229feae568..a711e4d303 100644 --- a/packages/README.md +++ b/packages/README.md @@ -19,7 +19,7 @@ Packages live at `packages///`; groups are containers, while names r | [`subprocess/`](subprocess/README.md) | Subprocess capability family: spawn seam + local process-tree implementation | Product — stable surface | | [`bash/`](bash/README.md) | Bash capability family: executor seam, local impl, model-facing tool | Product — stable surface | | [`pty/`](pty/README.md) | Persistent PTY capability family: owner-scoped sessions, local implementation, and model-facing tools | Product — stable surface | -| [`code-runtime/`](code-runtime/README.md) | Code-execution capability family: the runtime seam for model-written programs + a worker-thread backend | Product — stable surface | +| [`code-runtime/`](code-runtime/README.md) | Code-execution capability family: runtime seam plus local worker and filesystem/subprocess backends | Product — stable surface | | [`sandbox/`](sandbox/README.md) | Process-confinement seam; bwrap/Landlock/Seatbelt backends | Product — stable surface | | [`fs/`](fs/README.md) | Filesystem capability family: seam, local impl, model-facing file tools, bash-backed discovery tools | Product — stable surface | | [`lsp/`](lsp/README.md) | LSP capability family: seam, generic stdio provider, and the `lsp` tool | Product — stable surface | diff --git a/packages/README.zh.md b/packages/README.zh.md index b84aef020a..41380317f8 100644 --- a/packages/README.zh.md +++ b/packages/README.zh.md @@ -19,7 +19,7 @@ | [`subprocess/`](subprocess/README.md) | 进程管理能力系列:spawn seam + 本地进程树实现 | 产品:稳定表面 | | [`bash/`](bash/README.md) | Bash 能力系列:执行器 seam、本地实现、面向模型的工具 | 产品:稳定表面 | | [`pty/`](pty/README.md) | 持久 PTY 能力系列:按所有者隔离的会话、本地实现和面向模型的工具 | 产品:稳定表面 | -| [`code-runtime/`](code-runtime/README.md) | 代码执行能力系列:面向模型所写程序的运行时 seam + worker 线程后端 | 产品:稳定表面 | +| [`code-runtime/`](code-runtime/README.md) | 代码执行能力系列:运行时 seam、本地 worker 后端及文件系统/进程管理后端 | 产品:稳定表面 | | [`sandbox/`](sandbox/README.md) | 进程限制 seam;bwrap/Landlock/Seatbelt 后端 | 产品:稳定表面 | | [`fs/`](fs/README.md) | 文件系统能力系列:seam、本地实现、面向模型的文件工具、bash 后端发现工具 | 产品:稳定表面 | | [`lsp/`](lsp/README.md) | LSP 能力系列:seam、通用 stdio 提供方和 `lsp` 工具 | 产品:稳定表面 | diff --git a/packages/code-runtime/code-runtime-worker/README.md b/packages/code-runtime/code-runtime-worker/README.md index 590b79dcd1..3de3bc622e 100644 --- a/packages/code-runtime/code-runtime-worker/README.md +++ b/packages/code-runtime/code-runtime-worker/README.md @@ -34,7 +34,7 @@ Every field is validated and defaulted; `maxOutputBytes` is a safe integer of at Source mode loads erasable-only `src/worker.ts` through Node's native type stripping. Its transitive runtime closure contains only Node built-ins and relative source modules, so a fresh checkout never requires a sibling workspace package's unbuilt `lib/` export. The worker-local and session-owned JSON boundaries both flatten and rebuild validated values around the message port so application nesting never reaches structured clone. Built mode passes the sibling `lib/worker.cjs` as a filesystem path because pkg's VFS Worker hook expects CommonJS; the same path works under ordinary Node. The repository-wide requirement to exercise this published entry path belongs to the [testing policy](../../../docs/testing.md). -The SDK surface is the default/named `WorkerCodeRuntime` class plus `Config`. The operational `./worker` subpath exists only as the packaged spawn entry; the wire protocol and bootstrap helpers are source-private implementation details. +The SDK surface is the default/named `WorkerCodeRuntime` class plus `Config`. The `./runtime-host` subpath shares type stripping, binding validation/dispatch, lossless JSON transport, and output accounting with sibling worker-based implementations; it is implementation support, not a plugin. The operational `./worker` subpath exists only as the packaged spawn entry; the wire protocol and bootstrap helpers remain source-private. ## Model Experience diff --git a/packages/code-runtime/code-runtime-worker/src/runtime-host.ts b/packages/code-runtime/code-runtime-worker/src/runtime-host.ts new file mode 100644 index 0000000000..334bd82e38 --- /dev/null +++ b/packages/code-runtime/code-runtime-worker/src/runtime-host.ts @@ -0,0 +1,222 @@ +/** Shared host mechanics for local and subprocess-hosted TypeScript worker runtimes. */ + +import { stripTypeScriptTypes } from 'node:module' +import type { + CodeBindingNamespace, + CodeJsonValue, + CodeRunFailure, + CodeRunRequest, + CodeRunResult, +} from '@deepseek-ai/dsh-code-runtime' +import { jsonStringBytesUpTo, jsonValueBytesUpTo, truncateJsonStringBytes } from './output-json.ts' +import { decodeWorkerJson, encodeWorkerJson, snapshotCodeJsonValue } from './worker-json.ts' +import type { WorkerJsonWire } from './worker-json.ts' + +/** Smallest cap that can represent an empty log array and failure message. */ +export const MIN_RUNTIME_OUTPUT_BYTES = 4 + +const IDENTIFIER = /^[A-Za-z_$][A-Za-z0-9_$]*$/ +const RESERVED_WORDS = new Set([ + 'await', 'break', 'case', 'catch', 'class', 'const', 'continue', 'debugger', 'default', 'delete', 'do', + 'else', 'enum', 'export', 'extends', 'false', 'finally', 'for', 'function', 'if', 'import', 'in', + 'instanceof', 'new', 'null', 'return', 'super', 'switch', 'this', 'throw', 'true', 'try', 'typeof', + 'var', 'void', 'while', 'with', 'yield', 'let', 'static', 'implements', 'interface', 'package', + 'private', 'protected', 'public', 'arguments', 'eval', +]) +const RESERVED_ERROR_PROPERTIES = new Set(['name', 'message', 'stack']) +const STRIP_WRAP = { prefix: 'async function __dsh_program__() {\n', suffix: '\n}' } as const + +/** One validated binding call received from an isolated worker. */ +export interface RuntimeBindingCall { + /** Correlation id supplied by the isolated worker. */ + readonly id: number + /** Injected namespace global. */ + readonly global: string + /** Declared namespace function. */ + readonly name: string + /** Untrusted lossless-JSON wire payload. */ + readonly args: unknown +} + +/** One host reply to an isolated worker binding call. */ +export type RuntimeBindingReply = + | { readonly type: 'reply'; readonly id: number; readonly ok: true; readonly value: WorkerJsonWire } + | { readonly type: 'reply'; readonly id: number; readonly ok: false; readonly message: string } + +/** + * Render an unknown thrown value without assuming it is an Error. + * @param error - thrown or rejected value. + * @returns the caller-facing diagnostic text. + */ +export function runtimeErrorMessage(error: unknown): string { + return error instanceof Error ? error.message : String(error) +} + +/** + * Strip erasable TypeScript while preserving the program's body coordinates. + * @param program - model-written async-function body. + * @returns JavaScript source with the wrapper removed. + */ +export function stripRuntimeProgram(program: string): string { + const stripped = stripTypeScriptTypes(STRIP_WRAP.prefix + program + STRIP_WRAP.suffix) + return stripped.slice(STRIP_WRAP.prefix.length, stripped.length - STRIP_WRAP.suffix.length) +} + +/** + * Validate binding globals and typed-error declarations shared by worker runtimes. + * @param request - code-runtime request carrying the namespaces. + * @param implementationName - package name used in seam-misuse diagnostics. + * @returns namespaces indexed by their injected global. + */ +export function validateRuntimeBindings( + request: CodeRunRequest, + implementationName: string, +): Map { + const bindings = new Map() + for (const namespace of request.bindings) { + if (!IDENTIFIER.test(namespace.global) || RESERVED_WORDS.has(namespace.global)) { + throw new Error(`${implementationName}: binding global ${JSON.stringify(namespace.global)} is not a usable identifier`) + } + if (namespace.global === 'console' || bindings.has(namespace.global)) { + throw new Error(`${implementationName}: duplicate binding global ${JSON.stringify(namespace.global)}`) + } + bindings.set(namespace.global, namespace) + } + + const errorClassNames = new Set() + for (const namespace of request.bindings) { + const descriptor = namespace.errorClass + if (descriptor === undefined) continue + if (!IDENTIFIER.test(descriptor.name) || RESERVED_WORDS.has(descriptor.name)) { + throw new Error(`${implementationName}: binding error class ${JSON.stringify(descriptor.name)} is not a usable identifier`) + } + if (descriptor.name === 'console' || bindings.has(descriptor.name) || errorClassNames.has(descriptor.name)) { + throw new Error(`${implementationName}: duplicate injected global ${JSON.stringify(descriptor.name)}`) + } + if (descriptor.memberNameProperty.length === 0 || RESERVED_ERROR_PROPERTIES.has(descriptor.memberNameProperty)) { + throw new Error(`${implementationName}: binding error member property ${JSON.stringify(descriptor.memberNameProperty)} is not usable`) + } + errorClassNames.add(descriptor.name) + } + return bindings +} + +/** + * Resolve one untrusted worker call through a declared host binding. + * @param call - parsed call envelope from the isolated worker. + * @param bindings - namespaces returned by {@link validateRuntimeBindings}. + * @returns a lossless-JSON success or stable rejection reply. + */ +export async function invokeRuntimeBinding( + call: RuntimeBindingCall, + bindings: ReadonlyMap, +): Promise { + const functions = bindings.get(call.global)?.functions + const fn = functions !== undefined && Object.hasOwn(functions, call.name) ? functions[call.name] : undefined + if (typeof fn !== 'function') { + return { type: 'reply', id: call.id, ok: false, message: `unknown binding ${JSON.stringify(`${call.global}.${call.name}`)}` } + } + const args = decodeWorkerJson(call.args) + if (args === undefined) { + return { type: 'reply', id: call.id, ok: false, message: 'binding arguments must be lossless JSON' } + } + try { + const resolved = await fn(args) + let value: CodeJsonValue | undefined + try { + value = snapshotCodeJsonValue(resolved) + } catch { + value = undefined + } + if (value === undefined) { + return { type: 'reply', id: call.id, ok: false, message: 'binding resolution must be lossless JSON' } + } + return { type: 'reply', id: call.id, ok: true, value: encodeWorkerJson(value) } + } catch (error: unknown) { + return { type: 'reply', id: call.id, ok: false, message: runtimeErrorMessage(error) } + } +} + +/** One run's combined outer-output ledger; binding values never enter it. */ +export class RuntimeOutputLedger { + private bytes = 2 + private entries = 0 + + /** @param maxBytes - hard cap for logs plus completion or failure payload. */ + constructor(private readonly maxBytes: number) {} + + /** + * Admit one exact log entry. + * @param text - candidate log entry. + * @param sink - ordered retained log list. + * @returns false when the hard cap was crossed. + */ + admit(text: string, sink: string[]): boolean { + const separatorBytes = this.entries > 0 ? 1 : 0 + const stringBytes = jsonStringBytesUpTo(text, this.maxBytes - this.bytes - separatorBytes) + if (stringBytes === undefined) return false + this.bytes += stringBytes + separatorBytes + this.entries += 1 + sink.push(text) + return true + } + + /** + * Finalize a successful completion against the combined cap. + * @param logs - retained ordered logs. + * @param value - optional lossless-JSON completion. + * @returns the completion or output-limit result. + */ + success(logs: string[], value?: CodeJsonValue): CodeRunResult { + if (value !== undefined && jsonValueBytesUpTo(value, this.maxBytes - this.bytes) === undefined) return this.limit(logs) + return { logs, ...value !== undefined ? { value } : {} } + } + + /** + * Finalize one failure diagnostic against the combined cap. + * @param logs - retained ordered logs. + * @param error - structured runtime failure. + * @returns the failure or output-limit result. + */ + failure(logs: string[], error: CodeRunFailure): CodeRunResult { + if (jsonStringBytesUpTo(error.message, this.maxBytes - this.bytes) === undefined) return this.limit(logs) + return { logs, error } + } + + /** + * Build an explicit output-limit failure with a fitting log prefix. + * @param logs - ordered logs observed before the limit. + * @returns bounded output-limit result. + */ + limit(logs: string[]): CodeRunResult { + const fullMessage = `outer output exceeded ${this.maxBytes} bytes` + const messageBytes = fullMessage.length + 2 + const retained: string[] = [] + let retainedBytes = 2 + const logBudget = this.maxBytes - messageBytes + for (const text of logs) { + const separatorBytes = retained.length > 0 ? 1 : 0 + const availableBytes = logBudget - retainedBytes - separatorBytes + const stringBytes = jsonStringBytesUpTo(text, availableBytes) + if (stringBytes !== undefined) { + retained.push(text) + retainedBytes += stringBytes + separatorBytes + continue + } + const prefix = truncateJsonStringBytes(text, availableBytes) + if (prefix.length > 0) { + const prefixBytes = jsonStringBytesUpTo(prefix, availableBytes) + /* v8 ignore next -- truncateJsonStringBytes guarantees the same bound. */ + if (prefixBytes === undefined) throw new Error('output ledger produced an oversized log prefix') + retained.push(prefix) + retainedBytes += prefixBytes + separatorBytes + } + break + } + const message = truncateJsonStringBytes(fullMessage, this.maxBytes - retainedBytes) + return { logs: retained, error: { kind: 'output-limit', message } } + } +} + +export { decodeWorkerJson, encodeWorkerJson, snapshotCodeJsonValue } from './worker-json.ts' +export type { WorkerJsonWire } from './worker-json.ts' diff --git a/packages/code-runtime/code-runtime-worker/tsconfig.json b/packages/code-runtime/code-runtime-worker/tsconfig.json index 555b3d161e..340e68e848 100644 --- a/packages/code-runtime/code-runtime-worker/tsconfig.json +++ b/packages/code-runtime/code-runtime-worker/tsconfig.json @@ -8,9 +8,6 @@ "src" ], "references": [ - { - "path": "../../core/session" - }, { "path": "../../../vendor/cosmokit" }, diff --git a/packages/code-runtime/code-runtime-worker/tsdown.config.ts b/packages/code-runtime/code-runtime-worker/tsdown.config.ts index 1c40637722..dac6f12a65 100644 --- a/packages/code-runtime/code-runtime-worker/tsdown.config.ts +++ b/packages/code-runtime/code-runtime-worker/tsdown.config.ts @@ -1,9 +1,10 @@ import { defineConfig } from 'tsdown' /** - * Build the index and worker as separate single-entry bundles. The sibling `worker.cjs` is loaded - * by file and must be CommonJS for pkg's VFS Worker hook. A multi-entry build emits an unlisted - * shared chunk omitted by the package's exact `files` whitelist; separate builds inline it. + * Build the plugin, reusable runtime host, and worker as separate bundles. The + * sibling `worker.cjs` is loaded by file and must be CommonJS for pkg's VFS + * Worker hook. Separate builds inline shared implementation instead of + * emitting an unlisted chunk outside the exact `files` whitelist. */ export default defineConfig([ { @@ -16,6 +17,16 @@ export default defineConfig([ dts: false, clean: false, }, + { + entry: ['lib/types/runtime-host.js'], + outDir: 'lib', + format: ['esm'], + platform: 'node', + target: 'es2024', + fixedExtension: false, + dts: false, + clean: false, + }, { entry: ['lib/types/worker.js'], outDir: 'lib', diff --git a/packages/code-runtime/code-runtime/README.md b/packages/code-runtime/code-runtime/README.md index bb1c20d00a..7014bf517a 100644 --- a/packages/code-runtime/code-runtime/README.md +++ b/packages/code-runtime/code-runtime/README.md @@ -34,5 +34,5 @@ No direct invalidation; the named consumer owns any request-prefix changes. - **`run()` is one-shot** — `logs` arrive only on the resolved `CodeRunResult`; the seam exposes no streaming-log or progress surface for a live program's output. - **A persistent REPL-style kernel is recorded future work** — the no-state-between-runs contract stands until a persistent-kernel backend brings its own logging story ([Code Mode Agent Note](../../../.agents/notes/implemented/feature/2026-06-15-code-mode.md)). -- **Only the worker-thread backend ships** — `'process'`/`'container'` are declared well-known `isolation` values with no implementation; a hard security boundary awaits a container backend. -- **Intermediate binding values have no byte cap** — implementations remain subject to structured-clone cost and process memory, while a provider or executor may already have imposed its own acquisition bound. +- **No runtime claims a hard security boundary** — both shipped implementations use fresh worker threads; the filesystem/subprocess backend can place them inside a stronger execution world, but no runtime reports `'container'` today. +- **Intermediate binding values are implementation-bounded** — the direct worker backend has no per-binding byte cap; the filesystem/subprocess backend bounds each bridge frame, but repeated or concurrent binding traffic remains subject to process memory. diff --git a/packages/context/workspace-context/tests/workspace-context.spec.ts b/packages/context/workspace-context/tests/workspace-context.spec.ts index 0af12596cc..8620c38981 100644 --- a/packages/context/workspace-context/tests/workspace-context.spec.ts +++ b/packages/context/workspace-context/tests/workspace-context.spec.ts @@ -75,6 +75,14 @@ class RecordingFileSystem extends FileSystem { return { targetKey: FsTargetKey(absolute), displayPath: absolute } } + override processPath(target: FsTarget): string { return String(target.targetKey) } + + override fileUrl(target: FsTarget): string { return `file://${target.targetKey}` } + + override contains(parent: FsTarget, child: FsTarget): boolean { + return child.targetKey === parent.targetKey || String(child.targetKey).startsWith(`${parent.targetKey}/`) + } + override async stat(target: FsTarget, signal?: AbortSignal): Promise { if (signal !== undefined) this.signals.push(signal) signal?.throwIfAborted() @@ -109,6 +117,12 @@ class RecordingFileSystem extends FileSystem { return this.entries.get(target.targetKey)?.content ?? '' } + override async readTextBounded(target: FsTarget, maxBytes: number, signal?: AbortSignal): Promise { + const text = await this.readText(target, signal) + if (Buffer.byteLength(text) > maxBytes) throw new Error('too large') + return text + } + override async streamText(target: FsTarget, signal?: AbortSignal): Promise> { if (signal !== undefined) this.signals.push(signal) signal?.throwIfAborted() diff --git a/packages/cordis/tool-cordis/src/api-catalog.ts b/packages/cordis/tool-cordis/src/api-catalog.ts index bf179462ee..c7b6df254e 100644 --- a/packages/cordis/tool-cordis/src/api-catalog.ts +++ b/packages/cordis/tool-cordis/src/api-catalog.ts @@ -312,6 +312,18 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [ signature: 'abstract resolve(path: string, opts?: { cwd?: string; signal?: AbortSignal }): Promise', jsDoc: '/**\n * Resolve a model/plugin-supplied path into a stable {@link FsTarget}. May perform I/O (a\n * remote/sandboxed backend may need a round-trip to map a path to a stable identity), hence\n * async even though the local backend only normalizes + realpaths.\n *\n * @param path - the path to resolve; relative paths resolve against `opts.cwd`.\n * @param opts - optional cwd override and cancellation signal.\n * @returns the stable target; the same file yields the same `targetKey`.\n */', }, + { + signature: 'abstract processPath(target: FsTarget): string', + jsDoc: '/**\n * Return the canonical absolute path a subprocess in this filesystem\'s\n * execution world can open. The path is deliberately separate from\n * {@link FsTarget.targetKey}: consumers may pass this value to another OS\n * capability, but must continue treating the target key as opaque.\n * @param target - the resolved target whose process path is required.\n * @returns an absolute path in the backend\'s execution world.\n */', + }, + { + signature: 'abstract fileUrl(target: FsTarget): string', + jsDoc: '/**\n * Return the canonical `file:` URI for a target in this filesystem\'s\n * execution world. Backends own URI encoding because the host platform may\n * differ from the execution platform.\n * @param target - the resolved target to encode.\n * @returns the target\'s canonical file URI.\n */', + }, + { + signature: 'abstract contains(parent: FsTarget, child: FsTarget): boolean', + jsDoc: '/**\n * Test canonical containment without exposing or parsing backend target\n * keys. Both targets must come from this provider.\n * @param parent - canonical directory target.\n * @param child - canonical candidate target.\n * @returns true when `child` is `parent` or a descendant of it.\n */', + }, { signature: 'abstract stat(target: FsTarget, signal?: AbortSignal): Promise', jsDoc: '/**\n * Return target metadata, or `undefined` when the target does not exist.\n * @param target - the resolved target to stat.\n * @param signal - aborts the metadata round-trip.\n * @returns metadata only, never content; undefined for an absent target.\n */', @@ -324,6 +336,10 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [ signature: 'abstract readText(target: FsTarget, signal?: AbortSignal): Promise', jsDoc: '/**\n * Read the whole regular text file as a single decoded string.\n * @param target - the resolved target to read.\n * @param signal - aborts the read.\n * @returns the full decoded UTF-8 content.\n */', }, + { + signature: 'abstract readTextBounded(target: FsTarget, maxBytes: number, signal?: AbortSignal): Promise', + jsDoc: '/**\n * Read one regular UTF-8 text file through a backend-owned stable handle,\n * rejecting before more than `maxBytes` are retained. The size check and\n * bytes read are one operation: a caller must not emulate this with\n * {@link stat} followed by {@link readText}, which admits growth and path\n * replacement races between the two calls.\n * @param target - the resolved target to read.\n * @param maxBytes - positive safe-integer byte ceiling.\n * @param signal - aborts the open/read operation.\n * @returns the complete decoded text when it fits.\n */', + }, { signature: 'abstract streamText(target: FsTarget, signal?: AbortSignal): Promise>', jsDoc: '/**\n * Stream the whole regular text file as decoded text chunks (same text\n * semantics as {@link readText}, for large files). The backend owns\n * cross-chunk UTF-8 decoding and binary rejection so the policy layer never\n * touches raw bytes.\n * @param target - the resolved target to read.\n * @param signal - aborts the stream, including between chunks.\n * @returns the chunk iterable, decoded and validated like {@link readText}.\n */', @@ -974,10 +990,18 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [ key: 'subprocess', summary: 'Abstract subprocess service.', methods: [ + { + signature: 'abstract resolveExecutable( command: string, env?: Readonly>, signal?: AbortSignal, ): Promise', + jsDoc: '/**\n * Resolve one configured executable in this provider\'s execution world.\n * Absolute paths are verified; bare names use the provider\'s scrubbed PATH\n * plus explicit environment overrides.\n * @param command - absolute executable path or bare PATH name.\n * @param env - explicit environment entries used for lookup.\n * @param signal - aborts remote or local lookup.\n * @returns a canonical executable path.\n */', + }, { signature: 'abstract spawn(spec: SubprocessSpawnSpec): SubprocessHandle', jsDoc: '/**\n * Start one managed child process from a fully-specified spec; this seam\n * applies no defaults.\n * @param spec - argv, directory, stdio dispositions, grace, cancellation, and environment.\n * @returns the live process handle (streams/readers, signalling, outcome promise).\n */', }, + { + signature: 'abstract spawnTerminal(spec: SubprocessTerminalSpawnSpec): Promise', + jsDoc: '/**\n * Allocate a real terminal and start one owned process session. This is the\n * only non-pipe process primitive: implementations own terminal byte I/O,\n * foreground groups, signals, and complete session-tree cleanup.\n * @param spec - fully specified argv, cwd, environment, dimensions, grace, and cancellation.\n * @returns the live terminal handle after allocation succeeds.\n */', + }, ], }, { @@ -2879,6 +2903,22 @@ export const TYPE_API: readonly TypeApiEntry[] = [ name: 'SubprocessStdio', declaration: 'export interface SubprocessStdio {\n stdin: SubprocessStdinMode;\n stdout: SubprocessOutputMode;\n stderr: SubprocessOutputMode;\n}', }, + { + name: 'SubprocessTerminalForeground', + declaration: 'export interface SubprocessTerminalForeground {\n processGroupId: number;\n inputWaiting: boolean;\n}', + }, + { + name: 'SubprocessTerminalHandle', + declaration: 'export interface SubprocessTerminalHandle {\n readonly pid: number;\n readonly output: Readable;\n readonly done: Promise;\n write(data: Uint8Array): Promise;\n inspectForeground(): Promise;\n signalForeground(signal: SubprocessTerminalSignal): Promise;\n terminate(): void;\n waitForExit(signal?: AbortSignal): Promise;\n}', + }, + { + name: 'SubprocessTerminalSignal', + declaration: 'export type SubprocessTerminalSignal = \'SIGINT\' | \'SIGTERM\' | \'SIGKILL\' | \'SIGTSTP\' | \'SIGHUP\';', + }, + { + name: 'SubprocessTerminalSpawnSpec', + declaration: 'export interface SubprocessTerminalSpawnSpec {\n argv: readonly string[];\n cwd: string;\n env?: Record | undefined;\n rows: number;\n cols: number;\n graceMs: number;\n signal?: AbortSignal | undefined;\n}', + }, { name: 'SurfaceEvent', declaration: 'export type SurfaceEvent = SessionEvent & {\n surfaceOp: SurfaceOp;\n};', diff --git a/packages/fs/README.i18n.yaml b/packages/fs/README.i18n.yaml index e435f166ad..c1a381cb81 100644 --- a/packages/fs/README.i18n.yaml +++ b/packages/fs/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/fs/README.md -README.md: 108d7d862a1dc6307268e2a93fa00789c952e440 -README.zh.md: 8b037cc3192bf6ceb0f0671d62911a8e035db24c +README.md: b6adabd5744cb2b3dcee78b71815f8e95ba780f1 +README.zh.md: 0841f538932452921d2b0d7d9534f023672f241d diff --git a/packages/fs/README.md b/packages/fs/README.md index 108d7d862a..b6adabd574 100644 --- a/packages/fs/README.md +++ b/packages/fs/README.md @@ -2,16 +2,19 @@ English | [中文](README.zh.md) -The filesystem capability family: provider seam, interchangeable backends, policy, and model-facing tools. All **product** packages. +The filesystem stack: a provider seam (execution-world paths, bounded text IO, and atomic mutation with an optional version guard), a local implementation, a policy gate plugin (observed-state + read-before-edit + version-guarded write/edit), the model-facing file tools + executor, and the bash-backed discovery tools. All **product** packages. | Package | Role | ctx key | |---|---|---| -| [`fs/`](fs/README.md) | Filesystem provider seam and policy-event vocabulary | `ctx.fs` | -| [`fs-local/`](fs-local/README.md) | Local-filesystem backend | registers `ctx.fs` | -| [`fs-sandbox/`](fs-sandbox/README.md) | Sandbox-enforcing backend | registers `ctx.fs` | -| [`fs-policy/`](fs-policy/README.md) | Observed-state and mutation policy | `fs/*` listeners | -| [`tool-fs/`](tool-fs/README.md) | Model-facing file tools | registers on `ctx.tools` | -| [`tool-fs-search/`](tool-fs-search/README.md) | Process-backed discovery tools | registers on `ctx.tools` | -| [`tool-str-replace-editor/`](tool-str-replace-editor/README.md) | Model-facing string-replacement editor | registers on `ctx.tools` | +| `fs/` | Provider seam: canonical process paths/file URIs/containment, text IO, and atomic mutation primitives; owns the `fs/*` policy events | `ctx.fs` | +| `fs-local/` | Local-filesystem `FileSystem` implementation | (registers `ctx.fs`) | +| `fs-sandbox/` | Sandbox-enforcing `FileSystem`: extends `fs-local` and fences write/edit by the per-call mode + workspace root policy (read-only denies, workspace-write contains to the session workspace + temp roots), reads pass through | (registers `ctx.fs`) | +| `fs-policy/` | Policy gate plugin: observed-state + read-before-edit + version-guarded write/edit, via the `fs/*` event gate | (no service — `fs/*` listeners) | +| `tool-fs/` | Model-facing `read`/`write`/`edit` tools AND the executor (reads via `ctx.fs`, owns read windowing, dispatches `fs/*`); preserves filesystem semantics for session-cwd-relative paths and advertises sandbox escalation fields when the mounted `ctx.fs` confines | (registers on `ctx.tools`) | +| `tool-fs-search/` | Model-facing `glob`/`grep` discovery tools when `rg` is available on the bash executor `PATH`, backed by fixed ripgrep commands through `ctx.bash`, NOT by `ctx.fs` provider methods | (registers on `ctx.tools`) | -Backends replace one another behind `ctx.fs`; policy and tools consume the seam independently. Discovery remains process-backed instead of expanding the provider contract. Child READMEs own containment, mutation, schema, and timeout details. +The interface lives at `fs/fs/`. A sandboxed, remote, or project-scoped filesystem backend can replace `fs-local` without touching the seam, the policy gate, or the model-facing tool schemas — `fs-sandbox` is the first such replacement (an in-process path fence over the shared sandbox mode; see [the cross-family fs sandbox Agent Note](../../.agents/notes/implemented/feature/2026-07-14-cross-family-fs-sandbox.md)). The policy (`fs-policy/`) is a plugin that participates only through the `fs/*` event gate, not a service the tool injects — so dropping it gracefully loses the policy and leaves the unconstrained bare provider rather than breaking the tool. A deployment that loads `tool-fs/` is expected to also load it. The mode fence and the read-before-edit gate are orthogonal and compose. Discovery (`tool-fs-search/`) deliberately does NOT extend the provider seam: search is a process-backed `rg` workflow on the bash executor, so filesystem backends stay free of a universal search contract; its tools register only when that executor can find `rg`, and its results are follow-up-readable when the bash workdir and the `read` root are the same workspace (the co-located deployment its README documents). + +## No timeouts on file IO + +`read`/`write`/`edit` take **no** `timeoutMs`, and the provider seam arms no deadline — unlike bash and web (which consume [`@deepseek-ai/dsh-timeout`](../util/timeout/README.md)) and the bash-backed `glob`/`grep` (whose declared `timeoutMs` is enforced by `@deepseek-ai/dsh-timeout-policy`): those are process-backed, where a deadline can really kill the work. A local syscall is best-effort-abortable at most: a timeout could not force an in-progress `fsync`/`rename` to stop, so a deadline here would be a knob that cannot deliver on its promise. Adding one would also be an implicit default in the exact place explicit-over-implicit forbids. Both reference agents (Claude Code, Codex) leave file IO untimed for the same reason; cancellation still propagates through the tool-execution signal for best-effort abort at syscall boundaries. diff --git a/packages/fs/README.zh.md b/packages/fs/README.zh.md index 8b037cc319..0841f53893 100644 --- a/packages/fs/README.zh.md +++ b/packages/fs/README.zh.md @@ -1,17 +1,20 @@ -# fs/ - 文件系统能力家族 +# fs/:文件系统能力族 [English](README.md) | 中文 -文件系统能力家族:提供方 seam、可互换后端、策略和面向模型工具。这些全是**产品**包。 +文件系统栈包括:提供方 seam(执行世界路径、有界文本 I/O 与带可选版本防护的原子变更)、本地实现、政策门禁插件(已观察状态、编辑前读取、版本防护的写入/编辑)、面向模型的文件工具与执行器,以及基于 bash 的发现工具。全部都是**产品** 包。 -| 包 | 职责 | ctx key | +| 包 | 角色 | ctx 键 | |---|---|---| -| [`fs/`](fs/README.md) | 文件系统提供方 seam 和策略事件词汇 | `ctx.fs` | -| [`fs-local/`](fs-local/README.md) | 本地文件系统后端 | 注册 `ctx.fs` | -| [`fs-sandbox/`](fs-sandbox/README.md) | 强制执行沙箱的后端 | 注册 `ctx.fs` | -| [`fs-policy/`](fs-policy/README.md) | 已观察状态和修改策略 | `fs/*` 监听器 | -| [`tool-fs/`](tool-fs/README.md) | 面向模型的文件工具 | 注册到 `ctx.tools` | -| [`tool-fs-search/`](tool-fs-search/README.md) | 基于进程的发现工具 | 注册到 `ctx.tools` | -| [`tool-str-replace-editor/`](tool-str-replace-editor/README.md) | 面向模型的字符串替换编辑器 | 注册到 `ctx.tools` | +| `fs/` | 提供方 seam:规范化进程路径、文件 URI 与包含关系、文本 I/O 和原子变更原语;拥有 `fs/*` 政策事件 | `ctx.fs` | +| `fs-local/` | 本地文件系统 `FileSystem` 实现 | (注册 `ctx.fs`) | +| `fs-sandbox/` | 强制沙箱的 `FileSystem`:扩展 `fs-local`,并按每次调用的模式与工作区根政策约束写入/编辑(只读模式拒绝,工作区写入模式限制在会话工作区与临时根目录内);读取直接通过 | (注册 `ctx.fs`) | +| `fs-policy/` | 政策门禁插件:通过 `fs/*` 事件门禁提供已观察状态、编辑前读取和版本防护的写入/编辑 | (无服务,仅有 `fs/*` 监听器) | +| `tool-fs/` | 面向模型的 `read`/`write`/`edit` 工具以及执行器(通过 `ctx.fs` 读取,拥有读取窗口逻辑,分派 `fs/*`);为会话 cwd 相对路径保留文件系统语义,并在已挂载的 `ctx.fs` 实施约束时声明沙箱升级字段 | (注册到 `ctx.tools`) | +| `tool-fs-search/` | 面向模型的 `glob`/`grep` 发现工具;当 `rg` 位于 bash 执行器 `PATH` 上时注册,通过 `ctx.bash` 运行固定 ripgrep 命令,而不是使用 `ctx.fs` 提供方方法 | (注册到 `ctx.tools`) | -后端可在 `ctx.fs` 后互相替换;策略和工具独立消费该 seam。发现功能仍由进程提供,不扩展提供方契约。子 README 负责围堵、修改、schema 和超时细节。 +接口位于 `fs/fs/`。沙箱化、远程或限定项目作用域的文件系统后端可以替换 `fs-local`,而无需更改 seam、政策门禁或面向模型的工具 schema;`fs-sandbox` 是第一个这样的替代实现(基于共享沙箱模式的进程内路径围栏;见[跨能力族 fs 沙箱 Agent Note](../../.agents/notes/implemented/feature/2026-07-14-cross-family-fs-sandbox.md))。政策(`fs-policy/`)是一个只通过 `fs/*` 事件门禁参与的插件,不是工具注入的服务;因此移除它会平稳失去政策,留下不受约束的裸提供方,而不会破坏工具。加载 `tool-fs/` 的部署也应加载该插件。模式围栏与编辑前读取门禁彼此正交,可以组合。发现(`tool-fs-search/`)有意不扩展提供方 seam:搜索是在 bash 执行器上运行 `rg`、由进程支持的工作流,因此文件系统后端无需承担通用搜索契约;只有当执行器能找到 `rg` 时,其工具才会注册。如果 bash 工作目录与 `read` 根目录是同一工作区,结果就能继续读取,这也是其 README 所述的共置部署。 + +## 文件 I/O 不设超时 + +`read`/`write`/`edit` **不** 接受 `timeoutMs`,提供方 seam 也不启动 deadline。这与 bash 和 web(两者使用 [`@deepseek-ai/dsh-timeout`](../util/timeout/README.md))及基于 bash 的 `glob`/`grep` 不同(它们声明的 `timeoutMs` 由 `@deepseek-ai/dsh-timeout-policy` 强制执行):这些工作由进程支持,deadline 可以实际终止工作。本地系统调用至多只能尽力中止:超时无法强制正在进行的 `fsync`/`rename` 停止,因此这里的 deadline 会成为无法兑现承诺的配置项。在此添加 deadline 还会在「显式优于隐式」明确禁止的地方引入隐式默认值。两个参考 agent(Claude Code、Codex)出于同一原因都不为文件 I/O 计时;取消仍通过工具执行信号传播,在系统调用边界尽力中止。 diff --git a/packages/fs/fs-local/README.i18n.yaml b/packages/fs/fs-local/README.i18n.yaml index cc14be584f..57d2a32f97 100644 --- a/packages/fs/fs-local/README.i18n.yaml +++ b/packages/fs/fs-local/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/fs/fs-local/README.md -README.md: 6d344fa3fef7f6bda6c0daa50184661156a925a7 -README.zh.md: 4c94de64561d805684f91f02d5b0dfc375f0d04f +README.md: f5cf2441adcd62e55c6169ceb766f88382e314f5 +README.zh.md: 40f5a83626780bae8f335c80662721bae6cc0b0b diff --git a/packages/fs/fs-local/README.md b/packages/fs/fs-local/README.md index 6d344fa3fe..f5cf2441ad 100644 --- a/packages/fs/fs-local/README.md +++ b/packages/fs/fs-local/README.md @@ -2,7 +2,7 @@ English | [中文](README.zh.md) -The **local-filesystem implementation** of the `ctx.fs` provider seam ([`@deepseek-ai/dsh-fs`](../fs)). Backs the eight `FileSystem` primitives with the host filesystem; loading it as a plugin populates `ctx.fs`. +The **local-filesystem implementation** of the `ctx.fs` provider seam ([`@deepseek-ai/dsh-fs`](../fs)). Backs the twelve `FileSystem` primitives with the host filesystem; loading it as a plugin populates `ctx.fs`. ```ts ignore-check import { LocalFileSystem } from '@deepseek-ai/dsh-fs-local' @@ -15,8 +15,9 @@ await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) ## Behavior - **`resolve(path, opts?)`** — a relative `path` resolves against `opts.cwd` when the caller supplies one (the model-facing tools pass the calling agent's session cwd — see [the per-session cwd Agent Note](../../../.agents/notes/implemented/architecture/2026-07-02-fs-per-session-cwd.md)), else `config.cwd` (default `process.cwd()`); an absolute `path` ignores both. `opts.signal` is checked before and after local resolution, while a remote sibling backend may use it to abort its round-trip. The `targetKey` is the file's `realpath`, so two input paths reaching the same file through symlinks share one identity, and writes/edits land on the link target (preserving the link). A not-yet-existing path uses the realpathed parent directory plus basename when the parent exists; only an unresolvable parent falls back to the absolute path. `displayPath` is the absolute (un-resolved) path. +- **Execution-world coordinates** — `processPath` exposes the target's canonical host path, `fileUrl` encodes that path through Node's platform-aware URL conversion, and `contains` uses platform path semantics to test identity or descendant containment without consumers parsing `targetKey`. - **`stat` / `lstat`** — return target metadata or `undefined` when absent. `stat` reports `FsInfo` for an already resolved target (`version` = an opaque token derived from bigint `dev:ino:size:mtimeNs:ctimeNs`, `type` of `file`/`directory`/`other`, byte `size`); path-shaped `lstat` reports `FsPathInfo` without following the final symlink and can therefore return `symlink`. Both check cancellation before and after their asynchronous metadata probe, so an abort that lands in flight reports `FS_ABORTED` rather than stale absence. -- **`readText` / `streamText`** — UTF-8 only. `readText` reads the whole file; `streamText` streams it in chunks (cross-chunk decoding) so a huge file never has to be held whole in memory. Both reject invalid UTF-8 and NUL-byte binary samples (`FS_NOT_TEXT`) and non-regular targets. The `read` tool (`@deepseek-ai/dsh-tool-fs`) decides which to call by size and owns the line windowing. +- **`readText` / `readTextBounded` / `streamText`** — UTF-8 only. `readText` reads the whole file; `readTextBounded` opens one no-follow, nonblocking handle, verifies it is regular, and retains at most `maxBytes + 1` bytes so growth cannot bypass the cap; `streamText` decodes chunks so a huge file need not be held whole in memory. All reject invalid UTF-8 and NUL-byte binary samples (`FS_NOT_TEXT`) and non-regular targets. The `read` tool (`@deepseek-ai/dsh-tool-fs`) owns line windowing; protocol consumers such as the LSP host use the stable bounded operation. - **`listDir`** — lists one directory level in stable `name.localeCompare()` order. Each entry carries the child basename, type, resolved child target (`displayPath` under the listed directory, `targetKey` as the realpath identity), and cheap stat metadata (`version`, plus `size` for regular files). It never opens or decodes file contents. Missing targets report `FS_NOT_FOUND`, file/special-file targets report `FS_NOT_DIRECTORY`, aborted calls report `FS_ABORTED`, permission failures report `FS_PERMISSION_DENIED`, and other listing or child metadata I/O failures report `FS_IO_ERROR`. Broken/disappeared children are returned as `other` without metadata, but permission/IO failures while resolving a child fail the whole listing with a structured `FsError`. - **`writeText`** — atomic: writes to a temp file opened exclusively (`wx`, `0o600`) inside a randomly-named private staging dir (`0o700`) next to the target, fsyncs, then renames over the target. An existing file's mode is preserved, while new files default to `0o600`; on Windows a new file inherits the destination directory's DACL, while replacement copies the target DACL onto the empty temp before writing and publishes through `ReplaceFileW` so the original access policy survives ([Windows DACL preservation Agent Note](../../../.agents/notes/implemented/bug-fix/2026-07-19-windows-atomic-write-dacl-preservation.md)). The `expected` guard is OPTIONAL: omitting it unconditionally creates-or-overwrites; `createIfAbsent` creates a missing target and rejects an existing one (`FS_NOT_OBSERVED`); `replaceIfVersion` replaces only at the observed version (a missing target or mismatch is `FS_STALE_VERSION`). - **`editText`** — atomic literal read-modify-write over the same primitive, serialized per target by a mutation lock. The `expected` guard is OPTIONAL: when supplied it verifies the version BEFORE literal matching (a stale edit reports `FS_STALE_VERSION`, never `FS_EDIT_NOT_FOUND`/`FS_AMBIGUOUS_EDIT` against newer content); omitting it edits the current content unconditionally. A missing target reports `FS_STALE_VERSION` either way. LF-normalizes for matching, restores the file's dominant CRLF/LF style, and rejects empty `oldString` / zero matches (`FS_EDIT_NOT_FOUND`) or ambiguous multi-matches without `replace_all` (`FS_AMBIGUOUS_EDIT`). @@ -35,7 +36,7 @@ No direct invalidation; the named consumer owns any request-prefix changes. - **`config.cwd` is not a sandbox** — it is a resolution default, not containment: absolute paths and `..` escape it. Enforce containment with a stricter `ctx.fs` backend or a permission plugin on the `tools/execute` waterfall ([capability-seam Agent Note](../../../.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.md#consequences)). - **An overwrite reads the whole prior file into memory** — solely as the UI diff basis; bounding that pre-read above a size threshold is deferred (`TODO(overwrite-diff-bound)`). -- **Version tokens are `mtimeMs:size`** — an external change that preserves both within the filesystem's timestamp granularity defeats the stale guard. +- **Version tokens depend on filesystem metadata** — they combine device, inode, size, nanosecond mtime, and nanosecond ctime; a storage layer that cannot update any of those facts for a rewrite can still defeat the stale guard. - **`editText` holds the whole file (plus the edited copy) in memory** — streaming exists only on the read path. - **Binary detection is asymmetric** — reads NUL-sample only the first 8192 bytes while edits scan the whole buffer, so a file with a late NUL reads fine but rejects edits. - **The per-target mutation lock is in-process only** — a writer in another process is caught only by the optional version guard, never serialized. diff --git a/packages/fs/fs-local/README.zh.md b/packages/fs/fs-local/README.zh.md index 4c94de6456..40f5a83626 100644 --- a/packages/fs/fs-local/README.zh.md +++ b/packages/fs/fs-local/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -`ctx.fs` 提供方 seam([`@deepseek-ai/dsh-fs`](../fs))的**本地文件系统实现**。它使用宿主文件系统支持八个 `FileSystem` 原语;将其作为插件加载会填充 `ctx.fs`。 +`ctx.fs` 提供方 seam([`@deepseek-ai/dsh-fs`](../fs))的**本地文件系统实现**。它使用宿主文件系统支持十二个 `FileSystem` 原语;将其作为插件加载会填充 `ctx.fs`。 ```ts ignore-check import { LocalFileSystem } from '@deepseek-ai/dsh-fs-local' @@ -15,27 +15,28 @@ await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) ## 行为 - **`resolve(path, opts?)`**:相对 `path` 在调用方提供 `opts.cwd` 时以该值为基准解析(面向模型的工具会传入调用 agent(智能体)的会话 cwd;见[每会话 cwd Agent Note](../../../.agents/notes/implemented/architecture/2026-07-02-fs-per-session-cwd.md)),否则以 `config.cwd` 为基准(默认 `process.cwd()`);绝对 `path` 会忽略两者。`opts.signal` 会在本地解析前后检查,远程同级后端则可以用它中止往返。`targetKey` 是文件的 `realpath`,因此经符号链接到达同一文件的两个输入路径会共享一个身份,写入/编辑落在链接目标上,同时保留链接。尚不存在的路径在父目录存在时使用 realpath 后的父目录加 basename;只有父目录无法解析时才回退到绝对路径。`displayPath` 是绝对但未经解析的路径。 -- **`stat` / `lstat`**:返回目标元数据;目标不存在时返回 `undefined`。`stat` 为已解析目标报告 `FsInfo`(`version` 是由 bigint `dev:ino:size:mtimeNs:ctimeNs` 派生的不透明 token,`type` 为 `file`/`directory`/`other`,`size` 以字节计);路径形态的 `lstat` 不跟随最后一个符号链接,报告 `FsPathInfo`,因此可以返回 `symlink`。两者都会在异步元数据探测前后检查取消,因此异步探测进行期间发生的中止会报告 `FS_ABORTED`,而非已失效的「不存在」结果。 -- **`readText` / `streamText`**:只支持 UTF-8。`readText` 读取整个文件;`streamText` 按分片流式读取(跨分片解码),因此超大文件无需整体保存在内存中。两者都会拒绝无效 UTF-8、包含 NUL 字节的二进制样本(`FS_NOT_TEXT`)以及非普通文件目标。`read` 工具(`@deepseek-ai/dsh-tool-fs`)按大小决定调用哪个方法,并负责行窗口逻辑。 +- **执行世界坐标**:`processPath` 公开目标的规范化宿主路径,`fileUrl` 通过 Node 的平台感知 URL 转换对该路径编码,`contains` 则使用平台路径语义检查身份相等或后代包含关系,消费方无需解析 `targetKey`。 +- **`stat` / `lstat`**:返回目标元数据;目标不存在时返回 `undefined`。`stat` 为已解析目标报告 `FsInfo`(`version` 是由 bigint `dev:ino:size:mtimeNs:ctimeNs` 派生的不透明 token,`type` 为 `file`/`directory`/`other`,`size` 以字节计);路径形态的 `lstat` 不跟随最后一个符号链接,报告 `FsPathInfo`,因此可以返回 `symlink`。两者都会在异步元数据探测前后检查取消,因此飞行中的中止会报告 `FS_ABORTED`,而非陈旧的不存在结果。 +- **`readText` / `readTextBounded` / `streamText`**:只支持 UTF-8。`readText` 读取整个文件;`readTextBounded` 打开一个不跟随符号链接的非阻塞句柄,确认其为普通文件,并最多保留 `maxBytes + 1` 字节,使文件增长无法绕过上限;`streamText` 按分片解码,因此超大文件无需整体保存在内存中。三者都会拒绝无效 UTF-8、包含 NUL 字节的二进制样本(`FS_NOT_TEXT`)以及非普通文件目标。`read` 工具(`@deepseek-ai/dsh-tool-fs`)拥有行窗口逻辑;LSP 主机等协议消费方使用稳定的有界操作。 - **`listDir`**:按稳定的 `name.localeCompare()` 顺序列出一层目录。每个条目携带子项 basename、类型、解析后的子目标(`displayPath` 位于所列目录下,`targetKey` 是 realpath 身份)和低成本 stat 元数据(`version`,普通文件另有 `size`)。它绝不会打开或解码文件内容。缺失目标报告 `FS_NOT_FOUND`,文件/特殊文件目标报告 `FS_NOT_DIRECTORY`,已中止调用报告 `FS_ABORTED`,权限失败报告 `FS_PERMISSION_DENIED`,其他列出或子项元数据 I/O 失败报告 `FS_IO_ERROR`。损坏/消失的子项以无元数据的 `other` 返回,但解析子项时出现权限/I/O 失败会让整个列表以结构化 `FsError` 失败。 -- **`writeText`**:原子写入。它会向排他打开的临时文件(`wx`、`0o600`)写入;该文件位于目标旁随机命名的私有暂存目录(`0o700`)内。完成写入和 fsync 后,以 rename 覆盖目标。现有文件的 mode 会保留,新文件默认为 `0o600`;Windows 上的新文件继承目标目录的 DACL,而替换会在写入前把目标 DACL 复制到空临时文件,并通过 `ReplaceFileW` 发布,使原访问策略得以保留(见 [Windows DACL 保留 Agent Note](../../../.agents/notes/implemented/bug-fix/2026-07-19-windows-atomic-write-dacl-preservation.md))。`expected` 防护是可选(OPTIONAL)的:省略时无条件创建或覆盖;`createIfAbsent` 创建缺失目标并拒绝现有目标(`FS_NOT_OBSERVED`);`replaceIfVersion` 只在观察到的版本上替换(目标缺失或版本不匹配均为 `FS_STALE_VERSION`)。 -- **`editText`**:在同一原语之上执行原子式的字面量读取-修改-写入,并通过变更锁按目标串行化。`expected` 防护是可选(OPTIONAL)的:提供时,会在字面量匹配之前校验版本(陈旧编辑报告 `FS_STALE_VERSION`,绝不会针对较新内容报告 `FS_EDIT_NOT_FOUND`/`FS_AMBIGUOUS_EDIT`);省略时,无条件编辑当前内容。无论哪种情况,目标缺失都报告 `FS_STALE_VERSION`。匹配时规范化为 LF,随后恢复文件主要的 CRLF/LF 风格;空 `oldString` / 零匹配报告 `FS_EDIT_NOT_FOUND`,未设置 `replace_all` 的多个匹配则报告 `FS_AMBIGUOUS_EDIT`。 +- **`writeText`**:原子写入。它会向排他打开的临时文件(`wx`、`0o600`)写入;该文件位于目标旁随机命名的私有暂存目录(`0o700`)内。完成写入和 fsync 后,以 rename 覆盖目标。现有文件的 mode 会保留,新文件默认为 `0o600`;Windows 上的新文件继承目标目录的 DACL,而替换会在写入前把目标 DACL 复制到空临时文件,并通过 `ReplaceFileW` 发布,使原访问政策得以保留(见 [Windows DACL 保留 Agent Note](../../../.agents/notes/implemented/bug-fix/2026-07-19-windows-atomic-write-dacl-preservation.md))。`expected` 防护是可选的:省略时无条件创建或覆盖;`createIfAbsent` 创建缺失目标并拒绝现有目标(`FS_NOT_OBSERVED`);`replaceIfVersion` 只在观察到的版本上替换(目标缺失或版本不匹配均为 `FS_STALE_VERSION`)。 +- **`editText`**:在同一原语之上依次执行原子的字面量读取、修改和写入,并通过变更锁按目标串行化。`expected` 防护是可选的:提供时,会在字面量匹配之前校验版本(陈旧编辑报告 `FS_STALE_VERSION`,绝不会针对较新内容报告 `FS_EDIT_NOT_FOUND`/`FS_AMBIGUOUS_EDIT`);省略时,无条件编辑当前内容。无论哪种情况,目标缺失都报告 `FS_STALE_VERSION`。匹配时规范化为 LF,随后恢复文件主要的 CRLF/LF 风格;空 `oldString` / 零匹配报告 `FS_EDIT_NOT_FOUND`,未设置 `replace_all` 的多个匹配则报告 `FS_AMBIGUOUS_EDIT`。 -包根目录的 SDK 接口包含默认/具名 `LocalFileSystem` 类和 `Config`。原始 I/O 位于 `src/fsio.ts`(不依赖 Cordis,单独进行单元测试);`src/index.ts` 是轻量服务接线。 +包根 SDK 接口包含默认/具名 `LocalFileSystem` 类和 `Config`。原始 I/O 位于 `src/fsio.ts`(不依赖 Cordis,单独进行单元测试);`src/index.ts` 是轻量服务接线。 ## 模型体验 -通过 [`dsh-tool-fs`](../tool-fs/README.md) 间接产生影响;该消费方在有上限的保留结果中渲染本提供方带行窗口的 UTF-8 内容、变更确认和精确提供方消息,而版本、原子写入机制和目录元数据仍属内部实现。 +通过 [`dsh-tool-fs`](../tool-fs/README.md) 间接产生影响;该消费方把本提供方带行窗口的 UTF-8 内容、变更确认和精确提供方消息渲染为有上限且保留的结果,而版本、原子写入机制和目录元数据保持内部可见。 #### KV Cache 影响 不会直接使缓存失效;具名消费方负责请求前缀的任何变化。 -## 已知限制与暂缓事项 +## 已知限制与延期工作 - **`config.cwd` 不是沙箱**:它是解析默认值,而非约束;绝对路径和 `..` 可以逃逸。请使用更严格的 `ctx.fs` 后端或 `tools/execute` waterfall(瀑布式事件)上的权限插件实施约束(见[能力 seam Agent Note](../../../.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.md#consequences))。 - **覆盖会把整个旧文件读入内存**:只用于 UI diff;在大小阈值之上限制这次预读取的工作延期处理(`TODO(overwrite-diff-bound)`)。 -- **版本 token 是 `mtimeMs:size`**:如果外部变更在文件系统时间戳粒度内保持两者不变,就能绕过陈旧防护。 +- **版本 token 依赖文件系统元数据**:它们组合设备、inode、大小、纳秒级 mtime 和纳秒级 ctime;如果存储层在重写时无法更新其中任何一项事实,仍可能绕过陈旧防护。 - **`editText` 会把整个文件及编辑后的副本保存在内存中**:只有读取路径支持流式处理。 - **二进制检测不对称**:读取只对前 8192 字节执行 NUL 采样,编辑则扫描整个 buffer,因此 NUL 出现在后部的文件可以读取,但编辑会被拒绝。 - **每目标变更锁仅限进程内**:其他进程中的写入方只会被可选版本防护发现,绝不会被串行化。 diff --git a/packages/fs/fs-local/src/fsio.ts b/packages/fs/fs-local/src/fsio.ts index c93e4ddaeb..e95cb14cca 100644 --- a/packages/fs/fs-local/src/fsio.ts +++ b/packages/fs/fs-local/src/fsio.ts @@ -6,7 +6,7 @@ */ import { randomUUID } from 'node:crypto' -import { createReadStream } from 'node:fs' +import { constants, createReadStream } from 'node:fs' import { chmod, lstat, mkdir, open, readFile, realpath, readdir, rename, rm, stat } from 'node:fs/promises' import type { BigIntStats, Dirent, Stats } from 'node:fs' import { basename, dirname, join, resolve } from 'node:path' @@ -369,6 +369,77 @@ export async function readWholeText(target: LocalTarget, signal?: AbortSignal): return decodeUtf8(raw, 'read', target.displayPath) } +/** + * Read one regular UTF-8 file through a single no-follow handle, retaining at + * most `maxBytes + 1` bytes so a concurrent grow cannot bypass the bound. + * @param target - the resolved file to read. + * @param maxBytes - positive safe-integer byte ceiling. + * @param signal - aborts between handle operations. + * @returns the complete decoded text when it fits. + */ +export async function readWholeTextBounded( + target: LocalTarget, + maxBytes: number, + signal?: AbortSignal, +): Promise { + if (!Number.isSafeInteger(maxBytes) || maxBytes <= 0) { + throw new Error('bounded read maxBytes must be a positive safe integer') + } + throwIfAborted(signal, 'read') + let handle: Awaited> + try { + handle = await open( + target.targetKey, + constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK, + ) + } catch (error: unknown) { + if (isENOENT(error)) throw new FsError(`cannot read "${target.displayPath}": not found`, 'FS_NOT_FOUND', { cause: error }) + if (isPermissionError(error)) throw new FsError(`cannot read "${target.displayPath}": permission denied`, 'FS_PERMISSION_DENIED', { cause: error }) + throw new FsError(`cannot read "${target.displayPath}" safely: ${errorMessage(error)}`, 'FS_IO_ERROR', { cause: error }) + } + try { + throwIfAborted(signal, 'read') + const info = await handle.stat() + if (!info.isFile()) { + throw new FsError(`cannot read "${target.displayPath}": not a regular file`, 'FS_NOT_REGULAR_FILE') + } + if (info.size > maxBytes) { + throw new FsError( + `cannot read "${target.displayPath}": ${info.size} bytes exceeds the ${maxBytes}-byte limit`, + 'FS_IO_ERROR', + ) + } + const chunks: Buffer[] = [] + let total = 0 + for (;;) { + throwIfAborted(signal, 'read') + // Allocate in fixed internal chunks so a permissive deployment cap does + // not reserve that entire cap for a small file. Once exactly at the + // bound, one final byte detects concurrent growth without retaining it. + const remaining = total === maxBytes ? 1 : Math.min(64 * 1024, maxBytes - total) + const chunk = Buffer.allocUnsafe(remaining) + const { bytesRead } = await handle.read(chunk, 0, chunk.length, total) + if (bytesRead === 0) break + total += bytesRead + if (total > maxBytes) { + throw new FsError( + `cannot read "${target.displayPath}": file grew past the ${maxBytes}-byte limit while reading`, + 'FS_IO_ERROR', + ) + } + chunks.push(chunk.subarray(0, bytesRead)) + } + throwIfAborted(signal, 'read') + const bytes = chunks.length === 1 ? chunks[0] as Buffer : Buffer.concat(chunks, total) + if (bytes.subarray(0, BINARY_SAMPLE_BYTES).includes(0)) { + throw new FsError(`cannot read "${target.displayPath}": binary file`, 'FS_NOT_TEXT') + } + return decodeUtf8(bytes, 'read', target.displayPath) + } finally { + await handle.close() + } +} + /** * Stream a whole regular UTF-8 text file as decoded text chunks. Same text * semantics as {@link readWholeText} (regular-file check, binary/NUL rejection, diff --git a/packages/fs/fs-local/src/index.ts b/packages/fs/fs-local/src/index.ts index 18433f3f7c..1a266301db 100644 --- a/packages/fs/fs-local/src/index.ts +++ b/packages/fs/fs-local/src/index.ts @@ -5,7 +5,8 @@ */ import { Context } from 'cordis' -import { resolve } from 'node:path' +import { isAbsolute, relative, resolve, sep } from 'node:path' +import { pathToFileURL } from 'node:url' import z from 'schemastery' import { FileSystem, FsError, FsVersion } from '@deepseek-ai/dsh-fs' import type { @@ -27,6 +28,7 @@ import { readForEdit, readTextForDiff, readWholeText, + readWholeTextBounded, resolveLocalTarget, restoreLineEndings, streamWholeText, @@ -90,6 +92,19 @@ export class LocalFileSystem extends FileSystem { return { targetKey: local.targetKey, displayPath: local.displayPath } } + override processPath(target: FsTarget): string { + return String(target.targetKey) + } + + override fileUrl(target: FsTarget): string { + return pathToFileURL(this.processPath(target)).href + } + + override contains(parent: FsTarget, child: FsTarget): boolean { + const path = relative(this.processPath(parent), this.processPath(child)) + return path === '' || (path !== '..' && !path.startsWith(`..${sep}`) && !isAbsolute(path)) + } + override async stat(target: FsTarget, signal?: AbortSignal): Promise { if (signal?.aborted) throw new FsError('stat aborted', 'FS_ABORTED') const info = await probe(target.targetKey) @@ -111,6 +126,10 @@ export class LocalFileSystem extends FileSystem { return readWholeText({ displayPath: target.displayPath, targetKey: target.targetKey }, signal) } + override async readTextBounded(target: FsTarget, maxBytes: number, signal?: AbortSignal): Promise { + return readWholeTextBounded({ displayPath: target.displayPath, targetKey: target.targetKey }, maxBytes, signal) + } + override streamText(target: FsTarget, signal?: AbortSignal): Promise> { return Promise.resolve(streamWholeText({ displayPath: target.displayPath, targetKey: target.targetKey }, signal)) } diff --git a/packages/fs/fs-local/tests/filesystem.spec.ts b/packages/fs/fs-local/tests/filesystem.spec.ts index 61e2c0e999..02beab7323 100644 --- a/packages/fs/fs-local/tests/filesystem.spec.ts +++ b/packages/fs/fs-local/tests/filesystem.spec.ts @@ -10,6 +10,7 @@ import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' import { mkdir, mkdtemp, readFile, realpath, rm, stat, symlink, unlink, utimes, writeFile } from 'node:fs/promises' import { tmpdir } from 'node:os' import { join } from 'node:path' +import { pathToFileURL } from 'node:url' import { Context } from 'cordis' import { LocalFileSystem } from '@deepseek-ai/dsh-fs-local' import { FsVersion } from '@deepseek-ai/dsh-fs' @@ -85,6 +86,20 @@ describe('resolve', () => { await expect(pending).rejects.toMatchObject({ code: 'FS_ABORTED' }) }) + + it('projects process paths, file URLs, and canonical containment', async () => { + await mkdir(join(dir, 'nested')) + await writeFile(join(dir, 'nested', 'file.txt'), 'text') + const root = await fs.resolve('.') + const child = await fs.resolve('nested/file.txt') + const outside = await fs.resolve('..') + + expect(fs.processPath(child)).toBe(await realpath(join(dir, 'nested', 'file.txt'))) + expect(fs.fileUrl(child)).toBe(pathToFileURL(await realpath(join(dir, 'nested', 'file.txt'))).href) + expect(fs.contains(root, root)).toBe(true) + expect(fs.contains(root, child)).toBe(true) + expect(fs.contains(root, outside)).toBe(false) + }) }) describe('stat', () => { @@ -203,6 +218,15 @@ describe('readText / streamText', () => { expect(await fs.readText(await fs.resolve('a.txt'))).toBe('one\ntwo\nthree') }) + it('reads complete text through the stable byte bound', async () => { + await writeFile(join(dir, 'bounded.txt'), '€abc') + const target = await fs.resolve('bounded.txt') + expect(await fs.readTextBounded(target, 6)).toBe('€abc') + await expect(fs.readTextBounded(target, 5)).rejects.toThrow('exceeds the 5-byte limit') + await expect(fs.readTextBounded(target, 0)).rejects.toThrow('positive safe integer') + await expect(fs.readTextBounded(target, 6, AbortSignal.abort())).rejects.toMatchObject({ code: 'FS_ABORTED' }) + }) + it('streams the same text', async () => { await writeFile(join(dir, 'a.txt'), 'one\ntwo\nthree') const target = await fs.resolve('a.txt') diff --git a/packages/fs/fs-local/tests/fsio.spec.ts b/packages/fs/fs-local/tests/fsio.spec.ts index 15588e40b9..2c5fd7a8f4 100644 --- a/packages/fs/fs-local/tests/fsio.spec.ts +++ b/packages/fs/fs-local/tests/fsio.spec.ts @@ -5,7 +5,7 @@ * policy and lives in `dsh-fs-policy`, so it is not tested here. */ -import { afterEach, beforeEach, describe, expect, it } from 'vitest' +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' import { chmod, mkdtemp, readFile, rename, rm, stat, symlink, unlink, writeFile, mkdir, readdir, realpath } from 'node:fs/promises' import { tmpdir } from 'node:os' import { join } from 'node:path' @@ -17,6 +17,7 @@ import { probeNoFollow, readForEdit, readWholeText, + readWholeTextBounded, resolveLocalTarget, restoreLineEndings, streamWholeText, @@ -316,6 +317,66 @@ describe('readWholeText', () => { }) }) +describe('readWholeTextBounded', () => { + it('rejects non-files, binary text, and initial oversize without a full read', async () => { + await expect(readWholeTextBounded(localTarget(dir), 10)).rejects.toMatchObject({ code: 'FS_NOT_REGULAR_FILE' }) + await writeFile(join(dir, 'large'), '12345') + await expect(readWholeTextBounded(localTarget(join(dir, 'large')), 4)).rejects.toThrow('exceeds the 4-byte limit') + await writeFile(join(dir, 'binary'), Buffer.from([0x61, 0x00, 0x62])) + await expect(readWholeTextBounded(localTarget(join(dir, 'binary')), 3)).rejects.toMatchObject({ code: 'FS_NOT_TEXT' }) + }) + + it('detects growth past the bound on the same open handle', async () => { + const close = vi.fn(async () => {}) + const read = vi.fn(async (buffer: Buffer, offset: number, length: number, position: number) => { + const bytes = position === 0 ? Buffer.from('abc') : Buffer.from('d') + bytes.copy(buffer, offset, 0, Math.min(length, bytes.length)) + return { bytesRead: Math.min(length, bytes.length), buffer } + }) + vi.resetModules() + vi.doMock('node:fs/promises', async (importOriginal) => { + const actual = await importOriginal() + return { + ...actual, + open: async () => ({ + stat: async () => ({ isFile: () => true, size: 3 }), + read, + close, + }), + } + }) + try { + const isolated = await import('../src/fsio.ts') + await expect(isolated.readWholeTextBounded(localTarget('/virtual/growing'), 3)) + .rejects.toThrow('grew past the 3-byte limit') + expect(close).toHaveBeenCalledOnce() + } finally { + vi.doUnmock('node:fs/promises') + vi.resetModules() + } + }) + + it('translates permission and generic open failures', async () => { + const failure: { current: Error & { code?: string } } = { current: Object.assign(new Error('denied'), { code: 'EACCES' }) } + vi.resetModules() + vi.doMock('node:fs/promises', async (importOriginal) => { + const actual = await importOriginal() + return { ...actual, open: async () => { throw failure.current } } + }) + try { + const isolated = await import('../src/fsio.ts') + await expect(isolated.readWholeTextBounded(localTarget('/virtual/denied'), 3)) + .rejects.toMatchObject({ code: 'FS_PERMISSION_DENIED' }) + failure.current = new Error('open broke') + await expect(isolated.readWholeTextBounded(localTarget('/virtual/broken'), 3)) + .rejects.toMatchObject({ code: 'FS_IO_ERROR' }) + } finally { + vi.doUnmock('node:fs/promises') + vi.resetModules() + } + }) +}) + describe('streamWholeText', () => { it('streams the whole file as decoded text', async () => { const file = join(dir, 'a.txt') diff --git a/packages/fs/fs/README.i18n.yaml b/packages/fs/fs/README.i18n.yaml index ddf57a98d0..4a0e675c0c 100644 --- a/packages/fs/fs/README.i18n.yaml +++ b/packages/fs/fs/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/fs/fs/README.md -README.md: 6c80cc22f6f28e792c3458df3390b241b51d8202 -README.zh.md: 4772d799221efbfff9667cbc8b9e1df6af07dcff +README.md: 67079f795c705ab4c9cfa476e0458be04a48c6c8 +README.zh.md: d812a94fab9f4b7e9d15ff78bd1fea3bcc00c0d9 diff --git a/packages/fs/fs/README.md b/packages/fs/fs/README.md index 6c80cc22f6..67079f795c 100644 --- a/packages/fs/fs/README.md +++ b/packages/fs/fs/README.md @@ -2,20 +2,33 @@ English | [中文](README.zh.md) -The **filesystem provider seam**: an abstract `FileSystem` service (`ctx.fs`) defining the storage primitives a backend provides — resolve a path, stat metadata, no-follow path metadata, read/stream text, list directories, write atomically, and apply a literal edit — without saying HOW. Both mutations take their version guard **optionally**, so `ctx.fs` on its own is a complete, unconstrained text-storage seam. This package also owns the `fs/*` policy event vocabulary the tool dispatches and the policy plugin listens for. +The **filesystem provider seam**: an abstract `FileSystem` service (`ctx.fs`) defining the storage primitives in one execution world — resolve paths, expose canonical process paths and file URIs, test containment, read bounded or streaming text, inspect/list metadata, write atomically, and apply a literal edit — without saying HOW. Both mutations take their version guard **optionally**, so `ctx.fs` on its own is a complete, unconstrained text-storage seam. This package also owns the `fs/*` policy event vocabulary the tool dispatches and the policy plugin listens for. -This package is the provider-seam layer of the [filesystem family](../README.md). The [tool](../tool-fs/README.md), [policy](../fs-policy/README.md), and [local](../fs-local/README.md) and [sandboxed](../fs-sandbox/README.md) backends remain separate consumers and implementations; the capability-seam decisions own the split ([foundation](../../../.agents/notes/implemented/architecture/2026-06-13-capability-seams.md), [filesystem seam](../../../.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.md), [provider split](../../../.agents/notes/implemented/simplification/2026-06-26-fsspec-style-fs-seam.md), [event gate](../../../.agents/notes/implemented/architecture/2026-06-26-file-context-as-event-gate.md)). +This package is the provider-seam layer of the four-layer filesystem stack, split so each concern can evolve (and be swapped) independently (see [the capability-seam Agent Note](../../../.agents/notes/implemented/architecture/2026-06-13-capability-seams.md), [the filesystem capability-seam Agent Note](../../../.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.md), [the split-the-filesystem-seam Agent Note](../../../.agents/notes/implemented/simplification/2026-06-26-fsspec-style-fs-seam.md), and [the file-context event-gate Agent Note](../../../.agents/notes/implemented/architecture/2026-06-26-file-context-as-event-gate.md)): + +| Layer | Package | Role | +|---|---|---| +| tool / executor | `@deepseek-ai/dsh-tool-fs` | model-facing `read`/`write`/`edit` schemas + read windowing + text rendering; reads/writes/edits via `ctx.fs`, dispatches the `fs/*` events | +| policy | `@deepseek-ai/dsh-fs-policy` | observed-state + read-before-edit + version-guarded write/edit, contributed through the `fs/*` event gate (no service) | +| provider seam | `@deepseek-ai/dsh-fs` (this) | `ctx.fs`: execution-world paths, text IO, and atomic mutation primitives (optional version guard); owns the `fs/*` event vocabulary | +| provider | `@deepseek-ai/dsh-fs-local` | the host-filesystem implementation | + +A future sandboxed, virtual, or remote backend implements this interface and the policy/tool layers don't change. ## Service API (`ctx.fs`) -A backend subclasses `FileSystem` and implements eight primitives. +A backend subclasses `FileSystem` and implements twelve primitives. | Member | Semantics | |---|---| | `resolve(path, opts?)` | Resolve a path into a stable `FsTarget` (opaque `targetKey`, `displayPath`). `opts.cwd` is the base a relative `path` resolves against (a caller supplies its session workspace; absolute paths ignore it; omitted ⇒ the backend default), while `opts.signal` aborts a backend round-trip. Async — a remote backend may need I/O. The same file via different paths must yield the same `targetKey`. | +| `processPath(target)` | Return the canonical absolute path that a subprocess in this provider's execution world can open. This is intentionally distinct from opaque `targetKey`. | +| `fileUrl(target)` | Return the canonical `file:` URI in the execution world's platform syntax. The backend, not the host process, owns encoding. | +| `contains(parent, child)` | Test canonical identity/descendant containment without exposing or parsing target keys. Both targets come from this provider. | | `stat(target, signal?)` | Return `FsInfo` metadata (`version`, `type`, optional `size`), or `undefined` when the target is absent. Never content. | | `lstat(path, opts?, signal?)` | Return `FsPathInfo` metadata without following the final path component when it is a symlink. This is path-shaped so consumers can reject repository-owned symlinks before `resolve` follows them into a target. | | `readText(target, signal?)` | Read the whole regular text file as one decoded string. Owns regular-file checks, UTF-8 decoding, binary/NUL rejection (`FS_NOT_TEXT`). | +| `readTextBounded(target, maxBytes, signal?)` | Read one complete regular UTF-8 file through a backend-owned stable operation, rejecting before retaining more than `maxBytes`. Consumers must not emulate this with `stat` then `readText`, which admits growth and replacement races. | | `streamText(target, signal?)` | Stream the same text as decoded chunks for large files (cross-chunk UTF-8 decoding stays here). | | `listDir(target, signal?)` | List direct directory children in stable name order. Returns entry names, entry types, resolved child targets, and cheap metadata (`version`/file `size` when available); never reads file contents. Missing targets throw `FS_NOT_FOUND`, non-directories throw `FS_NOT_DIRECTORY`, permission failures throw `FS_PERMISSION_DENIED`, and other backend I/O failures throw `FS_IO_ERROR`. Broken/disappeared children may be returned as `other` without metadata; child permission/IO failures fail the whole listing with the same structured codes. | | `writeText(target, content, expected?, signal?)` | Atomic create/replace. `expected` is OPTIONAL: omit ⇒ unconditional create-or-overwrite; supply an `FsWriteIntent` (`createIfAbsent`/`replaceIfVersion`) to guard. | @@ -37,10 +50,6 @@ This package declares three events (see the generated [events catalog](../../../ `FsTargetKey` / `FsVersion` are branded opaque ids ([the branded-ids Agent Note](../../../.agents/notes/implemented/architecture/2026-06-20-branded-ids.md)) — consumers must not parse `targetKey` or interpret `version`; only `displayPath` is for model/UI output. `FsWriteIntent` is the explicit GUARDED write intent (`createIfAbsent` creates a missing target and rejects an existing one with `FS_NOT_OBSERVED`; `replaceIfVersion` replaces only at the observed version, else `FS_STALE_VERSION`); omitting it from `writeText` is the third, unconditional state. `FsPathInfo` is the no-follow metadata shape that can report `symlink`, unlike target-level `FsInfo`. Failures throw `FsError` (extends `HarnessError`, [the structured error taxonomy Agent Note](../../../.agents/notes/implemented/architecture/2026-06-11-structured-error-taxonomy.md)) carrying a stable `FsErrorCode` (`FS_NOT_FOUND`, `FS_NOT_DIRECTORY`, `FS_NOT_TEXT`, `FS_NOT_REGULAR_FILE`, `FS_PERMISSION_DENIED`, `FS_IO_ERROR`, `FS_STALE_VERSION`, `FS_NOT_OBSERVED`, `FS_AMBIGUOUS_EDIT`, `FS_EDIT_NOT_FOUND`, `FS_ABORTED`); the tool registry surfaces `{ name, code }` on `isError` results. See `src/types.ts` for the full contracts. -## No IO deadline - -Filesystem primitives accept an optional `AbortSignal` but arm no deadline. Local IO is only best-effort abortable: a timeout cannot force an in-progress `fsync` or `rename` to stop, so a fixed deadline would promise control the backend cannot provide. Process-backed discovery owns its separate timeout contract. - ## Model Experience Indirectly, through `dsh-tool-fs`, which renders provider text and errors as bounded, retained filesystem tool results. @@ -52,6 +61,6 @@ No direct invalidation; the named consumer owns any request-prefix changes. ## Known Limitations and Deferred Work - **Text-only by contract** — backends reject binary/non-UTF-8 content with `FS_NOT_TEXT`; binary-safe operations are a deliberate deferral of [the tool-schemas Agent Note](../../../.agents/notes/implemented/feature/2026-06-17-filesystem-tool-schemas.md). -- **Eight primitives only** — no delete, rename/move, copy, or watch; `listDir` is single-level, with recursion, globbing, pagination, and search out of scope per [the directory-listing Agent Note](../../../.agents/notes/archived/architecture/2026-07-03-filesystem-directory-listing-seam.md). -- **No IO deadline** — cancellation is best-effort at primitive boundaries. +- **Twelve primitives only** — no delete, rename/move, copy, or watch; `listDir` is single-level, with recursion, globbing, pagination, and search out of scope per [the directory-listing Agent Note](../../../.agents/notes/archived/architecture/2026-07-03-filesystem-directory-listing-seam.md). +- **No IO deadline** — the seam arms no timeout; cancellation is a best-effort optional `AbortSignal` per primitive (the deliberate [fs-family stance](../README.md)). - **Resolve-then-operate costs a remote backend two round-trips per tool call** — folding or caching resolution is left to such a backend. diff --git a/packages/fs/fs/README.zh.md b/packages/fs/fs/README.zh.md index 4772d79922..d812a94fab 100644 --- a/packages/fs/fs/README.zh.md +++ b/packages/fs/fs/README.zh.md @@ -2,56 +2,65 @@ [English](README.md) | 中文 -**文件系统提供方 seam**:抽象 `FileSystem` 服务(`ctx.fs`),定义后端提供的存储原语,包括路径解析、stat 元数据、不跟随链接的路径元数据、读取/流式读取文本、列出目录、原子写入和应用字面量编辑,但不规定实现方式。两个变更操作都**可选**接收版本防护,因此 `ctx.fs` 本身就是完整且不受约束的文本存储 seam。本包还拥有由工具分派、策略插件监听的 `fs/*` 策略事件词汇。 +**文件系统提供方 seam**:抽象 `FileSystem` 服务(`ctx.fs`),定义同一个执行世界中的存储原语,包括解析路径、公开规范化进程路径与文件 URI、检查包含关系、有界或流式读取文本、检查/列出元数据、原子写入和应用字面量编辑,但不规定实现方式。两个变更操作都**可选** 接收版本防护,因此 `ctx.fs` 本身就是完整且不受约束的文本存储 seam。本包还拥有由工具分派、政策插件监听的 `fs/*` 政策事件词汇。 -本包是[文件系统家族](../README.md)中的提供方 seam 层。[工具](../tool-fs/README.md)、[策略](../fs-policy/README.md)、[本地](../fs-local/README.md)与[沙箱化](../fs-sandbox/README.md)后端分别作为消费方与实现保持独立;能力 seam 决策负责该拆分([基础](../../../.agents/notes/implemented/architecture/2026-06-13-capability-seams.md)、[文件系统 seam](../../../.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.md)、[提供方拆分](../../../.agents/notes/implemented/simplification/2026-06-26-fsspec-style-fs-seam.md)、[事件门禁](../../../.agents/notes/implemented/architecture/2026-06-26-file-context-as-event-gate.md))。 +本包是四层文件系统栈中的提供方 seam 层;该拆分使每个关注点可以独立演进和替换(见[能力 seam Agent Note](../../../.agents/notes/implemented/architecture/2026-06-13-capability-seams.md)、[文件系统能力 seam Agent Note](../../../.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.md)、[拆分文件系统 seam Agent Note](../../../.agents/notes/implemented/simplification/2026-06-26-fsspec-style-fs-seam.md)和[文件上下文事件门禁 Agent Note](../../../.agents/notes/implemented/architecture/2026-06-26-file-context-as-event-gate.md)): + +| 层 | 包 | 角色 | +|---|---|---| +| 工具/执行器 | `@deepseek-ai/dsh-tool-fs` | 面向模型的 `read`/`write`/`edit` schema、读取窗口和文本渲染;通过 `ctx.fs` 读取/写入/编辑,并分派 `fs/*` 事件 | +| 政策 | `@deepseek-ai/dsh-fs-policy` | 已观察状态、编辑前读取和版本防护的写入/编辑,通过 `fs/*` 事件门禁贡献(无服务) | +| 提供方 seam | `@deepseek-ai/dsh-fs`(本包) | `ctx.fs`:执行世界路径、文本 I/O 与原子变更原语(可选版本防护);拥有 `fs/*` 事件词汇 | +| 提供方 | `@deepseek-ai/dsh-fs-local` | 宿主文件系统实现 | + +未来的沙箱化、虚拟或远程后端只需实现该接口,政策层和工具层无需改变。 ## 服务 API(`ctx.fs`) -后端继承 `FileSystem` 并实现八个原语。 +后端继承 `FileSystem` 并实现十二个原语。 | 成员 | 语义 | |---|---| | `resolve(path, opts?)` | 把路径解析为稳定的 `FsTarget`(不透明 `targetKey`、`displayPath`)。`opts.cwd` 是相对 `path` 解析所依据的基准(调用方提供其会话工作区;绝对路径忽略该值;省略时使用后端默认值),`opts.signal` 则中止后端往返。该方法是异步的,因为远程后端可能需要 I/O。经不同路径到达的同一文件必须产生相同 `targetKey`。 | +| `processPath(target)` | 返回该提供方执行世界中的子进程可以打开的规范化绝对路径。该路径有意与不透明的 `targetKey` 分离。 | +| `fileUrl(target)` | 返回采用执行世界平台语法的规范化 `file:` URI。编码由后端而非宿主进程负责。 | +| `contains(parent, child)` | 在不公开或解析目标 key 的情况下,检查规范化身份相等或后代包含关系。两个目标都来自该提供方。 | | `stat(target, signal?)` | 返回 `FsInfo` 元数据(`version`、`type`、可选 `size`);目标不存在时返回 `undefined`。绝不返回内容。 | -| `lstat(path, opts?, signal?)` | 当最后一个路径组件是符号链接时,不跟随该组件,返回 `FsPathInfo` 元数据。该方法采用路径形态,使消费方能在 `resolve` 跟随仓库自有的符号链接进入目标前拒绝它。 | +| `lstat(path, opts?, signal?)` | 当最后一个路径组件是符号链接时,不跟随该组件,返回 `FsPathInfo` 元数据。该方法采用路径形态,使消费方能在 `resolve` 跟随仓库所有的符号链接进入目标前拒绝它。 | | `readText(target, signal?)` | 把整个普通文本文件读取为一个解码后的字符串。负责普通文件检查、UTF-8 解码和二进制/NUL 拒绝(`FS_NOT_TEXT`)。 | +| `readTextBounded(target, maxBytes, signal?)` | 通过后端自有的稳定操作读取一个完整的普通 UTF-8 文件,在保留超过 `maxBytes` 前拒绝。消费方不得以先 `stat` 再 `readText` 模拟此操作,因为那会容许文件增长与替换竞态。 | | `streamText(target, signal?)` | 为大文件按解码后的分片流式读取相同文本(跨分片 UTF-8 解码仍由此处负责)。 | -| `listDir(target, signal?)` | 按稳定名称顺序列出直接子项。返回条目名称、条目类型、解析后的子目标和低成本元数据(若可用则包括 `version`/文件 `size`);绝不读取文件内容。缺失目标抛出 `FS_NOT_FOUND`,非目录抛出 `FS_NOT_DIRECTORY`,权限失败抛出 `FS_PERMISSION_DENIED`,其他后端 I/O 失败抛出 `FS_IO_ERROR`。损坏/消失的子项可以作为无元数据的 `other` 返回;子项权限/I/O 失败会使用相同结构化代码使整个列出操作失败。 | +| `listDir(target, signal?)` | 按稳定名称顺序列出直接子项。返回条目名称、条目类型、解析后的子目标和低成本元数据(若可用则包括 `version`/文件 `size`);绝不读取文件内容。缺失目标抛出 `FS_NOT_FOUND`,非目录抛出 `FS_NOT_DIRECTORY`,权限失败抛出 `FS_PERMISSION_DENIED`,其他后端 I/O 失败抛出 `FS_IO_ERROR`。损坏/消失的子项可以作为无元数据的 `other` 返回;子项权限/I/O 失败会使用相同结构化代码使整个列表失败。 | | `writeText(target, content, expected?, signal?)` | 原子创建/替换。`expected` 是可选的:省略 ⇒ 无条件创建或覆盖;提供 `FsWriteIntent`(`createIfAbsent`/`replaceIfVersion`)⇒ 添加防护。 | | `editText(target, edit, expected?, signal?)` | 字面量编辑。`expected` 是可选的:省略 ⇒ 无条件编辑当前内容;提供 `{ version }` ⇒ 添加防护,并在匹配之前校验。无论哪种情况,目标缺失都报告 `FS_STALE_VERSION`。应用和写入以原子方式完成,使用同一个变更临界区。 | 无论是否有版本防护,变更都在后端的每目标锁内运行,因此无条件写入/编辑仍是原子的;「无条件」只移除*版本*前置条件,不移除原子性。 -## `fs/*` 策略事件 +## `fs/*` 政策事件 -本包声明三个事件(见已生成的[事件目录](../../../docs/cordis-catalog/events.md)),使发出方(`@deepseek-ai/dsh-tool-fs`)和策略监听器(`@deepseek-ai/dsh-fs-policy`)共享词汇,而无需让发出方依赖策略插件。`fs/write-intent` 和 `fs/edit-intent` 是单槽决策 waterfall(瀑布式事件)(监听器完整决策,绝不调用 `next()`);`fs/observed` 是发后即忘的记录事件。它们只携带 `dsh-fs` 词汇和一个不透明 `object` 参与者,不含面向模型的概念或 agent(智能体)/会话所有者结构。 +本包声明三个事件(见已生成的[事件目录](../../../docs/cordis-catalog/events.md)),使发出方(`@deepseek-ai/dsh-tool-fs`)和政策监听器(`@deepseek-ai/dsh-fs-policy`)共享词汇,而无需让发出方依赖政策插件。`fs/write-intent` 和 `fs/edit-intent` 是单槽决策 waterfall(监听器完整决策,绝不调用 `next()`);`fs/observed` 是发后即忘的记录事件。它们只携带 `dsh-fs` 词汇和一个不透明 `object` 参与者,不含面向模型的概念或 agent(智能体)/会话所有者结构。 -## 提供方 seam,不是策略层 +## 提供方 seam,不是政策层 -`ctx.fs` 有意接近 fsspec 风格的存储原语,比字节级 `cat`/`open` 高半层,因为它会解码文本并拒绝二进制,使策略层绝不接触原始字节。它负责 UTF-8 解码、二进制拒绝、原子写入和字面量编辑临界区。它**不**负责行窗口、编号行、渲染 footer 或已观察状态。已观察状态、编辑前读取和版本防护的写入/编辑属于插件(`@deepseek-ai/dsh-fs-policy`)通过提供可选防护而添加的策略,并非提供方行为,因此沙箱化/远程后端不会继承任何面向模型的观察策略。 +`ctx.fs` 有意接近 fsspec 风格的存储原语,比字节级 `cat`/`open` 高半层,因为它会解码文本并拒绝二进制,使政策层绝不接触原始字节。它负责 UTF-8 解码、二进制拒绝、原子写入和字面量编辑临界区。它**不** 负责行窗口、编号行、渲染 footer 或已观察状态。已观察状态、编辑前读取和版本防护的写入/编辑属于插件(`@deepseek-ai/dsh-fs-policy`)通过提供可选防护而添加的政策,并非提供方行为,因此沙箱化/远程后端不会继承任何面向模型的观察政策。 -`editText` 留在该 seam 上,不由策略层通过读取加写入组合,因为版本防护、字面量匹配和原子重写必须处于同一临界区内,才能正确归因错误并实现一方胜出/一方陈旧的并发;远程后端也可以将其实现为原生比较并编辑操作。 +`editText` 留在该 seam 上,不由政策层通过读取加写入组合,因为版本防护、字面量匹配和原子重写必须处于同一临界区内,才能正确归因错误并实现一方胜出/一方陈旧的并发;远程后端也可以将其实现为原生比较并编辑操作。 ## 词汇 `FsTargetKey` / `FsVersion` 是带品牌的不透明 id(见[品牌 id Agent Note](../../../.agents/notes/implemented/architecture/2026-06-20-branded-ids.md));消费方不得解析 `targetKey` 或解释 `version`,只有 `displayPath` 用于模型/UI 输出。`FsWriteIntent` 是显式的防护写入意图(`createIfAbsent` 创建缺失目标,并以 `FS_NOT_OBSERVED` 拒绝现有目标;`replaceIfVersion` 只在观察版本上替换,否则为 `FS_STALE_VERSION`);从 `writeText` 中省略该值就是第三种无条件状态。`FsPathInfo` 是可报告 `symlink` 的不跟随链接元数据形态,区别于目标级 `FsInfo`。失败会抛出 `FsError`(继承 `HarnessError`;见[结构化错误分类 Agent Note](../../../.agents/notes/implemented/architecture/2026-06-11-structured-error-taxonomy.md)),并携带稳定的 `FsErrorCode`(`FS_NOT_FOUND`、`FS_NOT_DIRECTORY`、`FS_NOT_TEXT`、`FS_NOT_REGULAR_FILE`、`FS_PERMISSION_DENIED`、`FS_IO_ERROR`、`FS_STALE_VERSION`、`FS_NOT_OBSERVED`、`FS_AMBIGUOUS_EDIT`、`FS_EDIT_NOT_FOUND`、`FS_ABORTED`);工具注册表公开 `{ name, code }`,并将其附在 `isError` 结果上。完整契约见 `src/types.ts`。 -## 无 I/O deadline - -文件系统原语接受可选 `AbortSignal`,但不会启动 deadline。本地 I/O 只能尽力取消:超时无法强制进行中的 `fsync` 或 `rename` 停止,因此固定 deadline 会承诺后端无法提供的控制能力。基于进程的发现功能拥有独立的超时契约。 - ## 模型体验 通过 `dsh-tool-fs` 间接产生影响;该消费方把提供方文本和错误渲染为有界且保留的文件系统工具结果。 #### KV Cache 影响 -不会直接使缓存失效;上述消费方负责请求前缀的任何变化。 +不会直接使缓存失效;具名消费方负责请求前缀的任何变化。 -## 已知限制与暂缓事项 +## 已知限制与延期工作 - **契约只支持文本**:后端以 `FS_NOT_TEXT` 拒绝二进制/非 UTF-8 内容;二进制安全操作是[工具 schema Agent Note](../../../.agents/notes/implemented/feature/2026-06-17-filesystem-tool-schemas.md)有意延期的工作。 -- **只有八个原语**:没有删除、重命名/移动、复制或监视;`listDir` 只支持一层,递归、glob、分页和搜索不在范围内,见[目录列出 Agent Note](../../../.agents/notes/archived/architecture/2026-07-03-filesystem-directory-listing-seam.md)。 -- **没有 I/O deadline**:取消只能在原语边界尽力执行。 +- **只有十二个原语**:没有删除、重命名/移动、复制或监视;`listDir` 只支持一层,递归、glob、分页和搜索不在范围内,见[目录列出 Agent Note](../../../.agents/notes/archived/architecture/2026-07-03-filesystem-directory-listing-seam.md)。 +- **没有 I/O deadline**:该 seam 不启动超时;取消只是每个原语上尽力而为的可选 `AbortSignal`(见有意采用的 [fs 能力族立场](../README.md))。 - **先解析后操作使远程后端每次工具调用需要两次往返**:折叠或缓存解析由这种后端自行决定。 diff --git a/packages/fs/fs/src/index.ts b/packages/fs/fs/src/index.ts index b43fa48c4b..8bb20ece18 100644 --- a/packages/fs/fs/src/index.ts +++ b/packages/fs/fs/src/index.ts @@ -1,8 +1,10 @@ /** - * Filesystem text-storage provider seam. Backends own stable target identity, - * text decoding, binary rejection, and atomic mutations. Read windows and - * observed-state policy stay in consumer and policy plugins; `editText` remains - * here so version check, literal match, and rewrite share one critical section. + * Filesystem provider seam for one execution world. Backends own stable target + * identity, process paths and file URIs, containment, stable bounded text + * reads, decoding, binary rejection, and atomic mutations. Read windows and + * observed-state policy stay in consumer and policy plugins; `editText` + * remains here so version check, literal match, and rewrite share one critical + * section. * @module @deepseek-ai/dsh-fs */ @@ -83,7 +85,6 @@ export abstract class FileSystem extends Service { super(ctx, 'fs') } - /** /** * The sandbox mode this backend enforces on mutations BY DEFAULT, or * `undefined` when it does not confine at all — the capability fact the tool @@ -111,6 +112,34 @@ export abstract class FileSystem extends Service { */ abstract resolve(path: string, opts?: { cwd?: string; signal?: AbortSignal }): Promise + /** + * Return the canonical absolute path a subprocess in this filesystem's + * execution world can open. The path is deliberately separate from + * {@link FsTarget.targetKey}: consumers may pass this value to another OS + * capability, but must continue treating the target key as opaque. + * @param target - the resolved target whose process path is required. + * @returns an absolute path in the backend's execution world. + */ + abstract processPath(target: FsTarget): string + + /** + * Return the canonical `file:` URI for a target in this filesystem's + * execution world. Backends own URI encoding because the host platform may + * differ from the execution platform. + * @param target - the resolved target to encode. + * @returns the target's canonical file URI. + */ + abstract fileUrl(target: FsTarget): string + + /** + * Test canonical containment without exposing or parsing backend target + * keys. Both targets must come from this provider. + * @param parent - canonical directory target. + * @param child - canonical candidate target. + * @returns true when `child` is `parent` or a descendant of it. + */ + abstract contains(parent: FsTarget, child: FsTarget): boolean + /** * Return target metadata, or `undefined` when the target does not exist. * @param target - the resolved target to stat. @@ -143,6 +172,19 @@ export abstract class FileSystem extends Service { */ abstract readText(target: FsTarget, signal?: AbortSignal): Promise + /** + * Read one regular UTF-8 text file through a backend-owned stable handle, + * rejecting before more than `maxBytes` are retained. The size check and + * bytes read are one operation: a caller must not emulate this with + * {@link stat} followed by {@link readText}, which admits growth and path + * replacement races between the two calls. + * @param target - the resolved target to read. + * @param maxBytes - positive safe-integer byte ceiling. + * @param signal - aborts the open/read operation. + * @returns the complete decoded text when it fits. + */ + abstract readTextBounded(target: FsTarget, maxBytes: number, signal?: AbortSignal): Promise + /** * Stream the whole regular text file as decoded text chunks (same text * semantics as {@link readText}, for large files). The backend owns diff --git a/packages/fs/fs/tests/service.spec.ts b/packages/fs/fs/tests/service.spec.ts index 19ee033cce..92f1cfc37f 100644 --- a/packages/fs/fs/tests/service.spec.ts +++ b/packages/fs/fs/tests/service.spec.ts @@ -19,13 +19,18 @@ import type { FsWriteOutcome, } from '@deepseek-ai/dsh-fs' -/** A minimal in-memory fake implementing the eight provider primitives. */ +/** A minimal in-memory fake implementing the provider primitives. */ class FakeFileSystem extends FileSystem { files = new Map() override async resolve(path: string): Promise { return { targetKey: FsTargetKey(path), displayPath: path } } + override processPath(target: FsTarget): string { return String(target.targetKey) } + override fileUrl(target: FsTarget): string { return `file:///${encodeURIComponent(String(target.targetKey))}` } + override contains(parent: FsTarget, child: FsTarget): boolean { + return child.targetKey === parent.targetKey || String(child.targetKey).startsWith(`${parent.targetKey}/`) + } override async stat(target: FsTarget): Promise { const content = this.files.get(target.targetKey) if (content === undefined) return undefined @@ -41,6 +46,11 @@ class FakeFileSystem extends FileSystem { if (content === undefined) throw new FsError(`not found: ${target.displayPath}`, 'FS_NOT_FOUND') return content } + override async readTextBounded(target: FsTarget, maxBytes: number): Promise { + const content = await this.readText(target) + if (Buffer.byteLength(content) > maxBytes) throw new FsError('too large', 'FS_IO_ERROR') + return content + } override async streamText(target: FsTarget): Promise> { const content = await this.readText(target) return (async function* () { yield content })() @@ -75,6 +85,7 @@ describe('FileSystem provider seam', () => { const ctx = new Context() await ctx.plugin(FakeFileSystem) const fs = ctx.fs as FakeFileSystem + expect(fs.sandboxMode).toBeUndefined() fs.files.set('a.txt', 'hi') const target = await fs.resolve('a.txt') expect((await fs.stat(target))?.type).toBe('file') diff --git a/packages/fs/tool-fs/tests/tools.spec.ts b/packages/fs/tool-fs/tests/tools.spec.ts index ad01237c2b..f54701246a 100644 --- a/packages/fs/tool-fs/tests/tools.spec.ts +++ b/packages/fs/tool-fs/tests/tools.spec.ts @@ -48,6 +48,11 @@ class FakeFs extends FileSystem { override async resolve(path: string): Promise { return { targetKey: FsTargetKey(`key:${path}`), displayPath: `/abs/${path}` } } + override processPath(target: FsTarget): string { return String(target.targetKey) } + override fileUrl(target: FsTarget): string { return `file://${target.targetKey}` } + override contains(parent: FsTarget, child: FsTarget): boolean { + return child.targetKey === parent.targetKey || String(child.targetKey).startsWith(`${parent.targetKey}/`) + } override async stat(target: FsTarget): Promise { this.throwIfArmed() const content = this.files.get(target.targetKey) @@ -62,6 +67,11 @@ class FakeFs extends FileSystem { override async readText(target: FsTarget): Promise { return this.files.get(target.targetKey) ?? '' } + override async readTextBounded(target: FsTarget, maxBytes: number): Promise { + const content = await this.readText(target) + if (Buffer.byteLength(content) > maxBytes) throw new Error('too large') + return content + } override async streamText(target: FsTarget): Promise> { const content = this.files.get(target.targetKey) ?? '' return (async function* () { yield content })() diff --git a/packages/lsp/README.i18n.yaml b/packages/lsp/README.i18n.yaml index a41b05f148..9919cdaba5 100644 --- a/packages/lsp/README.i18n.yaml +++ b/packages/lsp/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/lsp/README.md -README.md: 4964d78f1096d1a4bc78fa80c6b5febaf24a5661 -README.zh.md: 93b872002cf1f53cdbb96ff42402bfeb9557575f +README.md: 7fbdf071735673fb0158f6fa66148be1c644a433 +README.zh.md: e059dbd80b7e38c0e447e54178162316dfd127c7 diff --git a/packages/lsp/README.md b/packages/lsp/README.md index 4964d78f10..7fbdf07173 100644 --- a/packages/lsp/README.md +++ b/packages/lsp/README.md @@ -6,8 +6,10 @@ The language-server capability seam: an abstract LSP interface, a generic stdio | Package | Role | ctx key | |---|---|---| -| [`lsp/`](lsp/README.md) | LSP provider seam and shared vocabulary | `ctx.lsp` | -| [`lsp-local/`](lsp-local/README.md) | Local stdio language-server backend | registers providers on `ctx.lsp` | -| [`tool-lsp/`](tool-lsp/README.md) | Model-facing semantic-navigation tool | registers on `ctx.tools` | +| `lsp/` | Abstract LSP seam (provider registry by branded id + extension mapping, per-query selection, vocabulary, `LspError`) | `ctx.lsp` | +| `lsp-local/` | Generic multi-server stdio backend over `ctx.fs` and `ctx.subprocess` (JSON-RPC, transient-open queries) | (registers providers on `ctx.lsp`) | +| `tool-lsp/` | Model-facing `lsp` tool (four operations, one-based UTF-16 cursor coordinates) | (registers on `ctx.tools`) | -Providers register semantic capabilities; the tool owns the model-facing contract. The child READMEs document operation, protocol, and presentation details, while the [LSP capability-seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md) owns the rationale. +The interface lives at `lsp/lsp/`. The seam exposes exactly four semantic operations — `goToDefinition`, `findReferences`, `goToImplementation`, `hover` — and no generic JSON-RPC escape hatch, so a provider swap does not change how the model asks for navigation and no protocol payload or unreviewed mutation reaches the model contract. Providers register **capabilities**, not tools; `tool-lsp` is the only owner of the model-facing name, schema, prompt guidance, and presentation. + +See the [LSP capability seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md) for the design rationale, including why documents open transiently per query, why the stdio host consumes the shared filesystem/subprocess execution world, and why extension ownership is exclusive within one runtime. diff --git a/packages/lsp/README.zh.md b/packages/lsp/README.zh.md index 93b872002c..e059dbd80b 100644 --- a/packages/lsp/README.zh.md +++ b/packages/lsp/README.zh.md @@ -2,12 +2,14 @@ [English](README.md) | 中文 -语言服务器能力 seam:抽象 LSP 接口、通用 stdio 提供方和面向模型的 `lsp` 工具。这些全是**产品**包。 +语言服务器能力 seam:抽象 LSP 接口、通用 stdio 提供方,以及面向模型的 `lsp` 工具。这些全是**产品** 包。 | 包 | 职责 | ctx key | |---|---|---| -| [`lsp/`](lsp/README.md) | LSP 提供方 seam 和共享词汇 | `ctx.lsp` | -| [`lsp-local/`](lsp-local/README.md) | 本地 stdio 语言服务器后端 | 在 `ctx.lsp` 上注册提供方 | -| [`tool-lsp/`](tool-lsp/README.md) | 面向模型的语义导航工具 | 注册到 `ctx.tools` | +| `lsp/` | 抽象 LSP seam(按品牌化 id + 扩展名映射组织的提供方注册表、逐查询选择、词汇、`LspError`) | `ctx.lsp` | +| `lsp-local/` | 基于 `ctx.fs` 与 `ctx.subprocess` 的通用多服务器 stdio 后端(JSON-RPC、临时打开查询) | (在 `ctx.lsp` 上注册提供方) | +| `tool-lsp/` | 面向模型的 `lsp` 工具(四种操作、从 1 开始的 UTF-16 光标坐标) | (注册到 `ctx.tools`) | -提供方注册语义能力;工具负责面向模型的契约。子 README 记录操作、协议和呈现细节,[LSP 能力 seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md)负责设计原理。 +接口位于 `lsp/lsp/`。该 seam 恰好公开四种语义操作:`goToDefinition`、`findReferences`、`goToImplementation`、`hover`,且不提供通用 JSON-RPC 逃生口;因此,替换提供方不会改变模型请求导航的方式,也不会让协议载荷或未经评审的修改进入模型契约。提供方注册的是**能力** 而非工具;`tool-lsp` 是面向模型名称、schema、提示词指引和呈现的唯一 owner。 + +设计原理见 [LSP 能力 seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md),其中也解释了文档为何在每次查询时临时打开、stdio 主机为何使用共享的文件系统/子进程执行环境,以及扩展名归属为何在同一运行时内互斥。 diff --git a/packages/lsp/lsp-local/README.i18n.yaml b/packages/lsp/lsp-local/README.i18n.yaml index b786ec1a60..1504e6f726 100644 --- a/packages/lsp/lsp-local/README.i18n.yaml +++ b/packages/lsp/lsp-local/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/lsp/lsp-local/README.md -README.md: 37676a82fb5d45b40ca86507259aca9509d25a43 -README.zh.md: 9e8b7f4f4395985bdbc29c1d911520b3559d7e0c +README.md: 2c5ab309f3557ad77696a881b41d164e65bd24fe +README.zh.md: d4493832964a5ac0d804602c18774bb106ddcbd9 diff --git a/packages/lsp/lsp-local/README.md b/packages/lsp/lsp-local/README.md index 37676a82fb..2c5ab309f3 100644 --- a/packages/lsp/lsp-local/README.md +++ b/packages/lsp/lsp-local/README.md @@ -2,18 +2,19 @@ English | [中文](README.zh.md) -A **generic local stdio language-server backend** for `ctx.lsp`. One plugin instance accepts a named server table and registers one isolated provider per entry. This is a generic host, not a language-server catalog or installer — deployments configure commands and mappings explicitly; presets belong in `cordis.yml` overlays. +A **generic stdio language-server backend** for `ctx.lsp`. One plugin instance accepts a named server table and registers one isolated provider per entry. It reads through `ctx.fs` and launches through `ctx.subprocess`, so the server and source always inhabit the mounted execution world. This is a generic host, not a language-server catalog or installer — deployments configure commands and mappings explicitly; presets belong in `cordis.yml` overlays. Namespace plugin (`name` / `inject` / `Config` / `apply`, no default export). ## What it does - Resolves every server-local setting before registration; an invalid mapping or registration conflict rolls back earlier entries, so a failed load leaves no provider routes. -- Lazily single-flights one server process per `(server id, canonical workspace realpath)`. A live server error is not replayed; if the selected pooled transport fails before or during a read-only query, the provider awaits its disposal and retries that query once on a fresh process. -- Uses a compatibility-first **transient-open** sequence per query: canonicalize and read the source with Node APIs, `textDocument/didOpen` (version 1, full text), the requested request, then `textDocument/didClose` in `finally`. A failed or canceled `didOpen` write terminates the instance before the pool can reuse it. Documents close after each call, so the first version needs no `didChange`, content cache, or document LRU. +- Lazily single-flights one server process per `(server id, canonical workspace target)`. A live server error is not replayed; if the selected pooled transport fails before or during a read-only query, the provider awaits its disposal and retries that query once on a fresh process. +- Uses a compatibility-first **transient-open** sequence per query: resolve and boundedly read the source through `ctx.fs`, `textDocument/didOpen` (version 1, full text), the requested request, then `textDocument/didClose` in `finally`. A failed or canceled `didOpen` write terminates the instance before the pool can reuse it. Documents close after each call, so the first version needs no `didChange`, content cache, or document LRU. - Serializes each source-read/open/query/close lifecycle through one abortable per-workspace queue so queued calls read current source only when their turn starts; distinct workspaces run in parallel. - After protocol shutdown fails, terminates the server's descendant tree through the subprocess seam (POSIX process-group signaling; Windows `taskkill /T /F`). Tree-kill delivery is contained like every group signal — it races server exit — and quiescence is confirmed by the handle's tree-liveness wait rather than by the kill's own outcome. -- Reads sources through Node filesystem APIs in the subprocess's host namespace — NOT `ctx.fs`, and emits no `fs/observed`: only the LSP result is model-visible, so a query does not satisfy read-before-write policy. +- Resolves the server executable, cwd, process, and protocol streams through `ctx.subprocess`; `initialize.processId` is `null` because another machine or PID namespace must not monitor the harness process. +- Uses `ctx.fs` canonical containment, file URIs, and stable bounded reads, but emits no `fs/observed`: only the LSP result is model-visible, so a query does not satisfy read-before-write policy. ## Configuration @@ -23,7 +24,7 @@ The `servers` record key is the stable provider id reserved on `ctx.lsp`; each v |---|---|---| | `command` | (required) | Executable to spawn — absolute, or resolved on the child PATH at load. Launch uses no shell. | | `args` | `[]` | Arguments passed to the executable. | -| `env` | `{}` | Extra env merged on top of the credential-scrubbed ambient env (vars matching `KEY`/`PASSWORD`/`SECRET`/`TOKEN` are not forwarded); an explicit `DSH_*` entry merges after the seam's scrub of ambient ones. | +| `env` | `{}` | Extra env merged on top of the credential-scrubbed ambient env (vars matching `KEY`/`SECRET`/`TOKEN` are not forwarded); an explicit `DSH_*` entry merges after the seam's scrub of ambient ones. | | `extensionToLanguage` | (required) | Lowercase leading-dot extension → LSP language id (e.g. `{ '.ts': 'typescript' }`). | | `initializationOptions` | `null` | Static `initialize` options forwarded to the server. | | `configuration` | `null` | Static answer to every `workspace/configuration` item. | @@ -41,7 +42,7 @@ Initialization advertises `general.positionEncodings: ['utf-16']`, `workspace: { ## Security boundary -The provider trusts its configured server and claims no sandbox confinement. It canonicalizes and reads source through Node APIs, rejecting a source that is missing, non-regular, non-UTF-8, oversized, or whose canonical path resolves outside the canonical workspace (symlink aliases share one instance). Result locations may be external, but an external path cannot become a query source. The first implementation therefore requires trusted host-local deployment; restricted, remote, or virtual workspaces require another provider. +The provider trusts its configured server and claims no sandbox confinement. It delegates canonical identity, containment, no-follow/stable bounded reads, UTF-8 validation, and file-URI encoding to `ctx.fs`; it rejects missing, non-regular, non-UTF-8, oversized, or canonically out-of-workspace query sources before server startup. Result locations may be external, but an external path cannot become a query source. A deployment must mount filesystem and subprocess providers for the same execution world; split-world composition is invalid. ## Model Experience @@ -53,6 +54,7 @@ No direct invalidation; `dsh-tool-lsp` owns request-prefix changes. ## Known Limitations and Deferred Work -- **Trusted host-local only** — no sandbox confinement, no private cache/temp write contract; supporting untrusted binaries or restricted/remote/virtual workspaces requires a later process/filesystem contract and a different provider ([seam Agent Note](../../../.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md)). Containment resolves `realpath`, then opens the source through one handle with `O_NOFOLLOW | O_NONBLOCK` (final-component symlink guard plus nonblocking rejection of FIFOs) and a bounded read; a concurrent mutator that swaps an *ancestor* directory for a symlink between the resolve and the open is an accepted residual TOCTOU under this trusted-deployment model, not closed with non-portable `openat` segment walks. -- **Transient-open compatibility floor** — servers whose synchronization omits open/close (or advertise `None`) are unsupported even if closed-document queries would work; compatibility with one TypeScript server does not imply cross-language support. +- **No confinement policy** — this package trusts the configured server and does not sandbox its process; a restricted deployment must supply appropriate process/filesystem providers or a same-world sandbox wrapper. +- **Execution-world URI rendering** — the stdio host produces provider-owned `file:` URIs. The current model tool renders them with the harness host's path library, so a Windows harness paired with a POSIX remote execution world may show remote locations as URIs or non-native paths; protocol queries remain correct. +- **Transient-open compatibility floor** — servers whose synchronization omits open/close (or advertise `None`) are unsupported even if closed-document queries would work; the pinned TypeScript e2e establishes one compatibility floor, not a cross-language claim. - **Per-server/workspace serialization latency** — parallel agents sharing one server and workspace queue behind one process; long-lived workspace processes consume memory until disposal. diff --git a/packages/lsp/lsp-local/README.zh.md b/packages/lsp/lsp-local/README.zh.md index 9e8b7f4f43..d449383296 100644 --- a/packages/lsp/lsp-local/README.zh.md +++ b/packages/lsp/lsp-local/README.zh.md @@ -2,18 +2,19 @@ [English](README.md) | 中文 -`ctx.lsp` 的**通用本地 stdio 语言服务器后端**。一个插件实例接受一张命名服务器表,并逐配置项注册一个隔离的提供方。这是通用主机,而不是语言服务器目录或安装器:部署需要显式配置命令与映射,预设应放在 `cordis.yml` overlay 中。 +`ctx.lsp` 的**通用 stdio 语言服务器后端**。一个插件实例接受一张命名服务器表,并逐配置项注册一个隔离的提供方。它通过 `ctx.fs` 读取,并通过 `ctx.subprocess` 启动,因此服务器与源文件始终位于所挂载的同一执行环境。这是通用主机,而不是语言服务器目录或安装器:部署需要显式配置命令与映射,preset 应放在 `cordis.yml` overlay 中。 Namespace 插件(`name`/`inject`/`Config`/`apply`,无默认导出)。 ## 功能 - 在注册前解析每项服务器局部设置;无效映射或注册冲突会回滚较早配置项,因此加载失败不会留下提供方路由。 -- 每个 `(server id, canonical workspace realpath)` 惰性 single-flight 一个服务器进程。存活服务器错误不会回放;如果选中的池化传输在只读查询之前或期间失败,提供方会等待其 dispose(资源释放)完成,并在新进程上重试该查询一次。 -- 每次查询都使用兼容性优先的**临时打开**序列:通过 Node API 规范化并读取源文件、`textDocument/didOpen`(版本 1、完整文本)、所请求操作,然后执行 `textDocument/didClose`,该操作位于 `finally` 中。写入 `didOpen` 失败或取消时,会在池复用该实例前将其终止。文档在每次调用后关闭,因此第一版不需要 `didChange`、内容 cache 或文档 LRU。 +- 每个 `(server id, canonical workspace target)` 惰性 single-flight 一个服务器进程。存活服务器错误不会回放;如果选中的池化传输在只读查询之前或期间失败,提供方会等待其释放,并在新进程上重试该查询一次。 +- 每次查询都使用兼容性优先的**临时打开** 序列:通过 `ctx.fs` 解析源文件并进行有界读取、`textDocument/didOpen`(版本 1、完整文本)、所请求操作,然后执行 `textDocument/didClose`,该操作位于 `finally` 中。写入 `didOpen` 失败或取消时,会先终止实例再允许池复用。文档在每次调用后关闭,因此第一版不需要 `didChange`、内容 cache 或文档 LRU。 - 通过一条逐 Workspace、可中止的队列,串行执行每个源读取/打开/查询/关闭生命周期,因此排队调用只会在轮到自身时读取当前源;不同 Workspace 并行运行。 -- 协议 shutdown 失败后,经由子进程 seam 终止服务器后代树(POSIX 进程组信号;Windows `taskkill /T /F`)。树终止的投递结果与所有进程组信号一样被就地吸收,不向外抛出(投递与服务器退出存在竞态);服务器是否完全停稳,由句柄的进程树存活等待确认,而非由这次终止自身的结果确认。 -- 通过子进程 host namespace 中的 Node 文件系统 API 读取源文件,绝不使用 `ctx.fs`,也不发出 `fs/observed`:只有 LSP 结果对模型可见,因此查询不满足先读后写策略。 +- 协议 shutdown 失败后,经由进程管理器 seam 终止服务器后代树(POSIX 进程组信号;Windows `taskkill /T /F`)。树终止的投递结果与所有进程组信号一样被就地吸收,不向外抛出(投递与服务器退出存在竞态);服务器是否完全停稳,由句柄的进程树存活等待确认,而非由这次终止自身的结果确认。 +- 通过 `ctx.subprocess` 解析服务器可执行文件、cwd、进程与协议流;`initialize.processId` 为 `null`,因为另一台机器或 PID 命名空间不得监控 harness 进程。 +- 使用 `ctx.fs` 提供的规范 containment、文件 URI 与稳定有界读取,但不发出 `fs/observed`:只有 LSP 结果对模型可见,因此查询不满足先读后写策略。 ## 配置 @@ -23,13 +24,13 @@ Namespace 插件(`name`/`inject`/`Config`/`apply`,无默认导出) |---|---|---| | `command` | (必填) | 要 spawn 的可执行文件:绝对路径,或在加载时从子进程 PATH 解析。不使用 shell 启动。 | | `args` | `[]` | 传给可执行文件的参数。 | -| `env` | `{}` | 合并到已清理 credential 的环境之上的额外 env(匹配 `KEY`/`PASSWORD`/`SECRET`/`TOKEN` 的变量不会转发);显式 `DSH_*` 条目在 seam 清除环境中同名值之后合并。 | +| `env` | `{}` | 合并到已清理 credential 的环境之上的额外 env(匹配 `KEY`/`SECRET`/`TOKEN` 的变量不会转发);显式 `DSH_*` 条目在 seam 清除环境中同名值之后合并。 | | `extensionToLanguage` | (必填) | 小写、以点开头的扩展名 → LSP language id(例如 `{ '.ts': 'typescript' }`)。 | | `initializationOptions` | `null` | 转发给服务器的静态 `initialize` 选项。 | | `configuration` | `null` | 每个 `workspace/configuration` 配置项的静态答案。 | | `maxMessageBytes` | `16000000` | 从服务器接受的单条 framed 消息最大大小。 | | `maxStderrBytes` | `1000000` | 为诊断保留的 stderr 尾部最大大小。 | -| `maxDocumentBytes` | `4000000` | 该主机可打开的源文件大小上限。 | +| `maxDocumentBytes` | `4000000` | 该主机可打开的最大源文件。 | | `shutdownTimeoutMs` | `5000` | 升级前用于优雅 `shutdown`/`exit` 的预算。 | | `killGraceMs` | `2000` | 请求取消及 SIGTERM→SIGKILL 升级的宽限期。 | @@ -37,11 +38,11 @@ Namespace 插件(`name`/`inject`/`Config`/`apply`,无默认导出) ## 协议行为 -初始化会声明 `general.positionEncodings: ['utf-16']`、`workspace: { workspaceFolders: true, configuration: true }`、`textDocument.hover.contentFormat: ['markdown', 'plaintext']`,以及定义与实现使用的 `linkSupport: true`,且不进行动态注册。服务器返回的能力具有最终决定权:不受支持的操作,或缺少临时打开/关闭的同步方式,会使查询失败。服务器省略 `positionEncoding` 时默认为 `utf-16`;其他值都属于协议错误。客户端通过静态配置回答 `workspace/configuration`,接受生命周期记账请求,并拒绝 `workspace/applyEdit`:它绝不应用编辑或运行命令。导航直接映射 `Location`,并从 `LocationLink` 的 `targetUri` + `targetSelectionRange` 映射;hover 规范化会取得有效的 `MarkupContent.value`,保留 string `MarkedString`,把带 language tag 的值渲染为围栏代码,并用一个空行连接数组。缺失结果、格式错误的范围或位置,以及格式错误的 hover 编码,都会以结构化 `LSP_MALFORMED_RESPONSE` 错误的形式失败。 +初始化会声明 `general.positionEncodings: ['utf-16']`、`workspace: { workspaceFolders: true, configuration: true }`、`textDocument.hover.contentFormat: ['markdown', 'plaintext']`,以及定义与实现使用的 `linkSupport: true`,且不进行动态注册。服务器返回的能力具有最终决定权:不受支持的操作,或缺少临时打开/关闭的同步方式,会使查询失败。服务器省略 `positionEncoding` 时默认为 `utf-16`;其他值都属于协议错误。客户端通过静态配置回答 `workspace/configuration`,接受生命周期记账请求,并拒绝 `workspace/applyEdit`:它绝不应用编辑或运行命令。导航直接映射 `Location`,并从 `LocationLink` 的 `targetUri` + `targetSelectionRange` 映射;hover 规范化会取得有效的 `MarkupContent.value`,保留 string `MarkedString`,把带 language tag 的值渲染为围栏代码,并用一个空行连接数组。缺失结果、格式错误的范围或位置,以及格式错误的 hover 编码,都会作为结构化 `LSP_MALFORMED_RESPONSE` 错误失败。 ## 安全边界 -提供方信任其配置的服务器,不提供任何沙箱隔离。它通过 Node API 规范化并读取源文件,拒绝缺失、非普通文件、非 UTF-8、过大,或规范路径位于规范 Workspace 外部的源文件(符号链接别名共享一个实例)。结果位置可以在外部,但外部路径不能成为查询源。因此,第一版要求可信的主机本地部署;受限、远程或虚拟 Workspace 需要另一个提供方。 +提供方信任其配置的服务器,不声明任何沙箱限制。它把规范身份、containment、不跟随符号链接的稳定有界读取、UTF-8 校验与文件 URI 编码委托给 `ctx.fs`;服务器启动前,系统会拒绝缺失、非普通文件、非 UTF-8、过大或规范路径位于工作区外的查询源。结果位置可以在外部,但外部路径不能成为查询源。部署必须为同一执行环境挂载文件系统与子进程提供方;分裂执行环境的组合无效。 ## 模型体验 @@ -53,6 +54,7 @@ Namespace 插件(`name`/`inject`/`Config`/`apply`,无默认导出) ## 已知限制与暂缓事项 -- **仅限可信主机本地环境**:没有沙箱隔离,也没有私有 cache/temp 写入契约;支持不受信任 binary 或受限/远程/虚拟 Workspace,需要后续的进程/文件系统契约及不同提供方(见 [seam Agent Note](../../../.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md))。限制逻辑先解析 `realpath`,再通过一个带 `O_NOFOLLOW | O_NONBLOCK` 的 handle 打开源文件(最终组件符号链接防护,并以非阻塞方式拒绝 FIFO),同时进行有界读取;并发修改方如果在解析与打开之间把*祖先*目录替换为符号链接,会造成残余 TOCTOU。在该可信部署模型下接受此风险,不使用不可移植的 `openat` 逐 segment 遍历来封闭。 -- **临时打开兼容性下限**:同步能力省略打开/关闭(或声明 `None`)的服务器不受支持,即使关闭文档查询能够工作;与一个 TypeScript 服务器兼容,并不表示支持其他语言。 -- **逐服务器/Workspace 串行化延迟**:共享同一个服务器与 Workspace 的并行 agent(智能体)会在一个进程后排队;长生命周期 Workspace 进程会占用内存直到 dispose。 +- **不提供隔离策略**:这个包(package)信任配置的服务器,不会对其进程执行沙箱化;受限部署必须提供适当的进程/文件系统提供方,或包装同一执行环境的沙箱。 +- **执行环境 URI 渲染**:stdio 主机生成归提供方所有的 `file:` URI。当前面向模型的工具使用 harness 宿主的路径库渲染这些 URI,因此 Windows harness 与 POSIX 远程执行环境配对时,可能把远程位置显示为 URI 或非本机路径;协议查询仍然正确。 +- **临时打开兼容性下限**:同步能力省略打开/关闭(或声明 `None`)的服务器不受支持,即使关闭文档查询能够工作;固定的 TypeScript e2e 只建立一项兼容性下限,不代表跨语言承诺。 +- **逐服务器/Workspace 串行化延迟**:共享同一个服务器与 Workspace 的并行 agent 会在一个进程后排队;长生命周期 Workspace 进程会占用内存直到释放。 diff --git a/packages/lsp/lsp-local/package.json b/packages/lsp/lsp-local/package.json index b68745d617..a5411bab55 100644 --- a/packages/lsp/lsp-local/package.json +++ b/packages/lsp/lsp-local/package.json @@ -26,6 +26,7 @@ "license": "BSD-3-Clause", "peerDependencies": { "@deepseek-ai/dsh-brand": "^0.0.1", + "@deepseek-ai/dsh-fs": "^0.0.1", "@deepseek-ai/dsh-invariants": "^0.0.1", "@deepseek-ai/dsh-llm": "^0.0.1", "@deepseek-ai/dsh-lsp": "^0.0.1", @@ -38,6 +39,8 @@ }, "devDependencies": { "@deepseek-ai/dsh-brand": "workspace:^", + "@deepseek-ai/dsh-fs": "workspace:^", + "@deepseek-ai/dsh-fs-local": "workspace:^", "@deepseek-ai/dsh-invariants": "workspace:^", "@deepseek-ai/dsh-llm": "workspace:^", "@deepseek-ai/dsh-lsp": "workspace:^", diff --git a/packages/lsp/lsp-local/src/connection.ts b/packages/lsp/lsp-local/src/connection.ts index e19718eed6..4be75cbd33 100644 --- a/packages/lsp/lsp-local/src/connection.ts +++ b/packages/lsp/lsp-local/src/connection.ts @@ -22,7 +22,7 @@ export interface ConnectionSpec { readonly args: readonly string[] /** The child's working directory (the canonical workspace). */ readonly cwd: string - /** The child's environment (credential-scrubbed, with overrides applied). */ + /** Explicit child environment overrides; the subprocess provider owns its ambient scrub. */ readonly env: Record /** Largest single framed message accepted from the server. */ readonly maxMessageBytes: number @@ -98,9 +98,8 @@ export class LspConnection { stderr: { maxBytes: spec.maxStderrBytes }, }, graceMs: spec.killGraceMs, - // spec.env mixes the scrubbed base with explicit config entries; the - // seam merges the whole map after its own ambient scrub, so a - // configured DSH_* fact reaches the child. + // The seam merges explicit config entries after its ambient scrub, so a + // configured credential or DSH_* fact reaches the child deliberately. env: spec.env, }) /* v8 ignore start -- 'pipe' dispositions expose both streams by the seam contract; defensive. */ diff --git a/packages/lsp/lsp-local/src/host.ts b/packages/lsp/lsp-local/src/host.ts index 11996908ac..3a2b9fbd7b 100644 --- a/packages/lsp/lsp-local/src/host.ts +++ b/packages/lsp/lsp-local/src/host.ts @@ -1,154 +1,106 @@ -/** - * Host-filesystem source access for the local provider, using Node APIs directly in the - * subprocess's namespace (never `ctx.fs`): only the LSP result is model-visible, so a query does not - * satisfy read-before-write policy and emits no `fs/observed`. Canonicalization derives target - * identity from `realpath`, so symlink aliases share a workspace; a source is rejected before server - * startup when it is missing, non-regular, non-UTF-8, oversized, or canonically outside the - * workspace. External result locations are allowed, but an external path can never become a query - * source. - * @module @deepseek-ai/dsh-lsp-local/host - */ +/** Filesystem-seam source access for the generic stdio LSP provider. */ -import { constants } from 'node:fs' -import { open, realpath, stat } from 'node:fs/promises' -import type { FileHandle } from 'node:fs/promises' -import { isAbsolute, resolve as resolvePath, sep } from 'node:path' +import type { FileSystem, FsTarget } from '@deepseek-ai/dsh-fs' import { throwIfAborted } from './abort.ts' -/** A validated source: its canonical absolute path and current UTF-8 text. */ -export interface HostSource { - /** The canonical (realpath-resolved) absolute path, inside the canonical workspace. */ +/** A canonical workspace in the filesystem/subprocess execution world. */ +export interface HostWorkspace { + /** Stable filesystem identity used for provider pooling. */ + readonly target: FsTarget + /** Canonical absolute path accepted as a subprocess cwd. */ readonly canonicalPath: string - /** The file's current text, read as UTF-8. */ + /** Canonical file URI sent during LSP initialization. */ + readonly fileUrl: string +} + +/** A validated source and the exact URI sent to the language server. */ +export interface HostSource { + /** Canonical file URI in the execution world's platform syntax. */ + readonly fileUrl: string + /** Current complete UTF-8 text. */ readonly text: string } /** - * Canonicalize a workspace root: it must exist and be a directory. The returned realpath supplies - * process cwd, `rootUri`, the sole `workspaceFolders` entry, and pool identity, so symlinked roots - * collapse to one instance. - * @param workspaceRoot - the caller's workspace root (absolute). - * @param signal - optional cancellation observed around each filesystem operation. - * @returns the canonical directory path. - * @throws Error when the path is missing or not a directory. + * Resolve and validate one workspace through `ctx.fs`. + * @param fs - filesystem provider sharing the language server's execution world. + * @param workspaceRoot - caller-supplied workspace path. + * @param signal - optional cancellation around provider operations. + * @returns stable identity plus process path and file URI. */ -export async function canonicalizeWorkspace(workspaceRoot: string, signal?: AbortSignal): Promise { +export async function canonicalizeWorkspace( + fs: FileSystem, + workspaceRoot: string, + signal?: AbortSignal, +): Promise { throwIfAborted(signal) - let canonical: string + let target: FsTarget try { - canonical = await realpath(workspaceRoot) - } catch (error) { - throw new Error(`workspace root "${workspaceRoot}" cannot be resolved: ${messageOf(error)}`) + target = await fs.resolve(workspaceRoot, signal === undefined ? {} : { signal }) + } catch (error: unknown) { + throwIfAborted(signal) + throw new Error(`workspace root "${workspaceRoot}" cannot be resolved: ${messageOf(error)}`, { cause: error }) } throwIfAborted(signal) - const info = await stat(canonical) + const info = await fs.stat(target, signal) throwIfAborted(signal) - if (!info.isDirectory()) { + if (info?.type !== 'directory') { throw new Error(`workspace root "${workspaceRoot}" is not a directory`) } - return canonical + return { + target, + canonicalPath: fs.processPath(target), + fileUrl: fs.fileUrl(target), + } } /** - * Resolve, canonicalize, validate, and read a query source in one pass. A relative `filePath` - * resolves against `canonicalWorkspace`; an absolute one is taken directly. The canonical target - * must be a regular UTF-8 file no larger than `maxDocumentBytes`, and must lie inside the canonical - * workspace. - * @param filePath - the model-supplied source path (relative or absolute). - * @param canonicalWorkspace - the already-canonicalized workspace root. - * @param maxDocumentBytes - the largest source this host will open. - * @param signal - optional cancellation observed throughout resolution, validation, and reading. - * @returns the canonical path and current UTF-8 text. - * @throws Error when the source is missing, non-regular, oversized, non-UTF-8, or out of workspace. + * Resolve, contain, and atomically read one bounded query source through + * `ctx.fs`. The provider's bounded read owns stable-handle and no-follow + * mechanics; this layer owns only LSP-facing validation and messages. + * @param fs - filesystem provider sharing the server's execution world. + * @param filePath - absolute source path or path relative to `workspace`. + * @param workspace - already-canonical workspace. + * @param maxDocumentBytes - largest complete source accepted by this host. + * @param signal - optional cancellation. + * @returns canonical file URI and current text. */ export async function readHostSource( + fs: FileSystem, filePath: string, - canonicalWorkspace: string, + workspace: HostWorkspace, maxDocumentBytes: number, signal?: AbortSignal, ): Promise { throwIfAborted(signal) - const requested = isAbsolute(filePath) ? filePath : resolvePath(canonicalWorkspace, filePath) - let canonicalPath: string + let target: FsTarget try { - canonicalPath = await realpath(requested) - } catch (error) { - throw new Error(`source "${filePath}" cannot be resolved: ${messageOf(error)}`) + target = await fs.resolve(filePath, { + cwd: workspace.canonicalPath, + ...signal === undefined ? {} : { signal }, + }) + } catch (error: unknown) { + throwIfAborted(signal) + throw new Error(`source "${filePath}" cannot be resolved: ${messageOf(error)}`, { cause: error }) } throwIfAborted(signal) - if (!isInside(canonicalWorkspace, canonicalPath)) { + if (!fs.contains(workspace.target, target)) { throw new Error(`source "${filePath}" resolves outside the workspace`) } - // Open ONE handle after containment, then stat and read through it: a concurrent replace between - // realpath and read cannot swap the target, so the regular-file and size checks bind the bytes we - // actually read (no path-based TOCTOU). O_NOFOLLOW rejects the final component being swapped for a - // symlink between realpath and open (which would otherwise escape the workspace). - // O_NONBLOCK prevents a FIFO with no writer from hanging before fstat can reject it as nonregular. - const handle = await open(canonicalPath, constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK) + let text: string try { + text = await fs.readTextBounded(target, maxDocumentBytes, signal) + } catch (error: unknown) { throwIfAborted(signal) - const info = await handle.stat() - throwIfAborted(signal) - if (!info.isFile()) { - throw new Error(`source "${filePath}" is not a regular file`) - } - if (info.size > maxDocumentBytes) { - throw new Error(`source "${filePath}" is ${info.size} bytes, over the ${maxDocumentBytes}-byte limit`) - } - // Bound the read to the cap even if the file grew after stat: read one extra byte and reject on - // overflow, so a concurrent grow cannot defeat the memory bound. - const buffer = await readCapped(handle, maxDocumentBytes, filePath, signal) - const text = decodeUtf8Strict(buffer, filePath) - throwIfAborted(signal) - return { canonicalPath, text } - } finally { - await handle.close() + throw new Error(`source "${filePath}" could not be opened safely: ${messageOf(error)}`, { cause: error }) + } + throwIfAborted(signal) + return { + fileUrl: fs.fileUrl(target), + text, } } -/** Read at most `maxBytes` from the handle, rejecting when the source overflows the cap. */ -async function readCapped( - handle: FileHandle, - maxBytes: number, - filePath: string, - signal?: AbortSignal, -): Promise { - const limit = maxBytes + 1 - const chunk = Buffer.allocUnsafe(limit) - let total = 0 - for (;;) { - throwIfAborted(signal) - const { bytesRead } = await handle.read(chunk, total, limit - total, total) - throwIfAborted(signal) - if (bytesRead === 0) break - total += bytesRead - /* v8 ignore next 3 -- overflow requires the file to grow past the cap between stat and read (a concurrent mutation); defensive. */ - if (total > maxBytes) { - throw new Error(`source "${filePath}" grew past the ${maxBytes}-byte limit while reading`) - } - } - return chunk.subarray(0, total) -} - -/** Whether `child` is the workspace itself or a descendant of it (both already canonical). */ -function isInside(workspace: string, child: string): boolean { - if (child === workspace) return true - /* v8 ignore next -- a canonical non-root workspace never ends with a separator; the guard covers the filesystem root. */ - const base = workspace.endsWith(sep) ? workspace : workspace + sep - return child.startsWith(base) -} - -/** Decode strictly as UTF-8: a fatal decoder rejects only malformed bytes, keeping a legitimate U+FFFD. */ -function decodeUtf8Strict(buffer: Buffer, filePath: string): string { - try { - return new TextDecoder('utf-8', { fatal: true }).decode(buffer) - } catch { - throw new Error(`source "${filePath}" is not valid UTF-8 text`) - } -} - -/** Extract a message from an unknown thrown value without leaking `any`. */ function messageOf(error: unknown): string { - /* v8 ignore next -- Node fs rejections are always Error instances; the String() fallback is defensive. */ return error instanceof Error ? error.message : String(error) } diff --git a/packages/lsp/lsp-local/src/index.ts b/packages/lsp/lsp-local/src/index.ts index d926ae9428..d12e8986fc 100644 --- a/packages/lsp/lsp-local/src/index.ts +++ b/packages/lsp/lsp-local/src/index.ts @@ -1,18 +1,16 @@ /** * Generic stdio language-server backend for `ctx.lsp`. One plugin instance configures a named table * of server commands and registers one isolated provider for each entry. Every provider lazily - * single-flights one server process per canonical workspace realpath, serves transient-open queries + * single-flights one server process per canonical workspace target, serves transient-open queries * through it, and replaces a selected transport that fails before or during the next read-only - * query. Providers read sources through Node APIs in the host namespace (not `ctx.fs`) - * and trust their configured servers — no sandbox confinement. + * query. Providers read sources through `ctx.fs` and launch servers through + * `ctx.subprocess`, so both local and remote implementations share one host. * * Namespace plugin (named exports, no default export). Lifecycle is effect-scoped: disposal * unregisters from `ctx.lsp` and tears down every live server. * @module @deepseek-ai/dsh-lsp-local */ -import { accessSync, constants, statSync } from 'node:fs' -import { delimiter, isAbsolute, join } from 'node:path' import type { Context } from 'cordis' import z from 'schemastery' import { LspError, LspProviderId } from '@deepseek-ai/dsh-lsp' @@ -24,9 +22,9 @@ import type { import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout' import { abortable, abortError } from './abort.ts' import { canonicalizeWorkspace, readHostSource } from './host.ts' +import type { HostWorkspace } from './host.ts' import { LspInstance } from './instance.ts' import type { ConnectionSpawner } from './connection.ts' -import { scrubbedParentEnv } from '@deepseek-ai/dsh-subprocess' import type { InstanceSpec } from './instance.ts' export { canonicalizeWorkspace, readHostSource } from './host.ts' @@ -46,10 +44,7 @@ export { LspConnection } from './connection.ts' export const name = 'lsp-local' /** Services required by this plugin. */ -export const inject = ['lsp', 'subprocess'] - -/** Credential-shaped ambient env vars are NOT forwarded to the child by default. */ - +export const inject = ['fs', 'lsp', 'subprocess'] const DEFAULT_MAX_MESSAGE_BYTES = 16_000_000 const DEFAULT_MAX_STDERR_BYTES = 1_000_000 @@ -91,6 +86,7 @@ export interface Config { /** One server config after schemastery fills every default. */ type ResolvedServerConfig = Required +type WorkspaceKey = HostWorkspace['target']['targetKey'] const LspLocalServerConfig: z = z.object({ command: z.string().required(), @@ -114,23 +110,28 @@ export const Config: z = z.object({ * Register the configured stdio LSP providers. Resolves every executable at load (after credential * scrubbing) before publishing any provider; each process launches lazily on its first matching * query. - * @param ctx - the plugin context (must inject `lsp`). + * @param ctx - the plugin context carrying `fs`, `lsp`, and `subprocess`. * @param config - the resolved plugin configuration (schemastery has filled every default). */ -export function apply(ctx: Context, config: Config): void { +export async function apply(ctx: Context, config: Config): Promise { const entries = Object.entries(config.servers) if (entries.length === 0) throw new Error('lsp-local: servers must contain at least one server') // Resolve every server-local setting before registration so a bad later command or bound cannot // publish an earlier provider. Registry-level mapping conflicts are rolled back below. - const providers = entries.map(([providerId, rawConfig]) => { + const providers = await Promise.all(entries.map(async ([providerId, rawConfig]) => { if (providerId.trim() === '') throw new Error('lsp-local: server ids must be non-empty strings') const resolved = rawConfig as ResolvedServerConfig validateServerConfig(providerId, resolved) - const childEnv = buildChildEnv(resolved.env) - const executable = resolveExecutable(resolved.command, childEnv) - return new LocalLspProvider(providerId, resolved, childEnv, executable, spec => ctx.subprocess.spawn(spec)) - }) + const executable = await ctx.subprocess.resolveExecutable(resolved.command, resolved.env) + return new LocalLspProvider( + providerId, + ctx.fs, + resolved, + executable, + spec => ctx.subprocess.spawn(spec), + ) + })) ctx.effect(() => { const disposers: Array<() => void> = [] @@ -180,16 +181,16 @@ function assertPositiveInteger(providerId: string, name: string, value: number): class LocalLspProvider implements LspProvider { readonly id: LspProviderId readonly extensionToLanguage: Readonly> - /** One live instance per canonical workspace realpath. */ - private readonly instances = new Map() + /** One live instance per stable canonical workspace identity. */ + private readonly instances = new Map() /** One complete source-read→open→query→close serialization tail per canonical workspace. */ - private readonly queues = new Map>() + private readonly queues = new Map>() private disposed = false constructor( providerId: string, + private readonly fs: Context['fs'], private readonly config: ResolvedServerConfig, - private readonly childEnv: Record, private readonly executable: string, private readonly spawner: ConnectionSpawner, ) { @@ -211,19 +212,20 @@ class LocalLspProvider implements LspProvider { } async query(request: LspProviderQuery, signal?: AbortSignal): Promise { - // Honor an already-aborted signal before host I/O so a canceled request never starts a server. + // Honor an already-aborted signal before provider I/O so a canceled request never starts a server. this.assertActive(signal) - const workspace = await canonicalizeWorkspace(request.workspaceRoot, signal) + const workspace = await canonicalizeWorkspace(this.fs, request.workspaceRoot, signal) this.assertActive(signal) - return this.enqueue(workspace, signal, async () => { + const workspaceKey = workspace.target.targetKey + return this.enqueue(workspaceKey, signal, async () => { this.assertActive(signal) // Read inside the workspace queue but before spawning: a queued query sees current bytes when // its turn starts, while an invalid source still cannot leave an idle process pooled. - const source = await readHostSource(request.filePath, workspace, this.config.maxDocumentBytes, signal) + const source = await readHostSource(this.fs, request.filePath, workspace, this.config.maxDocumentBytes, signal) // Disposal may have snapshotted the instance map while host I/O was pending. Re-check before a // synchronous get-or-create so every spawned process remains owned by teardown. this.assertActive(signal) - let instance = this.instanceFor(workspace) + let instance = this.instanceFor(workspaceKey, workspace) try { return await instance.query(request, source, signal) } catch (error) { @@ -231,22 +233,22 @@ class LocalLspProvider implements LspProvider { // read-only, so replace that transport once and retry transparently. if (!instance.isTransportFailure(error)) throw error await instance.dispose() - this.evictIfCurrent(workspace, instance) + this.evictIfCurrent(workspaceKey, instance) this.assertActive(signal) - instance = this.instanceFor(workspace) + instance = this.instanceFor(workspaceKey, workspace) return await instance.query(request, source, signal) } finally { // Reach quiescence before dropping a dead slot; a replacement must survive this ownership check. if (instance.dead) { await instance.dispose() - this.evictIfCurrent(workspace, instance) + this.evictIfCurrent(workspaceKey, instance) } } }) } /** Serialize one complete query lifecycle for a canonical workspace. */ - private enqueue(workspace: string, signal: AbortSignal | undefined, run: () => Promise): Promise { + private enqueue(workspace: WorkspaceKey, signal: AbortSignal | undefined, run: () => Promise): Promise { const previous = this.queues.get(workspace) ?? Promise.resolve() const result = abortable(previous, signal).then(run) // The tail follows the actual prior work even when this caller aborts its wait. It never rejects, @@ -260,27 +262,28 @@ class LocalLspProvider implements LspProvider { } /** Return or synchronously publish the one instance for a canonical workspace. */ - private instanceFor(workspace: string): LspInstance { + private instanceFor(workspaceKey: WorkspaceKey, workspace: HostWorkspace): LspInstance { this.assertActive() - const existing = this.instances.get(workspace) + const existing = this.instances.get(workspaceKey) if (existing !== undefined) return existing const created = this.createInstance(workspace) - this.instances.set(workspace, created) + this.instances.set(workspaceKey, created) return created } /** Drop the slot iff it still contains this instance. */ - private evictIfCurrent(workspace: string, instance: LspInstance): void { + private evictIfCurrent(workspace: WorkspaceKey, instance: LspInstance): void { /* v8 ignore next -- mismatch requires another query to replace the slot before this finally runs. */ if (this.instances.get(workspace) === instance) this.instances.delete(workspace) } - private createInstance(workspace: string): LspInstance { + private createInstance(workspace: HostWorkspace): LspInstance { const spec: InstanceSpec = { command: this.executable, args: this.config.args, - cwd: workspace, - env: this.childEnv, + cwd: workspace.canonicalPath, + workspaceUri: workspace.fileUrl, + env: this.config.env, configuration: this.config.configuration, initializationOptions: this.config.initializationOptions, maxMessageBytes: this.config.maxMessageBytes, @@ -304,41 +307,3 @@ class LocalLspProvider implements LspProvider { this.queues.clear() } } - -/** The seam's scrubbed parent env (credential-shaped and DSH_* names dropped), plus the config's explicit env. */ -function buildChildEnv(extra: Record): Record { - return { ...scrubbedParentEnv(), ...extra } -} - -/** - * Resolve the server executable to an absolute path: an absolute command is verified directly; a - * bare command is looked up on the child's PATH. Fails loudly when nothing is executable. - */ -function resolveExecutable(command: string, childEnv: Record): string { - if (isAbsolute(command)) { - // Verify an absolute command too, so an unavailable one fails at load, not on the first query. - if (!isExecutableFileSync(command)) { - throw new Error(`lsp-local: command "${command}" is not an executable file`) - } - return command - } - /* v8 ignore next -- buildChildEnv always sets PATH from the ambient env; the further fallbacks are defensive. */ - const pathValue = childEnv.PATH ?? process.env.PATH ?? '' - for (const dir of pathValue.split(delimiter)) { - if (dir === '') continue - const candidate = join(dir, command) - if (isExecutableFileSync(candidate)) return candidate - } - throw new Error(`lsp-local: command "${command}" was not found on PATH`) -} - -/** Synchronous regular-file and executable check used only at load-time resolution. */ -function isExecutableFileSync(path: string): boolean { - try { - if (!statSync(path).isFile()) return false - accessSync(path, constants.X_OK) - return true - } catch { - return false - } -} diff --git a/packages/lsp/lsp-local/src/instance.ts b/packages/lsp/lsp-local/src/instance.ts index c1f78eaa38..8584f64f48 100644 --- a/packages/lsp/lsp-local/src/instance.ts +++ b/packages/lsp/lsp-local/src/instance.ts @@ -7,7 +7,6 @@ * @module @deepseek-ai/dsh-lsp-local/instance */ -import { pathToFileURL } from 'node:url' import { LspError } from '@deepseek-ai/dsh-lsp' import type { LspOperation, @@ -31,6 +30,8 @@ import { /** Everything an instance needs beyond the connection spec. */ export interface InstanceSpec extends ConnectionSpec { + /** Canonical workspace file URI supplied by the filesystem provider. */ + readonly workspaceUri: string /** Static `initialize` options forwarded to the server. */ readonly initializationOptions: unknown /** Graceful `shutdown`/`exit` budget before escalation (ms). */ @@ -108,9 +109,11 @@ export class LspInstance { private async initialize(): Promise { const initializeResult = await this.connection.request('initialize', { - processId: process.pid, - rootUri: pathToFileURL(this.spec.cwd).href, - workspaceFolders: [{ uri: pathToFileURL(this.spec.cwd).href, name: 'workspace' }], + // A subprocess provider may run in another PID namespace or machine; + // the host PID would let the server monitor an unrelated process. + processId: null, + rootUri: this.spec.workspaceUri, + workspaceFolders: [{ uri: this.spec.workspaceUri, name: 'workspace' }], capabilities: CLIENT_CAPABILITIES, initializationOptions: this.spec.initializationOptions, }) as WireInitializeResult @@ -147,7 +150,7 @@ export class LspInstance { throw new LspError('server does not support the transient textDocument/didOpen this host requires', 'LSP_UNSUPPORTED_OPERATION') } - const uri = pathToFileURL(source.canonicalPath).href + const uri = source.fileUrl let opened = false try { /* v8 ignore next -- guards an abort landing between the ready wait and didOpen; not deterministically reproducible. */ diff --git a/packages/lsp/lsp-local/tests/built-lib.e2e.ts b/packages/lsp/lsp-local/tests/built-lib.e2e.ts index bdd0ec1067..75f9f227c3 100644 --- a/packages/lsp/lsp-local/tests/built-lib.e2e.ts +++ b/packages/lsp/lsp-local/tests/built-lib.e2e.ts @@ -16,8 +16,9 @@ import { afterAll, beforeAll, describe, expect, it } from 'vitest' const pkgDir = fileURLToPath(new URL('..', import.meta.url)) const seamLib = join(pkgDir, '../lsp/lib/index.js') +const fsLib = join(pkgDir, '../../fs/fs-local/lib/index.js') const subprocessLib = join(pkgDir, '../../subprocess/subprocess-local/lib/index.js') -const built = existsSync(join(pkgDir, 'lib/index.js')) && existsSync(seamLib) && existsSync(subprocessLib) +const built = existsSync(join(pkgDir, 'lib/index.js')) && existsSync(seamLib) && existsSync(fsLib) && existsSync(subprocessLib) const fixtureServer = fileURLToPath(new URL('./fixture-server.ts', import.meta.url)) @@ -42,10 +43,12 @@ describe.skipIf(!built)('built lib real load path (plain node)', () => { const { Context } = await import('cordis') const { default: Lsp } = await import('@deepseek-ai/dsh-lsp') const LspLocal = await import('@deepseek-ai/dsh-lsp-local') + const { default: LocalFileSystem } = await import('@deepseek-ai/dsh-fs-local') const { default: LocalSubprocessService } = await import('@deepseek-ai/dsh-subprocess-local') const ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) await ctx.plugin(LspLocal, { servers: { fake: { diff --git a/packages/lsp/lsp-local/tests/host.spec.ts b/packages/lsp/lsp-local/tests/host.spec.ts index 26aacdc1f4..cd7ed87e17 100644 --- a/packages/lsp/lsp-local/tests/host.spec.ts +++ b/packages/lsp/lsp-local/tests/host.spec.ts @@ -4,7 +4,10 @@ import { tmpdir } from 'node:os' import { join } from 'node:path' import { realpath } from 'node:fs/promises' import { execFile } from 'node:child_process' +import { pathToFileURL } from 'node:url' import { promisify } from 'node:util' +import { Context } from 'cordis' +import LocalFileSystem from '@deepseek-ai/dsh-fs-local' import { deadline } from '@deepseek-ai/dsh-timeout' import { canonicalizeWorkspace, readHostSource } from '@deepseek-ai/dsh-lsp-local' @@ -12,84 +15,105 @@ const execFileAsync = promisify(execFile) let root: string let ws: string +let ctx: Context +let fs: LocalFileSystem beforeEach(async () => { root = await realpath(await mkdtemp(join(tmpdir(), 'lsp-host-'))) ws = join(root, 'ws') await mkdir(ws) + ctx = new Context() + await ctx.plugin(LocalFileSystem, { cwd: root }) + fs = ctx.fs as LocalFileSystem }) afterEach(async () => { + await ctx.fiber.dispose() await rm(root, { recursive: true, force: true }) }) const BIG = 1_000_000 +async function workspace() { + return await canonicalizeWorkspace(fs, ws) +} + +async function readSource(filePath: string, maxBytes = BIG, signal?: AbortSignal) { + return await readHostSource(fs, filePath, await workspace(), maxBytes, signal) +} + describe('canonicalizeWorkspace', () => { it('returns the realpath of a directory', async () => { - expect(await canonicalizeWorkspace(ws)).toBe(ws) + expect((await workspace()).canonicalPath).toBe(ws) }) it('resolves a symlinked workspace to its target so aliases share identity', async () => { const link = join(root, 'ws-link') await symlink(ws, link) - expect(await canonicalizeWorkspace(link)).toBe(ws) + expect((await canonicalizeWorkspace(fs, link)).canonicalPath).toBe(ws) }) it('rejects a missing workspace', async () => { - await expect(canonicalizeWorkspace(join(root, 'nope'))).rejects.toThrow(/cannot be resolved/) + await expect(canonicalizeWorkspace(fs, join(root, 'nope'))).rejects.toThrow(/not a directory/) }) it('rejects a non-directory workspace', async () => { const file = join(root, 'file.txt') await writeFile(file, 'x') - await expect(canonicalizeWorkspace(file)).rejects.toThrow(/not a directory/) + await expect(canonicalizeWorkspace(fs, file)).rejects.toThrow(/not a directory/) }) }) describe('readHostSource', () => { it('reads a relative path against the workspace', async () => { await writeFile(join(ws, 'a.ts'), 'const x = 1\n') - const source = await readHostSource('a.ts', ws, BIG) - expect(source.canonicalPath).toBe(join(ws, 'a.ts')) + const source = await readSource('a.ts') + expect(source.fileUrl).toBe(pathToFileURL(join(ws, 'a.ts')).href) expect(source.text).toBe('const x = 1\n') }) it('reads an absolute path inside the workspace', async () => { const abs = join(ws, 'b.ts') await writeFile(abs, 'b') - const source = await readHostSource(abs, ws, BIG) - expect(source.canonicalPath).toBe(abs) + const source = await readSource(abs) + expect(source.fileUrl).toBe(pathToFileURL(abs).href) }) it('accepts a source reached through a symlink that stays inside the workspace', async () => { await mkdir(join(ws, 'real')) await writeFile(join(ws, 'real', 'c.ts'), 'c') await symlink(join(ws, 'real'), join(ws, 'linked')) - const source = await readHostSource('linked/c.ts', ws, BIG) - expect(source.canonicalPath).toBe(join(ws, 'real', 'c.ts')) + const source = await readSource('linked/c.ts') + expect(source.fileUrl).toBe(pathToFileURL(join(ws, 'real', 'c.ts')).href) }) it('rejects a source whose canonical path escapes the workspace via symlink', async () => { const outside = join(root, 'outside.ts') await writeFile(outside, 'secret') await symlink(outside, join(ws, 'escape.ts')) - await expect(readHostSource('escape.ts', ws, BIG)).rejects.toThrow(/outside the workspace/) + await expect(readSource('escape.ts')).rejects.toThrow(/outside the workspace/) }) it('rejects an absolute source outside the workspace', async () => { const outside = join(root, 'out.ts') await writeFile(outside, 'x') - await expect(readHostSource(outside, ws, BIG)).rejects.toThrow(/outside the workspace/) + await expect(readSource(outside)).rejects.toThrow(/outside the workspace/) }) it('rejects a missing source', async () => { - await expect(readHostSource('nope.ts', ws, BIG)).rejects.toThrow(/cannot be resolved/) + await expect(readSource('nope.ts')).rejects.toThrow(/not found/) + }) + + it('wraps a provider failure while resolving the source', async () => { + const canonical = await workspace() + fs.resolve = async () => { throw 'raw resolve failure' } + await expect(readHostSource(fs, 'broken.ts', canonical, BIG)) + .rejects.toThrow('source "broken.ts" cannot be resolved: raw resolve failure') }) it('rejects a non-regular source (directory)', async () => { await mkdir(join(ws, 'dir')) - await expect(readHostSource('dir', ws, BIG)).rejects.toThrow(/not a regular file/) + await expect(readSource('dir')).rejects.toThrow(/not a regular file/) }) // Windows has no filesystem FIFO; the directory case above pins non-regular rejection there. @@ -97,36 +121,36 @@ describe('readHostSource', () => { const fifo = join(ws, 'pipe.ts') await execFileAsync('mkfifo', [fifo]) using d = deadline(undefined, 1000, 'FIFO_READ_TIMEOUT') - await expect(readHostSource('pipe.ts', ws, BIG, d.signal)).rejects.toThrow(/not a regular file/) + await expect(readSource('pipe.ts', BIG, d.signal)).rejects.toThrow(/not a regular file/) }) it('honors a pre-aborted source read before filesystem work', async () => { const controller = new AbortController() controller.abort(new Error('source read cancelled')) - await expect(readHostSource('missing.ts', ws, BIG, controller.signal)).rejects.toThrow(/source read cancelled/) + await expect(readSource('missing.ts', BIG, controller.signal)).rejects.toThrow(/source read cancelled/) }) it('treats the workspace root itself as inside, then rejects it as non-regular', async () => { - // filePath '.' canonicalizes to the workspace dir: isInside's identity branch is taken, and the - // directory then fails the regular-file check. - await expect(readHostSource('.', ws, BIG)).rejects.toThrow(/not a regular file/) + // The filesystem containment primitive accepts the workspace itself; the + // bounded read then rejects the directory as non-regular. + await expect(readSource('.')).rejects.toThrow(/not a regular file/) }) it('rejects an oversized source', async () => { await writeFile(join(ws, 'big.ts'), 'x'.repeat(100)) - await expect(readHostSource('big.ts', ws, 10)).rejects.toThrow(/over the 10-byte limit/) + await expect(readSource('big.ts', 10)).rejects.toThrow(/10-byte limit/) }) it('rejects a non-UTF-8 source', async () => { await writeFile(join(ws, 'bin.ts'), Buffer.from([0xff, 0xfe, 0x00])) - await expect(readHostSource('bin.ts', ws, BIG)).rejects.toThrow(/not valid UTF-8/) + await expect(readSource('bin.ts')).rejects.toThrow(/invalid UTF-8|binary file/) }) it('keeps a valid U+FFFD replacement character in otherwise-valid UTF-8', async () => { // The literal replacement char is valid UTF-8; a fatal decoder must accept it (only malformed // byte sequences are rejected). await writeFile(join(ws, 'repl.ts'), 'const s = "�"\n') - const source = await readHostSource('repl.ts', ws, BIG) + const source = await readSource('repl.ts') expect(source.text).toBe('const s = "�"\n') }) }) diff --git a/packages/lsp/lsp-local/tests/instance.spec.ts b/packages/lsp/lsp-local/tests/instance.spec.ts index 75b9e541fc..3027cbbd29 100644 --- a/packages/lsp/lsp-local/tests/instance.spec.ts +++ b/packages/lsp/lsp-local/tests/instance.spec.ts @@ -3,6 +3,8 @@ import { mkdtemp, mkdir, readFile, rm, writeFile, realpath } from 'node:fs/promi import { tmpdir } from 'node:os' import { join } from 'node:path' import { pathToFileURL, fileURLToPath } from 'node:url' +import { Context } from 'cordis' +import LocalFileSystem from '@deepseek-ai/dsh-fs-local' import { LspInstance, readHostSource } from '@deepseek-ai/dsh-lsp-local' import { encodeMessage } from '@deepseek-ai/dsh-lsp-local' import type { ConnectionWriter } from '@deepseek-ai/dsh-lsp-local/src/connection.ts' @@ -15,6 +17,8 @@ const fixtureServer = fileURLToPath(new URL('./fixture-server.ts', import.meta.u let root: string let ws: string +let ctx: Context +let fs: LocalFileSystem let live: LspInstance[] = [] beforeEach(async () => { @@ -22,11 +26,15 @@ beforeEach(async () => { ws = join(root, 'ws') await mkdir(ws) await writeFile(join(ws, 'a.ts'), 'const x = 1\n') + ctx = new Context() + await ctx.plugin(LocalFileSystem, { cwd: root }) + fs = ctx.fs as LocalFileSystem }) afterEach(async () => { for (const instance of live) await instance.dispose() live = [] + await ctx.fiber.dispose() await rm(root, { recursive: true, force: true }) }) @@ -39,6 +47,7 @@ function makeInstance( command: process.execPath, args: [fixtureServer], cwd: ws, + workspaceUri: pathToFileURL(ws).href, env: { ...scrubbedParentEnv(), ...env }, configuration: { setting: 42 }, initializationOptions: { init: true }, @@ -58,7 +67,12 @@ function query(operation: LspProviderQuery['operation'] = 'goToDefinition'): Lsp /** Run a query against an instance, reading the source first the way the provider does. */ async function run(instance: LspInstance, operation: LspProviderQuery['operation'] = 'goToDefinition', signal?: AbortSignal): Promise { - const source = await readHostSource('a.ts', ws, 4_000_000) + const workspace = { + target: await fs.resolve(ws), + canonicalPath: ws, + fileUrl: pathToFileURL(ws).href, + } + const source = await readHostSource(fs, 'a.ts', workspace, 4_000_000) return instance.query(query(operation), source, signal) } @@ -68,6 +82,7 @@ function scriptInstance(script: string, overrides: Partial = {}): command: process.execPath, args: ['-e', script], cwd: ws, + workspaceUri: pathToFileURL(ws).href, env: scrubbedParentEnv(), configuration: null, initializationOptions: null, diff --git a/packages/lsp/lsp-local/tests/lifecycle.spec.ts b/packages/lsp/lsp-local/tests/lifecycle.spec.ts index ccd85a1e14..a7a006c06e 100644 --- a/packages/lsp/lsp-local/tests/lifecycle.spec.ts +++ b/packages/lsp/lsp-local/tests/lifecycle.spec.ts @@ -8,6 +8,7 @@ import { Context } from 'cordis' import Lsp, { type LspProvider, type LspQueryRequest, type LspQueryResult } from '@deepseek-ai/dsh-lsp' import { deadline } from '@deepseek-ai/dsh-timeout' import LocalSubprocessService from '@deepseek-ai/dsh-subprocess-local' +import LocalFileSystem from '@deepseek-ai/dsh-fs-local' import * as LspLocal from '@deepseek-ai/dsh-lsp-local' import type { LspLocalServerConfig } from '@deepseek-ai/dsh-lsp-local' @@ -47,6 +48,7 @@ async function mount( const ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) const register = ctx.lsp.registerProvider.bind(ctx.lsp) const registrationSpy = captureProvider === undefined ? undefined @@ -79,6 +81,7 @@ describe('lsp-local end to end over a fake server', () => { const ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) await ctx.plugin(LspLocal, { servers: { typescript: fakeServer({ LSP_FAKE_HOVER: JSON.stringify({ contents: 'ts' }) }), @@ -322,6 +325,7 @@ describe('lsp-local end to end over a fake server', () => { const ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) await expect(ctx.plugin(LspLocal, { servers: { missing: { diff --git a/packages/lsp/lsp-local/tests/provider.spec.ts b/packages/lsp/lsp-local/tests/provider.spec.ts index 77fcbe7851..b0209e09b8 100644 --- a/packages/lsp/lsp-local/tests/provider.spec.ts +++ b/packages/lsp/lsp-local/tests/provider.spec.ts @@ -4,6 +4,7 @@ import { tmpdir } from 'node:os' import { delimiter, join } from 'node:path' import { Context } from 'cordis' import LocalSubprocessService from '@deepseek-ai/dsh-subprocess-local' +import LocalFileSystem from '@deepseek-ai/dsh-fs-local' import Lsp, { type LspQueryRequest } from '@deepseek-ai/dsh-lsp' import * as LspLocal from '@deepseek-ai/dsh-lsp-local' import type { Config, LspLocalServerConfig } from '@deepseek-ai/dsh-lsp-local' @@ -44,6 +45,7 @@ describe('lsp-local provider resolution', () => { const ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) await expect(ctx.plugin(LspLocal, config('onpath', { command: 'fake-lsp', args: [], @@ -57,6 +59,7 @@ describe('lsp-local provider resolution', () => { const ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) await expect(ctx.plugin(LspLocal, config('nope', { command: 'fake-lsp', args: [], @@ -71,6 +74,7 @@ describe('lsp-local provider resolution', () => { const ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) // Grab the provider instance by registering, then dispose the whole plugin fiber. const lsp = ctx.lsp const fiber = await ctx.plugin(LspLocal, config('disp', { @@ -88,6 +92,7 @@ describe('lsp-local provider resolution', () => { const ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) await expect(ctx.plugin(LspLocal, config('bad-budget', { command: process.execPath, args: ['-e', ''], @@ -101,6 +106,7 @@ describe('lsp-local provider resolution', () => { const ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) await expect(ctx.plugin(LspLocal, config('bad-cap', { command: process.execPath, args: ['-e', ''], @@ -114,6 +120,7 @@ describe('lsp-local provider resolution', () => { const ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) await expect(ctx.plugin(LspLocal, config('bad-timer', { command: process.execPath, args: ['-e', ''], @@ -130,6 +137,7 @@ describe('lsp-local provider resolution', () => { const ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) await expect(ctx.plugin(LspLocal, config('abs-bad', { command: notExe, args: [], @@ -142,6 +150,7 @@ describe('lsp-local provider resolution', () => { const ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) await expect(ctx.plugin(LspLocal, config('abs-directory', { command: ws, args: [], @@ -154,6 +163,7 @@ describe('lsp-local provider resolution', () => { const ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) await expect(ctx.plugin(LspLocal, { servers: {} })).rejects.toThrow(/servers must contain at least one server/) await ctx.fiber.dispose() }) @@ -162,6 +172,7 @@ describe('lsp-local provider resolution', () => { const ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) await expect(ctx.plugin(LspLocal, config('', { command: process.execPath, extensionToLanguage: { '.ts': 'typescript' }, @@ -173,6 +184,7 @@ describe('lsp-local provider resolution', () => { const ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) await expect(ctx.plugin(LspLocal, { servers: { valid: { command: process.execPath, extensionToLanguage: { '.ts': 'typescript' } }, @@ -187,6 +199,7 @@ describe('lsp-local provider resolution', () => { const ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) await expect(ctx.plugin(LspLocal, { servers: { first: { command: process.execPath, extensionToLanguage: { '.ts': 'typescript' } }, diff --git a/packages/lsp/lsp-local/tests/typescript-server.e2e.ts b/packages/lsp/lsp-local/tests/typescript-server.e2e.ts index fe9230ce14..9bdb7d1fd4 100644 --- a/packages/lsp/lsp-local/tests/typescript-server.e2e.ts +++ b/packages/lsp/lsp-local/tests/typescript-server.e2e.ts @@ -11,6 +11,7 @@ import { tmpdir } from 'node:os' import { join } from 'node:path' import { Context } from 'cordis' import LocalSubprocessService from '@deepseek-ai/dsh-subprocess-local' +import LocalFileSystem from '@deepseek-ai/dsh-fs-local' import Lsp, { type LspQueryRequest, type LspQueryResult } from '@deepseek-ai/dsh-lsp' import * as LspLocal from '@deepseek-ai/dsh-lsp-local' @@ -54,6 +55,7 @@ beforeAll(async () => { ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) await ctx.plugin(LspLocal, { servers: { typescript: { diff --git a/packages/lsp/lsp-local/tsconfig.json b/packages/lsp/lsp-local/tsconfig.json index 2b106ddc81..04fdeb9a84 100644 --- a/packages/lsp/lsp-local/tsconfig.json +++ b/packages/lsp/lsp-local/tsconfig.json @@ -26,6 +26,9 @@ { "path": "../../llm/llm" }, + { + "path": "../../fs/fs" + }, { "path": "../lsp" }, diff --git a/packages/lsp/tool-lsp/package.json b/packages/lsp/tool-lsp/package.json index 28b5a34bdb..f757df35fd 100644 --- a/packages/lsp/tool-lsp/package.json +++ b/packages/lsp/tool-lsp/package.json @@ -38,11 +38,13 @@ }, "devDependencies": { "@deepseek-ai/dsh-agent": "workspace:^", + "@deepseek-ai/dsh-fs-local": "workspace:^", "@deepseek-ai/dsh-invariants": "workspace:^", "@deepseek-ai/dsh-llm": "workspace:^", "@deepseek-ai/dsh-lsp": "workspace:^", "@deepseek-ai/dsh-lsp-local": "workspace:^", "@deepseek-ai/dsh-session": "workspace:^", + "@deepseek-ai/dsh-subprocess-local": "workspace:^", "@deepseek-ai/dsh-system-prompt": "workspace:^", "@deepseek-ai/dsh-timeout": "workspace:^", "@deepseek-ai/dsh-timeout-policy": "workspace:^", diff --git a/packages/lsp/tool-lsp/tests/integration.spec.ts b/packages/lsp/tool-lsp/tests/integration.spec.ts index 4d790cbd6b..8312a6e3ab 100644 --- a/packages/lsp/tool-lsp/tests/integration.spec.ts +++ b/packages/lsp/tool-lsp/tests/integration.spec.ts @@ -6,6 +6,7 @@ import { pathToFileURL } from 'node:url' import { Context } from 'cordis' import SystemPrompt from '@deepseek-ai/dsh-system-prompt' import ToolRegistry from '@deepseek-ai/dsh-tools' +import LocalFileSystem from '@deepseek-ai/dsh-fs-local' import LocalSubprocessService from '@deepseek-ai/dsh-subprocess-local' import Lsp from '@deepseek-ai/dsh-lsp' import * as LspLocal from '@deepseek-ai/dsh-lsp-local' @@ -50,6 +51,7 @@ async function mount(hang: boolean, timeoutMs?: number): Promise { await ctx.plugin(SystemPrompt) await ctx.plugin(ToolRegistry) await ctx.plugin(Lsp) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) await ctx.plugin(LocalSubprocessService) await ctx.plugin(LspLocal, { servers: { diff --git a/packages/pty/README.i18n.yaml b/packages/pty/README.i18n.yaml index 083a144b55..65d541a7c1 100644 --- a/packages/pty/README.i18n.yaml +++ b/packages/pty/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/pty/README.md -README.md: 9c8206464d45b1be1d6ee3861c57c128e77686c5 -README.zh.md: 70d081e60a7db61443ed616b64586a93c119a640 +README.md: a4f743056b4a524be9623b0f700f37e0534b463f +README.zh.md: c84ad3f1b59afcdbbd111f1b82c57c56aa24fdcf diff --git a/packages/pty/README.md b/packages/pty/README.md index 9c8206464d..a4f743056b 100644 --- a/packages/pty/README.md +++ b/packages/pty/README.md @@ -2,13 +2,12 @@ English | [中文](README.zh.md) -This family provides persistent, owner-scoped pseudo-terminal sessions for interactive or stateful terminal work. It complements one-shot bash execution. +`PTY` stands for **Pseudo-Terminal**(伪终端). This capability provides persistent, owner-scoped terminal sessions for workflows that require state across tool calls or interactive stdin. PTY complements the one-shot bash and filesystem tools; it does not replace their stronger per-operation contracts. | Package | Role | ctx key | |---|---|---| -| [`pty/`](pty/README.md) | Defines the PTY service and session lifecycle | `ctx.pty` | -| [`pty-local/`](pty-local/README.md) | Provides local persistent terminal sessions | registers on `ctx.pty` | -| [`tool-pty/`](tool-pty/README.md) | Exposes PTY session operations to the model | registers on `ctx.tools` | -| [`tool-bash-persistent/`](tool-bash-persistent/README.md) | Exposes a reusable PTY-backed bash tool | registers on `ctx.tools` | +| [`pty`](pty/README.md) (`@deepseek-ai/dsh-pty`) | Backend registry, branded ids, exact-Agent ownership, session operations, and awaited cleanup | `ctx.pty` | +| `pty-local` (`@deepseek-ai/dsh-pty-local`) | Shell backend over `ctx.subprocess.spawnTerminal`: readiness detection, bounded terminal state, sandbox policy, and session operations | registers on `ctx.pty` | +| `tool-pty` (`@deepseek-ai/dsh-tool-pty`) | Six model-facing tools and generic task integration for background sends | registers on `ctx.tools` | -The [persistent PTY decision](../../.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md) records the family boundary. +The design and deferred boundaries live in the [persistent PTY Agent Note](../../.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md). diff --git a/packages/pty/README.zh.md b/packages/pty/README.zh.md index 70d081e60a..c84ad3f1b5 100644 --- a/packages/pty/README.zh.md +++ b/packages/pty/README.zh.md @@ -2,13 +2,12 @@ [English](README.md) | 中文 -本家族为交互式或有状态的终端工作提供持久且限定所有者范围的伪终端会话,是单次 bash 执行的补充。 +`PTY` 的全称是 **Pseudo-Terminal(伪终端)**。这项能力提供持久且限定所有者范围的终端会话,适用于需要跨工具调用保留状态或使用交互式 stdin 的工作流。PTY 是单次 bash 与文件系统工具的补充,不会取代后两者更严格的逐操作契约。 | 包 | 职责 | ctx 键 | |---|---|---| -| [`pty/`](pty/README.md) | 定义 PTY 服务和会话生命周期 | `ctx.pty` | -| [`pty-local/`](pty-local/README.md) | 提供本地持久终端会话 | 注册到 `ctx.pty` | -| [`tool-pty/`](tool-pty/README.md) | 向模型公开 PTY 会话操作 | 注册到 `ctx.tools` | -| [`tool-bash-persistent/`](tool-bash-persistent/README.md) | 公开可复用的 PTY 后端 bash 工具 | 注册到 `ctx.tools` | +| [`pty`](pty/README.md)(`@deepseek-ai/dsh-pty`) | 后端注册表、品牌化 id、精确的 Agent 所有权、会话操作与等待完成的清理 | `ctx.pty` | +| `pty-local`(`@deepseek-ai/dsh-pty-local`) | `ctx.subprocess.spawnTerminal` 之上的 shell 后端:就绪检测、有界终端状态、沙箱策略与会话操作 | 注册到 `ctx.pty` | +| `tool-pty`(`@deepseek-ai/dsh-tool-pty`) | 6 个面向模型的工具,并为后台发送集成通用任务 | 注册到 `ctx.tools` | -[持久 PTY 决策](../../.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md)记录了该家族的边界。 +设计与暂缓边界记录在[持久 PTY Agent Note](../../.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md) 中。 diff --git a/packages/pty/pty-local/README.i18n.yaml b/packages/pty/pty-local/README.i18n.yaml index 48ae675425..d8f5c2e2d0 100644 --- a/packages/pty/pty-local/README.i18n.yaml +++ b/packages/pty/pty-local/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/pty/pty-local/README.md -README.md: ba05495318127b63b3d2a6a60ec743e1ff1c5821 -README.zh.md: 81987ea0685d761507b535b7ed6eefa0888fbd54 +README.md: 8a0a60139e27d98c4f506245a204723da997cd4a +README.zh.md: 3df845931dac33abecbcd5030a1b204c68acc638 diff --git a/packages/pty/pty-local/README.md b/packages/pty/pty-local/README.md index ba05495318..8a0a60139e 100644 --- a/packages/pty/pty-local/README.md +++ b/packages/pty/pty-local/README.md @@ -2,35 +2,35 @@ English | [中文](README.zh.md) -Local Linux/macOS `node-pty` backend for `ctx.pty`; loading it on another platform fails as unsupported. It starts an interactive shell under the shared `ctx.sandboxPolicy`, strips credential-shaped ambient environment variables, retains bounded line-oriented output, detects readiness, and tears down the captured process tree rooted at the `node-pty` child. +Persistent shell backend for `ctx.pty` over `ctx.subprocess.spawnTerminal`. It starts an interactive shell under the shared `ctx.sandboxPolicy`, retains bounded line-oriented output, and detects readiness while the subprocess provider owns PTY allocation, environment scrubbing, foreground process groups, signalling, and complete terminal-session cleanup. The same PTY backend therefore composes with local or remote execution-world providers. ## Plugin (`pty-local`) -The plugin injects `pty`, `sandbox`, and `sandboxPolicy`, then registers the configured backend type (`shell`). `danger-full-access` starts the shell directly; confined modes wrap the exact shell argv through `ctx.sandbox`. At spawn, one `ctx.sandboxPolicy.resolve({ session })` call supplies both the effective mode and the session workspace root; the same root is the default shell cwd when the caller omits one. A change to a different effective mode is rejected before its `sandbox/mode` event commits while that owner has an open PTY or a spawn in progress; the fence is attached to the exact owner and therefore outlives a local-provider reload that retains existing sessions. Wait for creation to settle and close the sessions before changing modes, so a terminal opened with wider access cannot survive a downgrade. +The plugin injects `pty`, `sandbox`, `sandboxPolicy`, and `subprocess`, then registers the configured backend type (`shell`). `danger-full-access` starts the shell directly; confined modes wrap the exact shell argv through `ctx.sandbox`. The effective session mode is resolved at spawn. A change to a different effective mode is rejected before its `sandbox/mode` event commits while that owner has an open PTY or a spawn in progress; the fence is attached to the exact owner and therefore outlives a provider reload that retains existing sessions. Wait for creation to settle and close the sessions before changing modes, so a terminal opened with wider access cannot survive a downgrade. -Linux readiness combines a foreground-verified private bash prompt marker, foreground-process-group syscall inspection, silence fallback, and absolute timeout. macOS uses the verified prompt marker plus silence/timeout because it has no `/proc` syscall surface. A marker is not ready until printable prompt text arrives, including when the OSC marker and `PS1` are split across data callbacks; when bash prints the marker before the kernel publishes its return to the foreground process group, polling retains the candidate for `handoffGraceMs` past the ordinary silence bound so a coincident handoff can win; that grace must cover at least one `pollIntervalMs` and is rejected at load otherwise. An interactive child that inherits `PROMPT_COMMAND` therefore cannot suppress inferred-idle readiness until the absolute timeout. Unrecognized or unreadable process state is never a positive exact-idle signal. A foreground group's stdin wait that already existed before a send is likewise not post-write readiness: the same group must be observed outside that wait before a later wait can settle the send, while a changed foreground group is new evidence. During unpublished startup, a fallback requires observed output; zero-output silence cannot publish an empty session, and timeout rejects the spawn. Cancellation closes the unpublished shell and rejects with the caller's exact abort reason even when its foreground process group is not observable yet; if that close fails, `PtyBackendCleanupError` separately preserves the cleanup failure for registry disposal. Incomplete terminal-control sequences are bounded by `maxReadBytes` and discarded through their terminator after crossing that limit; a trailing carriage return is carried across callbacks so split CRLF becomes one newline. +Readiness combines a foreground-verified private bash prompt marker, provider-reported foreground stdin-wait facts, silence fallback, and absolute timeout. A marker is not ready until printable prompt text arrives, including when the OSC marker and `PS1` are split across data callbacks; when bash prints the marker before the terminal provider publishes its return to the foreground process group, polling retains the candidate for `handoffGraceMs` past the ordinary silence bound so a coincident handoff can win. An interactive child that inherits `PROMPT_COMMAND` therefore cannot suppress inferred-idle readiness until the absolute timeout. Unknown foreground state is never a positive exact-idle signal. A foreground group's stdin wait that existed before a send is likewise not post-write readiness: the same group must be observed outside that wait before a later wait can settle the send, while a changed foreground group is new evidence. During unpublished startup, a fallback requires observed output; zero-output silence cannot publish an empty session, and timeout rejects the spawn. Cancellation closes the unpublished shell and rejects with the caller's exact abort reason; `PtyBackendCleanupError` separately preserves a cleanup failure. Incomplete terminal-control sequences are bounded by `maxReadBytes` and discarded through their terminator after crossing that limit; a trailing carriage return is carried across callbacks so split CRLF becomes one newline. -Send cancellation resolves the current foreground process group and delivers a real `SIGINT`; it never emulates interruption by writing `\x03`, so raw-mode programs remain cancellable. Close sends `SIGTERM` to descendants, waits, then sends `SIGKILL` to the union of captured survivors and newly scanned descendants so reparenting cannot hide a process from teardown. It verifies that every retained identity is gone or, on Linux, a non-executing zombie before stopping the shell; zombie entries are quiescent and are reaped as the shell exits. A survivor failure does not cache a permanently rejected close; a later close retries the teardown. +Send cancellation asks the terminal handle to signal the current foreground process group with a real `SIGINT`; it never emulates interruption by writing `\x03`, so raw-mode programs remain cancellable. Close starts provider-owned TERM-to-KILL whole-session cleanup and awaits quiescence after the terminal outcome. A cleanup failure does not cache a permanently rejected close; a later close retries the provider operation. ## Model Experience -### Current file policy and indirect consumer +### Indirect consumer #### What the model sees -The policy owner contributes capability-neutral `sandbox:policy` context. Through `@deepseek-ai/dsh-tool-pty` or another PTY consumer, the model may also receive bounded MOTD, send deltas, scrollback pages, readiness reasons, and cleanup errors. +Nothing directly. Through `@deepseek-ai/dsh-tool-pty`, the model may receive bounded MOTD, send deltas, scrollback pages, readiness reasons, and cleanup errors. #### Token effect -The current-policy clause is present while this backend is mounted. Retained PTY scrollback is not placed in model history until a consumer returns bounded output. +None until a consumer returns bounded backend output. Retained PTY scrollback is not placed in model history by this package. #### KV Cache effect -A standing-policy change appends an owner-rendered superseding runtime-context snapshot after retained history; consumer results remain append-only. +No direct invalidation; the consumer owns prompts, schemas, and appended results. ## Known Limitations and Deferred Work - Line-oriented output is normalized; full-screen alternate-buffer interaction is unsupported. -- Linux exact probes support x64 and arm64 UAPI tables; other architectures use prompt-marker and silence/timeout readiness. -- A descendant that daemonizes and reparents before teardown leaves the captured tree; cleanup never broadens to the launcher PID's POSIX session because that can include unrelated processes. +- Exact stdin-wait detection depends on the mounted subprocess provider; providers that cannot prove it use prompt-marker and silence/timeout readiness. +- Cleanup guarantees are those of `SubprocessTerminalHandle`; provider-specific gaps belong to that implementation's contract rather than this PTY consumer. - Sessions do not survive harness process exit. diff --git a/packages/pty/pty-local/README.zh.md b/packages/pty/pty-local/README.zh.md index 81987ea068..3df845931d 100644 --- a/packages/pty/pty-local/README.zh.md +++ b/packages/pty/pty-local/README.zh.md @@ -2,35 +2,35 @@ [English](README.md) | 中文 -这个本地 Linux/macOS `node-pty` 后端实现 `ctx.pty`;在其他平台加载时会以不支持为由失败。它在共享 `ctx.sandboxPolicy` 下启动交互式 shell,移除形似凭据的环境变量,保留有界的逐行输出,检测就绪状态,并清理以 `node-pty` 子进程为根的已捕获进程树。 +这是基于 `ctx.subprocess.spawnTerminal` 的 `ctx.pty` 持久 shell 后端。它在共享 `ctx.sandboxPolicy` 下启动交互式 shell,保留有界的逐行输出并检测就绪状态;进程管理提供方负责 PTY 分配、环境清理、前台进程组、信号发送和完整终端会话清理。因此,同一个 PTY 后端可以与本地或远程执行世界提供方组合。 ## 插件(`pty-local`) -该插件注入 `pty`、`sandbox` 和 `sandboxPolicy`,然后注册所配置的后端类型(`shell`)。`danger-full-access` 会直接启动 shell;受限模式则通过 `ctx.sandbox` 包装确切的 shell argv。Spawn 时,一次 `ctx.sandboxPolicy.resolve({ session })` 调用会同时给出实际模式与会话工作区根目录;调用方省略 cwd 时,同一根目录也是 shell 的默认 cwd。当某个所有者存在开放的 PTY 或正在进行 spawn 时,如果配置变更会得到不同的实际模式,系统会在对应 `sandbox/mode` 事件提交前拒绝该变更。该限制绑定到确切所有者,因此即使本地提供方重新加载并保留现有会话,它仍然有效。更改模式前,请等待创建完成并关闭会话,避免以更宽权限打开的终端在权限降级后继续存在。 +该插件注入 `pty`、`sandbox`、`sandboxPolicy` 和 `subprocess`,然后注册所配置的后端类型(`shell`)。`danger-full-access` 会直接启动 shell;受限模式则通过 `ctx.sandbox` 包装确切的 shell argv。系统在 spawn 时解析会话的实际模式。当某个所有者存在开放的 PTY 或正在进行 spawn 时,如果配置变更会得到不同的实际模式,系统会在对应 `sandbox/mode` 事件提交前拒绝该变更。该限制绑定到确切所有者,因此即使提供方重新加载并保留现有会话,它仍然有效。更改模式前,请等待创建结算并关闭会话,避免以更宽权限打开的终端在权限降级后继续存在。 -Linux 的就绪检测结合以下机制:由前台状态验证的私有 bash 提示符标记、前台进程组 syscall 检查、静默回退和绝对超时。macOS 没有 `/proc` syscall 接口,因此使用经过验证的提示符标记以及静默/超时。当可打印的提示符文本尚未到达时,即使 OSC 标记和 `PS1` 被拆到多个数据回调中,系统也不会把标记视为就绪。如果 bash 在内核发布其重新取得前台进程组的状态前打印标记,轮询会在普通静默上限之后再保留该候选状态 `handoffGraceMs`,使恰好同时发生的前台交接有机会胜出;该宽限至少要覆盖一个 `pollIntervalMs`,否则加载时即被拒绝。因此,继承 `PROMPT_COMMAND` 的交互式子进程无法一直抑制推断空闲就绪直至绝对超时。无法识别或读取的进程状态绝不会作为精确空闲的正向信号。同样,一次 send 之前就已存在的前台进程组 stdin 等待并不代表写入后就绪:必须先观察到同一进程组脱离该等待,之后再次进入等待才能使该次 send 完成;前台进程组发生变化则构成新的证据。尚未发布的启动过程中,回退路径要求已经观察到输出;零输出静默不能发布空会话,超时则拒绝 spawn。取消操作会关闭尚未发布的 shell,并以调用方提供的确切中止原因拒绝,即使当时还无法观察其前台进程组。如果关闭失败,`PtyBackendCleanupError` 会单独保留清理失败,供注册表 dispose(资源释放)时处理。未完成的终端控制序列受 `maxReadBytes` 限制;超过上限后,系统会丢弃内容直到其终止符。末尾的回车会跨回调保留,使拆分的 CRLF 合并为一个换行。 +就绪检测结合以下机制:由前台状态验证的私有 bash 提示符标记、提供方报告的前台 stdin 等待事实、静默回退和绝对超时。可打印的提示符文本尚未到达时,即使 OSC 标记和 `PS1` 被拆到多个数据回调中,系统也不会把标记视为就绪。如果 bash 在终端提供方发布其重新取得前台进程组的状态前打印标记,轮询会在普通静默上限之后再保留该候选状态 `handoffGraceMs`,使恰好同时发生的前台交接有机会胜出。因此,继承 `PROMPT_COMMAND` 的交互式子进程无法持续压制推断空闲就绪,最多只能延续到绝对超时。未知的前台状态绝不会作为精确空闲的正向信号。同样,一次 send 之前就已存在的前台进程组 stdin 等待并不代表写入后就绪:必须先观察到同一进程组脱离该等待,之后再次进入等待才能使该次 send 完成;前台进程组发生变化则构成新的证据。尚未发布的启动过程中,回退路径要求已经观察到输出;零输出静默不能发布空会话,超时则拒绝 spawn。取消操作会关闭尚未发布的 shell,并以调用方提供的确切中止原因拒绝;`PtyBackendCleanupError` 会单独保留清理失败。未完成的终端控制序列受 `maxReadBytes` 限制;超过上限后,系统会丢弃内容直到其终止符。末尾的回车会跨回调保留,使拆分的 CRLF 合并为一个换行。 -取消发送时,系统会解析当前前台进程组并发送真正的 `SIGINT`;它绝不会通过写入 `\x03` 模拟中断,因此,即使程序运行在 raw 模式下,也仍可取消。关闭操作先向后代发送 `SIGTERM` 并等待,再向已捕获的存活进程与新扫描到的后代之并集发送 `SIGKILL`,防止进程通过重新设定父进程而逃避清理。在停止 shell 前,系统会确认每个保留的进程身份都已消失,或者在 Linux 上已成为不再执行的僵尸进程;僵尸进程条目视为完全停稳,并会随 shell 退出而回收。如果仍有进程存活,失败结果不会缓存成永久拒绝的关闭操作;后续关闭仍会重试清理。 +取消发送时,系统会请求终端句柄向当前前台进程组发送真正的 `SIGINT`;它绝不会通过写入 `\x03` 模拟中断,因此原始模式程序仍可取消。关闭操作启动由提供方负责的 TERM→KILL 全会话清理,并在终端结果之后等待完全停稳。清理失败不会缓存成永久拒绝的关闭操作;后续关闭会重试提供方操作。 ## 模型体验 -### 当前文件策略与间接消费方 +### 间接消费方 #### 模型看到的内容 -策略归属方会贡献与具体能力无关的 `sandbox:policy` 上下文。模型通过 `@deepseek-ai/dsh-tool-pty` 或其他 PTY 消费方还可能收到有界的 MOTD、发送增量、scrollback 页、就绪原因和清理错误。 +没有直接可见内容。模型通过 `@deepseek-ai/dsh-tool-pty` 可能收到有界的 MOTD、发送增量、scrollback 页、就绪原因和清理错误。 #### Token 影响 -装载该后端期间,当前策略子句会一直存在。消费方返回有界输出前,保留的 PTY scrollback 不会进入模型历史。 +消费方返回有界的后端输出前没有影响。此包不会把保留的 PTY scrollback 放入模型历史。 #### KV Cache 影响 -常驻策略发生变化时,会在保留的历史之后追加一份由归属方渲染、取代先前状态的运行时上下文快照;消费方结果保持仅追加。 +不会直接失效;提示词、schema 与追加结果由消费方负责。 -## 已知限制与暂缓事项 +## 已知限制与暂缓工作 - 输出按行规范化;不支持全屏备用缓冲区交互。 -- Linux 精确探针支持 x64 与 arm64 UAPI 表;其他架构使用提示符标记和静默/超时就绪机制。 -- 如果后代进程在清理前守护化并重新设定父进程,它会脱离已捕获的进程树;清理绝不会扩大到启动器 PID 所属的整个 POSIX 会话,因为其中可能包含无关进程。 -- harness 进程退出后,会话无法继续存在。 +- 精确 stdin 等待检测取决于挂载的进程管理提供方;无法证明该事实的提供方使用提示符标记和静默/超时就绪机制。 +- 清理保证遵循 `SubprocessTerminalHandle`;提供方特有缺口属于该实现的契约,而非此 PTY 消费方。 +- 会话无法跨 harness 进程退出保留。 diff --git a/packages/pty/pty-local/package.json b/packages/pty/pty-local/package.json index a859f482fa..94f98c02d5 100644 --- a/packages/pty/pty-local/package.json +++ b/packages/pty/pty-local/package.json @@ -1,6 +1,6 @@ { "name": "@deepseek-ai/dsh-pty-local", - "description": "Local node-pty backend for persistent DeepSeek Harness PTY sessions", + "description": "Persistent shell PTY backend over the DeepSeek Harness subprocess terminal primitive", "version": "0.0.1", "private": true, "type": "module", @@ -21,12 +21,8 @@ "files": [ "lib/index.js", "lib/invariant.js", - "scripts/ensure-spawn-helper.mjs", "lib/types/**/*.d.ts" ], - "scripts": { - "postinstall": "node scripts/ensure-spawn-helper.mjs" - }, "license": "BSD-3-Clause", "peerDependencies": { "@deepseek-ai/dsh-agent": "^0.0.1", @@ -39,7 +35,6 @@ "cordis": "^4.0.0-rc.7" }, "dependencies": { - "node-pty": "^1.1.0", "schemastery": "^3.18.0" }, "devDependencies": { @@ -50,6 +45,7 @@ "@deepseek-ai/dsh-sandbox-policy": "workspace:^", "@deepseek-ai/dsh-session": "workspace:^", "@deepseek-ai/dsh-subprocess": "workspace:^", + "@deepseek-ai/dsh-subprocess-local": "workspace:^", "cordis": "^4.0.0-rc.7" } } diff --git a/packages/pty/pty-local/src/index.ts b/packages/pty/pty-local/src/index.ts index e59d53446a..0a54a0b804 100644 --- a/packages/pty/pty-local/src/index.ts +++ b/packages/pty/pty-local/src/index.ts @@ -1,22 +1,18 @@ /** - * Local persistent PTY backend using public `node-pty` APIs, shared sandbox - * policy, bounded output, platform readiness probes, and process-session cleanup. + * Persistent shell PTY backend over the subprocess terminal primitive, shared + * sandbox policy, bounded output, and provider-owned session cleanup. * @module @deepseek-ai/dsh-pty-local */ import { Context } from 'cordis' -import * as nodePty from 'node-pty' -import type { IPtyForkOptions } from 'node-pty' import type { Agent } from '@deepseek-ai/dsh-agent' import type { Session, SessionEvent } from '@deepseek-ai/dsh-session' import { PtyBackendCleanupError } from '@deepseek-ai/dsh-pty' -import { scrubbedParentEnv } from '@deepseek-ai/dsh-subprocess' import type { PtyBackend, PtyBackendSpawnSpec } from '@deepseek-ai/dsh-pty' -import type { SandboxExecutionPolicy } from '@deepseek-ai/dsh-sandbox' +import type { SubprocessTerminalHandle, SubprocessTerminalSpawnSpec } from '@deepseek-ai/dsh-subprocess' +import type { SandboxMode } from '@deepseek-ai/dsh-sandbox' import { effectiveSandboxMode } from '@deepseek-ai/dsh-sandbox-policy' import { type Config, type ResolvedConfig, validateConfig } from './config.ts' -import { createProcessInspector } from './process-inspector.ts' -import type { ProcessInspector } from './process-inspector.ts' import { LocalPtySession } from './session.ts' export { Config } from './config.ts' @@ -24,8 +20,8 @@ export type { Config as PtyLocalConfig } from './config.ts' /** Cordis plugin name. */ export const name = 'pty-local' -/** Required services: PTY registry plus the one shared confinement policy. */ -export const inject = ['pty', 'sandbox', 'sandboxPolicy'] +/** Required services: PTY registry, shared confinement policy, and process substrate. */ +export const inject = ['pty', 'sandbox', 'sandboxPolicy', 'subprocess'] interface SandboxModeFenceState { pty: Context['pty'] @@ -55,10 +51,10 @@ function ensureSandboxModeFence(ctx: Context, owner: Agent): void { }, { global: true }) } -function childEnvironment(spec: PtyBackendSpawnSpec): NodeJS.ProcessEnv { - // node-pty owns the spawn; the base env shares the subprocess seam's scrub. +function childEnvironment(spec: PtyBackendSpawnSpec): Record { + // The subprocess provider supplies its own scrubbed ambient base; these are + // deliberate terminal-specific overrides layered after it. return { - ...scrubbedParentEnv(), TERM: 'dumb', PAGER: 'cat', GIT_PAGER: 'cat', @@ -71,11 +67,14 @@ function childEnvironment(spec: PtyBackendSpawnSpec): NodeJS.ProcessEnv { } } -function spawnArgv(ctx: Context, config: ResolvedConfig, policy: SandboxExecutionPolicy): string[] { +function spawnArgv(ctx: Context, config: ResolvedConfig, spec: PtyBackendSpawnSpec): string[] { const argv = [config.shellPath, ...config.shellArgs] - if (policy.mode === 'danger-full-access') return argv - // Re-state the discriminant because object spread does not preserve its narrowed type. - return ctx.sandbox.confine(argv, { ...policy, mode: policy.mode }).argv + const mode: SandboxMode = effectiveSandboxMode(spec.owner.session.events) ?? ctx.sandboxPolicy.defaultMode + if (mode === 'danger-full-access') return argv + return ctx.sandbox.confine(argv, { + mode: mode, + workspaceRoot: ctx.sandboxPolicy.workspaceRoot, + }).argv } /** Local shell backend registered under the configured type. */ @@ -85,13 +84,13 @@ export class LocalPtyBackend implements PtyBackend { constructor( private readonly ctx: Context, private readonly config: ResolvedConfig, - private readonly inspector: ProcessInspector, - private readonly spawnTerminal: typeof nodePty.spawn = nodePty.spawn, + private readonly spawnTerminal: ( + spec: SubprocessTerminalSpawnSpec, + ) => Promise = spec => ctx.subprocess.spawnTerminal(spec), private readonly createSession: ( - terminal: ReturnType, - inspector: ProcessInspector, + terminal: SubprocessTerminalHandle, config: ResolvedConfig, - ) => LocalPtySession = (terminal, inspector, config) => new LocalPtySession(terminal, inspector, config), + ) => LocalPtySession = (terminal, config) => new LocalPtySession(terminal, config), ) { this.type = config.backendType } @@ -99,19 +98,18 @@ export class LocalPtyBackend implements PtyBackend { async spawn(spec: PtyBackendSpawnSpec): Promise { spec.signal?.throwIfAborted() ensureSandboxModeFence(this.ctx, spec.owner) - const policy = this.ctx.sandboxPolicy.resolve({ session: spec.owner.session }) - const argv = spawnArgv(this.ctx, this.config, policy) - const file = argv[0] - if (file === undefined) throw new Error('pty-local: sandbox returned empty argv') - const options: IPtyForkOptions = { - name: 'dumb', - cols: this.config.cols, - rows: this.config.rows, - cwd: spec.cwd ?? policy.workspaceRoot, + const argv = spawnArgv(this.ctx, this.config, spec) + if (argv[0] === undefined) throw new Error('pty-local: sandbox returned empty argv') + const terminal = await this.spawnTerminal({ + argv, + cwd: spec.cwd ?? this.ctx.sandboxPolicy.workspaceRoot, env: childEnvironment(spec), - } - const terminal = this.spawnTerminal(file, argv.slice(1), options) - const session = this.createSession(terminal, this.inspector, this.config) + rows: this.config.rows, + cols: this.config.cols, + graceMs: this.config.disposeGraceMs, + signal: spec.signal, + }) + const session = this.createSession(terminal, this.config) try { await session.initialize(spec.signal) return session @@ -129,6 +127,5 @@ export class LocalPtyBackend implements PtyBackend { /** Register the local PTY backend. */ export function apply(ctx: Context, config: Config): void { validateConfig(config) - const inspector = createProcessInspector() - ctx.pty.registerBackend(new LocalPtyBackend(ctx, config, inspector)) + ctx.pty.registerBackend(new LocalPtyBackend(ctx, config)) } diff --git a/packages/pty/pty-local/src/session.ts b/packages/pty/pty-local/src/session.ts index 46f3be2ca8..a6abaf7785 100644 --- a/packages/pty/pty-local/src/session.ts +++ b/packages/pty/pty-local/src/session.ts @@ -1,8 +1,11 @@ -/** Local `node-pty` session: bounded output, readiness, signals, and teardown. */ +/** Persistent PTY session over the subprocess seam's terminal primitive. */ -import { constants } from 'node:os' import { Buffer } from 'node:buffer' -import type { IDisposable, IPty } from 'node-pty' +import type { + SubprocessOutcome, + SubprocessTerminalForeground, + SubprocessTerminalHandle, +} from '@deepseek-ai/dsh-subprocess' import type { PtyBackendSession, PtyReadRequest, @@ -17,13 +20,8 @@ import type { PtyWaitReason, } from '@deepseek-ai/dsh-pty' import type { ResolvedConfig } from './config.ts' -import type { ProcessIdentity, ProcessInspector } from './process-inspector.ts' import { TerminalSanitizer } from './sanitize.ts' -function delay(ms: number): Promise { - return new Promise(resolve => setTimeout(resolve, ms)) -} - function utf8Tail(text: string, maxBytes: number): { text: string; truncated: boolean } { if (Buffer.byteLength(text) <= maxBytes) return { text, truncated: false } const chars = Array.from(text) @@ -80,17 +78,16 @@ class LocalSendOperation implements PtySendOperation { private readonly promise: PromiseWithResolvers private finished = false private initialForegroundLeftWait: boolean + private initialForegroundPgid: number | undefined constructor( maxBytes: number, readonly startedAt: number, - private readonly initialForegroundPgid: number | undefined, - initialForegroundWasWaiting: boolean, private readonly onCancel: () => void, ) { this.output = new BoundedTextBuffer(maxBytes) this.promise = Promise.withResolvers() - this.initialForegroundLeftWait = !initialForegroundWasWaiting + this.initialForegroundLeftWait = true } get done(): Promise { @@ -123,6 +120,11 @@ class LocalSendOperation implements PtySendOperation { return this.output.consume() } + setInitialForeground(foreground: SubprocessTerminalForeground | undefined): void { + this.initialForegroundPgid = foreground?.processGroupId + this.initialForegroundLeftWait = foreground?.inputWaiting !== true + } + acceptsStdinWait(pgid: number, waiting: boolean): boolean { // The same group may still expose the wait that existed before terminal.write. // Observe every poll so a departure before the exact-settlement threshold @@ -139,27 +141,21 @@ class LocalSendOperation implements PtySendOperation { } } -function signalName(number: number | undefined): NodeJS.Signals | null { - if (number === undefined || number === 0) return null - for (const [name, value] of Object.entries(constants.signals)) { - if (value === number) return name as NodeJS.Signals - } - return null -} - -/** Backend session wrapping one `node-pty` process and its captured process tree. */ +/** Backend session wrapping one provider-owned terminal process. */ export class LocalPtySession implements PtyBackendSession { motd = '' readonly pid: number + private readonly decoder = new TextDecoder('utf-8', { fatal: true }) private readonly sanitizer: TerminalSanitizer private readonly scrollback: BoundedTextBuffer - private readonly exitPromise: PromiseWithResolvers = Promise.withResolvers() - private readonly dataDisposable: IDisposable - private readonly exitDisposable: IDisposable + private readonly outputEnded = Promise.withResolvers() + private readonly completion: Promise private statusValue: PtySessionStatus = { kind: 'running' } private active: LocalSendOperation | undefined private activeTimer: NodeJS.Timeout | undefined + private activeDeadlineTimer: NodeJS.Timeout | undefined private activeAbort: (() => void) | undefined + private polling = false private promptSeen = false private promptTextSeen = false private shellPgid: number | undefined @@ -167,23 +163,22 @@ export class LocalPtySession implements PtyBackendSession { private lastOutputAt = Date.now() private closing = false private closePromise: Promise | undefined + private transportFailure: Error | undefined constructor( - private readonly terminal: IPty, - private readonly inspector: ProcessInspector, + private readonly terminal: SubprocessTerminalHandle, private readonly config: ResolvedConfig, ) { this.pid = terminal.pid this.sanitizer = new TerminalSanitizer(config.maxReadBytes) this.scrollback = new BoundedTextBuffer(config.scrollbackMaxBytes, config.scrollbackLines) - this.dataDisposable = terminal.onData((data) => { this.onData(data) }) - this.exitDisposable = terminal.onExit(({ exitCode, signal }) => { - const tail = this.sanitizer.flush() - this.appendOutput(tail) - this.statusValue = { kind: 'exited', exitCode, signal: signalName(signal) } - this.settleActive('session_exit') - this.exitPromise.resolve() - }) + terminal.output.on('data', this.onTerminalData) + terminal.output.once('end', this.onTerminalEnd) + terminal.output.once('error', this.onTerminalError) + this.completion = terminal.done.then( + outcome => this.onExit(outcome), + (error: unknown) => { this.onTransportFailure(error) }, + ) } /** @@ -213,14 +208,9 @@ export class LocalPtySession implements PtyBackendSession { if (this.active !== undefined) throw new Error('PTY session already has an active send') if (request.signal?.aborted === true) throw new Error('PTY send aborted before write') - const initialForegroundPgid = this.inspector.foregroundPgid(this.pid) - const initialForegroundWasWaiting = initialForegroundPgid !== undefined - && this.inspector.isStdinWaiting(initialForegroundPgid) const operation = new LocalSendOperation( this.config.maxReadBytes, Date.now(), - initialForegroundPgid, - initialForegroundWasWaiting, () => { this.interrupt(operation) }, ) this.active = operation @@ -233,20 +223,28 @@ export class LocalPtySession implements PtyBackendSession { request.signal.addEventListener('abort', onAbort, { once: true }) this.activeAbort = () => request.signal?.removeEventListener('abort', onAbort) } - - try { - if (request.text.length > 0) this.terminal.write(request.text) - if (request.submit) this.terminal.write('\r') - } catch (error: unknown) { - this.clearActive() - operation.fail(error) - return operation - } - - this.activeTimer = setInterval(() => { this.pollReadiness(operation) }, this.config.pollIntervalMs) + this.activeDeadlineTimer = setTimeout(() => { + if (this.active === operation) this.settleActive('timeout') + }, this.config.timeoutMs) + void this.beginSend(operation, request) return operation } + private async beginSend(operation: LocalSendOperation, request: PtySendRequest): Promise { + try { + const foreground = await this.terminal.inspectForeground() + if (this.active !== operation || this.closing) return + operation.setInitialForeground(foreground) + const input = `${request.text}${request.submit ? '\r' : ''}` + if (input.length > 0) await this.terminal.write(Buffer.from(input, 'utf8')) + // Closing can race the awaited provider write even though static analysis sees only local assignments. + // eslint-disable-next-line @typescript-eslint/no-unnecessary-condition + if (this.active === operation && !this.closing) this.schedulePoll(operation, 0) + } catch (error: unknown) { + if (this.active === operation) this.failActive(error) + } + } + read(request: PtyReadRequest): PtyReadResult { const snapshot = this.scrollback.snapshot() const lines = snapshot.text.split('\n') @@ -272,16 +270,9 @@ export class LocalPtySession implements PtyBackendSession { } } - signal(signal: PtySignal): Promise { - return Promise.resolve().then(() => { - const pgid = this.inspector.foregroundPgid(this.pid) - if (pgid === undefined) throw new Error(`cannot resolve foreground process group for PTY ${this.pid}`) - if (signal === 'SIGKILL' && pgid === this.pid) { - throw new Error('refusing to SIGKILL the PTY shell; use terminal_close') - } - this.inspector.signalGroup(pgid, signal) - return { delivered: true, targetPgid: pgid } - }) + async signal(signal: PtySignal): Promise { + const targetPgid = await this.terminal.signalForeground(signal) + return { delivered: true, targetPgid } } status(): PtySessionStatus { @@ -300,12 +291,35 @@ export class LocalPtySession implements PtyBackendSession { return closing } + private readonly onTerminalData = (chunk: Buffer | Uint8Array | string): void => { + try { + const bytes = typeof chunk === 'string' ? Buffer.from(chunk, 'utf8') : chunk + this.onData(this.decoder.decode(bytes, { stream: true })) + } catch (error: unknown) { + this.onTransportFailure(new Error('PTY emitted invalid UTF-8', { cause: error })) + } + } + + private readonly onTerminalEnd = (): void => { + try { + this.onData(this.decoder.decode()) + this.appendOutput(this.sanitizer.flush()) + } catch (error: unknown) { + this.onTransportFailure(new Error('PTY ended with invalid UTF-8', { cause: error })) + } finally { + this.outputEnded.resolve() + } + } + + private readonly onTerminalError = (error: Error): void => { + this.onTransportFailure(error) + this.outputEnded.resolve() + } + private onData(data: string): void { const sanitized = this.sanitizer.push(data) this.appendOutput(sanitized.text) if (sanitized.prompt) { - const foregroundPgid = this.inspector.foregroundPgid(this.pid) - if (this.shellPgid === undefined) this.shellPgid = foregroundPgid // Bash can print PROMPT_COMMAND before the kernel publishes its return // to the foreground process group. Retain the marker; polling below is // the authority that accepts it only after bash owns the foreground. @@ -317,6 +331,21 @@ export class LocalPtySession implements PtyBackendSession { } } + private async onExit(outcome: SubprocessOutcome): Promise { + await this.outputEnded.promise + if (this.transportFailure !== undefined) return + this.statusValue = { kind: 'exited', exitCode: outcome.exitCode, signal: outcome.signal } + this.settleActive('session_exit') + } + + private onTransportFailure(error: unknown): void { + const failure = error instanceof Error ? error : new Error(String(error)) + this.transportFailure ??= failure + this.statusValue = { kind: 'exited', exitCode: null, signal: null } + this.failActive(failure) + this.terminal.terminate() + } + private appendOutput(text: string): void { if (text.length === 0) return this.lastOutputAt = Date.now() @@ -324,44 +353,56 @@ export class LocalPtySession implements PtyBackendSession { this.active?.append(text) } - private pollReadiness(operation: LocalSendOperation): void { - if (this.active !== operation) return - if (this.statusValue.kind === 'exited') { - this.settleActive('session_exit') - return - } - if (this.promptSeen && this.promptTextSeen && Date.now() - this.lastOutputAt >= this.config.pollIntervalMs) { - const pgid = this.inspector.foregroundPgid(this.pid) - if (this.shellPgid !== undefined && pgid === this.shellPgid) { + private schedulePoll(operation: LocalSendOperation, delayMs = this.config.pollIntervalMs): void { + if (this.active !== operation || this.polling) return + if (this.activeTimer !== undefined) clearTimeout(this.activeTimer) + this.activeTimer = setTimeout(() => { + this.activeTimer = undefined + void this.pollReadiness(operation) + }, delayMs) + } + + private async pollReadiness(operation: LocalSendOperation): Promise { + if (this.active !== operation || this.polling) return + this.polling = true + try { + if (this.statusValue.kind === 'exited') { + this.settleActive('session_exit') + return + } + const foreground = await this.terminal.inspectForeground() + if (this.active !== operation) return + const idleFor = Date.now() - this.lastOutputAt + if (this.promptSeen && foreground !== undefined && this.shellPgid === undefined) { + this.shellPgid = foreground.processGroupId + } + if (this.promptSeen && this.promptTextSeen && idleFor >= this.config.pollIntervalMs + && foreground?.processGroupId === this.shellPgid) { this.settleActive('stdin_read') return } + const elapsed = Date.now() - operation.startedAt + const startupHasOutput = !this.initializing || this.scrollback.snapshot().text.length > 0 + const acceptsStdinWait = startupHasOutput && foreground !== undefined + && operation.acceptsStdinWait(foreground.processGroupId, foreground.inputWaiting) + if (elapsed >= this.config.exactProbeAfterMs && acceptsStdinWait) { + this.settleActive('stdin_read') + return + } + // A prompt candidate can race bash's foreground handoff, but an interactive + // child also inherits PROMPT_COMMAND. Silence therefore remains the bound + // on waiting for shell ownership instead of letting a child marker suppress + // readiness until the absolute timeout. + const handoffGrace = this.promptSeen ? this.config.handoffGraceMs : 0 + if (startupHasOutput && idleFor >= this.config.idleSilenceMs + handoffGrace) { + this.settleActive('inferred_idle') + } + } catch (error: unknown) { + if (this.active === operation) this.failActive(error) + } finally { + this.polling = false + if (this.active === operation) this.schedulePoll(operation) } - const elapsed = Date.now() - operation.startedAt - const startupHasOutput = !this.initializing || this.scrollback.snapshot().text.length > 0 - let acceptsStdinWait = false - if (startupHasOutput) { - const pgid = this.inspector.foregroundPgid(this.pid) - acceptsStdinWait = pgid !== undefined - && operation.acceptsStdinWait(pgid, this.inspector.isStdinWaiting(pgid)) - } - if (elapsed >= this.config.exactProbeAfterMs && acceptsStdinWait) { - this.settleActive('stdin_read') - return - } - // A prompt candidate can race bash's foreground handoff, but an interactive - // child also inherits PROMPT_COMMAND. Silence therefore remains the bound - // on waiting for shell ownership instead of letting a child marker suppress - // readiness until the absolute timeout. When a prompt marker was seen, the - // configured grace holds the fallback past the silence bound so polls in - // that window can observe the foreground handoff and settle as stdin_read. - const idleFor = Date.now() - this.lastOutputAt - const handoffGrace = this.promptSeen ? this.config.handoffGraceMs : 0 - if (startupHasOutput && idleFor >= this.config.idleSilenceMs + handoffGrace) { - this.settleActive('inferred_idle') - return - } - if (elapsed >= this.config.timeoutMs) this.settleActive('timeout') } private settleActive(waitReason: PtyWaitReason): void { @@ -373,8 +414,10 @@ export class LocalPtySession implements PtyBackendSession { } private stopPolling(): void { - if (this.activeTimer !== undefined) clearInterval(this.activeTimer) + if (this.activeTimer !== undefined) clearTimeout(this.activeTimer) this.activeTimer = undefined + if (this.activeDeadlineTimer !== undefined) clearTimeout(this.activeDeadlineTimer) + this.activeDeadlineTimer = undefined } private clearActive(): void { @@ -393,104 +436,30 @@ export class LocalPtySession implements PtyBackendSession { private interrupt(operation: LocalSendOperation): void { if (this.active !== operation) return - try { - const pgid = this.inspector.foregroundPgid(this.pid) - if (pgid === undefined) throw new Error(`cannot resolve foreground process group for PTY ${this.pid}`) - this.inspector.signalGroup(pgid, 'SIGINT') - } catch (error: unknown) { - this.failActive(error) - } - } - - private survivors(members: ProcessIdentity[]): ProcessIdentity[] { - return members.filter(member => this.inspector.isAlive(member)) - } - - private descendants(): ProcessIdentity[] { - return this.inspector.processTree(this.pid).filter(member => member.pid !== this.pid) - } - - private async waitForExit(members: ProcessIdentity[]): Promise { - const deadline = Date.now() + this.config.disposeGraceMs - let survivors = this.survivors(members) - while (survivors.length > 0 && Date.now() < deadline) { - await delay(Math.min(25, Math.max(1, deadline - Date.now()))) - survivors = this.survivors(members) - } - return survivors - } - - private signalMembers(members: ProcessIdentity[], signal: 'SIGTERM' | 'SIGKILL'): void { - for (const member of members) { - try { - this.inspector.signalProcess(member, signal) - } catch (_alreadyExitedDuringSignal) { - // Identity is rechecked by the inspector; a same-tick exit is success. - } - } - } - - private unionMembers(...groups: ProcessIdentity[][]): ProcessIdentity[] { - const members: ProcessIdentity[] = [] - const seen = new Set() - for (const group of groups) { - for (const member of group) { - const key = JSON.stringify([member.pid, member.started]) - if (seen.has(key)) continue - seen.add(key) - members.push(member) - } - } - return members - } - - private async stopDescendants(): Promise { - const captured = this.descendants() - this.signalMembers(captured, 'SIGTERM') - const capturedSurvivors = await this.waitForExit(captured) - // A TERM-handling descendant may have forked while winding down. Rescan - // while the shell can still reap every member, then kill both the fresh - // tree and captured survivors that were reparented out of that tree. - const members = this.unionMembers(capturedSurvivors, this.descendants()) - this.signalMembers(members, 'SIGKILL') - const survivors = await this.waitForExit(members) - return this.survivors(this.unionMembers(survivors, this.descendants())) - } - - private async stopShell(): Promise { - try { - this.terminal.kill('SIGTERM') - } catch (_topLevelAlreadyExitedDuringTerm) { - // The exit notification remains authoritative. - } - if (this.statusValue.kind === 'running') { - await Promise.race([this.exitPromise.promise, delay(this.config.disposeGraceMs)]) - } - if (this.statusValue.kind === 'running') { - try { - this.terminal.kill('SIGKILL') - } catch (_topLevelAlreadyExitedDuringKill) { - // The exit notification remains authoritative. - } - await Promise.race([this.exitPromise.promise, delay(this.config.disposeGraceMs)]) - } - if (this.statusValue.kind === 'running') { - throw new Error(`PTY cleanup failed; surviving pids: ${this.pid}`) - } + void this.terminal.signalForeground('SIGINT').catch((error: unknown) => { + if (this.active === operation) this.failActive(error) + }) } private async closeOnce(reason: string): Promise { - this.dataDisposable.dispose() // Stop readiness polling but retain the active operation: teardown settles // it as session_exit below, so an in-flight send is never mis-settled as // stdin_read/inferred_idle/timeout during the grace period. this.stopPolling() - const survivors = await this.stopDescendants() - if (survivors.length > 0) { - throw new Error(`PTY cleanup failed (${reason}); surviving pids: ${survivors.map(member => member.pid).join(', ')}`) + this.terminal.terminate() + const quiescent = await this.terminal.waitForExit() + if (!quiescent) { + throw new Error(`PTY cleanup failed (${reason}); terminal session did not reach quiescence`) } - await this.stopShell() + // Whole-session cleanup can fail before the top-level process exits. Wait + // for it first so that failure is reported instead of blocking forever on + // `done`; successful quiescence guarantees `done` can now settle status and + // drain the terminal output. + await this.completion this.settleActive('session_exit') - this.exitDisposable.dispose() + this.terminal.output.off('data', this.onTerminalData) + this.terminal.output.off('end', this.onTerminalEnd) + this.terminal.output.off('error', this.onTerminalError) + if (this.transportFailure !== undefined) throw this.transportFailure } } diff --git a/packages/pty/pty-local/tests/index.spec.ts b/packages/pty/pty-local/tests/index.spec.ts index 1ad29d7d01..a8753deaf3 100644 --- a/packages/pty/pty-local/tests/index.spec.ts +++ b/packages/pty/pty-local/tests/index.spec.ts @@ -1,9 +1,9 @@ import { describe, expect, it, vi } from 'vitest' -import type { IPty, IPtyForkOptions } from 'node-pty' +import { PassThrough } from 'node:stream' import { Context } from 'cordis' import Loader from '@cordisjs/plugin-loader' import SessionStore, { Session, SessionId } from '@deepseek-ai/dsh-session' -import AgentRegistry, { Inbox, type Agent } from '@deepseek-ai/dsh-agent' +import AgentRegistry, { type Agent } from '@deepseek-ai/dsh-agent' import SandboxProvider from '@deepseek-ai/dsh-sandbox' import type { ConfinedArgv, SandboxPolicy } from '@deepseek-ai/dsh-sandbox' import SandboxPolicyService, { setSandboxMode } from '@deepseek-ai/dsh-sandbox-policy' @@ -11,12 +11,18 @@ import PtyService, { PtyBackendCleanupError, PtySessionId } from '@deepseek-ai/d import { LocalPtyBackend } from '@deepseek-ai/dsh-pty-local' import * as ptyLocal from '@deepseek-ai/dsh-pty-local' import type { ResolvedConfig } from '@deepseek-ai/dsh-pty-local/src/config.ts' -import type { ProcessInspector } from '@deepseek-ai/dsh-pty-local/src/process-inspector.ts' import type { LocalPtySession } from '@deepseek-ai/dsh-pty-local/src/session.ts' +import { SubprocessService } from '@deepseek-ai/dsh-subprocess' +import type { + SubprocessHandle, + SubprocessSpawnSpec, + SubprocessTerminalHandle, + SubprocessTerminalSpawnSpec, +} from '@deepseek-ai/dsh-subprocess' class EmptySandbox extends SandboxProvider { confine(_argv: readonly string[], _policy: SandboxPolicy): ConfinedArgv { - return { argv: [], enforcement: 'full', denialSignatures: [], runnerFailureRules: [] } + return { argv: [], enforcement: 'full', denialSignatures: [], runnerFailureSignatures: [] } } } @@ -25,7 +31,7 @@ class RecordingSandbox extends SandboxProvider { confine(argv: readonly string[], policy: SandboxPolicy): ConfinedArgv { this.calls.push({ argv, policy }) - return { argv: ['/sandbox', '--', ...argv], enforcement: 'full', denialSignatures: [], runnerFailureRules: [] } + return { argv: ['/sandbox', '--', ...argv], enforcement: 'full', denialSignatures: [], runnerFailureSignatures: [] } } } @@ -38,28 +44,37 @@ function config(): ResolvedConfig { } } -function agent(ctx: Context, cwd?: string): Agent { +function agent(ctx: Context): Agent { const id = SessionId('agent') - const session = Session.create(id, undefined, { version: 0, id, createdAt: 0, ...cwd === undefined ? {} : { cwd } }) return { - id, options: {}, session, inbox: new Inbox(session, { inserted: () => {}, discarded: () => {}, claimed: () => {} }), - status: 'idle', - ctx, - send: () => {}, - followup: () => {}, steer: () => {}, inject: () => {}, cancel() {}, - runMaintenance: task => task(new AbortController().signal), - whenIdle: () => Promise.resolve(), + id, options: {}, session: new Session(id), status: 'idle', acceptsNextStep: false, ctx, + followup: () => {}, steer: () => {}, inject: () => {}, send: () => {}, cancel() {}, whenIdle: () => Promise.resolve(), } } -const inspector = { - foregroundPgid: () => undefined, - isStdinWaiting: () => false, - processTree: () => [], - isAlive: () => false, - signalGroup() {}, - signalProcess() {}, -} satisfies ProcessInspector +function terminalHandle(): SubprocessTerminalHandle { + const output = new PassThrough() + return { + pid: 123, + output, + done: Promise.resolve({ exitCode: 0, signal: null }), + write: async () => {}, + inspectForeground: async () => ({ processGroupId: 123, inputWaiting: true }), + signalForeground: async () => 123, + terminate: () => { output.end() }, + waitForExit: async () => true, + } +} + +class StubSubprocessService extends SubprocessService { + readonly cwd = '/tmp' + readonly runtimeRoot = '/tmp/dsh-runtime' + async resolveExecutable(command: string): Promise { return command } + spawn(_spec: SubprocessSpawnSpec): SubprocessHandle { throw new Error('unused') } + async spawnTerminal(_spec: SubprocessTerminalSpawnSpec): Promise { + return terminalHandle() + } +} function spec(owner: Agent, signal?: AbortSignal) { return { @@ -81,12 +96,11 @@ function stubLocalSession(initialize: () => Promise = () => Promise.resolv } function registerStubLocalBackend(ctx: Context, createSession: () => LocalPtySession) { - return ctx.inject(['pty', 'sandbox', 'sandboxPolicy'], (providerCtx) => { + return ctx.inject(['pty', 'sandbox', 'sandboxPolicy', 'subprocess'], (providerCtx) => { providerCtx.pty.registerBackend(new LocalPtyBackend( providerCtx, { ...config(), backendType: 'stub' }, - inspector, - (() => ({})) as never, + async () => terminalHandle(), createSession, )) }) @@ -97,7 +111,7 @@ describe('LocalPtyBackend startup rollback', () => { const ctx = new Context() await ctx.plugin(EmptySandbox) await ctx.plugin(SandboxPolicyService, { mode: 'read-only', workspaceRoot: '/tmp' }) - const backend = new LocalPtyBackend(ctx, config(), inspector) + const backend = new LocalPtyBackend(ctx, config(), async () => terminalHandle()) const controller = new AbortController() const abortReason = new Error('spawn aborted') controller.abort(abortReason) @@ -109,11 +123,11 @@ describe('LocalPtyBackend startup rollback', () => { const ctx = new Context() await ctx.plugin(EmptySandbox) await ctx.plugin(SandboxPolicyService, { mode: 'danger-full-access', workspaceRoot: '/tmp' }) - const spawnTerminal = (() => ({} as IPty)) as never + const spawnTerminal = async (): Promise => terminalHandle() const closed = vi.fn<() => Promise>().mockResolvedValue(undefined) const failed = { initialize: () => Promise.reject(new Error('startup failed')), close: closed } as unknown as LocalPtySession - const backend = new LocalPtyBackend(ctx, config(), inspector, spawnTerminal, () => failed) + const backend = new LocalPtyBackend(ctx, config(), spawnTerminal, () => failed) await expect(backend.spawn(spec(agent(ctx)))).rejects.toThrow('startup failed') expect(closed).toHaveBeenCalledWith('PTY startup failed') @@ -123,7 +137,7 @@ describe('LocalPtyBackend startup rollback', () => { initialize: () => Promise.reject(startupFailure), close: () => Promise.reject(cleanupFailure), } as unknown as LocalPtySession - const aggregate = new LocalPtyBackend(ctx, config(), inspector, spawnTerminal, () => doublyFailed) + const aggregate = new LocalPtyBackend(ctx, config(), spawnTerminal, () => doublyFailed) await expect(aggregate.spawn(spec(agent(ctx)))).rejects.toEqual(expect.objectContaining({ name: 'PtyBackendCleanupError', spawnError: startupFailure, @@ -131,79 +145,72 @@ describe('LocalPtyBackend startup rollback', () => { } satisfies Partial)) }) - it('resolves session mode and root together before wrapping the shell', async () => { + it('wraps confined argv, scrubs the environment, and returns initialized sessions', async () => { const ctx = new Context() await ctx.plugin(RecordingSandbox) - await ctx.plugin(SandboxPolicyService, { mode: 'read-only', workspaceRoot: '/deployment-fallback' }) - const terminal = {} as IPty - let spawned: { file: string; args: string[]; options: IPtyForkOptions } | undefined - const spawnTerminal = ((file: string, args: string[], options: IPtyForkOptions) => { - spawned = { file, args, options } + await ctx.plugin(SandboxPolicyService, { mode: 'workspace-write', workspaceRoot: '/workspace' }) + const terminal = terminalHandle() + let spawned: SubprocessTerminalSpawnSpec | undefined + const spawnTerminal = async (spec: SubprocessTerminalSpawnSpec): Promise => { + spawned = spec return terminal - }) as never + } const initialized = vi.fn<() => Promise>().mockResolvedValue(undefined) const session = { initialize: initialized } as unknown as LocalPtySession const backend = new LocalPtyBackend( ctx, { ...config(), shellArgs: ['-i'] }, - inspector, spawnTerminal, () => session, ) const previous = process.env.PTY_TEST_SECRET process.env.PTY_TEST_SECRET = 'must-not-leak' - const owner = agent(ctx, '/session-workspace') - setSandboxMode(owner.session, 'workspace-write') try { - expect(await backend.spawn(spec(owner))).toBe(session) + expect(await backend.spawn({ ...spec(agent(ctx)), cwd: '/work' })).toBe(session) } finally { if (previous === undefined) delete process.env.PTY_TEST_SECRET else process.env.PTY_TEST_SECRET = previous } expect(spawned).toMatchObject({ - file: '/sandbox', - args: ['--', '/bin/bash', '-i'], - options: { - name: 'dumb', cols: 80, rows: 24, cwd: '/session-workspace', - env: { - TERM: 'dumb', PAGER: 'cat', GIT_PAGER: 'cat', PS1: 'dsh> ', BASH_SILENCE_DEPRECATION_WARNING: '1', - DSH_SHELL: '1', DSH_SESSION_ID: 'agent', DSH_PTY_SESSION_ID: 'pty-1', - }, + argv: ['/sandbox', '--', '/bin/bash', '-i'], + cols: 80, + rows: 24, + cwd: '/work', + graceMs: 10, + env: { + TERM: 'dumb', PAGER: 'cat', GIT_PAGER: 'cat', PS1: 'dsh> ', BASH_SILENCE_DEPRECATION_WARNING: '1', + DSH_SHELL: '1', DSH_SESSION_ID: 'agent', DSH_PTY_SESSION_ID: 'pty-1', }, }) - expect(spawned?.options.env?.PTY_TEST_SECRET).toBeUndefined() + expect(spawned?.env?.PTY_TEST_SECRET).toBeUndefined() expect(initialized).toHaveBeenCalledWith(undefined) - expect((ctx.sandbox as RecordingSandbox).calls).toEqual([{ - argv: ['/bin/bash', '-i'], - policy: { mode: 'workspace-write', workspaceRoot: '/session-workspace' }, - }]) }) it('composes the default local session around a spawned terminal', async () => { const ctx = new Context() await ctx.plugin(EmptySandbox) await ctx.plugin(SandboxPolicyService, { mode: 'danger-full-access', workspaceRoot: '/workspace' }) - let exitListener: ((event: { exitCode: number; signal?: number }) => void) | undefined - const terminal = { - pid: 123, cols: 80, rows: 24, process: 'bash', handleFlowControl: false, - onData(listener: (data: string) => void) { - queueMicrotask(() => { listener('\x1b]133;D;0\x07dsh> ') }) - return { dispose() {} } + const output = new PassThrough() + const outcome = Promise.withResolvers<{ exitCode: number | null; signal: NodeJS.Signals | null }>() + const terminal: SubprocessTerminalHandle = { + pid: 123, + output, + done: outcome.promise, + write: async () => {}, + inspectForeground: async () => ({ processGroupId: 123, inputWaiting: true }), + signalForeground: async () => 123, + terminate() { + output.end() + outcome.resolve({ exitCode: null, signal: 'SIGTERM' }) }, - onExit(listener: (event: { exitCode: number; signal?: number }) => void) { - exitListener = listener - return { dispose() {} } - }, - write() {}, - kill() { exitListener?.({ exitCode: 0, signal: 15 }) }, - resize() {}, clear() {}, pause() {}, resume() {}, - } as IPty + waitForExit: async () => true, + } + queueMicrotask(() => { output.write(Buffer.from('\x1b]133;D;0\x07dsh> ')) }) const backend = new LocalPtyBackend( ctx, config(), - { ...inspector, foregroundPgid: () => terminal.pid }, - () => terminal, + async () => terminal, ) const session = await backend.spawn(spec(agent(ctx))) expect(session.motd).toBe('dsh> ') @@ -217,7 +224,7 @@ describe('pty-local plugin shape', () => { const loader = Object.create(Loader.prototype) as Loader const unwrapped = loader.unwrapExports(ptyLocal) as Record expect(unwrapped.name).toBe('pty-local') - expect(unwrapped.inject).toEqual(['pty', 'sandbox', 'sandboxPolicy']) + expect(unwrapped.inject).toEqual(['pty', 'sandbox', 'sandboxPolicy', 'subprocess']) expect(unwrapped.Config).toBeDefined() }) @@ -227,6 +234,7 @@ describe('pty-local plugin shape', () => { await ctx.plugin(PtyService) await ctx.plugin(EmptySandbox) await ctx.plugin(SandboxPolicyService, { mode: 'danger-full-access', workspaceRoot: '/tmp' }) + await ctx.plugin(StubSubprocessService) const fiber = await ctx.plugin(ptyLocal, config()) expect(ctx.pty.listBackends()).toEqual(['shell']) await fiber.dispose() @@ -240,11 +248,12 @@ describe('pty-local plugin shape', () => { await ctx.plugin(PtyService) await ctx.plugin(EmptySandbox) await ctx.plugin(SandboxPolicyService, { mode: 'danger-full-access', workspaceRoot: '/tmp' }) + await ctx.plugin(StubSubprocessService) await ctx.plugin(ptyLocal, config()) const session = ctx.sessions.create(SessionId('unowned-mode')) expect(() => { - session.append('turn/start', { turn: 1 }) + session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) }).not.toThrow() expect(() => { setSandboxMode(session, 'read-only') }).not.toThrow() }) @@ -256,17 +265,13 @@ describe('pty-local plugin shape', () => { await ctx.plugin(PtyService) await ctx.plugin(RecordingSandbox) await ctx.plugin(SandboxPolicyService, { mode: 'danger-full-access', workspaceRoot: '/tmp' }) + await ctx.plugin(StubSubprocessService) const session = ctx.sessions.create(SessionId('mode-owner')) const ownerFiber = await ctx.plugin(() => {}) const owner: Agent = { - id: session.id, options: {}, session, inbox: new Inbox(session, { inserted: () => {}, discarded: () => {}, claimed: () => {} }), - status: 'idle', - ctx: ownerFiber.ctx, - send: () => {}, - followup: () => {}, steer: () => {}, inject: () => {}, cancel() {}, - runMaintenance: task => task(new AbortController().signal), - whenIdle: () => Promise.resolve(), + id: session.id, options: {}, session, status: 'idle', acceptsNextStep: false, ctx: ownerFiber.ctx, + followup: () => {}, steer: () => {}, inject: () => {}, send: () => {}, cancel() {}, whenIdle: () => Promise.resolve(), } ctx.agents.register(owner) const providerFiber = await registerStubLocalBackend(ctx, () => stubLocalSession()) @@ -275,7 +280,7 @@ describe('pty-local plugin shape', () => { const unrelated = ctx.sessions.create(SessionId('unrelated-mode')) expect(() => { setSandboxMode(unrelated, 'read-only') }).not.toThrow() expect(() => { - session.append('turn/start', { turn: 1 }) + session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) }).not.toThrow() expect(() => { setSandboxMode(session, 'danger-full-access') }).not.toThrow() @@ -304,17 +309,13 @@ describe('pty-local plugin shape', () => { await ctx.plugin(PtyService) await ctx.plugin(RecordingSandbox) await ctx.plugin(SandboxPolicyService, { mode: 'danger-full-access', workspaceRoot: '/tmp' }) + await ctx.plugin(StubSubprocessService) const session = ctx.sessions.create(SessionId('pending-mode-owner')) const ownerFiber = await ctx.plugin(() => {}) const owner: Agent = { - id: session.id, options: {}, session, inbox: new Inbox(session, { inserted: () => {}, discarded: () => {}, claimed: () => {} }), - status: 'idle', - ctx: ownerFiber.ctx, - send: () => {}, - followup: () => {}, steer: () => {}, inject: () => {}, cancel() {}, - runMaintenance: task => task(new AbortController().signal), - whenIdle: () => Promise.resolve(), + id: session.id, options: {}, session, status: 'idle', acceptsNextStep: false, ctx: ownerFiber.ctx, + followup: () => {}, steer: () => {}, inject: () => {}, send: () => {}, cancel() {}, whenIdle: () => Promise.resolve(), } ctx.agents.register(owner) const gate = Promise.withResolvers() diff --git a/packages/pty/pty-local/tests/local.spec.ts b/packages/pty/pty-local/tests/local.spec.ts index c57b0ebb8e..c02a9563ea 100644 --- a/packages/pty/pty-local/tests/local.spec.ts +++ b/packages/pty/pty-local/tests/local.spec.ts @@ -11,6 +11,7 @@ import type { PtySendOperation } from '@deepseek-ai/dsh-pty' import SandboxProvider from '@deepseek-ai/dsh-sandbox' import type { ConfinedArgv, SandboxPolicy } from '@deepseek-ai/dsh-sandbox' import SandboxPolicyService from '@deepseek-ai/dsh-sandbox-policy' +import LocalSubprocessService from '@deepseek-ai/dsh-subprocess-local' import * as ptyLocal from '@deepseek-ai/dsh-pty-local' const roots: string[] = [] @@ -57,6 +58,7 @@ async function harness( await ctx.plugin(PtyService) await ctx.plugin(PassthroughSandbox) await ctx.plugin(SandboxPolicyService, { mode, workspaceRoot: root }) + await ctx.plugin(LocalSubprocessService) const fiber = await ctx.plugin(ptyLocal, { pollIntervalMs: 10, exactProbeAfterMs: 20, diff --git a/packages/pty/pty-local/tests/session.spec.ts b/packages/pty/pty-local/tests/session.spec.ts index 53ce61eebc..7a775eb9a0 100644 --- a/packages/pty/pty-local/tests/session.spec.ts +++ b/packages/pty/pty-local/tests/session.spec.ts @@ -1,62 +1,17 @@ import { afterEach, describe, expect, it, vi } from 'vitest' -import type { IDisposable, IPty } from 'node-pty' +import { PassThrough } from 'node:stream' import { LocalPtySession } from '@deepseek-ai/dsh-pty-local/src/session.ts' import type { ResolvedConfig } from '@deepseek-ai/dsh-pty-local/src/config.ts' -import type { ProcessIdentity, ProcessInspector } from '@deepseek-ai/dsh-pty-local/src/process-inspector.ts' import type { PtySendOperation, PtySessionStatus, PtySignal } from '@deepseek-ai/dsh-pty' - -class FakeTerminal { - pid = 123 - cols = 80 - rows = 24 - process = 'bash' - handleFlowControl = false - writes: string[] = [] - kills: string[] = [] - throwWrite = false - throwKill = false - autoExitOnKill = true - private dataListeners = new Set<(data: string) => void>() - private exitListeners = new Set<(event: { exitCode: number; signal?: number }) => void>() - - readonly onData = (listener: (data: string) => void): IDisposable => { - this.dataListeners.add(listener) - return { dispose: () => this.dataListeners.delete(listener) } - } - - readonly onExit = (listener: (event: { exitCode: number; signal?: number }) => void): IDisposable => { - this.exitListeners.add(listener) - return { dispose: () => this.exitListeners.delete(listener) } - } - - emitData(data: string): void { - for (const listener of this.dataListeners) listener(data) - } - - emitExit(exitCode = 0, signal?: number): void { - for (const listener of this.exitListeners) listener({ exitCode, ...signal === undefined ? {} : { signal } }) - } - - write(data: string): void { - if (this.throwWrite) throw new Error('write failed') - this.writes.push(data) - } - - kill(signal?: string): void { - if (this.throwKill) throw new Error('kill failed') - this.kills.push(signal ?? 'SIGHUP') - if (this.autoExitOnKill) this.emitExit(0, signal === 'SIGKILL' ? 9 : 15) - } - - resize() {} - clear() {} - pause() {} - resume() {} - - asPty(): IPty { - return this - } -} +import type { + SubprocessOutcome, + SubprocessTerminalHandle, + SubprocessTerminalSignal, +} from '@deepseek-ai/dsh-subprocess' +import type { + ProcessIdentity, + ProcessInspector, +} from '@deepseek-ai/dsh-subprocess-local/src/process-inspector.ts' class FakeInspector implements ProcessInspector { pgid: number | undefined = 456 @@ -84,6 +39,89 @@ class FakeInspector implements ProcessInspector { } } +class FakeTerminal implements SubprocessTerminalHandle { + pid = 123 + readonly output = new PassThrough() + readonly writes: string[] = [] + readonly kills: string[] = [] + readonly outcome = Promise.withResolvers() + readonly done = this.outcome.promise + throwWrite = false + throwKill = false + autoExitOnKill = true + quiescent = true + waitError: Error | undefined + + constructor(public inspector = new FakeInspector()) {} + + emitData(data: string): void { + this.output.write(Buffer.from(data, 'utf8')) + } + + emitBytes(data: Uint8Array): void { + this.output.write(data) + } + + emitError(error: Error): void { + this.output.emit('error', error) + } + + emitFailure(error: unknown): void { + this.output.end() + this.outcome.reject(error) + } + + emitExit(exitCode = 0, signal?: number): void { + this.output.end() + this.outcome.resolve({ + exitCode: signal === undefined || signal === 0 ? exitCode : null, + signal: signal === 9 ? 'SIGKILL' : signal === 15 ? 'SIGTERM' : null, + }) + } + + async write(data: Uint8Array): Promise { + if (this.throwWrite) throw new Error('write failed') + this.writes.push(Buffer.from(data).toString('utf8')) + } + + async inspectForeground() { + const processGroupId = this.inspector.foregroundPgid() + return processGroupId === undefined + ? undefined + : { processGroupId, inputWaiting: this.inspector.isStdinWaiting() } + } + + async signalForeground(signal: SubprocessTerminalSignal): Promise { + const foreground = await this.inspectForeground() + if (foreground === undefined) throw new Error(`cannot resolve foreground process group for terminal ${this.pid}`) + if (signal === 'SIGKILL' && foreground.processGroupId === this.pid) { + throw new Error('refusing to SIGKILL the terminal shell; terminate the terminal session instead') + } + this.inspector.signalGroup(foreground.processGroupId, signal) + return foreground.processGroupId + } + + terminate(): void { + if (this.throwKill) throw new Error('kill failed') + this.kills.push('SIGTERM') + if (this.autoExitOnKill) this.emitExit(0, 15) + } + + async waitForExit(): Promise { + if (this.waitError !== undefined) throw this.waitError + return this.quiescent + } +} + +function makeSession( + terminal: FakeTerminal, + inspector: FakeInspector, + resolved: ResolvedConfig, +): LocalPtySession { + terminal.inspector = inspector + return new LocalPtySession(terminal, resolved) +} + function config(overrides: Partial = {}): ResolvedConfig { return { backendType: 'shell', shellPath: '/bin/bash', shellArgs: [], rows: 24, cols: 80, @@ -108,13 +146,15 @@ describe('LocalPtySession readiness and output', () => { vi.useFakeTimers() const terminal = new FakeTerminal() const inspector = new FakeInspector() - const session = new LocalPtySession(terminal.asPty(), inspector, config()) + const session = makeSession(terminal, inspector, config()) await initialize(session, terminal) expect(session.motd).toBe('dsh> ') inspector.waiting = true const operation = session.startSend({ text: 'python3', submit: true }) - expect(terminal.writes).toEqual(['python3', '\r']) + await Promise.resolve() + await Promise.resolve() + expect(terminal.writes).toEqual(['python3\r']) inspector.pgid = 789 terminal.emitData('Python\r\n>>> ') await vi.advanceTimersByTimeAsync(20) @@ -126,7 +166,7 @@ describe('LocalPtySession readiness and output', () => { vi.useFakeTimers() const terminal = new FakeTerminal() const inspector = new FakeInspector() - const session = new LocalPtySession(terminal.asPty(), inspector, config()) + const session = makeSession(terminal, inspector, config()) await initialize(session, terminal) inspector.waiting = true @@ -148,7 +188,7 @@ describe('LocalPtySession readiness and output', () => { vi.useFakeTimers() const terminal = new FakeTerminal() const inspector = new FakeInspector() - const session = new LocalPtySession(terminal.asPty(), inspector, config({ + const session = makeSession(terminal, inspector, config({ exactProbeAfterMs: 50, idleSilenceMs: 100, timeoutMs: 200, @@ -173,7 +213,7 @@ describe('LocalPtySession readiness and output', () => { vi.useFakeTimers() const terminal = new FakeTerminal() const inspector = new FakeInspector() - const session = new LocalPtySession(terminal.asPty(), inspector, config()) + const session = makeSession(terminal, inspector, config()) await initialize(session, terminal) inspector.pgid = undefined @@ -193,7 +233,7 @@ describe('LocalPtySession readiness and output', () => { const exiting = session.startSend({ text: 'exit', submit: true }) terminal.emitExit(7, 9) - expect(await exiting.done).toMatchObject({ waitReason: 'session_exit', sessionStatus: { kind: 'exited', exitCode: 7, signal: 'SIGKILL' } }) + expect(await exiting.done).toMatchObject({ waitReason: 'session_exit', sessionStatus: { kind: 'exited', exitCode: null, signal: 'SIGKILL' } }) expect(() => session.startSend({ text: '', submit: false })).toThrow('has exited') }) @@ -201,13 +241,15 @@ describe('LocalPtySession readiness and output', () => { vi.useFakeTimers() const terminal = new FakeTerminal() const inspector = new FakeInspector() - const session = new LocalPtySession(terminal.asPty(), inspector, config()) + const session = makeSession(terminal, inspector, config()) await initialize(session, terminal) const controller = new AbortController() const operation = session.startSend({ text: 'sleep', submit: true, signal: controller.signal }) expect(() => session.startSend({ text: 'again', submit: true })).toThrow('active send') controller.abort() + await Promise.resolve() + await Promise.resolve() expect(inspector.groups).toContainEqual([456, 'SIGINT']) expect(terminal.writes).not.toContain('\x03') terminal.emitData('\x1b]133;D;130\x07dsh> ') @@ -229,14 +271,14 @@ describe('LocalPtySession readiness and output', () => { it('handles startup exit, unknown exit signals, cancel-write failure, and stale polls', async () => { vi.useFakeTimers() const startupTerminal = new FakeTerminal() - const startup = new LocalPtySession(startupTerminal.asPty(), new FakeInspector(), config()) + const startup = new LocalPtySession(startupTerminal, config()) const initializing = startup.initialize(new AbortController().signal) startupTerminal.emitExit(1) await expect(initializing).rejects.toThrow('exited during startup') expect(startup.status()).toEqual({ kind: 'exited', exitCode: 1, signal: null }) const terminal = new FakeTerminal() - const session = new LocalPtySession(terminal.asPty(), new FakeInspector(), config()) + const session = new LocalPtySession(terminal, config()) await initialize(session, terminal) const operation = session.startSend({ text: '', submit: false }) const operationInternal = operation as unknown as { @@ -247,11 +289,17 @@ describe('LocalPtySession readiness and output', () => { const sessionInternal = session as unknown as { pollReadiness(operation: PtySendOperation): void interrupt(operation: PtySendOperation): void + schedulePoll(operation: PtySendOperation): void + polling: boolean statusValue: PtySessionStatus appendOutput(text: string): void } sessionInternal.appendOutput('') sessionInternal.pollReadiness({} as PtySendOperation) + sessionInternal.schedulePoll({} as PtySendOperation) + sessionInternal.polling = true + sessionInternal.schedulePoll(operation) + sessionInternal.polling = false sessionInternal.interrupt({} as PtySendOperation) sessionInternal.statusValue = { kind: 'exited', exitCode: 2, signal: null } sessionInternal.pollReadiness(operation) @@ -259,13 +307,15 @@ describe('LocalPtySession readiness and output', () => { operationInternal.settle('timeout', { kind: 'running' }, false) const unknownTerminal = new FakeTerminal() - const unknown = new LocalPtySession(unknownTerminal.asPty(), new FakeInspector(), config()) + const unknown = new LocalPtySession(unknownTerminal, config()) unknownTerminal.emitExit(1, 999) - expect(unknown.status()).toEqual({ kind: 'exited', exitCode: 1, signal: null }) + await vi.waitFor(() => { + expect(unknown.status()).toEqual({ kind: 'exited', exitCode: null, signal: null }) + }) const cancelTerminal = new FakeTerminal() const cancelInspector = new FakeInspector() - const cancel = new LocalPtySession(cancelTerminal.asPty(), cancelInspector, config()) + const cancel = makeSession(cancelTerminal, cancelInspector, config()) await initialize(cancel, cancelTerminal) const cancellable = cancel.startSend({ text: '', submit: false }) cancelInspector.throwGroup = true @@ -275,7 +325,7 @@ describe('LocalPtySession readiness and output', () => { const missingGroupTerminal = new FakeTerminal() const missingGroupInspector = new FakeInspector() - const missingGroup = new LocalPtySession(missingGroupTerminal.asPty(), missingGroupInspector, config()) + const missingGroup = makeSession(missingGroupTerminal, missingGroupInspector, config()) await initialize(missingGroup, missingGroupTerminal) missingGroupInspector.pgid = undefined const unresolved = missingGroup.startSend({ text: '', submit: false }) @@ -286,7 +336,7 @@ describe('LocalPtySession readiness and output', () => { it('does not treat zero-output startup silence as readiness and fails on startup timeout', async () => { vi.useFakeTimers() const terminal = new FakeTerminal() - const session = new LocalPtySession(terminal.asPty(), new FakeInspector(), config()) + const session = new LocalPtySession(terminal, config()) let settled = false const initializing = session.initialize().then(() => { settled = true }) await vi.advanceTimersByTimeAsync(60) @@ -296,7 +346,7 @@ describe('LocalPtySession readiness and output', () => { await initializing const timeoutTerminal = new FakeTerminal() - const timeout = new LocalPtySession(timeoutTerminal.asPty(), new FakeInspector(), config()) + const timeout = new LocalPtySession(timeoutTerminal, config()) const timedOut = expect(timeout.initialize()).rejects.toThrow('startup timeout') await vi.advanceTimersByTimeAsync(100) await timedOut @@ -306,7 +356,7 @@ describe('LocalPtySession readiness and output', () => { const terminal = new FakeTerminal() const inspector = new FakeInspector() inspector.pgid = undefined - const session = new LocalPtySession(terminal.asPty(), inspector, config()) + const session = makeSession(terminal, inspector, config()) const controller = new AbortController() const reason = new Error('startup cancelled') @@ -320,7 +370,7 @@ describe('LocalPtySession readiness and output', () => { it('waits for printable prompt text when the startup marker is split from PS1', async () => { vi.useFakeTimers() const terminal = new FakeTerminal() - const session = new LocalPtySession(terminal.asPty(), new FakeInspector(), config()) + const session = new LocalPtySession(terminal, config()) let settled = false const initializing = session.initialize().then(() => { settled = true }) @@ -338,7 +388,7 @@ describe('LocalPtySession readiness and output', () => { vi.useFakeTimers() const terminal = new FakeTerminal() const inspector = new FakeInspector() - const session = new LocalPtySession(terminal.asPty(), inspector, config()) + const session = makeSession(terminal, inspector, config()) await initialize(session, terminal) const operation = session.startSend({ text: 'run', submit: true }) @@ -359,7 +409,7 @@ describe('LocalPtySession readiness and output', () => { vi.useFakeTimers() const terminal = new FakeTerminal() const inspector = new FakeInspector() - const session = new LocalPtySession(terminal.asPty(), inspector, config({ handoffGraceMs: 40 })) + const session = makeSession(terminal, inspector, config({ handoffGraceMs: 40 })) await initialize(session, terminal) const operation = session.startSend({ text: 'run', submit: true }) @@ -380,7 +430,7 @@ describe('LocalPtySession readiness and output', () => { vi.useFakeTimers() const terminal = new FakeTerminal() const inspector = new FakeInspector() - const session = new LocalPtySession(terminal.asPty(), inspector, config()) + const session = makeSession(terminal, inspector, config()) await initialize(session, terminal) const operation = session.startSend({ text: 'bash -i', submit: true }) @@ -390,6 +440,156 @@ describe('LocalPtySession readiness and output', () => { expect((await operation.done).waitReason).toBe('inferred_idle') }) + + it('contains terminal transport failures and preserves the first failure', async () => { + const terminal = new FakeTerminal() + const session = new LocalPtySession(terminal, config()) + const operation = session.startSend({ text: '', submit: false }) + terminal.output.emit('data', 'plain text') + terminal.emitError(new Error('output transport failed')) + ;(session as unknown as { onTransportFailure(error: unknown): void }) + .onTransportFailure(new Error('later failure')) + await expect(operation.done).rejects.toThrow('output transport failed') + expect(session.status()).toEqual({ kind: 'exited', exitCode: null, signal: null }) + await expect(session.close('transport')).rejects.toThrow('output transport failed') + + const rejectedTerminal = new FakeTerminal() + const rejected = new LocalPtySession(rejectedTerminal, config()) + const rejectedOperation = rejected.startSend({ text: '', submit: false }) + rejectedTerminal.emitFailure('raw transport failure') + await expect(rejectedOperation.done).rejects.toThrow('raw transport failure') + }) + + it('rejects invalid UTF-8 in a data chunk and at stream end', async () => { + const chunkTerminal = new FakeTerminal() + const chunkSession = new LocalPtySession(chunkTerminal, config()) + const chunkOperation = chunkSession.startSend({ text: '', submit: false }) + chunkTerminal.emitBytes(Uint8Array.from([0xff])) + await expect(chunkOperation.done).rejects.toThrow('PTY emitted invalid UTF-8') + + const endTerminal = new FakeTerminal() + const endSession = new LocalPtySession(endTerminal, config()) + const endOperation = endSession.startSend({ text: '', submit: false }) + endTerminal.emitBytes(Uint8Array.from([0xe2])) + endTerminal.emitExit() + await expect(endOperation.done).rejects.toThrow('PTY ended with invalid UTF-8') + }) + + it('contains readiness inspection failure and a stale inspection result', async () => { + vi.useFakeTimers() + const failedTerminal = new FakeTerminal() + const failedSession = new LocalPtySession(failedTerminal, config()) + const failedOperation = failedSession.startSend({ text: '', submit: false }) + await Promise.resolve() + await Promise.resolve() + failedTerminal.inspectForeground = async () => { throw new Error('inspect failed') } + const failedInternal = failedSession as unknown as { + pollReadiness(operation: PtySendOperation): Promise + } + await failedInternal.pollReadiness(failedOperation) + await expect(failedOperation.done).rejects.toThrow('inspect failed') + + const staleTerminal = new FakeTerminal() + const staleSession = new LocalPtySession(staleTerminal, config()) + const staleOperation = staleSession.startSend({ text: '', submit: false }) + await Promise.resolve() + await Promise.resolve() + const gate = Promise.withResolvers extends Promise ? T : never>() + staleTerminal.inspectForeground = async () => await gate.promise + const staleInternal = staleSession as unknown as { + active: PtySendOperation | undefined + pollReadiness(operation: PtySendOperation): Promise + } + const polling = staleInternal.pollReadiness(staleOperation) + staleInternal.active = undefined + gate.resolve({ processGroupId: 456, inputWaiting: false }) + await polling + ;(staleOperation as unknown as { + settle(reason: 'timeout', status: PtySessionStatus, inherited: boolean): void + }).settle('timeout', { kind: 'running' }, false) + }) + + it('contains stale timer, write, inspection, and interrupt continuations', async () => { + vi.useFakeTimers() + const settle = (operation: PtySendOperation): void => { + ;(operation as unknown as { + settle(reason: 'timeout', status: PtySessionStatus, inherited: boolean): void + }).settle('timeout', { kind: 'running' }, false) + } + + const deadlineTerminal = new FakeTerminal() + const deadlineSession = new LocalPtySession(deadlineTerminal, config()) + const deadlineOperation = deadlineSession.startSend({ text: '', submit: false }) + ;(deadlineSession as unknown as { active: PtySendOperation | undefined }).active = undefined + await vi.advanceTimersByTimeAsync(100) + settle(deadlineOperation) + + const writeTerminal = new FakeTerminal() + const writeGate = Promise.withResolvers() + writeTerminal.write = async () => { await writeGate.promise } + const writeSession = new LocalPtySession(writeTerminal, config()) + const writeOperation = writeSession.startSend({ text: 'x', submit: false }) + await Promise.resolve() + await Promise.resolve() + ;(writeSession as unknown as { closing: boolean }).closing = true + writeGate.resolve(undefined) + await Promise.resolve() + await Promise.resolve() + settle(writeOperation) + + const beginTerminal = new FakeTerminal() + const beginGate = Promise.withResolvers() + beginTerminal.inspectForeground = async () => await beginGate.promise + const beginSession = new LocalPtySession(beginTerminal, config()) + const beginOperation = beginSession.startSend({ text: '', submit: false }) + ;(beginSession as unknown as { active: PtySendOperation | undefined }).active = undefined + beginGate.reject(new Error('stale begin failure')) + await Promise.resolve() + await Promise.resolve() + settle(beginOperation) + + const scheduledTerminal = new FakeTerminal() + const scheduledSession = new LocalPtySession(scheduledTerminal, config()) + const scheduledOperation = scheduledSession.startSend({ text: '', submit: false }) + await Promise.resolve() + await Promise.resolve() + const scheduledInternal = scheduledSession as unknown as { + schedulePoll(operation: PtySendOperation, delayMs?: number): void + settleActive(reason: 'timeout'): void + } + scheduledInternal.schedulePoll(scheduledOperation, 5) + scheduledInternal.settleActive('timeout') + await scheduledOperation.done + + const pollTerminal = new FakeTerminal() + const pollSession = new LocalPtySession(pollTerminal, config()) + const pollOperation = pollSession.startSend({ text: '', submit: false }) + await Promise.resolve() + await Promise.resolve() + const pollGate = Promise.withResolvers() + pollTerminal.inspectForeground = async () => await pollGate.promise + const pollInternal = pollSession as unknown as { + active: PtySendOperation | undefined + pollReadiness(operation: PtySendOperation): Promise + } + const stalePoll = pollInternal.pollReadiness(pollOperation) + pollInternal.active = undefined + pollGate.reject(new Error('stale poll failure')) + await stalePoll + settle(pollOperation) + + const interruptTerminal = new FakeTerminal() + const interruptGate = Promise.withResolvers() + interruptTerminal.signalForeground = async () => await interruptGate.promise + const interruptSession = new LocalPtySession(interruptTerminal, config()) + const interruptOperation = interruptSession.startSend({ text: '', submit: false }) + expect(interruptOperation.cancel()).toBe(true) + ;(interruptSession as unknown as { active: PtySendOperation | undefined }).active = undefined + interruptGate.reject(new Error('stale interrupt failure')) + await Promise.resolve() + await Promise.resolve() + settle(interruptOperation) + }) }) describe('LocalPtySession bounds, signals, and teardown', () => { @@ -397,8 +597,7 @@ describe('LocalPtySession bounds, signals, and teardown', () => { vi.useFakeTimers() const terminal = new FakeTerminal() const session = new LocalPtySession( - terminal.asPty(), - new FakeInspector(), + terminal, config({ scrollbackLines: 3, scrollbackMaxBytes: 12, maxReadBytes: 6 }), ) expect(session.read({})).toMatchObject({ text: '' }) @@ -415,7 +614,7 @@ describe('LocalPtySession bounds, signals, and teardown', () => { expect(() => session.read({ count: 0 })).toThrow('count') const tinyTerminal = new FakeTerminal() - const tiny = new LocalPtySession(tinyTerminal.asPty(), new FakeInspector(), config({ maxReadBytes: 1 })) + const tiny = new LocalPtySession(tinyTerminal, config({ maxReadBytes: 1 })) await initialize(tiny, tinyTerminal) const tinyOperation = tiny.startSend({ text: '', submit: false }) tinyTerminal.emitData('一') @@ -427,32 +626,38 @@ describe('LocalPtySession bounds, signals, and teardown', () => { it('signals verified groups and refuses unresolved or shell-targeted hard kills', async () => { const terminal = new FakeTerminal() const inspector = new FakeInspector() - const session = new LocalPtySession(terminal.asPty(), inspector, config()) + const session = makeSession(terminal, inspector, config()) expect(await session.signal('SIGINT')).toEqual({ delivered: true, targetPgid: 456 }) inspector.pgid = terminal.pid - await expect(session.signal('SIGKILL')).rejects.toThrow('use terminal_close') + await expect(session.signal('SIGKILL')).rejects.toThrow('terminate the terminal session') inspector.pgid = undefined await expect(session.signal('SIGTERM')).rejects.toThrow('cannot resolve') }) it('closes idempotently, contains signal races, and reports survivors', async () => { const terminal = new FakeTerminal() - const inspector = new FakeInspector() - inspector.members = [{ pid: 123, started: 'a' }] - inspector.alive.add(123) - inspector.throwProcess = true - terminal.throwKill = true - const session = new LocalPtySession(terminal.asPty(), inspector, config({ disposeGraceMs: 1 })) + terminal.quiescent = false + const session = new LocalPtySession(terminal, config({ disposeGraceMs: 1 })) const closing = session.close('test') expect(session.close('other')).toBe(closing) - await expect(closing).rejects.toThrow('surviving pids: 123') + await expect(closing).rejects.toThrow('did not reach quiescence') expect(() => session.startSend({ text: '', submit: false })).toThrow('closing') }) + it('reports cleanup failure without waiting for top-level exit', async () => { + const terminal = new FakeTerminal() + terminal.autoExitOnKill = false + terminal.waitError = new Error('terminal cleanup failed; surviving pids: 456') + const session = new LocalPtySession(terminal, config()) + + await expect(session.close('survivor')).rejects.toThrow('surviving pids: 456') + expect(terminal.kills).toEqual(['SIGTERM']) + }) + it('settles an active send as session_exit when closed mid-operation', async () => { vi.useFakeTimers() const terminal = new FakeTerminal() - const session = new LocalPtySession(terminal.asPty(), new FakeInspector(), config({ disposeGraceMs: 50 })) + const session = new LocalPtySession(terminal, config({ disposeGraceMs: 50 })) await initialize(session, terminal) const operation = session.startSend({ text: 'run', submit: true }) // The shell returns to its prompt while the send is active; a running @@ -467,94 +672,4 @@ describe('LocalPtySession bounds, signals, and teardown', () => { await closing }) - it('keeps the shell alive until SIGKILL recipients leave the process table', async () => { - vi.useFakeTimers() - const terminal = new FakeTerminal() - const inspector = new FakeInspector() - inspector.members = [{ pid: 124, started: 'child' }] - inspector.alive.add(124) - inspector.removeOnSignal = false - const session = new LocalPtySession(terminal.asPty(), inspector, config({ disposeGraceMs: 20 })) - - let settled = false - const closing = session.close('test').then(() => { settled = true }) - await vi.advanceTimersByTimeAsync(20) - expect(inspector.processes).toContainEqual([124, 'SIGKILL']) - expect(terminal.kills).toEqual([]) - expect(settled).toBe(false) - - inspector.alive.delete(124) - await vi.advanceTimersByTimeAsync(20) - await closing - expect(terminal.kills).toEqual(['SIGTERM']) - expect(settled).toBe(true) - }) - - it('rescans for descendants forked during TERM before stopping the shell', async () => { - const terminal = new FakeTerminal() - const inspector = new FakeInspector() - let reads = 0 - inspector.processTree = () => { - reads += 1 - if (reads === 1) { - inspector.alive.add(124) - return [{ pid: 124, started: 'first' }] - } - if (reads === 2) { - inspector.alive.add(125) - return [{ pid: 125, started: 'late' }] - } - return [] - } - const session = new LocalPtySession(terminal.asPty(), inspector, config()) - - await session.close('test') - - expect(inspector.processes).toEqual([[124, 'SIGTERM'], [125, 'SIGKILL']]) - expect(terminal.kills).toEqual(['SIGTERM']) - }) - - it('retains captured survivors that are reparented out of the teardown rescan', async () => { - vi.useFakeTimers() - const terminal = new FakeTerminal() - const inspector = new FakeInspector() - const captured = { pid: 124, started: 'captured' } - let reads = 0 - inspector.alive.add(captured.pid) - inspector.processTree = () => reads++ === 0 ? [captured] : [] - inspector.signalProcess = (identity, signal) => { - inspector.processes.push([identity.pid, signal]) - if (signal === 'SIGKILL') inspector.alive.delete(identity.pid) - } - const session = new LocalPtySession(terminal.asPty(), inspector, config({ disposeGraceMs: 20 })) - - const closing = session.close('test') - await vi.advanceTimersByTimeAsync(25) - await closing - - expect(inspector.processes).toEqual([[124, 'SIGTERM'], [124, 'SIGKILL']]) - expect(terminal.kills).toEqual(['SIGTERM']) - }) - - it('allows teardown to retry after a descendant-survivor failure', async () => { - vi.useFakeTimers() - const terminal = new FakeTerminal() - const inspector = new FakeInspector() - inspector.members = [{ pid: 124, started: 'child' }] - inspector.alive.add(124) - inspector.removeOnSignal = false - const session = new LocalPtySession(terminal.asPty(), inspector, config({ disposeGraceMs: 10 })) - - const first = session.close('first') - const rejected = expect(first).rejects.toThrow('surviving pids: 124') - await vi.advanceTimersByTimeAsync(25) - await rejected - expect(terminal.kills).toEqual([]) - - inspector.alive.delete(124) - const second = session.close('retry') - expect(second).not.toBe(first) - await second - expect(terminal.kills).toEqual(['SIGTERM']) - }) }) diff --git a/packages/pty/tool-pty/package.json b/packages/pty/tool-pty/package.json index 5512daba92..9027169941 100644 --- a/packages/pty/tool-pty/package.json +++ b/packages/pty/tool-pty/package.json @@ -51,6 +51,7 @@ "@deepseek-ai/dsh-sandbox-policy": "workspace:^", "@deepseek-ai/dsh-session": "workspace:^", "@deepseek-ai/dsh-system-prompt": "workspace:^", + "@deepseek-ai/dsh-subprocess-local": "workspace:^", "@deepseek-ai/dsh-tasks": "workspace:^", "@deepseek-ai/dsh-tasks-local": "workspace:^", "@deepseek-ai/dsh-tool-tasks": "workspace:^", diff --git a/packages/pty/tool-pty/tests/loader-composition.spec.ts b/packages/pty/tool-pty/tests/loader-composition.spec.ts index 835df86abc..35d0ea5d0a 100644 --- a/packages/pty/tool-pty/tests/loader-composition.spec.ts +++ b/packages/pty/tool-pty/tests/loader-composition.spec.ts @@ -16,6 +16,7 @@ import PtyService from '@deepseek-ai/dsh-pty' import SandboxProvider from '@deepseek-ai/dsh-sandbox' import type { ConfinedArgv, SandboxPolicy } from '@deepseek-ai/dsh-sandbox' import SandboxPolicyService from '@deepseek-ai/dsh-sandbox-policy' +import LocalSubprocessService from '@deepseek-ai/dsh-subprocess-local' import * as PtyLocal from '@deepseek-ai/dsh-pty-local' import * as ToolPty from '@deepseek-ai/dsh-tool-pty' @@ -72,6 +73,7 @@ suite('terminal real Loader composition through cordis.yml', () => { ' config:', ' mode: danger-full-access', ` workspaceRoot: ${JSON.stringify(root)}`, + "- name: '@deepseek-ai/dsh-subprocess-local'", "- name: '@deepseek-ai/dsh-pty-local'", ' config:', ' pollIntervalMs: 10', @@ -95,6 +97,7 @@ suite('terminal real Loader composition through cordis.yml', () => { ['@deepseek-ai/dsh-pty', PtyService], ['@deepseek-ai/dsh-test-sandbox', PassthroughSandbox], ['@deepseek-ai/dsh-sandbox-policy', SandboxPolicyService], + ['@deepseek-ai/dsh-subprocess-local', LocalSubprocessService], ['@deepseek-ai/dsh-pty-local', PtyLocal], ['@deepseek-ai/dsh-tool-pty', ToolPty], ]) diff --git a/packages/skill/skill-local/tests/skill-local.spec.ts b/packages/skill/skill-local/tests/skill-local.spec.ts index 0f92cac1d6..9dc723fb58 100644 --- a/packages/skill/skill-local/tests/skill-local.spec.ts +++ b/packages/skill/skill-local/tests/skill-local.spec.ts @@ -42,6 +42,14 @@ class TestFileSystem extends FileSystem { return { targetKey: path as never, displayPath: path } } + override processPath(target: FsTarget): string { return String(target.targetKey) } + + override fileUrl(target: FsTarget): string { return `file://${target.targetKey}` } + + override contains(parent: FsTarget, child: FsTarget): boolean { + return child.targetKey === parent.targetKey || String(child.targetKey).startsWith(`${parent.targetKey}/`) + } + override async stat(target: FsTarget, signal?: AbortSignal): Promise { this.statSignals.push(signal) if (this.failStatPaths.has(target.displayPath)) throw new FsError('stat failed', 'FS_NOT_FOUND') @@ -84,6 +92,12 @@ class TestFileSystem extends FileSystem { return text } + override async readTextBounded(target: FsTarget, maxBytes: number, signal?: AbortSignal): Promise { + const text = await this.readText(target, signal) + if (Buffer.byteLength(text) > maxBytes) throw new Error('too large') + return text + } + override async streamText(_target: FsTarget): Promise> { throw new Error('not needed in skill tests') } diff --git a/packages/subprocess/README.i18n.yaml b/packages/subprocess/README.i18n.yaml index 9c0cb0c25d..256d4ad2d9 100644 --- a/packages/subprocess/README.i18n.yaml +++ b/packages/subprocess/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/subprocess/README.md -README.md: 187ea5b778a4bc1f9c3c9121adb18bda11cc7b58 -README.zh.md: dd3a975daec014131877d7b1523810bd932619d8 +README.md: c4bb1da1a172b05afa63834db4e5b1fa974baabb +README.zh.md: f68bc1c720eeb5282bb9c94c951524a021e38b2f diff --git a/packages/subprocess/README.md b/packages/subprocess/README.md index 187ea5b778..c4bb1da1a1 100644 --- a/packages/subprocess/README.md +++ b/packages/subprocess/README.md @@ -2,11 +2,11 @@ English | [中文](README.zh.md) -This family runs host subprocesses behind an explicit process-lifecycle service. +The shared process substrate for one execution world: canonical cwd/runtime storage, executable lookup, fully-specified managed child-process trees with raw or collected stdio, and one deep terminal-process primitive that owns PTY allocation, foreground groups, and complete session cleanup. Command defaulting, shell semantics, deadlines, protocol framing, readiness, and presentation stay with consumers — the [bash executors](../bash/README.md), [LSP host](../lsp/README.md), [PTY shell backend](../pty/README.md), [subprocess code runtime](../code-runtime/code-runtime-subprocess/README.md), and [ACP subagent backend](../subagent/README.md). See the [subprocess seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md). -| Package | Role | ctx key | +| Package | ctx key | Role | |---|---|---| -| [`subprocess/`](subprocess/README.md) | Defines subprocess launch, stream, termination, and disposal contracts | `ctx.subprocess` | -| [`subprocess-local/`](subprocess-local/README.md) | Implements local process-tree execution | registers on `ctx.subprocess` | +| [`subprocess`](subprocess/README.md) (`@deepseek-ai/dsh-subprocess`) | `ctx.subprocess` | The seam: execution-world coordinates and executable lookup, ordinary managed spawns, the terminal-process primitive, handle lifecycles, and shared environment/output vocabulary | +| [`subprocess-local`](subprocess-local/README.md) (`@deepseek-ai/dsh-subprocess-local`) | — | The local implementation: detached process trees, bounded collection/spill, `node-pty`, foreground/session inspection, tree signalling, runtime storage, and terminate-and-join disposal | -The service owns process lifetime; each consumer owns what the process does and which defaults apply. +The service owns process lifetime across consumer reloads; consumers own what a process means (a bash command, a future non-shell runner) and every default that shapes one. diff --git a/packages/subprocess/README.zh.md b/packages/subprocess/README.zh.md index dd3a975dae..f68bc1c720 100644 --- a/packages/subprocess/README.zh.md +++ b/packages/subprocess/README.zh.md @@ -1,12 +1,12 @@ -# subprocess/:子进程能力家族 +# subprocess/:进程管理能力家族 [English](README.md) | 中文 -本家族通过显式的进程生命周期服务运行宿主子进程。 +同一执行世界中的共享进程基底:规范化 cwd/运行时存储、可执行文件查找、采用原始或收集式 stdio 的完全显式受管子进程树,以及一项负责 PTY 分配、前台进程组和完整会话清理的深层终端进程原语。命令默认值补全、shell 语义、deadline、协议分帧、就绪检测与呈现留在消费方:[bash 执行器](../bash/README.md)、[LSP 主机](../lsp/README.md)、[PTY shell 后端](../pty/README.md)、[基于进程管理的 Code Runtime](../code-runtime/code-runtime-subprocess/README.md)与 [ACP(Agent Client Protocol)subagent 后端](../subagent/README.md)。参见[进程管理器 seam Agent Note(agent 决策记录)](../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md)。 -| 包 | 职责 | ctx 键 | +| 包(package) | ctx 键 | 角色 | |---|---|---| -| [`subprocess/`](subprocess/README.md) | 定义子进程启动、流、终止和 dispose(资源释放)契约 | `ctx.subprocess` | -| [`subprocess-local/`](subprocess-local/README.md) | 实现本地进程树执行 | 注册到 `ctx.subprocess` | +| [`subprocess`](subprocess/README.md)(`@deepseek-ai/dsh-subprocess`) | `ctx.subprocess` | seam 本体:执行世界坐标与可执行文件查找、普通受管 spawn、终端进程原语、句柄生命周期,以及共享的环境/输出词汇 | +| [`subprocess-local`](subprocess-local/README.md)(`@deepseek-ai/dsh-subprocess-local`) | 无 | 本地实现:detached 进程树、有界收集/spill、`node-pty`、前台/会话检查、进程树信号发送、运行时存储,以及先终止再等待退出的资源释放 | -服务负责进程生命周期;每个消费方负责进程执行的工作以及所应用的默认值。 +服务拥有跨消费方重载的进程存续期;消费方拥有一个进程的含义(一条 bash 命令、未来的非 shell 运行器)以及塑造它的每一项默认值。 diff --git a/packages/subprocess/subprocess-local/README.i18n.yaml b/packages/subprocess/subprocess-local/README.i18n.yaml index 28ebc2c110..d931d3574b 100644 --- a/packages/subprocess/subprocess-local/README.i18n.yaml +++ b/packages/subprocess/subprocess-local/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/subprocess/subprocess-local/README.md -README.md: af9f92db714398dd52c9b5e7aeb64d5af71021da -README.zh.md: da78cbcfbff174eb5fda5b8323fbdc84b50c9997 +README.md: 6bc1003ae5903bb5640728bc79c3f9042fddbe7a +README.zh.md: 3c9ce73c7fcbeec9b17d73f212ecdcb6842ec133 diff --git a/packages/subprocess/subprocess-local/README.md b/packages/subprocess/subprocess-local/README.md index af9f92db71..6bc1003ae5 100644 --- a/packages/subprocess/subprocess-local/README.md +++ b/packages/subprocess/subprocess-local/README.md @@ -2,14 +2,16 @@ English | [中文](README.zh.md) -Local implementation of the [`@deepseek-ai/dsh-subprocess`](../subprocess/README.md) seam: `LocalSubprocessService` spawns each spec's argv as a detached process tree, wires the spec's per-stream stdio dispositions (raw pipes, inherit, bounded tail-keep collection with optional spill files), and signals tree-scoped with SIGTERM→SIGKILL escalation. It has no config: every disposition, limit, and directory arrives on the spawn spec, so the deployment-varying knobs stay with the calling seams' configs ([`dsh-bash-local`](../../bash/bash-local/README.md), [`dsh-lsp-local`](../../lsp/lsp-local/README.md), [`dsh-subagent-acp`](../../subagent/subagent-acp/README.md)). +Local implementation of the [`@deepseek-ai/dsh-subprocess`](../subprocess/README.md) seam. `LocalSubprocessService` owns a private runtime directory, resolves local executables, spawns ordinary detached process trees with explicit stdio, and implements terminal processes through `node-pty` plus platform process inspection. It has no config: every disposition, limit, terminal dimension, grace, and directory arrives from the calling seams ([`dsh-bash-local`](../../bash/bash-local/README.md), [`dsh-lsp-local`](../../lsp/lsp-local/README.md), [`dsh-pty-local`](../../pty/pty-local/README.md), and [`dsh-code-runtime-subprocess`](../../code-runtime/code-runtime-subprocess/README.md)). ## Behavior (and where it came from) -- **Detached process trees with platform-correct signalling** — POSIX children are spawned `detached` (own process group) and signalled by negative pgid with a direct-child fallback; Windows terminates the tree via `taskkill /PID /T /F`. `terminate()` — the handle's only termination verb — sends SIGTERM then SIGKILL after the spec's grace (OpenCode's escalation; pipelines and subshells die with the parent) and is a no-op once the tree is gone; `waitForExit()` polls whole-tree liveness so consumer teardown confirms real quiescence. After the leader exits, still-open pipes receive the same bounded drain grace so a surviving descendant cannot hold the outcome open indefinitely. ESRCH is tolerated; daemons that re-parent away from the group can still survive — the same caveat as the surveyed tools. +- **Detached process trees with platform-correct signalling** — POSIX children are spawned `detached` (own process group) and signalled by negative pgid with a direct-child fallback; Windows terminates the tree via `taskkill /PID /T /F` (injectable for tests). `terminate()` — the handle's only termination verb — sends SIGTERM then SIGKILL after the spec's grace (OpenCode's escalation; pipelines and subshells die with the parent) and is a no-op once the tree is gone; `waitForExit()` polls whole-tree liveness so consumer teardown confirms real quiescence. After the leader exits, still-open pipes receive the same bounded drain grace so a surviving descendant cannot hold the outcome open indefinitely. ESRCH is tolerated; daemons that re-parent away from the group can still survive — the same caveat as the surveyed tools. - **Per-stream dispositions** — `'pipe'` hands the raw stream to the caller untouched (protocol framing stays consumer-owned); `'inherit'` passes the parent descriptor through; collect mode keeps the in-memory TAIL beyond its cap (errors and results cluster at the end — pi/OpenCode rationale) while the FULL stream is appended to a private temp file when a spill cap is configured — omitting `spill` keeps only the tail, the diagnostic shape. A stream larger than the spill cap discards its now-incomplete spill and returns only the marked truncated tail; spill fds are sealed at settlement, and a failed final close withholds the path rather than advertising an incomplete file. Spill files are `0600` with random names under a lazily-created `0700` per-process directory. -- **Credential scrub + explicit merge** — `process.env` minus credential-shaped vars (`*KEY*`/`*PASSWORD*`/`*SECRET*`/`*TOKEN*`) and all ambient `DSH_*` names; the spec's explicit `env` merges after that scrub with no namespace validation, so a deliberately supplied credential or current `DSH_*` fact wins while stale nested-harness identity cannot leak in ambiently. Supplied stdin is written and closed; otherwise fd 0 is `/dev/null`. See the [stdin/env Agent Note](../../../.agents/notes/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-surface.md) and [managed environment Agent Note](../../../.agents/notes/implemented/feature/2026-07-10-agent-session-identity-and-log-location.md). +- **Credential scrub + explicit merge** — `process.env` minus credential-shaped vars (`*KEY*`/`*SECRET*`/`*TOKEN*`) and all ambient `DSH_*` names; the spec's explicit `env` merges after that scrub with no namespace validation, so a deliberately supplied credential or current `DSH_*` fact wins while stale nested-harness identity cannot leak in ambiently. Supplied stdin is written and closed; otherwise fd 0 is `/dev/null`. See the [stdin/env Agent Note](../../../.agents/notes/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-surface.md) and [managed environment Agent Note](../../../.agents/notes/implemented/feature/2026-07-10-agent-session-identity-and-log-location.md). - **Offset-based reads** — collect-mode readers return deltas in whole-stream byte coordinates; the service never holds a cursor, so consumer-owned cursors (the bash background read path) and full-stream re-reads coexist, before and after settlement. +- **Execution-world coordinates** — `cwd` is the host process cwd, `runtimeRoot` is an owner-private temporary directory removed on disposal, and `resolveExecutable` checks absolute files or searches the scrubbed effective PATH with platform-aware executable extensions. +- **Terminal-process ownership** — `spawnTerminal` allocates `node-pty`, bridges UTF-8 terminal bytes, inspects and signals the current foreground process group, and cleans descendants before the top-level shell. Linux `/proc`/syscall and macOS `ps` inspectors retain exact pid/start identity so pid reuse cannot redirect cleanup; the higher PTY backend owns prompt readiness, buffers, and model-facing operations. - **Terminate-and-join disposal** — the service retains live handles only so its own disposal can escalate every running tree and await its exit; settled and spawn-failed handles leave the live set on settlement. ## Model Experience @@ -22,8 +24,10 @@ No direct invalidation; the named consumers own any request-prefix changes. ## Known Limitations and Deferred Work -- **Windows tree support is best-effort** — termination routes through `taskkill /PID /T /F` with all outcomes contained (absent tree, races, missing binary), and liveness falls back to the direct-child boundary. -- **The credential scrub is a name heuristic** — `*KEY*`/`*PASSWORD*`/`*SECRET*`/`*TOKEN*` only; differently-named secrets (e.g. `*PASSPHRASE*`) pass through, and a whitelist for over-scrubbed vars is noted future work. +- **Windows tree support is best-effort and untested in CI** — termination routes through `taskkill /PID /T /F` with all outcomes contained (absent tree, races, missing binary), and liveness falls back to the direct-child boundary; the suites cover the routing through an injected runner only, and `packages/subprocess/*` is excluded from the Windows test matrix. +- **Terminal process inspection is Linux/macOS only** — the terminal primitive fails when its inspector has no supported platform implementation; Linux exact probes cover x64 and arm64, while macOS uses `ps` snapshots. +- **A daemonized terminal descendant can escape the captured tree** — a child that reparents before teardown is no longer discoverable from the `node-pty` root. The local provider accepts this gap rather than signal the root PID's POSIX session, which can include unrelated launcher processes. +- **The credential scrub is a name heuristic** — `*KEY*`/`*SECRET*`/`*TOKEN*` only; differently-named secrets (e.g. `*PASSWORD*`) pass through, and a whitelist for over-scrubbed vars is noted future work. - **Completed spill files are not deleted** — bounded full-output recovery files (and the private per-process spill dir) accumulate under the OS tmpdir until something external cleans them; oversize incomplete spills are discarded and deletion is attempted immediately, but a cleanup failure can leave a bounded file behind. The raw process handling lives in `src/spawn.ts`; `src/index.ts` is the service wiring. diff --git a/packages/subprocess/subprocess-local/README.zh.md b/packages/subprocess/subprocess-local/README.zh.md index da78cbcfbf..3c9ce73c7f 100644 --- a/packages/subprocess/subprocess-local/README.zh.md +++ b/packages/subprocess/subprocess-local/README.zh.md @@ -2,15 +2,17 @@ [English](README.md) | 中文 -[`@deepseek-ai/dsh-subprocess`](../subprocess/README.md) seam 的本地实现:`LocalSubprocessService` 将每个 spec 的 argv spawn 为 detached 进程树,依照 spec 中按流划分的 stdio 处置方式(disposition)完成接线(原始管道、inherit、附带可选 spill 文件的有界尾部保留收集),并以进程树为范围发送信号,按 SIGTERM→SIGKILL 逐级升级。该实现没有任何配置:每项处置方式、限制与目录都随 spawn spec 传入,因此随部署变化的可调参数留在各调用方 seam 的配置里([`dsh-bash-local`](../../bash/bash-local/README.md)、[`dsh-lsp-local`](../../lsp/lsp-local/README.md)、[`dsh-subagent-acp`](../../subagent/subagent-acp/README.md))。 +[`@deepseek-ai/dsh-subprocess`](../subprocess/README.md) seam 的本地实现。`LocalSubprocessService` 拥有私有运行时目录,解析本地可执行文件,以显式 stdio spawn 普通 detached 进程树,并通过 `node-pty` 与平台进程检查实现终端进程。该实现没有任何配置:每项处置方式、限制、终端尺寸、宽限期与目录都来自调用方 seam([`dsh-bash-local`](../../bash/bash-local/README.md)、[`dsh-lsp-local`](../../lsp/lsp-local/README.md)、[`dsh-pty-local`](../../pty/pty-local/README.md)和 [`dsh-code-runtime-subprocess`](../../code-runtime/code-runtime-subprocess/README.md))。 ## 行为(以及设计来源) -- **以适合平台的方式发送信号的 detached 进程树**:POSIX 子进程使用 `detached` spawn(拥有独立进程组),信号以负 pgid 发送并以直接子进程作为回退;Windows 通过 `taskkill /PID /T /F` 终止进程树。`terminate()`(句柄唯一的终止操作)先发送 SIGTERM,经过 spec 的宽限期后再发送 SIGKILL(沿用 OpenCode 的升级策略;流水线与子 shell 会随父进程一起结束),进程树消亡后为空操作;`waitForExit()` 轮询整棵进程树的存活状态,使消费方的拆卸能确认真正的完全停稳。组长进程退出后,仍然打开的管道也只获得同样有界的排空宽限期,因此存活的后代进程无法无限期地拖住结果不结算。系统会容忍 ESRCH;重新指定父进程并脱离该组的 daemon 仍可能存活,这与所调研工具的局限相同。 -- **按流划分的处置方式**:`'pipe'` 把原始流原样交给调用方(协议分帧仍归消费方所有);`'inherit'` 直通父进程的描述符;收集模式(collect)在输出超过上限后于内存中保留尾部(错误与结果通常聚集在末尾,沿用 pi/OpenCode 的理由),并在配置了 spill 上限时把完整流追加到一个私有临时文件;省略 `spill` 则只保留用于诊断的尾部。某条流大于 spill 上限时,会丢弃已不完整的 spill,仅返回带截断标记的尾部;spill 文件描述符在结算时封存,最终关闭失败时则不公布路径,以免声称存在不完整的文件。spill 文件权限为 `0600`、名称随机,位于按需创建、权限为 `0700` 的每进程目录之下。 -- **凭据清除 + 显式合并**:以 `process.env` 为基础,移除形似凭据的变量(`*KEY*`/`*PASSWORD*`/`*SECRET*`/`*TOKEN*`)和所有环境中已有的 `DSH_*` 名称;spec 的显式 `env` 在该清除之后合并且不做命名空间校验,因此有意提供的凭据或当前 `DSH_*` 事实会胜出,而陈旧的嵌套 harness 身份无法从环境中隐式漏入。提供的 stdin 会被写入后关闭;否则 fd 0 指向 `/dev/null`。参见 [stdin/env Agent Note](../../../.agents/notes/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-surface.md)与[受管环境 Agent Note](../../../.agents/notes/implemented/feature/2026-07-10-agent-session-identity-and-log-location.md)。 -- **基于偏移量的读取**:收集模式的读取器按完整流的字节坐标返回增量;服务自身从不持有游标,因此消费方自有的游标(bash 的后台读取路径)与完整流重读可以共存,结算前后皆然。 -- **先终止再等待退出的 dispose(资源释放)**:服务保留存活句柄,只为让自身的 dispose 能对每个仍在运行的进程树执行升级并等待其退出;已结算与 spawn 失败的句柄在结算时即离开存活集合。 +- **带平台正确信号发送的 detached 进程树**:POSIX 子进程使用 `detached` spawn(拥有独立进程组),信号以负 pgid 发送并以直接子进程作为回退;Windows 通过 `taskkill /PID /T /F` 终止进程树(可为测试注入)。`terminate()`(句柄唯一的终止动词)先发送 SIGTERM,经过 spec 的宽限期后再发送 SIGKILL(沿用 OpenCode 的升级策略;管道与子 shell 会随父进程一起结束),进程树消亡后为空操作;`waitForExit()` 轮询整棵进程树的存活状态,使消费方的拆卸能确认真正的完全停稳。组长进程退出后,仍然打开的管道也只获得同样有界的排空宽限期,因此存活的后代进程无法无限期地拖住结果不结算。系统会容忍 ESRCH;脱离该组重新挂载的 daemon 仍可能存活,这与调研工具的局限相同。 +- **按流划分的处置方式**:`'pipe'` 把原始流原样交给调用方(协议分帧仍归消费方所有);`'inherit'` 直通父进程的描述符;收集模式(collect)在输出超过上限后于内存中保留尾部(错误与结果通常聚集在末尾,沿用 pi/OpenCode 的理由),并在配置了 spill 上限时把完整流追加到一个私有临时文件;省略 `spill` 则只保留尾部,即诊断尾部的形状。某条流大于 spill 上限时,会丢弃已不完整的 spill,仅返回带截断标记的尾部;spill 文件描述符在结算时封存,最终关闭失败时则不公布路径,以免声称存在不完整的文件。spill 文件权限为 `0600`、名称随机,位于按需延迟创建的 `0700` 每进程目录之下。 +- **凭据清除 + 显式合并**:以 `process.env` 为基础,移除形似凭据的变量(`*KEY*`/`*SECRET*`/`*TOKEN*`)和所有环境中已有的 `DSH_*` 名称;spec 的显式 `env` 在该清除之后合并且不做命名空间校验,因此有意提供的凭据或当前 `DSH_*` 事实会胜出,而陈旧的嵌套 harness 身份无法从环境中隐式漏入。提供的 stdin 会被写入后关闭;否则 fd 0 指向 `/dev/null`。参见 [stdin/env Agent Note(agent 决策记录)](../../../.agents/notes/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-surface.md)与[受管环境 Agent Note](../../../.agents/notes/implemented/feature/2026-07-10-agent-session-identity-and-log-location.md)。 +- **基于偏移量的读取**:收集模式的读取器以全流字节坐标返回增量;服务自身从不持有游标,因此消费方自有的游标(bash 的后台读取路径)与完整流重读可以共存,结算前后皆然。 +- **执行世界坐标**:`cwd` 是宿主进程 cwd,`runtimeRoot` 是所有者私有的临时目录,在资源释放时删除;`resolveExecutable` 检查绝对文件,或使用平台感知的可执行扩展名在清理后的有效 PATH 中查找。 +- **终端进程所有权**:`spawnTerminal` 分配 `node-pty`,桥接 UTF-8 终端字节,检查当前前台进程组并向其发送信号,并先于顶层 shell 清理后代。Linux 的 `/proc`/syscall 检查器与 macOS 的 `ps` 检查器会保留精确的 pid/启动身份,使 PID 复用无法把清理重定向到其他进程;上层 PTY 后端负责提示符就绪检测、缓冲和面向模型的操作。 +- **先终止再等待退出的 dispose**:服务保留存活句柄,只为让自身的 dispose 能对每个仍在运行的进程树执行升级并等待其退出;已结算与 spawn 失败的句柄在结算时即离开存活集合。 ## 模型体验 @@ -18,12 +20,14 @@ #### KV Cache 影响 -不会直接导致 KV Cache 失效;请求前缀变更由上述消费方负责。 +不会直接失效;请求前缀变更由具名消费方负责。 ## 已知限制与暂缓事项 -- **Windows 进程树支持仅为尽力而为**:终止经由 `taskkill /PID /T /F` 完成,所有结果都被就地吸收,不向外抛出(进程树已不存在、竞态、二进制缺失),存活探测则回退到直接子进程边界。 -- **凭据清除依赖名称启发式规则**:只匹配 `*KEY*`/`*PASSWORD*`/`*SECRET*`/`*TOKEN*`;名称不同的 secret(例如 `*PASSPHRASE*`)会继续传递,对误删变量引入白名单属于已记录的后续工作。 +- **Windows 进程树支持仅为尽力而为,且未经 CI 测试**:终止经由 `taskkill /PID /T /F` 完成,所有结果都被就地吸收,不向外抛出(进程树已不存在、竞态、二进制缺失),存活探测则回退到直接子进程边界;测试套件只通过注入的运行器覆盖这条路由,且 `packages/subprocess/*` 被排除在 Windows 测试矩阵之外。 +- **终端进程检查仅支持 Linux/macOS**:检查器没有受支持的平台实现时,终端原语会失败;Linux 精确探针覆盖 x64 与 arm64,macOS 使用 `ps` 快照。 +- **守护化的终端后代可能逃离已捕获进程树**:子进程若在拆卸前重新设定父进程,便无法再从 `node-pty` 根发现。本地提供方接受这个缺口,不向根 PID 的 POSIX 会话发送信号,因为其中可能包含无关的启动器进程。 +- **凭据清除依赖名称启发式规则**:只匹配 `*KEY*`/`*SECRET*`/`*TOKEN*`;名称不同的 secret(例如 `*PASSWORD*`)会继续传递,对误删变量引入白名单属于已记录的后续工作。 - **不会删除已完成的 spill 文件**:有界的完整输出恢复文件(以及每个进程的私有 spill 目录)会在 OS tmpdir 下累积,直到外部机制进行清理;超大的不完整 spill 会被丢弃并立即尝试删除,但清理失败可能留下一个有界文件。 原始进程处理位于 `src/spawn.ts`;`src/index.ts` 负责服务接线。 diff --git a/packages/subprocess/subprocess-local/package.json b/packages/subprocess/subprocess-local/package.json index e1429bffd0..7a68a9b91b 100644 --- a/packages/subprocess/subprocess-local/package.json +++ b/packages/subprocess/subprocess-local/package.json @@ -21,8 +21,12 @@ "files": [ "lib/index.js", "lib/invariant.js", + "scripts/ensure-spawn-helper.mjs", "lib/types/**/*.d.ts" ], + "scripts": { + "postinstall": "node scripts/ensure-spawn-helper.mjs" + }, "license": "BSD-3-Clause", "peerDependencies": { "@deepseek-ai/dsh-invariants": "^0.0.1", @@ -30,6 +34,9 @@ "@deepseek-ai/dsh-timeout": "^0.0.1", "cordis": "^4.0.0-rc.7" }, + "dependencies": { + "node-pty": "^1.1.0" + }, "devDependencies": { "@deepseek-ai/dsh-invariants": "workspace:^", "@deepseek-ai/dsh-subprocess": "workspace:^", diff --git a/packages/pty/pty-local/scripts/ensure-spawn-helper.mjs b/packages/subprocess/subprocess-local/scripts/ensure-spawn-helper.mjs similarity index 100% rename from packages/pty/pty-local/scripts/ensure-spawn-helper.mjs rename to packages/subprocess/subprocess-local/scripts/ensure-spawn-helper.mjs diff --git a/packages/subprocess/subprocess-local/src/index.ts b/packages/subprocess/subprocess-local/src/index.ts index eea1f1e739..40d2f52552 100644 --- a/packages/subprocess/subprocess-local/src/index.ts +++ b/packages/subprocess/subprocess-local/src/index.ts @@ -7,11 +7,26 @@ * @module @deepseek-ai/dsh-subprocess-local */ +import { constants } from 'node:fs' +import { mkdtempSync } from 'node:fs' +import { access, rm, stat } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { delimiter, extname, isAbsolute, join } from 'node:path' import { Context } from 'cordis' +import * as nodePty from 'node-pty' +import type { IPtyForkOptions } from 'node-pty' import { SubprocessService } from '@deepseek-ai/dsh-subprocess' -import type { SubprocessHandle, SubprocessSpawnSpec } from '@deepseek-ai/dsh-subprocess' -import { spawnSubprocess } from './spawn.ts' +import type { + SubprocessHandle, + SubprocessSpawnSpec, + SubprocessTerminalHandle, + SubprocessTerminalSpawnSpec, +} from '@deepseek-ai/dsh-subprocess' +import { childEnv, spawnSubprocess } from './spawn.ts' import type { SpawnInternals } from './spawn.ts' +import { createProcessInspector } from './process-inspector.ts' +import type { ProcessInspector } from './process-inspector.ts' +import { LocalTerminalHandle } from './terminal.ts' /** * Local subprocess service: detached process trees, Node-shaped stdio @@ -20,10 +35,16 @@ import type { SpawnInternals } from './spawn.ts' * SIGTERM→grace→SIGKILL escalation. */ export class LocalSubprocessService extends SubprocessService { + readonly cwd = process.cwd() + readonly runtimeRoot = mkdtempSync(join(tmpdir(), 'dsh-subprocess-runtime-')) /** Live handles retained only so disposal can terminate and join them. */ private live = new Set() + /** Live terminal sessions retained through whole-session quiescence. */ + private terminals = new Set() /** Test seam: spill and platform knobs forwarded to spawnSubprocess. */ internals: SpawnInternals = {} + /** Test seam for platform process inspection; production resolves lazily on terminal spawn. */ + terminalInspector: ProcessInspector | undefined constructor(ctx: Context) { super(ctx) @@ -37,11 +58,58 @@ export class LocalSubprocessService extends SubprocessService { // Spawn-failure rejections already settled and left the live set. pending.push(handle.done.catch(() => {}).then(() => handle.waitForExit())) } + for (const terminal of this.terminals) { + terminal.terminate() + // Cleanup may reject before the top-level process exits (for example, + // an identity-fenced descendant survives escalation). Await the cleanup + // transaction directly so disposal reports that failure rather than + // waiting forever on `done`. + pending.push(terminal.waitForExit()) + } this.live.clear() + this.terminals.clear() await Promise.all(pending) + await rm(this.runtimeRoot, { recursive: true, force: true }) }, 'local subprocess teardown') } + async resolveExecutable( + command: string, + env?: Readonly>, + signal?: AbortSignal, + ): Promise { + if (command.length === 0) throw new Error('subprocess-local: executable must be non-empty') + signal?.throwIfAborted() + const environment = childEnv(env) + const absolute = isAbsolute(command) + const candidates = absolute ? [command] : this.executableCandidates(command, environment) + for (const candidate of candidates) { + signal?.throwIfAborted() + try { + const info = await stat(candidate) + if (!info.isFile()) continue + await access(candidate, constants.X_OK) + signal?.throwIfAborted() + return candidate + } catch { + // Try the next PATH candidate; the final miss receives one stable error. + } + } + signal?.throwIfAborted() + throw new Error(absolute + ? `subprocess-local: command ${JSON.stringify(command)} is not an executable file` + : `subprocess-local: command ${JSON.stringify(command)} was not found on PATH`) + } + + private executableCandidates(command: string, env: NodeJS.ProcessEnv): string[] { + const path = env.PATH ?? '' + const extensions = process.platform === 'win32' && extname(command) === '' + ? (env.PATHEXT ?? '.COM;.EXE;.BAT;.CMD').split(';') + : [''] + return path.split(delimiter).flatMap(directory => + directory === '' ? [] : extensions.map(extension => join(directory, command + extension))) + } + spawn(spec: SubprocessSpawnSpec): SubprocessHandle { const handle = spawnSubprocess(spec, this.internals) this.live.add(handle) @@ -54,6 +122,38 @@ export class LocalSubprocessService extends SubprocessService { handle.done.then(release, release) return handle } + + // Local PTY allocation is synchronous, but the provider seam permits remote asynchronous allocation. + // eslint-disable-next-line @typescript-eslint/require-await + async spawnTerminal(spec: SubprocessTerminalSpawnSpec): Promise { + const file = spec.argv[0] + if (file === undefined || file.length === 0) { + throw new Error('subprocess-local: terminal argv must contain a program') + } + for (const [name, value] of [['rows', spec.rows], ['cols', spec.cols], ['graceMs', spec.graceMs]] as const) { + if (!Number.isSafeInteger(value) || value <= 0) { + throw new Error(`subprocess-local: terminal ${name} must be a positive safe integer`) + } + } + spec.signal?.throwIfAborted() + const options: IPtyForkOptions = { + name: 'dumb', + rows: spec.rows, + cols: spec.cols, + cwd: spec.cwd, + env: childEnv(spec.env), + } + const inspector = this.terminalInspector ?? createProcessInspector() + const terminal = nodePty.spawn(file, [...spec.argv.slice(1)], options) + const handle = new LocalTerminalHandle(terminal, inspector, spec.graceMs, spec.signal) + this.terminals.add(handle) + const release = async (): Promise => { + await handle.waitForExit() + this.terminals.delete(handle) + } + void handle.done.then(release, release).catch(() => {}) + return handle + } } export default LocalSubprocessService diff --git a/packages/pty/pty-local/src/process-inspector.ts b/packages/subprocess/subprocess-local/src/process-inspector.ts similarity index 96% rename from packages/pty/pty-local/src/process-inspector.ts rename to packages/subprocess/subprocess-local/src/process-inspector.ts index 5be25a224a..f57184f289 100644 --- a/packages/pty/pty-local/src/process-inspector.ts +++ b/packages/subprocess/subprocess-local/src/process-inspector.ts @@ -1,8 +1,8 @@ -/** Platform process-table inspection used for readiness, signals, and teardown. */ +/** Platform process-table inspection for terminal readiness, signals, and teardown. */ import { closeSync, openSync, readFileSync, readdirSync, readSync } from 'node:fs' import { execFileSync } from 'node:child_process' -import type { PtySignal } from '@deepseek-ai/dsh-pty' +import type { SubprocessTerminalSignal } from '@deepseek-ai/dsh-subprocess' /** PID plus start identity, preventing teardown escalation after PID reuse. */ export interface ProcessIdentity { @@ -18,7 +18,7 @@ export interface ProcessInspector { processTree(rootPid: number): ProcessIdentity[] /** Return whether the exact identity remains a non-quiescent process. */ isAlive(identity: ProcessIdentity): boolean - signalGroup(pgid: number, signal: PtySignal): void + signalGroup(pgid: number, signal: SubprocessTerminalSignal): void signalProcess(identity: ProcessIdentity, signal: 'SIGTERM' | 'SIGKILL'): void } @@ -203,7 +203,7 @@ abstract class PosixProcessInspector implements ProcessInspector { abstract processTree(rootPid: number): ProcessIdentity[] abstract isAlive(identity: ProcessIdentity): boolean - signalGroup(pgid: number, signal: PtySignal): void { + signalGroup(pgid: number, signal: SubprocessTerminalSignal): void { this.internals.kill(-pgid, signal) } @@ -327,5 +327,5 @@ export function createProcessInspector( ): ProcessInspector { if (platform === 'linux') return new LinuxProcessInspector(arch, internals) if (platform === 'darwin') return new MacProcessInspector(internals) - throw new Error(`pty-local: unsupported platform ${platform}`) + throw new Error(`subprocess-local: terminal inspection is unsupported on platform ${platform}`) } diff --git a/packages/subprocess/subprocess-local/src/terminal.ts b/packages/subprocess/subprocess-local/src/terminal.ts new file mode 100644 index 0000000000..d4b7bd26d1 --- /dev/null +++ b/packages/subprocess/subprocess-local/src/terminal.ts @@ -0,0 +1,226 @@ +/** Local node-pty terminal-process implementation for the subprocess seam. */ + +import { Buffer } from 'node:buffer' +import { constants } from 'node:os' +import { PassThrough } from 'node:stream' +import type { IDisposable, IPty } from 'node-pty' +import type { + SubprocessOutcome, + SubprocessTerminalForeground, + SubprocessTerminalHandle, + SubprocessTerminalSignal, +} from '@deepseek-ai/dsh-subprocess' +import type { ProcessIdentity, ProcessInspector } from './process-inspector.ts' + +function delay(ms: number): Promise { + return new Promise(resolve => setTimeout(resolve, ms)) +} + +function signalName(number: number | undefined): NodeJS.Signals | null { + if (number === undefined || number === 0) return null + for (const [name, value] of Object.entries(constants.signals)) { + if (value === number) return name as NodeJS.Signals + } + return null +} + +/** A local terminal whose process-session ownership stays below the PTY backend. */ +export class LocalTerminalHandle implements SubprocessTerminalHandle { + readonly pid: number + readonly output = new PassThrough() + readonly done: Promise + + private readonly outcome = Promise.withResolvers() + private readonly dataDisposable: IDisposable + private readonly exitDisposable: IDisposable + private exited = false + private termination: Promise | undefined + private removeAbort: (() => void) | undefined + + /** + * @param terminal - allocated node-pty process. + * @param inspector - platform process/session operations. + * @param graceMs - TERM-to-KILL and exit-wait grace. + * @param signal - optional lifetime cancellation. + */ + constructor( + private readonly terminal: IPty, + private readonly inspector: ProcessInspector, + private readonly graceMs: number, + signal?: AbortSignal, + ) { + this.pid = terminal.pid + this.done = this.outcome.promise + this.dataDisposable = terminal.onData((data) => { this.output.write(Buffer.from(data, 'utf8')) }) + this.exitDisposable = terminal.onExit(({ exitCode, signal: exitSignal }) => { + if (this.exited) return + this.exited = true + this.output.end() + this.outcome.resolve({ + exitCode: exitSignal === undefined || exitSignal === 0 ? exitCode : null, + signal: signalName(exitSignal), + }) + this.terminate() + }) + if (signal !== undefined) { + const onAbort = (): void => { this.terminate() } + signal.addEventListener('abort', onAbort, { once: true }) + this.removeAbort = () => { signal.removeEventListener('abort', onAbort) } + if (signal.aborted) this.terminate() + } + } + + // node-pty writes synchronously; the seam returns a promise for remote transports. + // eslint-disable-next-line @typescript-eslint/require-await + async write(data: Uint8Array): Promise { + if (this.exited) throw new Error('terminal process has exited') + let text: string + try { + text = new TextDecoder('utf-8', { fatal: true }).decode(data) + } catch (error: unknown) { + throw new Error('terminal input must be valid UTF-8', { cause: error }) + } + this.terminal.write(text) + } + + // Local inspection is synchronous; the seam returns a promise for remote transports. + // eslint-disable-next-line @typescript-eslint/require-await + async inspectForeground(): Promise { + const processGroupId = this.inspector.foregroundPgid(this.pid) + if (processGroupId === undefined) return undefined + return { + processGroupId, + inputWaiting: this.inspector.isStdinWaiting(processGroupId), + } + } + + async signalForeground(signal: SubprocessTerminalSignal): Promise { + const foreground = await this.inspectForeground() + if (foreground === undefined) { + throw new Error(`cannot resolve foreground process group for terminal ${this.pid}`) + } + if (signal === 'SIGKILL' && foreground.processGroupId === this.pid) { + throw new Error('refusing to SIGKILL the terminal shell; terminate the terminal session instead') + } + this.inspector.signalGroup(foreground.processGroupId, signal) + return foreground.processGroupId + } + + terminate(): void { + this.termination ??= this.closeOnce().catch((error: unknown) => { + this.termination = undefined + throw error + }) + void this.termination.catch(() => {}) + } + + async waitForExit(signal?: AbortSignal): Promise { + // A caller may begin waiting before the top-level process exits. The exit + // callback starts descendant cleanup in the same turn, so resolve that + // eventual transaction after `done` instead of snapshotting only `done`. + const quiescence = this.termination ?? this.done.then(() => this.termination) + if (signal === undefined) { + await quiescence + return true + } + if (signal.aborted) return false + return await new Promise((resolve, reject) => { + const onAbort = (): void => { cleanup(); resolve(false) } + const cleanup = (): void => { signal.removeEventListener('abort', onAbort) } + signal.addEventListener('abort', onAbort, { once: true }) + void quiescence.then( + () => { cleanup(); resolve(true) }, + (error: unknown) => { + cleanup() + // The owned cleanup transaction only throws Error diagnostics. + // eslint-disable-next-line @typescript-eslint/prefer-promise-reject-errors + reject(error) + }, + ) + }) + } + + private survivors(members: ProcessIdentity[]): ProcessIdentity[] { + return members.filter(member => this.inspector.isAlive(member)) + } + + private descendants(): ProcessIdentity[] { + return this.inspector.processTree(this.pid).filter(member => member.pid !== this.pid) + } + + private async waitForMembers(members: ProcessIdentity[]): Promise { + const until = Date.now() + this.graceMs + let survivors = this.survivors(members) + while (survivors.length > 0 && Date.now() < until) { + await delay(Math.min(25, Math.max(1, until - Date.now()))) + survivors = this.survivors(members) + } + return survivors + } + + private signalMembers(members: ProcessIdentity[], signal: 'SIGTERM' | 'SIGKILL'): void { + for (const member of members) { + try { + this.inspector.signalProcess(member, signal) + } catch (_alreadyExitedDuringSignal) { + // The exact process identity is rechecked; a same-tick exit is success. + } + } + } + + private unionMembers(...groups: ProcessIdentity[][]): ProcessIdentity[] { + const members: ProcessIdentity[] = [] + const seen = new Set() + for (const group of groups) { + for (const member of group) { + const key = `${member.pid}:${member.started}` + if (seen.has(key)) continue + seen.add(key) + members.push(member) + } + } + return members + } + + private async stopDescendants(): Promise { + const captured = this.descendants() + this.signalMembers(captured, 'SIGTERM') + const capturedSurvivors = await this.waitForMembers(captured) + const members = this.unionMembers(capturedSurvivors, this.descendants()) + this.signalMembers(members, 'SIGKILL') + const survivors = await this.waitForMembers(members) + return this.survivors(this.unionMembers(survivors, this.descendants())) + } + + private async stopShell(): Promise { + if (!this.exited) { + try { + this.terminal.kill('SIGTERM') + } catch (_topLevelAlreadyExitedDuringTerm) { + // The exit callback is authoritative. + } + await Promise.race([this.done.then(() => undefined), delay(this.graceMs)]) + } + if (!this.exited) { + try { + this.terminal.kill('SIGKILL') + } catch (_topLevelAlreadyExitedDuringKill) { + // The exit callback is authoritative. + } + await Promise.race([this.done.then(() => undefined), delay(this.graceMs)]) + } + if (!this.exited) throw new Error(`terminal cleanup failed; surviving pid: ${this.pid}`) + } + + private async closeOnce(): Promise { + const survivors = await this.stopDescendants() + if (survivors.length > 0) { + throw new Error(`terminal cleanup failed; surviving pids: ${survivors.map(member => member.pid).join(', ')}`) + } + await this.stopShell() + this.removeAbort?.() + this.removeAbort = undefined + this.dataDisposable.dispose() + this.exitDisposable.dispose() + } +} diff --git a/packages/subprocess/subprocess-local/tests/local.spec.ts b/packages/subprocess/subprocess-local/tests/local.spec.ts index ca723fdb87..bc77d24cfd 100644 --- a/packages/subprocess/subprocess-local/tests/local.spec.ts +++ b/packages/subprocess/subprocess-local/tests/local.spec.ts @@ -1,7 +1,10 @@ -import { describe, expect, it } from 'vitest' +import { PassThrough } from 'node:stream' +import { describe, expect, it, vi } from 'vitest' +import { stat } from 'node:fs/promises' +import { basename, delimiter, dirname } from 'node:path' import { Context } from 'cordis' import LocalSubprocessService from '@deepseek-ai/dsh-subprocess-local' -import type { SubprocessSpawnSpec } from '@deepseek-ai/dsh-subprocess' +import type { SubprocessSpawnSpec, SubprocessTerminalHandle, SubprocessTerminalSpawnSpec } from '@deepseek-ai/dsh-subprocess' function spec(command: string, overrides: Partial = {}): SubprocessSpawnSpec { return { @@ -18,6 +21,133 @@ function spec(command: string, overrides: Partial = {}): Su } describe('LocalSubprocessService', () => { + it('publishes execution-world paths and removes its private runtime directory', async () => { + const ctx = new Context() + const fiber = await ctx.plugin(LocalSubprocessService) + const root = ctx.subprocess.runtimeRoot + expect(ctx.subprocess.cwd).toBe(process.cwd()) + expect((await stat(root)).isDirectory()).toBe(true) + await fiber.dispose() + await expect(stat(root)).rejects.toMatchObject({ code: 'ENOENT' }) + }) + + it('resolves absolute and PATH executables and honors lookup cancellation', async () => { + const ctx = new Context() + const fiber = await ctx.plugin(LocalSubprocessService) + expect(await ctx.subprocess.resolveExecutable(process.execPath)).toBe(process.execPath) + expect(await ctx.subprocess.resolveExecutable(basename(process.execPath), { + PATH: dirname(process.execPath), + })).toBe(process.execPath) + await expect(ctx.subprocess.resolveExecutable('')).rejects.toThrow('must be non-empty') + await expect(ctx.subprocess.resolveExecutable('dsh-command-that-does-not-exist', { PATH: '' })) + .rejects.toThrow('was not found on PATH') + await expect(ctx.subprocess.resolveExecutable('/dsh-absolute-command-that-does-not-exist')) + .rejects.toThrow('is not an executable file') + await expect(ctx.subprocess.resolveExecutable(process.cwd())) + .rejects.toThrow('is not an executable file') + await expect(ctx.subprocess.resolveExecutable(process.execPath, {}, AbortSignal.abort('stop'))) + .rejects.toBe('stop') + await fiber.dispose() + }) + + it('builds Windows executable candidates without empty PATH entries', async () => { + const ctx = new Context() + const fiber = await ctx.plugin(LocalSubprocessService) + const service = ctx.subprocess as LocalSubprocessService + const candidates = (service as unknown as { + executableCandidates(command: string, env: NodeJS.ProcessEnv): string[] + }).executableCandidates.bind(service) + const platform = vi.spyOn(process, 'platform', 'get').mockReturnValue('win32') + try { + expect(candidates('tool', { PATH: `${delimiter}/bin`, PATHEXT: '.EXE;.CMD' })) + .toEqual(['/bin/tool.EXE', '/bin/tool.CMD']) + expect(candidates('tool.exe', {})).toEqual([]) + expect(candidates('tool', { PATH: '/bin' })).toHaveLength(4) + } finally { + platform.mockRestore() + await fiber.dispose() + } + }) + + it('validates terminal spawn specs before allocating a PTY', async () => { + const ctx = new Context() + const fiber = await ctx.plugin(LocalSubprocessService) + const base: SubprocessTerminalSpawnSpec = { + argv: ['bash'], cwd: process.cwd(), rows: 24, cols: 80, graceMs: 10, + } + await expect(ctx.subprocess.spawnTerminal({ ...base, argv: [] })).rejects.toThrow('must contain a program') + await expect(ctx.subprocess.spawnTerminal({ ...base, argv: [''] })).rejects.toThrow('must contain a program') + await expect(ctx.subprocess.spawnTerminal({ ...base, rows: 1.5 })).rejects.toThrow('rows') + await expect(ctx.subprocess.spawnTerminal({ ...base, cols: 0 })).rejects.toThrow('cols') + await expect(ctx.subprocess.spawnTerminal({ ...base, graceMs: 0 })).rejects.toThrow('graceMs') + await expect(ctx.subprocess.spawnTerminal({ ...base, signal: AbortSignal.abort('stop') })).rejects.toBe('stop') + await fiber.dispose() + }) + + it('terminates and joins an owned terminal during disposal', async () => { + const ctx = new Context() + const fiber = await ctx.plugin(LocalSubprocessService) + const terminate = vi.fn() + const waitForExit = vi.fn(async () => true) + const terminal: SubprocessTerminalHandle = { + pid: 1, + output: new PassThrough(), + done: Promise.resolve({ exitCode: 0, signal: null }), + write: async () => {}, + inspectForeground: async () => undefined, + signalForeground: async () => 1, + terminate, + waitForExit, + } + ;(ctx.subprocess as unknown as { terminals: Set }).terminals.add(terminal) + await fiber.dispose() + expect(terminate).toHaveBeenCalledOnce() + expect(waitForExit).toHaveBeenCalledOnce() + }) + + it('contains a terminal release failure after top-level exit', async () => { + let exitListener: ((event: { exitCode: number; signal?: number }) => void) | undefined + const terminal = { + pid: 123, + onData: () => ({ dispose: () => {} }), + onExit: (listener: (event: { exitCode: number; signal?: number }) => void) => { + exitListener = listener + return { dispose: () => {} } + }, + write: () => {}, + kill: () => {}, + } + vi.resetModules() + vi.doMock('node-pty', () => ({ spawn: () => terminal })) + try { + const { default: IsolatedLocalSubprocessService } = await import('../src/index.ts') + const ctx = new Context() + const fiber = await ctx.plugin(IsolatedLocalSubprocessService) + const alive = new Set([124]) + ;(ctx.subprocess as InstanceType).terminalInspector = { + foregroundPgid: () => 123, + isStdinWaiting: () => false, + processTree: () => [{ pid: 124, started: 'child' }], + isAlive: identity => alive.has(identity.pid), + signalGroup: () => {}, + signalProcess: () => {}, + } + const handle = await ctx.subprocess.spawnTerminal({ + argv: ['shell'], cwd: process.cwd(), rows: 24, cols: 80, graceMs: 1, + }) + exitListener?.({ exitCode: 0 }) + await handle.done + await new Promise(resolve => setTimeout(resolve, 10)) + alive.clear() + handle.terminate() + await handle.waitForExit() + await fiber.dispose() + } finally { + vi.doUnmock('node-pty') + vi.resetModules() + } + }) + it('registers as ctx.subprocess and spawns managed handles', async () => { const ctx = new Context() const fiber = await ctx.plugin(LocalSubprocessService) diff --git a/packages/pty/pty-local/tests/process-inspector.spec.ts b/packages/subprocess/subprocess-local/tests/process-inspector.spec.ts similarity index 98% rename from packages/pty/pty-local/tests/process-inspector.spec.ts rename to packages/subprocess/subprocess-local/tests/process-inspector.spec.ts index 218a3d77fe..e8ad2fd11e 100644 --- a/packages/pty/pty-local/tests/process-inspector.spec.ts +++ b/packages/subprocess/subprocess-local/tests/process-inspector.spec.ts @@ -1,6 +1,6 @@ import { describe, expect, it } from 'vitest' -import { createProcessInspector, parseProcStat } from '@deepseek-ai/dsh-pty-local/src/process-inspector.ts' -import type { ProcessInspectorInternals } from '@deepseek-ai/dsh-pty-local/src/process-inspector.ts' +import { createProcessInspector, parseProcStat } from '@deepseek-ai/dsh-subprocess-local/src/process-inspector.ts' +import type { ProcessInspectorInternals } from '@deepseek-ai/dsh-subprocess-local/src/process-inspector.ts' function stat(pid: number, pgrp: number, session: number, tpgid: number, started: string, parentPid = 1, state = 'S'): string { const rest = [state, String(parentPid), String(pgrp), String(session), '99', String(tpgid)] @@ -216,6 +216,6 @@ describe('macOS process inspector', () => { expect(createProcessInspector('darwin', 'arm64', fake.internals).foregroundPgid(1)).toBeUndefined() fake.internals.exec = () => { throw new Error('gone') } expect(createProcessInspector('darwin', 'arm64', fake.internals).foregroundPgid(1)).toBeUndefined() - expect(() => createProcessInspector('win32', 'x64', fake.internals)).toThrow('unsupported platform win32') + expect(() => createProcessInspector('win32', 'x64', fake.internals)).toThrow('unsupported on platform win32') }) }) diff --git a/packages/subprocess/subprocess-local/tests/terminal.spec.ts b/packages/subprocess/subprocess-local/tests/terminal.spec.ts new file mode 100644 index 0000000000..417fd62517 --- /dev/null +++ b/packages/subprocess/subprocess-local/tests/terminal.spec.ts @@ -0,0 +1,275 @@ +import { afterEach, describe, expect, it, vi } from 'vitest' +import type { IDisposable, IPty } from 'node-pty' +import { LocalTerminalHandle } from '@deepseek-ai/dsh-subprocess-local/src/terminal.ts' +import type { + ProcessIdentity, + ProcessInspector, +} from '@deepseek-ai/dsh-subprocess-local/src/process-inspector.ts' +import type { SubprocessTerminalSignal } from '@deepseek-ai/dsh-subprocess' + +class FakePty { + pid = 123 + readonly writes: string[] = [] + readonly kills: string[] = [] + autoExitOnKill = true + throwKill = false + private readonly dataListeners = new Set<(data: string) => void>() + private readonly exitListeners = new Set<(event: { exitCode: number; signal?: number }) => void>() + + readonly onData = (listener: (data: string) => void): IDisposable => { + this.dataListeners.add(listener) + return { dispose: () => { this.dataListeners.delete(listener) } } + } + + readonly onExit = (listener: (event: { exitCode: number; signal?: number }) => void): IDisposable => { + this.exitListeners.add(listener) + return { dispose: () => { this.exitListeners.delete(listener) } } + } + + emitData(data: string): void { + for (const listener of this.dataListeners) listener(data) + } + + emitExit(exitCode = 0, signal?: number): void { + for (const listener of this.exitListeners) listener({ exitCode, ...signal === undefined ? {} : { signal } }) + } + + write(data: string): void { this.writes.push(data) } + + kill(signal?: string): void { + if (this.throwKill) throw new Error('process raced') + this.kills.push(signal ?? 'SIGHUP') + if (this.autoExitOnKill) this.emitExit(0, signal === 'SIGKILL' ? 9 : 15) + } + + asPty(): IPty { + return this as unknown as IPty + } +} + +class FakeInspector implements ProcessInspector { + pgid: number | undefined = 456 + waiting = false + members: ProcessIdentity[] = [] + readonly alive = new Set() + readonly groups: Array<[number, SubprocessTerminalSignal]> = [] + readonly processes: Array<[number, 'SIGTERM' | 'SIGKILL']> = [] + throwGroup = false + throwProcess = false + removeOnSignal = true + + foregroundPgid() { return this.pgid } + isStdinWaiting() { return this.waiting } + processTree() { return this.members } + isAlive(identity: ProcessIdentity) { return this.alive.has(identity.pid) } + signalGroup(pgid: number, signal: SubprocessTerminalSignal) { + if (this.throwGroup) throw new Error('group failed') + this.groups.push([pgid, signal]) + } + signalProcess(identity: ProcessIdentity, signal: 'SIGTERM' | 'SIGKILL') { + if (this.throwProcess) throw new Error('process raced') + this.processes.push([identity.pid, signal]) + if (this.removeOnSignal) this.alive.delete(identity.pid) + } +} + +afterEach(() => { vi.useRealTimers() }) + +describe('LocalTerminalHandle', () => { + it('bridges terminal bytes, foreground control, and signalled exit facts', async () => { + const pty = new FakePty() + const inspector = new FakeInspector() + inspector.waiting = true + const handle = new LocalTerminalHandle(pty.asPty(), inspector, 10) + const chunks: Buffer[] = [] + handle.output.on('data', (chunk: Buffer) => { chunks.push(chunk) }) + + pty.emitData('hello €') + await handle.write(Buffer.from('input\r')) + expect(pty.writes).toEqual(['input\r']) + expect(await handle.inspectForeground()).toEqual({ processGroupId: 456, inputWaiting: true }) + expect(await handle.signalForeground('SIGINT')).toBe(456) + expect(inspector.groups).toEqual([[456, 'SIGINT']]) + + pty.emitExit(7, 9) + pty.emitExit(0) + expect(await handle.done).toEqual({ exitCode: null, signal: 'SIGKILL' }) + expect(await handle.waitForExit()).toBe(true) + expect(Buffer.concat(chunks).toString('utf8')).toBe('hello €') + }) + + it('rejects invalid input and unsafe foreground signals', async () => { + const pty = new FakePty() + const inspector = new FakeInspector() + const handle = new LocalTerminalHandle(pty.asPty(), inspector, 10) + await expect(handle.write(Uint8Array.from([0xff]))).rejects.toThrow('valid UTF-8') + + inspector.pgid = handle.pid + await expect(handle.signalForeground('SIGKILL')).rejects.toThrow('terminate the terminal session') + inspector.pgid = undefined + expect(await handle.inspectForeground()).toBeUndefined() + await expect(handle.signalForeground('SIGTERM')).rejects.toThrow('cannot resolve') + + pty.emitExit(3) + expect(await handle.done).toEqual({ exitCode: 3, signal: null }) + await handle.waitForExit() + await expect(handle.write(Buffer.from('late'))).rejects.toThrow('has exited') + }) + + it('keeps the shell alive until forced descendants leave', async () => { + vi.useFakeTimers() + const pty = new FakePty() + const inspector = new FakeInspector() + inspector.members = [{ pid: 124, started: 'child' }] + inspector.alive.add(124) + inspector.removeOnSignal = false + const handle = new LocalTerminalHandle(pty.asPty(), inspector, 20) + + handle.terminate() + const quiescent = handle.waitForExit() + await vi.advanceTimersByTimeAsync(20) + expect(inspector.processes).toContainEqual([124, 'SIGKILL']) + expect(pty.kills).toEqual([]) + + inspector.alive.delete(124) + await vi.advanceTimersByTimeAsync(20) + expect(await quiescent).toBe(true) + expect(pty.kills).toEqual(['SIGTERM']) + }) + + it('keeps an early exit wait pending through descendant cleanup', async () => { + vi.useFakeTimers() + const pty = new FakePty() + const inspector = new FakeInspector() + inspector.members = [{ pid: 124, started: 'child' }] + inspector.alive.add(124) + inspector.removeOnSignal = false + const handle = new LocalTerminalHandle(pty.asPty(), inspector, 20) + const waiting = handle.waitForExit() + let settled = false + void waiting.then(() => { settled = true }) + + pty.emitExit() + await vi.advanceTimersByTimeAsync(10) + expect(settled).toBe(false) + + inspector.alive.delete(124) + await vi.advanceTimersByTimeAsync(20) + expect(await waiting).toBe(true) + }) + + it('rescans for descendants forked during TERM', async () => { + const pty = new FakePty() + const inspector = new FakeInspector() + let reads = 0 + inspector.processTree = () => { + reads += 1 + if (reads === 1) { + inspector.alive.add(124) + return [{ pid: 124, started: 'first' }] + } + if (reads === 2) { + inspector.alive.add(125) + return [{ pid: 125, started: 'late' }] + } + return [] + } + const handle = new LocalTerminalHandle(pty.asPty(), inspector, 10) + handle.terminate() + await handle.waitForExit() + expect(inspector.processes).toEqual([[124, 'SIGTERM'], [125, 'SIGKILL']]) + expect(pty.kills).toEqual(['SIGTERM']) + }) + + it('retains captured descendants after reparenting', async () => { + vi.useFakeTimers() + const pty = new FakePty() + const inspector = new FakeInspector() + const captured = { pid: 124, started: 'captured' } + let reads = 0 + inspector.alive.add(captured.pid) + inspector.processTree = () => reads++ === 0 ? [captured] : [] + inspector.signalProcess = (identity, signal) => { + inspector.processes.push([identity.pid, signal]) + if (signal === 'SIGKILL') inspector.alive.delete(identity.pid) + } + const handle = new LocalTerminalHandle(pty.asPty(), inspector, 20) + handle.terminate() + const quiescent = handle.waitForExit() + await vi.advanceTimersByTimeAsync(25) + expect(await quiescent).toBe(true) + expect(inspector.processes).toEqual([[124, 'SIGTERM'], [124, 'SIGKILL']]) + }) + + it('allows cleanup to retry after a surviving descendant leaves', async () => { + vi.useFakeTimers() + const pty = new FakePty() + const inspector = new FakeInspector() + inspector.members = [{ pid: 124, started: 'child' }] + inspector.alive.add(124) + inspector.removeOnSignal = false + const handle = new LocalTerminalHandle(pty.asPty(), inspector, 10) + + handle.terminate() + const first = expect(handle.waitForExit(new AbortController().signal)).rejects.toThrow('surviving pids: 124') + await vi.advanceTimersByTimeAsync(25) + await first + + inspector.alive.delete(124) + handle.terminate() + expect(await handle.waitForExit()).toBe(true) + expect(pty.kills).toEqual(['SIGTERM']) + }) + + it('bounds waits and reports a top-level process that ignores escalation', async () => { + vi.useFakeTimers() + const pty = new FakePty() + pty.autoExitOnKill = false + const handle = new LocalTerminalHandle(pty.asPty(), new FakeInspector(), 10) + expect(await handle.waitForExit(AbortSignal.abort())).toBe(false) + const controller = new AbortController() + const bounded = handle.waitForExit(controller.signal) + controller.abort() + expect(await bounded).toBe(false) + + handle.terminate() + const failed = expect(handle.waitForExit()).rejects.toThrow('surviving pid: 123') + await vi.advanceTimersByTimeAsync(25) + await failed + expect(pty.kills).toEqual(['SIGTERM', 'SIGKILL']) + + pty.emitExit(0, 999) + expect(await handle.done).toEqual({ exitCode: null, signal: null }) + handle.terminate() + expect(await handle.waitForExit()).toBe(true) + }) + + it('contains process races and reacts to lifetime cancellation', async () => { + const pty = new FakePty() + pty.throwKill = true + const inspector = new FakeInspector() + inspector.members = [{ pid: 124, started: 'child' }] + inspector.alive.add(124) + inspector.throwProcess = true + const controller = new AbortController() + const handle = new LocalTerminalHandle(pty.asPty(), inspector, 1, controller.signal) + controller.abort() + const failed = expect(handle.waitForExit()).rejects.toThrow('surviving pids: 124') + await failed + + inspector.alive.delete(124) + pty.throwKill = false + handle.terminate() + await handle.waitForExit() + + const preAbortedPty = new FakePty() + const preAborted = new LocalTerminalHandle( + preAbortedPty.asPty(), + new FakeInspector(), + 1, + AbortSignal.abort('stop'), + ) + await preAborted.waitForExit() + expect(preAbortedPty.kills).toEqual(['SIGTERM']) + }) +}) diff --git a/packages/subprocess/subprocess/README.i18n.yaml b/packages/subprocess/subprocess/README.i18n.yaml index b178daafb2..84d7d4a666 100644 --- a/packages/subprocess/subprocess/README.i18n.yaml +++ b/packages/subprocess/subprocess/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/subprocess/subprocess/README.md -README.md: e59dd96df036826f36bd0286c977438d2d87d1cf -README.zh.md: e8fb89dfd1f8c41a0caefc96469c13d9ae7d415d +README.md: 84b4b0c11c74c96929fa97b58fb33156d44e6ef1 +README.zh.md: dbd80a1c975719884481501f5cc43798e464a4fb diff --git a/packages/subprocess/subprocess/README.md b/packages/subprocess/subprocess/README.md index e59dd96df0..84b4b0c11c 100644 --- a/packages/subprocess/subprocess/README.md +++ b/packages/subprocess/subprocess/README.md @@ -2,15 +2,17 @@ English | [中文](README.zh.md) -The subprocess seam (`ctx.subprocess`). The abstract `SubprocessService` exposes one method — `spawn(spec): SubprocessHandle` — plus the vocabulary shared by every consumer: the fully-explicit `SubprocessSpawnSpec`, `SubprocessHandle` with its non-consuming offset-based output readers, `SubprocessOutcome`, `CollectedOutput`, and the managed `DSH_*` environment namespace (`DSH_ENV_PREFIX`, `DshEnvironment`). The local implementation lives in [`dsh-subprocess-local`](../subprocess-local/README.md). +The subprocess seam (`ctx.subprocess`) is the process half of one execution world. The abstract `SubprocessService` exposes its canonical `cwd`, private `runtimeRoot`, executable lookup, ordinary managed `spawn`, and one terminal-process primitive; its vocabulary covers raw/collected stdio, process and terminal handles, exit facts, tree/session cleanup, and the managed `DSH_*` environment namespace. The local implementation lives in [`dsh-subprocess-local`](../subprocess-local/README.md). ## Contract - `spawn(spec)` returns immediately with a live handle; `done` resolves at process close with exit facts (`SubprocessOutcome` carries no output and no cause classification) and rejects only for spawn-level failures. -- The spec is fully explicit — argv, cwd, per-stream stdio dispositions, grace — because deployment-varying defaults belong to the calling seam's config, not to a hidden subprocess-service default (the `dsh-bash` request/spec split is the owning template). Grace must be positive, finite, and no greater than [`MAX_TIMER_DELAY_MS`](../../util/timeout/README.md), so the implementation can represent it with one Node timer instead of accepting a value that Node collapses to one millisecond. `argv` is never shell-interpreted; a consumer that wants a shell passes `['bash', '-c', command]` itself. +- `cwd` and `runtimeRoot` are absolute paths in the provider's execution world. Consumers materialize private helpers below `runtimeRoot`, never in a host-only temp directory. `resolveExecutable(command, env?, signal?)` verifies absolute commands or resolves bare names against that world's scrubbed PATH plus explicit overrides. +- The spec is fully explicit — argv, cwd, per-stream stdio dispositions, grace — because deployment-varying defaults belong to the calling seam's config, not to a hidden subprocess-service default (the `dsh-bash` request/spec split is the owning template). `argv` is never shell-interpreted; a consumer that wants a shell passes `['bash', '-c', command]` itself. - Stdio is Node-shaped per stream: `'pipe'` hands the caller the raw stream for its own protocol framing (LSP JSON-RPC, ACP ndjson), `'inherit'` passes the parent descriptor through for diagnostics, and collect mode (`{ maxBytes, spill? }`) buffers a bounded tail with an optional full-stream spill file. Collect readers take whole-stream byte offsets and never consume, so independent readers cannot steal one another's deltas; a read whose offset slid out of the in-memory tail is `lossy` and points at the spill file when one exists. Collected output stays readable after settlement. - Termination is tree-scoped on every platform (POSIX detached groups with direct-child fallback; Windows `taskkill /T`): `terminate()` — the only termination verb — escalates SIGTERM→grace→SIGKILL (idempotent, driven by the spec's abort signal too, a no-op once the tree is gone), and `waitForExit(signal?)` observes whole-tree liveness so a consumer-owned teardown ladder holds each tier on real quiescence — the manager reacts but never classifies why (callers own deadlines, teardown ladders, and cause classification). -- `scrubbedParentEnv()` / `SENSITIVE_ENV_PATTERN` are the one shared scrub definition: ambient credential-shaped and `DSH_*` names are dropped, and the spec's explicit `env` merges after the scrub with no namespace validation — a string deliberately forwards or overrides a value, while an `undefined` tombstone removes an ordinary ambient entry. Spawners that cannot route through the service (node-pty backends, SDK-managed transports) import the scrub. +- `spawnTerminal(spec)` is the only non-pipe primitive. Its handle owns a real PTY, valid-UTF-8 byte I/O, foreground-process-group inspection/signalling, TERM-to-KILL whole-session cleanup, and a quiescence wait. The output stream ends after queued output when the top-level process exits; a live transport failure rejects `done`. These operations remain one substrate primitive because ordinary pipes cannot allocate a controlling terminal or prove and clean the complete terminal session; readiness, scrollback, and owner policy remain in the PTY consumer. +- `scrubbedParentEnv()` / `SENSITIVE_ENV_PATTERN` are the one shared scrub definition: ambient credential-shaped and `DSH_*` names are dropped, and explicit `env` merges after the scrub. The local ordinary and terminal spawns both apply it; SDK-managed transports that own their spawn may import it directly. - Disposal of the service terminates all still-running managed processes and awaits their exit. See the [subprocess data-structure catalog](../../../docs/core-data-structures/subprocess.md) and the [seam Agent Note](../../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md). @@ -25,5 +27,5 @@ No direct invalidation; the named consumers own any request-prefix changes. ## Known Limitations and Deferred Work -- **node-pty and SDK-managed spawns share only the scrub** — the PTY backend's terminal fork and the MCP SDK's own stdio transport cannot route their spawns through this seam (the library owns the fork/spawn call); they import `scrubbedParentEnv` so the environment policy stays single-sourced. +- **SDK-managed spawns remain outside** — an SDK transport that owns its internal spawn cannot route that call through this service; it can still import `scrubbedParentEnv` so environment policy stays single-sourced. - **Teardown ladders are consumer-owned** — the seam ships signalling verbs and the tree-liveness wait, not a canned quiesce sequence; each out-of-process consumer encodes its child's cooperation shape itself (the ACP backend's stdin-EOF-first ladder is the in-repo template). diff --git a/packages/subprocess/subprocess/README.zh.md b/packages/subprocess/subprocess/README.zh.md index e8fb89dfd1..dbd80a1c97 100644 --- a/packages/subprocess/subprocess/README.zh.md +++ b/packages/subprocess/subprocess/README.zh.md @@ -2,28 +2,30 @@ [English](README.md) | 中文 -子进程 seam(`ctx.subprocess`)。抽象的 `SubprocessService` 只暴露一个方法:`spawn(spec): SubprocessHandle`,外加所有消费方共享的词汇:完全显式的 `SubprocessSpawnSpec`、携带基于偏移量的非消费式输出读取器的 `SubprocessHandle`、`SubprocessOutcome`、`CollectedOutput`,以及受管的 `DSH_*` 环境命名空间(`DSH_ENV_PREFIX`、`DshEnvironment`)。本地实现位于 [`dsh-subprocess-local`](../subprocess-local/README.md)。 +进程管理器 seam(`ctx.subprocess`)是同一执行世界中的进程侧。抽象的 `SubprocessService` 公开其规范化 `cwd`、私有 `runtimeRoot`、可执行文件查找、普通受管 `spawn` 和一项终端进程原语;其词汇涵盖原始/收集式 stdio、进程与终端句柄、退出事实、进程树/会话清理,以及受管的 `DSH_*` 环境命名空间。本地实现位于 [`dsh-subprocess-local`](../subprocess-local/README.md)。 ## 契约 -- `spawn(spec)` 立即返回一个活动句柄;`done` 在进程关闭时以退出事实 resolve(`SubprocessOutcome` 不携带输出,也不携带原因分类),仅在 spawn 层面失败时 reject。 -- spec 完全显式(argv、cwd、按流划分的 stdio 处置方式(disposition)、宽限期),因为随部署变化的默认值属于调用方 seam 的配置,而不属于某个隐藏的子进程默认值(`dsh-bash` 的 request/spec 拆分是这条规则的所属模板)。宽限期须为正有限值,且不得大于 [`MAX_TIMER_DELAY_MS`](../../util/timeout/README.md),这样实现便可用一个 Node 定时器表示它,而不会接受会被 Node 折叠为 1 毫秒的值。`argv` 绝不经过 shell 解释;需要 shell 的消费方自行传入 `['bash', '-c', command]`。 -- stdio 按流采用 Node 风格:`'pipe'` 把原始流交给调用方做自己的协议分帧(LSP 的 JSON-RPC、ACP(Agent Client Protocol)的 ndjson),`'inherit'` 直通父进程描述符以承载诊断输出,收集模式(collect)`{ maxBytes, spill? }` 则缓冲一段有界尾部,外加可选的完整流 spill 文件。收集模式的读取器接受全流字节偏移量且从不消费,因此独立的读取器不会抢走彼此的增量;偏移量滑出内存尾部窗口的读取标记为 `lossy`,并在 spill 文件存在时指向它。收集到的输出在结算后仍可读取。 +- `spawn(spec)` 立即返回一个实时句柄;`done` 在进程关闭时以退出事实 resolve(`SubprocessOutcome` 不携带输出,也不携带原因分类),仅在 spawn 层面失败时 reject。 +- `cwd` 与 `runtimeRoot` 是提供方执行世界中的绝对路径。消费方在 `runtimeRoot` 下物化私有辅助程序,绝不使用仅宿主可见的临时目录。`resolveExecutable(command, env?, signal?)` 验证绝对命令,或根据该执行世界清理后的 PATH 加显式覆盖来解析裸名称。 +- spec 完全显式(argv、cwd、按流划分的 stdio 处置方式(disposition)、宽限期),因为随部署变化的默认值属于调用方 seam 的配置,而不属于某个隐藏的进程管理器默认值(`dsh-bash` 的 request/spec 拆分是这条规则的所属模板)。`argv` 绝不经过 shell 解释;需要 shell 的消费方自行传入 `['bash', '-c', command]`。 +- stdio 按流采用 Node 形状:`'pipe'` 把原始流交给调用方做自己的协议分帧(LSP 的 JSON-RPC、ACP(Agent Client Protocol)的 ndjson),`'inherit'` 直通父进程描述符以承载诊断输出,收集模式(collect)`{ maxBytes, spill? }` 则缓冲一段有界尾部,外加可选的完整流 spill 文件。收集模式的读取器接受全流字节偏移量且从不消费,因此独立的读取器不会抢走彼此的增量;偏移量滑出内存尾部窗口的读取标记为 `lossy`,并在 spill 文件存在时指向它。收集到的输出在结算后仍可读取。 - 终止在每个平台上都以进程树为范围(POSIX 用 detached 进程组并以直接子进程回退;Windows 用 `taskkill /T`):`terminate()`(唯一的终止动词)执行 SIGTERM→宽限期→SIGKILL 升级(幂等,也由 spec 的 abort 信号驱动,进程树消亡后为空操作);`waitForExit(signal?)` 观察整棵进程树的存活状态,使消费方自有的拆卸阶梯能在真正完全停稳后才进入下一层。管理器只响应中止,但绝不判定原因(deadline、拆卸阶梯与原因分类归调用方所有)。 -- `scrubbedParentEnv()` / `SENSITIVE_ENV_PATTERN` 是唯一一份共享的环境清理定义:环境中形似凭据的名称与 `DSH_*` 名称都会被丢弃,spec 的显式 `env` 在清理后合并且不做命名空间校验——字符串会有意转发或覆盖某个值,而 `undefined` tombstone 则会删除普通的环境条目。无法把 spawn 路由到该服务的进程启动方(node-pty 后端、由 SDK 管理的传输层)会导入该环境清理定义。 +- `spawnTerminal(spec)` 是唯一的非管道原语。其句柄负责真实 PTY、有效 UTF-8 字节 I/O、前台进程组检查/信号发送、TERM→KILL 全会话清理,以及等待完全停稳。顶层进程退出后,输出流会在排完队列中的输出后结束;存活期间的传输故障会拒绝 `done`。这些操作仍属于一项基底原语,因为普通管道无法分配控制终端,也无法证明并清理完整的终端会话;就绪检测、scrollback 与所有者策略仍归 PTY 消费方所有。 +- `scrubbedParentEnv()` / `SENSITIVE_ENV_PATTERN` 是唯一一份共享的凭据清除定义:环境中形似凭据的名称与 `DSH_*` 名称都会被丢弃,显式 `env` 在清除之后合并。本地普通 spawn 与终端 spawn 都应用这一定义;自行拥有 spawn 的 SDK 管理传输层可以直接导入它。 - 服务自身的 dispose(资源释放)会终止所有仍在运行的受管进程并等待其退出。 -参见[子进程数据结构目录](../../../docs/core-data-structures/subprocess.md)与[seam Agent Note](../../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md)。 +参见[进程管理器数据结构目录](../../../docs/core-data-structures/subprocess.md)与 [seam Agent Note(agent 决策记录)](../../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md)。 ## 模型体验 -通过消费方 seam 间接影响(目前是 `dsh-tool-bash` 背后的 bash 执行器家族);进程输出和生命周期的全部面向模型渲染均由消费方负责。 +通过消费方 seam 间接影响(目前是 `dsh-tool-bash` 背后的 bash 执行器家族);进程输出与生命周期面向模型的全部渲染归消费方所有。 #### KV Cache 影响 -不会直接导致 KV Cache 失效;请求前缀变更由上述消费方负责。 +不会直接失效;请求前缀变更由具名消费方负责。 ## 已知限制与暂缓事项 -- **node-pty 与由 SDK 管理的 spawn 只共享环境清理**:PTY 后端的终端 fork 与 MCP SDK 自己的 stdio 传输层无法把 spawn 路由到这道 seam(fork/spawn 调用归库所有);它们改为导入 `scrubbedParentEnv`,使环境策略保持单一来源。 -- **拆卸阶梯归消费方所有**:该 seam 只提供信号动词与进程树存活等待,不提供现成的停稳序列;每个进程外消费方自行编码其子进程的配合方式(ACP 后端以 stdin EOF 打头的阶梯是仓库内模板)。 +- **由 SDK 管理的 spawn 仍在服务之外**:自行拥有内部 spawn 的 SDK 传输层无法经该服务路由这次调用;它仍可导入 `scrubbedParentEnv`,使环境策略保持单一来源。 +- **拆卸阶梯归消费方所有**:该 seam 只提供信号动词与进程树存活等待,不提供现成的停稳序列;每个进程外消费方自行编码其子进程的配合形状(ACP 后端以 stdin EOF 打头的阶梯是仓库内模板)。 diff --git a/packages/subprocess/subprocess/src/index.ts b/packages/subprocess/subprocess/src/index.ts index d484faa24d..3c58a07a9e 100644 --- a/packages/subprocess/subprocess/src/index.ts +++ b/packages/subprocess/subprocess/src/index.ts @@ -1,10 +1,9 @@ /** - * The subprocess seam (`ctx.subprocess`): spawn fully-specified commands into - * managed process trees with Node-shaped stdio dispositions — raw pipes for - * protocol streams, inherit for diagnostics, bounded spill-backed collection - * for batch output — plus tree-scoped signalling. Command defaulting, shell - * semantics, deadlines, teardown ladders, framing, and presentation belong to - * consumers; the bash executor seam is the owning template. The local implementation lives in + * The subprocess seam (`ctx.subprocess`): execution-world process coordinates, + * executable lookup, fully specified managed process trees with raw or + * collected stdio, and one terminal-process primitive. Command defaulting, + * shell semantics, deadlines, protocol framing, terminal readiness, and + * presentation belong to consumers. The local implementation lives in * `@deepseek-ai/dsh-subprocess-local`. * @module @deepseek-ai/dsh-subprocess */ @@ -12,6 +11,7 @@ import { Context, Service } from 'cordis' import { DSH_ENV_PREFIX } from './types.ts' import type { SubprocessHandle, SubprocessSpawnSpec } from './types.ts' +import type { SubprocessTerminalHandle, SubprocessTerminalSpawnSpec } from './types.ts' export { DSH_ENV_PREFIX } from './types.ts' export type { @@ -28,6 +28,10 @@ export type { SubprocessSpawnSpec, SubprocessStdinMode, SubprocessStdio, + SubprocessTerminalForeground, + SubprocessTerminalHandle, + SubprocessTerminalSignal, + SubprocessTerminalSpawnSpec, } from './types.ts' /** @@ -74,6 +78,8 @@ declare module 'cordis' { * duplicate-service behavior). * * Implementations must honor these semantics: + * - {@link cwd}, {@link runtimeRoot}, and executable paths belong to one + * execution world shared with the mounted filesystem provider. * - {@link spawn} returns immediately with a live handle; `done` resolves at * process close with exit facts and rejects only for spawn-level failures. * - Collect-mode readers are offset-based and non-consuming, so independent @@ -87,12 +93,37 @@ declare module 'cordis' { * quiescence. * - Disposal of the service terminates all still-running managed processes * and awaits their exit. + * - {@link spawnTerminal} owns terminal allocation, byte transport, + * foreground groups, signalling, and whole-session quiescence; readiness + * and persistent-shell policy stay in the PTY consumer. Its output stream + * ends after queued terminal output when the top-level process exits. */ export abstract class SubprocessService extends Service { constructor(ctx: Context) { super(ctx, 'subprocess') } + /** Canonical default cwd in this provider's execution world. */ + abstract readonly cwd: string + + /** Private directory for runtime artifacts in this provider's execution world. */ + abstract readonly runtimeRoot: string + + /** + * Resolve one configured executable in this provider's execution world. + * Absolute paths are verified; bare names use the provider's scrubbed PATH + * plus explicit environment overrides. + * @param command - absolute executable path or bare PATH name. + * @param env - explicit environment entries used for lookup. + * @param signal - aborts remote or local lookup. + * @returns a canonical executable path. + */ + abstract resolveExecutable( + command: string, + env?: Readonly>, + signal?: AbortSignal, + ): Promise + /** * Start one managed child process from a fully-specified spec; this seam * applies no defaults. @@ -100,6 +131,15 @@ export abstract class SubprocessService extends Service { * @returns the live process handle (streams/readers, signalling, outcome promise). */ abstract spawn(spec: SubprocessSpawnSpec): SubprocessHandle + + /** + * Allocate a real terminal and start one owned process session. This is the + * only non-pipe process primitive: implementations own terminal byte I/O, + * foreground groups, signals, and complete session-tree cleanup. + * @param spec - fully specified argv, cwd, environment, dimensions, grace, and cancellation. + * @returns the live terminal handle after allocation succeeds. + */ + abstract spawnTerminal(spec: SubprocessTerminalSpawnSpec): Promise } export default SubprocessService diff --git a/packages/subprocess/subprocess/src/types.ts b/packages/subprocess/subprocess/src/types.ts index 6084cc8c8e..206104ad5b 100644 --- a/packages/subprocess/subprocess/src/types.ts +++ b/packages/subprocess/subprocess/src/types.ts @@ -192,3 +192,71 @@ export interface SubprocessHandle { */ waitForExit(signal?: AbortSignal): Promise } + +/** Signals supported by the terminal-process primitive. */ +export type SubprocessTerminalSignal = 'SIGINT' | 'SIGTERM' | 'SIGKILL' | 'SIGTSTP' | 'SIGHUP' + +/** A fully specified terminal-process spawn. */ +export interface SubprocessTerminalSpawnSpec { + /** Executable and arguments; `argv[0]` is the program. */ + argv: readonly string[] + /** Working directory in this subprocess provider's execution world. */ + cwd: string + /** Explicit environment layered after the provider's ambient scrub. */ + env?: Record | undefined + /** Initial terminal row count. */ + rows: number + /** Initial terminal column count. */ + cols: number + /** TERM-to-KILL cleanup grace for the complete terminal session. */ + graceMs: number + /** Cancellation of setup or the live terminal session. */ + signal?: AbortSignal | undefined +} + +/** Current foreground process-group facts for one terminal. */ +export interface SubprocessTerminalForeground { + /** Foreground process-group id published by the terminal driver. */ + processGroupId: number + /** Whether the provider can currently prove that group is waiting on terminal input. */ + inputWaiting: boolean +} + +/** + * One live terminal process and its owned OS session. Terminal allocation, + * foreground-group inspection/signalling, and session-tree cleanup are one + * deep subprocess primitive because none can be reconstructed from ordinary + * piped stdio without substrate-specific process control. + */ +export interface SubprocessTerminalHandle { + /** Top-level terminal process id. */ + readonly pid: number + /** UTF-8 terminal output bytes in delivery order; ends after queued output when the terminal exits. */ + readonly output: Readable + /** Resolves when the top-level process exits; rejects only for a live transport failure. */ + readonly done: Promise + /** + * Write bytes to the terminal input. + * @param data - valid UTF-8 bytes to deliver without implicit newline conversion. + */ + write(data: Uint8Array): Promise + /** + * Inspect the current foreground process group. + * @returns its id and input-wait fact, or undefined when no foreground group can be resolved. + */ + inspectForeground(): Promise + /** + * Deliver a signal to the current foreground process group. + * @param signal - permitted terminal signal. + * @returns the exact group id that received it. + */ + signalForeground(signal: SubprocessTerminalSignal): Promise + /** Begin idempotent TERM-to-KILL cleanup of the complete terminal session. */ + terminate(): void + /** + * Await whole-session quiescence, not only top-level process exit. + * @param signal - optional bound for this wait. + * @returns true after quiescence, false when `signal` aborts first. + */ + waitForExit(signal?: AbortSignal): Promise +} diff --git a/packages/subprocess/subprocess/tests/service.spec.ts b/packages/subprocess/subprocess/tests/service.spec.ts index d0ef5c9fd6..853b333400 100644 --- a/packages/subprocess/subprocess/tests/service.spec.ts +++ b/packages/subprocess/subprocess/tests/service.spec.ts @@ -1,7 +1,14 @@ import { describe, expect, it } from 'vitest' +import { PassThrough } from 'node:stream' import { Context } from 'cordis' import { scrubbedParentEnv, SubprocessService } from '@deepseek-ai/dsh-subprocess' -import type { SubprocessHandle, SubprocessOutputRead, SubprocessSpawnSpec } from '@deepseek-ai/dsh-subprocess' +import type { + SubprocessHandle, + SubprocessOutputRead, + SubprocessSpawnSpec, + SubprocessTerminalHandle, + SubprocessTerminalSpawnSpec, +} from '@deepseek-ai/dsh-subprocess' /** * Minimal concrete service: a hand-built handle. The seam is spawn-only — @@ -9,6 +16,13 @@ import type { SubprocessHandle, SubprocessOutputRead, SubprocessSpawnSpec } from * is all an implementation owes the abstract class. */ class StubSubprocessService extends SubprocessService { + readonly cwd = '/stub' + readonly runtimeRoot = '/stub/.runtime' + + async resolveExecutable(command: string): Promise { + return `/bin/${command}` + } + spawn(spec: SubprocessSpawnSpec): SubprocessHandle { const read: SubprocessOutputRead = { text: '', nextOffset: 0, lossy: false } const collected = spec.stdio.stdout !== 'pipe' && spec.stdio.stdout !== 'inherit' @@ -25,6 +39,19 @@ class StubSubprocessService extends SubprocessService { waitForExit: () => Promise.resolve(true), } } + + async spawnTerminal(spec: SubprocessTerminalSpawnSpec): Promise { + return { + pid: spec.argv.length, + output: new PassThrough(), + done: Promise.resolve({ exitCode: 0, signal: null }), + write: async () => {}, + inspectForeground: async () => ({ processGroupId: 1, inputWaiting: true }), + signalForeground: async () => 1, + terminate: () => {}, + waitForExit: async () => true, + } + } } describe('SubprocessService seam', () => { diff --git a/scripts/check-workspace-constraints.ts b/scripts/check-workspace-constraints.ts index 355b817e68..b20d5b5f3b 100644 --- a/scripts/check-workspace-constraints.ts +++ b/scripts/check-workspace-constraints.ts @@ -148,6 +148,7 @@ function expectedDshPackageFiles(manifest: PackageManifest): readonly string[] { 'lib/invariant.js', ...manifest.bin ? ['lib/bin.js'] : [], ...manifest.exports?.['./worker'] ? ['lib/worker.cjs'] : [], + ...exportDefault(manifest, './runtime-host') === './lib/runtime-host.js' ? ['lib/runtime-host.js'] : [], // UI plugin packages ship their browser bundle beside the node lib // (single-artifact ruling: dist/ retired, ./client resolves lib/client.js). // Keyed on the artifact path, not the subpath name: apiproxy's ./client is diff --git a/scripts/gen-cordis-catalog.ts b/scripts/gen-cordis-catalog.ts index 6d9665236a..256ae90caa 100644 --- a/scripts/gen-cordis-catalog.ts +++ b/scripts/gen-cordis-catalog.ts @@ -73,6 +73,8 @@ export const LINK_MAP: Readonly> = { SubprocessOutputRead: 'subprocess.md', SubprocessOutputReader: 'subprocess.md', SubprocessSpawnSpec: 'subprocess.md', + SubprocessTerminalHandle: 'subprocess.md', + SubprocessTerminalSpawnSpec: 'subprocess.md', CodeRunRequest: 'code-runtime.md', CodeRunResult: 'code-runtime.md', CompactionResult: 'compaction.md', @@ -242,6 +244,7 @@ export const FOUNDATION_TYPE_NAMES: ReadonlySet = new Set([ 'Partial', 'Pick', 'Promise', + 'Record', 'Readonly', ]) diff --git a/scripts/run-gates.ts b/scripts/run-gates.ts index 77432e7dbe..74a09a874e 100644 --- a/scripts/run-gates.ts +++ b/scripts/run-gates.ts @@ -623,11 +623,12 @@ function builtBinSmokeGate(needs: string[] = ['build']): Gate { 'packages/subagent/subagent-codex/tests/loader-composition.e2e.ts', 'packages/subagent/subagent-claude-code/tests/loader-composition.e2e.ts', 'packages/api/remotes/tests/built-lib.e2e.ts', - // The worker-entry packages' built bundles: the only automated proof - // that lib/index.js resolves its sibling lib/worker.cjs under plain node - // (the e2e lane runs unbuilt, so these files self-skip there). + // Built execution consumers: the only automated proof that package-name + // imports reach their lib/ entrypoints under plain Node. The e2e lane runs + // unbuilt, so these files self-skip there. 'packages/workflow/workflow-workerthread/tests/built-worker.e2e.ts', 'packages/code-runtime/code-runtime-worker/tests/built-lib.e2e.ts', + 'packages/lsp/lsp-local/tests/built-lib.e2e.ts', ], { label: 'built-bin smoke', needs, diff --git a/scripts/verify-package-readme-model-experience.ts b/scripts/verify-package-readme-model-experience.ts index 3ca0317cf4..9a249ca844 100644 --- a/scripts/verify-package-readme-model-experience.ts +++ b/scripts/verify-package-readme-model-experience.ts @@ -47,6 +47,7 @@ const SENTENCE_MODEL_EXPERIENCE: Readonly> = { 'packages/bash/bash-local': { kind: 'indirect', reason: 'The executor backend delegates model rendering to dsh-tool-bash.' }, 'packages/bash/pwsh-local': { kind: 'indirect', reason: 'The executor backend delegates model rendering to dsh-tool-pwsh.' }, 'packages/code-runtime/code-runtime': { kind: 'indirect', reason: 'The service interface delegates model rendering to Code Mode in dsh-tools.' }, + 'packages/code-runtime/code-runtime-subprocess': { kind: 'indirect', reason: 'The subprocess backend delegates model rendering to Code Mode in dsh-tools.' }, 'packages/code-runtime/code-runtime-worker': { kind: 'indirect', reason: 'The worker backend delegates model rendering to Code Mode in dsh-tools.' }, 'packages/typert/registry': { kind: 'none', reason: 'Runtime type registry; consumers (cordis_inspect, wire faces, gates) own any model-visible projection of registry contents.' }, 'packages/typert/loader': { kind: 'none', reason: 'Loader integration only registers generated artifacts; consumers own any model-visible projection.' }, diff --git a/tsconfig.host.json b/tsconfig.host.json index 1a8ef6508a..6252733511 100644 --- a/tsconfig.host.json +++ b/tsconfig.host.json @@ -169,6 +169,7 @@ { "path": "./packages/pty/tool-pty" }, { "path": "./packages/code-runtime/code-runtime" }, { "path": "./packages/code-runtime/code-runtime-worker" }, + { "path": "./packages/code-runtime/code-runtime-subprocess" }, { "path": "./packages/llm/llm-deepseek" }, { "path": "./packages/llm/llm-pi-ai" }, { "path": "./packages/bash/bash-local" }, From 16cac3b5afb7e829a3f02d9445e5991a6c30a5d3 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 00:05:46 +0800 Subject: [PATCH 005/108] fix(runtime): close remote cleanup and readiness races --- ...rtable-execution-world-consumers.i18n.yaml | 4 +- ...7-28-portable-execution-world-consumers.md | 4 +- ...8-portable-execution-world-consumers.zh.md | 4 +- packages/pty/pty-local/README.i18n.yaml | 4 +- packages/pty/pty-local/README.md | 2 +- packages/pty/pty-local/README.zh.md | 2 +- packages/pty/pty-local/src/session.ts | 48 ++++++-- packages/pty/pty-local/tests/session.spec.ts | 112 ++++++++++++++++++ packages/typert/README.i18n.yaml | 6 +- 9 files changed, 166 insertions(+), 20 deletions(-) diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml index 8b4288edb4..0eb1611fe1 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md -2026-07-28-portable-execution-world-consumers.md: 000fa1b98964eb85550d0c607281937f8d7932c6 -2026-07-28-portable-execution-world-consumers.zh.md: 94e6bfcf94e450fafea2b33e152384da400e4b63 +2026-07-28-portable-execution-world-consumers.md: 5a15c2c0a4e552fadbdad9f7d09e00229afb033b +2026-07-28-portable-execution-world-consumers.zh.md: 5ae7f3b96f4df4c3756d294ba12147a3b1b54d34 diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md index 000fa1b989..5a15c2c0a4 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md @@ -22,8 +22,8 @@ Generic consumers use that execution world: - `dsh-bash-local` continues to map Bash semantics onto ordinary `ctx.subprocess.spawn()`. - `dsh-lsp-local` reads and contains source through `ctx.fs`, resolves and launches language servers through `ctx.subprocess`, and sends provider-owned file URIs. Its JSON-RPC, pooling, synchronization, cancellation, and normalization stay unchanged. -- `dsh-pty-local` maps persistent-shell semantics onto `ctx.subprocess.spawnTerminal()`. The local `node-pty` and process-inspection implementation moves into `dsh-subprocess-local`; another subprocess provider supplies the same primitive. -- `dsh-code-runtime-subprocess` materializes a dependency-free runner through `ctx.fs` and launches it through `ctx.subprocess`, preserving the Code Runtime binding and output contract across local or remote worlds. It shares host-side worker mechanics through the non-plugin `dsh-code-runtime-worker/runtime-host` subpath instead of copying them. +- `dsh-pty-local` maps persistent-shell semantics onto `ctx.subprocess.spawnTerminal()`. The local `node-pty` and process-inspection implementation moves into `dsh-subprocess-local`; another subprocess provider supplies the same primitive. A timed-out asynchronous write retains the send reservation until the provider settles it, and completion of a stale inspection resumes polling for the current send. +- `dsh-code-runtime-subprocess` materializes a dependency-free runner through `ctx.fs` and launches it through `ctx.subprocess`, preserving the Code Runtime binding and output contract across local or remote worlds. It shares host-side worker mechanics through the non-plugin `dsh-code-runtime-worker/runtime-host` subpath instead of copying them. The heap-bounded worker rejects oversized binding frames before transfer, each outer hop enforces the same bound before serialization, and the launcher reaps its controller before group termination becomes the fallback. `dsh-code-runtime-worker` remains a separate implementation. It is the smaller in-process backend and works in single-file distributions that cannot assume an installed Node executable. Remote filesystem/process compositions select `dsh-code-runtime-subprocess`; they do not need a provider-specific Code Runtime package. diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md index 94e6bfcf94..5ae7f3b96f 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md @@ -22,8 +22,8 @@ Status: implemented - `dsh-bash-local` 继续把 Bash 语义映射到普通的 `ctx.subprocess.spawn()`。 - `dsh-lsp-local` 通过 `ctx.fs` 读取源文件并验证包含关系,通过 `ctx.subprocess` 解析和启动语言服务器,并发送由提供方负责的文件 URI。其 JSON-RPC、池化、同步、取消和规范化保持不变。 -- `dsh-pty-local` 把持久 shell 语义映射到 `ctx.subprocess.spawnTerminal()`。本地 `node-pty` 与进程检查实现移入 `dsh-subprocess-local`;其他进程管理提供方则提供相同原语。 -- `dsh-code-runtime-subprocess` 通过 `ctx.fs` 物化无依赖 runner,并通过 `ctx.subprocess` 启动它,从而在本地或远程执行世界中保留代码运行时的绑定与输出契约。它通过非插件子路径 `dsh-code-runtime-worker/runtime-host` 共享宿主侧 worker 机制,而不是复制这些机制。 +- `dsh-pty-local` 把持久 shell 语义映射到 `ctx.subprocess.spawnTerminal()`。本地 `node-pty` 与进程检查实现移入 `dsh-subprocess-local`;其他进程管理提供方则提供相同原语。异步写入超时后仍保留发送预留,直至提供方将写入结算;陈旧检查完成后,会针对当前发送恢复轮询。 +- `dsh-code-runtime-subprocess` 通过 `ctx.fs` 物化无依赖 runner,并通过 `ctx.subprocess` 启动它,从而在本地或远程执行世界中保留代码运行时的绑定与输出契约。它通过非插件子路径 `dsh-code-runtime-worker/runtime-host` 共享宿主侧 worker 机制,而不是复制这些机制。受堆上限约束的 worker 会在传输前拒绝过大的绑定帧;每个外层转发环节都会在序列化前执行相同的上限检查;launcher 会先回收 controller,进程组终止才会成为后备手段。 `dsh-code-runtime-worker` 仍是独立实现。它是较小的进程内后端,可用于无法假定已安装 Node 可执行文件的单文件分发。远程文件系统/进程组合选择 `dsh-code-runtime-subprocess`;它们不需要提供方专用的代码运行时包。 diff --git a/packages/pty/pty-local/README.i18n.yaml b/packages/pty/pty-local/README.i18n.yaml index d8f5c2e2d0..828e767a60 100644 --- a/packages/pty/pty-local/README.i18n.yaml +++ b/packages/pty/pty-local/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/pty/pty-local/README.md -README.md: 8a0a60139e27d98c4f506245a204723da997cd4a -README.zh.md: 3df845931dac33abecbcd5030a1b204c68acc638 +README.md: 235599737d21289d291ccab8f06fbcca91c83223 +README.zh.md: ca01c9a10430644b0d6ea6c5cfa70472f0447b92 diff --git a/packages/pty/pty-local/README.md b/packages/pty/pty-local/README.md index 8a0a60139e..235599737d 100644 --- a/packages/pty/pty-local/README.md +++ b/packages/pty/pty-local/README.md @@ -10,7 +10,7 @@ The plugin injects `pty`, `sandbox`, `sandboxPolicy`, and `subprocess`, then reg Readiness combines a foreground-verified private bash prompt marker, provider-reported foreground stdin-wait facts, silence fallback, and absolute timeout. A marker is not ready until printable prompt text arrives, including when the OSC marker and `PS1` are split across data callbacks; when bash prints the marker before the terminal provider publishes its return to the foreground process group, polling retains the candidate for `handoffGraceMs` past the ordinary silence bound so a coincident handoff can win. An interactive child that inherits `PROMPT_COMMAND` therefore cannot suppress inferred-idle readiness until the absolute timeout. Unknown foreground state is never a positive exact-idle signal. A foreground group's stdin wait that existed before a send is likewise not post-write readiness: the same group must be observed outside that wait before a later wait can settle the send, while a changed foreground group is new evidence. During unpublished startup, a fallback requires observed output; zero-output silence cannot publish an empty session, and timeout rejects the spawn. Cancellation closes the unpublished shell and rejects with the caller's exact abort reason; `PtyBackendCleanupError` separately preserves a cleanup failure. Incomplete terminal-control sequences are bounded by `maxReadBytes` and discarded through their terminator after crossing that limit; a trailing carriage return is carried across callbacks so split CRLF becomes one newline. -Send cancellation asks the terminal handle to signal the current foreground process group with a real `SIGINT`; it never emulates interruption by writing `\x03`, so raw-mode programs remain cancellable. Close starts provider-owned TERM-to-KILL whole-session cleanup and awaits quiescence after the terminal outcome. A cleanup failure does not cache a permanently rejected close; a later close retries the provider operation. +Send cancellation asks the terminal handle to signal the current foreground process group with a real `SIGINT`; it never emulates interruption by writing `\x03`, so raw-mode programs remain cancellable. A send that times out during an asynchronous provider write reports the timeout but retains the session's send slot until that write settles, so late bytes cannot interleave with a successor. Close starts provider-owned TERM-to-KILL whole-session cleanup and awaits quiescence after the terminal outcome. A cleanup failure does not cache a permanently rejected close; a later close retries the provider operation. ## Model Experience diff --git a/packages/pty/pty-local/README.zh.md b/packages/pty/pty-local/README.zh.md index 3df845931d..ca01c9a104 100644 --- a/packages/pty/pty-local/README.zh.md +++ b/packages/pty/pty-local/README.zh.md @@ -10,7 +10,7 @@ 就绪检测结合以下机制:由前台状态验证的私有 bash 提示符标记、提供方报告的前台 stdin 等待事实、静默回退和绝对超时。可打印的提示符文本尚未到达时,即使 OSC 标记和 `PS1` 被拆到多个数据回调中,系统也不会把标记视为就绪。如果 bash 在终端提供方发布其重新取得前台进程组的状态前打印标记,轮询会在普通静默上限之后再保留该候选状态 `handoffGraceMs`,使恰好同时发生的前台交接有机会胜出。因此,继承 `PROMPT_COMMAND` 的交互式子进程无法持续压制推断空闲就绪,最多只能延续到绝对超时。未知的前台状态绝不会作为精确空闲的正向信号。同样,一次 send 之前就已存在的前台进程组 stdin 等待并不代表写入后就绪:必须先观察到同一进程组脱离该等待,之后再次进入等待才能使该次 send 完成;前台进程组发生变化则构成新的证据。尚未发布的启动过程中,回退路径要求已经观察到输出;零输出静默不能发布空会话,超时则拒绝 spawn。取消操作会关闭尚未发布的 shell,并以调用方提供的确切中止原因拒绝;`PtyBackendCleanupError` 会单独保留清理失败。未完成的终端控制序列受 `maxReadBytes` 限制;超过上限后,系统会丢弃内容直到其终止符。末尾的回车会跨回调保留,使拆分的 CRLF 合并为一个换行。 -取消发送时,系统会请求终端句柄向当前前台进程组发送真正的 `SIGINT`;它绝不会通过写入 `\x03` 模拟中断,因此原始模式程序仍可取消。关闭操作启动由提供方负责的 TERM→KILL 全会话清理,并在终端结果之后等待完全停稳。清理失败不会缓存成永久拒绝的关闭操作;后续关闭会重试提供方操作。 +取消发送时,系统会请求终端句柄向当前前台进程组发送真正的 `SIGINT`;它绝不会通过写入 `\x03` 模拟中断,因此原始模式程序仍可取消。在提供方异步写入期间超时的发送会报告超时,但会继续占用该会话的发送槽位,直至写入结算,从而避免延迟到达的字节与后续发送交错。关闭操作启动由提供方负责的 TERM→KILL 全会话清理,并在终端结果之后等待完全停稳。清理失败不会缓存成永久拒绝的关闭操作;后续关闭会重试提供方操作。 ## 模型体验 diff --git a/packages/pty/pty-local/src/session.ts b/packages/pty/pty-local/src/session.ts index a6abaf7785..5ac4dbca73 100644 --- a/packages/pty/pty-local/src/session.ts +++ b/packages/pty/pty-local/src/session.ts @@ -94,6 +94,10 @@ class LocalSendOperation implements PtySendOperation { return this.promise.promise } + get settled(): boolean { + return this.finished + } + append(text: string): void { if (!this.finished) this.output.append(text) } @@ -155,6 +159,8 @@ export class LocalPtySession implements PtyBackendSession { private activeTimer: NodeJS.Timeout | undefined private activeDeadlineTimer: NodeJS.Timeout | undefined private activeAbort: (() => void) | undefined + private writing: LocalSendOperation | undefined + private pollingReady: LocalSendOperation | undefined private polling = false private promptSeen = false private promptTextSeen = false @@ -224,7 +230,7 @@ export class LocalPtySession implements PtyBackendSession { this.activeAbort = () => request.signal?.removeEventListener('abort', onAbort) } this.activeDeadlineTimer = setTimeout(() => { - if (this.active === operation) this.settleActive('timeout') + if (this.active === operation) this.settleActive('timeout', this.writing === operation) }, this.config.timeoutMs) void this.beginSend(operation, request) return operation @@ -236,12 +242,29 @@ export class LocalPtySession implements PtyBackendSession { if (this.active !== operation || this.closing) return operation.setInitialForeground(foreground) const input = `${request.text}${request.submit ? '\r' : ''}` - if (input.length > 0) await this.terminal.write(Buffer.from(input, 'utf8')) + if (input.length > 0) { + this.writing = operation + try { + await this.terminal.write(Buffer.from(input, 'utf8')) + } finally { + this.writing = undefined + } + } + if (this.active === operation && operation.settled) { + this.clearActive() + return + } // Closing can race the awaited provider write even though static analysis sees only local assignments. // eslint-disable-next-line @typescript-eslint/no-unnecessary-condition - if (this.active === operation && !this.closing) this.schedulePoll(operation, 0) + if (this.active === operation && !this.closing) { + this.pollingReady = operation + this.schedulePoll(operation, 0) + } } catch (error: unknown) { - if (this.active === operation) this.failActive(error) + if (this.active === operation) { + if (operation.settled) this.clearActive() + else this.failActive(error) + } } } @@ -401,15 +424,24 @@ export class LocalPtySession implements PtyBackendSession { if (this.active === operation) this.failActive(error) } finally { this.polling = false - if (this.active === operation) this.schedulePoll(operation) + const active = this.active + // Awaited provider inspection can clear or replace the active send despite static analysis. + // eslint-disable-next-line @typescript-eslint/no-unnecessary-condition + if (active !== undefined && this.pollingReady === active) this.schedulePoll(active) } } - private settleActive(waitReason: PtyWaitReason): void { + private settleActive(waitReason: PtyWaitReason, retainOwnership = false): void { const operation = this.active if (operation === undefined) return const scrollbackTruncated = this.scrollback.snapshot().truncated - this.clearActive() + if (retainOwnership) { + this.stopPolling() + this.activeAbort?.() + this.activeAbort = undefined + } else { + this.clearActive() + } operation.settle(waitReason, this.statusValue, scrollbackTruncated) } @@ -424,6 +456,8 @@ export class LocalPtySession implements PtyBackendSession { this.stopPolling() this.activeAbort?.() this.activeAbort = undefined + this.writing = undefined + this.pollingReady = undefined this.active = undefined } diff --git a/packages/pty/pty-local/tests/session.spec.ts b/packages/pty/pty-local/tests/session.spec.ts index 7a775eb9a0..7e6b9407a0 100644 --- a/packages/pty/pty-local/tests/session.spec.ts +++ b/packages/pty/pty-local/tests/session.spec.ts @@ -268,6 +268,40 @@ describe('LocalPtySession readiness and output', () => { failedInternal.fail(new Error('ignored')) }) + it('retains send ownership after timeout until an asynchronous provider write settles', async () => { + vi.useFakeTimers() + const terminal = new FakeTerminal() + const inspector = new FakeInspector() + const session = makeSession(terminal, inspector, config()) + await initialize(session, terminal) + + const writeGate = Promise.withResolvers() + terminal.write = async () => { await writeGate.promise } + const operation = session.startSend({ text: 'slow write', submit: true }) + await Promise.resolve() + await Promise.resolve() + await vi.advanceTimersByTimeAsync(100) + + expect((await operation.done).waitReason).toBe('timeout') + expect(() => session.startSend({ text: 'must wait', submit: true })).toThrow('active send') + + writeGate.resolve(undefined) + await Promise.resolve() + await Promise.resolve() + const rejectedWrite = Promise.withResolvers() + terminal.write = async () => { await rejectedWrite.promise } + const rejected = session.startSend({ text: 'late rejection', submit: true }) + await vi.advanceTimersByTimeAsync(100) + expect((await rejected.done).waitReason).toBe('timeout') + rejectedWrite.reject(new Error('write failed after timeout')) + await Promise.resolve() + await Promise.resolve() + + const next = session.startSend({ text: '', submit: false }) + await vi.advanceTimersByTimeAsync(100) + expect((await next.done).waitReason).toBe('inferred_idle') + }) + it('handles startup exit, unknown exit signals, cancel-write failure, and stale polls', async () => { vi.useFakeTimers() const startupTerminal = new FakeTerminal() @@ -509,6 +543,84 @@ describe('LocalPtySession readiness and output', () => { }).settle('timeout', { kind: 'running' }, false) }) + it('reschedules readiness for a new send after a stale remote inspection releases the poll slot', async () => { + vi.useFakeTimers() + const terminal = new FakeTerminal() + const inspector = new FakeInspector() + const session = makeSession(terminal, inspector, config()) + await initialize(session, terminal) + + const old = session.startSend({ text: '', submit: false }) + await Promise.resolve() + await Promise.resolve() + const inspection = Promise.withResolvers<{ processGroupId: number; inputWaiting: boolean }>() + let block = true + terminal.inspectForeground = async () => block + ? await inspection.promise + : { processGroupId: 456, inputWaiting: false } + const internals = session as unknown as { + pollReadiness(operation: PtySendOperation): Promise + settleActive(reason: 'timeout'): void + } + const stalePoll = internals.pollReadiness(old) + internals.settleActive('timeout') + await old.done + + block = false + const current = session.startSend({ text: '', submit: false }) + terminal.emitData('\x1b]133;D;0\x07dsh> ') + await Promise.resolve() + await Promise.resolve() + inspection.resolve({ processGroupId: 456, inputWaiting: false }) + await stalePoll + await vi.advanceTimersByTimeAsync(10) + + expect((await current.done).waitReason).toBe('stdin_read') + }) + + it('does not poll a successor before its own pre-write inspection and write complete', async () => { + vi.useFakeTimers() + const terminal = new FakeTerminal() + const inspector = new FakeInspector() + const session = makeSession(terminal, inspector, config()) + await initialize(session, terminal) + + const old = session.startSend({ text: '', submit: false }) + await Promise.resolve() + await Promise.resolve() + const staleInspection = Promise.withResolvers<{ processGroupId: number; inputWaiting: boolean }>() + const successorInspection = Promise.withResolvers<{ processGroupId: number; inputWaiting: boolean }>() + let inspectCalls = 0 + terminal.inspectForeground = async () => { + inspectCalls += 1 + return inspectCalls === 1 ? await staleInspection.promise : await successorInspection.promise + } + const internals = session as unknown as { + pollReadiness(operation: PtySendOperation): Promise + settleActive(reason: 'timeout'): void + } + const stalePoll = internals.pollReadiness(old) + internals.settleActive('timeout') + await old.done + + const current = session.startSend({ text: 'successor', submit: true }) + await Promise.resolve() + await Promise.resolve() + staleInspection.resolve({ processGroupId: 456, inputWaiting: false }) + await stalePoll + await vi.advanceTimersByTimeAsync(10) + expect(inspectCalls).toBe(2) + expect(terminal.writes).toEqual([]) + + successorInspection.resolve({ processGroupId: 456, inputWaiting: false }) + await Promise.resolve() + await Promise.resolve() + terminal.emitData('\x1b]133;D;0\x07dsh> ') + await vi.advanceTimersByTimeAsync(10) + expect(terminal.writes).toEqual(['successor\r']) + expect((await current.done).waitReason).toBe('stdin_read') + }) + it('contains stale timer, write, inspection, and interrupt continuations', async () => { vi.useFakeTimers() const settle = (operation: PtySendOperation): void => { diff --git a/packages/typert/README.i18n.yaml b/packages/typert/README.i18n.yaml index cf44d9aace..d075078b94 100644 --- a/packages/typert/README.i18n.yaml +++ b/packages/typert/README.i18n.yaml @@ -1,6 +1,6 @@ # Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: -# pnpm run verify-translation-pairing --write packages/typert/README.md -README.md: ad9f843e48be0e3be85921ed8fd3ca4e2c327160 -README.zh.md: 0d4be016b766178b54f7e269583fa4200e3eb24b +# pnpm run verify-translation-pairing --write packages/code-runtime/code-runtime-subprocess/README.md +README.md: 9e0ea9065ea4e42696f6afcd243c385e514f7c2d +README.zh.md: 6946248ec58054fcdd93a4c94affb55f90041301 From ac81165662eeee7fb9b4f77be358e78c8b0c6263 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 01:18:01 +0800 Subject: [PATCH 006/108] fix(runtime): close async cleanup and framing races --- .../feature/2026-06-15-code-mode.i18n.yaml | 4 +- .../feature/2026-06-15-code-mode.md | 4 +- .../feature/2026-06-15-code-mode.zh.md | 4 +- ...26-07-16-persistent-pty-sessions.i18n.yaml | 4 +- .../2026-07-16-persistent-pty-sessions.md | 6 +-- .../2026-07-16-persistent-pty-sessions.zh.md | 6 +-- packages/pty/pty-local/README.i18n.yaml | 4 +- packages/pty/pty-local/README.md | 4 +- packages/pty/pty-local/README.zh.md | 4 +- packages/pty/pty-local/src/index.ts | 3 +- packages/pty/pty-local/src/sanitize.ts | 37 +++++++++------- packages/pty/pty-local/src/session.ts | 22 +++++++--- packages/pty/pty-local/tests/sanitize.spec.ts | 6 +-- packages/pty/pty-local/tests/session.spec.ts | 44 +++++++++++++++++++ packages/typert/README.i18n.yaml | 4 +- 15 files changed, 109 insertions(+), 47 deletions(-) diff --git a/.agents/notes/implemented/feature/2026-06-15-code-mode.i18n.yaml b/.agents/notes/implemented/feature/2026-06-15-code-mode.i18n.yaml index a878942993..42a11b8fb8 100644 --- a/.agents/notes/implemented/feature/2026-06-15-code-mode.i18n.yaml +++ b/.agents/notes/implemented/feature/2026-06-15-code-mode.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-06-15-code-mode.md -2026-06-15-code-mode.md: 7e51b9fa726afe4c34b87457b562b4092ee6c93c -2026-06-15-code-mode.zh.md: 4ba098d87d8c944e88c4cbe11ff78ac4383d3ac5 +2026-06-15-code-mode.md: b0778b2c9a8dfce8f786faccf6cc47f477c6b7c0 +2026-06-15-code-mode.zh.md: 437692eac3fc6740e0f1d4dd9001130fd6dd448a diff --git a/.agents/notes/implemented/feature/2026-06-15-code-mode.md b/.agents/notes/implemented/feature/2026-06-15-code-mode.md index 7e51b9fa72..b0778b2c9a 100644 --- a/.agents/notes/implemented/feature/2026-06-15-code-mode.md +++ b/.agents/notes/implemented/feature/2026-06-15-code-mode.md @@ -79,7 +79,7 @@ Requests contain every runtime input; implementations own validated timeout and 5. **Enforce independent budgets.** `computeMs` meters worker busy time, allowing slow awaited tools without excusing a hot loop. `maxWallMs` bounds total elapsed time, including unresolved waits. `maxOutputBytes` bounds only the combined serialized outer logs, completion, or diagnostic; intermediate binding values have no byte cap. Expiry, cancellation, and completion terminate the worker, and heap exits or outer overflow are explicit failures. 6. **Dispose to quiescence**: the service's own disposal terminates in-flight workers and *awaits* their exits before resolving, per [defensive patterns](../../../../docs/defensive-patterns.md). -`@deepseek-ai/dsh-code-runtime-subprocess` preserves those program, binding, output, and worker-budget semantics across a filesystem/subprocess execution world. It writes a dependency-free runner below `ctx.subprocess.runtimeRoot`, resolves Node through the provider, and carries binding traffic over bounded base64 JSON frames on raw pipes. The [portable execution-world decision](../architecture/2026-07-28-portable-execution-world-consumers.md) owns why this generic backend replaces provider-specific Code Runtime packages; `dsh-code-runtime-worker` remains the single-process and single-file-distribution path. +`@deepseek-ai/dsh-code-runtime-subprocess` preserves those program, binding, output, and worker-budget semantics across a filesystem/subprocess execution world. It writes a dependency-free runner below `ctx.subprocess.runtimeRoot`, resolves Node through the provider, and carries binding traffic over bounded base64 JSON frames on raw pipes. The heap-bounded worker rejects expanded completion wires before MessagePort transfer; terminal settlement asks the launcher to reap its controller and keeps process-group escalation armed until `waitForExit()` confirms whole-tree quiescence. The [portable execution-world decision](../architecture/2026-07-28-portable-execution-world-consumers.md) owns why this generic backend replaces provider-specific Code Runtime packages; `dsh-code-runtime-worker` remains the single-process and single-file-distribution path. ### Trust posture @@ -95,7 +95,7 @@ Deployments switching to `'code'` must update any native-only `toolOrder`. Assem ## Testing -- **Runtime implementations:** Real-worker suites cover typed binding values and failures, every lossless JSON completion root, invalid and over-limit output, exact combined ledger boundaries, compute and wall budgets, hostile binding traffic, empty environment, and disposal to quiescence. Built-package tests run both the direct worker entry and the filesystem/subprocess composition under plain Node; the latter also has a Loader-driven `cordis.yml` test. +- **Runtime implementations:** Real-worker suites cover typed binding values and failures, every lossless JSON completion root, invalid and over-limit output, exact combined ledger and per-hop frame boundaries, compute and wall budgets, hostile binding traffic, empty environment, descendant lifetime cleanup, and disposal to quiescence. Built-package tests run both the direct worker entry and the filesystem/subprocess composition under plain Node; the latter also has a Loader-driven `cordis.yml` test. - **Registry integration:** Tests cover code generation, all presentation modes, reserved-name and restriction rules, scoped visibility, authoritative assembly rewrites, `toolOrder`, runtime compatibility failures, full-pipeline sub-dispatch, parent-token correlation, serialization, cancellation and queue drain, JSON normalization, error propagation, log events, ordered context deferral across successful and failed programs, outer-block suppression, and HMR cleanup. - **With-key e2e:** A real model composes two bash calls in one program; another discovers nested workspace instructions through a Code Mode fs dispatch. The tests verify collapsed request headers, correlated dispatch events, resulting files, deferred context, and model behavior. - **Snapshot:** The `code-mode-turn`, `both-mode-turn`, and `code-mode-workspace-context` fixtures pin SDK text, header tool lists, dispatch events, deferred context, and result cards. diff --git a/.agents/notes/implemented/feature/2026-06-15-code-mode.zh.md b/.agents/notes/implemented/feature/2026-06-15-code-mode.zh.md index 4ba098d87d..437692eac3 100644 --- a/.agents/notes/implemented/feature/2026-06-15-code-mode.zh.md +++ b/.agents/notes/implemented/feature/2026-06-15-code-mode.zh.md @@ -79,7 +79,7 @@ Cloudflare 的 [Code Mode](https://blog.cloudflare.com/code-mode/) 提出了一 5. **强制独立预算。** `computeMs` 计量 worker 忙碌时间,允许慢速的 awaited 工具而不放过热循环。`maxWallMs` 约束总经过时间,包括未解析的等待。`maxOutputBytes` 只约束序列化后的外层日志、完成值或诊断的组合;中间绑定值没有字节数上限。到期、取消和完成都终止 worker,堆退出或外层溢出会作为显式失败报告。 6. **dispose 至完全停稳**:服务自身的 dispose(资源释放)终止进行中的 worker 并*等待*其退出后再 resolve,遵循[防御性模式](../../../../docs/defensive-patterns.md)。 -`@deepseek-ai/dsh-code-runtime-subprocess` 在文件系统/子进程执行环境中保持相同的程序、绑定、输出与 worker 预算语义。它在 `ctx.subprocess.runtimeRoot` 下写入一个无依赖 runner,通过提供方解析 Node,并在原始管道上使用有界 base64 JSON 帧承载绑定通信。[可移植执行环境决策](../architecture/2026-07-28-portable-execution-world-consumers.md)说明为何这个通用后端会取代提供方专用的 Code Runtime 包;`dsh-code-runtime-worker` 仍用于单进程和单文件发行版。 +`@deepseek-ai/dsh-code-runtime-subprocess` 在文件系统/子进程执行环境中保持相同的程序、绑定、输出与 worker 预算语义。它在 `ctx.subprocess.runtimeRoot` 下写入一个无依赖 runner,通过提供方解析 Node,并在原始管道上使用有界 base64 JSON 帧承载绑定通信。受堆上限约束的 worker 会在通过 MessagePort 传输前拒绝展开后的完成值 wire;终态结算会请求 launcher 回收其 controller,并让进程组升级终止机制保持待命,直至 `waitForExit()` 确认整棵进程树完全停稳。[可移植执行环境决策](../architecture/2026-07-28-portable-execution-world-consumers.md)说明为何这个通用后端会取代提供方专用的 Code Runtime 包;`dsh-code-runtime-worker` 仍用于单进程和单文件发行版。 ### 信任姿态 @@ -95,7 +95,7 @@ SDK 指示模型编写一个异步的可擦除 TypeScript 函数体,通过 `aw ## 测试 -- **运行时实现:** 真实 worker 测试套件覆盖类型化的绑定值与失败、每一种无损 JSON 根类型的完成值、无效和超限输出、精确的组合账本边界、compute 和 wall 预算、恶意绑定流量、空环境以及 dispose 至完全停稳。构建后包测试会在纯 Node 下分别运行直接 worker 入口与文件系统/子进程组合;后者另有一个由 Loader 驱动的 `cordis.yml` 测试。 +- **运行时实现:** 真实 worker 测试套件覆盖类型化的绑定值与失败、每一种无损 JSON 根类型的完成值、无效和超限输出、精确的组合账本边界与逐跳帧边界、compute 和 wall 预算、恶意绑定流量、空环境、后代进程生命周期清理以及 dispose 至完全停稳。构建后包测试会在纯 Node 下分别运行直接 worker 入口与文件系统/子进程组合;后者另有一个由 Loader 驱动的 `cordis.yml` 测试。 - **注册表集成:** 测试覆盖代码生成、所有呈现模式、保留名称和限制规则、scoped 可见性、权威组装重写、`toolOrder`、运行时兼容性失败、完整流水线子分发、parent-token 关联、序列化、取消和队列排空、JSON 规范化、错误传播、日志事件、成功与失败程序中的有序上下文延后、外层阻止抑制以及 HMR(热模块替换)清理。 - **带密钥 e2e:** 真实模型在一个程序中组合两次 bash 调用;另一个模型通过 Code Mode fs 分发发现嵌套的工作区指令。测试验证折叠的请求头、关联的分发事件、结果文件、延后上下文和模型行为。 - **快照:** `code-mode-turn`、`both-mode-turn` 和 `code-mode-workspace-context` fixture(测试前置数据)固定 SDK 文本、请求头工具列表、分发事件、延后上下文和结果卡片。 diff --git a/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.i18n.yaml b/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.i18n.yaml index 6a801df865..8b33466ea7 100644 --- a/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.i18n.yaml +++ b/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md -2026-07-16-persistent-pty-sessions.md: 65d265b83ab5ad89c2b919364043eb28e75977c7 -2026-07-16-persistent-pty-sessions.zh.md: 6ea92da9a1b2870f3773d84e7f5629bec220dd1c +2026-07-16-persistent-pty-sessions.md: ef87c2806a237e6e6d44c1e62942146af24b8c19 +2026-07-16-persistent-pty-sessions.zh.md: 5f1bde39de5eca7aa57897d87c95675ce0da5abd diff --git a/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md b/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md index 65d265b83a..ef87c2806a 100644 --- a/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md +++ b/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md @@ -60,7 +60,7 @@ The local subprocess terminal primitive uses only public `node-pty` capabilities The UI render contract is exact and location-free. `terminal_send` uses terminal call/result cards only for foreground sends; its background form is generic `execute`. `terminal_open`, `terminal_read`, `terminal_signal`, `terminal_close`, and `terminal_list` use generic `execute`, `read`, `execute`, `delete`, and `read` cards respectively. No PTY tool emits `locations`. -`terminal_send({ sessionId, text, submit?, run_in_background? })` treats `text` as UTF-8 bytes and resolves `submit` to `true` in the tool implementation. When `submit` is true it writes the platform Enter sequence after the text; when false it writes only the text, allowing control characters and REPL fragments without hidden content heuristics. `enableRunInBackground` defaults to true; false removes `run_in_background` from the schema and rejects the same undeclared argument if a caller forces it through execution. +`terminal_send({ sessionId, text, submit?, run_in_background? })` treats `text` as UTF-8 bytes and resolves `submit` to `true` in the tool implementation. When `submit` is true it writes the platform Enter sequence after the text; when false it writes only the text, allowing control characters and REPL fragments without hidden content heuristics. Cancellation marks queued input before signaling the real foreground group, so input cannot execute if an asynchronous pre-write inspection settles afterward. `enableRunInBackground` defaults to true; false removes `run_in_background` from the schema and rejects the same undeclared argument if a caller forces it through execution. Foreground sends return a bounded rendered delta and two independent facts: `waitReason` (`stdin_read | inferred_idle | timeout | session_exit`) and `sessionStatus` (`running` or `exited` with exit code or signal). `session_exit` refers to the PTY's top-level shell process, not an arbitrary foreground command whose status the shell consumes. A timeout never implies process exit. `dsh-tool-pty.maxResultBytes` defaults to 262144, rejects values below 64 so creation acknowledgements retain registry-issued ids, and caps each single-text UTF-8 result after normalized tool or pipeline errors, wait, session, pagination, truncation, generic task-status wrappers, policy denials or short-circuits, and post-execute replacements or blocks; the terminal definitions' last-mile `finalizeContent` callback leaves deliberately structured multi-block policy content unchanged. The renderer reserves suffix space and preserves code-point boundaries instead of treating the backend payload cap as the final model bound. @@ -72,7 +72,7 @@ With `run_in_background: true`, `dsh-tool-pty` registers the in-flight send on ` ### Local readiness detection -The local backend first recognizes a private OSC prompt marker emitted by its controlled bash startup, then requires printable prompt text after that marker before declaring prompt readiness and runs three bounded fallback tiers. Carrying that state across data callbacks covers macOS delivery where the OSC marker and `PS1` arrive separately; the marker alone can no longer publish an empty MOTD. The marker is removed before output reaches the model and avoids a fixed silence delay for ordinary shell commands on both platforms. Unpublished startup does not accept zero-output silence as readiness; timeout rejects the spawn. If caller cancellation wins during startup, the backend closes the private session and propagates the exact `AbortSignal.reason`; a foreground PGID that is not observable yet cannot replace cancellation with a lookup error. All timings are validated config fields: `pollIntervalMs`, `exactProbeAfterMs`, `idleSilenceMs`, `handoffGraceMs`, and `timeoutMs`. +The local backend first recognizes a private OSC prompt marker emitted by its controlled bash startup, then requires the printable tail after the latest marker to exactly equal the controlled `PS1` before declaring prompt readiness and runs three bounded fallback tiers. Carrying that tail across data callbacks covers delivery where the marker and prompt arrive separately; requiring the exact tail rejects a delayed earlier prompt once echoed input or output follows it, so it cannot settle the current send. The marker is removed before output reaches the model and avoids a fixed silence delay for ordinary shell commands on both platforms. Unpublished startup does not accept zero-output silence as readiness; timeout rejects the spawn. If caller cancellation wins during startup, the backend closes the private session and propagates the exact `AbortSignal.reason`; a foreground PGID that is not observable yet cannot replace cancellation with a lookup error. All timings are validated config fields: `pollIntervalMs`, `exactProbeAfterMs`, `idleSilenceMs`, `handoffGraceMs`, and `timeoutMs`. On Linux, the inspector reads the shell's terminal foreground PGID from `/proc//stat`, enumerates every process and thread in that process group, and probes their current syscalls. A positive Tier 1 result requires an observed stdin wait: direct `read(0)`, a permitted read of a `select`/`pselect6` or `poll`/`ppoll` argument containing fd 0, or an epoll interest list containing fd 0. A wait already present before terminal input is not post-write readiness: the same PGID must be observed outside that wait before re-entering it, while a changed foreground PGID is new evidence. Unreadable process memory and unrecognized syscalls are misses, never positive guesses. Architecture tables contain only syscall numbers defined by the corresponding Linux UAPI; unsupported architectures skip Tier 1. @@ -156,7 +156,7 @@ The package ships concise tool guidance explaining persistent state, owner isola ## Verification -- Per-file coverage pins owner fencing, concurrent reservations, unpublished-spawn cancellation and awaited teardown, sandbox-mode change rejection, retriable lifecycle cleanup, readiness tiers, rejection of pre-write stdin waits, the configured handoff grace holding the idle fallback past one poll and its rejection below `pollIntervalMs`, sanitizer carry state, complete UTF-8 bounds, task integration, schemas, and exact render intents. +- Per-file coverage pins owner fencing, concurrent reservations, cancellation during pre-write inspection, unpublished-spawn cancellation and awaited teardown, sandbox-mode change rejection, retriable lifecycle cleanup, readiness tiers, rejection of pre-write stdin waits and delayed earlier prompts, the configured handoff grace holding the idle fallback past one poll and its rejection below `pollIntervalMs`, sanitizer carry state, complete UTF-8 bounds, task integration, schemas, and exact render intents. - Subprocess process fixtures cover non-leader and non-main-thread stdin waits, zombie quiescence, unreadable process state, supported syscall tables, unsupported architectures, and false-positive rejection; macOS inspector logic is injected into the same unit suite. - Real `node-pty` and PTY-consumer tests jointly exercise shell state, shared sandbox policy, environment scrubbing, raw-mode foreground `SIGINT`, a TERM-ignoring descendant, and immediate post-disposal quiescence on supported hosts. - A Loader-driven `cordis.yml` test mounts the real three-package composition. ACP and headless snapshots pin the six schemas, bounded results, and errors through opt-in overlays; TUI snapshots pin terminal and generic card presentation. diff --git a/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.zh.md b/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.zh.md index 6ea92da9a1..5f1bde39de 100644 --- a/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.zh.md +++ b/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.zh.md @@ -60,7 +60,7 @@ agent scope dispose 时先关闭注册,再等待全部所属 PTY 静默退出 UI 渲染契约精确且不携带位置信息。`terminal_send` 只为前台发送使用 terminal 调用卡片和结果卡片;后台形式使用通用 `execute` 卡片。`terminal_open`、`terminal_read`、`terminal_signal`、`terminal_close` 和 `terminal_list` 分别使用通用 `execute`、`read`、`execute`、`delete` 和 `read` 卡片。所有 PTY 工具都不发出 `locations`。 -`terminal_send({ sessionId, text, submit?, run_in_background? })` 将 `text` 视为 UTF-8 字节,并由工具实现在解析阶段把 `submit` 默认成 `true`。`submit` 为 true 时先写入文本,再写入平台 Enter 序列;为 false 时只写文本,使控制字符和 REPL 片段无需隐藏的内容启发式即可发送。`enableRunInBackground` 默认为 true;设为 false 时,schema 中会移除 `run_in_background`,调用方即使强行把这个未声明参数传入执行流程,也会被拒绝。 +`terminal_send({ sessionId, text, submit?, run_in_background? })` 将 `text` 视为 UTF-8 字节,并由工具实现在解析阶段把 `submit` 默认成 `true`。`submit` 为 true 时先写入文本,再写入平台 Enter 序列;为 false 时只写文本,使控制字符和 REPL 片段无需隐藏的内容启发式即可发送。取消会在向真实前台进程组发送信号前将排队输入标记为已取消,因此即使异步的写入前检查随后才结算,该输入也无法执行。`enableRunInBackground` 默认为 true;设为 false 时,schema 中会移除 `run_in_background`,调用方即使强行把这个未声明参数传入执行流程,也会被拒绝。 前台发送返回有界的渲染增量和两个独立事实:`waitReason`(`stdin_read | inferred_idle | timeout | session_exit`)与 `sessionStatus`(`running`,或携带退出码或信号的 `exited`)。`session_exit` 指 PTY 顶层 shell 进程退出,不指由 shell 消费状态的任意前台命令。timeout 从不意味着进程已经退出。`dsh-tool-pty.maxResultBytes` 默认为 262144;低于 64 的值会被拒绝,以确保创建确认保留 registry 签发的 id;每个单文本 UTF-8 结果在加入规范化的工具或流水线错误、等待、会话、分页、截断、通用 task 状态包装、策略拒绝或短路以及 post-execute 替换或阻断后,仍受该值限制;终端定义自有的末端 `finalizeContent` callback 会原样保留策略刻意返回的结构化多 block 内容。渲染器会为后缀预留空间并保持代码点边界,而不会把后端载荷上限当作面向模型结果的最终上限。 @@ -72,7 +72,7 @@ UI 渲染契约精确且不携带位置信息。`terminal_send` 只为前台发 ### 本地就绪检测 -本地后端先识别受控 bash 启动时发出的私有 OSC prompt marker,并且只有在该 marker 后出现可打印的 prompt 文本时才据此声明 prompt 就绪;除此之外,它还运行 3 个有界 fallback 层级。在 data callback 之间保留这项状态,可以适配 macOS 分开交付 OSC marker 与 `PS1` 的情况;单独的 marker 不会发布空 MOTD。marker 在输出到达模型前被移除,使两个平台上的普通 shell 命令都无需固定等待静默阈值。尚未发布的 startup 不会把零输出静默视为就绪;timeout 会拒绝 spawn。若调用方取消在 startup 期间胜出,后端会关闭私有会话并原样抛出 `AbortSignal.reason`;尚不可观察的前台 PGID 不会再用查找错误覆盖取消原因。所有时间参数都是经校验的配置字段:`pollIntervalMs`、`exactProbeAfterMs`、`idleSilenceMs`、`handoffGraceMs` 和 `timeoutMs`。 +本地后端先识别受控 bash 启动时发出的私有 OSC prompt marker,并且只有在最近一个 marker 后的可打印尾部与受控 `PS1` 完全相等时才声明 prompt 就绪;除此之外,它还运行 3 个有界 fallback 层级。在 data callback 之间保留该尾部,可以适配 marker 与 prompt 被分开交付的情况;如果回显的输入或输出跟在延迟到达的先前 prompt 之后,要求尾部完全相等会拒绝该 prompt,使其无法完成当前 send。marker 在输出到达模型前被移除,使两个平台上的普通 shell 命令都无需固定等待静默阈值。尚未发布的 startup 不会把零输出静默视为就绪;timeout 会拒绝 spawn。若调用方取消在 startup 期间胜出,后端会关闭私有会话并原样抛出 `AbortSignal.reason`;尚不可观察的前台 PGID 不会再用查找错误覆盖取消原因。所有时间参数都是经校验的配置字段:`pollIntervalMs`、`exactProbeAfterMs`、`idleSilenceMs`、`handoffGraceMs` 和 `timeoutMs`。 在 Linux 上,检查器从 `/proc//stat` 读取 shell 的终端前台 PGID,枚举该进程组中的每个进程与线程,并检查它们当前的 syscall。Tier 1 只有观察到 stdin 等待才返回正结果:直接 `read(0)`、获准读取且含 fd 0 的 `select`/`pselect6` 或 `poll`/`ppoll` 参数,或者含 fd 0 的 epoll interest list。终端输入前就已存在的等待并不代表写入后就绪:必须先观察到同一 PGID 脱离该等待,之后再次进入等待才能使该次 send 完成;前台 PGID 发生变化则构成新的证据。无法读取的进程内存和未识别的 syscall 都是 miss,绝不作为正向猜测。架构表只包含对应 Linux UAPI 定义的 syscall number;不支持的架构跳过 Tier 1。 @@ -156,7 +156,7 @@ plugins: ## 验证 -- 每文件覆盖率固定 owner 隔离、并发预留、未发布 spawn 的取消与等待式 teardown、沙箱模式变更拒绝、可重试的生命周期清理、就绪层级、对写入前 stdin 等待的拒绝、配置化交接宽限把 idle fallback 顶过一次轮询以及低于 `pollIntervalMs` 时的拒绝、sanitizer carry state、完整 UTF-8 结果上限、task 集成、schema 和精确 render intent。 +- 每文件覆盖率固定 owner 隔离、并发预留、写入前检查期间的取消、未发布 spawn 的取消与等待式 teardown、沙箱模式变更拒绝、可重试的生命周期清理、就绪层级、对写入前 stdin 等待与延迟到达的先前 prompt 的拒绝、配置化交接宽限把 idle fallback 顶过一次轮询以及低于 `pollIntervalMs` 时的拒绝、sanitizer carry state、完整 UTF-8 结果上限、task 集成、schema 和精确 render intent。 - 子进程 fixture 覆盖非 leader 与非主线程的 stdin 等待、僵尸进程完全停稳、不可读进程状态、受支持的 syscall 表、不支持的架构和误报拒绝;同一单元测试套件通过注入覆盖 macOS 检查器逻辑。 - 真实 `node-pty` 与 PTY 消费方测试共同在受支持宿主上覆盖 shell 状态、共享沙箱策略、环境清洗、raw mode 前台 `SIGINT`、忽略 `SIGTERM` 的后代进程,以及 dispose 返回后立即完全停稳。 - Loader 驱动的 `cordis.yml` 测试挂载真实三包组合。ACP 与 headless 快照通过 opt-in overlay 固定 6 个 schema、有界结果和错误;TUI 快照固定 terminal 与 generic 卡片展示。 diff --git a/packages/pty/pty-local/README.i18n.yaml b/packages/pty/pty-local/README.i18n.yaml index 828e767a60..e29a581f57 100644 --- a/packages/pty/pty-local/README.i18n.yaml +++ b/packages/pty/pty-local/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/pty/pty-local/README.md -README.md: 235599737d21289d291ccab8f06fbcca91c83223 -README.zh.md: ca01c9a10430644b0d6ea6c5cfa70472f0447b92 +README.md: 2c4ee44e8489cc6b53179769301e125dcca0ac8e +README.zh.md: 947bab00b9d6aa5fb7b321fbee84f4b2e420133c diff --git a/packages/pty/pty-local/README.md b/packages/pty/pty-local/README.md index 235599737d..2c4ee44e84 100644 --- a/packages/pty/pty-local/README.md +++ b/packages/pty/pty-local/README.md @@ -8,9 +8,9 @@ Persistent shell backend for `ctx.pty` over `ctx.subprocess.spawnTerminal`. It s The plugin injects `pty`, `sandbox`, `sandboxPolicy`, and `subprocess`, then registers the configured backend type (`shell`). `danger-full-access` starts the shell directly; confined modes wrap the exact shell argv through `ctx.sandbox`. The effective session mode is resolved at spawn. A change to a different effective mode is rejected before its `sandbox/mode` event commits while that owner has an open PTY or a spawn in progress; the fence is attached to the exact owner and therefore outlives a provider reload that retains existing sessions. Wait for creation to settle and close the sessions before changing modes, so a terminal opened with wider access cannot survive a downgrade. -Readiness combines a foreground-verified private bash prompt marker, provider-reported foreground stdin-wait facts, silence fallback, and absolute timeout. A marker is not ready until printable prompt text arrives, including when the OSC marker and `PS1` are split across data callbacks; when bash prints the marker before the terminal provider publishes its return to the foreground process group, polling retains the candidate for `handoffGraceMs` past the ordinary silence bound so a coincident handoff can win. An interactive child that inherits `PROMPT_COMMAND` therefore cannot suppress inferred-idle readiness until the absolute timeout. Unknown foreground state is never a positive exact-idle signal. A foreground group's stdin wait that existed before a send is likewise not post-write readiness: the same group must be observed outside that wait before a later wait can settle the send, while a changed foreground group is new evidence. During unpublished startup, a fallback requires observed output; zero-output silence cannot publish an empty session, and timeout rejects the spawn. Cancellation closes the unpublished shell and rejects with the caller's exact abort reason; `PtyBackendCleanupError` separately preserves a cleanup failure. Incomplete terminal-control sequences are bounded by `maxReadBytes` and discarded through their terminator after crossing that limit; a trailing carriage return is carried across callbacks so split CRLF becomes one newline. +Readiness combines a foreground-verified private bash prompt marker, provider-reported foreground stdin-wait facts, silence fallback, and absolute timeout. A marker is not ready until the printable tail after the latest owned marker exactly equals the controlled `PS1`, including when the OSC marker and prompt are split across data callbacks; echoed input or output following a delayed earlier prompt therefore cannot settle the current send. When bash prints the marker before the terminal provider publishes its return to the foreground process group, polling retains the candidate for `handoffGraceMs` past the ordinary silence bound so a coincident handoff can win. An interactive child that inherits `PROMPT_COMMAND` therefore cannot suppress inferred-idle readiness until the absolute timeout. Unknown foreground state is never a positive exact-idle signal. A foreground group's stdin wait that existed before a send is likewise not post-write readiness: the same group must be observed outside that wait before a later wait can settle the send, while a changed foreground group is new evidence. During unpublished startup, a fallback requires observed output; zero-output silence cannot publish an empty session, and timeout rejects the spawn. Cancellation closes the unpublished shell and rejects with the caller's exact abort reason; `PtyBackendCleanupError` separately preserves a cleanup failure. Incomplete terminal-control sequences are bounded by `maxReadBytes` and discarded through their terminator after crossing that limit; a trailing carriage return is carried across callbacks so split CRLF becomes one newline. -Send cancellation asks the terminal handle to signal the current foreground process group with a real `SIGINT`; it never emulates interruption by writing `\x03`, so raw-mode programs remain cancellable. A send that times out during an asynchronous provider write reports the timeout but retains the session's send slot until that write settles, so late bytes cannot interleave with a successor. Close starts provider-owned TERM-to-KILL whole-session cleanup and awaits quiescence after the terminal outcome. A cleanup failure does not cache a permanently rejected close; a later close retries the provider operation. +Send cancellation marks queued input as canceled before asking the terminal handle to signal the current foreground process group with a real `SIGINT`; if asynchronous pre-write inspection later settles, it cannot execute that input. Cancellation never emulates interruption by writing `\x03`, so raw-mode programs remain cancellable. A send that times out during an asynchronous provider write reports the timeout but retains the session's send slot until that write settles, so late bytes cannot interleave with a successor. Close starts provider-owned TERM-to-KILL whole-session cleanup and awaits quiescence after the terminal outcome. A cleanup failure does not cache a permanently rejected close; a later close retries the provider operation. ## Model Experience diff --git a/packages/pty/pty-local/README.zh.md b/packages/pty/pty-local/README.zh.md index ca01c9a104..947bab00b9 100644 --- a/packages/pty/pty-local/README.zh.md +++ b/packages/pty/pty-local/README.zh.md @@ -8,9 +8,9 @@ 该插件注入 `pty`、`sandbox`、`sandboxPolicy` 和 `subprocess`,然后注册所配置的后端类型(`shell`)。`danger-full-access` 会直接启动 shell;受限模式则通过 `ctx.sandbox` 包装确切的 shell argv。系统在 spawn 时解析会话的实际模式。当某个所有者存在开放的 PTY 或正在进行 spawn 时,如果配置变更会得到不同的实际模式,系统会在对应 `sandbox/mode` 事件提交前拒绝该变更。该限制绑定到确切所有者,因此即使提供方重新加载并保留现有会话,它仍然有效。更改模式前,请等待创建结算并关闭会话,避免以更宽权限打开的终端在权限降级后继续存在。 -就绪检测结合以下机制:由前台状态验证的私有 bash 提示符标记、提供方报告的前台 stdin 等待事实、静默回退和绝对超时。可打印的提示符文本尚未到达时,即使 OSC 标记和 `PS1` 被拆到多个数据回调中,系统也不会把标记视为就绪。如果 bash 在终端提供方发布其重新取得前台进程组的状态前打印标记,轮询会在普通静默上限之后再保留该候选状态 `handoffGraceMs`,使恰好同时发生的前台交接有机会胜出。因此,继承 `PROMPT_COMMAND` 的交互式子进程无法持续压制推断空闲就绪,最多只能延续到绝对超时。未知的前台状态绝不会作为精确空闲的正向信号。同样,一次 send 之前就已存在的前台进程组 stdin 等待并不代表写入后就绪:必须先观察到同一进程组脱离该等待,之后再次进入等待才能使该次 send 完成;前台进程组发生变化则构成新的证据。尚未发布的启动过程中,回退路径要求已经观察到输出;零输出静默不能发布空会话,超时则拒绝 spawn。取消操作会关闭尚未发布的 shell,并以调用方提供的确切中止原因拒绝;`PtyBackendCleanupError` 会单独保留清理失败。未完成的终端控制序列受 `maxReadBytes` 限制;超过上限后,系统会丢弃内容直到其终止符。末尾的回车会跨回调保留,使拆分的 CRLF 合并为一个换行。 +就绪检测结合以下机制:由前台状态验证的私有 bash 提示符标记、提供方报告的前台 stdin 等待事实、静默回退和绝对超时。只有最近一个自有标记之后的可打印尾部与受控 `PS1` 完全相等时,系统才会把标记视为就绪;即使 OSC 标记和提示符被拆到多个数据回调中也是如此。因此,如果回显的输入或输出跟在延迟到达的先前提示符之后,该提示符无法使当前 send 完成。如果 bash 在终端提供方发布其重新取得前台进程组的状态前打印标记,轮询会在普通静默上限之后再保留该候选状态 `handoffGraceMs`,使恰好同时发生的前台交接有机会胜出。因此,继承 `PROMPT_COMMAND` 的交互式子进程无法持续压制推断空闲就绪,最多只能延续到绝对超时。未知的前台状态绝不会作为精确空闲的正向信号。同样,一次 send 之前就已存在的前台进程组 stdin 等待并不代表写入后就绪:必须先观察到同一进程组脱离该等待,之后再次进入等待才能使该次 send 完成;前台进程组发生变化则构成新的证据。尚未发布的启动过程中,回退路径要求已经观察到输出;零输出静默不能发布空会话,超时则拒绝 spawn。取消操作会关闭尚未发布的 shell,并以调用方提供的确切中止原因拒绝;`PtyBackendCleanupError` 会单独保留清理失败。未完成的终端控制序列受 `maxReadBytes` 限制;超过上限后,系统会丢弃内容直到其终止符。末尾的回车会跨回调保留,使拆分的 CRLF 合并为一个换行。 -取消发送时,系统会请求终端句柄向当前前台进程组发送真正的 `SIGINT`;它绝不会通过写入 `\x03` 模拟中断,因此原始模式程序仍可取消。在提供方异步写入期间超时的发送会报告超时,但会继续占用该会话的发送槽位,直至写入结算,从而避免延迟到达的字节与后续发送交错。关闭操作启动由提供方负责的 TERM→KILL 全会话清理,并在终端结果之后等待完全停稳。清理失败不会缓存成永久拒绝的关闭操作;后续关闭会重试提供方操作。 +取消发送会先把排队输入标记为已取消,再请求终端句柄向当前前台进程组发送真正的 `SIGINT`;如果异步的写入前检查随后才结算,也无法执行该输入。取消绝不会通过写入 `\x03` 模拟中断,因此原始模式程序仍可取消。在提供方异步写入期间超时的发送会报告超时,但会继续占用该会话的发送槽位,直至写入结算,从而避免延迟到达的字节与后续发送交错。关闭操作启动由提供方负责的 TERM→KILL 全会话清理,并在终端结果之后等待完全停稳。清理失败不会缓存成永久拒绝的关闭操作;后续关闭会重试提供方操作。 ## 模型体验 diff --git a/packages/pty/pty-local/src/index.ts b/packages/pty/pty-local/src/index.ts index 0a54a0b804..a8d99cb243 100644 --- a/packages/pty/pty-local/src/index.ts +++ b/packages/pty/pty-local/src/index.ts @@ -14,6 +14,7 @@ import type { SandboxMode } from '@deepseek-ai/dsh-sandbox' import { effectiveSandboxMode } from '@deepseek-ai/dsh-sandbox-policy' import { type Config, type ResolvedConfig, validateConfig } from './config.ts' import { LocalPtySession } from './session.ts' +import { CONTROLLED_PROMPT } from './sanitize.ts' export { Config } from './config.ts' export type { Config as PtyLocalConfig } from './config.ts' @@ -58,7 +59,7 @@ function childEnvironment(spec: PtyBackendSpawnSpec): Record { TERM: 'dumb', PAGER: 'cat', GIT_PAGER: 'cat', - PS1: 'dsh> ', + PS1: CONTROLLED_PROMPT, PROMPT_COMMAND: 'printf "\\033]133;D;%s\\007" "$?"', BASH_SILENCE_DEPRECATION_WARNING: '1', DSH_SHELL: '1', diff --git a/packages/pty/pty-local/src/sanitize.ts b/packages/pty/pty-local/src/sanitize.ts index cc22c29c01..1f28315fb1 100644 --- a/packages/pty/pty-local/src/sanitize.ts +++ b/packages/pty/pty-local/src/sanitize.ts @@ -5,12 +5,15 @@ import { Buffer } from 'node:buffer' /** OSC marker emitted by the controlled bash before each prompt. */ export const PROMPT_MARKER_PREFIX = '133;D;' +/** Exact printable prompt emitted after the private marker. */ +export const CONTROLLED_PROMPT = 'dsh> ' + /** One sanitized chunk plus whether it contained the owned prompt marker. */ export interface SanitizedChunk { text: string prompt: boolean - /** Present when printable text followed the latest owned prompt marker. */ - promptText?: true + /** Printable text after the latest owned marker in this chunk. */ + promptTail?: string } /** @@ -23,7 +26,7 @@ export class TerminalSanitizer { private discardMode: 'osc' | 'csi' | undefined private discardOscEscape = false private trailingCarriageReturn = false - private awaitingPromptText = false + private trackingPromptTail = false constructor(private readonly maxPendingBytes: number) {} @@ -36,24 +39,21 @@ export class TerminalSanitizer { this.pending += this.discardPrefix(chunk) let text = '' let prompt = false - let promptText = false + let includePromptTail = this.trackingPromptTail + let promptTail = '' let index = 0 - const appendText = (value: string): boolean => { + const appendText = (value: string): void => { text += value - if (this.awaitingPromptText && value.replace(/[\r\n\x07]/g, '').length > 0) { - this.awaitingPromptText = false - return true - } - return false + if (this.trackingPromptTail) promptTail += value } while (index < this.pending.length) { const escape = this.pending.indexOf('\x1b', index) if (escape < 0) { - promptText = appendText(this.pending.slice(index)) || promptText + appendText(this.pending.slice(index)) index = this.pending.length break } - promptText = appendText(this.pending.slice(index, escape)) || promptText + appendText(this.pending.slice(index, escape)) if (escape + 1 >= this.pending.length) { index = escape break @@ -74,8 +74,9 @@ export class TerminalSanitizer { const content = this.pending.slice(escape + 2, end - terminatorBytes) if (content.startsWith(PROMPT_MARKER_PREFIX)) { prompt = true - promptText = false - this.awaitingPromptText = true + this.trackingPromptTail = true + includePromptTail = true + promptTail = '' } index = end continue @@ -99,7 +100,11 @@ export class TerminalSanitizer { } this.pending = this.pending.slice(index) this.enforcePendingBound() - return { text: this.normalizeText(text), prompt, ...promptText ? { promptText: true } : {} } + return { + text: this.normalizeText(text), + prompt, + ...includePromptTail ? { promptTail } : {}, + } } /** @@ -111,7 +116,7 @@ export class TerminalSanitizer { this.pending = '' this.discardMode = undefined this.discardOscEscape = false - this.awaitingPromptText = false + this.trackingPromptTail = false const normalized = this.normalizeText(text) if (!this.trailingCarriageReturn) return normalized this.trailingCarriageReturn = false diff --git a/packages/pty/pty-local/src/session.ts b/packages/pty/pty-local/src/session.ts index 5ac4dbca73..f984648f27 100644 --- a/packages/pty/pty-local/src/session.ts +++ b/packages/pty/pty-local/src/session.ts @@ -20,7 +20,7 @@ import type { PtyWaitReason, } from '@deepseek-ai/dsh-pty' import type { ResolvedConfig } from './config.ts' -import { TerminalSanitizer } from './sanitize.ts' +import { CONTROLLED_PROMPT, TerminalSanitizer } from './sanitize.ts' function utf8Tail(text: string, maxBytes: number): { text: string; truncated: boolean } { if (Buffer.byteLength(text) <= maxBytes) return { text, truncated: false } @@ -77,6 +77,7 @@ class LocalSendOperation implements PtySendOperation { private readonly output: BoundedTextBuffer private readonly promise: PromiseWithResolvers private finished = false + private cancellationRequested = false private initialForegroundLeftWait: boolean private initialForegroundPgid: number | undefined @@ -98,6 +99,10 @@ class LocalSendOperation implements PtySendOperation { return this.finished } + get cancelRequested(): boolean { + return this.cancellationRequested + } + append(text: string): void { if (!this.finished) this.output.append(text) } @@ -140,6 +145,7 @@ class LocalSendOperation implements PtySendOperation { cancel(): boolean { if (this.finished) return false + this.cancellationRequested = true this.onCancel() return true } @@ -164,6 +170,7 @@ export class LocalPtySession implements PtyBackendSession { private polling = false private promptSeen = false private promptTextSeen = false + private promptTail = '' private shellPgid: number | undefined private initializing = false private lastOutputAt = Date.now() @@ -223,6 +230,7 @@ export class LocalPtySession implements PtyBackendSession { this.lastOutputAt = Date.now() this.promptSeen = false this.promptTextSeen = false + this.promptTail = '' if (request.signal !== undefined) { const onAbort = (): void => { operation.cancel() } @@ -242,7 +250,7 @@ export class LocalPtySession implements PtyBackendSession { if (this.active !== operation || this.closing) return operation.setInitialForeground(foreground) const input = `${request.text}${request.submit ? '\r' : ''}` - if (input.length > 0) { + if (input.length > 0 && !operation.cancelRequested) { this.writing = operation try { await this.terminal.write(Buffer.from(input, 'utf8')) @@ -347,10 +355,14 @@ export class LocalPtySession implements PtyBackendSession { // to the foreground process group. Retain the marker; polling below is // the authority that accepts it only after bash owns the foreground. this.promptSeen = true - this.promptTextSeen = sanitized.promptText === true + this.promptTail = '' this.lastOutputAt = Date.now() - } else if (this.promptSeen && sanitized.promptText === true) { - this.promptTextSeen = true + } + if (this.promptSeen && sanitized.promptTail !== undefined) { + const remaining = Math.max(0, CONTROLLED_PROMPT.length + 1 - this.promptTail.length) + this.promptTail += sanitized.promptTail.slice(0, remaining) + if (sanitized.promptTail.length > remaining) this.promptTail = `${CONTROLLED_PROMPT}\0` + this.promptTextSeen = this.promptTail === CONTROLLED_PROMPT } } diff --git a/packages/pty/pty-local/tests/sanitize.spec.ts b/packages/pty/pty-local/tests/sanitize.spec.ts index 4da4ab0d73..f6649b3c6f 100644 --- a/packages/pty/pty-local/tests/sanitize.spec.ts +++ b/packages/pty/pty-local/tests/sanitize.spec.ts @@ -7,7 +7,7 @@ describe('TerminalSanitizer', () => { expect(sanitizer.push('red\x1b[3')).toEqual({ text: 'red', prompt: false }) expect(sanitizer.push('1m text\x1b[0m\r\n')).toEqual({ text: ' text\n', prompt: false }) expect(sanitizer.push('\x1b]133;')).toEqual({ text: '', prompt: false }) - expect(sanitizer.push('D;0\x07dsh> ')).toEqual({ text: 'dsh> ', prompt: true, promptText: true }) + expect(sanitizer.push('D;0\x07dsh> ')).toEqual({ text: 'dsh> ', prompt: true, promptTail: 'dsh> ' }) }) it('drops unrelated OSC, short escapes, BEL, and incomplete trailing escape', () => { @@ -35,8 +35,8 @@ describe('TerminalSanitizer', () => { it('reports printable prompt text that follows a marker in a later chunk', () => { const sanitizer = new TerminalSanitizer(64) - expect(sanitizer.push('\x1b]133;D;0\x07')).toEqual({ text: '', prompt: true }) - expect(sanitizer.push('dsh> ')).toEqual({ text: 'dsh> ', prompt: false, promptText: true }) + expect(sanitizer.push('\x1b]133;D;0\x07')).toEqual({ text: '', prompt: true, promptTail: '' }) + expect(sanitizer.push('dsh> ')).toEqual({ text: 'dsh> ', prompt: false, promptTail: 'dsh> ' }) }) it('bounds and discards unterminated control sequences through their terminators', () => { diff --git a/packages/pty/pty-local/tests/session.spec.ts b/packages/pty/pty-local/tests/session.spec.ts index 7e6b9407a0..6346a6bba2 100644 --- a/packages/pty/pty-local/tests/session.spec.ts +++ b/packages/pty/pty-local/tests/session.spec.ts @@ -268,6 +268,29 @@ describe('LocalPtySession readiness and output', () => { failedInternal.fail(new Error('ignored')) }) + it('does not write a send canceled during asynchronous foreground inspection', async () => { + vi.useFakeTimers() + const terminal = new FakeTerminal() + const inspector = new FakeInspector() + const session = makeSession(terminal, inspector, config()) + await initialize(session, terminal) + + const inspection = Promise.withResolvers<{ processGroupId: number; inputWaiting: boolean }>() + terminal.inspectForeground = async () => await inspection.promise + const controller = new AbortController() + const operation = session.startSend({ text: 'must not execute', submit: true, signal: controller.signal }) + controller.abort() + inspection.resolve({ processGroupId: 456, inputWaiting: false }) + await Promise.resolve() + await Promise.resolve() + + expect(terminal.writes).toEqual([]) + expect(inspector.groups).toContainEqual([456, 'SIGINT']) + terminal.emitData('\x1b]133;D;130\x07dsh> ') + await vi.advanceTimersByTimeAsync(10) + await operation.done + }) + it('retains send ownership after timeout until an asynchronous provider write settles', async () => { vi.useFakeTimers() const terminal = new FakeTerminal() @@ -418,6 +441,27 @@ describe('LocalPtySession readiness and output', () => { expect(session.motd).toBe('dsh> ') }) + it('does not attribute a delayed prior prompt to the current send', async () => { + vi.useFakeTimers() + const terminal = new FakeTerminal() + const session = new LocalPtySession(terminal, config({ idleSilenceMs: 100, timeoutMs: 200 })) + await initialize(session, terminal) + + const operation = session.startSend({ text: "printf 'PID=%s\\n' \"$!\"", submit: true }) + let settled = false + void operation.done.then(() => { settled = true }) + await Promise.resolve() + await Promise.resolve() + + terminal.emitData('\x1b]133;D;0\x07dsh> printf \'PID=%s\\n\' "$!"\r\n') + await vi.advanceTimersByTimeAsync(20) + expect(settled).toBe(false) + + terminal.emitData('PID=123\r\n\x1b]133;D;0\x07dsh> ') + await vi.advanceTimersByTimeAsync(10) + expect(await operation.done).toMatchObject({ waitReason: 'stdin_read' }) + }) + it('retains a prompt marker until the startup shell regains the foreground group', async () => { vi.useFakeTimers() const terminal = new FakeTerminal() diff --git a/packages/typert/README.i18n.yaml b/packages/typert/README.i18n.yaml index d075078b94..d1678caf7e 100644 --- a/packages/typert/README.i18n.yaml +++ b/packages/typert/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/code-runtime/code-runtime-subprocess/README.md -README.md: 9e0ea9065ea4e42696f6afcd243c385e514f7c2d -README.zh.md: 6946248ec58054fcdd93a4c94affb55f90041301 +README.md: 620728744fb2a63a730ee06ef3eb5c36505ae572 +README.zh.md: d3201d2e49e9dc89a9d274ca1075f450ad13c683 From e5f3a6736357c33d3d889ebda316646a409b3eb8 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 01:15:35 +0800 Subject: [PATCH 007/108] fix(runtime): publish completion before pipe cleanup --- .../2026-07-28-portable-execution-world-consumers.i18n.yaml | 4 ++-- .../2026-07-28-portable-execution-world-consumers.md | 2 +- .../2026-07-28-portable-execution-world-consumers.zh.md | 2 +- packages/typert/README.i18n.yaml | 4 ++-- 4 files changed, 6 insertions(+), 6 deletions(-) diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml index 0eb1611fe1..f72f7c1d89 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md -2026-07-28-portable-execution-world-consumers.md: 5a15c2c0a4e552fadbdad9f7d09e00229afb033b -2026-07-28-portable-execution-world-consumers.zh.md: 5ae7f3b96f4df4c3756d294ba12147a3b1b54d34 +2026-07-28-portable-execution-world-consumers.md: 145b23317552a51f608404a3308011fb8540319f +2026-07-28-portable-execution-world-consumers.zh.md: 50e27ccce19516a02e4e5024efb8303231a2236e diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md index 5a15c2c0a4..145b233175 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md @@ -23,7 +23,7 @@ Generic consumers use that execution world: - `dsh-bash-local` continues to map Bash semantics onto ordinary `ctx.subprocess.spawn()`. - `dsh-lsp-local` reads and contains source through `ctx.fs`, resolves and launches language servers through `ctx.subprocess`, and sends provider-owned file URIs. Its JSON-RPC, pooling, synchronization, cancellation, and normalization stay unchanged. - `dsh-pty-local` maps persistent-shell semantics onto `ctx.subprocess.spawnTerminal()`. The local `node-pty` and process-inspection implementation moves into `dsh-subprocess-local`; another subprocess provider supplies the same primitive. A timed-out asynchronous write retains the send reservation until the provider settles it, and completion of a stale inspection resumes polling for the current send. -- `dsh-code-runtime-subprocess` materializes a dependency-free runner through `ctx.fs` and launches it through `ctx.subprocess`, preserving the Code Runtime binding and output contract across local or remote worlds. It shares host-side worker mechanics through the non-plugin `dsh-code-runtime-worker/runtime-host` subpath instead of copying them. The heap-bounded worker rejects oversized binding frames before transfer, each outer hop enforces the same bound before serialization, and the launcher reaps its controller before group termination becomes the fallback. +- `dsh-code-runtime-subprocess` materializes a dependency-free runner through `ctx.fs` and launches it through `ctx.subprocess`, preserving the Code Runtime binding and output contract across local or remote worlds. It shares host-side worker mechanics through the non-plugin `dsh-code-runtime-worker/runtime-host` subpath instead of copying them. The heap-bounded worker rejects oversized binding frames before transfer, each outer hop enforces the same bound before serialization, and the launcher publishes an accepted terminal frame before reaping its controller so a descendant that inherits controller pipes cannot suppress completion; the host still awaits process-group quiescence. `dsh-code-runtime-worker` remains a separate implementation. It is the smaller in-process backend and works in single-file distributions that cannot assume an installed Node executable. Remote filesystem/process compositions select `dsh-code-runtime-subprocess`; they do not need a provider-specific Code Runtime package. diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md index 5ae7f3b96f..50e27ccce1 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md @@ -23,7 +23,7 @@ Status: implemented - `dsh-bash-local` 继续把 Bash 语义映射到普通的 `ctx.subprocess.spawn()`。 - `dsh-lsp-local` 通过 `ctx.fs` 读取源文件并验证包含关系,通过 `ctx.subprocess` 解析和启动语言服务器,并发送由提供方负责的文件 URI。其 JSON-RPC、池化、同步、取消和规范化保持不变。 - `dsh-pty-local` 把持久 shell 语义映射到 `ctx.subprocess.spawnTerminal()`。本地 `node-pty` 与进程检查实现移入 `dsh-subprocess-local`;其他进程管理提供方则提供相同原语。异步写入超时后仍保留发送预留,直至提供方将写入结算;陈旧检查完成后,会针对当前发送恢复轮询。 -- `dsh-code-runtime-subprocess` 通过 `ctx.fs` 物化无依赖 runner,并通过 `ctx.subprocess` 启动它,从而在本地或远程执行世界中保留代码运行时的绑定与输出契约。它通过非插件子路径 `dsh-code-runtime-worker/runtime-host` 共享宿主侧 worker 机制,而不是复制这些机制。受堆上限约束的 worker 会在传输前拒绝过大的绑定帧;每个外层转发环节都会在序列化前执行相同的上限检查;launcher 会先回收 controller,进程组终止才会成为后备手段。 +- `dsh-code-runtime-subprocess` 通过 `ctx.fs` 物化无依赖 runner,并通过 `ctx.subprocess` 启动它,从而在本地或远程执行世界中保留代码运行时的绑定与输出契约。它通过非插件子路径 `dsh-code-runtime-worker/runtime-host` 共享宿主侧 worker 机制,而不是复制这些机制。受堆上限约束的 worker 会在传输前拒绝过大的绑定帧;每个外层转发环节都会在序列化前执行相同的上限检查;launcher 会在回收 controller 前发布已接纳的终态帧,使继承 controller 管道的后代进程无法阻止完成;宿主仍会等待进程组完全停稳。 `dsh-code-runtime-worker` 仍是独立实现。它是较小的进程内后端,可用于无法假定已安装 Node 可执行文件的单文件分发。远程文件系统/进程组合选择 `dsh-code-runtime-subprocess`;它们不需要提供方专用的代码运行时包。 diff --git a/packages/typert/README.i18n.yaml b/packages/typert/README.i18n.yaml index d1678caf7e..384c9cdb94 100644 --- a/packages/typert/README.i18n.yaml +++ b/packages/typert/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/code-runtime/code-runtime-subprocess/README.md -README.md: 620728744fb2a63a730ee06ef3eb5c36505ae572 -README.zh.md: d3201d2e49e9dc89a9d274ca1075f450ad13c683 +README.md: db4478fe799dce1bbdb7e3ae3d31947515aae151 +README.zh.md: 238f5f66db58edb99945cfa43299635b4f8f36a8 From c17727d43dcb118158bc28bfec6cebe60097fae3 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 01:15:50 +0800 Subject: [PATCH 008/108] test(subprocess): cover terminal release cleanup --- .../subprocess-local/tests/local.spec.ts | 47 +++++++++++++++++++ 1 file changed, 47 insertions(+) diff --git a/packages/subprocess/subprocess-local/tests/local.spec.ts b/packages/subprocess/subprocess-local/tests/local.spec.ts index bc77d24cfd..7ff10a7348 100644 --- a/packages/subprocess/subprocess-local/tests/local.spec.ts +++ b/packages/subprocess/subprocess-local/tests/local.spec.ts @@ -105,6 +105,53 @@ describe('LocalSubprocessService', () => { expect(waitForExit).toHaveBeenCalledOnce() }) + it('releases a terminal after top-level exit reaches quiescence', async () => { + let exitListener: ((event: { exitCode: number; signal?: number }) => void) | undefined + const inspector = { + foregroundPgid: () => undefined, + isStdinWaiting: () => false, + processTree: () => [], + isAlive: () => false, + signalGroup: () => {}, + signalProcess: () => {}, + } + const terminal = { + pid: 123, + onData: () => ({ dispose: () => {} }), + onExit: (listener: (event: { exitCode: number; signal?: number }) => void) => { + exitListener = listener + return { dispose: () => {} } + }, + write: () => {}, + kill: () => {}, + } + vi.resetModules() + vi.doMock('node-pty', () => ({ spawn: () => terminal })) + vi.doMock('../src/process-inspector.ts', async importOriginal => ({ + ...await importOriginal(), + createProcessInspector: () => inspector, + })) + try { + const { default: IsolatedLocalSubprocessService } = await import('../src/index.ts') + const ctx = new Context() + const fiber = await ctx.plugin(IsolatedLocalSubprocessService) + const service = ctx.subprocess as InstanceType + const handle = await ctx.subprocess.spawnTerminal({ + argv: ['shell'], cwd: process.cwd(), rows: 24, cols: 80, graceMs: 1, + }) + expect((service as unknown as { terminals: Set }).terminals.size).toBe(1) + exitListener?.({ exitCode: 0 }) + await handle.done + await new Promise(resolve => setImmediate(resolve)) + expect((service as unknown as { terminals: Set }).terminals.size).toBe(0) + await fiber.dispose() + } finally { + vi.doUnmock('node-pty') + vi.doUnmock('../src/process-inspector.ts') + vi.resetModules() + } + }) + it('contains a terminal release failure after top-level exit', async () => { let exitListener: ((event: { exitCode: number; signal?: number }) => void) | undefined const terminal = { From 32bce339da84917b696f1387d8e7e8417c73c27d Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 02:40:02 +0800 Subject: [PATCH 009/108] fix(runtime): close lifecycle teardown races --- ...rtable-execution-world-consumers.i18n.yaml | 4 +- ...7-28-portable-execution-world-consumers.md | 6 +-- ...8-portable-execution-world-consumers.zh.md | 6 +-- packages/pty/pty-local/README.i18n.yaml | 4 +- packages/pty/pty-local/README.md | 4 +- packages/pty/pty-local/README.zh.md | 4 +- packages/pty/pty-local/src/index.ts | 33 ++++++++++---- packages/pty/pty-local/src/session.ts | 12 +++-- packages/pty/pty-local/tests/index.spec.ts | 45 +++++++++++++++++++ packages/pty/pty-local/tests/local.spec.ts | 43 +++++++++++++++++- packages/pty/pty-local/tests/session.spec.ts | 27 +++++++++++ .../subprocess-local/README.i18n.yaml | 4 +- .../subprocess/subprocess-local/README.md | 4 +- .../subprocess/subprocess-local/README.zh.md | 4 +- .../subprocess-local/src/process-inspector.ts | 14 ++++++ .../subprocess-local/src/terminal.ts | 9 +++- .../subprocess-local/tests/local.spec.ts | 2 + .../tests/process-inspector.spec.ts | 8 ++++ .../subprocess-local/tests/terminal.spec.ts | 32 +++++++++++++ packages/typert/README.i18n.yaml | 4 +- 20 files changed, 233 insertions(+), 36 deletions(-) diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml index f72f7c1d89..f97b09b940 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md -2026-07-28-portable-execution-world-consumers.md: 145b23317552a51f608404a3308011fb8540319f -2026-07-28-portable-execution-world-consumers.zh.md: 50e27ccce19516a02e4e5024efb8303231a2236e +2026-07-28-portable-execution-world-consumers.md: 3d1928be882fb3fb621e733649146445c69e8a5b +2026-07-28-portable-execution-world-consumers.zh.md: f385e502d5d737fe0d5ae6e28ac7a3a097d85a6b diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md index 145b233175..3d1928be88 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md @@ -22,8 +22,8 @@ Generic consumers use that execution world: - `dsh-bash-local` continues to map Bash semantics onto ordinary `ctx.subprocess.spawn()`. - `dsh-lsp-local` reads and contains source through `ctx.fs`, resolves and launches language servers through `ctx.subprocess`, and sends provider-owned file URIs. Its JSON-RPC, pooling, synchronization, cancellation, and normalization stay unchanged. -- `dsh-pty-local` maps persistent-shell semantics onto `ctx.subprocess.spawnTerminal()`. The local `node-pty` and process-inspection implementation moves into `dsh-subprocess-local`; another subprocess provider supplies the same primitive. A timed-out asynchronous write retains the send reservation until the provider settles it, and completion of a stale inspection resumes polling for the current send. -- `dsh-code-runtime-subprocess` materializes a dependency-free runner through `ctx.fs` and launches it through `ctx.subprocess`, preserving the Code Runtime binding and output contract across local or remote worlds. It shares host-side worker mechanics through the non-plugin `dsh-code-runtime-worker/runtime-host` subpath instead of copying them. The heap-bounded worker rejects oversized binding frames before transfer, each outer hop enforces the same bound before serialization, and the launcher publishes an accepted terminal frame before reaping its controller so a descendant that inherits controller pipes cannot suppress completion; the host still awaits process-group quiescence. +- `dsh-pty-local` maps persistent-shell semantics onto `ctx.subprocess.spawnTerminal()`. The local `node-pty` and process-inspection implementation moves into `dsh-subprocess-local`; another subprocess provider supplies the same primitive. The allocation signal is detached before publication, while readiness initialization retains setup cancellation. A timed-out asynchronous write, or a cancellation whose foreground signal fails during that write, retains the send reservation until the provider settles it; completion of a stale inspection resumes polling for the current send. +- `dsh-code-runtime-subprocess` materializes a dependency-free runner through `ctx.fs` and launches it through `ctx.subprocess`, preserving the Code Runtime binding and output contract across local or remote worlds. It shares host-side worker mechanics through the non-plugin `dsh-code-runtime-worker/runtime-host` subpath instead of copying them. Preparation carries one lifecycle signal through filesystem resolution, materialization, and executable lookup so disposal can abort a stalled provider. The heap-bounded worker rejects oversized binding frames before transfer, each outer hop enforces the same bound before serialization, and the launcher publishes an accepted terminal frame before reaping its controller so a descendant that inherits controller pipes cannot suppress completion; the host still awaits process-group quiescence. `dsh-code-runtime-worker` remains a separate implementation. It is the smaller in-process backend and works in single-file distributions that cannot assume an installed Node executable. Remote filesystem/process compositions select `dsh-code-runtime-subprocess`; they do not need a provider-specific Code Runtime package. @@ -45,4 +45,4 @@ A remote execution provider implements only its shared sandbox owner plus filesy The fundamental interfaces are wider, and a filesystem/subprocess pair must agree on one execution world. The added operations are limited to facts and lifecycle mechanics that current generic consumers require; model schemas, protocol framing, readiness policy, and presentation do not leak into the providers. -The local implementation absorbs `node-pty` and platform process inspection because it owns local terminal mechanics. This moves code without weakening terminal teardown: disposal still waits for exact PID-identity-fenced descendants and the top-level terminal process to reach quiescence. +The local implementation absorbs `node-pty` and platform process inspection because it owns local terminal mechanics. This moves code without weakening terminal teardown: disposal waits for exact PID-identity-fenced descendants retained during foreground inspection, Linux session members that survive top-level exit, and the top-level terminal process to reach quiescence. macOS cannot enumerate a POSIX session after its leader exits, so a child that reparents between inspection snapshots remains an explicit local-provider limitation rather than a reason to move process mechanics back into the PTY consumer. diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md index 50e27ccce1..f385e502d5 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md @@ -22,8 +22,8 @@ Status: implemented - `dsh-bash-local` 继续把 Bash 语义映射到普通的 `ctx.subprocess.spawn()`。 - `dsh-lsp-local` 通过 `ctx.fs` 读取源文件并验证包含关系,通过 `ctx.subprocess` 解析和启动语言服务器,并发送由提供方负责的文件 URI。其 JSON-RPC、池化、同步、取消和规范化保持不变。 -- `dsh-pty-local` 把持久 shell 语义映射到 `ctx.subprocess.spawnTerminal()`。本地 `node-pty` 与进程检查实现移入 `dsh-subprocess-local`;其他进程管理提供方则提供相同原语。异步写入超时后仍保留发送预留,直至提供方将写入结算;陈旧检查完成后,会针对当前发送恢复轮询。 -- `dsh-code-runtime-subprocess` 通过 `ctx.fs` 物化无依赖 runner,并通过 `ctx.subprocess` 启动它,从而在本地或远程执行世界中保留代码运行时的绑定与输出契约。它通过非插件子路径 `dsh-code-runtime-worker/runtime-host` 共享宿主侧 worker 机制,而不是复制这些机制。受堆上限约束的 worker 会在传输前拒绝过大的绑定帧;每个外层转发环节都会在序列化前执行相同的上限检查;launcher 会在回收 controller 前发布已接纳的终态帧,使继承 controller 管道的后代进程无法阻止完成;宿主仍会等待进程组完全停稳。 +- `dsh-pty-local` 把持久 shell 语义映射到 `ctx.subprocess.spawnTerminal()`。本地 `node-pty` 与进程检查实现移入 `dsh-subprocess-local`;其他进程管理提供方则提供相同原语。分配信号会在发布前解除关联,而就绪初始化仍保留设置阶段的取消。异步写入超时,或在该写入期间取消时前台信号发送失败,都会保留发送预留,直至提供方将写入结算;陈旧检查完成后,会针对当前发送恢复轮询。 +- `dsh-code-runtime-subprocess` 通过 `ctx.fs` 物化无依赖 runner,并通过 `ctx.subprocess` 启动它,从而在本地或远程执行世界中保留代码运行时的绑定与输出契约。它通过非插件子路径 `dsh-code-runtime-worker/runtime-host` 共享宿主侧 worker 机制,而不是复制这些机制。准备阶段让同一个生命周期信号贯穿文件系统解析、物化和可执行文件查找,使资源释放能够中止停滞的提供方操作。受堆上限约束的 worker 会在传输前拒绝过大的绑定帧;每个外层转发环节都会在序列化前执行相同的上限检查;launcher 会在回收 controller 前发布已接纳的终态帧,使继承 controller 管道的后代进程无法阻止完成;宿主仍会等待进程组完全停稳。 `dsh-code-runtime-worker` 仍是独立实现。它是较小的进程内后端,可用于无法假定已安装 Node 可执行文件的单文件分发。远程文件系统/进程组合选择 `dsh-code-runtime-subprocess`;它们不需要提供方专用的代码运行时包。 @@ -45,4 +45,4 @@ Status: implemented 基础接口更宽,一对文件系统/进程管理提供方必须在同一个执行世界上保持一致。新增操作仅限当前通用消费方所需的事实与生命周期机制;模型 schema、协议分帧、就绪策略和呈现不会渗入提供方。 -本地实现承接 `node-pty` 和平台进程检查,因为它负责本地终端机制。这种代码迁移不会削弱终端拆卸:dispose(资源释放)仍会等待受精确 PID 身份校验保护的后代进程和顶层终端进程完全停稳。 +本地实现承接 `node-pty` 和平台进程检查,因为它负责本地终端机制。这种代码迁移不会削弱终端拆卸:dispose(资源释放)会等待前台检查期间保留下来且受精确 PID 身份围栏保护的后代进程、在顶层进程退出后仍存活的 Linux 会话成员,以及顶层终端进程完全停稳。macOS 无法在 POSIX 会话 leader 退出后枚举该会话,因此在两次检查快照之间重新设定父进程的子进程仍是明确的本地提供方限制,而不是把进程机制移回 PTY 消费方的理由。 diff --git a/packages/pty/pty-local/README.i18n.yaml b/packages/pty/pty-local/README.i18n.yaml index e29a581f57..05e9b904f9 100644 --- a/packages/pty/pty-local/README.i18n.yaml +++ b/packages/pty/pty-local/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/pty/pty-local/README.md -README.md: 2c4ee44e8489cc6b53179769301e125dcca0ac8e -README.zh.md: 947bab00b9d6aa5fb7b321fbee84f4b2e420133c +README.md: 7d0cd2c26c6c58a64ae9d5fe5598c8aa6ff62915 +README.zh.md: 0b8216a9f599c224bd2888eb428213c9d725a338 diff --git a/packages/pty/pty-local/README.md b/packages/pty/pty-local/README.md index 2c4ee44e84..7d0cd2c26c 100644 --- a/packages/pty/pty-local/README.md +++ b/packages/pty/pty-local/README.md @@ -8,9 +8,9 @@ Persistent shell backend for `ctx.pty` over `ctx.subprocess.spawnTerminal`. It s The plugin injects `pty`, `sandbox`, `sandboxPolicy`, and `subprocess`, then registers the configured backend type (`shell`). `danger-full-access` starts the shell directly; confined modes wrap the exact shell argv through `ctx.sandbox`. The effective session mode is resolved at spawn. A change to a different effective mode is rejected before its `sandbox/mode` event commits while that owner has an open PTY or a spawn in progress; the fence is attached to the exact owner and therefore outlives a provider reload that retains existing sessions. Wait for creation to settle and close the sessions before changing modes, so a terminal opened with wider access cannot survive a downgrade. -Readiness combines a foreground-verified private bash prompt marker, provider-reported foreground stdin-wait facts, silence fallback, and absolute timeout. A marker is not ready until the printable tail after the latest owned marker exactly equals the controlled `PS1`, including when the OSC marker and prompt are split across data callbacks; echoed input or output following a delayed earlier prompt therefore cannot settle the current send. When bash prints the marker before the terminal provider publishes its return to the foreground process group, polling retains the candidate for `handoffGraceMs` past the ordinary silence bound so a coincident handoff can win. An interactive child that inherits `PROMPT_COMMAND` therefore cannot suppress inferred-idle readiness until the absolute timeout. Unknown foreground state is never a positive exact-idle signal. A foreground group's stdin wait that existed before a send is likewise not post-write readiness: the same group must be observed outside that wait before a later wait can settle the send, while a changed foreground group is new evidence. During unpublished startup, a fallback requires observed output; zero-output silence cannot publish an empty session, and timeout rejects the spawn. Cancellation closes the unpublished shell and rejects with the caller's exact abort reason; `PtyBackendCleanupError` separately preserves a cleanup failure. Incomplete terminal-control sequences are bounded by `maxReadBytes` and discarded through their terminator after crossing that limit; a trailing carriage return is carried across callbacks so split CRLF becomes one newline. +Readiness combines a foreground-verified private bash prompt marker, provider-reported foreground stdin-wait facts, silence fallback, and absolute timeout. A marker is not ready until the printable tail after the latest owned marker exactly equals the controlled `PS1`, including when the OSC marker and prompt are split across data callbacks; echoed input or output following a delayed earlier prompt therefore cannot settle the current send. When bash prints the marker before the terminal provider publishes its return to the foreground process group, polling retains the candidate for `handoffGraceMs` past the ordinary silence bound so a coincident handoff can win. An interactive child that inherits `PROMPT_COMMAND` therefore cannot suppress inferred-idle readiness until the absolute timeout. Unknown foreground state is never a positive exact-idle signal. A foreground group's stdin wait that existed before a send is likewise not post-write readiness: the same group must be observed outside that wait before a later wait can settle the send, while a changed foreground group is new evidence. During unpublished startup, a fallback requires observed output; zero-output silence cannot publish an empty session, and timeout rejects the spawn. Cancellation closes the unpublished shell and rejects with the caller's exact abort reason; `PtyBackendCleanupError` separately preserves a cleanup failure. The terminal-allocation signal is detached when allocation returns, while readiness initialization keeps the setup signal, so later cancellation cannot terminate a published persistent session. Incomplete terminal-control sequences are bounded by `maxReadBytes` and discarded through their terminator after crossing that limit; a trailing carriage return is carried across callbacks so split CRLF becomes one newline. -Send cancellation marks queued input as canceled before asking the terminal handle to signal the current foreground process group with a real `SIGINT`; if asynchronous pre-write inspection later settles, it cannot execute that input. Cancellation never emulates interruption by writing `\x03`, so raw-mode programs remain cancellable. A send that times out during an asynchronous provider write reports the timeout but retains the session's send slot until that write settles, so late bytes cannot interleave with a successor. Close starts provider-owned TERM-to-KILL whole-session cleanup and awaits quiescence after the terminal outcome. A cleanup failure does not cache a permanently rejected close; a later close retries the provider operation. +Send cancellation marks queued input as canceled before asking the terminal handle to signal the current foreground process group with a real `SIGINT`; if asynchronous pre-write inspection later settles, it cannot execute that input. Cancellation never emulates interruption by writing `\x03`, so raw-mode programs remain cancellable. A send that times out during an asynchronous provider write, or whose cancellation signal fails while that write remains pending, reports its result but retains the session's send slot until the write settles, so late bytes cannot interleave with a successor. Close starts provider-owned TERM-to-KILL whole-session cleanup and awaits quiescence after the terminal outcome. A cleanup failure does not cache a permanently rejected close; a later close retries the provider operation. ## Model Experience diff --git a/packages/pty/pty-local/README.zh.md b/packages/pty/pty-local/README.zh.md index 947bab00b9..0b8216a9f5 100644 --- a/packages/pty/pty-local/README.zh.md +++ b/packages/pty/pty-local/README.zh.md @@ -8,9 +8,9 @@ 该插件注入 `pty`、`sandbox`、`sandboxPolicy` 和 `subprocess`,然后注册所配置的后端类型(`shell`)。`danger-full-access` 会直接启动 shell;受限模式则通过 `ctx.sandbox` 包装确切的 shell argv。系统在 spawn 时解析会话的实际模式。当某个所有者存在开放的 PTY 或正在进行 spawn 时,如果配置变更会得到不同的实际模式,系统会在对应 `sandbox/mode` 事件提交前拒绝该变更。该限制绑定到确切所有者,因此即使提供方重新加载并保留现有会话,它仍然有效。更改模式前,请等待创建结算并关闭会话,避免以更宽权限打开的终端在权限降级后继续存在。 -就绪检测结合以下机制:由前台状态验证的私有 bash 提示符标记、提供方报告的前台 stdin 等待事实、静默回退和绝对超时。只有最近一个自有标记之后的可打印尾部与受控 `PS1` 完全相等时,系统才会把标记视为就绪;即使 OSC 标记和提示符被拆到多个数据回调中也是如此。因此,如果回显的输入或输出跟在延迟到达的先前提示符之后,该提示符无法使当前 send 完成。如果 bash 在终端提供方发布其重新取得前台进程组的状态前打印标记,轮询会在普通静默上限之后再保留该候选状态 `handoffGraceMs`,使恰好同时发生的前台交接有机会胜出。因此,继承 `PROMPT_COMMAND` 的交互式子进程无法持续压制推断空闲就绪,最多只能延续到绝对超时。未知的前台状态绝不会作为精确空闲的正向信号。同样,一次 send 之前就已存在的前台进程组 stdin 等待并不代表写入后就绪:必须先观察到同一进程组脱离该等待,之后再次进入等待才能使该次 send 完成;前台进程组发生变化则构成新的证据。尚未发布的启动过程中,回退路径要求已经观察到输出;零输出静默不能发布空会话,超时则拒绝 spawn。取消操作会关闭尚未发布的 shell,并以调用方提供的确切中止原因拒绝;`PtyBackendCleanupError` 会单独保留清理失败。未完成的终端控制序列受 `maxReadBytes` 限制;超过上限后,系统会丢弃内容直到其终止符。末尾的回车会跨回调保留,使拆分的 CRLF 合并为一个换行。 +就绪检测结合以下机制:由前台状态验证的私有 bash 提示符标记、提供方报告的前台 stdin 等待事实、静默回退和绝对超时。只有最近一个自有标记之后的可打印尾部与受控 `PS1` 完全相等时,系统才会把标记视为就绪;即使 OSC 标记和提示符被拆到多个数据回调中也是如此。因此,如果回显的输入或输出跟在延迟到达的先前提示符之后,该提示符无法使当前 send 完成。如果 bash 在终端提供方发布其重新取得前台进程组的状态前打印标记,轮询会在普通静默上限之后再保留该候选状态 `handoffGraceMs`,使恰好同时发生的前台交接有机会胜出。因此,继承 `PROMPT_COMMAND` 的交互式子进程无法持续压制推断空闲就绪,最多只能延续到绝对超时。未知的前台状态绝不会作为精确空闲的正向信号。同样,一次 send 之前就已存在的前台进程组 stdin 等待并不代表写入后就绪:必须先观察到同一进程组脱离该等待,之后再次进入等待才能使该次 send 完成;前台进程组发生变化则构成新的证据。尚未发布的启动过程中,回退路径要求已经观察到输出;零输出静默不能发布空会话,超时则拒绝 spawn。取消操作会关闭尚未发布的 shell,并以调用方提供的确切中止原因拒绝;`PtyBackendCleanupError` 会单独保留清理失败。终端分配返回时,分配信号会解除关联;就绪初始化则保留设置阶段信号,因此后续取消无法终止已发布的持久会话。未完成的终端控制序列受 `maxReadBytes` 限制;超过上限后,系统会丢弃内容直到其终止符。末尾的回车会跨回调保留,使拆分的 CRLF 合并为一个换行。 -取消发送会先把排队输入标记为已取消,再请求终端句柄向当前前台进程组发送真正的 `SIGINT`;如果异步的写入前检查随后才结算,也无法执行该输入。取消绝不会通过写入 `\x03` 模拟中断,因此原始模式程序仍可取消。在提供方异步写入期间超时的发送会报告超时,但会继续占用该会话的发送槽位,直至写入结算,从而避免延迟到达的字节与后续发送交错。关闭操作启动由提供方负责的 TERM→KILL 全会话清理,并在终端结果之后等待完全停稳。清理失败不会缓存成永久拒绝的关闭操作;后续关闭会重试提供方操作。 +取消发送会先把排队输入标记为已取消,再请求终端句柄向当前前台进程组发送真正的 `SIGINT`;如果异步的写入前检查随后才结算,也无法执行该输入。取消绝不会通过写入 `\x03` 模拟中断,因此原始模式程序仍可取消。发送在提供方异步写入期间超时,或在该写入仍未完成时其取消信号发送失败,都会报告各自结果,但继续占用会话的发送槽位,直至写入结算,从而避免延迟到达的字节与后续发送交错。关闭操作启动由提供方负责的 TERM→KILL 全会话清理,并在终端结果之后等待完全停稳。清理失败不会缓存成永久拒绝的关闭操作;后续关闭会重试提供方操作。 ## 模型体验 diff --git a/packages/pty/pty-local/src/index.ts b/packages/pty/pty-local/src/index.ts index a8d99cb243..62b5fdb71a 100644 --- a/packages/pty/pty-local/src/index.ts +++ b/packages/pty/pty-local/src/index.ts @@ -101,15 +101,30 @@ export class LocalPtyBackend implements PtyBackend { ensureSandboxModeFence(this.ctx, spec.owner) const argv = spawnArgv(this.ctx, this.config, spec) if (argv[0] === undefined) throw new Error('pty-local: sandbox returned empty argv') - const terminal = await this.spawnTerminal({ - argv, - cwd: spec.cwd ?? this.ctx.sandboxPolicy.workspaceRoot, - env: childEnvironment(spec), - rows: this.config.rows, - cols: this.config.cols, - graceMs: this.config.disposeGraceMs, - signal: spec.signal, - }) + let terminalSignal: AbortSignal | undefined + let detachSetupSignal: (() => void) | undefined + if (spec.signal !== undefined) { + const source = spec.signal + const controller = new AbortController() + const onAbort = (): void => { controller.abort(source.reason) } + source.addEventListener('abort', onAbort, { once: true }) + detachSetupSignal = () => { source.removeEventListener('abort', onAbort) } + terminalSignal = controller.signal + } + let terminal: SubprocessTerminalHandle + try { + terminal = await this.spawnTerminal({ + argv, + cwd: spec.cwd ?? this.ctx.sandboxPolicy.workspaceRoot, + env: childEnvironment(spec), + rows: this.config.rows, + cols: this.config.cols, + graceMs: this.config.disposeGraceMs, + signal: terminalSignal, + }) + } finally { + detachSetupSignal?.() + } const session = this.createSession(terminal, this.config) try { await session.initialize(spec.signal) diff --git a/packages/pty/pty-local/src/session.ts b/packages/pty/pty-local/src/session.ts index f984648f27..9a0beed6ff 100644 --- a/packages/pty/pty-local/src/session.ts +++ b/packages/pty/pty-local/src/session.ts @@ -473,17 +473,23 @@ export class LocalPtySession implements PtyBackendSession { this.active = undefined } - private failActive(error: unknown): void { + private failActive(error: unknown, retainOwnership = false): void { const operation = this.active if (operation === undefined) return - this.clearActive() + if (retainOwnership) { + this.stopPolling() + this.activeAbort?.() + this.activeAbort = undefined + } else { + this.clearActive() + } operation.fail(error) } private interrupt(operation: LocalSendOperation): void { if (this.active !== operation) return void this.terminal.signalForeground('SIGINT').catch((error: unknown) => { - if (this.active === operation) this.failActive(error) + if (this.active === operation) this.failActive(error, this.writing === operation) }) } diff --git a/packages/pty/pty-local/tests/index.spec.ts b/packages/pty/pty-local/tests/index.spec.ts index a8753deaf3..23e6f4bc89 100644 --- a/packages/pty/pty-local/tests/index.spec.ts +++ b/packages/pty/pty-local/tests/index.spec.ts @@ -187,6 +187,51 @@ describe('LocalPtyBackend startup rollback', () => { expect(initialized).toHaveBeenCalledWith(undefined) }) + it('forwards setup cancellation only while terminal allocation is unpublished', async () => { + const ctx = new Context() + await ctx.plugin(EmptySandbox) + await ctx.plugin(SandboxPolicyService, { mode: 'danger-full-access', workspaceRoot: '/tmp' }) + + const publishedController = new AbortController() + let publishedSignal: AbortSignal | undefined + const published = new LocalPtyBackend( + ctx, + config(), + async (spawnSpec) => { + publishedSignal = spawnSpec.signal + return terminalHandle() + }, + () => stubLocalSession(), + ) + await published.spawn(spec(agent(ctx), publishedController.signal)) + expect(publishedSignal).toBeDefined() + expect(publishedSignal).not.toBe(publishedController.signal) + publishedController.abort(new Error('originating turn ended')) + expect(publishedSignal?.aborted).toBe(false) + + const pendingController = new AbortController() + const seen = Promise.withResolvers() + const pending = new LocalPtyBackend( + ctx, + config(), + async spawnSpec => await new Promise((_resolve, reject) => { + const setupSignal = spawnSpec.signal as AbortSignal + seen.resolve(setupSignal) + const onAbort = (): void => { + reject(setupSignal.reason instanceof Error ? setupSignal.reason : new Error(String(setupSignal.reason))) + } + setupSignal.addEventListener('abort', onAbort, { once: true }) + }), + () => stubLocalSession(), + ) + const spawning = pending.spawn(spec(agent(ctx), pendingController.signal)) + const pendingSignal = await seen.promise + const reason = new Error('cancel pending allocation') + pendingController.abort(reason) + await expect(spawning).rejects.toBe(reason) + expect(pendingSignal.aborted).toBe(true) + }) + it('composes the default local session around a spawned terminal', async () => { const ctx = new Context() await ctx.plugin(EmptySandbox) diff --git a/packages/pty/pty-local/tests/local.spec.ts b/packages/pty/pty-local/tests/local.spec.ts index c02a9563ea..df05438864 100644 --- a/packages/pty/pty-local/tests/local.spec.ts +++ b/packages/pty/pty-local/tests/local.spec.ts @@ -1,4 +1,4 @@ -import { mkdtempSync, realpathSync, rmSync } from 'node:fs' +import { existsSync, mkdtempSync, readFileSync, realpathSync, rmSync } from 'node:fs' import { tmpdir } from 'node:os' import { join } from 'node:path' import { afterEach, describe, expect, it } from 'vitest' @@ -156,6 +156,47 @@ describe('pty-local real shell', () => { expect(() => process.kill(pid, 0)).toThrow() }, 10_000) + it('reaps a disowned same-session descendant after the shell exits naturally', async () => { + const { ctx, root, agent } = await harness('danger-full-access') + const created = await ctx.pty.spawn(agent, { type: 'shell' }) + const pidFile = join(root, 'disowned.pid') + let pid: number | undefined + try { + const background = ctx.pty.startSend(agent, created.sessionId, { + text: `sh -c 'trap "" TERM; printf "%s" "$$" > "$1"; sleep 60' dsh "${pidFile}" & disown`, + submit: true, + }) + await background.done + const pidDeadline = Date.now() + 2_000 + let childPid = 0 + while (childPid === 0 && Date.now() < pidDeadline) { + if (existsSync(pidFile)) childPid = Number(readFileSync(pidFile, 'utf8')) + if (childPid > 0) break + await new Promise(resolve => setTimeout(resolve, 10)) + } + expect(existsSync(pidFile), ctx.pty.read(agent, created.sessionId, { offset: 0, count: 100 }).text).toBe(true) + expect(childPid).toBeGreaterThan(0) + pid = childPid + expect(() => process.kill(childPid, 0)).not.toThrow() + await ctx.pty.startSend(agent, created.sessionId, { text: 'exit', submit: true }).done + const deadline = Date.now() + 2_000 + while (ctx.pty.list(agent)[0]?.status.kind !== 'exited' && Date.now() < deadline) { + await new Promise(resolve => setTimeout(resolve, 10)) + } + expect(ctx.pty.list(agent)[0]?.status.kind).toBe('exited') + await ctx.pty.kill(agent, created.sessionId) + expect(() => process.kill(childPid, 0)).toThrow() + } finally { + if (pid !== undefined) { + try { + process.kill(pid, 'SIGKILL') + } catch (_alreadyReaped) { + // Product cleanup is the expected path; this only contains a failed regression. + } + } + } + }, 10_000) + it('cancels a slow-starting raw-mode foreground process with a real SIGINT', async () => { const { ctx, agent } = await harness('danger-full-access', { idleSilenceMs: 10_000, diff --git a/packages/pty/pty-local/tests/session.spec.ts b/packages/pty/pty-local/tests/session.spec.ts index 6346a6bba2..120b2881e1 100644 --- a/packages/pty/pty-local/tests/session.spec.ts +++ b/packages/pty/pty-local/tests/session.spec.ts @@ -27,6 +27,7 @@ class FakeInspector implements ProcessInspector { foregroundPgid() { return this.pgid } isStdinWaiting() { return this.waiting } processTree() { return this.members } + processSession() { return [] } isAlive(identity: ProcessIdentity) { return this.alive.has(identity.pid) } signalGroup(pgid: number, signal: PtySignal) { if (this.throwGroup) throw new Error('group failed') @@ -325,6 +326,32 @@ describe('LocalPtySession readiness and output', () => { expect((await next.done).waitReason).toBe('inferred_idle') }) + it('retains send ownership when cancellation signalling fails during an asynchronous write', async () => { + vi.useFakeTimers() + const terminal = new FakeTerminal() + const inspector = new FakeInspector() + const session = makeSession(terminal, inspector, config()) + await initialize(session, terminal) + + const writeGate = Promise.withResolvers() + terminal.write = async () => { await writeGate.promise } + terminal.signalForeground = async () => { throw new Error('interrupt failed') } + const operation = session.startSend({ text: 'slow write', submit: true }) + await Promise.resolve() + await Promise.resolve() + const rejected = expect(operation.done).rejects.toThrow('interrupt failed') + expect(operation.cancel()).toBe(true) + await rejected + expect(() => session.startSend({ text: 'must wait', submit: true })).toThrow('active send') + + writeGate.resolve(undefined) + await Promise.resolve() + await Promise.resolve() + const next = session.startSend({ text: '', submit: false }) + await vi.advanceTimersByTimeAsync(100) + expect((await next.done).waitReason).toBe('inferred_idle') + }) + it('handles startup exit, unknown exit signals, cancel-write failure, and stale polls', async () => { vi.useFakeTimers() const startupTerminal = new FakeTerminal() diff --git a/packages/subprocess/subprocess-local/README.i18n.yaml b/packages/subprocess/subprocess-local/README.i18n.yaml index d931d3574b..72f3c93da9 100644 --- a/packages/subprocess/subprocess-local/README.i18n.yaml +++ b/packages/subprocess/subprocess-local/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/subprocess/subprocess-local/README.md -README.md: 6bc1003ae5903bb5640728bc79c3f9042fddbe7a -README.zh.md: 3c9ce73c7fcbeec9b17d73f212ecdcb6842ec133 +README.md: 38b26c33f3854acdb4a2bb48612da73361d727ba +README.zh.md: fc4a5d4961fb7618b9db84b21063f907361a26d8 diff --git a/packages/subprocess/subprocess-local/README.md b/packages/subprocess/subprocess-local/README.md index 6bc1003ae5..38b26c33f3 100644 --- a/packages/subprocess/subprocess-local/README.md +++ b/packages/subprocess/subprocess-local/README.md @@ -11,7 +11,7 @@ Local implementation of the [`@deepseek-ai/dsh-subprocess`](../subprocess/README - **Credential scrub + explicit merge** — `process.env` minus credential-shaped vars (`*KEY*`/`*SECRET*`/`*TOKEN*`) and all ambient `DSH_*` names; the spec's explicit `env` merges after that scrub with no namespace validation, so a deliberately supplied credential or current `DSH_*` fact wins while stale nested-harness identity cannot leak in ambiently. Supplied stdin is written and closed; otherwise fd 0 is `/dev/null`. See the [stdin/env Agent Note](../../../.agents/notes/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-surface.md) and [managed environment Agent Note](../../../.agents/notes/implemented/feature/2026-07-10-agent-session-identity-and-log-location.md). - **Offset-based reads** — collect-mode readers return deltas in whole-stream byte coordinates; the service never holds a cursor, so consumer-owned cursors (the bash background read path) and full-stream re-reads coexist, before and after settlement. - **Execution-world coordinates** — `cwd` is the host process cwd, `runtimeRoot` is an owner-private temporary directory removed on disposal, and `resolveExecutable` checks absolute files or searches the scrubbed effective PATH with platform-aware executable extensions. -- **Terminal-process ownership** — `spawnTerminal` allocates `node-pty`, bridges UTF-8 terminal bytes, inspects and signals the current foreground process group, and cleans descendants before the top-level shell. Linux `/proc`/syscall and macOS `ps` inspectors retain exact pid/start identity so pid reuse cannot redirect cleanup; the higher PTY backend owns prompt readiness, buffers, and model-facing operations. +- **Terminal-process ownership** — `spawnTerminal` allocates `node-pty`, bridges UTF-8 terminal bytes, inspects and signals the current foreground process group, and cleans descendants before the top-level shell. Each foreground inspection retains exact identities from the rooted tree; Linux also enumerates the POSIX session after its leader exits. A previously observed macOS descendant and any same-session Linux member therefore remain fenced after reparenting, while pid/start identity prevents cleanup from following PID reuse. The higher PTY backend owns prompt readiness, buffers, and model-facing operations. - **Terminate-and-join disposal** — the service retains live handles only so its own disposal can escalate every running tree and await its exit; settled and spawn-failed handles leave the live set on settlement. ## Model Experience @@ -26,7 +26,7 @@ No direct invalidation; the named consumers own any request-prefix changes. - **Windows tree support is best-effort and untested in CI** — termination routes through `taskkill /PID /T /F` with all outcomes contained (absent tree, races, missing binary), and liveness falls back to the direct-child boundary; the suites cover the routing through an injected runner only, and `packages/subprocess/*` is excluded from the Windows test matrix. - **Terminal process inspection is Linux/macOS only** — the terminal primitive fails when its inspector has no supported platform implementation; Linux exact probes cover x64 and arm64, while macOS uses `ps` snapshots. -- **A daemonized terminal descendant can escape the captured tree** — a child that reparents before teardown is no longer discoverable from the `node-pty` root. The local provider accepts this gap rather than signal the root PID's POSIX session, which can include unrelated launcher processes. +- **A daemonized terminal descendant can still escape the observable boundary** — on macOS, a child that reparents before any foreground-inspection snapshot is no longer discoverable from the `node-pty` root; on Linux, a child that calls `setsid` leaves both the tree and owned terminal session. The local provider does not add a continuous process-table monitor. - **The credential scrub is a name heuristic** — `*KEY*`/`*SECRET*`/`*TOKEN*` only; differently-named secrets (e.g. `*PASSWORD*`) pass through, and a whitelist for over-scrubbed vars is noted future work. - **Completed spill files are not deleted** — bounded full-output recovery files (and the private per-process spill dir) accumulate under the OS tmpdir until something external cleans them; oversize incomplete spills are discarded and deletion is attempted immediately, but a cleanup failure can leave a bounded file behind. diff --git a/packages/subprocess/subprocess-local/README.zh.md b/packages/subprocess/subprocess-local/README.zh.md index 3c9ce73c7f..fc4a5d4961 100644 --- a/packages/subprocess/subprocess-local/README.zh.md +++ b/packages/subprocess/subprocess-local/README.zh.md @@ -11,7 +11,7 @@ - **凭据清除 + 显式合并**:以 `process.env` 为基础,移除形似凭据的变量(`*KEY*`/`*SECRET*`/`*TOKEN*`)和所有环境中已有的 `DSH_*` 名称;spec 的显式 `env` 在该清除之后合并且不做命名空间校验,因此有意提供的凭据或当前 `DSH_*` 事实会胜出,而陈旧的嵌套 harness 身份无法从环境中隐式漏入。提供的 stdin 会被写入后关闭;否则 fd 0 指向 `/dev/null`。参见 [stdin/env Agent Note(agent 决策记录)](../../../.agents/notes/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-surface.md)与[受管环境 Agent Note](../../../.agents/notes/implemented/feature/2026-07-10-agent-session-identity-and-log-location.md)。 - **基于偏移量的读取**:收集模式的读取器以全流字节坐标返回增量;服务自身从不持有游标,因此消费方自有的游标(bash 的后台读取路径)与完整流重读可以共存,结算前后皆然。 - **执行世界坐标**:`cwd` 是宿主进程 cwd,`runtimeRoot` 是所有者私有的临时目录,在资源释放时删除;`resolveExecutable` 检查绝对文件,或使用平台感知的可执行扩展名在清理后的有效 PATH 中查找。 -- **终端进程所有权**:`spawnTerminal` 分配 `node-pty`,桥接 UTF-8 终端字节,检查当前前台进程组并向其发送信号,并先于顶层 shell 清理后代。Linux 的 `/proc`/syscall 检查器与 macOS 的 `ps` 检查器会保留精确的 pid/启动身份,使 PID 复用无法把清理重定向到其他进程;上层 PTY 后端负责提示符就绪检测、缓冲和面向模型的操作。 +- **终端进程所有权**:`spawnTerminal` 分配 `node-pty`,桥接 UTF-8 终端字节,检查当前前台进程组并向其发送信号,并先于顶层 shell 清理后代。每次前台检查都会保留有根进程树中的精确身份;Linux 还会在会话 leader 退出后枚举该 POSIX 会话。因此,先前观察到的 macOS 后代以及任何同会话 Linux 成员在重新设定父进程后仍受身份围栏保护,而 pid/启动身份可防止清理因 PID 复用而跟随到其他进程。上层 PTY 后端负责提示符就绪检测、缓冲和面向模型的操作。 - **先终止再等待退出的 dispose**:服务保留存活句柄,只为让自身的 dispose 能对每个仍在运行的进程树执行升级并等待其退出;已结算与 spawn 失败的句柄在结算时即离开存活集合。 ## 模型体验 @@ -26,7 +26,7 @@ - **Windows 进程树支持仅为尽力而为,且未经 CI 测试**:终止经由 `taskkill /PID /T /F` 完成,所有结果都被就地吸收,不向外抛出(进程树已不存在、竞态、二进制缺失),存活探测则回退到直接子进程边界;测试套件只通过注入的运行器覆盖这条路由,且 `packages/subprocess/*` 被排除在 Windows 测试矩阵之外。 - **终端进程检查仅支持 Linux/macOS**:检查器没有受支持的平台实现时,终端原语会失败;Linux 精确探针覆盖 x64 与 arm64,macOS 使用 `ps` 快照。 -- **守护化的终端后代可能逃离已捕获进程树**:子进程若在拆卸前重新设定父进程,便无法再从 `node-pty` 根发现。本地提供方接受这个缺口,不向根 PID 的 POSIX 会话发送信号,因为其中可能包含无关的启动器进程。 +- **守护化的终端后代仍可能逃出可观察边界**:在 macOS 上,子进程若在任何前台检查快照产生前重新设定父进程,便无法再从 `node-pty` 根发现;在 Linux 上,调用 `setsid` 的子进程会同时离开进程树与该提供方拥有的终端会话。本地提供方不会增加持续运行的进程表监视器。 - **凭据清除依赖名称启发式规则**:只匹配 `*KEY*`/`*SECRET*`/`*TOKEN*`;名称不同的 secret(例如 `*PASSWORD*`)会继续传递,对误删变量引入白名单属于已记录的后续工作。 - **不会删除已完成的 spill 文件**:有界的完整输出恢复文件(以及每个进程的私有 spill 目录)会在 OS tmpdir 下累积,直到外部机制进行清理;超大的不完整 spill 会被丢弃并立即尝试删除,但清理失败可能留下一个有界文件。 diff --git a/packages/subprocess/subprocess-local/src/process-inspector.ts b/packages/subprocess/subprocess-local/src/process-inspector.ts index f57184f289..48b01e18e3 100644 --- a/packages/subprocess/subprocess-local/src/process-inspector.ts +++ b/packages/subprocess/subprocess-local/src/process-inspector.ts @@ -16,6 +16,8 @@ export interface ProcessInspector { isStdinWaiting(pgid: number): boolean /** Return the root and its current transitive descendants, children first. */ processTree(rootPid: number): ProcessIdentity[] + /** Return current members of one POSIX process session when the platform exposes them. */ + processSession(sessionId: number): ProcessIdentity[] /** Return whether the exact identity remains a non-quiescent process. */ isAlive(identity: ProcessIdentity): boolean signalGroup(pgid: number, signal: SubprocessTerminalSignal): void @@ -201,6 +203,7 @@ abstract class PosixProcessInspector implements ProcessInspector { abstract foregroundPgid(shellPid: number): number | undefined abstract isStdinWaiting(pgid: number): boolean abstract processTree(rootPid: number): ProcessIdentity[] + abstract processSession(sessionId: number): ProcessIdentity[] abstract isAlive(identity: ProcessIdentity): boolean signalGroup(pgid: number, signal: SubprocessTerminalSignal): void { @@ -272,6 +275,13 @@ class LinuxProcessInspector extends PosixProcessInspector { return processTree(entries, rootPid) } + processSession(sessionId: number): ProcessIdentity[] { + return numericEntries(this.internals, '/proc').flatMap((pid) => { + const stat = readLinuxStat(this.internals, pid) + return stat?.session === sessionId ? [{ pid, started: stat.started }] : [] + }) + } + isAlive(identity: ProcessIdentity): boolean { const stat = readLinuxStat(this.internals, identity.pid) return stat?.started === identity.started && !/^[ZXx]$/.test(stat.state) @@ -307,6 +317,10 @@ class MacProcessInspector extends PosixProcessInspector { return processTree(macProcessTable(this.internals), rootPid) } + processSession(_sessionId: number): ProcessIdentity[] { + return [] + } + isAlive(identity: ProcessIdentity): boolean { return macProcessTable(this.internals).some(entry => entry.pid === identity.pid && entry.started === identity.started) } diff --git a/packages/subprocess/subprocess-local/src/terminal.ts b/packages/subprocess/subprocess-local/src/terminal.ts index d4b7bd26d1..8446f11b8f 100644 --- a/packages/subprocess/subprocess-local/src/terminal.ts +++ b/packages/subprocess/subprocess-local/src/terminal.ts @@ -36,6 +36,7 @@ export class LocalTerminalHandle implements SubprocessTerminalHandle { private exited = false private termination: Promise | undefined private removeAbort: (() => void) | undefined + private trackedDescendants: ProcessIdentity[] = [] /** * @param terminal - allocated node-pty process. @@ -86,6 +87,7 @@ export class LocalTerminalHandle implements SubprocessTerminalHandle { // Local inspection is synchronous; the seam returns a promise for remote transports. // eslint-disable-next-line @typescript-eslint/require-await async inspectForeground(): Promise { + this.descendants() const processGroupId = this.inspector.foregroundPgid(this.pid) if (processGroupId === undefined) return undefined return { @@ -145,7 +147,12 @@ export class LocalTerminalHandle implements SubprocessTerminalHandle { } private descendants(): ProcessIdentity[] { - return this.inspector.processTree(this.pid).filter(member => member.pid !== this.pid) + this.trackedDescendants = this.survivors(this.unionMembers( + this.trackedDescendants, + this.inspector.processTree(this.pid), + this.inspector.processSession(this.pid), + ).filter(member => member.pid !== this.pid)) + return this.trackedDescendants } private async waitForMembers(members: ProcessIdentity[]): Promise { diff --git a/packages/subprocess/subprocess-local/tests/local.spec.ts b/packages/subprocess/subprocess-local/tests/local.spec.ts index 7ff10a7348..93be25acd0 100644 --- a/packages/subprocess/subprocess-local/tests/local.spec.ts +++ b/packages/subprocess/subprocess-local/tests/local.spec.ts @@ -111,6 +111,7 @@ describe('LocalSubprocessService', () => { foregroundPgid: () => undefined, isStdinWaiting: () => false, processTree: () => [], + processSession: () => [], isAlive: () => false, signalGroup: () => {}, signalProcess: () => {}, @@ -175,6 +176,7 @@ describe('LocalSubprocessService', () => { foregroundPgid: () => 123, isStdinWaiting: () => false, processTree: () => [{ pid: 124, started: 'child' }], + processSession: () => [], isAlive: identity => alive.has(identity.pid), signalGroup: () => {}, signalProcess: () => {}, diff --git a/packages/subprocess/subprocess-local/tests/process-inspector.spec.ts b/packages/subprocess/subprocess-local/tests/process-inspector.spec.ts index e8ad2fd11e..bcc3ddb091 100644 --- a/packages/subprocess/subprocess-local/tests/process-inspector.spec.ts +++ b/packages/subprocess/subprocess-local/tests/process-inspector.spec.ts @@ -86,6 +86,13 @@ describe('Linux process inspector', () => { { pid: 10, started: '500' }, ]) expect(inspector.processTree(99)).toEqual([]) + expect(inspector.processSession(30)).toEqual([ + { pid: 10, started: '500' }, + { pid: 11, started: '501' }, + { pid: 12, started: '502' }, + { pid: 13, started: '503' }, + ]) + expect(inspector.processSession(99)).toEqual([]) expect(inspector.isAlive({ pid: 10, started: '500' })).toBe(true) expect(inspector.isAlive({ pid: 10, started: 'old' })).toBe(false) inspector.signalGroup(40, 'SIGINT') @@ -197,6 +204,7 @@ describe('macOS process inspector', () => { { pid: 10, started: 'Mon Jul 21 10:00:00 2026' }, ]) expect(inspector.processTree(99)).toEqual([]) + expect(inspector.processSession(10)).toEqual([]) expect(inspector.isAlive({ pid: 11, started: 'Mon Jul 21 10:00:01 2026' })).toBe(true) inspector.signalGroup(55, 'SIGTSTP') inspector.signalProcess({ pid: 11, started: 'Mon Jul 21 10:00:01 2026' }, 'SIGKILL') diff --git a/packages/subprocess/subprocess-local/tests/terminal.spec.ts b/packages/subprocess/subprocess-local/tests/terminal.spec.ts index 417fd62517..0202a3f33e 100644 --- a/packages/subprocess/subprocess-local/tests/terminal.spec.ts +++ b/packages/subprocess/subprocess-local/tests/terminal.spec.ts @@ -51,6 +51,7 @@ class FakeInspector implements ProcessInspector { pgid: number | undefined = 456 waiting = false members: ProcessIdentity[] = [] + sessionMembers: ProcessIdentity[] = [] readonly alive = new Set() readonly groups: Array<[number, SubprocessTerminalSignal]> = [] readonly processes: Array<[number, 'SIGTERM' | 'SIGKILL']> = [] @@ -61,6 +62,7 @@ class FakeInspector implements ProcessInspector { foregroundPgid() { return this.pgid } isStdinWaiting() { return this.waiting } processTree() { return this.members } + processSession() { return this.sessionMembers } isAlive(identity: ProcessIdentity) { return this.alive.has(identity.pid) } signalGroup(pgid: number, signal: SubprocessTerminalSignal) { if (this.throwGroup) throw new Error('group failed') @@ -158,6 +160,36 @@ describe('LocalTerminalHandle', () => { expect(await waiting).toBe(true) }) + it('cleans a same-session descendant after the top-level shell exits naturally', async () => { + const pty = new FakePty() + const inspector = new FakeInspector() + const disowned = { pid: 124, started: 'disowned' } + inspector.processSession = () => inspector.alive.has(disowned.pid) ? [disowned] : [] + inspector.alive.add(124) + const handle = new LocalTerminalHandle(pty.asPty(), inspector, 20) + + pty.emitExit() + + expect(await handle.waitForExit()).toBe(true) + expect(inspector.processes).toEqual([[124, 'SIGTERM']]) + }) + + it('retains an inspected descendant after it reparents away from the shell', async () => { + const pty = new FakePty() + const inspector = new FakeInspector() + const descendant = { pid: 124, started: 'observed' } + inspector.members = [descendant] + inspector.alive.add(descendant.pid) + const handle = new LocalTerminalHandle(pty.asPty(), inspector, 20) + + await handle.inspectForeground() + inspector.members = [] + pty.emitExit() + + expect(await handle.waitForExit()).toBe(true) + expect(inspector.processes).toEqual([[124, 'SIGTERM']]) + }) + it('rescans for descendants forked during TERM', async () => { const pty = new FakePty() const inspector = new FakeInspector() diff --git a/packages/typert/README.i18n.yaml b/packages/typert/README.i18n.yaml index 384c9cdb94..13b8df9f71 100644 --- a/packages/typert/README.i18n.yaml +++ b/packages/typert/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/code-runtime/code-runtime-subprocess/README.md -README.md: db4478fe799dce1bbdb7e3ae3d31947515aae151 -README.zh.md: 238f5f66db58edb99945cfa43299635b4f8f36a8 +README.md: fd8cfccdb333018e551459195564daad157d3867 +README.zh.md: b5b495e22bbf4328d82925cc793588ee45e5c6e4 From 65264f6c80f034b0c1c0bd448c5635506b98b792 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 03:17:40 +0800 Subject: [PATCH 010/108] refactor(subprocess): share terminal lifecycle --- ...rtable-execution-world-consumers.i18n.yaml | 4 +- ...7-28-portable-execution-world-consumers.md | 2 +- ...8-portable-execution-world-consumers.zh.md | 2 +- .../subprocess-local/src/terminal.ts | 47 ++----- .../subprocess/subprocess/README.i18n.yaml | 4 +- packages/subprocess/subprocess/README.md | 1 + packages/subprocess/subprocess/README.zh.md | 1 + packages/subprocess/subprocess/src/index.ts | 2 + .../subprocess/src/terminal-lifecycle.ts | 104 +++++++++++++++ .../tests/terminal-lifecycle.spec.ts | 125 ++++++++++++++++++ 10 files changed, 248 insertions(+), 44 deletions(-) create mode 100644 packages/subprocess/subprocess/src/terminal-lifecycle.ts create mode 100644 packages/subprocess/subprocess/tests/terminal-lifecycle.spec.ts diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml index f97b09b940..231037f054 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md -2026-07-28-portable-execution-world-consumers.md: 3d1928be882fb3fb621e733649146445c69e8a5b -2026-07-28-portable-execution-world-consumers.zh.md: f385e502d5d737fe0d5ae6e28ac7a3a097d85a6b +2026-07-28-portable-execution-world-consumers.md: a5df393f31a25b7d16e63af240882e1d94093e34 +2026-07-28-portable-execution-world-consumers.zh.md: 87f11fa53b5b7579969dcb15dc3aaad978018773 diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md index 3d1928be88..a5df393f31 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md @@ -16,7 +16,7 @@ Ordinary pipes do not cover one requirement. A persistent terminal needs PTY all The filesystem interface owns the path facts that another capability needs without exposing its opaque target identity: a canonical process path, canonical `file:` URI, containment, and a bounded stable-handle text read. The existing text and mutation operations remain filesystem-owned. -The subprocess interface owns the process coordinates and primitives: canonical cwd, private runtime storage, executable lookup, ordinary raw or collected process spawning, and `spawnTerminal()`. The terminal operation is one deep primitive whose handle owns byte I/O, foreground groups, signalling, TERM-to-KILL session cleanup, and a quiescence wait. Prompt detection, idle inference, scrollback, sandbox policy, and owner lifecycle remain in the PTY consumer. +The subprocess interface owns the process coordinates and primitives: canonical cwd, private runtime storage, executable lookup, ordinary raw or collected process spawning, and `spawnTerminal()`. The terminal operation is one deep primitive whose handle owns byte I/O, foreground groups, signalling, TERM-to-KILL session cleanup, and a quiescence wait. The interface package also exports a provider-neutral lifecycle controller that joins top-level settlement, lifetime cancellation, retryable provider cleanup, and bounded quiescence observation; each implementation supplies only its session-cleanup transaction. Prompt detection, idle inference, scrollback, sandbox policy, and owner lifecycle remain in the PTY consumer. Generic consumers use that execution world: diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md index f385e502d5..87f11fa53b 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md @@ -16,7 +16,7 @@ Status: implemented 文件系统接口负责其他能力需要的路径事实,同时不公开其不透明目标身份:规范化进程路径、规范化 `file:` URI、包含关系,以及通过稳定句柄执行的有界文本读取。现有文本与变更操作仍归文件系统负责。 -进程管理接口负责进程运行坐标与原语:规范化 cwd、私有运行时存储、可执行文件查找、以原始或收集模式 spawn 普通进程,以及 `spawnTerminal()`。终端操作是一项深层原语,其句柄负责字节 I/O、前台进程组管理、信号发送、TERM→KILL 会话清理以及等待完全停稳。提示符检测、空闲推断、scrollback、沙箱策略和所有者生命周期仍由 PTY 消费方负责。 +进程管理接口负责进程运行坐标与原语:规范化 cwd、私有运行时存储、可执行文件查找、以原始或收集模式 spawn 普通进程,以及 `spawnTerminal()`。终端操作是一项深层原语,其句柄负责字节 I/O、前台进程组管理、信号发送、TERM→KILL 会话清理以及等待完全停稳。接口包还导出一个提供方无关的生命周期控制器,用于组合顶层结算、生命周期取消、可重试的提供方清理与有界的完全停稳观测;每个实现只需提供自身的会话清理事务。提示符检测、空闲推断、scrollback、沙箱策略和所有者生命周期仍由 PTY 消费方负责。 通用消费方使用该执行世界: diff --git a/packages/subprocess/subprocess-local/src/terminal.ts b/packages/subprocess/subprocess-local/src/terminal.ts index 8446f11b8f..2f1e4d4058 100644 --- a/packages/subprocess/subprocess-local/src/terminal.ts +++ b/packages/subprocess/subprocess-local/src/terminal.ts @@ -4,6 +4,7 @@ import { Buffer } from 'node:buffer' import { constants } from 'node:os' import { PassThrough } from 'node:stream' import type { IDisposable, IPty } from 'node-pty' +import { SubprocessTerminalLifecycle } from '@deepseek-ai/dsh-subprocess' import type { SubprocessOutcome, SubprocessTerminalForeground, @@ -33,9 +34,8 @@ export class LocalTerminalHandle implements SubprocessTerminalHandle { private readonly outcome = Promise.withResolvers() private readonly dataDisposable: IDisposable private readonly exitDisposable: IDisposable + private readonly lifecycle: SubprocessTerminalLifecycle private exited = false - private termination: Promise | undefined - private removeAbort: (() => void) | undefined private trackedDescendants: ProcessIdentity[] = [] /** @@ -63,12 +63,11 @@ export class LocalTerminalHandle implements SubprocessTerminalHandle { }) this.terminate() }) - if (signal !== undefined) { - const onAbort = (): void => { this.terminate() } - signal.addEventListener('abort', onAbort, { once: true }) - this.removeAbort = () => { signal.removeEventListener('abort', onAbort) } - if (signal.aborted) this.terminate() - } + this.lifecycle = new SubprocessTerminalLifecycle({ + done: this.done, + cleanup: () => this.closeOnce(), + signal, + }) } // node-pty writes synchronously; the seam returns a promise for remote transports. @@ -109,37 +108,11 @@ export class LocalTerminalHandle implements SubprocessTerminalHandle { } terminate(): void { - this.termination ??= this.closeOnce().catch((error: unknown) => { - this.termination = undefined - throw error - }) - void this.termination.catch(() => {}) + this.lifecycle.terminate() } async waitForExit(signal?: AbortSignal): Promise { - // A caller may begin waiting before the top-level process exits. The exit - // callback starts descendant cleanup in the same turn, so resolve that - // eventual transaction after `done` instead of snapshotting only `done`. - const quiescence = this.termination ?? this.done.then(() => this.termination) - if (signal === undefined) { - await quiescence - return true - } - if (signal.aborted) return false - return await new Promise((resolve, reject) => { - const onAbort = (): void => { cleanup(); resolve(false) } - const cleanup = (): void => { signal.removeEventListener('abort', onAbort) } - signal.addEventListener('abort', onAbort, { once: true }) - void quiescence.then( - () => { cleanup(); resolve(true) }, - (error: unknown) => { - cleanup() - // The owned cleanup transaction only throws Error diagnostics. - // eslint-disable-next-line @typescript-eslint/prefer-promise-reject-errors - reject(error) - }, - ) - }) + return await this.lifecycle.waitForExit(signal) } private survivors(members: ProcessIdentity[]): ProcessIdentity[] { @@ -225,8 +198,6 @@ export class LocalTerminalHandle implements SubprocessTerminalHandle { throw new Error(`terminal cleanup failed; surviving pids: ${survivors.map(member => member.pid).join(', ')}`) } await this.stopShell() - this.removeAbort?.() - this.removeAbort = undefined this.dataDisposable.dispose() this.exitDisposable.dispose() } diff --git a/packages/subprocess/subprocess/README.i18n.yaml b/packages/subprocess/subprocess/README.i18n.yaml index 84d7d4a666..8e6c6a88e4 100644 --- a/packages/subprocess/subprocess/README.i18n.yaml +++ b/packages/subprocess/subprocess/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/subprocess/subprocess/README.md -README.md: 84b4b0c11c74c96929fa97b58fb33156d44e6ef1 -README.zh.md: dbd80a1c975719884481501f5cc43798e464a4fb +README.md: d03824da33bb44b2525b1343a27557ed15823418 +README.zh.md: 34cc75c3cfc8148754343e92b0c204760b1fb543 diff --git a/packages/subprocess/subprocess/README.md b/packages/subprocess/subprocess/README.md index 84b4b0c11c..d03824da33 100644 --- a/packages/subprocess/subprocess/README.md +++ b/packages/subprocess/subprocess/README.md @@ -12,6 +12,7 @@ The subprocess seam (`ctx.subprocess`) is the process half of one execution worl - Stdio is Node-shaped per stream: `'pipe'` hands the caller the raw stream for its own protocol framing (LSP JSON-RPC, ACP ndjson), `'inherit'` passes the parent descriptor through for diagnostics, and collect mode (`{ maxBytes, spill? }`) buffers a bounded tail with an optional full-stream spill file. Collect readers take whole-stream byte offsets and never consume, so independent readers cannot steal one another's deltas; a read whose offset slid out of the in-memory tail is `lossy` and points at the spill file when one exists. Collected output stays readable after settlement. - Termination is tree-scoped on every platform (POSIX detached groups with direct-child fallback; Windows `taskkill /T`): `terminate()` — the only termination verb — escalates SIGTERM→grace→SIGKILL (idempotent, driven by the spec's abort signal too, a no-op once the tree is gone), and `waitForExit(signal?)` observes whole-tree liveness so a consumer-owned teardown ladder holds each tier on real quiescence — the manager reacts but never classifies why (callers own deadlines, teardown ladders, and cause classification). - `spawnTerminal(spec)` is the only non-pipe primitive. Its handle owns a real PTY, valid-UTF-8 byte I/O, foreground-process-group inspection/signalling, TERM-to-KILL whole-session cleanup, and a quiescence wait. The output stream ends after queued output when the top-level process exits; a live transport failure rejects `done`. These operations remain one substrate primitive because ordinary pipes cannot allocate a controlling terminal or prove and clean the complete terminal session; readiness, scrollback, and owner policy remain in the PTY consumer. +- `SubprocessTerminalLifecycle` composes a handle's top-level `done` promise with its provider-owned session cleanup. It binds lifetime cancellation, shares one active cleanup attempt, permits a failed attempt to retry, normalizes cleanup rejections, and bounds quiescence observation without knowing the provider's process mechanics. - `scrubbedParentEnv()` / `SENSITIVE_ENV_PATTERN` are the one shared scrub definition: ambient credential-shaped and `DSH_*` names are dropped, and explicit `env` merges after the scrub. The local ordinary and terminal spawns both apply it; SDK-managed transports that own their spawn may import it directly. - Disposal of the service terminates all still-running managed processes and awaits their exit. diff --git a/packages/subprocess/subprocess/README.zh.md b/packages/subprocess/subprocess/README.zh.md index dbd80a1c97..34cc75c3cf 100644 --- a/packages/subprocess/subprocess/README.zh.md +++ b/packages/subprocess/subprocess/README.zh.md @@ -12,6 +12,7 @@ - stdio 按流采用 Node 形状:`'pipe'` 把原始流交给调用方做自己的协议分帧(LSP 的 JSON-RPC、ACP(Agent Client Protocol)的 ndjson),`'inherit'` 直通父进程描述符以承载诊断输出,收集模式(collect)`{ maxBytes, spill? }` 则缓冲一段有界尾部,外加可选的完整流 spill 文件。收集模式的读取器接受全流字节偏移量且从不消费,因此独立的读取器不会抢走彼此的增量;偏移量滑出内存尾部窗口的读取标记为 `lossy`,并在 spill 文件存在时指向它。收集到的输出在结算后仍可读取。 - 终止在每个平台上都以进程树为范围(POSIX 用 detached 进程组并以直接子进程回退;Windows 用 `taskkill /T`):`terminate()`(唯一的终止动词)执行 SIGTERM→宽限期→SIGKILL 升级(幂等,也由 spec 的 abort 信号驱动,进程树消亡后为空操作);`waitForExit(signal?)` 观察整棵进程树的存活状态,使消费方自有的拆卸阶梯能在真正完全停稳后才进入下一层。管理器只响应中止,但绝不判定原因(deadline、拆卸阶梯与原因分类归调用方所有)。 - `spawnTerminal(spec)` 是唯一的非管道原语。其句柄负责真实 PTY、有效 UTF-8 字节 I/O、前台进程组检查/信号发送、TERM→KILL 全会话清理,以及等待完全停稳。顶层进程退出后,输出流会在排完队列中的输出后结束;存活期间的传输故障会拒绝 `done`。这些操作仍属于一项基底原语,因为普通管道无法分配控制终端,也无法证明并清理完整的终端会话;就绪检测、scrollback 与所有者策略仍归 PTY 消费方所有。 +- `SubprocessTerminalLifecycle` 把句柄的顶层 `done` promise 与由提供方负责的会话清理组合起来。它绑定生命周期取消,共享同一个进行中的清理尝试,允许失败的尝试重试,规范化清理拒绝,并在不了解提供方进程机制的情况下对完全停稳观测施加上限。 - `scrubbedParentEnv()` / `SENSITIVE_ENV_PATTERN` 是唯一一份共享的凭据清除定义:环境中形似凭据的名称与 `DSH_*` 名称都会被丢弃,显式 `env` 在清除之后合并。本地普通 spawn 与终端 spawn 都应用这一定义;自行拥有 spawn 的 SDK 管理传输层可以直接导入它。 - 服务自身的 dispose(资源释放)会终止所有仍在运行的受管进程并等待其退出。 diff --git a/packages/subprocess/subprocess/src/index.ts b/packages/subprocess/subprocess/src/index.ts index 3c58a07a9e..df79ee5f3e 100644 --- a/packages/subprocess/subprocess/src/index.ts +++ b/packages/subprocess/subprocess/src/index.ts @@ -14,6 +14,8 @@ import type { SubprocessHandle, SubprocessSpawnSpec } from './types.ts' import type { SubprocessTerminalHandle, SubprocessTerminalSpawnSpec } from './types.ts' export { DSH_ENV_PREFIX } from './types.ts' +export { SubprocessTerminalLifecycle } from './terminal-lifecycle.ts' +export type { SubprocessTerminalLifecycleOptions } from './terminal-lifecycle.ts' export type { CollectedOutput, DshEnvironment, diff --git a/packages/subprocess/subprocess/src/terminal-lifecycle.ts b/packages/subprocess/subprocess/src/terminal-lifecycle.ts new file mode 100644 index 0000000000..76430fe7c6 --- /dev/null +++ b/packages/subprocess/subprocess/src/terminal-lifecycle.ts @@ -0,0 +1,104 @@ +/** Provider-neutral lifecycle transaction for terminal-process handles. */ + +/** Inputs owned by one terminal-process lifecycle controller. */ +export interface SubprocessTerminalLifecycleOptions { + /** Settlement of the top-level terminal process or its live transport. */ + readonly done: Promise + /** Provider-owned cleanup that reaches whole-session quiescence. */ + readonly cleanup: () => Promise + /** Optional cancellation for the complete terminal lifetime. */ + readonly signal?: AbortSignal | undefined +} + +function normalizeCleanupError(error: unknown): Error { + return error instanceof Error ? error : new Error(String(error)) +} + +/** + * Coordinates terminal cleanup without knowing how a provider allocates or + * terminates its process session. One active cleanup attempt is shared by all + * callers; a rejected attempt may be retried, and successful cleanup removes + * the lifetime abort listener. + */ +export class SubprocessTerminalLifecycle { + private cleanupAttempt: Promise | undefined + private removeLifetimeAbort: (() => void) | undefined + + /** + * @param options - top-level settlement, provider cleanup, and lifetime cancellation. + */ + constructor(private readonly options: SubprocessTerminalLifecycleOptions) { + const onDone = (): void => { this.terminate() } + void options.done.then(onDone, onDone) + + if (options.signal !== undefined) { + const onAbort = (): void => { this.terminate() } + options.signal.addEventListener('abort', onAbort, { once: true }) + this.removeLifetimeAbort = () => { options.signal?.removeEventListener('abort', onAbort) } + if (options.signal.aborted) this.terminate() + } + } + + /** Begin an idempotent provider cleanup attempt. */ + terminate(): void { + void this.startCleanup().catch(() => {}) + } + + /** + * Wait for top-level settlement and successful whole-session cleanup. + * @param signal - optional bound for this observation only. + * @returns true after quiescence, false when the observer signal aborts first. + */ + async waitForExit(signal?: AbortSignal): Promise { + const quiescence = this.cleanupAttempt ?? this.options.done.then( + () => this.startCleanup(), + () => this.startCleanup(), + ) + if (signal === undefined) { + await quiescence + return true + } + if (signal.aborted) return false + + return await new Promise((resolve, reject) => { + let settled = false + const finish = (complete: () => void): void => { + if (settled) return + settled = true + signal.removeEventListener('abort', onAbort) + complete() + } + const onAbort = (): void => { finish(() => { resolve(false) }) } + + signal.addEventListener('abort', onAbort, { once: true }) + if (signal.aborted) onAbort() + void quiescence.then( + () => { finish(() => { resolve(true) }) }, + (error: unknown) => { finish(() => { reject(normalizeCleanupError(error)) }) }, + ) + }) + } + + private startCleanup(): Promise { + if (this.cleanupAttempt !== undefined) return this.cleanupAttempt + + const outcome = Promise.withResolvers() + this.cleanupAttempt = outcome.promise.catch((error: unknown) => { + this.cleanupAttempt = undefined + throw normalizeCleanupError(error) + }) + void this.cleanupAttempt.then( + () => { + this.removeLifetimeAbort?.() + this.removeLifetimeAbort = undefined + }, + () => {}, + ) + try { + void this.options.cleanup().then(outcome.resolve, outcome.reject) + } catch (error: unknown) { + outcome.reject(error) + } + return this.cleanupAttempt + } +} diff --git a/packages/subprocess/subprocess/tests/terminal-lifecycle.spec.ts b/packages/subprocess/subprocess/tests/terminal-lifecycle.spec.ts new file mode 100644 index 0000000000..2dd688bbc0 --- /dev/null +++ b/packages/subprocess/subprocess/tests/terminal-lifecycle.spec.ts @@ -0,0 +1,125 @@ +import { describe, expect, it, vi } from 'vitest' +import { SubprocessTerminalLifecycle } from '@deepseek-ai/dsh-subprocess' + +describe('SubprocessTerminalLifecycle', () => { + it('waits for top-level settlement and the provider cleanup transaction', async () => { + const done = Promise.withResolvers() + const cleanupGate = Promise.withResolvers() + const cleanup = vi.fn(() => cleanupGate.promise) + const lifecycle = new SubprocessTerminalLifecycle({ done: done.promise, cleanup }) + + const waiting = lifecycle.waitForExit() + expect(cleanup).not.toHaveBeenCalled() + done.resolve(undefined) + await vi.waitFor(() => { expect(cleanup).toHaveBeenCalledOnce() }) + + const observed = vi.fn() + void waiting.then(observed) + await Promise.resolve() + expect(observed).not.toHaveBeenCalled() + + cleanupGate.resolve(undefined) + await expect(waiting).resolves.toBe(true) + lifecycle.terminate() + await expect(lifecycle.waitForExit()).resolves.toBe(true) + expect(cleanup).toHaveBeenCalledOnce() + }) + + it('normalizes cleanup failures, permits retries, and retains lifetime cancellation until success', async () => { + const done = Promise.withResolvers() + const lifetime = new AbortController() + const removeListener = vi.spyOn(lifetime.signal, 'removeEventListener') + const synchronousFailure = new Error('first cleanup failed') + let attempt = 0 + const cleanup = vi.fn((): Promise => { + attempt += 1 + if (attempt === 1) throw synchronousFailure + if (attempt === 2) { + return Promise.resolve().then(() => { + const nonErrorRejection: unknown = 'cleanup transport gone' + throw nonErrorRejection + }) + } + return Promise.resolve() + }) + const lifecycle = new SubprocessTerminalLifecycle({ + done: done.promise, + cleanup, + signal: lifetime.signal, + }) + + lifecycle.terminate() + await expect(lifecycle.waitForExit()).rejects.toBe(synchronousFailure) + lifecycle.terminate() + await expect(lifecycle.waitForExit()).rejects.toThrow('cleanup transport gone') + + lifetime.abort() + await expect(lifecycle.waitForExit()).resolves.toBe(true) + expect(cleanup).toHaveBeenCalledTimes(3) + expect(removeListener).toHaveBeenCalledWith('abort', expect.any(Function)) + + done.reject(new Error('top-level transport failed')) + await Promise.resolve() + expect(cleanup).toHaveBeenCalledTimes(3) + }) + + it('starts cleanup for a pre-aborted lifetime and bounds a wait that is already aborted', async () => { + const cleanupGate = Promise.withResolvers() + const cleanup = vi.fn(() => cleanupGate.promise) + const lifecycle = new SubprocessTerminalLifecycle({ + done: new Promise(() => {}), + cleanup, + signal: AbortSignal.abort(new Error('lifetime cancelled')), + }) + + expect(cleanup).toHaveBeenCalledOnce() + await expect(lifecycle.waitForExit(AbortSignal.abort())).resolves.toBe(false) + cleanupGate.resolve(undefined) + await expect(lifecycle.waitForExit()).resolves.toBe(true) + }) + + it('contains cleanup settlement after an observer aborts between signal checks', async () => { + const firstCleanup = Promise.withResolvers() + const cleanup = vi.fn() + .mockImplementationOnce(() => firstCleanup.promise) + .mockResolvedValueOnce(undefined) + const lifecycle = new SubprocessTerminalLifecycle({ done: Promise.resolve(), cleanup }) + const observer = new AbortController().signal + vi.spyOn(observer, 'aborted', 'get') + .mockReturnValueOnce(false) + .mockReturnValueOnce(true) + + await expect(lifecycle.waitForExit(observer)).resolves.toBe(false) + firstCleanup.reject(new Error('late cleanup failure')) + await vi.waitFor(() => { expect(cleanup).toHaveBeenCalledOnce() }) + await Promise.resolve() + + lifecycle.terminate() + await expect(lifecycle.waitForExit()).resolves.toBe(true) + expect(cleanup).toHaveBeenCalledTimes(2) + }) + + it('reports bounded cleanup success and failure', async () => { + const successful = new SubprocessTerminalLifecycle({ + done: Promise.resolve(), + cleanup: async () => {}, + }) + await expect(successful.waitForExit(new AbortController().signal)).resolves.toBe(true) + + const failure = new Error('quiescence failed') + const failed = new SubprocessTerminalLifecycle({ + done: Promise.resolve(), + cleanup: () => Promise.reject(failure), + }) + await expect(failed.waitForExit(new AbortController().signal)).rejects.toBe(failure) + + const failedDone = Promise.withResolvers() + const afterTransportFailure = new SubprocessTerminalLifecycle({ + done: failedDone.promise, + cleanup: async () => {}, + }) + const waiting = afterTransportFailure.waitForExit() + failedDone.reject(new Error('transport failed')) + await expect(waiting).resolves.toBe(true) + }) +}) From a0200376a1aaa35e00c1e2583b47f5a39c5ba157 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 03:44:57 +0800 Subject: [PATCH 011/108] fix(pty): defer the first readiness poll --- packages/pty/pty-local/src/session.ts | 2 +- packages/pty/pty-local/tests/session.spec.ts | 25 ++++++++++++++++++++ 2 files changed, 26 insertions(+), 1 deletion(-) diff --git a/packages/pty/pty-local/src/session.ts b/packages/pty/pty-local/src/session.ts index 9a0beed6ff..03809aec9f 100644 --- a/packages/pty/pty-local/src/session.ts +++ b/packages/pty/pty-local/src/session.ts @@ -266,7 +266,7 @@ export class LocalPtySession implements PtyBackendSession { // eslint-disable-next-line @typescript-eslint/no-unnecessary-condition if (this.active === operation && !this.closing) { this.pollingReady = operation - this.schedulePoll(operation, 0) + this.schedulePoll(operation) } } catch (error: unknown) { if (this.active === operation) { diff --git a/packages/pty/pty-local/tests/session.spec.ts b/packages/pty/pty-local/tests/session.spec.ts index 120b2881e1..5f0065d0ee 100644 --- a/packages/pty/pty-local/tests/session.spec.ts +++ b/packages/pty/pty-local/tests/session.spec.ts @@ -143,6 +143,31 @@ async function initialize(session: LocalPtySession, terminal: FakeTerminal): Pro } describe('LocalPtySession readiness and output', () => { + it('lets queued terminal output run before the first post-write readiness poll', async () => { + vi.useFakeTimers() + const terminal = new FakeTerminal() + const inspector = new FakeInspector() + const session = makeSession(terminal, inspector, config()) + await initialize(session, terminal) + + const inspect = terminal.inspectForeground.bind(terminal) + let inspections = 0 + terminal.inspectForeground = async () => { + inspections += 1 + return await inspect() + } + const operation = session.startSend({ text: 'true', submit: true }) + await Promise.resolve() + await Promise.resolve() + expect(inspections).toBe(1) + + await vi.advanceTimersByTimeAsync(0) + expect(inspections).toBe(1) + terminal.emitData('\x1b]133;D;0\x07dsh> ') + await vi.advanceTimersByTimeAsync(10) + expect((await operation.done).waitReason).toBe('stdin_read') + }) + it('captures prompt MOTD, writes submit explicitly, and settles exact stdin waits', async () => { vi.useFakeTimers() const terminal = new FakeTerminal() From 03f1c8eb437885469e3478a6982c525b4de6cb50 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 03:45:06 +0800 Subject: [PATCH 012/108] fix(subprocess): absolutize relative PATH lookups --- packages/subprocess/subprocess-local/src/index.ts | 4 ++-- packages/subprocess/subprocess-local/tests/local.spec.ts | 5 ++++- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/packages/subprocess/subprocess-local/src/index.ts b/packages/subprocess/subprocess-local/src/index.ts index 40d2f52552..03755cd9ea 100644 --- a/packages/subprocess/subprocess-local/src/index.ts +++ b/packages/subprocess/subprocess-local/src/index.ts @@ -11,7 +11,7 @@ import { constants } from 'node:fs' import { mkdtempSync } from 'node:fs' import { access, rm, stat } from 'node:fs/promises' import { tmpdir } from 'node:os' -import { delimiter, extname, isAbsolute, join } from 'node:path' +import { delimiter, extname, isAbsolute, join, resolve } from 'node:path' import { Context } from 'cordis' import * as nodePty from 'node-pty' import type { IPtyForkOptions } from 'node-pty' @@ -107,7 +107,7 @@ export class LocalSubprocessService extends SubprocessService { ? (env.PATHEXT ?? '.COM;.EXE;.BAT;.CMD').split(';') : [''] return path.split(delimiter).flatMap(directory => - directory === '' ? [] : extensions.map(extension => join(directory, command + extension))) + directory === '' ? [] : extensions.map(extension => resolve(this.cwd, directory, command + extension))) } spawn(spec: SubprocessSpawnSpec): SubprocessHandle { diff --git a/packages/subprocess/subprocess-local/tests/local.spec.ts b/packages/subprocess/subprocess-local/tests/local.spec.ts index 93be25acd0..6ce3d06b8e 100644 --- a/packages/subprocess/subprocess-local/tests/local.spec.ts +++ b/packages/subprocess/subprocess-local/tests/local.spec.ts @@ -1,7 +1,7 @@ import { PassThrough } from 'node:stream' import { describe, expect, it, vi } from 'vitest' import { stat } from 'node:fs/promises' -import { basename, delimiter, dirname } from 'node:path' +import { basename, delimiter, dirname, relative } from 'node:path' import { Context } from 'cordis' import LocalSubprocessService from '@deepseek-ai/dsh-subprocess-local' import type { SubprocessSpawnSpec, SubprocessTerminalHandle, SubprocessTerminalSpawnSpec } from '@deepseek-ai/dsh-subprocess' @@ -38,6 +38,9 @@ describe('LocalSubprocessService', () => { expect(await ctx.subprocess.resolveExecutable(basename(process.execPath), { PATH: dirname(process.execPath), })).toBe(process.execPath) + expect(await ctx.subprocess.resolveExecutable(basename(process.execPath), { + PATH: relative(process.cwd(), dirname(process.execPath)) || '.', + })).toBe(process.execPath) await expect(ctx.subprocess.resolveExecutable('')).rejects.toThrow('must be non-empty') await expect(ctx.subprocess.resolveExecutable('dsh-command-that-does-not-exist', { PATH: '' })) .rejects.toThrow('was not found on PATH') From 35376658061cc35aaf9316eca3100af6d0445357 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 04:19:04 +0800 Subject: [PATCH 013/108] fix(subprocess): honor Windows environment key casing --- packages/subprocess/subprocess-local/src/index.ts | 12 ++++++++++-- .../subprocess/subprocess-local/tests/local.spec.ts | 4 +++- 2 files changed, 13 insertions(+), 3 deletions(-) diff --git a/packages/subprocess/subprocess-local/src/index.ts b/packages/subprocess/subprocess-local/src/index.ts index 03755cd9ea..4c5cad8018 100644 --- a/packages/subprocess/subprocess-local/src/index.ts +++ b/packages/subprocess/subprocess-local/src/index.ts @@ -102,9 +102,9 @@ export class LocalSubprocessService extends SubprocessService { } private executableCandidates(command: string, env: NodeJS.ProcessEnv): string[] { - const path = env.PATH ?? '' + const path = environmentValue(env, 'PATH') ?? '' const extensions = process.platform === 'win32' && extname(command) === '' - ? (env.PATHEXT ?? '.COM;.EXE;.BAT;.CMD').split(';') + ? (environmentValue(env, 'PATHEXT') ?? '.COM;.EXE;.BAT;.CMD').split(';') : [''] return path.split(delimiter).flatMap(directory => directory === '' ? [] : extensions.map(extension => resolve(this.cwd, directory, command + extension))) @@ -156,4 +156,12 @@ export class LocalSubprocessService extends SubprocessService { } } +/** Read a Windows environment key using the platform's case-insensitive semantics. */ +function environmentValue(env: NodeJS.ProcessEnv, name: 'PATH' | 'PATHEXT'): string | undefined { + const exact = env[name] + if (exact !== undefined || process.platform !== 'win32') return exact + const normalized = name.toUpperCase() + return Object.entries(env).find(([key]) => key.toUpperCase() === normalized)?.[1] +} + export default LocalSubprocessService diff --git a/packages/subprocess/subprocess-local/tests/local.spec.ts b/packages/subprocess/subprocess-local/tests/local.spec.ts index 6ce3d06b8e..4662283aad 100644 --- a/packages/subprocess/subprocess-local/tests/local.spec.ts +++ b/packages/subprocess/subprocess-local/tests/local.spec.ts @@ -62,8 +62,10 @@ describe('LocalSubprocessService', () => { }).executableCandidates.bind(service) const platform = vi.spyOn(process, 'platform', 'get').mockReturnValue('win32') try { - expect(candidates('tool', { PATH: `${delimiter}/bin`, PATHEXT: '.EXE;.CMD' })) + expect(candidates('tool', { Path: `${delimiter}/bin`, PathExt: '.EXE;.CMD' })) .toEqual(['/bin/tool.EXE', '/bin/tool.CMD']) + expect(candidates('tool', { Path: '/ambient', PATH: '/explicit', PATHEXT: '.EXE' })) + .toEqual(['/explicit/tool.EXE']) expect(candidates('tool.exe', {})).toEqual([]) expect(candidates('tool', { PATH: '/bin' })).toHaveLength(4) } finally { From 4997fb219e38e26ef7e76d4c0531f029849bfc96 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 04:19:18 +0800 Subject: [PATCH 014/108] fix(lsp): preserve execution-world URI semantics --- .../2026-07-15-lsp-capability-seam.i18n.yaml | 4 +- .../2026-07-15-lsp-capability-seam.md | 6 +- .../2026-07-15-lsp-capability-seam.zh.md | 6 +- ...rtable-execution-world-consumers.i18n.yaml | 4 +- ...7-28-portable-execution-world-consumers.md | 2 +- ...8-portable-execution-world-consumers.zh.md | 2 +- docs/core-data-structures/lsp.i18n.yaml | 4 +- docs/core-data-structures/lsp.md | 12 ++-- docs/core-data-structures/lsp.zh.md | 12 ++-- packages/lsp/lsp-local/src/instance.ts | 7 +- packages/lsp/lsp-local/tests/instance.spec.ts | 8 +-- .../lsp/lsp-local/tests/lifecycle.spec.ts | 8 +-- packages/lsp/lsp/README.i18n.yaml | 4 +- packages/lsp/lsp/README.md | 2 +- packages/lsp/lsp/README.zh.md | 14 ++-- packages/lsp/lsp/src/types.ts | 10 +-- packages/lsp/lsp/tests/lsp.spec.ts | 8 +-- packages/lsp/tool-lsp/README.i18n.yaml | 4 +- packages/lsp/tool-lsp/README.md | 2 +- packages/lsp/tool-lsp/README.zh.md | 16 ++--- packages/lsp/tool-lsp/src/index.ts | 6 +- packages/lsp/tool-lsp/src/render.ts | 72 ++++++++++++++----- packages/lsp/tool-lsp/tests/render.spec.ts | 45 ++++++++---- packages/lsp/tool-lsp/tests/tool-lsp.spec.ts | 11 +-- 24 files changed, 162 insertions(+), 107 deletions(-) diff --git a/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.i18n.yaml index afb9251d33..bba045d606 100644 --- a/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md -2026-07-15-lsp-capability-seam.md: a9dab45ebf034f6d5218b25a0868a7e86803719b -2026-07-15-lsp-capability-seam.zh.md: b411a2456ea2a3838e56422f58954473da51ae23 +2026-07-15-lsp-capability-seam.md: 6e46fac0611c1b7282766cf57965a07e3ee6bb9b +2026-07-15-lsp-capability-seam.zh.md: b535b60df70e8a6132bb1596581be3ab1daba932 diff --git a/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md b/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md index a9dab45ebf..6e46fac061 100644 --- a/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md +++ b/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md @@ -62,7 +62,7 @@ interface LspProviderQuery extends LspQueryRequest { } type LspQueryResult = - | { readonly kind: 'locations'; readonly locations: readonly { readonly uri: string; readonly range: LspRange }[]; readonly resolvedWorkspaceRoot: string } + | { readonly kind: 'locations'; readonly locations: readonly { readonly uri: string; readonly range: LspRange }[]; readonly resolvedWorkspaceUri: string } | { readonly kind: 'hover'; readonly hover: { readonly contents: string; readonly range?: LspRange } | null } interface LspProvider { @@ -77,7 +77,7 @@ interface LspService { } ``` -Mapping keys normalize to lowercase, leading-dot extensions selected from `filePath`'s final extension; language ids only synchronize documents. Seam positions and ranges are zero-based UTF-16. `findReferences` always includes declarations: providers enforce this internally, the local mapping sets `context.includeDeclaration: true`, and callers get no flag. Closed result unions normalize navigation to locations and hover to content or `null`; navigation results carry the provider's resolved workspace root so consumers relativize file URIs in the same canonical namespace. The seam exposes no protocol types, process or document controls, or generic request escape hatch. +Mapping keys normalize to lowercase, leading-dot extensions selected from `filePath`'s final extension; language ids only synchronize documents. Seam positions and ranges are zero-based UTF-16. `findReferences` always includes declarations: providers enforce this internally, the local mapping sets `context.includeDeclaration: true`, and callers get no flag. Closed result unions normalize navigation to locations and hover to content or `null`; navigation results carry the provider's canonical workspace URI so consumers relativize file URIs in the execution world's namespace. The seam exposes no protocol types, process or document controls, or generic request escape hatch. `dsh-lsp-local` owns server configuration, JSON-RPC, process and transient-document state, and protocol translation. It reads through `ctx.fs` and launches through `ctx.subprocess`, depending on their interface packages rather than concrete providers; the [portable execution-world decision](2026-07-28-portable-execution-world-consumers.md) owns that pairing. The server-table key is its provider id. The plugin resolves every server-local setting before registration, rolls back earlier registrations if a later mapping is invalid or conflicts, and retains an independent process pool per provider. `dsh-tool-lsp` runtime-injects only `tools`, `lsp`, and `systemPrompt`, obtains the workspace from `exec.agent?.session.header.cwd` through a package-local `sessionCwd(exec)` helper matching the filesystem tools' lookup, and imports no provider. @@ -98,7 +98,7 @@ interface LspToolInput { The tool requires `workspaceRoot` from session `header.cwd`, with no fallback; absence fails as `LSP_WORKSPACE_REQUIRED` before querying or startup. The local provider resolves relative paths against that root and accepts absolute paths directly; both forms are canonicalized and rejected before startup when the target is outside the canonical workspace. -Locations render as stable, file-grouped `path:line:character` entries. A `file:` URI accepted by Node `fileURLToPath()` becomes a relative path inside the workspace or an absolute path outside it; other URIs remain verbatim. `maxLocations` defaults to `100` and reports omitted items; `maxResultChars` defaults to `16_000` and bounds every complete rendered result, including its truncation metadata. Empty locations and `null` hover are successful no-result responses; missing or malformed server payloads fail with structured `LSP_MALFORMED_RESPONSE` errors. +Locations render as stable, file-grouped `path:line:character` entries without applying harness-host path rules. A valid `file:` URI becomes a relative path inside the provider's canonical workspace URI or a URI-derived absolute path outside it; malformed and non-`file:` URIs remain verbatim. `maxLocations` defaults to `100` and reports omitted items; `maxResultChars` defaults to `16_000` and bounds every complete rendered result, including its truncation metadata. Empty locations and `null` hover are successful no-result responses; missing or malformed server payloads fail with structured `LSP_MALFORMED_RESPONSE` errors. The transport-neutral presenter uses `{ card: 'generic', kind: 'search', title, locations: [{ path: file_path, line }] }` with an args-derived operation/cursor `title`. Because `FileLocation` has no character, follow-along focuses the input line while the title preserves the cursor; presentation remains pure. diff --git a/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.zh.md b/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.zh.md index b411a2456e..b535b60df7 100644 --- a/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.zh.md @@ -62,7 +62,7 @@ interface LspProviderQuery extends LspQueryRequest { } type LspQueryResult = - | { readonly kind: 'locations'; readonly locations: readonly { readonly uri: string; readonly range: LspRange }[]; readonly resolvedWorkspaceRoot: string } + | { readonly kind: 'locations'; readonly locations: readonly { readonly uri: string; readonly range: LspRange }[]; readonly resolvedWorkspaceUri: string } | { readonly kind: 'hover'; readonly hover: { readonly contents: string; readonly range?: LspRange } | null } interface LspProvider { @@ -77,7 +77,7 @@ interface LspService { } ``` -映射键规范化为带前导点的小写扩展名,并按 `filePath` 的最后一个扩展名选择;语言 id 仅用于文档同步。服务边界中的位置和范围从零开始按 UTF-16 计数。`findReferences` 始终包含声明:提供方在内部执行该约束,本地映射设置 `context.includeDeclaration: true`,调用方不能配置。封闭结果联合将导航统一为位置,将 `hover` 统一为内容或 `null`;导航结果携带提供方解析后的工作区根目录,使消费方依据同一规范化根目录相对化文件 URI。服务边界不公开协议类型、进程或文档控制,也不提供通用请求逃生口。 +映射键规范化为带前导点的小写扩展名,并按 `filePath` 的最后一个扩展名选择;语言 id 仅用于文档同步。服务边界中的位置和范围从零开始按 UTF-16 计数。`findReferences` 始终包含声明:提供方在内部执行该约束,本地映射设置 `context.includeDeclaration: true`,调用方不能配置。封闭结果联合将导航统一为位置,将 `hover` 统一为内容或 `null`;导航结果携带提供方的规范工作区 URI,使消费方在执行世界的命名空间内相对化文件 URI。服务边界不公开协议类型、进程或文档控制,也不提供通用请求逃生口。 `dsh-lsp-local` 负责服务器配置、JSON-RPC、进程与临时文档状态和协议转换。它通过 `ctx.fs` 读取,通过 `ctx.subprocess` 启动,只依赖二者的接口包而非具体提供方;[可移植执行环境决策](2026-07-28-portable-execution-world-consumers.md)负责定义这种配对。服务器表的键是提供方 id。插件在注册前解析每个服务器的本地设置;如果后续映射无效或发生冲突,插件会撤销此前的注册,并为每个提供方保留独立进程池。`dsh-tool-lsp` 在运行时只注入 `tools`、`lsp` 和 `systemPrompt`,通过包内的 `sessionCwd(exec)` 辅助函数从 `exec.agent?.session.header.cwd` 取得工作区,其取值方式与文件系统工具一致,也不导入提供方。 @@ -98,7 +98,7 @@ interface LspToolInput { 工具必须从会话 `header.cwd` 取得 `workspaceRoot`,没有后备值;缺失时在查询或启动前以 `LSP_WORKSPACE_REQUIRED` 失败。本地提供方基于根目录解析相对路径并直接接受绝对路径;两种路径都会进行规范化,如果目标位于规范工作区外,则在启动前拒绝。 -位置按文件稳定分组并渲染为 `path:line:character`。Node `fileURLToPath()` 可接受的 `file:` URI 在工作区内转换为相对路径,在工作区外转换为绝对路径;其他 URI 保持原样。`maxLocations` 默认值为 `100`,并报告省略的条目;`maxResultChars` 默认值为 `16_000`,并限制每个完整渲染结果,其中包括截断元数据。空位置与 `null` hover 是成功的无结果响应;服务器载荷缺失或格式错误时,以结构化 `LSP_MALFORMED_RESPONSE` 错误失败。 +位置在不应用 harness 宿主路径规则的情况下按文件稳定分组并渲染为 `path:line:character`。有效的 `file:` URI 落在提供方的规范工作区 URI 内时转换为相对路径,位于其外时转换为从 URI 派生的绝对路径;格式错误的 URI 与非 `file:` URI 保持原样。`maxLocations` 默认值为 `100`,并报告省略的条目;`maxResultChars` 默认值为 `16_000`,并限制每个完整渲染结果,其中包括截断元数据。空位置与 `null` hover 是成功的无结果响应;服务器载荷缺失或格式错误时,以结构化 `LSP_MALFORMED_RESPONSE` 错误失败。 与传输方式无关的展示器使用 `{ card: 'generic', kind: 'search', title, locations: [{ path: file_path, line }] }`,`title` 由参数推导并标明操作与光标。由于 `FileLocation` 没有 character,跟随位置聚焦输入行,标题保留完整光标;展示保持纯函数。 diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml index 231037f054..1033fffcc6 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md -2026-07-28-portable-execution-world-consumers.md: a5df393f31a25b7d16e63af240882e1d94093e34 -2026-07-28-portable-execution-world-consumers.zh.md: 87f11fa53b5b7579969dcb15dc3aaad978018773 +2026-07-28-portable-execution-world-consumers.md: 6f75778398e8f204c85b64fe096838a3b62d398e +2026-07-28-portable-execution-world-consumers.zh.md: 22edc591f7716a92fd2a68d74c84168189743fbd diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md index a5df393f31..6f75778398 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md @@ -21,7 +21,7 @@ The subprocess interface owns the process coordinates and primitives: canonical Generic consumers use that execution world: - `dsh-bash-local` continues to map Bash semantics onto ordinary `ctx.subprocess.spawn()`. -- `dsh-lsp-local` reads and contains source through `ctx.fs`, resolves and launches language servers through `ctx.subprocess`, and sends provider-owned file URIs. Its JSON-RPC, pooling, synchronization, cancellation, and normalization stay unchanged. +- `dsh-lsp-local` reads and contains source through `ctx.fs`, resolves and launches language servers through `ctx.subprocess`, and carries provider-owned file URIs through initialization and result rendering. Its JSON-RPC, pooling, synchronization, cancellation, and normalization stay unchanged. - `dsh-pty-local` maps persistent-shell semantics onto `ctx.subprocess.spawnTerminal()`. The local `node-pty` and process-inspection implementation moves into `dsh-subprocess-local`; another subprocess provider supplies the same primitive. The allocation signal is detached before publication, while readiness initialization retains setup cancellation. A timed-out asynchronous write, or a cancellation whose foreground signal fails during that write, retains the send reservation until the provider settles it; completion of a stale inspection resumes polling for the current send. - `dsh-code-runtime-subprocess` materializes a dependency-free runner through `ctx.fs` and launches it through `ctx.subprocess`, preserving the Code Runtime binding and output contract across local or remote worlds. It shares host-side worker mechanics through the non-plugin `dsh-code-runtime-worker/runtime-host` subpath instead of copying them. Preparation carries one lifecycle signal through filesystem resolution, materialization, and executable lookup so disposal can abort a stalled provider. The heap-bounded worker rejects oversized binding frames before transfer, each outer hop enforces the same bound before serialization, and the launcher publishes an accepted terminal frame before reaping its controller so a descendant that inherits controller pipes cannot suppress completion; the host still awaits process-group quiescence. diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md index 87f11fa53b..22edc591f7 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md @@ -21,7 +21,7 @@ Status: implemented 通用消费方使用该执行世界: - `dsh-bash-local` 继续把 Bash 语义映射到普通的 `ctx.subprocess.spawn()`。 -- `dsh-lsp-local` 通过 `ctx.fs` 读取源文件并验证包含关系,通过 `ctx.subprocess` 解析和启动语言服务器,并发送由提供方负责的文件 URI。其 JSON-RPC、池化、同步、取消和规范化保持不变。 +- `dsh-lsp-local` 通过 `ctx.fs` 读取源文件并验证包含关系,通过 `ctx.subprocess` 解析和启动语言服务器,并让由提供方负责的文件 URI 贯穿初始化与结果渲染。其 JSON-RPC、池化、同步、取消和规范化保持不变。 - `dsh-pty-local` 把持久 shell 语义映射到 `ctx.subprocess.spawnTerminal()`。本地 `node-pty` 与进程检查实现移入 `dsh-subprocess-local`;其他进程管理提供方则提供相同原语。分配信号会在发布前解除关联,而就绪初始化仍保留设置阶段的取消。异步写入超时,或在该写入期间取消时前台信号发送失败,都会保留发送预留,直至提供方将写入结算;陈旧检查完成后,会针对当前发送恢复轮询。 - `dsh-code-runtime-subprocess` 通过 `ctx.fs` 物化无依赖 runner,并通过 `ctx.subprocess` 启动它,从而在本地或远程执行世界中保留代码运行时的绑定与输出契约。它通过非插件子路径 `dsh-code-runtime-worker/runtime-host` 共享宿主侧 worker 机制,而不是复制这些机制。准备阶段让同一个生命周期信号贯穿文件系统解析、物化和可执行文件查找,使资源释放能够中止停滞的提供方操作。受堆上限约束的 worker 会在传输前拒绝过大的绑定帧;每个外层转发环节都会在序列化前执行相同的上限检查;launcher 会在回收 controller 前发布已接纳的终态帧,使继承 controller 管道的后代进程无法阻止完成;宿主仍会等待进程组完全停稳。 diff --git a/docs/core-data-structures/lsp.i18n.yaml b/docs/core-data-structures/lsp.i18n.yaml index 5bed4a1a23..329d5086e0 100644 --- a/docs/core-data-structures/lsp.i18n.yaml +++ b/docs/core-data-structures/lsp.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write docs/core-data-structures/lsp.md -lsp.md: 62b133cbfdf521e067c56355664d7514a613397f -lsp.zh.md: 51a19a51a8ad92e744cb920a51f3214f68ae0036 +lsp.md: 03d0ce2dbe246aadb6f6c9a702fa50e3e792eb09 +lsp.zh.md: ef9623f01124bc459d3bbceba340c3870e678baa diff --git a/docs/core-data-structures/lsp.md b/docs/core-data-structures/lsp.md index 62b133cbfd..03d0ce2dbe 100644 --- a/docs/core-data-structures/lsp.md +++ b/docs/core-data-structures/lsp.md @@ -73,7 +73,7 @@ interface LspProviderQuery extends LspQueryRequest { ## Result -A CLOSED discriminated union: navigation operations normalize to `locations`, `hover` to content or `null`. Consumers `switch` on `kind` to exhaustiveness so a new arm breaks compilation until handled. `findReferences` always includes declarations — the provider enforces this internally, so callers get no flag. The `locations` variant carries `resolvedWorkspaceRoot`: the provider's canonical form of the request's `workspaceRoot` and the root its `file:` URIs are relative to, so a caller relativizing display paths uses it rather than the possibly-symlinked request root. +A CLOSED discriminated union: navigation operations normalize to `locations`, `hover` to content or `null`. Consumers `switch` on `kind` to exhaustiveness so a new arm breaks compilation until handled. `findReferences` always includes declarations — the provider enforces this internally, so callers get no flag. The `locations` variant carries `resolvedWorkspaceUri`, the provider's canonical workspace `file:` URI. A caller relativizing location URIs uses that coordinate rather than applying host-platform path rules to the possibly-symlinked request root. ```ts type-equiv /** One resolved location: a document URI and the range within it. */ @@ -101,13 +101,13 @@ interface LspHover { * `goToImplementation`) normalize to `locations`; `hover` normalizes to content or `null`. * Consumers `switch` on `kind` to exhaustiveness so a new arm breaks compilation until handled. * - * The `locations` variant carries `resolvedWorkspaceRoot`: the provider's canonical form of the - * request's `workspaceRoot`, and the root its `file:` location URIs are relative to. A caller that - * relativizes display paths MUST use this, not the request's (possibly symlinked) `workspaceRoot`; - * otherwise a symlinked workspace misclassifies in-workspace results as external. + * The `locations` variant carries `resolvedWorkspaceUri`: the provider's canonical `file:` URI for + * the request's workspace root. A caller that relativizes location URIs MUST use this, not parse the + * request's possibly symlinked process path with host-platform rules; the execution platform may + * differ from the caller's. */ type LspQueryResult = - | { readonly kind: 'locations'; readonly locations: readonly LspLocation[]; readonly resolvedWorkspaceRoot: string } + | { readonly kind: 'locations'; readonly locations: readonly LspLocation[]; readonly resolvedWorkspaceUri: string } | { readonly kind: 'hover'; readonly hover: LspHover | null } ``` diff --git a/docs/core-data-structures/lsp.zh.md b/docs/core-data-structures/lsp.zh.md index 51a19a51a8..d428f2b5f9 100644 --- a/docs/core-data-structures/lsp.zh.md +++ b/docs/core-data-structures/lsp.zh.md @@ -73,7 +73,7 @@ interface LspProviderQuery extends LspQueryRequest { ## 结果 -这是一个闭合的可辨识联合:导航操作规范化为 `locations`,`hover` 规范化为内容或 `null`。消费方使用 `switch` 对 `kind` 做穷尽处理,因此新增分支会使编译失败,直到完成处理。`findReferences` 始终包含声明;提供方在内部强制保证这一点,因此调用方没有对应 flag。`locations` 变体携带 `resolvedWorkspaceRoot`,即提供方对请求中 `workspaceRoot` 的规范形式,也是其 `file:` URI 所相对的根目录;调用方在相对化显示路径时应使用它,而不是可能经过符号链接的请求根目录。 +这是一个闭合的可辨识联合:导航操作规范化为 `locations`,`hover` 规范化为内容或 `null`。消费方使用 `switch` 对 `kind` 做穷尽处理,因此新增分支会使编译失败,直到完成处理。`findReferences` 始终包含声明;提供方在内部强制保证这一点,因此调用方没有对应 flag。`locations` 变体携带 `resolvedWorkspaceUri`,即提供方的规范工作区 `file:` URI。调用方相对化位置 URI 时应使用这一坐标,而不是对可能经过符号链接的请求根目录应用宿主平台路径规则。 ```ts type-equiv /** One resolved location: a document URI and the range within it. */ @@ -101,13 +101,13 @@ interface LspHover { * `goToImplementation`) normalize to `locations`; `hover` normalizes to content or `null`. * Consumers `switch` on `kind` to exhaustiveness so a new arm breaks compilation until handled. * - * The `locations` variant carries `resolvedWorkspaceRoot`: the provider's canonical form of the - * request's `workspaceRoot`, and the root its `file:` location URIs are relative to. A caller that - * relativizes display paths MUST use this, not the request's (possibly symlinked) `workspaceRoot`; - * otherwise a symlinked workspace misclassifies in-workspace results as external. + * The `locations` variant carries `resolvedWorkspaceUri`: the provider's canonical `file:` URI for + * the request's workspace root. A caller that relativizes location URIs MUST use this, not parse the + * request's possibly symlinked process path with host-platform rules; the execution platform may + * differ from the caller's. */ type LspQueryResult = - | { readonly kind: 'locations'; readonly locations: readonly LspLocation[]; readonly resolvedWorkspaceRoot: string } + | { readonly kind: 'locations'; readonly locations: readonly LspLocation[]; readonly resolvedWorkspaceUri: string } | { readonly kind: 'hover'; readonly hover: LspHover | null } ``` diff --git a/packages/lsp/lsp-local/src/instance.ts b/packages/lsp/lsp-local/src/instance.ts index 8584f64f48..318704b9fc 100644 --- a/packages/lsp/lsp-local/src/instance.ts +++ b/packages/lsp/lsp-local/src/instance.ts @@ -244,10 +244,9 @@ export class LspInstance { if (operation === 'hover') { return { kind: 'hover', hover: normalizeHover(payload) } } - // `spec.cwd` is the canonical workspace realpath (the provider canonicalizes before spawning), - // and every `file:` location URI is relative to it — so it is the root a caller must relativize - // display paths against, not the request's possibly-symlinked workspaceRoot. - return { kind: 'locations', locations: normalizeLocations(payload), resolvedWorkspaceRoot: this.spec.cwd } + // The filesystem provider owns URI syntax for the execution platform, which may differ from the + // harness host. Preserve that coordinate through rendering instead of reparsing `spec.cwd` there. + return { kind: 'locations', locations: normalizeLocations(payload), resolvedWorkspaceUri: this.spec.workspaceUri } } private answerServerRequest(method: string, params: unknown): Promise { diff --git a/packages/lsp/lsp-local/tests/instance.spec.ts b/packages/lsp/lsp-local/tests/instance.spec.ts index 3027cbbd29..811f15c9fa 100644 --- a/packages/lsp/lsp-local/tests/instance.spec.ts +++ b/packages/lsp/lsp-local/tests/instance.spec.ts @@ -117,17 +117,17 @@ describe('LspInstance server-request handling', () => { it('accepts a lifecycle client/registerCapability request', async () => { const instance = makeInstance({ LSP_FAKE_ON_OPEN: 'lifecycle', LSP_FAKE_DEF: 'null' }) - await expect(run(instance, 'goToDefinition')).resolves.toEqual({ kind: 'locations', locations: [], resolvedWorkspaceRoot: ws }) + await expect(run(instance, 'goToDefinition')).resolves.toEqual({ kind: 'locations', locations: [], resolvedWorkspaceUri: pathToFileURL(ws).href }) }) it('rejects a workspace/applyEdit request but keeps serving', async () => { const instance = makeInstance({ LSP_FAKE_ON_OPEN: 'applyEdit', LSP_FAKE_DEF: 'null' }) - await expect(run(instance, 'goToDefinition')).resolves.toEqual({ kind: 'locations', locations: [], resolvedWorkspaceRoot: ws }) + await expect(run(instance, 'goToDefinition')).resolves.toEqual({ kind: 'locations', locations: [], resolvedWorkspaceUri: pathToFileURL(ws).href }) }) it('rejects an unknown server request but keeps serving', async () => { const instance = makeInstance({ LSP_FAKE_ON_OPEN: 'unknown', LSP_FAKE_DEF: 'null' }) - await expect(run(instance, 'goToDefinition')).resolves.toEqual({ kind: 'locations', locations: [], resolvedWorkspaceRoot: ws }) + await expect(run(instance, 'goToDefinition')).resolves.toEqual({ kind: 'locations', locations: [], resolvedWorkspaceUri: pathToFileURL(ws).href }) }) }) @@ -263,7 +263,7 @@ describe('LspInstance query and abort', () => { await expect(run(instance, 'goToDefinition')).resolves.toEqual({ kind: 'locations', locations: [], - resolvedWorkspaceRoot: ws, + resolvedWorkspaceUri: pathToFileURL(ws).href, }) expect(instance.dead).toBe(true) }) diff --git a/packages/lsp/lsp-local/tests/lifecycle.spec.ts b/packages/lsp/lsp-local/tests/lifecycle.spec.ts index a7a006c06e..ce92e2c8fb 100644 --- a/packages/lsp/lsp-local/tests/lifecycle.spec.ts +++ b/packages/lsp/lsp-local/tests/lifecycle.spec.ts @@ -102,7 +102,7 @@ describe('lsp-local end to end over a fake server', () => { expect(result).toEqual({ kind: 'locations', locations: [{ uri: pathToFileURL(join(ws, 'a.ts')).href, range: { start: { line: 0, character: 0 }, end: { line: 0, character: 3 } } }], - resolvedWorkspaceRoot: ws, + resolvedWorkspaceUri: pathToFileURL(ws).href, }) await ctx.fiber.dispose() }) @@ -133,7 +133,7 @@ describe('lsp-local end to end over a fake server', () => { it('returns an empty locations result for a null definition', async () => { const ctx = await mount({ LSP_FAKE_DEF: 'null' }) - expect(await ctx.lsp.query(query('goToDefinition'))).toEqual({ kind: 'locations', locations: [], resolvedWorkspaceRoot: ws }) + expect(await ctx.lsp.query(query('goToDefinition'))).toEqual({ kind: 'locations', locations: [], resolvedWorkspaceUri: pathToFileURL(ws).href }) await ctx.fiber.dispose() }) @@ -173,7 +173,7 @@ describe('lsp-local end to end over a fake server', () => { it('accepts openClose options sync', async () => { const ctx = await mount({ LSP_FAKE_SYNC: JSON.stringify({ openClose: true, change: 2 }), LSP_FAKE_DEF: 'null' }) - expect(await ctx.lsp.query(query('goToDefinition'))).toEqual({ kind: 'locations', locations: [], resolvedWorkspaceRoot: ws }) + expect(await ctx.lsp.query(query('goToDefinition'))).toEqual({ kind: 'locations', locations: [], resolvedWorkspaceUri: pathToFileURL(ws).href }) await ctx.fiber.dispose() }) @@ -297,7 +297,7 @@ describe('lsp-local end to end over a fake server', () => { controller.abort(new Error('mid-read cancel')) await expect(pending).rejects.toThrow(/mid-read cancel/) // A subsequent live query still works, proving no half-created instance poisoned the pool. - expect(await ctx.lsp.query(query('goToDefinition'))).toEqual({ kind: 'locations', locations: [], resolvedWorkspaceRoot: ws }) + expect(await ctx.lsp.query(query('goToDefinition'))).toEqual({ kind: 'locations', locations: [], resolvedWorkspaceUri: pathToFileURL(ws).href }) await ctx.fiber.dispose() }) diff --git a/packages/lsp/lsp/README.i18n.yaml b/packages/lsp/lsp/README.i18n.yaml index 3faf906868..02649a8d39 100644 --- a/packages/lsp/lsp/README.i18n.yaml +++ b/packages/lsp/lsp/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/lsp/lsp/README.md -README.md: f96fc67ec8cb95f423eff9b312b7b591ec9d3008 -README.zh.md: 9757147de692684747d9965efda6329b3bbe2638 +README.md: 5c1044be50368acf13d8c36a15d5b2bd99d02701 +README.zh.md: cc412333e63b9469319240d67269bf0192ad3858 diff --git a/packages/lsp/lsp/README.md b/packages/lsp/lsp/README.md index f96fc67ec8..5c1044be50 100644 --- a/packages/lsp/lsp/README.md +++ b/packages/lsp/lsp/README.md @@ -27,7 +27,7 @@ Providers register **capabilities**, not tools. `dsh-tool-lsp` is the only owner ## Vocabulary -`LspQueryRequest` (`operation`, `filePath`, `position`, `workspaceRoot`) — every field required, so no field needs implementation defaulting and there is no `resolve()` step. Positions and ranges are zero-based UTF-16, matching the protocol; the tool owns the one-based cursor convention. `findReferences` always includes declarations — providers enforce this internally, so callers get no flag. `LspQueryResult` is a CLOSED discriminated union: `{ kind: 'locations'; locations; resolvedWorkspaceRoot }` for navigation, `{ kind: 'hover'; hover }` for hover (content or `null`) — consumers `switch` to exhaustiveness so a new arm breaks compilation until handled. `resolvedWorkspaceRoot` is the provider's canonical form of the request's `workspaceRoot` and the root its `file:` URIs are relative to; a caller relativizing display paths uses it, not the (possibly symlinked) request root. See `src/types.ts` for the full contracts and `src/index.ts` for the `LspError` codes, including `LSP_DISPOSED` and `LSP_MALFORMED_RESPONSE`. +`LspQueryRequest` (`operation`, `filePath`, `position`, `workspaceRoot`) — every field required, so no field needs implementation defaulting and there is no `resolve()` step. Positions and ranges are zero-based UTF-16, matching the protocol; the tool owns the one-based cursor convention. `findReferences` always includes declarations — providers enforce this internally, so callers get no flag. `LspQueryResult` is a CLOSED discriminated union: `{ kind: 'locations'; locations; resolvedWorkspaceUri }` for navigation, `{ kind: 'hover'; hover }` for hover (content or `null`) — consumers `switch` to exhaustiveness so a new arm breaks compilation until handled. `resolvedWorkspaceUri` is the provider's canonical workspace `file:` URI; callers relativize location URIs against it instead of applying host-platform path rules to the possibly symlinked request root. See `src/types.ts` for the full contracts and `src/index.ts` for the `LspError` codes, including `LSP_DISPOSED` and `LSP_MALFORMED_RESPONSE`. ## Model Experience diff --git a/packages/lsp/lsp/README.zh.md b/packages/lsp/lsp/README.zh.md index 9757147de6..cc412333e6 100644 --- a/packages/lsp/lsp/README.zh.md +++ b/packages/lsp/lsp/README.zh.md @@ -8,7 +8,7 @@ | 包 | 职责 | |---|---| -| `@deepseek-ai/dsh-lsp`(本包) | 接口:服务、以带品牌类型的 id 和扩展名映射为键的提供方注册表、逐查询选择、请求/结果词汇、`LspError` 分类体系 | +| `@deepseek-ai/dsh-lsp`(本包) | 接口:服务、以品牌化 id + 扩展名映射为 key 的提供方注册表、逐查询选择、请求/结果词汇、`LspError` 分类体系 | | `@deepseek-ai/dsh-lsp-local` | 通用本地后端,注册已配置的 stdio 语言服务器提供方 | | `@deepseek-ai/dsh-tool-lsp` | 面向模型的 `lsp` 工具,基于 `ctx.lsp` | @@ -18,16 +18,16 @@ | 成员 | 语义 | |---|---| -| `registerProvider(provider)` | 注册后端,以原子方式保留其带品牌类型的 `id` 与每个规范化文件扩展名。任何无效输入或冲突都不会发布内容,并抛出 `LspError`(`LSP_INVALID_PROVIDER`/`LSP_CONFLICT`)。返回释放所有保留项的 disposer。随调用 fiber 一同 dispose(资源释放)。 | +| `registerProvider(provider)` | 注册后端,以原子方式保留其品牌化 `id` 与每个规范化文件扩展名。任何无效输入或冲突都不会发布内容,并抛出 `LspError`(`LSP_INVALID_PROVIDER`/`LSP_CONFLICT`)。返回释放所有保留项的 disposer。随调用 fiber 释放。 | | `query(request, signal?)` | 按文件最终扩展名选择提供方,从该提供方的映射派生 `languageId`,并运行一次查询。没有匹配项时抛出 `LspError` `LSP_UNAVAILABLE`。 | -选择逐查询进行且与顺序无关:一个提供方独占一组扩展名,因此注册和 HMR(热模块替换)顺序绝不会改变路由。扩展名 key 规范化为小写且以点开头;`languageId` 只用于同步临时文档,绝不参与选择。第一版没有 glob、language-id 或显式路由 selector。 +选择逐查询进行且与顺序无关:一个提供方独占一组扩展名,因此注册和 HMR 顺序绝不会改变路由。扩展名 key 规范化为小写且以点开头;`languageId` 只用于同步临时文档,绝不参与选择。第一版没有 glob、language-id 或显式路由 selector。 -提供方注册的是**能力**,而非工具。`dsh-tool-lsp` 是面向模型名称、描述、提示词指引、schema 和呈现的唯一 owner。 +提供方注册的是**能力** 而非工具。`dsh-tool-lsp` 是面向模型名称、描述、提示词指引、schema 和呈现的唯一 owner。 ## 词汇 -`LspQueryRequest`(`operation`、`filePath`、`position`、`workspaceRoot`):每个字段都必填,因此没有字段需要实现默认值,也不存在 `resolve()` 步骤。位置与范围使用从零开始的 UTF-16,与协议一致;工具负责从 1 开始的光标约定。`findReferences` 始终包含声明,提供方在内部强制执行,因此调用方没有 flag。`LspQueryResult` 是封闭的判别联合:导航使用 `{ kind: 'locations'; locations; resolvedWorkspaceRoot }`,悬停使用 `{ kind: 'hover'; hover }`(内容或 `null`);消费方通过 `switch` 实现穷尽检查,因此新增分支会使编译失败,直到完成处理。`resolvedWorkspaceRoot` 是提供方对请求 `workspaceRoot` 的规范形式,也是其 `file:` URI 所相对的根;调用方把显示路径相对化时使用该值,而非可能含符号链接的请求根。完整契约见 `src/types.ts`;`src/index.ts` 给出 `LspError` 代码,包括 `LSP_DISPOSED` 和 `LSP_MALFORMED_RESPONSE`。 +`LspQueryRequest`(`operation`、`filePath`、`position`、`workspaceRoot`):每个字段都必填,因此没有字段需要实现默认值,也不存在 `resolve()` 步骤。位置与范围使用从零开始的 UTF-16,与协议一致;工具拥有从 1 开始的光标约定。`findReferences` 始终包含声明,提供方在内部强制执行,因此调用方没有 flag。`LspQueryResult` 是封闭的判别联合:导航使用 `{ kind: 'locations'; locations; resolvedWorkspaceUri }`,悬停使用 `{ kind: 'hover'; hover }`(内容或 `null`);消费方通过 `switch` 实现穷尽检查,因此新增分支会使编译失败,直到完成处理。`resolvedWorkspaceUri` 是提供方的规范工作区 `file:` URI;调用方相对化位置 URI 时以它为基准,而不是对可能含符号链接的请求根应用宿主平台路径规则。完整契约见 `src/types.ts`;`src/index.ts` 给出 `LspError` code,包括 `LSP_DISPOSED` 和 `LSP_MALFORMED_RESPONSE`。 ## 模型体验 @@ -39,6 +39,6 @@ ## 已知限制与暂缓事项 -- **同一运行时内扩展名归属互斥**:两个提供方不能同时声明 `.ts`,即使语言 ID 不同;重叠会使注册失败。预期扩展是在注册之上增加部署配置的 selector;它可以放宽互斥保留,而无需把提供方选择加入模型输入(见 [seam Agent Note](../../../.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md))。 +- **同一运行时内扩展名归属互斥**:两个提供方不能同时声明 `.ts`,即使 language id 不同;重叠会使注册失败。预期扩展是在注册之上增加部署配置的 selector;它可以放宽互斥保留,而无需把提供方选择加入模型输入(见 [seam Agent Note](../../../.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md))。 - **仅四种操作**:symbol 与 call hierarchy 暂缓(它们需要不同 schema);diagnostics 需要独立的新鲜度/累积规则;修改操作(rename、code action、formatting)需要独立工具,并集成预览、权限和写入策略。 -- **没有观测接口**:可用性只能通过运行 `query()` 并按抛出的 `LspError` 代码进行路由来观测;没有提供方变更事件或能力状态查询。 +- **没有观测表层**:可用性只能通过运行 `query()` 并按抛出的 `LspError` code 路由来观测;没有提供方变更事件或能力状态查询。 diff --git a/packages/lsp/lsp/src/types.ts b/packages/lsp/lsp/src/types.ts index d0c84f606d..d1bebe2a58 100644 --- a/packages/lsp/lsp/src/types.ts +++ b/packages/lsp/lsp/src/types.ts @@ -77,13 +77,13 @@ export interface LspHover { * `goToImplementation`) normalize to `locations`; `hover` normalizes to content or `null`. * Consumers `switch` on `kind` to exhaustiveness so a new arm breaks compilation until handled. * - * The `locations` variant carries `resolvedWorkspaceRoot`: the provider's canonical form of the - * request's `workspaceRoot`, and the root its `file:` location URIs are relative to. A caller that - * relativizes display paths MUST use this, not the request's (possibly symlinked) `workspaceRoot`; - * otherwise a symlinked workspace misclassifies in-workspace results as external. + * The `locations` variant carries `resolvedWorkspaceUri`: the provider's canonical `file:` URI for + * the request's workspace root. A caller that relativizes location URIs MUST use this, not parse the + * request's possibly symlinked process path with host-platform rules; the execution platform may + * differ from the caller's. */ export type LspQueryResult = - | { readonly kind: 'locations'; readonly locations: readonly LspLocation[]; readonly resolvedWorkspaceRoot: string } + | { readonly kind: 'locations'; readonly locations: readonly LspLocation[]; readonly resolvedWorkspaceUri: string } | { readonly kind: 'hover'; readonly hover: LspHover | null } /** diff --git a/packages/lsp/lsp/tests/lsp.spec.ts b/packages/lsp/lsp/tests/lsp.spec.ts index 77ea9687c7..86c9b0ac53 100644 --- a/packages/lsp/lsp/tests/lsp.spec.ts +++ b/packages/lsp/lsp/tests/lsp.spec.ts @@ -13,7 +13,7 @@ import Lsp, { function makeProvider( id: string, extensionToLanguage: Record, - result: LspQueryResult = { kind: 'locations', locations: [], resolvedWorkspaceRoot: '/ws' }, + result: LspQueryResult = { kind: 'locations', locations: [], resolvedWorkspaceUri: 'file:///ws' }, ): LspProvider & { seen: LspProviderQuery[]; seenSignals: (AbortSignal | undefined)[] } { const seen: LspProviderQuery[] = [] const seenSignals: (AbortSignal | undefined)[] = [] @@ -63,7 +63,7 @@ describe('Lsp registration', () => { const provider = makeProvider('ts', { '.ts': 'typescript' }) const dispose = lsp.registerProvider(provider) - await expect(lsp.query(query('a.ts'))).resolves.toEqual({ kind: 'locations', locations: [], resolvedWorkspaceRoot: '/ws' }) + await expect(lsp.query(query('a.ts'))).resolves.toEqual({ kind: 'locations', locations: [], resolvedWorkspaceUri: 'file:///ws' }) expect(provider.seen[0]).toMatchObject({ filePath: 'a.ts', languageId: 'typescript' }) dispose() @@ -148,7 +148,7 @@ describe('Lsp registration', () => { const py = makeProvider('py', { '.py': 'python' }) lsp.registerProvider(ts) lsp.registerProvider(py) - await expect(lsp.query(query('a.py'))).resolves.toEqual({ kind: 'locations', locations: [], resolvedWorkspaceRoot: '/ws' }) + await expect(lsp.query(query('a.py'))).resolves.toEqual({ kind: 'locations', locations: [], resolvedWorkspaceUri: 'file:///ws' }) await expect(lsp.query(query('a.ts', 'hover'))).resolves.toEqual(hover) }) @@ -172,7 +172,7 @@ describe('Lsp registration', () => { const fiber = await ctx.plugin(Object.assign((inner: Context) => { inner.lsp.registerProvider(makeProvider('ts', { '.ts': 'typescript' })) }, { inject: ['lsp'] })) - await expect(lsp.query(query('a.ts'))).resolves.toEqual({ kind: 'locations', locations: [], resolvedWorkspaceRoot: '/ws' }) + await expect(lsp.query(query('a.ts'))).resolves.toEqual({ kind: 'locations', locations: [], resolvedWorkspaceUri: 'file:///ws' }) await fiber.dispose() await expect(lsp.query(query('a.ts'))).rejects.toThrow(expect.objectContaining({ code: 'LSP_UNAVAILABLE' })) }) diff --git a/packages/lsp/tool-lsp/README.i18n.yaml b/packages/lsp/tool-lsp/README.i18n.yaml index e2e7c7517f..85a33f73af 100644 --- a/packages/lsp/tool-lsp/README.i18n.yaml +++ b/packages/lsp/tool-lsp/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/lsp/tool-lsp/README.md -README.md: 9b4130015ddf7e1cad6fa9a0e131be86f3bd4bcc -README.zh.md: a3f59fb6e39bb6c19cc2ef06d0bc256633e75386 +README.md: 1e89abf22e853735c094d17047b2c946af210905 +README.zh.md: 396b4d22ada8761b17c01d0c40e4bb4cfe80ca67 diff --git a/packages/lsp/tool-lsp/README.md b/packages/lsp/tool-lsp/README.md index 9b4130015d..1e89abf22e 100644 --- a/packages/lsp/tool-lsp/README.md +++ b/packages/lsp/tool-lsp/README.md @@ -10,7 +10,7 @@ Namespace plugin (`name` / `inject` / `Config` / `apply`, no default export). In `lsp` accepts `operation` (`goToDefinition` | `findReferences` | `goToImplementation` | `hover`), `file_path`, `line`, and `character`. `line` and `character` are positive, one-based UTF-16 cursor coordinates; the tool converts them to the seam's zero-based positions and converts rendered locations back. `findReferences` includes declarations so impact analysis does not omit the defining site. Provider, language id, workspace root, limits, timeout, initialization, and executable stay outside model input. -The tool requires the workspace root from the session `header.cwd`, with no fallback: absence fails as `LSP_WORKSPACE_REQUIRED` before querying. Its canonical result is the complete normalized seam union: `{ kind: "locations", locations, resolvedWorkspaceRoot }` or `{ kind: "hover", hover }`; Code Mode can inspect every acquired location and zero-based range directly. Native rendering then projects stable, file-grouped `path:line:character` entries relativized against the result's `resolvedWorkspaceRoot` (the provider's canonical root), not the session cwd — so a symlinked cwd still renders in-workspace results as workspace-relative paths; a `file:` URI becomes a workspace-relative path (inside) or absolute path (outside), and any other URI stays verbatim. Empty locations and `null` hover are successful no-result responses; malformed provider payloads remain structured errors. +The tool requires the workspace root from the session `header.cwd`, with no fallback: absence fails as `LSP_WORKSPACE_REQUIRED` before querying. Its canonical result is the complete normalized seam union: `{ kind: "locations", locations, resolvedWorkspaceUri }` or `{ kind: "hover", hover }`; Code Mode can inspect every acquired location and zero-based range directly. Native rendering projects stable, file-grouped `path:line:character` entries against the provider's canonical workspace URI rather than applying host-platform path rules to the session cwd. A `file:` URI becomes a workspace-relative path inside that URI or a URI-derived absolute path outside it; malformed and non-`file:` URIs stay verbatim. Empty locations and `null` hover are successful no-result responses; malformed provider payloads remain structured errors. ## Configuration diff --git a/packages/lsp/tool-lsp/README.zh.md b/packages/lsp/tool-lsp/README.zh.md index a3f59fb6e3..396b4d22ad 100644 --- a/packages/lsp/tool-lsp/README.zh.md +++ b/packages/lsp/tool-lsp/README.zh.md @@ -8,13 +8,13 @@ Namespace 插件(`name`/`inject`/`Config`/`apply`,无默认导出) ## 工具 -`lsp` 接受 `operation`(`goToDefinition` | `findReferences` | `goToImplementation` | `hover`)、`file_path`、`line` 和 `character`。`line` 与 `character` 是正的、从 1 开始的 UTF-16 光标坐标;工具将其转换为 seam 从零开始的位置,并把渲染位置转换回来。`findReferences` 包含声明,因此影响分析不会遗漏定义位置。提供方、语言 ID、工作区根目录、限制、超时、初始化和可执行文件均不进入模型输入。 +`lsp` 接受 `operation`(`goToDefinition` | `findReferences` | `goToImplementation` | `hover`)、`file_path`、`line` 和 `character`。`line` 与 `character` 是正的、从 1 开始的 UTF-16 光标坐标;工具将其转换为 seam 从零开始的位置,并把渲染位置转换回来。`findReferences` 包含声明,因此影响分析不会遗漏定义位置。提供方、language id、Workspace 根、限制、超时、初始化和可执行文件均不进入模型输入。 -该工具要求从会话 `header.cwd` 取得工作区根目录,没有回退值:缺失时会在查询前以 `LSP_WORKSPACE_REQUIRED` 失败。其规范结果是完整的已规范化 seam 联合:`{ kind: "locations", locations, resolvedWorkspaceRoot }` 或 `{ kind: "hover", hover }`;Code Mode 可以直接检查每个已取得的位置和从零开始的范围。Native 渲染随后投影出稳定的、按文件分组的 `path:line:character` 条目,并相对于结果的 `resolvedWorkspaceRoot`(提供方的规范根目录)而非会话 cwd;因此,即使 cwd 包含符号链接,工作区内的结果仍渲染为相对路径。`file:` URI 位于工作区内时成为工作区相对路径,位于工作区外时成为绝对路径,其他 URI 保持原样。空位置和 `null` hover 都是成功的无结果响应;格式错误的提供方载荷仍是结构化错误。 +该工具要求从会话 `header.cwd` 取得 Workspace 根,没有回退值:缺失时会在查询前以 `LSP_WORKSPACE_REQUIRED` 失败。其规范结果是完整的已规范化 seam 联合:`{ kind: "locations", locations, resolvedWorkspaceUri }` 或 `{ kind: "hover", hover }`;Code Mode 可以直接检查每个已取得的位置和从零开始的范围。原生渲染以提供方的规范工作区 URI 为基准,投影按文件稳定分组的 `path:line:character` 条目,而不对会话 cwd 应用宿主平台路径规则。`file:` URI 落在该工作区 URI 内时成为工作区相对路径,位于其外时成为从 URI 派生的绝对路径;格式错误的 URI 与非 `file:` URI 保持原样。空位置和 `null` hover 都是成功的无结果响应;格式错误的提供方载荷仍是结构化错误。 ## 配置 -| 配置键 | 默认值 | 含义 | +| Key | 默认值 | 含义 | |---|---|---| | `maxLocations` | `100` | 出现省略标记前可渲染位置的最大数量。 | | `maxResultChars` | `16000` | 完整渲染结果的最大长度,包括截断元数据。 | @@ -40,7 +40,7 @@ Use search/read for ordinary navigation. Use lsp when textual matches are ambigu #### KV Cache 影响 -只要插件作用域与指引文本不变,前缀就保持稳定;激活或 dispose(资源释放)可能使从该区段起的复用失效。 +只要插件 scope 与指引文本不变,前缀就保持稳定;激活或释放可能使从该区段起的复用失效。 ### 工具 schema @@ -54,13 +54,13 @@ Use search/read for ordinary navigation. Use lsp when textual matches are ambigu #### KV Cache 影响 -只要可见工具定义与顺序不变,前缀就保持稳定;注册生命周期或作用域限制可能使从第一个变化的 schema token 起的复用失效。 +只要可见工具定义与顺序不变,前缀就保持稳定;注册生命周期或 scope 限制可能使从第一个变化的 schema token 起的复用失效。 ### 结果 #### 模型看到的内容 -按文件分组的 `path:line:character` 位置行或规范化 hover 文本,先由 `maxLocations` 限制,再由 `maxResultChars` 限制;省略与截断标记计入完整字符上限。这些上限只影响 Native/模型呈现,不影响规范值。空结果使用不同的 `No results.`/`No hover information.` 行。 +按文件分组的 `path:line:character` 位置行或规范化 hover 文本,先由 `maxLocations` 限制,再由 `maxResultChars` 限制;省略与截断标记计入完整字符上限。这些上限只影响原生/模型呈现,不影响规范值。空结果使用不同的 `No results.`/`No hover information.` 行。 #### Token 影响 @@ -74,7 +74,7 @@ Use search/read for ordinary navigation. Use lsp when textual matches are ambigu #### 模型看到的内容 -无。客户端渲染通用搜索卡片:`{ card: 'generic', kind: 'search', title, locations: [{ path, line }] }`;从 args 派生的标题携带操作与从 1 开始的光标,编辑器跟随定位会聚焦所查询的行,标题则保留列号。 +无。客户端渲染通用搜索卡片:`{ card: 'generic', kind: 'search', title, locations: [{ path, line }] }`;从 args 派生的标题携带操作与从 1 开始的光标,跟随焦点对准查询行,标题则保留列号。 #### Token 影响 @@ -86,5 +86,5 @@ Use search/read for ordinary navigation. Use lsp when textual matches are ambigu ## 已知限制与暂缓事项 -- **UTF-16 光标坐标**:列坐标与协议精确一致,但模型难以在非 BMP 字符周围计数;不在符号上的位置可能返回空结果,因此提示词解释了该约定,但不会鼓励宽泛使用 LSP(见 [seam Agent Note](../../../.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md))。 +- **UTF-16 光标坐标**:列坐标与协议精确一致,但模型难以在非 BMP 字符周围计数;非 symbol 位置可能返回空结果,因此提示词解释了该约定,但不会鼓励宽泛使用 LSP(见 [seam Agent Note](../../../.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md))。 - **不承诺跨服务器完整性**:受支持的服务器仍可能根据索引就绪情况返回空或部分结果;该工具不承诺跨语言或服务器的完整性。 diff --git a/packages/lsp/tool-lsp/src/index.ts b/packages/lsp/tool-lsp/src/index.ts index e47dbc87db..688b48b369 100644 --- a/packages/lsp/tool-lsp/src/index.ts +++ b/packages/lsp/tool-lsp/src/index.ts @@ -138,7 +138,7 @@ export function apply(ctx: Context, config: Config): void { }, }, }, - resolvedWorkspaceRoot: { type: 'string', required: true }, + resolvedWorkspaceUri: { type: 'string', required: true }, }, }, { @@ -167,7 +167,7 @@ export function apply(ctx: Context, config: Config): void { render: (_args, value) => { switch (value.kind) { case 'locations': - return [{ type: 'text', text: formatLocations(value.locations, value.resolvedWorkspaceRoot, resolved.maxLocations, resolved.maxResultChars) }] + return [{ type: 'text', text: formatLocations(value.locations, value.resolvedWorkspaceUri, resolved.maxLocations, resolved.maxResultChars) }] case 'hover': return [{ type: 'text', text: formatHover(value.hover, resolved.maxResultChars) }] /* v8 ignore next -- exhaustive over the output schema's closed union; unreachable. */ @@ -200,7 +200,7 @@ export function apply(ctx: Context, config: Config): void { end: { line: location.range.end.line, character: location.range.end.character }, }, })), - resolvedWorkspaceRoot: result.resolvedWorkspaceRoot, + resolvedWorkspaceUri: result.resolvedWorkspaceUri, } case 'hover': return { diff --git a/packages/lsp/tool-lsp/src/render.ts b/packages/lsp/tool-lsp/src/render.ts index 215dfb60e6..3f49d1edfd 100644 --- a/packages/lsp/tool-lsp/src/render.ts +++ b/packages/lsp/tool-lsp/src/render.ts @@ -6,8 +6,6 @@ * @module @deepseek-ai/dsh-tool-lsp/render */ -import { fileURLToPath } from 'node:url' -import { isAbsolute, relative, sep } from 'node:path' import type { GenericCallView } from '@deepseek-ai/dsh-tools' import type { LspHover, LspLocation, LspOperation, LspPosition } from '@deepseek-ai/dsh-lsp' @@ -74,17 +72,17 @@ function oneBased(value: number, name: string): number { /** * Render a locations result grouped by file, converting each zero-based location back to a one-based * `path:line:character` entry. A `file:` URI inside the workspace becomes a workspace-relative path; - * outside it, an absolute path; a non-`file:` URI is kept verbatim. Applies `maxLocations` and + * outside it, a URI-derived absolute path; a non-`file:` URI is kept verbatim. Applies `maxLocations` and * appends an omission marker when it truncates by count, then applies the complete result cap. * @param locations - the seam's locations (possibly empty). - * @param workspaceRoot - the canonical workspace root for relativizing `file:` paths. + * @param workspaceUri - the provider's canonical workspace `file:` URI. * @param maxLocations - the cap before truncation. * @param maxResultChars - the complete rendered-text cap, including truncation metadata. * @returns the rendered text; a distinct no-result line when there are none. */ export function formatLocations( locations: readonly LspLocation[], - workspaceRoot: string, + workspaceUri: string, maxLocations: number, maxResultChars: number, ): string { @@ -93,7 +91,7 @@ export function formatLocations( const omitted = locations.length - shown.length const grouped = new Map() for (const location of shown) { - const path = renderUri(location.uri, workspaceRoot) + const path = renderUri(location.uri, workspaceUri) const line = location.range.start.line + 1 const character = location.range.start.character + 1 const entries = grouped.get(path) ?? [] @@ -128,27 +126,63 @@ function boundResult(text: string, maxChars: number, label: string): string { } /** - * Resolve a location URI to a display path. A `file:` URI accepted by Node becomes workspace-relative - * (inside) or absolute (outside); any other URI is returned verbatim. + * Resolve a location URI without applying the harness host's path rules. A valid `file:` URI becomes + * workspace-relative when it is under the provider's canonical workspace URI, or a URI-derived + * absolute path otherwise; malformed and non-`file:` URIs remain verbatim. * @param uri - the target URI from the seam. - * @param workspaceRoot - the canonical workspace root. + * @param workspaceUri - the provider's canonical workspace `file:` URI. * @returns the display path or the verbatim URI. */ -export function renderUri(uri: string, workspaceRoot: string): string { +export function renderUri(uri: string, workspaceUri: string): string { if (!uri.startsWith('file:')) return uri - let absolute: string + let target: URL + let workspace: URL try { - absolute = fileURLToPath(uri) + target = new URL(uri) + workspace = new URL(workspaceUri) } catch { - // A malformed file: URI is not a path we can resolve; show it verbatim. return uri } - const rel = relative(workspaceRoot, absolute) - if (rel === '') return '.' - // A leading `..` SEGMENT (or an absolute rel) means outside the workspace; guard against a false - // positive on an in-workspace path whose first component merely starts with dots (e.g. `..gen/x`). - const outside = rel === '..' || rel.startsWith(`..${sep}`) || isAbsolute(rel) - return outside ? absolute : rel.split(sep).join('/') + if (workspace.protocol !== 'file:') return uri + const targetSegments = decodeFileSegments(target) + const workspaceSegments = decodeFileSegments(workspace) + if (targetSegments === undefined || workspaceSegments === undefined) return uri + const sameAuthority = target.hostname === workspace.hostname + const windowsWorld = /^[A-Za-z]:$/.test(workspaceSegments[0] ?? '') + const inside = sameAuthority + && targetSegments.length >= workspaceSegments.length + && workspaceSegments.every((segment, index) => samePathSegment(segment, targetSegments[index] as string, windowsWorld)) + if (inside) { + const relative = targetSegments.slice(workspaceSegments.length) + return relative.length === 0 ? '.' : relative.join('/') + } + return absoluteUriPath(target, targetSegments, workspaceSegments) +} + +/** Decode URI path segments while rejecting encoded separators that would change path structure. */ +function decodeFileSegments(url: URL): string[] | undefined { + try { + const decoded = url.pathname.split('/').map(segment => decodeURIComponent(segment)) + if (decoded.some(segment => /[/\\\0]/u.test(segment))) return undefined + while (decoded.at(-1) === '') decoded.pop() + decoded.shift() + return decoded + } catch { + return undefined + } +} + +/** Windows execution-world path segments are case-insensitive even on a non-Windows harness host. */ +function samePathSegment(left: string, right: string, windowsWorld: boolean): boolean { + return windowsWorld ? left.toUpperCase() === right.toUpperCase() : left === right +} + +/** Render an external file URL according to the execution-world style implied by its workspace URI. */ +function absoluteUriPath(target: URL, segments: readonly string[], workspaceSegments: readonly string[]): string { + if (target.hostname.length > 0) return `//${target.hostname}/${segments.join('/')}` + const windowsWorld = /^[A-Za-z]:$/.test(workspaceSegments[0] ?? '') + if (windowsWorld && /^[A-Za-z]:$/.test(segments[0] ?? '')) return segments.join('/') + return `/${segments.join('/')}` } /** diff --git a/packages/lsp/tool-lsp/tests/render.spec.ts b/packages/lsp/tool-lsp/tests/render.spec.ts index 1fd0eeba51..9f87d663fb 100644 --- a/packages/lsp/tool-lsp/tests/render.spec.ts +++ b/packages/lsp/tool-lsp/tests/render.spec.ts @@ -14,6 +14,7 @@ import { import type { LspLocation } from '@deepseek-ai/dsh-lsp' const WS = resolve('/home/u/proj') +const WS_URI = pathToFileURL(WS).href function loc(uri: string, line: number, character = 0): LspLocation { return { uri, range: { start: { line, character }, end: { line, character: character + 1 } } } @@ -48,64 +49,84 @@ describe('parseLspArgs', () => { describe('renderUri', () => { it('relativizes a file: URI inside the workspace with forward slashes', () => { const uri = pathToFileURL(join(WS, 'src', 'a.ts')).href - expect(renderUri(uri, WS)).toBe('src/a.ts') + expect(renderUri(uri, WS_URI)).toBe('src/a.ts') }) it('returns an absolute path for a file: URI outside the workspace', () => { const outside = resolve(WS, '..', 'other', 'lib', 'b.ts') const uri = pathToFileURL(outside).href - expect(renderUri(uri, WS)).toBe(outside) + expect(renderUri(uri, WS_URI)).toBe(outside) }) it('renders the workspace root itself as "."', () => { - expect(renderUri(pathToFileURL(WS).href, WS)).toBe('.') + expect(renderUri(WS_URI, WS_URI)).toBe('.') }) it('keeps an in-workspace path whose first segment starts with dots relative', () => { // `..generated` is a real in-workspace dir, not a parent escape; only a `..` segment is external. const uri = pathToFileURL(join(WS, '..generated', 'a.ts')).href - expect(renderUri(uri, WS)).toBe('..generated/a.ts') + expect(renderUri(uri, WS_URI)).toBe('..generated/a.ts') + }) + + it('relativizes Windows execution-world URIs on a non-Windows host', () => { + expect(renderUri('file:///C:/WORKSPACE/src/a.ts', 'file:///c:/workspace')).toBe('src/a.ts') + expect(renderUri('file:///D:/lib/b.ts', 'file:///C:/workspace')).toBe('D:/lib/b.ts') + }) + + it('renders remote file authorities without host path conversion', () => { + expect(renderUri('file://server/share/workspace/a.ts', 'file://server/share/workspace')).toBe('a.ts') + expect(renderUri('file://other/share/b.ts', 'file://server/share/workspace')).toBe('//other/share/b.ts') + expect(renderUri('file:///a.ts', 'file://server/')).toBe('/a.ts') + }) + + it('keeps malformed or mismatched URI coordinates verbatim', () => { + expect(renderUri('file://[', WS_URI)).toBe('file://[') + expect(renderUri('file:///a.ts', 'https://example.com/workspace')).toBe('file:///a.ts') + expect(renderUri('file:///a.ts', 'file:///bad%ZZ')).toBe('file:///a.ts') + expect(renderUri('file:///bad%5Cpath', WS_URI)).toBe('file:///bad%5Cpath') + expect(renderUri('file:///short', 'file:///short/deeper')).toBe('/short') + expect(renderUri('file:///', 'file:///C:/workspace')).toBe('/') }) it('keeps a non-file URI verbatim', () => { - expect(renderUri('untitled:Untitled-1', WS)).toBe('untitled:Untitled-1') - expect(renderUri('jdt://contents/Foo.class', WS)).toBe('jdt://contents/Foo.class') + expect(renderUri('untitled:Untitled-1', WS_URI)).toBe('untitled:Untitled-1') + expect(renderUri('jdt://contents/Foo.class', WS_URI)).toBe('jdt://contents/Foo.class') }) it('keeps a malformed file: URI verbatim when it cannot be parsed to a path', () => { // An encoded path separator is invalid on every platform and must remain verbatim. - expect(renderUri('file:///bad%2Fpath', WS)).toBe('file:///bad%2Fpath') + expect(renderUri('file:///bad%2Fpath', WS_URI)).toBe('file:///bad%2Fpath') }) }) describe('formatLocations', () => { it('renders a no-result line for an empty list', () => { - expect(formatLocations([], WS, DEFAULT_MAX_LOCATIONS, DEFAULT_MAX_RESULT_CHARS)).toBe('No results.') + expect(formatLocations([], WS_URI, DEFAULT_MAX_LOCATIONS, DEFAULT_MAX_RESULT_CHARS)).toBe('No results.') }) it('renders one-based path:line:character grouped by file', () => { const a = pathToFileURL(join(WS, 'a.ts')).href - const text = formatLocations([loc(a, 0, 0), loc(a, 4, 2)], WS, DEFAULT_MAX_LOCATIONS, DEFAULT_MAX_RESULT_CHARS) + const text = formatLocations([loc(a, 0, 0), loc(a, 4, 2)], WS_URI, DEFAULT_MAX_LOCATIONS, DEFAULT_MAX_RESULT_CHARS) expect(text).toBe('a.ts:1:1\na.ts:5:3') }) it('caps at maxLocations and marks the omission', () => { const a = pathToFileURL(join(WS, 'a.ts')).href const many = Array.from({ length: 5 }, (_, i) => loc(a, i)) - const text = formatLocations(many, WS, 2, DEFAULT_MAX_RESULT_CHARS) + const text = formatLocations(many, WS_URI, 2, DEFAULT_MAX_RESULT_CHARS) expect(text).toContain('a.ts:1:1') expect(text).toContain('3 more locations omitted (limit 2).') }) it('uses the singular omission marker for exactly one extra', () => { const a = pathToFileURL(join(WS, 'a.ts')).href - const text = formatLocations([loc(a, 0), loc(a, 1)], WS, 1, DEFAULT_MAX_RESULT_CHARS) + const text = formatLocations([loc(a, 0), loc(a, 1)], WS_URI, 1, DEFAULT_MAX_RESULT_CHARS) expect(text).toContain('1 more location omitted (limit 1).') }) it('caps the complete location text even when one URI is enormous', () => { const maxResultChars = 80 - const text = formatLocations([loc(`custom:${'x'.repeat(1_000_000)}`, 0)], WS, 1, maxResultChars) + const text = formatLocations([loc(`custom:${'x'.repeat(1_000_000)}`, 0)], WS_URI, 1, maxResultChars) expect(text).toHaveLength(maxResultChars) expect(text).toContain('locations truncated') }) diff --git a/packages/lsp/tool-lsp/tests/tool-lsp.spec.ts b/packages/lsp/tool-lsp/tests/tool-lsp.spec.ts index de7a5319ef..846bd702ea 100644 --- a/packages/lsp/tool-lsp/tests/tool-lsp.spec.ts +++ b/packages/lsp/tool-lsp/tests/tool-lsp.spec.ts @@ -44,6 +44,7 @@ let seq = 0 const testToolSignal = new AbortController().signal const workspaceRoot = resolve('/virtual/workspace') const resolvedWorkspaceRoot = resolve('/virtual/real-workspace') +const resolvedWorkspaceUri = pathToFileURL(resolvedWorkspaceRoot).href const workspaceAlias = resolve('/virtual/workspace-alias') /** `cwd: null` means "no agent" (tests LSP_WORKSPACE_REQUIRED); a string is the session cwd. */ function call(ctx: Context, args: unknown, cwd: string | null = workspaceRoot) { @@ -59,7 +60,7 @@ function call(ctx: Context, args: unknown, cwd: string | null = workspaceRoot) { const okLocations: LspQueryResult = { kind: 'locations', locations: [{ uri: pathToFileURL(join(workspaceRoot, 'a.ts')).href, range: { start: { line: 0, character: 0 }, end: { line: 0, character: 1 } } }], - resolvedWorkspaceRoot: workspaceRoot, + resolvedWorkspaceUri: pathToFileURL(workspaceRoot).href, } describe('tool-lsp registration', () => { @@ -138,7 +139,7 @@ describe('tool-lsp execution', () => { const { ctx } = await mount(stubProvider(() => ({ kind: 'locations', locations, - resolvedWorkspaceRoot: cappedWorkspaceRoot, + resolvedWorkspaceUri: pathToFileURL(cappedWorkspaceRoot).href, })), { maxLocations: 1 }) const result = await call(ctx, { operation: 'findReferences', file_path: 'a.ts', line: 1, character: 1 }, cappedWorkspaceRoot) expect(result.content[0]).toEqual({ @@ -147,17 +148,17 @@ describe('tool-lsp execution', () => { }) expect(result).toMatchObject({ isError: false, - value: { kind: 'locations', locations, resolvedWorkspaceRoot: cappedWorkspaceRoot }, + value: { kind: 'locations', locations, resolvedWorkspaceUri: pathToFileURL(cappedWorkspaceRoot).href }, }) }) - it('relativizes against the provider resolvedWorkspaceRoot, not the session cwd', async () => { + it('relativizes against the provider resolvedWorkspaceUri, not the session cwd', async () => { // A symlinked session cwd resolves to the real path that contains the provider's location URIs. // Relativizing against the alias would misclassify the location as external. const provider = stubProvider(() => ({ kind: 'locations', locations: [{ uri: pathToFileURL(join(resolvedWorkspaceRoot, 'a.ts')).href, range: { start: { line: 0, character: 0 }, end: { line: 0, character: 1 } } }], - resolvedWorkspaceRoot, + resolvedWorkspaceUri, })) const { ctx } = await mount(provider) const result = await call(ctx, { operation: 'goToDefinition', file_path: 'a.ts', line: 1, character: 1 }, workspaceAlias) From 587fff9fe79170905352ee8639996f623e929158 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 04:43:54 +0800 Subject: [PATCH 015/108] fix(runtime): honor platform quiescence semantics --- packages/pty/pty-local/tests/local.spec.ts | 20 ++++- .../subprocess/subprocess-local/src/spawn.ts | 84 ++++++------------- .../subprocess-local/tests/local.spec.ts | 8 +- 3 files changed, 48 insertions(+), 64 deletions(-) diff --git a/packages/pty/pty-local/tests/local.spec.ts b/packages/pty/pty-local/tests/local.spec.ts index df05438864..1c0de1660a 100644 --- a/packages/pty/pty-local/tests/local.spec.ts +++ b/packages/pty/pty-local/tests/local.spec.ts @@ -98,6 +98,22 @@ function expectReadyForNextSend(waitReason: string): void { expect(['stdin_read', 'inferred_idle']).toContain(waitReason) } +function processIsRunning(pid: number): boolean { + try { + process.kill(pid, 0) + } catch (_missingProcess) { + return false + } + if (process.platform !== 'linux') return true + try { + const stat = readFileSync(`/proc/${pid}/stat`, 'utf8') + const state = stat.slice(stat.lastIndexOf(')') + 2).split(/\s+/, 1)[0] + return !/^[ZXx]$/.test(state ?? '') + } catch (_unreadableProcEntry) { + return false + } +} + describe('pty-local real shell', () => { it('persists cwd and environment across sends, scrubs secrets, and closes', async () => { const previous = process.env.DSH_TEST_SECRET @@ -156,7 +172,7 @@ describe('pty-local real shell', () => { expect(() => process.kill(pid, 0)).toThrow() }, 10_000) - it('reaps a disowned same-session descendant after the shell exits naturally', async () => { + it('quiesces a disowned same-session descendant after the shell exits naturally', async () => { const { ctx, root, agent } = await harness('danger-full-access') const created = await ctx.pty.spawn(agent, { type: 'shell' }) const pidFile = join(root, 'disowned.pid') @@ -185,7 +201,7 @@ describe('pty-local real shell', () => { } expect(ctx.pty.list(agent)[0]?.status.kind).toBe('exited') await ctx.pty.kill(agent, created.sessionId) - expect(() => process.kill(childPid, 0)).toThrow() + expect(processIsRunning(childPid)).toBe(false) } finally { if (pid !== undefined) { try { diff --git a/packages/subprocess/subprocess-local/src/spawn.ts b/packages/subprocess/subprocess-local/src/spawn.ts index 462da41382..feb0ed0767 100644 --- a/packages/subprocess/subprocess-local/src/spawn.ts +++ b/packages/subprocess/subprocess-local/src/spawn.ts @@ -15,7 +15,6 @@ import { tmpdir } from 'node:os' import { join } from 'node:path' import { setTimeout as sleepMs } from 'node:timers/promises' import { scrubbedParentEnv } from '@deepseek-ai/dsh-subprocess' -import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout' import type { CollectedOutput, SubprocessCollect, @@ -26,14 +25,23 @@ import type { } from '@deepseek-ai/dsh-subprocess' /** - * Build a child environment: explicit caller entries merge after the scrubbed - * parent base. A string deliberately restores or overrides an entry; an - * explicit `undefined` tombstone removes an ordinary ambient entry. - * @param extra - explicit caller entries and tombstones, merged after the scrub. + * Build a child environment: explicit caller entries override the scrubbed + * parent base using the target platform's environment-key semantics, so a + * deliberately supplied credential or current `DSH_*` fact wins over the + * scrub that dropped its ambient namesake. + * @param extra - explicit caller entries merged after the scrubbed parent. * @returns the environment to hand to `spawn` for the child process. */ -export function childEnv(extra?: Readonly): NodeJS.ProcessEnv { - return { ...scrubbedParentEnv(), ...extra } +export function childEnv(extra?: Readonly>): NodeJS.ProcessEnv { + const env = scrubbedParentEnv() + if (process.platform !== 'win32') return { ...env, ...extra } + let entries = Object.entries(env) + for (const [key, value] of Object.entries(extra ?? {})) { + const normalized = key.toUpperCase() + entries = entries.filter(([inherited]) => inherited.toUpperCase() !== normalized) + entries.push([key, value]) + } + return Object.fromEntries(entries) } /** Injectable knobs so tests can exercise spill and platform behavior deterministically. */ @@ -299,12 +307,8 @@ function signalTree( * @param spec - fully resolved argv, cwd, stdio, grace, cancellation, environment. * @param internals - test-only spill-directory, platform, and taskkill overrides. * @returns live subprocess handle. - * @throws when `graceMs` cannot be represented by one Node timer. */ export function spawnSubprocess(spec: SubprocessSpawnSpec, internals: SpawnInternals = {}): SubprocessHandle { - if (!Number.isFinite(spec.graceMs) || spec.graceMs <= 0 || spec.graceMs > MAX_TIMER_DELAY_MS) { - throw new Error(`subprocess graceMs must be a positive finite number no greater than ${MAX_TIMER_DELAY_MS}`) - } const spillDir = internals.spillDir ?? privateSpillDir() const platform = internals.platform ?? process.platform const taskkill = internals.taskkill ?? taskkillProcessTree @@ -346,9 +350,7 @@ export function spawnSubprocess(spec: SubprocessSpawnSpec, internals: SpawnInter const stdoutCollector = collectStream(outMode, child.stdout, 'stdout') const stderrCollector = collectStream(errMode, child.stderr, 'stderr') - let graceTimer: ReturnType | undefined - let treeExitObserved = false - let treeExitObservation: Promise | undefined + let graceTimer: NodeJS.Timeout | undefined let settled = false // Failed spawns use pid -1 so signalling remains a no-op. @@ -356,9 +358,6 @@ export function spawnSubprocess(spec: SubprocessSpawnSpec, internals: SpawnInter /** Whether the detached tree's root (or POSIX group) is still alive. */ const treeAlive = (): boolean => { - /* v8 ignore next -- only a timer callback already queued when the observer settles can enter here; - the guard is the final defense against probing an id after its tree was confirmed absent. */ - if (treeExitObserved) return false if (pid <= 0) return false if (platform === 'win32') { // Windows has no group-liveness probe; the direct child's exit is the @@ -381,40 +380,19 @@ export function spawnSubprocess(spec: SubprocessSpawnSpec, internals: SpawnInter } } - /** - * Start or reuse the handle's single whole-tree exit observer. The first - * confirmed absence is a permanent no-more-signals boundary: it cancels a - * pending escalation before this process-group id can be reused. - */ - const observeTreeExit = (): Promise => { - treeExitObservation ??= (async () => { - while (treeAlive()) await sleepTick() - treeExitObserved = true - if (graceTimer !== undefined) clearTimeout(graceTimer) - graceTimer = undefined - })() - return treeExitObservation - } - // The escalation's tier primitive (not on the handle — terminate() is the // only consumer-facing termination verb). Guards on TREE liveness, not // outcome settlement: a TERM-trapping helper can outlive the settled direct // child and must stay signalable, while a fully-dead tree (possible pid // reuse) must not be re-signalled by a later tier. const kill = (sig: NodeJS.Signals): void => { - /* v8 ignore next -- the shared exit observer cancels the ordinary dead-tree timer; - this remains the timer/death race guard and cannot be staged deterministically. */ if (!treeAlive()) return signalTree(platform, pid, sig, child, taskkill) } const terminate = (): void => { - if (treeExitObserved || graceTimer !== undefined) return - // Observe from the first termination tier onward, even when inherited - // pipes delay `done` and no consumer has begun its own teardown wait. - void observeTreeExit() - // oxlint-disable-next-line typescript/no-unnecessary-condition -- observer can record absence before its first await. - if (treeExitObserved) return + if (graceTimer !== undefined) return // escalation already in flight + if (!treeAlive()) return kill('SIGTERM') // The escalation must survive direct-child settlement — the leader dying // does not mean the tree died — so settle does not clear this timer, and @@ -436,7 +414,7 @@ export function spawnSubprocess(spec: SubprocessSpawnSpec, internals: SpawnInter } const done = new Promise((resolve, reject) => { - let pipeDrainTimer: ReturnType | undefined + let pipeDrainTimer: NodeJS.Timeout | undefined const settle = (exitCode: number | null, signal: NodeJS.Signals | null): void => { if (settled) return settled = true @@ -459,9 +437,7 @@ export function spawnSubprocess(spec: SubprocessSpawnSpec, internals: SpawnInter // A surviving descendant that inherited a pipe must not hold the // outcome open indefinitely: after exit, the same bounded grace that // governs kills also bounds the close wait. - pipeDrainTimer = setTimeout(() => { - settle(exitCode, signal) - }, spec.graceMs) + pipeDrainTimer = setTimeout(() => { settle(exitCode, signal) }, spec.graceMs) }) child.on('close', settle) function cleanup(): void { @@ -473,23 +449,11 @@ export function spawnSubprocess(spec: SubprocessSpawnSpec, internals: SpawnInter }) const waitForExit = async (signal?: AbortSignal): Promise => { - const observed = observeTreeExit() - if (treeExitObserved) return true - if (signal?.aborted) return false - if (signal === undefined) { - await observed - return true - } - const aborted = Promise.withResolvers() - const onAbort = (): void => { aborted.resolve(false) } - signal.addEventListener('abort', onAbort, { once: true }) - /* v8 ignore next -- closes the event-loop race between the preceding aborted check and listener registration. */ - if (signal.aborted) onAbort() - try { - return await Promise.race([observed.then(() => true), aborted.promise]) - } finally { - signal.removeEventListener('abort', onAbort) + while (treeAlive()) { + if (signal?.aborted) return false + await sleepTick() } + return true } return { diff --git a/packages/subprocess/subprocess-local/tests/local.spec.ts b/packages/subprocess/subprocess-local/tests/local.spec.ts index 4662283aad..b2ebfcfed0 100644 --- a/packages/subprocess/subprocess-local/tests/local.spec.ts +++ b/packages/subprocess/subprocess-local/tests/local.spec.ts @@ -5,6 +5,7 @@ import { basename, delimiter, dirname, relative } from 'node:path' import { Context } from 'cordis' import LocalSubprocessService from '@deepseek-ai/dsh-subprocess-local' import type { SubprocessSpawnSpec, SubprocessTerminalHandle, SubprocessTerminalSpawnSpec } from '@deepseek-ai/dsh-subprocess' +import { childEnv } from '../src/spawn.ts' function spec(command: string, overrides: Partial = {}): SubprocessSpawnSpec { return { @@ -62,8 +63,11 @@ describe('LocalSubprocessService', () => { }).executableCandidates.bind(service) const platform = vi.spyOn(process, 'platform', 'get').mockReturnValue('win32') try { - expect(candidates('tool', { Path: `${delimiter}/bin`, PathExt: '.EXE;.CMD' })) - .toEqual(['/bin/tool.EXE', '/bin/tool.CMD']) + expect(Object.keys(childEnv()).filter(key => key.toUpperCase() === 'PATH')).toHaveLength(1) + const explicit = childEnv({ Path: `${delimiter}/bin`, PathExt: '.EXE;.CMD' }) + expect(Object.keys(explicit).filter(key => key.toUpperCase() === 'PATH')).toEqual(['Path']) + expect(Object.keys(explicit).filter(key => key.toUpperCase() === 'PATHEXT')).toEqual(['PathExt']) + expect(candidates('tool', explicit)).toEqual(['/bin/tool.EXE', '/bin/tool.CMD']) expect(candidates('tool', { Path: '/ambient', PATH: '/explicit', PATHEXT: '.EXE' })) .toEqual(['/explicit/tool.EXE']) expect(candidates('tool.exe', {})).toEqual([]) From f5f40c740377d37a7da1bb1e0c8143b9d3201ca7 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 05:20:40 +0800 Subject: [PATCH 016/108] fix(pty): discard pre-write readiness evidence --- ...rtable-execution-world-consumers.i18n.yaml | 4 +-- ...7-28-portable-execution-world-consumers.md | 2 +- ...8-portable-execution-world-consumers.zh.md | 2 +- packages/pty/pty-local/README.i18n.yaml | 4 +-- packages/pty/pty-local/README.md | 2 +- packages/pty/pty-local/README.zh.md | 2 +- packages/pty/pty-local/src/session.ts | 13 ++++++--- packages/pty/pty-local/tests/session.spec.ts | 27 +++++++++++++++++++ 8 files changed, 44 insertions(+), 12 deletions(-) diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml index 1033fffcc6..30e672416a 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md -2026-07-28-portable-execution-world-consumers.md: 6f75778398e8f204c85b64fe096838a3b62d398e -2026-07-28-portable-execution-world-consumers.zh.md: 22edc591f7716a92fd2a68d74c84168189743fbd +2026-07-28-portable-execution-world-consumers.md: ccb7a13a9677c106976efbe95629d0d2c027b982 +2026-07-28-portable-execution-world-consumers.zh.md: 0a7b840049a826a47f4dc0476e5991bfd0578970 diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md index 6f75778398..ccb7a13a96 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md @@ -22,7 +22,7 @@ Generic consumers use that execution world: - `dsh-bash-local` continues to map Bash semantics onto ordinary `ctx.subprocess.spawn()`. - `dsh-lsp-local` reads and contains source through `ctx.fs`, resolves and launches language servers through `ctx.subprocess`, and carries provider-owned file URIs through initialization and result rendering. Its JSON-RPC, pooling, synchronization, cancellation, and normalization stay unchanged. -- `dsh-pty-local` maps persistent-shell semantics onto `ctx.subprocess.spawnTerminal()`. The local `node-pty` and process-inspection implementation moves into `dsh-subprocess-local`; another subprocess provider supplies the same primitive. The allocation signal is detached before publication, while readiness initialization retains setup cancellation. A timed-out asynchronous write, or a cancellation whose foreground signal fails during that write, retains the send reservation until the provider settles it; completion of a stale inspection resumes polling for the current send. +- `dsh-pty-local` maps persistent-shell semantics onto `ctx.subprocess.spawnTerminal()`. The local `node-pty` and process-inspection implementation moves into `dsh-subprocess-local`; another subprocess provider supplies the same primitive. The allocation signal is detached before publication, while readiness initialization retains setup cancellation. Prompt and silence evidence collected during asynchronous pre-write inspection is discarded when the provider write begins. A timed-out asynchronous write, or a cancellation whose foreground signal fails during that write, retains the send reservation until the provider settles it; completion of a stale inspection resumes polling for the current send. - `dsh-code-runtime-subprocess` materializes a dependency-free runner through `ctx.fs` and launches it through `ctx.subprocess`, preserving the Code Runtime binding and output contract across local or remote worlds. It shares host-side worker mechanics through the non-plugin `dsh-code-runtime-worker/runtime-host` subpath instead of copying them. Preparation carries one lifecycle signal through filesystem resolution, materialization, and executable lookup so disposal can abort a stalled provider. The heap-bounded worker rejects oversized binding frames before transfer, each outer hop enforces the same bound before serialization, and the launcher publishes an accepted terminal frame before reaping its controller so a descendant that inherits controller pipes cannot suppress completion; the host still awaits process-group quiescence. `dsh-code-runtime-worker` remains a separate implementation. It is the smaller in-process backend and works in single-file distributions that cannot assume an installed Node executable. Remote filesystem/process compositions select `dsh-code-runtime-subprocess`; they do not need a provider-specific Code Runtime package. diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md index 22edc591f7..0a7b840049 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md @@ -22,7 +22,7 @@ Status: implemented - `dsh-bash-local` 继续把 Bash 语义映射到普通的 `ctx.subprocess.spawn()`。 - `dsh-lsp-local` 通过 `ctx.fs` 读取源文件并验证包含关系,通过 `ctx.subprocess` 解析和启动语言服务器,并让由提供方负责的文件 URI 贯穿初始化与结果渲染。其 JSON-RPC、池化、同步、取消和规范化保持不变。 -- `dsh-pty-local` 把持久 shell 语义映射到 `ctx.subprocess.spawnTerminal()`。本地 `node-pty` 与进程检查实现移入 `dsh-subprocess-local`;其他进程管理提供方则提供相同原语。分配信号会在发布前解除关联,而就绪初始化仍保留设置阶段的取消。异步写入超时,或在该写入期间取消时前台信号发送失败,都会保留发送预留,直至提供方将写入结算;陈旧检查完成后,会针对当前发送恢复轮询。 +- `dsh-pty-local` 把持久 shell 语义映射到 `ctx.subprocess.spawnTerminal()`。本地 `node-pty` 与进程检查实现移入 `dsh-subprocess-local`;其他进程管理提供方则提供相同原语。分配信号会在发布前解除关联,而就绪初始化仍保留设置阶段的取消。提供方开始写入时,系统会丢弃异步写入前检查期间收集的提示符与静默证据。异步写入超时,或在该写入期间取消时前台信号发送失败,都会保留发送预留,直至提供方将写入结算;陈旧检查完成后,会针对当前发送恢复轮询。 - `dsh-code-runtime-subprocess` 通过 `ctx.fs` 物化无依赖 runner,并通过 `ctx.subprocess` 启动它,从而在本地或远程执行世界中保留代码运行时的绑定与输出契约。它通过非插件子路径 `dsh-code-runtime-worker/runtime-host` 共享宿主侧 worker 机制,而不是复制这些机制。准备阶段让同一个生命周期信号贯穿文件系统解析、物化和可执行文件查找,使资源释放能够中止停滞的提供方操作。受堆上限约束的 worker 会在传输前拒绝过大的绑定帧;每个外层转发环节都会在序列化前执行相同的上限检查;launcher 会在回收 controller 前发布已接纳的终态帧,使继承 controller 管道的后代进程无法阻止完成;宿主仍会等待进程组完全停稳。 `dsh-code-runtime-worker` 仍是独立实现。它是较小的进程内后端,可用于无法假定已安装 Node 可执行文件的单文件分发。远程文件系统/进程组合选择 `dsh-code-runtime-subprocess`;它们不需要提供方专用的代码运行时包。 diff --git a/packages/pty/pty-local/README.i18n.yaml b/packages/pty/pty-local/README.i18n.yaml index 05e9b904f9..4c153eff47 100644 --- a/packages/pty/pty-local/README.i18n.yaml +++ b/packages/pty/pty-local/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/pty/pty-local/README.md -README.md: 7d0cd2c26c6c58a64ae9d5fe5598c8aa6ff62915 -README.zh.md: 0b8216a9f599c224bd2888eb428213c9d725a338 +README.md: 74dfd7f59d45649f6daf0b42408285d9ede830a7 +README.zh.md: 2c7780f7e7db896c959e017dc39299e0e16f187d diff --git a/packages/pty/pty-local/README.md b/packages/pty/pty-local/README.md index 7d0cd2c26c..74dfd7f59d 100644 --- a/packages/pty/pty-local/README.md +++ b/packages/pty/pty-local/README.md @@ -8,7 +8,7 @@ Persistent shell backend for `ctx.pty` over `ctx.subprocess.spawnTerminal`. It s The plugin injects `pty`, `sandbox`, `sandboxPolicy`, and `subprocess`, then registers the configured backend type (`shell`). `danger-full-access` starts the shell directly; confined modes wrap the exact shell argv through `ctx.sandbox`. The effective session mode is resolved at spawn. A change to a different effective mode is rejected before its `sandbox/mode` event commits while that owner has an open PTY or a spawn in progress; the fence is attached to the exact owner and therefore outlives a provider reload that retains existing sessions. Wait for creation to settle and close the sessions before changing modes, so a terminal opened with wider access cannot survive a downgrade. -Readiness combines a foreground-verified private bash prompt marker, provider-reported foreground stdin-wait facts, silence fallback, and absolute timeout. A marker is not ready until the printable tail after the latest owned marker exactly equals the controlled `PS1`, including when the OSC marker and prompt are split across data callbacks; echoed input or output following a delayed earlier prompt therefore cannot settle the current send. When bash prints the marker before the terminal provider publishes its return to the foreground process group, polling retains the candidate for `handoffGraceMs` past the ordinary silence bound so a coincident handoff can win. An interactive child that inherits `PROMPT_COMMAND` therefore cannot suppress inferred-idle readiness until the absolute timeout. Unknown foreground state is never a positive exact-idle signal. A foreground group's stdin wait that existed before a send is likewise not post-write readiness: the same group must be observed outside that wait before a later wait can settle the send, while a changed foreground group is new evidence. During unpublished startup, a fallback requires observed output; zero-output silence cannot publish an empty session, and timeout rejects the spawn. Cancellation closes the unpublished shell and rejects with the caller's exact abort reason; `PtyBackendCleanupError` separately preserves a cleanup failure. The terminal-allocation signal is detached when allocation returns, while readiness initialization keeps the setup signal, so later cancellation cannot terminate a published persistent session. Incomplete terminal-control sequences are bounded by `maxReadBytes` and discarded through their terminator after crossing that limit; a trailing carriage return is carried across callbacks so split CRLF becomes one newline. +Readiness combines a foreground-verified private bash prompt marker, provider-reported foreground stdin-wait facts, silence fallback, and absolute timeout. A marker is not ready until the printable tail after the latest owned marker exactly equals the controlled `PS1`, including when the OSC marker and prompt are split across data callbacks; echoed input or output following a delayed earlier prompt therefore cannot settle the current send. Prompt and silence evidence collected before the provider write, including while pre-write foreground inspection is pending, is discarded at the write boundary. When bash prints the marker before the terminal provider publishes its return to the foreground process group, polling retains the candidate for `handoffGraceMs` past the ordinary silence bound so a coincident handoff can win. An interactive child that inherits `PROMPT_COMMAND` therefore cannot suppress inferred-idle readiness until the absolute timeout. Unknown foreground state is never a positive exact-idle signal. A foreground group's stdin wait that existed before a send is likewise not post-write readiness: the same group must be observed outside that wait before a later wait can settle the send, while a changed foreground group is new evidence. During unpublished startup, a fallback requires observed output; zero-output silence cannot publish an empty session, and timeout rejects the spawn. Cancellation closes the unpublished shell and rejects with the caller's exact abort reason; `PtyBackendCleanupError` separately preserves a cleanup failure. The terminal-allocation signal is detached when allocation returns, while readiness initialization keeps the setup signal, so later cancellation cannot terminate a published persistent session. Incomplete terminal-control sequences are bounded by `maxReadBytes` and discarded through their terminator after crossing that limit; a trailing carriage return is carried across callbacks so split CRLF becomes one newline. Send cancellation marks queued input as canceled before asking the terminal handle to signal the current foreground process group with a real `SIGINT`; if asynchronous pre-write inspection later settles, it cannot execute that input. Cancellation never emulates interruption by writing `\x03`, so raw-mode programs remain cancellable. A send that times out during an asynchronous provider write, or whose cancellation signal fails while that write remains pending, reports its result but retains the session's send slot until the write settles, so late bytes cannot interleave with a successor. Close starts provider-owned TERM-to-KILL whole-session cleanup and awaits quiescence after the terminal outcome. A cleanup failure does not cache a permanently rejected close; a later close retries the provider operation. diff --git a/packages/pty/pty-local/README.zh.md b/packages/pty/pty-local/README.zh.md index 0b8216a9f5..2c7780f7e7 100644 --- a/packages/pty/pty-local/README.zh.md +++ b/packages/pty/pty-local/README.zh.md @@ -8,7 +8,7 @@ 该插件注入 `pty`、`sandbox`、`sandboxPolicy` 和 `subprocess`,然后注册所配置的后端类型(`shell`)。`danger-full-access` 会直接启动 shell;受限模式则通过 `ctx.sandbox` 包装确切的 shell argv。系统在 spawn 时解析会话的实际模式。当某个所有者存在开放的 PTY 或正在进行 spawn 时,如果配置变更会得到不同的实际模式,系统会在对应 `sandbox/mode` 事件提交前拒绝该变更。该限制绑定到确切所有者,因此即使提供方重新加载并保留现有会话,它仍然有效。更改模式前,请等待创建结算并关闭会话,避免以更宽权限打开的终端在权限降级后继续存在。 -就绪检测结合以下机制:由前台状态验证的私有 bash 提示符标记、提供方报告的前台 stdin 等待事实、静默回退和绝对超时。只有最近一个自有标记之后的可打印尾部与受控 `PS1` 完全相等时,系统才会把标记视为就绪;即使 OSC 标记和提示符被拆到多个数据回调中也是如此。因此,如果回显的输入或输出跟在延迟到达的先前提示符之后,该提示符无法使当前 send 完成。如果 bash 在终端提供方发布其重新取得前台进程组的状态前打印标记,轮询会在普通静默上限之后再保留该候选状态 `handoffGraceMs`,使恰好同时发生的前台交接有机会胜出。因此,继承 `PROMPT_COMMAND` 的交互式子进程无法持续压制推断空闲就绪,最多只能延续到绝对超时。未知的前台状态绝不会作为精确空闲的正向信号。同样,一次 send 之前就已存在的前台进程组 stdin 等待并不代表写入后就绪:必须先观察到同一进程组脱离该等待,之后再次进入等待才能使该次 send 完成;前台进程组发生变化则构成新的证据。尚未发布的启动过程中,回退路径要求已经观察到输出;零输出静默不能发布空会话,超时则拒绝 spawn。取消操作会关闭尚未发布的 shell,并以调用方提供的确切中止原因拒绝;`PtyBackendCleanupError` 会单独保留清理失败。终端分配返回时,分配信号会解除关联;就绪初始化则保留设置阶段信号,因此后续取消无法终止已发布的持久会话。未完成的终端控制序列受 `maxReadBytes` 限制;超过上限后,系统会丢弃内容直到其终止符。末尾的回车会跨回调保留,使拆分的 CRLF 合并为一个换行。 +就绪检测结合以下机制:由前台状态验证的私有 bash 提示符标记、提供方报告的前台 stdin 等待事实、静默回退和绝对超时。只有最近一个自有标记之后的可打印尾部与受控 `PS1` 完全相等时,系统才会把标记视为就绪;即使 OSC 标记和提示符被拆到多个数据回调中也是如此。因此,如果回显的输入或输出跟在延迟到达的先前提示符之后,该提示符无法使当前 send 完成。系统会在写入边界丢弃提供方写入前收集的提示符与静默证据,包括写入前的前台检查尚未完成时收集的证据。如果 bash 在终端提供方发布其重新取得前台进程组的状态前打印标记,轮询会在普通静默上限之后再保留该候选状态 `handoffGraceMs`,使恰好同时发生的前台交接有机会胜出。因此,继承 `PROMPT_COMMAND` 的交互式子进程无法持续压制推断空闲就绪,最多只能延续到绝对超时。未知的前台状态绝不会作为精确空闲的正向信号。同样,一次 send 之前就已存在的前台进程组 stdin 等待并不代表写入后就绪:必须先观察到同一进程组脱离该等待,之后再次进入等待才能使该次 send 完成;前台进程组发生变化则构成新的证据。尚未发布的启动过程中,回退路径要求已经观察到输出;零输出静默不能发布空会话,超时则拒绝 spawn。取消操作会关闭尚未发布的 shell,并以调用方提供的确切中止原因拒绝;`PtyBackendCleanupError` 会单独保留清理失败。终端分配返回时,分配信号会解除关联;就绪初始化则保留设置阶段信号,因此后续取消无法终止已发布的持久会话。未完成的终端控制序列受 `maxReadBytes` 限制;超过上限后,系统会丢弃内容直到其终止符。末尾的回车会跨回调保留,使拆分的 CRLF 合并为一个换行。 取消发送会先把排队输入标记为已取消,再请求终端句柄向当前前台进程组发送真正的 `SIGINT`;如果异步的写入前检查随后才结算,也无法执行该输入。取消绝不会通过写入 `\x03` 模拟中断,因此原始模式程序仍可取消。发送在提供方异步写入期间超时,或在该写入仍未完成时其取消信号发送失败,都会报告各自结果,但继续占用会话的发送槽位,直至写入结算,从而避免延迟到达的字节与后续发送交错。关闭操作启动由提供方负责的 TERM→KILL 全会话清理,并在终端结果之后等待完全停稳。清理失败不会缓存成永久拒绝的关闭操作;后续关闭会重试提供方操作。 diff --git a/packages/pty/pty-local/src/session.ts b/packages/pty/pty-local/src/session.ts index 03809aec9f..d780719c54 100644 --- a/packages/pty/pty-local/src/session.ts +++ b/packages/pty/pty-local/src/session.ts @@ -227,10 +227,7 @@ export class LocalPtySession implements PtyBackendSession { () => { this.interrupt(operation) }, ) this.active = operation - this.lastOutputAt = Date.now() - this.promptSeen = false - this.promptTextSeen = false - this.promptTail = '' + this.resetReadinessEvidence() if (request.signal !== undefined) { const onAbort = (): void => { operation.cancel() } @@ -251,6 +248,7 @@ export class LocalPtySession implements PtyBackendSession { operation.setInitialForeground(foreground) const input = `${request.text}${request.submit ? '\r' : ''}` if (input.length > 0 && !operation.cancelRequested) { + this.resetReadinessEvidence() this.writing = operation try { await this.terminal.write(Buffer.from(input, 'utf8')) @@ -276,6 +274,13 @@ export class LocalPtySession implements PtyBackendSession { } } + private resetReadinessEvidence(): void { + this.lastOutputAt = Date.now() + this.promptSeen = false + this.promptTextSeen = false + this.promptTail = '' + } + read(request: PtyReadRequest): PtyReadResult { const snapshot = this.scrollback.snapshot() const lines = snapshot.text.split('\n') diff --git a/packages/pty/pty-local/tests/session.spec.ts b/packages/pty/pty-local/tests/session.spec.ts index 5f0065d0ee..40c60c570d 100644 --- a/packages/pty/pty-local/tests/session.spec.ts +++ b/packages/pty/pty-local/tests/session.spec.ts @@ -168,6 +168,29 @@ describe('LocalPtySession readiness and output', () => { expect((await operation.done).waitReason).toBe('stdin_read') }) + it('discards prompt readiness observed during asynchronous pre-write inspection', async () => { + vi.useFakeTimers() + const terminal = new FakeTerminal() + const session = new LocalPtySession(terminal, config()) + await initialize(session, terminal) + + const inspection = Promise.withResolvers<{ processGroupId: number; inputWaiting: boolean }>() + terminal.inspectForeground = async () => await inspection.promise + const operation = session.startSend({ text: 'long-running-command', submit: true }) + let settled = false + void operation.done.then(() => { settled = true }) + + terminal.emitData('\x1b]133;D;0\x07dsh> ') + inspection.resolve({ processGroupId: 456, inputWaiting: true }) + await vi.advanceTimersByTimeAsync(20) + expect(terminal.writes).toEqual(['long-running-command\r']) + expect(settled).toBe(false) + + terminal.emitData('\x1b]133;D;0\x07dsh> ') + await vi.advanceTimersByTimeAsync(10) + expect((await operation.done).waitReason).toBe('stdin_read') + }) + it('captures prompt MOTD, writes submit explicitly, and settles exact stdin waits', async () => { vi.useFakeTimers() const terminal = new FakeTerminal() @@ -524,6 +547,8 @@ describe('LocalPtySession readiness and output', () => { const operation = session.startSend({ text: 'run', submit: true }) let settled = false void operation.done.then(() => { settled = true }) + await Promise.resolve() + await Promise.resolve() inspector.pgid = 789 terminal.emitData('\x1b]133;D;0\x07dsh> ') await vi.advanceTimersByTimeAsync(50) @@ -545,6 +570,8 @@ describe('LocalPtySession readiness and output', () => { const operation = session.startSend({ text: 'run', submit: true }) let settled = false void operation.done.then(() => { settled = true }) + await Promise.resolve() + await Promise.resolve() inspector.pgid = 789 terminal.emitData('\x1b]133;D;0\x07dsh> ') // One poll past the silence bound would already have settled inferred_idle. From ea75333c92e815a633664323dcd41565c83c4565 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 05:32:54 +0800 Subject: [PATCH 017/108] fix(lsp): compare UNC paths case-insensitively --- packages/lsp/tool-lsp/src/render.ts | 7 ++++++- packages/lsp/tool-lsp/tests/render.spec.ts | 2 ++ 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/packages/lsp/tool-lsp/src/render.ts b/packages/lsp/tool-lsp/src/render.ts index 3f49d1edfd..4613312c87 100644 --- a/packages/lsp/tool-lsp/src/render.ts +++ b/packages/lsp/tool-lsp/src/render.ts @@ -148,7 +148,7 @@ export function renderUri(uri: string, workspaceUri: string): string { const workspaceSegments = decodeFileSegments(workspace) if (targetSegments === undefined || workspaceSegments === undefined) return uri const sameAuthority = target.hostname === workspace.hostname - const windowsWorld = /^[A-Za-z]:$/.test(workspaceSegments[0] ?? '') + const windowsWorld = isWindowsFileWorld(workspace, workspaceSegments) const inside = sameAuthority && targetSegments.length >= workspaceSegments.length && workspaceSegments.every((segment, index) => samePathSegment(segment, targetSegments[index] as string, windowsWorld)) @@ -159,6 +159,11 @@ export function renderUri(uri: string, workspaceUri: string): string { return absoluteUriPath(target, targetSegments, workspaceSegments) } +/** Whether a canonical file URI names a drive path or UNC path in a Windows execution world. */ +function isWindowsFileWorld(url: URL, segments: readonly string[]): boolean { + return url.hostname.length > 0 || /^[A-Za-z]:$/.test(segments[0] ?? '') +} + /** Decode URI path segments while rejecting encoded separators that would change path structure. */ function decodeFileSegments(url: URL): string[] | undefined { try { diff --git a/packages/lsp/tool-lsp/tests/render.spec.ts b/packages/lsp/tool-lsp/tests/render.spec.ts index 9f87d663fb..d76600ea6f 100644 --- a/packages/lsp/tool-lsp/tests/render.spec.ts +++ b/packages/lsp/tool-lsp/tests/render.spec.ts @@ -75,8 +75,10 @@ describe('renderUri', () => { it('renders remote file authorities without host path conversion', () => { expect(renderUri('file://server/share/workspace/a.ts', 'file://server/share/workspace')).toBe('a.ts') + expect(renderUri('file://SERVER/share/workspace/src/A.ts', 'file://server/Share/Workspace')).toBe('src/A.ts') expect(renderUri('file://other/share/b.ts', 'file://server/share/workspace')).toBe('//other/share/b.ts') expect(renderUri('file:///a.ts', 'file://server/')).toBe('/a.ts') + expect(renderUri('file:///a.ts', 'file:///')).toBe('a.ts') }) it('keeps malformed or mismatched URI coordinates verbatim', () => { From d29c73ddd6cbd4d5f4e8feab980f06961f5414f8 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 06:04:36 +0800 Subject: [PATCH 018/108] fix(subprocess): treat zombie-only groups as quiescent --- .../subprocess-local/src/process-inspector.ts | 29 +++++++++++++++++++ .../subprocess/subprocess-local/src/spawn.ts | 9 ++++++ .../tests/process-inspector.spec.ts | 21 +++++++++++++- .../subprocess-local/tests/spawn.spec.ts | 18 ++++++++++++ 4 files changed, 76 insertions(+), 1 deletion(-) diff --git a/packages/subprocess/subprocess-local/src/process-inspector.ts b/packages/subprocess/subprocess-local/src/process-inspector.ts index 48b01e18e3..f31de010de 100644 --- a/packages/subprocess/subprocess-local/src/process-inspector.ts +++ b/packages/subprocess/subprocess-local/src/process-inspector.ts @@ -87,6 +87,35 @@ function readLinuxStat(internals: ProcessInspectorInternals, pid: number): ProcS } } +/** + * Report whether a Linux process group has an executing member. `false` + * means the group contains only zombie/dead entries; `undefined` means the + * process table could not prove either outcome. + * @param processGroupId - POSIX process-group id to inspect. + * @param internals - injectable process-table operations. + * @returns Live-member presence, or `undefined` when unavailable/absent. + */ +export function linuxProcessGroupHasLiveMembers( + processGroupId: number, + internals: ProcessInspectorInternals = DEFAULT_INTERNALS, +): boolean | undefined { + let entries: string[] + try { + entries = internals.readDir('/proc') + } catch (_unreadableProcDirectory) { + return undefined + } + let matched = false + for (const entry of entries) { + if (!/^\d+$/.test(entry)) continue + const stat = readLinuxStat(internals, Number(entry)) + if (stat?.pgrp !== processGroupId) continue + matched = true + if (!/^[ZXx]$/.test(stat.state)) return true + } + return matched ? false : undefined +} + function numericEntries(internals: ProcessInspectorInternals, path: string): number[] { try { return internals.readDir(path).filter(entry => /^\d+$/.test(entry)).map(Number) diff --git a/packages/subprocess/subprocess-local/src/spawn.ts b/packages/subprocess/subprocess-local/src/spawn.ts index feb0ed0767..9fa70250dc 100644 --- a/packages/subprocess/subprocess-local/src/spawn.ts +++ b/packages/subprocess/subprocess-local/src/spawn.ts @@ -23,6 +23,7 @@ import type { SubprocessOutputMode, SubprocessSpawnSpec, } from '@deepseek-ai/dsh-subprocess' +import { linuxProcessGroupHasLiveMembers } from './process-inspector.ts' /** * Build a child environment: explicit caller entries override the scrubbed @@ -52,6 +53,8 @@ export interface SpawnInternals { taskkill?: (pid: number) => void /** Host platform override for signalling decisions. */ platform?: NodeJS.Platform + /** Linux process-group member probe (defaults to `/proc` inspection). */ + linuxProcessGroupHasLiveMembers?: (processGroupId: number) => boolean | undefined } /** @@ -312,6 +315,7 @@ export function spawnSubprocess(spec: SubprocessSpawnSpec, internals: SpawnInter const spillDir = internals.spillDir ?? privateSpillDir() const platform = internals.platform ?? process.platform const taskkill = internals.taskkill ?? taskkillProcessTree + const linuxGroupHasLiveMembers = internals.linuxProcessGroupHasLiveMembers ?? linuxProcessGroupHasLiveMembers if (spec.signal?.aborted) { throw new Error(`aborted before spawn: ${String(spec.signal.reason ?? 'aborted')}`) @@ -366,6 +370,11 @@ export function spawnSubprocess(spec: SubprocessSpawnSpec, internals: SpawnInter } try { process.kill(-pid, 0) + // A group containing only unreaped zombies still answers kill(0), but + // it can execute no work and cannot be signalled into quiescence. Only + // inspect after direct-child settlement so live-process polls remain a + // syscall rather than repeated process-table scans. + if (settled && platform === 'linux' && linuxGroupHasLiveMembers(pid) === false) return false return true } catch (error) { const code = (error as NodeJS.ErrnoException).code diff --git a/packages/subprocess/subprocess-local/tests/process-inspector.spec.ts b/packages/subprocess/subprocess-local/tests/process-inspector.spec.ts index bcc3ddb091..c90a7b3490 100644 --- a/packages/subprocess/subprocess-local/tests/process-inspector.spec.ts +++ b/packages/subprocess/subprocess-local/tests/process-inspector.spec.ts @@ -1,5 +1,9 @@ import { describe, expect, it } from 'vitest' -import { createProcessInspector, parseProcStat } from '@deepseek-ai/dsh-subprocess-local/src/process-inspector.ts' +import { + createProcessInspector, + linuxProcessGroupHasLiveMembers, + parseProcStat, +} from '@deepseek-ai/dsh-subprocess-local/src/process-inspector.ts' import type { ProcessInspectorInternals } from '@deepseek-ai/dsh-subprocess-local/src/process-inspector.ts' function stat(pid: number, pgrp: number, session: number, tpgid: number, started: string, parentPid = 1, state = 'S'): string { @@ -63,6 +67,21 @@ function fakeInternals() { } describe('Linux process inspector', () => { + it('treats zombie-only process groups as quiescent and fails closed when unobservable', () => { + const fake = fakeInternals() + expect(linuxProcessGroupHasLiveMembers(77, fake.internals)).toBeUndefined() + + fake.dirs.set('/proc', ['self', '10', '11', '12']) + fake.files.set('/proc/10/stat', stat(10, 77, 10, -1, '500', 1, 'Z')) + fake.files.set('/proc/11/stat', stat(11, 77, 10, -1, '501', 1, 'X')) + fake.files.set('/proc/12/stat', stat(12, 88, 12, -1, '502')) + expect(linuxProcessGroupHasLiveMembers(77, fake.internals)).toBe(false) + expect(linuxProcessGroupHasLiveMembers(99, fake.internals)).toBeUndefined() + + fake.files.set('/proc/11/stat', stat(11, 77, 10, -1, '501')) + expect(linuxProcessGroupHasLiveMembers(77, fake.internals)).toBe(true) + }) + it('parses stat safely, captures only the rooted process tree, and signals identities', () => { expect(parseProcStat('bad')).toBeUndefined() expect(parseProcStat('1 () ')).toBeUndefined() diff --git a/packages/subprocess/subprocess-local/tests/spawn.spec.ts b/packages/subprocess/subprocess-local/tests/spawn.spec.ts index 08b6b1dbf8..7d92447794 100644 --- a/packages/subprocess/subprocess-local/tests/spawn.spec.ts +++ b/packages/subprocess/subprocess-local/tests/spawn.spec.ts @@ -268,6 +268,24 @@ describe('spawnSubprocess', () => { expect(result.signal).toBe('SIGTERM') }) + it('does not wait for a Linux group that has only zombie members', async () => { + const pidFile = join(spillDir, `zombie-group-${Date.now()}.pid`) + let hasLiveMembers = false + const running = spawnSubprocess(spec(`sleep 60 & echo $! > ${pidFile}; echo leader-done`, { graceMs: 100 }), { + platform: 'linux', + linuxProcessGroupHasLiveMembers: () => hasLiveMembers, + }) + const descendant = await waitForPidFile(pidFile) + try { + await running.done + await expect(running.waitForExit()).resolves.toBe(true) + } finally { + hasLiveMembers = true + running.terminate() + await waitGone(descendant) + } + }) + it('bounds inherited-pipe draining after the shell exits', async () => { const pidFile = join(spillDir, `pipe-holder-${Date.now()}.pid`) const started = Date.now() From 3572c65d57214cd66819239616b5238e3482bb13 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 06:04:49 +0800 Subject: [PATCH 019/108] fix(pty): join asynchronous terminal operations --- packages/pty/pty-local/src/session.ts | 19 ++++++-- packages/pty/pty-local/tests/session.spec.ts | 47 ++++++++++++++++++++ 2 files changed, 62 insertions(+), 4 deletions(-) diff --git a/packages/pty/pty-local/src/session.ts b/packages/pty/pty-local/src/session.ts index d780719c54..0fb9f4ab0b 100644 --- a/packages/pty/pty-local/src/session.ts +++ b/packages/pty/pty-local/src/session.ts @@ -165,6 +165,7 @@ export class LocalPtySession implements PtyBackendSession { private activeTimer: NodeJS.Timeout | undefined private activeDeadlineTimer: NodeJS.Timeout | undefined private activeAbort: (() => void) | undefined + private readonly terminalOperations = new Set>() private writing: LocalSendOperation | undefined private pollingReady: LocalSendOperation | undefined private polling = false @@ -237,10 +238,18 @@ export class LocalPtySession implements PtyBackendSession { this.activeDeadlineTimer = setTimeout(() => { if (this.active === operation) this.settleActive('timeout', this.writing === operation) }, this.config.timeoutMs) - void this.beginSend(operation, request) + this.ownTerminalOperation(this.beginSend(operation, request)) return operation } + /** Retain one contained provider operation until its asynchronous work finishes. */ + private ownTerminalOperation(operation: Promise): void { + const tracked = operation.finally(() => { this.terminalOperations.delete(tracked) }) + this.terminalOperations.add(tracked) + // beginSend(), pollReadiness(), and interrupt() contain their own boundary errors. + void tracked + } + private async beginSend(operation: LocalSendOperation, request: PtySendRequest): Promise { try { const foreground = await this.terminal.inspectForeground() @@ -398,7 +407,7 @@ export class LocalPtySession implements PtyBackendSession { if (this.activeTimer !== undefined) clearTimeout(this.activeTimer) this.activeTimer = setTimeout(() => { this.activeTimer = undefined - void this.pollReadiness(operation) + this.ownTerminalOperation(this.pollReadiness(operation)) }, delayMs) } @@ -467,6 +476,7 @@ export class LocalPtySession implements PtyBackendSession { this.activeTimer = undefined if (this.activeDeadlineTimer !== undefined) clearTimeout(this.activeDeadlineTimer) this.activeDeadlineTimer = undefined + this.pollingReady = undefined } private clearActive(): void { @@ -493,9 +503,9 @@ export class LocalPtySession implements PtyBackendSession { private interrupt(operation: LocalSendOperation): void { if (this.active !== operation) return - void this.terminal.signalForeground('SIGINT').catch((error: unknown) => { + this.ownTerminalOperation(this.terminal.signalForeground('SIGINT').then(() => {}, (error: unknown) => { if (this.active === operation) this.failActive(error, this.writing === operation) - }) + })) } private async closeOnce(reason: string): Promise { @@ -508,6 +518,7 @@ export class LocalPtySession implements PtyBackendSession { if (!quiescent) { throw new Error(`PTY cleanup failed (${reason}); terminal session did not reach quiescence`) } + await Promise.all(this.terminalOperations) // Whole-session cleanup can fail before the top-level process exits. Wait // for it first so that failure is reported instead of blocking forever on // `done`; successful quiescence guarantees `done` can now settle status and diff --git a/packages/pty/pty-local/tests/session.spec.ts b/packages/pty/pty-local/tests/session.spec.ts index 40c60c570d..c5fc354106 100644 --- a/packages/pty/pty-local/tests/session.spec.ts +++ b/packages/pty/pty-local/tests/session.spec.ts @@ -907,4 +907,51 @@ describe('LocalPtySession bounds, signals, and teardown', () => { await closing }) + it('does not finish close while a pre-write terminal operation is pending', async () => { + vi.useFakeTimers() + const terminal = new FakeTerminal() + const session = new LocalPtySession(terminal, config()) + await initialize(session, terminal) + + const inspection = Promise.withResolvers<{ processGroupId: number; inputWaiting: boolean }>() + terminal.inspectForeground = async () => await inspection.promise + const operation = session.startSend({ text: 'must not run', submit: true }) + const closing = session.close('pending inspection') + let closed = false + void closing.then(() => { closed = true }) + await Promise.resolve() + await Promise.resolve() + expect(closed).toBe(false) + + inspection.resolve({ processGroupId: 456, inputWaiting: false }) + await closing + expect(closed).toBe(true) + expect(terminal.writes).toEqual([]) + expect((await operation.done).waitReason).toBe('session_exit') + }) + + it('does not finish close while a terminal write is pending', async () => { + vi.useFakeTimers() + const terminal = new FakeTerminal() + const session = new LocalPtySession(terminal, config()) + await initialize(session, terminal) + + const write = Promise.withResolvers() + terminal.write = async () => { await write.promise } + const operation = session.startSend({ text: 'pending write', submit: true }) + await Promise.resolve() + await Promise.resolve() + const closing = session.close('pending write') + let closed = false + void closing.then(() => { closed = true }) + await Promise.resolve() + await Promise.resolve() + expect(closed).toBe(false) + + write.resolve(undefined) + await closing + expect(closed).toBe(true) + expect((await operation.done).waitReason).toBe('session_exit') + }) + }) From f2a3cc450e988bdb36083f57f2fb0303b96b7e89 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 06:04:59 +0800 Subject: [PATCH 020/108] fix(lsp): retain drive syntax in UNC worlds --- packages/lsp/tool-lsp/src/render.ts | 5 ++--- packages/lsp/tool-lsp/tests/render.spec.ts | 1 + 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/packages/lsp/tool-lsp/src/render.ts b/packages/lsp/tool-lsp/src/render.ts index 4613312c87..1e73e7ec6a 100644 --- a/packages/lsp/tool-lsp/src/render.ts +++ b/packages/lsp/tool-lsp/src/render.ts @@ -156,7 +156,7 @@ export function renderUri(uri: string, workspaceUri: string): string { const relative = targetSegments.slice(workspaceSegments.length) return relative.length === 0 ? '.' : relative.join('/') } - return absoluteUriPath(target, targetSegments, workspaceSegments) + return absoluteUriPath(target, targetSegments, windowsWorld) } /** Whether a canonical file URI names a drive path or UNC path in a Windows execution world. */ @@ -183,9 +183,8 @@ function samePathSegment(left: string, right: string, windowsWorld: boolean): bo } /** Render an external file URL according to the execution-world style implied by its workspace URI. */ -function absoluteUriPath(target: URL, segments: readonly string[], workspaceSegments: readonly string[]): string { +function absoluteUriPath(target: URL, segments: readonly string[], windowsWorld: boolean): string { if (target.hostname.length > 0) return `//${target.hostname}/${segments.join('/')}` - const windowsWorld = /^[A-Za-z]:$/.test(workspaceSegments[0] ?? '') if (windowsWorld && /^[A-Za-z]:$/.test(segments[0] ?? '')) return segments.join('/') return `/${segments.join('/')}` } diff --git a/packages/lsp/tool-lsp/tests/render.spec.ts b/packages/lsp/tool-lsp/tests/render.spec.ts index d76600ea6f..953f7ef4c1 100644 --- a/packages/lsp/tool-lsp/tests/render.spec.ts +++ b/packages/lsp/tool-lsp/tests/render.spec.ts @@ -77,6 +77,7 @@ describe('renderUri', () => { expect(renderUri('file://server/share/workspace/a.ts', 'file://server/share/workspace')).toBe('a.ts') expect(renderUri('file://SERVER/share/workspace/src/A.ts', 'file://server/Share/Workspace')).toBe('src/A.ts') expect(renderUri('file://other/share/b.ts', 'file://server/share/workspace')).toBe('//other/share/b.ts') + expect(renderUri('file:///D:/lib/a.ts', 'file://server/share/workspace')).toBe('D:/lib/a.ts') expect(renderUri('file:///a.ts', 'file://server/')).toBe('/a.ts') expect(renderUri('file:///a.ts', 'file:///')).toBe('a.ts') }) From 4d96ff40affe56f7301930cb45c9ee990bfe567c Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 06:23:30 +0800 Subject: [PATCH 021/108] docs(lsp): align canonical output field --- .../2026-07-20-canonical-tool-output-contract.i18n.yaml | 4 ++-- .../architecture/2026-07-20-canonical-tool-output-contract.md | 2 +- .../2026-07-20-canonical-tool-output-contract.zh.md | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/.agents/notes/implemented/architecture/2026-07-20-canonical-tool-output-contract.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-20-canonical-tool-output-contract.i18n.yaml index f8ce3ee5f5..b8db543510 100644 --- a/.agents/notes/implemented/architecture/2026-07-20-canonical-tool-output-contract.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-20-canonical-tool-output-contract.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-20-canonical-tool-output-contract.md -2026-07-20-canonical-tool-output-contract.md: b2de9480d2659153dfb8a76ee07438d12e5b07c3 -2026-07-20-canonical-tool-output-contract.zh.md: 1852ae849aec27ef3af28895e902d840af5de14d +2026-07-20-canonical-tool-output-contract.md: 2429e1c141ad8c8ee932c6d654a6ba74bc4f7618 +2026-07-20-canonical-tool-output-contract.zh.md: fd1e55eab38e5929a2dc104073b49f9de7a493e2 diff --git a/.agents/notes/implemented/architecture/2026-07-20-canonical-tool-output-contract.md b/.agents/notes/implemented/architecture/2026-07-20-canonical-tool-output-contract.md index b2de9480d2..2429e1c141 100644 --- a/.agents/notes/implemented/architecture/2026-07-20-canonical-tool-output-contract.md +++ b/.agents/notes/implemented/architecture/2026-07-20-canonical-tool-output-contract.md @@ -46,7 +46,7 @@ The first-party tools preserve their existing Native text while returning domain | `glob` | `{ paths: string[] }` | | `grep` | `{ matches: [{ path, lineNumber, line }] }` | | `web_search` / `web_fetch` | The normalized `WebSearchResult` / `WebFetchResult` | -| `lsp` | `{ kind: "locations", locations, resolvedWorkspaceRoot }` or `{ kind: "hover", hover }` | +| `lsp` | `{ kind: "locations", locations, resolvedWorkspaceUri }` or `{ kind: "hover", hover }` | | `bash` | `{ kind: "background", taskId }` or `{ kind: "foreground" } & BashRunResult` | | `terminal_open` / `terminal_list` / `terminal_send` / `terminal_read` / `terminal_signal` / `terminal_close` | Public session snapshots, bounded read/send DTOs, signal/close outcomes, or a background task handle | | `task_output` / `task_list` / `task_kill` | Public task snapshots without owner or notification bookkeeping | diff --git a/.agents/notes/implemented/architecture/2026-07-20-canonical-tool-output-contract.zh.md b/.agents/notes/implemented/architecture/2026-07-20-canonical-tool-output-contract.zh.md index 1852ae849a..26653ea769 100644 --- a/.agents/notes/implemented/architecture/2026-07-20-canonical-tool-output-contract.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-20-canonical-tool-output-contract.zh.md @@ -46,7 +46,7 @@ type ToolExecutionResult = | `glob` | `{ paths: string[] }` | | `grep` | `{ matches: [{ path, lineNumber, line }] }` | | `web_search` / `web_fetch` | 归一化后的 `WebSearchResult` / `WebFetchResult` | -| `lsp` | `{ kind: "locations", locations, resolvedWorkspaceRoot }` 或 `{ kind: "hover", hover }` | +| `lsp` | `{ kind: "locations", locations, resolvedWorkspaceUri }` 或 `{ kind: "hover", hover }` | | `bash` | `{ kind: "background", taskId }` 或 `{ kind: "foreground" } & BashRunResult` | | `terminal_open` / `terminal_list` / `terminal_send` / `terminal_read` / `terminal_signal` / `terminal_close` | 公开会话快照、有界的读取/发送 DTO、信号/关闭操作结果,或后台任务句柄 | | `task_output` / `task_list` / `task_kill` | 不含所有者或通知管理信息的公开任务快照 | From 6e5d711099a1db4f14bf897a9416c705d9fd2d4f Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 07:59:55 +0800 Subject: [PATCH 022/108] fix(lsp): cancel executable setup on disposal --- packages/lsp/lsp-local/src/index.ts | 49 ++++++++++++++----- packages/lsp/lsp-local/tests/provider.spec.ts | 37 +++++++++++++- 2 files changed, 72 insertions(+), 14 deletions(-) diff --git a/packages/lsp/lsp-local/src/index.ts b/packages/lsp/lsp-local/src/index.ts index d12e8986fc..0d511588bd 100644 --- a/packages/lsp/lsp-local/src/index.ts +++ b/packages/lsp/lsp-local/src/index.ts @@ -117,21 +117,44 @@ export async function apply(ctx: Context, config: Config): Promise { const entries = Object.entries(config.servers) if (entries.length === 0) throw new Error('lsp-local: servers must contain at least one server') + const setupAbort = new AbortController() + const stopSetupCancellation = ctx.on('internal/plugin', (fiber) => { + // An async plugin callback must observe its own disposal before Cordis can + // run effect cleanup, because unload otherwise waits for this callback. + if (fiber === ctx.fiber && fiber.uid === null) { + setupAbort.abort(new Error('lsp-local setup disposed')) + } + }) + // Resolve every server-local setting before registration so a bad later command or bound cannot // publish an earlier provider. Registry-level mapping conflicts are rolled back below. - const providers = await Promise.all(entries.map(async ([providerId, rawConfig]) => { - if (providerId.trim() === '') throw new Error('lsp-local: server ids must be non-empty strings') - const resolved = rawConfig as ResolvedServerConfig - validateServerConfig(providerId, resolved) - const executable = await ctx.subprocess.resolveExecutable(resolved.command, resolved.env) - return new LocalLspProvider( - providerId, - ctx.fs, - resolved, - executable, - spec => ctx.subprocess.spawn(spec), - ) - })) + const providers = await (async () => { + try { + return await Promise.all(entries.map(async ([providerId, rawConfig]) => { + if (providerId.trim() === '') throw new Error('lsp-local: server ids must be non-empty strings') + const resolved = rawConfig as ResolvedServerConfig + validateServerConfig(providerId, resolved) + const executable = await ctx.subprocess.resolveExecutable( + resolved.command, + resolved.env, + setupAbort.signal, + ) + setupAbort.signal.throwIfAborted() + return new LocalLspProvider( + providerId, + ctx.fs, + resolved, + executable, + spec => ctx.subprocess.spawn(spec), + ) + })) + } catch (error: unknown) { + setupAbort.abort(error) + throw error + } finally { + stopSetupCancellation() + } + })() ctx.effect(() => { const disposers: Array<() => void> = [] diff --git a/packages/lsp/lsp-local/tests/provider.spec.ts b/packages/lsp/lsp-local/tests/provider.spec.ts index b0209e09b8..8d8a7ad356 100644 --- a/packages/lsp/lsp-local/tests/provider.spec.ts +++ b/packages/lsp/lsp-local/tests/provider.spec.ts @@ -1,4 +1,4 @@ -import { afterEach, beforeEach, describe, expect, it } from 'vitest' +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' import { chmod, mkdtemp, mkdir, rm, writeFile, realpath } from 'node:fs/promises' import { tmpdir } from 'node:os' import { delimiter, join } from 'node:path' @@ -195,6 +195,41 @@ describe('lsp-local provider resolution', () => { await ctx.fiber.dispose() }) + it('aborts executable resolution when disposed during setup', async () => { + const ctx = new Context() + await ctx.plugin(Lsp) + await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) + const subprocess = ctx.subprocess + const lookupStarted = Promise.withResolvers() + vi.spyOn(subprocess, 'resolveExecutable').mockImplementation(async (_command, _env, signal) => { + if (signal === undefined) throw new Error('missing setup signal') + lookupStarted.resolve(signal) + return await new Promise((_resolve, reject) => { + const onAbort = (): void => { + reject(signal.reason instanceof Error ? signal.reason : new Error(String(signal.reason))) + } + signal.addEventListener('abort', onAbort, { once: true }) + if (signal.aborted) onAbort() + }) + }) + + const loading = ctx.plugin(LspLocal, config('pending', { + command: 'pending-lsp', + extensionToLanguage: { '.ts': 'typescript' }, + })) + const signal = await lookupStarted.promise + const unrelated = await ctx.plugin(() => {}) + await unrelated.dispose() + expect(signal.aborted).toBe(false) + const disposing = loading.dispose() + + await expect(loading).rejects.toThrow('lsp-local setup disposed') + await expect(disposing).resolves.toBeUndefined() + expect(signal.aborted).toBe(true) + await ctx.fiber.dispose() + }) + it('rolls back earlier registrations when a later server conflicts', async () => { const ctx = new Context() await ctx.plugin(Lsp) From a95e01d8b02bc4882d840441b59d680b34d9ab39 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 08:00:08 +0800 Subject: [PATCH 023/108] fix(pty): detach cancellation before close drain --- packages/pty/pty-local/src/session.ts | 5 ++++- packages/pty/pty-local/tests/session.spec.ts | 22 ++++++++++++++++++++ 2 files changed, 26 insertions(+), 1 deletion(-) diff --git a/packages/pty/pty-local/src/session.ts b/packages/pty/pty-local/src/session.ts index 0fb9f4ab0b..267c7c17fc 100644 --- a/packages/pty/pty-local/src/session.ts +++ b/packages/pty/pty-local/src/session.ts @@ -518,13 +518,16 @@ export class LocalPtySession implements PtyBackendSession { if (!quiescent) { throw new Error(`PTY cleanup failed (${reason}); terminal session did not reach quiescence`) } + // Quiescence is the active send's terminal outcome. Detach its abort + // listener before snapshotting provider operations so no late interrupt + // can enter the owned set after the drain starts. + this.settleActive('session_exit') await Promise.all(this.terminalOperations) // Whole-session cleanup can fail before the top-level process exits. Wait // for it first so that failure is reported instead of blocking forever on // `done`; successful quiescence guarantees `done` can now settle status and // drain the terminal output. await this.completion - this.settleActive('session_exit') this.terminal.output.off('data', this.onTerminalData) this.terminal.output.off('end', this.onTerminalEnd) this.terminal.output.off('error', this.onTerminalError) diff --git a/packages/pty/pty-local/tests/session.spec.ts b/packages/pty/pty-local/tests/session.spec.ts index c5fc354106..cce384bf6b 100644 --- a/packages/pty/pty-local/tests/session.spec.ts +++ b/packages/pty/pty-local/tests/session.spec.ts @@ -954,4 +954,26 @@ describe('LocalPtySession bounds, signals, and teardown', () => { expect((await operation.done).waitReason).toBe('session_exit') }) + it('detaches active cancellation before draining terminal operations', async () => { + vi.useFakeTimers() + const terminal = new FakeTerminal() + const session = new LocalPtySession(terminal, config()) + await initialize(session, terminal) + + const inspection = Promise.withResolvers<{ processGroupId: number; inputWaiting: boolean }>() + terminal.inspectForeground = async () => await inspection.promise + const signalForeground = vi.spyOn(terminal, 'signalForeground') + const controller = new AbortController() + const operation = session.startSend({ text: 'pending inspection', submit: true, signal: controller.signal }) + const closing = session.close('pending cancellation') + await Promise.resolve() + await Promise.resolve() + + controller.abort('late cancellation') + expect(signalForeground).not.toHaveBeenCalled() + inspection.resolve({ processGroupId: 456, inputWaiting: false }) + await closing + expect((await operation.done).waitReason).toBe('session_exit') + }) + }) From 06a922490418f6dea751328e8ee0970762036d6e Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 08:00:21 +0800 Subject: [PATCH 024/108] fix(subprocess): retain failed terminal cleanup --- .../subprocess/subprocess-local/src/index.ts | 3 +- .../subprocess-local/tests/local.spec.ts | 30 +++++++++++++++++-- 2 files changed, 29 insertions(+), 4 deletions(-) diff --git a/packages/subprocess/subprocess-local/src/index.ts b/packages/subprocess/subprocess-local/src/index.ts index 4c5cad8018..6d7684eeeb 100644 --- a/packages/subprocess/subprocess-local/src/index.ts +++ b/packages/subprocess/subprocess-local/src/index.ts @@ -64,10 +64,9 @@ export class LocalSubprocessService extends SubprocessService { // an identity-fenced descendant survives escalation). Await the cleanup // transaction directly so disposal reports that failure rather than // waiting forever on `done`. - pending.push(terminal.waitForExit()) + pending.push(terminal.waitForExit().then(() => { this.terminals.delete(terminal) })) } this.live.clear() - this.terminals.clear() await Promise.all(pending) await rm(this.runtimeRoot, { recursive: true, force: true }) }, 'local subprocess teardown') diff --git a/packages/subprocess/subprocess-local/tests/local.spec.ts b/packages/subprocess/subprocess-local/tests/local.spec.ts index b2ebfcfed0..b772545f7f 100644 --- a/packages/subprocess/subprocess-local/tests/local.spec.ts +++ b/packages/subprocess/subprocess-local/tests/local.spec.ts @@ -1,6 +1,6 @@ import { PassThrough } from 'node:stream' import { describe, expect, it, vi } from 'vitest' -import { stat } from 'node:fs/promises' +import { rm, stat } from 'node:fs/promises' import { basename, delimiter, dirname, relative } from 'node:path' import { Context } from 'cordis' import LocalSubprocessService from '@deepseek-ai/dsh-subprocess-local' @@ -108,10 +108,36 @@ describe('LocalSubprocessService', () => { terminate, waitForExit, } - ;(ctx.subprocess as unknown as { terminals: Set }).terminals.add(terminal) + const terminals = (ctx.subprocess as unknown as { terminals: Set }).terminals + terminals.add(terminal) await fiber.dispose() expect(terminate).toHaveBeenCalledOnce() expect(waitForExit).toHaveBeenCalledOnce() + expect(terminals.size).toBe(0) + }) + + it('retains an owned terminal when disposal cleanup rejects', async () => { + const ctx = new Context() + const fiber = await ctx.plugin(LocalSubprocessService) + const service = ctx.subprocess + const runtimeRoot = service.runtimeRoot + const terminal: SubprocessTerminalHandle = { + pid: 1, + output: new PassThrough(), + done: Promise.resolve({ exitCode: 0, signal: null }), + write: async () => {}, + inspectForeground: async () => undefined, + signalForeground: async () => 1, + terminate: vi.fn(), + waitForExit: vi.fn(async () => { throw new Error('retryable cleanup failure') }), + } + const terminals = (service as unknown as { terminals: Set }).terminals + terminals.add(terminal) + + await fiber.dispose() + expect(terminals).toEqual(new Set([terminal])) + expect((await stat(runtimeRoot)).isDirectory()).toBe(true) + await rm(runtimeRoot, { recursive: true, force: true }) }) it('releases a terminal after top-level exit reaches quiescence', async () => { From ea047cc634bf1674f0d563a4f68d72a284e1b97e Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 08:00:33 +0800 Subject: [PATCH 025/108] test(subprocess): accept zombie quiescence --- .../subprocess/subprocess-local/tests/spawn.spec.ts | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/packages/subprocess/subprocess-local/tests/spawn.spec.ts b/packages/subprocess/subprocess-local/tests/spawn.spec.ts index 7d92447794..b9f0bf0321 100644 --- a/packages/subprocess/subprocess-local/tests/spawn.spec.ts +++ b/packages/subprocess/subprocess-local/tests/spawn.spec.ts @@ -60,7 +60,7 @@ function spec(command: string, overrides: SpecOverrides = {}) { } } -/** Poll until a pid no longer exists (kill(pid, 0) throws ESRCH). */ +/** Poll until a pid no longer exists, or is only a zombie on Linux. */ async function waitGone(pid: number, timeoutMs = 5_000): Promise { const deadline = Date.now() + timeoutMs while (Date.now() < deadline) { @@ -69,6 +69,16 @@ async function waitGone(pid: number, timeoutMs = 5_000): Promise { } catch { return } + if (process.platform === 'linux') { + try { + const stat = readFileSync(`/proc/${pid}/stat`, 'utf8') + const state = stat.slice(stat.lastIndexOf(')') + 2, stat.lastIndexOf(')') + 3) + if (state === 'Z' || state === 'X') return + } catch (error: unknown) { + if ((error as NodeJS.ErrnoException).code === 'ENOENT') return + throw error + } + } await new Promise(resolve => setTimeout(resolve, 20)) } throw new Error(`pid ${pid} still alive after ${timeoutMs}ms`) From 562cd154a827ee2fc619f9d246a31f3f927e87d9 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 08:00:44 +0800 Subject: [PATCH 026/108] docs(lsp): remove resolved URI limitation --- packages/lsp/lsp-local/README.i18n.yaml | 4 ++-- packages/lsp/lsp-local/README.md | 1 - packages/lsp/lsp-local/README.zh.md | 1 - 3 files changed, 2 insertions(+), 4 deletions(-) diff --git a/packages/lsp/lsp-local/README.i18n.yaml b/packages/lsp/lsp-local/README.i18n.yaml index 1504e6f726..e2b67d46d9 100644 --- a/packages/lsp/lsp-local/README.i18n.yaml +++ b/packages/lsp/lsp-local/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/lsp/lsp-local/README.md -README.md: 2c5ab309f3557ad77696a881b41d164e65bd24fe -README.zh.md: d4493832964a5ac0d804602c18774bb106ddcbd9 +README.md: 299a5282bbca0a6d517af9342c4101cbdae33d4c +README.zh.md: edf706fdb424756aa43b303b018b11e3764c2559 diff --git a/packages/lsp/lsp-local/README.md b/packages/lsp/lsp-local/README.md index 2c5ab309f3..299a5282bb 100644 --- a/packages/lsp/lsp-local/README.md +++ b/packages/lsp/lsp-local/README.md @@ -55,6 +55,5 @@ No direct invalidation; `dsh-tool-lsp` owns request-prefix changes. ## Known Limitations and Deferred Work - **No confinement policy** — this package trusts the configured server and does not sandbox its process; a restricted deployment must supply appropriate process/filesystem providers or a same-world sandbox wrapper. -- **Execution-world URI rendering** — the stdio host produces provider-owned `file:` URIs. The current model tool renders them with the harness host's path library, so a Windows harness paired with a POSIX remote execution world may show remote locations as URIs or non-native paths; protocol queries remain correct. - **Transient-open compatibility floor** — servers whose synchronization omits open/close (or advertise `None`) are unsupported even if closed-document queries would work; the pinned TypeScript e2e establishes one compatibility floor, not a cross-language claim. - **Per-server/workspace serialization latency** — parallel agents sharing one server and workspace queue behind one process; long-lived workspace processes consume memory until disposal. diff --git a/packages/lsp/lsp-local/README.zh.md b/packages/lsp/lsp-local/README.zh.md index d449383296..edf706fdb4 100644 --- a/packages/lsp/lsp-local/README.zh.md +++ b/packages/lsp/lsp-local/README.zh.md @@ -55,6 +55,5 @@ Namespace 插件(`name`/`inject`/`Config`/`apply`,无默认导出) ## 已知限制与暂缓事项 - **不提供隔离策略**:这个包(package)信任配置的服务器,不会对其进程执行沙箱化;受限部署必须提供适当的进程/文件系统提供方,或包装同一执行环境的沙箱。 -- **执行环境 URI 渲染**:stdio 主机生成归提供方所有的 `file:` URI。当前面向模型的工具使用 harness 宿主的路径库渲染这些 URI,因此 Windows harness 与 POSIX 远程执行环境配对时,可能把远程位置显示为 URI 或非本机路径;协议查询仍然正确。 - **临时打开兼容性下限**:同步能力省略打开/关闭(或声明 `None`)的服务器不受支持,即使关闭文档查询能够工作;固定的 TypeScript e2e 只建立一项兼容性下限,不代表跨语言承诺。 - **逐服务器/Workspace 串行化延迟**:共享同一个服务器与 Workspace 的并行 agent 会在一个进程后排队;长生命周期 Workspace 进程会占用内存直到释放。 From f84f63ccd777aee8961f7631008a7f30a68faaa6 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 08:26:21 +0800 Subject: [PATCH 027/108] test(runtime): finish zombie quiescence migration --- packages/lsp/lsp-local/tests/instance.spec.ts | 13 +++++++++++-- .../subprocess/subprocess-local/tests/spawn.spec.ts | 6 +++--- 2 files changed, 14 insertions(+), 5 deletions(-) diff --git a/packages/lsp/lsp-local/tests/instance.spec.ts b/packages/lsp/lsp-local/tests/instance.spec.ts index 811f15c9fa..08bad67ae2 100644 --- a/packages/lsp/lsp-local/tests/instance.spec.ts +++ b/packages/lsp/lsp-local/tests/instance.spec.ts @@ -1,4 +1,5 @@ import { afterEach, beforeEach, describe, expect, it } from 'vitest' +import { readFileSync } from 'node:fs' import { mkdtemp, mkdir, readFile, rm, writeFile, realpath } from 'node:fs/promises' import { tmpdir } from 'node:os' import { join } from 'node:path' @@ -346,14 +347,22 @@ describe('LspInstance disposal', () => { function processAlive(pid: number): boolean { try { process.kill(pid, 0) - return true } catch (error) { if ((error as NodeJS.ErrnoException).code === 'ESRCH') return false throw error } + if (process.platform !== 'linux') return true + try { + const stat = readFileSync(`/proc/${pid}/stat`, 'utf8') + const state = stat.slice(stat.lastIndexOf(')') + 2).split(/\s+/, 1)[0] + return !/^[ZXx]$/.test(state ?? '') + } catch (error) { + if ((error as NodeJS.ErrnoException).code === 'ENOENT') return false + throw error + } } -/** Wait until a process id disappears so temporary-workspace cleanup cannot race handle release. */ +/** Wait until a process can no longer execute so temporary-workspace cleanup cannot race handle release. */ async function waitForProcessExit(pid: number, timeoutMs = 3_000): Promise { const started = Date.now() while (processAlive(pid)) { diff --git a/packages/subprocess/subprocess-local/tests/spawn.spec.ts b/packages/subprocess/subprocess-local/tests/spawn.spec.ts index b9f0bf0321..f28f741992 100644 --- a/packages/subprocess/subprocess-local/tests/spawn.spec.ts +++ b/packages/subprocess/subprocess-local/tests/spawn.spec.ts @@ -665,7 +665,7 @@ describe('tree-survivor escalation (terminate and bounded waits reach helpers th clearTimeout(timer) running.terminate() await expect(running.waitForExit()).resolves.toBe(true) - expect(() => process.kill(helper, 0)).toThrow() + await expect(waitGone(helper)).resolves.toBeUndefined() }) it('service teardown awaits tree survivors, not just handle settlement', async () => { @@ -682,8 +682,8 @@ describe('tree-survivor escalation (terminate and bounded waits reach helpers th const helper = await waitForPidFile(pidFile) await running.done await fiber.dispose() - // Teardown itself waited for the survivor to die. - expect(() => process.kill(helper, 0)).toThrow() + // Teardown itself waited for the survivor to become quiescent. + await expect(waitGone(helper)).resolves.toBeUndefined() }) }) From 71e0eeac40231c943490c328bd71be3fafdadc9b Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 09:28:50 +0800 Subject: [PATCH 028/108] fix(code-runtime): contain hostile boundary failures --- .../code-runtime-worker/src/runtime-host.ts | 6 +++++- .../code-runtime-worker/tests/runtime.spec.ts | 21 +++++++++++++++++++ 2 files changed, 26 insertions(+), 1 deletion(-) diff --git a/packages/code-runtime/code-runtime-worker/src/runtime-host.ts b/packages/code-runtime/code-runtime-worker/src/runtime-host.ts index 334bd82e38..474a058e9f 100644 --- a/packages/code-runtime/code-runtime-worker/src/runtime-host.ts +++ b/packages/code-runtime/code-runtime-worker/src/runtime-host.ts @@ -49,7 +49,11 @@ export type RuntimeBindingReply = * @returns the caller-facing diagnostic text. */ export function runtimeErrorMessage(error: unknown): string { - return error instanceof Error ? error.message : String(error) + try { + return error instanceof Error ? error.message : String(error) + } catch { + return 'binding rejected with an unrenderable value' + } } /** diff --git a/packages/code-runtime/code-runtime-worker/tests/runtime.spec.ts b/packages/code-runtime/code-runtime-worker/tests/runtime.spec.ts index 54f58eb414..db2c167354 100644 --- a/packages/code-runtime/code-runtime-worker/tests/runtime.spec.ts +++ b/packages/code-runtime/code-runtime-worker/tests/runtime.spec.ts @@ -597,6 +597,27 @@ describe('WorkerCodeRuntime — hostile programs (real workers)', () => { expect(result.value).toEqual({ name: 'ToolCallError', toolName: 'bad', message: 'binding resolution must be lossless JSON' }) }) + it('contains binding rejections whose thrown values cannot be rendered', async () => { + const { runtime } = await setup() + const result = await runtime.run({ + program: 'try { await tools.bad({}) } catch (error) { return { name: error.name, toolName: error.toolName, message: error.message } }', + bindings: tools({ + bad: async () => { + const hostile = new Error('hidden') + Object.defineProperty(hostile, 'message', { + get() { throw new Error('message getter failed') }, + }) + throw hostile + }, + }), + }) + expect(result.value).toEqual({ + name: 'ToolCallError', + toolName: 'bad', + message: 'binding rejected with an unrenderable value', + }) + }) + it('rejects lossy binding arguments in the worker before invoking the host binding', async () => { const { runtime } = await setup() let calls = 0 From f61a4943bdad0874a662f6fbfb8389cc140fd38f Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 09:29:01 +0800 Subject: [PATCH 029/108] fix(lsp): preserve POSIX backslash paths --- packages/lsp/tool-lsp/src/render.ts | 5 +++-- packages/lsp/tool-lsp/tests/render.spec.ts | 6 +++++- 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/packages/lsp/tool-lsp/src/render.ts b/packages/lsp/tool-lsp/src/render.ts index 1e73e7ec6a..2f376707db 100644 --- a/packages/lsp/tool-lsp/src/render.ts +++ b/packages/lsp/tool-lsp/src/render.ts @@ -149,6 +149,7 @@ export function renderUri(uri: string, workspaceUri: string): string { if (targetSegments === undefined || workspaceSegments === undefined) return uri const sameAuthority = target.hostname === workspace.hostname const windowsWorld = isWindowsFileWorld(workspace, workspaceSegments) + if (windowsWorld && [...targetSegments, ...workspaceSegments].some(segment => segment.includes('\\'))) return uri const inside = sameAuthority && targetSegments.length >= workspaceSegments.length && workspaceSegments.every((segment, index) => samePathSegment(segment, targetSegments[index] as string, windowsWorld)) @@ -164,11 +165,11 @@ function isWindowsFileWorld(url: URL, segments: readonly string[]): boolean { return url.hostname.length > 0 || /^[A-Za-z]:$/.test(segments[0] ?? '') } -/** Decode URI path segments while rejecting encoded separators that would change path structure. */ +/** Decode URI path segments while rejecting encoded POSIX separators and NUL. */ function decodeFileSegments(url: URL): string[] | undefined { try { const decoded = url.pathname.split('/').map(segment => decodeURIComponent(segment)) - if (decoded.some(segment => /[/\\\0]/u.test(segment))) return undefined + if (decoded.some(segment => /[/\0]/u.test(segment))) return undefined while (decoded.at(-1) === '') decoded.pop() decoded.shift() return decoded diff --git a/packages/lsp/tool-lsp/tests/render.spec.ts b/packages/lsp/tool-lsp/tests/render.spec.ts index 953f7ef4c1..f09297d3d4 100644 --- a/packages/lsp/tool-lsp/tests/render.spec.ts +++ b/packages/lsp/tool-lsp/tests/render.spec.ts @@ -82,11 +82,15 @@ describe('renderUri', () => { expect(renderUri('file:///a.ts', 'file:///')).toBe('a.ts') }) + it('preserves backslashes as ordinary POSIX filename characters', () => { + expect(renderUri('file:///home/u/proj/dir%5Cname/a.ts', WS_URI)).toBe('dir\\name/a.ts') + }) + it('keeps malformed or mismatched URI coordinates verbatim', () => { expect(renderUri('file://[', WS_URI)).toBe('file://[') expect(renderUri('file:///a.ts', 'https://example.com/workspace')).toBe('file:///a.ts') expect(renderUri('file:///a.ts', 'file:///bad%ZZ')).toBe('file:///a.ts') - expect(renderUri('file:///bad%5Cpath', WS_URI)).toBe('file:///bad%5Cpath') + expect(renderUri('file:///C:/workspace/bad%5Cpath', 'file:///C:/workspace')).toBe('file:///C:/workspace/bad%5Cpath') expect(renderUri('file:///short', 'file:///short/deeper')).toBe('/short') expect(renderUri('file:///', 'file:///C:/workspace')).toBe('/') }) From 098f826001cc407e785cc70d2f8a89a77cfa8002 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 10:02:17 +0800 Subject: [PATCH 030/108] fix(subprocess): honor teardown and PATH semantics --- .../subprocess/subprocess-local/src/index.ts | 7 ++-- .../subprocess-local/tests/local.spec.ts | 36 ++++++++++++++----- 2 files changed, 32 insertions(+), 11 deletions(-) diff --git a/packages/subprocess/subprocess-local/src/index.ts b/packages/subprocess/subprocess-local/src/index.ts index 6d7684eeeb..80d0b4c76b 100644 --- a/packages/subprocess/subprocess-local/src/index.ts +++ b/packages/subprocess/subprocess-local/src/index.ts @@ -67,7 +67,10 @@ export class LocalSubprocessService extends SubprocessService { pending.push(terminal.waitForExit().then(() => { this.terminals.delete(terminal) })) } this.live.clear() - await Promise.all(pending) + const outcomes = await Promise.allSettled(pending) + for (const outcome of outcomes) { + if (outcome.status === 'rejected') throw outcome.reason + } await rm(this.runtimeRoot, { recursive: true, force: true }) }, 'local subprocess teardown') } @@ -106,7 +109,7 @@ export class LocalSubprocessService extends SubprocessService { ? (environmentValue(env, 'PATHEXT') ?? '.COM;.EXE;.BAT;.CMD').split(';') : [''] return path.split(delimiter).flatMap(directory => - directory === '' ? [] : extensions.map(extension => resolve(this.cwd, directory, command + extension))) + extensions.map(extension => resolve(this.cwd, directory, command + extension))) } spawn(spec: SubprocessSpawnSpec): SubprocessHandle { diff --git a/packages/subprocess/subprocess-local/tests/local.spec.ts b/packages/subprocess/subprocess-local/tests/local.spec.ts index b772545f7f..46f3b14bea 100644 --- a/packages/subprocess/subprocess-local/tests/local.spec.ts +++ b/packages/subprocess/subprocess-local/tests/local.spec.ts @@ -1,7 +1,7 @@ import { PassThrough } from 'node:stream' import { describe, expect, it, vi } from 'vitest' import { rm, stat } from 'node:fs/promises' -import { basename, delimiter, dirname, relative } from 'node:path' +import { basename, dirname, relative, resolve } from 'node:path' import { Context } from 'cordis' import LocalSubprocessService from '@deepseek-ai/dsh-subprocess-local' import type { SubprocessSpawnSpec, SubprocessTerminalHandle, SubprocessTerminalSpawnSpec } from '@deepseek-ai/dsh-subprocess' @@ -42,6 +42,9 @@ describe('LocalSubprocessService', () => { expect(await ctx.subprocess.resolveExecutable(basename(process.execPath), { PATH: relative(process.cwd(), dirname(process.execPath)) || '.', })).toBe(process.execPath) + Reflect.set(ctx.subprocess, 'cwd', dirname(process.execPath)) + expect(await ctx.subprocess.resolveExecutable(basename(process.execPath), { PATH: '' })) + .toBe(process.execPath) await expect(ctx.subprocess.resolveExecutable('')).rejects.toThrow('must be non-empty') await expect(ctx.subprocess.resolveExecutable('dsh-command-that-does-not-exist', { PATH: '' })) .rejects.toThrow('was not found on PATH') @@ -54,7 +57,7 @@ describe('LocalSubprocessService', () => { await fiber.dispose() }) - it('builds Windows executable candidates without empty PATH entries', async () => { + it('builds Windows executable candidates with case-insensitive overrides', async () => { const ctx = new Context() const fiber = await ctx.plugin(LocalSubprocessService) const service = ctx.subprocess as LocalSubprocessService @@ -64,13 +67,13 @@ describe('LocalSubprocessService', () => { const platform = vi.spyOn(process, 'platform', 'get').mockReturnValue('win32') try { expect(Object.keys(childEnv()).filter(key => key.toUpperCase() === 'PATH')).toHaveLength(1) - const explicit = childEnv({ Path: `${delimiter}/bin`, PathExt: '.EXE;.CMD' }) + const explicit = childEnv({ Path: '/bin', PathExt: '.EXE;.CMD' }) expect(Object.keys(explicit).filter(key => key.toUpperCase() === 'PATH')).toEqual(['Path']) expect(Object.keys(explicit).filter(key => key.toUpperCase() === 'PATHEXT')).toEqual(['PathExt']) expect(candidates('tool', explicit)).toEqual(['/bin/tool.EXE', '/bin/tool.CMD']) expect(candidates('tool', { Path: '/ambient', PATH: '/explicit', PATHEXT: '.EXE' })) .toEqual(['/explicit/tool.EXE']) - expect(candidates('tool.exe', {})).toEqual([]) + expect(candidates('tool.exe', {})).toEqual([resolve(process.cwd(), 'tool.exe')]) expect(candidates('tool', { PATH: '/bin' })).toHaveLength(4) } finally { platform.mockRestore() @@ -116,12 +119,12 @@ describe('LocalSubprocessService', () => { expect(terminals.size).toBe(0) }) - it('retains an owned terminal when disposal cleanup rejects', async () => { + it('waits for every terminal cleanup and retains rejections', async () => { const ctx = new Context() const fiber = await ctx.plugin(LocalSubprocessService) const service = ctx.subprocess const runtimeRoot = service.runtimeRoot - const terminal: SubprocessTerminalHandle = { + const failedTerminal: SubprocessTerminalHandle = { pid: 1, output: new PassThrough(), done: Promise.resolve({ exitCode: 0, signal: null }), @@ -131,11 +134,26 @@ describe('LocalSubprocessService', () => { terminate: vi.fn(), waitForExit: vi.fn(async () => { throw new Error('retryable cleanup failure') }), } + let finishCleanup!: () => void + const cleanup = new Promise((resolve) => { + finishCleanup = () => { resolve(true) } + }) + const drainingTerminal: SubprocessTerminalHandle = { + ...failedTerminal, + terminate: vi.fn(), + waitForExit: vi.fn(() => cleanup), + } const terminals = (service as unknown as { terminals: Set }).terminals - terminals.add(terminal) + terminals.add(failedTerminal) + terminals.add(drainingTerminal) - await fiber.dispose() - expect(terminals).toEqual(new Set([terminal])) + let disposed = false + const disposing = fiber.dispose().then(() => { disposed = true }) + await new Promise(resolve => setImmediate(resolve)) + expect(disposed).toBe(false) + finishCleanup() + await disposing + expect(terminals).toEqual(new Set([failedTerminal])) expect((await stat(runtimeRoot)).isDirectory()).toBe(true) await rm(runtimeRoot, { recursive: true, force: true }) }) From 4d63552d009aad6e09c06f87c3a68e1b2a0e10ac Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 10:17:45 +0800 Subject: [PATCH 031/108] fix(lsp): preserve workspace cancellation reason --- packages/lsp/lsp-local/src/host.ts | 5 ++++- packages/lsp/lsp-local/tests/host.spec.ts | 20 ++++++++++++++++++++ 2 files changed, 24 insertions(+), 1 deletion(-) diff --git a/packages/lsp/lsp-local/src/host.ts b/packages/lsp/lsp-local/src/host.ts index 3a2b9fbd7b..55e13ed307 100644 --- a/packages/lsp/lsp-local/src/host.ts +++ b/packages/lsp/lsp-local/src/host.ts @@ -42,7 +42,10 @@ export async function canonicalizeWorkspace( throw new Error(`workspace root "${workspaceRoot}" cannot be resolved: ${messageOf(error)}`, { cause: error }) } throwIfAborted(signal) - const info = await fs.stat(target, signal) + const info = await fs.stat(target, signal).catch((error: unknown) => { + throwIfAborted(signal) + throw error + }) throwIfAborted(signal) if (info?.type !== 'directory') { throw new Error(`workspace root "${workspaceRoot}" is not a directory`) diff --git a/packages/lsp/lsp-local/tests/host.spec.ts b/packages/lsp/lsp-local/tests/host.spec.ts index cd7ed87e17..3db9deb50a 100644 --- a/packages/lsp/lsp-local/tests/host.spec.ts +++ b/packages/lsp/lsp-local/tests/host.spec.ts @@ -57,11 +57,31 @@ describe('canonicalizeWorkspace', () => { await expect(canonicalizeWorkspace(fs, join(root, 'nope'))).rejects.toThrow(/not a directory/) }) + it('wraps a provider failure while resolving the workspace', async () => { + fs.resolve = async () => { throw 'raw workspace resolve failure' } + await expect(canonicalizeWorkspace(fs, ws)) + .rejects.toThrow(`workspace root "${ws}" cannot be resolved: raw workspace resolve failure`) + }) + it('rejects a non-directory workspace', async () => { const file = join(root, 'file.txt') await writeFile(file, 'x') await expect(canonicalizeWorkspace(fs, file)).rejects.toThrow(/not a directory/) }) + + it('normalizes workspace metadata cancellation and preserves other provider failures', async () => { + const providerFailure = new Error('workspace metadata failed') + fs.stat = async () => { throw providerFailure } + await expect(canonicalizeWorkspace(fs, ws)).rejects.toBe(providerFailure) + + const controller = new AbortController() + fs.stat = async () => { + controller.abort(new Error('workspace metadata cancelled')) + throw providerFailure + } + await expect(canonicalizeWorkspace(fs, ws, controller.signal)) + .rejects.toThrow('workspace metadata cancelled') + }) }) describe('readHostSource', () => { From 0fec058a483a1dae658250878c149b4fafa04f65 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 10:52:37 +0800 Subject: [PATCH 032/108] fix(pty): retain send ownership through cancellation --- ...rtable-execution-world-consumers.i18n.yaml | 4 +- ...7-28-portable-execution-world-consumers.md | 2 +- ...8-portable-execution-world-consumers.zh.md | 2 +- ...26-07-16-persistent-pty-sessions.i18n.yaml | 4 +- .../2026-07-16-persistent-pty-sessions.md | 2 +- .../2026-07-16-persistent-pty-sessions.zh.md | 2 +- packages/pty/pty-local/README.i18n.yaml | 4 +- packages/pty/pty-local/README.md | 2 +- packages/pty/pty-local/README.zh.md | 2 +- packages/pty/pty-local/src/session.ts | 24 +++++++-- packages/pty/pty-local/tests/session.spec.ts | 54 +++++++++++++++++++ 11 files changed, 87 insertions(+), 15 deletions(-) diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml index 30e672416a..02545da311 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md -2026-07-28-portable-execution-world-consumers.md: ccb7a13a9677c106976efbe95629d0d2c027b982 -2026-07-28-portable-execution-world-consumers.zh.md: 0a7b840049a826a47f4dc0476e5991bfd0578970 +2026-07-28-portable-execution-world-consumers.md: 715529e5f24df0c2fe0afc40716737285afa223b +2026-07-28-portable-execution-world-consumers.zh.md: a9d864b581bd546c12776152ad4515816ee6b83a diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md index ccb7a13a96..715529e5f2 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md @@ -22,7 +22,7 @@ Generic consumers use that execution world: - `dsh-bash-local` continues to map Bash semantics onto ordinary `ctx.subprocess.spawn()`. - `dsh-lsp-local` reads and contains source through `ctx.fs`, resolves and launches language servers through `ctx.subprocess`, and carries provider-owned file URIs through initialization and result rendering. Its JSON-RPC, pooling, synchronization, cancellation, and normalization stay unchanged. -- `dsh-pty-local` maps persistent-shell semantics onto `ctx.subprocess.spawnTerminal()`. The local `node-pty` and process-inspection implementation moves into `dsh-subprocess-local`; another subprocess provider supplies the same primitive. The allocation signal is detached before publication, while readiness initialization retains setup cancellation. Prompt and silence evidence collected during asynchronous pre-write inspection is discarded when the provider write begins. A timed-out asynchronous write, or a cancellation whose foreground signal fails during that write, retains the send reservation until the provider settles it; completion of a stale inspection resumes polling for the current send. +- `dsh-pty-local` maps persistent-shell semantics onto `ctx.subprocess.spawnTerminal()`. The local `node-pty` and process-inspection implementation moves into `dsh-subprocess-local`; another subprocess provider supplies the same primitive. The allocation signal is detached before publication, while readiness initialization retains setup cancellation. Prompt and silence evidence collected during asynchronous pre-write inspection is discarded when the provider write begins. Cancellation retains the send reservation until asynchronous foreground signalling settles, so the signal cannot target a successor; a timed-out asynchronous write, or a signal failure during that write, retains the reservation until the provider settles the write. Completion of a stale inspection resumes polling for the current send. - `dsh-code-runtime-subprocess` materializes a dependency-free runner through `ctx.fs` and launches it through `ctx.subprocess`, preserving the Code Runtime binding and output contract across local or remote worlds. It shares host-side worker mechanics through the non-plugin `dsh-code-runtime-worker/runtime-host` subpath instead of copying them. Preparation carries one lifecycle signal through filesystem resolution, materialization, and executable lookup so disposal can abort a stalled provider. The heap-bounded worker rejects oversized binding frames before transfer, each outer hop enforces the same bound before serialization, and the launcher publishes an accepted terminal frame before reaping its controller so a descendant that inherits controller pipes cannot suppress completion; the host still awaits process-group quiescence. `dsh-code-runtime-worker` remains a separate implementation. It is the smaller in-process backend and works in single-file distributions that cannot assume an installed Node executable. Remote filesystem/process compositions select `dsh-code-runtime-subprocess`; they do not need a provider-specific Code Runtime package. diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md index 0a7b840049..a9d864b581 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md @@ -22,7 +22,7 @@ Status: implemented - `dsh-bash-local` 继续把 Bash 语义映射到普通的 `ctx.subprocess.spawn()`。 - `dsh-lsp-local` 通过 `ctx.fs` 读取源文件并验证包含关系,通过 `ctx.subprocess` 解析和启动语言服务器,并让由提供方负责的文件 URI 贯穿初始化与结果渲染。其 JSON-RPC、池化、同步、取消和规范化保持不变。 -- `dsh-pty-local` 把持久 shell 语义映射到 `ctx.subprocess.spawnTerminal()`。本地 `node-pty` 与进程检查实现移入 `dsh-subprocess-local`;其他进程管理提供方则提供相同原语。分配信号会在发布前解除关联,而就绪初始化仍保留设置阶段的取消。提供方开始写入时,系统会丢弃异步写入前检查期间收集的提示符与静默证据。异步写入超时,或在该写入期间取消时前台信号发送失败,都会保留发送预留,直至提供方将写入结算;陈旧检查完成后,会针对当前发送恢复轮询。 +- `dsh-pty-local` 把持久 shell 语义映射到 `ctx.subprocess.spawnTerminal()`。本地 `node-pty` 与进程检查实现移入 `dsh-subprocess-local`;其他进程管理提供方则提供相同原语。分配信号会在发布前解除关联,而就绪初始化仍保留设置阶段的取消。提供方开始写入时,系统会丢弃异步写入前检查期间收集的提示符与静默证据。取消会保留发送预留,直至异步前台信号发送结算,从而使该信号无法把后续发送作为目标;异步写入超时,或该写入期间发生信号发送失败,都会保留预留,直至提供方将写入结算。陈旧检查完成后,会针对当前发送恢复轮询。 - `dsh-code-runtime-subprocess` 通过 `ctx.fs` 物化无依赖 runner,并通过 `ctx.subprocess` 启动它,从而在本地或远程执行世界中保留代码运行时的绑定与输出契约。它通过非插件子路径 `dsh-code-runtime-worker/runtime-host` 共享宿主侧 worker 机制,而不是复制这些机制。准备阶段让同一个生命周期信号贯穿文件系统解析、物化和可执行文件查找,使资源释放能够中止停滞的提供方操作。受堆上限约束的 worker 会在传输前拒绝过大的绑定帧;每个外层转发环节都会在序列化前执行相同的上限检查;launcher 会在回收 controller 前发布已接纳的终态帧,使继承 controller 管道的后代进程无法阻止完成;宿主仍会等待进程组完全停稳。 `dsh-code-runtime-worker` 仍是独立实现。它是较小的进程内后端,可用于无法假定已安装 Node 可执行文件的单文件分发。远程文件系统/进程组合选择 `dsh-code-runtime-subprocess`;它们不需要提供方专用的代码运行时包。 diff --git a/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.i18n.yaml b/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.i18n.yaml index 8b33466ea7..2694664183 100644 --- a/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.i18n.yaml +++ b/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md -2026-07-16-persistent-pty-sessions.md: ef87c2806a237e6e6d44c1e62942146af24b8c19 -2026-07-16-persistent-pty-sessions.zh.md: 5f1bde39de5eca7aa57897d87c95675ce0da5abd +2026-07-16-persistent-pty-sessions.md: f94dd57622c555e195564cc2eee9b3228841d5cc +2026-07-16-persistent-pty-sessions.zh.md: 6228571a6496e9d4cb031a2df958b80dee102fd3 diff --git a/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md b/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md index ef87c2806a..f94dd57622 100644 --- a/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md +++ b/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md @@ -60,7 +60,7 @@ The local subprocess terminal primitive uses only public `node-pty` capabilities The UI render contract is exact and location-free. `terminal_send` uses terminal call/result cards only for foreground sends; its background form is generic `execute`. `terminal_open`, `terminal_read`, `terminal_signal`, `terminal_close`, and `terminal_list` use generic `execute`, `read`, `execute`, `delete`, and `read` cards respectively. No PTY tool emits `locations`. -`terminal_send({ sessionId, text, submit?, run_in_background? })` treats `text` as UTF-8 bytes and resolves `submit` to `true` in the tool implementation. When `submit` is true it writes the platform Enter sequence after the text; when false it writes only the text, allowing control characters and REPL fragments without hidden content heuristics. Cancellation marks queued input before signaling the real foreground group, so input cannot execute if an asynchronous pre-write inspection settles afterward. `enableRunInBackground` defaults to true; false removes `run_in_background` from the schema and rejects the same undeclared argument if a caller forces it through execution. +`terminal_send({ sessionId, text, submit?, run_in_background? })` treats `text` as UTF-8 bytes and resolves `submit` to `true` in the tool implementation. When `submit` is true it writes the platform Enter sequence after the text; when false it writes only the text, allowing control characters and REPL fragments without hidden content heuristics. Cancellation marks queued input before signaling the real foreground group, so input cannot execute if an asynchronous pre-write inspection settles afterward. The canceled send retains its reservation until asynchronous foreground signalling settles, so a successor cannot become that signal's target. `enableRunInBackground` defaults to true; false removes `run_in_background` from the schema and rejects the same undeclared argument if a caller forces it through execution. Foreground sends return a bounded rendered delta and two independent facts: `waitReason` (`stdin_read | inferred_idle | timeout | session_exit`) and `sessionStatus` (`running` or `exited` with exit code or signal). `session_exit` refers to the PTY's top-level shell process, not an arbitrary foreground command whose status the shell consumes. A timeout never implies process exit. `dsh-tool-pty.maxResultBytes` defaults to 262144, rejects values below 64 so creation acknowledgements retain registry-issued ids, and caps each single-text UTF-8 result after normalized tool or pipeline errors, wait, session, pagination, truncation, generic task-status wrappers, policy denials or short-circuits, and post-execute replacements or blocks; the terminal definitions' last-mile `finalizeContent` callback leaves deliberately structured multi-block policy content unchanged. The renderer reserves suffix space and preserves code-point boundaries instead of treating the backend payload cap as the final model bound. diff --git a/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.zh.md b/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.zh.md index 5f1bde39de..6228571a64 100644 --- a/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.zh.md +++ b/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.zh.md @@ -60,7 +60,7 @@ agent scope dispose 时先关闭注册,再等待全部所属 PTY 静默退出 UI 渲染契约精确且不携带位置信息。`terminal_send` 只为前台发送使用 terminal 调用卡片和结果卡片;后台形式使用通用 `execute` 卡片。`terminal_open`、`terminal_read`、`terminal_signal`、`terminal_close` 和 `terminal_list` 分别使用通用 `execute`、`read`、`execute`、`delete` 和 `read` 卡片。所有 PTY 工具都不发出 `locations`。 -`terminal_send({ sessionId, text, submit?, run_in_background? })` 将 `text` 视为 UTF-8 字节,并由工具实现在解析阶段把 `submit` 默认成 `true`。`submit` 为 true 时先写入文本,再写入平台 Enter 序列;为 false 时只写文本,使控制字符和 REPL 片段无需隐藏的内容启发式即可发送。取消会在向真实前台进程组发送信号前将排队输入标记为已取消,因此即使异步的写入前检查随后才结算,该输入也无法执行。`enableRunInBackground` 默认为 true;设为 false 时,schema 中会移除 `run_in_background`,调用方即使强行把这个未声明参数传入执行流程,也会被拒绝。 +`terminal_send({ sessionId, text, submit?, run_in_background? })` 将 `text` 视为 UTF-8 字节,并由工具实现在解析阶段把 `submit` 默认成 `true`。`submit` 为 true 时先写入文本,再写入平台 Enter 序列;为 false 时只写文本,使控制字符和 REPL 片段无需隐藏的内容启发式即可发送。取消会在向真实前台进程组发送信号前将排队输入标记为已取消,因此即使异步的写入前检查随后才结算,该输入也无法执行。被取消的发送会保留其预留,直至异步前台信号发送结算,因此后续发送不会成为该信号的目标。`enableRunInBackground` 默认为 true;设为 false 时,schema 中会移除 `run_in_background`,调用方即使强行把这个未声明参数传入执行流程,也会被拒绝。 前台发送返回有界的渲染增量和两个独立事实:`waitReason`(`stdin_read | inferred_idle | timeout | session_exit`)与 `sessionStatus`(`running`,或携带退出码或信号的 `exited`)。`session_exit` 指 PTY 顶层 shell 进程退出,不指由 shell 消费状态的任意前台命令。timeout 从不意味着进程已经退出。`dsh-tool-pty.maxResultBytes` 默认为 262144;低于 64 的值会被拒绝,以确保创建确认保留 registry 签发的 id;每个单文本 UTF-8 结果在加入规范化的工具或流水线错误、等待、会话、分页、截断、通用 task 状态包装、策略拒绝或短路以及 post-execute 替换或阻断后,仍受该值限制;终端定义自有的末端 `finalizeContent` callback 会原样保留策略刻意返回的结构化多 block 内容。渲染器会为后缀预留空间并保持代码点边界,而不会把后端载荷上限当作面向模型结果的最终上限。 diff --git a/packages/pty/pty-local/README.i18n.yaml b/packages/pty/pty-local/README.i18n.yaml index 4c153eff47..c89122066a 100644 --- a/packages/pty/pty-local/README.i18n.yaml +++ b/packages/pty/pty-local/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/pty/pty-local/README.md -README.md: 74dfd7f59d45649f6daf0b42408285d9ede830a7 -README.zh.md: 2c7780f7e7db896c959e017dc39299e0e16f187d +README.md: 7e31adfbc486a190252be867e2113948b17e7924 +README.zh.md: eb628aef9ad6b0fea14a5e833197b4fac7856785 diff --git a/packages/pty/pty-local/README.md b/packages/pty/pty-local/README.md index 74dfd7f59d..7e31adfbc4 100644 --- a/packages/pty/pty-local/README.md +++ b/packages/pty/pty-local/README.md @@ -10,7 +10,7 @@ The plugin injects `pty`, `sandbox`, `sandboxPolicy`, and `subprocess`, then reg Readiness combines a foreground-verified private bash prompt marker, provider-reported foreground stdin-wait facts, silence fallback, and absolute timeout. A marker is not ready until the printable tail after the latest owned marker exactly equals the controlled `PS1`, including when the OSC marker and prompt are split across data callbacks; echoed input or output following a delayed earlier prompt therefore cannot settle the current send. Prompt and silence evidence collected before the provider write, including while pre-write foreground inspection is pending, is discarded at the write boundary. When bash prints the marker before the terminal provider publishes its return to the foreground process group, polling retains the candidate for `handoffGraceMs` past the ordinary silence bound so a coincident handoff can win. An interactive child that inherits `PROMPT_COMMAND` therefore cannot suppress inferred-idle readiness until the absolute timeout. Unknown foreground state is never a positive exact-idle signal. A foreground group's stdin wait that existed before a send is likewise not post-write readiness: the same group must be observed outside that wait before a later wait can settle the send, while a changed foreground group is new evidence. During unpublished startup, a fallback requires observed output; zero-output silence cannot publish an empty session, and timeout rejects the spawn. Cancellation closes the unpublished shell and rejects with the caller's exact abort reason; `PtyBackendCleanupError` separately preserves a cleanup failure. The terminal-allocation signal is detached when allocation returns, while readiness initialization keeps the setup signal, so later cancellation cannot terminate a published persistent session. Incomplete terminal-control sequences are bounded by `maxReadBytes` and discarded through their terminator after crossing that limit; a trailing carriage return is carried across callbacks so split CRLF becomes one newline. -Send cancellation marks queued input as canceled before asking the terminal handle to signal the current foreground process group with a real `SIGINT`; if asynchronous pre-write inspection later settles, it cannot execute that input. Cancellation never emulates interruption by writing `\x03`, so raw-mode programs remain cancellable. A send that times out during an asynchronous provider write, or whose cancellation signal fails while that write remains pending, reports its result but retains the session's send slot until the write settles, so late bytes cannot interleave with a successor. Close starts provider-owned TERM-to-KILL whole-session cleanup and awaits quiescence after the terminal outcome. A cleanup failure does not cache a permanently rejected close; a later close retries the provider operation. +Send cancellation marks queued input as canceled before asking the terminal handle to signal the current foreground process group with a real `SIGINT`; if asynchronous pre-write inspection later settles, it cannot execute that input. The canceled send retains its slot until foreground signalling settles, so a successor cannot become that signal's target. Cancellation never emulates interruption by writing `\x03`, so raw-mode programs remain cancellable. A send that times out during an asynchronous provider write, or whose cancellation signal fails while that write remains pending, reports its result but retains the slot until the write settles, so late bytes cannot interleave with a successor. Close starts provider-owned TERM-to-KILL whole-session cleanup and awaits quiescence after the terminal outcome. A cleanup failure does not cache a permanently rejected close; a later close retries the provider operation. ## Model Experience diff --git a/packages/pty/pty-local/README.zh.md b/packages/pty/pty-local/README.zh.md index 2c7780f7e7..eb628aef9a 100644 --- a/packages/pty/pty-local/README.zh.md +++ b/packages/pty/pty-local/README.zh.md @@ -10,7 +10,7 @@ 就绪检测结合以下机制:由前台状态验证的私有 bash 提示符标记、提供方报告的前台 stdin 等待事实、静默回退和绝对超时。只有最近一个自有标记之后的可打印尾部与受控 `PS1` 完全相等时,系统才会把标记视为就绪;即使 OSC 标记和提示符被拆到多个数据回调中也是如此。因此,如果回显的输入或输出跟在延迟到达的先前提示符之后,该提示符无法使当前 send 完成。系统会在写入边界丢弃提供方写入前收集的提示符与静默证据,包括写入前的前台检查尚未完成时收集的证据。如果 bash 在终端提供方发布其重新取得前台进程组的状态前打印标记,轮询会在普通静默上限之后再保留该候选状态 `handoffGraceMs`,使恰好同时发生的前台交接有机会胜出。因此,继承 `PROMPT_COMMAND` 的交互式子进程无法持续压制推断空闲就绪,最多只能延续到绝对超时。未知的前台状态绝不会作为精确空闲的正向信号。同样,一次 send 之前就已存在的前台进程组 stdin 等待并不代表写入后就绪:必须先观察到同一进程组脱离该等待,之后再次进入等待才能使该次 send 完成;前台进程组发生变化则构成新的证据。尚未发布的启动过程中,回退路径要求已经观察到输出;零输出静默不能发布空会话,超时则拒绝 spawn。取消操作会关闭尚未发布的 shell,并以调用方提供的确切中止原因拒绝;`PtyBackendCleanupError` 会单独保留清理失败。终端分配返回时,分配信号会解除关联;就绪初始化则保留设置阶段信号,因此后续取消无法终止已发布的持久会话。未完成的终端控制序列受 `maxReadBytes` 限制;超过上限后,系统会丢弃内容直到其终止符。末尾的回车会跨回调保留,使拆分的 CRLF 合并为一个换行。 -取消发送会先把排队输入标记为已取消,再请求终端句柄向当前前台进程组发送真正的 `SIGINT`;如果异步的写入前检查随后才结算,也无法执行该输入。取消绝不会通过写入 `\x03` 模拟中断,因此原始模式程序仍可取消。发送在提供方异步写入期间超时,或在该写入仍未完成时其取消信号发送失败,都会报告各自结果,但继续占用会话的发送槽位,直至写入结算,从而避免延迟到达的字节与后续发送交错。关闭操作启动由提供方负责的 TERM→KILL 全会话清理,并在终端结果之后等待完全停稳。清理失败不会缓存成永久拒绝的关闭操作;后续关闭会重试提供方操作。 +取消发送会先把排队输入标记为已取消,再请求终端句柄向当前前台进程组发送真正的 `SIGINT`;如果异步的写入前检查随后才结算,也无法执行该输入。被取消的发送会保留其槽位,直至前台信号发送结算,因此后续发送不会成为该信号的目标。取消绝不会通过写入 `\x03` 模拟中断,因此原始模式程序仍可取消。发送在提供方异步写入期间超时,或在该写入仍未完成时其取消信号发送失败,都会报告各自结果,但继续占用该槽位,直至写入结算,从而避免延迟到达的字节与后续发送交错。关闭操作启动由提供方负责的 TERM→KILL 全会话清理,并在终端结果之后等待完全停稳。清理失败不会缓存成永久拒绝的关闭操作;后续关闭会重试提供方操作。 ## 模型体验 diff --git a/packages/pty/pty-local/src/session.ts b/packages/pty/pty-local/src/session.ts index 267c7c17fc..922db21152 100644 --- a/packages/pty/pty-local/src/session.ts +++ b/packages/pty/pty-local/src/session.ts @@ -166,6 +166,7 @@ export class LocalPtySession implements PtyBackendSession { private activeDeadlineTimer: NodeJS.Timeout | undefined private activeAbort: (() => void) | undefined private readonly terminalOperations = new Set>() + private interrupting: LocalSendOperation | undefined private writing: LocalSendOperation | undefined private pollingReady: LocalSendOperation | undefined private polling = false @@ -403,7 +404,7 @@ export class LocalPtySession implements PtyBackendSession { } private schedulePoll(operation: LocalSendOperation, delayMs = this.config.pollIntervalMs): void { - if (this.active !== operation || this.polling) return + if (this.active !== operation || this.interrupting === operation || this.polling) return if (this.activeTimer !== undefined) clearTimeout(this.activeTimer) this.activeTimer = setTimeout(() => { this.activeTimer = undefined @@ -480,9 +481,11 @@ export class LocalPtySession implements PtyBackendSession { } private clearActive(): void { + const operation = this.active this.stopPolling() this.activeAbort?.() this.activeAbort = undefined + if (this.interrupting === operation) this.interrupting = undefined this.writing = undefined this.pollingReady = undefined this.active = undefined @@ -503,9 +506,24 @@ export class LocalPtySession implements PtyBackendSession { private interrupt(operation: LocalSendOperation): void { if (this.active !== operation) return - this.ownTerminalOperation(this.terminal.signalForeground('SIGINT').then(() => {}, (error: unknown) => { + this.interrupting = operation + this.stopPolling() + this.ownTerminalOperation(this.interruptOnce(operation)) + } + + private async interruptOnce(operation: LocalSendOperation): Promise { + try { + await this.terminal.signalForeground('SIGINT') + } catch (error: unknown) { if (this.active === operation) this.failActive(error, this.writing === operation) - })) + return + } finally { + if (this.interrupting === operation) this.interrupting = undefined + } + if (this.active === operation && !operation.settled && !this.closing && this.writing !== operation) { + this.pollingReady = operation + this.schedulePoll(operation, 0) + } } private async closeOnce(reason: string): Promise { diff --git a/packages/pty/pty-local/tests/session.spec.ts b/packages/pty/pty-local/tests/session.spec.ts index cce384bf6b..513dcbbbb4 100644 --- a/packages/pty/pty-local/tests/session.spec.ts +++ b/packages/pty/pty-local/tests/session.spec.ts @@ -340,6 +340,60 @@ describe('LocalPtySession readiness and output', () => { await operation.done }) + it('retains a canceled send until asynchronous foreground signalling settles', async () => { + vi.useFakeTimers() + const terminal = new FakeTerminal() + const inspector = new FakeInspector() + const session = makeSession(terminal, inspector, config()) + await initialize(session, terminal) + + const signalGate = Promise.withResolvers() + terminal.signalForeground = async (signal) => { + await signalGate.promise + const foreground = await terminal.inspectForeground() + if (foreground === undefined) throw new Error('cannot resolve foreground') + inspector.signalGroup(foreground.processGroupId, signal) + return foreground.processGroupId + } + const operation = session.startSend({ text: 'first', submit: true }) + await Promise.resolve() + await Promise.resolve() + expect(operation.cancel()).toBe(true) + + terminal.emitData('\x1b]133;D;130\x07dsh> ') + await vi.advanceTimersByTimeAsync(10) + expect(() => session.startSend({ text: 'successor', submit: true })).toThrow('active send') + signalGate.resolve(undefined) + await vi.advanceTimersByTimeAsync(10) + await operation.done + expect(inspector.groups).toContainEqual([456, 'SIGINT']) + expect(inspector.groups).not.toContainEqual([789, 'SIGINT']) + }) + + it('does not resume cancellation polling after the terminal exits during signalling', async () => { + vi.useFakeTimers() + const terminal = new FakeTerminal() + const inspector = new FakeInspector() + const session = makeSession(terminal, inspector, config()) + await initialize(session, terminal) + + const signalGate = Promise.withResolvers() + terminal.signalForeground = async () => { + await signalGate.promise + return 456 + } + const operation = session.startSend({ text: 'first', submit: true }) + await Promise.resolve() + await Promise.resolve() + expect(operation.cancel()).toBe(true) + terminal.emitExit(0) + await expect(operation.done).resolves.toMatchObject({ waitReason: 'session_exit' }) + + signalGate.resolve(undefined) + await vi.advanceTimersByTimeAsync(10) + expect(session.status()).toEqual({ kind: 'exited', exitCode: 0, signal: null }) + }) + it('retains send ownership after timeout until an asynchronous provider write settles', async () => { vi.useFakeTimers() const terminal = new FakeTerminal() From be3533fa19438cea98daad5727eb1a21e44a3f03 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 10:53:27 +0800 Subject: [PATCH 033/108] fix(subprocess): clean runtime state after teardown failure --- .../subprocess-local/README.i18n.yaml | 4 +- .../subprocess/subprocess-local/README.md | 2 +- .../subprocess/subprocess-local/README.zh.md | 2 +- .../subprocess/subprocess-local/src/index.ts | 14 +++-- .../subprocess-local/tests/local.spec.ts | 53 ++++++++++++++++--- 5 files changed, 60 insertions(+), 15 deletions(-) diff --git a/packages/subprocess/subprocess-local/README.i18n.yaml b/packages/subprocess/subprocess-local/README.i18n.yaml index 72f3c93da9..efbb1612bd 100644 --- a/packages/subprocess/subprocess-local/README.i18n.yaml +++ b/packages/subprocess/subprocess-local/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/subprocess/subprocess-local/README.md -README.md: 38b26c33f3854acdb4a2bb48612da73361d727ba -README.zh.md: fc4a5d4961fb7618b9db84b21063f907361a26d8 +README.md: 61ee8ff9e09d5177ff6d5b3805dd84fb74c59961 +README.zh.md: 06a8e4bdd225e212aa67817ebbf66581314ab2bf diff --git a/packages/subprocess/subprocess-local/README.md b/packages/subprocess/subprocess-local/README.md index 38b26c33f3..61ee8ff9e0 100644 --- a/packages/subprocess/subprocess-local/README.md +++ b/packages/subprocess/subprocess-local/README.md @@ -10,7 +10,7 @@ Local implementation of the [`@deepseek-ai/dsh-subprocess`](../subprocess/README - **Per-stream dispositions** — `'pipe'` hands the raw stream to the caller untouched (protocol framing stays consumer-owned); `'inherit'` passes the parent descriptor through; collect mode keeps the in-memory TAIL beyond its cap (errors and results cluster at the end — pi/OpenCode rationale) while the FULL stream is appended to a private temp file when a spill cap is configured — omitting `spill` keeps only the tail, the diagnostic shape. A stream larger than the spill cap discards its now-incomplete spill and returns only the marked truncated tail; spill fds are sealed at settlement, and a failed final close withholds the path rather than advertising an incomplete file. Spill files are `0600` with random names under a lazily-created `0700` per-process directory. - **Credential scrub + explicit merge** — `process.env` minus credential-shaped vars (`*KEY*`/`*SECRET*`/`*TOKEN*`) and all ambient `DSH_*` names; the spec's explicit `env` merges after that scrub with no namespace validation, so a deliberately supplied credential or current `DSH_*` fact wins while stale nested-harness identity cannot leak in ambiently. Supplied stdin is written and closed; otherwise fd 0 is `/dev/null`. See the [stdin/env Agent Note](../../../.agents/notes/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-surface.md) and [managed environment Agent Note](../../../.agents/notes/implemented/feature/2026-07-10-agent-session-identity-and-log-location.md). - **Offset-based reads** — collect-mode readers return deltas in whole-stream byte coordinates; the service never holds a cursor, so consumer-owned cursors (the bash background read path) and full-stream re-reads coexist, before and after settlement. -- **Execution-world coordinates** — `cwd` is the host process cwd, `runtimeRoot` is an owner-private temporary directory removed on disposal, and `resolveExecutable` checks absolute files or searches the scrubbed effective PATH with platform-aware executable extensions. +- **Execution-world coordinates** — `cwd` is the host process cwd, `runtimeRoot` is an owner-private temporary directory removed on disposal before any process-cleanup failure is reported, and `resolveExecutable` checks absolute files or searches the scrubbed effective PATH with platform-aware executable extensions. - **Terminal-process ownership** — `spawnTerminal` allocates `node-pty`, bridges UTF-8 terminal bytes, inspects and signals the current foreground process group, and cleans descendants before the top-level shell. Each foreground inspection retains exact identities from the rooted tree; Linux also enumerates the POSIX session after its leader exits. A previously observed macOS descendant and any same-session Linux member therefore remain fenced after reparenting, while pid/start identity prevents cleanup from following PID reuse. The higher PTY backend owns prompt readiness, buffers, and model-facing operations. - **Terminate-and-join disposal** — the service retains live handles only so its own disposal can escalate every running tree and await its exit; settled and spawn-failed handles leave the live set on settlement. diff --git a/packages/subprocess/subprocess-local/README.zh.md b/packages/subprocess/subprocess-local/README.zh.md index fc4a5d4961..06a8e4bdd2 100644 --- a/packages/subprocess/subprocess-local/README.zh.md +++ b/packages/subprocess/subprocess-local/README.zh.md @@ -10,7 +10,7 @@ - **按流划分的处置方式**:`'pipe'` 把原始流原样交给调用方(协议分帧仍归消费方所有);`'inherit'` 直通父进程的描述符;收集模式(collect)在输出超过上限后于内存中保留尾部(错误与结果通常聚集在末尾,沿用 pi/OpenCode 的理由),并在配置了 spill 上限时把完整流追加到一个私有临时文件;省略 `spill` 则只保留尾部,即诊断尾部的形状。某条流大于 spill 上限时,会丢弃已不完整的 spill,仅返回带截断标记的尾部;spill 文件描述符在结算时封存,最终关闭失败时则不公布路径,以免声称存在不完整的文件。spill 文件权限为 `0600`、名称随机,位于按需延迟创建的 `0700` 每进程目录之下。 - **凭据清除 + 显式合并**:以 `process.env` 为基础,移除形似凭据的变量(`*KEY*`/`*SECRET*`/`*TOKEN*`)和所有环境中已有的 `DSH_*` 名称;spec 的显式 `env` 在该清除之后合并且不做命名空间校验,因此有意提供的凭据或当前 `DSH_*` 事实会胜出,而陈旧的嵌套 harness 身份无法从环境中隐式漏入。提供的 stdin 会被写入后关闭;否则 fd 0 指向 `/dev/null`。参见 [stdin/env Agent Note(agent 决策记录)](../../../.agents/notes/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-surface.md)与[受管环境 Agent Note](../../../.agents/notes/implemented/feature/2026-07-10-agent-session-identity-and-log-location.md)。 - **基于偏移量的读取**:收集模式的读取器以全流字节坐标返回增量;服务自身从不持有游标,因此消费方自有的游标(bash 的后台读取路径)与完整流重读可以共存,结算前后皆然。 -- **执行世界坐标**:`cwd` 是宿主进程 cwd,`runtimeRoot` 是所有者私有的临时目录,在资源释放时删除;`resolveExecutable` 检查绝对文件,或使用平台感知的可执行扩展名在清理后的有效 PATH 中查找。 +- **执行世界坐标**:`cwd` 是宿主进程 cwd,`runtimeRoot` 是所有者私有的临时目录,会在资源释放时删除,并且删除发生在报告任何进程清理失败之前;`resolveExecutable` 检查绝对文件,或使用平台感知的可执行扩展名在清理后的有效 PATH 中查找。 - **终端进程所有权**:`spawnTerminal` 分配 `node-pty`,桥接 UTF-8 终端字节,检查当前前台进程组并向其发送信号,并先于顶层 shell 清理后代。每次前台检查都会保留有根进程树中的精确身份;Linux 还会在会话 leader 退出后枚举该 POSIX 会话。因此,先前观察到的 macOS 后代以及任何同会话 Linux 成员在重新设定父进程后仍受身份围栏保护,而 pid/启动身份可防止清理因 PID 复用而跟随到其他进程。上层 PTY 后端负责提示符就绪检测、缓冲和面向模型的操作。 - **先终止再等待退出的 dispose**:服务保留存活句柄,只为让自身的 dispose 能对每个仍在运行的进程树执行升级并等待其退出;已结算与 spawn 失败的句柄在结算时即离开存活集合。 diff --git a/packages/subprocess/subprocess-local/src/index.ts b/packages/subprocess/subprocess-local/src/index.ts index 80d0b4c76b..0d71cb0e33 100644 --- a/packages/subprocess/subprocess-local/src/index.ts +++ b/packages/subprocess/subprocess-local/src/index.ts @@ -67,11 +67,15 @@ export class LocalSubprocessService extends SubprocessService { pending.push(terminal.waitForExit().then(() => { this.terminals.delete(terminal) })) } this.live.clear() - const outcomes = await Promise.allSettled(pending) - for (const outcome of outcomes) { - if (outcome.status === 'rejected') throw outcome.reason - } - await rm(this.runtimeRoot, { recursive: true, force: true }) + const outcomes = [ + ...await Promise.allSettled(pending), + ...await Promise.allSettled([rm(this.runtimeRoot, { recursive: true, force: true })]), + ] + const failures = outcomes.flatMap(outcome => outcome.status === 'rejected' + ? [outcome.reason as unknown] + : []) + if (failures.length === 1) throw failures[0] + if (failures.length > 1) throw new AggregateError(failures, 'local subprocess teardown failed') }, 'local subprocess teardown') } diff --git a/packages/subprocess/subprocess-local/tests/local.spec.ts b/packages/subprocess/subprocess-local/tests/local.spec.ts index 46f3b14bea..e4dd4cfb44 100644 --- a/packages/subprocess/subprocess-local/tests/local.spec.ts +++ b/packages/subprocess/subprocess-local/tests/local.spec.ts @@ -1,6 +1,6 @@ import { PassThrough } from 'node:stream' import { describe, expect, it, vi } from 'vitest' -import { rm, stat } from 'node:fs/promises' +import { stat } from 'node:fs/promises' import { basename, dirname, relative, resolve } from 'node:path' import { Context } from 'cordis' import LocalSubprocessService from '@deepseek-ai/dsh-subprocess-local' @@ -119,11 +119,15 @@ describe('LocalSubprocessService', () => { expect(terminals.size).toBe(0) }) - it('waits for every terminal cleanup and retains rejections', async () => { + it('waits for every terminal cleanup, removes runtime state, and retains rejections', async () => { const ctx = new Context() const fiber = await ctx.plugin(LocalSubprocessService) const service = ctx.subprocess const runtimeRoot = service.runtimeRoot + const firstFailure = new Error('first retryable cleanup failure') + const secondFailure = new Error('second retryable cleanup failure') + const disposalErrors: unknown[] = [] + ctx.logger.error = ((error: unknown) => { disposalErrors.push(error) }) as typeof ctx.logger.error const failedTerminal: SubprocessTerminalHandle = { pid: 1, output: new PassThrough(), @@ -132,7 +136,12 @@ describe('LocalSubprocessService', () => { inspectForeground: async () => undefined, signalForeground: async () => 1, terminate: vi.fn(), - waitForExit: vi.fn(async () => { throw new Error('retryable cleanup failure') }), + waitForExit: vi.fn(async () => { throw firstFailure }), + } + const secondFailedTerminal: SubprocessTerminalHandle = { + ...failedTerminal, + terminate: vi.fn(), + waitForExit: vi.fn(async () => { throw secondFailure }), } let finishCleanup!: () => void const cleanup = new Promise((resolve) => { @@ -145,6 +154,7 @@ describe('LocalSubprocessService', () => { } const terminals = (service as unknown as { terminals: Set }).terminals terminals.add(failedTerminal) + terminals.add(secondFailedTerminal) terminals.add(drainingTerminal) let disposed = false @@ -153,9 +163,40 @@ describe('LocalSubprocessService', () => { expect(disposed).toBe(false) finishCleanup() await disposing - expect(terminals).toEqual(new Set([failedTerminal])) - expect((await stat(runtimeRoot)).isDirectory()).toBe(true) - await rm(runtimeRoot, { recursive: true, force: true }) + expect(terminals).toEqual(new Set([failedTerminal, secondFailedTerminal])) + await expect(stat(runtimeRoot)).rejects.toMatchObject({ code: 'ENOENT' }) + expect(disposalErrors).toHaveLength(1) + expect(disposalErrors[0]).toMatchObject({ + errors: [firstFailure, secondFailure], + message: 'local subprocess teardown failed', + }) + }) + + it('reports one cleanup failure without wrapping it after removing runtime state', async () => { + const ctx = new Context() + const failure = new Error('single cleanup failure') + const disposalErrors: unknown[] = [] + ctx.logger.error = ((error: unknown) => { disposalErrors.push(error) }) as typeof ctx.logger.error + const fiber = await ctx.plugin(LocalSubprocessService) + const service = ctx.subprocess + const runtimeRoot = service.runtimeRoot + const terminal: SubprocessTerminalHandle = { + pid: 1, + output: new PassThrough(), + done: Promise.resolve({ exitCode: 0, signal: null }), + write: async () => {}, + inspectForeground: async () => undefined, + signalForeground: async () => 1, + terminate: vi.fn(), + waitForExit: vi.fn(async () => { throw failure }), + } + const terminals = (service as unknown as { terminals: Set }).terminals + terminals.add(terminal) + + await fiber.dispose() + + await expect(stat(runtimeRoot)).rejects.toMatchObject({ code: 'ENOENT' }) + expect(disposalErrors).toEqual([failure]) }) it('releases a terminal after top-level exit reaches quiescence', async () => { From d6b3de6e0da064f5b95e29a4134e30da39ab2d1a Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 11:57:12 +0800 Subject: [PATCH 034/108] fix(pty): fence delayed signal prompts --- ...rtable-execution-world-consumers.i18n.yaml | 4 +-- ...7-28-portable-execution-world-consumers.md | 2 +- ...8-portable-execution-world-consumers.zh.md | 2 +- packages/pty/pty-local/README.i18n.yaml | 4 +-- packages/pty/pty-local/README.md | 2 +- packages/pty/pty-local/README.zh.md | 2 +- packages/pty/pty-local/src/session.ts | 16 +++++++++- packages/pty/pty-local/tests/session.spec.ts | 29 +++++++++++++++++++ 8 files changed, 52 insertions(+), 9 deletions(-) diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml index 02545da311..803a2f83fe 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md -2026-07-28-portable-execution-world-consumers.md: 715529e5f24df0c2fe0afc40716737285afa223b -2026-07-28-portable-execution-world-consumers.zh.md: a9d864b581bd546c12776152ad4515816ee6b83a +2026-07-28-portable-execution-world-consumers.md: 54b8ec19bc74e734f2cb940a475d2c7b4c9336fd +2026-07-28-portable-execution-world-consumers.zh.md: f21be00fd35267fe93e2fbb62642809cc42c0742 diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md index 715529e5f2..54b8ec19bc 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md @@ -22,7 +22,7 @@ Generic consumers use that execution world: - `dsh-bash-local` continues to map Bash semantics onto ordinary `ctx.subprocess.spawn()`. - `dsh-lsp-local` reads and contains source through `ctx.fs`, resolves and launches language servers through `ctx.subprocess`, and carries provider-owned file URIs through initialization and result rendering. Its JSON-RPC, pooling, synchronization, cancellation, and normalization stay unchanged. -- `dsh-pty-local` maps persistent-shell semantics onto `ctx.subprocess.spawnTerminal()`. The local `node-pty` and process-inspection implementation moves into `dsh-subprocess-local`; another subprocess provider supplies the same primitive. The allocation signal is detached before publication, while readiness initialization retains setup cancellation. Prompt and silence evidence collected during asynchronous pre-write inspection is discarded when the provider write begins. Cancellation retains the send reservation until asynchronous foreground signalling settles, so the signal cannot target a successor; a timed-out asynchronous write, or a signal failure during that write, retains the reservation until the provider settles the write. Completion of a stale inspection resumes polling for the current send. +- `dsh-pty-local` maps persistent-shell semantics onto `ctx.subprocess.spawnTerminal()`. The local `node-pty` and process-inspection implementation moves into `dsh-subprocess-local`; another subprocess provider supplies the same primitive. The allocation signal is detached before publication, while readiness initialization retains setup cancellation. Prompt and silence evidence collected during asynchronous pre-write inspection is discarded when the provider write begins. Cancellation retains the send reservation until asynchronous foreground signalling settles, so the signal cannot target a successor; if a signaled send returns `inferred_idle` before its prompt arrives, that marker remains attributed to the prior send instead of settling a successor after its echo. A timed-out asynchronous write, or a signal failure during that write, retains the reservation until the provider settles the write. Completion of a stale inspection resumes polling for the current send. - `dsh-code-runtime-subprocess` materializes a dependency-free runner through `ctx.fs` and launches it through `ctx.subprocess`, preserving the Code Runtime binding and output contract across local or remote worlds. It shares host-side worker mechanics through the non-plugin `dsh-code-runtime-worker/runtime-host` subpath instead of copying them. Preparation carries one lifecycle signal through filesystem resolution, materialization, and executable lookup so disposal can abort a stalled provider. The heap-bounded worker rejects oversized binding frames before transfer, each outer hop enforces the same bound before serialization, and the launcher publishes an accepted terminal frame before reaping its controller so a descendant that inherits controller pipes cannot suppress completion; the host still awaits process-group quiescence. `dsh-code-runtime-worker` remains a separate implementation. It is the smaller in-process backend and works in single-file distributions that cannot assume an installed Node executable. Remote filesystem/process compositions select `dsh-code-runtime-subprocess`; they do not need a provider-specific Code Runtime package. diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md index a9d864b581..f21be00fd3 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md @@ -22,7 +22,7 @@ Status: implemented - `dsh-bash-local` 继续把 Bash 语义映射到普通的 `ctx.subprocess.spawn()`。 - `dsh-lsp-local` 通过 `ctx.fs` 读取源文件并验证包含关系,通过 `ctx.subprocess` 解析和启动语言服务器,并让由提供方负责的文件 URI 贯穿初始化与结果渲染。其 JSON-RPC、池化、同步、取消和规范化保持不变。 -- `dsh-pty-local` 把持久 shell 语义映射到 `ctx.subprocess.spawnTerminal()`。本地 `node-pty` 与进程检查实现移入 `dsh-subprocess-local`;其他进程管理提供方则提供相同原语。分配信号会在发布前解除关联,而就绪初始化仍保留设置阶段的取消。提供方开始写入时,系统会丢弃异步写入前检查期间收集的提示符与静默证据。取消会保留发送预留,直至异步前台信号发送结算,从而使该信号无法把后续发送作为目标;异步写入超时,或该写入期间发生信号发送失败,都会保留预留,直至提供方将写入结算。陈旧检查完成后,会针对当前发送恢复轮询。 +- `dsh-pty-local` 把持久 shell 语义映射到 `ctx.subprocess.spawnTerminal()`。本地 `node-pty` 与进程检查实现移入 `dsh-subprocess-local`;其他进程管理提供方则提供相同原语。分配信号会在发布前解除关联,而就绪初始化仍保留设置阶段的取消。提供方开始写入时,系统会丢弃异步写入前检查期间收集的提示符与静默证据。取消会保留发送预留,直至异步前台信号发送结算,从而使该信号无法把后续发送作为目标;如果收到信号的 send 在其提示符到达前返回 `inferred_idle`,随后到达的标记仍归属于先前的 send,而不会在后续 send 回显后使其完成。异步写入超时,或该写入期间发生信号发送失败,都会保留预留,直至提供方将写入结算。陈旧检查完成后,会针对当前发送恢复轮询。 - `dsh-code-runtime-subprocess` 通过 `ctx.fs` 物化无依赖 runner,并通过 `ctx.subprocess` 启动它,从而在本地或远程执行世界中保留代码运行时的绑定与输出契约。它通过非插件子路径 `dsh-code-runtime-worker/runtime-host` 共享宿主侧 worker 机制,而不是复制这些机制。准备阶段让同一个生命周期信号贯穿文件系统解析、物化和可执行文件查找,使资源释放能够中止停滞的提供方操作。受堆上限约束的 worker 会在传输前拒绝过大的绑定帧;每个外层转发环节都会在序列化前执行相同的上限检查;launcher 会在回收 controller 前发布已接纳的终态帧,使继承 controller 管道的后代进程无法阻止完成;宿主仍会等待进程组完全停稳。 `dsh-code-runtime-worker` 仍是独立实现。它是较小的进程内后端,可用于无法假定已安装 Node 可执行文件的单文件分发。远程文件系统/进程组合选择 `dsh-code-runtime-subprocess`;它们不需要提供方专用的代码运行时包。 diff --git a/packages/pty/pty-local/README.i18n.yaml b/packages/pty/pty-local/README.i18n.yaml index c89122066a..3f21fb2bd3 100644 --- a/packages/pty/pty-local/README.i18n.yaml +++ b/packages/pty/pty-local/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/pty/pty-local/README.md -README.md: 7e31adfbc486a190252be867e2113948b17e7924 -README.zh.md: eb628aef9ad6b0fea14a5e833197b4fac7856785 +README.md: c7b7a363f22821363a0fed81982165fd35326d12 +README.zh.md: 859b94e4f29b82abb298468f028f9eba7cc067e6 diff --git a/packages/pty/pty-local/README.md b/packages/pty/pty-local/README.md index 7e31adfbc4..c7b7a363f2 100644 --- a/packages/pty/pty-local/README.md +++ b/packages/pty/pty-local/README.md @@ -8,7 +8,7 @@ Persistent shell backend for `ctx.pty` over `ctx.subprocess.spawnTerminal`. It s The plugin injects `pty`, `sandbox`, `sandboxPolicy`, and `subprocess`, then registers the configured backend type (`shell`). `danger-full-access` starts the shell directly; confined modes wrap the exact shell argv through `ctx.sandbox`. The effective session mode is resolved at spawn. A change to a different effective mode is rejected before its `sandbox/mode` event commits while that owner has an open PTY or a spawn in progress; the fence is attached to the exact owner and therefore outlives a provider reload that retains existing sessions. Wait for creation to settle and close the sessions before changing modes, so a terminal opened with wider access cannot survive a downgrade. -Readiness combines a foreground-verified private bash prompt marker, provider-reported foreground stdin-wait facts, silence fallback, and absolute timeout. A marker is not ready until the printable tail after the latest owned marker exactly equals the controlled `PS1`, including when the OSC marker and prompt are split across data callbacks; echoed input or output following a delayed earlier prompt therefore cannot settle the current send. Prompt and silence evidence collected before the provider write, including while pre-write foreground inspection is pending, is discarded at the write boundary. When bash prints the marker before the terminal provider publishes its return to the foreground process group, polling retains the candidate for `handoffGraceMs` past the ordinary silence bound so a coincident handoff can win. An interactive child that inherits `PROMPT_COMMAND` therefore cannot suppress inferred-idle readiness until the absolute timeout. Unknown foreground state is never a positive exact-idle signal. A foreground group's stdin wait that existed before a send is likewise not post-write readiness: the same group must be observed outside that wait before a later wait can settle the send, while a changed foreground group is new evidence. During unpublished startup, a fallback requires observed output; zero-output silence cannot publish an empty session, and timeout rejects the spawn. Cancellation closes the unpublished shell and rejects with the caller's exact abort reason; `PtyBackendCleanupError` separately preserves a cleanup failure. The terminal-allocation signal is detached when allocation returns, while readiness initialization keeps the setup signal, so later cancellation cannot terminate a published persistent session. Incomplete terminal-control sequences are bounded by `maxReadBytes` and discarded through their terminator after crossing that limit; a trailing carriage return is carried across callbacks so split CRLF becomes one newline. +Readiness combines a foreground-verified private bash prompt marker, provider-reported foreground stdin-wait facts, silence fallback, and absolute timeout. A marker is not ready until the printable tail after the latest owned marker exactly equals the controlled `PS1`, including when the OSC marker and prompt are split across data callbacks; echoed input or output following a delayed earlier prompt therefore cannot settle the current send. After a signaled send returns `inferred_idle` without a prompt marker, the next marker remains attributed to that prior send and cannot settle a successor even when it follows the successor's echoed input. Prompt and silence evidence collected before the provider write, including while pre-write foreground inspection is pending, is discarded at the write boundary. When bash prints the marker before the terminal provider publishes its return to the foreground process group, polling retains the candidate for `handoffGraceMs` past the ordinary silence bound so a coincident handoff can win. An interactive child that inherits `PROMPT_COMMAND` therefore cannot suppress inferred-idle readiness until the absolute timeout. Unknown foreground state is never a positive exact-idle signal. A foreground group's stdin wait that existed before a send is likewise not post-write readiness: the same group must be observed outside that wait before a later wait can settle the send, while a changed foreground group is new evidence. During unpublished startup, a fallback requires observed output; zero-output silence cannot publish an empty session, and timeout rejects the spawn. Cancellation closes the unpublished shell and rejects with the caller's exact abort reason; `PtyBackendCleanupError` separately preserves a cleanup failure. The terminal-allocation signal is detached when allocation returns, while readiness initialization keeps the setup signal, so later cancellation cannot terminate a published persistent session. Incomplete terminal-control sequences are bounded by `maxReadBytes` and discarded through their terminator after crossing that limit; a trailing carriage return is carried across callbacks so split CRLF becomes one newline. Send cancellation marks queued input as canceled before asking the terminal handle to signal the current foreground process group with a real `SIGINT`; if asynchronous pre-write inspection later settles, it cannot execute that input. The canceled send retains its slot until foreground signalling settles, so a successor cannot become that signal's target. Cancellation never emulates interruption by writing `\x03`, so raw-mode programs remain cancellable. A send that times out during an asynchronous provider write, or whose cancellation signal fails while that write remains pending, reports its result but retains the slot until the write settles, so late bytes cannot interleave with a successor. Close starts provider-owned TERM-to-KILL whole-session cleanup and awaits quiescence after the terminal outcome. A cleanup failure does not cache a permanently rejected close; a later close retries the provider operation. diff --git a/packages/pty/pty-local/README.zh.md b/packages/pty/pty-local/README.zh.md index eb628aef9a..859b94e4f2 100644 --- a/packages/pty/pty-local/README.zh.md +++ b/packages/pty/pty-local/README.zh.md @@ -8,7 +8,7 @@ 该插件注入 `pty`、`sandbox`、`sandboxPolicy` 和 `subprocess`,然后注册所配置的后端类型(`shell`)。`danger-full-access` 会直接启动 shell;受限模式则通过 `ctx.sandbox` 包装确切的 shell argv。系统在 spawn 时解析会话的实际模式。当某个所有者存在开放的 PTY 或正在进行 spawn 时,如果配置变更会得到不同的实际模式,系统会在对应 `sandbox/mode` 事件提交前拒绝该变更。该限制绑定到确切所有者,因此即使提供方重新加载并保留现有会话,它仍然有效。更改模式前,请等待创建结算并关闭会话,避免以更宽权限打开的终端在权限降级后继续存在。 -就绪检测结合以下机制:由前台状态验证的私有 bash 提示符标记、提供方报告的前台 stdin 等待事实、静默回退和绝对超时。只有最近一个自有标记之后的可打印尾部与受控 `PS1` 完全相等时,系统才会把标记视为就绪;即使 OSC 标记和提示符被拆到多个数据回调中也是如此。因此,如果回显的输入或输出跟在延迟到达的先前提示符之后,该提示符无法使当前 send 完成。系统会在写入边界丢弃提供方写入前收集的提示符与静默证据,包括写入前的前台检查尚未完成时收集的证据。如果 bash 在终端提供方发布其重新取得前台进程组的状态前打印标记,轮询会在普通静默上限之后再保留该候选状态 `handoffGraceMs`,使恰好同时发生的前台交接有机会胜出。因此,继承 `PROMPT_COMMAND` 的交互式子进程无法持续压制推断空闲就绪,最多只能延续到绝对超时。未知的前台状态绝不会作为精确空闲的正向信号。同样,一次 send 之前就已存在的前台进程组 stdin 等待并不代表写入后就绪:必须先观察到同一进程组脱离该等待,之后再次进入等待才能使该次 send 完成;前台进程组发生变化则构成新的证据。尚未发布的启动过程中,回退路径要求已经观察到输出;零输出静默不能发布空会话,超时则拒绝 spawn。取消操作会关闭尚未发布的 shell,并以调用方提供的确切中止原因拒绝;`PtyBackendCleanupError` 会单独保留清理失败。终端分配返回时,分配信号会解除关联;就绪初始化则保留设置阶段信号,因此后续取消无法终止已发布的持久会话。未完成的终端控制序列受 `maxReadBytes` 限制;超过上限后,系统会丢弃内容直到其终止符。末尾的回车会跨回调保留,使拆分的 CRLF 合并为一个换行。 +就绪检测结合以下机制:由前台状态验证的私有 bash 提示符标记、提供方报告的前台 stdin 等待事实、静默回退和绝对超时。只有最近一个自有标记之后的可打印尾部与受控 `PS1` 完全相等时,系统才会把标记视为就绪;即使 OSC 标记和提示符被拆到多个数据回调中也是如此。因此,如果回显的输入或输出跟在延迟到达的先前提示符之后,该提示符无法使当前 send 完成。在一次经过信号处理的 send 未出现提示符标记却返回 `inferred_idle` 后,下一个标记仍归属于该先前 send;即使该标记出现在后续 send 的输入回显之后,也不能使后续 send 完成。系统会在写入边界丢弃提供方写入前收集的提示符与静默证据,包括写入前的前台检查尚未完成时收集的证据。如果 bash 在终端提供方发布其重新取得前台进程组的状态前打印标记,轮询会在普通静默上限之后再保留该候选状态 `handoffGraceMs`,使恰好同时发生的前台交接有机会胜出。因此,继承 `PROMPT_COMMAND` 的交互式子进程无法持续压制推断空闲就绪,最多只能延续到绝对超时。未知的前台状态绝不会作为精确空闲的正向信号。同样,一次 send 之前就已存在的前台进程组 stdin 等待并不代表写入后就绪:必须先观察到同一进程组脱离该等待,之后再次进入等待才能使该次 send 完成;前台进程组发生变化则构成新的证据。尚未发布的启动过程中,回退路径要求已经观察到输出;零输出静默不能发布空会话,超时则拒绝 spawn。取消操作会关闭尚未发布的 shell,并以调用方提供的确切中止原因拒绝;`PtyBackendCleanupError` 会单独保留清理失败。终端分配返回时,分配信号会解除关联;就绪初始化则保留设置阶段信号,因此后续取消无法终止已发布的持久会话。未完成的终端控制序列受 `maxReadBytes` 限制;超过上限后,系统会丢弃内容直到其终止符。末尾的回车会跨回调保留,使拆分的 CRLF 合并为一个换行。 取消发送会先把排队输入标记为已取消,再请求终端句柄向当前前台进程组发送真正的 `SIGINT`;如果异步的写入前检查随后才结算,也无法执行该输入。被取消的发送会保留其槽位,直至前台信号发送结算,因此后续发送不会成为该信号的目标。取消绝不会通过写入 `\x03` 模拟中断,因此原始模式程序仍可取消。发送在提供方异步写入期间超时,或在该写入仍未完成时其取消信号发送失败,都会报告各自结果,但继续占用该槽位,直至写入结算,从而避免延迟到达的字节与后续发送交错。关闭操作启动由提供方负责的 TERM→KILL 全会话清理,并在终端结果之后等待完全停稳。清理失败不会缓存成永久拒绝的关闭操作;后续关闭会重试提供方操作。 diff --git a/packages/pty/pty-local/src/session.ts b/packages/pty/pty-local/src/session.ts index 922db21152..65a10c36ea 100644 --- a/packages/pty/pty-local/src/session.ts +++ b/packages/pty/pty-local/src/session.ts @@ -166,6 +166,7 @@ export class LocalPtySession implements PtyBackendSession { private activeDeadlineTimer: NodeJS.Timeout | undefined private activeAbort: (() => void) | undefined private readonly terminalOperations = new Set>() + private signaledOperation: LocalSendOperation | undefined private interrupting: LocalSendOperation | undefined private writing: LocalSendOperation | undefined private pollingReady: LocalSendOperation | undefined @@ -173,6 +174,7 @@ export class LocalPtySession implements PtyBackendSession { private promptSeen = false private promptTextSeen = false private promptTail = '' + private delayedSignaledPrompt = false private shellPgid: number | undefined private initializing = false private lastOutputAt = Date.now() @@ -317,6 +319,7 @@ export class LocalPtySession implements PtyBackendSession { } async signal(signal: PtySignal): Promise { + if (this.active !== undefined) this.signaledOperation = this.active const targetPgid = await this.terminal.signalForeground(signal) return { delivered: true, targetPgid } } @@ -365,7 +368,9 @@ export class LocalPtySession implements PtyBackendSession { private onData(data: string): void { const sanitized = this.sanitizer.push(data) this.appendOutput(sanitized.text) - if (sanitized.prompt) { + if (sanitized.prompt && this.delayedSignaledPrompt) { + this.delayedSignaledPrompt = false + } else if (sanitized.prompt) { // Bash can print PROMPT_COMMAND before the kernel publishes its return // to the foreground process group. Retain the marker; polling below is // the authority that accepts it only after bash owns the foreground. @@ -461,6 +466,13 @@ export class LocalPtySession implements PtyBackendSession { private settleActive(waitReason: PtyWaitReason, retainOwnership = false): void { const operation = this.active if (operation === undefined) return + // A signaled command can return by silence before bash emits its prompt. + // Reserve that marker so it cannot become successor readiness after echo. + const signaled = this.signaledOperation === operation + if (signaled) this.signaledOperation = undefined + if (waitReason === 'inferred_idle' && !this.promptSeen && signaled) { + this.delayedSignaledPrompt = true + } const scrollbackTruncated = this.scrollback.snapshot().truncated if (retainOwnership) { this.stopPolling() @@ -486,6 +498,7 @@ export class LocalPtySession implements PtyBackendSession { this.activeAbort?.() this.activeAbort = undefined if (this.interrupting === operation) this.interrupting = undefined + if (this.signaledOperation === operation) this.signaledOperation = undefined this.writing = undefined this.pollingReady = undefined this.active = undefined @@ -506,6 +519,7 @@ export class LocalPtySession implements PtyBackendSession { private interrupt(operation: LocalSendOperation): void { if (this.active !== operation) return + this.signaledOperation = operation this.interrupting = operation this.stopPolling() this.ownTerminalOperation(this.interruptOnce(operation)) diff --git a/packages/pty/pty-local/tests/session.spec.ts b/packages/pty/pty-local/tests/session.spec.ts index 513dcbbbb4..28ad58f04d 100644 --- a/packages/pty/pty-local/tests/session.spec.ts +++ b/packages/pty/pty-local/tests/session.spec.ts @@ -591,6 +591,35 @@ describe('LocalPtySession readiness and output', () => { expect(await operation.done).toMatchObject({ waitReason: 'stdin_read' }) }) + it('does not attribute a post-echo prompt from an inferred prior send to its successor', async () => { + vi.useFakeTimers() + const terminal = new FakeTerminal() + const session = new LocalPtySession(terminal, config({ idleSilenceMs: 50, timeoutMs: 200 })) + await initialize(session, terminal) + + const interrupted = session.startSend({ text: 'sleep', submit: true }) + await Promise.resolve() + await Promise.resolve() + await session.signal('SIGINT') + await vi.advanceTimersByTimeAsync(50) + expect((await interrupted.done).waitReason).toBe('inferred_idle') + + const successor = session.startSend({ text: "printf 'PID=%s\\n' \"$!\"", submit: true }) + let settled = false + void successor.done.then(() => { settled = true }) + await Promise.resolve() + await Promise.resolve() + terminal.emitData('printf \'PID=%s\\n\' "$!"\r\n\x1b]133;D;130\x07dsh> ') + await vi.advanceTimersByTimeAsync(10) + expect(settled).toBe(false) + + terminal.emitData('PID=123\r\n\x1b]133;D;0\x07dsh> ') + await vi.advanceTimersByTimeAsync(10) + const result = await successor.done + expect(result.waitReason).toBe('stdin_read') + expect(result.viewport).toContain('PID=123') + }) + it('retains a prompt marker until the startup shell regains the foreground group', async () => { vi.useFakeTimers() const terminal = new FakeTerminal() From 277870dac275292ce12f1de060421ba9cfd24661 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 12:05:00 +0800 Subject: [PATCH 035/108] Revert "fix(pty): fence delayed signal prompts" This reverts commit a78958eeeaf04a3acdf22ef8a26dec370661a51d. --- ...rtable-execution-world-consumers.i18n.yaml | 4 +-- ...7-28-portable-execution-world-consumers.md | 2 +- ...8-portable-execution-world-consumers.zh.md | 2 +- packages/pty/pty-local/README.i18n.yaml | 4 +-- packages/pty/pty-local/README.md | 2 +- packages/pty/pty-local/README.zh.md | 2 +- packages/pty/pty-local/src/session.ts | 16 +--------- packages/pty/pty-local/tests/session.spec.ts | 29 ------------------- 8 files changed, 9 insertions(+), 52 deletions(-) diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml index 803a2f83fe..02545da311 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md -2026-07-28-portable-execution-world-consumers.md: 54b8ec19bc74e734f2cb940a475d2c7b4c9336fd -2026-07-28-portable-execution-world-consumers.zh.md: f21be00fd35267fe93e2fbb62642809cc42c0742 +2026-07-28-portable-execution-world-consumers.md: 715529e5f24df0c2fe0afc40716737285afa223b +2026-07-28-portable-execution-world-consumers.zh.md: a9d864b581bd546c12776152ad4515816ee6b83a diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md index 54b8ec19bc..715529e5f2 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md @@ -22,7 +22,7 @@ Generic consumers use that execution world: - `dsh-bash-local` continues to map Bash semantics onto ordinary `ctx.subprocess.spawn()`. - `dsh-lsp-local` reads and contains source through `ctx.fs`, resolves and launches language servers through `ctx.subprocess`, and carries provider-owned file URIs through initialization and result rendering. Its JSON-RPC, pooling, synchronization, cancellation, and normalization stay unchanged. -- `dsh-pty-local` maps persistent-shell semantics onto `ctx.subprocess.spawnTerminal()`. The local `node-pty` and process-inspection implementation moves into `dsh-subprocess-local`; another subprocess provider supplies the same primitive. The allocation signal is detached before publication, while readiness initialization retains setup cancellation. Prompt and silence evidence collected during asynchronous pre-write inspection is discarded when the provider write begins. Cancellation retains the send reservation until asynchronous foreground signalling settles, so the signal cannot target a successor; if a signaled send returns `inferred_idle` before its prompt arrives, that marker remains attributed to the prior send instead of settling a successor after its echo. A timed-out asynchronous write, or a signal failure during that write, retains the reservation until the provider settles the write. Completion of a stale inspection resumes polling for the current send. +- `dsh-pty-local` maps persistent-shell semantics onto `ctx.subprocess.spawnTerminal()`. The local `node-pty` and process-inspection implementation moves into `dsh-subprocess-local`; another subprocess provider supplies the same primitive. The allocation signal is detached before publication, while readiness initialization retains setup cancellation. Prompt and silence evidence collected during asynchronous pre-write inspection is discarded when the provider write begins. Cancellation retains the send reservation until asynchronous foreground signalling settles, so the signal cannot target a successor; a timed-out asynchronous write, or a signal failure during that write, retains the reservation until the provider settles the write. Completion of a stale inspection resumes polling for the current send. - `dsh-code-runtime-subprocess` materializes a dependency-free runner through `ctx.fs` and launches it through `ctx.subprocess`, preserving the Code Runtime binding and output contract across local or remote worlds. It shares host-side worker mechanics through the non-plugin `dsh-code-runtime-worker/runtime-host` subpath instead of copying them. Preparation carries one lifecycle signal through filesystem resolution, materialization, and executable lookup so disposal can abort a stalled provider. The heap-bounded worker rejects oversized binding frames before transfer, each outer hop enforces the same bound before serialization, and the launcher publishes an accepted terminal frame before reaping its controller so a descendant that inherits controller pipes cannot suppress completion; the host still awaits process-group quiescence. `dsh-code-runtime-worker` remains a separate implementation. It is the smaller in-process backend and works in single-file distributions that cannot assume an installed Node executable. Remote filesystem/process compositions select `dsh-code-runtime-subprocess`; they do not need a provider-specific Code Runtime package. diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md index f21be00fd3..a9d864b581 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md @@ -22,7 +22,7 @@ Status: implemented - `dsh-bash-local` 继续把 Bash 语义映射到普通的 `ctx.subprocess.spawn()`。 - `dsh-lsp-local` 通过 `ctx.fs` 读取源文件并验证包含关系,通过 `ctx.subprocess` 解析和启动语言服务器,并让由提供方负责的文件 URI 贯穿初始化与结果渲染。其 JSON-RPC、池化、同步、取消和规范化保持不变。 -- `dsh-pty-local` 把持久 shell 语义映射到 `ctx.subprocess.spawnTerminal()`。本地 `node-pty` 与进程检查实现移入 `dsh-subprocess-local`;其他进程管理提供方则提供相同原语。分配信号会在发布前解除关联,而就绪初始化仍保留设置阶段的取消。提供方开始写入时,系统会丢弃异步写入前检查期间收集的提示符与静默证据。取消会保留发送预留,直至异步前台信号发送结算,从而使该信号无法把后续发送作为目标;如果收到信号的 send 在其提示符到达前返回 `inferred_idle`,随后到达的标记仍归属于先前的 send,而不会在后续 send 回显后使其完成。异步写入超时,或该写入期间发生信号发送失败,都会保留预留,直至提供方将写入结算。陈旧检查完成后,会针对当前发送恢复轮询。 +- `dsh-pty-local` 把持久 shell 语义映射到 `ctx.subprocess.spawnTerminal()`。本地 `node-pty` 与进程检查实现移入 `dsh-subprocess-local`;其他进程管理提供方则提供相同原语。分配信号会在发布前解除关联,而就绪初始化仍保留设置阶段的取消。提供方开始写入时,系统会丢弃异步写入前检查期间收集的提示符与静默证据。取消会保留发送预留,直至异步前台信号发送结算,从而使该信号无法把后续发送作为目标;异步写入超时,或该写入期间发生信号发送失败,都会保留预留,直至提供方将写入结算。陈旧检查完成后,会针对当前发送恢复轮询。 - `dsh-code-runtime-subprocess` 通过 `ctx.fs` 物化无依赖 runner,并通过 `ctx.subprocess` 启动它,从而在本地或远程执行世界中保留代码运行时的绑定与输出契约。它通过非插件子路径 `dsh-code-runtime-worker/runtime-host` 共享宿主侧 worker 机制,而不是复制这些机制。准备阶段让同一个生命周期信号贯穿文件系统解析、物化和可执行文件查找,使资源释放能够中止停滞的提供方操作。受堆上限约束的 worker 会在传输前拒绝过大的绑定帧;每个外层转发环节都会在序列化前执行相同的上限检查;launcher 会在回收 controller 前发布已接纳的终态帧,使继承 controller 管道的后代进程无法阻止完成;宿主仍会等待进程组完全停稳。 `dsh-code-runtime-worker` 仍是独立实现。它是较小的进程内后端,可用于无法假定已安装 Node 可执行文件的单文件分发。远程文件系统/进程组合选择 `dsh-code-runtime-subprocess`;它们不需要提供方专用的代码运行时包。 diff --git a/packages/pty/pty-local/README.i18n.yaml b/packages/pty/pty-local/README.i18n.yaml index 3f21fb2bd3..c89122066a 100644 --- a/packages/pty/pty-local/README.i18n.yaml +++ b/packages/pty/pty-local/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/pty/pty-local/README.md -README.md: c7b7a363f22821363a0fed81982165fd35326d12 -README.zh.md: 859b94e4f29b82abb298468f028f9eba7cc067e6 +README.md: 7e31adfbc486a190252be867e2113948b17e7924 +README.zh.md: eb628aef9ad6b0fea14a5e833197b4fac7856785 diff --git a/packages/pty/pty-local/README.md b/packages/pty/pty-local/README.md index c7b7a363f2..7e31adfbc4 100644 --- a/packages/pty/pty-local/README.md +++ b/packages/pty/pty-local/README.md @@ -8,7 +8,7 @@ Persistent shell backend for `ctx.pty` over `ctx.subprocess.spawnTerminal`. It s The plugin injects `pty`, `sandbox`, `sandboxPolicy`, and `subprocess`, then registers the configured backend type (`shell`). `danger-full-access` starts the shell directly; confined modes wrap the exact shell argv through `ctx.sandbox`. The effective session mode is resolved at spawn. A change to a different effective mode is rejected before its `sandbox/mode` event commits while that owner has an open PTY or a spawn in progress; the fence is attached to the exact owner and therefore outlives a provider reload that retains existing sessions. Wait for creation to settle and close the sessions before changing modes, so a terminal opened with wider access cannot survive a downgrade. -Readiness combines a foreground-verified private bash prompt marker, provider-reported foreground stdin-wait facts, silence fallback, and absolute timeout. A marker is not ready until the printable tail after the latest owned marker exactly equals the controlled `PS1`, including when the OSC marker and prompt are split across data callbacks; echoed input or output following a delayed earlier prompt therefore cannot settle the current send. After a signaled send returns `inferred_idle` without a prompt marker, the next marker remains attributed to that prior send and cannot settle a successor even when it follows the successor's echoed input. Prompt and silence evidence collected before the provider write, including while pre-write foreground inspection is pending, is discarded at the write boundary. When bash prints the marker before the terminal provider publishes its return to the foreground process group, polling retains the candidate for `handoffGraceMs` past the ordinary silence bound so a coincident handoff can win. An interactive child that inherits `PROMPT_COMMAND` therefore cannot suppress inferred-idle readiness until the absolute timeout. Unknown foreground state is never a positive exact-idle signal. A foreground group's stdin wait that existed before a send is likewise not post-write readiness: the same group must be observed outside that wait before a later wait can settle the send, while a changed foreground group is new evidence. During unpublished startup, a fallback requires observed output; zero-output silence cannot publish an empty session, and timeout rejects the spawn. Cancellation closes the unpublished shell and rejects with the caller's exact abort reason; `PtyBackendCleanupError` separately preserves a cleanup failure. The terminal-allocation signal is detached when allocation returns, while readiness initialization keeps the setup signal, so later cancellation cannot terminate a published persistent session. Incomplete terminal-control sequences are bounded by `maxReadBytes` and discarded through their terminator after crossing that limit; a trailing carriage return is carried across callbacks so split CRLF becomes one newline. +Readiness combines a foreground-verified private bash prompt marker, provider-reported foreground stdin-wait facts, silence fallback, and absolute timeout. A marker is not ready until the printable tail after the latest owned marker exactly equals the controlled `PS1`, including when the OSC marker and prompt are split across data callbacks; echoed input or output following a delayed earlier prompt therefore cannot settle the current send. Prompt and silence evidence collected before the provider write, including while pre-write foreground inspection is pending, is discarded at the write boundary. When bash prints the marker before the terminal provider publishes its return to the foreground process group, polling retains the candidate for `handoffGraceMs` past the ordinary silence bound so a coincident handoff can win. An interactive child that inherits `PROMPT_COMMAND` therefore cannot suppress inferred-idle readiness until the absolute timeout. Unknown foreground state is never a positive exact-idle signal. A foreground group's stdin wait that existed before a send is likewise not post-write readiness: the same group must be observed outside that wait before a later wait can settle the send, while a changed foreground group is new evidence. During unpublished startup, a fallback requires observed output; zero-output silence cannot publish an empty session, and timeout rejects the spawn. Cancellation closes the unpublished shell and rejects with the caller's exact abort reason; `PtyBackendCleanupError` separately preserves a cleanup failure. The terminal-allocation signal is detached when allocation returns, while readiness initialization keeps the setup signal, so later cancellation cannot terminate a published persistent session. Incomplete terminal-control sequences are bounded by `maxReadBytes` and discarded through their terminator after crossing that limit; a trailing carriage return is carried across callbacks so split CRLF becomes one newline. Send cancellation marks queued input as canceled before asking the terminal handle to signal the current foreground process group with a real `SIGINT`; if asynchronous pre-write inspection later settles, it cannot execute that input. The canceled send retains its slot until foreground signalling settles, so a successor cannot become that signal's target. Cancellation never emulates interruption by writing `\x03`, so raw-mode programs remain cancellable. A send that times out during an asynchronous provider write, or whose cancellation signal fails while that write remains pending, reports its result but retains the slot until the write settles, so late bytes cannot interleave with a successor. Close starts provider-owned TERM-to-KILL whole-session cleanup and awaits quiescence after the terminal outcome. A cleanup failure does not cache a permanently rejected close; a later close retries the provider operation. diff --git a/packages/pty/pty-local/README.zh.md b/packages/pty/pty-local/README.zh.md index 859b94e4f2..eb628aef9a 100644 --- a/packages/pty/pty-local/README.zh.md +++ b/packages/pty/pty-local/README.zh.md @@ -8,7 +8,7 @@ 该插件注入 `pty`、`sandbox`、`sandboxPolicy` 和 `subprocess`,然后注册所配置的后端类型(`shell`)。`danger-full-access` 会直接启动 shell;受限模式则通过 `ctx.sandbox` 包装确切的 shell argv。系统在 spawn 时解析会话的实际模式。当某个所有者存在开放的 PTY 或正在进行 spawn 时,如果配置变更会得到不同的实际模式,系统会在对应 `sandbox/mode` 事件提交前拒绝该变更。该限制绑定到确切所有者,因此即使提供方重新加载并保留现有会话,它仍然有效。更改模式前,请等待创建结算并关闭会话,避免以更宽权限打开的终端在权限降级后继续存在。 -就绪检测结合以下机制:由前台状态验证的私有 bash 提示符标记、提供方报告的前台 stdin 等待事实、静默回退和绝对超时。只有最近一个自有标记之后的可打印尾部与受控 `PS1` 完全相等时,系统才会把标记视为就绪;即使 OSC 标记和提示符被拆到多个数据回调中也是如此。因此,如果回显的输入或输出跟在延迟到达的先前提示符之后,该提示符无法使当前 send 完成。在一次经过信号处理的 send 未出现提示符标记却返回 `inferred_idle` 后,下一个标记仍归属于该先前 send;即使该标记出现在后续 send 的输入回显之后,也不能使后续 send 完成。系统会在写入边界丢弃提供方写入前收集的提示符与静默证据,包括写入前的前台检查尚未完成时收集的证据。如果 bash 在终端提供方发布其重新取得前台进程组的状态前打印标记,轮询会在普通静默上限之后再保留该候选状态 `handoffGraceMs`,使恰好同时发生的前台交接有机会胜出。因此,继承 `PROMPT_COMMAND` 的交互式子进程无法持续压制推断空闲就绪,最多只能延续到绝对超时。未知的前台状态绝不会作为精确空闲的正向信号。同样,一次 send 之前就已存在的前台进程组 stdin 等待并不代表写入后就绪:必须先观察到同一进程组脱离该等待,之后再次进入等待才能使该次 send 完成;前台进程组发生变化则构成新的证据。尚未发布的启动过程中,回退路径要求已经观察到输出;零输出静默不能发布空会话,超时则拒绝 spawn。取消操作会关闭尚未发布的 shell,并以调用方提供的确切中止原因拒绝;`PtyBackendCleanupError` 会单独保留清理失败。终端分配返回时,分配信号会解除关联;就绪初始化则保留设置阶段信号,因此后续取消无法终止已发布的持久会话。未完成的终端控制序列受 `maxReadBytes` 限制;超过上限后,系统会丢弃内容直到其终止符。末尾的回车会跨回调保留,使拆分的 CRLF 合并为一个换行。 +就绪检测结合以下机制:由前台状态验证的私有 bash 提示符标记、提供方报告的前台 stdin 等待事实、静默回退和绝对超时。只有最近一个自有标记之后的可打印尾部与受控 `PS1` 完全相等时,系统才会把标记视为就绪;即使 OSC 标记和提示符被拆到多个数据回调中也是如此。因此,如果回显的输入或输出跟在延迟到达的先前提示符之后,该提示符无法使当前 send 完成。系统会在写入边界丢弃提供方写入前收集的提示符与静默证据,包括写入前的前台检查尚未完成时收集的证据。如果 bash 在终端提供方发布其重新取得前台进程组的状态前打印标记,轮询会在普通静默上限之后再保留该候选状态 `handoffGraceMs`,使恰好同时发生的前台交接有机会胜出。因此,继承 `PROMPT_COMMAND` 的交互式子进程无法持续压制推断空闲就绪,最多只能延续到绝对超时。未知的前台状态绝不会作为精确空闲的正向信号。同样,一次 send 之前就已存在的前台进程组 stdin 等待并不代表写入后就绪:必须先观察到同一进程组脱离该等待,之后再次进入等待才能使该次 send 完成;前台进程组发生变化则构成新的证据。尚未发布的启动过程中,回退路径要求已经观察到输出;零输出静默不能发布空会话,超时则拒绝 spawn。取消操作会关闭尚未发布的 shell,并以调用方提供的确切中止原因拒绝;`PtyBackendCleanupError` 会单独保留清理失败。终端分配返回时,分配信号会解除关联;就绪初始化则保留设置阶段信号,因此后续取消无法终止已发布的持久会话。未完成的终端控制序列受 `maxReadBytes` 限制;超过上限后,系统会丢弃内容直到其终止符。末尾的回车会跨回调保留,使拆分的 CRLF 合并为一个换行。 取消发送会先把排队输入标记为已取消,再请求终端句柄向当前前台进程组发送真正的 `SIGINT`;如果异步的写入前检查随后才结算,也无法执行该输入。被取消的发送会保留其槽位,直至前台信号发送结算,因此后续发送不会成为该信号的目标。取消绝不会通过写入 `\x03` 模拟中断,因此原始模式程序仍可取消。发送在提供方异步写入期间超时,或在该写入仍未完成时其取消信号发送失败,都会报告各自结果,但继续占用该槽位,直至写入结算,从而避免延迟到达的字节与后续发送交错。关闭操作启动由提供方负责的 TERM→KILL 全会话清理,并在终端结果之后等待完全停稳。清理失败不会缓存成永久拒绝的关闭操作;后续关闭会重试提供方操作。 diff --git a/packages/pty/pty-local/src/session.ts b/packages/pty/pty-local/src/session.ts index 65a10c36ea..922db21152 100644 --- a/packages/pty/pty-local/src/session.ts +++ b/packages/pty/pty-local/src/session.ts @@ -166,7 +166,6 @@ export class LocalPtySession implements PtyBackendSession { private activeDeadlineTimer: NodeJS.Timeout | undefined private activeAbort: (() => void) | undefined private readonly terminalOperations = new Set>() - private signaledOperation: LocalSendOperation | undefined private interrupting: LocalSendOperation | undefined private writing: LocalSendOperation | undefined private pollingReady: LocalSendOperation | undefined @@ -174,7 +173,6 @@ export class LocalPtySession implements PtyBackendSession { private promptSeen = false private promptTextSeen = false private promptTail = '' - private delayedSignaledPrompt = false private shellPgid: number | undefined private initializing = false private lastOutputAt = Date.now() @@ -319,7 +317,6 @@ export class LocalPtySession implements PtyBackendSession { } async signal(signal: PtySignal): Promise { - if (this.active !== undefined) this.signaledOperation = this.active const targetPgid = await this.terminal.signalForeground(signal) return { delivered: true, targetPgid } } @@ -368,9 +365,7 @@ export class LocalPtySession implements PtyBackendSession { private onData(data: string): void { const sanitized = this.sanitizer.push(data) this.appendOutput(sanitized.text) - if (sanitized.prompt && this.delayedSignaledPrompt) { - this.delayedSignaledPrompt = false - } else if (sanitized.prompt) { + if (sanitized.prompt) { // Bash can print PROMPT_COMMAND before the kernel publishes its return // to the foreground process group. Retain the marker; polling below is // the authority that accepts it only after bash owns the foreground. @@ -466,13 +461,6 @@ export class LocalPtySession implements PtyBackendSession { private settleActive(waitReason: PtyWaitReason, retainOwnership = false): void { const operation = this.active if (operation === undefined) return - // A signaled command can return by silence before bash emits its prompt. - // Reserve that marker so it cannot become successor readiness after echo. - const signaled = this.signaledOperation === operation - if (signaled) this.signaledOperation = undefined - if (waitReason === 'inferred_idle' && !this.promptSeen && signaled) { - this.delayedSignaledPrompt = true - } const scrollbackTruncated = this.scrollback.snapshot().truncated if (retainOwnership) { this.stopPolling() @@ -498,7 +486,6 @@ export class LocalPtySession implements PtyBackendSession { this.activeAbort?.() this.activeAbort = undefined if (this.interrupting === operation) this.interrupting = undefined - if (this.signaledOperation === operation) this.signaledOperation = undefined this.writing = undefined this.pollingReady = undefined this.active = undefined @@ -519,7 +506,6 @@ export class LocalPtySession implements PtyBackendSession { private interrupt(operation: LocalSendOperation): void { if (this.active !== operation) return - this.signaledOperation = operation this.interrupting = operation this.stopPolling() this.ownTerminalOperation(this.interruptOnce(operation)) diff --git a/packages/pty/pty-local/tests/session.spec.ts b/packages/pty/pty-local/tests/session.spec.ts index 28ad58f04d..513dcbbbb4 100644 --- a/packages/pty/pty-local/tests/session.spec.ts +++ b/packages/pty/pty-local/tests/session.spec.ts @@ -591,35 +591,6 @@ describe('LocalPtySession readiness and output', () => { expect(await operation.done).toMatchObject({ waitReason: 'stdin_read' }) }) - it('does not attribute a post-echo prompt from an inferred prior send to its successor', async () => { - vi.useFakeTimers() - const terminal = new FakeTerminal() - const session = new LocalPtySession(terminal, config({ idleSilenceMs: 50, timeoutMs: 200 })) - await initialize(session, terminal) - - const interrupted = session.startSend({ text: 'sleep', submit: true }) - await Promise.resolve() - await Promise.resolve() - await session.signal('SIGINT') - await vi.advanceTimersByTimeAsync(50) - expect((await interrupted.done).waitReason).toBe('inferred_idle') - - const successor = session.startSend({ text: "printf 'PID=%s\\n' \"$!\"", submit: true }) - let settled = false - void successor.done.then(() => { settled = true }) - await Promise.resolve() - await Promise.resolve() - terminal.emitData('printf \'PID=%s\\n\' "$!"\r\n\x1b]133;D;130\x07dsh> ') - await vi.advanceTimersByTimeAsync(10) - expect(settled).toBe(false) - - terminal.emitData('PID=123\r\n\x1b]133;D;0\x07dsh> ') - await vi.advanceTimersByTimeAsync(10) - const result = await successor.done - expect(result.waitReason).toBe('stdin_read') - expect(result.viewport).toContain('PID=123') - }) - it('retains a prompt marker until the startup shell regains the foreground group', async () => { vi.useFakeTimers() const terminal = new FakeTerminal() From 7eedfda961d4e30e10b3d4756650bab402542d0f Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 11:57:12 +0800 Subject: [PATCH 036/108] fix(pty): fence delayed signal prompts --- ...rtable-execution-world-consumers.i18n.yaml | 4 +-- ...7-28-portable-execution-world-consumers.md | 2 +- ...8-portable-execution-world-consumers.zh.md | 2 +- packages/pty/pty-local/README.i18n.yaml | 4 +-- packages/pty/pty-local/README.md | 2 +- packages/pty/pty-local/README.zh.md | 2 +- packages/pty/pty-local/src/session.ts | 16 +++++++++- packages/pty/pty-local/tests/session.spec.ts | 29 +++++++++++++++++++ 8 files changed, 52 insertions(+), 9 deletions(-) diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml index 02545da311..803a2f83fe 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md -2026-07-28-portable-execution-world-consumers.md: 715529e5f24df0c2fe0afc40716737285afa223b -2026-07-28-portable-execution-world-consumers.zh.md: a9d864b581bd546c12776152ad4515816ee6b83a +2026-07-28-portable-execution-world-consumers.md: 54b8ec19bc74e734f2cb940a475d2c7b4c9336fd +2026-07-28-portable-execution-world-consumers.zh.md: f21be00fd35267fe93e2fbb62642809cc42c0742 diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md index 715529e5f2..54b8ec19bc 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md @@ -22,7 +22,7 @@ Generic consumers use that execution world: - `dsh-bash-local` continues to map Bash semantics onto ordinary `ctx.subprocess.spawn()`. - `dsh-lsp-local` reads and contains source through `ctx.fs`, resolves and launches language servers through `ctx.subprocess`, and carries provider-owned file URIs through initialization and result rendering. Its JSON-RPC, pooling, synchronization, cancellation, and normalization stay unchanged. -- `dsh-pty-local` maps persistent-shell semantics onto `ctx.subprocess.spawnTerminal()`. The local `node-pty` and process-inspection implementation moves into `dsh-subprocess-local`; another subprocess provider supplies the same primitive. The allocation signal is detached before publication, while readiness initialization retains setup cancellation. Prompt and silence evidence collected during asynchronous pre-write inspection is discarded when the provider write begins. Cancellation retains the send reservation until asynchronous foreground signalling settles, so the signal cannot target a successor; a timed-out asynchronous write, or a signal failure during that write, retains the reservation until the provider settles the write. Completion of a stale inspection resumes polling for the current send. +- `dsh-pty-local` maps persistent-shell semantics onto `ctx.subprocess.spawnTerminal()`. The local `node-pty` and process-inspection implementation moves into `dsh-subprocess-local`; another subprocess provider supplies the same primitive. The allocation signal is detached before publication, while readiness initialization retains setup cancellation. Prompt and silence evidence collected during asynchronous pre-write inspection is discarded when the provider write begins. Cancellation retains the send reservation until asynchronous foreground signalling settles, so the signal cannot target a successor; if a signaled send returns `inferred_idle` before its prompt arrives, that marker remains attributed to the prior send instead of settling a successor after its echo. A timed-out asynchronous write, or a signal failure during that write, retains the reservation until the provider settles the write. Completion of a stale inspection resumes polling for the current send. - `dsh-code-runtime-subprocess` materializes a dependency-free runner through `ctx.fs` and launches it through `ctx.subprocess`, preserving the Code Runtime binding and output contract across local or remote worlds. It shares host-side worker mechanics through the non-plugin `dsh-code-runtime-worker/runtime-host` subpath instead of copying them. Preparation carries one lifecycle signal through filesystem resolution, materialization, and executable lookup so disposal can abort a stalled provider. The heap-bounded worker rejects oversized binding frames before transfer, each outer hop enforces the same bound before serialization, and the launcher publishes an accepted terminal frame before reaping its controller so a descendant that inherits controller pipes cannot suppress completion; the host still awaits process-group quiescence. `dsh-code-runtime-worker` remains a separate implementation. It is the smaller in-process backend and works in single-file distributions that cannot assume an installed Node executable. Remote filesystem/process compositions select `dsh-code-runtime-subprocess`; they do not need a provider-specific Code Runtime package. diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md index a9d864b581..f21be00fd3 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md @@ -22,7 +22,7 @@ Status: implemented - `dsh-bash-local` 继续把 Bash 语义映射到普通的 `ctx.subprocess.spawn()`。 - `dsh-lsp-local` 通过 `ctx.fs` 读取源文件并验证包含关系,通过 `ctx.subprocess` 解析和启动语言服务器,并让由提供方负责的文件 URI 贯穿初始化与结果渲染。其 JSON-RPC、池化、同步、取消和规范化保持不变。 -- `dsh-pty-local` 把持久 shell 语义映射到 `ctx.subprocess.spawnTerminal()`。本地 `node-pty` 与进程检查实现移入 `dsh-subprocess-local`;其他进程管理提供方则提供相同原语。分配信号会在发布前解除关联,而就绪初始化仍保留设置阶段的取消。提供方开始写入时,系统会丢弃异步写入前检查期间收集的提示符与静默证据。取消会保留发送预留,直至异步前台信号发送结算,从而使该信号无法把后续发送作为目标;异步写入超时,或该写入期间发生信号发送失败,都会保留预留,直至提供方将写入结算。陈旧检查完成后,会针对当前发送恢复轮询。 +- `dsh-pty-local` 把持久 shell 语义映射到 `ctx.subprocess.spawnTerminal()`。本地 `node-pty` 与进程检查实现移入 `dsh-subprocess-local`;其他进程管理提供方则提供相同原语。分配信号会在发布前解除关联,而就绪初始化仍保留设置阶段的取消。提供方开始写入时,系统会丢弃异步写入前检查期间收集的提示符与静默证据。取消会保留发送预留,直至异步前台信号发送结算,从而使该信号无法把后续发送作为目标;如果收到信号的 send 在其提示符到达前返回 `inferred_idle`,随后到达的标记仍归属于先前的 send,而不会在后续 send 回显后使其完成。异步写入超时,或该写入期间发生信号发送失败,都会保留预留,直至提供方将写入结算。陈旧检查完成后,会针对当前发送恢复轮询。 - `dsh-code-runtime-subprocess` 通过 `ctx.fs` 物化无依赖 runner,并通过 `ctx.subprocess` 启动它,从而在本地或远程执行世界中保留代码运行时的绑定与输出契约。它通过非插件子路径 `dsh-code-runtime-worker/runtime-host` 共享宿主侧 worker 机制,而不是复制这些机制。准备阶段让同一个生命周期信号贯穿文件系统解析、物化和可执行文件查找,使资源释放能够中止停滞的提供方操作。受堆上限约束的 worker 会在传输前拒绝过大的绑定帧;每个外层转发环节都会在序列化前执行相同的上限检查;launcher 会在回收 controller 前发布已接纳的终态帧,使继承 controller 管道的后代进程无法阻止完成;宿主仍会等待进程组完全停稳。 `dsh-code-runtime-worker` 仍是独立实现。它是较小的进程内后端,可用于无法假定已安装 Node 可执行文件的单文件分发。远程文件系统/进程组合选择 `dsh-code-runtime-subprocess`;它们不需要提供方专用的代码运行时包。 diff --git a/packages/pty/pty-local/README.i18n.yaml b/packages/pty/pty-local/README.i18n.yaml index c89122066a..3f21fb2bd3 100644 --- a/packages/pty/pty-local/README.i18n.yaml +++ b/packages/pty/pty-local/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/pty/pty-local/README.md -README.md: 7e31adfbc486a190252be867e2113948b17e7924 -README.zh.md: eb628aef9ad6b0fea14a5e833197b4fac7856785 +README.md: c7b7a363f22821363a0fed81982165fd35326d12 +README.zh.md: 859b94e4f29b82abb298468f028f9eba7cc067e6 diff --git a/packages/pty/pty-local/README.md b/packages/pty/pty-local/README.md index 7e31adfbc4..c7b7a363f2 100644 --- a/packages/pty/pty-local/README.md +++ b/packages/pty/pty-local/README.md @@ -8,7 +8,7 @@ Persistent shell backend for `ctx.pty` over `ctx.subprocess.spawnTerminal`. It s The plugin injects `pty`, `sandbox`, `sandboxPolicy`, and `subprocess`, then registers the configured backend type (`shell`). `danger-full-access` starts the shell directly; confined modes wrap the exact shell argv through `ctx.sandbox`. The effective session mode is resolved at spawn. A change to a different effective mode is rejected before its `sandbox/mode` event commits while that owner has an open PTY or a spawn in progress; the fence is attached to the exact owner and therefore outlives a provider reload that retains existing sessions. Wait for creation to settle and close the sessions before changing modes, so a terminal opened with wider access cannot survive a downgrade. -Readiness combines a foreground-verified private bash prompt marker, provider-reported foreground stdin-wait facts, silence fallback, and absolute timeout. A marker is not ready until the printable tail after the latest owned marker exactly equals the controlled `PS1`, including when the OSC marker and prompt are split across data callbacks; echoed input or output following a delayed earlier prompt therefore cannot settle the current send. Prompt and silence evidence collected before the provider write, including while pre-write foreground inspection is pending, is discarded at the write boundary. When bash prints the marker before the terminal provider publishes its return to the foreground process group, polling retains the candidate for `handoffGraceMs` past the ordinary silence bound so a coincident handoff can win. An interactive child that inherits `PROMPT_COMMAND` therefore cannot suppress inferred-idle readiness until the absolute timeout. Unknown foreground state is never a positive exact-idle signal. A foreground group's stdin wait that existed before a send is likewise not post-write readiness: the same group must be observed outside that wait before a later wait can settle the send, while a changed foreground group is new evidence. During unpublished startup, a fallback requires observed output; zero-output silence cannot publish an empty session, and timeout rejects the spawn. Cancellation closes the unpublished shell and rejects with the caller's exact abort reason; `PtyBackendCleanupError` separately preserves a cleanup failure. The terminal-allocation signal is detached when allocation returns, while readiness initialization keeps the setup signal, so later cancellation cannot terminate a published persistent session. Incomplete terminal-control sequences are bounded by `maxReadBytes` and discarded through their terminator after crossing that limit; a trailing carriage return is carried across callbacks so split CRLF becomes one newline. +Readiness combines a foreground-verified private bash prompt marker, provider-reported foreground stdin-wait facts, silence fallback, and absolute timeout. A marker is not ready until the printable tail after the latest owned marker exactly equals the controlled `PS1`, including when the OSC marker and prompt are split across data callbacks; echoed input or output following a delayed earlier prompt therefore cannot settle the current send. After a signaled send returns `inferred_idle` without a prompt marker, the next marker remains attributed to that prior send and cannot settle a successor even when it follows the successor's echoed input. Prompt and silence evidence collected before the provider write, including while pre-write foreground inspection is pending, is discarded at the write boundary. When bash prints the marker before the terminal provider publishes its return to the foreground process group, polling retains the candidate for `handoffGraceMs` past the ordinary silence bound so a coincident handoff can win. An interactive child that inherits `PROMPT_COMMAND` therefore cannot suppress inferred-idle readiness until the absolute timeout. Unknown foreground state is never a positive exact-idle signal. A foreground group's stdin wait that existed before a send is likewise not post-write readiness: the same group must be observed outside that wait before a later wait can settle the send, while a changed foreground group is new evidence. During unpublished startup, a fallback requires observed output; zero-output silence cannot publish an empty session, and timeout rejects the spawn. Cancellation closes the unpublished shell and rejects with the caller's exact abort reason; `PtyBackendCleanupError` separately preserves a cleanup failure. The terminal-allocation signal is detached when allocation returns, while readiness initialization keeps the setup signal, so later cancellation cannot terminate a published persistent session. Incomplete terminal-control sequences are bounded by `maxReadBytes` and discarded through their terminator after crossing that limit; a trailing carriage return is carried across callbacks so split CRLF becomes one newline. Send cancellation marks queued input as canceled before asking the terminal handle to signal the current foreground process group with a real `SIGINT`; if asynchronous pre-write inspection later settles, it cannot execute that input. The canceled send retains its slot until foreground signalling settles, so a successor cannot become that signal's target. Cancellation never emulates interruption by writing `\x03`, so raw-mode programs remain cancellable. A send that times out during an asynchronous provider write, or whose cancellation signal fails while that write remains pending, reports its result but retains the slot until the write settles, so late bytes cannot interleave with a successor. Close starts provider-owned TERM-to-KILL whole-session cleanup and awaits quiescence after the terminal outcome. A cleanup failure does not cache a permanently rejected close; a later close retries the provider operation. diff --git a/packages/pty/pty-local/README.zh.md b/packages/pty/pty-local/README.zh.md index eb628aef9a..859b94e4f2 100644 --- a/packages/pty/pty-local/README.zh.md +++ b/packages/pty/pty-local/README.zh.md @@ -8,7 +8,7 @@ 该插件注入 `pty`、`sandbox`、`sandboxPolicy` 和 `subprocess`,然后注册所配置的后端类型(`shell`)。`danger-full-access` 会直接启动 shell;受限模式则通过 `ctx.sandbox` 包装确切的 shell argv。系统在 spawn 时解析会话的实际模式。当某个所有者存在开放的 PTY 或正在进行 spawn 时,如果配置变更会得到不同的实际模式,系统会在对应 `sandbox/mode` 事件提交前拒绝该变更。该限制绑定到确切所有者,因此即使提供方重新加载并保留现有会话,它仍然有效。更改模式前,请等待创建结算并关闭会话,避免以更宽权限打开的终端在权限降级后继续存在。 -就绪检测结合以下机制:由前台状态验证的私有 bash 提示符标记、提供方报告的前台 stdin 等待事实、静默回退和绝对超时。只有最近一个自有标记之后的可打印尾部与受控 `PS1` 完全相等时,系统才会把标记视为就绪;即使 OSC 标记和提示符被拆到多个数据回调中也是如此。因此,如果回显的输入或输出跟在延迟到达的先前提示符之后,该提示符无法使当前 send 完成。系统会在写入边界丢弃提供方写入前收集的提示符与静默证据,包括写入前的前台检查尚未完成时收集的证据。如果 bash 在终端提供方发布其重新取得前台进程组的状态前打印标记,轮询会在普通静默上限之后再保留该候选状态 `handoffGraceMs`,使恰好同时发生的前台交接有机会胜出。因此,继承 `PROMPT_COMMAND` 的交互式子进程无法持续压制推断空闲就绪,最多只能延续到绝对超时。未知的前台状态绝不会作为精确空闲的正向信号。同样,一次 send 之前就已存在的前台进程组 stdin 等待并不代表写入后就绪:必须先观察到同一进程组脱离该等待,之后再次进入等待才能使该次 send 完成;前台进程组发生变化则构成新的证据。尚未发布的启动过程中,回退路径要求已经观察到输出;零输出静默不能发布空会话,超时则拒绝 spawn。取消操作会关闭尚未发布的 shell,并以调用方提供的确切中止原因拒绝;`PtyBackendCleanupError` 会单独保留清理失败。终端分配返回时,分配信号会解除关联;就绪初始化则保留设置阶段信号,因此后续取消无法终止已发布的持久会话。未完成的终端控制序列受 `maxReadBytes` 限制;超过上限后,系统会丢弃内容直到其终止符。末尾的回车会跨回调保留,使拆分的 CRLF 合并为一个换行。 +就绪检测结合以下机制:由前台状态验证的私有 bash 提示符标记、提供方报告的前台 stdin 等待事实、静默回退和绝对超时。只有最近一个自有标记之后的可打印尾部与受控 `PS1` 完全相等时,系统才会把标记视为就绪;即使 OSC 标记和提示符被拆到多个数据回调中也是如此。因此,如果回显的输入或输出跟在延迟到达的先前提示符之后,该提示符无法使当前 send 完成。在一次经过信号处理的 send 未出现提示符标记却返回 `inferred_idle` 后,下一个标记仍归属于该先前 send;即使该标记出现在后续 send 的输入回显之后,也不能使后续 send 完成。系统会在写入边界丢弃提供方写入前收集的提示符与静默证据,包括写入前的前台检查尚未完成时收集的证据。如果 bash 在终端提供方发布其重新取得前台进程组的状态前打印标记,轮询会在普通静默上限之后再保留该候选状态 `handoffGraceMs`,使恰好同时发生的前台交接有机会胜出。因此,继承 `PROMPT_COMMAND` 的交互式子进程无法持续压制推断空闲就绪,最多只能延续到绝对超时。未知的前台状态绝不会作为精确空闲的正向信号。同样,一次 send 之前就已存在的前台进程组 stdin 等待并不代表写入后就绪:必须先观察到同一进程组脱离该等待,之后再次进入等待才能使该次 send 完成;前台进程组发生变化则构成新的证据。尚未发布的启动过程中,回退路径要求已经观察到输出;零输出静默不能发布空会话,超时则拒绝 spawn。取消操作会关闭尚未发布的 shell,并以调用方提供的确切中止原因拒绝;`PtyBackendCleanupError` 会单独保留清理失败。终端分配返回时,分配信号会解除关联;就绪初始化则保留设置阶段信号,因此后续取消无法终止已发布的持久会话。未完成的终端控制序列受 `maxReadBytes` 限制;超过上限后,系统会丢弃内容直到其终止符。末尾的回车会跨回调保留,使拆分的 CRLF 合并为一个换行。 取消发送会先把排队输入标记为已取消,再请求终端句柄向当前前台进程组发送真正的 `SIGINT`;如果异步的写入前检查随后才结算,也无法执行该输入。被取消的发送会保留其槽位,直至前台信号发送结算,因此后续发送不会成为该信号的目标。取消绝不会通过写入 `\x03` 模拟中断,因此原始模式程序仍可取消。发送在提供方异步写入期间超时,或在该写入仍未完成时其取消信号发送失败,都会报告各自结果,但继续占用该槽位,直至写入结算,从而避免延迟到达的字节与后续发送交错。关闭操作启动由提供方负责的 TERM→KILL 全会话清理,并在终端结果之后等待完全停稳。清理失败不会缓存成永久拒绝的关闭操作;后续关闭会重试提供方操作。 diff --git a/packages/pty/pty-local/src/session.ts b/packages/pty/pty-local/src/session.ts index 922db21152..65a10c36ea 100644 --- a/packages/pty/pty-local/src/session.ts +++ b/packages/pty/pty-local/src/session.ts @@ -166,6 +166,7 @@ export class LocalPtySession implements PtyBackendSession { private activeDeadlineTimer: NodeJS.Timeout | undefined private activeAbort: (() => void) | undefined private readonly terminalOperations = new Set>() + private signaledOperation: LocalSendOperation | undefined private interrupting: LocalSendOperation | undefined private writing: LocalSendOperation | undefined private pollingReady: LocalSendOperation | undefined @@ -173,6 +174,7 @@ export class LocalPtySession implements PtyBackendSession { private promptSeen = false private promptTextSeen = false private promptTail = '' + private delayedSignaledPrompt = false private shellPgid: number | undefined private initializing = false private lastOutputAt = Date.now() @@ -317,6 +319,7 @@ export class LocalPtySession implements PtyBackendSession { } async signal(signal: PtySignal): Promise { + if (this.active !== undefined) this.signaledOperation = this.active const targetPgid = await this.terminal.signalForeground(signal) return { delivered: true, targetPgid } } @@ -365,7 +368,9 @@ export class LocalPtySession implements PtyBackendSession { private onData(data: string): void { const sanitized = this.sanitizer.push(data) this.appendOutput(sanitized.text) - if (sanitized.prompt) { + if (sanitized.prompt && this.delayedSignaledPrompt) { + this.delayedSignaledPrompt = false + } else if (sanitized.prompt) { // Bash can print PROMPT_COMMAND before the kernel publishes its return // to the foreground process group. Retain the marker; polling below is // the authority that accepts it only after bash owns the foreground. @@ -461,6 +466,13 @@ export class LocalPtySession implements PtyBackendSession { private settleActive(waitReason: PtyWaitReason, retainOwnership = false): void { const operation = this.active if (operation === undefined) return + // A signaled command can return by silence before bash emits its prompt. + // Reserve that marker so it cannot become successor readiness after echo. + const signaled = this.signaledOperation === operation + if (signaled) this.signaledOperation = undefined + if (waitReason === 'inferred_idle' && !this.promptSeen && signaled) { + this.delayedSignaledPrompt = true + } const scrollbackTruncated = this.scrollback.snapshot().truncated if (retainOwnership) { this.stopPolling() @@ -486,6 +498,7 @@ export class LocalPtySession implements PtyBackendSession { this.activeAbort?.() this.activeAbort = undefined if (this.interrupting === operation) this.interrupting = undefined + if (this.signaledOperation === operation) this.signaledOperation = undefined this.writing = undefined this.pollingReady = undefined this.active = undefined @@ -506,6 +519,7 @@ export class LocalPtySession implements PtyBackendSession { private interrupt(operation: LocalSendOperation): void { if (this.active !== operation) return + this.signaledOperation = operation this.interrupting = operation this.stopPolling() this.ownTerminalOperation(this.interruptOnce(operation)) diff --git a/packages/pty/pty-local/tests/session.spec.ts b/packages/pty/pty-local/tests/session.spec.ts index 513dcbbbb4..28ad58f04d 100644 --- a/packages/pty/pty-local/tests/session.spec.ts +++ b/packages/pty/pty-local/tests/session.spec.ts @@ -591,6 +591,35 @@ describe('LocalPtySession readiness and output', () => { expect(await operation.done).toMatchObject({ waitReason: 'stdin_read' }) }) + it('does not attribute a post-echo prompt from an inferred prior send to its successor', async () => { + vi.useFakeTimers() + const terminal = new FakeTerminal() + const session = new LocalPtySession(terminal, config({ idleSilenceMs: 50, timeoutMs: 200 })) + await initialize(session, terminal) + + const interrupted = session.startSend({ text: 'sleep', submit: true }) + await Promise.resolve() + await Promise.resolve() + await session.signal('SIGINT') + await vi.advanceTimersByTimeAsync(50) + expect((await interrupted.done).waitReason).toBe('inferred_idle') + + const successor = session.startSend({ text: "printf 'PID=%s\\n' \"$!\"", submit: true }) + let settled = false + void successor.done.then(() => { settled = true }) + await Promise.resolve() + await Promise.resolve() + terminal.emitData('printf \'PID=%s\\n\' "$!"\r\n\x1b]133;D;130\x07dsh> ') + await vi.advanceTimersByTimeAsync(10) + expect(settled).toBe(false) + + terminal.emitData('PID=123\r\n\x1b]133;D;0\x07dsh> ') + await vi.advanceTimersByTimeAsync(10) + const result = await successor.done + expect(result.waitReason).toBe('stdin_read') + expect(result.viewport).toContain('PID=123') + }) + it('retains a prompt marker until the startup shell regains the foreground group', async () => { vi.useFakeTimers() const terminal = new FakeTerminal() From 727546ce6d8cb4a12294d63da5f9cb43ee19443f Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 12:59:49 +0800 Subject: [PATCH 037/108] fix(code-runtime): bound every subprocess frame hop --- packages/code-runtime/code-runtime-worker/src/runtime-host.ts | 1 + 1 file changed, 1 insertion(+) diff --git a/packages/code-runtime/code-runtime-worker/src/runtime-host.ts b/packages/code-runtime/code-runtime-worker/src/runtime-host.ts index 474a058e9f..d2bb1082b8 100644 --- a/packages/code-runtime/code-runtime-worker/src/runtime-host.ts +++ b/packages/code-runtime/code-runtime-worker/src/runtime-host.ts @@ -223,4 +223,5 @@ export class RuntimeOutputLedger { } export { decodeWorkerJson, encodeWorkerJson, snapshotCodeJsonValue } from './worker-json.ts' +export { jsonValueBytesUpTo } from './output-json.ts' export type { WorkerJsonWire } from './worker-json.ts' From 045e8462ee5ba25d14e399060a4ccb3c40f89c5d Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 13:00:12 +0800 Subject: [PATCH 038/108] fix(pty): drain public signals during close --- ...rtable-execution-world-consumers.i18n.yaml | 4 ++-- ...7-28-portable-execution-world-consumers.md | 2 +- ...8-portable-execution-world-consumers.zh.md | 2 +- packages/pty/pty-local/README.i18n.yaml | 4 ++-- packages/pty/pty-local/README.md | 2 +- packages/pty/pty-local/README.zh.md | 2 +- packages/pty/pty-local/src/session.ts | 12 +++++++---- packages/pty/pty-local/tests/session.spec.ts | 20 +++++++++++++++++++ 8 files changed, 36 insertions(+), 12 deletions(-) diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml index 803a2f83fe..fecaacac05 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md -2026-07-28-portable-execution-world-consumers.md: 54b8ec19bc74e734f2cb940a475d2c7b4c9336fd -2026-07-28-portable-execution-world-consumers.zh.md: f21be00fd35267fe93e2fbb62642809cc42c0742 +2026-07-28-portable-execution-world-consumers.md: fe9d7bdd38199a031989b093fd3e9d931d62c80a +2026-07-28-portable-execution-world-consumers.zh.md: 9f4f38e6e4c6c7dba74e8de4c260ec633dade910 diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md index 54b8ec19bc..fe9d7bdd38 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md @@ -22,7 +22,7 @@ Generic consumers use that execution world: - `dsh-bash-local` continues to map Bash semantics onto ordinary `ctx.subprocess.spawn()`. - `dsh-lsp-local` reads and contains source through `ctx.fs`, resolves and launches language servers through `ctx.subprocess`, and carries provider-owned file URIs through initialization and result rendering. Its JSON-RPC, pooling, synchronization, cancellation, and normalization stay unchanged. -- `dsh-pty-local` maps persistent-shell semantics onto `ctx.subprocess.spawnTerminal()`. The local `node-pty` and process-inspection implementation moves into `dsh-subprocess-local`; another subprocess provider supplies the same primitive. The allocation signal is detached before publication, while readiness initialization retains setup cancellation. Prompt and silence evidence collected during asynchronous pre-write inspection is discarded when the provider write begins. Cancellation retains the send reservation until asynchronous foreground signalling settles, so the signal cannot target a successor; if a signaled send returns `inferred_idle` before its prompt arrives, that marker remains attributed to the prior send instead of settling a successor after its echo. A timed-out asynchronous write, or a signal failure during that write, retains the reservation until the provider settles the write. Completion of a stale inspection resumes polling for the current send. +- `dsh-pty-local` maps persistent-shell semantics onto `ctx.subprocess.spawnTerminal()`. The local `node-pty` and process-inspection implementation moves into `dsh-subprocess-local`; another subprocess provider supplies the same primitive. The allocation signal is detached before publication, while readiness initialization retains setup cancellation. Prompt and silence evidence collected during asynchronous pre-write inspection is discarded when the provider write begins. Cancellation retains the send reservation until asynchronous foreground signalling settles, so the signal cannot target a successor; if a signaled send returns `inferred_idle` before its prompt arrives, that marker remains attributed to the prior send instead of settling a successor after its echo. A timed-out asynchronous write, or a signal failure during that write, retains the reservation until the provider settles the write. Completion of a stale inspection resumes polling for the current send. Close rejects new public signals after shutdown begins and drains public signals already in flight before returning. - `dsh-code-runtime-subprocess` materializes a dependency-free runner through `ctx.fs` and launches it through `ctx.subprocess`, preserving the Code Runtime binding and output contract across local or remote worlds. It shares host-side worker mechanics through the non-plugin `dsh-code-runtime-worker/runtime-host` subpath instead of copying them. Preparation carries one lifecycle signal through filesystem resolution, materialization, and executable lookup so disposal can abort a stalled provider. The heap-bounded worker rejects oversized binding frames before transfer, each outer hop enforces the same bound before serialization, and the launcher publishes an accepted terminal frame before reaping its controller so a descendant that inherits controller pipes cannot suppress completion; the host still awaits process-group quiescence. `dsh-code-runtime-worker` remains a separate implementation. It is the smaller in-process backend and works in single-file distributions that cannot assume an installed Node executable. Remote filesystem/process compositions select `dsh-code-runtime-subprocess`; they do not need a provider-specific Code Runtime package. diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md index f21be00fd3..9f4f38e6e4 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md @@ -22,7 +22,7 @@ Status: implemented - `dsh-bash-local` 继续把 Bash 语义映射到普通的 `ctx.subprocess.spawn()`。 - `dsh-lsp-local` 通过 `ctx.fs` 读取源文件并验证包含关系,通过 `ctx.subprocess` 解析和启动语言服务器,并让由提供方负责的文件 URI 贯穿初始化与结果渲染。其 JSON-RPC、池化、同步、取消和规范化保持不变。 -- `dsh-pty-local` 把持久 shell 语义映射到 `ctx.subprocess.spawnTerminal()`。本地 `node-pty` 与进程检查实现移入 `dsh-subprocess-local`;其他进程管理提供方则提供相同原语。分配信号会在发布前解除关联,而就绪初始化仍保留设置阶段的取消。提供方开始写入时,系统会丢弃异步写入前检查期间收集的提示符与静默证据。取消会保留发送预留,直至异步前台信号发送结算,从而使该信号无法把后续发送作为目标;如果收到信号的 send 在其提示符到达前返回 `inferred_idle`,随后到达的标记仍归属于先前的 send,而不会在后续 send 回显后使其完成。异步写入超时,或该写入期间发生信号发送失败,都会保留预留,直至提供方将写入结算。陈旧检查完成后,会针对当前发送恢复轮询。 +- `dsh-pty-local` 把持久 shell 语义映射到 `ctx.subprocess.spawnTerminal()`。本地 `node-pty` 与进程检查实现移入 `dsh-subprocess-local`;其他进程管理提供方则提供相同原语。分配信号会在发布前解除关联,而就绪初始化仍保留设置阶段的取消。提供方开始写入时,系统会丢弃异步写入前检查期间收集的提示符与静默证据。取消会保留发送预留,直至异步前台信号发送结算,从而使该信号无法把后续发送作为目标;如果收到信号的 send 在其提示符到达前返回 `inferred_idle`,随后到达的标记仍归属于先前的 send,而不会在后续 send 回显后使其完成。异步写入超时,或该写入期间发生信号发送失败,都会保留预留,直至提供方将写入结算。陈旧检查完成后,会针对当前发送恢复轮询。关闭操作开始后会拒绝新的公开信号,并在返回前等待所有已在途的公开信号结算。 - `dsh-code-runtime-subprocess` 通过 `ctx.fs` 物化无依赖 runner,并通过 `ctx.subprocess` 启动它,从而在本地或远程执行世界中保留代码运行时的绑定与输出契约。它通过非插件子路径 `dsh-code-runtime-worker/runtime-host` 共享宿主侧 worker 机制,而不是复制这些机制。准备阶段让同一个生命周期信号贯穿文件系统解析、物化和可执行文件查找,使资源释放能够中止停滞的提供方操作。受堆上限约束的 worker 会在传输前拒绝过大的绑定帧;每个外层转发环节都会在序列化前执行相同的上限检查;launcher 会在回收 controller 前发布已接纳的终态帧,使继承 controller 管道的后代进程无法阻止完成;宿主仍会等待进程组完全停稳。 `dsh-code-runtime-worker` 仍是独立实现。它是较小的进程内后端,可用于无法假定已安装 Node 可执行文件的单文件分发。远程文件系统/进程组合选择 `dsh-code-runtime-subprocess`;它们不需要提供方专用的代码运行时包。 diff --git a/packages/pty/pty-local/README.i18n.yaml b/packages/pty/pty-local/README.i18n.yaml index 3f21fb2bd3..2bd653983e 100644 --- a/packages/pty/pty-local/README.i18n.yaml +++ b/packages/pty/pty-local/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/pty/pty-local/README.md -README.md: c7b7a363f22821363a0fed81982165fd35326d12 -README.zh.md: 859b94e4f29b82abb298468f028f9eba7cc067e6 +README.md: bccef05ca73ec8d826621f814f232206fa9fea4b +README.zh.md: 27d4d1193047eb1bb3bf2289697be3c0cc0253aa diff --git a/packages/pty/pty-local/README.md b/packages/pty/pty-local/README.md index c7b7a363f2..bccef05ca7 100644 --- a/packages/pty/pty-local/README.md +++ b/packages/pty/pty-local/README.md @@ -10,7 +10,7 @@ The plugin injects `pty`, `sandbox`, `sandboxPolicy`, and `subprocess`, then reg Readiness combines a foreground-verified private bash prompt marker, provider-reported foreground stdin-wait facts, silence fallback, and absolute timeout. A marker is not ready until the printable tail after the latest owned marker exactly equals the controlled `PS1`, including when the OSC marker and prompt are split across data callbacks; echoed input or output following a delayed earlier prompt therefore cannot settle the current send. After a signaled send returns `inferred_idle` without a prompt marker, the next marker remains attributed to that prior send and cannot settle a successor even when it follows the successor's echoed input. Prompt and silence evidence collected before the provider write, including while pre-write foreground inspection is pending, is discarded at the write boundary. When bash prints the marker before the terminal provider publishes its return to the foreground process group, polling retains the candidate for `handoffGraceMs` past the ordinary silence bound so a coincident handoff can win. An interactive child that inherits `PROMPT_COMMAND` therefore cannot suppress inferred-idle readiness until the absolute timeout. Unknown foreground state is never a positive exact-idle signal. A foreground group's stdin wait that existed before a send is likewise not post-write readiness: the same group must be observed outside that wait before a later wait can settle the send, while a changed foreground group is new evidence. During unpublished startup, a fallback requires observed output; zero-output silence cannot publish an empty session, and timeout rejects the spawn. Cancellation closes the unpublished shell and rejects with the caller's exact abort reason; `PtyBackendCleanupError` separately preserves a cleanup failure. The terminal-allocation signal is detached when allocation returns, while readiness initialization keeps the setup signal, so later cancellation cannot terminate a published persistent session. Incomplete terminal-control sequences are bounded by `maxReadBytes` and discarded through their terminator after crossing that limit; a trailing carriage return is carried across callbacks so split CRLF becomes one newline. -Send cancellation marks queued input as canceled before asking the terminal handle to signal the current foreground process group with a real `SIGINT`; if asynchronous pre-write inspection later settles, it cannot execute that input. The canceled send retains its slot until foreground signalling settles, so a successor cannot become that signal's target. Cancellation never emulates interruption by writing `\x03`, so raw-mode programs remain cancellable. A send that times out during an asynchronous provider write, or whose cancellation signal fails while that write remains pending, reports its result but retains the slot until the write settles, so late bytes cannot interleave with a successor. Close starts provider-owned TERM-to-KILL whole-session cleanup and awaits quiescence after the terminal outcome. A cleanup failure does not cache a permanently rejected close; a later close retries the provider operation. +Send cancellation marks queued input as canceled before asking the terminal handle to signal the current foreground process group with a real `SIGINT`; if asynchronous pre-write inspection later settles, it cannot execute that input. The canceled send retains its slot until foreground signalling settles, so a successor cannot become that signal's target. Cancellation never emulates interruption by writing `\x03`, so raw-mode programs remain cancellable. A send that times out during an asynchronous provider write, or whose cancellation signal fails while that write remains pending, reports its result but retains the slot until the write settles, so late bytes cannot interleave with a successor. Close rejects new public signals after shutdown begins, drains public signals already in flight, starts provider-owned TERM-to-KILL whole-session cleanup, and awaits quiescence after the terminal outcome. A cleanup failure does not cache a permanently rejected close; a later close retries the provider operation. ## Model Experience diff --git a/packages/pty/pty-local/README.zh.md b/packages/pty/pty-local/README.zh.md index 859b94e4f2..27d4d11930 100644 --- a/packages/pty/pty-local/README.zh.md +++ b/packages/pty/pty-local/README.zh.md @@ -10,7 +10,7 @@ 就绪检测结合以下机制:由前台状态验证的私有 bash 提示符标记、提供方报告的前台 stdin 等待事实、静默回退和绝对超时。只有最近一个自有标记之后的可打印尾部与受控 `PS1` 完全相等时,系统才会把标记视为就绪;即使 OSC 标记和提示符被拆到多个数据回调中也是如此。因此,如果回显的输入或输出跟在延迟到达的先前提示符之后,该提示符无法使当前 send 完成。在一次经过信号处理的 send 未出现提示符标记却返回 `inferred_idle` 后,下一个标记仍归属于该先前 send;即使该标记出现在后续 send 的输入回显之后,也不能使后续 send 完成。系统会在写入边界丢弃提供方写入前收集的提示符与静默证据,包括写入前的前台检查尚未完成时收集的证据。如果 bash 在终端提供方发布其重新取得前台进程组的状态前打印标记,轮询会在普通静默上限之后再保留该候选状态 `handoffGraceMs`,使恰好同时发生的前台交接有机会胜出。因此,继承 `PROMPT_COMMAND` 的交互式子进程无法持续压制推断空闲就绪,最多只能延续到绝对超时。未知的前台状态绝不会作为精确空闲的正向信号。同样,一次 send 之前就已存在的前台进程组 stdin 等待并不代表写入后就绪:必须先观察到同一进程组脱离该等待,之后再次进入等待才能使该次 send 完成;前台进程组发生变化则构成新的证据。尚未发布的启动过程中,回退路径要求已经观察到输出;零输出静默不能发布空会话,超时则拒绝 spawn。取消操作会关闭尚未发布的 shell,并以调用方提供的确切中止原因拒绝;`PtyBackendCleanupError` 会单独保留清理失败。终端分配返回时,分配信号会解除关联;就绪初始化则保留设置阶段信号,因此后续取消无法终止已发布的持久会话。未完成的终端控制序列受 `maxReadBytes` 限制;超过上限后,系统会丢弃内容直到其终止符。末尾的回车会跨回调保留,使拆分的 CRLF 合并为一个换行。 -取消发送会先把排队输入标记为已取消,再请求终端句柄向当前前台进程组发送真正的 `SIGINT`;如果异步的写入前检查随后才结算,也无法执行该输入。被取消的发送会保留其槽位,直至前台信号发送结算,因此后续发送不会成为该信号的目标。取消绝不会通过写入 `\x03` 模拟中断,因此原始模式程序仍可取消。发送在提供方异步写入期间超时,或在该写入仍未完成时其取消信号发送失败,都会报告各自结果,但继续占用该槽位,直至写入结算,从而避免延迟到达的字节与后续发送交错。关闭操作启动由提供方负责的 TERM→KILL 全会话清理,并在终端结果之后等待完全停稳。清理失败不会缓存成永久拒绝的关闭操作;后续关闭会重试提供方操作。 +取消发送会先把排队输入标记为已取消,再请求终端句柄向当前前台进程组发送真正的 `SIGINT`;如果异步的写入前检查随后才结算,也无法执行该输入。被取消的发送会保留其槽位,直至前台信号发送结算,因此后续发送不会成为该信号的目标。取消绝不会通过写入 `\x03` 模拟中断,因此原始模式程序仍可取消。发送在提供方异步写入期间超时,或在该写入仍未完成时其取消信号发送失败,都会报告各自结果,但继续占用该槽位,直至写入结算,从而避免延迟到达的字节与后续发送交错。关闭操作会在关闭开始后拒绝新的公开信号请求,等待已经在途的公开信号请求全部结算,再启动由提供方负责的 TERM→KILL 全会话清理,并在终端结果之后等待完全停稳。清理失败不会缓存成永久拒绝的关闭操作;后续关闭会重试提供方操作。 ## 模型体验 diff --git a/packages/pty/pty-local/src/session.ts b/packages/pty/pty-local/src/session.ts index 65a10c36ea..08b42466aa 100644 --- a/packages/pty/pty-local/src/session.ts +++ b/packages/pty/pty-local/src/session.ts @@ -165,7 +165,7 @@ export class LocalPtySession implements PtyBackendSession { private activeTimer: NodeJS.Timeout | undefined private activeDeadlineTimer: NodeJS.Timeout | undefined private activeAbort: (() => void) | undefined - private readonly terminalOperations = new Set>() + private readonly terminalOperations = new Set>() private signaledOperation: LocalSendOperation | undefined private interrupting: LocalSendOperation | undefined private writing: LocalSendOperation | undefined @@ -247,10 +247,13 @@ export class LocalPtySession implements PtyBackendSession { /** Retain one contained provider operation until its asynchronous work finishes. */ private ownTerminalOperation(operation: Promise): void { + void this.trackTerminalOperation(operation) + } + + private trackTerminalOperation(operation: Promise): Promise { const tracked = operation.finally(() => { this.terminalOperations.delete(tracked) }) this.terminalOperations.add(tracked) - // beginSend(), pollReadiness(), and interrupt() contain their own boundary errors. - void tracked + return tracked } private async beginSend(operation: LocalSendOperation, request: PtySendRequest): Promise { @@ -319,8 +322,9 @@ export class LocalPtySession implements PtyBackendSession { } async signal(signal: PtySignal): Promise { + if (this.closing) throw new Error('PTY session is closing') if (this.active !== undefined) this.signaledOperation = this.active - const targetPgid = await this.terminal.signalForeground(signal) + const targetPgid = await this.trackTerminalOperation(this.terminal.signalForeground(signal)) return { delivered: true, targetPgid } } diff --git a/packages/pty/pty-local/tests/session.spec.ts b/packages/pty/pty-local/tests/session.spec.ts index 28ad58f04d..dc03da2a63 100644 --- a/packages/pty/pty-local/tests/session.spec.ts +++ b/packages/pty/pty-local/tests/session.spec.ts @@ -952,6 +952,26 @@ describe('LocalPtySession bounds, signals, and teardown', () => { await expect(session.signal('SIGTERM')).rejects.toThrow('cannot resolve') }) + it('drains an in-flight public signal and rejects signals after close starts', async () => { + const terminal = new FakeTerminal() + const session = new LocalPtySession(terminal, config()) + const signal = Promise.withResolvers() + terminal.signalForeground = async () => await signal.promise + + const signaling = session.signal('SIGINT') + const closing = session.close('public signal') + let closed = false + void closing.then(() => { closed = true }) + await Promise.resolve() + expect(closed).toBe(false) + await expect(session.signal('SIGTERM')).rejects.toThrow('closing') + + signal.resolve(456) + await expect(signaling).resolves.toEqual({ delivered: true, targetPgid: 456 }) + await closing + expect(closed).toBe(true) + }) + it('closes idempotently, contains signal races, and reports survivors', async () => { const terminal = new FakeTerminal() terminal.quiescent = false From 4fecc549981ebb572ad25bed32b78aa123d0dd78 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 13:51:55 +0800 Subject: [PATCH 039/108] fix(runtime): close teardown gaps and simplify framing --- .../2026-07-15-lsp-capability-seam.i18n.yaml | 4 +- .../2026-07-15-lsp-capability-seam.md | 2 +- .../2026-07-15-lsp-capability-seam.zh.md | 2 +- ...rtable-execution-world-consumers.i18n.yaml | 4 +- ...7-28-portable-execution-world-consumers.md | 6 +- ...8-portable-execution-world-consumers.zh.md | 6 +- .../feature/2026-06-15-code-mode.i18n.yaml | 4 +- .../feature/2026-06-15-code-mode.md | 2 +- .../feature/2026-06-15-code-mode.zh.md | 2 +- packages/lsp/lsp-local/README.i18n.yaml | 4 +- packages/lsp/lsp-local/README.md | 2 +- packages/lsp/lsp-local/README.zh.md | 2 +- packages/lsp/lsp-local/src/index.ts | 41 ++++++++++--- .../lsp/lsp-local/tests/lifecycle.spec.ts | 59 +++++++++++++++++++ .../subprocess-local/README.i18n.yaml | 4 +- .../subprocess/subprocess-local/README.md | 2 +- .../subprocess/subprocess-local/README.zh.md | 2 +- .../subprocess-local/src/terminal.ts | 6 +- .../subprocess-local/tests/terminal.spec.ts | 42 +++++++++++++ packages/typert/README.i18n.yaml | 4 +- 20 files changed, 164 insertions(+), 36 deletions(-) diff --git a/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.i18n.yaml index bba045d606..c2ed39b07d 100644 --- a/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md -2026-07-15-lsp-capability-seam.md: 6e46fac0611c1b7282766cf57965a07e3ee6bb9b -2026-07-15-lsp-capability-seam.zh.md: b535b60df70e8a6132bb1596581be3ab1daba932 +2026-07-15-lsp-capability-seam.md: 233a7321e1b4fded955144ed84fbc34df49e8964 +2026-07-15-lsp-capability-seam.zh.md: 547249fc22844878954fe7120dd8b5c87aabfc1a diff --git a/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md b/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md index 6e46fac061..233a7321e1 100644 --- a/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md +++ b/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md @@ -112,7 +112,7 @@ Provider disposal occurs outside tool execution, so `dsh-lsp-local` keeps `shutd ## Workspace, filesystem, and document synchronization -`dsh-lsp-local` canonicalizes and reads through `ctx.fs` in the language server's execution world. It requires the workspace target to be a directory, rejects out-of-workspace sources through provider-owned containment, and uses `readTextBounded` so regular-file validation, UTF-8 decoding, the byte ceiling, and path replacement/growth safety stay one filesystem operation. It observes caller cancellation around each provider operation. It does not emit `fs/observed`: only the LSP result is model-visible, so the query does not satisfy read-before-write policy. +`dsh-lsp-local` canonicalizes and reads through `ctx.fs` in the language server's execution world. It requires the workspace target to be a directory, rejects out-of-workspace sources through provider-owned containment, and uses `readTextBounded` so regular-file validation, UTF-8 decoding, the byte ceiling, and path replacement/growth safety stay one filesystem operation. It fuses caller cancellation with provider disposal across each filesystem operation, tracks workspace lookups before they enter a queue, and awaits those lookups during disposal. It does not emit `fs/observed`: only the LSP result is model-visible, so the query does not satisfy read-before-write policy. The `read` tool is unsuitable source because its output is windowed, numbered, transcript-visible, and observed. Reading in `tool-lsp` would also assign provider-specific synchronization to the consumer and preclude non-local providers. diff --git a/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.zh.md b/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.zh.md index b535b60df7..547249fc22 100644 --- a/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.zh.md @@ -112,7 +112,7 @@ interface LspToolInput { ## 工作区、文件系统与文档同步 -`dsh-lsp-local` 在语言服务器的执行环境中通过 `ctx.fs` 规范化并读取文件。它要求工作区目标是目录,使用提供方自有的 containment 拒绝工作区外的源文件,并通过 `readTextBounded` 把普通文件校验、UTF-8 解码、字节上限和路径替换/增长安全性保留在同一项文件系统操作中。它在每项提供方操作前后检查调用方是否取消。它不发送 `fs/observed`:只有 LSP 结果对模型可见,因此查询不满足写前读取策略。 +`dsh-lsp-local` 在语言服务器的执行环境中通过 `ctx.fs` 规范化并读取文件。它要求工作区目标是目录,使用提供方自有的 containment 拒绝工作区外的源文件,并通过 `readTextBounded` 把普通文件校验、UTF-8 解码、字节上限和路径替换/增长安全性保留在同一项文件系统操作中。它会针对每项文件系统操作合并调用方取消与提供方资源释放,跟踪尚未进入队列的工作区查找,并在资源释放期间等待这些查找结算。它不发送 `fs/observed`:只有 LSP 结果对模型可见,因此查询不满足写前读取策略。 `read` 工具的输出带窗口与行号,进入 transcript(文本记录)且已被观察,不适合作为源文件。在 `tool-lsp` 内读取还会把提供方专用同步职责交给消费方,并排除非本地提供方。 diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml index fecaacac05..de79bd2d8d 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md -2026-07-28-portable-execution-world-consumers.md: fe9d7bdd38199a031989b093fd3e9d931d62c80a -2026-07-28-portable-execution-world-consumers.zh.md: 9f4f38e6e4c6c7dba74e8de4c260ec633dade910 +2026-07-28-portable-execution-world-consumers.md: e943c54a3f3a4c93a95db999d4c60790457ba644 +2026-07-28-portable-execution-world-consumers.zh.md: 7755791d5763436866763e8d4ea9e0fab8203bb9 diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md index fe9d7bdd38..e943c54a3f 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md @@ -21,9 +21,9 @@ The subprocess interface owns the process coordinates and primitives: canonical Generic consumers use that execution world: - `dsh-bash-local` continues to map Bash semantics onto ordinary `ctx.subprocess.spawn()`. -- `dsh-lsp-local` reads and contains source through `ctx.fs`, resolves and launches language servers through `ctx.subprocess`, and carries provider-owned file URIs through initialization and result rendering. Its JSON-RPC, pooling, synchronization, cancellation, and normalization stay unchanged. +- `dsh-lsp-local` reads and contains source through `ctx.fs`, resolves and launches language servers through `ctx.subprocess`, and carries provider-owned file URIs through initialization and result rendering. One provider-lifetime signal aborts filesystem and protocol work during disposal, including workspace lookup before queue ownership; its JSON-RPC, pooling, synchronization, and normalization stay unchanged. - `dsh-pty-local` maps persistent-shell semantics onto `ctx.subprocess.spawnTerminal()`. The local `node-pty` and process-inspection implementation moves into `dsh-subprocess-local`; another subprocess provider supplies the same primitive. The allocation signal is detached before publication, while readiness initialization retains setup cancellation. Prompt and silence evidence collected during asynchronous pre-write inspection is discarded when the provider write begins. Cancellation retains the send reservation until asynchronous foreground signalling settles, so the signal cannot target a successor; if a signaled send returns `inferred_idle` before its prompt arrives, that marker remains attributed to the prior send instead of settling a successor after its echo. A timed-out asynchronous write, or a signal failure during that write, retains the reservation until the provider settles the write. Completion of a stale inspection resumes polling for the current send. Close rejects new public signals after shutdown begins and drains public signals already in flight before returning. -- `dsh-code-runtime-subprocess` materializes a dependency-free runner through `ctx.fs` and launches it through `ctx.subprocess`, preserving the Code Runtime binding and output contract across local or remote worlds. It shares host-side worker mechanics through the non-plugin `dsh-code-runtime-worker/runtime-host` subpath instead of copying them. Preparation carries one lifecycle signal through filesystem resolution, materialization, and executable lookup so disposal can abort a stalled provider. The heap-bounded worker rejects oversized binding frames before transfer, each outer hop enforces the same bound before serialization, and the launcher publishes an accepted terminal frame before reaping its controller so a descendant that inherits controller pipes cannot suppress completion; the host still awaits process-group quiescence. +- `dsh-code-runtime-subprocess` materializes a dependency-free runner through `ctx.fs` and launches it through `ctx.subprocess`, preserving the Code Runtime binding and output contract across local or remote worlds. It shares host-side worker mechanics through the non-plugin `dsh-code-runtime-worker/runtime-host` subpath instead of copying them. Preparation carries one lifecycle signal through filesystem resolution, materialization, and executable lookup so disposal can abort a stalled provider. The heap-bounded worker rejects oversized binding frames before transfer, each outer hop enforces the same bound before serialization, and raw subprocess pipes carry newline-delimited UTF-8 JSON without a redundant base64 representation. The launcher publishes an accepted terminal frame before reaping its controller so a descendant that inherits controller pipes cannot suppress completion; the host still awaits process-group quiescence. `dsh-code-runtime-worker` remains a separate implementation. It is the smaller in-process backend and works in single-file distributions that cannot assume an installed Node executable. Remote filesystem/process compositions select `dsh-code-runtime-subprocess`; they do not need a provider-specific Code Runtime package. @@ -45,4 +45,4 @@ A remote execution provider implements only its shared sandbox owner plus filesy The fundamental interfaces are wider, and a filesystem/subprocess pair must agree on one execution world. The added operations are limited to facts and lifecycle mechanics that current generic consumers require; model schemas, protocol framing, readiness policy, and presentation do not leak into the providers. -The local implementation absorbs `node-pty` and platform process inspection because it owns local terminal mechanics. This moves code without weakening terminal teardown: disposal waits for exact PID-identity-fenced descendants retained during foreground inspection, Linux session members that survive top-level exit, and the top-level terminal process to reach quiescence. macOS cannot enumerate a POSIX session after its leader exits, so a child that reparents between inspection snapshots remains an explicit local-provider limitation rather than a reason to move process mechanics back into the PTY consumer. +The local implementation absorbs `node-pty` and platform process inspection because it owns local terminal mechanics. This moves code without weakening terminal teardown: disposal sweeps descendants before and after terminating the top-level shell, waits for exact PID-identity-fenced descendants retained during foreground inspection, and retains Linux session members that survive top-level exit. macOS cannot enumerate a POSIX session after its leader exits, so a child that reparents between inspection snapshots remains an explicit local-provider limitation rather than a reason to move process mechanics back into the PTY consumer. diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md index 9f4f38e6e4..7755791d57 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md @@ -21,9 +21,9 @@ Status: implemented 通用消费方使用该执行世界: - `dsh-bash-local` 继续把 Bash 语义映射到普通的 `ctx.subprocess.spawn()`。 -- `dsh-lsp-local` 通过 `ctx.fs` 读取源文件并验证包含关系,通过 `ctx.subprocess` 解析和启动语言服务器,并让由提供方负责的文件 URI 贯穿初始化与结果渲染。其 JSON-RPC、池化、同步、取消和规范化保持不变。 +- `dsh-lsp-local` 通过 `ctx.fs` 读取源文件并验证包含关系,通过 `ctx.subprocess` 解析和启动语言服务器,并让由提供方负责的文件 URI 贯穿初始化与结果渲染。一个提供方生命周期信号会在资源释放期间中止文件系统与协议操作,包括取得队列所有权之前的工作区查找;其 JSON-RPC、池化、同步和规范化保持不变。 - `dsh-pty-local` 把持久 shell 语义映射到 `ctx.subprocess.spawnTerminal()`。本地 `node-pty` 与进程检查实现移入 `dsh-subprocess-local`;其他进程管理提供方则提供相同原语。分配信号会在发布前解除关联,而就绪初始化仍保留设置阶段的取消。提供方开始写入时,系统会丢弃异步写入前检查期间收集的提示符与静默证据。取消会保留发送预留,直至异步前台信号发送结算,从而使该信号无法把后续发送作为目标;如果收到信号的 send 在其提示符到达前返回 `inferred_idle`,随后到达的标记仍归属于先前的 send,而不会在后续 send 回显后使其完成。异步写入超时,或该写入期间发生信号发送失败,都会保留预留,直至提供方将写入结算。陈旧检查完成后,会针对当前发送恢复轮询。关闭操作开始后会拒绝新的公开信号,并在返回前等待所有已在途的公开信号结算。 -- `dsh-code-runtime-subprocess` 通过 `ctx.fs` 物化无依赖 runner,并通过 `ctx.subprocess` 启动它,从而在本地或远程执行世界中保留代码运行时的绑定与输出契约。它通过非插件子路径 `dsh-code-runtime-worker/runtime-host` 共享宿主侧 worker 机制,而不是复制这些机制。准备阶段让同一个生命周期信号贯穿文件系统解析、物化和可执行文件查找,使资源释放能够中止停滞的提供方操作。受堆上限约束的 worker 会在传输前拒绝过大的绑定帧;每个外层转发环节都会在序列化前执行相同的上限检查;launcher 会在回收 controller 前发布已接纳的终态帧,使继承 controller 管道的后代进程无法阻止完成;宿主仍会等待进程组完全停稳。 +- `dsh-code-runtime-subprocess` 通过 `ctx.fs` 物化无依赖 runner,并通过 `ctx.subprocess` 启动它,从而在本地或远程执行世界中保留代码运行时的绑定与输出契约。它通过非插件子路径 `dsh-code-runtime-worker/runtime-host` 共享宿主侧 worker 机制,而不是复制这些机制。准备阶段让同一个生命周期信号贯穿文件系统解析、物化和可执行文件查找,使资源释放能够中止停滞的提供方操作。受堆上限约束的 worker 会在传输前拒绝过大的绑定帧;每个外层转发环节都会在序列化前执行相同的上限检查;原始子进程管道承载以换行符分隔的 UTF-8 JSON,无需冗余的 base64 表示;launcher 会在回收 controller 前发布已接纳的终态帧,使继承 controller 管道的后代进程无法阻止完成;宿主仍会等待进程组完全停稳。 `dsh-code-runtime-worker` 仍是独立实现。它是较小的进程内后端,可用于无法假定已安装 Node 可执行文件的单文件分发。远程文件系统/进程组合选择 `dsh-code-runtime-subprocess`;它们不需要提供方专用的代码运行时包。 @@ -45,4 +45,4 @@ Status: implemented 基础接口更宽,一对文件系统/进程管理提供方必须在同一个执行世界上保持一致。新增操作仅限当前通用消费方所需的事实与生命周期机制;模型 schema、协议分帧、就绪策略和呈现不会渗入提供方。 -本地实现承接 `node-pty` 和平台进程检查,因为它负责本地终端机制。这种代码迁移不会削弱终端拆卸:dispose(资源释放)会等待前台检查期间保留下来且受精确 PID 身份围栏保护的后代进程、在顶层进程退出后仍存活的 Linux 会话成员,以及顶层终端进程完全停稳。macOS 无法在 POSIX 会话 leader 退出后枚举该会话,因此在两次检查快照之间重新设定父进程的子进程仍是明确的本地提供方限制,而不是把进程机制移回 PTY 消费方的理由。 +本地实现承接 `node-pty` 和平台进程检查,因为它负责本地终端机制。这种代码迁移不会削弱终端拆卸:dispose(资源释放)会在终止顶层 shell 前后清理后代进程,等待前台检查期间保留下来且受精确 PID 身份围栏保护的后代进程,并继续追踪在顶层进程退出后仍存活的 Linux 会话成员。macOS 无法在 POSIX 会话 leader 退出后枚举该会话,因此在两次检查快照之间重新设定父进程的子进程仍是明确的本地提供方限制,而不是把进程机制移回 PTY 消费方的理由。 diff --git a/.agents/notes/implemented/feature/2026-06-15-code-mode.i18n.yaml b/.agents/notes/implemented/feature/2026-06-15-code-mode.i18n.yaml index 42a11b8fb8..112e4eed83 100644 --- a/.agents/notes/implemented/feature/2026-06-15-code-mode.i18n.yaml +++ b/.agents/notes/implemented/feature/2026-06-15-code-mode.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-06-15-code-mode.md -2026-06-15-code-mode.md: b0778b2c9a8dfce8f786faccf6cc47f477c6b7c0 -2026-06-15-code-mode.zh.md: 437692eac3fc6740e0f1d4dd9001130fd6dd448a +2026-06-15-code-mode.md: 1be25f5b145e50d8fe391d22910b71d85181582e +2026-06-15-code-mode.zh.md: a4e8fbbd339899937c4f39091b4befbbb78ec347 diff --git a/.agents/notes/implemented/feature/2026-06-15-code-mode.md b/.agents/notes/implemented/feature/2026-06-15-code-mode.md index b0778b2c9a..1be25f5b14 100644 --- a/.agents/notes/implemented/feature/2026-06-15-code-mode.md +++ b/.agents/notes/implemented/feature/2026-06-15-code-mode.md @@ -79,7 +79,7 @@ Requests contain every runtime input; implementations own validated timeout and 5. **Enforce independent budgets.** `computeMs` meters worker busy time, allowing slow awaited tools without excusing a hot loop. `maxWallMs` bounds total elapsed time, including unresolved waits. `maxOutputBytes` bounds only the combined serialized outer logs, completion, or diagnostic; intermediate binding values have no byte cap. Expiry, cancellation, and completion terminate the worker, and heap exits or outer overflow are explicit failures. 6. **Dispose to quiescence**: the service's own disposal terminates in-flight workers and *awaits* their exits before resolving, per [defensive patterns](../../../../docs/defensive-patterns.md). -`@deepseek-ai/dsh-code-runtime-subprocess` preserves those program, binding, output, and worker-budget semantics across a filesystem/subprocess execution world. It writes a dependency-free runner below `ctx.subprocess.runtimeRoot`, resolves Node through the provider, and carries binding traffic over bounded base64 JSON frames on raw pipes. The heap-bounded worker rejects expanded completion wires before MessagePort transfer; terminal settlement asks the launcher to reap its controller and keeps process-group escalation armed until `waitForExit()` confirms whole-tree quiescence. The [portable execution-world decision](../architecture/2026-07-28-portable-execution-world-consumers.md) owns why this generic backend replaces provider-specific Code Runtime packages; `dsh-code-runtime-worker` remains the single-process and single-file-distribution path. +`@deepseek-ai/dsh-code-runtime-subprocess` preserves those program, binding, output, and worker-budget semantics across a filesystem/subprocess execution world. It writes a dependency-free runner below `ctx.subprocess.runtimeRoot`, resolves Node through the provider, and carries binding traffic over bounded newline-delimited UTF-8 JSON frames on raw pipes. The heap-bounded worker rejects expanded completion wires before MessagePort transfer; terminal settlement asks the launcher to reap its controller and keeps process-group escalation armed until `waitForExit()` confirms whole-tree quiescence. The [portable execution-world decision](../architecture/2026-07-28-portable-execution-world-consumers.md) owns why this generic backend replaces provider-specific Code Runtime packages; `dsh-code-runtime-worker` remains the single-process and single-file-distribution path. ### Trust posture diff --git a/.agents/notes/implemented/feature/2026-06-15-code-mode.zh.md b/.agents/notes/implemented/feature/2026-06-15-code-mode.zh.md index 437692eac3..a4e8fbbd33 100644 --- a/.agents/notes/implemented/feature/2026-06-15-code-mode.zh.md +++ b/.agents/notes/implemented/feature/2026-06-15-code-mode.zh.md @@ -79,7 +79,7 @@ Cloudflare 的 [Code Mode](https://blog.cloudflare.com/code-mode/) 提出了一 5. **强制独立预算。** `computeMs` 计量 worker 忙碌时间,允许慢速的 awaited 工具而不放过热循环。`maxWallMs` 约束总经过时间,包括未解析的等待。`maxOutputBytes` 只约束序列化后的外层日志、完成值或诊断的组合;中间绑定值没有字节数上限。到期、取消和完成都终止 worker,堆退出或外层溢出会作为显式失败报告。 6. **dispose 至完全停稳**:服务自身的 dispose(资源释放)终止进行中的 worker 并*等待*其退出后再 resolve,遵循[防御性模式](../../../../docs/defensive-patterns.md)。 -`@deepseek-ai/dsh-code-runtime-subprocess` 在文件系统/子进程执行环境中保持相同的程序、绑定、输出与 worker 预算语义。它在 `ctx.subprocess.runtimeRoot` 下写入一个无依赖 runner,通过提供方解析 Node,并在原始管道上使用有界 base64 JSON 帧承载绑定通信。受堆上限约束的 worker 会在通过 MessagePort 传输前拒绝展开后的完成值 wire;终态结算会请求 launcher 回收其 controller,并让进程组升级终止机制保持待命,直至 `waitForExit()` 确认整棵进程树完全停稳。[可移植执行环境决策](../architecture/2026-07-28-portable-execution-world-consumers.md)说明为何这个通用后端会取代提供方专用的 Code Runtime 包;`dsh-code-runtime-worker` 仍用于单进程和单文件发行版。 +`@deepseek-ai/dsh-code-runtime-subprocess` 在文件系统/子进程执行环境中保持相同的程序、绑定、输出与 worker 预算语义。它在 `ctx.subprocess.runtimeRoot` 下写入一个无依赖 runner,通过提供方解析 Node,并在原始管道上使用有界的、以换行符分隔的 UTF-8 JSON 帧承载绑定通信。受堆上限约束的 worker 会在通过 MessagePort 传输前拒绝展开后的完成值 wire;终态结算会请求 launcher 回收其 controller,并让进程组升级终止机制保持待命,直至 `waitForExit()` 确认整棵进程树完全停稳。[可移植执行环境决策](../architecture/2026-07-28-portable-execution-world-consumers.md)说明为何这个通用后端会取代提供方专用的 Code Runtime 包;`dsh-code-runtime-worker` 仍用于单进程和单文件发行版。 ### 信任姿态 diff --git a/packages/lsp/lsp-local/README.i18n.yaml b/packages/lsp/lsp-local/README.i18n.yaml index e2b67d46d9..5a1036a349 100644 --- a/packages/lsp/lsp-local/README.i18n.yaml +++ b/packages/lsp/lsp-local/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/lsp/lsp-local/README.md -README.md: 299a5282bbca0a6d517af9342c4101cbdae33d4c -README.zh.md: edf706fdb424756aa43b303b018b11e3764c2559 +README.md: c96c4febc9d047b41789f2b7a73e3eaf4d35012b +README.zh.md: 5bc2c3c8bb7a89afb673797f5a3b25bb9fc06748 diff --git a/packages/lsp/lsp-local/README.md b/packages/lsp/lsp-local/README.md index 299a5282bb..c96c4febc9 100644 --- a/packages/lsp/lsp-local/README.md +++ b/packages/lsp/lsp-local/README.md @@ -11,7 +11,7 @@ Namespace plugin (`name` / `inject` / `Config` / `apply`, no default export). - Resolves every server-local setting before registration; an invalid mapping or registration conflict rolls back earlier entries, so a failed load leaves no provider routes. - Lazily single-flights one server process per `(server id, canonical workspace target)`. A live server error is not replayed; if the selected pooled transport fails before or during a read-only query, the provider awaits its disposal and retries that query once on a fresh process. - Uses a compatibility-first **transient-open** sequence per query: resolve and boundedly read the source through `ctx.fs`, `textDocument/didOpen` (version 1, full text), the requested request, then `textDocument/didClose` in `finally`. A failed or canceled `didOpen` write terminates the instance before the pool can reuse it. Documents close after each call, so the first version needs no `didChange`, content cache, or document LRU. -- Serializes each source-read/open/query/close lifecycle through one abortable per-workspace queue so queued calls read current source only when their turn starts; distinct workspaces run in parallel. +- Serializes each source-read/open/query/close lifecycle through one abortable per-workspace queue so queued calls read current source only when their turn starts; distinct workspaces run in parallel. Provider disposal aborts filesystem and protocol work, awaits workspace lookups that have not entered a queue, then drains every queue and server. - After protocol shutdown fails, terminates the server's descendant tree through the subprocess seam (POSIX process-group signaling; Windows `taskkill /T /F`). Tree-kill delivery is contained like every group signal — it races server exit — and quiescence is confirmed by the handle's tree-liveness wait rather than by the kill's own outcome. - Resolves the server executable, cwd, process, and protocol streams through `ctx.subprocess`; `initialize.processId` is `null` because another machine or PID namespace must not monitor the harness process. - Uses `ctx.fs` canonical containment, file URIs, and stable bounded reads, but emits no `fs/observed`: only the LSP result is model-visible, so a query does not satisfy read-before-write policy. diff --git a/packages/lsp/lsp-local/README.zh.md b/packages/lsp/lsp-local/README.zh.md index edf706fdb4..5bc2c3c8bb 100644 --- a/packages/lsp/lsp-local/README.zh.md +++ b/packages/lsp/lsp-local/README.zh.md @@ -11,7 +11,7 @@ Namespace 插件(`name`/`inject`/`Config`/`apply`,无默认导出) - 在注册前解析每项服务器局部设置;无效映射或注册冲突会回滚较早配置项,因此加载失败不会留下提供方路由。 - 每个 `(server id, canonical workspace target)` 惰性 single-flight 一个服务器进程。存活服务器错误不会回放;如果选中的池化传输在只读查询之前或期间失败,提供方会等待其释放,并在新进程上重试该查询一次。 - 每次查询都使用兼容性优先的**临时打开** 序列:通过 `ctx.fs` 解析源文件并进行有界读取、`textDocument/didOpen`(版本 1、完整文本)、所请求操作,然后执行 `textDocument/didClose`,该操作位于 `finally` 中。写入 `didOpen` 失败或取消时,会先终止实例再允许池复用。文档在每次调用后关闭,因此第一版不需要 `didChange`、内容 cache 或文档 LRU。 -- 通过一条逐 Workspace、可中止的队列,串行执行每个源读取/打开/查询/关闭生命周期,因此排队调用只会在轮到自身时读取当前源;不同 Workspace 并行运行。 +- 通过一条逐 Workspace、可中止的队列,串行执行每个源读取/打开/查询/关闭生命周期,因此排队调用只会在轮到自身时读取当前源;不同 Workspace 并行运行。提供方资源释放会中止文件系统与协议工作,等待尚未进入队列的 Workspace 查找结算,再排空所有队列并等待所有服务器结算。 - 协议 shutdown 失败后,经由进程管理器 seam 终止服务器后代树(POSIX 进程组信号;Windows `taskkill /T /F`)。树终止的投递结果与所有进程组信号一样被就地吸收,不向外抛出(投递与服务器退出存在竞态);服务器是否完全停稳,由句柄的进程树存活等待确认,而非由这次终止自身的结果确认。 - 通过 `ctx.subprocess` 解析服务器可执行文件、cwd、进程与协议流;`initialize.processId` 为 `null`,因为另一台机器或 PID 命名空间不得监控 harness 进程。 - 使用 `ctx.fs` 提供的规范 containment、文件 URI 与稳定有界读取,但不发出 `fs/observed`:只有 LSP 结果对模型可见,因此查询不满足先读后写策略。 diff --git a/packages/lsp/lsp-local/src/index.ts b/packages/lsp/lsp-local/src/index.ts index 0d511588bd..f3f45959e5 100644 --- a/packages/lsp/lsp-local/src/index.ts +++ b/packages/lsp/lsp-local/src/index.ts @@ -208,6 +208,9 @@ class LocalLspProvider implements LspProvider { private readonly instances = new Map() /** One complete source-read→open→query→close serialization tail per canonical workspace. */ private readonly queues = new Map>() + /** Workspace canonicalizations that have not entered a provider-owned queue yet. */ + private readonly workspaceLookups = new Set>() + private readonly lifetime = new AbortController() private disposed = false constructor( @@ -234,32 +237,48 @@ class LocalLspProvider implements LspProvider { if (signal?.aborted) throw abortError(signal) } + /** Fuse caller cancellation with provider disposal for every filesystem and protocol await. */ + private querySignal(signal?: AbortSignal): AbortSignal { + return signal === undefined + ? this.lifetime.signal + : AbortSignal.any([signal, this.lifetime.signal]) + } + async query(request: LspProviderQuery, signal?: AbortSignal): Promise { // Honor an already-aborted signal before provider I/O so a canceled request never starts a server. this.assertActive(signal) - const workspace = await canonicalizeWorkspace(this.fs, request.workspaceRoot, signal) - this.assertActive(signal) + const querySignal = this.querySignal(signal) + const workspaceResult = canonicalizeWorkspace(this.fs, request.workspaceRoot, querySignal) + const workspaceLookup = workspaceResult.then(() => undefined, () => undefined) + this.workspaceLookups.add(workspaceLookup) + let workspace: HostWorkspace + try { + workspace = await workspaceResult + } finally { + this.workspaceLookups.delete(workspaceLookup) + } + this.assertActive(querySignal) const workspaceKey = workspace.target.targetKey - return this.enqueue(workspaceKey, signal, async () => { - this.assertActive(signal) + return this.enqueue(workspaceKey, querySignal, async () => { + this.assertActive(querySignal) // Read inside the workspace queue but before spawning: a queued query sees current bytes when // its turn starts, while an invalid source still cannot leave an idle process pooled. - const source = await readHostSource(this.fs, request.filePath, workspace, this.config.maxDocumentBytes, signal) + const source = await readHostSource(this.fs, request.filePath, workspace, this.config.maxDocumentBytes, querySignal) // Disposal may have snapshotted the instance map while host I/O was pending. Re-check before a // synchronous get-or-create so every spawned process remains owned by teardown. - this.assertActive(signal) + this.assertActive(querySignal) let instance = this.instanceFor(workspaceKey, workspace) try { - return await instance.query(request, source, signal) + return await instance.query(request, source, querySignal) } catch (error) { // A selected child can have died while idle or fail during the next write. Queries are // read-only, so replace that transport once and retry transparently. if (!instance.isTransportFailure(error)) throw error await instance.dispose() this.evictIfCurrent(workspaceKey, instance) - this.assertActive(signal) + this.assertActive(querySignal) instance = this.instanceFor(workspaceKey, workspace) - return await instance.query(request, source, signal) + return await instance.query(request, source, querySignal) } finally { // Reach quiescence before dropping a dead slot; a replacement must survive this ownership check. if (instance.dead) { @@ -320,13 +339,17 @@ class LocalLspProvider implements LspProvider { /** Dispose every live instance and block further queries. */ async disposeAll(): Promise { this.disposed = true + this.lifetime.abort(new LspError('lsp-local provider is disposed', 'LSP_DISPOSED')) const live = [...this.instances.values()] const draining = [...this.queues.values()] + const resolving = [...this.workspaceLookups] this.instances.clear() await Promise.all([ ...live.map(instance => instance.dispose()), ...draining, + ...resolving, ]) this.queues.clear() + this.workspaceLookups.clear() } } diff --git a/packages/lsp/lsp-local/tests/lifecycle.spec.ts b/packages/lsp/lsp-local/tests/lifecycle.spec.ts index ce92e2c8fb..0337b0fec2 100644 --- a/packages/lsp/lsp-local/tests/lifecycle.spec.ts +++ b/packages/lsp/lsp-local/tests/lifecycle.spec.ts @@ -301,6 +301,52 @@ describe('lsp-local end to end over a fake server', () => { await ctx.fiber.dispose() }) + it('aborts and awaits a workspace lookup when the provider is disposed', async () => { + const ctx = await mount({ LSP_FAKE_DEF: 'null' }) + const fs = ctx.fs + const resolve = fs.resolve.bind(fs) + const started = Promise.withResolvers() + const release = Promise.withResolvers() + vi.spyOn(fs, 'resolve').mockImplementation(async (path, options) => { + if (path !== ws) return await resolve(path, options) + const signal = options?.signal + if (signal === undefined) throw new Error('workspace lookup missing provider lifetime signal') + started.resolve(signal) + return await rejectWhenAborted(signal, release.promise) + }) + + const pending = ctx.lsp.query(query('goToDefinition')) + const signal = await started.promise + let disposed = false + const disposing = ctx.fiber.dispose().then(() => { disposed = true }) + await new Promise(resolve => setImmediate(resolve)) + + expect(signal.aborted).toBe(true) + expect(disposed).toBe(false) + release.resolve(undefined) + await expect(pending).rejects.toThrow('provider is disposed') + await expect(disposing).resolves.toBeUndefined() + }) + + it('aborts a queued source read when the provider is disposed', async () => { + const ctx = await mount({ LSP_FAKE_DEF: 'null' }) + const fs = ctx.fs + const started = Promise.withResolvers() + vi.spyOn(fs, 'readTextBounded').mockImplementation(async (_target, _maxBytes, signal) => { + if (signal === undefined) throw new Error('source read missing provider lifetime signal') + started.resolve(signal) + return await rejectWhenAborted(signal) + }) + + const pending = ctx.lsp.query(query('goToDefinition')) + const signal = await started.promise + const disposing = ctx.fiber.dispose() + + await expect(pending).rejects.toThrow('provider is disposed') + await expect(disposing).resolves.toBeUndefined() + expect(signal.aborted).toBe(true) + }) + it('runs distinct workspaces in parallel instances', async () => { const ws2 = join(root, 'ws2') await mkdir(ws2) @@ -358,3 +404,16 @@ async function waitFor(condition: () => Promise, timeoutMs = 3000): Pro await new Promise(resolve => setTimeout(resolve, 10)) } } + +/** Hold one fake provider operation until cancellation, optionally behind a cleanup gate. */ +function rejectWhenAborted(signal: AbortSignal, release: Promise = Promise.resolve()): Promise { + return new Promise((_resolve, reject) => { + const onAbort = (): void => { + void release.then(() => { + reject(signal.reason instanceof Error ? signal.reason : new Error(String(signal.reason))) + }) + } + signal.addEventListener('abort', onAbort, { once: true }) + if (signal.aborted) onAbort() + }) +} diff --git a/packages/subprocess/subprocess-local/README.i18n.yaml b/packages/subprocess/subprocess-local/README.i18n.yaml index efbb1612bd..ded1d12b30 100644 --- a/packages/subprocess/subprocess-local/README.i18n.yaml +++ b/packages/subprocess/subprocess-local/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/subprocess/subprocess-local/README.md -README.md: 61ee8ff9e09d5177ff6d5b3805dd84fb74c59961 -README.zh.md: 06a8e4bdd225e212aa67817ebbf66581314ab2bf +README.md: 9f30fa6dc676e7b82f87b75b78b7d3143f204c94 +README.zh.md: 0b6813ec7d754f42e8bf4c65a1dee33d77bf3787 diff --git a/packages/subprocess/subprocess-local/README.md b/packages/subprocess/subprocess-local/README.md index 61ee8ff9e0..9f30fa6dc6 100644 --- a/packages/subprocess/subprocess-local/README.md +++ b/packages/subprocess/subprocess-local/README.md @@ -11,7 +11,7 @@ Local implementation of the [`@deepseek-ai/dsh-subprocess`](../subprocess/README - **Credential scrub + explicit merge** — `process.env` minus credential-shaped vars (`*KEY*`/`*SECRET*`/`*TOKEN*`) and all ambient `DSH_*` names; the spec's explicit `env` merges after that scrub with no namespace validation, so a deliberately supplied credential or current `DSH_*` fact wins while stale nested-harness identity cannot leak in ambiently. Supplied stdin is written and closed; otherwise fd 0 is `/dev/null`. See the [stdin/env Agent Note](../../../.agents/notes/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-surface.md) and [managed environment Agent Note](../../../.agents/notes/implemented/feature/2026-07-10-agent-session-identity-and-log-location.md). - **Offset-based reads** — collect-mode readers return deltas in whole-stream byte coordinates; the service never holds a cursor, so consumer-owned cursors (the bash background read path) and full-stream re-reads coexist, before and after settlement. - **Execution-world coordinates** — `cwd` is the host process cwd, `runtimeRoot` is an owner-private temporary directory removed on disposal before any process-cleanup failure is reported, and `resolveExecutable` checks absolute files or searches the scrubbed effective PATH with platform-aware executable extensions. -- **Terminal-process ownership** — `spawnTerminal` allocates `node-pty`, bridges UTF-8 terminal bytes, inspects and signals the current foreground process group, and cleans descendants before the top-level shell. Each foreground inspection retains exact identities from the rooted tree; Linux also enumerates the POSIX session after its leader exits. A previously observed macOS descendant and any same-session Linux member therefore remain fenced after reparenting, while pid/start identity prevents cleanup from following PID reuse. The higher PTY backend owns prompt readiness, buffers, and model-facing operations. +- **Terminal-process ownership** — `spawnTerminal` allocates `node-pty`, bridges UTF-8 terminal bytes, inspects and signals the current foreground process group, and sweeps descendants before and after terminating the top-level shell. Each foreground inspection retains exact identities from the rooted tree; Linux also enumerates the POSIX session after its leader exits. A previously observed macOS descendant and any same-session Linux member therefore remain fenced after reparenting, while pid/start identity prevents cleanup from following PID reuse. The higher PTY backend owns prompt readiness, buffers, and model-facing operations. - **Terminate-and-join disposal** — the service retains live handles only so its own disposal can escalate every running tree and await its exit; settled and spawn-failed handles leave the live set on settlement. ## Model Experience diff --git a/packages/subprocess/subprocess-local/README.zh.md b/packages/subprocess/subprocess-local/README.zh.md index 06a8e4bdd2..0b6813ec7d 100644 --- a/packages/subprocess/subprocess-local/README.zh.md +++ b/packages/subprocess/subprocess-local/README.zh.md @@ -11,7 +11,7 @@ - **凭据清除 + 显式合并**:以 `process.env` 为基础,移除形似凭据的变量(`*KEY*`/`*SECRET*`/`*TOKEN*`)和所有环境中已有的 `DSH_*` 名称;spec 的显式 `env` 在该清除之后合并且不做命名空间校验,因此有意提供的凭据或当前 `DSH_*` 事实会胜出,而陈旧的嵌套 harness 身份无法从环境中隐式漏入。提供的 stdin 会被写入后关闭;否则 fd 0 指向 `/dev/null`。参见 [stdin/env Agent Note(agent 决策记录)](../../../.agents/notes/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-surface.md)与[受管环境 Agent Note](../../../.agents/notes/implemented/feature/2026-07-10-agent-session-identity-and-log-location.md)。 - **基于偏移量的读取**:收集模式的读取器以全流字节坐标返回增量;服务自身从不持有游标,因此消费方自有的游标(bash 的后台读取路径)与完整流重读可以共存,结算前后皆然。 - **执行世界坐标**:`cwd` 是宿主进程 cwd,`runtimeRoot` 是所有者私有的临时目录,会在资源释放时删除,并且删除发生在报告任何进程清理失败之前;`resolveExecutable` 检查绝对文件,或使用平台感知的可执行扩展名在清理后的有效 PATH 中查找。 -- **终端进程所有权**:`spawnTerminal` 分配 `node-pty`,桥接 UTF-8 终端字节,检查当前前台进程组并向其发送信号,并先于顶层 shell 清理后代。每次前台检查都会保留有根进程树中的精确身份;Linux 还会在会话 leader 退出后枚举该 POSIX 会话。因此,先前观察到的 macOS 后代以及任何同会话 Linux 成员在重新设定父进程后仍受身份围栏保护,而 pid/启动身份可防止清理因 PID 复用而跟随到其他进程。上层 PTY 后端负责提示符就绪检测、缓冲和面向模型的操作。 +- **终端进程所有权**:`spawnTerminal` 分配 `node-pty`,桥接 UTF-8 终端字节,检查当前前台进程组并向其发送信号,并在终止顶层 shell 前后清理后代。每次前台检查都会保留有根进程树中的精确身份;Linux 还会在会话 leader 退出后枚举该 POSIX 会话。因此,先前观察到的 macOS 后代以及任何同会话 Linux 成员在重新设定父进程后仍受身份围栏保护,而 pid/启动身份可防止清理因 PID 复用而跟随到其他进程。上层 PTY 后端负责提示符就绪检测、缓冲和面向模型的操作。 - **先终止再等待退出的 dispose**:服务保留存活句柄,只为让自身的 dispose 能对每个仍在运行的进程树执行升级并等待其退出;已结算与 spawn 失败的句柄在结算时即离开存活集合。 ## 模型体验 diff --git a/packages/subprocess/subprocess-local/src/terminal.ts b/packages/subprocess/subprocess-local/src/terminal.ts index 2f1e4d4058..30e38bd422 100644 --- a/packages/subprocess/subprocess-local/src/terminal.ts +++ b/packages/subprocess/subprocess-local/src/terminal.ts @@ -193,11 +193,15 @@ export class LocalTerminalHandle implements SubprocessTerminalHandle { } private async closeOnce(): Promise { - const survivors = await this.stopDescendants() + let survivors = await this.stopDescendants() if (survivors.length > 0) { throw new Error(`terminal cleanup failed; surviving pids: ${survivors.map(member => member.pid).join(', ')}`) } await this.stopShell() + survivors = await this.stopDescendants() + if (survivors.length > 0) { + throw new Error(`terminal cleanup failed; surviving pids: ${survivors.map(member => member.pid).join(', ')}`) + } this.dataDisposable.dispose() this.exitDisposable.dispose() } diff --git a/packages/subprocess/subprocess-local/tests/terminal.spec.ts b/packages/subprocess/subprocess-local/tests/terminal.spec.ts index 0202a3f33e..456ce4ee8f 100644 --- a/packages/subprocess/subprocess-local/tests/terminal.spec.ts +++ b/packages/subprocess/subprocess-local/tests/terminal.spec.ts @@ -13,6 +13,7 @@ class FakePty { readonly kills: string[] = [] autoExitOnKill = true throwKill = false + onKill?: () => void private readonly dataListeners = new Set<(data: string) => void>() private readonly exitListeners = new Set<(event: { exitCode: number; signal?: number }) => void>() @@ -39,6 +40,7 @@ class FakePty { kill(signal?: string): void { if (this.throwKill) throw new Error('process raced') this.kills.push(signal ?? 'SIGHUP') + this.onKill?.() if (this.autoExitOnKill) this.emitExit(0, signal === 'SIGKILL' ? 9 : 15) } @@ -213,6 +215,46 @@ describe('LocalTerminalHandle', () => { expect(pty.kills).toEqual(['SIGTERM']) }) + it('sweeps a same-session descendant forked while the shell handles TERM', async () => { + const pty = new FakePty() + const inspector = new FakeInspector() + const late = { pid: 124, started: 'shell-term-trap' } + pty.onKill = () => { + inspector.sessionMembers = [late] + inspector.alive.add(late.pid) + } + const handle = new LocalTerminalHandle(pty.asPty(), inspector, 10) + + handle.terminate() + await handle.waitForExit() + + expect(inspector.processes).toEqual([[late.pid, 'SIGTERM']]) + expect(pty.kills).toEqual(['SIGTERM']) + }) + + it('keeps a failed post-shell sweep retryable until its survivor leaves', async () => { + vi.useFakeTimers() + const pty = new FakePty() + const inspector = new FakeInspector() + const late = { pid: 124, started: 'shell-term-survivor' } + inspector.removeOnSignal = false + pty.onKill = () => { + inspector.sessionMembers = [late] + inspector.alive.add(late.pid) + } + const handle = new LocalTerminalHandle(pty.asPty(), inspector, 10) + + handle.terminate() + const failed = expect(handle.waitForExit()).rejects.toThrow('surviving pids: 124') + await vi.advanceTimersByTimeAsync(25) + await failed + + inspector.alive.delete(late.pid) + handle.terminate() + expect(await handle.waitForExit()).toBe(true) + expect(inspector.processes).toEqual([[late.pid, 'SIGTERM'], [late.pid, 'SIGKILL']]) + }) + it('retains captured descendants after reparenting', async () => { vi.useFakeTimers() const pty = new FakePty() diff --git a/packages/typert/README.i18n.yaml b/packages/typert/README.i18n.yaml index 13b8df9f71..7a36ab0690 100644 --- a/packages/typert/README.i18n.yaml +++ b/packages/typert/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/code-runtime/code-runtime-subprocess/README.md -README.md: fd8cfccdb333018e551459195564daad157d3867 -README.zh.md: b5b495e22bbf4328d82925cc793588ee45e5c6e4 +README.md: 38ee201a1754c6f50b7fae60b77af7dae734c8f9 +README.zh.md: 6fec39de550b7e3f57cd613dbe23c9687cd840ac From c1d550de5809abea2a5db920c02396d412a1f013 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 18:26:21 +0800 Subject: [PATCH 040/108] refactor(runtime): collapse speculative portability layers Remove the one-consumer bounded-read primitive and shared terminal lifecycle controller, make terminal cleanup one awaited provider operation, and reuse one Code Runtime contract suite. Keep only reproduced cancellation and policy fixes; defer unproven replacement, prompt-attribution, and streaming-frame concerns to scoped markers. --- ...06-17-filesystem-capability-seam.i18n.yaml | 4 +- .../2026-06-17-filesystem-capability-seam.md | 4 +- ...026-06-17-filesystem-capability-seam.zh.md | 4 +- .../2026-07-15-lsp-capability-seam.i18n.yaml | 4 +- .../2026-07-15-lsp-capability-seam.md | 8 +- .../2026-07-15-lsp-capability-seam.zh.md | 8 +- ...rtable-execution-world-consumers.i18n.yaml | 4 +- ...7-28-portable-execution-world-consumers.md | 12 +- ...8-portable-execution-world-consumers.zh.md | 12 +- docs/config-catalog.md | 2 +- .../core-data-structures/filesystem.i18n.yaml | 4 +- docs/core-data-structures/filesystem.md | 4 +- docs/core-data-structures/filesystem.zh.md | 4 +- .../core-data-structures/subprocess.i18n.yaml | 4 +- docs/core-data-structures/subprocess.md | 4 +- docs/core-data-structures/subprocess.zh.md | 4 +- .../code-runtime-worker/src/runtime-host.ts | 1 - .../code-runtime/tests/contract.ts | 838 ++++++++++++++++++ .../tests/workspace-context.spec.ts | 6 - .../cordis/tool-cordis/src/api-catalog.ts | 8 +- packages/fs/fs-local/README.i18n.yaml | 4 +- packages/fs/fs-local/README.md | 2 +- packages/fs/fs-local/README.zh.md | 2 +- packages/fs/fs-local/src/fsio.ts | 65 +- packages/fs/fs-local/src/index.ts | 5 - packages/fs/fs-local/tests/filesystem.spec.ts | 9 - packages/fs/fs-local/tests/fsio.spec.ts | 63 +- packages/fs/fs/README.i18n.yaml | 4 +- packages/fs/fs/README.md | 9 +- packages/fs/fs/README.zh.md | 9 +- packages/fs/fs/src/index.ts | 17 +- packages/fs/fs/tests/service.spec.ts | 5 - packages/fs/tool-fs/tests/tools.spec.ts | 5 - packages/lsp/lsp-local/README.i18n.yaml | 4 +- packages/lsp/lsp-local/README.md | 6 +- packages/lsp/lsp-local/README.zh.md | 6 +- packages/lsp/lsp-local/src/host.ts | 26 +- packages/lsp/lsp-local/tests/host.spec.ts | 6 + .../lsp/lsp-local/tests/lifecycle.spec.ts | 9 +- packages/lsp/tool-lsp/src/render.ts | 57 +- packages/lsp/tool-lsp/tests/render.spec.ts | 1 + packages/pty/pty-local/README.i18n.yaml | 4 +- packages/pty/pty-local/README.md | 4 +- packages/pty/pty-local/README.zh.md | 4 +- packages/pty/pty-local/src/index.ts | 33 +- packages/pty/pty-local/src/session.ts | 130 ++- packages/pty/pty-local/tests/index.spec.ts | 13 +- packages/pty/pty-local/tests/session.spec.ts | 285 +++--- .../skill-local/tests/skill-local.spec.ts | 6 - .../subprocess-local/README.i18n.yaml | 4 +- .../subprocess/subprocess-local/README.md | 2 +- .../subprocess/subprocess-local/README.zh.md | 2 +- .../subprocess/subprocess-local/src/index.ts | 16 +- .../subprocess-local/src/terminal.ts | 34 +- .../subprocess-local/tests/local.spec.ts | 39 +- .../subprocess-local/tests/terminal.spec.ts | 108 +-- .../subprocess/subprocess/README.i18n.yaml | 4 +- packages/subprocess/subprocess/README.md | 3 +- packages/subprocess/subprocess/README.zh.md | 3 +- packages/subprocess/subprocess/src/index.ts | 13 +- .../subprocess/src/terminal-lifecycle.ts | 104 --- packages/subprocess/subprocess/src/types.ts | 17 +- .../subprocess/tests/service.spec.ts | 3 +- .../tests/terminal-lifecycle.spec.ts | 125 --- packages/typert/README.i18n.yaml | 4 +- 65 files changed, 1265 insertions(+), 948 deletions(-) create mode 100644 packages/code-runtime/code-runtime/tests/contract.ts delete mode 100644 packages/subprocess/subprocess/src/terminal-lifecycle.ts delete mode 100644 packages/subprocess/subprocess/tests/terminal-lifecycle.spec.ts diff --git a/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.i18n.yaml b/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.i18n.yaml index dc9b924f54..ddb215be9f 100644 --- a/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.md -2026-06-17-filesystem-capability-seam.md: 2fb360e5ad3c972f4a8de50c602a79af158dfc22 -2026-06-17-filesystem-capability-seam.zh.md: 1f70113b1583d890047ee4a17161f76b6390d8e8 +2026-06-17-filesystem-capability-seam.md: 7436d2a9402c76f17c7d6571eb489a86e550ebfb +2026-06-17-filesystem-capability-seam.zh.md: fdeafba387b562352b377321327c526a67669ef6 diff --git a/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.md b/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.md index 2fb360e5ad..7436d2a940 100644 --- a/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.md +++ b/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.md @@ -64,7 +64,7 @@ The interface covers these semantic operations: - Resolve a model/plugin-supplied path into a backend-defined target. - Convert a resolved target to the canonical process path or `file:` URI for the same execution world, and test containment without parsing its opaque key. - Stat target metadata without reading file contents. -- Read complete or streamed UTF-8 text, including one stable-handle byte-bounded whole-file operation. +- Read complete or streamed UTF-8 text; consumers apply their own view and retention limits. - Create or replace a UTF-8 text file. - Edit an existing UTF-8 text file by literal replacement. @@ -88,7 +88,7 @@ Resolved targets must expose at least three concepts: Read and mutation results must include an opaque file `version`. The local backend derives its token from bigint stat metadata (`dev`, `ino`, `size`, `mtimeNs`, and `ctimeNs`) so same-size rewrites and inode replacement invalidate consumers reliably; a remote backend can use a revision id or hash-like token. The `dsh-fs-policy` plugin records versions for stale checks; consumers may display related metadata but must not interpret the version token. -The provider hands back decoded text: `readText` returns a whole regular text file, `streamText` streams the same text semantics for large files, and `readTextBounded` holds one backend-owned stable handle while rejecting a complete file above its byte ceiling. Line windowing, numbered-line rendering, and total-line accounting live in the executor (`dsh-tool-fs`). The provider owns regular-file checks, UTF-8 decoding, binary/NUL rejection, and the bounded read's replacement/growth race; it does not know about line windows or views. +The provider hands back decoded text: `readText` returns a whole regular text file and `streamText` streams the same text semantics for large files or consumer-owned retention limits. Line windowing, byte ceilings, numbered-line rendering, and total-line accounting live in consumers such as `dsh-tool-fs` and `dsh-lsp-local`. The provider owns regular-file checks, UTF-8 decoding, and binary/NUL rejection; it does not know about line windows, protocol limits, or views. Observed-state recording is not on `ctx.fs`: after a successful read the executor emits `fs/observed`, and the `dsh-fs-policy` plugin records `{ version }` for the deriving owner. There is no `full`/`partial` view — a read at any window records the version, and freshness (not view completeness) authorizes a later write/edit. diff --git a/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.zh.md b/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.zh.md index 1f70113b15..fdeafba387 100644 --- a/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.zh.md +++ b/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.zh.md @@ -64,7 +64,7 @@ harness 已有一个具体的 `bash` 能力 seam(`dsh-bash` / `dsh-bash-local` - 将模型/插件提供的路径解析为后端定义的目标。 - 将解析后的目标转换为同一执行环境的规范进程路径或 `file:` URI,并在不解析其不透明键的情况下检查包含关系。 - 获取目标元数据而不读取文件内容。 -- 读取完整或流式 UTF-8 文本,其中包括一项持有稳定句柄、以字节为上限的全文件读取操作。 +- 读取完整或流式 UTF-8 文本;消费方执行各自的视图与保留上限。 - 创建或替换一个 UTF-8 文本文件。 - 通过字面替换编辑一个已有的 UTF-8 文本文件。 @@ -88,7 +88,7 @@ harness 已有一个具体的 `bash` 能力 seam(`dsh-bash` / `dsh-bash-local` 读取和变更结果必须包含不透明的文件 `version`。本地后端从 bigint stat 元数据(`dev`、`ino`、`size`、`mtimeNs` 和 `ctimeNs`)派生令牌,因此同大小重写和 inode 替换都会可靠地使消费方失效;远程后端可以使用 revision id 或类似 hash 的令牌。`dsh-fs-policy` 插件记录版本用于陈旧检查;消费方可以展示相关元数据但禁止解释版本令牌。 -提供方返回已解码的文本:`readText` 返回整个常规文本文件,`streamText` 为大文件流式传输相同的文本语义,`readTextBounded` 则持有一个归后端所有的稳定句柄,并在完整文件超过字节上限时拒绝。行窗口化、带行号渲染和总行数统计位于执行器(`dsh-tool-fs`)中。提供方负责普通文件检查、UTF-8 解码、二进制/NUL 拒绝,以及有界读取期间的路径替换/增长竞态;它不知道行窗口或视图。 +提供方返回已解码的文本:`readText` 返回整个普通文本文件,`streamText` 为大文件或消费方自有的保留上限流式传输相同的文本语义。行窗口化、字节上限、带行号渲染和总行数统计归 `dsh-tool-fs`、`dsh-lsp-local` 等消费方所有。提供方负责普通文件检查、UTF-8 解码和二进制/NUL 拒绝;它不知道行窗口、协议上限或视图。 观测状态记录不在 `ctx.fs` 上:成功读取后,执行器发出 `fs/observed`,`dsh-fs-policy` 插件为推导出的 owner 记录 `{ version }`。没有 `full`/`partial` 视图——任何窗口的读取都记录版本,新鲜度(而非视图完整性)授权后续的写入/编辑。 diff --git a/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.i18n.yaml index c2ed39b07d..aac1c86e11 100644 --- a/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md -2026-07-15-lsp-capability-seam.md: 233a7321e1b4fded955144ed84fbc34df49e8964 -2026-07-15-lsp-capability-seam.zh.md: 547249fc22844878954fe7120dd8b5c87aabfc1a +2026-07-15-lsp-capability-seam.md: 63cec0a4e349ffc5be27a84e955b15e9168f898b +2026-07-15-lsp-capability-seam.zh.md: cdb1289f6956d8a4e46ed92847ddd8a58e588afa diff --git a/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md b/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md index 233a7321e1..63cec0a4e3 100644 --- a/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md +++ b/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md @@ -112,13 +112,13 @@ Provider disposal occurs outside tool execution, so `dsh-lsp-local` keeps `shutd ## Workspace, filesystem, and document synchronization -`dsh-lsp-local` canonicalizes and reads through `ctx.fs` in the language server's execution world. It requires the workspace target to be a directory, rejects out-of-workspace sources through provider-owned containment, and uses `readTextBounded` so regular-file validation, UTF-8 decoding, the byte ceiling, and path replacement/growth safety stay one filesystem operation. It fuses caller cancellation with provider disposal across each filesystem operation, tracks workspace lookups before they enter a queue, and awaits those lookups during disposal. It does not emit `fs/observed`: only the LSP result is model-visible, so the query does not satisfy read-before-write policy. +`dsh-lsp-local` canonicalizes and reads through `ctx.fs` in the language server's execution world. It requires the workspace target to be a directory, rejects out-of-workspace sources through provider-owned containment, consumes `streamText`, and enforces `maxDocumentBytes` as chunks arrive; the provider retains regular-file validation and UTF-8 decoding while the protocol consumer owns its document limit. It fuses caller cancellation with provider disposal across each filesystem operation, tracks workspace lookups before they enter a queue, and awaits those lookups during disposal. It does not emit `fs/observed`: only the LSP result is model-visible, so the query does not satisfy read-before-write policy. The `read` tool is unsuitable source because its output is windowed, numbered, transcript-visible, and observed. Reading in `tool-lsp` would also assign provider-specific synchronization to the consumer and preclude non-local providers. The local provider uses a compatibility-first transient-open sequence for every query. It accepts legacy `textDocumentSync` `Full` or `Incremental`, or options with `openClose: true`; omitted, `None`, or explicitly incompatible synchronization fails as unsupported before `didOpen`. -1. Resolve and contain the source through `ctx.fs`, then read its current bounded text through the same provider. +1. Resolve and contain the source through `ctx.fs`, then stream its current text through the same provider while enforcing the document byte limit. 2. Send `textDocument/didOpen` with version `1`, full text, and the configured language id. Its write remains abortable; failure or cancellation invalidates the instance and awaits bounded process termination before the pool can reuse it. 3. Send the requested `textDocument/definition`, `textDocument/references`, `textDocument/implementation`, or `textDocument/hover` request. 4. If `didOpen` succeeded, attempt `textDocument/didClose` in `finally` after the request settles or aborts. A close-write failure does not replace the settled result or error, but invalidates the instance and awaits bounded process termination. @@ -157,7 +157,7 @@ The provider trusts its configured server. Its filesystem visibility and process **Wrap the signal in a per-seam execution-context object.** Web passes a bare `AbortSignal`; wrapping this single field would add unexplained asymmetry. `query()` gains a context object only when another field requires it. -**Read through the model-facing `read` tool.** Rejected because tool output is windowed, numbered, transcript-visible, and observed. The provider reads bounded full text directly through the same `ctx.fs` execution world used by its subprocess. +**Read through the model-facing `read` tool.** Rejected because tool output is windowed, numbered, transcript-visible, and observed. The provider consumes streamed full text directly through the same `ctx.fs` execution world used by its subprocess. **Keep documents open.** Mirroring edits requires version ownership, all-path `didChange`, HMR recovery, eviction, and stale-state rules. Transient opens avoid that MVP state machine. @@ -195,4 +195,4 @@ Extension ownership is exclusive within one runtime. Two providers cannot both c UTF-16 cursor columns are exact for the protocol but difficult for a model to count around non-BMP characters. Invalid or off-symbol positions may produce empty results, so error text and prompt examples must explain the coordinate convention without encouraging broad LSP use. -The paired filesystem/subprocess providers align the query snapshot with the server index but do not make a trusted language server safe. Canonical containment rejects query sources outside the workspace; the server itself receives the execution world's configured authority and may read other paths or use caches. +The paired filesystem/subprocess providers align the query snapshot with the server index but do not make a trusted language server safe. Canonical containment rejects query sources outside the workspace at resolution time, but stream opening does not add stable-handle identity across a concurrent path replacement; the server itself receives the execution world's configured authority and may read other paths or use caches. diff --git a/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.zh.md b/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.zh.md index 547249fc22..cdb1289f69 100644 --- a/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.zh.md @@ -112,13 +112,13 @@ interface LspToolInput { ## 工作区、文件系统与文档同步 -`dsh-lsp-local` 在语言服务器的执行环境中通过 `ctx.fs` 规范化并读取文件。它要求工作区目标是目录,使用提供方自有的 containment 拒绝工作区外的源文件,并通过 `readTextBounded` 把普通文件校验、UTF-8 解码、字节上限和路径替换/增长安全性保留在同一项文件系统操作中。它会针对每项文件系统操作合并调用方取消与提供方资源释放,跟踪尚未进入队列的工作区查找,并在资源释放期间等待这些查找结算。它不发送 `fs/observed`:只有 LSP 结果对模型可见,因此查询不满足写前读取策略。 +`dsh-lsp-local` 在语言服务器的执行环境中通过 `ctx.fs` 规范化并读取文件。它要求工作区目标是目录,使用提供方自有的 containment 拒绝工作区外的源文件,消费 `streamText`,并在分片到达时执行 `maxDocumentBytes` 上限;普通文件校验和 UTF-8 解码仍由提供方负责,文档上限则由协议消费方负责。它会针对每项文件系统操作合并调用方取消与提供方资源释放,跟踪尚未进入队列的工作区查找,并在资源释放期间等待这些查找结算。它不发送 `fs/observed`:只有 LSP 结果对模型可见,因此查询不满足写前读取策略。 `read` 工具的输出带窗口与行号,进入 transcript(文本记录)且已被观察,不适合作为源文件。在 `tool-lsp` 内读取还会把提供方专用同步职责交给消费方,并排除非本地提供方。 本地提供方对每次查询都采用兼容优先的临时打开流程。它接受旧式 `textDocumentSync` 的 `Full` 或 `Incremental`,也接受设置了 `openClose: true` 的选项;同步能力缺失、为 `None` 或明确不兼容时,在 `didOpen` 前以不支持错误失败。 -1. 通过 `ctx.fs` 解析源文件并检查其位于工作区内,再通过同一提供方对当前文本进行有界读取。 +1. 通过 `ctx.fs` 解析源文件并检查其位于工作区内,再通过同一提供方流式读取当前文本,同时执行文档字节上限。 2. 发送 `textDocument/didOpen`,其中包含版本 `1`、完整文本和配置的语言 id。该写入仍可取消;写入失败或遭取消会使实例失效,并等待有界进程终止完成,池才能复用它。 3. 发送所请求的 `textDocument/definition`、`textDocument/references`、`textDocument/implementation` 或 `textDocument/hover` 请求。 4. 如果 `didOpen` 成功,则在请求完成或取消后于 `finally` 中尝试发送 `textDocument/didClose`。关闭写入失败不会覆盖已经确定的结果或错误,但会使实例失效,并等待有界进程终止完成。 @@ -157,7 +157,7 @@ interface LspToolInput { **将信号包装为每服务边界的执行上下文对象。** Web 传递裸 `AbortSignal`;仅包装这一个字段会造成无谓的不对称。只有另一个字段确有需要时,`query()` 才引入上下文对象。 -**通过面向模型的 `read` 工具读取。**拒绝,因为工具输出带窗口与行号,会进入 transcript 且已被观察。提供方直接通过子进程所用的同一 `ctx.fs` 执行环境读取有界的完整文本。 +**通过面向模型的 `read` 工具读取。**拒绝,因为工具输出带窗口与行号,会进入 transcript 且已被观察。提供方直接通过子进程所用的同一 `ctx.fs` 执行环境消费流式传输的完整文本。 **保持文档打开。** 镜像编辑需要版本归属、覆盖所有路径的 `didChange`、HMR 恢复、淘汰和陈旧状态规则。临时打开避免在 MVP 引入这套状态机。 @@ -195,4 +195,4 @@ interface LspToolInput { UTF-16 光标列与协议完全一致,但模型难以在包含非 BMP 字符的文本中准确计数。无效位置或不在符号上的位置可能返回空结果,因此错误文本和提示词示例必须说明坐标约定,同时避免鼓励模型广泛使用 LSP。 -配对的文件系统/子进程提供方会对齐查询快照与服务器索引,但不会因此使受信任的语言服务器变得安全。规范 containment 会拒绝工作区外的查询源;服务器本身获得执行环境所配置的权限,仍可读取其他路径或使用缓存。 +配对的文件系统/子进程提供方会对齐查询快照与服务器索引,但不会因此使受信任的语言服务器变得安全。规范 containment 会在解析时拒绝工作区外的查询源,但打开流不会在路径并发替换期间额外保证稳定句柄身份;服务器本身获得执行环境所配置的权限,仍可读取其他路径或使用缓存。 diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml index de79bd2d8d..7b6a74b7e8 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md -2026-07-28-portable-execution-world-consumers.md: e943c54a3f3a4c93a95db999d4c60790457ba644 -2026-07-28-portable-execution-world-consumers.zh.md: 7755791d5763436866763e8d4ea9e0fab8203bb9 +2026-07-28-portable-execution-world-consumers.md: d68fc92308079aa68a8f4e1accadde0addb0e5aa +2026-07-28-portable-execution-world-consumers.zh.md: 5341cf1643cabb1043b977e18179dc857e37e121 diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md index e943c54a3f..d68fc92308 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md @@ -14,16 +14,16 @@ Ordinary pipes do not cover one requirement. A persistent terminal needs PTY all `ctx.fs` and `ctx.subprocess` together define one execution world. Providers mounted together must describe the same path namespace, executables, processes, and terminal sessions; higher capabilities consume those two interfaces rather than name the provider. -The filesystem interface owns the path facts that another capability needs without exposing its opaque target identity: a canonical process path, canonical `file:` URI, containment, and a bounded stable-handle text read. The existing text and mutation operations remain filesystem-owned. +The filesystem interface owns the path facts that another capability needs without exposing its opaque target identity: a canonical process path, canonical `file:` URI, and containment. Existing whole and streaming text operations remain filesystem-owned; protocol consumers enforce their own retention limits while consuming the stream. -The subprocess interface owns the process coordinates and primitives: canonical cwd, private runtime storage, executable lookup, ordinary raw or collected process spawning, and `spawnTerminal()`. The terminal operation is one deep primitive whose handle owns byte I/O, foreground groups, signalling, TERM-to-KILL session cleanup, and a quiescence wait. The interface package also exports a provider-neutral lifecycle controller that joins top-level settlement, lifetime cancellation, retryable provider cleanup, and bounded quiescence observation; each implementation supplies only its session-cleanup transaction. Prompt detection, idle inference, scrollback, sandbox policy, and owner lifecycle remain in the PTY consumer. +The subprocess interface owns the process coordinates and primitives: canonical cwd, private runtime storage, executable lookup, ordinary raw or collected process spawning, and `spawnTerminal()`. The terminal operation is one deep primitive whose handle owns text I/O, foreground groups, signalling, and one awaited TERM-to-KILL operation that settles in-flight handle calls and reaches whole-session quiescence. Its signal cancels allocation only; the published handle owns its lifetime. Prompt detection, idle inference, scrollback, sandbox policy, and owner lifecycle remain in the PTY consumer. Generic consumers use that execution world: - `dsh-bash-local` continues to map Bash semantics onto ordinary `ctx.subprocess.spawn()`. - `dsh-lsp-local` reads and contains source through `ctx.fs`, resolves and launches language servers through `ctx.subprocess`, and carries provider-owned file URIs through initialization and result rendering. One provider-lifetime signal aborts filesystem and protocol work during disposal, including workspace lookup before queue ownership; its JSON-RPC, pooling, synchronization, and normalization stay unchanged. -- `dsh-pty-local` maps persistent-shell semantics onto `ctx.subprocess.spawnTerminal()`. The local `node-pty` and process-inspection implementation moves into `dsh-subprocess-local`; another subprocess provider supplies the same primitive. The allocation signal is detached before publication, while readiness initialization retains setup cancellation. Prompt and silence evidence collected during asynchronous pre-write inspection is discarded when the provider write begins. Cancellation retains the send reservation until asynchronous foreground signalling settles, so the signal cannot target a successor; if a signaled send returns `inferred_idle` before its prompt arrives, that marker remains attributed to the prior send instead of settling a successor after its echo. A timed-out asynchronous write, or a signal failure during that write, retains the reservation until the provider settles the write. Completion of a stale inspection resumes polling for the current send. Close rejects new public signals after shutdown begins and drains public signals already in flight before returning. -- `dsh-code-runtime-subprocess` materializes a dependency-free runner through `ctx.fs` and launches it through `ctx.subprocess`, preserving the Code Runtime binding and output contract across local or remote worlds. It shares host-side worker mechanics through the non-plugin `dsh-code-runtime-worker/runtime-host` subpath instead of copying them. Preparation carries one lifecycle signal through filesystem resolution, materialization, and executable lookup so disposal can abort a stalled provider. The heap-bounded worker rejects oversized binding frames before transfer, each outer hop enforces the same bound before serialization, and raw subprocess pipes carry newline-delimited UTF-8 JSON without a redundant base64 representation. The launcher publishes an accepted terminal frame before reaping its controller so a descendant that inherits controller pipes cannot suppress completion; the host still awaits process-group quiescence. +- `dsh-pty-local` maps persistent-shell semantics onto `ctx.subprocess.spawnTerminal()`. The local `node-pty` and process-inspection implementation moves into `dsh-subprocess-local`; another subprocess provider supplies the same primitive. Prompt and silence evidence collected during asynchronous pre-write inspection is discarded when the provider write begins. Cancellation retains the send reservation while an in-flight write settles and then signals the foreground group, so late bytes or the signal cannot target a successor; a rejected write sends no signal. The absolute deadline remains armed throughout cancellation. A signal failure becomes terminal transport failure. Completion of a stale inspection resumes polling for the current send. Close rejects new public signals and delegates complete-session quiescence to the handle's awaited termination operation. +- `dsh-code-runtime-subprocess` materializes a dependency-free runner through `ctx.fs` and launches it through `ctx.subprocess`, preserving the Code Runtime binding and output contract across local or remote worlds. The fixed runner is adapter-owned infrastructure below `ctx.subprocess.runtimeRoot`, so its write carries an explicit `danger-full-access` policy instead of inheriting the model-facing filesystem mode. It shares host-side worker mechanics through the non-plugin `dsh-code-runtime-worker/runtime-host` subpath instead of copying them. Preparation carries one lifecycle signal through filesystem resolution, materialization, and executable lookup so disposal can abort a stalled provider. The heap-bounded worker rejects oversized binding frames before transfer, each outer hop enforces the same bound before forwarding, and raw subprocess pipes carry newline-delimited UTF-8 JSON without a redundant base64 representation. The launcher publishes an accepted terminal frame before reaping its controller so a descendant that inherits controller pipes cannot suppress completion; the host still awaits process-group quiescence. `dsh-code-runtime-worker` remains a separate implementation. It is the smaller in-process backend and works in single-file distributions that cannot assume an installed Node executable. Remote filesystem/process compositions select `dsh-code-runtime-subprocess`; they do not need a provider-specific Code Runtime package. @@ -35,6 +35,10 @@ Generic consumers use that execution world: **Move PTY readiness and session policy into the subprocess service.** Rejected because those are persistent-terminal consumer semantics, not OS process mechanics. A subprocess provider owns what only its substrate can do; `dsh-pty-local` owns what a Harness terminal means. +**Expose separate terminal termination and quiescence operations plus a shared lifecycle controller.** Rejected because every terminal consumer needs the same single cleanup outcome. Separate operations export provider bookkeeping, bounded-observer, and retry semantics without a production consumer; one awaited provider operation is a deeper interface. + +**Add a stable bounded-read primitive to the filesystem seam.** Rejected because only LSP needs a complete-document byte ceiling, which it can enforce while consuming the existing text stream. A second primitive forces every provider to implement stable-handle and no-follow mechanics, including a remote helper protocol, without an observed concurrent-replacement defect. + **Delete the worker-thread Code Runtime.** Rejected because portability does not erase its current deployment need. The subprocess backend requires a Node executable and filesystem materialization; the worker backend has neither requirement and remains the supported single-process path. **Run the whole harness inside the remote environment.** Rejected as a different deployment model. Making execution capabilities portable does not move model calls, session state, plugin state, or the agent loop. diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md index 7755791d57..5341cf1643 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md @@ -14,16 +14,16 @@ Status: implemented `ctx.fs` 与 `ctx.subprocess` 共同定义一个执行世界。共同挂载的提供方必须描述相同的路径命名空间、可执行文件、进程和终端会话;上层能力消费这两个接口,而不引用具体提供方。 -文件系统接口负责其他能力需要的路径事实,同时不公开其不透明目标身份:规范化进程路径、规范化 `file:` URI、包含关系,以及通过稳定句柄执行的有界文本读取。现有文本与变更操作仍归文件系统负责。 +文件系统接口负责其他能力需要的路径事实,同时不公开其不透明目标身份:规范化进程路径、规范化 `file:` URI 和包含关系。现有完整文本与流式文本操作仍归文件系统负责;协议消费方在消费流时执行各自的保留上限。 -进程管理接口负责进程运行坐标与原语:规范化 cwd、私有运行时存储、可执行文件查找、以原始或收集模式 spawn 普通进程,以及 `spawnTerminal()`。终端操作是一项深层原语,其句柄负责字节 I/O、前台进程组管理、信号发送、TERM→KILL 会话清理以及等待完全停稳。接口包还导出一个提供方无关的生命周期控制器,用于组合顶层结算、生命周期取消、可重试的提供方清理与有界的完全停稳观测;每个实现只需提供自身的会话清理事务。提示符检测、空闲推断、scrollback、沙箱策略和所有者生命周期仍由 PTY 消费方负责。 +进程管理接口负责进程运行坐标与原语:规范化 cwd、私有运行时存储、可执行文件查找、以原始或收集模式 spawn 普通进程,以及 `spawnTerminal()`。终端操作是一项深层原语,其句柄负责文本 I/O、前台进程组、信号发送,以及一项须等待的 TERM→KILL 操作;该操作会结算所有在途句柄调用,并使整个会话完全停稳。其信号只取消分配;句柄一经发布,便负责自身生命周期。提示符检测、空闲推断、scrollback、沙箱策略和所有者生命周期仍由 PTY 消费方负责。 通用消费方使用该执行世界: - `dsh-bash-local` 继续把 Bash 语义映射到普通的 `ctx.subprocess.spawn()`。 - `dsh-lsp-local` 通过 `ctx.fs` 读取源文件并验证包含关系,通过 `ctx.subprocess` 解析和启动语言服务器,并让由提供方负责的文件 URI 贯穿初始化与结果渲染。一个提供方生命周期信号会在资源释放期间中止文件系统与协议操作,包括取得队列所有权之前的工作区查找;其 JSON-RPC、池化、同步和规范化保持不变。 -- `dsh-pty-local` 把持久 shell 语义映射到 `ctx.subprocess.spawnTerminal()`。本地 `node-pty` 与进程检查实现移入 `dsh-subprocess-local`;其他进程管理提供方则提供相同原语。分配信号会在发布前解除关联,而就绪初始化仍保留设置阶段的取消。提供方开始写入时,系统会丢弃异步写入前检查期间收集的提示符与静默证据。取消会保留发送预留,直至异步前台信号发送结算,从而使该信号无法把后续发送作为目标;如果收到信号的 send 在其提示符到达前返回 `inferred_idle`,随后到达的标记仍归属于先前的 send,而不会在后续 send 回显后使其完成。异步写入超时,或该写入期间发生信号发送失败,都会保留预留,直至提供方将写入结算。陈旧检查完成后,会针对当前发送恢复轮询。关闭操作开始后会拒绝新的公开信号,并在返回前等待所有已在途的公开信号结算。 -- `dsh-code-runtime-subprocess` 通过 `ctx.fs` 物化无依赖 runner,并通过 `ctx.subprocess` 启动它,从而在本地或远程执行世界中保留代码运行时的绑定与输出契约。它通过非插件子路径 `dsh-code-runtime-worker/runtime-host` 共享宿主侧 worker 机制,而不是复制这些机制。准备阶段让同一个生命周期信号贯穿文件系统解析、物化和可执行文件查找,使资源释放能够中止停滞的提供方操作。受堆上限约束的 worker 会在传输前拒绝过大的绑定帧;每个外层转发环节都会在序列化前执行相同的上限检查;原始子进程管道承载以换行符分隔的 UTF-8 JSON,无需冗余的 base64 表示;launcher 会在回收 controller 前发布已接纳的终态帧,使继承 controller 管道的后代进程无法阻止完成;宿主仍会等待进程组完全停稳。 +- `dsh-pty-local` 把持久 shell 语义映射到 `ctx.subprocess.spawnTerminal()`。本地 `node-pty` 与进程检查实现移入 `dsh-subprocess-local`;其他进程管理提供方则提供相同原语。提供方开始写入时,系统会丢弃异步写入前检查期间收集的提示符与静默证据。取消会在在途写入结算期间保留发送预留,随后向前台进程组发送信号,因此延迟字节和该信号都无法落到后续发送;写入被拒绝时不会发送信号。绝对截止时间会在整个取消期间保持启用。信号发送失败会成为终结性传输失败。陈旧检查完成后,会针对当前发送恢复轮询。关闭操作会拒绝新的公开信号,并把完整会话的完全停稳委托给句柄上须等待的终止操作。 +- `dsh-code-runtime-subprocess` 通过 `ctx.fs` 物化无依赖 runner,并通过 `ctx.subprocess` 启动它,从而在本地或远程执行世界中保留代码运行时的绑定与输出契约。固定 runner 是位于 `ctx.subprocess.runtimeRoot` 下的适配器自有基础设施,因此其写入携带显式 `danger-full-access` 策略,而不继承面向模型的文件系统模式。它通过非插件子路径 `dsh-code-runtime-worker/runtime-host` 共享宿主侧 worker 机制,而不是复制这些机制。准备阶段让同一个生命周期信号贯穿文件系统解析、物化和可执行文件查找,使资源释放能够中止停滞的提供方操作。受堆上限约束的 worker 会在传输前拒绝过大的绑定帧;每个外层转发环节都会在转发前执行相同的上限检查;原始子进程管道承载以换行符分隔的 UTF-8 JSON,无需冗余的 base64 表示;launcher 会在回收 controller 前发布已接纳的终态帧,使继承 controller 管道的后代进程无法阻止完成;宿主仍会等待进程组完全停稳。 `dsh-code-runtime-worker` 仍是独立实现。它是较小的进程内后端,可用于无法假定已安装 Node 可执行文件的单文件分发。远程文件系统/进程组合选择 `dsh-code-runtime-subprocess`;它们不需要提供方专用的代码运行时包。 @@ -35,6 +35,10 @@ Status: implemented **把 PTY 就绪判断与会话策略移入进程管理服务。** 不予采纳,因为这些属于持久终端消费方的语义,而非 OS 进程机制。进程管理提供方负责只有其执行基底才能完成的操作;`dsh-pty-local` 负责 Harness 终端的语义。 +**分别公开终端终止与完全停稳操作,并提供共享生命周期控制器。** 不予采纳,因为每个终端消费方都需要相同的单一清理结果。拆分操作会把提供方簿记、有界观察者和重试语义暴露出来,却没有生产消费方;由提供方提供一个须等待的操作,接口更深。 + +**在文件系统 seam 中新增稳定的有界读取原语。** 不予采纳,因为只有 LSP 需要完整文档字节上限,而它可以在消费现有文本流时执行该上限。第二项原语会迫使每个提供方实现稳定句柄和不跟随符号链接的机制,远程提供方甚至需要辅助协议,却没有已观察到的并发替换缺陷。 + **删除 worker 线程代码运行时。** 不予采纳,因为可移植性不会消除其当前部署需求。进程管理后端需要 Node 可执行文件和文件系统物化,而 worker 后端两者都不需要,并且仍是受支持的单进程路径。 **在远程环境中运行整个 harness。** 不予采纳,因为这是另一种部署模型。让执行能力可移植,并不意味着移动模型调用、会话状态、插件状态或 agent loop(智能体循环)。 diff --git a/docs/config-catalog.md b/docs/config-catalog.md index c07e81089c..b70f468960 100644 --- a/docs/config-catalog.md +++ b/docs/config-catalog.md @@ -447,7 +447,7 @@ export interface Config { } ``` -Source: [`packages/fs/fs-local/src/index.ts:40`](../packages/fs/fs-local/src/index.ts) +Source: [`packages/fs/fs-local/src/index.ts:39`](../packages/fs/fs-local/src/index.ts) ## `@deepseek-ai/dsh-fs-sandbox` diff --git a/docs/core-data-structures/filesystem.i18n.yaml b/docs/core-data-structures/filesystem.i18n.yaml index 129c5569b9..78386c54fd 100644 --- a/docs/core-data-structures/filesystem.i18n.yaml +++ b/docs/core-data-structures/filesystem.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write docs/core-data-structures/filesystem.md -filesystem.md: 50b936bdef7535769f19d6cc0a75eb11eea9a869 -filesystem.zh.md: 61a885203b0718b34b0fd936c318b03c65ce4770 +filesystem.md: addded9f673ed435e95109d4fb772967514c0b87 +filesystem.zh.md: 3ef16ede1e137d2a1310861941ac3244ca105650 diff --git a/docs/core-data-structures/filesystem.md b/docs/core-data-structures/filesystem.md index 50b936bdef..addded9f67 100644 --- a/docs/core-data-structures/filesystem.md +++ b/docs/core-data-structures/filesystem.md @@ -52,7 +52,7 @@ type FsTargetKey = Branded<'FsTargetKey'> type FsVersion = Branded<'FsVersion'> ``` -`stat` returns metadata (never content), or `undefined` when the target is absent. `type` lets the tool reject directories/special files before reading, and `size` lets it choose `readText` vs `streamText` without probing by failure. Protocol consumers use `readTextBounded(target, maxBytes)` when size validation and the complete UTF-8 read must remain one backend-owned stable operation; composing `stat` with `readText` would admit growth and replacement races. +`stat` returns metadata (never content), or `undefined` when the target is absent. `type` lets the tool reject directories/special files before reading, and `size` lets it choose `readText` vs `streamText` without probing by failure. A protocol consumer that needs a byte ceiling applies it while consuming `streamText`, so the filesystem seam needs no consumer-specific bounded-read primitive. ```ts type-equiv /** @@ -258,4 +258,4 @@ type FsErrorCode = ## The service and the plugin -`FileSystem` (`ctx.fs`, abstract) owns the provider primitives: `resolve`, `processPath`, `fileUrl`, `contains`, `stat`, `lstat`, `readText`, `readTextBounded`, `streamText`, `listDir`, `writeText`, and `editText`. `dsh-fs-policy` registers **no service** — it is a plugin that adds policy through the `fs/*` event gate: it decides the write/edit intent waterfalls (supplying `createIfAbsent`/`replaceIfVersion`/`{ version }` or throwing `FS_NOT_OBSERVED`) and records on `fs/observed`. The executor is `dsh-tool-fs`: it reads/writes/edits through `ctx.fs`, dispatches the waterfalls, and emits the recording event. The generated wiring catalog shows the exact `ctx.fs` signatures on [services.md](../cordis-catalog/services.md#ctxfs--filesystem-abstract-seam). +`FileSystem` (`ctx.fs`, abstract) owns the provider primitives: `resolve`, `processPath`, `fileUrl`, `contains`, `stat`, `lstat`, `readText`, `streamText`, `listDir`, `writeText`, and `editText`. `dsh-fs-policy` registers **no service** — it is a plugin that adds policy through the `fs/*` event gate: it decides the write/edit intent waterfalls (supplying `createIfAbsent`/`replaceIfVersion`/`{ version }` or throwing `FS_NOT_OBSERVED`) and records on `fs/observed`. The executor is `dsh-tool-fs`: it reads/writes/edits through `ctx.fs`, dispatches the waterfalls, and emits the recording event. The generated wiring catalog shows the exact `ctx.fs` signatures on [services.md](../cordis-catalog/services.md#ctxfs--filesystem-abstract-seam). diff --git a/docs/core-data-structures/filesystem.zh.md b/docs/core-data-structures/filesystem.zh.md index 6b7642b3f7..1e378928ed 100644 --- a/docs/core-data-structures/filesystem.zh.md +++ b/docs/core-data-structures/filesystem.zh.md @@ -52,7 +52,7 @@ type FsTargetKey = Branded<'FsTargetKey'> type FsVersion = Branded<'FsVersion'> ``` -`stat` 返回元数据(从不返回内容),目标不存在时返回 `undefined`。`type` 让工具在读取前拒绝目录或特殊文件;`size` 让工具无需通过失败探测即可选择 `readText` 还是 `streamText`。当大小校验与完整 UTF-8 读取必须保持为一个由后端负责的稳定操作时,协议消费方使用 `readTextBounded(target, maxBytes)`;组合 `stat` 与 `readText` 会容许文件增长与替换竞态。 +`stat` 返回元数据(从不返回内容),目标不存在时返回 `undefined`。`type` 让工具在读取前拒绝目录或特殊文件;`size` 让工具无需通过失败探测即可选择 `readText` 还是 `streamText`。需要字节上限的协议消费方在消费 `streamText` 时执行该上限,因此文件系统 seam 无需消费方专用的有界读取原语。 ```ts type-equiv /** @@ -258,4 +258,4 @@ type FsErrorCode = ## 服务与插件 -`FileSystem`(`ctx.fs`,abstract)拥有提供方原语:`resolve`、`processPath`、`fileUrl`、`contains`、`stat`、`lstat`、`readText`、`readTextBounded`、`streamText`、`listDir`、`writeText` 与 `editText`。`dsh-fs-policy` **不注册服务**——它是一个通过 `fs/*` 事件门禁添加策略的插件:对写入/编辑意图 waterfall 作出决策(提供 `createIfAbsent`/`replaceIfVersion`/`{ version }`,或抛出 `FS_NOT_OBSERVED`),并在 `fs/observed` 上记录。执行器是 `dsh-tool-fs`:它通过 `ctx.fs` 读取/写入/编辑,分发 waterfall,并 emit 记录事件。生成的 wiring 目录在 [services.md](../cordis-catalog/services.md#ctxfs--filesystem-abstract-seam) 中展示确切的 `ctx.fs` 签名。 +`FileSystem`(`ctx.fs`,abstract)拥有提供方原语:`resolve`、`processPath`、`fileUrl`、`contains`、`stat`、`lstat`、`readText`、`streamText`、`listDir`、`writeText` 与 `editText`。`dsh-fs-policy` **不注册服务**——它是一个通过 `fs/*` 事件门禁添加策略的插件:对写入/编辑意图 waterfall 作出决策(提供 `createIfAbsent`/`replaceIfVersion`/`{ version }`,或抛出 `FS_NOT_OBSERVED`),并在 `fs/observed` 上记录。执行器是 `dsh-tool-fs`:它通过 `ctx.fs` 读取/写入/编辑,分发 waterfall,并 emit 记录事件。生成的 wiring 目录在 [services.md](../cordis-catalog/services.md#ctxfs--filesystem-abstract-seam) 中展示确切的 `ctx.fs` 签名。 diff --git a/docs/core-data-structures/subprocess.i18n.yaml b/docs/core-data-structures/subprocess.i18n.yaml index fca7947aae..7cba3857bb 100644 --- a/docs/core-data-structures/subprocess.i18n.yaml +++ b/docs/core-data-structures/subprocess.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write docs/core-data-structures/subprocess.md -subprocess.md: eae497530ff54a7a259c23a2d0f665f3313488f5 -subprocess.zh.md: c3504ef7cd42448a15341f7f92d0f309a6216de1 +subprocess.md: 89c75aea3393a000612e3b8f87d263c7b4fa7c60 +subprocess.zh.md: 83fb44f4750ba31db1f4336c0c293eb9e236be22 diff --git a/docs/core-data-structures/subprocess.md b/docs/core-data-structures/subprocess.md index c866276b8e..d6e0105c59 100644 --- a/docs/core-data-structures/subprocess.md +++ b/docs/core-data-structures/subprocess.md @@ -240,9 +240,9 @@ interface SubprocessOutcome { ## Terminal-process primitive -`spawnTerminal(spec)` is the non-pipe process primitive. The provider allocates the controlling terminal and owns raw UTF-8 byte transport, foreground-process-group inspection and signalling, TERM-to-KILL cleanup, and whole-session quiescence. The PTY backend remains responsible for prompt detection, readiness inference, scrollback, sandbox policy, and persistent-session ownership; ordinary `spawn()` cannot reconstruct controlling-terminal semantics. +`spawnTerminal(spec)` is the non-pipe process primitive. The provider allocates the controlling terminal and owns UTF-8 text transport, foreground-process-group inspection and signalling, and one awaited TERM-to-KILL operation that reaches whole-session quiescence. The PTY backend remains responsible for prompt detection, readiness inference, scrollback, sandbox policy, and persistent-session ownership; ordinary `spawn()` cannot reconstruct controlling-terminal semantics. -The terminal spec fully specifies argv, cwd, environment overrides, dimensions, cleanup grace, and optional cancellation. Its handle exposes `pid`, ordered output, `done`, `write`, `inspectForeground`, `signalForeground`, `terminate`, and `waitForExit`; the exact public shapes are generated into the [`ctx.subprocess` service catalog](../cordis-catalog/services.md#ctxsubprocess--subprocessservice-abstract-seam). +The terminal spec fully specifies argv, cwd, environment overrides, dimensions, cleanup grace, and optional allocation cancellation. Its handle exposes `pid`, ordered output, `done`, `write`, `inspectForeground`, `signalForeground`, and awaited `terminate`; the exact public shapes are generated into the [`ctx.subprocess` service catalog](../cordis-catalog/services.md#ctxsubprocess--subprocessservice-abstract-seam). ## Service behavior diff --git a/docs/core-data-structures/subprocess.zh.md b/docs/core-data-structures/subprocess.zh.md index c3504ef7cd..83fb44f475 100644 --- a/docs/core-data-structures/subprocess.zh.md +++ b/docs/core-data-structures/subprocess.zh.md @@ -240,9 +240,9 @@ interface SubprocessOutcome { ## 终端进程原语 -`spawnTerminal(spec)` 是非管道进程原语。提供方分配控制终端,并负责原始 UTF-8 字节传输、前台进程组检查与信号发送、TERM→KILL 清理,以及整个会话的完全停稳。PTY 后端仍负责提示符检测、就绪推断、scrollback、沙箱策略和持久会话所有权;普通 `spawn()` 无法重建控制终端语义。 +`spawnTerminal(spec)` 是非管道进程原语。提供方分配控制终端,并负责 UTF-8 文本传输、前台进程组检查与信号发送,以及一项须等待的 TERM→KILL 操作;该操作会使整个会话完全停稳。PTY 后端仍负责提示符检测、就绪推断、scrollback、沙箱策略和持久会话所有权;普通 `spawn()` 无法重建控制终端语义。 -终端 spec 完全指定 argv、cwd、环境覆盖、尺寸、清理宽限期与可选取消。其句柄公开 `pid`、有序输出、`done`、`write`、`inspectForeground`、`signalForeground`、`terminate` 和 `waitForExit`;确切的公共形状生成到 [`ctx.subprocess` 服务目录](../cordis-catalog/services.md#ctxsubprocess--subprocessservice-abstract-seam)中。 +终端 spec 完全指定 argv、cwd、环境覆盖、尺寸、清理宽限期与可选的分配取消。其句柄公开 `pid`、有序输出、`done`、`write`、`inspectForeground`、`signalForeground` 和须等待的 `terminate`;确切的公共形状生成到 [`ctx.subprocess` 服务目录](../cordis-catalog/services.md#ctxsubprocess--subprocessservice-abstract-seam)中。 ## 服务行为 diff --git a/packages/code-runtime/code-runtime-worker/src/runtime-host.ts b/packages/code-runtime/code-runtime-worker/src/runtime-host.ts index d2bb1082b8..474a058e9f 100644 --- a/packages/code-runtime/code-runtime-worker/src/runtime-host.ts +++ b/packages/code-runtime/code-runtime-worker/src/runtime-host.ts @@ -223,5 +223,4 @@ export class RuntimeOutputLedger { } export { decodeWorkerJson, encodeWorkerJson, snapshotCodeJsonValue } from './worker-json.ts' -export { jsonValueBytesUpTo } from './output-json.ts' export type { WorkerJsonWire } from './worker-json.ts' diff --git a/packages/code-runtime/code-runtime/tests/contract.ts b/packages/code-runtime/code-runtime/tests/contract.ts new file mode 100644 index 0000000000..f28545f8c1 --- /dev/null +++ b/packages/code-runtime/code-runtime/tests/contract.ts @@ -0,0 +1,838 @@ +import { describe, expect, it } from 'vitest' +import type { + CodeBindingFunction, + CodeBindingNamespace, + CodeRunResult, + CodeRuntime, +} from '@deepseek-ai/dsh-code-runtime' + +interface WorkerCodeRuntimeContractConfig { + computeMs?: number + maxWallMs?: number + maxOutputBytes?: number + maxOldGenerationSizeMb?: number +} + +interface WorkerCodeRuntimeContractHarness { + runtime: CodeRuntime + dispose: () => Promise +} + +type WorkerCodeRuntimeContractSetup = ( + config?: WorkerCodeRuntimeContractConfig, +) => Promise + +/** Convenience: one namespace `tools` with the given functions. */ +export function workerRuntimeTools( + functions: Record Promise>, +): CodeBindingNamespace[] { + return [{ + global: 'tools', + functions: functions as Record, + errorClass: { name: 'ToolCallError', memberNameProperty: 'toolName' }, + }] +} + +/** Run behavior shared by the direct and subprocess-hosted worker runtimes. */ +export function runWorkerCodeRuntimeContract( + label: string, + setup: WorkerCodeRuntimeContractSetup, +): void { + describe(`${label} — programs and bindings (real workers)`, () => { + it('registers with the seam descriptors', async () => { + const { runtime } = await setup() + expect(runtime.language).toBe('typescript') + expect(runtime.isolation).toBe('worker-thread') + }) + + it('runs TypeScript (erasable syntax), captures output in order, returns the value', async () => { + const { runtime } = await setup() + const result = await runtime.run({ + program: ` + interface Point { x: number; y: number } + const p: Point = { x: 1, y: 2 } as Point; + console.log('point', p); + process.stdout.write('raw-out\\n'); + console.warn('careful'); + return p.x + p.y; + `, + bindings: [], + }) + expect(result.error).toBeUndefined() + expect(result.value).toBe(3) + expect(result.logs).toEqual(['point { x: 1, y: 2 }', 'raw-out\n', 'careful']) + }) + + it('bridges binding calls both ways and rejects the program-side call on a host rejection', async () => { + const { runtime } = await setup() + const calls: unknown[] = [] + const result = await runtime.run({ + program: ` + const first = await tools.echo({ n: 1 }); + let caught = {}; + try { await tools.fail({}) } catch (error) { caught = { isTyped: error instanceof ToolCallError, name: error.name, toolName: error.toolName, message: error.message } } + let caughtRaw = {}; + try { await tools.failRaw({}) } catch (error) { caughtRaw = { name: error.name, toolName: error.toolName, message: error.message } } + return { first, caught, caughtRaw }; + `, + bindings: workerRuntimeTools({ + echo: async (args) => { calls.push(args); return { echoed: args } }, + fail: async () => { throw new Error('nope') }, + // A non-Error throw: the host renders it, the program still catches. + failRaw: async () => { throw 'raw-nope' }, + }), + }) + expect(result.error).toBeUndefined() + expect(result.value).toEqual({ + first: { echoed: { n: 1 } }, + caught: { isTyped: true, name: 'ToolCallError', toolName: 'fail', message: 'nope' }, + caughtRaw: { name: 'ToolCallError', toolName: 'failRaw', message: 'raw-nope' }, + }) + expect(calls).toEqual([{ n: 1 }]) + }) + + it('materializes a typed rejection from a generic namespace descriptor', async () => { + const { runtime } = await setup() + const result = await runtime.run({ + program: ` + try { await helpers.fail({}) } catch (error) { + return { + isTyped: error instanceof HelperCallError, + name: error.name, + helperName: error.helperName, + message: error.message, + }; + } + `, + bindings: [{ + global: 'helpers', + functions: { fail: async () => { throw new Error('nope') } }, + errorClass: { name: 'HelperCallError', memberNameProperty: 'helperName' }, + }], + }) + expect(result.value).toEqual({ + isTyped: true, + name: 'HelperCallError', + helperName: 'fail', + message: 'nope', + }) + }) + + it('bridges a deeply nested lossless JSON argument, resolution, and completion', async () => { + const { runtime } = await setup() + const result = await runtime.run({ + program: ` + let value = 'leaf'; + for (let depth = 0; depth < 3_000; depth++) value = [value]; + return await tools.echo(value); + `, + bindings: workerRuntimeTools({ echo: async args => args }), + }) + + expect(result.error).toBeUndefined() + let cursor = result.value + for (let depth = 0; depth < 3_000; depth++) { + expect(Array.isArray(cursor)).toBe(true) + cursor = Array.isArray(cursor) ? cursor[0] : undefined + } + expect(cursor).toBe('leaf') + }, 15_000) + + it('reports non-erasable syntax as an exception without spawning a worker', async () => { + const { runtime } = await setup() + const result = await runtime.run({ program: 'enum E { A }\nreturn 1', bindings: [] }) + expect(result.error?.kind).toBe('exception') + expect(result.error?.message).toMatch(/enum|strip/i) + }) + + it('reports a runtime throw as an exception with the message', async () => { + const { runtime } = await setup() + const result = await runtime.run({ program: 'throw new Error("kaboom")', bindings: [] }) + expect(result.error?.kind).toBe('exception') + expect(result.error?.message).toContain('kaboom') + }) + + it('gives the program an EMPTY environment', async () => { + const { runtime } = await setup() + const result = await runtime.run({ program: 'return JSON.stringify(process.env)', bindings: [] }) + expect(result.value).toBe('{}') + }) + + it('rejects a non-lossless completion instead of replacing it with rendered text', async () => { + const { runtime } = await setup() + const result = await runtime.run({ program: 'return { f: () => 1 }', bindings: [] }) + expect(result.value).toBeUndefined() + expect(result.error).toEqual({ kind: 'invalid-output', message: 'program completion must be lossless JSON' }) + }) + + it('completes a program that returns nothing with no value at all', async () => { + const { runtime } = await setup() + const result = await runtime.run({ program: 'const x = 1', bindings: [] }) + expect(result.error).toBeUndefined() + expect('value' in result).toBe(false) + }) + + it('keeps logs streamed before a failure', async () => { + const { runtime } = await setup() + const result = await runtime.run({ + program: 'console.log("before"); throw new Error("after-log")', + bindings: [], + }) + expect(result.error?.kind).toBe('exception') + expect(result.logs).toContain('before') + }) + }) + + describe(`${label} — budgets and containment (real workers)`, () => { + it('ends a hot loop at the compute budget — including behind a pending decoy dispatch', async () => { + const { runtime } = await setup({ computeMs: 300, maxWallMs: 30_000 }) + const result = await runtime.run({ + // The decoy: fire a call at a never-resolving binding WITHOUT awaiting, + // then spin. Host-side pending-call bookkeeping would pause a naive + // budget here; measured busy time cannot be fooled. + program: 'void tools.slow({}); for (;;) {}', + bindings: workerRuntimeTools({ slow: () => new Promise(() => {}) }), + }) + expect(result.error?.kind).toBe('timeout') + expect(result.error?.message).toContain('compute budget') + }, 15_000) + + it('does not charge time spent awaiting a slow binding against the compute budget', async () => { + // Keep the binding delay above the compute allowance while leaving enough + // headroom for worker bootstrap on loaded CI hosts. + const { runtime } = await setup({ computeMs: 1_000, maxWallMs: 30_000 }) + const result = await runtime.run({ + program: 'return await tools.slow({})', + bindings: workerRuntimeTools({ slow: () => new Promise(resolve => setTimeout(() => { resolve('slow-done') }, 1_500)) }), + }) + expect(result.error).toBeUndefined() + expect(result.value).toBe('slow-done') + }, 15_000) + + it('ends an idle-forever run at the wall-clock ceiling', async () => { + const { runtime } = await setup({ computeMs: 30_000, maxWallMs: 400 }) + const result = await runtime.run({ + program: 'await tools.never({}); return 1', + bindings: workerRuntimeTools({ never: () => new Promise(() => {}) }), + }) + expect(result.error?.kind).toBe('timeout') + expect(result.error?.message).toContain('wall-clock ceiling') + }, 15_000) + + it('reports an abort mid-run and stops the worker', async () => { + const { runtime } = await setup() + const controller = new AbortController() + setTimeout(() => { controller.abort('user-cancel') }, 150) + const result = await runtime.run({ program: 'for (;;) {}', bindings: [], signal: controller.signal }) + expect(result.error).toEqual({ kind: 'abort', message: 'user-cancel' }) + }, 15_000) + + it('reports a pre-aborted signal without spawning', async () => { + const { runtime } = await setup() + const controller = new AbortController() + controller.abort('too-late') + const result = await runtime.run({ program: 'return 1', bindings: [], signal: controller.signal }) + expect(result.error).toEqual({ kind: 'abort', message: 'too-late' }) + }) + + it('applies the outer-output cap to failures before worker startup', async () => { + const capped = await setup({ maxOutputBytes: 64 }) + const controller = new AbortController() + controller.abort('A'.repeat(1_000)) + const aborted = await capped.runtime.run({ program: 'return 1', bindings: [], signal: controller.signal }) + expect(aborted).toEqual({ logs: [], error: { kind: 'output-limit', message: 'outer output exceeded 64 bytes' } }) + + const minimal = await setup({ maxOutputBytes: 4 }) + const invalid = await minimal.runtime.run({ program: 'enum E { A }\nreturn 1', bindings: [] }) + expect(invalid.error?.kind).toBe('output-limit') + expect(Buffer.byteLength(JSON.stringify(invalid.logs), 'utf8') + Buffer.byteLength(JSON.stringify(invalid.error?.message), 'utf8')).toBeLessThanOrEqual(4) + }) + + it('drops a binding resolution that lands after the run settled', async () => { + const { runtime } = await setup() + const controller = new AbortController() + let replyDelivered!: Promise + const result = await runtime.run({ + program: 'void tools.late({}); for (;;) {}', + bindings: workerRuntimeTools({ + // Anchored on invocation: abort 100ms after the call reaches the + // host, resolve 400ms after — by then the run has settled, so the + // resolution's reply hits the post-settlement drop. + late: () => new Promise((resolve) => { + setTimeout(() => { controller.abort('cancel-now') }, 100) + replyDelivered = new Promise(done => setTimeout(() => { resolve('too-late'); done() }, 400)) + }), + }), + signal: controller.signal, + }) + expect(result.error).toEqual({ kind: 'abort', message: 'cancel-now' }) + // Let the late resolution actually fire so its reply executes instead of + // being cancelled with the test. + await replyDelivered + }, 15_000) + + it('contains an OOM under resourceLimits as worker-exit, host process healthy', async () => { + const { runtime } = await setup({ maxOldGenerationSizeMb: 32 }) + const result = await runtime.run({ + program: 'const hog = []; for (;;) hog.push(new Array(1e6).fill(1));', + bindings: [], + }) + expect(result.error?.kind).toBe('worker-exit') + // And the host is fine: run something else. + const after = await runtime.run({ program: 'return "alive"', bindings: [] }) + expect(after.value).toBe('alive') + }, 30_000) + + it('reports a worker that exits before publishing a completion', async () => { + const { runtime } = await setup() + const result = await runtime.run({ program: 'process.exit(7)', bindings: [] }) + expect(result).toEqual({ + logs: [], + error: { kind: 'worker-exit', message: 'worker exited with code 7 before completing' }, + }) + }) + + it('fails runaway log output explicitly while retaining a bounded prefix', async () => { + const { runtime } = await setup({ maxOutputBytes: 300 }) + const result = await runtime.run({ + program: 'for (let i = 0; i < 1000; i++) console.log("spam line", i); return 1', + bindings: [], + }) + expect(result.error).toEqual({ kind: 'output-limit', message: 'outer output exceeded 300 bytes' }) + expect(result.value).toBeUndefined() + expect(result.logs.length).toBeGreaterThan(0) + expect(Buffer.byteLength(JSON.stringify(result.logs), 'utf8')).toBeLessThan(300) + }) + + it('retains a fitting prefix when one oversized log is the first output', async () => { + const { runtime } = await setup({ maxOutputBytes: 96 }) + const result = await runtime.run({ + program: 'console.log(`start-${`😀"\\\\\\n`.repeat(100)}`); return null', + bindings: [], + }) + expect(result.error).toEqual({ kind: 'output-limit', message: 'outer output exceeded 96 bytes' }) + expect(result.logs).toHaveLength(1) + expect(result.logs[0]?.startsWith('start-')).toBe(true) + expect(Buffer.byteLength(JSON.stringify(result.logs), 'utf8') + + Buffer.byteLength(JSON.stringify(result.error?.message), 'utf8')).toBeLessThanOrEqual(96) + }) + + it('fails an oversized return value without substituting a string', async () => { + const { runtime } = await setup({ maxOutputBytes: 64 }) + const result = await runtime.run({ program: 'return "y".repeat(10_000)', bindings: [] }) + expect(result.value).toBeUndefined() + expect(result.error).toEqual({ kind: 'output-limit', message: 'outer output exceeded 64 bytes' }) + }) + + it('uses UTF-8 serialized bytes at the exact completion boundary', async () => { + const exact = await setup({ maxOutputBytes: 7 }) + const exactResult = await exact.runtime.run({ program: 'return "€"', bindings: [] }) + // [] costs two bytes and JSON serialization of "€" costs five. + expect(exactResult).toEqual({ logs: [], value: '€' }) + + const over = await setup({ maxOutputBytes: 6 }) + const overResult = await over.runtime.run({ program: 'return "€"', bindings: [] }) + expect(overResult.error?.kind).toBe('output-limit') + }) + + it('accounts logs and completion in one exact combined ledger', async () => { + // JSON(["abc"]) is seven bytes and JSON("xy") is four. + const exact = await setup({ maxOutputBytes: 11 }) + expect(await exact.runtime.run({ program: 'console.log("abc"); return "xy"', bindings: [] })) + .toEqual({ logs: ['abc'], value: 'xy' }) + + const over = await setup({ maxOutputBytes: 10 }) + const result = await over.runtime.run({ program: 'console.log("abc"); return "xy"', bindings: [] }) + expect(result.value).toBeUndefined() + expect(result.error?.kind).toBe('output-limit') + expect(Buffer.byteLength(JSON.stringify(result.logs), 'utf8') + Buffer.byteLength(JSON.stringify(result.error?.message), 'utf8')).toBeLessThanOrEqual(10) + }) + + it('accounts logs and exception diagnostics before the worker port boundary', async () => { + // JSON(["abc"]) is seven bytes and JSON("xy") is four. + const exact = await setup({ maxOutputBytes: 11 }) + expect(await exact.runtime.run({ program: 'console.log("abc"); throw "xy"', bindings: [] })) + .toEqual({ logs: ['abc'], error: { kind: 'exception', message: 'xy' } }) + + const over = await setup({ maxOutputBytes: 10 }) + const result = await over.runtime.run({ program: 'console.log("abc"); throw "xy"', bindings: [] }) + expect(result.error?.kind).toBe('output-limit') + expect(Buffer.byteLength(JSON.stringify(result.logs), 'utf8') + + Buffer.byteLength(JSON.stringify(result.error?.message), 'utf8')).toBeLessThanOrEqual(10) + }) + + it('does not send a giant Error stack across the worker port', async () => { + const { runtime } = await setup({ maxOutputBytes: 64 }) + const result = await runtime.run({ + program: 'throw new Error("x".repeat(1_000_000))', + bindings: [], + }) + expect(result).toEqual({ + logs: [], + error: { kind: 'output-limit', message: 'outer output exceeded 64 bytes' }, + }) + }) + + it('completes a program that awaits its write callback, capturing the chunk', async () => { + // Node's write(chunk[, encoding][, callback]) contract: dropping the + // callback would leave this promise pending until the wall ceiling and + // misreport a completed program as a timeout. + const { runtime } = await setup({ maxWallMs: 2_000 }) + const result = await runtime.run({ + program: 'await new Promise(resolve => process.stdout.write("flushed", resolve)); return "done"', + bindings: [], + }) + expect(result.error).toBeUndefined() + expect(result.value).toBe('done') + expect(result.logs).toContain('flushed') + }) + + it('returns a large JSON container exactly when the outer cap permits it', async () => { + const { runtime } = await setup() + const result = await runtime.run({ program: 'return new Array(50_000).fill(7)', bindings: [] }) + expect(result.error).toBeUndefined() + expect(result.value).toEqual(new Array(50_000).fill(7)) + }) + + it('returns an exact completion at the default 64 MiB combined boundary', async () => { + const { runtime } = await setup() + // [] costs two bytes and the JSON string contributes two quotes, leaving + // exactly this many payload bytes under the 67_108_864-byte default. + const result = await runtime.run({ program: 'return "x".repeat(67_108_860)', bindings: [] }) + expect(result.error).toBeUndefined() + expect(result.logs).toEqual([]) + expect(result.value).toHaveLength(67_108_860) + }, 60_000) + + it('fails one byte over the default 64 MiB combined boundary', async () => { + const { runtime } = await setup() + const result = await runtime.run({ program: 'return "x".repeat(67_108_861)', bindings: [] }) + expect(result.value).toBeUndefined() + expect(result.error).toEqual({ kind: 'output-limit', message: 'outer output exceeded 67108864 bytes' }) + }, 60_000) + + it('drains pipe output queued before terminal worker teardown completes', async () => { + const { runtime } = await setup({ maxOutputBytes: 200_000 }) + const payload = `late-pipe-${'x'.repeat(100_000)}` + const result = await runtime.run({ + program: ` + const { parentPort } = await import('node:worker_threads'); + const write = (text) => Object.getPrototypeOf(process.stdout).write.call(process.stdout, text); + write('late-pipe-' + 'x'.repeat(100_000)); + parentPort.postMessage({ type: 'done', value: ['done'] }); + for (;;) {} + `, + bindings: [], + }) + expect(result.error).toBeUndefined() + expect(result.value).toBe('done') + expect(result.logs.join('') === payload).toBe(true) + }, 15_000) + }) + + describe(`${label} — hostile programs (real workers)`, () => { + it('survives forged port traffic: unknown binding names, duplicate ids, junk shapes', async () => { + const { runtime } = await setup() + const result = await runtime.run({ + program: ` + const { parentPort } = await import('node:worker_threads'); + parentPort.postMessage({ type: 'call', id: 7777, global: 'tools', name: 'missing', args: {} }); + parentPort.postMessage({ type: 'call', id: 7777, global: 'tools', name: 'missing', args: {} }); + parentPort.postMessage({ type: 'call', id: 7778, global: 'tools', name: 'constructor', args: {} }); + parentPort.postMessage({ type: 'junk' }); + return await tools.real({}); + `, + bindings: workerRuntimeTools({ real: async () => 'still-works' }), + }) + expect(result.error).toBeUndefined() + expect(result.value).toBe('still-works') + }) + + it('survives arbitrary junk on the port: non-objects, junk types, malformed calls, logs, and dones', async () => { + const { runtime } = await setup() + const result = await runtime.run({ + program: ` + const { parentPort } = await import('node:worker_threads'); + for (const junk of [ + null, 42, 'junk', [], + { type: 'nope' }, + { type: 'call' }, + { type: 'call', id: 'x', global: 'tools', name: 'real', args: {} }, + { type: 'call', id: 1e9, global: 7, name: 'real', args: {} }, + { type: 'call', id: 1e9, global: 'tools', name: 7, args: {} }, + { type: 'log' }, + { type: 'log', text: null }, + { type: 'log', text: 7 }, + { type: 'log', text: {} }, + { type: 'done', error: 5 }, + { type: 'done', error: { kind: 'exception', message: 5 } }, + { type: 'done', error: { kind: 'invented', message: 'bad kind' } }, + ]) parentPort.postMessage(junk); + return await tools.real({}); + `, + bindings: workerRuntimeTools({ real: async () => 'still-works' }), + }) + expect(result.error).toBeUndefined() + expect(result.value).toBe('still-works') + expect(result.logs).toEqual([]) + }) + + it('fails forged log floods and forged done values through the same outer cap', async () => { + const { runtime } = await setup({ maxOutputBytes: 200 }) + const result = await runtime.run({ + // Forged messages bypass worker-side capture and completion checks; + // the outer ledger must still contain them. + program: ` + const { parentPort } = await import('node:worker_threads'); + for (let i = 0; i < 50; i++) parentPort.postMessage({ type: 'log', text: 'F'.repeat(100), forged: true }); + parentPort.postMessage({ type: 'done', value: ['V'.repeat(100000)] }); + for (;;) {} + `, + bindings: [], + }) + expect(result.value).toBeUndefined() + expect(result.error).toEqual({ kind: 'output-limit', message: 'outer output exceeded 200 bytes' }) + expect(Buffer.byteLength(JSON.stringify(result.logs), 'utf8')).toBeLessThan(200) + }) + + it('re-caps an oversized forged done value at the host boundary', async () => { + const { runtime } = await setup({ maxOutputBytes: 64 }) + const result = await runtime.run({ + program: ` + const { parentPort } = await import('node:worker_threads'); + parentPort.postMessage({ type: 'done', value: ['V'.repeat(100_000)] }); + for (;;) {} + `, + bindings: [], + }) + expect(result).toEqual({ + logs: [], + error: { kind: 'output-limit', message: 'outer output exceeded 64 bytes' }, + }) + }) + + it('drops a malformed forged done carrying both value and error', async () => { + const { runtime } = await setup() + const result = await runtime.run({ + program: ` + const { parentPort } = await import('node:worker_threads'); + parentPort.postMessage({ type: 'done', value: 'lied', error: { kind: 'exception', message: 'fake failure' } }); + return 'honest'; + `, + bindings: [], + }) + expect(result).toEqual({ logs: [], error: { kind: 'exception', message: 'fake failure' } }) + }) + + it('contains a deeply nested forged completion without overflowing the host meter', async () => { + const { runtime } = await setup() + const result = await runtime.run({ + program: ` + const { parentPort } = await import('node:worker_threads'); + const value = []; + for (let depth = 0; depth < 3_000; depth++) value.push({ kind: 'array', length: 1 }); + value.push(null); + setTimeout(() => { parentPort.postMessage({ type: 'done', value }) }, 25); + // Prevent bootstrap's normal undefined completion from racing the forged terminal. + await new Promise(() => {}); + `, + bindings: [], + }) + expect(result.error).toBeUndefined() + let value = result.value + let depth = 0 + while (Array.isArray(value)) { + expect(value).toHaveLength(1) + value = value[0] + depth += 1 + } + expect(depth).toBe(3_000) + expect(value).toBeNull() + }, 15_000) + + it('turns forged over-limit error text into output-limit at the host', async () => { + const { runtime } = await setup({ maxOutputBytes: 64 }) + const result = await runtime.run({ + program: ` + const { parentPort } = await import('node:worker_threads'); + parentPort.postMessage({ type: 'done', error: { kind: 'exception', message: '€'.repeat(1000) } }); + for (;;) {} + `, + bindings: [], + }) + expect(result.error).toEqual({ kind: 'output-limit', message: 'outer output exceeded 64 bytes' }) + }) + + it('answers a binding whose resolution is not lossless JSON with a typed failure reply', async () => { + const { runtime } = await setup() + const result = await runtime.run({ + program: 'try { await tools.bad({}) } catch (error) { return { name: error.name, toolName: error.toolName, message: error.message } }', + bindings: workerRuntimeTools({ bad: async () => (() => 1) }), + }) + expect(result.value).toEqual({ name: 'ToolCallError', toolName: 'bad', message: 'binding resolution must be lossless JSON' }) + }) + + it('rejects lossy binding arguments in the worker before invoking the host binding', async () => { + const { runtime } = await setup() + let calls = 0 + const result = await runtime.run({ + program: ` + const decorated = [1]; Object.defineProperty(decorated, 'extra', { value: true }); + const values = [new Date(), decorated, () => 1]; + const failures = []; + for (const value of values) { + try { await tools.never(value) } catch (error) { + failures.push({ typed: error instanceof ToolCallError, name: error.name, toolName: error.toolName, message: error.message }); + } + } + return failures; + `, + bindings: workerRuntimeTools({ never: async () => { calls += 1; return null } }), + }) + expect(calls).toBe(0) + expect(result.value).toEqual(new Array(3).fill({ + typed: true, + name: 'ToolCallError', + toolName: 'never', + message: 'binding arguments must be lossless JSON', + })) + }) + + it('rejects intrinsic-looking exotic objects as arguments and completions', async () => { + const { runtime } = await setup() + let calls = 0 + const forgeObject = ` + const prototype = Object.create(null); + const SpoofedObject = function Object() {}; + SpoofedObject.prototype = prototype; + Object.defineProperty(prototype, 'constructor', { value: SpoofedObject }); + const forged = Object.assign(Object.create(prototype), { value: 1 }); + Function.prototype.toString = () => 'function Object() { [native code] }'; + ` + const argument = await runtime.run({ + program: `${forgeObject} + try { await tools.never(forged) } catch (error) { + return { typed: error instanceof ToolCallError, name: error.name, toolName: error.toolName, message: error.message }; + } + `, + bindings: workerRuntimeTools({ never: async () => { calls += 1; return null } }), + }) + expect(calls).toBe(0) + expect(argument.value).toEqual({ + typed: true, + name: 'ToolCallError', + toolName: 'never', + message: 'binding arguments must be lossless JSON', + }) + + const completion = await runtime.run({ program: `${forgeObject}\nreturn forged`, bindings: [] }) + expect(completion).toEqual({ + logs: [], + error: { kind: 'invalid-output', message: 'program completion must be lossless JSON' }, + }) + }) + + it('preserves binding and completion JSON after model code mutates boundary globals', async () => { + const { runtime } = await setup() + const result = await runtime.run({ + program: ` + const arrayPrototype = Array.prototype; + const objectPrototype = Object.prototype; + const setPrototype = Set.prototype; + const stringPrototype = String.prototype; + Array.isArray = () => false; + arrayPrototype.at = arrayPrototype.includes = arrayPrototype.pop = arrayPrototype.push = () => { throw new Error('mutated array method') }; + Object.defineProperty = Object.getOwnPropertyDescriptor = Object.getPrototypeOf = Object.keys = () => { throw new Error('mutated object method') }; + Object.hasOwn = () => false; + Object.is = () => true; + objectPrototype.propertyIsEnumerable = () => false; + Number.isFinite = Number.isSafeInteger = () => false; + Reflect.apply = Reflect.ownKeys = () => { throw new Error('mutated reflect method') }; + setPrototype.add = setPrototype.delete = setPrototype.has = () => { throw new Error('mutated set method') }; + stringPrototype.charCodeAt = stringPrototype.codePointAt = stringPrototype.slice = () => { throw new Error('mutated string method') }; + Buffer.byteLength = () => 0; + Function.prototype.toString = () => 'mutated'; + objectPrototype.get = () => undefined; + objectPrototype.constructor = arrayPrototype.constructor = null; + globalThis.Array = globalThis.Buffer = globalThis.Error = globalThis.Function = globalThis.Number = globalThis.Object = globalThis.Reflect = globalThis.Set = globalThis.String = undefined; + const echoed = await tools.echo({ request: ['€', 1] }); + let failure; + try { await tools.fail({}) } catch (error) { + failure = { typed: error instanceof ToolCallError, name: error.name, toolName: error.toolName, message: error.message }; + } + return { echoed, failure, completion: { ok: true, amount: 42 } }; + `, + bindings: workerRuntimeTools({ echo: async args => args, fail: async () => { throw new Error('nope') } }), + }) + expect(result).toEqual({ + logs: [], + value: { + echoed: { request: ['€', 1] }, + failure: { typed: true, name: 'ToolCallError', toolName: 'fail', message: 'nope' }, + completion: { ok: true, amount: 42 }, + }, + }) + }) + + it('rejects forged lossy binding arguments again at the host boundary', async () => { + const { runtime } = await setup() + let calls = 0 + const result = await runtime.run({ + program: ` + const { parentPort } = await import('node:worker_threads'); + const forged = (id, args) => new Promise((resolve) => { + const receive = (message) => { + if (message?.type !== 'reply' || message.id !== id) return; + parentPort.off('message', receive); + resolve(message); + }; + parentPort.on('message', receive); + parentPort.postMessage({ type: 'call', id, global: 'tools', name: 'never', args }); + }); + const sparse = []; sparse.length = 1; + const cycle = {}; cycle.self = cycle; + return await Promise.all([ + forged(8001, new Date()), + forged(8002, -0), + forged(8003, sparse), + forged(8004, cycle), + ]); + `, + bindings: workerRuntimeTools({ never: async () => { calls += 1; return null } }), + }) + expect(calls).toBe(0) + expect(result.value).toEqual([8001, 8002, 8003, 8004].map(id => ({ + type: 'reply', + id, + ok: false, + message: 'binding arguments must be lossless JSON', + }))) + }) + + it('contains throwing getters while snapshotting binding resolutions', async () => { + const { runtime } = await setup() + const result = await runtime.run({ + program: 'try { await tools.bad({}) } catch (error) { return { name: error.name, toolName: error.toolName, message: error.message } }', + bindings: workerRuntimeTools({ bad: async () => Object.defineProperty({}, 'bad', { enumerable: true, get() { throw new Error('getter exploded') } }) }), + }) + expect(result.value).toEqual({ name: 'ToolCallError', toolName: 'bad', message: 'binding resolution must be lossless JSON' }) + }) + + it('revalidates a forged lossy completion at the host boundary', async () => { + const { runtime } = await setup() + const result = await runtime.run({ + program: ` + const { parentPort } = await import('node:worker_threads'); + parentPort.postMessage({ type: 'done', value: -0 }); + for (;;) {} + `, + bindings: [], + }) + expect(result).toEqual({ logs: [], error: { kind: 'invalid-output', message: 'program completion must be lossless JSON' } }) + }) + + it('honors a forged worker-side output-limit signal', async () => { + const { runtime } = await setup() + const result = await runtime.run({ + program: ` + const { parentPort } = await import('node:worker_threads'); + parentPort.postMessage({ type: 'output-limit' }); + for (;;) {} + `, + bindings: [], + }) + expect(result).toEqual({ logs: [], error: { kind: 'output-limit', message: 'outer output exceeded 67108864 bytes' } }) + }) + + it('exposes binding names that collide with Object.prototype as ordinary functions', async () => { + const { runtime } = await setup() + const result = await runtime.run({ + program: 'return [await tools["__proto__"]({}), await tools["constructor"]({}), typeof tools["hasOwnProperty"]]', + // Computed keys: a literal `'__proto__': …` entry would SET the record's + // prototype instead of declaring a binding of that name. + bindings: workerRuntimeTools({ ['__proto__']: async () => 'proto-ok', ['constructor']: async () => 'ctor-ok' }), + }) + expect(result.value).toEqual(['proto-ok', 'ctor-ok', 'undefined']) + }) + }) + + describe(`${label} — seam misuse and lifecycle`, () => { + it('rejects invalid and duplicate binding globals loudly', async () => { + const { runtime } = await setup() + const cases: [string, RegExp][] = [ + ['not valid!', /not a usable identifier/], + ['await', /not a usable identifier/], + ['console', /duplicate binding global/], + ] + for (const [global, message] of cases) { + await expect(runtime.run({ program: 'return 1', bindings: [{ global, functions: {} }] })).rejects.toThrow(message) + } + await expect(runtime.run({ + program: 'return 1', + bindings: [{ global: 'tools', functions: {} }, { global: 'tools', functions: {} }], + })).rejects.toThrow(/duplicate binding global/) + + await expect(runtime.run({ + program: 'return typeof ToolCallError', + bindings: [{ global: 'ToolCallError', functions: {} }], + })).resolves.toMatchObject({ value: 'object' }) + }) + + it('rejects malformed or colliding binding error-class declarations', async () => { + const { runtime } = await setup() + const run = async (bindings: CodeBindingNamespace[]) => await runtime.run({ program: 'return 1', bindings }) + const namespace = (global: string, name: string, memberNameProperty = 'memberName'): CodeBindingNamespace => ({ + global, + functions: {}, + errorClass: { name, memberNameProperty }, + }) + + await expect(run([namespace('tools', 'not valid!')])).rejects.toThrow(/error class.*not a usable identifier/) + await expect(run([namespace('tools', 'await')])).rejects.toThrow(/error class.*not a usable identifier/) + await expect(run([namespace('tools', 'console')])).rejects.toThrow(/duplicate injected global/) + await expect(run([namespace('tools', 'tools')])).rejects.toThrow(/duplicate injected global/) + await expect(run([ + namespace('tools', 'CallError'), + namespace('helpers', 'CallError'), + ])).rejects.toThrow(/duplicate injected global/) + await expect(run([namespace('tools', 'CallError', '')])).rejects.toThrow(/member property.*not usable/) + await expect(run([namespace('tools', 'CallError', 'message')])).rejects.toThrow(/member property.*not usable/) + }) + + it('rejects config values that are not positive numbers', async () => { + await expect(setup({ computeMs: -1 })).rejects.toThrow(/positive number/) + }) + + it('rejects a maxWallMs above Node\'s maximum timer delay', async () => { + // setTimeout clamps a delay past 2^31-1 ms to 1 ms, so the positivity check + // alone would accept a 25-day ceiling that expires on the first tick. + await expect(setup({ maxWallMs: 2_147_483_648 })) + .rejects.toThrow(/maxWallMs must be at most 2147483647/) + // The boundary itself is usable. + await expect(setup({ maxWallMs: 2_147_483_647 })).resolves.toBeTruthy() + }) + + it('requires maxOutputBytes to fit the smallest counted outer payloads', async () => { + await expect(setup({ maxOutputBytes: 3 })).rejects.toThrow(/safe integer of at least 4/) + await expect(setup({ maxOutputBytes: 4.5 })).rejects.toThrow(/safe integer of at least 4/) + }) + + it('keeps runs isolated: no state survives from one run to the next', async () => { + const { runtime } = await setup() + await runtime.run({ program: 'globalThis.leak = "value"; return 1', bindings: [] }) + const second = await runtime.run({ program: 'return typeof globalThis.leak', bindings: [] }) + expect(second.value).toBe('undefined') + }) + + it('disposal aborts in-flight runs, awaits worker exit, and rejects later runs', async () => { + const { runtime, dispose } = await setup() + const inflight: Promise = runtime.run({ program: 'for (;;) {}', bindings: [] }) + // Give the worker a moment to actually start spinning. + await new Promise(resolve => setTimeout(resolve, 200)) + await dispose() + const result = await inflight + expect(result.error).toEqual({ kind: 'abort', message: 'runtime disposed' }) + await expect(runtime.run({ program: 'return 1', bindings: [] })).rejects.toThrow(/after disposal/) + }, 15_000) + }) +} diff --git a/packages/context/workspace-context/tests/workspace-context.spec.ts b/packages/context/workspace-context/tests/workspace-context.spec.ts index 8620c38981..fcdcd7c422 100644 --- a/packages/context/workspace-context/tests/workspace-context.spec.ts +++ b/packages/context/workspace-context/tests/workspace-context.spec.ts @@ -117,12 +117,6 @@ class RecordingFileSystem extends FileSystem { return this.entries.get(target.targetKey)?.content ?? '' } - override async readTextBounded(target: FsTarget, maxBytes: number, signal?: AbortSignal): Promise { - const text = await this.readText(target, signal) - if (Buffer.byteLength(text) > maxBytes) throw new Error('too large') - return text - } - override async streamText(target: FsTarget, signal?: AbortSignal): Promise> { if (signal !== undefined) this.signals.push(signal) signal?.throwIfAborted() diff --git a/packages/cordis/tool-cordis/src/api-catalog.ts b/packages/cordis/tool-cordis/src/api-catalog.ts index c7b6df254e..2eb410acc6 100644 --- a/packages/cordis/tool-cordis/src/api-catalog.ts +++ b/packages/cordis/tool-cordis/src/api-catalog.ts @@ -336,10 +336,6 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [ signature: 'abstract readText(target: FsTarget, signal?: AbortSignal): Promise', jsDoc: '/**\n * Read the whole regular text file as a single decoded string.\n * @param target - the resolved target to read.\n * @param signal - aborts the read.\n * @returns the full decoded UTF-8 content.\n */', }, - { - signature: 'abstract readTextBounded(target: FsTarget, maxBytes: number, signal?: AbortSignal): Promise', - jsDoc: '/**\n * Read one regular UTF-8 text file through a backend-owned stable handle,\n * rejecting before more than `maxBytes` are retained. The size check and\n * bytes read are one operation: a caller must not emulate this with\n * {@link stat} followed by {@link readText}, which admits growth and path\n * replacement races between the two calls.\n * @param target - the resolved target to read.\n * @param maxBytes - positive safe-integer byte ceiling.\n * @param signal - aborts the open/read operation.\n * @returns the complete decoded text when it fits.\n */', - }, { signature: 'abstract streamText(target: FsTarget, signal?: AbortSignal): Promise>', jsDoc: '/**\n * Stream the whole regular text file as decoded text chunks (same text\n * semantics as {@link readText}, for large files). The backend owns\n * cross-chunk UTF-8 decoding and binary rejection so the policy layer never\n * touches raw bytes.\n * @param target - the resolved target to read.\n * @param signal - aborts the stream, including between chunks.\n * @returns the chunk iterable, decoded and validated like {@link readText}.\n */', @@ -1000,7 +996,7 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [ }, { signature: 'abstract spawnTerminal(spec: SubprocessTerminalSpawnSpec): Promise', - jsDoc: '/**\n * Allocate a real terminal and start one owned process session. This is the\n * only non-pipe process primitive: implementations own terminal byte I/O,\n * foreground groups, signals, and complete session-tree cleanup.\n * @param spec - fully specified argv, cwd, environment, dimensions, grace, and cancellation.\n * @returns the live terminal handle after allocation succeeds.\n */', + jsDoc: '/**\n * Allocate a real terminal and start one owned process session. This is the\n * only non-pipe process primitive: implementations own terminal byte I/O,\n * foreground groups, signals, and complete session-tree cleanup.\n * @param spec - fully specified argv, cwd, environment, dimensions, grace, and allocation cancellation.\n * @returns the live terminal handle after allocation succeeds.\n */', }, ], }, @@ -2909,7 +2905,7 @@ export const TYPE_API: readonly TypeApiEntry[] = [ }, { name: 'SubprocessTerminalHandle', - declaration: 'export interface SubprocessTerminalHandle {\n readonly pid: number;\n readonly output: Readable;\n readonly done: Promise;\n write(data: Uint8Array): Promise;\n inspectForeground(): Promise;\n signalForeground(signal: SubprocessTerminalSignal): Promise;\n terminate(): void;\n waitForExit(signal?: AbortSignal): Promise;\n}', + declaration: 'export interface SubprocessTerminalHandle {\n readonly pid: number;\n readonly output: Readable;\n readonly done: Promise;\n write(data: string): Promise;\n inspectForeground(): Promise;\n signalForeground(signal: SubprocessTerminalSignal): Promise;\n terminate(): Promise;\n}', }, { name: 'SubprocessTerminalSignal', diff --git a/packages/fs/fs-local/README.i18n.yaml b/packages/fs/fs-local/README.i18n.yaml index 57d2a32f97..e1870cbf41 100644 --- a/packages/fs/fs-local/README.i18n.yaml +++ b/packages/fs/fs-local/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/fs/fs-local/README.md -README.md: f5cf2441adcd62e55c6169ceb766f88382e314f5 -README.zh.md: 40f5a83626780bae8f335c80662721bae6cc0b0b +README.md: fe0e5e9dec07fad745d6bea28da9009e517c7d85 +README.zh.md: 195f3963328035e9c6c382dd924cd04ee9e8c642 diff --git a/packages/fs/fs-local/README.md b/packages/fs/fs-local/README.md index f5cf2441ad..fe0e5e9dec 100644 --- a/packages/fs/fs-local/README.md +++ b/packages/fs/fs-local/README.md @@ -17,7 +17,7 @@ await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) - **`resolve(path, opts?)`** — a relative `path` resolves against `opts.cwd` when the caller supplies one (the model-facing tools pass the calling agent's session cwd — see [the per-session cwd Agent Note](../../../.agents/notes/implemented/architecture/2026-07-02-fs-per-session-cwd.md)), else `config.cwd` (default `process.cwd()`); an absolute `path` ignores both. `opts.signal` is checked before and after local resolution, while a remote sibling backend may use it to abort its round-trip. The `targetKey` is the file's `realpath`, so two input paths reaching the same file through symlinks share one identity, and writes/edits land on the link target (preserving the link). A not-yet-existing path uses the realpathed parent directory plus basename when the parent exists; only an unresolvable parent falls back to the absolute path. `displayPath` is the absolute (un-resolved) path. - **Execution-world coordinates** — `processPath` exposes the target's canonical host path, `fileUrl` encodes that path through Node's platform-aware URL conversion, and `contains` uses platform path semantics to test identity or descendant containment without consumers parsing `targetKey`. - **`stat` / `lstat`** — return target metadata or `undefined` when absent. `stat` reports `FsInfo` for an already resolved target (`version` = an opaque token derived from bigint `dev:ino:size:mtimeNs:ctimeNs`, `type` of `file`/`directory`/`other`, byte `size`); path-shaped `lstat` reports `FsPathInfo` without following the final symlink and can therefore return `symlink`. Both check cancellation before and after their asynchronous metadata probe, so an abort that lands in flight reports `FS_ABORTED` rather than stale absence. -- **`readText` / `readTextBounded` / `streamText`** — UTF-8 only. `readText` reads the whole file; `readTextBounded` opens one no-follow, nonblocking handle, verifies it is regular, and retains at most `maxBytes + 1` bytes so growth cannot bypass the cap; `streamText` decodes chunks so a huge file need not be held whole in memory. All reject invalid UTF-8 and NUL-byte binary samples (`FS_NOT_TEXT`) and non-regular targets. The `read` tool (`@deepseek-ai/dsh-tool-fs`) owns line windowing; protocol consumers such as the LSP host use the stable bounded operation. +- **`readText` / `streamText`** — UTF-8 only. `readText` reads the whole file; `streamText` decodes chunks so a huge file need not be held whole in memory and consumers can enforce their own retention bounds. Both reject invalid UTF-8 and NUL-byte binary samples (`FS_NOT_TEXT`) and non-regular targets. The `read` tool (`@deepseek-ai/dsh-tool-fs`) owns line windowing. - **`listDir`** — lists one directory level in stable `name.localeCompare()` order. Each entry carries the child basename, type, resolved child target (`displayPath` under the listed directory, `targetKey` as the realpath identity), and cheap stat metadata (`version`, plus `size` for regular files). It never opens or decodes file contents. Missing targets report `FS_NOT_FOUND`, file/special-file targets report `FS_NOT_DIRECTORY`, aborted calls report `FS_ABORTED`, permission failures report `FS_PERMISSION_DENIED`, and other listing or child metadata I/O failures report `FS_IO_ERROR`. Broken/disappeared children are returned as `other` without metadata, but permission/IO failures while resolving a child fail the whole listing with a structured `FsError`. - **`writeText`** — atomic: writes to a temp file opened exclusively (`wx`, `0o600`) inside a randomly-named private staging dir (`0o700`) next to the target, fsyncs, then renames over the target. An existing file's mode is preserved, while new files default to `0o600`; on Windows a new file inherits the destination directory's DACL, while replacement copies the target DACL onto the empty temp before writing and publishes through `ReplaceFileW` so the original access policy survives ([Windows DACL preservation Agent Note](../../../.agents/notes/implemented/bug-fix/2026-07-19-windows-atomic-write-dacl-preservation.md)). The `expected` guard is OPTIONAL: omitting it unconditionally creates-or-overwrites; `createIfAbsent` creates a missing target and rejects an existing one (`FS_NOT_OBSERVED`); `replaceIfVersion` replaces only at the observed version (a missing target or mismatch is `FS_STALE_VERSION`). - **`editText`** — atomic literal read-modify-write over the same primitive, serialized per target by a mutation lock. The `expected` guard is OPTIONAL: when supplied it verifies the version BEFORE literal matching (a stale edit reports `FS_STALE_VERSION`, never `FS_EDIT_NOT_FOUND`/`FS_AMBIGUOUS_EDIT` against newer content); omitting it edits the current content unconditionally. A missing target reports `FS_STALE_VERSION` either way. LF-normalizes for matching, restores the file's dominant CRLF/LF style, and rejects empty `oldString` / zero matches (`FS_EDIT_NOT_FOUND`) or ambiguous multi-matches without `replace_all` (`FS_AMBIGUOUS_EDIT`). diff --git a/packages/fs/fs-local/README.zh.md b/packages/fs/fs-local/README.zh.md index 40f5a83626..195f396332 100644 --- a/packages/fs/fs-local/README.zh.md +++ b/packages/fs/fs-local/README.zh.md @@ -17,7 +17,7 @@ await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) - **`resolve(path, opts?)`**:相对 `path` 在调用方提供 `opts.cwd` 时以该值为基准解析(面向模型的工具会传入调用 agent(智能体)的会话 cwd;见[每会话 cwd Agent Note](../../../.agents/notes/implemented/architecture/2026-07-02-fs-per-session-cwd.md)),否则以 `config.cwd` 为基准(默认 `process.cwd()`);绝对 `path` 会忽略两者。`opts.signal` 会在本地解析前后检查,远程同级后端则可以用它中止往返。`targetKey` 是文件的 `realpath`,因此经符号链接到达同一文件的两个输入路径会共享一个身份,写入/编辑落在链接目标上,同时保留链接。尚不存在的路径在父目录存在时使用 realpath 后的父目录加 basename;只有父目录无法解析时才回退到绝对路径。`displayPath` 是绝对但未经解析的路径。 - **执行世界坐标**:`processPath` 公开目标的规范化宿主路径,`fileUrl` 通过 Node 的平台感知 URL 转换对该路径编码,`contains` 则使用平台路径语义检查身份相等或后代包含关系,消费方无需解析 `targetKey`。 - **`stat` / `lstat`**:返回目标元数据;目标不存在时返回 `undefined`。`stat` 为已解析目标报告 `FsInfo`(`version` 是由 bigint `dev:ino:size:mtimeNs:ctimeNs` 派生的不透明 token,`type` 为 `file`/`directory`/`other`,`size` 以字节计);路径形态的 `lstat` 不跟随最后一个符号链接,报告 `FsPathInfo`,因此可以返回 `symlink`。两者都会在异步元数据探测前后检查取消,因此飞行中的中止会报告 `FS_ABORTED`,而非陈旧的不存在结果。 -- **`readText` / `readTextBounded` / `streamText`**:只支持 UTF-8。`readText` 读取整个文件;`readTextBounded` 打开一个不跟随符号链接的非阻塞句柄,确认其为普通文件,并最多保留 `maxBytes + 1` 字节,使文件增长无法绕过上限;`streamText` 按分片解码,因此超大文件无需整体保存在内存中。三者都会拒绝无效 UTF-8、包含 NUL 字节的二进制样本(`FS_NOT_TEXT`)以及非普通文件目标。`read` 工具(`@deepseek-ai/dsh-tool-fs`)拥有行窗口逻辑;LSP 主机等协议消费方使用稳定的有界操作。 +- **`readText` / `streamText`**:只支持 UTF-8。`readText` 读取整个文件;`streamText` 按分片解码,因此超大文件无需整体保存在内存中,消费方也可以执行各自的保留上限。两者都会拒绝无效 UTF-8、包含 NUL 字节的二进制样本(`FS_NOT_TEXT`)以及非普通文件目标。`read` 工具(`@deepseek-ai/dsh-tool-fs`)拥有行窗口逻辑。 - **`listDir`**:按稳定的 `name.localeCompare()` 顺序列出一层目录。每个条目携带子项 basename、类型、解析后的子目标(`displayPath` 位于所列目录下,`targetKey` 是 realpath 身份)和低成本 stat 元数据(`version`,普通文件另有 `size`)。它绝不会打开或解码文件内容。缺失目标报告 `FS_NOT_FOUND`,文件/特殊文件目标报告 `FS_NOT_DIRECTORY`,已中止调用报告 `FS_ABORTED`,权限失败报告 `FS_PERMISSION_DENIED`,其他列出或子项元数据 I/O 失败报告 `FS_IO_ERROR`。损坏/消失的子项以无元数据的 `other` 返回,但解析子项时出现权限/I/O 失败会让整个列表以结构化 `FsError` 失败。 - **`writeText`**:原子写入。它会向排他打开的临时文件(`wx`、`0o600`)写入;该文件位于目标旁随机命名的私有暂存目录(`0o700`)内。完成写入和 fsync 后,以 rename 覆盖目标。现有文件的 mode 会保留,新文件默认为 `0o600`;Windows 上的新文件继承目标目录的 DACL,而替换会在写入前把目标 DACL 复制到空临时文件,并通过 `ReplaceFileW` 发布,使原访问政策得以保留(见 [Windows DACL 保留 Agent Note](../../../.agents/notes/implemented/bug-fix/2026-07-19-windows-atomic-write-dacl-preservation.md))。`expected` 防护是可选的:省略时无条件创建或覆盖;`createIfAbsent` 创建缺失目标并拒绝现有目标(`FS_NOT_OBSERVED`);`replaceIfVersion` 只在观察到的版本上替换(目标缺失或版本不匹配均为 `FS_STALE_VERSION`)。 - **`editText`**:在同一原语之上依次执行原子的字面量读取、修改和写入,并通过变更锁按目标串行化。`expected` 防护是可选的:提供时,会在字面量匹配之前校验版本(陈旧编辑报告 `FS_STALE_VERSION`,绝不会针对较新内容报告 `FS_EDIT_NOT_FOUND`/`FS_AMBIGUOUS_EDIT`);省略时,无条件编辑当前内容。无论哪种情况,目标缺失都报告 `FS_STALE_VERSION`。匹配时规范化为 LF,随后恢复文件主要的 CRLF/LF 风格;空 `oldString` / 零匹配报告 `FS_EDIT_NOT_FOUND`,未设置 `replace_all` 的多个匹配则报告 `FS_AMBIGUOUS_EDIT`。 diff --git a/packages/fs/fs-local/src/fsio.ts b/packages/fs/fs-local/src/fsio.ts index e95cb14cca..4925357e45 100644 --- a/packages/fs/fs-local/src/fsio.ts +++ b/packages/fs/fs-local/src/fsio.ts @@ -6,7 +6,7 @@ */ import { randomUUID } from 'node:crypto' -import { constants, createReadStream } from 'node:fs' +import { createReadStream } from 'node:fs' import { chmod, lstat, mkdir, open, readFile, realpath, readdir, rename, rm, stat } from 'node:fs/promises' import type { BigIntStats, Dirent, Stats } from 'node:fs' import { basename, dirname, join, resolve } from 'node:path' @@ -377,69 +377,6 @@ export async function readWholeText(target: LocalTarget, signal?: AbortSignal): * @param signal - aborts between handle operations. * @returns the complete decoded text when it fits. */ -export async function readWholeTextBounded( - target: LocalTarget, - maxBytes: number, - signal?: AbortSignal, -): Promise { - if (!Number.isSafeInteger(maxBytes) || maxBytes <= 0) { - throw new Error('bounded read maxBytes must be a positive safe integer') - } - throwIfAborted(signal, 'read') - let handle: Awaited> - try { - handle = await open( - target.targetKey, - constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK, - ) - } catch (error: unknown) { - if (isENOENT(error)) throw new FsError(`cannot read "${target.displayPath}": not found`, 'FS_NOT_FOUND', { cause: error }) - if (isPermissionError(error)) throw new FsError(`cannot read "${target.displayPath}": permission denied`, 'FS_PERMISSION_DENIED', { cause: error }) - throw new FsError(`cannot read "${target.displayPath}" safely: ${errorMessage(error)}`, 'FS_IO_ERROR', { cause: error }) - } - try { - throwIfAborted(signal, 'read') - const info = await handle.stat() - if (!info.isFile()) { - throw new FsError(`cannot read "${target.displayPath}": not a regular file`, 'FS_NOT_REGULAR_FILE') - } - if (info.size > maxBytes) { - throw new FsError( - `cannot read "${target.displayPath}": ${info.size} bytes exceeds the ${maxBytes}-byte limit`, - 'FS_IO_ERROR', - ) - } - const chunks: Buffer[] = [] - let total = 0 - for (;;) { - throwIfAborted(signal, 'read') - // Allocate in fixed internal chunks so a permissive deployment cap does - // not reserve that entire cap for a small file. Once exactly at the - // bound, one final byte detects concurrent growth without retaining it. - const remaining = total === maxBytes ? 1 : Math.min(64 * 1024, maxBytes - total) - const chunk = Buffer.allocUnsafe(remaining) - const { bytesRead } = await handle.read(chunk, 0, chunk.length, total) - if (bytesRead === 0) break - total += bytesRead - if (total > maxBytes) { - throw new FsError( - `cannot read "${target.displayPath}": file grew past the ${maxBytes}-byte limit while reading`, - 'FS_IO_ERROR', - ) - } - chunks.push(chunk.subarray(0, bytesRead)) - } - throwIfAborted(signal, 'read') - const bytes = chunks.length === 1 ? chunks[0] as Buffer : Buffer.concat(chunks, total) - if (bytes.subarray(0, BINARY_SAMPLE_BYTES).includes(0)) { - throw new FsError(`cannot read "${target.displayPath}": binary file`, 'FS_NOT_TEXT') - } - return decodeUtf8(bytes, 'read', target.displayPath) - } finally { - await handle.close() - } -} - /** * Stream a whole regular UTF-8 text file as decoded text chunks. Same text * semantics as {@link readWholeText} (regular-file check, binary/NUL rejection, diff --git a/packages/fs/fs-local/src/index.ts b/packages/fs/fs-local/src/index.ts index 1a266301db..f6049b1795 100644 --- a/packages/fs/fs-local/src/index.ts +++ b/packages/fs/fs-local/src/index.ts @@ -28,7 +28,6 @@ import { readForEdit, readTextForDiff, readWholeText, - readWholeTextBounded, resolveLocalTarget, restoreLineEndings, streamWholeText, @@ -126,10 +125,6 @@ export class LocalFileSystem extends FileSystem { return readWholeText({ displayPath: target.displayPath, targetKey: target.targetKey }, signal) } - override async readTextBounded(target: FsTarget, maxBytes: number, signal?: AbortSignal): Promise { - return readWholeTextBounded({ displayPath: target.displayPath, targetKey: target.targetKey }, maxBytes, signal) - } - override streamText(target: FsTarget, signal?: AbortSignal): Promise> { return Promise.resolve(streamWholeText({ displayPath: target.displayPath, targetKey: target.targetKey }, signal)) } diff --git a/packages/fs/fs-local/tests/filesystem.spec.ts b/packages/fs/fs-local/tests/filesystem.spec.ts index 02beab7323..4906bb0adc 100644 --- a/packages/fs/fs-local/tests/filesystem.spec.ts +++ b/packages/fs/fs-local/tests/filesystem.spec.ts @@ -218,15 +218,6 @@ describe('readText / streamText', () => { expect(await fs.readText(await fs.resolve('a.txt'))).toBe('one\ntwo\nthree') }) - it('reads complete text through the stable byte bound', async () => { - await writeFile(join(dir, 'bounded.txt'), '€abc') - const target = await fs.resolve('bounded.txt') - expect(await fs.readTextBounded(target, 6)).toBe('€abc') - await expect(fs.readTextBounded(target, 5)).rejects.toThrow('exceeds the 5-byte limit') - await expect(fs.readTextBounded(target, 0)).rejects.toThrow('positive safe integer') - await expect(fs.readTextBounded(target, 6, AbortSignal.abort())).rejects.toMatchObject({ code: 'FS_ABORTED' }) - }) - it('streams the same text', async () => { await writeFile(join(dir, 'a.txt'), 'one\ntwo\nthree') const target = await fs.resolve('a.txt') diff --git a/packages/fs/fs-local/tests/fsio.spec.ts b/packages/fs/fs-local/tests/fsio.spec.ts index 2c5fd7a8f4..15588e40b9 100644 --- a/packages/fs/fs-local/tests/fsio.spec.ts +++ b/packages/fs/fs-local/tests/fsio.spec.ts @@ -5,7 +5,7 @@ * policy and lives in `dsh-fs-policy`, so it is not tested here. */ -import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' +import { afterEach, beforeEach, describe, expect, it } from 'vitest' import { chmod, mkdtemp, readFile, rename, rm, stat, symlink, unlink, writeFile, mkdir, readdir, realpath } from 'node:fs/promises' import { tmpdir } from 'node:os' import { join } from 'node:path' @@ -17,7 +17,6 @@ import { probeNoFollow, readForEdit, readWholeText, - readWholeTextBounded, resolveLocalTarget, restoreLineEndings, streamWholeText, @@ -317,66 +316,6 @@ describe('readWholeText', () => { }) }) -describe('readWholeTextBounded', () => { - it('rejects non-files, binary text, and initial oversize without a full read', async () => { - await expect(readWholeTextBounded(localTarget(dir), 10)).rejects.toMatchObject({ code: 'FS_NOT_REGULAR_FILE' }) - await writeFile(join(dir, 'large'), '12345') - await expect(readWholeTextBounded(localTarget(join(dir, 'large')), 4)).rejects.toThrow('exceeds the 4-byte limit') - await writeFile(join(dir, 'binary'), Buffer.from([0x61, 0x00, 0x62])) - await expect(readWholeTextBounded(localTarget(join(dir, 'binary')), 3)).rejects.toMatchObject({ code: 'FS_NOT_TEXT' }) - }) - - it('detects growth past the bound on the same open handle', async () => { - const close = vi.fn(async () => {}) - const read = vi.fn(async (buffer: Buffer, offset: number, length: number, position: number) => { - const bytes = position === 0 ? Buffer.from('abc') : Buffer.from('d') - bytes.copy(buffer, offset, 0, Math.min(length, bytes.length)) - return { bytesRead: Math.min(length, bytes.length), buffer } - }) - vi.resetModules() - vi.doMock('node:fs/promises', async (importOriginal) => { - const actual = await importOriginal() - return { - ...actual, - open: async () => ({ - stat: async () => ({ isFile: () => true, size: 3 }), - read, - close, - }), - } - }) - try { - const isolated = await import('../src/fsio.ts') - await expect(isolated.readWholeTextBounded(localTarget('/virtual/growing'), 3)) - .rejects.toThrow('grew past the 3-byte limit') - expect(close).toHaveBeenCalledOnce() - } finally { - vi.doUnmock('node:fs/promises') - vi.resetModules() - } - }) - - it('translates permission and generic open failures', async () => { - const failure: { current: Error & { code?: string } } = { current: Object.assign(new Error('denied'), { code: 'EACCES' }) } - vi.resetModules() - vi.doMock('node:fs/promises', async (importOriginal) => { - const actual = await importOriginal() - return { ...actual, open: async () => { throw failure.current } } - }) - try { - const isolated = await import('../src/fsio.ts') - await expect(isolated.readWholeTextBounded(localTarget('/virtual/denied'), 3)) - .rejects.toMatchObject({ code: 'FS_PERMISSION_DENIED' }) - failure.current = new Error('open broke') - await expect(isolated.readWholeTextBounded(localTarget('/virtual/broken'), 3)) - .rejects.toMatchObject({ code: 'FS_IO_ERROR' }) - } finally { - vi.doUnmock('node:fs/promises') - vi.resetModules() - } - }) -}) - describe('streamWholeText', () => { it('streams the whole file as decoded text', async () => { const file = join(dir, 'a.txt') diff --git a/packages/fs/fs/README.i18n.yaml b/packages/fs/fs/README.i18n.yaml index 4a0e675c0c..d9b5f0ebc0 100644 --- a/packages/fs/fs/README.i18n.yaml +++ b/packages/fs/fs/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/fs/fs/README.md -README.md: 67079f795c705ab4c9cfa476e0458be04a48c6c8 -README.zh.md: d812a94fab9f4b7e9d15ff78bd1fea3bcc00c0d9 +README.md: bf1dd1c1eb65146258cd64e450749845522e7057 +README.zh.md: f3fcc0c3794b972233dc418e93bdd80b1cc8570a diff --git a/packages/fs/fs/README.md b/packages/fs/fs/README.md index 67079f795c..bf1dd1c1eb 100644 --- a/packages/fs/fs/README.md +++ b/packages/fs/fs/README.md @@ -2,7 +2,7 @@ English | [中文](README.zh.md) -The **filesystem provider seam**: an abstract `FileSystem` service (`ctx.fs`) defining the storage primitives in one execution world — resolve paths, expose canonical process paths and file URIs, test containment, read bounded or streaming text, inspect/list metadata, write atomically, and apply a literal edit — without saying HOW. Both mutations take their version guard **optionally**, so `ctx.fs` on its own is a complete, unconstrained text-storage seam. This package also owns the `fs/*` policy event vocabulary the tool dispatches and the policy plugin listens for. +The **filesystem provider seam**: an abstract `FileSystem` service (`ctx.fs`) defining the storage primitives in one execution world — resolve paths, expose canonical process paths and file URIs, test containment, read whole or streaming text, inspect/list metadata, write atomically, and apply a literal edit — without saying HOW. Both mutations take their version guard **optionally**, so `ctx.fs` on its own is a complete, unconstrained text-storage seam. This package also owns the `fs/*` policy event vocabulary the tool dispatches and the policy plugin listens for. This package is the provider-seam layer of the four-layer filesystem stack, split so each concern can evolve (and be swapped) independently (see [the capability-seam Agent Note](../../../.agents/notes/implemented/architecture/2026-06-13-capability-seams.md), [the filesystem capability-seam Agent Note](../../../.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.md), [the split-the-filesystem-seam Agent Note](../../../.agents/notes/implemented/simplification/2026-06-26-fsspec-style-fs-seam.md), and [the file-context event-gate Agent Note](../../../.agents/notes/implemented/architecture/2026-06-26-file-context-as-event-gate.md)): @@ -17,7 +17,7 @@ A future sandboxed, virtual, or remote backend implements this interface and the ## Service API (`ctx.fs`) -A backend subclasses `FileSystem` and implements twelve primitives. +A backend subclasses `FileSystem` and implements eleven primitives. | Member | Semantics | |---|---| @@ -28,8 +28,7 @@ A backend subclasses `FileSystem` and implements twelve primitives. | `stat(target, signal?)` | Return `FsInfo` metadata (`version`, `type`, optional `size`), or `undefined` when the target is absent. Never content. | | `lstat(path, opts?, signal?)` | Return `FsPathInfo` metadata without following the final path component when it is a symlink. This is path-shaped so consumers can reject repository-owned symlinks before `resolve` follows them into a target. | | `readText(target, signal?)` | Read the whole regular text file as one decoded string. Owns regular-file checks, UTF-8 decoding, binary/NUL rejection (`FS_NOT_TEXT`). | -| `readTextBounded(target, maxBytes, signal?)` | Read one complete regular UTF-8 file through a backend-owned stable operation, rejecting before retaining more than `maxBytes`. Consumers must not emulate this with `stat` then `readText`, which admits growth and replacement races. | -| `streamText(target, signal?)` | Stream the same text as decoded chunks for large files (cross-chunk UTF-8 decoding stays here). | +| `streamText(target, signal?)` | Stream the same text as decoded chunks for large files (cross-chunk UTF-8 decoding stays here); consumers that need a byte ceiling enforce it while consuming the stream. | | `listDir(target, signal?)` | List direct directory children in stable name order. Returns entry names, entry types, resolved child targets, and cheap metadata (`version`/file `size` when available); never reads file contents. Missing targets throw `FS_NOT_FOUND`, non-directories throw `FS_NOT_DIRECTORY`, permission failures throw `FS_PERMISSION_DENIED`, and other backend I/O failures throw `FS_IO_ERROR`. Broken/disappeared children may be returned as `other` without metadata; child permission/IO failures fail the whole listing with the same structured codes. | | `writeText(target, content, expected?, signal?)` | Atomic create/replace. `expected` is OPTIONAL: omit ⇒ unconditional create-or-overwrite; supply an `FsWriteIntent` (`createIfAbsent`/`replaceIfVersion`) to guard. | | `editText(target, edit, expected?, signal?)` | Literal edit. `expected` is OPTIONAL: omit ⇒ unconditional edit of the current content; supply `{ version }` to guard (verified BEFORE matching). A missing target reports `FS_STALE_VERSION` either way. Applies and writes atomically — one mutation critical section. | @@ -61,6 +60,6 @@ No direct invalidation; the named consumer owns any request-prefix changes. ## Known Limitations and Deferred Work - **Text-only by contract** — backends reject binary/non-UTF-8 content with `FS_NOT_TEXT`; binary-safe operations are a deliberate deferral of [the tool-schemas Agent Note](../../../.agents/notes/implemented/feature/2026-06-17-filesystem-tool-schemas.md). -- **Twelve primitives only** — no delete, rename/move, copy, or watch; `listDir` is single-level, with recursion, globbing, pagination, and search out of scope per [the directory-listing Agent Note](../../../.agents/notes/archived/architecture/2026-07-03-filesystem-directory-listing-seam.md). +- **Eleven primitives only** — no delete, rename/move, copy, or watch; `listDir` is single-level, with recursion, globbing, pagination, and search out of scope per [the directory-listing Agent Note](../../../.agents/notes/archived/architecture/2026-07-03-filesystem-directory-listing-seam.md). - **No IO deadline** — the seam arms no timeout; cancellation is a best-effort optional `AbortSignal` per primitive (the deliberate [fs-family stance](../README.md)). - **Resolve-then-operate costs a remote backend two round-trips per tool call** — folding or caching resolution is left to such a backend. diff --git a/packages/fs/fs/README.zh.md b/packages/fs/fs/README.zh.md index d812a94fab..f3fcc0c379 100644 --- a/packages/fs/fs/README.zh.md +++ b/packages/fs/fs/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -**文件系统提供方 seam**:抽象 `FileSystem` 服务(`ctx.fs`),定义同一个执行世界中的存储原语,包括解析路径、公开规范化进程路径与文件 URI、检查包含关系、有界或流式读取文本、检查/列出元数据、原子写入和应用字面量编辑,但不规定实现方式。两个变更操作都**可选** 接收版本防护,因此 `ctx.fs` 本身就是完整且不受约束的文本存储 seam。本包还拥有由工具分派、政策插件监听的 `fs/*` 政策事件词汇。 +**文件系统提供方 seam**:抽象 `FileSystem` 服务(`ctx.fs`),定义同一个执行世界中的存储原语,包括解析路径、公开规范化进程路径与文件 URI、检查包含关系、完整或流式读取文本、检查/列出元数据、原子写入和应用字面量编辑,但不规定实现方式。两个变更操作都**可选** 接收版本防护,因此 `ctx.fs` 本身就是完整且不受约束的文本存储 seam。本包还拥有由工具分派、政策插件监听的 `fs/*` 政策事件词汇。 本包是四层文件系统栈中的提供方 seam 层;该拆分使每个关注点可以独立演进和替换(见[能力 seam Agent Note](../../../.agents/notes/implemented/architecture/2026-06-13-capability-seams.md)、[文件系统能力 seam Agent Note](../../../.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.md)、[拆分文件系统 seam Agent Note](../../../.agents/notes/implemented/simplification/2026-06-26-fsspec-style-fs-seam.md)和[文件上下文事件门禁 Agent Note](../../../.agents/notes/implemented/architecture/2026-06-26-file-context-as-event-gate.md)): @@ -17,7 +17,7 @@ ## 服务 API(`ctx.fs`) -后端继承 `FileSystem` 并实现十二个原语。 +后端继承 `FileSystem` 并实现十一个原语。 | 成员 | 语义 | |---|---| @@ -28,8 +28,7 @@ | `stat(target, signal?)` | 返回 `FsInfo` 元数据(`version`、`type`、可选 `size`);目标不存在时返回 `undefined`。绝不返回内容。 | | `lstat(path, opts?, signal?)` | 当最后一个路径组件是符号链接时,不跟随该组件,返回 `FsPathInfo` 元数据。该方法采用路径形态,使消费方能在 `resolve` 跟随仓库所有的符号链接进入目标前拒绝它。 | | `readText(target, signal?)` | 把整个普通文本文件读取为一个解码后的字符串。负责普通文件检查、UTF-8 解码和二进制/NUL 拒绝(`FS_NOT_TEXT`)。 | -| `readTextBounded(target, maxBytes, signal?)` | 通过后端自有的稳定操作读取一个完整的普通 UTF-8 文件,在保留超过 `maxBytes` 前拒绝。消费方不得以先 `stat` 再 `readText` 模拟此操作,因为那会容许文件增长与替换竞态。 | -| `streamText(target, signal?)` | 为大文件按解码后的分片流式读取相同文本(跨分片 UTF-8 解码仍由此处负责)。 | +| `streamText(target, signal?)` | 为大文件按解码后的分片流式读取相同文本(跨分片 UTF-8 解码仍由此处负责);需要字节上限的消费方在消费流时执行该上限。 | | `listDir(target, signal?)` | 按稳定名称顺序列出直接子项。返回条目名称、条目类型、解析后的子目标和低成本元数据(若可用则包括 `version`/文件 `size`);绝不读取文件内容。缺失目标抛出 `FS_NOT_FOUND`,非目录抛出 `FS_NOT_DIRECTORY`,权限失败抛出 `FS_PERMISSION_DENIED`,其他后端 I/O 失败抛出 `FS_IO_ERROR`。损坏/消失的子项可以作为无元数据的 `other` 返回;子项权限/I/O 失败会使用相同结构化代码使整个列表失败。 | | `writeText(target, content, expected?, signal?)` | 原子创建/替换。`expected` 是可选的:省略 ⇒ 无条件创建或覆盖;提供 `FsWriteIntent`(`createIfAbsent`/`replaceIfVersion`)⇒ 添加防护。 | | `editText(target, edit, expected?, signal?)` | 字面量编辑。`expected` 是可选的:省略 ⇒ 无条件编辑当前内容;提供 `{ version }` ⇒ 添加防护,并在匹配之前校验。无论哪种情况,目标缺失都报告 `FS_STALE_VERSION`。应用和写入以原子方式完成,使用同一个变更临界区。 | @@ -61,6 +60,6 @@ ## 已知限制与延期工作 - **契约只支持文本**:后端以 `FS_NOT_TEXT` 拒绝二进制/非 UTF-8 内容;二进制安全操作是[工具 schema Agent Note](../../../.agents/notes/implemented/feature/2026-06-17-filesystem-tool-schemas.md)有意延期的工作。 -- **只有十二个原语**:没有删除、重命名/移动、复制或监视;`listDir` 只支持一层,递归、glob、分页和搜索不在范围内,见[目录列出 Agent Note](../../../.agents/notes/archived/architecture/2026-07-03-filesystem-directory-listing-seam.md)。 +- **只有十一个原语**:没有删除、重命名/移动、复制或监视;`listDir` 只支持一层,递归、glob、分页和搜索不在范围内,见[目录列出 Agent Note](../../../.agents/notes/archived/architecture/2026-07-03-filesystem-directory-listing-seam.md)。 - **没有 I/O deadline**:该 seam 不启动超时;取消只是每个原语上尽力而为的可选 `AbortSignal`(见有意采用的 [fs 能力族立场](../README.md))。 - **先解析后操作使远程后端每次工具调用需要两次往返**:折叠或缓存解析由这种后端自行决定。 diff --git a/packages/fs/fs/src/index.ts b/packages/fs/fs/src/index.ts index 8bb20ece18..2d46868a57 100644 --- a/packages/fs/fs/src/index.ts +++ b/packages/fs/fs/src/index.ts @@ -1,7 +1,7 @@ /** * Filesystem provider seam for one execution world. Backends own stable target - * identity, process paths and file URIs, containment, stable bounded text - * reads, decoding, binary rejection, and atomic mutations. Read windows and + * identity, process paths and file URIs, containment, text reads, decoding, + * binary rejection, and atomic mutations. Read windows and * observed-state policy stay in consumer and policy plugins; `editText` * remains here so version check, literal match, and rewrite share one critical * section. @@ -172,19 +172,6 @@ export abstract class FileSystem extends Service { */ abstract readText(target: FsTarget, signal?: AbortSignal): Promise - /** - * Read one regular UTF-8 text file through a backend-owned stable handle, - * rejecting before more than `maxBytes` are retained. The size check and - * bytes read are one operation: a caller must not emulate this with - * {@link stat} followed by {@link readText}, which admits growth and path - * replacement races between the two calls. - * @param target - the resolved target to read. - * @param maxBytes - positive safe-integer byte ceiling. - * @param signal - aborts the open/read operation. - * @returns the complete decoded text when it fits. - */ - abstract readTextBounded(target: FsTarget, maxBytes: number, signal?: AbortSignal): Promise - /** * Stream the whole regular text file as decoded text chunks (same text * semantics as {@link readText}, for large files). The backend owns diff --git a/packages/fs/fs/tests/service.spec.ts b/packages/fs/fs/tests/service.spec.ts index 92f1cfc37f..f225e51cc0 100644 --- a/packages/fs/fs/tests/service.spec.ts +++ b/packages/fs/fs/tests/service.spec.ts @@ -46,11 +46,6 @@ class FakeFileSystem extends FileSystem { if (content === undefined) throw new FsError(`not found: ${target.displayPath}`, 'FS_NOT_FOUND') return content } - override async readTextBounded(target: FsTarget, maxBytes: number): Promise { - const content = await this.readText(target) - if (Buffer.byteLength(content) > maxBytes) throw new FsError('too large', 'FS_IO_ERROR') - return content - } override async streamText(target: FsTarget): Promise> { const content = await this.readText(target) return (async function* () { yield content })() diff --git a/packages/fs/tool-fs/tests/tools.spec.ts b/packages/fs/tool-fs/tests/tools.spec.ts index f54701246a..b217260972 100644 --- a/packages/fs/tool-fs/tests/tools.spec.ts +++ b/packages/fs/tool-fs/tests/tools.spec.ts @@ -67,11 +67,6 @@ class FakeFs extends FileSystem { override async readText(target: FsTarget): Promise { return this.files.get(target.targetKey) ?? '' } - override async readTextBounded(target: FsTarget, maxBytes: number): Promise { - const content = await this.readText(target) - if (Buffer.byteLength(content) > maxBytes) throw new Error('too large') - return content - } override async streamText(target: FsTarget): Promise> { const content = this.files.get(target.targetKey) ?? '' return (async function* () { yield content })() diff --git a/packages/lsp/lsp-local/README.i18n.yaml b/packages/lsp/lsp-local/README.i18n.yaml index 5a1036a349..55a9f97165 100644 --- a/packages/lsp/lsp-local/README.i18n.yaml +++ b/packages/lsp/lsp-local/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/lsp/lsp-local/README.md -README.md: c96c4febc9d047b41789f2b7a73e3eaf4d35012b -README.zh.md: 5bc2c3c8bb7a89afb673797f5a3b25bb9fc06748 +README.md: ad8f4bc2318a58202f9596a18d402d2c6d45dae1 +README.zh.md: ff2686856a9e518a7fa846edf700da1b32e8a114 diff --git a/packages/lsp/lsp-local/README.md b/packages/lsp/lsp-local/README.md index c96c4febc9..ad8f4bc231 100644 --- a/packages/lsp/lsp-local/README.md +++ b/packages/lsp/lsp-local/README.md @@ -10,11 +10,11 @@ Namespace plugin (`name` / `inject` / `Config` / `apply`, no default export). - Resolves every server-local setting before registration; an invalid mapping or registration conflict rolls back earlier entries, so a failed load leaves no provider routes. - Lazily single-flights one server process per `(server id, canonical workspace target)`. A live server error is not replayed; if the selected pooled transport fails before or during a read-only query, the provider awaits its disposal and retries that query once on a fresh process. -- Uses a compatibility-first **transient-open** sequence per query: resolve and boundedly read the source through `ctx.fs`, `textDocument/didOpen` (version 1, full text), the requested request, then `textDocument/didClose` in `finally`. A failed or canceled `didOpen` write terminates the instance before the pool can reuse it. Documents close after each call, so the first version needs no `didChange`, content cache, or document LRU. +- Uses a compatibility-first **transient-open** sequence per query: resolve and byte-bound the source while streaming it through `ctx.fs`, `textDocument/didOpen` (version 1, full text), the requested request, then `textDocument/didClose` in `finally`. A failed or canceled `didOpen` write terminates the instance before the pool can reuse it. Documents close after each call, so the first version needs no `didChange`, content cache, or document LRU. - Serializes each source-read/open/query/close lifecycle through one abortable per-workspace queue so queued calls read current source only when their turn starts; distinct workspaces run in parallel. Provider disposal aborts filesystem and protocol work, awaits workspace lookups that have not entered a queue, then drains every queue and server. - After protocol shutdown fails, terminates the server's descendant tree through the subprocess seam (POSIX process-group signaling; Windows `taskkill /T /F`). Tree-kill delivery is contained like every group signal — it races server exit — and quiescence is confirmed by the handle's tree-liveness wait rather than by the kill's own outcome. - Resolves the server executable, cwd, process, and protocol streams through `ctx.subprocess`; `initialize.processId` is `null` because another machine or PID namespace must not monitor the harness process. -- Uses `ctx.fs` canonical containment, file URIs, and stable bounded reads, but emits no `fs/observed`: only the LSP result is model-visible, so a query does not satisfy read-before-write policy. +- Uses `ctx.fs` canonical containment, file URIs, and streamed text validation, but emits no `fs/observed`: only the LSP result is model-visible, so a query does not satisfy read-before-write policy. ## Configuration @@ -42,7 +42,7 @@ Initialization advertises `general.positionEncodings: ['utf-16']`, `workspace: { ## Security boundary -The provider trusts its configured server and claims no sandbox confinement. It delegates canonical identity, containment, no-follow/stable bounded reads, UTF-8 validation, and file-URI encoding to `ctx.fs`; it rejects missing, non-regular, non-UTF-8, oversized, or canonically out-of-workspace query sources before server startup. Result locations may be external, but an external path cannot become a query source. A deployment must mount filesystem and subprocess providers for the same execution world; split-world composition is invalid. +The provider trusts its configured server and claims no sandbox confinement. It delegates canonical identity, containment, regular-file streaming, UTF-8 validation, and file-URI encoding to `ctx.fs`; it rejects missing, non-regular, non-UTF-8, oversized, or canonically out-of-workspace query sources before server startup. Containment is evaluated before the stream opens and does not promise stable-handle identity across concurrent path replacement. Result locations may be external, but an external path cannot become a query source. A deployment must mount filesystem and subprocess providers for the same execution world; split-world composition is invalid. ## Model Experience diff --git a/packages/lsp/lsp-local/README.zh.md b/packages/lsp/lsp-local/README.zh.md index 5bc2c3c8bb..ff2686856a 100644 --- a/packages/lsp/lsp-local/README.zh.md +++ b/packages/lsp/lsp-local/README.zh.md @@ -10,11 +10,11 @@ Namespace 插件(`name`/`inject`/`Config`/`apply`,无默认导出) - 在注册前解析每项服务器局部设置;无效映射或注册冲突会回滚较早配置项,因此加载失败不会留下提供方路由。 - 每个 `(server id, canonical workspace target)` 惰性 single-flight 一个服务器进程。存活服务器错误不会回放;如果选中的池化传输在只读查询之前或期间失败,提供方会等待其释放,并在新进程上重试该查询一次。 -- 每次查询都使用兼容性优先的**临时打开** 序列:通过 `ctx.fs` 解析源文件并进行有界读取、`textDocument/didOpen`(版本 1、完整文本)、所请求操作,然后执行 `textDocument/didClose`,该操作位于 `finally` 中。写入 `didOpen` 失败或取消时,会先终止实例再允许池复用。文档在每次调用后关闭,因此第一版不需要 `didChange`、内容 cache 或文档 LRU。 +- 每次查询都使用兼容性优先的**临时打开** 序列:通过 `ctx.fs` 解析并流式读取源文件,同时执行字节上限;随后执行 `textDocument/didOpen`(版本 1、完整文本)、所请求操作,以及位于 `finally` 中的 `textDocument/didClose`。写入 `didOpen` 失败或取消时,会先终止实例再允许池复用。文档在每次调用后关闭,因此第一版不需要 `didChange`、内容 cache 或文档 LRU。 - 通过一条逐 Workspace、可中止的队列,串行执行每个源读取/打开/查询/关闭生命周期,因此排队调用只会在轮到自身时读取当前源;不同 Workspace 并行运行。提供方资源释放会中止文件系统与协议工作,等待尚未进入队列的 Workspace 查找结算,再排空所有队列并等待所有服务器结算。 - 协议 shutdown 失败后,经由进程管理器 seam 终止服务器后代树(POSIX 进程组信号;Windows `taskkill /T /F`)。树终止的投递结果与所有进程组信号一样被就地吸收,不向外抛出(投递与服务器退出存在竞态);服务器是否完全停稳,由句柄的进程树存活等待确认,而非由这次终止自身的结果确认。 - 通过 `ctx.subprocess` 解析服务器可执行文件、cwd、进程与协议流;`initialize.processId` 为 `null`,因为另一台机器或 PID 命名空间不得监控 harness 进程。 -- 使用 `ctx.fs` 提供的规范 containment、文件 URI 与稳定有界读取,但不发出 `fs/observed`:只有 LSP 结果对模型可见,因此查询不满足先读后写策略。 +- 使用 `ctx.fs` 提供的规范 containment、文件 URI 与流式文本校验,但不发出 `fs/observed`:只有 LSP 结果对模型可见,因此查询不满足先读后写策略。 ## 配置 @@ -42,7 +42,7 @@ Namespace 插件(`name`/`inject`/`Config`/`apply`,无默认导出) ## 安全边界 -提供方信任其配置的服务器,不声明任何沙箱限制。它把规范身份、containment、不跟随符号链接的稳定有界读取、UTF-8 校验与文件 URI 编码委托给 `ctx.fs`;服务器启动前,系统会拒绝缺失、非普通文件、非 UTF-8、过大或规范路径位于工作区外的查询源。结果位置可以在外部,但外部路径不能成为查询源。部署必须为同一执行环境挂载文件系统与子进程提供方;分裂执行环境的组合无效。 +提供方信任其配置的服务器,不声明任何沙箱限制。它把规范身份、containment、普通文件流式读取、UTF-8 校验与文件 URI 编码委托给 `ctx.fs`;服务器启动前,系统会拒绝缺失、非普通文件、非 UTF-8、过大或规范路径位于工作区外的查询源。系统在打开流之前检查 containment,但不保证路径并发替换期间的稳定句柄身份。结果位置可以在外部,但外部路径不能成为查询源。部署必须为同一执行环境挂载文件系统与子进程提供方;分裂执行环境的组合无效。 ## 模型体验 diff --git a/packages/lsp/lsp-local/src/host.ts b/packages/lsp/lsp-local/src/host.ts index 55e13ed307..b8939a2ae4 100644 --- a/packages/lsp/lsp-local/src/host.ts +++ b/packages/lsp/lsp-local/src/host.ts @@ -1,5 +1,6 @@ /** Filesystem-seam source access for the generic stdio LSP provider. */ +import { Buffer } from 'node:buffer' import type { FileSystem, FsTarget } from '@deepseek-ai/dsh-fs' import { throwIfAborted } from './abort.ts' @@ -58,9 +59,9 @@ export async function canonicalizeWorkspace( } /** - * Resolve, contain, and atomically read one bounded query source through - * `ctx.fs`. The provider's bounded read owns stable-handle and no-follow - * mechanics; this layer owns only LSP-facing validation and messages. + * Resolve, contain, and read one byte-bounded query source through `ctx.fs`. + * This layer owns the LSP-specific complete-document cap while the filesystem + * provider owns streaming, regular-file checks, and UTF-8 validation. * @param fs - filesystem provider sharing the server's execution world. * @param filePath - absolute source path or path relative to `workspace`. * @param workspace - already-canonical workspace. @@ -90,17 +91,28 @@ export async function readHostSource( if (!fs.contains(workspace.target, target)) { throw new Error(`source "${filePath}" resolves outside the workspace`) } - let text: string + const chunks: string[] = [] + let bytes = 0 try { - text = await fs.readTextBounded(target, maxDocumentBytes, signal) + // XXX(lsp-source-replacement): Revisit stable-handle identity only if a real query observes + // replacement between canonical containment and the provider opening this stream. + const stream = await fs.streamText(target, signal) + for await (const chunk of stream) { + throwIfAborted(signal) + bytes += Buffer.byteLength(chunk) + if (bytes > maxDocumentBytes) { + throw new Error(`source "${filePath}" exceeds the ${maxDocumentBytes}-byte limit`) + } + chunks.push(chunk) + } } catch (error: unknown) { throwIfAborted(signal) - throw new Error(`source "${filePath}" could not be opened safely: ${messageOf(error)}`, { cause: error }) + throw new Error(`source "${filePath}" could not be read: ${messageOf(error)}`, { cause: error }) } throwIfAborted(signal) return { fileUrl: fs.fileUrl(target), - text, + text: chunks.join(''), } } diff --git a/packages/lsp/lsp-local/tests/host.spec.ts b/packages/lsp/lsp-local/tests/host.spec.ts index 3db9deb50a..eea3851414 100644 --- a/packages/lsp/lsp-local/tests/host.spec.ts +++ b/packages/lsp/lsp-local/tests/host.spec.ts @@ -161,6 +161,12 @@ describe('readHostSource', () => { await expect(readSource('big.ts', 10)).rejects.toThrow(/10-byte limit/) }) + it('counts the complete UTF-8 byte length at the configured boundary', async () => { + await writeFile(join(ws, 'multibyte.ts'), '€abc') + await expect(readSource('multibyte.ts', 6)).resolves.toMatchObject({ text: '€abc' }) + await expect(readSource('multibyte.ts', 5)).rejects.toThrow(/5-byte limit/) + }) + it('rejects a non-UTF-8 source', async () => { await writeFile(join(ws, 'bin.ts'), Buffer.from([0xff, 0xfe, 0x00])) await expect(readSource('bin.ts')).rejects.toThrow(/invalid UTF-8|binary file/) diff --git a/packages/lsp/lsp-local/tests/lifecycle.spec.ts b/packages/lsp/lsp-local/tests/lifecycle.spec.ts index 0337b0fec2..63aea8742c 100644 --- a/packages/lsp/lsp-local/tests/lifecycle.spec.ts +++ b/packages/lsp/lsp-local/tests/lifecycle.spec.ts @@ -328,14 +328,17 @@ describe('lsp-local end to end over a fake server', () => { await expect(disposing).resolves.toBeUndefined() }) - it('aborts a queued source read when the provider is disposed', async () => { + it('aborts a queued source stream when the provider is disposed', async () => { const ctx = await mount({ LSP_FAKE_DEF: 'null' }) const fs = ctx.fs const started = Promise.withResolvers() - vi.spyOn(fs, 'readTextBounded').mockImplementation(async (_target, _maxBytes, signal) => { + vi.spyOn(fs, 'streamText').mockImplementation(async (_target, signal) => { if (signal === undefined) throw new Error('source read missing provider lifetime signal') started.resolve(signal) - return await rejectWhenAborted(signal) + return (async function* () { + await rejectWhenAborted(signal) + yield '' + })() }) const pending = ctx.lsp.query(query('goToDefinition')) diff --git a/packages/lsp/tool-lsp/src/render.ts b/packages/lsp/tool-lsp/src/render.ts index 2f376707db..761f5a7d34 100644 --- a/packages/lsp/tool-lsp/src/render.ts +++ b/packages/lsp/tool-lsp/src/render.ts @@ -8,6 +8,8 @@ import type { GenericCallView } from '@deepseek-ai/dsh-tools' import type { LspHover, LspLocation, LspOperation, LspPosition } from '@deepseek-ai/dsh-lsp' +import { posix, win32 } from 'node:path' +import { fileURLToPath } from 'node:url' /** The four operations the tool exposes, as a runtime tuple for schema enum + validation. */ export const LSP_OPERATIONS: readonly LspOperation[] = ['goToDefinition', 'findReferences', 'goToImplementation', 'hover'] @@ -144,52 +146,31 @@ export function renderUri(uri: string, workspaceUri: string): string { return uri } if (workspace.protocol !== 'file:') return uri - const targetSegments = decodeFileSegments(target) - const workspaceSegments = decodeFileSegments(workspace) - if (targetSegments === undefined || workspaceSegments === undefined) return uri - const sameAuthority = target.hostname === workspace.hostname - const windowsWorld = isWindowsFileWorld(workspace, workspaceSegments) - if (windowsWorld && [...targetSegments, ...workspaceSegments].some(segment => segment.includes('\\'))) return uri - const inside = sameAuthority - && targetSegments.length >= workspaceSegments.length - && workspaceSegments.every((segment, index) => samePathSegment(segment, targetSegments[index] as string, windowsWorld)) - if (inside) { - const relative = targetSegments.slice(workspaceSegments.length) - return relative.length === 0 ? '.' : relative.join('/') - } - return absoluteUriPath(target, targetSegments, windowsWorld) + const drivePath = /^\/[a-z](?::|%3A)/iu + const windowsWorld = workspace.hostname.length > 0 || drivePath.test(workspace.pathname) + const targetWindowsWorld = windowsWorld && (target.hostname.length > 0 || drivePath.test(target.pathname)) + const workspacePath = filePath(workspace, windowsWorld) + const targetPath = filePath(target, targetWindowsWorld) + if (workspacePath === undefined || targetPath === undefined) return uri + if (windowsWorld !== targetWindowsWorld) return targetPath + const path = windowsWorld ? win32 : posix + const relative = path.relative(workspacePath, targetPath) + const outside = relative === '..' || relative.startsWith(`..${path.sep}`) || path.isAbsolute(relative) + const rendered = relative === '' ? '.' : outside ? targetPath : relative + return windowsWorld ? rendered.replaceAll('\\', '/') : rendered } -/** Whether a canonical file URI names a drive path or UNC path in a Windows execution world. */ -function isWindowsFileWorld(url: URL, segments: readonly string[]): boolean { - return url.hostname.length > 0 || /^[A-Za-z]:$/.test(segments[0] ?? '') -} - -/** Decode URI path segments while rejecting encoded POSIX separators and NUL. */ -function decodeFileSegments(url: URL): string[] | undefined { +/** Decode a file URL for its execution world while containing malformed URL failures. */ +function filePath(url: URL, windows: boolean): string | undefined { try { - const decoded = url.pathname.split('/').map(segment => decodeURIComponent(segment)) - if (decoded.some(segment => /[/\0]/u.test(segment))) return undefined - while (decoded.at(-1) === '') decoded.pop() - decoded.shift() - return decoded + const path = fileURLToPath(url, { windows }) + return path.includes('\0') ? undefined : path } catch { + // `fileURLToPath` rejects malformed escapes, authorities, and encoded path separators. return undefined } } -/** Windows execution-world path segments are case-insensitive even on a non-Windows harness host. */ -function samePathSegment(left: string, right: string, windowsWorld: boolean): boolean { - return windowsWorld ? left.toUpperCase() === right.toUpperCase() : left === right -} - -/** Render an external file URL according to the execution-world style implied by its workspace URI. */ -function absoluteUriPath(target: URL, segments: readonly string[], windowsWorld: boolean): string { - if (target.hostname.length > 0) return `//${target.hostname}/${segments.join('/')}` - if (windowsWorld && /^[A-Za-z]:$/.test(segments[0] ?? '')) return segments.join('/') - return `/${segments.join('/')}` -} - /** * UI presentation for a pending `lsp` call. Uses a generic search card; the title carries the * operation and one-based cursor, and `locations` focuses the queried line. The shared location diff --git a/packages/lsp/tool-lsp/tests/render.spec.ts b/packages/lsp/tool-lsp/tests/render.spec.ts index f09297d3d4..2d181f25b3 100644 --- a/packages/lsp/tool-lsp/tests/render.spec.ts +++ b/packages/lsp/tool-lsp/tests/render.spec.ts @@ -103,6 +103,7 @@ describe('renderUri', () => { it('keeps a malformed file: URI verbatim when it cannot be parsed to a path', () => { // An encoded path separator is invalid on every platform and must remain verbatim. expect(renderUri('file:///bad%2Fpath', WS_URI)).toBe('file:///bad%2Fpath') + expect(renderUri('file:///bad%00path', WS_URI)).toBe('file:///bad%00path') }) }) diff --git a/packages/pty/pty-local/README.i18n.yaml b/packages/pty/pty-local/README.i18n.yaml index 2bd653983e..c2a0f9b647 100644 --- a/packages/pty/pty-local/README.i18n.yaml +++ b/packages/pty/pty-local/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/pty/pty-local/README.md -README.md: bccef05ca73ec8d826621f814f232206fa9fea4b -README.zh.md: 27d4d1193047eb1bb3bf2289697be3c0cc0253aa +README.md: c4af7bc8293689d64c58eebab2606d4f9b52fd2f +README.zh.md: 21075ccbf53d173a54220b58b384f01cfe50ced1 diff --git a/packages/pty/pty-local/README.md b/packages/pty/pty-local/README.md index bccef05ca7..c4af7bc829 100644 --- a/packages/pty/pty-local/README.md +++ b/packages/pty/pty-local/README.md @@ -8,9 +8,9 @@ Persistent shell backend for `ctx.pty` over `ctx.subprocess.spawnTerminal`. It s The plugin injects `pty`, `sandbox`, `sandboxPolicy`, and `subprocess`, then registers the configured backend type (`shell`). `danger-full-access` starts the shell directly; confined modes wrap the exact shell argv through `ctx.sandbox`. The effective session mode is resolved at spawn. A change to a different effective mode is rejected before its `sandbox/mode` event commits while that owner has an open PTY or a spawn in progress; the fence is attached to the exact owner and therefore outlives a provider reload that retains existing sessions. Wait for creation to settle and close the sessions before changing modes, so a terminal opened with wider access cannot survive a downgrade. -Readiness combines a foreground-verified private bash prompt marker, provider-reported foreground stdin-wait facts, silence fallback, and absolute timeout. A marker is not ready until the printable tail after the latest owned marker exactly equals the controlled `PS1`, including when the OSC marker and prompt are split across data callbacks; echoed input or output following a delayed earlier prompt therefore cannot settle the current send. After a signaled send returns `inferred_idle` without a prompt marker, the next marker remains attributed to that prior send and cannot settle a successor even when it follows the successor's echoed input. Prompt and silence evidence collected before the provider write, including while pre-write foreground inspection is pending, is discarded at the write boundary. When bash prints the marker before the terminal provider publishes its return to the foreground process group, polling retains the candidate for `handoffGraceMs` past the ordinary silence bound so a coincident handoff can win. An interactive child that inherits `PROMPT_COMMAND` therefore cannot suppress inferred-idle readiness until the absolute timeout. Unknown foreground state is never a positive exact-idle signal. A foreground group's stdin wait that existed before a send is likewise not post-write readiness: the same group must be observed outside that wait before a later wait can settle the send, while a changed foreground group is new evidence. During unpublished startup, a fallback requires observed output; zero-output silence cannot publish an empty session, and timeout rejects the spawn. Cancellation closes the unpublished shell and rejects with the caller's exact abort reason; `PtyBackendCleanupError` separately preserves a cleanup failure. The terminal-allocation signal is detached when allocation returns, while readiness initialization keeps the setup signal, so later cancellation cannot terminate a published persistent session. Incomplete terminal-control sequences are bounded by `maxReadBytes` and discarded through their terminator after crossing that limit; a trailing carriage return is carried across callbacks so split CRLF becomes one newline. +Readiness combines a foreground-verified private bash prompt marker, provider-reported foreground stdin-wait facts, silence fallback, and absolute timeout. A marker is not ready until the printable tail after the latest owned marker exactly equals the controlled `PS1`, including when the OSC marker and prompt are split across data callbacks; echoed input or output following an earlier prompt therefore cannot settle the current send. Prompt and silence evidence collected before the provider write, including while pre-write foreground inspection is pending, is discarded at the write boundary. When bash prints the marker before the terminal provider publishes its return to the foreground process group, polling retains the candidate for `handoffGraceMs` past the ordinary silence bound so a coincident handoff can win. An interactive child that inherits `PROMPT_COMMAND` therefore cannot suppress inferred-idle readiness until the absolute timeout. Unknown foreground state is never a positive exact-idle signal. A foreground group's stdin wait that existed before a send is likewise not post-write readiness: the same group must be observed outside that wait before a later wait can settle the send, while a changed foreground group is new evidence. During unpublished startup, a fallback requires observed output; zero-output silence cannot publish an empty session, and timeout rejects the spawn. Cancellation closes the unpublished shell and rejects with the caller's exact abort reason; `PtyBackendCleanupError` separately preserves a cleanup failure. The caller's signal is forwarded for terminal allocation and readiness initialization; after publication the handle owns its lifetime. Incomplete terminal-control sequences are bounded by `maxReadBytes` and discarded through their terminator after crossing that limit; malformed UTF-8 terminal output uses replacement characters, and a trailing carriage return is carried across callbacks so split CRLF becomes one newline. -Send cancellation marks queued input as canceled before asking the terminal handle to signal the current foreground process group with a real `SIGINT`; if asynchronous pre-write inspection later settles, it cannot execute that input. The canceled send retains its slot until foreground signalling settles, so a successor cannot become that signal's target. Cancellation never emulates interruption by writing `\x03`, so raw-mode programs remain cancellable. A send that times out during an asynchronous provider write, or whose cancellation signal fails while that write remains pending, reports its result but retains the slot until the write settles, so late bytes cannot interleave with a successor. Close rejects new public signals after shutdown begins, drains public signals already in flight, starts provider-owned TERM-to-KILL whole-session cleanup, and awaits quiescence after the terminal outcome. A cleanup failure does not cache a permanently rejected close; a later close retries the provider operation. +Send cancellation marks queued input as canceled before asking the terminal handle to signal the current foreground process group with a real `SIGINT`; if asynchronous pre-write inspection later settles, it cannot execute that input. If a provider write is already in flight, signalling waits for it to settle; a rejected write sends no signal. The canceled send retains its slot until the write and foreground signalling settle, so a successor cannot receive either late bytes or that signal. The absolute deadline remains armed while cancellation waits. A signal failure is a terminal transport failure and rejects the active send. Cancellation never emulates interruption by writing `\x03`, so raw-mode programs remain cancellable. Close rejects new public signals, stops readiness polling, and awaits the handle's provider-owned complete-session termination before settling the active send as `session_exit`. ## Model Experience diff --git a/packages/pty/pty-local/README.zh.md b/packages/pty/pty-local/README.zh.md index 27d4d11930..21075ccbf5 100644 --- a/packages/pty/pty-local/README.zh.md +++ b/packages/pty/pty-local/README.zh.md @@ -8,9 +8,9 @@ 该插件注入 `pty`、`sandbox`、`sandboxPolicy` 和 `subprocess`,然后注册所配置的后端类型(`shell`)。`danger-full-access` 会直接启动 shell;受限模式则通过 `ctx.sandbox` 包装确切的 shell argv。系统在 spawn 时解析会话的实际模式。当某个所有者存在开放的 PTY 或正在进行 spawn 时,如果配置变更会得到不同的实际模式,系统会在对应 `sandbox/mode` 事件提交前拒绝该变更。该限制绑定到确切所有者,因此即使提供方重新加载并保留现有会话,它仍然有效。更改模式前,请等待创建结算并关闭会话,避免以更宽权限打开的终端在权限降级后继续存在。 -就绪检测结合以下机制:由前台状态验证的私有 bash 提示符标记、提供方报告的前台 stdin 等待事实、静默回退和绝对超时。只有最近一个自有标记之后的可打印尾部与受控 `PS1` 完全相等时,系统才会把标记视为就绪;即使 OSC 标记和提示符被拆到多个数据回调中也是如此。因此,如果回显的输入或输出跟在延迟到达的先前提示符之后,该提示符无法使当前 send 完成。在一次经过信号处理的 send 未出现提示符标记却返回 `inferred_idle` 后,下一个标记仍归属于该先前 send;即使该标记出现在后续 send 的输入回显之后,也不能使后续 send 完成。系统会在写入边界丢弃提供方写入前收集的提示符与静默证据,包括写入前的前台检查尚未完成时收集的证据。如果 bash 在终端提供方发布其重新取得前台进程组的状态前打印标记,轮询会在普通静默上限之后再保留该候选状态 `handoffGraceMs`,使恰好同时发生的前台交接有机会胜出。因此,继承 `PROMPT_COMMAND` 的交互式子进程无法持续压制推断空闲就绪,最多只能延续到绝对超时。未知的前台状态绝不会作为精确空闲的正向信号。同样,一次 send 之前就已存在的前台进程组 stdin 等待并不代表写入后就绪:必须先观察到同一进程组脱离该等待,之后再次进入等待才能使该次 send 完成;前台进程组发生变化则构成新的证据。尚未发布的启动过程中,回退路径要求已经观察到输出;零输出静默不能发布空会话,超时则拒绝 spawn。取消操作会关闭尚未发布的 shell,并以调用方提供的确切中止原因拒绝;`PtyBackendCleanupError` 会单独保留清理失败。终端分配返回时,分配信号会解除关联;就绪初始化则保留设置阶段信号,因此后续取消无法终止已发布的持久会话。未完成的终端控制序列受 `maxReadBytes` 限制;超过上限后,系统会丢弃内容直到其终止符。末尾的回车会跨回调保留,使拆分的 CRLF 合并为一个换行。 +就绪检测结合以下机制:由前台状态验证的私有 bash 提示符标记、提供方报告的前台 stdin 等待事实、静默回退和绝对超时。只有最近一个自有标记之后的可打印尾部与受控 `PS1` 完全相等时,系统才会把标记视为就绪;即使 OSC 标记和提示符被拆到多个数据回调中也是如此。因此,如果回显的输入或输出跟在先前提示符之后,该提示符无法使当前 send 完成。系统会在写入边界丢弃提供方写入前收集的提示符与静默证据,包括写入前的前台检查尚未完成时收集的证据。如果 bash 在终端提供方发布其重新取得前台进程组的状态前打印标记,轮询会在普通静默上限之后再保留该候选状态 `handoffGraceMs`,使恰好同时发生的前台交接有机会胜出。因此,继承 `PROMPT_COMMAND` 的交互式子进程无法持续压制推断空闲就绪,最多只能延续到绝对超时。未知的前台状态绝不会作为精确空闲的正向信号。同样,一次 send 之前就已存在的前台进程组 stdin 等待并不代表写入后就绪:必须先观察到同一进程组脱离该等待,之后再次进入等待才能使该次 send 完成;前台进程组发生变化则构成新的证据。尚未发布的启动过程中,回退路径要求已经观察到输出;零输出静默不能发布空会话,超时则拒绝 spawn。取消操作会关闭尚未发布的 shell,并以调用方提供的确切中止原因拒绝;`PtyBackendCleanupError` 会单独保留清理失败。调用方信号会转发给终端分配和就绪初始化;句柄一经发布,便负责自身生命周期。未完成的终端控制序列受 `maxReadBytes` 限制;超过上限后,系统会丢弃内容直到其终止符。格式错误的 UTF-8 终端输出使用替换字符;末尾的回车会跨回调保留,使拆分的 CRLF 合并为一个换行。 -取消发送会先把排队输入标记为已取消,再请求终端句柄向当前前台进程组发送真正的 `SIGINT`;如果异步的写入前检查随后才结算,也无法执行该输入。被取消的发送会保留其槽位,直至前台信号发送结算,因此后续发送不会成为该信号的目标。取消绝不会通过写入 `\x03` 模拟中断,因此原始模式程序仍可取消。发送在提供方异步写入期间超时,或在该写入仍未完成时其取消信号发送失败,都会报告各自结果,但继续占用该槽位,直至写入结算,从而避免延迟到达的字节与后续发送交错。关闭操作会在关闭开始后拒绝新的公开信号请求,等待已经在途的公开信号请求全部结算,再启动由提供方负责的 TERM→KILL 全会话清理,并在终端结果之后等待完全停稳。清理失败不会缓存成永久拒绝的关闭操作;后续关闭会重试提供方操作。 +取消发送会先把排队输入标记为已取消,再请求终端句柄向当前前台进程组发送真正的 `SIGINT`;如果异步的写入前检查随后才结算,也无法执行该输入。如果提供方写入已在途,信号发送会等待写入结算;写入被拒绝时不会发送信号。被取消的发送会保留其槽位,直至写入和前台信号发送都结算,因此后续发送既不会收到延迟字节,也不会成为该信号的目标。取消等待期间,绝对截止时间仍保持启用。信号发送失败属于终结性传输失败,并会使当前发送被拒绝。取消绝不会通过写入 `\x03` 模拟中断,因此原始模式程序仍可取消。关闭操作会拒绝新的公开信号、停止就绪轮询,并等待句柄执行由提供方负责的完整会话终止,之后才将当前发送以 `session_exit` 结算。 ## 模型体验 diff --git a/packages/pty/pty-local/src/index.ts b/packages/pty/pty-local/src/index.ts index 62b5fdb71a..a8d99cb243 100644 --- a/packages/pty/pty-local/src/index.ts +++ b/packages/pty/pty-local/src/index.ts @@ -101,30 +101,15 @@ export class LocalPtyBackend implements PtyBackend { ensureSandboxModeFence(this.ctx, spec.owner) const argv = spawnArgv(this.ctx, this.config, spec) if (argv[0] === undefined) throw new Error('pty-local: sandbox returned empty argv') - let terminalSignal: AbortSignal | undefined - let detachSetupSignal: (() => void) | undefined - if (spec.signal !== undefined) { - const source = spec.signal - const controller = new AbortController() - const onAbort = (): void => { controller.abort(source.reason) } - source.addEventListener('abort', onAbort, { once: true }) - detachSetupSignal = () => { source.removeEventListener('abort', onAbort) } - terminalSignal = controller.signal - } - let terminal: SubprocessTerminalHandle - try { - terminal = await this.spawnTerminal({ - argv, - cwd: spec.cwd ?? this.ctx.sandboxPolicy.workspaceRoot, - env: childEnvironment(spec), - rows: this.config.rows, - cols: this.config.cols, - graceMs: this.config.disposeGraceMs, - signal: terminalSignal, - }) - } finally { - detachSetupSignal?.() - } + const terminal = await this.spawnTerminal({ + argv, + cwd: spec.cwd ?? this.ctx.sandboxPolicy.workspaceRoot, + env: childEnvironment(spec), + rows: this.config.rows, + cols: this.config.cols, + graceMs: this.config.disposeGraceMs, + signal: spec.signal, + }) const session = this.createSession(terminal, this.config) try { await session.initialize(spec.signal) diff --git a/packages/pty/pty-local/src/session.ts b/packages/pty/pty-local/src/session.ts index 08b42466aa..076f21e3b9 100644 --- a/packages/pty/pty-local/src/session.ts +++ b/packages/pty/pty-local/src/session.ts @@ -155,7 +155,7 @@ class LocalSendOperation implements PtySendOperation { export class LocalPtySession implements PtyBackendSession { motd = '' readonly pid: number - private readonly decoder = new TextDecoder('utf-8', { fatal: true }) + private readonly decoder = new TextDecoder() private readonly sanitizer: TerminalSanitizer private readonly scrollback: BoundedTextBuffer private readonly outputEnded = Promise.withResolvers() @@ -165,16 +165,13 @@ export class LocalPtySession implements PtyBackendSession { private activeTimer: NodeJS.Timeout | undefined private activeDeadlineTimer: NodeJS.Timeout | undefined private activeAbort: (() => void) | undefined - private readonly terminalOperations = new Set>() - private signaledOperation: LocalSendOperation | undefined private interrupting: LocalSendOperation | undefined - private writing: LocalSendOperation | undefined + private activeWrite: { operation: LocalSendOperation; settled: Promise } | undefined private pollingReady: LocalSendOperation | undefined private polling = false private promptSeen = false private promptTextSeen = false private promptTail = '' - private delayedSignaledPrompt = false private shellPgid: number | undefined private initializing = false private lastOutputAt = Date.now() @@ -239,23 +236,14 @@ export class LocalPtySession implements PtyBackendSession { this.activeAbort = () => request.signal?.removeEventListener('abort', onAbort) } this.activeDeadlineTimer = setTimeout(() => { - if (this.active === operation) this.settleActive('timeout', this.writing === operation) + if (this.active === operation) { + this.settleActive('timeout', this.activeWrite?.operation === operation) + } }, this.config.timeoutMs) - this.ownTerminalOperation(this.beginSend(operation, request)) + void this.beginSend(operation, request) return operation } - /** Retain one contained provider operation until its asynchronous work finishes. */ - private ownTerminalOperation(operation: Promise): void { - void this.trackTerminalOperation(operation) - } - - private trackTerminalOperation(operation: Promise): Promise { - const tracked = operation.finally(() => { this.terminalOperations.delete(tracked) }) - this.terminalOperations.add(tracked) - return tracked - } - private async beginSend(operation: LocalSendOperation, request: PtySendRequest): Promise { try { const foreground = await this.terminal.inspectForeground() @@ -264,13 +252,20 @@ export class LocalPtySession implements PtyBackendSession { const input = `${request.text}${request.submit ? '\r' : ''}` if (input.length > 0 && !operation.cancelRequested) { this.resetReadinessEvidence() - this.writing = operation + const write = this.terminal.write(input) + const activeWrite = { + operation, + settled: write.then(() => true, () => false), + } + this.activeWrite = activeWrite try { - await this.terminal.write(Buffer.from(input, 'utf8')) + await write } finally { - this.writing = undefined + this.activeWrite = undefined } } + // Cancellation owns post-write signalling and reservation release. + if (operation.cancelRequested) return if (this.active === operation && operation.settled) { this.clearActive() return @@ -282,7 +277,7 @@ export class LocalPtySession implements PtyBackendSession { this.schedulePoll(operation) } } catch (error: unknown) { - if (this.active === operation) { + if (this.active === operation && !this.closing) { if (operation.settled) this.clearActive() else this.failActive(error) } @@ -323,8 +318,7 @@ export class LocalPtySession implements PtyBackendSession { async signal(signal: PtySignal): Promise { if (this.closing) throw new Error('PTY session is closing') - if (this.active !== undefined) this.signaledOperation = this.active - const targetPgid = await this.trackTerminalOperation(this.terminal.signalForeground(signal)) + const targetPgid = await this.terminal.signalForeground(signal) return { delivered: true, targetPgid } } @@ -345,23 +339,14 @@ export class LocalPtySession implements PtyBackendSession { } private readonly onTerminalData = (chunk: Buffer | Uint8Array | string): void => { - try { - const bytes = typeof chunk === 'string' ? Buffer.from(chunk, 'utf8') : chunk - this.onData(this.decoder.decode(bytes, { stream: true })) - } catch (error: unknown) { - this.onTransportFailure(new Error('PTY emitted invalid UTF-8', { cause: error })) - } + const bytes = typeof chunk === 'string' ? Buffer.from(chunk, 'utf8') : chunk + this.onData(this.decoder.decode(bytes, { stream: true })) } private readonly onTerminalEnd = (): void => { - try { - this.onData(this.decoder.decode()) - this.appendOutput(this.sanitizer.flush()) - } catch (error: unknown) { - this.onTransportFailure(new Error('PTY ended with invalid UTF-8', { cause: error })) - } finally { - this.outputEnded.resolve() - } + this.onData(this.decoder.decode()) + this.appendOutput(this.sanitizer.flush()) + this.outputEnded.resolve() } private readonly onTerminalError = (error: Error): void => { @@ -372,9 +357,9 @@ export class LocalPtySession implements PtyBackendSession { private onData(data: string): void { const sanitized = this.sanitizer.push(data) this.appendOutput(sanitized.text) - if (sanitized.prompt && this.delayedSignaledPrompt) { - this.delayedSignaledPrompt = false - } else if (sanitized.prompt) { + if (sanitized.prompt) { + // TODO(pty-delayed-signal-prompt): With a reproducer, define a marker-generation boundary + // before attributing a signal-delayed prompt to a later send. // Bash can print PROMPT_COMMAND before the kernel publishes its return // to the foreground process group. Retain the marker; polling below is // the authority that accepts it only after bash owns the foreground. @@ -402,7 +387,7 @@ export class LocalPtySession implements PtyBackendSession { this.transportFailure ??= failure this.statusValue = { kind: 'exited', exitCode: null, signal: null } this.failActive(failure) - this.terminal.terminate() + void this.terminal.terminate().catch(() => {}) } private appendOutput(text: string): void { @@ -417,7 +402,7 @@ export class LocalPtySession implements PtyBackendSession { if (this.activeTimer !== undefined) clearTimeout(this.activeTimer) this.activeTimer = setTimeout(() => { this.activeTimer = undefined - this.ownTerminalOperation(this.pollReadiness(operation)) + void this.pollReadiness(operation) }, delayMs) } @@ -457,7 +442,7 @@ export class LocalPtySession implements PtyBackendSession { this.settleActive('inferred_idle') } } catch (error: unknown) { - if (this.active === operation) this.failActive(error) + if (this.active === operation && !this.closing) this.failActive(error) } finally { this.polling = false const active = this.active @@ -470,13 +455,6 @@ export class LocalPtySession implements PtyBackendSession { private settleActive(waitReason: PtyWaitReason, retainOwnership = false): void { const operation = this.active if (operation === undefined) return - // A signaled command can return by silence before bash emits its prompt. - // Reserve that marker so it cannot become successor readiness after echo. - const signaled = this.signaledOperation === operation - if (signaled) this.signaledOperation = undefined - if (waitReason === 'inferred_idle' && !this.promptSeen && signaled) { - this.delayedSignaledPrompt = true - } const scrollbackTruncated = this.scrollback.snapshot().truncated if (retainOwnership) { this.stopPolling() @@ -489,10 +467,14 @@ export class LocalPtySession implements PtyBackendSession { } private stopPolling(): void { - if (this.activeTimer !== undefined) clearTimeout(this.activeTimer) - this.activeTimer = undefined + this.stopReadinessPolling() if (this.activeDeadlineTimer !== undefined) clearTimeout(this.activeDeadlineTimer) this.activeDeadlineTimer = undefined + } + + private stopReadinessPolling(): void { + if (this.activeTimer !== undefined) clearTimeout(this.activeTimer) + this.activeTimer = undefined this.pollingReady = undefined } @@ -502,43 +484,38 @@ export class LocalPtySession implements PtyBackendSession { this.activeAbort?.() this.activeAbort = undefined if (this.interrupting === operation) this.interrupting = undefined - if (this.signaledOperation === operation) this.signaledOperation = undefined - this.writing = undefined this.pollingReady = undefined this.active = undefined } - private failActive(error: unknown, retainOwnership = false): void { + private failActive(error: unknown): void { const operation = this.active if (operation === undefined) return - if (retainOwnership) { - this.stopPolling() - this.activeAbort?.() - this.activeAbort = undefined - } else { - this.clearActive() - } + this.clearActive() operation.fail(error) } private interrupt(operation: LocalSendOperation): void { if (this.active !== operation) return - this.signaledOperation = operation this.interrupting = operation - this.stopPolling() - this.ownTerminalOperation(this.interruptOnce(operation)) + this.stopReadinessPolling() + void this.interruptOnce(operation) } private async interruptOnce(operation: LocalSendOperation): Promise { try { + const activeWrite = this.activeWrite + if (activeWrite?.operation === operation && !await activeWrite.settled) return await this.terminal.signalForeground('SIGINT') } catch (error: unknown) { - if (this.active === operation) this.failActive(error, this.writing === operation) + if (this.active === operation && !this.closing) this.onTransportFailure(error) return } finally { if (this.interrupting === operation) this.interrupting = undefined } - if (this.active === operation && !operation.settled && !this.closing && this.writing !== operation) { + if (this.active === operation && operation.settled) { + this.clearActive() + } else if (this.active === operation && !this.closing) { this.pollingReady = operation this.schedulePoll(operation, 0) } @@ -549,20 +526,13 @@ export class LocalPtySession implements PtyBackendSession { // it as session_exit below, so an in-flight send is never mis-settled as // stdin_read/inferred_idle/timeout during the grace period. this.stopPolling() - this.terminal.terminate() - const quiescent = await this.terminal.waitForExit() - if (!quiescent) { - throw new Error(`PTY cleanup failed (${reason}); terminal session did not reach quiescence`) + try { + await this.terminal.terminate() + } catch (error: unknown) { + throw new Error(`PTY cleanup failed (${reason})`, { cause: error }) } - // Quiescence is the active send's terminal outcome. Detach its abort - // listener before snapshotting provider operations so no late interrupt - // can enter the owned set after the drain starts. + // Quiescence is the active send's terminal outcome. this.settleActive('session_exit') - await Promise.all(this.terminalOperations) - // Whole-session cleanup can fail before the top-level process exits. Wait - // for it first so that failure is reported instead of blocking forever on - // `done`; successful quiescence guarantees `done` can now settle status and - // drain the terminal output. await this.completion this.terminal.output.off('data', this.onTerminalData) this.terminal.output.off('end', this.onTerminalEnd) diff --git a/packages/pty/pty-local/tests/index.spec.ts b/packages/pty/pty-local/tests/index.spec.ts index 23e6f4bc89..48c8272106 100644 --- a/packages/pty/pty-local/tests/index.spec.ts +++ b/packages/pty/pty-local/tests/index.spec.ts @@ -61,8 +61,7 @@ function terminalHandle(): SubprocessTerminalHandle { write: async () => {}, inspectForeground: async () => ({ processGroupId: 123, inputWaiting: true }), signalForeground: async () => 123, - terminate: () => { output.end() }, - waitForExit: async () => true, + terminate: async () => { output.end() }, } } @@ -187,7 +186,7 @@ describe('LocalPtyBackend startup rollback', () => { expect(initialized).toHaveBeenCalledWith(undefined) }) - it('forwards setup cancellation only while terminal allocation is unpublished', async () => { + it('forwards terminal allocation cancellation directly', async () => { const ctx = new Context() await ctx.plugin(EmptySandbox) await ctx.plugin(SandboxPolicyService, { mode: 'danger-full-access', workspaceRoot: '/tmp' }) @@ -204,10 +203,9 @@ describe('LocalPtyBackend startup rollback', () => { () => stubLocalSession(), ) await published.spawn(spec(agent(ctx), publishedController.signal)) - expect(publishedSignal).toBeDefined() - expect(publishedSignal).not.toBe(publishedController.signal) + expect(publishedSignal).toBe(publishedController.signal) publishedController.abort(new Error('originating turn ended')) - expect(publishedSignal?.aborted).toBe(false) + expect(publishedSignal?.aborted).toBe(true) const pendingController = new AbortController() const seen = Promise.withResolvers() @@ -245,11 +243,10 @@ describe('LocalPtyBackend startup rollback', () => { write: async () => {}, inspectForeground: async () => ({ processGroupId: 123, inputWaiting: true }), signalForeground: async () => 123, - terminate() { + async terminate() { output.end() outcome.resolve({ exitCode: null, signal: 'SIGTERM' }) }, - waitForExit: async () => true, } queueMicrotask(() => { output.write(Buffer.from('\x1b]133;D;0\x07dsh> ')) }) const backend = new LocalPtyBackend( diff --git a/packages/pty/pty-local/tests/session.spec.ts b/packages/pty/pty-local/tests/session.spec.ts index dc03da2a63..22b07a0f8f 100644 --- a/packages/pty/pty-local/tests/session.spec.ts +++ b/packages/pty/pty-local/tests/session.spec.ts @@ -50,8 +50,8 @@ class FakeTerminal implements SubprocessTerminalHandle { throwWrite = false throwKill = false autoExitOnKill = true - quiescent = true - waitError: Error | undefined + terminateError: Error | undefined + private cleanup: Promise | undefined constructor(public inspector = new FakeInspector()) {} @@ -80,9 +80,9 @@ class FakeTerminal implements SubprocessTerminalHandle { }) } - async write(data: Uint8Array): Promise { + async write(data: string): Promise { if (this.throwWrite) throw new Error('write failed') - this.writes.push(Buffer.from(data).toString('utf8')) + this.writes.push(data) } async inspectForeground() { @@ -102,16 +102,20 @@ class FakeTerminal implements SubprocessTerminalHandle { return foreground.processGroupId } - terminate(): void { + terminate(): Promise { + if (this.cleanup !== undefined) return this.cleanup + const cleanup = this.terminateOnce() + this.cleanup = cleanup + void cleanup.catch(() => { this.cleanup = undefined }) + return cleanup + } + + private async terminateOnce(): Promise { + if (this.terminateError !== undefined) throw this.terminateError if (this.throwKill) throw new Error('kill failed') this.kills.push('SIGTERM') if (this.autoExitOnKill) this.emitExit(0, 15) } - - async waitForExit(): Promise { - if (this.waitError !== undefined) throw this.waitError - return this.quiescent - } } function makeSession( @@ -370,6 +374,100 @@ describe('LocalPtySession readiness and output', () => { expect(inspector.groups).not.toContainEqual([789, 'SIGINT']) }) + it('signals only after an in-flight provider write lands', async () => { + vi.useFakeTimers() + const terminal = new FakeTerminal() + const inspector = new FakeInspector() + const session = makeSession(terminal, inspector, config()) + await initialize(session, terminal) + + const writeGate = Promise.withResolvers() + terminal.write = async () => { await writeGate.promise } + const operation = session.startSend({ text: 'must be interrupted', submit: true }) + await Promise.resolve() + await Promise.resolve() + expect(operation.cancel()).toBe(true) + await vi.advanceTimersByTimeAsync(20) + expect(inspector.groups).toEqual([]) + + writeGate.resolve(undefined) + await vi.advanceTimersByTimeAsync(0) + expect(inspector.groups).toContainEqual([456, 'SIGINT']) + terminal.emitData('\x1b]133;D;130\x07dsh> ') + await vi.advanceTimersByTimeAsync(10) + await operation.done + }) + + it('does not signal when a cancelled provider write rejects', async () => { + vi.useFakeTimers() + const terminal = new FakeTerminal() + const inspector = new FakeInspector() + const session = makeSession(terminal, inspector, config()) + await initialize(session, terminal) + + const writeGate = Promise.withResolvers() + terminal.write = async () => { await writeGate.promise } + const operation = session.startSend({ text: 'rejected write', submit: true }) + await Promise.resolve() + await Promise.resolve() + expect(operation.cancel()).toBe(true) + + const rejected = expect(operation.done).rejects.toThrow('write failed after cancellation') + writeGate.reject(new Error('write failed after cancellation')) + await rejected + expect(inspector.groups).toEqual([]) + + const next = session.startSend({ text: '', submit: false }) + await vi.advanceTimersByTimeAsync(100) + expect((await next.done).waitReason).toBe('inferred_idle') + }) + + it('releases a timed-out cancellation after the provider write and signal settle', async () => { + vi.useFakeTimers() + const terminal = new FakeTerminal() + const inspector = new FakeInspector() + const session = makeSession(terminal, inspector, config()) + await initialize(session, terminal) + + const writeGate = Promise.withResolvers() + terminal.write = async () => { await writeGate.promise } + const operation = session.startSend({ text: 'slow cancelled write', submit: true }) + await Promise.resolve() + await Promise.resolve() + expect(operation.cancel()).toBe(true) + await vi.advanceTimersByTimeAsync(100) + + expect((await operation.done).waitReason).toBe('timeout') + expect(() => session.startSend({ text: 'must wait', submit: true })).toThrow('active send') + + writeGate.resolve(undefined) + await vi.advanceTimersByTimeAsync(0) + expect(inspector.groups).toContainEqual([456, 'SIGINT']) + + const next = session.startSend({ text: '', submit: false }) + await vi.advanceTimersByTimeAsync(100) + expect((await next.done).waitReason).toBe('inferred_idle') + }) + + it('retains the absolute timeout after cancellation while output stays active', async () => { + vi.useFakeTimers() + const terminal = new FakeTerminal() + const inspector = new FakeInspector() + const session = makeSession(terminal, inspector, config()) + await initialize(session, terminal) + + const operation = session.startSend({ text: 'ignore-sigint-and-write', submit: true }) + await Promise.resolve() + await Promise.resolve() + expect(operation.cancel()).toBe(true) + for (let elapsed = 20; elapsed <= 100; elapsed += 20) { + terminal.emitData('.') + await vi.advanceTimersByTimeAsync(20) + } + + expect(await operation.done).toMatchObject({ waitReason: 'timeout' }) + }) + it('does not resume cancellation polling after the terminal exits during signalling', async () => { vi.useFakeTimers() const terminal = new FakeTerminal() @@ -437,21 +535,24 @@ describe('LocalPtySession readiness and output', () => { const writeGate = Promise.withResolvers() terminal.write = async () => { await writeGate.promise } - terminal.signalForeground = async () => { throw new Error('interrupt failed') } + let signalCalls = 0 + terminal.signalForeground = async () => { + signalCalls += 1 + throw new Error('interrupt failed') + } const operation = session.startSend({ text: 'slow write', submit: true }) await Promise.resolve() await Promise.resolve() - const rejected = expect(operation.done).rejects.toThrow('interrupt failed') expect(operation.cancel()).toBe(true) - await rejected + await vi.advanceTimersByTimeAsync(20) + expect(signalCalls).toBe(0) expect(() => session.startSend({ text: 'must wait', submit: true })).toThrow('active send') writeGate.resolve(undefined) - await Promise.resolve() - await Promise.resolve() - const next = session.startSend({ text: '', submit: false }) - await vi.advanceTimersByTimeAsync(100) - expect((await next.done).waitReason).toBe('inferred_idle') + await expect(operation.done).rejects.toThrow('interrupt failed') + expect(signalCalls).toBe(1) + expect(session.status()).toEqual({ kind: 'exited', exitCode: null, signal: null }) + expect(() => session.startSend({ text: '', submit: false })).toThrow('has exited') }) it('handles startup exit, unknown exit signals, cancel-write failure, and stale polls', async () => { @@ -591,35 +692,6 @@ describe('LocalPtySession readiness and output', () => { expect(await operation.done).toMatchObject({ waitReason: 'stdin_read' }) }) - it('does not attribute a post-echo prompt from an inferred prior send to its successor', async () => { - vi.useFakeTimers() - const terminal = new FakeTerminal() - const session = new LocalPtySession(terminal, config({ idleSilenceMs: 50, timeoutMs: 200 })) - await initialize(session, terminal) - - const interrupted = session.startSend({ text: 'sleep', submit: true }) - await Promise.resolve() - await Promise.resolve() - await session.signal('SIGINT') - await vi.advanceTimersByTimeAsync(50) - expect((await interrupted.done).waitReason).toBe('inferred_idle') - - const successor = session.startSend({ text: "printf 'PID=%s\\n' \"$!\"", submit: true }) - let settled = false - void successor.done.then(() => { settled = true }) - await Promise.resolve() - await Promise.resolve() - terminal.emitData('printf \'PID=%s\\n\' "$!"\r\n\x1b]133;D;130\x07dsh> ') - await vi.advanceTimersByTimeAsync(10) - expect(settled).toBe(false) - - terminal.emitData('PID=123\r\n\x1b]133;D;0\x07dsh> ') - await vi.advanceTimersByTimeAsync(10) - const result = await successor.done - expect(result.waitReason).toBe('stdin_read') - expect(result.viewport).toContain('PID=123') - }) - it('retains a prompt marker until the startup shell regains the foreground group', async () => { vi.useFakeTimers() const terminal = new FakeTerminal() @@ -683,6 +755,7 @@ describe('LocalPtySession readiness and output', () => { it('contains terminal transport failures and preserves the first failure', async () => { const terminal = new FakeTerminal() + terminal.terminateError = new Error('cleanup after transport failure') const session = new LocalPtySession(terminal, config()) const operation = session.startSend({ text: '', submit: false }) terminal.output.emit('data', 'plain text') @@ -691,6 +764,7 @@ describe('LocalPtySession readiness and output', () => { .onTransportFailure(new Error('later failure')) await expect(operation.done).rejects.toThrow('output transport failed') expect(session.status()).toEqual({ kind: 'exited', exitCode: null, signal: null }) + terminal.terminateError = undefined await expect(session.close('transport')).rejects.toThrow('output transport failed') const rejectedTerminal = new FakeTerminal() @@ -700,19 +774,21 @@ describe('LocalPtySession readiness and output', () => { await expect(rejectedOperation.done).rejects.toThrow('raw transport failure') }) - it('rejects invalid UTF-8 in a data chunk and at stream end', async () => { + it('replaces invalid UTF-8 terminal output', async () => { const chunkTerminal = new FakeTerminal() const chunkSession = new LocalPtySession(chunkTerminal, config()) const chunkOperation = chunkSession.startSend({ text: '', submit: false }) chunkTerminal.emitBytes(Uint8Array.from([0xff])) - await expect(chunkOperation.done).rejects.toThrow('PTY emitted invalid UTF-8') + expect(chunkOperation.readOutput()).toEqual({ delta: '�', truncated: false }) + chunkTerminal.emitExit() + await chunkOperation.done const endTerminal = new FakeTerminal() const endSession = new LocalPtySession(endTerminal, config()) const endOperation = endSession.startSend({ text: '', submit: false }) endTerminal.emitBytes(Uint8Array.from([0xe2])) endTerminal.emitExit() - await expect(endOperation.done).rejects.toThrow('PTY ended with invalid UTF-8') + expect((await endOperation.done).viewport).toBe('�') }) it('contains readiness inspection failure and a stale inspection result', async () => { @@ -952,43 +1028,32 @@ describe('LocalPtySession bounds, signals, and teardown', () => { await expect(session.signal('SIGTERM')).rejects.toThrow('cannot resolve') }) - it('drains an in-flight public signal and rejects signals after close starts', async () => { + it('closes idempotently and rejects new signals', async () => { const terminal = new FakeTerminal() - const session = new LocalPtySession(terminal, config()) - const signal = Promise.withResolvers() - terminal.signalForeground = async () => await signal.promise - - const signaling = session.signal('SIGINT') - const closing = session.close('public signal') - let closed = false - void closing.then(() => { closed = true }) - await Promise.resolve() - expect(closed).toBe(false) - await expect(session.signal('SIGTERM')).rejects.toThrow('closing') - - signal.resolve(456) - await expect(signaling).resolves.toEqual({ delivered: true, targetPgid: 456 }) - await closing - expect(closed).toBe(true) - }) - - it('closes idempotently, contains signal races, and reports survivors', async () => { - const terminal = new FakeTerminal() - terminal.quiescent = false + terminal.throwKill = true const session = new LocalPtySession(terminal, config({ disposeGraceMs: 1 })) const closing = session.close('test') expect(session.close('other')).toBe(closing) - await expect(closing).rejects.toThrow('did not reach quiescence') + await expect(closing).rejects.toThrow('PTY cleanup failed (test)') expect(() => session.startSend({ text: '', submit: false })).toThrow('closing') + await expect(session.signal('SIGTERM')).rejects.toThrow('closing') }) - it('reports cleanup failure without waiting for top-level exit', async () => { + it('reports cleanup failure without waiting for top-level exit and permits retry', async () => { const terminal = new FakeTerminal() terminal.autoExitOnKill = false - terminal.waitError = new Error('terminal cleanup failed; surviving pids: 456') + terminal.terminateError = new Error('terminal cleanup failed; surviving pids: 456') const session = new LocalPtySession(terminal, config()) - await expect(session.close('survivor')).rejects.toThrow('surviving pids: 456') + await expect(session.close('survivor')).rejects.toMatchObject({ + message: 'PTY cleanup failed (survivor)', + cause: terminal.terminateError, + }) + expect(terminal.kills).toEqual([]) + + terminal.terminateError = undefined + terminal.autoExitOnKill = true + await expect(session.close('retry')).resolves.toBeUndefined() expect(terminal.kills).toEqual(['SIGTERM']) }) @@ -1010,73 +1075,25 @@ describe('LocalPtySession bounds, signals, and teardown', () => { await closing }) - it('does not finish close while a pre-write terminal operation is pending', async () => { + it('settles a closing send when provider termination cancels inspection', async () => { vi.useFakeTimers() const terminal = new FakeTerminal() const session = new LocalPtySession(terminal, config()) await initialize(session, terminal) - const inspection = Promise.withResolvers<{ processGroupId: number; inputWaiting: boolean }>() terminal.inspectForeground = async () => await inspection.promise - const operation = session.startSend({ text: 'must not run', submit: true }) - const closing = session.close('pending inspection') - let closed = false - void closing.then(() => { closed = true }) + const terminate = terminal.terminate.bind(terminal) + terminal.terminate = async () => { + inspection.reject(new Error('terminal terminated')) + await terminate() + } + const operation = session.startSend({ text: 'pending inspection', submit: true }) await Promise.resolve() - await Promise.resolve() - expect(closed).toBe(false) - inspection.resolve({ processGroupId: 456, inputWaiting: false }) - await closing - expect(closed).toBe(true) + await session.close('pending inspection') + + expect((await operation.done).waitReason).toBe('session_exit') expect(terminal.writes).toEqual([]) - expect((await operation.done).waitReason).toBe('session_exit') - }) - - it('does not finish close while a terminal write is pending', async () => { - vi.useFakeTimers() - const terminal = new FakeTerminal() - const session = new LocalPtySession(terminal, config()) - await initialize(session, terminal) - - const write = Promise.withResolvers() - terminal.write = async () => { await write.promise } - const operation = session.startSend({ text: 'pending write', submit: true }) - await Promise.resolve() - await Promise.resolve() - const closing = session.close('pending write') - let closed = false - void closing.then(() => { closed = true }) - await Promise.resolve() - await Promise.resolve() - expect(closed).toBe(false) - - write.resolve(undefined) - await closing - expect(closed).toBe(true) - expect((await operation.done).waitReason).toBe('session_exit') - }) - - it('detaches active cancellation before draining terminal operations', async () => { - vi.useFakeTimers() - const terminal = new FakeTerminal() - const session = new LocalPtySession(terminal, config()) - await initialize(session, terminal) - - const inspection = Promise.withResolvers<{ processGroupId: number; inputWaiting: boolean }>() - terminal.inspectForeground = async () => await inspection.promise - const signalForeground = vi.spyOn(terminal, 'signalForeground') - const controller = new AbortController() - const operation = session.startSend({ text: 'pending inspection', submit: true, signal: controller.signal }) - const closing = session.close('pending cancellation') - await Promise.resolve() - await Promise.resolve() - - controller.abort('late cancellation') - expect(signalForeground).not.toHaveBeenCalled() - inspection.resolve({ processGroupId: 456, inputWaiting: false }) - await closing - expect((await operation.done).waitReason).toBe('session_exit') }) }) diff --git a/packages/skill/skill-local/tests/skill-local.spec.ts b/packages/skill/skill-local/tests/skill-local.spec.ts index 9dc723fb58..75f4a4ed65 100644 --- a/packages/skill/skill-local/tests/skill-local.spec.ts +++ b/packages/skill/skill-local/tests/skill-local.spec.ts @@ -92,12 +92,6 @@ class TestFileSystem extends FileSystem { return text } - override async readTextBounded(target: FsTarget, maxBytes: number, signal?: AbortSignal): Promise { - const text = await this.readText(target, signal) - if (Buffer.byteLength(text) > maxBytes) throw new Error('too large') - return text - } - override async streamText(_target: FsTarget): Promise> { throw new Error('not needed in skill tests') } diff --git a/packages/subprocess/subprocess-local/README.i18n.yaml b/packages/subprocess/subprocess-local/README.i18n.yaml index ded1d12b30..d34e7475f9 100644 --- a/packages/subprocess/subprocess-local/README.i18n.yaml +++ b/packages/subprocess/subprocess-local/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/subprocess/subprocess-local/README.md -README.md: 9f30fa6dc676e7b82f87b75b78b7d3143f204c94 -README.zh.md: 0b6813ec7d754f42e8bf4c65a1dee33d77bf3787 +README.md: 31c5539750c4af2b1c4169ac1dce4c91da587af7 +README.zh.md: 74aed52084c0db3ed9bfe4fcb991f781976fb7b5 diff --git a/packages/subprocess/subprocess-local/README.md b/packages/subprocess/subprocess-local/README.md index 9f30fa6dc6..31c5539750 100644 --- a/packages/subprocess/subprocess-local/README.md +++ b/packages/subprocess/subprocess-local/README.md @@ -11,7 +11,7 @@ Local implementation of the [`@deepseek-ai/dsh-subprocess`](../subprocess/README - **Credential scrub + explicit merge** — `process.env` minus credential-shaped vars (`*KEY*`/`*SECRET*`/`*TOKEN*`) and all ambient `DSH_*` names; the spec's explicit `env` merges after that scrub with no namespace validation, so a deliberately supplied credential or current `DSH_*` fact wins while stale nested-harness identity cannot leak in ambiently. Supplied stdin is written and closed; otherwise fd 0 is `/dev/null`. See the [stdin/env Agent Note](../../../.agents/notes/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-surface.md) and [managed environment Agent Note](../../../.agents/notes/implemented/feature/2026-07-10-agent-session-identity-and-log-location.md). - **Offset-based reads** — collect-mode readers return deltas in whole-stream byte coordinates; the service never holds a cursor, so consumer-owned cursors (the bash background read path) and full-stream re-reads coexist, before and after settlement. - **Execution-world coordinates** — `cwd` is the host process cwd, `runtimeRoot` is an owner-private temporary directory removed on disposal before any process-cleanup failure is reported, and `resolveExecutable` checks absolute files or searches the scrubbed effective PATH with platform-aware executable extensions. -- **Terminal-process ownership** — `spawnTerminal` allocates `node-pty`, bridges UTF-8 terminal bytes, inspects and signals the current foreground process group, and sweeps descendants before and after terminating the top-level shell. Each foreground inspection retains exact identities from the rooted tree; Linux also enumerates the POSIX session after its leader exits. A previously observed macOS descendant and any same-session Linux member therefore remain fenced after reparenting, while pid/start identity prevents cleanup from following PID reuse. The higher PTY backend owns prompt readiness, buffers, and model-facing operations. +- **Terminal-process ownership** — `spawnTerminal` allocates `node-pty`, bridges UTF-8 terminal text, inspects and signals the current foreground process group, and exposes one awaited termination operation that sweeps descendants before and after terminating the top-level shell. Each foreground inspection retains exact identities from the rooted tree; Linux also enumerates the POSIX session after its leader exits. A previously observed macOS descendant and any same-session Linux member therefore remain fenced after reparenting, while pid/start identity prevents cleanup from following PID reuse. The higher PTY backend owns prompt readiness, buffers, and model-facing operations. - **Terminate-and-join disposal** — the service retains live handles only so its own disposal can escalate every running tree and await its exit; settled and spawn-failed handles leave the live set on settlement. ## Model Experience diff --git a/packages/subprocess/subprocess-local/README.zh.md b/packages/subprocess/subprocess-local/README.zh.md index 0b6813ec7d..74aed52084 100644 --- a/packages/subprocess/subprocess-local/README.zh.md +++ b/packages/subprocess/subprocess-local/README.zh.md @@ -11,7 +11,7 @@ - **凭据清除 + 显式合并**:以 `process.env` 为基础,移除形似凭据的变量(`*KEY*`/`*SECRET*`/`*TOKEN*`)和所有环境中已有的 `DSH_*` 名称;spec 的显式 `env` 在该清除之后合并且不做命名空间校验,因此有意提供的凭据或当前 `DSH_*` 事实会胜出,而陈旧的嵌套 harness 身份无法从环境中隐式漏入。提供的 stdin 会被写入后关闭;否则 fd 0 指向 `/dev/null`。参见 [stdin/env Agent Note(agent 决策记录)](../../../.agents/notes/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-surface.md)与[受管环境 Agent Note](../../../.agents/notes/implemented/feature/2026-07-10-agent-session-identity-and-log-location.md)。 - **基于偏移量的读取**:收集模式的读取器以全流字节坐标返回增量;服务自身从不持有游标,因此消费方自有的游标(bash 的后台读取路径)与完整流重读可以共存,结算前后皆然。 - **执行世界坐标**:`cwd` 是宿主进程 cwd,`runtimeRoot` 是所有者私有的临时目录,会在资源释放时删除,并且删除发生在报告任何进程清理失败之前;`resolveExecutable` 检查绝对文件,或使用平台感知的可执行扩展名在清理后的有效 PATH 中查找。 -- **终端进程所有权**:`spawnTerminal` 分配 `node-pty`,桥接 UTF-8 终端字节,检查当前前台进程组并向其发送信号,并在终止顶层 shell 前后清理后代。每次前台检查都会保留有根进程树中的精确身份;Linux 还会在会话 leader 退出后枚举该 POSIX 会话。因此,先前观察到的 macOS 后代以及任何同会话 Linux 成员在重新设定父进程后仍受身份围栏保护,而 pid/启动身份可防止清理因 PID 复用而跟随到其他进程。上层 PTY 后端负责提示符就绪检测、缓冲和面向模型的操作。 +- **终端进程所有权**:`spawnTerminal` 分配 `node-pty`,桥接 UTF-8 终端文本,检查当前前台进程组并向其发送信号,并公开一项须等待的终止操作,该操作会在终止顶层 shell 前后清理后代。每次前台检查都会保留有根进程树中的精确身份;Linux 还会在会话 leader 退出后枚举该 POSIX 会话。因此,先前观察到的 macOS 后代以及任何同会话 Linux 成员在重新设定父进程后仍受身份围栏保护,而 pid/启动身份可防止清理因 PID 复用而跟随到其他进程。上层 PTY 后端负责提示符就绪检测、缓冲和面向模型的操作。 - **先终止再等待退出的 dispose**:服务保留存活句柄,只为让自身的 dispose 能对每个仍在运行的进程树执行升级并等待其退出;已结算与 spawn 失败的句柄在结算时即离开存活集合。 ## 模型体验 diff --git a/packages/subprocess/subprocess-local/src/index.ts b/packages/subprocess/subprocess-local/src/index.ts index 0d71cb0e33..0d02f0e387 100644 --- a/packages/subprocess/subprocess-local/src/index.ts +++ b/packages/subprocess/subprocess-local/src/index.ts @@ -59,12 +59,7 @@ export class LocalSubprocessService extends SubprocessService { pending.push(handle.done.catch(() => {}).then(() => handle.waitForExit())) } for (const terminal of this.terminals) { - terminal.terminate() - // Cleanup may reject before the top-level process exits (for example, - // an identity-fenced descendant survives escalation). Await the cleanup - // transaction directly so disposal reports that failure rather than - // waiting forever on `done`. - pending.push(terminal.waitForExit().then(() => { this.terminals.delete(terminal) })) + pending.push(terminal.terminate().then(() => { this.terminals.delete(terminal) })) } this.live.clear() const outcomes = [ @@ -136,11 +131,6 @@ export class LocalSubprocessService extends SubprocessService { if (file === undefined || file.length === 0) { throw new Error('subprocess-local: terminal argv must contain a program') } - for (const [name, value] of [['rows', spec.rows], ['cols', spec.cols], ['graceMs', spec.graceMs]] as const) { - if (!Number.isSafeInteger(value) || value <= 0) { - throw new Error(`subprocess-local: terminal ${name} must be a positive safe integer`) - } - } spec.signal?.throwIfAborted() const options: IPtyForkOptions = { name: 'dumb', @@ -151,10 +141,10 @@ export class LocalSubprocessService extends SubprocessService { } const inspector = this.terminalInspector ?? createProcessInspector() const terminal = nodePty.spawn(file, [...spec.argv.slice(1)], options) - const handle = new LocalTerminalHandle(terminal, inspector, spec.graceMs, spec.signal) + const handle = new LocalTerminalHandle(terminal, inspector, spec.graceMs) this.terminals.add(handle) const release = async (): Promise => { - await handle.waitForExit() + await handle.terminate() this.terminals.delete(handle) } void handle.done.then(release, release).catch(() => {}) diff --git a/packages/subprocess/subprocess-local/src/terminal.ts b/packages/subprocess/subprocess-local/src/terminal.ts index 30e38bd422..93b7a8fc2e 100644 --- a/packages/subprocess/subprocess-local/src/terminal.ts +++ b/packages/subprocess/subprocess-local/src/terminal.ts @@ -4,7 +4,6 @@ import { Buffer } from 'node:buffer' import { constants } from 'node:os' import { PassThrough } from 'node:stream' import type { IDisposable, IPty } from 'node-pty' -import { SubprocessTerminalLifecycle } from '@deepseek-ai/dsh-subprocess' import type { SubprocessOutcome, SubprocessTerminalForeground, @@ -34,7 +33,7 @@ export class LocalTerminalHandle implements SubprocessTerminalHandle { private readonly outcome = Promise.withResolvers() private readonly dataDisposable: IDisposable private readonly exitDisposable: IDisposable - private readonly lifecycle: SubprocessTerminalLifecycle + private cleanup: Promise | undefined private exited = false private trackedDescendants: ProcessIdentity[] = [] @@ -42,13 +41,11 @@ export class LocalTerminalHandle implements SubprocessTerminalHandle { * @param terminal - allocated node-pty process. * @param inspector - platform process/session operations. * @param graceMs - TERM-to-KILL and exit-wait grace. - * @param signal - optional lifetime cancellation. */ constructor( private readonly terminal: IPty, private readonly inspector: ProcessInspector, private readonly graceMs: number, - signal?: AbortSignal, ) { this.pid = terminal.pid this.done = this.outcome.promise @@ -61,26 +58,15 @@ export class LocalTerminalHandle implements SubprocessTerminalHandle { exitCode: exitSignal === undefined || exitSignal === 0 ? exitCode : null, signal: signalName(exitSignal), }) - this.terminate() - }) - this.lifecycle = new SubprocessTerminalLifecycle({ - done: this.done, - cleanup: () => this.closeOnce(), - signal, + void this.terminate().catch(() => {}) }) } // node-pty writes synchronously; the seam returns a promise for remote transports. // eslint-disable-next-line @typescript-eslint/require-await - async write(data: Uint8Array): Promise { + async write(data: string): Promise { if (this.exited) throw new Error('terminal process has exited') - let text: string - try { - text = new TextDecoder('utf-8', { fatal: true }).decode(data) - } catch (error: unknown) { - throw new Error('terminal input must be valid UTF-8', { cause: error }) - } - this.terminal.write(text) + this.terminal.write(data) } // Local inspection is synchronous; the seam returns a promise for remote transports. @@ -107,12 +93,12 @@ export class LocalTerminalHandle implements SubprocessTerminalHandle { return foreground.processGroupId } - terminate(): void { - this.lifecycle.terminate() - } - - async waitForExit(signal?: AbortSignal): Promise { - return await this.lifecycle.waitForExit(signal) + terminate(): Promise { + if (this.cleanup !== undefined) return this.cleanup + const cleanup = this.closeOnce() + this.cleanup = cleanup + void cleanup.catch(() => { this.cleanup = undefined }) + return cleanup } private survivors(members: ProcessIdentity[]): ProcessIdentity[] { diff --git a/packages/subprocess/subprocess-local/tests/local.spec.ts b/packages/subprocess/subprocess-local/tests/local.spec.ts index e4dd4cfb44..e58c66f7fa 100644 --- a/packages/subprocess/subprocess-local/tests/local.spec.ts +++ b/packages/subprocess/subprocess-local/tests/local.spec.ts @@ -81,7 +81,7 @@ describe('LocalSubprocessService', () => { } }) - it('validates terminal spawn specs before allocating a PTY', async () => { + it('validates terminal allocation inputs before allocating a PTY', async () => { const ctx = new Context() const fiber = await ctx.plugin(LocalSubprocessService) const base: SubprocessTerminalSpawnSpec = { @@ -89,9 +89,6 @@ describe('LocalSubprocessService', () => { } await expect(ctx.subprocess.spawnTerminal({ ...base, argv: [] })).rejects.toThrow('must contain a program') await expect(ctx.subprocess.spawnTerminal({ ...base, argv: [''] })).rejects.toThrow('must contain a program') - await expect(ctx.subprocess.spawnTerminal({ ...base, rows: 1.5 })).rejects.toThrow('rows') - await expect(ctx.subprocess.spawnTerminal({ ...base, cols: 0 })).rejects.toThrow('cols') - await expect(ctx.subprocess.spawnTerminal({ ...base, graceMs: 0 })).rejects.toThrow('graceMs') await expect(ctx.subprocess.spawnTerminal({ ...base, signal: AbortSignal.abort('stop') })).rejects.toBe('stop') await fiber.dispose() }) @@ -99,8 +96,7 @@ describe('LocalSubprocessService', () => { it('terminates and joins an owned terminal during disposal', async () => { const ctx = new Context() const fiber = await ctx.plugin(LocalSubprocessService) - const terminate = vi.fn() - const waitForExit = vi.fn(async () => true) + const terminate = vi.fn(async () => {}) const terminal: SubprocessTerminalHandle = { pid: 1, output: new PassThrough(), @@ -109,13 +105,11 @@ describe('LocalSubprocessService', () => { inspectForeground: async () => undefined, signalForeground: async () => 1, terminate, - waitForExit, } const terminals = (ctx.subprocess as unknown as { terminals: Set }).terminals terminals.add(terminal) await fiber.dispose() expect(terminate).toHaveBeenCalledOnce() - expect(waitForExit).toHaveBeenCalledOnce() expect(terminals.size).toBe(0) }) @@ -124,8 +118,8 @@ describe('LocalSubprocessService', () => { const fiber = await ctx.plugin(LocalSubprocessService) const service = ctx.subprocess const runtimeRoot = service.runtimeRoot - const firstFailure = new Error('first retryable cleanup failure') - const secondFailure = new Error('second retryable cleanup failure') + const firstFailure = new Error('first cleanup failure') + const secondFailure = new Error('second cleanup failure') const disposalErrors: unknown[] = [] ctx.logger.error = ((error: unknown) => { disposalErrors.push(error) }) as typeof ctx.logger.error const failedTerminal: SubprocessTerminalHandle = { @@ -135,22 +129,19 @@ describe('LocalSubprocessService', () => { write: async () => {}, inspectForeground: async () => undefined, signalForeground: async () => 1, - terminate: vi.fn(), - waitForExit: vi.fn(async () => { throw firstFailure }), + terminate: vi.fn(async () => { throw firstFailure }), } const secondFailedTerminal: SubprocessTerminalHandle = { ...failedTerminal, - terminate: vi.fn(), - waitForExit: vi.fn(async () => { throw secondFailure }), + terminate: vi.fn(async () => { throw secondFailure }), } let finishCleanup!: () => void - const cleanup = new Promise((resolve) => { - finishCleanup = () => { resolve(true) } + const cleanup = new Promise((resolve) => { + finishCleanup = resolve }) const drainingTerminal: SubprocessTerminalHandle = { ...failedTerminal, - terminate: vi.fn(), - waitForExit: vi.fn(() => cleanup), + terminate: vi.fn(() => cleanup), } const terminals = (service as unknown as { terminals: Set }).terminals terminals.add(failedTerminal) @@ -187,8 +178,7 @@ describe('LocalSubprocessService', () => { write: async () => {}, inspectForeground: async () => undefined, signalForeground: async () => 1, - terminate: vi.fn(), - waitForExit: vi.fn(async () => { throw failure }), + terminate: vi.fn(async () => { throw failure }), } const terminals = (service as unknown as { terminals: Set }).terminals terminals.add(terminal) @@ -247,7 +237,7 @@ describe('LocalSubprocessService', () => { } }) - it('contains a terminal release failure after top-level exit', async () => { + it('retains a terminal whose automatic cleanup fails', async () => { let exitListener: ((event: { exitCode: number; signal?: number }) => void) | undefined const terminal = { pid: 123, @@ -264,6 +254,8 @@ describe('LocalSubprocessService', () => { try { const { default: IsolatedLocalSubprocessService } = await import('../src/index.ts') const ctx = new Context() + const disposalErrors: unknown[] = [] + ctx.logger.error = ((error: unknown) => { disposalErrors.push(error) }) as typeof ctx.logger.error const fiber = await ctx.plugin(IsolatedLocalSubprocessService) const alive = new Set([124]) ;(ctx.subprocess as InstanceType).terminalInspector = { @@ -281,10 +273,9 @@ describe('LocalSubprocessService', () => { exitListener?.({ exitCode: 0 }) await handle.done await new Promise(resolve => setTimeout(resolve, 10)) - alive.clear() - handle.terminate() - await handle.waitForExit() + expect((ctx.subprocess as unknown as { terminals: Set }).terminals.size).toBe(1) await fiber.dispose() + expect(disposalErrors).toHaveLength(1) } finally { vi.doUnmock('node-pty') vi.resetModules() diff --git a/packages/subprocess/subprocess-local/tests/terminal.spec.ts b/packages/subprocess/subprocess-local/tests/terminal.spec.ts index 456ce4ee8f..2eeb7c6a27 100644 --- a/packages/subprocess/subprocess-local/tests/terminal.spec.ts +++ b/packages/subprocess/subprocess-local/tests/terminal.spec.ts @@ -89,7 +89,7 @@ describe('LocalTerminalHandle', () => { handle.output.on('data', (chunk: Buffer) => { chunks.push(chunk) }) pty.emitData('hello €') - await handle.write(Buffer.from('input\r')) + await handle.write('input\r') expect(pty.writes).toEqual(['input\r']) expect(await handle.inspectForeground()).toEqual({ processGroupId: 456, inputWaiting: true }) expect(await handle.signalForeground('SIGINT')).toBe(456) @@ -98,16 +98,14 @@ describe('LocalTerminalHandle', () => { pty.emitExit(7, 9) pty.emitExit(0) expect(await handle.done).toEqual({ exitCode: null, signal: 'SIGKILL' }) - expect(await handle.waitForExit()).toBe(true) + await handle.terminate() expect(Buffer.concat(chunks).toString('utf8')).toBe('hello €') }) - it('rejects invalid input and unsafe foreground signals', async () => { + it('rejects unsafe foreground signals and writes after exit', async () => { const pty = new FakePty() const inspector = new FakeInspector() const handle = new LocalTerminalHandle(pty.asPty(), inspector, 10) - await expect(handle.write(Uint8Array.from([0xff]))).rejects.toThrow('valid UTF-8') - inspector.pgid = handle.pid await expect(handle.signalForeground('SIGKILL')).rejects.toThrow('terminate the terminal session') inspector.pgid = undefined @@ -116,8 +114,8 @@ describe('LocalTerminalHandle', () => { pty.emitExit(3) expect(await handle.done).toEqual({ exitCode: 3, signal: null }) - await handle.waitForExit() - await expect(handle.write(Buffer.from('late'))).rejects.toThrow('has exited') + await handle.terminate() + await expect(handle.write('late')).rejects.toThrow('has exited') }) it('keeps the shell alive until forced descendants leave', async () => { @@ -129,15 +127,14 @@ describe('LocalTerminalHandle', () => { inspector.removeOnSignal = false const handle = new LocalTerminalHandle(pty.asPty(), inspector, 20) - handle.terminate() - const quiescent = handle.waitForExit() + const quiescent = handle.terminate() await vi.advanceTimersByTimeAsync(20) expect(inspector.processes).toContainEqual([124, 'SIGKILL']) expect(pty.kills).toEqual([]) inspector.alive.delete(124) await vi.advanceTimersByTimeAsync(20) - expect(await quiescent).toBe(true) + await quiescent expect(pty.kills).toEqual(['SIGTERM']) }) @@ -149,17 +146,16 @@ describe('LocalTerminalHandle', () => { inspector.alive.add(124) inspector.removeOnSignal = false const handle = new LocalTerminalHandle(pty.asPty(), inspector, 20) - const waiting = handle.waitForExit() + pty.emitExit() + const waiting = handle.terminate() let settled = false void waiting.then(() => { settled = true }) - - pty.emitExit() await vi.advanceTimersByTimeAsync(10) expect(settled).toBe(false) inspector.alive.delete(124) await vi.advanceTimersByTimeAsync(20) - expect(await waiting).toBe(true) + await waiting }) it('cleans a same-session descendant after the top-level shell exits naturally', async () => { @@ -172,7 +168,7 @@ describe('LocalTerminalHandle', () => { pty.emitExit() - expect(await handle.waitForExit()).toBe(true) + await handle.terminate() expect(inspector.processes).toEqual([[124, 'SIGTERM']]) }) @@ -188,7 +184,7 @@ describe('LocalTerminalHandle', () => { inspector.members = [] pty.emitExit() - expect(await handle.waitForExit()).toBe(true) + await handle.terminate() expect(inspector.processes).toEqual([[124, 'SIGTERM']]) }) @@ -209,8 +205,7 @@ describe('LocalTerminalHandle', () => { return [] } const handle = new LocalTerminalHandle(pty.asPty(), inspector, 10) - handle.terminate() - await handle.waitForExit() + await handle.terminate() expect(inspector.processes).toEqual([[124, 'SIGTERM'], [125, 'SIGKILL']]) expect(pty.kills).toEqual(['SIGTERM']) }) @@ -225,14 +220,13 @@ describe('LocalTerminalHandle', () => { } const handle = new LocalTerminalHandle(pty.asPty(), inspector, 10) - handle.terminate() - await handle.waitForExit() + await handle.terminate() expect(inspector.processes).toEqual([[late.pid, 'SIGTERM']]) expect(pty.kills).toEqual(['SIGTERM']) }) - it('keeps a failed post-shell sweep retryable until its survivor leaves', async () => { + it('retries failed cleanup after a surviving descendant leaves', async () => { vi.useFakeTimers() const pty = new FakePty() const inspector = new FakeInspector() @@ -244,14 +238,15 @@ describe('LocalTerminalHandle', () => { } const handle = new LocalTerminalHandle(pty.asPty(), inspector, 10) - handle.terminate() - const failed = expect(handle.waitForExit()).rejects.toThrow('surviving pids: 124') + const first = handle.terminate() + const failed = expect(first).rejects.toThrow('surviving pids: 124') await vi.advanceTimersByTimeAsync(25) await failed inspector.alive.delete(late.pid) - handle.terminate() - expect(await handle.waitForExit()).toBe(true) + const retry = handle.terminate() + expect(retry).not.toBe(first) + await retry expect(inspector.processes).toEqual([[late.pid, 'SIGTERM'], [late.pid, 'SIGKILL']]) }) @@ -268,82 +263,35 @@ describe('LocalTerminalHandle', () => { if (signal === 'SIGKILL') inspector.alive.delete(identity.pid) } const handle = new LocalTerminalHandle(pty.asPty(), inspector, 20) - handle.terminate() - const quiescent = handle.waitForExit() + const quiescent = handle.terminate() await vi.advanceTimersByTimeAsync(25) - expect(await quiescent).toBe(true) + await quiescent expect(inspector.processes).toEqual([[124, 'SIGTERM'], [124, 'SIGKILL']]) }) - it('allows cleanup to retry after a surviving descendant leaves', async () => { - vi.useFakeTimers() - const pty = new FakePty() - const inspector = new FakeInspector() - inspector.members = [{ pid: 124, started: 'child' }] - inspector.alive.add(124) - inspector.removeOnSignal = false - const handle = new LocalTerminalHandle(pty.asPty(), inspector, 10) - - handle.terminate() - const first = expect(handle.waitForExit(new AbortController().signal)).rejects.toThrow('surviving pids: 124') - await vi.advanceTimersByTimeAsync(25) - await first - - inspector.alive.delete(124) - handle.terminate() - expect(await handle.waitForExit()).toBe(true) - expect(pty.kills).toEqual(['SIGTERM']) - }) - - it('bounds waits and reports a top-level process that ignores escalation', async () => { + it('reports a top-level process that ignores escalation', async () => { vi.useFakeTimers() const pty = new FakePty() pty.autoExitOnKill = false const handle = new LocalTerminalHandle(pty.asPty(), new FakeInspector(), 10) - expect(await handle.waitForExit(AbortSignal.abort())).toBe(false) - const controller = new AbortController() - const bounded = handle.waitForExit(controller.signal) - controller.abort() - expect(await bounded).toBe(false) - - handle.terminate() - const failed = expect(handle.waitForExit()).rejects.toThrow('surviving pid: 123') + const failed = expect(handle.terminate()).rejects.toThrow('surviving pid: 123') await vi.advanceTimersByTimeAsync(25) await failed expect(pty.kills).toEqual(['SIGTERM', 'SIGKILL']) pty.emitExit(0, 999) expect(await handle.done).toEqual({ exitCode: null, signal: null }) - handle.terminate() - expect(await handle.waitForExit()).toBe(true) + await handle.terminate() }) - it('contains process races and reacts to lifetime cancellation', async () => { + it('contains process races while reporting surviving descendants', async () => { const pty = new FakePty() pty.throwKill = true const inspector = new FakeInspector() inspector.members = [{ pid: 124, started: 'child' }] inspector.alive.add(124) inspector.throwProcess = true - const controller = new AbortController() - const handle = new LocalTerminalHandle(pty.asPty(), inspector, 1, controller.signal) - controller.abort() - const failed = expect(handle.waitForExit()).rejects.toThrow('surviving pids: 124') - await failed - - inspector.alive.delete(124) - pty.throwKill = false - handle.terminate() - await handle.waitForExit() - - const preAbortedPty = new FakePty() - const preAborted = new LocalTerminalHandle( - preAbortedPty.asPty(), - new FakeInspector(), - 1, - AbortSignal.abort('stop'), - ) - await preAborted.waitForExit() - expect(preAbortedPty.kills).toEqual(['SIGTERM']) + const handle = new LocalTerminalHandle(pty.asPty(), inspector, 1) + await expect(handle.terminate()).rejects.toThrow('surviving pids: 124') }) }) diff --git a/packages/subprocess/subprocess/README.i18n.yaml b/packages/subprocess/subprocess/README.i18n.yaml index 8e6c6a88e4..f8678464a9 100644 --- a/packages/subprocess/subprocess/README.i18n.yaml +++ b/packages/subprocess/subprocess/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/subprocess/subprocess/README.md -README.md: d03824da33bb44b2525b1343a27557ed15823418 -README.zh.md: 34cc75c3cfc8148754343e92b0c204760b1fb543 +README.md: 2fca5cdd08f3cbd36d8ef492226c91681b5568d0 +README.zh.md: 5830d0fc394160bc557e24646cc76d4bcfead13c diff --git a/packages/subprocess/subprocess/README.md b/packages/subprocess/subprocess/README.md index d03824da33..2fca5cdd08 100644 --- a/packages/subprocess/subprocess/README.md +++ b/packages/subprocess/subprocess/README.md @@ -11,8 +11,7 @@ The subprocess seam (`ctx.subprocess`) is the process half of one execution worl - The spec is fully explicit — argv, cwd, per-stream stdio dispositions, grace — because deployment-varying defaults belong to the calling seam's config, not to a hidden subprocess-service default (the `dsh-bash` request/spec split is the owning template). `argv` is never shell-interpreted; a consumer that wants a shell passes `['bash', '-c', command]` itself. - Stdio is Node-shaped per stream: `'pipe'` hands the caller the raw stream for its own protocol framing (LSP JSON-RPC, ACP ndjson), `'inherit'` passes the parent descriptor through for diagnostics, and collect mode (`{ maxBytes, spill? }`) buffers a bounded tail with an optional full-stream spill file. Collect readers take whole-stream byte offsets and never consume, so independent readers cannot steal one another's deltas; a read whose offset slid out of the in-memory tail is `lossy` and points at the spill file when one exists. Collected output stays readable after settlement. - Termination is tree-scoped on every platform (POSIX detached groups with direct-child fallback; Windows `taskkill /T`): `terminate()` — the only termination verb — escalates SIGTERM→grace→SIGKILL (idempotent, driven by the spec's abort signal too, a no-op once the tree is gone), and `waitForExit(signal?)` observes whole-tree liveness so a consumer-owned teardown ladder holds each tier on real quiescence — the manager reacts but never classifies why (callers own deadlines, teardown ladders, and cause classification). -- `spawnTerminal(spec)` is the only non-pipe primitive. Its handle owns a real PTY, valid-UTF-8 byte I/O, foreground-process-group inspection/signalling, TERM-to-KILL whole-session cleanup, and a quiescence wait. The output stream ends after queued output when the top-level process exits; a live transport failure rejects `done`. These operations remain one substrate primitive because ordinary pipes cannot allocate a controlling terminal or prove and clean the complete terminal session; readiness, scrollback, and owner policy remain in the PTY consumer. -- `SubprocessTerminalLifecycle` composes a handle's top-level `done` promise with its provider-owned session cleanup. It binds lifetime cancellation, shares one active cleanup attempt, permits a failed attempt to retry, normalizes cleanup rejections, and bounds quiescence observation without knowing the provider's process mechanics. +- `spawnTerminal(spec)` is the only non-pipe primitive. Its handle owns a real PTY, UTF-8 text I/O, foreground-process-group inspection/signalling, and one awaited `terminate()` operation that reaches whole-session quiescence and settles in-flight handle calls. The spec signal cancels allocation only; the published handle owns its lifetime. The output stream ends after queued output when the top-level process exits, and a live transport failure rejects `done`. These operations remain one substrate primitive because ordinary pipes cannot allocate a controlling terminal or clean the complete terminal session; readiness, scrollback, and owner policy remain in the PTY consumer. - `scrubbedParentEnv()` / `SENSITIVE_ENV_PATTERN` are the one shared scrub definition: ambient credential-shaped and `DSH_*` names are dropped, and explicit `env` merges after the scrub. The local ordinary and terminal spawns both apply it; SDK-managed transports that own their spawn may import it directly. - Disposal of the service terminates all still-running managed processes and awaits their exit. diff --git a/packages/subprocess/subprocess/README.zh.md b/packages/subprocess/subprocess/README.zh.md index 34cc75c3cf..5830d0fc39 100644 --- a/packages/subprocess/subprocess/README.zh.md +++ b/packages/subprocess/subprocess/README.zh.md @@ -11,8 +11,7 @@ - spec 完全显式(argv、cwd、按流划分的 stdio 处置方式(disposition)、宽限期),因为随部署变化的默认值属于调用方 seam 的配置,而不属于某个隐藏的进程管理器默认值(`dsh-bash` 的 request/spec 拆分是这条规则的所属模板)。`argv` 绝不经过 shell 解释;需要 shell 的消费方自行传入 `['bash', '-c', command]`。 - stdio 按流采用 Node 形状:`'pipe'` 把原始流交给调用方做自己的协议分帧(LSP 的 JSON-RPC、ACP(Agent Client Protocol)的 ndjson),`'inherit'` 直通父进程描述符以承载诊断输出,收集模式(collect)`{ maxBytes, spill? }` 则缓冲一段有界尾部,外加可选的完整流 spill 文件。收集模式的读取器接受全流字节偏移量且从不消费,因此独立的读取器不会抢走彼此的增量;偏移量滑出内存尾部窗口的读取标记为 `lossy`,并在 spill 文件存在时指向它。收集到的输出在结算后仍可读取。 - 终止在每个平台上都以进程树为范围(POSIX 用 detached 进程组并以直接子进程回退;Windows 用 `taskkill /T`):`terminate()`(唯一的终止动词)执行 SIGTERM→宽限期→SIGKILL 升级(幂等,也由 spec 的 abort 信号驱动,进程树消亡后为空操作);`waitForExit(signal?)` 观察整棵进程树的存活状态,使消费方自有的拆卸阶梯能在真正完全停稳后才进入下一层。管理器只响应中止,但绝不判定原因(deadline、拆卸阶梯与原因分类归调用方所有)。 -- `spawnTerminal(spec)` 是唯一的非管道原语。其句柄负责真实 PTY、有效 UTF-8 字节 I/O、前台进程组检查/信号发送、TERM→KILL 全会话清理,以及等待完全停稳。顶层进程退出后,输出流会在排完队列中的输出后结束;存活期间的传输故障会拒绝 `done`。这些操作仍属于一项基底原语,因为普通管道无法分配控制终端,也无法证明并清理完整的终端会话;就绪检测、scrollback 与所有者策略仍归 PTY 消费方所有。 -- `SubprocessTerminalLifecycle` 把句柄的顶层 `done` promise 与由提供方负责的会话清理组合起来。它绑定生命周期取消,共享同一个进行中的清理尝试,允许失败的尝试重试,规范化清理拒绝,并在不了解提供方进程机制的情况下对完全停稳观测施加上限。 +- `spawnTerminal(spec)` 是唯一的非管道原语。其句柄负责真实 PTY、UTF-8 文本 I/O、前台进程组检查/信号发送,以及一项须等待的 `terminate()` 操作;该操作会使整个会话完全停稳,并让所有在途句柄调用结算。spec 信号只取消分配;句柄一经发布,便负责自身生命周期。顶层进程退出后,输出流会在排完队列中的输出后结束;存活期间的传输故障会拒绝 `done`。这些操作仍属于一项基底原语,因为普通管道无法分配控制终端,也无法清理完整的终端会话;就绪检测、scrollback 与所有者策略仍归 PTY 消费方所有。 - `scrubbedParentEnv()` / `SENSITIVE_ENV_PATTERN` 是唯一一份共享的凭据清除定义:环境中形似凭据的名称与 `DSH_*` 名称都会被丢弃,显式 `env` 在清除之后合并。本地普通 spawn 与终端 spawn 都应用这一定义;自行拥有 spawn 的 SDK 管理传输层可以直接导入它。 - 服务自身的 dispose(资源释放)会终止所有仍在运行的受管进程并等待其退出。 diff --git a/packages/subprocess/subprocess/src/index.ts b/packages/subprocess/subprocess/src/index.ts index df79ee5f3e..e1749203dd 100644 --- a/packages/subprocess/subprocess/src/index.ts +++ b/packages/subprocess/subprocess/src/index.ts @@ -14,8 +14,6 @@ import type { SubprocessHandle, SubprocessSpawnSpec } from './types.ts' import type { SubprocessTerminalHandle, SubprocessTerminalSpawnSpec } from './types.ts' export { DSH_ENV_PREFIX } from './types.ts' -export { SubprocessTerminalLifecycle } from './terminal-lifecycle.ts' -export type { SubprocessTerminalLifecycleOptions } from './terminal-lifecycle.ts' export type { CollectedOutput, DshEnvironment, @@ -95,10 +93,11 @@ declare module 'cordis' { * quiescence. * - Disposal of the service terminates all still-running managed processes * and awaits their exit. - * - {@link spawnTerminal} owns terminal allocation, byte transport, - * foreground groups, signalling, and whole-session quiescence; readiness - * and persistent-shell policy stay in the PTY consumer. Its output stream - * ends after queued terminal output when the top-level process exits. + * - {@link spawnTerminal} owns terminal allocation, text transport, + * foreground groups, signalling, and whole-session quiescence behind one + * awaited termination method; readiness and persistent-shell policy stay + * in the PTY consumer. Its output stream ends after queued terminal output + * when the top-level process exits. */ export abstract class SubprocessService extends Service { constructor(ctx: Context) { @@ -138,7 +137,7 @@ export abstract class SubprocessService extends Service { * Allocate a real terminal and start one owned process session. This is the * only non-pipe process primitive: implementations own terminal byte I/O, * foreground groups, signals, and complete session-tree cleanup. - * @param spec - fully specified argv, cwd, environment, dimensions, grace, and cancellation. + * @param spec - fully specified argv, cwd, environment, dimensions, grace, and allocation cancellation. * @returns the live terminal handle after allocation succeeds. */ abstract spawnTerminal(spec: SubprocessTerminalSpawnSpec): Promise diff --git a/packages/subprocess/subprocess/src/terminal-lifecycle.ts b/packages/subprocess/subprocess/src/terminal-lifecycle.ts deleted file mode 100644 index 76430fe7c6..0000000000 --- a/packages/subprocess/subprocess/src/terminal-lifecycle.ts +++ /dev/null @@ -1,104 +0,0 @@ -/** Provider-neutral lifecycle transaction for terminal-process handles. */ - -/** Inputs owned by one terminal-process lifecycle controller. */ -export interface SubprocessTerminalLifecycleOptions { - /** Settlement of the top-level terminal process or its live transport. */ - readonly done: Promise - /** Provider-owned cleanup that reaches whole-session quiescence. */ - readonly cleanup: () => Promise - /** Optional cancellation for the complete terminal lifetime. */ - readonly signal?: AbortSignal | undefined -} - -function normalizeCleanupError(error: unknown): Error { - return error instanceof Error ? error : new Error(String(error)) -} - -/** - * Coordinates terminal cleanup without knowing how a provider allocates or - * terminates its process session. One active cleanup attempt is shared by all - * callers; a rejected attempt may be retried, and successful cleanup removes - * the lifetime abort listener. - */ -export class SubprocessTerminalLifecycle { - private cleanupAttempt: Promise | undefined - private removeLifetimeAbort: (() => void) | undefined - - /** - * @param options - top-level settlement, provider cleanup, and lifetime cancellation. - */ - constructor(private readonly options: SubprocessTerminalLifecycleOptions) { - const onDone = (): void => { this.terminate() } - void options.done.then(onDone, onDone) - - if (options.signal !== undefined) { - const onAbort = (): void => { this.terminate() } - options.signal.addEventListener('abort', onAbort, { once: true }) - this.removeLifetimeAbort = () => { options.signal?.removeEventListener('abort', onAbort) } - if (options.signal.aborted) this.terminate() - } - } - - /** Begin an idempotent provider cleanup attempt. */ - terminate(): void { - void this.startCleanup().catch(() => {}) - } - - /** - * Wait for top-level settlement and successful whole-session cleanup. - * @param signal - optional bound for this observation only. - * @returns true after quiescence, false when the observer signal aborts first. - */ - async waitForExit(signal?: AbortSignal): Promise { - const quiescence = this.cleanupAttempt ?? this.options.done.then( - () => this.startCleanup(), - () => this.startCleanup(), - ) - if (signal === undefined) { - await quiescence - return true - } - if (signal.aborted) return false - - return await new Promise((resolve, reject) => { - let settled = false - const finish = (complete: () => void): void => { - if (settled) return - settled = true - signal.removeEventListener('abort', onAbort) - complete() - } - const onAbort = (): void => { finish(() => { resolve(false) }) } - - signal.addEventListener('abort', onAbort, { once: true }) - if (signal.aborted) onAbort() - void quiescence.then( - () => { finish(() => { resolve(true) }) }, - (error: unknown) => { finish(() => { reject(normalizeCleanupError(error)) }) }, - ) - }) - } - - private startCleanup(): Promise { - if (this.cleanupAttempt !== undefined) return this.cleanupAttempt - - const outcome = Promise.withResolvers() - this.cleanupAttempt = outcome.promise.catch((error: unknown) => { - this.cleanupAttempt = undefined - throw normalizeCleanupError(error) - }) - void this.cleanupAttempt.then( - () => { - this.removeLifetimeAbort?.() - this.removeLifetimeAbort = undefined - }, - () => {}, - ) - try { - void this.options.cleanup().then(outcome.resolve, outcome.reject) - } catch (error: unknown) { - outcome.reject(error) - } - return this.cleanupAttempt - } -} diff --git a/packages/subprocess/subprocess/src/types.ts b/packages/subprocess/subprocess/src/types.ts index 206104ad5b..6b606fd870 100644 --- a/packages/subprocess/subprocess/src/types.ts +++ b/packages/subprocess/subprocess/src/types.ts @@ -210,7 +210,7 @@ export interface SubprocessTerminalSpawnSpec { cols: number /** TERM-to-KILL cleanup grace for the complete terminal session. */ graceMs: number - /** Cancellation of setup or the live terminal session. */ + /** Cancellation of terminal allocation; a published handle owns its later lifetime. */ signal?: AbortSignal | undefined } @@ -236,10 +236,10 @@ export interface SubprocessTerminalHandle { /** Resolves when the top-level process exits; rejects only for a live transport failure. */ readonly done: Promise /** - * Write bytes to the terminal input. - * @param data - valid UTF-8 bytes to deliver without implicit newline conversion. + * Write text to the terminal input. + * @param data - text to deliver without implicit newline conversion. */ - write(data: Uint8Array): Promise + write(data: string): Promise /** * Inspect the current foreground process group. * @returns its id and input-wait fact, or undefined when no foreground group can be resolved. @@ -251,12 +251,9 @@ export interface SubprocessTerminalHandle { * @returns the exact group id that received it. */ signalForeground(signal: SubprocessTerminalSignal): Promise - /** Begin idempotent TERM-to-KILL cleanup of the complete terminal session. */ - terminate(): void /** - * Await whole-session quiescence, not only top-level process exit. - * @param signal - optional bound for this wait. - * @returns true after quiescence, false when `signal` aborts first. + * Idempotently terminate the complete terminal session and await whole-session quiescence. + * After settlement, no write, inspection, or signal call remains in flight. */ - waitForExit(signal?: AbortSignal): Promise + terminate(): Promise } diff --git a/packages/subprocess/subprocess/tests/service.spec.ts b/packages/subprocess/subprocess/tests/service.spec.ts index 853b333400..9f321caf54 100644 --- a/packages/subprocess/subprocess/tests/service.spec.ts +++ b/packages/subprocess/subprocess/tests/service.spec.ts @@ -48,8 +48,7 @@ class StubSubprocessService extends SubprocessService { write: async () => {}, inspectForeground: async () => ({ processGroupId: 1, inputWaiting: true }), signalForeground: async () => 1, - terminate: () => {}, - waitForExit: async () => true, + terminate: async () => {}, } } } diff --git a/packages/subprocess/subprocess/tests/terminal-lifecycle.spec.ts b/packages/subprocess/subprocess/tests/terminal-lifecycle.spec.ts deleted file mode 100644 index 2dd688bbc0..0000000000 --- a/packages/subprocess/subprocess/tests/terminal-lifecycle.spec.ts +++ /dev/null @@ -1,125 +0,0 @@ -import { describe, expect, it, vi } from 'vitest' -import { SubprocessTerminalLifecycle } from '@deepseek-ai/dsh-subprocess' - -describe('SubprocessTerminalLifecycle', () => { - it('waits for top-level settlement and the provider cleanup transaction', async () => { - const done = Promise.withResolvers() - const cleanupGate = Promise.withResolvers() - const cleanup = vi.fn(() => cleanupGate.promise) - const lifecycle = new SubprocessTerminalLifecycle({ done: done.promise, cleanup }) - - const waiting = lifecycle.waitForExit() - expect(cleanup).not.toHaveBeenCalled() - done.resolve(undefined) - await vi.waitFor(() => { expect(cleanup).toHaveBeenCalledOnce() }) - - const observed = vi.fn() - void waiting.then(observed) - await Promise.resolve() - expect(observed).not.toHaveBeenCalled() - - cleanupGate.resolve(undefined) - await expect(waiting).resolves.toBe(true) - lifecycle.terminate() - await expect(lifecycle.waitForExit()).resolves.toBe(true) - expect(cleanup).toHaveBeenCalledOnce() - }) - - it('normalizes cleanup failures, permits retries, and retains lifetime cancellation until success', async () => { - const done = Promise.withResolvers() - const lifetime = new AbortController() - const removeListener = vi.spyOn(lifetime.signal, 'removeEventListener') - const synchronousFailure = new Error('first cleanup failed') - let attempt = 0 - const cleanup = vi.fn((): Promise => { - attempt += 1 - if (attempt === 1) throw synchronousFailure - if (attempt === 2) { - return Promise.resolve().then(() => { - const nonErrorRejection: unknown = 'cleanup transport gone' - throw nonErrorRejection - }) - } - return Promise.resolve() - }) - const lifecycle = new SubprocessTerminalLifecycle({ - done: done.promise, - cleanup, - signal: lifetime.signal, - }) - - lifecycle.terminate() - await expect(lifecycle.waitForExit()).rejects.toBe(synchronousFailure) - lifecycle.terminate() - await expect(lifecycle.waitForExit()).rejects.toThrow('cleanup transport gone') - - lifetime.abort() - await expect(lifecycle.waitForExit()).resolves.toBe(true) - expect(cleanup).toHaveBeenCalledTimes(3) - expect(removeListener).toHaveBeenCalledWith('abort', expect.any(Function)) - - done.reject(new Error('top-level transport failed')) - await Promise.resolve() - expect(cleanup).toHaveBeenCalledTimes(3) - }) - - it('starts cleanup for a pre-aborted lifetime and bounds a wait that is already aborted', async () => { - const cleanupGate = Promise.withResolvers() - const cleanup = vi.fn(() => cleanupGate.promise) - const lifecycle = new SubprocessTerminalLifecycle({ - done: new Promise(() => {}), - cleanup, - signal: AbortSignal.abort(new Error('lifetime cancelled')), - }) - - expect(cleanup).toHaveBeenCalledOnce() - await expect(lifecycle.waitForExit(AbortSignal.abort())).resolves.toBe(false) - cleanupGate.resolve(undefined) - await expect(lifecycle.waitForExit()).resolves.toBe(true) - }) - - it('contains cleanup settlement after an observer aborts between signal checks', async () => { - const firstCleanup = Promise.withResolvers() - const cleanup = vi.fn() - .mockImplementationOnce(() => firstCleanup.promise) - .mockResolvedValueOnce(undefined) - const lifecycle = new SubprocessTerminalLifecycle({ done: Promise.resolve(), cleanup }) - const observer = new AbortController().signal - vi.spyOn(observer, 'aborted', 'get') - .mockReturnValueOnce(false) - .mockReturnValueOnce(true) - - await expect(lifecycle.waitForExit(observer)).resolves.toBe(false) - firstCleanup.reject(new Error('late cleanup failure')) - await vi.waitFor(() => { expect(cleanup).toHaveBeenCalledOnce() }) - await Promise.resolve() - - lifecycle.terminate() - await expect(lifecycle.waitForExit()).resolves.toBe(true) - expect(cleanup).toHaveBeenCalledTimes(2) - }) - - it('reports bounded cleanup success and failure', async () => { - const successful = new SubprocessTerminalLifecycle({ - done: Promise.resolve(), - cleanup: async () => {}, - }) - await expect(successful.waitForExit(new AbortController().signal)).resolves.toBe(true) - - const failure = new Error('quiescence failed') - const failed = new SubprocessTerminalLifecycle({ - done: Promise.resolve(), - cleanup: () => Promise.reject(failure), - }) - await expect(failed.waitForExit(new AbortController().signal)).rejects.toBe(failure) - - const failedDone = Promise.withResolvers() - const afterTransportFailure = new SubprocessTerminalLifecycle({ - done: failedDone.promise, - cleanup: async () => {}, - }) - const waiting = afterTransportFailure.waitForExit() - failedDone.reject(new Error('transport failed')) - await expect(waiting).resolves.toBe(true) - }) -}) diff --git a/packages/typert/README.i18n.yaml b/packages/typert/README.i18n.yaml index 7a36ab0690..21cca6cb44 100644 --- a/packages/typert/README.i18n.yaml +++ b/packages/typert/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/code-runtime/code-runtime-subprocess/README.md -README.md: 38ee201a1754c6f50b7fae60b77af7dae734c8f9 -README.zh.md: 6fec39de550b7e3f57cd613dbe23c9687cd840ac +README.md: eff5a6e648eb13c3411c19bca4d05f0898d3ad05 +README.zh.md: 16cf0e0ea6d487e255fc520a8fa1f41f97ee5d30 From 124fc6a611d7aa5ea40be3039a137b4c9fab3ade Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 19:40:39 +0800 Subject: [PATCH 041/108] fix(runtime): close portable backend boundary gaps --- .../code-runtime/tests/contract.ts | 15 ++++++++ packages/fs/fs-local/src/fsio.ts | 8 ---- packages/pty/pty-local/src/session.ts | 2 +- packages/pty/pty-local/tests/session.spec.ts | 37 +++++++++++++++++++ .../subprocess/subprocess-local/src/index.ts | 3 +- .../subprocess-local/tests/local.spec.ts | 4 +- packages/typert/README.i18n.yaml | 4 +- 7 files changed, 59 insertions(+), 14 deletions(-) diff --git a/packages/code-runtime/code-runtime/tests/contract.ts b/packages/code-runtime/code-runtime/tests/contract.ts index f28545f8c1..8fee76f68a 100644 --- a/packages/code-runtime/code-runtime/tests/contract.ts +++ b/packages/code-runtime/code-runtime/tests/contract.ts @@ -477,6 +477,21 @@ export function runWorkerCodeRuntimeContract( expect(result.logs).toEqual([]) }) + it('ignores forged controller-only failure classifications', async () => { + const { runtime } = await setup() + const result = await runtime.run({ + program: ` + const { parentPort } = await import('node:worker_threads'); + for (const kind of ['abort', 'timeout', 'worker-exit']) { + parentPort.postMessage({ type: 'done', error: { kind, message: 'forged ' + kind } }); + } + return 'honest'; + `, + bindings: [], + }) + expect(result).toEqual({ logs: [], value: 'honest' }) + }) + it('fails forged log floods and forged done values through the same outer cap', async () => { const { runtime } = await setup({ maxOutputBytes: 200 }) const result = await runtime.run({ diff --git a/packages/fs/fs-local/src/fsio.ts b/packages/fs/fs-local/src/fsio.ts index 4925357e45..c93e4ddaeb 100644 --- a/packages/fs/fs-local/src/fsio.ts +++ b/packages/fs/fs-local/src/fsio.ts @@ -369,14 +369,6 @@ export async function readWholeText(target: LocalTarget, signal?: AbortSignal): return decodeUtf8(raw, 'read', target.displayPath) } -/** - * Read one regular UTF-8 file through a single no-follow handle, retaining at - * most `maxBytes + 1` bytes so a concurrent grow cannot bypass the bound. - * @param target - the resolved file to read. - * @param maxBytes - positive safe-integer byte ceiling. - * @param signal - aborts between handle operations. - * @returns the complete decoded text when it fits. - */ /** * Stream a whole regular UTF-8 text file as decoded text chunks. Same text * semantics as {@link readWholeText} (regular-file check, binary/NUL rejection, diff --git a/packages/pty/pty-local/src/session.ts b/packages/pty/pty-local/src/session.ts index 076f21e3b9..f9b35dc665 100644 --- a/packages/pty/pty-local/src/session.ts +++ b/packages/pty/pty-local/src/session.ts @@ -415,7 +415,7 @@ export class LocalPtySession implements PtyBackendSession { return } const foreground = await this.terminal.inspectForeground() - if (this.active !== operation) return + if (this.active !== operation || this.closing) return const idleFor = Date.now() - this.lastOutputAt if (this.promptSeen && foreground !== undefined && this.shellPgid === undefined) { this.shellPgid = foreground.processGroupId diff --git a/packages/pty/pty-local/tests/session.spec.ts b/packages/pty/pty-local/tests/session.spec.ts index 22b07a0f8f..14cf252a0e 100644 --- a/packages/pty/pty-local/tests/session.spec.ts +++ b/packages/pty/pty-local/tests/session.spec.ts @@ -1096,4 +1096,41 @@ describe('LocalPtySession bounds, signals, and teardown', () => { expect(terminal.writes).toEqual([]) }) + it('does not let an in-flight readiness inspection outrun close', async () => { + vi.useFakeTimers() + const terminal = new FakeTerminal() + const session = new LocalPtySession(terminal, config()) + await initialize(session, terminal) + const inspection = Promise.withResolvers<{ processGroupId: number; inputWaiting: boolean }>() + const originalInspect = terminal.inspectForeground.bind(terminal) + let inspections = 0 + terminal.inspectForeground = async () => { + inspections += 1 + return inspections === 1 ? await originalInspect() : await inspection.promise + } + const operation = session.startSend({ text: 'pending readiness', submit: true }) + await Promise.resolve() + await Promise.resolve() + terminal.emitData('\x1b]133;D;0\x07dsh> ') + await vi.advanceTimersByTimeAsync(10) + expect(inspections).toBe(2) + + const termination = Promise.withResolvers() + terminal.terminate = async () => { + await termination.promise + terminal.emitExit(0, 15) + } + const closing = session.close('in-flight readiness') + let settled = false + void operation.done.then(() => { settled = true }) + inspection.resolve({ processGroupId: 456, inputWaiting: true }) + await Promise.resolve() + await Promise.resolve() + expect(settled).toBe(false) + + termination.resolve(undefined) + await closing + expect((await operation.done).waitReason).toBe('session_exit') + }) + }) diff --git a/packages/subprocess/subprocess-local/src/index.ts b/packages/subprocess/subprocess-local/src/index.ts index 0d02f0e387..46d179a7b2 100644 --- a/packages/subprocess/subprocess-local/src/index.ts +++ b/packages/subprocess/subprocess-local/src/index.ts @@ -59,9 +59,10 @@ export class LocalSubprocessService extends SubprocessService { pending.push(handle.done.catch(() => {}).then(() => handle.waitForExit())) } for (const terminal of this.terminals) { - pending.push(terminal.terminate().then(() => { this.terminals.delete(terminal) })) + pending.push(terminal.terminate()) } this.live.clear() + this.terminals.clear() const outcomes = [ ...await Promise.allSettled(pending), ...await Promise.allSettled([rm(this.runtimeRoot, { recursive: true, force: true })]), diff --git a/packages/subprocess/subprocess-local/tests/local.spec.ts b/packages/subprocess/subprocess-local/tests/local.spec.ts index e58c66f7fa..ff83141fab 100644 --- a/packages/subprocess/subprocess-local/tests/local.spec.ts +++ b/packages/subprocess/subprocess-local/tests/local.spec.ts @@ -113,7 +113,7 @@ describe('LocalSubprocessService', () => { expect(terminals.size).toBe(0) }) - it('waits for every terminal cleanup, removes runtime state, and retains rejections', async () => { + it('waits for every terminal cleanup and clears single-shot teardown ownership', async () => { const ctx = new Context() const fiber = await ctx.plugin(LocalSubprocessService) const service = ctx.subprocess @@ -154,7 +154,7 @@ describe('LocalSubprocessService', () => { expect(disposed).toBe(false) finishCleanup() await disposing - expect(terminals).toEqual(new Set([failedTerminal, secondFailedTerminal])) + expect(terminals.size).toBe(0) await expect(stat(runtimeRoot)).rejects.toMatchObject({ code: 'ENOENT' }) expect(disposalErrors).toHaveLength(1) expect(disposalErrors[0]).toMatchObject({ diff --git a/packages/typert/README.i18n.yaml b/packages/typert/README.i18n.yaml index 21cca6cb44..5e93edd997 100644 --- a/packages/typert/README.i18n.yaml +++ b/packages/typert/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/code-runtime/code-runtime-subprocess/README.md -README.md: eff5a6e648eb13c3411c19bca4d05f0898d3ad05 -README.zh.md: 16cf0e0ea6d487e255fc520a8fa1f41f97ee5d30 +README.md: 152921a1ed595676781aad170e3396f7ce613161 +README.zh.md: 3e8223c33c7ce85f342c9a36ab5a24c4d2a6cb94 From 674d23118bfa894e13142d8cc1746dd74c39fba0 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 20:20:51 +0800 Subject: [PATCH 042/108] fix(runtime): close measured portability defects --- packages/lsp/lsp-local/src/host.ts | 5 +-- packages/lsp/lsp-local/src/index.ts | 38 ++++++++-------- packages/lsp/lsp-local/tests/host.spec.ts | 4 +- packages/lsp/lsp-local/tests/provider.spec.ts | 44 +++++++++++++++++++ packages/pty/pty-local/src/session.ts | 2 +- packages/pty/pty-local/tests/session.spec.ts | 6 +-- 6 files changed, 73 insertions(+), 26 deletions(-) diff --git a/packages/lsp/lsp-local/src/host.ts b/packages/lsp/lsp-local/src/host.ts index b8939a2ae4..ffba6a1344 100644 --- a/packages/lsp/lsp-local/src/host.ts +++ b/packages/lsp/lsp-local/src/host.ts @@ -100,15 +100,14 @@ export async function readHostSource( for await (const chunk of stream) { throwIfAborted(signal) bytes += Buffer.byteLength(chunk) - if (bytes > maxDocumentBytes) { - throw new Error(`source "${filePath}" exceeds the ${maxDocumentBytes}-byte limit`) - } + if (bytes > maxDocumentBytes) break chunks.push(chunk) } } catch (error: unknown) { throwIfAborted(signal) throw new Error(`source "${filePath}" could not be read: ${messageOf(error)}`, { cause: error }) } + if (bytes > maxDocumentBytes) throw new Error(`source "${filePath}" exceeds the ${maxDocumentBytes}-byte limit`) throwIfAborted(signal) return { fileUrl: fs.fileUrl(target), diff --git a/packages/lsp/lsp-local/src/index.ts b/packages/lsp/lsp-local/src/index.ts index f3f45959e5..0e9468cba9 100644 --- a/packages/lsp/lsp-local/src/index.ts +++ b/packages/lsp/lsp-local/src/index.ts @@ -129,27 +129,29 @@ export async function apply(ctx: Context, config: Config): Promise { // Resolve every server-local setting before registration so a bad later command or bound cannot // publish an earlier provider. Registry-level mapping conflicts are rolled back below. const providers = await (async () => { + const lookups = entries.map(async ([providerId, rawConfig]) => { + if (providerId.trim() === '') throw new Error('lsp-local: server ids must be non-empty strings') + const resolved = rawConfig as ResolvedServerConfig + validateServerConfig(providerId, resolved) + const executable = await ctx.subprocess.resolveExecutable( + resolved.command, + resolved.env, + setupAbort.signal, + ) + setupAbort.signal.throwIfAborted() + return new LocalLspProvider( + providerId, + ctx.fs, + resolved, + executable, + spec => ctx.subprocess.spawn(spec), + ) + }) try { - return await Promise.all(entries.map(async ([providerId, rawConfig]) => { - if (providerId.trim() === '') throw new Error('lsp-local: server ids must be non-empty strings') - const resolved = rawConfig as ResolvedServerConfig - validateServerConfig(providerId, resolved) - const executable = await ctx.subprocess.resolveExecutable( - resolved.command, - resolved.env, - setupAbort.signal, - ) - setupAbort.signal.throwIfAborted() - return new LocalLspProvider( - providerId, - ctx.fs, - resolved, - executable, - spec => ctx.subprocess.spawn(spec), - ) - })) + return await Promise.all(lookups) } catch (error: unknown) { setupAbort.abort(error) + await Promise.allSettled(lookups) throw error } finally { stopSetupCancellation() diff --git a/packages/lsp/lsp-local/tests/host.spec.ts b/packages/lsp/lsp-local/tests/host.spec.ts index eea3851414..c97defc086 100644 --- a/packages/lsp/lsp-local/tests/host.spec.ts +++ b/packages/lsp/lsp-local/tests/host.spec.ts @@ -158,7 +158,9 @@ describe('readHostSource', () => { it('rejects an oversized source', async () => { await writeFile(join(ws, 'big.ts'), 'x'.repeat(100)) - await expect(readSource('big.ts', 10)).rejects.toThrow(/10-byte limit/) + await expect(readSource('big.ts', 10)).rejects.toMatchObject({ + message: 'source "big.ts" exceeds the 10-byte limit', + }) }) it('counts the complete UTF-8 byte length at the configured boundary', async () => { diff --git a/packages/lsp/lsp-local/tests/provider.spec.ts b/packages/lsp/lsp-local/tests/provider.spec.ts index 8d8a7ad356..88b2b0144f 100644 --- a/packages/lsp/lsp-local/tests/provider.spec.ts +++ b/packages/lsp/lsp-local/tests/provider.spec.ts @@ -195,6 +195,50 @@ describe('lsp-local provider resolution', () => { await ctx.fiber.dispose() }) + it('waits for aborted sibling executable lookups before setup rejects', async () => { + const ctx = new Context() + await ctx.plugin(Lsp) + await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) + const slowStarted = Promise.withResolvers() + const slowAborted = Promise.withResolvers() + const releaseCleanup = Promise.withResolvers() + vi.spyOn(ctx.subprocess, 'resolveExecutable').mockImplementation(async (command, _env, signal) => { + if (signal === undefined) throw new Error('missing setup signal') + if (command === 'slow-lsp') { + return await new Promise((_resolve, reject) => { + const onAbort = (): void => { + slowAborted.resolve(undefined) + void releaseCleanup.promise.then(() => { + reject(signal.reason instanceof Error ? signal.reason : new Error(String(signal.reason))) + }) + } + signal.addEventListener('abort', onAbort, { once: true }) + slowStarted.resolve(undefined) + if (signal.aborted) onAbort() + }) + } + await slowStarted.promise + throw new Error('lookup failed') + }) + + const loading = ctx.plugin(LspLocal, { + servers: { + slow: { command: 'slow-lsp', extensionToLanguage: { '.ts': 'typescript' } }, + failing: { command: 'failing-lsp', extensionToLanguage: { '.js': 'javascript' } }, + }, + }) + await slowAborted.promise + let settled = false + void loading.then(() => { settled = true }, () => { settled = true }) + await new Promise((resolve) => { setImmediate(resolve) }) + expect(settled).toBe(false) + + releaseCleanup.resolve(undefined) + await expect(loading).rejects.toThrow('lookup failed') + await ctx.fiber.dispose() + }) + it('aborts executable resolution when disposed during setup', async () => { const ctx = new Context() await ctx.plugin(Lsp) diff --git a/packages/pty/pty-local/src/session.ts b/packages/pty/pty-local/src/session.ts index f9b35dc665..5517cd902d 100644 --- a/packages/pty/pty-local/src/session.ts +++ b/packages/pty/pty-local/src/session.ts @@ -237,7 +237,7 @@ export class LocalPtySession implements PtyBackendSession { } this.activeDeadlineTimer = setTimeout(() => { if (this.active === operation) { - this.settleActive('timeout', this.activeWrite?.operation === operation) + this.settleActive('timeout', this.activeWrite?.operation === operation || this.interrupting === operation) } }, this.config.timeoutMs) void this.beginSend(operation, request) diff --git a/packages/pty/pty-local/tests/session.spec.ts b/packages/pty/pty-local/tests/session.spec.ts index 14cf252a0e..fbe0aa807b 100644 --- a/packages/pty/pty-local/tests/session.spec.ts +++ b/packages/pty/pty-local/tests/session.spec.ts @@ -365,11 +365,11 @@ describe('LocalPtySession readiness and output', () => { expect(operation.cancel()).toBe(true) terminal.emitData('\x1b]133;D;130\x07dsh> ') - await vi.advanceTimersByTimeAsync(10) + await vi.advanceTimersByTimeAsync(100) + expect((await operation.done).waitReason).toBe('timeout') expect(() => session.startSend({ text: 'successor', submit: true })).toThrow('active send') signalGate.resolve(undefined) - await vi.advanceTimersByTimeAsync(10) - await operation.done + await vi.advanceTimersByTimeAsync(0) expect(inspector.groups).toContainEqual([456, 'SIGINT']) expect(inspector.groups).not.toContainEqual([789, 'SIGINT']) }) From b71dbbe766edc0646a4f7c073d45867bb3248684 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 21:19:27 +0800 Subject: [PATCH 043/108] fix(runtime): close review lifecycle and bound gaps --- ...rtable-execution-world-consumers.i18n.yaml | 4 +- ...7-28-portable-execution-world-consumers.md | 2 +- ...8-portable-execution-world-consumers.zh.md | 2 +- ...26-07-16-persistent-pty-sessions.i18n.yaml | 4 +- .../2026-07-16-persistent-pty-sessions.md | 2 +- .../2026-07-16-persistent-pty-sessions.zh.md | 2 +- docs/config-catalog.md | 2 +- .../code-runtime-worker/src/runtime-host.ts | 1 + packages/lsp/lsp-local/src/index.ts | 16 ++++- .../lsp/lsp-local/tests/lifecycle.spec.ts | 72 +++++++++++++++++++ packages/pty/pty-local/README.i18n.yaml | 4 +- packages/pty/pty-local/README.md | 2 +- packages/pty/pty-local/README.zh.md | 20 +++--- packages/pty/pty-local/src/index.ts | 8 ++- packages/pty/pty-local/tests/index.spec.ts | 18 ++++- 15 files changed, 130 insertions(+), 29 deletions(-) diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml index 7b6a74b7e8..a7eaaa2e3a 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md -2026-07-28-portable-execution-world-consumers.md: d68fc92308079aa68a8f4e1accadde0addb0e5aa -2026-07-28-portable-execution-world-consumers.zh.md: 5341cf1643cabb1043b977e18179dc857e37e121 +2026-07-28-portable-execution-world-consumers.md: 260f38981ea0cd034f506594057483c6ccfa9617 +2026-07-28-portable-execution-world-consumers.zh.md: 0bf7b4dd1a87378b76f6a71343a8cb78766b7b3a diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md index d68fc92308..260f38981e 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md @@ -22,7 +22,7 @@ Generic consumers use that execution world: - `dsh-bash-local` continues to map Bash semantics onto ordinary `ctx.subprocess.spawn()`. - `dsh-lsp-local` reads and contains source through `ctx.fs`, resolves and launches language servers through `ctx.subprocess`, and carries provider-owned file URIs through initialization and result rendering. One provider-lifetime signal aborts filesystem and protocol work during disposal, including workspace lookup before queue ownership; its JSON-RPC, pooling, synchronization, and normalization stay unchanged. -- `dsh-pty-local` maps persistent-shell semantics onto `ctx.subprocess.spawnTerminal()`. The local `node-pty` and process-inspection implementation moves into `dsh-subprocess-local`; another subprocess provider supplies the same primitive. Prompt and silence evidence collected during asynchronous pre-write inspection is discarded when the provider write begins. Cancellation retains the send reservation while an in-flight write settles and then signals the foreground group, so late bytes or the signal cannot target a successor; a rejected write sends no signal. The absolute deadline remains armed throughout cancellation. A signal failure becomes terminal transport failure. Completion of a stale inspection resumes polling for the current send. Close rejects new public signals and delegates complete-session quiescence to the handle's awaited termination operation. +- `dsh-pty-local` maps persistent-shell semantics onto `ctx.subprocess.spawnTerminal()`. The local `node-pty` and process-inspection implementation moves into `dsh-subprocess-local`; another subprocess provider supplies the same primitive. `danger-full-access` needs no `ctx.sandbox`; a confined mode requires a same-world sandbox provider and fails before spawn when none is mounted. Prompt and silence evidence collected during asynchronous pre-write inspection is discarded when the provider write begins. Cancellation retains the send reservation while an in-flight write settles and then signals the foreground group, so late bytes or the signal cannot target a successor; a rejected write sends no signal. The absolute deadline remains armed throughout cancellation. A signal failure becomes terminal transport failure. Completion of a stale inspection resumes polling for the current send. Close rejects new public signals and delegates complete-session quiescence to the handle's awaited termination operation. - `dsh-code-runtime-subprocess` materializes a dependency-free runner through `ctx.fs` and launches it through `ctx.subprocess`, preserving the Code Runtime binding and output contract across local or remote worlds. The fixed runner is adapter-owned infrastructure below `ctx.subprocess.runtimeRoot`, so its write carries an explicit `danger-full-access` policy instead of inheriting the model-facing filesystem mode. It shares host-side worker mechanics through the non-plugin `dsh-code-runtime-worker/runtime-host` subpath instead of copying them. Preparation carries one lifecycle signal through filesystem resolution, materialization, and executable lookup so disposal can abort a stalled provider. The heap-bounded worker rejects oversized binding frames before transfer, each outer hop enforces the same bound before forwarding, and raw subprocess pipes carry newline-delimited UTF-8 JSON without a redundant base64 representation. The launcher publishes an accepted terminal frame before reaping its controller so a descendant that inherits controller pipes cannot suppress completion; the host still awaits process-group quiescence. `dsh-code-runtime-worker` remains a separate implementation. It is the smaller in-process backend and works in single-file distributions that cannot assume an installed Node executable. Remote filesystem/process compositions select `dsh-code-runtime-subprocess`; they do not need a provider-specific Code Runtime package. diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md index 5341cf1643..0bf7b4dd1a 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md @@ -22,7 +22,7 @@ Status: implemented - `dsh-bash-local` 继续把 Bash 语义映射到普通的 `ctx.subprocess.spawn()`。 - `dsh-lsp-local` 通过 `ctx.fs` 读取源文件并验证包含关系,通过 `ctx.subprocess` 解析和启动语言服务器,并让由提供方负责的文件 URI 贯穿初始化与结果渲染。一个提供方生命周期信号会在资源释放期间中止文件系统与协议操作,包括取得队列所有权之前的工作区查找;其 JSON-RPC、池化、同步和规范化保持不变。 -- `dsh-pty-local` 把持久 shell 语义映射到 `ctx.subprocess.spawnTerminal()`。本地 `node-pty` 与进程检查实现移入 `dsh-subprocess-local`;其他进程管理提供方则提供相同原语。提供方开始写入时,系统会丢弃异步写入前检查期间收集的提示符与静默证据。取消会在在途写入结算期间保留发送预留,随后向前台进程组发送信号,因此延迟字节和该信号都无法落到后续发送;写入被拒绝时不会发送信号。绝对截止时间会在整个取消期间保持启用。信号发送失败会成为终结性传输失败。陈旧检查完成后,会针对当前发送恢复轮询。关闭操作会拒绝新的公开信号,并把完整会话的完全停稳委托给句柄上须等待的终止操作。 +- `dsh-pty-local` 把持久 shell 语义映射到 `ctx.subprocess.spawnTerminal()`。本地 `node-pty` 与进程检查实现移入 `dsh-subprocess-local`;其他进程管理提供方则提供相同原语。`danger-full-access` 不需要 `ctx.sandbox`;受限模式要求同一执行世界中存在沙箱提供方,未挂载时会在 spawn 前失败。提供方开始写入时,系统会丢弃异步写入前检查期间收集的提示符与静默证据。取消会在在途写入结算期间保留发送预留,随后向前台进程组发送信号,因此延迟字节和该信号都无法落到后续发送;写入被拒绝时不会发送信号。绝对截止时间会在整个取消期间保持启用。信号发送失败会成为终结性传输失败。陈旧检查完成后,会针对当前发送恢复轮询。关闭操作会拒绝新的公开信号,并把完整会话的完全停稳委托给句柄上须等待的终止操作。 - `dsh-code-runtime-subprocess` 通过 `ctx.fs` 物化无依赖 runner,并通过 `ctx.subprocess` 启动它,从而在本地或远程执行世界中保留代码运行时的绑定与输出契约。固定 runner 是位于 `ctx.subprocess.runtimeRoot` 下的适配器自有基础设施,因此其写入携带显式 `danger-full-access` 策略,而不继承面向模型的文件系统模式。它通过非插件子路径 `dsh-code-runtime-worker/runtime-host` 共享宿主侧 worker 机制,而不是复制这些机制。准备阶段让同一个生命周期信号贯穿文件系统解析、物化和可执行文件查找,使资源释放能够中止停滞的提供方操作。受堆上限约束的 worker 会在传输前拒绝过大的绑定帧;每个外层转发环节都会在转发前执行相同的上限检查;原始子进程管道承载以换行符分隔的 UTF-8 JSON,无需冗余的 base64 表示;launcher 会在回收 controller 前发布已接纳的终态帧,使继承 controller 管道的后代进程无法阻止完成;宿主仍会等待进程组完全停稳。 `dsh-code-runtime-worker` 仍是独立实现。它是较小的进程内后端,可用于无法假定已安装 Node 可执行文件的单文件分发。远程文件系统/进程组合选择 `dsh-code-runtime-subprocess`;它们不需要提供方专用的代码运行时包。 diff --git a/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.i18n.yaml b/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.i18n.yaml index 2694664183..72944d7433 100644 --- a/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.i18n.yaml +++ b/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md -2026-07-16-persistent-pty-sessions.md: f94dd57622c555e195564cc2eee9b3228841d5cc -2026-07-16-persistent-pty-sessions.zh.md: 6228571a6496e9d4cb031a2df958b80dee102fd3 +2026-07-16-persistent-pty-sessions.md: 88296a9318b2c398fead382153bd2c652dbd9e68 +2026-07-16-persistent-pty-sessions.zh.md: 40f26c67fe046ad292f05919501600bd2e146c62 diff --git a/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md b/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md index f94dd57622..88296a9318 100644 --- a/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md +++ b/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md @@ -41,7 +41,7 @@ Agent-scope disposal closes registrations first, then awaits quiescent teardown A registered `shell` backend constrains how a terminal starts; it does not constrain commands typed after startup. `dsh-pty-local` therefore applies two protections before spawning: - It supplies only terminal-specific environment overrides; the mounted subprocess provider applies the shared credential-shaped-name scrub before merging them. -- It requires `ctx.sandbox` and the shared `ctx.sandboxPolicy`. At spawn, the backend resolves the owner's effective session mode over the deployment default and wraps the shell argv once; that mode and workspace root remain the process boundary for the PTY lifetime. A write that would change the effective `sandbox/mode` is rejected before commit while the owner has any open PTY or unpublished spawn, with an instruction to wait for creation to settle and close those sessions first; same-effective-mode writes remain valid. The pending reservation spans backend setup through publication, so there is no race in which a wider terminal appears after a downgrade. `danger-full-access` is the existing explicit unconfined choice rather than a PTY-specific bypass. +- It requires the shared `ctx.sandboxPolicy`. At spawn, the backend resolves the owner's effective session mode over the deployment default; `danger-full-access` starts the shell directly, while confined modes require a same-world `ctx.sandbox` provider and wrap the shell argv once. That mode and workspace root remain the process boundary for the PTY lifetime. A write that would change the effective `sandbox/mode` is rejected before commit while the owner has any open PTY or unpublished spawn, with an instruction to wait for creation to settle and close those sessions first; same-effective-mode writes remain valid. The pending reservation spans backend setup through publication, so there is no race in which a wider terminal appears after a downgrade. `danger-full-access` is the existing explicit unconfined choice rather than a PTY-specific bypass. Sandboxing confines local process effects but does not make arbitrary shell input safe: network calls and other external side effects remain governed by deployment policy. Tool descriptions state that PTY sessions are less auditable than one-shot tools and should be used only when persistence or interactive stdin is necessary. diff --git a/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.zh.md b/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.zh.md index 6228571a64..40f26c67fe 100644 --- a/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.zh.md +++ b/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.zh.md @@ -41,7 +41,7 @@ agent scope dispose 时先关闭注册,再等待全部所属 PTY 静默退出 注册的 `shell` 后端只约束终端如何启动,不约束启动后输入的命令。因此 `dsh-pty-local` 在 spawn 前应用两层保护: - 它只提供终端专用的环境覆盖;挂载的子进程提供方先清除名称形似凭据的环境变量,再合并这些覆盖。 -- 它要求 `ctx.sandbox` 和共享的 `ctx.sandboxPolicy`。后端在 spawn 时,以部署默认值为底折叠 owner 的有效 session mode,并只包装一次 shell argv;该 mode 与 workspace root 在 PTY 的整个生命周期中充当进程边界。只要 owner 有任何已打开的 PTY 或尚未发布的 spawn,任何会改变生效 `sandbox/mode` 的写入都会在提交前被拒绝,并提示先等待创建操作结算,再关闭这些会话;不会改变生效模式的写入仍然有效。这项进行中的预留从后端 setup 持续到发布完成,因此不存在降级后又出现权限更宽的终端这一竞态。`danger-full-access` 是现有的显式无约束选择,不另设 PTY 私有 bypass。 +- 它要求共享的 `ctx.sandboxPolicy`。后端在 spawn 时,以部署默认值为底折叠 owner 的有效 session mode;`danger-full-access` 会直接启动 shell,受限模式则要求同一执行世界中存在 `ctx.sandbox` 提供方,并只包装一次 shell argv。该 mode 与 workspace root 在 PTY 的整个生命周期中充当进程边界。只要 owner 有任何已打开的 PTY 或尚未发布的 spawn,任何会改变生效 `sandbox/mode` 的写入都会在提交前被拒绝,并提示先等待创建操作结算,再关闭这些会话;不会改变生效模式的写入仍然有效。这项进行中的预留从后端 setup 持续到发布完成,因此不存在降级后又出现权限更宽的终端这一竞态。`danger-full-access` 是现有的显式无约束选择,不另设 PTY 私有 bypass。 沙箱限制本地进程副作用,但不会让任意 shell 输入自动安全:网络调用和其他外部副作用仍由部署策略治理。工具描述会说明 PTY 会话比一次性工具更难审计,只应在确实需要持久状态或交互式 stdin 时使用。 diff --git a/docs/config-catalog.md b/docs/config-catalog.md index b70f468960..0514cdd969 100644 --- a/docs/config-catalog.md +++ b/docs/config-catalog.md @@ -1072,7 +1072,7 @@ Source: [`packages/plan/plan-mode/src/index.ts:69`](../packages/plan/plan-mode/s ## `@deepseek-ai/dsh-pty-local` -Requires: `pty` · `sandbox` · `sandboxPolicy` · `subprocess` +Requires: `pty` · `sandboxPolicy` · `subprocess` ```ts config-catalog /** Public plugin configuration. */ diff --git a/packages/code-runtime/code-runtime-worker/src/runtime-host.ts b/packages/code-runtime/code-runtime-worker/src/runtime-host.ts index 474a058e9f..d2bb1082b8 100644 --- a/packages/code-runtime/code-runtime-worker/src/runtime-host.ts +++ b/packages/code-runtime/code-runtime-worker/src/runtime-host.ts @@ -223,4 +223,5 @@ export class RuntimeOutputLedger { } export { decodeWorkerJson, encodeWorkerJson, snapshotCodeJsonValue } from './worker-json.ts' +export { jsonValueBytesUpTo } from './output-json.ts' export type { WorkerJsonWire } from './worker-json.ts' diff --git a/packages/lsp/lsp-local/src/index.ts b/packages/lsp/lsp-local/src/index.ts index 0e9468cba9..fc7e50787f 100644 --- a/packages/lsp/lsp-local/src/index.ts +++ b/packages/lsp/lsp-local/src/index.ts @@ -106,6 +106,16 @@ export const Config: z = z.object({ servers: z.dict(LspLocalServerConfig).required(), }) +/** Propagate teardown failures only after every sibling has settled. */ +function throwTeardownFailures(results: readonly PromiseSettledResult[], message: string): void { + const failures: unknown[] = [] + for (const result of results) { + if (result.status === 'rejected') failures.push(result.reason) + } + if (failures.length === 1) throw failures[0] + if (failures.length > 1) throw new AggregateError(failures, message) +} + /** * Register the configured stdio LSP providers. Resolves every executable at load (after credential * scrubbing) before publishing any provider; each process launches lazily on its first matching @@ -169,7 +179,8 @@ export async function apply(ctx: Context, config: Config): Promise { return async () => { // Remove every route before process teardown so no new query can enter a draining provider. for (const dispose of disposers.reverse()) dispose() - await Promise.all(providers.map(provider => provider.disposeAll())) + const results = await Promise.allSettled(providers.map(provider => provider.disposeAll())) + throwTeardownFailures(results, 'lsp-local provider teardown failed') } }, 'lsp-local.registerProviders') } @@ -346,12 +357,13 @@ class LocalLspProvider implements LspProvider { const draining = [...this.queues.values()] const resolving = [...this.workspaceLookups] this.instances.clear() - await Promise.all([ + const results = await Promise.allSettled([ ...live.map(instance => instance.dispose()), ...draining, ...resolving, ]) this.queues.clear() this.workspaceLookups.clear() + throwTeardownFailures(results, 'lsp-local instance teardown failed') } } diff --git a/packages/lsp/lsp-local/tests/lifecycle.spec.ts b/packages/lsp/lsp-local/tests/lifecycle.spec.ts index 63aea8742c..7a30232e22 100644 --- a/packages/lsp/lsp-local/tests/lifecycle.spec.ts +++ b/packages/lsp/lsp-local/tests/lifecycle.spec.ts @@ -350,6 +350,78 @@ describe('lsp-local end to end over a fake server', () => { expect(signal.aborted).toBe(true) }) + it('waits for every owned teardown before aggregating instance failures', async () => { + let provider: LspProvider | undefined + const ctx = await mount({ LSP_FAKE_DEF: 'null' }, {}, (registered) => { provider = registered }) + if (provider === undefined) throw new Error('expected lsp-local to register a provider') + const internals = provider as unknown as { + readonly instances: Map }> + readonly queues: Map> + readonly workspaceLookups: Set> + disposeAll(): Promise + } + const firstFailure = new Error('first instance cleanup failed') + const secondFailure = new Error('second instance cleanup failed') + const release = Promise.withResolvers() + internals.instances.set('first', { dispose: async () => { throw firstFailure } }) + internals.instances.set('second', { dispose: async () => { throw secondFailure } }) + internals.queues.set('pending', release.promise) + internals.workspaceLookups.add(Promise.resolve()) + + let settled = false + const disposing = internals.disposeAll().finally(() => { settled = true }) + await new Promise(resolve => setImmediate(resolve)) + expect(settled).toBe(false) + release.resolve(undefined) + await expect(disposing).rejects.toMatchObject({ + errors: [firstFailure, secondFailure], + message: 'lsp-local instance teardown failed', + }) + expect(internals.instances.size).toBe(0) + expect(internals.queues.size).toBe(0) + expect(internals.workspaceLookups.size).toBe(0) + await ctx.fiber.dispose() + }) + + it('waits for every provider before reporting plugin teardown failure', async () => { + const ctx = new Context() + const disposalErrors: unknown[] = [] + ctx.logger.error = ((error: unknown) => { disposalErrors.push(error) }) as typeof ctx.logger.error + await ctx.plugin(Lsp) + await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) + const providers: LspProvider[] = [] + const register = ctx.lsp.registerProvider.bind(ctx.lsp) + const registrationSpy = vi.spyOn(ctx.lsp, 'registerProvider').mockImplementation((provider) => { + providers.push(provider) + return register(provider) + }) + const fiber = await ctx.plugin(LspLocal, { + servers: { + first: fakeServer(), + second: fakeServer({}, { extensionToLanguage: { '.js': 'javascript' } }), + }, + }) + registrationSpy.mockRestore() + expect(providers).toHaveLength(2) + const failure = new Error('provider cleanup failed') + const release = Promise.withResolvers() + const first = providers[0] as LspProvider & { disposeAll(): Promise } + const second = providers[1] as LspProvider & { disposeAll(): Promise } + first.disposeAll = async () => { throw failure } + second.disposeAll = async () => { await release.promise } + + let disposed = false + const disposing = fiber.dispose().then(() => { disposed = true }) + await new Promise(resolve => setImmediate(resolve)) + expect(disposed).toBe(false) + expect(disposalErrors).toEqual([]) + release.resolve(undefined) + await disposing + expect(disposalErrors).toEqual([failure]) + await ctx.fiber.dispose() + }) + it('runs distinct workspaces in parallel instances', async () => { const ws2 = join(root, 'ws2') await mkdir(ws2) diff --git a/packages/pty/pty-local/README.i18n.yaml b/packages/pty/pty-local/README.i18n.yaml index c2a0f9b647..2689103443 100644 --- a/packages/pty/pty-local/README.i18n.yaml +++ b/packages/pty/pty-local/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/pty/pty-local/README.md -README.md: c4af7bc8293689d64c58eebab2606d4f9b52fd2f -README.zh.md: 21075ccbf53d173a54220b58b384f01cfe50ced1 +README.md: 92a3d7be68ada6f38ab3c4ca5bdd2622ebefc8ee +README.zh.md: 016043f7842df8bb963cdc5eea015666c9ce6941 diff --git a/packages/pty/pty-local/README.md b/packages/pty/pty-local/README.md index c4af7bc829..92a3d7be68 100644 --- a/packages/pty/pty-local/README.md +++ b/packages/pty/pty-local/README.md @@ -6,7 +6,7 @@ Persistent shell backend for `ctx.pty` over `ctx.subprocess.spawnTerminal`. It s ## Plugin (`pty-local`) -The plugin injects `pty`, `sandbox`, `sandboxPolicy`, and `subprocess`, then registers the configured backend type (`shell`). `danger-full-access` starts the shell directly; confined modes wrap the exact shell argv through `ctx.sandbox`. The effective session mode is resolved at spawn. A change to a different effective mode is rejected before its `sandbox/mode` event commits while that owner has an open PTY or a spawn in progress; the fence is attached to the exact owner and therefore outlives a provider reload that retains existing sessions. Wait for creation to settle and close the sessions before changing modes, so a terminal opened with wider access cannot survive a downgrade. +The plugin injects `pty`, `sandboxPolicy`, and `subprocess`, then registers the configured backend type (`shell`). `danger-full-access` starts the shell directly without requiring a sandbox provider; confined modes require a same-world `ctx.sandbox` and wrap the exact shell argv through it, failing before spawn when none is mounted. The effective session mode is resolved at spawn. A change to a different effective mode is rejected before its `sandbox/mode` event commits while that owner has an open PTY or a spawn in progress; the fence is attached to the exact owner and therefore outlives a provider reload that retains existing sessions. Wait for creation to settle and close the sessions before changing modes, so a terminal opened with wider access cannot survive a downgrade. Readiness combines a foreground-verified private bash prompt marker, provider-reported foreground stdin-wait facts, silence fallback, and absolute timeout. A marker is not ready until the printable tail after the latest owned marker exactly equals the controlled `PS1`, including when the OSC marker and prompt are split across data callbacks; echoed input or output following an earlier prompt therefore cannot settle the current send. Prompt and silence evidence collected before the provider write, including while pre-write foreground inspection is pending, is discarded at the write boundary. When bash prints the marker before the terminal provider publishes its return to the foreground process group, polling retains the candidate for `handoffGraceMs` past the ordinary silence bound so a coincident handoff can win. An interactive child that inherits `PROMPT_COMMAND` therefore cannot suppress inferred-idle readiness until the absolute timeout. Unknown foreground state is never a positive exact-idle signal. A foreground group's stdin wait that existed before a send is likewise not post-write readiness: the same group must be observed outside that wait before a later wait can settle the send, while a changed foreground group is new evidence. During unpublished startup, a fallback requires observed output; zero-output silence cannot publish an empty session, and timeout rejects the spawn. Cancellation closes the unpublished shell and rejects with the caller's exact abort reason; `PtyBackendCleanupError` separately preserves a cleanup failure. The caller's signal is forwarded for terminal allocation and readiness initialization; after publication the handle owns its lifetime. Incomplete terminal-control sequences are bounded by `maxReadBytes` and discarded through their terminator after crossing that limit; malformed UTF-8 terminal output uses replacement characters, and a trailing carriage return is carried across callbacks so split CRLF becomes one newline. diff --git a/packages/pty/pty-local/README.zh.md b/packages/pty/pty-local/README.zh.md index 21075ccbf5..016043f784 100644 --- a/packages/pty/pty-local/README.zh.md +++ b/packages/pty/pty-local/README.zh.md @@ -2,15 +2,15 @@ [English](README.md) | 中文 -这是基于 `ctx.subprocess.spawnTerminal` 的 `ctx.pty` 持久 shell 后端。它在共享 `ctx.sandboxPolicy` 下启动交互式 shell,保留有界的逐行输出并检测就绪状态;进程管理提供方负责 PTY 分配、环境清理、前台进程组、信号发送和完整终端会话清理。因此,同一个 PTY 后端可以与本地或远程执行世界提供方组合。 +这是一个基于 `ctx.subprocess.spawnTerminal`、为 `ctx.pty` 提供的持久 shell 后端。它在共享 `ctx.sandboxPolicy` 下启动交互式 shell,保留有界的逐行输出并检测就绪状态;进程管理提供方则负责 PTY 分配、环境清理、前台进程组、信号发送和完整终端会话清理。因此,同一个 PTY 后端可以与本地或远程执行世界提供方组合。 ## 插件(`pty-local`) -该插件注入 `pty`、`sandbox`、`sandboxPolicy` 和 `subprocess`,然后注册所配置的后端类型(`shell`)。`danger-full-access` 会直接启动 shell;受限模式则通过 `ctx.sandbox` 包装确切的 shell argv。系统在 spawn 时解析会话的实际模式。当某个所有者存在开放的 PTY 或正在进行 spawn 时,如果配置变更会得到不同的实际模式,系统会在对应 `sandbox/mode` 事件提交前拒绝该变更。该限制绑定到确切所有者,因此即使提供方重新加载并保留现有会话,它仍然有效。更改模式前,请等待创建结算并关闭会话,避免以更宽权限打开的终端在权限降级后继续存在。 +该插件注入 `pty`、`sandboxPolicy` 和 `subprocess`,然后注册所配置的后端类型(`shell`)。`danger-full-access` 无需沙箱提供方即可直接启动 shell;受限模式要求同一执行世界中存在 `ctx.sandbox`,并通过它包装确切的 shell argv,未挂载时会在 spawn 前失败。系统在 spawn 时解析会话的实际模式。当某个所有者存在开放的 PTY 或正在进行 spawn 时,如果配置变更会得到不同的实际模式,系统会在对应 `sandbox/mode` 事件提交前拒绝该变更。该限制绑定到确切所有者,因此即使提供方重新加载并保留现有会话,它仍然有效。更改模式前,请等待创建完成并关闭会话,避免以更宽权限打开的终端在权限降级后继续存在。 -就绪检测结合以下机制:由前台状态验证的私有 bash 提示符标记、提供方报告的前台 stdin 等待事实、静默回退和绝对超时。只有最近一个自有标记之后的可打印尾部与受控 `PS1` 完全相等时,系统才会把标记视为就绪;即使 OSC 标记和提示符被拆到多个数据回调中也是如此。因此,如果回显的输入或输出跟在先前提示符之后,该提示符无法使当前 send 完成。系统会在写入边界丢弃提供方写入前收集的提示符与静默证据,包括写入前的前台检查尚未完成时收集的证据。如果 bash 在终端提供方发布其重新取得前台进程组的状态前打印标记,轮询会在普通静默上限之后再保留该候选状态 `handoffGraceMs`,使恰好同时发生的前台交接有机会胜出。因此,继承 `PROMPT_COMMAND` 的交互式子进程无法持续压制推断空闲就绪,最多只能延续到绝对超时。未知的前台状态绝不会作为精确空闲的正向信号。同样,一次 send 之前就已存在的前台进程组 stdin 等待并不代表写入后就绪:必须先观察到同一进程组脱离该等待,之后再次进入等待才能使该次 send 完成;前台进程组发生变化则构成新的证据。尚未发布的启动过程中,回退路径要求已经观察到输出;零输出静默不能发布空会话,超时则拒绝 spawn。取消操作会关闭尚未发布的 shell,并以调用方提供的确切中止原因拒绝;`PtyBackendCleanupError` 会单独保留清理失败。调用方信号会转发给终端分配和就绪初始化;句柄一经发布,便负责自身生命周期。未完成的终端控制序列受 `maxReadBytes` 限制;超过上限后,系统会丢弃内容直到其终止符。格式错误的 UTF-8 终端输出使用替换字符;末尾的回车会跨回调保留,使拆分的 CRLF 合并为一个换行。 +就绪检测结合以下机制:由前台状态验证的私有 bash 提示符标记、提供方报告的前台 stdin 等待事实、静默回退和绝对超时。只有最新自有标记之后的可打印尾部与受控 `PS1` 完全相等,标记才算就绪;即使 OSC 标记和提示符被拆到多个数据回调中也一样。因此,较早提示符之后的回显输入或输出无法使当前 send 完成。提供方写入前收集的提示符与静默证据,包括写入前前台检查仍在等待时收集的证据,都会在写入边界丢弃。如果 bash 在终端提供方发布其重新取得前台进程组的状态前打印标记,轮询会在普通静默上限之后再保留该候选状态 `handoffGraceMs`,使恰好同时发生的前台交接有机会胜出。因此,继承 `PROMPT_COMMAND` 的交互式子进程无法一直抑制推断空闲就绪直至绝对超时。未知的前台状态绝不会作为精确空闲的正向信号。同样,一次 send 之前就已存在的前台进程组 stdin 等待并不代表写入后就绪:必须先观察到同一进程组脱离该等待,之后再次进入等待才能使该次 send 完成;前台进程组发生变化则构成新的证据。尚未发布的启动过程中,回退路径要求已经观察到输出;零输出静默不能发布空会话,超时则拒绝 spawn。取消操作会关闭尚未发布的 shell,并以调用方提供的确切中止原因拒绝;`PtyBackendCleanupError` 会单独保留清理失败。调用方的 signal 会转发给终端分配与就绪初始化;发布后,句柄负责其生命周期。未完成的终端控制序列受 `maxReadBytes` 限制;超过上限后,系统会丢弃内容直到其终止符。格式错误的 UTF-8 终端输出使用替换字符;末尾的回车会跨回调保留,使拆分的 CRLF 合并为一个换行。 -取消发送会先把排队输入标记为已取消,再请求终端句柄向当前前台进程组发送真正的 `SIGINT`;如果异步的写入前检查随后才结算,也无法执行该输入。如果提供方写入已在途,信号发送会等待写入结算;写入被拒绝时不会发送信号。被取消的发送会保留其槽位,直至写入和前台信号发送都结算,因此后续发送既不会收到延迟字节,也不会成为该信号的目标。取消等待期间,绝对截止时间仍保持启用。信号发送失败属于终结性传输失败,并会使当前发送被拒绝。取消绝不会通过写入 `\x03` 模拟中断,因此原始模式程序仍可取消。关闭操作会拒绝新的公开信号、停止就绪轮询,并等待句柄执行由提供方负责的完整会话终止,之后才将当前发送以 `session_exit` 结算。 +取消发送时,系统会先把排队输入标记为已取消,再要求终端句柄向当前前台进程组发送真正的 `SIGINT`;异步写入前检查即使随后结算,也无法执行该输入。如果提供方写入已在途,信号发送会等待其结算;写入被拒绝时不会发送信号。已取消的 send 会保留其位置,直到写入与前台信号发送都结算,因此后继 send 不会收到延迟字节或该信号。取消等待期间,绝对 deadline 仍保持启用。信号发送失败是终端传输失败,会拒绝活跃 send。取消绝不会通过写入 `\x03` 模拟中断,因此,即使程序运行在 raw 模式下,也仍可取消。关闭操作会拒绝新的公开信号、停止就绪轮询,并等待由句柄提供方负责的完整会话终止,然后才把活跃 send 结算为 `session_exit`。 ## 模型体验 @@ -22,15 +22,15 @@ #### Token 影响 -消费方返回有界的后端输出前没有影响。此包不会把保留的 PTY scrollback 放入模型历史。 +消费方返回有界的后端输出前没有影响。此包(package)不会把保留的 PTY scrollback 放入模型历史。 #### KV Cache 影响 -不会直接失效;提示词、schema 与追加结果由消费方负责。 +不会直接使 KV Cache 失效;提示词、schema 与追加结果由消费方负责。 -## 已知限制与暂缓工作 +## 已知限制与暂缓事项 - 输出按行规范化;不支持全屏备用缓冲区交互。 -- 精确 stdin 等待检测取决于挂载的进程管理提供方;无法证明该事实的提供方使用提示符标记和静默/超时就绪机制。 -- 清理保证遵循 `SubprocessTerminalHandle`;提供方特有缺口属于该实现的契约,而非此 PTY 消费方。 -- 会话无法跨 harness 进程退出保留。 +- 精确 stdin 等待检测取决于已挂载的进程管理提供方;无法证明该状态的提供方使用提示符标记和静默/超时就绪机制。 +- 清理保证以 `SubprocessTerminalHandle` 的保证为准;提供方特定的缺口属于该实现的契约,而非这个 PTY 消费方。 +- harness 进程退出后,会话无法继续存在。 diff --git a/packages/pty/pty-local/src/index.ts b/packages/pty/pty-local/src/index.ts index a8d99cb243..1fa3357002 100644 --- a/packages/pty/pty-local/src/index.ts +++ b/packages/pty/pty-local/src/index.ts @@ -22,7 +22,7 @@ export type { Config as PtyLocalConfig } from './config.ts' /** Cordis plugin name. */ export const name = 'pty-local' /** Required services: PTY registry, shared confinement policy, and process substrate. */ -export const inject = ['pty', 'sandbox', 'sandboxPolicy', 'subprocess'] +export const inject = ['pty', 'sandboxPolicy', 'subprocess'] interface SandboxModeFenceState { pty: Context['pty'] @@ -72,7 +72,11 @@ function spawnArgv(ctx: Context, config: ResolvedConfig, spec: PtyBackendSpawnSp const argv = [config.shellPath, ...config.shellArgs] const mode: SandboxMode = effectiveSandboxMode(spec.owner.session.events) ?? ctx.sandboxPolicy.defaultMode if (mode === 'danger-full-access') return argv - return ctx.sandbox.confine(argv, { + const sandbox = ctx.get('sandbox') + if (sandbox === undefined) { + throw new Error(`pty-local: sandbox mode "${mode}" requires a ctx.sandbox provider in the execution world`) + } + return sandbox.confine(argv, { mode: mode, workspaceRoot: ctx.sandboxPolicy.workspaceRoot, }).argv diff --git a/packages/pty/pty-local/tests/index.spec.ts b/packages/pty/pty-local/tests/index.spec.ts index 48c8272106..9206dd89ef 100644 --- a/packages/pty/pty-local/tests/index.spec.ts +++ b/packages/pty/pty-local/tests/index.spec.ts @@ -120,7 +120,6 @@ describe('LocalPtyBackend startup rollback', () => { it('closes failed startup and aggregates cleanup failure', async () => { const ctx = new Context() - await ctx.plugin(EmptySandbox) await ctx.plugin(SandboxPolicyService, { mode: 'danger-full-access', workspaceRoot: '/tmp' }) const spawnTerminal = async (): Promise => terminalHandle() @@ -186,6 +185,20 @@ describe('LocalPtyBackend startup rollback', () => { expect(initialized).toHaveBeenCalledWith(undefined) }) + it('rejects a confined spawn without a sandbox provider', async () => { + const confinedCtx = new Context() + await confinedCtx.plugin(SandboxPolicyService, { mode: 'workspace-write', workspaceRoot: '/workspace' }) + const confined = new LocalPtyBackend( + confinedCtx, + config(), + async () => { throw new Error('terminal spawn must not run') }, + () => stubLocalSession(), + ) + await expect(confined.spawn(spec(agent(confinedCtx)))).rejects.toThrow( + 'sandbox mode "workspace-write" requires a ctx.sandbox provider in the execution world', + ) + }) + it('forwards terminal allocation cancellation directly', async () => { const ctx = new Context() await ctx.plugin(EmptySandbox) @@ -266,7 +279,7 @@ describe('pty-local plugin shape', () => { const loader = Object.create(Loader.prototype) as Loader const unwrapped = loader.unwrapExports(ptyLocal) as Record expect(unwrapped.name).toBe('pty-local') - expect(unwrapped.inject).toEqual(['pty', 'sandbox', 'sandboxPolicy', 'subprocess']) + expect(unwrapped.inject).toEqual(['pty', 'sandboxPolicy', 'subprocess']) expect(unwrapped.Config).toBeDefined() }) @@ -274,7 +287,6 @@ describe('pty-local plugin shape', () => { const ctx = new Context() await ctx.plugin(AgentRegistry) await ctx.plugin(PtyService) - await ctx.plugin(EmptySandbox) await ctx.plugin(SandboxPolicyService, { mode: 'danger-full-access', workspaceRoot: '/tmp' }) await ctx.plugin(StubSubprocessService) const fiber = await ctx.plugin(ptyLocal, config()) From ff86795b15576048529c7ccff4d00c38adbdb015 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 23:11:49 +0800 Subject: [PATCH 044/108] fix(pty): close provider cancellation races --- ...rtable-execution-world-consumers.i18n.yaml | 4 +- ...7-28-portable-execution-world-consumers.md | 4 +- ...8-portable-execution-world-consumers.zh.md | 4 +- .../core-data-structures/subprocess.i18n.yaml | 4 +- docs/core-data-structures/subprocess.md | 2 +- docs/core-data-structures/subprocess.zh.md | 2 +- packages/pty/pty-local/src/index.ts | 18 ++++++- packages/pty/pty-local/src/session.ts | 5 +- packages/pty/pty-local/tests/index.spec.ts | 26 +++++++++ packages/pty/pty-local/tests/session.spec.ts | 53 ++++++++++++++++++- packages/subprocess/README.i18n.yaml | 4 +- packages/subprocess/README.md | 2 +- packages/subprocess/README.zh.md | 6 +-- .../subprocess/subprocess/README.i18n.yaml | 4 +- packages/subprocess/subprocess/README.md | 2 +- packages/subprocess/subprocess/README.zh.md | 24 ++++----- packages/subprocess/subprocess/src/types.ts | 3 +- 17 files changed, 131 insertions(+), 36 deletions(-) diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml index a7eaaa2e3a..683a095229 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md -2026-07-28-portable-execution-world-consumers.md: 260f38981ea0cd034f506594057483c6ccfa9617 -2026-07-28-portable-execution-world-consumers.zh.md: 0bf7b4dd1a87378b76f6a71343a8cb78766b7b3a +2026-07-28-portable-execution-world-consumers.md: 03b274688e15661845c326afdd105b342dd6a513 +2026-07-28-portable-execution-world-consumers.zh.md: 1e3d8ae8e46d26f4bea47293104c52dbf33098d4 diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md index 260f38981e..03b274688e 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md @@ -16,13 +16,13 @@ Ordinary pipes do not cover one requirement. A persistent terminal needs PTY all The filesystem interface owns the path facts that another capability needs without exposing its opaque target identity: a canonical process path, canonical `file:` URI, and containment. Existing whole and streaming text operations remain filesystem-owned; protocol consumers enforce their own retention limits while consuming the stream. -The subprocess interface owns the process coordinates and primitives: canonical cwd, private runtime storage, executable lookup, ordinary raw or collected process spawning, and `spawnTerminal()`. The terminal operation is one deep primitive whose handle owns text I/O, foreground groups, signalling, and one awaited TERM-to-KILL operation that settles in-flight handle calls and reaches whole-session quiescence. Its signal cancels allocation only; the published handle owns its lifetime. Prompt detection, idle inference, scrollback, sandbox policy, and owner lifecycle remain in the PTY consumer. +The subprocess interface owns the process coordinates and primitives: canonical cwd, private runtime storage, executable lookup, ordinary raw or collected process spawning, and `spawnTerminal()`. The terminal operation is one deep primitive whose handle owns text I/O, foreground groups, signalling, and one awaited TERM-to-KILL operation that settles in-flight handle calls and reaches quiescence for every session member the provider can still observe. Its signal cancels allocation only; the published handle owns its lifetime. Prompt detection, idle inference, scrollback, sandbox policy, and owner lifecycle remain in the PTY consumer. Generic consumers use that execution world: - `dsh-bash-local` continues to map Bash semantics onto ordinary `ctx.subprocess.spawn()`. - `dsh-lsp-local` reads and contains source through `ctx.fs`, resolves and launches language servers through `ctx.subprocess`, and carries provider-owned file URIs through initialization and result rendering. One provider-lifetime signal aborts filesystem and protocol work during disposal, including workspace lookup before queue ownership; its JSON-RPC, pooling, synchronization, and normalization stay unchanged. -- `dsh-pty-local` maps persistent-shell semantics onto `ctx.subprocess.spawnTerminal()`. The local `node-pty` and process-inspection implementation moves into `dsh-subprocess-local`; another subprocess provider supplies the same primitive. `danger-full-access` needs no `ctx.sandbox`; a confined mode requires a same-world sandbox provider and fails before spawn when none is mounted. Prompt and silence evidence collected during asynchronous pre-write inspection is discarded when the provider write begins. Cancellation retains the send reservation while an in-flight write settles and then signals the foreground group, so late bytes or the signal cannot target a successor; a rejected write sends no signal. The absolute deadline remains armed throughout cancellation. A signal failure becomes terminal transport failure. Completion of a stale inspection resumes polling for the current send. Close rejects new public signals and delegates complete-session quiescence to the handle's awaited termination operation. +- `dsh-pty-local` maps persistent-shell semantics onto `ctx.subprocess.spawnTerminal()`. The local `node-pty` and process-inspection implementation moves into `dsh-subprocess-local`; another subprocess provider supplies the same primitive. `danger-full-access` needs no `ctx.sandbox`; a confined mode requires a same-world sandbox provider and fails before spawn when none is mounted. Prompt and silence evidence collected during asynchronous pre-write inspection is discarded when the provider write begins. Cancellation retains the send reservation while an in-flight write settles and then signals the foreground group, so late bytes or the signal cannot target a successor; an in-flight readiness poll cannot release that reservation, and a rejected write sends no signal. The absolute deadline remains armed throughout cancellation. A signal failure becomes terminal transport failure. Completion of a stale inspection resumes polling for the current send. Startup cancellation begins terminal rollback without waiting for a stalled readiness or signalling call. Close rejects new public signals and delegates provider-observable session quiescence to the handle's awaited termination operation. - `dsh-code-runtime-subprocess` materializes a dependency-free runner through `ctx.fs` and launches it through `ctx.subprocess`, preserving the Code Runtime binding and output contract across local or remote worlds. The fixed runner is adapter-owned infrastructure below `ctx.subprocess.runtimeRoot`, so its write carries an explicit `danger-full-access` policy instead of inheriting the model-facing filesystem mode. It shares host-side worker mechanics through the non-plugin `dsh-code-runtime-worker/runtime-host` subpath instead of copying them. Preparation carries one lifecycle signal through filesystem resolution, materialization, and executable lookup so disposal can abort a stalled provider. The heap-bounded worker rejects oversized binding frames before transfer, each outer hop enforces the same bound before forwarding, and raw subprocess pipes carry newline-delimited UTF-8 JSON without a redundant base64 representation. The launcher publishes an accepted terminal frame before reaping its controller so a descendant that inherits controller pipes cannot suppress completion; the host still awaits process-group quiescence. `dsh-code-runtime-worker` remains a separate implementation. It is the smaller in-process backend and works in single-file distributions that cannot assume an installed Node executable. Remote filesystem/process compositions select `dsh-code-runtime-subprocess`; they do not need a provider-specific Code Runtime package. diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md index 0bf7b4dd1a..1e3d8ae8e4 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md @@ -16,13 +16,13 @@ Status: implemented 文件系统接口负责其他能力需要的路径事实,同时不公开其不透明目标身份:规范化进程路径、规范化 `file:` URI 和包含关系。现有完整文本与流式文本操作仍归文件系统负责;协议消费方在消费流时执行各自的保留上限。 -进程管理接口负责进程运行坐标与原语:规范化 cwd、私有运行时存储、可执行文件查找、以原始或收集模式 spawn 普通进程,以及 `spawnTerminal()`。终端操作是一项深层原语,其句柄负责文本 I/O、前台进程组、信号发送,以及一项须等待的 TERM→KILL 操作;该操作会结算所有在途句柄调用,并使整个会话完全停稳。其信号只取消分配;句柄一经发布,便负责自身生命周期。提示符检测、空闲推断、scrollback、沙箱策略和所有者生命周期仍由 PTY 消费方负责。 +进程管理接口负责进程运行坐标与原语:规范化 cwd、私有运行时存储、可执行文件查找、以原始或收集模式 spawn 普通进程,以及 `spawnTerminal()`。终端操作是一项深层原语,其句柄负责文本 I/O、前台进程组、信号发送,以及一项须等待的 TERM→KILL 操作;该操作会结算所有在途句柄调用,并使提供方仍可观察到的每个会话成员完全停稳。其信号只取消分配;句柄一经发布,便负责自身生命周期。提示符检测、空闲推断、scrollback、沙箱策略和所有者生命周期仍由 PTY 消费方负责。 通用消费方使用该执行世界: - `dsh-bash-local` 继续把 Bash 语义映射到普通的 `ctx.subprocess.spawn()`。 - `dsh-lsp-local` 通过 `ctx.fs` 读取源文件并验证包含关系,通过 `ctx.subprocess` 解析和启动语言服务器,并让由提供方负责的文件 URI 贯穿初始化与结果渲染。一个提供方生命周期信号会在资源释放期间中止文件系统与协议操作,包括取得队列所有权之前的工作区查找;其 JSON-RPC、池化、同步和规范化保持不变。 -- `dsh-pty-local` 把持久 shell 语义映射到 `ctx.subprocess.spawnTerminal()`。本地 `node-pty` 与进程检查实现移入 `dsh-subprocess-local`;其他进程管理提供方则提供相同原语。`danger-full-access` 不需要 `ctx.sandbox`;受限模式要求同一执行世界中存在沙箱提供方,未挂载时会在 spawn 前失败。提供方开始写入时,系统会丢弃异步写入前检查期间收集的提示符与静默证据。取消会在在途写入结算期间保留发送预留,随后向前台进程组发送信号,因此延迟字节和该信号都无法落到后续发送;写入被拒绝时不会发送信号。绝对截止时间会在整个取消期间保持启用。信号发送失败会成为终结性传输失败。陈旧检查完成后,会针对当前发送恢复轮询。关闭操作会拒绝新的公开信号,并把完整会话的完全停稳委托给句柄上须等待的终止操作。 +- `dsh-pty-local` 把持久 shell 语义映射到 `ctx.subprocess.spawnTerminal()`。本地 `node-pty` 与进程检查实现移入 `dsh-subprocess-local`;其他进程管理提供方则提供相同原语。`danger-full-access` 不需要 `ctx.sandbox`;受限模式要求同一执行世界中存在沙箱提供方,未挂载时会在 spawn 前失败。提供方开始写入时,系统会丢弃异步写入前检查期间收集的提示符与静默证据。取消会在在途写入结算期间保留发送预留,随后向前台进程组发送信号,因此延迟字节和该信号都无法落到后续发送;在途就绪检查无法释放该预留,写入被拒绝时也不会发送信号。绝对截止时间会在整个取消期间保持启用。信号发送失败会成为终结性传输失败。陈旧检查完成后,会针对当前发送恢复轮询。启动取消会立即开始终端回滚,而不等待停滞的就绪检查或信号发送调用。关闭操作会拒绝新的公开信号,并把提供方可观察会话成员的完全停稳委托给句柄上须等待的终止操作。 - `dsh-code-runtime-subprocess` 通过 `ctx.fs` 物化无依赖 runner,并通过 `ctx.subprocess` 启动它,从而在本地或远程执行世界中保留代码运行时的绑定与输出契约。固定 runner 是位于 `ctx.subprocess.runtimeRoot` 下的适配器自有基础设施,因此其写入携带显式 `danger-full-access` 策略,而不继承面向模型的文件系统模式。它通过非插件子路径 `dsh-code-runtime-worker/runtime-host` 共享宿主侧 worker 机制,而不是复制这些机制。准备阶段让同一个生命周期信号贯穿文件系统解析、物化和可执行文件查找,使资源释放能够中止停滞的提供方操作。受堆上限约束的 worker 会在传输前拒绝过大的绑定帧;每个外层转发环节都会在转发前执行相同的上限检查;原始子进程管道承载以换行符分隔的 UTF-8 JSON,无需冗余的 base64 表示;launcher 会在回收 controller 前发布已接纳的终态帧,使继承 controller 管道的后代进程无法阻止完成;宿主仍会等待进程组完全停稳。 `dsh-code-runtime-worker` 仍是独立实现。它是较小的进程内后端,可用于无法假定已安装 Node 可执行文件的单文件分发。远程文件系统/进程组合选择 `dsh-code-runtime-subprocess`;它们不需要提供方专用的代码运行时包。 diff --git a/docs/core-data-structures/subprocess.i18n.yaml b/docs/core-data-structures/subprocess.i18n.yaml index 7cba3857bb..f3b308b0be 100644 --- a/docs/core-data-structures/subprocess.i18n.yaml +++ b/docs/core-data-structures/subprocess.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write docs/core-data-structures/subprocess.md -subprocess.md: 89c75aea3393a000612e3b8f87d263c7b4fa7c60 -subprocess.zh.md: 83fb44f4750ba31db1f4336c0c293eb9e236be22 +subprocess.md: 95e9454a75112cf98e1ed583f970c50fe90aac37 +subprocess.zh.md: 58127d662fa599d19f632449d42b8aa3259886d6 diff --git a/docs/core-data-structures/subprocess.md b/docs/core-data-structures/subprocess.md index d6e0105c59..935a7d60ac 100644 --- a/docs/core-data-structures/subprocess.md +++ b/docs/core-data-structures/subprocess.md @@ -240,7 +240,7 @@ interface SubprocessOutcome { ## Terminal-process primitive -`spawnTerminal(spec)` is the non-pipe process primitive. The provider allocates the controlling terminal and owns UTF-8 text transport, foreground-process-group inspection and signalling, and one awaited TERM-to-KILL operation that reaches whole-session quiescence. The PTY backend remains responsible for prompt detection, readiness inference, scrollback, sandbox policy, and persistent-session ownership; ordinary `spawn()` cannot reconstruct controlling-terminal semantics. +`spawnTerminal(spec)` is the non-pipe process primitive. The provider allocates the controlling terminal and owns UTF-8 text transport, foreground-process-group inspection and signalling, and one awaited TERM-to-KILL operation that reaches quiescence for every session member the provider can still observe; providers document substrate-specific observability limits. The PTY backend remains responsible for prompt detection, readiness inference, scrollback, sandbox policy, and persistent-session ownership; ordinary `spawn()` cannot reconstruct controlling-terminal semantics. The terminal spec fully specifies argv, cwd, environment overrides, dimensions, cleanup grace, and optional allocation cancellation. Its handle exposes `pid`, ordered output, `done`, `write`, `inspectForeground`, `signalForeground`, and awaited `terminate`; the exact public shapes are generated into the [`ctx.subprocess` service catalog](../cordis-catalog/services.md#ctxsubprocess--subprocessservice-abstract-seam). diff --git a/docs/core-data-structures/subprocess.zh.md b/docs/core-data-structures/subprocess.zh.md index 83fb44f475..58127d662f 100644 --- a/docs/core-data-structures/subprocess.zh.md +++ b/docs/core-data-structures/subprocess.zh.md @@ -240,7 +240,7 @@ interface SubprocessOutcome { ## 终端进程原语 -`spawnTerminal(spec)` 是非管道进程原语。提供方分配控制终端,并负责 UTF-8 文本传输、前台进程组检查与信号发送,以及一项须等待的 TERM→KILL 操作;该操作会使整个会话完全停稳。PTY 后端仍负责提示符检测、就绪推断、scrollback、沙箱策略和持久会话所有权;普通 `spawn()` 无法重建控制终端语义。 +`spawnTerminal(spec)` 是非管道进程原语。提供方分配控制终端,并负责 UTF-8 文本传输、前台进程组检查与信号发送,以及一项须等待的 TERM→KILL 操作;该操作会使提供方仍可观察到的每个会话成员完全停稳,提供方则会记录执行基底特有的可观察性限制。PTY 后端仍负责提示符检测、就绪推断、scrollback、沙箱策略和持久会话所有权;普通 `spawn()` 无法重建控制终端语义。 终端 spec 完全指定 argv、cwd、环境覆盖、尺寸、清理宽限期与可选的分配取消。其句柄公开 `pid`、有序输出、`done`、`write`、`inspectForeground`、`signalForeground` 和须等待的 `terminate`;确切的公共形状生成到 [`ctx.subprocess` 服务目录](../cordis-catalog/services.md#ctxsubprocess--subprocessservice-abstract-seam)中。 diff --git a/packages/pty/pty-local/src/index.ts b/packages/pty/pty-local/src/index.ts index 1fa3357002..f91db5d386 100644 --- a/packages/pty/pty-local/src/index.ts +++ b/packages/pty/pty-local/src/index.ts @@ -82,6 +82,22 @@ function spawnArgv(ctx: Context, config: ResolvedConfig, spec: PtyBackendSpawnSp }).argv } +async function initializeSession(session: LocalPtySession, signal?: AbortSignal): Promise { + if (signal === undefined) { + await session.initialize(signal) + return + } + const aborted = Promise.withResolvers() + const onAbort = (): void => { aborted.reject(signal.reason) } + signal.addEventListener('abort', onAbort, { once: true }) + try { + signal.throwIfAborted() + await Promise.race([session.initialize(signal), aborted.promise]) + } finally { + signal.removeEventListener('abort', onAbort) + } +} + /** Local shell backend registered under the configured type. */ export class LocalPtyBackend implements PtyBackend { readonly type: string @@ -116,7 +132,7 @@ export class LocalPtyBackend implements PtyBackend { }) const session = this.createSession(terminal, this.config) try { - await session.initialize(spec.signal) + await initializeSession(session, spec.signal) return session } catch (error) { try { diff --git a/packages/pty/pty-local/src/session.ts b/packages/pty/pty-local/src/session.ts index 5517cd902d..ebd7e076cf 100644 --- a/packages/pty/pty-local/src/session.ts +++ b/packages/pty/pty-local/src/session.ts @@ -6,6 +6,7 @@ import type { SubprocessTerminalForeground, SubprocessTerminalHandle, } from '@deepseek-ai/dsh-subprocess' +import { PtyError } from '@deepseek-ai/dsh-pty' import type { PtyBackendSession, PtyReadRequest, @@ -219,7 +220,7 @@ export class LocalPtySession implements PtyBackendSession { startSend(request: PtySendRequest): PtySendOperation { if (this.closing) throw new Error('PTY session is closing') if (this.statusValue.kind === 'exited') throw new Error('PTY session has exited') - if (this.active !== undefined) throw new Error('PTY session already has an active send') + if (this.active !== undefined) throw new PtyError('PTY session already has an active send or draining provider operation', 'SEND_ACTIVE') if (request.signal?.aborted === true) throw new Error('PTY send aborted before write') const operation = new LocalSendOperation( @@ -247,7 +248,7 @@ export class LocalPtySession implements PtyBackendSession { private async beginSend(operation: LocalSendOperation, request: PtySendRequest): Promise { try { const foreground = await this.terminal.inspectForeground() - if (this.active !== operation || this.closing) return + if (this.active !== operation || this.closing || this.interrupting === operation) return operation.setInitialForeground(foreground) const input = `${request.text}${request.submit ? '\r' : ''}` if (input.length > 0 && !operation.cancelRequested) { diff --git a/packages/pty/pty-local/tests/index.spec.ts b/packages/pty/pty-local/tests/index.spec.ts index 9206dd89ef..f59cd3ab5a 100644 --- a/packages/pty/pty-local/tests/index.spec.ts +++ b/packages/pty/pty-local/tests/index.spec.ts @@ -143,6 +143,32 @@ describe('LocalPtyBackend startup rollback', () => { } satisfies Partial)) }) + it('starts startup rollback when cancellation wins a stalled initialization', async () => { + const ctx = new Context() + await ctx.plugin(SandboxPolicyService, { mode: 'danger-full-access', workspaceRoot: '/tmp' }) + const initialization = Promise.withResolvers() + const initializationStarted = Promise.withResolvers() + const close = vi.fn<() => Promise>().mockResolvedValue(undefined) + const session = { + initialize: () => { + initializationStarted.resolve(undefined) + return initialization.promise + }, + close, + } as unknown as LocalPtySession + const backend = new LocalPtyBackend(ctx, config(), async () => terminalHandle(), () => session) + const controller = new AbortController() + const reason = new Error('cancel stalled startup') + + const spawning = backend.spawn(spec(agent(ctx), controller.signal)) + await initializationStarted.promise + controller.abort(reason) + + await expect(spawning).rejects.toBe(reason) + expect(close).toHaveBeenCalledWith('PTY startup failed') + initialization.resolve(undefined) + }) + it('wraps confined argv, scrubs the environment, and returns initialized sessions', async () => { const ctx = new Context() await ctx.plugin(RecordingSandbox) diff --git a/packages/pty/pty-local/tests/session.spec.ts b/packages/pty/pty-local/tests/session.spec.ts index fbe0aa807b..8d11f8e6f4 100644 --- a/packages/pty/pty-local/tests/session.spec.ts +++ b/packages/pty/pty-local/tests/session.spec.ts @@ -8,6 +8,7 @@ import type { SubprocessTerminalHandle, SubprocessTerminalSignal, } from '@deepseek-ai/dsh-subprocess' +import { PtyError } from '@deepseek-ai/dsh-pty' import type { ProcessIdentity, ProcessInspector, @@ -374,6 +375,54 @@ describe('LocalPtySession readiness and output', () => { expect(inspector.groups).not.toContainEqual([789, 'SIGINT']) }) + it('does not let an in-flight readiness inspection release a canceled send before signalling settles', async () => { + vi.useFakeTimers() + const terminal = new FakeTerminal() + const inspector = new FakeInspector() + const session = makeSession(terminal, inspector, config()) + await initialize(session, terminal) + + const readiness = Promise.withResolvers<{ processGroupId: number; inputWaiting: boolean }>() + let inspections = 0 + terminal.inspectForeground = async () => { + inspections += 1 + if (inspections === 1) return { processGroupId: 456, inputWaiting: false } + if (inspections === 2) return await readiness.promise + return { processGroupId: 456, inputWaiting: true } + } + const signalling = Promise.withResolvers() + const signalled = Promise.withResolvers() + terminal.signalForeground = async (signal) => { + await signalling.promise + const foreground = await terminal.inspectForeground() + if (foreground === undefined) throw new Error('cannot resolve foreground') + inspector.signalGroup(foreground.processGroupId, signal) + signalled.resolve(foreground.processGroupId) + return foreground.processGroupId + } + + const operation = session.startSend({ text: 'first', submit: true }) + await Promise.resolve() + await Promise.resolve() + await vi.advanceTimersByTimeAsync(10) + expect(inspections).toBe(2) + expect(operation.cancel()).toBe(true) + let settled = false + void operation.done.then(() => { settled = true }) + + readiness.resolve({ processGroupId: 456, inputWaiting: true }) + await Promise.resolve() + await Promise.resolve() + expect(settled).toBe(false) + expect(() => session.startSend({ text: 'successor', submit: true })).toThrow(PtyError) + + signalling.resolve(undefined) + expect(await signalled.promise).toBe(456) + expect(inspector.groups).toContainEqual([456, 'SIGINT']) + await session.close('test complete') + expect((await operation.done).waitReason).toBe('session_exit') + }) + it('signals only after an in-flight provider write lands', async () => { vi.useFakeTimers() const terminal = new FakeTerminal() @@ -438,7 +487,9 @@ describe('LocalPtySession readiness and output', () => { await vi.advanceTimersByTimeAsync(100) expect((await operation.done).waitReason).toBe('timeout') - expect(() => session.startSend({ text: 'must wait', submit: true })).toThrow('active send') + expect(() => session.startSend({ text: 'must wait', submit: true })).toThrow(expect.objectContaining({ + code: 'SEND_ACTIVE', + })) writeGate.resolve(undefined) await vi.advanceTimersByTimeAsync(0) diff --git a/packages/subprocess/README.i18n.yaml b/packages/subprocess/README.i18n.yaml index 256d4ad2d9..f6de20e259 100644 --- a/packages/subprocess/README.i18n.yaml +++ b/packages/subprocess/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/subprocess/README.md -README.md: c4bb1da1a172b05afa63834db4e5b1fa974baabb -README.zh.md: f68bc1c720eeb5282bb9c94c951524a021e38b2f +README.md: b4aaf78454a51af4d3f7ea9f1aef450fde8d729d +README.zh.md: c5013ce19bacb582b6c8c93e0ea0a73ec776ee57 diff --git a/packages/subprocess/README.md b/packages/subprocess/README.md index c4bb1da1a1..b4aaf78454 100644 --- a/packages/subprocess/README.md +++ b/packages/subprocess/README.md @@ -2,7 +2,7 @@ English | [中文](README.zh.md) -The shared process substrate for one execution world: canonical cwd/runtime storage, executable lookup, fully-specified managed child-process trees with raw or collected stdio, and one deep terminal-process primitive that owns PTY allocation, foreground groups, and complete session cleanup. Command defaulting, shell semantics, deadlines, protocol framing, readiness, and presentation stay with consumers — the [bash executors](../bash/README.md), [LSP host](../lsp/README.md), [PTY shell backend](../pty/README.md), [subprocess code runtime](../code-runtime/code-runtime-subprocess/README.md), and [ACP subagent backend](../subagent/README.md). See the [subprocess seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md). +The shared process substrate for one execution world: canonical cwd/runtime storage, executable lookup, fully-specified managed child-process trees with raw or collected stdio, and one deep terminal-process primitive that owns PTY allocation, foreground groups, and provider-observable session cleanup. Command defaulting, shell semantics, deadlines, protocol framing, readiness, and presentation stay with consumers — the [bash executors](../bash/README.md), [LSP host](../lsp/README.md), [PTY shell backend](../pty/README.md), [subprocess code runtime](../code-runtime/code-runtime-subprocess/README.md), and [ACP subagent backend](../subagent/README.md). See the [subprocess seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md). | Package | ctx key | Role | |---|---|---| diff --git a/packages/subprocess/README.zh.md b/packages/subprocess/README.zh.md index f68bc1c720..c5013ce19b 100644 --- a/packages/subprocess/README.zh.md +++ b/packages/subprocess/README.zh.md @@ -1,12 +1,12 @@ -# subprocess/:进程管理能力家族 +# subprocess/:子进程能力家族 [English](README.md) | 中文 -同一执行世界中的共享进程基底:规范化 cwd/运行时存储、可执行文件查找、采用原始或收集式 stdio 的完全显式受管子进程树,以及一项负责 PTY 分配、前台进程组和完整会话清理的深层终端进程原语。命令默认值补全、shell 语义、deadline、协议分帧、就绪检测与呈现留在消费方:[bash 执行器](../bash/README.md)、[LSP 主机](../lsp/README.md)、[PTY shell 后端](../pty/README.md)、[基于进程管理的 Code Runtime](../code-runtime/code-runtime-subprocess/README.md)与 [ACP(Agent Client Protocol)subagent 后端](../subagent/README.md)。参见[进程管理器 seam Agent Note(agent 决策记录)](../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md)。 +这里集中提供一个执行世界的共享进程基底:规范化 cwd/运行时存储、可执行文件查找、具有原始或收集式 stdio 的完整指定受管子进程树,以及一项深层终端进程原语,负责 PTY 分配、前台进程组和提供方仍可观察到的会话成员清理。命令默认值补全、shell 语义、时限、协议分帧、就绪状态与呈现留在消费方:[bash 执行器](../bash/README.md)、[LSP 主机](../lsp/README.md)、[PTY shell 后端](../pty/README.md)、[子进程代码运行时](../code-runtime/code-runtime-subprocess/README.md)与 [ACP(Agent Client Protocol)subagent 后端](../subagent/README.md)。参见[subprocess seam Agent Note(agent 决策记录)](../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md)。 | 包(package) | ctx 键 | 角色 | |---|---|---| | [`subprocess`](subprocess/README.md)(`@deepseek-ai/dsh-subprocess`) | `ctx.subprocess` | seam 本体:执行世界坐标与可执行文件查找、普通受管 spawn、终端进程原语、句柄生命周期,以及共享的环境/输出词汇 | | [`subprocess-local`](subprocess-local/README.md)(`@deepseek-ai/dsh-subprocess-local`) | 无 | 本地实现:detached 进程树、有界收集/spill、`node-pty`、前台/会话检查、进程树信号发送、运行时存储,以及先终止再等待退出的资源释放 | -服务拥有跨消费方重载的进程存续期;消费方拥有一个进程的含义(一条 bash 命令、未来的非 shell 运行器)以及塑造它的每一项默认值。 +即使消费方重载,进程生命周期仍由服务负责管理;消费方负责定义进程的含义(一条 bash 命令、未来的非 shell 运行器),以及决定塑造该进程的每一项默认值。 diff --git a/packages/subprocess/subprocess/README.i18n.yaml b/packages/subprocess/subprocess/README.i18n.yaml index f8678464a9..92887cc247 100644 --- a/packages/subprocess/subprocess/README.i18n.yaml +++ b/packages/subprocess/subprocess/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/subprocess/subprocess/README.md -README.md: 2fca5cdd08f3cbd36d8ef492226c91681b5568d0 -README.zh.md: 5830d0fc394160bc557e24646cc76d4bcfead13c +README.md: 12a88a86c3cd7bcde2858bdbd874c99533100e09 +README.zh.md: f4c265a0f55c4e35e4557516593dd0da236164c5 diff --git a/packages/subprocess/subprocess/README.md b/packages/subprocess/subprocess/README.md index 2fca5cdd08..12a88a86c3 100644 --- a/packages/subprocess/subprocess/README.md +++ b/packages/subprocess/subprocess/README.md @@ -11,7 +11,7 @@ The subprocess seam (`ctx.subprocess`) is the process half of one execution worl - The spec is fully explicit — argv, cwd, per-stream stdio dispositions, grace — because deployment-varying defaults belong to the calling seam's config, not to a hidden subprocess-service default (the `dsh-bash` request/spec split is the owning template). `argv` is never shell-interpreted; a consumer that wants a shell passes `['bash', '-c', command]` itself. - Stdio is Node-shaped per stream: `'pipe'` hands the caller the raw stream for its own protocol framing (LSP JSON-RPC, ACP ndjson), `'inherit'` passes the parent descriptor through for diagnostics, and collect mode (`{ maxBytes, spill? }`) buffers a bounded tail with an optional full-stream spill file. Collect readers take whole-stream byte offsets and never consume, so independent readers cannot steal one another's deltas; a read whose offset slid out of the in-memory tail is `lossy` and points at the spill file when one exists. Collected output stays readable after settlement. - Termination is tree-scoped on every platform (POSIX detached groups with direct-child fallback; Windows `taskkill /T`): `terminate()` — the only termination verb — escalates SIGTERM→grace→SIGKILL (idempotent, driven by the spec's abort signal too, a no-op once the tree is gone), and `waitForExit(signal?)` observes whole-tree liveness so a consumer-owned teardown ladder holds each tier on real quiescence — the manager reacts but never classifies why (callers own deadlines, teardown ladders, and cause classification). -- `spawnTerminal(spec)` is the only non-pipe primitive. Its handle owns a real PTY, UTF-8 text I/O, foreground-process-group inspection/signalling, and one awaited `terminate()` operation that reaches whole-session quiescence and settles in-flight handle calls. The spec signal cancels allocation only; the published handle owns its lifetime. The output stream ends after queued output when the top-level process exits, and a live transport failure rejects `done`. These operations remain one substrate primitive because ordinary pipes cannot allocate a controlling terminal or clean the complete terminal session; readiness, scrollback, and owner policy remain in the PTY consumer. +- `spawnTerminal(spec)` is the only non-pipe primitive. Its handle owns a real PTY, UTF-8 text I/O, foreground-process-group inspection/signalling, and one awaited `terminate()` operation that reaches quiescence for every session member the provider can still observe and settles in-flight handle calls; providers document substrate-specific observability limits. The spec signal cancels allocation only; the published handle owns its lifetime. The output stream ends after queued output when the top-level process exits, and a live transport failure rejects `done`. These operations remain one substrate primitive because ordinary pipes cannot allocate a controlling terminal or clean terminal-session members; readiness, scrollback, and owner policy remain in the PTY consumer. - `scrubbedParentEnv()` / `SENSITIVE_ENV_PATTERN` are the one shared scrub definition: ambient credential-shaped and `DSH_*` names are dropped, and explicit `env` merges after the scrub. The local ordinary and terminal spawns both apply it; SDK-managed transports that own their spawn may import it directly. - Disposal of the service terminates all still-running managed processes and awaits their exit. diff --git a/packages/subprocess/subprocess/README.zh.md b/packages/subprocess/subprocess/README.zh.md index 5830d0fc39..f4c265a0f5 100644 --- a/packages/subprocess/subprocess/README.zh.md +++ b/packages/subprocess/subprocess/README.zh.md @@ -2,30 +2,30 @@ [English](README.md) | 中文 -进程管理器 seam(`ctx.subprocess`)是同一执行世界中的进程侧。抽象的 `SubprocessService` 公开其规范化 `cwd`、私有 `runtimeRoot`、可执行文件查找、普通受管 `spawn` 和一项终端进程原语;其词汇涵盖原始/收集式 stdio、进程与终端句柄、退出事实、进程树/会话清理,以及受管的 `DSH_*` 环境命名空间。本地实现位于 [`dsh-subprocess-local`](../subprocess-local/README.md)。 +子进程 seam(`ctx.subprocess`)是一个执行世界的进程部分。抽象的 `SubprocessService` 公开其规范化 `cwd`、私有 `runtimeRoot`、可执行文件查找、普通受管 `spawn` 和一项终端进程原语;其词汇涵盖原始/收集式 stdio、进程与终端句柄、退出事实、进程树/会话清理,以及受管的 `DSH_*` 环境命名空间。本地实现位于 [`dsh-subprocess-local`](../subprocess-local/README.md)。 ## 契约 -- `spawn(spec)` 立即返回一个实时句柄;`done` 在进程关闭时以退出事实 resolve(`SubprocessOutcome` 不携带输出,也不携带原因分类),仅在 spawn 层面失败时 reject。 -- `cwd` 与 `runtimeRoot` 是提供方执行世界中的绝对路径。消费方在 `runtimeRoot` 下物化私有辅助程序,绝不使用仅宿主可见的临时目录。`resolveExecutable(command, env?, signal?)` 验证绝对命令,或根据该执行世界清理后的 PATH 加显式覆盖来解析裸名称。 -- spec 完全显式(argv、cwd、按流划分的 stdio 处置方式(disposition)、宽限期),因为随部署变化的默认值属于调用方 seam 的配置,而不属于某个隐藏的进程管理器默认值(`dsh-bash` 的 request/spec 拆分是这条规则的所属模板)。`argv` 绝不经过 shell 解释;需要 shell 的消费方自行传入 `['bash', '-c', command]`。 -- stdio 按流采用 Node 形状:`'pipe'` 把原始流交给调用方做自己的协议分帧(LSP 的 JSON-RPC、ACP(Agent Client Protocol)的 ndjson),`'inherit'` 直通父进程描述符以承载诊断输出,收集模式(collect)`{ maxBytes, spill? }` 则缓冲一段有界尾部,外加可选的完整流 spill 文件。收集模式的读取器接受全流字节偏移量且从不消费,因此独立的读取器不会抢走彼此的增量;偏移量滑出内存尾部窗口的读取标记为 `lossy`,并在 spill 文件存在时指向它。收集到的输出在结算后仍可读取。 +- `spawn(spec)` 立即返回一个活动句柄;`done` 在进程关闭时以退出事实 resolve(`SubprocessOutcome` 不携带输出,也不携带原因分类),仅在 spawn 层面失败时 reject。 +- `cwd` 和 `runtimeRoot` 是提供方执行世界中的绝对路径。消费方在 `runtimeRoot` 之下物化私有辅助程序,绝不使用仅宿主可见的临时目录。`resolveExecutable(command, env?, signal?)` 验证绝对命令,或根据该执行世界清理后的 PATH 加显式覆盖来解析裸名称。 +- spec 完全显式(argv、cwd、按流划分的 stdio 处置方式(disposition)、宽限期),因为随部署变化的默认值属于调用方 seam 的配置,而不属于某个隐藏的子进程默认值(`dsh-bash` 的 request/spec 拆分是这条规则的所属模板)。`argv` 绝不经过 shell 解释;需要 shell 的消费方自行传入 `['bash', '-c', command]`。 +- stdio 按流采用 Node 风格:`'pipe'` 把原始流交给调用方做自己的协议分帧(LSP 的 JSON-RPC、ACP(Agent Client Protocol)的 ndjson),`'inherit'` 直通父进程描述符以承载诊断输出,收集模式(collect)`{ maxBytes, spill? }` 则缓冲一段有界尾部,外加可选的完整流 spill 文件。收集模式的读取器接受全流字节偏移量且从不消费,因此独立的读取器不会抢走彼此的增量;偏移量滑出内存尾部窗口的读取标记为 `lossy`,并在 spill 文件存在时指向它。收集到的输出在结算后仍可读取。 - 终止在每个平台上都以进程树为范围(POSIX 用 detached 进程组并以直接子进程回退;Windows 用 `taskkill /T`):`terminate()`(唯一的终止动词)执行 SIGTERM→宽限期→SIGKILL 升级(幂等,也由 spec 的 abort 信号驱动,进程树消亡后为空操作);`waitForExit(signal?)` 观察整棵进程树的存活状态,使消费方自有的拆卸阶梯能在真正完全停稳后才进入下一层。管理器只响应中止,但绝不判定原因(deadline、拆卸阶梯与原因分类归调用方所有)。 -- `spawnTerminal(spec)` 是唯一的非管道原语。其句柄负责真实 PTY、UTF-8 文本 I/O、前台进程组检查/信号发送,以及一项须等待的 `terminate()` 操作;该操作会使整个会话完全停稳,并让所有在途句柄调用结算。spec 信号只取消分配;句柄一经发布,便负责自身生命周期。顶层进程退出后,输出流会在排完队列中的输出后结束;存活期间的传输故障会拒绝 `done`。这些操作仍属于一项基底原语,因为普通管道无法分配控制终端,也无法清理完整的终端会话;就绪检测、scrollback 与所有者策略仍归 PTY 消费方所有。 -- `scrubbedParentEnv()` / `SENSITIVE_ENV_PATTERN` 是唯一一份共享的凭据清除定义:环境中形似凭据的名称与 `DSH_*` 名称都会被丢弃,显式 `env` 在清除之后合并。本地普通 spawn 与终端 spawn 都应用这一定义;自行拥有 spawn 的 SDK 管理传输层可以直接导入它。 +- `spawnTerminal(spec)` 是唯一的非管道原语。其句柄负责真实 PTY、UTF-8 文本 I/O、前台进程组检查/信号发送,以及一项须等待的 `terminate()` 操作;该操作会使提供方仍可观察到的每个会话成员完全停稳,并结算在途句柄调用;提供方会记录执行基底特有的可观察性限制。spec 信号只取消分配;句柄一经发布,便负责自身生命周期。顶层进程退出时,输出流在已排队输出之后结束;存活传输失败会拒绝 `done`。这些操作保留为一项执行基底原语,因为普通管道无法分配控制终端或清理终端会话成员;就绪状态、scrollback 和所有者策略仍归 PTY 消费方所有。 +- `scrubbedParentEnv()` / `SENSITIVE_ENV_PATTERN` 是唯一一份共享的环境清理定义:环境中形似凭据的名称与 `DSH_*` 名称都会被丢弃,显式 `env` 在清除之后合并。本地的普通 spawn 与终端 spawn 都应用该定义;拥有自身 spawn 的 SDK 管理传输可直接导入它。 - 服务自身的 dispose(资源释放)会终止所有仍在运行的受管进程并等待其退出。 -参见[进程管理器数据结构目录](../../../docs/core-data-structures/subprocess.md)与 [seam Agent Note(agent 决策记录)](../../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md)。 +参见[子进程数据结构目录](../../../docs/core-data-structures/subprocess.md)与[seam Agent Note(agent 决策记录)](../../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md)。 ## 模型体验 -通过消费方 seam 间接影响(目前是 `dsh-tool-bash` 背后的 bash 执行器家族);进程输出与生命周期面向模型的全部渲染归消费方所有。 +通过消费方 seam 间接影响(目前是 `dsh-tool-bash` 背后的 bash 执行器家族);进程输出和生命周期的全部面向模型渲染均由消费方负责。 #### KV Cache 影响 -不会直接失效;请求前缀变更由具名消费方负责。 +不会直接导致 KV Cache 失效;请求前缀变更由上述消费方负责。 ## 已知限制与暂缓事项 -- **由 SDK 管理的 spawn 仍在服务之外**:自行拥有内部 spawn 的 SDK 传输层无法经该服务路由这次调用;它仍可导入 `scrubbedParentEnv`,使环境策略保持单一来源。 -- **拆卸阶梯归消费方所有**:该 seam 只提供信号动词与进程树存活等待,不提供现成的停稳序列;每个进程外消费方自行编码其子进程的配合形状(ACP 后端以 stdin EOF 打头的阶梯是仓库内模板)。 +- **由 SDK 管理的 spawn 仍在服务之外**:拥有内部 spawn 的 SDK 传输无法把该调用路由到本服务;它仍可导入 `scrubbedParentEnv`,使环境策略保持单一来源。 +- **拆卸阶梯归消费方所有**:该 seam 只提供信号动词与进程树存活等待,不提供现成的停稳序列;每个进程外消费方自行编码其子进程的配合方式(ACP 后端以 stdin EOF 打头的阶梯是仓库内模板)。 diff --git a/packages/subprocess/subprocess/src/types.ts b/packages/subprocess/subprocess/src/types.ts index 6b606fd870..9801bf2f49 100644 --- a/packages/subprocess/subprocess/src/types.ts +++ b/packages/subprocess/subprocess/src/types.ts @@ -252,8 +252,9 @@ export interface SubprocessTerminalHandle { */ signalForeground(signal: SubprocessTerminalSignal): Promise /** - * Idempotently terminate the complete terminal session and await whole-session quiescence. + * Idempotently terminate every terminal-session member the provider can still observe and await quiescence. * After settlement, no write, inspection, or signal call remains in flight. + * Providers document substrate-specific observability limits. */ terminate(): Promise } From dde55d5afecb75e8cd3f07f04e8f79e8199b53ba Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Thu, 30 Jul 2026 02:04:11 +0800 Subject: [PATCH 045/108] docs(agent-notes): consolidate subprocess decisions --- ...26-subprocess-consumer-migration.i18n.yaml | 6 --- ...026-07-26-subprocess-consumer-migration.md | 38 ------------------- ...-07-26-subprocess-consumer-migration.zh.md | 38 ------------------- .../2026-07-26-subprocess-seam.i18n.yaml | 4 +- .../2026-07-26-subprocess-seam.md | 16 +++++--- .../2026-07-26-subprocess-seam.zh.md | 16 +++++--- 6 files changed, 24 insertions(+), 94 deletions(-) delete mode 100644 .agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.i18n.yaml delete mode 100644 .agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.md delete mode 100644 .agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.zh.md diff --git a/.agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.i18n.yaml deleted file mode 100644 index 469c3f9a24..0000000000 --- a/.agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.i18n.yaml +++ /dev/null @@ -1,6 +0,0 @@ -# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each -# side as of the last confirmed-consistent state. Both languages carry equal authority; -# after editing either side, bring the other along and re-record with: -# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.md -2026-07-26-subprocess-consumer-migration.md: abc7c8504595641b2b821df5ddbd4645c3a15336 -2026-07-26-subprocess-consumer-migration.zh.md: 500dcfe36f63315156b1a3cc48670eaf1c204b20 diff --git a/.agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.md b/.agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.md deleted file mode 100644 index abc7c85045..0000000000 --- a/.agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.md +++ /dev/null @@ -1,38 +0,0 @@ -# Agent Note: The subprocess seam goes Node-shaped and every eligible spawner rides it - -Status: implemented - -English | [中文](2026-07-26-subprocess-consumer-migration.zh.md) - -## Problem - -The [subprocess seam](2026-07-26-subprocess-seam.md) shipped shaped for exactly one consumer family: batch-collected stdout/stderr, batch stdin, a single escalating `kill()`. That was deliberate scope control, and its own note records "migrate the other spawn sites" as rejected-for-now. Review on the introducing PR reversed that deferral: the stacked follow-up should reshape the interface toward Node's API and move the remaining process-running places onto the service. The remaining spawners each carried a private copy of some slice of the same mechanics — lsp-local had its own detached-tree signalling (POSIX group + Windows taskkill + liveness polling), subagent-subprocess had the dispose ladder and its own scrub, mcp-client and pty-local and the SDK helper each had a third/fourth/fifth copy of the credential scrub — and none of it was swappable or centrally testable. - -## Decision - -The seam's vocabulary is now Node-shaped, and every spawner that can ride the service does: - -- **Per-stream stdio dispositions** on `SubprocessSpawnSpec`: `'pipe'` (the raw `Readable`/`Writable`, for consumer-owned protocol framing), `'inherit'` (diagnostics to the parent's stream), and collect mode `{ maxBytes, spill? }` — the original bounded tail-keep shape, with the spill file now optional so a diagnostic tail (a language server's stderr) buffers without touching disk. stdin is `'ignore'`, `'pipe'`, or `{ data }` (write-and-close batch). -- **`SubprocessOutcome` carries exit facts only** (Node's close-event vocabulary); collected output stays readable through `handle.collected` after settlement (spill fds seal at the settle boundary), so batch and streaming callers share one access path and nothing is copied into the outcome. -- **Tree-scoped termination behind one verb**: `terminate()` owns the SIGTERM→grace→SIGKILL escalation (serves the spec's abort signal too, and is a no-op once the tree is gone) — the handle exposes no single-signal `kill(signal?)`, so a consumer cannot skip the grace window; `waitForExit()` polls tree liveness (POSIX group probe; direct-child boundary on Windows); Windows tree termination (`taskkill /T`, injectable) moved in from lsp-local, so tree semantics are platform-correct for every consumer. (The stdin-EOF-first dispose ladder initially absorbed from `subagent-subprocess` later moved back out to its one consumer — see the [ladder-ownership Agent Note](2026-07-27-dispose-ladder-to-consumer.md).) -- **One scrub definition**: `scrubbedParentEnv()`/`SENSITIVE_ENV_PATTERN` live on the seam. Ordinary and terminal local spawns apply it inside `dsh-subprocess-local`; mcp-client still imports it because the MCP SDK owns that transport spawn, and the SDK helper's `scrubEnvironment()` defaults through it as well. - -Migrations landed with the reshape: **bash-local/bash-sandbox** (collect modes + batch stdin; the bash `kill()` maps to `terminate()` so `task_kill` keeps escalation semantics), **lsp-local** (piped protocol streams + a no-spill collected stderr tail; `LspConnection` takes the seam's spawn function; its private tree-op helpers deleted), **subagent-acp** (piped ndjson streams + inherited stderr; spawn failure surfaces through `done` rejection into the same startup race; disposal is the backend-owned `disposeAcpChild` ladder over the seam's verbs, with the plugin's configured graces), and **pty-local** through the later [portable execution-world decision](2026-07-28-portable-execution-world-consumers.md) (`node-pty` allocation and process inspection sit behind `spawnTerminal()`, while readiness and terminal policy remain in the consumer). **`dsh-subagent-subprocess` is deleted** — the dispose ladder and scrub are the seam's; the unused isolated-config-dir helper died with it (no consumer existed). - -Compositions mounting lsp-local or subagent-acp now load `dsh-subprocess-local` (the plugins inject `'subprocess'`); the acp/lsp test fixtures gained the row. - -## Alternatives considered - -**Keep the batch-only seam and let stream consumers stay bespoke.** The introducing note's position, rejected by review: it leaves three private copies of tree signalling and five of the scrub, and any future runner (containerized executor, remote process host) would have to pick which private copy to fork. The Node-shaped dispositions cover all three observed stream shapes without widening the outcome type or buffering piped streams. - -**A single `stdio: 'pipe' | 'inherit' | 'collect'` mode for all three streams at once.** Rejected: real consumers mix modes per stream (lsp: pipe/pipe/collect; acp: pipe/pipe/inherit; bash: data/collect/collect). Per-stream dispositions are exactly Node's shape and avoid a second spawn call for the mixed cases. - -**Keep pty-local and mcp-client spawns outside the service.** The MCP SDK still owns its transport spawn. PTY allocation is different: the [portable execution-world decision](2026-07-28-portable-execution-world-consumers.md) moves `node-pty` behind one deep terminal primitive, resolving the ownership objection without pretending an ordinary piped spawn can provide terminal semantics. - -**Migrate the test-support launchers (acp-snapshot, loader-smoke) and the SDK package-manager runner.** Rejected: the support packages are deliberately dependency-light test infrastructure that must not depend on product seams, and the SDK wizard's `stdio: 'inherit'`-with-redirect semantics plus its out-of-composition lifecycle (no cordis context at all) make the service a poor fit; it shares the scrub instead. - -## Consequences - -Bought: one implementation of tree signalling, escalation, bounded collection, terminal process mechanics, and the scrub, tested once in `dsh-subprocess-local`'s suites (including injected-platform Windows coverage that lsp-local's private copy never had); lsp-local, pty-local, and subagent-acp shed provider-specific process plumbing and their children die with composition teardown like bash's; a whole package (`dsh-subagent-subprocess`) is gone. The seam README's "one consumer family" limitation is retired. - -Cost: the seam is wider — execution-world coordinates, executable lookup, three stdio modes, process-tree lifecycle, and one terminal primitive — so a future backend implements more surface; the compositions for lsp-local, pty-local, and subagent-acp each carry the subprocess row; and `SubprocessOutcome` no longer carries output, a breaking shape change inside the still-unreleased stack. MCP/SDK/test-support spawns remain outside the service by ownership, with the scrub as the shared floor. diff --git a/.agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.zh.md b/.agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.zh.md deleted file mode 100644 index 500dcfe36f..0000000000 --- a/.agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.zh.md +++ /dev/null @@ -1,38 +0,0 @@ -# Agent Note: 进程 seam 转向 Node 形状,所有具备条件的 spawn 调用点一并迁入 - -Status: implemented - -[English](2026-07-26-subprocess-consumer-migration.md) | 中文 - -## 问题 - -[进程 seam](2026-07-26-subprocess-seam.md) 交付时恰好只为一个消费方家族塑形:批量收集的 stdout/stderr、批量 stdin、单一的升级式 `kill()`。那是有意的范围控制,其自身的 Agent Note 也把「迁移其余 spawn 调用点」记为暂缓否决项。引入该 seam 的 PR(Pull Request)上的评审推翻了这一暂缓决定:堆叠其上的后续变更应当把接口向 Node 的 API 方向重塑,并把其余运行进程之处迁到该服务上。其余各 spawn 调用点此前各自持有同一套机制中某个切片的私有副本——lsp-local 自带 detached 进程树信号发送(POSIX 进程组 + Windows taskkill + 存活轮询),subagent-subprocess 自带 dispose(资源释放)阶梯和自己的凭据清除,mcp-client、pty-local 与 SDK helper 则各自持有凭据清除的第三、第四、第五份副本——而这一切既不可替换,也无法集中测试。 - -## 决策 - -这道 seam 的词汇如今已是 Node 形状,凡能接入该服务的 spawn 调用点均已迁入: - -- **按流划分的 stdio 处置方式(disposition)**,位于 `SubprocessSpawnSpec` 上:`'pipe'`(原始的 `Readable`/`Writable`,供消费方自有的协议分帧使用)、`'inherit'`(诊断输出直通父进程的流),以及收集模式(collect)`{ maxBytes, spill? }`——即最初的有界尾部保留形状,只是 spill 文件改为可选,使诊断尾部(例如语言服务器的 stderr)无需落盘即可缓冲。stdin 则为 `'ignore'`、`'pipe'` 或 `{ data }`(写完即关闭的批量形式)。 -- **`SubprocessOutcome` 只承载退出事实**(Node close 事件的词汇);收集到的输出在结算后仍可经 `handle.collected` 读取(spill 文件描述符在结算边界封存),因此批量与流式调用方共用一条访问路径,也没有任何内容被复制进这份结果。 -- **以进程树为范围的终止,集中在一个动词后面**:`terminate()` 拥有 SIGTERM→宽限期→SIGKILL 升级(也承接 spec 的 abort 信号,进程树消亡后为空操作)——句柄不暴露单信号的 `kill(signal?)`,因此消费方无法跳过宽限窗口;`waitForExit()` 轮询进程树存活状态(POSIX 进程组探测;Windows 上以直接子进程为界)。Windows 进程树终止(`taskkill /T`,可注入)自 lsp-local 迁入,因此每个消费方拿到的进程树语义在各平台上都正确。(最初从 `subagent-subprocess` 吸收的以 stdin EOF 打头的 dispose 阶梯,后来又移回其唯一消费方——见[阶梯归属 Agent Note](2026-07-27-dispose-ladder-to-consumer.md)。) -- **凭据清除只有一份定义**:`scrubbedParentEnv()`/`SENSITIVE_ENV_PATTERN` 定义在 seam 上。普通本地 spawn 与终端本地 spawn 都在 `dsh-subprocess-local` 内部应用该定义;mcp-client 仍需导入它,因为 MCP SDK 拥有该传输层 spawn,SDK helper 的 `scrubEnvironment()` 默认同样委托给它。 - -各项迁移随这次重塑一并落地:**bash-local/bash-sandbox**(收集模式 + 批量 stdin;bash 的 `kill()` 映射到 `terminate()`,因此 `task_kill` 保有升级语义),**lsp-local**(管道化的协议流 + 无 spill 的 stderr 收集尾部;`LspConnection` 改为接收 seam 的 spawn 函数;其私有的进程树操作辅助函数已删除),**subagent-acp**(管道化的 ndjson 流 + inherit 的 stderr;spawn 失败经 `done` 的 reject 汇入同一个启动竞态;dispose 是后端自有的 `disposeAcpChild` 阶梯,经由 seam 的动词运行,携带插件所配置的宽限期),以及后来通过[可移植执行环境决策](2026-07-28-portable-execution-world-consumers.md)迁移的 **pty-local**(`node-pty` 分配与进程检查位于 `spawnTerminal()` 之后,就绪状态和终端策略仍归消费方所有)。**`dsh-subagent-subprocess` 已删除**——dispose 阶梯与凭据清除归 seam 所有;无人使用的隔离配置目录辅助函数随之消亡(其消费方本就不存在)。 - -挂载 lsp-local 或 subagent-acp 的组合如今都加载 `dsh-subprocess-local`(这两个插件注入 `'subprocess'`);acp/lsp 测试 fixture(测试前置数据)补上了这一行组合配置。 - -## 曾考虑的替代方案 - -**保持只支持批量的 seam,让流式消费方继续各自为政。**这正是引入该 seam 的 Agent Note 当初的立场,评审将其否决:这样会留下三份进程树信号发送的私有副本和五份凭据清除的私有副本,而未来任何运行器(容器化执行器、远程进程宿主)都得挑选去 fork 哪一份私有副本。Node 形状的处置方式覆盖已观察到的全部三种流形状,既不拓宽结果类型,也不缓冲管道化的流。 - -**用单个 `stdio: 'pipe' | 'inherit' | 'collect'` 模式一次性统辖全部三条流。**否决:真实消费方按流混用模式(lsp:pipe/pipe/collect;acp:pipe/pipe/inherit;bash:data/collect/collect)。按流划分的处置方式恰好就是 Node 的形状,也免去了混用场景的第二个 spawn 调用。 - -**让 pty-local 与 mcp-client 的 spawn 留在服务之外。**MCP SDK 仍拥有其传输层 spawn。PTY 分配则不同:[可移植执行环境决策](2026-07-28-portable-execution-world-consumers.md)把 `node-pty` 放到一个深层终端原语之后,既解决所有权异议,也不假装普通管道化 spawn 能够提供终端语义。 - -**迁移 test-support 启动器(acp-snapshot、loader-smoke)与 SDK package-manager 运行器。**否决:support 各包(package)是刻意保持轻依赖的测试基础设施,不得依赖产品 seam;而 SDK 向导那套附带重定向的 `stdio: 'inherit'` 语义,加上其完全脱离组合的生命周期(根本没有 cordis 上下文),使该服务并不合用;它改为共享凭据清除。 - -## 后果 - -换来的是:进程树信号发送、升级、有界收集、终端进程机制与凭据清除各自只剩一份实现,且只在 `dsh-subprocess-local` 的测试套件中测试一次(其中包括 lsp-local 的私有副本从未有过的、以注入平台方式实现的 Windows 覆盖);lsp-local、pty-local 与 subagent-acp 卸下了提供方专用的进程管道,其子进程如今像 bash 的一样随组合拆除而终止;一个完整的包(`dsh-subagent-subprocess`)就此消失。seam README 中「只有一个消费方家族」的限制说明也随之退役。 - -代价是:这道 seam 变宽了,涵盖执行环境坐标、可执行文件查找、三种 stdio 模式、进程树生命周期和一个终端原语,未来的后端因此要实现更宽的表面;lsp-local、pty-local 与 subagent-acp 的各组合如今都多出 subprocess 这一行组合配置;`SubprocessOutcome` 也不再承载输出,这是仍未发布的堆叠变更内部的一次破坏性形状变更。MCP/SDK/test-support 的 spawn 因所有权归属留在该服务之外,以凭据清除作为共享底线。 diff --git a/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.i18n.yaml index 94711572d6..0b62184128 100644 --- a/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md -2026-07-26-subprocess-seam.md: d5dee7361e48f18f3147444b2db74da3dd1c1161 -2026-07-26-subprocess-seam.zh.md: 884c572dc65c0fbf240f7e4e3b0387038e21f5ad +2026-07-26-subprocess-seam.md: 359977ac6a50394405163bc060f3a39eb68891fb +2026-07-26-subprocess-seam.zh.md: 551fcc8f4df82d0c893d8e6f5423f172dfd5d9c5 diff --git a/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md b/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md index d5dee7361e..359977ac6a 100644 --- a/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md +++ b/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md @@ -12,8 +12,8 @@ English | [中文](2026-07-26-subprocess-seam.zh.md) A new `subprocess/` capability family owns "run and manage a process"; the bash family keeps "run a bash command" and consumes it: -- **`@deepseek-ai/dsh-subprocess` (interface)** — the abstract `SubprocessService` owning `ctx.subprocess`: execution-world cwd and runtime storage, executable lookup, fully explicit ordinary spawns, and the terminal primitive added by the [portable execution-world decision](2026-07-28-portable-execution-world-consumers.md). Its vocabulary includes per-stream stdio dispositions, process and terminal handles, exit facts with deliberately no timeout/cancel classification, and the shared scrub plus `DSH_ENV_PREFIX`/`DshEnvironment`/`CollectedOutput` types. `argv` is never shell-interpreted. -- **`@deepseek-ai/dsh-subprocess-local` (implementation)** — `LocalSubprocessService` over the former `run.ts` plumbing (`spawn.ts`) plus `node-pty`: detached groups, bounded collection and private spill files, executable lookup, private runtime storage, foreground/session inspection, credential scrub with explicit env merged after it, tree cleanup, and disposal that terminates and joins every managed process. It has no config; every limit arrives on the spec. Bash and PTY presentation environment overrides stay in their consumers. +- **`@deepseek-ai/dsh-subprocess` (interface)** — the abstract `SubprocessService` owning `ctx.subprocess`: execution-world cwd and runtime storage, executable lookup, fully explicit ordinary spawns, and the terminal primitive added by the [portable execution-world decision](2026-07-28-portable-execution-world-consumers.md). Each stdio stream independently selects `'pipe'`, `'inherit'`, or bounded collection `{ maxBytes, spill? }`; stdin selects `'ignore'`, `'pipe'`, or `{ data }`. `SubprocessOutcome` carries exit facts with deliberately no timeout/cancel classification, while collected output remains on the handle after settlement. The seam also owns process and terminal handles, the shared scrub, and `DSH_ENV_PREFIX`/`DshEnvironment`/`CollectedOutput`; `argv` is never shell-interpreted. +- **`@deepseek-ai/dsh-subprocess-local` (implementation)** — `LocalSubprocessService` over the former `run.ts` plumbing (`spawn.ts`) plus `node-pty`: detached groups, bounded collection and private spill files, executable lookup, private runtime storage, foreground/session inspection, and disposal that terminates and joins every managed process. `terminate()` owns TERM→grace→KILL for the tree, `waitForExit()` observes tree liveness, and injected `taskkill /T` covers Windows. Ordinary and terminal spawns apply the seam's case-insensitive `KEY`/`PASSWORD`/`SECRET`/`TOKEN` scrub before explicit env. The implementation has no config; every limit arrives on the spec, while Bash and PTY presentation environment overrides stay in their consumers. - **`dsh-bash-local` (consumer)** — `inject: ['subprocess']`; maps each resolved `BashExecSpec` onto a `SubprocessSpawnSpec` (`['bash', '-c', command]`), keeps its config, `resolve()` defaulting, fused-deadline `timedOut`/`aborted` classification, the `[stderr]`-marked background read merge with its consuming cursor, and the `onProcessDone` subclass hook. `dsh-bash-sandbox` is unchanged apart from redeclaring the inherited inject; it still wraps at the command-string level and re-enters the inherited spawn path. - **`dsh-bash` (seam)** — re-exports the moved vocabulary from `dsh-subprocess`, so no bash consumer changes an import; `BashExecRequest`/`BashExecSpec`/`BashProcess` and the sandbox facts remain bash-owned. @@ -21,11 +21,17 @@ Every composition that loads a bash executor now also loads `@deepseek-ai/dsh-su Background-process lifetime moved from the executor to the subprocess service: the executor no longer retains a live-process set, so an executor reload leaves background work running and readable, and composition teardown (the service's disposal) remains the kill-and-join boundary. One behavioral seam shifted with it: a background spawn failure can no longer be buffered as fake stderr inside the plumbing (the service rejects `done` and buffers nothing for a process that never ran), so the executor injects the `spawn failed: …` note into exactly one `readOutput()` delta. +Observed stream and lifecycle needs then moved the eligible process consumers onto the seam: LSP uses piped protocol streams plus a collected stderr tail; the ACP backend uses piped ndjson, inherited stderr, and a consumer-owned stdin-EOF disposal ladder; PTY uses `spawnTerminal()` while keeping readiness and terminal policy; Code Runtime uses ordinary raw pipes. `dsh-subagent-subprocess` and the private LSP tree helpers were deleted. MCP transport spawning, the SDK package-manager runner, synchronous TUI Git probing, and dependency-light test-support launchers remain outside by ownership or execution shape; their production callers share the scrub where applicable. + ## Alternatives considered **Leave the process plumbing inside `dsh-bash-local` (status quo).** Rejected for the same reason the [task registry split](2026-07-26-task-registry-seam.md) landed: the boundary is stable and already documented in-code (`run.ts`'s module doc said "this layer reacts to an abort signal; the executor owns deadlines and classifies causes"), and keeping it private makes every future non-shell runner either fork the mechanics or depend on a bash-named package for non-bash work. The user-visible driver for this stack was exactly this split. -**Migrate the repo's other spawn sites in the introducing change.** Rejected as scope creep with real design risk at that scale. The later [consumer-migration](2026-07-26-subprocess-consumer-migration.md) and [portable execution-world](2026-07-28-portable-execution-world-consumers.md) decisions reshape the interface around the observed LSP, subagent, PTY, and Code Runtime consumers; SDK and test-support launchers remain outside by ownership. +**Keep the original batch-only interface and leave stream consumers bespoke.** Rejected after the observed LSP, ACP, PTY, and Code Runtime shapes showed that private process-tree signalling and environment scrubs would otherwise remain duplicated. The Node-shaped dispositions cover those consumers without buffering piped streams. + +**Use one `stdio: 'pipe' | 'inherit' | 'collect'` mode for all streams.** Rejected because real consumers mix modes per stream: LSP uses pipe/pipe/collect, ACP uses pipe/pipe/inherit, and Bash uses data/collect/collect. + +**Route every process launch through `ctx.subprocess`.** Rejected because the MCP SDK owns its transport spawn, the SDK wizard has no Cordis context and needs inherited redirection, the TUI probe is synchronous, and support launchers deliberately stay independent of product seams. PTY allocation did move behind `spawnTerminal()` because the provider, not the consumer, owns that substrate-specific primitive. **Put `run_in_background`/task semantics into the process seam instead.** Rejected: that boundary already exists — `ctx.tasks` owns ids, ownership, and notices, and the bash tool adapts a `BashProcess` into task hooks. The process seam sits *below* the bash executor, not beside the task registry. @@ -33,6 +39,6 @@ Background-process lifetime moved from the executor to the subprocess service: t ## Consequences -Bought: "run and manage a process" is a swappable capability used by Bash, LSP, PTY, Code Runtime, and ACP consumers; a containerized or remote process backend slots in without changing their domain semantics; the shared `DSH_*`/output vocabulary has a non-shell home; and background processes survive executor reloads, matching the task registry's lifetime model. Process and terminal plumbing is tested through `dsh-subprocess-local`; consumer suites pin only their owned behavior against the real service. +Bought: "run and manage a process" is a swappable capability used by Bash, LSP, PTY, Code Runtime, and ACP consumers; a containerized or remote process backend slots in without changing their domain semantics; tree signalling, escalation, bounded collection, terminal mechanics, and credential scrubbing each have one implementation; and background processes survive executor reloads, matching the task registry's lifetime model. Process and terminal plumbing is tested through `dsh-subprocess-local`; consumer suites pin only their owned behavior against the real service. -Cost: one more package pair and one more composition row everywhere a bash executor loads — a boot that loads an executor without the subprocess service leaves `ctx.bash` pending on `ctx.subprocess` (standard missing-service behavior). The moved-vocabulary re-exports keep `dsh-bash` imports working but mean two packages now name the same types; the subprocess seam is the owner and the bash seam documents the re-export. The spawn-failure note became single-delivery through the read path where the old plumbing retained it in the stderr buffer for repeated `readFrom(0)` reads — acceptable because the bash background read path was already a consuming cursor, and the note reaches the one reader that exists. +Cost: one more package pair and one more composition row wherever a consumer loads; a missing subprocess provider leaves the consumer pending by standard service-injection behavior. Every backend implements execution-world coordinates, executable lookup, three stdio modes, tree lifecycle, and one terminal primitive. The moved-vocabulary re-exports keep `dsh-bash` imports working but mean two packages name the same types; the subprocess seam is the owner. The spawn-failure note became single-delivery through Bash's consuming read cursor instead of repeatable stderr-buffer content. diff --git a/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.zh.md b/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.zh.md index 884c572dc6..551fcc8f4d 100644 --- a/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.zh.md @@ -12,8 +12,8 @@ Status: implemented 新的 `subprocess/` 能力家族拥有「运行并管理一个进程」;bash 家族保留「运行一条 bash 命令」,并成为前者的消费方: -- **`@deepseek-ai/dsh-subprocess`(接口)**——拥有 `ctx.subprocess` 的抽象 `SubprocessService`:执行环境 cwd 与运行时存储、可执行文件查找、完全显式的普通 spawn,以及[可移植执行环境决策](2026-07-28-portable-execution-world-consumers.md)新增的终端原语。其词汇包括按流划分的 stdio 处置方式(disposition)、进程与终端句柄、刻意不含超时/取消分类的退出事实,以及共享的凭据清除与 `DSH_ENV_PREFIX`/`DshEnvironment`/`CollectedOutput` 类型。`argv` 绝不经过 shell 解释。 -- **`@deepseek-ai/dsh-subprocess-local`(实现)**——`LocalSubprocessService` 构建在原 `run.ts` 管道(现为 `spawn.ts`)与 `node-pty` 之上:detached 进程组、有界收集与私有 spill 文件、可执行文件查找、私有运行时存储、前台/会话检查、清除之后合并显式 env 的凭据清除、进程树清理,以及终止每个受管进程并等待其退出的 dispose。该实现没有任何配置;每项限制都随 spec 到达。Bash 与 PTY 的呈现环境覆盖仍归各自消费方所有。 +- **`@deepseek-ai/dsh-subprocess`(接口)**——拥有 `ctx.subprocess` 的抽象 `SubprocessService`:执行环境 cwd 与运行时存储、可执行文件查找、完全显式的普通 spawn,以及[可移植执行环境决策](2026-07-28-portable-execution-world-consumers.md)新增的终端原语。每条 stdio 流独立选择 `'pipe'`、`'inherit'` 或有界收集 `{ maxBytes, spill? }`;stdin 选择 `'ignore'`、`'pipe'` 或 `{ data }`。`SubprocessOutcome` 只承载刻意不含超时/取消分类的退出事实,收集输出在结算后仍留在句柄上。该 seam 还拥有进程与终端句柄、共享凭据清除,以及 `DSH_ENV_PREFIX`/`DshEnvironment`/`CollectedOutput`;`argv` 绝不经过 shell 解释。 +- **`@deepseek-ai/dsh-subprocess-local`(实现)**——`LocalSubprocessService` 构建在原 `run.ts` 管道(现为 `spawn.ts`)与 `node-pty` 之上:detached 进程组、有界收集与私有 spill 文件、可执行文件查找、私有运行时存储、前台/会话检查,以及终止每个受管进程并等待其退出的 dispose。`terminate()` 拥有面向进程树的 TERM→宽限→KILL,`waitForExit()` 观察进程树存活性,可注入的 `taskkill /T` 覆盖 Windows。普通与终端 spawn 都先应用 seam 对 `KEY`/`PASSWORD`/`SECRET`/`TOKEN` 不区分大小写的清除,再合并显式 env。该实现没有配置;每项限制都随 spec 到达,Bash 与 PTY 的呈现环境覆盖仍归各自消费方所有。 - **`dsh-bash-local`(消费方)**——`inject: ['subprocess']`;把每个解析后的 `BashExecSpec` 映射为一个 `SubprocessSpawnSpec`(`['bash', '-c', command]`),并保留自身配置、`resolve()` 默认值补全、基于融合 deadline 的 `timedOut`/`aborted` 分类、带 `[stderr]` 标记的后台读取合并及其消费游标,以及 `onProcessDone` 子类钩子。`dsh-bash-sandbox` 除了重新声明继承来的 inject 之外没有变化;它仍在命令字符串层面做包装,并重新进入继承的 spawn 路径。 - **`dsh-bash`(seam)**——把迁走的词汇从 `dsh-subprocess` 重导出,因此没有任何 bash 消费方需要改动导入;`BashExecRequest`/`BashExecSpec`/`BashProcess` 与沙箱事实仍归 bash 所有。 @@ -21,11 +21,17 @@ Status: implemented 后台进程的存续期从执行器移到了管理器:执行器不再保有存活进程集合,于是重载执行器后,后台工作会继续运行且仍可读取,而组合拆除(管理器的 dispose)仍是先终止再等待退出的边界。一条行为 seam 随之挪动:后台 spawn 失败不再能在管道内部被缓冲成伪造的 stderr(对一个从未真正运行的进程,管理器会 reject `done`,且不缓冲任何内容),因此执行器把 `spawn failed: …` 提示注入恰好一个 `readOutput()` 增量。 +基于已观察到的流与生命周期需求,具备条件的进程消费方随后迁到该 seam:LSP 使用管道化协议流加收集式 stderr 尾部;ACP(Agent Client Protocol)后端使用管道化 ndjson、继承式 stderr 和消费方拥有的 stdin-EOF dispose 阶梯;PTY 使用 `spawnTerminal()`,同时保留就绪与终端策略;Code Runtime 使用普通原始管道。`dsh-subagent-subprocess` 与 LSP 私有进程树辅助函数均被删除。MCP 传输 spawn、SDK 包管理器运行器、同步 TUI Git 探测和刻意保持轻依赖的 test-support 启动器因所有权或执行形状仍留在外部;适用的生产调用方共享凭据清除。 + ## 曾考虑的替代方案 **把进程管道留在 `dsh-bash-local` 里(维持现状)。**否决的理由与[任务注册表拆分](2026-07-26-task-registry-seam.md)得以落地的理由相同:这条边界既稳定,也早已记录在代码里(`run.ts` 的模块文档曾写明「this layer reacts to an abort signal; the executor owns deadlines and classifies causes」),而若继续将它保持私有,未来每个非 shell 运行器就只能要么 fork 这套机制,要么为非 bash 工作去依赖一个以 bash 命名的包。这组堆叠变更对用户可见的动因正是这一拆分。 -**在引入 seam 的同一变更中迁移仓库其余 spawn 调用点。**否决,因为在当时的规模下属于带有真实设计风险的范围蔓延。后续的[消费方迁移](2026-07-26-subprocess-consumer-migration.md)与[可移植执行环境](2026-07-28-portable-execution-world-consumers.md)决策围绕已观察到的 LSP、subagent、PTY 与 Code Runtime 消费方重塑接口;SDK 与 test-support 启动器因所有权归属仍留在服务之外。 +**保留最初只支持批量的接口,让流式消费方继续各自实现。**否决:已观察到的 LSP、ACP、PTY 与 Code Runtime 形状表明,这会继续保留重复的私有进程树信号与环境清除。Node 形状的处置方式覆盖这些消费方,又不缓冲管道化流。 + +**用单个 `stdio: 'pipe' | 'inherit' | 'collect'` 模式统一全部流。**否决:真实消费方按流混用模式——LSP 使用 pipe/pipe/collect,ACP 使用 pipe/pipe/inherit,Bash 使用 data/collect/collect。 + +**把每一次进程启动都路由到 `ctx.subprocess`。**否决:MCP SDK 拥有其传输 spawn,SDK 向导没有 Cordis 上下文且需要继承式重定向,TUI 探测是同步的,support 启动器则刻意独立于产品 seam。PTY 分配迁到 `spawnTerminal()`,因为这项底层专用原语归提供方而非消费方所有。 **改把 `run_in_background`/任务语义放进进程 seam。**否决:那条边界已经存在。`ctx.tasks` 拥有 id、所有权与通知,bash 工具则把 `BashProcess` 适配成任务钩子。进程 seam 位于 bash 执行器*之下*,而不是与任务注册表并列。 @@ -33,6 +39,6 @@ Status: implemented ## 后果 -换来的是:「运行并管理一个进程」成为 Bash、LSP、PTY、Code Runtime 与 ACP(Agent Client Protocol)消费方共用的可替换能力;容器化或远程进程后端可以直接接入,而无需改变各领域语义;共享的 `DSH_*`/输出词汇有了一个不带 shell 含义的归属;后台进程也能在执行器重载后存活,与任务注册表的存续期模型一致。进程与终端管道通过 `dsh-subprocess-local` 测试;消费方测试套件只需针对真实服务固定各自拥有的行为。 +换来的是:「运行并管理一个进程」成为 Bash、LSP、PTY、Code Runtime 与 ACP 消费方共用的可替换能力;容器化或远程进程后端可以直接接入,而无需改变各领域语义;进程树信号、升级终止、有界收集、终端机制与凭据清除各自只剩一份实现;后台进程也能在执行器重载后存活,与任务注册表的存续期模型一致。进程与终端管道通过 `dsh-subprocess-local` 测试;消费方测试套件只需针对真实服务固定各自拥有的行为。 -代价是:多出一对包,而且凡加载 bash 执行器之处都多一行组合配置。若某次启动加载了执行器却没有加载管理器,`ctx.bash` 会因等待 `ctx.subprocess` 而保持挂起(标准的服务缺失行为)。迁移词汇的重导出让 `dsh-bash` 的导入继续可用,但也意味着两个包如今命名同一批类型;进程 seam 是所有者,bash seam 则记录这层重导出。spawn 失败提示经由读取路径变为单次交付,而旧管道曾把它保留在 stderr 缓冲区里,供重复的 `readFrom(0)` 读取;这一点可以接受,因为 bash 的后台读取路径本就是消费游标,该提示能到达唯一存在的那个读取方。 +代价是:多出一对包,而且凡加载消费方之处都多一行组合配置;缺少 subprocess 提供方时,消费方会按标准服务注入行为保持挂起。每个后端都要实现执行环境坐标、可执行文件查找、三种 stdio 模式、进程树生命周期和一个终端原语。迁移词汇的重导出让 `dsh-bash` 的导入继续可用,但也意味着两个包命名同一批类型;进程 seam 是所有者。spawn 失败提示经由 Bash 的消费式读取游标变为单次交付,不再是可重复读取的 stderr 缓冲内容。 From 68ccf50a77f34288de50d5008eb636851cd48e18 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Thu, 30 Jul 2026 02:04:21 +0800 Subject: [PATCH 046/108] refactor(code-runtime): generate shared subprocess runner --- package.json | 2 + .../code-runtime-worker/src/bootstrap.ts | 19 +- .../code-runtime-worker/src/runtime-host.ts | 26 +- .../tests/bootstrap.spec.ts | 27 ++ .../src/runner-source.generated.ts | 6 + packages/util/atomic-write/src/runner.ts | 390 ++++++++++++++++++ scripts/gen-code-runtime-runner.ts | 70 ++++ scripts/run-gates.ts | 1 + vitest.config.ts | 96 +---- 9 files changed, 555 insertions(+), 82 deletions(-) create mode 100644 packages/util/atomic-write/src/runner-source.generated.ts create mode 100644 packages/util/atomic-write/src/runner.ts create mode 100644 scripts/gen-code-runtime-runner.ts diff --git a/package.json b/package.json index 423cbf813c..2b4971a6b7 100644 --- a/package.json +++ b/package.json @@ -112,6 +112,8 @@ "gen-module-graph": "tsx scripts/gen-module-graph.ts", "gen-scoped-events": "tsx scripts/gen-scoped-events.ts", "verify-scoped-events": "tsx scripts/gen-scoped-events.ts --check", + "gen-code-runtime-runner": "tsx scripts/gen-code-runtime-runner.ts", + "verify-code-runtime-runner": "tsx scripts/gen-code-runtime-runner.ts --check", "verify-module-graph": "tsx scripts/gen-module-graph.ts --check", "constraints": "tsx scripts/check-workspace-constraints.ts", "doc-sync": "tsx scripts/run-gates.ts doc-sync", diff --git a/packages/code-runtime/code-runtime-worker/src/bootstrap.ts b/packages/code-runtime/code-runtime-worker/src/bootstrap.ts index cbe0d0ac55..2207389c97 100644 --- a/packages/code-runtime/code-runtime-worker/src/bootstrap.ts +++ b/packages/code-runtime/code-runtime-worker/src/bootstrap.ts @@ -6,6 +6,7 @@ */ import { inspect } from 'node:util' +import type { CodeJsonValue } from '@deepseek-ai/dsh-code-runtime' import type { DoneMessage, ReplyMessage, WorkerBootData, WorkerToHost } from './protocol.ts' import { jsonStringBytesUpTo, jsonValueBytesUpTo, truncateJsonStringBytes } from './output-json.ts' import { decodeWorkerJson, encodeWorkerJson, snapshotCodeJsonValue } from './worker-json.ts' @@ -310,6 +311,7 @@ export function wireReplies(port: BootstrapPort, pending: Map, nextId: { value: number }, errorClasses: Map = makeBindingErrorClasses(data), + maxFrameBytes?: number, ): Record[] { return data.namespaces.map(({ global, names }) => { const errorClass = errorClasses.get(global) @@ -335,6 +338,11 @@ export function makeNamespaces( if (detached === undefined) { return Promise.reject(bindingFailure(errorClass, name, 'binding arguments must be lossless JSON')) } + const call = { type: 'call' as const, id: nextId.value, global, name, args: encodeWorkerJson(detached) } + if (maxFrameBytes !== undefined + && jsonValueBytesUpTo(call as unknown as CodeJsonValue, maxFrameBytes) === undefined) { + return Promise.reject(bindingFailure(errorClass, name, 'binding arguments exceed maxFrameBytes')) + } return new Promise((resolve, reject) => { const id = nextId.value++ pending.set(id, { @@ -344,7 +352,7 @@ export function makeNamespaces( }, }) try { - port.postMessage({ type: 'call', id, global, name, args: encodeWorkerJson(detached) }) + port.postMessage(call) } catch (error: unknown) { pending.delete(id) const message = `binding arguments must be structured-cloneable: ${error instanceof CapturedError ? error.message : String(error)}` @@ -364,12 +372,14 @@ export function makeNamespaces( * @param port - host message port or test double. * @param data - the boot payload the host sent. * @param streams - stdout/stderr objects captured as program logs. + * @param maxFrameBytes - optional serialized transport cap checked before posting. * @returns after posting the done message. */ export async function runWorkerMain( port: BootstrapPort, data: WorkerBootData, streams: { stdout: PatchableStream; stderr: PatchableStream }, + maxFrameBytes?: number, ): Promise { const logs = new LogBuffer( data.maxOutputBytes, @@ -384,7 +394,7 @@ export async function runWorkerMain( const nextId = { value: 1 } const errorClasses = makeBindingErrorClasses(data) - const namespaces = makeNamespaces(data, port, pending, nextId, errorClasses) + const namespaces = makeNamespaces(data, port, pending, nextId, errorClasses, maxFrameBytes) const errorClassParameters: string[] = [] const errorClassValues: BindingErrorConstructor[] = [] for (const namespace of data.namespaces) { @@ -420,5 +430,8 @@ export async function runWorkerMain( ...prepareException(error, logs.remainingOutputBytes(), data.maxOutputBytes), } } - port.postMessage(done) + port.postMessage(maxFrameBytes !== undefined + && jsonValueBytesUpTo(done as unknown as CodeJsonValue, maxFrameBytes) === undefined + ? { type: 'output-limit' } + : done) } diff --git a/packages/code-runtime/code-runtime-worker/src/runtime-host.ts b/packages/code-runtime/code-runtime-worker/src/runtime-host.ts index d2bb1082b8..47bfacfe91 100644 --- a/packages/code-runtime/code-runtime-worker/src/runtime-host.ts +++ b/packages/code-runtime/code-runtime-worker/src/runtime-host.ts @@ -1,6 +1,7 @@ /** Shared host mechanics for local and subprocess-hosted TypeScript worker runtimes. */ import { stripTypeScriptTypes } from 'node:module' +import type { Readable } from 'node:stream' import type { CodeBindingNamespace, CodeJsonValue, @@ -15,6 +16,28 @@ import type { WorkerJsonWire } from './worker-json.ts' /** Smallest cap that can represent an empty log array and failure message. */ export const MIN_RUNTIME_OUTPUT_BYTES = 4 +/** + * Resolve after a worker pipe emits queued data or closes during termination. + * @param stream - captured worker or child-process pipe. + * @returns after no more queued bytes can arrive. + */ +export function waitForRuntimePipeDrain(stream: Readable): Promise { + if (stream.readableEnded || stream.destroyed) return Promise.resolve() + return new Promise((resolve) => { + const done = (): void => { + stream.off('end', done) + stream.off('close', done) + stream.off('error', done) + resolve() + } + stream.once('end', done) + stream.once('close', done) + stream.once('error', done) + /* v8 ignore next -- termination can win the adjacent listener-registration race. */ + if (stream.readableEnded || stream.destroyed) done() + }) +} + const IDENTIFIER = /^[A-Za-z_$][A-Za-z0-9_$]*$/ const RESERVED_WORDS = new Set([ 'await', 'break', 'case', 'catch', 'class', 'const', 'continue', 'debugger', 'default', 'delete', 'do', @@ -223,5 +246,6 @@ export class RuntimeOutputLedger { } export { decodeWorkerJson, encodeWorkerJson, snapshotCodeJsonValue } from './worker-json.ts' -export { jsonValueBytesUpTo } from './output-json.ts' +export { jsonStringBytesUpTo, jsonValueBytesUpTo } from './output-json.ts' +export { runWorkerMain } from './bootstrap.ts' export type { WorkerJsonWire } from './worker-json.ts' diff --git a/packages/code-runtime/code-runtime-worker/tests/bootstrap.spec.ts b/packages/code-runtime/code-runtime-worker/tests/bootstrap.spec.ts index a2aac6d9c2..247501df29 100644 --- a/packages/code-runtime/code-runtime-worker/tests/bootstrap.spec.ts +++ b/packages/code-runtime/code-runtime-worker/tests/bootstrap.spec.ts @@ -286,6 +286,23 @@ describe('makeNamespaces', () => { expect(nextId.value).toBe(1) }) + it('rejects an oversized transport frame before posting or allocating a call id', async () => { + const port = new FakePort() + const pending = new Map() + const nextId = { value: 1 } + const data = { namespaces: [toolNamespace(['x'])] } + const [tools] = makeNamespaces( + data, port, pending, nextId, makeBindingErrorClasses(data), 64, + ) as [Record Promise>] + + await expect(tools.x?.({ text: 'x'.repeat(64) })).rejects.toMatchObject({ + name: 'ToolCallError', toolName: 'x', message: 'binding arguments exceed maxFrameBytes', + }) + expect(port.sent).toEqual([]) + expect(pending.size).toBe(0) + expect(nextId.value).toBe(1) + }) + it('uses ordinary Error for non-tools namespace failures', async () => { const deniedPort = new FakePort() deniedPort.respond = message => message.type === 'call' @@ -343,6 +360,16 @@ describe('runWorkerMain', () => { }) }) + it('reports output-limit before posting a completion that expands past the transport cap', async () => { + const port = new FakePort() + await runWorkerMain(port, { + maxOutputBytes: 1_000, + code: 'return Array.from({ length: 100 }, () => [])', + namespaces: [], + }, fakeStreams(), 100) + expect(port.sent.at(-1)).toEqual({ type: 'output-limit' }) + }) + it('reports a thrown program error on the done message', async () => { const port = new FakePort() await runWorkerMain(port, { ...BOOT, code: 'throw new Error("boom")', namespaces: [] }, fakeStreams()) diff --git a/packages/util/atomic-write/src/runner-source.generated.ts b/packages/util/atomic-write/src/runner-source.generated.ts new file mode 100644 index 0000000000..d40c77783c --- /dev/null +++ b/packages/util/atomic-write/src/runner-source.generated.ts @@ -0,0 +1,6 @@ +/** + * Generated dependency-free execution-world runner. + * Do not edit by hand; run `pnpm run gen-code-runtime-runner`. + */ + +export const CODE_RUNNER_SOURCE = "import{Buffer as e}from\"node:buffer\";import{fork as t}from\"node:child_process\";import{createInterface as n}from\"node:readline\";import{inspect as r}from\"node:util\";import{fileURLToPath as i}from\"node:url\";import{Worker as a,isMainThread as o,parentPort as s,workerData as c}from\"node:worker_threads\";const l=Reflect.apply,u=Array.isArray,d=Buffer,f=Reflect.get(Buffer,`byteLength`),ee=Object.create,p=Object.defineProperty,te=Object.keys,ne=String,re=Reflect.get(String.prototype,`charCodeAt`),ie=Reflect.get(String.prototype,`codePointAt`),m=Reflect.get(String.prototype,`slice`);function h(e){let t=ee(null);return t.value=e,t}function ae(e,t,n){let r=h(n);r.enumerable=!0,r.configurable=!0,r.writable=!0,p(e,t,r)}function g(e){return l(f,d,[e,`utf8`])}function _(e,t){ae(e,e.length,t)}function oe(e){if(e.length===0)return;let t=e.length-1,n=e[t];return p(e,`length`,h(t)),n}function se(e,t){return l(m,e,[t,t+(l(ie,e,[t])>65535?2:1)])}function ce(e){if(e.length===2)return 4;if(e===`\"`||e===`\\\\`)return 2;let t=l(re,e,[0]);return t>=55296&&t<=57343?6:t<32?t===8||t===9||t===10||t===12||t===13?2:6:g(e)}function v(e,t){if(t<2)return;let n=2;for(let r=0;rt)return;r+=i.length}return n}function y(e,t){let n=0,r=e=>(n+=e,n<=t),i=[{kind:`value`,value:e}];for(let e=oe(i);e!==void 0;e=oe(i)){if(e.kind===`value`){let a=e.value;if(a===null){if(!r(4))return}else if(typeof a==`string`){let e=v(a,t-n);if(e===void 0)return;n+=e}else if(typeof a==`number`){if(!r(g(ne(a))))return}else if(typeof a==`boolean`){if(!r(a?4:5))return}else if(u(a)){if(!r(2))return;a.length>0&&_(i,{kind:`array`,value:a,index:0})}else{if(!r(2))return;let e=te(a);e.length>0&&_(i,{kind:`object`,value:a,keys:e,index:0})}continue}if(e.index>0&&!r(1))return;if(e.kind===`array`){let t=e.value[e.index];if(t===void 0)return;e.index+1t)break;n+=o,r+=a.length,i+=a.length}return r===e.length?e:l(m,e,[0,r])}const ue=Reflect.get(Function.prototype,`toString`),b=Reflect.get(Reflect,`apply`),x=Error,S=Set,C=Array.isArray,de=Array.prototype,w=Number.isFinite,fe=Number.isSafeInteger,pe=Object.create,me=Object.defineProperty,he=Object.getOwnPropertyDescriptor,T=Object.getPrototypeOf,ge=Object.hasOwn,E=Object.is,_e=Object.keys,D=Object.prototype,ve=Reflect.get(D,`propertyIsEnumerable`),O=Reflect.ownKeys,ye=Reflect.get(Set.prototype,`add`),be=Reflect.get(Set.prototype,`delete`),xe=Reflect.get(Set.prototype,`has`);function k(e){let t=pe(null);return t.value=e,t}function A(e,t,n){let r=k(n);r.enumerable=!0,r.configurable=!0,r.writable=!0,me(e,t,r)}function j(e,t){A(e,e.length,t)}function M(e){if(e.length===0)return;let t=e.length-1,n=e[t];return me(e,`length`,k(t)),n}function N(e,t){return b(xe,e,[t])}function P(e,t){b(ye,e,[t])}function Se(e,t){b(be,e,[t])}function F(e,t){let n=he(e,`constructor`)?.value;if(typeof n!=`function`)return!1;try{return n.name===t&&n.prototype===e&&b(ue,n,[])===`function ${t}() { [native code] }`}catch{return!1}}function I(e){return T(e)===null&&F(e,`Object`)}function L(e){let t=T(e);if(t===de)return!0;if(!C(t)||!F(t,`Array`))return!1;let n=T(t);return typeof n==`object`&&!!n&&I(n)}function R(e){let t=T(e);return t===null||t===D||typeof t==`object`&&I(t)}function z(e){let t=O(e);for(let n=0;n{e.kind===`root`?n=t:e.kind===`array`?A(e.target,e.index,t):A(e.target,e.key,t)},i=[{kind:`visit`,value:e,destination:{kind:`root`}}];for(let e=M(i);e!==void 0;e=M(i)){if(e.kind===`leave`){Se(t,e.source);continue}if(e.kind===`array-item`){if(!ge(e.source,e.index))return;j(i,{kind:`visit`,value:e.source[e.index],destination:{kind:`array`,target:e.target,index:e.index}});continue}if(e.kind===`object-property`){j(i,{kind:`visit`,value:e.source[e.key],destination:{kind:`object`,target:e.target,key:e.key}});continue}let n=e.value;if(n===null){r(e.destination,null);continue}if(typeof n==`boolean`||typeof n==`string`){r(e.destination,n);continue}if(typeof n==`number`){if(!w(n)||E(n,-0))return;r(e.destination,n);continue}if(typeof n!=`object`||N(t,n))return;if(C(n)){if(!L(n))return;let a=n.length;if(O(n).length!==a+1)return;let o=[];r(e.destination,o),P(t,n),j(i,{kind:`leave`,source:n});for(let e=a-1;e>=0;e--)j(i,{kind:`array-item`,source:n,index:e,target:o});continue}if(!R(n))return;let a=z(n);if(a===void 0)return;let o={};r(e.destination,o),P(t,n),j(i,{kind:`leave`,source:n});for(let e=a.length-1;e>=0;e--){let t=a[e];if(t===void 0)return;j(i,{kind:`object-property`,source:n,key:t,target:o})}}return n}function V(e){let t=[],n=[e];for(let e=M(n);e!==void 0;e=M(n)){if(e===null||typeof e==`boolean`||typeof e==`number`||typeof e==`string`){j(t,e);continue}if(C(e)){j(t,{kind:`array`,length:e.length});for(let t=e.length-1;t>=0;t--){let r=e[t];if(r===void 0)throw new x(`cannot encode a sparse JSON array`);j(n,r)}continue}let r=_e(e);j(t,{kind:`object`,keys:r});for(let t=r.length-1;t>=0;t--){let i=r[t];if(i===void 0)throw new x(`cannot encode a missing JSON object key`);let a=e[i];if(a===void 0)throw new x(`cannot encode an undefined JSON object property`);j(n,a)}}return t}function H(e){if(!L(e)||O(e).length!==e.length+1)return!1;for(let t=0;t=0?{kind:`array`,length:e}:void 0}if(n.kind===`object`){if(t.length!==2||!U(t,`kind`)||!U(t,`keys`))return;let e=n.keys;if(!C(e)||!H(e))return;let r=new S,i=[],a=e;for(let e=0;e{let t=n[n.length-1];if(!t)return i?!1:(r=e,i=!0,!0);if(t.index>=(t.kind===`array`?t.length:t.keys.length))return!1;if(t.kind===`array`)j(t.target,e);else{let n=t.keys[t.index];if(n===void 0)return!1;A(t.target,n,e)}return t.index+=1,!0};for(let e=0;ea)return;let e=[];i=e,n.length>0&&(o={kind:`array`,target:e,length:n.length,index:0})}else{if(n.keys.length>a)return;let e={};i=e,n.keys.length>0&&(o={kind:`object`,target:e,keys:n.keys,index:0})}}if(!a(i))return;for(o&&j(n,o);n.length>0;){let e=n[n.length-1];if(e===void 0||e.index<(e.kind===`array`?e.length:e.keys.length))break;M(n)}}return n.length===0?r:void 0}catch{return}}const G=Error,we=Object.create,Te=Object.defineProperty;function K(e,t,n){let r=we(null);r.enumerable=!0,r.value=n,Te(e,t,r)}var Ee=class{bytes=2;entries=0;truncated=!1;sink;onLimit;maxBytes;constructor(e,t,n=()=>{}){this.maxBytes=e,this.sink=t,this.onLimit=n}push(e){if(this.truncated)return;let t=+(this.entries>0),n=this.maxBytes-this.bytes-t,r=v(e,n);if(r===void 0){this.truncated=!0;let r=le(e,n);if(r.length>0){let e=v(r,n);if(e===void 0)throw new G(`worker output ledger produced an oversized log prefix`);this.bytes+=e+t,this.entries+=1,this.sink(r)}this.onLimit();return}this.bytes+=r+t,this.entries+=1,this.sink(e)}remainingOutputBytes(){return this.maxBytes-this.bytes}};const De=[`log`,`info`,`warn`,`error`,`debug`];function Oe(e){let t=e=>e.map(e=>typeof e==`string`?e:r(e,Ae)).join(` `),n=Object.create(null);for(let r of De)n[r]=(...n)=>{e.push(t(n))};return n}function ke(e,t){let n=t.write;return t.write=(t,...n)=>{e.push(typeof t==`string`?t:String(t));let r=[n[0],n[1]].find(e=>typeof e==`function`);return r&&queueMicrotask(()=>{r(null)}),!0},()=>{t.write=n}}const Ae={depth:4,maxArrayLength:100,maxStringLength:1e4};function je(e,t,n=t){if(e===void 0)return{};let r;try{r=B(e)}catch{r=void 0}return r===void 0?Ne(`invalid-output`,`program completion must be lossless JSON`,t,n):y(r,t)===void 0?Me(n):{value:V(r)}}function Me(e){return{error:{kind:`output-limit`,message:`outer output exceeded ${e} bytes`}}}function Ne(e,t,n,r){return v(t,n)===void 0?Me(r):{error:{kind:e,message:t}}}function Pe(e,t,n=t){let r;try{let t=e instanceof G?e.stack??e.message:e;r=typeof t==`string`?t:String(t)}catch{r=`program threw an unrenderable value`}return Ne(`exception`,r,t,n)}function Fe(e){return class extends G{constructor(t,n){super(n),K(this,`name`,e.name),K(this,e.memberNameProperty,t)}}}function q(e,t,n){return e?new e(t,n):new G(n)}function Ie(e){let t=new Map;for(let n of e.namespaces)n.errorClass&&t.set(n.global,Fe(n.errorClass));return t}function Le(e,t){e.on(`message`,e=>{let n=t.get(e.id);if(n)if(t.delete(e.id),e.ok){let t=W(e.value);t===void 0?n.reject(new G(`binding resolution must be lossless JSON`)):n.resolve(t)}else n.reject(new G(e.message))})}function Re(e,t,n,r,i=Ie(e),a){return e.namespaces.map(({global:e,names:o})=>{let s=i.get(e),c=Object.create(null);for(let i of o)Object.defineProperty(c,i,{enumerable:!0,value:o=>{let c;try{c=B(o)}catch{c=void 0}if(c===void 0)return Promise.reject(q(s,i,`binding arguments must be lossless JSON`));let l={type:`call`,id:r.value,global:e,name:i,args:V(c)};return a!==void 0&&y(l,a)===void 0?Promise.reject(q(s,i,`binding arguments exceed maxFrameBytes`)):new Promise((e,a)=>{let o=r.value++;n.set(o,{resolve:e,reject:e=>{a(q(s,i,e.message))}});try{t.postMessage(l)}catch(e){n.delete(o),a(q(s,i,`binding arguments must be structured-cloneable: ${e instanceof G?e.message:String(e)}`))}})}});return c})}async function ze(e,t,n,r){let i=new Ee(t.maxOutputBytes,t=>{e.postMessage({type:`log`,text:t})},()=>{e.postMessage({type:`output-limit`})});ke(i,n.stdout),ke(i,n.stderr);let a=new Map;Le(e,a);let o={value:1},s=Ie(t),c=Re(t,e,a,o,s,r),l=[],u=[];for(let e of t.namespaces){if(!e.errorClass)continue;l.push(e.errorClass.name);let t=s.get(e.global);if(!t)throw new G(`missing binding error class for ${e.global}`);u.push(t)}let d=Oe(i),f;try{let e=(async()=>{}).constructor;f={type:`done`,...je(await new e(...t.namespaces.map(e=>e.global),...l,`console`,`'use strict';\\n${t.code}`)(...c,...u,d),i.remainingOutputBytes(),t.maxOutputBytes)}}catch(e){f={type:`done`,...Pe(e,i.remainingOutputBytes(),t.maxOutputBytes)}}e.postMessage(r!==void 0&&y(f,r)===void 0?{type:`output-limit`}:f)}function J(e){return e.readableEnded||e.destroyed?Promise.resolve():new Promise(t=>{let n=()=>{e.off(`end`,n),e.off(`close`,n),e.off(`error`,n),t()};e.once(`end`,n),e.once(`close`,n),e.once(`error`,n),(e.readableEnded||e.destroyed)&&n()})}const Be=new Set([`exception`,`timeout`,`abort`,`worker-exit`,`invalid-output`,`output-limit`]);let Y=0;function X(e){return typeof e==`object`&&e?e:void 0}function Z(t,n){try{let r=JSON.stringify(t);return typeof r==`string`&&e.byteLength(r)<=n?r:void 0}catch{return}}function Ve(e){process.stdout.write(e),process.stdout.write(`\n`)}function He(e){let t=Z(e,Y);return t===void 0?!1:(Ve(t),!0)}function Ue(e){let t=X(e);return t!==void 0&&typeof t.kind==`string`&&Be.has(t.kind)&&typeof t.message==`string`}function We(e){return Ue(e)&&(e.kind===`exception`||e.kind===`invalid-output`||e.kind===`output-limit`)}function Ge(e){let t=X(e);if(!(t===void 0||t.type!==`boot`||typeof t.code!=`string`||!Array.isArray(t.namespaces)||!Number.isSafeInteger(t.maxOutputBytes)||t.maxOutputBytes<4||!Number.isSafeInteger(t.maxFrameBytes)||t.maxFrameBytes{e.once(`exit`,()=>{t()})})}function $(){return{type:`done`,error:{kind:`worker-exit`,message:`code runtime bridge frame exceeded maxFrameBytes`}}}function Je(){let e=n({input:process.stdin,crlfDelay:1/0}),r,a=0,o=2,s=0,c=!1,l=t=>{if(c)return;let n=Z(t,Y)??Z($(),Y);c=!0,n!==void 0&&Ve(n);let i=r;r=void 0,(i===void 0?Promise.resolve():new Promise(e=>{setImmediate(e)}).then(async()=>{let e=J(i.stdout),t=J(i.stderr),n=qe(i);i.kill(`SIGKILL`),await Promise.all([n,e,t])})).catch(e=>{process.stderr.write(`dsh-code-runtime-subprocess controller cleanup error: ${String(e)}\\n`)}).then(()=>{e.close(),process.stdin.destroy()})},u=e=>{if(c)return;let t=+(s>0),n=v(e,a-o-t);if(n===void 0){l({type:`output-limit`});return}o+=n+t,s+=1,He({type:`log`,text:e})||l($())},d=e=>{a=e.maxOutputBytes,Y=e.maxFrameBytes,r=t(i(import.meta.url),[],{env:{DSH_CODE_RUNTIME_CONTROLLER:`1`},detached:!1,execArgv:[],stdio:[`ignore`,`pipe`,`pipe`,`ipc`]});let n=r;n.stdout.on(`data`,e=>{u(e.toString(`utf8`))}),n.stderr.on(`data`,e=>{u(e.toString(`utf8`))}),n.on(`message`,e=>{let t=X(e);if(t!==void 0){if(t.type===`log`&&typeof t.text==`string`){u(t.text);return}c||(t.type===`call`&&typeof t.id==`number`&&typeof t.global==`string`&&typeof t.name==`string`?He({type:`call`,id:t.id,global:t.global,name:t.name,args:Q(t.args)})||l($()):t.type===`output-limit`?l({type:`output-limit`}):t.type===`done`&&(t.error===void 0?l({type:`done`,...t.value===void 0?{}:{value:Q(t.value)}}):Ue(t.error)&&l({type:`done`,error:t.error})))}}),n.on(`error`,e=>{l({type:`done`,error:{kind:`worker-exit`,message:`remote controller error: ${e.message}`}})}),n.on(`exit`,e=>{c||l({type:`done`,error:{kind:`worker-exit`,message:`remote controller exited with code ${e} before completing`}})}),n.send(e,e=>{e!==null&&l({type:`done`,error:{kind:`worker-exit`,message:`remote controller boot failed: ${e.message}`}})})};e.on(`line`,e=>{let t;try{t=JSON.parse(e)}catch(e){process.stderr.write(`dsh-code-runtime-subprocess frame error: ${String(e)}\\n`),l({type:`done`,error:{kind:`worker-exit`,message:`remote runner received a malformed frame`}});return}if(r===void 0){let e=Ge(t);if(e===void 0){l({type:`done`,error:{kind:`worker-exit`,message:`remote runner received an invalid boot frame`}});return}d(e);return}let n=Ke(t);n!==void 0&&r.send(n,e=>{e!==null&&l({type:`done`,error:{kind:`worker-exit`,message:`remote controller reply failed: ${e.message}`}})})}),e.on(`close`,()=>{r!==void 0&&!c&&l({type:`done`,error:{kind:`abort`,message:`remote runner input closed`}})})}function Ye(){let e,t=!1,n,i=0,o=e=>process.send===void 0||i>0&&Z(e,i)===void 0?!1:(process.send(e),!0),s=r=>{if(t)return;t=!0,clearInterval(n);let a=i>0&&Z(r,i)===void 0?$():r,s=e;e=void 0,(s===void 0?Promise.resolve():new Promise(e=>{setImmediate(e)}).then(async()=>{let e=J(s.stdout),t=J(s.stderr);await Promise.all([s.terminate(),e,t])})).catch(e=>{o({type:`log`,text:`dsh-code-runtime-subprocess worker cleanup error: ${String(e)}\\n`})}).then(()=>{if(process.send===void 0){process.exitCode=1;return}process.send(a,()=>{process.connected&&process.disconnect()})})};process.on(`message`,c=>{if(e===void 0){let l=Ge(c);if(l===void 0){s({type:`done`,error:{kind:`worker-exit`,message:`remote controller received an invalid boot frame`}});return}i=l.maxFrameBytes,e=new a(new URL(import.meta.url),{workerData:l,env:{},execArgv:[],stdout:!0,stderr:!0,resourceLimits:{maxOldGenerationSizeMb:l.maxOldGenerationSizeMb}});let u=e;u.stdout.on(`data`,e=>{o({type:`log`,text:e.toString(`utf8`)})||s($())}),u.stderr.on(`data`,e=>{o({type:`log`,text:e.toString(`utf8`)})||s($())}),u.on(`message`,e=>{let t=X(e);t!==void 0&&(t.type===`call`&&typeof t.id==`number`&&typeof t.global==`string`&&typeof t.name==`string`?o({type:`call`,id:t.id,global:t.global,name:t.name,args:Q(t.args)})||s($()):t.type===`log`&&typeof t.text==`string`?o({type:`log`,text:t.text})||s($()):t.type===`output-limit`?s({type:`output-limit`}):t.type===`done`&&(t.error===void 0?s({type:`done`,...t.value===void 0?{}:{value:Q(t.value)}}):We(t.error)&&s({type:`done`,error:t.error})))}),u.on(`error`,e=>{s({type:`done`,error:{kind:`worker-exit`,message:`worker error: ${e.stack||e.message||r(e)}`}})}),u.on(`exit`,e=>{t||s({type:`done`,error:{kind:`worker-exit`,message:`worker exited with code ${e} before completing`}})}),n=setInterval(()=>{e!==void 0&&e.performance.eventLoopUtilization().active>l.computeMs&&s({type:`done`,error:{kind:`timeout`,message:`compute budget exhausted (${l.computeMs}ms busy)`}})},25);return}let l=Ke(c);l!==void 0&&e.postMessage(l)}),process.on(`disconnect`,()=>{e!==void 0&&!t&&e.terminate()})}if(o)process.env.DSH_CODE_RUNTIME_CONTROLLER===`1`?Ye():Je();else{if(s===null)throw Error(`remote worker requires parentPort`);let e=c;ze(s,e,{stdout:process.stdout,stderr:process.stderr},e.maxFrameBytes)}export{};" diff --git a/packages/util/atomic-write/src/runner.ts b/packages/util/atomic-write/src/runner.ts new file mode 100644 index 0000000000..7a0b3eb678 --- /dev/null +++ b/packages/util/atomic-write/src/runner.ts @@ -0,0 +1,390 @@ +/** Typed source for the dependency-free execution-world runner bundle. */ + +import { Buffer } from 'node:buffer' +import { fork } from 'node:child_process' +import type { ChildProcess } from 'node:child_process' +import { createInterface } from 'node:readline' +import type { Readable } from 'node:stream' +import { inspect } from 'node:util' +import { fileURLToPath } from 'node:url' +import { Worker, isMainThread, parentPort, workerData } from 'node:worker_threads' +import { + decodeWorkerJson, + encodeWorkerJson, + jsonStringBytesUpTo, + runWorkerMain, + waitForRuntimePipeDrain, +} from '@deepseek-ai/dsh-code-runtime-worker/runtime-host' +import type { WorkerJsonWire } from '@deepseek-ai/dsh-code-runtime-worker/runtime-host' + +type WorkerBootData = Parameters[1] +type Controller = ChildProcess & { stdout: Readable; stderr: Readable } +type FailureKind = 'exception' | 'timeout' | 'abort' | 'worker-exit' | 'invalid-output' | 'output-limit' + +interface RuntimeBootData extends WorkerBootData { + type: 'boot' + maxFrameBytes: number + maxOldGenerationSizeMb: number + computeMs: number +} + +interface RuntimeFailure { + kind: FailureKind + message: string +} + +interface RuntimeCall { + type: 'call' + id: number + global: string + name: string + args: WorkerJsonWire | null +} + +type RuntimeReply = + | { type: 'reply'; id: number; ok: true; value: unknown } + | { type: 'reply'; id: number; ok: false; message: string } + +type RuntimeMessage = RuntimeCall + | { type: 'log'; text: string } + | { type: 'output-limit' } + | { type: 'done'; value?: WorkerJsonWire | null; error?: RuntimeFailure } + +const failureKinds = new Set([ + 'exception', 'timeout', 'abort', 'worker-exit', 'invalid-output', 'output-limit', +]) + +let maxFrameBytes = 0 + +function recordOf(value: unknown): Record | undefined { + return typeof value === 'object' && value !== null ? value as Record : undefined +} + +function encodeJsonBounded(value: unknown, maxBytes: number): string | undefined { + try { + const json: unknown = JSON.stringify(value) + return typeof json === 'string' && Buffer.byteLength(json) <= maxBytes ? json : undefined + } catch { + return undefined + } +} + +function emitJson(json: string): void { + process.stdout.write(json) + process.stdout.write('\n') +} + +function emitFrame(message: RuntimeMessage): boolean { + const json = encodeJsonBounded(message, maxFrameBytes) + if (json === undefined) return false + emitJson(json) + return true +} + +function validFailure(value: unknown): value is RuntimeFailure { + const record = recordOf(value) + return record !== undefined + && typeof record.kind === 'string' + && failureKinds.has(record.kind as FailureKind) + && typeof record.message === 'string' +} + +function validWorkerFailure(value: unknown): value is RuntimeFailure { + return validFailure(value) + && (value.kind === 'exception' || value.kind === 'invalid-output' || value.kind === 'output-limit') +} + +function runtimeBoot(value: unknown): RuntimeBootData | undefined { + const record = recordOf(value) + if (record === undefined + || record.type !== 'boot' + || typeof record.code !== 'string' + || !Array.isArray(record.namespaces) + || !Number.isSafeInteger(record.maxOutputBytes) + || (record.maxOutputBytes as number) < 4 + || !Number.isSafeInteger(record.maxFrameBytes) + || (record.maxFrameBytes as number) < (record.maxOutputBytes as number) + || typeof record.computeMs !== 'number' + || !Number.isFinite(record.computeMs) + || (record.computeMs) <= 0 + || typeof record.maxOldGenerationSizeMb !== 'number' + || !Number.isFinite(record.maxOldGenerationSizeMb) + || (record.maxOldGenerationSizeMb) <= 0) return undefined + return record as unknown as RuntimeBootData +} + +function runtimeReply(value: unknown): RuntimeReply | undefined { + const record = recordOf(value) + if (record === undefined || record.type !== 'reply' || typeof record.id !== 'number' || typeof record.ok !== 'boolean') return undefined + return record.ok + ? { type: 'reply', id: record.id, ok: true, value: record.value } + : { type: 'reply', id: record.id, ok: false, message: String(record.message) } +} + +function transportWireOrNull(input: unknown): WorkerJsonWire | null { + const value = decodeWorkerJson(input) + return value === undefined ? null : encodeWorkerJson(value) +} + +function waitForChildExit(child: ChildProcess): Promise { + if (child.exitCode !== null || child.signalCode !== null) return Promise.resolve() + return new Promise((resolve) => { child.once('exit', () => { resolve() }) }) +} + +function frameLimitFailure(): RuntimeMessage { + return { + type: 'done', + error: { kind: 'worker-exit', message: 'code runtime bridge frame exceeded maxFrameBytes' }, + } +} + +function runLauncher(): void { + const input = createInterface({ input: process.stdin, crlfDelay: Infinity }) + let controller: Controller | undefined + let maxOutputBytes = 0 + let logBytes = 2 + let logEntries = 0 + let settling = false + + const finish = (message: RuntimeMessage): void => { + if (settling) return + const encoded = encodeJsonBounded(message, maxFrameBytes) + ?? encodeJsonBounded(frameLimitFailure(), maxFrameBytes) + settling = true + if (encoded !== undefined) emitJson(encoded) + const current = controller + controller = undefined + const drain = current === undefined + ? Promise.resolve() + : new Promise((resolve) => { setImmediate(resolve) }).then(async () => { + const stdoutDrained = waitForRuntimePipeDrain(current.stdout) + const stderrDrained = waitForRuntimePipeDrain(current.stderr) + const exited = waitForChildExit(current) + current.kill('SIGKILL') + await Promise.all([exited, stdoutDrained, stderrDrained]) + }) + void drain.catch((error: unknown) => { + process.stderr.write(`dsh-code-runtime-subprocess controller cleanup error: ${String(error)}\n`) + }).then(() => { + input.close() + process.stdin.destroy() + }) + } + + const forwardLog = (text: string): void => { + if (settling) return + const separator = logEntries > 0 ? 1 : 0 + const cost = jsonStringBytesUpTo(text, maxOutputBytes - logBytes - separator) + if (cost === undefined) { + finish({ type: 'output-limit' }) + return + } + logBytes += cost + separator + logEntries += 1 + if (!emitFrame({ type: 'log', text })) finish(frameLimitFailure()) + } + + const startController = (boot: RuntimeBootData): void => { + maxOutputBytes = boot.maxOutputBytes + maxFrameBytes = boot.maxFrameBytes + controller = fork(fileURLToPath(import.meta.url), [], { + env: { DSH_CODE_RUNTIME_CONTROLLER: '1' }, + detached: false, + execArgv: [], + stdio: ['ignore', 'pipe', 'pipe', 'ipc'], + }) as Controller + const current = controller + current.stdout.on('data', (data: Buffer) => { forwardLog(data.toString('utf8')) }) + current.stderr.on('data', (data: Buffer) => { forwardLog(data.toString('utf8')) }) + current.on('message', (raw: unknown) => { + const message = recordOf(raw) + if (message === undefined) return + if (message.type === 'log' && typeof message.text === 'string') { + forwardLog(message.text) + return + } + if (settling) return + if (message.type === 'call' + && typeof message.id === 'number' + && typeof message.global === 'string' + && typeof message.name === 'string') { + if (!emitFrame({ + type: 'call', id: message.id, global: message.global, name: message.name, args: transportWireOrNull(message.args), + })) finish(frameLimitFailure()) + } else if (message.type === 'output-limit') { + finish({ type: 'output-limit' }) + } else if (message.type === 'done') { + if (message.error !== undefined) { + if (validFailure(message.error)) finish({ type: 'done', error: message.error }) + } else { + finish({ type: 'done', ...message.value === undefined ? {} : { value: transportWireOrNull(message.value) } }) + } + } + }) + current.on('error', (error: Error) => { + finish({ type: 'done', error: { kind: 'worker-exit', message: `remote controller error: ${error.message}` } }) + }) + current.on('exit', (code: number | null) => { + if (!settling) { + finish({ type: 'done', error: { kind: 'worker-exit', message: `remote controller exited with code ${code} before completing` } }) + } + }) + current.send(boot, (error: Error | null) => { + if (error !== null) { + finish({ type: 'done', error: { kind: 'worker-exit', message: `remote controller boot failed: ${error.message}` } }) + } + }) + } + + input.on('line', (line: string) => { + let raw: unknown + try { + raw = JSON.parse(line) as unknown + } catch (error: unknown) { + process.stderr.write(`dsh-code-runtime-subprocess frame error: ${String(error)}\n`) + finish({ type: 'done', error: { kind: 'worker-exit', message: 'remote runner received a malformed frame' } }) + return + } + if (controller === undefined) { + const boot = runtimeBoot(raw) + if (boot === undefined) { + finish({ type: 'done', error: { kind: 'worker-exit', message: 'remote runner received an invalid boot frame' } }) + return + } + startController(boot) + return + } + const reply = runtimeReply(raw) + if (reply !== undefined) { + controller.send(reply, (error: Error | null) => { + if (error !== null) { + finish({ type: 'done', error: { kind: 'worker-exit', message: `remote controller reply failed: ${error.message}` } }) + } + }) + } + }) + input.on('close', () => { + if (controller !== undefined && !settling) { + finish({ type: 'done', error: { kind: 'abort', message: 'remote runner input closed' } }) + } + }) +} + +function runController(): void { + let worker: Worker | undefined + let finished = false + let computeTimer: NodeJS.Timeout | undefined + let controllerMaxFrameBytes = 0 + + const send = (message: RuntimeMessage): boolean => { + if (process.send === undefined) return false + if (controllerMaxFrameBytes > 0 && encodeJsonBounded(message, controllerMaxFrameBytes) === undefined) return false + process.send(message) + return true + } + + const finish = (message: RuntimeMessage): void => { + if (finished) return + finished = true + clearInterval(computeTimer) + const bounded = controllerMaxFrameBytes > 0 && encodeJsonBounded(message, controllerMaxFrameBytes) === undefined + ? frameLimitFailure() + : message + const current = worker + worker = undefined + const drain = current === undefined + ? Promise.resolve() + : new Promise((resolve) => { setImmediate(resolve) }).then(async () => { + const stdoutDrained = waitForRuntimePipeDrain(current.stdout) + const stderrDrained = waitForRuntimePipeDrain(current.stderr) + await Promise.all([current.terminate(), stdoutDrained, stderrDrained]) + }) + void drain.catch((error: unknown) => { + send({ type: 'log', text: `dsh-code-runtime-subprocess worker cleanup error: ${String(error)}\n` }) + }).then(() => { + if (process.send === undefined) { + process.exitCode = 1 + return + } + process.send(bounded, () => { if (process.connected) process.disconnect() }) + }) + } + + process.on('message', (raw: unknown) => { + if (worker === undefined) { + const boot = runtimeBoot(raw) + if (boot === undefined) { + finish({ type: 'done', error: { kind: 'worker-exit', message: 'remote controller received an invalid boot frame' } }) + return + } + controllerMaxFrameBytes = boot.maxFrameBytes + worker = new Worker(new URL(import.meta.url), { + workerData: boot, + env: {}, + execArgv: [], + stdout: true, + stderr: true, + resourceLimits: { maxOldGenerationSizeMb: boot.maxOldGenerationSizeMb }, + }) + const current = worker + current.stdout.on('data', (data: Buffer) => { + if (!send({ type: 'log', text: data.toString('utf8') })) finish(frameLimitFailure()) + }) + current.stderr.on('data', (data: Buffer) => { + if (!send({ type: 'log', text: data.toString('utf8') })) finish(frameLimitFailure()) + }) + current.on('message', (messageRaw: unknown) => { + const message = recordOf(messageRaw) + if (message === undefined) return + if (message.type === 'call' + && typeof message.id === 'number' + && typeof message.global === 'string' + && typeof message.name === 'string') { + if (!send({ + type: 'call', id: message.id, global: message.global, name: message.name, args: transportWireOrNull(message.args), + })) finish(frameLimitFailure()) + } else if (message.type === 'log' && typeof message.text === 'string') { + if (!send({ type: 'log', text: message.text })) finish(frameLimitFailure()) + } else if (message.type === 'output-limit') { + finish({ type: 'output-limit' }) + } else if (message.type === 'done') { + if (message.error !== undefined) { + if (validWorkerFailure(message.error)) finish({ type: 'done', error: message.error }) + } else { + finish({ type: 'done', ...message.value === undefined ? {} : { value: transportWireOrNull(message.value) } }) + } + } + }) + current.on('error', (error: Error) => { + finish({ + type: 'done', + error: { kind: 'worker-exit', message: `worker error: ${error.stack || error.message || inspect(error)}` }, + }) + }) + current.on('exit', (code: number) => { + if (!finished) { + finish({ type: 'done', error: { kind: 'worker-exit', message: `worker exited with code ${code} before completing` } }) + } + }) + computeTimer = setInterval(() => { + if (worker !== undefined && worker.performance.eventLoopUtilization().active > boot.computeMs) { + finish({ type: 'done', error: { kind: 'timeout', message: `compute budget exhausted (${boot.computeMs}ms busy)` } }) + } + }, 25) + return + } + const reply = runtimeReply(raw) + if (reply !== undefined) worker.postMessage(reply) + }) + process.on('disconnect', () => { if (worker !== undefined && !finished) void worker.terminate() }) +} + +if (!isMainThread) { + if (parentPort === null) throw new Error('remote worker requires parentPort') + const boot = workerData as RuntimeBootData + void runWorkerMain(parentPort, boot, { stdout: process.stdout, stderr: process.stderr }, boot.maxFrameBytes) +} else if (process.env.DSH_CODE_RUNTIME_CONTROLLER === '1') { + runController() +} else { + runLauncher() +} diff --git a/scripts/gen-code-runtime-runner.ts b/scripts/gen-code-runtime-runner.ts new file mode 100644 index 0000000000..7e856e7651 --- /dev/null +++ b/scripts/gen-code-runtime-runner.ts @@ -0,0 +1,70 @@ +/** Generate the subprocess Code Runtime's dependency-free runner bundle. */ + +import { existsSync, readFileSync, writeFileSync } from 'node:fs' +import { resolve } from 'node:path' +import { build } from 'tsdown' + +const root = resolve(import.meta.dirname, '..') +const ENTRY = 'packages/code-runtime/code-runtime-subprocess/src/runner.ts' +const OUT = 'packages/code-runtime/code-runtime-subprocess/src/runner-source.generated.ts' + +/** + * Bundle the typed runner and shared worker implementation into one source literal. + * @returns generated TypeScript module consumed by the subprocess backend. + */ +export async function renderCodeRuntimeRunner(): Promise { + const bundles = await build({ + config: false, + entry: [resolve(root, ENTRY)], + format: ['esm'], + platform: 'node', + target: 'es2024', + write: false, + dts: false, + clean: false, + minify: true, + logLevel: 'silent', + report: false, + deps: { alwaysBundle: ['@deepseek-ai/dsh-code-runtime-worker'] }, + }) + try { + const chunks = bundles.flatMap(bundle => bundle.chunks).filter(chunk => chunk.type === 'chunk') + if (chunks.length !== 1) throw new Error(`gen-code-runtime-runner: expected one chunk, received ${chunks.length}`) + const chunk = chunks[0] + if (chunk === undefined) throw new Error('gen-code-runtime-runner: runner chunk is missing') + const external = chunk.imports.filter(specifier => !specifier.startsWith('node:')) + if (external.length > 0) { + throw new Error(`gen-code-runtime-runner: runner retained external imports: ${external.join(', ')}`) + } + return [ + '/**', + ' * Generated dependency-free execution-world runner.', + ' * Do not edit by hand; run `pnpm run gen-code-runtime-runner`.', + ' */', + '', + `export const CODE_RUNNER_SOURCE = ${JSON.stringify(chunk.code)}`, + '', + ].join('\n') + } finally { + await Promise.all(bundles.map(async (bundle) => { await bundle[Symbol.asyncDispose]() })) + } +} + +async function main(): Promise { + const content = await renderCodeRuntimeRunner() + const output = resolve(root, OUT) + if (process.argv.includes('--check')) { + const committed = existsSync(output) ? readFileSync(output, 'utf8') : null + if (committed === content) { + console.log(`gen-code-runtime-runner: ${OUT} is up to date.`) + return + } + console.error(`gen-code-runtime-runner: ${OUT} is stale. Run \`pnpm run gen-code-runtime-runner\` and commit it.`) + process.exitCode = 1 + return + } + writeFileSync(output, content) + console.log(`gen-code-runtime-runner: wrote ${OUT}.`) +} + +if (process.argv[1] && import.meta.filename === resolve(process.argv[1])) await main() diff --git a/scripts/run-gates.ts b/scripts/run-gates.ts index 74a09a874e..6f59b64826 100644 --- a/scripts/run-gates.ts +++ b/scripts/run-gates.ts @@ -585,6 +585,7 @@ function docSyncLeafGates(options: { pnpmScript('persistence-catalog', 'verify-persistence-catalog', { label: 'persistence catalog' }), pnpmScript('doc-graphs', 'verify-doc-graphs', { label: 'doc graphs' }), pnpmScript('scoped-events', 'verify-scoped-events', { label: 'scoped events' }), + pnpmScript('code-runtime-runner', 'verify-code-runtime-runner', { label: 'code-runtime runner' }), pnpmScript('markdown-wrap', 'verify-md-wrap', { label: 'markdown wrap' }), pnpmScript('markdown-links', 'verify-md-links', { label: 'markdown links' }), pnpmScript('public-repository-links', 'verify-public-repository-links', { label: 'public repository links' }), diff --git a/vitest.config.ts b/vitest.config.ts index 6eb1bda354..996f2456c7 100644 --- a/vitest.config.ts +++ b/vitest.config.ts @@ -1,16 +1,5 @@ -import { spawnSync } from 'node:child_process' -import { fileURLToPath } from 'node:url' import tsconfigPaths from 'vite-tsconfig-paths' -import { resolvePwshPath } from './packages/bash/pwsh-local/src/resolve.ts' import { defineConfig } from 'vitest/config' -import { standardDecoratorPlugin, vitestExecArgv } from './vitest.shared.ts' -import { COVERAGE_EXEMPT_ENV, coverageExemptHeavySuites } from './scripts/coverage-exempt.ts' - -// Prints exact `path:line:col` records for every uncovered statement, branch -// path, and function when a file misses the per-file 100% gate — the built-in -// threshold ERRORs name only the file. Absolute path because istanbul-reports -// require()s custom reporters (which is also why the reporter is CJS). -const uncoveredLocationsReporter = fileURLToPath(new URL('./scripts/coverage-uncovered-locations.cjs', import.meta.url)) // Resolution facade shared by every plugin instance below: tsconfig.base.json // has no include, which vite-tsconfig-paths treats as match-all, so its paths @@ -20,14 +9,7 @@ const pathsPlugin = (): ReturnType => tsconfigPaths({ proj const windowsUnsupportedPackages = process.platform === 'win32' ? [ - // Bash-requiring suites (a real POSIX shell is unavailable on Windows). - // The pwsh-requiring suites (pwsh-local, tool-pwsh) deliberately stay - // INCLUDED: PowerShell ships with Windows, so they run natively here. - // Replacing the old 'packages/bash/*' glob with this explicit list also - // newly INCLUDES packages/bash/bash (the pure seam package) on Windows. - 'packages/bash/bash-local', - 'packages/bash/bash-sandbox', - 'packages/bash/tool-bash', + 'packages/bash/*', 'packages/hooks/*', 'packages/subprocess/*', 'packages/pty/pty-local', @@ -44,20 +26,10 @@ const windowsCoverageExclusions = process.platform === 'win32' 'packages/lsp/lsp-local/src/connection.ts', 'packages/lsp/lsp-local/src/index.ts', 'packages/lsp/lsp-local/src/instance.ts', + 'packages/ui/tui/src/index.ts', ] : [] -// Mirrors windowsCoverageExclusions: pwsh-local's run/start/lifecycle suites -// self-skip without a real pwsh (executor.spec.ts hasPwsh), leaving this file -// far below per-file 100% on pwsh-less hosts; the exemption keeps those hosts -// green while CI runners ship pwsh and still enforce the full bar. The probe -// runs the suites' own resolution (the dependency-free resolve.ts module), -// so the exemption is active exactly when the suites skip — a mismatched -// narrower probe could exempt the file on hosts whose suites actually run. -const pwshCoverageExclusions = spawnSync(resolvePwshPath(), ['-NoLogo', '-NoProfile', '-NonInteractive', '-Command', '$true'], { encoding: 'utf8' }).status === 0 - ? [] - : ['packages/bash/pwsh-local/src/index.ts'] - const testIncludes = [ 'packages/*/*/tests/**/*.spec.{ts,tsx}', 'apps/*/tests/**/*.spec.ts', @@ -65,17 +37,6 @@ const testIncludes = [ 'scripts/**/*.spec.ts', ] -// The instrumented coverage gate sets this env; the exempt heavy suites then -// run beside it uninstrumented (membership contract in scripts/coverage-exempt.ts). -// A set-but-not-'1' value is a misconfiguration, not a silent no-op. -const coverageExemptRaw = process.env[COVERAGE_EXEMPT_ENV] -if (coverageExemptRaw !== undefined && coverageExemptRaw !== '' && coverageExemptRaw !== '1') { - throw new Error(`vitest config: ${COVERAGE_EXEMPT_ENV} must be '1' or unset, got ${JSON.stringify(coverageExemptRaw)}.`) -} -const coverageExemptExcludes = coverageExemptRaw === '1' - ? coverageExemptHeavySuites.map(suite => suite.exclude) - : [] - // These suites exercise process-global state, process APIs, or timing-sensitive process I/O // that worker threads cannot isolate reliably under aggregate gate contention. // Keep the narrow exception in forks while the rest of the inventory avoids per-file processes. @@ -88,48 +49,39 @@ const processBoundTests = [ ] export default defineConfig({ - plugins: [pathsPlugin(), standardDecoratorPlugin()], + plugins: [pathsPlugin()], test: { setupFiles: ['./scripts/test-invariants.ts'], // .tsx: client component specs (jsdom via per-file @vitest-environment pragma). include: testIncludes, exclude: windowsUnsupportedPackages.map(path => `${path}/tests/**/*.spec.ts`), - // One coverage invocation aggregates both projects. Regular suites fork on - // POSIX for Node stability and use threads on Windows; process-bound suites - // always fork. + // One coverage invocation aggregates both projects. Most suites use threads + // for lower startup/IPC overhead; only explicit process-bound suites fork. projects: [ { - plugins: [pathsPlugin(), standardDecoratorPlugin()], + plugins: [pathsPlugin()], test: { name: 'thread-safe', - execArgv: vitestExecArgv, - // Node 24 has aborted in its CJS lexer (v8::ToLocalChecked Empty - // MaybeLocal in cjs_lexer::Parse) from worker threads on macOS - // arm64 and later on Linux. A fork contains that external runtime - // failure to the test process; Windows keeps the thread pool, where - // the abort has not reproduced and process spawn is costlier. - pool: process.platform === 'win32' ? 'threads' : 'forks', + // Node 24 has aborted in its CJS lexer from a macOS arm64 worker + // thread. A fork contains that external runtime failure to the test + // process; other hosts retain the lower-overhead thread pool. + pool: process.platform === 'darwin' ? 'forks' : 'threads', setupFiles: ['./scripts/test-invariants.ts'], include: testIncludes, exclude: [ ...windowsUnsupportedPackages.map(path => `${path}/tests/**/*.spec.ts`), ...processBoundTests, - ...coverageExemptExcludes, ], }, }, { - plugins: [pathsPlugin(), standardDecoratorPlugin()], + plugins: [pathsPlugin()], test: { name: 'process-bound', - execArgv: vitestExecArgv, pool: 'forks', setupFiles: ['./scripts/test-invariants.ts'], include: processBoundTests, - exclude: [ - ...windowsUnsupportedPackages.map(path => `${path}/tests/**/*.spec.ts`), - ...coverageExemptExcludes, - ], + exclude: windowsUnsupportedPackages.map(path => `${path}/tests/**/*.spec.ts`), }, }, ], @@ -146,8 +98,7 @@ export default defineConfig({ 'packages/*/*/src/types.ts', 'packages/*/*/src/bin.ts', 'packages/*/*/src/worker.ts', - // A killed executable lint-contract test can leave a non-product source probe behind. - 'packages/*/*/src/oxlint-contract-*.ts', + 'packages/code-runtime/code-runtime-subprocess/src/runner.ts', // GUI step-1 skeleton (PR #500): client/web UI files whose remaining // branches need a browser-grade harness the jsdom lane doesn't cover // yet. TODO(gui): cover and remove as the client test lane matures. @@ -156,7 +107,6 @@ export default defineConfig({ 'packages/client/ui-primitives/src/markdown/plain-text.ts', 'packages/client/ui-question/src/client/QuestionComposer.tsx', 'packages/client/ui-primitives/src/Menu.tsx', - 'packages/client/ui-primitives/src/RiskConfirmation.tsx', 'packages/client/ui-workspace/src/client/WorkspaceBrowser.tsx', 'packages/client/ui-workspace/src/client/WorkspacePicker.tsx', 'packages/client/web-react/src/*', @@ -181,10 +131,6 @@ export default defineConfig({ 'packages/client/hmr/src/invariant.ts', 'packages/client/connection/src/index.ts', 'packages/client/connection/src/http-bridge.ts', - // This assembly imports generated Host-for-Client code that exists - // only in lib; the post-build built-bin smoke executes both entries. - 'packages/api/remotes/src/index.ts', - 'packages/api/remotes/src/client/index.ts', // Slash/command/input round: per-file gaps deferred with the same // client-lane debt. TODO(gui): cover and remove with the lane above. 'packages/client/connection/src/client/fixture.ts', @@ -209,13 +155,9 @@ export default defineConfig({ 'packages/client/ui-sidebar/src/client/index.ts', 'packages/client/ui-skill/src/client/index.ts', 'packages/client/ui-workspace/src/client/index.ts', - 'packages/client/test-runtime/src/translate.ts', - 'packages/client/ui-primitives/src/JsonTree.tsx', - // Typert generator: correctness is pinned by its fixture suites and - // the byte-for-byte catalog reproduction test; per-file coverage - // would put whole-workspace compiler analysis under v8 - // instrumentation — the coverage lane's longest tail. - 'packages/typert/generator/src/*.ts', + 'packages/typert/generator/src/analyzer.ts', + 'packages/typert/generator/src/renderer.ts', + 'packages/typert/generator/src/cordis-catalog.ts', 'packages/host/apiproxy/src/index.ts', 'packages/host/apiproxy/src/invariant.ts', 'packages/host/apiproxy/src/api-proxy.ts', @@ -225,9 +167,9 @@ export default defineConfig({ 'packages/ui/commands/src/index.ts', 'packages/ui/commands/src/invariant.ts', 'packages/session-projection/session-projection/src/index.ts', + 'packages/ui/tui/src/index.ts', ...windowsUnsupportedPackages.map(path => `${path}/src/**/*.ts`), ...windowsCoverageExclusions, - ...pwshCoverageExclusions, ], // 100% or it doesn't merge (docs/testing.md: excessive tests are welcome). // Per-file so a well-covered big file can't subsidize a bare one. @@ -240,9 +182,7 @@ export default defineConfig({ functions: 100, lines: 100, }, - reporter: process.env.CI - ? ['text', uncoveredLocationsReporter] - : ['text', 'html', uncoveredLocationsReporter], + reporter: process.env.CI ? ['text'] : ['text', 'html'], }, }, }) From e8409cf928a3f848f8c2e6a8ab85c32ecbd52de6 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Thu, 30 Jul 2026 02:27:12 +0800 Subject: [PATCH 047/108] fix(code-runtime): track generated runner source --- knip.json | 1 + .../src/runner-source.generated.ts | 2 +- packages/util/atomic-write/src/runner.ts | 24 +++++++++++-------- 3 files changed, 16 insertions(+), 11 deletions(-) diff --git a/knip.json b/knip.json index 20bcb5ff2d..413b740578 100644 --- a/knip.json +++ b/knip.json @@ -385,6 +385,7 @@ }, "packages/code-runtime/code-runtime-subprocess": { "entry": [ + "src/runner.ts", "tests/**/*.spec.ts", "tests/**/*.e2e.ts" ], diff --git a/packages/util/atomic-write/src/runner-source.generated.ts b/packages/util/atomic-write/src/runner-source.generated.ts index d40c77783c..8592fa2399 100644 --- a/packages/util/atomic-write/src/runner-source.generated.ts +++ b/packages/util/atomic-write/src/runner-source.generated.ts @@ -3,4 +3,4 @@ * Do not edit by hand; run `pnpm run gen-code-runtime-runner`. */ -export const CODE_RUNNER_SOURCE = "import{Buffer as e}from\"node:buffer\";import{fork as t}from\"node:child_process\";import{createInterface as n}from\"node:readline\";import{inspect as r}from\"node:util\";import{fileURLToPath as i}from\"node:url\";import{Worker as a,isMainThread as o,parentPort as s,workerData as c}from\"node:worker_threads\";const l=Reflect.apply,u=Array.isArray,d=Buffer,f=Reflect.get(Buffer,`byteLength`),ee=Object.create,p=Object.defineProperty,te=Object.keys,ne=String,re=Reflect.get(String.prototype,`charCodeAt`),ie=Reflect.get(String.prototype,`codePointAt`),m=Reflect.get(String.prototype,`slice`);function h(e){let t=ee(null);return t.value=e,t}function ae(e,t,n){let r=h(n);r.enumerable=!0,r.configurable=!0,r.writable=!0,p(e,t,r)}function g(e){return l(f,d,[e,`utf8`])}function _(e,t){ae(e,e.length,t)}function oe(e){if(e.length===0)return;let t=e.length-1,n=e[t];return p(e,`length`,h(t)),n}function se(e,t){return l(m,e,[t,t+(l(ie,e,[t])>65535?2:1)])}function ce(e){if(e.length===2)return 4;if(e===`\"`||e===`\\\\`)return 2;let t=l(re,e,[0]);return t>=55296&&t<=57343?6:t<32?t===8||t===9||t===10||t===12||t===13?2:6:g(e)}function v(e,t){if(t<2)return;let n=2;for(let r=0;rt)return;r+=i.length}return n}function y(e,t){let n=0,r=e=>(n+=e,n<=t),i=[{kind:`value`,value:e}];for(let e=oe(i);e!==void 0;e=oe(i)){if(e.kind===`value`){let a=e.value;if(a===null){if(!r(4))return}else if(typeof a==`string`){let e=v(a,t-n);if(e===void 0)return;n+=e}else if(typeof a==`number`){if(!r(g(ne(a))))return}else if(typeof a==`boolean`){if(!r(a?4:5))return}else if(u(a)){if(!r(2))return;a.length>0&&_(i,{kind:`array`,value:a,index:0})}else{if(!r(2))return;let e=te(a);e.length>0&&_(i,{kind:`object`,value:a,keys:e,index:0})}continue}if(e.index>0&&!r(1))return;if(e.kind===`array`){let t=e.value[e.index];if(t===void 0)return;e.index+1t)break;n+=o,r+=a.length,i+=a.length}return r===e.length?e:l(m,e,[0,r])}const ue=Reflect.get(Function.prototype,`toString`),b=Reflect.get(Reflect,`apply`),x=Error,S=Set,C=Array.isArray,de=Array.prototype,w=Number.isFinite,fe=Number.isSafeInteger,pe=Object.create,me=Object.defineProperty,he=Object.getOwnPropertyDescriptor,T=Object.getPrototypeOf,ge=Object.hasOwn,E=Object.is,_e=Object.keys,D=Object.prototype,ve=Reflect.get(D,`propertyIsEnumerable`),O=Reflect.ownKeys,ye=Reflect.get(Set.prototype,`add`),be=Reflect.get(Set.prototype,`delete`),xe=Reflect.get(Set.prototype,`has`);function k(e){let t=pe(null);return t.value=e,t}function A(e,t,n){let r=k(n);r.enumerable=!0,r.configurable=!0,r.writable=!0,me(e,t,r)}function j(e,t){A(e,e.length,t)}function M(e){if(e.length===0)return;let t=e.length-1,n=e[t];return me(e,`length`,k(t)),n}function N(e,t){return b(xe,e,[t])}function P(e,t){b(ye,e,[t])}function Se(e,t){b(be,e,[t])}function F(e,t){let n=he(e,`constructor`)?.value;if(typeof n!=`function`)return!1;try{return n.name===t&&n.prototype===e&&b(ue,n,[])===`function ${t}() { [native code] }`}catch{return!1}}function I(e){return T(e)===null&&F(e,`Object`)}function L(e){let t=T(e);if(t===de)return!0;if(!C(t)||!F(t,`Array`))return!1;let n=T(t);return typeof n==`object`&&!!n&&I(n)}function R(e){let t=T(e);return t===null||t===D||typeof t==`object`&&I(t)}function z(e){let t=O(e);for(let n=0;n{e.kind===`root`?n=t:e.kind===`array`?A(e.target,e.index,t):A(e.target,e.key,t)},i=[{kind:`visit`,value:e,destination:{kind:`root`}}];for(let e=M(i);e!==void 0;e=M(i)){if(e.kind===`leave`){Se(t,e.source);continue}if(e.kind===`array-item`){if(!ge(e.source,e.index))return;j(i,{kind:`visit`,value:e.source[e.index],destination:{kind:`array`,target:e.target,index:e.index}});continue}if(e.kind===`object-property`){j(i,{kind:`visit`,value:e.source[e.key],destination:{kind:`object`,target:e.target,key:e.key}});continue}let n=e.value;if(n===null){r(e.destination,null);continue}if(typeof n==`boolean`||typeof n==`string`){r(e.destination,n);continue}if(typeof n==`number`){if(!w(n)||E(n,-0))return;r(e.destination,n);continue}if(typeof n!=`object`||N(t,n))return;if(C(n)){if(!L(n))return;let a=n.length;if(O(n).length!==a+1)return;let o=[];r(e.destination,o),P(t,n),j(i,{kind:`leave`,source:n});for(let e=a-1;e>=0;e--)j(i,{kind:`array-item`,source:n,index:e,target:o});continue}if(!R(n))return;let a=z(n);if(a===void 0)return;let o={};r(e.destination,o),P(t,n),j(i,{kind:`leave`,source:n});for(let e=a.length-1;e>=0;e--){let t=a[e];if(t===void 0)return;j(i,{kind:`object-property`,source:n,key:t,target:o})}}return n}function V(e){let t=[],n=[e];for(let e=M(n);e!==void 0;e=M(n)){if(e===null||typeof e==`boolean`||typeof e==`number`||typeof e==`string`){j(t,e);continue}if(C(e)){j(t,{kind:`array`,length:e.length});for(let t=e.length-1;t>=0;t--){let r=e[t];if(r===void 0)throw new x(`cannot encode a sparse JSON array`);j(n,r)}continue}let r=_e(e);j(t,{kind:`object`,keys:r});for(let t=r.length-1;t>=0;t--){let i=r[t];if(i===void 0)throw new x(`cannot encode a missing JSON object key`);let a=e[i];if(a===void 0)throw new x(`cannot encode an undefined JSON object property`);j(n,a)}}return t}function H(e){if(!L(e)||O(e).length!==e.length+1)return!1;for(let t=0;t=0?{kind:`array`,length:e}:void 0}if(n.kind===`object`){if(t.length!==2||!U(t,`kind`)||!U(t,`keys`))return;let e=n.keys;if(!C(e)||!H(e))return;let r=new S,i=[],a=e;for(let e=0;e{let t=n[n.length-1];if(!t)return i?!1:(r=e,i=!0,!0);if(t.index>=(t.kind===`array`?t.length:t.keys.length))return!1;if(t.kind===`array`)j(t.target,e);else{let n=t.keys[t.index];if(n===void 0)return!1;A(t.target,n,e)}return t.index+=1,!0};for(let e=0;ea)return;let e=[];i=e,n.length>0&&(o={kind:`array`,target:e,length:n.length,index:0})}else{if(n.keys.length>a)return;let e={};i=e,n.keys.length>0&&(o={kind:`object`,target:e,keys:n.keys,index:0})}}if(!a(i))return;for(o&&j(n,o);n.length>0;){let e=n[n.length-1];if(e===void 0||e.index<(e.kind===`array`?e.length:e.keys.length))break;M(n)}}return n.length===0?r:void 0}catch{return}}const G=Error,we=Object.create,Te=Object.defineProperty;function K(e,t,n){let r=we(null);r.enumerable=!0,r.value=n,Te(e,t,r)}var Ee=class{bytes=2;entries=0;truncated=!1;sink;onLimit;maxBytes;constructor(e,t,n=()=>{}){this.maxBytes=e,this.sink=t,this.onLimit=n}push(e){if(this.truncated)return;let t=+(this.entries>0),n=this.maxBytes-this.bytes-t,r=v(e,n);if(r===void 0){this.truncated=!0;let r=le(e,n);if(r.length>0){let e=v(r,n);if(e===void 0)throw new G(`worker output ledger produced an oversized log prefix`);this.bytes+=e+t,this.entries+=1,this.sink(r)}this.onLimit();return}this.bytes+=r+t,this.entries+=1,this.sink(e)}remainingOutputBytes(){return this.maxBytes-this.bytes}};const De=[`log`,`info`,`warn`,`error`,`debug`];function Oe(e){let t=e=>e.map(e=>typeof e==`string`?e:r(e,Ae)).join(` `),n=Object.create(null);for(let r of De)n[r]=(...n)=>{e.push(t(n))};return n}function ke(e,t){let n=t.write;return t.write=(t,...n)=>{e.push(typeof t==`string`?t:String(t));let r=[n[0],n[1]].find(e=>typeof e==`function`);return r&&queueMicrotask(()=>{r(null)}),!0},()=>{t.write=n}}const Ae={depth:4,maxArrayLength:100,maxStringLength:1e4};function je(e,t,n=t){if(e===void 0)return{};let r;try{r=B(e)}catch{r=void 0}return r===void 0?Ne(`invalid-output`,`program completion must be lossless JSON`,t,n):y(r,t)===void 0?Me(n):{value:V(r)}}function Me(e){return{error:{kind:`output-limit`,message:`outer output exceeded ${e} bytes`}}}function Ne(e,t,n,r){return v(t,n)===void 0?Me(r):{error:{kind:e,message:t}}}function Pe(e,t,n=t){let r;try{let t=e instanceof G?e.stack??e.message:e;r=typeof t==`string`?t:String(t)}catch{r=`program threw an unrenderable value`}return Ne(`exception`,r,t,n)}function Fe(e){return class extends G{constructor(t,n){super(n),K(this,`name`,e.name),K(this,e.memberNameProperty,t)}}}function q(e,t,n){return e?new e(t,n):new G(n)}function Ie(e){let t=new Map;for(let n of e.namespaces)n.errorClass&&t.set(n.global,Fe(n.errorClass));return t}function Le(e,t){e.on(`message`,e=>{let n=t.get(e.id);if(n)if(t.delete(e.id),e.ok){let t=W(e.value);t===void 0?n.reject(new G(`binding resolution must be lossless JSON`)):n.resolve(t)}else n.reject(new G(e.message))})}function Re(e,t,n,r,i=Ie(e),a){return e.namespaces.map(({global:e,names:o})=>{let s=i.get(e),c=Object.create(null);for(let i of o)Object.defineProperty(c,i,{enumerable:!0,value:o=>{let c;try{c=B(o)}catch{c=void 0}if(c===void 0)return Promise.reject(q(s,i,`binding arguments must be lossless JSON`));let l={type:`call`,id:r.value,global:e,name:i,args:V(c)};return a!==void 0&&y(l,a)===void 0?Promise.reject(q(s,i,`binding arguments exceed maxFrameBytes`)):new Promise((e,a)=>{let o=r.value++;n.set(o,{resolve:e,reject:e=>{a(q(s,i,e.message))}});try{t.postMessage(l)}catch(e){n.delete(o),a(q(s,i,`binding arguments must be structured-cloneable: ${e instanceof G?e.message:String(e)}`))}})}});return c})}async function ze(e,t,n,r){let i=new Ee(t.maxOutputBytes,t=>{e.postMessage({type:`log`,text:t})},()=>{e.postMessage({type:`output-limit`})});ke(i,n.stdout),ke(i,n.stderr);let a=new Map;Le(e,a);let o={value:1},s=Ie(t),c=Re(t,e,a,o,s,r),l=[],u=[];for(let e of t.namespaces){if(!e.errorClass)continue;l.push(e.errorClass.name);let t=s.get(e.global);if(!t)throw new G(`missing binding error class for ${e.global}`);u.push(t)}let d=Oe(i),f;try{let e=(async()=>{}).constructor;f={type:`done`,...je(await new e(...t.namespaces.map(e=>e.global),...l,`console`,`'use strict';\\n${t.code}`)(...c,...u,d),i.remainingOutputBytes(),t.maxOutputBytes)}}catch(e){f={type:`done`,...Pe(e,i.remainingOutputBytes(),t.maxOutputBytes)}}e.postMessage(r!==void 0&&y(f,r)===void 0?{type:`output-limit`}:f)}function J(e){return e.readableEnded||e.destroyed?Promise.resolve():new Promise(t=>{let n=()=>{e.off(`end`,n),e.off(`close`,n),e.off(`error`,n),t()};e.once(`end`,n),e.once(`close`,n),e.once(`error`,n),(e.readableEnded||e.destroyed)&&n()})}const Be=new Set([`exception`,`timeout`,`abort`,`worker-exit`,`invalid-output`,`output-limit`]);let Y=0;function X(e){return typeof e==`object`&&e?e:void 0}function Z(t,n){try{let r=JSON.stringify(t);return typeof r==`string`&&e.byteLength(r)<=n?r:void 0}catch{return}}function Ve(e){process.stdout.write(e),process.stdout.write(`\n`)}function He(e){let t=Z(e,Y);return t===void 0?!1:(Ve(t),!0)}function Ue(e){let t=X(e);return t!==void 0&&typeof t.kind==`string`&&Be.has(t.kind)&&typeof t.message==`string`}function We(e){return Ue(e)&&(e.kind===`exception`||e.kind===`invalid-output`||e.kind===`output-limit`)}function Ge(e){let t=X(e);if(!(t===void 0||t.type!==`boot`||typeof t.code!=`string`||!Array.isArray(t.namespaces)||!Number.isSafeInteger(t.maxOutputBytes)||t.maxOutputBytes<4||!Number.isSafeInteger(t.maxFrameBytes)||t.maxFrameBytes{e.once(`exit`,()=>{t()})})}function $(){return{type:`done`,error:{kind:`worker-exit`,message:`code runtime bridge frame exceeded maxFrameBytes`}}}function Je(){let e=n({input:process.stdin,crlfDelay:1/0}),r,a=0,o=2,s=0,c=!1,l=t=>{if(c)return;let n=Z(t,Y)??Z($(),Y);c=!0,n!==void 0&&Ve(n);let i=r;r=void 0,(i===void 0?Promise.resolve():new Promise(e=>{setImmediate(e)}).then(async()=>{let e=J(i.stdout),t=J(i.stderr),n=qe(i);i.kill(`SIGKILL`),await Promise.all([n,e,t])})).catch(e=>{process.stderr.write(`dsh-code-runtime-subprocess controller cleanup error: ${String(e)}\\n`)}).then(()=>{e.close(),process.stdin.destroy()})},u=e=>{if(c)return;let t=+(s>0),n=v(e,a-o-t);if(n===void 0){l({type:`output-limit`});return}o+=n+t,s+=1,He({type:`log`,text:e})||l($())},d=e=>{a=e.maxOutputBytes,Y=e.maxFrameBytes,r=t(i(import.meta.url),[],{env:{DSH_CODE_RUNTIME_CONTROLLER:`1`},detached:!1,execArgv:[],stdio:[`ignore`,`pipe`,`pipe`,`ipc`]});let n=r;n.stdout.on(`data`,e=>{u(e.toString(`utf8`))}),n.stderr.on(`data`,e=>{u(e.toString(`utf8`))}),n.on(`message`,e=>{let t=X(e);if(t!==void 0){if(t.type===`log`&&typeof t.text==`string`){u(t.text);return}c||(t.type===`call`&&typeof t.id==`number`&&typeof t.global==`string`&&typeof t.name==`string`?He({type:`call`,id:t.id,global:t.global,name:t.name,args:Q(t.args)})||l($()):t.type===`output-limit`?l({type:`output-limit`}):t.type===`done`&&(t.error===void 0?l({type:`done`,...t.value===void 0?{}:{value:Q(t.value)}}):Ue(t.error)&&l({type:`done`,error:t.error})))}}),n.on(`error`,e=>{l({type:`done`,error:{kind:`worker-exit`,message:`remote controller error: ${e.message}`}})}),n.on(`exit`,e=>{c||l({type:`done`,error:{kind:`worker-exit`,message:`remote controller exited with code ${e} before completing`}})}),n.send(e,e=>{e!==null&&l({type:`done`,error:{kind:`worker-exit`,message:`remote controller boot failed: ${e.message}`}})})};e.on(`line`,e=>{let t;try{t=JSON.parse(e)}catch(e){process.stderr.write(`dsh-code-runtime-subprocess frame error: ${String(e)}\\n`),l({type:`done`,error:{kind:`worker-exit`,message:`remote runner received a malformed frame`}});return}if(r===void 0){let e=Ge(t);if(e===void 0){l({type:`done`,error:{kind:`worker-exit`,message:`remote runner received an invalid boot frame`}});return}d(e);return}let n=Ke(t);n!==void 0&&r.send(n,e=>{e!==null&&l({type:`done`,error:{kind:`worker-exit`,message:`remote controller reply failed: ${e.message}`}})})}),e.on(`close`,()=>{r!==void 0&&!c&&l({type:`done`,error:{kind:`abort`,message:`remote runner input closed`}})})}function Ye(){let e,t=!1,n,i=0,o=e=>process.send===void 0||i>0&&Z(e,i)===void 0?!1:(process.send(e),!0),s=r=>{if(t)return;t=!0,clearInterval(n);let a=i>0&&Z(r,i)===void 0?$():r,s=e;e=void 0,(s===void 0?Promise.resolve():new Promise(e=>{setImmediate(e)}).then(async()=>{let e=J(s.stdout),t=J(s.stderr);await Promise.all([s.terminate(),e,t])})).catch(e=>{o({type:`log`,text:`dsh-code-runtime-subprocess worker cleanup error: ${String(e)}\\n`})}).then(()=>{if(process.send===void 0){process.exitCode=1;return}process.send(a,()=>{process.connected&&process.disconnect()})})};process.on(`message`,c=>{if(e===void 0){let l=Ge(c);if(l===void 0){s({type:`done`,error:{kind:`worker-exit`,message:`remote controller received an invalid boot frame`}});return}i=l.maxFrameBytes,e=new a(new URL(import.meta.url),{workerData:l,env:{},execArgv:[],stdout:!0,stderr:!0,resourceLimits:{maxOldGenerationSizeMb:l.maxOldGenerationSizeMb}});let u=e;u.stdout.on(`data`,e=>{o({type:`log`,text:e.toString(`utf8`)})||s($())}),u.stderr.on(`data`,e=>{o({type:`log`,text:e.toString(`utf8`)})||s($())}),u.on(`message`,e=>{let t=X(e);t!==void 0&&(t.type===`call`&&typeof t.id==`number`&&typeof t.global==`string`&&typeof t.name==`string`?o({type:`call`,id:t.id,global:t.global,name:t.name,args:Q(t.args)})||s($()):t.type===`log`&&typeof t.text==`string`?o({type:`log`,text:t.text})||s($()):t.type===`output-limit`?s({type:`output-limit`}):t.type===`done`&&(t.error===void 0?s({type:`done`,...t.value===void 0?{}:{value:Q(t.value)}}):We(t.error)&&s({type:`done`,error:t.error})))}),u.on(`error`,e=>{s({type:`done`,error:{kind:`worker-exit`,message:`worker error: ${e.stack||e.message||r(e)}`}})}),u.on(`exit`,e=>{t||s({type:`done`,error:{kind:`worker-exit`,message:`worker exited with code ${e} before completing`}})}),n=setInterval(()=>{e!==void 0&&e.performance.eventLoopUtilization().active>l.computeMs&&s({type:`done`,error:{kind:`timeout`,message:`compute budget exhausted (${l.computeMs}ms busy)`}})},25);return}let l=Ke(c);l!==void 0&&e.postMessage(l)}),process.on(`disconnect`,()=>{e!==void 0&&!t&&e.terminate()})}if(o)process.env.DSH_CODE_RUNTIME_CONTROLLER===`1`?Ye():Je();else{if(s===null)throw Error(`remote worker requires parentPort`);let e=c;ze(s,e,{stdout:process.stdout,stderr:process.stderr},e.maxFrameBytes)}export{};" +export const CODE_RUNNER_SOURCE = "import{Buffer as e}from\"node:buffer\";import{fork as t}from\"node:child_process\";import{createInterface as n}from\"node:readline\";import{inspect as r}from\"node:util\";import{fileURLToPath as i}from\"node:url\";import{Worker as a,isMainThread as o,parentPort as s,workerData as c}from\"node:worker_threads\";const l=Reflect.apply,u=Array.isArray,d=Buffer,f=Reflect.get(Buffer,`byteLength`),ee=Object.create,p=Object.defineProperty,te=Object.keys,ne=String,re=Reflect.get(String.prototype,`charCodeAt`),ie=Reflect.get(String.prototype,`codePointAt`),m=Reflect.get(String.prototype,`slice`);function h(e){let t=ee(null);return t.value=e,t}function ae(e,t,n){let r=h(n);r.enumerable=!0,r.configurable=!0,r.writable=!0,p(e,t,r)}function g(e){return l(f,d,[e,`utf8`])}function _(e,t){ae(e,e.length,t)}function oe(e){if(e.length===0)return;let t=e.length-1,n=e[t];return p(e,`length`,h(t)),n}function se(e,t){return l(m,e,[t,t+(l(ie,e,[t])>65535?2:1)])}function ce(e){if(e.length===2)return 4;if(e===`\"`||e===`\\\\`)return 2;let t=l(re,e,[0]);return t>=55296&&t<=57343?6:t<32?t===8||t===9||t===10||t===12||t===13?2:6:g(e)}function v(e,t){if(t<2)return;let n=2;for(let r=0;rt)return;r+=i.length}return n}function y(e,t){let n=0,r=e=>(n+=e,n<=t),i=[{kind:`value`,value:e}];for(let e=oe(i);e!==void 0;e=oe(i)){if(e.kind===`value`){let a=e.value;if(a===null){if(!r(4))return}else if(typeof a==`string`){let e=v(a,t-n);if(e===void 0)return;n+=e}else if(typeof a==`number`){if(!r(g(ne(a))))return}else if(typeof a==`boolean`){if(!r(a?4:5))return}else if(u(a)){if(!r(2))return;a.length>0&&_(i,{kind:`array`,value:a,index:0})}else{if(!r(2))return;let e=te(a);e.length>0&&_(i,{kind:`object`,value:a,keys:e,index:0})}continue}if(e.index>0&&!r(1))return;if(e.kind===`array`){let t=e.value[e.index];if(t===void 0)return;e.index+1t)break;n+=o,r+=a.length,i+=a.length}return r===e.length?e:l(m,e,[0,r])}const ue=Reflect.get(Function.prototype,`toString`),b=Reflect.get(Reflect,`apply`),x=Error,S=Set,C=Array.isArray,de=Array.prototype,w=Number.isFinite,fe=Number.isSafeInteger,pe=Object.create,T=Object.defineProperty,me=Object.getOwnPropertyDescriptor,E=Object.getPrototypeOf,D=Object.hasOwn,O=Object.is,he=Object.keys,k=Object.prototype,ge=Reflect.get(k,`propertyIsEnumerable`),A=Reflect.ownKeys,_e=Reflect.get(Set.prototype,`add`),ve=Reflect.get(Set.prototype,`delete`),ye=Reflect.get(Set.prototype,`has`);function j(e){let t=pe(null);return t.value=e,t}function M(e,t,n){let r=j(n);r.enumerable=!0,r.configurable=!0,r.writable=!0,T(e,t,r)}function N(e,t){M(e,e.length,t)}function P(e){if(e.length===0)return;let t=e.length-1,n=e[t];return T(e,`length`,j(t)),n}function F(e,t){return b(ye,e,[t])}function I(e,t){b(_e,e,[t])}function be(e,t){b(ve,e,[t])}function L(e,t){let n=me(e,`constructor`)?.value;if(typeof n!=`function`)return!1;try{return n.name===t&&n.prototype===e&&b(ue,n,[])===`function ${t}() { [native code] }`}catch{return!1}}function R(e){return E(e)===null&&L(e,`Object`)}function z(e){let t=E(e);if(t===de)return!0;if(!C(t)||!L(t,`Array`))return!1;let n=E(t);return typeof n==`object`&&!!n&&R(n)}function B(e){let t=E(e);return t===null||t===k||typeof t==`object`&&R(t)}function V(e){let t=A(e);for(let n=0;n{e.kind===`root`?n=t:e.kind===`array`?M(e.target,e.index,t):M(e.target,e.key,t)},i=[{kind:`visit`,value:e,destination:{kind:`root`}}];for(let e=P(i);e!==void 0;e=P(i)){if(e.kind===`leave`){be(t,e.source);continue}if(e.kind===`array-item`){if(!D(e.source,e.index))return;N(i,{kind:`visit`,value:e.source[e.index],destination:{kind:`array`,target:e.target,index:e.index}});continue}if(e.kind===`object-property`){N(i,{kind:`visit`,value:e.source[e.key],destination:{kind:`object`,target:e.target,key:e.key}});continue}let n=e.value;if(n===null){r(e.destination,null);continue}if(typeof n==`boolean`||typeof n==`string`){r(e.destination,n);continue}if(typeof n==`number`){if(!w(n)||O(n,-0))return;r(e.destination,n);continue}if(typeof n!=`object`||F(t,n))return;if(C(n)){if(!z(n))return;let a=n.length;if(A(n).length!==a+1)return;let o=[];r(e.destination,o),I(t,n),N(i,{kind:`leave`,source:n});for(let e=a-1;e>=0;e--)N(i,{kind:`array-item`,source:n,index:e,target:o});continue}if(!B(n))return;let a=V(n);if(a===void 0)return;let o={};r(e.destination,o),I(t,n),N(i,{kind:`leave`,source:n});for(let e=a.length-1;e>=0;e--){let t=a[e];if(t===void 0)return;N(i,{kind:`object-property`,source:n,key:t,target:o})}}return n}function U(e){let t=[],n=[e];for(let e=P(n);e!==void 0;e=P(n)){if(e===null||typeof e==`boolean`||typeof e==`number`||typeof e==`string`){N(t,e);continue}if(C(e)){N(t,{kind:`array`,length:e.length});for(let t=e.length-1;t>=0;t--){let r=e[t];if(r===void 0)throw new x(`cannot encode a sparse JSON array`);N(n,r)}continue}let r=he(e);N(t,{kind:`object`,keys:r});for(let t=r.length-1;t>=0;t--){let i=r[t];if(i===void 0)throw new x(`cannot encode a missing JSON object key`);let a=e[i];if(a===void 0)throw new x(`cannot encode an undefined JSON object property`);N(n,a)}}return t}function xe(e){if(!z(e)||A(e).length!==e.length+1)return!1;for(let t=0;t=0?{kind:`array`,length:e}:void 0}if(n.kind===`object`){if(t.length!==2||!W(t,`kind`)||!W(t,`keys`))return;let e=n.keys;if(!C(e)||!xe(e))return;let r=new S,i=[],a=e;for(let e=0;e{let t=n[n.length-1];if(!t)return i?!1:(r=e,i=!0,!0);if(t.index>=(t.kind===`array`?t.length:t.keys.length))return!1;if(t.kind===`array`)N(t.target,e);else{let n=t.keys[t.index];if(n===void 0)return!1;M(t.target,n,e)}return t.index+=1,!0};for(let e=0;ea)return;let e=[];i=e,n.length>0&&(o={kind:`array`,target:e,length:n.length,index:0})}else{if(n.keys.length>a)return;let e={};i=e,n.keys.length>0&&(o={kind:`object`,target:e,keys:n.keys,index:0})}}if(!a(i))return;for(o&&N(n,o);n.length>0;){let e=n[n.length-1];if(e===void 0||e.index<(e.kind===`array`?e.length:e.keys.length))break;P(n)}}return n.length===0?r:void 0}catch{return}}const K=Error,Ce=Object.create,we=Object.defineProperty;function Te(e,t,n){let r=Ce(null);r.enumerable=!0,r.value=n,we(e,t,r)}var Ee=class{bytes=2;entries=0;truncated=!1;sink;onLimit;maxBytes;constructor(e,t,n=()=>{}){this.maxBytes=e,this.sink=t,this.onLimit=n}push(e){if(this.truncated)return;let t=+(this.entries>0),n=this.maxBytes-this.bytes-t,r=v(e,n);if(r===void 0){this.truncated=!0;let r=le(e,n);if(r.length>0){let e=v(r,n);if(e===void 0)throw new K(`worker output ledger produced an oversized log prefix`);this.bytes+=e+t,this.entries+=1,this.sink(r)}this.onLimit();return}this.bytes+=r+t,this.entries+=1,this.sink(e)}remainingOutputBytes(){return this.maxBytes-this.bytes}};const De=[`log`,`info`,`warn`,`error`,`debug`];function Oe(e){let t=e=>e.map(e=>typeof e==`string`?e:r(e,Ae)).join(` `),n=Object.create(null);for(let r of De)n[r]=(...n)=>{e.push(t(n))};return n}function ke(e,t){let n=t.write;return t.write=(t,...n)=>{e.push(typeof t==`string`?t:String(t));let r=[n[0],n[1]].find(e=>typeof e==`function`);return r&&queueMicrotask(()=>{r(null)}),!0},()=>{t.write=n}}const Ae={depth:4,maxArrayLength:100,maxStringLength:1e4};function je(e,t,n=t){if(e===void 0)return{};let r;try{r=H(e)}catch{r=void 0}return r===void 0?Ne(`invalid-output`,`program completion must be lossless JSON`,t,n):y(r,t)===void 0?Me(n):{value:U(r)}}function Me(e){return{error:{kind:`output-limit`,message:`outer output exceeded ${e} bytes`}}}function Ne(e,t,n,r){return v(t,n)===void 0?Me(r):{error:{kind:e,message:t}}}function Pe(e,t,n=t){let r;try{let t=e instanceof K?e.stack??e.message:e;r=typeof t==`string`?t:String(t)}catch{r=`program threw an unrenderable value`}return Ne(`exception`,r,t,n)}function Fe(e){return class extends K{constructor(t,n){super(n),Te(this,`name`,e.name),Te(this,e.memberNameProperty,t)}}}function q(e,t,n){return e?new e(t,n):new K(n)}function Ie(e){let t=new Map;for(let n of e.namespaces)n.errorClass&&t.set(n.global,Fe(n.errorClass));return t}function Le(e,t){e.on(`message`,e=>{let n=t.get(e.id);if(n)if(t.delete(e.id),e.ok){let t=G(e.value);t===void 0?n.reject(new K(`binding resolution must be lossless JSON`)):n.resolve(t)}else n.reject(new K(e.message))})}function Re(e,t,n,r,i=Ie(e),a){return e.namespaces.map(({global:e,names:o})=>{let s=i.get(e),c=Object.create(null);for(let i of o)Object.defineProperty(c,i,{enumerable:!0,value:o=>{let c;try{c=H(o)}catch{c=void 0}if(c===void 0)return Promise.reject(q(s,i,`binding arguments must be lossless JSON`));let l={type:`call`,id:r.value,global:e,name:i,args:U(c)};return a!==void 0&&y(l,a)===void 0?Promise.reject(q(s,i,`binding arguments exceed maxFrameBytes`)):new Promise((e,a)=>{let o=r.value++;n.set(o,{resolve:e,reject:e=>{a(q(s,i,e.message))}});try{t.postMessage(l)}catch(e){n.delete(o),a(q(s,i,`binding arguments must be structured-cloneable: ${e instanceof K?e.message:String(e)}`))}})}});return c})}async function ze(e,t,n,r){let i=new Ee(t.maxOutputBytes,t=>{e.postMessage({type:`log`,text:t})},()=>{e.postMessage({type:`output-limit`})});ke(i,n.stdout),ke(i,n.stderr);let a=new Map;Le(e,a);let o={value:1},s=Ie(t),c=Re(t,e,a,o,s,r),l=[],u=[];for(let e of t.namespaces){if(!e.errorClass)continue;l.push(e.errorClass.name);let t=s.get(e.global);if(!t)throw new K(`missing binding error class for ${e.global}`);u.push(t)}let d=Oe(i),f;try{let e=(async()=>{}).constructor;f={type:`done`,...je(await new e(...t.namespaces.map(e=>e.global),...l,`console`,`'use strict';\\n${t.code}`)(...c,...u,d),i.remainingOutputBytes(),t.maxOutputBytes)}}catch(e){f={type:`done`,...Pe(e,i.remainingOutputBytes(),t.maxOutputBytes)}}e.postMessage(r!==void 0&&y(f,r)===void 0?{type:`output-limit`}:f)}function J(e){return e.readableEnded||e.destroyed?Promise.resolve():new Promise(t=>{let n=()=>{e.off(`end`,n),e.off(`close`,n),e.off(`error`,n),t()};e.once(`end`,n),e.once(`close`,n),e.once(`error`,n),(e.readableEnded||e.destroyed)&&n()})}const Be=new Set([`exception`,`timeout`,`abort`,`worker-exit`,`invalid-output`,`output-limit`]);let Y=0;function X(e){return typeof e==`object`&&e?e:void 0}function Z(t,n){try{let r=JSON.stringify(t);return typeof r==`string`&&e.byteLength(r)<=n?r:void 0}catch{return}}function Ve(e){process.stdout.write(e),process.stdout.write(`\n`)}function He(e){let t=Z(e,Y);return t===void 0?!1:(Ve(t),!0)}function Ue(e){let t=X(e);return t!==void 0&&typeof t.kind==`string`&&Be.has(t.kind)&&typeof t.message==`string`}function We(e){return Ue(e)&&(e.kind===`exception`||e.kind===`invalid-output`||e.kind===`output-limit`)}function Ge(e,t){return e.error===void 0?{type:`done`,...e.value===void 0?{}:{value:Q(e.value)}}:t(e.error)?{type:`done`,error:e.error}:void 0}function Ke(e){let t=X(e);if(!(t===void 0||t.type!==`boot`||typeof t.code!=`string`||!Array.isArray(t.namespaces)||!Number.isSafeInteger(t.maxOutputBytes)||t.maxOutputBytes<4||!Number.isSafeInteger(t.maxFrameBytes)||t.maxFrameBytes{e.once(`exit`,()=>{t()})})}function $(){return{type:`done`,error:{kind:`worker-exit`,message:`code runtime bridge frame exceeded maxFrameBytes`}}}function Ye(){let e=n({input:process.stdin,crlfDelay:1/0}),r,a=0,o=2,s=0,c=!1,l=t=>{if(c)return;let n=Z(t,Y)??Z($(),Y);c=!0,n!==void 0&&Ve(n);let i=r;r=void 0,(i===void 0?Promise.resolve():new Promise(e=>{setImmediate(e)}).then(async()=>{let e=J(i.stdout),t=J(i.stderr),n=Je(i);i.kill(`SIGKILL`),await Promise.all([n,e,t])})).catch(e=>{process.stderr.write(`dsh-code-runtime-subprocess controller cleanup error: ${String(e)}\\n`)}).then(()=>{e.close(),process.stdin.destroy()})},u=e=>{if(c)return;let t=+(s>0),n=v(e,a-o-t);if(n===void 0){l({type:`output-limit`});return}o+=n+t,s+=1,He({type:`log`,text:e})||l($())},d=e=>{a=e.maxOutputBytes,Y=e.maxFrameBytes,r=t(i(import.meta.url),[],{env:{DSH_CODE_RUNTIME_CONTROLLER:`1`},detached:!1,execArgv:[],stdio:[`ignore`,`pipe`,`pipe`,`ipc`]});let n=r;n.stdout.on(`data`,e=>{u(e.toString(`utf8`))}),n.stderr.on(`data`,e=>{u(e.toString(`utf8`))}),n.on(`message`,e=>{let t=X(e);if(t!==void 0){if(t.type===`log`&&typeof t.text==`string`){u(t.text);return}if(!c){if(t.type===`call`&&typeof t.id==`number`&&typeof t.global==`string`&&typeof t.name==`string`)He({type:`call`,id:t.id,global:t.global,name:t.name,args:Q(t.args)})||l($());else if(t.type===`output-limit`)l({type:`output-limit`});else if(t.type===`done`){let e=Ge(t,Ue);e!==void 0&&l(e)}}}}),n.on(`error`,e=>{l({type:`done`,error:{kind:`worker-exit`,message:`remote controller error: ${e.message}`}})}),n.on(`exit`,e=>{c||l({type:`done`,error:{kind:`worker-exit`,message:`remote controller exited with code ${e} before completing`}})}),n.send(e,e=>{e!==null&&l({type:`done`,error:{kind:`worker-exit`,message:`remote controller boot failed: ${e.message}`}})})};e.on(`line`,e=>{let t;try{t=JSON.parse(e)}catch(e){process.stderr.write(`dsh-code-runtime-subprocess frame error: ${String(e)}\\n`),l({type:`done`,error:{kind:`worker-exit`,message:`remote runner received a malformed frame`}});return}if(r===void 0){let e=Ke(t);if(e===void 0){l({type:`done`,error:{kind:`worker-exit`,message:`remote runner received an invalid boot frame`}});return}d(e);return}let n=qe(t);n!==void 0&&r.send(n,e=>{e!==null&&l({type:`done`,error:{kind:`worker-exit`,message:`remote controller reply failed: ${e.message}`}})})}),e.on(`close`,()=>{r!==void 0&&!c&&l({type:`done`,error:{kind:`abort`,message:`remote runner input closed`}})})}function Xe(){let e,t=!1,n,i=0,o=e=>process.send===void 0||i>0&&Z(e,i)===void 0?!1:(process.send(e),!0),s=r=>{if(t)return;t=!0,clearInterval(n);let a=i>0&&Z(r,i)===void 0?$():r,s=e;e=void 0,(s===void 0?Promise.resolve():new Promise(e=>{setImmediate(e)}).then(async()=>{let e=J(s.stdout),t=J(s.stderr);await Promise.all([s.terminate(),e,t])})).catch(e=>{o({type:`log`,text:`dsh-code-runtime-subprocess worker cleanup error: ${String(e)}\\n`})}).then(()=>{if(process.send===void 0){process.exitCode=1;return}process.send(a,()=>{process.connected&&process.disconnect()})})};process.on(`message`,c=>{if(e===void 0){let l=Ke(c);if(l===void 0){s({type:`done`,error:{kind:`worker-exit`,message:`remote controller received an invalid boot frame`}});return}i=l.maxFrameBytes,e=new a(new URL(import.meta.url),{workerData:l,env:{},execArgv:[],stdout:!0,stderr:!0,resourceLimits:{maxOldGenerationSizeMb:l.maxOldGenerationSizeMb}});let u=e;u.stdout.on(`data`,e=>{o({type:`log`,text:e.toString(`utf8`)})||s($())}),u.stderr.on(`data`,e=>{o({type:`log`,text:e.toString(`utf8`)})||s($())}),u.on(`message`,e=>{let t=X(e);if(t!==void 0){if(t.type===`call`&&typeof t.id==`number`&&typeof t.global==`string`&&typeof t.name==`string`)o({type:`call`,id:t.id,global:t.global,name:t.name,args:Q(t.args)})||s($());else if(t.type===`log`&&typeof t.text==`string`)o({type:`log`,text:t.text})||s($());else if(t.type===`output-limit`)s({type:`output-limit`});else if(t.type===`done`){let e=Ge(t,We);e!==void 0&&s(e)}}}),u.on(`error`,e=>{s({type:`done`,error:{kind:`worker-exit`,message:`worker error: ${e.stack||e.message||r(e)}`}})}),u.on(`exit`,e=>{t||s({type:`done`,error:{kind:`worker-exit`,message:`worker exited with code ${e} before completing`}})}),n=setInterval(()=>{e!==void 0&&e.performance.eventLoopUtilization().active>l.computeMs&&s({type:`done`,error:{kind:`timeout`,message:`compute budget exhausted (${l.computeMs}ms busy)`}})},25);return}let l=qe(c);l!==void 0&&e.postMessage(l)}),process.on(`disconnect`,()=>{e!==void 0&&!t&&e.terminate()})}if(o)process.env.DSH_CODE_RUNTIME_CONTROLLER===`1`?Xe():Ye();else{if(s===null)throw Error(`remote worker requires parentPort`);let e=c;ze(s,e,{stdout:process.stdout,stderr:process.stderr},e.maxFrameBytes)}export{};" diff --git a/packages/util/atomic-write/src/runner.ts b/packages/util/atomic-write/src/runner.ts index 7a0b3eb678..92fad6db2e 100644 --- a/packages/util/atomic-write/src/runner.ts +++ b/packages/util/atomic-write/src/runner.ts @@ -94,6 +94,16 @@ function validWorkerFailure(value: unknown): value is RuntimeFailure { && (value.kind === 'exception' || value.kind === 'invalid-output' || value.kind === 'output-limit') } +function doneMessage( + message: Record, + acceptsFailure: (value: unknown) => value is RuntimeFailure, +): RuntimeMessage | undefined { + if (message.error !== undefined) { + return acceptsFailure(message.error) ? { type: 'done', error: message.error } : undefined + } + return { type: 'done', ...message.value === undefined ? {} : { value: transportWireOrNull(message.value) } } +} + function runtimeBoot(value: unknown): RuntimeBootData | undefined { const record = recordOf(value) if (record === undefined @@ -214,11 +224,8 @@ function runLauncher(): void { } else if (message.type === 'output-limit') { finish({ type: 'output-limit' }) } else if (message.type === 'done') { - if (message.error !== undefined) { - if (validFailure(message.error)) finish({ type: 'done', error: message.error }) - } else { - finish({ type: 'done', ...message.value === undefined ? {} : { value: transportWireOrNull(message.value) } }) - } + const done = doneMessage(message, validFailure) + if (done !== undefined) finish(done) } }) current.on('error', (error: Error) => { @@ -348,11 +355,8 @@ function runController(): void { } else if (message.type === 'output-limit') { finish({ type: 'output-limit' }) } else if (message.type === 'done') { - if (message.error !== undefined) { - if (validWorkerFailure(message.error)) finish({ type: 'done', error: message.error }) - } else { - finish({ type: 'done', ...message.value === undefined ? {} : { value: transportWireOrNull(message.value) } }) - } + const done = doneMessage(message, validWorkerFailure) + if (done !== undefined) finish(done) } }) current.on('error', (error: Error) => { From 22fc228b9757626fbc0c3c9ab920ec9472b258ea Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Thu, 30 Jul 2026 04:23:20 +0800 Subject: [PATCH 048/108] refactor(runtime): collapse unused setup layers --- ...rtable-execution-world-consumers.i18n.yaml | 4 ++-- ...7-28-portable-execution-world-consumers.md | 4 ++-- ...8-portable-execution-world-consumers.zh.md | 4 ++-- .../feature/2026-06-15-code-mode.i18n.yaml | 4 ++-- .../feature/2026-06-15-code-mode.md | 6 ++--- .../feature/2026-06-15-code-mode.zh.md | 6 ++--- docs/config-catalog.md | 4 ++-- .../core-data-structures/subprocess.i18n.yaml | 4 ++-- docs/core-data-structures/subprocess.md | 2 +- docs/core-data-structures/subprocess.zh.md | 2 +- packages/README.i18n.yaml | 4 ++-- packages/README.md | 2 +- packages/README.zh.md | 2 +- packages/bash/pwsh-local/tsconfig.json | 10 ++++---- packages/code-runtime/code-runtime/README.md | 4 ++-- packages/pty/pty-local/tests/index.spec.ts | 1 - .../subprocess-local/README.i18n.yaml | 4 ++-- .../subprocess/subprocess-local/README.md | 4 ++-- .../subprocess/subprocess-local/README.zh.md | 24 +++++++++---------- .../subprocess/subprocess-local/src/index.ts | 14 ++++------- .../subprocess-local/tests/local.spec.ts | 19 ++------------- .../subprocess/subprocess/README.i18n.yaml | 4 ++-- packages/subprocess/subprocess/README.md | 4 ++-- packages/subprocess/subprocess/README.zh.md | 4 ++-- packages/subprocess/subprocess/src/index.ts | 7 ++---- .../subprocess/tests/service.spec.ts | 1 - packages/ui/app-boot/tsdown.config.ts | 8 +------ packages/util/atomic-write/README.i18n.yaml | 6 ++--- .../src/runner-source.generated.ts | 2 +- packages/util/atomic-write/src/runner.ts | 17 +++++++++---- 30 files changed, 78 insertions(+), 103 deletions(-) diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml index 683a095229..3fce989f78 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md -2026-07-28-portable-execution-world-consumers.md: 03b274688e15661845c326afdd105b342dd6a513 -2026-07-28-portable-execution-world-consumers.zh.md: 1e3d8ae8e46d26f4bea47293104c52dbf33098d4 +2026-07-28-portable-execution-world-consumers.md: 74fa2687782a2cf59be8dddf98fa4a08a953d565 +2026-07-28-portable-execution-world-consumers.zh.md: 35d4f3640a44564894fecfaa22c7255d7452dec9 diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md index 03b274688e..74fa268778 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md @@ -23,7 +23,7 @@ Generic consumers use that execution world: - `dsh-bash-local` continues to map Bash semantics onto ordinary `ctx.subprocess.spawn()`. - `dsh-lsp-local` reads and contains source through `ctx.fs`, resolves and launches language servers through `ctx.subprocess`, and carries provider-owned file URIs through initialization and result rendering. One provider-lifetime signal aborts filesystem and protocol work during disposal, including workspace lookup before queue ownership; its JSON-RPC, pooling, synchronization, and normalization stay unchanged. - `dsh-pty-local` maps persistent-shell semantics onto `ctx.subprocess.spawnTerminal()`. The local `node-pty` and process-inspection implementation moves into `dsh-subprocess-local`; another subprocess provider supplies the same primitive. `danger-full-access` needs no `ctx.sandbox`; a confined mode requires a same-world sandbox provider and fails before spawn when none is mounted. Prompt and silence evidence collected during asynchronous pre-write inspection is discarded when the provider write begins. Cancellation retains the send reservation while an in-flight write settles and then signals the foreground group, so late bytes or the signal cannot target a successor; an in-flight readiness poll cannot release that reservation, and a rejected write sends no signal. The absolute deadline remains armed throughout cancellation. A signal failure becomes terminal transport failure. Completion of a stale inspection resumes polling for the current send. Startup cancellation begins terminal rollback without waiting for a stalled readiness or signalling call. Close rejects new public signals and delegates provider-observable session quiescence to the handle's awaited termination operation. -- `dsh-code-runtime-subprocess` materializes a dependency-free runner through `ctx.fs` and launches it through `ctx.subprocess`, preserving the Code Runtime binding and output contract across local or remote worlds. The fixed runner is adapter-owned infrastructure below `ctx.subprocess.runtimeRoot`, so its write carries an explicit `danger-full-access` policy instead of inheriting the model-facing filesystem mode. It shares host-side worker mechanics through the non-plugin `dsh-code-runtime-worker/runtime-host` subpath instead of copying them. Preparation carries one lifecycle signal through filesystem resolution, materialization, and executable lookup so disposal can abort a stalled provider. The heap-bounded worker rejects oversized binding frames before transfer, each outer hop enforces the same bound before forwarding, and raw subprocess pipes carry newline-delimited UTF-8 JSON without a redundant base64 representation. The launcher publishes an accepted terminal frame before reaping its controller so a descendant that inherits controller pipes cannot suppress completion; the host still awaits process-group quiescence. +- `dsh-code-runtime-subprocess` passes a bundled dependency-free eval runner directly to `ctx.subprocess`, preserving the Code Runtime binding and output contract across local or remote process worlds without a filesystem dependency or provider-specific package. It shares host-side worker mechanics through the non-plugin `dsh-code-runtime-worker/runtime-host` subpath instead of copying them. The heap-bounded worker rejects oversized binding frames before transfer, each outer hop enforces the same bound before forwarding, and raw subprocess pipes carry newline-delimited UTF-8 JSON without a redundant base64 representation. The launcher publishes an accepted terminal frame before reaping its controller so a descendant that inherits controller pipes cannot suppress completion; the host still awaits process-group quiescence. `dsh-code-runtime-worker` remains a separate implementation. It is the smaller in-process backend and works in single-file distributions that cannot assume an installed Node executable. Remote filesystem/process compositions select `dsh-code-runtime-subprocess`; they do not need a provider-specific Code Runtime package. @@ -39,7 +39,7 @@ Generic consumers use that execution world: **Add a stable bounded-read primitive to the filesystem seam.** Rejected because only LSP needs a complete-document byte ceiling, which it can enforce while consuming the existing text stream. A second primitive forces every provider to implement stable-handle and no-follow mechanics, including a remote helper protocol, without an observed concurrent-replacement defect. -**Delete the worker-thread Code Runtime.** Rejected because portability does not erase its current deployment need. The subprocess backend requires a Node executable and filesystem materialization; the worker backend has neither requirement and remains the supported single-process path. +**Delete the worker-thread Code Runtime.** Rejected because portability does not erase its current deployment need. The subprocess backend requires a Node executable; the worker backend does not and remains the supported single-process path. **Run the whole harness inside the remote environment.** Rejected as a different deployment model. Making execution capabilities portable does not move model calls, session state, plugin state, or the agent loop. diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md index 1e3d8ae8e4..35d4f3640a 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md @@ -23,7 +23,7 @@ Status: implemented - `dsh-bash-local` 继续把 Bash 语义映射到普通的 `ctx.subprocess.spawn()`。 - `dsh-lsp-local` 通过 `ctx.fs` 读取源文件并验证包含关系,通过 `ctx.subprocess` 解析和启动语言服务器,并让由提供方负责的文件 URI 贯穿初始化与结果渲染。一个提供方生命周期信号会在资源释放期间中止文件系统与协议操作,包括取得队列所有权之前的工作区查找;其 JSON-RPC、池化、同步和规范化保持不变。 - `dsh-pty-local` 把持久 shell 语义映射到 `ctx.subprocess.spawnTerminal()`。本地 `node-pty` 与进程检查实现移入 `dsh-subprocess-local`;其他进程管理提供方则提供相同原语。`danger-full-access` 不需要 `ctx.sandbox`;受限模式要求同一执行世界中存在沙箱提供方,未挂载时会在 spawn 前失败。提供方开始写入时,系统会丢弃异步写入前检查期间收集的提示符与静默证据。取消会在在途写入结算期间保留发送预留,随后向前台进程组发送信号,因此延迟字节和该信号都无法落到后续发送;在途就绪检查无法释放该预留,写入被拒绝时也不会发送信号。绝对截止时间会在整个取消期间保持启用。信号发送失败会成为终结性传输失败。陈旧检查完成后,会针对当前发送恢复轮询。启动取消会立即开始终端回滚,而不等待停滞的就绪检查或信号发送调用。关闭操作会拒绝新的公开信号,并把提供方可观察会话成员的完全停稳委托给句柄上须等待的终止操作。 -- `dsh-code-runtime-subprocess` 通过 `ctx.fs` 物化无依赖 runner,并通过 `ctx.subprocess` 启动它,从而在本地或远程执行世界中保留代码运行时的绑定与输出契约。固定 runner 是位于 `ctx.subprocess.runtimeRoot` 下的适配器自有基础设施,因此其写入携带显式 `danger-full-access` 策略,而不继承面向模型的文件系统模式。它通过非插件子路径 `dsh-code-runtime-worker/runtime-host` 共享宿主侧 worker 机制,而不是复制这些机制。准备阶段让同一个生命周期信号贯穿文件系统解析、物化和可执行文件查找,使资源释放能够中止停滞的提供方操作。受堆上限约束的 worker 会在传输前拒绝过大的绑定帧;每个外层转发环节都会在转发前执行相同的上限检查;原始子进程管道承载以换行符分隔的 UTF-8 JSON,无需冗余的 base64 表示;launcher 会在回收 controller 前发布已接纳的终态帧,使继承 controller 管道的后代进程无法阻止完成;宿主仍会等待进程组完全停稳。 +- `dsh-code-runtime-subprocess` 将一个内置的无依赖 eval runner 直接传给 `ctx.subprocess`,从而在本地或远程进程执行环境中保留代码运行时的绑定与输出契约,且不依赖文件系统或提供方专用包。它通过非插件子路径 `dsh-code-runtime-worker/runtime-host` 共享宿主侧 worker 机制,而不是复制这些机制。受堆上限约束的 worker 会在传输前拒绝过大的绑定帧;每个外层转发环节都会在转发前执行相同的上限检查;原始子进程管道承载以换行符分隔的 UTF-8 JSON,无需冗余的 base64 表示;launcher 会在回收 controller 前发布已接纳的终态帧,使继承 controller 管道的后代进程无法阻止完成;宿主仍会等待进程组完全停稳。 `dsh-code-runtime-worker` 仍是独立实现。它是较小的进程内后端,可用于无法假定已安装 Node 可执行文件的单文件分发。远程文件系统/进程组合选择 `dsh-code-runtime-subprocess`;它们不需要提供方专用的代码运行时包。 @@ -39,7 +39,7 @@ Status: implemented **在文件系统 seam 中新增稳定的有界读取原语。** 不予采纳,因为只有 LSP 需要完整文档字节上限,而它可以在消费现有文本流时执行该上限。第二项原语会迫使每个提供方实现稳定句柄和不跟随符号链接的机制,远程提供方甚至需要辅助协议,却没有已观察到的并发替换缺陷。 -**删除 worker 线程代码运行时。** 不予采纳,因为可移植性不会消除其当前部署需求。进程管理后端需要 Node 可执行文件和文件系统物化,而 worker 后端两者都不需要,并且仍是受支持的单进程路径。 +**删除 worker 线程代码运行时。** 不予采纳,因为可移植性不会消除其当前部署需求。进程管理后端需要 Node 可执行文件,而 worker 后端不需要,并且仍是受支持的单进程路径。 **在远程环境中运行整个 harness。** 不予采纳,因为这是另一种部署模型。让执行能力可移植,并不意味着移动模型调用、会话状态、插件状态或 agent loop(智能体循环)。 diff --git a/.agents/notes/implemented/feature/2026-06-15-code-mode.i18n.yaml b/.agents/notes/implemented/feature/2026-06-15-code-mode.i18n.yaml index 112e4eed83..d2655b6d02 100644 --- a/.agents/notes/implemented/feature/2026-06-15-code-mode.i18n.yaml +++ b/.agents/notes/implemented/feature/2026-06-15-code-mode.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-06-15-code-mode.md -2026-06-15-code-mode.md: 1be25f5b145e50d8fe391d22910b71d85181582e -2026-06-15-code-mode.zh.md: a4e8fbbd339899937c4f39091b4befbbb78ec347 +2026-06-15-code-mode.md: a075ea66c150afb1b48f98c600b72eec57869962 +2026-06-15-code-mode.zh.md: c269c0a8709087e7b1e27051cbda3924f8dd9109 diff --git a/.agents/notes/implemented/feature/2026-06-15-code-mode.md b/.agents/notes/implemented/feature/2026-06-15-code-mode.md index 1be25f5b14..a075ea66c1 100644 --- a/.agents/notes/implemented/feature/2026-06-15-code-mode.md +++ b/.agents/notes/implemented/feature/2026-06-15-code-mode.md @@ -20,7 +20,7 @@ Three decisions, each elaborated in its own section below: 1. **Code Mode is a first-class presentation mode of `ToolRegistry`** (`dsh-tools`), selected by a validated `mode` config: `'native'` (the default, contributing the visible capability schemas), `'code'` (the registry contributes only its reserved `run_code` transport plus a generated SDK `.d.ts` in the system prompt), or `'both'` (native schemas and the transport + SDK). The registry shapes its canonical contribution at the source; the cooperative prompt-assembly result remains authoritative, and the logged request header records exactly that returned presentation. 2. **Code execution is a capability seam** — `packages/code-runtime/` contains the interface package `@deepseek-ai/dsh-code-runtime`, which owns `ctx.codeRuntime` ([capability seams](../architecture/2026-06-13-capability-seams.md); consumer = `dsh-tools`, with core-consumes-a-seam precedent in `agent-loop` → `dsh-llm`). The runtime knows nothing about tools: it is handed a program and named async bindings, runs the program, and reports `{ value, logs, error? }`. Language and substrate are backend properties, so a future Python or container backend is another implementation package, not a redesign. -3. **Two implementations preserve one fresh-worker contract**: `@deepseek-ai/dsh-code-runtime-worker` runs the worker in the harness process, while `@deepseek-ai/dsh-code-runtime-subprocess` materializes a runner through `ctx.fs` and launches it through `ctx.subprocess` for another execution world. Both execute host-stripped TypeScript in a fresh Node worker with an empty environment, bridged bindings, configurable heap/output/time caps, and hard termination. Their trust posture is bash-equivalent by design; stronger isolation comes from the mounted execution world. +3. **Two implementations preserve one fresh-worker contract**: `@deepseek-ai/dsh-code-runtime-worker` runs the worker in the harness process, while `@deepseek-ai/dsh-code-runtime-subprocess` passes a bundled eval runner to `ctx.subprocess` for another execution world. Both execute host-stripped TypeScript in a fresh Node worker with an empty environment, bridged bindings, configurable heap/output/time caps, and hard termination. Their trust posture is bash-equivalent by design; stronger isolation comes from the mounted execution world. This note owns Code Mode's presentation, composition, isolation, and settlement foundation. The later [typed tool-return Agent Note](2026-07-20-code-mode-typed-tool-returns.md) owns the generated output map, canonical binding values, `ToolCallError`, and the lossless outer-output boundary. @@ -79,7 +79,7 @@ Requests contain every runtime input; implementations own validated timeout and 5. **Enforce independent budgets.** `computeMs` meters worker busy time, allowing slow awaited tools without excusing a hot loop. `maxWallMs` bounds total elapsed time, including unresolved waits. `maxOutputBytes` bounds only the combined serialized outer logs, completion, or diagnostic; intermediate binding values have no byte cap. Expiry, cancellation, and completion terminate the worker, and heap exits or outer overflow are explicit failures. 6. **Dispose to quiescence**: the service's own disposal terminates in-flight workers and *awaits* their exits before resolving, per [defensive patterns](../../../../docs/defensive-patterns.md). -`@deepseek-ai/dsh-code-runtime-subprocess` preserves those program, binding, output, and worker-budget semantics across a filesystem/subprocess execution world. It writes a dependency-free runner below `ctx.subprocess.runtimeRoot`, resolves Node through the provider, and carries binding traffic over bounded newline-delimited UTF-8 JSON frames on raw pipes. The heap-bounded worker rejects expanded completion wires before MessagePort transfer; terminal settlement asks the launcher to reap its controller and keeps process-group escalation armed until `waitForExit()` confirms whole-tree quiescence. The [portable execution-world decision](../architecture/2026-07-28-portable-execution-world-consumers.md) owns why this generic backend replaces provider-specific Code Runtime packages; `dsh-code-runtime-worker` remains the single-process and single-file-distribution path. +`@deepseek-ai/dsh-code-runtime-subprocess` preserves those program, binding, output, and worker-budget semantics across another process world. It passes a dependency-free eval runner directly through the subprocess provider and carries binding traffic over bounded newline-delimited UTF-8 JSON frames on raw pipes. The heap-bounded worker rejects expanded completion wires before MessagePort transfer; terminal settlement asks the launcher to reap its controller and keeps process-group escalation armed until `waitForExit()` confirms whole-tree quiescence. The [portable execution-world decision](../architecture/2026-07-28-portable-execution-world-consumers.md) owns why this generic backend replaces provider-specific Code Runtime packages; `dsh-code-runtime-worker` remains the single-process and single-file-distribution path. ### Trust posture @@ -95,7 +95,7 @@ Deployments switching to `'code'` must update any native-only `toolOrder`. Assem ## Testing -- **Runtime implementations:** Real-worker suites cover typed binding values and failures, every lossless JSON completion root, invalid and over-limit output, exact combined ledger and per-hop frame boundaries, compute and wall budgets, hostile binding traffic, empty environment, descendant lifetime cleanup, and disposal to quiescence. Built-package tests run both the direct worker entry and the filesystem/subprocess composition under plain Node; the latter also has a Loader-driven `cordis.yml` test. +- **Runtime implementations:** Real-worker suites cover typed binding values and failures, every lossless JSON completion root, invalid and over-limit output, exact combined ledger and per-hop frame boundaries, compute and wall budgets, hostile binding traffic, empty environment, descendant lifetime cleanup, and disposal to quiescence. Built-package tests run both the direct worker entry and the subprocess composition under plain Node; the latter also has a Loader-driven `cordis.yml` test. - **Registry integration:** Tests cover code generation, all presentation modes, reserved-name and restriction rules, scoped visibility, authoritative assembly rewrites, `toolOrder`, runtime compatibility failures, full-pipeline sub-dispatch, parent-token correlation, serialization, cancellation and queue drain, JSON normalization, error propagation, log events, ordered context deferral across successful and failed programs, outer-block suppression, and HMR cleanup. - **With-key e2e:** A real model composes two bash calls in one program; another discovers nested workspace instructions through a Code Mode fs dispatch. The tests verify collapsed request headers, correlated dispatch events, resulting files, deferred context, and model behavior. - **Snapshot:** The `code-mode-turn`, `both-mode-turn`, and `code-mode-workspace-context` fixtures pin SDK text, header tool lists, dispatch events, deferred context, and result cards. diff --git a/.agents/notes/implemented/feature/2026-06-15-code-mode.zh.md b/.agents/notes/implemented/feature/2026-06-15-code-mode.zh.md index a4e8fbbd33..c269c0a870 100644 --- a/.agents/notes/implemented/feature/2026-06-15-code-mode.zh.md +++ b/.agents/notes/implemented/feature/2026-06-15-code-mode.zh.md @@ -20,7 +20,7 @@ Cloudflare 的 [Code Mode](https://blog.cloudflare.com/code-mode/) 提出了一 1. **Code Mode 是 `ToolRegistry`(`dsh-tools`)的一等呈现模式**,通过经校验的 `mode` 配置选择:`'native'`(默认,贡献可见能力 schema)、`'code'`(注册表仅贡献其保留的 `run_code` 传输通道加一份生成的 SDK `.d.ts` 到系统提示词中)或 `'both'`(原生 schema 加传输通道 + SDK)。注册表在源头塑造其权威贡献;协作式提示词组装的结果仍具权威性,记录在日志中的请求头精确反映该返回的呈现。 2. **代码执行是一个能力 seam**——`packages/code-runtime/` 包含接口包 `@deepseek-ai/dsh-code-runtime`,拥有 `ctx.codeRuntime`([能力 seam](../architecture/2026-06-13-capability-seams.md);消费方 = `dsh-tools`,core 消费 seam 的先例见 `agent-loop` → `dsh-llm`)。运行时对工具一无所知:它接收一段程序和命名的异步绑定,执行程序,报告 `{ value, logs, error? }`。语言和基底是后端属性,因此未来的 Python 或容器后端只是另一个实现包,而非重新设计。 -3. **两种实现保持同一份全新 worker 契约**:`@deepseek-ai/dsh-code-runtime-worker` 在 harness 进程内运行 worker,`@deepseek-ai/dsh-code-runtime-subprocess` 则通过 `ctx.fs` 物化 runner,再通过 `ctx.subprocess` 在另一执行环境中启动。二者都在具有空环境的全新 Node worker 内执行由宿主剥离类型的 TypeScript,并提供桥接绑定、可配置的堆/输出/时间上限和硬终止。其信任姿态在设计上等同于 bash;更强的隔离来自挂载的执行环境。 +3. **两种实现保持同一份全新 worker 契约**:`@deepseek-ai/dsh-code-runtime-worker` 在 harness 进程内运行 worker,`@deepseek-ai/dsh-code-runtime-subprocess` 则将一个内置的 eval runner 传给 `ctx.subprocess`,用于另一执行环境。二者都在具有空环境的全新 Node worker 内执行由宿主剥离类型的 TypeScript,并提供桥接绑定、可配置的堆/输出/时间上限和硬终止。其信任姿态在设计上等同于 bash;更强的隔离来自挂载的执行环境。 本说明负责定义 Code Mode 的呈现、组合、隔离与结算基础。后续的[类型化工具返回值 Agent Note](2026-07-20-code-mode-typed-tool-returns.md)负责定义生成的输出映射、规范绑定值、`ToolCallError` 和无损外层输出边界。 @@ -79,7 +79,7 @@ Cloudflare 的 [Code Mode](https://blog.cloudflare.com/code-mode/) 提出了一 5. **强制独立预算。** `computeMs` 计量 worker 忙碌时间,允许慢速的 awaited 工具而不放过热循环。`maxWallMs` 约束总经过时间,包括未解析的等待。`maxOutputBytes` 只约束序列化后的外层日志、完成值或诊断的组合;中间绑定值没有字节数上限。到期、取消和完成都终止 worker,堆退出或外层溢出会作为显式失败报告。 6. **dispose 至完全停稳**:服务自身的 dispose(资源释放)终止进行中的 worker 并*等待*其退出后再 resolve,遵循[防御性模式](../../../../docs/defensive-patterns.md)。 -`@deepseek-ai/dsh-code-runtime-subprocess` 在文件系统/子进程执行环境中保持相同的程序、绑定、输出与 worker 预算语义。它在 `ctx.subprocess.runtimeRoot` 下写入一个无依赖 runner,通过提供方解析 Node,并在原始管道上使用有界的、以换行符分隔的 UTF-8 JSON 帧承载绑定通信。受堆上限约束的 worker 会在通过 MessagePort 传输前拒绝展开后的完成值 wire;终态结算会请求 launcher 回收其 controller,并让进程组升级终止机制保持待命,直至 `waitForExit()` 确认整棵进程树完全停稳。[可移植执行环境决策](../architecture/2026-07-28-portable-execution-world-consumers.md)说明为何这个通用后端会取代提供方专用的 Code Runtime 包;`dsh-code-runtime-worker` 仍用于单进程和单文件发行版。 +`@deepseek-ai/dsh-code-runtime-subprocess` 在另一进程执行环境中保持相同的程序、绑定、输出与 worker 预算语义。它通过子进程提供方直接传递一个无依赖的 eval runner,并在原始管道上使用有界的、以换行符分隔的 UTF-8 JSON 帧承载绑定通信。受堆上限约束的 worker 会在通过 MessagePort 传输前拒绝展开后的完成值 wire;终态结算会请求 launcher 回收其 controller,并让进程组升级终止机制保持待命,直至 `waitForExit()` 确认整棵进程树完全停稳。[可移植执行环境决策](../architecture/2026-07-28-portable-execution-world-consumers.md)说明为何这个通用后端会取代提供方专用的 Code Runtime 包;`dsh-code-runtime-worker` 仍用于单进程和单文件发行版。 ### 信任姿态 @@ -95,7 +95,7 @@ SDK 指示模型编写一个异步的可擦除 TypeScript 函数体,通过 `aw ## 测试 -- **运行时实现:** 真实 worker 测试套件覆盖类型化的绑定值与失败、每一种无损 JSON 根类型的完成值、无效和超限输出、精确的组合账本边界与逐跳帧边界、compute 和 wall 预算、恶意绑定流量、空环境、后代进程生命周期清理以及 dispose 至完全停稳。构建后包测试会在纯 Node 下分别运行直接 worker 入口与文件系统/子进程组合;后者另有一个由 Loader 驱动的 `cordis.yml` 测试。 +- **运行时实现:** 真实 worker 测试套件覆盖类型化的绑定值与失败、每一种无损 JSON 根类型的完成值、无效和超限输出、精确的组合账本边界与逐跳帧边界、compute 和 wall 预算、恶意绑定流量、空环境、后代进程生命周期清理以及 dispose 至完全停稳。构建后包测试会在纯 Node 下分别运行直接 worker 入口与子进程组合;后者另有一个由 Loader 驱动的 `cordis.yml` 测试。 - **注册表集成:** 测试覆盖代码生成、所有呈现模式、保留名称和限制规则、scoped 可见性、权威组装重写、`toolOrder`、运行时兼容性失败、完整流水线子分发、parent-token 关联、序列化、取消和队列排空、JSON 规范化、错误传播、日志事件、成功与失败程序中的有序上下文延后、外层阻止抑制以及 HMR(热模块替换)清理。 - **带密钥 e2e:** 真实模型在一个程序中组合两次 bash 调用;另一个模型通过 Code Mode fs 分发发现嵌套的工作区指令。测试验证折叠的请求头、关联的分发事件、结果文件、延后上下文和模型行为。 - **快照:** `code-mode-turn`、`both-mode-turn` 和 `code-mode-workspace-context` fixture(测试前置数据)固定 SDK 文本、请求头工具列表、分发事件、延后上下文和结果卡片。 diff --git a/docs/config-catalog.md b/docs/config-catalog.md index 0514cdd969..a875c190fe 100644 --- a/docs/config-catalog.md +++ b/docs/config-catalog.md @@ -286,7 +286,7 @@ Source: [`packages/client/hmr/src/index.ts:29`](../packages/client/hmr/src/index ## `@deepseek-ai/dsh-code-runtime-subprocess` -Requires: `fs` · `subprocess` +Requires: `subprocess` ```ts config-catalog /** Runtime configuration; every execution and bridge bound is deployment-tunable. */ @@ -306,7 +306,7 @@ export interface Config { } ``` -Source: [`packages/code-runtime/code-runtime-subprocess/src/index.ts:28`](../packages/code-runtime/code-runtime-subprocess/src/index.ts) +Source: [`packages/code-runtime/code-runtime-subprocess/src/index.ts:27`](../packages/code-runtime/code-runtime-subprocess/src/index.ts) ## `@deepseek-ai/dsh-code-runtime-worker` diff --git a/docs/core-data-structures/subprocess.i18n.yaml b/docs/core-data-structures/subprocess.i18n.yaml index f3b308b0be..f6c61bd691 100644 --- a/docs/core-data-structures/subprocess.i18n.yaml +++ b/docs/core-data-structures/subprocess.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write docs/core-data-structures/subprocess.md -subprocess.md: 95e9454a75112cf98e1ed583f970c50fe90aac37 -subprocess.zh.md: 58127d662fa599d19f632449d42b8aa3259886d6 +subprocess.md: f0fe691219df3afba2b836da8f1594fd5fe5a87b +subprocess.zh.md: f59e787820ba2727fdc0c4b62c4aec7b8d2d6b7a diff --git a/docs/core-data-structures/subprocess.md b/docs/core-data-structures/subprocess.md index 935a7d60ac..89f30095bc 100644 --- a/docs/core-data-structures/subprocess.md +++ b/docs/core-data-structures/subprocess.md @@ -8,7 +8,7 @@ Source: [`packages/subprocess/subprocess/src/types.ts`](../../packages/subproces ## Execution-world coordinates -One provider's `cwd`, `runtimeRoot`, executable paths, ordinary processes, and terminal sessions inhabit the same path and process namespace as the mounted filesystem provider. `resolveExecutable(command, env?, signal?)` verifies absolute executable paths or resolves bare names through the provider's scrubbed `PATH` plus deliberate overrides. Consumers use `runtimeRoot` for private materialized helpers and never assume a host path exists in that world. +One provider's `cwd`, executable paths, ordinary processes, and terminal sessions inhabit the same path and process namespace as the mounted filesystem provider. `resolveExecutable(command, env?, signal?)` verifies absolute executable paths or resolves bare names through the provider's scrubbed `PATH` plus deliberate overrides. ## Managed environment namespace and captured output diff --git a/docs/core-data-structures/subprocess.zh.md b/docs/core-data-structures/subprocess.zh.md index 58127d662f..f59e787820 100644 --- a/docs/core-data-structures/subprocess.zh.md +++ b/docs/core-data-structures/subprocess.zh.md @@ -8,7 +8,7 @@ ## 执行世界坐标 -一个提供方的 `cwd`、`runtimeRoot`、可执行文件路径、普通进程与终端会话,和挂载的文件系统提供方处于同一路径与进程命名空间。`resolveExecutable(command, env?, signal?)` 验证绝对可执行文件路径,或通过提供方清理后的 `PATH` 加有意覆盖来解析裸名称。消费方使用 `runtimeRoot` 存放私有物化辅助程序,绝不假设该执行世界中存在某条宿主路径。 +一个提供方的 `cwd`、可执行文件路径、普通进程与终端会话,和挂载的文件系统提供方处于同一路径与进程命名空间。`resolveExecutable(command, env?, signal?)` 验证绝对可执行文件路径,或通过提供方清理后的 `PATH` 加有意覆盖来解析裸名称。 ## 受管环境命名空间与捕获的输出 diff --git a/packages/README.i18n.yaml b/packages/README.i18n.yaml index c5be7e67d6..eaa48fac05 100644 --- a/packages/README.i18n.yaml +++ b/packages/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/README.md -README.md: 7fcd98bd19ca3291f0472af57841f2f763bb346f -README.zh.md: 7c4aed9b343ec57001883e094ea3dd0d73e2a920 +README.md: b498c146202f1a69d89805fa3e8966c4d0735d6c +README.zh.md: 2eefc73ed0ea22aae5fde93284c25c5d03fe86c4 diff --git a/packages/README.md b/packages/README.md index a711e4d303..7d591605ef 100644 --- a/packages/README.md +++ b/packages/README.md @@ -19,7 +19,7 @@ Packages live at `packages///`; groups are containers, while names r | [`subprocess/`](subprocess/README.md) | Subprocess capability family: spawn seam + local process-tree implementation | Product — stable surface | | [`bash/`](bash/README.md) | Bash capability family: executor seam, local impl, model-facing tool | Product — stable surface | | [`pty/`](pty/README.md) | Persistent PTY capability family: owner-scoped sessions, local implementation, and model-facing tools | Product — stable surface | -| [`code-runtime/`](code-runtime/README.md) | Code-execution capability family: runtime seam plus local worker and filesystem/subprocess backends | Product — stable surface | +| [`code-runtime/`](code-runtime/README.md) | Code-execution capability family: runtime seam plus local worker and subprocess backends | Product — stable surface | | [`sandbox/`](sandbox/README.md) | Process-confinement seam; bwrap/Landlock/Seatbelt backends | Product — stable surface | | [`fs/`](fs/README.md) | Filesystem capability family: seam, local impl, model-facing file tools, bash-backed discovery tools | Product — stable surface | | [`lsp/`](lsp/README.md) | LSP capability family: seam, generic stdio provider, and the `lsp` tool | Product — stable surface | diff --git a/packages/README.zh.md b/packages/README.zh.md index 41380317f8..a9201d08a9 100644 --- a/packages/README.zh.md +++ b/packages/README.zh.md @@ -19,7 +19,7 @@ | [`subprocess/`](subprocess/README.md) | 进程管理能力系列:spawn seam + 本地进程树实现 | 产品:稳定表面 | | [`bash/`](bash/README.md) | Bash 能力系列:执行器 seam、本地实现、面向模型的工具 | 产品:稳定表面 | | [`pty/`](pty/README.md) | 持久 PTY 能力系列:按所有者隔离的会话、本地实现和面向模型的工具 | 产品:稳定表面 | -| [`code-runtime/`](code-runtime/README.md) | 代码执行能力系列:运行时 seam、本地 worker 后端及文件系统/进程管理后端 | 产品:稳定表面 | +| [`code-runtime/`](code-runtime/README.md) | 代码执行能力系列:运行时 seam、本地 worker 后端及进程管理后端 | 产品:稳定表面 | | [`sandbox/`](sandbox/README.md) | 进程限制 seam;bwrap/Landlock/Seatbelt 后端 | 产品:稳定表面 | | [`fs/`](fs/README.md) | 文件系统能力系列:seam、本地实现、面向模型的文件工具、bash 后端发现工具 | 产品:稳定表面 | | [`lsp/`](lsp/README.md) | LSP 能力系列:seam、通用 stdio 提供方和 `lsp` 工具 | 产品:稳定表面 | diff --git a/packages/bash/pwsh-local/tsconfig.json b/packages/bash/pwsh-local/tsconfig.json index 53ccc94926..7381575674 100644 --- a/packages/bash/pwsh-local/tsconfig.json +++ b/packages/bash/pwsh-local/tsconfig.json @@ -18,19 +18,19 @@ "path": "../../../vendor/schemastery" }, { - "path": "../../util/brand" + "path": "../code-runtime" }, { - "path": "../../util/timeout" - }, - { - "path": "../../bash/bash" + "path": "../code-runtime-worker" }, { "path": "../../subprocess/subprocess" }, { "path": "../../support/invariants" + }, + { + "path": "../../util/timeout" } ] } diff --git a/packages/code-runtime/code-runtime/README.md b/packages/code-runtime/code-runtime/README.md index 7014bf517a..d58508cd08 100644 --- a/packages/code-runtime/code-runtime/README.md +++ b/packages/code-runtime/code-runtime/README.md @@ -34,5 +34,5 @@ No direct invalidation; the named consumer owns any request-prefix changes. - **`run()` is one-shot** — `logs` arrive only on the resolved `CodeRunResult`; the seam exposes no streaming-log or progress surface for a live program's output. - **A persistent REPL-style kernel is recorded future work** — the no-state-between-runs contract stands until a persistent-kernel backend brings its own logging story ([Code Mode Agent Note](../../../.agents/notes/implemented/feature/2026-06-15-code-mode.md)). -- **No runtime claims a hard security boundary** — both shipped implementations use fresh worker threads; the filesystem/subprocess backend can place them inside a stronger execution world, but no runtime reports `'container'` today. -- **Intermediate binding values are implementation-bounded** — the direct worker backend has no per-binding byte cap; the filesystem/subprocess backend bounds each bridge frame, but repeated or concurrent binding traffic remains subject to process memory. +- **No runtime claims a hard security boundary** — both shipped implementations use fresh worker threads; the subprocess backend can place them inside a stronger execution world, but no runtime reports `'container'` today. +- **Intermediate binding values are implementation-bounded** — the direct worker backend has no per-binding byte cap; the subprocess backend bounds each bridge frame, but repeated or concurrent binding traffic remains subject to process memory. diff --git a/packages/pty/pty-local/tests/index.spec.ts b/packages/pty/pty-local/tests/index.spec.ts index f59cd3ab5a..15e73ddc1c 100644 --- a/packages/pty/pty-local/tests/index.spec.ts +++ b/packages/pty/pty-local/tests/index.spec.ts @@ -67,7 +67,6 @@ function terminalHandle(): SubprocessTerminalHandle { class StubSubprocessService extends SubprocessService { readonly cwd = '/tmp' - readonly runtimeRoot = '/tmp/dsh-runtime' async resolveExecutable(command: string): Promise { return command } spawn(_spec: SubprocessSpawnSpec): SubprocessHandle { throw new Error('unused') } async spawnTerminal(_spec: SubprocessTerminalSpawnSpec): Promise { diff --git a/packages/subprocess/subprocess-local/README.i18n.yaml b/packages/subprocess/subprocess-local/README.i18n.yaml index d34e7475f9..8b668b43f6 100644 --- a/packages/subprocess/subprocess-local/README.i18n.yaml +++ b/packages/subprocess/subprocess-local/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/subprocess/subprocess-local/README.md -README.md: 31c5539750c4af2b1c4169ac1dce4c91da587af7 -README.zh.md: 74aed52084c0db3ed9bfe4fcb991f781976fb7b5 +README.md: 4892ad49e571662b833b1dd6d64dc966f343ae3c +README.zh.md: ca0df5ff18e9feb992cc3f6d0db90dc06cb275d8 diff --git a/packages/subprocess/subprocess-local/README.md b/packages/subprocess/subprocess-local/README.md index 31c5539750..13b51ec6d1 100644 --- a/packages/subprocess/subprocess-local/README.md +++ b/packages/subprocess/subprocess-local/README.md @@ -2,7 +2,7 @@ English | [中文](README.zh.md) -Local implementation of the [`@deepseek-ai/dsh-subprocess`](../subprocess/README.md) seam. `LocalSubprocessService` owns a private runtime directory, resolves local executables, spawns ordinary detached process trees with explicit stdio, and implements terminal processes through `node-pty` plus platform process inspection. It has no config: every disposition, limit, terminal dimension, grace, and directory arrives from the calling seams ([`dsh-bash-local`](../../bash/bash-local/README.md), [`dsh-lsp-local`](../../lsp/lsp-local/README.md), [`dsh-pty-local`](../../pty/pty-local/README.md), and [`dsh-code-runtime-subprocess`](../../code-runtime/code-runtime-subprocess/README.md)). +Local implementation of the [`@deepseek-ai/dsh-subprocess`](../subprocess/README.md) seam. `LocalSubprocessService` resolves local executables, spawns ordinary detached process trees with explicit stdio, and implements terminal processes through `node-pty` plus platform process inspection. It has no config: every disposition, limit, terminal dimension, grace, and directory arrives from the calling seams ([`dsh-bash-local`](../../bash/bash-local/README.md), [`dsh-lsp-local`](../../lsp/lsp-local/README.md), [`dsh-pty-local`](../../pty/pty-local/README.md), and [`dsh-code-runtime-subprocess`](../../code-runtime/code-runtime-subprocess/README.md)). ## Behavior (and where it came from) @@ -10,7 +10,7 @@ Local implementation of the [`@deepseek-ai/dsh-subprocess`](../subprocess/README - **Per-stream dispositions** — `'pipe'` hands the raw stream to the caller untouched (protocol framing stays consumer-owned); `'inherit'` passes the parent descriptor through; collect mode keeps the in-memory TAIL beyond its cap (errors and results cluster at the end — pi/OpenCode rationale) while the FULL stream is appended to a private temp file when a spill cap is configured — omitting `spill` keeps only the tail, the diagnostic shape. A stream larger than the spill cap discards its now-incomplete spill and returns only the marked truncated tail; spill fds are sealed at settlement, and a failed final close withholds the path rather than advertising an incomplete file. Spill files are `0600` with random names under a lazily-created `0700` per-process directory. - **Credential scrub + explicit merge** — `process.env` minus credential-shaped vars (`*KEY*`/`*SECRET*`/`*TOKEN*`) and all ambient `DSH_*` names; the spec's explicit `env` merges after that scrub with no namespace validation, so a deliberately supplied credential or current `DSH_*` fact wins while stale nested-harness identity cannot leak in ambiently. Supplied stdin is written and closed; otherwise fd 0 is `/dev/null`. See the [stdin/env Agent Note](../../../.agents/notes/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-surface.md) and [managed environment Agent Note](../../../.agents/notes/implemented/feature/2026-07-10-agent-session-identity-and-log-location.md). - **Offset-based reads** — collect-mode readers return deltas in whole-stream byte coordinates; the service never holds a cursor, so consumer-owned cursors (the bash background read path) and full-stream re-reads coexist, before and after settlement. -- **Execution-world coordinates** — `cwd` is the host process cwd, `runtimeRoot` is an owner-private temporary directory removed on disposal before any process-cleanup failure is reported, and `resolveExecutable` checks absolute files or searches the scrubbed effective PATH with platform-aware executable extensions. +- **Execution-world coordinates** — `cwd` is the host process cwd, and `resolveExecutable` checks absolute files or searches the scrubbed effective PATH with platform-aware executable extensions. - **Terminal-process ownership** — `spawnTerminal` allocates `node-pty`, bridges UTF-8 terminal text, inspects and signals the current foreground process group, and exposes one awaited termination operation that sweeps descendants before and after terminating the top-level shell. Each foreground inspection retains exact identities from the rooted tree; Linux also enumerates the POSIX session after its leader exits. A previously observed macOS descendant and any same-session Linux member therefore remain fenced after reparenting, while pid/start identity prevents cleanup from following PID reuse. The higher PTY backend owns prompt readiness, buffers, and model-facing operations. - **Terminate-and-join disposal** — the service retains live handles only so its own disposal can escalate every running tree and await its exit; settled and spawn-failed handles leave the live set on settlement. diff --git a/packages/subprocess/subprocess-local/README.zh.md b/packages/subprocess/subprocess-local/README.zh.md index 74aed52084..ca0df5ff18 100644 --- a/packages/subprocess/subprocess-local/README.zh.md +++ b/packages/subprocess/subprocess-local/README.zh.md @@ -2,17 +2,17 @@ [English](README.md) | 中文 -[`@deepseek-ai/dsh-subprocess`](../subprocess/README.md) seam 的本地实现。`LocalSubprocessService` 拥有私有运行时目录,解析本地可执行文件,以显式 stdio spawn 普通 detached 进程树,并通过 `node-pty` 与平台进程检查实现终端进程。该实现没有任何配置:每项处置方式、限制、终端尺寸、宽限期与目录都来自调用方 seam([`dsh-bash-local`](../../bash/bash-local/README.md)、[`dsh-lsp-local`](../../lsp/lsp-local/README.md)、[`dsh-pty-local`](../../pty/pty-local/README.md)和 [`dsh-code-runtime-subprocess`](../../code-runtime/code-runtime-subprocess/README.md))。 +[`@deepseek-ai/dsh-subprocess`](../subprocess/README.md) seam 的本地实现。`LocalSubprocessService` 解析本地可执行文件,以显式 stdio spawn 普通 detached 进程树,并通过 `node-pty` 加平台进程检查实现终端进程。该实现没有任何配置:每项处置方式、限制、终端尺寸、宽限期与目录都来自调用方 seam([`dsh-bash-local`](../../bash/bash-local/README.md)、[`dsh-lsp-local`](../../lsp/lsp-local/README.md)、[`dsh-pty-local`](../../pty/pty-local/README.md) 和 [`dsh-code-runtime-subprocess`](../../code-runtime/code-runtime-subprocess/README.md))。 ## 行为(以及设计来源) -- **带平台正确信号发送的 detached 进程树**:POSIX 子进程使用 `detached` spawn(拥有独立进程组),信号以负 pgid 发送并以直接子进程作为回退;Windows 通过 `taskkill /PID /T /F` 终止进程树(可为测试注入)。`terminate()`(句柄唯一的终止动词)先发送 SIGTERM,经过 spec 的宽限期后再发送 SIGKILL(沿用 OpenCode 的升级策略;管道与子 shell 会随父进程一起结束),进程树消亡后为空操作;`waitForExit()` 轮询整棵进程树的存活状态,使消费方的拆卸能确认真正的完全停稳。组长进程退出后,仍然打开的管道也只获得同样有界的排空宽限期,因此存活的后代进程无法无限期地拖住结果不结算。系统会容忍 ESRCH;脱离该组重新挂载的 daemon 仍可能存活,这与调研工具的局限相同。 -- **按流划分的处置方式**:`'pipe'` 把原始流原样交给调用方(协议分帧仍归消费方所有);`'inherit'` 直通父进程的描述符;收集模式(collect)在输出超过上限后于内存中保留尾部(错误与结果通常聚集在末尾,沿用 pi/OpenCode 的理由),并在配置了 spill 上限时把完整流追加到一个私有临时文件;省略 `spill` 则只保留尾部,即诊断尾部的形状。某条流大于 spill 上限时,会丢弃已不完整的 spill,仅返回带截断标记的尾部;spill 文件描述符在结算时封存,最终关闭失败时则不公布路径,以免声称存在不完整的文件。spill 文件权限为 `0600`、名称随机,位于按需延迟创建的 `0700` 每进程目录之下。 -- **凭据清除 + 显式合并**:以 `process.env` 为基础,移除形似凭据的变量(`*KEY*`/`*SECRET*`/`*TOKEN*`)和所有环境中已有的 `DSH_*` 名称;spec 的显式 `env` 在该清除之后合并且不做命名空间校验,因此有意提供的凭据或当前 `DSH_*` 事实会胜出,而陈旧的嵌套 harness 身份无法从环境中隐式漏入。提供的 stdin 会被写入后关闭;否则 fd 0 指向 `/dev/null`。参见 [stdin/env Agent Note(agent 决策记录)](../../../.agents/notes/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-surface.md)与[受管环境 Agent Note](../../../.agents/notes/implemented/feature/2026-07-10-agent-session-identity-and-log-location.md)。 -- **基于偏移量的读取**:收集模式的读取器以全流字节坐标返回增量;服务自身从不持有游标,因此消费方自有的游标(bash 的后台读取路径)与完整流重读可以共存,结算前后皆然。 -- **执行世界坐标**:`cwd` 是宿主进程 cwd,`runtimeRoot` 是所有者私有的临时目录,会在资源释放时删除,并且删除发生在报告任何进程清理失败之前;`resolveExecutable` 检查绝对文件,或使用平台感知的可执行扩展名在清理后的有效 PATH 中查找。 -- **终端进程所有权**:`spawnTerminal` 分配 `node-pty`,桥接 UTF-8 终端文本,检查当前前台进程组并向其发送信号,并公开一项须等待的终止操作,该操作会在终止顶层 shell 前后清理后代。每次前台检查都会保留有根进程树中的精确身份;Linux 还会在会话 leader 退出后枚举该 POSIX 会话。因此,先前观察到的 macOS 后代以及任何同会话 Linux 成员在重新设定父进程后仍受身份围栏保护,而 pid/启动身份可防止清理因 PID 复用而跟随到其他进程。上层 PTY 后端负责提示符就绪检测、缓冲和面向模型的操作。 -- **先终止再等待退出的 dispose**:服务保留存活句柄,只为让自身的 dispose 能对每个仍在运行的进程树执行升级并等待其退出;已结算与 spawn 失败的句柄在结算时即离开存活集合。 +- **以适合平台的方式发送信号的 detached 进程树**:POSIX 子进程使用 `detached` spawn(拥有独立进程组),信号以负 pgid 发送并以直接子进程作为回退;Windows 通过 `taskkill /PID /T /F` 终止进程树(可为测试注入)。`terminate()`(句柄唯一的终止操作)先发送 SIGTERM,经过 spec 的宽限期后再发送 SIGKILL(沿用 OpenCode 的升级策略;流水线与子 shell 会随父进程一起结束),进程树消亡后为空操作;`waitForExit()` 轮询整棵进程树的存活状态,使消费方的拆卸能确认真正的完全停稳。组长进程退出后,仍然打开的管道也只获得同样有界的排空宽限期,因此存活的后代进程无法无限期地拖住结果不结算。系统会容忍 ESRCH;重新指定父进程并脱离该组的 daemon 仍可能存活,这与所调研工具的局限相同。 +- **按流划分的处置方式**:`'pipe'` 把原始流原样交给调用方(协议分帧仍归消费方所有);`'inherit'` 直通父进程的描述符;收集模式(collect)在输出超过上限后于内存中保留尾部(错误与结果通常聚集在末尾,沿用 pi/OpenCode 的理由),并在配置了 spill 上限时把完整流追加到一个私有临时文件;省略 `spill` 则只保留用于诊断的尾部。某条流大于 spill 上限时,会丢弃已不完整的 spill,仅返回带截断标记的尾部;spill 文件描述符在结算时封存,最终关闭失败时则不公布路径,以免声称存在不完整的文件。spill 文件权限为 `0600`、名称随机,位于按需创建、权限为 `0700` 的每进程目录之下。 +- **凭据清除 + 显式合并**:以 `process.env` 为基础,移除形似凭据的变量(`*KEY*`/`*PASSWORD*`/`*SECRET*`/`*TOKEN*`)和所有环境中已有的 `DSH_*` 名称;spec 的显式 `env` 在该清除之后合并且不做命名空间校验,因此有意提供的凭据或当前 `DSH_*` 事实会胜出,而陈旧的嵌套 harness 身份无法从环境中隐式漏入。提供的 stdin 会被写入后关闭;否则 fd 0 指向 `/dev/null`。参见 [stdin/env Agent Note(agent 决策记录)](../../../.agents/notes/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-surface.md)与[受管环境 Agent Note](../../../.agents/notes/implemented/feature/2026-07-10-agent-session-identity-and-log-location.md)。 +- **基于偏移量的读取**:收集模式的读取器按完整流的字节坐标返回增量;服务自身从不持有游标,因此消费方自有的游标(bash 的后台读取路径)与完整流重读可以共存,结算前后皆然。 +- **执行世界坐标**:`cwd` 是宿主进程 cwd,`resolveExecutable` 检查绝对文件,或根据平台可执行文件扩展名在清理后的有效 PATH 中搜索。 +- **终端进程所有权**:`spawnTerminal` 分配 `node-pty`,桥接 UTF-8 终端文本,检查当前前台进程组并向其发送信号,还会公开一项须等待的终止操作,在终止顶层 shell 前后清理后代进程。每次前台检查都会保留根进程树中的精确身份;Linux 还会在 POSIX 会话 leader 退出后枚举该会话。因此,之前观察到的 macOS 后代以及同会话 Linux 成员在重新设定父进程后仍受围栏保护,pid/start 身份则防止清理跟随 PID 复用。上层 PTY 后端负责提示符就绪、缓冲区与面向模型的操作。 +- **先终止再等待退出的 dispose(资源释放)**:服务保留存活句柄,只为让自身的 dispose 能对每个仍在运行的进程树执行升级并等待其退出;已结算与 spawn 失败的句柄在结算时即离开存活集合。 ## 模型体验 @@ -20,14 +20,14 @@ #### KV Cache 影响 -不会直接失效;请求前缀变更由具名消费方负责。 +不会直接导致 KV Cache 失效;请求前缀变更由上述消费方负责。 ## 已知限制与暂缓事项 - **Windows 进程树支持仅为尽力而为,且未经 CI 测试**:终止经由 `taskkill /PID /T /F` 完成,所有结果都被就地吸收,不向外抛出(进程树已不存在、竞态、二进制缺失),存活探测则回退到直接子进程边界;测试套件只通过注入的运行器覆盖这条路由,且 `packages/subprocess/*` 被排除在 Windows 测试矩阵之外。 -- **终端进程检查仅支持 Linux/macOS**:检查器没有受支持的平台实现时,终端原语会失败;Linux 精确探针覆盖 x64 与 arm64,macOS 使用 `ps` 快照。 -- **守护化的终端后代仍可能逃出可观察边界**:在 macOS 上,子进程若在任何前台检查快照产生前重新设定父进程,便无法再从 `node-pty` 根发现;在 Linux 上,调用 `setsid` 的子进程会同时离开进程树与该提供方拥有的终端会话。本地提供方不会增加持续运行的进程表监视器。 -- **凭据清除依赖名称启发式规则**:只匹配 `*KEY*`/`*SECRET*`/`*TOKEN*`;名称不同的 secret(例如 `*PASSWORD*`)会继续传递,对误删变量引入白名单属于已记录的后续工作。 +- **终端进程检查仅支持 Linux/macOS**:检查器没有受支持的平台实现时,终端原语会失败;Linux 精确探针覆盖 x64 与 arm64,macOS 则使用 `ps` 快照。 +- **守护化的终端后代仍可能逃出可观察边界**:在 macOS 上,子进程如果在任何前台检查快照之前重新设定父进程,将无法再从 `node-pty` 根进程发现;在 Linux 上,调用 `setsid` 的子进程会同时离开进程树与自有终端会话。本地提供方不会新增持续进程表监视器。 +- **凭据清除依赖名称启发式规则**:只匹配 `*KEY*`/`*PASSWORD*`/`*SECRET*`/`*TOKEN*`;名称不同的 secret(例如 `*PASSPHRASE*`)会继续传递,对误删变量引入白名单属于已记录的后续工作。 - **不会删除已完成的 spill 文件**:有界的完整输出恢复文件(以及每个进程的私有 spill 目录)会在 OS tmpdir 下累积,直到外部机制进行清理;超大的不完整 spill 会被丢弃并立即尝试删除,但清理失败可能留下一个有界文件。 原始进程处理位于 `src/spawn.ts`;`src/index.ts` 负责服务接线。 diff --git a/packages/subprocess/subprocess-local/src/index.ts b/packages/subprocess/subprocess-local/src/index.ts index 46d179a7b2..262a2567b8 100644 --- a/packages/subprocess/subprocess-local/src/index.ts +++ b/packages/subprocess/subprocess-local/src/index.ts @@ -8,10 +8,8 @@ */ import { constants } from 'node:fs' -import { mkdtempSync } from 'node:fs' -import { access, rm, stat } from 'node:fs/promises' -import { tmpdir } from 'node:os' -import { delimiter, extname, isAbsolute, join, resolve } from 'node:path' +import { access, stat } from 'node:fs/promises' +import { delimiter, extname, isAbsolute, resolve } from 'node:path' import { Context } from 'cordis' import * as nodePty from 'node-pty' import type { IPtyForkOptions } from 'node-pty' @@ -36,7 +34,6 @@ import { LocalTerminalHandle } from './terminal.ts' */ export class LocalSubprocessService extends SubprocessService { readonly cwd = process.cwd() - readonly runtimeRoot = mkdtempSync(join(tmpdir(), 'dsh-subprocess-runtime-')) /** Live handles retained only so disposal can terminate and join them. */ private live = new Set() /** Live terminal sessions retained through whole-session quiescence. */ @@ -63,10 +60,7 @@ export class LocalSubprocessService extends SubprocessService { } this.live.clear() this.terminals.clear() - const outcomes = [ - ...await Promise.allSettled(pending), - ...await Promise.allSettled([rm(this.runtimeRoot, { recursive: true, force: true })]), - ] + const outcomes = await Promise.allSettled(pending) const failures = outcomes.flatMap(outcome => outcome.status === 'rejected' ? [outcome.reason as unknown] : []) @@ -126,7 +120,7 @@ export class LocalSubprocessService extends SubprocessService { } // Local PTY allocation is synchronous, but the provider seam permits remote asynchronous allocation. - // eslint-disable-next-line @typescript-eslint/require-await + // oxlint-disable-next-line typescript/require-await -- Preserve promise rejection semantics at the async provider seam. async spawnTerminal(spec: SubprocessTerminalSpawnSpec): Promise { const file = spec.argv[0] if (file === undefined || file.length === 0) { diff --git a/packages/subprocess/subprocess-local/tests/local.spec.ts b/packages/subprocess/subprocess-local/tests/local.spec.ts index ff83141fab..e74927448d 100644 --- a/packages/subprocess/subprocess-local/tests/local.spec.ts +++ b/packages/subprocess/subprocess-local/tests/local.spec.ts @@ -1,6 +1,5 @@ import { PassThrough } from 'node:stream' import { describe, expect, it, vi } from 'vitest' -import { stat } from 'node:fs/promises' import { basename, dirname, relative, resolve } from 'node:path' import { Context } from 'cordis' import LocalSubprocessService from '@deepseek-ai/dsh-subprocess-local' @@ -22,16 +21,6 @@ function spec(command: string, overrides: Partial = {}): Su } describe('LocalSubprocessService', () => { - it('publishes execution-world paths and removes its private runtime directory', async () => { - const ctx = new Context() - const fiber = await ctx.plugin(LocalSubprocessService) - const root = ctx.subprocess.runtimeRoot - expect(ctx.subprocess.cwd).toBe(process.cwd()) - expect((await stat(root)).isDirectory()).toBe(true) - await fiber.dispose() - await expect(stat(root)).rejects.toMatchObject({ code: 'ENOENT' }) - }) - it('resolves absolute and PATH executables and honors lookup cancellation', async () => { const ctx = new Context() const fiber = await ctx.plugin(LocalSubprocessService) @@ -113,11 +102,10 @@ describe('LocalSubprocessService', () => { expect(terminals.size).toBe(0) }) - it('waits for every terminal cleanup and clears single-shot teardown ownership', async () => { + it('waits for every terminal cleanup and aggregates teardown failures', async () => { const ctx = new Context() const fiber = await ctx.plugin(LocalSubprocessService) const service = ctx.subprocess - const runtimeRoot = service.runtimeRoot const firstFailure = new Error('first cleanup failure') const secondFailure = new Error('second cleanup failure') const disposalErrors: unknown[] = [] @@ -155,7 +143,6 @@ describe('LocalSubprocessService', () => { finishCleanup() await disposing expect(terminals.size).toBe(0) - await expect(stat(runtimeRoot)).rejects.toMatchObject({ code: 'ENOENT' }) expect(disposalErrors).toHaveLength(1) expect(disposalErrors[0]).toMatchObject({ errors: [firstFailure, secondFailure], @@ -163,14 +150,13 @@ describe('LocalSubprocessService', () => { }) }) - it('reports one cleanup failure without wrapping it after removing runtime state', async () => { + it('reports one cleanup failure without wrapping it', async () => { const ctx = new Context() const failure = new Error('single cleanup failure') const disposalErrors: unknown[] = [] ctx.logger.error = ((error: unknown) => { disposalErrors.push(error) }) as typeof ctx.logger.error const fiber = await ctx.plugin(LocalSubprocessService) const service = ctx.subprocess - const runtimeRoot = service.runtimeRoot const terminal: SubprocessTerminalHandle = { pid: 1, output: new PassThrough(), @@ -185,7 +171,6 @@ describe('LocalSubprocessService', () => { await fiber.dispose() - await expect(stat(runtimeRoot)).rejects.toMatchObject({ code: 'ENOENT' }) expect(disposalErrors).toEqual([failure]) }) diff --git a/packages/subprocess/subprocess/README.i18n.yaml b/packages/subprocess/subprocess/README.i18n.yaml index 92887cc247..d12516f11f 100644 --- a/packages/subprocess/subprocess/README.i18n.yaml +++ b/packages/subprocess/subprocess/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/subprocess/subprocess/README.md -README.md: 12a88a86c3cd7bcde2858bdbd874c99533100e09 -README.zh.md: f4c265a0f55c4e35e4557516593dd0da236164c5 +README.md: 08fb03d6f145c1026180d92de2421b647cb3ebbe +README.zh.md: 592fc83223092b3406e9070e11de2b20a63232a6 diff --git a/packages/subprocess/subprocess/README.md b/packages/subprocess/subprocess/README.md index 12a88a86c3..08fb03d6f1 100644 --- a/packages/subprocess/subprocess/README.md +++ b/packages/subprocess/subprocess/README.md @@ -2,12 +2,12 @@ English | [中文](README.zh.md) -The subprocess seam (`ctx.subprocess`) is the process half of one execution world. The abstract `SubprocessService` exposes its canonical `cwd`, private `runtimeRoot`, executable lookup, ordinary managed `spawn`, and one terminal-process primitive; its vocabulary covers raw/collected stdio, process and terminal handles, exit facts, tree/session cleanup, and the managed `DSH_*` environment namespace. The local implementation lives in [`dsh-subprocess-local`](../subprocess-local/README.md). +The subprocess seam (`ctx.subprocess`) is the process half of one execution world. The abstract `SubprocessService` exposes its canonical `cwd`, executable lookup, ordinary managed `spawn`, and one terminal-process primitive; its vocabulary covers raw/collected stdio, process and terminal handles, exit facts, tree/session cleanup, and the managed `DSH_*` environment namespace. The local implementation lives in [`dsh-subprocess-local`](../subprocess-local/README.md). ## Contract - `spawn(spec)` returns immediately with a live handle; `done` resolves at process close with exit facts (`SubprocessOutcome` carries no output and no cause classification) and rejects only for spawn-level failures. -- `cwd` and `runtimeRoot` are absolute paths in the provider's execution world. Consumers materialize private helpers below `runtimeRoot`, never in a host-only temp directory. `resolveExecutable(command, env?, signal?)` verifies absolute commands or resolves bare names against that world's scrubbed PATH plus explicit overrides. +- `cwd` and executable paths belong to the provider's execution world. `resolveExecutable(command, env?, signal?)` verifies absolute commands or resolves bare names against that world's scrubbed PATH plus explicit overrides. - The spec is fully explicit — argv, cwd, per-stream stdio dispositions, grace — because deployment-varying defaults belong to the calling seam's config, not to a hidden subprocess-service default (the `dsh-bash` request/spec split is the owning template). `argv` is never shell-interpreted; a consumer that wants a shell passes `['bash', '-c', command]` itself. - Stdio is Node-shaped per stream: `'pipe'` hands the caller the raw stream for its own protocol framing (LSP JSON-RPC, ACP ndjson), `'inherit'` passes the parent descriptor through for diagnostics, and collect mode (`{ maxBytes, spill? }`) buffers a bounded tail with an optional full-stream spill file. Collect readers take whole-stream byte offsets and never consume, so independent readers cannot steal one another's deltas; a read whose offset slid out of the in-memory tail is `lossy` and points at the spill file when one exists. Collected output stays readable after settlement. - Termination is tree-scoped on every platform (POSIX detached groups with direct-child fallback; Windows `taskkill /T`): `terminate()` — the only termination verb — escalates SIGTERM→grace→SIGKILL (idempotent, driven by the spec's abort signal too, a no-op once the tree is gone), and `waitForExit(signal?)` observes whole-tree liveness so a consumer-owned teardown ladder holds each tier on real quiescence — the manager reacts but never classifies why (callers own deadlines, teardown ladders, and cause classification). diff --git a/packages/subprocess/subprocess/README.zh.md b/packages/subprocess/subprocess/README.zh.md index f4c265a0f5..592fc83223 100644 --- a/packages/subprocess/subprocess/README.zh.md +++ b/packages/subprocess/subprocess/README.zh.md @@ -2,12 +2,12 @@ [English](README.md) | 中文 -子进程 seam(`ctx.subprocess`)是一个执行世界的进程部分。抽象的 `SubprocessService` 公开其规范化 `cwd`、私有 `runtimeRoot`、可执行文件查找、普通受管 `spawn` 和一项终端进程原语;其词汇涵盖原始/收集式 stdio、进程与终端句柄、退出事实、进程树/会话清理,以及受管的 `DSH_*` 环境命名空间。本地实现位于 [`dsh-subprocess-local`](../subprocess-local/README.md)。 +子进程 seam(`ctx.subprocess`)是一个执行世界的进程部分。抽象的 `SubprocessService` 公开其规范化 `cwd`、可执行文件查找、普通受管 `spawn` 和一项终端进程原语;其词汇涵盖原始/收集式 stdio、进程与终端句柄、退出事实、进程树/会话清理,以及受管的 `DSH_*` 环境命名空间。本地实现位于 [`dsh-subprocess-local`](../subprocess-local/README.md)。 ## 契约 - `spawn(spec)` 立即返回一个活动句柄;`done` 在进程关闭时以退出事实 resolve(`SubprocessOutcome` 不携带输出,也不携带原因分类),仅在 spawn 层面失败时 reject。 -- `cwd` 和 `runtimeRoot` 是提供方执行世界中的绝对路径。消费方在 `runtimeRoot` 之下物化私有辅助程序,绝不使用仅宿主可见的临时目录。`resolveExecutable(command, env?, signal?)` 验证绝对命令,或根据该执行世界清理后的 PATH 加显式覆盖来解析裸名称。 +- `cwd` 和可执行文件路径属于提供方的执行世界。`resolveExecutable(command, env?, signal?)` 验证绝对命令,或根据该执行世界清理后的 PATH 加显式覆盖来解析裸名称。 - spec 完全显式(argv、cwd、按流划分的 stdio 处置方式(disposition)、宽限期),因为随部署变化的默认值属于调用方 seam 的配置,而不属于某个隐藏的子进程默认值(`dsh-bash` 的 request/spec 拆分是这条规则的所属模板)。`argv` 绝不经过 shell 解释;需要 shell 的消费方自行传入 `['bash', '-c', command]`。 - stdio 按流采用 Node 风格:`'pipe'` 把原始流交给调用方做自己的协议分帧(LSP 的 JSON-RPC、ACP(Agent Client Protocol)的 ndjson),`'inherit'` 直通父进程描述符以承载诊断输出,收集模式(collect)`{ maxBytes, spill? }` 则缓冲一段有界尾部,外加可选的完整流 spill 文件。收集模式的读取器接受全流字节偏移量且从不消费,因此独立的读取器不会抢走彼此的增量;偏移量滑出内存尾部窗口的读取标记为 `lossy`,并在 spill 文件存在时指向它。收集到的输出在结算后仍可读取。 - 终止在每个平台上都以进程树为范围(POSIX 用 detached 进程组并以直接子进程回退;Windows 用 `taskkill /T`):`terminate()`(唯一的终止动词)执行 SIGTERM→宽限期→SIGKILL 升级(幂等,也由 spec 的 abort 信号驱动,进程树消亡后为空操作);`waitForExit(signal?)` 观察整棵进程树的存活状态,使消费方自有的拆卸阶梯能在真正完全停稳后才进入下一层。管理器只响应中止,但绝不判定原因(deadline、拆卸阶梯与原因分类归调用方所有)。 diff --git a/packages/subprocess/subprocess/src/index.ts b/packages/subprocess/subprocess/src/index.ts index e1749203dd..df3186c9b0 100644 --- a/packages/subprocess/subprocess/src/index.ts +++ b/packages/subprocess/subprocess/src/index.ts @@ -78,8 +78,8 @@ declare module 'cordis' { * duplicate-service behavior). * * Implementations must honor these semantics: - * - {@link cwd}, {@link runtimeRoot}, and executable paths belong to one - * execution world shared with the mounted filesystem provider. + * - {@link cwd} and executable paths belong to one execution world shared + * with the mounted filesystem provider. * - {@link spawn} returns immediately with a live handle; `done` resolves at * process close with exit facts and rejects only for spawn-level failures. * - Collect-mode readers are offset-based and non-consuming, so independent @@ -107,9 +107,6 @@ export abstract class SubprocessService extends Service { /** Canonical default cwd in this provider's execution world. */ abstract readonly cwd: string - /** Private directory for runtime artifacts in this provider's execution world. */ - abstract readonly runtimeRoot: string - /** * Resolve one configured executable in this provider's execution world. * Absolute paths are verified; bare names use the provider's scrubbed PATH diff --git a/packages/subprocess/subprocess/tests/service.spec.ts b/packages/subprocess/subprocess/tests/service.spec.ts index 9f321caf54..2084310da9 100644 --- a/packages/subprocess/subprocess/tests/service.spec.ts +++ b/packages/subprocess/subprocess/tests/service.spec.ts @@ -17,7 +17,6 @@ import type { */ class StubSubprocessService extends SubprocessService { readonly cwd = '/stub' - readonly runtimeRoot = '/stub/.runtime' async resolveExecutable(command: string): Promise { return `/bin/${command}` diff --git a/packages/ui/app-boot/tsdown.config.ts b/packages/ui/app-boot/tsdown.config.ts index 88492d7c26..33db6be1b8 100644 --- a/packages/ui/app-boot/tsdown.config.ts +++ b/packages/ui/app-boot/tsdown.config.ts @@ -1,9 +1,6 @@ import { defineConfig } from 'tsdown' -/** - * Embed Include while keeping Loader external so the built include tree and - * app host bind to one Loader peer. - */ +/** Bundle the host plugin and its dependency-free eval runner. */ export default defineConfig({ entry: ['lib/types/index.js', 'lib/types/invariant.js'], outDir: 'lib', @@ -13,7 +10,4 @@ export default defineConfig({ fixedExtension: false, dts: false, clean: false, - deps: { - alwaysBundle: ['@cordisjs/plugin-include'], - }, }) diff --git a/packages/util/atomic-write/README.i18n.yaml b/packages/util/atomic-write/README.i18n.yaml index 33e1f6014b..efcd06f845 100644 --- a/packages/util/atomic-write/README.i18n.yaml +++ b/packages/util/atomic-write/README.i18n.yaml @@ -1,6 +1,6 @@ # Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: -# pnpm run verify-translation-pairing --write packages/util/atomic-write/README.md -README.md: 2ff4abb6ac10d8b592ccd2056b4f1f92cc8518b0 -README.zh.md: 4284d06422564268bb9e31d1a1ed06ab5271e562 +# pnpm run verify-translation-pairing --write packages/code-runtime/code-runtime-subprocess/README.md +README.md: 897112a32fa3e776df756d4caaba97b617c6af9e +README.zh.md: d9ab3713640fe955d2b36a8f0674f77686e4791b diff --git a/packages/util/atomic-write/src/runner-source.generated.ts b/packages/util/atomic-write/src/runner-source.generated.ts index 8592fa2399..95da563dbb 100644 --- a/packages/util/atomic-write/src/runner-source.generated.ts +++ b/packages/util/atomic-write/src/runner-source.generated.ts @@ -3,4 +3,4 @@ * Do not edit by hand; run `pnpm run gen-code-runtime-runner`. */ -export const CODE_RUNNER_SOURCE = "import{Buffer as e}from\"node:buffer\";import{fork as t}from\"node:child_process\";import{createInterface as n}from\"node:readline\";import{inspect as r}from\"node:util\";import{fileURLToPath as i}from\"node:url\";import{Worker as a,isMainThread as o,parentPort as s,workerData as c}from\"node:worker_threads\";const l=Reflect.apply,u=Array.isArray,d=Buffer,f=Reflect.get(Buffer,`byteLength`),ee=Object.create,p=Object.defineProperty,te=Object.keys,ne=String,re=Reflect.get(String.prototype,`charCodeAt`),ie=Reflect.get(String.prototype,`codePointAt`),m=Reflect.get(String.prototype,`slice`);function h(e){let t=ee(null);return t.value=e,t}function ae(e,t,n){let r=h(n);r.enumerable=!0,r.configurable=!0,r.writable=!0,p(e,t,r)}function g(e){return l(f,d,[e,`utf8`])}function _(e,t){ae(e,e.length,t)}function oe(e){if(e.length===0)return;let t=e.length-1,n=e[t];return p(e,`length`,h(t)),n}function se(e,t){return l(m,e,[t,t+(l(ie,e,[t])>65535?2:1)])}function ce(e){if(e.length===2)return 4;if(e===`\"`||e===`\\\\`)return 2;let t=l(re,e,[0]);return t>=55296&&t<=57343?6:t<32?t===8||t===9||t===10||t===12||t===13?2:6:g(e)}function v(e,t){if(t<2)return;let n=2;for(let r=0;rt)return;r+=i.length}return n}function y(e,t){let n=0,r=e=>(n+=e,n<=t),i=[{kind:`value`,value:e}];for(let e=oe(i);e!==void 0;e=oe(i)){if(e.kind===`value`){let a=e.value;if(a===null){if(!r(4))return}else if(typeof a==`string`){let e=v(a,t-n);if(e===void 0)return;n+=e}else if(typeof a==`number`){if(!r(g(ne(a))))return}else if(typeof a==`boolean`){if(!r(a?4:5))return}else if(u(a)){if(!r(2))return;a.length>0&&_(i,{kind:`array`,value:a,index:0})}else{if(!r(2))return;let e=te(a);e.length>0&&_(i,{kind:`object`,value:a,keys:e,index:0})}continue}if(e.index>0&&!r(1))return;if(e.kind===`array`){let t=e.value[e.index];if(t===void 0)return;e.index+1t)break;n+=o,r+=a.length,i+=a.length}return r===e.length?e:l(m,e,[0,r])}const ue=Reflect.get(Function.prototype,`toString`),b=Reflect.get(Reflect,`apply`),x=Error,S=Set,C=Array.isArray,de=Array.prototype,w=Number.isFinite,fe=Number.isSafeInteger,pe=Object.create,T=Object.defineProperty,me=Object.getOwnPropertyDescriptor,E=Object.getPrototypeOf,D=Object.hasOwn,O=Object.is,he=Object.keys,k=Object.prototype,ge=Reflect.get(k,`propertyIsEnumerable`),A=Reflect.ownKeys,_e=Reflect.get(Set.prototype,`add`),ve=Reflect.get(Set.prototype,`delete`),ye=Reflect.get(Set.prototype,`has`);function j(e){let t=pe(null);return t.value=e,t}function M(e,t,n){let r=j(n);r.enumerable=!0,r.configurable=!0,r.writable=!0,T(e,t,r)}function N(e,t){M(e,e.length,t)}function P(e){if(e.length===0)return;let t=e.length-1,n=e[t];return T(e,`length`,j(t)),n}function F(e,t){return b(ye,e,[t])}function I(e,t){b(_e,e,[t])}function be(e,t){b(ve,e,[t])}function L(e,t){let n=me(e,`constructor`)?.value;if(typeof n!=`function`)return!1;try{return n.name===t&&n.prototype===e&&b(ue,n,[])===`function ${t}() { [native code] }`}catch{return!1}}function R(e){return E(e)===null&&L(e,`Object`)}function z(e){let t=E(e);if(t===de)return!0;if(!C(t)||!L(t,`Array`))return!1;let n=E(t);return typeof n==`object`&&!!n&&R(n)}function B(e){let t=E(e);return t===null||t===k||typeof t==`object`&&R(t)}function V(e){let t=A(e);for(let n=0;n{e.kind===`root`?n=t:e.kind===`array`?M(e.target,e.index,t):M(e.target,e.key,t)},i=[{kind:`visit`,value:e,destination:{kind:`root`}}];for(let e=P(i);e!==void 0;e=P(i)){if(e.kind===`leave`){be(t,e.source);continue}if(e.kind===`array-item`){if(!D(e.source,e.index))return;N(i,{kind:`visit`,value:e.source[e.index],destination:{kind:`array`,target:e.target,index:e.index}});continue}if(e.kind===`object-property`){N(i,{kind:`visit`,value:e.source[e.key],destination:{kind:`object`,target:e.target,key:e.key}});continue}let n=e.value;if(n===null){r(e.destination,null);continue}if(typeof n==`boolean`||typeof n==`string`){r(e.destination,n);continue}if(typeof n==`number`){if(!w(n)||O(n,-0))return;r(e.destination,n);continue}if(typeof n!=`object`||F(t,n))return;if(C(n)){if(!z(n))return;let a=n.length;if(A(n).length!==a+1)return;let o=[];r(e.destination,o),I(t,n),N(i,{kind:`leave`,source:n});for(let e=a-1;e>=0;e--)N(i,{kind:`array-item`,source:n,index:e,target:o});continue}if(!B(n))return;let a=V(n);if(a===void 0)return;let o={};r(e.destination,o),I(t,n),N(i,{kind:`leave`,source:n});for(let e=a.length-1;e>=0;e--){let t=a[e];if(t===void 0)return;N(i,{kind:`object-property`,source:n,key:t,target:o})}}return n}function U(e){let t=[],n=[e];for(let e=P(n);e!==void 0;e=P(n)){if(e===null||typeof e==`boolean`||typeof e==`number`||typeof e==`string`){N(t,e);continue}if(C(e)){N(t,{kind:`array`,length:e.length});for(let t=e.length-1;t>=0;t--){let r=e[t];if(r===void 0)throw new x(`cannot encode a sparse JSON array`);N(n,r)}continue}let r=he(e);N(t,{kind:`object`,keys:r});for(let t=r.length-1;t>=0;t--){let i=r[t];if(i===void 0)throw new x(`cannot encode a missing JSON object key`);let a=e[i];if(a===void 0)throw new x(`cannot encode an undefined JSON object property`);N(n,a)}}return t}function xe(e){if(!z(e)||A(e).length!==e.length+1)return!1;for(let t=0;t=0?{kind:`array`,length:e}:void 0}if(n.kind===`object`){if(t.length!==2||!W(t,`kind`)||!W(t,`keys`))return;let e=n.keys;if(!C(e)||!xe(e))return;let r=new S,i=[],a=e;for(let e=0;e{let t=n[n.length-1];if(!t)return i?!1:(r=e,i=!0,!0);if(t.index>=(t.kind===`array`?t.length:t.keys.length))return!1;if(t.kind===`array`)N(t.target,e);else{let n=t.keys[t.index];if(n===void 0)return!1;M(t.target,n,e)}return t.index+=1,!0};for(let e=0;ea)return;let e=[];i=e,n.length>0&&(o={kind:`array`,target:e,length:n.length,index:0})}else{if(n.keys.length>a)return;let e={};i=e,n.keys.length>0&&(o={kind:`object`,target:e,keys:n.keys,index:0})}}if(!a(i))return;for(o&&N(n,o);n.length>0;){let e=n[n.length-1];if(e===void 0||e.index<(e.kind===`array`?e.length:e.keys.length))break;P(n)}}return n.length===0?r:void 0}catch{return}}const K=Error,Ce=Object.create,we=Object.defineProperty;function Te(e,t,n){let r=Ce(null);r.enumerable=!0,r.value=n,we(e,t,r)}var Ee=class{bytes=2;entries=0;truncated=!1;sink;onLimit;maxBytes;constructor(e,t,n=()=>{}){this.maxBytes=e,this.sink=t,this.onLimit=n}push(e){if(this.truncated)return;let t=+(this.entries>0),n=this.maxBytes-this.bytes-t,r=v(e,n);if(r===void 0){this.truncated=!0;let r=le(e,n);if(r.length>0){let e=v(r,n);if(e===void 0)throw new K(`worker output ledger produced an oversized log prefix`);this.bytes+=e+t,this.entries+=1,this.sink(r)}this.onLimit();return}this.bytes+=r+t,this.entries+=1,this.sink(e)}remainingOutputBytes(){return this.maxBytes-this.bytes}};const De=[`log`,`info`,`warn`,`error`,`debug`];function Oe(e){let t=e=>e.map(e=>typeof e==`string`?e:r(e,Ae)).join(` `),n=Object.create(null);for(let r of De)n[r]=(...n)=>{e.push(t(n))};return n}function ke(e,t){let n=t.write;return t.write=(t,...n)=>{e.push(typeof t==`string`?t:String(t));let r=[n[0],n[1]].find(e=>typeof e==`function`);return r&&queueMicrotask(()=>{r(null)}),!0},()=>{t.write=n}}const Ae={depth:4,maxArrayLength:100,maxStringLength:1e4};function je(e,t,n=t){if(e===void 0)return{};let r;try{r=H(e)}catch{r=void 0}return r===void 0?Ne(`invalid-output`,`program completion must be lossless JSON`,t,n):y(r,t)===void 0?Me(n):{value:U(r)}}function Me(e){return{error:{kind:`output-limit`,message:`outer output exceeded ${e} bytes`}}}function Ne(e,t,n,r){return v(t,n)===void 0?Me(r):{error:{kind:e,message:t}}}function Pe(e,t,n=t){let r;try{let t=e instanceof K?e.stack??e.message:e;r=typeof t==`string`?t:String(t)}catch{r=`program threw an unrenderable value`}return Ne(`exception`,r,t,n)}function Fe(e){return class extends K{constructor(t,n){super(n),Te(this,`name`,e.name),Te(this,e.memberNameProperty,t)}}}function q(e,t,n){return e?new e(t,n):new K(n)}function Ie(e){let t=new Map;for(let n of e.namespaces)n.errorClass&&t.set(n.global,Fe(n.errorClass));return t}function Le(e,t){e.on(`message`,e=>{let n=t.get(e.id);if(n)if(t.delete(e.id),e.ok){let t=G(e.value);t===void 0?n.reject(new K(`binding resolution must be lossless JSON`)):n.resolve(t)}else n.reject(new K(e.message))})}function Re(e,t,n,r,i=Ie(e),a){return e.namespaces.map(({global:e,names:o})=>{let s=i.get(e),c=Object.create(null);for(let i of o)Object.defineProperty(c,i,{enumerable:!0,value:o=>{let c;try{c=H(o)}catch{c=void 0}if(c===void 0)return Promise.reject(q(s,i,`binding arguments must be lossless JSON`));let l={type:`call`,id:r.value,global:e,name:i,args:U(c)};return a!==void 0&&y(l,a)===void 0?Promise.reject(q(s,i,`binding arguments exceed maxFrameBytes`)):new Promise((e,a)=>{let o=r.value++;n.set(o,{resolve:e,reject:e=>{a(q(s,i,e.message))}});try{t.postMessage(l)}catch(e){n.delete(o),a(q(s,i,`binding arguments must be structured-cloneable: ${e instanceof K?e.message:String(e)}`))}})}});return c})}async function ze(e,t,n,r){let i=new Ee(t.maxOutputBytes,t=>{e.postMessage({type:`log`,text:t})},()=>{e.postMessage({type:`output-limit`})});ke(i,n.stdout),ke(i,n.stderr);let a=new Map;Le(e,a);let o={value:1},s=Ie(t),c=Re(t,e,a,o,s,r),l=[],u=[];for(let e of t.namespaces){if(!e.errorClass)continue;l.push(e.errorClass.name);let t=s.get(e.global);if(!t)throw new K(`missing binding error class for ${e.global}`);u.push(t)}let d=Oe(i),f;try{let e=(async()=>{}).constructor;f={type:`done`,...je(await new e(...t.namespaces.map(e=>e.global),...l,`console`,`'use strict';\\n${t.code}`)(...c,...u,d),i.remainingOutputBytes(),t.maxOutputBytes)}}catch(e){f={type:`done`,...Pe(e,i.remainingOutputBytes(),t.maxOutputBytes)}}e.postMessage(r!==void 0&&y(f,r)===void 0?{type:`output-limit`}:f)}function J(e){return e.readableEnded||e.destroyed?Promise.resolve():new Promise(t=>{let n=()=>{e.off(`end`,n),e.off(`close`,n),e.off(`error`,n),t()};e.once(`end`,n),e.once(`close`,n),e.once(`error`,n),(e.readableEnded||e.destroyed)&&n()})}const Be=new Set([`exception`,`timeout`,`abort`,`worker-exit`,`invalid-output`,`output-limit`]);let Y=0;function X(e){return typeof e==`object`&&e?e:void 0}function Z(t,n){try{let r=JSON.stringify(t);return typeof r==`string`&&e.byteLength(r)<=n?r:void 0}catch{return}}function Ve(e){process.stdout.write(e),process.stdout.write(`\n`)}function He(e){let t=Z(e,Y);return t===void 0?!1:(Ve(t),!0)}function Ue(e){let t=X(e);return t!==void 0&&typeof t.kind==`string`&&Be.has(t.kind)&&typeof t.message==`string`}function We(e){return Ue(e)&&(e.kind===`exception`||e.kind===`invalid-output`||e.kind===`output-limit`)}function Ge(e,t){return e.error===void 0?{type:`done`,...e.value===void 0?{}:{value:Q(e.value)}}:t(e.error)?{type:`done`,error:e.error}:void 0}function Ke(e){let t=X(e);if(!(t===void 0||t.type!==`boot`||typeof t.code!=`string`||!Array.isArray(t.namespaces)||!Number.isSafeInteger(t.maxOutputBytes)||t.maxOutputBytes<4||!Number.isSafeInteger(t.maxFrameBytes)||t.maxFrameBytes{e.once(`exit`,()=>{t()})})}function $(){return{type:`done`,error:{kind:`worker-exit`,message:`code runtime bridge frame exceeded maxFrameBytes`}}}function Ye(){let e=n({input:process.stdin,crlfDelay:1/0}),r,a=0,o=2,s=0,c=!1,l=t=>{if(c)return;let n=Z(t,Y)??Z($(),Y);c=!0,n!==void 0&&Ve(n);let i=r;r=void 0,(i===void 0?Promise.resolve():new Promise(e=>{setImmediate(e)}).then(async()=>{let e=J(i.stdout),t=J(i.stderr),n=Je(i);i.kill(`SIGKILL`),await Promise.all([n,e,t])})).catch(e=>{process.stderr.write(`dsh-code-runtime-subprocess controller cleanup error: ${String(e)}\\n`)}).then(()=>{e.close(),process.stdin.destroy()})},u=e=>{if(c)return;let t=+(s>0),n=v(e,a-o-t);if(n===void 0){l({type:`output-limit`});return}o+=n+t,s+=1,He({type:`log`,text:e})||l($())},d=e=>{a=e.maxOutputBytes,Y=e.maxFrameBytes,r=t(i(import.meta.url),[],{env:{DSH_CODE_RUNTIME_CONTROLLER:`1`},detached:!1,execArgv:[],stdio:[`ignore`,`pipe`,`pipe`,`ipc`]});let n=r;n.stdout.on(`data`,e=>{u(e.toString(`utf8`))}),n.stderr.on(`data`,e=>{u(e.toString(`utf8`))}),n.on(`message`,e=>{let t=X(e);if(t!==void 0){if(t.type===`log`&&typeof t.text==`string`){u(t.text);return}if(!c){if(t.type===`call`&&typeof t.id==`number`&&typeof t.global==`string`&&typeof t.name==`string`)He({type:`call`,id:t.id,global:t.global,name:t.name,args:Q(t.args)})||l($());else if(t.type===`output-limit`)l({type:`output-limit`});else if(t.type===`done`){let e=Ge(t,Ue);e!==void 0&&l(e)}}}}),n.on(`error`,e=>{l({type:`done`,error:{kind:`worker-exit`,message:`remote controller error: ${e.message}`}})}),n.on(`exit`,e=>{c||l({type:`done`,error:{kind:`worker-exit`,message:`remote controller exited with code ${e} before completing`}})}),n.send(e,e=>{e!==null&&l({type:`done`,error:{kind:`worker-exit`,message:`remote controller boot failed: ${e.message}`}})})};e.on(`line`,e=>{let t;try{t=JSON.parse(e)}catch(e){process.stderr.write(`dsh-code-runtime-subprocess frame error: ${String(e)}\\n`),l({type:`done`,error:{kind:`worker-exit`,message:`remote runner received a malformed frame`}});return}if(r===void 0){let e=Ke(t);if(e===void 0){l({type:`done`,error:{kind:`worker-exit`,message:`remote runner received an invalid boot frame`}});return}d(e);return}let n=qe(t);n!==void 0&&r.send(n,e=>{e!==null&&l({type:`done`,error:{kind:`worker-exit`,message:`remote controller reply failed: ${e.message}`}})})}),e.on(`close`,()=>{r!==void 0&&!c&&l({type:`done`,error:{kind:`abort`,message:`remote runner input closed`}})})}function Xe(){let e,t=!1,n,i=0,o=e=>process.send===void 0||i>0&&Z(e,i)===void 0?!1:(process.send(e),!0),s=r=>{if(t)return;t=!0,clearInterval(n);let a=i>0&&Z(r,i)===void 0?$():r,s=e;e=void 0,(s===void 0?Promise.resolve():new Promise(e=>{setImmediate(e)}).then(async()=>{let e=J(s.stdout),t=J(s.stderr);await Promise.all([s.terminate(),e,t])})).catch(e=>{o({type:`log`,text:`dsh-code-runtime-subprocess worker cleanup error: ${String(e)}\\n`})}).then(()=>{if(process.send===void 0){process.exitCode=1;return}process.send(a,()=>{process.connected&&process.disconnect()})})};process.on(`message`,c=>{if(e===void 0){let l=Ke(c);if(l===void 0){s({type:`done`,error:{kind:`worker-exit`,message:`remote controller received an invalid boot frame`}});return}i=l.maxFrameBytes,e=new a(new URL(import.meta.url),{workerData:l,env:{},execArgv:[],stdout:!0,stderr:!0,resourceLimits:{maxOldGenerationSizeMb:l.maxOldGenerationSizeMb}});let u=e;u.stdout.on(`data`,e=>{o({type:`log`,text:e.toString(`utf8`)})||s($())}),u.stderr.on(`data`,e=>{o({type:`log`,text:e.toString(`utf8`)})||s($())}),u.on(`message`,e=>{let t=X(e);if(t!==void 0){if(t.type===`call`&&typeof t.id==`number`&&typeof t.global==`string`&&typeof t.name==`string`)o({type:`call`,id:t.id,global:t.global,name:t.name,args:Q(t.args)})||s($());else if(t.type===`log`&&typeof t.text==`string`)o({type:`log`,text:t.text})||s($());else if(t.type===`output-limit`)s({type:`output-limit`});else if(t.type===`done`){let e=Ge(t,We);e!==void 0&&s(e)}}}),u.on(`error`,e=>{s({type:`done`,error:{kind:`worker-exit`,message:`worker error: ${e.stack||e.message||r(e)}`}})}),u.on(`exit`,e=>{t||s({type:`done`,error:{kind:`worker-exit`,message:`worker exited with code ${e} before completing`}})}),n=setInterval(()=>{e!==void 0&&e.performance.eventLoopUtilization().active>l.computeMs&&s({type:`done`,error:{kind:`timeout`,message:`compute budget exhausted (${l.computeMs}ms busy)`}})},25);return}let l=qe(c);l!==void 0&&e.postMessage(l)}),process.on(`disconnect`,()=>{e!==void 0&&!t&&e.terminate()})}if(o)process.env.DSH_CODE_RUNTIME_CONTROLLER===`1`?Xe():Ye();else{if(s===null)throw Error(`remote worker requires parentPort`);let e=c;ze(s,e,{stdout:process.stdout,stderr:process.stderr},e.maxFrameBytes)}export{};" +export const CODE_RUNNER_SOURCE = "import{Buffer as e}from\"node:buffer\";import{spawn as t}from\"node:child_process\";import{createInterface as n}from\"node:readline\";import{inspect as r}from\"node:util\";import{Worker as i,isMainThread as a,parentPort as o,workerData as s}from\"node:worker_threads\";const c=Reflect.apply,l=Array.isArray,u=Buffer,d=Reflect.get(Buffer,`byteLength`),f=Object.create,p=Object.defineProperty,ee=Object.keys,te=String,ne=Reflect.get(String.prototype,`charCodeAt`),re=Reflect.get(String.prototype,`codePointAt`),ie=Reflect.get(String.prototype,`slice`);function m(e){let t=f(null);return t.value=e,t}function ae(e,t,n){let r=m(n);r.enumerable=!0,r.configurable=!0,r.writable=!0,p(e,t,r)}function h(e){return c(d,u,[e,`utf8`])}function g(e,t){ae(e,e.length,t)}function _(e){if(e.length===0)return;let t=e.length-1,n=e[t];return p(e,`length`,m(t)),n}function v(e,t){return c(ie,e,[t,t+(c(re,e,[t])>65535?2:1)])}function y(e){if(e.length===2)return 4;if(e===`\"`||e===`\\\\`)return 2;let t=c(ne,e,[0]);return t>=55296&&t<=57343?6:t<32?t===8||t===9||t===10||t===12||t===13?2:6:h(e)}function b(e,t){if(t<2)return;let n=2;for(let r=0;rt)return;r+=i.length}return n}function x(e,t){let n=0,r=e=>(n+=e,n<=t),i=[{kind:`value`,value:e}];for(let e=_(i);e!==void 0;e=_(i)){if(e.kind===`value`){let a=e.value;if(a===null){if(!r(4))return}else if(typeof a==`string`){let e=b(a,t-n);if(e===void 0)return;n+=e}else if(typeof a==`number`){if(!r(h(te(a))))return}else if(typeof a==`boolean`){if(!r(a?4:5))return}else if(l(a)){if(!r(2))return;a.length>0&&g(i,{kind:`array`,value:a,index:0})}else{if(!r(2))return;let e=ee(a);e.length>0&&g(i,{kind:`object`,value:a,keys:e,index:0})}continue}if(e.index>0&&!r(1))return;if(e.kind===`array`){let t=e.value[e.index];if(t===void 0)return;e.index+1t)break;n+=o,r+=a.length,i+=a.length}return r===e.length?e:c(ie,e,[0,r])}const se=Reflect.get(Function.prototype,`toString`),S=Reflect.get(Reflect,`apply`),C=Error,ce=Set,w=Array.isArray,le=Array.prototype,T=Number.isFinite,ue=Number.isSafeInteger,de=Object.create,E=Object.defineProperty,fe=Object.getOwnPropertyDescriptor,D=Object.getPrototypeOf,O=Object.hasOwn,k=Object.is,pe=Object.keys,A=Object.prototype,me=Reflect.get(A,`propertyIsEnumerable`),j=Reflect.ownKeys,he=Reflect.get(Set.prototype,`add`),ge=Reflect.get(Set.prototype,`delete`),_e=Reflect.get(Set.prototype,`has`);function M(e){let t=de(null);return t.value=e,t}function N(e,t,n){let r=M(n);r.enumerable=!0,r.configurable=!0,r.writable=!0,E(e,t,r)}function P(e,t){N(e,e.length,t)}function F(e){if(e.length===0)return;let t=e.length-1,n=e[t];return E(e,`length`,M(t)),n}function I(e,t){return S(_e,e,[t])}function L(e,t){S(he,e,[t])}function ve(e,t){S(ge,e,[t])}function R(e,t){let n=fe(e,`constructor`)?.value;if(typeof n!=`function`)return!1;try{return n.name===t&&n.prototype===e&&S(se,n,[])===`function ${t}() { [native code] }`}catch{return!1}}function z(e){return D(e)===null&&R(e,`Object`)}function B(e){let t=D(e);if(t===le)return!0;if(!w(t)||!R(t,`Array`))return!1;let n=D(t);return typeof n==`object`&&!!n&&z(n)}function V(e){let t=D(e);return t===null||t===A||typeof t==`object`&&z(t)}function ye(e){let t=j(e);for(let n=0;n{e.kind===`root`?n=t:e.kind===`array`?N(e.target,e.index,t):N(e.target,e.key,t)},i=[{kind:`visit`,value:e,destination:{kind:`root`}}];for(let e=F(i);e!==void 0;e=F(i)){if(e.kind===`leave`){ve(t,e.source);continue}if(e.kind===`array-item`){if(!O(e.source,e.index))return;P(i,{kind:`visit`,value:e.source[e.index],destination:{kind:`array`,target:e.target,index:e.index}});continue}if(e.kind===`object-property`){P(i,{kind:`visit`,value:e.source[e.key],destination:{kind:`object`,target:e.target,key:e.key}});continue}let n=e.value;if(n===null){r(e.destination,null);continue}if(typeof n==`boolean`||typeof n==`string`){r(e.destination,n);continue}if(typeof n==`number`){if(!T(n)||k(n,-0))return;r(e.destination,n);continue}if(typeof n!=`object`||I(t,n))return;if(w(n)){if(!B(n))return;let a=n.length;if(j(n).length!==a+1)return;let o=[];r(e.destination,o),L(t,n),P(i,{kind:`leave`,source:n});for(let e=a-1;e>=0;e--)P(i,{kind:`array-item`,source:n,index:e,target:o});continue}if(!V(n))return;let a=ye(n);if(a===void 0)return;let o={};r(e.destination,o),L(t,n),P(i,{kind:`leave`,source:n});for(let e=a.length-1;e>=0;e--){let t=a[e];if(t===void 0)return;P(i,{kind:`object-property`,source:n,key:t,target:o})}}return n}function U(e){let t=[],n=[e];for(let e=F(n);e!==void 0;e=F(n)){if(e===null||typeof e==`boolean`||typeof e==`number`||typeof e==`string`){P(t,e);continue}if(w(e)){P(t,{kind:`array`,length:e.length});for(let t=e.length-1;t>=0;t--){let r=e[t];if(r===void 0)throw new C(`cannot encode a sparse JSON array`);P(n,r)}continue}let r=pe(e);P(t,{kind:`object`,keys:r});for(let t=r.length-1;t>=0;t--){let i=r[t];if(i===void 0)throw new C(`cannot encode a missing JSON object key`);let a=e[i];if(a===void 0)throw new C(`cannot encode an undefined JSON object property`);P(n,a)}}return t}function be(e){if(!B(e)||j(e).length!==e.length+1)return!1;for(let t=0;t=0?{kind:`array`,length:e}:void 0}if(n.kind===`object`){if(t.length!==2||!W(t,`kind`)||!W(t,`keys`))return;let e=n.keys;if(!w(e)||!be(e))return;let r=new ce,i=[],a=e;for(let e=0;e{let t=n[n.length-1];if(!t)return i?!1:(r=e,i=!0,!0);if(t.index>=(t.kind===`array`?t.length:t.keys.length))return!1;if(t.kind===`array`)P(t.target,e);else{let n=t.keys[t.index];if(n===void 0)return!1;N(t.target,n,e)}return t.index+=1,!0};for(let e=0;ea)return;let e=[];i=e,n.length>0&&(o={kind:`array`,target:e,length:n.length,index:0})}else{if(n.keys.length>a)return;let e={};i=e,n.keys.length>0&&(o={kind:`object`,target:e,keys:n.keys,index:0})}}if(!a(i))return;for(o&&P(n,o);n.length>0;){let e=n[n.length-1];if(e===void 0||e.index<(e.kind===`array`?e.length:e.keys.length))break;F(n)}}return n.length===0?r:void 0}catch{return}}const G=Error,Ce=Object.create,we=Object.defineProperty;function Te(e,t,n){let r=Ce(null);r.enumerable=!0,r.value=n,we(e,t,r)}var Ee=class{bytes=2;entries=0;truncated=!1;sink;onLimit;maxBytes;constructor(e,t,n=()=>{}){this.maxBytes=e,this.sink=t,this.onLimit=n}push(e){if(this.truncated)return;let t=+(this.entries>0),n=this.maxBytes-this.bytes-t,r=b(e,n);if(r===void 0){this.truncated=!0;let r=oe(e,n);if(r.length>0){let e=b(r,n);if(e===void 0)throw new G(`worker output ledger produced an oversized log prefix`);this.bytes+=e+t,this.entries+=1,this.sink(r)}this.onLimit();return}this.bytes+=r+t,this.entries+=1,this.sink(e)}remainingOutputBytes(){return this.maxBytes-this.bytes}};const De=[`log`,`info`,`warn`,`error`,`debug`];function Oe(e){let t=e=>e.map(e=>typeof e==`string`?e:r(e,Ae)).join(` `),n=Object.create(null);for(let r of De)n[r]=(...n)=>{e.push(t(n))};return n}function ke(e,t){let n=t.write;return t.write=(t,...n)=>{e.push(typeof t==`string`?t:String(t));let r=[n[0],n[1]].find(e=>typeof e==`function`);return r&&queueMicrotask(()=>{r(null)}),!0},()=>{t.write=n}}const Ae={depth:4,maxArrayLength:100,maxStringLength:1e4};function je(e,t,n=t){if(e===void 0)return{};let r;try{r=H(e)}catch{r=void 0}return r===void 0?Ne(`invalid-output`,`program completion must be lossless JSON`,t,n):x(r,t)===void 0?Me(n):{value:U(r)}}function Me(e){return{error:{kind:`output-limit`,message:`outer output exceeded ${e} bytes`}}}function Ne(e,t,n,r){return b(t,n)===void 0?Me(r):{error:{kind:e,message:t}}}function Pe(e,t,n=t){let r;try{let t=e instanceof G?e.stack??e.message:e;r=typeof t==`string`?t:String(t)}catch{r=`program threw an unrenderable value`}return Ne(`exception`,r,t,n)}function Fe(e){return class extends G{constructor(t,n){super(n),Te(this,`name`,e.name),Te(this,e.memberNameProperty,t)}}}function K(e,t,n){return e?new e(t,n):new G(n)}function Ie(e){let t=new Map;for(let n of e.namespaces)n.errorClass&&t.set(n.global,Fe(n.errorClass));return t}function Le(e,t){e.on(`message`,e=>{let n=t.get(e.id);if(n)if(t.delete(e.id),e.ok){let t=Se(e.value);t===void 0?n.reject(new G(`binding resolution must be lossless JSON`)):n.resolve(t)}else n.reject(new G(e.message))})}function Re(e,t,n,r,i=Ie(e),a){return e.namespaces.map(({global:e,names:o})=>{let s=i.get(e),c=Object.create(null);for(let i of o)Object.defineProperty(c,i,{enumerable:!0,value:o=>{let c;try{c=H(o)}catch{c=void 0}if(c===void 0)return Promise.reject(K(s,i,`binding arguments must be lossless JSON`));let l={type:`call`,id:r.value,global:e,name:i,args:U(c)};return a!==void 0&&x(l,a)===void 0?Promise.reject(K(s,i,`binding arguments exceed maxFrameBytes`)):new Promise((e,a)=>{let o=r.value++;n.set(o,{resolve:e,reject:e=>{a(K(s,i,e.message))}});try{t.postMessage(l)}catch(e){n.delete(o),a(K(s,i,`binding arguments must be structured-cloneable: ${e instanceof G?e.message:String(e)}`))}})}});return c})}async function ze(e,t,n,r){let i=new Ee(t.maxOutputBytes,t=>{e.postMessage({type:`log`,text:t})},()=>{e.postMessage({type:`output-limit`})});ke(i,n.stdout),ke(i,n.stderr);let a=new Map;Le(e,a);let o={value:1},s=Ie(t),c=Re(t,e,a,o,s,r),l=[],u=[];for(let e of t.namespaces){if(!e.errorClass)continue;l.push(e.errorClass.name);let t=s.get(e.global);if(!t)throw new G(`missing binding error class for ${e.global}`);u.push(t)}let d=Oe(i),f;try{let e=(async()=>{}).constructor;f={type:`done`,...je(await new e(...t.namespaces.map(e=>e.global),...l,`console`,`'use strict';\\n${t.code}`)(...c,...u,d),i.remainingOutputBytes(),t.maxOutputBytes)}}catch(e){f={type:`done`,...Pe(e,i.remainingOutputBytes(),t.maxOutputBytes)}}e.postMessage(r!==void 0&&x(f,r)===void 0?{type:`output-limit`}:f)}function q(e){return e.readableEnded||e.destroyed?Promise.resolve():new Promise(t=>{let n=()=>{e.off(`end`,n),e.off(`close`,n),e.off(`error`,n),t()};e.once(`end`,n),e.once(`close`,n),e.once(`error`,n),(e.readableEnded||e.destroyed)&&n()})}const Be=new Set([`exception`,`timeout`,`abort`,`worker-exit`,`invalid-output`,`output-limit`]);let J=0;function Ve(){let e=process.execArgv.indexOf(`--eval`);if(e<0)throw Error(`code runtime runner requires its eval source`);let t=process.execArgv[e+1];if(t===void 0)throw Error(`code runtime runner requires its eval source`);return t}function Y(e){return typeof e==`object`&&e?e:void 0}function X(t,n){try{let r=JSON.stringify(t);return typeof r==`string`&&e.byteLength(r)<=n?r:void 0}catch{return}}function He(e){process.stdout.write(e),process.stdout.write(`\n`)}function Ue(e){let t=X(e,J);return t===void 0?!1:(He(t),!0)}function We(e){let t=Y(e);return t!==void 0&&typeof t.kind==`string`&&Be.has(t.kind)&&typeof t.message==`string`}function Ge(e){return We(e)&&(e.kind===`exception`||e.kind===`invalid-output`||e.kind===`output-limit`)}function Ke(e,t){return e.error===void 0?{type:`done`,...e.value===void 0?{}:{value:Q(e.value)}}:t(e.error)?{type:`done`,error:e.error}:void 0}function Z(e){let t=Y(e);if(!(t===void 0||t.type!==`boot`||typeof t.code!=`string`||!Array.isArray(t.namespaces)||!Number.isSafeInteger(t.maxOutputBytes)||t.maxOutputBytes<4||!Number.isSafeInteger(t.maxFrameBytes)||t.maxFrameBytes{e.once(`exit`,()=>{t()})})}function $(){return{type:`done`,error:{kind:`worker-exit`,message:`code runtime bridge frame exceeded maxFrameBytes`}}}function Ye(){let e=n({input:process.stdin,crlfDelay:1/0}),r,i=0,a=2,o=0,s=!1,c=t=>{if(s)return;let n=X(t,J)??X($(),J);s=!0,n!==void 0&&He(n);let i=r;r=void 0,(i===void 0?Promise.resolve():new Promise(e=>{setImmediate(e)}).then(async()=>{let e=q(i.stdout),t=q(i.stderr),n=Je(i);i.kill(`SIGKILL`),await Promise.all([n,e,t])})).catch(e=>{process.stderr.write(`dsh-code-runtime-subprocess controller cleanup error: ${String(e)}\\n`)}).then(()=>{e.close(),process.stdin.destroy()})},l=e=>{if(s)return;let t=+(o>0),n=b(e,i-a-t);if(n===void 0){c({type:`output-limit`});return}a+=n+t,o+=1,Ue({type:`log`,text:e})||c($())},u=e=>{i=e.maxOutputBytes,J=e.maxFrameBytes,r=t(process.execPath,process.execArgv,{env:{DSH_CODE_RUNTIME_CONTROLLER:`1`},detached:!1,stdio:[`ignore`,`pipe`,`pipe`,`ipc`]});let n=r;n.stdout.on(`data`,e=>{l(e.toString(`utf8`))}),n.stderr.on(`data`,e=>{l(e.toString(`utf8`))}),n.on(`message`,e=>{let t=Y(e);if(t!==void 0){if(t.type===`log`&&typeof t.text==`string`){l(t.text);return}if(!s){if(t.type===`call`&&typeof t.id==`number`&&typeof t.global==`string`&&typeof t.name==`string`)Ue({type:`call`,id:t.id,global:t.global,name:t.name,args:Q(t.args)})||c($());else if(t.type===`output-limit`)c({type:`output-limit`});else if(t.type===`done`){let e=Ke(t,We);e!==void 0&&c(e)}}}}),n.on(`error`,e=>{c({type:`done`,error:{kind:`worker-exit`,message:`remote controller error: ${e.message}`}})}),n.on(`exit`,e=>{s||c({type:`done`,error:{kind:`worker-exit`,message:`remote controller exited with code ${e} before completing`}})}),n.send(e,e=>{e!==null&&c({type:`done`,error:{kind:`worker-exit`,message:`remote controller boot failed: ${e.message}`}})})};e.on(`line`,e=>{let t;try{t=JSON.parse(e)}catch(e){process.stderr.write(`dsh-code-runtime-subprocess frame error: ${String(e)}\\n`),c({type:`done`,error:{kind:`worker-exit`,message:`remote runner received a malformed frame`}});return}if(r===void 0){let e=Z(t);if(e===void 0){c({type:`done`,error:{kind:`worker-exit`,message:`remote runner received an invalid boot frame`}});return}u(e);return}let n=qe(t);n!==void 0&&r.send(n,e=>{e!==null&&c({type:`done`,error:{kind:`worker-exit`,message:`remote controller reply failed: ${e.message}`}})})}),e.on(`close`,()=>{r!==void 0&&!s&&c({type:`done`,error:{kind:`abort`,message:`remote runner input closed`}})})}function Xe(){let t,n=!1,a,o=0,s=e=>process.send===void 0||o>0&&X(e,o)===void 0?!1:(process.send(e),!0),c=e=>{if(n)return;n=!0,clearInterval(a);let r=o>0&&X(e,o)===void 0?$():e,i=t;t=void 0,(i===void 0?Promise.resolve():new Promise(e=>{setImmediate(e)}).then(async()=>{let e=q(i.stdout),t=q(i.stderr);await Promise.all([i.terminate(),e,t])})).catch(e=>{s({type:`log`,text:`dsh-code-runtime-subprocess worker cleanup error: ${String(e)}\\n`})}).then(()=>{if(process.send===void 0){process.exitCode=1;return}process.send(r,()=>{process.connected&&process.disconnect()})})};process.on(`message`,l=>{if(t===void 0){let u=Z(l);if(u===void 0){c({type:`done`,error:{kind:`worker-exit`,message:`remote controller received an invalid boot frame`}});return}o=u.maxFrameBytes,t=new i(new URL(`data:text/javascript;base64,${e.from(Ve()).toString(`base64`)}`),{workerData:u,env:{},execArgv:[],stdout:!0,stderr:!0,resourceLimits:{maxOldGenerationSizeMb:u.maxOldGenerationSizeMb}});let d=t;d.stdout.on(`data`,e=>{s({type:`log`,text:e.toString(`utf8`)})||c($())}),d.stderr.on(`data`,e=>{s({type:`log`,text:e.toString(`utf8`)})||c($())}),d.on(`message`,e=>{let t=Y(e);if(t!==void 0){if(t.type===`call`&&typeof t.id==`number`&&typeof t.global==`string`&&typeof t.name==`string`)s({type:`call`,id:t.id,global:t.global,name:t.name,args:Q(t.args)})||c($());else if(t.type===`log`&&typeof t.text==`string`)s({type:`log`,text:t.text})||c($());else if(t.type===`output-limit`)c({type:`output-limit`});else if(t.type===`done`){let e=Ke(t,Ge);e!==void 0&&c(e)}}}),d.on(`error`,e=>{c({type:`done`,error:{kind:`worker-exit`,message:`worker error: ${e.stack||e.message||r(e)}`}})}),d.on(`exit`,e=>{n||c({type:`done`,error:{kind:`worker-exit`,message:`worker exited with code ${e} before completing`}})}),a=setInterval(()=>{t!==void 0&&t.performance.eventLoopUtilization().active>u.computeMs&&c({type:`done`,error:{kind:`timeout`,message:`compute budget exhausted (${u.computeMs}ms busy)`}})},25);return}let u=qe(l);u!==void 0&&t.postMessage(u)}),process.on(`disconnect`,()=>{t!==void 0&&!n&&t.terminate()})}if(a)process.env.DSH_CODE_RUNTIME_CONTROLLER===`1`?Xe():Ye();else{if(o===null)throw Error(`remote worker requires parentPort`);let e=s;ze(o,e,{stdout:process.stdout,stderr:process.stderr},e.maxFrameBytes)}export{};" diff --git a/packages/util/atomic-write/src/runner.ts b/packages/util/atomic-write/src/runner.ts index 92fad6db2e..2655d22026 100644 --- a/packages/util/atomic-write/src/runner.ts +++ b/packages/util/atomic-write/src/runner.ts @@ -1,12 +1,11 @@ /** Typed source for the dependency-free execution-world runner bundle. */ import { Buffer } from 'node:buffer' -import { fork } from 'node:child_process' +import { spawn } from 'node:child_process' import type { ChildProcess } from 'node:child_process' import { createInterface } from 'node:readline' import type { Readable } from 'node:stream' import { inspect } from 'node:util' -import { fileURLToPath } from 'node:url' import { Worker, isMainThread, parentPort, workerData } from 'node:worker_threads' import { decodeWorkerJson, @@ -56,6 +55,14 @@ const failureKinds = new Set([ let maxFrameBytes = 0 +function runnerSource(): string { + const evalIndex = process.execArgv.indexOf('--eval') + if (evalIndex < 0) throw new Error('code runtime runner requires its eval source') + const source = process.execArgv[evalIndex + 1] + if (source === undefined) throw new Error('code runtime runner requires its eval source') + return source +} + function recordOf(value: unknown): Record | undefined { return typeof value === 'object' && value !== null ? value as Record : undefined } @@ -197,10 +204,9 @@ function runLauncher(): void { const startController = (boot: RuntimeBootData): void => { maxOutputBytes = boot.maxOutputBytes maxFrameBytes = boot.maxFrameBytes - controller = fork(fileURLToPath(import.meta.url), [], { + controller = spawn(process.execPath, process.execArgv, { env: { DSH_CODE_RUNTIME_CONTROLLER: '1' }, detached: false, - execArgv: [], stdio: ['ignore', 'pipe', 'pipe', 'ipc'], }) as Controller const current = controller @@ -325,7 +331,8 @@ function runController(): void { return } controllerMaxFrameBytes = boot.maxFrameBytes - worker = new Worker(new URL(import.meta.url), { + const sourceUrl = new URL(`data:text/javascript;base64,${Buffer.from(runnerSource()).toString('base64')}`) + worker = new Worker(sourceUrl, { workerData: boot, env: {}, execArgv: [], From 87b2c5d93f44bd52bfd52b536251e98ae7c274d2 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Thu, 30 Jul 2026 05:04:01 +0800 Subject: [PATCH 049/108] docs(runtime): remove stale runtime storage claims --- .../architecture/2026-07-26-subprocess-seam.i18n.yaml | 4 ++-- .../implemented/architecture/2026-07-26-subprocess-seam.md | 4 ++-- .../implemented/architecture/2026-07-26-subprocess-seam.zh.md | 4 ++-- .../2026-07-28-portable-execution-world-consumers.i18n.yaml | 4 ++-- .../2026-07-28-portable-execution-world-consumers.md | 2 +- .../2026-07-28-portable-execution-world-consumers.zh.md | 2 +- docs/core-data-structures/subprocess.i18n.yaml | 4 ++-- docs/core-data-structures/subprocess.md | 2 +- docs/core-data-structures/subprocess.zh.md | 2 +- packages/subprocess/README.i18n.yaml | 4 ++-- packages/subprocess/README.md | 4 ++-- packages/subprocess/README.zh.md | 4 ++-- 12 files changed, 20 insertions(+), 20 deletions(-) diff --git a/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.i18n.yaml index 0b62184128..fde94ee5cf 100644 --- a/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md -2026-07-26-subprocess-seam.md: 359977ac6a50394405163bc060f3a39eb68891fb -2026-07-26-subprocess-seam.zh.md: 551fcc8f4df82d0c893d8e6f5423f172dfd5d9c5 +2026-07-26-subprocess-seam.md: 214a46959c48862176e3c790d541def3444e31ad +2026-07-26-subprocess-seam.zh.md: 13ba397458a041ec358828bab53e75516fd3e76d diff --git a/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md b/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md index 359977ac6a..214a46959c 100644 --- a/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md +++ b/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md @@ -12,8 +12,8 @@ English | [中文](2026-07-26-subprocess-seam.zh.md) A new `subprocess/` capability family owns "run and manage a process"; the bash family keeps "run a bash command" and consumes it: -- **`@deepseek-ai/dsh-subprocess` (interface)** — the abstract `SubprocessService` owning `ctx.subprocess`: execution-world cwd and runtime storage, executable lookup, fully explicit ordinary spawns, and the terminal primitive added by the [portable execution-world decision](2026-07-28-portable-execution-world-consumers.md). Each stdio stream independently selects `'pipe'`, `'inherit'`, or bounded collection `{ maxBytes, spill? }`; stdin selects `'ignore'`, `'pipe'`, or `{ data }`. `SubprocessOutcome` carries exit facts with deliberately no timeout/cancel classification, while collected output remains on the handle after settlement. The seam also owns process and terminal handles, the shared scrub, and `DSH_ENV_PREFIX`/`DshEnvironment`/`CollectedOutput`; `argv` is never shell-interpreted. -- **`@deepseek-ai/dsh-subprocess-local` (implementation)** — `LocalSubprocessService` over the former `run.ts` plumbing (`spawn.ts`) plus `node-pty`: detached groups, bounded collection and private spill files, executable lookup, private runtime storage, foreground/session inspection, and disposal that terminates and joins every managed process. `terminate()` owns TERM→grace→KILL for the tree, `waitForExit()` observes tree liveness, and injected `taskkill /T` covers Windows. Ordinary and terminal spawns apply the seam's case-insensitive `KEY`/`PASSWORD`/`SECRET`/`TOKEN` scrub before explicit env. The implementation has no config; every limit arrives on the spec, while Bash and PTY presentation environment overrides stay in their consumers. +- **`@deepseek-ai/dsh-subprocess` (interface)** — the abstract `SubprocessService` owning `ctx.subprocess`: execution-world cwd, executable lookup, fully explicit ordinary spawns, and the terminal primitive added by the [portable execution-world decision](2026-07-28-portable-execution-world-consumers.md). Each stdio stream independently selects `'pipe'`, `'inherit'`, or bounded collection `{ maxBytes, spill? }`; stdin selects `'ignore'`, `'pipe'`, or `{ data }`. `SubprocessOutcome` carries exit facts with deliberately no timeout/cancel classification, while collected output remains on the handle after settlement. The seam also owns process and terminal handles, the shared scrub, and `DSH_ENV_PREFIX`/`DshEnvironment`/`CollectedOutput`; `argv` is never shell-interpreted. +- **`@deepseek-ai/dsh-subprocess-local` (implementation)** — `LocalSubprocessService` over the former `run.ts` plumbing (`spawn.ts`) plus `node-pty`: detached groups, bounded collection and private spill files, executable lookup, foreground/session inspection, and disposal that terminates and joins every managed process. `terminate()` owns TERM→grace→KILL for the tree, `waitForExit()` observes tree liveness, and injected `taskkill /T` covers Windows. Ordinary and terminal spawns apply the seam's case-insensitive `KEY`/`PASSWORD`/`SECRET`/`TOKEN` scrub before explicit env. The implementation has no config; every limit arrives on the spec, while Bash and PTY presentation environment overrides stay in their consumers. - **`dsh-bash-local` (consumer)** — `inject: ['subprocess']`; maps each resolved `BashExecSpec` onto a `SubprocessSpawnSpec` (`['bash', '-c', command]`), keeps its config, `resolve()` defaulting, fused-deadline `timedOut`/`aborted` classification, the `[stderr]`-marked background read merge with its consuming cursor, and the `onProcessDone` subclass hook. `dsh-bash-sandbox` is unchanged apart from redeclaring the inherited inject; it still wraps at the command-string level and re-enters the inherited spawn path. - **`dsh-bash` (seam)** — re-exports the moved vocabulary from `dsh-subprocess`, so no bash consumer changes an import; `BashExecRequest`/`BashExecSpec`/`BashProcess` and the sandbox facts remain bash-owned. diff --git a/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.zh.md b/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.zh.md index 551fcc8f4d..13ba397458 100644 --- a/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.zh.md @@ -12,8 +12,8 @@ Status: implemented 新的 `subprocess/` 能力家族拥有「运行并管理一个进程」;bash 家族保留「运行一条 bash 命令」,并成为前者的消费方: -- **`@deepseek-ai/dsh-subprocess`(接口)**——拥有 `ctx.subprocess` 的抽象 `SubprocessService`:执行环境 cwd 与运行时存储、可执行文件查找、完全显式的普通 spawn,以及[可移植执行环境决策](2026-07-28-portable-execution-world-consumers.md)新增的终端原语。每条 stdio 流独立选择 `'pipe'`、`'inherit'` 或有界收集 `{ maxBytes, spill? }`;stdin 选择 `'ignore'`、`'pipe'` 或 `{ data }`。`SubprocessOutcome` 只承载刻意不含超时/取消分类的退出事实,收集输出在结算后仍留在句柄上。该 seam 还拥有进程与终端句柄、共享凭据清除,以及 `DSH_ENV_PREFIX`/`DshEnvironment`/`CollectedOutput`;`argv` 绝不经过 shell 解释。 -- **`@deepseek-ai/dsh-subprocess-local`(实现)**——`LocalSubprocessService` 构建在原 `run.ts` 管道(现为 `spawn.ts`)与 `node-pty` 之上:detached 进程组、有界收集与私有 spill 文件、可执行文件查找、私有运行时存储、前台/会话检查,以及终止每个受管进程并等待其退出的 dispose。`terminate()` 拥有面向进程树的 TERM→宽限→KILL,`waitForExit()` 观察进程树存活性,可注入的 `taskkill /T` 覆盖 Windows。普通与终端 spawn 都先应用 seam 对 `KEY`/`PASSWORD`/`SECRET`/`TOKEN` 不区分大小写的清除,再合并显式 env。该实现没有配置;每项限制都随 spec 到达,Bash 与 PTY 的呈现环境覆盖仍归各自消费方所有。 +- **`@deepseek-ai/dsh-subprocess`(接口)**——拥有 `ctx.subprocess` 的抽象 `SubprocessService`:执行环境 cwd、可执行文件查找、完全显式的普通 spawn,以及[可移植执行环境决策](2026-07-28-portable-execution-world-consumers.md)新增的终端原语。每条 stdio 流独立选择 `'pipe'`、`'inherit'` 或有界收集 `{ maxBytes, spill? }`;stdin 选择 `'ignore'`、`'pipe'` 或 `{ data }`。`SubprocessOutcome` 只承载刻意不含超时/取消分类的退出事实,收集输出在结算后仍留在句柄上。该 seam 还拥有进程与终端句柄、共享凭据清除,以及 `DSH_ENV_PREFIX`/`DshEnvironment`/`CollectedOutput`;`argv` 绝不经过 shell 解释。 +- **`@deepseek-ai/dsh-subprocess-local`(实现)**——`LocalSubprocessService` 构建在原 `run.ts` 管道(现为 `spawn.ts`)与 `node-pty` 之上:detached 进程组、有界收集与私有 spill 文件、可执行文件查找、前台/会话检查,以及终止每个受管进程并等待其退出的 dispose。`terminate()` 拥有面向进程树的 TERM→宽限→KILL,`waitForExit()` 观察进程树存活性,可注入的 `taskkill /T` 覆盖 Windows。普通与终端 spawn 都先应用 seam 对 `KEY`/`PASSWORD`/`SECRET`/`TOKEN` 不区分大小写的清除,再合并显式 env。该实现没有配置;每项限制都随 spec 到达,Bash 与 PTY 的呈现环境覆盖仍归各自消费方所有。 - **`dsh-bash-local`(消费方)**——`inject: ['subprocess']`;把每个解析后的 `BashExecSpec` 映射为一个 `SubprocessSpawnSpec`(`['bash', '-c', command]`),并保留自身配置、`resolve()` 默认值补全、基于融合 deadline 的 `timedOut`/`aborted` 分类、带 `[stderr]` 标记的后台读取合并及其消费游标,以及 `onProcessDone` 子类钩子。`dsh-bash-sandbox` 除了重新声明继承来的 inject 之外没有变化;它仍在命令字符串层面做包装,并重新进入继承的 spawn 路径。 - **`dsh-bash`(seam)**——把迁走的词汇从 `dsh-subprocess` 重导出,因此没有任何 bash 消费方需要改动导入;`BashExecRequest`/`BashExecSpec`/`BashProcess` 与沙箱事实仍归 bash 所有。 diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml index 3fce989f78..62592b5be0 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md -2026-07-28-portable-execution-world-consumers.md: 74fa2687782a2cf59be8dddf98fa4a08a953d565 -2026-07-28-portable-execution-world-consumers.zh.md: 35d4f3640a44564894fecfaa22c7255d7452dec9 +2026-07-28-portable-execution-world-consumers.md: 97c8bcfc7bf84db4b6a2862fcc9d991ee7f4a7f6 +2026-07-28-portable-execution-world-consumers.zh.md: 025393f9e5ae99feba41fa8badeef8276151ba6f diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md index 74fa268778..97c8bcfc7b 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md @@ -16,7 +16,7 @@ Ordinary pipes do not cover one requirement. A persistent terminal needs PTY all The filesystem interface owns the path facts that another capability needs without exposing its opaque target identity: a canonical process path, canonical `file:` URI, and containment. Existing whole and streaming text operations remain filesystem-owned; protocol consumers enforce their own retention limits while consuming the stream. -The subprocess interface owns the process coordinates and primitives: canonical cwd, private runtime storage, executable lookup, ordinary raw or collected process spawning, and `spawnTerminal()`. The terminal operation is one deep primitive whose handle owns text I/O, foreground groups, signalling, and one awaited TERM-to-KILL operation that settles in-flight handle calls and reaches quiescence for every session member the provider can still observe. Its signal cancels allocation only; the published handle owns its lifetime. Prompt detection, idle inference, scrollback, sandbox policy, and owner lifecycle remain in the PTY consumer. +The subprocess interface owns the process coordinates and primitives: canonical cwd, executable lookup, ordinary raw or collected process spawning, and `spawnTerminal()`. The terminal operation is one deep primitive whose handle owns text I/O, foreground groups, signalling, and one awaited TERM-to-KILL operation that settles in-flight handle calls and reaches quiescence for every session member the provider can still observe. Its signal cancels allocation only; the published handle owns its lifetime. Prompt detection, idle inference, scrollback, sandbox policy, and owner lifecycle remain in the PTY consumer. Generic consumers use that execution world: diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md index 35d4f3640a..025393f9e5 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md @@ -16,7 +16,7 @@ Status: implemented 文件系统接口负责其他能力需要的路径事实,同时不公开其不透明目标身份:规范化进程路径、规范化 `file:` URI 和包含关系。现有完整文本与流式文本操作仍归文件系统负责;协议消费方在消费流时执行各自的保留上限。 -进程管理接口负责进程运行坐标与原语:规范化 cwd、私有运行时存储、可执行文件查找、以原始或收集模式 spawn 普通进程,以及 `spawnTerminal()`。终端操作是一项深层原语,其句柄负责文本 I/O、前台进程组、信号发送,以及一项须等待的 TERM→KILL 操作;该操作会结算所有在途句柄调用,并使提供方仍可观察到的每个会话成员完全停稳。其信号只取消分配;句柄一经发布,便负责自身生命周期。提示符检测、空闲推断、scrollback、沙箱策略和所有者生命周期仍由 PTY 消费方负责。 +进程管理接口负责进程运行坐标与原语:规范化 cwd、可执行文件查找、以原始或收集模式 spawn 普通进程,以及 `spawnTerminal()`。终端操作是一项深层原语,其句柄负责文本 I/O、前台进程组、信号发送,以及一项须等待的 TERM→KILL 操作;该操作会结算所有在途句柄调用,并使提供方仍可观察到的每个会话成员完全停稳。其信号只取消分配;句柄一经发布,便负责自身生命周期。提示符检测、空闲推断、scrollback、沙箱策略和所有者生命周期仍由 PTY 消费方负责。 通用消费方使用该执行世界: diff --git a/docs/core-data-structures/subprocess.i18n.yaml b/docs/core-data-structures/subprocess.i18n.yaml index f6c61bd691..cfa3762222 100644 --- a/docs/core-data-structures/subprocess.i18n.yaml +++ b/docs/core-data-structures/subprocess.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write docs/core-data-structures/subprocess.md -subprocess.md: f0fe691219df3afba2b836da8f1594fd5fe5a87b -subprocess.zh.md: f59e787820ba2727fdc0c4b62c4aec7b8d2d6b7a +subprocess.md: c68a2f77059f62b273c536987b321017f2a71bbb +subprocess.zh.md: 061c681430ba1f011b4ed9e5c4d9fc8595fe6738 diff --git a/docs/core-data-structures/subprocess.md b/docs/core-data-structures/subprocess.md index 89f30095bc..a258ff4767 100644 --- a/docs/core-data-structures/subprocess.md +++ b/docs/core-data-structures/subprocess.md @@ -246,4 +246,4 @@ The terminal spec fully specifies argv, cwd, environment overrides, dimensions, ## Service behavior -The abstract [`SubprocessService`](../../packages/subprocess/subprocess/src/index.ts) seam defines execution-world coordinates, executable lookup, ordinary `spawn`, and `spawnTerminal`. [`LocalSubprocessService`](../../packages/subprocess/subprocess-local/src/index.ts) implements them with temporary runtime storage, detached process trees, per-disposition wiring, credential scrubbing, `node-pty`, platform process inspection, and terminate-and-join disposal. See [`dsh-subprocess`](../../packages/subprocess/subprocess/README.md) for the interface contract and [`dsh-subprocess-local`](../../packages/subprocess/subprocess-local/README.md) for local mechanics. +The abstract [`SubprocessService`](../../packages/subprocess/subprocess/src/index.ts) seam defines execution-world coordinates, executable lookup, ordinary `spawn`, and `spawnTerminal`. [`LocalSubprocessService`](../../packages/subprocess/subprocess-local/src/index.ts) implements them with detached process trees, per-disposition wiring, credential scrubbing, `node-pty`, platform process inspection, and terminate-and-join disposal. See [`dsh-subprocess`](../../packages/subprocess/subprocess/README.md) for the interface contract and [`dsh-subprocess-local`](../../packages/subprocess/subprocess-local/README.md) for local mechanics. diff --git a/docs/core-data-structures/subprocess.zh.md b/docs/core-data-structures/subprocess.zh.md index f59e787820..061c681430 100644 --- a/docs/core-data-structures/subprocess.zh.md +++ b/docs/core-data-structures/subprocess.zh.md @@ -246,4 +246,4 @@ interface SubprocessOutcome { ## 服务行为 -抽象的 [`SubprocessService`](../../packages/subprocess/subprocess/src/index.ts) seam 定义执行世界坐标、可执行文件查找、普通 `spawn` 与 `spawnTerminal`。[`LocalSubprocessService`](../../packages/subprocess/subprocess-local/src/index.ts) 以临时运行时存储、detached 进程树、按处置方式接线、凭据清除、`node-pty`、平台进程检查,以及先终止再等待退出的资源释放实现这些能力。接口契约见 [`dsh-subprocess`](../../packages/subprocess/subprocess/README.md),本地机制见 [`dsh-subprocess-local`](../../packages/subprocess/subprocess-local/README.md)。 +抽象的 [`SubprocessService`](../../packages/subprocess/subprocess/src/index.ts) seam 定义执行世界坐标、可执行文件查找、普通 `spawn` 与 `spawnTerminal`。[`LocalSubprocessService`](../../packages/subprocess/subprocess-local/src/index.ts) 以 detached 进程树、按处置方式接线、凭据清除、`node-pty`、平台进程检查,以及先终止再等待退出的资源释放实现这些能力。接口契约见 [`dsh-subprocess`](../../packages/subprocess/subprocess/README.md),本地机制见 [`dsh-subprocess-local`](../../packages/subprocess/subprocess-local/README.md)。 diff --git a/packages/subprocess/README.i18n.yaml b/packages/subprocess/README.i18n.yaml index f6de20e259..af424b8809 100644 --- a/packages/subprocess/README.i18n.yaml +++ b/packages/subprocess/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/subprocess/README.md -README.md: b4aaf78454a51af4d3f7ea9f1aef450fde8d729d -README.zh.md: c5013ce19bacb582b6c8c93e0ea0a73ec776ee57 +README.md: 5ea1ae3681f9340cfba798e659f454d6d05fdbfe +README.zh.md: 4e1cb9472d6e90062ecde61db4200a79a56b16f1 diff --git a/packages/subprocess/README.md b/packages/subprocess/README.md index b4aaf78454..5ea1ae3681 100644 --- a/packages/subprocess/README.md +++ b/packages/subprocess/README.md @@ -2,11 +2,11 @@ English | [中文](README.zh.md) -The shared process substrate for one execution world: canonical cwd/runtime storage, executable lookup, fully-specified managed child-process trees with raw or collected stdio, and one deep terminal-process primitive that owns PTY allocation, foreground groups, and provider-observable session cleanup. Command defaulting, shell semantics, deadlines, protocol framing, readiness, and presentation stay with consumers — the [bash executors](../bash/README.md), [LSP host](../lsp/README.md), [PTY shell backend](../pty/README.md), [subprocess code runtime](../code-runtime/code-runtime-subprocess/README.md), and [ACP subagent backend](../subagent/README.md). See the [subprocess seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md). +The shared process substrate for one execution world: canonical cwd, executable lookup, fully-specified managed child-process trees with raw or collected stdio, and one deep terminal-process primitive that owns PTY allocation, foreground groups, and provider-observable session cleanup. Command defaulting, shell semantics, deadlines, protocol framing, readiness, and presentation stay with consumers — the [bash executors](../bash/README.md), [LSP host](../lsp/README.md), [PTY shell backend](../pty/README.md), [subprocess code runtime](../code-runtime/code-runtime-subprocess/README.md), and [ACP subagent backend](../subagent/README.md). See the [subprocess seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md). | Package | ctx key | Role | |---|---|---| | [`subprocess`](subprocess/README.md) (`@deepseek-ai/dsh-subprocess`) | `ctx.subprocess` | The seam: execution-world coordinates and executable lookup, ordinary managed spawns, the terminal-process primitive, handle lifecycles, and shared environment/output vocabulary | -| [`subprocess-local`](subprocess-local/README.md) (`@deepseek-ai/dsh-subprocess-local`) | — | The local implementation: detached process trees, bounded collection/spill, `node-pty`, foreground/session inspection, tree signalling, runtime storage, and terminate-and-join disposal | +| [`subprocess-local`](subprocess-local/README.md) (`@deepseek-ai/dsh-subprocess-local`) | — | The local implementation: detached process trees, bounded collection/spill, `node-pty`, foreground/session inspection, tree signalling, and terminate-and-join disposal | The service owns process lifetime across consumer reloads; consumers own what a process means (a bash command, a future non-shell runner) and every default that shapes one. diff --git a/packages/subprocess/README.zh.md b/packages/subprocess/README.zh.md index c5013ce19b..4e1cb9472d 100644 --- a/packages/subprocess/README.zh.md +++ b/packages/subprocess/README.zh.md @@ -2,11 +2,11 @@ [English](README.md) | 中文 -这里集中提供一个执行世界的共享进程基底:规范化 cwd/运行时存储、可执行文件查找、具有原始或收集式 stdio 的完整指定受管子进程树,以及一项深层终端进程原语,负责 PTY 分配、前台进程组和提供方仍可观察到的会话成员清理。命令默认值补全、shell 语义、时限、协议分帧、就绪状态与呈现留在消费方:[bash 执行器](../bash/README.md)、[LSP 主机](../lsp/README.md)、[PTY shell 后端](../pty/README.md)、[子进程代码运行时](../code-runtime/code-runtime-subprocess/README.md)与 [ACP(Agent Client Protocol)subagent 后端](../subagent/README.md)。参见[subprocess seam Agent Note(agent 决策记录)](../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md)。 +这里集中提供一个执行世界的共享进程基底:规范化 cwd、可执行文件查找、具有原始或收集式 stdio 的完整指定受管子进程树,以及一项深层终端进程原语,负责 PTY 分配、前台进程组和提供方仍可观察到的会话成员清理。命令默认值补全、shell 语义、时限、协议分帧、就绪状态与呈现留在消费方:[bash 执行器](../bash/README.md)、[LSP 主机](../lsp/README.md)、[PTY shell 后端](../pty/README.md)、[子进程代码运行时](../code-runtime/code-runtime-subprocess/README.md)与 [ACP(Agent Client Protocol)subagent 后端](../subagent/README.md)。参见[subprocess seam Agent Note(agent 决策记录)](../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md)。 | 包(package) | ctx 键 | 角色 | |---|---|---| | [`subprocess`](subprocess/README.md)(`@deepseek-ai/dsh-subprocess`) | `ctx.subprocess` | seam 本体:执行世界坐标与可执行文件查找、普通受管 spawn、终端进程原语、句柄生命周期,以及共享的环境/输出词汇 | -| [`subprocess-local`](subprocess-local/README.md)(`@deepseek-ai/dsh-subprocess-local`) | 无 | 本地实现:detached 进程树、有界收集/spill、`node-pty`、前台/会话检查、进程树信号发送、运行时存储,以及先终止再等待退出的资源释放 | +| [`subprocess-local`](subprocess-local/README.md)(`@deepseek-ai/dsh-subprocess-local`) | 无 | 本地实现:detached 进程树、有界收集/spill、`node-pty`、前台/会话检查、进程树信号发送,以及先终止再等待退出的资源释放 | 即使消费方重载,进程生命周期仍由服务负责管理;消费方负责定义进程的含义(一条 bash 命令、未来的非 shell 运行器),以及决定塑造该进程的每一项默认值。 From c1d08edd8324092c4981baee3bd030fbc2cd740b Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Thu, 30 Jul 2026 05:19:53 +0800 Subject: [PATCH 050/108] refactor(runtime): remove duplicate terminal ownership state --- packages/pty/pty-local/src/session.ts | 16 ++++++---------- .../subprocess/subprocess-local/src/terminal.ts | 5 ++--- .../subprocess-local/tests/terminal.spec.ts | 1 + 3 files changed, 9 insertions(+), 13 deletions(-) diff --git a/packages/pty/pty-local/src/session.ts b/packages/pty/pty-local/src/session.ts index ebd7e076cf..faea1ef84d 100644 --- a/packages/pty/pty-local/src/session.ts +++ b/packages/pty/pty-local/src/session.ts @@ -167,7 +167,7 @@ export class LocalPtySession implements PtyBackendSession { private activeDeadlineTimer: NodeJS.Timeout | undefined private activeAbort: (() => void) | undefined private interrupting: LocalSendOperation | undefined - private activeWrite: { operation: LocalSendOperation; settled: Promise } | undefined + private activeWrite: Promise | undefined private pollingReady: LocalSendOperation | undefined private polling = false private promptSeen = false @@ -238,7 +238,7 @@ export class LocalPtySession implements PtyBackendSession { } this.activeDeadlineTimer = setTimeout(() => { if (this.active === operation) { - this.settleActive('timeout', this.activeWrite?.operation === operation || this.interrupting === operation) + this.settleActive('timeout', this.activeWrite !== undefined || this.interrupting === operation) } }, this.config.timeoutMs) void this.beginSend(operation, request) @@ -254,11 +254,7 @@ export class LocalPtySession implements PtyBackendSession { if (input.length > 0 && !operation.cancelRequested) { this.resetReadinessEvidence() const write = this.terminal.write(input) - const activeWrite = { - operation, - settled: write.then(() => true, () => false), - } - this.activeWrite = activeWrite + this.activeWrite = write.then(() => true, () => false) try { await write } finally { @@ -272,7 +268,7 @@ export class LocalPtySession implements PtyBackendSession { return } // Closing can race the awaited provider write even though static analysis sees only local assignments. - // eslint-disable-next-line @typescript-eslint/no-unnecessary-condition + // oxlint-disable-next-line typescript/no-unnecessary-condition -- awaited provider writes can close the session. if (this.active === operation && !this.closing) { this.pollingReady = operation this.schedulePoll(operation) @@ -448,7 +444,7 @@ export class LocalPtySession implements PtyBackendSession { this.polling = false const active = this.active // Awaited provider inspection can clear or replace the active send despite static analysis. - // eslint-disable-next-line @typescript-eslint/no-unnecessary-condition + // oxlint-disable-next-line typescript/no-unnecessary-condition -- awaited inspection can replace the active send. if (active !== undefined && this.pollingReady === active) this.schedulePoll(active) } } @@ -506,7 +502,7 @@ export class LocalPtySession implements PtyBackendSession { private async interruptOnce(operation: LocalSendOperation): Promise { try { const activeWrite = this.activeWrite - if (activeWrite?.operation === operation && !await activeWrite.settled) return + if (activeWrite !== undefined && !await activeWrite) return await this.terminal.signalForeground('SIGINT') } catch (error: unknown) { if (this.active === operation && !this.closing) this.onTransportFailure(error) diff --git a/packages/subprocess/subprocess-local/src/terminal.ts b/packages/subprocess/subprocess-local/src/terminal.ts index 93b7a8fc2e..53c531675f 100644 --- a/packages/subprocess/subprocess-local/src/terminal.ts +++ b/packages/subprocess/subprocess-local/src/terminal.ts @@ -58,19 +58,18 @@ export class LocalTerminalHandle implements SubprocessTerminalHandle { exitCode: exitSignal === undefined || exitSignal === 0 ? exitCode : null, signal: signalName(exitSignal), }) - void this.terminate().catch(() => {}) }) } // node-pty writes synchronously; the seam returns a promise for remote transports. - // eslint-disable-next-line @typescript-eslint/require-await + // oxlint-disable-next-line typescript/require-await -- Preserve promise rejection semantics at the async provider seam. async write(data: string): Promise { if (this.exited) throw new Error('terminal process has exited') this.terminal.write(data) } // Local inspection is synchronous; the seam returns a promise for remote transports. - // eslint-disable-next-line @typescript-eslint/require-await + // oxlint-disable-next-line typescript/require-await -- Preserve promise rejection semantics at the async provider seam. async inspectForeground(): Promise { this.descendants() const processGroupId = this.inspector.foregroundPgid(this.pid) diff --git a/packages/subprocess/subprocess-local/tests/terminal.spec.ts b/packages/subprocess/subprocess-local/tests/terminal.spec.ts index 2eeb7c6a27..8a6270883f 100644 --- a/packages/subprocess/subprocess-local/tests/terminal.spec.ts +++ b/packages/subprocess/subprocess-local/tests/terminal.spec.ts @@ -128,6 +128,7 @@ describe('LocalTerminalHandle', () => { const handle = new LocalTerminalHandle(pty.asPty(), inspector, 20) const quiescent = handle.terminate() + expect(handle.terminate()).toBe(quiescent) await vi.advanceTimersByTimeAsync(20) expect(inspector.processes).toContainEqual([124, 'SIGKILL']) expect(pty.kills).toEqual([]) From 4d5345794d73e51d09620cce859303d642a76bf1 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Thu, 30 Jul 2026 14:24:47 +0800 Subject: [PATCH 051/108] refactor(code-runtime): remove subprocess backend --- .../2026-07-26-subprocess-seam.i18n.yaml | 4 +- .../2026-07-26-subprocess-seam.md | 10 +- .../2026-07-26-subprocess-seam.zh.md | 10 +- ...rtable-execution-world-consumers.i18n.yaml | 4 +- ...7-28-portable-execution-world-consumers.md | 13 +- ...8-portable-execution-world-consumers.zh.md | 14 +- .../feature/2026-06-15-code-mode.i18n.yaml | 4 +- .../feature/2026-06-15-code-mode.md | 14 +- .../feature/2026-06-15-code-mode.zh.md | 14 +- docs/capability-seams.md | 79 +- docs/config-catalog.md | 26 +- .../core-data-structures/subprocess.i18n.yaml | 4 +- docs/core-data-structures/subprocess.md | 6 +- docs/core-data-structures/subprocess.zh.md | 6 +- knip.json | 11 - package.json | 2 - packages/README.i18n.yaml | 4 +- packages/README.md | 2 +- packages/README.zh.md | 2 +- .../code-runtime-worker/src/bootstrap.ts | 19 +- .../code-runtime-worker/src/runtime-host.ts | 251 ------ .../tests/bootstrap.spec.ts | 27 - .../code-runtime-worker/tsconfig.json | 3 + .../code-runtime-worker/tsdown.config.ts | 17 +- packages/code-runtime/code-runtime/README.md | 4 +- .../code-runtime/tests/contract.ts | 853 ------------------ packages/subprocess/README.i18n.yaml | 4 +- packages/subprocess/README.md | 4 +- packages/subprocess/README.zh.md | 4 +- .../subprocess-local/README.i18n.yaml | 4 +- .../subprocess/subprocess-local/README.md | 4 +- .../subprocess/subprocess-local/README.zh.md | 4 +- .../subprocess/subprocess-local/src/index.ts | 3 +- .../subprocess-local/tests/local.spec.ts | 3 - .../subprocess/subprocess/README.i18n.yaml | 4 +- packages/subprocess/subprocess/README.md | 4 +- packages/subprocess/subprocess/README.zh.md | 4 +- packages/subprocess/subprocess/src/index.ts | 11 +- scripts/check-workspace-constraints.ts | 1 - scripts/gen-code-runtime-runner.ts | 70 -- scripts/gen-doc-graphs.ts | 276 ++---- scripts/run-gates.ts | 1 - .../verify-package-readme-model-experience.ts | 1 - tsconfig.host.json | 1 - 44 files changed, 192 insertions(+), 1614 deletions(-) delete mode 100644 packages/code-runtime/code-runtime-worker/src/runtime-host.ts delete mode 100644 packages/code-runtime/code-runtime/tests/contract.ts delete mode 100644 scripts/gen-code-runtime-runner.ts diff --git a/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.i18n.yaml index fde94ee5cf..f805e7cde3 100644 --- a/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md -2026-07-26-subprocess-seam.md: 214a46959c48862176e3c790d541def3444e31ad -2026-07-26-subprocess-seam.zh.md: 13ba397458a041ec358828bab53e75516fd3e76d +2026-07-26-subprocess-seam.md: 8797102c65ebab663bcf72fced5791364fe2c6ae +2026-07-26-subprocess-seam.zh.md: 8a7ff14079374d6d74a1ec729dd02c8961fa23e6 diff --git a/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md b/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md index 214a46959c..8797102c65 100644 --- a/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md +++ b/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md @@ -12,7 +12,7 @@ English | [中文](2026-07-26-subprocess-seam.zh.md) A new `subprocess/` capability family owns "run and manage a process"; the bash family keeps "run a bash command" and consumes it: -- **`@deepseek-ai/dsh-subprocess` (interface)** — the abstract `SubprocessService` owning `ctx.subprocess`: execution-world cwd, executable lookup, fully explicit ordinary spawns, and the terminal primitive added by the [portable execution-world decision](2026-07-28-portable-execution-world-consumers.md). Each stdio stream independently selects `'pipe'`, `'inherit'`, or bounded collection `{ maxBytes, spill? }`; stdin selects `'ignore'`, `'pipe'`, or `{ data }`. `SubprocessOutcome` carries exit facts with deliberately no timeout/cancel classification, while collected output remains on the handle after settlement. The seam also owns process and terminal handles, the shared scrub, and `DSH_ENV_PREFIX`/`DshEnvironment`/`CollectedOutput`; `argv` is never shell-interpreted. +- **`@deepseek-ai/dsh-subprocess` (interface)** — the abstract `SubprocessService` owning `ctx.subprocess`: executable lookup, fully explicit ordinary spawns, and the terminal primitive added by the [portable execution-world decision](2026-07-28-portable-execution-world-consumers.md). Each stdio stream independently selects `'pipe'`, `'inherit'`, or bounded collection `{ maxBytes, spill? }`; stdin selects `'ignore'`, `'pipe'`, or `{ data }`. `SubprocessOutcome` carries exit facts with deliberately no timeout/cancel classification, while collected output remains on the handle after settlement. The seam also owns process and terminal handles, the shared scrub, and `DSH_ENV_PREFIX`/`DshEnvironment`/`CollectedOutput`; `argv` is never shell-interpreted. - **`@deepseek-ai/dsh-subprocess-local` (implementation)** — `LocalSubprocessService` over the former `run.ts` plumbing (`spawn.ts`) plus `node-pty`: detached groups, bounded collection and private spill files, executable lookup, foreground/session inspection, and disposal that terminates and joins every managed process. `terminate()` owns TERM→grace→KILL for the tree, `waitForExit()` observes tree liveness, and injected `taskkill /T` covers Windows. Ordinary and terminal spawns apply the seam's case-insensitive `KEY`/`PASSWORD`/`SECRET`/`TOKEN` scrub before explicit env. The implementation has no config; every limit arrives on the spec, while Bash and PTY presentation environment overrides stay in their consumers. - **`dsh-bash-local` (consumer)** — `inject: ['subprocess']`; maps each resolved `BashExecSpec` onto a `SubprocessSpawnSpec` (`['bash', '-c', command]`), keeps its config, `resolve()` defaulting, fused-deadline `timedOut`/`aborted` classification, the `[stderr]`-marked background read merge with its consuming cursor, and the `onProcessDone` subclass hook. `dsh-bash-sandbox` is unchanged apart from redeclaring the inherited inject; it still wraps at the command-string level and re-enters the inherited spawn path. - **`dsh-bash` (seam)** — re-exports the moved vocabulary from `dsh-subprocess`, so no bash consumer changes an import; `BashExecRequest`/`BashExecSpec`/`BashProcess` and the sandbox facts remain bash-owned. @@ -21,13 +21,13 @@ Every composition that loads a bash executor now also loads `@deepseek-ai/dsh-su Background-process lifetime moved from the executor to the subprocess service: the executor no longer retains a live-process set, so an executor reload leaves background work running and readable, and composition teardown (the service's disposal) remains the kill-and-join boundary. One behavioral seam shifted with it: a background spawn failure can no longer be buffered as fake stderr inside the plumbing (the service rejects `done` and buffers nothing for a process that never ran), so the executor injects the `spawn failed: …` note into exactly one `readOutput()` delta. -Observed stream and lifecycle needs then moved the eligible process consumers onto the seam: LSP uses piped protocol streams plus a collected stderr tail; the ACP backend uses piped ndjson, inherited stderr, and a consumer-owned stdin-EOF disposal ladder; PTY uses `spawnTerminal()` while keeping readiness and terminal policy; Code Runtime uses ordinary raw pipes. `dsh-subagent-subprocess` and the private LSP tree helpers were deleted. MCP transport spawning, the SDK package-manager runner, synchronous TUI Git probing, and dependency-light test-support launchers remain outside by ownership or execution shape; their production callers share the scrub where applicable. +Observed stream and lifecycle needs then moved the eligible process consumers onto the seam: LSP uses piped protocol streams plus a collected stderr tail; the ACP backend uses piped ndjson, inherited stderr, and a consumer-owned stdin-EOF disposal ladder; PTY uses `spawnTerminal()` while keeping readiness and terminal policy. `dsh-subagent-subprocess` and the private LSP tree helpers were deleted. MCP transport spawning, the SDK package-manager runner, synchronous TUI Git probing, and dependency-light test-support launchers remain outside by ownership or execution shape; their production callers share the scrub where applicable. ## Alternatives considered **Leave the process plumbing inside `dsh-bash-local` (status quo).** Rejected for the same reason the [task registry split](2026-07-26-task-registry-seam.md) landed: the boundary is stable and already documented in-code (`run.ts`'s module doc said "this layer reacts to an abort signal; the executor owns deadlines and classifies causes"), and keeping it private makes every future non-shell runner either fork the mechanics or depend on a bash-named package for non-bash work. The user-visible driver for this stack was exactly this split. -**Keep the original batch-only interface and leave stream consumers bespoke.** Rejected after the observed LSP, ACP, PTY, and Code Runtime shapes showed that private process-tree signalling and environment scrubs would otherwise remain duplicated. The Node-shaped dispositions cover those consumers without buffering piped streams. +**Keep the original batch-only interface and leave stream consumers bespoke.** Rejected after the observed LSP, ACP, and PTY shapes showed that private process-tree signalling and environment scrubs would otherwise remain duplicated. The Node-shaped dispositions cover those consumers without buffering piped streams. **Use one `stdio: 'pipe' | 'inherit' | 'collect'` mode for all streams.** Rejected because real consumers mix modes per stream: LSP uses pipe/pipe/collect, ACP uses pipe/pipe/inherit, and Bash uses data/collect/collect. @@ -39,6 +39,6 @@ Observed stream and lifecycle needs then moved the eligible process consumers on ## Consequences -Bought: "run and manage a process" is a swappable capability used by Bash, LSP, PTY, Code Runtime, and ACP consumers; a containerized or remote process backend slots in without changing their domain semantics; tree signalling, escalation, bounded collection, terminal mechanics, and credential scrubbing each have one implementation; and background processes survive executor reloads, matching the task registry's lifetime model. Process and terminal plumbing is tested through `dsh-subprocess-local`; consumer suites pin only their owned behavior against the real service. +Bought: "run and manage a process" is a swappable capability used by Bash, LSP, PTY, and ACP consumers; a containerized or remote process backend slots in without changing their domain semantics; tree signalling, escalation, bounded collection, terminal mechanics, and credential scrubbing each have one implementation; and background processes survive executor reloads, matching the task registry's lifetime model. Process and terminal plumbing is tested through `dsh-subprocess-local`; consumer suites pin only their owned behavior against the real service. -Cost: one more package pair and one more composition row wherever a consumer loads; a missing subprocess provider leaves the consumer pending by standard service-injection behavior. Every backend implements execution-world coordinates, executable lookup, three stdio modes, tree lifecycle, and one terminal primitive. The moved-vocabulary re-exports keep `dsh-bash` imports working but mean two packages name the same types; the subprocess seam is the owner. The spawn-failure note became single-delivery through Bash's consuming read cursor instead of repeatable stderr-buffer content. +Cost: one more package pair and one more composition row wherever a consumer loads; a missing subprocess provider leaves the consumer pending by standard service-injection behavior. Every backend implements executable lookup, three stdio modes, tree lifecycle, and one terminal primitive. The moved-vocabulary re-exports keep `dsh-bash` imports working but mean two packages name the same types; the subprocess seam is the owner. The spawn-failure note became single-delivery through Bash's consuming read cursor instead of repeatable stderr-buffer content. diff --git a/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.zh.md b/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.zh.md index 13ba397458..8a7ff14079 100644 --- a/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.zh.md @@ -12,7 +12,7 @@ Status: implemented 新的 `subprocess/` 能力家族拥有「运行并管理一个进程」;bash 家族保留「运行一条 bash 命令」,并成为前者的消费方: -- **`@deepseek-ai/dsh-subprocess`(接口)**——拥有 `ctx.subprocess` 的抽象 `SubprocessService`:执行环境 cwd、可执行文件查找、完全显式的普通 spawn,以及[可移植执行环境决策](2026-07-28-portable-execution-world-consumers.md)新增的终端原语。每条 stdio 流独立选择 `'pipe'`、`'inherit'` 或有界收集 `{ maxBytes, spill? }`;stdin 选择 `'ignore'`、`'pipe'` 或 `{ data }`。`SubprocessOutcome` 只承载刻意不含超时/取消分类的退出事实,收集输出在结算后仍留在句柄上。该 seam 还拥有进程与终端句柄、共享凭据清除,以及 `DSH_ENV_PREFIX`/`DshEnvironment`/`CollectedOutput`;`argv` 绝不经过 shell 解释。 +- **`@deepseek-ai/dsh-subprocess`(接口)**——拥有 `ctx.subprocess` 的抽象 `SubprocessService`:可执行文件查找、完全显式的普通 spawn,以及[可移植执行环境决策](2026-07-28-portable-execution-world-consumers.md)新增的终端原语。每条 stdio 流独立选择 `'pipe'`、`'inherit'` 或有界收集 `{ maxBytes, spill? }`;stdin 选择 `'ignore'`、`'pipe'` 或 `{ data }`。`SubprocessOutcome` 只承载刻意不含超时/取消分类的退出事实,收集输出在结算后仍留在句柄上。该 seam 还拥有进程与终端句柄、共享凭据清除,以及 `DSH_ENV_PREFIX`/`DshEnvironment`/`CollectedOutput`;`argv` 绝不经过 shell 解释。 - **`@deepseek-ai/dsh-subprocess-local`(实现)**——`LocalSubprocessService` 构建在原 `run.ts` 管道(现为 `spawn.ts`)与 `node-pty` 之上:detached 进程组、有界收集与私有 spill 文件、可执行文件查找、前台/会话检查,以及终止每个受管进程并等待其退出的 dispose。`terminate()` 拥有面向进程树的 TERM→宽限→KILL,`waitForExit()` 观察进程树存活性,可注入的 `taskkill /T` 覆盖 Windows。普通与终端 spawn 都先应用 seam 对 `KEY`/`PASSWORD`/`SECRET`/`TOKEN` 不区分大小写的清除,再合并显式 env。该实现没有配置;每项限制都随 spec 到达,Bash 与 PTY 的呈现环境覆盖仍归各自消费方所有。 - **`dsh-bash-local`(消费方)**——`inject: ['subprocess']`;把每个解析后的 `BashExecSpec` 映射为一个 `SubprocessSpawnSpec`(`['bash', '-c', command]`),并保留自身配置、`resolve()` 默认值补全、基于融合 deadline 的 `timedOut`/`aborted` 分类、带 `[stderr]` 标记的后台读取合并及其消费游标,以及 `onProcessDone` 子类钩子。`dsh-bash-sandbox` 除了重新声明继承来的 inject 之外没有变化;它仍在命令字符串层面做包装,并重新进入继承的 spawn 路径。 - **`dsh-bash`(seam)**——把迁走的词汇从 `dsh-subprocess` 重导出,因此没有任何 bash 消费方需要改动导入;`BashExecRequest`/`BashExecSpec`/`BashProcess` 与沙箱事实仍归 bash 所有。 @@ -21,13 +21,13 @@ Status: implemented 后台进程的存续期从执行器移到了管理器:执行器不再保有存活进程集合,于是重载执行器后,后台工作会继续运行且仍可读取,而组合拆除(管理器的 dispose)仍是先终止再等待退出的边界。一条行为 seam 随之挪动:后台 spawn 失败不再能在管道内部被缓冲成伪造的 stderr(对一个从未真正运行的进程,管理器会 reject `done`,且不缓冲任何内容),因此执行器把 `spawn failed: …` 提示注入恰好一个 `readOutput()` 增量。 -基于已观察到的流与生命周期需求,具备条件的进程消费方随后迁到该 seam:LSP 使用管道化协议流加收集式 stderr 尾部;ACP(Agent Client Protocol)后端使用管道化 ndjson、继承式 stderr 和消费方拥有的 stdin-EOF dispose 阶梯;PTY 使用 `spawnTerminal()`,同时保留就绪与终端策略;Code Runtime 使用普通原始管道。`dsh-subagent-subprocess` 与 LSP 私有进程树辅助函数均被删除。MCP 传输 spawn、SDK 包管理器运行器、同步 TUI Git 探测和刻意保持轻依赖的 test-support 启动器因所有权或执行形状仍留在外部;适用的生产调用方共享凭据清除。 +基于已观察到的流与生命周期需求,具备条件的进程消费方随后迁到该 seam:LSP 使用管道化协议流加收集式 stderr 尾部;ACP(Agent Client Protocol)后端使用管道化 ndjson、继承式 stderr 和消费方拥有的 stdin-EOF dispose 阶梯;PTY 使用 `spawnTerminal()`,同时保留就绪与终端策略。`dsh-subagent-subprocess` 与 LSP 私有进程树辅助函数均被删除。MCP 传输 spawn、SDK 包管理器运行器、同步 TUI Git 探测和刻意保持轻依赖的 test-support 启动器因所有权或执行形状仍留在外部;适用的生产调用方共享凭据清除。 ## 曾考虑的替代方案 **把进程管道留在 `dsh-bash-local` 里(维持现状)。**否决的理由与[任务注册表拆分](2026-07-26-task-registry-seam.md)得以落地的理由相同:这条边界既稳定,也早已记录在代码里(`run.ts` 的模块文档曾写明「this layer reacts to an abort signal; the executor owns deadlines and classifies causes」),而若继续将它保持私有,未来每个非 shell 运行器就只能要么 fork 这套机制,要么为非 bash 工作去依赖一个以 bash 命名的包。这组堆叠变更对用户可见的动因正是这一拆分。 -**保留最初只支持批量的接口,让流式消费方继续各自实现。**否决:已观察到的 LSP、ACP、PTY 与 Code Runtime 形状表明,这会继续保留重复的私有进程树信号与环境清除。Node 形状的处置方式覆盖这些消费方,又不缓冲管道化流。 +**保留最初只支持批量的接口,让流式消费方继续各自实现。**否决:已观察到的 LSP、ACP 与 PTY 形状表明,这会继续保留重复的私有进程树信号与环境清除。Node 形状的处置方式覆盖这些消费方,又不缓冲管道化流。 **用单个 `stdio: 'pipe' | 'inherit' | 'collect'` 模式统一全部流。**否决:真实消费方按流混用模式——LSP 使用 pipe/pipe/collect,ACP 使用 pipe/pipe/inherit,Bash 使用 data/collect/collect。 @@ -39,6 +39,6 @@ Status: implemented ## 后果 -换来的是:「运行并管理一个进程」成为 Bash、LSP、PTY、Code Runtime 与 ACP 消费方共用的可替换能力;容器化或远程进程后端可以直接接入,而无需改变各领域语义;进程树信号、升级终止、有界收集、终端机制与凭据清除各自只剩一份实现;后台进程也能在执行器重载后存活,与任务注册表的存续期模型一致。进程与终端管道通过 `dsh-subprocess-local` 测试;消费方测试套件只需针对真实服务固定各自拥有的行为。 +换来的是:「运行并管理一个进程」成为 Bash、LSP、PTY 与 ACP 消费方共用的可替换能力;容器化或远程进程后端可以直接接入,而无需改变各领域语义;进程树信号、升级终止、有界收集、终端机制与凭据清除各自只剩一份实现;后台进程也能在执行器重载后存活,与任务注册表的存续期模型一致。进程与终端管道通过 `dsh-subprocess-local` 测试;消费方测试套件只需针对真实服务固定各自拥有的行为。 -代价是:多出一对包,而且凡加载消费方之处都多一行组合配置;缺少 subprocess 提供方时,消费方会按标准服务注入行为保持挂起。每个后端都要实现执行环境坐标、可执行文件查找、三种 stdio 模式、进程树生命周期和一个终端原语。迁移词汇的重导出让 `dsh-bash` 的导入继续可用,但也意味着两个包命名同一批类型;进程 seam 是所有者。spawn 失败提示经由 Bash 的消费式读取游标变为单次交付,不再是可重复读取的 stderr 缓冲内容。 +代价是:多出一对包,而且凡加载消费方之处都多一行组合配置;缺少 subprocess 提供方时,消费方会按标准服务注入行为保持挂起。每个后端都要实现可执行文件查找、三种 stdio 模式、进程树生命周期和一个终端原语。迁移词汇的重导出让 `dsh-bash` 的导入继续可用,但也意味着两个包命名同一批类型;进程 seam 是所有者。spawn 失败提示经由 Bash 的消费式读取游标变为单次交付,不再是可重复读取的 stderr 缓冲内容。 diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml index 62592b5be0..06fd47f329 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md -2026-07-28-portable-execution-world-consumers.md: 97c8bcfc7bf84db4b6a2862fcc9d991ee7f4a7f6 -2026-07-28-portable-execution-world-consumers.zh.md: 025393f9e5ae99feba41fa8badeef8276151ba6f +2026-07-28-portable-execution-world-consumers.md: 402aa580255a5bd6aa0d046e3bcc16f712da520d +2026-07-28-portable-execution-world-consumers.zh.md: 26d493a79f8adb9e729ff69b0673f2f2775868b0 diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md index 97c8bcfc7b..402aa58025 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md @@ -6,7 +6,7 @@ English | [中文](2026-07-28-portable-execution-world-consumers.zh.md) ## Problem -The filesystem and subprocess seams made file and ordinary process access replaceable, but several higher capabilities still reached host Node APIs directly. A remote execution provider therefore appeared to need separate PTY, LSP, and Code Runtime packages even though their domain behavior did not change. Those packages would be shallow adapters: each would duplicate an existing consumer merely to replace its file and process operations. +The filesystem and subprocess seams made file and ordinary process access replaceable, but PTY and LSP still reached host Node APIs directly. A remote execution provider therefore appeared to need separate PTY and LSP packages even though their domain behavior did not change. Those packages would be shallow adapters: each would duplicate an existing consumer merely to replace its file and process operations. Ordinary pipes do not cover one requirement. A persistent terminal needs PTY allocation, foreground-process-group inspection and signalling, and cleanup of the complete terminal session. Pretending those operations can be rebuilt in `dsh-pty-local` from an ordinary `spawn()` handle would either leak provider internals or weaken its lifecycle contract. @@ -16,20 +16,17 @@ Ordinary pipes do not cover one requirement. A persistent terminal needs PTY all The filesystem interface owns the path facts that another capability needs without exposing its opaque target identity: a canonical process path, canonical `file:` URI, and containment. Existing whole and streaming text operations remain filesystem-owned; protocol consumers enforce their own retention limits while consuming the stream. -The subprocess interface owns the process coordinates and primitives: canonical cwd, executable lookup, ordinary raw or collected process spawning, and `spawnTerminal()`. The terminal operation is one deep primitive whose handle owns text I/O, foreground groups, signalling, and one awaited TERM-to-KILL operation that settles in-flight handle calls and reaches quiescence for every session member the provider can still observe. Its signal cancels allocation only; the published handle owns its lifetime. Prompt detection, idle inference, scrollback, sandbox policy, and owner lifecycle remain in the PTY consumer. +The subprocess interface owns executable lookup and process primitives: ordinary raw or collected process spawning and `spawnTerminal()`. The terminal operation is one deep primitive whose handle owns text I/O, foreground groups, signalling, and one awaited TERM-to-KILL operation that settles in-flight handle calls and reaches quiescence for every session member the provider can still observe. Its signal cancels allocation only; the published handle owns its lifetime. Prompt detection, idle inference, scrollback, sandbox policy, and owner lifecycle remain in the PTY consumer. Generic consumers use that execution world: - `dsh-bash-local` continues to map Bash semantics onto ordinary `ctx.subprocess.spawn()`. - `dsh-lsp-local` reads and contains source through `ctx.fs`, resolves and launches language servers through `ctx.subprocess`, and carries provider-owned file URIs through initialization and result rendering. One provider-lifetime signal aborts filesystem and protocol work during disposal, including workspace lookup before queue ownership; its JSON-RPC, pooling, synchronization, and normalization stay unchanged. - `dsh-pty-local` maps persistent-shell semantics onto `ctx.subprocess.spawnTerminal()`. The local `node-pty` and process-inspection implementation moves into `dsh-subprocess-local`; another subprocess provider supplies the same primitive. `danger-full-access` needs no `ctx.sandbox`; a confined mode requires a same-world sandbox provider and fails before spawn when none is mounted. Prompt and silence evidence collected during asynchronous pre-write inspection is discarded when the provider write begins. Cancellation retains the send reservation while an in-flight write settles and then signals the foreground group, so late bytes or the signal cannot target a successor; an in-flight readiness poll cannot release that reservation, and a rejected write sends no signal. The absolute deadline remains armed throughout cancellation. A signal failure becomes terminal transport failure. Completion of a stale inspection resumes polling for the current send. Startup cancellation begins terminal rollback without waiting for a stalled readiness or signalling call. Close rejects new public signals and delegates provider-observable session quiescence to the handle's awaited termination operation. -- `dsh-code-runtime-subprocess` passes a bundled dependency-free eval runner directly to `ctx.subprocess`, preserving the Code Runtime binding and output contract across local or remote process worlds without a filesystem dependency or provider-specific package. It shares host-side worker mechanics through the non-plugin `dsh-code-runtime-worker/runtime-host` subpath instead of copying them. The heap-bounded worker rejects oversized binding frames before transfer, each outer hop enforces the same bound before forwarding, and raw subprocess pipes carry newline-delimited UTF-8 JSON without a redundant base64 representation. The launcher publishes an accepted terminal frame before reaping its controller so a descendant that inherits controller pipes cannot suppress completion; the host still awaits process-group quiescence. - -`dsh-code-runtime-worker` remains a separate implementation. It is the smaller in-process backend and works in single-file distributions that cannot assume an installed Node executable. Remote filesystem/process compositions select `dsh-code-runtime-subprocess`; they do not need a provider-specific Code Runtime package. ## Alternatives considered -**Keep one PTY, LSP, and Code Runtime package per remote provider.** Rejected because provider mechanics would be repeated above the existing seams. The deletion test exposes the problem: deleting those adapters should not scatter domain behavior into the remote provider; the generic consumers already own it. +**Keep one PTY and LSP package per remote provider.** Rejected because provider mechanics would be repeated above the existing seams. The deletion test exposes the problem: deleting those adapters should not scatter domain behavior into the remote provider; the generic consumers already own it. **Model a terminal as an ordinary piped subprocess.** Rejected because pipes cannot allocate a controlling terminal, resolve the current foreground process group, or prove complete terminal-session cleanup. One terminal primitive is smaller and more honest than exposing substrate-specific escape hatches. @@ -39,13 +36,11 @@ Generic consumers use that execution world: **Add a stable bounded-read primitive to the filesystem seam.** Rejected because only LSP needs a complete-document byte ceiling, which it can enforce while consuming the existing text stream. A second primitive forces every provider to implement stable-handle and no-follow mechanics, including a remote helper protocol, without an observed concurrent-replacement defect. -**Delete the worker-thread Code Runtime.** Rejected because portability does not erase its current deployment need. The subprocess backend requires a Node executable; the worker backend does not and remains the supported single-process path. - **Run the whole harness inside the remote environment.** Rejected as a different deployment model. Making execution capabilities portable does not move model calls, session state, plugin state, or the agent loop. ## Consequences -A remote execution provider implements only its shared sandbox owner plus filesystem and subprocess adapters. Bash, PTY, LSP, and subprocess Code Runtime compose above them, so fixes to those capabilities remain provider-neutral. +A remote execution provider implements only its shared sandbox owner plus filesystem and subprocess adapters. Bash, PTY, and LSP compose above them, so fixes to those capabilities remain provider-neutral. The fundamental interfaces are wider, and a filesystem/subprocess pair must agree on one execution world. The added operations are limited to facts and lifecycle mechanics that current generic consumers require; model schemas, protocol framing, readiness policy, and presentation do not leak into the providers. diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md index 025393f9e5..26d493a79f 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md @@ -6,7 +6,7 @@ Status: implemented ## 问题 -文件系统与进程管理 seam 使文件访问和普通进程访问具备可替换性,但若干上层能力仍直接调用宿主 Node API。因此,即使领域行为没有变化,远程执行提供方看起来仍需要独立的 PTY、LSP 与代码运行时包(package)。这些包只会成为浅层适配器:每个包都仅为替换文件与进程操作而复制一个现有消费方。 +文件系统与进程管理 seam 使文件访问和普通进程访问具备可替换性,但 PTY 和 LSP 仍直接调用宿主 Node API。因此,即使领域行为没有变化,远程执行提供方看起来仍需要独立的 PTY 与 LSP 包(package)。这些包只会成为浅层适配器:每个包都仅为替换文件与进程操作而复制一个现有消费方。 普通管道无法满足其中一项要求。持久终端需要分配 PTY、检查前台进程组并发送信号,以及清理完整的终端会话。如果假设可以在 `dsh-pty-local` 中基于普通 `spawn()` 句柄重建这些操作,最终不是泄漏提供方内部细节,就是削弱其生命周期契约。 @@ -16,20 +16,16 @@ Status: implemented 文件系统接口负责其他能力需要的路径事实,同时不公开其不透明目标身份:规范化进程路径、规范化 `file:` URI 和包含关系。现有完整文本与流式文本操作仍归文件系统负责;协议消费方在消费流时执行各自的保留上限。 -进程管理接口负责进程运行坐标与原语:规范化 cwd、可执行文件查找、以原始或收集模式 spawn 普通进程,以及 `spawnTerminal()`。终端操作是一项深层原语,其句柄负责文本 I/O、前台进程组、信号发送,以及一项须等待的 TERM→KILL 操作;该操作会结算所有在途句柄调用,并使提供方仍可观察到的每个会话成员完全停稳。其信号只取消分配;句柄一经发布,便负责自身生命周期。提示符检测、空闲推断、scrollback、沙箱策略和所有者生命周期仍由 PTY 消费方负责。 +进程管理接口负责可执行文件查找与进程原语:以原始或收集模式 spawn 普通进程,以及 `spawnTerminal()`。终端操作是一项深层原语,其句柄负责文本 I/O、前台进程组、信号发送,以及一项须等待的 TERM→KILL 操作;该操作会结算所有在途句柄调用,并使提供方仍可观察到的每个会话成员完全停稳。其信号只取消分配;句柄一经发布,便负责自身生命周期。提示符检测、空闲推断、scrollback、沙箱策略和所有者生命周期仍由 PTY 消费方负责。 通用消费方使用该执行世界: - `dsh-bash-local` 继续把 Bash 语义映射到普通的 `ctx.subprocess.spawn()`。 - `dsh-lsp-local` 通过 `ctx.fs` 读取源文件并验证包含关系,通过 `ctx.subprocess` 解析和启动语言服务器,并让由提供方负责的文件 URI 贯穿初始化与结果渲染。一个提供方生命周期信号会在资源释放期间中止文件系统与协议操作,包括取得队列所有权之前的工作区查找;其 JSON-RPC、池化、同步和规范化保持不变。 - `dsh-pty-local` 把持久 shell 语义映射到 `ctx.subprocess.spawnTerminal()`。本地 `node-pty` 与进程检查实现移入 `dsh-subprocess-local`;其他进程管理提供方则提供相同原语。`danger-full-access` 不需要 `ctx.sandbox`;受限模式要求同一执行世界中存在沙箱提供方,未挂载时会在 spawn 前失败。提供方开始写入时,系统会丢弃异步写入前检查期间收集的提示符与静默证据。取消会在在途写入结算期间保留发送预留,随后向前台进程组发送信号,因此延迟字节和该信号都无法落到后续发送;在途就绪检查无法释放该预留,写入被拒绝时也不会发送信号。绝对截止时间会在整个取消期间保持启用。信号发送失败会成为终结性传输失败。陈旧检查完成后,会针对当前发送恢复轮询。启动取消会立即开始终端回滚,而不等待停滞的就绪检查或信号发送调用。关闭操作会拒绝新的公开信号,并把提供方可观察会话成员的完全停稳委托给句柄上须等待的终止操作。 -- `dsh-code-runtime-subprocess` 将一个内置的无依赖 eval runner 直接传给 `ctx.subprocess`,从而在本地或远程进程执行环境中保留代码运行时的绑定与输出契约,且不依赖文件系统或提供方专用包。它通过非插件子路径 `dsh-code-runtime-worker/runtime-host` 共享宿主侧 worker 机制,而不是复制这些机制。受堆上限约束的 worker 会在传输前拒绝过大的绑定帧;每个外层转发环节都会在转发前执行相同的上限检查;原始子进程管道承载以换行符分隔的 UTF-8 JSON,无需冗余的 base64 表示;launcher 会在回收 controller 前发布已接纳的终态帧,使继承 controller 管道的后代进程无法阻止完成;宿主仍会等待进程组完全停稳。 - -`dsh-code-runtime-worker` 仍是独立实现。它是较小的进程内后端,可用于无法假定已安装 Node 可执行文件的单文件分发。远程文件系统/进程组合选择 `dsh-code-runtime-subprocess`;它们不需要提供方专用的代码运行时包。 - ## 考虑过的替代方案 -**为每个远程提供方分别保留 PTY、LSP 和代码运行时包。** 不予采纳,因为这会在现有 seam 之上重复实现提供方机制。删除检验揭示了这一问题:删除这些适配器不应使领域行为散落到远程提供方中;通用消费方本已负责这些行为。 +**为每个远程提供方分别保留 PTY 与 LSP 包。** 不予采纳,因为这会在现有 seam 之上重复实现提供方机制。删除检验揭示了这一问题:删除这些适配器不应使领域行为散落到远程提供方中;通用消费方本已负责这些行为。 **把终端建模为普通的管道子进程。** 不予采纳,因为管道无法分配控制终端、确定当前前台进程组或证明完整终端会话已清理。一项终端原语比公开特定于执行基底的逃生口更小,也更能如实表达契约。 @@ -39,13 +35,11 @@ Status: implemented **在文件系统 seam 中新增稳定的有界读取原语。** 不予采纳,因为只有 LSP 需要完整文档字节上限,而它可以在消费现有文本流时执行该上限。第二项原语会迫使每个提供方实现稳定句柄和不跟随符号链接的机制,远程提供方甚至需要辅助协议,却没有已观察到的并发替换缺陷。 -**删除 worker 线程代码运行时。** 不予采纳,因为可移植性不会消除其当前部署需求。进程管理后端需要 Node 可执行文件,而 worker 后端不需要,并且仍是受支持的单进程路径。 - **在远程环境中运行整个 harness。** 不予采纳,因为这是另一种部署模型。让执行能力可移植,并不意味着移动模型调用、会话状态、插件状态或 agent loop(智能体循环)。 ## 后果 -远程执行提供方只需实现共享沙箱所有者,以及文件系统与进程管理适配器。Bash、PTY、LSP 和基于进程管理的代码运行时组合在这些适配器之上,因此对这些能力的修复仍与提供方无关。 +远程执行提供方只需实现共享沙箱所有者,以及文件系统与进程管理适配器。Bash、PTY 与 LSP 组合在这些适配器之上,因此对这些能力的修复仍与提供方无关。 基础接口更宽,一对文件系统/进程管理提供方必须在同一个执行世界上保持一致。新增操作仅限当前通用消费方所需的事实与生命周期机制;模型 schema、协议分帧、就绪策略和呈现不会渗入提供方。 diff --git a/.agents/notes/implemented/feature/2026-06-15-code-mode.i18n.yaml b/.agents/notes/implemented/feature/2026-06-15-code-mode.i18n.yaml index d2655b6d02..8773a797e9 100644 --- a/.agents/notes/implemented/feature/2026-06-15-code-mode.i18n.yaml +++ b/.agents/notes/implemented/feature/2026-06-15-code-mode.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-06-15-code-mode.md -2026-06-15-code-mode.md: a075ea66c150afb1b48f98c600b72eec57869962 -2026-06-15-code-mode.zh.md: c269c0a8709087e7b1e27051cbda3924f8dd9109 +2026-06-15-code-mode.md: b6a24ecd9700e32912b8112b59cbd8b6ab131eb5 +2026-06-15-code-mode.zh.md: 4d0a4cf8fa31cf9d9954e5bd95f823dfc0668444 diff --git a/.agents/notes/implemented/feature/2026-06-15-code-mode.md b/.agents/notes/implemented/feature/2026-06-15-code-mode.md index a075ea66c1..b6a24ecd97 100644 --- a/.agents/notes/implemented/feature/2026-06-15-code-mode.md +++ b/.agents/notes/implemented/feature/2026-06-15-code-mode.md @@ -20,7 +20,7 @@ Three decisions, each elaborated in its own section below: 1. **Code Mode is a first-class presentation mode of `ToolRegistry`** (`dsh-tools`), selected by a validated `mode` config: `'native'` (the default, contributing the visible capability schemas), `'code'` (the registry contributes only its reserved `run_code` transport plus a generated SDK `.d.ts` in the system prompt), or `'both'` (native schemas and the transport + SDK). The registry shapes its canonical contribution at the source; the cooperative prompt-assembly result remains authoritative, and the logged request header records exactly that returned presentation. 2. **Code execution is a capability seam** — `packages/code-runtime/` contains the interface package `@deepseek-ai/dsh-code-runtime`, which owns `ctx.codeRuntime` ([capability seams](../architecture/2026-06-13-capability-seams.md); consumer = `dsh-tools`, with core-consumes-a-seam precedent in `agent-loop` → `dsh-llm`). The runtime knows nothing about tools: it is handed a program and named async bindings, runs the program, and reports `{ value, logs, error? }`. Language and substrate are backend properties, so a future Python or container backend is another implementation package, not a redesign. -3. **Two implementations preserve one fresh-worker contract**: `@deepseek-ai/dsh-code-runtime-worker` runs the worker in the harness process, while `@deepseek-ai/dsh-code-runtime-subprocess` passes a bundled eval runner to `ctx.subprocess` for another execution world. Both execute host-stripped TypeScript in a fresh Node worker with an empty environment, bridged bindings, configurable heap/output/time caps, and hard termination. Their trust posture is bash-equivalent by design; stronger isolation comes from the mounted execution world. +3. **The shipped implementation is `@deepseek-ai/dsh-code-runtime-worker`**: one fresh Node worker thread per run, executing the model's TypeScript after type-strip, with bindings bridged over the message port, an empty environment, configurable heap/output/time caps, and hard termination. Its trust posture is bash-equivalent by design — no unsafe-acknowledgement flags — because the harness already ships `dsh-bash-local`, which executes arbitrary model-written shell commands with strictly *more* ambient authority. This note owns Code Mode's presentation, composition, isolation, and settlement foundation. The later [typed tool-return Agent Note](2026-07-20-code-mode-typed-tool-returns.md) owns the generated output map, canonical binding values, `ToolCallError`, and the lossless outer-output boundary. @@ -64,7 +64,7 @@ Each sub-dispatch appends a log-only `tool/code-dispatch-start` event at pool en - `CodeBindingNamespace = { global: string; functions: Record Promise>; errorClass?: { name: string; memberNameProperty: string } }` — the runtime exposes each namespace as a global object of async functions inside the program; the optional descriptor asks the runtime to inject a real program-visible rejection class without teaching the seam consumer-specific names. `CodeJsonValue` is this dependency-light seam's structural lossless-JSON type, so binding arguments and resolutions cross the implementation's serialization boundary whole. - `CodeRunResult = { value?: CodeJsonValue; logs: string[]; error?: CodeRunFailure }` — program execution outcomes resolve as the `error` field. `run()` may reject only for caller/seam misuse (for example a duplicate binding namespace); consumers still contain a non-conforming backend rejection at their own error boundary. - `CodeRunFailure = { kind: 'exception' | 'timeout' | 'abort' | 'worker-exit' | 'invalid-output' | 'output-limit'; message: string }` — orthogonal outcomes reported independently per [defensive patterns](../../../../docs/defensive-patterns.md); a timed-out run is not an exception, an abort is not a timeout, a lossy completion is not an overflow, and a substrate exit is none of them. -- Two readonly backend descriptors, informational not gating: `language` (what the program must be written in — `'typescript'` for both shipped backends; a Python backend would pair with its own SDK generator) and `isolation` (`'worker-thread'` for both shipped backends; the subprocess provider may add a container boundary). `dsh-tools` requires `language === 'typescript'` in the MVP — its codegen emits TS — and fails the assembly loudly otherwise, the same misconfiguration idiom as `toolOrder` violations (as when `mode` is non-native with no `ctx.codeRuntime` loaded at all). +- Two readonly backend descriptors, informational not gating: `language` (what the program must be written in — `'typescript'` for the shipped backend; a Python backend would say so, and pair with its own SDK generator on the presentation side) and `isolation` (`'worker-thread'` for the shipped backend; `'process'`, `'container'`, … for future ones). `dsh-tools` requires `language === 'typescript'` in the MVP — its codegen emits TS — and fails the assembly loudly otherwise, the same misconfiguration idiom as `toolOrder` violations (as when `mode` is non-native with no `ctx.codeRuntime` loaded at all). Requests contain every runtime input; implementations own validated timeout and cap defaults. The registry looks up the optional runtime only when Code Mode is assembled, so native mode does not depend on one. Missing or language-incompatible runtimes fail loudly. Alternate substrates or languages can replace the implementation behind the same seam, paired with the appropriate SDK generator. @@ -79,11 +79,9 @@ Requests contain every runtime input; implementations own validated timeout and 5. **Enforce independent budgets.** `computeMs` meters worker busy time, allowing slow awaited tools without excusing a hot loop. `maxWallMs` bounds total elapsed time, including unresolved waits. `maxOutputBytes` bounds only the combined serialized outer logs, completion, or diagnostic; intermediate binding values have no byte cap. Expiry, cancellation, and completion terminate the worker, and heap exits or outer overflow are explicit failures. 6. **Dispose to quiescence**: the service's own disposal terminates in-flight workers and *awaits* their exits before resolving, per [defensive patterns](../../../../docs/defensive-patterns.md). -`@deepseek-ai/dsh-code-runtime-subprocess` preserves those program, binding, output, and worker-budget semantics across another process world. It passes a dependency-free eval runner directly through the subprocess provider and carries binding traffic over bounded newline-delimited UTF-8 JSON frames on raw pipes. The heap-bounded worker rejects expanded completion wires before MessagePort transfer; terminal settlement asks the launcher to reap its controller and keeps process-group escalation armed until `waitForExit()` confirms whole-tree quiescence. The [portable execution-world decision](../architecture/2026-07-28-portable-execution-world-consumers.md) owns why this generic backend replaces provider-specific Code Runtime packages; `dsh-code-runtime-worker` remains the single-process and single-file-distribution path. - ### Trust posture -The runtimes provide containment, not an independent security boundary: model code can reach Node APIs and has the authority of its mounted execution world. Code Mode uses the same `tools/pre-execute` policy gate as bash and adds an empty worker environment, heap limits, a separate isolate, and hard termination of the program itself. Deployments that need a hard multi-tenant boundary mount container-class filesystem/subprocess providers for both code and bash. +The worker runtime provides containment, not a security boundary: model code can reach Node APIs and has authority comparable to the bash tool. `worker.terminate()` stops the thread but not OS processes it spawned. Code Mode uses the same `tools/pre-execute` policy gate as bash and adds an empty environment, heap limits, a separate isolate, and hard termination of the program itself. Deployments that need a hard multi-tenant boundary need a container-class backend for both code and bash; the runtime's isolation descriptor lets them distinguish that backend. ### What the model sees @@ -95,7 +93,7 @@ Deployments switching to `'code'` must update any native-only `toolOrder`. Assem ## Testing -- **Runtime implementations:** Real-worker suites cover typed binding values and failures, every lossless JSON completion root, invalid and over-limit output, exact combined ledger and per-hop frame boundaries, compute and wall budgets, hostile binding traffic, empty environment, descendant lifetime cleanup, and disposal to quiescence. Built-package tests run both the direct worker entry and the subprocess composition under plain Node; the latter also has a Loader-driven `cordis.yml` test. +- **Worker runtime:** Real-worker tests cover typed binding values and failures, every lossless JSON completion root, invalid and over-limit output, exact combined ledger boundaries, compute and wall budgets, hostile binding traffic, empty environment, and disposal to quiescence. A built-package test runs the worker entry under plain Node. - **Registry integration:** Tests cover code generation, all presentation modes, reserved-name and restriction rules, scoped visibility, authoritative assembly rewrites, `toolOrder`, runtime compatibility failures, full-pipeline sub-dispatch, parent-token correlation, serialization, cancellation and queue drain, JSON normalization, error propagation, log events, ordered context deferral across successful and failed programs, outer-block suppression, and HMR cleanup. - **With-key e2e:** A real model composes two bash calls in one program; another discovers nested workspace instructions through a Code Mode fs dispatch. The tests verify collapsed request headers, correlated dispatch events, resulting files, deferred context, and model behavior. - **Snapshot:** The `code-mode-turn`, `both-mode-turn`, and `code-mode-workspace-context` fixtures pin SDK text, header tool lists, dispatch events, deferred context, and result cards. @@ -120,7 +118,7 @@ Deployments switching to `'code'` must update any native-only `toolOrder`. Assem ## Risks -**A worker is not a hard security boundary.** Deliberate and documented (§Trust posture): posture equals the existing bash tool and gating uses the same seams. The subprocess implementation can run inside a container-class execution world, but its worker descriptor does not itself claim that boundary. +**The worker is not a hard security boundary.** Deliberate and documented (§Trust posture): posture equals the existing bash tool, containment exceeds it, gating uses the same seams. Deployments needing more need a future `isolation: 'container'` backend — tracked as the seam's designed extension, not a TODO on this design. **`stripTypeScriptTypes` is marked experimental.** It is the same engine (amaro/swc) behind Node's own native `.ts` execution, exposed as an API across this repo's whole engines range. Mitigations: the runtime's unit suite pins the behaviors relied on (position preservation, erasable-only rejection message shape loosely), the call sits behind one private function, and `amaro`/`sucrase` are drop-in replacements if the API shifts. The erasable-only subset is a model-facing contract line, and the error path is a working feedback loop, not a dead end. @@ -128,7 +126,7 @@ Deployments switching to `'code'` must update any native-only `toolOrder`. Assem **Registry scope growth.** `dsh-tools` absorbs codegen, a tool, a bridge, and an event. Contained by module boundaries inside the package (`ts-types.ts`, `code-mode.ts` beside `schema.ts`/`json-schema.ts`/`presentation.ts`) and by the seam: everything substrate-shaped lives behind `ctx.codeRuntime`. -**Large lossless JSON values can exhaust memory.** Tool bindings snapshot lossless JSON before dispatch and return canonical JSON resolutions whole. The direct worker runtime applies no per-binding byte cap; the subprocess runtime bounds each transport frame with `maxFrameBytes`, but repeated or concurrent calls can still consume process or worker memory. The combined outer-output ledger for logs, the completion value, and a failure diagnostic is separately bounded by `maxOutputBytes`. +**Large lossless JSON values can exhaust memory.** Tool bindings snapshot lossless JSON before dispatch and return canonical JSON resolutions whole. The runtime validates both sides of the worker port and applies no per-binding byte cap; structured-clone cost and process or worker memory are the practical bounds. The combined outer-output ledger for logs, the completion value, and a failure diagnostic is the only byte-capped boundary. **Serialized-only sub-dispatch.** `Promise.all` gains no wall-clock parallelism yet, only fewer round-trips; models may over-expect. The instructions state it; lifting it is tied to the same concurrency-safety metadata the native parallel-dispatch TODO needs. diff --git a/.agents/notes/implemented/feature/2026-06-15-code-mode.zh.md b/.agents/notes/implemented/feature/2026-06-15-code-mode.zh.md index c269c0a870..4d0a4cf8fa 100644 --- a/.agents/notes/implemented/feature/2026-06-15-code-mode.zh.md +++ b/.agents/notes/implemented/feature/2026-06-15-code-mode.zh.md @@ -20,7 +20,7 @@ Cloudflare 的 [Code Mode](https://blog.cloudflare.com/code-mode/) 提出了一 1. **Code Mode 是 `ToolRegistry`(`dsh-tools`)的一等呈现模式**,通过经校验的 `mode` 配置选择:`'native'`(默认,贡献可见能力 schema)、`'code'`(注册表仅贡献其保留的 `run_code` 传输通道加一份生成的 SDK `.d.ts` 到系统提示词中)或 `'both'`(原生 schema 加传输通道 + SDK)。注册表在源头塑造其权威贡献;协作式提示词组装的结果仍具权威性,记录在日志中的请求头精确反映该返回的呈现。 2. **代码执行是一个能力 seam**——`packages/code-runtime/` 包含接口包 `@deepseek-ai/dsh-code-runtime`,拥有 `ctx.codeRuntime`([能力 seam](../architecture/2026-06-13-capability-seams.md);消费方 = `dsh-tools`,core 消费 seam 的先例见 `agent-loop` → `dsh-llm`)。运行时对工具一无所知:它接收一段程序和命名的异步绑定,执行程序,报告 `{ value, logs, error? }`。语言和基底是后端属性,因此未来的 Python 或容器后端只是另一个实现包,而非重新设计。 -3. **两种实现保持同一份全新 worker 契约**:`@deepseek-ai/dsh-code-runtime-worker` 在 harness 进程内运行 worker,`@deepseek-ai/dsh-code-runtime-subprocess` 则将一个内置的 eval runner 传给 `ctx.subprocess`,用于另一执行环境。二者都在具有空环境的全新 Node worker 内执行由宿主剥离类型的 TypeScript,并提供桥接绑定、可配置的堆/输出/时间上限和硬终止。其信任姿态在设计上等同于 bash;更强的隔离来自挂载的执行环境。 +3. **交付的实现是 `@deepseek-ai/dsh-code-runtime-worker`**:每次运行 spawn 一个全新的 Node worker 线程,对模型的 TypeScript 进行 type-strip 后执行,绑定通过消息端口桥接,环境为空,堆/输出/时间上限可配置,并支持硬终止。其信任姿态在设计上等同于 bash——无需 unsafe-acknowledgement flag——因为 harness 已经交付了 `dsh-bash-local`,后者以严格*更高*的环境权限执行模型编写的任意 shell 命令。 本说明负责定义 Code Mode 的呈现、组合、隔离与结算基础。后续的[类型化工具返回值 Agent Note](2026-07-20-code-mode-typed-tool-returns.md)负责定义生成的输出映射、规范绑定值、`ToolCallError` 和无损外层输出边界。 @@ -64,7 +64,7 @@ Cloudflare 的 [Code Mode](https://blog.cloudflare.com/code-mode/) 提出了一 - `CodeBindingNamespace = { global: string; functions: Record Promise>; errorClass?: { name: string; memberNameProperty: string } }`——运行时将每个命名空间作为程序内部的全局异步函数对象暴露;可选描述符要求运行时注入真正的、程序可见的 reject 类,而无需让 seam 获知消费方专用名称。`CodeJsonValue` 是这个低依赖 seam 的结构化无损 JSON 类型,因此绑定参数与解析值可以完整跨越实现的序列化边界。 - `CodeRunResult = { value?: CodeJsonValue; logs: string[]; error?: CodeRunFailure }`——程序执行失败时,执行 promise 仍会 fulfill,并通过 `error` 字段返回失败结果。只有调用方/seam 误用(例如重复的绑定命名空间)时,`run()` 才会 reject;消费方仍在自己的错误边界处理不合规后端的拒绝。 - `CodeRunFailure = { kind: 'exception' | 'timeout' | 'abort' | 'worker-exit' | 'invalid-output' | 'output-limit'; message: string }`——按[防御性模式](../../../../docs/defensive-patterns.md)独立报告的正交结果;超时的 run 不是异常,abort 不是超时,有损完成值不是溢出,基底退出也与上述情况相互独立。 -- 两个只读的后端描述符,仅供信息参考而非门禁判定:`language`(程序必须使用的语言——两种已交付后端均为 `'typescript'`;Python 后端会配对自己的 SDK 生成器)和 `isolation`(两种已交付后端均为 `'worker-thread'`;子进程提供方可以增加容器边界)。`dsh-tools` 在 MVP 中要求 `language === 'typescript'`——其代码生成输出 TS——否则组装会大声失败,与 `toolOrder` 违规时的配置错误惯用法相同(如 `mode` 为非 native 但根本没有加载 `ctx.codeRuntime`)。 +- 两个只读的后端描述符,仅供信息参考而非门禁判定:`language`(程序必须使用的语言——交付的后端为 `'typescript'`;Python 后端会声明自己,并在呈现侧配对自己的 SDK 生成器)和 `isolation`(交付的后端为 `'worker-thread'`;未来可为 `'process'`、`'container'` 等)。`dsh-tools` 在 MVP 中要求 `language === 'typescript'`——其代码生成输出 TS——否则组装会大声失败,与 `toolOrder` 违规时的配置错误惯用法相同(如 `mode` 为非 native 但根本没有加载 `ctx.codeRuntime`)。 请求包含所有运行时输入;实现方拥有经校验的超时和上限默认值。注册表仅在组装 Code Mode 时查找可选的运行时,因此 native 模式不依赖它。缺失或语言不兼容的运行时会大声失败。替代基底或语言可以在同一 seam 背后替换实现,配对相应的 SDK 生成器。 @@ -79,11 +79,9 @@ Cloudflare 的 [Code Mode](https://blog.cloudflare.com/code-mode/) 提出了一 5. **强制独立预算。** `computeMs` 计量 worker 忙碌时间,允许慢速的 awaited 工具而不放过热循环。`maxWallMs` 约束总经过时间,包括未解析的等待。`maxOutputBytes` 只约束序列化后的外层日志、完成值或诊断的组合;中间绑定值没有字节数上限。到期、取消和完成都终止 worker,堆退出或外层溢出会作为显式失败报告。 6. **dispose 至完全停稳**:服务自身的 dispose(资源释放)终止进行中的 worker 并*等待*其退出后再 resolve,遵循[防御性模式](../../../../docs/defensive-patterns.md)。 -`@deepseek-ai/dsh-code-runtime-subprocess` 在另一进程执行环境中保持相同的程序、绑定、输出与 worker 预算语义。它通过子进程提供方直接传递一个无依赖的 eval runner,并在原始管道上使用有界的、以换行符分隔的 UTF-8 JSON 帧承载绑定通信。受堆上限约束的 worker 会在通过 MessagePort 传输前拒绝展开后的完成值 wire;终态结算会请求 launcher 回收其 controller,并让进程组升级终止机制保持待命,直至 `waitForExit()` 确认整棵进程树完全停稳。[可移植执行环境决策](../architecture/2026-07-28-portable-execution-world-consumers.md)说明为何这个通用后端会取代提供方专用的 Code Runtime 包;`dsh-code-runtime-worker` 仍用于单进程和单文件发行版。 - ### 信任姿态 -这些运行时提供的是隔离,而非独立安全边界:模型代码可以访问 Node API,并拥有挂载的执行环境所授予的权限。Code Mode 使用与 bash 相同的 `tools/pre-execute` 策略门禁,并额外提供空 worker 环境、堆限制、独立 isolate 和对程序本身的硬终止。需要硬多租户边界的部署应当为代码和 bash 都挂载容器级文件系统/子进程提供方。 +worker 运行时提供的是隔离,而非安全边界:模型代码可以访问 Node API,权限与 bash 工具相当。`worker.terminate()` 停止线程但不停止它 spawn 的 OS 进程。Code Mode 使用与 bash 相同的 `tools/pre-execute` 策略门禁,并额外提供空环境、堆限制、独立 isolate 和对程序本身的硬终止。需要硬多租户边界的部署需要为代码和 bash 都使用容器级后端;运行时的 isolation 描述符让它们能区分该后端。 ### 模型看到的内容 @@ -95,7 +93,7 @@ SDK 指示模型编写一个异步的可擦除 TypeScript 函数体,通过 `aw ## 测试 -- **运行时实现:** 真实 worker 测试套件覆盖类型化的绑定值与失败、每一种无损 JSON 根类型的完成值、无效和超限输出、精确的组合账本边界与逐跳帧边界、compute 和 wall 预算、恶意绑定流量、空环境、后代进程生命周期清理以及 dispose 至完全停稳。构建后包测试会在纯 Node 下分别运行直接 worker 入口与子进程组合;后者另有一个由 Loader 驱动的 `cordis.yml` 测试。 +- **Worker 运行时:** 真实 worker 测试覆盖类型化的绑定值与失败、每一种无损 JSON 根类型的完成值、无效和超限输出、精确的组合账本边界、compute 和 wall 预算、恶意绑定流量、空环境以及 dispose 至完全停稳。一个构建后包测试在纯 Node 下运行 worker 入口。 - **注册表集成:** 测试覆盖代码生成、所有呈现模式、保留名称和限制规则、scoped 可见性、权威组装重写、`toolOrder`、运行时兼容性失败、完整流水线子分发、parent-token 关联、序列化、取消和队列排空、JSON 规范化、错误传播、日志事件、成功与失败程序中的有序上下文延后、外层阻止抑制以及 HMR(热模块替换)清理。 - **带密钥 e2e:** 真实模型在一个程序中组合两次 bash 调用;另一个模型通过 Code Mode fs 分发发现嵌套的工作区指令。测试验证折叠的请求头、关联的分发事件、结果文件、延后上下文和模型行为。 - **快照:** `code-mode-turn`、`both-mode-turn` 和 `code-mode-workspace-context` fixture(测试前置数据)固定 SDK 文本、请求头工具列表、分发事件、延后上下文和结果卡片。 @@ -120,7 +118,7 @@ SDK 指示模型编写一个异步的可擦除 TypeScript 函数体,通过 `aw ## 风险 -**worker 不是硬安全边界。** 有意为之且已文档化(§信任姿态):姿态等同于既有的 bash 工具,门禁使用相同的 seam。子进程实现可以在容器级执行环境中运行,但其 worker 描述符本身不声称具备该边界。 +**Worker 不是硬安全边界。** 有意为之且已文档化(§信任姿态):姿态等同于既有的 bash 工具,隔离程度超过它,门禁使用相同的 seam。需要更强隔离的部署需要未来的 `isolation: 'container'` 后端——作为 seam 设计的扩展点跟踪,而非本设计的 TODO。 **`stripTypeScriptTypes` 标记为 experimental。** 它与 Node 自身原生 `.ts` 执行背后的引擎(amaro/swc)相同,在本仓库的整个引擎范围内作为 API 暴露。缓解措施:运行时的单元测试套件固定了所依赖的行为(位置保持、可擦除限制的拒绝消息形状宽松匹配),调用位于一个私有函数之后,且 `amaro`/`sucrase` 是 API 变化时的直接替代品。仅可擦除子集是面向模型的契约线,错误路径是一个可工作的反馈循环,而非死胡同。 @@ -128,7 +126,7 @@ SDK 指示模型编写一个异步的可擦除 TypeScript 函数体,通过 `aw **注册表 scope 增长。** `dsh-tools` 吸收了代码生成、一个工具、一个桥和一个事件。通过包内的模块边界(`ts-types.ts`、`code-mode.ts` 与 `schema.ts`/`json-schema.ts`/`presentation.ts` 并列)和 seam 约束:所有基底相关的内容都在 `ctx.codeRuntime` 之后。 -**大型无损 JSON 值可能耗尽内存。** 工具绑定会在分发前对无损 JSON 创建快照,并完整返回规范 JSON 解析值。直接 worker 运行时不对单次绑定设置字节数上限;子进程运行时使用 `maxFrameBytes` 限制每个传输帧,但重复或并发调用仍可能消耗进程或 worker 内存。包含日志、完成值和失败诊断的组合外层输出账本另由 `maxOutputBytes` 限制。 +**大型无损 JSON 值可能耗尽内存。** 工具绑定会在分发前对无损 JSON 创建快照,并完整返回规范 JSON 解析值。运行时会校验 worker 端口两侧,但不对单次绑定设置字节数上限;结构化克隆成本以及进程或 worker 内存构成实际边界。只有包含日志、完成值和失败诊断的组合外层输出账本受字节数上限约束。 **仅序列化的子分发。** `Promise.all` 尚未获得挂钟并行性,仅减少往返次数;模型可能过度期望。说明中已声明;解除此限制与原生并行分发 TODO 所需的并发安全元数据绑定。 diff --git a/docs/capability-seams.md b/docs/capability-seams.md index 1aee225b5d..c5ec654d0f 100644 --- a/docs/capability-seams.md +++ b/docs/capability-seams.md @@ -21,6 +21,7 @@ flowchart LR pkg_session["session"] svc_sessions["ctx.sessions
In-memory session store"] pkg_agent["agent"] + pkg_cli_demo["cli-demo"] pkg_session_persistence["session-persistence"] pkg_session_query["session-query"] pkg_session_query_sqlite["session-query-sqlite"] @@ -39,13 +40,6 @@ flowchart LR pkg_tool_bash["tool-bash"] pkg_hooks_claude["hooks-claude"] pkg_hooks_codex["hooks-codex"] - pkg_settings["settings"] - svc_settings["ctx.settings
User-settings seam"] - pkg_settings_local["settings-local"] - pkg_apiproxy["apiproxy"] - pkg_credentials["credentials"] - svc_credentials["ctx.credentials
Credential seam"] - pkg_credentials_local["credentials-local"] pkg_session_telemetry["session-telemetry"] svc_telemetry["ctx.telemetry
Session telemetry seam"] pkg_session_telemetry_otel["session-telemetry-otel"] @@ -57,10 +51,12 @@ flowchart LR svc_storageDomain["ctx.storageDomain
Domain data facility"] pkg_workspace["workspace"] svc_workspace["ctx.workspace
Workspace entity registry"] + pkg_apiproxy["apiproxy"] svc_sessionQuery["ctx.sessionQuery
Session reads, traces, filters, and search"] pkg_session_reference["session-reference"] pkg_tool_session_query["tool-session-query"] svc_sessionReferences["ctx.sessionReferences
Cross-session snapshot preparation"] + pkg_tui["tui"] pkg_session_title["session-title"] svc_sessionTitle["ctx.sessionTitle
Log-backed session titles"] pkg_session_title_first_message_llm["session-title-first-message-llm"] @@ -88,12 +84,13 @@ flowchart LR pkg_host_apiproxy["host-apiproxy"] pkg_session_projection_cache["session-projection-cache"] svc_sessionProjectionCache["ctx.sessionProjectionCache
Persisted projection cache"] + svc_tui["ctx.tui
Mounted-terminal interaction service"] pkg_skill["skill"] svc_skills["ctx.skills
Skill provider registry"] - pkg_skill_badge["skill-badge"] pkg_skill_local["skill-local"] svc_agents["ctx.agents
Agent service"] pkg_acp["acp"] + pkg_tui_demo["tui-demo"] svc_agentLoop["ctx.agentLoop
Concrete loop driver"] pkg_agent_spine_demo["agent-spine-demo"] pkg_goal["goal"] @@ -103,19 +100,14 @@ flowchart LR pkg_subprocess_local["subprocess-local"] pkg_bash_local["bash-local"] pkg_bash_sandbox["bash-sandbox"] + pkg_pty_local["pty-local"] pkg_lsp_local["lsp-local"] pkg_subagent_acp["subagent-acp"] - pkg_subagent_codex["subagent-codex"] - pkg_subagent_claude_code["subagent-claude-code"] pkg_bash["bash"] svc_bash["ctx.bash
Bash executor seam"] - pkg_pwsh_local["pwsh-local"] - pkg_tool_pwsh["tool-pwsh"] - pkg_bash_env["bash-env"] svc_bashEnv["ctx.bashEnv
Managed bash environment registry"] pkg_pty["pty"] svc_pty["ctx.pty
Persistent PTY session registry"] - pkg_pty_local["pty-local"] pkg_sandbox["sandbox"] svc_sandbox["ctx.sandbox
Process-sandbox seam"] pkg_sandbox_local["sandbox-local"] @@ -136,11 +128,9 @@ flowchart LR pkg_compact["compact"] svc_compact["ctx.compact
Compaction seam"] pkg_subagent["subagent"] - svc_subagents["ctx.subagents
Subagent provider and continuation service"] + svc_subagents["ctx.subagents
Subagent provider registry"] pkg_subagent_spawn["subagent-spawn"] pkg_subagent_fork["subagent-fork"] - pkg_subagent_dsh_sdk["subagent-dsh-sdk"] - pkg_tool_subagent_control["tool-subagent-control"] pkg_tool_ralph["tool-ralph"] pkg_tasks["tasks"] svc_tasks["ctx.tasks
Background task registry"] @@ -176,7 +166,6 @@ flowchart LR pkg_api_gateway --> svc_typertGateway pkg_approval --> svc_approval pkg_bash --> svc_bash - pkg_bash_env --> svc_bashEnv pkg_bash_local --> svc_bash pkg_bash_sandbox --> svc_bash pkg_code_runtime --> svc_codeRuntime @@ -185,8 +174,6 @@ flowchart LR pkg_compact --> svc_compact pkg_compact_basic --> svc_compact pkg_compact_tool_result_prune --> svc_toolResultPrune - pkg_credentials --> svc_credentials - pkg_credentials_local --> svc_credentials pkg_directory_picker --> svc_directoryPicker pkg_directory_picker_browse --> svc_directoryPicker pkg_directory_picker_native --> svc_directoryPicker @@ -204,7 +191,6 @@ flowchart LR pkg_plan_mode --> svc_planMode pkg_pty --> svc_pty pkg_pty_local --> svc_pty - pkg_pwsh_local --> svc_bash pkg_sandbox --> svc_sandbox pkg_sandbox_local --> svc_sandbox pkg_sandbox_policy --> svc_sandboxPolicy @@ -222,10 +208,7 @@ flowchart LR pkg_session_title --> svc_sessionTitle pkg_session_title_all_messages_llm --> svc_sessionTitle pkg_session_title_first_message_llm --> svc_sessionTitle - pkg_settings --> svc_settings - pkg_settings_local --> svc_settings pkg_skill --> svc_skills - pkg_skill_badge --> svc_skills pkg_skill_local --> svc_skills pkg_spill --> svc_spillStore pkg_spill_local --> svc_spillStore @@ -235,9 +218,6 @@ flowchart LR pkg_storage_sqlite --> svc_storage pkg_subagent --> svc_subagents pkg_subagent_acp --> svc_subagents - pkg_subagent_claude_code --> svc_subagents - pkg_subagent_codex --> svc_subagents - pkg_subagent_dsh_sdk --> svc_subagents pkg_subagent_fork --> svc_subagents pkg_subagent_spawn --> svc_subagents pkg_subprocess --> svc_subprocess @@ -246,7 +226,10 @@ flowchart LR pkg_tasks --> svc_tasks pkg_tasks_local --> svc_tasks pkg_token_meter --> svc_tokenMeter + pkg_tool_bash --> svc_bashEnv pkg_tools --> svc_tools + pkg_tui --> svc_tui + pkg_tui --> svc_userInteraction pkg_typert_registry --> svc_typert pkg_user_interaction --> svc_userInteraction pkg_web --> svc_web @@ -261,21 +244,18 @@ flowchart LR svc_agentLoop --> pkg_agent_spine_demo svc_agents --> pkg_acp svc_agents --> pkg_agent_loop + svc_agents --> pkg_cli_demo svc_agents --> pkg_subagent_inprocess + svc_agents --> pkg_tui_demo svc_approval --> pkg_tool_bash svc_approval --> pkg_tools svc_bash --> pkg_hooks_claude svc_bash --> pkg_hooks_codex svc_bash --> pkg_tool_bash - svc_bash --> pkg_tool_pwsh - svc_bashEnv --> pkg_tool_bash - svc_bashEnv --> pkg_tool_pwsh svc_clientModuleHost --> pkg_hmr svc_codeRuntime --> pkg_tools + svc_commands --> pkg_tui svc_compact --> pkg_compact_basic - svc_credentials --> pkg_apiproxy - svc_credentials --> pkg_llm_deepseek - svc_credentials --> pkg_llm_pi_ai svc_directoryPicker --> pkg_apiproxy svc_fs --> pkg_tool_fs svc_httpServer --> pkg_connection @@ -305,29 +285,26 @@ flowchart LR svc_sessionProjections --> pkg_tool_todo svc_sessionQuery --> pkg_session_reference svc_sessionQuery --> pkg_tool_session_query + svc_sessionReferences --> pkg_tui svc_sessions --> pkg_agent svc_sessions --> pkg_agent_loop + svc_sessions --> pkg_cli_demo svc_sessions --> pkg_invariants svc_sessions --> pkg_session_persistence svc_sessions --> pkg_session_query svc_sessions --> pkg_session_query_sqlite svc_sessions --> pkg_subagent_inprocess - svc_settings --> pkg_apiproxy - svc_settings --> pkg_llm_deepseek - svc_settings --> pkg_llm_pi_ai svc_skills --> pkg_tool_skill svc_spillStore --> pkg_spill_policy svc_storage --> pkg_storage_domain svc_storageDomain --> pkg_workspace svc_subagents --> pkg_tool_ralph svc_subagents --> pkg_tool_subagent - svc_subagents --> pkg_tool_subagent_control svc_subprocess --> pkg_bash_local svc_subprocess --> pkg_bash_sandbox svc_subprocess --> pkg_lsp_local + svc_subprocess --> pkg_pty_local svc_subprocess --> pkg_subagent_acp - svc_subprocess --> pkg_subagent_claude_code - svc_subprocess --> pkg_subagent_codex svc_systemPrompt --> pkg_agent_loop svc_systemPrompt --> pkg_tool_fs svc_systemPrompt --> pkg_tool_pty @@ -352,6 +329,7 @@ flowchart LR svc_typert --> pkg_api_gateway svc_typert --> pkg_typert_loader svc_userInteraction --> pkg_tool_ask_user + svc_userInteraction --> pkg_tui svc_web --> pkg_tool_web svc_workflows --> pkg_tool_ralph svc_workflows --> pkg_tool_workflow @@ -364,34 +342,33 @@ flowchart LR | `ctx.llm` | `seam` | [`llm`](../packages/llm/llm) | [`llm-deepseek`](../packages/llm/llm-deepseek), [`llm-pi-ai`](../packages/llm/llm-pi-ai), [`llm-replay`](../packages/support/llm-replay) | [`agent-loop`](../packages/core/agent-loop), [`compact-basic`](../packages/compact/compact-basic) | - | Adapters register provider implementations; the loop and compaction call the provider-neutral stream service. | | `ctx.tokenMeter` | `core` | [`token-meter`](../packages/llm/token-meter) | - | [`compact-basic`](../packages/compact/compact-basic) | - | Owns isolated per-session replay folds; pressure consumers share immutable revisioned measurements. | | `ctx.toolResultPrune` | `core` | [`compact-tool-result-prune`](../packages/compact/compact-tool-result-prune) | - | [`compact-basic`](../packages/compact/compact-basic) | - | Rewrites oversized current tool results through replayable single-node surface replacements before summary compaction. | -| `ctx.sessions` | `core` | [`session`](../packages/core/session) | - | [`agent-loop`](../packages/core/agent-loop), [`agent`](../packages/core/agent), [`session-persistence`](../packages/session-persistence/session-persistence), [`session-query`](../packages/session-query/session-query), [`session-query-sqlite`](../packages/session-query/session-query-sqlite), [`subagent-inprocess`](../packages/subagent/subagent-inprocess), [`invariants`](../packages/support/invariants) | - | Owns append-only Session instances and emits the durable session event feed. | +| `ctx.sessions` | `core` | [`session`](../packages/core/session) | - | [`agent-loop`](../packages/core/agent-loop), [`agent`](../packages/core/agent), [`cli-demo`](../packages/examples/cli-demo), [`session-persistence`](../packages/session-persistence/session-persistence), [`session-query`](../packages/session-query/session-query), [`session-query-sqlite`](../packages/session-query/session-query-sqlite), [`subagent-inprocess`](../packages/subagent/subagent-inprocess), [`invariants`](../packages/support/invariants) | - | Owns append-only Session instances and emits the durable session event feed. | | `ctx.invariants` | `core` | [`invariants`](../packages/support/invariants) | - | [`session`](../packages/core/session), [`agent`](../packages/core/agent), [`scope`](../packages/core/scope), [`agent-loop`](../packages/core/agent-loop) | - | Companion subpaths register owner-local checks; the service owns selection, uniqueness, child fibers, and package-attributed failures. | | `ctx.typert` | `core` | [`typert-registry`](../packages/typert/registry) | - | [`typert-loader`](../packages/typert/loader), [`api-gateway`](../packages/api/gateway) | - | Plugins register live zod contributions directly or through dsh-typert-loader; the API gateway consumes invocation descriptors and providers, while other runtime consumers query schemas and reflection metadata at their own edges. | | `ctx.typertGateway` | `core` | [`api-gateway`](../packages/api/gateway) | - | - | - | Associates generated Remote descriptors with live Cordis services, resolves registered identities, and exposes unary calls through the shared Connection RPC carrier. | | `ctx.sessionPersistence` | `seam` | [`session-persistence`](../packages/session-persistence/session-persistence) | [`session-persistence-jsonl`](../packages/session-persistence/session-persistence-jsonl), [`session-persistence-sqlite`](../packages/session-persistence/session-persistence-sqlite) | [`agent-loop`](../packages/core/agent-loop), [`tool-bash`](../packages/bash/tool-bash), [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex), [`session-query`](../packages/session-query/session-query), [`session-query-sqlite`](../packages/session-query/session-query-sqlite) | - | Backends persist the same SessionEvent vocabulary; apps choose a backend at composition time. | -| `ctx.settings` | `seam` | [`settings`](../packages/settings/settings) | [`settings-local`](../packages/settings/settings-local) | [`llm-deepseek`](../packages/llm/llm-deepseek), [`llm-pi-ai`](../packages/llm/llm-pi-ai), `apiproxy` | - | Plugins register namespace schemas and resolve layered values; providers store the raw document. The LLM adapters register their entry config as the composition base under the user section; the web gateway serves redacted layered descriptors and writes the user layer. | -| `ctx.credentials` | `seam` | [`credentials`](../packages/credentials/credentials) | [`credentials-local`](../packages/credentials/credentials-local) | [`llm-deepseek`](../packages/llm/llm-deepseek), [`llm-pi-ai`](../packages/llm/llm-pi-ai), `apiproxy` | - | Configuration carries references to secrets; providers own the values. Consumers resolve per operation, so a rotated credential reaches the very next request; the web gateway exposes value-free views and write-only storage. | | `ctx.telemetry` | `seam` | [`session-telemetry`](../packages/telemetry/session-telemetry) | [`session-telemetry-otel`](../packages/telemetry/session-telemetry-otel) | - | - | The seam captures, redacts, and hands session records to one backend; nothing else consumes the service — its output leaves the process. | | `ctx.storage` | `seam` | [`storage`](../packages/storage/storage) | [`storage-json`](../packages/storage/storage-json), [`storage-sqlite`](../packages/storage/storage-sqlite) | [`storage-domain`](../packages/storage/storage-domain) | - | Backends register side by side under names; data forms (domain first) mount on the hub and translate typed operations into opaque KV-unit primitives. | | `ctx.storageDomain` | `core` | [`storage-domain`](../packages/storage/storage-domain) | - | [`workspace`](../packages/workspace/workspace) | - | Waits for every configured backend, then publishes the domain form as one lifecycle-bound service for typed durable state. | | `ctx.workspace` | `core` | [`workspace`](../packages/workspace/workspace) | - | `apiproxy` | - | Owns WorkspaceId-branded records over the domain facility; stable sessionIds accounts drive Host RPC and GUI projections. | | `ctx.sessionQuery` | `seam` | [`session-query`](../packages/session-query/session-query) | [`session-query-sqlite`](../packages/session-query/session-query-sqlite) | [`session-reference`](../packages/context/session-reference), [`tool-session-query`](../packages/session-query/tool-session-query) | - | The interface supplies exact reads, filters, and traces; its concrete backend adds full-text reconciliation, ranking, snippets, and cursor generations, while the model consumer owns workspace authority and cursor-free rendering. | -| `ctx.sessionReferences` | `core` | [`session-reference`](../packages/context/session-reference) | - | - | - | Projects bounded current-surface conversation snapshots into durable untrusted message context; host adapters own mention syntax. | +| `ctx.sessionReferences` | `core` | [`session-reference`](../packages/context/session-reference) | - | [`tui`](../packages/ui/tui) | - | Projects bounded current-surface conversation snapshots into durable untrusted message context; host adapters own mention syntax. | | `ctx.sessionTitle` | `seam` | [`session-title`](../packages/session-title/session-title) | [`session-title-first-message-llm`](../packages/session-title/session-title-first-message-llm), [`session-title-all-messages-llm`](../packages/session-title/session-title-all-messages-llm) | - | - | Owns the deterministic fallback, latest-title fold, and sole optional asynchronous provider registration. | | `ctx.systemPrompt` | `core` | [`system-prompt`](../packages/core/system-prompt) | - | [`agent-loop`](../packages/core/agent-loop), [`tools`](../packages/core/tools), [`tool-fs`](../packages/fs/tool-fs), [`tool-pty`](../packages/pty/tool-pty), [`tool-web`](../packages/web/tool-web) | - | Collects prompt sections and model-facing tool schemas for each step. | | `ctx.tools` | `core` | [`tools`](../packages/core/tools) | - | [`agent-loop`](../packages/core/agent-loop), [`tool-ask-user`](../packages/ui/tool-ask-user), [`tool-bash`](../packages/bash/tool-bash), [`tool-cordis`](../packages/cordis/tool-cordis), [`tool-fs`](../packages/fs/tool-fs), [`tool-pty`](../packages/pty/tool-pty), [`tool-skill`](../packages/skill/tool-skill), [`tool-subagent`](../packages/subagent/tool-subagent), [`tool-todo`](../packages/todo/tool-todo), [`tool-web`](../packages/web/tool-web) | - | Registers capabilities, owns Code Mode transport, and routes calls through pre-policy, monotonic guards, around dispatch, post-policy, and final-result observation. | -| `ctx.userInteraction` | `seam` | [`user-interaction`](../packages/ui/user-interaction) | - | [`tool-ask-user`](../packages/ui/tool-ask-user) | - | UI front doors provide the active human-answer provider; tool-ask-user pauses a tool call on the provider-neutral ask() promise. | +| `ctx.userInteraction` | `seam` | [`user-interaction`](../packages/ui/user-interaction) | [`tui`](../packages/ui/tui) | [`tool-ask-user`](../packages/ui/tool-ask-user), [`tui`](../packages/ui/tui) | - | UI front doors provide the active human-answer provider; tool-ask-user pauses a tool call on the provider-neutral ask() promise. | | `ctx.planMode` | `core` | [`plan-mode`](../packages/plan/plan-mode) | - | - | - | Folds logged plan/mode state, flushes user selections at turn boundaries, renders deployment-owned guidance, registers /plan, and keeps the plan-exit schema stable across transitions. | -| `ctx.commands` | `core` | [`commands`](../packages/ui/commands) | - | - | - | Plugins register direct human commands without sending invocations to the model. | +| `ctx.commands` | `core` | [`commands`](../packages/ui/commands) | - | [`tui`](../packages/ui/tui) | - | Plugins register direct human commands; TUI consumes the effective per-agent catalog without sending invocations to the model. | | `ctx.sessionProjections` | `core` | [`session-projection`](../packages/session-projection/session-projection) | - | [`tool-todo`](../packages/todo/tool-todo), [`session-title`](../packages/session-title/session-title), [`host-apiproxy`](../packages/host/apiproxy) | - | Domains register state-driven fold units; the eager drive keeps per-session watermark states and api-proxy serves baselines and pushes changed values. | | `ctx.sessionProjectionCache` | `core` | [`session-projection-cache`](../packages/session-projection/session-projection-cache) | - | [`host-apiproxy`](../packages/host/apiproxy) | - | Durably checkpoints projection unit states per session (throttled + turn/end/detach mandatory points) and serves the cold-read ladder: cache row + persistence tail replay, so listings never load full logs. | -| `ctx.skills` | `seam` | [`skill`](../packages/skill/skill) | [`skill-badge`](../packages/skill/skill-badge), [`skill-local`](../packages/skill/skill-local) | [`tool-skill`](../packages/skill/tool-skill) | - | Merges provider skill catalogs; tool-skill renders the session-prefix catalog and loads complete skill bodies. | -| `ctx.agents` | `core` | [`agent`](../packages/core/agent) | - | [`agent-loop`](../packages/core/agent-loop), [`acp`](../packages/acp/acp), [`subagent-inprocess`](../packages/subagent/subagent-inprocess) | - | Owns live Agent handles, the create/resume factory seam, and process-local initiator propagation. | +| `ctx.tui` | `bundle` | [`tui`](../packages/ui/tui) | - | - | - | One TUI front door provides a FIFO overlay host; injected plugins receive caller-fiber ownership without access to pi-tui or terminal lifecycle state. | +| `ctx.skills` | `seam` | [`skill`](../packages/skill/skill) | [`skill-local`](../packages/skill/skill-local) | [`tool-skill`](../packages/skill/tool-skill) | - | Merges provider skill catalogs; tool-skill renders the session-prefix catalog and loads complete skill bodies. | +| `ctx.agents` | `core` | [`agent`](../packages/core/agent) | - | [`agent-loop`](../packages/core/agent-loop), [`acp`](../packages/acp/acp), [`cli-demo`](../packages/examples/cli-demo), [`subagent-inprocess`](../packages/subagent/subagent-inprocess), [`tui-demo`](../packages/examples/tui-demo) | - | Owns live Agent handles, the create/resume factory seam, and process-local initiator propagation. | | `ctx.agentLoop` | `bundle` | [`agent-loop`](../packages/core/agent-loop) | - | [`agent-spine-demo`](../packages/examples/agent-spine-demo) | - | The one concrete loop plugin; extension packages depend on dsh-agent events and services, not on this package. | | `ctx.goals` | `core` | [`goal`](../packages/goal/goal) | - | - | - | Folds revisioned objective state from the session log and keeps live continuation activation process-local. | -| `ctx.subprocess` | `seam` | [`subprocess`](../packages/subprocess/subprocess) | [`subprocess-local`](../packages/subprocess/subprocess-local) | [`bash-local`](../packages/bash/bash-local), [`bash-sandbox`](../packages/bash/bash-sandbox), [`lsp-local`](../packages/lsp/lsp-local), [`subagent-acp`](../packages/subagent/subagent-acp), [`subagent-codex`](../packages/subagent/subagent-codex), [`subagent-claude-code`](../packages/subagent/subagent-claude-code) | - | The bash executors, the LSP host, and the out-of-process ACP, Codex, and Claude Code subagent backends spawn their children through ctx.subprocess; the service owns tree lifetime, stdio dispositions (pipes, inherit, bounded spill-backed collection), and kill escalation. | -| `ctx.bash` | `seam` | [`bash`](../packages/bash/bash) | [`bash-local`](../packages/bash/bash-local), [`bash-sandbox`](../packages/bash/bash-sandbox), [`pwsh-local`](../packages/bash/pwsh-local) | [`tool-bash`](../packages/bash/tool-bash), [`tool-pwsh`](../packages/bash/tool-pwsh), [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex) | - | The model-facing shell tools and hook bridges consume this seam; sandboxed, remote, or PowerShell executors replace bash-local without touching them. | -| `ctx.bashEnv` | `core` | [`bash-env`](../packages/bash/bash-env) | - | [`tool-bash`](../packages/bash/tool-bash), [`tool-pwsh`](../packages/bash/tool-pwsh) | - | Plugins declare effect-scoped DSH_* facts; each shell tool collects one trusted snapshot per execution and its executor rebuilds the namespace. | +| `ctx.subprocess` | `seam` | [`subprocess`](../packages/subprocess/subprocess) | [`subprocess-local`](../packages/subprocess/subprocess-local) | [`bash-local`](../packages/bash/bash-local), [`bash-sandbox`](../packages/bash/bash-sandbox), [`pty-local`](../packages/pty/pty-local), [`lsp-local`](../packages/lsp/lsp-local), [`subagent-acp`](../packages/subagent/subagent-acp) | - | The bash executors, PTY shell backend, LSP host, and ACP subagent backend spawn through ctx.subprocess; the service owns process coordinates, tree/session lifetime, stdio dispositions, terminal mechanics, and kill escalation. | +| `ctx.bash` | `seam` | [`bash`](../packages/bash/bash) | [`bash-local`](../packages/bash/bash-local), [`bash-sandbox`](../packages/bash/bash-sandbox) | [`tool-bash`](../packages/bash/tool-bash), [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex) | - | The model-facing bash tools and hook bridges consume this seam; sandboxed or remote executors replace bash-local without touching them. | +| `ctx.bashEnv` | `core` | [`tool-bash`](../packages/bash/tool-bash) | - | - | - | Plugins declare effect-scoped DSH_* facts; tool-bash collects one trusted snapshot per execution and the executor rebuilds the namespace. | | `ctx.pty` | `seam` | [`pty`](../packages/pty/pty) | [`pty-local`](../packages/pty/pty-local) | [`tool-pty`](../packages/pty/tool-pty) | - | The registry owns exact-Agent session identity and cleanup; backends own terminal mechanics, while tool-pty exposes the owner-scoped model surface. | | `ctx.sandbox` | `seam` | [`sandbox`](../packages/sandbox/sandbox) | [`sandbox-local`](../packages/sandbox/sandbox-local) | [`bash-sandbox`](../packages/bash/bash-sandbox), [`pty-local`](../packages/pty/pty-local) | - | Consumers hand over the exact argv they are about to spawn; same-world backends wrap it under a per-call policy and report enforcement. | | `ctx.sandboxPolicy` | `core` | [`sandbox-policy`](../packages/sandbox/sandbox-policy) | - | [`bash-sandbox`](../packages/bash/bash-sandbox), [`fs-sandbox`](../packages/fs/fs-sandbox), [`pty-local`](../packages/pty/pty-local) | - | The one home for the deployment default mode + workspace root; only the sandboxed executor and provider read the service (the tool layers use the pure `sandbox/mode` fold it also exports). Both enforcing families read it so bash and fs cannot confine to different roots. | @@ -400,7 +377,7 @@ flowchart LR | `ctx.codeRuntime` | `seam` | [`code-runtime`](../packages/code-runtime/code-runtime) | [`code-runtime-worker`](../packages/code-runtime/code-runtime-worker) | [`tools`](../packages/core/tools) | - | Runs one model-written program against host-provided async bindings; backends differ by substrate and language (the tool registry consumes it for Code Mode). | | `ctx.fs` | `seam` | [`fs`](../packages/fs/fs) | [`fs-local`](../packages/fs/fs-local), [`fs-sandbox`](../packages/fs/fs-sandbox) | [`tool-fs`](../packages/fs/tool-fs) | [`fs-policy`](../packages/fs/fs-policy) | tool-fs executes read/write/edit through ctx.fs; fs-sandbox fences mutations by the shared sandbox mode; fs-policy contributes observed-state checks through the fs/* event gate. | | `ctx.compact` | `seam` | [`compact`](../packages/compact/compact) | [`compact-basic`](../packages/compact/compact-basic) | [`compact-basic`](../packages/compact/compact-basic) | - | The basic backend consumes post-step pressure and request-error recovery events; a model-facing compact tool remains deferred. | -| `ctx.subagents` | `seam` | [`subagent`](../packages/subagent/subagent) | [`subagent-spawn`](../packages/subagent/subagent-spawn), [`subagent-fork`](../packages/subagent/subagent-fork), [`subagent-acp`](../packages/subagent/subagent-acp), [`subagent-codex`](../packages/subagent/subagent-codex), [`subagent-claude-code`](../packages/subagent/subagent-claude-code), [`subagent-dsh-sdk`](../packages/subagent/subagent-dsh-sdk) | [`tool-subagent`](../packages/subagent/tool-subagent), [`tool-subagent-control`](../packages/subagent/tool-subagent-control), [`tool-ralph`](../packages/workflow/tool-ralph) | - | Providers implement transports; the service also owns optional Activation-based continuation orchestration, tool-subagent selects one-shot or continuable delegation, tool-subagent-control delivers follow-ups, and tool-ralph requires one fresh structured-output route. | +| `ctx.subagents` | `seam` | [`subagent`](../packages/subagent/subagent) | [`subagent-spawn`](../packages/subagent/subagent-spawn), [`subagent-fork`](../packages/subagent/subagent-fork), [`subagent-acp`](../packages/subagent/subagent-acp) | [`tool-subagent`](../packages/subagent/tool-subagent), [`tool-ralph`](../packages/workflow/tool-ralph) | - | Providers implement transports; tool-subagent exposes configured delegation while tool-ralph requires one fresh structured-output route. | | `ctx.tasks` | `seam` | [`tasks`](../packages/tasks/tasks) | [`tasks-local`](../packages/tasks/tasks-local) | [`tool-bash`](../packages/bash/tool-bash), [`tool-pty`](../packages/pty/tool-pty), [`tool-subagent`](../packages/subagent/tool-subagent), [`tool-tasks`](../packages/tasks/tool-tasks) | - | Producers (background bash, PTY sends, and subagent delegations) register running work; tool-tasks is the model-facing control surface that reads, lists, and kills it; tasks-local is the process-local registry. | | `ctx.web` | `seam` | [`web`](../packages/web/web) | [`web-search-exa`](../packages/web/web-search-exa), [`web-search-perplexity`](../packages/web/web-search-perplexity), [`web-search-deepseek`](../packages/web/web-search-deepseek), [`web-fetch-local`](../packages/web/web-fetch-local) | [`tool-web`](../packages/web/tool-web) | - | Search and fetch providers register into one ctx.web seam; tool-web owns the stable model-facing names. | | `ctx.spillStore` | `seam` | [`spill`](../packages/spill/spill) | [`spill-local`](../packages/spill/spill-local) | [`spill-policy`](../packages/spill/spill-policy) | - | The backend saves oversized tool text and returns a model-facing locator plus retrieval hint; spill-policy is the tools/post-execute consumer that decides when to spill. | diff --git a/docs/config-catalog.md b/docs/config-catalog.md index a875c190fe..b8a49a7ba2 100644 --- a/docs/config-catalog.md +++ b/docs/config-catalog.md @@ -284,30 +284,6 @@ export interface Config { Source: [`packages/client/hmr/src/index.ts:29`](../packages/client/hmr/src/index.ts) -## `@deepseek-ai/dsh-code-runtime-subprocess` - -Requires: `subprocess` - -```ts config-catalog -/** Runtime configuration; every execution and bridge bound is deployment-tunable. */ -export interface Config { - /** Worker measured event-loop busy-time budget. */ - computeMs?: number - /** Host-observed wall-clock ceiling. */ - maxWallMs?: number - /** Combined serialized outer logs/value/diagnostic cap. */ - maxOutputBytes?: number - /** Worker old-generation heap cap in MiB. */ - maxOldGenerationSizeMb?: number - /** Largest decoded bridge frame, including binding traffic. */ - maxFrameBytes?: number - /** Process-tree TERM-to-KILL grace. */ - killGraceMs?: number -} -``` - -Source: [`packages/code-runtime/code-runtime-subprocess/src/index.ts:27`](../packages/code-runtime/code-runtime-subprocess/src/index.ts) - ## `@deepseek-ai/dsh-code-runtime-worker` ```ts config-catalog @@ -341,7 +317,7 @@ export interface Config { } ``` -Source: [`packages/code-runtime/code-runtime-worker/src/index.ts:30`](../packages/code-runtime/code-runtime-worker/src/index.ts) +Source: [`packages/code-runtime/code-runtime-worker/src/index.ts:25`](../packages/code-runtime/code-runtime-worker/src/index.ts) ## `@deepseek-ai/dsh-compact-basic` diff --git a/docs/core-data-structures/subprocess.i18n.yaml b/docs/core-data-structures/subprocess.i18n.yaml index cfa3762222..50e219a2c6 100644 --- a/docs/core-data-structures/subprocess.i18n.yaml +++ b/docs/core-data-structures/subprocess.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write docs/core-data-structures/subprocess.md -subprocess.md: c68a2f77059f62b273c536987b321017f2a71bbb -subprocess.zh.md: 061c681430ba1f011b4ed9e5c4d9fc8595fe6738 +subprocess.md: 895bfe3d763853a86648e8aaab8b091a26006255 +subprocess.zh.md: 044e924674ecf5dd8f8a1c71fe39653e7525dfb0 diff --git a/docs/core-data-structures/subprocess.md b/docs/core-data-structures/subprocess.md index a258ff4767..023b122218 100644 --- a/docs/core-data-structures/subprocess.md +++ b/docs/core-data-structures/subprocess.md @@ -2,13 +2,13 @@ English | [中文](subprocess.zh.md) -The subprocess seam is split across interface ([dsh-subprocess](../../packages/subprocess/subprocess), `ctx.subprocess`) and implementation ([dsh-subprocess-local](../../packages/subprocess/subprocess-local)); its consumers are other capability seams and out-of-process backends: the [bash executor family](bash.md) uses collected batch output, the LSP and Code Runtime hosts use raw protocol pipes, the PTY backend uses the terminal primitive, and the ACP subagent backend uses piped ndjson plus inherited stderr. This seam owns the managed `DSH_*` environment namespace, the shared credential scrub (`scrubbedParentEnv`), and the `CollectedOutput` shape; [dsh-bash](../../packages/bash/bash) re-exports the vocabulary so bash consumers keep one import root. +The subprocess seam is split across interface ([dsh-subprocess](../../packages/subprocess/subprocess), `ctx.subprocess`) and implementation ([dsh-subprocess-local](../../packages/subprocess/subprocess-local)); its consumers are other capability seams and out-of-process backends: the [bash executor family](bash.md) uses collected batch output, LSP uses raw protocol pipes, the PTY backend uses the terminal primitive, and the ACP subagent backend uses piped ndjson plus inherited stderr. This seam owns the managed `DSH_*` environment namespace, the shared credential scrub (`scrubbedParentEnv`), and the `CollectedOutput` shape; [dsh-bash](../../packages/bash/bash) re-exports the vocabulary so bash consumers keep one import root. Source: [`packages/subprocess/subprocess/src/types.ts`](../../packages/subprocess/subprocess/src/types.ts) and [`packages/subprocess/subprocess/src/index.ts`](../../packages/subprocess/subprocess/src/index.ts) -## Execution-world coordinates +## Executable lookup -One provider's `cwd`, executable paths, ordinary processes, and terminal sessions inhabit the same path and process namespace as the mounted filesystem provider. `resolveExecutable(command, env?, signal?)` verifies absolute executable paths or resolves bare names through the provider's scrubbed `PATH` plus deliberate overrides. +One provider's spawn working directories, executable paths, ordinary processes, and terminal sessions inhabit the same path and process namespace as the mounted filesystem provider. `resolveExecutable(command, env?, signal?)` verifies absolute executable paths or resolves bare names through the provider's scrubbed `PATH` plus deliberate overrides. ## Managed environment namespace and captured output diff --git a/docs/core-data-structures/subprocess.zh.md b/docs/core-data-structures/subprocess.zh.md index 061c681430..044e924674 100644 --- a/docs/core-data-structures/subprocess.zh.md +++ b/docs/core-data-structures/subprocess.zh.md @@ -2,13 +2,13 @@ [English](subprocess.md) | 中文 -进程管理器 seam 分为接口([dsh-subprocess](../../packages/subprocess/subprocess),`ctx.subprocess`)与实现([dsh-subprocess-local](../../packages/subprocess/subprocess-local));它的消费方是其他能力 seam 与进程外后端:[bash 执行器家族](bash.md)使用收集模式的批量输出,LSP 与 Code Runtime 主机使用原始协议管道,PTY 后端使用终端原语,ACP(Agent Client Protocol)subagent 后端则使用管道化 ndjson 加 inherit 的 stderr。该 seam 拥有受管的 `DSH_*` 环境命名空间、共享的凭据清除(`scrubbedParentEnv`)与 `CollectedOutput` 形状;[dsh-bash](../../packages/bash/bash) 重导出这套词汇,使 bash 消费方保持单一导入入口。 +进程管理器 seam 分为接口([dsh-subprocess](../../packages/subprocess/subprocess),`ctx.subprocess`)与实现([dsh-subprocess-local](../../packages/subprocess/subprocess-local));它的消费方是其他能力 seam 与进程外后端:[bash 执行器家族](bash.md)使用收集模式的批量输出,LSP 使用原始协议管道,PTY 后端使用终端原语,ACP(Agent Client Protocol)subagent 后端则使用管道化 ndjson 加 inherit 的 stderr。该 seam 拥有受管的 `DSH_*` 环境命名空间、共享的凭据清除(`scrubbedParentEnv`)与 `CollectedOutput` 形状;[dsh-bash](../../packages/bash/bash) 重导出这套词汇,使 bash 消费方保持单一导入入口。 源码:[`packages/subprocess/subprocess/src/types.ts`](../../packages/subprocess/subprocess/src/types.ts) 与 [`packages/subprocess/subprocess/src/index.ts`](../../packages/subprocess/subprocess/src/index.ts) -## 执行世界坐标 +## 可执行文件查找 -一个提供方的 `cwd`、可执行文件路径、普通进程与终端会话,和挂载的文件系统提供方处于同一路径与进程命名空间。`resolveExecutable(command, env?, signal?)` 验证绝对可执行文件路径,或通过提供方清理后的 `PATH` 加有意覆盖来解析裸名称。 +一个提供方的 spawn 工作目录、可执行文件路径、普通进程与终端会话,和挂载的文件系统提供方处于同一路径与进程命名空间。`resolveExecutable(command, env?, signal?)` 验证绝对可执行文件路径,或通过提供方清理后的 `PATH` 加有意覆盖来解析裸名称。 ## 受管环境命名空间与捕获的输出 diff --git a/knip.json b/knip.json index 413b740578..ede8b2c738 100644 --- a/knip.json +++ b/knip.json @@ -383,17 +383,6 @@ "tests/**/*.ts" ] }, - "packages/code-runtime/code-runtime-subprocess": { - "entry": [ - "src/runner.ts", - "tests/**/*.spec.ts", - "tests/**/*.e2e.ts" - ], - "project": [ - "src/**/*.ts", - "tests/**/*.ts" - ] - }, "packages/llm/llm-deepseek": { "entry": [ "tests/**/*.spec.ts", diff --git a/package.json b/package.json index 2b4971a6b7..423cbf813c 100644 --- a/package.json +++ b/package.json @@ -112,8 +112,6 @@ "gen-module-graph": "tsx scripts/gen-module-graph.ts", "gen-scoped-events": "tsx scripts/gen-scoped-events.ts", "verify-scoped-events": "tsx scripts/gen-scoped-events.ts --check", - "gen-code-runtime-runner": "tsx scripts/gen-code-runtime-runner.ts", - "verify-code-runtime-runner": "tsx scripts/gen-code-runtime-runner.ts --check", "verify-module-graph": "tsx scripts/gen-module-graph.ts --check", "constraints": "tsx scripts/check-workspace-constraints.ts", "doc-sync": "tsx scripts/run-gates.ts doc-sync", diff --git a/packages/README.i18n.yaml b/packages/README.i18n.yaml index eaa48fac05..1484411cca 100644 --- a/packages/README.i18n.yaml +++ b/packages/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/README.md -README.md: b498c146202f1a69d89805fa3e8966c4d0735d6c -README.zh.md: 2eefc73ed0ea22aae5fde93284c25c5d03fe86c4 +README.md: 3f467641bbc9eae14a94aa2d3bff0402116a9d3f +README.zh.md: faa879f3b5b62527c1d76ab3aff6a737f876ce4d diff --git a/packages/README.md b/packages/README.md index 7d591605ef..229feae568 100644 --- a/packages/README.md +++ b/packages/README.md @@ -19,7 +19,7 @@ Packages live at `packages///`; groups are containers, while names r | [`subprocess/`](subprocess/README.md) | Subprocess capability family: spawn seam + local process-tree implementation | Product — stable surface | | [`bash/`](bash/README.md) | Bash capability family: executor seam, local impl, model-facing tool | Product — stable surface | | [`pty/`](pty/README.md) | Persistent PTY capability family: owner-scoped sessions, local implementation, and model-facing tools | Product — stable surface | -| [`code-runtime/`](code-runtime/README.md) | Code-execution capability family: runtime seam plus local worker and subprocess backends | Product — stable surface | +| [`code-runtime/`](code-runtime/README.md) | Code-execution capability family: the runtime seam for model-written programs + a worker-thread backend | Product — stable surface | | [`sandbox/`](sandbox/README.md) | Process-confinement seam; bwrap/Landlock/Seatbelt backends | Product — stable surface | | [`fs/`](fs/README.md) | Filesystem capability family: seam, local impl, model-facing file tools, bash-backed discovery tools | Product — stable surface | | [`lsp/`](lsp/README.md) | LSP capability family: seam, generic stdio provider, and the `lsp` tool | Product — stable surface | diff --git a/packages/README.zh.md b/packages/README.zh.md index a9201d08a9..bed31b657c 100644 --- a/packages/README.zh.md +++ b/packages/README.zh.md @@ -19,7 +19,7 @@ | [`subprocess/`](subprocess/README.md) | 进程管理能力系列:spawn seam + 本地进程树实现 | 产品:稳定表面 | | [`bash/`](bash/README.md) | Bash 能力系列:执行器 seam、本地实现、面向模型的工具 | 产品:稳定表面 | | [`pty/`](pty/README.md) | 持久 PTY 能力系列:按所有者隔离的会话、本地实现和面向模型的工具 | 产品:稳定表面 | -| [`code-runtime/`](code-runtime/README.md) | 代码执行能力系列:运行时 seam、本地 worker 后端及进程管理后端 | 产品:稳定表面 | +| [`code-runtime/`](code-runtime/README.md) | 代码执行能力系列:用于模型编写程序的运行时 seam + worker 线程后端 | 产品:稳定表面 | | [`sandbox/`](sandbox/README.md) | 进程限制 seam;bwrap/Landlock/Seatbelt 后端 | 产品:稳定表面 | | [`fs/`](fs/README.md) | 文件系统能力系列:seam、本地实现、面向模型的文件工具、bash 后端发现工具 | 产品:稳定表面 | | [`lsp/`](lsp/README.md) | LSP 能力系列:seam、通用 stdio 提供方和 `lsp` 工具 | 产品:稳定表面 | diff --git a/packages/code-runtime/code-runtime-worker/src/bootstrap.ts b/packages/code-runtime/code-runtime-worker/src/bootstrap.ts index 2207389c97..cbe0d0ac55 100644 --- a/packages/code-runtime/code-runtime-worker/src/bootstrap.ts +++ b/packages/code-runtime/code-runtime-worker/src/bootstrap.ts @@ -6,7 +6,6 @@ */ import { inspect } from 'node:util' -import type { CodeJsonValue } from '@deepseek-ai/dsh-code-runtime' import type { DoneMessage, ReplyMessage, WorkerBootData, WorkerToHost } from './protocol.ts' import { jsonStringBytesUpTo, jsonValueBytesUpTo, truncateJsonStringBytes } from './output-json.ts' import { decodeWorkerJson, encodeWorkerJson, snapshotCodeJsonValue } from './worker-json.ts' @@ -311,7 +310,6 @@ export function wireReplies(port: BootstrapPort, pending: Map, nextId: { value: number }, errorClasses: Map = makeBindingErrorClasses(data), - maxFrameBytes?: number, ): Record[] { return data.namespaces.map(({ global, names }) => { const errorClass = errorClasses.get(global) @@ -338,11 +335,6 @@ export function makeNamespaces( if (detached === undefined) { return Promise.reject(bindingFailure(errorClass, name, 'binding arguments must be lossless JSON')) } - const call = { type: 'call' as const, id: nextId.value, global, name, args: encodeWorkerJson(detached) } - if (maxFrameBytes !== undefined - && jsonValueBytesUpTo(call as unknown as CodeJsonValue, maxFrameBytes) === undefined) { - return Promise.reject(bindingFailure(errorClass, name, 'binding arguments exceed maxFrameBytes')) - } return new Promise((resolve, reject) => { const id = nextId.value++ pending.set(id, { @@ -352,7 +344,7 @@ export function makeNamespaces( }, }) try { - port.postMessage(call) + port.postMessage({ type: 'call', id, global, name, args: encodeWorkerJson(detached) }) } catch (error: unknown) { pending.delete(id) const message = `binding arguments must be structured-cloneable: ${error instanceof CapturedError ? error.message : String(error)}` @@ -372,14 +364,12 @@ export function makeNamespaces( * @param port - host message port or test double. * @param data - the boot payload the host sent. * @param streams - stdout/stderr objects captured as program logs. - * @param maxFrameBytes - optional serialized transport cap checked before posting. * @returns after posting the done message. */ export async function runWorkerMain( port: BootstrapPort, data: WorkerBootData, streams: { stdout: PatchableStream; stderr: PatchableStream }, - maxFrameBytes?: number, ): Promise { const logs = new LogBuffer( data.maxOutputBytes, @@ -394,7 +384,7 @@ export async function runWorkerMain( const nextId = { value: 1 } const errorClasses = makeBindingErrorClasses(data) - const namespaces = makeNamespaces(data, port, pending, nextId, errorClasses, maxFrameBytes) + const namespaces = makeNamespaces(data, port, pending, nextId, errorClasses) const errorClassParameters: string[] = [] const errorClassValues: BindingErrorConstructor[] = [] for (const namespace of data.namespaces) { @@ -430,8 +420,5 @@ export async function runWorkerMain( ...prepareException(error, logs.remainingOutputBytes(), data.maxOutputBytes), } } - port.postMessage(maxFrameBytes !== undefined - && jsonValueBytesUpTo(done as unknown as CodeJsonValue, maxFrameBytes) === undefined - ? { type: 'output-limit' } - : done) + port.postMessage(done) } diff --git a/packages/code-runtime/code-runtime-worker/src/runtime-host.ts b/packages/code-runtime/code-runtime-worker/src/runtime-host.ts deleted file mode 100644 index 47bfacfe91..0000000000 --- a/packages/code-runtime/code-runtime-worker/src/runtime-host.ts +++ /dev/null @@ -1,251 +0,0 @@ -/** Shared host mechanics for local and subprocess-hosted TypeScript worker runtimes. */ - -import { stripTypeScriptTypes } from 'node:module' -import type { Readable } from 'node:stream' -import type { - CodeBindingNamespace, - CodeJsonValue, - CodeRunFailure, - CodeRunRequest, - CodeRunResult, -} from '@deepseek-ai/dsh-code-runtime' -import { jsonStringBytesUpTo, jsonValueBytesUpTo, truncateJsonStringBytes } from './output-json.ts' -import { decodeWorkerJson, encodeWorkerJson, snapshotCodeJsonValue } from './worker-json.ts' -import type { WorkerJsonWire } from './worker-json.ts' - -/** Smallest cap that can represent an empty log array and failure message. */ -export const MIN_RUNTIME_OUTPUT_BYTES = 4 - -/** - * Resolve after a worker pipe emits queued data or closes during termination. - * @param stream - captured worker or child-process pipe. - * @returns after no more queued bytes can arrive. - */ -export function waitForRuntimePipeDrain(stream: Readable): Promise { - if (stream.readableEnded || stream.destroyed) return Promise.resolve() - return new Promise((resolve) => { - const done = (): void => { - stream.off('end', done) - stream.off('close', done) - stream.off('error', done) - resolve() - } - stream.once('end', done) - stream.once('close', done) - stream.once('error', done) - /* v8 ignore next -- termination can win the adjacent listener-registration race. */ - if (stream.readableEnded || stream.destroyed) done() - }) -} - -const IDENTIFIER = /^[A-Za-z_$][A-Za-z0-9_$]*$/ -const RESERVED_WORDS = new Set([ - 'await', 'break', 'case', 'catch', 'class', 'const', 'continue', 'debugger', 'default', 'delete', 'do', - 'else', 'enum', 'export', 'extends', 'false', 'finally', 'for', 'function', 'if', 'import', 'in', - 'instanceof', 'new', 'null', 'return', 'super', 'switch', 'this', 'throw', 'true', 'try', 'typeof', - 'var', 'void', 'while', 'with', 'yield', 'let', 'static', 'implements', 'interface', 'package', - 'private', 'protected', 'public', 'arguments', 'eval', -]) -const RESERVED_ERROR_PROPERTIES = new Set(['name', 'message', 'stack']) -const STRIP_WRAP = { prefix: 'async function __dsh_program__() {\n', suffix: '\n}' } as const - -/** One validated binding call received from an isolated worker. */ -export interface RuntimeBindingCall { - /** Correlation id supplied by the isolated worker. */ - readonly id: number - /** Injected namespace global. */ - readonly global: string - /** Declared namespace function. */ - readonly name: string - /** Untrusted lossless-JSON wire payload. */ - readonly args: unknown -} - -/** One host reply to an isolated worker binding call. */ -export type RuntimeBindingReply = - | { readonly type: 'reply'; readonly id: number; readonly ok: true; readonly value: WorkerJsonWire } - | { readonly type: 'reply'; readonly id: number; readonly ok: false; readonly message: string } - -/** - * Render an unknown thrown value without assuming it is an Error. - * @param error - thrown or rejected value. - * @returns the caller-facing diagnostic text. - */ -export function runtimeErrorMessage(error: unknown): string { - try { - return error instanceof Error ? error.message : String(error) - } catch { - return 'binding rejected with an unrenderable value' - } -} - -/** - * Strip erasable TypeScript while preserving the program's body coordinates. - * @param program - model-written async-function body. - * @returns JavaScript source with the wrapper removed. - */ -export function stripRuntimeProgram(program: string): string { - const stripped = stripTypeScriptTypes(STRIP_WRAP.prefix + program + STRIP_WRAP.suffix) - return stripped.slice(STRIP_WRAP.prefix.length, stripped.length - STRIP_WRAP.suffix.length) -} - -/** - * Validate binding globals and typed-error declarations shared by worker runtimes. - * @param request - code-runtime request carrying the namespaces. - * @param implementationName - package name used in seam-misuse diagnostics. - * @returns namespaces indexed by their injected global. - */ -export function validateRuntimeBindings( - request: CodeRunRequest, - implementationName: string, -): Map { - const bindings = new Map() - for (const namespace of request.bindings) { - if (!IDENTIFIER.test(namespace.global) || RESERVED_WORDS.has(namespace.global)) { - throw new Error(`${implementationName}: binding global ${JSON.stringify(namespace.global)} is not a usable identifier`) - } - if (namespace.global === 'console' || bindings.has(namespace.global)) { - throw new Error(`${implementationName}: duplicate binding global ${JSON.stringify(namespace.global)}`) - } - bindings.set(namespace.global, namespace) - } - - const errorClassNames = new Set() - for (const namespace of request.bindings) { - const descriptor = namespace.errorClass - if (descriptor === undefined) continue - if (!IDENTIFIER.test(descriptor.name) || RESERVED_WORDS.has(descriptor.name)) { - throw new Error(`${implementationName}: binding error class ${JSON.stringify(descriptor.name)} is not a usable identifier`) - } - if (descriptor.name === 'console' || bindings.has(descriptor.name) || errorClassNames.has(descriptor.name)) { - throw new Error(`${implementationName}: duplicate injected global ${JSON.stringify(descriptor.name)}`) - } - if (descriptor.memberNameProperty.length === 0 || RESERVED_ERROR_PROPERTIES.has(descriptor.memberNameProperty)) { - throw new Error(`${implementationName}: binding error member property ${JSON.stringify(descriptor.memberNameProperty)} is not usable`) - } - errorClassNames.add(descriptor.name) - } - return bindings -} - -/** - * Resolve one untrusted worker call through a declared host binding. - * @param call - parsed call envelope from the isolated worker. - * @param bindings - namespaces returned by {@link validateRuntimeBindings}. - * @returns a lossless-JSON success or stable rejection reply. - */ -export async function invokeRuntimeBinding( - call: RuntimeBindingCall, - bindings: ReadonlyMap, -): Promise { - const functions = bindings.get(call.global)?.functions - const fn = functions !== undefined && Object.hasOwn(functions, call.name) ? functions[call.name] : undefined - if (typeof fn !== 'function') { - return { type: 'reply', id: call.id, ok: false, message: `unknown binding ${JSON.stringify(`${call.global}.${call.name}`)}` } - } - const args = decodeWorkerJson(call.args) - if (args === undefined) { - return { type: 'reply', id: call.id, ok: false, message: 'binding arguments must be lossless JSON' } - } - try { - const resolved = await fn(args) - let value: CodeJsonValue | undefined - try { - value = snapshotCodeJsonValue(resolved) - } catch { - value = undefined - } - if (value === undefined) { - return { type: 'reply', id: call.id, ok: false, message: 'binding resolution must be lossless JSON' } - } - return { type: 'reply', id: call.id, ok: true, value: encodeWorkerJson(value) } - } catch (error: unknown) { - return { type: 'reply', id: call.id, ok: false, message: runtimeErrorMessage(error) } - } -} - -/** One run's combined outer-output ledger; binding values never enter it. */ -export class RuntimeOutputLedger { - private bytes = 2 - private entries = 0 - - /** @param maxBytes - hard cap for logs plus completion or failure payload. */ - constructor(private readonly maxBytes: number) {} - - /** - * Admit one exact log entry. - * @param text - candidate log entry. - * @param sink - ordered retained log list. - * @returns false when the hard cap was crossed. - */ - admit(text: string, sink: string[]): boolean { - const separatorBytes = this.entries > 0 ? 1 : 0 - const stringBytes = jsonStringBytesUpTo(text, this.maxBytes - this.bytes - separatorBytes) - if (stringBytes === undefined) return false - this.bytes += stringBytes + separatorBytes - this.entries += 1 - sink.push(text) - return true - } - - /** - * Finalize a successful completion against the combined cap. - * @param logs - retained ordered logs. - * @param value - optional lossless-JSON completion. - * @returns the completion or output-limit result. - */ - success(logs: string[], value?: CodeJsonValue): CodeRunResult { - if (value !== undefined && jsonValueBytesUpTo(value, this.maxBytes - this.bytes) === undefined) return this.limit(logs) - return { logs, ...value !== undefined ? { value } : {} } - } - - /** - * Finalize one failure diagnostic against the combined cap. - * @param logs - retained ordered logs. - * @param error - structured runtime failure. - * @returns the failure or output-limit result. - */ - failure(logs: string[], error: CodeRunFailure): CodeRunResult { - if (jsonStringBytesUpTo(error.message, this.maxBytes - this.bytes) === undefined) return this.limit(logs) - return { logs, error } - } - - /** - * Build an explicit output-limit failure with a fitting log prefix. - * @param logs - ordered logs observed before the limit. - * @returns bounded output-limit result. - */ - limit(logs: string[]): CodeRunResult { - const fullMessage = `outer output exceeded ${this.maxBytes} bytes` - const messageBytes = fullMessage.length + 2 - const retained: string[] = [] - let retainedBytes = 2 - const logBudget = this.maxBytes - messageBytes - for (const text of logs) { - const separatorBytes = retained.length > 0 ? 1 : 0 - const availableBytes = logBudget - retainedBytes - separatorBytes - const stringBytes = jsonStringBytesUpTo(text, availableBytes) - if (stringBytes !== undefined) { - retained.push(text) - retainedBytes += stringBytes + separatorBytes - continue - } - const prefix = truncateJsonStringBytes(text, availableBytes) - if (prefix.length > 0) { - const prefixBytes = jsonStringBytesUpTo(prefix, availableBytes) - /* v8 ignore next -- truncateJsonStringBytes guarantees the same bound. */ - if (prefixBytes === undefined) throw new Error('output ledger produced an oversized log prefix') - retained.push(prefix) - retainedBytes += prefixBytes + separatorBytes - } - break - } - const message = truncateJsonStringBytes(fullMessage, this.maxBytes - retainedBytes) - return { logs: retained, error: { kind: 'output-limit', message } } - } -} - -export { decodeWorkerJson, encodeWorkerJson, snapshotCodeJsonValue } from './worker-json.ts' -export { jsonStringBytesUpTo, jsonValueBytesUpTo } from './output-json.ts' -export { runWorkerMain } from './bootstrap.ts' -export type { WorkerJsonWire } from './worker-json.ts' diff --git a/packages/code-runtime/code-runtime-worker/tests/bootstrap.spec.ts b/packages/code-runtime/code-runtime-worker/tests/bootstrap.spec.ts index 247501df29..a2aac6d9c2 100644 --- a/packages/code-runtime/code-runtime-worker/tests/bootstrap.spec.ts +++ b/packages/code-runtime/code-runtime-worker/tests/bootstrap.spec.ts @@ -286,23 +286,6 @@ describe('makeNamespaces', () => { expect(nextId.value).toBe(1) }) - it('rejects an oversized transport frame before posting or allocating a call id', async () => { - const port = new FakePort() - const pending = new Map() - const nextId = { value: 1 } - const data = { namespaces: [toolNamespace(['x'])] } - const [tools] = makeNamespaces( - data, port, pending, nextId, makeBindingErrorClasses(data), 64, - ) as [Record Promise>] - - await expect(tools.x?.({ text: 'x'.repeat(64) })).rejects.toMatchObject({ - name: 'ToolCallError', toolName: 'x', message: 'binding arguments exceed maxFrameBytes', - }) - expect(port.sent).toEqual([]) - expect(pending.size).toBe(0) - expect(nextId.value).toBe(1) - }) - it('uses ordinary Error for non-tools namespace failures', async () => { const deniedPort = new FakePort() deniedPort.respond = message => message.type === 'call' @@ -360,16 +343,6 @@ describe('runWorkerMain', () => { }) }) - it('reports output-limit before posting a completion that expands past the transport cap', async () => { - const port = new FakePort() - await runWorkerMain(port, { - maxOutputBytes: 1_000, - code: 'return Array.from({ length: 100 }, () => [])', - namespaces: [], - }, fakeStreams(), 100) - expect(port.sent.at(-1)).toEqual({ type: 'output-limit' }) - }) - it('reports a thrown program error on the done message', async () => { const port = new FakePort() await runWorkerMain(port, { ...BOOT, code: 'throw new Error("boom")', namespaces: [] }, fakeStreams()) diff --git a/packages/code-runtime/code-runtime-worker/tsconfig.json b/packages/code-runtime/code-runtime-worker/tsconfig.json index 340e68e848..555b3d161e 100644 --- a/packages/code-runtime/code-runtime-worker/tsconfig.json +++ b/packages/code-runtime/code-runtime-worker/tsconfig.json @@ -8,6 +8,9 @@ "src" ], "references": [ + { + "path": "../../core/session" + }, { "path": "../../../vendor/cosmokit" }, diff --git a/packages/code-runtime/code-runtime-worker/tsdown.config.ts b/packages/code-runtime/code-runtime-worker/tsdown.config.ts index dac6f12a65..1c40637722 100644 --- a/packages/code-runtime/code-runtime-worker/tsdown.config.ts +++ b/packages/code-runtime/code-runtime-worker/tsdown.config.ts @@ -1,10 +1,9 @@ import { defineConfig } from 'tsdown' /** - * Build the plugin, reusable runtime host, and worker as separate bundles. The - * sibling `worker.cjs` is loaded by file and must be CommonJS for pkg's VFS - * Worker hook. Separate builds inline shared implementation instead of - * emitting an unlisted chunk outside the exact `files` whitelist. + * Build the index and worker as separate single-entry bundles. The sibling `worker.cjs` is loaded + * by file and must be CommonJS for pkg's VFS Worker hook. A multi-entry build emits an unlisted + * shared chunk omitted by the package's exact `files` whitelist; separate builds inline it. */ export default defineConfig([ { @@ -17,16 +16,6 @@ export default defineConfig([ dts: false, clean: false, }, - { - entry: ['lib/types/runtime-host.js'], - outDir: 'lib', - format: ['esm'], - platform: 'node', - target: 'es2024', - fixedExtension: false, - dts: false, - clean: false, - }, { entry: ['lib/types/worker.js'], outDir: 'lib', diff --git a/packages/code-runtime/code-runtime/README.md b/packages/code-runtime/code-runtime/README.md index d58508cd08..bb1c20d00a 100644 --- a/packages/code-runtime/code-runtime/README.md +++ b/packages/code-runtime/code-runtime/README.md @@ -34,5 +34,5 @@ No direct invalidation; the named consumer owns any request-prefix changes. - **`run()` is one-shot** — `logs` arrive only on the resolved `CodeRunResult`; the seam exposes no streaming-log or progress surface for a live program's output. - **A persistent REPL-style kernel is recorded future work** — the no-state-between-runs contract stands until a persistent-kernel backend brings its own logging story ([Code Mode Agent Note](../../../.agents/notes/implemented/feature/2026-06-15-code-mode.md)). -- **No runtime claims a hard security boundary** — both shipped implementations use fresh worker threads; the subprocess backend can place them inside a stronger execution world, but no runtime reports `'container'` today. -- **Intermediate binding values are implementation-bounded** — the direct worker backend has no per-binding byte cap; the subprocess backend bounds each bridge frame, but repeated or concurrent binding traffic remains subject to process memory. +- **Only the worker-thread backend ships** — `'process'`/`'container'` are declared well-known `isolation` values with no implementation; a hard security boundary awaits a container backend. +- **Intermediate binding values have no byte cap** — implementations remain subject to structured-clone cost and process memory, while a provider or executor may already have imposed its own acquisition bound. diff --git a/packages/code-runtime/code-runtime/tests/contract.ts b/packages/code-runtime/code-runtime/tests/contract.ts deleted file mode 100644 index 8fee76f68a..0000000000 --- a/packages/code-runtime/code-runtime/tests/contract.ts +++ /dev/null @@ -1,853 +0,0 @@ -import { describe, expect, it } from 'vitest' -import type { - CodeBindingFunction, - CodeBindingNamespace, - CodeRunResult, - CodeRuntime, -} from '@deepseek-ai/dsh-code-runtime' - -interface WorkerCodeRuntimeContractConfig { - computeMs?: number - maxWallMs?: number - maxOutputBytes?: number - maxOldGenerationSizeMb?: number -} - -interface WorkerCodeRuntimeContractHarness { - runtime: CodeRuntime - dispose: () => Promise -} - -type WorkerCodeRuntimeContractSetup = ( - config?: WorkerCodeRuntimeContractConfig, -) => Promise - -/** Convenience: one namespace `tools` with the given functions. */ -export function workerRuntimeTools( - functions: Record Promise>, -): CodeBindingNamespace[] { - return [{ - global: 'tools', - functions: functions as Record, - errorClass: { name: 'ToolCallError', memberNameProperty: 'toolName' }, - }] -} - -/** Run behavior shared by the direct and subprocess-hosted worker runtimes. */ -export function runWorkerCodeRuntimeContract( - label: string, - setup: WorkerCodeRuntimeContractSetup, -): void { - describe(`${label} — programs and bindings (real workers)`, () => { - it('registers with the seam descriptors', async () => { - const { runtime } = await setup() - expect(runtime.language).toBe('typescript') - expect(runtime.isolation).toBe('worker-thread') - }) - - it('runs TypeScript (erasable syntax), captures output in order, returns the value', async () => { - const { runtime } = await setup() - const result = await runtime.run({ - program: ` - interface Point { x: number; y: number } - const p: Point = { x: 1, y: 2 } as Point; - console.log('point', p); - process.stdout.write('raw-out\\n'); - console.warn('careful'); - return p.x + p.y; - `, - bindings: [], - }) - expect(result.error).toBeUndefined() - expect(result.value).toBe(3) - expect(result.logs).toEqual(['point { x: 1, y: 2 }', 'raw-out\n', 'careful']) - }) - - it('bridges binding calls both ways and rejects the program-side call on a host rejection', async () => { - const { runtime } = await setup() - const calls: unknown[] = [] - const result = await runtime.run({ - program: ` - const first = await tools.echo({ n: 1 }); - let caught = {}; - try { await tools.fail({}) } catch (error) { caught = { isTyped: error instanceof ToolCallError, name: error.name, toolName: error.toolName, message: error.message } } - let caughtRaw = {}; - try { await tools.failRaw({}) } catch (error) { caughtRaw = { name: error.name, toolName: error.toolName, message: error.message } } - return { first, caught, caughtRaw }; - `, - bindings: workerRuntimeTools({ - echo: async (args) => { calls.push(args); return { echoed: args } }, - fail: async () => { throw new Error('nope') }, - // A non-Error throw: the host renders it, the program still catches. - failRaw: async () => { throw 'raw-nope' }, - }), - }) - expect(result.error).toBeUndefined() - expect(result.value).toEqual({ - first: { echoed: { n: 1 } }, - caught: { isTyped: true, name: 'ToolCallError', toolName: 'fail', message: 'nope' }, - caughtRaw: { name: 'ToolCallError', toolName: 'failRaw', message: 'raw-nope' }, - }) - expect(calls).toEqual([{ n: 1 }]) - }) - - it('materializes a typed rejection from a generic namespace descriptor', async () => { - const { runtime } = await setup() - const result = await runtime.run({ - program: ` - try { await helpers.fail({}) } catch (error) { - return { - isTyped: error instanceof HelperCallError, - name: error.name, - helperName: error.helperName, - message: error.message, - }; - } - `, - bindings: [{ - global: 'helpers', - functions: { fail: async () => { throw new Error('nope') } }, - errorClass: { name: 'HelperCallError', memberNameProperty: 'helperName' }, - }], - }) - expect(result.value).toEqual({ - isTyped: true, - name: 'HelperCallError', - helperName: 'fail', - message: 'nope', - }) - }) - - it('bridges a deeply nested lossless JSON argument, resolution, and completion', async () => { - const { runtime } = await setup() - const result = await runtime.run({ - program: ` - let value = 'leaf'; - for (let depth = 0; depth < 3_000; depth++) value = [value]; - return await tools.echo(value); - `, - bindings: workerRuntimeTools({ echo: async args => args }), - }) - - expect(result.error).toBeUndefined() - let cursor = result.value - for (let depth = 0; depth < 3_000; depth++) { - expect(Array.isArray(cursor)).toBe(true) - cursor = Array.isArray(cursor) ? cursor[0] : undefined - } - expect(cursor).toBe('leaf') - }, 15_000) - - it('reports non-erasable syntax as an exception without spawning a worker', async () => { - const { runtime } = await setup() - const result = await runtime.run({ program: 'enum E { A }\nreturn 1', bindings: [] }) - expect(result.error?.kind).toBe('exception') - expect(result.error?.message).toMatch(/enum|strip/i) - }) - - it('reports a runtime throw as an exception with the message', async () => { - const { runtime } = await setup() - const result = await runtime.run({ program: 'throw new Error("kaboom")', bindings: [] }) - expect(result.error?.kind).toBe('exception') - expect(result.error?.message).toContain('kaboom') - }) - - it('gives the program an EMPTY environment', async () => { - const { runtime } = await setup() - const result = await runtime.run({ program: 'return JSON.stringify(process.env)', bindings: [] }) - expect(result.value).toBe('{}') - }) - - it('rejects a non-lossless completion instead of replacing it with rendered text', async () => { - const { runtime } = await setup() - const result = await runtime.run({ program: 'return { f: () => 1 }', bindings: [] }) - expect(result.value).toBeUndefined() - expect(result.error).toEqual({ kind: 'invalid-output', message: 'program completion must be lossless JSON' }) - }) - - it('completes a program that returns nothing with no value at all', async () => { - const { runtime } = await setup() - const result = await runtime.run({ program: 'const x = 1', bindings: [] }) - expect(result.error).toBeUndefined() - expect('value' in result).toBe(false) - }) - - it('keeps logs streamed before a failure', async () => { - const { runtime } = await setup() - const result = await runtime.run({ - program: 'console.log("before"); throw new Error("after-log")', - bindings: [], - }) - expect(result.error?.kind).toBe('exception') - expect(result.logs).toContain('before') - }) - }) - - describe(`${label} — budgets and containment (real workers)`, () => { - it('ends a hot loop at the compute budget — including behind a pending decoy dispatch', async () => { - const { runtime } = await setup({ computeMs: 300, maxWallMs: 30_000 }) - const result = await runtime.run({ - // The decoy: fire a call at a never-resolving binding WITHOUT awaiting, - // then spin. Host-side pending-call bookkeeping would pause a naive - // budget here; measured busy time cannot be fooled. - program: 'void tools.slow({}); for (;;) {}', - bindings: workerRuntimeTools({ slow: () => new Promise(() => {}) }), - }) - expect(result.error?.kind).toBe('timeout') - expect(result.error?.message).toContain('compute budget') - }, 15_000) - - it('does not charge time spent awaiting a slow binding against the compute budget', async () => { - // Keep the binding delay above the compute allowance while leaving enough - // headroom for worker bootstrap on loaded CI hosts. - const { runtime } = await setup({ computeMs: 1_000, maxWallMs: 30_000 }) - const result = await runtime.run({ - program: 'return await tools.slow({})', - bindings: workerRuntimeTools({ slow: () => new Promise(resolve => setTimeout(() => { resolve('slow-done') }, 1_500)) }), - }) - expect(result.error).toBeUndefined() - expect(result.value).toBe('slow-done') - }, 15_000) - - it('ends an idle-forever run at the wall-clock ceiling', async () => { - const { runtime } = await setup({ computeMs: 30_000, maxWallMs: 400 }) - const result = await runtime.run({ - program: 'await tools.never({}); return 1', - bindings: workerRuntimeTools({ never: () => new Promise(() => {}) }), - }) - expect(result.error?.kind).toBe('timeout') - expect(result.error?.message).toContain('wall-clock ceiling') - }, 15_000) - - it('reports an abort mid-run and stops the worker', async () => { - const { runtime } = await setup() - const controller = new AbortController() - setTimeout(() => { controller.abort('user-cancel') }, 150) - const result = await runtime.run({ program: 'for (;;) {}', bindings: [], signal: controller.signal }) - expect(result.error).toEqual({ kind: 'abort', message: 'user-cancel' }) - }, 15_000) - - it('reports a pre-aborted signal without spawning', async () => { - const { runtime } = await setup() - const controller = new AbortController() - controller.abort('too-late') - const result = await runtime.run({ program: 'return 1', bindings: [], signal: controller.signal }) - expect(result.error).toEqual({ kind: 'abort', message: 'too-late' }) - }) - - it('applies the outer-output cap to failures before worker startup', async () => { - const capped = await setup({ maxOutputBytes: 64 }) - const controller = new AbortController() - controller.abort('A'.repeat(1_000)) - const aborted = await capped.runtime.run({ program: 'return 1', bindings: [], signal: controller.signal }) - expect(aborted).toEqual({ logs: [], error: { kind: 'output-limit', message: 'outer output exceeded 64 bytes' } }) - - const minimal = await setup({ maxOutputBytes: 4 }) - const invalid = await minimal.runtime.run({ program: 'enum E { A }\nreturn 1', bindings: [] }) - expect(invalid.error?.kind).toBe('output-limit') - expect(Buffer.byteLength(JSON.stringify(invalid.logs), 'utf8') + Buffer.byteLength(JSON.stringify(invalid.error?.message), 'utf8')).toBeLessThanOrEqual(4) - }) - - it('drops a binding resolution that lands after the run settled', async () => { - const { runtime } = await setup() - const controller = new AbortController() - let replyDelivered!: Promise - const result = await runtime.run({ - program: 'void tools.late({}); for (;;) {}', - bindings: workerRuntimeTools({ - // Anchored on invocation: abort 100ms after the call reaches the - // host, resolve 400ms after — by then the run has settled, so the - // resolution's reply hits the post-settlement drop. - late: () => new Promise((resolve) => { - setTimeout(() => { controller.abort('cancel-now') }, 100) - replyDelivered = new Promise(done => setTimeout(() => { resolve('too-late'); done() }, 400)) - }), - }), - signal: controller.signal, - }) - expect(result.error).toEqual({ kind: 'abort', message: 'cancel-now' }) - // Let the late resolution actually fire so its reply executes instead of - // being cancelled with the test. - await replyDelivered - }, 15_000) - - it('contains an OOM under resourceLimits as worker-exit, host process healthy', async () => { - const { runtime } = await setup({ maxOldGenerationSizeMb: 32 }) - const result = await runtime.run({ - program: 'const hog = []; for (;;) hog.push(new Array(1e6).fill(1));', - bindings: [], - }) - expect(result.error?.kind).toBe('worker-exit') - // And the host is fine: run something else. - const after = await runtime.run({ program: 'return "alive"', bindings: [] }) - expect(after.value).toBe('alive') - }, 30_000) - - it('reports a worker that exits before publishing a completion', async () => { - const { runtime } = await setup() - const result = await runtime.run({ program: 'process.exit(7)', bindings: [] }) - expect(result).toEqual({ - logs: [], - error: { kind: 'worker-exit', message: 'worker exited with code 7 before completing' }, - }) - }) - - it('fails runaway log output explicitly while retaining a bounded prefix', async () => { - const { runtime } = await setup({ maxOutputBytes: 300 }) - const result = await runtime.run({ - program: 'for (let i = 0; i < 1000; i++) console.log("spam line", i); return 1', - bindings: [], - }) - expect(result.error).toEqual({ kind: 'output-limit', message: 'outer output exceeded 300 bytes' }) - expect(result.value).toBeUndefined() - expect(result.logs.length).toBeGreaterThan(0) - expect(Buffer.byteLength(JSON.stringify(result.logs), 'utf8')).toBeLessThan(300) - }) - - it('retains a fitting prefix when one oversized log is the first output', async () => { - const { runtime } = await setup({ maxOutputBytes: 96 }) - const result = await runtime.run({ - program: 'console.log(`start-${`😀"\\\\\\n`.repeat(100)}`); return null', - bindings: [], - }) - expect(result.error).toEqual({ kind: 'output-limit', message: 'outer output exceeded 96 bytes' }) - expect(result.logs).toHaveLength(1) - expect(result.logs[0]?.startsWith('start-')).toBe(true) - expect(Buffer.byteLength(JSON.stringify(result.logs), 'utf8') - + Buffer.byteLength(JSON.stringify(result.error?.message), 'utf8')).toBeLessThanOrEqual(96) - }) - - it('fails an oversized return value without substituting a string', async () => { - const { runtime } = await setup({ maxOutputBytes: 64 }) - const result = await runtime.run({ program: 'return "y".repeat(10_000)', bindings: [] }) - expect(result.value).toBeUndefined() - expect(result.error).toEqual({ kind: 'output-limit', message: 'outer output exceeded 64 bytes' }) - }) - - it('uses UTF-8 serialized bytes at the exact completion boundary', async () => { - const exact = await setup({ maxOutputBytes: 7 }) - const exactResult = await exact.runtime.run({ program: 'return "€"', bindings: [] }) - // [] costs two bytes and JSON serialization of "€" costs five. - expect(exactResult).toEqual({ logs: [], value: '€' }) - - const over = await setup({ maxOutputBytes: 6 }) - const overResult = await over.runtime.run({ program: 'return "€"', bindings: [] }) - expect(overResult.error?.kind).toBe('output-limit') - }) - - it('accounts logs and completion in one exact combined ledger', async () => { - // JSON(["abc"]) is seven bytes and JSON("xy") is four. - const exact = await setup({ maxOutputBytes: 11 }) - expect(await exact.runtime.run({ program: 'console.log("abc"); return "xy"', bindings: [] })) - .toEqual({ logs: ['abc'], value: 'xy' }) - - const over = await setup({ maxOutputBytes: 10 }) - const result = await over.runtime.run({ program: 'console.log("abc"); return "xy"', bindings: [] }) - expect(result.value).toBeUndefined() - expect(result.error?.kind).toBe('output-limit') - expect(Buffer.byteLength(JSON.stringify(result.logs), 'utf8') + Buffer.byteLength(JSON.stringify(result.error?.message), 'utf8')).toBeLessThanOrEqual(10) - }) - - it('accounts logs and exception diagnostics before the worker port boundary', async () => { - // JSON(["abc"]) is seven bytes and JSON("xy") is four. - const exact = await setup({ maxOutputBytes: 11 }) - expect(await exact.runtime.run({ program: 'console.log("abc"); throw "xy"', bindings: [] })) - .toEqual({ logs: ['abc'], error: { kind: 'exception', message: 'xy' } }) - - const over = await setup({ maxOutputBytes: 10 }) - const result = await over.runtime.run({ program: 'console.log("abc"); throw "xy"', bindings: [] }) - expect(result.error?.kind).toBe('output-limit') - expect(Buffer.byteLength(JSON.stringify(result.logs), 'utf8') - + Buffer.byteLength(JSON.stringify(result.error?.message), 'utf8')).toBeLessThanOrEqual(10) - }) - - it('does not send a giant Error stack across the worker port', async () => { - const { runtime } = await setup({ maxOutputBytes: 64 }) - const result = await runtime.run({ - program: 'throw new Error("x".repeat(1_000_000))', - bindings: [], - }) - expect(result).toEqual({ - logs: [], - error: { kind: 'output-limit', message: 'outer output exceeded 64 bytes' }, - }) - }) - - it('completes a program that awaits its write callback, capturing the chunk', async () => { - // Node's write(chunk[, encoding][, callback]) contract: dropping the - // callback would leave this promise pending until the wall ceiling and - // misreport a completed program as a timeout. - const { runtime } = await setup({ maxWallMs: 2_000 }) - const result = await runtime.run({ - program: 'await new Promise(resolve => process.stdout.write("flushed", resolve)); return "done"', - bindings: [], - }) - expect(result.error).toBeUndefined() - expect(result.value).toBe('done') - expect(result.logs).toContain('flushed') - }) - - it('returns a large JSON container exactly when the outer cap permits it', async () => { - const { runtime } = await setup() - const result = await runtime.run({ program: 'return new Array(50_000).fill(7)', bindings: [] }) - expect(result.error).toBeUndefined() - expect(result.value).toEqual(new Array(50_000).fill(7)) - }) - - it('returns an exact completion at the default 64 MiB combined boundary', async () => { - const { runtime } = await setup() - // [] costs two bytes and the JSON string contributes two quotes, leaving - // exactly this many payload bytes under the 67_108_864-byte default. - const result = await runtime.run({ program: 'return "x".repeat(67_108_860)', bindings: [] }) - expect(result.error).toBeUndefined() - expect(result.logs).toEqual([]) - expect(result.value).toHaveLength(67_108_860) - }, 60_000) - - it('fails one byte over the default 64 MiB combined boundary', async () => { - const { runtime } = await setup() - const result = await runtime.run({ program: 'return "x".repeat(67_108_861)', bindings: [] }) - expect(result.value).toBeUndefined() - expect(result.error).toEqual({ kind: 'output-limit', message: 'outer output exceeded 67108864 bytes' }) - }, 60_000) - - it('drains pipe output queued before terminal worker teardown completes', async () => { - const { runtime } = await setup({ maxOutputBytes: 200_000 }) - const payload = `late-pipe-${'x'.repeat(100_000)}` - const result = await runtime.run({ - program: ` - const { parentPort } = await import('node:worker_threads'); - const write = (text) => Object.getPrototypeOf(process.stdout).write.call(process.stdout, text); - write('late-pipe-' + 'x'.repeat(100_000)); - parentPort.postMessage({ type: 'done', value: ['done'] }); - for (;;) {} - `, - bindings: [], - }) - expect(result.error).toBeUndefined() - expect(result.value).toBe('done') - expect(result.logs.join('') === payload).toBe(true) - }, 15_000) - }) - - describe(`${label} — hostile programs (real workers)`, () => { - it('survives forged port traffic: unknown binding names, duplicate ids, junk shapes', async () => { - const { runtime } = await setup() - const result = await runtime.run({ - program: ` - const { parentPort } = await import('node:worker_threads'); - parentPort.postMessage({ type: 'call', id: 7777, global: 'tools', name: 'missing', args: {} }); - parentPort.postMessage({ type: 'call', id: 7777, global: 'tools', name: 'missing', args: {} }); - parentPort.postMessage({ type: 'call', id: 7778, global: 'tools', name: 'constructor', args: {} }); - parentPort.postMessage({ type: 'junk' }); - return await tools.real({}); - `, - bindings: workerRuntimeTools({ real: async () => 'still-works' }), - }) - expect(result.error).toBeUndefined() - expect(result.value).toBe('still-works') - }) - - it('survives arbitrary junk on the port: non-objects, junk types, malformed calls, logs, and dones', async () => { - const { runtime } = await setup() - const result = await runtime.run({ - program: ` - const { parentPort } = await import('node:worker_threads'); - for (const junk of [ - null, 42, 'junk', [], - { type: 'nope' }, - { type: 'call' }, - { type: 'call', id: 'x', global: 'tools', name: 'real', args: {} }, - { type: 'call', id: 1e9, global: 7, name: 'real', args: {} }, - { type: 'call', id: 1e9, global: 'tools', name: 7, args: {} }, - { type: 'log' }, - { type: 'log', text: null }, - { type: 'log', text: 7 }, - { type: 'log', text: {} }, - { type: 'done', error: 5 }, - { type: 'done', error: { kind: 'exception', message: 5 } }, - { type: 'done', error: { kind: 'invented', message: 'bad kind' } }, - ]) parentPort.postMessage(junk); - return await tools.real({}); - `, - bindings: workerRuntimeTools({ real: async () => 'still-works' }), - }) - expect(result.error).toBeUndefined() - expect(result.value).toBe('still-works') - expect(result.logs).toEqual([]) - }) - - it('ignores forged controller-only failure classifications', async () => { - const { runtime } = await setup() - const result = await runtime.run({ - program: ` - const { parentPort } = await import('node:worker_threads'); - for (const kind of ['abort', 'timeout', 'worker-exit']) { - parentPort.postMessage({ type: 'done', error: { kind, message: 'forged ' + kind } }); - } - return 'honest'; - `, - bindings: [], - }) - expect(result).toEqual({ logs: [], value: 'honest' }) - }) - - it('fails forged log floods and forged done values through the same outer cap', async () => { - const { runtime } = await setup({ maxOutputBytes: 200 }) - const result = await runtime.run({ - // Forged messages bypass worker-side capture and completion checks; - // the outer ledger must still contain them. - program: ` - const { parentPort } = await import('node:worker_threads'); - for (let i = 0; i < 50; i++) parentPort.postMessage({ type: 'log', text: 'F'.repeat(100), forged: true }); - parentPort.postMessage({ type: 'done', value: ['V'.repeat(100000)] }); - for (;;) {} - `, - bindings: [], - }) - expect(result.value).toBeUndefined() - expect(result.error).toEqual({ kind: 'output-limit', message: 'outer output exceeded 200 bytes' }) - expect(Buffer.byteLength(JSON.stringify(result.logs), 'utf8')).toBeLessThan(200) - }) - - it('re-caps an oversized forged done value at the host boundary', async () => { - const { runtime } = await setup({ maxOutputBytes: 64 }) - const result = await runtime.run({ - program: ` - const { parentPort } = await import('node:worker_threads'); - parentPort.postMessage({ type: 'done', value: ['V'.repeat(100_000)] }); - for (;;) {} - `, - bindings: [], - }) - expect(result).toEqual({ - logs: [], - error: { kind: 'output-limit', message: 'outer output exceeded 64 bytes' }, - }) - }) - - it('drops a malformed forged done carrying both value and error', async () => { - const { runtime } = await setup() - const result = await runtime.run({ - program: ` - const { parentPort } = await import('node:worker_threads'); - parentPort.postMessage({ type: 'done', value: 'lied', error: { kind: 'exception', message: 'fake failure' } }); - return 'honest'; - `, - bindings: [], - }) - expect(result).toEqual({ logs: [], error: { kind: 'exception', message: 'fake failure' } }) - }) - - it('contains a deeply nested forged completion without overflowing the host meter', async () => { - const { runtime } = await setup() - const result = await runtime.run({ - program: ` - const { parentPort } = await import('node:worker_threads'); - const value = []; - for (let depth = 0; depth < 3_000; depth++) value.push({ kind: 'array', length: 1 }); - value.push(null); - setTimeout(() => { parentPort.postMessage({ type: 'done', value }) }, 25); - // Prevent bootstrap's normal undefined completion from racing the forged terminal. - await new Promise(() => {}); - `, - bindings: [], - }) - expect(result.error).toBeUndefined() - let value = result.value - let depth = 0 - while (Array.isArray(value)) { - expect(value).toHaveLength(1) - value = value[0] - depth += 1 - } - expect(depth).toBe(3_000) - expect(value).toBeNull() - }, 15_000) - - it('turns forged over-limit error text into output-limit at the host', async () => { - const { runtime } = await setup({ maxOutputBytes: 64 }) - const result = await runtime.run({ - program: ` - const { parentPort } = await import('node:worker_threads'); - parentPort.postMessage({ type: 'done', error: { kind: 'exception', message: '€'.repeat(1000) } }); - for (;;) {} - `, - bindings: [], - }) - expect(result.error).toEqual({ kind: 'output-limit', message: 'outer output exceeded 64 bytes' }) - }) - - it('answers a binding whose resolution is not lossless JSON with a typed failure reply', async () => { - const { runtime } = await setup() - const result = await runtime.run({ - program: 'try { await tools.bad({}) } catch (error) { return { name: error.name, toolName: error.toolName, message: error.message } }', - bindings: workerRuntimeTools({ bad: async () => (() => 1) }), - }) - expect(result.value).toEqual({ name: 'ToolCallError', toolName: 'bad', message: 'binding resolution must be lossless JSON' }) - }) - - it('rejects lossy binding arguments in the worker before invoking the host binding', async () => { - const { runtime } = await setup() - let calls = 0 - const result = await runtime.run({ - program: ` - const decorated = [1]; Object.defineProperty(decorated, 'extra', { value: true }); - const values = [new Date(), decorated, () => 1]; - const failures = []; - for (const value of values) { - try { await tools.never(value) } catch (error) { - failures.push({ typed: error instanceof ToolCallError, name: error.name, toolName: error.toolName, message: error.message }); - } - } - return failures; - `, - bindings: workerRuntimeTools({ never: async () => { calls += 1; return null } }), - }) - expect(calls).toBe(0) - expect(result.value).toEqual(new Array(3).fill({ - typed: true, - name: 'ToolCallError', - toolName: 'never', - message: 'binding arguments must be lossless JSON', - })) - }) - - it('rejects intrinsic-looking exotic objects as arguments and completions', async () => { - const { runtime } = await setup() - let calls = 0 - const forgeObject = ` - const prototype = Object.create(null); - const SpoofedObject = function Object() {}; - SpoofedObject.prototype = prototype; - Object.defineProperty(prototype, 'constructor', { value: SpoofedObject }); - const forged = Object.assign(Object.create(prototype), { value: 1 }); - Function.prototype.toString = () => 'function Object() { [native code] }'; - ` - const argument = await runtime.run({ - program: `${forgeObject} - try { await tools.never(forged) } catch (error) { - return { typed: error instanceof ToolCallError, name: error.name, toolName: error.toolName, message: error.message }; - } - `, - bindings: workerRuntimeTools({ never: async () => { calls += 1; return null } }), - }) - expect(calls).toBe(0) - expect(argument.value).toEqual({ - typed: true, - name: 'ToolCallError', - toolName: 'never', - message: 'binding arguments must be lossless JSON', - }) - - const completion = await runtime.run({ program: `${forgeObject}\nreturn forged`, bindings: [] }) - expect(completion).toEqual({ - logs: [], - error: { kind: 'invalid-output', message: 'program completion must be lossless JSON' }, - }) - }) - - it('preserves binding and completion JSON after model code mutates boundary globals', async () => { - const { runtime } = await setup() - const result = await runtime.run({ - program: ` - const arrayPrototype = Array.prototype; - const objectPrototype = Object.prototype; - const setPrototype = Set.prototype; - const stringPrototype = String.prototype; - Array.isArray = () => false; - arrayPrototype.at = arrayPrototype.includes = arrayPrototype.pop = arrayPrototype.push = () => { throw new Error('mutated array method') }; - Object.defineProperty = Object.getOwnPropertyDescriptor = Object.getPrototypeOf = Object.keys = () => { throw new Error('mutated object method') }; - Object.hasOwn = () => false; - Object.is = () => true; - objectPrototype.propertyIsEnumerable = () => false; - Number.isFinite = Number.isSafeInteger = () => false; - Reflect.apply = Reflect.ownKeys = () => { throw new Error('mutated reflect method') }; - setPrototype.add = setPrototype.delete = setPrototype.has = () => { throw new Error('mutated set method') }; - stringPrototype.charCodeAt = stringPrototype.codePointAt = stringPrototype.slice = () => { throw new Error('mutated string method') }; - Buffer.byteLength = () => 0; - Function.prototype.toString = () => 'mutated'; - objectPrototype.get = () => undefined; - objectPrototype.constructor = arrayPrototype.constructor = null; - globalThis.Array = globalThis.Buffer = globalThis.Error = globalThis.Function = globalThis.Number = globalThis.Object = globalThis.Reflect = globalThis.Set = globalThis.String = undefined; - const echoed = await tools.echo({ request: ['€', 1] }); - let failure; - try { await tools.fail({}) } catch (error) { - failure = { typed: error instanceof ToolCallError, name: error.name, toolName: error.toolName, message: error.message }; - } - return { echoed, failure, completion: { ok: true, amount: 42 } }; - `, - bindings: workerRuntimeTools({ echo: async args => args, fail: async () => { throw new Error('nope') } }), - }) - expect(result).toEqual({ - logs: [], - value: { - echoed: { request: ['€', 1] }, - failure: { typed: true, name: 'ToolCallError', toolName: 'fail', message: 'nope' }, - completion: { ok: true, amount: 42 }, - }, - }) - }) - - it('rejects forged lossy binding arguments again at the host boundary', async () => { - const { runtime } = await setup() - let calls = 0 - const result = await runtime.run({ - program: ` - const { parentPort } = await import('node:worker_threads'); - const forged = (id, args) => new Promise((resolve) => { - const receive = (message) => { - if (message?.type !== 'reply' || message.id !== id) return; - parentPort.off('message', receive); - resolve(message); - }; - parentPort.on('message', receive); - parentPort.postMessage({ type: 'call', id, global: 'tools', name: 'never', args }); - }); - const sparse = []; sparse.length = 1; - const cycle = {}; cycle.self = cycle; - return await Promise.all([ - forged(8001, new Date()), - forged(8002, -0), - forged(8003, sparse), - forged(8004, cycle), - ]); - `, - bindings: workerRuntimeTools({ never: async () => { calls += 1; return null } }), - }) - expect(calls).toBe(0) - expect(result.value).toEqual([8001, 8002, 8003, 8004].map(id => ({ - type: 'reply', - id, - ok: false, - message: 'binding arguments must be lossless JSON', - }))) - }) - - it('contains throwing getters while snapshotting binding resolutions', async () => { - const { runtime } = await setup() - const result = await runtime.run({ - program: 'try { await tools.bad({}) } catch (error) { return { name: error.name, toolName: error.toolName, message: error.message } }', - bindings: workerRuntimeTools({ bad: async () => Object.defineProperty({}, 'bad', { enumerable: true, get() { throw new Error('getter exploded') } }) }), - }) - expect(result.value).toEqual({ name: 'ToolCallError', toolName: 'bad', message: 'binding resolution must be lossless JSON' }) - }) - - it('revalidates a forged lossy completion at the host boundary', async () => { - const { runtime } = await setup() - const result = await runtime.run({ - program: ` - const { parentPort } = await import('node:worker_threads'); - parentPort.postMessage({ type: 'done', value: -0 }); - for (;;) {} - `, - bindings: [], - }) - expect(result).toEqual({ logs: [], error: { kind: 'invalid-output', message: 'program completion must be lossless JSON' } }) - }) - - it('honors a forged worker-side output-limit signal', async () => { - const { runtime } = await setup() - const result = await runtime.run({ - program: ` - const { parentPort } = await import('node:worker_threads'); - parentPort.postMessage({ type: 'output-limit' }); - for (;;) {} - `, - bindings: [], - }) - expect(result).toEqual({ logs: [], error: { kind: 'output-limit', message: 'outer output exceeded 67108864 bytes' } }) - }) - - it('exposes binding names that collide with Object.prototype as ordinary functions', async () => { - const { runtime } = await setup() - const result = await runtime.run({ - program: 'return [await tools["__proto__"]({}), await tools["constructor"]({}), typeof tools["hasOwnProperty"]]', - // Computed keys: a literal `'__proto__': …` entry would SET the record's - // prototype instead of declaring a binding of that name. - bindings: workerRuntimeTools({ ['__proto__']: async () => 'proto-ok', ['constructor']: async () => 'ctor-ok' }), - }) - expect(result.value).toEqual(['proto-ok', 'ctor-ok', 'undefined']) - }) - }) - - describe(`${label} — seam misuse and lifecycle`, () => { - it('rejects invalid and duplicate binding globals loudly', async () => { - const { runtime } = await setup() - const cases: [string, RegExp][] = [ - ['not valid!', /not a usable identifier/], - ['await', /not a usable identifier/], - ['console', /duplicate binding global/], - ] - for (const [global, message] of cases) { - await expect(runtime.run({ program: 'return 1', bindings: [{ global, functions: {} }] })).rejects.toThrow(message) - } - await expect(runtime.run({ - program: 'return 1', - bindings: [{ global: 'tools', functions: {} }, { global: 'tools', functions: {} }], - })).rejects.toThrow(/duplicate binding global/) - - await expect(runtime.run({ - program: 'return typeof ToolCallError', - bindings: [{ global: 'ToolCallError', functions: {} }], - })).resolves.toMatchObject({ value: 'object' }) - }) - - it('rejects malformed or colliding binding error-class declarations', async () => { - const { runtime } = await setup() - const run = async (bindings: CodeBindingNamespace[]) => await runtime.run({ program: 'return 1', bindings }) - const namespace = (global: string, name: string, memberNameProperty = 'memberName'): CodeBindingNamespace => ({ - global, - functions: {}, - errorClass: { name, memberNameProperty }, - }) - - await expect(run([namespace('tools', 'not valid!')])).rejects.toThrow(/error class.*not a usable identifier/) - await expect(run([namespace('tools', 'await')])).rejects.toThrow(/error class.*not a usable identifier/) - await expect(run([namespace('tools', 'console')])).rejects.toThrow(/duplicate injected global/) - await expect(run([namespace('tools', 'tools')])).rejects.toThrow(/duplicate injected global/) - await expect(run([ - namespace('tools', 'CallError'), - namespace('helpers', 'CallError'), - ])).rejects.toThrow(/duplicate injected global/) - await expect(run([namespace('tools', 'CallError', '')])).rejects.toThrow(/member property.*not usable/) - await expect(run([namespace('tools', 'CallError', 'message')])).rejects.toThrow(/member property.*not usable/) - }) - - it('rejects config values that are not positive numbers', async () => { - await expect(setup({ computeMs: -1 })).rejects.toThrow(/positive number/) - }) - - it('rejects a maxWallMs above Node\'s maximum timer delay', async () => { - // setTimeout clamps a delay past 2^31-1 ms to 1 ms, so the positivity check - // alone would accept a 25-day ceiling that expires on the first tick. - await expect(setup({ maxWallMs: 2_147_483_648 })) - .rejects.toThrow(/maxWallMs must be at most 2147483647/) - // The boundary itself is usable. - await expect(setup({ maxWallMs: 2_147_483_647 })).resolves.toBeTruthy() - }) - - it('requires maxOutputBytes to fit the smallest counted outer payloads', async () => { - await expect(setup({ maxOutputBytes: 3 })).rejects.toThrow(/safe integer of at least 4/) - await expect(setup({ maxOutputBytes: 4.5 })).rejects.toThrow(/safe integer of at least 4/) - }) - - it('keeps runs isolated: no state survives from one run to the next', async () => { - const { runtime } = await setup() - await runtime.run({ program: 'globalThis.leak = "value"; return 1', bindings: [] }) - const second = await runtime.run({ program: 'return typeof globalThis.leak', bindings: [] }) - expect(second.value).toBe('undefined') - }) - - it('disposal aborts in-flight runs, awaits worker exit, and rejects later runs', async () => { - const { runtime, dispose } = await setup() - const inflight: Promise = runtime.run({ program: 'for (;;) {}', bindings: [] }) - // Give the worker a moment to actually start spinning. - await new Promise(resolve => setTimeout(resolve, 200)) - await dispose() - const result = await inflight - expect(result.error).toEqual({ kind: 'abort', message: 'runtime disposed' }) - await expect(runtime.run({ program: 'return 1', bindings: [] })).rejects.toThrow(/after disposal/) - }, 15_000) - }) -} diff --git a/packages/subprocess/README.i18n.yaml b/packages/subprocess/README.i18n.yaml index af424b8809..0ebb5bd4af 100644 --- a/packages/subprocess/README.i18n.yaml +++ b/packages/subprocess/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/subprocess/README.md -README.md: 5ea1ae3681f9340cfba798e659f454d6d05fdbfe -README.zh.md: 4e1cb9472d6e90062ecde61db4200a79a56b16f1 +README.md: f2b19436da40feb14d067e2cfc706222625680b5 +README.zh.md: 938312448dd5c0a691ed07ddc9843cf2c4445637 diff --git a/packages/subprocess/README.md b/packages/subprocess/README.md index 5ea1ae3681..f2b19436da 100644 --- a/packages/subprocess/README.md +++ b/packages/subprocess/README.md @@ -2,11 +2,11 @@ English | [中文](README.zh.md) -The shared process substrate for one execution world: canonical cwd, executable lookup, fully-specified managed child-process trees with raw or collected stdio, and one deep terminal-process primitive that owns PTY allocation, foreground groups, and provider-observable session cleanup. Command defaulting, shell semantics, deadlines, protocol framing, readiness, and presentation stay with consumers — the [bash executors](../bash/README.md), [LSP host](../lsp/README.md), [PTY shell backend](../pty/README.md), [subprocess code runtime](../code-runtime/code-runtime-subprocess/README.md), and [ACP subagent backend](../subagent/README.md). See the [subprocess seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md). +The shared process substrate for one execution world: executable lookup, fully-specified managed child-process trees with raw or collected stdio, and one deep terminal-process primitive that owns PTY allocation, foreground groups, and provider-observable session cleanup. Command defaulting, shell semantics, deadlines, protocol framing, readiness, and presentation stay with consumers — the [bash executors](../bash/README.md), [LSP host](../lsp/README.md), [PTY shell backend](../pty/README.md), and [ACP subagent backend](../subagent/README.md). See the [subprocess seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md). | Package | ctx key | Role | |---|---|---| -| [`subprocess`](subprocess/README.md) (`@deepseek-ai/dsh-subprocess`) | `ctx.subprocess` | The seam: execution-world coordinates and executable lookup, ordinary managed spawns, the terminal-process primitive, handle lifecycles, and shared environment/output vocabulary | +| [`subprocess`](subprocess/README.md) (`@deepseek-ai/dsh-subprocess`) | `ctx.subprocess` | The seam: executable lookup, ordinary managed spawns, the terminal-process primitive, handle lifecycles, and shared environment/output vocabulary | | [`subprocess-local`](subprocess-local/README.md) (`@deepseek-ai/dsh-subprocess-local`) | — | The local implementation: detached process trees, bounded collection/spill, `node-pty`, foreground/session inspection, tree signalling, and terminate-and-join disposal | The service owns process lifetime across consumer reloads; consumers own what a process means (a bash command, a future non-shell runner) and every default that shapes one. diff --git a/packages/subprocess/README.zh.md b/packages/subprocess/README.zh.md index 4e1cb9472d..938312448d 100644 --- a/packages/subprocess/README.zh.md +++ b/packages/subprocess/README.zh.md @@ -2,11 +2,11 @@ [English](README.md) | 中文 -这里集中提供一个执行世界的共享进程基底:规范化 cwd、可执行文件查找、具有原始或收集式 stdio 的完整指定受管子进程树,以及一项深层终端进程原语,负责 PTY 分配、前台进程组和提供方仍可观察到的会话成员清理。命令默认值补全、shell 语义、时限、协议分帧、就绪状态与呈现留在消费方:[bash 执行器](../bash/README.md)、[LSP 主机](../lsp/README.md)、[PTY shell 后端](../pty/README.md)、[子进程代码运行时](../code-runtime/code-runtime-subprocess/README.md)与 [ACP(Agent Client Protocol)subagent 后端](../subagent/README.md)。参见[subprocess seam Agent Note(agent 决策记录)](../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md)。 +这里集中提供一个执行世界的共享进程基底:可执行文件查找、具有原始或收集式 stdio 的完整指定受管子进程树,以及一项深层终端进程原语,负责 PTY 分配、前台进程组和提供方仍可观察到的会话成员清理。命令默认值补全、shell 语义、时限、协议分帧、就绪状态与呈现留在消费方:[bash 执行器](../bash/README.md)、[LSP 主机](../lsp/README.md)、[PTY shell 后端](../pty/README.md)与 [ACP(Agent Client Protocol)subagent 后端](../subagent/README.md)。参见[subprocess seam Agent Note(agent 决策记录)](../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md)。 | 包(package) | ctx 键 | 角色 | |---|---|---| -| [`subprocess`](subprocess/README.md)(`@deepseek-ai/dsh-subprocess`) | `ctx.subprocess` | seam 本体:执行世界坐标与可执行文件查找、普通受管 spawn、终端进程原语、句柄生命周期,以及共享的环境/输出词汇 | +| [`subprocess`](subprocess/README.md)(`@deepseek-ai/dsh-subprocess`) | `ctx.subprocess` | seam 本体:可执行文件查找、普通受管 spawn、终端进程原语、句柄生命周期,以及共享的环境/输出词汇 | | [`subprocess-local`](subprocess-local/README.md)(`@deepseek-ai/dsh-subprocess-local`) | 无 | 本地实现:detached 进程树、有界收集/spill、`node-pty`、前台/会话检查、进程树信号发送,以及先终止再等待退出的资源释放 | 即使消费方重载,进程生命周期仍由服务负责管理;消费方负责定义进程的含义(一条 bash 命令、未来的非 shell 运行器),以及决定塑造该进程的每一项默认值。 diff --git a/packages/subprocess/subprocess-local/README.i18n.yaml b/packages/subprocess/subprocess-local/README.i18n.yaml index 8b668b43f6..ee90e9e607 100644 --- a/packages/subprocess/subprocess-local/README.i18n.yaml +++ b/packages/subprocess/subprocess-local/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/subprocess/subprocess-local/README.md -README.md: 4892ad49e571662b833b1dd6d64dc966f343ae3c -README.zh.md: ca0df5ff18e9feb992cc3f6d0db90dc06cb275d8 +README.md: 85103c2634bd35b188acd71c7f037e3678a2542e +README.zh.md: 505eac4f2650e6723e042cdb1122f9ef504ad7df diff --git a/packages/subprocess/subprocess-local/README.md b/packages/subprocess/subprocess-local/README.md index 13b51ec6d1..c211ca26f1 100644 --- a/packages/subprocess/subprocess-local/README.md +++ b/packages/subprocess/subprocess-local/README.md @@ -2,7 +2,7 @@ English | [中文](README.zh.md) -Local implementation of the [`@deepseek-ai/dsh-subprocess`](../subprocess/README.md) seam. `LocalSubprocessService` resolves local executables, spawns ordinary detached process trees with explicit stdio, and implements terminal processes through `node-pty` plus platform process inspection. It has no config: every disposition, limit, terminal dimension, grace, and directory arrives from the calling seams ([`dsh-bash-local`](../../bash/bash-local/README.md), [`dsh-lsp-local`](../../lsp/lsp-local/README.md), [`dsh-pty-local`](../../pty/pty-local/README.md), and [`dsh-code-runtime-subprocess`](../../code-runtime/code-runtime-subprocess/README.md)). +Local implementation of the [`@deepseek-ai/dsh-subprocess`](../subprocess/README.md) seam. `LocalSubprocessService` resolves local executables, spawns ordinary detached process trees with explicit stdio, and implements terminal processes through `node-pty` plus platform process inspection. It has no config: every disposition, limit, terminal dimension, grace, and directory arrives from the calling seams ([`dsh-bash-local`](../../bash/bash-local/README.md), [`dsh-lsp-local`](../../lsp/lsp-local/README.md), and [`dsh-pty-local`](../../pty/pty-local/README.md)). ## Behavior (and where it came from) @@ -10,7 +10,7 @@ Local implementation of the [`@deepseek-ai/dsh-subprocess`](../subprocess/README - **Per-stream dispositions** — `'pipe'` hands the raw stream to the caller untouched (protocol framing stays consumer-owned); `'inherit'` passes the parent descriptor through; collect mode keeps the in-memory TAIL beyond its cap (errors and results cluster at the end — pi/OpenCode rationale) while the FULL stream is appended to a private temp file when a spill cap is configured — omitting `spill` keeps only the tail, the diagnostic shape. A stream larger than the spill cap discards its now-incomplete spill and returns only the marked truncated tail; spill fds are sealed at settlement, and a failed final close withholds the path rather than advertising an incomplete file. Spill files are `0600` with random names under a lazily-created `0700` per-process directory. - **Credential scrub + explicit merge** — `process.env` minus credential-shaped vars (`*KEY*`/`*SECRET*`/`*TOKEN*`) and all ambient `DSH_*` names; the spec's explicit `env` merges after that scrub with no namespace validation, so a deliberately supplied credential or current `DSH_*` fact wins while stale nested-harness identity cannot leak in ambiently. Supplied stdin is written and closed; otherwise fd 0 is `/dev/null`. See the [stdin/env Agent Note](../../../.agents/notes/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-surface.md) and [managed environment Agent Note](../../../.agents/notes/implemented/feature/2026-07-10-agent-session-identity-and-log-location.md). - **Offset-based reads** — collect-mode readers return deltas in whole-stream byte coordinates; the service never holds a cursor, so consumer-owned cursors (the bash background read path) and full-stream re-reads coexist, before and after settlement. -- **Execution-world coordinates** — `cwd` is the host process cwd, and `resolveExecutable` checks absolute files or searches the scrubbed effective PATH with platform-aware executable extensions. +- **Executable lookup** — `resolveExecutable` checks absolute files or searches the scrubbed effective PATH with platform-aware executable extensions; relative PATH entries resolve from the host process cwd. - **Terminal-process ownership** — `spawnTerminal` allocates `node-pty`, bridges UTF-8 terminal text, inspects and signals the current foreground process group, and exposes one awaited termination operation that sweeps descendants before and after terminating the top-level shell. Each foreground inspection retains exact identities from the rooted tree; Linux also enumerates the POSIX session after its leader exits. A previously observed macOS descendant and any same-session Linux member therefore remain fenced after reparenting, while pid/start identity prevents cleanup from following PID reuse. The higher PTY backend owns prompt readiness, buffers, and model-facing operations. - **Terminate-and-join disposal** — the service retains live handles only so its own disposal can escalate every running tree and await its exit; settled and spawn-failed handles leave the live set on settlement. diff --git a/packages/subprocess/subprocess-local/README.zh.md b/packages/subprocess/subprocess-local/README.zh.md index ca0df5ff18..505eac4f26 100644 --- a/packages/subprocess/subprocess-local/README.zh.md +++ b/packages/subprocess/subprocess-local/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -[`@deepseek-ai/dsh-subprocess`](../subprocess/README.md) seam 的本地实现。`LocalSubprocessService` 解析本地可执行文件,以显式 stdio spawn 普通 detached 进程树,并通过 `node-pty` 加平台进程检查实现终端进程。该实现没有任何配置:每项处置方式、限制、终端尺寸、宽限期与目录都来自调用方 seam([`dsh-bash-local`](../../bash/bash-local/README.md)、[`dsh-lsp-local`](../../lsp/lsp-local/README.md)、[`dsh-pty-local`](../../pty/pty-local/README.md) 和 [`dsh-code-runtime-subprocess`](../../code-runtime/code-runtime-subprocess/README.md))。 +[`@deepseek-ai/dsh-subprocess`](../subprocess/README.md) seam 的本地实现。`LocalSubprocessService` 解析本地可执行文件,以显式 stdio spawn 普通 detached 进程树,并通过 `node-pty` 加平台进程检查实现终端进程。该实现没有任何配置:每项处置方式、限制、终端尺寸、宽限期与目录都来自调用方 seam([`dsh-bash-local`](../../bash/bash-local/README.md)、[`dsh-lsp-local`](../../lsp/lsp-local/README.md) 和 [`dsh-pty-local`](../../pty/pty-local/README.md))。 ## 行为(以及设计来源) @@ -10,7 +10,7 @@ - **按流划分的处置方式**:`'pipe'` 把原始流原样交给调用方(协议分帧仍归消费方所有);`'inherit'` 直通父进程的描述符;收集模式(collect)在输出超过上限后于内存中保留尾部(错误与结果通常聚集在末尾,沿用 pi/OpenCode 的理由),并在配置了 spill 上限时把完整流追加到一个私有临时文件;省略 `spill` 则只保留用于诊断的尾部。某条流大于 spill 上限时,会丢弃已不完整的 spill,仅返回带截断标记的尾部;spill 文件描述符在结算时封存,最终关闭失败时则不公布路径,以免声称存在不完整的文件。spill 文件权限为 `0600`、名称随机,位于按需创建、权限为 `0700` 的每进程目录之下。 - **凭据清除 + 显式合并**:以 `process.env` 为基础,移除形似凭据的变量(`*KEY*`/`*PASSWORD*`/`*SECRET*`/`*TOKEN*`)和所有环境中已有的 `DSH_*` 名称;spec 的显式 `env` 在该清除之后合并且不做命名空间校验,因此有意提供的凭据或当前 `DSH_*` 事实会胜出,而陈旧的嵌套 harness 身份无法从环境中隐式漏入。提供的 stdin 会被写入后关闭;否则 fd 0 指向 `/dev/null`。参见 [stdin/env Agent Note(agent 决策记录)](../../../.agents/notes/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-surface.md)与[受管环境 Agent Note](../../../.agents/notes/implemented/feature/2026-07-10-agent-session-identity-and-log-location.md)。 - **基于偏移量的读取**:收集模式的读取器按完整流的字节坐标返回增量;服务自身从不持有游标,因此消费方自有的游标(bash 的后台读取路径)与完整流重读可以共存,结算前后皆然。 -- **执行世界坐标**:`cwd` 是宿主进程 cwd,`resolveExecutable` 检查绝对文件,或根据平台可执行文件扩展名在清理后的有效 PATH 中搜索。 +- **可执行文件查找**:`resolveExecutable` 检查绝对文件,或根据平台可执行文件扩展名在清理后的有效 PATH 中搜索;相对 PATH 条目从宿主进程 cwd 解析。 - **终端进程所有权**:`spawnTerminal` 分配 `node-pty`,桥接 UTF-8 终端文本,检查当前前台进程组并向其发送信号,还会公开一项须等待的终止操作,在终止顶层 shell 前后清理后代进程。每次前台检查都会保留根进程树中的精确身份;Linux 还会在 POSIX 会话 leader 退出后枚举该会话。因此,之前观察到的 macOS 后代以及同会话 Linux 成员在重新设定父进程后仍受围栏保护,pid/start 身份则防止清理跟随 PID 复用。上层 PTY 后端负责提示符就绪、缓冲区与面向模型的操作。 - **先终止再等待退出的 dispose(资源释放)**:服务保留存活句柄,只为让自身的 dispose 能对每个仍在运行的进程树执行升级并等待其退出;已结算与 spawn 失败的句柄在结算时即离开存活集合。 diff --git a/packages/subprocess/subprocess-local/src/index.ts b/packages/subprocess/subprocess-local/src/index.ts index 262a2567b8..4a740a1e3b 100644 --- a/packages/subprocess/subprocess-local/src/index.ts +++ b/packages/subprocess/subprocess-local/src/index.ts @@ -33,7 +33,6 @@ import { LocalTerminalHandle } from './terminal.ts' * SIGTERM→grace→SIGKILL escalation. */ export class LocalSubprocessService extends SubprocessService { - readonly cwd = process.cwd() /** Live handles retained only so disposal can terminate and join them. */ private live = new Set() /** Live terminal sessions retained through whole-session quiescence. */ @@ -103,7 +102,7 @@ export class LocalSubprocessService extends SubprocessService { ? (environmentValue(env, 'PATHEXT') ?? '.COM;.EXE;.BAT;.CMD').split(';') : [''] return path.split(delimiter).flatMap(directory => - extensions.map(extension => resolve(this.cwd, directory, command + extension))) + extensions.map(extension => resolve(process.cwd(), directory, command + extension))) } spawn(spec: SubprocessSpawnSpec): SubprocessHandle { diff --git a/packages/subprocess/subprocess-local/tests/local.spec.ts b/packages/subprocess/subprocess-local/tests/local.spec.ts index e74927448d..a0455edef0 100644 --- a/packages/subprocess/subprocess-local/tests/local.spec.ts +++ b/packages/subprocess/subprocess-local/tests/local.spec.ts @@ -31,9 +31,6 @@ describe('LocalSubprocessService', () => { expect(await ctx.subprocess.resolveExecutable(basename(process.execPath), { PATH: relative(process.cwd(), dirname(process.execPath)) || '.', })).toBe(process.execPath) - Reflect.set(ctx.subprocess, 'cwd', dirname(process.execPath)) - expect(await ctx.subprocess.resolveExecutable(basename(process.execPath), { PATH: '' })) - .toBe(process.execPath) await expect(ctx.subprocess.resolveExecutable('')).rejects.toThrow('must be non-empty') await expect(ctx.subprocess.resolveExecutable('dsh-command-that-does-not-exist', { PATH: '' })) .rejects.toThrow('was not found on PATH') diff --git a/packages/subprocess/subprocess/README.i18n.yaml b/packages/subprocess/subprocess/README.i18n.yaml index d12516f11f..e67bf1a87f 100644 --- a/packages/subprocess/subprocess/README.i18n.yaml +++ b/packages/subprocess/subprocess/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/subprocess/subprocess/README.md -README.md: 08fb03d6f145c1026180d92de2421b647cb3ebbe -README.zh.md: 592fc83223092b3406e9070e11de2b20a63232a6 +README.md: ec4a4e3328a5a600441d2e7983b3844f4ccc5e91 +README.zh.md: 3da79995fad9a2c8f2a6020ae18152a0e4459c71 diff --git a/packages/subprocess/subprocess/README.md b/packages/subprocess/subprocess/README.md index 08fb03d6f1..ec4a4e3328 100644 --- a/packages/subprocess/subprocess/README.md +++ b/packages/subprocess/subprocess/README.md @@ -2,12 +2,12 @@ English | [中文](README.zh.md) -The subprocess seam (`ctx.subprocess`) is the process half of one execution world. The abstract `SubprocessService` exposes its canonical `cwd`, executable lookup, ordinary managed `spawn`, and one terminal-process primitive; its vocabulary covers raw/collected stdio, process and terminal handles, exit facts, tree/session cleanup, and the managed `DSH_*` environment namespace. The local implementation lives in [`dsh-subprocess-local`](../subprocess-local/README.md). +The subprocess seam (`ctx.subprocess`) is the process half of one execution world. The abstract `SubprocessService` exposes executable lookup, ordinary managed `spawn`, and one terminal-process primitive; its vocabulary covers raw/collected stdio, process and terminal handles, exit facts, tree/session cleanup, and the managed `DSH_*` environment namespace. The local implementation lives in [`dsh-subprocess-local`](../subprocess-local/README.md). ## Contract - `spawn(spec)` returns immediately with a live handle; `done` resolves at process close with exit facts (`SubprocessOutcome` carries no output and no cause classification) and rejects only for spawn-level failures. -- `cwd` and executable paths belong to the provider's execution world. `resolveExecutable(command, env?, signal?)` verifies absolute commands or resolves bare names against that world's scrubbed PATH plus explicit overrides. +- Spawn working directories and executable paths belong to the provider's execution world. `resolveExecutable(command, env?, signal?)` verifies absolute commands or resolves bare names against that world's scrubbed PATH plus explicit overrides. - The spec is fully explicit — argv, cwd, per-stream stdio dispositions, grace — because deployment-varying defaults belong to the calling seam's config, not to a hidden subprocess-service default (the `dsh-bash` request/spec split is the owning template). `argv` is never shell-interpreted; a consumer that wants a shell passes `['bash', '-c', command]` itself. - Stdio is Node-shaped per stream: `'pipe'` hands the caller the raw stream for its own protocol framing (LSP JSON-RPC, ACP ndjson), `'inherit'` passes the parent descriptor through for diagnostics, and collect mode (`{ maxBytes, spill? }`) buffers a bounded tail with an optional full-stream spill file. Collect readers take whole-stream byte offsets and never consume, so independent readers cannot steal one another's deltas; a read whose offset slid out of the in-memory tail is `lossy` and points at the spill file when one exists. Collected output stays readable after settlement. - Termination is tree-scoped on every platform (POSIX detached groups with direct-child fallback; Windows `taskkill /T`): `terminate()` — the only termination verb — escalates SIGTERM→grace→SIGKILL (idempotent, driven by the spec's abort signal too, a no-op once the tree is gone), and `waitForExit(signal?)` observes whole-tree liveness so a consumer-owned teardown ladder holds each tier on real quiescence — the manager reacts but never classifies why (callers own deadlines, teardown ladders, and cause classification). diff --git a/packages/subprocess/subprocess/README.zh.md b/packages/subprocess/subprocess/README.zh.md index 592fc83223..3da79995fa 100644 --- a/packages/subprocess/subprocess/README.zh.md +++ b/packages/subprocess/subprocess/README.zh.md @@ -2,12 +2,12 @@ [English](README.md) | 中文 -子进程 seam(`ctx.subprocess`)是一个执行世界的进程部分。抽象的 `SubprocessService` 公开其规范化 `cwd`、可执行文件查找、普通受管 `spawn` 和一项终端进程原语;其词汇涵盖原始/收集式 stdio、进程与终端句柄、退出事实、进程树/会话清理,以及受管的 `DSH_*` 环境命名空间。本地实现位于 [`dsh-subprocess-local`](../subprocess-local/README.md)。 +子进程 seam(`ctx.subprocess`)是一个执行世界的进程部分。抽象的 `SubprocessService` 公开可执行文件查找、普通受管 `spawn` 和一项终端进程原语;其词汇涵盖原始/收集式 stdio、进程与终端句柄、退出事实、进程树/会话清理,以及受管的 `DSH_*` 环境命名空间。本地实现位于 [`dsh-subprocess-local`](../subprocess-local/README.md)。 ## 契约 - `spawn(spec)` 立即返回一个活动句柄;`done` 在进程关闭时以退出事实 resolve(`SubprocessOutcome` 不携带输出,也不携带原因分类),仅在 spawn 层面失败时 reject。 -- `cwd` 和可执行文件路径属于提供方的执行世界。`resolveExecutable(command, env?, signal?)` 验证绝对命令,或根据该执行世界清理后的 PATH 加显式覆盖来解析裸名称。 +- spawn 工作目录和可执行文件路径属于提供方的执行世界。`resolveExecutable(command, env?, signal?)` 验证绝对命令,或根据该执行世界清理后的 PATH 加显式覆盖来解析裸名称。 - spec 完全显式(argv、cwd、按流划分的 stdio 处置方式(disposition)、宽限期),因为随部署变化的默认值属于调用方 seam 的配置,而不属于某个隐藏的子进程默认值(`dsh-bash` 的 request/spec 拆分是这条规则的所属模板)。`argv` 绝不经过 shell 解释;需要 shell 的消费方自行传入 `['bash', '-c', command]`。 - stdio 按流采用 Node 风格:`'pipe'` 把原始流交给调用方做自己的协议分帧(LSP 的 JSON-RPC、ACP(Agent Client Protocol)的 ndjson),`'inherit'` 直通父进程描述符以承载诊断输出,收集模式(collect)`{ maxBytes, spill? }` 则缓冲一段有界尾部,外加可选的完整流 spill 文件。收集模式的读取器接受全流字节偏移量且从不消费,因此独立的读取器不会抢走彼此的增量;偏移量滑出内存尾部窗口的读取标记为 `lossy`,并在 spill 文件存在时指向它。收集到的输出在结算后仍可读取。 - 终止在每个平台上都以进程树为范围(POSIX 用 detached 进程组并以直接子进程回退;Windows 用 `taskkill /T`):`terminate()`(唯一的终止动词)执行 SIGTERM→宽限期→SIGKILL 升级(幂等,也由 spec 的 abort 信号驱动,进程树消亡后为空操作);`waitForExit(signal?)` 观察整棵进程树的存活状态,使消费方自有的拆卸阶梯能在真正完全停稳后才进入下一层。管理器只响应中止,但绝不判定原因(deadline、拆卸阶梯与原因分类归调用方所有)。 diff --git a/packages/subprocess/subprocess/src/index.ts b/packages/subprocess/subprocess/src/index.ts index df3186c9b0..14866c9784 100644 --- a/packages/subprocess/subprocess/src/index.ts +++ b/packages/subprocess/subprocess/src/index.ts @@ -1,6 +1,6 @@ /** - * The subprocess seam (`ctx.subprocess`): execution-world process coordinates, - * executable lookup, fully specified managed process trees with raw or + * The subprocess seam (`ctx.subprocess`): execution-world executable lookup, + * fully specified managed process trees with raw or * collected stdio, and one terminal-process primitive. Command defaulting, * shell semantics, deadlines, protocol framing, terminal readiness, and * presentation belong to consumers. The local implementation lives in @@ -78,8 +78,8 @@ declare module 'cordis' { * duplicate-service behavior). * * Implementations must honor these semantics: - * - {@link cwd} and executable paths belong to one execution world shared - * with the mounted filesystem provider. + * - Executable paths belong to one execution world shared with the mounted + * filesystem provider. * - {@link spawn} returns immediately with a live handle; `done` resolves at * process close with exit facts and rejects only for spawn-level failures. * - Collect-mode readers are offset-based and non-consuming, so independent @@ -104,9 +104,6 @@ export abstract class SubprocessService extends Service { super(ctx, 'subprocess') } - /** Canonical default cwd in this provider's execution world. */ - abstract readonly cwd: string - /** * Resolve one configured executable in this provider's execution world. * Absolute paths are verified; bare names use the provider's scrubbed PATH diff --git a/scripts/check-workspace-constraints.ts b/scripts/check-workspace-constraints.ts index b20d5b5f3b..355b817e68 100644 --- a/scripts/check-workspace-constraints.ts +++ b/scripts/check-workspace-constraints.ts @@ -148,7 +148,6 @@ function expectedDshPackageFiles(manifest: PackageManifest): readonly string[] { 'lib/invariant.js', ...manifest.bin ? ['lib/bin.js'] : [], ...manifest.exports?.['./worker'] ? ['lib/worker.cjs'] : [], - ...exportDefault(manifest, './runtime-host') === './lib/runtime-host.js' ? ['lib/runtime-host.js'] : [], // UI plugin packages ship their browser bundle beside the node lib // (single-artifact ruling: dist/ retired, ./client resolves lib/client.js). // Keyed on the artifact path, not the subpath name: apiproxy's ./client is diff --git a/scripts/gen-code-runtime-runner.ts b/scripts/gen-code-runtime-runner.ts deleted file mode 100644 index 7e856e7651..0000000000 --- a/scripts/gen-code-runtime-runner.ts +++ /dev/null @@ -1,70 +0,0 @@ -/** Generate the subprocess Code Runtime's dependency-free runner bundle. */ - -import { existsSync, readFileSync, writeFileSync } from 'node:fs' -import { resolve } from 'node:path' -import { build } from 'tsdown' - -const root = resolve(import.meta.dirname, '..') -const ENTRY = 'packages/code-runtime/code-runtime-subprocess/src/runner.ts' -const OUT = 'packages/code-runtime/code-runtime-subprocess/src/runner-source.generated.ts' - -/** - * Bundle the typed runner and shared worker implementation into one source literal. - * @returns generated TypeScript module consumed by the subprocess backend. - */ -export async function renderCodeRuntimeRunner(): Promise { - const bundles = await build({ - config: false, - entry: [resolve(root, ENTRY)], - format: ['esm'], - platform: 'node', - target: 'es2024', - write: false, - dts: false, - clean: false, - minify: true, - logLevel: 'silent', - report: false, - deps: { alwaysBundle: ['@deepseek-ai/dsh-code-runtime-worker'] }, - }) - try { - const chunks = bundles.flatMap(bundle => bundle.chunks).filter(chunk => chunk.type === 'chunk') - if (chunks.length !== 1) throw new Error(`gen-code-runtime-runner: expected one chunk, received ${chunks.length}`) - const chunk = chunks[0] - if (chunk === undefined) throw new Error('gen-code-runtime-runner: runner chunk is missing') - const external = chunk.imports.filter(specifier => !specifier.startsWith('node:')) - if (external.length > 0) { - throw new Error(`gen-code-runtime-runner: runner retained external imports: ${external.join(', ')}`) - } - return [ - '/**', - ' * Generated dependency-free execution-world runner.', - ' * Do not edit by hand; run `pnpm run gen-code-runtime-runner`.', - ' */', - '', - `export const CODE_RUNNER_SOURCE = ${JSON.stringify(chunk.code)}`, - '', - ].join('\n') - } finally { - await Promise.all(bundles.map(async (bundle) => { await bundle[Symbol.asyncDispose]() })) - } -} - -async function main(): Promise { - const content = await renderCodeRuntimeRunner() - const output = resolve(root, OUT) - if (process.argv.includes('--check')) { - const committed = existsSync(output) ? readFileSync(output, 'utf8') : null - if (committed === content) { - console.log(`gen-code-runtime-runner: ${OUT} is up to date.`) - return - } - console.error(`gen-code-runtime-runner: ${OUT} is stale. Run \`pnpm run gen-code-runtime-runner\` and commit it.`) - process.exitCode = 1 - return - } - writeFileSync(output, content) - console.log(`gen-code-runtime-runner: wrote ${OUT}.`) -} - -if (process.argv[1] && import.meta.filename === resolve(process.argv[1])) await main() diff --git a/scripts/gen-doc-graphs.ts b/scripts/gen-doc-graphs.ts index 7be0e1fa58..00520bdbe7 100644 --- a/scripts/gen-doc-graphs.ts +++ b/scripts/gen-doc-graphs.ts @@ -48,13 +48,9 @@ interface EventRelation { listeners: Set } -/** One scanned package source file and its owning package short name. */ -export interface PackageSource { - /** Repository-relative path. */ +interface PackageSource { rel: string - /** Package short name from the `packages///src` path. */ pkg: string - /** The bound program source file. */ sourceFile: ts.SourceFile } @@ -124,7 +120,7 @@ const SERVICE_ROLES: ServiceRole[] = [ pkg: 'session', title: 'In-memory session store', mode: 'core', - consumers: ['agent-loop', 'agent', 'session-persistence', 'session-query', 'session-query-sqlite', 'subagent-inprocess', 'invariants'], + consumers: ['agent-loop', 'agent', 'cli-demo', 'session-persistence', 'session-query', 'session-query-sqlite', 'subagent-inprocess', 'invariants'], note: 'Owns append-only Session instances and emits the durable session event feed.', }, { @@ -159,24 +155,6 @@ const SERVICE_ROLES: ServiceRole[] = [ consumers: ['agent-loop', 'tool-bash', 'hooks-claude', 'hooks-codex', 'session-query', 'session-query-sqlite'], note: 'Backends persist the same SessionEvent vocabulary; apps choose a backend at composition time.', }, - { - key: 'settings', - pkg: 'settings', - title: 'User-settings seam', - mode: 'seam', - implementations: ['settings-local'], - consumers: ['llm-deepseek', 'llm-pi-ai', 'apiproxy'], - note: 'Plugins register namespace schemas and resolve layered values; providers store the raw document. The LLM adapters register their entry config as the composition base under the user section; the web gateway serves redacted layered descriptors and writes the user layer.', - }, - { - key: 'credentials', - pkg: 'credentials', - title: 'Credential seam', - mode: 'seam', - implementations: ['credentials-local'], - consumers: ['llm-deepseek', 'llm-pi-ai', 'apiproxy'], - note: 'Configuration carries references to secrets; providers own the values. Consumers resolve per operation, so a rotated credential reaches the very next request; the web gateway exposes value-free views and write-only storage.', - }, { key: 'telemetry', pkg: 'session-telemetry', @@ -225,6 +203,7 @@ const SERVICE_ROLES: ServiceRole[] = [ pkg: 'session-reference', title: 'Cross-session snapshot preparation', mode: 'core', + consumers: ['tui'], note: 'Projects bounded current-surface conversation snapshots into durable untrusted message context; host adapters own mention syntax.', }, { @@ -256,7 +235,8 @@ const SERVICE_ROLES: ServiceRole[] = [ pkg: 'user-interaction', title: 'Human question/answer seam', mode: 'seam', - consumers: ['tool-ask-user'], + implementations: ['tui'], + consumers: ['tool-ask-user', 'tui'], note: 'UI front doors provide the active human-answer provider; tool-ask-user pauses a tool call on the provider-neutral ask() promise.', }, { @@ -271,7 +251,8 @@ const SERVICE_ROLES: ServiceRole[] = [ pkg: 'commands', title: 'Human command registry', mode: 'core', - note: 'Plugins register direct human commands without sending invocations to the model.', + consumers: ['tui'], + note: 'Plugins register direct human commands; TUI consumes the effective per-agent catalog without sending invocations to the model.', }, { key: 'sessionProjections', @@ -289,12 +270,19 @@ const SERVICE_ROLES: ServiceRole[] = [ consumers: ['host-apiproxy'], note: 'Durably checkpoints projection unit states per session (throttled + turn/end/detach mandatory points) and serves the cold-read ladder: cache row + persistence tail replay, so listings never load full logs.', }, + { + key: 'tui', + pkg: 'tui', + title: 'Mounted-terminal interaction service', + mode: 'bundle', + note: 'One TUI front door provides a FIFO overlay host; injected plugins receive caller-fiber ownership without access to pi-tui or terminal lifecycle state.', + }, { key: 'skills', pkg: 'skill', title: 'Skill provider registry', mode: 'seam', - implementations: ['skill-badge', 'skill-local'], + implementations: ['skill-local'], consumers: ['tool-skill'], note: 'Merges provider skill catalogs; tool-skill renders the session-prefix catalog and loads complete skill bodies.', }, @@ -303,7 +291,7 @@ const SERVICE_ROLES: ServiceRole[] = [ pkg: 'agent', title: 'Agent service', mode: 'core', - consumers: ['agent-loop', 'acp', 'subagent-inprocess'], + consumers: ['agent-loop', 'acp', 'cli-demo', 'subagent-inprocess', 'tui-demo'], note: 'Owns live Agent handles, the create/resume factory seam, and process-local initiator propagation.', }, { @@ -327,25 +315,24 @@ const SERVICE_ROLES: ServiceRole[] = [ title: 'Subprocess seam', mode: 'seam', implementations: ['subprocess-local'], - consumers: ['bash-local', 'bash-sandbox', 'lsp-local', 'subagent-acp', 'subagent-codex', 'subagent-claude-code'], - note: 'The bash executors, the LSP host, and the out-of-process ACP, Codex, and Claude Code subagent backends spawn their children through ctx.subprocess; the service owns tree lifetime, stdio dispositions (pipes, inherit, bounded spill-backed collection), and kill escalation.', + consumers: ['bash-local', 'bash-sandbox', 'pty-local', 'lsp-local', 'subagent-acp'], + note: 'The bash executors, PTY shell backend, LSP host, and ACP subagent backend spawn through ctx.subprocess; the service owns process coordinates, tree/session lifetime, stdio dispositions, terminal mechanics, and kill escalation.', }, { key: 'bash', pkg: 'bash', title: 'Bash executor seam', mode: 'seam', - implementations: ['bash-local', 'bash-sandbox', 'pwsh-local'], - consumers: ['tool-bash', 'tool-pwsh', 'hooks-claude', 'hooks-codex'], - note: 'The model-facing shell tools and hook bridges consume this seam; sandboxed, remote, or PowerShell executors replace bash-local without touching them.', + implementations: ['bash-local', 'bash-sandbox'], + consumers: ['tool-bash', 'hooks-claude', 'hooks-codex'], + note: 'The model-facing bash tools and hook bridges consume this seam; sandboxed or remote executors replace bash-local without touching them.', }, { key: 'bashEnv', - pkg: 'bash-env', + pkg: 'tool-bash', title: 'Managed bash environment registry', mode: 'core', - consumers: ['tool-bash', 'tool-pwsh'], - note: 'Plugins declare effect-scoped DSH_* facts; each shell tool collects one trusted snapshot per execution and its executor rebuilds the namespace.', + note: 'Plugins declare effect-scoped DSH_* facts; tool-bash collects one trusted snapshot per execution and the executor rebuilds the namespace.', }, { key: 'pty', @@ -422,11 +409,11 @@ const SERVICE_ROLES: ServiceRole[] = [ { key: 'subagents', pkg: 'subagent', - title: 'Subagent provider and continuation service', + title: 'Subagent provider registry', mode: 'seam', - implementations: ['subagent-spawn', 'subagent-fork', 'subagent-acp', 'subagent-codex', 'subagent-claude-code', 'subagent-dsh-sdk'], - consumers: ['tool-subagent', 'tool-subagent-control', 'tool-ralph'], - note: 'Providers implement transports; the service also owns optional Activation-based continuation orchestration, tool-subagent selects one-shot or continuable delegation, tool-subagent-control delivers follow-ups, and tool-ralph requires one fresh structured-output route.', + implementations: ['subagent-spawn', 'subagent-fork', 'subagent-acp'], + consumers: ['tool-subagent', 'tool-ralph'], + note: 'Providers implement transports; tool-subagent exposes configured delegation while tool-ralph requires one fresh structured-output route.', }, { key: 'tasks', @@ -605,8 +592,7 @@ function parseExampleCordis(rel: string): ExamplePlugin[] { if (current?.name) plugins.push({ id: current.id, name: current.name }) } for (const line of text.split('\n')) { - // Top-level rows (`- id:`) and bundle-patch insert rows (` - id:`). - const id = /^\s*-\s+id:\s+(.+?)\s*$/.exec(line) + const id = /^-\s+id:\s+(.+?)\s*$/.exec(line) if (id?.[1] !== undefined) { flush() current = { id: stripYamlScalar(id[1]) } @@ -625,20 +611,28 @@ function stripYamlScalar(value: string): string { const APP_EXAMPLES = [ { - id: 'dsh_base', - rel: 'apps/cli/composition.md', - title: 'DSH Base Composition', - label: 'packages/bundle/base/cordis.patch.yml', - config: 'packages/bundle/base/cordis.patch.yml', - summary: 'The dsh-base bundle patch every profile applies first; mode bundles (dsh-web-app, dsh-headless) and the user\'s profile layer patch over it.', + id: 'tui', + rel: 'examples/tui-agent/composition.md', + title: 'TUI Agent App Composition', + label: 'examples/tui-agent', + config: 'examples/tui-agent/cordis.yml', + summary: 'The TUI agent combines the real DeepSeek adapter, coding tools, compaction, subagents, and workflows with the full-screen terminal app package.', }, { id: 'headless', rel: 'examples/headless-agent/composition.md', - title: 'Headless Agent Snapshot Composition', + title: 'Headless Agent App Composition', label: 'examples/headless-agent', config: 'examples/headless-agent/cordis.yml', - summary: 'The headless snapshot composition combines the real DeepSeek adapter and coding capabilities with one explicitly configured persisted top-level agent; its JSONL driver is test-only.', + summary: 'The headless demo combines the real DeepSeek adapter and coding capabilities with the one-shot app package, format-pure stdout, and one fresh persisted top-level session.', + }, + { + id: 'cordis', + rel: 'examples/cordis-agent/composition.md', + title: 'Cordis Agent App Composition', + label: 'examples/cordis-agent', + config: 'examples/cordis-agent/cordis.yml', + summary: 'The self-referential demo puts @deepseek-ai/dsh-tool-cordis on the coding spine, letting the agent inspect its current-process runtime and mount or unmount in-memory temporary Plugins.', }, { id: 'acp', @@ -657,7 +651,11 @@ function renderAppExpansion(lines: string[], appNode: string, pluginName: string const jsonl = nodeId('bundle', 'jsonl') lines.push(` ${appNode} --> ${agentCore}["@deepseek-ai/dsh-agent-spine-demo"]`) lines.push(` ${appNode} --> ${jsonl}["@deepseek-ai/dsh-session-persistence-jsonl"]`) - if (pluginName === '@deepseek-ai/dsh-acp-demo') { + if (pluginName === '@deepseek-ai/dsh-tui-demo') { + lines.push(` ${appNode} --> ${nodeId('frontdoor', 'tui')}["@deepseek-ai/dsh-tui
pre-created main agent"]`) + } else if (pluginName === '@deepseek-ai/dsh-cli-demo') { + lines.push(` ${appNode} --> ${nodeId('frontdoor', 'cli')}["one-shot driver
format-pure stdout
fresh top-level agent"]`) + } else if (pluginName === '@deepseek-ai/dsh-acp-demo') { lines.push(` ${appNode} --> ${nodeId('frontdoor', 'acp')}["@deepseek-ai/dsh-acp
automation-only JSON-RPC stdio
fresh sessions created by client"]`) } lines.push( @@ -683,7 +681,7 @@ function renderAppComposition(example: AppExample): string { const pluginNode = nodeId(`plugin_${example.id}`, plugin.id) lines.push(` ${pluginNode}["${escLabel(plugin.id)}
${escLabel(plugin.name)}"]`) lines.push(` cfg --> ${pluginNode}`) - if (plugin.name === '@deepseek-ai/dsh-acp-demo') { + if (plugin.name === '@deepseek-ai/dsh-tui-demo' || plugin.name === '@deepseek-ai/dsh-cli-demo' || plugin.name === '@deepseek-ai/dsh-acp-demo') { renderAppExpansion(lines, pluginNode, plugin.name) } } @@ -700,26 +698,13 @@ function renderAppComposition(example: AppExample): string { return lines.join('\n') } -type CallSiteIndex = Map - -/** - * The only method names visitSource classifies; receiver typing runs on these - * alone. Obligation: every method name matched by a branch inside visitSource - * must appear here — the prefilter drops non-members before any branch runs, - * so a branch for an unlisted name is silently dead. - */ -const EVENT_API_METHODS = new Set(['on', 'once', 'emit', 'parallel', 'serial', 'waterfall', 'dispatch']) - /** Collect event dispatch/listener relations from real cross-file receiver types. */ -export class EventRelationCollector { +class EventRelationCollector { private readonly relations = new Map() - private readonly fileCallSites = new Map() - private readonly localCalleeProofs = new Map() - private globalCallSites: CallSiteIndex | null = null + private readonly callSites = new Map() private readonly contextType: ts.Type private readonly agentDispatchType: ts.Type private readonly eventsServiceType: ts.Type - private readonly packageSourceFiles: ReadonlySet constructor( private readonly project: TypeScriptProject, @@ -728,7 +713,7 @@ export class EventRelationCollector { this.contextType = this.declaredType('vendor/cordis/src/context.ts', 'Context') this.agentDispatchType = this.declaredType('packages/core/agent/src/dispatch.ts', 'AgentEventDispatch') this.eventsServiceType = this.declaredType('vendor/cordis/src/events.ts', 'EventsService') - this.packageSourceFiles = new Set(sources.map(source => source.sourceFile)) + this.indexCallSites() } /** Return all event relations discovered from the Program. */ @@ -748,88 +733,20 @@ export class EventRelationCollector { return this.project.checker.getDeclaredTypeOfSymbol(symbol) } - /** Index resolved function calls in the given files for narrow argument-flow recovery. */ - private buildCallSiteIndex(files: Iterable): CallSiteIndex { - const index: CallSiteIndex = new Map() + /** Index resolved local function calls for narrow argument-flow recovery. */ + private indexCallSites(): void { const visit = (node: ts.Node): void => { if (ts.isCallExpression(node)) { const declaration = this.project.checker.getResolvedSignature(node)?.declaration if (declaration) { - const calls = index.get(declaration) ?? [] + const calls = this.callSites.get(declaration) ?? [] calls.push(node) - index.set(declaration, calls) + this.callSites.set(declaration, calls) } } ts.forEachChild(node, visit) } - for (const file of files) visit(file) - return index - } - - /** - * Return every indexed call resolving to one local helper declaration. - * Fast path: when every same-file reference to the non-exported helper is - * provably a direct callee, module scoping confines all of its calls to that - * file, so only that file is indexed. Any other reference shape may alias - * the function value outward, so the original full package-source index - * decides instead. - */ - private callSitesFor(owner: ts.FunctionDeclaration): ts.CallExpression[] { - if (!this.globalCallSites && !this.provenLocalCallee(owner)) { - this.globalCallSites = this.buildCallSiteIndex(this.packageSourceFiles) - } - if (this.globalCallSites) return this.globalCallSites.get(owner) ?? [] - const file = owner.getSourceFile() - let index = this.fileCallSites.get(file) - if (!index) { - index = this.buildCallSiteIndex([file]) - this.fileCallSites.set(file, index) - } - return index.get(owner) ?? [] - } - - /** - * Prove every same-file reference to one helper is a direct callee. The - * proof owns its premises: an exported helper or a helper in a global - * script file (no import/export means program-wide scope, callable from - * another file with no same-file reference at all) fails immediately. - * Alias escapes (re-export statements, default exports, value reads) - * resolve back to the owner symbol at a non-callee position and fail the - * proof, as does anything the scan cannot positively classify. - */ - private provenLocalCallee(owner: ts.FunctionDeclaration): boolean { - const cached = this.localCalleeProofs.get(owner) - if (cached !== undefined) return cached - if (hasExportModifier(owner) || !ts.isExternalModule(owner.getSourceFile())) { - this.localCalleeProofs.set(owner, false) - return false - } - const name = owner.name - const ownerSymbol = name && this.project.checker.getSymbolAtLocation(name) - let proven = !!ownerSymbol - const refersToOwner = (identifier: ts.Identifier): boolean => { - // Shorthand properties resolve to the property symbol; ask for the value side. - const local = ts.isShorthandPropertyAssignment(identifier.parent) - ? this.project.checker.getShorthandAssignmentValueSymbol(identifier.parent) - : this.project.checker.getSymbolAtLocation(identifier) - if (!local) return false - const symbol = local.flags & ts.SymbolFlags.Alias - ? this.project.checker.getAliasedSymbol(local) - : local - return symbol === ownerSymbol - } - const visit = (node: ts.Node): void => { - if (!proven) return - if (ts.isIdentifier(node) && node !== name && node.text === name?.text - && !isDirectCallee(node) && refersToOwner(node)) { - proven = false - return - } - ts.forEachChild(node, visit) - } - visit(owner.getSourceFile()) - this.localCalleeProofs.set(owner, proven) - return proven + for (const source of this.sources) visit(source.sourceFile) } /** Walk one package source file and classify event API calls by receiver type. */ @@ -843,7 +760,7 @@ export class EventRelationCollector { this.addDispatcher(name, source.pkg, 'emitAgentEvent') } } - } else if (ts.isPropertyAccessExpression(node.expression) && EVENT_API_METHODS.has(node.expression.name.text)) { + } else if (ts.isPropertyAccessExpression(node.expression)) { const receiverKind = this.receiverKind(node.expression.expression) const method = node.expression.name.text if (receiverKind === 'events-service' && method === 'dispatch') { @@ -946,7 +863,7 @@ export class EventRelationCollector { const index = owner.parameters.indexOf(parameter) if (index < 0) return new Set() const events = new Set() - for (const call of this.callSitesFor(owner)) { + for (const call of this.callSites.get(owner) ?? []) { const argument = call.arguments[index] if (argument) addAll(events, this.eventNamesFromArgumentList(argument, new Set(seen))) } @@ -993,21 +910,6 @@ export class EventRelationCollector { } } -/** Return whether an identifier is the callee of a call, seen through value-preserving wrappers. */ -function isDirectCallee(identifier: ts.Identifier): boolean { - let current: ts.Node = identifier - while ( - ts.isParenthesizedExpression(current.parent) - || ts.isAsExpression(current.parent) - || ts.isTypeAssertionExpression(current.parent) - || ts.isNonNullExpression(current.parent) - || ts.isSatisfiesExpression(current.parent) - ) { - current = current.parent - } - return ts.isCallExpression(current.parent) && current.parent.expression === current -} - /** Peel syntax-only wrappers that do not change an expression's runtime value. */ function unwrapExpression(expression: ts.Expression): ts.Expression { let current = expression @@ -1063,22 +965,14 @@ function unionSets(left: ReadonlySet, right: ReadonlySet): Set { return out } -/** - * Select the package source files of one project in deterministic order. - * @param project - the loaded repository TypeScript project. - * @returns `packages///src` files tagged with their package name. - */ -export function collectPackageSources(project: TypeScriptProject): PackageSource[] { - return project.sourceFiles().flatMap((sourceFile): PackageSource[] => { +function collectEventRelations(): Map { + const project = new TypeScriptProject(root) + const sources = project.sourceFiles().flatMap((sourceFile): PackageSource[] => { const rel = project.relativePath(sourceFile) const match = /^packages\/[^/]+\/([^/]+)\/src\/.+\.ts$/.exec(rel) return match?.[1] ? [{ rel, pkg: match[1], sourceFile }] : [] }).sort((left, right) => left.rel.localeCompare(right.rel)) -} - -function collectEventRelations(): Map { - const project = new TypeScriptProject(root) - return new EventRelationCollector(project, collectPackageSources(project)).collect() + return new EventRelationCollector(project, sources).collect() } function relationPackages(map: Map>, pkgsByShort: Map): string { @@ -1159,22 +1053,20 @@ function renderLifecycle(): string { ' participant Session', ' participant SDK as UI or SDK listener', ' User->>Agent: followup(content)', - ` Agent-->>SDK: ${mermaidCode('agent/inbox/spliced')}`, - ` Agent-->>SDK: ${mermaidCode('agent/inbox/inserted')} { message }`, + ` Agent-->>SDK: ${mermaidCode('agent/inbox/enqueue')}`, ' Agent->>Driver: queued work wakes driver', ` Driver-->>SDK: ${mermaidCode('agent/status')} running`, - ' Note over Agent,Driver: claim pending next-step input plus one queued prompt', - ` Driver-->>SDK: ${mermaidCode('agent/inbox/spliced')} pure deletion`, - ` Driver-->>SDK: ${mermaidCode('agent/inbox/claimed')} { message, turn } per message`, - ` Driver->>Hooks: ${mermaidCode('agent/pre-step')} waterfall`, - ' Hooks-->>Driver: authoritative reject or enter(messages)', - ' alt proposed step rejected or pre-step failed', - ' Driver-->>Driver: claimed batch stays removed, no turn opens', - ' else enter proposed step', + ' Note over Agent,Driver: next-step acceptance window opens', + ` Driver->>Hooks: ${mermaidCode('agent/prompt-submit')} waterfall`, + ' Hooks-->>Driver: authoritative allow, block, or add context', + ' alt prompt blocked or admission failed', + ' Driver-->>Driver: append context-only batch or keep steering boundary pending', + ' else prompt allowed', ` Driver->>Session: ${mermaidCode('turn/start')}`, - ` Driver->>Session: ${mermaidCode('step/start')}`, - ` Driver->>Session: ${mermaidCode('user/message')} per entered message`, + ` Driver->>Session: ${mermaidCode('user/message')}`, ` Driver->>Prompt: ${mermaidCode('system-prompt/assemble')} waterfall`, + ` Driver-->>Driver: ${mermaidCode('agent/step')} serial checkpoint`, + ` Driver->>Session: ${mermaidCode('step/start')}`, ` Driver->>LLM: ${mermaidCode('agent/request')} waterfall, then ${mermaidCode('llm/stream')} waterfall`, ' LLM-->>Driver: StreamChunk*', ` Driver->>Session: ${mermaidCode('assistant/chunk')}*`, @@ -1197,17 +1089,11 @@ function renderLifecycle(): string { ` Driver->>Session: ${mermaidCode('tool/result')}`, ' end', ' end', + ' Driver->>Session: post-tool context and steering (no prompt-submit)', ` Driver->>Session: ${mermaidCode('step/end')}`, - ' opt natural stop and next-step inbox empty', - ` Driver->>Hooks: ${mermaidCode('agent/turn-stopping')} serial terminal checkpoint`, - ' end', - ' opt next-step input is pending', - ' Driver-->>Driver: claim pending next-step input', - ` Driver-->>SDK: ${mermaidCode('agent/inbox/claimed')} { message, turn } per message`, - ` Driver->>Hooks: ${mermaidCode('agent/pre-step')} waterfall`, - ' Hooks-->>Driver: authoritative reject or enter(messages)', - ' end', + ` Driver->>Hooks: ${mermaidCode('agent/turn-stopping')} serial terminal checkpoint`, ' end', + ' Note over Agent,Driver: next-step acceptance window closes', ` Driver->>Session: ${mermaidCode('turn/end')}`, ' end', ` Driver-->>SDK: ${mermaidCode('agent/status')} idle`, @@ -1215,9 +1101,9 @@ function renderLifecycle(): string { '', 'The `assistant/message` edge records every successful provider call, including content-less and `max-tokens` finishes. Empty content stays out of derived history while the durable anchor retains usage and exact chunk provenance, including an explicit empty source set.', '', - '`dsh-compact-basic` uses `agent/pre-step` for pressure before request derivation and `agent/request-error` only for canonical context overflow. Once either trigger qualifies, optional tool-result pruning runs before summary selection. Recovery works between the closed failed step and failed turn close, and opens a fresh retry turn only when pruning or summarization advances the surface replacement generation; otherwise the original request error remains authoritative.', + '`dsh-compact-basic` uses `agent/step` for pressure before request derivation and `agent/request-error` only for canonical context overflow. Once either trigger qualifies, optional tool-result pruning runs before summary selection. Recovery works between the closed failed step and failed turn close, and opens a fresh retry turn only when pruning or summarization advances the surface replacement generation; otherwise the original request error remains authoritative.', '', - 'The returned `agent/pre-step` decision is authoritative; listeners wrapping `next()` preserve downstream messages unless replacement is intentional. Steering and injected context pass through the same waterfall after a later boundary claims their next-step batch.', + 'The returned `agent/prompt-submit` allow is authoritative; listeners wrapping `next()` preserve downstream content and additional contexts unless replacement is intentional. Steering bypasses that waterfall and joins at its durable checkpoint.', '', 'SDK users that need replayable transcript data should consume `session/event`; `agent/*` is the live coordination surface for queue/status, prompt interception, request shaping, steering, continuation, and errors.', '', @@ -1306,8 +1192,8 @@ function renderDocs(): GraphDoc[] { function renderIndex(docs: GraphDoc[]): string { const labels: Record = { 'docs/capability-seams.md': 'capability seams and core services', - 'apps/cli/composition.md': 'dsh shared base composition', 'examples/headless-agent/composition.md': 'headless-agent app composition', + 'examples/tui-agent/composition.md': 'tui-agent app composition', 'examples/cordis-agent/composition.md': 'cordis-agent app composition', 'examples/acp-agent/composition.md': 'acp-agent app composition', 'docs/event-producer-consumer.md': 'event producer/consumer matrix', @@ -1316,8 +1202,8 @@ function renderIndex(docs: GraphDoc[]): string { } const modes: Record = { 'docs/capability-seams.md': 'hybrid generated', - 'apps/cli/composition.md': 'hybrid generated', 'examples/headless-agent/composition.md': 'hybrid generated', + 'examples/tui-agent/composition.md': 'hybrid generated', 'examples/cordis-agent/composition.md': 'hybrid generated', 'examples/acp-agent/composition.md': 'hybrid generated', 'docs/event-producer-consumer.md': 'hybrid generated', diff --git a/scripts/run-gates.ts b/scripts/run-gates.ts index 6f59b64826..74a09a874e 100644 --- a/scripts/run-gates.ts +++ b/scripts/run-gates.ts @@ -585,7 +585,6 @@ function docSyncLeafGates(options: { pnpmScript('persistence-catalog', 'verify-persistence-catalog', { label: 'persistence catalog' }), pnpmScript('doc-graphs', 'verify-doc-graphs', { label: 'doc graphs' }), pnpmScript('scoped-events', 'verify-scoped-events', { label: 'scoped events' }), - pnpmScript('code-runtime-runner', 'verify-code-runtime-runner', { label: 'code-runtime runner' }), pnpmScript('markdown-wrap', 'verify-md-wrap', { label: 'markdown wrap' }), pnpmScript('markdown-links', 'verify-md-links', { label: 'markdown links' }), pnpmScript('public-repository-links', 'verify-public-repository-links', { label: 'public repository links' }), diff --git a/scripts/verify-package-readme-model-experience.ts b/scripts/verify-package-readme-model-experience.ts index 9a249ca844..3ca0317cf4 100644 --- a/scripts/verify-package-readme-model-experience.ts +++ b/scripts/verify-package-readme-model-experience.ts @@ -47,7 +47,6 @@ const SENTENCE_MODEL_EXPERIENCE: Readonly> = { 'packages/bash/bash-local': { kind: 'indirect', reason: 'The executor backend delegates model rendering to dsh-tool-bash.' }, 'packages/bash/pwsh-local': { kind: 'indirect', reason: 'The executor backend delegates model rendering to dsh-tool-pwsh.' }, 'packages/code-runtime/code-runtime': { kind: 'indirect', reason: 'The service interface delegates model rendering to Code Mode in dsh-tools.' }, - 'packages/code-runtime/code-runtime-subprocess': { kind: 'indirect', reason: 'The subprocess backend delegates model rendering to Code Mode in dsh-tools.' }, 'packages/code-runtime/code-runtime-worker': { kind: 'indirect', reason: 'The worker backend delegates model rendering to Code Mode in dsh-tools.' }, 'packages/typert/registry': { kind: 'none', reason: 'Runtime type registry; consumers (cordis_inspect, wire faces, gates) own any model-visible projection of registry contents.' }, 'packages/typert/loader': { kind: 'none', reason: 'Loader integration only registers generated artifacts; consumers own any model-visible projection.' }, diff --git a/tsconfig.host.json b/tsconfig.host.json index 6252733511..1a8ef6508a 100644 --- a/tsconfig.host.json +++ b/tsconfig.host.json @@ -169,7 +169,6 @@ { "path": "./packages/pty/tool-pty" }, { "path": "./packages/code-runtime/code-runtime" }, { "path": "./packages/code-runtime/code-runtime-worker" }, - { "path": "./packages/code-runtime/code-runtime-subprocess" }, { "path": "./packages/llm/llm-deepseek" }, { "path": "./packages/llm/llm-pi-ai" }, { "path": "./packages/bash/bash-local" }, From e67a824604c3c098f01cb3e24fa189f56d4c8fc0 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Thu, 30 Jul 2026 14:38:03 +0800 Subject: [PATCH 052/108] docs(code-runtime): restore worker-only package summary --- packages/README.i18n.yaml | 2 +- packages/README.zh.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/packages/README.i18n.yaml b/packages/README.i18n.yaml index 1484411cca..458da3b713 100644 --- a/packages/README.i18n.yaml +++ b/packages/README.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/README.md README.md: 3f467641bbc9eae14a94aa2d3bff0402116a9d3f -README.zh.md: faa879f3b5b62527c1d76ab3aff6a737f876ce4d +README.zh.md: c9d11bbf6239b4239a4e037dac63b05d3a9a58f7 diff --git a/packages/README.zh.md b/packages/README.zh.md index bed31b657c..b84aef020a 100644 --- a/packages/README.zh.md +++ b/packages/README.zh.md @@ -19,7 +19,7 @@ | [`subprocess/`](subprocess/README.md) | 进程管理能力系列:spawn seam + 本地进程树实现 | 产品:稳定表面 | | [`bash/`](bash/README.md) | Bash 能力系列:执行器 seam、本地实现、面向模型的工具 | 产品:稳定表面 | | [`pty/`](pty/README.md) | 持久 PTY 能力系列:按所有者隔离的会话、本地实现和面向模型的工具 | 产品:稳定表面 | -| [`code-runtime/`](code-runtime/README.md) | 代码执行能力系列:用于模型编写程序的运行时 seam + worker 线程后端 | 产品:稳定表面 | +| [`code-runtime/`](code-runtime/README.md) | 代码执行能力系列:面向模型所写程序的运行时 seam + worker 线程后端 | 产品:稳定表面 | | [`sandbox/`](sandbox/README.md) | 进程限制 seam;bwrap/Landlock/Seatbelt 后端 | 产品:稳定表面 | | [`fs/`](fs/README.md) | 文件系统能力系列:seam、本地实现、面向模型的文件工具、bash 后端发现工具 | 产品:稳定表面 | | [`lsp/`](lsp/README.md) | LSP 能力系列:seam、通用 stdio 提供方和 `lsp` 工具 | 产品:稳定表面 | From 921f4ff7a32d3c8189e4983071281cdbc1525800 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Thu, 30 Jul 2026 14:53:28 +0800 Subject: [PATCH 053/108] fix(pty): retain sends through cancellation polling --- packages/pty/pty-local/src/session.ts | 4 +- packages/pty/pty-local/tests/session.spec.ts | 49 ++++++++++++++++++++ 2 files changed, 51 insertions(+), 2 deletions(-) diff --git a/packages/pty/pty-local/src/session.ts b/packages/pty/pty-local/src/session.ts index faea1ef84d..7116f44611 100644 --- a/packages/pty/pty-local/src/session.ts +++ b/packages/pty/pty-local/src/session.ts @@ -412,7 +412,7 @@ export class LocalPtySession implements PtyBackendSession { return } const foreground = await this.terminal.inspectForeground() - if (this.active !== operation || this.closing) return + if (this.active !== operation || this.closing || this.interrupting === operation) return const idleFor = Date.now() - this.lastOutputAt if (this.promptSeen && foreground !== undefined && this.shellPgid === undefined) { this.shellPgid = foreground.processGroupId @@ -439,7 +439,7 @@ export class LocalPtySession implements PtyBackendSession { this.settleActive('inferred_idle') } } catch (error: unknown) { - if (this.active === operation && !this.closing) this.failActive(error) + if (this.active === operation && !this.closing && this.interrupting !== operation) this.failActive(error) } finally { this.polling = false const active = this.active diff --git a/packages/pty/pty-local/tests/session.spec.ts b/packages/pty/pty-local/tests/session.spec.ts index 8d11f8e6f4..27abfcf74e 100644 --- a/packages/pty/pty-local/tests/session.spec.ts +++ b/packages/pty/pty-local/tests/session.spec.ts @@ -406,6 +406,7 @@ describe('LocalPtySession readiness and output', () => { await Promise.resolve() await vi.advanceTimersByTimeAsync(10) expect(inspections).toBe(2) + await vi.advanceTimersByTimeAsync(50) expect(operation.cancel()).toBe(true) let settled = false void operation.done.then(() => { settled = true }) @@ -423,6 +424,54 @@ describe('LocalPtySession readiness and output', () => { expect((await operation.done).waitReason).toBe('session_exit') }) + it('does not let an in-flight readiness failure release a canceled send before signalling settles', async () => { + vi.useFakeTimers() + const terminal = new FakeTerminal() + const inspector = new FakeInspector() + const session = makeSession(terminal, inspector, config()) + await initialize(session, terminal) + + const readiness = Promise.withResolvers() + let inspections = 0 + terminal.inspectForeground = async () => { + inspections += 1 + if (inspections === 1) return { processGroupId: 456, inputWaiting: false } + if (inspections === 2) return await readiness.promise + return { processGroupId: 456, inputWaiting: true } + } + const signalling = Promise.withResolvers() + const signalled = Promise.withResolvers() + terminal.signalForeground = async (signal) => { + await signalling.promise + const foreground = await terminal.inspectForeground() + if (foreground === undefined) throw new Error('cannot resolve foreground') + inspector.signalGroup(foreground.processGroupId, signal) + signalled.resolve(foreground.processGroupId) + return foreground.processGroupId + } + + const operation = session.startSend({ text: 'first', submit: true }) + await Promise.resolve() + await Promise.resolve() + await vi.advanceTimersByTimeAsync(10) + expect(inspections).toBe(2) + expect(operation.cancel()).toBe(true) + let settled = false + void operation.done.then(() => { settled = true }) + + readiness.reject(new Error('inspection failed during cancellation')) + await Promise.resolve() + await Promise.resolve() + expect(settled).toBe(false) + expect(() => session.startSend({ text: 'successor', submit: true })).toThrow(PtyError) + + signalling.resolve(undefined) + expect(await signalled.promise).toBe(456) + expect(inspector.groups).toContainEqual([456, 'SIGINT']) + await session.close('test complete') + expect((await operation.done).waitReason).toBe('session_exit') + }) + it('signals only after an in-flight provider write lands', async () => { vi.useFakeTimers() const terminal = new FakeTerminal() From 6e4bdbf0df411caca431986f39f8221ba6b64664 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Thu, 30 Jul 2026 16:01:38 +0800 Subject: [PATCH 054/108] fix(pty): wire persistent Bash through subprocess --- examples/jsonrpc-agent/persistent-tools.cordis.yml | 3 +++ packages/pty/tool-bash-persistent/package.json | 1 + .../pty/tool-bash-persistent/tests/loader-composition.spec.ts | 3 +++ pnpm-lock.yaml | 3 +++ 4 files changed, 10 insertions(+) diff --git a/examples/jsonrpc-agent/persistent-tools.cordis.yml b/examples/jsonrpc-agent/persistent-tools.cordis.yml index 6f42441ca7..ebe0a00e61 100644 --- a/examples/jsonrpc-agent/persistent-tools.cordis.yml +++ b/examples/jsonrpc-agent/persistent-tools.cordis.yml @@ -18,6 +18,9 @@ mode: danger-full-access workspaceRoot: !!js process.env.DSH_CWD ?? process.cwd() +- id: subprocess + name: '@deepseek-ai/dsh-subprocess-local' + - id: pty name: '@deepseek-ai/dsh-pty' diff --git a/packages/pty/tool-bash-persistent/package.json b/packages/pty/tool-bash-persistent/package.json index b5cede3920..23aad9f32e 100644 --- a/packages/pty/tool-bash-persistent/package.json +++ b/packages/pty/tool-bash-persistent/package.json @@ -45,6 +45,7 @@ "@deepseek-ai/dsh-sandbox": "workspace:^", "@deepseek-ai/dsh-sandbox-policy": "workspace:^", "@deepseek-ai/dsh-session": "workspace:^", + "@deepseek-ai/dsh-subprocess-local": "workspace:^", "@deepseek-ai/dsh-system-prompt": "workspace:^", "@deepseek-ai/dsh-timeout": "workspace:^", "@deepseek-ai/dsh-tools": "workspace:^", diff --git a/packages/pty/tool-bash-persistent/tests/loader-composition.spec.ts b/packages/pty/tool-bash-persistent/tests/loader-composition.spec.ts index 16c4a7fea5..a69048e8d6 100644 --- a/packages/pty/tool-bash-persistent/tests/loader-composition.spec.ts +++ b/packages/pty/tool-bash-persistent/tests/loader-composition.spec.ts @@ -15,6 +15,7 @@ import * as PtyLocal from '@deepseek-ai/dsh-pty-local' import SandboxProvider from '@deepseek-ai/dsh-sandbox' import type { ConfinedArgv, SandboxPolicy } from '@deepseek-ai/dsh-sandbox' import SandboxPolicyService from '@deepseek-ai/dsh-sandbox-policy' +import LocalSubprocessService from '@deepseek-ai/dsh-subprocess-local' import SystemPrompt from '@deepseek-ai/dsh-system-prompt' import ToolRegistry from '@deepseek-ai/dsh-tools' import * as ToolBashPersistent from '@deepseek-ai/dsh-tool-bash-persistent' @@ -78,6 +79,7 @@ suite('persistent Bash through a real cordis.yml Loader composition', () => { ' config:', ' mode: danger-full-access', ` workspaceRoot: ${JSON.stringify(root)}`, + "- name: '@deepseek-ai/dsh-subprocess-local'", "- name: '@deepseek-ai/dsh-pty-local'", ' config:', ' pollIntervalMs: 10', @@ -104,6 +106,7 @@ suite('persistent Bash through a real cordis.yml Loader composition', () => { ['@deepseek-ai/dsh-pty', PtyService], ['@deepseek-ai/dsh-test-sandbox', PassthroughSandbox], ['@deepseek-ai/dsh-sandbox-policy', SandboxPolicyService], + ['@deepseek-ai/dsh-subprocess-local', LocalSubprocessService], ['@deepseek-ai/dsh-pty-local', PtyLocal], ['@deepseek-ai/dsh-tool-bash-persistent', ToolBashPersistent], ]) diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index ccbc67c764..666d01c1c1 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -4508,6 +4508,9 @@ importers: '@deepseek-ai/dsh-session': specifier: workspace:^ version: link:../../core/session + '@deepseek-ai/dsh-subprocess-local': + specifier: workspace:^ + version: link:../../subprocess/subprocess-local '@deepseek-ai/dsh-system-prompt': specifier: workspace:^ version: link:../../core/system-prompt From d689907c121ea1f012cb18ecee835b4f567ece2e Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Thu, 30 Jul 2026 17:25:44 +0800 Subject: [PATCH 055/108] test(subprocess): drop obsolete cwd stubs --- packages/pty/pty-local/tests/index.spec.ts | 1 - packages/subprocess/subprocess/tests/service.spec.ts | 2 -- 2 files changed, 3 deletions(-) diff --git a/packages/pty/pty-local/tests/index.spec.ts b/packages/pty/pty-local/tests/index.spec.ts index 15e73ddc1c..e6c4fde452 100644 --- a/packages/pty/pty-local/tests/index.spec.ts +++ b/packages/pty/pty-local/tests/index.spec.ts @@ -66,7 +66,6 @@ function terminalHandle(): SubprocessTerminalHandle { } class StubSubprocessService extends SubprocessService { - readonly cwd = '/tmp' async resolveExecutable(command: string): Promise { return command } spawn(_spec: SubprocessSpawnSpec): SubprocessHandle { throw new Error('unused') } async spawnTerminal(_spec: SubprocessTerminalSpawnSpec): Promise { diff --git a/packages/subprocess/subprocess/tests/service.spec.ts b/packages/subprocess/subprocess/tests/service.spec.ts index 2084310da9..e4f770a9a3 100644 --- a/packages/subprocess/subprocess/tests/service.spec.ts +++ b/packages/subprocess/subprocess/tests/service.spec.ts @@ -16,8 +16,6 @@ import type { * is all an implementation owes the abstract class. */ class StubSubprocessService extends SubprocessService { - readonly cwd = '/stub' - async resolveExecutable(command: string): Promise { return `/bin/${command}` } From a662af94df3e628645bca2ca223b90e5f88bc2a6 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Thu, 30 Jul 2026 19:08:07 +0800 Subject: [PATCH 056/108] test(lsp): accept eager dead-slot eviction --- packages/lsp/lsp-local/tests/lifecycle.spec.ts | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/packages/lsp/lsp-local/tests/lifecycle.spec.ts b/packages/lsp/lsp-local/tests/lifecycle.spec.ts index 7a30232e22..20c4bc7bf0 100644 --- a/packages/lsp/lsp-local/tests/lifecycle.spec.ts +++ b/packages/lsp/lsp-local/tests/lifecycle.spec.ts @@ -282,8 +282,9 @@ describe('lsp-local end to end over a fake server', () => { readonly instances: ReadonlyMap }).instances const instance = [...instances.values()][0] - if (instance === undefined) throw new Error('expected one pooled LSP instance') - await waitFor(async () => instance.dead) + // The query's finally may already have observed the exit and evicted the dead slot. When the + // slot remains, synchronize with its close before proving the next query replaces it. + if (instance !== undefined) await waitFor(async () => instance.dead) expect(await ctx.lsp.query(query('goToDefinition'))).toMatchObject({ kind: 'locations' }) await ctx.fiber.dispose() }) From e385c11e8e7b5d5d787c457760184b42fbd23767 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Sun, 2 Aug 2026 14:11:57 +0800 Subject: [PATCH 057/108] fix(pty,subprocess,lsp): address review round on cancellation, lookup, and contracts - pty-local: a pre-write inspection rejection no longer releases a canceled send while its foreground SIGINT is in flight; the interrupt path retains the slot and its post-signal tail resumes polling. Regression pins the failure-shaped cancellation and a close-during-write release. - pty-local: SEND_ACTIVE names which provider operation is draining; README states the never-settling-provider outcome (slot retained, close recovers). - subprocess-local: resolveExecutable rejects relative paths containing separators instead of expanding them per PATH entry with a misleading not-found error; seam JSDoc pins the rule for every provider. - subprocess-local: LocalTerminalHandle documents why the seam's in-flight-join promise holds without operation tracking. - lsp-local: the oversized-source diagnostic reports the observed byte lower bound; README documents that processId: null trades away server-side orphan cleanup after a hard-killed harness. - pty/subprocess seams: cross-reference the twin five-member signal unions. - pty-local: TODO markers for send-state consolidation and initialize-race relocation. --- .../cordis/tool-cordis/src/api-catalog.ts | 2 +- packages/lsp/lsp-local/README.i18n.yaml | 4 +- packages/lsp/lsp-local/README.md | 1 + packages/lsp/lsp-local/README.zh.md | 27 +++++---- packages/lsp/lsp-local/src/host.ts | 6 +- packages/lsp/lsp-local/tests/host.spec.ts | 4 +- packages/pty/pty-local/README.i18n.yaml | 4 +- packages/pty/pty-local/README.md | 2 +- packages/pty/pty-local/README.zh.md | 2 +- packages/pty/pty-local/src/index.ts | 3 + packages/pty/pty-local/src/session.ts | 30 +++++++++- packages/pty/pty-local/tests/session.spec.ts | 59 +++++++++++++++++++ packages/pty/pty/src/types.ts | 6 +- .../subprocess-local/README.i18n.yaml | 4 +- .../subprocess/subprocess-local/README.md | 2 +- .../subprocess/subprocess-local/README.zh.md | 2 +- .../subprocess/subprocess-local/src/index.ts | 5 ++ .../subprocess-local/src/terminal.ts | 9 ++- .../subprocess-local/tests/local.spec.ts | 6 ++ packages/subprocess/subprocess/src/index.ts | 4 +- packages/subprocess/subprocess/src/types.ts | 6 +- 21 files changed, 155 insertions(+), 33 deletions(-) diff --git a/packages/cordis/tool-cordis/src/api-catalog.ts b/packages/cordis/tool-cordis/src/api-catalog.ts index 2eb410acc6..21848e55e2 100644 --- a/packages/cordis/tool-cordis/src/api-catalog.ts +++ b/packages/cordis/tool-cordis/src/api-catalog.ts @@ -988,7 +988,7 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [ methods: [ { signature: 'abstract resolveExecutable( command: string, env?: Readonly>, signal?: AbortSignal, ): Promise', - jsDoc: '/**\n * Resolve one configured executable in this provider\'s execution world.\n * Absolute paths are verified; bare names use the provider\'s scrubbed PATH\n * plus explicit environment overrides.\n * @param command - absolute executable path or bare PATH name.\n * @param env - explicit environment entries used for lookup.\n * @param signal - aborts remote or local lookup.\n * @returns a canonical executable path.\n */', + jsDoc: '/**\n * Resolve one configured executable in this provider\'s execution world.\n * Absolute paths are verified; bare names use the provider\'s scrubbed PATH\n * plus explicit environment overrides. Relative paths containing separators\n * are rejected: no current consumer defines which directory they would\n * resolve against, so providers fail loud instead of guessing.\n * @param command - absolute executable path or bare PATH name.\n * @param env - explicit environment entries used for lookup.\n * @param signal - aborts remote or local lookup.\n * @returns a canonical executable path.\n */', }, { signature: 'abstract spawn(spec: SubprocessSpawnSpec): SubprocessHandle', diff --git a/packages/lsp/lsp-local/README.i18n.yaml b/packages/lsp/lsp-local/README.i18n.yaml index 55a9f97165..03487d5719 100644 --- a/packages/lsp/lsp-local/README.i18n.yaml +++ b/packages/lsp/lsp-local/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/lsp/lsp-local/README.md -README.md: ad8f4bc2318a58202f9596a18d402d2c6d45dae1 -README.zh.md: ff2686856a9e518a7fa846edf700da1b32e8a114 +README.md: 661c27d3326adfc3408b33550a63fe3ebe183a39 +README.zh.md: 83f0eaa1fb4caccc381a1623b791355b2f5c5b65 diff --git a/packages/lsp/lsp-local/README.md b/packages/lsp/lsp-local/README.md index ad8f4bc231..93d123fac7 100644 --- a/packages/lsp/lsp-local/README.md +++ b/packages/lsp/lsp-local/README.md @@ -57,3 +57,4 @@ No direct invalidation; `dsh-tool-lsp` owns request-prefix changes. - **No confinement policy** — this package trusts the configured server and does not sandbox its process; a restricted deployment must supply appropriate process/filesystem providers or a same-world sandbox wrapper. - **Transient-open compatibility floor** — servers whose synchronization omits open/close (or advertise `None`) are unsupported even if closed-document queries would work; the pinned TypeScript e2e establishes one compatibility floor, not a cross-language claim. - **Per-server/workspace serialization latency** — parallel agents sharing one server and workspace queue behind one process; long-lived workspace processes consume memory until disposal. +- **A hard-killed harness orphans language servers** — `initialize.processId: null` removes server-side client-PID monitoring, so servers are cleaned only by graceful service disposal; a SIGKILL'd harness leaves them running until they exit on their own. diff --git a/packages/lsp/lsp-local/README.zh.md b/packages/lsp/lsp-local/README.zh.md index ff2686856a..83f0eaa1fb 100644 --- a/packages/lsp/lsp-local/README.zh.md +++ b/packages/lsp/lsp-local/README.zh.md @@ -2,19 +2,19 @@ [English](README.md) | 中文 -`ctx.lsp` 的**通用 stdio 语言服务器后端**。一个插件实例接受一张命名服务器表,并逐配置项注册一个隔离的提供方。它通过 `ctx.fs` 读取,并通过 `ctx.subprocess` 启动,因此服务器与源文件始终位于所挂载的同一执行环境。这是通用主机,而不是语言服务器目录或安装器:部署需要显式配置命令与映射,preset 应放在 `cordis.yml` overlay 中。 +`ctx.lsp` 的**通用 stdio 语言服务器后端**。一个插件实例接受一张命名服务器表,并逐配置项注册一个隔离的提供方。它通过 `ctx.fs` 读取,并通过 `ctx.subprocess` 启动,因此服务器与源文件始终位于已挂载的执行世界中。这是通用主机,而不是语言服务器目录或安装器:部署需要显式配置命令与映射,预设应放在 `cordis.yml` overlay 中。 Namespace 插件(`name`/`inject`/`Config`/`apply`,无默认导出)。 ## 功能 - 在注册前解析每项服务器局部设置;无效映射或注册冲突会回滚较早配置项,因此加载失败不会留下提供方路由。 -- 每个 `(server id, canonical workspace target)` 惰性 single-flight 一个服务器进程。存活服务器错误不会回放;如果选中的池化传输在只读查询之前或期间失败,提供方会等待其释放,并在新进程上重试该查询一次。 -- 每次查询都使用兼容性优先的**临时打开** 序列:通过 `ctx.fs` 解析并流式读取源文件,同时执行字节上限;随后执行 `textDocument/didOpen`(版本 1、完整文本)、所请求操作,以及位于 `finally` 中的 `textDocument/didClose`。写入 `didOpen` 失败或取消时,会先终止实例再允许池复用。文档在每次调用后关闭,因此第一版不需要 `didChange`、内容 cache 或文档 LRU。 -- 通过一条逐 Workspace、可中止的队列,串行执行每个源读取/打开/查询/关闭生命周期,因此排队调用只会在轮到自身时读取当前源;不同 Workspace 并行运行。提供方资源释放会中止文件系统与协议工作,等待尚未进入队列的 Workspace 查找结算,再排空所有队列并等待所有服务器结算。 -- 协议 shutdown 失败后,经由进程管理器 seam 终止服务器后代树(POSIX 进程组信号;Windows `taskkill /T /F`)。树终止的投递结果与所有进程组信号一样被就地吸收,不向外抛出(投递与服务器退出存在竞态);服务器是否完全停稳,由句柄的进程树存活等待确认,而非由这次终止自身的结果确认。 -- 通过 `ctx.subprocess` 解析服务器可执行文件、cwd、进程与协议流;`initialize.processId` 为 `null`,因为另一台机器或 PID 命名空间不得监控 harness 进程。 -- 使用 `ctx.fs` 提供的规范 containment、文件 URI 与流式文本校验,但不发出 `fs/observed`:只有 LSP 结果对模型可见,因此查询不满足先读后写策略。 +- 每个 `(server id, canonical workspace target)` 惰性 single-flight 一个服务器进程。存活服务器错误不会回放;如果选中的池化传输在只读查询之前或期间失败,提供方会等待其 dispose(资源释放)完成,并在新进程上重试该查询一次。 +- 每次查询都使用兼容性优先的**临时打开**序列:通过 `ctx.fs` 流式读取源文件,同时解析并限制其字节数;随后执行 `textDocument/didOpen`(版本 1、完整文本)、所请求操作,再执行位于 `finally` 中的 `textDocument/didClose`。写入 `didOpen` 失败或取消时,会在池复用该实例前将其终止。文档在每次调用后关闭,因此第一版不需要 `didChange`、内容 cache 或文档 LRU。 +- 通过一条逐 Workspace、可中止的队列,串行执行每个源读取/打开/查询/关闭生命周期,因此排队调用只会在轮到自身时读取当前源;不同 Workspace 并行运行。提供方 dispose 会中止文件系统与协议工作,等待尚未进入队列的 Workspace 查找完成,随后排空每条队列与每个服务器。 +- 协议 shutdown 失败后,经由子进程 seam 终止服务器后代树(POSIX 进程组信号;Windows `taskkill /T /F`)。树终止的投递结果与所有进程组信号一样被就地吸收,不向外抛出(投递与服务器退出存在竞态);服务器是否完全停稳,由句柄的进程树存活等待确认,而非由这次终止自身的结果确认。 +- 通过 `ctx.subprocess` 解析服务器可执行文件、cwd、进程和协议流;`initialize.processId` 为 `null`,因为另一台机器或 PID namespace 不得监视 harness 进程。 +- 使用 `ctx.fs` 提供的规范化包含关系、文件 URI 与流式文本验证,但不发出 `fs/observed`:只有 LSP 结果对模型可见,因此查询不满足先读后写策略。 ## 配置 @@ -24,13 +24,13 @@ Namespace 插件(`name`/`inject`/`Config`/`apply`,无默认导出) |---|---|---| | `command` | (必填) | 要 spawn 的可执行文件:绝对路径,或在加载时从子进程 PATH 解析。不使用 shell 启动。 | | `args` | `[]` | 传给可执行文件的参数。 | -| `env` | `{}` | 合并到已清理 credential 的环境之上的额外 env(匹配 `KEY`/`SECRET`/`TOKEN` 的变量不会转发);显式 `DSH_*` 条目在 seam 清除环境中同名值之后合并。 | +| `env` | `{}` | 合并到已清理 credential 的环境之上的额外 env(匹配 `KEY`/`PASSWORD`/`SECRET`/`TOKEN` 的变量不会转发);显式 `DSH_*` 条目在 seam 清除环境中同名值之后合并。 | | `extensionToLanguage` | (必填) | 小写、以点开头的扩展名 → LSP language id(例如 `{ '.ts': 'typescript' }`)。 | | `initializationOptions` | `null` | 转发给服务器的静态 `initialize` 选项。 | | `configuration` | `null` | 每个 `workspace/configuration` 配置项的静态答案。 | | `maxMessageBytes` | `16000000` | 从服务器接受的单条 framed 消息最大大小。 | | `maxStderrBytes` | `1000000` | 为诊断保留的 stderr 尾部最大大小。 | -| `maxDocumentBytes` | `4000000` | 该主机可打开的最大源文件。 | +| `maxDocumentBytes` | `4000000` | 该主机可打开的源文件大小上限。 | | `shutdownTimeoutMs` | `5000` | 升级前用于优雅 `shutdown`/`exit` 的预算。 | | `killGraceMs` | `2000` | 请求取消及 SIGTERM→SIGKILL 升级的宽限期。 | @@ -38,11 +38,11 @@ Namespace 插件(`name`/`inject`/`Config`/`apply`,无默认导出) ## 协议行为 -初始化会声明 `general.positionEncodings: ['utf-16']`、`workspace: { workspaceFolders: true, configuration: true }`、`textDocument.hover.contentFormat: ['markdown', 'plaintext']`,以及定义与实现使用的 `linkSupport: true`,且不进行动态注册。服务器返回的能力具有最终决定权:不受支持的操作,或缺少临时打开/关闭的同步方式,会使查询失败。服务器省略 `positionEncoding` 时默认为 `utf-16`;其他值都属于协议错误。客户端通过静态配置回答 `workspace/configuration`,接受生命周期记账请求,并拒绝 `workspace/applyEdit`:它绝不应用编辑或运行命令。导航直接映射 `Location`,并从 `LocationLink` 的 `targetUri` + `targetSelectionRange` 映射;hover 规范化会取得有效的 `MarkupContent.value`,保留 string `MarkedString`,把带 language tag 的值渲染为围栏代码,并用一个空行连接数组。缺失结果、格式错误的范围或位置,以及格式错误的 hover 编码,都会作为结构化 `LSP_MALFORMED_RESPONSE` 错误失败。 +初始化会声明 `general.positionEncodings: ['utf-16']`、`workspace: { workspaceFolders: true, configuration: true }`、`textDocument.hover.contentFormat: ['markdown', 'plaintext']`,以及定义与实现使用的 `linkSupport: true`,且不进行动态注册。服务器返回的能力具有最终决定权:不受支持的操作,或缺少临时打开/关闭的同步方式,会使查询失败。服务器省略 `positionEncoding` 时默认为 `utf-16`;其他值都属于协议错误。客户端通过静态配置回答 `workspace/configuration`,接受生命周期记账请求,并拒绝 `workspace/applyEdit`:它绝不应用编辑或运行命令。导航直接映射 `Location`,并从 `LocationLink` 的 `targetUri` + `targetSelectionRange` 映射;hover 规范化会取得有效的 `MarkupContent.value`,保留 string `MarkedString`,把带 language tag 的值渲染为围栏代码,并用一个空行连接数组。缺失结果、格式错误的范围或位置,以及格式错误的 hover 编码,都会以结构化 `LSP_MALFORMED_RESPONSE` 错误的形式失败。 ## 安全边界 -提供方信任其配置的服务器,不声明任何沙箱限制。它把规范身份、containment、普通文件流式读取、UTF-8 校验与文件 URI 编码委托给 `ctx.fs`;服务器启动前,系统会拒绝缺失、非普通文件、非 UTF-8、过大或规范路径位于工作区外的查询源。系统在打开流之前检查 containment,但不保证路径并发替换期间的稳定句柄身份。结果位置可以在外部,但外部路径不能成为查询源。部署必须为同一执行环境挂载文件系统与子进程提供方;分裂执行环境的组合无效。 +提供方信任其配置的服务器,不提供任何沙箱隔离。它把规范化身份、包含关系、普通文件流式读取、UTF-8 验证和文件 URI 编码委托给 `ctx.fs`;并在服务器启动前拒绝缺失、非普通文件、非 UTF-8、过大,或规范化后位于 Workspace 外部的查询源。包含关系在打开流之前评估,不承诺在并发路径替换期间保持稳定句柄身份。结果位置可以在外部,但外部路径不能成为查询源。部署必须挂载描述同一执行世界的文件系统与进程管理提供方;分裂世界组合无效。 ## 模型体验 @@ -54,6 +54,7 @@ Namespace 插件(`name`/`inject`/`Config`/`apply`,无默认导出) ## 已知限制与暂缓事项 -- **不提供隔离策略**:这个包(package)信任配置的服务器,不会对其进程执行沙箱化;受限部署必须提供适当的进程/文件系统提供方,或包装同一执行环境的沙箱。 +- **不提供隔离策略**:本包(package)信任所配置的服务器,不对其进程实施沙箱;受限部署必须提供适当的进程/文件系统提供方,或使用同一执行世界的沙箱包装层。 - **临时打开兼容性下限**:同步能力省略打开/关闭(或声明 `None`)的服务器不受支持,即使关闭文档查询能够工作;固定的 TypeScript e2e 只建立一项兼容性下限,不代表跨语言承诺。 -- **逐服务器/Workspace 串行化延迟**:共享同一个服务器与 Workspace 的并行 agent 会在一个进程后排队;长生命周期 Workspace 进程会占用内存直到释放。 +- **逐服务器/Workspace 串行化延迟**:共享同一个服务器与 Workspace 的并行 agent(智能体)会在一个进程后排队;长生命周期 Workspace 进程会占用内存直到 dispose。 +- **被强制杀死的 harness 会遗留语言服务器**:`initialize.processId: null` 取消了服务器侧的客户端 PID 监视,因此服务器只能由服务的优雅 dispose 清理;被 SIGKILL 的 harness 会让它们继续运行,直到自行退出。 diff --git a/packages/lsp/lsp-local/src/host.ts b/packages/lsp/lsp-local/src/host.ts index ffba6a1344..ef4b56809d 100644 --- a/packages/lsp/lsp-local/src/host.ts +++ b/packages/lsp/lsp-local/src/host.ts @@ -107,7 +107,11 @@ export async function readHostSource( throwIfAborted(signal) throw new Error(`source "${filePath}" could not be read: ${messageOf(error)}`, { cause: error }) } - if (bytes > maxDocumentBytes) throw new Error(`source "${filePath}" exceeds the ${maxDocumentBytes}-byte limit`) + if (bytes > maxDocumentBytes) { + throw new Error( + `source "${filePath}" exceeds the ${maxDocumentBytes}-byte limit; reading stopped after ${bytes} bytes`, + ) + } throwIfAborted(signal) return { fileUrl: fs.fileUrl(target), diff --git a/packages/lsp/lsp-local/tests/host.spec.ts b/packages/lsp/lsp-local/tests/host.spec.ts index c97defc086..c06b6d9df8 100644 --- a/packages/lsp/lsp-local/tests/host.spec.ts +++ b/packages/lsp/lsp-local/tests/host.spec.ts @@ -156,10 +156,10 @@ describe('readHostSource', () => { await expect(readSource('.')).rejects.toThrow(/not a regular file/) }) - it('rejects an oversized source', async () => { + it('rejects an oversized source and reports the observed lower bound', async () => { await writeFile(join(ws, 'big.ts'), 'x'.repeat(100)) await expect(readSource('big.ts', 10)).rejects.toMatchObject({ - message: 'source "big.ts" exceeds the 10-byte limit', + message: 'source "big.ts" exceeds the 10-byte limit; reading stopped after 100 bytes', }) }) diff --git a/packages/pty/pty-local/README.i18n.yaml b/packages/pty/pty-local/README.i18n.yaml index 2689103443..bd835a091d 100644 --- a/packages/pty/pty-local/README.i18n.yaml +++ b/packages/pty/pty-local/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/pty/pty-local/README.md -README.md: 92a3d7be68ada6f38ab3c4ca5bdd2622ebefc8ee -README.zh.md: 016043f7842df8bb963cdc5eea015666c9ce6941 +README.md: 5acc92853e6e8fcb8938c48e391559bf4a28fb75 +README.zh.md: 353c2a4bdac7e8402fc63071dfb6fb85dcff66d5 diff --git a/packages/pty/pty-local/README.md b/packages/pty/pty-local/README.md index 92a3d7be68..3b1112b025 100644 --- a/packages/pty/pty-local/README.md +++ b/packages/pty/pty-local/README.md @@ -10,7 +10,7 @@ The plugin injects `pty`, `sandboxPolicy`, and `subprocess`, then registers the Readiness combines a foreground-verified private bash prompt marker, provider-reported foreground stdin-wait facts, silence fallback, and absolute timeout. A marker is not ready until the printable tail after the latest owned marker exactly equals the controlled `PS1`, including when the OSC marker and prompt are split across data callbacks; echoed input or output following an earlier prompt therefore cannot settle the current send. Prompt and silence evidence collected before the provider write, including while pre-write foreground inspection is pending, is discarded at the write boundary. When bash prints the marker before the terminal provider publishes its return to the foreground process group, polling retains the candidate for `handoffGraceMs` past the ordinary silence bound so a coincident handoff can win. An interactive child that inherits `PROMPT_COMMAND` therefore cannot suppress inferred-idle readiness until the absolute timeout. Unknown foreground state is never a positive exact-idle signal. A foreground group's stdin wait that existed before a send is likewise not post-write readiness: the same group must be observed outside that wait before a later wait can settle the send, while a changed foreground group is new evidence. During unpublished startup, a fallback requires observed output; zero-output silence cannot publish an empty session, and timeout rejects the spawn. Cancellation closes the unpublished shell and rejects with the caller's exact abort reason; `PtyBackendCleanupError` separately preserves a cleanup failure. The caller's signal is forwarded for terminal allocation and readiness initialization; after publication the handle owns its lifetime. Incomplete terminal-control sequences are bounded by `maxReadBytes` and discarded through their terminator after crossing that limit; malformed UTF-8 terminal output uses replacement characters, and a trailing carriage return is carried across callbacks so split CRLF becomes one newline. -Send cancellation marks queued input as canceled before asking the terminal handle to signal the current foreground process group with a real `SIGINT`; if asynchronous pre-write inspection later settles, it cannot execute that input. If a provider write is already in flight, signalling waits for it to settle; a rejected write sends no signal. The canceled send retains its slot until the write and foreground signalling settle, so a successor cannot receive either late bytes or that signal. The absolute deadline remains armed while cancellation waits. A signal failure is a terminal transport failure and rejects the active send. Cancellation never emulates interruption by writing `\x03`, so raw-mode programs remain cancellable. Close rejects new public signals, stops readiness polling, and awaits the handle's provider-owned complete-session termination before settling the active send as `session_exit`. +Send cancellation marks queued input as canceled before asking the terminal handle to signal the current foreground process group with a real `SIGINT`; if asynchronous pre-write inspection later settles, it cannot execute that input. If a provider write is already in flight, signalling waits for it to settle; a rejected write sends no signal. The canceled send retains its slot until the write and foreground signalling settle, so a successor cannot receive either late bytes or that signal. A provider write or signal that never settles therefore retains the slot indefinitely; closing the session (`terminal_close`) is the recovery. The absolute deadline remains armed while cancellation waits. A signal failure is a terminal transport failure and rejects the active send. Cancellation never emulates interruption by writing `\x03`, so raw-mode programs remain cancellable. Close rejects new public signals, stops readiness polling, and awaits the handle's provider-owned complete-session termination before settling the active send as `session_exit`. ## Model Experience diff --git a/packages/pty/pty-local/README.zh.md b/packages/pty/pty-local/README.zh.md index 016043f784..a23a2f3338 100644 --- a/packages/pty/pty-local/README.zh.md +++ b/packages/pty/pty-local/README.zh.md @@ -10,7 +10,7 @@ 就绪检测结合以下机制:由前台状态验证的私有 bash 提示符标记、提供方报告的前台 stdin 等待事实、静默回退和绝对超时。只有最新自有标记之后的可打印尾部与受控 `PS1` 完全相等,标记才算就绪;即使 OSC 标记和提示符被拆到多个数据回调中也一样。因此,较早提示符之后的回显输入或输出无法使当前 send 完成。提供方写入前收集的提示符与静默证据,包括写入前前台检查仍在等待时收集的证据,都会在写入边界丢弃。如果 bash 在终端提供方发布其重新取得前台进程组的状态前打印标记,轮询会在普通静默上限之后再保留该候选状态 `handoffGraceMs`,使恰好同时发生的前台交接有机会胜出。因此,继承 `PROMPT_COMMAND` 的交互式子进程无法一直抑制推断空闲就绪直至绝对超时。未知的前台状态绝不会作为精确空闲的正向信号。同样,一次 send 之前就已存在的前台进程组 stdin 等待并不代表写入后就绪:必须先观察到同一进程组脱离该等待,之后再次进入等待才能使该次 send 完成;前台进程组发生变化则构成新的证据。尚未发布的启动过程中,回退路径要求已经观察到输出;零输出静默不能发布空会话,超时则拒绝 spawn。取消操作会关闭尚未发布的 shell,并以调用方提供的确切中止原因拒绝;`PtyBackendCleanupError` 会单独保留清理失败。调用方的 signal 会转发给终端分配与就绪初始化;发布后,句柄负责其生命周期。未完成的终端控制序列受 `maxReadBytes` 限制;超过上限后,系统会丢弃内容直到其终止符。格式错误的 UTF-8 终端输出使用替换字符;末尾的回车会跨回调保留,使拆分的 CRLF 合并为一个换行。 -取消发送时,系统会先把排队输入标记为已取消,再要求终端句柄向当前前台进程组发送真正的 `SIGINT`;异步写入前检查即使随后结算,也无法执行该输入。如果提供方写入已在途,信号发送会等待其结算;写入被拒绝时不会发送信号。已取消的 send 会保留其位置,直到写入与前台信号发送都结算,因此后继 send 不会收到延迟字节或该信号。取消等待期间,绝对 deadline 仍保持启用。信号发送失败是终端传输失败,会拒绝活跃 send。取消绝不会通过写入 `\x03` 模拟中断,因此,即使程序运行在 raw 模式下,也仍可取消。关闭操作会拒绝新的公开信号、停止就绪轮询,并等待由句柄提供方负责的完整会话终止,然后才把活跃 send 结算为 `session_exit`。 +取消发送时,系统会先把排队输入标记为已取消,再要求终端句柄向当前前台进程组发送真正的 `SIGINT`;异步写入前检查即使随后结算,也无法执行该输入。如果提供方写入已在途,信号发送会等待其结算;写入被拒绝时不会发送信号。已取消的 send 会保留其位置,直到写入与前台信号发送都结算,因此后继 send 不会收到延迟字节或该信号。因此,永不结算的提供方写入或信号会无限期保留该位置;恢复手段是关闭会话(`terminal_close`)。取消等待期间,绝对 deadline 仍保持启用。信号发送失败是终端传输失败,会拒绝活跃 send。取消绝不会通过写入 `\x03` 模拟中断,因此,即使程序运行在 raw 模式下,也仍可取消。关闭操作会拒绝新的公开信号、停止就绪轮询,并等待由句柄提供方负责的完整会话终止,然后才把活跃 send 结算为 `session_exit`。 ## 模型体验 diff --git a/packages/pty/pty-local/src/index.ts b/packages/pty/pty-local/src/index.ts index f91db5d386..b1e0e6edbd 100644 --- a/packages/pty/pty-local/src/index.ts +++ b/packages/pty/pty-local/src/index.ts @@ -82,6 +82,9 @@ function spawnArgv(ctx: Context, config: ResolvedConfig, spec: PtyBackendSpawnSp }).argv } +// TODO(pty-initialize-race-home): Fold this outer abort race into +// LocalPtySession.initialize when the send-state consolidation lands; the +// session already owns the send lifecycle the race protects. async function initializeSession(session: LocalPtySession, signal?: AbortSignal): Promise { if (signal === undefined) { await session.initialize(signal) diff --git a/packages/pty/pty-local/src/session.ts b/packages/pty/pty-local/src/session.ts index 7116f44611..25ed8fe586 100644 --- a/packages/pty/pty-local/src/session.ts +++ b/packages/pty/pty-local/src/session.ts @@ -162,6 +162,11 @@ export class LocalPtySession implements PtyBackendSession { private readonly outputEnded = Promise.withResolvers() private readonly completion: Promise private statusValue: PtySessionStatus = { kind: 'running' } + // TODO(pty-send-state-consolidation): Fold the per-send fields below + // (active/activeTimer/activeDeadlineTimer/activeAbort/interrupting/ + // activeWrite/pollingReady/polling) into one send-lifecycle owner; the + // cancellation/readiness interplay now has enough pinned tests to carry + // that refactor safely. private active: LocalSendOperation | undefined private activeTimer: NodeJS.Timeout | undefined private activeDeadlineTimer: NodeJS.Timeout | undefined @@ -220,7 +225,14 @@ export class LocalPtySession implements PtyBackendSession { startSend(request: PtySendRequest): PtySendOperation { if (this.closing) throw new Error('PTY session is closing') if (this.statusValue.kind === 'exited') throw new Error('PTY session has exited') - if (this.active !== undefined) throw new PtyError('PTY session already has an active send or draining provider operation', 'SEND_ACTIVE') + if (this.active !== undefined) { + const draining = this.activeWrite !== undefined + ? ' or draining provider write' + : this.interrupting !== undefined + ? ' or draining foreground interrupt' + : '' + throw new PtyError(`PTY session already has an active send${draining}`, 'SEND_ACTIVE') + } if (request.signal?.aborted === true) throw new Error('PTY send aborted before write') const operation = new LocalSendOperation( @@ -246,8 +258,22 @@ export class LocalPtySession implements PtyBackendSession { } private async beginSend(operation: LocalSendOperation, request: PtySendRequest): Promise { + let foreground: SubprocessTerminalForeground | undefined + try { + foreground = await this.terminal.inspectForeground() + } catch (error: unknown) { + // A pre-write inspection failure while cancellation owns the slot must not + // release it: interruptOnce's in-flight foreground signal could land on a + // successor's foreground group. The interrupt path's post-signal tail + // resumes polling, whose guarded catch propagates a persistent failure. + // A retained settled operation implies that same in-flight interrupt, so + // this guard admits only an unsettled active send. + if (this.active === operation && !this.closing && this.interrupting !== operation) { + this.failActive(error) + } + return + } try { - const foreground = await this.terminal.inspectForeground() if (this.active !== operation || this.closing || this.interrupting === operation) return operation.setInitialForeground(foreground) const input = `${request.text}${request.submit ? '\r' : ''}` diff --git a/packages/pty/pty-local/tests/session.spec.ts b/packages/pty/pty-local/tests/session.spec.ts index 27abfcf74e..0ca1835e53 100644 --- a/packages/pty/pty-local/tests/session.spec.ts +++ b/packages/pty/pty-local/tests/session.spec.ts @@ -345,6 +345,47 @@ describe('LocalPtySession readiness and output', () => { await operation.done }) + it('retains a canceled send when the pre-write inspection rejects while its signal is in flight', async () => { + vi.useFakeTimers() + const terminal = new FakeTerminal() + const inspector = new FakeInspector() + const session = makeSession(terminal, inspector, config()) + await initialize(session, terminal) + + const failed = Promise.withResolvers<{ processGroupId: number; inputWaiting: boolean }>() + terminal.inspectForeground = async () => await failed.promise + const uncanceled = session.startSend({ text: 'plain failure', submit: true }) + failed.reject(new Error('inspect failed before write')) + await expect(uncanceled.done).rejects.toThrow('inspect failed before write') + + terminal.inspectForeground = FakeTerminal.prototype.inspectForeground.bind(terminal) + const inspection = Promise.withResolvers<{ processGroupId: number; inputWaiting: boolean }>() + terminal.inspectForeground = async () => await inspection.promise + const signalGate = Promise.withResolvers() + terminal.signalForeground = async (signal) => { + await signalGate.promise + inspector.signalGroup(456, signal) + return 456 + } + const controller = new AbortController() + const operation = session.startSend({ text: 'must stay owned', submit: true, signal: controller.signal }) + controller.abort() + inspection.reject(new Error('transient inspection failure')) + await Promise.resolve() + await Promise.resolve() + + // The slot stays reserved while the cancellation's foreground signal is in flight. + expect(() => session.startSend({ text: 'successor', submit: true })).toThrow('active send') + terminal.inspectForeground = async () => ({ processGroupId: 456, inputWaiting: false }) + signalGate.resolve(undefined) + await vi.advanceTimersByTimeAsync(0) + expect(inspector.groups).toContainEqual([456, 'SIGINT']) + + terminal.emitData('\x1b]133;D;130\x07dsh> ') + await vi.advanceTimersByTimeAsync(10) + await operation.done + }) + it('retains a canceled send until asynchronous foreground signalling settles', async () => { vi.useFakeTimers() const terminal = new FakeTerminal() @@ -1176,6 +1217,24 @@ describe('LocalPtySession bounds, signals, and teardown', () => { }) it('settles a closing send when provider termination cancels inspection', async () => { + vi.useFakeTimers() + const terminal = new FakeTerminal() + const session = new LocalPtySession(terminal, config()) + await initialize(session, terminal) + const write = Promise.withResolvers() + terminal.write = async () => { await write.promise } + const writeOperation = session.startSend({ text: 'pending write', submit: true }) + await Promise.resolve() + await Promise.resolve() + await session.close('pending write') + expect((await writeOperation.done).waitReason).toBe('session_exit') + // The rejection lands after close released the send; it must stay contained. + write.reject(new Error('write rejected during close')) + await Promise.resolve() + await Promise.resolve() + }) + + it('settles a closing send when provider termination cancels a pending inspection', async () => { vi.useFakeTimers() const terminal = new FakeTerminal() const session = new LocalPtySession(terminal, config()) diff --git a/packages/pty/pty/src/types.ts b/packages/pty/pty/src/types.ts index a4985ead93..0317bb9d43 100644 --- a/packages/pty/pty/src/types.ts +++ b/packages/pty/pty/src/types.ts @@ -28,7 +28,11 @@ export class PtyBackendCleanupError extends AggregateError { /** Why one interactive send returned control to its caller. */ export type PtyWaitReason = 'stdin_read' | 'inferred_idle' | 'timeout' | 'session_exit' -/** Signals the model-facing PTY surface permits for foreground process groups. */ +/** + * Signals the model-facing PTY surface permits for foreground process groups. + * Kept member-identical to `SubprocessTerminalSignal` in + * `@deepseek-ai/dsh-subprocess` without a cross-seam dependency; change both together. + */ export type PtySignal = 'SIGINT' | 'SIGTERM' | 'SIGKILL' | 'SIGTSTP' | 'SIGHUP' /** Top-level PTY process status, independent of a send's wait reason. */ diff --git a/packages/subprocess/subprocess-local/README.i18n.yaml b/packages/subprocess/subprocess-local/README.i18n.yaml index ee90e9e607..87a95c3170 100644 --- a/packages/subprocess/subprocess-local/README.i18n.yaml +++ b/packages/subprocess/subprocess-local/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/subprocess/subprocess-local/README.md -README.md: 85103c2634bd35b188acd71c7f037e3678a2542e -README.zh.md: 505eac4f2650e6723e042cdb1122f9ef504ad7df +README.md: b901a69d6cfd45a084711ba0d32e555c481fd626 +README.zh.md: 5723dc99ff955c5f4b07ea96e7d2134100d9fd6a diff --git a/packages/subprocess/subprocess-local/README.md b/packages/subprocess/subprocess-local/README.md index c211ca26f1..9e3e8f7f2d 100644 --- a/packages/subprocess/subprocess-local/README.md +++ b/packages/subprocess/subprocess-local/README.md @@ -10,7 +10,7 @@ Local implementation of the [`@deepseek-ai/dsh-subprocess`](../subprocess/README - **Per-stream dispositions** — `'pipe'` hands the raw stream to the caller untouched (protocol framing stays consumer-owned); `'inherit'` passes the parent descriptor through; collect mode keeps the in-memory TAIL beyond its cap (errors and results cluster at the end — pi/OpenCode rationale) while the FULL stream is appended to a private temp file when a spill cap is configured — omitting `spill` keeps only the tail, the diagnostic shape. A stream larger than the spill cap discards its now-incomplete spill and returns only the marked truncated tail; spill fds are sealed at settlement, and a failed final close withholds the path rather than advertising an incomplete file. Spill files are `0600` with random names under a lazily-created `0700` per-process directory. - **Credential scrub + explicit merge** — `process.env` minus credential-shaped vars (`*KEY*`/`*SECRET*`/`*TOKEN*`) and all ambient `DSH_*` names; the spec's explicit `env` merges after that scrub with no namespace validation, so a deliberately supplied credential or current `DSH_*` fact wins while stale nested-harness identity cannot leak in ambiently. Supplied stdin is written and closed; otherwise fd 0 is `/dev/null`. See the [stdin/env Agent Note](../../../.agents/notes/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-surface.md) and [managed environment Agent Note](../../../.agents/notes/implemented/feature/2026-07-10-agent-session-identity-and-log-location.md). - **Offset-based reads** — collect-mode readers return deltas in whole-stream byte coordinates; the service never holds a cursor, so consumer-owned cursors (the bash background read path) and full-stream re-reads coexist, before and after settlement. -- **Executable lookup** — `resolveExecutable` checks absolute files or searches the scrubbed effective PATH with platform-aware executable extensions; relative PATH entries resolve from the host process cwd. +- **Executable lookup** — `resolveExecutable` checks absolute files or searches the scrubbed effective PATH with platform-aware executable extensions; relative paths containing separators are rejected at the seam, and relative PATH entries resolve from the host process cwd. - **Terminal-process ownership** — `spawnTerminal` allocates `node-pty`, bridges UTF-8 terminal text, inspects and signals the current foreground process group, and exposes one awaited termination operation that sweeps descendants before and after terminating the top-level shell. Each foreground inspection retains exact identities from the rooted tree; Linux also enumerates the POSIX session after its leader exits. A previously observed macOS descendant and any same-session Linux member therefore remain fenced after reparenting, while pid/start identity prevents cleanup from following PID reuse. The higher PTY backend owns prompt readiness, buffers, and model-facing operations. - **Terminate-and-join disposal** — the service retains live handles only so its own disposal can escalate every running tree and await its exit; settled and spawn-failed handles leave the live set on settlement. diff --git a/packages/subprocess/subprocess-local/README.zh.md b/packages/subprocess/subprocess-local/README.zh.md index 505eac4f26..5723dc99ff 100644 --- a/packages/subprocess/subprocess-local/README.zh.md +++ b/packages/subprocess/subprocess-local/README.zh.md @@ -10,7 +10,7 @@ - **按流划分的处置方式**:`'pipe'` 把原始流原样交给调用方(协议分帧仍归消费方所有);`'inherit'` 直通父进程的描述符;收集模式(collect)在输出超过上限后于内存中保留尾部(错误与结果通常聚集在末尾,沿用 pi/OpenCode 的理由),并在配置了 spill 上限时把完整流追加到一个私有临时文件;省略 `spill` 则只保留用于诊断的尾部。某条流大于 spill 上限时,会丢弃已不完整的 spill,仅返回带截断标记的尾部;spill 文件描述符在结算时封存,最终关闭失败时则不公布路径,以免声称存在不完整的文件。spill 文件权限为 `0600`、名称随机,位于按需创建、权限为 `0700` 的每进程目录之下。 - **凭据清除 + 显式合并**:以 `process.env` 为基础,移除形似凭据的变量(`*KEY*`/`*PASSWORD*`/`*SECRET*`/`*TOKEN*`)和所有环境中已有的 `DSH_*` 名称;spec 的显式 `env` 在该清除之后合并且不做命名空间校验,因此有意提供的凭据或当前 `DSH_*` 事实会胜出,而陈旧的嵌套 harness 身份无法从环境中隐式漏入。提供的 stdin 会被写入后关闭;否则 fd 0 指向 `/dev/null`。参见 [stdin/env Agent Note(agent 决策记录)](../../../.agents/notes/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-surface.md)与[受管环境 Agent Note](../../../.agents/notes/implemented/feature/2026-07-10-agent-session-identity-and-log-location.md)。 - **基于偏移量的读取**:收集模式的读取器按完整流的字节坐标返回增量;服务自身从不持有游标,因此消费方自有的游标(bash 的后台读取路径)与完整流重读可以共存,结算前后皆然。 -- **可执行文件查找**:`resolveExecutable` 检查绝对文件,或根据平台可执行文件扩展名在清理后的有效 PATH 中搜索;相对 PATH 条目从宿主进程 cwd 解析。 +- **可执行文件查找**:`resolveExecutable` 检查绝对文件,或根据平台可执行文件扩展名在清理后的有效 PATH 中搜索;含分隔符的相对路径在接缝处被拒绝,相对 PATH 条目从宿主进程 cwd 解析。 - **终端进程所有权**:`spawnTerminal` 分配 `node-pty`,桥接 UTF-8 终端文本,检查当前前台进程组并向其发送信号,还会公开一项须等待的终止操作,在终止顶层 shell 前后清理后代进程。每次前台检查都会保留根进程树中的精确身份;Linux 还会在 POSIX 会话 leader 退出后枚举该会话。因此,之前观察到的 macOS 后代以及同会话 Linux 成员在重新设定父进程后仍受围栏保护,pid/start 身份则防止清理跟随 PID 复用。上层 PTY 后端负责提示符就绪、缓冲区与面向模型的操作。 - **先终止再等待退出的 dispose(资源释放)**:服务保留存活句柄,只为让自身的 dispose 能对每个仍在运行的进程树执行升级并等待其退出;已结算与 spawn 失败的句柄在结算时即离开存活集合。 diff --git a/packages/subprocess/subprocess-local/src/index.ts b/packages/subprocess/subprocess-local/src/index.ts index 4a740a1e3b..e441944225 100644 --- a/packages/subprocess/subprocess-local/src/index.ts +++ b/packages/subprocess/subprocess-local/src/index.ts @@ -77,6 +77,11 @@ export class LocalSubprocessService extends SubprocessService { signal?.throwIfAborted() const environment = childEnv(env) const absolute = isAbsolute(command) + if (!absolute && (command.includes('/') || (process.platform === 'win32' && command.includes('\\')))) { + throw new Error( + `subprocess-local: command ${JSON.stringify(command)} is a relative path; use an absolute path or a bare PATH name`, + ) + } const candidates = absolute ? [command] : this.executableCandidates(command, environment) for (const candidate of candidates) { signal?.throwIfAborted() diff --git a/packages/subprocess/subprocess-local/src/terminal.ts b/packages/subprocess/subprocess-local/src/terminal.ts index 53c531675f..2d3d0bb78e 100644 --- a/packages/subprocess/subprocess-local/src/terminal.ts +++ b/packages/subprocess/subprocess-local/src/terminal.ts @@ -24,7 +24,14 @@ function signalName(number: number | undefined): NodeJS.Signals | null { return null } -/** A local terminal whose process-session ownership stays below the PTY backend. */ +/** + * A local terminal whose process-session ownership stays below the PTY backend. + * The seam's terminate() promise — no write, inspection, or signal in flight + * after settlement — holds here without operation tracking only because every + * handle call completes synchronously under the hood (node-pty write, ps-based + * inspection). A first genuinely asynchronous step in any handle call must add + * the tracking a remote provider needs. + */ export class LocalTerminalHandle implements SubprocessTerminalHandle { readonly pid: number readonly output = new PassThrough() diff --git a/packages/subprocess/subprocess-local/tests/local.spec.ts b/packages/subprocess/subprocess-local/tests/local.spec.ts index a0455edef0..347b9c9330 100644 --- a/packages/subprocess/subprocess-local/tests/local.spec.ts +++ b/packages/subprocess/subprocess-local/tests/local.spec.ts @@ -32,6 +32,10 @@ describe('LocalSubprocessService', () => { PATH: relative(process.cwd(), dirname(process.execPath)) || '.', })).toBe(process.execPath) await expect(ctx.subprocess.resolveExecutable('')).rejects.toThrow('must be non-empty') + await expect(ctx.subprocess.resolveExecutable('./bin/tsserver')) + .rejects.toThrow('is a relative path') + await expect(ctx.subprocess.resolveExecutable('node_modules/.bin/server')) + .rejects.toThrow('is a relative path') await expect(ctx.subprocess.resolveExecutable('dsh-command-that-does-not-exist', { PATH: '' })) .rejects.toThrow('was not found on PATH') await expect(ctx.subprocess.resolveExecutable('/dsh-absolute-command-that-does-not-exist')) @@ -61,6 +65,8 @@ describe('LocalSubprocessService', () => { .toEqual(['/explicit/tool.EXE']) expect(candidates('tool.exe', {})).toEqual([resolve(process.cwd(), 'tool.exe')]) expect(candidates('tool', { PATH: '/bin' })).toHaveLength(4) + await expect(ctx.subprocess.resolveExecutable(String.raw`bin\server.exe`)) + .rejects.toThrow('is a relative path') } finally { platform.mockRestore() await fiber.dispose() diff --git a/packages/subprocess/subprocess/src/index.ts b/packages/subprocess/subprocess/src/index.ts index 14866c9784..6b9904078f 100644 --- a/packages/subprocess/subprocess/src/index.ts +++ b/packages/subprocess/subprocess/src/index.ts @@ -107,7 +107,9 @@ export abstract class SubprocessService extends Service { /** * Resolve one configured executable in this provider's execution world. * Absolute paths are verified; bare names use the provider's scrubbed PATH - * plus explicit environment overrides. + * plus explicit environment overrides. Relative paths containing separators + * are rejected: no current consumer defines which directory they would + * resolve against, so providers fail loud instead of guessing. * @param command - absolute executable path or bare PATH name. * @param env - explicit environment entries used for lookup. * @param signal - aborts remote or local lookup. diff --git a/packages/subprocess/subprocess/src/types.ts b/packages/subprocess/subprocess/src/types.ts index 9801bf2f49..2aa2de0c91 100644 --- a/packages/subprocess/subprocess/src/types.ts +++ b/packages/subprocess/subprocess/src/types.ts @@ -193,7 +193,11 @@ export interface SubprocessHandle { waitForExit(signal?: AbortSignal): Promise } -/** Signals supported by the terminal-process primitive. */ +/** + * Signals supported by the terminal-process primitive. Kept member-identical + * to `PtySignal` in `@deepseek-ai/dsh-pty` without a cross-seam dependency; + * change both together. + */ export type SubprocessTerminalSignal = 'SIGINT' | 'SIGTERM' | 'SIGKILL' | 'SIGTSTP' | 'SIGHUP' /** A fully specified terminal-process spawn. */ From 18ab9f6db201468d6cc68de3a7f4a6176e97b93c Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Fri, 7 Aug 2026 20:34:31 +0800 Subject: [PATCH 058/108] fix(rebase): migrate the replayed stack onto current master APIs The linear replay carried each commit's own lineage, so this checkpoint restores the master-owned surfaces the conflicted regions clobbered and migrates branch-owned code to master's post-rebase APIs: - rebuild subprocess-local spawn.ts on master's tree-exit-observer machinery, keeping the branch's win32 childEnv key semantics and the Linux zombie-quiescence probe; the zombie test reaps its survivor directly since a confirmed-absent verdict is a permanent no-more-signals boundary - migrate pty-local test stubs to the Inbox-model Agent interface, Session.create, runnerFailureRules, and the new turn/start payload - implement the seam's resolveExecutable/spawnTerminal abstracts in the new pwsh-local and tool-fs-search test fakes - restore code-runtime, atomic-write, pwsh-local, and app-boot to master's exact content (the net-zero code-runtime churn is pruned from this history) and drop rename-detection graft debris - re-apply the PR's architecture rows and execution-world paragraph, re-record bilingual pairings, regenerate catalogs, and reconcile the lockfile --- ...0-canonical-tool-output-contract.i18n.yaml | 2 +- .../feature/2026-06-15-code-mode.i18n.yaml | 4 +- .../feature/2026-06-15-code-mode.md | 16 +- .../feature/2026-06-15-code-mode.zh.md | 42 +- docs/architecture.md | 6 +- docs/architecture.zh.md | 6 +- docs/capability-seams.md | 67 ++- docs/cordis-catalog/services.md | 58 ++- .../core-data-structures/filesystem.i18n.yaml | 2 +- docs/core-data-structures/lsp.i18n.yaml | 2 +- .../core-data-structures/subprocess.i18n.yaml | 4 +- docs/core-data-structures/subprocess.zh.md | 30 +- packages/README.i18n.yaml | 4 +- .../bash/pwsh-local/tests/executor.spec.ts | 2 + packages/bash/pwsh-local/tsconfig.json | 10 +- .../code-runtime-worker/README.md | 2 +- .../code-runtime-worker/tests/runtime.spec.ts | 21 - .../fs/tool-fs-search/tests/tools.spec.ts | 2 + packages/lsp/lsp-local/README.md | 2 +- packages/pty/pty-local/README.md | 10 +- packages/pty/pty-local/README.zh.md | 10 +- packages/pty/pty-local/tests/index.spec.ts | 38 +- .../subprocess-local/README.i18n.yaml | 4 +- .../subprocess/subprocess-local/README.md | 8 +- .../subprocess/subprocess-local/README.zh.md | 6 +- .../subprocess/subprocess-local/src/spawn.ts | 75 +++- .../subprocess-local/tests/spawn.spec.ts | 8 +- packages/typert/README.i18n.yaml | 6 +- packages/ui/app-boot/tsdown.config.ts | 8 +- packages/util/atomic-write/README.i18n.yaml | 6 +- .../src/runner-source.generated.ts | 6 - packages/util/atomic-write/src/runner.ts | 401 ------------------ pnpm-lock.yaml | 25 +- scripts/gen-doc-graphs.ts | 276 ++++++++---- vitest.config.ts | 86 +++- 35 files changed, 582 insertions(+), 673 deletions(-) delete mode 100644 packages/util/atomic-write/src/runner-source.generated.ts delete mode 100644 packages/util/atomic-write/src/runner.ts diff --git a/.agents/notes/implemented/architecture/2026-07-20-canonical-tool-output-contract.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-20-canonical-tool-output-contract.i18n.yaml index b8db543510..680d2dd94a 100644 --- a/.agents/notes/implemented/architecture/2026-07-20-canonical-tool-output-contract.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-20-canonical-tool-output-contract.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-20-canonical-tool-output-contract.md 2026-07-20-canonical-tool-output-contract.md: 2429e1c141ad8c8ee932c6d654a6ba74bc4f7618 -2026-07-20-canonical-tool-output-contract.zh.md: fd1e55eab38e5929a2dc104073b49f9de7a493e2 +2026-07-20-canonical-tool-output-contract.zh.md: 26653ea769b8a642b70c4d3dd2f5ab907a70bd85 diff --git a/.agents/notes/implemented/feature/2026-06-15-code-mode.i18n.yaml b/.agents/notes/implemented/feature/2026-06-15-code-mode.i18n.yaml index 8773a797e9..bc05e497c5 100644 --- a/.agents/notes/implemented/feature/2026-06-15-code-mode.i18n.yaml +++ b/.agents/notes/implemented/feature/2026-06-15-code-mode.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-06-15-code-mode.md -2026-06-15-code-mode.md: b6a24ecd9700e32912b8112b59cbd8b6ab131eb5 -2026-06-15-code-mode.zh.md: 4d0a4cf8fa31cf9d9954e5bd95f823dfc0668444 +2026-06-15-code-mode.md: 99bbed3edab32512f88ece9694d6519a1f89c2dd +2026-06-15-code-mode.zh.md: ca1bbe9ed3e412186763d1ed4fca9ed06669d4c3 diff --git a/.agents/notes/implemented/feature/2026-06-15-code-mode.md b/.agents/notes/implemented/feature/2026-06-15-code-mode.md index b6a24ecd97..99bbed3eda 100644 --- a/.agents/notes/implemented/feature/2026-06-15-code-mode.md +++ b/.agents/notes/implemented/feature/2026-06-15-code-mode.md @@ -6,7 +6,7 @@ English | [中文](2026-06-15-code-mode.zh.md) ## Problem -In the registry's native presentation, the agent loop advertises every visible capability as a JSON-schema function definition. `ToolRegistry` contributes its schemas to the system-prompt assembly, the assembly's `tools` land on the wire (and in the logged request header), the model invokes one `tool-call` block per step, and the loop dispatches each call through `ctx.tools.execute()` **sequentially** (parallel tool execution is an explicit open TODO in `dsh-tools` and [docs/architecture.md](../../../../docs/architecture.md)), with **every** intermediate `tool-result` re-entering the model's context on the next request. +In the registry's native presentation, the agent loop advertises every visible capability as a JSON-schema function definition. `ToolRegistry` contributes its schemas to the system-prompt assembly, the assembly's `tools` land on the wire (and in the logged request header), the model invokes one `tool-call` block per step, and at the time of this note the loop dispatched each call through `ctx.tools.execute()` **sequentially** (parallel tool execution was an open TODO then; bounded parallel dispatch has since shipped — the [parallel tool-call note](2026-07-10-parallel-tool-call-execution.md), the rolling pool in [docs/architecture.md](../../../../docs/architecture.md)) — with **every** intermediate `tool-result` re-entering the model's context on the next request. For multi-step tool work this is token-heavy and serial. The model cannot compose tools — loop over a result set, branch on an intermediate value, fan out, post-process — without a full model round-trip per call, and each round-trip drags the entire intermediate result back into context whether the model needs it or not. @@ -32,7 +32,7 @@ This note owns Code Mode's presentation, composition, isolation, and settlement **Interaction with `toolOrder`, stated up front:** a configured `systemPrompt.toolOrder` naming native capabilities rejects every assembly under `mode: 'code'`, because those names are outside that mode's wire-validation universe. This is correct behavior, not a bug: a deployment using Code Mode updates its order config or drops it. -**SDK prompt section.** In `'code'` and `'both'`, the lazy `tools:sdk` section in the tool-guidance order band renders TypeScript declarations plus fixed usage instructions for the scope's visible capabilities. It shares lookup and execution visibility, excludes `run_code`, and sorts tools lexicographically for byte-stable output. +**SDK prompt section.** In `'code'` and `'both'`, the lazy `tools:sdk` section in the tool-guidance order band renders the loaded runtime's language declarations plus fixed usage instructions for the scope's visible capabilities (TypeScript by default; the [language-dispatch note](2026-07-31-code-mode-language-dispatch.md) added Python and the `ctx.codeRuntime.language` renderer table). It shares lookup and execution visibility, excludes `run_code`, and sorts tools lexicographically for byte-stable output. **Assembly ownership.** `run_code` and `tools:sdk` enter the trusted `system-prompt/assemble` waterfall as normal assembly inputs. A scoped `tools:sdk` section may shadow the global default before dispatch, and a listener may remove or replace either contribution. The waterfall's returned assembly is final, so whoever changes these inputs owns preserving a viable Code Mode protocol when the deployment expects Code Mode to remain usable; no restoration pass overrides deliberate composition. @@ -48,7 +48,7 @@ Under `'code'` and `'both'` the registry owns `run_code` as a reserved presentat **Sub-call contexts are deferred through the parent.** Injecting inside `run_code` would break parent call/result adjacency, so `ToolRunContext.deferContext()` collects every sub-result `additionalContexts` entry in dispatch order. The registry carries that array even when the program later throws, and the loop appends each entry only after the outer result and every sibling result in the step. An outer post-execute block discards tool-deferred entries and exposes only contexts explicitly attached by the blocking decision. -**Concurrency is serialized.** Each run owns a dispatch queue, so even `Promise.all` executes tool calls in submission order. Settlement abandons queued calls that have not started. Parallelism requires per-tool concurrency-safety metadata. +**Concurrency is bounded, not serialized.** Each run owns a dispatch queue that starts calls strictly in submission order and classifies each one through `registry.executionMode`, the same fail-closed `isConcurrencySafe` contract the native loop uses. Consecutive parallel-classified calls overlap up to `maxParallelSubCalls` (default 10; `1` restores serial dispatch); an exclusive call drains the pool and runs alone. Settlement abandons queued calls that have not started. This note shipped the serialized placeholder; the [live-parallel Agent Note](2026-07-26-code-mode-live-parallel-dispatch.md) owns the scheduler that replaced it. **Presentation.** `run_code`'s render intent is decided here per the [render-intent Agent Note](../architecture/2026-07-02-tool-render-intent-union.md): `presentCall` creates a `generic` card with `kind: 'execute'`, the program text as its title, and the same program text as `rawInput`; `run_code` intentionally declares no `presentResult`, so the TUI and host/client runtime (Web) complete that card through their generic raw-content fallback using the final durable `tool/result.content`, including captured logs plus the returned value, failure, or post-policy spill preview. This is not a `terminal` card: that card's semantics are "a shell command in a working directory", which a program is not. See the [result-card completeness note](../../archived/bug-fix/2026-07-20-code-mode-result-card-completeness.md). @@ -64,7 +64,7 @@ Each sub-dispatch appends a log-only `tool/code-dispatch-start` event at pool en - `CodeBindingNamespace = { global: string; functions: Record Promise>; errorClass?: { name: string; memberNameProperty: string } }` — the runtime exposes each namespace as a global object of async functions inside the program; the optional descriptor asks the runtime to inject a real program-visible rejection class without teaching the seam consumer-specific names. `CodeJsonValue` is this dependency-light seam's structural lossless-JSON type, so binding arguments and resolutions cross the implementation's serialization boundary whole. - `CodeRunResult = { value?: CodeJsonValue; logs: string[]; error?: CodeRunFailure }` — program execution outcomes resolve as the `error` field. `run()` may reject only for caller/seam misuse (for example a duplicate binding namespace); consumers still contain a non-conforming backend rejection at their own error boundary. - `CodeRunFailure = { kind: 'exception' | 'timeout' | 'abort' | 'worker-exit' | 'invalid-output' | 'output-limit'; message: string }` — orthogonal outcomes reported independently per [defensive patterns](../../../../docs/defensive-patterns.md); a timed-out run is not an exception, an abort is not a timeout, a lossy completion is not an overflow, and a substrate exit is none of them. -- Two readonly backend descriptors, informational not gating: `language` (what the program must be written in — `'typescript'` for the shipped backend; a Python backend would say so, and pair with its own SDK generator on the presentation side) and `isolation` (`'worker-thread'` for the shipped backend; `'process'`, `'container'`, … for future ones). `dsh-tools` requires `language === 'typescript'` in the MVP — its codegen emits TS — and fails the assembly loudly otherwise, the same misconfiguration idiom as `toolOrder` violations (as when `mode` is non-native with no `ctx.codeRuntime` loaded at all). +- Two readonly backend descriptors, informational not gating: `language` (what the program must be written in — `'typescript'` for the first backend; a Python backend says `'python'` and pairs with its own SDK generator on the presentation side) and `isolation` (`'worker-thread'` for the shipped backend; `'process'`, `'container'`, … for future ones). `dsh-tools` accepts any `language` with a registered SDK renderer and `run_code` flavor (TypeScript and Python ship; see the [language-dispatch note](2026-07-31-code-mode-language-dispatch.md)) and fails the assembly loudly otherwise, the same misconfiguration idiom as `toolOrder` violations (as when `mode` is non-native with no `ctx.codeRuntime` loaded at all). Requests contain every runtime input; implementations own validated timeout and cap defaults. The registry looks up the optional runtime only when Code Mode is assembled, so native mode does not depend on one. Missing or language-incompatible runtimes fail loudly. Alternate substrates or languages can replace the implementation behind the same seam, paired with the appropriate SDK generator. @@ -85,11 +85,11 @@ The worker runtime provides containment, not a security boundary: model code can ### What the model sees -The SDK instructs the model to write an async erasable-TypeScript body, call tools through `await tools.name(args)`, catch rejected tool calls when needed, and return or log only the output that should re-enter context. Calls remain sequential even under `Promise.all`. The declaration prefix can be as large as native schemas, especially in `'both'`, but remains stable for provider caching. +The SDK instructs the model to write an async body in the loaded runtime's language (an erasable-TypeScript body by default; a Python `async` body under a Python runtime — see the [language-dispatch note](2026-07-31-code-mode-language-dispatch.md)), call tools through `await tools.name(args)`, catch rejected tool calls when needed, and return or log only the output that should re-enter context. Both flavors state the same contract in their own primitive: independent read-only calls MAY overlap under `Promise.all` (TypeScript) or `asyncio.gather` (Python), mutating calls run alone in submission order, and dependent work sequences with `await`. The declaration prefix can be as large as native schemas, especially in `'both'`, but remains stable for provider caching. ## Consequences -Deployments switching to `'code'` must update any native-only `toolOrder`. Assembly listeners own the integrity of any rewritten protocol surface. Sub-dispatch remains serialized, while per-call contexts retain their source, envelope, and metadata through the outer result. +Deployments switching to `'code'` must update any native-only `toolOrder`. Assembly listeners own the integrity of any rewritten protocol surface. Sub-dispatch starts in submission order under a bounded overlap pool, while per-call contexts retain their source, envelope, and metadata through the outer result. ## Testing @@ -106,7 +106,7 @@ Deployments switching to `'code'` must update any native-only `toolOrder`. Assem **Result elision / summarization over native tool-calling.** Addresses only the context-bloat half of the problem: trimming old `tool-result`s is cheap to add as a logged surface replacement under reconstructable requests, but still pays one model round-trip per call and cannot express loops, branches, or joins. Complementary, not competing; it can layer under Code Mode for residual native calls. -**Parallel native dispatch in the loop.** The other answer to round-trip cost; still valid future work (the open TODO), still blocked on concurrency-safety metadata, and still no composition — it parallelizes calls the model already decided on in one step. Code Mode's serialized-queue decision keeps the two compatible: when the metadata lands, both native parallel dispatch and per-tool binding parallelism unlock together. +**Parallel native dispatch in the loop.** The other answer to round-trip cost at decision time; it was blocked on concurrency-safety metadata and offers no composition either way — it parallelizes calls the model already decided on in one step. Code Mode's queue decision kept the two compatible, and that is how it played out: the metadata landed as `isConcurrencySafe` (the [parallel tool-call note](2026-07-10-parallel-tool-call-execution.md)), and native rolling-pool dispatch and per-tool binding parallelism unlocked on the same classifier. **Always-exclusive (Cloudflare-faithful, no mode).** Rejected for this SDK's primary consumer: a coding agent's bread-and-butter single calls (`bash`, `read`, `edit`) are already ideal as native calls, and forcing every edit through a program taxes the common case. The mode config keeps the faithful form (`'code'`) one line away without imposing it. @@ -128,6 +128,6 @@ Deployments switching to `'code'` must update any native-only `toolOrder`. Assem **Large lossless JSON values can exhaust memory.** Tool bindings snapshot lossless JSON before dispatch and return canonical JSON resolutions whole. The runtime validates both sides of the worker port and applies no per-binding byte cap; structured-clone cost and process or worker memory are the practical bounds. The combined outer-output ledger for logs, the completion value, and a failure diagnostic is the only byte-capped boundary. -**Serialized-only sub-dispatch.** `Promise.all` gains no wall-clock parallelism yet, only fewer round-trips; models may over-expect. The instructions state it; lifting it is tied to the same concurrency-safety metadata the native parallel-dispatch TODO needs. +**Sub-dispatch overlap is bounded by tool safety claims, not by the caller.** A program's `Promise.all` or `asyncio.gather` buys wall-clock parallelism only across calls the tool itself classifies concurrency-safe; a run of exclusive calls still costs its round-trips in sequence, and models may over-expect. Both flavors' SDK instructions state the real contract. This note shipped the serialized placeholder that made the risk absolute; the [live-parallel Agent Note](2026-07-26-code-mode-live-parallel-dispatch.md) owns the scheduler and its overlap cap. **Budget metering reads the event loop, not a flag.** Busy-time polling (`eventLoopUtilization()`) is coarser than an exact CPU meter — a budget expires up to one poll interval late — and its correctness claim ("a pending dispatch cannot pause it") is load-bearing against a hostile program. Both sides are unit-tested (hot loop with a pending decoy dispatch dies at `computeMs`; idle-on-slow-binding survives to `maxWallMs`), and the poll interval is an internal constant, not config — nothing a deployment could mis-tune into a bypass. `maxWallMs` is config, and it reaches `setTimeout`, which clamps a delay above `MAX_TIMER_DELAY_MS` (2^31-1 ms) to 1 ms; a positivity check alone therefore accepts a 25-day ceiling that expires on the first tick and times out every run. The worker runtime range-checks the field at load for that reason. `computeMs` needs no upper bound because it is compared against measured utilization instead of being handed to a timer. diff --git a/.agents/notes/implemented/feature/2026-06-15-code-mode.zh.md b/.agents/notes/implemented/feature/2026-06-15-code-mode.zh.md index 4d0a4cf8fa..ca1bbe9ed3 100644 --- a/.agents/notes/implemented/feature/2026-06-15-code-mode.zh.md +++ b/.agents/notes/implemented/feature/2026-06-15-code-mode.zh.md @@ -6,7 +6,7 @@ Status: implemented ## 问题 -在注册表的原生呈现方式下,agent loop(智能体循环)将每个可见能力以 JSON Schema 函数定义的形式通告给模型。`ToolRegistry` 将其 schema 贡献给系统提示词组装,组装结果中的 `tools` 落到协议格式(wire format)上(也记录在请求头日志中),模型每步调用一个 `tool-call` 块,循环通过 `ctx.tools.execute()` **逐个**分发每次调用(并行工具执行是 `dsh-tools` 和 [docs/architecture.md](../../../../docs/architecture.md) 中明确标注的 open TODO),且**每一个**中间 `tool-result` 都会在下一次请求时重新进入模型上下文。 +在注册表的原生呈现方式下,agent loop(智能体循环)将每个可见能力以 JSON Schema 函数定义的形式通告给模型。`ToolRegistry` 将其 schema 贡献给系统提示词组装,组装结果中的 `tools` 落到协议格式(wire format)上(也记录在请求头日志中),模型每步调用一个 `tool-call` 块,而在本 note 写作时,循环通过 `ctx.tools.execute()` **逐个**分发每次调用(并行工具执行当时还是 open TODO;此后有界的并行分发已经交付——见[并行工具调用 note](2026-07-10-parallel-tool-call-execution.md),以及 [docs/architecture.md](../../../../docs/architecture.md) 中的 rolling pool)——且**每一个**中间 `tool-result` 都会在下一次请求时重新进入模型上下文。 对于多步工具操作,这种方式 token 开销大且串行。模型无法组合工具——遍历结果集、根据中间值分支、扇出、后处理——每次调用都需要一次完整的模型往返,而每次往返都会把完整的中间结果拖回上下文,不管模型是否需要。 @@ -19,7 +19,7 @@ Cloudflare 的 [Code Mode](https://blog.cloudflare.com/code-mode/) 提出了一 三项决策,各自在下方独立小节中展开: 1. **Code Mode 是 `ToolRegistry`(`dsh-tools`)的一等呈现模式**,通过经校验的 `mode` 配置选择:`'native'`(默认,贡献可见能力 schema)、`'code'`(注册表仅贡献其保留的 `run_code` 传输通道加一份生成的 SDK `.d.ts` 到系统提示词中)或 `'both'`(原生 schema 加传输通道 + SDK)。注册表在源头塑造其权威贡献;协作式提示词组装的结果仍具权威性,记录在日志中的请求头精确反映该返回的呈现。 -2. **代码执行是一个能力 seam**——`packages/code-runtime/` 包含接口包 `@deepseek-ai/dsh-code-runtime`,拥有 `ctx.codeRuntime`([能力 seam](../architecture/2026-06-13-capability-seams.md);消费方 = `dsh-tools`,core 消费 seam 的先例见 `agent-loop` → `dsh-llm`)。运行时对工具一无所知:它接收一段程序和命名的异步绑定,执行程序,报告 `{ value, logs, error? }`。语言和基底是后端属性,因此未来的 Python 或容器后端只是另一个实现包,而非重新设计。 +2. **代码执行是一个能力 seam**——`packages/code-runtime/` 包含接口包`@deepseek-ai/dsh-code-runtime`,拥有 `ctx.codeRuntime`([能力 seam](../architecture/2026-06-13-capability-seams.md);消费方 = `dsh-tools`,core 消费 seam 的先例见 `agent-loop` → `dsh-llm`)。运行时对工具一无所知:它接收一段程序和命名的异步绑定,执行程序,报告 `{ value, logs, error? }`。语言和基底是后端属性,因此未来的 Python 或容器后端只是另一个实现包,而非重新设计。 3. **交付的实现是 `@deepseek-ai/dsh-code-runtime-worker`**:每次运行 spawn 一个全新的 Node worker 线程,对模型的 TypeScript 进行 type-strip 后执行,绑定通过消息端口桥接,环境为空,堆/输出/时间上限可配置,并支持硬终止。其信任姿态在设计上等同于 bash——无需 unsafe-acknowledgement flag——因为 harness 已经交付了 `dsh-bash-local`,后者以严格*更高*的环境权限执行模型编写的任意 shell 命令。 本说明负责定义 Code Mode 的呈现、组合、隔离与结算基础。后续的[类型化工具返回值 Agent Note](2026-07-20-code-mode-typed-tool-returns.md)负责定义生成的输出映射、规范绑定值、`ToolCallError` 和无损外层输出边界。 @@ -32,7 +32,7 @@ Cloudflare 的 [Code Mode](https://blog.cloudflare.com/code-mode/) 提出了一 **与 `toolOrder` 的交互,预先说明:** 如果配置的 `systemPrompt.toolOrder` 引用了原生能力名称,在 `mode: 'code'` 下会拒绝所有组装,因为那些名称不在该模式的协议校验范围内。这是正确行为而非 bug:使用 Code Mode 的部署需要更新其 order 配置或移除它。 -**SDK 提示词段。** 在 `'code'` 和 `'both'` 下,tool-guidance order band 中的惰性 `tools:sdk` 段为当前 scope 的可见能力渲染 TypeScript 声明加固定的使用说明。它共享查找和执行可见性,排除 `run_code`,并按字典序排列工具以获得字节稳定的输出。 +**SDK 提示词段。** 在 `'code'` 和 `'both'` 下,tool-guidance order band 中的惰性 `tools:sdk` 段为当前 scope 的可见能力渲染所加载运行时语言的声明加固定的使用说明(默认 TypeScript;[语言分发 note](2026-07-31-code-mode-language-dispatch.md) 加入了 Python 与按 `ctx.codeRuntime.language` 选择的渲染器表)。它共享查找和执行可见性,排除 `run_code`,并按字典序排列工具以获得字节稳定的输出。 **组装所有权。** `run_code` 和 `tools:sdk` 作为正常的组装输入进入受信任的 `system-prompt/assemble` waterfall。一个 scoped 的 `tools:sdk` 段可以在分发前遮蔽全局默认值,监听器也可以移除或替换任一贡献。waterfall 返回的组装结果是最终的,因此修改这些输入的人有责任在部署期望 Code Mode 可用时保持协议面的完整性;没有恢复 pass 会覆盖有意的组合。 @@ -40,7 +40,7 @@ Cloudflare 的 [Code Mode](https://blog.cloudflare.com/code-mode/) 提出了一 ### run_code 工具与分发桥 -在 `'code'` 和 `'both'` 下,注册表拥有 `run_code` 作为保留的呈现传输通道,带两个必需参数 `{ code: string; description: string }`(description 为 UI 标注该调用,沿用 bash 的先例)。它由一个正常的 `ToolDefinition` 表示以供分发,但位于可过滤的能力层之外,因此限制规则不会意外移除 Code Mode 的唯一入口。调用遍历完整的工具流水线——`tools/pre-execute` → 单调守卫 → `tools/execute` 包裹分发 → `tools/post-execute` → 由定义拥有的可选 `finalizeContent` → 不可变的 `tools/result` 通知——与原生调用完全一致;权限插件可以在程序运行前检查程序文本,最终结果观察者看到的是规范化的外层结果。其 `execute(args, exec)`: +在 `'code'` 和 `'both'` 下,注册表拥有 `run_code` 作为保留的呈现传输通道,带两个必需参数 `{ code: string; description: string }`(description 为 UI 标注该调用,沿用 bash 的先例)。它由一个正常的 `ToolDefinition` 表示以供分发,但位于可过滤的能力层之外,因此限制规则不会意外移除 Code Mode 的唯一入口。调用遍历完整的工具流水线——`tools/pre-execute` → 单调性守卫 → `tools/execute` 包裹分发 → `tools/post-execute` → 由定义拥有的可选 `finalizeContent` → 不可变的 `tools/result` 通知——与原生调用完全一致;权限插件可以在程序运行前检查程序文本,最终结果观察者看到的是规范化的外层结果。其 `execute(args, exec)`: 1. **构建绑定。** 一个 run 级别的 signal 跟随外层取消,并在 run 结算时被 abort。每个可见工具绑定都会对无损 JSON 参数创建快照,进入原生契约的分发池(调度设计由[实时并行 Agent Note](2026-07-26-code-mode-live-parallel-dispatch.md) 负责),以确定性的 call id 和外层 token 作为 `parent` 执行,通过外层 execution 延后返回的上下文,并记录 `tool/code-dispatch-start`/`tool/code-dispatch` 事件对,其中结算侧携带完整渲染后的结果内容。成功时返回工具最终的规范 JSON 值;失败则变为程序可见的 `ToolCallError`。每个子调用保留自己不可变的执行标识,并遍历完整的工具流水线。 2. **运行程序**:`ctx.codeRuntime.run({ program: args.code, bindings: [{ global: 'tools', functions }], signal: runController.signal })`。运行时接收的是 run 级别的 signal 而非仅调用方的外层 signal,因此外层 run 以任何方式结算都会同时 abort 运行时内部的工作。 @@ -48,7 +48,7 @@ Cloudflare 的 [Code Mode](https://blog.cloudflare.com/code-mode/) 提出了一 **子调用上下文通过父调用延后。** 在 `run_code` 内部注入会破坏父调用/结果的相邻性,因此 `ToolRunContext.deferContext()` 按分发顺序收集每个子结果的 `additionalContexts` 条目。即使程序后来抛出异常,注册表仍携带该数组;循环只在外层结果与步骤中所有兄弟结果之后追加每个条目。外层 post-execute 阻止会丢弃工具延后的条目,只暴露阻止 decision 显式附加的上下文。 -**并发被序列化。** 每次 run 拥有一个分发队列,因此即使 `Promise.all` 也按提交顺序执行工具调用。结算时放弃尚未开始的排队调用。并行化需要每个工具的并发安全元数据。 +**并发是有界的,而非被序列化。** 每次 run 拥有一个分发队列,严格按提交顺序启动调用,并通过 `registry.executionMode` 对每个调用分类——与原生循环所用的 fail-closed `isConcurrencySafe` 契约相同。连续的 parallel 类调用最多重叠 `maxParallelSubCalls` 个(默认 10;设为 `1` 恢复串行分发);exclusive 类调用会排空池并单独运行。结算时放弃尚未开始的排队调用。本 note 交付的是被序列化的占位实现;取代它的调度器由[实时并行 Agent Note](2026-07-26-code-mode-live-parallel-dispatch.md) 负责。 **呈现。** `run_code` 的 render intent 按[呈现意图 Agent Note](../architecture/2026-07-02-tool-render-intent-union.md)在此决定:`presentCall` 创建一个 `generic` 卡片,`kind: 'execute'`,以程序文本作为标题,并将同一程序文本作为 `rawInput`;`run_code` 有意不声明 `presentResult`,因此 TUI 和宿主/客户端运行时(Web)会通过通用原始内容回退机制,使用最终持久化的 `tool/result.content` 补全该卡片,其中包括捕获的日志,以及返回值、失败信息或 post-policy 输出落盘预览。这不是 `terminal` 卡片:该卡片的语义是「工作目录中的 shell 命令」,程序不是。参见[结果卡片完整性说明](../../archived/bug-fix/2026-07-20-code-mode-result-card-completeness.md)。 @@ -61,35 +61,35 @@ Cloudflare 的 [Code Mode](https://blog.cloudflare.com/code-mode/) 提出了一 `packages/code-runtime/code-runtime/`——`@deepseek-ai/dsh-code-runtime`,仅依赖 `cordis`。一个抽象的 `CodeRuntime extends Service`(`super(ctx, 'codeRuntime')`)加上词汇: - `CodeRunRequest = { program: string; bindings: CodeBindingNamespace[]; signal?: AbortSignal }` -- `CodeBindingNamespace = { global: string; functions: Record Promise>; errorClass?: { name: string; memberNameProperty: string } }`——运行时将每个命名空间作为程序内部的全局异步函数对象暴露;可选描述符要求运行时注入真正的、程序可见的 reject 类,而无需让 seam 获知消费方专用名称。`CodeJsonValue` 是这个低依赖 seam 的结构化无损 JSON 类型,因此绑定参数与解析值可以完整跨越实现的序列化边界。 +- `CodeBindingNamespace = { global: string; functions: Record Promise>; errorClass?: { name: string; memberNameProperty: string } }`——运行时将每个命名空间作为程序内部的全局异步函数对象暴露;可选描述符要求运行时注入真正的、程序可见的 reject 类,而无需让 seam 获知消费方专用名称。`CodeJsonValue` 是这个低依赖 seam 的结构化无损 JSON 类型,因此绑定参数与返回值可以完整跨越实现的序列化边界。 - `CodeRunResult = { value?: CodeJsonValue; logs: string[]; error?: CodeRunFailure }`——程序执行失败时,执行 promise 仍会 fulfill,并通过 `error` 字段返回失败结果。只有调用方/seam 误用(例如重复的绑定命名空间)时,`run()` 才会 reject;消费方仍在自己的错误边界处理不合规后端的拒绝。 - `CodeRunFailure = { kind: 'exception' | 'timeout' | 'abort' | 'worker-exit' | 'invalid-output' | 'output-limit'; message: string }`——按[防御性模式](../../../../docs/defensive-patterns.md)独立报告的正交结果;超时的 run 不是异常,abort 不是超时,有损完成值不是溢出,基底退出也与上述情况相互独立。 -- 两个只读的后端描述符,仅供信息参考而非门禁判定:`language`(程序必须使用的语言——交付的后端为 `'typescript'`;Python 后端会声明自己,并在呈现侧配对自己的 SDK 生成器)和 `isolation`(交付的后端为 `'worker-thread'`;未来可为 `'process'`、`'container'` 等)。`dsh-tools` 在 MVP 中要求 `language === 'typescript'`——其代码生成输出 TS——否则组装会大声失败,与 `toolOrder` 违规时的配置错误惯用法相同(如 `mode` 为非 native 但根本没有加载 `ctx.codeRuntime`)。 +- 两个只读的后端描述符,仅供信息参考而非门禁判定:`language`(程序必须使用的语言——首个后端为 `'typescript'`;Python 后端声明 `'python'`,并在呈现侧配对自己的 SDK 生成器)和 `isolation`(交付的后端为 `'worker-thread'`;未来可为 `'process'`、`'container'` 等)。`dsh-tools` 接受任何注册了 SDK 渲染器与 `run_code` flavor 的 `language`(TypeScript 与 Python 已交付;见[语言分发 note](2026-07-31-code-mode-language-dispatch.md)),否则组装会大声失败,与 `toolOrder` 违规时的配置错误惯用法相同(如 `mode` 为非 native 但根本没有加载 `ctx.codeRuntime`)。 请求包含所有运行时输入;实现方拥有经校验的超时和上限默认值。注册表仅在组装 Code Mode 时查找可选的运行时,因此 native 模式不依赖它。缺失或语言不兼容的运行时会大声失败。替代基底或语言可以在同一 seam 背后替换实现,配对相应的 SDK 生成器。 ### worker-thread 运行时 -`@deepseek-ai/dsh-code-runtime-worker`,`packages/code-runtime/` 组的第二个包(package)。每次 `run()`: +`@deepseek-ai/dsh-code-runtime-worker`,`packages/code-runtime/` 组的第二个包。每次 `run()`: -1. **宿主侧 type-strip**,使用 Node 内置的 `stripTypeScriptTypes`(`node:module`;在本仓库的整个引擎范围 `^22.19.0 || >=24.0.0` 内可用,且保持位置不变,因此运行时错误行号与模型源码一致)。仅剥离模式拒绝不可擦除的语法(`enum`、namespaces)——该拒绝以 `error.kind: 'exception'` 加 Node 的消息返回,SDK 说明写明「仅限可擦除 TypeScript」,模型像处理其他程序错误一样自我修正。语法级失败不会 spawn worker。 +1. **宿主侧 type-strip**,使用 Node 内置的 `stripTypeScriptTypes`(`node:module`;在本仓库的整个引擎范围 `^22.19.0 || >=24.0.0` 内可用,且会保留源码位置,因此运行时错误行号与模型源码一致)。仅剥离模式拒绝不可擦除的语法(`enum`、namespaces)——该拒绝以 `error.kind: 'exception'` 加 Node 的消息返回,SDK 说明写明「仅限可擦除 TypeScript」,模型像处理其他程序错误一样自我修正。语法级失败不会 spawn worker。 2. **每次 run spawn 一个全新 `Worker`**,来自包自身的 bootstrap 模块:`env: {}`(真正为空——比 spawn 命令的 scrubbed-env 规则更严格),`resourceLimits` 来自配置,`stdout`/`stderr` 捕获到 `logs` 而非继承。不做池化,不跨 run 保留状态:程序的世界随 worker 消亡,这使得 run 仅从日志即可重建,状态泄漏不可表达。 3. **在 bootstrap 中执行**:剥离后的程序成为一个 `AsyncFunction` 的函数体,其参数是绑定全局变量、消费方声明的 reject 类和一个捕获式 `console` shim,因此顶层 `await` 和 `return` 可用。Code Mode 声明 `ToolCallError`,成员属性为 `toolName`;运行时无需硬编码工具即可实体化真正的构造函数。无损 JSON 完成值会精确跨越边界;`undefined` 仍表示缺席,有损值产生 `invalid-output`,过大的外层结果产生 `output-limit`,而不会退化为检查格式化后的字符串替代品。 4. **通过消息端口桥接绑定**:worker 中的每个绑定函数发送 `{ id, global, name, args }` 并等待回复;宿主根据请求的绑定校验名称、调用、并回复 `{ id, ok, value }` 或 `{ id, ok: false, message }`(宿主侧绑定拒绝变为程序侧 rejection)。worker 侧的命名空间对象通过 `defineProperty` 构建为 null-prototype,因此名为 `__proto__`、`constructor` 或 `toString` 的绑定是普通自有属性,而非原型链碰撞。未知名称、重复 id 和结算后消息被拒绝或忽略——端口协议假设对端是恶意的,因为对端运行的是模型代码。 5. **强制独立预算。** `computeMs` 计量 worker 忙碌时间,允许慢速的 awaited 工具而不放过热循环。`maxWallMs` 约束总经过时间,包括未解析的等待。`maxOutputBytes` 只约束序列化后的外层日志、完成值或诊断的组合;中间绑定值没有字节数上限。到期、取消和完成都终止 worker,堆退出或外层溢出会作为显式失败报告。 -6. **dispose 至完全停稳**:服务自身的 dispose(资源释放)终止进行中的 worker 并*等待*其退出后再 resolve,遵循[防御性模式](../../../../docs/defensive-patterns.md)。 +6. **dispose(资源释放)至完全停稳**:服务自身的 dispose 终止进行中的 worker 并*等待*其退出后再 resolve,遵循[防御性模式](../../../../docs/defensive-patterns.md)。 ### 信任姿态 -worker 运行时提供的是隔离,而非安全边界:模型代码可以访问 Node API,权限与 bash 工具相当。`worker.terminate()` 停止线程但不停止它 spawn 的 OS 进程。Code Mode 使用与 bash 相同的 `tools/pre-execute` 策略门禁,并额外提供空环境、堆限制、独立 isolate 和对程序本身的硬终止。需要硬多租户边界的部署需要为代码和 bash 都使用容器级后端;运行时的 isolation 描述符让它们能区分该后端。 +worker 运行时只能约束程序的运行,而不构成安全边界:模型代码可以访问 Node API,权限与 bash 工具相当。`worker.terminate()` 停止线程但不停止它 spawn 的 OS 进程。Code Mode 使用与 bash 相同的 `tools/pre-execute` 策略门禁,并额外提供空环境、堆限制、独立 isolate 和对程序本身的硬终止。需要硬多租户边界的部署需要为代码和 bash 都使用容器级后端;运行时的 isolation 描述符让它们能区分该后端。 ### 模型看到的内容 -SDK 指示模型编写一个异步的可擦除 TypeScript 函数体,通过 `await tools.name(args)` 调用工具,在需要时捕获被拒绝的工具调用,并仅 return 或 log 应重新进入上下文的输出。即使在 `Promise.all` 下调用仍保持顺序。声明前缀可能与原生 schema 一样大,尤其在 `'both'` 下,但对提供方缓存保持稳定。 +SDK 指示模型编写一个所加载运行时语言的异步函数体(默认可擦除 TypeScript;Python 运行时下为 Python `async` 函数体——见[语言分发 note](2026-07-31-code-mode-language-dispatch.md)),通过 `await tools.name(args)` 调用工具,在需要时捕获被拒绝的工具调用,并仅 return 或 log 应重新进入上下文的输出。两种 flavor 用各自的原语陈述同一契约:相互独立的只读调用可以在 `Promise.all`(TypeScript)或 `asyncio.gather`(Python)下重叠,有副作用的调用按提交顺序单独运行,有依赖的工作用 `await` 排序。声明前缀可能与原生 schema 一样大,尤其在 `'both'` 下,但对提供方缓存保持稳定。 ## 后果 -切换到 `'code'` 的部署必须更新任何仅限 native 的 `toolOrder`。组装监听器有责任维护任何被重写的协议面的完整性。子分发保持序列化,而每次调用的上下文会通过外层结果保留其 source、信封与元数据。 +切换到 `'code'` 的部署必须更新任何仅限 native 的 `toolOrder`。组装监听器有责任维护任何被重写的协议面的完整性。子分发在有界的重叠池下按提交顺序启动,而每次调用的上下文会通过外层结果保留其 source、信封与元数据。 ## 测试 @@ -104,9 +104,9 @@ SDK 指示模型编写一个异步的可擦除 TypeScript 函数体,通过 `aw **`node:vm` 作为参考运行时,加固推迟。** 否决:`node:vm` 不是隔离(原型链逃逸可达宿主 realm)且无法中断热循环。worker 线程提供独立 isolate、空环境、`resourceLimits` 和可靠的 `terminate()`,信任等级等同于 bash,因此参考实现和生产实现是同一个包,无需 unsafe-acknowledgement 仪式。 -**在原生工具调用上做结果省略/摘要。** 仅解决问题的上下文膨胀一半:裁剪旧 `tool-result` 作为可重建请求下的日志化表面替换成本低,但仍需每次调用一次模型往返,且无法表达循环、分支或汇合。互补而非竞争;它可以在 Code Mode 下为残余的原生调用分层。 +**在原生工具调用上做结果省略/摘要。** 仅解决问题中上下文膨胀这一半:裁剪旧 `tool-result` 作为可重建请求下的日志化表面替换成本低,但仍需每次调用一次模型往返,且无法表达循环、分支或汇合。互补而非竞争;它可以在 Code Mode 下为残余的原生调用分层。 -**循环中的并行原生分发。** 往返成本的另一个答案;仍是有效的未来工作(open TODO),仍被并发安全元数据阻塞,且仍无组合能力——它并行化的是模型在一步中已经决定的调用。Code Mode 的序列化队列决策保持两者兼容:当元数据就绪时,原生并行分发和每工具绑定并行化一起解锁。 +**循环中的并行原生分发。** 决策当时对往返成本的另一个答案;它被并发安全元数据阻塞,且无论如何都不提供组合能力——它并行化的是模型在一步中已经决定的调用。Code Mode 的队列决策保持了两者兼容,后续也正是这样落地的:元数据以 `isConcurrencySafe` 的形式就绪(见[并行工具调用 note](2026-07-10-parallel-tool-call-execution.md)),原生 rolling-pool 分发与每工具绑定并行化基于同一个分类器一起解锁。 **始终排他(忠于 Cloudflare,无模式)。** 否决,因为本 SDK 的主要消费方是编码 agent:其日常的单次调用(`bash`、`read`、`edit`)作为原生调用已经是最优的,强制每次编辑都通过程序会给常见场景增加负担。mode 配置让忠实形式(`'code'`)只需一行配置即可启用,而不强加于人。 @@ -114,20 +114,20 @@ SDK 指示模型编写一个异步的可擦除 TypeScript 函数体,通过 `aw **SDK 中的清洁化标识符别名**(`my-tool` → `my_tool`,Cloudflare 的做法)。否决:`declare const` 上的带引号键使每个名称可达,零别名碰撞逻辑;模型能正常处理 `tools["my-tool"](…)`。 -**REPL 风格的持久内核**(状态跨 `run_code` 调用存活)。在 MVP 中否决:跨调用状态对会话日志不可见,破坏了「每个请求是日志的纯函数」这一可重建性保证;每次 run 全新保持了这一点。内核风格的后端在未来仍可通过同一 seam 表达,配合自己的日志方案。 +**REPL 风格的持久内核**(状态跨 `run_code` 调用存活)。在 MVP 中否决:跨调用状态对会话日志不可见,破坏了「每个请求是日志的纯函数」这一可重建性保证;每次 run 均使用全新实例则维持了这一保证。内核风格的后端在未来仍可通过同一 seam 表达,配合自己的日志方案。 ## 风险 -**Worker 不是硬安全边界。** 有意为之且已文档化(§信任姿态):姿态等同于既有的 bash 工具,隔离程度超过它,门禁使用相同的 seam。需要更强隔离的部署需要未来的 `isolation: 'container'` 后端——作为 seam 设计的扩展点跟踪,而非本设计的 TODO。 +**Worker 不是硬安全边界。** 有意为之且已文档化(§信任姿态):姿态等同于既有的 bash 工具,约束能力强于它,门禁使用相同的 seam。需要更强隔离的部署需要未来的 `isolation: 'container'` 后端——作为 seam 设计的扩展点跟踪,而非本设计的 TODO。 **`stripTypeScriptTypes` 标记为 experimental。** 它与 Node 自身原生 `.ts` 执行背后的引擎(amaro/swc)相同,在本仓库的整个引擎范围内作为 API 暴露。缓解措施:运行时的单元测试套件固定了所依赖的行为(位置保持、可擦除限制的拒绝消息形状宽松匹配),调用位于一个私有函数之后,且 `amaro`/`sucrase` 是 API 变化时的直接替代品。仅可擦除子集是面向模型的契约线,错误路径是一个可工作的反馈循环,而非死胡同。 **SDK 的提示词成本,尤其在 `'both'` 下。** `.d.ts` 可能与它补充的原生 schema 体量相当;`'both'` 携带两种表示。前缀稳定性 + 提供方缓存摊销了每会话成本;mode 是每部署的;本 Agent Note 不做无条件节省的声明。何时优先使用哪种模式的量化指导明确属于上线后学习。 -**注册表 scope 增长。** `dsh-tools` 吸收了代码生成、一个工具、一个桥和一个事件。通过包内的模块边界(`ts-types.ts`、`code-mode.ts` 与 `schema.ts`/`json-schema.ts`/`presentation.ts` 并列)和 seam 约束:所有基底相关的内容都在 `ctx.codeRuntime` 之后。 +**注册表 scope 增长。** `dsh-tools` 吸收了代码生成、一个工具、一个桥和一个事件。包内的模块边界(`ts-types.ts`、`code-mode.ts` 与 `schema.ts`/`json-schema.ts`/`presentation.ts` 并列)和 seam 共同约束了这一增长:所有基底相关的内容都在 `ctx.codeRuntime` 之后。 -**大型无损 JSON 值可能耗尽内存。** 工具绑定会在分发前对无损 JSON 创建快照,并完整返回规范 JSON 解析值。运行时会校验 worker 端口两侧,但不对单次绑定设置字节数上限;结构化克隆成本以及进程或 worker 内存构成实际边界。只有包含日志、完成值和失败诊断的组合外层输出账本受字节数上限约束。 +**大型无损 JSON 值可能耗尽内存。** 工具绑定会在分发前对无损 JSON 创建快照,并完整返回规范 JSON 返回值。运行时会校验 worker 端口两侧,但不对单次绑定设置字节数上限;结构化克隆成本以及进程或 worker 内存构成实际边界。只有包含日志、完成值和失败诊断的组合外层输出账本受字节数上限约束。 -**仅序列化的子分发。** `Promise.all` 尚未获得挂钟并行性,仅减少往返次数;模型可能过度期望。说明中已声明;解除此限制与原生并行分发 TODO 所需的并发安全元数据绑定。 +**子分发的重叠由工具自身的安全声明限定,而非由调用方决定。** 程序里的 `Promise.all` 或 `asyncio.gather` 只在工具自己分类为并发安全的调用之间换来挂钟并行性;一串 exclusive 调用仍要按顺序付出各自的往返开销,模型可能过度期望。两种 flavor 的 SDK 说明都陈述了真实契约。本 note 交付的是使该风险绝对化的序列化占位实现;调度器及其重叠上限由[实时并行 Agent Note](2026-07-26-code-mode-live-parallel-dispatch.md) 负责。 -**预算计量读取事件循环,而非 flag。** 忙碌时间轮询(`eventLoopUtilization()`)比精确 CPU 计量更粗糙——预算到期最多延迟一个轮询间隔——且其正确性声明(「pending 的分发不能暂停它」)对恶意程序是承重的。两侧都有单元测试(带 pending 诱饵分发的热循环在 `computeMs` 处死亡;在慢绑定上空闲的程序存活到 `maxWallMs`),轮询间隔是内部常量而非配置——部署无法将其误调为绕过手段。`maxWallMs` 是配置项,且会传入 `setTimeout`,后者会把超过 `MAX_TIMER_DELAY_MS`(2^31-1 ms)的延迟夹到 1 ms;因此仅有正数校验会放行一个 25 天的上限,它在第一个 tick 就到期,使每次运行都超时。worker 运行时正因如此在加载时对该字段做范围校验。`computeMs` 不需要上界,因为它对照的是实测占用率,而不是交给定时器。 +**预算计量读取事件循环,而非 flag。** 忙碌时间轮询(`eventLoopUtilization()`)比精确 CPU 计量更粗糙——预算到期最多延迟一个轮询间隔——且其正确性声明(「pending 的分发不能暂停它」)是抵御恶意程序的关键。两种情况均有单元测试(带 pending 诱饵分发的热循环会在耗尽 `computeMs` 预算时终止;等待慢速绑定的空闲程序则会持续运行至 `maxWallMs`),轮询间隔是内部常量而非配置——部署无法将其误调为绕过手段。`maxWallMs` 是配置项,且会传入 `setTimeout`,后者会把超过 `MAX_TIMER_DELAY_MS`(2^31-1 ms)的延迟夹到 1 ms;因此仅有正数校验会放行一个 25 天的上限,它在第一个 tick 就到期,使每次运行都超时。worker 运行时正因如此在加载时对该字段做范围校验。`computeMs` 不需要上界,因为它对照的是实测占用率,而不是交给定时器。 diff --git a/docs/architecture.md b/docs/architecture.md index ea78faa627..e88bdac118 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -26,12 +26,12 @@ Harnesses are [Cordis](cordis-primer.md) contexts; packages contribute services, | `ctx.llm` | [`llm/`](../packages/llm/README.md) | adapter registry, streaming model calls | | `ctx.tokenMeter` | [`llm/token-meter`](../packages/llm/token-meter/README.md) | replay-aware request and surface pressure | | `ctx.bash` | [`bash/`](../packages/bash/README.md) | foreground/background command execution | -| `ctx.subprocess` | [`subprocess/`](../packages/subprocess/README.md) | managed child-process trees for bash, LSP, and ACP subagent backends | +| `ctx.subprocess` | [`subprocess/`](../packages/subprocess/README.md) | executable lookup, managed trees, terminals | | `ctx.pty` | [`pty/`](../packages/pty/README.md) | owner-scoped persistent terminal sessions | | `ctx.sandbox` | [`sandbox/`](../packages/sandbox/README.md) | same-world process confinement through argv wrapping and per-call policy | | `ctx.sandboxPolicy` | [`sandbox/`](../packages/sandbox/README.md) | shared sandbox policy home | | `ctx.codeRuntime` | [`code-runtime/`](../packages/code-runtime/README.md) | model-written program execution | -| `ctx.fs` | [`fs/`](../packages/fs/README.md) | filesystem provider primitives and policy events | +| `ctx.fs` | [`fs/`](../packages/fs/README.md) | execution-world paths, bounded IO, and policy events | | `ctx.lsp` | [`lsp/`](../packages/lsp/README.md) | semantic navigation registry | | `ctx.skills` | [`skill/`](../packages/skill/README.md) | skill provider registry, progressive disclosure | | `ctx.web` | [`web/`](../packages/web/README.md) | search/fetch provider registries | @@ -155,7 +155,7 @@ Streaming uses raw chunks and `BlockAssembler`. Each `LlmAdapter.stream()` is on ### Capability Pattern -A swappable capability usually has **interface / implementation / consumer** layers: service/events, backend, and model-facing tools/prompts. Bash is the reference; the [capability graph](capability-seams.md) maps each family. +Capabilities separate **interface / implementation / consumer** layers. Filesystem and subprocess providers define one execution world; Bash, PTY, and LSP run there without provider forks. See the [capability graph](capability-seams.md). Exceptions combine LLM interface/consumer, filesystem policy, web registries, and named skill/subagent providers. Subagents spawn fresh, fork a completed-turn prefix, use ACP children, or delegate one self-contained turn to a real product provider such as Codex ([subagent.md](core-data-structures/subagent.md)). diff --git a/docs/architecture.zh.md b/docs/architecture.zh.md index 5a46dfaf84..9314a3fbfc 100644 --- a/docs/architecture.zh.md +++ b/docs/architecture.zh.md @@ -26,12 +26,12 @@ | `ctx.llm` | [`llm/`](../packages/llm/README.md) | 适配器注册表和模型流式调用 | | `ctx.tokenMeter` | [`llm/token-meter`](../packages/llm/token-meter/README.md) | 感知回放的单实例请求压力与表面压力 | | `ctx.bash` | [`bash/`](../packages/bash/README.md) | 前台和后台命令执行 | -| `ctx.subprocess` | [`subprocess/`](../packages/subprocess/README.md) | 供 bash、LSP 与 ACP subagent 后端使用的受管子进程树 | +| `ctx.subprocess` | [`subprocess/`](../packages/subprocess/README.md) | 可执行文件查找、受管进程树、终端 | | `ctx.pty` | [`pty/`](../packages/pty/README.md) | 按 owner 隔离的持久化终端会话 | | `ctx.sandbox` | [`sandbox/`](../packages/sandbox/README.md) | 通过 argv 包装和逐调用策略限制同一执行环境内的进程 | | `ctx.sandboxPolicy` | [`sandbox/`](../packages/sandbox/README.md) | 共享沙箱策略归属点 | | `ctx.codeRuntime` | [`code-runtime/`](../packages/code-runtime/README.md) | 执行模型编写的程序 | -| `ctx.fs` | [`fs/`](../packages/fs/README.md) | 文件系统提供方原语和策略事件 | +| `ctx.fs` | [`fs/`](../packages/fs/README.md) | 执行世界路径、有界 I/O 和策略事件 | | `ctx.lsp` | [`lsp/`](../packages/lsp/README.md) | 语义导航注册表 | | `ctx.skills` | [`skill/`](../packages/skill/README.md) | skill(技能)提供方注册表和渐进式披露 | | `ctx.web` | [`web/`](../packages/web/README.md) | 搜索与抓取提供方注册表 | @@ -155,7 +155,7 @@ idle inject: ### 功能模式 -可替换功能通常具有**接口/实现/消费方**三层:服务和事件、后端、面向模型的工具和提示词。Bash 是参考实现;[功能图](capability-seams.md)映射了每个包族。 +能力分为**接口/实现/消费方**三层。文件系统与进程管理提供方共同定义一个执行世界;Bash、PTY 和 LSP 都在其中运行,无需提供方专用 fork。参见[功能图](capability-seams.md)。 例外情况包括 LLM(大语言模型)合并接口和消费方、文件系统整合策略、web 使用注册表、skill 和 subagent 使用具名提供方。subagent 可以通过 spawn 创建全新实例、fork 一个已完成轮次的前缀、使用 ACP(Agent Client Protocol)子 agent,或将一个独立完整的轮次委派给 Codex 等真实产品提供方([subagent.md](core-data-structures/subagent.md))。 diff --git a/docs/capability-seams.md b/docs/capability-seams.md index c5ec654d0f..ae90e4a9bf 100644 --- a/docs/capability-seams.md +++ b/docs/capability-seams.md @@ -40,6 +40,13 @@ flowchart LR pkg_tool_bash["tool-bash"] pkg_hooks_claude["hooks-claude"] pkg_hooks_codex["hooks-codex"] + pkg_settings["settings"] + svc_settings["ctx.settings
User-settings seam"] + pkg_settings_local["settings-local"] + pkg_apiproxy["apiproxy"] + pkg_credentials["credentials"] + svc_credentials["ctx.credentials
Credential seam"] + pkg_credentials_local["credentials-local"] pkg_session_telemetry["session-telemetry"] svc_telemetry["ctx.telemetry
Session telemetry seam"] pkg_session_telemetry_otel["session-telemetry-otel"] @@ -51,12 +58,10 @@ flowchart LR svc_storageDomain["ctx.storageDomain
Domain data facility"] pkg_workspace["workspace"] svc_workspace["ctx.workspace
Workspace entity registry"] - pkg_apiproxy["apiproxy"] svc_sessionQuery["ctx.sessionQuery
Session reads, traces, filters, and search"] pkg_session_reference["session-reference"] pkg_tool_session_query["tool-session-query"] svc_sessionReferences["ctx.sessionReferences
Cross-session snapshot preparation"] - pkg_tui["tui"] pkg_session_title["session-title"] svc_sessionTitle["ctx.sessionTitle
Log-backed session titles"] pkg_session_title_first_message_llm["session-title-first-message-llm"] @@ -84,13 +89,11 @@ flowchart LR pkg_host_apiproxy["host-apiproxy"] pkg_session_projection_cache["session-projection-cache"] svc_sessionProjectionCache["ctx.sessionProjectionCache
Persisted projection cache"] - svc_tui["ctx.tui
Mounted-terminal interaction service"] pkg_skill["skill"] svc_skills["ctx.skills
Skill provider registry"] pkg_skill_local["skill-local"] svc_agents["ctx.agents
Agent service"] pkg_acp["acp"] - pkg_tui_demo["tui-demo"] svc_agentLoop["ctx.agentLoop
Concrete loop driver"] pkg_agent_spine_demo["agent-spine-demo"] pkg_goal["goal"] @@ -103,8 +106,13 @@ flowchart LR pkg_pty_local["pty-local"] pkg_lsp_local["lsp-local"] pkg_subagent_acp["subagent-acp"] + pkg_subagent_codex["subagent-codex"] + pkg_subagent_claude_code["subagent-claude-code"] pkg_bash["bash"] svc_bash["ctx.bash
Bash executor seam"] + pkg_pwsh_local["pwsh-local"] + pkg_tool_pwsh["tool-pwsh"] + pkg_bash_env["bash-env"] svc_bashEnv["ctx.bashEnv
Managed bash environment registry"] pkg_pty["pty"] svc_pty["ctx.pty
Persistent PTY session registry"] @@ -128,9 +136,11 @@ flowchart LR pkg_compact["compact"] svc_compact["ctx.compact
Compaction seam"] pkg_subagent["subagent"] - svc_subagents["ctx.subagents
Subagent provider registry"] + svc_subagents["ctx.subagents
Subagent provider and continuation service"] pkg_subagent_spawn["subagent-spawn"] pkg_subagent_fork["subagent-fork"] + pkg_subagent_dsh_sdk["subagent-dsh-sdk"] + pkg_tool_subagent_control["tool-subagent-control"] pkg_tool_ralph["tool-ralph"] pkg_tasks["tasks"] svc_tasks["ctx.tasks
Background task registry"] @@ -166,6 +176,7 @@ flowchart LR pkg_api_gateway --> svc_typertGateway pkg_approval --> svc_approval pkg_bash --> svc_bash + pkg_bash_env --> svc_bashEnv pkg_bash_local --> svc_bash pkg_bash_sandbox --> svc_bash pkg_code_runtime --> svc_codeRuntime @@ -174,6 +185,8 @@ flowchart LR pkg_compact --> svc_compact pkg_compact_basic --> svc_compact pkg_compact_tool_result_prune --> svc_toolResultPrune + pkg_credentials --> svc_credentials + pkg_credentials_local --> svc_credentials pkg_directory_picker --> svc_directoryPicker pkg_directory_picker_browse --> svc_directoryPicker pkg_directory_picker_native --> svc_directoryPicker @@ -191,6 +204,7 @@ flowchart LR pkg_plan_mode --> svc_planMode pkg_pty --> svc_pty pkg_pty_local --> svc_pty + pkg_pwsh_local --> svc_bash pkg_sandbox --> svc_sandbox pkg_sandbox_local --> svc_sandbox pkg_sandbox_policy --> svc_sandboxPolicy @@ -208,6 +222,8 @@ flowchart LR pkg_session_title --> svc_sessionTitle pkg_session_title_all_messages_llm --> svc_sessionTitle pkg_session_title_first_message_llm --> svc_sessionTitle + pkg_settings --> svc_settings + pkg_settings_local --> svc_settings pkg_skill --> svc_skills pkg_skill_local --> svc_skills pkg_spill --> svc_spillStore @@ -218,6 +234,9 @@ flowchart LR pkg_storage_sqlite --> svc_storage pkg_subagent --> svc_subagents pkg_subagent_acp --> svc_subagents + pkg_subagent_claude_code --> svc_subagents + pkg_subagent_codex --> svc_subagents + pkg_subagent_dsh_sdk --> svc_subagents pkg_subagent_fork --> svc_subagents pkg_subagent_spawn --> svc_subagents pkg_subprocess --> svc_subprocess @@ -226,10 +245,7 @@ flowchart LR pkg_tasks --> svc_tasks pkg_tasks_local --> svc_tasks pkg_token_meter --> svc_tokenMeter - pkg_tool_bash --> svc_bashEnv pkg_tools --> svc_tools - pkg_tui --> svc_tui - pkg_tui --> svc_userInteraction pkg_typert_registry --> svc_typert pkg_user_interaction --> svc_userInteraction pkg_web --> svc_web @@ -246,16 +262,20 @@ flowchart LR svc_agents --> pkg_agent_loop svc_agents --> pkg_cli_demo svc_agents --> pkg_subagent_inprocess - svc_agents --> pkg_tui_demo svc_approval --> pkg_tool_bash svc_approval --> pkg_tools svc_bash --> pkg_hooks_claude svc_bash --> pkg_hooks_codex svc_bash --> pkg_tool_bash + svc_bash --> pkg_tool_pwsh + svc_bashEnv --> pkg_tool_bash + svc_bashEnv --> pkg_tool_pwsh svc_clientModuleHost --> pkg_hmr svc_codeRuntime --> pkg_tools - svc_commands --> pkg_tui svc_compact --> pkg_compact_basic + svc_credentials --> pkg_apiproxy + svc_credentials --> pkg_llm_deepseek + svc_credentials --> pkg_llm_pi_ai svc_directoryPicker --> pkg_apiproxy svc_fs --> pkg_tool_fs svc_httpServer --> pkg_connection @@ -285,7 +305,6 @@ flowchart LR svc_sessionProjections --> pkg_tool_todo svc_sessionQuery --> pkg_session_reference svc_sessionQuery --> pkg_tool_session_query - svc_sessionReferences --> pkg_tui svc_sessions --> pkg_agent svc_sessions --> pkg_agent_loop svc_sessions --> pkg_cli_demo @@ -294,17 +313,23 @@ flowchart LR svc_sessions --> pkg_session_query svc_sessions --> pkg_session_query_sqlite svc_sessions --> pkg_subagent_inprocess + svc_settings --> pkg_apiproxy + svc_settings --> pkg_llm_deepseek + svc_settings --> pkg_llm_pi_ai svc_skills --> pkg_tool_skill svc_spillStore --> pkg_spill_policy svc_storage --> pkg_storage_domain svc_storageDomain --> pkg_workspace svc_subagents --> pkg_tool_ralph svc_subagents --> pkg_tool_subagent + svc_subagents --> pkg_tool_subagent_control svc_subprocess --> pkg_bash_local svc_subprocess --> pkg_bash_sandbox svc_subprocess --> pkg_lsp_local svc_subprocess --> pkg_pty_local svc_subprocess --> pkg_subagent_acp + svc_subprocess --> pkg_subagent_claude_code + svc_subprocess --> pkg_subagent_codex svc_systemPrompt --> pkg_agent_loop svc_systemPrompt --> pkg_tool_fs svc_systemPrompt --> pkg_tool_pty @@ -329,7 +354,6 @@ flowchart LR svc_typert --> pkg_api_gateway svc_typert --> pkg_typert_loader svc_userInteraction --> pkg_tool_ask_user - svc_userInteraction --> pkg_tui svc_web --> pkg_tool_web svc_workflows --> pkg_tool_ralph svc_workflows --> pkg_tool_workflow @@ -347,28 +371,29 @@ flowchart LR | `ctx.typert` | `core` | [`typert-registry`](../packages/typert/registry) | - | [`typert-loader`](../packages/typert/loader), [`api-gateway`](../packages/api/gateway) | - | Plugins register live zod contributions directly or through dsh-typert-loader; the API gateway consumes invocation descriptors and providers, while other runtime consumers query schemas and reflection metadata at their own edges. | | `ctx.typertGateway` | `core` | [`api-gateway`](../packages/api/gateway) | - | - | - | Associates generated Remote descriptors with live Cordis services, resolves registered identities, and exposes unary calls through the shared Connection RPC carrier. | | `ctx.sessionPersistence` | `seam` | [`session-persistence`](../packages/session-persistence/session-persistence) | [`session-persistence-jsonl`](../packages/session-persistence/session-persistence-jsonl), [`session-persistence-sqlite`](../packages/session-persistence/session-persistence-sqlite) | [`agent-loop`](../packages/core/agent-loop), [`tool-bash`](../packages/bash/tool-bash), [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex), [`session-query`](../packages/session-query/session-query), [`session-query-sqlite`](../packages/session-query/session-query-sqlite) | - | Backends persist the same SessionEvent vocabulary; apps choose a backend at composition time. | +| `ctx.settings` | `seam` | [`settings`](../packages/settings/settings) | [`settings-local`](../packages/settings/settings-local) | [`llm-deepseek`](../packages/llm/llm-deepseek), [`llm-pi-ai`](../packages/llm/llm-pi-ai), `apiproxy` | - | Plugins register namespace schemas and resolve layered values; providers store the raw document. The LLM adapters register their entry config as the composition base under the user section; the web gateway serves redacted layered descriptors and writes the user layer. | +| `ctx.credentials` | `seam` | [`credentials`](../packages/credentials/credentials) | [`credentials-local`](../packages/credentials/credentials-local) | [`llm-deepseek`](../packages/llm/llm-deepseek), [`llm-pi-ai`](../packages/llm/llm-pi-ai), `apiproxy` | - | Configuration carries references to secrets; providers own the values. Consumers resolve per operation, so a rotated credential reaches the very next request; the web gateway exposes value-free views and write-only storage. | | `ctx.telemetry` | `seam` | [`session-telemetry`](../packages/telemetry/session-telemetry) | [`session-telemetry-otel`](../packages/telemetry/session-telemetry-otel) | - | - | The seam captures, redacts, and hands session records to one backend; nothing else consumes the service — its output leaves the process. | | `ctx.storage` | `seam` | [`storage`](../packages/storage/storage) | [`storage-json`](../packages/storage/storage-json), [`storage-sqlite`](../packages/storage/storage-sqlite) | [`storage-domain`](../packages/storage/storage-domain) | - | Backends register side by side under names; data forms (domain first) mount on the hub and translate typed operations into opaque KV-unit primitives. | | `ctx.storageDomain` | `core` | [`storage-domain`](../packages/storage/storage-domain) | - | [`workspace`](../packages/workspace/workspace) | - | Waits for every configured backend, then publishes the domain form as one lifecycle-bound service for typed durable state. | | `ctx.workspace` | `core` | [`workspace`](../packages/workspace/workspace) | - | `apiproxy` | - | Owns WorkspaceId-branded records over the domain facility; stable sessionIds accounts drive Host RPC and GUI projections. | | `ctx.sessionQuery` | `seam` | [`session-query`](../packages/session-query/session-query) | [`session-query-sqlite`](../packages/session-query/session-query-sqlite) | [`session-reference`](../packages/context/session-reference), [`tool-session-query`](../packages/session-query/tool-session-query) | - | The interface supplies exact reads, filters, and traces; its concrete backend adds full-text reconciliation, ranking, snippets, and cursor generations, while the model consumer owns workspace authority and cursor-free rendering. | -| `ctx.sessionReferences` | `core` | [`session-reference`](../packages/context/session-reference) | - | [`tui`](../packages/ui/tui) | - | Projects bounded current-surface conversation snapshots into durable untrusted message context; host adapters own mention syntax. | +| `ctx.sessionReferences` | `core` | [`session-reference`](../packages/context/session-reference) | - | - | - | Projects bounded current-surface conversation snapshots into durable untrusted message context; host adapters own mention syntax. | | `ctx.sessionTitle` | `seam` | [`session-title`](../packages/session-title/session-title) | [`session-title-first-message-llm`](../packages/session-title/session-title-first-message-llm), [`session-title-all-messages-llm`](../packages/session-title/session-title-all-messages-llm) | - | - | Owns the deterministic fallback, latest-title fold, and sole optional asynchronous provider registration. | | `ctx.systemPrompt` | `core` | [`system-prompt`](../packages/core/system-prompt) | - | [`agent-loop`](../packages/core/agent-loop), [`tools`](../packages/core/tools), [`tool-fs`](../packages/fs/tool-fs), [`tool-pty`](../packages/pty/tool-pty), [`tool-web`](../packages/web/tool-web) | - | Collects prompt sections and model-facing tool schemas for each step. | | `ctx.tools` | `core` | [`tools`](../packages/core/tools) | - | [`agent-loop`](../packages/core/agent-loop), [`tool-ask-user`](../packages/ui/tool-ask-user), [`tool-bash`](../packages/bash/tool-bash), [`tool-cordis`](../packages/cordis/tool-cordis), [`tool-fs`](../packages/fs/tool-fs), [`tool-pty`](../packages/pty/tool-pty), [`tool-skill`](../packages/skill/tool-skill), [`tool-subagent`](../packages/subagent/tool-subagent), [`tool-todo`](../packages/todo/tool-todo), [`tool-web`](../packages/web/tool-web) | - | Registers capabilities, owns Code Mode transport, and routes calls through pre-policy, monotonic guards, around dispatch, post-policy, and final-result observation. | -| `ctx.userInteraction` | `seam` | [`user-interaction`](../packages/ui/user-interaction) | [`tui`](../packages/ui/tui) | [`tool-ask-user`](../packages/ui/tool-ask-user), [`tui`](../packages/ui/tui) | - | UI front doors provide the active human-answer provider; tool-ask-user pauses a tool call on the provider-neutral ask() promise. | +| `ctx.userInteraction` | `seam` | [`user-interaction`](../packages/ui/user-interaction) | - | [`tool-ask-user`](../packages/ui/tool-ask-user) | - | UI front doors provide the active human-answer provider; tool-ask-user pauses a tool call on the provider-neutral ask() promise. | | `ctx.planMode` | `core` | [`plan-mode`](../packages/plan/plan-mode) | - | - | - | Folds logged plan/mode state, flushes user selections at turn boundaries, renders deployment-owned guidance, registers /plan, and keeps the plan-exit schema stable across transitions. | -| `ctx.commands` | `core` | [`commands`](../packages/ui/commands) | - | [`tui`](../packages/ui/tui) | - | Plugins register direct human commands; TUI consumes the effective per-agent catalog without sending invocations to the model. | +| `ctx.commands` | `core` | [`commands`](../packages/ui/commands) | - | - | - | Plugins register direct human commands without sending invocations to the model. | | `ctx.sessionProjections` | `core` | [`session-projection`](../packages/session-projection/session-projection) | - | [`tool-todo`](../packages/todo/tool-todo), [`session-title`](../packages/session-title/session-title), [`host-apiproxy`](../packages/host/apiproxy) | - | Domains register state-driven fold units; the eager drive keeps per-session watermark states and api-proxy serves baselines and pushes changed values. | | `ctx.sessionProjectionCache` | `core` | [`session-projection-cache`](../packages/session-projection/session-projection-cache) | - | [`host-apiproxy`](../packages/host/apiproxy) | - | Durably checkpoints projection unit states per session (throttled + turn/end/detach mandatory points) and serves the cold-read ladder: cache row + persistence tail replay, so listings never load full logs. | -| `ctx.tui` | `bundle` | [`tui`](../packages/ui/tui) | - | - | - | One TUI front door provides a FIFO overlay host; injected plugins receive caller-fiber ownership without access to pi-tui or terminal lifecycle state. | | `ctx.skills` | `seam` | [`skill`](../packages/skill/skill) | [`skill-local`](../packages/skill/skill-local) | [`tool-skill`](../packages/skill/tool-skill) | - | Merges provider skill catalogs; tool-skill renders the session-prefix catalog and loads complete skill bodies. | -| `ctx.agents` | `core` | [`agent`](../packages/core/agent) | - | [`agent-loop`](../packages/core/agent-loop), [`acp`](../packages/acp/acp), [`cli-demo`](../packages/examples/cli-demo), [`subagent-inprocess`](../packages/subagent/subagent-inprocess), [`tui-demo`](../packages/examples/tui-demo) | - | Owns live Agent handles, the create/resume factory seam, and process-local initiator propagation. | +| `ctx.agents` | `core` | [`agent`](../packages/core/agent) | - | [`agent-loop`](../packages/core/agent-loop), [`acp`](../packages/acp/acp), [`cli-demo`](../packages/examples/cli-demo), [`subagent-inprocess`](../packages/subagent/subagent-inprocess) | - | Owns live Agent handles, the create/resume factory seam, and process-local initiator propagation. | | `ctx.agentLoop` | `bundle` | [`agent-loop`](../packages/core/agent-loop) | - | [`agent-spine-demo`](../packages/examples/agent-spine-demo) | - | The one concrete loop plugin; extension packages depend on dsh-agent events and services, not on this package. | | `ctx.goals` | `core` | [`goal`](../packages/goal/goal) | - | - | - | Folds revisioned objective state from the session log and keeps live continuation activation process-local. | -| `ctx.subprocess` | `seam` | [`subprocess`](../packages/subprocess/subprocess) | [`subprocess-local`](../packages/subprocess/subprocess-local) | [`bash-local`](../packages/bash/bash-local), [`bash-sandbox`](../packages/bash/bash-sandbox), [`pty-local`](../packages/pty/pty-local), [`lsp-local`](../packages/lsp/lsp-local), [`subagent-acp`](../packages/subagent/subagent-acp) | - | The bash executors, PTY shell backend, LSP host, and ACP subagent backend spawn through ctx.subprocess; the service owns process coordinates, tree/session lifetime, stdio dispositions, terminal mechanics, and kill escalation. | -| `ctx.bash` | `seam` | [`bash`](../packages/bash/bash) | [`bash-local`](../packages/bash/bash-local), [`bash-sandbox`](../packages/bash/bash-sandbox) | [`tool-bash`](../packages/bash/tool-bash), [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex) | - | The model-facing bash tools and hook bridges consume this seam; sandboxed or remote executors replace bash-local without touching them. | -| `ctx.bashEnv` | `core` | [`tool-bash`](../packages/bash/tool-bash) | - | - | - | Plugins declare effect-scoped DSH_* facts; tool-bash collects one trusted snapshot per execution and the executor rebuilds the namespace. | +| `ctx.subprocess` | `seam` | [`subprocess`](../packages/subprocess/subprocess) | [`subprocess-local`](../packages/subprocess/subprocess-local) | [`bash-local`](../packages/bash/bash-local), [`bash-sandbox`](../packages/bash/bash-sandbox), [`pty-local`](../packages/pty/pty-local), [`lsp-local`](../packages/lsp/lsp-local), [`subagent-acp`](../packages/subagent/subagent-acp), [`subagent-codex`](../packages/subagent/subagent-codex), [`subagent-claude-code`](../packages/subagent/subagent-claude-code) | - | The bash executors, the PTY shell backend, the LSP host, and the out-of-process ACP, Codex, and Claude Code subagent backends spawn through ctx.subprocess; the service owns process coordinates, tree/session lifetime, stdio dispositions, terminal mechanics, and kill escalation. | +| `ctx.bash` | `seam` | [`bash`](../packages/bash/bash) | [`bash-local`](../packages/bash/bash-local), [`bash-sandbox`](../packages/bash/bash-sandbox), [`pwsh-local`](../packages/bash/pwsh-local) | [`tool-bash`](../packages/bash/tool-bash), [`tool-pwsh`](../packages/bash/tool-pwsh), [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex) | - | The model-facing shell tools and hook bridges consume this seam; sandboxed, remote, or PowerShell executors replace bash-local without touching them. | +| `ctx.bashEnv` | `core` | [`bash-env`](../packages/bash/bash-env) | - | [`tool-bash`](../packages/bash/tool-bash), [`tool-pwsh`](../packages/bash/tool-pwsh) | - | Plugins declare effect-scoped DSH_* facts; each shell tool collects one trusted snapshot per execution and its executor rebuilds the namespace. | | `ctx.pty` | `seam` | [`pty`](../packages/pty/pty) | [`pty-local`](../packages/pty/pty-local) | [`tool-pty`](../packages/pty/tool-pty) | - | The registry owns exact-Agent session identity and cleanup; backends own terminal mechanics, while tool-pty exposes the owner-scoped model surface. | | `ctx.sandbox` | `seam` | [`sandbox`](../packages/sandbox/sandbox) | [`sandbox-local`](../packages/sandbox/sandbox-local) | [`bash-sandbox`](../packages/bash/bash-sandbox), [`pty-local`](../packages/pty/pty-local) | - | Consumers hand over the exact argv they are about to spawn; same-world backends wrap it under a per-call policy and report enforcement. | | `ctx.sandboxPolicy` | `core` | [`sandbox-policy`](../packages/sandbox/sandbox-policy) | - | [`bash-sandbox`](../packages/bash/bash-sandbox), [`fs-sandbox`](../packages/fs/fs-sandbox), [`pty-local`](../packages/pty/pty-local) | - | The one home for the deployment default mode + workspace root; only the sandboxed executor and provider read the service (the tool layers use the pure `sandbox/mode` fold it also exports). Both enforcing families read it so bash and fs cannot confine to different roots. | @@ -377,7 +402,7 @@ flowchart LR | `ctx.codeRuntime` | `seam` | [`code-runtime`](../packages/code-runtime/code-runtime) | [`code-runtime-worker`](../packages/code-runtime/code-runtime-worker) | [`tools`](../packages/core/tools) | - | Runs one model-written program against host-provided async bindings; backends differ by substrate and language (the tool registry consumes it for Code Mode). | | `ctx.fs` | `seam` | [`fs`](../packages/fs/fs) | [`fs-local`](../packages/fs/fs-local), [`fs-sandbox`](../packages/fs/fs-sandbox) | [`tool-fs`](../packages/fs/tool-fs) | [`fs-policy`](../packages/fs/fs-policy) | tool-fs executes read/write/edit through ctx.fs; fs-sandbox fences mutations by the shared sandbox mode; fs-policy contributes observed-state checks through the fs/* event gate. | | `ctx.compact` | `seam` | [`compact`](../packages/compact/compact) | [`compact-basic`](../packages/compact/compact-basic) | [`compact-basic`](../packages/compact/compact-basic) | - | The basic backend consumes post-step pressure and request-error recovery events; a model-facing compact tool remains deferred. | -| `ctx.subagents` | `seam` | [`subagent`](../packages/subagent/subagent) | [`subagent-spawn`](../packages/subagent/subagent-spawn), [`subagent-fork`](../packages/subagent/subagent-fork), [`subagent-acp`](../packages/subagent/subagent-acp) | [`tool-subagent`](../packages/subagent/tool-subagent), [`tool-ralph`](../packages/workflow/tool-ralph) | - | Providers implement transports; tool-subagent exposes configured delegation while tool-ralph requires one fresh structured-output route. | +| `ctx.subagents` | `seam` | [`subagent`](../packages/subagent/subagent) | [`subagent-spawn`](../packages/subagent/subagent-spawn), [`subagent-fork`](../packages/subagent/subagent-fork), [`subagent-acp`](../packages/subagent/subagent-acp), [`subagent-codex`](../packages/subagent/subagent-codex), [`subagent-claude-code`](../packages/subagent/subagent-claude-code), [`subagent-dsh-sdk`](../packages/subagent/subagent-dsh-sdk) | [`tool-subagent`](../packages/subagent/tool-subagent), [`tool-subagent-control`](../packages/subagent/tool-subagent-control), [`tool-ralph`](../packages/workflow/tool-ralph) | - | Providers implement transports; the service also owns optional Activation-based continuation orchestration, tool-subagent selects one-shot or continuable delegation, tool-subagent-control delivers follow-ups, and tool-ralph requires one fresh structured-output route. | | `ctx.tasks` | `seam` | [`tasks`](../packages/tasks/tasks) | [`tasks-local`](../packages/tasks/tasks-local) | [`tool-bash`](../packages/bash/tool-bash), [`tool-pty`](../packages/pty/tool-pty), [`tool-subagent`](../packages/subagent/tool-subagent), [`tool-tasks`](../packages/tasks/tool-tasks) | - | Producers (background bash, PTY sends, and subagent delegations) register running work; tool-tasks is the model-facing control surface that reads, lists, and kills it; tasks-local is the process-local registry. | | `ctx.web` | `seam` | [`web`](../packages/web/web) | [`web-search-exa`](../packages/web/web-search-exa), [`web-search-perplexity`](../packages/web/web-search-perplexity), [`web-search-deepseek`](../packages/web/web-search-deepseek), [`web-fetch-local`](../packages/web/web-fetch-local) | [`tool-web`](../packages/web/tool-web) | - | Search and fetch providers register into one ctx.web seam; tool-web owns the stable model-facing names. | | `ctx.spillStore` | `seam` | [`spill`](../packages/spill/spill) | [`spill-local`](../packages/spill/spill-local) | [`spill-policy`](../packages/spill/spill-policy) | - | The backend saves oversized tool text and returns a model-facing locator plus retrieval hint; spill-policy is the tools/post-execute consumer that decides when to spill. | diff --git a/docs/cordis-catalog/services.md b/docs/cordis-catalog/services.md index a9077ae139..7e125cd02c 100644 --- a/docs/cordis-catalog/services.md +++ b/docs/cordis-catalog/services.md @@ -594,6 +594,34 @@ Abstract filesystem provider. Targets must preserve identity across aliases; rea */ abstract resolve(path: string, opts?: { cwd?: string; signal?: AbortSignal }): Promise +/** + * Return the canonical absolute path a subprocess in this filesystem's + * execution world can open. The path is deliberately separate from + * {@link FsTarget.targetKey}: consumers may pass this value to another OS + * capability, but must continue treating the target key as opaque. + * @param target - the resolved target whose process path is required. + * @returns an absolute path in the backend's execution world. + */ +abstract processPath(target: FsTarget): string + +/** + * Return the canonical `file:` URI for a target in this filesystem's + * execution world. Backends own URI encoding because the host platform may + * differ from the execution platform. + * @param target - the resolved target to encode. + * @returns the target's canonical file URI. + */ +abstract fileUrl(target: FsTarget): string + +/** + * Test canonical containment without exposing or parsing backend target + * keys. Both targets must come from this provider. + * @param parent - canonical directory target. + * @param child - canonical candidate target. + * @returns true when `child` is `parent` or a descendant of it. + */ +abstract contains(parent: FsTarget, child: FsTarget): boolean + /** * Return target metadata, or `undefined` when the target does not exist. * @param target - the resolved target to stat. @@ -678,7 +706,7 @@ abstract editText( target: FsTarget, edit: FsEditRequest, expected?: { version: Types: [FsDirEntry](../core-data-structures/filesystem.md) · [FsEditOutcome](../core-data-structures/filesystem.md) · [FsEditRequest](../core-data-structures/filesystem.md) · [FsInfo](../core-data-structures/filesystem.md) · [FsPathInfo](../core-data-structures/filesystem.md) · [FsTarget](../core-data-structures/filesystem.md) · [FsVersion](../core-data-structures/filesystem.md) · [FsWriteIntent](../core-data-structures/filesystem.md) · [FsWriteOutcome](../core-data-structures/filesystem.md) · [SandboxExecutionPolicy](../core-data-structures/sandbox.md) -Source: [`packages/fs/fs/src/index.ts:81`](../../packages/fs/fs/src/index.ts) +Source: [`packages/fs/fs/src/index.ts:83`](../../packages/fs/fs/src/index.ts) ## `ctx.goals` — `GoalService` @@ -2215,12 +2243,27 @@ Abstract subprocess service. Subclass, implement spawn, and load the subclass as Implementations must honor these semantics: +- Executable paths belong to one execution world shared with the mounted filesystem provider. - spawn returns immediately with a live handle; `done` resolves at process close with exit facts and rejects only for spawn-level failures. - Collect-mode readers are offset-based and non-consuming, so independent readers never consume one another's output; lossy reads report truncation and the spill file holding the complete stream when one exists. Piped streams are handed to the caller raw and never buffered here. - SubprocessHandle.terminate (and the spec's abort signal) escalates SIGTERM→grace→SIGKILL — the only termination verb — tree-scoped on every platform. SubprocessHandle.waitForExit observes whole-tree liveness, so a consumer-owned teardown ladder can hold each tier on real quiescence. - Disposal of the service terminates all still-running managed processes and awaits their exit. +- spawnTerminal owns terminal allocation, text transport, foreground groups, signalling, and whole-session quiescence behind one awaited termination method; readiness and persistent-shell policy stay in the PTY consumer. Its output stream ends after queued terminal output when the top-level process exits. ```ts cordis-catalog +/** + * Resolve one configured executable in this provider's execution world. + * Absolute paths are verified; bare names use the provider's scrubbed PATH + * plus explicit environment overrides. Relative paths containing separators + * are rejected: no current consumer defines which directory they would + * resolve against, so providers fail loud instead of guessing. + * @param command - absolute executable path or bare PATH name. + * @param env - explicit environment entries used for lookup. + * @param signal - aborts remote or local lookup. + * @returns a canonical executable path. + */ +abstract resolveExecutable( command: string, env?: Readonly>, signal?: AbortSignal, ): Promise + /** * Start one managed child process from a fully-specified spec; this seam * applies no defaults. @@ -2228,11 +2271,20 @@ Implementations must honor these semantics: * @returns the live process handle (streams/readers, signalling, outcome promise). */ abstract spawn(spec: SubprocessSpawnSpec): SubprocessHandle + +/** + * Allocate a real terminal and start one owned process session. This is the + * only non-pipe process primitive: implementations own terminal byte I/O, + * foreground groups, signals, and complete session-tree cleanup. + * @param spec - fully specified argv, cwd, environment, dimensions, grace, and allocation cancellation. + * @returns the live terminal handle after allocation succeeds. + */ +abstract spawnTerminal(spec: SubprocessTerminalSpawnSpec): Promise ``` -Types: [SubprocessHandle](../core-data-structures/subprocess.md) · [SubprocessSpawnSpec](../core-data-structures/subprocess.md) +Types: [SubprocessHandle](../core-data-structures/subprocess.md) · [SubprocessSpawnSpec](../core-data-structures/subprocess.md) · [SubprocessTerminalHandle](../core-data-structures/subprocess.md) · [SubprocessTerminalSpawnSpec](../core-data-structures/subprocess.md) -Source: [`packages/subprocess/subprocess/src/index.ts:91`](../../packages/subprocess/subprocess/src/index.ts) +Source: [`packages/subprocess/subprocess/src/index.ts:102`](../../packages/subprocess/subprocess/src/index.ts) ## `ctx.systemPrompt` — `SystemPrompt` diff --git a/docs/core-data-structures/filesystem.i18n.yaml b/docs/core-data-structures/filesystem.i18n.yaml index 78386c54fd..2f9478a7ba 100644 --- a/docs/core-data-structures/filesystem.i18n.yaml +++ b/docs/core-data-structures/filesystem.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write docs/core-data-structures/filesystem.md filesystem.md: addded9f673ed435e95109d4fb772967514c0b87 -filesystem.zh.md: 3ef16ede1e137d2a1310861941ac3244ca105650 +filesystem.zh.md: 1e378928ed570b97dbec73b836a7e6ff18726ff8 diff --git a/docs/core-data-structures/lsp.i18n.yaml b/docs/core-data-structures/lsp.i18n.yaml index 329d5086e0..25b54df775 100644 --- a/docs/core-data-structures/lsp.i18n.yaml +++ b/docs/core-data-structures/lsp.i18n.yaml @@ -3,4 +3,4 @@ # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write docs/core-data-structures/lsp.md lsp.md: 03d0ce2dbe246aadb6f6c9a702fa50e3e792eb09 -lsp.zh.md: ef9623f01124bc459d3bbceba340c3870e678baa +lsp.zh.md: d428f2b5f9568b231b0b50a13bd4a41f7ea81346 diff --git a/docs/core-data-structures/subprocess.i18n.yaml b/docs/core-data-structures/subprocess.i18n.yaml index 50e219a2c6..b2d38854b2 100644 --- a/docs/core-data-structures/subprocess.i18n.yaml +++ b/docs/core-data-structures/subprocess.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write docs/core-data-structures/subprocess.md -subprocess.md: 895bfe3d763853a86648e8aaab8b091a26006255 -subprocess.zh.md: 044e924674ecf5dd8f8a1c71fe39653e7525dfb0 +subprocess.md: 023b122218ad1caa2b8e16c26b0bc8b0d4183c28 +subprocess.zh.md: 5ee9c782c0ce73ca9a2e694450ed38b8d548dc09 diff --git a/docs/core-data-structures/subprocess.zh.md b/docs/core-data-structures/subprocess.zh.md index 044e924674..5ee9c782c0 100644 --- a/docs/core-data-structures/subprocess.zh.md +++ b/docs/core-data-structures/subprocess.zh.md @@ -1,4 +1,4 @@ -# 进程管理器 +# 子进程 [English](subprocess.md) | 中文 @@ -12,7 +12,7 @@ ## 受管环境命名空间与捕获的输出 -`DSH_*` 变量是归 Harness 所有的子进程事实;实现会在合并调用方显式 `env` 之前丢弃环境中已有的 `DSH_*` 名称,因此当前事实只会以有意提供的条目形式到达,每条被收集的流都通过 `CollectedOutput` 报告自身的截断与 spill 恢复状态。 +`DSH_*` 变量是归 Harness 所有的子进程事实;实现会在合并调用方显式 `env` 之前丢弃环境中已有的 `DSH_*` 名称,因此当前事实只会以有意提供的字符串条目形式到达,而显式的 `undefined` tombstone 会删除普通环境中已有的值。每条被收集的流都通过 `CollectedOutput` 报告自身的截断与 spill 恢复状态。 ```ts type-equiv /** One environment key inside the managed {@link DSH_ENV_PREFIX} namespace. */ @@ -36,7 +36,7 @@ interface CollectedOutput { } ``` -## Node 形状的 stdio 处置方式(disposition) +## Node 风格的 stdio 处置方式(disposition) 每条流的处置方式都显式给出,由各消费方自行选择:原始管道用于协议分帧(LSP JSON-RPC、ACP ndjson),inherit 用于直通的诊断输出,收集模式用于有界的批量输出;其中 spill 文件是可选的,因此诊断尾部(语言服务器的 stderr)可以只在内存中缓冲,不留下任何文件。 @@ -88,7 +88,7 @@ interface SubprocessStdio { ## 完全显式的 spawn spec -该 seam 不应用任何默认值:每项处置方式、限制与目录都在 spec 上显式给出,因此由调用方自己的配置决定它们,而不是由某个隐藏的进程管理器默认值决定。`argv` 绝不经过 shell 解释。 +该 seam 不应用任何默认值:每项处置方式、限制与目录都在 spec 上显式给出,因此由调用方自己的配置决定它们,而不是由某个隐藏的子进程服务默认值决定。`argv` 绝不经过 shell 解释。 ```ts type-equiv /** @@ -105,10 +105,11 @@ interface SubprocessSpawnSpec { /** Per-stream stdio dispositions. */ stdio: SubprocessStdio /** - * Grace period in milliseconds for the {@link SubprocessHandle.terminate} - * escalation and for draining still-open collected pipes after the process - * exits (an inherited descriptor held by a surviving descendant cannot hold - * the outcome open indefinitely). + * Positive finite grace period in milliseconds, no greater than + * `MAX_TIMER_DELAY_MS`, for the {@link SubprocessHandle.terminate} escalation + * and for draining still-open collected pipes after the process exits (an + * inherited descriptor held by a surviving descendant cannot hold the + * outcome open indefinitely). */ graceMs: number /** @@ -119,19 +120,18 @@ interface SubprocessSpawnSpec { signal?: AbortSignal | undefined /** * Explicit environment entries merged onto the implementation's scrubbed - * parent base (see `scrubbedParentEnv`), with no namespace validation: - * every entry is a deliberate caller opt-in, so a forwarded - * credential-shaped entry or a current `DSH_*` fact survives precisely - * because this layer merges after the scrub that drops its ambient - * namesake. + * parent base (see `scrubbedParentEnv`), with no namespace validation. A + * string is a deliberate caller opt-in, so a forwarded credential-shaped + * entry or current `DSH_*` fact survives the scrub; `undefined` is a + * tombstone that removes an ordinary ambient entry from the child. */ - env?: Record | undefined + env?: NodeJS.ProcessEnv | undefined } ``` ## 句柄:流、读取器与以进程树为范围的终止 -spawn 会立即返回一个实时句柄。收集模式的读取器接受全流字节偏移量且从不消费,因此独立的读取器不会抢走彼此的增量;管道化的流归调用方所有。终止在每个平台上都以进程树为范围:`terminate()`(唯一的终止动词)执行 SIGTERM→宽限期→SIGKILL 升级,`waitForExit()` 观察整棵进程树——这足以让消费方构建自己的拆卸阶梯(ACP 后端以 stdin EOF 打头的 `disposeAcpChild` 即是模板)。 +spawn 会立即返回一个活动句柄。收集模式的读取器接受全流字节偏移量且从不消费,因此独立的读取器不会抢走彼此的增量;管道化的流归调用方所有。终止在每个平台上都以进程树为范围:`terminate()`(唯一的终止动词)执行 SIGTERM→宽限期→SIGKILL 升级,`waitForExit()` 观察整棵进程树。这足以让消费方构建自己的拆卸阶梯;ACP 后端的 `disposeAcpChild` 以 stdin EOF 开始,即为仓库内模板。 ```ts type-equiv /** diff --git a/packages/README.i18n.yaml b/packages/README.i18n.yaml index 458da3b713..d10f476f79 100644 --- a/packages/README.i18n.yaml +++ b/packages/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/README.md -README.md: 3f467641bbc9eae14a94aa2d3bff0402116a9d3f -README.zh.md: c9d11bbf6239b4239a4e037dac63b05d3a9a58f7 +README.md: 8fbb6069a784a5bd45423a4e1ae11834a597750d +README.zh.md: 42a8d691344c716021188df6fd870a841d543f36 diff --git a/packages/bash/pwsh-local/tests/executor.spec.ts b/packages/bash/pwsh-local/tests/executor.spec.ts index c7da16c44c..350ea4de89 100644 --- a/packages/bash/pwsh-local/tests/executor.spec.ts +++ b/packages/bash/pwsh-local/tests/executor.spec.ts @@ -119,6 +119,8 @@ describe('spawn construction (pure, every platform)', () => { /** A subprocess service that records spawn specs and settles instantly. */ class CapturingSubprocessService extends SubprocessService { specs: SubprocessSpawnSpec[] = [] + override async resolveExecutable(command: string): Promise { return command } + override spawnTerminal(): Promise { throw new Error('pwsh spawns pipes, never terminals') } private readonly reader: SubprocessOutputReader = { readFrom: () => ({ text: '', lossy: false, nextOffset: 0 }), } diff --git a/packages/bash/pwsh-local/tsconfig.json b/packages/bash/pwsh-local/tsconfig.json index 7381575674..53ccc94926 100644 --- a/packages/bash/pwsh-local/tsconfig.json +++ b/packages/bash/pwsh-local/tsconfig.json @@ -18,19 +18,19 @@ "path": "../../../vendor/schemastery" }, { - "path": "../code-runtime" + "path": "../../util/brand" }, { - "path": "../code-runtime-worker" + "path": "../../util/timeout" + }, + { + "path": "../../bash/bash" }, { "path": "../../subprocess/subprocess" }, { "path": "../../support/invariants" - }, - { - "path": "../../util/timeout" } ] } diff --git a/packages/code-runtime/code-runtime-worker/README.md b/packages/code-runtime/code-runtime-worker/README.md index 3de3bc622e..590b79dcd1 100644 --- a/packages/code-runtime/code-runtime-worker/README.md +++ b/packages/code-runtime/code-runtime-worker/README.md @@ -34,7 +34,7 @@ Every field is validated and defaulted; `maxOutputBytes` is a safe integer of at Source mode loads erasable-only `src/worker.ts` through Node's native type stripping. Its transitive runtime closure contains only Node built-ins and relative source modules, so a fresh checkout never requires a sibling workspace package's unbuilt `lib/` export. The worker-local and session-owned JSON boundaries both flatten and rebuild validated values around the message port so application nesting never reaches structured clone. Built mode passes the sibling `lib/worker.cjs` as a filesystem path because pkg's VFS Worker hook expects CommonJS; the same path works under ordinary Node. The repository-wide requirement to exercise this published entry path belongs to the [testing policy](../../../docs/testing.md). -The SDK surface is the default/named `WorkerCodeRuntime` class plus `Config`. The `./runtime-host` subpath shares type stripping, binding validation/dispatch, lossless JSON transport, and output accounting with sibling worker-based implementations; it is implementation support, not a plugin. The operational `./worker` subpath exists only as the packaged spawn entry; the wire protocol and bootstrap helpers remain source-private. +The SDK surface is the default/named `WorkerCodeRuntime` class plus `Config`. The operational `./worker` subpath exists only as the packaged spawn entry; the wire protocol and bootstrap helpers are source-private implementation details. ## Model Experience diff --git a/packages/code-runtime/code-runtime-worker/tests/runtime.spec.ts b/packages/code-runtime/code-runtime-worker/tests/runtime.spec.ts index db2c167354..54f58eb414 100644 --- a/packages/code-runtime/code-runtime-worker/tests/runtime.spec.ts +++ b/packages/code-runtime/code-runtime-worker/tests/runtime.spec.ts @@ -597,27 +597,6 @@ describe('WorkerCodeRuntime — hostile programs (real workers)', () => { expect(result.value).toEqual({ name: 'ToolCallError', toolName: 'bad', message: 'binding resolution must be lossless JSON' }) }) - it('contains binding rejections whose thrown values cannot be rendered', async () => { - const { runtime } = await setup() - const result = await runtime.run({ - program: 'try { await tools.bad({}) } catch (error) { return { name: error.name, toolName: error.toolName, message: error.message } }', - bindings: tools({ - bad: async () => { - const hostile = new Error('hidden') - Object.defineProperty(hostile, 'message', { - get() { throw new Error('message getter failed') }, - }) - throw hostile - }, - }), - }) - expect(result.value).toEqual({ - name: 'ToolCallError', - toolName: 'bad', - message: 'binding rejected with an unrenderable value', - }) - }) - it('rejects lossy binding arguments in the worker before invoking the host binding', async () => { const { runtime } = await setup() let calls = 0 diff --git a/packages/fs/tool-fs-search/tests/tools.spec.ts b/packages/fs/tool-fs-search/tests/tools.spec.ts index e9ae5588ee..c02e4f93db 100644 --- a/packages/fs/tool-fs-search/tests/tools.spec.ts +++ b/packages/fs/tool-fs-search/tests/tools.spec.ts @@ -148,6 +148,8 @@ class FakeHandle implements SubprocessHandle { */ class FakeSubprocess extends SubprocessService { spawns: SubprocessSpawnSpec[] = [] + override async resolveExecutable(command: string): Promise { return command } + override spawnTerminal(): Promise { throw new Error('search tools spawn pipes, never terminals') } handles: FakeHandle[] = [] /** Arms the per-spawn script; a `{ reject }` return scripts a spawn-level failure. */ handler: (spec: SubprocessSpawnSpec) => ScriptedRun | { reject: Error } = () => runResult('') diff --git a/packages/lsp/lsp-local/README.md b/packages/lsp/lsp-local/README.md index 93d123fac7..661c27d332 100644 --- a/packages/lsp/lsp-local/README.md +++ b/packages/lsp/lsp-local/README.md @@ -24,7 +24,7 @@ The `servers` record key is the stable provider id reserved on `ctx.lsp`; each v |---|---|---| | `command` | (required) | Executable to spawn — absolute, or resolved on the child PATH at load. Launch uses no shell. | | `args` | `[]` | Arguments passed to the executable. | -| `env` | `{}` | Extra env merged on top of the credential-scrubbed ambient env (vars matching `KEY`/`SECRET`/`TOKEN` are not forwarded); an explicit `DSH_*` entry merges after the seam's scrub of ambient ones. | +| `env` | `{}` | Extra env merged on top of the credential-scrubbed ambient env (vars matching `KEY`/`PASSWORD`/`SECRET`/`TOKEN` are not forwarded); an explicit `DSH_*` entry merges after the seam's scrub of ambient ones. | | `extensionToLanguage` | (required) | Lowercase leading-dot extension → LSP language id (e.g. `{ '.ts': 'typescript' }`). | | `initializationOptions` | `null` | Static `initialize` options forwarded to the server. | | `configuration` | `null` | Static answer to every `workspace/configuration` item. | diff --git a/packages/pty/pty-local/README.md b/packages/pty/pty-local/README.md index 3b1112b025..5acc92853e 100644 --- a/packages/pty/pty-local/README.md +++ b/packages/pty/pty-local/README.md @@ -6,7 +6,7 @@ Persistent shell backend for `ctx.pty` over `ctx.subprocess.spawnTerminal`. It s ## Plugin (`pty-local`) -The plugin injects `pty`, `sandboxPolicy`, and `subprocess`, then registers the configured backend type (`shell`). `danger-full-access` starts the shell directly without requiring a sandbox provider; confined modes require a same-world `ctx.sandbox` and wrap the exact shell argv through it, failing before spawn when none is mounted. The effective session mode is resolved at spawn. A change to a different effective mode is rejected before its `sandbox/mode` event commits while that owner has an open PTY or a spawn in progress; the fence is attached to the exact owner and therefore outlives a provider reload that retains existing sessions. Wait for creation to settle and close the sessions before changing modes, so a terminal opened with wider access cannot survive a downgrade. +The plugin injects `pty`, `sandboxPolicy`, and `subprocess`, then registers the configured backend type (`shell`). `danger-full-access` starts the shell directly without requiring a sandbox provider; confined modes require a same-world `ctx.sandbox` and wrap the exact shell argv through it, failing before spawn when none is mounted. At spawn, one `ctx.sandboxPolicy.resolve({ session })` call supplies both the effective mode and the session workspace root; the same root is the default shell cwd when the caller omits one. A change to a different effective mode is rejected before its `sandbox/mode` event commits while that owner has an open PTY or a spawn in progress; the fence is attached to the exact owner and therefore outlives a provider reload that retains existing sessions. Wait for creation to settle and close the sessions before changing modes, so a terminal opened with wider access cannot survive a downgrade. Readiness combines a foreground-verified private bash prompt marker, provider-reported foreground stdin-wait facts, silence fallback, and absolute timeout. A marker is not ready until the printable tail after the latest owned marker exactly equals the controlled `PS1`, including when the OSC marker and prompt are split across data callbacks; echoed input or output following an earlier prompt therefore cannot settle the current send. Prompt and silence evidence collected before the provider write, including while pre-write foreground inspection is pending, is discarded at the write boundary. When bash prints the marker before the terminal provider publishes its return to the foreground process group, polling retains the candidate for `handoffGraceMs` past the ordinary silence bound so a coincident handoff can win. An interactive child that inherits `PROMPT_COMMAND` therefore cannot suppress inferred-idle readiness until the absolute timeout. Unknown foreground state is never a positive exact-idle signal. A foreground group's stdin wait that existed before a send is likewise not post-write readiness: the same group must be observed outside that wait before a later wait can settle the send, while a changed foreground group is new evidence. During unpublished startup, a fallback requires observed output; zero-output silence cannot publish an empty session, and timeout rejects the spawn. Cancellation closes the unpublished shell and rejects with the caller's exact abort reason; `PtyBackendCleanupError` separately preserves a cleanup failure. The caller's signal is forwarded for terminal allocation and readiness initialization; after publication the handle owns its lifetime. Incomplete terminal-control sequences are bounded by `maxReadBytes` and discarded through their terminator after crossing that limit; malformed UTF-8 terminal output uses replacement characters, and a trailing carriage return is carried across callbacks so split CRLF becomes one newline. @@ -14,19 +14,19 @@ Send cancellation marks queued input as canceled before asking the terminal hand ## Model Experience -### Indirect consumer +### Current file policy and indirect consumer #### What the model sees -Nothing directly. Through `@deepseek-ai/dsh-tool-pty`, the model may receive bounded MOTD, send deltas, scrollback pages, readiness reasons, and cleanup errors. +The policy owner contributes capability-neutral `sandbox:policy` context. Through `@deepseek-ai/dsh-tool-pty` or another PTY consumer, the model may also receive bounded MOTD, send deltas, scrollback pages, readiness reasons, and cleanup errors. #### Token effect -None until a consumer returns bounded backend output. Retained PTY scrollback is not placed in model history by this package. +The current-policy clause is present while this backend is mounted. Retained PTY scrollback is not placed in model history until a consumer returns bounded output. #### KV Cache effect -No direct invalidation; the consumer owns prompts, schemas, and appended results. +A standing-policy change appends an owner-rendered superseding runtime-context snapshot after retained history; consumer results remain append-only. ## Known Limitations and Deferred Work diff --git a/packages/pty/pty-local/README.zh.md b/packages/pty/pty-local/README.zh.md index a23a2f3338..353c2a4bda 100644 --- a/packages/pty/pty-local/README.zh.md +++ b/packages/pty/pty-local/README.zh.md @@ -6,7 +6,7 @@ ## 插件(`pty-local`) -该插件注入 `pty`、`sandboxPolicy` 和 `subprocess`,然后注册所配置的后端类型(`shell`)。`danger-full-access` 无需沙箱提供方即可直接启动 shell;受限模式要求同一执行世界中存在 `ctx.sandbox`,并通过它包装确切的 shell argv,未挂载时会在 spawn 前失败。系统在 spawn 时解析会话的实际模式。当某个所有者存在开放的 PTY 或正在进行 spawn 时,如果配置变更会得到不同的实际模式,系统会在对应 `sandbox/mode` 事件提交前拒绝该变更。该限制绑定到确切所有者,因此即使提供方重新加载并保留现有会话,它仍然有效。更改模式前,请等待创建完成并关闭会话,避免以更宽权限打开的终端在权限降级后继续存在。 +该插件注入 `pty`、`sandboxPolicy` 和 `subprocess`,然后注册所配置的后端类型(`shell`)。`danger-full-access` 无需沙箱提供方即可直接启动 shell;受限模式要求同一执行世界中存在 `ctx.sandbox`,并通过它包装确切的 shell argv,未挂载时会在 spawn 前失败。Spawn 时,一次 `ctx.sandboxPolicy.resolve({ session })` 调用会同时给出实际模式与会话工作区根目录;调用方省略 cwd 时,同一根目录也是 shell 的默认 cwd。当某个所有者存在开放的 PTY 或正在进行 spawn 时,如果配置变更会得到不同的实际模式,系统会在对应 `sandbox/mode` 事件提交前拒绝该变更。该限制绑定到确切所有者,因此即使提供方重新加载并保留现有会话,它仍然有效。更改模式前,请等待创建完成并关闭会话,避免以更宽权限打开的终端在权限降级后继续存在。 就绪检测结合以下机制:由前台状态验证的私有 bash 提示符标记、提供方报告的前台 stdin 等待事实、静默回退和绝对超时。只有最新自有标记之后的可打印尾部与受控 `PS1` 完全相等,标记才算就绪;即使 OSC 标记和提示符被拆到多个数据回调中也一样。因此,较早提示符之后的回显输入或输出无法使当前 send 完成。提供方写入前收集的提示符与静默证据,包括写入前前台检查仍在等待时收集的证据,都会在写入边界丢弃。如果 bash 在终端提供方发布其重新取得前台进程组的状态前打印标记,轮询会在普通静默上限之后再保留该候选状态 `handoffGraceMs`,使恰好同时发生的前台交接有机会胜出。因此,继承 `PROMPT_COMMAND` 的交互式子进程无法一直抑制推断空闲就绪直至绝对超时。未知的前台状态绝不会作为精确空闲的正向信号。同样,一次 send 之前就已存在的前台进程组 stdin 等待并不代表写入后就绪:必须先观察到同一进程组脱离该等待,之后再次进入等待才能使该次 send 完成;前台进程组发生变化则构成新的证据。尚未发布的启动过程中,回退路径要求已经观察到输出;零输出静默不能发布空会话,超时则拒绝 spawn。取消操作会关闭尚未发布的 shell,并以调用方提供的确切中止原因拒绝;`PtyBackendCleanupError` 会单独保留清理失败。调用方的 signal 会转发给终端分配与就绪初始化;发布后,句柄负责其生命周期。未完成的终端控制序列受 `maxReadBytes` 限制;超过上限后,系统会丢弃内容直到其终止符。格式错误的 UTF-8 终端输出使用替换字符;末尾的回车会跨回调保留,使拆分的 CRLF 合并为一个换行。 @@ -14,19 +14,19 @@ ## 模型体验 -### 间接消费方 +### 当前文件策略与间接消费方 #### 模型看到的内容 -没有直接可见内容。模型通过 `@deepseek-ai/dsh-tool-pty` 可能收到有界的 MOTD、发送增量、scrollback 页、就绪原因和清理错误。 +策略归属方会贡献与具体能力无关的 `sandbox:policy` 上下文。模型通过 `@deepseek-ai/dsh-tool-pty` 或其他 PTY 消费方还可能收到有界的 MOTD、发送增量、scrollback 页、就绪原因和清理错误。 #### Token 影响 -消费方返回有界的后端输出前没有影响。此包(package)不会把保留的 PTY scrollback 放入模型历史。 +装载该后端期间,当前策略子句会一直存在。消费方返回有界输出前,保留的 PTY scrollback 不会进入模型历史。 #### KV Cache 影响 -不会直接使 KV Cache 失效;提示词、schema 与追加结果由消费方负责。 +常驻策略发生变化时,会在保留的历史之后追加一份由归属方渲染、取代先前状态的运行时上下文快照;消费方结果保持仅追加。 ## 已知限制与暂缓事项 diff --git a/packages/pty/pty-local/tests/index.spec.ts b/packages/pty/pty-local/tests/index.spec.ts index e6c4fde452..9f10a2cde8 100644 --- a/packages/pty/pty-local/tests/index.spec.ts +++ b/packages/pty/pty-local/tests/index.spec.ts @@ -3,7 +3,7 @@ import { PassThrough } from 'node:stream' import { Context } from 'cordis' import Loader from '@cordisjs/plugin-loader' import SessionStore, { Session, SessionId } from '@deepseek-ai/dsh-session' -import AgentRegistry, { type Agent } from '@deepseek-ai/dsh-agent' +import AgentRegistry, { Inbox, type Agent } from '@deepseek-ai/dsh-agent' import SandboxProvider from '@deepseek-ai/dsh-sandbox' import type { ConfinedArgv, SandboxPolicy } from '@deepseek-ai/dsh-sandbox' import SandboxPolicyService, { setSandboxMode } from '@deepseek-ai/dsh-sandbox-policy' @@ -22,7 +22,7 @@ import type { class EmptySandbox extends SandboxProvider { confine(_argv: readonly string[], _policy: SandboxPolicy): ConfinedArgv { - return { argv: [], enforcement: 'full', denialSignatures: [], runnerFailureSignatures: [] } + return { argv: [], enforcement: 'full', denialSignatures: [], runnerFailureRules: [] } } } @@ -31,7 +31,7 @@ class RecordingSandbox extends SandboxProvider { confine(argv: readonly string[], policy: SandboxPolicy): ConfinedArgv { this.calls.push({ argv, policy }) - return { argv: ['/sandbox', '--', ...argv], enforcement: 'full', denialSignatures: [], runnerFailureSignatures: [] } + return { argv: ['/sandbox', '--', ...argv], enforcement: 'full', denialSignatures: [], runnerFailureRules: [] } } } @@ -46,9 +46,15 @@ function config(): ResolvedConfig { function agent(ctx: Context): Agent { const id = SessionId('agent') + const session = Session.create(id, undefined, { version: 0, id, createdAt: 0 }) return { - id, options: {}, session: new Session(id), status: 'idle', acceptsNextStep: false, ctx, - followup: () => {}, steer: () => {}, inject: () => {}, send: () => {}, cancel() {}, whenIdle: () => Promise.resolve(), + id, options: {}, session, inbox: new Inbox(session, { inserted: () => {}, discarded: () => {}, claimed: () => {} }), + status: 'idle', + ctx, + send: () => {}, + followup: () => {}, steer: () => {}, inject: () => {}, cancel() {}, + runMaintenance: task => task(new AbortController().signal), + whenIdle: () => Promise.resolve(), } } @@ -331,7 +337,7 @@ describe('pty-local plugin shape', () => { const session = ctx.sessions.create(SessionId('unowned-mode')) expect(() => { - session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) + session.append('turn/start', { turn: 1 }) }).not.toThrow() expect(() => { setSandboxMode(session, 'read-only') }).not.toThrow() }) @@ -348,8 +354,13 @@ describe('pty-local plugin shape', () => { const session = ctx.sessions.create(SessionId('mode-owner')) const ownerFiber = await ctx.plugin(() => {}) const owner: Agent = { - id: session.id, options: {}, session, status: 'idle', acceptsNextStep: false, ctx: ownerFiber.ctx, - followup: () => {}, steer: () => {}, inject: () => {}, send: () => {}, cancel() {}, whenIdle: () => Promise.resolve(), + id: session.id, options: {}, session, inbox: new Inbox(session, { inserted: () => {}, discarded: () => {}, claimed: () => {} }), + status: 'idle', + ctx: ownerFiber.ctx, + send: () => {}, + followup: () => {}, steer: () => {}, inject: () => {}, cancel() {}, + runMaintenance: task => task(new AbortController().signal), + whenIdle: () => Promise.resolve(), } ctx.agents.register(owner) const providerFiber = await registerStubLocalBackend(ctx, () => stubLocalSession()) @@ -358,7 +369,7 @@ describe('pty-local plugin shape', () => { const unrelated = ctx.sessions.create(SessionId('unrelated-mode')) expect(() => { setSandboxMode(unrelated, 'read-only') }).not.toThrow() expect(() => { - session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) + session.append('turn/start', { turn: 1 }) }).not.toThrow() expect(() => { setSandboxMode(session, 'danger-full-access') }).not.toThrow() @@ -392,8 +403,13 @@ describe('pty-local plugin shape', () => { const session = ctx.sessions.create(SessionId('pending-mode-owner')) const ownerFiber = await ctx.plugin(() => {}) const owner: Agent = { - id: session.id, options: {}, session, status: 'idle', acceptsNextStep: false, ctx: ownerFiber.ctx, - followup: () => {}, steer: () => {}, inject: () => {}, send: () => {}, cancel() {}, whenIdle: () => Promise.resolve(), + id: session.id, options: {}, session, inbox: new Inbox(session, { inserted: () => {}, discarded: () => {}, claimed: () => {} }), + status: 'idle', + ctx: ownerFiber.ctx, + send: () => {}, + followup: () => {}, steer: () => {}, inject: () => {}, cancel() {}, + runMaintenance: task => task(new AbortController().signal), + whenIdle: () => Promise.resolve(), } ctx.agents.register(owner) const gate = Promise.withResolvers() diff --git a/packages/subprocess/subprocess-local/README.i18n.yaml b/packages/subprocess/subprocess-local/README.i18n.yaml index 87a95c3170..22d971b251 100644 --- a/packages/subprocess/subprocess-local/README.i18n.yaml +++ b/packages/subprocess/subprocess-local/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/subprocess/subprocess-local/README.md -README.md: b901a69d6cfd45a084711ba0d32e555c481fd626 -README.zh.md: 5723dc99ff955c5f4b07ea96e7d2134100d9fd6a +README.md: 087ca24a3207cb8cb1568769a462fbbb010aaa35 +README.zh.md: 4d400906f71b653ce2be95a22751fd9893f447bd diff --git a/packages/subprocess/subprocess-local/README.md b/packages/subprocess/subprocess-local/README.md index 9e3e8f7f2d..087ca24a32 100644 --- a/packages/subprocess/subprocess-local/README.md +++ b/packages/subprocess/subprocess-local/README.md @@ -6,9 +6,9 @@ Local implementation of the [`@deepseek-ai/dsh-subprocess`](../subprocess/README ## Behavior (and where it came from) -- **Detached process trees with platform-correct signalling** — POSIX children are spawned `detached` (own process group) and signalled by negative pgid with a direct-child fallback; Windows terminates the tree via `taskkill /PID /T /F` (injectable for tests). `terminate()` — the handle's only termination verb — sends SIGTERM then SIGKILL after the spec's grace (OpenCode's escalation; pipelines and subshells die with the parent) and is a no-op once the tree is gone; `waitForExit()` polls whole-tree liveness so consumer teardown confirms real quiescence. After the leader exits, still-open pipes receive the same bounded drain grace so a surviving descendant cannot hold the outcome open indefinitely. ESRCH is tolerated; daemons that re-parent away from the group can still survive — the same caveat as the surveyed tools. +- **Detached process trees with platform-correct signalling** — POSIX children are spawned `detached` (own process group) and signalled by negative pgid with a direct-child fallback; Windows terminates the tree via `taskkill /PID /T /F`. `terminate()` — the handle's only termination verb — sends SIGTERM then SIGKILL after the spec's grace (OpenCode's escalation; pipelines and subshells die with the parent) and is a no-op once the tree is gone; `waitForExit()` polls whole-tree liveness so consumer teardown confirms real quiescence. After the leader exits, still-open pipes receive the same bounded drain grace so a surviving descendant cannot hold the outcome open indefinitely. ESRCH is tolerated; daemons that re-parent away from the group can still survive — the same caveat as the surveyed tools. - **Per-stream dispositions** — `'pipe'` hands the raw stream to the caller untouched (protocol framing stays consumer-owned); `'inherit'` passes the parent descriptor through; collect mode keeps the in-memory TAIL beyond its cap (errors and results cluster at the end — pi/OpenCode rationale) while the FULL stream is appended to a private temp file when a spill cap is configured — omitting `spill` keeps only the tail, the diagnostic shape. A stream larger than the spill cap discards its now-incomplete spill and returns only the marked truncated tail; spill fds are sealed at settlement, and a failed final close withholds the path rather than advertising an incomplete file. Spill files are `0600` with random names under a lazily-created `0700` per-process directory. -- **Credential scrub + explicit merge** — `process.env` minus credential-shaped vars (`*KEY*`/`*SECRET*`/`*TOKEN*`) and all ambient `DSH_*` names; the spec's explicit `env` merges after that scrub with no namespace validation, so a deliberately supplied credential or current `DSH_*` fact wins while stale nested-harness identity cannot leak in ambiently. Supplied stdin is written and closed; otherwise fd 0 is `/dev/null`. See the [stdin/env Agent Note](../../../.agents/notes/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-surface.md) and [managed environment Agent Note](../../../.agents/notes/implemented/feature/2026-07-10-agent-session-identity-and-log-location.md). +- **Credential scrub + explicit merge** — `process.env` minus credential-shaped vars (`*KEY*`/`*PASSWORD*`/`*SECRET*`/`*TOKEN*`) and all ambient `DSH_*` names; the spec's explicit `env` merges after that scrub with no namespace validation, so a deliberately supplied credential or current `DSH_*` fact wins while stale nested-harness identity cannot leak in ambiently. Supplied stdin is written and closed; otherwise fd 0 is `/dev/null`. See the [stdin/env Agent Note](../../../.agents/notes/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-surface.md) and [managed environment Agent Note](../../../.agents/notes/implemented/feature/2026-07-10-agent-session-identity-and-log-location.md). - **Offset-based reads** — collect-mode readers return deltas in whole-stream byte coordinates; the service never holds a cursor, so consumer-owned cursors (the bash background read path) and full-stream re-reads coexist, before and after settlement. - **Executable lookup** — `resolveExecutable` checks absolute files or searches the scrubbed effective PATH with platform-aware executable extensions; relative paths containing separators are rejected at the seam, and relative PATH entries resolve from the host process cwd. - **Terminal-process ownership** — `spawnTerminal` allocates `node-pty`, bridges UTF-8 terminal text, inspects and signals the current foreground process group, and exposes one awaited termination operation that sweeps descendants before and after terminating the top-level shell. Each foreground inspection retains exact identities from the rooted tree; Linux also enumerates the POSIX session after its leader exits. A previously observed macOS descendant and any same-session Linux member therefore remain fenced after reparenting, while pid/start identity prevents cleanup from following PID reuse. The higher PTY backend owns prompt readiness, buffers, and model-facing operations. @@ -24,10 +24,10 @@ No direct invalidation; the named consumers own any request-prefix changes. ## Known Limitations and Deferred Work -- **Windows tree support is best-effort and untested in CI** — termination routes through `taskkill /PID /T /F` with all outcomes contained (absent tree, races, missing binary), and liveness falls back to the direct-child boundary; the suites cover the routing through an injected runner only, and `packages/subprocess/*` is excluded from the Windows test matrix. +- **Windows tree support is best-effort** — termination routes through `taskkill /PID /T /F` with all outcomes contained (absent tree, races, missing binary), and liveness falls back to the direct-child boundary. - **Terminal process inspection is Linux/macOS only** — the terminal primitive fails when its inspector has no supported platform implementation; Linux exact probes cover x64 and arm64, while macOS uses `ps` snapshots. - **A daemonized terminal descendant can still escape the observable boundary** — on macOS, a child that reparents before any foreground-inspection snapshot is no longer discoverable from the `node-pty` root; on Linux, a child that calls `setsid` leaves both the tree and owned terminal session. The local provider does not add a continuous process-table monitor. -- **The credential scrub is a name heuristic** — `*KEY*`/`*SECRET*`/`*TOKEN*` only; differently-named secrets (e.g. `*PASSWORD*`) pass through, and a whitelist for over-scrubbed vars is noted future work. +- **The credential scrub is a name heuristic** — `*KEY*`/`*PASSWORD*`/`*SECRET*`/`*TOKEN*` only; differently-named secrets (e.g. `*PASSPHRASE*`) pass through, and a whitelist for over-scrubbed vars is noted future work. - **Completed spill files are not deleted** — bounded full-output recovery files (and the private per-process spill dir) accumulate under the OS tmpdir until something external cleans them; oversize incomplete spills are discarded and deletion is attempted immediately, but a cleanup failure can leave a bounded file behind. The raw process handling lives in `src/spawn.ts`; `src/index.ts` is the service wiring. diff --git a/packages/subprocess/subprocess-local/README.zh.md b/packages/subprocess/subprocess-local/README.zh.md index 5723dc99ff..4d400906f7 100644 --- a/packages/subprocess/subprocess-local/README.zh.md +++ b/packages/subprocess/subprocess-local/README.zh.md @@ -6,9 +6,9 @@ ## 行为(以及设计来源) -- **以适合平台的方式发送信号的 detached 进程树**:POSIX 子进程使用 `detached` spawn(拥有独立进程组),信号以负 pgid 发送并以直接子进程作为回退;Windows 通过 `taskkill /PID /T /F` 终止进程树(可为测试注入)。`terminate()`(句柄唯一的终止操作)先发送 SIGTERM,经过 spec 的宽限期后再发送 SIGKILL(沿用 OpenCode 的升级策略;流水线与子 shell 会随父进程一起结束),进程树消亡后为空操作;`waitForExit()` 轮询整棵进程树的存活状态,使消费方的拆卸能确认真正的完全停稳。组长进程退出后,仍然打开的管道也只获得同样有界的排空宽限期,因此存活的后代进程无法无限期地拖住结果不结算。系统会容忍 ESRCH;重新指定父进程并脱离该组的 daemon 仍可能存活,这与所调研工具的局限相同。 +- **以适合平台的方式发送信号的 detached 进程树**:POSIX 子进程使用 `detached` spawn(拥有独立进程组),信号以负 pgid 发送并以直接子进程作为回退;Windows 通过 `taskkill /PID /T /F` 终止进程树。`terminate()`(句柄唯一的终止操作)先发送 SIGTERM,经过 spec 的宽限期后再发送 SIGKILL(沿用 OpenCode 的升级策略;流水线与子 shell 会随父进程一起结束),进程树消亡后为空操作;`waitForExit()` 轮询整棵进程树的存活状态,使消费方的拆卸能确认真正的完全停稳。组长进程退出后,仍然打开的管道也只获得同样有界的排空宽限期,因此存活的后代进程无法无限期地拖住结果不结算。系统会容忍 ESRCH;重新指定父进程并脱离该组的 daemon 仍可能存活,这与所调研工具的局限相同。 - **按流划分的处置方式**:`'pipe'` 把原始流原样交给调用方(协议分帧仍归消费方所有);`'inherit'` 直通父进程的描述符;收集模式(collect)在输出超过上限后于内存中保留尾部(错误与结果通常聚集在末尾,沿用 pi/OpenCode 的理由),并在配置了 spill 上限时把完整流追加到一个私有临时文件;省略 `spill` 则只保留用于诊断的尾部。某条流大于 spill 上限时,会丢弃已不完整的 spill,仅返回带截断标记的尾部;spill 文件描述符在结算时封存,最终关闭失败时则不公布路径,以免声称存在不完整的文件。spill 文件权限为 `0600`、名称随机,位于按需创建、权限为 `0700` 的每进程目录之下。 -- **凭据清除 + 显式合并**:以 `process.env` 为基础,移除形似凭据的变量(`*KEY*`/`*PASSWORD*`/`*SECRET*`/`*TOKEN*`)和所有环境中已有的 `DSH_*` 名称;spec 的显式 `env` 在该清除之后合并且不做命名空间校验,因此有意提供的凭据或当前 `DSH_*` 事实会胜出,而陈旧的嵌套 harness 身份无法从环境中隐式漏入。提供的 stdin 会被写入后关闭;否则 fd 0 指向 `/dev/null`。参见 [stdin/env Agent Note(agent 决策记录)](../../../.agents/notes/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-surface.md)与[受管环境 Agent Note](../../../.agents/notes/implemented/feature/2026-07-10-agent-session-identity-and-log-location.md)。 +- **凭据清除 + 显式合并**:以 `process.env` 为基础,移除形似凭据的变量(`*KEY*`/`*PASSWORD*`/`*SECRET*`/`*TOKEN*`)和所有环境中已有的 `DSH_*` 名称;spec 的显式 `env` 在该清除之后合并且不做命名空间校验,因此有意提供的凭据或当前 `DSH_*` 事实会胜出,而陈旧的嵌套 harness 身份无法从环境中隐式漏入。提供的 stdin 会被写入后关闭;否则 fd 0 指向 `/dev/null`。参见 [stdin/env Agent Note](../../../.agents/notes/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-surface.md)与[受管环境 Agent Note](../../../.agents/notes/implemented/feature/2026-07-10-agent-session-identity-and-log-location.md)。 - **基于偏移量的读取**:收集模式的读取器按完整流的字节坐标返回增量;服务自身从不持有游标,因此消费方自有的游标(bash 的后台读取路径)与完整流重读可以共存,结算前后皆然。 - **可执行文件查找**:`resolveExecutable` 检查绝对文件,或根据平台可执行文件扩展名在清理后的有效 PATH 中搜索;含分隔符的相对路径在接缝处被拒绝,相对 PATH 条目从宿主进程 cwd 解析。 - **终端进程所有权**:`spawnTerminal` 分配 `node-pty`,桥接 UTF-8 终端文本,检查当前前台进程组并向其发送信号,还会公开一项须等待的终止操作,在终止顶层 shell 前后清理后代进程。每次前台检查都会保留根进程树中的精确身份;Linux 还会在 POSIX 会话 leader 退出后枚举该会话。因此,之前观察到的 macOS 后代以及同会话 Linux 成员在重新设定父进程后仍受围栏保护,pid/start 身份则防止清理跟随 PID 复用。上层 PTY 后端负责提示符就绪、缓冲区与面向模型的操作。 @@ -24,7 +24,7 @@ ## 已知限制与暂缓事项 -- **Windows 进程树支持仅为尽力而为,且未经 CI 测试**:终止经由 `taskkill /PID /T /F` 完成,所有结果都被就地吸收,不向外抛出(进程树已不存在、竞态、二进制缺失),存活探测则回退到直接子进程边界;测试套件只通过注入的运行器覆盖这条路由,且 `packages/subprocess/*` 被排除在 Windows 测试矩阵之外。 +- **Windows 进程树支持仅为尽力而为**:终止经由 `taskkill /PID /T /F` 完成,所有结果都被就地吸收,不向外抛出(进程树已不存在、竞态、二进制缺失),存活探测则回退到直接子进程边界。 - **终端进程检查仅支持 Linux/macOS**:检查器没有受支持的平台实现时,终端原语会失败;Linux 精确探针覆盖 x64 与 arm64,macOS 则使用 `ps` 快照。 - **守护化的终端后代仍可能逃出可观察边界**:在 macOS 上,子进程如果在任何前台检查快照之前重新设定父进程,将无法再从 `node-pty` 根进程发现;在 Linux 上,调用 `setsid` 的子进程会同时离开进程树与自有终端会话。本地提供方不会新增持续进程表监视器。 - **凭据清除依赖名称启发式规则**:只匹配 `*KEY*`/`*PASSWORD*`/`*SECRET*`/`*TOKEN*`;名称不同的 secret(例如 `*PASSPHRASE*`)会继续传递,对误删变量引入白名单属于已记录的后续工作。 diff --git a/packages/subprocess/subprocess-local/src/spawn.ts b/packages/subprocess/subprocess-local/src/spawn.ts index 9fa70250dc..5b977cacc1 100644 --- a/packages/subprocess/subprocess-local/src/spawn.ts +++ b/packages/subprocess/subprocess-local/src/spawn.ts @@ -15,6 +15,7 @@ import { tmpdir } from 'node:os' import { join } from 'node:path' import { setTimeout as sleepMs } from 'node:timers/promises' import { scrubbedParentEnv } from '@deepseek-ai/dsh-subprocess' +import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout' import type { CollectedOutput, SubprocessCollect, @@ -27,16 +28,16 @@ import { linuxProcessGroupHasLiveMembers } from './process-inspector.ts' /** * Build a child environment: explicit caller entries override the scrubbed - * parent base using the target platform's environment-key semantics, so a - * deliberately supplied credential or current `DSH_*` fact wins over the - * scrub that dropped its ambient namesake. - * @param extra - explicit caller entries merged after the scrubbed parent. + * parent base using the target platform's environment-key semantics. A string + * deliberately restores or overrides an entry; an explicit `undefined` + * tombstone removes an ordinary ambient entry. + * @param extra - explicit caller entries and tombstones, merged after the scrub. * @returns the environment to hand to `spawn` for the child process. */ -export function childEnv(extra?: Readonly>): NodeJS.ProcessEnv { +export function childEnv(extra?: Readonly): NodeJS.ProcessEnv { const env = scrubbedParentEnv() if (process.platform !== 'win32') return { ...env, ...extra } - let entries = Object.entries(env) + let entries: [string, string | undefined][] = Object.entries(env) for (const [key, value] of Object.entries(extra ?? {})) { const normalized = key.toUpperCase() entries = entries.filter(([inherited]) => inherited.toUpperCase() !== normalized) @@ -310,8 +311,12 @@ function signalTree( * @param spec - fully resolved argv, cwd, stdio, grace, cancellation, environment. * @param internals - test-only spill-directory, platform, and taskkill overrides. * @returns live subprocess handle. + * @throws when `graceMs` cannot be represented by one Node timer. */ export function spawnSubprocess(spec: SubprocessSpawnSpec, internals: SpawnInternals = {}): SubprocessHandle { + if (!Number.isFinite(spec.graceMs) || spec.graceMs <= 0 || spec.graceMs > MAX_TIMER_DELAY_MS) { + throw new Error(`subprocess graceMs must be a positive finite number no greater than ${MAX_TIMER_DELAY_MS}`) + } const spillDir = internals.spillDir ?? privateSpillDir() const platform = internals.platform ?? process.platform const taskkill = internals.taskkill ?? taskkillProcessTree @@ -354,7 +359,9 @@ export function spawnSubprocess(spec: SubprocessSpawnSpec, internals: SpawnInter const stdoutCollector = collectStream(outMode, child.stdout, 'stdout') const stderrCollector = collectStream(errMode, child.stderr, 'stderr') - let graceTimer: NodeJS.Timeout | undefined + let graceTimer: ReturnType | undefined + let treeExitObserved = false + let treeExitObservation: Promise | undefined let settled = false // Failed spawns use pid -1 so signalling remains a no-op. @@ -362,6 +369,9 @@ export function spawnSubprocess(spec: SubprocessSpawnSpec, internals: SpawnInter /** Whether the detached tree's root (or POSIX group) is still alive. */ const treeAlive = (): boolean => { + /* v8 ignore next -- only a timer callback already queued when the observer settles can enter here; + the guard is the final defense against probing an id after its tree was confirmed absent. */ + if (treeExitObserved) return false if (pid <= 0) return false if (platform === 'win32') { // Windows has no group-liveness probe; the direct child's exit is the @@ -389,19 +399,40 @@ export function spawnSubprocess(spec: SubprocessSpawnSpec, internals: SpawnInter } } + /** + * Start or reuse the handle's single whole-tree exit observer. The first + * confirmed absence is a permanent no-more-signals boundary: it cancels a + * pending escalation before this process-group id can be reused. + */ + const observeTreeExit = (): Promise => { + treeExitObservation ??= (async () => { + while (treeAlive()) await sleepTick() + treeExitObserved = true + if (graceTimer !== undefined) clearTimeout(graceTimer) + graceTimer = undefined + })() + return treeExitObservation + } + // The escalation's tier primitive (not on the handle — terminate() is the // only consumer-facing termination verb). Guards on TREE liveness, not // outcome settlement: a TERM-trapping helper can outlive the settled direct // child and must stay signalable, while a fully-dead tree (possible pid // reuse) must not be re-signalled by a later tier. const kill = (sig: NodeJS.Signals): void => { + /* v8 ignore next -- the shared exit observer cancels the ordinary dead-tree timer; + this remains the timer/death race guard and cannot be staged deterministically. */ if (!treeAlive()) return signalTree(platform, pid, sig, child, taskkill) } const terminate = (): void => { - if (graceTimer !== undefined) return // escalation already in flight - if (!treeAlive()) return + if (treeExitObserved || graceTimer !== undefined) return + // Observe from the first termination tier onward, even when inherited + // pipes delay `done` and no consumer has begun its own teardown wait. + void observeTreeExit() + // oxlint-disable-next-line typescript/no-unnecessary-condition -- observer can record absence before its first await. + if (treeExitObserved) return kill('SIGTERM') // The escalation must survive direct-child settlement — the leader dying // does not mean the tree died — so settle does not clear this timer, and @@ -423,7 +454,7 @@ export function spawnSubprocess(spec: SubprocessSpawnSpec, internals: SpawnInter } const done = new Promise((resolve, reject) => { - let pipeDrainTimer: NodeJS.Timeout | undefined + let pipeDrainTimer: ReturnType | undefined const settle = (exitCode: number | null, signal: NodeJS.Signals | null): void => { if (settled) return settled = true @@ -446,7 +477,9 @@ export function spawnSubprocess(spec: SubprocessSpawnSpec, internals: SpawnInter // A surviving descendant that inherited a pipe must not hold the // outcome open indefinitely: after exit, the same bounded grace that // governs kills also bounds the close wait. - pipeDrainTimer = setTimeout(() => { settle(exitCode, signal) }, spec.graceMs) + pipeDrainTimer = setTimeout(() => { + settle(exitCode, signal) + }, spec.graceMs) }) child.on('close', settle) function cleanup(): void { @@ -458,11 +491,23 @@ export function spawnSubprocess(spec: SubprocessSpawnSpec, internals: SpawnInter }) const waitForExit = async (signal?: AbortSignal): Promise => { - while (treeAlive()) { - if (signal?.aborted) return false - await sleepTick() + const observed = observeTreeExit() + if (treeExitObserved) return true + if (signal?.aborted) return false + if (signal === undefined) { + await observed + return true + } + const aborted = Promise.withResolvers() + const onAbort = (): void => { aborted.resolve(false) } + signal.addEventListener('abort', onAbort, { once: true }) + /* v8 ignore next -- closes the event-loop race between the preceding aborted check and listener registration. */ + if (signal.aborted) onAbort() + try { + return await Promise.race([observed.then(() => true), aborted.promise]) + } finally { + signal.removeEventListener('abort', onAbort) } - return true } return { diff --git a/packages/subprocess/subprocess-local/tests/spawn.spec.ts b/packages/subprocess/subprocess-local/tests/spawn.spec.ts index f28f741992..6bac46ac16 100644 --- a/packages/subprocess/subprocess-local/tests/spawn.spec.ts +++ b/packages/subprocess/subprocess-local/tests/spawn.spec.ts @@ -280,18 +280,18 @@ describe('spawnSubprocess', () => { it('does not wait for a Linux group that has only zombie members', async () => { const pidFile = join(spillDir, `zombie-group-${Date.now()}.pid`) - let hasLiveMembers = false const running = spawnSubprocess(spec(`sleep 60 & echo $! > ${pidFile}; echo leader-done`, { graceMs: 100 }), { platform: 'linux', - linuxProcessGroupHasLiveMembers: () => hasLiveMembers, + linuxProcessGroupHasLiveMembers: () => false, }) const descendant = await waitForPidFile(pidFile) try { await running.done await expect(running.waitForExit()).resolves.toBe(true) } finally { - hasLiveMembers = true - running.terminate() + // The confirmed-absent verdict is a permanent no-more-signals boundary, + // so terminate() must stay inert here; reap the live survivor directly. + process.kill(descendant, 'SIGKILL') await waitGone(descendant) } }) diff --git a/packages/typert/README.i18n.yaml b/packages/typert/README.i18n.yaml index 5e93edd997..cf44d9aace 100644 --- a/packages/typert/README.i18n.yaml +++ b/packages/typert/README.i18n.yaml @@ -1,6 +1,6 @@ # Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: -# pnpm run verify-translation-pairing --write packages/code-runtime/code-runtime-subprocess/README.md -README.md: 152921a1ed595676781aad170e3396f7ce613161 -README.zh.md: 3e8223c33c7ce85f342c9a36ab5a24c4d2a6cb94 +# pnpm run verify-translation-pairing --write packages/typert/README.md +README.md: ad9f843e48be0e3be85921ed8fd3ca4e2c327160 +README.zh.md: 0d4be016b766178b54f7e269583fa4200e3eb24b diff --git a/packages/ui/app-boot/tsdown.config.ts b/packages/ui/app-boot/tsdown.config.ts index 33db6be1b8..88492d7c26 100644 --- a/packages/ui/app-boot/tsdown.config.ts +++ b/packages/ui/app-boot/tsdown.config.ts @@ -1,6 +1,9 @@ import { defineConfig } from 'tsdown' -/** Bundle the host plugin and its dependency-free eval runner. */ +/** + * Embed Include while keeping Loader external so the built include tree and + * app host bind to one Loader peer. + */ export default defineConfig({ entry: ['lib/types/index.js', 'lib/types/invariant.js'], outDir: 'lib', @@ -10,4 +13,7 @@ export default defineConfig({ fixedExtension: false, dts: false, clean: false, + deps: { + alwaysBundle: ['@cordisjs/plugin-include'], + }, }) diff --git a/packages/util/atomic-write/README.i18n.yaml b/packages/util/atomic-write/README.i18n.yaml index efcd06f845..33e1f6014b 100644 --- a/packages/util/atomic-write/README.i18n.yaml +++ b/packages/util/atomic-write/README.i18n.yaml @@ -1,6 +1,6 @@ # Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: -# pnpm run verify-translation-pairing --write packages/code-runtime/code-runtime-subprocess/README.md -README.md: 897112a32fa3e776df756d4caaba97b617c6af9e -README.zh.md: d9ab3713640fe955d2b36a8f0674f77686e4791b +# pnpm run verify-translation-pairing --write packages/util/atomic-write/README.md +README.md: 2ff4abb6ac10d8b592ccd2056b4f1f92cc8518b0 +README.zh.md: 4284d06422564268bb9e31d1a1ed06ab5271e562 diff --git a/packages/util/atomic-write/src/runner-source.generated.ts b/packages/util/atomic-write/src/runner-source.generated.ts deleted file mode 100644 index 95da563dbb..0000000000 --- a/packages/util/atomic-write/src/runner-source.generated.ts +++ /dev/null @@ -1,6 +0,0 @@ -/** - * Generated dependency-free execution-world runner. - * Do not edit by hand; run `pnpm run gen-code-runtime-runner`. - */ - -export const CODE_RUNNER_SOURCE = "import{Buffer as e}from\"node:buffer\";import{spawn as t}from\"node:child_process\";import{createInterface as n}from\"node:readline\";import{inspect as r}from\"node:util\";import{Worker as i,isMainThread as a,parentPort as o,workerData as s}from\"node:worker_threads\";const c=Reflect.apply,l=Array.isArray,u=Buffer,d=Reflect.get(Buffer,`byteLength`),f=Object.create,p=Object.defineProperty,ee=Object.keys,te=String,ne=Reflect.get(String.prototype,`charCodeAt`),re=Reflect.get(String.prototype,`codePointAt`),ie=Reflect.get(String.prototype,`slice`);function m(e){let t=f(null);return t.value=e,t}function ae(e,t,n){let r=m(n);r.enumerable=!0,r.configurable=!0,r.writable=!0,p(e,t,r)}function h(e){return c(d,u,[e,`utf8`])}function g(e,t){ae(e,e.length,t)}function _(e){if(e.length===0)return;let t=e.length-1,n=e[t];return p(e,`length`,m(t)),n}function v(e,t){return c(ie,e,[t,t+(c(re,e,[t])>65535?2:1)])}function y(e){if(e.length===2)return 4;if(e===`\"`||e===`\\\\`)return 2;let t=c(ne,e,[0]);return t>=55296&&t<=57343?6:t<32?t===8||t===9||t===10||t===12||t===13?2:6:h(e)}function b(e,t){if(t<2)return;let n=2;for(let r=0;rt)return;r+=i.length}return n}function x(e,t){let n=0,r=e=>(n+=e,n<=t),i=[{kind:`value`,value:e}];for(let e=_(i);e!==void 0;e=_(i)){if(e.kind===`value`){let a=e.value;if(a===null){if(!r(4))return}else if(typeof a==`string`){let e=b(a,t-n);if(e===void 0)return;n+=e}else if(typeof a==`number`){if(!r(h(te(a))))return}else if(typeof a==`boolean`){if(!r(a?4:5))return}else if(l(a)){if(!r(2))return;a.length>0&&g(i,{kind:`array`,value:a,index:0})}else{if(!r(2))return;let e=ee(a);e.length>0&&g(i,{kind:`object`,value:a,keys:e,index:0})}continue}if(e.index>0&&!r(1))return;if(e.kind===`array`){let t=e.value[e.index];if(t===void 0)return;e.index+1t)break;n+=o,r+=a.length,i+=a.length}return r===e.length?e:c(ie,e,[0,r])}const se=Reflect.get(Function.prototype,`toString`),S=Reflect.get(Reflect,`apply`),C=Error,ce=Set,w=Array.isArray,le=Array.prototype,T=Number.isFinite,ue=Number.isSafeInteger,de=Object.create,E=Object.defineProperty,fe=Object.getOwnPropertyDescriptor,D=Object.getPrototypeOf,O=Object.hasOwn,k=Object.is,pe=Object.keys,A=Object.prototype,me=Reflect.get(A,`propertyIsEnumerable`),j=Reflect.ownKeys,he=Reflect.get(Set.prototype,`add`),ge=Reflect.get(Set.prototype,`delete`),_e=Reflect.get(Set.prototype,`has`);function M(e){let t=de(null);return t.value=e,t}function N(e,t,n){let r=M(n);r.enumerable=!0,r.configurable=!0,r.writable=!0,E(e,t,r)}function P(e,t){N(e,e.length,t)}function F(e){if(e.length===0)return;let t=e.length-1,n=e[t];return E(e,`length`,M(t)),n}function I(e,t){return S(_e,e,[t])}function L(e,t){S(he,e,[t])}function ve(e,t){S(ge,e,[t])}function R(e,t){let n=fe(e,`constructor`)?.value;if(typeof n!=`function`)return!1;try{return n.name===t&&n.prototype===e&&S(se,n,[])===`function ${t}() { [native code] }`}catch{return!1}}function z(e){return D(e)===null&&R(e,`Object`)}function B(e){let t=D(e);if(t===le)return!0;if(!w(t)||!R(t,`Array`))return!1;let n=D(t);return typeof n==`object`&&!!n&&z(n)}function V(e){let t=D(e);return t===null||t===A||typeof t==`object`&&z(t)}function ye(e){let t=j(e);for(let n=0;n{e.kind===`root`?n=t:e.kind===`array`?N(e.target,e.index,t):N(e.target,e.key,t)},i=[{kind:`visit`,value:e,destination:{kind:`root`}}];for(let e=F(i);e!==void 0;e=F(i)){if(e.kind===`leave`){ve(t,e.source);continue}if(e.kind===`array-item`){if(!O(e.source,e.index))return;P(i,{kind:`visit`,value:e.source[e.index],destination:{kind:`array`,target:e.target,index:e.index}});continue}if(e.kind===`object-property`){P(i,{kind:`visit`,value:e.source[e.key],destination:{kind:`object`,target:e.target,key:e.key}});continue}let n=e.value;if(n===null){r(e.destination,null);continue}if(typeof n==`boolean`||typeof n==`string`){r(e.destination,n);continue}if(typeof n==`number`){if(!T(n)||k(n,-0))return;r(e.destination,n);continue}if(typeof n!=`object`||I(t,n))return;if(w(n)){if(!B(n))return;let a=n.length;if(j(n).length!==a+1)return;let o=[];r(e.destination,o),L(t,n),P(i,{kind:`leave`,source:n});for(let e=a-1;e>=0;e--)P(i,{kind:`array-item`,source:n,index:e,target:o});continue}if(!V(n))return;let a=ye(n);if(a===void 0)return;let o={};r(e.destination,o),L(t,n),P(i,{kind:`leave`,source:n});for(let e=a.length-1;e>=0;e--){let t=a[e];if(t===void 0)return;P(i,{kind:`object-property`,source:n,key:t,target:o})}}return n}function U(e){let t=[],n=[e];for(let e=F(n);e!==void 0;e=F(n)){if(e===null||typeof e==`boolean`||typeof e==`number`||typeof e==`string`){P(t,e);continue}if(w(e)){P(t,{kind:`array`,length:e.length});for(let t=e.length-1;t>=0;t--){let r=e[t];if(r===void 0)throw new C(`cannot encode a sparse JSON array`);P(n,r)}continue}let r=pe(e);P(t,{kind:`object`,keys:r});for(let t=r.length-1;t>=0;t--){let i=r[t];if(i===void 0)throw new C(`cannot encode a missing JSON object key`);let a=e[i];if(a===void 0)throw new C(`cannot encode an undefined JSON object property`);P(n,a)}}return t}function be(e){if(!B(e)||j(e).length!==e.length+1)return!1;for(let t=0;t=0?{kind:`array`,length:e}:void 0}if(n.kind===`object`){if(t.length!==2||!W(t,`kind`)||!W(t,`keys`))return;let e=n.keys;if(!w(e)||!be(e))return;let r=new ce,i=[],a=e;for(let e=0;e{let t=n[n.length-1];if(!t)return i?!1:(r=e,i=!0,!0);if(t.index>=(t.kind===`array`?t.length:t.keys.length))return!1;if(t.kind===`array`)P(t.target,e);else{let n=t.keys[t.index];if(n===void 0)return!1;N(t.target,n,e)}return t.index+=1,!0};for(let e=0;ea)return;let e=[];i=e,n.length>0&&(o={kind:`array`,target:e,length:n.length,index:0})}else{if(n.keys.length>a)return;let e={};i=e,n.keys.length>0&&(o={kind:`object`,target:e,keys:n.keys,index:0})}}if(!a(i))return;for(o&&P(n,o);n.length>0;){let e=n[n.length-1];if(e===void 0||e.index<(e.kind===`array`?e.length:e.keys.length))break;F(n)}}return n.length===0?r:void 0}catch{return}}const G=Error,Ce=Object.create,we=Object.defineProperty;function Te(e,t,n){let r=Ce(null);r.enumerable=!0,r.value=n,we(e,t,r)}var Ee=class{bytes=2;entries=0;truncated=!1;sink;onLimit;maxBytes;constructor(e,t,n=()=>{}){this.maxBytes=e,this.sink=t,this.onLimit=n}push(e){if(this.truncated)return;let t=+(this.entries>0),n=this.maxBytes-this.bytes-t,r=b(e,n);if(r===void 0){this.truncated=!0;let r=oe(e,n);if(r.length>0){let e=b(r,n);if(e===void 0)throw new G(`worker output ledger produced an oversized log prefix`);this.bytes+=e+t,this.entries+=1,this.sink(r)}this.onLimit();return}this.bytes+=r+t,this.entries+=1,this.sink(e)}remainingOutputBytes(){return this.maxBytes-this.bytes}};const De=[`log`,`info`,`warn`,`error`,`debug`];function Oe(e){let t=e=>e.map(e=>typeof e==`string`?e:r(e,Ae)).join(` `),n=Object.create(null);for(let r of De)n[r]=(...n)=>{e.push(t(n))};return n}function ke(e,t){let n=t.write;return t.write=(t,...n)=>{e.push(typeof t==`string`?t:String(t));let r=[n[0],n[1]].find(e=>typeof e==`function`);return r&&queueMicrotask(()=>{r(null)}),!0},()=>{t.write=n}}const Ae={depth:4,maxArrayLength:100,maxStringLength:1e4};function je(e,t,n=t){if(e===void 0)return{};let r;try{r=H(e)}catch{r=void 0}return r===void 0?Ne(`invalid-output`,`program completion must be lossless JSON`,t,n):x(r,t)===void 0?Me(n):{value:U(r)}}function Me(e){return{error:{kind:`output-limit`,message:`outer output exceeded ${e} bytes`}}}function Ne(e,t,n,r){return b(t,n)===void 0?Me(r):{error:{kind:e,message:t}}}function Pe(e,t,n=t){let r;try{let t=e instanceof G?e.stack??e.message:e;r=typeof t==`string`?t:String(t)}catch{r=`program threw an unrenderable value`}return Ne(`exception`,r,t,n)}function Fe(e){return class extends G{constructor(t,n){super(n),Te(this,`name`,e.name),Te(this,e.memberNameProperty,t)}}}function K(e,t,n){return e?new e(t,n):new G(n)}function Ie(e){let t=new Map;for(let n of e.namespaces)n.errorClass&&t.set(n.global,Fe(n.errorClass));return t}function Le(e,t){e.on(`message`,e=>{let n=t.get(e.id);if(n)if(t.delete(e.id),e.ok){let t=Se(e.value);t===void 0?n.reject(new G(`binding resolution must be lossless JSON`)):n.resolve(t)}else n.reject(new G(e.message))})}function Re(e,t,n,r,i=Ie(e),a){return e.namespaces.map(({global:e,names:o})=>{let s=i.get(e),c=Object.create(null);for(let i of o)Object.defineProperty(c,i,{enumerable:!0,value:o=>{let c;try{c=H(o)}catch{c=void 0}if(c===void 0)return Promise.reject(K(s,i,`binding arguments must be lossless JSON`));let l={type:`call`,id:r.value,global:e,name:i,args:U(c)};return a!==void 0&&x(l,a)===void 0?Promise.reject(K(s,i,`binding arguments exceed maxFrameBytes`)):new Promise((e,a)=>{let o=r.value++;n.set(o,{resolve:e,reject:e=>{a(K(s,i,e.message))}});try{t.postMessage(l)}catch(e){n.delete(o),a(K(s,i,`binding arguments must be structured-cloneable: ${e instanceof G?e.message:String(e)}`))}})}});return c})}async function ze(e,t,n,r){let i=new Ee(t.maxOutputBytes,t=>{e.postMessage({type:`log`,text:t})},()=>{e.postMessage({type:`output-limit`})});ke(i,n.stdout),ke(i,n.stderr);let a=new Map;Le(e,a);let o={value:1},s=Ie(t),c=Re(t,e,a,o,s,r),l=[],u=[];for(let e of t.namespaces){if(!e.errorClass)continue;l.push(e.errorClass.name);let t=s.get(e.global);if(!t)throw new G(`missing binding error class for ${e.global}`);u.push(t)}let d=Oe(i),f;try{let e=(async()=>{}).constructor;f={type:`done`,...je(await new e(...t.namespaces.map(e=>e.global),...l,`console`,`'use strict';\\n${t.code}`)(...c,...u,d),i.remainingOutputBytes(),t.maxOutputBytes)}}catch(e){f={type:`done`,...Pe(e,i.remainingOutputBytes(),t.maxOutputBytes)}}e.postMessage(r!==void 0&&x(f,r)===void 0?{type:`output-limit`}:f)}function q(e){return e.readableEnded||e.destroyed?Promise.resolve():new Promise(t=>{let n=()=>{e.off(`end`,n),e.off(`close`,n),e.off(`error`,n),t()};e.once(`end`,n),e.once(`close`,n),e.once(`error`,n),(e.readableEnded||e.destroyed)&&n()})}const Be=new Set([`exception`,`timeout`,`abort`,`worker-exit`,`invalid-output`,`output-limit`]);let J=0;function Ve(){let e=process.execArgv.indexOf(`--eval`);if(e<0)throw Error(`code runtime runner requires its eval source`);let t=process.execArgv[e+1];if(t===void 0)throw Error(`code runtime runner requires its eval source`);return t}function Y(e){return typeof e==`object`&&e?e:void 0}function X(t,n){try{let r=JSON.stringify(t);return typeof r==`string`&&e.byteLength(r)<=n?r:void 0}catch{return}}function He(e){process.stdout.write(e),process.stdout.write(`\n`)}function Ue(e){let t=X(e,J);return t===void 0?!1:(He(t),!0)}function We(e){let t=Y(e);return t!==void 0&&typeof t.kind==`string`&&Be.has(t.kind)&&typeof t.message==`string`}function Ge(e){return We(e)&&(e.kind===`exception`||e.kind===`invalid-output`||e.kind===`output-limit`)}function Ke(e,t){return e.error===void 0?{type:`done`,...e.value===void 0?{}:{value:Q(e.value)}}:t(e.error)?{type:`done`,error:e.error}:void 0}function Z(e){let t=Y(e);if(!(t===void 0||t.type!==`boot`||typeof t.code!=`string`||!Array.isArray(t.namespaces)||!Number.isSafeInteger(t.maxOutputBytes)||t.maxOutputBytes<4||!Number.isSafeInteger(t.maxFrameBytes)||t.maxFrameBytes{e.once(`exit`,()=>{t()})})}function $(){return{type:`done`,error:{kind:`worker-exit`,message:`code runtime bridge frame exceeded maxFrameBytes`}}}function Ye(){let e=n({input:process.stdin,crlfDelay:1/0}),r,i=0,a=2,o=0,s=!1,c=t=>{if(s)return;let n=X(t,J)??X($(),J);s=!0,n!==void 0&&He(n);let i=r;r=void 0,(i===void 0?Promise.resolve():new Promise(e=>{setImmediate(e)}).then(async()=>{let e=q(i.stdout),t=q(i.stderr),n=Je(i);i.kill(`SIGKILL`),await Promise.all([n,e,t])})).catch(e=>{process.stderr.write(`dsh-code-runtime-subprocess controller cleanup error: ${String(e)}\\n`)}).then(()=>{e.close(),process.stdin.destroy()})},l=e=>{if(s)return;let t=+(o>0),n=b(e,i-a-t);if(n===void 0){c({type:`output-limit`});return}a+=n+t,o+=1,Ue({type:`log`,text:e})||c($())},u=e=>{i=e.maxOutputBytes,J=e.maxFrameBytes,r=t(process.execPath,process.execArgv,{env:{DSH_CODE_RUNTIME_CONTROLLER:`1`},detached:!1,stdio:[`ignore`,`pipe`,`pipe`,`ipc`]});let n=r;n.stdout.on(`data`,e=>{l(e.toString(`utf8`))}),n.stderr.on(`data`,e=>{l(e.toString(`utf8`))}),n.on(`message`,e=>{let t=Y(e);if(t!==void 0){if(t.type===`log`&&typeof t.text==`string`){l(t.text);return}if(!s){if(t.type===`call`&&typeof t.id==`number`&&typeof t.global==`string`&&typeof t.name==`string`)Ue({type:`call`,id:t.id,global:t.global,name:t.name,args:Q(t.args)})||c($());else if(t.type===`output-limit`)c({type:`output-limit`});else if(t.type===`done`){let e=Ke(t,We);e!==void 0&&c(e)}}}}),n.on(`error`,e=>{c({type:`done`,error:{kind:`worker-exit`,message:`remote controller error: ${e.message}`}})}),n.on(`exit`,e=>{s||c({type:`done`,error:{kind:`worker-exit`,message:`remote controller exited with code ${e} before completing`}})}),n.send(e,e=>{e!==null&&c({type:`done`,error:{kind:`worker-exit`,message:`remote controller boot failed: ${e.message}`}})})};e.on(`line`,e=>{let t;try{t=JSON.parse(e)}catch(e){process.stderr.write(`dsh-code-runtime-subprocess frame error: ${String(e)}\\n`),c({type:`done`,error:{kind:`worker-exit`,message:`remote runner received a malformed frame`}});return}if(r===void 0){let e=Z(t);if(e===void 0){c({type:`done`,error:{kind:`worker-exit`,message:`remote runner received an invalid boot frame`}});return}u(e);return}let n=qe(t);n!==void 0&&r.send(n,e=>{e!==null&&c({type:`done`,error:{kind:`worker-exit`,message:`remote controller reply failed: ${e.message}`}})})}),e.on(`close`,()=>{r!==void 0&&!s&&c({type:`done`,error:{kind:`abort`,message:`remote runner input closed`}})})}function Xe(){let t,n=!1,a,o=0,s=e=>process.send===void 0||o>0&&X(e,o)===void 0?!1:(process.send(e),!0),c=e=>{if(n)return;n=!0,clearInterval(a);let r=o>0&&X(e,o)===void 0?$():e,i=t;t=void 0,(i===void 0?Promise.resolve():new Promise(e=>{setImmediate(e)}).then(async()=>{let e=q(i.stdout),t=q(i.stderr);await Promise.all([i.terminate(),e,t])})).catch(e=>{s({type:`log`,text:`dsh-code-runtime-subprocess worker cleanup error: ${String(e)}\\n`})}).then(()=>{if(process.send===void 0){process.exitCode=1;return}process.send(r,()=>{process.connected&&process.disconnect()})})};process.on(`message`,l=>{if(t===void 0){let u=Z(l);if(u===void 0){c({type:`done`,error:{kind:`worker-exit`,message:`remote controller received an invalid boot frame`}});return}o=u.maxFrameBytes,t=new i(new URL(`data:text/javascript;base64,${e.from(Ve()).toString(`base64`)}`),{workerData:u,env:{},execArgv:[],stdout:!0,stderr:!0,resourceLimits:{maxOldGenerationSizeMb:u.maxOldGenerationSizeMb}});let d=t;d.stdout.on(`data`,e=>{s({type:`log`,text:e.toString(`utf8`)})||c($())}),d.stderr.on(`data`,e=>{s({type:`log`,text:e.toString(`utf8`)})||c($())}),d.on(`message`,e=>{let t=Y(e);if(t!==void 0){if(t.type===`call`&&typeof t.id==`number`&&typeof t.global==`string`&&typeof t.name==`string`)s({type:`call`,id:t.id,global:t.global,name:t.name,args:Q(t.args)})||c($());else if(t.type===`log`&&typeof t.text==`string`)s({type:`log`,text:t.text})||c($());else if(t.type===`output-limit`)c({type:`output-limit`});else if(t.type===`done`){let e=Ke(t,Ge);e!==void 0&&c(e)}}}),d.on(`error`,e=>{c({type:`done`,error:{kind:`worker-exit`,message:`worker error: ${e.stack||e.message||r(e)}`}})}),d.on(`exit`,e=>{n||c({type:`done`,error:{kind:`worker-exit`,message:`worker exited with code ${e} before completing`}})}),a=setInterval(()=>{t!==void 0&&t.performance.eventLoopUtilization().active>u.computeMs&&c({type:`done`,error:{kind:`timeout`,message:`compute budget exhausted (${u.computeMs}ms busy)`}})},25);return}let u=qe(l);u!==void 0&&t.postMessage(u)}),process.on(`disconnect`,()=>{t!==void 0&&!n&&t.terminate()})}if(a)process.env.DSH_CODE_RUNTIME_CONTROLLER===`1`?Xe():Ye();else{if(o===null)throw Error(`remote worker requires parentPort`);let e=s;ze(o,e,{stdout:process.stdout,stderr:process.stderr},e.maxFrameBytes)}export{};" diff --git a/packages/util/atomic-write/src/runner.ts b/packages/util/atomic-write/src/runner.ts deleted file mode 100644 index 2655d22026..0000000000 --- a/packages/util/atomic-write/src/runner.ts +++ /dev/null @@ -1,401 +0,0 @@ -/** Typed source for the dependency-free execution-world runner bundle. */ - -import { Buffer } from 'node:buffer' -import { spawn } from 'node:child_process' -import type { ChildProcess } from 'node:child_process' -import { createInterface } from 'node:readline' -import type { Readable } from 'node:stream' -import { inspect } from 'node:util' -import { Worker, isMainThread, parentPort, workerData } from 'node:worker_threads' -import { - decodeWorkerJson, - encodeWorkerJson, - jsonStringBytesUpTo, - runWorkerMain, - waitForRuntimePipeDrain, -} from '@deepseek-ai/dsh-code-runtime-worker/runtime-host' -import type { WorkerJsonWire } from '@deepseek-ai/dsh-code-runtime-worker/runtime-host' - -type WorkerBootData = Parameters[1] -type Controller = ChildProcess & { stdout: Readable; stderr: Readable } -type FailureKind = 'exception' | 'timeout' | 'abort' | 'worker-exit' | 'invalid-output' | 'output-limit' - -interface RuntimeBootData extends WorkerBootData { - type: 'boot' - maxFrameBytes: number - maxOldGenerationSizeMb: number - computeMs: number -} - -interface RuntimeFailure { - kind: FailureKind - message: string -} - -interface RuntimeCall { - type: 'call' - id: number - global: string - name: string - args: WorkerJsonWire | null -} - -type RuntimeReply = - | { type: 'reply'; id: number; ok: true; value: unknown } - | { type: 'reply'; id: number; ok: false; message: string } - -type RuntimeMessage = RuntimeCall - | { type: 'log'; text: string } - | { type: 'output-limit' } - | { type: 'done'; value?: WorkerJsonWire | null; error?: RuntimeFailure } - -const failureKinds = new Set([ - 'exception', 'timeout', 'abort', 'worker-exit', 'invalid-output', 'output-limit', -]) - -let maxFrameBytes = 0 - -function runnerSource(): string { - const evalIndex = process.execArgv.indexOf('--eval') - if (evalIndex < 0) throw new Error('code runtime runner requires its eval source') - const source = process.execArgv[evalIndex + 1] - if (source === undefined) throw new Error('code runtime runner requires its eval source') - return source -} - -function recordOf(value: unknown): Record | undefined { - return typeof value === 'object' && value !== null ? value as Record : undefined -} - -function encodeJsonBounded(value: unknown, maxBytes: number): string | undefined { - try { - const json: unknown = JSON.stringify(value) - return typeof json === 'string' && Buffer.byteLength(json) <= maxBytes ? json : undefined - } catch { - return undefined - } -} - -function emitJson(json: string): void { - process.stdout.write(json) - process.stdout.write('\n') -} - -function emitFrame(message: RuntimeMessage): boolean { - const json = encodeJsonBounded(message, maxFrameBytes) - if (json === undefined) return false - emitJson(json) - return true -} - -function validFailure(value: unknown): value is RuntimeFailure { - const record = recordOf(value) - return record !== undefined - && typeof record.kind === 'string' - && failureKinds.has(record.kind as FailureKind) - && typeof record.message === 'string' -} - -function validWorkerFailure(value: unknown): value is RuntimeFailure { - return validFailure(value) - && (value.kind === 'exception' || value.kind === 'invalid-output' || value.kind === 'output-limit') -} - -function doneMessage( - message: Record, - acceptsFailure: (value: unknown) => value is RuntimeFailure, -): RuntimeMessage | undefined { - if (message.error !== undefined) { - return acceptsFailure(message.error) ? { type: 'done', error: message.error } : undefined - } - return { type: 'done', ...message.value === undefined ? {} : { value: transportWireOrNull(message.value) } } -} - -function runtimeBoot(value: unknown): RuntimeBootData | undefined { - const record = recordOf(value) - if (record === undefined - || record.type !== 'boot' - || typeof record.code !== 'string' - || !Array.isArray(record.namespaces) - || !Number.isSafeInteger(record.maxOutputBytes) - || (record.maxOutputBytes as number) < 4 - || !Number.isSafeInteger(record.maxFrameBytes) - || (record.maxFrameBytes as number) < (record.maxOutputBytes as number) - || typeof record.computeMs !== 'number' - || !Number.isFinite(record.computeMs) - || (record.computeMs) <= 0 - || typeof record.maxOldGenerationSizeMb !== 'number' - || !Number.isFinite(record.maxOldGenerationSizeMb) - || (record.maxOldGenerationSizeMb) <= 0) return undefined - return record as unknown as RuntimeBootData -} - -function runtimeReply(value: unknown): RuntimeReply | undefined { - const record = recordOf(value) - if (record === undefined || record.type !== 'reply' || typeof record.id !== 'number' || typeof record.ok !== 'boolean') return undefined - return record.ok - ? { type: 'reply', id: record.id, ok: true, value: record.value } - : { type: 'reply', id: record.id, ok: false, message: String(record.message) } -} - -function transportWireOrNull(input: unknown): WorkerJsonWire | null { - const value = decodeWorkerJson(input) - return value === undefined ? null : encodeWorkerJson(value) -} - -function waitForChildExit(child: ChildProcess): Promise { - if (child.exitCode !== null || child.signalCode !== null) return Promise.resolve() - return new Promise((resolve) => { child.once('exit', () => { resolve() }) }) -} - -function frameLimitFailure(): RuntimeMessage { - return { - type: 'done', - error: { kind: 'worker-exit', message: 'code runtime bridge frame exceeded maxFrameBytes' }, - } -} - -function runLauncher(): void { - const input = createInterface({ input: process.stdin, crlfDelay: Infinity }) - let controller: Controller | undefined - let maxOutputBytes = 0 - let logBytes = 2 - let logEntries = 0 - let settling = false - - const finish = (message: RuntimeMessage): void => { - if (settling) return - const encoded = encodeJsonBounded(message, maxFrameBytes) - ?? encodeJsonBounded(frameLimitFailure(), maxFrameBytes) - settling = true - if (encoded !== undefined) emitJson(encoded) - const current = controller - controller = undefined - const drain = current === undefined - ? Promise.resolve() - : new Promise((resolve) => { setImmediate(resolve) }).then(async () => { - const stdoutDrained = waitForRuntimePipeDrain(current.stdout) - const stderrDrained = waitForRuntimePipeDrain(current.stderr) - const exited = waitForChildExit(current) - current.kill('SIGKILL') - await Promise.all([exited, stdoutDrained, stderrDrained]) - }) - void drain.catch((error: unknown) => { - process.stderr.write(`dsh-code-runtime-subprocess controller cleanup error: ${String(error)}\n`) - }).then(() => { - input.close() - process.stdin.destroy() - }) - } - - const forwardLog = (text: string): void => { - if (settling) return - const separator = logEntries > 0 ? 1 : 0 - const cost = jsonStringBytesUpTo(text, maxOutputBytes - logBytes - separator) - if (cost === undefined) { - finish({ type: 'output-limit' }) - return - } - logBytes += cost + separator - logEntries += 1 - if (!emitFrame({ type: 'log', text })) finish(frameLimitFailure()) - } - - const startController = (boot: RuntimeBootData): void => { - maxOutputBytes = boot.maxOutputBytes - maxFrameBytes = boot.maxFrameBytes - controller = spawn(process.execPath, process.execArgv, { - env: { DSH_CODE_RUNTIME_CONTROLLER: '1' }, - detached: false, - stdio: ['ignore', 'pipe', 'pipe', 'ipc'], - }) as Controller - const current = controller - current.stdout.on('data', (data: Buffer) => { forwardLog(data.toString('utf8')) }) - current.stderr.on('data', (data: Buffer) => { forwardLog(data.toString('utf8')) }) - current.on('message', (raw: unknown) => { - const message = recordOf(raw) - if (message === undefined) return - if (message.type === 'log' && typeof message.text === 'string') { - forwardLog(message.text) - return - } - if (settling) return - if (message.type === 'call' - && typeof message.id === 'number' - && typeof message.global === 'string' - && typeof message.name === 'string') { - if (!emitFrame({ - type: 'call', id: message.id, global: message.global, name: message.name, args: transportWireOrNull(message.args), - })) finish(frameLimitFailure()) - } else if (message.type === 'output-limit') { - finish({ type: 'output-limit' }) - } else if (message.type === 'done') { - const done = doneMessage(message, validFailure) - if (done !== undefined) finish(done) - } - }) - current.on('error', (error: Error) => { - finish({ type: 'done', error: { kind: 'worker-exit', message: `remote controller error: ${error.message}` } }) - }) - current.on('exit', (code: number | null) => { - if (!settling) { - finish({ type: 'done', error: { kind: 'worker-exit', message: `remote controller exited with code ${code} before completing` } }) - } - }) - current.send(boot, (error: Error | null) => { - if (error !== null) { - finish({ type: 'done', error: { kind: 'worker-exit', message: `remote controller boot failed: ${error.message}` } }) - } - }) - } - - input.on('line', (line: string) => { - let raw: unknown - try { - raw = JSON.parse(line) as unknown - } catch (error: unknown) { - process.stderr.write(`dsh-code-runtime-subprocess frame error: ${String(error)}\n`) - finish({ type: 'done', error: { kind: 'worker-exit', message: 'remote runner received a malformed frame' } }) - return - } - if (controller === undefined) { - const boot = runtimeBoot(raw) - if (boot === undefined) { - finish({ type: 'done', error: { kind: 'worker-exit', message: 'remote runner received an invalid boot frame' } }) - return - } - startController(boot) - return - } - const reply = runtimeReply(raw) - if (reply !== undefined) { - controller.send(reply, (error: Error | null) => { - if (error !== null) { - finish({ type: 'done', error: { kind: 'worker-exit', message: `remote controller reply failed: ${error.message}` } }) - } - }) - } - }) - input.on('close', () => { - if (controller !== undefined && !settling) { - finish({ type: 'done', error: { kind: 'abort', message: 'remote runner input closed' } }) - } - }) -} - -function runController(): void { - let worker: Worker | undefined - let finished = false - let computeTimer: NodeJS.Timeout | undefined - let controllerMaxFrameBytes = 0 - - const send = (message: RuntimeMessage): boolean => { - if (process.send === undefined) return false - if (controllerMaxFrameBytes > 0 && encodeJsonBounded(message, controllerMaxFrameBytes) === undefined) return false - process.send(message) - return true - } - - const finish = (message: RuntimeMessage): void => { - if (finished) return - finished = true - clearInterval(computeTimer) - const bounded = controllerMaxFrameBytes > 0 && encodeJsonBounded(message, controllerMaxFrameBytes) === undefined - ? frameLimitFailure() - : message - const current = worker - worker = undefined - const drain = current === undefined - ? Promise.resolve() - : new Promise((resolve) => { setImmediate(resolve) }).then(async () => { - const stdoutDrained = waitForRuntimePipeDrain(current.stdout) - const stderrDrained = waitForRuntimePipeDrain(current.stderr) - await Promise.all([current.terminate(), stdoutDrained, stderrDrained]) - }) - void drain.catch((error: unknown) => { - send({ type: 'log', text: `dsh-code-runtime-subprocess worker cleanup error: ${String(error)}\n` }) - }).then(() => { - if (process.send === undefined) { - process.exitCode = 1 - return - } - process.send(bounded, () => { if (process.connected) process.disconnect() }) - }) - } - - process.on('message', (raw: unknown) => { - if (worker === undefined) { - const boot = runtimeBoot(raw) - if (boot === undefined) { - finish({ type: 'done', error: { kind: 'worker-exit', message: 'remote controller received an invalid boot frame' } }) - return - } - controllerMaxFrameBytes = boot.maxFrameBytes - const sourceUrl = new URL(`data:text/javascript;base64,${Buffer.from(runnerSource()).toString('base64')}`) - worker = new Worker(sourceUrl, { - workerData: boot, - env: {}, - execArgv: [], - stdout: true, - stderr: true, - resourceLimits: { maxOldGenerationSizeMb: boot.maxOldGenerationSizeMb }, - }) - const current = worker - current.stdout.on('data', (data: Buffer) => { - if (!send({ type: 'log', text: data.toString('utf8') })) finish(frameLimitFailure()) - }) - current.stderr.on('data', (data: Buffer) => { - if (!send({ type: 'log', text: data.toString('utf8') })) finish(frameLimitFailure()) - }) - current.on('message', (messageRaw: unknown) => { - const message = recordOf(messageRaw) - if (message === undefined) return - if (message.type === 'call' - && typeof message.id === 'number' - && typeof message.global === 'string' - && typeof message.name === 'string') { - if (!send({ - type: 'call', id: message.id, global: message.global, name: message.name, args: transportWireOrNull(message.args), - })) finish(frameLimitFailure()) - } else if (message.type === 'log' && typeof message.text === 'string') { - if (!send({ type: 'log', text: message.text })) finish(frameLimitFailure()) - } else if (message.type === 'output-limit') { - finish({ type: 'output-limit' }) - } else if (message.type === 'done') { - const done = doneMessage(message, validWorkerFailure) - if (done !== undefined) finish(done) - } - }) - current.on('error', (error: Error) => { - finish({ - type: 'done', - error: { kind: 'worker-exit', message: `worker error: ${error.stack || error.message || inspect(error)}` }, - }) - }) - current.on('exit', (code: number) => { - if (!finished) { - finish({ type: 'done', error: { kind: 'worker-exit', message: `worker exited with code ${code} before completing` } }) - } - }) - computeTimer = setInterval(() => { - if (worker !== undefined && worker.performance.eventLoopUtilization().active > boot.computeMs) { - finish({ type: 'done', error: { kind: 'timeout', message: `compute budget exhausted (${boot.computeMs}ms busy)` } }) - } - }, 25) - return - } - const reply = runtimeReply(raw) - if (reply !== undefined) worker.postMessage(reply) - }) - process.on('disconnect', () => { if (worker !== undefined && !finished) void worker.terminate() }) -} - -if (!isMainThread) { - if (parentPort === null) throw new Error('remote worker requires parentPort') - const boot = workerData as RuntimeBootData - void runWorkerMain(parentPort, boot, { stdout: process.stdout, stderr: process.stderr }, boot.maxFrameBytes) -} else if (process.env.DSH_CODE_RUNTIME_CONTROLLER === '1') { - runController() -} else { - runLauncher() -} diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 666d01c1c1..87648e01e1 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -4275,6 +4275,12 @@ importers: '@deepseek-ai/dsh-brand': specifier: workspace:^ version: link:../../util/brand + '@deepseek-ai/dsh-fs': + specifier: workspace:^ + version: link:../../fs/fs + '@deepseek-ai/dsh-fs-local': + specifier: workspace:^ + version: link:../../fs/fs-local '@deepseek-ai/dsh-invariants': specifier: workspace:^ version: link:../../support/invariants @@ -4312,6 +4318,9 @@ importers: '@deepseek-ai/dsh-agent': specifier: workspace:^ version: link:../../core/agent + '@deepseek-ai/dsh-fs-local': + specifier: workspace:^ + version: link:../../fs/fs-local '@deepseek-ai/dsh-invariants': specifier: workspace:^ version: link:../../support/invariants @@ -4327,6 +4336,9 @@ importers: '@deepseek-ai/dsh-session': specifier: workspace:^ version: link:../../core/session + '@deepseek-ai/dsh-subprocess-local': + specifier: workspace:^ + version: link:../../subprocess/subprocess-local '@deepseek-ai/dsh-system-prompt': specifier: workspace:^ version: link:../../core/system-prompt @@ -4440,9 +4452,6 @@ importers: packages/pty/pty-local: dependencies: - node-pty: - specifier: ^1.1.0 - version: 1.1.0(patch_hash=7a0c04f1f49d798a9ffe2f7f414c01064a44ca2489772d0c3e1235ab336755e6) schemastery: specifier: ^3.18.0 version: link:../../../vendor/schemastery @@ -4468,6 +4477,9 @@ importers: '@deepseek-ai/dsh-subprocess': specifier: workspace:^ version: link:../../subprocess/subprocess + '@deepseek-ai/dsh-subprocess-local': + specifier: workspace:^ + version: link:../../subprocess/subprocess-local cordis: specifier: ^4.0.0-rc.7 version: link:../../../vendor/cordis @@ -4563,6 +4575,9 @@ importers: '@deepseek-ai/dsh-session': specifier: workspace:^ version: link:../../core/session + '@deepseek-ai/dsh-subprocess-local': + specifier: workspace:^ + version: link:../../subprocess/subprocess-local '@deepseek-ai/dsh-system-prompt': specifier: workspace:^ version: link:../../core/system-prompt @@ -5940,6 +5955,10 @@ importers: version: link:../../../vendor/cordis packages/subprocess/subprocess-local: + dependencies: + node-pty: + specifier: ^1.1.0 + version: 1.1.0(patch_hash=7a0c04f1f49d798a9ffe2f7f414c01064a44ca2489772d0c3e1235ab336755e6) devDependencies: '@deepseek-ai/dsh-invariants': specifier: workspace:^ diff --git a/scripts/gen-doc-graphs.ts b/scripts/gen-doc-graphs.ts index 00520bdbe7..e254696491 100644 --- a/scripts/gen-doc-graphs.ts +++ b/scripts/gen-doc-graphs.ts @@ -48,9 +48,13 @@ interface EventRelation { listeners: Set } -interface PackageSource { +/** One scanned package source file and its owning package short name. */ +export interface PackageSource { + /** Repository-relative path. */ rel: string + /** Package short name from the `packages///src` path. */ pkg: string + /** The bound program source file. */ sourceFile: ts.SourceFile } @@ -120,7 +124,7 @@ const SERVICE_ROLES: ServiceRole[] = [ pkg: 'session', title: 'In-memory session store', mode: 'core', - consumers: ['agent-loop', 'agent', 'cli-demo', 'session-persistence', 'session-query', 'session-query-sqlite', 'subagent-inprocess', 'invariants'], + consumers: ['agent-loop', 'agent', 'session-persistence', 'session-query', 'session-query-sqlite', 'subagent-inprocess', 'invariants'], note: 'Owns append-only Session instances and emits the durable session event feed.', }, { @@ -155,6 +159,24 @@ const SERVICE_ROLES: ServiceRole[] = [ consumers: ['agent-loop', 'tool-bash', 'hooks-claude', 'hooks-codex', 'session-query', 'session-query-sqlite'], note: 'Backends persist the same SessionEvent vocabulary; apps choose a backend at composition time.', }, + { + key: 'settings', + pkg: 'settings', + title: 'User-settings seam', + mode: 'seam', + implementations: ['settings-local'], + consumers: ['llm-deepseek', 'llm-pi-ai', 'apiproxy'], + note: 'Plugins register namespace schemas and resolve layered values; providers store the raw document. The LLM adapters register their entry config as the composition base under the user section; the web gateway serves redacted layered descriptors and writes the user layer.', + }, + { + key: 'credentials', + pkg: 'credentials', + title: 'Credential seam', + mode: 'seam', + implementations: ['credentials-local'], + consumers: ['llm-deepseek', 'llm-pi-ai', 'apiproxy'], + note: 'Configuration carries references to secrets; providers own the values. Consumers resolve per operation, so a rotated credential reaches the very next request; the web gateway exposes value-free views and write-only storage.', + }, { key: 'telemetry', pkg: 'session-telemetry', @@ -203,7 +225,6 @@ const SERVICE_ROLES: ServiceRole[] = [ pkg: 'session-reference', title: 'Cross-session snapshot preparation', mode: 'core', - consumers: ['tui'], note: 'Projects bounded current-surface conversation snapshots into durable untrusted message context; host adapters own mention syntax.', }, { @@ -235,8 +256,7 @@ const SERVICE_ROLES: ServiceRole[] = [ pkg: 'user-interaction', title: 'Human question/answer seam', mode: 'seam', - implementations: ['tui'], - consumers: ['tool-ask-user', 'tui'], + consumers: ['tool-ask-user'], note: 'UI front doors provide the active human-answer provider; tool-ask-user pauses a tool call on the provider-neutral ask() promise.', }, { @@ -251,8 +271,7 @@ const SERVICE_ROLES: ServiceRole[] = [ pkg: 'commands', title: 'Human command registry', mode: 'core', - consumers: ['tui'], - note: 'Plugins register direct human commands; TUI consumes the effective per-agent catalog without sending invocations to the model.', + note: 'Plugins register direct human commands without sending invocations to the model.', }, { key: 'sessionProjections', @@ -270,19 +289,12 @@ const SERVICE_ROLES: ServiceRole[] = [ consumers: ['host-apiproxy'], note: 'Durably checkpoints projection unit states per session (throttled + turn/end/detach mandatory points) and serves the cold-read ladder: cache row + persistence tail replay, so listings never load full logs.', }, - { - key: 'tui', - pkg: 'tui', - title: 'Mounted-terminal interaction service', - mode: 'bundle', - note: 'One TUI front door provides a FIFO overlay host; injected plugins receive caller-fiber ownership without access to pi-tui or terminal lifecycle state.', - }, { key: 'skills', pkg: 'skill', title: 'Skill provider registry', mode: 'seam', - implementations: ['skill-local'], + implementations: ['skill-badge', 'skill-local'], consumers: ['tool-skill'], note: 'Merges provider skill catalogs; tool-skill renders the session-prefix catalog and loads complete skill bodies.', }, @@ -291,7 +303,7 @@ const SERVICE_ROLES: ServiceRole[] = [ pkg: 'agent', title: 'Agent service', mode: 'core', - consumers: ['agent-loop', 'acp', 'cli-demo', 'subagent-inprocess', 'tui-demo'], + consumers: ['agent-loop', 'acp', 'subagent-inprocess'], note: 'Owns live Agent handles, the create/resume factory seam, and process-local initiator propagation.', }, { @@ -315,24 +327,25 @@ const SERVICE_ROLES: ServiceRole[] = [ title: 'Subprocess seam', mode: 'seam', implementations: ['subprocess-local'], - consumers: ['bash-local', 'bash-sandbox', 'pty-local', 'lsp-local', 'subagent-acp'], - note: 'The bash executors, PTY shell backend, LSP host, and ACP subagent backend spawn through ctx.subprocess; the service owns process coordinates, tree/session lifetime, stdio dispositions, terminal mechanics, and kill escalation.', + consumers: ['bash-local', 'bash-sandbox', 'pty-local', 'lsp-local', 'subagent-acp', 'subagent-codex', 'subagent-claude-code'], + note: 'The bash executors, the PTY shell backend, the LSP host, and the out-of-process ACP, Codex, and Claude Code subagent backends spawn through ctx.subprocess; the service owns process coordinates, tree/session lifetime, stdio dispositions, terminal mechanics, and kill escalation.', }, { key: 'bash', pkg: 'bash', title: 'Bash executor seam', mode: 'seam', - implementations: ['bash-local', 'bash-sandbox'], - consumers: ['tool-bash', 'hooks-claude', 'hooks-codex'], - note: 'The model-facing bash tools and hook bridges consume this seam; sandboxed or remote executors replace bash-local without touching them.', + implementations: ['bash-local', 'bash-sandbox', 'pwsh-local'], + consumers: ['tool-bash', 'tool-pwsh', 'hooks-claude', 'hooks-codex'], + note: 'The model-facing shell tools and hook bridges consume this seam; sandboxed, remote, or PowerShell executors replace bash-local without touching them.', }, { key: 'bashEnv', - pkg: 'tool-bash', + pkg: 'bash-env', title: 'Managed bash environment registry', mode: 'core', - note: 'Plugins declare effect-scoped DSH_* facts; tool-bash collects one trusted snapshot per execution and the executor rebuilds the namespace.', + consumers: ['tool-bash', 'tool-pwsh'], + note: 'Plugins declare effect-scoped DSH_* facts; each shell tool collects one trusted snapshot per execution and its executor rebuilds the namespace.', }, { key: 'pty', @@ -409,11 +422,11 @@ const SERVICE_ROLES: ServiceRole[] = [ { key: 'subagents', pkg: 'subagent', - title: 'Subagent provider registry', + title: 'Subagent provider and continuation service', mode: 'seam', - implementations: ['subagent-spawn', 'subagent-fork', 'subagent-acp'], - consumers: ['tool-subagent', 'tool-ralph'], - note: 'Providers implement transports; tool-subagent exposes configured delegation while tool-ralph requires one fresh structured-output route.', + implementations: ['subagent-spawn', 'subagent-fork', 'subagent-acp', 'subagent-codex', 'subagent-claude-code', 'subagent-dsh-sdk'], + consumers: ['tool-subagent', 'tool-subagent-control', 'tool-ralph'], + note: 'Providers implement transports; the service also owns optional Activation-based continuation orchestration, tool-subagent selects one-shot or continuable delegation, tool-subagent-control delivers follow-ups, and tool-ralph requires one fresh structured-output route.', }, { key: 'tasks', @@ -592,7 +605,8 @@ function parseExampleCordis(rel: string): ExamplePlugin[] { if (current?.name) plugins.push({ id: current.id, name: current.name }) } for (const line of text.split('\n')) { - const id = /^-\s+id:\s+(.+?)\s*$/.exec(line) + // Top-level rows (`- id:`) and bundle-patch insert rows (` - id:`). + const id = /^\s*-\s+id:\s+(.+?)\s*$/.exec(line) if (id?.[1] !== undefined) { flush() current = { id: stripYamlScalar(id[1]) } @@ -611,28 +625,20 @@ function stripYamlScalar(value: string): string { const APP_EXAMPLES = [ { - id: 'tui', - rel: 'examples/tui-agent/composition.md', - title: 'TUI Agent App Composition', - label: 'examples/tui-agent', - config: 'examples/tui-agent/cordis.yml', - summary: 'The TUI agent combines the real DeepSeek adapter, coding tools, compaction, subagents, and workflows with the full-screen terminal app package.', + id: 'dsh_base', + rel: 'apps/cli/composition.md', + title: 'DSH Base Composition', + label: 'packages/bundle/base/cordis.patch.yml', + config: 'packages/bundle/base/cordis.patch.yml', + summary: 'The dsh-base bundle patch every profile applies first; mode bundles (dsh-web-app, dsh-headless) and the user\'s profile layer patch over it.', }, { id: 'headless', rel: 'examples/headless-agent/composition.md', - title: 'Headless Agent App Composition', + title: 'Headless Agent Snapshot Composition', label: 'examples/headless-agent', config: 'examples/headless-agent/cordis.yml', - summary: 'The headless demo combines the real DeepSeek adapter and coding capabilities with the one-shot app package, format-pure stdout, and one fresh persisted top-level session.', - }, - { - id: 'cordis', - rel: 'examples/cordis-agent/composition.md', - title: 'Cordis Agent App Composition', - label: 'examples/cordis-agent', - config: 'examples/cordis-agent/cordis.yml', - summary: 'The self-referential demo puts @deepseek-ai/dsh-tool-cordis on the coding spine, letting the agent inspect its current-process runtime and mount or unmount in-memory temporary Plugins.', + summary: 'The headless snapshot composition combines the real DeepSeek adapter and coding capabilities with one explicitly configured persisted top-level agent; its JSONL driver is test-only.', }, { id: 'acp', @@ -651,11 +657,7 @@ function renderAppExpansion(lines: string[], appNode: string, pluginName: string const jsonl = nodeId('bundle', 'jsonl') lines.push(` ${appNode} --> ${agentCore}["@deepseek-ai/dsh-agent-spine-demo"]`) lines.push(` ${appNode} --> ${jsonl}["@deepseek-ai/dsh-session-persistence-jsonl"]`) - if (pluginName === '@deepseek-ai/dsh-tui-demo') { - lines.push(` ${appNode} --> ${nodeId('frontdoor', 'tui')}["@deepseek-ai/dsh-tui
pre-created main agent"]`) - } else if (pluginName === '@deepseek-ai/dsh-cli-demo') { - lines.push(` ${appNode} --> ${nodeId('frontdoor', 'cli')}["one-shot driver
format-pure stdout
fresh top-level agent"]`) - } else if (pluginName === '@deepseek-ai/dsh-acp-demo') { + if (pluginName === '@deepseek-ai/dsh-acp-demo') { lines.push(` ${appNode} --> ${nodeId('frontdoor', 'acp')}["@deepseek-ai/dsh-acp
automation-only JSON-RPC stdio
fresh sessions created by client"]`) } lines.push( @@ -681,7 +683,7 @@ function renderAppComposition(example: AppExample): string { const pluginNode = nodeId(`plugin_${example.id}`, plugin.id) lines.push(` ${pluginNode}["${escLabel(plugin.id)}
${escLabel(plugin.name)}"]`) lines.push(` cfg --> ${pluginNode}`) - if (plugin.name === '@deepseek-ai/dsh-tui-demo' || plugin.name === '@deepseek-ai/dsh-cli-demo' || plugin.name === '@deepseek-ai/dsh-acp-demo') { + if (plugin.name === '@deepseek-ai/dsh-acp-demo') { renderAppExpansion(lines, pluginNode, plugin.name) } } @@ -698,13 +700,26 @@ function renderAppComposition(example: AppExample): string { return lines.join('\n') } +type CallSiteIndex = Map + +/** + * The only method names visitSource classifies; receiver typing runs on these + * alone. Obligation: every method name matched by a branch inside visitSource + * must appear here — the prefilter drops non-members before any branch runs, + * so a branch for an unlisted name is silently dead. + */ +const EVENT_API_METHODS = new Set(['on', 'once', 'emit', 'parallel', 'serial', 'waterfall', 'dispatch']) + /** Collect event dispatch/listener relations from real cross-file receiver types. */ -class EventRelationCollector { +export class EventRelationCollector { private readonly relations = new Map() - private readonly callSites = new Map() + private readonly fileCallSites = new Map() + private readonly localCalleeProofs = new Map() + private globalCallSites: CallSiteIndex | null = null private readonly contextType: ts.Type private readonly agentDispatchType: ts.Type private readonly eventsServiceType: ts.Type + private readonly packageSourceFiles: ReadonlySet constructor( private readonly project: TypeScriptProject, @@ -713,7 +728,7 @@ class EventRelationCollector { this.contextType = this.declaredType('vendor/cordis/src/context.ts', 'Context') this.agentDispatchType = this.declaredType('packages/core/agent/src/dispatch.ts', 'AgentEventDispatch') this.eventsServiceType = this.declaredType('vendor/cordis/src/events.ts', 'EventsService') - this.indexCallSites() + this.packageSourceFiles = new Set(sources.map(source => source.sourceFile)) } /** Return all event relations discovered from the Program. */ @@ -733,20 +748,88 @@ class EventRelationCollector { return this.project.checker.getDeclaredTypeOfSymbol(symbol) } - /** Index resolved local function calls for narrow argument-flow recovery. */ - private indexCallSites(): void { + /** Index resolved function calls in the given files for narrow argument-flow recovery. */ + private buildCallSiteIndex(files: Iterable): CallSiteIndex { + const index: CallSiteIndex = new Map() const visit = (node: ts.Node): void => { if (ts.isCallExpression(node)) { const declaration = this.project.checker.getResolvedSignature(node)?.declaration if (declaration) { - const calls = this.callSites.get(declaration) ?? [] + const calls = index.get(declaration) ?? [] calls.push(node) - this.callSites.set(declaration, calls) + index.set(declaration, calls) } } ts.forEachChild(node, visit) } - for (const source of this.sources) visit(source.sourceFile) + for (const file of files) visit(file) + return index + } + + /** + * Return every indexed call resolving to one local helper declaration. + * Fast path: when every same-file reference to the non-exported helper is + * provably a direct callee, module scoping confines all of its calls to that + * file, so only that file is indexed. Any other reference shape may alias + * the function value outward, so the original full package-source index + * decides instead. + */ + private callSitesFor(owner: ts.FunctionDeclaration): ts.CallExpression[] { + if (!this.globalCallSites && !this.provenLocalCallee(owner)) { + this.globalCallSites = this.buildCallSiteIndex(this.packageSourceFiles) + } + if (this.globalCallSites) return this.globalCallSites.get(owner) ?? [] + const file = owner.getSourceFile() + let index = this.fileCallSites.get(file) + if (!index) { + index = this.buildCallSiteIndex([file]) + this.fileCallSites.set(file, index) + } + return index.get(owner) ?? [] + } + + /** + * Prove every same-file reference to one helper is a direct callee. The + * proof owns its premises: an exported helper or a helper in a global + * script file (no import/export means program-wide scope, callable from + * another file with no same-file reference at all) fails immediately. + * Alias escapes (re-export statements, default exports, value reads) + * resolve back to the owner symbol at a non-callee position and fail the + * proof, as does anything the scan cannot positively classify. + */ + private provenLocalCallee(owner: ts.FunctionDeclaration): boolean { + const cached = this.localCalleeProofs.get(owner) + if (cached !== undefined) return cached + if (hasExportModifier(owner) || !ts.isExternalModule(owner.getSourceFile())) { + this.localCalleeProofs.set(owner, false) + return false + } + const name = owner.name + const ownerSymbol = name && this.project.checker.getSymbolAtLocation(name) + let proven = !!ownerSymbol + const refersToOwner = (identifier: ts.Identifier): boolean => { + // Shorthand properties resolve to the property symbol; ask for the value side. + const local = ts.isShorthandPropertyAssignment(identifier.parent) + ? this.project.checker.getShorthandAssignmentValueSymbol(identifier.parent) + : this.project.checker.getSymbolAtLocation(identifier) + if (!local) return false + const symbol = local.flags & ts.SymbolFlags.Alias + ? this.project.checker.getAliasedSymbol(local) + : local + return symbol === ownerSymbol + } + const visit = (node: ts.Node): void => { + if (!proven) return + if (ts.isIdentifier(node) && node !== name && node.text === name?.text + && !isDirectCallee(node) && refersToOwner(node)) { + proven = false + return + } + ts.forEachChild(node, visit) + } + visit(owner.getSourceFile()) + this.localCalleeProofs.set(owner, proven) + return proven } /** Walk one package source file and classify event API calls by receiver type. */ @@ -760,7 +843,7 @@ class EventRelationCollector { this.addDispatcher(name, source.pkg, 'emitAgentEvent') } } - } else if (ts.isPropertyAccessExpression(node.expression)) { + } else if (ts.isPropertyAccessExpression(node.expression) && EVENT_API_METHODS.has(node.expression.name.text)) { const receiverKind = this.receiverKind(node.expression.expression) const method = node.expression.name.text if (receiverKind === 'events-service' && method === 'dispatch') { @@ -863,7 +946,7 @@ class EventRelationCollector { const index = owner.parameters.indexOf(parameter) if (index < 0) return new Set() const events = new Set() - for (const call of this.callSites.get(owner) ?? []) { + for (const call of this.callSitesFor(owner)) { const argument = call.arguments[index] if (argument) addAll(events, this.eventNamesFromArgumentList(argument, new Set(seen))) } @@ -910,6 +993,21 @@ class EventRelationCollector { } } +/** Return whether an identifier is the callee of a call, seen through value-preserving wrappers. */ +function isDirectCallee(identifier: ts.Identifier): boolean { + let current: ts.Node = identifier + while ( + ts.isParenthesizedExpression(current.parent) + || ts.isAsExpression(current.parent) + || ts.isTypeAssertionExpression(current.parent) + || ts.isNonNullExpression(current.parent) + || ts.isSatisfiesExpression(current.parent) + ) { + current = current.parent + } + return ts.isCallExpression(current.parent) && current.parent.expression === current +} + /** Peel syntax-only wrappers that do not change an expression's runtime value. */ function unwrapExpression(expression: ts.Expression): ts.Expression { let current = expression @@ -965,14 +1063,22 @@ function unionSets(left: ReadonlySet, right: ReadonlySet): Set { return out } -function collectEventRelations(): Map { - const project = new TypeScriptProject(root) - const sources = project.sourceFiles().flatMap((sourceFile): PackageSource[] => { +/** + * Select the package source files of one project in deterministic order. + * @param project - the loaded repository TypeScript project. + * @returns `packages///src` files tagged with their package name. + */ +export function collectPackageSources(project: TypeScriptProject): PackageSource[] { + return project.sourceFiles().flatMap((sourceFile): PackageSource[] => { const rel = project.relativePath(sourceFile) const match = /^packages\/[^/]+\/([^/]+)\/src\/.+\.ts$/.exec(rel) return match?.[1] ? [{ rel, pkg: match[1], sourceFile }] : [] }).sort((left, right) => left.rel.localeCompare(right.rel)) - return new EventRelationCollector(project, sources).collect() +} + +function collectEventRelations(): Map { + const project = new TypeScriptProject(root) + return new EventRelationCollector(project, collectPackageSources(project)).collect() } function relationPackages(map: Map>, pkgsByShort: Map): string { @@ -1053,20 +1159,22 @@ function renderLifecycle(): string { ' participant Session', ' participant SDK as UI or SDK listener', ' User->>Agent: followup(content)', - ` Agent-->>SDK: ${mermaidCode('agent/inbox/enqueue')}`, + ` Agent-->>SDK: ${mermaidCode('agent/inbox/spliced')}`, + ` Agent-->>SDK: ${mermaidCode('agent/inbox/inserted')} { message }`, ' Agent->>Driver: queued work wakes driver', ` Driver-->>SDK: ${mermaidCode('agent/status')} running`, - ' Note over Agent,Driver: next-step acceptance window opens', - ` Driver->>Hooks: ${mermaidCode('agent/prompt-submit')} waterfall`, - ' Hooks-->>Driver: authoritative allow, block, or add context', - ' alt prompt blocked or admission failed', - ' Driver-->>Driver: append context-only batch or keep steering boundary pending', - ' else prompt allowed', + ' Note over Agent,Driver: claim pending next-step input plus one queued prompt', + ` Driver-->>SDK: ${mermaidCode('agent/inbox/spliced')} pure deletion`, + ` Driver-->>SDK: ${mermaidCode('agent/inbox/claimed')} { message, turn } per message`, + ` Driver->>Hooks: ${mermaidCode('agent/pre-step')} waterfall`, + ' Hooks-->>Driver: authoritative reject or enter(messages)', + ' alt proposed step rejected or pre-step failed', + ' Driver-->>Driver: claimed batch stays removed, no turn opens', + ' else enter proposed step', ` Driver->>Session: ${mermaidCode('turn/start')}`, - ` Driver->>Session: ${mermaidCode('user/message')}`, - ` Driver->>Prompt: ${mermaidCode('system-prompt/assemble')} waterfall`, - ` Driver-->>Driver: ${mermaidCode('agent/step')} serial checkpoint`, ` Driver->>Session: ${mermaidCode('step/start')}`, + ` Driver->>Session: ${mermaidCode('user/message')} per entered message`, + ` Driver->>Prompt: ${mermaidCode('system-prompt/assemble')} waterfall`, ` Driver->>LLM: ${mermaidCode('agent/request')} waterfall, then ${mermaidCode('llm/stream')} waterfall`, ' LLM-->>Driver: StreamChunk*', ` Driver->>Session: ${mermaidCode('assistant/chunk')}*`, @@ -1089,11 +1197,17 @@ function renderLifecycle(): string { ` Driver->>Session: ${mermaidCode('tool/result')}`, ' end', ' end', - ' Driver->>Session: post-tool context and steering (no prompt-submit)', ` Driver->>Session: ${mermaidCode('step/end')}`, - ` Driver->>Hooks: ${mermaidCode('agent/turn-stopping')} serial terminal checkpoint`, + ' opt natural stop and next-step inbox empty', + ` Driver->>Hooks: ${mermaidCode('agent/turn-stopping')} serial terminal checkpoint`, + ' end', + ' opt next-step input is pending', + ' Driver-->>Driver: claim pending next-step input', + ` Driver-->>SDK: ${mermaidCode('agent/inbox/claimed')} { message, turn } per message`, + ` Driver->>Hooks: ${mermaidCode('agent/pre-step')} waterfall`, + ' Hooks-->>Driver: authoritative reject or enter(messages)', + ' end', ' end', - ' Note over Agent,Driver: next-step acceptance window closes', ` Driver->>Session: ${mermaidCode('turn/end')}`, ' end', ` Driver-->>SDK: ${mermaidCode('agent/status')} idle`, @@ -1101,9 +1215,9 @@ function renderLifecycle(): string { '', 'The `assistant/message` edge records every successful provider call, including content-less and `max-tokens` finishes. Empty content stays out of derived history while the durable anchor retains usage and exact chunk provenance, including an explicit empty source set.', '', - '`dsh-compact-basic` uses `agent/step` for pressure before request derivation and `agent/request-error` only for canonical context overflow. Once either trigger qualifies, optional tool-result pruning runs before summary selection. Recovery works between the closed failed step and failed turn close, and opens a fresh retry turn only when pruning or summarization advances the surface replacement generation; otherwise the original request error remains authoritative.', + '`dsh-compact-basic` uses `agent/pre-step` for pressure before request derivation and `agent/request-error` only for canonical context overflow. Once either trigger qualifies, optional tool-result pruning runs before summary selection. Recovery works between the closed failed step and failed turn close, and opens a fresh retry turn only when pruning or summarization advances the surface replacement generation; otherwise the original request error remains authoritative.', '', - 'The returned `agent/prompt-submit` allow is authoritative; listeners wrapping `next()` preserve downstream content and additional contexts unless replacement is intentional. Steering bypasses that waterfall and joins at its durable checkpoint.', + 'The returned `agent/pre-step` decision is authoritative; listeners wrapping `next()` preserve downstream messages unless replacement is intentional. Steering and injected context pass through the same waterfall after a later boundary claims their next-step batch.', '', 'SDK users that need replayable transcript data should consume `session/event`; `agent/*` is the live coordination surface for queue/status, prompt interception, request shaping, steering, continuation, and errors.', '', @@ -1192,8 +1306,8 @@ function renderDocs(): GraphDoc[] { function renderIndex(docs: GraphDoc[]): string { const labels: Record = { 'docs/capability-seams.md': 'capability seams and core services', + 'apps/cli/composition.md': 'dsh shared base composition', 'examples/headless-agent/composition.md': 'headless-agent app composition', - 'examples/tui-agent/composition.md': 'tui-agent app composition', 'examples/cordis-agent/composition.md': 'cordis-agent app composition', 'examples/acp-agent/composition.md': 'acp-agent app composition', 'docs/event-producer-consumer.md': 'event producer/consumer matrix', @@ -1202,8 +1316,8 @@ function renderIndex(docs: GraphDoc[]): string { } const modes: Record = { 'docs/capability-seams.md': 'hybrid generated', + 'apps/cli/composition.md': 'hybrid generated', 'examples/headless-agent/composition.md': 'hybrid generated', - 'examples/tui-agent/composition.md': 'hybrid generated', 'examples/cordis-agent/composition.md': 'hybrid generated', 'examples/acp-agent/composition.md': 'hybrid generated', 'docs/event-producer-consumer.md': 'hybrid generated', diff --git a/vitest.config.ts b/vitest.config.ts index 996f2456c7..5e6ac400ef 100644 --- a/vitest.config.ts +++ b/vitest.config.ts @@ -1,5 +1,16 @@ +import { spawnSync } from 'node:child_process' +import { fileURLToPath } from 'node:url' import tsconfigPaths from 'vite-tsconfig-paths' +import { resolvePwshPath } from './packages/bash/pwsh-local/src/resolve.ts' import { defineConfig } from 'vitest/config' +import { vitestExecArgv } from './vitest.shared.ts' +import { COVERAGE_EXEMPT_ENV, coverageExemptHeavySuites } from './scripts/coverage-exempt.ts' + +// Prints exact `path:line:col` records for every uncovered statement, branch +// path, and function when a file misses the per-file 100% gate — the built-in +// threshold ERRORs name only the file. Absolute path because istanbul-reports +// require()s custom reporters (which is also why the reporter is CJS). +const uncoveredLocationsReporter = fileURLToPath(new URL('./scripts/coverage-uncovered-locations.cjs', import.meta.url)) // Resolution facade shared by every plugin instance below: tsconfig.base.json // has no include, which vite-tsconfig-paths treats as match-all, so its paths @@ -9,7 +20,14 @@ const pathsPlugin = (): ReturnType => tsconfigPaths({ proj const windowsUnsupportedPackages = process.platform === 'win32' ? [ - 'packages/bash/*', + // Bash-requiring suites (a real POSIX shell is unavailable on Windows). + // The pwsh-requiring suites (pwsh-local, tool-pwsh) deliberately stay + // INCLUDED: PowerShell ships with Windows, so they run natively here. + // Replacing the old 'packages/bash/*' glob with this explicit list also + // newly INCLUDES packages/bash/bash (the pure seam package) on Windows. + 'packages/bash/bash-local', + 'packages/bash/bash-sandbox', + 'packages/bash/tool-bash', 'packages/hooks/*', 'packages/subprocess/*', 'packages/pty/pty-local', @@ -26,10 +44,20 @@ const windowsCoverageExclusions = process.platform === 'win32' 'packages/lsp/lsp-local/src/connection.ts', 'packages/lsp/lsp-local/src/index.ts', 'packages/lsp/lsp-local/src/instance.ts', - 'packages/ui/tui/src/index.ts', ] : [] +// Mirrors windowsCoverageExclusions: pwsh-local's run/start/lifecycle suites +// self-skip without a real pwsh (executor.spec.ts hasPwsh), leaving this file +// far below per-file 100% on pwsh-less hosts; the exemption keeps those hosts +// green while CI runners ship pwsh and still enforce the full bar. The probe +// runs the suites' own resolution (the dependency-free resolve.ts module), +// so the exemption is active exactly when the suites skip — a mismatched +// narrower probe could exempt the file on hosts whose suites actually run. +const pwshCoverageExclusions = spawnSync(resolvePwshPath(), ['-NoLogo', '-NoProfile', '-NonInteractive', '-Command', '$true'], { encoding: 'utf8' }).status === 0 + ? [] + : ['packages/bash/pwsh-local/src/index.ts'] + const testIncludes = [ 'packages/*/*/tests/**/*.spec.{ts,tsx}', 'apps/*/tests/**/*.spec.ts', @@ -37,6 +65,17 @@ const testIncludes = [ 'scripts/**/*.spec.ts', ] +// The instrumented coverage gate sets this env; the exempt heavy suites then +// run beside it uninstrumented (membership contract in scripts/coverage-exempt.ts). +// A set-but-not-'1' value is a misconfiguration, not a silent no-op. +const coverageExemptRaw = process.env[COVERAGE_EXEMPT_ENV] +if (coverageExemptRaw !== undefined && coverageExemptRaw !== '' && coverageExemptRaw !== '1') { + throw new Error(`vitest config: ${COVERAGE_EXEMPT_ENV} must be '1' or unset, got ${JSON.stringify(coverageExemptRaw)}.`) +} +const coverageExemptExcludes = coverageExemptRaw === '1' + ? coverageExemptHeavySuites.map(suite => suite.exclude) + : [] + // These suites exercise process-global state, process APIs, or timing-sensitive process I/O // that worker threads cannot isolate reliably under aggregate gate contention. // Keep the narrow exception in forks while the rest of the inventory avoids per-file processes. @@ -55,22 +94,27 @@ export default defineConfig({ // .tsx: client component specs (jsdom via per-file @vitest-environment pragma). include: testIncludes, exclude: windowsUnsupportedPackages.map(path => `${path}/tests/**/*.spec.ts`), - // One coverage invocation aggregates both projects. Most suites use threads - // for lower startup/IPC overhead; only explicit process-bound suites fork. + // One coverage invocation aggregates both projects. Regular suites fork on + // POSIX for Node stability and use threads on Windows; process-bound suites + // always fork. projects: [ { plugins: [pathsPlugin()], test: { name: 'thread-safe', - // Node 24 has aborted in its CJS lexer from a macOS arm64 worker - // thread. A fork contains that external runtime failure to the test - // process; other hosts retain the lower-overhead thread pool. - pool: process.platform === 'darwin' ? 'forks' : 'threads', + execArgv: vitestExecArgv, + // Node 24 has aborted in its CJS lexer (v8::ToLocalChecked Empty + // MaybeLocal in cjs_lexer::Parse) from worker threads on macOS + // arm64 and later on Linux. A fork contains that external runtime + // failure to the test process; Windows keeps the thread pool, where + // the abort has not reproduced and process spawn is costlier. + pool: process.platform === 'win32' ? 'threads' : 'forks', setupFiles: ['./scripts/test-invariants.ts'], include: testIncludes, exclude: [ ...windowsUnsupportedPackages.map(path => `${path}/tests/**/*.spec.ts`), ...processBoundTests, + ...coverageExemptExcludes, ], }, }, @@ -78,10 +122,14 @@ export default defineConfig({ plugins: [pathsPlugin()], test: { name: 'process-bound', + execArgv: vitestExecArgv, pool: 'forks', setupFiles: ['./scripts/test-invariants.ts'], include: processBoundTests, - exclude: windowsUnsupportedPackages.map(path => `${path}/tests/**/*.spec.ts`), + exclude: [ + ...windowsUnsupportedPackages.map(path => `${path}/tests/**/*.spec.ts`), + ...coverageExemptExcludes, + ], }, }, ], @@ -98,7 +146,8 @@ export default defineConfig({ 'packages/*/*/src/types.ts', 'packages/*/*/src/bin.ts', 'packages/*/*/src/worker.ts', - 'packages/code-runtime/code-runtime-subprocess/src/runner.ts', + // A killed executable lint-contract test can leave a non-product source probe behind. + 'packages/*/*/src/oxlint-contract-*.ts', // GUI step-1 skeleton (PR #500): client/web UI files whose remaining // branches need a browser-grade harness the jsdom lane doesn't cover // yet. TODO(gui): cover and remove as the client test lane matures. @@ -107,6 +156,7 @@ export default defineConfig({ 'packages/client/ui-primitives/src/markdown/plain-text.ts', 'packages/client/ui-question/src/client/QuestionComposer.tsx', 'packages/client/ui-primitives/src/Menu.tsx', + 'packages/client/ui-primitives/src/RiskConfirmation.tsx', 'packages/client/ui-workspace/src/client/WorkspaceBrowser.tsx', 'packages/client/ui-workspace/src/client/WorkspacePicker.tsx', 'packages/client/web-react/src/*', @@ -155,9 +205,13 @@ export default defineConfig({ 'packages/client/ui-sidebar/src/client/index.ts', 'packages/client/ui-skill/src/client/index.ts', 'packages/client/ui-workspace/src/client/index.ts', - 'packages/typert/generator/src/analyzer.ts', - 'packages/typert/generator/src/renderer.ts', - 'packages/typert/generator/src/cordis-catalog.ts', + 'packages/client/test-runtime/src/translate.ts', + 'packages/client/ui-primitives/src/JsonTree.tsx', + // Typert generator: correctness is pinned by its fixture suites and + // the byte-for-byte catalog reproduction test; per-file coverage + // would put whole-workspace compiler analysis under v8 + // instrumentation — the coverage lane's longest tail. + 'packages/typert/generator/src/*.ts', 'packages/host/apiproxy/src/index.ts', 'packages/host/apiproxy/src/invariant.ts', 'packages/host/apiproxy/src/api-proxy.ts', @@ -167,9 +221,9 @@ export default defineConfig({ 'packages/ui/commands/src/index.ts', 'packages/ui/commands/src/invariant.ts', 'packages/session-projection/session-projection/src/index.ts', - 'packages/ui/tui/src/index.ts', ...windowsUnsupportedPackages.map(path => `${path}/src/**/*.ts`), ...windowsCoverageExclusions, + ...pwshCoverageExclusions, ], // 100% or it doesn't merge (docs/testing.md: excessive tests are welcome). // Per-file so a well-covered big file can't subsidize a bare one. @@ -182,7 +236,9 @@ export default defineConfig({ functions: 100, lines: 100, }, - reporter: process.env.CI ? ['text'] : ['text', 'html'], + reporter: process.env.CI + ? ['text', uncoveredLocationsReporter] + : ['text', 'html', uncoveredLocationsReporter], }, }, }) From d0701296223994d82323b7c249aa20ce9a325b35 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Fri, 7 Aug 2026 21:09:51 +0800 Subject: [PATCH 059/108] fix(constraints): follow the spawn-helper relocation to subprocess-local The workspace-constraints allowlist pinned scripts/ensure-spawn-helper.mjs to dsh-pty-local; this branch relocates the helper (with node-pty ownership) into dsh-subprocess-local, so the published-files expectation moves with it. --- scripts/check-workspace-constraints.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/check-workspace-constraints.ts b/scripts/check-workspace-constraints.ts index 355b817e68..1c7bedc696 100644 --- a/scripts/check-workspace-constraints.ts +++ b/scripts/check-workspace-constraints.ts @@ -125,8 +125,8 @@ const packageFileExtras: Readonly> = { '@deepseek-ai/dsh-headless': ['cordis.patch.yml'], '@deepseek-ai/dsh-client-ui-theme': ['lib/styles'], '@deepseek-ai/dsh-helper': ['lib/assets'], - '@deepseek-ai/dsh-pty-local': ['scripts/ensure-spawn-helper.mjs'], '@deepseek-ai/dsh-skill-badge': ['assets'], + '@deepseek-ai/dsh-subprocess-local': ['scripts/ensure-spawn-helper.mjs'], '@deepseek-ai/dsh-scripts': [ 'lib/dev/tsdown-config.js', 'lib/local-plugin-loader-hooks.js', From fbceb8dcd55535e84f5b2ced570cbc613a00891a Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Fri, 7 Aug 2026 23:40:45 +0800 Subject: [PATCH 060/108] fix(pty-local): restore the session-resolved workspace policy MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The linear replay resurrected the July spawn path: mode folded inline and both confinement and the default cwd fenced to the deployment workspaceRoot. Master (and the pre-rebase merge result) resolve the policy once per spawn — resolve({ session }) — so session.header.cwd is both the workspace-write boundary and the default shell cwd, as the README already states. Restores the single resolve, threads the policy through spawnArgv and the cwd fallback, re-expresses master's session-root test on the terminal spec, and pins the recorded confine policy in the explicit-cwd test. Also from the same review round: the Python SDK exe build stages pty.node from subprocess-local (node-pty's home since the relocation), and fs-local's README counts the seam's eleven primitives. --- packages/fs/fs-local/README.i18n.yaml | 4 +-- packages/fs/fs-local/README.md | 2 +- packages/fs/fs-local/README.zh.md | 2 +- packages/pty/pty-local/src/index.ts | 20 +++++------ packages/pty/pty-local/tests/index.spec.ts | 40 ++++++++++++++++++++-- scripts/build-exe-for-python-sdk.ts | 2 +- 6 files changed, 52 insertions(+), 18 deletions(-) diff --git a/packages/fs/fs-local/README.i18n.yaml b/packages/fs/fs-local/README.i18n.yaml index e1870cbf41..250b94dacf 100644 --- a/packages/fs/fs-local/README.i18n.yaml +++ b/packages/fs/fs-local/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/fs/fs-local/README.md -README.md: fe0e5e9dec07fad745d6bea28da9009e517c7d85 -README.zh.md: 195f3963328035e9c6c382dd924cd04ee9e8c642 +README.md: 2e934298ceff75440357b0742770010c8b1c3904 +README.zh.md: 14f4867ce0f9eaae2a1dea98dbd7d401c98d4535 diff --git a/packages/fs/fs-local/README.md b/packages/fs/fs-local/README.md index fe0e5e9dec..2e934298ce 100644 --- a/packages/fs/fs-local/README.md +++ b/packages/fs/fs-local/README.md @@ -2,7 +2,7 @@ English | [中文](README.zh.md) -The **local-filesystem implementation** of the `ctx.fs` provider seam ([`@deepseek-ai/dsh-fs`](../fs)). Backs the twelve `FileSystem` primitives with the host filesystem; loading it as a plugin populates `ctx.fs`. +The **local-filesystem implementation** of the `ctx.fs` provider seam ([`@deepseek-ai/dsh-fs`](../fs)). Backs the eleven `FileSystem` primitives with the host filesystem; loading it as a plugin populates `ctx.fs`. ```ts ignore-check import { LocalFileSystem } from '@deepseek-ai/dsh-fs-local' diff --git a/packages/fs/fs-local/README.zh.md b/packages/fs/fs-local/README.zh.md index 195f396332..14f4867ce0 100644 --- a/packages/fs/fs-local/README.zh.md +++ b/packages/fs/fs-local/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -`ctx.fs` 提供方 seam([`@deepseek-ai/dsh-fs`](../fs))的**本地文件系统实现**。它使用宿主文件系统支持十二个 `FileSystem` 原语;将其作为插件加载会填充 `ctx.fs`。 +`ctx.fs` 提供方 seam([`@deepseek-ai/dsh-fs`](../fs))的**本地文件系统实现**。它使用宿主文件系统支持十一个 `FileSystem` 原语;将其作为插件加载会填充 `ctx.fs`。 ```ts ignore-check import { LocalFileSystem } from '@deepseek-ai/dsh-fs-local' diff --git a/packages/pty/pty-local/src/index.ts b/packages/pty/pty-local/src/index.ts index b1e0e6edbd..fa2237c959 100644 --- a/packages/pty/pty-local/src/index.ts +++ b/packages/pty/pty-local/src/index.ts @@ -10,7 +10,7 @@ import type { Session, SessionEvent } from '@deepseek-ai/dsh-session' import { PtyBackendCleanupError } from '@deepseek-ai/dsh-pty' import type { PtyBackend, PtyBackendSpawnSpec } from '@deepseek-ai/dsh-pty' import type { SubprocessTerminalHandle, SubprocessTerminalSpawnSpec } from '@deepseek-ai/dsh-subprocess' -import type { SandboxMode } from '@deepseek-ai/dsh-sandbox' +import type { SandboxExecutionPolicy } from '@deepseek-ai/dsh-sandbox' import { effectiveSandboxMode } from '@deepseek-ai/dsh-sandbox-policy' import { type Config, type ResolvedConfig, validateConfig } from './config.ts' import { LocalPtySession } from './session.ts' @@ -68,18 +68,15 @@ function childEnvironment(spec: PtyBackendSpawnSpec): Record { } } -function spawnArgv(ctx: Context, config: ResolvedConfig, spec: PtyBackendSpawnSpec): string[] { +function spawnArgv(ctx: Context, config: ResolvedConfig, policy: SandboxExecutionPolicy): string[] { const argv = [config.shellPath, ...config.shellArgs] - const mode: SandboxMode = effectiveSandboxMode(spec.owner.session.events) ?? ctx.sandboxPolicy.defaultMode - if (mode === 'danger-full-access') return argv + if (policy.mode === 'danger-full-access') return argv const sandbox = ctx.get('sandbox') if (sandbox === undefined) { - throw new Error(`pty-local: sandbox mode "${mode}" requires a ctx.sandbox provider in the execution world`) + throw new Error(`pty-local: sandbox mode "${policy.mode}" requires a ctx.sandbox provider in the execution world`) } - return sandbox.confine(argv, { - mode: mode, - workspaceRoot: ctx.sandboxPolicy.workspaceRoot, - }).argv + // Re-state the discriminant because object spread does not preserve its narrowed type. + return sandbox.confine(argv, { ...policy, mode: policy.mode }).argv } // TODO(pty-initialize-race-home): Fold this outer abort race into @@ -122,11 +119,12 @@ export class LocalPtyBackend implements PtyBackend { async spawn(spec: PtyBackendSpawnSpec): Promise { spec.signal?.throwIfAborted() ensureSandboxModeFence(this.ctx, spec.owner) - const argv = spawnArgv(this.ctx, this.config, spec) + const policy = this.ctx.sandboxPolicy.resolve({ session: spec.owner.session }) + const argv = spawnArgv(this.ctx, this.config, policy) if (argv[0] === undefined) throw new Error('pty-local: sandbox returned empty argv') const terminal = await this.spawnTerminal({ argv, - cwd: spec.cwd ?? this.ctx.sandboxPolicy.workspaceRoot, + cwd: spec.cwd ?? policy.workspaceRoot, env: childEnvironment(spec), rows: this.config.rows, cols: this.config.cols, diff --git a/packages/pty/pty-local/tests/index.spec.ts b/packages/pty/pty-local/tests/index.spec.ts index 9f10a2cde8..d61eb58da5 100644 --- a/packages/pty/pty-local/tests/index.spec.ts +++ b/packages/pty/pty-local/tests/index.spec.ts @@ -44,9 +44,9 @@ function config(): ResolvedConfig { } } -function agent(ctx: Context): Agent { +function agent(ctx: Context, cwd?: string): Agent { const id = SessionId('agent') - const session = Session.create(id, undefined, { version: 0, id, createdAt: 0 }) + const session = Session.create(id, undefined, { version: 0, id, createdAt: 0, ...cwd === undefined ? {} : { cwd } }) return { id, options: {}, session, inbox: new Inbox(session, { inserted: () => {}, discarded: () => {}, claimed: () => {} }), status: 'idle', @@ -213,6 +213,42 @@ describe('LocalPtyBackend startup rollback', () => { }) expect(spawned?.env?.PTY_TEST_SECRET).toBeUndefined() expect(initialized).toHaveBeenCalledWith(undefined) + expect((ctx.sandbox as RecordingSandbox).calls).toEqual([{ + argv: ['/bin/bash', '-i'], + policy: { mode: 'workspace-write', workspaceRoot: '/workspace' }, + }]) + }) + + it('resolves session mode and root together before wrapping the shell', async () => { + const ctx = new Context() + await ctx.plugin(RecordingSandbox) + await ctx.plugin(SandboxPolicyService, { mode: 'read-only', workspaceRoot: '/deployment-fallback' }) + const terminal = terminalHandle() + let spawned: SubprocessTerminalSpawnSpec | undefined + const spawnTerminal = async (spec: SubprocessTerminalSpawnSpec): Promise => { + spawned = spec + return terminal + } + const initialized = vi.fn<() => Promise>().mockResolvedValue(undefined) + const session = { initialize: initialized } as unknown as LocalPtySession + const backend = new LocalPtyBackend( + ctx, + { ...config(), shellArgs: ['-i'] }, + spawnTerminal, + () => session, + ) + const owner = agent(ctx, '/session-workspace') + setSandboxMode(owner.session, 'workspace-write') + expect(await backend.spawn(spec(owner))).toBe(session) + + expect(spawned).toMatchObject({ + argv: ['/sandbox', '--', '/bin/bash', '-i'], + cwd: '/session-workspace', + }) + expect((ctx.sandbox as RecordingSandbox).calls).toEqual([{ + argv: ['/bin/bash', '-i'], + policy: { mode: 'workspace-write', workspaceRoot: '/session-workspace' }, + }]) }) it('rejects a confined spawn without a sandbox provider', async () => { diff --git a/scripts/build-exe-for-python-sdk.ts b/scripts/build-exe-for-python-sdk.ts index db72f4c52e..536342dc89 100644 --- a/scripts/build-exe-for-python-sdk.ts +++ b/scripts/build-exe-for-python-sdk.ts @@ -326,7 +326,7 @@ class SingleExeBuild { if (this.cli.dryRun) console.log(`build-exe-for-python-sdk: [dry-run] rm -rf ${stagedBuild}`) else await rm(stagedBuild, { recursive: true, force: true }) if (target.platform !== 'linux') return - const source = join(root, 'packages', 'pty', 'pty-local', 'node_modules', 'node-pty', 'build', 'Release', 'pty.node') + const source = join(root, 'packages', 'subprocess', 'subprocess-local', 'node_modules', 'node-pty', 'build', 'Release', 'pty.node') const destination = join(stagedBuild, 'Release', 'pty.node') if (this.cli.dryRun) { console.log(`build-exe-for-python-sdk: [dry-run] cp ${source} ${destination}`) From d47df8ff9dc9d2eaabac8ae09449e325212c5bbf Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Sat, 8 Aug 2026 00:10:29 +0800 Subject: [PATCH 061/108] chore(rebase): refresh generated docs after the round-two replay Restores master's vitest config and packages-README pairing record (the replay had carried old-lineage copies; this branch claims neither), and regenerates the doc graphs, catalogs, and architecture pairing the replay's ours-side resolutions had flattened to master's content. --- docs/architecture.i18n.yaml | 4 ++-- docs/capability-seams.md | 11 +++++------ docs/event-producer-consumer.md | 8 ++++---- docs/module-graph.md | 15 ++++++++------- packages/README.i18n.yaml | 4 ++-- vitest.config.ts | 12 ++++++++---- 6 files changed, 29 insertions(+), 25 deletions(-) diff --git a/docs/architecture.i18n.yaml b/docs/architecture.i18n.yaml index de15631197..e0ddc02423 100644 --- a/docs/architecture.i18n.yaml +++ b/docs/architecture.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write docs/architecture.md -architecture.md: ea78faa62773a6b8ac98e5baab6e181ad6a3b7f0 -architecture.zh.md: 5a46dfaf84276de5f4dc0352a529bdd4e5d8c267 +architecture.md: e88bdac118e382d91c41c471d03182952d8a4508 +architecture.zh.md: 9314a3fbfc87d467a0f902fa1dae9e8f07e2eb28 diff --git a/docs/capability-seams.md b/docs/capability-seams.md index ae90e4a9bf..50ebe88d4f 100644 --- a/docs/capability-seams.md +++ b/docs/capability-seams.md @@ -21,7 +21,6 @@ flowchart LR pkg_session["session"] svc_sessions["ctx.sessions
In-memory session store"] pkg_agent["agent"] - pkg_cli_demo["cli-demo"] pkg_session_persistence["session-persistence"] pkg_session_query["session-query"] pkg_session_query_sqlite["session-query-sqlite"] @@ -91,6 +90,7 @@ flowchart LR svc_sessionProjectionCache["ctx.sessionProjectionCache
Persisted projection cache"] pkg_skill["skill"] svc_skills["ctx.skills
Skill provider registry"] + pkg_skill_badge["skill-badge"] pkg_skill_local["skill-local"] svc_agents["ctx.agents
Agent service"] pkg_acp["acp"] @@ -225,6 +225,7 @@ flowchart LR pkg_settings --> svc_settings pkg_settings_local --> svc_settings pkg_skill --> svc_skills + pkg_skill_badge --> svc_skills pkg_skill_local --> svc_skills pkg_spill --> svc_spillStore pkg_spill_local --> svc_spillStore @@ -260,7 +261,6 @@ flowchart LR svc_agentLoop --> pkg_agent_spine_demo svc_agents --> pkg_acp svc_agents --> pkg_agent_loop - svc_agents --> pkg_cli_demo svc_agents --> pkg_subagent_inprocess svc_approval --> pkg_tool_bash svc_approval --> pkg_tools @@ -307,7 +307,6 @@ flowchart LR svc_sessionQuery --> pkg_tool_session_query svc_sessions --> pkg_agent svc_sessions --> pkg_agent_loop - svc_sessions --> pkg_cli_demo svc_sessions --> pkg_invariants svc_sessions --> pkg_session_persistence svc_sessions --> pkg_session_query @@ -366,7 +365,7 @@ flowchart LR | `ctx.llm` | `seam` | [`llm`](../packages/llm/llm) | [`llm-deepseek`](../packages/llm/llm-deepseek), [`llm-pi-ai`](../packages/llm/llm-pi-ai), [`llm-replay`](../packages/support/llm-replay) | [`agent-loop`](../packages/core/agent-loop), [`compact-basic`](../packages/compact/compact-basic) | - | Adapters register provider implementations; the loop and compaction call the provider-neutral stream service. | | `ctx.tokenMeter` | `core` | [`token-meter`](../packages/llm/token-meter) | - | [`compact-basic`](../packages/compact/compact-basic) | - | Owns isolated per-session replay folds; pressure consumers share immutable revisioned measurements. | | `ctx.toolResultPrune` | `core` | [`compact-tool-result-prune`](../packages/compact/compact-tool-result-prune) | - | [`compact-basic`](../packages/compact/compact-basic) | - | Rewrites oversized current tool results through replayable single-node surface replacements before summary compaction. | -| `ctx.sessions` | `core` | [`session`](../packages/core/session) | - | [`agent-loop`](../packages/core/agent-loop), [`agent`](../packages/core/agent), [`cli-demo`](../packages/examples/cli-demo), [`session-persistence`](../packages/session-persistence/session-persistence), [`session-query`](../packages/session-query/session-query), [`session-query-sqlite`](../packages/session-query/session-query-sqlite), [`subagent-inprocess`](../packages/subagent/subagent-inprocess), [`invariants`](../packages/support/invariants) | - | Owns append-only Session instances and emits the durable session event feed. | +| `ctx.sessions` | `core` | [`session`](../packages/core/session) | - | [`agent-loop`](../packages/core/agent-loop), [`agent`](../packages/core/agent), [`session-persistence`](../packages/session-persistence/session-persistence), [`session-query`](../packages/session-query/session-query), [`session-query-sqlite`](../packages/session-query/session-query-sqlite), [`subagent-inprocess`](../packages/subagent/subagent-inprocess), [`invariants`](../packages/support/invariants) | - | Owns append-only Session instances and emits the durable session event feed. | | `ctx.invariants` | `core` | [`invariants`](../packages/support/invariants) | - | [`session`](../packages/core/session), [`agent`](../packages/core/agent), [`scope`](../packages/core/scope), [`agent-loop`](../packages/core/agent-loop) | - | Companion subpaths register owner-local checks; the service owns selection, uniqueness, child fibers, and package-attributed failures. | | `ctx.typert` | `core` | [`typert-registry`](../packages/typert/registry) | - | [`typert-loader`](../packages/typert/loader), [`api-gateway`](../packages/api/gateway) | - | Plugins register live zod contributions directly or through dsh-typert-loader; the API gateway consumes invocation descriptors and providers, while other runtime consumers query schemas and reflection metadata at their own edges. | | `ctx.typertGateway` | `core` | [`api-gateway`](../packages/api/gateway) | - | - | - | Associates generated Remote descriptors with live Cordis services, resolves registered identities, and exposes unary calls through the shared Connection RPC carrier. | @@ -387,8 +386,8 @@ flowchart LR | `ctx.commands` | `core` | [`commands`](../packages/ui/commands) | - | - | - | Plugins register direct human commands without sending invocations to the model. | | `ctx.sessionProjections` | `core` | [`session-projection`](../packages/session-projection/session-projection) | - | [`tool-todo`](../packages/todo/tool-todo), [`session-title`](../packages/session-title/session-title), [`host-apiproxy`](../packages/host/apiproxy) | - | Domains register state-driven fold units; the eager drive keeps per-session watermark states and api-proxy serves baselines and pushes changed values. | | `ctx.sessionProjectionCache` | `core` | [`session-projection-cache`](../packages/session-projection/session-projection-cache) | - | [`host-apiproxy`](../packages/host/apiproxy) | - | Durably checkpoints projection unit states per session (throttled + turn/end/detach mandatory points) and serves the cold-read ladder: cache row + persistence tail replay, so listings never load full logs. | -| `ctx.skills` | `seam` | [`skill`](../packages/skill/skill) | [`skill-local`](../packages/skill/skill-local) | [`tool-skill`](../packages/skill/tool-skill) | - | Merges provider skill catalogs; tool-skill renders the session-prefix catalog and loads complete skill bodies. | -| `ctx.agents` | `core` | [`agent`](../packages/core/agent) | - | [`agent-loop`](../packages/core/agent-loop), [`acp`](../packages/acp/acp), [`cli-demo`](../packages/examples/cli-demo), [`subagent-inprocess`](../packages/subagent/subagent-inprocess) | - | Owns live Agent handles, the create/resume factory seam, and process-local initiator propagation. | +| `ctx.skills` | `seam` | [`skill`](../packages/skill/skill) | [`skill-badge`](../packages/skill/skill-badge), [`skill-local`](../packages/skill/skill-local) | [`tool-skill`](../packages/skill/tool-skill) | - | Merges provider skill catalogs; tool-skill renders the session-prefix catalog and loads complete skill bodies. | +| `ctx.agents` | `core` | [`agent`](../packages/core/agent) | - | [`agent-loop`](../packages/core/agent-loop), [`acp`](../packages/acp/acp), [`subagent-inprocess`](../packages/subagent/subagent-inprocess) | - | Owns live Agent handles, the create/resume factory seam, and process-local initiator propagation. | | `ctx.agentLoop` | `bundle` | [`agent-loop`](../packages/core/agent-loop) | - | [`agent-spine-demo`](../packages/examples/agent-spine-demo) | - | The one concrete loop plugin; extension packages depend on dsh-agent events and services, not on this package. | | `ctx.goals` | `core` | [`goal`](../packages/goal/goal) | - | - | - | Folds revisioned objective state from the session log and keeps live continuation activation process-local. | | `ctx.subprocess` | `seam` | [`subprocess`](../packages/subprocess/subprocess) | [`subprocess-local`](../packages/subprocess/subprocess-local) | [`bash-local`](../packages/bash/bash-local), [`bash-sandbox`](../packages/bash/bash-sandbox), [`pty-local`](../packages/pty/pty-local), [`lsp-local`](../packages/lsp/lsp-local), [`subagent-acp`](../packages/subagent/subagent-acp), [`subagent-codex`](../packages/subagent/subagent-codex), [`subagent-claude-code`](../packages/subagent/subagent-claude-code) | - | The bash executors, the PTY shell backend, the LSP host, and the out-of-process ACP, Codex, and Claude Code subagent backends spawn through ctx.subprocess; the service owns process coordinates, tree/session lifetime, stdio dispositions, terminal mechanics, and kill escalation. | diff --git a/docs/event-producer-consumer.md b/docs/event-producer-consumer.md index acd2bc5e91..9699870ecc 100644 --- a/docs/event-producer-consumer.md +++ b/docs/event-producer-consumer.md @@ -24,9 +24,9 @@ This matrix shows which packages dispatch each harness-owned event and which pac | `commands/change` | `emit` | [`packages/ui/commands/src/index.ts:172`](../packages/ui/commands/src/index.ts) | [`commands`](../packages/ui/commands) (`events.dispatch`) | `apiproxy` | | `credentials/updated` | `emit` | [`packages/credentials/credentials/src/index.ts:67`](../packages/credentials/credentials/src/index.ts) | [`credentials`](../packages/credentials/credentials) (`events.dispatch`) | `apiproxy`, [`credentials`](../packages/credentials/credentials) | | `domain/changed` | `emit` | [`packages/storage/storage-domain/src/events.ts:46`](../packages/storage/storage-domain/src/events.ts) | [`storage-domain`](../packages/storage/storage-domain) (`emit`) | `apiproxy`, [`storage-domain`](../packages/storage/storage-domain), [`workspace`](../packages/workspace/workspace) | -| `fs/edit-intent` | `waterfall` | [`packages/fs/fs/src/index.ts:62`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`waterfall`), [`tool-str-replace-editor`](../packages/fs/tool-str-replace-editor) (`waterfall`) | [`fs-policy`](../packages/fs/fs-policy) | -| `fs/observed` | `emit` | [`packages/fs/fs/src/index.ts:71`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`emit`), [`tool-str-replace-editor`](../packages/fs/tool-str-replace-editor) (`emit`) | [`fs-policy`](../packages/fs/fs-policy), [`skill-local`](../packages/skill/skill-local) | -| `fs/write-intent` | `waterfall` | [`packages/fs/fs/src/index.ts:54`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`waterfall`), [`tool-str-replace-editor`](../packages/fs/tool-str-replace-editor) (`waterfall`) | [`fs-policy`](../packages/fs/fs-policy) | +| `fs/edit-intent` | `waterfall` | [`packages/fs/fs/src/index.ts:64`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`waterfall`), [`tool-str-replace-editor`](../packages/fs/tool-str-replace-editor) (`waterfall`) | [`fs-policy`](../packages/fs/fs-policy) | +| `fs/observed` | `emit` | [`packages/fs/fs/src/index.ts:73`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`emit`), [`tool-str-replace-editor`](../packages/fs/tool-str-replace-editor) (`emit`) | [`fs-policy`](../packages/fs/fs-policy), [`skill-local`](../packages/skill/skill-local) | +| `fs/write-intent` | `waterfall` | [`packages/fs/fs/src/index.ts:56`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`waterfall`), [`tool-str-replace-editor`](../packages/fs/tool-str-replace-editor) (`waterfall`) | [`fs-policy`](../packages/fs/fs-policy) | | `goal/changed` | `emit` | [`packages/goal/goal/src/domain.ts:114`](../packages/goal/goal/src/domain.ts) | [`goal`](../packages/goal/goal) (`emit`) | [`goal-session`](../packages/goal/goal-session) | | `llm/adapters-updated` | `emit` | [`packages/llm/llm/src/index.ts:73`](../packages/llm/llm/src/index.ts) | [`llm`](../packages/llm/llm) (`events.dispatch`) | `apiproxy`, [`llm`](../packages/llm/llm) | | `llm/stream` | `waterfall` | [`packages/llm/llm/src/index.ts:62`](../packages/llm/llm/src/index.ts) | [`llm`](../packages/llm/llm) (`waterfall`) | [`agent-loop`](../packages/core/agent-loop), [`llm`](../packages/llm/llm), [`llm-replay`](../packages/support/llm-replay), [`session-checkpoint-policy`](../packages/session-persistence/session-checkpoint-policy), [`session-title`](../packages/session-title/session-title) | @@ -65,7 +65,7 @@ This matrix shows which packages dispatch each harness-owned event and which pac | `connection/reset` | `runtime` (`emit`) | `ui-command`, `ui-models`, `ui-permission`, `ui-settings-general` | | `credentials/changed` | `runtime` (`emit`) | `ui-models` | | `internal/dispatch` | - | [`commands`](../packages/ui/commands), [`compact`](../packages/compact/compact), [`fs`](../packages/fs/fs), [`goal`](../packages/goal/goal), [`goal-session`](../packages/goal/goal-session), [`hook-protocol`](../packages/hooks/hook-protocol), [`llm-retry`](../packages/llm/llm-retry), [`permission`](../packages/ui/permission), [`plan-mode`](../packages/plan/plan-mode), [`pty-local`](../packages/pty/pty-local), `runtime`, [`sandbox-policy`](../packages/sandbox/sandbox-policy), [`scope`](../packages/core/scope), [`session`](../packages/core/session), [`session-title`](../packages/session-title/session-title), [`subagent`](../packages/subagent/subagent), [`time-context`](../packages/context/time-context), [`tool-todo`](../packages/todo/tool-todo), [`tools`](../packages/core/tools), [`user-approval`](../packages/ui/user-approval), [`workflow`](../packages/workflow/workflow) | -| `internal/plugin` | - | `hmr`, `loader`, `modules`, `webserver` | +| `internal/plugin` | - | `hmr`, `loader`, [`lsp-local`](../packages/lsp/lsp-local), `modules`, `webserver` | | `internal/service` | - | `gateway` | | `internal/status` | - | [`agent`](../packages/core/agent) | | `locale/change` | `locale` (`emit`) | `locale` | diff --git a/docs/module-graph.md b/docs/module-graph.md index 22af77f9c5..38a85a508a 100644 --- a/docs/module-graph.md +++ b/docs/module-graph.md @@ -491,12 +491,6 @@ flowchart TD pkg_code_runtime_worker --> pkg_invariants pkg_code_runtime_worker --> pkg_session pkg_code_runtime_worker --> pkg_timeout - pkg_lsp_local --> pkg_brand - pkg_lsp_local --> pkg_invariants - pkg_lsp_local --> pkg_llm - pkg_lsp_local --> pkg_lsp - pkg_lsp_local --> pkg_subprocess - pkg_lsp_local --> pkg_timeout pkg_sandbox_local --> pkg_invariants pkg_sandbox_local --> pkg_llm pkg_sandbox_local --> pkg_sandbox @@ -607,6 +601,13 @@ flowchart TD pkg_host_directory_picker_native --> pkg_client_ui_slots pkg_host_directory_picker_native --> pkg_client_ui_workspace pkg_host_directory_picker_native --> pkg_invariants + pkg_lsp_local --> pkg_brand + pkg_lsp_local --> pkg_fs + pkg_lsp_local --> pkg_invariants + pkg_lsp_local --> pkg_llm + pkg_lsp_local --> pkg_lsp + pkg_lsp_local --> pkg_subprocess + pkg_lsp_local --> pkg_timeout pkg_pty --> pkg_agent pkg_pty --> pkg_brand pkg_pty --> pkg_invariants @@ -1238,7 +1239,6 @@ flowchart TD | [`client-ui-theme`](../packages/client/ui-theme) | `client` | [`client-locale`](../packages/client/locale), [`client-runtime`](../packages/client/runtime), [`client-ui-primitives`](../packages/client/ui-primitives), [`client-ui-slots`](../packages/client/ui-slots), [`invariants`](../packages/support/invariants) | | [`client-ui-workspace`](../packages/client/ui-workspace) | `client` | [`client-locale`](../packages/client/locale), [`client-runtime`](../packages/client/runtime), [`client-ui-primitives`](../packages/client/ui-primitives), [`client-ui-slots`](../packages/client/ui-slots), [`invariants`](../packages/support/invariants) | | [`code-runtime-worker`](../packages/code-runtime/code-runtime-worker) | `code-runtime` | [`code-runtime`](../packages/code-runtime/code-runtime), [`invariants`](../packages/support/invariants), [`session`](../packages/core/session), [`timeout`](../packages/util/timeout) | -| [`lsp-local`](../packages/lsp/lsp-local) | `lsp` | [`brand`](../packages/util/brand), [`invariants`](../packages/support/invariants), [`llm`](../packages/llm/llm), [`lsp`](../packages/lsp/lsp), [`subprocess`](../packages/subprocess/subprocess), [`timeout`](../packages/util/timeout) | | [`sandbox-local`](../packages/sandbox/sandbox-local) | `sandbox` | [`invariants`](../packages/support/invariants), [`llm`](../packages/llm/llm), [`sandbox`](../packages/sandbox/sandbox) | | [`session-projection`](../packages/session-projection/session-projection) | `session-projection` | [`invariants`](../packages/support/invariants), [`session`](../packages/core/session) | | [`llm-retry`](../packages/llm/llm-retry) | `llm` | [`agent`](../packages/core/agent), [`invariants`](../packages/support/invariants), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`timeout`](../packages/util/timeout) | @@ -1266,6 +1266,7 @@ flowchart TD | [`tmux-context`](../packages/context/tmux-context) | `context` | [`agent`](../packages/core/agent), [`bash`](../packages/bash/bash), [`invariants`](../packages/support/invariants), [`session`](../packages/core/session) | | [`host-directory-picker-browse`](../packages/host/directory-picker-browse) | `host` | [`client-locale`](../packages/client/locale), [`client-runtime`](../packages/client/runtime), [`client-ui-primitives`](../packages/client/ui-primitives), [`client-ui-slots`](../packages/client/ui-slots), [`client-ui-workspace`](../packages/client/ui-workspace), [`invariants`](../packages/support/invariants) | | [`host-directory-picker-native`](../packages/host/directory-picker-native) | `host` | [`client-runtime`](../packages/client/runtime), [`client-ui-slots`](../packages/client/ui-slots), [`client-ui-workspace`](../packages/client/ui-workspace), [`invariants`](../packages/support/invariants) | +| [`lsp-local`](../packages/lsp/lsp-local) | `lsp` | [`brand`](../packages/util/brand), [`fs`](../packages/fs/fs), [`invariants`](../packages/support/invariants), [`llm`](../packages/llm/llm), [`lsp`](../packages/lsp/lsp), [`subprocess`](../packages/subprocess/subprocess), [`timeout`](../packages/util/timeout) | | [`pty`](../packages/pty/pty) | `pty` | [`agent`](../packages/core/agent), [`brand`](../packages/util/brand), [`invariants`](../packages/support/invariants) | | [`sandbox-policy`](../packages/sandbox/sandbox-policy) | `sandbox` | [`agent`](../packages/core/agent), [`invariants`](../packages/support/invariants), [`sandbox`](../packages/sandbox/sandbox), [`session`](../packages/core/session), [`system-prompt`](../packages/core/system-prompt) | | [`scripts`](../packages/sdk/scripts) | `sdk` | [`app-boot`](../packages/ui/app-boot), [`invariants`](../packages/support/invariants) | diff --git a/packages/README.i18n.yaml b/packages/README.i18n.yaml index d10f476f79..8aa9b92b91 100644 --- a/packages/README.i18n.yaml +++ b/packages/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/README.md -README.md: 8fbb6069a784a5bd45423a4e1ae11834a597750d -README.zh.md: 42a8d691344c716021188df6fd870a841d543f36 +README.md: 229feae568ba6e40a9c633696097eff46fd5bc95 +README.zh.md: b84aef020a7e3edf305df709d399fbc7b093b6a3 diff --git a/vitest.config.ts b/vitest.config.ts index 5e6ac400ef..6eb1bda354 100644 --- a/vitest.config.ts +++ b/vitest.config.ts @@ -3,7 +3,7 @@ import { fileURLToPath } from 'node:url' import tsconfigPaths from 'vite-tsconfig-paths' import { resolvePwshPath } from './packages/bash/pwsh-local/src/resolve.ts' import { defineConfig } from 'vitest/config' -import { vitestExecArgv } from './vitest.shared.ts' +import { standardDecoratorPlugin, vitestExecArgv } from './vitest.shared.ts' import { COVERAGE_EXEMPT_ENV, coverageExemptHeavySuites } from './scripts/coverage-exempt.ts' // Prints exact `path:line:col` records for every uncovered statement, branch @@ -88,7 +88,7 @@ const processBoundTests = [ ] export default defineConfig({ - plugins: [pathsPlugin()], + plugins: [pathsPlugin(), standardDecoratorPlugin()], test: { setupFiles: ['./scripts/test-invariants.ts'], // .tsx: client component specs (jsdom via per-file @vitest-environment pragma). @@ -99,7 +99,7 @@ export default defineConfig({ // always fork. projects: [ { - plugins: [pathsPlugin()], + plugins: [pathsPlugin(), standardDecoratorPlugin()], test: { name: 'thread-safe', execArgv: vitestExecArgv, @@ -119,7 +119,7 @@ export default defineConfig({ }, }, { - plugins: [pathsPlugin()], + plugins: [pathsPlugin(), standardDecoratorPlugin()], test: { name: 'process-bound', execArgv: vitestExecArgv, @@ -181,6 +181,10 @@ export default defineConfig({ 'packages/client/hmr/src/invariant.ts', 'packages/client/connection/src/index.ts', 'packages/client/connection/src/http-bridge.ts', + // This assembly imports generated Host-for-Client code that exists + // only in lib; the post-build built-bin smoke executes both entries. + 'packages/api/remotes/src/index.ts', + 'packages/api/remotes/src/client/index.ts', // Slash/command/input round: per-file gaps deferred with the same // client-lane debt. TODO(gui): cover and remove with the lane above. 'packages/client/connection/src/client/fixture.ts', From b9b25f81cb5659b7b73ab6ec049a53a36e26407a Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Sat, 8 Aug 2026 22:17:54 +0800 Subject: [PATCH 062/108] fix(subprocess-local): fence descendant adoption on the shell's start identity A recycled shell pid could donate an unrelated process's children to the terminal session's cleanup signalling: post-exit rescans queried processTree/processSession by numeric pid alone. The handle now captures the spawned shell's start identity at construction and adopts newly scanned members only while the root pid still carries it; already-adopted members keep their own identities, which every signal already rechecks. Regressions cover a recycled root donating an imposter child and a shell whose identity was never observable; the terminal fakes now model the root row the real /proc and ps scans include. Also from the review round: tool-pty's dependency list is re-sorted, and the LSP renderer documents the deliberate drive-letter reading of ambiguous file: URIs (display-only blast radius). --- packages/lsp/tool-lsp/src/render.ts | 3 ++ packages/pty/tool-pty/package.json | 2 +- .../subprocess-local/src/terminal.ts | 16 ++++++- .../subprocess-local/tests/local.spec.ts | 2 +- .../subprocess-local/tests/terminal.spec.ts | 48 ++++++++++++++++--- 5 files changed, 60 insertions(+), 11 deletions(-) diff --git a/packages/lsp/tool-lsp/src/render.ts b/packages/lsp/tool-lsp/src/render.ts index 761f5a7d34..ad77b2a6e2 100644 --- a/packages/lsp/tool-lsp/src/render.ts +++ b/packages/lsp/tool-lsp/src/render.ts @@ -146,6 +146,9 @@ export function renderUri(uri: string, workspaceUri: string): string { return uri } if (workspace.protocol !== 'file:') return uri + // A `file:` URI does not carry its world's OS, so a leading `/X:` segment is + // read as a Windows drive. A POSIX workspace literally rooted at `/c:/...` + // would mis-render (display only; edits and reads use the exact URI). const drivePath = /^\/[a-z](?::|%3A)/iu const windowsWorld = workspace.hostname.length > 0 || drivePath.test(workspace.pathname) const targetWindowsWorld = windowsWorld && (target.hostname.length > 0 || drivePath.test(target.pathname)) diff --git a/packages/pty/tool-pty/package.json b/packages/pty/tool-pty/package.json index 9027169941..f443bbbdf4 100644 --- a/packages/pty/tool-pty/package.json +++ b/packages/pty/tool-pty/package.json @@ -50,8 +50,8 @@ "@deepseek-ai/dsh-sandbox": "workspace:^", "@deepseek-ai/dsh-sandbox-policy": "workspace:^", "@deepseek-ai/dsh-session": "workspace:^", - "@deepseek-ai/dsh-system-prompt": "workspace:^", "@deepseek-ai/dsh-subprocess-local": "workspace:^", + "@deepseek-ai/dsh-system-prompt": "workspace:^", "@deepseek-ai/dsh-tasks": "workspace:^", "@deepseek-ai/dsh-tasks-local": "workspace:^", "@deepseek-ai/dsh-tool-tasks": "workspace:^", diff --git a/packages/subprocess/subprocess-local/src/terminal.ts b/packages/subprocess/subprocess-local/src/terminal.ts index 2d3d0bb78e..2a763948bb 100644 --- a/packages/subprocess/subprocess-local/src/terminal.ts +++ b/packages/subprocess/subprocess-local/src/terminal.ts @@ -43,6 +43,8 @@ export class LocalTerminalHandle implements SubprocessTerminalHandle { private cleanup: Promise | undefined private exited = false private trackedDescendants: ProcessIdentity[] = [] + /** The spawned shell's start identity; scans stop adopting members once the root pid no longer carries it. */ + private readonly rootIdentity: ProcessIdentity | undefined /** * @param terminal - allocated node-pty process. @@ -55,6 +57,7 @@ export class LocalTerminalHandle implements SubprocessTerminalHandle { private readonly graceMs: number, ) { this.pid = terminal.pid + this.rootIdentity = inspector.processTree(this.pid).find(member => member.pid === this.pid) this.done = this.outcome.promise this.dataDisposable = terminal.onData((data) => { this.output.write(Buffer.from(data, 'utf8')) }) this.exitDisposable = terminal.onExit(({ exitCode, signal: exitSignal }) => { @@ -112,10 +115,19 @@ export class LocalTerminalHandle implements SubprocessTerminalHandle { } private descendants(): ProcessIdentity[] { + // Adopt newly scanned members only while the numeric root pid provably + // still carries the spawned shell's start identity: after the shell dies, + // a recycled pid's tree and session must not donate an unrelated + // process's children to this session's signalling. Already-adopted + // members keep their own start identities, which every signal rechecks. + const tree = this.inspector.processTree(this.pid) + const root = tree.find(member => member.pid === this.pid) + const rootVerified = this.rootIdentity !== undefined + && root !== undefined + && root.started === this.rootIdentity.started this.trackedDescendants = this.survivors(this.unionMembers( this.trackedDescendants, - this.inspector.processTree(this.pid), - this.inspector.processSession(this.pid), + ...rootVerified ? [tree, this.inspector.processSession(this.pid)] : [], ).filter(member => member.pid !== this.pid)) return this.trackedDescendants } diff --git a/packages/subprocess/subprocess-local/tests/local.spec.ts b/packages/subprocess/subprocess-local/tests/local.spec.ts index 347b9c9330..c4e3f91522 100644 --- a/packages/subprocess/subprocess-local/tests/local.spec.ts +++ b/packages/subprocess/subprocess-local/tests/local.spec.ts @@ -249,7 +249,7 @@ describe('LocalSubprocessService', () => { ;(ctx.subprocess as InstanceType).terminalInspector = { foregroundPgid: () => 123, isStdinWaiting: () => false, - processTree: () => [{ pid: 124, started: 'child' }], + processTree: () => [{ pid: 123, started: 'shell' }, { pid: 124, started: 'child' }], processSession: () => [], isAlive: identity => alive.has(identity.pid), signalGroup: () => {}, diff --git a/packages/subprocess/subprocess-local/tests/terminal.spec.ts b/packages/subprocess/subprocess-local/tests/terminal.spec.ts index 8a6270883f..79501c7dc4 100644 --- a/packages/subprocess/subprocess-local/tests/terminal.spec.ts +++ b/packages/subprocess/subprocess-local/tests/terminal.spec.ts @@ -52,6 +52,8 @@ class FakePty { class FakeInspector implements ProcessInspector { pgid: number | undefined = 456 waiting = false + /** The shell's own row, present like the real /proc- and ps-backed scans; tests recycle or drop it. */ + root: ProcessIdentity | undefined = { pid: 123, started: 'shell' } members: ProcessIdentity[] = [] sessionMembers: ProcessIdentity[] = [] readonly alive = new Set() @@ -63,7 +65,7 @@ class FakeInspector implements ProcessInspector { foregroundPgid() { return this.pgid } isStdinWaiting() { return this.waiting } - processTree() { return this.members } + processTree() { return this.root === undefined ? this.members : [this.root, ...this.members] } processSession() { return this.sessionMembers } isAlive(identity: ProcessIdentity) { return this.alive.has(identity.pid) } signalGroup(pgid: number, signal: SubprocessTerminalSignal) { @@ -189,19 +191,50 @@ describe('LocalTerminalHandle', () => { expect(inspector.processes).toEqual([[124, 'SIGTERM']]) }) + it('does not adopt the children of a recycled shell pid', async () => { + const pty = new FakePty() + const inspector = new FakeInspector() + const handle = new LocalTerminalHandle(pty.asPty(), inspector, 10) + + pty.emitExit() + const imposterChild = { pid: 999, started: 'imposter-child' } + inspector.root = { pid: 123, started: 'imposter' } + inspector.members = [imposterChild] + inspector.alive.add(imposterChild.pid) + + await handle.terminate() + expect(inspector.processes).toEqual([]) + }) + + it('adopts nothing when the shell identity was never observable', async () => { + const pty = new FakePty() + const inspector = new FakeInspector() + inspector.root = undefined + const orphan = { pid: 321, started: 'unverifiable' } + inspector.members = [orphan] + inspector.alive.add(orphan.pid) + const handle = new LocalTerminalHandle(pty.asPty(), inspector, 10) + + await handle.terminate() + expect(inspector.processes).toEqual([]) + expect(pty.kills).toEqual(['SIGTERM']) + }) + it('rescans for descendants forked during TERM', async () => { const pty = new FakePty() const inspector = new FakeInspector() + const root = { pid: 123, started: 'shell' } let reads = 0 inspector.processTree = () => { reads += 1 - if (reads === 1) { - inspector.alive.add(124) - return [{ pid: 124, started: 'first' }] - } + if (reads === 1) return [root] if (reads === 2) { + inspector.alive.add(124) + return [root, { pid: 124, started: 'first' }] + } + if (reads === 3) { inspector.alive.add(125) - return [{ pid: 125, started: 'late' }] + return [root, { pid: 125, started: 'late' }] } return [] } @@ -256,9 +289,10 @@ describe('LocalTerminalHandle', () => { const pty = new FakePty() const inspector = new FakeInspector() const captured = { pid: 124, started: 'captured' } + const root = { pid: 123, started: 'shell' } let reads = 0 inspector.alive.add(captured.pid) - inspector.processTree = () => reads++ === 0 ? [captured] : [] + inspector.processTree = () => { reads += 1; return reads === 1 ? [root] : reads === 2 ? [root, captured] : [] } inspector.signalProcess = (identity, signal) => { inspector.processes.push([identity.pid, signal]) if (signal === 'SIGKILL') inspector.alive.delete(identity.pid) From e7b682f1f65fa9de386edffd94bd4c46c232a786 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Tue, 28 Jul 2026 10:05:30 +0800 Subject: [PATCH 063/108] feat(e2b): add remote runtime providers --- ...026-07-27-e2b-remote-runtime-poc.i18n.yaml | 6 + .../2026-07-27-e2b-remote-runtime-poc.md | 46 ++ .../2026-07-27-e2b-remote-runtime-poc.zh.md | 46 ++ docs/architecture.zh.md | 1 + docs/cordis-catalog/services.md | 15 + examples/headless-agent/README.i18n.yaml | 4 +- examples/headless-agent/README.md | 26 +- examples/headless-agent/README.zh.md | 26 +- examples/headless-agent/e2b.cordis.yml | 30 + .../tests/fixtures/e2b/e2b/bin.ts | 32 + .../tests/fixtures/e2b/e2b/cordis.yml | 19 + examples/package.json | 3 + knip.json | 11 + packages/README.i18n.yaml | 4 +- packages/README.md | 1 + packages/README.zh.md | 1 + packages/e2b/README.i18n.yaml | 6 + packages/e2b/README.md | 13 + packages/e2b/README.zh.md | 13 + packages/e2b/e2b/README.i18n.yaml | 6 + packages/e2b/e2b/README.md | 50 ++ packages/e2b/e2b/README.zh.md | 50 ++ packages/e2b/e2b/package.json | 44 ++ packages/e2b/e2b/src/index.ts | 240 ++++++ packages/e2b/e2b/src/invariant.ts | 30 + packages/e2b/e2b/tests/composition.e2e.ts | 39 + packages/e2b/e2b/tests/e2b.spec.ts | 218 ++++++ packages/e2b/e2b/tsconfig.json | 25 + packages/fs/README.i18n.yaml | 4 +- packages/fs/README.md | 3 +- packages/fs/README.zh.md | 3 +- packages/fs/fs-e2b/README.i18n.yaml | 6 + packages/fs/fs-e2b/README.md | 29 + packages/fs/fs-e2b/README.zh.md | 29 + packages/fs/fs-e2b/package.json | 41 + packages/fs/fs-e2b/src/index.ts | 423 ++++++++++ packages/fs/fs-e2b/src/invariant.ts | 27 + packages/fs/fs-e2b/tests/filesystem.spec.ts | 537 +++++++++++++ packages/fs/fs-e2b/tsconfig.json | 25 + packages/subprocess/README.i18n.yaml | 4 +- packages/subprocess/README.md | 7 +- packages/subprocess/README.zh.md | 11 +- .../subprocess-e2b/README.i18n.yaml | 6 + packages/subprocess/subprocess-e2b/README.md | 32 + .../subprocess/subprocess-e2b/README.zh.md | 32 + .../subprocess/subprocess-e2b/package.json | 41 + .../subprocess/subprocess-e2b/src/index.ts | 58 ++ .../subprocess-e2b/src/invariant.ts | 27 + .../subprocess/subprocess-e2b/src/output.ts | 70 ++ .../subprocess/subprocess-e2b/src/process.ts | 417 ++++++++++ .../subprocess-e2b/tests/subprocess.spec.ts | 725 ++++++++++++++++++ .../subprocess/subprocess-e2b/tsconfig.json | 25 + scripts/gen-cordis-catalog.ts | 1 + .../verify-package-readme-model-experience.ts | 3 + tsconfig.base.json | 1 + tsconfig.host.json | 3 + 56 files changed, 3565 insertions(+), 30 deletions(-) create mode 100644 .agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.i18n.yaml create mode 100644 .agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md create mode 100644 .agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.zh.md create mode 100644 examples/headless-agent/e2b.cordis.yml create mode 100644 examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts create mode 100644 examples/headless-agent/tests/fixtures/e2b/e2b/cordis.yml create mode 100644 packages/e2b/README.i18n.yaml create mode 100644 packages/e2b/README.md create mode 100644 packages/e2b/README.zh.md create mode 100644 packages/e2b/e2b/README.i18n.yaml create mode 100644 packages/e2b/e2b/README.md create mode 100644 packages/e2b/e2b/README.zh.md create mode 100644 packages/e2b/e2b/package.json create mode 100644 packages/e2b/e2b/src/index.ts create mode 100644 packages/e2b/e2b/src/invariant.ts create mode 100644 packages/e2b/e2b/tests/composition.e2e.ts create mode 100644 packages/e2b/e2b/tests/e2b.spec.ts create mode 100644 packages/e2b/e2b/tsconfig.json create mode 100644 packages/fs/fs-e2b/README.i18n.yaml create mode 100644 packages/fs/fs-e2b/README.md create mode 100644 packages/fs/fs-e2b/README.zh.md create mode 100644 packages/fs/fs-e2b/package.json create mode 100644 packages/fs/fs-e2b/src/index.ts create mode 100644 packages/fs/fs-e2b/src/invariant.ts create mode 100644 packages/fs/fs-e2b/tests/filesystem.spec.ts create mode 100644 packages/fs/fs-e2b/tsconfig.json create mode 100644 packages/subprocess/subprocess-e2b/README.i18n.yaml create mode 100644 packages/subprocess/subprocess-e2b/README.md create mode 100644 packages/subprocess/subprocess-e2b/README.zh.md create mode 100644 packages/subprocess/subprocess-e2b/package.json create mode 100644 packages/subprocess/subprocess-e2b/src/index.ts create mode 100644 packages/subprocess/subprocess-e2b/src/invariant.ts create mode 100644 packages/subprocess/subprocess-e2b/src/output.ts create mode 100644 packages/subprocess/subprocess-e2b/src/process.ts create mode 100644 packages/subprocess/subprocess-e2b/tests/subprocess.spec.ts create mode 100644 packages/subprocess/subprocess-e2b/tsconfig.json diff --git a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.i18n.yaml b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.i18n.yaml new file mode 100644 index 0000000000..13d081c28f --- /dev/null +++ b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md +2026-07-27-e2b-remote-runtime-poc.md: 307a08d6ca77c83bf836bc8d50b071fbf33ac49a +2026-07-27-e2b-remote-runtime-poc.zh.md: bb4e657d999760f000949d509228bbb65f22773a diff --git a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md new file mode 100644 index 0000000000..307a08d6ca --- /dev/null +++ b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md @@ -0,0 +1,46 @@ +# Agent Note: Shared E2B remote runtime POC + +Status: implemented + +English | [中文](2026-07-27-e2b-remote-runtime-poc.zh.md) + +## Problem + +A remote coding-agent backend is useful only when file operations and commands observe one coherent world. Attaching E2B independently at individual tools would allow a Bash command and a filesystem edit to address different sandboxes, while moving the complete harness into a remote VM would couple provider experimentation to agent, session, model, persistence, and deployment changes. + +## Decision + +The E2B integration is a provider-composition POC with one shared lifecycle owner and two capability implementations: + +- `@deepseek-ai/dsh-e2b` creates or reconnects one secure E2B sandbox, creates its working and private runtime directories, and owns kill/pause/leave disposal. +- `@deepseek-ai/dsh-fs-e2b` implements `ctx.fs` over that sandbox's Filesystem API. +- `@deepseek-ai/dsh-subprocess-e2b` implements `ctx.subprocess` over E2B Commands and remote Linux process groups. +- The existing `@deepseek-ai/dsh-bash-local` remains the Bash implementation because it delegates all process mechanics to `ctx.subprocess`. + +The owner is the sole source of sandbox identity. Providers inject it and never create private sandboxes. The composition therefore gives filesystem tools and Bash one remote cwd, process namespace, and spill/state directory while preserving the existing capability interfaces and model-facing tools. + +## POC boundary + +Only filesystem state, command processes during the provider lifetime, and adapter-owned remote files move into E2B. The host retains Cordis and plugin objects, the agent loop, agent/session state, session logs and persistence, model requests, skills, subagent orchestration, and E2B SDK buffers. The overlay does not upload or mount the host workspace; identical cwd strings name independent host and remote directories. Managed process groups still terminate and join when the subprocess service disposes, including before a retained-sandbox pause or leave disposition. + +The POC has no PTY adapter, LSP-specific integration, session-persistence backend, code-runtime backend, template builder, volume, snapshot, network-policy layer, sandbox catalog, or workspace synchronization. Retained sandbox reconnect proves lifecycle continuity only; it does not reconstruct host process handles, output cursors, or locks. + +## Verification + +Package tests pin lifecycle cleanup, filesystem seam semantics, subprocess group/stdio/abort behavior, and the package-owned invariant registrations. A credential-gated real Loader composition creates one sandbox, proves FS-write→Bash-read and Bash-write→FS-read in the same remote cwd, proves neither file appears in the host cwd, disposes the composition, and confirms the sandbox id is gone. + +## Alternatives considered + +**A separate E2B sandbox per capability or tool** — rejected because file and command operations would not share identity or state, defeating the coding-agent use case and multiplying lifecycle ownership. + +**Run the entire harness process inside E2B** — rejected because it changes deployment, credential flow, model transport, session durability, plugin loading, and supervision at once. Those questions are independent of proving the provider seams. + +**Add E2B-specific Bash, PTY, LSP, persistence, and synchronization packages together** — rejected because Bash already has the required subprocess seam and the other capabilities need separate consumer evidence and lifecycle designs. Their absence is an explicit fidelity boundary, not an incomplete hidden plan. + +**Implement filesystem operations through shell commands only** — rejected because that bypasses `ctx.fs` identity, structured errors, version guards, streaming reads, and atomic mutation semantics already consumed by the file tools. + +## Consequences + +The small composition demonstrates that existing capability seams can move an agent's mutable coding world off-host without changing the loop or model-facing tool packages. `sandboxId` plus pause/leave permits manual state retention for experiments, while kill remains the demo's cleanup policy. + +The provider is not interchangeable with the local subprocess backend for every consumer: remote startup cannot synchronously expose a PID, E2B retains complete command output in SDK memory, callback output is not byte-faithful, signal attribution is inferred, and reconnect cannot restore handles. Remote process/spill artifacts accumulate in a retained sandbox. These gaps remain documented POC constraints rather than compatibility shims or new cross-cutting abstractions. diff --git a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.zh.md b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.zh.md new file mode 100644 index 0000000000..bb4e657d99 --- /dev/null +++ b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.zh.md @@ -0,0 +1,46 @@ +# Agent Note: 共享 E2B 远程运行时 POC + +Status: implemented + +[English](2026-07-27-e2b-remote-runtime-poc.md) | 中文 + +## 问题 + +远程 coding agent(智能体)后端只有在文件操作与命令观察到同一个一致环境时才有用。若在各工具上分别接入 E2B,Bash 命令和文件系统编辑可能访问不同的沙箱;若把完整 harness 迁入远程 VM,则会把提供方实验与 agent、会话、模型、持久化及部署变更耦合在一起。 + +## 决策 + +E2B 集成是一个提供方组合 POC,由一个共享生命周期所有者和两个功能实现组成: + +- `@deepseek-ai/dsh-e2b` 创建或重新连接一个安全的 E2B 沙箱,创建其工作目录与私有运行时目录,并拥有 kill/pause/leave 资源释放操作。 +- `@deepseek-ai/dsh-fs-e2b` 在该沙箱的 Filesystem API 之上实现 `ctx.fs`。 +- `@deepseek-ai/dsh-subprocess-e2b` 在 E2B Commands 和远程 Linux 进程组之上实现 `ctx.subprocess`。 +- 现有的 `@deepseek-ai/dsh-bash-local` 继续作为 Bash 实现,因为它把所有进程机制委托给 `ctx.subprocess`。 + +该所有者是沙箱身份的唯一真源。提供方会注入该所有者,绝不创建私有沙箱。因此,该组合让文件系统工具与 Bash 共享一个远程 cwd、进程命名空间和 spill/状态目录,同时保留现有功能接口与面向模型的工具。 + +## POC 边界 + +只有文件系统状态、提供方存续期内的命令进程,以及适配器拥有的远程文件会迁入 E2B。宿主仍保留 Cordis 和插件对象、agent loop(智能体循环)、agent/会话状态、会话日志及其持久化、模型请求、skill(技能)、subagent 编排和 E2B SDK 缓冲。该 overlay 不会上传或挂载宿主工作区;拼写相同的 cwd 字符串分别指向彼此独立的宿主与远程目录。受管进程组仍会在进程管理服务 dispose(资源释放)时终止并等待退出,包括保留沙箱采用 `pause` 或 `leave` 处置方式之前。 + +本 POC 没有 PTY 适配器、LSP 专用集成、会话持久化后端、代码运行时后端、模板构建器、卷、快照、网络策略层、沙箱目录或工作区同步。保留沙箱后重新连接只能证明生命周期连续性;它不会重建宿主进程句柄、输出游标或锁。 + +## 验证 + +包测试固定生命周期清理、文件系统 seam 语义、进程管理的进程组/stdio/中止行为,以及包自有不变式注册。凭据门控的真实 Loader 组合会创建一个沙箱,证明同一远程 cwd 中 FS-write→Bash-read 和 Bash-write→FS-read 双向可见,证明两个文件均未出现在宿主 cwd 中,释放组合,并确认该沙箱 id 已不存在。 + +## 曾考虑的替代方案 + +**每项功能或每个工具使用独立的 E2B 沙箱。** 不予采纳,因为文件操作和命令操作将无法共享身份或状态,既违背 coding agent 用例,也会增加生命周期所有者的数量。 + +**在 E2B 内运行完整 harness 进程。** 不予采纳,因为这会同时改变部署、凭据流、模型传输、会话持久性、插件加载和监管方式。要证明提供方 seam,并不需要同时回答这些彼此独立的问题。 + +**同时添加 E2B 专用 Bash、PTY、LSP、持久化和同步包。** 不予采纳,因为 Bash 已经具备所需的进程管理 seam,其他功能则需要各自的消费方证据和生命周期设计。缺少它们是显式保真边界,而不是尚未公开的不完整计划。 + +**仅通过 shell 命令实现文件系统操作。** 不予采纳,因为这会绕过文件工具已经使用的 `ctx.fs` 身份、结构化错误、版本防护、流式读取和原子变更语义。 + +## 后果 + +这个小型组合证明,现有功能 seam 可以把 agent 的可变 coding 环境移出宿主,而无需改变循环或面向模型的工具包。`sandboxId` 与 `pause`/`leave` 允许实验手动保留状态,演示仍以 `kill` 作为清理策略。 + +该提供方并不能对所有消费方与本地进程管理后端互换:远程启动无法同步公开 PID,E2B 会在 SDK 内存中保留完整命令输出,回调输出并非字节保真,信号归因依靠推断,重新连接也无法恢复句柄。保留沙箱后会累积远程进程/spill 产物。这些缺口作为 POC 约束明确记录,而不会引入兼容垫片或新的跨领域抽象。 diff --git a/docs/architecture.zh.md b/docs/architecture.zh.md index 9314a3fbfc..2d1a0b43d5 100644 --- a/docs/architecture.zh.md +++ b/docs/architecture.zh.md @@ -25,6 +25,7 @@ |---|---|---| | `ctx.llm` | [`llm/`](../packages/llm/README.md) | 适配器注册表和模型流式调用 | | `ctx.tokenMeter` | [`llm/token-meter`](../packages/llm/token-meter/README.md) | 感知回放的单实例请求压力与表面压力 | +| `ctx.e2b` | [`e2b/`](../packages/e2b/README.md) | 共享 E2B 沙箱 | | `ctx.bash` | [`bash/`](../packages/bash/README.md) | 前台和后台命令执行 | | `ctx.subprocess` | [`subprocess/`](../packages/subprocess/README.md) | 可执行文件查找、受管进程树、终端 | | `ctx.pty` | [`pty/`](../packages/pty/README.md) | 按 owner 隔离的持久化终端会话 | diff --git a/docs/cordis-catalog/services.md b/docs/cordis-catalog/services.md index 7e125cd02c..7ce45c6817 100644 --- a/docs/cordis-catalog/services.md +++ b/docs/cordis-catalog/services.md @@ -578,6 +578,21 @@ abstract capability(): DirectoryPickerCapability Source: [`packages/host/directory-picker/src/index.ts:131`](../../packages/host/directory-picker/src/index.ts) +## `ctx.e2b` — `E2BSandboxService` + +Owns one lazily consumable E2B SDK handle and its final kill/pause/leave decision. The connection begins at plugin construction; adapters await getSandbox before their first operation. + +```ts cordis-catalog +/** + * Return the shared live SDK handle. + * @returns the created or reconnected sandbox after the configured cwd exists. + * @throws when E2B rejects creation/reconnection or the service is disposing. + */ +async getSandbox(): Promise +``` + +Source: [`packages/e2b/e2b/src/index.ts:97`](../../packages/e2b/e2b/src/index.ts) + ## `ctx.fs` — `FileSystem` (abstract seam) Abstract filesystem provider. Targets must preserve identity across aliases; reads expose regular UTF-8 text or typed errors, listings are stable and content-free, and mutations are atomic. Optional guards add stale protection without changing the unguarded provider contract. diff --git a/examples/headless-agent/README.i18n.yaml b/examples/headless-agent/README.i18n.yaml index 7b6139f28b..56c02d3e0a 100644 --- a/examples/headless-agent/README.i18n.yaml +++ b/examples/headless-agent/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write examples/headless-agent/README.md -README.md: e00f3d2d4fd21a860239f3d3a3e5eb2d7520f14a -README.zh.md: 6cd845783b1c112ba73676474b176ec28a4d0b78 +README.md: 16c45d304eb0f06a2b062518b053db683cd402c0 +README.zh.md: 72d2813d174e74a3bf93c363b268499a44f1cd4f diff --git a/examples/headless-agent/README.md b/examples/headless-agent/README.md index e00f3d2d4f..16c45d304e 100644 --- a/examples/headless-agent/README.md +++ b/examples/headless-agent/README.md @@ -2,7 +2,7 @@ English | [中文](README.zh.md) -This directory owns the replay and real-model test composition for a headless coding agent: DeepSeek V4 + local bash and filesystem tools + subagent delegation + workflows and fresh-agent Ralph iteration + `todo_write` + JSONL persistence. It explicitly mounts the shared agent spine, one root agent, persistence, and checkpoint policy; it is not a second product front door. +Headless one-shot agent wiring: DeepSeek V4 + local bash and filesystem tools + subagent delegation + workflows and fresh-agent Ralph iteration + `todo_write` + JSONL persistence, with [`@deepseek-ai/dsh-cli-demo`](../../packages/examples/cli-demo) as the app front door. ## Run it @@ -10,13 +10,27 @@ This directory owns the replay and real-model test composition for a headless co # repo root .env (gitignored) or exported env: # DEEPSEEK_API_KEY=sk-… # DEEPSEEK_BASE_URL=https://… # optional; defaults to the public API -pnpm run dsh run "fix the failing test in this workspace" +pnpm run demo:headless "fix the failing test in this workspace" +pnpm run demo:headless --output-format json -- "summarize the implementation" +pnpm run demo:headless --output-format stream-json -- "run the focused tests" ``` -The product command is [`dsh run`](../../apps/cli/README.md): it accepts one nonblank task, creates and persists a fresh session, prints the final assistant text, and exits. The root `demo:headless` script is only an alias of that command. +Exactly one nonblank positional task is required; quote tasks containing spaces. There is no `-p` flag. `text` prints the last text-bearing assistant message, `json` prints one DSH-native result record, and `stream-json` emits the top-level session's canonical task-turn events before that record. Child sessions surface only through parent tool events and results. -Snapshot suites run this directory's configuration through [`tests/fixtures/headless-driver.ts`](tests/fixtures/headless-driver.ts), an unexported test-only process that emits canonical session events as JSONL before its result record. That stream is test infrastructure, not a supported CLI output format. Child sessions surface only through parent tool events and results. +Each invocation creates and persists a fresh session, runs all model and tool steps in one turn, flushes, disposes, and exits. This is non-interactive automation: there is no prompt, approval, resume, second turn, or stdin context. The configured tools can mutate the launch workspace, run commands, spawn child agents, and consume provider tokens. -## Advanced configuration +## E2B POC overlay -[`advanced.cordis.yml`](advanced.cordis.yml) adds Code Mode and the Cordis tools to the test composition. +[`e2b.cordis.yml`](e2b.cordis.yml) replaces the local filesystem and subprocess providers with one shared E2B sandbox while retaining `dsh-bash-local` and the same model-facing tools. Put `E2B_API_KEY` beside `DEEPSEEK_API_KEY` in the gitignored root `.env`, then run: + +```sh +pnpm run demo:e2b "create hello.txt, read it back, and run pwd" +``` + +The overlay creates the same absolute cwd inside the sandbox, but it does not upload or mount the host workspace. File and Bash mutations exist only in E2B; Cordis, model calls, agent/session state, session logs, skills, and SDK buffers remain on the host. The demo kills its sandbox on timeout and disposal. It is a provider-composition POC, not a whole-harness migration or a workspace-sync feature. + +## Advanced and snapshot wiring + +[`advanced.cordis.yml`](advanced.cordis.yml) adds Code Mode and the Cordis tools to the shipped leaf. [`advanced.cordis.snapshot.yml`](advanced.cordis.snapshot.yml) replaces only the live LLM with replay. The tests under [`tests/`](tests/) own the keyless real-Loader smoke, key-gated world-verified smoke, and the `stream-json` replay snapshot with its parent and child session fixtures. + +The package-level [CLI contract](../../packages/examples/cli-demo/README.md) documents output records, exit status, cancellation, persistence, and model/token effects. diff --git a/examples/headless-agent/README.zh.md b/examples/headless-agent/README.zh.md index 6cd845783b..72d2813d17 100644 --- a/examples/headless-agent/README.zh.md +++ b/examples/headless-agent/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -本目录负责 headless coding agent(智能体)的回放和真实模型测试组装:DeepSeek V4 + 本地 bash 与文件系统工具 + subagent 委托 + 工作流与全新 agent Ralph 迭代 + `todo_write` + JSONL 持久化。本目录显式挂载共享 agent 主干、一个根 agent、持久化和检查点策略;它不是第二个产品入口。 +无头单次 agent(智能体)接线:DeepSeek V4 + 本地 bash 与文件系统工具 + subagent 委托 + 工作流与新 agent Ralph 迭代 + `todo_write` + JSONL 持久化,并以 [`@deepseek-ai/dsh-cli-demo`](../../packages/examples/cli-demo) 作为应用入口。 ## 运行 @@ -10,13 +10,27 @@ # repo root .env (gitignored) or exported env: # DEEPSEEK_API_KEY=sk-… # DEEPSEEK_BASE_URL=https://… # optional; defaults to the public API -pnpm run dsh run "fix the failing test in this workspace" +pnpm run demo:headless "fix the failing test in this workspace" +pnpm run demo:headless --output-format json -- "summarize the implementation" +pnpm run demo:headless --output-format stream-json -- "run the focused tests" ``` -产品命令是 [`dsh run`](../../apps/cli/README.md):它接受一项非空任务,创建并持久化新会话,打印最终 assistant 文本,然后退出。根目录的 `demo:headless` 脚本只是该命令的别名。 +必须提供且只能提供一个非空位置任务;含空格的任务需要加引号。没有 `-p` 标志。`text` 打印最后一条包含文本的 assistant 消息,`json` 打印一条 DSH 原生结果记录,`stream-json` 则在该记录之前发出顶层会话的规范任务轮次事件。子会话只通过父工具事件和结果对外显示。 -快照套件通过 [`tests/fixtures/headless-driver.ts`](tests/fixtures/headless-driver.ts) 运行本目录的配置。这个未导出且仅供测试使用的进程会在结果记录之前,以 JSONL 发出规范会话事件。该事件流属于测试基础设施,不是受支持的 CLI(命令行界面)输出格式。子会话只通过父会话的工具事件和结果对外显示。 +每次调用都会创建并持久化新会话,在一个轮次中运行所有模型和工具步骤,然后刷新、释放并退出。这是非交互式自动化:没有提示符、批准、恢复、第二轮次或 stdin 上下文。已配置工具可以修改启动 workspace、运行命令、spawn 子 agent,并消耗提供方 token。 -## 高级配置 +## E2B POC overlay -[`advanced.cordis.yml`](advanced.cordis.yml) 在测试组装中添加 Code Mode 和 Cordis 工具。 +[`e2b.cordis.yml`](e2b.cordis.yml) 使用一个共享 E2B 沙箱替换本地文件系统与进程管理提供方,同时保留 `dsh-bash-local` 和相同的面向模型工具。请在 git 忽略的根目录 `.env` 中,将 `E2B_API_KEY` 与 `DEEPSEEK_API_KEY` 放在一起,然后运行: + +```sh +pnpm run demo:e2b "create hello.txt, read it back, and run pwd" +``` + +该 overlay 会在沙箱中创建拼写相同的绝对 cwd,但不会上传或挂载宿主工作区。文件与 Bash 变更只存在于 E2B;Cordis、模型调用、agent/会话状态、会话日志、skill(技能)和 SDK 缓冲仍在宿主上。演示会在超时和资源释放时终止其沙箱。它是提供方组合 POC,而不是完整 harness 迁移或工作区同步功能。 + +## 高级与快照接线 + +[`advanced.cordis.yml`](advanced.cordis.yml) 在已交付叶节点上添加 Code Mode 和 Cordis 工具。[`advanced.cordis.snapshot.yml`](advanced.cordis.snapshot.yml) 只将实时 LLM(大语言模型)替换为回放。[`tests/`](tests/) 下的测试拥有无密钥真实 Loader 冒烟测试、密钥门控的外部状态验证冒烟测试,以及带父子会话 fixture(测试前置数据)的 `stream-json` 回放快照。 + +包级 [CLI 契约](../../packages/examples/cli-demo/README.md)记录输出记录、退出状态、取消、持久化以及模型/token 影响。 diff --git a/examples/headless-agent/e2b.cordis.yml b/examples/headless-agent/e2b.cordis.yml new file mode 100644 index 0000000000..4f961100b4 --- /dev/null +++ b/examples/headless-agent/e2b.cordis.yml @@ -0,0 +1,30 @@ +# POC overlay: keep the headless agent and model-facing tools, but place its +# filesystem and managed Bash process world in one short-lived E2B sandbox. +- id: base + name: '@cordisjs/plugin-include' + config: + path: ./cordis.yml + patches: + - id: subprocess + name: '@deepseek-ai/dsh-subprocess-local' + disabled: true + - id: fs-local + name: '@deepseek-ai/dsh-fs-local' + disabled: true + - id: bash + name: '@deepseek-ai/dsh-bash-local' + config: + cwd: !!js process.cwd() + timeoutMs: 60000 + - insert: + - id: e2b + name: '@deepseek-ai/dsh-e2b' + config: + cwd: !!js process.cwd() + timeoutMs: 300000 + onTimeout: kill + onDispose: kill + - id: subprocess-e2b + name: '@deepseek-ai/dsh-subprocess-e2b' + - id: fs-e2b + name: '@deepseek-ai/dsh-fs-e2b' diff --git a/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts b/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts new file mode 100644 index 0000000000..3b2cef609b --- /dev/null +++ b/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts @@ -0,0 +1,32 @@ +import { resolve } from 'node:path' +import { boot } from '@deepseek-ai/dsh-app-boot' +import type {} from '@deepseek-ai/dsh-e2b' +import type {} from '@deepseek-ai/dsh-fs-e2b' +import type {} from '@deepseek-ai/dsh-bash-local' + +const configPath = process.argv[2] +if (configPath === undefined) throw new Error('usage: bin.ts ') + +const ctx = await boot('e2b-composition', resolve(configPath)) +try { + const fromFs = await ctx.fs.resolve('from-fs.txt') + await ctx.fs.writeText(fromFs, 'written-by-fs\n', { kind: 'createIfAbsent' }) + const bashRead = await ctx.bash.run(ctx.bash.resolve({ command: 'cat from-fs.txt' })) + if (bashRead.exitCode !== 0 || bashRead.stdout.text !== 'written-by-fs\n') { + throw new Error(`E2B Bash could not read the FS write: ${JSON.stringify(bashRead)}`) + } + + const bashWrite = await ctx.bash.run(ctx.bash.resolve({ command: "printf 'written-by-bash\\n' > from-bash.txt" })) + if (bashWrite.exitCode !== 0) { + throw new Error(`E2B Bash could not write the shared filesystem: ${JSON.stringify(bashWrite)}`) + } + const fromBash = await ctx.fs.resolve('from-bash.txt') + const fsRead = await ctx.fs.readText(fromBash) + process.stdout.write(`${JSON.stringify({ + sandboxId: await ctx.e2b.sandboxId, + bashRead: bashRead.stdout.text, + fsRead, + })}\n`) +} finally { + await ctx.fiber.dispose() +} diff --git a/examples/headless-agent/tests/fixtures/e2b/e2b/cordis.yml b/examples/headless-agent/tests/fixtures/e2b/e2b/cordis.yml new file mode 100644 index 0000000000..08a7e7225c --- /dev/null +++ b/examples/headless-agent/tests/fixtures/e2b/e2b/cordis.yml @@ -0,0 +1,19 @@ +- id: e2b + name: '@deepseek-ai/dsh-e2b' + config: + cwd: !!js process.cwd() + timeoutMs: 60000 + onTimeout: kill + onDispose: kill + +- id: subprocess-e2b + name: '@deepseek-ai/dsh-subprocess-e2b' + +- id: bash + name: '@deepseek-ai/dsh-bash-local' + config: + cwd: !!js process.cwd() + timeoutMs: 30000 + +- id: fs-e2b + name: '@deepseek-ai/dsh-fs-e2b' diff --git a/examples/package.json b/examples/package.json index 1f6e99d240..48ac39f5ca 100644 --- a/examples/package.json +++ b/examples/package.json @@ -26,7 +26,9 @@ "@deepseek-ai/dsh-compact-basic": "workspace:*", "@deepseek-ai/dsh-compact-tool-result-prune": "workspace:*", "@deepseek-ai/dsh-credentials-local": "workspace:*", + "@deepseek-ai/dsh-e2b": "workspace:*", "@deepseek-ai/dsh-fs-local": "workspace:*", + "@deepseek-ai/dsh-fs-e2b": "workspace:*", "@deepseek-ai/dsh-fs-policy": "workspace:*", "@deepseek-ai/dsh-fs-sandbox": "workspace:^", "@deepseek-ai/dsh-goal": "workspace:*", @@ -76,6 +78,7 @@ "@deepseek-ai/dsh-subagent-fork": "workspace:*", "@deepseek-ai/dsh-subagent-spawn": "workspace:*", "@deepseek-ai/dsh-subprocess-local": "workspace:*", + "@deepseek-ai/dsh-subprocess-e2b": "workspace:*", "@deepseek-ai/dsh-system-prompt": "workspace:*", "@deepseek-ai/dsh-tasks-local": "workspace:*", "@deepseek-ai/dsh-time-context": "workspace:*", diff --git a/knip.json b/knip.json index ede8b2c738..d154cd646b 100644 --- a/knip.json +++ b/knip.json @@ -45,6 +45,7 @@ "headless-agent/tests/fixtures/time-context-mock-llm.ts", "headless-agent/tests/fixtures/telemetry-otel-driver.ts", "headless-agent/tests/fixtures/telemetry-redact-rule.ts", + "headless-agent/tests/fixtures/e2b/e2b/bin.ts", "acp-agent/tests/snapshots/lsp-definition/workspace/subject.ts", "acp-agent/tests/fixtures/child-question-tripwire.ts", "acp-agent/tests/fixtures/partial-landlock-sandbox.ts", @@ -229,6 +230,16 @@ "tests/**/*.ts" ] }, + "packages/e2b/e2b": { + "entry": [ + "tests/**/*.spec.ts", + "tests/**/*.e2e.ts" + ], + "project": [ + "src/**/*.ts", + "tests/**/*.ts" + ] + }, "packages/context/time-context": { "entry": [ "tests/**/*.spec.ts", diff --git a/packages/README.i18n.yaml b/packages/README.i18n.yaml index 8aa9b92b91..3eee08f35b 100644 --- a/packages/README.i18n.yaml +++ b/packages/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/README.md -README.md: 229feae568ba6e40a9c633696097eff46fd5bc95 -README.zh.md: b84aef020a7e3edf305df709d399fbc7b093b6a3 +README.md: 1b2a1737bcac33829a65fa937668ca9496035330 +README.zh.md: 2ef61e9c38750f4420ce99abc9943af1f3308b56 diff --git a/packages/README.md b/packages/README.md index 229feae568..54a5779016 100644 --- a/packages/README.md +++ b/packages/README.md @@ -16,6 +16,7 @@ Packages live at `packages///`; groups are containers, while names r | [`goal/`](goal/README.md) | Same-session goal persistence and lifecycle | Product — stable surface | | [`feedback/`](feedback/README.md) | Human feedback | Product — stable surface | | [`llm/`](llm/README.md) | LLM capability family: the abstract service + provider adapters | Product — stable surface | +| [`e2b/`](e2b/README.md) | E2B remote filesystem/process providers | POC | | [`subprocess/`](subprocess/README.md) | Subprocess capability family: spawn seam + local process-tree implementation | Product — stable surface | | [`bash/`](bash/README.md) | Bash capability family: executor seam, local impl, model-facing tool | Product — stable surface | | [`pty/`](pty/README.md) | Persistent PTY capability family: owner-scoped sessions, local implementation, and model-facing tools | Product — stable surface | diff --git a/packages/README.zh.md b/packages/README.zh.md index b84aef020a..46b112ec8f 100644 --- a/packages/README.zh.md +++ b/packages/README.zh.md @@ -16,6 +16,7 @@ | [`goal/`](goal/README.md) | 同会话 goal 的持久化与生命周期 | 产品:稳定表面 | | [`feedback/`](feedback/README.md) | 人类反馈 | 产品:稳定表面 | | [`llm/`](llm/README.md) | LLM(大语言模型)能力系列:抽象服务 + 提供方适配器 | 产品:稳定表面 | +| [`e2b/`](e2b/README.md) | E2B 远程文件系统/进程管理提供方 | POC | | [`subprocess/`](subprocess/README.md) | 进程管理能力系列:spawn seam + 本地进程树实现 | 产品:稳定表面 | | [`bash/`](bash/README.md) | Bash 能力系列:执行器 seam、本地实现、面向模型的工具 | 产品:稳定表面 | | [`pty/`](pty/README.md) | 持久 PTY 能力系列:按所有者隔离的会话、本地实现和面向模型的工具 | 产品:稳定表面 | diff --git a/packages/e2b/README.i18n.yaml b/packages/e2b/README.i18n.yaml new file mode 100644 index 0000000000..1bc9761930 --- /dev/null +++ b/packages/e2b/README.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write packages/e2b/README.md +README.md: 4b28ec7c0452189a579a6d7e0cd22a0b8561784d +README.zh.md: ab254eecad82c05cd6a6021e7f8bc4eda6cf3076 diff --git a/packages/e2b/README.md b/packages/e2b/README.md new file mode 100644 index 0000000000..4b28ec7c04 --- /dev/null +++ b/packages/e2b/README.md @@ -0,0 +1,13 @@ +# e2b/ — E2B remote runtime family + +English | [中文](README.zh.md) + +An experimental provider-composition POC that places the filesystem and managed subprocess world in one E2B Linux sandbox. The shared owner is separate from the capability adapters so every remote provider awaits the same sandbox identity and lifecycle. + +| Package | ctx key | Role | +|---|---|---| +| [`e2b`](e2b/README.md) (`@deepseek-ai/dsh-e2b`) | `ctx.e2b` | Create or reconnect one sandbox, create its working/runtime directories, expose the shared SDK handle, and apply the configured kill/pause/leave disposition | +| [`fs-e2b`](../fs/fs-e2b/README.md) (`@deepseek-ai/dsh-fs-e2b`) | `ctx.fs` | Implement the filesystem seam over E2B Filesystem APIs | +| [`subprocess-e2b`](../subprocess/subprocess-e2b/README.md) (`@deepseek-ai/dsh-subprocess-e2b`) | `ctx.subprocess` | Implement managed process groups, stdio projection, and remote spill files over E2B Commands | + +The existing [`dsh-bash-local`](../bash/bash-local/README.md) needs no E2B-specific fork: it already delegates process mechanics to `ctx.subprocess`, so replacing that provider places Bash in the same remote world as `ctx.fs`. This boundary does not move the harness process, Cordis objects, model calls, agent/session state, session persistence, skills, or E2B SDK buffers. The [decision record](../../.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md) owns the POC boundary and rejected expansion. diff --git a/packages/e2b/README.zh.md b/packages/e2b/README.zh.md new file mode 100644 index 0000000000..ab254eecad --- /dev/null +++ b/packages/e2b/README.zh.md @@ -0,0 +1,13 @@ +# e2b/ — E2B 远程运行时家族 + +[English](README.md) | 中文 + +这是一个实验性提供方组合 POC,把文件系统和受管子进程环境放进同一个 E2B Linux 沙箱。共享所有者与功能适配器彼此分离,使每个远程提供方都等待同一个沙箱身份和生命周期。 + +| 包(package) | ctx 键 | 职责 | +|---|---|---| +| [`e2b`](e2b/README.md)(`@deepseek-ai/dsh-e2b`) | `ctx.e2b` | 创建或重新连接一个沙箱,创建其工作目录与运行时目录,公开共享 SDK 句柄,并应用配置的 kill/pause/leave 处置方式 | +| [`fs-e2b`](../fs/fs-e2b/README.md)(`@deepseek-ai/dsh-fs-e2b`) | `ctx.fs` | 通过 E2B Filesystem API 实现文件系统 seam | +| [`subprocess-e2b`](../subprocess/subprocess-e2b/README.md)(`@deepseek-ai/dsh-subprocess-e2b`) | `ctx.subprocess` | 通过 E2B Commands 实现受管进程组、stdio 投影与远程 spill 文件 | + +现有的 [`dsh-bash-local`](../bash/bash-local/README.md) 无需 E2B 专用 fork:它已经把进程机制委托给 `ctx.subprocess`,因此替换该提供方即可让 Bash 与 `ctx.fs` 进入同一个远程环境。该边界不会迁移 harness 进程、Cordis 对象、模型调用、agent(智能体)/会话状态、会话持久化、skill(技能)或 E2B SDK 缓冲。[决策记录](../../.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md)负责说明 POC 边界及未采纳的扩展方案。 diff --git a/packages/e2b/e2b/README.i18n.yaml b/packages/e2b/e2b/README.i18n.yaml new file mode 100644 index 0000000000..9255c21577 --- /dev/null +++ b/packages/e2b/e2b/README.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write packages/e2b/e2b/README.md +README.md: bf62cb7d4811ca92f263bd3d337fff9fe41ce223 +README.zh.md: 8c7603aa501a2481aba6dab1f22750056e43ad26 diff --git a/packages/e2b/e2b/README.md b/packages/e2b/e2b/README.md new file mode 100644 index 0000000000..bf62cb7d48 --- /dev/null +++ b/packages/e2b/e2b/README.md @@ -0,0 +1,50 @@ +# @deepseek-ai/dsh-e2b + +English | [中文](README.zh.md) + +Shared lifecycle owner for one E2B sandbox. Filesystem and subprocess adapters inject `ctx.e2b`, await its single SDK handle, and therefore inhabit the same remote Linux working tree and process world. The package pins `e2b@2.29.1`. + +## Configuration + +```yaml +- id: e2b + name: '@deepseek-ai/dsh-e2b' + config: + cwd: /home/user/workspace + timeoutMs: 300000 + onTimeout: pause + onDispose: kill + +- id: subprocess-e2b + name: '@deepseek-ai/dsh-subprocess-e2b' + +- id: fs-e2b + name: '@deepseek-ai/dsh-fs-e2b' +``` + +`apiKey` is optional and otherwise reads `E2B_API_KEY`; the key configures the host SDK connection and is never installed in the sandbox. `cwd` defaults to `/home/user/workspace` and must be an absolute POSIX path. `timeoutMs` defaults to five minutes. `onTimeout` is `pause` by default and accepts `pause | kill`; it applies only when this service creates a sandbox. Pause-on-timeout enables E2B auto-resume so the shared SDK handle wakes on its next operation. `onDispose` defaults to `kill` and accepts `kill | pause | leave`. + +Set `sandboxId` to reconnect a running or paused sandbox instead of creating one. E2B resumes a paused sandbox during connect; `template` is creation-only and cannot accompany `sandboxId`. Omitting `template` uses E2B's default base template. + +## Lifecycle and ownership + +Construction starts one create/connect operation. Before resolving `getSandbox()`, the service creates `cwd` and the private `cwd/.dsh-e2b` adapter-state directory, then sets that directory to mode `0700`. `sandboxId` resolves to a branded `E2BSandboxId` after setup. + +Disposal first prevents new handle acquisition, then awaits setup and applies exactly one configured disposition. A newly created sandbox is killed when initial directory setup fails; a reconnected sandbox is not killed on setup failure because the service did not create it. Provider plugins must load after this owner and dispose before it. + +`pause` and `leave` retain remote filesystem and adapter artifacts for a later `sandboxId` connection, but a later harness process receives only a new SDK handle. The subprocess service still fulfills its seam contract by terminating managed groups before owner disposal; neither disposition recovers prior process objects, output cursors, or in-memory adapter locks. + +## Model Experience + +None, as this shared runtime owner registers no model-visible context; provider adapters and their consumers own any rendered effects. + +#### KV Cache effect + +No direct invalidation; this package does not contribute request tokens. + +## Known Limitations and Deferred Work + +- **This is not a whole-harness runtime** — Cordis services, agent/session state, session logs, LLM requests, skills, and SDK-side buffers stay in the host process. +- **Retained sandboxes do not restore host handles** — reconnect preserves remote files and adapter artifacts, but cannot reconstruct subprocess handles, stream cursors, or mutation locks; managed subprocesses terminate when their service disposes. +- **No deployment platform is configured** — templates, volumes, snapshots, network policy, host-workspace synchronization, and sandbox discovery are outside this POC. +- **`cwd` is a resolution convention, not containment** — adapters and commands can address other sandbox paths; E2B network access also retains the template's policy. diff --git a/packages/e2b/e2b/README.zh.md b/packages/e2b/e2b/README.zh.md new file mode 100644 index 0000000000..8c7603aa50 --- /dev/null +++ b/packages/e2b/e2b/README.zh.md @@ -0,0 +1,50 @@ +# @deepseek-ai/dsh-e2b + +[English](README.md) | 中文 + +一个 E2B 沙箱的共享生命周期所有者。文件系统与进程管理适配器注入 `ctx.e2b`,等待其唯一的 SDK 句柄,因此处于同一个远程 Linux 工作树与进程环境中。本包固定使用 `e2b@2.29.1`。 + +## 配置 + +```yaml +- id: e2b + name: '@deepseek-ai/dsh-e2b' + config: + cwd: /home/user/workspace + timeoutMs: 300000 + onTimeout: pause + onDispose: kill + +- id: subprocess-e2b + name: '@deepseek-ai/dsh-subprocess-e2b' + +- id: fs-e2b + name: '@deepseek-ai/dsh-fs-e2b' +``` + +`apiKey` 可省略;省略时读取 `E2B_API_KEY`。该密钥只配置宿主 SDK 连接,绝不会安装进沙箱。`cwd` 默认为 `/home/user/workspace`,并且必须是绝对 POSIX 路径。`timeoutMs` 默认为 5 分钟。`onTimeout` 默认为 `pause`,接受 `pause | kill`;它只在本服务创建沙箱时生效。超时时 pause 会启用 E2B 自动恢复,使共享 SDK 句柄在下一次操作时唤醒。`onDispose` 默认为 `kill`,接受 `kill | pause | leave`。 + +设置 `sandboxId` 可重新连接正在运行或已经暂停的沙箱,而不是创建新沙箱。连接时,E2B 会恢复已经暂停的沙箱;`template` 仅用于创建,不能与 `sandboxId` 同时使用。省略 `template` 时使用 E2B 的默认基础模板。 + +## 生命周期与所有权 + +构造阶段会启动一次 create/connect 操作。服务在 `getSandbox()` 结算前创建 `cwd` 和私有的 `cwd/.dsh-e2b` 适配器状态目录,再把该目录的 mode 设为 `0700`。初始化完成后,`sandboxId` 会结算为品牌类型 `E2BSandboxId`。 + +资源释放会先阻止继续获取新句柄,再等待初始化完成,并且只应用一种已配置的处置方式。新建沙箱的初始目录设置失败时,服务会终止该沙箱;重新连接的沙箱设置失败时不会被终止,因为它不是由本服务创建的。提供方插件必须在该所有者之后加载,并在其之前 dispose(资源释放)。 + +`pause` 和 `leave` 会保留远程文件系统及适配器产物,供稍后的 `sandboxId` 连接使用,但后续 harness 进程只会获得新的 SDK 句柄。进程管理服务仍会履行其 seam 契约,在所有者释放前终止受管进程组;这两种处置方式都不会恢复先前的进程对象、输出游标或内存中的适配器锁。 + +## 模型体验 + +无。本共享运行时所有者不注册模型可见上下文;提供方适配器及其消费方拥有所有渲染效果。 + +#### KV Cache 影响 + +不会直接失效;本包不会贡献请求 token。 + +## 已知限制与延后工作 + +- **这不是完整的 harness 运行时**:Cordis 服务、agent(智能体)/会话状态、会话日志、LLM(大语言模型)请求、skill(技能)和 SDK 侧缓冲仍留在宿主进程中。 +- **保留的沙箱不会恢复宿主句柄**:重新连接会保留远程文件和适配器产物,但无法重建进程管理句柄、流游标或变更锁;进程管理服务 dispose 时会终止受管子进程。 +- **没有配置部署平台**:模板、卷、快照、网络策略、宿主工作区同步和沙箱发现均不在本 POC 范围内。 +- **`cwd` 是解析约定,而不是包含边界**:适配器和命令可以访问沙箱中的其他路径;E2B 网络访问也继续采用模板的策略。 diff --git a/packages/e2b/e2b/package.json b/packages/e2b/e2b/package.json new file mode 100644 index 0000000000..7ecc5fabed --- /dev/null +++ b/packages/e2b/e2b/package.json @@ -0,0 +1,44 @@ +{ + "name": "@deepseek-ai/dsh-e2b", + "description": "Shared E2B sandbox lifecycle for DeepSeek Harness provider adapters", + "version": "0.0.1", + "private": true, + "type": "module", + "main": "lib/index.js", + "types": "lib/types/index.d.ts", + "exports": { + ".": { + "types": "./lib/types/index.d.ts", + "default": "./lib/index.js" + }, + "./invariant": { + "types": "./lib/types/invariant.d.ts", + "default": "./lib/invariant.js" + }, + "./src/*": "./src/*", + "./package.json": "./package.json" + }, + "files": [ + "lib/index.js", + "lib/invariant.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", + "src" + ], + "license": "BSD-3-Clause", + "peerDependencies": { + "@deepseek-ai/dsh-brand": "^0.0.1", + "@deepseek-ai/dsh-invariants": "^0.0.1", + "cordis": "^4.0.0-rc.7" + }, + "dependencies": { + "e2b": "2.29.1", + "schemastery": "^3.18.0" + }, + "devDependencies": { + "@deepseek-ai/dsh-brand": "workspace:^", + "@deepseek-ai/dsh-invariants": "workspace:^", + "@deepseek-ai/dsh-loader-smoke": "workspace:^", + "cordis": "^4.0.0-rc.7" + } +} diff --git a/packages/e2b/e2b/src/index.ts b/packages/e2b/e2b/src/index.ts new file mode 100644 index 0000000000..29cbf1dc53 --- /dev/null +++ b/packages/e2b/e2b/src/index.ts @@ -0,0 +1,240 @@ +/** + * Shared ownership of one E2B sandbox. Capability adapters await the same SDK + * handle, so filesystem and process operations inhabit one remote Linux world. + * @module @deepseek-ai/dsh-e2b + */ + +import { posix } from 'node:path' +import { Context, Service } from 'cordis' +import z from 'schemastery' +import { Sandbox } from 'e2b' +import type { Branded } from '@deepseek-ai/dsh-brand' + +export { + CommandExitError, + FileNotFoundError, + FileType, + Sandbox, + SandboxError, + SandboxNotFoundError, + TimeoutError, +} from 'e2b' +export type { CommandHandle, CommandResult, EntryInfo } from 'e2b' + +/** Opaque E2B sandbox identity used for reconnecting a later harness process. */ +export type E2BSandboxId = Branded<'E2BSandboxId'> + +/** + * Brand an SDK sandbox id after E2B has created or resolved it. + * @param value - E2B's opaque sandbox id. + * @returns the same string with the harness brand. + */ +export function E2BSandboxId(value: string): E2BSandboxId { + return value as E2BSandboxId +} + +/** + * Quote one opaque argument for the SDK's unavoidable `/bin/bash -l -c` layer. + * @param value - Exact argument value to preserve. + * @returns A single shell word with no interpolation. + */ +export function quoteE2BShellArg(value: string): string { + return `'${value.replaceAll('\'', "'\"'\"'")}'` +} + +/** Action taken on the owned sandbox when the Cordis service is disposed. */ +export type E2BDisposeMode = 'kill' | 'pause' | 'leave' + +/** Action E2B takes when a newly created sandbox reaches its lifetime. */ +export type E2BTimeoutMode = 'kill' | 'pause' + +/** Configuration for the shared E2B sandbox owner. */ +export interface Config { + /** API key; omission reads `E2B_API_KEY`. It is never forwarded into the sandbox. */ + apiKey?: string + /** Existing sandbox to reconnect instead of creating a new one. */ + sandboxId?: string + /** Template name or id for a newly created sandbox. */ + template?: string + /** Shared remote working directory, created before adapters receive the sandbox. */ + cwd?: string + /** E2B sandbox lifetime in milliseconds. */ + timeoutMs?: number + /** E2B action when a newly created sandbox reaches `timeoutMs`. */ + onTimeout?: E2BTimeoutMode + /** Disposal policy; `pause` and `leave` retain remote state for reconnect. */ + onDispose?: E2BDisposeMode +} + +interface ResolvedConfig { + apiKey: string + cwd: string + timeoutMs: number + onTimeout: E2BTimeoutMode + onDispose: E2BDisposeMode + sandboxId?: string + template?: string +} + +interface SchemaResolvedConfig extends Config { + cwd: string + timeoutMs: number + onTimeout: E2BTimeoutMode + onDispose: E2BDisposeMode +} + +declare module 'cordis' { + interface Context { + e2b: E2BSandboxService + } +} + +/** + * Owns one lazily consumable E2B SDK handle and its final kill/pause/leave + * decision. The connection begins at plugin construction; adapters await + * {@link getSandbox} before their first operation. + */ +export class E2BSandboxService extends Service { + static Config: z = z.object({ + apiKey: z.string(), + sandboxId: z.string(), + template: z.string(), + cwd: z.string().default('/home/user/workspace'), + timeoutMs: z.number().default(300_000), + onTimeout: z.union(['kill', 'pause'] as const).default('pause'), + onDispose: z.union(['kill', 'pause', 'leave'] as const).default('kill'), + }) + + /** Validated remote working directory shared by provider adapters. */ + readonly cwd: string + /** Remote directory reserved for adapter-owned process and terminal state. */ + readonly runtimeRoot: string + /** Whether this service creates a sandbox rather than reconnecting one. */ + readonly created: boolean + /** Configured action when a newly created sandbox reaches its lifetime. */ + readonly timeoutMode: E2BTimeoutMode + /** Configured final sandbox disposition. */ + readonly disposeMode: E2BDisposeMode + /** Sandbox id once E2B has created or resolved the remote runtime. */ + readonly sandboxId: Promise + + private readonly config: ResolvedConfig + private readonly ready: Promise + private disposed = false + + constructor(ctx: Context, config: Config) { + super(ctx, 'e2b') + // Schemastery fills these fields before construction; the type does not encode that step. + const resolved = config as SchemaResolvedConfig + const apiKey = config.apiKey ?? process.env.E2B_API_KEY + this.config = { + apiKey: apiKey ?? '', + cwd: resolved.cwd, + timeoutMs: resolved.timeoutMs, + onTimeout: resolved.onTimeout, + onDispose: resolved.onDispose, + ...(config.sandboxId !== undefined ? { sandboxId: config.sandboxId } : {}), + ...(config.template !== undefined ? { template: config.template } : {}), + } + this.validate() + this.cwd = this.config.cwd + this.runtimeRoot = posix.join(this.cwd, '.dsh-e2b') + this.created = this.config.sandboxId === undefined + this.timeoutMode = this.config.onTimeout + this.disposeMode = this.config.onDispose + this.ready = this.open() + // A deployment may load the owner before any adapter uses it. Keep a + // failed eager connection observed; getSandbox() still returns the error. + void this.ready.catch(() => {}) + this.sandboxId = this.ready.then(sandbox => E2BSandboxId(sandbox.sandboxId)) + void this.sandboxId.catch(() => {}) + + ctx.effect(() => async () => { + this.disposed = true + let sandbox: Sandbox + try { + sandbox = await this.ready + } catch { + // Connection creation already failed and is exposed by getSandbox(); + // there is no remote resource for teardown to own. + return + } + switch (this.config.onDispose) { + case 'kill': + await sandbox.kill() + return + case 'pause': { + await sandbox.pause() + return + } + case 'leave': + return + } + }, 'e2b sandbox teardown') + } + + /** + * Return the shared live SDK handle. + * @returns the created or reconnected sandbox after the configured cwd exists. + * @throws when E2B rejects creation/reconnection or the service is disposing. + */ + async getSandbox(): Promise { + if (this.disposed) throw new Error('E2B sandbox service is disposing') + return await this.ready + } + + private validate(): void { + if (this.config.apiKey.length === 0) { + throw new Error('dsh-e2b: configure apiKey or set E2B_API_KEY') + } + if (!posix.isAbsolute(this.config.cwd)) { + throw new Error(`dsh-e2b: cwd must be an absolute Linux path: ${this.config.cwd}`) + } + if (!Number.isFinite(this.config.timeoutMs) || this.config.timeoutMs <= 0) { + throw new Error('dsh-e2b: timeoutMs must be a positive finite number') + } + if (this.config.sandboxId !== undefined && this.config.sandboxId.length === 0) { + throw new Error('dsh-e2b: sandboxId must be non-empty when provided') + } + if (this.config.sandboxId !== undefined && this.config.template !== undefined) { + throw new Error('dsh-e2b: template applies only when creating; omit it when sandboxId reconnects') + } + } + + private async open(): Promise { + const connection = { + apiKey: this.config.apiKey, + timeoutMs: this.config.timeoutMs, + } + const sandbox = this.config.sandboxId === undefined + ? this.config.template === undefined + ? await Sandbox.create({ + ...connection, + secure: true, + lifecycle: { onTimeout: this.config.onTimeout, autoResume: this.config.onTimeout === 'pause' }, + }) + : await Sandbox.create(this.config.template, { + ...connection, + secure: true, + lifecycle: { onTimeout: this.config.onTimeout, autoResume: this.config.onTimeout === 'pause' }, + }) + : await Sandbox.connect(this.config.sandboxId, connection) + try { + await sandbox.files.makeDir(this.cwd) + await sandbox.files.makeDir(this.runtimeRoot) + await sandbox.commands.run(`chmod 700 -- ${quoteE2BShellArg(this.runtimeRoot)}`) + return sandbox + } catch (error: unknown) { + if (this.created) { + try { + await sandbox.kill() + } catch (_cleanupFailure) { + // The setup failure remains authoritative; E2B will still apply the configured lifetime. + } + } + throw error + } + } +} + +export default E2BSandboxService diff --git a/packages/e2b/e2b/src/invariant.ts b/packages/e2b/e2b/src/invariant.ts new file mode 100644 index 0000000000..891cabb2db --- /dev/null +++ b/packages/e2b/e2b/src/invariant.ts @@ -0,0 +1,30 @@ +/** + * Package-owned invariant companion for `@deepseek-ai/dsh-e2b`. + * @module @deepseek-ai/dsh-e2b/invariant + */ + +/* jscpd:ignore-start */ +import type { Context } from 'cordis' +import type { InvariantInstaller } from '@deepseek-ai/dsh-invariants' + +const PACKAGE_NAME = '@deepseek-ai/dsh-e2b' + +/** Cordis companion plugin name. */ +export const name = 'e2b-invariant' +/** Service required before the companion can reserve package ownership. */ +export const inject = ['invariants'] + +/** + * No runtime invariant: sandbox creation and teardown have one SDK promise and + * no independent event or mutable-data relationship to cross-check. + */ +const install: InvariantInstaller = () => {} + +/** + * Register this package's invariant companion. + * @param ctx - Cordis context carrying the invariant service. + * @returns the installed registration's disposer after setup succeeds. + */ +export const apply = (ctx: Context): Promise<() => void> => + Promise.resolve(ctx.invariants.register(PACKAGE_NAME, install)) +/* jscpd:ignore-end */ diff --git a/packages/e2b/e2b/tests/composition.e2e.ts b/packages/e2b/e2b/tests/composition.e2e.ts new file mode 100644 index 0000000000..8c01fae7b4 --- /dev/null +++ b/packages/e2b/e2b/tests/composition.e2e.ts @@ -0,0 +1,39 @@ +import { access } from 'node:fs/promises' +import { join } from 'node:path' +import { fileURLToPath } from 'node:url' +import { describe, expect, it } from 'vitest' +import { runLoaderSmoke } from '@deepseek-ai/dsh-loader-smoke' +import { Sandbox, SandboxNotFoundError } from '@deepseek-ai/dsh-e2b' + +const fixtureRoot = fileURLToPath(new URL('../../../../examples/headless-agent/tests/fixtures/e2b/e2b/', import.meta.url)) +const binScript = join(fixtureRoot, 'bin.ts') +const configPath = join(fixtureRoot, 'cordis.yml') +const tsconfigPath = fileURLToPath(new URL('../../../../tsconfig.json', import.meta.url)) + +describe.skipIf(!process.env.E2B_API_KEY)('E2B live Loader composition', () => { + it('shares remote state across FS and Bash without creating host workspace files', async () => { + const { stdout, stderr } = await runLoaderSmoke({ + label: 'E2B composition', + tempDirPrefix: 'dsh-e2b-composition-', + binScript, + libBinScript: binScript, + configPath, + tsconfigPath, + processTimeoutMs: 90_000, + inspect: async (cwd) => { + await expect(access(join(cwd, 'from-fs.txt'))).rejects.toMatchObject({ code: 'ENOENT' }) + await expect(access(join(cwd, 'from-bash.txt'))).rejects.toMatchObject({ code: 'ENOENT' }) + }, + }) + + expect(stderr).toBe('') + const output = JSON.parse(stdout) as Record + expect(output).toMatchObject({ + bashRead: 'written-by-fs\n', + fsRead: 'written-by-bash\n', + }) + const apiKey = process.env.E2B_API_KEY + if (apiKey === undefined) throw new Error('E2B_API_KEY disappeared during the live composition test') + await expect(Sandbox.getInfo(String(output.sandboxId), { apiKey })).rejects.toBeInstanceOf(SandboxNotFoundError) + }, 105_000) +}) diff --git a/packages/e2b/e2b/tests/e2b.spec.ts b/packages/e2b/e2b/tests/e2b.spec.ts new file mode 100644 index 0000000000..c842731f5c --- /dev/null +++ b/packages/e2b/e2b/tests/e2b.spec.ts @@ -0,0 +1,218 @@ +import { beforeEach, describe, expect, it, vi } from 'vitest' +import { Context } from 'cordis' +import type { Sandbox as SandboxType } from 'e2b' +import E2BSandboxService, { + E2BSandboxId, + quoteE2BShellArg, +} from '@deepseek-ai/dsh-e2b' +import * as E2BInvariant from '../src/invariant.ts' +import InvariantService from '@deepseek-ai/dsh-invariants' + +const sdk = vi.hoisted(() => ({ + create: vi.fn(), + connect: vi.fn(), +})) + +vi.mock('e2b', async (importOriginal) => { + const actual = await importOriginal() + // The mock replaces only the SDK's static factory surface and is never constructed. + // eslint-disable-next-line @typescript-eslint/no-extraneous-class + class FakeSandbox { + static create(...args: unknown[]): unknown { + return sdk.create(...args) + } + + static connect(...args: unknown[]): unknown { + return sdk.connect(...args) + } + } + return { ...actual, Sandbox: FakeSandbox } +}) + +interface SandboxFixture { + sandbox: SandboxType + makeDir: ReturnType + run: ReturnType + kill: ReturnType + pause: ReturnType +} + +function fakeSandbox(id = 'sandbox-1'): SandboxFixture { + const makeDir = vi.fn().mockResolvedValue(true) + const run = vi.fn().mockResolvedValue({ exitCode: 0, stdout: '', stderr: '' }) + const kill = vi.fn().mockResolvedValue(undefined) + const pause = vi.fn().mockResolvedValue(true) + const sandbox = { + sandboxId: id, + files: { makeDir }, + commands: { run }, + kill, + pause, + } as unknown as SandboxType + return { sandbox, makeDir, run, kill, pause } +} + +beforeEach(() => { + sdk.create.mockReset() + sdk.connect.mockReset() + vi.unstubAllEnvs() +}) + +describe('E2BSandboxService', () => { + it('creates one protected shared sandbox and kills it on default disposal', async () => { + const fixture = fakeSandbox() + sdk.create.mockResolvedValue(fixture.sandbox) + const ctx = new Context() + const fiber = await ctx.plugin(E2BSandboxService, { apiKey: 'test-key' }) + + const service = ctx.e2b + await expect(service.getSandbox()).resolves.toBe(fixture.sandbox) + await expect(service.sandboxId).resolves.toBe(E2BSandboxId('sandbox-1')) + expect(service.cwd).toBe('/home/user/workspace') + expect(service.runtimeRoot).toBe('/home/user/workspace/.dsh-e2b') + expect(service.created).toBe(true) + expect(service.timeoutMode).toBe('pause') + expect(service.disposeMode).toBe('kill') + expect(sdk.create).toHaveBeenCalledWith({ + apiKey: 'test-key', + timeoutMs: 300_000, + secure: true, + lifecycle: { onTimeout: 'pause', autoResume: true }, + }) + expect(fixture.makeDir).toHaveBeenNthCalledWith(1, '/home/user/workspace') + expect(fixture.makeDir).toHaveBeenNthCalledWith(2, '/home/user/workspace/.dsh-e2b') + expect(fixture.run).toHaveBeenCalledWith("chmod 700 -- '/home/user/workspace/.dsh-e2b'") + + await fiber.dispose() + expect(fixture.kill).toHaveBeenCalledOnce() + await expect(service.getSandbox()).rejects.toThrow(/disposing/) + }) + + it('creates from a template, honors timeout and pause policies, and reads the key from the environment', async () => { + vi.stubEnv('E2B_API_KEY', 'environment-key') + const fixture = fakeSandbox('template-sandbox') + sdk.create.mockResolvedValue(fixture.sandbox) + const ctx = new Context() + const fiber = await ctx.plugin(E2BSandboxService, { + template: 'agent-template', + cwd: '/workspace/project', + timeoutMs: 60_000, + onTimeout: 'kill', + onDispose: 'pause', + }) + await ctx.e2b.getSandbox() + + expect(sdk.create).toHaveBeenCalledWith('agent-template', { + apiKey: 'environment-key', + timeoutMs: 60_000, + secure: true, + lifecycle: { onTimeout: 'kill', autoResume: false }, + }) + await fiber.dispose() + expect(fixture.pause).toHaveBeenCalledOnce() + expect(fixture.kill).not.toHaveBeenCalled() + }) + + it('accepts an already-paused result during configured pause disposal', async () => { + const fixture = fakeSandbox() + fixture.pause.mockResolvedValue(false) + sdk.create.mockResolvedValue(fixture.sandbox) + const ctx = new Context() + const fiber = await ctx.plugin(E2BSandboxService, { apiKey: 'test-key', onDispose: 'pause' }) + await ctx.e2b.getSandbox() + await fiber.dispose() + expect(fixture.pause).toHaveBeenCalledOnce() + }) + + it('reconnects without applying creation lifecycle options and can leave state running', async () => { + const fixture = fakeSandbox('existing') + sdk.connect.mockResolvedValue(fixture.sandbox) + const ctx = new Context() + const fiber = await ctx.plugin(E2BSandboxService, { + apiKey: 'test-key', + sandboxId: 'existing', + timeoutMs: 90_000, + onDispose: 'leave', + }) + await ctx.e2b.getSandbox() + + expect(ctx.e2b.created).toBe(false) + expect(sdk.connect).toHaveBeenCalledWith('existing', { apiKey: 'test-key', timeoutMs: 90_000 }) + expect(sdk.create).not.toHaveBeenCalled() + await fiber.dispose() + expect(fixture.kill).not.toHaveBeenCalled() + expect(fixture.pause).not.toHaveBeenCalled() + }) + + it('kills a newly created sandbox when remote directory setup fails', async () => { + const fixture = fakeSandbox() + fixture.makeDir.mockRejectedValueOnce(new Error('setup failed')) + sdk.create.mockResolvedValue(fixture.sandbox) + const ctx = new Context() + const fiber = await ctx.plugin(E2BSandboxService, { apiKey: 'test-key' }) + + await expect(ctx.e2b.getSandbox()).rejects.toThrow('setup failed') + await expect(ctx.e2b.sandboxId).rejects.toThrow('setup failed') + expect(fixture.kill).toHaveBeenCalledOnce() + await fiber.dispose() + }) + + it('preserves the setup failure even when cleanup also fails', async () => { + const fixture = fakeSandbox() + fixture.run.mockRejectedValueOnce(new Error('chmod failed')) + fixture.kill.mockRejectedValueOnce(new Error('cleanup failed')) + sdk.create.mockResolvedValue(fixture.sandbox) + const ctx = new Context() + await ctx.plugin(E2BSandboxService, { apiKey: 'test-key' }) + await expect(ctx.e2b.getSandbox()).rejects.toThrow('chmod failed') + }) + + it('does not kill a reconnected sandbox when setup fails', async () => { + const fixture = fakeSandbox() + fixture.makeDir.mockRejectedValueOnce(new Error('setup failed')) + sdk.connect.mockResolvedValue(fixture.sandbox) + const ctx = new Context() + await ctx.plugin(E2BSandboxService, { apiKey: 'test-key', sandboxId: 'existing' }) + await expect(ctx.e2b.getSandbox()).rejects.toThrow('setup failed') + expect(fixture.kill).not.toHaveBeenCalled() + }) + + it.each([ + [{ apiKey: '' }, /configure apiKey/], + [{ apiKey: 'x', cwd: 'relative' }, /absolute Linux path/], + [{ apiKey: 'x', timeoutMs: 0 }, /positive finite/], + [{ apiKey: 'x', sandboxId: '' }, /sandboxId must be non-empty/], + [{ apiKey: 'x', sandboxId: 'one', template: 'two' }, /template applies only/], + ] as const)('fails self-contained configuration before opening E2B: %j', async (config, message) => { + vi.stubEnv('E2B_API_KEY', '') + const ctx = new Context() + await expect(ctx.plugin(E2BSandboxService, config)).rejects.toThrow(message) + expect(sdk.create).not.toHaveBeenCalled() + expect(sdk.connect).not.toHaveBeenCalled() + }) + + it('requires a key when both config and the environment omit it', async () => { + const original = process.env.E2B_API_KEY + delete process.env.E2B_API_KEY + try { + const ctx = new Context() + await expect(ctx.plugin(E2BSandboxService, {})).rejects.toThrow(/configure apiKey/) + } finally { + if (original === undefined) delete process.env.E2B_API_KEY + else process.env.E2B_API_KEY = original + } + }) +}) + +describe('E2B helpers and invariant companion', () => { + it('quotes opaque shell arguments without interpolation', () => { + expect(quoteE2BShellArg("a'b $HOME")).toBe("'a'\"'\"'b $HOME'") + }) + + it('registers the package-owned empty invariant installer', async () => { + const ctx = new Context() + await ctx.plugin(InvariantService, { enabled: true }) + const fiber = await ctx.plugin(E2BInvariant).await() + await fiber.dispose() + }) +}) diff --git a/packages/e2b/e2b/tsconfig.json b/packages/e2b/e2b/tsconfig.json new file mode 100644 index 0000000000..5890a41d66 --- /dev/null +++ b/packages/e2b/e2b/tsconfig.json @@ -0,0 +1,25 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": ["src"], + "references": [ + { + "path": "../../../vendor/cosmokit" + }, + { + "path": "../../../vendor/cordis" + }, + { + "path": "../../../vendor/schemastery" + }, + { + "path": "../../util/brand" + }, + { + "path": "../../support/invariants" + } + ] +} diff --git a/packages/fs/README.i18n.yaml b/packages/fs/README.i18n.yaml index c1a381cb81..eabb6a602e 100644 --- a/packages/fs/README.i18n.yaml +++ b/packages/fs/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/fs/README.md -README.md: b6adabd5744cb2b3dcee78b71815f8e95ba780f1 -README.zh.md: 0841f538932452921d2b0d7d9534f023672f241d +README.md: 8ac8807cc9ca4b5cc31b686cef9796a8e42b7f81 +README.zh.md: 49705790cc0cdf1f7199251e09af0af01f704efb diff --git a/packages/fs/README.md b/packages/fs/README.md index b6adabd574..460af7c53b 100644 --- a/packages/fs/README.md +++ b/packages/fs/README.md @@ -8,12 +8,13 @@ The filesystem stack: a provider seam (execution-world paths, bounded text IO, a |---|---|---| | `fs/` | Provider seam: canonical process paths/file URIs/containment, text IO, and atomic mutation primitives; owns the `fs/*` policy events | `ctx.fs` | | `fs-local/` | Local-filesystem `FileSystem` implementation | (registers `ctx.fs`) | +| `fs-e2b/` | E2B-backed `FileSystem` implementation sharing the remote runtime owned by `ctx.e2b` | (registers `ctx.fs`) | | `fs-sandbox/` | Sandbox-enforcing `FileSystem`: extends `fs-local` and fences write/edit by the per-call mode + workspace root policy (read-only denies, workspace-write contains to the session workspace + temp roots), reads pass through | (registers `ctx.fs`) | | `fs-policy/` | Policy gate plugin: observed-state + read-before-edit + version-guarded write/edit, via the `fs/*` event gate | (no service — `fs/*` listeners) | | `tool-fs/` | Model-facing `read`/`write`/`edit` tools AND the executor (reads via `ctx.fs`, owns read windowing, dispatches `fs/*`); preserves filesystem semantics for session-cwd-relative paths and advertises sandbox escalation fields when the mounted `ctx.fs` confines | (registers on `ctx.tools`) | | `tool-fs-search/` | Model-facing `glob`/`grep` discovery tools when `rg` is available on the bash executor `PATH`, backed by fixed ripgrep commands through `ctx.bash`, NOT by `ctx.fs` provider methods | (registers on `ctx.tools`) | -The interface lives at `fs/fs/`. A sandboxed, remote, or project-scoped filesystem backend can replace `fs-local` without touching the seam, the policy gate, or the model-facing tool schemas — `fs-sandbox` is the first such replacement (an in-process path fence over the shared sandbox mode; see [the cross-family fs sandbox Agent Note](../../.agents/notes/implemented/feature/2026-07-14-cross-family-fs-sandbox.md)). The policy (`fs-policy/`) is a plugin that participates only through the `fs/*` event gate, not a service the tool injects — so dropping it gracefully loses the policy and leaves the unconstrained bare provider rather than breaking the tool. A deployment that loads `tool-fs/` is expected to also load it. The mode fence and the read-before-edit gate are orthogonal and compose. Discovery (`tool-fs-search/`) deliberately does NOT extend the provider seam: search is a process-backed `rg` workflow on the bash executor, so filesystem backends stay free of a universal search contract; its tools register only when that executor can find `rg`, and its results are follow-up-readable when the bash workdir and the `read` root are the same workspace (the co-located deployment its README documents). +The interface lives at `fs/fs/`. A sandboxed, remote, or project-scoped filesystem backend can replace `fs-local` without touching the seam, the policy gate, or the model-facing tool schemas: `fs-sandbox` provides an in-process path fence over the shared sandbox mode ([decision](../../.agents/notes/implemented/feature/2026-07-14-cross-family-fs-sandbox.md)), while `fs-e2b` places file state in the remote runtime shared with the E2B subprocess provider ([POC decision](../../.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md)). The policy (`fs-policy/`) is a plugin that participates only through the `fs/*` event gate, not a service the tool injects — so dropping it gracefully loses the policy and leaves the unconstrained bare provider rather than breaking the tool. A deployment that loads `tool-fs/` is expected to also load it. The mode fence and the read-before-edit gate are orthogonal and compose. Discovery (`tool-fs-search/`) deliberately does NOT extend the provider seam: search is a process-backed `rg` workflow on the bash executor, so filesystem backends stay free of a universal search contract; its tools register only when that executor can find `rg`, and its results are follow-up-readable when the bash workdir and the `read` root are the same workspace (the co-located deployment its README documents). ## No timeouts on file IO diff --git a/packages/fs/README.zh.md b/packages/fs/README.zh.md index 0841f53893..5f70e3cb50 100644 --- a/packages/fs/README.zh.md +++ b/packages/fs/README.zh.md @@ -8,12 +8,13 @@ |---|---|---| | `fs/` | 提供方 seam:规范化进程路径、文件 URI 与包含关系、文本 I/O 和原子变更原语;拥有 `fs/*` 政策事件 | `ctx.fs` | | `fs-local/` | 本地文件系统 `FileSystem` 实现 | (注册 `ctx.fs`) | +| `fs-e2b/` | 以 E2B 为后端的 `FileSystem` 实现,共享由 `ctx.e2b` 拥有的远程运行时 | (注册 `ctx.fs`) | | `fs-sandbox/` | 强制沙箱的 `FileSystem`:扩展 `fs-local`,并按每次调用的模式与工作区根政策约束写入/编辑(只读模式拒绝,工作区写入模式限制在会话工作区与临时根目录内);读取直接通过 | (注册 `ctx.fs`) | | `fs-policy/` | 政策门禁插件:通过 `fs/*` 事件门禁提供已观察状态、编辑前读取和版本防护的写入/编辑 | (无服务,仅有 `fs/*` 监听器) | | `tool-fs/` | 面向模型的 `read`/`write`/`edit` 工具以及执行器(通过 `ctx.fs` 读取,拥有读取窗口逻辑,分派 `fs/*`);为会话 cwd 相对路径保留文件系统语义,并在已挂载的 `ctx.fs` 实施约束时声明沙箱升级字段 | (注册到 `ctx.tools`) | | `tool-fs-search/` | 面向模型的 `glob`/`grep` 发现工具;当 `rg` 位于 bash 执行器 `PATH` 上时注册,通过 `ctx.bash` 运行固定 ripgrep 命令,而不是使用 `ctx.fs` 提供方方法 | (注册到 `ctx.tools`) | -接口位于 `fs/fs/`。沙箱化、远程或限定项目作用域的文件系统后端可以替换 `fs-local`,而无需更改 seam、政策门禁或面向模型的工具 schema;`fs-sandbox` 是第一个这样的替代实现(基于共享沙箱模式的进程内路径围栏;见[跨能力族 fs 沙箱 Agent Note](../../.agents/notes/implemented/feature/2026-07-14-cross-family-fs-sandbox.md))。政策(`fs-policy/`)是一个只通过 `fs/*` 事件门禁参与的插件,不是工具注入的服务;因此移除它会平稳失去政策,留下不受约束的裸提供方,而不会破坏工具。加载 `tool-fs/` 的部署也应加载该插件。模式围栏与编辑前读取门禁彼此正交,可以组合。发现(`tool-fs-search/`)有意不扩展提供方 seam:搜索是在 bash 执行器上运行 `rg`、由进程支持的工作流,因此文件系统后端无需承担通用搜索契约;只有当执行器能找到 `rg` 时,其工具才会注册。如果 bash 工作目录与 `read` 根目录是同一工作区,结果就能继续读取,这也是其 README 所述的共置部署。 +接口位于 `fs/fs/`。沙箱化、远程或限定项目作用域的文件系统后端可以替换 `fs-local`,而无需更改 seam、政策门禁或面向模型的工具 schema:`fs-sandbox` 基于共享沙箱模式提供进程内路径围栏([决策](../../.agents/notes/implemented/feature/2026-07-14-cross-family-fs-sandbox.md)),而 `fs-e2b` 则把文件状态置于与 E2B 进程管理提供方共享的远程运行时中([POC 决策](../../.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md))。政策(`fs-policy/`)是一个只通过 `fs/*` 事件门禁参与的插件,不是工具注入的服务;因此移除它会平稳失去政策,留下不受约束的裸提供方,而不会破坏工具。加载 `tool-fs/` 的部署也应加载该插件。模式围栏与编辑前读取门禁彼此正交,可以组合。发现(`tool-fs-search/`)有意不扩展提供方 seam:搜索是在 bash 执行器上运行 `rg`、由进程支持的工作流,因此文件系统后端无需承担通用搜索契约;只有当执行器能找到 `rg` 时,其工具才会注册。如果 bash 工作目录与 `read` 根目录是同一工作区,结果就能继续读取,这也是其 README 所述的共置部署。 ## 文件 I/O 不设超时 diff --git a/packages/fs/fs-e2b/README.i18n.yaml b/packages/fs/fs-e2b/README.i18n.yaml new file mode 100644 index 0000000000..d3ea2786b2 --- /dev/null +++ b/packages/fs/fs-e2b/README.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write packages/fs/fs-e2b/README.md +README.md: 2b23884e345e7c39b0559465afa0bc588699b70a +README.zh.md: 1a9a174ac90c59b68b02887303f58ee2a99dc72e diff --git a/packages/fs/fs-e2b/README.md b/packages/fs/fs-e2b/README.md new file mode 100644 index 0000000000..2b23884e34 --- /dev/null +++ b/packages/fs/fs-e2b/README.md @@ -0,0 +1,29 @@ +# @deepseek-ai/dsh-fs-e2b + +English | [中文](README.zh.md) + +E2B implementation of the [`@deepseek-ai/dsh-fs`](../fs/README.md) provider seam. It has no config: load [`@deepseek-ai/dsh-e2b`](../../e2b/e2b/README.md) first, then this service in place of `dsh-fs-local`. The provider uses the owner's remote cwd and SDK handle, so file tools observe the same world as E2B-backed Bash processes. + +## Behavior + +- **Remote identity and metadata** — relative paths resolve as POSIX paths against the caller cwd or `ctx.e2b.cwd`; `realpath -m` supplies canonical target identity without requiring the final file to exist. `stat`, no-follow `lstat`, and stable one-level directory listings project E2B metadata into the filesystem seam. Versions are opaque hashes of E2B metadata plus a per-write extended attribute. +- **UTF-8 reads** — whole reads and streamed reads preserve cross-chunk decoding, reject invalid UTF-8, and use the seam's 8192-byte NUL sample for binary detection. The model-facing tool still owns size selection and line windowing. +- **Atomic mutations** — writes upload a mode-`0600` temporary sibling, preserve an existing file's POSIX mode, and publish through same-directory Linux `mv -f`. E2B creates missing parent directories. Literal edits LF-normalize for matching, restore dominant CRLF storage, and serialize mutations per canonical target within the host process. Optional create/version guards keep the base seam's observed-state semantics. +- **Failures and cancellation** — E2B not-found, permission, abort, and other controller failures map to the existing `FsError` vocabulary. Cancellation is best-effort at SDK request boundaries; a successful rename is the commit point. + +The provider does not copy, mount, or reconcile the host workspace. Giving it a host path as `cwd` creates a remote directory with the same spelling only. + +## Model Experience + +Indirectly, through [`dsh-tool-fs`](../tool-fs/README.md), which renders remote UTF-8 content, directory results, mutation acknowledgements, and provider errors while E2B identity and transport remain internal. + +#### KV Cache effect + +No direct invalidation; the named consumer owns any request-prefix changes. + +## Known Limitations and Deferred Work + +- **No host synchronization** — an empty E2B cwd stays empty until a tool, command, template, or external process populates it; local files are neither uploaded nor reflected back. +- **Mutation coordination is host-process-local** — another harness connection or remote command can race the adapter; version guards detect only metadata changes represented by E2B. +- **Whole-file mutation costs remain** — overwrite diffs and literal edits read complete files into host memory, and every operation incurs E2B controller latency. +- **Custom templates must support the used Linux and envd features** — `realpath`, `chmod`, `mv`, same-filesystem POSIX rename, streaming reads, and file metadata extended attributes are required; unsupported templates fail rather than degrade silently. diff --git a/packages/fs/fs-e2b/README.zh.md b/packages/fs/fs-e2b/README.zh.md new file mode 100644 index 0000000000..1a9a174ac9 --- /dev/null +++ b/packages/fs/fs-e2b/README.zh.md @@ -0,0 +1,29 @@ +# @deepseek-ai/dsh-fs-e2b + +[English](README.md) | 中文 + +[`@deepseek-ai/dsh-fs`](../fs/README.md) 提供方 seam 的 E2B 实现。它没有配置:先加载 [`@deepseek-ai/dsh-e2b`](../../e2b/e2b/README.md),再用本服务取代 `dsh-fs-local`。该提供方使用所有者的远程 cwd 和 SDK 句柄,因此文件工具观察到的环境与 E2B 后端 Bash 进程相同。 + +## 行为 + +- **远程身份与元数据**:相对路径以调用方 cwd 或 `ctx.e2b.cwd` 为基准,按照 POSIX 路径解析;`realpath -m` 提供规范化目标身份,且不要求最终文件存在。`stat`、不跟随链接的 `lstat` 和稳定的单层目录列表会把 E2B 元数据投影到文件系统 seam。版本是 E2B 元数据与每次写入设置的扩展属性所组成的不透明哈希。 +- **UTF-8 读取**:完整读取和流式读取会保留跨分片解码、拒绝无效 UTF-8,并使用 seam 的 8192 字节 NUL 样本检测二进制内容。面向模型的工具仍负责选择大小和行窗口。 +- **原子变更**:写入会上传 mode 为 `0600` 的同级临时文件,保留现有文件的 POSIX mode,并通过同目录 Linux `mv -f` 发布。E2B 会创建缺失的父目录。字面量编辑匹配时会规范化为 LF,存储时恢复占主导的 CRLF,并在宿主进程内按规范化目标串行执行变更。可选的创建/版本防护会保留基础 seam 的已观察状态语义。 +- **失败与取消**:E2B 的未找到、权限、中止及其他控制器故障会映射到现有 `FsError` 词汇。取消在 SDK 请求边界上采用尽力而为语义;成功 rename 是提交点。 + +该提供方不会复制、挂载或协调宿主工作区。把宿主路径用作 `cwd`,只会在远程创建一个拼写相同的目录。 + +## 模型体验 + +通过 [`dsh-tool-fs`](../tool-fs/README.md) 间接影响模型;该工具会渲染远程 UTF-8 内容、目录结果、变更确认和提供方错误,而 E2B 身份及传输保持内部实现。 + +#### KV Cache 影响 + +不会直接失效;请求前缀变更由具名消费方负责。 + +## 已知限制与延后工作 + +- **不提供宿主同步**:空的 E2B cwd 会一直为空,直到工具、命令、模板或外部进程填充它;本地文件既不会上传,也不会同步回本地。 +- **变更协调仅限宿主进程内**:另一个 harness 连接或远程命令可能与适配器发生竞态;版本防护只能检测 E2B 元数据所体现的变更。 +- **仍需承担完整文件变更成本**:覆盖差异和字面量编辑会把完整文件读入宿主内存,每项操作也都会产生 E2B 控制器延迟。 +- **自定义模板必须支持所用的 Linux 与 envd 功能**:必须支持 `realpath`、`chmod`、`mv`、同一文件系统内的 POSIX rename、流式读取和文件元数据扩展属性;不支持的模板会失败,而不会静默降级。 diff --git a/packages/fs/fs-e2b/package.json b/packages/fs/fs-e2b/package.json new file mode 100644 index 0000000000..cc96cba6c9 --- /dev/null +++ b/packages/fs/fs-e2b/package.json @@ -0,0 +1,41 @@ +{ + "name": "@deepseek-ai/dsh-fs-e2b", + "description": "E2B filesystem implementation for DeepSeek Harness", + "version": "0.0.1", + "private": true, + "type": "module", + "main": "lib/index.js", + "types": "lib/types/index.d.ts", + "exports": { + ".": { + "types": "./lib/types/index.d.ts", + "default": "./lib/index.js" + }, + "./invariant": { + "types": "./lib/types/invariant.d.ts", + "default": "./lib/invariant.js" + }, + "./src/*": "./src/*", + "./package.json": "./package.json" + }, + "files": [ + "lib/index.js", + "lib/invariant.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", + "src" + ], + "license": "BSD-3-Clause", + "peerDependencies": { + "@deepseek-ai/dsh-e2b": "^0.0.1", + "@deepseek-ai/dsh-fs": "^0.0.1", + "@deepseek-ai/dsh-invariants": "^0.0.1", + "cordis": "^4.0.0-rc.7" + }, + "devDependencies": { + "@deepseek-ai/dsh-e2b": "workspace:^", + "@deepseek-ai/dsh-fs": "workspace:^", + "@deepseek-ai/dsh-invariants": "workspace:^", + "cordis": "^4.0.0-rc.7" + } +} diff --git a/packages/fs/fs-e2b/src/index.ts b/packages/fs/fs-e2b/src/index.ts new file mode 100644 index 0000000000..020b0f4857 --- /dev/null +++ b/packages/fs/fs-e2b/src/index.ts @@ -0,0 +1,423 @@ +/** + * E2B implementation of the filesystem provider seam. Paths, contents, and + * atomic staging files remain inside the shared remote sandbox. + * @module @deepseek-ai/dsh-fs-e2b + */ + +import { createHash, randomUUID } from 'node:crypto' +import { posix } from 'node:path' +import { FileSystem, FsError, FsTargetKey, FsVersion } from '@deepseek-ai/dsh-fs' +import type { + FsDirEntry, + FsEditOutcome, + FsEditRequest, + FsInfo, + FsPathInfo, + FsTarget, + FsWriteIntent, + FsWriteOutcome, +} from '@deepseek-ai/dsh-fs' +import { + CommandExitError, + FileNotFoundError, + FileType, + quoteE2BShellArg, +} from '@deepseek-ai/dsh-e2b' +import type { EntryInfo, Sandbox } from '@deepseek-ai/dsh-e2b' + +const VERSION_METADATA_KEY = 'dsh-version' +const BINARY_SAMPLE_BYTES = 8192 + +function assertNotAborted(signal: AbortSignal | undefined, operation: string): void { + if (signal?.aborted === true) throw new FsError(`${operation} aborted`, 'FS_ABORTED') +} + +function normalizeLineEndings(value: string): string { + return value.replaceAll('\r\n', '\n') +} + +function detectsCrlf(value: string): boolean { + const sample = value.slice(0, 4096) + const crlf = sample.split('\r\n').length - 1 + const lf = sample.split('\n').length - 1 - crlf + return crlf > lf +} + +function restoreLineEndings(value: string, crlf: boolean): string { + return crlf ? normalizeLineEndings(value).replaceAll('\n', '\r\n') : value +} + +function decodeText(bytes: Uint8Array, displayPath: string, binarySampleBytes: number): string { + if (bytes.subarray(0, binarySampleBytes).includes(0)) { + throw new FsError(`cannot read "${displayPath}": binary file`, 'FS_NOT_TEXT') + } + try { + return new TextDecoder('utf-8', { fatal: true }).decode(bytes) + } catch (error: unknown) { + throw new FsError(`cannot read "${displayPath}": invalid UTF-8 text`, 'FS_NOT_TEXT', { cause: error }) + } +} + +function signalOpts(signal: AbortSignal | undefined): { signal?: AbortSignal } { + return signal === undefined ? {} : { signal } +} + +function entryType(entry: EntryInfo): FsInfo['type'] { + switch (entry.type) { + case FileType.FILE: + return 'file' + case FileType.DIR: + return 'directory' + default: + return 'other' + } +} + +function entryVersion(entry: EntryInfo): ReturnType { + const facts = JSON.stringify([ + entry.metadata?.[VERSION_METADATA_KEY], + entry.path, + entry.type, + entry.size, + entry.mode, + entry.modifiedTime?.toISOString(), + entry.symlinkTarget, + ]) + return FsVersion(`e2b:${createHash('sha256').update(facts).digest('hex')}`) +} + +function mapError(error: unknown, operation: string, displayPath: string, signal?: AbortSignal): FsError { + if (error instanceof FsError) return error + if (signal?.aborted === true || (error instanceof DOMException && error.name === 'AbortError')) { + return new FsError(`${operation} aborted`, 'FS_ABORTED', { cause: error }) + } + if (error instanceof FileNotFoundError) { + return new FsError(`cannot ${operation} "${displayPath}": not found`, 'FS_NOT_FOUND', { cause: error }) + } + if (/permission denied|operation not permitted/i.test(String(error))) { + return new FsError(`cannot ${operation} "${displayPath}": permission denied`, 'FS_PERMISSION_DENIED', { cause: error }) + } + return new FsError(`cannot ${operation} "${displayPath}": ${String(error)}`, 'FS_IO_ERROR', { cause: error }) +} + +function literalEdit(content: string, request: FsEditRequest, displayPath: string): string { + const oldString = normalizeLineEndings(request.oldString) + const newString = normalizeLineEndings(request.newString) + if (oldString.length === 0) { + throw new FsError(`cannot edit "${displayPath}": old_string must be non-empty`, 'FS_EDIT_NOT_FOUND') + } + let matches = 0 + let offset = 0 + while (true) { + const found = content.indexOf(oldString, offset) + if (found < 0) break + matches += 1 + offset = found + oldString.length + } + if (matches === 0) throw new FsError(`cannot edit "${displayPath}": old_string was not found`, 'FS_EDIT_NOT_FOUND') + if (!request.replaceAll && matches !== 1) { + throw new FsError(`cannot edit "${displayPath}": old_string matched ${matches} times`, 'FS_AMBIGUOUS_EDIT') + } + return request.replaceAll ? content.split(oldString).join(newString) : content.replace(oldString, newString) +} + +/** Remote filesystem backend sharing the sandbox owned by `ctx.e2b`. */ +export class E2BFileSystem extends FileSystem { + static inject = ['e2b'] + + private readonly locks = new Map>() + + override async resolve(path: string, opts?: { cwd?: string; signal?: AbortSignal }): Promise { + assertNotAborted(opts?.signal, 'resolve') + if (path.trim().length === 0) throw new FsError('file_path must be a non-empty string', 'FS_NOT_FOUND') + const displayPath = posix.resolve(opts?.cwd ?? this.ctx.e2b.cwd, path) + try { + const sandbox = await this.ctx.e2b.getSandbox() + const targetKey = await this.canonicalPath(sandbox, displayPath, opts?.signal) + assertNotAborted(opts?.signal, 'resolve') + return { targetKey: FsTargetKey(targetKey), displayPath } + } catch (error: unknown) { + throw mapError(error, 'resolve', displayPath, opts?.signal) + } + } + + override async stat(target: FsTarget, signal?: AbortSignal): Promise { + assertNotAborted(signal, 'stat') + const entry = await this.probe(String(target.targetKey), target.displayPath, signal) + if (entry === undefined) return undefined + return { + version: entryVersion(entry), + type: entryType(entry), + ...(entry.type === FileType.FILE ? { size: entry.size } : {}), + } + } + + override async lstat(path: string, opts?: { cwd?: string }, signal?: AbortSignal): Promise { + assertNotAborted(signal, 'lstat') + if (path.trim().length === 0) throw new FsError('file_path must be a non-empty string', 'FS_NOT_FOUND') + const displayPath = posix.resolve(opts?.cwd ?? this.ctx.e2b.cwd, path) + const entry = await this.probe(displayPath, displayPath, signal) + if (entry === undefined) return undefined + const type = entry.symlinkTarget !== undefined + ? 'symlink' as const + : entry.type === FileType.FILE + ? 'file' as const + : entry.type === FileType.DIR + ? 'directory' as const + : 'other' as const + return { + version: entryVersion(entry), + type, + ...(entry.type === FileType.FILE ? { size: entry.size } : {}), + } + } + + override async readText(target: FsTarget, signal?: AbortSignal): Promise { + const sandbox = await this.ctx.e2b.getSandbox() + await this.requireRegular(target, signal) + try { + const bytes = await sandbox.files.read(String(target.targetKey), { format: 'bytes', ...signalOpts(signal) }) + assertNotAborted(signal, 'read') + return decodeText(bytes, target.displayPath, BINARY_SAMPLE_BYTES) + } catch (error: unknown) { + throw mapError(error, 'read', target.displayPath, signal) + } + } + + override async streamText(target: FsTarget, signal?: AbortSignal): Promise> { + const sandbox = await this.ctx.e2b.getSandbox() + await this.requireRegular(target, signal) + let stream: ReadableStream + try { + stream = await sandbox.files.read(String(target.targetKey), { format: 'stream', ...signalOpts(signal) }) + } catch (error: unknown) { + throw mapError(error, 'read', target.displayPath, signal) + } + const displayPath = target.displayPath + return { + async *[Symbol.asyncIterator](): AsyncGenerator { + const reader = stream.getReader() + const decoder = new TextDecoder('utf-8', { fatal: true }) + let sampledBytes = 0 + try { + while (true) { + assertNotAborted(signal, 'read') + const next = await reader.read() + if (next.done) break + if (sampledBytes < BINARY_SAMPLE_BYTES) { + const sample = next.value.subarray(0, BINARY_SAMPLE_BYTES - sampledBytes) + if (sample.includes(0)) throw new FsError(`cannot read "${displayPath}": binary file`, 'FS_NOT_TEXT') + sampledBytes += sample.length + } + let text: string + try { + text = decoder.decode(next.value, { stream: true }) + } catch (error: unknown) { + throw new FsError(`cannot read "${displayPath}": invalid UTF-8 text`, 'FS_NOT_TEXT', { cause: error }) + } + if (text.length > 0) yield text + } + try { + decoder.decode() + } catch (error: unknown) { + throw new FsError(`cannot read "${displayPath}": invalid UTF-8 text`, 'FS_NOT_TEXT', { cause: error }) + } + } catch (error: unknown) { + throw mapError(error, 'read', displayPath, signal) + } finally { + reader.releaseLock() + } + }, + } + } + + override async listDir(target: FsTarget, signal?: AbortSignal): Promise { + const info = await this.stat(target, signal) + if (info === undefined) throw new FsError(`cannot list "${target.displayPath}": not found`, 'FS_NOT_FOUND') + if (info.type !== 'directory') throw new FsError(`cannot list "${target.displayPath}": not a directory`, 'FS_NOT_DIRECTORY') + try { + const sandbox = await this.ctx.e2b.getSandbox() + const listed = await sandbox.files.list(String(target.targetKey), { depth: 1, ...signalOpts(signal) }) + const entries = await Promise.all(listed.map(async (entry): Promise => { + const displayPath = posix.join(target.displayPath, entry.name) + const canonical = await this.canonicalPath(sandbox, entry.path, signal) + const resolved = await this.probe(canonical, displayPath, signal) + return { + name: entry.name, + type: resolved === undefined ? 'other' : entryType(resolved), + target: { targetKey: FsTargetKey(canonical), displayPath }, + ...(resolved !== undefined ? { version: entryVersion(resolved) } : {}), + ...(resolved?.type === FileType.FILE ? { size: resolved.size } : {}), + } + })) + return entries.sort((left, right) => left.name.localeCompare(right.name)) + } catch (error: unknown) { + throw mapError(error, 'list', target.displayPath, signal) + } + } + + override async writeText( + target: FsTarget, + content: string, + expected?: FsWriteIntent, + signal?: AbortSignal, + ): Promise { + return this.withLock(String(target.targetKey), async () => { + const existing = await this.probe(String(target.targetKey), target.displayPath, signal) + if (existing !== undefined && entryType(existing) !== 'file') { + throw new FsError(`cannot write "${target.displayPath}": not a regular file`, 'FS_NOT_REGULAR_FILE') + } + this.checkWriteIntent(existing, expected, target) + const before = existing === undefined ? null : await this.readForDiff(target, signal) + const version = await this.writeAtomic(target, content, existing, signal) + return { + operation: existing === undefined ? 'create' : 'update', + version, + before, + after: normalizeLineEndings(content), + } + }) + } + + override async editText( + target: FsTarget, + edit: FsEditRequest, + expected?: { version: ReturnType }, + signal?: AbortSignal, + ): Promise { + return this.withLock(String(target.targetKey), async () => { + const existing = await this.probe(String(target.targetKey), target.displayPath, signal) + if (existing === undefined) { + throw new FsError(`cannot edit "${target.displayPath}": file changed since it was read`, 'FS_STALE_VERSION') + } + if (entryType(existing) !== 'file') { + throw new FsError(`cannot edit "${target.displayPath}": not a regular file`, 'FS_NOT_REGULAR_FILE') + } + if (expected !== undefined && entryVersion(existing) !== expected.version) { + throw new FsError(`cannot edit "${target.displayPath}": file changed since it was read`, 'FS_STALE_VERSION') + } + const raw = await this.readForEdit(target, signal) + const before = normalizeLineEndings(raw) + const after = literalEdit(before, edit, target.displayPath) + const storage = restoreLineEndings(after, detectsCrlf(raw)) + const version = await this.writeAtomic(target, storage, existing, signal) + return { version, before, after } + }) + } + + private async withLock(targetKey: string, operation: () => Promise): Promise { + const prior = this.locks.get(targetKey) ?? Promise.resolve() + const run = prior.then(operation, operation) + const tail = run.then(() => undefined, () => undefined) + this.locks.set(targetKey, tail) + try { + return await run + } finally { + if (this.locks.get(targetKey) === tail) this.locks.delete(targetKey) + } + } + + private async canonicalPath(sandbox: Sandbox, path: string, signal?: AbortSignal): Promise { + try { + const result = await sandbox.commands.run(`realpath -m -- ${quoteE2BShellArg(path)}`, signalOpts(signal)) + return result.stdout.replace(/\n$/, '') + } catch (error: unknown) { + if (error instanceof CommandExitError) throw new Error(error.stderr || error.message, { cause: error }) + throw error + } + } + + private async probe(path: string, displayPath: string, signal?: AbortSignal): Promise { + assertNotAborted(signal, 'stat') + try { + const sandbox = await this.ctx.e2b.getSandbox() + const entry = await sandbox.files.getInfo(path, signalOpts(signal)) + assertNotAborted(signal, 'stat') + return entry + } catch (error: unknown) { + if (error instanceof FileNotFoundError) return undefined + throw mapError(error, 'stat', displayPath, signal) + } + } + + private async requireRegular(target: FsTarget, signal?: AbortSignal): Promise { + const info = await this.stat(target, signal) + if (info === undefined) throw new FsError(`cannot read "${target.displayPath}": not found`, 'FS_NOT_FOUND') + if (info.type !== 'file') throw new FsError(`cannot read "${target.displayPath}": not a regular file`, 'FS_NOT_REGULAR_FILE') + } + + private checkWriteIntent(existing: EntryInfo | undefined, expected: FsWriteIntent | undefined, target: FsTarget): void { + if (expected?.kind === 'createIfAbsent' && existing !== undefined) { + throw new FsError(`cannot overwrite existing "${target.displayPath}" without reading it first`, 'FS_NOT_OBSERVED') + } + if (expected?.kind === 'replaceIfVersion') { + if (existing === undefined || entryVersion(existing) !== expected.version) { + throw new FsError(`cannot write "${target.displayPath}": file changed since it was read`, 'FS_STALE_VERSION') + } + } + } + + private async readForDiff(target: FsTarget, signal?: AbortSignal): Promise { + try { + const sandbox = await this.ctx.e2b.getSandbox() + const bytes = await sandbox.files.read(String(target.targetKey), { format: 'bytes', ...signalOpts(signal) }) + assertNotAborted(signal, 'read') + return normalizeLineEndings(decodeText(bytes, target.displayPath, bytes.length)) + } catch (error: unknown) { + if (error instanceof FsError && error.code === 'FS_NOT_TEXT') return null + throw mapError(error, 'read', target.displayPath, signal) + } + } + + private async readForEdit(target: FsTarget, signal?: AbortSignal): Promise { + try { + const sandbox = await this.ctx.e2b.getSandbox() + const bytes = await sandbox.files.read(String(target.targetKey), { format: 'bytes', ...signalOpts(signal) }) + assertNotAborted(signal, 'edit') + return decodeText(bytes, target.displayPath, bytes.length) + } catch (error: unknown) { + throw mapError(error, 'edit', target.displayPath, signal) + } + } + + private async writeAtomic( + target: FsTarget, + content: string, + existing: EntryInfo | undefined, + signal?: AbortSignal, + ): Promise> { + assertNotAborted(signal, 'write') + const sandbox = await this.ctx.e2b.getSandbox() + const targetPath = String(target.targetKey) + const versionId = randomUUID() + const temporary = posix.join(posix.dirname(targetPath), `.${posix.basename(targetPath)}.dsh-${randomUUID()}.tmp`) + try { + await sandbox.files.write(temporary, content, { + metadata: { [VERSION_METADATA_KEY]: versionId }, + ...signalOpts(signal), + }) + assertNotAborted(signal, 'write') + const mode = existing === undefined ? 0o600 : existing.mode & 0o777 + await sandbox.commands.run( + `chmod ${mode.toString(8)} -- ${quoteE2BShellArg(temporary)}`, + signalOpts(signal), + ) + assertNotAborted(signal, 'write') + await sandbox.commands.run( + `mv -f -- ${quoteE2BShellArg(temporary)} ${quoteE2BShellArg(targetPath)}`, + signalOpts(signal), + ) + const committed = await sandbox.files.getInfo(targetPath) + return entryVersion(committed) + } catch (error: unknown) { + try { + await sandbox.files.remove(temporary) + } catch (_temporaryAlreadyAbsent) { + // Only the private staging path is swallowed; the original failure owns the operation. + } + throw mapError(error, 'write', target.displayPath, signal) + } + } +} + +export default E2BFileSystem diff --git a/packages/fs/fs-e2b/src/invariant.ts b/packages/fs/fs-e2b/src/invariant.ts new file mode 100644 index 0000000000..6294d66ace --- /dev/null +++ b/packages/fs/fs-e2b/src/invariant.ts @@ -0,0 +1,27 @@ +/** Package-owned invariant companion for `@deepseek-ai/dsh-fs-e2b`. */ + +/* jscpd:ignore-start */ +import type { Context } from 'cordis' +import type { InvariantInstaller } from '@deepseek-ai/dsh-invariants' + +const PACKAGE_NAME = '@deepseek-ai/dsh-fs-e2b' + +/** Cordis companion plugin name. */ +export const name = 'fs-e2b-invariant' +/** Service required before reserving package ownership. */ +export const inject = ['invariants'] + +/** + * No runtime invariant: each operation returns the E2B controller's committed + * result directly, with no independent event or cache to cross-check. + */ +const install: InvariantInstaller = () => {} + +/** + * Register this package's invariant companion. + * @param ctx - Cordis context carrying the invariant service. + * @returns the installed registration's disposer after setup succeeds. + */ +export const apply = (ctx: Context): Promise<() => void> => + Promise.resolve(ctx.invariants.register(PACKAGE_NAME, install)) +/* jscpd:ignore-end */ diff --git a/packages/fs/fs-e2b/tests/filesystem.spec.ts b/packages/fs/fs-e2b/tests/filesystem.spec.ts new file mode 100644 index 0000000000..b8eb5447d6 --- /dev/null +++ b/packages/fs/fs-e2b/tests/filesystem.spec.ts @@ -0,0 +1,537 @@ +import { dirname, posix } from 'node:path' +import { Context } from 'cordis' +import { + CommandExitError, + FileNotFoundError, + FileType, + type EntryInfo, + type Sandbox, +} from '@deepseek-ai/dsh-e2b' +import type E2BSandboxService from '@deepseek-ai/dsh-e2b' +import { FsVersion } from '@deepseek-ai/dsh-fs' +import E2BFileSystem from '@deepseek-ai/dsh-fs-e2b' +import * as E2BFsInvariant from '../src/invariant.ts' +import InvariantService from '@deepseek-ai/dsh-invariants' +import { describe, expect, it } from 'vitest' + +interface RemoteNode { + type: FileType + data: Uint8Array + mode: number + modified: number + metadata?: Record + symlinkTarget?: string +} + +function bytes(value: string | readonly number[]): Uint8Array { + return typeof value === 'string' ? new TextEncoder().encode(value) : Uint8Array.from(value) +} + +function commandError(exitCode: number, stderr = ''): CommandExitError { + return new CommandExitError({ exitCode, stdout: '', stderr, error: stderr }) +} + +class FakeRemote { + readonly nodes = new Map() + readonly writes: Array<{ path: string; data: string; metadata?: Record }> = [] + readonly renames: Array<{ from: string; to: string }> = [] + readonly removals: string[] = [] + readonly commands: string[] = [] + streamChunks: Uint8Array[] | undefined + nextCommandError: unknown + nextInfoError: unknown + nextListError: unknown + nextReadError: unknown + nextRenameError: unknown + nextRemoveError: unknown + abortAfterRename: AbortController | undefined + disappearOnInfo = new Set() + private clock = 1 + + constructor() { + this.dir('/') + this.dir('/workspace') + } + + dir(path: string): void { + this.nodes.set(path, { type: FileType.DIR, data: bytes(''), mode: 0o755, modified: this.clock++ }) + } + + file(path: string, data: string | readonly number[], mode = 0o644): void { + this.nodes.set(path, { type: FileType.FILE, data: bytes(data), mode, modified: this.clock++ }) + } + + other(path: string): void { + this.nodes.set(path, { type: 'other' as FileType, data: bytes(''), mode: 0o600, modified: this.clock++ }) + } + + symlink(path: string, target: string): void { + this.nodes.set(path, { + type: FileType.FILE, + data: bytes(''), + mode: 0o777, + modified: this.clock++, + symlinkTarget: target, + }) + } + + mutate(path: string, data: string): void { + const node = this.required(path) + node.data = bytes(data) + node.modified = this.clock++ + } + + private required(path: string): RemoteNode { + const node = this.nodes.get(path) + if (node === undefined) throw new FileNotFoundError(`missing: ${path}`) + return node + } + + private followed(path: string): { path: string; node: RemoteNode; link?: RemoteNode } { + const node = this.required(path) + if (node.symlinkTarget === undefined) return { path, node } + return { path: node.symlinkTarget, node: this.required(node.symlinkTarget), link: node } + } + + private info(path: string): EntryInfo { + if (this.disappearOnInfo.delete(path)) throw new FileNotFoundError(`missing: ${path}`) + return this.rawInfo(path) + } + + private rawInfo(path: string): EntryInfo { + const followed = this.followed(path) + const node = followed.node + return { + name: posix.basename(path), + path, + type: node.type, + size: node.data.byteLength, + mode: node.mode, + permissions: 'rw-------', + owner: 'user', + group: 'user', + modifiedTime: new Date(node.modified), + ...(node.metadata !== undefined ? { metadata: { ...node.metadata } } : {}), + ...(followed.link?.symlinkTarget !== undefined ? { symlinkTarget: followed.link.symlinkTarget } : {}), + } + } + + private checkAbort(options: { signal?: AbortSignal } | undefined): void { + if (options?.signal?.aborted === true) throw new DOMException('aborted', 'AbortError') + } + + readonly sandbox = { + sandboxId: 'fake', + files: { + makeDir: async (path: string): Promise => { + if (this.nodes.has(path)) return false + this.dir(path) + return true + }, + getInfo: async (path: string, options?: { signal?: AbortSignal }): Promise => { + this.checkAbort(options) + if (this.nextInfoError !== undefined) { + const error = this.nextInfoError + this.nextInfoError = undefined + throw error + } + return this.info(path) + }, + read: async (path: string, options: { format: 'bytes' | 'stream'; signal?: AbortSignal }): Promise> => { + this.checkAbort(options) + if (this.nextReadError !== undefined) { + const error = this.nextReadError + this.nextReadError = undefined + throw error + } + const data = this.followed(path).node.data + if (options.format === 'bytes') return data.slice() + const chunks = this.streamChunks ?? [data.slice()] + return new ReadableStream({ + start(controller) { + for (const chunk of chunks) controller.enqueue(chunk) + controller.close() + }, + }) + }, + list: async (path: string, options?: { depth?: number; signal?: AbortSignal }): Promise => { + this.checkAbort(options) + if (this.nextListError !== undefined) { + const error = this.nextListError + this.nextListError = undefined + throw error + } + this.required(path) + return [...this.nodes.keys()] + .filter(candidate => candidate !== path && dirname(candidate) === path) + .map(candidate => this.rawInfo(candidate)) + }, + write: async (path: string, data: string, options?: { metadata?: Record; signal?: AbortSignal }): Promise => { + this.checkAbort(options) + const parent = dirname(path) + if (!this.nodes.has(parent)) this.dir(parent) + this.nodes.set(path, { + type: FileType.FILE, + data: bytes(data), + mode: 0o644, + modified: this.clock++, + ...(options?.metadata !== undefined ? { metadata: { ...options.metadata } } : {}), + }) + this.writes.push({ path, data, ...(options?.metadata !== undefined ? { metadata: options.metadata } : {}) }) + return {} + }, + rename: async (from: string, to: string, options?: { signal?: AbortSignal }): Promise => { + this.checkAbort(options) + if (this.nextRenameError !== undefined) { + const error = this.nextRenameError + this.nextRenameError = undefined + throw error + } + const node = this.required(from) + this.nodes.delete(from) + this.nodes.set(to, node) + this.renames.push({ from, to }) + this.abortAfterRename?.abort('after commit') + return this.info(to) + }, + remove: async (path: string): Promise => { + this.removals.push(path) + if (this.nextRemoveError !== undefined) { + const error = this.nextRemoveError + this.nextRemoveError = undefined + throw error + } + this.nodes.delete(path) + }, + }, + commands: { + run: async (command: string, options?: { signal?: AbortSignal }): Promise<{ exitCode: number; stdout: string; stderr: string }> => { + this.checkAbort(options) + this.commands.push(command) + if (this.nextCommandError !== undefined) { + const error = this.nextCommandError + this.nextCommandError = undefined + throw error + } + if (command.startsWith('realpath -m -- ')) { + const input = command.slice('realpath -m -- '.length).slice(1, -1) + const node = this.nodes.get(input) + return { exitCode: 0, stdout: `${node?.symlinkTarget ?? input}\n`, stderr: '' } + } + const chmod = /^chmod ([0-7]+) -- '([^']+)'$/.exec(command) + if (chmod !== null) this.required(chmod[2]!).mode = Number.parseInt(chmod[1]!, 8) + const move = /^mv -f -- '([^']+)' '([^']+)'$/.exec(command) + if (move !== null) { + if (this.nextRenameError !== undefined) { + const error = this.nextRenameError + this.nextRenameError = undefined + throw error + } + const node = this.required(move[1]!) + this.nodes.delete(move[1]!) + this.nodes.set(move[2]!, node) + this.renames.push({ from: move[1]!, to: move[2]! }) + this.abortAfterRename?.abort('after commit') + } + return { exitCode: 0, stdout: '', stderr: '' } + }, + }, + } as unknown as Sandbox +} + +async function setup(remote = new FakeRemote()): Promise<{ ctx: Context; fs: E2BFileSystem; remote: FakeRemote }> { + const ctx = new Context() + const runtime = { + cwd: '/workspace', + runtimeRoot: '/workspace/.dsh-e2b', + disposeMode: 'kill', + getSandbox: async () => remote.sandbox, + } as unknown as E2BSandboxService + ctx.provide('e2b', runtime) + await ctx.plugin(E2BFileSystem) + return { ctx, fs: ctx.fs as E2BFileSystem, remote } +} + +async function expectCode(promise: Promise, code: string): Promise { + await expect(promise).rejects.toMatchObject({ code }) +} + +describe('E2BFileSystem identity, metadata, and reads', () => { + it('resolves remote paths, reports symlinks, and lists direct children in stable order', async () => { + const remote = new FakeRemote() + remote.file('/workspace/z.txt', 'z') + remote.file('/workspace/a.txt', 'a') + remote.dir('/workspace/dir') + remote.other('/workspace/special') + remote.file('/workspace/dir/nested.txt', 'nested') + remote.symlink('/workspace/link.txt', '/workspace/a.txt') + const { fs } = await setup(remote) + + const link = await fs.resolve('link.txt') + expect(link).toEqual({ targetKey: '/workspace/a.txt', displayPath: '/workspace/link.txt' }) + await expect(fs.lstat('link.txt')).resolves.toMatchObject({ type: 'symlink', size: 1 }) + await expect(fs.lstat('a.txt')).resolves.toMatchObject({ type: 'file', size: 1 }) + await expect(fs.lstat('dir')).resolves.toEqual(expect.objectContaining({ type: 'directory' })) + await expect(fs.lstat('special')).resolves.toEqual(expect.objectContaining({ type: 'other' })) + await expect(fs.lstat('missing')).resolves.toBeUndefined() + await expect(fs.stat(link)).resolves.toMatchObject({ type: 'file', size: 1 }) + const directory = await fs.resolve('.') + const listed = await fs.listDir(directory) + expect(listed.map(entry => entry.name)).toEqual(['a.txt', 'dir', 'link.txt', 'special', 'z.txt']) + expect(listed.find(entry => entry.name === 'dir')).toMatchObject({ type: 'directory' }) + expect(listed.find(entry => entry.name === 'link.txt')).toMatchObject({ + type: 'file', + target: { targetKey: '/workspace/a.txt', displayPath: '/workspace/link.txt' }, + }) + expect(listed.some(entry => entry.name === 'nested.txt')).toBe(false) + }) + + it('reads whole and streamed UTF-8 across chunk boundaries', async () => { + const remote = new FakeRemote() + remote.file('/workspace/text.txt', 'A€B') + remote.streamChunks = [bytes([65, 0xe2]), bytes([0x82, 0xac, 66])] + const { fs } = await setup(remote) + const target = await fs.resolve('text.txt') + await expect(fs.readText(target)).resolves.toBe('A€B') + let streamed = '' + for await (const chunk of await fs.streamText(target)) streamed += chunk + expect(streamed).toBe('A€B') + + remote.streamChunks = [bytes([0xe2]), bytes([0x82, 0xac])] + let initiallyBuffered = '' + for await (const chunk of await fs.streamText(target)) initiallyBuffered += chunk + expect(initiallyBuffered).toBe('€') + }) + + it('matches local binary sampling while edits still reject any NUL byte', async () => { + const remote = new FakeRemote() + remote.file('/workspace/late-nul.txt', `${'a'.repeat(8192)}\0tail`) + const { fs } = await setup(remote) + const target = await fs.resolve('late-nul.txt') + await expect(fs.readText(target)).resolves.toContain('\0tail') + remote.streamChunks = [bytes('a'.repeat(8192)), bytes([0, 116])] + let streamed = '' + for await (const chunk of await fs.streamText(target)) streamed += chunk + expect(streamed).toBe(`${'a'.repeat(8192)}\0t`) + await expectCode(fs.editText(target, { oldString: 'tail', newString: 'end', replaceAll: false }), 'FS_NOT_TEXT') + }) + + it('maps binary, invalid UTF-8, missing, and non-regular read failures', async () => { + const remote = new FakeRemote() + remote.file('/workspace/binary', [0, 1]) + remote.file('/workspace/invalid', [0xff]) + remote.dir('/workspace/directory') + const { fs } = await setup(remote) + await expectCode(fs.readText(await fs.resolve('binary')), 'FS_NOT_TEXT') + await expectCode(fs.readText(await fs.resolve('invalid')), 'FS_NOT_TEXT') + await expectCode(fs.readText(await fs.resolve('missing')), 'FS_NOT_FOUND') + await expectCode(fs.readText(await fs.resolve('directory')), 'FS_NOT_REGULAR_FILE') + + remote.streamChunks = [bytes([0xff])] + const invalid = await fs.streamText(await fs.resolve('invalid')) + await expect((async () => { for await (const _chunk of invalid) void _chunk })()).rejects.toMatchObject({ code: 'FS_NOT_TEXT' }) + remote.streamChunks = [bytes([0])] + const binary = await fs.streamText(await fs.resolve('binary')) + await expect((async () => { for await (const _chunk of binary) void _chunk })()).rejects.toMatchObject({ code: 'FS_NOT_TEXT' }) + + remote.streamChunks = [bytes([0xe2])] + const incomplete = await fs.streamText(await fs.resolve('invalid')) + await expect((async () => { for await (const _chunk of incomplete) void _chunk })()).rejects.toMatchObject({ code: 'FS_NOT_TEXT' }) + + const raced = await fs.resolve('invalid') + remote.nextReadError = new FileNotFoundError('gone after stat') + await expectCode(fs.streamText(raced), 'FS_NOT_FOUND') + }) + + it('honors aborts before and during remote reads', async () => { + const remote = new FakeRemote() + remote.file('/workspace/a', 'a') + const { fs } = await setup(remote) + await expectCode(fs.resolve('a', { signal: AbortSignal.abort() }), 'FS_ABORTED') + await expectCode(fs.lstat('a', undefined, AbortSignal.abort()), 'FS_ABORTED') + await expectCode(fs.stat(await fs.resolve('a'), AbortSignal.abort()), 'FS_ABORTED') + remote.nextReadError = new DOMException('aborted', 'AbortError') + await expectCode(fs.readText(await fs.resolve('a')), 'FS_ABORTED') + }) + + it('rejects empty paths and directory-listing type errors', async () => { + const remote = new FakeRemote() + remote.file('/workspace/file', 'x') + const { fs } = await setup(remote) + await expectCode(fs.resolve(' '), 'FS_NOT_FOUND') + await expectCode(fs.lstat(''), 'FS_NOT_FOUND') + await expectCode(fs.listDir(await fs.resolve('missing')), 'FS_NOT_FOUND') + await expectCode(fs.listDir(await fs.resolve('/workspace/file')), 'FS_NOT_DIRECTORY') + remote.nextListError = new Error('listing transport failed') + await expectCode(fs.listDir(await fs.resolve('/workspace')), 'FS_IO_ERROR') + }) +}) + +describe('E2BFileSystem atomic writes and edits', () => { + it('creates owner-only files and returns metadata after the committed move', async () => { + const { fs, remote } = await setup() + const target = await fs.resolve('new.txt') + const outcome = await fs.writeText(target, 'one\r\ntwo\rthree', { kind: 'createIfAbsent' }) + expect(outcome).toMatchObject({ operation: 'create', before: null, after: 'one\ntwo\rthree' }) + expect(remote.nodes.get('/workspace/new.txt')?.mode).toBe(0o600) + expect(remote.nodes.get('/workspace/new.txt')?.metadata?.['dsh-version']).toBeDefined() + await expect(fs.stat(target)).resolves.toMatchObject({ version: outcome.version, size: 14 }) + }) + + it('preserves replacement mode, normalizes only CRLF for diffs, and changes version on external writes', async () => { + const remote = new FakeRemote() + remote.file('/workspace/file.txt', 'old\r\nline\rlone', 0o640) + const { fs } = await setup(remote) + const target = await fs.resolve('file.txt') + const before = (await fs.stat(target))!.version + const outcome = await fs.writeText(target, 'new', { kind: 'replaceIfVersion', version: before }) + expect(outcome).toMatchObject({ operation: 'update', before: 'old\nline\rlone', after: 'new' }) + expect(remote.nodes.get('/workspace/file.txt')?.mode).toBe(0o640) + const committed = outcome.version + remote.mutate('/workspace/file.txt', 'external') + expect((await fs.stat(target))!.version).not.toBe(committed) + }) + + it('returns null as the overwrite diff basis for binary or invalid prior content', async () => { + const remote = new FakeRemote() + remote.file('/workspace/file.txt', [0xff]) + const { fs } = await setup(remote) + const target = await fs.resolve('file.txt') + await expect(fs.writeText(target, 'valid')).resolves.toMatchObject({ before: null, after: 'valid' }) + }) + + it('fails an overwrite when reading its text diff basis fails for another reason', async () => { + const remote = new FakeRemote() + remote.file('/workspace/file.txt', 'prior') + const { fs } = await setup(remote) + const target = await fs.resolve('file.txt') + remote.nextReadError = new Error('read transport failed') + await expectCode(fs.writeText(target, 'replacement'), 'FS_IO_ERROR') + expect(new TextDecoder().decode(remote.nodes.get('/workspace/file.txt')?.data)).toBe('prior') + }) + + it('enforces create and version intents before publication', async () => { + const remote = new FakeRemote() + remote.file('/workspace/file.txt', 'v1') + const { fs } = await setup(remote) + const target = await fs.resolve('file.txt') + const version = (await fs.stat(target))!.version + await expectCode(fs.writeText(target, 'blind', { kind: 'createIfAbsent' }), 'FS_NOT_OBSERVED') + remote.mutate('/workspace/file.txt', 'v2') + await expectCode(fs.writeText(target, 'stale', { kind: 'replaceIfVersion', version }), 'FS_STALE_VERSION') + await expectCode(fs.writeText(await fs.resolve('missing'), 'stale', { kind: 'replaceIfVersion', version }), 'FS_STALE_VERSION') + remote.dir('/workspace/dir') + await expectCode(fs.writeText(await fs.resolve('dir'), 'x'), 'FS_NOT_REGULAR_FILE') + }) + + it('does not turn an abort observed after a successful move into a failed write', async () => { + const remote = new FakeRemote() + const controller = new AbortController() + remote.abortAfterRename = controller + const { fs } = await setup(remote) + await expect(fs.writeText(await fs.resolve('committed'), 'yes', undefined, controller.signal)) + .resolves.toMatchObject({ operation: 'create' }) + expect(controller.signal.aborted).toBe(true) + }) + + it('cleans staging files and maps command, permission, and abort failures', async () => { + const remote = new FakeRemote() + const { fs } = await setup(remote) + const commandTarget = await fs.resolve('command') + remote.nextCommandError = commandError(1, 'chmod failed') + await expectCode(fs.writeText(commandTarget, 'x'), 'FS_IO_ERROR') + expect(remote.removals).toHaveLength(1) + + remote.nextRenameError = new Error('permission denied') + await expectCode(fs.writeText(await fs.resolve('permission'), 'x'), 'FS_PERMISSION_DENIED') + remote.nextRemoveError = new Error('cleanup also failed') + remote.nextRenameError = new DOMException('aborted', 'AbortError') + await expectCode(fs.writeText(await fs.resolve('abort'), 'x'), 'FS_ABORTED') + }) + + it('applies literal edits atomically and restores the detected CRLF style', async () => { + const remote = new FakeRemote() + remote.file('/workspace/file.txt', 'one\r\ntwo\r\nthree\n') + const { fs } = await setup(remote) + const target = await fs.resolve('file.txt') + const version = (await fs.stat(target))!.version + const outcome = await fs.editText( + target, + { oldString: 'two\r\n', newString: 'TWO\r\n', replaceAll: false }, + { version }, + ) + expect(outcome).toMatchObject({ before: 'one\ntwo\nthree\n', after: 'one\nTWO\nthree\n' }) + expect(new TextDecoder().decode(remote.nodes.get('/workspace/file.txt')?.data)).toBe('one\r\nTWO\r\nthree\r\n') + }) + + it('reports stale and literal-match failures with stable codes', async () => { + const remote = new FakeRemote() + remote.file('/workspace/file.txt', 'a a') + remote.dir('/workspace/dir') + const { fs } = await setup(remote) + const target = await fs.resolve('file.txt') + await expectCode(fs.editText(target, { oldString: '', newString: 'x', replaceAll: false }), 'FS_EDIT_NOT_FOUND') + await expectCode(fs.editText(target, { oldString: 'z', newString: 'x', replaceAll: false }), 'FS_EDIT_NOT_FOUND') + await expectCode(fs.editText(target, { oldString: 'a', newString: 'x', replaceAll: false }), 'FS_AMBIGUOUS_EDIT') + await expect(fs.editText(target, { oldString: 'a', newString: 'x', replaceAll: true })) + .resolves.toMatchObject({ after: 'x x' }) + await expectCode(fs.editText(target, { oldString: 'x', newString: 'y', replaceAll: false }, { version: FsVersion('stale') }), 'FS_STALE_VERSION') + await expectCode(fs.editText(await fs.resolve('missing'), { oldString: 'x', newString: 'y', replaceAll: false }), 'FS_STALE_VERSION') + await expectCode(fs.editText(await fs.resolve('dir'), { oldString: 'x', newString: 'y', replaceAll: false }), 'FS_NOT_REGULAR_FILE') + }) + + it('serializes guarded mutations so only one stale version can win', async () => { + const remote = new FakeRemote() + remote.file('/workspace/file.txt', 'base') + const { fs } = await setup(remote) + const target = await fs.resolve('file.txt') + const version = (await fs.stat(target))!.version + const results = await Promise.allSettled([ + fs.writeText(target, 'one', { kind: 'replaceIfVersion', version }), + fs.editText(target, { oldString: 'base', newString: 'two', replaceAll: false }, { version }), + ]) + expect(results.filter(result => result.status === 'fulfilled')).toHaveLength(1) + expect(results.filter(result => result.status === 'rejected')).toHaveLength(1) + }) +}) + +describe('E2B filesystem adapter integration edges', () => { + it('maps canonicalization, permission, and generic provider failures', async () => { + const remote = new FakeRemote() + const { fs } = await setup(remote) + remote.nextCommandError = commandError(1, 'not a directory') + await expectCode(fs.resolve('bad'), 'FS_IO_ERROR') + remote.nextCommandError = commandError(1) + await expectCode(fs.resolve('bad-again'), 'FS_IO_ERROR') + remote.nextCommandError = new Error('canonical transport failed') + await expectCode(fs.resolve('bad-transport'), 'FS_IO_ERROR') + remote.file('/workspace/a', 'a') + const target = await fs.resolve('a') + remote.nextInfoError = new Error('metadata transport failed') + await expectCode(fs.stat(target), 'FS_IO_ERROR') + remote.nextReadError = new Error('operation not permitted') + await expectCode(fs.readText(target), 'FS_PERMISSION_DENIED') + remote.nextReadError = 'transport vanished' + await expectCode(fs.readText(target), 'FS_IO_ERROR') + }) + + it('keeps a listed child whose metadata disappears as an other entry', async () => { + const remote = new FakeRemote() + remote.file('/workspace/a', 'a') + remote.disappearOnInfo.add('/workspace/a') + const { fs } = await setup(remote) + const listed = await fs.listDir(await fs.resolve('/workspace')) + expect(listed).toEqual([{ + name: 'a', + type: 'other', + target: { targetKey: '/workspace/a', displayPath: '/workspace/a' }, + }]) + }) + + it('registers the package-owned empty invariant installer', async () => { + const ctx = new Context() + await ctx.plugin(InvariantService, { enabled: true }) + const fiber = await ctx.plugin(E2BFsInvariant).await() + await fiber.dispose() + }) +}) diff --git a/packages/fs/fs-e2b/tsconfig.json b/packages/fs/fs-e2b/tsconfig.json new file mode 100644 index 0000000000..c424efd2e5 --- /dev/null +++ b/packages/fs/fs-e2b/tsconfig.json @@ -0,0 +1,25 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": ["src"], + "references": [ + { + "path": "../../../vendor/cosmokit" + }, + { + "path": "../../../vendor/cordis" + }, + { + "path": "../../e2b/e2b" + }, + { + "path": "../fs" + }, + { + "path": "../../support/invariants" + } + ] +} diff --git a/packages/subprocess/README.i18n.yaml b/packages/subprocess/README.i18n.yaml index 0ebb5bd4af..0efaf7838e 100644 --- a/packages/subprocess/README.i18n.yaml +++ b/packages/subprocess/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/subprocess/README.md -README.md: f2b19436da40feb14d067e2cfc706222625680b5 -README.zh.md: 938312448dd5c0a691ed07ddc9843cf2c4445637 +README.md: ae18c55205edd6085a0ed8de1bb7f875c411c79f +README.zh.md: e27fd240e4c1e96d8b859b326574e441241e8508 diff --git a/packages/subprocess/README.md b/packages/subprocess/README.md index f2b19436da..ae18c55205 100644 --- a/packages/subprocess/README.md +++ b/packages/subprocess/README.md @@ -2,11 +2,12 @@ English | [中文](README.zh.md) -The shared process substrate for one execution world: executable lookup, fully-specified managed child-process trees with raw or collected stdio, and one deep terminal-process primitive that owns PTY allocation, foreground groups, and provider-observable session cleanup. Command defaulting, shell semantics, deadlines, protocol framing, readiness, and presentation stay with consumers — the [bash executors](../bash/README.md), [LSP host](../lsp/README.md), [PTY shell backend](../pty/README.md), and [ACP subagent backend](../subagent/README.md). See the [subprocess seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md). +The shared home for spawning managed child-process trees: fully-specified spawn specs with Node-shaped per-stream stdio dispositions (raw pipes, inherit, bounded tail-keep collection with spill files), the one credential scrub every harness spawner uses, offset-based incremental reads, tree-scoped signalling with SIGTERM→grace→SIGKILL escalation, and the cooperative dispose ladder. Command defaulting, shell semantics, deadlines, protocol framing, and presentation stay with consumers — the [bash executors](../bash/README.md), the [LSP host](../lsp/README.md), and the [ACP subagent backend](../subagent/README.md). See the [subprocess seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md). | Package | ctx key | Role | |---|---|---| -| [`subprocess`](subprocess/README.md) (`@deepseek-ai/dsh-subprocess`) | `ctx.subprocess` | The seam: executable lookup, ordinary managed spawns, the terminal-process primitive, handle lifecycles, and shared environment/output vocabulary | -| [`subprocess-local`](subprocess-local/README.md) (`@deepseek-ai/dsh-subprocess-local`) | — | The local implementation: detached process trees, bounded collection/spill, `node-pty`, foreground/session inspection, tree signalling, and terminate-and-join disposal | +| [`subprocess`](subprocess/README.md) (`@deepseek-ai/dsh-subprocess`) | `ctx.subprocess` | The seam: abstract `SubprocessService.spawn(spec)`, the fully-explicit `SubprocessSpawnSpec` with per-stream stdio dispositions, `SubprocessHandle` (streams, offset-based readers, terminate/waitForExit/dispose), and the shared scrub + `DSH_*`/`CollectedOutput` vocabulary | +| [`subprocess-local`](subprocess-local/README.md) (`@deepseek-ai/dsh-subprocess-local`) | — | The local implementation: detached process trees, per-disposition stream wiring, tail-keep truncation with bounded private spill files, the `DSH_*` merge order, tree signalling with escalation, the dispose ladder, and terminate-and-join disposal | +| [`subprocess-e2b`](subprocess-e2b/README.md) (`@deepseek-ai/dsh-subprocess-e2b`) | — | Experimental E2B implementation: remote Linux process groups and spill state in the shared `ctx.e2b` sandbox, with asynchronous PID acquisition and SDK buffering limitations | The service owns process lifetime across consumer reloads; consumers own what a process means (a bash command, a future non-shell runner) and every default that shapes one. diff --git a/packages/subprocess/README.zh.md b/packages/subprocess/README.zh.md index 938312448d..e27fd240e4 100644 --- a/packages/subprocess/README.zh.md +++ b/packages/subprocess/README.zh.md @@ -1,12 +1,13 @@ -# subprocess/:子进程能力家族 +# subprocess/:进程管理能力家族 [English](README.md) | 中文 -这里集中提供一个执行世界的共享进程基底:可执行文件查找、具有原始或收集式 stdio 的完整指定受管子进程树,以及一项深层终端进程原语,负责 PTY 分配、前台进程组和提供方仍可观察到的会话成员清理。命令默认值补全、shell 语义、时限、协议分帧、就绪状态与呈现留在消费方:[bash 执行器](../bash/README.md)、[LSP 主机](../lsp/README.md)、[PTY shell 后端](../pty/README.md)与 [ACP(Agent Client Protocol)subagent 后端](../subagent/README.md)。参见[subprocess seam Agent Note(agent 决策记录)](../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md)。 +spawn 受管子进程树的共用归属位置:完全显式的 spawn spec,其 stdio 处置方式(disposition)为 Node 形状、按流划分(原始管道、inherit、附带 spill 文件的有界尾部保留收集);harness 中所有 spawn 调用方共用的那一份凭据清除;基于偏移量的增量读取;以进程树为范围、带 SIGTERM→宽限期→SIGKILL 升级的信号发送;以及协作式 dispose(资源释放)阶梯。命令默认值补全、shell 语义、deadline、协议分帧与呈现留在消费方:[bash 执行器](../bash/README.md)、[LSP 主机](../lsp/README.md)与 [ACP(Agent Client Protocol)subagent 后端](../subagent/README.md)。参见[进程管理器 seam Agent Note(agent 决策记录)](../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md)。 | 包(package) | ctx 键 | 角色 | |---|---|---| -| [`subprocess`](subprocess/README.md)(`@deepseek-ai/dsh-subprocess`) | `ctx.subprocess` | seam 本体:可执行文件查找、普通受管 spawn、终端进程原语、句柄生命周期,以及共享的环境/输出词汇 | -| [`subprocess-local`](subprocess-local/README.md)(`@deepseek-ai/dsh-subprocess-local`) | 无 | 本地实现:detached 进程树、有界收集/spill、`node-pty`、前台/会话检查、进程树信号发送,以及先终止再等待退出的资源释放 | +| [`subprocess`](subprocess/README.md)(`@deepseek-ai/dsh-subprocess`) | `ctx.subprocess` | seam 本体:抽象的 `SubprocessService.spawn(spec)`、完全显式且带按流划分 stdio 处置方式的 `SubprocessSpawnSpec`、`SubprocessHandle`(流、基于偏移量的读取器、terminate/waitForExit/dispose),以及共享的凭据清除 + `DSH_*`/`CollectedOutput` 词汇 | +| [`subprocess-local`](subprocess-local/README.md)(`@deepseek-ai/dsh-subprocess-local`) | 无 | 本地实现:detached 进程树、按处置方式接线的流、附带有界私有 spill 文件的尾部保留截断、`DSH_*` 合并次序、带升级的进程树信号发送、dispose 阶梯,以及先终止再等待退出的 dispose | +| [`subprocess-e2b`](subprocess-e2b/README.md)(`@deepseek-ai/dsh-subprocess-e2b`) | 无 | 实验性 E2B 实现:远程 Linux 进程组和共享 `ctx.e2b` 沙箱中的 spill 状态,但 PID 异步获取,且受 SDK 缓冲限制 | -即使消费方重载,进程生命周期仍由服务负责管理;消费方负责定义进程的含义(一条 bash 命令、未来的非 shell 运行器),以及决定塑造该进程的每一项默认值。 +服务拥有跨消费方重载的进程存续期;消费方拥有一个进程的含义(一条 bash 命令、未来的非 shell 运行器)以及塑造它的每一项默认值。 diff --git a/packages/subprocess/subprocess-e2b/README.i18n.yaml b/packages/subprocess/subprocess-e2b/README.i18n.yaml new file mode 100644 index 0000000000..dfbe97e8b9 --- /dev/null +++ b/packages/subprocess/subprocess-e2b/README.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write packages/subprocess/subprocess-e2b/README.md +README.md: 9ec1518103413f33a52023c17722bf5a5fe275ce +README.zh.md: f50ff2f3d85a5ddd9bd352e95d0eb0a23e756404 diff --git a/packages/subprocess/subprocess-e2b/README.md b/packages/subprocess/subprocess-e2b/README.md new file mode 100644 index 0000000000..9ec1518103 --- /dev/null +++ b/packages/subprocess/subprocess-e2b/README.md @@ -0,0 +1,32 @@ +# @deepseek-ai/dsh-subprocess-e2b + +English | [中文](README.zh.md) + +E2B implementation of the [`@deepseek-ai/dsh-subprocess`](../subprocess/README.md) seam. It has no config: load [`@deepseek-ai/dsh-e2b`](../../e2b/e2b/README.md) first, then this service in place of `dsh-subprocess-local`. Existing consumers such as [`dsh-bash-local`](../../bash/bash-local/README.md) then execute in the shared remote sandbox without an E2B-specific Bash adapter. + +## Behavior + +- **Asynchronous remote start** — the synchronous seam returns a handle immediately while `Sandbox.commands.run(..., { background: true })` starts remotely. `pid` is `-1` until the SDK returns the command PID; `done`, stdin, termination, and `waitForExit()` wait for readiness internally. +- **Linux process groups** — a quoted wrapper starts each argv under `setsid --wait` and records its actual process-group id plus private status files beneath `ctx.e2b.runtimeRoot/processes`. The handle waits for that file instead of assuming the SDK command PID is the group id. Termination signals the negative recorded id with `SIGTERM`, waits the caller's `graceMs`, then escalates to `SIGKILL` and the SDK kill fallback. Service disposal terminates and joins every retained handle before the sandbox owner disposes. +- **Environment boundary** — the wrapper starts from the sandbox command environment, removes ambient `DSH_*` and credential-shaped (`*KEY*`, `*SECRET*`, `*TOKEN*`) names, then restores every `spec.env` entry as an explicit caller opt-in. Host ambient variables never enter the sandbox implicitly. +- **Stdio projection** — pipe mode forwards E2B callbacks into host Node streams; inherit mode forwards them to the harness process streams; collect mode retains a bounded host tail with offset reads. Optional complete spill files are written remotely and advertised only while within their cap. Batch and streaming stdin use the SDK handle. + +The base E2B image supplies the Bash/GNU utilities this adapter invokes: `bash`, `setsid`, `ps`, `tr`, `env`, `chmod`, `tee`, and `kill`. A custom template must retain compatible commands. + +## Model Experience + +Indirectly, through consumer seams such as the Bash executor behind `dsh-tool-bash`, which render remote output, exit facts, background deltas, and spill paths. + +#### KV Cache effect + +No direct invalidation; the named consumers own any request-prefix changes. + +## Known Limitations and Deferred Work + +- **The SDK still retains complete command output in host memory** — E2B `CommandHandle.stdout` and `.stderr` accumulate even when this adapter exposes bounded tails, so the subprocess seam's normal host-memory bound is not achieved. +- **Pipe output is not byte-faithful** — E2B delivers separately decoded strings rather than raw bytes, so split multibyte sequences and arbitrary binary protocols can be corrupted; LSP and other framed byte-stream consumers are unsupported. +- **Synchronous-PID consumers are unsupported** — `pid` remains `-1` during remote startup; consumers that require a positive PID immediately, including the ACP child backend, cannot use this provider unchanged. +- **Reconnect does not reconstruct handles** — remote PID/status/spill files survive a retained sandbox, but a new harness process does not rebuild live `SubprocessHandle` objects or output cursors from them. +- **Remote state accumulates when retained** — process directories and valid spill files remain under `.dsh-e2b`; this POC supplies no retention sweep. +- **Signal attribution is inferred** — when termination was requested and E2B reports a nonzero exit code, the adapter reports the last requested signal because the SDK result does not identify the terminating signal. +- **Linux utility and E2B transport semantics are assumed** — there is no PTY, Windows, arbitrary-template, or network-partition fidelity layer. diff --git a/packages/subprocess/subprocess-e2b/README.zh.md b/packages/subprocess/subprocess-e2b/README.zh.md new file mode 100644 index 0000000000..f50ff2f3d8 --- /dev/null +++ b/packages/subprocess/subprocess-e2b/README.zh.md @@ -0,0 +1,32 @@ +# @deepseek-ai/dsh-subprocess-e2b + +[English](README.md) | 中文 + +[`@deepseek-ai/dsh-subprocess`](../subprocess/README.md) seam 的 E2B 实现。它没有配置:先加载 [`@deepseek-ai/dsh-e2b`](../../e2b/e2b/README.md),再用本服务取代 `dsh-subprocess-local`。随后,[`dsh-bash-local`](../../bash/bash-local/README.md) 等现有消费方会在共享远程沙箱中执行,无需 E2B 专用 Bash 适配器。 + +## 行为 + +- **异步远程启动**:同步 seam 会立即返回一个句柄,同时由 `Sandbox.commands.run(..., { background: true })` 在远程启动进程。SDK 返回命令 PID 之前,`pid` 为 `-1`;`done`、stdin、终止和 `waitForExit()` 会在内部等待就绪。 +- **Linux 进程组**:带引号保护的包装层会在 `setsid --wait` 下启动每组 argv,并在 `ctx.e2b.runtimeRoot/processes` 下记录实际进程组 ID 和私有状态文件。句柄会等待该文件,而不会假设 SDK 命令 PID 就是进程组 ID。终止操作以记录的负数 ID 发送 `SIGTERM`,等待调用方的 `graceMs`,再升级到 `SIGKILL` 和 SDK kill 回退。服务 dispose(资源释放)会在沙箱所有者释放前终止并等待每个保留句柄退出。 +- **环境边界**:包装层从沙箱命令环境开始,移除环境中的 `DSH_*` 和形似凭据的名称(`*KEY*`、`*SECRET*`、`*TOKEN*`),再把每个 `spec.env` 条目恢复为调用方显式选择。宿主环境变量绝不会隐式进入沙箱。 +- **stdio 投影**:pipe 模式把 E2B 回调转发到宿主 Node 流;inherit 模式把回调转发到 harness 进程流;collect 模式保留有界的宿主尾部,并支持基于偏移量读取。可选的完整 spill 文件写在远程,并且只有未超过其上限时才会对外公布。批量 stdin 和流式 stdin 都使用 SDK 句柄。 + +基础 E2B 镜像提供该适配器调用的 Bash/GNU 工具:`bash`、`setsid`、`ps`、`tr`、`env`、`chmod`、`tee` 和 `kill`。自定义模板必须保留兼容的命令。 + +## 模型体验 + +通过消费方 seam 间接影响模型,例如 `dsh-tool-bash` 背后的 Bash 执行器;这些消费方会渲染远程输出、退出事实、后台增量和 spill 路径。 + +#### KV Cache 影响 + +不会直接失效;请求前缀变更由具名消费方负责。 + +## 已知限制与延后工作 + +- **SDK 仍会在宿主内存中保留完整命令输出**:即使本适配器公开的是有界尾部,E2B `CommandHandle.stdout` 和 `.stderr` 仍会持续累积,因此无法达到进程管理 seam 通常提供的宿主内存边界。 +- **Pipe 输出并非字节保真**:E2B 交付的是分别解码后的字符串,而不是原始字节,因此拆分的多字节序列和任意二进制协议可能损坏;不支持 LSP 及其他带帧字节流消费方。 +- **不支持需要同步 PID 的消费方**:远程启动期间,`pid` 保持为 `-1`;包括 ACP 子进程后端在内,要求立即获得正 PID 的消费方无法原样使用本提供方。 +- **重新连接不会重建句柄**:保留沙箱后,远程 PID/状态/spill 文件仍然存在,但新的 harness 进程不会据此重建实时 `SubprocessHandle` 对象或输出游标。 +- **保留沙箱时会累积远程状态**:进程目录和有效的 spill 文件会留在 `.dsh-e2b` 下;本 POC 不提供保留清理。 +- **信号归因依靠推断**:如果已经请求终止,而 E2B 报告非零退出码,适配器会报告最后请求的信号,因为 SDK 结果不标识终止信号。 +- **依赖 Linux 工具与 E2B 传输语义**:没有 PTY、Windows、任意模板或网络分区的保真层。 diff --git a/packages/subprocess/subprocess-e2b/package.json b/packages/subprocess/subprocess-e2b/package.json new file mode 100644 index 0000000000..d73607d1f3 --- /dev/null +++ b/packages/subprocess/subprocess-e2b/package.json @@ -0,0 +1,41 @@ +{ + "name": "@deepseek-ai/dsh-subprocess-e2b", + "description": "E2B subprocess implementation for DeepSeek Harness", + "version": "0.0.1", + "private": true, + "type": "module", + "main": "lib/index.js", + "types": "lib/types/index.d.ts", + "exports": { + ".": { + "types": "./lib/types/index.d.ts", + "default": "./lib/index.js" + }, + "./invariant": { + "types": "./lib/types/invariant.d.ts", + "default": "./lib/invariant.js" + }, + "./src/*": "./src/*", + "./package.json": "./package.json" + }, + "files": [ + "lib/index.js", + "lib/invariant.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", + "src" + ], + "license": "BSD-3-Clause", + "peerDependencies": { + "@deepseek-ai/dsh-e2b": "^0.0.1", + "@deepseek-ai/dsh-invariants": "^0.0.1", + "@deepseek-ai/dsh-subprocess": "^0.0.1", + "cordis": "^4.0.0-rc.7" + }, + "devDependencies": { + "@deepseek-ai/dsh-e2b": "workspace:^", + "@deepseek-ai/dsh-invariants": "workspace:^", + "@deepseek-ai/dsh-subprocess": "workspace:^", + "cordis": "^4.0.0-rc.7" + } +} diff --git a/packages/subprocess/subprocess-e2b/src/index.ts b/packages/subprocess/subprocess-e2b/src/index.ts new file mode 100644 index 0000000000..2856df67bd --- /dev/null +++ b/packages/subprocess/subprocess-e2b/src/index.ts @@ -0,0 +1,58 @@ +/** + * E2B implementation of the subprocess seam. Each handle starts through the + * shared sandbox and retains command output/status paths in that remote world. + * @module @deepseek-ai/dsh-subprocess-e2b + */ + +import { randomUUID } from 'node:crypto' +import { posix } from 'node:path' +import { Context } from 'cordis' +import { SubprocessService } from '@deepseek-ai/dsh-subprocess' +import type { SubprocessHandle, SubprocessSpawnSpec } from '@deepseek-ai/dsh-subprocess' +import { E2BSubprocessHandle } from './process.ts' + +/** E2B command manager registered as `ctx.subprocess`. */ +export class E2BSubprocessService extends SubprocessService { + static inject = ['e2b'] + + private readonly live = new Set() + + /** Create the E2B subprocess service and bind its disposal policy. */ + constructor(ctx: Context) { + super(ctx) + ctx.effect(() => async () => { + const handles = [...this.live] + for (const handle of handles) handle.terminate() + await Promise.all(handles.map(async (handle) => { + await handle.done.catch(() => {}) + await handle.waitForExit() + })) + this.live.clear() + }, 'e2b subprocess teardown') + } + + /** @inheritdoc */ + spawn(spec: SubprocessSpawnSpec): SubprocessHandle { + const program = spec.argv[0] + if (program === undefined || program.length === 0) { + throw new Error('invalid argv: expected a non-empty program name at argv[0]') + } + if (!Number.isFinite(spec.graceMs) || spec.graceMs <= 0) { + throw new Error('subprocess-e2b: graceMs must be a positive finite number') + } + if (spec.signal?.aborted === true) { + throw new Error(`aborted before spawn: ${String(spec.signal.reason ?? 'aborted')}`) + } + const stateDir = posix.join(this.ctx.e2b.runtimeRoot, 'processes', randomUUID()) + const handle = new E2BSubprocessHandle(this.ctx.e2b, spec, stateDir) + this.live.add(handle) + const release = async (): Promise => { + await handle.waitForExit() + this.live.delete(handle) + } + void handle.done.then(release, release).catch(() => {}) + return handle + } +} + +export default E2BSubprocessService diff --git a/packages/subprocess/subprocess-e2b/src/invariant.ts b/packages/subprocess/subprocess-e2b/src/invariant.ts new file mode 100644 index 0000000000..4416175b1b --- /dev/null +++ b/packages/subprocess/subprocess-e2b/src/invariant.ts @@ -0,0 +1,27 @@ +/** Package-owned invariant companion for `@deepseek-ai/dsh-subprocess-e2b`. */ + +/* jscpd:ignore-start */ +import type { Context } from 'cordis' +import type { InvariantInstaller } from '@deepseek-ai/dsh-invariants' + +const PACKAGE_NAME = '@deepseek-ai/dsh-subprocess-e2b' + +/** Cordis companion plugin name. */ +export const name = 'subprocess-e2b-invariant' +/** Service required before reserving package ownership. */ +export const inject = ['invariants'] + +/** + * No runtime invariant: live remote handles are private teardown ownership, + * and the E2B command event stream is the sole outcome authority. + */ +const install: InvariantInstaller = () => {} + +/** + * Register this package's invariant companion. + * @param ctx - Cordis context carrying the invariant service. + * @returns the installed registration's disposer after setup succeeds. + */ +export const apply = (ctx: Context): Promise<() => void> => + Promise.resolve(ctx.invariants.register(PACKAGE_NAME, install)) +/* jscpd:ignore-end */ diff --git a/packages/subprocess/subprocess-e2b/src/output.ts b/packages/subprocess/subprocess-e2b/src/output.ts new file mode 100644 index 0000000000..b6551983a0 --- /dev/null +++ b/packages/subprocess/subprocess-e2b/src/output.ts @@ -0,0 +1,70 @@ +/** Bounded host-side projection of a complete output file retained in E2B. */ + +import { Buffer } from 'node:buffer' +import type { SubprocessOutputRead, SubprocessOutputReader } from '@deepseek-ai/dsh-subprocess' + +/** Offset reader used for one collect-mode E2B stream. */ +export class E2BOutputReader implements SubprocessOutputReader { + private chunks: Buffer[] = [] + private retainedBytes = 0 + private totalBytes = 0 + + /** + * Create a bounded reader over one remote spill path. + * @param maxBytes - In-memory tail cap. + * @param maxSpillBytes - Maximum complete remote file size the caller accepts. + * @param spillPath - Remote full-output path. + */ + constructor( + private readonly maxBytes: number, + private readonly maxSpillBytes: number | undefined, + private readonly spillPath: string, + ) {} + + /** Total bytes observed from the SDK stream. */ + get size(): number { + return this.totalBytes + } + + /** + * Append one decoded SDK output event. + * @param text - Event text delivered by E2B. + */ + push(text: string): void { + if (text.length === 0) return + const chunk = Buffer.from(text) + this.totalBytes += chunk.length + this.chunks.push(chunk) + this.retainedBytes += chunk.length + while (this.retainedBytes > this.maxBytes) { + const head = this.chunks[0] as Buffer + const excess = this.retainedBytes - this.maxBytes + if (head.length <= excess) { + this.chunks.shift() + this.retainedBytes -= head.length + } else { + this.chunks[0] = head.subarray(excess) + this.retainedBytes -= excess + } + } + } + + /** @inheritdoc */ + readFrom(fromByte: number): SubprocessOutputRead { + if (!Number.isSafeInteger(fromByte) || fromByte < 0) { + throw new Error('subprocess output offset must be a non-negative safe integer') + } + const retained = Buffer.concat(this.chunks, this.retainedBytes) + const firstRetained = this.totalBytes - this.retainedBytes + const lossy = fromByte < firstRetained + const start = lossy ? 0 : Math.min(retained.length, Math.max(0, fromByte - firstRetained)) + return { + text: retained.subarray(start).toString('utf8'), + nextOffset: this.totalBytes, + lossy, + ...(lossy && this.maxSpillBytes !== undefined && this.totalBytes <= this.maxSpillBytes + ? { spillPath: this.spillPath } + : {}), + } + } +} diff --git a/packages/subprocess/subprocess-e2b/src/process.ts b/packages/subprocess/subprocess-e2b/src/process.ts new file mode 100644 index 0000000000..507ea700c4 --- /dev/null +++ b/packages/subprocess/subprocess-e2b/src/process.ts @@ -0,0 +1,417 @@ +/** One asynchronously-started E2B command projected onto the subprocess seam. */ + +import { Buffer } from 'node:buffer' +import { PassThrough, Writable } from 'node:stream' +import { posix } from 'node:path' +import { + CommandExitError, + quoteE2BShellArg, +} from '@deepseek-ai/dsh-e2b' +import type { CommandHandle, CommandResult, Sandbox } from '@deepseek-ai/dsh-e2b' +import type { + SubprocessCollect, + SubprocessHandle, + SubprocessOutcome, + SubprocessOutputMode, + SubprocessSpawnSpec, +} from '@deepseek-ai/dsh-subprocess' +import type E2BSandboxService from '@deepseek-ai/dsh-e2b' +import { E2BOutputReader } from './output.ts' + +const GROUP_POLL_MS = 20 + +function isCollect(mode: SubprocessOutputMode): mode is SubprocessCollect { + return mode !== 'pipe' && mode !== 'inherit' +} + +function hasSpill(mode: SubprocessOutputMode): mode is SubprocessCollect & { spill: { maxBytes: number } } { + return isCollect(mode) && mode.spill !== undefined +} + +function asError(error: unknown): Error { + return error instanceof Error ? error : new Error(String(error)) +} + +class DeferredStdin extends Writable { + constructor(private readonly ready: Promise) { + super({ decodeStrings: false }) + } + + override _write(chunk: string | Buffer, _encoding: BufferEncoding, callback: (error?: Error | null) => void): void { + void this.ready.then(handle => handle.sendStdin(chunk)).then( + () => { callback() }, + (error: unknown) => { callback(asError(error)) }, + ) + } + + override _final(callback: (error?: Error | null) => void): void { + void this.ready.then(handle => handle.closeStdin()).then( + () => { callback() }, + (error: unknown) => { callback(asError(error)) }, + ) + } +} + +interface RemotePaths { + pid: string + status: string + stdout: string + stderr: string +} + +function explicitEnvironmentNames(env: Readonly> | undefined): string { + return Object.keys(env ?? {}) + .map(quoteE2BShellArg) + .join(' ') +} + +function commandText(spec: SubprocessSpawnSpec, paths: RemotePaths): string { + const stdoutRedirect = hasSpill(spec.stdio.stdout) + ? `> >(tee -a -- ${quoteE2BShellArg(paths.stdout)})` + : '' + const stderrRedirect = hasSpill(spec.stdio.stderr) + ? `2> >(tee -a -- ${quoteE2BShellArg(paths.stderr)} >&2)` + : '' + const environmentNames = explicitEnvironmentNames(spec.env) + const inner = [ + 'set +e', + 'umask 077', + 'dsh_e2b_pgid="$(ps -o pgid= -p "$$" | tr -d " ")"', + `printf '%s\\n' "$dsh_e2b_pgid" > ${quoteE2BShellArg(paths.pid)}`, + 'dsh_e2b_env=()', + `dsh_e2b_explicit=(${environmentNames})`, + 'while IFS= read -r dsh_e2b_name; do', + ' case "${dsh_e2b_name^^}" in DSH_*|*KEY*|*SECRET*|*TOKEN*) continue ;; esac', + ' dsh_e2b_env+=("$dsh_e2b_name=${!dsh_e2b_name}")', + 'done < <(compgen -e)', + 'for dsh_e2b_name in "${dsh_e2b_explicit[@]}"; do dsh_e2b_env+=("$dsh_e2b_name=${!dsh_e2b_name}"); done', + `env -i "\${dsh_e2b_env[@]}" "$@" ${stdoutRedirect} ${stderrRedirect}`.trimEnd(), + 'dsh_e2b_status=$?', + 'wait', + `printf '%s\\n' "$dsh_e2b_status" > ${quoteE2BShellArg(paths.status)}`, + 'exit "$dsh_e2b_status"', + ].join('\n') + const argv = spec.argv.map(quoteE2BShellArg).join(' ') + return `exec setsid --wait -- bash -c ${quoteE2BShellArg(inner)} dsh-e2b ${argv}` +} + +function signalOpts(signal: AbortSignal | undefined): { signal?: AbortSignal } { + return signal === undefined ? {} : { signal } +} + +function isAborted(signal: AbortSignal | undefined): boolean { + return signal?.aborted === true +} + +function waitTick(signal?: AbortSignal): Promise { + if (signal?.aborted === true) return Promise.resolve(false) + return new Promise((resolve) => { + const timer = setTimeout(() => { + signal?.removeEventListener('abort', onAbort) + resolve(true) + }, GROUP_POLL_MS) + const onAbort = (): void => { + clearTimeout(timer) + resolve(false) + } + signal?.addEventListener('abort', onAbort, { once: true }) + }) +} + +/** E2B-backed subprocess handle with deferred remote PID acquisition. */ +export class E2BSubprocessHandle implements SubprocessHandle { + readonly stdin: Writable | undefined + readonly stdout: PassThrough | undefined + readonly stderr: PassThrough | undefined + readonly collected: SubprocessHandle['collected'] + readonly done: Promise + + private readonly readyState = Promise.withResolvers() + private readonly stdoutReader: E2BOutputReader | undefined + private readonly stderrReader: E2BOutputReader | undefined + private readonly paths: RemotePaths + private remotePid = -1 + private settled = false + private terminationRequested = false + private terminationSignal: NodeJS.Signals | null = null + private termination: Promise | undefined + + /** + * Begin an E2B command without blocking the synchronous subprocess spawn seam. + * @param runtime - Shared E2B sandbox owner. + * @param spec - Fully resolved subprocess request. + * @param stateDir - Remote directory retaining process identity, status, and valid spills. + */ + constructor( + private readonly runtime: E2BSandboxService, + private readonly spec: SubprocessSpawnSpec, + readonly stateDir: string, + ) { + this.paths = { + pid: posix.join(stateDir, 'pid'), + status: posix.join(stateDir, 'exit-code'), + stdout: posix.join(stateDir, 'stdout.log'), + stderr: posix.join(stateDir, 'stderr.log'), + } + const outMode = spec.stdio.stdout + const errMode = spec.stdio.stderr + this.stdout = outMode === 'pipe' ? new PassThrough() : undefined + this.stderr = errMode === 'pipe' ? new PassThrough() : undefined + this.stdoutReader = isCollect(outMode) + ? new E2BOutputReader(outMode.maxBytes, outMode.spill?.maxBytes, this.paths.stdout) + : undefined + this.stderrReader = isCollect(errMode) + ? new E2BOutputReader(errMode.maxBytes, errMode.spill?.maxBytes, this.paths.stderr) + : undefined + this.collected = { + ...(this.stdoutReader !== undefined ? { stdout: this.stdoutReader } : {}), + ...(this.stderrReader !== undefined ? { stderr: this.stderrReader } : {}), + } + this.stdin = spec.stdio.stdin === 'pipe' ? new DeferredStdin(this.readyState.promise) : undefined + void this.readyState.promise.catch(() => {}) + spec.signal?.addEventListener('abort', this.onAbort, { once: true }) + this.done = this.run() + void this.done.catch(() => {}) + if (spec.signal?.aborted === true) this.terminate() + } + + /** Remote process id after start; `-1` while E2B startup is pending or after it fails. */ + get pid(): number { + return this.remotePid + } + + /** @inheritdoc */ + terminate(): void { + if (this.terminationRequested || this.settled) return + this.terminationRequested = true + this.termination = this.terminateRemote() + void this.termination.catch(() => {}) + } + + /** @inheritdoc */ + async waitForExit(signal?: AbortSignal): Promise { + let handle: CommandHandle | undefined + try { + handle = await this.readyForWait(signal) + } catch { + return true + } + if (handle === undefined) return false + let sandbox: Sandbox + try { + sandbox = await this.runtime.getSandbox() + } catch (error: unknown) { + if (isAborted(signal)) return false + throw error + } + while (await this.groupAlive(sandbox, this.remotePid, signal)) { + if (!await waitTick(signal)) return false + } + return !isAborted(signal) + } + + private readyForWait(signal: AbortSignal | undefined): Promise { + if (signal === undefined) return this.readyState.promise + return new Promise((resolve, reject) => { + const onAbort = (): void => { cleanup(); resolve(undefined) } + const cleanup = (): void => { signal.removeEventListener('abort', onAbort) } + signal.addEventListener('abort', onAbort, { once: true }) + if (signal.aborted) { + onAbort() + return + } + void this.readyState.promise.then( + (handle) => { cleanup(); resolve(handle) }, + (error: unknown) => { cleanup(); reject(asError(error)) }, + ) + }) + } + + private readonly onAbort = (): void => { this.terminate() } + + private async run(): Promise { + try { + const sandbox = await this.runtime.getSandbox() + await this.prepareState(sandbox) + const handle = await sandbox.commands.run( + commandText(this.spec, this.paths), + { + background: true, + cwd: this.spec.cwd, + stdin: this.spec.stdio.stdin !== 'ignore', + timeoutMs: 0, + ...(this.spec.env !== undefined ? { envs: this.spec.env } : {}), + onStdout: async (data) => { await this.dispatchOutput('stdout', data) }, + onStderr: async (data) => { await this.dispatchOutput('stderr', data) }, + }, + ) + if (!Number.isSafeInteger(handle.pid) || handle.pid <= 0) { + throw new Error(`subprocess-e2b: E2B returned invalid command pid ${handle.pid}`) + } + const completion = handle.wait() + void completion.catch(() => {}) + this.remotePid = await this.waitForProcessGroupId(sandbox, completion) + this.readyState.resolve(handle) + await this.writeBatchStdin(handle) + const outcome = await this.waitForCommand(completion) + await this.finalizeSpills(sandbox) + return outcome + } catch (error: unknown) { + this.readyState.reject(error) + throw error + } finally { + this.settled = true + this.spec.signal?.removeEventListener('abort', this.onAbort) + this.stdout?.end() + this.stderr?.end() + } + } + + private async prepareState(sandbox: Sandbox): Promise { + await sandbox.files.makeDir(this.stateDir) + const files = [ + { path: this.paths.pid, data: '' }, + { path: this.paths.status, data: '' }, + ...(hasSpill(this.spec.stdio.stdout) ? [{ path: this.paths.stdout, data: '' }] : []), + ...(hasSpill(this.spec.stdio.stderr) ? [{ path: this.paths.stderr, data: '' }] : []), + ] + await sandbox.files.write(files) + await sandbox.commands.run([ + `chmod 700 -- ${quoteE2BShellArg(this.stateDir)}`, + `chmod 600 -- ${files.map(file => quoteE2BShellArg(file.path)).join(' ')}`, + ].join('\n')) + } + + private async writeBatchStdin(handle: CommandHandle): Promise { + if (typeof this.spec.stdio.stdin !== 'object') return + try { + await handle.sendStdin(this.spec.stdio.stdin.data) + await handle.closeStdin() + } catch (_processClosedItsInput) { + // Like the local adapter, batch stdin is best-effort; exit and output remain authoritative. + } + } + + private async dispatchOutput(stream: 'stdout' | 'stderr', data: string): Promise { + try { + if (stream === 'stdout') { + this.stdoutReader?.push(data) + await this.writeOutput(this.stdout, this.spec.stdio.stdout === 'inherit' ? process.stdout : undefined, data) + return + } + this.stderrReader?.push(data) + await this.writeOutput(this.stderr, this.spec.stdio.stderr === 'inherit' ? process.stderr : undefined, data) + } catch (error: unknown) { + const target = stream === 'stdout' ? this.stdout : this.stderr + target?.destroy(asError(error)) + } + } + + private async writeOutput(pipe: PassThrough | undefined, inherited: NodeJS.WriteStream | undefined, data: string): Promise { + const target = pipe ?? inherited + if (target === undefined || data.length === 0) return + if (target.destroyed) throw new Error('subprocess output stream is closed') + if (target.write(Buffer.from(data))) return + await new Promise((resolve, reject) => { + const onDrain = (): void => { cleanup(); resolve() } + const onError = (error: Error): void => { cleanup(); reject(error) } + const cleanup = (): void => { + target.removeListener('drain', onDrain) + target.removeListener('error', onError) + } + target.once('drain', onDrain) + target.once('error', onError) + }) + } + + private async waitForProcessGroupId(sandbox: Sandbox, completion: Promise): Promise { + const commandSettled = completion.then( + () => true, + () => true, + ) + while (true) { + const raw = await sandbox.files.read(this.paths.pid) + const value = raw.trim() + if (value.length > 0) { + const pid = Number(value) + if (!/^[1-9][0-9]*$/.test(value) || !Number.isSafeInteger(pid)) { + throw new Error(`subprocess-e2b: remote wrapper published invalid process-group id ${JSON.stringify(value)}`) + } + return pid + } + const settled = await Promise.race([commandSettled, waitTick().then(() => false)]) + if (settled) throw new Error('subprocess-e2b: remote command exited before publishing its process-group id') + } + } + + private async waitForCommand(completion: Promise): Promise { + try { + const result = await completion + return { exitCode: result.exitCode, signal: null } + } catch (error: unknown) { + if (error instanceof CommandExitError) { + return this.terminationSignal === null + ? { exitCode: error.exitCode, signal: null } + : { exitCode: null, signal: this.terminationSignal } + } + throw error + } + } + + private async terminateRemote(): Promise { + let handle: CommandHandle + try { + handle = await this.readyState.promise + } catch { + return + } + const sandbox = await this.runtime.getSandbox() + this.terminationSignal = 'SIGTERM' + await this.signalGroup(sandbox, this.remotePid, 'TERM') + const deadline = Date.now() + this.spec.graceMs + while (Date.now() < deadline && await this.groupAlive(sandbox, this.remotePid)) { + await waitTick() + } + if (!await this.groupAlive(sandbox, this.remotePid)) return + this.terminationSignal = 'SIGKILL' + try { + await this.signalGroup(sandbox, this.remotePid, 'KILL') + } finally { + await handle.kill().catch(() => false) + } + } + + private async signalGroup(sandbox: Sandbox, pid: number, signal: 'TERM' | 'KILL'): Promise { + try { + await sandbox.commands.run(`kill -${signal} -- -${pid}`) + } catch (error: unknown) { + if (!(error instanceof CommandExitError)) throw error + } + } + + private async groupAlive(sandbox: Sandbox, pid: number, signal?: AbortSignal): Promise { + try { + await sandbox.commands.run(`kill -0 -- -${pid}`, signalOpts(signal)) + return true + } catch (error: unknown) { + if (signal?.aborted === true) return false + if (error instanceof CommandExitError) return false + throw error + } + } + + private async finalizeSpills(sandbox: Sandbox): Promise { + const removals: Promise[] = [] + const collect = (mode: SubprocessOutputMode, reader: E2BOutputReader | undefined, path: string): void => { + if (!hasSpill(mode)) return + // A spill mode is a collect mode, so construction always created its reader. + const size = (reader as E2BOutputReader).size + if (size <= mode.maxBytes || size > mode.spill.maxBytes) { + removals.push(sandbox.files.remove(path).catch(() => {})) + } + } + collect(this.spec.stdio.stdout, this.stdoutReader, this.paths.stdout) + collect(this.spec.stdio.stderr, this.stderrReader, this.paths.stderr) + await Promise.all(removals) + } +} diff --git a/packages/subprocess/subprocess-e2b/tests/subprocess.spec.ts b/packages/subprocess/subprocess-e2b/tests/subprocess.spec.ts new file mode 100644 index 0000000000..ff9d48073c --- /dev/null +++ b/packages/subprocess/subprocess-e2b/tests/subprocess.spec.ts @@ -0,0 +1,725 @@ +import { once } from 'node:events' +import { Context } from 'cordis' +import { + CommandExitError, + type CommandHandle, + type CommandResult, + type Sandbox, +} from '@deepseek-ai/dsh-e2b' +import type E2BSandboxService from '@deepseek-ai/dsh-e2b' +import type { SubprocessSpawnSpec } from '@deepseek-ai/dsh-subprocess' +import E2BSubprocessService from '@deepseek-ai/dsh-subprocess-e2b' +import * as E2BSubprocessInvariant from '../src/invariant.ts' +import { E2BOutputReader } from '../src/output.ts' +import { E2BSubprocessHandle } from '../src/process.ts' +import InvariantService from '@deepseek-ai/dsh-invariants' +import { describe, expect, it, vi } from 'vitest' + +function commandError(exitCode: number): CommandExitError { + return new CommandExitError({ exitCode, stdout: '', stderr: '', error: `exit ${exitCode}` }) +} + +interface StartOptions { + background: true + cwd: string + stdin: boolean + timeoutMs: number + signal?: AbortSignal + envs?: Record + onStdout?: (data: string) => void | Promise + onStderr?: (data: string) => void | Promise +} + +class FakeCommandHandle { + pid = 4242 + readonly sent: Array = [] + closes = 0 + kills = 0 + killError: unknown + private readonly result = Promise.withResolvers() + private settled = false + + wait(): Promise { + return this.result.promise + } + + async sendStdin(data: string | Uint8Array): Promise { + this.sent.push(data) + } + + async closeStdin(): Promise { + this.closes += 1 + } + + async kill(): Promise { + this.kills += 1 + if (this.killError !== undefined) throw this.killError + return true + } + + succeed(exitCode = 0): void { + if (this.settled) return + this.settled = true + this.result.resolve({ exitCode, stdout: '', stderr: '' }) + } + + fail(exitCode: number): void { + if (this.settled) return + this.settled = true + this.result.reject(commandError(exitCode)) + } + + crash(error: unknown): void { + if (this.settled) return + this.settled = true + this.result.reject(error) + } +} + +class FakeSandbox { + readonly handle = new FakeCommandHandle() + readonly commandsSeen: string[] = [] + readonly writtenFiles: string[][] = [] + readonly removed: string[] = [] + readonly directories: string[] = [] + startOptions: StartOptions | undefined + backgroundError: unknown + nextRemoveError: unknown + probeError: unknown + signalError: unknown + trapsTerm = false + alive = true + processGroupId = '4242\n' + readonly processGroupReads: string[] = [] + beforeProbe: (() => void) | undefined + afterProbe: (() => void) | undefined + private startGate: Promise | undefined + private openStart: (() => void) | undefined + + deferStart(): void { + const gate = Promise.withResolvers() + this.startGate = gate.promise + this.openStart = () => { gate.resolve(undefined) } + } + + releaseStart(): void { + this.openStart?.() + } + + finish(exitCode = 0): void { + this.alive = false + if (exitCode === 0) this.handle.succeed(0) + else this.handle.fail(exitCode) + } + + async stdout(data: string): Promise { + await this.startOptions?.onStdout?.(data) + } + + async stderr(data: string): Promise { + await this.startOptions?.onStderr?.(data) + } + + readonly sandbox = { + sandboxId: 'fake', + files: { + makeDir: async (path: string): Promise => { + this.directories.push(path) + return true + }, + write: async (files: Array<{ path: string; data: string }>): Promise => { + this.writtenFiles.push(files.map(file => file.path)) + return files.map(() => ({})) + }, + read: async (): Promise => this.processGroupReads.shift() ?? this.processGroupId, + remove: async (path: string): Promise => { + this.removed.push(path) + if (this.nextRemoveError !== undefined) { + const error = this.nextRemoveError + this.nextRemoveError = undefined + throw error + } + }, + }, + commands: { + run: async (command: string, options?: StartOptions | { signal?: AbortSignal }): Promise => { + this.commandsSeen.push(command) + if (command.startsWith('kill -0 ')) { + this.beforeProbe?.() + if (options?.signal?.aborted === true) throw new DOMException('aborted', 'AbortError') + if (this.probeError !== undefined) { + const error = this.probeError + this.probeError = undefined + throw error + } + if (!this.alive) throw commandError(1) + this.afterProbe?.() + return { exitCode: 0, stdout: '', stderr: '' } + } + if (command.startsWith('kill -TERM ')) { + if (this.signalError !== undefined) { + const error = this.signalError + this.signalError = undefined + throw error + } + if (!this.trapsTerm) { + this.alive = false + this.handle.fail(143) + } + return { exitCode: 0, stdout: '', stderr: '' } + } + if (command.startsWith('kill -KILL ')) { + if (this.signalError !== undefined) { + const error = this.signalError + this.signalError = undefined + throw error + } + this.alive = false + this.handle.fail(137) + return { exitCode: 0, stdout: '', stderr: '' } + } + if ((options as StartOptions | undefined)?.background === true) { + this.startOptions = options as StartOptions + await this.startGate + if (this.backgroundError !== undefined) throw this.backgroundError + return this.handle as unknown as CommandHandle + } + return { exitCode: 0, stdout: '', stderr: '' } + }, + }, + } as unknown as Sandbox +} + +function spec(overrides: Partial = {}): SubprocessSpawnSpec { + return { + argv: ['bash', '-c', 'printf ok'], + cwd: '/workspace', + stdio: { + stdin: 'ignore', + stdout: { maxBytes: 4, spill: { maxBytes: 16 } }, + stderr: { maxBytes: 4 }, + }, + graceMs: 5, + ...overrides, + } +} + +function runtime(fake: FakeSandbox, getSandbox: () => Promise = async () => fake.sandbox): E2BSandboxService { + return { + cwd: '/workspace', + runtimeRoot: '/workspace/.dsh-e2b', + disposeMode: 'kill', + getSandbox, + } as unknown as E2BSandboxService +} + +async function flush(): Promise { + await new Promise(resolve => setTimeout(resolve, 0)) +} + +describe('E2BOutputReader', () => { + it('keeps a byte-exact tail with independent whole-stream cursors', () => { + const reader = new E2BOutputReader(4, 10, '/remote/spill') + reader.push('') + reader.push('ab') + reader.push('cdef') + expect(reader.size).toBe(6) + expect(reader.readFrom(0)).toEqual({ text: 'cdef', nextOffset: 6, lossy: true, spillPath: '/remote/spill' }) + expect(reader.readFrom(2)).toEqual({ text: 'cdef', nextOffset: 6, lossy: false }) + expect(reader.readFrom(5)).toEqual({ text: 'f', nextOffset: 6, lossy: false }) + expect(reader.readFrom(99)).toEqual({ text: '', nextOffset: 6, lossy: false }) + }) + + it('drops whole head chunks and withholds absent or over-cap spills', () => { + const withoutSpill = new E2BOutputReader(2, undefined, '/unused') + withoutSpill.push('ab') + withoutSpill.push('cd') + expect(withoutSpill.readFrom(0)).toEqual({ text: 'cd', nextOffset: 4, lossy: true }) + const overCap = new E2BOutputReader(2, 3, '/too-small') + overCap.push('abcd') + expect(overCap.readFrom(0)).toEqual({ text: 'cd', nextOffset: 4, lossy: true }) + expect(() => overCap.readFrom(-1)).toThrow(/non-negative safe integer/) + expect(() => overCap.readFrom(1.5)).toThrow(/non-negative safe integer/) + }) +}) + +describe('E2BSubprocessHandle', () => { + it('starts asynchronously, keeps secrets out of the command, and supports deferred piped stdin/output', async () => { + const fake = new FakeSandbox() + fake.processGroupId = '4343\n' + fake.deferStart() + const handle = new E2BSubprocessHandle(runtime(fake), spec({ + argv: ['tool', 'argument with spaces'], + stdio: { stdin: 'pipe', stdout: 'pipe', stderr: { maxBytes: 8, spill: { maxBytes: 32 } } }, + env: { PATH: '/bin', DEEPSEEK_API_KEY: 'explicit-secret', DSH_MODE: 'test' }, + }), '/workspace/.dsh-e2b/processes/one') + expect(handle.pid).toBe(-1) + handle.stdin!.write('hello') + handle.stdin!.end() + fake.releaseStart() + await flush() + expect(handle.pid).toBe(4343) + expect(fake.handle.sent.map(value => String(value))).toEqual(['hello']) + expect(fake.handle.closes).toBe(1) + expect(fake.startOptions?.envs).toEqual({ PATH: '/bin', DEEPSEEK_API_KEY: 'explicit-secret', DSH_MODE: 'test' }) + const command = fake.commandsSeen.find(value => value.startsWith('exec setsid'))! + expect(command).toContain('exec setsid --wait -- bash -c') + expect(command).toContain('DEEPSEEK_API_KEY') + expect(command).toContain('DSH_MODE') + expect(command).not.toContain('explicit-secret') + expect(fake.writtenFiles[0]).toEqual([ + '/workspace/.dsh-e2b/processes/one/pid', + '/workspace/.dsh-e2b/processes/one/exit-code', + '/workspace/.dsh-e2b/processes/one/stderr.log', + ]) + + let piped = '' + handle.stdout!.on('data', (chunk) => { piped += String(chunk) }) + await fake.stdout('pipe-data') + await fake.stderr('err') + fake.finish() + await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) + expect(piped).toBe('pipe-data') + expect(handle.collected.stderr!.readFrom(0)).toMatchObject({ text: 'err', lossy: false }) + expect(fake.removed).toContain('/workspace/.dsh-e2b/processes/one/stderr.log') + await expect(handle.waitForExit()).resolves.toBe(true) + }) + + it('surfaces deferred piped-stdin write and close failures as stream errors', async () => { + const writeFake = new FakeSandbox() + writeFake.deferStart() + vi.spyOn(writeFake.handle, 'sendStdin').mockRejectedValueOnce('stdin rejected') + const writeHandle = new E2BSubprocessHandle(runtime(writeFake), spec({ + stdio: { stdin: 'pipe', stdout: { maxBytes: 4 }, stderr: { maxBytes: 4 } }, + }), '/runtime/stdin-write-error') + const writeError = once(writeHandle.stdin!, 'error') + writeHandle.stdin!.write('input') + writeFake.releaseStart() + await expect(writeError).resolves.toMatchObject([{ message: 'stdin rejected' }]) + writeFake.finish() + await writeHandle.done + + const closeFake = new FakeSandbox() + vi.spyOn(closeFake.handle, 'closeStdin').mockRejectedValueOnce(new Error('close rejected')) + const closeHandle = new E2BSubprocessHandle(runtime(closeFake), spec({ + stdio: { stdin: 'pipe', stdout: { maxBytes: 4 }, stderr: { maxBytes: 4 } }, + }), '/runtime/stdin-close-error') + await flush() + const closeError = once(closeHandle.stdin!, 'error') + closeHandle.stdin!.end() + await expect(closeError).resolves.toMatchObject([{ message: 'close rejected' }]) + closeFake.finish() + await closeHandle.done + }) + + it('collects bounded tails, retains valid spills, and maps natural nonzero exits', async () => { + const fake = new FakeSandbox() + const handle = new E2BSubprocessHandle(runtime(fake), spec({ + stdio: { + stdin: { data: 'batch' }, + stdout: { maxBytes: 4, spill: { maxBytes: 16 } }, + stderr: { maxBytes: 3 }, + }, + }), '/runtime/two') + await flush() + await fake.stdout('abcdef') + await fake.stderr('12345') + fake.finish(7) + await expect(handle.done).resolves.toEqual({ exitCode: 7, signal: null }) + expect(fake.handle.sent).toEqual(['batch']) + expect(fake.handle.closes).toBe(1) + expect(handle.collected.stdout!.readFrom(0)).toEqual({ + text: 'cdef', + nextOffset: 6, + lossy: true, + spillPath: '/runtime/two/stdout.log', + }) + expect(handle.collected.stderr!.readFrom(0)).toEqual({ text: '345', nextOffset: 5, lossy: true }) + expect(fake.removed).not.toContain('/runtime/two/stdout.log') + }) + + it('removes a spill once the complete stream exceeds its cap', async () => { + const fake = new FakeSandbox() + const handle = new E2BSubprocessHandle(runtime(fake), spec({ + stdio: { stdin: 'ignore', stdout: { maxBytes: 2, spill: { maxBytes: 3 } }, stderr: 'inherit' }, + }), '/runtime/oversize') + await flush() + await fake.stdout('abcd') + await fake.stderr('') + fake.finish() + await handle.done + expect(handle.collected.stdout!.readFrom(0)).toEqual({ text: 'cd', nextOffset: 4, lossy: true }) + expect(fake.removed).toContain('/runtime/oversize/stdout.log') + }) + + it('contains remote spill-removal failures and routes empty inherited output', async () => { + const fake = new FakeSandbox() + fake.nextRemoveError = new Error('already removed') + const handle = new E2BSubprocessHandle(runtime(fake), spec({ + stdio: { stdin: 'ignore', stdout: 'inherit', stderr: { maxBytes: 4, spill: { maxBytes: 8 } } }, + }), '/runtime/remove-error') + await flush() + await fake.stdout('') + await fake.stderr('') + fake.finish() + await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) + expect(fake.removed).toContain('/runtime/remove-error/stderr.log') + }) + + it('terminates a process group with TERM and reports the signal outcome', async () => { + const fake = new FakeSandbox() + const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/term') + await flush() + handle.terminate() + handle.terminate() + await expect(handle.done).resolves.toEqual({ exitCode: null, signal: 'SIGTERM' }) + await expect(handle.waitForExit()).resolves.toBe(true) + expect(fake.commandsSeen).toContain('kill -TERM -- -4242') + expect(fake.commandsSeen).not.toContain('kill -KILL -- -4242') + }) + + it('escalates a TERM-trapping process group to KILL and uses the SDK kill as fallback', async () => { + const fake = new FakeSandbox() + fake.trapsTerm = true + fake.handle.killError = new Error('already gone') + const handle = new E2BSubprocessHandle(runtime(fake), spec({ graceMs: 1 }), '/runtime/kill') + await flush() + handle.terminate() + await expect(handle.done).resolves.toEqual({ exitCode: null, signal: 'SIGKILL' }) + await expect(handle.waitForExit()).resolves.toBe(true) + expect(fake.commandsSeen).toContain('kill -KILL -- -4242') + expect(fake.handle.kills).toBe(1) + }) + + it('honors termination requested before asynchronous startup finishes', async () => { + const fake = new FakeSandbox() + fake.deferStart() + const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/deferred-kill') + handle.terminate() + fake.releaseStart() + await expect(handle.done).resolves.toEqual({ exitCode: null, signal: 'SIGTERM' }) + }) + + it('honors an already-aborted signal when constructing the asynchronous handle directly', async () => { + const fake = new FakeSandbox() + const handle = new E2BSubprocessHandle(runtime(fake), spec({ signal: AbortSignal.abort('stop') }), '/runtime/pre-aborted') + await expect(handle.done).resolves.toEqual({ exitCode: null, signal: 'SIGTERM' }) + }) + + it('reacts to a signal that aborts after the remote command has started', async () => { + const fake = new FakeSandbox() + const controller = new AbortController() + const handle = new E2BSubprocessHandle(runtime(fake), spec({ signal: controller.signal }), '/runtime/live-abort') + await flush() + controller.abort('stop') + await expect(handle.done).resolves.toEqual({ exitCode: null, signal: 'SIGTERM' }) + }) + + it('bounds waitForExit while startup or a live group is pending', async () => { + const fake = new FakeSandbox() + fake.deferStart() + const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/wait') + const beforeStart = new AbortController() + const pending = handle.waitForExit(beforeStart.signal) + beforeStart.abort() + await expect(pending).resolves.toBe(false) + await expect(handle.waitForExit(AbortSignal.abort())).resolves.toBe(false) + fake.releaseStart() + await flush() + const live = new AbortController() + const liveWait = handle.waitForExit(live.signal) + live.abort() + await expect(liveWait).resolves.toBe(false) + fake.finish() + await handle.done + }) + + it('bounds both sides of the liveness-poll abort race', async () => { + const fake = new FakeSandbox() + const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/poll-abort') + await flush() + + const beforeTick = new AbortController() + fake.afterProbe = () => { beforeTick.abort(); fake.afterProbe = undefined } + await expect(handle.waitForExit(beforeTick.signal)).resolves.toBe(false) + + const duringTick = new AbortController() + fake.afterProbe = () => { + fake.afterProbe = undefined + setTimeout(() => { duringTick.abort() }, 0) + } + await expect(handle.waitForExit(duringTick.signal)).resolves.toBe(false) + + const duringProbe = new AbortController() + fake.beforeProbe = () => { duringProbe.abort(); fake.beforeProbe = undefined } + await expect(handle.waitForExit(duringProbe.signal)).resolves.toBe(false) + fake.finish() + await handle.done + }) + + it('observes a live group across one successful bounded poll', async () => { + const fake = new FakeSandbox() + const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/poll-success') + await flush() + setTimeout(() => { fake.finish() }, 1) + await expect(handle.waitForExit(new AbortController().signal)).resolves.toBe(true) + await handle.done + }) + + it('treats startup failure as no live tree and contains readiness rejection', async () => { + const fake = new FakeSandbox() + fake.backgroundError = new Error('start failed') + const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/fail') + await expect(handle.done).rejects.toThrow('start failed') + expect(handle.pid).toBe(-1) + await expect(handle.waitForExit()).resolves.toBe(true) + handle.terminate() + }) + + it('bounds a readiness rejection with a still-live caller signal', async () => { + const fake = new FakeSandbox() + fake.deferStart() + fake.backgroundError = new Error('start failed') + const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/fail-with-signal') + const waiting = handle.waitForExit(new AbortController().signal) + fake.releaseStart() + await expect(handle.done).rejects.toThrow('start failed') + await expect(waiting).resolves.toBe(true) + }) + + it('propagates an unavailable sandbox unless the caller aborts the wait', async () => { + const fake = new FakeSandbox() + let calls = 0 + const unavailable = runtime(fake, async () => { + calls += 1 + if (calls === 1) return fake.sandbox + throw new Error('connection unavailable') + }) + const handle = new E2BSubprocessHandle(unavailable, spec(), '/runtime/unavailable') + await flush() + await expect(handle.waitForExit()).rejects.toThrow('connection unavailable') + fake.finish() + await handle.done + }) + + it('returns false when the caller aborts while reconnecting for liveness', async () => { + const fake = new FakeSandbox() + const reconnect = Promise.withResolvers() + let calls = 0 + const unavailable = runtime(fake, async () => { + calls += 1 + return calls === 1 ? fake.sandbox : await reconnect.promise + }) + const handle = new E2BSubprocessHandle(unavailable, spec(), '/runtime/reconnect-abort') + await flush() + const controller = new AbortController() + const waiting = handle.waitForExit(controller.signal) + await flush() + controller.abort() + reconnect.reject(new Error('connection unavailable')) + await expect(waiting).resolves.toBe(false) + fake.finish() + await handle.done + }) + + it('returns false when a liveness request itself is aborted and surfaces other probe failures', async () => { + const fake = new FakeSandbox() + const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/probe') + await flush() + const controller = new AbortController() + controller.abort() + await expect(handle.waitForExit(controller.signal)).resolves.toBe(false) + fake.probeError = new Error('probe failed') + await expect(handle.waitForExit()).rejects.toThrow('probe failed') + fake.finish() + await handle.done + }) + + it('makes batch stdin close failures best-effort', async () => { + const fake = new FakeSandbox() + vi.spyOn(fake.handle, 'sendStdin').mockRejectedValueOnce(new Error('closed')) + const handle = new E2BSubprocessHandle(runtime(fake), spec({ + stdio: { stdin: { data: 'ignored' }, stdout: { maxBytes: 4 }, stderr: { maxBytes: 4 } }, + }), '/runtime/stdin-closed') + await flush() + fake.finish() + await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) + }) + + it('rejects malformed SDK process ids and non-command settlement failures', async () => { + const invalidPid = new FakeSandbox() + invalidPid.handle.pid = 0 + const invalid = new E2BSubprocessHandle(runtime(invalidPid), spec(), '/runtime/invalid-pid') + await expect(invalid.done).rejects.toThrow(/invalid command pid 0/) + await expect(invalid.waitForExit()).resolves.toBe(true) + + const crashedFake = new FakeSandbox() + const crashed = new E2BSubprocessHandle(runtime(crashedFake), spec(), '/runtime/crashed') + await flush() + crashedFake.alive = false + crashedFake.handle.crash(new Error('command transport failed')) + await expect(crashed.done).rejects.toThrow('command transport failed') + }) + + it('rejects invalid or absent process-group publication', async () => { + const invalidGroup = new FakeSandbox() + invalidGroup.processGroupId = 'not-a-pid\n' + const invalid = new E2BSubprocessHandle(runtime(invalidGroup), spec(), '/runtime/invalid-group') + await expect(invalid.done).rejects.toThrow(/invalid process-group id/) + + const absentGroup = new FakeSandbox() + absentGroup.processGroupId = '' + const absent = new E2BSubprocessHandle(runtime(absentGroup), spec(), '/runtime/absent-group') + await flush() + absentGroup.finish() + await expect(absent.done).rejects.toThrow(/exited before publishing/) + }) + + it('waits for delayed process-group publication', async () => { + const fake = new FakeSandbox() + fake.processGroupReads.push('', '4242\n') + const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/delayed-group') + await vi.waitFor(() => { expect(handle.pid).toBe(4242) }) + fake.finish() + await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) + }) + + it('handles output backpressure and contains a stderr sink failure', async () => { + const fake = new FakeSandbox() + const handle = new E2BSubprocessHandle(runtime(fake), spec({ + stdio: { stdin: 'ignore', stdout: 'pipe', stderr: 'pipe' }, + }), '/runtime/backpressure') + await flush() + + handle.stdout!.on('error', () => {}) + const stdoutWrite = vi.spyOn(handle.stdout!, 'write').mockReturnValueOnce(false) + const stdoutPending = fake.stdout('blocked') + queueMicrotask(() => { handle.stdout!.emit('drain') }) + await stdoutPending + stdoutWrite.mockRestore() + + handle.stderr!.on('error', () => {}) + const stderrWrite = vi.spyOn(handle.stderr!, 'write').mockReturnValueOnce(false) + const stderrPending = fake.stderr('broken') + queueMicrotask(() => { handle.stderr!.emit('error', new Error('sink failed')) }) + await stderrPending + stderrWrite.mockRestore() + + fake.finish() + await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) + }) + + it('contains a pipe callback failure instead of rejecting command settlement', async () => { + const fake = new FakeSandbox() + const handle = new E2BSubprocessHandle(runtime(fake), spec({ + stdio: { stdin: 'ignore', stdout: 'pipe', stderr: { maxBytes: 4 } }, + }), '/runtime/pipe-error') + await flush() + const emitted = once(handle.stdout!, 'error') + handle.stdout!.destroy(new Error('consumer failed')) + await emitted + await fake.stdout('late output') + fake.finish() + await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) + }) + + it('contains an already-gone group signal and observes non-command signal failures', async () => { + const gone = new FakeSandbox() + gone.trapsTerm = true + gone.signalError = commandError(1) + const goneHandle = new E2BSubprocessHandle(runtime(gone), spec({ graceMs: 1 }), '/runtime/gone-signal') + await flush() + goneHandle.terminate() + await expect(goneHandle.done).resolves.toEqual({ exitCode: null, signal: 'SIGKILL' }) + + const failed = new FakeSandbox() + failed.signalError = new Error('signal transport failed') + const failedHandle = new E2BSubprocessHandle(runtime(failed), spec(), '/runtime/failed-signal') + await flush() + failedHandle.terminate() + await flush() + failed.finish() + await expect(failedHandle.done).resolves.toEqual({ exitCode: 0, signal: null }) + }) +}) + +describe('E2BSubprocessService', () => { + async function service( + fake = new FakeSandbox(), + providedRuntime: E2BSandboxService = runtime(fake), + ): Promise<{ ctx: Context; fiber: Awaited> }> { + const ctx = new Context() + ctx.provide('e2b', providedRuntime) + const fiber = await ctx.plugin(E2BSubprocessService) + return { ctx, fiber } + } + + it('registers handles and disposal terminates and joins live remote groups regardless of sandbox policy', async () => { + const fake = new FakeSandbox() + fake.trapsTerm = true + const { ctx, fiber } = await service(fake) + const handle = ctx.subprocess.spawn(spec({ graceMs: 1 })) + await flush() + await fiber.dispose() + await expect(handle.done).resolves.toEqual({ exitCode: null, signal: 'SIGKILL' }) + expect(fake.alive).toBe(false) + }) + + it('releases naturally settled handles before later service disposal', async () => { + const fake = new FakeSandbox() + const { ctx, fiber } = await service(fake) + const handle = ctx.subprocess.spawn(spec()) + await flush() + fake.finish() + await handle.done + await flush() + const signalsBefore = fake.commandsSeen.filter(command => command.startsWith('kill -')).length + await fiber.dispose() + expect(fake.commandsSeen.filter(command => command.startsWith('kill -')).length).toBe(signalsBefore) + }) + + it('contains a release liveness failure and retries quiescence during disposal', async () => { + const fake = new FakeSandbox() + let calls = 0 + const reconnecting = runtime(fake, async () => { + calls += 1 + if (calls === 2) throw new Error('transient liveness failure') + return fake.sandbox + }) + const { ctx, fiber } = await service(fake, reconnecting) + const handle = ctx.subprocess.spawn(spec()) + await flush() + fake.finish() + await handle.done + await flush() + await fiber.dispose() + expect(calls).toBeGreaterThanOrEqual(3) + }) + + it('contains spawn rejection while disposal is joining the pending handle', async () => { + const fake = new FakeSandbox() + fake.deferStart() + fake.backgroundError = new Error('start failed during disposal') + const { ctx, fiber } = await service(fake) + const handle = ctx.subprocess.spawn(spec()) + const disposing = fiber.dispose() + fake.releaseStart() + await expect(disposing).resolves.toBeUndefined() + await expect(handle.done).rejects.toThrow('start failed during disposal') + }) + + it('validates synchronous spawn preconditions', async () => { + const { ctx } = await service() + expect(() => ctx.subprocess.spawn(spec({ argv: [] }))).toThrow(/non-empty program/) + expect(() => ctx.subprocess.spawn(spec({ graceMs: 0 }))).toThrow(/positive finite/) + expect(() => ctx.subprocess.spawn(spec({ signal: AbortSignal.abort('stop') }))).toThrow(/aborted before spawn/) + expect(() => ctx.subprocess.spawn(spec({ signal: { aborted: true, reason: undefined } as AbortSignal }))).toThrow(/aborted$/) + }) + + it('registers the package-owned empty invariant installer', async () => { + const ctx = new Context() + await ctx.plugin(InvariantService, { enabled: true }) + const fiber = await ctx.plugin(E2BSubprocessInvariant).await() + await fiber.dispose() + }) +}) diff --git a/packages/subprocess/subprocess-e2b/tsconfig.json b/packages/subprocess/subprocess-e2b/tsconfig.json new file mode 100644 index 0000000000..f3bb8c2a26 --- /dev/null +++ b/packages/subprocess/subprocess-e2b/tsconfig.json @@ -0,0 +1,25 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": ["src"], + "references": [ + { + "path": "../../../vendor/cosmokit" + }, + { + "path": "../../../vendor/cordis" + }, + { + "path": "../../e2b/e2b" + }, + { + "path": "../subprocess" + }, + { + "path": "../../support/invariants" + } + ] +} diff --git a/scripts/gen-cordis-catalog.ts b/scripts/gen-cordis-catalog.ts index 256ae90caa..b17cc5b4f8 100644 --- a/scripts/gen-cordis-catalog.ts +++ b/scripts/gen-cordis-catalog.ts @@ -301,6 +301,7 @@ export const TYPE_LINK_EXEMPTIONS: Readonly> = { PermissionSelect: 'permissions projection payload is owned by packages/ui/permission/src/types.ts', PromptAssembly: 'assembly result is owned by packages/core/system-prompt/README.md', ResumeAgentOptions: 'agent resume contract is owned by packages/core/agent/README.md', + Sandbox: 'external E2B SDK handle is owned by packages/e2b/e2b/README.md', SessionForkSource: 'service-local fork input is owned by packages/core/session/src/index.ts', SubagentRunEndInfo: 'event payload contract is owned by packages/subagent/subagent/src/types.ts', SubagentRunInfo: 'event payload contract is owned by packages/subagent/subagent/src/types.ts', diff --git a/scripts/verify-package-readme-model-experience.ts b/scripts/verify-package-readme-model-experience.ts index 3ca0317cf4..90aa22e322 100644 --- a/scripts/verify-package-readme-model-experience.ts +++ b/scripts/verify-package-readme-model-experience.ts @@ -50,6 +50,7 @@ const SENTENCE_MODEL_EXPERIENCE: Readonly> = { 'packages/code-runtime/code-runtime-worker': { kind: 'indirect', reason: 'The worker backend delegates model rendering to Code Mode in dsh-tools.' }, 'packages/typert/registry': { kind: 'none', reason: 'Runtime type registry; consumers (cordis_inspect, wire faces, gates) own any model-visible projection of registry contents.' }, 'packages/typert/loader': { kind: 'none', reason: 'Loader integration only registers generated artifacts; consumers own any model-visible projection.' }, + 'packages/e2b/e2b': { kind: 'none', reason: 'The shared remote-runtime owner registers no model context; provider adapters and consumers own rendered effects.' }, 'packages/client/hmr': { kind: 'none', reason: 'Browser-side UI plugin layer; registers no model surface.' }, 'packages/client/modules': { kind: 'none', reason: 'Browser-side module-loading kernel machinery; registers no model surface.' }, 'packages/client/test-runtime': { kind: 'none', reason: 'Browser-side test infrastructure (jsdom bench); registers no model surface.' }, @@ -82,6 +83,7 @@ const SENTENCE_MODEL_EXPERIENCE: Readonly> = { 'packages/client/web': { kind: 'none', reason: 'Browser-side UI plugin layer; registers no model surface.' }, 'packages/examples/agent-spine-demo': { kind: 'indirect', reason: 'The bundle only mounts model-facing child plugins.' }, 'packages/fs/fs': { kind: 'indirect', reason: 'The service interface delegates model rendering to dsh-tool-fs.' }, + 'packages/fs/fs-e2b': { kind: 'indirect', reason: 'The provider backend delegates model rendering to dsh-tool-fs.' }, 'packages/fs/fs-local': { kind: 'indirect', reason: 'The provider backend delegates model rendering to dsh-tool-fs.' }, 'packages/hooks/hook-protocol': { kind: 'indirect', reason: 'Only the hook bridge plugins render decoded hook output to a model.' }, 'packages/host/apiproxy': { kind: 'none', reason: 'The wire contract and fetch carriers move already-composed messages and register no model surface.' }, @@ -98,6 +100,7 @@ const SENTENCE_MODEL_EXPERIENCE: Readonly> = { 'packages/lsp/lsp': { kind: 'indirect', reason: 'The provider registry delegates model rendering to dsh-tool-lsp.' }, 'packages/lsp/lsp-local': { kind: 'indirect', reason: 'The provider backend delegates model rendering to dsh-tool-lsp.' }, 'packages/subprocess/subprocess': { kind: 'indirect', reason: 'The seam delegates all model rendering to consumer seams such as the bash executor family.' }, + 'packages/subprocess/subprocess-e2b': { kind: 'indirect', reason: 'The remote spawn backend delegates model rendering to consumer seams such as the bash executor family.' }, 'packages/subprocess/subprocess-local': { kind: 'indirect', reason: 'The spawn backend delegates model rendering to consumer seams such as the bash executor family.' }, 'packages/sandbox/sandbox-local': { kind: 'indirect', reason: 'The provider backend delegates model rendering to dsh-bash-sandbox and dsh-tool-bash.' }, 'packages/sdk/create-sdk': { kind: 'indirect', reason: 'The initializer only writes project files; selected runtime plugins provide the generated project model surface.' }, diff --git a/tsconfig.base.json b/tsconfig.base.json index de9b63459c..8380182e83 100644 --- a/tsconfig.base.json +++ b/tsconfig.base.json @@ -188,6 +188,7 @@ "./packages/bash/*/src", "./packages/pty/*/src", "./packages/subprocess/*/src", + "./packages/e2b/*/src", "./packages/code-runtime/*/src", "./packages/fs/*/src", "./packages/lsp/*/src", diff --git a/tsconfig.host.json b/tsconfig.host.json index 1a8ef6508a..03b1bcd2dd 100644 --- a/tsconfig.host.json +++ b/tsconfig.host.json @@ -162,6 +162,8 @@ { "path": "./packages/examples/agent-spine-demo" }, { "path": "./packages/subprocess/subprocess" }, { "path": "./packages/subprocess/subprocess-local" }, + { "path": "./packages/e2b/e2b" }, + { "path": "./packages/subprocess/subprocess-e2b" }, { "path": "./packages/bash/bash" }, { "path": "./packages/pty/pty" }, { "path": "./packages/pty/pty-local" }, @@ -183,6 +185,7 @@ { "path": "./packages/bash/tool-bash" }, { "path": "./packages/fs/fs" }, { "path": "./packages/fs/fs-local" }, + { "path": "./packages/fs/fs-e2b" }, { "path": "./packages/fs/fs-policy" }, { "path": "./packages/fs/fs-sandbox" }, { "path": "./packages/fs/tool-fs" }, From 6667102890ce0cf50f6122ecb676a4deccd60499 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Tue, 28 Jul 2026 13:59:47 +0800 Subject: [PATCH 064/108] Add E2B PTY, LSP, and code runtime providers --- ...026-07-27-e2b-remote-runtime-poc.i18n.yaml | 4 +- .../2026-07-27-e2b-remote-runtime-poc.md | 11 +- .../2026-07-27-e2b-remote-runtime-poc.zh.md | 11 +- ...active-semantic-code-runtime-poc.i18n.yaml | 6 + ...b-interactive-semantic-code-runtime-poc.md | 57 ++ ...nteractive-semantic-code-runtime-poc.zh.md | 57 ++ docs/cordis-catalog/services.md | 4 +- examples/headless-agent/e2b.cordis.yml | 34 +- .../tests/fixtures/e2b/e2b/bin.ts | 143 +++++ .../tests/fixtures/e2b/e2b/cordis.yml | 40 ++ .../tests/fixtures/e2b/e2b/fixture-lsp.mjs | 85 +++ examples/package.json | 5 + packages/code-runtime/README.i18n.yaml | 4 +- packages/code-runtime/README.md | 9 +- packages/code-runtime/README.zh.md | 13 +- .../code-runtime-e2b/README.i18n.yaml | 6 + .../code-runtime/code-runtime-e2b/README.md | 44 ++ .../code-runtime-e2b/README.zh.md | 44 ++ .../code-runtime-e2b/package.json | 52 ++ .../code-runtime-e2b/src/index.ts | 441 +++++++++++++++ .../code-runtime-e2b/src/invariant.ts | 20 + .../code-runtime-e2b/src/runner-source.ts | 460 ++++++++++++++++ .../code-runtime-e2b/tests/runtime.spec.ts | 460 ++++++++++++++++ .../code-runtime-e2b/tsconfig.json | 20 + .../code-runtime-worker/README.i18n.yaml | 4 +- .../code-runtime-worker/README.md | 6 +- .../code-runtime-worker/README.zh.md | 26 +- .../code-runtime-worker/src/index.ts | 35 +- .../code-runtime/README.i18n.yaml | 4 +- packages/code-runtime/code-runtime/README.md | 2 +- .../code-runtime/code-runtime/README.zh.md | 20 +- packages/e2b/README.i18n.yaml | 4 +- packages/e2b/README.md | 7 +- packages/e2b/README.zh.md | 7 +- packages/e2b/e2b/README.i18n.yaml | 4 +- packages/e2b/e2b/README.md | 2 +- packages/e2b/e2b/README.zh.md | 2 +- packages/e2b/e2b/src/frame.ts | 75 +++ packages/e2b/e2b/src/index.ts | 24 +- packages/e2b/e2b/tests/composition.e2e.ts | 42 +- packages/e2b/e2b/tests/e2b.spec.ts | 46 ++ packages/lsp/README.i18n.yaml | 4 +- packages/lsp/README.md | 5 +- packages/lsp/README.zh.md | 5 +- packages/lsp/lsp-e2b/README.i18n.yaml | 6 + packages/lsp/lsp-e2b/README.md | 47 ++ packages/lsp/lsp-e2b/README.zh.md | 47 ++ packages/lsp/lsp-e2b/package.json | 52 ++ packages/lsp/lsp-e2b/src/index.ts | 358 ++++++++++++ packages/lsp/lsp-e2b/src/invariant.ts | 20 + packages/lsp/lsp-e2b/src/proxy-source.ts | 55 ++ packages/lsp/lsp-e2b/src/transport.ts | 183 +++++++ packages/lsp/lsp-e2b/tests/provider.spec.ts | 397 ++++++++++++++ packages/lsp/lsp-e2b/tests/transport.spec.ts | 181 +++++++ packages/lsp/lsp-e2b/tsconfig.json | 20 + packages/lsp/lsp-local/src/instance.ts | 22 +- .../lsp/lsp-local/tests/fixture-server.ts | 6 + packages/lsp/lsp-local/tests/instance.spec.ts | 5 + packages/lsp/lsp/README.i18n.yaml | 4 +- packages/lsp/lsp/README.md | 1 + packages/lsp/lsp/README.zh.md | 1 + packages/pty/README.i18n.yaml | 4 +- packages/pty/README.md | 5 +- packages/pty/README.zh.md | 5 +- packages/pty/pty-e2b/README.i18n.yaml | 6 + packages/pty/pty-e2b/README.md | 54 ++ packages/pty/pty-e2b/README.zh.md | 54 ++ packages/pty/pty-e2b/package.json | 46 ++ packages/pty/pty-e2b/src/config.ts | 64 +++ packages/pty/pty-e2b/src/index.ts | 93 ++++ packages/pty/pty-e2b/src/invariant.ts | 20 + packages/pty/pty-e2b/src/session.ts | 367 +++++++++++++ packages/pty/pty-e2b/tests/index.spec.ts | 195 +++++++ packages/pty/pty-e2b/tests/session.spec.ts | 398 ++++++++++++++ packages/pty/pty-e2b/tsconfig.json | 16 + packages/pty/pty-local/src/sanitize.ts | 188 ------- packages/pty/pty-local/src/session.ts | 511 ++++++------------ packages/pty/pty-local/tests/sanitize.spec.ts | 34 +- packages/pty/pty/src/index.ts | 9 + packages/pty/pty/src/terminal.ts | 273 ++++++++++ .../verify-package-readme-model-experience.ts | 2 + tsconfig.host.json | 3 + 82 files changed, 5462 insertions(+), 644 deletions(-) create mode 100644 .agents/notes/implemented/feature/2026-07-28-e2b-interactive-semantic-code-runtime-poc.i18n.yaml create mode 100644 .agents/notes/implemented/feature/2026-07-28-e2b-interactive-semantic-code-runtime-poc.md create mode 100644 .agents/notes/implemented/feature/2026-07-28-e2b-interactive-semantic-code-runtime-poc.zh.md create mode 100644 examples/headless-agent/tests/fixtures/e2b/e2b/fixture-lsp.mjs create mode 100644 packages/code-runtime/code-runtime-e2b/README.i18n.yaml create mode 100644 packages/code-runtime/code-runtime-e2b/README.md create mode 100644 packages/code-runtime/code-runtime-e2b/README.zh.md create mode 100644 packages/code-runtime/code-runtime-e2b/package.json create mode 100644 packages/code-runtime/code-runtime-e2b/src/index.ts create mode 100644 packages/code-runtime/code-runtime-e2b/src/invariant.ts create mode 100644 packages/code-runtime/code-runtime-e2b/src/runner-source.ts create mode 100644 packages/code-runtime/code-runtime-e2b/tests/runtime.spec.ts create mode 100644 packages/code-runtime/code-runtime-e2b/tsconfig.json create mode 100644 packages/e2b/e2b/src/frame.ts create mode 100644 packages/lsp/lsp-e2b/README.i18n.yaml create mode 100644 packages/lsp/lsp-e2b/README.md create mode 100644 packages/lsp/lsp-e2b/README.zh.md create mode 100644 packages/lsp/lsp-e2b/package.json create mode 100644 packages/lsp/lsp-e2b/src/index.ts create mode 100644 packages/lsp/lsp-e2b/src/invariant.ts create mode 100644 packages/lsp/lsp-e2b/src/proxy-source.ts create mode 100644 packages/lsp/lsp-e2b/src/transport.ts create mode 100644 packages/lsp/lsp-e2b/tests/provider.spec.ts create mode 100644 packages/lsp/lsp-e2b/tests/transport.spec.ts create mode 100644 packages/lsp/lsp-e2b/tsconfig.json create mode 100644 packages/pty/pty-e2b/README.i18n.yaml create mode 100644 packages/pty/pty-e2b/README.md create mode 100644 packages/pty/pty-e2b/README.zh.md create mode 100644 packages/pty/pty-e2b/package.json create mode 100644 packages/pty/pty-e2b/src/config.ts create mode 100644 packages/pty/pty-e2b/src/index.ts create mode 100644 packages/pty/pty-e2b/src/invariant.ts create mode 100644 packages/pty/pty-e2b/src/session.ts create mode 100644 packages/pty/pty-e2b/tests/index.spec.ts create mode 100644 packages/pty/pty-e2b/tests/session.spec.ts create mode 100644 packages/pty/pty-e2b/tsconfig.json delete mode 100644 packages/pty/pty-local/src/sanitize.ts create mode 100644 packages/pty/pty/src/terminal.ts diff --git a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.i18n.yaml b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.i18n.yaml index 13d081c28f..46f1849b07 100644 --- a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.i18n.yaml +++ b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md -2026-07-27-e2b-remote-runtime-poc.md: 307a08d6ca77c83bf836bc8d50b071fbf33ac49a -2026-07-27-e2b-remote-runtime-poc.zh.md: bb4e657d999760f000949d509228bbb65f22773a +2026-07-27-e2b-remote-runtime-poc.md: 730e6f0e1958ccab43a5a6a11931f9407b38ef16 +2026-07-27-e2b-remote-runtime-poc.zh.md: 5f1ae07b985787a7dc1bd61f2a5b7ff0512d85c1 diff --git a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md index 307a08d6ca..730e6f0e19 100644 --- a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md +++ b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md @@ -16,18 +16,19 @@ The E2B integration is a provider-composition POC with one shared lifecycle owne - `@deepseek-ai/dsh-fs-e2b` implements `ctx.fs` over that sandbox's Filesystem API. - `@deepseek-ai/dsh-subprocess-e2b` implements `ctx.subprocess` over E2B Commands and remote Linux process groups. - The existing `@deepseek-ai/dsh-bash-local` remains the Bash implementation because it delegates all process mechanics to `ctx.subprocess`. +- PTY, LSP, and Code Runtime remain separate capability providers over the same owner, with their runtime split recorded in the [extension decision](2026-07-28-e2b-interactive-semantic-code-runtime-poc.md). The owner is the sole source of sandbox identity. Providers inject it and never create private sandboxes. The composition therefore gives filesystem tools and Bash one remote cwd, process namespace, and spill/state directory while preserving the existing capability interfaces and model-facing tools. ## POC boundary -Only filesystem state, command processes during the provider lifetime, and adapter-owned remote files move into E2B. The host retains Cordis and plugin objects, the agent loop, agent/session state, session logs and persistence, model requests, skills, subagent orchestration, and E2B SDK buffers. The overlay does not upload or mount the host workspace; identical cwd strings name independent host and remote directories. Managed process groups still terminate and join when the subprocess service disposes, including before a retained-sandbox pause or leave disposition. +Filesystem state, managed commands, interactive shells, language servers, code workers, and adapter-owned files move into E2B. The host retains Cordis and plugin objects, the agent loop, agent/session state, session logs and persistence, model requests, skills, subagent orchestration, capability protocol state, and E2B SDK buffers. The overlay does not upload or mount the host workspace; identical cwd strings name independent host and remote directories. Managed process groups still terminate and join when their provider disposes, including before a retained-sandbox pause or leave disposition. -The POC has no PTY adapter, LSP-specific integration, session-persistence backend, code-runtime backend, template builder, volume, snapshot, network-policy layer, sandbox catalog, or workspace synchronization. Retained sandbox reconnect proves lifecycle continuity only; it does not reconstruct host process handles, output cursors, or locks. +The POC has no session-persistence backend, template builder, volume, snapshot, network-policy layer, sandbox catalog, or workspace synchronization. Retained sandbox reconnect proves lifecycle continuity only; it does not reconstruct host PTY/LSP/code-runtime state, process handles, output cursors, pending calls, or locks. ## Verification -Package tests pin lifecycle cleanup, filesystem seam semantics, subprocess group/stdio/abort behavior, and the package-owned invariant registrations. A credential-gated real Loader composition creates one sandbox, proves FS-write→Bash-read and Bash-write→FS-read in the same remote cwd, proves neither file appears in the host cwd, disposes the composition, and confirms the sandbox id is gone. +Package tests pin lifecycle cleanup, filesystem semantics, subprocess groups, byte framing, PTY readiness/signals, LSP transport and containment, Code Runtime binding/output behavior, and package-owned invariant registrations. A credential-gated real Loader composition creates one sandbox, exercises filesystem, Bash/subprocess, PTY, LSP, and Code Runtime through source and built package paths, proves host-workspace isolation, disposes the composition, and confirms the sandbox id is gone. ## Alternatives considered @@ -35,7 +36,7 @@ Package tests pin lifecycle cleanup, filesystem seam semantics, subprocess group **Run the entire harness process inside E2B** — rejected because it changes deployment, credential flow, model transport, session durability, plugin loading, and supervision at once. Those questions are independent of proving the provider seams. -**Add E2B-specific Bash, PTY, LSP, persistence, and synchronization packages together** — rejected because Bash already has the required subprocess seam and the other capabilities need separate consumer evidence and lifecycle designs. Their absence is an explicit fidelity boundary, not an incomplete hidden plan. +**Put every E2B capability in the shared owner package** — rejected because lifecycle identity is the owner's only concern. Filesystem, subprocess, PTY, LSP, and Code Runtime retain separate provider contracts, configuration, tests, and consumers; Bash continues to reuse its subprocess seam. **Implement filesystem operations through shell commands only** — rejected because that bypasses `ctx.fs` identity, structured errors, version guards, streaming reads, and atomic mutation semantics already consumed by the file tools. @@ -43,4 +44,4 @@ Package tests pin lifecycle cleanup, filesystem seam semantics, subprocess group The small composition demonstrates that existing capability seams can move an agent's mutable coding world off-host without changing the loop or model-facing tool packages. `sandboxId` plus pause/leave permits manual state retention for experiments, while kill remains the demo's cleanup policy. -The provider is not interchangeable with the local subprocess backend for every consumer: remote startup cannot synchronously expose a PID, E2B retains complete command output in SDK memory, callback output is not byte-faithful, signal attribution is inferred, and reconnect cannot restore handles. Remote process/spill artifacts accumulate in a retained sandbox. These gaps remain documented POC constraints rather than compatibility shims or new cross-cutting abstractions. +The provider is not interchangeable with local backends for every consumer: remote startup cannot synchronously expose a PID, E2B retains complete command output in SDK memory, ordinary command callbacks are not byte-faithful, signal attribution is partly inferred, and reconnect cannot restore handles or protocol state. PTY uses E2B's byte API; LSP and Code Runtime add validated ASCII framing where protocol bytes matter. Remote process/spill artifacts accumulate in a retained sandbox. These gaps remain documented POC constraints rather than compatibility shims or new cross-cutting abstractions. diff --git a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.zh.md b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.zh.md index bb4e657d99..5f1ae07b98 100644 --- a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.zh.md +++ b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.zh.md @@ -16,18 +16,19 @@ E2B 集成是一个提供方组合 POC,由一个共享生命周期所有者和 - `@deepseek-ai/dsh-fs-e2b` 在该沙箱的 Filesystem API 之上实现 `ctx.fs`。 - `@deepseek-ai/dsh-subprocess-e2b` 在 E2B Commands 和远程 Linux 进程组之上实现 `ctx.subprocess`。 - 现有的 `@deepseek-ai/dsh-bash-local` 继续作为 Bash 实现,因为它把所有进程机制委托给 `ctx.subprocess`。 +- PTY、LSP 与 Code Runtime 仍是基于同一所有者的独立功能提供方;其运行时划分见[扩展决策](2026-07-28-e2b-interactive-semantic-code-runtime-poc.md)。 该所有者是沙箱身份的唯一真源。提供方会注入该所有者,绝不创建私有沙箱。因此,该组合让文件系统工具与 Bash 共享一个远程 cwd、进程命名空间和 spill/状态目录,同时保留现有功能接口与面向模型的工具。 ## POC 边界 -只有文件系统状态、提供方存续期内的命令进程,以及适配器拥有的远程文件会迁入 E2B。宿主仍保留 Cordis 和插件对象、agent loop(智能体循环)、agent/会话状态、会话日志及其持久化、模型请求、skill(技能)、subagent 编排和 E2B SDK 缓冲。该 overlay 不会上传或挂载宿主工作区;拼写相同的 cwd 字符串分别指向彼此独立的宿主与远程目录。受管进程组仍会在进程管理服务 dispose(资源释放)时终止并等待退出,包括保留沙箱采用 `pause` 或 `leave` 处置方式之前。 +文件系统状态、受管命令、交互式 shell、语言服务器、代码 worker 和适配器拥有的文件会迁入 E2B。宿主仍保留 Cordis 和插件对象、agent loop(智能体循环)、agent/会话状态、会话日志及其持久化、模型请求、skill(技能)、subagent 编排、功能协议状态和 E2B SDK 缓冲。该 overlay 不会上传或挂载宿主工作区;拼写相同的 cwd 字符串分别指向彼此独立的宿主与远程目录。受管进程组仍会在所属提供方 dispose(资源释放)时终止并等待退出,包括保留沙箱采用 `pause` 或 `leave` 处置方式之前。 -本 POC 没有 PTY 适配器、LSP 专用集成、会话持久化后端、代码运行时后端、模板构建器、卷、快照、网络策略层、沙箱目录或工作区同步。保留沙箱后重新连接只能证明生命周期连续性;它不会重建宿主进程句柄、输出游标或锁。 +本 POC 没有会话持久化后端、模板构建器、卷、快照、网络策略层、沙箱目录或工作区同步。保留沙箱后重新连接只能证明生命周期连续性;它不会重建宿主 PTY/LSP/Code Runtime 状态、进程句柄、输出游标、待处理调用或锁。 ## 验证 -包测试固定生命周期清理、文件系统 seam 语义、进程管理的进程组/stdio/中止行为,以及包自有不变式注册。凭据门控的真实 Loader 组合会创建一个沙箱,证明同一远程 cwd 中 FS-write→Bash-read 和 Bash-write→FS-read 双向可见,证明两个文件均未出现在宿主 cwd 中,释放组合,并确认该沙箱 id 已不存在。 +包测试固定生命周期清理、文件系统语义、进程管理的进程组、字节分帧、PTY 就绪状态/信号、LSP 传输与源码路径约束、Code Runtime 绑定/输出行为,以及包自有不变式注册。凭据门控的真实 Loader 组合会创建一个沙箱,通过源代码与已构建包路径演练文件系统、Bash/进程管理、PTY、LSP 和 Code Runtime,证明宿主工作区隔离,释放组合,并确认该沙箱 id 已不存在。 ## 曾考虑的替代方案 @@ -35,7 +36,7 @@ E2B 集成是一个提供方组合 POC,由一个共享生命周期所有者和 **在 E2B 内运行完整 harness 进程。** 不予采纳,因为这会同时改变部署、凭据流、模型传输、会话持久性、插件加载和监管方式。要证明提供方 seam,并不需要同时回答这些彼此独立的问题。 -**同时添加 E2B 专用 Bash、PTY、LSP、持久化和同步包。** 不予采纳,因为 Bash 已经具备所需的进程管理 seam,其他功能则需要各自的消费方证据和生命周期设计。缺少它们是显式保真边界,而不是尚未公开的不完整计划。 +**把所有 E2B 功能放入共享所有者包。** 不予采纳,因为生命周期身份是该所有者唯一负责的事项。文件系统、进程管理、PTY、LSP 与 Code Runtime 各自保留独立的提供方契约、配置、测试和消费方;Bash 继续复用其进程管理 seam。 **仅通过 shell 命令实现文件系统操作。** 不予采纳,因为这会绕过文件工具已经使用的 `ctx.fs` 身份、结构化错误、版本防护、流式读取和原子变更语义。 @@ -43,4 +44,4 @@ E2B 集成是一个提供方组合 POC,由一个共享生命周期所有者和 这个小型组合证明,现有功能 seam 可以把 agent 的可变 coding 环境移出宿主,而无需改变循环或面向模型的工具包。`sandboxId` 与 `pause`/`leave` 允许实验手动保留状态,演示仍以 `kill` 作为清理策略。 -该提供方并不能对所有消费方与本地进程管理后端互换:远程启动无法同步公开 PID,E2B 会在 SDK 内存中保留完整命令输出,回调输出并非字节保真,信号归因依靠推断,重新连接也无法恢复句柄。保留沙箱后会累积远程进程/spill 产物。这些缺口作为 POC 约束明确记录,而不会引入兼容垫片或新的跨领域抽象。 +该提供方并不能对所有消费方与本地后端互换:远程启动无法同步公开 PID,E2B 会在 SDK 内存中保留完整命令输出,普通命令回调并非字节保真,信号归因部分依靠推断,重新连接也无法恢复句柄或协议状态。PTY 使用 E2B 的字节 API;LSP 与 Code Runtime 则在必须保真处理协议字节之处增加经过验证的 ASCII 分帧。保留沙箱后会累积远程进程/spill 产物。这些缺口作为 POC 约束明确记录,而不会引入兼容垫片或新的跨领域抽象。 diff --git a/.agents/notes/implemented/feature/2026-07-28-e2b-interactive-semantic-code-runtime-poc.i18n.yaml b/.agents/notes/implemented/feature/2026-07-28-e2b-interactive-semantic-code-runtime-poc.i18n.yaml new file mode 100644 index 0000000000..e34f939a7f --- /dev/null +++ b/.agents/notes/implemented/feature/2026-07-28-e2b-interactive-semantic-code-runtime-poc.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-28-e2b-interactive-semantic-code-runtime-poc.md +2026-07-28-e2b-interactive-semantic-code-runtime-poc.md: befaa47dd56b7f425e72978ae040bd0815cc9f92 +2026-07-28-e2b-interactive-semantic-code-runtime-poc.zh.md: 677c3a09f6846320d3e000e9607becb3dee865cf diff --git a/.agents/notes/implemented/feature/2026-07-28-e2b-interactive-semantic-code-runtime-poc.md b/.agents/notes/implemented/feature/2026-07-28-e2b-interactive-semantic-code-runtime-poc.md new file mode 100644 index 0000000000..befaa47dd5 --- /dev/null +++ b/.agents/notes/implemented/feature/2026-07-28-e2b-interactive-semantic-code-runtime-poc.md @@ -0,0 +1,57 @@ +# Agent Note: E2B interactive, semantic, and code-runtime POC + +Status: implemented + +English | [中文](2026-07-28-e2b-interactive-semantic-code-runtime-poc.zh.md) + +## Problem + +The [shared E2B runtime](2026-07-27-e2b-remote-runtime-poc.md) proves that filesystem operations and one-shot commands can inhabit one remote coding world, but an assembled coding agent also uses persistent terminals, language servers, and model-written Code Mode programs. Falling back to host implementations for those capabilities splits observable state: a Bash edit exists in E2B while a host PTY, LSP process, or code worker addresses a different filesystem and process namespace. + +Moving the complete harness process into E2B would unify that state but also changes plugin loading, credentials, model transport, session durability, supervision, and deployment. The POC needs to test the existing capability boundaries without taking on those independent concerns. + +## Decision + +Three opt-in providers extend the existing shared sandbox: + +- `@deepseek-ai/dsh-pty-e2b` registers an E2B byte-PTY backend on `ctx.pty` and keeps exact-Agent ownership in the existing registry. +- `@deepseek-ai/dsh-lsp-e2b` registers configured remote language servers on `ctx.lsp`, reads source through E2B Filesystem APIs, and runs servers through `dsh-subprocess-e2b`. +- `@deepseek-ai/dsh-code-runtime-e2b` registers `ctx.codeRuntime`, runs each model program in a fresh remote worker, and dispatches binding functions in the host process. + +All three inject `ctx.e2b`; none creates another sandbox. The opt-in overlay composes them with `dsh-fs-e2b`, `dsh-subprocess-e2b`, and the existing `dsh-bash-local`, so files, foreground commands, interactive shell processes, language servers, and code workers observe one remote cwd. + +The providers reuse the PTY, LSP, Code Runtime, and subprocess seams without changing their model-facing consumers or the agent loop. Backend-neutral PTY text handling moves into `dsh-pty`; the LSP protocol engine accepts `processId: null` for a server in another process namespace; Code Runtime exports its output-ledger and lossless-JSON helpers for backend parity. + +## Runtime boundary + +E2B owns the mutable filesystem, command and Bash processes, PTY shell and foreground process groups, language-server processes and source reads, the Code Runtime runner and worker, and adapter-private files under `.dsh-e2b`. + +The host owns Cordis and plugin objects, agent/session/goal state, session logs and persistence, LLM calls, prompts and tools, authority decisions, PTY buffers and readiness state, LSP JSON-RPC ids/queues/protocol state, Code Runtime type stripping/output accounting/binding dispatch, and E2B SDK/network orchestration. The host workspace is not mounted or synchronized merely because its absolute cwd string is reused remotely. + +Byte-sensitive protocols use the narrowest adapter needed for E2B's callback shapes. PTY consumes the SDK's byte callback directly. LSP and Code Runtime install dependency-free remote helpers that encode raw payloads as validated newline-delimited base64 JSON, keeping E2B's decoded command callbacks on an ASCII transport. + +Retaining a sandbox preserves remote files and any unmanaged remote state only. Reconnect does not reconstruct host PTY sessions, buffers, process handles, LSP connections or requests, code workers, binding calls, timers, output cursors, or locks. Managed groups are terminated and awaited when their provider disposes before the shared owner pauses, leaves, or kills the sandbox. + +## Verification + +Focused unit suites pin configuration, publication rollback, byte framing, multibyte boundaries, readiness, signals, timeout/abort ordering, output limits, hostile Code Runtime traffic, and disposal to quiescence. Adjacent local-backend suites pin the shared PTY utilities and the LSP cross-namespace `processId` behavior. + +A credential-gated Loader composition creates one real E2B sandbox and exercises FS-to-Bash and Bash-to-FS visibility, multibyte PTY output and `SIGINT`, multibyte LSP hover and definition results, Code Runtime host bindings and typed rejection under mutation of adapter-captured intrinsics, wall timeout, abort, runner cleanup, host-workspace isolation, and final sandbox deletion. The same scenario runs through source imports and built package exports. + +## Alternatives considered + +**Run the complete harness inside E2B** — rejected because it couples this provider experiment to credentials, LLM transport, plugin deployment, session persistence, supervision, and remote package installation. None is necessary to prove the capability seams. + +**Use the host PTY, LSP, and worker backends unchanged** — rejected because they use host process and filesystem APIs; sharing an absolute cwd string does not share state across machines. + +**Expose E2B Commands as one generic transport and bypass capability providers** — rejected because PTY needs byte callbacks and foreground signaling, LSP needs byte-faithful stdio plus remote source containment, and Code Runtime needs bidirectional host binding calls and hostile-peer validation. Bypassing their registries would also fork model-facing behavior. + +**Add a generic distributed-runtime abstraction first** — rejected because the three existing capability seams already carry the required contracts. A new cross-cutting interface would speculate about persistence, synchronization, and reconnect semantics beyond the POC. + +**Restore live capability handles after `sandboxId` reconnect** — rejected because remote identity alone cannot reconstruct host callbacks, pending promises, authority, protocol state, or output cursors. Claiming continuity would make stale remote processes appear managed when they are not. + +## Consequences + +The assembled POC keeps the coding world remote without moving the agent runtime or changing model-visible tool contracts. It demonstrates that PTY, LSP, and Code Runtime can share E2B state through existing plugins, while making the remaining host state explicit. + +This is not a deployment platform. Language-server installation, templates, volumes, snapshots, network policy, workspace synchronization, durable remote handles, and whole-harness execution remain outside scope. E2B SDK buffering and host protocol state remain memory costs. Code programs share a JavaScript realm with Node worker internals, and a process that deliberately escapes a captured remote process group is not made reconnectable or owned by this composition. diff --git a/.agents/notes/implemented/feature/2026-07-28-e2b-interactive-semantic-code-runtime-poc.zh.md b/.agents/notes/implemented/feature/2026-07-28-e2b-interactive-semantic-code-runtime-poc.zh.md new file mode 100644 index 0000000000..677c3a09f6 --- /dev/null +++ b/.agents/notes/implemented/feature/2026-07-28-e2b-interactive-semantic-code-runtime-poc.zh.md @@ -0,0 +1,57 @@ +# Agent Note: E2B 交互式、语义与代码运行时 POC + +Status: implemented + +[English](2026-07-28-e2b-interactive-semantic-code-runtime-poc.md) | 中文 + +## 问题 + +[共享 E2B 运行时](2026-07-27-e2b-remote-runtime-poc.md)证明,文件系统操作与一次性命令可以共处一个远程 coding 环境,但组装完成的 coding agent(智能体)还会使用持久终端、语言服务器,以及模型编写的 Code Mode 程序。若这些功能回退到宿主实现,可观测状态就会分裂:Bash 修改存在于 E2B 中,而宿主 PTY、LSP 进程或代码 worker 面向的却是另一个文件系统与进程命名空间。 + +把完整 harness 进程迁入 E2B 可以统一这些状态,但也会改变插件加载、凭据、模型传输、会话持久性、监管和部署。这个 POC 只需测试现有功能边界,不应把这些彼此独立的问题纳入范围。 + +## 决策 + +三个可选提供方扩展现有共享沙箱: + +- `@deepseek-ai/dsh-pty-e2b` 在 `ctx.pty` 上注册 E2B 字节 PTY 后端,并把精确的 Agent 所有权保留在现有注册表中。 +- `@deepseek-ai/dsh-lsp-e2b` 在 `ctx.lsp` 上注册已配置的远程语言服务器,通过 E2B Filesystem API 读取源代码,并通过 `dsh-subprocess-e2b` 运行服务器。 +- `@deepseek-ai/dsh-code-runtime-e2b` 注册 `ctx.codeRuntime`,在全新的远程 worker 中运行每个模型程序,并在宿主进程中分发绑定函数。 + +三个提供方均注入 `ctx.e2b`,无一创建其他沙箱。可选叠加配置将它们与 `dsh-fs-e2b`、`dsh-subprocess-e2b` 和现有的 `dsh-bash-local` 组合,使文件、前台命令、交互式 shell 进程、语言服务器和代码 worker 观察到同一个远程 cwd。 + +这些提供方复用 PTY、LSP、Code Runtime 与进程管理 seam,不更改面向模型的消费方或 agent loop(智能体循环)。后端无关的 PTY 文本处理移入 `dsh-pty`;LSP 协议引擎允许位于另一个进程命名空间的服务器使用 `processId: null`;Code Runtime 导出输出账本与无损 JSON 辅助函数,以保持各后端一致。 + +## 运行时边界 + +E2B 拥有可变文件系统、命令和 Bash 进程、PTY shell 与前台进程组、语言服务器进程及源码读取、Code Runtime 运行器和 worker,以及 `.dsh-e2b` 下的适配器私有文件。 + +宿主拥有 Cordis 与插件对象、agent/会话/goal 状态、会话日志及持久化、LLM(大语言模型)调用、提示词与工具、权限决策、PTY 缓冲与就绪状态、LSP JSON-RPC id/队列/协议状态、Code Runtime 类型剥离/输出计量/绑定分发,以及 E2B SDK/网络编排。宿主工作区不会仅因远程复用了其绝对 cwd 字符串就被挂载或同步。 + +对字节敏感的协议只使用适配 E2B 回调形状所需的最窄适配器。PTY 直接消费 SDK 的字节回调。LSP 与 Code Runtime 会安装无依赖的远程辅助程序,把原始载荷编码为经过验证、以换行分隔的 base64 JSON,并通过 ASCII 传输承载 E2B 已解码的命令回调。 + +保留沙箱只会保存远程文件与任何未受管的远程状态。重新连接不会重建宿主 PTY 会话、缓冲、进程句柄、LSP 连接或请求、代码 worker、绑定调用、定时器、输出游标或锁。受管进程组会在所属提供方 dispose(资源释放)时终止并等待退出,之后共享所有者才会暂停、脱离或终止沙箱。 + +## 验证 + +聚焦单元测试固定配置、发布回滚、字节分帧、多字节边界、就绪状态、信号、超时/中止顺序、输出上限、恶意 Code Runtime 通信,以及等待完全停稳的资源释放。相邻本地后端测试固定共享 PTY 工具函数,以及 LSP 跨命名空间 `processId` 行为。 + +凭据门控的 Loader 组合会创建一个真实 E2B 沙箱,并演练 FS-to-Bash 与 Bash-to-FS 可见性、多字节 PTY 输出和 `SIGINT`、多字节 LSP 悬停与定义结果、Code Runtime 宿主绑定,以及适配器已捕获 intrinsic 被修改时的类型化 reject、墙钟超时、中止、运行器清理、宿主工作区隔离,以及最终删除沙箱。同一场景分别通过源代码导入与已构建包导出运行。 + +## 曾考虑的替代方案 + +**在 E2B 内运行完整 harness。** 不予采纳,因为这会把这项提供方实验与凭据、LLM 传输、插件部署、会话持久化、监管和远程包安装耦合。要证明功能 seam,无需引入任何一项。 + +**原样使用宿主 PTY、LSP 与 worker 后端。** 不予采纳,因为它们使用宿主的进程与文件系统 API;在不同机器上复用同一个绝对 cwd 字符串并不会共享状态。 + +**把 E2B Commands 公开为通用传输并绕过功能提供方。** 不予采纳,因为 PTY 需要字节回调和前台信号,LSP 需要字节保真的 stdio 与远程源码路径约束,Code Runtime 则需要双向宿主绑定调用与不可信对等方验证。绕过其注册表还会使面向模型的行为产生分叉。 + +**先添加通用分布式运行时抽象。** 不予采纳,因为现有三个功能 seam 已承载所需契约。新的跨领域接口会预先假定 POC 范围之外的持久化、同步与重连语义。 + +**在 `sandboxId` 重连后恢复活动功能句柄。** 不予采纳,因为只有远程身份,无法重建宿主回调、待处理 promise、权限、协议状态或输出游标。若声称保持连续性,就会让陈旧的远程进程看似仍受管理,实际并非如此。 + +## 后果 + +组装后的 POC 在不迁移 agent 运行时、不改变模型可见工具契约的前提下,把 coding 环境保留在远程。它证明 PTY、LSP 与 Code Runtime 可以通过现有插件共享 E2B 状态,同时明确列出仍留在宿主的状态。 + +这不是部署平台。语言服务器安装、模板、卷、快照、网络策略、工作区同步、持久远程句柄和完整 harness 执行仍不在范围内。E2B SDK 缓冲与宿主协议状态仍会占用内存。模型程序与 Node worker 内部机制共享一个 JavaScript realm;有意逃离已捕获远程进程组的进程,也不会因此变得可重新连接或由该组合管理。 diff --git a/docs/cordis-catalog/services.md b/docs/cordis-catalog/services.md index 7ce45c6817..33f357d2bf 100644 --- a/docs/cordis-catalog/services.md +++ b/docs/cordis-catalog/services.md @@ -591,7 +591,7 @@ Owns one lazily consumable E2B SDK handle and its final kill/pause/leave decisio async getSandbox(): Promise ``` -Source: [`packages/e2b/e2b/src/index.ts:97`](../../packages/e2b/e2b/src/index.ts) +Source: [`packages/e2b/e2b/src/index.ts:119`](../../packages/e2b/e2b/src/index.ts) ## `ctx.fs` — `FileSystem` (abstract seam) @@ -1180,7 +1180,7 @@ list(owner: Agent): PtySessionSnapshot[] Types: [Agent](../core-data-structures/core.md) · [PtyBackend](../core-data-structures/pty.md) · [PtyReadRequest](../core-data-structures/pty.md) · [PtyReadResult](../core-data-structures/pty.md) · [PtySendOperation](../core-data-structures/pty.md) · [PtySendRequest](../core-data-structures/pty.md) · [PtySessionId](../core-data-structures/pty.md) · [PtySessionSnapshot](../core-data-structures/pty.md) · [PtySignal](../core-data-structures/pty.md) · [PtySignalResult](../core-data-structures/pty.md) · [PtySpawnRequest](../core-data-structures/pty.md) · [PtySpawnResult](../core-data-structures/pty.md) -Source: [`packages/pty/pty/src/index.ts:105`](../../packages/pty/pty/src/index.ts) +Source: [`packages/pty/pty/src/index.ts:114`](../../packages/pty/pty/src/index.ts) ## `ctx.sandbox` — `SandboxProvider` (abstract seam) diff --git a/examples/headless-agent/e2b.cordis.yml b/examples/headless-agent/e2b.cordis.yml index 4f961100b4..877172c343 100644 --- a/examples/headless-agent/e2b.cordis.yml +++ b/examples/headless-agent/e2b.cordis.yml @@ -1,9 +1,10 @@ -# POC overlay: keep the headless agent and model-facing tools, but place its -# filesystem and managed Bash process world in one short-lived E2B sandbox. +# POC overlay: keep the advanced headless agent and model-facing tools, but +# place its filesystem, processes, terminals, language servers, and Code Mode +# execution in one short-lived E2B sandbox. - id: base name: '@cordisjs/plugin-include' config: - path: ./cordis.yml + path: ./advanced.cordis.yml patches: - id: subprocess name: '@deepseek-ai/dsh-subprocess-local' @@ -11,6 +12,9 @@ - id: fs-local name: '@deepseek-ai/dsh-fs-local' disabled: true + - id: code-runtime + name: '@deepseek-ai/dsh-code-runtime-worker' + disabled: true - id: bash name: '@deepseek-ai/dsh-bash-local' config: @@ -28,3 +32,27 @@ name: '@deepseek-ai/dsh-subprocess-e2b' - id: fs-e2b name: '@deepseek-ai/dsh-fs-e2b' + - id: code-runtime-e2b + name: '@deepseek-ai/dsh-code-runtime-e2b' + - id: pty + name: '@deepseek-ai/dsh-pty' + - id: pty-e2b + name: '@deepseek-ai/dsh-pty-e2b' + - id: tool-pty + name: '@deepseek-ai/dsh-tool-pty' + - id: lsp + name: '@deepseek-ai/dsh-lsp' + - id: lsp-e2b + name: '@deepseek-ai/dsh-lsp-e2b' + config: + servers: + typescript: + command: npx + args: [--yes, typescript-language-server@5.0.0, --stdio] + extensionToLanguage: + .ts: typescript + .tsx: typescriptreact + .js: javascript + .jsx: javascriptreact + - id: tool-lsp + name: '@deepseek-ai/dsh-tool-lsp' diff --git a/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts b/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts index 3b2cef609b..33e9530708 100644 --- a/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts +++ b/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts @@ -1,13 +1,38 @@ +import { readFile } from 'node:fs/promises' import { resolve } from 'node:path' import { boot } from '@deepseek-ai/dsh-app-boot' +import { AgentMessageId } from '@deepseek-ai/dsh-agent' +import type { Agent } from '@deepseek-ai/dsh-agent' +import { Session, SessionId } from '@deepseek-ai/dsh-session' +import type {} from '@deepseek-ai/dsh-code-runtime-e2b' import type {} from '@deepseek-ai/dsh-e2b' import type {} from '@deepseek-ai/dsh-fs-e2b' import type {} from '@deepseek-ai/dsh-bash-local' +import type {} from '@deepseek-ai/dsh-lsp-e2b' +import type {} from '@deepseek-ai/dsh-pty-e2b' const configPath = process.argv[2] if (configPath === undefined) throw new Error('usage: bin.ts ') const ctx = await boot('e2b-composition', resolve(configPath)) +const ownerFiber = ctx.plugin(() => {}) +const ownerId = SessionId('e2b-live-owner') +const owner: Agent = { + id: ownerId, + options: {}, + session: new Session(ownerId), + status: 'idle', + acceptsNextStep: false, + ctx: ownerFiber.ctx, + followup: () => AgentMessageId('unused'), + steer: () => AgentMessageId('unused'), + inject: () => AgentMessageId('unused'), + send: () => AgentMessageId('unused'), + cancel() {}, + whenIdle: () => Promise.resolve(), +} +const unregisterOwner = ctx.agents.register(owner) +let terminalId: Awaited>['sessionId'] | undefined try { const fromFs = await ctx.fs.resolve('from-fs.txt') await ctx.fs.writeText(fromFs, 'written-by-fs\n', { kind: 'createIfAbsent' }) @@ -22,11 +47,129 @@ try { } const fromBash = await ctx.fs.resolve('from-bash.txt') const fsRead = await ctx.fs.readText(fromBash) + + const lspFixture = await readFile(new URL('./fixture-lsp.mjs', import.meta.url), 'utf8') + const remoteLspFixture = await ctx.fs.resolve('fixture-lsp.mjs') + await ctx.fs.writeText(remoteLspFixture, lspFixture, { kind: 'createIfAbsent' }) + const remoteSource = await ctx.fs.resolve('multibyte.ts') + await ctx.fs.writeText(remoteSource, 'const café = "你好"\nconsole.log(café)\n', { kind: 'createIfAbsent' }) + const hover = await ctx.lsp.query({ + operation: 'hover', + filePath: 'multibyte.ts', + position: { line: 0, character: 7 }, + workspaceRoot: process.cwd(), + }) + const definition = await ctx.lsp.query({ + operation: 'goToDefinition', + filePath: 'multibyte.ts', + position: { line: 0, character: 7 }, + workspaceRoot: process.cwd(), + }) + + const terminal = await ctx.pty.spawn(owner, { type: 'shell' }) + terminalId = terminal.sessionId + const terminalEcho = await ctx.pty.startSend(owner, terminal.sessionId, { + text: "printf 'PTY-你好\\n'", + submit: true, + }).done + const sleeping = ctx.pty.startSend(owner, terminal.sessionId, { text: 'sleep 30', submit: true }) + await new Promise(resolveDelay => setTimeout(resolveDelay, 150)) + const terminalSignal = await ctx.pty.signal(owner, terminal.sessionId, 'SIGINT') + const interrupted = await sleeping.done + const terminalScrollback = ctx.pty.read(owner, terminal.sessionId, { count: 50 }) + await ctx.pty.kill(owner, terminal.sessionId, 'live E2B composition complete') + terminalId = undefined + + const code = await ctx.codeRuntime.run({ + program: ` + console.log('remote-log 你好', 42) + const arrayPrototype = Array.prototype + const objectPrototype = Object.prototype + const setPrototype = Set.prototype + const stringPrototype = String.prototype + Array.isArray = () => false + Object.defineProperty = Object.getPrototypeOf = Object.keys = () => { throw new Error('mutated object method') } + Object.hasOwn = () => false + Object.is = () => true + objectPrototype.propertyIsEnumerable = () => false + Number.isFinite = Number.isSafeInteger = () => false + Reflect.apply = Reflect.ownKeys = () => { throw new Error('mutated reflect method') } + setPrototype.add = setPrototype.delete = setPrototype.has = () => { throw new Error('mutated set method') } + stringPrototype.charCodeAt = stringPrototype.codePointAt = stringPrototype.slice = () => { throw new Error('mutated string method') } + Buffer.byteLength = () => 0 + Function.prototype.toString = () => 'mutated' + objectPrototype.constructor = arrayPrototype.constructor = null + globalThis.Array = globalThis.Buffer = globalThis.Function = globalThis.Number = globalThis.Object = globalThis.Promise = globalThis.Reflect = globalThis.Set = globalThis.String = undefined + process.stdout.write('post-mutation', () => {}) + const doubled: number = await bridge.double({ value: 21 }) + let typed = false + try { + await bridge.fail({ reason: 'expected' }) + } catch (error) { + typed = error instanceof BridgeError && (error as { member: string }).member === 'fail' + } + return { doubled, typed } + `, + bindings: [{ + global: 'bridge', + errorClass: { name: 'BridgeError', memberNameProperty: 'member' }, + functions: { + double: async (args) => { + const value = (args as { value: number }).value + return value * 2 + }, + fail: async () => { throw new Error('binding rejected') }, + }, + }], + }) + const hostileOutput = await ctx.codeRuntime.run({ + program: ` + const payload = '🙂'.repeat(4096) + String.prototype[Symbol.iterator] = () => { throw new Error('mutated string iterator') } + console.log(payload) + return true + `, + bindings: [], + }) + const timedOut = await ctx.codeRuntime.run({ + program: 'await new Promise(() => {})', + bindings: [], + }) + const abortController = new AbortController() + const aborting = ctx.codeRuntime.run({ + program: 'await new Promise(() => {})', + bindings: [], + signal: abortController.signal, + }) + setTimeout(() => { abortController.abort('live abort') }, 50) + const aborted = await aborting + const remoteProcesses = await (await ctx.e2b.getSandbox()).commands.list() + const lingeringCodeRunners = remoteProcesses.filter(processInfo => + JSON.stringify([processInfo.cmd, processInfo.args]).includes('code-runtime-runner.mjs'), + ) + process.stdout.write(`${JSON.stringify({ sandboxId: await ctx.e2b.sandboxId, bashRead: bashRead.stdout.text, fsRead, + hover, + definition, + terminal: { + motd: terminal.motd, + echo: terminalEcho, + signal: terminalSignal, + interrupted, + scrollback: terminalScrollback.text, + }, + code, + hostileOutput, + timedOut, + aborted, + lingeringCodeRunners: lingeringCodeRunners.length, })}\n`) } finally { + if (terminalId !== undefined) await ctx.pty.kill(owner, terminalId, 'fixture cleanup').catch(() => false) + unregisterOwner() + await ownerFiber.dispose() await ctx.fiber.dispose() } diff --git a/examples/headless-agent/tests/fixtures/e2b/e2b/cordis.yml b/examples/headless-agent/tests/fixtures/e2b/e2b/cordis.yml index 08a7e7225c..f5cf403c4b 100644 --- a/examples/headless-agent/tests/fixtures/e2b/e2b/cordis.yml +++ b/examples/headless-agent/tests/fixtures/e2b/e2b/cordis.yml @@ -17,3 +17,43 @@ - id: fs-e2b name: '@deepseek-ai/dsh-fs-e2b' + +- id: agents + name: '@deepseek-ai/dsh-agent' + +- id: pty + name: '@deepseek-ai/dsh-pty' + +- id: pty-e2b + name: '@deepseek-ai/dsh-pty-e2b' + config: + pollIntervalMs: 25 + idleSilenceMs: 2000 + timeoutMs: 5000 + disposeGraceMs: 1000 + +- id: lsp + name: '@deepseek-ai/dsh-lsp' + +- id: lsp-e2b + name: '@deepseek-ai/dsh-lsp-e2b' + config: + servers: + fixture: + command: node + args: + - !!js process.cwd() + '/fixture-lsp.mjs' + extensionToLanguage: + .ts: typescript + shutdownTimeoutMs: 1000 + killGraceMs: 500 + +- id: code-runtime-e2b + name: '@deepseek-ai/dsh-code-runtime-e2b' + config: + computeMs: 500 + maxWallMs: 5000 + maxOutputBytes: 4096 + maxOldGenerationSizeMb: 128 + maxFrameBytes: 4194304 + killGraceMs: 500 diff --git a/examples/headless-agent/tests/fixtures/e2b/e2b/fixture-lsp.mjs b/examples/headless-agent/tests/fixtures/e2b/e2b/fixture-lsp.mjs new file mode 100644 index 0000000000..dc4f7bb219 --- /dev/null +++ b/examples/headless-agent/tests/fixtures/e2b/e2b/fixture-lsp.mjs @@ -0,0 +1,85 @@ +import { Buffer } from 'node:buffer' + +let pending = Buffer.alloc(0) +let source = '' +let sourceUri = '' + +function send(message) { + const body = Buffer.from(JSON.stringify(message)) + process.stdout.write(`Content-Length: ${body.length}\r\n\r\n`) + process.stdout.write(body) +} + +function respond(id, result) { + send({ jsonrpc: '2.0', id, result }) +} + +function dispatch(message) { + switch (message.method) { + case 'initialize': + respond(message.id, { + capabilities: { + positionEncoding: 'utf-16', + textDocumentSync: { openClose: true, change: 1 }, + definitionProvider: true, + referencesProvider: true, + implementationProvider: true, + hoverProvider: true, + }, + }) + return + case 'textDocument/didOpen': + source = message.params.textDocument.text + sourceUri = message.params.textDocument.uri + return + case 'textDocument/didClose': + source = '' + sourceUri = '' + return + case 'textDocument/hover': + if (!source.includes('const café = "你好"')) { + send({ jsonrpc: '2.0', id: message.id, error: { code: -32000, message: 'multibyte source was corrupted' } }) + return + } + respond(message.id, { + contents: { kind: 'markdown', value: '**remote hover** 你好 café' }, + range: { start: { line: 0, character: 6 }, end: { line: 0, character: 10 } }, + }) + return + case 'textDocument/definition': + case 'textDocument/references': + case 'textDocument/implementation': + respond(message.id, [{ + uri: sourceUri, + range: { start: { line: 0, character: 6 }, end: { line: 0, character: 10 } }, + }]) + return + case 'shutdown': + respond(message.id, null) + return + case 'exit': + process.exit(0) + return + } +} + +function drain() { + for (;;) { + const headerEnd = pending.indexOf('\r\n\r\n') + if (headerEnd < 0) return + const header = pending.subarray(0, headerEnd).toString('ascii') + const match = /(?:^|\r\n)Content-Length: ([0-9]+)(?:\r\n|$)/i.exec(header) + if (!match) throw new Error('missing Content-Length') + const length = Number(match[1]) + const bodyStart = headerEnd + 4 + if (pending.length < bodyStart + length) return + const body = pending.subarray(bodyStart, bodyStart + length) + pending = pending.subarray(bodyStart + length) + dispatch(JSON.parse(body.toString('utf8'))) + } +} + +process.stdin.on('data', chunk => { + pending = Buffer.concat([pending, chunk]) + drain() +}) diff --git a/examples/package.json b/examples/package.json index 48ac39f5ca..8dd631e760 100644 --- a/examples/package.json +++ b/examples/package.json @@ -43,10 +43,12 @@ "@deepseek-ai/dsh-llm-replay": "workspace:*", "@deepseek-ai/dsh-loader-smoke": "workspace:*", "@deepseek-ai/dsh-lsp": "workspace:*", + "@deepseek-ai/dsh-lsp-e2b": "workspace:*", "@deepseek-ai/dsh-lsp-local": "workspace:*", "@deepseek-ai/dsh-permission": "workspace:*", "@deepseek-ai/dsh-plan-mode": "workspace:*", "@deepseek-ai/dsh-pty": "workspace:*", + "@deepseek-ai/dsh-pty-e2b": "workspace:*", "@deepseek-ai/dsh-pty-local": "workspace:*", "@deepseek-ai/dsh-pwsh-local": "workspace:*", "@deepseek-ai/dsh-repeat-tool-guard": "workspace:*", @@ -54,7 +56,10 @@ "@deepseek-ai/dsh-sandbox": "workspace:*", "@deepseek-ai/dsh-sandbox-local": "workspace:*", "@deepseek-ai/dsh-sandbox-policy": "workspace:^", +<<<<<<< HEAD "@deepseek-ai/dsh-scope": "workspace:*", +======= +>>>>>>> 3f6c905d5e (Add E2B PTY, LSP, and code runtime providers) "@deepseek-ai/dsh-session": "workspace:*", "@deepseek-ai/dsh-session-checkpoint-policy": "workspace:*", "@deepseek-ai/dsh-session-persistence-jsonl": "workspace:*", diff --git a/packages/code-runtime/README.i18n.yaml b/packages/code-runtime/README.i18n.yaml index 3eb462d513..f9160b4f8e 100644 --- a/packages/code-runtime/README.i18n.yaml +++ b/packages/code-runtime/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/code-runtime/README.md -README.md: f20a287419b94b1a9dc1d8da7303fc4d3032cfd3 -README.zh.md: f5cd4c9949f2bd7a7d6d7cd078144910712a3819 +README.md: f59fc3b15331b4799cbc7a3fbe27ce8cd9a12e51 +README.zh.md: 02ade97887fab8a8a0c22efcf73bb2dd1f2cf33c diff --git a/packages/code-runtime/README.md b/packages/code-runtime/README.md index f20a287419..f59fc3b153 100644 --- a/packages/code-runtime/README.md +++ b/packages/code-runtime/README.md @@ -2,11 +2,12 @@ English | [中文](README.zh.md) -The code-execution capability seam (see [capability seams](../../.agents/notes/implemented/architecture/2026-06-13-capability-seams.md)): an abstract runtime interface for executing one model-written program against host-provided async bindings, capturing what it printed and returned. The consumer is the tool registry's [Code Mode](../core/tools/README.md) (`tools: { mode: code }` — the `run_code` tool and the SDK generated in the loaded runtime's `language`); design in the [Code Mode Agent Note](../../.agents/notes/implemented/feature/2026-06-15-code-mode.md). **Product** packages. +The code-execution capability seam (see [capability seams](../../.agents/notes/implemented/architecture/2026-06-13-capability-seams.md)): an abstract runtime interface for executing one model-written program against host-provided async bindings, capturing what it printed and returned. The consumer is the tool registry's [Code Mode](../core/tools/README.md) (`tools: { mode: code }` — the `run_code` tool and the generated TypeScript SDK); design in the [Code Mode Agent Note](../../.agents/notes/implemented/feature/2026-06-15-code-mode.md). **Product** packages. | Package | Role | ctx key | |---|---|---| -| [`code-runtime/`](code-runtime/README.md) | Code-execution seam and shared vocabulary | `ctx.codeRuntime` | -| [`code-runtime-worker/`](code-runtime-worker/README.md) | Worker-thread backend | registers `ctx.codeRuntime` | +| `code-runtime/` | Abstract code-execution seam (interface + vocabulary) | `ctx.codeRuntime` | +| [`code-runtime-worker/`](code-runtime-worker/README.md) | Worker-thread backend: fresh worker per run, TypeScript via host-side type-strip (annotations advisory, never type-checked), port-bridged bindings, budget/heap containment | registers `ctx.codeRuntime` | +| [`code-runtime-e2b/`](code-runtime-e2b/README.md) | E2B backend: host type-strip and bindings, fresh remote runner/worker, framed bridge, remote process-group cleanup | registers `ctx.codeRuntime` | -Backends register the seam without changing its consumer. The child READMEs own language, isolation, and execution-budget details. +Backends differ by execution substrate and source language—both readonly descriptors on the service—and register `ctx.codeRuntime` without touching the interface or its consumer. The E2B ownership split is recorded in the [remote extension note](../../.agents/notes/implemented/feature/2026-07-28-e2b-interactive-semantic-code-runtime-poc.md). diff --git a/packages/code-runtime/README.zh.md b/packages/code-runtime/README.zh.md index f5cd4c9949..02ade97887 100644 --- a/packages/code-runtime/README.zh.md +++ b/packages/code-runtime/README.zh.md @@ -1,12 +1,13 @@ -# code-runtime/ — 代码执行能力家族 +# code-runtime/:代码执行能力家族 [English](README.md) | 中文 -代码执行能力 seam(参见[能力 seam](../../.agents/notes/implemented/architecture/2026-06-13-capability-seams.md)):一个抽象运行时接口,用于对宿主提供的异步绑定执行模型编写的程序,并捕获它打印和返回的内容。消费方是工具注册表的 [Code Mode](../core/tools/README.md)(`tools: { mode: code }`,即 `run_code` 工具和按所加载运行时 `language` 生成的 SDK);设计见 [Code Mode Agent Note](../../.agents/notes/implemented/feature/2026-06-15-code-mode.md)。这些全是**产品**包。 +代码执行能力 seam(参见[能力 seam](../../.agents/notes/implemented/architecture/2026-06-13-capability-seams.md)):一个抽象运行时接口,用于针对宿主提供的异步绑定执行一段模型编写的程序,并捕获程序打印和返回的内容。消费方是工具注册表的 [Code Mode](../core/tools/README.md)(`tools: { mode: code }`,即 `run_code` 工具与生成的 TypeScript SDK);设计记录在 [Code Mode Agent Note](../../.agents/notes/implemented/feature/2026-06-15-code-mode.md) 中。这些都是**产品** 包。 -| 包 | 职责 | ctx key | +| 包 | 职责 | ctx 键 | |---|---|---| -| [`code-runtime/`](code-runtime/README.md) | 代码执行 seam 与共享词汇 | `ctx.codeRuntime` | -| [`code-runtime-worker/`](code-runtime-worker/README.md) | Worker 线程后端 | 注册 `ctx.codeRuntime` | +| `code-runtime/` | 抽象代码执行 seam(接口 + 词汇) | `ctx.codeRuntime` | +| [`code-runtime-worker/`](code-runtime-worker/README.md) | worker 线程后端:每次运行使用全新 worker,由宿主侧剥离 TypeScript 类型(类型注解仅供参考,绝不执行类型检查)、端口桥接绑定、预算/堆隔离 | 注册 `ctx.codeRuntime` | +| [`code-runtime-e2b/`](code-runtime-e2b/README.md) | E2B 后端:宿主侧类型剥离与绑定、全新远程 runner/worker、分帧桥、远程进程组清理 | 注册 `ctx.codeRuntime` | -后端在不改变消费方的情况下注册该 seam。子 README 负责语言、隔离和执行预算细节。 +不同后端的执行基底和源语言各异,二者都是服务上的只读描述符;后端注册 `ctx.codeRuntime`,无需修改接口或消费方。E2B 所有权拆分记录在[远程扩展 Agent Note](../../.agents/notes/implemented/feature/2026-07-28-e2b-interactive-semantic-code-runtime-poc.md) 中。 diff --git a/packages/code-runtime/code-runtime-e2b/README.i18n.yaml b/packages/code-runtime/code-runtime-e2b/README.i18n.yaml new file mode 100644 index 0000000000..c00b40f4a2 --- /dev/null +++ b/packages/code-runtime/code-runtime-e2b/README.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write packages/code-runtime/code-runtime-e2b/README.md +README.md: bf26174c59c1df18f63278acb7923fd409860b44 +README.zh.md: c87945da30a6f8d77eb3722879510bb6159b3ee8 diff --git a/packages/code-runtime/code-runtime-e2b/README.md b/packages/code-runtime/code-runtime-e2b/README.md new file mode 100644 index 0000000000..bf26174c59 --- /dev/null +++ b/packages/code-runtime/code-runtime-e2b/README.md @@ -0,0 +1,44 @@ +# @deepseek-ai/dsh-code-runtime-e2b + +English | [中文](README.zh.md) + +E2B implementation of [`ctx.codeRuntime`](../code-runtime/README.md). Each run executes one model-written TypeScript program in a fresh remote Node worker while binding functions, type stripping, output accounting, and lifecycle orchestration remain on the host. + +## Configuration + +| Key | Default | Meaning | +|---|---|---| +| `computeMs` | `60000` | Remote worker event-loop busy-time budget. | +| `maxWallMs` | `600000` | Host-observed wall-clock ceiling. | +| `maxOutputBytes` | `67108864` | Combined serialized outer logs/value/diagnostic cap. | +| `maxOldGenerationSizeMb` | `512` | Remote worker old-generation heap cap in MiB. | +| `maxFrameBytes` | `268435456` | Largest decoded bridge frame, including binding traffic. | +| `killGraceMs` | `2000` | Remote process-group TERM-to-KILL grace. | + +Every value is a positive safe integer. `maxOutputBytes` is at least four bytes, `maxWallMs` cannot exceed Node's maximum timer delay, and `maxFrameBytes` cannot be smaller than `maxOutputBytes`. The service requires the concrete `dsh-subprocess-e2b` backend so run cleanup has remote process-group semantics. + +## Execution and bridge contract + +Setup uploads one dependency-free runner under `ctx.e2b.runtimeRoot` and resolves remote Node. For each run, the host wraps and type-strips erasable TypeScript with Node's `stripTypeScriptTypes`, then starts the runner in `ctx.e2b.cwd`. The runner creates a fresh worker thread with an empty environment and heap limit, measures active event-loop time, and destroys that worker after one completion. The enclosing E2B subprocess group is terminated and awaited after every result, timeout, abort, or disposal, so ordinary child processes in that group stop with the run. + +The bridge uses validated newline-delimited base64 JSON frames because E2B subprocess callbacks expose decoded text. Binding arguments and resolutions use the worker runtime's iterative lossless-JSON wire shape; binding functions execute on the host and typed rejection classes are materialized inside the remote worker. The worker captures the JavaScript intrinsics that its adapter boundary invokes before model code runs, hardening binding transport, output accounting, and completion validation against mutation of those references. The host repeats message validation, call-id deduplication, lossless-JSON checks, and the outer-output ledger. + +Program failures resolve as `CodeRunResult.error`; only seam misuse rejects. `isolation` is reported as `container`, which is a deployment descriptor rather than a security claim. + +## Model Experience + +Indirectly, through Code Mode in `dsh-tools`, which returns program logs, values, or typed failures through the existing `run_code` result contract. + +#### KV Cache effect + +No direct invalidation; Code Mode owns request-prefix changes. + +## Known Limitations and Deferred Work + +- **Not a whole-agent runtime** — Cordis, sessions, LLM calls, binding dispatch, TypeScript stripping, output ledgers, and E2B SDK state remain on the host. +- **No reconnectable runs** — retaining a sandbox preserves files but not worker/subprocess handles, binding calls, timers, or output cursors. +- **Node worker internals share the model realm** — mutating realm-wide globals or prototypes that Node itself uses can terminate the worker; captured adapter intrinsics are not a separate JavaScript realm or a security boundary. +- **Deliberate process-group escape is not captured** — model code can create a new POSIX session; that unmanaged process is outside this backend's cleanup identity. +- **Intermediate binding traffic is memory-bounded only per frame** — it does not enter model context or the outer-output ledger, but aggregate host/remote process memory remains the limit. +- **Experimental type stripping** — the backend shares the worker implementation's reliance on Node's experimental erasable-syntax API. +- **Sandbox policy is template-owned** — this package adds no network, volume, snapshot, or workspace-synchronization policy. diff --git a/packages/code-runtime/code-runtime-e2b/README.zh.md b/packages/code-runtime/code-runtime-e2b/README.zh.md new file mode 100644 index 0000000000..c87945da30 --- /dev/null +++ b/packages/code-runtime/code-runtime-e2b/README.zh.md @@ -0,0 +1,44 @@ +# @deepseek-ai/dsh-code-runtime-e2b + +[English](README.md) | 中文 + +[`ctx.codeRuntime`](../code-runtime/README.md) 的 E2B 实现。每次运行都会在全新的远程 Node worker 中执行一段模型编写的 TypeScript 程序;绑定函数、类型剥离、输出记账和生命周期编排仍保留在宿主侧。 + +## 配置 + +| 配置键 | 默认值 | 含义 | +|---|---|---| +| `computeMs` | `60000` | 远程 worker 的事件循环忙碌时间预算。 | +| `maxWallMs` | `600000` | 宿主观测到的墙钟时间上限。 | +| `maxOutputBytes` | `67108864` | 外层日志、值和诊断合计的序列化上限。 | +| `maxOldGenerationSizeMb` | `512` | 远程 worker 的老生代堆上限(MiB)。 | +| `maxFrameBytes` | `268435456` | 已解码桥接帧的最大大小,包括绑定流量。 | +| `killGraceMs` | `2000` | 远程进程组 TERM 到 KILL 的宽限期。 | + +每个值都必须是正的安全整数。`maxOutputBytes` 必须至少为 4 字节,`maxWallMs` 不得超过 Node 的最大定时器延迟,且 `maxFrameBytes` 不得小于 `maxOutputBytes`。本服务要求使用具体的 `dsh-subprocess-e2b` 后端,使运行清理具备远程进程组语义。 + +## 执行与桥接契约 + +设置阶段会在 `ctx.e2b.runtimeRoot` 下上传一个无依赖的 runner,并解析远程 Node。每次运行时,宿主会包装仅使用可擦除语法的 TypeScript,再用 Node 的 `stripTypeScriptTypes` 剥离类型,然后在 `ctx.e2b.cwd` 中启动 runner。runner 会创建一个具有空环境与堆上限的全新 worker 线程,测量事件循环活跃时间,并在一次运行结算后销毁该 worker。每当运行返回结果、超时、中止或因资源释放终止时,系统都会终止外围的 E2B 进程组并等待其退出,因此组内的普通子进程会随本次运行一同停止。 + +由于 E2B 进程管理回调公开的是已解码文本,桥接层使用经过验证、以换行分隔的 base64 JSON 帧。绑定参数与 resolve 值使用 worker 运行时的迭代式无损 JSON wire 形状;绑定函数在宿主执行,类型化的 reject 类则在远程 worker 内物化。worker 会在模型代码运行前捕获其适配器边界调用的 JavaScript intrinsic,从而增强绑定传输、输出记账与完成值验证对这些引用修改的抵御能力。宿主会再次执行消息验证、调用 id 去重和无损 JSON 检查,并用外层输出账本再次计量。 + +程序失败会 resolve 为 `CodeRunResult.error`;只有 seam 误用才会 reject。`isolation` 报告为 `container`;这是部署描述符,不构成安全声明。 + +## 模型体验 + +通过 `dsh-tools` 中的 Code Mode 间接影响模型;它会通过现有 `run_code` 结果契约返回程序日志、值或类型化失败。 + +#### KV Cache 影响 + +不会直接失效;请求前缀变更由 Code Mode 负责。 + +## 已知限制与暂缓工作 + +- **并非完整的 agent(智能体)运行时**:Cordis、会话、LLM(大语言模型)调用、绑定分发、TypeScript 类型剥离、输出账本和 E2B SDK 状态仍保留在宿主侧。 +- **运行不可重连**:保留沙箱会保留文件,但不会保留 worker/进程管理句柄、绑定调用、定时器或输出游标。 +- **Node worker 内部机制与模型共享同一 realm**:修改 Node 自身使用、影响整个 realm 的全局对象或原型可能会终止 worker;已捕获的适配器 intrinsic 并不构成独立的 JavaScript realm 或安全边界。 +- **不会捕获有意逃逸进程组的行为**:模型代码可以创建新的 POSIX 会话;该非受管进程不属于此后端的清理身份范围。 +- **中间绑定流量的内存边界仅适用于单帧**:它不会进入模型上下文或外层输出账本,但其总量仍只受宿主/远程进程内存限制。 +- **实验性类型剥离**:该后端与 worker 实现一样,依赖 Node 的实验性可擦除语法 API。 +- **沙箱策略归模板负责**:本包不会额外增加网络、卷、快照或工作区同步策略。 diff --git a/packages/code-runtime/code-runtime-e2b/package.json b/packages/code-runtime/code-runtime-e2b/package.json new file mode 100644 index 0000000000..222ae71bef --- /dev/null +++ b/packages/code-runtime/code-runtime-e2b/package.json @@ -0,0 +1,52 @@ +{ + "name": "@deepseek-ai/dsh-code-runtime-e2b", + "description": "E2B code-runtime implementation for DeepSeek Harness", + "version": "0.0.1", + "private": true, + "type": "module", + "main": "lib/index.js", + "types": "lib/types/index.d.ts", + "exports": { + ".": { + "types": "./lib/types/index.d.ts", + "default": "./lib/index.js" + }, + "./invariant": { + "types": "./lib/types/invariant.d.ts", + "default": "./lib/invariant.js" + }, + "./src/*": "./src/*", + "./package.json": "./package.json" + }, + "files": [ + "lib/index.js", + "lib/invariant.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", + "src" + ], + "license": "BSD-3-Clause", + "peerDependencies": { + "@deepseek-ai/dsh-code-runtime": "^0.0.1", + "@deepseek-ai/dsh-code-runtime-worker": "^0.0.1", + "@deepseek-ai/dsh-e2b": "^0.0.1", + "@deepseek-ai/dsh-invariants": "^0.0.1", + "@deepseek-ai/dsh-session": "^0.0.1", + "@deepseek-ai/dsh-subprocess-e2b": "^0.0.1", + "@deepseek-ai/dsh-timeout": "^0.0.1", + "cordis": "^4.0.0-rc.7" + }, + "dependencies": { + "schemastery": "^3.18.0" + }, + "devDependencies": { + "@deepseek-ai/dsh-code-runtime": "workspace:^", + "@deepseek-ai/dsh-code-runtime-worker": "workspace:^", + "@deepseek-ai/dsh-e2b": "workspace:^", + "@deepseek-ai/dsh-invariants": "workspace:^", + "@deepseek-ai/dsh-session": "workspace:^", + "@deepseek-ai/dsh-subprocess-e2b": "workspace:^", + "@deepseek-ai/dsh-timeout": "workspace:^", + "cordis": "^4.0.0-rc.7" + } +} diff --git a/packages/code-runtime/code-runtime-e2b/src/index.ts b/packages/code-runtime/code-runtime-e2b/src/index.ts new file mode 100644 index 0000000000..fbfbdea91f --- /dev/null +++ b/packages/code-runtime/code-runtime-e2b/src/index.ts @@ -0,0 +1,441 @@ +/** E2B process/worker implementation of the harness code-runtime seam. */ + +import { posix } from 'node:path' +import { stripTypeScriptTypes } from 'node:module' +import type { Context } from 'cordis' +import z from 'schemastery' +import { CodeRuntime } from '@deepseek-ai/dsh-code-runtime' +import type { + CodeBindingNamespace, + CodeJsonValue, + CodeRunFailure, + CodeRunRequest, + CodeRunResult, +} from '@deepseek-ai/dsh-code-runtime' +import { + E2BFrameDecoder, + encodeE2BFrame, + quoteE2BShellArg, + resolveE2BExecutable, +} from '@deepseek-ai/dsh-e2b' +import { + decodeWorkerJson, + encodeWorkerJson, + OutputLedger, +} from '@deepseek-ai/dsh-code-runtime-worker' +import type { WorkerJsonWire } from '@deepseek-ai/dsh-code-runtime-worker' +import { snapshotJsonValue } from '@deepseek-ai/dsh-session' +import E2BSubprocessService from '@deepseek-ai/dsh-subprocess-e2b' +import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout' +import { CODE_RUNNER_SOURCE } from './runner-source.ts' + +/** Runtime configuration; every execution and bridge bound is deployment-tunable. */ +export interface Config { + /** Remote worker measured event-loop busy-time budget. */ + computeMs?: number + /** Host-observed wall-clock ceiling. */ + maxWallMs?: number + /** Combined serialized outer logs/value/diagnostic cap. */ + maxOutputBytes?: number + /** Remote worker old-generation heap cap in MiB. */ + maxOldGenerationSizeMb?: number + /** Largest decoded bridge frame, including binding traffic. */ + maxFrameBytes?: number + /** Remote process-group TERM-to-KILL grace. */ + killGraceMs?: number +} + +type ResolvedConfig = Required + +interface LiveRun { + settle(failure: CodeRunFailure): void + finished: Promise +} + +interface CallMessage { + type: 'call' + id: number + global: string + name: string + args: WorkerJsonWire +} + +interface LogMessage { + type: 'log' + text: string +} + +interface DoneMessage { + type: 'done' + value?: WorkerJsonWire + error?: CodeRunFailure +} + +type RunnerMessage = CallMessage | LogMessage | DoneMessage | { type: 'output-limit' } + +const STRIP_WRAP = { prefix: 'async function __dsh_program__() {\n', suffix: '\n}' } as const +const MIN_OUTPUT_BYTES = 4 +const IDENTIFIER = /^[A-Za-z_$][A-Za-z0-9_$]*$/ +/* jscpd:ignore-start -- Backends enforce the same injected-global vocabulary without coupling lifecycle implementations. */ +const RESERVED_WORDS = new Set([ + 'await', 'break', 'case', 'catch', 'class', 'const', 'continue', 'debugger', 'default', 'delete', 'do', + 'else', 'enum', 'export', 'extends', 'false', 'finally', 'for', 'function', 'if', 'import', 'in', + 'instanceof', 'new', 'null', 'return', 'super', 'switch', 'this', 'throw', 'true', 'try', 'typeof', + 'var', 'void', 'while', 'with', 'yield', 'let', 'static', 'implements', 'interface', 'package', + 'private', 'protected', 'public', 'arguments', 'eval', +]) +const RESERVED_ERROR_PROPERTIES = new Set(['name', 'message', 'stack']) +/* jscpd:ignore-end */ +const FAILURE_KINDS = new Set([ + 'exception', 'timeout', 'abort', 'worker-exit', 'invalid-output', 'output-limit', +]) + +function messageOf(error: unknown): string { + return error instanceof Error ? error.message : String(error) +} + +function parseRunnerMessage(raw: unknown): RunnerMessage | undefined { + if (typeof raw !== 'object' || raw === null) return undefined + const record = raw as Record + if (record.type === 'output-limit') return { type: 'output-limit' } + if (record.type === 'log') return typeof record.text === 'string' ? { type: 'log', text: record.text } : undefined + if (record.type === 'call') { + if (!Number.isSafeInteger(record.id) || (record.id as number) < 1 || typeof record.global !== 'string' || typeof record.name !== 'string' || !Array.isArray(record.args)) return undefined + return { type: 'call', id: record.id as number, global: record.global, name: record.name, args: record.args as WorkerJsonWire } + } + if (record.type !== 'done') return undefined + if (record.error === undefined) { + return { type: 'done', ...record.value === undefined ? {} : { value: record.value as WorkerJsonWire } } + } + if (typeof record.error !== 'object' || record.error === null) return undefined + const error = record.error as Record + if (typeof error.kind !== 'string' || !FAILURE_KINDS.has(error.kind as CodeRunFailure['kind']) || typeof error.message !== 'string') return undefined + return { type: 'done', error: { kind: error.kind as CodeRunFailure['kind'], message: error.message } } +} + +/** E2B-backed runtime: host-side type stripping, remote worker execution, host binding dispatch. */ +export class E2BCodeRuntime extends CodeRuntime { + static inject = ['e2b', 'subprocess'] + + static Config: z = z.object({ + computeMs: z.number().default(60_000), + maxWallMs: z.number().default(600_000), + maxOutputBytes: z.number().default(67_108_864), + maxOldGenerationSizeMb: z.number().default(512), + maxFrameBytes: z.number().default(268_435_456), + killGraceMs: z.number().default(2_000), + }) + + readonly language = 'typescript' + readonly isolation = 'container' + + private readonly config: ResolvedConfig + private readonly ready: Promise<{ node: string; runner: string }> + private readonly live = new Set() + private readonly subprocess: E2BSubprocessService + private disposed = false + + constructor(ctx: Context, config: Config) { + super(ctx) + if (!(ctx.subprocess instanceof E2BSubprocessService)) { + throw new Error('code-runtime-e2b requires @deepseek-ai/dsh-subprocess-e2b as ctx.subprocess') + } + this.subprocess = ctx.subprocess + this.config = config as ResolvedConfig + for (const [key, value] of Object.entries(this.config)) { + if (!Number.isSafeInteger(value) || value <= 0) { + throw new Error(`code-runtime-e2b: config.${key} must be a positive safe integer`) + } + } + if (this.config.maxOutputBytes < MIN_OUTPUT_BYTES) { + throw new Error(`code-runtime-e2b: config.maxOutputBytes must be at least ${MIN_OUTPUT_BYTES}`) + } + if (this.config.maxWallMs > MAX_TIMER_DELAY_MS) { + throw new Error(`code-runtime-e2b: config.maxWallMs must be at most ${MAX_TIMER_DELAY_MS}`) + } + if (this.config.maxFrameBytes < this.config.maxOutputBytes) { + throw new Error('code-runtime-e2b: config.maxFrameBytes must be at least maxOutputBytes') + } + this.ready = this.prepare() + void this.ready.catch(() => {}) + ctx.effect(() => () => this.teardown(), 'E2B code-runtime teardown') + } + + /* jscpd:ignore-start -- Seam-level abort and type-strip results remain identical across execution substrates. */ + /** Execute one type-stripped program in a fresh E2B worker process. */ + async run(request: CodeRunRequest): Promise { + if (this.disposed) throw new Error('code-runtime-e2b: run() after disposal') + const bindings = this.validateBindings(request) + if (request.signal?.aborted === true) { + return this.failure({ kind: 'abort', message: String(request.signal.reason) }) + } + let code: string + try { + const stripped = stripTypeScriptTypes(STRIP_WRAP.prefix + request.program + STRIP_WRAP.suffix) + code = stripped.slice(STRIP_WRAP.prefix.length, stripped.length - STRIP_WRAP.suffix.length) + } catch (error: unknown) { + return this.failure({ kind: 'exception', message: messageOf(error) }) + } + let runtime: Awaited + try { + runtime = await this.ready + } catch (error: unknown) { + return this.failure({ kind: 'worker-exit', message: `E2B runtime setup failed: ${messageOf(error)}` }) + } + // Disposal can race the awaited remote setup after the pre-await check. + // eslint-disable-next-line @typescript-eslint/no-unnecessary-condition + if (this.disposed) return this.failure({ kind: 'abort', message: 'runtime disposed' }) + return await this.execute(request, code, bindings, runtime) + } + /* jscpd:ignore-end */ + + private async prepare(): Promise<{ node: string; runner: string }> { + const sandbox = await this.ctx.e2b.getSandbox() + const runner = posix.join(this.ctx.e2b.runtimeRoot, 'code-runtime-runner.mjs') + await sandbox.files.write([{ path: runner, data: CODE_RUNNER_SOURCE }]) + await sandbox.commands.run(`chmod 600 -- ${quoteE2BShellArg(runner)}`) + const node = await resolveE2BExecutable(sandbox, 'node') + return { node, runner } + } + + private failure(error: CodeRunFailure): CodeRunResult { + return new OutputLedger(this.config.maxOutputBytes).failure([], error) + } + + /* jscpd:ignore-start -- Binding names have one seam contract while dispatch and teardown remain backend-owned. */ + private validateBindings(request: CodeRunRequest): Map { + const bindings = new Map() + for (const namespace of request.bindings) { + if (!IDENTIFIER.test(namespace.global) || RESERVED_WORDS.has(namespace.global)) { + throw new Error(`code-runtime-e2b: binding global ${JSON.stringify(namespace.global)} is not a usable identifier`) + } + if (namespace.global === 'console' || bindings.has(namespace.global)) { + throw new Error(`code-runtime-e2b: duplicate binding global ${JSON.stringify(namespace.global)}`) + } + bindings.set(namespace.global, namespace) + } + const errorClassNames = new Set() + for (const namespace of request.bindings) { + const descriptor = namespace.errorClass + if (descriptor === undefined) continue + if (!IDENTIFIER.test(descriptor.name) || RESERVED_WORDS.has(descriptor.name)) { + throw new Error(`code-runtime-e2b: binding error class ${JSON.stringify(descriptor.name)} is not a usable identifier`) + } + if (descriptor.name === 'console' || bindings.has(descriptor.name) || errorClassNames.has(descriptor.name)) { + throw new Error(`code-runtime-e2b: duplicate injected global ${JSON.stringify(descriptor.name)}`) + } + if (descriptor.memberNameProperty.length === 0 || RESERVED_ERROR_PROPERTIES.has(descriptor.memberNameProperty)) { + throw new Error(`code-runtime-e2b: binding error member property ${JSON.stringify(descriptor.memberNameProperty)} is not usable`) + } + errorClassNames.add(descriptor.name) + } + return bindings + } + /* jscpd:ignore-end */ + + private async execute( + request: CodeRunRequest, + code: string, + bindings: Map, + runtime: { node: string; runner: string }, + ): Promise { + const handle = this.subprocess.spawn({ + argv: [runtime.node, runtime.runner], + cwd: this.ctx.e2b.cwd, + stdio: { stdin: 'pipe', stdout: 'pipe', stderr: { maxBytes: this.config.maxOutputBytes } }, + graceMs: this.config.killGraceMs, + ...request.signal === undefined ? {} : { signal: request.signal }, + env: {}, + }) + if (handle.stdin === undefined || handle.stdout === undefined) { + handle.terminate() + await Promise.allSettled([handle.done]) + try { + await handle.waitForExit() + } catch (error: unknown) { + return this.failure({ kind: 'worker-exit', message: `E2B runtime cleanup failed: ${messageOf(error)}` }) + } + return this.failure({ kind: 'worker-exit', message: 'E2B subprocess dropped a piped runtime stream' }) + } + const stdin = handle.stdin + const stdout = handle.stdout + + return new Promise((resolve) => { + const output = new OutputLedger(this.config.maxOutputBytes) + const logs: string[] = [] + const answered = new Set() + const decoder = new E2BFrameDecoder(this.config.maxFrameBytes) + let settled = false + let finishResolve!: () => void + const finished = new Promise((done) => { finishResolve = done }) + const wallTimer: { current: NodeJS.Timeout | undefined } = { current: undefined } + const live: LiveRun = { + finished, + settle: (failure) => { finish(() => output.failure(logs, failure)) }, + } + + const finish = (result: CodeRunResult | (() => CodeRunResult)): void => { + if (settled) return + settled = true + clearTimeout(wallTimer.current) + request.signal?.removeEventListener('abort', onAbort) + this.live.delete(live) + void new Promise((resume) => { setImmediate(resume) }).then(async () => { + handle.terminate() + await handle.done.catch(() => {}) + let cleanupError: unknown + try { + await handle.waitForExit() + } catch (error: unknown) { + cleanupError = error + } + try { + decoder.finish() + } catch (error: unknown) { + result = output.failure(logs, { kind: 'worker-exit', message: messageOf(error) }) + } + if (cleanupError !== undefined) { + result = output.failure(logs, { kind: 'worker-exit', message: `E2B runtime cleanup failed: ${messageOf(cleanupError)}` }) + } + const final = typeof result === 'function' ? result() : result + finishResolve() + resolve(final) + }) + } + + const sendReply = (message: unknown): void => { + if (settled) return + stdin.write(encodeE2BFrame(message), (error?: Error | null) => { + if (error !== undefined && error !== null) { + finish(() => output.failure(logs, { kind: 'worker-exit', message: `E2B runtime bridge write failed: ${error.message}` })) + } + }) + } + + /* jscpd:ignore-start -- Host binding resolution mirrors worker semantics over a different transport. */ + const onCall = (message: CallMessage): void => { + if (answered.has(message.id)) return + answered.add(message.id) + const functions = bindings.get(message.global)?.functions + const fn = functions !== undefined && Object.hasOwn(functions, message.name) ? functions[message.name] : undefined + if (typeof fn !== 'function') { + sendReply({ type: 'reply', id: message.id, ok: false, message: `unknown binding ${JSON.stringify(`${message.global}.${message.name}`)}` }) + return + } + const args = decodeWorkerJson(message.args) + if (args === undefined) { + sendReply({ type: 'reply', id: message.id, ok: false, message: 'binding arguments must be lossless JSON' }) + return + } + void (async () => { + try { + const resolved = await fn(args) + let value: CodeJsonValue | undefined + try { + value = snapshotJsonValue(resolved) + } catch { + value = undefined + } + if (value === undefined) { + sendReply({ type: 'reply', id: message.id, ok: false, message: 'binding resolution must be lossless JSON' }) + } else { + sendReply({ type: 'reply', id: message.id, ok: true, value: encodeWorkerJson(value) }) + } + } catch (error: unknown) { + sendReply({ type: 'reply', id: message.id, ok: false, message: messageOf(error) }) + } + })() + } + /* jscpd:ignore-end */ + + const onMessage = (raw: unknown): void => { + if (settled) return + const message = parseRunnerMessage(raw) + if (message === undefined) return + if (message.type === 'log') { + if (!output.admit(message.text, logs)) finish(output.limit([...logs, message.text])) + return + } + if (message.type === 'output-limit') { + finish(output.limit(logs)) + return + } + if (message.type === 'call') { + onCall(message) + return + } + if (message.error !== undefined) { + finish(() => output.failure(logs, message.error as CodeRunFailure)) + } else if (message.value === undefined) { + finish(() => output.success(logs)) + } else { + const value = decodeWorkerJson(message.value) + if (value === undefined) finish(() => output.failure(logs, { kind: 'invalid-output', message: 'program completion must be lossless JSON' })) + else finish(() => output.success(logs, value)) + } + } + + stdout.on('data', (chunk: Buffer) => { + if (settled) return + try { + for (const frame of decoder.push(chunk.toString('utf8'))) onMessage(frame) + } catch (error: unknown) { + finish(() => output.failure(logs, { kind: 'worker-exit', message: `E2B runtime bridge failed: ${messageOf(error)}` })) + } + }) + stdout.on('error', (error: Error) => { + finish(() => output.failure(logs, { kind: 'worker-exit', message: `E2B runtime stdout failed: ${error.message}` })) + }) + stdin.on('error', (error: Error) => { + finish(() => output.failure(logs, { kind: 'worker-exit', message: `E2B runtime stdin failed: ${error.message}` })) + }) + void handle.done.then( + () => { + if (!settled) { + const stderr = handle.collected.stderr?.readFrom(0).text.trim() + finish(() => output.failure(logs, { kind: 'worker-exit', message: stderr === undefined || stderr === '' ? 'E2B runtime exited before completing' : `E2B runtime exited before completing: ${stderr}` })) + } + }, + (error: unknown) => { + finish(() => output.failure(logs, { kind: 'worker-exit', message: `E2B runtime spawn failed: ${messageOf(error)}` })) + }, + ) + + const onAbort = (): void => { + finish(() => output.failure(logs, { kind: 'abort', message: String(request.signal?.reason) })) + } + request.signal?.addEventListener('abort', onAbort, { once: true }) + wallTimer.current = setTimeout(() => { + finish(() => output.failure(logs, { kind: 'timeout', message: `wall-clock ceiling reached (${this.config.maxWallMs}ms)` })) + }, this.config.maxWallMs) + this.live.add(live) + if (request.signal?.aborted === true) { + onAbort() + return + } + sendReply({ + type: 'boot', + code, + namespaces: [...bindings].map(([global, namespace]) => ({ + global, + names: Object.keys(namespace.functions), + ...namespace.errorClass === undefined ? {} : { errorClass: namespace.errorClass }, + })), + computeMs: this.config.computeMs, + maxOutputBytes: this.config.maxOutputBytes, + maxOldGenerationSizeMb: this.config.maxOldGenerationSizeMb, + }) + }) + } + + /* jscpd:ignore-start -- Code-runtime backends share the service lifecycle but own different child identities. */ + private async teardown(): Promise { + this.disposed = true + const runs = [...this.live] + for (const run of runs) run.settle({ kind: 'abort', message: 'runtime disposed' }) + await Promise.all(runs.map(run => run.finished)) + } + /* jscpd:ignore-end */ +} + +export default E2BCodeRuntime diff --git a/packages/code-runtime/code-runtime-e2b/src/invariant.ts b/packages/code-runtime/code-runtime-e2b/src/invariant.ts new file mode 100644 index 0000000000..4112cdc76b --- /dev/null +++ b/packages/code-runtime/code-runtime-e2b/src/invariant.ts @@ -0,0 +1,20 @@ +/** Package-owned invariant companion for `@deepseek-ai/dsh-code-runtime-e2b`. */ + +/* jscpd:ignore-start */ +import type { Context } from 'cordis' +import type { InvariantInstaller } from '@deepseek-ai/dsh-invariants' + +const PACKAGE_NAME = '@deepseek-ai/dsh-code-runtime-e2b' + +/** Cordis companion plugin name. */ +export const name = 'code-runtime-e2b-invariant' +/** Service required before the companion can reserve package ownership. */ +export const inject = ['invariants'] + +/** No runtime invariant: the service owns every one-shot remote run. */ +const install: InvariantInstaller = () => {} + +/** Register this package's invariant companion. */ +export const apply = (ctx: Context): Promise<() => void> => + Promise.resolve(ctx.invariants.register(PACKAGE_NAME, install)) +/* jscpd:ignore-end */ diff --git a/packages/code-runtime/code-runtime-e2b/src/runner-source.ts b/packages/code-runtime/code-runtime-e2b/src/runner-source.ts new file mode 100644 index 0000000000..b0355f20ce --- /dev/null +++ b/packages/code-runtime/code-runtime-e2b/src/runner-source.ts @@ -0,0 +1,460 @@ +/** Dependency-free remote code runner installed inside the E2B sandbox. */ + +/** Node program that runs one model program in a fresh remote worker thread. */ +export const CODE_RUNNER_SOURCE = String.raw`import { Buffer } from 'node:buffer' +import { inspect } from 'node:util' +import { Worker, isMainThread, parentPort, workerData } from 'node:worker_threads' +import { createInterface } from 'node:readline' + +const emitFrame = message => { + process.stdout.write(Buffer.from(JSON.stringify(message)).toString('base64') + '\n') +} + +const parseFrame = line => JSON.parse(Buffer.from(line, 'base64').toString('utf8')) + +if (isMainThread) { + const input = createInterface({ input: process.stdin, crlfDelay: Infinity }) + let worker + let finished = false + let computeTimer + const finish = message => { + if (finished) return + finished = true + clearInterval(computeTimer) + emitFrame(message) + const current = worker + worker = undefined + Promise.resolve(current ? current.terminate() : undefined).finally(() => { + input.close() + process.stdin.destroy() + }) + } + input.on('line', line => { + let message + try { + message = parseFrame(line) + } catch (error) { + process.stderr.write('code-runtime-e2b frame error: ' + String(error) + '\n') + finish({ type: 'done', error: { kind: 'worker-exit', message: 'remote runner received a malformed frame' } }) + return + } + if (!worker) { + if (!message || message.type !== 'boot' || typeof message.code !== 'string' || !Array.isArray(message.namespaces)) { + finish({ type: 'done', error: { kind: 'worker-exit', message: 'remote runner received an invalid boot frame' } }) + return + } + worker = new Worker(new URL(import.meta.url), { + workerData: message, + env: {}, + stdout: true, + stderr: true, + resourceLimits: { maxOldGenerationSizeMb: message.maxOldGenerationSizeMb }, + }) + worker.stdout.on('data', data => { emitFrame({ type: 'log', text: data.toString('utf8') }) }) + worker.stderr.on('data', data => { emitFrame({ type: 'log', text: data.toString('utf8') }) }) + worker.on('message', raw => { + if (!raw || typeof raw !== 'object') return + if (raw.type === 'call' && typeof raw.id === 'number' && typeof raw.global === 'string' && typeof raw.name === 'string' && Array.isArray(raw.args)) { + emitFrame({ type: 'call', id: raw.id, global: raw.global, name: raw.name, args: raw.args }) + } else if (raw.type === 'log' && typeof raw.text === 'string') { + emitFrame({ type: 'log', text: raw.text }) + } else if (raw.type === 'output-limit') { + finish({ type: 'output-limit' }) + } else if (raw.type === 'done') { + if (raw.error && typeof raw.error === 'object' && typeof raw.error.kind === 'string' && typeof raw.error.message === 'string') { + finish({ type: 'done', error: { kind: raw.error.kind, message: raw.error.message } }) + } else if (raw.value === undefined || Array.isArray(raw.value)) { + finish({ type: 'done', ...(raw.value === undefined ? {} : { value: raw.value }) }) + } + } + }) + worker.on('error', error => { + finish({ type: 'done', error: { kind: 'worker-exit', message: 'remote worker error: ' + error.message } }) + }) + worker.on('exit', code => { + if (!finished) finish({ type: 'done', error: { kind: 'worker-exit', message: 'remote worker exited with code ' + code + ' before completing' } }) + }) + computeTimer = setInterval(() => { + if (!worker) return + if (worker.performance.eventLoopUtilization().active > message.computeMs) { + finish({ type: 'done', error: { kind: 'timeout', message: 'compute budget exhausted (' + message.computeMs + 'ms busy)' } }) + } + }, 25) + return + } + if (message && message.type === 'reply' && typeof message.id === 'number' && typeof message.ok === 'boolean') { + worker.postMessage(message.ok + ? { type: 'reply', id: message.id, ok: true, value: message.value } + : { type: 'reply', id: message.id, ok: false, message: String(message.message) }) + } + }) + input.on('close', () => { if (worker && !finished) void worker.terminate() }) +} else { + const port = parentPort + if (!port) throw new Error('remote worker requires parentPort') + + const CapturedError = Error + const ArrayIsArray = Array.isArray + const ArrayPrototype = Array.prototype + const ObjectPrototype = Object.prototype + const ObjectCreate = Object.create + const ObjectDefineProperty = Object.defineProperty + const ObjectGetPrototypeOf = Object.getPrototypeOf + const ObjectHasOwn = Object.hasOwn + const ObjectKeys = Object.keys + const ObjectIs = Object.is + const ObjectPropertyIsEnumerable = Object.prototype.propertyIsEnumerable + const ReflectOwnKeys = Reflect.ownKeys + const ReflectApply = Reflect.apply + const NumberIsFinite = Number.isFinite + const NumberIsSafeInteger = Number.isSafeInteger + const PromiseCtor = Promise + const PromiseReject = Promise.reject + const QueueMicrotask = queueMicrotask + const BufferByteLength = Buffer.byteLength + const SetCtor = Set + const SetAdd = Set.prototype.add + const SetDelete = Set.prototype.delete + const SetHas = Set.prototype.has + const MapDelete = Map.prototype.delete + const MapGet = Map.prototype.get + const MapSet = Map.prototype.set + const ArrayJoin = Array.prototype.join + const ArrayPop = Array.prototype.pop + const StringCharCodeAt = String.prototype.charCodeAt + const StringSlice = String.prototype.slice + const JSONStringify = JSON.stringify + const StringValue = String + + const define = (target, key, value) => { + const descriptor = ObjectCreate(null) + descriptor.value = value + descriptor.enumerable = true + descriptor.configurable = true + descriptor.writable = true + ObjectDefineProperty(target, key, descriptor) + } + const append = (target, value) => { define(target, target.length, value) } + const pop = target => ReflectApply(ArrayPop, target, []) + const setAdd = (target, value) => { ReflectApply(SetAdd, target, [value]) } + const setDelete = (target, value) => { ReflectApply(SetDelete, target, [value]) } + const setHas = (target, value) => ReflectApply(SetHas, target, [value]) + const mapDelete = (target, key) => { ReflectApply(MapDelete, target, [key]) } + const mapGet = (target, key) => ReflectApply(MapGet, target, [key]) + const mapSet = (target, key, value) => { ReflectApply(MapSet, target, [key, value]) } + const plainObject = value => { + const prototype = ObjectGetPrototypeOf(value) + return prototype === null || prototype === ObjectPrototype + } + const ownEnumerableStringKeys = value => { + const keys = ReflectOwnKeys(value) + for (let index = 0; index < keys.length; index++) { + const key = keys[index] + if (typeof key !== 'string' || !ReflectApply(ObjectPropertyIsEnumerable, value, [key])) return undefined + } + return keys + } + const assign = (destination, value) => { + if (destination.kind === 'root') destination.holder.value = value + else define(destination.target, destination.key, value) + } + const snapshot = input => { + const active = new SetCtor() + const holder = ObjectCreate(null) + const tasks = [{ kind: 'visit', value: input, destination: { kind: 'root', holder } }] + while (tasks.length) { + const task = pop(tasks) + if (task.kind === 'leave') { setDelete(active, task.source); continue } + const candidate = task.value + if (candidate === null || typeof candidate === 'boolean' || typeof candidate === 'string') { + assign(task.destination, candidate); continue + } + if (typeof candidate === 'number') { + if (!NumberIsFinite(candidate) || ObjectIs(candidate, -0)) return undefined + assign(task.destination, candidate); continue + } + if (typeof candidate !== 'object' || setHas(active, candidate)) return undefined + if (ArrayIsArray(candidate)) { + if (ObjectGetPrototypeOf(candidate) !== ArrayPrototype || ReflectOwnKeys(candidate).length !== candidate.length + 1) return undefined + const target = [] + assign(task.destination, target) + setAdd(active, candidate) + append(tasks, { kind: 'leave', source: candidate }) + for (let index = candidate.length - 1; index >= 0; index--) { + if (!ObjectHasOwn(candidate, index)) return undefined + append(tasks, { kind: 'visit', value: candidate[index], destination: { kind: 'slot', target, key: index } }) + } + continue + } + if (!plainObject(candidate)) return undefined + const keys = ownEnumerableStringKeys(candidate) + if (!keys) return undefined + const target = {} + assign(task.destination, target) + setAdd(active, candidate) + append(tasks, { kind: 'leave', source: candidate }) + for (let index = keys.length - 1; index >= 0; index--) { + const key = keys[index] + append(tasks, { kind: 'visit', value: candidate[key], destination: { kind: 'slot', target, key } }) + } + } + return holder.value + } + const encodeWire = value => { + const wire = [] + const pending = [value] + while (pending.length) { + const current = pop(pending) + if (current === null || typeof current === 'boolean' || typeof current === 'number' || typeof current === 'string') { + append(wire, current); continue + } + if (ArrayIsArray(current)) { + append(wire, { kind: 'array', length: current.length }) + for (let index = current.length - 1; index >= 0; index--) append(pending, current[index]) + } else { + const keys = ObjectKeys(current) + append(wire, { kind: 'object', keys }) + for (let index = keys.length - 1; index >= 0; index--) append(pending, current[keys[index]]) + } + } + return wire + } + const decodeWire = wire => { + if (!ArrayIsArray(wire) || wire.length === 0) return undefined + const frames = [] + let root + let assigned = false + const attach = value => { + const parent = frames[frames.length - 1] + if (!parent) { + if (assigned) return false + root = value; assigned = true; return true + } + if (parent.kind === 'array') append(parent.target, value) + else define(parent.target, parent.keys[parent.index], value) + parent.index += 1 + return true + } + for (let tokenIndex = 0; tokenIndex < wire.length; tokenIndex++) { + const token = wire[tokenIndex] + let value + let frame + if (token === null || typeof token === 'boolean' || typeof token === 'string') value = token + else if (typeof token === 'number') { + if (!NumberIsFinite(token) || ObjectIs(token, -0)) return undefined + value = token + } else { + if (!plainObject(token)) return undefined + const keys = ownEnumerableStringKeys(token) + if (!keys || keys.length !== 2 || keys[0] !== 'kind') return undefined + if (token.kind === 'array' && keys[1] === 'length' && NumberIsSafeInteger(token.length) && token.length >= 0) { + value = [] + if (token.length > wire.length - tokenIndex - 1) return undefined + if (token.length) frame = { kind: 'array', target: value, length: token.length, index: 0 } + } else if (token.kind === 'object' && keys[1] === 'keys' && ArrayIsArray(token.keys)) { + const unique = new SetCtor() + const objectKeys = [] + for (const key of token.keys) { + if (typeof key !== 'string' || setHas(unique, key)) return undefined + setAdd(unique, key); append(objectKeys, key) + } + if (objectKeys.length > wire.length - tokenIndex - 1) return undefined + value = {} + if (objectKeys.length) frame = { kind: 'object', target: value, keys: objectKeys, index: 0 } + } else return undefined + } + if (!attach(value)) return undefined + if (frame) append(frames, frame) + while (frames.length) { + const current = frames[frames.length - 1] + const length = current.kind === 'array' ? current.length : current.keys.length + if (current.index < length) break + pop(frames) + } + } + return frames.length === 0 ? root : undefined + } + const byteLength = text => ReflectApply(BufferByteLength, Buffer, [text]) + const jsonStringBytes = text => byteLength(JSONStringify(text)) + const jsonValueBytes = value => { + let bytes = 0 + const tasks = [{ kind: 'value', value }] + while (tasks.length) { + const task = pop(tasks) + if (task.kind === 'separator') { bytes += 1; continue } + if (task.kind === 'key') { bytes += jsonStringBytes(task.value) + 1; continue } + const current = task.value + if (current === null) bytes += 4 + else if (typeof current === 'string') bytes += jsonStringBytes(current) + else if (typeof current === 'number' || typeof current === 'boolean') bytes += byteLength(StringValue(current)) + else if (ArrayIsArray(current)) { + bytes += 2 + for (let index = current.length - 1; index >= 0; index--) { + append(tasks, { kind: 'value', value: current[index] }) + if (index > 0) append(tasks, { kind: 'separator' }) + } + } else { + bytes += 2 + const keys = ObjectKeys(current) + for (let index = keys.length - 1; index >= 0; index--) { + const key = keys[index] + append(tasks, { kind: 'value', value: current[key] }) + append(tasks, { kind: 'key', value: key }) + if (index > 0) append(tasks, { kind: 'separator' }) + } + } + } + return bytes + } + const truncate = (text, available) => { + if (available < 2) return '' + let result = '' + let bytes = 2 + let index = 0 + while (index < text.length) { + const first = ReflectApply(StringCharCodeAt, text, [index]) + let end = index + 1 + if (first >= 0xd800 && first <= 0xdbff && end < text.length) { + const second = ReflectApply(StringCharCodeAt, text, [end]) + if (second >= 0xdc00 && second <= 0xdfff) end += 1 + } + const character = ReflectApply(StringSlice, text, [index, end]) + const cost = jsonStringBytes(character) - 2 + if (bytes + cost > available) break + bytes += cost + result += character + index = end + } + return result + } + let logBytes = 2 + let logEntries = 0 + let limited = false + const pushLog = text => { + if (limited) return + const separator = logEntries > 0 ? 1 : 0 + const available = workerData.maxOutputBytes - logBytes - separator + const cost = jsonStringBytes(text) + if (cost > available) { + const prefix = truncate(text, available) + if (prefix) { + logBytes += jsonStringBytes(prefix) + separator + logEntries += 1 + port.postMessage({ type: 'log', text: prefix }) + } + limited = true + port.postMessage({ type: 'output-limit' }) + return + } + logBytes += cost + separator + logEntries += 1 + port.postMessage({ type: 'log', text }) + } + const originalStdout = process.stdout.write + const originalStderr = process.stderr.write + process.stdout.write = (chunk, ...rest) => { + pushLog(typeof chunk === 'string' ? chunk : StringValue(chunk)) + let callback + for (let index = 0; index < rest.length; index++) { + if (typeof rest[index] === 'function') { callback = rest[index]; break } + } + if (callback) QueueMicrotask(() => { callback(null) }) + return true + } + process.stderr.write = process.stdout.write + const consoleShim = ObjectCreate(null) + for (const level of ['log', 'info', 'warn', 'error', 'debug']) { + define(consoleShim, level, (...args) => { + const rendered = [] + for (let index = 0; index < args.length; index++) { + const value = args[index] + append(rendered, typeof value === 'string' ? value : inspect(value, { depth: 4, maxArrayLength: 100, maxStringLength: 10000 })) + } + pushLog(ReflectApply(ArrayJoin, rendered, [' '])) + }) + } + const pending = new Map() + let nextId = 1 + const errorClasses = new Map() + for (const namespace of workerData.namespaces) { + if (!namespace.errorClass) continue + const descriptor = namespace.errorClass + mapSet(errorClasses, namespace.global, class BindingCallError extends CapturedError { + constructor(memberName, message) { + super(message) + ObjectDefineProperty(this, 'name', { value: descriptor.name, enumerable: true }) + ObjectDefineProperty(this, descriptor.memberNameProperty, { value: memberName, enumerable: true }) + } + }) + } + port.on('message', message => { + if (!message || message.type !== 'reply' || typeof message.id !== 'number') return + const entry = mapGet(pending, message.id) + if (!entry) return + mapDelete(pending, message.id) + if (!message.ok) { entry.reject(new CapturedError(StringValue(message.message))); return } + const value = decodeWire(message.value) + if (value === undefined) entry.reject(new CapturedError('binding resolution must be lossless JSON')) + else entry.resolve(value) + }) + const namespaces = workerData.namespaces.map(namespace => { + const target = ObjectCreate(null) + const ErrorClass = mapGet(errorClasses, namespace.global) + for (const name of namespace.names) { + define(target, name, args => { + const detached = snapshot(args) + if (detached === undefined) { + return ReflectApply(PromiseReject, PromiseCtor, [ErrorClass ? new ErrorClass(name, 'binding arguments must be lossless JSON') : new CapturedError('binding arguments must be lossless JSON')]) + } + return new PromiseCtor((resolve, reject) => { + const id = nextId++ + mapSet(pending, id, { + resolve, + reject: error => { reject(ErrorClass ? new ErrorClass(name, error.message) : error) }, + }) + port.postMessage({ type: 'call', id, global: namespace.global, name, args: encodeWire(detached) }) + }) + }) + } + return target + }) + const errorClassNames = [] + const errorClassValues = [] + for (const namespace of workerData.namespaces) { + if (!namespace.errorClass) continue + append(errorClassNames, namespace.errorClass.name) + append(errorClassValues, mapGet(errorClasses, namespace.global)) + } + const AsyncFunction = ObjectGetPrototypeOf(async function () {}).constructor + try { + const fn = new AsyncFunction(...workerData.namespaces.map(value => value.global), ...errorClassNames, 'console', '"use strict";\n' + workerData.code) + const value = await fn(...namespaces, ...errorClassValues, consoleShim) + if (!limited) { + if (value === undefined) port.postMessage({ type: 'done' }) + else { + const detached = snapshot(value) + if (detached === undefined) { + const message = 'program completion must be lossless JSON' + if (jsonStringBytes(message) > workerData.maxOutputBytes - logBytes) port.postMessage({ type: 'output-limit' }) + else port.postMessage({ type: 'done', error: { kind: 'invalid-output', message } }) + } else if (jsonValueBytes(detached) > workerData.maxOutputBytes - logBytes) { + port.postMessage({ type: 'output-limit' }) + } else { + port.postMessage({ type: 'done', value: encodeWire(detached) }) + } + } + } + } catch (error) { + if (!limited) { + let message + try { message = error instanceof CapturedError ? error.stack || error.message : StringValue(error) } + catch { message = 'program threw an unrenderable value' } + if (jsonStringBytes(message) > workerData.maxOutputBytes - logBytes) port.postMessage({ type: 'output-limit' }) + else port.postMessage({ type: 'done', error: { kind: 'exception', message } }) + } + } finally { + process.stdout.write = originalStdout + process.stderr.write = originalStderr + } +} +` diff --git a/packages/code-runtime/code-runtime-e2b/tests/runtime.spec.ts b/packages/code-runtime/code-runtime-e2b/tests/runtime.spec.ts new file mode 100644 index 0000000000..ff1c5dd84a --- /dev/null +++ b/packages/code-runtime/code-runtime-e2b/tests/runtime.spec.ts @@ -0,0 +1,460 @@ +import { PassThrough, Writable } from 'node:stream' +import { Context } from 'cordis' +import { describe, expect, it, vi } from 'vitest' +import type { Sandbox } from '@deepseek-ai/dsh-e2b' +import { + E2BFrameDecoder, + encodeE2BFrame, +} from '@deepseek-ai/dsh-e2b' +import type E2BSandboxService from '@deepseek-ai/dsh-e2b' +import type { + SubprocessHandle, + SubprocessOutcome, + SubprocessSpawnSpec, +} from '@deepseek-ai/dsh-subprocess' +import E2BSubprocessService from '@deepseek-ai/dsh-subprocess-e2b' +import { + encodeWorkerJson, +} from '@deepseek-ai/dsh-code-runtime-worker' +import E2BCodeRuntime from '@deepseek-ai/dsh-code-runtime-e2b' +import * as E2BCodeRuntimeInvariant from '../src/invariant.ts' +import { CODE_RUNNER_SOURCE } from '../src/runner-source.ts' +import InvariantService from '@deepseek-ai/dsh-invariants' + +class FakeHandle implements SubprocessHandle { + readonly pid = 123 + readonly stdin: Writable | undefined + readonly stdout: PassThrough | undefined + readonly stderr = undefined + readonly collected: SubprocessHandle['collected'] + readonly done: Promise + readonly writes: unknown[] = [] + readonly result = Promise.withResolvers() + terminated = 0 + waitCalls = 0 + private readonly decoder = new E2BFrameDecoder(10_000_000) + private readonly waitError: Error | undefined + private settled = false + + constructor( + private readonly onMessage: (message: unknown, handle: FakeHandle) => void = () => {}, + options: { stdin?: boolean; stdout?: boolean; stderr?: string; writeError?: Error; waitError?: Error } = {}, + ) { + this.waitError = options.waitError + this.stdin = options.stdin === false + ? undefined + : options.writeError === undefined + ? new PassThrough() + : new Writable({ write: (_chunk, _encoding, callback) => { callback(options.writeError) } }) + this.stdout = options.stdout === false ? undefined : new PassThrough() + this.collected = options.stderr === undefined + ? {} + : { stderr: { readFrom: () => ({ text: options.stderr as string, nextOffset: 0, lossy: false }) } } + this.done = this.result.promise + this.stdin?.on('data', (chunk: Buffer) => { + for (const message of this.decoder.push(chunk.toString('ascii'))) { + this.writes.push(message) + this.onMessage(message, this) + } + }) + } + + emit(message: unknown): void { + this.stdout?.write(encodeE2BFrame(message)) + } + + emitRaw(text: string): void { + this.stdout?.write(text) + } + + exit(outcome: SubprocessOutcome = { exitCode: 0, signal: null }): void { + if (this.settled) return + this.settled = true + this.stdout?.end() + this.result.resolve(outcome) + } + + crash(error: unknown): void { + if (this.settled) return + this.settled = true + this.stdout?.end() + this.result.reject(error) + } + + terminate(): void { + this.terminated += 1 + this.exit({ exitCode: null, signal: 'SIGTERM' }) + } + + async waitForExit(): Promise { + this.waitCalls += 1 + if (this.waitError !== undefined) throw this.waitError + return true + } +} + +interface RuntimeFixture { + ctx: Context + fiber: Awaited> + runtime: E2BCodeRuntime + sandbox: Sandbox + spawn: ReturnType SubprocessHandle>> + write: ReturnType + run: ReturnType +} + +async function setup( + handles: FakeHandle[] = [], + config: Record = {}, + sandboxOverrides: Partial = {}, + getSandbox?: () => Promise, +): Promise { + const write = vi.fn().mockResolvedValue([]) + const run = vi.fn().mockImplementation(async (command: string) => ({ + exitCode: 0, + stdout: command.startsWith('command -v') ? '/usr/bin/node\n' : '', + stderr: '', + })) + const sandbox = { + files: { write }, + commands: { run }, + ...sandboxOverrides, + } as unknown as Sandbox + const e2b = { + cwd: '/workspace', + runtimeRoot: '/workspace/.dsh-e2b', + getSandbox: getSandbox ?? (async () => sandbox), + } as unknown as E2BSandboxService + const spawn = vi.fn<(spec: SubprocessSpawnSpec) => SubprocessHandle>(() => { + const handle = handles.shift() + if (handle === undefined) throw new Error('no fake handle queued') + return handle + }) + const subprocess = Object.create(E2BSubprocessService.prototype) as E2BSubprocessService + Object.defineProperty(subprocess, 'spawn', { value: spawn }) + const ctx = new Context() + ctx.provide('e2b', e2b) + ctx.provide('subprocess', subprocess) + const fiber = await ctx.plugin(E2BCodeRuntime, config) + return { ctx, fiber, runtime: ctx.codeRuntime as E2BCodeRuntime, sandbox, spawn, write, run } +} + +function request(program = 'return 1') { + return { program, bindings: [] } +} + +describe('E2BCodeRuntime', () => { + it('prepares the remote runner and returns logs and a lossless completion', async () => { + const handle = new FakeHandle((message, current) => { + if ((message as { type?: string }).type !== 'boot') return + current.emit({ type: 'log', text: 'remote 你好' }) + current.emitRaw( + encodeE2BFrame({ type: 'done', value: encodeWorkerJson({ answer: 42 }) }) + + encodeE2BFrame({ type: 'log', text: 'ignored after done' }), + ) + current.emit({ type: 'log', text: 'also ignored after done' }) + }) + const fixture = await setup([handle]) + + await expect(fixture.runtime.run(request('const answer: number = 42; return { answer }'))) + .resolves.toEqual({ logs: ['remote 你好'], value: { answer: 42 } }) + expect(fixture.runtime.language).toBe('typescript') + expect(fixture.runtime.isolation).toBe('container') + expect(fixture.write).toHaveBeenCalledWith([{ path: '/workspace/.dsh-e2b/code-runtime-runner.mjs', data: CODE_RUNNER_SOURCE }]) + expect(fixture.run).toHaveBeenCalledWith("chmod 600 -- '/workspace/.dsh-e2b/code-runtime-runner.mjs'") + expect(fixture.spawn).toHaveBeenCalledWith(expect.objectContaining({ + argv: ['/usr/bin/node', '/workspace/.dsh-e2b/code-runtime-runner.mjs'], + cwd: '/workspace', + env: {}, + })) + expect(handle.terminated).toBe(1) + expect(handle.waitCalls).toBe(1) + await fixture.fiber.dispose() + }) + + it('bridges binding success, host rejection, unknown members, and invalid values', async () => { + const replies: unknown[] = [] + const handle = new FakeHandle((message, current) => { + const record = message as { type?: string; id?: number; ok?: boolean } + if (record.type === 'boot') { + current.emit({ type: 'call', id: 1, global: 'bridge', name: 'double', args: encodeWorkerJson({ value: 4 }) }) + current.emit({ type: 'call', id: 2, global: 'bridge', name: 'fail', args: encodeWorkerJson(null) }) + current.emit({ type: 'call', id: 3, global: 'bridge', name: 'missing', args: encodeWorkerJson(null) }) + current.emit({ type: 'call', id: 4, global: 'bridge', name: 'double', args: [] }) + current.emit({ type: 'call', id: 5, global: 'bridge', name: 'invalid', args: encodeWorkerJson(null) }) + current.emit({ type: 'call', id: 6, global: 'bridge', name: 'throwing', args: encodeWorkerJson(null) }) + current.emit({ type: 'call', id: 1, global: 'bridge', name: 'double', args: encodeWorkerJson({ value: 99 }) }) + return + } + if (record.type === 'reply') { + replies.push(message) + if (replies.length === 6) current.emit({ type: 'done', value: encodeWorkerJson('done') }) + } + }) + const fixture = await setup([handle]) + const result = await fixture.runtime.run({ + program: 'return await bridge.double({ value: 4 })', + bindings: [ + { + global: 'bridge', + errorClass: { name: 'BridgeError', memberNameProperty: 'member' }, + functions: { + double: async args => (args as { value: number }).value * 2, + fail: async () => { throw 'nope' }, + invalid: (async () => undefined) as never, + throwing: async () => Object.defineProperty({}, 'value', { + enumerable: true, + get: () => { throw new Error('getter failed') }, + }), + }, + }, + { global: 'plain', functions: {} }, + ], + }) + + expect(result).toEqual({ logs: [], value: 'done' }) + expect(replies.sort((left, right) => (left as { id: number }).id - (right as { id: number }).id)).toEqual([ + { type: 'reply', id: 1, ok: true, value: encodeWorkerJson(8) }, + { type: 'reply', id: 2, ok: false, message: 'nope' }, + { type: 'reply', id: 3, ok: false, message: 'unknown binding "bridge.missing"' }, + { type: 'reply', id: 4, ok: false, message: 'binding arguments must be lossless JSON' }, + { type: 'reply', id: 5, ok: false, message: 'binding resolution must be lossless JSON' }, + { type: 'reply', id: 6, ok: false, message: 'binding resolution must be lossless JSON' }, + ]) + await fixture.fiber.dispose() + }) + + it('ignores malformed runner traffic and classifies terminal runner messages', async () => { + const ignored = [ + null, 1, {}, { type: 'log' }, { type: 'call' }, + { type: 'call', id: 0, global: 'x', name: 'y', args: [] }, + { type: 'call', id: 1, global: 1, name: 'y', args: [] }, + { type: 'call', id: 1, global: 'x', name: 1, args: [] }, + { type: 'call', id: 1, global: 'x', name: 'y', args: {} }, + { type: 'done', error: null }, + { type: 'done', error: { kind: 'invented', message: 'x' } }, + { type: 'done', error: { kind: 'exception', message: 1 } }, + ] + const handles = [ + new FakeHandle((message, current) => { + if ((message as { type?: string }).type !== 'boot') return + for (const item of ignored) current.emit(item) + current.emit({ type: 'done' }) + }), + new FakeHandle((message, current) => { + if ((message as { type?: string }).type === 'boot') current.emit({ type: 'done', error: { kind: 'exception', message: 'boom' } }) + }), + new FakeHandle((message, current) => { + if ((message as { type?: string }).type === 'boot') current.emit({ type: 'done', value: [] }) + }), + new FakeHandle((message, current) => { + if ((message as { type?: string }).type === 'boot') current.emit({ type: 'output-limit' }) + }), + ] + const fixture = await setup(handles, { maxOutputBytes: 64, maxFrameBytes: 128 }) + + await expect(fixture.runtime.run(request())).resolves.toEqual({ logs: [] }) + await expect(fixture.runtime.run(request())).resolves.toEqual({ logs: [], error: { kind: 'exception', message: 'boom' } }) + await expect(fixture.runtime.run(request())).resolves.toEqual({ logs: [], error: { kind: 'invalid-output', message: 'program completion must be lossless JSON' } }) + await expect(fixture.runtime.run(request())).resolves.toEqual({ logs: [], error: { kind: 'output-limit', message: 'outer output exceeded 64 bytes' } }) + await fixture.fiber.dispose() + }) + + it('enforces the host output ledger and catches malformed bridge output', async () => { + const handles = [ + new FakeHandle((message, current) => { + if ((message as { type?: string }).type === 'boot') current.emit({ type: 'log', text: 'x'.repeat(1_000) }) + }), + new FakeHandle((message, current) => { + if ((message as { type?: string }).type === 'boot') current.emitRaw('not-base64\n') + }), + new FakeHandle((message, current) => { + if ((message as { type?: string }).type === 'boot') current.emitRaw('é') + }), + new FakeHandle((message, current) => { + if ((message as { type?: string }).type === 'boot') current.stdout?.emit('error', new Error('stdout broke')) + }), + ] + const fixture = await setup(handles, { maxOutputBytes: 128, maxFrameBytes: 4_096 }) + + expect((await fixture.runtime.run(request())).error?.kind).toBe('output-limit') + const malformed = (await fixture.runtime.run(request())).error + expect(malformed?.kind).toBe('worker-exit') + expect(malformed?.message).toContain('bridge failed') + expect((await fixture.runtime.run(request())).error?.message).toContain('non-ASCII') + expect((await fixture.runtime.run(request())).error).toEqual({ kind: 'worker-exit', message: 'E2B runtime stdout failed: stdout broke' }) + await fixture.fiber.dispose() + }) + + it('contains stdin errors, process exits, spawn failures, and missing pipes', async () => { + const writeError = new FakeHandle(() => {}, { writeError: new Error('write callback broke') }) + const stdinError = new FakeHandle((message, current) => { + if ((message as { type?: string }).type === 'boot') current.stdin?.emit('error', new Error('stdin broke')) + }) + const earlyExit = new FakeHandle(() => {}, { stderr: 'remote diagnostic' }) + const quietExit = new FakeHandle() + const emptyStderrExit = new FakeHandle(() => {}, { stderr: '' }) + const spawnFailure = new FakeHandle() + const missingStdin = new FakeHandle(() => {}, { stdin: false }) + const missingStdout = new FakeHandle(() => {}, { stdout: false, waitError: new Error('missing-stream process query failed') }) + const truncated = new FakeHandle((message, current) => { + if ((message as { type?: string }).type === 'boot') { + current.emitRaw('YQ==') + setImmediate(() => { current.exit() }) + } + }) + const cleanupFailure = new FakeHandle((message, current) => { + if ((message as { type?: string }).type === 'boot') current.emit({ type: 'done' }) + }, { waitError: new Error('process query failed') }) + const fixture = await setup([ + writeError, stdinError, earlyExit, quietExit, emptyStderrExit, + spawnFailure, missingStdin, missingStdout, truncated, cleanupFailure, + ]) + + expect((await fixture.runtime.run(request())).error).toEqual({ kind: 'worker-exit', message: 'E2B runtime bridge write failed: write callback broke' }) + expect((await fixture.runtime.run(request())).error).toEqual({ kind: 'worker-exit', message: 'E2B runtime stdin failed: stdin broke' }) + setImmediate(() => { earlyExit.exit() }) + expect((await fixture.runtime.run(request())).error).toEqual({ kind: 'worker-exit', message: 'E2B runtime exited before completing: remote diagnostic' }) + setImmediate(() => { quietExit.exit() }) + expect((await fixture.runtime.run(request())).error).toEqual({ kind: 'worker-exit', message: 'E2B runtime exited before completing' }) + setImmediate(() => { emptyStderrExit.exit() }) + expect((await fixture.runtime.run(request())).error).toEqual({ kind: 'worker-exit', message: 'E2B runtime exited before completing' }) + setImmediate(() => { spawnFailure.crash('spawn rejected') }) + expect((await fixture.runtime.run(request())).error).toEqual({ kind: 'worker-exit', message: 'E2B runtime spawn failed: spawn rejected' }) + expect((await fixture.runtime.run(request())).error?.message).toContain('dropped a piped runtime stream') + expect((await fixture.runtime.run(request())).error).toEqual({ kind: 'worker-exit', message: 'E2B runtime cleanup failed: missing-stream process query failed' }) + expect(missingStdin.terminated).toBe(1) + expect(missingStdin.waitCalls).toBe(1) + expect(missingStdout.terminated).toBe(1) + expect(missingStdout.waitCalls).toBe(1) + expect((await fixture.runtime.run(request())).error).toEqual({ kind: 'worker-exit', message: 'E2B frame stream ended mid-frame' }) + expect((await fixture.runtime.run(request())).error).toEqual({ kind: 'worker-exit', message: 'E2B runtime cleanup failed: process query failed' }) + await fixture.fiber.dispose() + }) + + it('reports wall timeout, abort, pre-abort, type-strip failure, and disposal', async () => { + const timeout = new FakeHandle() + const abort = new FakeHandle() + const disposing = new FakeHandle() + const fixture = await setup([timeout, abort, disposing], { maxWallMs: 20 }) + + expect((await fixture.runtime.run(request())).error).toEqual({ kind: 'timeout', message: 'wall-clock ceiling reached (20ms)' }) + const controller = new AbortController() + const aborting = fixture.runtime.run({ ...request(), signal: controller.signal }) + controller.abort('stop') + expect((await aborting).error).toEqual({ kind: 'abort', message: 'stop' }) + expect((await fixture.runtime.run({ ...request(), signal: AbortSignal.abort('already') })).error) + .toEqual({ kind: 'abort', message: 'already' }) + expect((await fixture.runtime.run(request('enum E { A }'))).error?.kind).toBe('exception') + + const live = fixture.runtime.run(request()) + await new Promise(resolve => setImmediate(resolve)) + await fixture.fiber.dispose() + expect((await live).error).toEqual({ kind: 'abort', message: 'runtime disposed' }) + await expect(fixture.runtime.run(request())).rejects.toThrow('after disposal') + }) + + it('drops binding replies that settle after abort', async () => { + const controller = new AbortController() + const resolution = Promise.withResolvers() + const invoked = Promise.withResolvers() + const handle = new FakeHandle((message, current) => { + if ((message as { type?: string }).type === 'boot') { + current.emit({ type: 'call', id: 1, global: 'bridge', name: 'late', args: encodeWorkerJson(null) }) + } + }) + const fixture = await setup([handle]) + const running = fixture.runtime.run({ + program: 'return await bridge.late(null)', + bindings: [{ + global: 'bridge', + functions: { + late: async () => { + invoked.resolve(undefined) + return await resolution.promise + }, + }, + }], + signal: controller.signal, + }) + await invoked.promise + controller.abort('stop') + expect((await running).error).toEqual({ kind: 'abort', message: 'stop' }) + resolution.resolve('late') + await new Promise(resolve => setImmediate(resolve)) + expect(handle.writes).toHaveLength(1) + await fixture.fiber.dispose() + }) + + it('validates binding and runtime configuration before remote execution', async () => { + const fixture = await setup([]) + const invalidRequests = [ + { global: 'not-valid!', functions: {} }, + { global: 'await', functions: {} }, + { global: 'console', functions: {} }, + { global: 'same', functions: {} }, + { global: 'same', functions: {} }, + { global: 'ok', functions: {}, errorClass: { name: 'not-valid!', memberNameProperty: 'member' } }, + { global: 'ok', functions: {}, errorClass: { name: 'await', memberNameProperty: 'member' } }, + { global: 'Clash', functions: {}, errorClass: { name: 'Clash', memberNameProperty: 'member' } }, + { global: 'one', functions: {}, errorClass: { name: 'Err', memberNameProperty: 'member' } }, + { global: 'two', functions: {}, errorClass: { name: 'Err', memberNameProperty: 'member' } }, + { global: 'ok', functions: {}, errorClass: { name: 'Err', memberNameProperty: '' } }, + { global: 'ok', functions: {}, errorClass: { name: 'Err', memberNameProperty: 'message' } }, + ] + for (const bindings of [ + [invalidRequests[0]], [invalidRequests[1]], [invalidRequests[2]], + invalidRequests.slice(3, 5), [invalidRequests[5]], [invalidRequests[6]], + [invalidRequests[7]], invalidRequests.slice(8, 10), [invalidRequests[10]], [invalidRequests[11]], + ]) { + await expect(fixture.runtime.run({ program: 'return 1', bindings: bindings as never })).rejects.toThrow() + } + await fixture.fiber.dispose() + + for (const config of [ + { computeMs: 0 }, { computeMs: 1.5 }, { maxOutputBytes: 3 }, + { maxWallMs: 2_147_483_648 }, { maxFrameBytes: 10, maxOutputBytes: 20 }, + ]) { + const ctx = new Context() + const subprocess = Object.create(E2BSubprocessService.prototype) as E2BSubprocessService + ctx.provide('e2b', { getSandbox: async () => ({}) } as never) + ctx.provide('subprocess', subprocess) + await expect(ctx.plugin(E2BCodeRuntime, config)).rejects.toThrow() + } + + const wrong = new Context() + wrong.provide('e2b', { getSandbox: async () => ({}) } as never) + wrong.provide('subprocess', {} as never) + await expect(wrong.plugin(E2BCodeRuntime, {})).rejects.toThrow('dsh-subprocess-e2b') + }) + + it('turns asynchronous runtime preparation failure into a run result', async () => { + const sandbox = { + files: { write: vi.fn().mockRejectedValue(new Error('upload failed')) }, + commands: { run: vi.fn() }, + } as unknown as Sandbox + const fixture = await setup([], {}, sandbox) + expect((await fixture.runtime.run(request())).error).toEqual({ + kind: 'worker-exit', + message: 'E2B runtime setup failed: upload failed', + }) + await fixture.fiber.dispose() + }) + + it('returns disposal when remote preparation completes after teardown', async () => { + const gate = Promise.withResolvers() + const fixture = await setup([], {}, {}, () => gate.promise) + const running = fixture.runtime.run(request()) + await (fixture.runtime as unknown as { teardown(): Promise }).teardown() + gate.resolve(fixture.sandbox) + expect((await running).error).toEqual({ kind: 'abort', message: 'runtime disposed' }) + await fixture.fiber.dispose() + }) + + it('registers the package-owned invariant companion', async () => { + const ctx = new Context() + await ctx.plugin(InvariantService, { enabled: true }) + const fiber = await ctx.plugin(E2BCodeRuntimeInvariant).await() + await fiber.dispose() + }) +}) diff --git a/packages/code-runtime/code-runtime-e2b/tsconfig.json b/packages/code-runtime/code-runtime-e2b/tsconfig.json new file mode 100644 index 0000000000..4ddc3b5ffe --- /dev/null +++ b/packages/code-runtime/code-runtime-e2b/tsconfig.json @@ -0,0 +1,20 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": ["src"], + "references": [ + { "path": "../../../vendor/cosmokit" }, + { "path": "../../../vendor/cordis" }, + { "path": "../../../vendor/schemastery" }, + { "path": "../code-runtime" }, + { "path": "../code-runtime-worker" }, + { "path": "../../e2b/e2b" }, + { "path": "../../core/session" }, + { "path": "../../subprocess/subprocess-e2b" }, + { "path": "../../util/timeout" }, + { "path": "../../support/invariants" } + ] +} diff --git a/packages/code-runtime/code-runtime-worker/README.i18n.yaml b/packages/code-runtime/code-runtime-worker/README.i18n.yaml index 8d3568e82e..d3472a697e 100644 --- a/packages/code-runtime/code-runtime-worker/README.i18n.yaml +++ b/packages/code-runtime/code-runtime-worker/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/code-runtime/code-runtime-worker/README.md -README.md: 590b79dcd1bc322350060b55767b09c6305edacc -README.zh.md: 12c25f892bd20892cb47e593f16b6aadd9ffa84c +README.md: 4b14c6fad5d1e491faeb54c9cb0e4403c8e2d8dd +README.zh.md: c522917835b562cf7648d8bc78f0315b7c52d2d2 diff --git a/packages/code-runtime/code-runtime-worker/README.md b/packages/code-runtime/code-runtime-worker/README.md index 590b79dcd1..4b14c6fad5 100644 --- a/packages/code-runtime/code-runtime-worker/README.md +++ b/packages/code-runtime/code-runtime-worker/README.md @@ -32,7 +32,7 @@ Every field is validated and defaulted; `maxOutputBytes` is a safe integer of at ## The worker entry, unbuilt and built -Source mode loads erasable-only `src/worker.ts` through Node's native type stripping. Its transitive runtime closure contains only Node built-ins and relative source modules, so a fresh checkout never requires a sibling workspace package's unbuilt `lib/` export. The worker-local and session-owned JSON boundaries both flatten and rebuild validated values around the message port so application nesting never reaches structured clone. Built mode passes the sibling `lib/worker.cjs` as a filesystem path because pkg's VFS Worker hook expects CommonJS; the same path works under ordinary Node. The repository-wide requirement to exercise this published entry path belongs to the [testing policy](../../../docs/testing.md). +Source mode loads erasable-only `src/worker.ts` through Node's native type stripping. Its transitive runtime closure contains only Node built-ins and relative source modules, so a fresh checkout never requires a sibling workspace package's unbuilt `lib/` export. The worker-local JSON snapshotter is parity-tested against the session-owned canonical boundary; both sides flatten and rebuild validated values around the message port so application nesting never reaches structured clone. Built mode passes the sibling `lib/worker.cjs` as a filesystem path because pkg's VFS Worker hook expects CommonJS; the same path works under ordinary Node. `tests/built-lib.e2e.ts` pins the real load path required by [docs/testing.md](../../../docs/testing.md). The SDK surface is the default/named `WorkerCodeRuntime` class plus `Config`. The operational `./worker` subpath exists only as the packaged spawn entry; the wire protocol and bootstrap helpers are source-private implementation details. @@ -46,8 +46,8 @@ No direct invalidation; the named consumer owns any request-prefix changes. ## Known Limitations and Deferred Work -- **OS processes a program spawns survive termination** — `worker.terminate()` ends the thread only, weaker than bash-local's process-group kill; orphan cleanup is a deployment concern until a container backend exists. -- **Type-strip rides Node's experimental `stripTypeScriptTypes` API** — amaro or sucrase are the named drop-in replacements if the relied-on behavior shifts. +- **OS processes a program spawns survive this backend's termination** — `worker.terminate()` ends only the thread; deployments needing remote process-group cleanup can select the E2B backend, whose separate limitations still apply. +- **Type-strip rides Node's experimental `stripTypeScriptTypes` API** — the relied-on behavior is pinned by unit tests, with amaro/sucrase as named drop-in replacements if it shifts. - **`computeMs` expiry can overshoot by up to one poll interval** — busy time is sampled every 25 ms (an internal constant, deliberately not config). - **Programs get a five-method `console` shim** (`log`/`info`/`warn`/`error`/`debug`) — deliberately not Node's full console surface. - **Intermediate binding values have no byte cap** — a program can exhaust process or worker memory with a value that never becomes outer output. diff --git a/packages/code-runtime/code-runtime-worker/README.zh.md b/packages/code-runtime/code-runtime-worker/README.zh.md index 12c25f892b..c522917835 100644 --- a/packages/code-runtime/code-runtime-worker/README.zh.md +++ b/packages/code-runtime/code-runtime-worker/README.zh.md @@ -23,18 +23,18 @@ - **每次运行使用一个全新 worker,不设池化**:程序所在的世界会随 worker 一同终止,不会留下需要记录的跨运行状态,也无法发生状态泄漏;仅凭会话日志即可重建运行。 - **在执行上下文中,由宿主侧剥离类型**:程序会包裹在异步函数外壳中,通过 `node:module` 的 `stripTypeScriptTypes` 剥离类型(只支持可擦除语法;`enum`/namespace 会作为程序 `exception` 被拒绝,且不会启动 worker),再按字节位置切回原内容。之后程序作为 `AsyncFunction` 的函数体执行,因此顶层 `await`/`return` 可用。 - **端口把对端视为不可信**:模型代码能够访问 `parentPort` 并伪造通信,因此任何代码读取入站消息前,系统都会验证其形状并重新构建(`null`、原始值、无效类型和格式错误的载荷会被静默丢弃;伪造的额外字段绝不会被带入);宿主对每个调用 id 最多响应一次,只将绑定名称解析为自有属性(伪造的 `constructor` 无法沿原型链访问),丢弃结算后的回复,并验证每个绑定 resolve 值与完成值是否为无损 JSON。伪造的 `log`/`done` 消息无法绕过外层上限:宿主会再次验证,并统计每条获准日志以及完成值或诊断。worker 侧命名空间使用 null-prototype 和 `defineProperty`,因此形似 `__proto__` 的绑定名称只是普通键。 -- **绑定调用被拒绝时使用的异常类属于请求数据**:可选命名空间描述符会指定构造器全局变量,以及用于接收调用失败的成员名称的自有属性。worker 会创建并注入该真实类,使 `instanceof` 生效,同时无需硬编码 `tools` 或 `ToolCallError`;全局变量无效或冲突的声明会在启动 worker 前失败。失败路径使用模块捕获的错误 intrinsic 与属性定义 intrinsic,以及 null-prototype 描述符,因此模型之后的修改无法把被拒绝的绑定变成 worker 崩溃。 -- **两个独立预算,因为对端不可信**:`computeMs` 统计 worker 实际测得的忙碌时间(轮询 `worker.performance.eventLoopUtilization()`);热循环无法借助待完成的诱饵 dispatch 隐藏,程序等待慢工具时则不累计。`maxWallMs` 为忙碌时间无法观测的情况兜底(例如等待永远不会 resolve 的 promise)。二者最终都会调用 `worker.terminate()`,连同步热循环也能终止;堆溢出会表现为 worker 的 OOM 退出(`kind: 'worker-exit'`)。`maxWallMs` 在加载时会对照 `MAX_TIMER_DELAY_MS` 做范围校验:`setTimeout` 会把更长的延迟限制为 1 ms,仅有正数校验会放行一个在第一个 tick 就到期的上限。`computeMs` 不需要这道上界,因为它对照的是实测占用率,而不是喂给定时器。 -- **中间绑定值是完整 JSON**:绑定参数与 resolve 值会接受迭代式无损 JSON 验证。程序执行前,worker 会捕获自己 realm 中的普通容器原型身份,以及只用于外部 realm 的原生函数源码检查,因此构造器槽修改和用户编写的仿冒对象都无法改变容器分类。它还会捕获该 JSON 边界使用的每一个结构与计量 intrinsic,以无原型对象创建属性描述符,并绕过可变集合原型管理私有遍历状态;因此,模型对全局对象、原型方法或 `Object.prototype` 上形似描述符字段的修改,都无法改变验证、wire 传输或字节计量。值会展平为自身嵌套深度有界的前序 wire 值,供 structured clone 使用,并在另一侧迭代式重建。它们没有字节、JavaScript 调用栈或嵌套 structured-clone 深度上限,绝不会进入外层输出账本或模型上下文;上限仍来自提供方/执行器获取限制与进程/worker 内存。 -- **日志主动流入一个外层账本**:console/stdout/stderr 文本按产生顺序经端口传输,因此超时或被终止的程序仍会显示已经打印的内容。worker 会精确统计 JSON 字符串的字节数,并在发送完成值和异常诊断前,根据组合预算的剩余量预检;因此,抛出的百万字节 stack 会在 worker 边界变成固定的 `output-limit` 诊断。绕过补丁 stream 槽的原生写入会到达独立于完成端口的 pipe,因此宿主会针对这些字节和不可信伪造通信再次执行账本统计;在物化结果前,结算过程会持续进行有界 pipe 捕获,直到 worker 完成终止。`maxOutputBytes` 统计外层 `logs` 数组加完成值或失败消息载荷的 JSON 序列化;固定的 `CodeRunResult` 字段名、花括号、有界错误 kind 标签,以及后续呈现空白不计入这份可变载荷账本。未超过上限时会返回精确值;有损完成值属于 `invalid-output`,组合溢出属于 `output-limit`,不会用 inspected string 代替。失败会保留能容纳的已捕获前缀,之后按普通外层 `run_code` 落盘策略处理。 -- **空环境**:worker 使用 `env: {}` 和 `execArgv: []`,既不会获得环境变量中的凭据(比 spawn 命令的清理环境规则更严格),也不会继承 loader 标志。 -- **dispose(资源释放)时等待完全停稳**:清理会使进行中的运行以 `abort` 失败,并会等待每个 worker 退出后再完成。 +- **绑定 reject 类属于请求数据**:可选命名空间描述符会指定构造器全局变量,以及用于接收失败成员名称的自有属性。worker 会创建并注入该真实类,使 `instanceof` 生效,同时无需硬编码 `tools` 或 `ToolCallError`;全局变量无效或冲突的声明会在启动 worker 前失败。失败路径使用模块捕获的错误与属性定义 intrinsic,以及 null-prototype 描述符,因此模型之后的修改无法把被拒绝的绑定变成 worker 崩溃。 +- **两个独立预算,因为对端不可信**:`computeMs` 统计 worker 实际测得的忙碌时间(轮询 `worker.performance.eventLoopUtilization()`);热循环无法借助待完成的诱饵 dispatch 隐藏,程序等待慢工具时则不累计。`maxWallMs` 为忙碌时间无法观测的情况兜底(例如等待永远不会 resolve 的 promise)。二者最终都会调用 `worker.terminate()`,连同步热循环也能终止;堆溢出会表现为 worker 的 OOM 退出(`kind: 'worker-exit'`)。`maxWallMs` 在加载时会对照 `MAX_TIMER_DELAY_MS` 做范围校验:`setTimeout` 会把更长的延迟夹到 1 ms,仅有正数校验会放行一个在第一个 tick 就到期的上限。`computeMs` 不需要这道上界,因为它对照的是实测占用率,而不是喂给定时器。 +- **中间绑定值是完整 JSON**:绑定参数与 resolve 值会接受迭代式无损 JSON 验证。程序执行前,worker 会捕获自己 realm 中的普通容器原型身份,以及只用于外部 realm 的原生函数源码检查,因此构造器槽修改和用户编写的仿冒对象都无法改变容器分类。它还会捕获该 JSON 边界使用的每一个结构与计量 intrinsic,以无原型对象创建属性描述符,并绕过可变集合原型管理私有遍历状态;因此,模型对全局对象、原型方法或 `Object.prototype` 上形似描述符字段的修改,都无法改变验证、wire 传输或字节计量。值会展平为有深度上限的前序 wire 值,供 structured clone 使用,并在另一侧迭代式重建。它们没有字节、JavaScript 调用栈或嵌套 structured-clone 深度上限,绝不会进入外层输出账本或模型上下文;上限仍来自提供方/执行器获取限制与进程/worker 内存。 +- **日志主动流入一个外层账本**:console/stdout/stderr 文本按发送顺序穿过端口,因此超时或被终止的程序仍会显示已经打印的内容。worker 会按 JSON 字符串精确计费,并在发送完成值和异常诊断前,根据组合预算的剩余量预检;因此,抛出的百万字节 stack 会在 worker 边界变成固定的 `output-limit` 诊断。绕过补丁 stream 槽的原生写入会到达独立于完成端口的 pipe,因此宿主会针对这些字节和不可信伪造通信再次执行账本统计;在物化结果前,结算过程会持续进行有界 pipe 捕获,直到 worker 完成终止。`maxOutputBytes` 统计外层 `logs` 数组加完成值或失败消息载荷的 JSON 序列化;固定的 `CodeRunResult` 字段名、花括号、有界错误 kind 标签,以及后续呈现空白不计入这份可变载荷账本。未超过上限时会返回精确值;有损完成值属于 `invalid-output`,组合溢出属于 `output-limit`,不会用 inspected string 代替。失败会保留能容纳的已捕获前缀,之后按普通外层 `run_code` 落盘策略处理。 +- **空环境**:worker 使用 `env: {}` 和 `execArgv: []`,既没有环境凭据(比 spawn 命令的清理环境规则更严格),也不会继承 loader 标志。 +- **释放资源时等待完全停稳**:清理会把进行中的运行标记为 `abort`,并在 resolve 前等待每个 worker 退出。 ## 未构建与已构建的 worker 入口 -源代码模式通过 Node 原生类型剥离加载只包含可擦除语法的 `src/worker.ts`。其传递运行时闭包只包含 Node 内置模块和相对源模块,因此全新 checkout 绝不需要兄弟工作区包尚未构建的 `lib/` 导出。worker 本地和会话自有的 JSON 边界都会在消息端口周围展平并重建已验证值,使应用嵌套永远不会进入 structured clone。构建模式会把兄弟文件 `lib/worker.cjs` 作为文件系统路径传入,因为 pkg 的虚拟文件系统(VFS)Worker hook 要求 CommonJS;同一路径也可在普通 Node 下使用。演练这个已发布入口路径的仓库级要求由[测试策略](../../../docs/testing.md)规定。 +源代码模式通过 Node 原生类型剥离加载只包含可擦除语法的 `src/worker.ts`。其传递运行时闭包只包含 Node 内置模块和相对源模块,因此全新 checkout 绝不需要兄弟工作区包尚未构建的 `lib/` 导出。worker 本地 JSON 快照器会与会话自有的规范边界执行一致性测试;消息端口两侧都会展平并重建已验证值,使应用嵌套永远不会进入 structured clone。构建模式会把兄弟文件 `lib/worker.cjs` 作为文件系统路径传入,因为 pkg 的 VFS Worker hook 要求 CommonJS;同一路径也可在普通 Node 下使用。`tests/built-lib.e2e.ts` 固定了 [docs/testing.md](../../../docs/testing.md) 要求的真实加载路径。 -SDK 对外提供默认及具名导出的 `WorkerCodeRuntime` 类,以及 `Config`。运行所用的 `./worker` 子路径仅作为打包后的 spawn 入口存在;wire 协议与启动辅助模块是源代码私有的实现细节。 +SDK 接口是默认/具名 `WorkerCodeRuntime` 类与 `Config`。可操作的 `./worker` 子路径仅作为打包后的 spawn 入口存在;wire 协议与启动辅助模块是源代码私有的实现细节。 ## 模型体验 @@ -42,13 +42,13 @@ SDK 对外提供默认及具名导出的 `WorkerCodeRuntime` 类,以及 `Confi #### KV Cache 影响 -不会直接失效;由上述消费方负责请求前缀变更。 +不会直接失效;由具名消费方负责请求前缀变更。 -## 已知限制与暂缓事项 +## 已知限制与暂缓工作 -- **程序派生的 OS 进程在程序终止后仍会存活**:`worker.terminate()` 只结束线程,比 bash-local 的进程组终止更弱;在容器后端出现前,孤儿进程清理属于部署职责。 -- **类型剥离依赖 Node 的实验性 `stripTypeScriptTypes` API**:如依赖的行为发生变化,amaro 或 sucrase 是已经点名的直接替代品。 +- **程序 spawn 的 OS 进程在该后端终止后仍会存活**:`worker.terminate()` 只结束线程;需要清理远程进程组的部署可以选择 E2B 后端,但该后端自身的限制仍然适用。 +- **类型剥离依赖 Node 的实验性 `stripTypeScriptTypes` API**:依赖的行为由单元测试固定;如其发生变化,amaro/sucrase 是已经点名的直接替代品。 - **`computeMs` 到期最多可能超过一个轮询间隔**:系统每 25 ms 采样一次忙碌时间(内部常量,有意不做成配置)。 -- **程序获得一个含 5 个方法的 `console` shim**(`log`/`info`/`warn`/`error`/`debug`):有意不提供 Node 的完整 console 接口。 +- **程序获得一个含 5 种方法的 `console` shim**(`log`/`info`/`warn`/`error`/`debug`):有意不提供 Node 的完整 console 接口。 - **中间绑定值没有字节上限**:程序可以用永远不会成为外层输出的值耗尽进程或 worker 内存。 - **默认 64 MiB 是拒绝边界,不是可恢复存储**:外层落盘只能保存发生 `output-limit` 后返回的有界日志和诊断;在运行时上限之外被拒绝的字节永远不会到达落盘层。 diff --git a/packages/code-runtime/code-runtime-worker/src/index.ts b/packages/code-runtime/code-runtime-worker/src/index.ts index 43801899a2..2ece0b5c39 100644 --- a/packages/code-runtime/code-runtime-worker/src/index.ts +++ b/packages/code-runtime/code-runtime-worker/src/index.ts @@ -21,6 +21,10 @@ import { jsonStringBytesUpTo, jsonValueBytesUpTo, truncateJsonStringBytes } from import { decodeWorkerJson, encodeWorkerJson } from './worker-json.ts' import type { WorkerJsonWire } from './worker-json.ts' +export { jsonStringBytesUpTo, jsonValueBytesUpTo, truncateJsonStringBytes } from './output-json.ts' +export { decodeWorkerJson, encodeWorkerJson } from './worker-json.ts' +export type { WorkerJsonWire } from './worker-json.ts' + /** Plugin config: every execution cap, changeable from `cordis.yml` (no hardcoded tunables). */ export interface Config { /** @@ -165,14 +169,19 @@ function parseWorkerMessage(raw: unknown): WorkerToHost | undefined { } -/** One run's combined outer-output ledger; binding values never enter it. */ -class OutputLedger { +/** Shared outer-output accounting for one isolated run; binding values never enter it. */ +export class OutputLedger { private bytes = 2 // JSON serialization of the empty logs array: [] private entries = 0 constructor(private readonly maxBytes: number) {} - /** Admit one exact log entry, or report that the hard cap was crossed. */ + /** + * Admit one exact log entry, or report that the hard cap was crossed. + * @param text - Candidate log entry. + * @param sink - Accepted log entries for the current run. + * @returns Whether the complete entry fits the remaining outer-output budget. + */ admit(text: string, sink: string[]): boolean { const separatorBytes = this.entries > 0 ? 1 : 0 const stringBytes = jsonStringBytesUpTo(text, this.maxBytes - this.bytes - separatorBytes) @@ -183,19 +192,33 @@ class OutputLedger { return true } - /** Finalize a successful absent-or-JSON completion against the combined cap. */ + /** + * Finalize a successful absent-or-JSON completion against the combined cap. + * @param logs - Already accepted log entries. + * @param value - Optional lossless-JSON completion value. + * @returns A success result or an output-limit failure. + */ success(logs: string[], value?: CodeJsonValue): CodeRunResult { if (value !== undefined && jsonValueBytesUpTo(value, this.maxBytes - this.bytes) === undefined) return this.limit(logs) return { logs, ...value !== undefined ? { value } : {} } } - /** Finalize a failure diagnostic, with output-limit taking precedence when combined bytes exceed the cap. */ + /** + * Finalize a failure diagnostic, with output-limit taking precedence when combined bytes exceed the cap. + * @param logs - Already accepted log entries. + * @param error - Candidate failure diagnostic. + * @returns The diagnostic result or an output-limit failure. + */ failure(logs: string[], error: CodeRunFailure): CodeRunResult { if (jsonStringBytesUpTo(error.message, this.maxBytes - this.bytes) === undefined) return this.limit(logs) return { logs, error } } - /** Build the explicit output-limit failure while retaining a fitting prefix of the final log. */ + /** + * Build the explicit output-limit failure while retaining a fitting prefix of the final log. + * @param logs - Candidate log entries in original order. + * @returns A capped output-limit result. + */ limit(logs: string[]): CodeRunResult { const fullMessage = `outer output exceeded ${this.maxBytes} bytes` // The fixed diagnostic is ASCII, so every character is one byte plus the quotes. diff --git a/packages/code-runtime/code-runtime/README.i18n.yaml b/packages/code-runtime/code-runtime/README.i18n.yaml index a6bd209c7b..51c1466872 100644 --- a/packages/code-runtime/code-runtime/README.i18n.yaml +++ b/packages/code-runtime/code-runtime/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/code-runtime/code-runtime/README.md -README.md: bb1c20d00a260f643f601c42c6e48722437d5aab -README.zh.md: 15fbcecf77b2318acf3b09101802cd032ae426d2 +README.md: c7690412d0f1bc8556fc758da4e94c6ce5a08d8f +README.zh.md: ecfa48a97c46113c52f13c7a6edfc5d6fbbc2285 diff --git a/packages/code-runtime/code-runtime/README.md b/packages/code-runtime/code-runtime/README.md index bb1c20d00a..4018cdf508 100644 --- a/packages/code-runtime/code-runtime/README.md +++ b/packages/code-runtime/code-runtime/README.md @@ -34,5 +34,5 @@ No direct invalidation; the named consumer owns any request-prefix changes. - **`run()` is one-shot** — `logs` arrive only on the resolved `CodeRunResult`; the seam exposes no streaming-log or progress surface for a live program's output. - **A persistent REPL-style kernel is recorded future work** — the no-state-between-runs contract stands until a persistent-kernel backend brings its own logging story ([Code Mode Agent Note](../../../.agents/notes/implemented/feature/2026-06-15-code-mode.md)). -- **Only the worker-thread backend ships** — `'process'`/`'container'` are declared well-known `isolation` values with no implementation; a hard security boundary awaits a container backend. +- **Isolation is backend-specific** — the worker backend is process-local, while the E2B backend reports `container` and keeps orchestration and bindings on the host; the descriptor remains informational rather than a security claim. - **Intermediate binding values have no byte cap** — implementations remain subject to structured-clone cost and process memory, while a provider or executor may already have imposed its own acquisition bound. diff --git a/packages/code-runtime/code-runtime/README.zh.md b/packages/code-runtime/code-runtime/README.zh.md index 15fbcecf77..ecfa48a97c 100644 --- a/packages/code-runtime/code-runtime/README.zh.md +++ b/packages/code-runtime/code-runtime/README.zh.md @@ -4,23 +4,21 @@ 这是**代码执行 seam**:抽象的 `CodeRuntime` 服务(`ctx.codeRuntime`)只定义代码运行时做什么,即针对宿主提供的一组异步绑定运行一段模型编写的程序,并报告 `{ value, logs, error? }`,而不规定如何实现。 -此包承担该能力三个组成部分中的接口职责(以 bash 三包结构为模板,参见[能力 seam](../../../.agents/notes/implemented/architecture/2026-06-13-capability-seams.md)):实现通过继承 `CodeRuntime` 并注册服务接入;消费方是工具注册表的 Code Mode,它生成面向模型的 SDK,并桥接工具分发。这两项职责均由 [Code Mode Agent Note](../../../.agents/notes/implemented/feature/2026-06-15-code-mode.md) 规定,首个实现是 Node worker 线程后端。运行时不了解工具或会话:调用方只向它提供具名异步函数与程序字符串;所有与工具有关的内容都留在消费方。 +此包是该能力的接口层(以 bash 三包结构为模板,参见[能力 seam](../../../.agents/notes/implemented/architecture/2026-06-13-capability-seams.md)):实现通过继承 `CodeRuntime` 并注册服务接入;消费方是工具注册表的 Code Mode,它生成面向模型的 SDK,并桥接工具分发。两者都由 [Code Mode Agent Note](../../../.agents/notes/implemented/feature/2026-06-15-code-mode.md) 规定,首个实现是 Node worker 线程后端。运行时不了解工具或会话:调用方只向它提供具名异步函数与程序字符串;所有工具形状的内容都留在消费方。 ## 服务 API(`ctx.codeRuntime`) | 成员 | 语义 | |---|---| -| `run(request)` | 针对请求的绑定执行一段程序。**所有程序失败结果都通过 resolve 结果中的 error 字段报告**:包括解析/转换失败、抛出异常、无效完成值、输出溢出、预算到期、中止或执行基底终止(由 `CodeRunFailure` 的正交 `kind` 分类表示);只有调用方误用 seam 本身时才 reject(例如 dispose(资源释放)后仍提交运行)。程序作为异步函数的函数体运行,因此顶层 `await`/`return` 可用,无损 JSON 完成值会成为 `result.value`。 | -| `language` | 只读描述符:`run` 期望的源语言。已知值为 `'typescript'` 与 `'python'`——`dsh-tools` 能呈现的那些;其中只有 `'typescript'` 有已发布的后端。仅供参考,不作门禁;生成语言专用呈现的消费方会根据该值选择分支,遇到无法呈现的语言时明确失败。 | +| `run(request)` | 针对请求的绑定执行一段程序。**每一种程序结果都通过 error 字段完成 resolve**:包括解析/转换失败、抛出异常、无效完成值、输出溢出、预算到期、中止或执行基底死亡(由 `CodeRunFailure` 的正交 `kind` 分类表示);只有调用方误用 seam 本身时才 reject(例如资源释放后仍提交运行)。程序作为异步函数的函数体运行,因此顶层 `await`/`return` 可用,无损 JSON 完成值会成为 `result.value`。 | +| `language` | 只读描述符:`run` 期望的源语言(已知值为 `'typescript'`)。仅供参考,不作门禁;生成语言专用呈现的消费方会对该值执行分支,遇到无法呈现的语言时明确失败。 | | `isolation` | 只读描述符:执行基底(`'worker-thread'`、`'process'`、`'container'`)。供部署与诊断使用,**不构成安全声明**。 | -每个实现都必须遵守以下语义(完整契约见类 JSDoc):绑定调用会桥接完整的无损 JSON 参数与 resolve 值,seam 层不设字节上限;程序被视为敌对对等方(任意绑定名称都会成为自有属性,格式错误的通信绝不能使宿主崩溃);不同运行之间不保留任何状态;dispose 会终止进行中的运行,并且在完成前等待其退出。 +每个实现都必须遵守以下语义(完整契约见类 JSDoc):绑定调用会桥接完整的无损 JSON 参数与 resolve 值,seam 层不设字节上限;程序被视为不可信对等方(任意绑定名称都是自有属性,格式错误的通信绝不能使宿主崩溃);不同运行之间不保留任何状态;资源释放会终止进行中的运行,并且在完成前等待其退出。 ## 词汇 -`CodeRunRequest`(`program`、`bindings`、`signal?`)携带运行时操作所需的全部内容;默认值解析(时间预算与外层输出上限)属于实现的已验证配置,绝不能是隐藏的 `??`,更不能藏在 `run()` 内部。`bindings` 是 `CodeBindingNamespace` 列表(`global` + `functions` + 可选 `errorClass`);每个命名空间会作为一个由异步可调用函数组成的全局对象公开给程序,这些函数返回 `CodeJsonValue`。后者是 seam 本地、与规范 `JsonValue` 结构等价的类型,使接口包保持独立于会话。`errorClass` 描述符点名真实的程序全局构造器,以及用于接收被拒绝成员名称的自有属性;运行时不依赖 `ToolCallError` 等消费方术语。`CodeRunResult` 报告无损 JSON 完成值 `value?`、有序的 `logs: string[]` 和 `error?`(`CodeRunFailure`:`kind` + 可反馈给模型的 `message`)。完整契约见 `src/types.ts`。 - -binding-global 与 error-class 名称是**语言可移植**的:必须匹配标识符子集 `[A-Za-z_][A-Za-z0-9_]*`(不含 JS 专有的 `$`)并通过 seam 导出的排除集,因此同一份 `bindings` 列表对每个后端都有效,无论其 `language` 为何。本包导出每个后端都执行的契约——`PORTABLE_RESERVED_WORDS`(ECMAScript ∪ Python 保留字)、`RESERVED_BINDING_GLOBALS`(如 `console` 等后端拥有的 global)、`RESERVED_ERROR_MEMBERS` 与 `DUNDER_MEMBER`(error-member 排除)——因此 `$tools`、`lambda`、`__dsh_main__` 之类的名称会让 `run()` 在任何后端上作为 seam 误用而 reject,而非只在某些后端。确切集合与理由见 `src/index.ts`。 +`CodeRunRequest`(`program`、`bindings`、`signal?`)携带运行时操作所需的全部内容;默认值解析(时间预算与外层输出上限)属于实现的已验证配置,绝不能是隐藏的 `??`,更不能藏在 `run()` 内部。`bindings` 是 `CodeBindingNamespace` 列表(`global` + `functions` + 可选 `errorClass`);每个命名空间会作为一个由异步可调用函数组成的全局对象公开给程序,这些函数返回 `CodeJsonValue`。后者是 seam 本地、与规范 `JsonValue` 结构等价的类型,使接口包保持独立于会话。`errorClass` 描述符点名真实的程序全局构造器,以及用于接收 reject 成员名称的自有属性;运行时不依赖 `ToolCallError` 等消费方术语。`CodeRunResult` 报告无损 JSON 完成值 `value?`、有序的 `logs: string[]` 和 `error?`(`CodeRunFailure`:`kind` + 可反馈给模型的 `message`)。完整契约见 `src/types.ts`。 ## 模型体验 @@ -28,11 +26,11 @@ binding-global 与 error-class 名称是**语言可移植**的:必须匹配标 #### KV Cache 影响 -不会直接失效;由上述消费方负责请求前缀变更。 +不会直接失效;由具名消费方负责请求前缀变更。 -## 已知限制与暂缓事项 +## 已知限制与暂缓工作 -- **`run()` 是一次性的**:`logs` 只有在 `CodeRunResult` resolve 后才能获得;seam 不提供正在运行的程序所产生输出的流式日志或进度接口。 +- **`run()` 是一次性的**:`logs` 只有在 `CodeRunResult` resolve 后才能获得;seam 不提供活跃程序输出的流式日志或进度接口。 - **持久 REPL 风格内核已记录为未来工作**:在持久内核后端带来自己的日志方案前,运行之间不保留状态的契约继续有效(参见 [Code Mode Agent Note](../../../.agents/notes/implemented/feature/2026-06-15-code-mode.md))。 -- **目前只提供 worker 线程后端**:`'process'`/`'container'` 是已经声明但没有实现的已知 `isolation` 值;强安全边界需要等待容器后端。 +- **隔离方式由后端决定**:worker 后端位于宿主进程内,而 E2B 后端报告 `container`,并把编排与绑定留在宿主;该描述符仍只提供信息,不构成安全声明。 - **中间绑定值没有字节上限**:实现仍受 structured-clone 成本与进程内存约束,而提供方或执行器可能已经应用自己的获取上限。 diff --git a/packages/e2b/README.i18n.yaml b/packages/e2b/README.i18n.yaml index 1bc9761930..444145fbb8 100644 --- a/packages/e2b/README.i18n.yaml +++ b/packages/e2b/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/e2b/README.md -README.md: 4b28ec7c0452189a579a6d7e0cd22a0b8561784d -README.zh.md: ab254eecad82c05cd6a6021e7f8bc4eda6cf3076 +README.md: 9e7f1a98ada9f56f3d1e3f74d2a906f8a1c7bc15 +README.zh.md: 53fdf30eaf405ab5313274a7fadde0b044b7fba1 diff --git a/packages/e2b/README.md b/packages/e2b/README.md index 4b28ec7c04..9e7f1a98ad 100644 --- a/packages/e2b/README.md +++ b/packages/e2b/README.md @@ -2,12 +2,15 @@ English | [中文](README.zh.md) -An experimental provider-composition POC that places the filesystem and managed subprocess world in one E2B Linux sandbox. The shared owner is separate from the capability adapters so every remote provider awaits the same sandbox identity and lifecycle. +An experimental provider-composition POC that places the mutable coding world in one E2B Linux sandbox. The shared owner is separate from capability adapters so every remote provider awaits the same sandbox identity and lifecycle. | Package | ctx key | Role | |---|---|---| | [`e2b`](e2b/README.md) (`@deepseek-ai/dsh-e2b`) | `ctx.e2b` | Create or reconnect one sandbox, create its working/runtime directories, expose the shared SDK handle, and apply the configured kill/pause/leave disposition | | [`fs-e2b`](../fs/fs-e2b/README.md) (`@deepseek-ai/dsh-fs-e2b`) | `ctx.fs` | Implement the filesystem seam over E2B Filesystem APIs | | [`subprocess-e2b`](../subprocess/subprocess-e2b/README.md) (`@deepseek-ai/dsh-subprocess-e2b`) | `ctx.subprocess` | Implement managed process groups, stdio projection, and remote spill files over E2B Commands | +| [`pty-e2b`](../pty/pty-e2b/README.md) (`@deepseek-ai/dsh-pty-e2b`) | `ctx.pty` backend | Run persistent interactive shells through E2B's byte PTY API | +| [`lsp-e2b`](../lsp/lsp-e2b/README.md) (`@deepseek-ai/dsh-lsp-e2b`) | `ctx.lsp` provider | Run configured language servers and read query sources inside E2B | +| [`code-runtime-e2b`](../code-runtime/code-runtime-e2b/README.md) (`@deepseek-ai/dsh-code-runtime-e2b`) | `ctx.codeRuntime` | Run model-written programs remotely while bridging bindings to the host | -The existing [`dsh-bash-local`](../bash/bash-local/README.md) needs no E2B-specific fork: it already delegates process mechanics to `ctx.subprocess`, so replacing that provider places Bash in the same remote world as `ctx.fs`. This boundary does not move the harness process, Cordis objects, model calls, agent/session state, session persistence, skills, or E2B SDK buffers. The [decision record](../../.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md) owns the POC boundary and rejected expansion. +The existing [`dsh-bash-local`](../bash/bash-local/README.md) needs no E2B-specific fork: it delegates process mechanics to `ctx.subprocess`, so replacing that provider places Bash in the same remote world. This boundary does not move the harness process, Cordis objects, model calls, agent/session state, session persistence, skills, protocol state, or E2B SDK buffers. The [base decision](../../.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md) and [runtime-extension decision](../../.agents/notes/implemented/feature/2026-07-28-e2b-interactive-semantic-code-runtime-poc.md) own the POC boundary. diff --git a/packages/e2b/README.zh.md b/packages/e2b/README.zh.md index ab254eecad..53fdf30eaf 100644 --- a/packages/e2b/README.zh.md +++ b/packages/e2b/README.zh.md @@ -2,12 +2,15 @@ [English](README.md) | 中文 -这是一个实验性提供方组合 POC,把文件系统和受管子进程环境放进同一个 E2B Linux 沙箱。共享所有者与功能适配器彼此分离,使每个远程提供方都等待同一个沙箱身份和生命周期。 +这是一个实验性提供方组合 POC,把可变的编码环境放进同一个 E2B Linux 沙箱。共享所有者与功能适配器分离,使每个远程提供方都等待同一个沙箱身份和生命周期。 | 包(package) | ctx 键 | 职责 | |---|---|---| | [`e2b`](e2b/README.md)(`@deepseek-ai/dsh-e2b`) | `ctx.e2b` | 创建或重新连接一个沙箱,创建其工作目录与运行时目录,公开共享 SDK 句柄,并应用配置的 kill/pause/leave 处置方式 | | [`fs-e2b`](../fs/fs-e2b/README.md)(`@deepseek-ai/dsh-fs-e2b`) | `ctx.fs` | 通过 E2B Filesystem API 实现文件系统 seam | | [`subprocess-e2b`](../subprocess/subprocess-e2b/README.md)(`@deepseek-ai/dsh-subprocess-e2b`) | `ctx.subprocess` | 通过 E2B Commands 实现受管进程组、stdio 投影与远程 spill 文件 | +| [`pty-e2b`](../pty/pty-e2b/README.md)(`@deepseek-ai/dsh-pty-e2b`) | `ctx.pty` 后端 | 通过 E2B 的字节 PTY API 运行持久交互式 shell | +| [`lsp-e2b`](../lsp/lsp-e2b/README.md)(`@deepseek-ai/dsh-lsp-e2b`) | `ctx.lsp` 提供方 | 在 E2B 内运行已配置的语言服务器并读取查询源代码 | +| [`code-runtime-e2b`](../code-runtime/code-runtime-e2b/README.md)(`@deepseek-ai/dsh-code-runtime-e2b`) | `ctx.codeRuntime` | 远程运行模型编写的程序,同时把绑定桥接到宿主 | -现有的 [`dsh-bash-local`](../bash/bash-local/README.md) 无需 E2B 专用 fork:它已经把进程机制委托给 `ctx.subprocess`,因此替换该提供方即可让 Bash 与 `ctx.fs` 进入同一个远程环境。该边界不会迁移 harness 进程、Cordis 对象、模型调用、agent(智能体)/会话状态、会话持久化、skill(技能)或 E2B SDK 缓冲。[决策记录](../../.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md)负责说明 POC 边界及未采纳的扩展方案。 +现有的 [`dsh-bash-local`](../bash/bash-local/README.md) 无需 E2B 专用 fork:它把进程机制委托给 `ctx.subprocess`,因此替换该提供方即可让 Bash 进入同一个远程环境。该边界不会迁移 harness 进程、Cordis 对象、模型调用、agent(智能体)/会话状态、会话持久化、skill(技能)、协议状态或 E2B SDK 缓冲。[基础决策](../../.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md)与[运行时扩展决策](../../.agents/notes/implemented/feature/2026-07-28-e2b-interactive-semantic-code-runtime-poc.md)共同界定 POC 边界。 diff --git a/packages/e2b/e2b/README.i18n.yaml b/packages/e2b/e2b/README.i18n.yaml index 9255c21577..164bb47784 100644 --- a/packages/e2b/e2b/README.i18n.yaml +++ b/packages/e2b/e2b/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/e2b/e2b/README.md -README.md: bf62cb7d4811ca92f263bd3d337fff9fe41ce223 -README.zh.md: 8c7603aa501a2481aba6dab1f22750056e43ad26 +README.md: 5f89a4bcffdfd11fef8929d2a2ceecb41af319e2 +README.zh.md: b4956033bae131bb8aa236276323ecba30f00115 diff --git a/packages/e2b/e2b/README.md b/packages/e2b/e2b/README.md index bf62cb7d48..5f89a4bcff 100644 --- a/packages/e2b/e2b/README.md +++ b/packages/e2b/e2b/README.md @@ -2,7 +2,7 @@ English | [中文](README.zh.md) -Shared lifecycle owner for one E2B sandbox. Filesystem and subprocess adapters inject `ctx.e2b`, await its single SDK handle, and therefore inhabit the same remote Linux working tree and process world. The package pins `e2b@2.29.1`. +Shared lifecycle owner for one E2B sandbox. Capability adapters inject `ctx.e2b`, await its single SDK handle, and therefore inhabit the same remote Linux working tree and process world. The package pins `e2b@2.29.1`; the [family map](../README.md) lists the opt-in adapters. ## Configuration diff --git a/packages/e2b/e2b/README.zh.md b/packages/e2b/e2b/README.zh.md index 8c7603aa50..b4956033ba 100644 --- a/packages/e2b/e2b/README.zh.md +++ b/packages/e2b/e2b/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -一个 E2B 沙箱的共享生命周期所有者。文件系统与进程管理适配器注入 `ctx.e2b`,等待其唯一的 SDK 句柄,因此处于同一个远程 Linux 工作树与进程环境中。本包固定使用 `e2b@2.29.1`。 +一个 E2B 沙箱的共享生命周期所有者。功能适配器注入 `ctx.e2b`,等待其唯一的 SDK 句柄,因此处于同一个远程 Linux 工作树与进程环境中。本包固定使用 `e2b@2.29.1`;可选适配器见[包族索引](../README.md)。 ## 配置 diff --git a/packages/e2b/e2b/src/frame.ts b/packages/e2b/e2b/src/frame.ts new file mode 100644 index 0000000000..39800b4428 --- /dev/null +++ b/packages/e2b/e2b/src/frame.ts @@ -0,0 +1,75 @@ +/** ASCII/base64 JSON framing for byte-faithful protocols over E2B text callbacks. */ + +import { Buffer } from 'node:buffer' + +const BASE64_LINE = /^(?:[A-Za-z0-9+/]{4})*(?:[A-Za-z0-9+/]{2}==|[A-Za-z0-9+/]{3}=)?$/ + +/** + * Encode one JSON-compatible value as a newline-delimited ASCII frame. + * @param value - Value accepted by `JSON.stringify`. + * @returns Base64-encoded UTF-8 JSON followed by one newline. + */ +export function encodeE2BFrame(value: unknown): string { + const json: unknown = JSON.stringify(value) + if (typeof json !== 'string') throw new Error('E2B frame value is not JSON-serializable') + return `${Buffer.from(json).toString('base64')}\n` +} + +/** Incremental decoder for newline-delimited base64 JSON frames. */ +export class E2BFrameDecoder { + private pending = '' + private readonly maxEncodedChars: number + + /** @param maxFrameBytes - Maximum decoded UTF-8 JSON bytes in one frame. */ + constructor(private readonly maxFrameBytes: number) { + if (!Number.isSafeInteger(maxFrameBytes) || maxFrameBytes <= 0) { + throw new Error('E2B frame maxFrameBytes must be a positive safe integer') + } + this.maxEncodedChars = Math.ceil(maxFrameBytes / 3) * 4 + } + + /** + * Consume one E2B callback chunk. + * @param chunk - ASCII text received from the remote helper. + * @returns Every complete decoded JSON value, in order. + */ + push(chunk: string): unknown[] { + if (/[^\x0a\x20-\x7e]/.test(chunk)) throw new Error('E2B frame stream contained non-ASCII data') + this.pending += chunk + const values: unknown[] = [] + for (;;) { + const newline = this.pending.indexOf('\n') + if (newline < 0) { + if (this.pending.length > this.maxEncodedChars) throw new Error('E2B frame exceeded its byte limit') + return values + } + const line = this.pending.slice(0, newline) + this.pending = this.pending.slice(newline + 1) + values.push(this.decode(line)) + } + } + + /** Reject a truncated final frame. */ + finish(): void { + if (this.pending.length !== 0) throw new Error('E2B frame stream ended mid-frame') + } + + private decode(line: string): unknown { + if (line.length === 0 || line.length > this.maxEncodedChars || !BASE64_LINE.test(line)) { + throw new Error('E2B frame contained invalid base64 or exceeded its byte limit') + } + const bytes = Buffer.from(line, 'base64') + if (bytes.length > this.maxFrameBytes) throw new Error('E2B frame exceeded its byte limit') + let json: string + try { + json = new TextDecoder('utf-8', { fatal: true }).decode(bytes) + } catch (error: unknown) { + throw new Error('E2B frame contained invalid UTF-8', { cause: error }) + } + try { + return JSON.parse(json) as unknown + } catch (error: unknown) { + throw new Error('E2B frame contained invalid JSON', { cause: error }) + } + } +} diff --git a/packages/e2b/e2b/src/index.ts b/packages/e2b/e2b/src/index.ts index 29cbf1dc53..4c4597f6ef 100644 --- a/packages/e2b/e2b/src/index.ts +++ b/packages/e2b/e2b/src/index.ts @@ -10,6 +10,8 @@ import z from 'schemastery' import { Sandbox } from 'e2b' import type { Branded } from '@deepseek-ai/dsh-brand' +export { E2BFrameDecoder, encodeE2BFrame } from './frame.ts' + export { CommandExitError, FileNotFoundError, @@ -19,7 +21,7 @@ export { SandboxNotFoundError, TimeoutError, } from 'e2b' -export type { CommandHandle, CommandResult, EntryInfo } from 'e2b' +export type { CommandHandle, CommandResult, EntryInfo, ProcessInfo, PtyOutput } from 'e2b' /** Opaque E2B sandbox identity used for reconnecting a later harness process. */ export type E2BSandboxId = Branded<'E2BSandboxId'> @@ -42,6 +44,26 @@ export function quoteE2BShellArg(value: string): string { return `'${value.replaceAll('\'', "'\"'\"'")}'` } +/** + * Resolve one executable inside an E2B sandbox and require an absolute result. + * @param sandbox - Sandbox whose PATH and filesystem own the executable. + * @param command - Absolute path or bare executable name. + * @returns Verified absolute remote executable path. + */ +export async function resolveE2BExecutable(sandbox: Sandbox, command: string): Promise { + if (command.length === 0) throw new Error('E2B executable name must be non-empty') + if (posix.isAbsolute(command)) { + await sandbox.commands.run(`test -f ${quoteE2BShellArg(command)} -a -x ${quoteE2BShellArg(command)}`) + return command + } + const result = await sandbox.commands.run(`command -v -- ${quoteE2BShellArg(command)}`) + const executable = result.stdout.trim() + if (!posix.isAbsolute(executable) || executable.includes('\n')) { + throw new Error(`E2B executable ${JSON.stringify(command)} did not resolve to one absolute path`) + } + return executable +} + /** Action taken on the owned sandbox when the Cordis service is disposed. */ export type E2BDisposeMode = 'kill' | 'pause' | 'leave' diff --git a/packages/e2b/e2b/tests/composition.e2e.ts b/packages/e2b/e2b/tests/composition.e2e.ts index 8c01fae7b4..d056d4b851 100644 --- a/packages/e2b/e2b/tests/composition.e2e.ts +++ b/packages/e2b/e2b/tests/composition.e2e.ts @@ -11,7 +11,7 @@ const configPath = join(fixtureRoot, 'cordis.yml') const tsconfigPath = fileURLToPath(new URL('../../../../tsconfig.json', import.meta.url)) describe.skipIf(!process.env.E2B_API_KEY)('E2B live Loader composition', () => { - it('shares remote state across FS and Bash without creating host workspace files', async () => { + it('runs FS, Bash, PTY, LSP, and Code Runtime in one sandbox and deletes it', async () => { const { stdout, stderr } = await runLoaderSmoke({ label: 'E2B composition', tempDirPrefix: 'dsh-e2b-composition-', @@ -19,10 +19,14 @@ describe.skipIf(!process.env.E2B_API_KEY)('E2B live Loader composition', () => { libBinScript: binScript, configPath, tsconfigPath, - processTimeoutMs: 90_000, + env: { + NODE_OPTIONS: [process.env.NODE_OPTIONS, '--disable-warning=ExperimentalWarning'].filter(Boolean).join(' '), + }, + processTimeoutMs: 120_000, inspect: async (cwd) => { - await expect(access(join(cwd, 'from-fs.txt'))).rejects.toMatchObject({ code: 'ENOENT' }) - await expect(access(join(cwd, 'from-bash.txt'))).rejects.toMatchObject({ code: 'ENOENT' }) + for (const name of ['from-fs.txt', 'from-bash.txt', 'multibyte.ts', 'fixture-lsp.mjs']) { + await expect(access(join(cwd, name))).rejects.toMatchObject({ code: 'ENOENT' }) + } }, }) @@ -31,9 +35,37 @@ describe.skipIf(!process.env.E2B_API_KEY)('E2B live Loader composition', () => { expect(output).toMatchObject({ bashRead: 'written-by-fs\n', fsRead: 'written-by-bash\n', + hover: { + kind: 'hover', + hover: { contents: '**remote hover** 你好 café' }, + }, + definition: { + kind: 'locations', + locations: [{ range: { start: { line: 0, character: 6 }, end: { line: 0, character: 10 } } }], + }, + terminal: { + echo: { waitReason: 'stdin_read', sessionStatus: { kind: 'running' } }, + signal: { delivered: true }, + interrupted: { sessionStatus: { kind: 'running' } }, + }, + hostileOutput: { error: { kind: 'output-limit' } }, + timedOut: { error: { kind: 'timeout' } }, + aborted: { error: { kind: 'abort', message: 'live abort' } }, + lingeringCodeRunners: 0, + }) + expect((output.terminal as { motd: string }).motd.length).toBeGreaterThan(0) + expect((output.terminal as { echo: { viewport: string } }).echo.viewport).toContain('PTY-你好') + expect((output.terminal as { scrollback: string }).scrollback).toContain('PTY-你好') + expect((output.terminal as { signal: { targetPgid: number } }).signal.targetPgid).toBeGreaterThan(0) + expect(['stdin_read', 'inferred_idle']).toContain( + (output.terminal as { interrupted: { waitReason: string } }).interrupted.waitReason, + ) + expect(output.code).toEqual({ + value: { doubled: 42, typed: true }, + logs: ['remote-log 你好 42', 'post-mutation'], }) const apiKey = process.env.E2B_API_KEY if (apiKey === undefined) throw new Error('E2B_API_KEY disappeared during the live composition test') await expect(Sandbox.getInfo(String(output.sandboxId), { apiKey })).rejects.toBeInstanceOf(SandboxNotFoundError) - }, 105_000) + }, 135_000) }) diff --git a/packages/e2b/e2b/tests/e2b.spec.ts b/packages/e2b/e2b/tests/e2b.spec.ts index c842731f5c..541e69df9c 100644 --- a/packages/e2b/e2b/tests/e2b.spec.ts +++ b/packages/e2b/e2b/tests/e2b.spec.ts @@ -2,8 +2,11 @@ import { beforeEach, describe, expect, it, vi } from 'vitest' import { Context } from 'cordis' import type { Sandbox as SandboxType } from 'e2b' import E2BSandboxService, { + E2BFrameDecoder, E2BSandboxId, + encodeE2BFrame, quoteE2BShellArg, + resolveE2BExecutable, } from '@deepseek-ai/dsh-e2b' import * as E2BInvariant from '../src/invariant.ts' import InvariantService from '@deepseek-ai/dsh-invariants' @@ -209,6 +212,49 @@ describe('E2B helpers and invariant companion', () => { expect(quoteE2BShellArg("a'b $HOME")).toBe("'a'\"'\"'b $HOME'") }) + it('resolves absolute and PATH executables inside the sandbox', async () => { + const run = vi.fn() + .mockResolvedValueOnce({ exitCode: 0, stdout: '', stderr: '' }) + .mockResolvedValueOnce({ exitCode: 0, stdout: '/usr/bin/node\n', stderr: '' }) + const sandbox = { commands: { run } } as unknown as SandboxType + await expect(resolveE2BExecutable(sandbox, '/bin/bash')).resolves.toBe('/bin/bash') + await expect(resolveE2BExecutable(sandbox, 'node')).resolves.toBe('/usr/bin/node') + expect(run).toHaveBeenNthCalledWith(1, "test -f '/bin/bash' -a -x '/bin/bash'") + expect(run).toHaveBeenNthCalledWith(2, "command -v -- 'node'") + }) + + it('rejects empty or non-absolute executable resolutions', async () => { + const sandbox = { + commands: { run: vi.fn().mockResolvedValue({ exitCode: 0, stdout: 'relative\npath\n', stderr: '' }) }, + } as unknown as SandboxType + await expect(resolveE2BExecutable(sandbox, '')).rejects.toThrow('non-empty') + await expect(resolveE2BExecutable(sandbox, 'tool')).rejects.toThrow('did not resolve') + }) + + it('round-trips split and adjacent ASCII/base64 JSON frames', () => { + const decoder = new E2BFrameDecoder(128) + const encoded = encodeE2BFrame({ text: '你好' }) + encodeE2BFrame([1, true]) + expect(decoder.push(encoded.slice(0, 5))).toEqual([]) + expect(decoder.push(encoded.slice(5))).toEqual([{ text: '你好' }, [1, true]]) + expect(() => { decoder.finish() }).not.toThrow() + expect(() => encodeE2BFrame(undefined)).toThrow('not JSON-serializable') + }) + + it('rejects malformed, oversized, and truncated frame streams', () => { + expect(() => new E2BFrameDecoder(0)).toThrow('positive safe integer') + expect(() => new E2BFrameDecoder(1.5)).toThrow('positive safe integer') + expect(() => new E2BFrameDecoder(4).push('é')).toThrow('non-ASCII') + expect(() => new E2BFrameDecoder(3).push('AAAAA')).toThrow('byte limit') + expect(() => new E2BFrameDecoder(8).push('\n')).toThrow('invalid base64') + expect(() => new E2BFrameDecoder(8).push('abc!\n')).toThrow('invalid base64') + expect(() => new E2BFrameDecoder(2).push(`${Buffer.from('abc').toString('base64')}\n`)).toThrow('byte limit') + expect(() => new E2BFrameDecoder(8).push('/w==\n')).toThrow('invalid UTF-8') + expect(() => new E2BFrameDecoder(16).push(`${Buffer.from('not-json').toString('base64')}\n`)).toThrow('invalid JSON') + const truncated = new E2BFrameDecoder(8) + truncated.push('YQ==') + expect(() => { truncated.finish() }).toThrow('mid-frame') + }) + it('registers the package-owned empty invariant installer', async () => { const ctx = new Context() await ctx.plugin(InvariantService, { enabled: true }) diff --git a/packages/lsp/README.i18n.yaml b/packages/lsp/README.i18n.yaml index 9919cdaba5..c28c0282c2 100644 --- a/packages/lsp/README.i18n.yaml +++ b/packages/lsp/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/lsp/README.md -README.md: 7fbdf071735673fb0158f6fa66148be1c644a433 -README.zh.md: e059dbd80b7e38c0e447e54178162316dfd127c7 +README.md: 74ed729d97013de3d7f502f728e3abe62348846f +README.zh.md: 5a08d54ca6c54d8355454fc3021a0da48772000b diff --git a/packages/lsp/README.md b/packages/lsp/README.md index 7fbdf07173..74ed729d97 100644 --- a/packages/lsp/README.md +++ b/packages/lsp/README.md @@ -7,9 +7,10 @@ The language-server capability seam: an abstract LSP interface, a generic stdio | Package | Role | ctx key | |---|---|---| | `lsp/` | Abstract LSP seam (provider registry by branded id + extension mapping, per-query selection, vocabulary, `LspError`) | `ctx.lsp` | -| `lsp-local/` | Generic multi-server stdio backend over `ctx.fs` and `ctx.subprocess` (JSON-RPC, transient-open queries) | (registers providers on `ctx.lsp`) | +| [`lsp-local/`](lsp-local/README.md) | Generic multi-server local backend (spawn, JSON-RPC, transient-open queries) | (registers providers on `ctx.lsp`) | +| [`lsp-e2b/`](lsp-e2b/README.md) | Remote E2B backend (remote source reads and servers, byte-framed stdio bridge) | (registers providers on `ctx.lsp`) | | `tool-lsp/` | Model-facing `lsp` tool (four operations, one-based UTF-16 cursor coordinates) | (registers on `ctx.tools`) | The interface lives at `lsp/lsp/`. The seam exposes exactly four semantic operations — `goToDefinition`, `findReferences`, `goToImplementation`, `hover` — and no generic JSON-RPC escape hatch, so a provider swap does not change how the model asks for navigation and no protocol payload or unreviewed mutation reaches the model contract. Providers register **capabilities**, not tools; `tool-lsp` is the only owner of the model-facing name, schema, prompt guidance, and presentation. -See the [LSP capability seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md) for the design rationale, including why documents open transiently per query, why the stdio host consumes the shared filesystem/subprocess execution world, and why extension ownership is exclusive within one runtime. +See the [LSP capability seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md) for the protocol design and the [E2B extension note](../../.agents/notes/implemented/feature/2026-07-28-e2b-interactive-semantic-code-runtime-poc.md) for the remote process/filesystem boundary. diff --git a/packages/lsp/README.zh.md b/packages/lsp/README.zh.md index e059dbd80b..5a08d54ca6 100644 --- a/packages/lsp/README.zh.md +++ b/packages/lsp/README.zh.md @@ -7,9 +7,10 @@ | 包 | 职责 | ctx key | |---|---|---| | `lsp/` | 抽象 LSP seam(按品牌化 id + 扩展名映射组织的提供方注册表、逐查询选择、词汇、`LspError`) | `ctx.lsp` | -| `lsp-local/` | 基于 `ctx.fs` 与 `ctx.subprocess` 的通用多服务器 stdio 后端(JSON-RPC、临时打开查询) | (在 `ctx.lsp` 上注册提供方) | +| [`lsp-local/`](lsp-local/README.md) | 通用多服务器本地后端(spawn、JSON-RPC、临时打开查询) | (在 `ctx.lsp` 上注册提供方) | +| [`lsp-e2b/`](lsp-e2b/README.md) | 远程 E2B 后端(在远程读取源文件并运行服务器、采用字节分帧的 stdio 桥) | (在 `ctx.lsp` 上注册提供方) | | `tool-lsp/` | 面向模型的 `lsp` 工具(四种操作、从 1 开始的 UTF-16 光标坐标) | (注册到 `ctx.tools`) | 接口位于 `lsp/lsp/`。该 seam 恰好公开四种语义操作:`goToDefinition`、`findReferences`、`goToImplementation`、`hover`,且不提供通用 JSON-RPC 逃生口;因此,替换提供方不会改变模型请求导航的方式,也不会让协议载荷或未经评审的修改进入模型契约。提供方注册的是**能力** 而非工具;`tool-lsp` 是面向模型名称、schema、提示词指引和呈现的唯一 owner。 -设计原理见 [LSP 能力 seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md),其中也解释了文档为何在每次查询时临时打开、stdio 主机为何使用共享的文件系统/子进程执行环境,以及扩展名归属为何在同一运行时内互斥。 +协议设计见 [LSP 能力 seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md),远程进程/文件系统边界见 [E2B 扩展 Agent Note](../../.agents/notes/implemented/feature/2026-07-28-e2b-interactive-semantic-code-runtime-poc.md)。 diff --git a/packages/lsp/lsp-e2b/README.i18n.yaml b/packages/lsp/lsp-e2b/README.i18n.yaml new file mode 100644 index 0000000000..fa159820a9 --- /dev/null +++ b/packages/lsp/lsp-e2b/README.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write packages/lsp/lsp-e2b/README.md +README.md: 968487b7c5f63b2165cf66797a4182642507a8e9 +README.zh.md: 2b937f01156c3194f64ed9fb54478fec5c77305b diff --git a/packages/lsp/lsp-e2b/README.md b/packages/lsp/lsp-e2b/README.md new file mode 100644 index 0000000000..968487b7c5 --- /dev/null +++ b/packages/lsp/lsp-e2b/README.md @@ -0,0 +1,47 @@ +# @deepseek-ai/dsh-lsp-e2b + +English | [中文](README.zh.md) + +Generic E2B language-server backend for [`ctx.lsp`](../lsp/README.md). It runs configured stdio servers and reads their source documents inside the shared `ctx.e2b` sandbox; the provider registry, normalized query results, queues, and protocol connection state remain on the host. + +## Plugin and configuration + +The `lsp-e2b` plugin injects `e2b`, `lsp`, and the concrete `dsh-subprocess-e2b` service. `servers` is a non-empty provider-id table: + +| Server key | Default | Meaning | +|---|---|---| +| `command` | required | Remote executable, absolute or resolved on the sandbox PATH at load. | +| `args` | `[]` | Remote server arguments. | +| `env` | `{}` | Explicit environment entries passed through the subprocess adapter. | +| `extensionToLanguage` | required | Lowercase leading-dot extension to LSP language id. | +| `initializationOptions` / `configuration` | `null` / `null` | Static initialize options and `workspace/configuration` answer. | +| `maxMessageBytes` | `16000000` | Largest LSP message accepted from the server. | +| `maxStderrBytes` | `1000000` | Retained raw server stderr tail. | +| `maxDocumentBytes` | `4000000` | Largest remote source opened for one query. | +| `shutdownTimeoutMs` | `5000` | Graceful protocol-shutdown budget. | +| `killGraceMs` | `2000` | Request-cancel and TERM-to-KILL grace. | + +Provider ids and commands are non-empty; numeric bounds are positive safe integers, and timer values cannot exceed Node's maximum timer delay. Setup uploads one owner-private proxy under `ctx.e2b.runtimeRoot`, resolves Node and every configured server executable remotely, then registers all providers atomically. + +## Remote protocol and filesystem + +E2B command callbacks are text, while LSP is byte-framed. The installed proxy therefore base64-frames raw server stdout, stderr, and stdin as newline-delimited ASCII JSON; the host validates and decodes every frame before handing bytes to the shared `LspInstance` protocol engine. `initialize.processId` is `null` because host and server do not share a process namespace. + +One language-server process is pooled per provider and canonical remote workspace. Queries serialize per workspace but different workspaces run concurrently. Each query canonicalizes the remote workspace and source with `realpath`, rejects paths outside that workspace, requires a regular file, enforces the size bound before and after reading, decodes strict UTF-8, and uses the ordinary transient `didOpen` / request / `didClose` lifecycle. A transport failure disposes the instance and retries the read-only query once on a fresh remote process. + +The subprocess adapter owns process groups and escalation, so cancellation and disposal await remote server quiescence. The host owns LSP request ids, pending requests, provider queues, and normalized results. + +## Model Experience + +Indirectly, through `@deepseek-ai/dsh-tool-lsp`, which exposes normalized semantic navigation and hover results without changing its model-facing schema. + +#### KV Cache effect + +No direct invalidation; `dsh-tool-lsp` owns request-prefix changes. + +## Known Limitations and Deferred Work + +- **Configured servers only** — this package does not install language servers, select presets, or synchronize a host workspace into E2B. +- **Host protocol state is not reconnectable** — retaining a sandbox does not restore provider queues, JSON-RPC requests, subprocess handles, or document lifecycle state. +- **SDK output retention remains** — ASCII framing preserves protocol bytes, but E2B and the subprocess adapter still retain callback output in host memory. +- **Sandbox policy is template-owned** — this provider adds no volume, snapshot, credential, or network-policy layer. diff --git a/packages/lsp/lsp-e2b/README.zh.md b/packages/lsp/lsp-e2b/README.zh.md new file mode 100644 index 0000000000..2b937f0115 --- /dev/null +++ b/packages/lsp/lsp-e2b/README.zh.md @@ -0,0 +1,47 @@ +# @deepseek-ai/dsh-lsp-e2b + +[English](README.md) | 中文 + +用于 [`ctx.lsp`](../lsp/README.md) 的通用 E2B 语言服务器后端。它在共享的 `ctx.e2b` 沙箱内运行已配置的 stdio 服务器并读取其源文档;提供方注册表、规范化查询结果、队列和协议连接状态仍保留在宿主侧。 + +## 插件与配置 + +`lsp-e2b` 插件注入 `e2b`、`lsp` 和具体的 `dsh-subprocess-e2b` 服务。`servers` 是一张非空的提供方 id 表: + +| 服务器键 | 默认值 | 含义 | +|---|---|---| +| `command` | 必填 | 远程可执行文件:绝对路径,或在加载时通过沙箱 PATH 解析。 | +| `args` | `[]` | 远程服务器参数。 | +| `env` | `{}` | 经由进程管理适配器传入的显式环境条目。 | +| `extensionToLanguage` | 必填 | 小写、以点开头的扩展名到 LSP language id 的映射。 | +| `initializationOptions` / `configuration` | `null` / `null` | 静态初始化选项和 `workspace/configuration` 应答。 | +| `maxMessageBytes` | `16000000` | 从服务器接受的 LSP 消息大小上限。 | +| `maxStderrBytes` | `1000000` | 保留的服务器原始 stderr 尾部上限。 | +| `maxDocumentBytes` | `4000000` | 单次查询可打开的最大远程源文件。 | +| `shutdownTimeoutMs` | `5000` | 协议优雅关闭预算。 | +| `killGraceMs` | `2000` | 请求取消与 TERM 到 KILL 升级的宽限期。 | + +提供方 id 与命令必须非空;数值上限必须是正的安全整数,定时器取值不得超过 Node 的最大定时器延迟。设置阶段会在 `ctx.e2b.runtimeRoot` 下上传一个仅供所有者使用的私有代理,在远程解析 Node 和每个已配置服务器的可执行文件,再以原子方式注册所有提供方。 + +## 远程协议与文件系统 + +E2B 命令回调交付的是文本,而 LSP 按字节分帧。因此,已安装的代理会把服务器 stdout、stderr 和 stdin 的原始字节进行 base64 编码,封装为以换行分隔的 ASCII JSON 帧;宿主会验证并解码每一帧,再把字节交给共享的 `LspInstance` 协议引擎。`initialize.processId` 为 `null`,因为宿主与服务器不共享进程命名空间。 + +每个提供方与规范化远程工作区的组合共享一个池化语言服务器进程。同一工作区的查询串行执行,不同工作区的查询并发运行。每项查询都会使用 `realpath` 规范化远程工作区与源文件,拒绝工作区外的路径,要求源文件为普通文件,在读取前后都检查大小上限,使用严格的 UTF-8 解码,并采用常规的临时 `didOpen`/请求/`didClose` 生命周期。传输失败会 dispose(资源释放)该实例,并在全新的远程进程上重试一次只读查询。 + +进程管理适配器负责进程组和终止升级,因此取消与资源释放都会等待远程服务器完全停稳。宿主负责 LSP 请求 id、待完成请求、提供方队列和规范化结果。 + +## 模型体验 + +通过 `@deepseek-ai/dsh-tool-lsp` 间接影响模型;该包会公开规范化的语义导航与悬停结果,而不改变其面向模型的 schema。 + +#### KV Cache 影响 + +不会直接失效;请求前缀变更由 `dsh-tool-lsp` 负责。 + +## 已知限制与暂缓工作 + +- **仅支持已配置的服务器**:本包不会安装语言服务器、选择 preset,或把宿主工作区同步到 E2B。 +- **宿主协议状态不可重连**:保留沙箱并不会恢复提供方队列、JSON-RPC 请求、进程管理句柄或文档生命周期状态。 +- **SDK 仍会保留输出**:ASCII 分帧能保留协议字节,但 E2B 和进程管理适配器仍会在宿主内存中保留回调输出。 +- **沙箱策略归模板负责**:本提供方不会额外增加卷、快照、凭据或网络策略层。 diff --git a/packages/lsp/lsp-e2b/package.json b/packages/lsp/lsp-e2b/package.json new file mode 100644 index 0000000000..0671341f3c --- /dev/null +++ b/packages/lsp/lsp-e2b/package.json @@ -0,0 +1,52 @@ +{ + "name": "@deepseek-ai/dsh-lsp-e2b", + "description": "E2B language-server provider for DeepSeek Harness", + "version": "0.0.1", + "private": true, + "type": "module", + "main": "lib/index.js", + "types": "lib/types/index.d.ts", + "exports": { + ".": { + "types": "./lib/types/index.d.ts", + "default": "./lib/index.js" + }, + "./invariant": { + "types": "./lib/types/invariant.d.ts", + "default": "./lib/invariant.js" + }, + "./src/*": "./src/*", + "./package.json": "./package.json" + }, + "files": [ + "lib/index.js", + "lib/invariant.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", + "src" + ], + "license": "BSD-3-Clause", + "peerDependencies": { + "@deepseek-ai/dsh-e2b": "^0.0.1", + "@deepseek-ai/dsh-invariants": "^0.0.1", + "@deepseek-ai/dsh-lsp": "^0.0.1", + "@deepseek-ai/dsh-lsp-local": "^0.0.1", + "@deepseek-ai/dsh-subprocess": "^0.0.1", + "@deepseek-ai/dsh-subprocess-e2b": "^0.0.1", + "@deepseek-ai/dsh-timeout": "^0.0.1", + "cordis": "^4.0.0-rc.7" + }, + "dependencies": { + "schemastery": "^3.18.0" + }, + "devDependencies": { + "@deepseek-ai/dsh-e2b": "workspace:^", + "@deepseek-ai/dsh-invariants": "workspace:^", + "@deepseek-ai/dsh-lsp": "workspace:^", + "@deepseek-ai/dsh-lsp-local": "workspace:^", + "@deepseek-ai/dsh-subprocess": "workspace:^", + "@deepseek-ai/dsh-subprocess-e2b": "workspace:^", + "@deepseek-ai/dsh-timeout": "workspace:^", + "cordis": "^4.0.0-rc.7" + } +} diff --git a/packages/lsp/lsp-e2b/src/index.ts b/packages/lsp/lsp-e2b/src/index.ts new file mode 100644 index 0000000000..eb9596db5f --- /dev/null +++ b/packages/lsp/lsp-e2b/src/index.ts @@ -0,0 +1,358 @@ +/** E2B filesystem and process backend for the harness LSP capability seam. */ + +import { Buffer } from 'node:buffer' +import { posix } from 'node:path' +import type { Context } from 'cordis' +import z from 'schemastery' +import { + FileType, + quoteE2BShellArg, + resolveE2BExecutable, +} from '@deepseek-ai/dsh-e2b' +import type { Sandbox } from '@deepseek-ai/dsh-e2b' +import { LspError, LspProviderId } from '@deepseek-ai/dsh-lsp' +import type { LspProvider, LspProviderQuery, LspQueryResult } from '@deepseek-ai/dsh-lsp' +import { LspInstance } from '@deepseek-ai/dsh-lsp-local' +import type { SubprocessSpawnSpec } from '@deepseek-ai/dsh-subprocess' +import E2BSubprocessService from '@deepseek-ai/dsh-subprocess-e2b' +import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout' +import { LSP_PROXY_SOURCE } from './proxy-source.ts' +import { E2BLspTransport } from './transport.ts' + +export { E2BLspTransport } from './transport.ts' + +/** Cordis plugin name. */ +export const name = 'lsp-e2b' +/** Services required by the remote provider. */ +export const inject = ['e2b', 'lsp', 'subprocess'] + +const DEFAULT_MAX_MESSAGE_BYTES = 16_000_000 +const DEFAULT_MAX_STDERR_BYTES = 1_000_000 +const DEFAULT_MAX_DOCUMENT_BYTES = 4_000_000 +const DEFAULT_SHUTDOWN_TIMEOUT_MS = 5_000 +const DEFAULT_KILL_GRACE_MS = 2_000 + +/* jscpd:ignore-start -- Loader requires each backend to expose its own statically walkable server schema. */ +/** One configured language server inside the shared E2B sandbox. */ +export interface LspE2BServerConfig { + /** Remote executable, absolute or resolved on the sandbox PATH. */ + command: string + /** Lowercase leading-dot extension to LSP language id. */ + extensionToLanguage: Record + /** Remote executable arguments. */ + args?: string[] + /** Explicit remote environment overrides. */ + env?: Record + /** Static `initialize` options. */ + initializationOptions?: unknown + /** Static answer to every `workspace/configuration` item. */ + configuration?: unknown + /** Largest LSP message accepted from the server. */ + maxMessageBytes?: number + /** Largest remote stderr tail retained for diagnostics. */ + maxStderrBytes?: number + /** Largest remote source opened for one query. */ + maxDocumentBytes?: number + /** Graceful LSP shutdown budget. */ + shutdownTimeoutMs?: number + /** Request-cancel and TERM-to-KILL grace. */ + killGraceMs?: number +} + +/** Plugin configuration. */ +export interface Config { + /** Non-empty provider-id to remote-server table. */ + servers: Record +} + +type ResolvedServerConfig = Required + +const ServerConfig: z = z.object({ + command: z.string().required(), + args: z.array(String).default([]), + env: z.dict(String).default({}), + extensionToLanguage: z.dict(String).required(), + initializationOptions: z.any().default(null), + configuration: z.any().default(null), + maxMessageBytes: z.number().default(DEFAULT_MAX_MESSAGE_BYTES), + maxStderrBytes: z.number().default(DEFAULT_MAX_STDERR_BYTES), + maxDocumentBytes: z.number().default(DEFAULT_MAX_DOCUMENT_BYTES), + shutdownTimeoutMs: z.number().max(MAX_TIMER_DELAY_MS).default(DEFAULT_SHUTDOWN_TIMEOUT_MS), + killGraceMs: z.number().max(MAX_TIMER_DELAY_MS).default(DEFAULT_KILL_GRACE_MS), +}) + +/** Schemastery plugin configuration. */ +export const Config: z = z.object({ + servers: z.dict(ServerConfig).required(), +}) +/* jscpd:ignore-end */ + +interface RemoteSource { + canonicalPath: string + text: string +} + +function abortReason(signal: AbortSignal): unknown { + try { + signal.throwIfAborted() + } catch (error: unknown) { + return error + } + return new DOMException('The operation was aborted', 'AbortError') +} + +function abortable(promise: Promise, signal?: AbortSignal): Promise { + if (signal === undefined) return promise + // AbortSignal permits opaque reasons, and callers observe the exact reason. + // eslint-disable-next-line @typescript-eslint/prefer-promise-reject-errors + if (signal.aborted) return Promise.reject(abortReason(signal)) + return new Promise((resolve, reject) => { + const onAbort = (): void => { + cleanup() + // eslint-disable-next-line @typescript-eslint/prefer-promise-reject-errors -- Preserve the signal's exact opaque reason. + reject(abortReason(signal)) + } + const cleanup = (): void => { signal.removeEventListener('abort', onAbort) } + signal.addEventListener('abort', onAbort, { once: true }) + void promise.then( + (value) => { cleanup(); resolve(value) }, + (error: unknown) => { cleanup(); reject(error instanceof Error ? error : new Error(String(error))) }, + ) + }) +} + +function validateServerConfig(providerId: string, config: ResolvedServerConfig): void { + if (config.command.length === 0) throw new Error(`lsp-e2b: servers.${providerId}.command must be non-empty`) + for (const name of ['maxMessageBytes', 'maxStderrBytes', 'maxDocumentBytes', 'shutdownTimeoutMs', 'killGraceMs'] as const) { + const value = config[name] + if (!Number.isSafeInteger(value) || value <= 0 || (name.endsWith('Ms') && value > MAX_TIMER_DELAY_MS)) { + throw new Error(`lsp-e2b: servers.${providerId}.${name} must be a positive safe integer${name.endsWith('Ms') ? ` no greater than ${MAX_TIMER_DELAY_MS}` : ''}`) + } + } +} + +async function canonicalRemotePath(sandbox: Sandbox, path: string, signal?: AbortSignal): Promise { + signal?.throwIfAborted() + const result = await sandbox.commands.run(`realpath -e -- ${quoteE2BShellArg(path)}`, signal === undefined ? {} : { signal }) + signal?.throwIfAborted() + const canonical = result.stdout.trim() + if (!posix.isAbsolute(canonical) || canonical.includes('\n')) throw new Error(`remote path ${JSON.stringify(path)} did not resolve canonically`) + return canonical +} + +/** + * Canonicalize and validate one workspace inside E2B. + * @param sandbox - Shared sandbox that owns the workspace. + * @param workspaceRoot - Remote workspace path supplied by the query. + * @param signal - Optional query cancellation signal. + * @returns The canonical remote directory path. + */ +export async function canonicalizeE2BWorkspace( + sandbox: Sandbox, + workspaceRoot: string, + signal?: AbortSignal, +): Promise { + const canonical = await canonicalRemotePath(sandbox, workspaceRoot, signal) + const info = await sandbox.files.getInfo(canonical, signal === undefined ? {} : { signal }) + signal?.throwIfAborted() + if (info.type !== FileType.DIR) throw new Error(`workspace root ${JSON.stringify(workspaceRoot)} is not a directory`) + return canonical +} + +/** + * Resolve, contain, and read one UTF-8 query source inside E2B. + * @param sandbox - Shared sandbox that owns the source. + * @param filePath - Absolute path or path relative to the canonical workspace. + * @param workspace - Canonical remote workspace directory. + * @param maxDocumentBytes - Maximum source size before and after reading. + * @param signal - Optional query cancellation signal. + * @returns The canonical source path and decoded text. + */ +export async function readE2BSource( + sandbox: Sandbox, + filePath: string, + workspace: string, + maxDocumentBytes: number, + signal?: AbortSignal, +): Promise { + const requested = posix.isAbsolute(filePath) ? filePath : posix.resolve(workspace, filePath) + const canonicalPath = await canonicalRemotePath(sandbox, requested, signal) + const relative = posix.relative(workspace, canonicalPath) + if (relative === '..' || relative.startsWith('../') || posix.isAbsolute(relative)) { + throw new Error(`source ${JSON.stringify(filePath)} resolves outside the workspace`) + } + const info = await sandbox.files.getInfo(canonicalPath, signal === undefined ? {} : { signal }) + if (info.type !== FileType.FILE) throw new Error(`source ${JSON.stringify(filePath)} is not a regular file`) + if (info.size > maxDocumentBytes) { + throw new Error(`source ${JSON.stringify(filePath)} is ${info.size} bytes, over the ${maxDocumentBytes}-byte limit`) + } + const bytes = await sandbox.files.read(canonicalPath, { format: 'bytes', ...signal === undefined ? {} : { signal } }) + signal?.throwIfAborted() + if (bytes.length > maxDocumentBytes) { + throw new Error(`source ${JSON.stringify(filePath)} grew past the ${maxDocumentBytes}-byte limit while reading`) + } + let text: string + try { + text = new TextDecoder('utf-8', { fatal: true }).decode(bytes) + } catch (error: unknown) { + throw new Error(`source ${JSON.stringify(filePath)} is not valid UTF-8 text`, { cause: error }) + } + return { canonicalPath, text } +} + +/* jscpd:ignore-start -- Provider identity mirrors the seam while remote source and process ownership stay local. */ +/** One pooled remote provider with an isolated server per canonical workspace. */ +export class E2BLspProvider implements LspProvider { + readonly id: LspProviderId + readonly extensionToLanguage: Readonly> + private readonly instances = new Map() + private readonly queues = new Map>() + private disposed = false + + constructor( + providerId: string, + private readonly sandbox: Sandbox, + private readonly subprocess: E2BSubprocessService, + private readonly config: ResolvedServerConfig, + private readonly executable: string, + private readonly nodeExecutable: string, + private readonly proxyPath: string, + ) { + this.id = LspProviderId(providerId) + this.extensionToLanguage = config.extensionToLanguage + } + /* jscpd:ignore-end */ + + async query(request: LspProviderQuery, signal?: AbortSignal): Promise { + this.assertActive(signal) + const workspace = await canonicalizeE2BWorkspace(this.sandbox, request.workspaceRoot, signal) + this.assertActive(signal) + return this.enqueue(workspace, signal, async () => { + this.assertActive(signal) + const source = await readE2BSource(this.sandbox, request.filePath, workspace, this.config.maxDocumentBytes, signal) + this.assertActive(signal) + let instance = this.instanceFor(workspace) + try { + return await instance.query(request, source, signal) + } catch (error: unknown) { + if (!instance.isTransportFailure(error)) throw error + await instance.dispose() + this.evict(workspace, instance) + this.assertActive(signal) + instance = this.instanceFor(workspace) + return await instance.query(request, source, signal) + } finally { + if (instance.dead) { + await instance.dispose() + this.evict(workspace, instance) + } + } + }) + } + + /* jscpd:ignore-start -- Queue and pooling semantics are shared; transport failure and disposal identities differ. */ + /** Stop accepting work and await every remote server and queued query. */ + async disposeAll(): Promise { + this.disposed = true + const instances = [...this.instances.values()] + const queues = [...this.queues.values()] + this.instances.clear() + await Promise.all([...instances.map(instance => instance.dispose()), ...queues]) + this.queues.clear() + } + + private assertActive(signal?: AbortSignal): void { + if (this.disposed) throw new LspError('lsp-e2b provider is disposed', 'LSP_DISPOSED') + signal?.throwIfAborted() + } + + private enqueue(workspace: string, signal: AbortSignal | undefined, run: () => Promise): Promise { + const previous = this.queues.get(workspace) ?? Promise.resolve() + const result = abortable(previous, signal).then(run) + const tail = previous.then(() => result).then(() => undefined, () => undefined) + this.queues.set(workspace, tail) + void tail.then(() => { if (this.queues.get(workspace) === tail) this.queues.delete(workspace) }) + return result + } + + private instanceFor(workspace: string): LspInstance { + this.assertActive() + const existing = this.instances.get(workspace) + if (existing !== undefined) return existing + const created = this.createInstance(workspace) + this.instances.set(workspace, created) + return created + } + /* jscpd:ignore-end */ + + private createInstance(workspace: string): LspInstance { + return new LspInstance({ + command: this.executable, + args: this.config.args, + cwd: workspace, + env: this.config.env, + configuration: this.config.configuration, + initializationOptions: this.config.initializationOptions, + maxMessageBytes: this.config.maxMessageBytes, + maxStderrBytes: this.config.maxStderrBytes, + shutdownTimeoutMs: this.config.shutdownTimeoutMs, + killGraceMs: this.config.killGraceMs, + clientProcessId: null, + }, (spec: SubprocessSpawnSpec) => { + const originalArgv = Buffer.from(JSON.stringify(spec.argv)).toString('base64') + const inner = this.subprocess.spawn({ + ...spec, + argv: [this.nodeExecutable, this.proxyPath, originalArgv], + stdio: { + stdin: 'pipe', + stdout: 'pipe', + stderr: { maxBytes: this.config.maxStderrBytes }, + }, + }) + const rawBound = Math.max(this.config.maxMessageBytes, this.config.maxStderrBytes) + return new E2BLspTransport(inner, rawBound * 2 + 1024, this.config.maxStderrBytes) + }) + } + + private evict(workspace: string, instance: LspInstance): void { + if (this.instances.get(workspace) === instance) this.instances.delete(workspace) + } +} + +/** Install the proxy, resolve remote commands, and atomically register providers. */ +export async function apply(ctx: Context, config: Config): Promise { + if (!(ctx.subprocess instanceof E2BSubprocessService)) { + throw new Error('lsp-e2b requires @deepseek-ai/dsh-subprocess-e2b as ctx.subprocess') + } + const subprocess = ctx.subprocess + const entries = Object.entries(config.servers) + if (entries.length === 0) throw new Error('lsp-e2b: servers must contain at least one server') + const sandbox = await ctx.e2b.getSandbox() + const proxyPath = posix.join(ctx.e2b.runtimeRoot, 'lsp-stdio-proxy.mjs') + await sandbox.files.write([{ path: proxyPath, data: LSP_PROXY_SOURCE }]) + await sandbox.commands.run(`chmod 600 -- ${quoteE2BShellArg(proxyPath)}`) + const nodeExecutable = await resolveE2BExecutable(sandbox, 'node') + const providers = await Promise.all(entries.map(async ([providerId, raw]) => { + if (providerId.trim() === '') throw new Error('lsp-e2b: server ids must be non-empty strings') + const resolved = raw as ResolvedServerConfig + validateServerConfig(providerId, resolved) + const executable = await resolveE2BExecutable(sandbox, resolved.command) + return new E2BLspProvider(providerId, sandbox, subprocess, resolved, executable, nodeExecutable, proxyPath) + })) + + /* jscpd:ignore-start -- Every provider table publishes atomically through the same registry contract. */ + ctx.effect(() => { + const disposers: Array<() => void> = [] + try { + for (const provider of providers) disposers.push(ctx.lsp.registerProvider(provider)) + } catch (error: unknown) { + for (const dispose of disposers.reverse()) dispose() + throw error + } + return async () => { + for (const dispose of disposers.reverse()) dispose() + await Promise.all(providers.map(provider => provider.disposeAll())) + } + }, 'lsp-e2b.registerProviders') + /* jscpd:ignore-end */ +} diff --git a/packages/lsp/lsp-e2b/src/invariant.ts b/packages/lsp/lsp-e2b/src/invariant.ts new file mode 100644 index 0000000000..129cf0a6d4 --- /dev/null +++ b/packages/lsp/lsp-e2b/src/invariant.ts @@ -0,0 +1,20 @@ +/** Package-owned invariant companion for `@deepseek-ai/dsh-lsp-e2b`. */ + +/* jscpd:ignore-start */ +import type { Context } from 'cordis' +import type { InvariantInstaller } from '@deepseek-ai/dsh-invariants' + +const PACKAGE_NAME = '@deepseek-ai/dsh-lsp-e2b' + +/** Cordis companion plugin name. */ +export const name = 'lsp-e2b-invariant' +/** Service required before the companion can reserve package ownership. */ +export const inject = ['invariants'] + +/** No runtime invariant: the LSP registry owns provider publication. */ +const install: InvariantInstaller = () => {} + +/** Register this package's invariant companion. */ +export const apply = (ctx: Context): Promise<() => void> => + Promise.resolve(ctx.invariants.register(PACKAGE_NAME, install)) +/* jscpd:ignore-end */ diff --git a/packages/lsp/lsp-e2b/src/proxy-source.ts b/packages/lsp/lsp-e2b/src/proxy-source.ts new file mode 100644 index 0000000000..5b6fe42d07 --- /dev/null +++ b/packages/lsp/lsp-e2b/src/proxy-source.ts @@ -0,0 +1,55 @@ +/** Dependency-free remote stdio proxy installed inside the E2B sandbox. */ + +/** + * Node program that base64-frames raw child stdio so E2B's text callbacks + * never decode the language server's byte stream. + */ +export const LSP_PROXY_SOURCE = String.raw`import { Buffer } from 'node:buffer' +import { spawn } from 'node:child_process' +import { createInterface } from 'node:readline' + +const emit = (message) => { + process.stdout.write(Buffer.from(JSON.stringify(message)).toString('base64') + '\n') +} + +let argv +try { + argv = JSON.parse(Buffer.from(process.argv[2], 'base64').toString('utf8')) + if (!Array.isArray(argv) || argv.length === 0 || argv.some(value => typeof value !== 'string')) throw new Error('invalid argv') +} catch (error) { + process.stderr.write('lsp-e2b proxy argv error: ' + String(error) + '\n') + process.exitCode = 125 + process.stdin.destroy() +} + +if (argv) { + const child = spawn(argv[0], argv.slice(1), { stdio: ['pipe', 'pipe', 'pipe'], env: process.env }) + const input = createInterface({ input: process.stdin, crlfDelay: Infinity }) + child.stdout.on('data', data => { emit({ type: 'stdout', data: data.toString('base64') }) }) + child.stderr.on('data', data => { emit({ type: 'stderr', data: data.toString('base64') }) }) + child.on('error', error => { + emit({ type: 'stderr', data: Buffer.from('language server spawn failed: ' + error.message).toString('base64') }) + }) + child.on('close', (code, signal) => { + emit({ type: 'exit', code, signal }) + input.close() + process.stdin.destroy() + process.exitCode = code === null ? 1 : code + }) + input.on('line', line => { + input.pause() + try { + const message = JSON.parse(Buffer.from(line, 'base64').toString('utf8')) + if (!message || message.type !== 'stdin' || typeof message.data !== 'string') throw new Error('invalid stdin frame') + const data = Buffer.from(message.data, 'base64') + if (data.toString('base64') !== message.data) throw new Error('invalid stdin base64') + if (child.stdin.write(data)) input.resume() + else child.stdin.once('drain', () => { input.resume() }) + } catch (error) { + process.stderr.write('lsp-e2b proxy stdin error: ' + String(error) + '\n') + child.kill('SIGTERM') + } + }) + input.on('close', () => { child.stdin.end() }) +} +` diff --git a/packages/lsp/lsp-e2b/src/transport.ts b/packages/lsp/lsp-e2b/src/transport.ts new file mode 100644 index 0000000000..ddd3445db8 --- /dev/null +++ b/packages/lsp/lsp-e2b/src/transport.ts @@ -0,0 +1,183 @@ +/** Byte-faithful stdio transport over an E2B subprocess and ASCII/base64 frames. */ + +import { Buffer } from 'node:buffer' +import { PassThrough, Writable } from 'node:stream' +import { E2BFrameDecoder, encodeE2BFrame } from '@deepseek-ai/dsh-e2b' +import type { + SubprocessHandle, + SubprocessOutcome, + SubprocessOutputRead, + SubprocessOutputReader, +} from '@deepseek-ai/dsh-subprocess' + +function asError(error: unknown): Error { + return error instanceof Error ? error : new Error(String(error)) +} + +class ByteTailReader implements SubprocessOutputReader { + private chunks: Buffer[] = [] + private totalBytes = 0 + private retainedBytes = 0 + private dropped = false + + constructor(private readonly maxBytes: number) {} + + append(data: Buffer): void { + if (data.length === 0) return + this.chunks.push(data) + this.totalBytes += data.length + this.retainedBytes += data.length + while (this.retainedBytes > this.maxBytes && this.chunks.length > 0) { + const first = this.chunks[0] as Buffer + const excess = this.retainedBytes - this.maxBytes + if (first.length <= excess) { + this.chunks.shift() + this.retainedBytes -= first.length + } else { + this.chunks[0] = first.subarray(excess) + this.retainedBytes -= excess + } + this.dropped = true + } + } + + readFrom(fromByte: number): SubprocessOutputRead { + if (!Number.isSafeInteger(fromByte) || fromByte < 0) { + throw new Error('subprocess output offset must be a non-negative safe integer') + } + const retainedStart = this.totalBytes - this.retainedBytes + const lossy = fromByte < retainedStart + const start = lossy ? 0 : Math.min(this.retainedBytes, fromByte - retainedStart) + const bytes = Buffer.concat(this.chunks).subarray(start) + return { text: bytes.toString('utf8'), nextOffset: this.totalBytes, lossy: lossy || this.dropped && fromByte === 0 } + } +} + +class FramedInput extends Writable { + constructor(private readonly target: Writable) { + super() + target.on('error', (error: Error) => { this.destroy(error) }) + } + + override _write(chunk: Buffer, _encoding: BufferEncoding, callback: (error?: Error | null) => void): void { + this.target.write(encodeE2BFrame({ type: 'stdin', data: chunk.toString('base64') }), callback) + } + + override _final(callback: (error?: Error | null) => void): void { + this.target.end(callback) + } +} + +/** Subprocess handle that decodes a remote proxy's stdout/stderr byte frames. */ +export class E2BLspTransport implements SubprocessHandle { + readonly stdin: Writable + readonly stdout = new PassThrough() + readonly stderr = undefined + readonly collected: SubprocessHandle['collected'] + readonly done: Promise + private readonly stderrTail: ByteTailReader + private readonly decoder: E2BFrameDecoder + private failed: Error | undefined + + /** + * @param inner - E2B subprocess running the installed proxy. + * @param maxFrameBytes - Maximum decoded proxy frame size. + * @param maxStderrBytes - Retained raw language-server stderr tail. + */ + constructor( + private readonly inner: SubprocessHandle, + maxFrameBytes: number, + maxStderrBytes: number, + ) { + if (inner.stdin === undefined || inner.stdout === undefined) { + inner.terminate() + throw new Error('lsp-e2b: proxy subprocess dropped a piped stream') + } + this.stdin = new FramedInput(inner.stdin) + this.stderrTail = new ByteTailReader(maxStderrBytes) + this.collected = { stderr: this.stderrTail } + this.decoder = new E2BFrameDecoder(maxFrameBytes) + inner.stdout.on('data', (chunk: Buffer) => { this.onProxyData(chunk) }) + inner.stdout.on('error', (error: Error) => { this.fail(error) }) + this.done = inner.done.then( + (outcome) => { + this.finishFrames() + this.captureProxyStderr() + this.stdout.end() + if (this.failed !== undefined) throw this.failed + return outcome + }, + (error: unknown) => { + this.captureProxyStderr() + this.stdout.end() + throw error + }, + ) + void this.done.catch(() => {}) + } + + get pid(): number { + return this.inner.pid + } + + terminate(): void { + this.inner.terminate() + } + + async waitForExit(signal?: AbortSignal): Promise { + return await this.inner.waitForExit(signal) + } + + private onProxyData(chunk: Buffer): void { + if (this.failed !== undefined) return + let frames: unknown[] + try { + frames = this.decoder.push(chunk.toString('utf8')) + } catch (error: unknown) { + this.fail(asError(error)) + return + } + for (const frame of frames) this.dispatch(frame) + } + + private dispatch(frame: unknown): void { + if (typeof frame !== 'object' || frame === null) { + this.fail(new Error('lsp-e2b: proxy emitted a malformed frame')) + return + } + const record = frame as Record + if (record.type === 'exit' && (record.code === null || typeof record.code === 'number') && (record.signal === null || typeof record.signal === 'string')) return + if ((record.type !== 'stdout' && record.type !== 'stderr') || typeof record.data !== 'string') { + this.fail(new Error('lsp-e2b: proxy emitted a malformed frame')) + return + } + const data = Buffer.from(record.data, 'base64') + if (data.toString('base64') !== record.data) { + this.fail(new Error('lsp-e2b: proxy emitted invalid base64')) + return + } + if (record.type === 'stdout') this.stdout.write(data) + else this.stderrTail.append(data) + } + + private finishFrames(): void { + if (this.failed !== undefined) return + try { + this.decoder.finish() + } catch (error: unknown) { + this.fail(asError(error)) + } + } + + private captureProxyStderr(): void { + const diagnostic = this.inner.collected.stderr?.readFrom(0).text + if (diagnostic !== undefined && diagnostic.length > 0) this.stderrTail.append(Buffer.from(diagnostic)) + } + + private fail(error: Error): void { + if (this.failed !== undefined) return + this.failed = error + this.inner.terminate() + this.stdout.end() + } +} diff --git a/packages/lsp/lsp-e2b/tests/provider.spec.ts b/packages/lsp/lsp-e2b/tests/provider.spec.ts new file mode 100644 index 0000000000..0b41be12a3 --- /dev/null +++ b/packages/lsp/lsp-e2b/tests/provider.spec.ts @@ -0,0 +1,397 @@ +import { PassThrough } from 'node:stream' +import { Context } from 'cordis' +import { beforeEach, describe, expect, it, vi } from 'vitest' +import { + FileType, + type Sandbox, +} from '@deepseek-ai/dsh-e2b' +import type { SubprocessHandle, SubprocessSpawnSpec } from '@deepseek-ai/dsh-subprocess' +import E2BSubprocessService from '@deepseek-ai/dsh-subprocess-e2b' + +const mockedLsp = vi.hoisted(() => { + interface Plan { + query?: (...args: unknown[]) => unknown + transportFailure?: unknown + dead?: boolean + deadAfterQuery?: boolean + disposeError?: unknown + } + + class FakeLspInstance { + static readonly instances: FakeLspInstance[] = [] + static readonly plans: Plan[] = [] + readonly plan: Plan + readonly transport: unknown + readonly queries: unknown[][] = [] + dead: boolean + disposals = 0 + + constructor( + readonly spec: Record, + spawner: (spec: SubprocessSpawnSpec) => unknown, + ) { + this.plan = FakeLspInstance.plans.shift() ?? {} + this.dead = this.plan.dead === true + this.transport = spawner({ + argv: [String(spec.command), ...(spec.args as string[])], + cwd: String(spec.cwd), + env: spec.env as Record, + graceMs: Number(spec.killGraceMs), + stdio: { stdin: 'pipe', stdout: 'pipe', stderr: { maxBytes: Number(spec.maxStderrBytes) } }, + }) + FakeLspInstance.instances.push(this) + } + + async query(...args: unknown[]): Promise { + this.queries.push(args) + const result = await Promise.resolve(this.plan.query?.(...args) ?? { kind: 'hover', hover: null }) + if (this.plan.deadAfterQuery === true) this.dead = true + return result + } + + isTransportFailure(error: unknown): boolean { + return error === this.plan.transportFailure + } + + async dispose(): Promise { + this.disposals += 1 + this.dead = true + if (this.plan.disposeError !== undefined) throw this.plan.disposeError + } + } + + return { FakeLspInstance } +}) + +vi.mock('@deepseek-ai/dsh-lsp-local', () => ({ LspInstance: mockedLsp.FakeLspInstance })) + +import { + E2BLspProvider, + apply, + canonicalizeE2BWorkspace, + readE2BSource, +} from '@deepseek-ai/dsh-lsp-e2b' +import type { LspE2BServerConfig } from '@deepseek-ai/dsh-lsp-e2b' +import * as E2BLspInvariant from '../src/invariant.ts' +import InvariantService from '@deepseek-ai/dsh-invariants' + +class FakeInnerHandle implements SubprocessHandle { + readonly pid = 777 + readonly stdin = new PassThrough() + readonly stdout = new PassThrough() + readonly stderr = undefined + readonly collected = { stderr: { readFrom: () => ({ text: '', nextOffset: 0, lossy: false }) } } + readonly done = Promise.resolve({ exitCode: 0, signal: null }) + terminate(): void {} + async waitForExit(): Promise { return true } +} + +class FakeRemote { + readonly writes: Array> = [] + readonly commands: string[] = [] + readonly infos = new Map() + readonly contents = new Map() + readonly realpaths = new Map() + forcedRealpath: string | undefined + + constructor() { + this.infos.set('/workspace', { type: FileType.DIR, size: 0 }) + this.infos.set('/workspace/file.ts', { type: FileType.FILE, size: 12 }) + this.contents.set('/workspace/file.ts', Buffer.from('const x = 1')) + } + + readonly sandbox = { + commands: { + run: async (command: string) => { + this.commands.push(command) + if (command.startsWith('realpath ')) { + const match = /'([^']*)'$/.exec(command) + const requested = match?.[1] ?? '' + return { exitCode: 0, stdout: `${this.forcedRealpath ?? this.realpaths.get(requested) ?? requested}\n`, stderr: '' } + } + if (command.startsWith('command -v')) return { exitCode: 0, stdout: '/usr/bin/node\n', stderr: '' } + return { exitCode: 0, stdout: '', stderr: '' } + }, + }, + files: { + write: async (files: Array<{ path: string; data: string }>) => { + this.writes.push(files) + return files.map(() => ({})) + }, + getInfo: async (path: string) => { + const info = this.infos.get(path) + if (info === undefined) throw new Error(`missing info for ${path}`) + return info + }, + read: async (path: string) => this.contents.get(path) ?? new Uint8Array(), + }, + } as unknown as Sandbox +} + +function subprocess(spawn = vi.fn((_spec: SubprocessSpawnSpec) => new FakeInnerHandle())): E2BSubprocessService { + const service = Object.create(E2BSubprocessService.prototype) as E2BSubprocessService + Object.defineProperty(service, 'spawn', { value: spawn }) + return service +} + +function server(overrides: Partial = {}): Required { + return { + command: '/usr/bin/server', args: ['--stdio'], env: {}, + extensionToLanguage: { '.ts': 'typescript' }, + initializationOptions: null, configuration: null, + maxMessageBytes: 1_024, maxStderrBytes: 128, maxDocumentBytes: 1_024, + shutdownTimeoutMs: 100, killGraceMs: 50, + ...overrides, + } +} + +function provider(remote = new FakeRemote(), service = subprocess()): E2BLspProvider { + return new E2BLspProvider( + 'fixture', remote.sandbox, service, server(), + '/usr/bin/server', '/usr/bin/node', '/workspace/.dsh-e2b/lsp-proxy.mjs', + ) +} + +function query(workspaceRoot = '/workspace') { + return { + operation: 'hover' as const, + filePath: 'file.ts', + position: { line: 0, character: 1 }, + workspaceRoot, + languageId: 'typescript', + } +} + +beforeEach(() => { + mockedLsp.FakeLspInstance.instances.length = 0 + mockedLsp.FakeLspInstance.plans.length = 0 +}) + +describe('E2B LSP filesystem boundary', () => { + it('canonicalizes a directory and reads a contained UTF-8 source', async () => { + const remote = new FakeRemote() + await expect(canonicalizeE2BWorkspace(remote.sandbox, '/workspace')).resolves.toBe('/workspace') + await expect(readE2BSource(remote.sandbox, 'file.ts', '/workspace', 1_024)).resolves.toEqual({ + canonicalPath: '/workspace/file.ts', + text: 'const x = 1', + }) + await expect(readE2BSource(remote.sandbox, '/workspace/file.ts', '/workspace', 1_024)).resolves.toMatchObject({ + canonicalPath: '/workspace/file.ts', + }) + const signal = new AbortController().signal + await expect(canonicalizeE2BWorkspace(remote.sandbox, '/workspace', signal)).resolves.toBe('/workspace') + await expect(readE2BSource(remote.sandbox, 'file.ts', '/workspace', 1_024, signal)).resolves.toMatchObject({ + canonicalPath: '/workspace/file.ts', + }) + }) + + it('rejects malformed workspaces and source containment/type/size/encoding failures', async () => { + const malformed = new FakeRemote() + malformed.forcedRealpath = 'relative' + await expect(canonicalizeE2BWorkspace(malformed.sandbox, '/workspace')).rejects.toThrow('did not resolve canonically') + malformed.forcedRealpath = '/workspace\nother' + await expect(canonicalizeE2BWorkspace(malformed.sandbox, '/workspace')).rejects.toThrow('did not resolve canonically') + + const notDirectory = new FakeRemote() + notDirectory.infos.set('/workspace', { type: FileType.FILE, size: 0 }) + await expect(canonicalizeE2BWorkspace(notDirectory.sandbox, '/workspace')).rejects.toThrow('not a directory') + + const outside = new FakeRemote() + outside.realpaths.set('/workspace/file.ts', '/outside/file.ts') + await expect(readE2BSource(outside.sandbox, 'file.ts', '/workspace', 20)).rejects.toThrow('outside the workspace') + + const notFile = new FakeRemote() + notFile.infos.set('/workspace/file.ts', { type: FileType.DIR, size: 0 }) + await expect(readE2BSource(notFile.sandbox, 'file.ts', '/workspace', 20)).rejects.toThrow('not a regular file') + + const tooLarge = new FakeRemote() + tooLarge.infos.set('/workspace/file.ts', { type: FileType.FILE, size: 21 }) + await expect(readE2BSource(tooLarge.sandbox, 'file.ts', '/workspace', 20)).rejects.toThrow('over the 20-byte limit') + + const grew = new FakeRemote() + grew.infos.set('/workspace/file.ts', { type: FileType.FILE, size: 1 }) + grew.contents.set('/workspace/file.ts', Buffer.alloc(21)) + await expect(readE2BSource(grew.sandbox, 'file.ts', '/workspace', 20)).rejects.toThrow('grew past') + + const invalid = new FakeRemote() + invalid.infos.set('/workspace/file.ts', { type: FileType.FILE, size: 1 }) + invalid.contents.set('/workspace/file.ts', Uint8Array.from([0xff])) + await expect(readE2BSource(invalid.sandbox, 'file.ts', '/workspace', 20)).rejects.toThrow('not valid UTF-8') + + await expect(canonicalizeE2BWorkspace(new FakeRemote().sandbox, '/workspace', AbortSignal.abort('stop'))) + .rejects.toBe('stop') + }) +}) + +describe('E2BLspProvider pooling and lifecycle', () => { + it('reuses one canonical-workspace instance and constructs the remote proxy transport', async () => { + const spawn = vi.fn((_spec: SubprocessSpawnSpec) => new FakeInnerHandle()) + const remote = new FakeRemote() + mockedLsp.FakeLspInstance.plans.push({ query: async () => ({ kind: 'hover', hover: { contents: 'ok' } }) }) + const current = provider(remote, subprocess(spawn)) + + await expect(current.query(query())).resolves.toEqual({ kind: 'hover', hover: { contents: 'ok' } }) + await expect(current.query(query())).resolves.toEqual({ kind: 'hover', hover: { contents: 'ok' } }) + expect(mockedLsp.FakeLspInstance.instances).toHaveLength(1) + expect(mockedLsp.FakeLspInstance.instances[0]?.spec).toMatchObject({ clientProcessId: null, cwd: '/workspace' }) + expect(spawn).toHaveBeenCalledWith(expect.objectContaining({ + argv: ['/usr/bin/node', '/workspace/.dsh-e2b/lsp-proxy.mjs', expect.any(String)], + stdio: { stdin: 'pipe', stdout: 'pipe', stderr: { maxBytes: 128 } }, + })) + expect(current.id).toBe('fixture') + expect(current.extensionToLanguage).toEqual({ '.ts': 'typescript' }) + await current.disposeAll() + expect(mockedLsp.FakeLspInstance.instances[0]?.disposals).toBe(1) + }) + + it('replaces one transport failure, but preserves ordinary query errors', async () => { + const transportFailure = new Error('transport failed') + mockedLsp.FakeLspInstance.plans.push( + { transportFailure, query: async () => { throw transportFailure } }, + { query: async () => ({ kind: 'hover', hover: { contents: 'retried' } }) }, + ) + const retried = provider() + await expect(retried.query(query())).resolves.toMatchObject({ hover: { contents: 'retried' } }) + expect(mockedLsp.FakeLspInstance.instances[0]?.disposals).toBe(1) + expect(mockedLsp.FakeLspInstance.instances).toHaveLength(2) + + const ordinary = new Error('ordinary failure') + mockedLsp.FakeLspInstance.plans.push( + { query: async () => { throw ordinary }, dead: true }, + { query: async () => ({ kind: 'hover', hover: null }) }, + ) + const failed = provider() + await expect(failed.query(query())).rejects.toBe(ordinary) + await expect(failed.query(query())).resolves.toMatchObject({ kind: 'hover' }) + }) + + it('evicts a server that dies after a successful query', async () => { + mockedLsp.FakeLspInstance.plans.push( + { deadAfterQuery: true, query: async () => ({ kind: 'hover', hover: null }) }, + { query: async () => ({ kind: 'hover', hover: null }) }, + ) + const current = provider() + await current.query(query()) + await current.query(query()) + expect(mockedLsp.FakeLspInstance.instances).toHaveLength(2) + expect(mockedLsp.FakeLspInstance.instances[0]?.disposals).toBe(1) + }) + + it('serializes a workspace queue, observes queued abort, and awaits work on disposal', async () => { + const first = Promise.withResolvers() + mockedLsp.FakeLspInstance.plans.push({ query: () => first.promise }) + const current = provider() + const running = current.query(query()) + const controller = new AbortController() + const queued = current.query(query(), controller.signal) + await new Promise(resolve => setImmediate(resolve)) + controller.abort('queued stop') + await expect(queued).rejects.toBe('queued stop') + + const disposing = current.disposeAll() + first.resolve({ kind: 'hover', hover: null }) + await expect(running).resolves.toMatchObject({ kind: 'hover' }) + await disposing + await expect(current.query(query())).rejects.toMatchObject({ code: 'LSP_DISPOSED' }) + }) + + it('covers pre-abort, synthetic abort, resolve, and rejection in the queue race', async () => { + const current = provider() + const internal = current as unknown as { + queues: Map> + enqueue(workspace: string, signal: AbortSignal | undefined, run: () => Promise): Promise + } + await expect(internal.enqueue('pre', AbortSignal.abort('pre-stop'), async () => 'unused')).rejects.toBe('pre-stop') + + const signal = new AbortController().signal + await expect(internal.enqueue('resolve', signal, async () => 'ok')).resolves.toBe('ok') + + const failure = new Error('queue failed') + const rejected = Promise.reject(failure) + void rejected.catch(() => {}) + internal.queues.set('reject', rejected) + await expect(internal.enqueue('reject', signal, async () => 'unused')).rejects.toBe(failure) + + // eslint-disable-next-line @typescript-eslint/prefer-promise-reject-errors -- Exercise normalization at the promise boundary. + const opaque = Promise.reject('opaque queue failure') + void opaque.catch(() => {}) + internal.queues.set('opaque', opaque) + await expect(internal.enqueue('opaque', signal, async () => 'unused')).rejects.toEqual(new Error('opaque queue failure')) + + const synthetic = { + aborted: false, + reason: undefined, + throwIfAborted() {}, + addEventListener(_type: string, listener: () => void) { listener() }, + removeEventListener() {}, + } as unknown as AbortSignal + await expect(internal.enqueue('synthetic', synthetic, async () => 'unused')).rejects.toMatchObject({ name: 'AbortError' }) + await current.disposeAll() + }) +}) + +describe('lsp-e2b plugin composition', () => { + function pluginContext( + remote: FakeRemote, + service: E2BSubprocessService, + registerProvider = vi.fn(() => vi.fn()), + ) { + const effects: Array<() => void | Promise> = [] + const ctx = { + subprocess: service, + e2b: { + runtimeRoot: '/workspace/.dsh-e2b', + getSandbox: async () => remote.sandbox, + }, + lsp: { registerProvider }, + effect: (callback: () => (() => void | Promise)) => { effects.push(callback()) }, + } as unknown as Context + return { ctx, effects, registerProvider } + } + + it('installs one proxy, resolves commands, registers providers, and disposes them', async () => { + const remote = new FakeRemote() + const fixture = pluginContext(remote, subprocess()) + await apply(fixture.ctx, { servers: { one: server(), two: server({ command: 'server-two' }) } }) + expect(remote.writes).toHaveLength(1) + expect(remote.writes[0]?.[0]?.path).toBe('/workspace/.dsh-e2b/lsp-stdio-proxy.mjs') + expect(remote.commands).toContain("chmod 600 -- '/workspace/.dsh-e2b/lsp-stdio-proxy.mjs'") + expect(fixture.registerProvider).toHaveBeenCalledTimes(2) + await fixture.effects[0]?.() + }) + + it('rolls back partial registration and rejects invalid composition/configuration', async () => { + const remote = new FakeRemote() + const firstDispose = vi.fn() + const register = vi.fn() + .mockReturnValueOnce(firstDispose) + .mockImplementationOnce(() => { throw new Error('duplicate provider') }) + const rollback = pluginContext(remote, subprocess(), register) + await expect(apply(rollback.ctx, { servers: { one: server(), two: server() } })).rejects.toThrow('duplicate provider') + expect(firstDispose).toHaveBeenCalledOnce() + + const wrong = pluginContext(remote, {} as E2BSubprocessService) + await expect(apply(wrong.ctx, { servers: { one: server() } })).rejects.toThrow('dsh-subprocess-e2b') + + const empty = pluginContext(remote, subprocess()) + await expect(apply(empty.ctx, { servers: {} })).rejects.toThrow('at least one server') + + for (const [id, config] of [ + ['', server()], + ['one', server({ command: '' })], + ['one', server({ maxMessageBytes: 0 })], + ['one', server({ maxStderrBytes: 1.5 })], + ['one', server({ shutdownTimeoutMs: 2_147_483_648 })], + ] as const) { + const fixture = pluginContext(new FakeRemote(), subprocess()) + await expect(apply(fixture.ctx, { servers: { [id]: config } })).rejects.toThrow() + } + }) + + it('registers the package-owned invariant companion', async () => { + const ctx = new Context() + await ctx.plugin(InvariantService, { enabled: true }) + const fiber = await ctx.plugin(E2BLspInvariant).await() + await fiber.dispose() + }) +}) diff --git a/packages/lsp/lsp-e2b/tests/transport.spec.ts b/packages/lsp/lsp-e2b/tests/transport.spec.ts new file mode 100644 index 0000000000..ae5106bc09 --- /dev/null +++ b/packages/lsp/lsp-e2b/tests/transport.spec.ts @@ -0,0 +1,181 @@ +import { once } from 'node:events' +import { PassThrough } from 'node:stream' +import { describe, expect, it } from 'vitest' +import { encodeE2BFrame } from '@deepseek-ai/dsh-e2b' +import type { SubprocessHandle, SubprocessOutcome } from '@deepseek-ai/dsh-subprocess' +import { E2BLspTransport } from '@deepseek-ai/dsh-lsp-e2b' + +class FakeHandle implements SubprocessHandle { + readonly pid = 321 + readonly stdin: PassThrough | undefined + readonly stdout: PassThrough | undefined + readonly stderr = undefined + readonly collected: SubprocessHandle['collected'] + readonly result = Promise.withResolvers() + readonly done = this.result.promise + terminated = 0 + waitResult = true + + constructor(options: { stdin?: boolean; stdout?: boolean; diagnostic?: string } = {}) { + this.stdin = options.stdin === false ? undefined : new PassThrough() + this.stdout = options.stdout === false ? undefined : new PassThrough() + this.collected = options.diagnostic === undefined + ? {} + : { stderr: { readFrom: () => ({ text: options.diagnostic as string, nextOffset: 0, lossy: false }) } } + } + + terminate(): void { + this.terminated += 1 + } + + async waitForExit(): Promise { + return this.waitResult + } + + resolve(outcome: SubprocessOutcome = { exitCode: 0, signal: null }): void { + this.stdout?.end() + this.result.resolve(outcome) + } + + reject(error: unknown): void { + this.stdout?.end() + this.result.reject(error) + } +} + +function proxyFrame(type: 'stdout' | 'stderr', data: string | Buffer): string { + return encodeE2BFrame({ type, data: Buffer.from(data).toString('base64') }) +} + +describe('E2BLspTransport', () => { + it('frames stdin, decodes split byte output, and exposes handle lifecycle', async () => { + const inner = new FakeHandle({ diagnostic: 'proxy diagnostic' }) + const transport = new E2BLspTransport(inner, 1_024, 64) + let stdin = '' + inner.stdin?.on('data', (chunk) => { stdin += String(chunk) }) + let stdout = Buffer.alloc(0) + transport.stdout.on('data', (chunk) => { stdout = Buffer.concat([stdout, chunk]) }) + + transport.stdin.write(Buffer.from([0, 0xff])) + await new Promise(resolve => setImmediate(resolve)) + const encodedInput = stdin.trim() + const input = JSON.parse(Buffer.from(encodedInput, 'base64').toString('utf8')) as Record + expect(input).toEqual({ type: 'stdin', data: 'AP8=' }) + + const frames = proxyFrame('stdout', Buffer.from([0, 0xff])) + + proxyFrame('stderr', 'server diagnostic') + + encodeE2BFrame({ type: 'exit', code: 0, signal: null }) + inner.stdout?.write(frames.slice(0, 7)) + inner.stdout?.write(frames.slice(7)) + inner.stdout?.write(proxyFrame('stderr', '')) + inner.stdout?.write(encodeE2BFrame({ type: 'exit', code: null, signal: 'SIGTERM' })) + inner.resolve() + + await expect(transport.done).resolves.toEqual({ exitCode: 0, signal: null }) + expect(stdout).toEqual(Buffer.from([0, 0xff])) + expect(transport.collected.stderr?.readFrom(0).text).toBe('server diagnosticproxy diagnostic') + expect(transport.pid).toBe(321) + inner.waitResult = false + await expect(transport.waitForExit()).resolves.toBe(false) + transport.terminate() + expect(inner.terminated).toBe(1) + }) + + it('ends the inner stdin and retains a bounded byte tail with independent offsets', async () => { + const inner = new FakeHandle() + const transport = new E2BLspTransport(inner, 1_024, 4) + const finished = once(inner.stdin!, 'finish') + transport.stdin.end() + await finished + + inner.stdout?.write(proxyFrame('stderr', 'ab')) + inner.stdout?.write(proxyFrame('stderr', 'cdef')) + const reader = transport.collected.stderr! + expect(reader.readFrom(0)).toEqual({ text: 'cdef', nextOffset: 6, lossy: true }) + expect(reader.readFrom(2)).toEqual({ text: 'cdef', nextOffset: 6, lossy: false }) + expect(reader.readFrom(5)).toEqual({ text: 'f', nextOffset: 6, lossy: false }) + expect(reader.readFrom(99)).toEqual({ text: '', nextOffset: 6, lossy: false }) + expect(() => reader.readFrom(-1)).toThrow('non-negative safe integer') + expect(() => reader.readFrom(1.5)).toThrow('non-negative safe integer') + inner.resolve() + await transport.done + + const partialInner = new FakeHandle() + const partial = new E2BLspTransport(partialInner, 1_024, 4) + partialInner.stdout?.write(proxyFrame('stderr', 'abcdef')) + expect(partial.collected.stderr?.readFrom(0)).toEqual({ text: 'cdef', nextOffset: 6, lossy: true }) + partialInner.resolve() + await partial.done + }) + + it.each([ + ['non-object', encodeE2BFrame(null), 'malformed frame'], + ['wrong type', encodeE2BFrame({ type: 'other', data: '' }), 'malformed frame'], + ['invalid exit', encodeE2BFrame({ type: 'exit', code: 'zero', signal: null }), 'malformed frame'], + ['missing data', encodeE2BFrame({ type: 'stdout' }), 'malformed frame'], + ['invalid base64', encodeE2BFrame({ type: 'stdout', data: 'abc' }), 'invalid base64'], + ['invalid outer frame', 'not-base64\n', 'invalid base64'], + ['non-ASCII outer frame', 'é', 'non-ASCII'], + ])('fails %s proxy traffic and terminates the group', async (_name, frame, message) => { + const inner = new FakeHandle() + const transport = new E2BLspTransport(inner, 1_024, 32) + inner.stdout?.write(frame) + inner.stdout?.write(proxyFrame('stdout', 'ignored')) + inner.stdout?.emit('error', new Error('second failure')) + inner.resolve() + await expect(transport.done).rejects.toThrow(message) + expect(inner.terminated).toBe(1) + }) + + it('fails truncated frames and preserves inner spawn rejection', async () => { + const truncatedInner = new FakeHandle() + const truncated = new E2BLspTransport(truncatedInner, 1_024, 32) + truncatedInner.stdout?.write('YQ==') + truncatedInner.resolve() + await expect(truncated.done).rejects.toThrow('mid-frame') + + const rejectedInner = new FakeHandle({ diagnostic: 'tail' }) + const rejected = new E2BLspTransport(rejectedInner, 1_024, 32) + rejectedInner.reject(new Error('spawn failed')) + await expect(rejected.done).rejects.toThrow('spawn failed') + expect(rejected.collected.stderr?.readFrom(0).text).toBe('tail') + }) + + it('forwards output and input stream errors without an unhandled inner error', async () => { + const outputInner = new FakeHandle() + const output = new E2BLspTransport(outputInner, 1_024, 32) + outputInner.stdout?.emit('error', new Error('proxy stdout failed')) + outputInner.resolve() + await expect(output.done).rejects.toThrow('proxy stdout failed') + + const inputInner = new FakeHandle() + const input = new E2BLspTransport(inputInner, 1_024, 32) + const outerError = once(input.stdin, 'error') + inputInner.stdin?.emit('error', new Error('proxy stdin failed')) + await expect(outerError).resolves.toMatchObject([{ message: 'proxy stdin failed' }]) + inputInner.resolve() + await input.done + }) + + it('normalizes a non-Error decoder throw', async () => { + const inner = new FakeHandle() + const transport = new E2BLspTransport(inner, 1_024, 32) + const internal = transport as unknown as { + decoder: { push(chunk: string): unknown[] } + onProxyData(chunk: Buffer): void + } + internal.decoder = { push: () => { throw 'raw decoder failure' } } + internal.onProxyData(Buffer.from('x')) + inner.resolve() + await expect(transport.done).rejects.toThrow('raw decoder failure') + }) + + it('rejects a subprocess that drops either required pipe', () => { + const missingStdin = new FakeHandle({ stdin: false }) + const missingStdout = new FakeHandle({ stdout: false }) + expect(() => new E2BLspTransport(missingStdin, 10, 10)).toThrow('dropped a piped stream') + expect(() => new E2BLspTransport(missingStdout, 10, 10)).toThrow('dropped a piped stream') + expect(missingStdin.terminated).toBe(1) + expect(missingStdout.terminated).toBe(1) + }) +}) diff --git a/packages/lsp/lsp-e2b/tsconfig.json b/packages/lsp/lsp-e2b/tsconfig.json new file mode 100644 index 0000000000..4679845db8 --- /dev/null +++ b/packages/lsp/lsp-e2b/tsconfig.json @@ -0,0 +1,20 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": ["src"], + "references": [ + { "path": "../../../vendor/cosmokit" }, + { "path": "../../../vendor/cordis" }, + { "path": "../../../vendor/schemastery" }, + { "path": "../../e2b/e2b" }, + { "path": "../lsp" }, + { "path": "../lsp-local" }, + { "path": "../../subprocess/subprocess" }, + { "path": "../../subprocess/subprocess-e2b" }, + { "path": "../../util/timeout" }, + { "path": "../../support/invariants" } + ] +} diff --git a/packages/lsp/lsp-local/src/instance.ts b/packages/lsp/lsp-local/src/instance.ts index 318704b9fc..d643db1528 100644 --- a/packages/lsp/lsp-local/src/instance.ts +++ b/packages/lsp/lsp-local/src/instance.ts @@ -7,6 +7,7 @@ * @module @deepseek-ai/dsh-lsp-local/instance */ +import { pathToFileURL } from 'node:url' import { LspError } from '@deepseek-ai/dsh-lsp' import type { LspOperation, @@ -30,12 +31,12 @@ import { /** Everything an instance needs beyond the connection spec. */ export interface InstanceSpec extends ConnectionSpec { - /** Canonical workspace file URI supplied by the filesystem provider. */ - readonly workspaceUri: string /** Static `initialize` options forwarded to the server. */ readonly initializationOptions: unknown /** Graceful `shutdown`/`exit` budget before escalation (ms). */ readonly shutdownTimeoutMs: number + /** PID advertised to the server; `null` when client and server do not share a process namespace. */ + readonly clientProcessId?: number | null } /** @@ -109,11 +110,9 @@ export class LspInstance { private async initialize(): Promise { const initializeResult = await this.connection.request('initialize', { - // A subprocess provider may run in another PID namespace or machine; - // the host PID would let the server monitor an unrelated process. - processId: null, - rootUri: this.spec.workspaceUri, - workspaceFolders: [{ uri: this.spec.workspaceUri, name: 'workspace' }], + processId: this.spec.clientProcessId === undefined ? process.pid : this.spec.clientProcessId, + rootUri: pathToFileURL(this.spec.cwd).href, + workspaceFolders: [{ uri: pathToFileURL(this.spec.cwd).href, name: 'workspace' }], capabilities: CLIENT_CAPABILITIES, initializationOptions: this.spec.initializationOptions, }) as WireInitializeResult @@ -150,7 +149,7 @@ export class LspInstance { throw new LspError('server does not support the transient textDocument/didOpen this host requires', 'LSP_UNSUPPORTED_OPERATION') } - const uri = source.fileUrl + const uri = pathToFileURL(source.canonicalPath).href let opened = false try { /* v8 ignore next -- guards an abort landing between the ready wait and didOpen; not deterministically reproducible. */ @@ -244,9 +243,10 @@ export class LspInstance { if (operation === 'hover') { return { kind: 'hover', hover: normalizeHover(payload) } } - // The filesystem provider owns URI syntax for the execution platform, which may differ from the - // harness host. Preserve that coordinate through rendering instead of reparsing `spec.cwd` there. - return { kind: 'locations', locations: normalizeLocations(payload), resolvedWorkspaceUri: this.spec.workspaceUri } + // `spec.cwd` is the canonical workspace realpath (the provider canonicalizes before spawning), + // and every `file:` location URI is relative to it — so it is the root a caller must relativize + // display paths against, not the request's possibly-symlinked workspaceRoot. + return { kind: 'locations', locations: normalizeLocations(payload), resolvedWorkspaceRoot: this.spec.cwd } } private answerServerRequest(method: string, params: unknown): Promise { diff --git a/packages/lsp/lsp-local/tests/fixture-server.ts b/packages/lsp/lsp-local/tests/fixture-server.ts index 9399b5a1a7..89053894ee 100644 --- a/packages/lsp/lsp-local/tests/fixture-server.ts +++ b/packages/lsp/lsp-local/tests/fixture-server.ts @@ -15,6 +15,7 @@ * - LSP_FAKE_REPLY_DELAY_MS: delays each textDocument/* response by this many milliseconds. * - LSP_FAKE_OPEN_MARKER: appends each didOpen document text as one JSON line to this path. * - LSP_FAKE_INITIALIZED_MARKER: records when the initialized notification is received. + * - LSP_FAKE_EXPECT_PROCESS_ID: expected JSON `initialize.processId`; mismatch exits nonzero. * - LSP_FAKE_PAUSE_STDIN_AFTER_INITIALIZED: "1" stops consuming stdin after initialized. * - LSP_FAKE_EXIT_DELAY_MS / LSP_FAKE_EXIT_MARKER: delay protocol exit and record exit/termination. * - LSP_FAKE_NO_SHUTDOWN: "1" ignores the shutdown request (forces kill escalation). @@ -37,6 +38,7 @@ const exitAfterReply = process.env.LSP_FAKE_EXIT_AFTER_REPLY === '1' const replyDelayMs = Number(process.env.LSP_FAKE_REPLY_DELAY_MS ?? 0) const openMarker = process.env.LSP_FAKE_OPEN_MARKER const initializedMarker = process.env.LSP_FAKE_INITIALIZED_MARKER +const expectedProcessId = process.env.LSP_FAKE_EXPECT_PROCESS_ID const pauseStdinAfterInitialized = process.env.LSP_FAKE_PAUSE_STDIN_AFTER_INITIALIZED === '1' const exitDelayMs = Number(process.env.LSP_FAKE_EXIT_DELAY_MS ?? 0) const exitMarker = process.env.LSP_FAKE_EXIT_MARKER @@ -102,6 +104,10 @@ function handle(message: { id?: number; method?: string; params?: unknown; resul return } if (method === 'initialize') { + if (expectedProcessId !== undefined) { + const params = message.params as { processId?: unknown } | undefined + if (JSON.stringify(params?.processId) !== expectedProcessId) process.exit(2) + } if (garbage) process.stdout.write('this is not a framed message\r\n') send({ id, diff --git a/packages/lsp/lsp-local/tests/instance.spec.ts b/packages/lsp/lsp-local/tests/instance.spec.ts index 08bad67ae2..939b58f0b1 100644 --- a/packages/lsp/lsp-local/tests/instance.spec.ts +++ b/packages/lsp/lsp-local/tests/instance.spec.ts @@ -109,6 +109,11 @@ const RESPONDING_SERVER = const locJson = () => JSON.stringify({ uri: pathToFileURL(join(ws, 'a.ts')).href, range: { start: { line: 0, character: 0 }, end: { line: 0, character: 3 } } }) describe('LspInstance server-request handling', () => { + it('advertises a null process id across process namespaces', async () => { + const instance = makeInstance({ LSP_FAKE_EXPECT_PROCESS_ID: 'null', LSP_FAKE_DEF: 'null' }, { clientProcessId: null }) + await expect(run(instance, 'goToDefinition')).resolves.toMatchObject({ kind: 'locations' }) + }) + it('answers workspace/configuration with the static config per item', async () => { const instance = makeInstance({ LSP_FAKE_ON_OPEN: 'configuration', LSP_FAKE_DEF: locJson() }) // The query drives didOpen, which makes the fake emit workspace/configuration; a healthy answer diff --git a/packages/lsp/lsp/README.i18n.yaml b/packages/lsp/lsp/README.i18n.yaml index 02649a8d39..b672a405de 100644 --- a/packages/lsp/lsp/README.i18n.yaml +++ b/packages/lsp/lsp/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/lsp/lsp/README.md -README.md: 5c1044be50368acf13d8c36a15d5b2bd99d02701 -README.zh.md: cc412333e63b9469319240d67269bf0192ad3858 +README.md: bcffd460f192b0e6ac214ad0bc0d43da4c81d68c +README.zh.md: e9df352e999834235a66ef7a500e300dadc2029c diff --git a/packages/lsp/lsp/README.md b/packages/lsp/lsp/README.md index 5c1044be50..4dc0436bbb 100644 --- a/packages/lsp/lsp/README.md +++ b/packages/lsp/lsp/README.md @@ -10,6 +10,7 @@ This package is the interface third of the LSP capability: |---|---| | `@deepseek-ai/dsh-lsp` (this) | the interface: the service, provider registry keyed by branded id + extension mapping, per-query selection, request/result vocabulary, the `LspError` taxonomy | | `@deepseek-ai/dsh-lsp-local` | a generic local backend that registers configured stdio language-server providers | +| `@deepseek-ai/dsh-lsp-e2b` | a generic E2B backend that registers configured remote stdio providers | | `@deepseek-ai/dsh-tool-lsp` | the model-facing `lsp` tool over `ctx.lsp` | The seam exposes exactly four semantic operations — `goToDefinition`, `findReferences`, `goToImplementation`, `hover` — and no generic JSON-RPC escape hatch, so no protocol payload or unreviewed command/mutation reaches a provider through `ctx.lsp`. diff --git a/packages/lsp/lsp/README.zh.md b/packages/lsp/lsp/README.zh.md index cc412333e6..fdaedc9a96 100644 --- a/packages/lsp/lsp/README.zh.md +++ b/packages/lsp/lsp/README.zh.md @@ -10,6 +10,7 @@ |---|---| | `@deepseek-ai/dsh-lsp`(本包) | 接口:服务、以品牌化 id + 扩展名映射为 key 的提供方注册表、逐查询选择、请求/结果词汇、`LspError` 分类体系 | | `@deepseek-ai/dsh-lsp-local` | 通用本地后端,注册已配置的 stdio 语言服务器提供方 | +| `@deepseek-ai/dsh-lsp-e2b` | 通用 E2B 后端,注册已配置的远程 stdio 提供方 | | `@deepseek-ai/dsh-tool-lsp` | 面向模型的 `lsp` 工具,基于 `ctx.lsp` | 该 seam 恰好公开四种语义操作:`goToDefinition`、`findReferences`、`goToImplementation`、`hover`,且没有通用 JSON-RPC 逃生口,因此任何协议载荷或未经评审的命令/修改都无法通过 `ctx.lsp` 到达提供方。 diff --git a/packages/pty/README.i18n.yaml b/packages/pty/README.i18n.yaml index 65d541a7c1..c0fa873642 100644 --- a/packages/pty/README.i18n.yaml +++ b/packages/pty/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/pty/README.md -README.md: a4f743056b4a524be9623b0f700f37e0534b463f -README.zh.md: c84ad3f1b59afcdbbd111f1b82c57c56aa24fdcf +README.md: 239c751f30f8e5c543e6737e65a67f9bb22f79ad +README.zh.md: edaae7f948d957e2fce2b3f331cc2af61cfabd36 diff --git a/packages/pty/README.md b/packages/pty/README.md index a4f743056b..239c751f30 100644 --- a/packages/pty/README.md +++ b/packages/pty/README.md @@ -7,7 +7,8 @@ English | [中文](README.zh.md) | Package | Role | ctx key | |---|---|---| | [`pty`](pty/README.md) (`@deepseek-ai/dsh-pty`) | Backend registry, branded ids, exact-Agent ownership, session operations, and awaited cleanup | `ctx.pty` | -| `pty-local` (`@deepseek-ai/dsh-pty-local`) | Shell backend over `ctx.subprocess.spawnTerminal`: readiness detection, bounded terminal state, sandbox policy, and session operations | registers on `ctx.pty` | +| [`pty-local`](pty-local/README.md) (`@deepseek-ai/dsh-pty-local`) | Local `node-pty` backend, readiness detection, bounded terminal state, sandboxing, and process-session supervision | registers on `ctx.pty` | +| [`pty-e2b`](pty-e2b/README.md) (`@deepseek-ai/dsh-pty-e2b`) | E2B byte-PTY backend, remote foreground signaling, bounded terminal state, and awaited remote cleanup | registers on `ctx.pty` | | `tool-pty` (`@deepseek-ai/dsh-tool-pty`) | Six model-facing tools and generic task integration for background sends | registers on `ctx.tools` | -The design and deferred boundaries live in the [persistent PTY Agent Note](../../.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md). +The core design lives in the [persistent PTY Agent Note](../../.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md); the remote ownership boundary lives in the [E2B extension note](../../.agents/notes/implemented/feature/2026-07-28-e2b-interactive-semantic-code-runtime-poc.md). diff --git a/packages/pty/README.zh.md b/packages/pty/README.zh.md index c84ad3f1b5..edaae7f948 100644 --- a/packages/pty/README.zh.md +++ b/packages/pty/README.zh.md @@ -7,7 +7,8 @@ | 包 | 职责 | ctx 键 | |---|---|---| | [`pty`](pty/README.md)(`@deepseek-ai/dsh-pty`) | 后端注册表、品牌化 id、精确的 Agent 所有权、会话操作与等待完成的清理 | `ctx.pty` | -| `pty-local`(`@deepseek-ai/dsh-pty-local`) | `ctx.subprocess.spawnTerminal` 之上的 shell 后端:就绪检测、有界终端状态、沙箱策略与会话操作 | 注册到 `ctx.pty` | +| [`pty-local`](pty-local/README.md)(`@deepseek-ai/dsh-pty-local`) | 本地 `node-pty` 后端、就绪检测、有界终端状态、沙箱与进程会话监管 | 注册到 `ctx.pty` | +| [`pty-e2b`](pty-e2b/README.md)(`@deepseek-ai/dsh-pty-e2b`) | E2B 字节 PTY 后端、远程前台信号传递、有界终端状态与等待完成的远程清理 | 注册到 `ctx.pty` | | `tool-pty`(`@deepseek-ai/dsh-tool-pty`) | 6 个面向模型的工具,并为后台发送集成通用任务 | 注册到 `ctx.tools` | -设计与暂缓边界记录在[持久 PTY Agent Note](../../.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md) 中。 +核心设计记录在[持久 PTY Agent Note](../../.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md) 中;远程所有权边界记录在 [E2B 扩展 Agent Note](../../.agents/notes/implemented/feature/2026-07-28-e2b-interactive-semantic-code-runtime-poc.md) 中。 diff --git a/packages/pty/pty-e2b/README.i18n.yaml b/packages/pty/pty-e2b/README.i18n.yaml new file mode 100644 index 0000000000..bfd7795349 --- /dev/null +++ b/packages/pty/pty-e2b/README.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write packages/pty/pty-e2b/README.md +README.md: 7a8ddc9e08f4d28d69b97825232b00e2d8f394b7 +README.zh.md: 2151ca1f2d91b01a7b0106153f708ace2c3d2726 diff --git a/packages/pty/pty-e2b/README.md b/packages/pty/pty-e2b/README.md new file mode 100644 index 0000000000..7a8ddc9e08 --- /dev/null +++ b/packages/pty/pty-e2b/README.md @@ -0,0 +1,54 @@ +# @deepseek-ai/dsh-pty-e2b + +English | [中文](README.zh.md) + +E2B byte-PTY backend for [`ctx.pty`](../pty/README.md). It creates persistent interactive shells inside the shared `ctx.e2b` sandbox while the PTY registry keeps session identity, exact-Agent ownership, and cleanup policy on the host. + +## Plugin and configuration + +The `pty-e2b` plugin injects `e2b` and `pty`, then registers one backend under `backendType`. + +| Key | Default | Meaning | +|---|---|---| +| `backendType` | `shell` | Registry type selected by `terminal_open`. | +| `rows` / `cols` | `40` / `160` | Initial remote PTY size. | +| `scrollbackLines` | `10000` | Maximum retained logical lines. | +| `scrollbackMaxBytes` | `4194304` | Maximum retained UTF-8 scrollback bytes. | +| `maxReadBytes` | `262144` | Maximum bytes returned by one read or settled send. | +| `pollIntervalMs` | `50` | Host readiness-poll interval. | +| `idleSilenceMs` | `3000` | Output silence that yields `inferred_idle`. | +| `timeoutMs` | `30000` | Absolute startup and send wait bound. | +| `disposeGraceMs` | `3000` | TERM-to-KILL cleanup grace. | + +Numeric values are positive safe integers, `backendType` is non-empty, and `maxReadBytes` cannot exceed `scrollbackMaxBytes`. A relative spawn cwd resolves against `ctx.e2b.cwd`; an absolute remote path remains absolute. + +## Runtime contract + +The backend uses E2B's byte-oriented PTY callback with a streaming fatal UTF-8 decoder, then the backend-neutral line sanitizer and bounded buffers from `dsh-pty`. It installs a controlled Bash prompt marker and waits for printable prompt text; when that marker is unavailable, observed output plus the configured silence bound yields `inferred_idle`. Startup with no output reaches the absolute timeout and fails instead of publishing an empty session. + +Each send writes UTF-8 bytes and an optional carriage-return submit sequence. Cancellation and explicit signals resolve the remote terminal's foreground process group through `ps`, then signal that group; `SIGKILL` refuses to target the shell itself. Close sends `SIGTERM` to the PTY process group, waits, escalates through E2B's PTY kill, and does not resolve until the SDK handle reports exit. A startup failure closes the unpublished PTY, and `PtyBackendCleanupError` preserves a concurrent cleanup failure. + +The remote PTY process and its child processes live in E2B. Prompt/readiness state, scrollback, operation handles, owner authority, and SDK event delivery remain in host memory. + +## Model Experience + +### Indirect consumer + +#### What the model sees + +Nothing directly. Through `@deepseek-ai/dsh-tool-pty`, the model may receive bounded MOTD, send deltas, scrollback pages, readiness reasons, signal results, and cleanup failures. + +#### Token effect + +None until a consumer returns bounded backend output. Retained host PTY scrollback is not placed in model history by this package. + +#### KV Cache effect + +No direct invalidation; the consumer owns prompts, schemas, and appended results. + +## Known Limitations and Deferred Work + +- **Line-oriented terminal model** — CSI/OSC control sequences are removed; alternate-screen and full terminal emulation remain unsupported. +- **Readiness is marker-or-silence based** — E2B exposes foreground process groups but not the local backend's Linux syscall inspection, so `inferred_idle` is deliberately possible. +- **UTF-8 only** — invalid byte sequences fail the session instead of returning lossy text. +- **No reconnectable terminal handles** — retaining an E2B sandbox preserves remote files, not host ownership, buffers, callbacks, or live PTY sessions. diff --git a/packages/pty/pty-e2b/README.zh.md b/packages/pty/pty-e2b/README.zh.md new file mode 100644 index 0000000000..2151ca1f2d --- /dev/null +++ b/packages/pty/pty-e2b/README.zh.md @@ -0,0 +1,54 @@ +# @deepseek-ai/dsh-pty-e2b + +[English](README.md) | 中文 + +用于 [`ctx.pty`](../pty/README.md) 的 E2B 字节 PTY 后端。它在共享的 `ctx.e2b` 沙箱内创建持久交互式 shell;PTY 注册表则在宿主侧维护会话身份、精确的 Agent 所有权和清理策略。 + +## 插件与配置 + +`pty-e2b` 插件注入 `e2b` 和 `pty`,然后以 `backendType` 注册一个后端。 + +| 配置键 | 默认值 | 含义 | +|---|---|---| +| `backendType` | `shell` | `terminal_open` 选择的注册表类型。 | +| `rows` / `cols` | `40` / `160` | 远程 PTY 的初始尺寸。 | +| `scrollbackLines` | `10000` | 保留的逻辑行数上限。 | +| `scrollbackMaxBytes` | `4194304` | 保留的 UTF-8 scrollback 字节数上限。 | +| `maxReadBytes` | `262144` | 单次读取或发送结算时返回的字节数上限。 | +| `pollIntervalMs` | `50` | 宿主就绪轮询间隔。 | +| `idleSilenceMs` | `3000` | 触发 `inferred_idle` 的输出静默时长。 | +| `timeoutMs` | `30000` | 启动与发送等待的绝对上限。 | +| `disposeGraceMs` | `3000` | TERM 到 KILL 的清理宽限期。 | + +数值必须是正的安全整数,`backendType` 必须非空,且 `maxReadBytes` 不得超过 `scrollbackMaxBytes`。相对的 spawn cwd 以 `ctx.e2b.cwd` 为基准解析;绝对远程路径保持不变。 + +## 运行时契约 + +该后端为 E2B 面向字节的 PTY 回调配备流式、遇到无效序列即失败的 UTF-8 解码器,随后使用 `dsh-pty` 提供的后端无关行清理器与有界缓冲区。它会安装受控的 Bash 提示符标记,并等待可打印的提示符文本;若该标记不可用,系统会在已经观察到输出且达到已配置的静默上限时得出 `inferred_idle`。零输出的启动过程会达到绝对超时并失败,不会发布空会话。 + +每次发送都会写入 UTF-8 字节,并可选写入回车提交序列。取消与显式信号会通过 `ps` 确定远程终端的前台进程组,再向该组发送信号;发送 `SIGKILL` 时拒绝以 shell 本身为目标。关闭操作向 PTY 进程组发送 `SIGTERM`,等待后通过 E2B 的 PTY kill 操作升级,并且直到 SDK 句柄报告退出才结算。如果启动失败,系统会关闭尚未发布的 PTY;若清理同时失败,`PtyBackendCleanupError` 会保留这项失败。 + +远程 PTY 进程及其子进程位于 E2B。提示符/就绪状态、scrollback、操作句柄、所有者权限和 SDK 事件交付仍保留在宿主内存中。 + +## 模型体验 + +### 间接消费方 + +#### 模型看到的内容 + +没有直接可见内容。模型通过 `@deepseek-ai/dsh-tool-pty` 可能收到有界的 MOTD、发送增量、scrollback 页、就绪原因、信号结果和清理失败。 + +#### Token 影响 + +消费方返回有界的后端输出前没有影响。本包不会把宿主保留的 PTY scrollback 放入模型历史。 + +#### KV Cache 影响 + +不会直接失效;提示词、schema 和追加结果由消费方负责。 + +## 已知限制与暂缓工作 + +- **面向行的终端模型**:CSI/OSC 控制序列会被移除;备用屏幕与完整终端仿真仍不受支持。 +- **就绪判断基于标记或静默**:E2B 会公开前台进程组,但不提供本地后端使用的 Linux syscall 检查,因此系统有意保留返回 `inferred_idle` 的可能性。 +- **仅支持 UTF-8**:无效字节序列会使会话失败,而不是返回有损文本。 +- **没有可重连的终端句柄**:保留 E2B 沙箱会保留远程文件,但不会保留宿主所有权、缓冲区、回调或实时 PTY 会话。 diff --git a/packages/pty/pty-e2b/package.json b/packages/pty/pty-e2b/package.json new file mode 100644 index 0000000000..cc31699ab1 --- /dev/null +++ b/packages/pty/pty-e2b/package.json @@ -0,0 +1,46 @@ +{ + "name": "@deepseek-ai/dsh-pty-e2b", + "description": "E2B PTY provider for DeepSeek Harness", + "version": "0.0.1", + "private": true, + "type": "module", + "main": "lib/index.js", + "types": "lib/types/index.d.ts", + "exports": { + ".": { + "types": "./lib/types/index.d.ts", + "default": "./lib/index.js" + }, + "./invariant": { + "types": "./lib/types/invariant.d.ts", + "default": "./lib/invariant.js" + }, + "./src/*": "./src/*", + "./package.json": "./package.json" + }, + "files": [ + "lib/index.js", + "lib/invariant.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", + "src" + ], + "license": "BSD-3-Clause", + "peerDependencies": { + "@deepseek-ai/dsh-e2b": "^0.0.1", + "@deepseek-ai/dsh-invariants": "^0.0.1", + "@deepseek-ai/dsh-pty": "^0.0.1", + "cordis": "^4.0.0-rc.7" + }, + "dependencies": { + "schemastery": "^3.18.0" + }, + "devDependencies": { + "@deepseek-ai/dsh-agent": "workspace:^", + "@deepseek-ai/dsh-e2b": "workspace:^", + "@deepseek-ai/dsh-invariants": "workspace:^", + "@deepseek-ai/dsh-pty": "workspace:^", + "@deepseek-ai/dsh-session": "workspace:^", + "cordis": "^4.0.0-rc.7" + } +} diff --git a/packages/pty/pty-e2b/src/config.ts b/packages/pty/pty-e2b/src/config.ts new file mode 100644 index 0000000000..e6849bf54d --- /dev/null +++ b/packages/pty/pty-e2b/src/config.ts @@ -0,0 +1,64 @@ +/** Validated configuration for the E2B PTY backend. */ + +import z from 'schemastery' + +/** Public plugin configuration. */ +export interface Config { + /** Backend registry type. */ + backendType?: string + /** Initial terminal rows. */ + rows?: number + /** Initial terminal columns. */ + cols?: number + /** Maximum retained logical lines. */ + scrollbackLines?: number + /** Maximum retained UTF-8 bytes. */ + scrollbackMaxBytes?: number + /** Maximum bytes returned by one read or settled viewport. */ + maxReadBytes?: number + /** Readiness polling interval. */ + pollIntervalMs?: number + /** Output silence duration that yields `inferred_idle`. */ + idleSilenceMs?: number + /** Absolute send and startup wait bound. */ + timeoutMs?: number + /** Grace before PTY teardown escalates from TERM to KILL. */ + disposeGraceMs?: number +} + +/** Configuration after Schemastery defaults. */ +export type ResolvedConfig = Required + +/* jscpd:ignore-start -- Loader requires a backend-local schema and load-time diagnostics. */ +/** Schemastery config exposed by the plugin. */ +export const Config: z = z.object({ + backendType: z.string().default('shell'), + rows: z.number().default(40), + cols: z.number().default(160), + scrollbackLines: z.number().default(10_000), + scrollbackMaxBytes: z.number().default(4 * 1024 * 1024), + maxReadBytes: z.number().default(256 * 1024), + pollIntervalMs: z.number().default(50), + idleSilenceMs: z.number().default(3_000), + timeoutMs: z.number().default(30_000), + disposeGraceMs: z.number().default(3_000), +}) + +/** + * Validate the resolved configuration before publishing the backend. + * @param config - Schemastery-resolved plugin configuration. + * @returns Nothing; success narrows every optional field to its resolved value. + */ +export function validateConfig(config: Config): asserts config is ResolvedConfig { + const resolved = config as ResolvedConfig + if (resolved.backendType.length === 0) throw new Error('pty-e2b: backendType must be non-empty') + for (const [name, value] of Object.entries(resolved)) { + if (typeof value === 'number' && (!Number.isSafeInteger(value) || value <= 0)) { + throw new Error(`pty-e2b: ${name} must be a positive safe integer`) + } + } + if (resolved.maxReadBytes > resolved.scrollbackMaxBytes) { + throw new Error('pty-e2b: maxReadBytes must not exceed scrollbackMaxBytes') + } +} +/* jscpd:ignore-end */ diff --git a/packages/pty/pty-e2b/src/index.ts b/packages/pty/pty-e2b/src/index.ts new file mode 100644 index 0000000000..fca16cf5c7 --- /dev/null +++ b/packages/pty/pty-e2b/src/index.ts @@ -0,0 +1,93 @@ +/** E2B byte-PTY backend for persistent interactive terminal sessions. */ + +import { posix } from 'node:path' +import type { Context } from 'cordis' +import type { CommandHandle, Sandbox } from '@deepseek-ai/dsh-e2b' +import { PtyBackendCleanupError } from '@deepseek-ai/dsh-pty' +import type { PtyBackend, PtyBackendSpawnSpec } from '@deepseek-ai/dsh-pty' +import { type Config, type ResolvedConfig, validateConfig } from './config.ts' +import { E2BPtySession } from './session.ts' + +export { Config } from './config.ts' +export type { Config as PtyE2BConfig } from './config.ts' +export { E2BPtySession } from './session.ts' + +/** Cordis plugin name. */ +export const name = 'pty-e2b' +/** Required shared sandbox owner and PTY registry. */ +export const inject = ['e2b', 'pty'] + +function terminalEnvironment(spec: PtyBackendSpawnSpec): Record { + return { + TERM: 'dumb', + PAGER: 'cat', + GIT_PAGER: 'cat', + PS1: 'dsh> ', + PROMPT_COMMAND: 'printf "\\033]133;D;%s\\007" "$?"', + BASH_SILENCE_DEPRECATION_WARNING: '1', + DSH_SHELL: '1', + DSH_SESSION_ID: spec.owner.id, + DSH_PTY_SESSION_ID: spec.sessionId, + } +} + +/** E2B backend registered under the configured terminal type. */ +export class E2BPtyBackend implements PtyBackend { + readonly type: string + + constructor( + private readonly ctx: Context, + private readonly config: ResolvedConfig, + private readonly createPty: ( + sandbox: Sandbox, + options: Parameters[0], + ) => Promise = (sandbox, options) => sandbox.pty.create(options), + ) { + this.type = config.backendType + } + + /** Create, initialize, and publish one remote PTY session. */ + async spawn(spec: PtyBackendSpawnSpec): Promise { + spec.signal?.throwIfAborted() + const sandbox = await this.ctx.e2b.getSandbox() + spec.signal?.throwIfAborted() + const pending: Uint8Array[] = [] + const created: { session?: E2BPtySession } = {} + const handle = await this.createPty(sandbox, { + rows: this.config.rows, + cols: this.config.cols, + cwd: posix.resolve(this.ctx.e2b.cwd, spec.cwd ?? this.ctx.e2b.cwd), + envs: terminalEnvironment(spec), + timeoutMs: 0, + ...spec.signal === undefined ? {} : { signal: spec.signal }, + onData: (data) => { + if (created.session === undefined) pending.push(Uint8Array.from(data)) + else created.session.onData(data) + }, + }) + if (!Number.isSafeInteger(handle.pid) || handle.pid <= 0) { + await handle.kill().catch(() => false) + throw new Error(`pty-e2b: E2B returned invalid PTY pid ${handle.pid}`) + } + const session = new E2BPtySession(sandbox, handle, this.config) + created.session = session + for (const data of pending) session.onData(data) + try { + await session.initialize(spec.signal) + return session + } catch (error: unknown) { + try { + await session.close('E2B PTY startup failed') + } catch (cleanupError: unknown) { + throw new PtyBackendCleanupError(error, cleanupError) + } + throw error + } + } +} + +/** Register the E2B PTY backend. */ +export function apply(ctx: Context, config: Config): void { + validateConfig(config) + ctx.pty.registerBackend(new E2BPtyBackend(ctx, config)) +} diff --git a/packages/pty/pty-e2b/src/invariant.ts b/packages/pty/pty-e2b/src/invariant.ts new file mode 100644 index 0000000000..b86fdffc5d --- /dev/null +++ b/packages/pty/pty-e2b/src/invariant.ts @@ -0,0 +1,20 @@ +/** Package-owned invariant companion for `@deepseek-ai/dsh-pty-e2b`. */ + +/* jscpd:ignore-start */ +import type { Context } from 'cordis' +import type { InvariantInstaller } from '@deepseek-ai/dsh-invariants' + +const PACKAGE_NAME = '@deepseek-ai/dsh-pty-e2b' + +/** Cordis companion plugin name. */ +export const name = 'pty-e2b-invariant' +/** Service required before the companion can reserve package ownership. */ +export const inject = ['invariants'] + +/** No runtime invariant: the PTY registry owns publication and cleanup. */ +const install: InvariantInstaller = () => {} + +/** Register this package's invariant companion. */ +export const apply = (ctx: Context): Promise<() => void> => + Promise.resolve(ctx.invariants.register(PACKAGE_NAME, install)) +/* jscpd:ignore-end */ diff --git a/packages/pty/pty-e2b/src/session.ts b/packages/pty/pty-e2b/src/session.ts new file mode 100644 index 0000000000..e5dfc05efe --- /dev/null +++ b/packages/pty/pty-e2b/src/session.ts @@ -0,0 +1,367 @@ +/** One byte-oriented E2B PTY session projected onto the harness PTY seam. */ + +import { Buffer } from 'node:buffer' +import type { CommandHandle, Sandbox } from '@deepseek-ai/dsh-e2b' +import { CommandExitError } from '@deepseek-ai/dsh-e2b' +import { + PtyTerminalSanitizer, + PtyTextBuffer, + ptySignalName, + ptyUtf8Tail, +} from '@deepseek-ai/dsh-pty' +import type { + PtyBackendSession, + PtyReadRequest, + PtyReadResult, + PtySendOperation, + PtySendRead, + PtySendRequest, + PtySendResult, + PtySessionStatus, + PtySignal, + PtySignalResult, + PtyWaitReason, +} from '@deepseek-ai/dsh-pty' +import type { ResolvedConfig } from './config.ts' + +function delay(ms: number): Promise { + return new Promise(resolve => setTimeout(resolve, ms)) +} + +/* jscpd:ignore-start -- Operation state stays backend-local because process readiness and cleanup identities diverge. */ +class E2BSendOperation implements PtySendOperation { + private readonly output: PtyTextBuffer + private readonly result = Promise.withResolvers() + private finished = false + + constructor( + maxBytes: number, + readonly startedAt: number, + private readonly onCancel: () => void, + ) { + this.output = new PtyTextBuffer(maxBytes) + } + + get done(): Promise { + return this.result.promise + } + + append(text: string): void { + if (!this.finished) this.output.append(text) + } + + settle(waitReason: PtyWaitReason, sessionStatus: PtySessionStatus, inheritedTruncation: boolean): void { + if (this.finished) return + this.finished = true + const read = this.output.snapshot() + this.result.resolve({ + viewport: read.text, + waitReason, + sessionStatus, + truncated: read.truncated || inheritedTruncation, + }) + } + + fail(error: unknown): void { + if (this.finished) return + this.finished = true + this.result.reject(error) + } + + readOutput(): PtySendRead { + return this.output.consume() + } + + cancel(): boolean { + if (this.finished) return false + this.onCancel() + return true + } +} +/* jscpd:ignore-end */ + +/** Live session around one E2B SDK PTY handle. */ +export class E2BPtySession implements PtyBackendSession { + motd = '' + readonly pid: number + private readonly decoder = new TextDecoder('utf-8', { fatal: true }) + private readonly sanitizer: PtyTerminalSanitizer + private readonly scrollback: PtyTextBuffer + private readonly exited = Promise.withResolvers() + private statusValue: PtySessionStatus = { kind: 'running' } + private active: E2BSendOperation | undefined + private activeTimer: NodeJS.Timeout | undefined + private activeAbort: (() => void) | undefined + private promptSeen = false + private promptTextSeen = false + private initializing = false + private lastOutputAt = Date.now() + private closing = false + private closePromise: Promise | undefined + private closeSignal: NodeJS.Signals | null = null + private transportFailure: Error | undefined + private remoteExited = false + + constructor( + private readonly sandbox: Sandbox, + private readonly handle: CommandHandle, + private readonly config: ResolvedConfig, + ) { + this.pid = handle.pid + this.sanitizer = new PtyTerminalSanitizer(config.maxReadBytes) + this.scrollback = new PtyTextBuffer(config.scrollbackMaxBytes, config.scrollbackLines) + const completion = handle.wait() + void completion.then( + (result) => { this.onExit(result.exitCode) }, + (error: unknown) => { + if (error instanceof CommandExitError) this.onExit(error.exitCode) + else this.onTransportFailure(error) + }, + ) + } + + /** + * Consume bytes received by the SDK's PTY callback. + * @param data - Exact callback bytes in delivery order. + */ + onData(data: Uint8Array): void { + let decoded: string + try { + decoded = this.decoder.decode(data, { stream: true }) + } catch (error: unknown) { + this.onTransportFailure(new Error('pty-e2b: PTY emitted invalid UTF-8', { cause: error })) + return + } + const sanitized = this.sanitizer.push(decoded) + this.appendOutput(sanitized.text) + if (sanitized.prompt) { + this.promptSeen = true + this.promptTextSeen = sanitized.promptText === true + this.lastOutputAt = Date.now() + } else if (this.promptSeen && sanitized.promptText === true) { + this.promptTextSeen = true + } + } + + /** + * Await the first prompt or bounded startup fallback. + * @param signal - Optional startup cancellation signal. + */ + async initialize(signal?: AbortSignal): Promise { + this.initializing = true + try { + const operation = this.startSend({ text: '', submit: false, ...signal === undefined ? {} : { signal } }) + const result = await operation.done + if (result.waitReason === 'session_exit') throw new Error('E2B PTY shell exited during startup') + if (result.waitReason === 'timeout') throw new Error('E2B PTY shell did not reach readiness before startup timeout') + this.motd = result.viewport + } catch (error: unknown) { + signal?.throwIfAborted() + throw error + } finally { + this.initializing = false + } + } + + /* jscpd:ignore-start -- PTY backends share request admission while owning distinct input and readiness transports. */ + startSend(request: PtySendRequest): PtySendOperation { + if (this.closing) throw new Error('E2B PTY session is closing') + if (this.statusValue.kind === 'exited') throw new Error('E2B PTY session has exited') + if (this.active !== undefined) throw new Error('E2B PTY session already has an active send') + if (request.signal?.aborted === true) throw new Error('E2B PTY send aborted before write') + + const operation = new E2BSendOperation( + this.config.maxReadBytes, + Date.now(), + () => { this.interrupt(operation) }, + ) + this.active = operation + this.lastOutputAt = Date.now() + this.promptSeen = false + this.promptTextSeen = false + if (request.signal !== undefined) { + const onAbort = (): void => { operation.cancel() } + request.signal.addEventListener('abort', onAbort, { once: true }) + this.activeAbort = () => request.signal?.removeEventListener('abort', onAbort) + } + + const input = `${request.text}${request.submit ? '\r' : ''}` + if (input.length > 0) { + void this.sandbox.pty.sendInput(this.pid, Buffer.from(input)).catch((error: unknown) => { + if (this.active === operation) this.failActive(error) + }) + } + this.activeTimer = setInterval(() => { this.pollReadiness(operation) }, this.config.pollIntervalMs) + return operation + } + /* jscpd:ignore-end */ + + /* jscpd:ignore-start -- The seam requires identical bounded-read coordinates across backend buffers. */ + read(request: PtyReadRequest): PtyReadResult { + const snapshot = this.scrollback.snapshot() + const lines = snapshot.text.split('\n') + const totalLines = snapshot.text.length === 0 ? 0 : lines.length + const offset = request.offset ?? 0 + const count = request.count ?? 500 + if (!Number.isSafeInteger(offset) || offset < 0) throw new Error('PTY read offset must be a non-negative safe integer') + if (!Number.isSafeInteger(count) || count <= 0) throw new Error('PTY read count must be a positive safe integer') + if (offset >= totalLines) { + return { text: '', totalLines, lineBegin: offset, lineEnd: offset, truncated: snapshot.truncated } + } + const end = totalLines - offset + const start = Math.max(0, end - count) + const bounded = ptyUtf8Tail(lines.slice(start, end).join('\n'), this.config.maxReadBytes) + const returnedLines = bounded.text.length === 0 ? 0 : bounded.text.split('\n').length + return { + text: bounded.text, + totalLines, + lineBegin: offset, + lineEnd: offset + returnedLines, + truncated: snapshot.truncated || bounded.truncated, + } + } + /* jscpd:ignore-end */ + + /* jscpd:ignore-start -- Signal, status, and close methods preserve the seam shape around remote identities. */ + async signal(signal: PtySignal): Promise { + const pgid = await this.foregroundPgid() + if (signal === 'SIGKILL' && pgid === this.pid) { + throw new Error('refusing to SIGKILL the E2B PTY shell; use terminal_close') + } + await this.sandbox.commands.run(`kill -${signal.slice(3)} -- -${pgid}`) + return { delivered: true, targetPgid: pgid } + } + + status(): PtySessionStatus { + return this.statusValue + } + + close(reason: string): Promise { + this.closing = true + if (this.closePromise !== undefined) return this.closePromise + const closing = this.closeOnce(reason).catch((error: unknown) => { + this.closePromise = undefined + this.failActive(error) + throw error + }) + this.closePromise = closing + return closing + } + /* jscpd:ignore-end */ + + private appendOutput(text: string): void { + if (text.length === 0) return + this.lastOutputAt = Date.now() + this.scrollback.append(text) + this.active?.append(text) + } + + private pollReadiness(operation: E2BSendOperation): void { + if (this.active !== operation) return + if (this.statusValue.kind === 'exited') { + this.settleActive('session_exit') + return + } + const elapsed = Date.now() - operation.startedAt + const idleFor = Date.now() - this.lastOutputAt + if (this.promptSeen && this.promptTextSeen && idleFor >= this.config.pollIntervalMs) { + this.settleActive('stdin_read') + return + } + const startupHasOutput = !this.initializing || this.scrollback.snapshot().text.length > 0 + if (startupHasOutput && idleFor >= this.config.idleSilenceMs) { + this.settleActive('inferred_idle') + return + } + if (elapsed >= this.config.timeoutMs) this.settleActive('timeout') + } + + private settleActive(waitReason: PtyWaitReason): void { + const operation = this.active + if (operation === undefined) return + const inherited = this.scrollback.snapshot().truncated + this.clearActive() + operation.settle(waitReason, this.statusValue, inherited) + } + + private clearActive(): void { + if (this.activeTimer !== undefined) clearInterval(this.activeTimer) + this.activeTimer = undefined + this.activeAbort?.() + this.activeAbort = undefined + this.active = undefined + } + + private failActive(error: unknown): void { + const operation = this.active + if (operation === undefined) return + this.clearActive() + operation.fail(error) + } + + private interrupt(operation: E2BSendOperation): void { + if (this.active !== operation) return + void this.signal('SIGINT').catch((error: unknown) => { this.failActive(error) }) + } + + private async foregroundPgid(): Promise { + const result = await this.sandbox.commands.run(`ps -o tpgid= -p ${this.pid}`) + const raw = result.stdout.trim() + const pgid = Number(raw) + if (!/^[1-9][0-9]*$/.test(raw) || !Number.isSafeInteger(pgid)) { + throw new Error(`cannot resolve foreground process group for E2B PTY ${this.pid}`) + } + return pgid + } + + private onExit(exitCode: number): void { + this.remoteExited = true + let tail = '' + try { + tail = this.decoder.decode() + } catch (error: unknown) { + this.transportFailure ??= new Error('pty-e2b: PTY ended with invalid UTF-8', { cause: error }) + } + this.appendOutput(this.sanitizer.push(tail).text) + this.appendOutput(this.sanitizer.flush()) + const inferredSignal = this.closeSignal ?? (exitCode > 128 ? ptySignalName(exitCode - 128) : null) + this.statusValue = { + kind: 'exited', + exitCode: inferredSignal === null ? exitCode : null, + signal: inferredSignal, + } + if (this.transportFailure === undefined) this.settleActive('session_exit') + else this.failActive(this.transportFailure) + this.exited.resolve() + } + + private onTransportFailure(error: unknown): void { + const failure = error instanceof Error ? error : new Error(String(error)) + this.transportFailure ??= failure + this.statusValue = { kind: 'exited', exitCode: null, signal: null } + this.failActive(failure) + } + + private async closeOnce(reason: string): Promise { + if (!this.remoteExited) { + this.closeSignal = 'SIGTERM' + try { + await this.sandbox.commands.run(`kill -TERM -- -${this.pid}`) + } catch (error: unknown) { + if (!(error instanceof CommandExitError)) throw error + } + await Promise.race([this.exited.promise, delay(this.config.disposeGraceMs)]) + } + if (!this.remoteExited) { + this.closeSignal = 'SIGKILL' + await this.sandbox.pty.kill(this.pid) + await Promise.race([this.exited.promise, delay(this.config.disposeGraceMs)]) + } + if (!this.remoteExited) { + throw new Error(`E2B PTY cleanup failed (${reason}); surviving pid: ${this.pid}`) + } + this.settleActive('session_exit') + await this.handle.disconnect().catch(() => {}) + if (this.transportFailure !== undefined) throw this.transportFailure + } +} diff --git a/packages/pty/pty-e2b/tests/index.spec.ts b/packages/pty/pty-e2b/tests/index.spec.ts new file mode 100644 index 0000000000..567b3b9215 --- /dev/null +++ b/packages/pty/pty-e2b/tests/index.spec.ts @@ -0,0 +1,195 @@ +import { Context } from 'cordis' +import { describe, expect, it, vi } from 'vitest' +import type { CommandHandle, Sandbox } from '@deepseek-ai/dsh-e2b' +import type E2BSandboxService from '@deepseek-ai/dsh-e2b' +import PtyService, { PtyBackendCleanupError, PtySessionId } from '@deepseek-ai/dsh-pty' +import { E2BPtyBackend, apply } from '@deepseek-ai/dsh-pty-e2b' +import { validateConfig } from '@deepseek-ai/dsh-pty-e2b/src/config.ts' +import * as E2BPtyInvariant from '../src/invariant.ts' +import InvariantService from '@deepseek-ai/dsh-invariants' +import { AgentMessageId, type Agent } from '@deepseek-ai/dsh-agent' +import { Session, SessionId } from '@deepseek-ai/dsh-session' + +function config() { + return { + backendType: 'shell', rows: 24, cols: 80, + scrollbackLines: 10, scrollbackMaxBytes: 128, maxReadBytes: 64, + pollIntervalMs: 1, idleSilenceMs: 2, timeoutMs: 5, disposeGraceMs: 1, + } +} + +function owner(ctx: Context): Agent { + const id = SessionId('owner') + return { + id, options: {}, session: new Session(id), status: 'idle', acceptsNextStep: false, ctx, + followup: () => AgentMessageId('unused'), steer: () => AgentMessageId('unused'), + inject: () => AgentMessageId('unused'), send: () => AgentMessageId('unused'), + cancel() {}, whenIdle: () => Promise.resolve(), + } +} + +function handle(pid = 123, kill = vi.fn().mockResolvedValue(true)): CommandHandle { + const result = Promise.withResolvers<{ exitCode: number; stdout: string; stderr: string }>() + return { + pid, + wait: () => result.promise, + kill, + disconnect: vi.fn().mockResolvedValue(undefined), + } as unknown as CommandHandle +} + +describe('E2BPtyBackend and plugin', () => { + it('creates a remote PTY with isolated environment and initializes the session', async () => { + vi.useFakeTimers() + const ctx = new Context() + const sandbox = {} as Sandbox + ctx.provide('e2b', { + cwd: '/workspace', + getSandbox: async () => sandbox, + } as E2BSandboxService) + const created = handle() + let options: Parameters[0] | undefined + const backend = new E2BPtyBackend(ctx, config(), async (_sandbox, received) => { + options = received + void received.onData(Buffer.from('banner\n')) + setTimeout(() => { void received.onData(Buffer.from('\x1b]133;D;0\x07dsh> ')) }, 0) + return created + }) + const pending = backend.spawn({ + sessionId: PtySessionId('pty-1'), owner: owner(ctx), type: 'shell', cwd: 'project', + signal: new AbortController().signal, + }) + await vi.advanceTimersByTimeAsync(2) + const session = await pending + + expect(session.motd).toBe('dsh> ') + expect(options).toMatchObject({ rows: 24, cols: 80, cwd: '/workspace/project', timeoutMs: 0 }) + expect(options?.envs).toMatchObject({ + TERM: 'dumb', PAGER: 'cat', GIT_PAGER: 'cat', PS1: 'dsh> ', + DSH_SHELL: '1', DSH_SESSION_ID: 'owner', DSH_PTY_SESSION_ID: 'pty-1', + }) + vi.useRealTimers() + }) + + it('uses the SDK PTY create method and the shared cwd by default', async () => { + vi.useFakeTimers() + const ctx = new Context() + const created = handle() + const create = vi.fn(async (received: Parameters[0]) => { + setTimeout(() => { void received.onData(Buffer.from('\x1b]133;D;0\x07dsh> ')) }, 0) + return created + }) + const sandbox = { pty: { create } } as unknown as Sandbox + ctx.provide('e2b', { cwd: '/workspace', getSandbox: async () => sandbox } as unknown as E2BSandboxService) + const backend = new E2BPtyBackend(ctx, config()) + const pending = backend.spawn({ sessionId: PtySessionId('default'), owner: owner(ctx), type: 'shell' }) + await vi.advanceTimersByTimeAsync(2) + await pending + expect(create).toHaveBeenCalledWith(expect.objectContaining({ cwd: '/workspace' })) + vi.useRealTimers() + }) + + it('rejects aborts and invalid pids, killing a malformed SDK handle', async () => { + const ctx = new Context() + const sandbox = {} as Sandbox + ctx.provide('e2b', { cwd: '/workspace', getSandbox: async () => sandbox } as E2BSandboxService) + const create = vi.fn().mockResolvedValue(handle(0)) + const backend = new E2BPtyBackend(ctx, config(), create) + const aborted = AbortSignal.abort(new Error('stop')) + await expect(backend.spawn({ sessionId: PtySessionId('one'), owner: owner(ctx), type: 'shell', signal: aborted })).rejects.toThrow('stop') + expect(create).not.toHaveBeenCalled() + + const malformedKill = vi.fn().mockResolvedValue(true) + const malformed = handle(0, malformedKill) + const invalid = new E2BPtyBackend(ctx, config(), async () => malformed) + await expect(invalid.spawn({ sessionId: PtySessionId('two'), owner: owner(ctx), type: 'shell' })).rejects.toThrow('invalid PTY pid') + expect(malformedKill).toHaveBeenCalledOnce() + + const killFailureKill = vi.fn().mockRejectedValue(new Error('already gone')) + const killFailure = handle(0, killFailureKill) + const raced = new E2BPtyBackend(ctx, config(), async () => killFailure) + await expect(raced.spawn({ sessionId: PtySessionId('three'), owner: owner(ctx), type: 'shell' })).rejects.toThrow('invalid PTY pid') + }) + + it('cleans failed startup and aggregates a cleanup failure', async () => { + vi.useFakeTimers() + const ctx = new Context() + const sandbox = { + commands: { run: vi.fn().mockResolvedValue({ exitCode: 0, stdout: '', stderr: '' }) }, + pty: { kill: vi.fn().mockRejectedValue(new Error('cleanup failed')) }, + } as unknown as Sandbox + ctx.provide('e2b', { cwd: '/workspace', getSandbox: async () => sandbox } as E2BSandboxService) + const failedHandle = handle() + const backend = new E2BPtyBackend(ctx, config(), async () => failedHandle) + const pending = backend.spawn({ sessionId: PtySessionId('failed'), owner: owner(ctx), type: 'shell' }) + const rejected = expect(pending).rejects.toMatchObject({ + name: 'PtyBackendCleanupError', + cleanupError: expect.objectContaining({ message: 'cleanup failed' }), + } satisfies Partial) + await vi.advanceTimersByTimeAsync(6) + await vi.advanceTimersByTimeAsync(2) + await rejected + vi.useRealTimers() + }) + + it('preserves startup failure when cleanup succeeds', async () => { + vi.useFakeTimers() + const ctx = new Context() + const completion = Promise.withResolvers<{ exitCode: number; stdout: string; stderr: string }>() + const created = { + pid: 123, + wait: () => completion.promise, + disconnect: vi.fn().mockResolvedValue(undefined), + } as unknown as CommandHandle + const sandbox = { + commands: { + run: vi.fn(async (command: string) => { + if (command.startsWith('kill -TERM')) completion.resolve({ exitCode: 143, stdout: '', stderr: '' }) + return { exitCode: 0, stdout: '', stderr: '' } + }), + }, + pty: { kill: vi.fn().mockResolvedValue(true) }, + } as unknown as Sandbox + ctx.provide('e2b', { cwd: '/workspace', getSandbox: async () => sandbox } as unknown as E2BSandboxService) + const backend = new E2BPtyBackend(ctx, config(), async () => created) + const rejected = expect(backend.spawn({ sessionId: PtySessionId('failed-clean'), owner: owner(ctx), type: 'shell' })) + .rejects.toThrow('startup timeout') + await vi.advanceTimersByTimeAsync(6) + await rejected + vi.useRealTimers() + }) + + it('validates configuration and registers the selected backend type', async () => { + const valid = config() + expect(() => { validateConfig(valid) }).not.toThrow() + for (const invalid of [ + { ...valid, backendType: '' }, + { ...valid, rows: 0 }, + { ...valid, rows: 1.5 }, + { ...valid, maxReadBytes: 129 }, + ]) { + expect(() => { validateConfig(invalid) }).toThrow() + } + + const registerBackend = vi.fn() + apply({ pty: { registerBackend } } as unknown as Context, valid) + expect(registerBackend).toHaveBeenCalledWith(expect.objectContaining({ type: 'shell' })) + + const ctx = new Context() + await ctx.plugin(PtyService) + ctx.provide('e2b', { cwd: '/workspace', getSandbox: async () => ({}) } as never) + const fiber = await ctx.plugin({ + inject: ['pty', 'e2b'], + apply: (pluginCtx: Context) => { apply(pluginCtx, valid) }, + }) + expect(ctx.pty.listBackends()).toEqual(['shell']) + await fiber.dispose() + }) + + it('registers the package-owned invariant companion', async () => { + const ctx = new Context() + await ctx.plugin(InvariantService, { enabled: true }) + const fiber = await ctx.plugin(E2BPtyInvariant).await() + await fiber.dispose() + }) +}) diff --git a/packages/pty/pty-e2b/tests/session.spec.ts b/packages/pty/pty-e2b/tests/session.spec.ts new file mode 100644 index 0000000000..6dced22ed2 --- /dev/null +++ b/packages/pty/pty-e2b/tests/session.spec.ts @@ -0,0 +1,398 @@ +import { Buffer } from 'node:buffer' +import { afterEach, describe, expect, it, vi } from 'vitest' +import { + CommandExitError, + type CommandHandle, + type CommandResult, + type Sandbox, +} from '@deepseek-ai/dsh-e2b' +import type { PtySendOperation, PtySessionStatus } from '@deepseek-ai/dsh-pty' +import { E2BPtySession } from '@deepseek-ai/dsh-pty-e2b' +import type { ResolvedConfig } from '@deepseek-ai/dsh-pty-e2b/src/config.ts' + +function commandError(exitCode: number): CommandExitError { + return new CommandExitError({ exitCode, stdout: '', stderr: '', error: `exit ${exitCode}` }) +} + +class FakePtyHandle { + pid = 123 + readonly result = Promise.withResolvers() + disconnects = 0 + kills = 0 + disconnectError: unknown + private settled = false + + wait(): Promise { + return this.result.promise + } + + async disconnect(): Promise { + this.disconnects += 1 + if (this.disconnectError !== undefined) throw this.disconnectError + } + + async kill(): Promise { + this.kills += 1 + return true + } + + exit(exitCode = 0): void { + if (this.settled) return + this.settled = true + this.result.resolve({ exitCode, stdout: '', stderr: '' }) + } + + failExit(exitCode: number): void { + if (this.settled) return + this.settled = true + this.result.reject(commandError(exitCode)) + } + + crash(error: unknown): void { + if (this.settled) return + this.settled = true + this.result.reject(error) + } + + asHandle(): CommandHandle { + return this as unknown as CommandHandle + } +} + +class FakeSandbox { + readonly sent: Array<{ pid: number; data: Buffer }> = [] + readonly commands: string[] = [] + readonly killed: number[] = [] + pgid = '456\n' + sendError: unknown + commandError: unknown + killError: unknown + onTerm: (() => void) | undefined + onKill: (() => void) | undefined + + readonly sandbox = { + pty: { + sendInput: async (pid: number, data: Uint8Array): Promise => { + this.sent.push({ pid, data: Buffer.from(data) }) + if (this.sendError !== undefined) throw this.sendError + }, + kill: async (pid: number): Promise => { + this.killed.push(pid) + if (this.killError !== undefined) throw this.killError + this.onKill?.() + return true + }, + }, + commands: { + run: async (command: string): Promise => { + this.commands.push(command) + if (this.commandError !== undefined) { + const error = this.commandError + this.commandError = undefined + throw error + } + if (command.startsWith('ps ')) return { exitCode: 0, stdout: this.pgid, stderr: '' } + if (command.startsWith('kill -TERM')) this.onTerm?.() + return { exitCode: 0, stdout: '', stderr: '' } + }, + }, + } as unknown as Sandbox +} + +function config(overrides: Partial = {}): ResolvedConfig { + return { + backendType: 'shell', rows: 24, cols: 80, + scrollbackLines: 10, scrollbackMaxBytes: 128, maxReadBytes: 64, + pollIntervalMs: 10, idleSilenceMs: 40, timeoutMs: 100, disposeGraceMs: 20, + ...overrides, + } +} + +async function initialize(session: E2BPtySession): Promise { + const pending = session.initialize() + session.onData(Buffer.from('\x1b]133;D;0\x07dsh> ')) + await vi.advanceTimersByTimeAsync(10) + await pending +} + +afterEach(() => { vi.useRealTimers() }) + +describe('E2BPtySession readiness, output, and signals', () => { + it('initializes, sends UTF-8 input, settles at a prompt, and reads bounded scrollback', async () => { + vi.useFakeTimers() + const fake = new FakeSandbox() + const handle = new FakePtyHandle() + const session = new E2BPtySession(fake.sandbox, handle.asHandle(), config({ maxReadBytes: 12 })) + expect(session.read({})).toMatchObject({ text: '', totalLines: 0 }) + await initialize(session) + expect(session.motd).toBe('dsh> ') + + const operation = session.startSend({ text: 'printf 你好', submit: true }) + expect(fake.sent).toEqual([{ pid: 123, data: Buffer.from('printf 你好\r') }]) + session.onData(Buffer.from('一\n二\n三\x1b]133;D;0\x07dsh> ')) + const bounded = operation.readOutput() + expect(bounded.delta).toContain('三') + expect(bounded.truncated).toBe(true) + await vi.advanceTimersByTimeAsync(10) + expect(await operation.done).toMatchObject({ waitReason: 'stdin_read', sessionStatus: { kind: 'running' } }) + expect(operation.cancel()).toBe(false) + expect(session.read({ count: 2 }).text).toContain('dsh>') + expect(session.read({ offset: 99 })).toMatchObject({ text: '', lineBegin: 99, lineEnd: 99 }) + expect(() => session.read({ offset: -1 })).toThrow('non-negative safe integer') + expect(() => session.read({ offset: 1.5 })).toThrow('non-negative safe integer') + expect(() => session.read({ count: 0 })).toThrow('positive safe integer') + expect(() => session.read({ count: 1.5 })).toThrow('positive safe integer') + + await expect(session.signal('SIGTERM')).resolves.toEqual({ delivered: true, targetPgid: 456 }) + expect(fake.commands).toContain('kill -TERM -- -456') + expect(session.status()).toEqual({ kind: 'running' }) + }) + + it('distinguishes inferred idle, timeout, session exit, and no-output startup timeout', async () => { + vi.useFakeTimers() + const fake = new FakeSandbox() + const handle = new FakePtyHandle() + const session = new E2BPtySession(fake.sandbox, handle.asHandle(), config()) + await initialize(session) + + const inferred = session.startSend({ text: '', submit: false }) + await vi.advanceTimersByTimeAsync(40) + expect((await inferred.done).waitReason).toBe('inferred_idle') + + const timeout = session.startSend({ text: '', submit: false }) + for (let index = 0; index < 3; index += 1) { + await vi.advanceTimersByTimeAsync(30) + session.onData(Buffer.from('.')) + } + await vi.advanceTimersByTimeAsync(10) + expect((await timeout.done).waitReason).toBe('timeout') + + const exiting = session.startSend({ text: '', submit: false }) + handle.failExit(143) + expect(await exiting.done).toMatchObject({ + waitReason: 'session_exit', + sessionStatus: { kind: 'exited', exitCode: null, signal: 'SIGTERM' }, + }) + expect(() => session.startSend({ text: '', submit: false })).toThrow('has exited') + + const startupHandle = new FakePtyHandle() + const startup = new E2BPtySession(fake.sandbox, startupHandle.asHandle(), config()) + const timedOut = expect(startup.initialize()).rejects.toThrow('startup timeout') + await vi.advanceTimersByTimeAsync(100) + await timedOut + }) + + it('handles split prompt text, stale operations, and explicit cancellation', async () => { + vi.useFakeTimers() + const fake = new FakeSandbox() + const handle = new FakePtyHandle() + const session = new E2BPtySession(fake.sandbox, handle.asHandle(), config()) + const initializing = session.initialize() + session.onData(Buffer.from('\x1b]133;D;0\x07')) + await vi.advanceTimersByTimeAsync(20) + session.onData(Buffer.from('dsh> ')) + await vi.advanceTimersByTimeAsync(10) + await initializing + + const operation = session.startSend({ text: 'sleep', submit: true }) + const internal = session as unknown as { + pollReadiness(operation: PtySendOperation): void + interrupt(operation: PtySendOperation): void + settleActive(reason: 'timeout'): void + failActive(error: unknown): void + appendOutput(text: string): void + statusValue: PtySessionStatus + } + internal.pollReadiness({} as PtySendOperation) + internal.interrupt({} as PtySendOperation) + internal.appendOutput('') + fake.pgid = '789\n' + expect(operation.cancel()).toBe(true) + await vi.advanceTimersByTimeAsync(0) + expect(fake.commands).toContain('kill -INT -- -789') + session.onData(Buffer.from('\x1b]133;D;130\x07dsh> ')) + await vi.advanceTimersByTimeAsync(10) + await operation.done + + internal.settleActive('timeout') + internal.failActive(new Error('ignored')) + const operationInternal = operation as unknown as { + append(text: string): void + settle(reason: 'timeout', status: PtySessionStatus, inherited: boolean): void + fail(error: unknown): void + } + operationInternal.append('ignored') + operationInternal.settle('timeout', { kind: 'running' }, false) + operationInternal.fail(new Error('ignored')) + }) + + it('observes AbortSignal and contains send or foreground lookup failures', async () => { + vi.useFakeTimers() + const fake = new FakeSandbox() + const handle = new FakePtyHandle() + const session = new E2BPtySession(fake.sandbox, handle.asHandle(), config()) + await initialize(session) + + const controller = new AbortController() + const aborting = session.startSend({ text: '', submit: false, signal: controller.signal }) + expect(() => session.startSend({ text: '', submit: false })).toThrow('active send') + fake.pgid = 'not-a-pgid\n' + controller.abort() + await expect(aborting.done).rejects.toThrow('cannot resolve foreground process group') + + const already = new AbortController() + already.abort() + expect(() => session.startSend({ text: '', submit: false, signal: already.signal })).toThrow('aborted before write') + + fake.sendError = new Error('send failed') + const failed = session.startSend({ text: 'x', submit: false }) + await expect(failed.done).rejects.toThrow('send failed') + + fake.pgid = '123\n' + await expect(session.signal('SIGKILL')).rejects.toThrow('refusing to SIGKILL') + fake.pgid = '0\n' + await expect(session.signal('SIGINT')).rejects.toThrow('cannot resolve') + + const deferred = Promise.withResolvers() + fake.sendError = undefined + const sendInput = vi.spyOn(fake.sandbox.pty, 'sendInput').mockReturnValueOnce(deferred.promise) + const late = session.startSend({ text: 'late', submit: false }) + session.onData(Buffer.from('\x1b]133;D;0\x07dsh> ')) + await vi.advanceTimersByTimeAsync(10) + await late.done + deferred.reject(new Error('late failure')) + await vi.advanceTimersByTimeAsync(0) + expect(sendInput).toHaveBeenCalled() + }) + + it('preserves startup abort reasons and classifies invalid UTF-8 transport failures', async () => { + const fake = new FakeSandbox() + const abortHandle = new FakePtyHandle() + const abortSession = new E2BPtySession(fake.sandbox, abortHandle.asHandle(), config()) + const controller = new AbortController() + const reason = new Error('startup cancelled') + const initializing = abortSession.initialize(controller.signal) + const rejected = expect(initializing).rejects.toBe(reason) + controller.abort(reason) + await rejected + + const invalidHandle = new FakePtyHandle() + const invalid = new E2BPtySession(fake.sandbox, invalidHandle.asHandle(), config()) + const pending = invalid.startSend({ text: '', submit: false }) + invalid.onData(Uint8Array.from([0xff])) + await expect(pending.done).rejects.toThrow('invalid UTF-8') + expect(invalid.status()).toEqual({ kind: 'exited', exitCode: null, signal: null }) + + const crashHandle = new FakePtyHandle() + const crashed = new E2BPtySession(fake.sandbox, crashHandle.asHandle(), config()) + const active = crashed.startSend({ text: '', submit: false }) + crashHandle.crash('transport gone') + await expect(active.done).rejects.toEqual(new Error('transport gone')) + + const startupExitHandle = new FakePtyHandle() + const startupExit = new E2BPtySession(fake.sandbox, startupExitHandle.asHandle(), config()) + const exitedDuringStartup = expect(startupExit.initialize()).rejects.toThrow('exited during startup') + startupExitHandle.exit(7) + await exitedDuringStartup + }) + + it('covers empty bounded reads and polling an exited active session', async () => { + vi.useFakeTimers() + const fake = new FakeSandbox() + const tinyHandle = new FakePtyHandle() + const tiny = new E2BPtySession(fake.sandbox, tinyHandle.asHandle(), config({ maxReadBytes: 1 })) + tiny.onData(Buffer.from('你')) + expect(tiny.read({ count: 1 })).toMatchObject({ text: '', lineEnd: 0 }) + + const handle = new FakePtyHandle() + const session = new E2BPtySession(fake.sandbox, handle.asHandle(), config()) + const operation = session.startSend({ text: '', submit: false }) + const internal = session as unknown as { + pollReadiness(operation: PtySendOperation): void + clearActive(): void + statusValue: PtySessionStatus + } + internal.statusValue = { kind: 'exited', exitCode: 7, signal: null } + internal.pollReadiness(operation) + expect((await operation.done).waitReason).toBe('session_exit') + internal.clearActive() + }) +}) + +describe('E2BPtySession teardown', () => { + it('terminates the process group once, awaits exit, and disconnects', async () => { + vi.useFakeTimers() + const fake = new FakeSandbox() + const handle = new FakePtyHandle() + fake.onTerm = () => { handle.failExit(143) } + const session = new E2BPtySession(fake.sandbox, handle.asHandle(), config()) + const first = session.close('done') + expect(session.close('again')).toBe(first) + await first + expect(session.status()).toEqual({ kind: 'exited', exitCode: null, signal: 'SIGTERM' }) + expect(handle.disconnects).toBe(1) + expect(() => session.startSend({ text: '', submit: false })).toThrow('closing') + }) + + it('contains an already-gone TERM, escalates to KILL, and reports a survivor', async () => { + vi.useFakeTimers() + const gone = new FakeSandbox() + const goneHandle = new FakePtyHandle() + gone.commandError = commandError(1) + gone.onKill = () => { goneHandle.failExit(137) } + const goneSession = new E2BPtySession(gone.sandbox, goneHandle.asHandle(), config()) + const closingGone = goneSession.close('gone') + await vi.advanceTimersByTimeAsync(20) + await closingGone + expect(gone.killed).toEqual([123]) + expect(goneSession.status()).toEqual({ kind: 'exited', exitCode: null, signal: 'SIGKILL' }) + + const survivor = new FakeSandbox() + const survivorHandle = new FakePtyHandle() + const survivorSession = new E2BPtySession(survivor.sandbox, survivorHandle.asHandle(), config()) + const failed = expect(survivorSession.close('still alive')).rejects.toThrow('surviving pid: 123') + await vi.advanceTimersByTimeAsync(40) + await failed + survivorHandle.exit() + await expect(survivorSession.close('retry')).resolves.toBeUndefined() + }) + + it('propagates cleanup transport failures and lets close retry', async () => { + const fake = new FakeSandbox() + const handle = new FakePtyHandle() + fake.commandError = new Error('TERM transport failed') + const session = new E2BPtySession(fake.sandbox, handle.asHandle(), config()) + await expect(session.close('failure')).rejects.toThrow('TERM transport failed') + handle.exit() + await expect(session.close('retry')).resolves.toBeUndefined() + + const invalidTailHandle = new FakePtyHandle() + const invalidTail = new E2BPtySession(fake.sandbox, invalidTailHandle.asHandle(), config()) + invalidTail.onData(Uint8Array.from([0xe2])) + invalidTailHandle.exit() + await expect(invalidTail.close('invalid tail')).rejects.toThrow('invalid UTF-8') + + const normalHandle = new FakePtyHandle() + normalHandle.disconnectError = new Error('disconnect raced') + const normal = new E2BPtySession(fake.sandbox, normalHandle.asHandle(), config()) + normalHandle.exit(7) + await Promise.resolve() + expect(normal.status()).toEqual({ kind: 'exited', exitCode: 7, signal: null }) + await expect(normal.close('already exited')).resolves.toBeUndefined() + }) + + it('kills a remotely live PTY after its host transport fails', async () => { + const fake = new FakeSandbox() + const handle = new FakePtyHandle() + const session = new E2BPtySession(fake.sandbox, handle.asHandle(), config()) + const active = session.startSend({ text: '', submit: false }) + session.onData(Uint8Array.from([0xff])) + await expect(active.done).rejects.toThrow('invalid UTF-8') + expect(session.status()).toEqual({ kind: 'exited', exitCode: null, signal: null }) + + fake.onTerm = () => { handle.failExit(143) } + await expect(session.close('transport failed')).rejects.toThrow('invalid UTF-8') + expect(fake.commands).toContain('kill -TERM -- -123') + expect(handle.disconnects).toBe(1) + }) +}) diff --git a/packages/pty/pty-e2b/tsconfig.json b/packages/pty/pty-e2b/tsconfig.json new file mode 100644 index 0000000000..0a7647aa80 --- /dev/null +++ b/packages/pty/pty-e2b/tsconfig.json @@ -0,0 +1,16 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": ["src"], + "references": [ + { "path": "../../../vendor/cosmokit" }, + { "path": "../../../vendor/cordis" }, + { "path": "../../../vendor/schemastery" }, + { "path": "../../e2b/e2b" }, + { "path": "../pty" }, + { "path": "../../support/invariants" } + ] +} diff --git a/packages/pty/pty-local/src/sanitize.ts b/packages/pty/pty-local/src/sanitize.ts deleted file mode 100644 index 1f28315fb1..0000000000 --- a/packages/pty/pty-local/src/sanitize.ts +++ /dev/null @@ -1,188 +0,0 @@ -/** Streaming terminal-control sanitizer for the line-oriented first release. */ - -import { Buffer } from 'node:buffer' - -/** OSC marker emitted by the controlled bash before each prompt. */ -export const PROMPT_MARKER_PREFIX = '133;D;' - -/** Exact printable prompt emitted after the private marker. */ -export const CONTROLLED_PROMPT = 'dsh> ' - -/** One sanitized chunk plus whether it contained the owned prompt marker. */ -export interface SanitizedChunk { - text: string - prompt: boolean - /** Printable text after the latest owned marker in this chunk. */ - promptTail?: string -} - -/** - * Remove CSI/OSC/short escape sequences while preserving split-sequence carry. - * Full terminal emulation is deliberately deferred; ordinary line output and - * the private prompt marker are the supported contract. - */ -export class TerminalSanitizer { - private pending = '' - private discardMode: 'osc' | 'csi' | undefined - private discardOscEscape = false - private trailingCarriageReturn = false - private trackingPromptTail = false - - constructor(private readonly maxPendingBytes: number) {} - - /** - * Consume one decoded `node-pty` data chunk. - * @param chunk - decoded terminal data. - * @returns Printable text and whether the private prompt marker completed. - */ - push(chunk: string): SanitizedChunk { - this.pending += this.discardPrefix(chunk) - let text = '' - let prompt = false - let includePromptTail = this.trackingPromptTail - let promptTail = '' - let index = 0 - const appendText = (value: string): void => { - text += value - if (this.trackingPromptTail) promptTail += value - } - while (index < this.pending.length) { - const escape = this.pending.indexOf('\x1b', index) - if (escape < 0) { - appendText(this.pending.slice(index)) - index = this.pending.length - break - } - appendText(this.pending.slice(index, escape)) - if (escape + 1 >= this.pending.length) { - index = escape - break - } - const kind = this.pending[escape + 1] - if (kind === ']') { - const bel = this.pending.indexOf('\x07', escape + 2) - const stringTerminator = this.pending.indexOf('\x1b\\', escape + 2) - let end = -1 - if (bel >= 0 && stringTerminator >= 0) end = Math.min(bel + 1, stringTerminator + 2) - else if (bel >= 0) end = bel + 1 - else if (stringTerminator >= 0) end = stringTerminator + 2 - if (end < 0) { - index = escape - break - } - const terminatorBytes = this.pending[end - 1] === '\x07' ? 1 : 2 - const content = this.pending.slice(escape + 2, end - terminatorBytes) - if (content.startsWith(PROMPT_MARKER_PREFIX)) { - prompt = true - this.trackingPromptTail = true - includePromptTail = true - promptTail = '' - } - index = end - continue - } - if (kind === '[') { - let end = escape + 2 - while (end < this.pending.length) { - const code = this.pending.charCodeAt(end) - if (code >= 0x40 && code <= 0x7e) break - end += 1 - } - if (end >= this.pending.length) { - index = escape - break - } - index = end + 1 - continue - } - // Two-byte escape family (save/restore cursor and similar). - index = escape + 2 - } - this.pending = this.pending.slice(index) - this.enforcePendingBound() - return { - text: this.normalizeText(text), - prompt, - ...includePromptTail ? { promptTail } : {}, - } - } - - /** - * Flush a trailing printable fragment when the PTY exits. - * @returns Remaining printable text; incomplete escapes are discarded. - */ - flush(): string { - const text = this.pending.startsWith('\x1b') ? '' : this.pending - this.pending = '' - this.discardMode = undefined - this.discardOscEscape = false - this.trackingPromptTail = false - const normalized = this.normalizeText(text) - if (!this.trailingCarriageReturn) return normalized - this.trailingCarriageReturn = false - return `${normalized}\n` - } - - private normalizeText(text: string): string { - let complete = this.trailingCarriageReturn ? `\r${text}` : text - this.trailingCarriageReturn = false - if (complete.endsWith('\r')) { - complete = complete.slice(0, -1) - this.trailingCarriageReturn = true - } - return normalizeTerminalText(complete) - } - - private enforcePendingBound(): void { - if (Buffer.byteLength(this.pending) <= this.maxPendingBytes) return - this.discardMode = this.pending[1] === ']' ? 'osc' : 'csi' - this.pending = '' - } - - private discardPrefix(chunk: string): string { - if (this.discardMode === undefined) return chunk - if (this.discardMode === 'csi') { - for (let index = 0; index < chunk.length; index += 1) { - const code = chunk.charCodeAt(index) - if (code >= 0x40 && code <= 0x7e) { - this.discardMode = undefined - return chunk.slice(index + 1) - } - } - return '' - } - - let index = 0 - if (this.discardOscEscape) { - this.discardOscEscape = false - if (chunk.startsWith('\\')) { - this.discardMode = undefined - return chunk.slice(1) - } - } - while (index < chunk.length) { - if (chunk[index] === '\x07') { - this.discardMode = undefined - return chunk.slice(index + 1) - } - if (chunk[index] === '\x1b') { - if (chunk[index + 1] === '\\') { - this.discardMode = undefined - return chunk.slice(index + 2) - } - if (index + 1 === chunk.length) this.discardOscEscape = true - } - index += 1 - } - return '' - } -} - -/** - * Normalize CRLF and standalone carriage returns for line-oriented rendering. - * @param text - sanitized terminal text. - * @returns Line-normalized text with BEL removed. - */ -export function normalizeTerminalText(text: string): string { - return text.replaceAll('\r\n', '\n').replaceAll('\r', '\n').replaceAll('\x07', '') -} diff --git a/packages/pty/pty-local/src/session.ts b/packages/pty/pty-local/src/session.ts index 25ed8fe586..ebf6b559d1 100644 --- a/packages/pty/pty-local/src/session.ts +++ b/packages/pty/pty-local/src/session.ts @@ -1,12 +1,7 @@ -/** Persistent PTY session over the subprocess seam's terminal primitive. */ +/** Local `node-pty` session: bounded output, readiness, signals, and teardown. */ -import { Buffer } from 'node:buffer' -import type { - SubprocessOutcome, - SubprocessTerminalForeground, - SubprocessTerminalHandle, -} from '@deepseek-ai/dsh-subprocess' -import { PtyError } from '@deepseek-ai/dsh-pty' +import type { IDisposable, IPty } from 'node-pty' +import { PtyTerminalSanitizer, PtyTextBuffer, ptySignalName, ptyUtf8Tail } from '@deepseek-ai/dsh-pty' import type { PtyBackendSession, PtyReadRequest, @@ -21,89 +16,30 @@ import type { PtyWaitReason, } from '@deepseek-ai/dsh-pty' import type { ResolvedConfig } from './config.ts' -import { CONTROLLED_PROMPT, TerminalSanitizer } from './sanitize.ts' +import type { ProcessIdentity, ProcessInspector } from './process-inspector.ts' -function utf8Tail(text: string, maxBytes: number): { text: string; truncated: boolean } { - if (Buffer.byteLength(text) <= maxBytes) return { text, truncated: false } - const chars = Array.from(text) - let bytes = 0 - let start = chars.length - while (start > 0) { - const next = Buffer.byteLength(chars[start - 1] as string) - if (bytes + next > maxBytes) break - bytes += next - start -= 1 - } - return { text: chars.slice(start).join(''), truncated: true } -} - -class BoundedTextBuffer { - private value = '' - private dropped = false - - constructor( - private readonly maxBytes: number, - private readonly maxLines?: number, - ) {} - - append(text: string): void { - if (text.length === 0) return - this.value += text - if (this.maxLines !== undefined) { - const lines = this.value.split('\n') - if (lines.length > this.maxLines) { - this.value = lines.slice(lines.length - this.maxLines).join('\n') - this.dropped = true - } - } - const tail = utf8Tail(this.value, this.maxBytes) - this.value = tail.text - this.dropped ||= tail.truncated - } - - consume(): PtySendRead { - const delta = this.value - const truncated = this.dropped - this.value = '' - this.dropped = false - return { delta, truncated } - } - - snapshot(): { text: string; truncated: boolean } { - return { text: this.value, truncated: this.dropped } - } +function delay(ms: number): Promise { + return new Promise(resolve => setTimeout(resolve, ms)) } class LocalSendOperation implements PtySendOperation { - private readonly output: BoundedTextBuffer + private readonly output: PtyTextBuffer private readonly promise: PromiseWithResolvers private finished = false - private cancellationRequested = false - private initialForegroundLeftWait: boolean - private initialForegroundPgid: number | undefined constructor( maxBytes: number, readonly startedAt: number, private readonly onCancel: () => void, ) { - this.output = new BoundedTextBuffer(maxBytes) + this.output = new PtyTextBuffer(maxBytes) this.promise = Promise.withResolvers() - this.initialForegroundLeftWait = true } get done(): Promise { return this.promise.promise } - get settled(): boolean { - return this.finished - } - - get cancelRequested(): boolean { - return this.cancellationRequested - } - append(text: string): void { if (!this.finished) this.output.append(text) } @@ -130,75 +66,50 @@ class LocalSendOperation implements PtySendOperation { return this.output.consume() } - setInitialForeground(foreground: SubprocessTerminalForeground | undefined): void { - this.initialForegroundPgid = foreground?.processGroupId - this.initialForegroundLeftWait = foreground?.inputWaiting !== true - } - - acceptsStdinWait(pgid: number, waiting: boolean): boolean { - // The same group may still expose the wait that existed before terminal.write. - // Observe every poll so a departure before the exact-settlement threshold - // still makes a later return to that wait post-write evidence. - if (pgid !== this.initialForegroundPgid) return waiting - if (!waiting) this.initialForegroundLeftWait = true - return waiting && this.initialForegroundLeftWait - } - cancel(): boolean { if (this.finished) return false - this.cancellationRequested = true this.onCancel() return true } } -/** Backend session wrapping one provider-owned terminal process. */ +/** Backend session wrapping one `node-pty` process and its captured process tree. */ export class LocalPtySession implements PtyBackendSession { motd = '' readonly pid: number - private readonly decoder = new TextDecoder() - private readonly sanitizer: TerminalSanitizer - private readonly scrollback: BoundedTextBuffer - private readonly outputEnded = Promise.withResolvers() - private readonly completion: Promise + private readonly sanitizer: PtyTerminalSanitizer + private readonly scrollback: PtyTextBuffer + private readonly exitPromise: PromiseWithResolvers = Promise.withResolvers() + private readonly dataDisposable: IDisposable + private readonly exitDisposable: IDisposable private statusValue: PtySessionStatus = { kind: 'running' } - // TODO(pty-send-state-consolidation): Fold the per-send fields below - // (active/activeTimer/activeDeadlineTimer/activeAbort/interrupting/ - // activeWrite/pollingReady/polling) into one send-lifecycle owner; the - // cancellation/readiness interplay now has enough pinned tests to carry - // that refactor safely. private active: LocalSendOperation | undefined private activeTimer: NodeJS.Timeout | undefined - private activeDeadlineTimer: NodeJS.Timeout | undefined private activeAbort: (() => void) | undefined - private interrupting: LocalSendOperation | undefined - private activeWrite: Promise | undefined - private pollingReady: LocalSendOperation | undefined - private polling = false private promptSeen = false private promptTextSeen = false - private promptTail = '' private shellPgid: number | undefined private initializing = false private lastOutputAt = Date.now() private closing = false private closePromise: Promise | undefined - private transportFailure: Error | undefined constructor( - private readonly terminal: SubprocessTerminalHandle, + private readonly terminal: IPty, + private readonly inspector: ProcessInspector, private readonly config: ResolvedConfig, ) { this.pid = terminal.pid - this.sanitizer = new TerminalSanitizer(config.maxReadBytes) - this.scrollback = new BoundedTextBuffer(config.scrollbackMaxBytes, config.scrollbackLines) - terminal.output.on('data', this.onTerminalData) - terminal.output.once('end', this.onTerminalEnd) - terminal.output.once('error', this.onTerminalError) - this.completion = terminal.done.then( - outcome => this.onExit(outcome), - (error: unknown) => { this.onTransportFailure(error) }, - ) + this.sanitizer = new PtyTerminalSanitizer(config.maxReadBytes) + this.scrollback = new PtyTextBuffer(config.scrollbackMaxBytes, config.scrollbackLines) + this.dataDisposable = terminal.onData((data) => { this.onData(data) }) + this.exitDisposable = terminal.onExit(({ exitCode, signal }) => { + const tail = this.sanitizer.flush() + this.appendOutput(tail) + this.statusValue = { kind: 'exited', exitCode, signal: ptySignalName(signal) } + this.settleActive('session_exit') + this.exitPromise.resolve() + }) } /** @@ -225,14 +136,7 @@ export class LocalPtySession implements PtyBackendSession { startSend(request: PtySendRequest): PtySendOperation { if (this.closing) throw new Error('PTY session is closing') if (this.statusValue.kind === 'exited') throw new Error('PTY session has exited') - if (this.active !== undefined) { - const draining = this.activeWrite !== undefined - ? ' or draining provider write' - : this.interrupting !== undefined - ? ' or draining foreground interrupt' - : '' - throw new PtyError(`PTY session already has an active send${draining}`, 'SEND_ACTIVE') - } + if (this.active !== undefined) throw new Error('PTY session already has an active send') if (request.signal?.aborted === true) throw new Error('PTY send aborted before write') const operation = new LocalSendOperation( @@ -241,79 +145,29 @@ export class LocalPtySession implements PtyBackendSession { () => { this.interrupt(operation) }, ) this.active = operation - this.resetReadinessEvidence() + this.lastOutputAt = Date.now() + this.promptSeen = false + this.promptTextSeen = false if (request.signal !== undefined) { const onAbort = (): void => { operation.cancel() } request.signal.addEventListener('abort', onAbort, { once: true }) this.activeAbort = () => request.signal?.removeEventListener('abort', onAbort) } - this.activeDeadlineTimer = setTimeout(() => { - if (this.active === operation) { - this.settleActive('timeout', this.activeWrite !== undefined || this.interrupting === operation) - } - }, this.config.timeoutMs) - void this.beginSend(operation, request) + + try { + if (request.text.length > 0) this.terminal.write(request.text) + if (request.submit) this.terminal.write('\r') + } catch (error: unknown) { + this.clearActive() + operation.fail(error) + return operation + } + + this.activeTimer = setInterval(() => { this.pollReadiness(operation) }, this.config.pollIntervalMs) return operation } - private async beginSend(operation: LocalSendOperation, request: PtySendRequest): Promise { - let foreground: SubprocessTerminalForeground | undefined - try { - foreground = await this.terminal.inspectForeground() - } catch (error: unknown) { - // A pre-write inspection failure while cancellation owns the slot must not - // release it: interruptOnce's in-flight foreground signal could land on a - // successor's foreground group. The interrupt path's post-signal tail - // resumes polling, whose guarded catch propagates a persistent failure. - // A retained settled operation implies that same in-flight interrupt, so - // this guard admits only an unsettled active send. - if (this.active === operation && !this.closing && this.interrupting !== operation) { - this.failActive(error) - } - return - } - try { - if (this.active !== operation || this.closing || this.interrupting === operation) return - operation.setInitialForeground(foreground) - const input = `${request.text}${request.submit ? '\r' : ''}` - if (input.length > 0 && !operation.cancelRequested) { - this.resetReadinessEvidence() - const write = this.terminal.write(input) - this.activeWrite = write.then(() => true, () => false) - try { - await write - } finally { - this.activeWrite = undefined - } - } - // Cancellation owns post-write signalling and reservation release. - if (operation.cancelRequested) return - if (this.active === operation && operation.settled) { - this.clearActive() - return - } - // Closing can race the awaited provider write even though static analysis sees only local assignments. - // oxlint-disable-next-line typescript/no-unnecessary-condition -- awaited provider writes can close the session. - if (this.active === operation && !this.closing) { - this.pollingReady = operation - this.schedulePoll(operation) - } - } catch (error: unknown) { - if (this.active === operation && !this.closing) { - if (operation.settled) this.clearActive() - else this.failActive(error) - } - } - } - - private resetReadinessEvidence(): void { - this.lastOutputAt = Date.now() - this.promptSeen = false - this.promptTextSeen = false - this.promptTail = '' - } - read(request: PtyReadRequest): PtyReadResult { const snapshot = this.scrollback.snapshot() const lines = snapshot.text.split('\n') @@ -328,7 +182,7 @@ export class LocalPtySession implements PtyBackendSession { const end = totalLines - offset const start = Math.max(0, end - count) const requested = lines.slice(start, end).join('\n') - const bounded = utf8Tail(requested, this.config.maxReadBytes) + const bounded = ptyUtf8Tail(requested, this.config.maxReadBytes) const returnedLines = bounded.text.length === 0 ? 0 : bounded.text.split('\n').length return { text: bounded.text, @@ -339,10 +193,16 @@ export class LocalPtySession implements PtyBackendSession { } } - async signal(signal: PtySignal): Promise { - if (this.closing) throw new Error('PTY session is closing') - const targetPgid = await this.terminal.signalForeground(signal) - return { delivered: true, targetPgid } + signal(signal: PtySignal): Promise { + return Promise.resolve().then(() => { + const pgid = this.inspector.foregroundPgid(this.pid) + if (pgid === undefined) throw new Error(`cannot resolve foreground process group for PTY ${this.pid}`) + if (signal === 'SIGKILL' && pgid === this.pid) { + throw new Error('refusing to SIGKILL the PTY shell; use terminal_close') + } + this.inspector.signalGroup(pgid, signal) + return { delivered: true, targetPgid: pgid } + }) } status(): PtySessionStatus { @@ -361,56 +221,21 @@ export class LocalPtySession implements PtyBackendSession { return closing } - private readonly onTerminalData = (chunk: Buffer | Uint8Array | string): void => { - const bytes = typeof chunk === 'string' ? Buffer.from(chunk, 'utf8') : chunk - this.onData(this.decoder.decode(bytes, { stream: true })) - } - - private readonly onTerminalEnd = (): void => { - this.onData(this.decoder.decode()) - this.appendOutput(this.sanitizer.flush()) - this.outputEnded.resolve() - } - - private readonly onTerminalError = (error: Error): void => { - this.onTransportFailure(error) - this.outputEnded.resolve() - } - private onData(data: string): void { const sanitized = this.sanitizer.push(data) this.appendOutput(sanitized.text) if (sanitized.prompt) { - // TODO(pty-delayed-signal-prompt): With a reproducer, define a marker-generation boundary - // before attributing a signal-delayed prompt to a later send. + const foregroundPgid = this.inspector.foregroundPgid(this.pid) + if (this.shellPgid === undefined) this.shellPgid = foregroundPgid // Bash can print PROMPT_COMMAND before the kernel publishes its return // to the foreground process group. Retain the marker; polling below is // the authority that accepts it only after bash owns the foreground. this.promptSeen = true - this.promptTail = '' + this.promptTextSeen = sanitized.promptText === true this.lastOutputAt = Date.now() + } else if (this.promptSeen && sanitized.promptText === true) { + this.promptTextSeen = true } - if (this.promptSeen && sanitized.promptTail !== undefined) { - const remaining = Math.max(0, CONTROLLED_PROMPT.length + 1 - this.promptTail.length) - this.promptTail += sanitized.promptTail.slice(0, remaining) - if (sanitized.promptTail.length > remaining) this.promptTail = `${CONTROLLED_PROMPT}\0` - this.promptTextSeen = this.promptTail === CONTROLLED_PROMPT - } - } - - private async onExit(outcome: SubprocessOutcome): Promise { - await this.outputEnded.promise - if (this.transportFailure !== undefined) return - this.statusValue = { kind: 'exited', exitCode: outcome.exitCode, signal: outcome.signal } - this.settleActive('session_exit') - } - - private onTransportFailure(error: unknown): void { - const failure = error instanceof Error ? error : new Error(String(error)) - this.transportFailure ??= failure - this.statusValue = { kind: 'exited', exitCode: null, signal: null } - this.failActive(failure) - void this.terminal.terminate().catch(() => {}) } private appendOutput(text: string): void { @@ -420,94 +245,60 @@ export class LocalPtySession implements PtyBackendSession { this.active?.append(text) } - private schedulePoll(operation: LocalSendOperation, delayMs = this.config.pollIntervalMs): void { - if (this.active !== operation || this.interrupting === operation || this.polling) return - if (this.activeTimer !== undefined) clearTimeout(this.activeTimer) - this.activeTimer = setTimeout(() => { - this.activeTimer = undefined - void this.pollReadiness(operation) - }, delayMs) - } - - private async pollReadiness(operation: LocalSendOperation): Promise { - if (this.active !== operation || this.polling) return - this.polling = true - try { - if (this.statusValue.kind === 'exited') { - this.settleActive('session_exit') - return - } - const foreground = await this.terminal.inspectForeground() - if (this.active !== operation || this.closing || this.interrupting === operation) return - const idleFor = Date.now() - this.lastOutputAt - if (this.promptSeen && foreground !== undefined && this.shellPgid === undefined) { - this.shellPgid = foreground.processGroupId - } - if (this.promptSeen && this.promptTextSeen && idleFor >= this.config.pollIntervalMs - && foreground?.processGroupId === this.shellPgid) { - this.settleActive('stdin_read') - return - } - const elapsed = Date.now() - operation.startedAt - const startupHasOutput = !this.initializing || this.scrollback.snapshot().text.length > 0 - const acceptsStdinWait = startupHasOutput && foreground !== undefined - && operation.acceptsStdinWait(foreground.processGroupId, foreground.inputWaiting) - if (elapsed >= this.config.exactProbeAfterMs && acceptsStdinWait) { - this.settleActive('stdin_read') - return - } - // A prompt candidate can race bash's foreground handoff, but an interactive - // child also inherits PROMPT_COMMAND. Silence therefore remains the bound - // on waiting for shell ownership instead of letting a child marker suppress - // readiness until the absolute timeout. - const handoffGrace = this.promptSeen ? this.config.handoffGraceMs : 0 - if (startupHasOutput && idleFor >= this.config.idleSilenceMs + handoffGrace) { - this.settleActive('inferred_idle') - } - } catch (error: unknown) { - if (this.active === operation && !this.closing && this.interrupting !== operation) this.failActive(error) - } finally { - this.polling = false - const active = this.active - // Awaited provider inspection can clear or replace the active send despite static analysis. - // oxlint-disable-next-line typescript/no-unnecessary-condition -- awaited inspection can replace the active send. - if (active !== undefined && this.pollingReady === active) this.schedulePoll(active) + private pollReadiness(operation: LocalSendOperation): void { + if (this.active !== operation) return + if (this.statusValue.kind === 'exited') { + this.settleActive('session_exit') + return } + if (this.promptSeen && this.promptTextSeen && Date.now() - this.lastOutputAt >= this.config.pollIntervalMs) { + const pgid = this.inspector.foregroundPgid(this.pid) + if (this.shellPgid !== undefined && pgid === this.shellPgid) { + this.settleActive('stdin_read') + return + } + } + const elapsed = Date.now() - operation.startedAt + const startupHasOutput = !this.initializing || this.scrollback.snapshot().text.length > 0 + if (startupHasOutput && elapsed >= this.config.exactProbeAfterMs) { + const pgid = this.inspector.foregroundPgid(this.pid) + if (pgid !== undefined && this.inspector.isStdinWaiting(pgid)) { + this.settleActive('stdin_read') + return + } + } + // A prompt candidate can race bash's foreground handoff, but an interactive + // child also inherits PROMPT_COMMAND. Silence therefore remains the bound + // on waiting for shell ownership instead of letting a child marker suppress + // readiness until the absolute timeout. When a prompt marker was seen, the + // configured grace holds the fallback past the silence bound so polls in + // that window can observe the foreground handoff and settle as stdin_read. + const idleFor = Date.now() - this.lastOutputAt + const handoffGrace = this.promptSeen ? this.config.handoffGraceMs : 0 + if (startupHasOutput && idleFor >= this.config.idleSilenceMs + handoffGrace) { + this.settleActive('inferred_idle') + return + } + if (elapsed >= this.config.timeoutMs) this.settleActive('timeout') } - private settleActive(waitReason: PtyWaitReason, retainOwnership = false): void { + private settleActive(waitReason: PtyWaitReason): void { const operation = this.active if (operation === undefined) return const scrollbackTruncated = this.scrollback.snapshot().truncated - if (retainOwnership) { - this.stopPolling() - this.activeAbort?.() - this.activeAbort = undefined - } else { - this.clearActive() - } + this.clearActive() operation.settle(waitReason, this.statusValue, scrollbackTruncated) } private stopPolling(): void { - this.stopReadinessPolling() - if (this.activeDeadlineTimer !== undefined) clearTimeout(this.activeDeadlineTimer) - this.activeDeadlineTimer = undefined - } - - private stopReadinessPolling(): void { - if (this.activeTimer !== undefined) clearTimeout(this.activeTimer) + if (this.activeTimer !== undefined) clearInterval(this.activeTimer) this.activeTimer = undefined - this.pollingReady = undefined } private clearActive(): void { - const operation = this.active this.stopPolling() this.activeAbort?.() this.activeAbort = undefined - if (this.interrupting === operation) this.interrupting = undefined - this.pollingReady = undefined this.active = undefined } @@ -520,46 +311,104 @@ export class LocalPtySession implements PtyBackendSession { private interrupt(operation: LocalSendOperation): void { if (this.active !== operation) return - this.interrupting = operation - this.stopReadinessPolling() - void this.interruptOnce(operation) + try { + const pgid = this.inspector.foregroundPgid(this.pid) + if (pgid === undefined) throw new Error(`cannot resolve foreground process group for PTY ${this.pid}`) + this.inspector.signalGroup(pgid, 'SIGINT') + } catch (error: unknown) { + this.failActive(error) + } } - private async interruptOnce(operation: LocalSendOperation): Promise { - try { - const activeWrite = this.activeWrite - if (activeWrite !== undefined && !await activeWrite) return - await this.terminal.signalForeground('SIGINT') - } catch (error: unknown) { - if (this.active === operation && !this.closing) this.onTransportFailure(error) - return - } finally { - if (this.interrupting === operation) this.interrupting = undefined + private survivors(members: ProcessIdentity[]): ProcessIdentity[] { + return members.filter(member => this.inspector.isAlive(member)) + } + + private descendants(): ProcessIdentity[] { + return this.inspector.processTree(this.pid).filter(member => member.pid !== this.pid) + } + + private async waitForExit(members: ProcessIdentity[]): Promise { + const deadline = Date.now() + this.config.disposeGraceMs + let survivors = this.survivors(members) + while (survivors.length > 0 && Date.now() < deadline) { + await delay(Math.min(25, Math.max(1, deadline - Date.now()))) + survivors = this.survivors(members) } - if (this.active === operation && operation.settled) { - this.clearActive() - } else if (this.active === operation && !this.closing) { - this.pollingReady = operation - this.schedulePoll(operation, 0) + return survivors + } + + private signalMembers(members: ProcessIdentity[], signal: 'SIGTERM' | 'SIGKILL'): void { + for (const member of members) { + try { + this.inspector.signalProcess(member, signal) + } catch (_alreadyExitedDuringSignal) { + // Identity is rechecked by the inspector; a same-tick exit is success. + } + } + } + + private unionMembers(...groups: ProcessIdentity[][]): ProcessIdentity[] { + const members: ProcessIdentity[] = [] + const seen = new Set() + for (const group of groups) { + for (const member of group) { + const key = JSON.stringify([member.pid, member.started]) + if (seen.has(key)) continue + seen.add(key) + members.push(member) + } + } + return members + } + + private async stopDescendants(): Promise { + const captured = this.descendants() + this.signalMembers(captured, 'SIGTERM') + const capturedSurvivors = await this.waitForExit(captured) + // A TERM-handling descendant may have forked while winding down. Rescan + // while the shell can still reap every member, then kill both the fresh + // tree and captured survivors that were reparented out of that tree. + const members = this.unionMembers(capturedSurvivors, this.descendants()) + this.signalMembers(members, 'SIGKILL') + const survivors = await this.waitForExit(members) + return this.survivors(this.unionMembers(survivors, this.descendants())) + } + + private async stopShell(): Promise { + try { + this.terminal.kill('SIGTERM') + } catch (_topLevelAlreadyExitedDuringTerm) { + // The exit notification remains authoritative. + } + if (this.statusValue.kind === 'running') { + await Promise.race([this.exitPromise.promise, delay(this.config.disposeGraceMs)]) + } + if (this.statusValue.kind === 'running') { + try { + this.terminal.kill('SIGKILL') + } catch (_topLevelAlreadyExitedDuringKill) { + // The exit notification remains authoritative. + } + await Promise.race([this.exitPromise.promise, delay(this.config.disposeGraceMs)]) + } + if (this.statusValue.kind === 'running') { + throw new Error(`PTY cleanup failed; surviving pids: ${this.pid}`) } } private async closeOnce(reason: string): Promise { + this.dataDisposable.dispose() // Stop readiness polling but retain the active operation: teardown settles // it as session_exit below, so an in-flight send is never mis-settled as // stdin_read/inferred_idle/timeout during the grace period. this.stopPolling() - try { - await this.terminal.terminate() - } catch (error: unknown) { - throw new Error(`PTY cleanup failed (${reason})`, { cause: error }) + const survivors = await this.stopDescendants() + if (survivors.length > 0) { + throw new Error(`PTY cleanup failed (${reason}); surviving pids: ${survivors.map(member => member.pid).join(', ')}`) } - // Quiescence is the active send's terminal outcome. + await this.stopShell() this.settleActive('session_exit') - await this.completion - this.terminal.output.off('data', this.onTerminalData) - this.terminal.output.off('end', this.onTerminalEnd) - this.terminal.output.off('error', this.onTerminalError) - if (this.transportFailure !== undefined) throw this.transportFailure + this.exitDisposable.dispose() } } diff --git a/packages/pty/pty-local/tests/sanitize.spec.ts b/packages/pty/pty-local/tests/sanitize.spec.ts index f6649b3c6f..06ec076358 100644 --- a/packages/pty/pty-local/tests/sanitize.spec.ts +++ b/packages/pty/pty-local/tests/sanitize.spec.ts @@ -1,17 +1,17 @@ import { describe, expect, it } from 'vitest' -import { normalizeTerminalText, TerminalSanitizer } from '@deepseek-ai/dsh-pty-local/src/sanitize.ts' +import { normalizePtyTerminalText, PtyTerminalSanitizer } from '@deepseek-ai/dsh-pty' -describe('TerminalSanitizer', () => { +describe('PtyTerminalSanitizer', () => { it('removes split CSI and owned OSC prompt markers', () => { - const sanitizer = new TerminalSanitizer(64) + const sanitizer = new PtyTerminalSanitizer(64) expect(sanitizer.push('red\x1b[3')).toEqual({ text: 'red', prompt: false }) expect(sanitizer.push('1m text\x1b[0m\r\n')).toEqual({ text: ' text\n', prompt: false }) expect(sanitizer.push('\x1b]133;')).toEqual({ text: '', prompt: false }) - expect(sanitizer.push('D;0\x07dsh> ')).toEqual({ text: 'dsh> ', prompt: true, promptTail: 'dsh> ' }) + expect(sanitizer.push('D;0\x07dsh> ')).toEqual({ text: 'dsh> ', prompt: true, promptText: true }) }) it('drops unrelated OSC, short escapes, BEL, and incomplete trailing escape', () => { - const sanitizer = new TerminalSanitizer(64) + const sanitizer = new PtyTerminalSanitizer(64) expect(sanitizer.push('a\x1b]0;title\x1b\\b\x1b7c\x07')).toEqual({ text: 'abc', prompt: false }) expect(sanitizer.push('tail\x1b')).toEqual({ text: 'tail', prompt: false }) expect(sanitizer.flush()).toBe('') @@ -22,11 +22,11 @@ describe('TerminalSanitizer', () => { }) it('normalizes CRLF and standalone carriage returns', () => { - expect(normalizeTerminalText('a\r\nb\rc\x07')).toBe('a\nb\nc') + expect(normalizePtyTerminalText('a\r\nb\rc\x07')).toBe('a\nb\nc') }) it('carries a trailing carriage return across data chunks and flushes standalone CR', () => { - const sanitizer = new TerminalSanitizer(64) + const sanitizer = new PtyTerminalSanitizer(64) expect(sanitizer.push('a\r')).toEqual({ text: 'a', prompt: false }) expect(sanitizer.push('\nb')).toEqual({ text: '\nb', prompt: false }) expect(sanitizer.push('\r')).toEqual({ text: '', prompt: false }) @@ -34,41 +34,41 @@ describe('TerminalSanitizer', () => { }) it('reports printable prompt text that follows a marker in a later chunk', () => { - const sanitizer = new TerminalSanitizer(64) - expect(sanitizer.push('\x1b]133;D;0\x07')).toEqual({ text: '', prompt: true, promptTail: '' }) - expect(sanitizer.push('dsh> ')).toEqual({ text: 'dsh> ', prompt: false, promptTail: 'dsh> ' }) + const sanitizer = new PtyTerminalSanitizer(64) + expect(sanitizer.push('\x1b]133;D;0\x07')).toEqual({ text: '', prompt: true }) + expect(sanitizer.push('dsh> ')).toEqual({ text: 'dsh> ', prompt: false, promptText: true }) }) it('bounds and discards unterminated control sequences through their terminators', () => { - const oscBel = new TerminalSanitizer(8) + const oscBel = new PtyTerminalSanitizer(8) expect(oscBel.push(`\x1b]0;${'x'.repeat(16)}`)).toEqual({ text: '', prompt: false }) expect(oscBel.push('more\x07tail')).toEqual({ text: 'tail', prompt: false }) - const oscSt = new TerminalSanitizer(8) + const oscSt = new PtyTerminalSanitizer(8) oscSt.push(`\x1b]0;${'x'.repeat(16)}`) expect(oscSt.push('more\x1b')).toEqual({ text: '', prompt: false }) expect(oscSt.push('\\tail')).toEqual({ text: 'tail', prompt: false }) - const oscDirectSt = new TerminalSanitizer(8) + const oscDirectSt = new PtyTerminalSanitizer(8) oscDirectSt.push(`\x1b]0;${'x'.repeat(16)}`) expect(oscDirectSt.push('more\x1b\\tail')).toEqual({ text: 'tail', prompt: false }) - const oscFalseSt = new TerminalSanitizer(8) + const oscFalseSt = new PtyTerminalSanitizer(8) oscFalseSt.push(`\x1b]0;${'x'.repeat(16)}`) oscFalseSt.push('\x1b') expect(oscFalseSt.push('more')).toEqual({ text: '', prompt: false }) expect(oscFalseSt.push('\x07tail')).toEqual({ text: 'tail', prompt: false }) - const oscNonTerminatingEscape = new TerminalSanitizer(8) + const oscNonTerminatingEscape = new PtyTerminalSanitizer(8) oscNonTerminatingEscape.push(`\x1b]0;${'x'.repeat(16)}`) expect(oscNonTerminatingEscape.push('more\x1bxmore\x07tail')).toEqual({ text: 'tail', prompt: false }) - const csi = new TerminalSanitizer(8) + const csi = new PtyTerminalSanitizer(8) expect(csi.push(`\x1b[${'1'.repeat(16)}`)).toEqual({ text: '', prompt: false }) expect(csi.push('123')).toEqual({ text: '', prompt: false }) expect(csi.push('mtext')).toEqual({ text: 'text', prompt: false }) - const flushed = new TerminalSanitizer(8) + const flushed = new PtyTerminalSanitizer(8) flushed.push(`\x1b]0;${'x'.repeat(16)}`) expect(flushed.flush()).toBe('') expect(flushed.push('text')).toEqual({ text: 'text', prompt: false }) diff --git a/packages/pty/pty/src/index.ts b/packages/pty/pty/src/index.ts index 8ff4168e1a..5cb375fbca 100644 --- a/packages/pty/pty/src/index.ts +++ b/packages/pty/pty/src/index.ts @@ -41,6 +41,15 @@ export type { PtyWaitReason, } from './types.ts' export { PtyBackendCleanupError } from './types.ts' +export { + normalizePtyTerminalText, + PTY_PROMPT_MARKER_PREFIX, + PtyTerminalSanitizer, + PtyTextBuffer, + ptySignalName, + ptyUtf8Tail, +} from './terminal.ts' +export type { PtySanitizedChunk } from './terminal.ts' /** Opaque identity minted by {@link PtyService} for one live PTY session. */ export type PtySessionId = PtySessionIdValue diff --git a/packages/pty/pty/src/terminal.ts b/packages/pty/pty/src/terminal.ts new file mode 100644 index 0000000000..ed005dda41 --- /dev/null +++ b/packages/pty/pty/src/terminal.ts @@ -0,0 +1,273 @@ +/** Backend-neutral line-oriented terminal buffering and control-sequence sanitization. */ + +import { Buffer } from 'node:buffer' +import { constants } from 'node:os' +import type { PtySendRead } from './types.ts' + +/** OSC marker emitted by a controlled bash before each prompt. */ +export const PTY_PROMPT_MARKER_PREFIX = '133;D;' + +/** One sanitized chunk plus whether it contained the controlled prompt marker. */ +export interface PtySanitizedChunk { + /** Printable, line-normalized terminal text. */ + text: string + /** Whether the chunk completed the controlled prompt marker. */ + prompt: boolean + /** Present when printable text followed the latest controlled prompt marker. */ + promptText?: true +} + +/** + * Return the largest code-point-aligned UTF-8 tail within a byte cap. + * @param text - Candidate terminal text. + * @param maxBytes - Maximum retained UTF-8 bytes. + * @returns The retained tail and whether its head was dropped. + */ +export function ptyUtf8Tail(text: string, maxBytes: number): { text: string; truncated: boolean } { + if (Buffer.byteLength(text) <= maxBytes) return { text, truncated: false } + const chars = Array.from(text) + let bytes = 0 + let start = chars.length + while (start > 0) { + const next = Buffer.byteLength(chars[start - 1] as string) + if (bytes + next > maxBytes) break + bytes += next + start -= 1 + } + return { text: chars.slice(start).join(''), truncated: true } +} + +/** UTF-8 and optionally line-bounded terminal text buffer. */ +export class PtyTextBuffer { + private value = '' + private dropped = false + + /** + * @param maxBytes - Maximum retained UTF-8 bytes. + * @param maxLines - Optional maximum retained logical lines. + */ + constructor( + private readonly maxBytes: number, + private readonly maxLines?: number, + ) {} + + /** + * Append terminal text and drop the oldest excess. + * @param text - Decoded and sanitized terminal text. + */ + append(text: string): void { + if (text.length === 0) return + this.value += text + if (this.maxLines !== undefined) { + const lines = this.value.split('\n') + if (lines.length > this.maxLines) { + this.value = lines.slice(lines.length - this.maxLines).join('\n') + this.dropped = true + } + } + const tail = ptyUtf8Tail(this.value, this.maxBytes) + this.value = tail.text + this.dropped ||= tail.truncated + } + + /** + * Consume all currently retained operation text. + * @returns The delta and whether older text was dropped. + */ + consume(): PtySendRead { + const delta = this.value + const truncated = this.dropped + this.value = '' + this.dropped = false + return { delta, truncated } + } + + /** + * Return the retained text without consuming it. + * @returns The retained text and whether its head was dropped. + */ + snapshot(): { text: string; truncated: boolean } { + return { text: this.value, truncated: this.dropped } + } +} + +/** + * Streaming terminal-control sanitizer for line-oriented PTY backends. + * Full terminal emulation is deliberately outside the PTY seam. + */ +export class PtyTerminalSanitizer { + private pending = '' + private discardMode: 'osc' | 'csi' | undefined + private discardOscEscape = false + private trailingCarriageReturn = false + private awaitingPromptText = false + + /** @param maxPendingBytes - Bound for an incomplete terminal-control sequence. */ + constructor(private readonly maxPendingBytes: number) {} + + /** + * Consume one decoded PTY data chunk. + * @param chunk - Decoded terminal data. + * @returns Printable text and prompt-marker facts. + */ + push(chunk: string): PtySanitizedChunk { + this.pending += this.discardPrefix(chunk) + let text = '' + let prompt = false + let promptText = false + let index = 0 + const appendText = (value: string): boolean => { + text += value + if (this.awaitingPromptText && value.replace(/[\r\n\x07]/g, '').length > 0) { + this.awaitingPromptText = false + return true + } + return false + } + while (index < this.pending.length) { + const escape = this.pending.indexOf('\x1b', index) + if (escape < 0) { + promptText = appendText(this.pending.slice(index)) || promptText + index = this.pending.length + break + } + promptText = appendText(this.pending.slice(index, escape)) || promptText + if (escape + 1 >= this.pending.length) { + index = escape + break + } + const kind = this.pending[escape + 1] + if (kind === ']') { + const bel = this.pending.indexOf('\x07', escape + 2) + const stringTerminator = this.pending.indexOf('\x1b\\', escape + 2) + let end = -1 + if (bel >= 0 && stringTerminator >= 0) end = Math.min(bel + 1, stringTerminator + 2) + else if (bel >= 0) end = bel + 1 + else if (stringTerminator >= 0) end = stringTerminator + 2 + if (end < 0) { + index = escape + break + } + const terminatorBytes = this.pending[end - 1] === '\x07' ? 1 : 2 + const content = this.pending.slice(escape + 2, end - terminatorBytes) + if (content.startsWith(PTY_PROMPT_MARKER_PREFIX)) { + prompt = true + promptText = false + this.awaitingPromptText = true + } + index = end + continue + } + if (kind === '[') { + let end = escape + 2 + while (end < this.pending.length) { + const code = this.pending.charCodeAt(end) + if (code >= 0x40 && code <= 0x7e) break + end += 1 + } + if (end >= this.pending.length) { + index = escape + break + } + index = end + 1 + continue + } + index = escape + 2 + } + this.pending = this.pending.slice(index) + this.enforcePendingBound() + return { text: this.normalizeText(text), prompt, ...promptText ? { promptText: true } : {} } + } + + /** + * Flush printable trailing data and discard incomplete controls. + * @returns Remaining normalized printable text. + */ + flush(): string { + const text = this.pending.startsWith('\x1b') ? '' : this.pending + this.pending = '' + this.discardMode = undefined + this.discardOscEscape = false + this.awaitingPromptText = false + const normalized = this.normalizeText(text) + if (!this.trailingCarriageReturn) return normalized + this.trailingCarriageReturn = false + return `${normalized}\n` + } + + private normalizeText(text: string): string { + let complete = this.trailingCarriageReturn ? `\r${text}` : text + this.trailingCarriageReturn = false + if (complete.endsWith('\r')) { + complete = complete.slice(0, -1) + this.trailingCarriageReturn = true + } + return normalizePtyTerminalText(complete) + } + + private enforcePendingBound(): void { + if (Buffer.byteLength(this.pending) <= this.maxPendingBytes) return + this.discardMode = this.pending[1] === ']' ? 'osc' : 'csi' + this.pending = '' + } + + private discardPrefix(chunk: string): string { + if (this.discardMode === undefined) return chunk + if (this.discardMode === 'csi') { + for (let index = 0; index < chunk.length; index += 1) { + const code = chunk.charCodeAt(index) + if (code >= 0x40 && code <= 0x7e) { + this.discardMode = undefined + return chunk.slice(index + 1) + } + } + return '' + } + + let index = 0 + if (this.discardOscEscape) { + this.discardOscEscape = false + if (chunk.startsWith('\\')) { + this.discardMode = undefined + return chunk.slice(1) + } + } + while (index < chunk.length) { + if (chunk[index] === '\x07') { + this.discardMode = undefined + return chunk.slice(index + 1) + } + if (chunk[index] === '\x1b') { + if (chunk[index + 1] === '\\') { + this.discardMode = undefined + return chunk.slice(index + 2) + } + if (index + 1 === chunk.length) this.discardOscEscape = true + } + index += 1 + } + return '' + } +} + +/** + * Normalize CRLF and standalone carriage returns for line-oriented rendering. + * @param text - Sanitized terminal text. + * @returns Line-normalized text with BEL removed. + */ +export function normalizePtyTerminalText(text: string): string { + return text.replaceAll('\r\n', '\n').replaceAll('\r', '\n').replaceAll('\x07', '') +} + +/** + * Convert a platform signal number into the seam's signal-name vocabulary. + * @param number - Platform signal number, zero, or an absent signal. + * @returns The matching Node signal name, or `null` when unknown or absent. + */ +export function ptySignalName(number: number | undefined): NodeJS.Signals | null { + if (number === undefined || number === 0) return null + for (const [name, value] of Object.entries(constants.signals)) { + if (value === number) return name as NodeJS.Signals + } + return null +} diff --git a/scripts/verify-package-readme-model-experience.ts b/scripts/verify-package-readme-model-experience.ts index 90aa22e322..5ab5dc2e4e 100644 --- a/scripts/verify-package-readme-model-experience.ts +++ b/scripts/verify-package-readme-model-experience.ts @@ -47,6 +47,7 @@ const SENTENCE_MODEL_EXPERIENCE: Readonly> = { 'packages/bash/bash-local': { kind: 'indirect', reason: 'The executor backend delegates model rendering to dsh-tool-bash.' }, 'packages/bash/pwsh-local': { kind: 'indirect', reason: 'The executor backend delegates model rendering to dsh-tool-pwsh.' }, 'packages/code-runtime/code-runtime': { kind: 'indirect', reason: 'The service interface delegates model rendering to Code Mode in dsh-tools.' }, + 'packages/code-runtime/code-runtime-e2b': { kind: 'indirect', reason: 'The E2B backend delegates model rendering to Code Mode in dsh-tools.' }, 'packages/code-runtime/code-runtime-worker': { kind: 'indirect', reason: 'The worker backend delegates model rendering to Code Mode in dsh-tools.' }, 'packages/typert/registry': { kind: 'none', reason: 'Runtime type registry; consumers (cordis_inspect, wire faces, gates) own any model-visible projection of registry contents.' }, 'packages/typert/loader': { kind: 'none', reason: 'Loader integration only registers generated artifacts; consumers own any model-visible projection.' }, @@ -98,6 +99,7 @@ const SENTENCE_MODEL_EXPERIENCE: Readonly> = { 'packages/llm/llm': { kind: 'none', reason: 'The adapter registry forwards already-assembled requests unchanged.' }, 'packages/llm/token-meter': { kind: 'indirect', reason: 'The measurement service leaves model-visible changes to its consumers.' }, 'packages/lsp/lsp': { kind: 'indirect', reason: 'The provider registry delegates model rendering to dsh-tool-lsp.' }, + 'packages/lsp/lsp-e2b': { kind: 'indirect', reason: 'The E2B provider backend delegates model rendering to dsh-tool-lsp.' }, 'packages/lsp/lsp-local': { kind: 'indirect', reason: 'The provider backend delegates model rendering to dsh-tool-lsp.' }, 'packages/subprocess/subprocess': { kind: 'indirect', reason: 'The seam delegates all model rendering to consumer seams such as the bash executor family.' }, 'packages/subprocess/subprocess-e2b': { kind: 'indirect', reason: 'The remote spawn backend delegates model rendering to consumer seams such as the bash executor family.' }, diff --git a/tsconfig.host.json b/tsconfig.host.json index 03b1bcd2dd..1f70e79a4f 100644 --- a/tsconfig.host.json +++ b/tsconfig.host.json @@ -166,10 +166,12 @@ { "path": "./packages/subprocess/subprocess-e2b" }, { "path": "./packages/bash/bash" }, { "path": "./packages/pty/pty" }, + { "path": "./packages/pty/pty-e2b" }, { "path": "./packages/pty/pty-local" }, { "path": "./packages/pty/tool-bash-persistent" }, { "path": "./packages/pty/tool-pty" }, { "path": "./packages/code-runtime/code-runtime" }, + { "path": "./packages/code-runtime/code-runtime-e2b" }, { "path": "./packages/code-runtime/code-runtime-worker" }, { "path": "./packages/llm/llm-deepseek" }, { "path": "./packages/llm/llm-pi-ai" }, @@ -266,6 +268,7 @@ { "path": "./packages/sdk/create-sdk" }, { "path": "./packages/sdk/telemetry" }, { "path": "./packages/lsp/lsp" }, + { "path": "./packages/lsp/lsp-e2b" }, { "path": "./packages/lsp/lsp-local" }, { "path": "./packages/lsp/tool-lsp" }, { "path": "./apps/cli" } From e64d40837c04bc6b39ea958ce8afc26cba97b2ed Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Tue, 28 Jul 2026 14:52:37 +0800 Subject: [PATCH 065/108] refactor(e2b): group remote providers --- ...026-07-27-e2b-remote-runtime-poc.i18n.yaml | 4 +- .../2026-07-27-e2b-remote-runtime-poc.md | 40 ++++++++++--- .../2026-07-27-e2b-remote-runtime-poc.zh.md | 40 ++++++++++--- ...active-semantic-code-runtime-poc.i18n.yaml | 6 -- ...b-interactive-semantic-code-runtime-poc.md | 57 ------------------- ...nteractive-semantic-code-runtime-poc.zh.md | 57 ------------------- AGENTS.md | 21 +++---- packages/README.i18n.yaml | 4 +- packages/README.md | 2 +- packages/README.zh.md | 2 +- packages/code-runtime/README.i18n.yaml | 4 +- packages/code-runtime/README.md | 4 +- packages/code-runtime/README.zh.md | 4 +- .../code-runtime-e2b/README.i18n.yaml | 6 -- packages/e2b/README.i18n.yaml | 4 +- packages/e2b/README.md | 12 ++-- packages/e2b/README.zh.md | 12 ++-- .../code-runtime-e2b}/README.i18n.yaml | 6 +- .../code-runtime-e2b/README.md | 2 +- .../code-runtime-e2b/README.zh.md | 2 +- .../code-runtime-e2b/package.json | 0 .../code-runtime-e2b/src/index.ts | 0 .../code-runtime-e2b/src/invariant.ts | 0 .../code-runtime-e2b/src/runner-source.ts | 0 .../code-runtime-e2b/tests/runtime.spec.ts | 0 .../code-runtime-e2b/tsconfig.json | 8 +-- packages/{fs => e2b}/fs-e2b/README.i18n.yaml | 6 +- packages/{fs => e2b}/fs-e2b/README.md | 4 +- packages/{fs => e2b}/fs-e2b/README.zh.md | 4 +- packages/{fs => e2b}/fs-e2b/package.json | 0 packages/{fs => e2b}/fs-e2b/src/index.ts | 0 packages/{fs => e2b}/fs-e2b/src/invariant.ts | 0 .../fs-e2b/tests/filesystem.spec.ts | 0 packages/{fs => e2b}/fs-e2b/tsconfig.json | 4 +- .../pty-e2b => e2b/lsp-e2b}/README.i18n.yaml | 6 +- packages/{lsp => e2b}/lsp-e2b/README.md | 2 +- packages/{lsp => e2b}/lsp-e2b/README.zh.md | 2 +- packages/{lsp => e2b}/lsp-e2b/package.json | 0 packages/{lsp => e2b}/lsp-e2b/src/index.ts | 0 .../{lsp => e2b}/lsp-e2b/src/invariant.ts | 0 .../{lsp => e2b}/lsp-e2b/src/proxy-source.ts | 0 .../{lsp => e2b}/lsp-e2b/src/transport.ts | 0 .../lsp-e2b/tests/provider.spec.ts | 0 .../lsp-e2b/tests/transport.spec.ts | 0 packages/{lsp => e2b}/lsp-e2b/tsconfig.json | 8 +-- .../lsp-e2b => e2b/pty-e2b}/README.i18n.yaml | 6 +- packages/{pty => e2b}/pty-e2b/README.md | 2 +- packages/{pty => e2b}/pty-e2b/README.zh.md | 2 +- packages/{pty => e2b}/pty-e2b/package.json | 0 packages/{pty => e2b}/pty-e2b/src/config.ts | 0 packages/{pty => e2b}/pty-e2b/src/index.ts | 0 .../{pty => e2b}/pty-e2b/src/invariant.ts | 0 packages/{pty => e2b}/pty-e2b/src/session.ts | 0 .../{pty => e2b}/pty-e2b/tests/index.spec.ts | 0 .../pty-e2b/tests/session.spec.ts | 0 packages/{pty => e2b}/pty-e2b/tsconfig.json | 4 +- packages/e2b/subprocess-e2b/README.i18n.yaml | 6 ++ .../subprocess-e2b/README.md | 2 +- .../subprocess-e2b/README.zh.md | 2 +- .../subprocess-e2b/package.json | 0 .../subprocess-e2b/src/index.ts | 0 .../subprocess-e2b/src/invariant.ts | 0 .../subprocess-e2b/src/output.ts | 0 .../subprocess-e2b/src/process.ts | 0 .../subprocess-e2b/tests/subprocess.spec.ts | 0 .../subprocess-e2b/tsconfig.json | 4 +- packages/fs/README.i18n.yaml | 4 +- packages/fs/README.md | 2 +- packages/fs/README.zh.md | 2 +- packages/lsp/README.i18n.yaml | 4 +- packages/lsp/README.md | 4 +- packages/lsp/README.zh.md | 4 +- packages/pty/README.i18n.yaml | 4 +- packages/pty/README.md | 4 +- packages/pty/README.zh.md | 4 +- packages/subprocess/README.i18n.yaml | 4 +- packages/subprocess/README.md | 2 +- packages/subprocess/README.zh.md | 2 +- .../verify-package-readme-model-experience.ts | 8 +-- tsconfig.base.json | 1 + tsconfig.host.json | 10 ++-- 81 files changed, 171 insertions(+), 249 deletions(-) delete mode 100644 .agents/notes/implemented/feature/2026-07-28-e2b-interactive-semantic-code-runtime-poc.i18n.yaml delete mode 100644 .agents/notes/implemented/feature/2026-07-28-e2b-interactive-semantic-code-runtime-poc.md delete mode 100644 .agents/notes/implemented/feature/2026-07-28-e2b-interactive-semantic-code-runtime-poc.zh.md delete mode 100644 packages/code-runtime/code-runtime-e2b/README.i18n.yaml rename packages/{subprocess/subprocess-e2b => e2b/code-runtime-e2b}/README.i18n.yaml (54%) rename packages/{code-runtime => e2b}/code-runtime-e2b/README.md (93%) rename packages/{code-runtime => e2b}/code-runtime-e2b/README.zh.md (93%) rename packages/{code-runtime => e2b}/code-runtime-e2b/package.json (100%) rename packages/{code-runtime => e2b}/code-runtime-e2b/src/index.ts (100%) rename packages/{code-runtime => e2b}/code-runtime-e2b/src/invariant.ts (100%) rename packages/{code-runtime => e2b}/code-runtime-e2b/src/runner-source.ts (100%) rename packages/{code-runtime => e2b}/code-runtime-e2b/tests/runtime.spec.ts (100%) rename packages/{code-runtime => e2b}/code-runtime-e2b/tsconfig.json (70%) rename packages/{fs => e2b}/fs-e2b/README.i18n.yaml (56%) rename packages/{fs => e2b}/fs-e2b/README.md (81%) rename packages/{fs => e2b}/fs-e2b/README.zh.md (81%) rename packages/{fs => e2b}/fs-e2b/package.json (100%) rename packages/{fs => e2b}/fs-e2b/src/index.ts (100%) rename packages/{fs => e2b}/fs-e2b/src/invariant.ts (100%) rename packages/{fs => e2b}/fs-e2b/tests/filesystem.spec.ts (100%) rename packages/{fs => e2b}/fs-e2b/tsconfig.json (87%) rename packages/{pty/pty-e2b => e2b/lsp-e2b}/README.i18n.yaml (60%) rename packages/{lsp => e2b}/lsp-e2b/README.md (92%) rename packages/{lsp => e2b}/lsp-e2b/README.zh.md (92%) rename packages/{lsp => e2b}/lsp-e2b/package.json (100%) rename packages/{lsp => e2b}/lsp-e2b/src/index.ts (100%) rename packages/{lsp => e2b}/lsp-e2b/src/invariant.ts (100%) rename packages/{lsp => e2b}/lsp-e2b/src/proxy-source.ts (100%) rename packages/{lsp => e2b}/lsp-e2b/src/transport.ts (100%) rename packages/{lsp => e2b}/lsp-e2b/tests/provider.spec.ts (100%) rename packages/{lsp => e2b}/lsp-e2b/tests/transport.spec.ts (100%) rename packages/{lsp => e2b}/lsp-e2b/tsconfig.json (75%) rename packages/{lsp/lsp-e2b => e2b/pty-e2b}/README.i18n.yaml (60%) rename packages/{pty => e2b}/pty-e2b/README.md (93%) rename packages/{pty => e2b}/pty-e2b/README.zh.md (93%) rename packages/{pty => e2b}/pty-e2b/package.json (100%) rename packages/{pty => e2b}/pty-e2b/src/config.ts (100%) rename packages/{pty => e2b}/pty-e2b/src/index.ts (100%) rename packages/{pty => e2b}/pty-e2b/src/invariant.ts (100%) rename packages/{pty => e2b}/pty-e2b/src/session.ts (100%) rename packages/{pty => e2b}/pty-e2b/tests/index.spec.ts (100%) rename packages/{pty => e2b}/pty-e2b/tests/session.spec.ts (100%) rename packages/{pty => e2b}/pty-e2b/tsconfig.json (85%) create mode 100644 packages/e2b/subprocess-e2b/README.i18n.yaml rename packages/{subprocess => e2b}/subprocess-e2b/README.md (89%) rename packages/{subprocess => e2b}/subprocess-e2b/README.zh.md (89%) rename packages/{subprocess => e2b}/subprocess-e2b/package.json (100%) rename packages/{subprocess => e2b}/subprocess-e2b/src/index.ts (100%) rename packages/{subprocess => e2b}/subprocess-e2b/src/invariant.ts (100%) rename packages/{subprocess => e2b}/subprocess-e2b/src/output.ts (100%) rename packages/{subprocess => e2b}/subprocess-e2b/src/process.ts (100%) rename packages/{subprocess => e2b}/subprocess-e2b/tests/subprocess.spec.ts (100%) rename packages/{subprocess => e2b}/subprocess-e2b/tsconfig.json (84%) diff --git a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.i18n.yaml b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.i18n.yaml index 46f1849b07..5ed4e445b8 100644 --- a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.i18n.yaml +++ b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md -2026-07-27-e2b-remote-runtime-poc.md: 730e6f0e1958ccab43a5a6a11931f9407b38ef16 -2026-07-27-e2b-remote-runtime-poc.zh.md: 5f1ae07b985787a7dc1bd61f2a5b7ff0512d85c1 +2026-07-27-e2b-remote-runtime-poc.md: a713cca3fd9e7ee11a2c487bf6ed2c3a205dc502 +2026-07-27-e2b-remote-runtime-poc.zh.md: 7afd54f72622d51a3d71719b3b4975ccd7f83e3a diff --git a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md index 730e6f0e19..a713cca3fd 100644 --- a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md +++ b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md @@ -6,29 +6,43 @@ English | [中文](2026-07-27-e2b-remote-runtime-poc.zh.md) ## Problem -A remote coding-agent backend is useful only when file operations and commands observe one coherent world. Attaching E2B independently at individual tools would allow a Bash command and a filesystem edit to address different sandboxes, while moving the complete harness into a remote VM would couple provider experimentation to agent, session, model, persistence, and deployment changes. +A remote coding-agent backend is useful only when filesystem operations, one-shot commands, persistent terminals, language servers, and model-written programs observe one coherent world. Attaching E2B independently at individual tools would let those capabilities address different sandboxes, while retaining host PTY, LSP, or worker backends would split state across machines even when the cwd strings match. + +Moving the complete harness into a remote VM would unify that state but also couple provider experimentation to plugin loading, credentials, model transport, agent/session durability, supervision, and deployment. The POC needs to test the existing capability boundaries without taking on those independent concerns. ## Decision -The E2B integration is a provider-composition POC with one shared lifecycle owner and two capability implementations: +The E2B integration is an opt-in provider-composition POC. Its six E2B-specific packages live under `packages/e2b/` while retaining seam-specific npm names: - `@deepseek-ai/dsh-e2b` creates or reconnects one secure E2B sandbox, creates its working and private runtime directories, and owns kill/pause/leave disposal. - `@deepseek-ai/dsh-fs-e2b` implements `ctx.fs` over that sandbox's Filesystem API. - `@deepseek-ai/dsh-subprocess-e2b` implements `ctx.subprocess` over E2B Commands and remote Linux process groups. +- `@deepseek-ai/dsh-pty-e2b` registers an E2B byte-PTY backend on `ctx.pty` while the existing registry retains exact-Agent ownership. +- `@deepseek-ai/dsh-lsp-e2b` registers configured remote language servers on `ctx.lsp`, reads source through E2B Filesystem APIs, and runs servers through `dsh-subprocess-e2b`. +- `@deepseek-ai/dsh-code-runtime-e2b` registers `ctx.codeRuntime`, runs each model program in a fresh remote worker, and dispatches binding functions in the host process. - The existing `@deepseek-ai/dsh-bash-local` remains the Bash implementation because it delegates all process mechanics to `ctx.subprocess`. -- PTY, LSP, and Code Runtime remain separate capability providers over the same owner, with their runtime split recorded in the [extension decision](2026-07-28-e2b-interactive-semantic-code-runtime-poc.md). -The owner is the sole source of sandbox identity. Providers inject it and never create private sandboxes. The composition therefore gives filesystem tools and Bash one remote cwd, process namespace, and spill/state directory while preserving the existing capability interfaces and model-facing tools. +The owner is the sole source of sandbox identity. Providers inject it and never create private sandboxes, so filesystem tools, Bash, interactive shells, language servers, and code workers share one remote cwd, process namespace, and adapter-private directory while preserving the existing capability interfaces and model-facing tools. + +The providers reuse the PTY, LSP, Code Runtime, and subprocess seams without changing their model-facing consumers or the agent loop. Backend-neutral PTY text handling lives in `dsh-pty`; the LSP protocol engine accepts `processId: null` for a server in another process namespace; Code Runtime exports its output-ledger and lossless-JSON helpers for backend parity. ## POC boundary -Filesystem state, managed commands, interactive shells, language servers, code workers, and adapter-owned files move into E2B. The host retains Cordis and plugin objects, the agent loop, agent/session state, session logs and persistence, model requests, skills, subagent orchestration, capability protocol state, and E2B SDK buffers. The overlay does not upload or mount the host workspace; identical cwd strings name independent host and remote directories. Managed process groups still terminate and join when their provider disposes, including before a retained-sandbox pause or leave disposition. +E2B owns the mutable filesystem, command and Bash processes, PTY shell and foreground process groups, language-server processes and source reads, the Code Runtime runner and worker, and adapter-private files under `.dsh-e2b`. -The POC has no session-persistence backend, template builder, volume, snapshot, network-policy layer, sandbox catalog, or workspace synchronization. Retained sandbox reconnect proves lifecycle continuity only; it does not reconstruct host PTY/LSP/code-runtime state, process handles, output cursors, pending calls, or locks. +The host owns Cordis and plugin objects, the agent loop, agent/session/goal state, session logs and persistence, LLM calls, prompts and tools, authority decisions, skills, subagent orchestration, PTY buffers and readiness state, LSP JSON-RPC ids/queues/protocol state, Code Runtime type stripping/output accounting/binding dispatch, and E2B SDK/network orchestration. The overlay does not upload, mount, or synchronize the host workspace; identical cwd strings name independent host and remote directories. + +Byte-sensitive protocols use the narrowest adapter required by E2B's callback shapes. PTY consumes the SDK's byte callback directly. LSP and Code Runtime install dependency-free remote helpers that encode raw payloads as validated newline-delimited base64 JSON, keeping E2B's decoded command callbacks on an ASCII transport. + +Retaining a sandbox preserves remote files and unmanaged remote state only. Reconnect does not reconstruct host PTY sessions, buffers, process handles, LSP connections or requests, code workers, binding calls, timers, output cursors, or locks. Managed groups terminate and join when their provider disposes before the shared owner pauses, leaves, or kills the sandbox. + +The POC has no session-persistence backend, template builder, volume, snapshot, network-policy layer, sandbox catalog, workspace synchronization, durable remote handles, or whole-harness execution. ## Verification -Package tests pin lifecycle cleanup, filesystem semantics, subprocess groups, byte framing, PTY readiness/signals, LSP transport and containment, Code Runtime binding/output behavior, and package-owned invariant registrations. A credential-gated real Loader composition creates one sandbox, exercises filesystem, Bash/subprocess, PTY, LSP, and Code Runtime through source and built package paths, proves host-workspace isolation, disposes the composition, and confirms the sandbox id is gone. +Focused package suites pin owner lifecycle cleanup, filesystem semantics, subprocess process groups, configuration and publication rollback, byte framing and multibyte boundaries, PTY readiness/signals, LSP transport and source containment, Code Runtime bindings, hostile traffic, output limits, timeout/abort ordering, disposal to quiescence, and package-owned invariant registrations. Adjacent local-backend suites pin the shared PTY utilities and the LSP cross-namespace `processId` behavior. + +A credential-gated Loader composition creates one real E2B sandbox and exercises FS-to-Bash and Bash-to-FS visibility, multibyte PTY output and `SIGINT`, multibyte LSP hover and definition results, Code Runtime host bindings and typed rejection under mutation of adapter-captured intrinsics, wall timeout, abort, runner cleanup, host-workspace isolation, and final sandbox deletion. The same scenario runs through source imports and built package exports. ## Alternatives considered @@ -40,8 +54,16 @@ Package tests pin lifecycle cleanup, filesystem semantics, subprocess groups, by **Implement filesystem operations through shell commands only** — rejected because that bypasses `ctx.fs` identity, structured errors, version guards, streaming reads, and atomic mutation semantics already consumed by the file tools. +**Use the host PTY, LSP, and worker backends unchanged** — rejected because they use host process and filesystem APIs; sharing an absolute cwd string does not share state across machines. + +**Expose E2B Commands as one generic transport and bypass capability providers** — rejected because PTY needs byte callbacks and foreground signaling, LSP needs byte-faithful stdio plus remote source containment, and Code Runtime needs bidirectional host binding calls and hostile-peer validation. Bypassing their registries would also fork model-facing behavior. + +**Add a generic distributed-runtime abstraction first** — rejected because the existing capability seams already carry the required contracts. A new cross-cutting interface would speculate about persistence, synchronization, and reconnect semantics beyond the POC. + +**Restore live capability handles after `sandboxId` reconnect** — rejected because remote identity alone cannot reconstruct host callbacks, pending promises, authority, protocol state, or output cursors. Claiming continuity would make stale remote processes appear managed when they are not. + ## Consequences -The small composition demonstrates that existing capability seams can move an agent's mutable coding world off-host without changing the loop or model-facing tool packages. `sandboxId` plus pause/leave permits manual state retention for experiments, while kill remains the demo's cleanup policy. +The small composition demonstrates that existing capability seams can move an agent's mutable coding world off-host without changing the loop or model-facing tool packages. `sandboxId` plus pause/leave permits manual remote-file retention for experiments, while kill remains the demo's cleanup policy. -The provider is not interchangeable with local backends for every consumer: remote startup cannot synchronously expose a PID, E2B retains complete command output in SDK memory, ordinary command callbacks are not byte-faithful, signal attribution is partly inferred, and reconnect cannot restore handles or protocol state. PTY uses E2B's byte API; LSP and Code Runtime add validated ASCII framing where protocol bytes matter. Remote process/spill artifacts accumulate in a retained sandbox. These gaps remain documented POC constraints rather than compatibility shims or new cross-cutting abstractions. +The providers are not interchangeable with local backends for every consumer: remote startup cannot synchronously expose a PID, E2B retains complete command output in SDK memory, ordinary command callbacks are not byte-faithful, signal attribution is partly inferred, and reconnect cannot restore handles or protocol state. PTY uses E2B's byte API; LSP and Code Runtime add validated ASCII framing where protocol bytes matter. Remote process/spill artifacts accumulate in a retained sandbox, Code programs share a JavaScript realm with Node worker internals, and a process that deliberately escapes a captured remote process group does not become reconnectable or owned. These gaps remain documented POC constraints rather than compatibility shims or new cross-cutting abstractions. diff --git a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.zh.md b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.zh.md index 5f1ae07b98..7afd54f726 100644 --- a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.zh.md +++ b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.zh.md @@ -6,29 +6,43 @@ Status: implemented ## 问题 -远程 coding agent(智能体)后端只有在文件操作与命令观察到同一个一致环境时才有用。若在各工具上分别接入 E2B,Bash 命令和文件系统编辑可能访问不同的沙箱;若把完整 harness 迁入远程 VM,则会把提供方实验与 agent、会话、模型、持久化及部署变更耦合在一起。 +远程 coding agent(智能体)后端只有在文件系统操作、一次性命令、持久终端、语言服务器和模型编写的程序观察到同一个一致环境时才有用。若在各工具上分别接入 E2B,这些功能可能访问不同的沙箱;即使 cwd 字符串相同,保留宿主 PTY、LSP 或 worker 后端也会让状态分散在不同机器上。 + +把完整 harness 迁入远程 VM 可以统一这些状态,但也会把提供方实验与插件加载、凭据、模型传输、agent/会话持久性、监管和部署耦合在一起。这个 POC 只需测试现有功能边界,不应把这些彼此独立的问题纳入范围。 ## 决策 -E2B 集成是一个提供方组合 POC,由一个共享生命周期所有者和两个功能实现组成: +E2B 集成是一个选择性启用的提供方组合 POC。它的 6 个 E2B 专用包(package)位于 `packages/e2b/` 下,同时保留按 seam 区分的 npm 名称: - `@deepseek-ai/dsh-e2b` 创建或重新连接一个安全的 E2B 沙箱,创建其工作目录与私有运行时目录,并拥有 kill/pause/leave 资源释放操作。 - `@deepseek-ai/dsh-fs-e2b` 在该沙箱的 Filesystem API 之上实现 `ctx.fs`。 - `@deepseek-ai/dsh-subprocess-e2b` 在 E2B Commands 和远程 Linux 进程组之上实现 `ctx.subprocess`。 +- `@deepseek-ai/dsh-pty-e2b` 在 `ctx.pty` 上注册 E2B 字节 PTY 后端,并把精确的 Agent 所有权保留在现有注册表中。 +- `@deepseek-ai/dsh-lsp-e2b` 在 `ctx.lsp` 上注册已配置的远程语言服务器,通过 E2B Filesystem API 读取源代码,并通过 `dsh-subprocess-e2b` 运行服务器。 +- `@deepseek-ai/dsh-code-runtime-e2b` 注册 `ctx.codeRuntime`,在全新的远程 worker 中运行每个模型程序,并在宿主进程中分发绑定函数。 - 现有的 `@deepseek-ai/dsh-bash-local` 继续作为 Bash 实现,因为它把所有进程机制委托给 `ctx.subprocess`。 -- PTY、LSP 与 Code Runtime 仍是基于同一所有者的独立功能提供方;其运行时划分见[扩展决策](2026-07-28-e2b-interactive-semantic-code-runtime-poc.md)。 -该所有者是沙箱身份的唯一真源。提供方会注入该所有者,绝不创建私有沙箱。因此,该组合让文件系统工具与 Bash 共享一个远程 cwd、进程命名空间和 spill/状态目录,同时保留现有功能接口与面向模型的工具。 +该所有者是沙箱身份的唯一真源。提供方会注入该所有者,绝不创建私有沙箱,因此文件系统工具、Bash、交互式 shell、语言服务器和代码 worker 会共享一个远程 cwd、进程命名空间和适配器私有目录,同时保留现有功能接口与面向模型的工具。 + +这些提供方复用 PTY、LSP、Code Runtime 与进程管理 seam,不更改面向模型的消费方或 agent loop(智能体循环)。后端无关的 PTY 文本处理位于 `dsh-pty`;LSP 协议引擎允许位于另一个进程命名空间的服务器使用 `processId: null`;Code Runtime 导出输出账本与无损 JSON 辅助函数,以保持各后端一致。 ## POC 边界 -文件系统状态、受管命令、交互式 shell、语言服务器、代码 worker 和适配器拥有的文件会迁入 E2B。宿主仍保留 Cordis 和插件对象、agent loop(智能体循环)、agent/会话状态、会话日志及其持久化、模型请求、skill(技能)、subagent 编排、功能协议状态和 E2B SDK 缓冲。该 overlay 不会上传或挂载宿主工作区;拼写相同的 cwd 字符串分别指向彼此独立的宿主与远程目录。受管进程组仍会在所属提供方 dispose(资源释放)时终止并等待退出,包括保留沙箱采用 `pause` 或 `leave` 处置方式之前。 +E2B 拥有可变文件系统、命令和 Bash 进程、PTY shell 与前台进程组、语言服务器进程及源码读取、Code Runtime 运行器和 worker,以及 `.dsh-e2b` 下的适配器私有文件。 -本 POC 没有会话持久化后端、模板构建器、卷、快照、网络策略层、沙箱目录或工作区同步。保留沙箱后重新连接只能证明生命周期连续性;它不会重建宿主 PTY/LSP/Code Runtime 状态、进程句柄、输出游标、待处理调用或锁。 +宿主拥有 Cordis 与插件对象、agent loop、agent/会话/goal 状态、会话日志及持久化、LLM(大语言模型)调用、提示词与工具、权限决策、skill(技能)、subagent 编排、PTY 缓冲与就绪状态、LSP JSON-RPC id/队列/协议状态、Code Runtime 类型剥离/输出计量/绑定分发,以及 E2B SDK/网络编排。该 overlay 不会上传、挂载或同步宿主工作区;拼写相同的 cwd 字符串分别指向彼此独立的宿主与远程目录。 + +对字节敏感的协议只使用适配 E2B 回调形状所需的最窄适配器。PTY 直接消费 SDK 的字节回调。LSP 与 Code Runtime 会安装无依赖的远程辅助程序,把原始载荷编码为经过验证、以换行分隔的 base64 JSON,并通过 ASCII 传输承载 E2B 已解码的命令回调。 + +保留沙箱只会保存远程文件与未受管的远程状态。重新连接不会重建宿主 PTY 会话、缓冲、进程句柄、LSP 连接或请求、代码 worker、绑定调用、定时器、输出游标或锁。受管进程组会在所属提供方 dispose(资源释放)时终止并等待退出,之后共享所有者才会暂停、脱离或终止沙箱。 + +本 POC 没有会话持久化后端、模板构建器、卷、快照、网络策略层、沙箱目录、工作区同步、持久远程句柄或完整 harness 执行。 ## 验证 -包测试固定生命周期清理、文件系统语义、进程管理的进程组、字节分帧、PTY 就绪状态/信号、LSP 传输与源码路径约束、Code Runtime 绑定/输出行为,以及包自有不变式注册。凭据门控的真实 Loader 组合会创建一个沙箱,通过源代码与已构建包路径演练文件系统、Bash/进程管理、PTY、LSP 和 Code Runtime,证明宿主工作区隔离,释放组合,并确认该沙箱 id 已不存在。 +聚焦包测试套件固定所有者生命周期清理、文件系统语义、进程管理的进程组、配置与发布回滚、字节分帧与多字节边界、PTY 就绪状态/信号、LSP 传输与源码路径约束、Code Runtime 绑定、恶意通信、输出上限、超时/中止顺序、等待完全停稳的资源释放,以及包自有不变式注册。相邻本地后端测试套件固定共享 PTY 工具函数,以及 LSP 跨命名空间 `processId` 行为。 + +凭据门控的 Loader 组合会创建一个真实 E2B 沙箱,并演练 FS-to-Bash 与 Bash-to-FS 可见性、多字节 PTY 输出和 `SIGINT`、多字节 LSP 悬停与定义结果、Code Runtime 宿主绑定,以及适配器已捕获 intrinsic 被修改时的类型化 reject、墙钟超时、中止、运行器清理、宿主工作区隔离,以及最终删除沙箱。同一场景分别通过源代码导入与已构建包导出运行。 ## 曾考虑的替代方案 @@ -40,8 +54,16 @@ E2B 集成是一个提供方组合 POC,由一个共享生命周期所有者和 **仅通过 shell 命令实现文件系统操作。** 不予采纳,因为这会绕过文件工具已经使用的 `ctx.fs` 身份、结构化错误、版本防护、流式读取和原子变更语义。 +**原样使用宿主 PTY、LSP 与 worker 后端。** 不予采纳,因为它们使用宿主的进程与文件系统 API;在不同机器上复用同一个绝对 cwd 字符串并不会共享状态。 + +**把 E2B Commands 公开为通用传输并绕过功能提供方。** 不予采纳,因为 PTY 需要字节回调和前台信号,LSP 需要字节保真的 stdio 与远程源码路径约束,Code Runtime 则需要双向宿主绑定调用与不可信对等方验证。绕过其注册表还会使面向模型的行为产生分叉。 + +**先添加通用分布式运行时抽象。** 不予采纳,因为现有功能 seam 已承载所需契约。新的跨领域接口会预先假定 POC 范围之外的持久化、同步与重连语义。 + +**在 `sandboxId` 重连后恢复活动功能句柄。** 不予采纳,因为只有远程身份,无法重建宿主回调、待处理 promise、权限、协议状态或输出游标。若声称保持连续性,就会让陈旧的远程进程看似仍受管理,实际并非如此。 + ## 后果 -这个小型组合证明,现有功能 seam 可以把 agent 的可变 coding 环境移出宿主,而无需改变循环或面向模型的工具包。`sandboxId` 与 `pause`/`leave` 允许实验手动保留状态,演示仍以 `kill` 作为清理策略。 +这个小型组合证明,现有功能 seam 可以把 agent 的可变 coding 环境移出宿主,而无需改变循环或面向模型的工具包。`sandboxId` 与 `pause`/`leave` 允许实验手动保留远程文件,演示仍以 `kill` 作为清理策略。 -该提供方并不能对所有消费方与本地后端互换:远程启动无法同步公开 PID,E2B 会在 SDK 内存中保留完整命令输出,普通命令回调并非字节保真,信号归因部分依靠推断,重新连接也无法恢复句柄或协议状态。PTY 使用 E2B 的字节 API;LSP 与 Code Runtime 则在必须保真处理协议字节之处增加经过验证的 ASCII 分帧。保留沙箱后会累积远程进程/spill 产物。这些缺口作为 POC 约束明确记录,而不会引入兼容垫片或新的跨领域抽象。 +这些提供方并不能对所有消费方与本地后端互换:远程启动无法同步公开 PID,E2B 会在 SDK 内存中保留完整命令输出,普通命令回调并非字节保真,信号归因部分依靠推断,重新连接也无法恢复句柄或协议状态。PTY 使用 E2B 的字节 API;LSP 与 Code Runtime 则在必须保真处理协议字节之处增加经过验证的 ASCII 分帧。保留沙箱后会累积远程进程/spill 产物,模型程序与 Node worker 内部机制共享一个 JavaScript realm,有意逃离已捕获远程进程组的进程也不会因此变得可重新连接或由该组合管理。这些缺口作为 POC 约束明确记录,而不会引入兼容垫片或新的跨领域抽象。 diff --git a/.agents/notes/implemented/feature/2026-07-28-e2b-interactive-semantic-code-runtime-poc.i18n.yaml b/.agents/notes/implemented/feature/2026-07-28-e2b-interactive-semantic-code-runtime-poc.i18n.yaml deleted file mode 100644 index e34f939a7f..0000000000 --- a/.agents/notes/implemented/feature/2026-07-28-e2b-interactive-semantic-code-runtime-poc.i18n.yaml +++ /dev/null @@ -1,6 +0,0 @@ -# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each -# side as of the last confirmed-consistent state. Both languages carry equal authority; -# after editing either side, bring the other along and re-record with: -# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-28-e2b-interactive-semantic-code-runtime-poc.md -2026-07-28-e2b-interactive-semantic-code-runtime-poc.md: befaa47dd56b7f425e72978ae040bd0815cc9f92 -2026-07-28-e2b-interactive-semantic-code-runtime-poc.zh.md: 677c3a09f6846320d3e000e9607becb3dee865cf diff --git a/.agents/notes/implemented/feature/2026-07-28-e2b-interactive-semantic-code-runtime-poc.md b/.agents/notes/implemented/feature/2026-07-28-e2b-interactive-semantic-code-runtime-poc.md deleted file mode 100644 index befaa47dd5..0000000000 --- a/.agents/notes/implemented/feature/2026-07-28-e2b-interactive-semantic-code-runtime-poc.md +++ /dev/null @@ -1,57 +0,0 @@ -# Agent Note: E2B interactive, semantic, and code-runtime POC - -Status: implemented - -English | [中文](2026-07-28-e2b-interactive-semantic-code-runtime-poc.zh.md) - -## Problem - -The [shared E2B runtime](2026-07-27-e2b-remote-runtime-poc.md) proves that filesystem operations and one-shot commands can inhabit one remote coding world, but an assembled coding agent also uses persistent terminals, language servers, and model-written Code Mode programs. Falling back to host implementations for those capabilities splits observable state: a Bash edit exists in E2B while a host PTY, LSP process, or code worker addresses a different filesystem and process namespace. - -Moving the complete harness process into E2B would unify that state but also changes plugin loading, credentials, model transport, session durability, supervision, and deployment. The POC needs to test the existing capability boundaries without taking on those independent concerns. - -## Decision - -Three opt-in providers extend the existing shared sandbox: - -- `@deepseek-ai/dsh-pty-e2b` registers an E2B byte-PTY backend on `ctx.pty` and keeps exact-Agent ownership in the existing registry. -- `@deepseek-ai/dsh-lsp-e2b` registers configured remote language servers on `ctx.lsp`, reads source through E2B Filesystem APIs, and runs servers through `dsh-subprocess-e2b`. -- `@deepseek-ai/dsh-code-runtime-e2b` registers `ctx.codeRuntime`, runs each model program in a fresh remote worker, and dispatches binding functions in the host process. - -All three inject `ctx.e2b`; none creates another sandbox. The opt-in overlay composes them with `dsh-fs-e2b`, `dsh-subprocess-e2b`, and the existing `dsh-bash-local`, so files, foreground commands, interactive shell processes, language servers, and code workers observe one remote cwd. - -The providers reuse the PTY, LSP, Code Runtime, and subprocess seams without changing their model-facing consumers or the agent loop. Backend-neutral PTY text handling moves into `dsh-pty`; the LSP protocol engine accepts `processId: null` for a server in another process namespace; Code Runtime exports its output-ledger and lossless-JSON helpers for backend parity. - -## Runtime boundary - -E2B owns the mutable filesystem, command and Bash processes, PTY shell and foreground process groups, language-server processes and source reads, the Code Runtime runner and worker, and adapter-private files under `.dsh-e2b`. - -The host owns Cordis and plugin objects, agent/session/goal state, session logs and persistence, LLM calls, prompts and tools, authority decisions, PTY buffers and readiness state, LSP JSON-RPC ids/queues/protocol state, Code Runtime type stripping/output accounting/binding dispatch, and E2B SDK/network orchestration. The host workspace is not mounted or synchronized merely because its absolute cwd string is reused remotely. - -Byte-sensitive protocols use the narrowest adapter needed for E2B's callback shapes. PTY consumes the SDK's byte callback directly. LSP and Code Runtime install dependency-free remote helpers that encode raw payloads as validated newline-delimited base64 JSON, keeping E2B's decoded command callbacks on an ASCII transport. - -Retaining a sandbox preserves remote files and any unmanaged remote state only. Reconnect does not reconstruct host PTY sessions, buffers, process handles, LSP connections or requests, code workers, binding calls, timers, output cursors, or locks. Managed groups are terminated and awaited when their provider disposes before the shared owner pauses, leaves, or kills the sandbox. - -## Verification - -Focused unit suites pin configuration, publication rollback, byte framing, multibyte boundaries, readiness, signals, timeout/abort ordering, output limits, hostile Code Runtime traffic, and disposal to quiescence. Adjacent local-backend suites pin the shared PTY utilities and the LSP cross-namespace `processId` behavior. - -A credential-gated Loader composition creates one real E2B sandbox and exercises FS-to-Bash and Bash-to-FS visibility, multibyte PTY output and `SIGINT`, multibyte LSP hover and definition results, Code Runtime host bindings and typed rejection under mutation of adapter-captured intrinsics, wall timeout, abort, runner cleanup, host-workspace isolation, and final sandbox deletion. The same scenario runs through source imports and built package exports. - -## Alternatives considered - -**Run the complete harness inside E2B** — rejected because it couples this provider experiment to credentials, LLM transport, plugin deployment, session persistence, supervision, and remote package installation. None is necessary to prove the capability seams. - -**Use the host PTY, LSP, and worker backends unchanged** — rejected because they use host process and filesystem APIs; sharing an absolute cwd string does not share state across machines. - -**Expose E2B Commands as one generic transport and bypass capability providers** — rejected because PTY needs byte callbacks and foreground signaling, LSP needs byte-faithful stdio plus remote source containment, and Code Runtime needs bidirectional host binding calls and hostile-peer validation. Bypassing their registries would also fork model-facing behavior. - -**Add a generic distributed-runtime abstraction first** — rejected because the three existing capability seams already carry the required contracts. A new cross-cutting interface would speculate about persistence, synchronization, and reconnect semantics beyond the POC. - -**Restore live capability handles after `sandboxId` reconnect** — rejected because remote identity alone cannot reconstruct host callbacks, pending promises, authority, protocol state, or output cursors. Claiming continuity would make stale remote processes appear managed when they are not. - -## Consequences - -The assembled POC keeps the coding world remote without moving the agent runtime or changing model-visible tool contracts. It demonstrates that PTY, LSP, and Code Runtime can share E2B state through existing plugins, while making the remaining host state explicit. - -This is not a deployment platform. Language-server installation, templates, volumes, snapshots, network policy, workspace synchronization, durable remote handles, and whole-harness execution remain outside scope. E2B SDK buffering and host protocol state remain memory costs. Code programs share a JavaScript realm with Node worker internals, and a process that deliberately escapes a captured remote process group is not made reconnectable or owned by this composition. diff --git a/.agents/notes/implemented/feature/2026-07-28-e2b-interactive-semantic-code-runtime-poc.zh.md b/.agents/notes/implemented/feature/2026-07-28-e2b-interactive-semantic-code-runtime-poc.zh.md deleted file mode 100644 index 677c3a09f6..0000000000 --- a/.agents/notes/implemented/feature/2026-07-28-e2b-interactive-semantic-code-runtime-poc.zh.md +++ /dev/null @@ -1,57 +0,0 @@ -# Agent Note: E2B 交互式、语义与代码运行时 POC - -Status: implemented - -[English](2026-07-28-e2b-interactive-semantic-code-runtime-poc.md) | 中文 - -## 问题 - -[共享 E2B 运行时](2026-07-27-e2b-remote-runtime-poc.md)证明,文件系统操作与一次性命令可以共处一个远程 coding 环境,但组装完成的 coding agent(智能体)还会使用持久终端、语言服务器,以及模型编写的 Code Mode 程序。若这些功能回退到宿主实现,可观测状态就会分裂:Bash 修改存在于 E2B 中,而宿主 PTY、LSP 进程或代码 worker 面向的却是另一个文件系统与进程命名空间。 - -把完整 harness 进程迁入 E2B 可以统一这些状态,但也会改变插件加载、凭据、模型传输、会话持久性、监管和部署。这个 POC 只需测试现有功能边界,不应把这些彼此独立的问题纳入范围。 - -## 决策 - -三个可选提供方扩展现有共享沙箱: - -- `@deepseek-ai/dsh-pty-e2b` 在 `ctx.pty` 上注册 E2B 字节 PTY 后端,并把精确的 Agent 所有权保留在现有注册表中。 -- `@deepseek-ai/dsh-lsp-e2b` 在 `ctx.lsp` 上注册已配置的远程语言服务器,通过 E2B Filesystem API 读取源代码,并通过 `dsh-subprocess-e2b` 运行服务器。 -- `@deepseek-ai/dsh-code-runtime-e2b` 注册 `ctx.codeRuntime`,在全新的远程 worker 中运行每个模型程序,并在宿主进程中分发绑定函数。 - -三个提供方均注入 `ctx.e2b`,无一创建其他沙箱。可选叠加配置将它们与 `dsh-fs-e2b`、`dsh-subprocess-e2b` 和现有的 `dsh-bash-local` 组合,使文件、前台命令、交互式 shell 进程、语言服务器和代码 worker 观察到同一个远程 cwd。 - -这些提供方复用 PTY、LSP、Code Runtime 与进程管理 seam,不更改面向模型的消费方或 agent loop(智能体循环)。后端无关的 PTY 文本处理移入 `dsh-pty`;LSP 协议引擎允许位于另一个进程命名空间的服务器使用 `processId: null`;Code Runtime 导出输出账本与无损 JSON 辅助函数,以保持各后端一致。 - -## 运行时边界 - -E2B 拥有可变文件系统、命令和 Bash 进程、PTY shell 与前台进程组、语言服务器进程及源码读取、Code Runtime 运行器和 worker,以及 `.dsh-e2b` 下的适配器私有文件。 - -宿主拥有 Cordis 与插件对象、agent/会话/goal 状态、会话日志及持久化、LLM(大语言模型)调用、提示词与工具、权限决策、PTY 缓冲与就绪状态、LSP JSON-RPC id/队列/协议状态、Code Runtime 类型剥离/输出计量/绑定分发,以及 E2B SDK/网络编排。宿主工作区不会仅因远程复用了其绝对 cwd 字符串就被挂载或同步。 - -对字节敏感的协议只使用适配 E2B 回调形状所需的最窄适配器。PTY 直接消费 SDK 的字节回调。LSP 与 Code Runtime 会安装无依赖的远程辅助程序,把原始载荷编码为经过验证、以换行分隔的 base64 JSON,并通过 ASCII 传输承载 E2B 已解码的命令回调。 - -保留沙箱只会保存远程文件与任何未受管的远程状态。重新连接不会重建宿主 PTY 会话、缓冲、进程句柄、LSP 连接或请求、代码 worker、绑定调用、定时器、输出游标或锁。受管进程组会在所属提供方 dispose(资源释放)时终止并等待退出,之后共享所有者才会暂停、脱离或终止沙箱。 - -## 验证 - -聚焦单元测试固定配置、发布回滚、字节分帧、多字节边界、就绪状态、信号、超时/中止顺序、输出上限、恶意 Code Runtime 通信,以及等待完全停稳的资源释放。相邻本地后端测试固定共享 PTY 工具函数,以及 LSP 跨命名空间 `processId` 行为。 - -凭据门控的 Loader 组合会创建一个真实 E2B 沙箱,并演练 FS-to-Bash 与 Bash-to-FS 可见性、多字节 PTY 输出和 `SIGINT`、多字节 LSP 悬停与定义结果、Code Runtime 宿主绑定,以及适配器已捕获 intrinsic 被修改时的类型化 reject、墙钟超时、中止、运行器清理、宿主工作区隔离,以及最终删除沙箱。同一场景分别通过源代码导入与已构建包导出运行。 - -## 曾考虑的替代方案 - -**在 E2B 内运行完整 harness。** 不予采纳,因为这会把这项提供方实验与凭据、LLM 传输、插件部署、会话持久化、监管和远程包安装耦合。要证明功能 seam,无需引入任何一项。 - -**原样使用宿主 PTY、LSP 与 worker 后端。** 不予采纳,因为它们使用宿主的进程与文件系统 API;在不同机器上复用同一个绝对 cwd 字符串并不会共享状态。 - -**把 E2B Commands 公开为通用传输并绕过功能提供方。** 不予采纳,因为 PTY 需要字节回调和前台信号,LSP 需要字节保真的 stdio 与远程源码路径约束,Code Runtime 则需要双向宿主绑定调用与不可信对等方验证。绕过其注册表还会使面向模型的行为产生分叉。 - -**先添加通用分布式运行时抽象。** 不予采纳,因为现有三个功能 seam 已承载所需契约。新的跨领域接口会预先假定 POC 范围之外的持久化、同步与重连语义。 - -**在 `sandboxId` 重连后恢复活动功能句柄。** 不予采纳,因为只有远程身份,无法重建宿主回调、待处理 promise、权限、协议状态或输出游标。若声称保持连续性,就会让陈旧的远程进程看似仍受管理,实际并非如此。 - -## 后果 - -组装后的 POC 在不迁移 agent 运行时、不改变模型可见工具契约的前提下,把 coding 环境保留在远程。它证明 PTY、LSP 与 Code Runtime 可以通过现有插件共享 E2B 状态,同时明确列出仍留在宿主的状态。 - -这不是部署平台。语言服务器安装、模板、卷、快照、网络策略、工作区同步、持久远程句柄和完整 harness 执行仍不在范围内。E2B SDK 缓冲与宿主协议状态仍会占用内存。模型程序与 Node worker 内部机制共享一个 JavaScript realm;有意逃离已捕获远程进程组的进程,也不会因此变得可重新连接或由该组合管理。 diff --git a/AGENTS.md b/AGENTS.md index 64f2e944a8..c663a5e099 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -12,10 +12,9 @@ DeepSeek Harness SDK is a plugin-based agent harness on vendored Cordis: **every vendor/ Vendored Cordis source — manifest + sync procedure in vendor/README.md packages/ @deepseek-ai/dsh- workspaces at packages/// core/ product API spine: session, system-prompt, tools, agent, agent-loop - api/ Remote BFF assembly and TypeRT RPC gateway - typert/ type graph generator, loader, and runtime registry llm/ LLM seam + DeepSeek adapters (direct-fetch + pi-ai design twin) - bash/ bash executor seam + local/pwsh impls + model-facing shell tools + e2b/ remote-runtime POC + bash/ bash executor seam + local impl + model-facing bash tools subprocess/ subprocess seam + local process-tree impl pty/ persistent PTY seam/backend/tools fs/ filesystem seam + local impl + policy gate + read/write/edit tools @@ -25,7 +24,6 @@ packages/ @deepseek-ai/dsh- workspaces at packages/// compact/ compaction seam + basic backend context/ request-context plugins subagent/ subagent seam + spawn/fork/ACP backends + delegation tool - bundle/ profile plugin bundles: installable patch layers for dsh --profile workflow/ workflow seam + worker-thread engine + workflow tool todo/ todo_write tool plan/ plan mode as logged per-agent collaboration state @@ -33,11 +31,9 @@ packages/ @deepseek-ai/dsh- workspaces at packages/// cordis/ self-referential toolset: the agent inspects/mounts plugins in its own runtime hooks/ Claude Code/Codex hook bridges + shared wire-protocol library session-persistence/ persistence seam + JSONL/SQLite backends - settings/ user-settings seam + file-backed provider - credentials/ credential-reference seam + env-over-.env provider acp/ automation-only Agent Client Protocol server - ui/ JSON-RPC bridge; boot, approval, interaction plugins - examples/ demo bundles (agent-spine + CLI/ACP/JSON-RPC bins) leaves load + ui/ TUI/JSON-RPC bridges; boot, approval, interaction plugins + examples/ demo bundles (agent-spine + TUI/CLI/ACP/JSON-RPC bins) leaves load support/ dev/test infrastructure util/ zero-dependency utilities python/ Python SDK and bundled runtime (see python/README.md) @@ -59,7 +55,7 @@ pnpm run clean # remove build outputs and safe residue from deleted pa pnpm run test # vitest unit tests pnpm run test:coverage # CI coverage gate: per-file 100% on packages/*/*/src pnpm run test:e2e # real-API tests; self-skip without DEEPSEEK_API_KEY -pnpm run test:snapshot # keyless ACP/headless replay vs expected outputs; filter: -t +pnpm run test:snapshot # keyless ACP/headless/TUI replay vs expected outputs; filter: -t pnpm run test:snapshot:record # re-record expected outputs (needs key) pnpm run typecheck pnpm run lint @@ -70,6 +66,7 @@ pnpm run hygiene # knip + publint + workspace constraints + NodeNext cons pnpm run doc-sync # all documentation gates; leaf list in scripts/run-gates.ts pnpm run website:build # VitePress build (doubles as dead-link check) pnpm run demo:headless "task" # one-shot agent (needs DEEPSEEK_API_KEY) +pnpm run demo:tui # full-screen TUI coding agent (needs DEEPSEEK_API_KEY) pnpm run demo:cordis # the agent modifies its own runtime (needs key) pnpm run demo:acp # ACP automation server (needs DEEPSEEK_API_KEY) ``` @@ -80,7 +77,7 @@ When required `gh`, `pnpm`, build, test, or generator commands fail because the ### Run relevant checks locally -Run checks before pushes via [dsh-pre-push-checks](.agents/skills/dsh-pre-push-checks/SKILL.md); report only commands run. After `gh stack sync`, validate immediately; do not merge before checks pass. +Agents MUST run relevant tests and checks before pushing; select them with [dsh-pre-push-checks](.agents/skills/dsh-pre-push-checks/SKILL.md) and report only commands run. - Match evidence to the surface: focused tests for behavior, snapshots for model or user output, `doc-sync` for docs, build/hygiene and built smokes for published paths, and real-API e2e for provider behavior. - Never default to the full suite or repeat a passing check for commit or push. CI owns exhaustive coverage and the platform matrix; rehearse all locally only by explicit request, for CI diagnosis, or for an irreducibly repository-wide change. @@ -93,7 +90,7 @@ Real-API tests and demos read `DEEPSEEK_API_KEY`, optional `DEEPSEEK_BASE_URL`, ## Conventions - Every npm package is `@deepseek-ai/dsh-`; vendored packages keep upstream names and are `private: true`. `cordis` is a peerDependency (+ dev) of every harness package. -- ESM everywhere (`"type": "module"`). Cross-package imports use package names; in-package relative imports include `.ts`. Config subprocesses run built `lib/` under plain Node; source regressions use their declared launcher ([testing policy](docs/testing.md#test-subprocess-launch-modes)). The `dsh` CLI source launch runs through tsx's ESM-only hook (`node --import tsx/esm`); modules it reaches must stay ESM (no CJS-only shapes) — Node's native TypeScript modes are unavailable across the engines range ([source-launch contract](.agents/notes/implemented/architecture/2026-07-29-dsh-source-launch-tsx-esm.md)). Raw/Web `cordis.yml` bare plugins must appear in their resolver manifest's `dependencies`; `verify-cordis-config` enforces it. +- ESM everywhere (`"type": "module"`). Cross-package imports use package names; in-package relative imports include `.ts`. CI subprocesses that boot examples or Cordis configs run built `lib/` under plain Node; only explicit source-path regressions use tsx ([testing policy](docs/testing.md#test-subprocess-launch-modes)). - **Registrations are effects**: every contribution goes through `ctx.effect()` / `ctx.on()`; a registry's `register()` returns the disposer. - **Runtime invariants assert owned relationships.** Check authoritative event streams or mutable data, not service or method presence, plugin metadata or effects, or fixed pure examples. If a package has no plausible relationship, an explained empty companion is correct ([package contract](packages/AGENTS.md)). - **Typed events use declaration merging** and merge-extensible maps. Event JSDoc needs `@mode` and payload `@param`; scoped keys absent from payloads need `@dshScopeScan unsupported`. Public service methods document parameters and non-void returns. @@ -117,7 +114,7 @@ Real-API tests and demos read `DEEPSEEK_API_KEY`, optional `DEEPSEEK_BASE_URL`, - **Testing policy** — [docs/testing.md](docs/testing.md). Every non-trivial model- or product-user-visible behavior change adds or updates a keyless snapshot through a real runnable example in the same PR; package tests, e2e-only assertions, and mock-only fixtures do not substitute for the assembled application transcript. Fixtures must replay on macOS/Linux; fix fixtures, not normalizers. - **A tool's UI render intent is part of its design**, decided up front (`generic`/`terminal`/`diff`, `locations`); presentation methods are pure functions of `args` ([cookbook](docs/cookbook/adding-a-tool.md)). - **Plan unit, e2e, and snapshot coverage** for new seams, lifecycle shapes, and transcript surfaces; missing snapshot-harness support is part of the implementation, not deferred follow-up. -- **Choose PR history deliberately.** Split independent changes; fix the introducing PR before propagation. Standalone PRs and official stacks may merge-forward or rebase after review. Rewrites use `--force-with-lease`, abort on remote movement, never raw `--force`; an in-progress merge-forward preserves its checkpoint before taking a newer base ([rationale](.agents/notes/implemented/process/2026-08-02-native-github-stacks-and-optional-rebases.md)). +- **Use incremental merge commits.** Split independent changes; never squash, rebase, or rewrite pushed history. Fix the introducing PR before merging down-stack. If the base advances mid-merge, never restart: finish the checkpoint, push when authorized, then merge the newer tip separately ([rationale](.agents/notes/implemented/process/2026-07-26-incremental-pr-base-retargeting.md)). - **Label PRs:** one kind (`feature`/`bug-fix`/`doc`/`testing`/`cleanup`), each matching area; the [taxonomy](.agents/notes/implemented/process/2026-07-25-semantic-pr-label-taxonomy.md) is extensible. - TODO markers: `FIXME`/`TODO`/`XXX` by urgency ([semantics](docs/development.md)). - Files end with exactly one trailing newline; `git diff --cached --check` (pre-commit) gates it. diff --git a/packages/README.i18n.yaml b/packages/README.i18n.yaml index 3eee08f35b..b8320d8c19 100644 --- a/packages/README.i18n.yaml +++ b/packages/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/README.md -README.md: 1b2a1737bcac33829a65fa937668ca9496035330 -README.zh.md: 2ef61e9c38750f4420ce99abc9943af1f3308b56 +README.md: c46a08f76f9f38f9620c03f05f28e7698487afc8 +README.zh.md: 65a5c6c88b9b37b3f948b11c6db778e42965643d diff --git a/packages/README.md b/packages/README.md index 54a5779016..b2a870f516 100644 --- a/packages/README.md +++ b/packages/README.md @@ -16,7 +16,7 @@ Packages live at `packages///`; groups are containers, while names r | [`goal/`](goal/README.md) | Same-session goal persistence and lifecycle | Product — stable surface | | [`feedback/`](feedback/README.md) | Human feedback | Product — stable surface | | [`llm/`](llm/README.md) | LLM capability family: the abstract service + provider adapters | Product — stable surface | -| [`e2b/`](e2b/README.md) | E2B remote filesystem/process providers | POC | +| [`e2b/`](e2b/README.md) | E2B remote-runtime provider family | POC | | [`subprocess/`](subprocess/README.md) | Subprocess capability family: spawn seam + local process-tree implementation | Product — stable surface | | [`bash/`](bash/README.md) | Bash capability family: executor seam, local impl, model-facing tool | Product — stable surface | | [`pty/`](pty/README.md) | Persistent PTY capability family: owner-scoped sessions, local implementation, and model-facing tools | Product — stable surface | diff --git a/packages/README.zh.md b/packages/README.zh.md index 46b112ec8f..c8079c3d03 100644 --- a/packages/README.zh.md +++ b/packages/README.zh.md @@ -16,7 +16,7 @@ | [`goal/`](goal/README.md) | 同会话 goal 的持久化与生命周期 | 产品:稳定表面 | | [`feedback/`](feedback/README.md) | 人类反馈 | 产品:稳定表面 | | [`llm/`](llm/README.md) | LLM(大语言模型)能力系列:抽象服务 + 提供方适配器 | 产品:稳定表面 | -| [`e2b/`](e2b/README.md) | E2B 远程文件系统/进程管理提供方 | POC | +| [`e2b/`](e2b/README.md) | E2B 远程运行时提供方系列 | POC | | [`subprocess/`](subprocess/README.md) | 进程管理能力系列:spawn seam + 本地进程树实现 | 产品:稳定表面 | | [`bash/`](bash/README.md) | Bash 能力系列:执行器 seam、本地实现、面向模型的工具 | 产品:稳定表面 | | [`pty/`](pty/README.md) | 持久 PTY 能力系列:按所有者隔离的会话、本地实现和面向模型的工具 | 产品:稳定表面 | diff --git a/packages/code-runtime/README.i18n.yaml b/packages/code-runtime/README.i18n.yaml index f9160b4f8e..30c5ca2572 100644 --- a/packages/code-runtime/README.i18n.yaml +++ b/packages/code-runtime/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/code-runtime/README.md -README.md: f59fc3b15331b4799cbc7a3fbe27ce8cd9a12e51 -README.zh.md: 02ade97887fab8a8a0c22efcf73bb2dd1f2cf33c +README.md: 34eecc4874afc5a576ed129e72eeeb944b95f67f +README.zh.md: 85d7a95a37cd1f4090062f3036a6634fde5011ee diff --git a/packages/code-runtime/README.md b/packages/code-runtime/README.md index f59fc3b153..34eecc4874 100644 --- a/packages/code-runtime/README.md +++ b/packages/code-runtime/README.md @@ -8,6 +8,6 @@ The code-execution capability seam (see [capability seams](../../.agents/notes/i |---|---|---| | `code-runtime/` | Abstract code-execution seam (interface + vocabulary) | `ctx.codeRuntime` | | [`code-runtime-worker/`](code-runtime-worker/README.md) | Worker-thread backend: fresh worker per run, TypeScript via host-side type-strip (annotations advisory, never type-checked), port-bridged bindings, budget/heap containment | registers `ctx.codeRuntime` | -| [`code-runtime-e2b/`](code-runtime-e2b/README.md) | E2B backend: host type-strip and bindings, fresh remote runner/worker, framed bridge, remote process-group cleanup | registers `ctx.codeRuntime` | +| [`e2b/code-runtime-e2b`](../e2b/code-runtime-e2b/README.md) | E2B backend: host type-strip and bindings, fresh remote runner/worker, framed bridge, remote process-group cleanup | registers `ctx.codeRuntime` | -Backends differ by execution substrate and source language—both readonly descriptors on the service—and register `ctx.codeRuntime` without touching the interface or its consumer. The E2B ownership split is recorded in the [remote extension note](../../.agents/notes/implemented/feature/2026-07-28-e2b-interactive-semantic-code-runtime-poc.md). +Backends differ by execution substrate and source language—both readonly descriptors on the service—and register `ctx.codeRuntime` without touching the interface or its consumer. The E2B ownership split is recorded in the [shared E2B runtime note](../../.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md). diff --git a/packages/code-runtime/README.zh.md b/packages/code-runtime/README.zh.md index 02ade97887..85d7a95a37 100644 --- a/packages/code-runtime/README.zh.md +++ b/packages/code-runtime/README.zh.md @@ -8,6 +8,6 @@ |---|---|---| | `code-runtime/` | 抽象代码执行 seam(接口 + 词汇) | `ctx.codeRuntime` | | [`code-runtime-worker/`](code-runtime-worker/README.md) | worker 线程后端:每次运行使用全新 worker,由宿主侧剥离 TypeScript 类型(类型注解仅供参考,绝不执行类型检查)、端口桥接绑定、预算/堆隔离 | 注册 `ctx.codeRuntime` | -| [`code-runtime-e2b/`](code-runtime-e2b/README.md) | E2B 后端:宿主侧类型剥离与绑定、全新远程 runner/worker、分帧桥、远程进程组清理 | 注册 `ctx.codeRuntime` | +| [`e2b/code-runtime-e2b`](../e2b/code-runtime-e2b/README.md) | E2B 后端:宿主侧类型剥离与绑定、全新远程 runner/worker、分帧桥、远程进程组清理 | 注册 `ctx.codeRuntime` | -不同后端的执行基底和源语言各异,二者都是服务上的只读描述符;后端注册 `ctx.codeRuntime`,无需修改接口或消费方。E2B 所有权拆分记录在[远程扩展 Agent Note](../../.agents/notes/implemented/feature/2026-07-28-e2b-interactive-semantic-code-runtime-poc.md) 中。 +不同后端的执行基底和源语言各异,二者都是服务上的只读描述符;后端注册 `ctx.codeRuntime`,无需修改接口或消费方。E2B 所有权拆分记录在[共享 E2B 运行时 Agent Note](../../.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md) 中。 diff --git a/packages/code-runtime/code-runtime-e2b/README.i18n.yaml b/packages/code-runtime/code-runtime-e2b/README.i18n.yaml deleted file mode 100644 index c00b40f4a2..0000000000 --- a/packages/code-runtime/code-runtime-e2b/README.i18n.yaml +++ /dev/null @@ -1,6 +0,0 @@ -# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each -# side as of the last confirmed-consistent state. Both languages carry equal authority; -# after editing either side, bring the other along and re-record with: -# pnpm run verify-translation-pairing --write packages/code-runtime/code-runtime-e2b/README.md -README.md: bf26174c59c1df18f63278acb7923fd409860b44 -README.zh.md: c87945da30a6f8d77eb3722879510bb6159b3ee8 diff --git a/packages/e2b/README.i18n.yaml b/packages/e2b/README.i18n.yaml index 444145fbb8..1ded2a1d34 100644 --- a/packages/e2b/README.i18n.yaml +++ b/packages/e2b/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/e2b/README.md -README.md: 9e7f1a98ada9f56f3d1e3f74d2a906f8a1c7bc15 -README.zh.md: 53fdf30eaf405ab5313274a7fadde0b044b7fba1 +README.md: b25f00fb5f32643008127f0cee4f3d758018404a +README.zh.md: fc3d6901f3cdf6c645cfde49dc46108759959aca diff --git a/packages/e2b/README.md b/packages/e2b/README.md index 9e7f1a98ad..b25f00fb5f 100644 --- a/packages/e2b/README.md +++ b/packages/e2b/README.md @@ -7,10 +7,10 @@ An experimental provider-composition POC that places the mutable coding world in | Package | ctx key | Role | |---|---|---| | [`e2b`](e2b/README.md) (`@deepseek-ai/dsh-e2b`) | `ctx.e2b` | Create or reconnect one sandbox, create its working/runtime directories, expose the shared SDK handle, and apply the configured kill/pause/leave disposition | -| [`fs-e2b`](../fs/fs-e2b/README.md) (`@deepseek-ai/dsh-fs-e2b`) | `ctx.fs` | Implement the filesystem seam over E2B Filesystem APIs | -| [`subprocess-e2b`](../subprocess/subprocess-e2b/README.md) (`@deepseek-ai/dsh-subprocess-e2b`) | `ctx.subprocess` | Implement managed process groups, stdio projection, and remote spill files over E2B Commands | -| [`pty-e2b`](../pty/pty-e2b/README.md) (`@deepseek-ai/dsh-pty-e2b`) | `ctx.pty` backend | Run persistent interactive shells through E2B's byte PTY API | -| [`lsp-e2b`](../lsp/lsp-e2b/README.md) (`@deepseek-ai/dsh-lsp-e2b`) | `ctx.lsp` provider | Run configured language servers and read query sources inside E2B | -| [`code-runtime-e2b`](../code-runtime/code-runtime-e2b/README.md) (`@deepseek-ai/dsh-code-runtime-e2b`) | `ctx.codeRuntime` | Run model-written programs remotely while bridging bindings to the host | +| [`fs-e2b`](fs-e2b/README.md) (`@deepseek-ai/dsh-fs-e2b`) | `ctx.fs` | Implement the filesystem seam over E2B Filesystem APIs | +| [`subprocess-e2b`](subprocess-e2b/README.md) (`@deepseek-ai/dsh-subprocess-e2b`) | `ctx.subprocess` | Implement managed process groups, stdio projection, and remote spill files over E2B Commands | +| [`pty-e2b`](pty-e2b/README.md) (`@deepseek-ai/dsh-pty-e2b`) | `ctx.pty` backend | Run persistent interactive shells through E2B's byte PTY API | +| [`lsp-e2b`](lsp-e2b/README.md) (`@deepseek-ai/dsh-lsp-e2b`) | `ctx.lsp` provider | Run configured language servers and read query sources inside E2B | +| [`code-runtime-e2b`](code-runtime-e2b/README.md) (`@deepseek-ai/dsh-code-runtime-e2b`) | `ctx.codeRuntime` | Run model-written programs remotely while bridging bindings to the host | -The existing [`dsh-bash-local`](../bash/bash-local/README.md) needs no E2B-specific fork: it delegates process mechanics to `ctx.subprocess`, so replacing that provider places Bash in the same remote world. This boundary does not move the harness process, Cordis objects, model calls, agent/session state, session persistence, skills, protocol state, or E2B SDK buffers. The [base decision](../../.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md) and [runtime-extension decision](../../.agents/notes/implemented/feature/2026-07-28-e2b-interactive-semantic-code-runtime-poc.md) own the POC boundary. +The existing [`dsh-bash-local`](../bash/bash-local/README.md) needs no E2B-specific fork: it delegates process mechanics to `ctx.subprocess`, so replacing that provider places Bash in the same remote world. This boundary does not move the harness process, Cordis objects, model calls, agent/session state, session persistence, skills, protocol state, or E2B SDK buffers. The [shared-runtime decision](../../.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md) owns the POC boundary. diff --git a/packages/e2b/README.zh.md b/packages/e2b/README.zh.md index 53fdf30eaf..fc3d6901f3 100644 --- a/packages/e2b/README.zh.md +++ b/packages/e2b/README.zh.md @@ -7,10 +7,10 @@ | 包(package) | ctx 键 | 职责 | |---|---|---| | [`e2b`](e2b/README.md)(`@deepseek-ai/dsh-e2b`) | `ctx.e2b` | 创建或重新连接一个沙箱,创建其工作目录与运行时目录,公开共享 SDK 句柄,并应用配置的 kill/pause/leave 处置方式 | -| [`fs-e2b`](../fs/fs-e2b/README.md)(`@deepseek-ai/dsh-fs-e2b`) | `ctx.fs` | 通过 E2B Filesystem API 实现文件系统 seam | -| [`subprocess-e2b`](../subprocess/subprocess-e2b/README.md)(`@deepseek-ai/dsh-subprocess-e2b`) | `ctx.subprocess` | 通过 E2B Commands 实现受管进程组、stdio 投影与远程 spill 文件 | -| [`pty-e2b`](../pty/pty-e2b/README.md)(`@deepseek-ai/dsh-pty-e2b`) | `ctx.pty` 后端 | 通过 E2B 的字节 PTY API 运行持久交互式 shell | -| [`lsp-e2b`](../lsp/lsp-e2b/README.md)(`@deepseek-ai/dsh-lsp-e2b`) | `ctx.lsp` 提供方 | 在 E2B 内运行已配置的语言服务器并读取查询源代码 | -| [`code-runtime-e2b`](../code-runtime/code-runtime-e2b/README.md)(`@deepseek-ai/dsh-code-runtime-e2b`) | `ctx.codeRuntime` | 远程运行模型编写的程序,同时把绑定桥接到宿主 | +| [`fs-e2b`](fs-e2b/README.md)(`@deepseek-ai/dsh-fs-e2b`) | `ctx.fs` | 通过 E2B Filesystem API 实现文件系统 seam | +| [`subprocess-e2b`](subprocess-e2b/README.md)(`@deepseek-ai/dsh-subprocess-e2b`) | `ctx.subprocess` | 通过 E2B Commands 实现受管进程组、stdio 投影与远程 spill 文件 | +| [`pty-e2b`](pty-e2b/README.md)(`@deepseek-ai/dsh-pty-e2b`) | `ctx.pty` 后端 | 通过 E2B 的字节 PTY API 运行持久交互式 shell | +| [`lsp-e2b`](lsp-e2b/README.md)(`@deepseek-ai/dsh-lsp-e2b`) | `ctx.lsp` 提供方 | 在 E2B 内运行已配置的语言服务器并读取查询源代码 | +| [`code-runtime-e2b`](code-runtime-e2b/README.md)(`@deepseek-ai/dsh-code-runtime-e2b`) | `ctx.codeRuntime` | 远程运行模型编写的程序,同时把绑定桥接到宿主 | -现有的 [`dsh-bash-local`](../bash/bash-local/README.md) 无需 E2B 专用 fork:它把进程机制委托给 `ctx.subprocess`,因此替换该提供方即可让 Bash 进入同一个远程环境。该边界不会迁移 harness 进程、Cordis 对象、模型调用、agent(智能体)/会话状态、会话持久化、skill(技能)、协议状态或 E2B SDK 缓冲。[基础决策](../../.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md)与[运行时扩展决策](../../.agents/notes/implemented/feature/2026-07-28-e2b-interactive-semantic-code-runtime-poc.md)共同界定 POC 边界。 +现有的 [`dsh-bash-local`](../bash/bash-local/README.md) 无需 E2B 专用 fork:它把进程机制委托给 `ctx.subprocess`,因此替换该提供方即可让 Bash 进入同一个远程环境。该边界不会迁移 harness 进程、Cordis 对象、模型调用、agent(智能体)/会话状态、会话持久化、skill(技能)、协议状态或 E2B SDK 缓冲。[共享运行时决策](../../.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md)界定 POC 边界。 diff --git a/packages/subprocess/subprocess-e2b/README.i18n.yaml b/packages/e2b/code-runtime-e2b/README.i18n.yaml similarity index 54% rename from packages/subprocess/subprocess-e2b/README.i18n.yaml rename to packages/e2b/code-runtime-e2b/README.i18n.yaml index dfbe97e8b9..9eb2633b86 100644 --- a/packages/subprocess/subprocess-e2b/README.i18n.yaml +++ b/packages/e2b/code-runtime-e2b/README.i18n.yaml @@ -1,6 +1,6 @@ # Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: -# pnpm run verify-translation-pairing --write packages/subprocess/subprocess-e2b/README.md -README.md: 9ec1518103413f33a52023c17722bf5a5fe275ce -README.zh.md: f50ff2f3d85a5ddd9bd352e95d0eb0a23e756404 +# pnpm run verify-translation-pairing --write packages/e2b/code-runtime-e2b/README.md +README.md: 1007800c59725d116197a52f6d087ea5e2860686 +README.zh.md: a56026bce760879079627dd63a80dfe95c4d2049 diff --git a/packages/code-runtime/code-runtime-e2b/README.md b/packages/e2b/code-runtime-e2b/README.md similarity index 93% rename from packages/code-runtime/code-runtime-e2b/README.md rename to packages/e2b/code-runtime-e2b/README.md index bf26174c59..1007800c59 100644 --- a/packages/code-runtime/code-runtime-e2b/README.md +++ b/packages/e2b/code-runtime-e2b/README.md @@ -2,7 +2,7 @@ English | [中文](README.zh.md) -E2B implementation of [`ctx.codeRuntime`](../code-runtime/README.md). Each run executes one model-written TypeScript program in a fresh remote Node worker while binding functions, type stripping, output accounting, and lifecycle orchestration remain on the host. +E2B implementation of [`ctx.codeRuntime`](../../code-runtime/code-runtime/README.md). Each run executes one model-written TypeScript program in a fresh remote Node worker while binding functions, type stripping, output accounting, and lifecycle orchestration remain on the host. ## Configuration diff --git a/packages/code-runtime/code-runtime-e2b/README.zh.md b/packages/e2b/code-runtime-e2b/README.zh.md similarity index 93% rename from packages/code-runtime/code-runtime-e2b/README.zh.md rename to packages/e2b/code-runtime-e2b/README.zh.md index c87945da30..a56026bce7 100644 --- a/packages/code-runtime/code-runtime-e2b/README.zh.md +++ b/packages/e2b/code-runtime-e2b/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -[`ctx.codeRuntime`](../code-runtime/README.md) 的 E2B 实现。每次运行都会在全新的远程 Node worker 中执行一段模型编写的 TypeScript 程序;绑定函数、类型剥离、输出记账和生命周期编排仍保留在宿主侧。 +[`ctx.codeRuntime`](../../code-runtime/code-runtime/README.md) 的 E2B 实现。每次运行都会在全新的远程 Node worker 中执行一段模型编写的 TypeScript 程序;绑定函数、类型剥离、输出记账和生命周期编排仍保留在宿主侧。 ## 配置 diff --git a/packages/code-runtime/code-runtime-e2b/package.json b/packages/e2b/code-runtime-e2b/package.json similarity index 100% rename from packages/code-runtime/code-runtime-e2b/package.json rename to packages/e2b/code-runtime-e2b/package.json diff --git a/packages/code-runtime/code-runtime-e2b/src/index.ts b/packages/e2b/code-runtime-e2b/src/index.ts similarity index 100% rename from packages/code-runtime/code-runtime-e2b/src/index.ts rename to packages/e2b/code-runtime-e2b/src/index.ts diff --git a/packages/code-runtime/code-runtime-e2b/src/invariant.ts b/packages/e2b/code-runtime-e2b/src/invariant.ts similarity index 100% rename from packages/code-runtime/code-runtime-e2b/src/invariant.ts rename to packages/e2b/code-runtime-e2b/src/invariant.ts diff --git a/packages/code-runtime/code-runtime-e2b/src/runner-source.ts b/packages/e2b/code-runtime-e2b/src/runner-source.ts similarity index 100% rename from packages/code-runtime/code-runtime-e2b/src/runner-source.ts rename to packages/e2b/code-runtime-e2b/src/runner-source.ts diff --git a/packages/code-runtime/code-runtime-e2b/tests/runtime.spec.ts b/packages/e2b/code-runtime-e2b/tests/runtime.spec.ts similarity index 100% rename from packages/code-runtime/code-runtime-e2b/tests/runtime.spec.ts rename to packages/e2b/code-runtime-e2b/tests/runtime.spec.ts diff --git a/packages/code-runtime/code-runtime-e2b/tsconfig.json b/packages/e2b/code-runtime-e2b/tsconfig.json similarity index 70% rename from packages/code-runtime/code-runtime-e2b/tsconfig.json rename to packages/e2b/code-runtime-e2b/tsconfig.json index 4ddc3b5ffe..3bc2e21873 100644 --- a/packages/code-runtime/code-runtime-e2b/tsconfig.json +++ b/packages/e2b/code-runtime-e2b/tsconfig.json @@ -9,11 +9,11 @@ { "path": "../../../vendor/cosmokit" }, { "path": "../../../vendor/cordis" }, { "path": "../../../vendor/schemastery" }, - { "path": "../code-runtime" }, - { "path": "../code-runtime-worker" }, - { "path": "../../e2b/e2b" }, + { "path": "../../code-runtime/code-runtime" }, + { "path": "../../code-runtime/code-runtime-worker" }, + { "path": "../e2b" }, { "path": "../../core/session" }, - { "path": "../../subprocess/subprocess-e2b" }, + { "path": "../subprocess-e2b" }, { "path": "../../util/timeout" }, { "path": "../../support/invariants" } ] diff --git a/packages/fs/fs-e2b/README.i18n.yaml b/packages/e2b/fs-e2b/README.i18n.yaml similarity index 56% rename from packages/fs/fs-e2b/README.i18n.yaml rename to packages/e2b/fs-e2b/README.i18n.yaml index d3ea2786b2..b416105166 100644 --- a/packages/fs/fs-e2b/README.i18n.yaml +++ b/packages/e2b/fs-e2b/README.i18n.yaml @@ -1,6 +1,6 @@ # Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: -# pnpm run verify-translation-pairing --write packages/fs/fs-e2b/README.md -README.md: 2b23884e345e7c39b0559465afa0bc588699b70a -README.zh.md: 1a9a174ac90c59b68b02887303f58ee2a99dc72e +# pnpm run verify-translation-pairing --write packages/e2b/fs-e2b/README.md +README.md: d8f3915fa281e8b9c062e3412c92bc6358616284 +README.zh.md: 93d27bcdd514ccbf87e07d0eff5958edf3c726c1 diff --git a/packages/fs/fs-e2b/README.md b/packages/e2b/fs-e2b/README.md similarity index 81% rename from packages/fs/fs-e2b/README.md rename to packages/e2b/fs-e2b/README.md index 2b23884e34..d8f3915fa2 100644 --- a/packages/fs/fs-e2b/README.md +++ b/packages/e2b/fs-e2b/README.md @@ -2,7 +2,7 @@ English | [中文](README.zh.md) -E2B implementation of the [`@deepseek-ai/dsh-fs`](../fs/README.md) provider seam. It has no config: load [`@deepseek-ai/dsh-e2b`](../../e2b/e2b/README.md) first, then this service in place of `dsh-fs-local`. The provider uses the owner's remote cwd and SDK handle, so file tools observe the same world as E2B-backed Bash processes. +E2B implementation of the [`@deepseek-ai/dsh-fs`](../../fs/fs/README.md) provider seam. It has no config: load [`@deepseek-ai/dsh-e2b`](../e2b/README.md) first, then this service in place of `dsh-fs-local`. The provider uses the owner's remote cwd and SDK handle, so file tools observe the same world as E2B-backed Bash processes. ## Behavior @@ -15,7 +15,7 @@ The provider does not copy, mount, or reconcile the host workspace. Giving it a ## Model Experience -Indirectly, through [`dsh-tool-fs`](../tool-fs/README.md), which renders remote UTF-8 content, directory results, mutation acknowledgements, and provider errors while E2B identity and transport remain internal. +Indirectly, through [`dsh-tool-fs`](../../fs/tool-fs/README.md), which renders remote UTF-8 content, directory results, mutation acknowledgements, and provider errors while E2B identity and transport remain internal. #### KV Cache effect diff --git a/packages/fs/fs-e2b/README.zh.md b/packages/e2b/fs-e2b/README.zh.md similarity index 81% rename from packages/fs/fs-e2b/README.zh.md rename to packages/e2b/fs-e2b/README.zh.md index 1a9a174ac9..93d27bcdd5 100644 --- a/packages/fs/fs-e2b/README.zh.md +++ b/packages/e2b/fs-e2b/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -[`@deepseek-ai/dsh-fs`](../fs/README.md) 提供方 seam 的 E2B 实现。它没有配置:先加载 [`@deepseek-ai/dsh-e2b`](../../e2b/e2b/README.md),再用本服务取代 `dsh-fs-local`。该提供方使用所有者的远程 cwd 和 SDK 句柄,因此文件工具观察到的环境与 E2B 后端 Bash 进程相同。 +[`@deepseek-ai/dsh-fs`](../../fs/fs/README.md) 提供方 seam 的 E2B 实现。它没有配置:先加载 [`@deepseek-ai/dsh-e2b`](../e2b/README.md),再用本服务取代 `dsh-fs-local`。该提供方使用所有者的远程 cwd 和 SDK 句柄,因此文件工具观察到的环境与 E2B 后端 Bash 进程相同。 ## 行为 @@ -15,7 +15,7 @@ ## 模型体验 -通过 [`dsh-tool-fs`](../tool-fs/README.md) 间接影响模型;该工具会渲染远程 UTF-8 内容、目录结果、变更确认和提供方错误,而 E2B 身份及传输保持内部实现。 +通过 [`dsh-tool-fs`](../../fs/tool-fs/README.md) 间接影响模型;该工具会渲染远程 UTF-8 内容、目录结果、变更确认和提供方错误,而 E2B 身份及传输保持内部实现。 #### KV Cache 影响 diff --git a/packages/fs/fs-e2b/package.json b/packages/e2b/fs-e2b/package.json similarity index 100% rename from packages/fs/fs-e2b/package.json rename to packages/e2b/fs-e2b/package.json diff --git a/packages/fs/fs-e2b/src/index.ts b/packages/e2b/fs-e2b/src/index.ts similarity index 100% rename from packages/fs/fs-e2b/src/index.ts rename to packages/e2b/fs-e2b/src/index.ts diff --git a/packages/fs/fs-e2b/src/invariant.ts b/packages/e2b/fs-e2b/src/invariant.ts similarity index 100% rename from packages/fs/fs-e2b/src/invariant.ts rename to packages/e2b/fs-e2b/src/invariant.ts diff --git a/packages/fs/fs-e2b/tests/filesystem.spec.ts b/packages/e2b/fs-e2b/tests/filesystem.spec.ts similarity index 100% rename from packages/fs/fs-e2b/tests/filesystem.spec.ts rename to packages/e2b/fs-e2b/tests/filesystem.spec.ts diff --git a/packages/fs/fs-e2b/tsconfig.json b/packages/e2b/fs-e2b/tsconfig.json similarity index 87% rename from packages/fs/fs-e2b/tsconfig.json rename to packages/e2b/fs-e2b/tsconfig.json index c424efd2e5..bcb54a88d9 100644 --- a/packages/fs/fs-e2b/tsconfig.json +++ b/packages/e2b/fs-e2b/tsconfig.json @@ -13,10 +13,10 @@ "path": "../../../vendor/cordis" }, { - "path": "../../e2b/e2b" + "path": "../e2b" }, { - "path": "../fs" + "path": "../../fs/fs" }, { "path": "../../support/invariants" diff --git a/packages/pty/pty-e2b/README.i18n.yaml b/packages/e2b/lsp-e2b/README.i18n.yaml similarity index 60% rename from packages/pty/pty-e2b/README.i18n.yaml rename to packages/e2b/lsp-e2b/README.i18n.yaml index bfd7795349..5f126d7e56 100644 --- a/packages/pty/pty-e2b/README.i18n.yaml +++ b/packages/e2b/lsp-e2b/README.i18n.yaml @@ -1,6 +1,6 @@ # Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: -# pnpm run verify-translation-pairing --write packages/pty/pty-e2b/README.md -README.md: 7a8ddc9e08f4d28d69b97825232b00e2d8f394b7 -README.zh.md: 2151ca1f2d91b01a7b0106153f708ace2c3d2726 +# pnpm run verify-translation-pairing --write packages/e2b/lsp-e2b/README.md +README.md: ded928ec073e6c0943d574d86359eeec586bb862 +README.zh.md: 91eb7cecb49adcf73f4be948e9a5da400981e6f1 diff --git a/packages/lsp/lsp-e2b/README.md b/packages/e2b/lsp-e2b/README.md similarity index 92% rename from packages/lsp/lsp-e2b/README.md rename to packages/e2b/lsp-e2b/README.md index 968487b7c5..ded928ec07 100644 --- a/packages/lsp/lsp-e2b/README.md +++ b/packages/e2b/lsp-e2b/README.md @@ -2,7 +2,7 @@ English | [中文](README.zh.md) -Generic E2B language-server backend for [`ctx.lsp`](../lsp/README.md). It runs configured stdio servers and reads their source documents inside the shared `ctx.e2b` sandbox; the provider registry, normalized query results, queues, and protocol connection state remain on the host. +Generic E2B language-server backend for [`ctx.lsp`](../../lsp/lsp/README.md). It runs configured stdio servers and reads their source documents inside the shared `ctx.e2b` sandbox; the provider registry, normalized query results, queues, and protocol connection state remain on the host. ## Plugin and configuration diff --git a/packages/lsp/lsp-e2b/README.zh.md b/packages/e2b/lsp-e2b/README.zh.md similarity index 92% rename from packages/lsp/lsp-e2b/README.zh.md rename to packages/e2b/lsp-e2b/README.zh.md index 2b937f0115..91eb7cecb4 100644 --- a/packages/lsp/lsp-e2b/README.zh.md +++ b/packages/e2b/lsp-e2b/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -用于 [`ctx.lsp`](../lsp/README.md) 的通用 E2B 语言服务器后端。它在共享的 `ctx.e2b` 沙箱内运行已配置的 stdio 服务器并读取其源文档;提供方注册表、规范化查询结果、队列和协议连接状态仍保留在宿主侧。 +用于 [`ctx.lsp`](../../lsp/lsp/README.md) 的通用 E2B 语言服务器后端。它在共享的 `ctx.e2b` 沙箱内运行已配置的 stdio 服务器并读取其源文档;提供方注册表、规范化查询结果、队列和协议连接状态仍保留在宿主侧。 ## 插件与配置 diff --git a/packages/lsp/lsp-e2b/package.json b/packages/e2b/lsp-e2b/package.json similarity index 100% rename from packages/lsp/lsp-e2b/package.json rename to packages/e2b/lsp-e2b/package.json diff --git a/packages/lsp/lsp-e2b/src/index.ts b/packages/e2b/lsp-e2b/src/index.ts similarity index 100% rename from packages/lsp/lsp-e2b/src/index.ts rename to packages/e2b/lsp-e2b/src/index.ts diff --git a/packages/lsp/lsp-e2b/src/invariant.ts b/packages/e2b/lsp-e2b/src/invariant.ts similarity index 100% rename from packages/lsp/lsp-e2b/src/invariant.ts rename to packages/e2b/lsp-e2b/src/invariant.ts diff --git a/packages/lsp/lsp-e2b/src/proxy-source.ts b/packages/e2b/lsp-e2b/src/proxy-source.ts similarity index 100% rename from packages/lsp/lsp-e2b/src/proxy-source.ts rename to packages/e2b/lsp-e2b/src/proxy-source.ts diff --git a/packages/lsp/lsp-e2b/src/transport.ts b/packages/e2b/lsp-e2b/src/transport.ts similarity index 100% rename from packages/lsp/lsp-e2b/src/transport.ts rename to packages/e2b/lsp-e2b/src/transport.ts diff --git a/packages/lsp/lsp-e2b/tests/provider.spec.ts b/packages/e2b/lsp-e2b/tests/provider.spec.ts similarity index 100% rename from packages/lsp/lsp-e2b/tests/provider.spec.ts rename to packages/e2b/lsp-e2b/tests/provider.spec.ts diff --git a/packages/lsp/lsp-e2b/tests/transport.spec.ts b/packages/e2b/lsp-e2b/tests/transport.spec.ts similarity index 100% rename from packages/lsp/lsp-e2b/tests/transport.spec.ts rename to packages/e2b/lsp-e2b/tests/transport.spec.ts diff --git a/packages/lsp/lsp-e2b/tsconfig.json b/packages/e2b/lsp-e2b/tsconfig.json similarity index 75% rename from packages/lsp/lsp-e2b/tsconfig.json rename to packages/e2b/lsp-e2b/tsconfig.json index 4679845db8..a6c0fd8ece 100644 --- a/packages/lsp/lsp-e2b/tsconfig.json +++ b/packages/e2b/lsp-e2b/tsconfig.json @@ -9,11 +9,11 @@ { "path": "../../../vendor/cosmokit" }, { "path": "../../../vendor/cordis" }, { "path": "../../../vendor/schemastery" }, - { "path": "../../e2b/e2b" }, - { "path": "../lsp" }, - { "path": "../lsp-local" }, + { "path": "../e2b" }, + { "path": "../../lsp/lsp" }, + { "path": "../../lsp/lsp-local" }, { "path": "../../subprocess/subprocess" }, - { "path": "../../subprocess/subprocess-e2b" }, + { "path": "../subprocess-e2b" }, { "path": "../../util/timeout" }, { "path": "../../support/invariants" } ] diff --git a/packages/lsp/lsp-e2b/README.i18n.yaml b/packages/e2b/pty-e2b/README.i18n.yaml similarity index 60% rename from packages/lsp/lsp-e2b/README.i18n.yaml rename to packages/e2b/pty-e2b/README.i18n.yaml index fa159820a9..1bd8485e8f 100644 --- a/packages/lsp/lsp-e2b/README.i18n.yaml +++ b/packages/e2b/pty-e2b/README.i18n.yaml @@ -1,6 +1,6 @@ # Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: -# pnpm run verify-translation-pairing --write packages/lsp/lsp-e2b/README.md -README.md: 968487b7c5f63b2165cf66797a4182642507a8e9 -README.zh.md: 2b937f01156c3194f64ed9fb54478fec5c77305b +# pnpm run verify-translation-pairing --write packages/e2b/pty-e2b/README.md +README.md: 3d363309f6bd1c4def5fbff668735958ca2de52d +README.zh.md: 78d61cc654bf6eb46a3bec7124b3cc06cc7e34e9 diff --git a/packages/pty/pty-e2b/README.md b/packages/e2b/pty-e2b/README.md similarity index 93% rename from packages/pty/pty-e2b/README.md rename to packages/e2b/pty-e2b/README.md index 7a8ddc9e08..3d363309f6 100644 --- a/packages/pty/pty-e2b/README.md +++ b/packages/e2b/pty-e2b/README.md @@ -2,7 +2,7 @@ English | [中文](README.zh.md) -E2B byte-PTY backend for [`ctx.pty`](../pty/README.md). It creates persistent interactive shells inside the shared `ctx.e2b` sandbox while the PTY registry keeps session identity, exact-Agent ownership, and cleanup policy on the host. +E2B byte-PTY backend for [`ctx.pty`](../../pty/pty/README.md). It creates persistent interactive shells inside the shared `ctx.e2b` sandbox while the PTY registry keeps session identity, exact-Agent ownership, and cleanup policy on the host. ## Plugin and configuration diff --git a/packages/pty/pty-e2b/README.zh.md b/packages/e2b/pty-e2b/README.zh.md similarity index 93% rename from packages/pty/pty-e2b/README.zh.md rename to packages/e2b/pty-e2b/README.zh.md index 2151ca1f2d..78d61cc654 100644 --- a/packages/pty/pty-e2b/README.zh.md +++ b/packages/e2b/pty-e2b/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -用于 [`ctx.pty`](../pty/README.md) 的 E2B 字节 PTY 后端。它在共享的 `ctx.e2b` 沙箱内创建持久交互式 shell;PTY 注册表则在宿主侧维护会话身份、精确的 Agent 所有权和清理策略。 +用于 [`ctx.pty`](../../pty/pty/README.md) 的 E2B 字节 PTY 后端。它在共享的 `ctx.e2b` 沙箱内创建持久交互式 shell;PTY 注册表则在宿主侧维护会话身份、精确的 Agent 所有权和清理策略。 ## 插件与配置 diff --git a/packages/pty/pty-e2b/package.json b/packages/e2b/pty-e2b/package.json similarity index 100% rename from packages/pty/pty-e2b/package.json rename to packages/e2b/pty-e2b/package.json diff --git a/packages/pty/pty-e2b/src/config.ts b/packages/e2b/pty-e2b/src/config.ts similarity index 100% rename from packages/pty/pty-e2b/src/config.ts rename to packages/e2b/pty-e2b/src/config.ts diff --git a/packages/pty/pty-e2b/src/index.ts b/packages/e2b/pty-e2b/src/index.ts similarity index 100% rename from packages/pty/pty-e2b/src/index.ts rename to packages/e2b/pty-e2b/src/index.ts diff --git a/packages/pty/pty-e2b/src/invariant.ts b/packages/e2b/pty-e2b/src/invariant.ts similarity index 100% rename from packages/pty/pty-e2b/src/invariant.ts rename to packages/e2b/pty-e2b/src/invariant.ts diff --git a/packages/pty/pty-e2b/src/session.ts b/packages/e2b/pty-e2b/src/session.ts similarity index 100% rename from packages/pty/pty-e2b/src/session.ts rename to packages/e2b/pty-e2b/src/session.ts diff --git a/packages/pty/pty-e2b/tests/index.spec.ts b/packages/e2b/pty-e2b/tests/index.spec.ts similarity index 100% rename from packages/pty/pty-e2b/tests/index.spec.ts rename to packages/e2b/pty-e2b/tests/index.spec.ts diff --git a/packages/pty/pty-e2b/tests/session.spec.ts b/packages/e2b/pty-e2b/tests/session.spec.ts similarity index 100% rename from packages/pty/pty-e2b/tests/session.spec.ts rename to packages/e2b/pty-e2b/tests/session.spec.ts diff --git a/packages/pty/pty-e2b/tsconfig.json b/packages/e2b/pty-e2b/tsconfig.json similarity index 85% rename from packages/pty/pty-e2b/tsconfig.json rename to packages/e2b/pty-e2b/tsconfig.json index 0a7647aa80..06bf0cd589 100644 --- a/packages/pty/pty-e2b/tsconfig.json +++ b/packages/e2b/pty-e2b/tsconfig.json @@ -9,8 +9,8 @@ { "path": "../../../vendor/cosmokit" }, { "path": "../../../vendor/cordis" }, { "path": "../../../vendor/schemastery" }, - { "path": "../../e2b/e2b" }, - { "path": "../pty" }, + { "path": "../e2b" }, + { "path": "../../pty/pty" }, { "path": "../../support/invariants" } ] } diff --git a/packages/e2b/subprocess-e2b/README.i18n.yaml b/packages/e2b/subprocess-e2b/README.i18n.yaml new file mode 100644 index 0000000000..26bc8f0b51 --- /dev/null +++ b/packages/e2b/subprocess-e2b/README.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write packages/e2b/subprocess-e2b/README.md +README.md: ce61de1100791be6e5c4db74c73ff43566281ed9 +README.zh.md: a4c619f0c002cc1d36310c5a9b4a3f7ae655ad1f diff --git a/packages/subprocess/subprocess-e2b/README.md b/packages/e2b/subprocess-e2b/README.md similarity index 89% rename from packages/subprocess/subprocess-e2b/README.md rename to packages/e2b/subprocess-e2b/README.md index 9ec1518103..ce61de1100 100644 --- a/packages/subprocess/subprocess-e2b/README.md +++ b/packages/e2b/subprocess-e2b/README.md @@ -2,7 +2,7 @@ English | [中文](README.zh.md) -E2B implementation of the [`@deepseek-ai/dsh-subprocess`](../subprocess/README.md) seam. It has no config: load [`@deepseek-ai/dsh-e2b`](../../e2b/e2b/README.md) first, then this service in place of `dsh-subprocess-local`. Existing consumers such as [`dsh-bash-local`](../../bash/bash-local/README.md) then execute in the shared remote sandbox without an E2B-specific Bash adapter. +E2B implementation of the [`@deepseek-ai/dsh-subprocess`](../../subprocess/subprocess/README.md) seam. It has no config: load [`@deepseek-ai/dsh-e2b`](../e2b/README.md) first, then this service in place of `dsh-subprocess-local`. Existing consumers such as [`dsh-bash-local`](../../bash/bash-local/README.md) then execute in the shared remote sandbox without an E2B-specific Bash adapter. ## Behavior diff --git a/packages/subprocess/subprocess-e2b/README.zh.md b/packages/e2b/subprocess-e2b/README.zh.md similarity index 89% rename from packages/subprocess/subprocess-e2b/README.zh.md rename to packages/e2b/subprocess-e2b/README.zh.md index f50ff2f3d8..a4c619f0c0 100644 --- a/packages/subprocess/subprocess-e2b/README.zh.md +++ b/packages/e2b/subprocess-e2b/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -[`@deepseek-ai/dsh-subprocess`](../subprocess/README.md) seam 的 E2B 实现。它没有配置:先加载 [`@deepseek-ai/dsh-e2b`](../../e2b/e2b/README.md),再用本服务取代 `dsh-subprocess-local`。随后,[`dsh-bash-local`](../../bash/bash-local/README.md) 等现有消费方会在共享远程沙箱中执行,无需 E2B 专用 Bash 适配器。 +[`@deepseek-ai/dsh-subprocess`](../../subprocess/subprocess/README.md) seam 的 E2B 实现。它没有配置:先加载 [`@deepseek-ai/dsh-e2b`](../e2b/README.md),再用本服务取代 `dsh-subprocess-local`。随后,[`dsh-bash-local`](../../bash/bash-local/README.md) 等现有消费方会在共享远程沙箱中执行,无需 E2B 专用 Bash 适配器。 ## 行为 diff --git a/packages/subprocess/subprocess-e2b/package.json b/packages/e2b/subprocess-e2b/package.json similarity index 100% rename from packages/subprocess/subprocess-e2b/package.json rename to packages/e2b/subprocess-e2b/package.json diff --git a/packages/subprocess/subprocess-e2b/src/index.ts b/packages/e2b/subprocess-e2b/src/index.ts similarity index 100% rename from packages/subprocess/subprocess-e2b/src/index.ts rename to packages/e2b/subprocess-e2b/src/index.ts diff --git a/packages/subprocess/subprocess-e2b/src/invariant.ts b/packages/e2b/subprocess-e2b/src/invariant.ts similarity index 100% rename from packages/subprocess/subprocess-e2b/src/invariant.ts rename to packages/e2b/subprocess-e2b/src/invariant.ts diff --git a/packages/subprocess/subprocess-e2b/src/output.ts b/packages/e2b/subprocess-e2b/src/output.ts similarity index 100% rename from packages/subprocess/subprocess-e2b/src/output.ts rename to packages/e2b/subprocess-e2b/src/output.ts diff --git a/packages/subprocess/subprocess-e2b/src/process.ts b/packages/e2b/subprocess-e2b/src/process.ts similarity index 100% rename from packages/subprocess/subprocess-e2b/src/process.ts rename to packages/e2b/subprocess-e2b/src/process.ts diff --git a/packages/subprocess/subprocess-e2b/tests/subprocess.spec.ts b/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts similarity index 100% rename from packages/subprocess/subprocess-e2b/tests/subprocess.spec.ts rename to packages/e2b/subprocess-e2b/tests/subprocess.spec.ts diff --git a/packages/subprocess/subprocess-e2b/tsconfig.json b/packages/e2b/subprocess-e2b/tsconfig.json similarity index 84% rename from packages/subprocess/subprocess-e2b/tsconfig.json rename to packages/e2b/subprocess-e2b/tsconfig.json index f3bb8c2a26..fda7cfc667 100644 --- a/packages/subprocess/subprocess-e2b/tsconfig.json +++ b/packages/e2b/subprocess-e2b/tsconfig.json @@ -13,10 +13,10 @@ "path": "../../../vendor/cordis" }, { - "path": "../../e2b/e2b" + "path": "../e2b" }, { - "path": "../subprocess" + "path": "../../subprocess/subprocess" }, { "path": "../../support/invariants" diff --git a/packages/fs/README.i18n.yaml b/packages/fs/README.i18n.yaml index eabb6a602e..6b5a5d8b76 100644 --- a/packages/fs/README.i18n.yaml +++ b/packages/fs/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/fs/README.md -README.md: 8ac8807cc9ca4b5cc31b686cef9796a8e42b7f81 -README.zh.md: 49705790cc0cdf1f7199251e09af0af01f704efb +README.md: 0b9a244d0c42542909460edf7109dfb9cee34503 +README.zh.md: 77da9b73f866fd88fcb81b28cbb622b0389bf5f0 diff --git a/packages/fs/README.md b/packages/fs/README.md index 460af7c53b..96bbe6c95b 100644 --- a/packages/fs/README.md +++ b/packages/fs/README.md @@ -8,7 +8,7 @@ The filesystem stack: a provider seam (execution-world paths, bounded text IO, a |---|---|---| | `fs/` | Provider seam: canonical process paths/file URIs/containment, text IO, and atomic mutation primitives; owns the `fs/*` policy events | `ctx.fs` | | `fs-local/` | Local-filesystem `FileSystem` implementation | (registers `ctx.fs`) | -| `fs-e2b/` | E2B-backed `FileSystem` implementation sharing the remote runtime owned by `ctx.e2b` | (registers `ctx.fs`) | +| [`e2b/fs-e2b`](../e2b/fs-e2b/README.md) | E2B-backed `FileSystem` implementation sharing the remote runtime owned by `ctx.e2b` | (registers `ctx.fs`) | | `fs-sandbox/` | Sandbox-enforcing `FileSystem`: extends `fs-local` and fences write/edit by the per-call mode + workspace root policy (read-only denies, workspace-write contains to the session workspace + temp roots), reads pass through | (registers `ctx.fs`) | | `fs-policy/` | Policy gate plugin: observed-state + read-before-edit + version-guarded write/edit, via the `fs/*` event gate | (no service — `fs/*` listeners) | | `tool-fs/` | Model-facing `read`/`write`/`edit` tools AND the executor (reads via `ctx.fs`, owns read windowing, dispatches `fs/*`); preserves filesystem semantics for session-cwd-relative paths and advertises sandbox escalation fields when the mounted `ctx.fs` confines | (registers on `ctx.tools`) | diff --git a/packages/fs/README.zh.md b/packages/fs/README.zh.md index 5f70e3cb50..7f091b4f09 100644 --- a/packages/fs/README.zh.md +++ b/packages/fs/README.zh.md @@ -8,7 +8,7 @@ |---|---|---| | `fs/` | 提供方 seam:规范化进程路径、文件 URI 与包含关系、文本 I/O 和原子变更原语;拥有 `fs/*` 政策事件 | `ctx.fs` | | `fs-local/` | 本地文件系统 `FileSystem` 实现 | (注册 `ctx.fs`) | -| `fs-e2b/` | 以 E2B 为后端的 `FileSystem` 实现,共享由 `ctx.e2b` 拥有的远程运行时 | (注册 `ctx.fs`) | +| [`e2b/fs-e2b`](../e2b/fs-e2b/README.md) | 以 E2B 为后端的 `FileSystem` 实现,共享由 `ctx.e2b` 拥有的远程运行时 | (注册 `ctx.fs`) | | `fs-sandbox/` | 强制沙箱的 `FileSystem`:扩展 `fs-local`,并按每次调用的模式与工作区根政策约束写入/编辑(只读模式拒绝,工作区写入模式限制在会话工作区与临时根目录内);读取直接通过 | (注册 `ctx.fs`) | | `fs-policy/` | 政策门禁插件:通过 `fs/*` 事件门禁提供已观察状态、编辑前读取和版本防护的写入/编辑 | (无服务,仅有 `fs/*` 监听器) | | `tool-fs/` | 面向模型的 `read`/`write`/`edit` 工具以及执行器(通过 `ctx.fs` 读取,拥有读取窗口逻辑,分派 `fs/*`);为会话 cwd 相对路径保留文件系统语义,并在已挂载的 `ctx.fs` 实施约束时声明沙箱升级字段 | (注册到 `ctx.tools`) | diff --git a/packages/lsp/README.i18n.yaml b/packages/lsp/README.i18n.yaml index c28c0282c2..be241ada6e 100644 --- a/packages/lsp/README.i18n.yaml +++ b/packages/lsp/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/lsp/README.md -README.md: 74ed729d97013de3d7f502f728e3abe62348846f -README.zh.md: 5a08d54ca6c54d8355454fc3021a0da48772000b +README.md: d8bc6047e7eadeb2993f1114564b41913b2970dc +README.zh.md: d838af346626bb96299a3b702df0dabb6231c15a diff --git a/packages/lsp/README.md b/packages/lsp/README.md index 74ed729d97..d8bc6047e7 100644 --- a/packages/lsp/README.md +++ b/packages/lsp/README.md @@ -8,9 +8,9 @@ The language-server capability seam: an abstract LSP interface, a generic stdio |---|---|---| | `lsp/` | Abstract LSP seam (provider registry by branded id + extension mapping, per-query selection, vocabulary, `LspError`) | `ctx.lsp` | | [`lsp-local/`](lsp-local/README.md) | Generic multi-server local backend (spawn, JSON-RPC, transient-open queries) | (registers providers on `ctx.lsp`) | -| [`lsp-e2b/`](lsp-e2b/README.md) | Remote E2B backend (remote source reads and servers, byte-framed stdio bridge) | (registers providers on `ctx.lsp`) | +| [`e2b/lsp-e2b`](../e2b/lsp-e2b/README.md) | Remote E2B backend (remote source reads and servers, byte-framed stdio bridge) | (registers providers on `ctx.lsp`) | | `tool-lsp/` | Model-facing `lsp` tool (four operations, one-based UTF-16 cursor coordinates) | (registers on `ctx.tools`) | The interface lives at `lsp/lsp/`. The seam exposes exactly four semantic operations — `goToDefinition`, `findReferences`, `goToImplementation`, `hover` — and no generic JSON-RPC escape hatch, so a provider swap does not change how the model asks for navigation and no protocol payload or unreviewed mutation reaches the model contract. Providers register **capabilities**, not tools; `tool-lsp` is the only owner of the model-facing name, schema, prompt guidance, and presentation. -See the [LSP capability seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md) for the protocol design and the [E2B extension note](../../.agents/notes/implemented/feature/2026-07-28-e2b-interactive-semantic-code-runtime-poc.md) for the remote process/filesystem boundary. +See the [LSP capability seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md) for the protocol design and the [shared E2B runtime note](../../.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md) for the remote process/filesystem boundary. diff --git a/packages/lsp/README.zh.md b/packages/lsp/README.zh.md index 5a08d54ca6..d838af3466 100644 --- a/packages/lsp/README.zh.md +++ b/packages/lsp/README.zh.md @@ -8,9 +8,9 @@ |---|---|---| | `lsp/` | 抽象 LSP seam(按品牌化 id + 扩展名映射组织的提供方注册表、逐查询选择、词汇、`LspError`) | `ctx.lsp` | | [`lsp-local/`](lsp-local/README.md) | 通用多服务器本地后端(spawn、JSON-RPC、临时打开查询) | (在 `ctx.lsp` 上注册提供方) | -| [`lsp-e2b/`](lsp-e2b/README.md) | 远程 E2B 后端(在远程读取源文件并运行服务器、采用字节分帧的 stdio 桥) | (在 `ctx.lsp` 上注册提供方) | +| [`e2b/lsp-e2b`](../e2b/lsp-e2b/README.md) | 远程 E2B 后端(在远程读取源文件并运行服务器、采用字节分帧的 stdio 桥) | (在 `ctx.lsp` 上注册提供方) | | `tool-lsp/` | 面向模型的 `lsp` 工具(四种操作、从 1 开始的 UTF-16 光标坐标) | (注册到 `ctx.tools`) | 接口位于 `lsp/lsp/`。该 seam 恰好公开四种语义操作:`goToDefinition`、`findReferences`、`goToImplementation`、`hover`,且不提供通用 JSON-RPC 逃生口;因此,替换提供方不会改变模型请求导航的方式,也不会让协议载荷或未经评审的修改进入模型契约。提供方注册的是**能力** 而非工具;`tool-lsp` 是面向模型名称、schema、提示词指引和呈现的唯一 owner。 -协议设计见 [LSP 能力 seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md),远程进程/文件系统边界见 [E2B 扩展 Agent Note](../../.agents/notes/implemented/feature/2026-07-28-e2b-interactive-semantic-code-runtime-poc.md)。 +协议设计见 [LSP 能力 seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md),远程进程/文件系统边界见 [共享 E2B 运行时 Agent Note](../../.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md)。 diff --git a/packages/pty/README.i18n.yaml b/packages/pty/README.i18n.yaml index c0fa873642..2dff12e3da 100644 --- a/packages/pty/README.i18n.yaml +++ b/packages/pty/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/pty/README.md -README.md: 239c751f30f8e5c543e6737e65a67f9bb22f79ad -README.zh.md: edaae7f948d957e2fce2b3f331cc2af61cfabd36 +README.md: a6706ed653bc60909a23b3598c41bb10ef499cbe +README.zh.md: 01fecf8d57d933168e91331c4d3c3e4666c13cdc diff --git a/packages/pty/README.md b/packages/pty/README.md index 239c751f30..a6706ed653 100644 --- a/packages/pty/README.md +++ b/packages/pty/README.md @@ -8,7 +8,7 @@ English | [中文](README.zh.md) |---|---|---| | [`pty`](pty/README.md) (`@deepseek-ai/dsh-pty`) | Backend registry, branded ids, exact-Agent ownership, session operations, and awaited cleanup | `ctx.pty` | | [`pty-local`](pty-local/README.md) (`@deepseek-ai/dsh-pty-local`) | Local `node-pty` backend, readiness detection, bounded terminal state, sandboxing, and process-session supervision | registers on `ctx.pty` | -| [`pty-e2b`](pty-e2b/README.md) (`@deepseek-ai/dsh-pty-e2b`) | E2B byte-PTY backend, remote foreground signaling, bounded terminal state, and awaited remote cleanup | registers on `ctx.pty` | +| [`e2b/pty-e2b`](../e2b/pty-e2b/README.md) (`@deepseek-ai/dsh-pty-e2b`) | E2B byte-PTY backend, remote foreground signaling, bounded terminal state, and awaited remote cleanup | registers on `ctx.pty` | | `tool-pty` (`@deepseek-ai/dsh-tool-pty`) | Six model-facing tools and generic task integration for background sends | registers on `ctx.tools` | -The core design lives in the [persistent PTY Agent Note](../../.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md); the remote ownership boundary lives in the [E2B extension note](../../.agents/notes/implemented/feature/2026-07-28-e2b-interactive-semantic-code-runtime-poc.md). +The core design lives in the [persistent PTY Agent Note](../../.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md); the remote ownership boundary lives in the [shared E2B runtime note](../../.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md). diff --git a/packages/pty/README.zh.md b/packages/pty/README.zh.md index edaae7f948..01fecf8d57 100644 --- a/packages/pty/README.zh.md +++ b/packages/pty/README.zh.md @@ -8,7 +8,7 @@ |---|---|---| | [`pty`](pty/README.md)(`@deepseek-ai/dsh-pty`) | 后端注册表、品牌化 id、精确的 Agent 所有权、会话操作与等待完成的清理 | `ctx.pty` | | [`pty-local`](pty-local/README.md)(`@deepseek-ai/dsh-pty-local`) | 本地 `node-pty` 后端、就绪检测、有界终端状态、沙箱与进程会话监管 | 注册到 `ctx.pty` | -| [`pty-e2b`](pty-e2b/README.md)(`@deepseek-ai/dsh-pty-e2b`) | E2B 字节 PTY 后端、远程前台信号传递、有界终端状态与等待完成的远程清理 | 注册到 `ctx.pty` | +| [`e2b/pty-e2b`](../e2b/pty-e2b/README.md)(`@deepseek-ai/dsh-pty-e2b`) | E2B 字节 PTY 后端、远程前台信号传递、有界终端状态与等待完成的远程清理 | 注册到 `ctx.pty` | | `tool-pty`(`@deepseek-ai/dsh-tool-pty`) | 6 个面向模型的工具,并为后台发送集成通用任务 | 注册到 `ctx.tools` | -核心设计记录在[持久 PTY Agent Note](../../.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md) 中;远程所有权边界记录在 [E2B 扩展 Agent Note](../../.agents/notes/implemented/feature/2026-07-28-e2b-interactive-semantic-code-runtime-poc.md) 中。 +核心设计记录在[持久 PTY Agent Note](../../.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md) 中;远程所有权边界记录在 [共享 E2B 运行时 Agent Note](../../.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md) 中。 diff --git a/packages/subprocess/README.i18n.yaml b/packages/subprocess/README.i18n.yaml index 0efaf7838e..4097420382 100644 --- a/packages/subprocess/README.i18n.yaml +++ b/packages/subprocess/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/subprocess/README.md -README.md: ae18c55205edd6085a0ed8de1bb7f875c411c79f -README.zh.md: e27fd240e4c1e96d8b859b326574e441241e8508 +README.md: cdea3803e903b31e895658745bc6dbf5a3c04c71 +README.zh.md: 95b02b6491077750f23ff12ffd14c1aa174609be diff --git a/packages/subprocess/README.md b/packages/subprocess/README.md index ae18c55205..cdea3803e9 100644 --- a/packages/subprocess/README.md +++ b/packages/subprocess/README.md @@ -8,6 +8,6 @@ The shared home for spawning managed child-process trees: fully-specified spawn |---|---|---| | [`subprocess`](subprocess/README.md) (`@deepseek-ai/dsh-subprocess`) | `ctx.subprocess` | The seam: abstract `SubprocessService.spawn(spec)`, the fully-explicit `SubprocessSpawnSpec` with per-stream stdio dispositions, `SubprocessHandle` (streams, offset-based readers, terminate/waitForExit/dispose), and the shared scrub + `DSH_*`/`CollectedOutput` vocabulary | | [`subprocess-local`](subprocess-local/README.md) (`@deepseek-ai/dsh-subprocess-local`) | — | The local implementation: detached process trees, per-disposition stream wiring, tail-keep truncation with bounded private spill files, the `DSH_*` merge order, tree signalling with escalation, the dispose ladder, and terminate-and-join disposal | -| [`subprocess-e2b`](subprocess-e2b/README.md) (`@deepseek-ai/dsh-subprocess-e2b`) | — | Experimental E2B implementation: remote Linux process groups and spill state in the shared `ctx.e2b` sandbox, with asynchronous PID acquisition and SDK buffering limitations | +| [`e2b/subprocess-e2b`](../e2b/subprocess-e2b/README.md) (`@deepseek-ai/dsh-subprocess-e2b`) | — | Experimental E2B implementation: remote Linux process groups and spill state in the shared `ctx.e2b` sandbox, with asynchronous PID acquisition and SDK buffering limitations | The service owns process lifetime across consumer reloads; consumers own what a process means (a bash command, a future non-shell runner) and every default that shapes one. diff --git a/packages/subprocess/README.zh.md b/packages/subprocess/README.zh.md index e27fd240e4..95b02b6491 100644 --- a/packages/subprocess/README.zh.md +++ b/packages/subprocess/README.zh.md @@ -8,6 +8,6 @@ spawn 受管子进程树的共用归属位置:完全显式的 spawn spec,其 |---|---|---| | [`subprocess`](subprocess/README.md)(`@deepseek-ai/dsh-subprocess`) | `ctx.subprocess` | seam 本体:抽象的 `SubprocessService.spawn(spec)`、完全显式且带按流划分 stdio 处置方式的 `SubprocessSpawnSpec`、`SubprocessHandle`(流、基于偏移量的读取器、terminate/waitForExit/dispose),以及共享的凭据清除 + `DSH_*`/`CollectedOutput` 词汇 | | [`subprocess-local`](subprocess-local/README.md)(`@deepseek-ai/dsh-subprocess-local`) | 无 | 本地实现:detached 进程树、按处置方式接线的流、附带有界私有 spill 文件的尾部保留截断、`DSH_*` 合并次序、带升级的进程树信号发送、dispose 阶梯,以及先终止再等待退出的 dispose | -| [`subprocess-e2b`](subprocess-e2b/README.md)(`@deepseek-ai/dsh-subprocess-e2b`) | 无 | 实验性 E2B 实现:远程 Linux 进程组和共享 `ctx.e2b` 沙箱中的 spill 状态,但 PID 异步获取,且受 SDK 缓冲限制 | +| [`e2b/subprocess-e2b`](../e2b/subprocess-e2b/README.md)(`@deepseek-ai/dsh-subprocess-e2b`) | 无 | 实验性 E2B 实现:远程 Linux 进程组和共享 `ctx.e2b` 沙箱中的 spill 状态,但 PID 异步获取,且受 SDK 缓冲限制 | 服务拥有跨消费方重载的进程存续期;消费方拥有一个进程的含义(一条 bash 命令、未来的非 shell 运行器)以及塑造它的每一项默认值。 diff --git a/scripts/verify-package-readme-model-experience.ts b/scripts/verify-package-readme-model-experience.ts index 5ab5dc2e4e..5ad70c0e68 100644 --- a/scripts/verify-package-readme-model-experience.ts +++ b/scripts/verify-package-readme-model-experience.ts @@ -47,7 +47,7 @@ const SENTENCE_MODEL_EXPERIENCE: Readonly> = { 'packages/bash/bash-local': { kind: 'indirect', reason: 'The executor backend delegates model rendering to dsh-tool-bash.' }, 'packages/bash/pwsh-local': { kind: 'indirect', reason: 'The executor backend delegates model rendering to dsh-tool-pwsh.' }, 'packages/code-runtime/code-runtime': { kind: 'indirect', reason: 'The service interface delegates model rendering to Code Mode in dsh-tools.' }, - 'packages/code-runtime/code-runtime-e2b': { kind: 'indirect', reason: 'The E2B backend delegates model rendering to Code Mode in dsh-tools.' }, + 'packages/e2b/code-runtime-e2b': { kind: 'indirect', reason: 'The E2B backend delegates model rendering to Code Mode in dsh-tools.' }, 'packages/code-runtime/code-runtime-worker': { kind: 'indirect', reason: 'The worker backend delegates model rendering to Code Mode in dsh-tools.' }, 'packages/typert/registry': { kind: 'none', reason: 'Runtime type registry; consumers (cordis_inspect, wire faces, gates) own any model-visible projection of registry contents.' }, 'packages/typert/loader': { kind: 'none', reason: 'Loader integration only registers generated artifacts; consumers own any model-visible projection.' }, @@ -84,7 +84,7 @@ const SENTENCE_MODEL_EXPERIENCE: Readonly> = { 'packages/client/web': { kind: 'none', reason: 'Browser-side UI plugin layer; registers no model surface.' }, 'packages/examples/agent-spine-demo': { kind: 'indirect', reason: 'The bundle only mounts model-facing child plugins.' }, 'packages/fs/fs': { kind: 'indirect', reason: 'The service interface delegates model rendering to dsh-tool-fs.' }, - 'packages/fs/fs-e2b': { kind: 'indirect', reason: 'The provider backend delegates model rendering to dsh-tool-fs.' }, + 'packages/e2b/fs-e2b': { kind: 'indirect', reason: 'The provider backend delegates model rendering to dsh-tool-fs.' }, 'packages/fs/fs-local': { kind: 'indirect', reason: 'The provider backend delegates model rendering to dsh-tool-fs.' }, 'packages/hooks/hook-protocol': { kind: 'indirect', reason: 'Only the hook bridge plugins render decoded hook output to a model.' }, 'packages/host/apiproxy': { kind: 'none', reason: 'The wire contract and fetch carriers move already-composed messages and register no model surface.' }, @@ -99,10 +99,10 @@ const SENTENCE_MODEL_EXPERIENCE: Readonly> = { 'packages/llm/llm': { kind: 'none', reason: 'The adapter registry forwards already-assembled requests unchanged.' }, 'packages/llm/token-meter': { kind: 'indirect', reason: 'The measurement service leaves model-visible changes to its consumers.' }, 'packages/lsp/lsp': { kind: 'indirect', reason: 'The provider registry delegates model rendering to dsh-tool-lsp.' }, - 'packages/lsp/lsp-e2b': { kind: 'indirect', reason: 'The E2B provider backend delegates model rendering to dsh-tool-lsp.' }, + 'packages/e2b/lsp-e2b': { kind: 'indirect', reason: 'The E2B provider backend delegates model rendering to dsh-tool-lsp.' }, 'packages/lsp/lsp-local': { kind: 'indirect', reason: 'The provider backend delegates model rendering to dsh-tool-lsp.' }, 'packages/subprocess/subprocess': { kind: 'indirect', reason: 'The seam delegates all model rendering to consumer seams such as the bash executor family.' }, - 'packages/subprocess/subprocess-e2b': { kind: 'indirect', reason: 'The remote spawn backend delegates model rendering to consumer seams such as the bash executor family.' }, + 'packages/e2b/subprocess-e2b': { kind: 'indirect', reason: 'The remote spawn backend delegates model rendering to consumer seams such as the bash executor family.' }, 'packages/subprocess/subprocess-local': { kind: 'indirect', reason: 'The spawn backend delegates model rendering to consumer seams such as the bash executor family.' }, 'packages/sandbox/sandbox-local': { kind: 'indirect', reason: 'The provider backend delegates model rendering to dsh-bash-sandbox and dsh-tool-bash.' }, 'packages/sdk/create-sdk': { kind: 'indirect', reason: 'The initializer only writes project files; selected runtime plugins provide the generated project model surface.' }, diff --git a/tsconfig.base.json b/tsconfig.base.json index 8380182e83..0673c35b90 100644 --- a/tsconfig.base.json +++ b/tsconfig.base.json @@ -85,6 +85,7 @@ "./packages/llm/*/src/invariant.ts", "./packages/bash/*/src/invariant.ts", "./packages/subprocess/*/src/invariant.ts", + "./packages/e2b/*/src/invariant.ts", "./packages/code-runtime/*/src/invariant.ts", "./packages/fs/*/src/invariant.ts", "./packages/skill/*/src/invariant.ts", diff --git a/tsconfig.host.json b/tsconfig.host.json index 1f70e79a4f..526d4ef4b8 100644 --- a/tsconfig.host.json +++ b/tsconfig.host.json @@ -163,15 +163,15 @@ { "path": "./packages/subprocess/subprocess" }, { "path": "./packages/subprocess/subprocess-local" }, { "path": "./packages/e2b/e2b" }, - { "path": "./packages/subprocess/subprocess-e2b" }, + { "path": "./packages/e2b/subprocess-e2b" }, { "path": "./packages/bash/bash" }, { "path": "./packages/pty/pty" }, - { "path": "./packages/pty/pty-e2b" }, + { "path": "./packages/e2b/pty-e2b" }, { "path": "./packages/pty/pty-local" }, { "path": "./packages/pty/tool-bash-persistent" }, { "path": "./packages/pty/tool-pty" }, { "path": "./packages/code-runtime/code-runtime" }, - { "path": "./packages/code-runtime/code-runtime-e2b" }, + { "path": "./packages/e2b/code-runtime-e2b" }, { "path": "./packages/code-runtime/code-runtime-worker" }, { "path": "./packages/llm/llm-deepseek" }, { "path": "./packages/llm/llm-pi-ai" }, @@ -187,7 +187,7 @@ { "path": "./packages/bash/tool-bash" }, { "path": "./packages/fs/fs" }, { "path": "./packages/fs/fs-local" }, - { "path": "./packages/fs/fs-e2b" }, + { "path": "./packages/e2b/fs-e2b" }, { "path": "./packages/fs/fs-policy" }, { "path": "./packages/fs/fs-sandbox" }, { "path": "./packages/fs/tool-fs" }, @@ -268,7 +268,7 @@ { "path": "./packages/sdk/create-sdk" }, { "path": "./packages/sdk/telemetry" }, { "path": "./packages/lsp/lsp" }, - { "path": "./packages/lsp/lsp-e2b" }, + { "path": "./packages/e2b/lsp-e2b" }, { "path": "./packages/lsp/lsp-local" }, { "path": "./packages/lsp/tool-lsp" }, { "path": "./apps/cli" } From 3dea36f1ce43d72413fb2b6e8fc3144b7e358d86 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Tue, 28 Jul 2026 16:26:20 +0800 Subject: [PATCH 066/108] fix(e2b): close remote lifecycle gaps --- .../tests/fixtures/e2b/e2b/bin.ts | 71 ++++++- .../tests/fixtures/e2b/e2b/cordis.yml | 4 +- packages/e2b/code-runtime-e2b/src/index.ts | 91 ++++++-- .../code-runtime-e2b/tests/runtime.spec.ts | 142 ++++++++++++- packages/e2b/e2b/src/frame.ts | 23 +- packages/e2b/e2b/src/index.ts | 8 +- packages/e2b/e2b/tests/composition.e2e.ts | 10 +- packages/e2b/e2b/tests/e2b.spec.ts | 25 +++ packages/e2b/fs-e2b/src/index.ts | 9 + packages/e2b/fs-e2b/tests/filesystem.spec.ts | 25 ++- packages/e2b/lsp-e2b/src/index.ts | 11 + packages/e2b/lsp-e2b/tests/provider.spec.ts | 5 + packages/e2b/pty-e2b/src/index.ts | 5 +- packages/e2b/pty-e2b/tests/index.spec.ts | 4 +- packages/e2b/subprocess-e2b/README.i18n.yaml | 4 +- packages/e2b/subprocess-e2b/README.md | 2 +- packages/e2b/subprocess-e2b/README.zh.md | 2 +- packages/e2b/subprocess-e2b/src/process.ts | 47 +++-- .../subprocess-e2b/tests/subprocess.spec.ts | 47 ++++- packages/lsp/lsp-local/src/index.ts | 199 ++++++++---------- packages/lsp/lsp-local/src/instance.ts | 13 +- packages/lsp/lsp-local/tests/instance.spec.ts | 2 + 22 files changed, 568 insertions(+), 181 deletions(-) diff --git a/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts b/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts index 33e9530708..89b09a6818 100644 --- a/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts +++ b/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts @@ -1,5 +1,5 @@ import { readFile } from 'node:fs/promises' -import { resolve } from 'node:path' +import { posix, resolve } from 'node:path' import { boot } from '@deepseek-ai/dsh-app-boot' import { AgentMessageId } from '@deepseek-ai/dsh-agent' import type { Agent } from '@deepseek-ai/dsh-agent' @@ -48,20 +48,68 @@ try { const fromBash = await ctx.fs.resolve('from-bash.txt') const fsRead = await ctx.fs.readText(fromBash) + const environmentHandle = ctx.subprocess.spawn({ + argv: ['env'], + cwd: process.cwd(), + stdio: { stdin: 'ignore', stdout: { maxBytes: 65_536 }, stderr: { maxBytes: 4_096 } }, + graceMs: 500, + env: { + 'FOO-BAR': 'hyphen-value', + DSH_EXPLICIT: 'managed-value', + TOKEN_EXPLICIT: 'credential-value', + }, + }) + const environmentOutcome = await environmentHandle.done + const environmentText = environmentHandle.collected.stdout?.readFrom(0).text + if (environmentOutcome.exitCode !== 0 || environmentText === undefined) { + throw new Error(`E2B subprocess environment probe failed: ${JSON.stringify(environmentOutcome)}`) + } + const environmentLines = new Set(environmentText.trimEnd().split('\n')) + const explicitEnvironment = [ + 'FOO-BAR=hyphen-value', + 'DSH_EXPLICIT=managed-value', + 'TOKEN_EXPLICIT=credential-value', + ].every(entry => environmentLines.has(entry)) + if (!explicitEnvironment) throw new Error(`E2B subprocess dropped an explicit environment entry: ${environmentText}`) + + const spillHandle = ctx.subprocess.spawn({ + argv: ['bash', '-c', "printf '0123456789'; sleep 30"], + cwd: process.cwd(), + stdio: { stdin: 'ignore', stdout: { maxBytes: 4, spill: { maxBytes: 6 } }, stderr: { maxBytes: 4_096 } }, + graceMs: 500, + env: {}, + }) + const spillReader = spillHandle.collected.stdout + if (spillReader === undefined) throw new Error('E2B subprocess omitted its configured stdout collector') + const spillDeadline = Date.now() + 5_000 + while (spillReader.readFrom(0).nextOffset < 10) { + if (Date.now() >= spillDeadline) throw new Error('E2B subprocess did not stream the spill probe output') + await new Promise(resolveDelay => setTimeout(resolveDelay, 20)) + } + const spillPath = posix.join((spillHandle as unknown as { stateDir: string }).stateDir, 'stdout.log') + const liveSpillBytes = (await (await ctx.e2b.getSandbox()).files.getInfo(spillPath)).size + spillHandle.terminate() + const spillOutcome = await spillHandle.done + const spillExited = await spillHandle.waitForExit(AbortSignal.timeout(5_000)) + const spillRead = spillReader.readFrom(0) + if (liveSpillBytes !== 6 || !spillExited || spillRead.spillPath !== undefined) { + throw new Error(`E2B subprocess spill bound failed: ${JSON.stringify({ liveSpillBytes, spillExited, spillRead })}`) + } + const lspFixture = await readFile(new URL('./fixture-lsp.mjs', import.meta.url), 'utf8') const remoteLspFixture = await ctx.fs.resolve('fixture-lsp.mjs') await ctx.fs.writeText(remoteLspFixture, lspFixture, { kind: 'createIfAbsent' }) - const remoteSource = await ctx.fs.resolve('multibyte.ts') + const remoteSource = await ctx.fs.resolve('multibyte # file.ts') await ctx.fs.writeText(remoteSource, 'const café = "你好"\nconsole.log(café)\n', { kind: 'createIfAbsent' }) const hover = await ctx.lsp.query({ operation: 'hover', - filePath: 'multibyte.ts', + filePath: 'multibyte # file.ts', position: { line: 0, character: 7 }, workspaceRoot: process.cwd(), }) const definition = await ctx.lsp.query({ operation: 'goToDefinition', - filePath: 'multibyte.ts', + filePath: 'multibyte # file.ts', position: { line: 0, character: 7 }, workspaceRoot: process.cwd(), }) @@ -143,6 +191,17 @@ try { }) setTimeout(() => { abortController.abort('live abort') }, 50) const aborted = await aborting + const oversizedBoot = await ctx.codeRuntime.run({ + program: `return ${JSON.stringify('x'.repeat(40_000))}`, + bindings: [], + }) + const oversizedReply = await ctx.codeRuntime.run({ + program: 'return await bridge.large(null)', + bindings: [{ + global: 'bridge', + functions: { large: async () => 'x'.repeat(40_000) }, + }], + }) const remoteProcesses = await (await ctx.e2b.getSandbox()).commands.list() const lingeringCodeRunners = remoteProcesses.filter(processInfo => JSON.stringify([processInfo.cmd, processInfo.args]).includes('code-runtime-runner.mjs'), @@ -152,6 +211,8 @@ try { sandboxId: await ctx.e2b.sandboxId, bashRead: bashRead.stdout.text, fsRead, + explicitEnvironment, + spill: { liveBytes: liveSpillBytes, outcome: spillOutcome, read: spillRead }, hover, definition, terminal: { @@ -165,6 +226,8 @@ try { hostileOutput, timedOut, aborted, + oversizedBoot, + oversizedReply, lingeringCodeRunners: lingeringCodeRunners.length, })}\n`) } finally { diff --git a/examples/headless-agent/tests/fixtures/e2b/e2b/cordis.yml b/examples/headless-agent/tests/fixtures/e2b/e2b/cordis.yml index f5cf403c4b..6b3350e49c 100644 --- a/examples/headless-agent/tests/fixtures/e2b/e2b/cordis.yml +++ b/examples/headless-agent/tests/fixtures/e2b/e2b/cordis.yml @@ -2,7 +2,7 @@ name: '@deepseek-ai/dsh-e2b' config: cwd: !!js process.cwd() - timeoutMs: 60000 + timeoutMs: 120000 onTimeout: kill onDispose: kill @@ -55,5 +55,5 @@ maxWallMs: 5000 maxOutputBytes: 4096 maxOldGenerationSizeMb: 128 - maxFrameBytes: 4194304 + maxFrameBytes: 32768 killGraceMs: 500 diff --git a/packages/e2b/code-runtime-e2b/src/index.ts b/packages/e2b/code-runtime-e2b/src/index.ts index fbfbdea91f..a567a2bb6d 100644 --- a/packages/e2b/code-runtime-e2b/src/index.ts +++ b/packages/e2b/code-runtime-e2b/src/index.ts @@ -14,7 +14,7 @@ import type { } from '@deepseek-ai/dsh-code-runtime' import { E2BFrameDecoder, - encodeE2BFrame, + encodeBoundedE2BFrame, quoteE2BShellArg, resolveE2BExecutable, } from '@deepseek-ai/dsh-e2b' @@ -25,6 +25,7 @@ import { } from '@deepseek-ai/dsh-code-runtime-worker' import type { WorkerJsonWire } from '@deepseek-ai/dsh-code-runtime-worker' import { snapshotJsonValue } from '@deepseek-ai/dsh-session' +import type { SubprocessHandle } from '@deepseek-ai/dsh-subprocess' import E2BSubprocessService from '@deepseek-ai/dsh-subprocess-e2b' import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout' import { CODE_RUNNER_SOURCE } from './runner-source.ts' @@ -46,6 +47,7 @@ export interface Config { } type ResolvedConfig = Required +type PreparedRuntime = { node: string; runner: string } interface LiveRun { settle(failure: CodeRunFailure): void @@ -130,7 +132,7 @@ export class E2BCodeRuntime extends CodeRuntime { readonly isolation = 'container' private readonly config: ResolvedConfig - private readonly ready: Promise<{ node: string; runner: string }> + private readonly ready: Promise private readonly live = new Set() private readonly subprocess: E2BSubprocessService private disposed = false @@ -176,25 +178,61 @@ export class E2BCodeRuntime extends CodeRuntime { } catch (error: unknown) { return this.failure({ kind: 'exception', message: messageOf(error) }) } - let runtime: Awaited + let runtime: PreparedRuntime | undefined try { - runtime = await this.ready + runtime = await this.awaitPreparation(request.signal) } catch (error: unknown) { + // Disposal can race the awaited setup despite the synchronous precheck. + // eslint-disable-next-line @typescript-eslint/no-unnecessary-condition + if (this.disposed) return this.failure({ kind: 'abort', message: 'runtime disposed' }) return this.failure({ kind: 'worker-exit', message: `E2B runtime setup failed: ${messageOf(error)}` }) } + if (runtime === undefined) { + return this.failure({ kind: 'abort', message: String(request.signal?.reason) }) + } // Disposal can race the awaited remote setup after the pre-await check. + /* v8 ignore start -- requires disposal between promise resolution and its awaiting continuation. */ // eslint-disable-next-line @typescript-eslint/no-unnecessary-condition if (this.disposed) return this.failure({ kind: 'abort', message: 'runtime disposed' }) + /* v8 ignore stop */ return await this.execute(request, code, bindings, runtime) } /* jscpd:ignore-end */ - private async prepare(): Promise<{ node: string; runner: string }> { + private awaitPreparation(signal: AbortSignal | undefined): Promise { + if (signal === undefined) return this.ready + return new Promise((resolve, reject) => { + const onAbort = (): void => { cleanup(); resolve(undefined) } + const cleanup = (): void => { signal.removeEventListener('abort', onAbort) } + signal.addEventListener('abort', onAbort, { once: true }) + if (signal.aborted) { + onAbort() + return + } + void this.ready.then( + (runtime) => { cleanup(); resolve(runtime) }, + (error: unknown) => { + cleanup() + reject(error instanceof Error ? error : new Error(String(error))) + }, + ) + }) + } + + private assertPreparationActive(): void { + if (this.disposed) throw new Error('code-runtime-e2b: runtime disposed during setup') + } + + private async prepare(): Promise { const sandbox = await this.ctx.e2b.getSandbox() + this.assertPreparationActive() const runner = posix.join(this.ctx.e2b.runtimeRoot, 'code-runtime-runner.mjs') await sandbox.files.write([{ path: runner, data: CODE_RUNNER_SOURCE }]) + this.assertPreparationActive() await sandbox.commands.run(`chmod 600 -- ${quoteE2BShellArg(runner)}`) + this.assertPreparationActive() const node = await resolveE2BExecutable(sandbox, 'node') + this.assertPreparationActive() return { node, runner } } @@ -237,16 +275,25 @@ export class E2BCodeRuntime extends CodeRuntime { request: CodeRunRequest, code: string, bindings: Map, - runtime: { node: string; runner: string }, + runtime: PreparedRuntime, ): Promise { - const handle = this.subprocess.spawn({ - argv: [runtime.node, runtime.runner], - cwd: this.ctx.e2b.cwd, - stdio: { stdin: 'pipe', stdout: 'pipe', stderr: { maxBytes: this.config.maxOutputBytes } }, - graceMs: this.config.killGraceMs, - ...request.signal === undefined ? {} : { signal: request.signal }, - env: {}, - }) + let handle: SubprocessHandle + try { + handle = this.subprocess.spawn({ + argv: [runtime.node, runtime.runner], + cwd: this.ctx.e2b.cwd, + stdio: { stdin: 'pipe', stdout: 'pipe', stderr: { maxBytes: this.config.maxOutputBytes } }, + graceMs: this.config.killGraceMs, + ...request.signal === undefined ? {} : { signal: request.signal }, + env: {}, + }) + } catch (error: unknown) { + if (this.disposed) return this.failure({ kind: 'abort', message: 'runtime disposed' }) + if (request.signal?.aborted === true) { + return this.failure({ kind: 'abort', message: String(request.signal.reason) }) + } + return this.failure({ kind: 'worker-exit', message: `E2B runtime spawn failed: ${messageOf(error)}` }) + } if (handle.stdin === undefined || handle.stdout === undefined) { handle.terminate() await Promise.allSettled([handle.done]) @@ -279,7 +326,6 @@ export class E2BCodeRuntime extends CodeRuntime { settled = true clearTimeout(wallTimer.current) request.signal?.removeEventListener('abort', onAbort) - this.live.delete(live) void new Promise((resume) => { setImmediate(resume) }).then(async () => { handle.terminate() await handle.done.catch(() => {}) @@ -298,6 +344,7 @@ export class E2BCodeRuntime extends CodeRuntime { result = output.failure(logs, { kind: 'worker-exit', message: `E2B runtime cleanup failed: ${messageOf(cleanupError)}` }) } const final = typeof result === 'function' ? result() : result + this.live.delete(live) finishResolve() resolve(final) }) @@ -305,7 +352,14 @@ export class E2BCodeRuntime extends CodeRuntime { const sendReply = (message: unknown): void => { if (settled) return - stdin.write(encodeE2BFrame(message), (error?: Error | null) => { + let frame: string + try { + frame = encodeBoundedE2BFrame(message, this.config.maxFrameBytes) + } catch (error: unknown) { + finish(() => output.failure(logs, { kind: 'worker-exit', message: `E2B runtime bridge failed: ${messageOf(error)}` })) + return + } + stdin.write(frame, (error?: Error | null) => { if (error !== undefined && error !== null) { finish(() => output.failure(logs, { kind: 'worker-exit', message: `E2B runtime bridge write failed: ${error.message}` })) } @@ -433,7 +487,10 @@ export class E2BCodeRuntime extends CodeRuntime { this.disposed = true const runs = [...this.live] for (const run of runs) run.settle({ kind: 'abort', message: 'runtime disposed' }) - await Promise.all(runs.map(run => run.finished)) + await Promise.all([ + this.ready.then(() => {}, () => {}), + ...runs.map(run => run.finished), + ]) } /* jscpd:ignore-end */ } diff --git a/packages/e2b/code-runtime-e2b/tests/runtime.spec.ts b/packages/e2b/code-runtime-e2b/tests/runtime.spec.ts index ff1c5dd84a..9824fb2cf0 100644 --- a/packages/e2b/code-runtime-e2b/tests/runtime.spec.ts +++ b/packages/e2b/code-runtime-e2b/tests/runtime.spec.ts @@ -34,13 +34,22 @@ class FakeHandle implements SubprocessHandle { waitCalls = 0 private readonly decoder = new E2BFrameDecoder(10_000_000) private readonly waitError: Error | undefined + private readonly waitResult: Promise | undefined private settled = false constructor( private readonly onMessage: (message: unknown, handle: FakeHandle) => void = () => {}, - options: { stdin?: boolean; stdout?: boolean; stderr?: string; writeError?: Error; waitError?: Error } = {}, + options: { + stdin?: boolean + stdout?: boolean + stderr?: string + writeError?: Error + waitError?: Error + waitResult?: Promise + } = {}, ) { this.waitError = options.waitError + this.waitResult = options.waitResult this.stdin = options.stdin === false ? undefined : options.writeError === undefined @@ -89,6 +98,7 @@ class FakeHandle implements SubprocessHandle { async waitForExit(): Promise { this.waitCalls += 1 if (this.waitError !== undefined) throw this.waitError + if (this.waitResult !== undefined) return await this.waitResult return true } } @@ -286,6 +296,30 @@ describe('E2BCodeRuntime', () => { await fixture.fiber.dispose() }) + it('enforces the outbound frame bound on boot and binding replies', async () => { + const oversizedBoot = new FakeHandle() + const oversizedReply = new FakeHandle((message, current) => { + if ((message as { type?: string }).type === 'boot') { + current.emit({ type: 'call', id: 1, global: 'bridge', name: 'large', args: encodeWorkerJson(null) }) + } + }) + const fixture = await setup([oversizedBoot, oversizedReply], { maxOutputBytes: 128, maxFrameBytes: 512 }) + + const bootResult = await fixture.runtime.run(request(`return ${JSON.stringify('x'.repeat(1_000))}`)) + expect(bootResult.error).toMatchObject({ kind: 'worker-exit' }) + expect(bootResult.error?.message).toContain('frame exceeded its byte limit') + expect(oversizedBoot.writes).toHaveLength(0) + + const replyResult = await fixture.runtime.run({ + program: 'return await bridge.large(null)', + bindings: [{ global: 'bridge', functions: { large: async () => 'x'.repeat(1_000) } }], + }) + expect(replyResult.error).toMatchObject({ kind: 'worker-exit' }) + expect(replyResult.error?.message).toContain('frame exceeded its byte limit') + expect(oversizedReply.writes).toHaveLength(1) + await fixture.fiber.dispose() + }) + it('contains stdin errors, process exits, spawn failures, and missing pipes', async () => { const writeError = new FakeHandle(() => {}, { writeError: new Error('write callback broke') }) const stdinError = new FakeHandle((message, current) => { @@ -445,12 +479,116 @@ describe('E2BCodeRuntime', () => { const gate = Promise.withResolvers() const fixture = await setup([], {}, {}, () => gate.promise) const running = fixture.runtime.run(request()) - await (fixture.runtime as unknown as { teardown(): Promise }).teardown() + const disposing = fixture.fiber.dispose() + let disposed = false + void disposing.then(() => { disposed = true }) + await new Promise(resolve => setImmediate(resolve)) + const disposedBeforeSetup = disposed gate.resolve(fixture.sandbox) + await disposing + expect(disposedBeforeSetup).toBe(false) expect((await running).error).toEqual({ kind: 'abort', message: 'runtime disposed' }) + expect(fixture.write).not.toHaveBeenCalled() + }) + + it('observes abort while runtime preparation is pending', async () => { + const gate = Promise.withResolvers() + const fixture = await setup([], {}, {}, () => gate.promise) + const controller = new AbortController() + const running = fixture.runtime.run({ ...request(), signal: controller.signal }) + + controller.abort('stop during setup') + const early = await Promise.race([ + running.then(result => ({ kind: 'result' as const, result })), + new Promise<{ kind: 'pending' }>((resolve) => { setImmediate(() => { resolve({ kind: 'pending' }) }) }), + ]) + expect(fixture.spawn).not.toHaveBeenCalled() + + gate.resolve(fixture.sandbox) + expect(early).toMatchObject({ kind: 'result', result: { error: { kind: 'abort', message: 'stop during setup' } } }) + await running await fixture.fiber.dispose() }) + it('classifies an abort that races synchronous subprocess spawn', async () => { + const fixture = await setup() + const controller = new AbortController() + fixture.spawn.mockImplementationOnce(() => { + controller.abort('stop at spawn') + throw new Error('aborted before spawn') + }) + + expect((await fixture.runtime.run({ ...request(), signal: controller.signal })).error) + .toEqual({ kind: 'abort', message: 'stop at spawn' }) + + fixture.spawn.mockImplementationOnce(() => { throw new Error('synchronous spawn failure') }) + expect((await fixture.runtime.run(request())).error).toEqual({ + kind: 'worker-exit', + message: 'E2B runtime spawn failed: synchronous spawn failure', + }) + await fixture.fiber.dispose() + + const disposingFixture = await setup() + disposingFixture.spawn.mockImplementationOnce(() => { + void (disposingFixture.runtime as unknown as { teardown(): Promise }).teardown() + throw new Error('spawn raced disposal') + }) + expect((await disposingFixture.runtime.run(request())).error) + .toEqual({ kind: 'abort', message: 'runtime disposed' }) + await disposingFixture.fiber.dispose() + }) + + it('closes both abort races around runtime readiness and live-run publication', async () => { + let preparationAborted = false + const preparationSignal = { + get aborted() { return preparationAborted }, + reason: 'preparation race', + addEventListener() { preparationAborted = true }, + removeEventListener() {}, + } as unknown as AbortSignal + const liveHandle = new FakeHandle() + const fixture = await setup([liveHandle]) + expect((await fixture.runtime.run({ ...request(), signal: preparationSignal })).error) + .toEqual({ kind: 'abort', message: 'preparation race' }) + expect(fixture.spawn).not.toHaveBeenCalled() + + let liveAborted = false + let registrations = 0 + const liveSignal = { + get aborted() { return liveAborted }, + reason: 'live publication race', + addEventListener() { + registrations += 1 + if (registrations === 2) liveAborted = true + }, + removeEventListener() {}, + } as unknown as AbortSignal + expect((await fixture.runtime.run({ ...request(), signal: liveSignal })).error) + .toEqual({ kind: 'abort', message: 'live publication race' }) + await fixture.fiber.dispose() + }) + + it('retains a live run until remote cleanup reaches quiescence', async () => { + const cleanup = Promise.withResolvers() + const handle = new FakeHandle((message, current) => { + if ((message as { type?: string }).type === 'boot') current.emit({ type: 'done' }) + }, { waitResult: cleanup.promise }) + const fixture = await setup([handle]) + const running = fixture.runtime.run(request()) + await vi.waitFor(() => { expect(handle.waitCalls).toBe(1) }) + + const disposing = fixture.fiber.dispose() + let disposed = false + void disposing.then(() => { disposed = true }) + await new Promise(resolve => setImmediate(resolve)) + const disposedBeforeCleanup = disposed + + cleanup.resolve(true) + await expect(running).resolves.toEqual({ logs: [] }) + await expect(disposing).resolves.toBeUndefined() + expect(disposedBeforeCleanup).toBe(false) + }) + it('registers the package-owned invariant companion', async () => { const ctx = new Context() await ctx.plugin(InvariantService, { enabled: true }) diff --git a/packages/e2b/e2b/src/frame.ts b/packages/e2b/e2b/src/frame.ts index 39800b4428..5b6fe536c0 100644 --- a/packages/e2b/e2b/src/frame.ts +++ b/packages/e2b/e2b/src/frame.ts @@ -10,9 +10,30 @@ const BASE64_LINE = /^(?:[A-Za-z0-9+/]{4})*(?:[A-Za-z0-9+/]{2}==|[A-Za-z0-9+/]{3 * @returns Base64-encoded UTF-8 JSON followed by one newline. */ export function encodeE2BFrame(value: unknown): string { + return encodeFrame(value) +} + +/** + * Encode one JSON-compatible value while enforcing the decoded frame bound. + * @param value - Value accepted by `JSON.stringify`. + * @param maxFrameBytes - Maximum UTF-8 JSON bytes in the encoded frame. + * @returns Base64-encoded UTF-8 JSON followed by one newline. + */ +export function encodeBoundedE2BFrame(value: unknown, maxFrameBytes: number): string { + if (!Number.isSafeInteger(maxFrameBytes) || maxFrameBytes <= 0) { + throw new Error('E2B frame maxFrameBytes must be a positive safe integer') + } + return encodeFrame(value, maxFrameBytes) +} + +function encodeFrame(value: unknown, maxFrameBytes?: number): string { const json: unknown = JSON.stringify(value) if (typeof json !== 'string') throw new Error('E2B frame value is not JSON-serializable') - return `${Buffer.from(json).toString('base64')}\n` + const bytes = Buffer.from(json) + if (maxFrameBytes !== undefined && bytes.length > maxFrameBytes) { + throw new Error('E2B frame exceeded its byte limit') + } + return `${bytes.toString('base64')}\n` } /** Incremental decoder for newline-delimited base64 JSON frames. */ diff --git a/packages/e2b/e2b/src/index.ts b/packages/e2b/e2b/src/index.ts index 4c4597f6ef..11625b3b21 100644 --- a/packages/e2b/e2b/src/index.ts +++ b/packages/e2b/e2b/src/index.ts @@ -10,7 +10,7 @@ import z from 'schemastery' import { Sandbox } from 'e2b' import type { Branded } from '@deepseek-ai/dsh-brand' -export { E2BFrameDecoder, encodeE2BFrame } from './frame.ts' +export { E2BFrameDecoder, encodeBoundedE2BFrame, encodeE2BFrame } from './frame.ts' export { CommandExitError, @@ -202,7 +202,11 @@ export class E2BSandboxService extends Service { */ async getSandbox(): Promise { if (this.disposed) throw new Error('E2B sandbox service is disposing') - return await this.ready + const sandbox = await this.ready + // Disposal can race the awaited sandbox readiness despite the synchronous precheck. + // eslint-disable-next-line @typescript-eslint/no-unnecessary-condition + if (this.disposed) throw new Error('E2B sandbox service is disposing') + return sandbox } private validate(): void { diff --git a/packages/e2b/e2b/tests/composition.e2e.ts b/packages/e2b/e2b/tests/composition.e2e.ts index d056d4b851..5c9bd56fc9 100644 --- a/packages/e2b/e2b/tests/composition.e2e.ts +++ b/packages/e2b/e2b/tests/composition.e2e.ts @@ -24,7 +24,7 @@ describe.skipIf(!process.env.E2B_API_KEY)('E2B live Loader composition', () => { }, processTimeoutMs: 120_000, inspect: async (cwd) => { - for (const name of ['from-fs.txt', 'from-bash.txt', 'multibyte.ts', 'fixture-lsp.mjs']) { + for (const name of ['from-fs.txt', 'from-bash.txt', 'multibyte # file.ts', 'fixture-lsp.mjs']) { await expect(access(join(cwd, name))).rejects.toMatchObject({ code: 'ENOENT' }) } }, @@ -35,6 +35,12 @@ describe.skipIf(!process.env.E2B_API_KEY)('E2B live Loader composition', () => { expect(output).toMatchObject({ bashRead: 'written-by-fs\n', fsRead: 'written-by-bash\n', + explicitEnvironment: true, + spill: { + liveBytes: 6, + outcome: { exitCode: null, signal: 'SIGTERM' }, + read: { text: '6789', nextOffset: 10, lossy: true }, + }, hover: { kind: 'hover', hover: { contents: '**remote hover** 你好 café' }, @@ -51,6 +57,8 @@ describe.skipIf(!process.env.E2B_API_KEY)('E2B live Loader composition', () => { hostileOutput: { error: { kind: 'output-limit' } }, timedOut: { error: { kind: 'timeout' } }, aborted: { error: { kind: 'abort', message: 'live abort' } }, + oversizedBoot: { error: { kind: 'worker-exit' } }, + oversizedReply: { error: { kind: 'worker-exit' } }, lingeringCodeRunners: 0, }) expect((output.terminal as { motd: string }).motd.length).toBeGreaterThan(0) diff --git a/packages/e2b/e2b/tests/e2b.spec.ts b/packages/e2b/e2b/tests/e2b.spec.ts index 541e69df9c..58d2ffbd7a 100644 --- a/packages/e2b/e2b/tests/e2b.spec.ts +++ b/packages/e2b/e2b/tests/e2b.spec.ts @@ -4,6 +4,7 @@ import type { Sandbox as SandboxType } from 'e2b' import E2BSandboxService, { E2BFrameDecoder, E2BSandboxId, + encodeBoundedE2BFrame, encodeE2BFrame, quoteE2BShellArg, resolveE2BExecutable, @@ -91,6 +92,22 @@ describe('E2BSandboxService', () => { await expect(service.getSandbox()).rejects.toThrow(/disposing/) }) + it('rejects handle acquisition when disposal starts during setup', async () => { + const fixture = fakeSandbox() + const opening = Promise.withResolvers() + sdk.create.mockReturnValue(opening.promise) + const ctx = new Context() + const fiber = await ctx.plugin(E2BSandboxService, { apiKey: 'test-key' }) + + const acquisition = ctx.e2b.getSandbox() + const disposing = fiber.dispose() + opening.resolve(fixture.sandbox) + + await expect(acquisition).rejects.toThrow(/disposing/) + await expect(disposing).resolves.toBeUndefined() + expect(fixture.kill).toHaveBeenCalledOnce() + }) + it('creates from a template, honors timeout and pause policies, and reads the key from the environment', async () => { vi.stubEnv('E2B_API_KEY', 'environment-key') const fixture = fakeSandbox('template-sandbox') @@ -240,6 +257,14 @@ describe('E2B helpers and invariant companion', () => { expect(() => encodeE2BFrame(undefined)).toThrow('not JSON-serializable') }) + it('bounds outbound frames by decoded UTF-8 bytes', () => { + const exact = encodeBoundedE2BFrame({ text: '你' }, 14) + expect(new E2BFrameDecoder(14).push(exact)).toEqual([{ text: '你' }]) + expect(() => encodeBoundedE2BFrame({ text: '你' }, 13)).toThrow('byte limit') + expect(() => encodeBoundedE2BFrame(null, 0)).toThrow('positive safe integer') + expect(() => encodeBoundedE2BFrame(null, 1.5)).toThrow('positive safe integer') + }) + it('rejects malformed, oversized, and truncated frame streams', () => { expect(() => new E2BFrameDecoder(0)).toThrow('positive safe integer') expect(() => new E2BFrameDecoder(1.5)).toThrow('positive safe integer') diff --git a/packages/e2b/fs-e2b/src/index.ts b/packages/e2b/fs-e2b/src/index.ts index 020b0f4857..4f733f1fba 100644 --- a/packages/e2b/fs-e2b/src/index.ts +++ b/packages/e2b/fs-e2b/src/index.ts @@ -199,6 +199,7 @@ export class E2BFileSystem extends FileSystem { const reader = stream.getReader() const decoder = new TextDecoder('utf-8', { fatal: true }) let sampledBytes = 0 + let completed = false try { while (true) { assertNotAborted(signal, 'read') @@ -222,9 +223,17 @@ export class E2BFileSystem extends FileSystem { } catch (error: unknown) { throw new FsError(`cannot read "${displayPath}": invalid UTF-8 text`, 'FS_NOT_TEXT', { cause: error }) } + completed = true } catch (error: unknown) { throw mapError(error, 'read', displayPath, signal) } finally { + if (!completed) { + try { + await reader.cancel() + } catch (_streamCancellationFailure) { + // The primary read outcome owns the result; cancellation is best-effort after early stop. + } + } reader.releaseLock() } }, diff --git a/packages/e2b/fs-e2b/tests/filesystem.spec.ts b/packages/e2b/fs-e2b/tests/filesystem.spec.ts index b8eb5447d6..177bb9e59c 100644 --- a/packages/e2b/fs-e2b/tests/filesystem.spec.ts +++ b/packages/e2b/fs-e2b/tests/filesystem.spec.ts @@ -12,7 +12,7 @@ import { FsVersion } from '@deepseek-ai/dsh-fs' import E2BFileSystem from '@deepseek-ai/dsh-fs-e2b' import * as E2BFsInvariant from '../src/invariant.ts' import InvariantService from '@deepseek-ai/dsh-invariants' -import { describe, expect, it } from 'vitest' +import { describe, expect, it, vi } from 'vitest' interface RemoteNode { type: FileType @@ -38,6 +38,8 @@ class FakeRemote { readonly removals: string[] = [] readonly commands: string[] = [] streamChunks: Uint8Array[] | undefined + streamKeepOpen = false + readonly streamCancel = vi.fn() nextCommandError: unknown nextInfoError: unknown nextListError: unknown @@ -148,10 +150,11 @@ class FakeRemote { if (options.format === 'bytes') return data.slice() const chunks = this.streamChunks ?? [data.slice()] return new ReadableStream({ - start(controller) { + start: (controller) => { for (const chunk of chunks) controller.enqueue(chunk) - controller.close() + if (!this.streamKeepOpen) controller.close() }, + cancel: () => { this.streamCancel() }, }) }, list: async (path: string, options?: { depth?: number; signal?: AbortSignal }): Promise => { @@ -303,6 +306,22 @@ describe('E2BFileSystem identity, metadata, and reads', () => { expect(initiallyBuffered).toBe('€') }) + it('cancels a remote stream when its consumer stops early', async () => { + const remote = new FakeRemote() + remote.file('/workspace/text.txt', 'ab') + remote.streamChunks = [bytes('a'), bytes('b')] + remote.streamKeepOpen = true + const { fs } = await setup(remote) + const stream = await fs.streamText(await fs.resolve('text.txt')) + + for await (const chunk of stream) { + expect(chunk).toBe('a') + break + } + + expect(remote.streamCancel).toHaveBeenCalledOnce() + }) + it('matches local binary sampling while edits still reject any NUL byte', async () => { const remote = new FakeRemote() remote.file('/workspace/late-nul.txt', `${'a'.repeat(8192)}\0tail`) diff --git a/packages/e2b/lsp-e2b/src/index.ts b/packages/e2b/lsp-e2b/src/index.ts index eb9596db5f..b378ebfc9a 100644 --- a/packages/e2b/lsp-e2b/src/index.ts +++ b/packages/e2b/lsp-e2b/src/index.ts @@ -200,6 +200,16 @@ export async function readE2BSource( return { canonicalPath, text } } +/** + * Encode one absolute remote Linux path as a host-independent file URI. + * @param path - Canonical POSIX path inside E2B. + * @returns The equivalent percent-encoded file URI. + */ +export function e2bFileUri(path: string): string { + if (!posix.isAbsolute(path)) throw new Error(`lsp-e2b: expected an absolute remote path, received ${JSON.stringify(path)}`) + return `file://${path.split('/').map(segment => encodeURIComponent(segment)).join('/')}` +} + /* jscpd:ignore-start -- Provider identity mirrors the seam while remote source and process ownership stay local. */ /** One pooled remote provider with an isolated server per canonical workspace. */ export class E2BLspProvider implements LspProvider { @@ -298,6 +308,7 @@ export class E2BLspProvider implements LspProvider { shutdownTimeoutMs: this.config.shutdownTimeoutMs, killGraceMs: this.config.killGraceMs, clientProcessId: null, + pathToFileUri: e2bFileUri, }, (spec: SubprocessSpawnSpec) => { const originalArgv = Buffer.from(JSON.stringify(spec.argv)).toString('base64') const inner = this.subprocess.spawn({ diff --git a/packages/e2b/lsp-e2b/tests/provider.spec.ts b/packages/e2b/lsp-e2b/tests/provider.spec.ts index 0b41be12a3..d746abe7f7 100644 --- a/packages/e2b/lsp-e2b/tests/provider.spec.ts +++ b/packages/e2b/lsp-e2b/tests/provider.spec.ts @@ -69,6 +69,7 @@ import { E2BLspProvider, apply, canonicalizeE2BWorkspace, + e2bFileUri, readE2BSource, } from '@deepseek-ai/dsh-lsp-e2b' import type { LspE2BServerConfig } from '@deepseek-ai/dsh-lsp-e2b' @@ -234,6 +235,10 @@ describe('E2BLspProvider pooling and lifecycle', () => { await expect(current.query(query())).resolves.toEqual({ kind: 'hover', hover: { contents: 'ok' } }) expect(mockedLsp.FakeLspInstance.instances).toHaveLength(1) expect(mockedLsp.FakeLspInstance.instances[0]?.spec).toMatchObject({ clientProcessId: null, cwd: '/workspace' }) + expect(e2bFileUri('/workspace/a b#c.ts')).toBe('file:///workspace/a%20b%23c.ts') + expect(() => e2bFileUri('relative.ts')).toThrow('absolute remote path') + const pathToFileUri = mockedLsp.FakeLspInstance.instances[0]?.spec.pathToFileUri as (path: string) => string + expect(pathToFileUri('/workspace/a b#c.ts')).toBe('file:///workspace/a%20b%23c.ts') expect(spawn).toHaveBeenCalledWith(expect.objectContaining({ argv: ['/usr/bin/node', '/workspace/.dsh-e2b/lsp-proxy.mjs', expect.any(String)], stdio: { stdin: 'pipe', stdout: 'pipe', stderr: { maxBytes: 128 } }, diff --git a/packages/e2b/pty-e2b/src/index.ts b/packages/e2b/pty-e2b/src/index.ts index fca16cf5c7..8fd7a519a0 100644 --- a/packages/e2b/pty-e2b/src/index.ts +++ b/packages/e2b/pty-e2b/src/index.ts @@ -71,9 +71,10 @@ export class E2BPtyBackend implements PtyBackend { } const session = new E2BPtySession(sandbox, handle, this.config) created.session = session - for (const data of pending) session.onData(data) try { - await session.initialize(spec.signal) + const initializing = session.initialize(spec.signal) + for (const data of pending) session.onData(data) + await initializing return session } catch (error: unknown) { try { diff --git a/packages/e2b/pty-e2b/tests/index.spec.ts b/packages/e2b/pty-e2b/tests/index.spec.ts index 567b3b9215..e306d8b86b 100644 --- a/packages/e2b/pty-e2b/tests/index.spec.ts +++ b/packages/e2b/pty-e2b/tests/index.spec.ts @@ -52,7 +52,7 @@ describe('E2BPtyBackend and plugin', () => { const backend = new E2BPtyBackend(ctx, config(), async (_sandbox, received) => { options = received void received.onData(Buffer.from('banner\n')) - setTimeout(() => { void received.onData(Buffer.from('\x1b]133;D;0\x07dsh> ')) }, 0) + void received.onData(Buffer.from('\x1b]133;D;0\x07dsh> ')) return created }) const pending = backend.spawn({ @@ -62,7 +62,7 @@ describe('E2BPtyBackend and plugin', () => { await vi.advanceTimersByTimeAsync(2) const session = await pending - expect(session.motd).toBe('dsh> ') + expect(session.motd).toBe('banner\ndsh> ') expect(options).toMatchObject({ rows: 24, cols: 80, cwd: '/workspace/project', timeoutMs: 0 }) expect(options?.envs).toMatchObject({ TERM: 'dumb', PAGER: 'cat', GIT_PAGER: 'cat', PS1: 'dsh> ', diff --git a/packages/e2b/subprocess-e2b/README.i18n.yaml b/packages/e2b/subprocess-e2b/README.i18n.yaml index 26bc8f0b51..8b8a70838c 100644 --- a/packages/e2b/subprocess-e2b/README.i18n.yaml +++ b/packages/e2b/subprocess-e2b/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/e2b/subprocess-e2b/README.md -README.md: ce61de1100791be6e5c4db74c73ff43566281ed9 -README.zh.md: a4c619f0c002cc1d36310c5a9b4a3f7ae655ad1f +README.md: 3b3bfa88e7e6483decfcdec11355942ae4ff7403 +README.zh.md: 3ff9a51c60636dea5789f9dd11b04aa902b91d0c diff --git a/packages/e2b/subprocess-e2b/README.md b/packages/e2b/subprocess-e2b/README.md index ce61de1100..3b3bfa88e7 100644 --- a/packages/e2b/subprocess-e2b/README.md +++ b/packages/e2b/subprocess-e2b/README.md @@ -11,7 +11,7 @@ E2B implementation of the [`@deepseek-ai/dsh-subprocess`](../../subprocess/subpr - **Environment boundary** — the wrapper starts from the sandbox command environment, removes ambient `DSH_*` and credential-shaped (`*KEY*`, `*SECRET*`, `*TOKEN*`) names, then restores every `spec.env` entry as an explicit caller opt-in. Host ambient variables never enter the sandbox implicitly. - **Stdio projection** — pipe mode forwards E2B callbacks into host Node streams; inherit mode forwards them to the harness process streams; collect mode retains a bounded host tail with offset reads. Optional complete spill files are written remotely and advertised only while within their cap. Batch and streaming stdin use the SDK handle. -The base E2B image supplies the Bash/GNU utilities this adapter invokes: `bash`, `setsid`, `ps`, `tr`, `env`, `chmod`, `tee`, and `kill`. A custom template must retain compatible commands. +The base E2B image supplies the Bash/GNU utilities this adapter invokes: `bash`, `setsid`, `ps`, `tr`, `env`, `chmod`, `tee`, `head`, and `kill`. A custom template must retain compatible commands. ## Model Experience diff --git a/packages/e2b/subprocess-e2b/README.zh.md b/packages/e2b/subprocess-e2b/README.zh.md index a4c619f0c0..3ff9a51c60 100644 --- a/packages/e2b/subprocess-e2b/README.zh.md +++ b/packages/e2b/subprocess-e2b/README.zh.md @@ -11,7 +11,7 @@ - **环境边界**:包装层从沙箱命令环境开始,移除环境中的 `DSH_*` 和形似凭据的名称(`*KEY*`、`*SECRET*`、`*TOKEN*`),再把每个 `spec.env` 条目恢复为调用方显式选择。宿主环境变量绝不会隐式进入沙箱。 - **stdio 投影**:pipe 模式把 E2B 回调转发到宿主 Node 流;inherit 模式把回调转发到 harness 进程流;collect 模式保留有界的宿主尾部,并支持基于偏移量读取。可选的完整 spill 文件写在远程,并且只有未超过其上限时才会对外公布。批量 stdin 和流式 stdin 都使用 SDK 句柄。 -基础 E2B 镜像提供该适配器调用的 Bash/GNU 工具:`bash`、`setsid`、`ps`、`tr`、`env`、`chmod`、`tee` 和 `kill`。自定义模板必须保留兼容的命令。 +基础 E2B 镜像提供该适配器调用的 Bash/GNU 工具:`bash`、`setsid`、`ps`、`tr`、`env`、`chmod`、`tee`、`head` 和 `kill`。自定义模板必须保留兼容的命令。 ## 模型体验 diff --git a/packages/e2b/subprocess-e2b/src/process.ts b/packages/e2b/subprocess-e2b/src/process.ts index 507ea700c4..86bdade589 100644 --- a/packages/e2b/subprocess-e2b/src/process.ts +++ b/packages/e2b/subprocess-e2b/src/process.ts @@ -55,37 +55,38 @@ class DeferredStdin extends Writable { interface RemotePaths { pid: string status: string + environment: string stdout: string stderr: string } -function explicitEnvironmentNames(env: Readonly> | undefined): string { - return Object.keys(env ?? {}) - .map(quoteE2BShellArg) - .join(' ') +function explicitEnvironment(env: Readonly> | undefined): string { + return Object.entries(env ?? {}) + .map(([name, value]) => `${name}=${value}\0`) + .join('') } function commandText(spec: SubprocessSpawnSpec, paths: RemotePaths): string { const stdoutRedirect = hasSpill(spec.stdio.stdout) - ? `> >(tee -a -- ${quoteE2BShellArg(paths.stdout)})` + ? `> >(tee --output-error=warn-nopipe >(head -c ${spec.stdio.stdout.spill.maxBytes} > ${quoteE2BShellArg(paths.stdout)}))` : '' const stderrRedirect = hasSpill(spec.stdio.stderr) - ? `2> >(tee -a -- ${quoteE2BShellArg(paths.stderr)} >&2)` + ? `2> >(tee --output-error=warn-nopipe >(head -c ${spec.stdio.stderr.spill.maxBytes} > ${quoteE2BShellArg(paths.stderr)}) >&2)` : '' - const environmentNames = explicitEnvironmentNames(spec.env) const inner = [ 'set +e', 'umask 077', 'dsh_e2b_pgid="$(ps -o pgid= -p "$$" | tr -d " ")"', `printf '%s\\n' "$dsh_e2b_pgid" > ${quoteE2BShellArg(paths.pid)}`, + `mapfile -d '' -t dsh_e2b_explicit < ${quoteE2BShellArg(paths.environment)}`, + `: > ${quoteE2BShellArg(paths.environment)}`, 'dsh_e2b_env=()', - `dsh_e2b_explicit=(${environmentNames})`, - 'while IFS= read -r dsh_e2b_name; do', + "while IFS= read -r -d '' dsh_e2b_entry; do", + ' dsh_e2b_name="${dsh_e2b_entry%%=*}"', ' case "${dsh_e2b_name^^}" in DSH_*|*KEY*|*SECRET*|*TOKEN*) continue ;; esac', - ' dsh_e2b_env+=("$dsh_e2b_name=${!dsh_e2b_name}")', - 'done < <(compgen -e)', - 'for dsh_e2b_name in "${dsh_e2b_explicit[@]}"; do dsh_e2b_env+=("$dsh_e2b_name=${!dsh_e2b_name}"); done', - `env -i "\${dsh_e2b_env[@]}" "$@" ${stdoutRedirect} ${stderrRedirect}`.trimEnd(), + ' dsh_e2b_env+=("$dsh_e2b_entry")', + 'done < <(env -0)', + `env -i "\${dsh_e2b_env[@]}" "\${dsh_e2b_explicit[@]}" "$@" ${stdoutRedirect} ${stderrRedirect}`.trimEnd(), 'dsh_e2b_status=$?', 'wait', `printf '%s\\n' "$dsh_e2b_status" > ${quoteE2BShellArg(paths.status)}`, @@ -131,7 +132,6 @@ export class E2BSubprocessHandle implements SubprocessHandle { private readonly stderrReader: E2BOutputReader | undefined private readonly paths: RemotePaths private remotePid = -1 - private settled = false private terminationRequested = false private terminationSignal: NodeJS.Signals | null = null private termination: Promise | undefined @@ -150,6 +150,7 @@ export class E2BSubprocessHandle implements SubprocessHandle { this.paths = { pid: posix.join(stateDir, 'pid'), status: posix.join(stateDir, 'exit-code'), + environment: posix.join(stateDir, 'environment'), stdout: posix.join(stateDir, 'stdout.log'), stderr: posix.join(stateDir, 'stderr.log'), } @@ -182,7 +183,7 @@ export class E2BSubprocessHandle implements SubprocessHandle { /** @inheritdoc */ terminate(): void { - if (this.terminationRequested || this.settled) return + if (this.terminationRequested) return this.terminationRequested = true this.termination = this.terminateRemote() void this.termination.catch(() => {}) @@ -240,7 +241,6 @@ export class E2BSubprocessHandle implements SubprocessHandle { cwd: this.spec.cwd, stdin: this.spec.stdio.stdin !== 'ignore', timeoutMs: 0, - ...(this.spec.env !== undefined ? { envs: this.spec.env } : {}), onStdout: async (data) => { await this.dispatchOutput('stdout', data) }, onStderr: async (data) => { await this.dispatchOutput('stderr', data) }, }, @@ -250,7 +250,12 @@ export class E2BSubprocessHandle implements SubprocessHandle { } const completion = handle.wait() void completion.catch(() => {}) - this.remotePid = await this.waitForProcessGroupId(sandbox, completion) + try { + this.remotePid = await this.waitForProcessGroupId(sandbox, completion) + } catch (error: unknown) { + await Promise.allSettled([handle.kill()]) + throw error + } this.readyState.resolve(handle) await this.writeBatchStdin(handle) const outcome = await this.waitForCommand(completion) @@ -260,7 +265,6 @@ export class E2BSubprocessHandle implements SubprocessHandle { this.readyState.reject(error) throw error } finally { - this.settled = true this.spec.signal?.removeEventListener('abort', this.onAbort) this.stdout?.end() this.stderr?.end() @@ -269,17 +273,16 @@ export class E2BSubprocessHandle implements SubprocessHandle { private async prepareState(sandbox: Sandbox): Promise { await sandbox.files.makeDir(this.stateDir) + await sandbox.commands.run(`chmod 700 -- ${quoteE2BShellArg(this.stateDir)}`) const files = [ { path: this.paths.pid, data: '' }, { path: this.paths.status, data: '' }, + { path: this.paths.environment, data: explicitEnvironment(this.spec.env) }, ...(hasSpill(this.spec.stdio.stdout) ? [{ path: this.paths.stdout, data: '' }] : []), ...(hasSpill(this.spec.stdio.stderr) ? [{ path: this.paths.stderr, data: '' }] : []), ] await sandbox.files.write(files) - await sandbox.commands.run([ - `chmod 700 -- ${quoteE2BShellArg(this.stateDir)}`, - `chmod 600 -- ${files.map(file => quoteE2BShellArg(file.path)).join(' ')}`, - ].join('\n')) + await sandbox.commands.run(`chmod 600 -- ${files.map(file => quoteE2BShellArg(file.path)).join(' ')}`) } private async writeBatchStdin(handle: CommandHandle): Promise { diff --git a/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts b/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts index ff9d48073c..6be645a713 100644 --- a/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts +++ b/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts @@ -80,6 +80,7 @@ class FakeSandbox { readonly handle = new FakeCommandHandle() readonly commandsSeen: string[] = [] readonly writtenFiles: string[][] = [] + readonly writtenFileData = new Map() readonly removed: string[] = [] readonly directories: string[] = [] startOptions: StartOptions | undefined @@ -129,6 +130,7 @@ class FakeSandbox { }, write: async (files: Array<{ path: string; data: string }>): Promise => { this.writtenFiles.push(files.map(file => file.path)) + for (const file of files) this.writtenFileData.set(file.path, file.data) return files.map(() => ({})) }, read: async (): Promise => this.processGroupReads.shift() ?? this.processGroupId, @@ -251,7 +253,7 @@ describe('E2BSubprocessHandle', () => { const handle = new E2BSubprocessHandle(runtime(fake), spec({ argv: ['tool', 'argument with spaces'], stdio: { stdin: 'pipe', stdout: 'pipe', stderr: { maxBytes: 8, spill: { maxBytes: 32 } } }, - env: { PATH: '/bin', DEEPSEEK_API_KEY: 'explicit-secret', DSH_MODE: 'test' }, + env: { PATH: '/bin', 'FOO-BAR': 'hyphen-value', DEEPSEEK_API_KEY: 'explicit-secret', DSH_MODE: 'test' }, }), '/workspace/.dsh-e2b/processes/one') expect(handle.pid).toBe(-1) handle.stdin!.write('hello') @@ -261,17 +263,26 @@ describe('E2BSubprocessHandle', () => { expect(handle.pid).toBe(4343) expect(fake.handle.sent.map(value => String(value))).toEqual(['hello']) expect(fake.handle.closes).toBe(1) - expect(fake.startOptions?.envs).toEqual({ PATH: '/bin', DEEPSEEK_API_KEY: 'explicit-secret', DSH_MODE: 'test' }) + expect(fake.startOptions?.envs).toBeUndefined() const command = fake.commandsSeen.find(value => value.startsWith('exec setsid'))! expect(command).toContain('exec setsid --wait -- bash -c') - expect(command).toContain('DEEPSEEK_API_KEY') - expect(command).toContain('DSH_MODE') + expect(command).not.toContain('DEEPSEEK_API_KEY') + expect(command).not.toContain('DSH_MODE') + expect(command).not.toContain('FOO-BAR') expect(command).not.toContain('explicit-secret') + expect(command).not.toContain('hyphen-value') + expect(command).not.toContain('${!dsh_e2b_name}') + expect(command).toContain('env -0') + expect(command).toContain('mapfile -d') expect(fake.writtenFiles[0]).toEqual([ '/workspace/.dsh-e2b/processes/one/pid', '/workspace/.dsh-e2b/processes/one/exit-code', + '/workspace/.dsh-e2b/processes/one/environment', '/workspace/.dsh-e2b/processes/one/stderr.log', ]) + expect(fake.writtenFileData.get('/workspace/.dsh-e2b/processes/one/environment')).toBe( + 'PATH=/bin\0FOO-BAR=hyphen-value\0DEEPSEEK_API_KEY=explicit-secret\0DSH_MODE=test\0', + ) let piped = '' handle.stdout!.on('data', (chunk) => { piped += String(chunk) }) @@ -350,6 +361,11 @@ describe('E2BSubprocessHandle', () => { await handle.done expect(handle.collected.stdout!.readFrom(0)).toEqual({ text: 'cd', nextOffset: 4, lossy: true }) expect(fake.removed).toContain('/runtime/oversize/stdout.log') + const command = fake.commandsSeen.find(value => value.startsWith('exec setsid'))! + expect(command).toContain('head -c 3') + expect(command).toContain('/runtime/oversize/stdout.log') + expect(command).toContain('tee --output-error=warn-nopipe') + expect(command).not.toContain('tee -a') }) it('contains remote spill-removal failures and routes empty inherited output', async () => { @@ -415,6 +431,22 @@ describe('E2BSubprocessHandle', () => { await expect(handle.done).resolves.toEqual({ exitCode: null, signal: 'SIGTERM' }) }) + it('can terminate a surviving process group after the command leader settles', async () => { + const fake = new FakeSandbox() + const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/surviving-group') + await flush() + fake.handle.succeed(0) + await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) + expect(fake.alive).toBe(true) + + handle.terminate() + await flush() + const signaled = fake.commandsSeen.includes('kill -TERM -- -4242') + if (!signaled) fake.finish() + await expect(handle.waitForExit()).resolves.toBe(true) + expect(signaled).toBe(true) + }) + it('bounds waitForExit while startup or a live group is pending', async () => { const fake = new FakeSandbox() fake.deferStart() @@ -564,8 +596,14 @@ describe('E2BSubprocessHandle', () => { it('rejects invalid or absent process-group publication', async () => { const invalidGroup = new FakeSandbox() invalidGroup.processGroupId = 'not-a-pid\n' + vi.spyOn(invalidGroup.handle, 'kill').mockImplementation(async () => { + invalidGroup.handle.kills += 1 + invalidGroup.finish() + return true + }) const invalid = new E2BSubprocessHandle(runtime(invalidGroup), spec(), '/runtime/invalid-group') await expect(invalid.done).rejects.toThrow(/invalid process-group id/) + expect(invalidGroup.handle.kills).toBe(1) const absentGroup = new FakeSandbox() absentGroup.processGroupId = '' @@ -573,6 +611,7 @@ describe('E2BSubprocessHandle', () => { await flush() absentGroup.finish() await expect(absent.done).rejects.toThrow(/exited before publishing/) + expect(absentGroup.handle.kills).toBe(1) }) it('waits for delayed process-group publication', async () => { diff --git a/packages/lsp/lsp-local/src/index.ts b/packages/lsp/lsp-local/src/index.ts index fc7e50787f..3a2baa0c55 100644 --- a/packages/lsp/lsp-local/src/index.ts +++ b/packages/lsp/lsp-local/src/index.ts @@ -1,16 +1,19 @@ /** * Generic stdio language-server backend for `ctx.lsp`. One plugin instance configures a named table * of server commands and registers one isolated provider for each entry. Every provider lazily - * single-flights one server process per canonical workspace target, serves transient-open queries + * single-flights one server process per canonical workspace realpath, serves transient-open queries * through it, and replaces a selected transport that fails before or during the next read-only - * query. Providers read sources through `ctx.fs` and launch servers through - * `ctx.subprocess`, so both local and remote implementations share one host. + * query. Providers read sources through Node APIs in the host namespace (not `ctx.fs`) + * and trust their configured servers — no sandbox confinement. * * Namespace plugin (named exports, no default export). Lifecycle is effect-scoped: disposal * unregisters from `ctx.lsp` and tears down every live server. * @module @deepseek-ai/dsh-lsp-local */ +import { accessSync, constants, statSync } from 'node:fs' +import { delimiter, isAbsolute, join } from 'node:path' +import { pathToFileURL } from 'node:url' import type { Context } from 'cordis' import z from 'schemastery' import { LspError, LspProviderId } from '@deepseek-ai/dsh-lsp' @@ -22,9 +25,9 @@ import type { import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout' import { abortable, abortError } from './abort.ts' import { canonicalizeWorkspace, readHostSource } from './host.ts' -import type { HostWorkspace } from './host.ts' import { LspInstance } from './instance.ts' import type { ConnectionSpawner } from './connection.ts' +import { scrubbedParentEnv } from '@deepseek-ai/dsh-subprocess' import type { InstanceSpec } from './instance.ts' export { canonicalizeWorkspace, readHostSource } from './host.ts' @@ -44,7 +47,10 @@ export { LspConnection } from './connection.ts' export const name = 'lsp-local' /** Services required by this plugin. */ -export const inject = ['fs', 'lsp', 'subprocess'] +export const inject = ['lsp', 'subprocess'] + +/** Credential-shaped ambient env vars are NOT forwarded to the child by default. */ + const DEFAULT_MAX_MESSAGE_BYTES = 16_000_000 const DEFAULT_MAX_STDERR_BYTES = 1_000_000 @@ -86,7 +92,6 @@ export interface Config { /** One server config after schemastery fills every default. */ type ResolvedServerConfig = Required -type WorkspaceKey = HostWorkspace['target']['targetKey'] const LspLocalServerConfig: z = z.object({ command: z.string().required(), @@ -106,67 +111,27 @@ export const Config: z = z.object({ servers: z.dict(LspLocalServerConfig).required(), }) -/** Propagate teardown failures only after every sibling has settled. */ -function throwTeardownFailures(results: readonly PromiseSettledResult[], message: string): void { - const failures: unknown[] = [] - for (const result of results) { - if (result.status === 'rejected') failures.push(result.reason) - } - if (failures.length === 1) throw failures[0] - if (failures.length > 1) throw new AggregateError(failures, message) -} - /** * Register the configured stdio LSP providers. Resolves every executable at load (after credential * scrubbing) before publishing any provider; each process launches lazily on its first matching * query. - * @param ctx - the plugin context carrying `fs`, `lsp`, and `subprocess`. + * @param ctx - the plugin context (must inject `lsp`). * @param config - the resolved plugin configuration (schemastery has filled every default). */ -export async function apply(ctx: Context, config: Config): Promise { +export function apply(ctx: Context, config: Config): void { const entries = Object.entries(config.servers) if (entries.length === 0) throw new Error('lsp-local: servers must contain at least one server') - const setupAbort = new AbortController() - const stopSetupCancellation = ctx.on('internal/plugin', (fiber) => { - // An async plugin callback must observe its own disposal before Cordis can - // run effect cleanup, because unload otherwise waits for this callback. - if (fiber === ctx.fiber && fiber.uid === null) { - setupAbort.abort(new Error('lsp-local setup disposed')) - } - }) - // Resolve every server-local setting before registration so a bad later command or bound cannot // publish an earlier provider. Registry-level mapping conflicts are rolled back below. - const providers = await (async () => { - const lookups = entries.map(async ([providerId, rawConfig]) => { - if (providerId.trim() === '') throw new Error('lsp-local: server ids must be non-empty strings') - const resolved = rawConfig as ResolvedServerConfig - validateServerConfig(providerId, resolved) - const executable = await ctx.subprocess.resolveExecutable( - resolved.command, - resolved.env, - setupAbort.signal, - ) - setupAbort.signal.throwIfAborted() - return new LocalLspProvider( - providerId, - ctx.fs, - resolved, - executable, - spec => ctx.subprocess.spawn(spec), - ) - }) - try { - return await Promise.all(lookups) - } catch (error: unknown) { - setupAbort.abort(error) - await Promise.allSettled(lookups) - throw error - } finally { - stopSetupCancellation() - } - })() + const providers = entries.map(([providerId, rawConfig]) => { + if (providerId.trim() === '') throw new Error('lsp-local: server ids must be non-empty strings') + const resolved = rawConfig as ResolvedServerConfig + validateServerConfig(providerId, resolved) + const childEnv = buildChildEnv(resolved.env) + const executable = resolveExecutable(resolved.command, childEnv) + return new LocalLspProvider(providerId, resolved, childEnv, executable, spec => ctx.subprocess.spawn(spec)) + }) ctx.effect(() => { const disposers: Array<() => void> = [] @@ -179,8 +144,7 @@ export async function apply(ctx: Context, config: Config): Promise { return async () => { // Remove every route before process teardown so no new query can enter a draining provider. for (const dispose of disposers.reverse()) dispose() - const results = await Promise.allSettled(providers.map(provider => provider.disposeAll())) - throwTeardownFailures(results, 'lsp-local provider teardown failed') + await Promise.all(providers.map(provider => provider.disposeAll())) } }, 'lsp-local.registerProviders') } @@ -217,19 +181,16 @@ function assertPositiveInteger(providerId: string, name: string, value: number): class LocalLspProvider implements LspProvider { readonly id: LspProviderId readonly extensionToLanguage: Readonly> - /** One live instance per stable canonical workspace identity. */ - private readonly instances = new Map() + /** One live instance per canonical workspace realpath. */ + private readonly instances = new Map() /** One complete source-read→open→query→close serialization tail per canonical workspace. */ - private readonly queues = new Map>() - /** Workspace canonicalizations that have not entered a provider-owned queue yet. */ - private readonly workspaceLookups = new Set>() - private readonly lifetime = new AbortController() + private readonly queues = new Map>() private disposed = false constructor( providerId: string, - private readonly fs: Context['fs'], private readonly config: ResolvedServerConfig, + private readonly childEnv: Record, private readonly executable: string, private readonly spawner: ConnectionSpawner, ) { @@ -250,60 +211,43 @@ class LocalLspProvider implements LspProvider { if (signal?.aborted) throw abortError(signal) } - /** Fuse caller cancellation with provider disposal for every filesystem and protocol await. */ - private querySignal(signal?: AbortSignal): AbortSignal { - return signal === undefined - ? this.lifetime.signal - : AbortSignal.any([signal, this.lifetime.signal]) - } - async query(request: LspProviderQuery, signal?: AbortSignal): Promise { - // Honor an already-aborted signal before provider I/O so a canceled request never starts a server. + // Honor an already-aborted signal before host I/O so a canceled request never starts a server. this.assertActive(signal) - const querySignal = this.querySignal(signal) - const workspaceResult = canonicalizeWorkspace(this.fs, request.workspaceRoot, querySignal) - const workspaceLookup = workspaceResult.then(() => undefined, () => undefined) - this.workspaceLookups.add(workspaceLookup) - let workspace: HostWorkspace - try { - workspace = await workspaceResult - } finally { - this.workspaceLookups.delete(workspaceLookup) - } - this.assertActive(querySignal) - const workspaceKey = workspace.target.targetKey - return this.enqueue(workspaceKey, querySignal, async () => { - this.assertActive(querySignal) + const workspace = await canonicalizeWorkspace(request.workspaceRoot, signal) + this.assertActive(signal) + return this.enqueue(workspace, signal, async () => { + this.assertActive(signal) // Read inside the workspace queue but before spawning: a queued query sees current bytes when // its turn starts, while an invalid source still cannot leave an idle process pooled. - const source = await readHostSource(this.fs, request.filePath, workspace, this.config.maxDocumentBytes, querySignal) + const source = await readHostSource(request.filePath, workspace, this.config.maxDocumentBytes, signal) // Disposal may have snapshotted the instance map while host I/O was pending. Re-check before a // synchronous get-or-create so every spawned process remains owned by teardown. - this.assertActive(querySignal) - let instance = this.instanceFor(workspaceKey, workspace) + this.assertActive(signal) + let instance = this.instanceFor(workspace) try { - return await instance.query(request, source, querySignal) + return await instance.query(request, source, signal) } catch (error) { // A selected child can have died while idle or fail during the next write. Queries are // read-only, so replace that transport once and retry transparently. if (!instance.isTransportFailure(error)) throw error await instance.dispose() - this.evictIfCurrent(workspaceKey, instance) - this.assertActive(querySignal) - instance = this.instanceFor(workspaceKey, workspace) - return await instance.query(request, source, querySignal) + this.evictIfCurrent(workspace, instance) + this.assertActive(signal) + instance = this.instanceFor(workspace) + return await instance.query(request, source, signal) } finally { // Reach quiescence before dropping a dead slot; a replacement must survive this ownership check. if (instance.dead) { await instance.dispose() - this.evictIfCurrent(workspaceKey, instance) + this.evictIfCurrent(workspace, instance) } } }) } /** Serialize one complete query lifecycle for a canonical workspace. */ - private enqueue(workspace: WorkspaceKey, signal: AbortSignal | undefined, run: () => Promise): Promise { + private enqueue(workspace: string, signal: AbortSignal | undefined, run: () => Promise): Promise { const previous = this.queues.get(workspace) ?? Promise.resolve() const result = abortable(previous, signal).then(run) // The tail follows the actual prior work even when this caller aborts its wait. It never rejects, @@ -317,34 +261,34 @@ class LocalLspProvider implements LspProvider { } /** Return or synchronously publish the one instance for a canonical workspace. */ - private instanceFor(workspaceKey: WorkspaceKey, workspace: HostWorkspace): LspInstance { + private instanceFor(workspace: string): LspInstance { this.assertActive() - const existing = this.instances.get(workspaceKey) + const existing = this.instances.get(workspace) if (existing !== undefined) return existing const created = this.createInstance(workspace) - this.instances.set(workspaceKey, created) + this.instances.set(workspace, created) return created } /** Drop the slot iff it still contains this instance. */ - private evictIfCurrent(workspace: WorkspaceKey, instance: LspInstance): void { + private evictIfCurrent(workspace: string, instance: LspInstance): void { /* v8 ignore next -- mismatch requires another query to replace the slot before this finally runs. */ if (this.instances.get(workspace) === instance) this.instances.delete(workspace) } - private createInstance(workspace: HostWorkspace): LspInstance { + private createInstance(workspace: string): LspInstance { const spec: InstanceSpec = { command: this.executable, args: this.config.args, - cwd: workspace.canonicalPath, - workspaceUri: workspace.fileUrl, - env: this.config.env, + cwd: workspace, + env: this.childEnv, configuration: this.config.configuration, initializationOptions: this.config.initializationOptions, maxMessageBytes: this.config.maxMessageBytes, maxStderrBytes: this.config.maxStderrBytes, shutdownTimeoutMs: this.config.shutdownTimeoutMs, killGraceMs: this.config.killGraceMs, + pathToFileUri: path => pathToFileURL(path).href, } return new LspInstance(spec, this.spawner) } @@ -352,18 +296,51 @@ class LocalLspProvider implements LspProvider { /** Dispose every live instance and block further queries. */ async disposeAll(): Promise { this.disposed = true - this.lifetime.abort(new LspError('lsp-local provider is disposed', 'LSP_DISPOSED')) const live = [...this.instances.values()] const draining = [...this.queues.values()] - const resolving = [...this.workspaceLookups] this.instances.clear() - const results = await Promise.allSettled([ + await Promise.all([ ...live.map(instance => instance.dispose()), ...draining, - ...resolving, ]) this.queues.clear() - this.workspaceLookups.clear() - throwTeardownFailures(results, 'lsp-local instance teardown failed') + } +} + +/** The seam's scrubbed parent env (credential-shaped and DSH_* names dropped), plus the config's explicit env. */ +function buildChildEnv(extra: Record): Record { + return { ...scrubbedParentEnv(), ...extra } +} + +/** + * Resolve the server executable to an absolute path: an absolute command is verified directly; a + * bare command is looked up on the child's PATH. Fails loudly when nothing is executable. + */ +function resolveExecutable(command: string, childEnv: Record): string { + if (isAbsolute(command)) { + // Verify an absolute command too, so an unavailable one fails at load, not on the first query. + if (!isExecutableFileSync(command)) { + throw new Error(`lsp-local: command "${command}" is not an executable file`) + } + return command + } + /* v8 ignore next -- buildChildEnv always sets PATH from the ambient env; the further fallbacks are defensive. */ + const pathValue = childEnv.PATH ?? process.env.PATH ?? '' + for (const dir of pathValue.split(delimiter)) { + if (dir === '') continue + const candidate = join(dir, command) + if (isExecutableFileSync(candidate)) return candidate + } + throw new Error(`lsp-local: command "${command}" was not found on PATH`) +} + +/** Synchronous regular-file and executable check used only at load-time resolution. */ +function isExecutableFileSync(path: string): boolean { + try { + if (!statSync(path).isFile()) return false + accessSync(path, constants.X_OK) + return true + } catch { + return false } } diff --git a/packages/lsp/lsp-local/src/instance.ts b/packages/lsp/lsp-local/src/instance.ts index d643db1528..5a12192c62 100644 --- a/packages/lsp/lsp-local/src/instance.ts +++ b/packages/lsp/lsp-local/src/instance.ts @@ -7,7 +7,6 @@ * @module @deepseek-ai/dsh-lsp-local/instance */ -import { pathToFileURL } from 'node:url' import { LspError } from '@deepseek-ai/dsh-lsp' import type { LspOperation, @@ -37,6 +36,12 @@ export interface InstanceSpec extends ConnectionSpec { readonly shutdownTimeoutMs: number /** PID advertised to the server; `null` when client and server do not share a process namespace. */ readonly clientProcessId?: number | null + /** + * Encode one implementation-native absolute path as a file URI. + * @param path - Canonical workspace or source path. + * @returns A file URI interpreted in the server's filesystem namespace. + */ + readonly pathToFileUri: (path: string) => string } /** @@ -111,8 +116,8 @@ export class LspInstance { private async initialize(): Promise { const initializeResult = await this.connection.request('initialize', { processId: this.spec.clientProcessId === undefined ? process.pid : this.spec.clientProcessId, - rootUri: pathToFileURL(this.spec.cwd).href, - workspaceFolders: [{ uri: pathToFileURL(this.spec.cwd).href, name: 'workspace' }], + rootUri: this.spec.pathToFileUri(this.spec.cwd), + workspaceFolders: [{ uri: this.spec.pathToFileUri(this.spec.cwd), name: 'workspace' }], capabilities: CLIENT_CAPABILITIES, initializationOptions: this.spec.initializationOptions, }) as WireInitializeResult @@ -149,7 +154,7 @@ export class LspInstance { throw new LspError('server does not support the transient textDocument/didOpen this host requires', 'LSP_UNSUPPORTED_OPERATION') } - const uri = pathToFileURL(source.canonicalPath).href + const uri = this.spec.pathToFileUri(source.canonicalPath) let opened = false try { /* v8 ignore next -- guards an abort landing between the ready wait and didOpen; not deterministically reproducible. */ diff --git a/packages/lsp/lsp-local/tests/instance.spec.ts b/packages/lsp/lsp-local/tests/instance.spec.ts index 939b58f0b1..fc1d05843c 100644 --- a/packages/lsp/lsp-local/tests/instance.spec.ts +++ b/packages/lsp/lsp-local/tests/instance.spec.ts @@ -56,6 +56,7 @@ function makeInstance( maxStderrBytes: 100_000, shutdownTimeoutMs: 200, killGraceMs: 200, + pathToFileUri: path => pathToFileURL(path).href, ...overrides, }, spawnSubprocess, writer) live.push(instance) @@ -91,6 +92,7 @@ function scriptInstance(script: string, overrides: Partial = {}): maxStderrBytes: 100_000, shutdownTimeoutMs: 150, killGraceMs: 150, + pathToFileUri: path => pathToFileURL(path).href, ...overrides, }, spawnSubprocess) live.push(instance) From bccad1bb31d21a0290cdcbab23584c86feba80b1 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Tue, 28 Jul 2026 16:30:28 +0800 Subject: [PATCH 067/108] fix(e2b): normalize setup rejection reasons --- packages/e2b/code-runtime-e2b/src/index.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/e2b/code-runtime-e2b/src/index.ts b/packages/e2b/code-runtime-e2b/src/index.ts index a567a2bb6d..b6245244ba 100644 --- a/packages/e2b/code-runtime-e2b/src/index.ts +++ b/packages/e2b/code-runtime-e2b/src/index.ts @@ -213,7 +213,7 @@ export class E2BCodeRuntime extends CodeRuntime { (runtime) => { cleanup(); resolve(runtime) }, (error: unknown) => { cleanup() - reject(error instanceof Error ? error : new Error(String(error))) + reject(new Error(messageOf(error), { cause: error })) }, ) }) From dc44f93c39d284d378bc687769a0616758a13f82 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Tue, 28 Jul 2026 16:54:29 +0800 Subject: [PATCH 068/108] fix(e2b): simplify setup abort wait --- packages/e2b/code-runtime-e2b/src/index.ts | 32 ++++++++++------------ 1 file changed, 14 insertions(+), 18 deletions(-) diff --git a/packages/e2b/code-runtime-e2b/src/index.ts b/packages/e2b/code-runtime-e2b/src/index.ts index b6245244ba..c37cb3cc64 100644 --- a/packages/e2b/code-runtime-e2b/src/index.ts +++ b/packages/e2b/code-runtime-e2b/src/index.ts @@ -199,24 +199,20 @@ export class E2BCodeRuntime extends CodeRuntime { } /* jscpd:ignore-end */ - private awaitPreparation(signal: AbortSignal | undefined): Promise { - if (signal === undefined) return this.ready - return new Promise((resolve, reject) => { - const onAbort = (): void => { cleanup(); resolve(undefined) } - const cleanup = (): void => { signal.removeEventListener('abort', onAbort) } - signal.addEventListener('abort', onAbort, { once: true }) - if (signal.aborted) { - onAbort() - return - } - void this.ready.then( - (runtime) => { cleanup(); resolve(runtime) }, - (error: unknown) => { - cleanup() - reject(new Error(messageOf(error), { cause: error })) - }, - ) - }) + private async awaitPreparation(signal: AbortSignal | undefined): Promise { + if (signal === undefined) return await this.ready + const aborted = Promise.withResolvers() + const onAbort = (): void => { aborted.resolve(undefined) } + signal.addEventListener('abort', onAbort, { once: true }) + if (signal.aborted) { + signal.removeEventListener('abort', onAbort) + return undefined + } + try { + return await Promise.race([this.ready, aborted.promise]) + } finally { + signal.removeEventListener('abort', onAbort) + } } private assertPreparationActive(): void { From 3e343b44778633350f02420c11cc45ea0168e91e Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Tue, 28 Jul 2026 18:31:37 +0800 Subject: [PATCH 069/108] fix(e2b): harden remote adapter boundaries --- ...026-07-27-e2b-remote-runtime-poc.i18n.yaml | 4 +- .../2026-07-27-e2b-remote-runtime-poc.md | 12 +- .../2026-07-27-e2b-remote-runtime-poc.zh.md | 12 +- .../tests/fixtures/e2b/e2b/bin.ts | 128 ++++++++++- .../tests/fixtures/e2b/e2b/cordis.yml | 2 +- .../e2b/code-runtime-e2b/README.i18n.yaml | 4 +- packages/e2b/code-runtime-e2b/README.md | 2 +- packages/e2b/code-runtime-e2b/README.zh.md | 2 +- .../e2b/code-runtime-e2b/src/runner-source.ts | 212 ++++++++++++++++-- .../code-runtime-e2b/tests/runtime.spec.ts | 117 ++++++++++ packages/e2b/e2b/tests/composition.e2e.ts | 64 +++++- packages/e2b/fs-e2b/README.i18n.yaml | 4 +- packages/e2b/fs-e2b/README.md | 2 +- packages/e2b/fs-e2b/README.zh.md | 2 +- packages/e2b/fs-e2b/src/index.ts | 6 +- packages/e2b/fs-e2b/tests/filesystem.spec.ts | 11 + packages/e2b/lsp-e2b/README.i18n.yaml | 4 +- packages/e2b/lsp-e2b/README.md | 2 +- packages/e2b/lsp-e2b/README.zh.md | 2 +- packages/e2b/lsp-e2b/src/index.ts | 90 +++++++- packages/e2b/lsp-e2b/tests/provider.spec.ts | 44 ++-- packages/e2b/pty-e2b/README.i18n.yaml | 4 +- packages/e2b/pty-e2b/README.md | 5 +- packages/e2b/pty-e2b/README.zh.md | 5 +- packages/e2b/pty-e2b/src/index.ts | 42 +++- packages/e2b/pty-e2b/src/session.ts | 56 ++++- packages/e2b/pty-e2b/tests/index.spec.ts | 63 +++++- packages/e2b/pty-e2b/tests/session.spec.ts | 109 ++++++--- packages/e2b/subprocess-e2b/README.i18n.yaml | 4 +- packages/e2b/subprocess-e2b/README.md | 4 +- packages/e2b/subprocess-e2b/README.zh.md | 4 +- packages/e2b/subprocess-e2b/src/process.ts | 22 +- .../subprocess-e2b/tests/subprocess.spec.ts | 41 +++- 33 files changed, 939 insertions(+), 146 deletions(-) diff --git a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.i18n.yaml b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.i18n.yaml index 5ed4e445b8..beb49fba76 100644 --- a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.i18n.yaml +++ b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md -2026-07-27-e2b-remote-runtime-poc.md: a713cca3fd9e7ee11a2c487bf6ed2c3a205dc502 -2026-07-27-e2b-remote-runtime-poc.zh.md: 7afd54f72622d51a3d71719b3b4975ccd7f83e3a +2026-07-27-e2b-remote-runtime-poc.md: efb91fc166f36dc8fb047d686e8cd82d93b9af6a +2026-07-27-e2b-remote-runtime-poc.zh.md: eb14cccc284d38cdc5cc704512b6899621a9a840 diff --git a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md index a713cca3fd..efb91fc166 100644 --- a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md +++ b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md @@ -18,7 +18,7 @@ The E2B integration is an opt-in provider-composition POC. Its six E2B-specific - `@deepseek-ai/dsh-fs-e2b` implements `ctx.fs` over that sandbox's Filesystem API. - `@deepseek-ai/dsh-subprocess-e2b` implements `ctx.subprocess` over E2B Commands and remote Linux process groups. - `@deepseek-ai/dsh-pty-e2b` registers an E2B byte-PTY backend on `ctx.pty` while the existing registry retains exact-Agent ownership. -- `@deepseek-ai/dsh-lsp-e2b` registers configured remote language servers on `ctx.lsp`, reads source through E2B Filesystem APIs, and runs servers through `dsh-subprocess-e2b`. +- `@deepseek-ai/dsh-lsp-e2b` registers configured remote language servers on `ctx.lsp`, reads source through a bounded no-follow helper in E2B, and runs servers through `dsh-subprocess-e2b`. - `@deepseek-ai/dsh-code-runtime-e2b` registers `ctx.codeRuntime`, runs each model program in a fresh remote worker, and dispatches binding functions in the host process. - The existing `@deepseek-ai/dsh-bash-local` remains the Bash implementation because it delegates all process mechanics to `ctx.subprocess`. @@ -28,11 +28,11 @@ The providers reuse the PTY, LSP, Code Runtime, and subprocess seams without cha ## POC boundary -E2B owns the mutable filesystem, command and Bash processes, PTY shell and foreground process groups, language-server processes and source reads, the Code Runtime runner and worker, and adapter-private files under `.dsh-e2b`. +E2B owns the mutable filesystem, command and Bash processes, PTY shell and terminal-session process groups, language-server processes and source reads, the Code Runtime launcher, controller, and worker, and adapter-private files under `.dsh-e2b`. The host owns Cordis and plugin objects, the agent loop, agent/session/goal state, session logs and persistence, LLM calls, prompts and tools, authority decisions, skills, subagent orchestration, PTY buffers and readiness state, LSP JSON-RPC ids/queues/protocol state, Code Runtime type stripping/output accounting/binding dispatch, and E2B SDK/network orchestration. The overlay does not upload, mount, or synchronize the host workspace; identical cwd strings name independent host and remote directories. -Byte-sensitive protocols use the narrowest adapter required by E2B's callback shapes. PTY consumes the SDK's byte callback directly. LSP and Code Runtime install dependency-free remote helpers that encode raw payloads as validated newline-delimited base64 JSON, keeping E2B's decoded command callbacks on an ASCII transport. +Byte-sensitive protocols use the narrowest adapter required by E2B's callback shapes. PTY consumes the SDK's byte callback directly. LSP installs a bounded remote source reader, while Code Runtime keeps framed stdout in a launcher process isolated from the controller and worker descriptors. Their dependency-free helpers encode protocol payloads as validated newline-delimited base64 JSON, keeping E2B's decoded command callbacks on an ASCII transport. Retaining a sandbox preserves remote files and unmanaged remote state only. Reconnect does not reconstruct host PTY sessions, buffers, process handles, LSP connections or requests, code workers, binding calls, timers, output cursors, or locks. Managed groups terminate and join when their provider disposes before the shared owner pauses, leaves, or kills the sandbox. @@ -40,9 +40,9 @@ The POC has no session-persistence backend, template builder, volume, snapshot, ## Verification -Focused package suites pin owner lifecycle cleanup, filesystem semantics, subprocess process groups, configuration and publication rollback, byte framing and multibyte boundaries, PTY readiness/signals, LSP transport and source containment, Code Runtime bindings, hostile traffic, output limits, timeout/abort ordering, disposal to quiescence, and package-owned invariant registrations. Adjacent local-backend suites pin the shared PTY utilities and the LSP cross-namespace `processId` behavior. +Focused package suites pin owner lifecycle cleanup, filesystem semantics and commit metadata, subprocess process groups, configuration and verified publication rollback, byte framing and multibyte boundaries, PTY readiness/signals/default-environment scrubbing/terminal-session cleanup, stable bounded LSP source reads, Code Runtime binding and descriptor isolation, worker-pipe draining, hostile traffic, output limits, timeout/abort ordering, disposal to quiescence, and package-owned invariant registrations. Adjacent local-backend suites pin the shared PTY utilities and the LSP cross-namespace `processId` behavior. -A credential-gated Loader composition creates one real E2B sandbox and exercises FS-to-Bash and Bash-to-FS visibility, multibyte PTY output and `SIGINT`, multibyte LSP hover and definition results, Code Runtime host bindings and typed rejection under mutation of adapter-captured intrinsics, wall timeout, abort, runner cleanup, host-workspace isolation, and final sandbox deletion. The same scenario runs through source imports and built package exports. +A credential-gated Loader composition creates real E2B sandboxes and exercises FS-to-Bash and Bash-to-FS visibility, process-publication rollback, bounded spill output, PTY default-secret scrubbing and process-tree cleanup, stable bounded LSP source reads, Code Runtime host bindings and descriptor-isolated output accounting, wall timeout, abort, runner cleanup, host-workspace isolation, and final sandbox deletion. The same composition runs through source imports and built package exports. ## Alternatives considered @@ -66,4 +66,4 @@ A credential-gated Loader composition creates one real E2B sandbox and exercises The small composition demonstrates that existing capability seams can move an agent's mutable coding world off-host without changing the loop or model-facing tool packages. `sandboxId` plus pause/leave permits manual remote-file retention for experiments, while kill remains the demo's cleanup policy. -The providers are not interchangeable with local backends for every consumer: remote startup cannot synchronously expose a PID, E2B retains complete command output in SDK memory, ordinary command callbacks are not byte-faithful, signal attribution is partly inferred, and reconnect cannot restore handles or protocol state. PTY uses E2B's byte API; LSP and Code Runtime add validated ASCII framing where protocol bytes matter. Remote process/spill artifacts accumulate in a retained sandbox, Code programs share a JavaScript realm with Node worker internals, and a process that deliberately escapes a captured remote process group does not become reconnectable or owned. These gaps remain documented POC constraints rather than compatibility shims or new cross-cutting abstractions. +The providers are not interchangeable with local backends for every consumer: remote startup cannot synchronously expose a PID, E2B retains complete command output in SDK memory, ordinary command callbacks are not byte-faithful, signal attribution is partly inferred, and reconnect cannot restore handles or protocol state. PTY uses E2B's byte API; LSP and Code Runtime add validated ASCII framing where protocol bytes matter. Remote process/spill artifacts accumulate in a retained sandbox, Code programs share a JavaScript realm with Node worker internals, and a process that deliberately escapes a managed process group or PTY session does not become reconnectable or owned. These gaps remain documented POC constraints rather than compatibility shims or new cross-cutting abstractions. diff --git a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.zh.md b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.zh.md index 7afd54f726..eb14cccc28 100644 --- a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.zh.md +++ b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.zh.md @@ -18,7 +18,7 @@ E2B 集成是一个选择性启用的提供方组合 POC。它的 6 个 E2B 专 - `@deepseek-ai/dsh-fs-e2b` 在该沙箱的 Filesystem API 之上实现 `ctx.fs`。 - `@deepseek-ai/dsh-subprocess-e2b` 在 E2B Commands 和远程 Linux 进程组之上实现 `ctx.subprocess`。 - `@deepseek-ai/dsh-pty-e2b` 在 `ctx.pty` 上注册 E2B 字节 PTY 后端,并把精确的 Agent 所有权保留在现有注册表中。 -- `@deepseek-ai/dsh-lsp-e2b` 在 `ctx.lsp` 上注册已配置的远程语言服务器,通过 E2B Filesystem API 读取源代码,并通过 `dsh-subprocess-e2b` 运行服务器。 +- `@deepseek-ai/dsh-lsp-e2b` 在 `ctx.lsp` 上注册已配置的远程语言服务器,通过 E2B 内有界且不跟随链接的辅助程序读取源代码,并通过 `dsh-subprocess-e2b` 运行服务器。 - `@deepseek-ai/dsh-code-runtime-e2b` 注册 `ctx.codeRuntime`,在全新的远程 worker 中运行每个模型程序,并在宿主进程中分发绑定函数。 - 现有的 `@deepseek-ai/dsh-bash-local` 继续作为 Bash 实现,因为它把所有进程机制委托给 `ctx.subprocess`。 @@ -28,11 +28,11 @@ E2B 集成是一个选择性启用的提供方组合 POC。它的 6 个 E2B 专 ## POC 边界 -E2B 拥有可变文件系统、命令和 Bash 进程、PTY shell 与前台进程组、语言服务器进程及源码读取、Code Runtime 运行器和 worker,以及 `.dsh-e2b` 下的适配器私有文件。 +E2B 拥有可变文件系统、命令和 Bash 进程、PTY shell 与终端会话进程组、语言服务器进程及源码读取、Code Runtime launcher、controller 和 worker,以及 `.dsh-e2b` 下的适配器私有文件。 宿主拥有 Cordis 与插件对象、agent loop、agent/会话/goal 状态、会话日志及持久化、LLM(大语言模型)调用、提示词与工具、权限决策、skill(技能)、subagent 编排、PTY 缓冲与就绪状态、LSP JSON-RPC id/队列/协议状态、Code Runtime 类型剥离/输出计量/绑定分发,以及 E2B SDK/网络编排。该 overlay 不会上传、挂载或同步宿主工作区;拼写相同的 cwd 字符串分别指向彼此独立的宿主与远程目录。 -对字节敏感的协议只使用适配 E2B 回调形状所需的最窄适配器。PTY 直接消费 SDK 的字节回调。LSP 与 Code Runtime 会安装无依赖的远程辅助程序,把原始载荷编码为经过验证、以换行分隔的 base64 JSON,并通过 ASCII 传输承载 E2B 已解码的命令回调。 +对字节敏感的协议只使用适配 E2B 回调形状所需的最窄适配器。PTY 直接消费 SDK 的字节回调。LSP 会安装一个有界的远程源码读取器;Code Runtime 则把分帧 stdout 保留在与 controller 和 worker 描述符隔离的 launcher 进程内。它们的无依赖辅助程序会把协议载荷编码为经过验证、以换行分隔的 base64 JSON,并通过 ASCII 传输承载 E2B 已解码的命令回调。 保留沙箱只会保存远程文件与未受管的远程状态。重新连接不会重建宿主 PTY 会话、缓冲、进程句柄、LSP 连接或请求、代码 worker、绑定调用、定时器、输出游标或锁。受管进程组会在所属提供方 dispose(资源释放)时终止并等待退出,之后共享所有者才会暂停、脱离或终止沙箱。 @@ -40,9 +40,9 @@ E2B 拥有可变文件系统、命令和 Bash 进程、PTY shell 与前台进程 ## 验证 -聚焦包测试套件固定所有者生命周期清理、文件系统语义、进程管理的进程组、配置与发布回滚、字节分帧与多字节边界、PTY 就绪状态/信号、LSP 传输与源码路径约束、Code Runtime 绑定、恶意通信、输出上限、超时/中止顺序、等待完全停稳的资源释放,以及包自有不变式注册。相邻本地后端测试套件固定共享 PTY 工具函数,以及 LSP 跨命名空间 `processId` 行为。 +聚焦包测试套件固定所有者生命周期清理、文件系统语义与提交元数据、进程管理的进程组、配置与经过验证的发布回滚、字节分帧与多字节边界、PTY 就绪状态/信号/默认环境清理/终端会话清理、稳定且有界的 LSP 源码读取、Code Runtime 绑定与描述符隔离、worker 管道排空、恶意通信、输出上限、超时/中止顺序、等待完全停稳的资源释放,以及包自有不变式注册。相邻本地后端测试套件固定共享 PTY 工具函数,以及 LSP 跨命名空间 `processId` 行为。 -凭据门控的 Loader 组合会创建一个真实 E2B 沙箱,并演练 FS-to-Bash 与 Bash-to-FS 可见性、多字节 PTY 输出和 `SIGINT`、多字节 LSP 悬停与定义结果、Code Runtime 宿主绑定,以及适配器已捕获 intrinsic 被修改时的类型化 reject、墙钟超时、中止、运行器清理、宿主工作区隔离,以及最终删除沙箱。同一场景分别通过源代码导入与已构建包导出运行。 +凭据门控的 Loader 组合会创建真实 E2B 沙箱,并演练 FS-to-Bash 与 Bash-to-FS 可见性、进程发布回滚、有界 spill 输出、PTY 默认秘密清理与进程树清理、稳定且有界的 LSP 源码读取、Code Runtime 宿主绑定与描述符隔离的输出记账、墙钟超时、中止、runner 清理、宿主工作区隔离,以及最终删除沙箱。同一组合分别通过源代码导入与已构建包导出运行。 ## 曾考虑的替代方案 @@ -66,4 +66,4 @@ E2B 拥有可变文件系统、命令和 Bash 进程、PTY shell 与前台进程 这个小型组合证明,现有功能 seam 可以把 agent 的可变 coding 环境移出宿主,而无需改变循环或面向模型的工具包。`sandboxId` 与 `pause`/`leave` 允许实验手动保留远程文件,演示仍以 `kill` 作为清理策略。 -这些提供方并不能对所有消费方与本地后端互换:远程启动无法同步公开 PID,E2B 会在 SDK 内存中保留完整命令输出,普通命令回调并非字节保真,信号归因部分依靠推断,重新连接也无法恢复句柄或协议状态。PTY 使用 E2B 的字节 API;LSP 与 Code Runtime 则在必须保真处理协议字节之处增加经过验证的 ASCII 分帧。保留沙箱后会累积远程进程/spill 产物,模型程序与 Node worker 内部机制共享一个 JavaScript realm,有意逃离已捕获远程进程组的进程也不会因此变得可重新连接或由该组合管理。这些缺口作为 POC 约束明确记录,而不会引入兼容垫片或新的跨领域抽象。 +这些提供方并不能对所有消费方与本地后端互换:远程启动无法同步公开 PID,E2B 会在 SDK 内存中保留完整命令输出,普通命令回调并非字节保真,信号归因部分依靠推断,重新连接也无法恢复句柄或协议状态。PTY 使用 E2B 的字节 API;LSP 与 Code Runtime 则在必须保真处理协议字节之处增加经过验证的 ASCII 分帧。保留沙箱后会累积远程进程/spill 产物,模型程序与 Node worker 内部机制共享一个 JavaScript realm,有意逃离受管理进程组或 PTY 会话的进程也不会因此变得可重新连接或由该组合管理。这些缺口作为 POC 约束明确记录,而不会引入兼容垫片或新的跨领域抽象。 diff --git a/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts b/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts index 89b09a6818..4f2401d2c7 100644 --- a/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts +++ b/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts @@ -5,7 +5,7 @@ import { AgentMessageId } from '@deepseek-ai/dsh-agent' import type { Agent } from '@deepseek-ai/dsh-agent' import { Session, SessionId } from '@deepseek-ai/dsh-session' import type {} from '@deepseek-ai/dsh-code-runtime-e2b' -import type {} from '@deepseek-ai/dsh-e2b' +import { quoteE2BShellArg } from '@deepseek-ai/dsh-e2b' import type {} from '@deepseek-ai/dsh-fs-e2b' import type {} from '@deepseek-ai/dsh-bash-local' import type {} from '@deepseek-ai/dsh-lsp-e2b' @@ -34,6 +34,7 @@ const owner: Agent = { const unregisterOwner = ctx.agents.register(owner) let terminalId: Awaited>['sessionId'] | undefined try { + const sandbox = await ctx.e2b.getSandbox() const fromFs = await ctx.fs.resolve('from-fs.txt') await ctx.fs.writeText(fromFs, 'written-by-fs\n', { kind: 'createIfAbsent' }) const bashRead = await ctx.bash.run(ctx.bash.resolve({ command: 'cat from-fs.txt' })) @@ -72,6 +73,38 @@ try { ].every(entry => environmentLines.has(entry)) if (!explicitEnvironment) throw new Error(`E2B subprocess dropped an explicit environment entry: ${environmentText}`) + const remoteFiles = sandbox.files as unknown as { + read(path: string, options?: unknown): Promise + } + const readRemoteFile = remoteFiles.read.bind(sandbox.files) + let publicationFaultInjected = false + remoteFiles.read = async (path, options) => { + if (!publicationFaultInjected && path.includes('/processes/') && path.endsWith('/pid')) { + publicationFaultInjected = true + throw new Error('injected process-group publication read failure') + } + return await readRemoteFile(path, options) + } + let publicationRollback = false + try { + const unpublished = ctx.subprocess.spawn({ + argv: ['bash', '-c', 'exec -a dsh-publication-survivor sleep 30 & wait'], + cwd: process.cwd(), + stdio: { stdin: 'ignore', stdout: { maxBytes: 4_096 }, stderr: { maxBytes: 4_096 } }, + graceMs: 500, + env: {}, + }) + await unpublished.done + throw new Error('E2B subprocess unexpectedly survived an injected publication failure') + } catch (error: unknown) { + if (!String(error).includes('injected process-group publication read failure')) throw error + const processes = await sandbox.commands.run('ps -eo args=') + publicationRollback = publicationFaultInjected && !processes.stdout.includes('dsh-publication-survivor') + if (!publicationRollback) throw new Error('E2B subprocess publication rollback left its remote process group alive') + } finally { + remoteFiles.read = readRemoteFile + } + const spillHandle = ctx.subprocess.spawn({ argv: ['bash', '-c', "printf '0123456789'; sleep 30"], cwd: process.cwd(), @@ -81,7 +114,7 @@ try { }) const spillReader = spillHandle.collected.stdout if (spillReader === undefined) throw new Error('E2B subprocess omitted its configured stdout collector') - const spillDeadline = Date.now() + 5_000 + const spillDeadline = Date.now() + 15_000 while (spillReader.readFrom(0).nextOffset < 10) { if (Date.now() >= spillDeadline) throw new Error('E2B subprocess did not stream the spill probe output') await new Promise(resolveDelay => setTimeout(resolveDelay, 20)) @@ -114,6 +147,53 @@ try { workspaceRoot: process.cwd(), }) + const swappedSource = await ctx.fs.resolve('swapped-source.ts') + const swappedSourcePath = posix.join(process.cwd(), 'swapped-source.ts') + await ctx.fs.writeText(swappedSource, 'const safe = true\n', { kind: 'createIfAbsent' }) + const remoteCommands = sandbox.commands as unknown as { + run(command: string, options?: unknown): Promise<{ exitCode: number; stdout: string; stderr: string }> + } + const runRemoteCommand = remoteCommands.run.bind(sandbox.commands) + let containmentFaultInjected = false + remoteCommands.run = async (command, options) => { + if (!containmentFaultInjected && command.includes('dsh-e2b-source-reader') && command.includes('swapped-source.ts')) { + containmentFaultInjected = true + await runRemoteCommand(`rm -f -- ${quoteE2BShellArg(swappedSourcePath)} && ln -s -- /etc/hosts ${quoteE2BShellArg(swappedSourcePath)}`) + } + return await runRemoteCommand(command, options) + } + let lspContainment = false + try { + await ctx.lsp.query({ + operation: 'hover', + filePath: 'swapped-source.ts', + position: { line: 0, character: 1 }, + workspaceRoot: process.cwd(), + }) + } catch (error: unknown) { + lspContainment = containmentFaultInjected && String(error).includes('opened safely') + if (!lspContainment) throw error + } finally { + remoteCommands.run = runRemoteCommand + } + if (!lspContainment) throw new Error('E2B LSP source swap was not rejected') + + const oversizedSourcePath = posix.join(process.cwd(), 'oversized-source.ts') + await sandbox.commands.run(`head -c 4000001 /dev/zero > ${quoteE2BShellArg(oversizedSourcePath)}`) + let lspDocumentBound = false + try { + await ctx.lsp.query({ + operation: 'hover', + filePath: 'oversized-source.ts', + position: { line: 0, character: 0 }, + workspaceRoot: process.cwd(), + }) + } catch (error: unknown) { + lspDocumentBound = String(error).includes('over the 4000000-byte limit') + if (!lspDocumentBound) throw error + } + if (!lspDocumentBound) throw new Error('E2B LSP accepted an oversized remote source') + const terminal = await ctx.pty.spawn(owner, { type: 'shell' }) terminalId = terminal.sessionId const terminalEcho = await ctx.pty.startSend(owner, terminal.sessionId, { @@ -124,9 +204,19 @@ try { await new Promise(resolveDelay => setTimeout(resolveDelay, 150)) const terminalSignal = await ctx.pty.signal(owner, terminal.sessionId, 'SIGINT') const interrupted = await sleeping.done + const stubborn = await ctx.pty.startSend(owner, terminal.sessionId, { + text: "bash -c 'trap \"\" TERM; exec sleep 30' & printf 'DSH_STUBBORN_PID=%s\\n' \"$!\"", + submit: true, + }).done + const stubbornMatch = /DSH_STUBBORN_PID=([1-9][0-9]*)/.exec(stubborn.viewport) + if (stubbornMatch?.[1] === undefined) throw new Error(`E2B PTY did not report its stubborn child: ${stubborn.viewport}`) + const stubbornPid = Number(stubbornMatch[1]) const terminalScrollback = ctx.pty.read(owner, terminal.sessionId, { count: 50 }) await ctx.pty.kill(owner, terminal.sessionId, 'live E2B composition complete') terminalId = undefined + const stubbornProbe = await sandbox.commands.run(`if kill -0 ${stubbornPid} 2>/dev/null; then printf alive; else printf gone; fi`) + const terminalTreeCleanup = stubbornProbe.stdout === 'gone' + if (!terminalTreeCleanup) throw new Error(`E2B PTY left process ${stubbornPid} alive after close`) const code = await ctx.codeRuntime.run({ program: ` @@ -179,6 +269,33 @@ try { `, bindings: [], }) + const nativeOutput = await ctx.codeRuntime.run({ + program: ` + let stdoutPrototype = Object.getPrototypeOf(process.stdout) + while (stdoutPrototype && !Object.hasOwn(stdoutPrototype, 'write')) stdoutPrototype = Object.getPrototypeOf(stdoutPrototype) + Reflect.apply(stdoutPrototype.write, process.stdout, ['x'.repeat(8192)]) + return true + `, + bindings: [], + }) + const descriptorOutput = await ctx.codeRuntime.run({ + program: ` + const fs = await import('node:fs') + const forged = Buffer.from(JSON.stringify({ type: 'done' })).toString('base64') + '\\n' + fs.writeSync(1, forged) + fs.writeSync(1, 'x'.repeat(8192)) + return true + `, + bindings: [], + }) + const inheritedOutput = await ctx.codeRuntime.run({ + program: ` + const childProcess = await import('node:child_process') + childProcess.spawnSync(process.execPath, ['-e', 'process.stdout.write("x".repeat(8192))'], { stdio: 'inherit' }) + return true + `, + bindings: [], + }) const timedOut = await ctx.codeRuntime.run({ program: 'await new Promise(() => {})', bindings: [], @@ -212,18 +329,25 @@ try { bashRead: bashRead.stdout.text, fsRead, explicitEnvironment, + publicationRollback, spill: { liveBytes: liveSpillBytes, outcome: spillOutcome, read: spillRead }, hover, definition, + lspContainment, + lspDocumentBound, terminal: { motd: terminal.motd, echo: terminalEcho, signal: terminalSignal, interrupted, + treeCleanup: terminalTreeCleanup, scrollback: terminalScrollback.text, }, code, hostileOutput, + nativeOutput, + descriptorOutput, + inheritedOutput, timedOut, aborted, oversizedBoot, diff --git a/examples/headless-agent/tests/fixtures/e2b/e2b/cordis.yml b/examples/headless-agent/tests/fixtures/e2b/e2b/cordis.yml index 6b3350e49c..65bd3d5246 100644 --- a/examples/headless-agent/tests/fixtures/e2b/e2b/cordis.yml +++ b/examples/headless-agent/tests/fixtures/e2b/e2b/cordis.yml @@ -2,7 +2,7 @@ name: '@deepseek-ai/dsh-e2b' config: cwd: !!js process.cwd() - timeoutMs: 120000 + timeoutMs: 180000 onTimeout: kill onDispose: kill diff --git a/packages/e2b/code-runtime-e2b/README.i18n.yaml b/packages/e2b/code-runtime-e2b/README.i18n.yaml index 9eb2633b86..ba43d787b8 100644 --- a/packages/e2b/code-runtime-e2b/README.i18n.yaml +++ b/packages/e2b/code-runtime-e2b/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/e2b/code-runtime-e2b/README.md -README.md: 1007800c59725d116197a52f6d087ea5e2860686 -README.zh.md: a56026bce760879079627dd63a80dfe95c4d2049 +README.md: 171e63a8c1ca70deb63e9860f5088b83401a37d0 +README.zh.md: b70984e765d32985436c15dd543263cc06a25715 diff --git a/packages/e2b/code-runtime-e2b/README.md b/packages/e2b/code-runtime-e2b/README.md index 1007800c59..171e63a8c1 100644 --- a/packages/e2b/code-runtime-e2b/README.md +++ b/packages/e2b/code-runtime-e2b/README.md @@ -19,7 +19,7 @@ Every value is a positive safe integer. `maxOutputBytes` is at least four bytes, ## Execution and bridge contract -Setup uploads one dependency-free runner under `ctx.e2b.runtimeRoot` and resolves remote Node. For each run, the host wraps and type-strips erasable TypeScript with Node's `stripTypeScriptTypes`, then starts the runner in `ctx.e2b.cwd`. The runner creates a fresh worker thread with an empty environment and heap limit, measures active event-loop time, and destroys that worker after one completion. The enclosing E2B subprocess group is terminated and awaited after every result, timeout, abort, or disposal, so ordinary child processes in that group stop with the run. +Setup uploads one dependency-free runner under `ctx.e2b.runtimeRoot` and resolves remote Node. For each run, the host wraps and type-strips erasable TypeScript with Node's `stripTypeScriptTypes`, then starts the runner in `ctx.e2b.cwd`. The runner keeps the framed host protocol in a launcher process, forks a controller whose stdout and stderr are bounded data pipes, and creates a fresh worker thread with an empty environment and heap limit. Model writes to native descriptors and inherited child output therefore cannot enter the frame stream; worker and controller pipes drain before the terminal frame. The worker measures active event-loop time and is destroyed after one completion. The enclosing E2B subprocess group is terminated and awaited after every result, timeout, abort, or disposal, so ordinary child processes in that group stop with the run. The bridge uses validated newline-delimited base64 JSON frames because E2B subprocess callbacks expose decoded text. Binding arguments and resolutions use the worker runtime's iterative lossless-JSON wire shape; binding functions execute on the host and typed rejection classes are materialized inside the remote worker. The worker captures the JavaScript intrinsics that its adapter boundary invokes before model code runs, hardening binding transport, output accounting, and completion validation against mutation of those references. The host repeats message validation, call-id deduplication, lossless-JSON checks, and the outer-output ledger. diff --git a/packages/e2b/code-runtime-e2b/README.zh.md b/packages/e2b/code-runtime-e2b/README.zh.md index a56026bce7..b70984e765 100644 --- a/packages/e2b/code-runtime-e2b/README.zh.md +++ b/packages/e2b/code-runtime-e2b/README.zh.md @@ -19,7 +19,7 @@ ## 执行与桥接契约 -设置阶段会在 `ctx.e2b.runtimeRoot` 下上传一个无依赖的 runner,并解析远程 Node。每次运行时,宿主会包装仅使用可擦除语法的 TypeScript,再用 Node 的 `stripTypeScriptTypes` 剥离类型,然后在 `ctx.e2b.cwd` 中启动 runner。runner 会创建一个具有空环境与堆上限的全新 worker 线程,测量事件循环活跃时间,并在一次运行结算后销毁该 worker。每当运行返回结果、超时、中止或因资源释放终止时,系统都会终止外围的 E2B 进程组并等待其退出,因此组内的普通子进程会随本次运行一同停止。 +设置阶段会在 `ctx.e2b.runtimeRoot` 下上传一个无依赖的 runner,并解析远程 Node。每次运行时,宿主会包装仅使用可擦除语法的 TypeScript,再用 Node 的 `stripTypeScriptTypes` 剥离类型,然后在 `ctx.e2b.cwd` 中启动 runner。runner 会把面向宿主的分帧协议保留在 launcher 进程内,派生一个以 stdout 和 stderr 作为有界数据管道的 controller,再创建一个具有空环境与堆上限的全新 worker 线程。因此,模型对原生描述符的写入和继承的子进程输出无法进入分帧流;worker 与 controller 管道会在发出终结帧前排空。worker 会测量事件循环活跃时间,并在一次运行结算后销毁。每当运行返回结果、超时、中止或因资源释放终止时,系统都会终止外围的 E2B 进程组并等待其退出,因此组内的普通子进程会随本次运行一同停止。 由于 E2B 进程管理回调公开的是已解码文本,桥接层使用经过验证、以换行分隔的 base64 JSON 帧。绑定参数与 resolve 值使用 worker 运行时的迭代式无损 JSON wire 形状;绑定函数在宿主执行,类型化的 reject 类则在远程 worker 内物化。worker 会在模型代码运行前捕获其适配器边界调用的 JavaScript intrinsic,从而增强绑定传输、输出记账与完成值验证对这些引用修改的抵御能力。宿主会再次执行消息验证、调用 id 去重和无损 JSON 检查,并用外层输出账本再次计量。 diff --git a/packages/e2b/code-runtime-e2b/src/runner-source.ts b/packages/e2b/code-runtime-e2b/src/runner-source.ts index b0355f20ce..e5cf6ebefa 100644 --- a/packages/e2b/code-runtime-e2b/src/runner-source.ts +++ b/packages/e2b/code-runtime-e2b/src/runner-source.ts @@ -2,9 +2,11 @@ /** Node program that runs one model program in a fresh remote worker thread. */ export const CODE_RUNNER_SOURCE = String.raw`import { Buffer } from 'node:buffer' +import { fork } from 'node:child_process' import { inspect } from 'node:util' import { Worker, isMainThread, parentPort, workerData } from 'node:worker_threads' import { createInterface } from 'node:readline' +import { fileURLToPath } from 'node:url' const emitFrame = message => { process.stdout.write(Buffer.from(JSON.stringify(message)).toString('base64') + '\n') @@ -12,23 +14,140 @@ const emitFrame = message => { const parseFrame = line => JSON.parse(Buffer.from(line, 'base64').toString('utf8')) -if (isMainThread) { +const waitForPipeDrain = stream => { + if (stream.readableEnded || stream.destroyed) return Promise.resolve() + return new Promise(resolve => { + const done = () => { + stream.off('end', done) + stream.off('close', done) + stream.off('error', done) + resolve() + } + stream.once('end', done) + stream.once('close', done) + stream.once('error', done) + if (stream.readableEnded || stream.destroyed) done() + }) +} + +const waitForChildExit = child => { + if (child.exitCode !== null || child.signalCode !== null) return Promise.resolve() + return new Promise(resolve => { child.once('exit', resolve) }) +} + +const jsonStringBytes = text => Buffer.byteLength(JSON.stringify(text)) + +const truncateLog = (text, available) => { + if (available < 2) return '' + let result = '' + let bytes = 2 + for (const character of text) { + const cost = jsonStringBytes(character) - 2 + if (bytes + cost > available) break + bytes += cost + result += character + } + return result +} + +const runLauncher = () => { const input = createInterface({ input: process.stdin, crlfDelay: Infinity }) - let worker - let finished = false - let computeTimer + let controller + let maxOutputBytes = 0 + let logBytes = 2 + let logEntries = 0 + let settling = false + let closed = false + let terminal + const finish = message => { - if (finished) return - finished = true - clearInterval(computeTimer) - emitFrame(message) - const current = worker - worker = undefined - Promise.resolve(current ? current.terminate() : undefined).finally(() => { + if (settling) { + if (message.type === 'output-limit') terminal = message + return + } + settling = true + terminal = message + const current = controller + controller = undefined + const drain = current + ? new Promise(resolve => { setImmediate(resolve) }).then(async () => { + const stdoutDrained = waitForPipeDrain(current.stdout) + const stderrDrained = waitForPipeDrain(current.stderr) + const exited = waitForChildExit(current) + current.kill('SIGKILL') + await Promise.all([exited, stdoutDrained, stderrDrained]) + }) + : Promise.resolve() + void drain.catch(error => { + process.stderr.write('code-runtime-e2b controller cleanup error: ' + String(error) + '\n') + }).then(() => { + closed = true + emitFrame(terminal) input.close() process.stdin.destroy() }) } + + const forwardLog = text => { + if (closed || terminal?.type === 'output-limit') return + const separator = logEntries > 0 ? 1 : 0 + const available = maxOutputBytes - logBytes - separator + const cost = jsonStringBytes(text) + if (cost > available) { + const prefix = truncateLog(text, available) + if (prefix) { + logBytes += jsonStringBytes(prefix) + separator + logEntries += 1 + emitFrame({ type: 'log', text: prefix }) + } + finish({ type: 'output-limit' }) + return + } + logBytes += cost + separator + logEntries += 1 + emitFrame({ type: 'log', text }) + } + + const startController = message => { + maxOutputBytes = message.maxOutputBytes + controller = fork(fileURLToPath(import.meta.url), [], { + env: { DSH_CODE_RUNTIME_CONTROLLER: '1' }, + execArgv: [], + stdio: ['ignore', 'pipe', 'pipe', 'ipc'], + }) + const current = controller + current.stdout.on('data', data => { forwardLog(data.toString('utf8')) }) + current.stderr.on('data', data => { forwardLog(data.toString('utf8')) }) + current.on('message', raw => { + if (!raw || typeof raw !== 'object') return + if (raw.type === 'log' && typeof raw.text === 'string') { + forwardLog(raw.text) + return + } + if (settling) return + if (raw.type === 'call' && typeof raw.id === 'number' && typeof raw.global === 'string' && typeof raw.name === 'string' && Array.isArray(raw.args)) { + emitFrame({ type: 'call', id: raw.id, global: raw.global, name: raw.name, args: raw.args }) + } else if (raw.type === 'output-limit') { + finish({ type: 'output-limit' }) + } else if (raw.type === 'done') { + if (raw.error && typeof raw.error === 'object' && typeof raw.error.kind === 'string' && typeof raw.error.message === 'string') { + finish({ type: 'done', error: { kind: raw.error.kind, message: raw.error.message } }) + } else if (raw.value === undefined || Array.isArray(raw.value)) { + finish({ type: 'done', ...(raw.value === undefined ? {} : { value: raw.value }) }) + } + } + }) + current.on('error', error => { + finish({ type: 'done', error: { kind: 'worker-exit', message: 'remote controller error: ' + error.message } }) + }) + current.on('exit', code => { + if (!settling) finish({ type: 'done', error: { kind: 'worker-exit', message: 'remote controller exited with code ' + code + ' before completing' } }) + }) + current.send(message, error => { + if (error) finish({ type: 'done', error: { kind: 'worker-exit', message: 'remote controller boot failed: ' + error.message } }) + }) + } + input.on('line', line => { let message try { @@ -38,26 +157,77 @@ if (isMainThread) { finish({ type: 'done', error: { kind: 'worker-exit', message: 'remote runner received a malformed frame' } }) return } + if (!controller) { + if (!message || message.type !== 'boot' || typeof message.code !== 'string' || !Array.isArray(message.namespaces) || !Number.isSafeInteger(message.maxOutputBytes) || message.maxOutputBytes < 4) { + finish({ type: 'done', error: { kind: 'worker-exit', message: 'remote runner received an invalid boot frame' } }) + return + } + startController(message) + return + } + if (message && message.type === 'reply' && typeof message.id === 'number' && typeof message.ok === 'boolean') { + controller.send(message.ok + ? { type: 'reply', id: message.id, ok: true, value: message.value } + : { type: 'reply', id: message.id, ok: false, message: String(message.message) }, error => { + if (error) finish({ type: 'done', error: { kind: 'worker-exit', message: 'remote controller reply failed: ' + error.message } }) + }) + } + }) + input.on('close', () => { if (controller && !settling) controller.kill('SIGKILL') }) +} + +const runController = () => { + let worker + let finished = false + let computeTimer + const send = message => { + if (process.send) process.send(message) + } + const finish = message => { + if (finished) return + finished = true + clearInterval(computeTimer) + const current = worker + worker = undefined + const drain = current + ? new Promise(resolve => { setImmediate(resolve) }).then(async () => { + const stdoutDrained = waitForPipeDrain(current.stdout) + const stderrDrained = waitForPipeDrain(current.stderr) + await Promise.all([current.terminate(), stdoutDrained, stderrDrained]) + }) + : Promise.resolve() + void drain.catch(error => { + send({ type: 'log', text: 'code-runtime-e2b worker cleanup error: ' + String(error) + '\n' }) + }).then(() => { + if (!process.send) { + process.exitCode = 1 + return + } + process.send(message, () => { if (process.connected) process.disconnect() }) + }) + } + process.on('message', message => { if (!worker) { if (!message || message.type !== 'boot' || typeof message.code !== 'string' || !Array.isArray(message.namespaces)) { - finish({ type: 'done', error: { kind: 'worker-exit', message: 'remote runner received an invalid boot frame' } }) + finish({ type: 'done', error: { kind: 'worker-exit', message: 'remote controller received an invalid boot frame' } }) return } worker = new Worker(new URL(import.meta.url), { workerData: message, env: {}, + execArgv: [], stdout: true, stderr: true, resourceLimits: { maxOldGenerationSizeMb: message.maxOldGenerationSizeMb }, }) - worker.stdout.on('data', data => { emitFrame({ type: 'log', text: data.toString('utf8') }) }) - worker.stderr.on('data', data => { emitFrame({ type: 'log', text: data.toString('utf8') }) }) + worker.stdout.on('data', data => { send({ type: 'log', text: data.toString('utf8') }) }) + worker.stderr.on('data', data => { send({ type: 'log', text: data.toString('utf8') }) }) worker.on('message', raw => { if (!raw || typeof raw !== 'object') return if (raw.type === 'call' && typeof raw.id === 'number' && typeof raw.global === 'string' && typeof raw.name === 'string' && Array.isArray(raw.args)) { - emitFrame({ type: 'call', id: raw.id, global: raw.global, name: raw.name, args: raw.args }) + send({ type: 'call', id: raw.id, global: raw.global, name: raw.name, args: raw.args }) } else if (raw.type === 'log' && typeof raw.text === 'string') { - emitFrame({ type: 'log', text: raw.text }) + send({ type: 'log', text: raw.text }) } else if (raw.type === 'output-limit') { finish({ type: 'output-limit' }) } else if (raw.type === 'done') { @@ -88,8 +258,10 @@ if (isMainThread) { : { type: 'reply', id: message.id, ok: false, message: String(message.message) }) } }) - input.on('close', () => { if (worker && !finished) void worker.terminate() }) -} else { + process.on('disconnect', () => { if (worker && !finished) void worker.terminate() }) +} + +if (!isMainThread) { const port = parentPort if (!port) throw new Error('remote worker requires parentPort') @@ -456,5 +628,9 @@ if (isMainThread) { process.stdout.write = originalStdout process.stderr.write = originalStderr } +} else if (process.env.DSH_CODE_RUNTIME_CONTROLLER === '1') { + runController() +} else { + runLauncher() } ` diff --git a/packages/e2b/code-runtime-e2b/tests/runtime.spec.ts b/packages/e2b/code-runtime-e2b/tests/runtime.spec.ts index 9824fb2cf0..9743c361c5 100644 --- a/packages/e2b/code-runtime-e2b/tests/runtime.spec.ts +++ b/packages/e2b/code-runtime-e2b/tests/runtime.spec.ts @@ -1,3 +1,7 @@ +import { spawn } from 'node:child_process' +import { mkdtemp, rm, writeFile } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' import { PassThrough, Writable } from 'node:stream' import { Context } from 'cordis' import { describe, expect, it, vi } from 'vitest' @@ -153,7 +157,120 @@ function request(program = 'return 1') { return { program, bindings: [] } } +async function runInstalledRunner( + code: string, + maxOutputBytes = 2_000_000, +): Promise<{ messages: unknown[]; stderr: string }> { + const directory = await mkdtemp(join(tmpdir(), 'dsh-e2b-code-runner-')) + const runner = join(directory, 'runner.mjs') + await writeFile(runner, CODE_RUNNER_SOURCE) + const child = spawn(process.execPath, [runner], { stdio: ['pipe', 'pipe', 'pipe'] }) + const decoder = new E2BFrameDecoder(4_000_000) + const messages: unknown[] = [] + let stderr = '' + let outputError: unknown + child.stdout.setEncoding('ascii') + child.stdout.on('data', (chunk: string) => { + try { + messages.push(...decoder.push(chunk)) + } catch (error: unknown) { + outputError = error + child.kill('SIGKILL') + } + }) + child.stderr.setEncoding('utf8') + child.stderr.on('data', (chunk: string) => { stderr += chunk }) + + try { + child.stdin.write(encodeE2BFrame({ + type: 'boot', + code, + namespaces: [], + computeMs: 1_000, + maxOutputBytes, + maxOldGenerationSizeMb: 128, + })) + await new Promise((resolve, reject) => { + const timeout = setTimeout(() => { + child.kill('SIGKILL') + reject(new Error('installed E2B code runner did not exit')) + }, 5_000) + child.once('error', (error) => { + clearTimeout(timeout) + reject(error) + }) + child.once('exit', () => { + clearTimeout(timeout) + resolve() + }) + }) + if (outputError !== undefined) throw outputError + decoder.finish() + return { messages, stderr } + } finally { + child.kill('SIGKILL') + await rm(directory, { recursive: true, force: true }) + } +} + describe('E2BCodeRuntime', () => { + it('keeps model-owned descriptors outside the host framing process', async () => { + const forged = Buffer.from(JSON.stringify({ type: 'done' })).toString('base64') + '\\n' + const { messages, stderr } = await runInstalledRunner( + ` + const fs = await import('node:fs') + const childProcess = await import('node:child_process') + fs.writeSync(1, ${JSON.stringify(forged)}) + childProcess.spawnSync(process.execPath, ['-e', 'process.stdout.write("child-native")'], { stdio: 'inherit' }) + return true + `, + ) + const records = messages as Array<{ type?: string; text?: string; value?: unknown }> + const terminal = records.filter(message => message.type === 'done') + + expect(stderr).toBe('') + expect(terminal).toEqual([{ type: 'done', value: [true] }]) + expect(records.at(-1)).toEqual(terminal[0]) + expect(records.filter(message => message.type === 'log').map(message => message.text).join('')) + .toContain(forged + 'child-native') + }) + + it('bounds native descriptor output before it reaches the host protocol', async () => { + const { messages, stderr } = await runInstalledRunner( + "(await import('node:fs')).writeSync(1, 'x'.repeat(4096)); return true", + 64, + ) + const records = messages as Array<{ type?: string; text?: string }> + + expect(stderr).toBe('') + expect(records.at(-1)).toEqual({ type: 'output-limit' }) + expect(Buffer.byteLength(records.filter(message => message.type === 'log').map(message => message.text).join(''))) + .toBeLessThanOrEqual(62) + }) + + it('drains native worker pipes before emitting the terminal frame', async () => { + const expectedBytes = 1_048_576 + const { messages, stderr } = await runInstalledRunner( + ` + let stdoutPrototype = Object.getPrototypeOf(process.stdout) + while (stdoutPrototype && !Object.hasOwn(stdoutPrototype, 'write')) stdoutPrototype = Object.getPrototypeOf(stdoutPrototype) + Reflect.apply(stdoutPrototype.write, process.stdout, ['x'.repeat(${expectedBytes})]) + return true + `, + ) + const records = messages as Array<{ type?: string; text?: string }> + const terminalIndex = records.findIndex(message => message.type === 'done') + const nativeOutput = records + .slice(0, terminalIndex) + .filter(message => message.type === 'log') + .map(message => message.text ?? '') + .join('') + + expect(stderr).toBe('') + expect(terminalIndex).toBe(records.length - 1) + expect(Buffer.byteLength(nativeOutput)).toBe(expectedBytes) + }) + it('prepares the remote runner and returns logs and a lossless completion', async () => { const handle = new FakeHandle((message, current) => { if ((message as { type?: string }).type !== 'boot') return diff --git a/packages/e2b/e2b/tests/composition.e2e.ts b/packages/e2b/e2b/tests/composition.e2e.ts index 5c9bd56fc9..4f4ea19eb9 100644 --- a/packages/e2b/e2b/tests/composition.e2e.ts +++ b/packages/e2b/e2b/tests/composition.e2e.ts @@ -1,9 +1,15 @@ import { access } from 'node:fs/promises' import { join } from 'node:path' import { fileURLToPath } from 'node:url' +import { Context } from 'cordis' import { describe, expect, it } from 'vitest' +import { AgentMessageId } from '@deepseek-ai/dsh-agent' +import type { Agent } from '@deepseek-ai/dsh-agent' import { runLoaderSmoke } from '@deepseek-ai/dsh-loader-smoke' import { Sandbox, SandboxNotFoundError } from '@deepseek-ai/dsh-e2b' +import { E2BPtyBackend } from '@deepseek-ai/dsh-pty-e2b' +import { PtySessionId } from '@deepseek-ai/dsh-pty' +import { Session, SessionId } from '@deepseek-ai/dsh-session' const fixtureRoot = fileURLToPath(new URL('../../../../examples/headless-agent/tests/fixtures/e2b/e2b/', import.meta.url)) const binScript = join(fixtureRoot, 'bin.ts') @@ -11,6 +17,53 @@ const configPath = join(fixtureRoot, 'cordis.yml') const tsconfigPath = fileURLToPath(new URL('../../../../tsconfig.json', import.meta.url)) describe.skipIf(!process.env.E2B_API_KEY)('E2B live Loader composition', () => { + it('scrubs sandbox-default credentials from an actual E2B PTY', async () => { + const apiKey = process.env.E2B_API_KEY + if (apiKey === undefined) throw new Error('E2B_API_KEY disappeared before the PTY environment test') + const sandbox = await Sandbox.create({ + apiKey, + envs: { NPM_TOKEN: 'sentinel-secret', DSH_STALE: 'sentinel-stale', KEEP: 'visible' }, + timeoutMs: 60_000, + secure: true, + lifecycle: { onTimeout: 'kill' }, + }) + try { + const ctx = new Context() + ctx.provide('e2b', { cwd: '/home/user', getSandbox: async () => sandbox } as never) + const ownerId = SessionId('e2b-pty-env-owner') + const owner: Agent = { + id: ownerId, + options: {}, + session: new Session(ownerId), + status: 'idle', + acceptsNextStep: false, + ctx, + followup: () => AgentMessageId('unused'), + steer: () => AgentMessageId('unused'), + inject: () => AgentMessageId('unused'), + send: () => AgentMessageId('unused'), + cancel() {}, + whenIdle: () => Promise.resolve(), + } + const backend = new E2BPtyBackend(ctx, { + backendType: 'shell', rows: 24, cols: 80, + scrollbackLines: 100, scrollbackMaxBytes: 65_536, maxReadBytes: 16_384, + pollIntervalMs: 25, idleSilenceMs: 1_000, timeoutMs: 5_000, disposeGraceMs: 1_000, + }) + const session = await backend.spawn({ sessionId: PtySessionId('env'), owner, type: 'shell' }) + const result = await session.startSend({ + text: "printf 'NPM=<%s> DSH=<%s> KEEP=<%s>\\n' \"$NPM_TOKEN\" \"$DSH_STALE\" \"$KEEP\"", + submit: true, + }).done + expect(result.viewport).toContain('NPM=<> DSH=<> KEEP=') + expect(result.viewport).not.toContain('sentinel-secret') + expect(result.viewport).not.toContain('sentinel-stale') + await session.close('environment test complete') + } finally { + await sandbox.kill().catch(() => false) + } + }, 70_000) + it('runs FS, Bash, PTY, LSP, and Code Runtime in one sandbox and deletes it', async () => { const { stdout, stderr } = await runLoaderSmoke({ label: 'E2B composition', @@ -22,7 +75,7 @@ describe.skipIf(!process.env.E2B_API_KEY)('E2B live Loader composition', () => { env: { NODE_OPTIONS: [process.env.NODE_OPTIONS, '--disable-warning=ExperimentalWarning'].filter(Boolean).join(' '), }, - processTimeoutMs: 120_000, + processTimeoutMs: 180_000, inspect: async (cwd) => { for (const name of ['from-fs.txt', 'from-bash.txt', 'multibyte # file.ts', 'fixture-lsp.mjs']) { await expect(access(join(cwd, name))).rejects.toMatchObject({ code: 'ENOENT' }) @@ -36,6 +89,7 @@ describe.skipIf(!process.env.E2B_API_KEY)('E2B live Loader composition', () => { bashRead: 'written-by-fs\n', fsRead: 'written-by-bash\n', explicitEnvironment: true, + publicationRollback: true, spill: { liveBytes: 6, outcome: { exitCode: null, signal: 'SIGTERM' }, @@ -49,12 +103,18 @@ describe.skipIf(!process.env.E2B_API_KEY)('E2B live Loader composition', () => { kind: 'locations', locations: [{ range: { start: { line: 0, character: 6 }, end: { line: 0, character: 10 } } }], }, + lspContainment: true, + lspDocumentBound: true, terminal: { echo: { waitReason: 'stdin_read', sessionStatus: { kind: 'running' } }, signal: { delivered: true }, interrupted: { sessionStatus: { kind: 'running' } }, + treeCleanup: true, }, hostileOutput: { error: { kind: 'output-limit' } }, + nativeOutput: { error: { kind: 'output-limit' } }, + descriptorOutput: { error: { kind: 'output-limit' } }, + inheritedOutput: { error: { kind: 'output-limit' } }, timedOut: { error: { kind: 'timeout' } }, aborted: { error: { kind: 'abort', message: 'live abort' } }, oversizedBoot: { error: { kind: 'worker-exit' } }, @@ -75,5 +135,5 @@ describe.skipIf(!process.env.E2B_API_KEY)('E2B live Loader composition', () => { const apiKey = process.env.E2B_API_KEY if (apiKey === undefined) throw new Error('E2B_API_KEY disappeared during the live composition test') await expect(Sandbox.getInfo(String(output.sandboxId), { apiKey })).rejects.toBeInstanceOf(SandboxNotFoundError) - }, 135_000) + }, 195_000) }) diff --git a/packages/e2b/fs-e2b/README.i18n.yaml b/packages/e2b/fs-e2b/README.i18n.yaml index b416105166..95a3a9eea6 100644 --- a/packages/e2b/fs-e2b/README.i18n.yaml +++ b/packages/e2b/fs-e2b/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/e2b/fs-e2b/README.md -README.md: d8f3915fa281e8b9c062e3412c92bc6358616284 -README.zh.md: 93d27bcdd514ccbf87e07d0eff5958edf3c726c1 +README.md: a505703fc764f8441fa54d2b1d922762eb3fafdf +README.zh.md: 626fdb52979d29f12d1bc1b67b0ea13730003af7 diff --git a/packages/e2b/fs-e2b/README.md b/packages/e2b/fs-e2b/README.md index d8f3915fa2..a505703fc7 100644 --- a/packages/e2b/fs-e2b/README.md +++ b/packages/e2b/fs-e2b/README.md @@ -8,7 +8,7 @@ E2B implementation of the [`@deepseek-ai/dsh-fs`](../../fs/fs/README.md) provide - **Remote identity and metadata** — relative paths resolve as POSIX paths against the caller cwd or `ctx.e2b.cwd`; `realpath -m` supplies canonical target identity without requiring the final file to exist. `stat`, no-follow `lstat`, and stable one-level directory listings project E2B metadata into the filesystem seam. Versions are opaque hashes of E2B metadata plus a per-write extended attribute. - **UTF-8 reads** — whole reads and streamed reads preserve cross-chunk decoding, reject invalid UTF-8, and use the seam's 8192-byte NUL sample for binary detection. The model-facing tool still owns size selection and line windowing. -- **Atomic mutations** — writes upload a mode-`0600` temporary sibling, preserve an existing file's POSIX mode, and publish through same-directory Linux `mv -f`. E2B creates missing parent directories. Literal edits LF-normalize for matching, restore dominant CRLF storage, and serialize mutations per canonical target within the host process. Optional create/version guards keep the base seam's observed-state semantics. +- **Atomic mutations** — writes upload a mode-`0600` temporary sibling, preserve an existing file's POSIX mode, and publish through E2B's same-directory atomic rename. The rename response supplies the committed version, so no fallible metadata request follows the commit point. E2B creates missing parent directories. Literal edits LF-normalize for matching, restore dominant CRLF storage, and serialize mutations per canonical target within the host process. Optional create/version guards keep the base seam's observed-state semantics. - **Failures and cancellation** — E2B not-found, permission, abort, and other controller failures map to the existing `FsError` vocabulary. Cancellation is best-effort at SDK request boundaries; a successful rename is the commit point. The provider does not copy, mount, or reconcile the host workspace. Giving it a host path as `cwd` creates a remote directory with the same spelling only. diff --git a/packages/e2b/fs-e2b/README.zh.md b/packages/e2b/fs-e2b/README.zh.md index 93d27bcdd5..626fdb5297 100644 --- a/packages/e2b/fs-e2b/README.zh.md +++ b/packages/e2b/fs-e2b/README.zh.md @@ -8,7 +8,7 @@ - **远程身份与元数据**:相对路径以调用方 cwd 或 `ctx.e2b.cwd` 为基准,按照 POSIX 路径解析;`realpath -m` 提供规范化目标身份,且不要求最终文件存在。`stat`、不跟随链接的 `lstat` 和稳定的单层目录列表会把 E2B 元数据投影到文件系统 seam。版本是 E2B 元数据与每次写入设置的扩展属性所组成的不透明哈希。 - **UTF-8 读取**:完整读取和流式读取会保留跨分片解码、拒绝无效 UTF-8,并使用 seam 的 8192 字节 NUL 样本检测二进制内容。面向模型的工具仍负责选择大小和行窗口。 -- **原子变更**:写入会上传 mode 为 `0600` 的同级临时文件,保留现有文件的 POSIX mode,并通过同目录 Linux `mv -f` 发布。E2B 会创建缺失的父目录。字面量编辑匹配时会规范化为 LF,存储时恢复占主导的 CRLF,并在宿主进程内按规范化目标串行执行变更。可选的创建/版本防护会保留基础 seam 的已观察状态语义。 +- **原子变更**:写入会上传 mode 为 `0600` 的同级临时文件,保留现有文件的 POSIX mode,并通过 E2B 的同目录原子重命名发布。重命名响应会提供已提交的版本,因此提交点之后不会再进行可能失败的元数据请求。E2B 会创建缺失的父目录。字面量编辑匹配时会规范化为 LF,存储时恢复占主导的 CRLF,并在宿主进程内按规范化目标串行执行变更。可选的创建/版本防护会保留基础 seam 的已观察状态语义。 - **失败与取消**:E2B 的未找到、权限、中止及其他控制器故障会映射到现有 `FsError` 词汇。取消在 SDK 请求边界上采用尽力而为语义;成功 rename 是提交点。 该提供方不会复制、挂载或协调宿主工作区。把宿主路径用作 `cwd`,只会在远程创建一个拼写相同的目录。 diff --git a/packages/e2b/fs-e2b/src/index.ts b/packages/e2b/fs-e2b/src/index.ts index 4f733f1fba..2a7fd1275f 100644 --- a/packages/e2b/fs-e2b/src/index.ts +++ b/packages/e2b/fs-e2b/src/index.ts @@ -412,11 +412,7 @@ export class E2BFileSystem extends FileSystem { signalOpts(signal), ) assertNotAborted(signal, 'write') - await sandbox.commands.run( - `mv -f -- ${quoteE2BShellArg(temporary)} ${quoteE2BShellArg(targetPath)}`, - signalOpts(signal), - ) - const committed = await sandbox.files.getInfo(targetPath) + const committed = await sandbox.files.rename(temporary, targetPath, signalOpts(signal)) return entryVersion(committed) } catch (error: unknown) { try { diff --git a/packages/e2b/fs-e2b/tests/filesystem.spec.ts b/packages/e2b/fs-e2b/tests/filesystem.spec.ts index 177bb9e59c..13d94e70c7 100644 --- a/packages/e2b/fs-e2b/tests/filesystem.spec.ts +++ b/packages/e2b/fs-e2b/tests/filesystem.spec.ts @@ -453,6 +453,17 @@ describe('E2BFileSystem atomic writes and edits', () => { expect(controller.signal.aborted).toBe(true) }) + it('returns committed rename metadata without a fallible post-commit lookup', async () => { + const remote = new FakeRemote() + const getInfo = vi.spyOn(remote.sandbox.files, 'getInfo') + const { fs } = await setup(remote) + + await expect(fs.writeText(await fs.resolve('committed'), 'yes')) + .resolves.toMatchObject({ operation: 'create' }) + expect(getInfo).toHaveBeenCalledTimes(1) + expect(remote.renames).toHaveLength(1) + }) + it('cleans staging files and maps command, permission, and abort failures', async () => { const remote = new FakeRemote() const { fs } = await setup(remote) diff --git a/packages/e2b/lsp-e2b/README.i18n.yaml b/packages/e2b/lsp-e2b/README.i18n.yaml index 5f126d7e56..066820d9fd 100644 --- a/packages/e2b/lsp-e2b/README.i18n.yaml +++ b/packages/e2b/lsp-e2b/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/e2b/lsp-e2b/README.md -README.md: ded928ec073e6c0943d574d86359eeec586bb862 -README.zh.md: 91eb7cecb49adcf73f4be948e9a5da400981e6f1 +README.md: 6cc118767adc5d2df855b009f48faab596928fb7 +README.zh.md: b384e9efd0e5c62b0ee6e69f73755e1bbe0192df diff --git a/packages/e2b/lsp-e2b/README.md b/packages/e2b/lsp-e2b/README.md index ded928ec07..6cc118767a 100644 --- a/packages/e2b/lsp-e2b/README.md +++ b/packages/e2b/lsp-e2b/README.md @@ -27,7 +27,7 @@ Provider ids and commands are non-empty; numeric bounds are positive safe intege E2B command callbacks are text, while LSP is byte-framed. The installed proxy therefore base64-frames raw server stdout, stderr, and stdin as newline-delimited ASCII JSON; the host validates and decodes every frame before handing bytes to the shared `LspInstance` protocol engine. `initialize.processId` is `null` because host and server do not share a process namespace. -One language-server process is pooled per provider and canonical remote workspace. Queries serialize per workspace but different workspaces run concurrently. Each query canonicalizes the remote workspace and source with `realpath`, rejects paths outside that workspace, requires a regular file, enforces the size bound before and after reading, decodes strict UTF-8, and uses the ordinary transient `didOpen` / request / `didClose` lifecycle. A transport failure disposes the instance and retries the read-only query once on a fresh remote process. +One language-server process is pooled per provider and canonical remote workspace. Queries serialize per workspace but different workspaces run concurrently. Each query canonicalizes the remote workspace and source with `realpath`, rejects paths outside that workspace, then uses a remote helper to open the canonical source without following the final symlink and to verify and read one stable descriptor. The helper requires a regular file and reads at most `maxDocumentBytes + 1` bytes before strict UTF-8 decoding. Queries use the ordinary transient `didOpen` / request / `didClose` lifecycle. A transport failure disposes the instance and retries the read-only query once on a fresh remote process. The subprocess adapter owns process groups and escalation, so cancellation and disposal await remote server quiescence. The host owns LSP request ids, pending requests, provider queues, and normalized results. diff --git a/packages/e2b/lsp-e2b/README.zh.md b/packages/e2b/lsp-e2b/README.zh.md index 91eb7cecb4..b384e9efd0 100644 --- a/packages/e2b/lsp-e2b/README.zh.md +++ b/packages/e2b/lsp-e2b/README.zh.md @@ -27,7 +27,7 @@ E2B 命令回调交付的是文本,而 LSP 按字节分帧。因此,已安装的代理会把服务器 stdout、stderr 和 stdin 的原始字节进行 base64 编码,封装为以换行分隔的 ASCII JSON 帧;宿主会验证并解码每一帧,再把字节交给共享的 `LspInstance` 协议引擎。`initialize.processId` 为 `null`,因为宿主与服务器不共享进程命名空间。 -每个提供方与规范化远程工作区的组合共享一个池化语言服务器进程。同一工作区的查询串行执行,不同工作区的查询并发运行。每项查询都会使用 `realpath` 规范化远程工作区与源文件,拒绝工作区外的路径,要求源文件为普通文件,在读取前后都检查大小上限,使用严格的 UTF-8 解码,并采用常规的临时 `didOpen`/请求/`didClose` 生命周期。传输失败会 dispose(资源释放)该实例,并在全新的远程进程上重试一次只读查询。 +每个提供方与规范化远程工作区的组合共享一个池化语言服务器进程。同一工作区的查询串行执行,不同工作区的查询并发运行。每项查询都会使用 `realpath` 规范化远程工作区与源文件,拒绝工作区外的路径,再由远程 helper 以不跟随最终符号链接的方式打开规范化源文件,并在同一个稳定描述符上完成验证与读取。该 helper 要求目标为普通文件,最多读取 `maxDocumentBytes + 1` 字节,随后执行严格的 UTF-8 解码。查询采用常规的临时 `didOpen`/请求/`didClose` 生命周期。传输失败会 dispose(资源释放)该实例,并在全新的远程进程上重试一次只读查询。 进程管理适配器负责进程组和终止升级,因此取消与资源释放都会等待远程服务器完全停稳。宿主负责 LSP 请求 id、待完成请求、提供方队列和规范化结果。 diff --git a/packages/e2b/lsp-e2b/src/index.ts b/packages/e2b/lsp-e2b/src/index.ts index b378ebfc9a..fa6b3729da 100644 --- a/packages/e2b/lsp-e2b/src/index.ts +++ b/packages/e2b/lsp-e2b/src/index.ts @@ -92,6 +92,46 @@ interface RemoteSource { text: string } +interface RemoteSourceReadResponse { + kind: 'ok' | 'not-file' | 'oversize' | 'grew' | 'open-error' + data?: string + size?: number + message?: string +} + +const SOURCE_READER_SOURCE = String.raw` +/* dsh-e2b-source-reader */ +const fs = require('node:fs') +const path = process.argv[1] +const maxBytes = Number(process.argv[2]) +let descriptor +let response +try { + descriptor = fs.openSync(path, fs.constants.O_RDONLY | fs.constants.O_NOFOLLOW | fs.constants.O_NONBLOCK) + const info = fs.fstatSync(descriptor) + if (!info.isFile()) response = { kind: 'not-file' } + else if (info.size > maxBytes) response = { kind: 'oversize', size: info.size } + else { + const chunks = [] + let total = 0 + while (total <= maxBytes) { + const chunk = Buffer.allocUnsafe(Math.min(65536, maxBytes - total + 1)) + const bytesRead = fs.readSync(descriptor, chunk, 0, chunk.length, null) + if (bytesRead === 0) break + chunks.push(chunk.subarray(0, bytesRead)) + total += bytesRead + } + response = total > maxBytes + ? { kind: 'grew' } + : { kind: 'ok', data: Buffer.concat(chunks, total).toString('base64') } + } +} catch (error) { + response = { kind: 'open-error', message: error instanceof Error ? error.message : String(error) } +} +if (descriptor !== undefined) fs.closeSync(descriptor) +process.stdout.write(JSON.stringify(response)) +` + function abortReason(signal: AbortSignal): unknown { try { signal.throwIfAborted() @@ -164,7 +204,8 @@ export async function canonicalizeE2BWorkspace( * @param sandbox - Shared sandbox that owns the source. * @param filePath - Absolute path or path relative to the canonical workspace. * @param workspace - Canonical remote workspace directory. - * @param maxDocumentBytes - Maximum source size before and after reading. + * @param maxDocumentBytes - Maximum bytes read through the stable remote handle. + * @param nodeExecutable - Resolved remote Node executable used by the bounded reader. * @param signal - Optional query cancellation signal. * @returns The canonical source path and decoded text. */ @@ -173,6 +214,7 @@ export async function readE2BSource( filePath: string, workspace: string, maxDocumentBytes: number, + nodeExecutable: string, signal?: AbortSignal, ): Promise { const requested = posix.isAbsolute(filePath) ? filePath : posix.resolve(workspace, filePath) @@ -181,16 +223,39 @@ export async function readE2BSource( if (relative === '..' || relative.startsWith('../') || posix.isAbsolute(relative)) { throw new Error(`source ${JSON.stringify(filePath)} resolves outside the workspace`) } - const info = await sandbox.files.getInfo(canonicalPath, signal === undefined ? {} : { signal }) - if (info.type !== FileType.FILE) throw new Error(`source ${JSON.stringify(filePath)} is not a regular file`) - if (info.size > maxDocumentBytes) { - throw new Error(`source ${JSON.stringify(filePath)} is ${info.size} bytes, over the ${maxDocumentBytes}-byte limit`) - } - const bytes = await sandbox.files.read(canonicalPath, { format: 'bytes', ...signal === undefined ? {} : { signal } }) + const command = [ + quoteE2BShellArg(nodeExecutable), + '--input-type=commonjs', + '-e', + quoteE2BShellArg(SOURCE_READER_SOURCE), + quoteE2BShellArg(canonicalPath), + String(maxDocumentBytes), + ].join(' ') + const result = await sandbox.commands.run(command, signal === undefined ? {} : { signal }) signal?.throwIfAborted() - if (bytes.length > maxDocumentBytes) { + let response: RemoteSourceReadResponse + try { + response = JSON.parse(result.stdout) as RemoteSourceReadResponse + } catch (error: unknown) { + throw new Error(`source ${JSON.stringify(filePath)} reader returned an invalid response`, { cause: error }) + } + if (response.kind === 'not-file') throw new Error(`source ${JSON.stringify(filePath)} is not a regular file`) + if (response.kind === 'oversize' && Number.isSafeInteger(response.size)) { + throw new Error(`source ${JSON.stringify(filePath)} is ${response.size} bytes, over the ${maxDocumentBytes}-byte limit`) + } + if (response.kind === 'grew') { throw new Error(`source ${JSON.stringify(filePath)} grew past the ${maxDocumentBytes}-byte limit while reading`) } + if (response.kind === 'open-error' && typeof response.message === 'string') { + throw new Error(`source ${JSON.stringify(filePath)} could not be opened safely: ${response.message}`) + } + if (response.kind !== 'ok' || typeof response.data !== 'string') { + throw new Error(`source ${JSON.stringify(filePath)} reader returned an invalid response`) + } + const bytes = Buffer.from(response.data, 'base64') + if (bytes.toString('base64') !== response.data || bytes.length > maxDocumentBytes) { + throw new Error(`source ${JSON.stringify(filePath)} reader returned invalid bounded bytes`) + } let text: string try { text = new TextDecoder('utf-8', { fatal: true }).decode(bytes) @@ -239,7 +304,14 @@ export class E2BLspProvider implements LspProvider { this.assertActive(signal) return this.enqueue(workspace, signal, async () => { this.assertActive(signal) - const source = await readE2BSource(this.sandbox, request.filePath, workspace, this.config.maxDocumentBytes, signal) + const source = await readE2BSource( + this.sandbox, + request.filePath, + workspace, + this.config.maxDocumentBytes, + this.nodeExecutable, + signal, + ) this.assertActive(signal) let instance = this.instanceFor(workspace) try { diff --git a/packages/e2b/lsp-e2b/tests/provider.spec.ts b/packages/e2b/lsp-e2b/tests/provider.spec.ts index d746abe7f7..dc92053e4a 100644 --- a/packages/e2b/lsp-e2b/tests/provider.spec.ts +++ b/packages/e2b/lsp-e2b/tests/provider.spec.ts @@ -94,11 +94,14 @@ class FakeRemote { readonly contents = new Map() readonly realpaths = new Map() forcedRealpath: string | undefined + readerResponse: unknown + readerOutput: string | undefined constructor() { this.infos.set('/workspace', { type: FileType.DIR, size: 0 }) this.infos.set('/workspace/file.ts', { type: FileType.FILE, size: 12 }) this.contents.set('/workspace/file.ts', Buffer.from('const x = 1')) + this.readerResponse = { kind: 'ok', data: Buffer.from('const x = 1').toString('base64') } } readonly sandbox = { @@ -110,6 +113,9 @@ class FakeRemote { const requested = match?.[1] ?? '' return { exitCode: 0, stdout: `${this.forcedRealpath ?? this.realpaths.get(requested) ?? requested}\n`, stderr: '' } } + if (command.includes('dsh-e2b-source-reader')) { + return { exitCode: 0, stdout: this.readerOutput ?? JSON.stringify(this.readerResponse), stderr: '' } + } if (command.startsWith('command -v')) return { exitCode: 0, stdout: '/usr/bin/node\n', stderr: '' } return { exitCode: 0, stdout: '', stderr: '' } }, @@ -172,16 +178,16 @@ describe('E2B LSP filesystem boundary', () => { it('canonicalizes a directory and reads a contained UTF-8 source', async () => { const remote = new FakeRemote() await expect(canonicalizeE2BWorkspace(remote.sandbox, '/workspace')).resolves.toBe('/workspace') - await expect(readE2BSource(remote.sandbox, 'file.ts', '/workspace', 1_024)).resolves.toEqual({ + await expect(readE2BSource(remote.sandbox, 'file.ts', '/workspace', 1_024, '/usr/bin/node')).resolves.toEqual({ canonicalPath: '/workspace/file.ts', text: 'const x = 1', }) - await expect(readE2BSource(remote.sandbox, '/workspace/file.ts', '/workspace', 1_024)).resolves.toMatchObject({ + await expect(readE2BSource(remote.sandbox, '/workspace/file.ts', '/workspace', 1_024, '/usr/bin/node')).resolves.toMatchObject({ canonicalPath: '/workspace/file.ts', }) const signal = new AbortController().signal await expect(canonicalizeE2BWorkspace(remote.sandbox, '/workspace', signal)).resolves.toBe('/workspace') - await expect(readE2BSource(remote.sandbox, 'file.ts', '/workspace', 1_024, signal)).resolves.toMatchObject({ + await expect(readE2BSource(remote.sandbox, 'file.ts', '/workspace', 1_024, '/usr/bin/node', signal)).resolves.toMatchObject({ canonicalPath: '/workspace/file.ts', }) }) @@ -199,25 +205,35 @@ describe('E2B LSP filesystem boundary', () => { const outside = new FakeRemote() outside.realpaths.set('/workspace/file.ts', '/outside/file.ts') - await expect(readE2BSource(outside.sandbox, 'file.ts', '/workspace', 20)).rejects.toThrow('outside the workspace') + await expect(readE2BSource(outside.sandbox, 'file.ts', '/workspace', 20, '/usr/bin/node')).rejects.toThrow('outside the workspace') const notFile = new FakeRemote() - notFile.infos.set('/workspace/file.ts', { type: FileType.DIR, size: 0 }) - await expect(readE2BSource(notFile.sandbox, 'file.ts', '/workspace', 20)).rejects.toThrow('not a regular file') + notFile.readerResponse = { kind: 'not-file' } + await expect(readE2BSource(notFile.sandbox, 'file.ts', '/workspace', 20, '/usr/bin/node')).rejects.toThrow('not a regular file') const tooLarge = new FakeRemote() - tooLarge.infos.set('/workspace/file.ts', { type: FileType.FILE, size: 21 }) - await expect(readE2BSource(tooLarge.sandbox, 'file.ts', '/workspace', 20)).rejects.toThrow('over the 20-byte limit') + tooLarge.readerResponse = { kind: 'oversize', size: 21 } + await expect(readE2BSource(tooLarge.sandbox, 'file.ts', '/workspace', 20, '/usr/bin/node')).rejects.toThrow('over the 20-byte limit') const grew = new FakeRemote() - grew.infos.set('/workspace/file.ts', { type: FileType.FILE, size: 1 }) - grew.contents.set('/workspace/file.ts', Buffer.alloc(21)) - await expect(readE2BSource(grew.sandbox, 'file.ts', '/workspace', 20)).rejects.toThrow('grew past') + grew.readerResponse = { kind: 'grew' } + await expect(readE2BSource(grew.sandbox, 'file.ts', '/workspace', 20, '/usr/bin/node')).rejects.toThrow('grew past') const invalid = new FakeRemote() - invalid.infos.set('/workspace/file.ts', { type: FileType.FILE, size: 1 }) - invalid.contents.set('/workspace/file.ts', Uint8Array.from([0xff])) - await expect(readE2BSource(invalid.sandbox, 'file.ts', '/workspace', 20)).rejects.toThrow('not valid UTF-8') + invalid.readerResponse = { kind: 'ok', data: Buffer.from([0xff]).toString('base64') } + await expect(readE2BSource(invalid.sandbox, 'file.ts', '/workspace', 20, '/usr/bin/node')).rejects.toThrow('not valid UTF-8') + + const swapped = new FakeRemote() + swapped.readerResponse = { kind: 'open-error', message: 'ELOOP: symbolic link encountered' } + await expect(readE2BSource(swapped.sandbox, 'file.ts', '/workspace', 20, '/usr/bin/node')).rejects.toThrow('opened safely') + + const malformedReader = new FakeRemote() + malformedReader.readerResponse = { kind: 'ok', data: '*' } + await expect(readE2BSource(malformedReader.sandbox, 'file.ts', '/workspace', 20, '/usr/bin/node')).rejects.toThrow('invalid bounded bytes') + malformedReader.readerResponse = { kind: 'oversize', size: 'large' } + await expect(readE2BSource(malformedReader.sandbox, 'file.ts', '/workspace', 20, '/usr/bin/node')).rejects.toThrow('invalid response') + malformedReader.readerOutput = '{' + await expect(readE2BSource(malformedReader.sandbox, 'file.ts', '/workspace', 20, '/usr/bin/node')).rejects.toThrow('invalid response') await expect(canonicalizeE2BWorkspace(new FakeRemote().sandbox, '/workspace', AbortSignal.abort('stop'))) .rejects.toBe('stop') diff --git a/packages/e2b/pty-e2b/README.i18n.yaml b/packages/e2b/pty-e2b/README.i18n.yaml index 1bd8485e8f..ddae4c21db 100644 --- a/packages/e2b/pty-e2b/README.i18n.yaml +++ b/packages/e2b/pty-e2b/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/e2b/pty-e2b/README.md -README.md: 3d363309f6bd1c4def5fbff668735958ca2de52d -README.zh.md: 78d61cc654bf6eb46a3bec7124b3cc06cc7e34e9 +README.md: d6d8e05f13701b85cee9a9646bfdd82ac07769da +README.zh.md: 265257c72e7bcbd2dfd805b343db5bab7173ab51 diff --git a/packages/e2b/pty-e2b/README.md b/packages/e2b/pty-e2b/README.md index 3d363309f6..d6d8e05f13 100644 --- a/packages/e2b/pty-e2b/README.md +++ b/packages/e2b/pty-e2b/README.md @@ -20,13 +20,13 @@ The `pty-e2b` plugin injects `e2b` and `pty`, then registers one backend under ` | `timeoutMs` | `30000` | Absolute startup and send wait bound. | | `disposeGraceMs` | `3000` | TERM-to-KILL cleanup grace. | -Numeric values are positive safe integers, `backendType` is non-empty, and `maxReadBytes` cannot exceed `scrollbackMaxBytes`. A relative spawn cwd resolves against `ctx.e2b.cwd`; an absolute remote path remains absolute. +Numeric values are positive safe integers, `backendType` is non-empty, and `maxReadBytes` cannot exceed `scrollbackMaxBytes`. A relative spawn cwd resolves against `ctx.e2b.cwd`; an absolute remote path remains absolute. Before launch, the backend enumerates sandbox-default environment names, blanks `DSH_*` and credential-shaped names, then overlays its controlled terminal values and explicit `spec.env` entries. ## Runtime contract The backend uses E2B's byte-oriented PTY callback with a streaming fatal UTF-8 decoder, then the backend-neutral line sanitizer and bounded buffers from `dsh-pty`. It installs a controlled Bash prompt marker and waits for printable prompt text; when that marker is unavailable, observed output plus the configured silence bound yields `inferred_idle`. Startup with no output reaches the absolute timeout and fails instead of publishing an empty session. -Each send writes UTF-8 bytes and an optional carriage-return submit sequence. Cancellation and explicit signals resolve the remote terminal's foreground process group through `ps`, then signal that group; `SIGKILL` refuses to target the shell itself. Close sends `SIGTERM` to the PTY process group, waits, escalates through E2B's PTY kill, and does not resolve until the SDK handle reports exit. A startup failure closes the unpublished PTY, and `PtyBackendCleanupError` preserves a concurrent cleanup failure. +Each send writes UTF-8 bytes and an optional carriage-return submit sequence. Cancellation and explicit signals resolve the remote terminal's foreground process group through `ps`, then signal that group; `SIGKILL` refuses to target the shell itself. The backend records the terminal's POSIX session id at startup. Close sends `SIGTERM` to every process group still in that session, escalates survivors to `SIGKILL`, verifies that the session is empty, and does not resolve until the SDK handle reports exit. A startup failure closes the unpublished PTY, and `PtyBackendCleanupError` preserves a concurrent cleanup failure. The remote PTY process and its child processes live in E2B. Prompt/readiness state, scrollback, operation handles, owner authority, and SDK event delivery remain in host memory. @@ -51,4 +51,5 @@ No direct invalidation; the consumer owns prompts, schemas, and appended results - **Line-oriented terminal model** — CSI/OSC control sequences are removed; alternate-screen and full terminal emulation remain unsupported. - **Readiness is marker-or-silence based** — E2B exposes foreground process groups but not the local backend's Linux syscall inspection, so `inferred_idle` is deliberately possible. - **UTF-8 only** — invalid byte sequences fail the session instead of returning lossy text. +- **Deliberate session escape is unmanaged** — a process that calls `setsid` leaves the terminal session and is outside this backend's cleanup identity. - **No reconnectable terminal handles** — retaining an E2B sandbox preserves remote files, not host ownership, buffers, callbacks, or live PTY sessions. diff --git a/packages/e2b/pty-e2b/README.zh.md b/packages/e2b/pty-e2b/README.zh.md index 78d61cc654..265257c72e 100644 --- a/packages/e2b/pty-e2b/README.zh.md +++ b/packages/e2b/pty-e2b/README.zh.md @@ -20,13 +20,13 @@ | `timeoutMs` | `30000` | 启动与发送等待的绝对上限。 | | `disposeGraceMs` | `3000` | TERM 到 KILL 的清理宽限期。 | -数值必须是正的安全整数,`backendType` 必须非空,且 `maxReadBytes` 不得超过 `scrollbackMaxBytes`。相对的 spawn cwd 以 `ctx.e2b.cwd` 为基准解析;绝对远程路径保持不变。 +数值必须是正的安全整数,`backendType` 必须非空,且 `maxReadBytes` 不得超过 `scrollbackMaxBytes`。相对的 spawn cwd 以 `ctx.e2b.cwd` 为基准解析;绝对远程路径保持不变。启动前,后端会枚举沙箱默认环境变量名,清空 `DSH_*` 和形似凭据的名称,再覆盖其受控终端值与显式 `spec.env` 条目。 ## 运行时契约 该后端为 E2B 面向字节的 PTY 回调配备流式、遇到无效序列即失败的 UTF-8 解码器,随后使用 `dsh-pty` 提供的后端无关行清理器与有界缓冲区。它会安装受控的 Bash 提示符标记,并等待可打印的提示符文本;若该标记不可用,系统会在已经观察到输出且达到已配置的静默上限时得出 `inferred_idle`。零输出的启动过程会达到绝对超时并失败,不会发布空会话。 -每次发送都会写入 UTF-8 字节,并可选写入回车提交序列。取消与显式信号会通过 `ps` 确定远程终端的前台进程组,再向该组发送信号;发送 `SIGKILL` 时拒绝以 shell 本身为目标。关闭操作向 PTY 进程组发送 `SIGTERM`,等待后通过 E2B 的 PTY kill 操作升级,并且直到 SDK 句柄报告退出才结算。如果启动失败,系统会关闭尚未发布的 PTY;若清理同时失败,`PtyBackendCleanupError` 会保留这项失败。 +每次发送都会写入 UTF-8 字节,并可选写入回车提交序列。取消与显式信号会通过 `ps` 确定远程终端的前台进程组,再向该组发送信号;发送 `SIGKILL` 时拒绝以 shell 本身为目标。后端会在启动时记录终端的 POSIX 会话 id。关闭操作会向该会话内仍存在的每个进程组发送 `SIGTERM`,对存活者升级为 `SIGKILL`,验证会话已经清空,并且直到 SDK 句柄报告退出才结算。如果启动失败,系统会关闭尚未发布的 PTY;若清理同时失败,`PtyBackendCleanupError` 会保留这项失败。 远程 PTY 进程及其子进程位于 E2B。提示符/就绪状态、scrollback、操作句柄、所有者权限和 SDK 事件交付仍保留在宿主内存中。 @@ -51,4 +51,5 @@ - **面向行的终端模型**:CSI/OSC 控制序列会被移除;备用屏幕与完整终端仿真仍不受支持。 - **就绪判断基于标记或静默**:E2B 会公开前台进程组,但不提供本地后端使用的 Linux syscall 检查,因此系统有意保留返回 `inferred_idle` 的可能性。 - **仅支持 UTF-8**:无效字节序列会使会话失败,而不是返回有损文本。 +- **主动逃离会话的进程不受管理**:调用 `setsid` 的进程会离开终端会话,因而不属于本后端的清理身份。 - **没有可重连的终端句柄**:保留 E2B 沙箱会保留远程文件,但不会保留宿主所有权、缓冲区、回调或实时 PTY 会话。 diff --git a/packages/e2b/pty-e2b/src/index.ts b/packages/e2b/pty-e2b/src/index.ts index 8fd7a519a0..d665a734b7 100644 --- a/packages/e2b/pty-e2b/src/index.ts +++ b/packages/e2b/pty-e2b/src/index.ts @@ -17,8 +17,22 @@ export const name = 'pty-e2b' /** Required shared sandbox owner and PTY registry. */ export const inject = ['e2b', 'pty'] -function terminalEnvironment(spec: PtyBackendSpawnSpec): Record { +const SENSITIVE_ENV_NAME = /KEY|SECRET|TOKEN/i + +async function terminalEnvironment( + sandbox: Sandbox, + spec: PtyBackendSpawnSpec, +): Promise> { + const discovered = await sandbox.commands.run( + 'env -0 | cut -z -d= -f1', + spec.signal === undefined ? {} : { signal: spec.signal }, + ) + spec.signal?.throwIfAborted() + const scrubbed = Object.fromEntries(discovered.stdout.split('\0') + .filter(name => name.startsWith('DSH_') || SENSITIVE_ENV_NAME.test(name)) + .map(name => [name, ''])) return { + ...scrubbed, TERM: 'dumb', PAGER: 'cat', GIT_PAGER: 'cat', @@ -31,6 +45,20 @@ function terminalEnvironment(spec: PtyBackendSpawnSpec): Record } } +async function resolveTerminalSessionId(sandbox: Sandbox, pid: number, signal?: AbortSignal): Promise { + const result = await sandbox.commands.run( + `ps -o sid= -p ${pid}`, + signal === undefined ? {} : { signal }, + ) + signal?.throwIfAborted() + const raw = result.stdout.trim() + const sessionId = Number(raw) + if (!/^[1-9][0-9]*$/.test(raw) || !Number.isSafeInteger(sessionId)) { + throw new Error(`pty-e2b: cannot resolve process session for E2B PTY ${pid}`) + } + return sessionId +} + /** E2B backend registered under the configured terminal type. */ export class E2BPtyBackend implements PtyBackend { readonly type: string @@ -57,7 +85,7 @@ export class E2BPtyBackend implements PtyBackend { rows: this.config.rows, cols: this.config.cols, cwd: posix.resolve(this.ctx.e2b.cwd, spec.cwd ?? this.ctx.e2b.cwd), - envs: terminalEnvironment(spec), + envs: await terminalEnvironment(sandbox, spec), timeoutMs: 0, ...spec.signal === undefined ? {} : { signal: spec.signal }, onData: (data) => { @@ -69,7 +97,15 @@ export class E2BPtyBackend implements PtyBackend { await handle.kill().catch(() => false) throw new Error(`pty-e2b: E2B returned invalid PTY pid ${handle.pid}`) } - const session = new E2BPtySession(sandbox, handle, this.config) + let terminalSessionId: number + try { + terminalSessionId = await resolveTerminalSessionId(sandbox, handle.pid, spec.signal) + } catch (error: unknown) { + await handle.kill().catch(() => false) + await Promise.allSettled([handle.wait()]) + throw error + } + const session = new E2BPtySession(sandbox, handle, terminalSessionId, this.config) created.session = session try { const initializing = session.initialize(spec.signal) diff --git a/packages/e2b/pty-e2b/src/session.ts b/packages/e2b/pty-e2b/src/session.ts index e5dfc05efe..bf8780103d 100644 --- a/packages/e2b/pty-e2b/src/session.ts +++ b/packages/e2b/pty-e2b/src/session.ts @@ -105,6 +105,7 @@ export class E2BPtySession implements PtyBackendSession { constructor( private readonly sandbox: Sandbox, private readonly handle: CommandHandle, + private readonly terminalSessionId: number, private readonly config: ResolvedConfig, ) { this.pid = handle.pid @@ -314,6 +315,40 @@ export class E2BPtySession implements PtyBackendSession { return pgid } + private async sessionProcessGroups(): Promise { + const result = await this.sandbox.commands.run( + `ps -eo sid=,pgid= | awk '$1 == ${this.terminalSessionId} { print $2 }'`, + ) + const groups = new Set() + for (const raw of result.stdout.trim().split(/\s+/)) { + if (raw.length === 0) continue + const pgid = Number(raw) + if (!/^[1-9][0-9]*$/.test(raw) || !Number.isSafeInteger(pgid) || pgid <= 1) { + throw new Error(`pty-e2b: invalid process group ${JSON.stringify(raw)} in terminal session ${this.terminalSessionId}`) + } + groups.add(pgid) + } + return [...groups] + } + + private async signalProcessGroups(groups: number[], signal: 'TERM' | 'KILL'): Promise { + try { + await this.sandbox.commands.run(`kill -${signal} -- ${groups.map(pgid => `-${pgid}`).join(' ')}`) + } catch (error: unknown) { + if (!(error instanceof CommandExitError)) throw error + } + } + + private async awaitSessionEmpty(timeoutMs: number, signal?: 'KILL'): Promise { + const deadline = Date.now() + timeoutMs + for (;;) { + const groups = await this.sessionProcessGroups() + if (groups.length === 0 || Date.now() >= deadline) return groups + if (signal !== undefined) await this.signalProcessGroups(groups, signal) + await delay(Math.min(this.config.pollIntervalMs, deadline - Date.now())) + } + } + private onExit(exitCode: number): void { this.remoteExited = true let tail = '' @@ -343,19 +378,20 @@ export class E2BPtySession implements PtyBackendSession { } private async closeOnce(reason: string): Promise { - if (!this.remoteExited) { + let survivingGroups = await this.sessionProcessGroups() + if (survivingGroups.length > 0) { this.closeSignal = 'SIGTERM' - try { - await this.sandbox.commands.run(`kill -TERM -- -${this.pid}`) - } catch (error: unknown) { - if (!(error instanceof CommandExitError)) throw error - } - await Promise.race([this.exited.promise, delay(this.config.disposeGraceMs)]) + await this.signalProcessGroups(survivingGroups, 'TERM') + survivingGroups = await this.awaitSessionEmpty(this.config.disposeGraceMs) } - if (!this.remoteExited) { + if (survivingGroups.length > 0 || !this.remoteExited) { this.closeSignal = 'SIGKILL' - await this.sandbox.pty.kill(this.pid) - await Promise.race([this.exited.promise, delay(this.config.disposeGraceMs)]) + if (!this.remoteExited) await this.sandbox.pty.kill(this.pid) + survivingGroups = await this.awaitSessionEmpty(this.config.disposeGraceMs, 'KILL') + if (!this.remoteExited) await Promise.race([this.exited.promise, delay(this.config.disposeGraceMs)]) + } + if (survivingGroups.length > 0) { + throw new Error(`E2B PTY cleanup failed (${reason}); surviving process groups: ${survivingGroups.join(', ')}`) } if (!this.remoteExited) { throw new Error(`E2B PTY cleanup failed (${reason}); surviving pid: ${this.pid}`) diff --git a/packages/e2b/pty-e2b/tests/index.spec.ts b/packages/e2b/pty-e2b/tests/index.spec.ts index e306d8b86b..476ddc38df 100644 --- a/packages/e2b/pty-e2b/tests/index.spec.ts +++ b/packages/e2b/pty-e2b/tests/index.spec.ts @@ -42,7 +42,10 @@ describe('E2BPtyBackend and plugin', () => { it('creates a remote PTY with isolated environment and initializes the session', async () => { vi.useFakeTimers() const ctx = new Context() - const sandbox = {} as Sandbox + const run = vi.fn(async (command: string) => command.startsWith('env -0') + ? { exitCode: 0, stdout: 'NPM_TOKEN\0DSH_STALE\0KEEP\0', stderr: '' } + : { exitCode: 0, stdout: '123\n', stderr: '' }) + const sandbox = { commands: { run } } as unknown as Sandbox ctx.provide('e2b', { cwd: '/workspace', getSandbox: async () => sandbox, @@ -65,9 +68,11 @@ describe('E2BPtyBackend and plugin', () => { expect(session.motd).toBe('banner\ndsh> ') expect(options).toMatchObject({ rows: 24, cols: 80, cwd: '/workspace/project', timeoutMs: 0 }) expect(options?.envs).toMatchObject({ + NPM_TOKEN: '', DSH_STALE: '', TERM: 'dumb', PAGER: 'cat', GIT_PAGER: 'cat', PS1: 'dsh> ', DSH_SHELL: '1', DSH_SESSION_ID: 'owner', DSH_PTY_SESSION_ID: 'pty-1', }) + expect(options?.envs).not.toHaveProperty('KEEP') vi.useRealTimers() }) @@ -79,7 +84,10 @@ describe('E2BPtyBackend and plugin', () => { setTimeout(() => { void received.onData(Buffer.from('\x1b]133;D;0\x07dsh> ')) }, 0) return created }) - const sandbox = { pty: { create } } as unknown as Sandbox + const sandbox = { + commands: { run: async (command: string) => ({ exitCode: 0, stdout: command.startsWith('env -0') ? '' : '123\n', stderr: '' }) }, + pty: { create }, + } as unknown as Sandbox ctx.provide('e2b', { cwd: '/workspace', getSandbox: async () => sandbox } as unknown as E2BSandboxService) const backend = new E2BPtyBackend(ctx, config()) const pending = backend.spawn({ sessionId: PtySessionId('default'), owner: owner(ctx), type: 'shell' }) @@ -91,7 +99,9 @@ describe('E2BPtyBackend and plugin', () => { it('rejects aborts and invalid pids, killing a malformed SDK handle', async () => { const ctx = new Context() - const sandbox = {} as Sandbox + const sandbox = { + commands: { run: async () => ({ exitCode: 0, stdout: '', stderr: '' }) }, + } as unknown as Sandbox ctx.provide('e2b', { cwd: '/workspace', getSandbox: async () => sandbox } as E2BSandboxService) const create = vi.fn().mockResolvedValue(handle(0)) const backend = new E2BPtyBackend(ctx, config(), create) @@ -109,13 +119,47 @@ describe('E2BPtyBackend and plugin', () => { const killFailure = handle(0, killFailureKill) const raced = new E2BPtyBackend(ctx, config(), async () => killFailure) await expect(raced.spawn({ sessionId: PtySessionId('three'), owner: owner(ctx), type: 'shell' })).rejects.toThrow('invalid PTY pid') + + const invalidSessionKill = vi.fn().mockRejectedValue(new Error('kill raced')) + const invalidSessionHandle = { + pid: 123, + wait: vi.fn().mockRejectedValue(new Error('already exited')), + kill: invalidSessionKill, + disconnect: vi.fn(), + } as unknown as CommandHandle + const invalidSessionSandbox = { + commands: { + run: async (command: string) => ({ + exitCode: 0, + stdout: command.startsWith('env -0') ? '' : '9007199254740992\n', + stderr: '', + }), + }, + } as unknown as Sandbox + const invalidSessionContext = new Context() + invalidSessionContext.provide('e2b', { + cwd: '/workspace', + getSandbox: async () => invalidSessionSandbox, + } as E2BSandboxService) + const invalidSession = new E2BPtyBackend(invalidSessionContext, config(), async () => invalidSessionHandle) + await expect(invalidSession.spawn({ + sessionId: PtySessionId('four'), owner: owner(invalidSessionContext), type: 'shell', + })) + .rejects.toThrow('cannot resolve process session') + expect(invalidSessionKill).toHaveBeenCalledOnce() }) it('cleans failed startup and aggregates a cleanup failure', async () => { vi.useFakeTimers() const ctx = new Context() const sandbox = { - commands: { run: vi.fn().mockResolvedValue({ exitCode: 0, stdout: '', stderr: '' }) }, + commands: { + run: vi.fn(async (command: string) => ({ + exitCode: 0, + stdout: command.startsWith('ps -o sid=') || command.startsWith('ps -eo sid=') ? '123\n' : '', + stderr: '', + })), + }, pty: { kill: vi.fn().mockRejectedValue(new Error('cleanup failed')) }, } as unknown as Sandbox ctx.provide('e2b', { cwd: '/workspace', getSandbox: async () => sandbox } as E2BSandboxService) @@ -141,10 +185,19 @@ describe('E2BPtyBackend and plugin', () => { wait: () => completion.promise, disconnect: vi.fn().mockResolvedValue(undefined), } as unknown as CommandHandle + let sessionRunning = true const sandbox = { commands: { run: vi.fn(async (command: string) => { - if (command.startsWith('kill -TERM')) completion.resolve({ exitCode: 143, stdout: '', stderr: '' }) + if (command.startsWith('env -0')) return { exitCode: 0, stdout: '', stderr: '' } + if (command.startsWith('ps -o sid=')) return { exitCode: 0, stdout: '123\n', stderr: '' } + if (command.startsWith('ps -eo sid=')) { + return { exitCode: 0, stdout: sessionRunning ? '123\n' : '', stderr: '' } + } + if (command.startsWith('kill -TERM')) { + sessionRunning = false + completion.resolve({ exitCode: 143, stdout: '', stderr: '' }) + } return { exitCode: 0, stdout: '', stderr: '' } }), }, diff --git a/packages/e2b/pty-e2b/tests/session.spec.ts b/packages/e2b/pty-e2b/tests/session.spec.ts index 6dced22ed2..554a4a3590 100644 --- a/packages/e2b/pty-e2b/tests/session.spec.ts +++ b/packages/e2b/pty-e2b/tests/session.spec.ts @@ -64,10 +64,12 @@ class FakeSandbox { readonly commands: string[] = [] readonly killed: number[] = [] pgid = '456\n' + sessionGroups = [123] sendError: unknown - commandError: unknown + signalError: unknown killError: unknown onTerm: (() => void) | undefined + onGroupKill: (() => void) | undefined onKill: (() => void) | undefined readonly sandbox = { @@ -86,13 +88,19 @@ class FakeSandbox { commands: { run: async (command: string): Promise => { this.commands.push(command) - if (this.commandError !== undefined) { - const error = this.commandError - this.commandError = undefined - throw error + if (command.startsWith('ps -o tpgid')) return { exitCode: 0, stdout: this.pgid, stderr: '' } + if (command.startsWith('ps -eo sid=')) { + return { exitCode: 0, stdout: this.sessionGroups.map(value => `${value}\n`).join(''), stderr: '' } + } + if (command.startsWith('kill -')) { + if (this.signalError !== undefined) { + const error = this.signalError + this.signalError = undefined + throw error + } + if (command.startsWith('kill -TERM')) this.onTerm?.() + if (command.startsWith('kill -KILL')) this.onGroupKill?.() } - if (command.startsWith('ps ')) return { exitCode: 0, stdout: this.pgid, stderr: '' } - if (command.startsWith('kill -TERM')) this.onTerm?.() return { exitCode: 0, stdout: '', stderr: '' } }, }, @@ -122,7 +130,7 @@ describe('E2BPtySession readiness, output, and signals', () => { vi.useFakeTimers() const fake = new FakeSandbox() const handle = new FakePtyHandle() - const session = new E2BPtySession(fake.sandbox, handle.asHandle(), config({ maxReadBytes: 12 })) + const session = new E2BPtySession(fake.sandbox, handle.asHandle(), 123, config({ maxReadBytes: 12 })) expect(session.read({})).toMatchObject({ text: '', totalLines: 0 }) await initialize(session) expect(session.motd).toBe('dsh> ') @@ -152,7 +160,7 @@ describe('E2BPtySession readiness, output, and signals', () => { vi.useFakeTimers() const fake = new FakeSandbox() const handle = new FakePtyHandle() - const session = new E2BPtySession(fake.sandbox, handle.asHandle(), config()) + const session = new E2BPtySession(fake.sandbox, handle.asHandle(), 123, config()) await initialize(session) const inferred = session.startSend({ text: '', submit: false }) @@ -176,7 +184,7 @@ describe('E2BPtySession readiness, output, and signals', () => { expect(() => session.startSend({ text: '', submit: false })).toThrow('has exited') const startupHandle = new FakePtyHandle() - const startup = new E2BPtySession(fake.sandbox, startupHandle.asHandle(), config()) + const startup = new E2BPtySession(fake.sandbox, startupHandle.asHandle(), 123, config()) const timedOut = expect(startup.initialize()).rejects.toThrow('startup timeout') await vi.advanceTimersByTimeAsync(100) await timedOut @@ -186,7 +194,7 @@ describe('E2BPtySession readiness, output, and signals', () => { vi.useFakeTimers() const fake = new FakeSandbox() const handle = new FakePtyHandle() - const session = new E2BPtySession(fake.sandbox, handle.asHandle(), config()) + const session = new E2BPtySession(fake.sandbox, handle.asHandle(), 123, config()) const initializing = session.initialize() session.onData(Buffer.from('\x1b]133;D;0\x07')) await vi.advanceTimersByTimeAsync(20) @@ -230,7 +238,7 @@ describe('E2BPtySession readiness, output, and signals', () => { vi.useFakeTimers() const fake = new FakeSandbox() const handle = new FakePtyHandle() - const session = new E2BPtySession(fake.sandbox, handle.asHandle(), config()) + const session = new E2BPtySession(fake.sandbox, handle.asHandle(), 123, config()) await initialize(session) const controller = new AbortController() @@ -268,7 +276,7 @@ describe('E2BPtySession readiness, output, and signals', () => { it('preserves startup abort reasons and classifies invalid UTF-8 transport failures', async () => { const fake = new FakeSandbox() const abortHandle = new FakePtyHandle() - const abortSession = new E2BPtySession(fake.sandbox, abortHandle.asHandle(), config()) + const abortSession = new E2BPtySession(fake.sandbox, abortHandle.asHandle(), 123, config()) const controller = new AbortController() const reason = new Error('startup cancelled') const initializing = abortSession.initialize(controller.signal) @@ -277,20 +285,20 @@ describe('E2BPtySession readiness, output, and signals', () => { await rejected const invalidHandle = new FakePtyHandle() - const invalid = new E2BPtySession(fake.sandbox, invalidHandle.asHandle(), config()) + const invalid = new E2BPtySession(fake.sandbox, invalidHandle.asHandle(), 123, config()) const pending = invalid.startSend({ text: '', submit: false }) invalid.onData(Uint8Array.from([0xff])) await expect(pending.done).rejects.toThrow('invalid UTF-8') expect(invalid.status()).toEqual({ kind: 'exited', exitCode: null, signal: null }) const crashHandle = new FakePtyHandle() - const crashed = new E2BPtySession(fake.sandbox, crashHandle.asHandle(), config()) + const crashed = new E2BPtySession(fake.sandbox, crashHandle.asHandle(), 123, config()) const active = crashed.startSend({ text: '', submit: false }) crashHandle.crash('transport gone') await expect(active.done).rejects.toEqual(new Error('transport gone')) const startupExitHandle = new FakePtyHandle() - const startupExit = new E2BPtySession(fake.sandbox, startupExitHandle.asHandle(), config()) + const startupExit = new E2BPtySession(fake.sandbox, startupExitHandle.asHandle(), 123, config()) const exitedDuringStartup = expect(startupExit.initialize()).rejects.toThrow('exited during startup') startupExitHandle.exit(7) await exitedDuringStartup @@ -300,12 +308,12 @@ describe('E2BPtySession readiness, output, and signals', () => { vi.useFakeTimers() const fake = new FakeSandbox() const tinyHandle = new FakePtyHandle() - const tiny = new E2BPtySession(fake.sandbox, tinyHandle.asHandle(), config({ maxReadBytes: 1 })) + const tiny = new E2BPtySession(fake.sandbox, tinyHandle.asHandle(), 123, config({ maxReadBytes: 1 })) tiny.onData(Buffer.from('你')) expect(tiny.read({ count: 1 })).toMatchObject({ text: '', lineEnd: 0 }) const handle = new FakePtyHandle() - const session = new E2BPtySession(fake.sandbox, handle.asHandle(), config()) + const session = new E2BPtySession(fake.sandbox, handle.asHandle(), 123, config()) const operation = session.startSend({ text: '', submit: false }) const internal = session as unknown as { pollReadiness(operation: PtySendOperation): void @@ -324,8 +332,8 @@ describe('E2BPtySession teardown', () => { vi.useFakeTimers() const fake = new FakeSandbox() const handle = new FakePtyHandle() - fake.onTerm = () => { handle.failExit(143) } - const session = new E2BPtySession(fake.sandbox, handle.asHandle(), config()) + fake.onTerm = () => { fake.sessionGroups = []; handle.failExit(143) } + const session = new E2BPtySession(fake.sandbox, handle.asHandle(), 123, config()) const first = session.close('done') expect(session.close('again')).toBe(first) await first @@ -334,63 +342,100 @@ describe('E2BPtySession teardown', () => { expect(() => session.startSend({ text: '', submit: false })).toThrow('closing') }) + it('escalates every job-control group that survives shell exit', async () => { + vi.useFakeTimers() + const fake = new FakeSandbox() + fake.sessionGroups = [123, 456] + const handle = new FakePtyHandle() + fake.onTerm = () => { fake.sessionGroups = [456]; handle.failExit(143) } + fake.onGroupKill = () => { fake.sessionGroups = [] } + const session = new E2BPtySession(fake.sandbox, handle.asHandle(), 123, config()) + + const closing = session.close('tree cleanup') + await vi.advanceTimersByTimeAsync(100) + await closing + + expect(fake.commands).toContain('kill -TERM -- -123 -456') + expect(fake.commands).toContain('kill -KILL -- -456') + }) + it('contains an already-gone TERM, escalates to KILL, and reports a survivor', async () => { vi.useFakeTimers() const gone = new FakeSandbox() const goneHandle = new FakePtyHandle() - gone.commandError = commandError(1) + gone.signalError = commandError(1) + gone.onGroupKill = () => { gone.sessionGroups = [] } gone.onKill = () => { goneHandle.failExit(137) } - const goneSession = new E2BPtySession(gone.sandbox, goneHandle.asHandle(), config()) + const goneSession = new E2BPtySession(gone.sandbox, goneHandle.asHandle(), 123, config()) const closingGone = goneSession.close('gone') - await vi.advanceTimersByTimeAsync(20) + await vi.advanceTimersByTimeAsync(100) await closingGone expect(gone.killed).toEqual([123]) expect(goneSession.status()).toEqual({ kind: 'exited', exitCode: null, signal: 'SIGKILL' }) const survivor = new FakeSandbox() const survivorHandle = new FakePtyHandle() - const survivorSession = new E2BPtySession(survivor.sandbox, survivorHandle.asHandle(), config()) - const failed = expect(survivorSession.close('still alive')).rejects.toThrow('surviving pid: 123') - await vi.advanceTimersByTimeAsync(40) + const survivorSession = new E2BPtySession(survivor.sandbox, survivorHandle.asHandle(), 123, config()) + const failed = expect(survivorSession.close('still alive')).rejects.toThrow('surviving process groups: 123') + await vi.advanceTimersByTimeAsync(100) await failed survivorHandle.exit() + survivor.sessionGroups = [] await expect(survivorSession.close('retry')).resolves.toBeUndefined() }) it('propagates cleanup transport failures and lets close retry', async () => { const fake = new FakeSandbox() const handle = new FakePtyHandle() - fake.commandError = new Error('TERM transport failed') - const session = new E2BPtySession(fake.sandbox, handle.asHandle(), config()) + fake.signalError = new Error('TERM transport failed') + const session = new E2BPtySession(fake.sandbox, handle.asHandle(), 123, config()) await expect(session.close('failure')).rejects.toThrow('TERM transport failed') handle.exit() + fake.sessionGroups = [] await expect(session.close('retry')).resolves.toBeUndefined() const invalidTailHandle = new FakePtyHandle() - const invalidTail = new E2BPtySession(fake.sandbox, invalidTailHandle.asHandle(), config()) + const invalidTail = new E2BPtySession(fake.sandbox, invalidTailHandle.asHandle(), 123, config()) invalidTail.onData(Uint8Array.from([0xe2])) invalidTailHandle.exit() await expect(invalidTail.close('invalid tail')).rejects.toThrow('invalid UTF-8') const normalHandle = new FakePtyHandle() normalHandle.disconnectError = new Error('disconnect raced') - const normal = new E2BPtySession(fake.sandbox, normalHandle.asHandle(), config()) + const normal = new E2BPtySession(fake.sandbox, normalHandle.asHandle(), 123, config()) normalHandle.exit(7) await Promise.resolve() expect(normal.status()).toEqual({ kind: 'exited', exitCode: 7, signal: null }) await expect(normal.close('already exited')).resolves.toBeUndefined() }) + it('rejects invalid session groups and a shell handle that survives SDK kill', async () => { + const invalid = new FakeSandbox() + invalid.sessionGroups = [1] + const invalidSession = new E2BPtySession(invalid.sandbox, new FakePtyHandle().asHandle(), 123, config()) + await expect(invalidSession.close('invalid group')).rejects.toThrow('invalid process group') + + vi.useFakeTimers() + const survivor = new FakeSandbox() + survivor.sessionGroups = [] + const survivorHandle = new FakePtyHandle() + const survivorSession = new E2BPtySession(survivor.sandbox, survivorHandle.asHandle(), 123, config()) + const failed = expect(survivorSession.close('shell survived')).rejects.toThrow('surviving pid: 123') + await vi.advanceTimersByTimeAsync(100) + await failed + expect(survivor.killed).toEqual([123]) + }) + it('kills a remotely live PTY after its host transport fails', async () => { const fake = new FakeSandbox() const handle = new FakePtyHandle() - const session = new E2BPtySession(fake.sandbox, handle.asHandle(), config()) + const session = new E2BPtySession(fake.sandbox, handle.asHandle(), 123, config()) const active = session.startSend({ text: '', submit: false }) session.onData(Uint8Array.from([0xff])) await expect(active.done).rejects.toThrow('invalid UTF-8') expect(session.status()).toEqual({ kind: 'exited', exitCode: null, signal: null }) - fake.onTerm = () => { handle.failExit(143) } + fake.onTerm = () => { fake.sessionGroups = []; handle.failExit(143) } await expect(session.close('transport failed')).rejects.toThrow('invalid UTF-8') expect(fake.commands).toContain('kill -TERM -- -123') expect(handle.disconnects).toBe(1) diff --git a/packages/e2b/subprocess-e2b/README.i18n.yaml b/packages/e2b/subprocess-e2b/README.i18n.yaml index 8b8a70838c..f05191a45b 100644 --- a/packages/e2b/subprocess-e2b/README.i18n.yaml +++ b/packages/e2b/subprocess-e2b/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/e2b/subprocess-e2b/README.md -README.md: 3b3bfa88e7e6483decfcdec11355942ae4ff7403 -README.zh.md: 3ff9a51c60636dea5789f9dd11b04aa902b91d0c +README.md: 066d35a099f560fe40fe629ede632c37535129f7 +README.zh.md: d3af14e8cc58c7d86331156358f78a0abbb24c39 diff --git a/packages/e2b/subprocess-e2b/README.md b/packages/e2b/subprocess-e2b/README.md index 3b3bfa88e7..066d35a099 100644 --- a/packages/e2b/subprocess-e2b/README.md +++ b/packages/e2b/subprocess-e2b/README.md @@ -6,8 +6,8 @@ E2B implementation of the [`@deepseek-ai/dsh-subprocess`](../../subprocess/subpr ## Behavior -- **Asynchronous remote start** — the synchronous seam returns a handle immediately while `Sandbox.commands.run(..., { background: true })` starts remotely. `pid` is `-1` until the SDK returns the command PID; `done`, stdin, termination, and `waitForExit()` wait for readiness internally. -- **Linux process groups** — a quoted wrapper starts each argv under `setsid --wait` and records its actual process-group id plus private status files beneath `ctx.e2b.runtimeRoot/processes`. The handle waits for that file instead of assuming the SDK command PID is the group id. Termination signals the negative recorded id with `SIGTERM`, waits the caller's `graceMs`, then escalates to `SIGKILL` and the SDK kill fallback. Service disposal terminates and joins every retained handle before the sandbox owner disposes. +- **Asynchronous remote start** — the synchronous seam returns a handle immediately while `Sandbox.commands.run(..., { background: true })` starts remotely. `pid` is `-1` until the wrapper publishes and the adapter validates its process-group id; `done`, stdin, termination, and `waitForExit()` wait for readiness internally. +- **Linux process groups** — a quoted wrapper starts each argv under `exec setsid --wait` and records its actual process-group id plus private status files beneath `ctx.e2b.runtimeRoot/processes`. The handle waits for that file instead of treating the SDK command PID as its published identity. Termination signals the negative recorded id with `SIGTERM`, waits the caller's `graceMs`, then escalates to `SIGKILL` and the SDK kill fallback. If publication fails, the SDK PID remains the provisional `exec setsid` group id; rollback kills and verifies that group before startup rejects. Service disposal terminates and joins every retained handle before the sandbox owner disposes. - **Environment boundary** — the wrapper starts from the sandbox command environment, removes ambient `DSH_*` and credential-shaped (`*KEY*`, `*SECRET*`, `*TOKEN*`) names, then restores every `spec.env` entry as an explicit caller opt-in. Host ambient variables never enter the sandbox implicitly. - **Stdio projection** — pipe mode forwards E2B callbacks into host Node streams; inherit mode forwards them to the harness process streams; collect mode retains a bounded host tail with offset reads. Optional complete spill files are written remotely and advertised only while within their cap. Batch and streaming stdin use the SDK handle. diff --git a/packages/e2b/subprocess-e2b/README.zh.md b/packages/e2b/subprocess-e2b/README.zh.md index 3ff9a51c60..d3af14e8cc 100644 --- a/packages/e2b/subprocess-e2b/README.zh.md +++ b/packages/e2b/subprocess-e2b/README.zh.md @@ -6,8 +6,8 @@ ## 行为 -- **异步远程启动**:同步 seam 会立即返回一个句柄,同时由 `Sandbox.commands.run(..., { background: true })` 在远程启动进程。SDK 返回命令 PID 之前,`pid` 为 `-1`;`done`、stdin、终止和 `waitForExit()` 会在内部等待就绪。 -- **Linux 进程组**:带引号保护的包装层会在 `setsid --wait` 下启动每组 argv,并在 `ctx.e2b.runtimeRoot/processes` 下记录实际进程组 ID 和私有状态文件。句柄会等待该文件,而不会假设 SDK 命令 PID 就是进程组 ID。终止操作以记录的负数 ID 发送 `SIGTERM`,等待调用方的 `graceMs`,再升级到 `SIGKILL` 和 SDK kill 回退。服务 dispose(资源释放)会在沙箱所有者释放前终止并等待每个保留句柄退出。 +- **异步远程启动**:同步 seam 会立即返回一个句柄,同时由 `Sandbox.commands.run(..., { background: true })` 在远程启动进程。包装层发布进程组 ID 并由适配器完成验证之前,`pid` 为 `-1`;`done`、stdin、终止和 `waitForExit()` 会在内部等待就绪。 +- **Linux 进程组**:带引号保护的包装层会在 `exec setsid --wait` 下启动每组 argv,并在 `ctx.e2b.runtimeRoot/processes` 下记录实际进程组 ID 和私有状态文件。句柄会等待该文件,而不会把 SDK 命令 PID 当作已发布的身份。终止操作以记录的负数 ID 发送 `SIGTERM`,等待调用方的 `graceMs`,再升级到 `SIGKILL` 和 SDK kill 回退。如果发布失败,SDK PID 仍为临时的 `exec setsid` 进程组 ID;回滚会终止并验证该进程组,随后启动操作才会以拒绝结束。服务 dispose(资源释放)会在沙箱所有者释放前终止并等待每个保留句柄退出。 - **环境边界**:包装层从沙箱命令环境开始,移除环境中的 `DSH_*` 和形似凭据的名称(`*KEY*`、`*SECRET*`、`*TOKEN*`),再把每个 `spec.env` 条目恢复为调用方显式选择。宿主环境变量绝不会隐式进入沙箱。 - **stdio 投影**:pipe 模式把 E2B 回调转发到宿主 Node 流;inherit 模式把回调转发到 harness 进程流;collect 模式保留有界的宿主尾部,并支持基于偏移量读取。可选的完整 spill 文件写在远程,并且只有未超过其上限时才会对外公布。批量 stdin 和流式 stdin 都使用 SDK 句柄。 diff --git a/packages/e2b/subprocess-e2b/src/process.ts b/packages/e2b/subprocess-e2b/src/process.ts index 86bdade589..3b9594a752 100644 --- a/packages/e2b/subprocess-e2b/src/process.ts +++ b/packages/e2b/subprocess-e2b/src/process.ts @@ -253,7 +253,14 @@ export class E2BSubprocessHandle implements SubprocessHandle { try { this.remotePid = await this.waitForProcessGroupId(sandbox, completion) } catch (error: unknown) { - await Promise.allSettled([handle.kill()]) + try { + await this.rollbackUnpublishedGroup(sandbox, handle) + } catch (cleanupError: unknown) { + throw new AggregateError( + [error, cleanupError], + 'subprocess-e2b: process-group publication failed and rollback did not reach quiescence', + ) + } throw error } this.readyState.resolve(handle) @@ -361,6 +368,19 @@ export class E2BSubprocessHandle implements SubprocessHandle { } } + private async rollbackUnpublishedGroup(sandbox: Sandbox, handle: CommandHandle): Promise { + // The background command begins with `exec setsid`, so E2B's command PID is + // the provisional group id even before the private publication file can be + // trusted. Kill that group before the SDK-PID fallback, then prove no group + // member survived before rejecting startup. + try { + await this.signalGroup(sandbox, handle.pid, 'KILL') + } finally { + await handle.kill().catch(() => false) + } + while (await this.groupAlive(sandbox, handle.pid)) await waitTick() + } + private async terminateRemote(): Promise { let handle: CommandHandle try { diff --git a/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts b/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts index 6be645a713..fd4cc97a83 100644 --- a/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts +++ b/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts @@ -89,6 +89,7 @@ class FakeSandbox { probeError: unknown signalError: unknown trapsTerm = false + delaysKill = false alive = true processGroupId = '4242\n' readonly processGroupReads: string[] = [] @@ -176,7 +177,7 @@ class FakeSandbox { this.signalError = undefined throw error } - this.alive = false + if (!this.delaysKill) this.alive = false this.handle.fail(137) return { exitCode: 0, stdout: '', stderr: '' } } @@ -596,14 +597,13 @@ describe('E2BSubprocessHandle', () => { it('rejects invalid or absent process-group publication', async () => { const invalidGroup = new FakeSandbox() invalidGroup.processGroupId = 'not-a-pid\n' - vi.spyOn(invalidGroup.handle, 'kill').mockImplementation(async () => { - invalidGroup.handle.kills += 1 - invalidGroup.finish() - return true - }) + invalidGroup.delaysKill = true + invalidGroup.afterProbe = () => { invalidGroup.alive = false } const invalid = new E2BSubprocessHandle(runtime(invalidGroup), spec(), '/runtime/invalid-group') await expect(invalid.done).rejects.toThrow(/invalid process-group id/) expect(invalidGroup.handle.kills).toBe(1) + expect(invalidGroup.commandsSeen).toContain('kill -KILL -- -4242') + await expect(invalid.waitForExit()).resolves.toBe(true) const absentGroup = new FakeSandbox() absentGroup.processGroupId = '' @@ -612,6 +612,35 @@ describe('E2BSubprocessHandle', () => { absentGroup.finish() await expect(absent.done).rejects.toThrow(/exited before publishing/) expect(absentGroup.handle.kills).toBe(1) + expect(absentGroup.commandsSeen).toContain('kill -KILL -- -4242') + await expect(absent.waitForExit()).resolves.toBe(true) + }) + + it('preserves publication and rollback failures when cleanup cannot be verified', async () => { + const fake = new FakeSandbox() + fake.processGroupId = 'not-a-pid\n' + fake.signalError = new Error('rollback signal failed') + fake.handle.killError = new Error('SDK kill failed') + const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/failed-rollback') + + let failure: unknown + try { + await handle.done + } catch (error: unknown) { + failure = error + } + expect(failure).toBeInstanceOf(AggregateError) + if (!(failure instanceof AggregateError)) throw new Error('expected AggregateError') + expect(failure.message).toBe('subprocess-e2b: process-group publication failed and rollback did not reach quiescence') + const failures = Array.from(failure.errors as Iterable) + expect(failures).toHaveLength(2) + expect(failures[0]).toBeInstanceOf(Error) + expect(failures[1]).toBeInstanceOf(Error) + if (!(failures[0] instanceof Error) || !(failures[1] instanceof Error)) throw new Error('expected nested errors') + expect(failures[0].message).toContain('invalid process-group id') + expect(failures[1].message).toBe('rollback signal failed') + expect(fake.handle.kills).toBe(1) + fake.finish() }) it('waits for delayed process-group publication', async () => { From 64b4669a742b975e717e1544316efee427925bad Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Tue, 28 Jul 2026 18:38:45 +0800 Subject: [PATCH 070/108] docs(packages): fit merged group map budget --- packages/README.i18n.yaml | 4 ++-- packages/README.md | 4 ++-- packages/README.zh.md | 4 ++-- 3 files changed, 6 insertions(+), 6 deletions(-) diff --git a/packages/README.i18n.yaml b/packages/README.i18n.yaml index b8320d8c19..a0827779db 100644 --- a/packages/README.i18n.yaml +++ b/packages/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/README.md -README.md: c46a08f76f9f38f9620c03f05f28e7698487afc8 -README.zh.md: 65a5c6c88b9b37b3f948b11c6db778e42965643d +README.md: 7dc52943e488a14e0c055ac72be0b2ad6f752455 +README.zh.md: bc3dbf0b4533d1d6adf6d49b0376a3dcdd6bfdb5 diff --git a/packages/README.md b/packages/README.md index b2a870f516..125b466d72 100644 --- a/packages/README.md +++ b/packages/README.md @@ -6,7 +6,7 @@ Packages use the `@deepseek-ai/dsh-*` scope. Cordis `Service` subclasses and fun ## Hierarchy -Packages live at `packages///`; groups are containers, while names remain `@deepseek-ai/dsh-`. **Each group README is the canonical package/ctx-key map.** +Groups contain packages at `packages///`; names remain `@deepseek-ai/dsh-`. **Group READMEs own package/ctx-key maps.** | Group | Role | Release expectation | |---|---|---| @@ -16,7 +16,7 @@ Packages live at `packages///`; groups are containers, while names r | [`goal/`](goal/README.md) | Same-session goal persistence and lifecycle | Product — stable surface | | [`feedback/`](feedback/README.md) | Human feedback | Product — stable surface | | [`llm/`](llm/README.md) | LLM capability family: the abstract service + provider adapters | Product — stable surface | -| [`e2b/`](e2b/README.md) | E2B remote-runtime provider family | POC | +| [`e2b/`](e2b/README.md) | E2B providers | POC | | [`subprocess/`](subprocess/README.md) | Subprocess capability family: spawn seam + local process-tree implementation | Product — stable surface | | [`bash/`](bash/README.md) | Bash capability family: executor seam, local impl, model-facing tool | Product — stable surface | | [`pty/`](pty/README.md) | Persistent PTY capability family: owner-scoped sessions, local implementation, and model-facing tools | Product — stable surface | diff --git a/packages/README.zh.md b/packages/README.zh.md index c8079c3d03..1667c88273 100644 --- a/packages/README.zh.md +++ b/packages/README.zh.md @@ -6,7 +6,7 @@ ## 层级结构 -包位于 `packages///`;组是容器,包名仍为 `@deepseek-ai/dsh-`。**每个组 README 是规范的包/ctx 键映射。** +组在 `packages///` 容纳包;包名仍为 `@deepseek-ai/dsh-`。**组 README 是规范的包/ctx 键映射。** | 组 | 职责 | 发布预期 | |---|---|---| @@ -16,7 +16,7 @@ | [`goal/`](goal/README.md) | 同会话 goal 的持久化与生命周期 | 产品:稳定表面 | | [`feedback/`](feedback/README.md) | 人类反馈 | 产品:稳定表面 | | [`llm/`](llm/README.md) | LLM(大语言模型)能力系列:抽象服务 + 提供方适配器 | 产品:稳定表面 | -| [`e2b/`](e2b/README.md) | E2B 远程运行时提供方系列 | POC | +| [`e2b/`](e2b/README.md) | E2B 提供方 | POC | | [`subprocess/`](subprocess/README.md) | 进程管理能力系列:spawn seam + 本地进程树实现 | 产品:稳定表面 | | [`bash/`](bash/README.md) | Bash 能力系列:执行器 seam、本地实现、面向模型的工具 | 产品:稳定表面 | | [`pty/`](pty/README.md) | 持久 PTY 能力系列:按所有者隔离的会话、本地实现和面向模型的工具 | 产品:稳定表面 | From 97496c3d00b4165dfb64d57b55d00e3462bedab4 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Tue, 28 Jul 2026 19:21:39 +0800 Subject: [PATCH 071/108] fix(e2b): close remaining remote lifecycle races --- ...026-07-27-e2b-remote-runtime-poc.i18n.yaml | 4 +- .../2026-07-27-e2b-remote-runtime-poc.md | 6 +- .../2026-07-27-e2b-remote-runtime-poc.zh.md | 6 +- .../tests/fixtures/e2b/e2b/bin.ts | 59 ++- .../e2b/code-runtime-e2b/README.i18n.yaml | 4 +- packages/e2b/code-runtime-e2b/README.md | 2 +- packages/e2b/code-runtime-e2b/README.zh.md | 2 +- .../e2b/code-runtime-e2b/src/runner-source.ts | 17 +- .../code-runtime-e2b/tests/runtime.spec.ts | 52 ++- packages/e2b/e2b/tests/composition.e2e.ts | 3 + packages/e2b/lsp-e2b/README.i18n.yaml | 6 - packages/e2b/lsp-e2b/README.md | 47 -- packages/e2b/lsp-e2b/README.zh.md | 47 -- packages/e2b/lsp-e2b/package.json | 52 --- packages/e2b/lsp-e2b/src/index.ts | 441 ------------------ packages/e2b/lsp-e2b/src/invariant.ts | 20 - packages/e2b/lsp-e2b/src/proxy-source.ts | 55 --- packages/e2b/lsp-e2b/src/transport.ts | 183 -------- packages/e2b/lsp-e2b/tests/provider.spec.ts | 418 ----------------- packages/e2b/lsp-e2b/tests/transport.spec.ts | 181 ------- packages/e2b/lsp-e2b/tsconfig.json | 20 - packages/e2b/pty-e2b/README.i18n.yaml | 4 +- packages/e2b/pty-e2b/README.md | 2 +- packages/e2b/pty-e2b/README.zh.md | 2 +- packages/e2b/pty-e2b/src/session.ts | 14 +- packages/e2b/pty-e2b/tests/session.spec.ts | 44 +- 26 files changed, 193 insertions(+), 1498 deletions(-) delete mode 100644 packages/e2b/lsp-e2b/README.i18n.yaml delete mode 100644 packages/e2b/lsp-e2b/README.md delete mode 100644 packages/e2b/lsp-e2b/README.zh.md delete mode 100644 packages/e2b/lsp-e2b/package.json delete mode 100644 packages/e2b/lsp-e2b/src/index.ts delete mode 100644 packages/e2b/lsp-e2b/src/invariant.ts delete mode 100644 packages/e2b/lsp-e2b/src/proxy-source.ts delete mode 100644 packages/e2b/lsp-e2b/src/transport.ts delete mode 100644 packages/e2b/lsp-e2b/tests/provider.spec.ts delete mode 100644 packages/e2b/lsp-e2b/tests/transport.spec.ts delete mode 100644 packages/e2b/lsp-e2b/tsconfig.json diff --git a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.i18n.yaml b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.i18n.yaml index beb49fba76..04ebe420ac 100644 --- a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.i18n.yaml +++ b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md -2026-07-27-e2b-remote-runtime-poc.md: efb91fc166f36dc8fb047d686e8cd82d93b9af6a -2026-07-27-e2b-remote-runtime-poc.zh.md: eb14cccc284d38cdc5cc704512b6899621a9a840 +2026-07-27-e2b-remote-runtime-poc.md: 4e6414b7d919d765712ac9314ce73a69c5c6344e +2026-07-27-e2b-remote-runtime-poc.zh.md: 4d39ed7eb07cdd17c5bfa9155bc7efabaa798117 diff --git a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md index efb91fc166..4e6414b7d9 100644 --- a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md +++ b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md @@ -32,7 +32,7 @@ E2B owns the mutable filesystem, command and Bash processes, PTY shell and termi The host owns Cordis and plugin objects, the agent loop, agent/session/goal state, session logs and persistence, LLM calls, prompts and tools, authority decisions, skills, subagent orchestration, PTY buffers and readiness state, LSP JSON-RPC ids/queues/protocol state, Code Runtime type stripping/output accounting/binding dispatch, and E2B SDK/network orchestration. The overlay does not upload, mount, or synchronize the host workspace; identical cwd strings name independent host and remote directories. -Byte-sensitive protocols use the narrowest adapter required by E2B's callback shapes. PTY consumes the SDK's byte callback directly. LSP installs a bounded remote source reader, while Code Runtime keeps framed stdout in a launcher process isolated from the controller and worker descriptors. Their dependency-free helpers encode protocol payloads as validated newline-delimited base64 JSON, keeping E2B's decoded command callbacks on an ASCII transport. +Byte-sensitive protocols use the narrowest adapter required by E2B's callback shapes. PTY consumes the SDK's byte callback directly and carries send identity across asynchronous foreground-group lookup. LSP installs a bounded remote source reader that walks no-follow directory descriptors beneath the canonical workspace. Code Runtime keeps framed stdout in a launcher process isolated from the controller and worker descriptors, and gives each controller a process group that is killed before its inherited pipes drain. Their dependency-free helpers encode protocol payloads as validated newline-delimited base64 JSON, keeping E2B's decoded command callbacks on an ASCII transport. Retaining a sandbox preserves remote files and unmanaged remote state only. Reconnect does not reconstruct host PTY sessions, buffers, process handles, LSP connections or requests, code workers, binding calls, timers, output cursors, or locks. Managed groups terminate and join when their provider disposes before the shared owner pauses, leaves, or kills the sandbox. @@ -40,9 +40,9 @@ The POC has no session-persistence backend, template builder, volume, snapshot, ## Verification -Focused package suites pin owner lifecycle cleanup, filesystem semantics and commit metadata, subprocess process groups, configuration and verified publication rollback, byte framing and multibyte boundaries, PTY readiness/signals/default-environment scrubbing/terminal-session cleanup, stable bounded LSP source reads, Code Runtime binding and descriptor isolation, worker-pipe draining, hostile traffic, output limits, timeout/abort ordering, disposal to quiescence, and package-owned invariant registrations. Adjacent local-backend suites pin the shared PTY utilities and the LSP cross-namespace `processId` behavior. +Focused package suites pin owner lifecycle cleanup, filesystem semantics and commit metadata, subprocess process groups, configuration and verified publication rollback, byte framing and multibyte boundaries, PTY readiness/signal identity/default-environment scrubbing/terminal-session cleanup, descriptor-walked bounded LSP source reads, Code Runtime binding and descriptor isolation, worker and descendant-held pipe draining, hostile traffic, output limits, timeout/abort ordering, disposal to quiescence, and package-owned invariant registrations. Adjacent local-backend suites pin the shared PTY utilities and the LSP cross-namespace `processId` behavior. -A credential-gated Loader composition creates real E2B sandboxes and exercises FS-to-Bash and Bash-to-FS visibility, process-publication rollback, bounded spill output, PTY default-secret scrubbing and process-tree cleanup, stable bounded LSP source reads, Code Runtime host bindings and descriptor-isolated output accounting, wall timeout, abort, runner cleanup, host-workspace isolation, and final sandbox deletion. The same composition runs through source imports and built package exports. +A credential-gated Loader composition creates real E2B sandboxes and exercises FS-to-Bash and Bash-to-FS visibility, process-publication rollback, bounded spill output, PTY default-secret scrubbing, stale-interrupt identity, and process-tree cleanup, parent-swap-safe bounded LSP source reads, Code Runtime host bindings, descriptor-isolated output accounting, descendant-held pipe cleanup, wall timeout, abort, runner cleanup, host-workspace isolation, and final sandbox deletion. The same composition runs through source imports and built package exports. ## Alternatives considered diff --git a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.zh.md b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.zh.md index eb14cccc28..4d39ed7eb0 100644 --- a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.zh.md +++ b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.zh.md @@ -32,7 +32,7 @@ E2B 拥有可变文件系统、命令和 Bash 进程、PTY shell 与终端会话 宿主拥有 Cordis 与插件对象、agent loop、agent/会话/goal 状态、会话日志及持久化、LLM(大语言模型)调用、提示词与工具、权限决策、skill(技能)、subagent 编排、PTY 缓冲与就绪状态、LSP JSON-RPC id/队列/协议状态、Code Runtime 类型剥离/输出计量/绑定分发,以及 E2B SDK/网络编排。该 overlay 不会上传、挂载或同步宿主工作区;拼写相同的 cwd 字符串分别指向彼此独立的宿主与远程目录。 -对字节敏感的协议只使用适配 E2B 回调形状所需的最窄适配器。PTY 直接消费 SDK 的字节回调。LSP 会安装一个有界的远程源码读取器;Code Runtime 则把分帧 stdout 保留在与 controller 和 worker 描述符隔离的 launcher 进程内。它们的无依赖辅助程序会把协议载荷编码为经过验证、以换行分隔的 base64 JSON,并通过 ASCII 传输承载 E2B 已解码的命令回调。 +对字节敏感的协议只使用适配 E2B 回调形状所需的最窄适配器。PTY 直接消费 SDK 的字节回调,并在异步查找前台进程组的过程中保留发送操作身份。LSP 会安装一个有界的远程源码读取器,通过不跟随符号链接打开的目录描述符在规范化工作区下逐级遍历。Code Runtime 则把分帧 stdout 保留在与 controller 和 worker 描述符隔离的 launcher 进程内,并为每个 controller 分配一个进程组,在 controller 继承的管道排空前终止该组。它们的无依赖辅助程序会把协议载荷编码为经过验证、以换行分隔的 base64 JSON,并通过 ASCII 传输承载 E2B 已解码的命令回调。 保留沙箱只会保存远程文件与未受管的远程状态。重新连接不会重建宿主 PTY 会话、缓冲、进程句柄、LSP 连接或请求、代码 worker、绑定调用、定时器、输出游标或锁。受管进程组会在所属提供方 dispose(资源释放)时终止并等待退出,之后共享所有者才会暂停、脱离或终止沙箱。 @@ -40,9 +40,9 @@ E2B 拥有可变文件系统、命令和 Bash 进程、PTY shell 与终端会话 ## 验证 -聚焦包测试套件固定所有者生命周期清理、文件系统语义与提交元数据、进程管理的进程组、配置与经过验证的发布回滚、字节分帧与多字节边界、PTY 就绪状态/信号/默认环境清理/终端会话清理、稳定且有界的 LSP 源码读取、Code Runtime 绑定与描述符隔离、worker 管道排空、恶意通信、输出上限、超时/中止顺序、等待完全停稳的资源释放,以及包自有不变式注册。相邻本地后端测试套件固定共享 PTY 工具函数,以及 LSP 跨命名空间 `processId` 行为。 +聚焦包测试套件固定所有者生命周期清理、文件系统语义与提交元数据、进程管理的进程组、配置与经过验证的发布回滚、字节分帧与多字节边界、PTY 就绪状态/信号身份/默认环境清理/终端会话清理、基于描述符逐级遍历的有界 LSP 源码读取、Code Runtime 绑定与描述符隔离、worker 管道及后代进程所持管道的排空、恶意通信、输出上限、超时/中止顺序、等待完全停稳的资源释放,以及包自有不变式注册。相邻本地后端测试套件固定共享 PTY 工具函数,以及 LSP 跨命名空间 `processId` 行为。 -凭据门控的 Loader 组合会创建真实 E2B 沙箱,并演练 FS-to-Bash 与 Bash-to-FS 可见性、进程发布回滚、有界 spill 输出、PTY 默认秘密清理与进程树清理、稳定且有界的 LSP 源码读取、Code Runtime 宿主绑定与描述符隔离的输出记账、墙钟超时、中止、runner 清理、宿主工作区隔离,以及最终删除沙箱。同一组合分别通过源代码导入与已构建包导出运行。 +凭据门控的 Loader 组合会创建真实 E2B 沙箱,并演练 FS-to-Bash 与 Bash-to-FS 可见性、进程发布回滚、有界 spill 输出、PTY 默认秘密清理、陈旧中断身份与进程树清理、可抵御父目录替换的有界 LSP 源码读取、Code Runtime 宿主绑定、描述符隔离的输出记账、后代进程所持管道的清理、墙钟超时、中止、runner 清理、宿主工作区隔离,以及最终删除沙箱。同一组合分别通过源代码导入与已构建包导出运行。 ## 曾考虑的替代方案 diff --git a/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts b/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts index 4f2401d2c7..a997752711 100644 --- a/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts +++ b/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts @@ -147,18 +147,23 @@ try { workspaceRoot: process.cwd(), }) - const swappedSource = await ctx.fs.resolve('swapped-source.ts') - const swappedSourcePath = posix.join(process.cwd(), 'swapped-source.ts') - await ctx.fs.writeText(swappedSource, 'const safe = true\n', { kind: 'createIfAbsent' }) + const swappedParentPath = posix.join(process.cwd(), 'swapped-parent') + const swappedSourcePath = posix.join(swappedParentPath, 'source.ts') + const swappedOutsidePath = '/tmp/dsh-e2b-lsp-outside' + await sandbox.commands.run( + `mkdir -p -- ${quoteE2BShellArg(swappedParentPath)} ${quoteE2BShellArg(swappedOutsidePath)} && printf 'const safe = true\\n' > ${quoteE2BShellArg(swappedSourcePath)} && printf 'const outside = true\\n' > ${quoteE2BShellArg(posix.join(swappedOutsidePath, 'source.ts'))}`, + ) const remoteCommands = sandbox.commands as unknown as { run(command: string, options?: unknown): Promise<{ exitCode: number; stdout: string; stderr: string }> } const runRemoteCommand = remoteCommands.run.bind(sandbox.commands) let containmentFaultInjected = false remoteCommands.run = async (command, options) => { - if (!containmentFaultInjected && command.includes('dsh-e2b-source-reader') && command.includes('swapped-source.ts')) { + if (!containmentFaultInjected && command.includes('dsh-e2b-source-reader') && command.includes('swapped-parent/source.ts')) { containmentFaultInjected = true - await runRemoteCommand(`rm -f -- ${quoteE2BShellArg(swappedSourcePath)} && ln -s -- /etc/hosts ${quoteE2BShellArg(swappedSourcePath)}`) + await runRemoteCommand( + `rm -rf -- ${quoteE2BShellArg(swappedParentPath)} && ln -s -- ${quoteE2BShellArg(swappedOutsidePath)} ${quoteE2BShellArg(swappedParentPath)}`, + ) } return await runRemoteCommand(command, options) } @@ -166,7 +171,7 @@ try { try { await ctx.lsp.query({ operation: 'hover', - filePath: 'swapped-source.ts', + filePath: 'swapped-parent/source.ts', position: { line: 0, character: 1 }, workspaceRoot: process.cwd(), }) @@ -200,8 +205,28 @@ try { text: "printf 'PTY-你好\\n'", submit: true, }).done + const foregroundLookup = Promise.withResolvers() + let delayedForegroundLookup = false + remoteCommands.run = async (command, options) => { + if (!delayedForegroundLookup && command.startsWith('ps -o tpgid=')) { + delayedForegroundLookup = true + await foregroundLookup.promise + } + return await runRemoteCommand(command, options) + } + const staleInterrupt = ctx.pty.startSend(owner, terminal.sessionId, { text: 'sleep 0.2', submit: true }) + if (!staleInterrupt.cancel()) throw new Error('E2B PTY refused the stale-interrupt probe cancellation') + await staleInterrupt.done const sleeping = ctx.pty.startSend(owner, terminal.sessionId, { text: 'sleep 30', submit: true }) - await new Promise(resolveDelay => setTimeout(resolveDelay, 150)) + foregroundLookup.resolve(undefined) + const interruptIdentitySafe = await Promise.race([ + sleeping.done.then(() => false), + new Promise(resolveDelay => setTimeout(() => { resolveDelay(true) }, 300)), + ]) + remoteCommands.run = runRemoteCommand + if (!delayedForegroundLookup || !interruptIdentitySafe) { + throw new Error('E2B PTY stale interrupt affected its successor send') + } const terminalSignal = await ctx.pty.signal(owner, terminal.sessionId, 'SIGINT') const interrupted = await sleeping.done const stubborn = await ctx.pty.startSend(owner, terminal.sessionId, { @@ -296,6 +321,23 @@ try { `, bindings: [], }) + const descendantPipe = await ctx.codeRuntime.run({ + program: ` + const childProcess = await import('node:child_process') + const child = childProcess.spawn( + process.execPath, + ['-e', 'setInterval(() => {}, 1000)', 'dsh-code-runtime-descendant'], + { stdio: ['ignore', 'inherit', 'inherit'] }, + ) + return child.pid > 0 + `, + bindings: [], + }) + const descendantProcesses = await sandbox.commands.list() + const descendantCleanup = !descendantProcesses.some(processInfo => + JSON.stringify([processInfo.cmd, processInfo.args]).includes('dsh-code-runtime-descendant'), + ) + if (!descendantCleanup) throw new Error('E2B Code Runtime left a pipe-holding descendant alive') const timedOut = await ctx.codeRuntime.run({ program: 'await new Promise(() => {})', bindings: [], @@ -340,6 +382,7 @@ try { echo: terminalEcho, signal: terminalSignal, interrupted, + interruptIdentitySafe, treeCleanup: terminalTreeCleanup, scrollback: terminalScrollback.text, }, @@ -348,6 +391,8 @@ try { nativeOutput, descriptorOutput, inheritedOutput, + descendantPipe, + descendantCleanup, timedOut, aborted, oversizedBoot, diff --git a/packages/e2b/code-runtime-e2b/README.i18n.yaml b/packages/e2b/code-runtime-e2b/README.i18n.yaml index ba43d787b8..eb4e75f297 100644 --- a/packages/e2b/code-runtime-e2b/README.i18n.yaml +++ b/packages/e2b/code-runtime-e2b/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/e2b/code-runtime-e2b/README.md -README.md: 171e63a8c1ca70deb63e9860f5088b83401a37d0 -README.zh.md: b70984e765d32985436c15dd543263cc06a25715 +README.md: a8623f95d16b54b29e53bb9cf2c528b36f121283 +README.zh.md: 2b4a37864e3c6755a74f0d2ef6ce38aa39aae24b diff --git a/packages/e2b/code-runtime-e2b/README.md b/packages/e2b/code-runtime-e2b/README.md index 171e63a8c1..a8623f95d1 100644 --- a/packages/e2b/code-runtime-e2b/README.md +++ b/packages/e2b/code-runtime-e2b/README.md @@ -19,7 +19,7 @@ Every value is a positive safe integer. `maxOutputBytes` is at least four bytes, ## Execution and bridge contract -Setup uploads one dependency-free runner under `ctx.e2b.runtimeRoot` and resolves remote Node. For each run, the host wraps and type-strips erasable TypeScript with Node's `stripTypeScriptTypes`, then starts the runner in `ctx.e2b.cwd`. The runner keeps the framed host protocol in a launcher process, forks a controller whose stdout and stderr are bounded data pipes, and creates a fresh worker thread with an empty environment and heap limit. Model writes to native descriptors and inherited child output therefore cannot enter the frame stream; worker and controller pipes drain before the terminal frame. The worker measures active event-loop time and is destroyed after one completion. The enclosing E2B subprocess group is terminated and awaited after every result, timeout, abort, or disposal, so ordinary child processes in that group stop with the run. +Setup uploads one dependency-free runner under `ctx.e2b.runtimeRoot` and resolves remote Node. For each run, the host wraps and type-strips erasable TypeScript with Node's `stripTypeScriptTypes`, then starts the runner in `ctx.e2b.cwd`. The runner keeps the framed host protocol in a launcher process, forks a controller process group whose stdout and stderr are bounded data pipes, and creates a fresh worker thread with an empty environment and heap limit. Model writes to native descriptors and inherited child output therefore cannot enter the frame stream; completion kills the controller group before draining its pipes and emitting the terminal frame. The worker measures active event-loop time and is destroyed after one completion. The enclosing E2B subprocess group is terminated and awaited after every result, timeout, abort, or disposal, so ordinary child processes in either managed group stop with the run. The bridge uses validated newline-delimited base64 JSON frames because E2B subprocess callbacks expose decoded text. Binding arguments and resolutions use the worker runtime's iterative lossless-JSON wire shape; binding functions execute on the host and typed rejection classes are materialized inside the remote worker. The worker captures the JavaScript intrinsics that its adapter boundary invokes before model code runs, hardening binding transport, output accounting, and completion validation against mutation of those references. The host repeats message validation, call-id deduplication, lossless-JSON checks, and the outer-output ledger. diff --git a/packages/e2b/code-runtime-e2b/README.zh.md b/packages/e2b/code-runtime-e2b/README.zh.md index b70984e765..2b4a37864e 100644 --- a/packages/e2b/code-runtime-e2b/README.zh.md +++ b/packages/e2b/code-runtime-e2b/README.zh.md @@ -19,7 +19,7 @@ ## 执行与桥接契约 -设置阶段会在 `ctx.e2b.runtimeRoot` 下上传一个无依赖的 runner,并解析远程 Node。每次运行时,宿主会包装仅使用可擦除语法的 TypeScript,再用 Node 的 `stripTypeScriptTypes` 剥离类型,然后在 `ctx.e2b.cwd` 中启动 runner。runner 会把面向宿主的分帧协议保留在 launcher 进程内,派生一个以 stdout 和 stderr 作为有界数据管道的 controller,再创建一个具有空环境与堆上限的全新 worker 线程。因此,模型对原生描述符的写入和继承的子进程输出无法进入分帧流;worker 与 controller 管道会在发出终结帧前排空。worker 会测量事件循环活跃时间,并在一次运行结算后销毁。每当运行返回结果、超时、中止或因资源释放终止时,系统都会终止外围的 E2B 进程组并等待其退出,因此组内的普通子进程会随本次运行一同停止。 +设置阶段会在 `ctx.e2b.runtimeRoot` 下上传一个无依赖的 runner,并解析远程 Node。每次运行时,宿主会包装仅使用可擦除语法的 TypeScript,再用 Node 的 `stripTypeScriptTypes` 剥离类型,然后在 `ctx.e2b.cwd` 中启动 runner。runner 会把面向宿主的分帧协议保留在 launcher 进程内,派生一个以 stdout 和 stderr 作为有界数据管道的 controller 进程组,再创建一个具有空环境与堆上限的全新 worker 线程。因此,模型对原生描述符的写入和继承的子进程输出无法进入分帧流;运行结算会先终止 controller 进程组,再排空其管道并发出终结帧。worker 会测量事件循环活跃时间,并在一次运行结算后销毁。每当运行返回结果、超时、中止或因资源释放终止时,系统都会终止外围的 E2B 进程组并等待其退出,因此任一受管组内的普通子进程会随本次运行一同停止。 由于 E2B 进程管理回调公开的是已解码文本,桥接层使用经过验证、以换行分隔的 base64 JSON 帧。绑定参数与 resolve 值使用 worker 运行时的迭代式无损 JSON wire 形状;绑定函数在宿主执行,类型化的 reject 类则在远程 worker 内物化。worker 会在模型代码运行前捕获其适配器边界调用的 JavaScript intrinsic,从而增强绑定传输、输出记账与完成值验证对这些引用修改的抵御能力。宿主会再次执行消息验证、调用 id 去重和无损 JSON 检查,并用外层输出账本再次计量。 diff --git a/packages/e2b/code-runtime-e2b/src/runner-source.ts b/packages/e2b/code-runtime-e2b/src/runner-source.ts index e5cf6ebefa..527f21114e 100644 --- a/packages/e2b/code-runtime-e2b/src/runner-source.ts +++ b/packages/e2b/code-runtime-e2b/src/runner-source.ts @@ -35,6 +35,18 @@ const waitForChildExit = child => { return new Promise(resolve => { child.once('exit', resolve) }) } +const killControllerGroup = child => { + if (process.platform !== 'win32' && Number.isSafeInteger(child.pid)) { + try { + process.kill(-child.pid, 'SIGKILL') + } catch (error) { + if (!error || typeof error !== 'object' || error.code !== 'ESRCH') throw error + } + return + } + child.kill('SIGKILL') +} + const jsonStringBytes = text => Buffer.byteLength(JSON.stringify(text)) const truncateLog = (text, available) => { @@ -74,7 +86,7 @@ const runLauncher = () => { const stdoutDrained = waitForPipeDrain(current.stdout) const stderrDrained = waitForPipeDrain(current.stderr) const exited = waitForChildExit(current) - current.kill('SIGKILL') + killControllerGroup(current) await Promise.all([exited, stdoutDrained, stderrDrained]) }) : Promise.resolve() @@ -112,6 +124,7 @@ const runLauncher = () => { maxOutputBytes = message.maxOutputBytes controller = fork(fileURLToPath(import.meta.url), [], { env: { DSH_CODE_RUNTIME_CONTROLLER: '1' }, + detached: process.platform !== 'win32', execArgv: [], stdio: ['ignore', 'pipe', 'pipe', 'ipc'], }) @@ -173,7 +186,7 @@ const runLauncher = () => { }) } }) - input.on('close', () => { if (controller && !settling) controller.kill('SIGKILL') }) + input.on('close', () => { if (controller && !settling) killControllerGroup(controller) }) } const runController = () => { diff --git a/packages/e2b/code-runtime-e2b/tests/runtime.spec.ts b/packages/e2b/code-runtime-e2b/tests/runtime.spec.ts index 9743c361c5..f99485a929 100644 --- a/packages/e2b/code-runtime-e2b/tests/runtime.spec.ts +++ b/packages/e2b/code-runtime-e2b/tests/runtime.spec.ts @@ -1,8 +1,9 @@ import { spawn } from 'node:child_process' -import { mkdtemp, rm, writeFile } from 'node:fs/promises' +import { access, mkdtemp, rm, writeFile } from 'node:fs/promises' import { tmpdir } from 'node:os' import { join } from 'node:path' import { PassThrough, Writable } from 'node:stream' +import { setTimeout as delay } from 'node:timers/promises' import { Context } from 'cordis' import { describe, expect, it, vi } from 'vitest' import type { Sandbox } from '@deepseek-ai/dsh-e2b' @@ -271,6 +272,55 @@ describe('E2BCodeRuntime', () => { expect(Buffer.byteLength(nativeOutput)).toBe(expectedBytes) }) + it.skipIf(process.platform === 'win32')('reaps descendant-held controller pipes before completion', async () => { + const directory = await mkdtemp(join(tmpdir(), 'dsh-e2b-code-descendant-')) + const marker = join(directory, 'started') + const release = join(directory, 'release') + const childSource = ` + const fs = require('node:fs') + fs.writeFileSync(${JSON.stringify(marker)}, 'started') + const timer = setInterval(() => { + if (fs.existsSync(${JSON.stringify(release)})) clearInterval(timer) + }, 10) + ` + let running: ReturnType | undefined + try { + running = runInstalledRunner(` + const fs = await import('node:fs') + const childProcess = await import('node:child_process') + childProcess.spawn(process.execPath, ['-e', ${JSON.stringify(childSource)}], { + stdio: ['ignore', 'inherit', 'inherit'], + }) + while (!fs.existsSync(${JSON.stringify(marker)})) await new Promise(resolve => setTimeout(resolve, 5)) + return true + `) + const deadline = Date.now() + 2_000 + for (;;) { + try { + await access(marker) + break + } catch (error: unknown) { + if (Date.now() >= deadline) throw error + await delay(10) + } + } + const completed = await Promise.race([ + running.then(() => true), + delay(500).then(() => false), + ]) + await writeFile(release, '') + const { messages, stderr } = await running + + expect(completed).toBe(true) + expect(stderr).toBe('') + expect(messages.at(-1)).toEqual({ type: 'done', value: [true] }) + } finally { + await writeFile(release, '').catch(() => undefined) + await running?.catch(() => undefined) + await rm(directory, { recursive: true, force: true }) + } + }) + it('prepares the remote runner and returns logs and a lossless completion', async () => { const handle = new FakeHandle((message, current) => { if ((message as { type?: string }).type !== 'boot') return diff --git a/packages/e2b/e2b/tests/composition.e2e.ts b/packages/e2b/e2b/tests/composition.e2e.ts index 4f4ea19eb9..1bfbab1e8e 100644 --- a/packages/e2b/e2b/tests/composition.e2e.ts +++ b/packages/e2b/e2b/tests/composition.e2e.ts @@ -109,12 +109,15 @@ describe.skipIf(!process.env.E2B_API_KEY)('E2B live Loader composition', () => { echo: { waitReason: 'stdin_read', sessionStatus: { kind: 'running' } }, signal: { delivered: true }, interrupted: { sessionStatus: { kind: 'running' } }, + interruptIdentitySafe: true, treeCleanup: true, }, hostileOutput: { error: { kind: 'output-limit' } }, nativeOutput: { error: { kind: 'output-limit' } }, descriptorOutput: { error: { kind: 'output-limit' } }, inheritedOutput: { error: { kind: 'output-limit' } }, + descendantPipe: { value: true, logs: [] }, + descendantCleanup: true, timedOut: { error: { kind: 'timeout' } }, aborted: { error: { kind: 'abort', message: 'live abort' } }, oversizedBoot: { error: { kind: 'worker-exit' } }, diff --git a/packages/e2b/lsp-e2b/README.i18n.yaml b/packages/e2b/lsp-e2b/README.i18n.yaml deleted file mode 100644 index 066820d9fd..0000000000 --- a/packages/e2b/lsp-e2b/README.i18n.yaml +++ /dev/null @@ -1,6 +0,0 @@ -# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each -# side as of the last confirmed-consistent state. Both languages carry equal authority; -# after editing either side, bring the other along and re-record with: -# pnpm run verify-translation-pairing --write packages/e2b/lsp-e2b/README.md -README.md: 6cc118767adc5d2df855b009f48faab596928fb7 -README.zh.md: b384e9efd0e5c62b0ee6e69f73755e1bbe0192df diff --git a/packages/e2b/lsp-e2b/README.md b/packages/e2b/lsp-e2b/README.md deleted file mode 100644 index 6cc118767a..0000000000 --- a/packages/e2b/lsp-e2b/README.md +++ /dev/null @@ -1,47 +0,0 @@ -# @deepseek-ai/dsh-lsp-e2b - -English | [中文](README.zh.md) - -Generic E2B language-server backend for [`ctx.lsp`](../../lsp/lsp/README.md). It runs configured stdio servers and reads their source documents inside the shared `ctx.e2b` sandbox; the provider registry, normalized query results, queues, and protocol connection state remain on the host. - -## Plugin and configuration - -The `lsp-e2b` plugin injects `e2b`, `lsp`, and the concrete `dsh-subprocess-e2b` service. `servers` is a non-empty provider-id table: - -| Server key | Default | Meaning | -|---|---|---| -| `command` | required | Remote executable, absolute or resolved on the sandbox PATH at load. | -| `args` | `[]` | Remote server arguments. | -| `env` | `{}` | Explicit environment entries passed through the subprocess adapter. | -| `extensionToLanguage` | required | Lowercase leading-dot extension to LSP language id. | -| `initializationOptions` / `configuration` | `null` / `null` | Static initialize options and `workspace/configuration` answer. | -| `maxMessageBytes` | `16000000` | Largest LSP message accepted from the server. | -| `maxStderrBytes` | `1000000` | Retained raw server stderr tail. | -| `maxDocumentBytes` | `4000000` | Largest remote source opened for one query. | -| `shutdownTimeoutMs` | `5000` | Graceful protocol-shutdown budget. | -| `killGraceMs` | `2000` | Request-cancel and TERM-to-KILL grace. | - -Provider ids and commands are non-empty; numeric bounds are positive safe integers, and timer values cannot exceed Node's maximum timer delay. Setup uploads one owner-private proxy under `ctx.e2b.runtimeRoot`, resolves Node and every configured server executable remotely, then registers all providers atomically. - -## Remote protocol and filesystem - -E2B command callbacks are text, while LSP is byte-framed. The installed proxy therefore base64-frames raw server stdout, stderr, and stdin as newline-delimited ASCII JSON; the host validates and decodes every frame before handing bytes to the shared `LspInstance` protocol engine. `initialize.processId` is `null` because host and server do not share a process namespace. - -One language-server process is pooled per provider and canonical remote workspace. Queries serialize per workspace but different workspaces run concurrently. Each query canonicalizes the remote workspace and source with `realpath`, rejects paths outside that workspace, then uses a remote helper to open the canonical source without following the final symlink and to verify and read one stable descriptor. The helper requires a regular file and reads at most `maxDocumentBytes + 1` bytes before strict UTF-8 decoding. Queries use the ordinary transient `didOpen` / request / `didClose` lifecycle. A transport failure disposes the instance and retries the read-only query once on a fresh remote process. - -The subprocess adapter owns process groups and escalation, so cancellation and disposal await remote server quiescence. The host owns LSP request ids, pending requests, provider queues, and normalized results. - -## Model Experience - -Indirectly, through `@deepseek-ai/dsh-tool-lsp`, which exposes normalized semantic navigation and hover results without changing its model-facing schema. - -#### KV Cache effect - -No direct invalidation; `dsh-tool-lsp` owns request-prefix changes. - -## Known Limitations and Deferred Work - -- **Configured servers only** — this package does not install language servers, select presets, or synchronize a host workspace into E2B. -- **Host protocol state is not reconnectable** — retaining a sandbox does not restore provider queues, JSON-RPC requests, subprocess handles, or document lifecycle state. -- **SDK output retention remains** — ASCII framing preserves protocol bytes, but E2B and the subprocess adapter still retain callback output in host memory. -- **Sandbox policy is template-owned** — this provider adds no volume, snapshot, credential, or network-policy layer. diff --git a/packages/e2b/lsp-e2b/README.zh.md b/packages/e2b/lsp-e2b/README.zh.md deleted file mode 100644 index b384e9efd0..0000000000 --- a/packages/e2b/lsp-e2b/README.zh.md +++ /dev/null @@ -1,47 +0,0 @@ -# @deepseek-ai/dsh-lsp-e2b - -[English](README.md) | 中文 - -用于 [`ctx.lsp`](../../lsp/lsp/README.md) 的通用 E2B 语言服务器后端。它在共享的 `ctx.e2b` 沙箱内运行已配置的 stdio 服务器并读取其源文档;提供方注册表、规范化查询结果、队列和协议连接状态仍保留在宿主侧。 - -## 插件与配置 - -`lsp-e2b` 插件注入 `e2b`、`lsp` 和具体的 `dsh-subprocess-e2b` 服务。`servers` 是一张非空的提供方 id 表: - -| 服务器键 | 默认值 | 含义 | -|---|---|---| -| `command` | 必填 | 远程可执行文件:绝对路径,或在加载时通过沙箱 PATH 解析。 | -| `args` | `[]` | 远程服务器参数。 | -| `env` | `{}` | 经由进程管理适配器传入的显式环境条目。 | -| `extensionToLanguage` | 必填 | 小写、以点开头的扩展名到 LSP language id 的映射。 | -| `initializationOptions` / `configuration` | `null` / `null` | 静态初始化选项和 `workspace/configuration` 应答。 | -| `maxMessageBytes` | `16000000` | 从服务器接受的 LSP 消息大小上限。 | -| `maxStderrBytes` | `1000000` | 保留的服务器原始 stderr 尾部上限。 | -| `maxDocumentBytes` | `4000000` | 单次查询可打开的最大远程源文件。 | -| `shutdownTimeoutMs` | `5000` | 协议优雅关闭预算。 | -| `killGraceMs` | `2000` | 请求取消与 TERM 到 KILL 升级的宽限期。 | - -提供方 id 与命令必须非空;数值上限必须是正的安全整数,定时器取值不得超过 Node 的最大定时器延迟。设置阶段会在 `ctx.e2b.runtimeRoot` 下上传一个仅供所有者使用的私有代理,在远程解析 Node 和每个已配置服务器的可执行文件,再以原子方式注册所有提供方。 - -## 远程协议与文件系统 - -E2B 命令回调交付的是文本,而 LSP 按字节分帧。因此,已安装的代理会把服务器 stdout、stderr 和 stdin 的原始字节进行 base64 编码,封装为以换行分隔的 ASCII JSON 帧;宿主会验证并解码每一帧,再把字节交给共享的 `LspInstance` 协议引擎。`initialize.processId` 为 `null`,因为宿主与服务器不共享进程命名空间。 - -每个提供方与规范化远程工作区的组合共享一个池化语言服务器进程。同一工作区的查询串行执行,不同工作区的查询并发运行。每项查询都会使用 `realpath` 规范化远程工作区与源文件,拒绝工作区外的路径,再由远程 helper 以不跟随最终符号链接的方式打开规范化源文件,并在同一个稳定描述符上完成验证与读取。该 helper 要求目标为普通文件,最多读取 `maxDocumentBytes + 1` 字节,随后执行严格的 UTF-8 解码。查询采用常规的临时 `didOpen`/请求/`didClose` 生命周期。传输失败会 dispose(资源释放)该实例,并在全新的远程进程上重试一次只读查询。 - -进程管理适配器负责进程组和终止升级,因此取消与资源释放都会等待远程服务器完全停稳。宿主负责 LSP 请求 id、待完成请求、提供方队列和规范化结果。 - -## 模型体验 - -通过 `@deepseek-ai/dsh-tool-lsp` 间接影响模型;该包会公开规范化的语义导航与悬停结果,而不改变其面向模型的 schema。 - -#### KV Cache 影响 - -不会直接失效;请求前缀变更由 `dsh-tool-lsp` 负责。 - -## 已知限制与暂缓工作 - -- **仅支持已配置的服务器**:本包不会安装语言服务器、选择 preset,或把宿主工作区同步到 E2B。 -- **宿主协议状态不可重连**:保留沙箱并不会恢复提供方队列、JSON-RPC 请求、进程管理句柄或文档生命周期状态。 -- **SDK 仍会保留输出**:ASCII 分帧能保留协议字节,但 E2B 和进程管理适配器仍会在宿主内存中保留回调输出。 -- **沙箱策略归模板负责**:本提供方不会额外增加卷、快照、凭据或网络策略层。 diff --git a/packages/e2b/lsp-e2b/package.json b/packages/e2b/lsp-e2b/package.json deleted file mode 100644 index 0671341f3c..0000000000 --- a/packages/e2b/lsp-e2b/package.json +++ /dev/null @@ -1,52 +0,0 @@ -{ - "name": "@deepseek-ai/dsh-lsp-e2b", - "description": "E2B language-server provider for DeepSeek Harness", - "version": "0.0.1", - "private": true, - "type": "module", - "main": "lib/index.js", - "types": "lib/types/index.d.ts", - "exports": { - ".": { - "types": "./lib/types/index.d.ts", - "default": "./lib/index.js" - }, - "./invariant": { - "types": "./lib/types/invariant.d.ts", - "default": "./lib/invariant.js" - }, - "./src/*": "./src/*", - "./package.json": "./package.json" - }, - "files": [ - "lib/index.js", - "lib/invariant.js", - "lib/types/**/*.d.ts", - "lib/types/**/*.d.ts.map", - "src" - ], - "license": "BSD-3-Clause", - "peerDependencies": { - "@deepseek-ai/dsh-e2b": "^0.0.1", - "@deepseek-ai/dsh-invariants": "^0.0.1", - "@deepseek-ai/dsh-lsp": "^0.0.1", - "@deepseek-ai/dsh-lsp-local": "^0.0.1", - "@deepseek-ai/dsh-subprocess": "^0.0.1", - "@deepseek-ai/dsh-subprocess-e2b": "^0.0.1", - "@deepseek-ai/dsh-timeout": "^0.0.1", - "cordis": "^4.0.0-rc.7" - }, - "dependencies": { - "schemastery": "^3.18.0" - }, - "devDependencies": { - "@deepseek-ai/dsh-e2b": "workspace:^", - "@deepseek-ai/dsh-invariants": "workspace:^", - "@deepseek-ai/dsh-lsp": "workspace:^", - "@deepseek-ai/dsh-lsp-local": "workspace:^", - "@deepseek-ai/dsh-subprocess": "workspace:^", - "@deepseek-ai/dsh-subprocess-e2b": "workspace:^", - "@deepseek-ai/dsh-timeout": "workspace:^", - "cordis": "^4.0.0-rc.7" - } -} diff --git a/packages/e2b/lsp-e2b/src/index.ts b/packages/e2b/lsp-e2b/src/index.ts deleted file mode 100644 index fa6b3729da..0000000000 --- a/packages/e2b/lsp-e2b/src/index.ts +++ /dev/null @@ -1,441 +0,0 @@ -/** E2B filesystem and process backend for the harness LSP capability seam. */ - -import { Buffer } from 'node:buffer' -import { posix } from 'node:path' -import type { Context } from 'cordis' -import z from 'schemastery' -import { - FileType, - quoteE2BShellArg, - resolveE2BExecutable, -} from '@deepseek-ai/dsh-e2b' -import type { Sandbox } from '@deepseek-ai/dsh-e2b' -import { LspError, LspProviderId } from '@deepseek-ai/dsh-lsp' -import type { LspProvider, LspProviderQuery, LspQueryResult } from '@deepseek-ai/dsh-lsp' -import { LspInstance } from '@deepseek-ai/dsh-lsp-local' -import type { SubprocessSpawnSpec } from '@deepseek-ai/dsh-subprocess' -import E2BSubprocessService from '@deepseek-ai/dsh-subprocess-e2b' -import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout' -import { LSP_PROXY_SOURCE } from './proxy-source.ts' -import { E2BLspTransport } from './transport.ts' - -export { E2BLspTransport } from './transport.ts' - -/** Cordis plugin name. */ -export const name = 'lsp-e2b' -/** Services required by the remote provider. */ -export const inject = ['e2b', 'lsp', 'subprocess'] - -const DEFAULT_MAX_MESSAGE_BYTES = 16_000_000 -const DEFAULT_MAX_STDERR_BYTES = 1_000_000 -const DEFAULT_MAX_DOCUMENT_BYTES = 4_000_000 -const DEFAULT_SHUTDOWN_TIMEOUT_MS = 5_000 -const DEFAULT_KILL_GRACE_MS = 2_000 - -/* jscpd:ignore-start -- Loader requires each backend to expose its own statically walkable server schema. */ -/** One configured language server inside the shared E2B sandbox. */ -export interface LspE2BServerConfig { - /** Remote executable, absolute or resolved on the sandbox PATH. */ - command: string - /** Lowercase leading-dot extension to LSP language id. */ - extensionToLanguage: Record - /** Remote executable arguments. */ - args?: string[] - /** Explicit remote environment overrides. */ - env?: Record - /** Static `initialize` options. */ - initializationOptions?: unknown - /** Static answer to every `workspace/configuration` item. */ - configuration?: unknown - /** Largest LSP message accepted from the server. */ - maxMessageBytes?: number - /** Largest remote stderr tail retained for diagnostics. */ - maxStderrBytes?: number - /** Largest remote source opened for one query. */ - maxDocumentBytes?: number - /** Graceful LSP shutdown budget. */ - shutdownTimeoutMs?: number - /** Request-cancel and TERM-to-KILL grace. */ - killGraceMs?: number -} - -/** Plugin configuration. */ -export interface Config { - /** Non-empty provider-id to remote-server table. */ - servers: Record -} - -type ResolvedServerConfig = Required - -const ServerConfig: z = z.object({ - command: z.string().required(), - args: z.array(String).default([]), - env: z.dict(String).default({}), - extensionToLanguage: z.dict(String).required(), - initializationOptions: z.any().default(null), - configuration: z.any().default(null), - maxMessageBytes: z.number().default(DEFAULT_MAX_MESSAGE_BYTES), - maxStderrBytes: z.number().default(DEFAULT_MAX_STDERR_BYTES), - maxDocumentBytes: z.number().default(DEFAULT_MAX_DOCUMENT_BYTES), - shutdownTimeoutMs: z.number().max(MAX_TIMER_DELAY_MS).default(DEFAULT_SHUTDOWN_TIMEOUT_MS), - killGraceMs: z.number().max(MAX_TIMER_DELAY_MS).default(DEFAULT_KILL_GRACE_MS), -}) - -/** Schemastery plugin configuration. */ -export const Config: z = z.object({ - servers: z.dict(ServerConfig).required(), -}) -/* jscpd:ignore-end */ - -interface RemoteSource { - canonicalPath: string - text: string -} - -interface RemoteSourceReadResponse { - kind: 'ok' | 'not-file' | 'oversize' | 'grew' | 'open-error' - data?: string - size?: number - message?: string -} - -const SOURCE_READER_SOURCE = String.raw` -/* dsh-e2b-source-reader */ -const fs = require('node:fs') -const path = process.argv[1] -const maxBytes = Number(process.argv[2]) -let descriptor -let response -try { - descriptor = fs.openSync(path, fs.constants.O_RDONLY | fs.constants.O_NOFOLLOW | fs.constants.O_NONBLOCK) - const info = fs.fstatSync(descriptor) - if (!info.isFile()) response = { kind: 'not-file' } - else if (info.size > maxBytes) response = { kind: 'oversize', size: info.size } - else { - const chunks = [] - let total = 0 - while (total <= maxBytes) { - const chunk = Buffer.allocUnsafe(Math.min(65536, maxBytes - total + 1)) - const bytesRead = fs.readSync(descriptor, chunk, 0, chunk.length, null) - if (bytesRead === 0) break - chunks.push(chunk.subarray(0, bytesRead)) - total += bytesRead - } - response = total > maxBytes - ? { kind: 'grew' } - : { kind: 'ok', data: Buffer.concat(chunks, total).toString('base64') } - } -} catch (error) { - response = { kind: 'open-error', message: error instanceof Error ? error.message : String(error) } -} -if (descriptor !== undefined) fs.closeSync(descriptor) -process.stdout.write(JSON.stringify(response)) -` - -function abortReason(signal: AbortSignal): unknown { - try { - signal.throwIfAborted() - } catch (error: unknown) { - return error - } - return new DOMException('The operation was aborted', 'AbortError') -} - -function abortable(promise: Promise, signal?: AbortSignal): Promise { - if (signal === undefined) return promise - // AbortSignal permits opaque reasons, and callers observe the exact reason. - // eslint-disable-next-line @typescript-eslint/prefer-promise-reject-errors - if (signal.aborted) return Promise.reject(abortReason(signal)) - return new Promise((resolve, reject) => { - const onAbort = (): void => { - cleanup() - // eslint-disable-next-line @typescript-eslint/prefer-promise-reject-errors -- Preserve the signal's exact opaque reason. - reject(abortReason(signal)) - } - const cleanup = (): void => { signal.removeEventListener('abort', onAbort) } - signal.addEventListener('abort', onAbort, { once: true }) - void promise.then( - (value) => { cleanup(); resolve(value) }, - (error: unknown) => { cleanup(); reject(error instanceof Error ? error : new Error(String(error))) }, - ) - }) -} - -function validateServerConfig(providerId: string, config: ResolvedServerConfig): void { - if (config.command.length === 0) throw new Error(`lsp-e2b: servers.${providerId}.command must be non-empty`) - for (const name of ['maxMessageBytes', 'maxStderrBytes', 'maxDocumentBytes', 'shutdownTimeoutMs', 'killGraceMs'] as const) { - const value = config[name] - if (!Number.isSafeInteger(value) || value <= 0 || (name.endsWith('Ms') && value > MAX_TIMER_DELAY_MS)) { - throw new Error(`lsp-e2b: servers.${providerId}.${name} must be a positive safe integer${name.endsWith('Ms') ? ` no greater than ${MAX_TIMER_DELAY_MS}` : ''}`) - } - } -} - -async function canonicalRemotePath(sandbox: Sandbox, path: string, signal?: AbortSignal): Promise { - signal?.throwIfAborted() - const result = await sandbox.commands.run(`realpath -e -- ${quoteE2BShellArg(path)}`, signal === undefined ? {} : { signal }) - signal?.throwIfAborted() - const canonical = result.stdout.trim() - if (!posix.isAbsolute(canonical) || canonical.includes('\n')) throw new Error(`remote path ${JSON.stringify(path)} did not resolve canonically`) - return canonical -} - -/** - * Canonicalize and validate one workspace inside E2B. - * @param sandbox - Shared sandbox that owns the workspace. - * @param workspaceRoot - Remote workspace path supplied by the query. - * @param signal - Optional query cancellation signal. - * @returns The canonical remote directory path. - */ -export async function canonicalizeE2BWorkspace( - sandbox: Sandbox, - workspaceRoot: string, - signal?: AbortSignal, -): Promise { - const canonical = await canonicalRemotePath(sandbox, workspaceRoot, signal) - const info = await sandbox.files.getInfo(canonical, signal === undefined ? {} : { signal }) - signal?.throwIfAborted() - if (info.type !== FileType.DIR) throw new Error(`workspace root ${JSON.stringify(workspaceRoot)} is not a directory`) - return canonical -} - -/** - * Resolve, contain, and read one UTF-8 query source inside E2B. - * @param sandbox - Shared sandbox that owns the source. - * @param filePath - Absolute path or path relative to the canonical workspace. - * @param workspace - Canonical remote workspace directory. - * @param maxDocumentBytes - Maximum bytes read through the stable remote handle. - * @param nodeExecutable - Resolved remote Node executable used by the bounded reader. - * @param signal - Optional query cancellation signal. - * @returns The canonical source path and decoded text. - */ -export async function readE2BSource( - sandbox: Sandbox, - filePath: string, - workspace: string, - maxDocumentBytes: number, - nodeExecutable: string, - signal?: AbortSignal, -): Promise { - const requested = posix.isAbsolute(filePath) ? filePath : posix.resolve(workspace, filePath) - const canonicalPath = await canonicalRemotePath(sandbox, requested, signal) - const relative = posix.relative(workspace, canonicalPath) - if (relative === '..' || relative.startsWith('../') || posix.isAbsolute(relative)) { - throw new Error(`source ${JSON.stringify(filePath)} resolves outside the workspace`) - } - const command = [ - quoteE2BShellArg(nodeExecutable), - '--input-type=commonjs', - '-e', - quoteE2BShellArg(SOURCE_READER_SOURCE), - quoteE2BShellArg(canonicalPath), - String(maxDocumentBytes), - ].join(' ') - const result = await sandbox.commands.run(command, signal === undefined ? {} : { signal }) - signal?.throwIfAborted() - let response: RemoteSourceReadResponse - try { - response = JSON.parse(result.stdout) as RemoteSourceReadResponse - } catch (error: unknown) { - throw new Error(`source ${JSON.stringify(filePath)} reader returned an invalid response`, { cause: error }) - } - if (response.kind === 'not-file') throw new Error(`source ${JSON.stringify(filePath)} is not a regular file`) - if (response.kind === 'oversize' && Number.isSafeInteger(response.size)) { - throw new Error(`source ${JSON.stringify(filePath)} is ${response.size} bytes, over the ${maxDocumentBytes}-byte limit`) - } - if (response.kind === 'grew') { - throw new Error(`source ${JSON.stringify(filePath)} grew past the ${maxDocumentBytes}-byte limit while reading`) - } - if (response.kind === 'open-error' && typeof response.message === 'string') { - throw new Error(`source ${JSON.stringify(filePath)} could not be opened safely: ${response.message}`) - } - if (response.kind !== 'ok' || typeof response.data !== 'string') { - throw new Error(`source ${JSON.stringify(filePath)} reader returned an invalid response`) - } - const bytes = Buffer.from(response.data, 'base64') - if (bytes.toString('base64') !== response.data || bytes.length > maxDocumentBytes) { - throw new Error(`source ${JSON.stringify(filePath)} reader returned invalid bounded bytes`) - } - let text: string - try { - text = new TextDecoder('utf-8', { fatal: true }).decode(bytes) - } catch (error: unknown) { - throw new Error(`source ${JSON.stringify(filePath)} is not valid UTF-8 text`, { cause: error }) - } - return { canonicalPath, text } -} - -/** - * Encode one absolute remote Linux path as a host-independent file URI. - * @param path - Canonical POSIX path inside E2B. - * @returns The equivalent percent-encoded file URI. - */ -export function e2bFileUri(path: string): string { - if (!posix.isAbsolute(path)) throw new Error(`lsp-e2b: expected an absolute remote path, received ${JSON.stringify(path)}`) - return `file://${path.split('/').map(segment => encodeURIComponent(segment)).join('/')}` -} - -/* jscpd:ignore-start -- Provider identity mirrors the seam while remote source and process ownership stay local. */ -/** One pooled remote provider with an isolated server per canonical workspace. */ -export class E2BLspProvider implements LspProvider { - readonly id: LspProviderId - readonly extensionToLanguage: Readonly> - private readonly instances = new Map() - private readonly queues = new Map>() - private disposed = false - - constructor( - providerId: string, - private readonly sandbox: Sandbox, - private readonly subprocess: E2BSubprocessService, - private readonly config: ResolvedServerConfig, - private readonly executable: string, - private readonly nodeExecutable: string, - private readonly proxyPath: string, - ) { - this.id = LspProviderId(providerId) - this.extensionToLanguage = config.extensionToLanguage - } - /* jscpd:ignore-end */ - - async query(request: LspProviderQuery, signal?: AbortSignal): Promise { - this.assertActive(signal) - const workspace = await canonicalizeE2BWorkspace(this.sandbox, request.workspaceRoot, signal) - this.assertActive(signal) - return this.enqueue(workspace, signal, async () => { - this.assertActive(signal) - const source = await readE2BSource( - this.sandbox, - request.filePath, - workspace, - this.config.maxDocumentBytes, - this.nodeExecutable, - signal, - ) - this.assertActive(signal) - let instance = this.instanceFor(workspace) - try { - return await instance.query(request, source, signal) - } catch (error: unknown) { - if (!instance.isTransportFailure(error)) throw error - await instance.dispose() - this.evict(workspace, instance) - this.assertActive(signal) - instance = this.instanceFor(workspace) - return await instance.query(request, source, signal) - } finally { - if (instance.dead) { - await instance.dispose() - this.evict(workspace, instance) - } - } - }) - } - - /* jscpd:ignore-start -- Queue and pooling semantics are shared; transport failure and disposal identities differ. */ - /** Stop accepting work and await every remote server and queued query. */ - async disposeAll(): Promise { - this.disposed = true - const instances = [...this.instances.values()] - const queues = [...this.queues.values()] - this.instances.clear() - await Promise.all([...instances.map(instance => instance.dispose()), ...queues]) - this.queues.clear() - } - - private assertActive(signal?: AbortSignal): void { - if (this.disposed) throw new LspError('lsp-e2b provider is disposed', 'LSP_DISPOSED') - signal?.throwIfAborted() - } - - private enqueue(workspace: string, signal: AbortSignal | undefined, run: () => Promise): Promise { - const previous = this.queues.get(workspace) ?? Promise.resolve() - const result = abortable(previous, signal).then(run) - const tail = previous.then(() => result).then(() => undefined, () => undefined) - this.queues.set(workspace, tail) - void tail.then(() => { if (this.queues.get(workspace) === tail) this.queues.delete(workspace) }) - return result - } - - private instanceFor(workspace: string): LspInstance { - this.assertActive() - const existing = this.instances.get(workspace) - if (existing !== undefined) return existing - const created = this.createInstance(workspace) - this.instances.set(workspace, created) - return created - } - /* jscpd:ignore-end */ - - private createInstance(workspace: string): LspInstance { - return new LspInstance({ - command: this.executable, - args: this.config.args, - cwd: workspace, - env: this.config.env, - configuration: this.config.configuration, - initializationOptions: this.config.initializationOptions, - maxMessageBytes: this.config.maxMessageBytes, - maxStderrBytes: this.config.maxStderrBytes, - shutdownTimeoutMs: this.config.shutdownTimeoutMs, - killGraceMs: this.config.killGraceMs, - clientProcessId: null, - pathToFileUri: e2bFileUri, - }, (spec: SubprocessSpawnSpec) => { - const originalArgv = Buffer.from(JSON.stringify(spec.argv)).toString('base64') - const inner = this.subprocess.spawn({ - ...spec, - argv: [this.nodeExecutable, this.proxyPath, originalArgv], - stdio: { - stdin: 'pipe', - stdout: 'pipe', - stderr: { maxBytes: this.config.maxStderrBytes }, - }, - }) - const rawBound = Math.max(this.config.maxMessageBytes, this.config.maxStderrBytes) - return new E2BLspTransport(inner, rawBound * 2 + 1024, this.config.maxStderrBytes) - }) - } - - private evict(workspace: string, instance: LspInstance): void { - if (this.instances.get(workspace) === instance) this.instances.delete(workspace) - } -} - -/** Install the proxy, resolve remote commands, and atomically register providers. */ -export async function apply(ctx: Context, config: Config): Promise { - if (!(ctx.subprocess instanceof E2BSubprocessService)) { - throw new Error('lsp-e2b requires @deepseek-ai/dsh-subprocess-e2b as ctx.subprocess') - } - const subprocess = ctx.subprocess - const entries = Object.entries(config.servers) - if (entries.length === 0) throw new Error('lsp-e2b: servers must contain at least one server') - const sandbox = await ctx.e2b.getSandbox() - const proxyPath = posix.join(ctx.e2b.runtimeRoot, 'lsp-stdio-proxy.mjs') - await sandbox.files.write([{ path: proxyPath, data: LSP_PROXY_SOURCE }]) - await sandbox.commands.run(`chmod 600 -- ${quoteE2BShellArg(proxyPath)}`) - const nodeExecutable = await resolveE2BExecutable(sandbox, 'node') - const providers = await Promise.all(entries.map(async ([providerId, raw]) => { - if (providerId.trim() === '') throw new Error('lsp-e2b: server ids must be non-empty strings') - const resolved = raw as ResolvedServerConfig - validateServerConfig(providerId, resolved) - const executable = await resolveE2BExecutable(sandbox, resolved.command) - return new E2BLspProvider(providerId, sandbox, subprocess, resolved, executable, nodeExecutable, proxyPath) - })) - - /* jscpd:ignore-start -- Every provider table publishes atomically through the same registry contract. */ - ctx.effect(() => { - const disposers: Array<() => void> = [] - try { - for (const provider of providers) disposers.push(ctx.lsp.registerProvider(provider)) - } catch (error: unknown) { - for (const dispose of disposers.reverse()) dispose() - throw error - } - return async () => { - for (const dispose of disposers.reverse()) dispose() - await Promise.all(providers.map(provider => provider.disposeAll())) - } - }, 'lsp-e2b.registerProviders') - /* jscpd:ignore-end */ -} diff --git a/packages/e2b/lsp-e2b/src/invariant.ts b/packages/e2b/lsp-e2b/src/invariant.ts deleted file mode 100644 index 129cf0a6d4..0000000000 --- a/packages/e2b/lsp-e2b/src/invariant.ts +++ /dev/null @@ -1,20 +0,0 @@ -/** Package-owned invariant companion for `@deepseek-ai/dsh-lsp-e2b`. */ - -/* jscpd:ignore-start */ -import type { Context } from 'cordis' -import type { InvariantInstaller } from '@deepseek-ai/dsh-invariants' - -const PACKAGE_NAME = '@deepseek-ai/dsh-lsp-e2b' - -/** Cordis companion plugin name. */ -export const name = 'lsp-e2b-invariant' -/** Service required before the companion can reserve package ownership. */ -export const inject = ['invariants'] - -/** No runtime invariant: the LSP registry owns provider publication. */ -const install: InvariantInstaller = () => {} - -/** Register this package's invariant companion. */ -export const apply = (ctx: Context): Promise<() => void> => - Promise.resolve(ctx.invariants.register(PACKAGE_NAME, install)) -/* jscpd:ignore-end */ diff --git a/packages/e2b/lsp-e2b/src/proxy-source.ts b/packages/e2b/lsp-e2b/src/proxy-source.ts deleted file mode 100644 index 5b6fe42d07..0000000000 --- a/packages/e2b/lsp-e2b/src/proxy-source.ts +++ /dev/null @@ -1,55 +0,0 @@ -/** Dependency-free remote stdio proxy installed inside the E2B sandbox. */ - -/** - * Node program that base64-frames raw child stdio so E2B's text callbacks - * never decode the language server's byte stream. - */ -export const LSP_PROXY_SOURCE = String.raw`import { Buffer } from 'node:buffer' -import { spawn } from 'node:child_process' -import { createInterface } from 'node:readline' - -const emit = (message) => { - process.stdout.write(Buffer.from(JSON.stringify(message)).toString('base64') + '\n') -} - -let argv -try { - argv = JSON.parse(Buffer.from(process.argv[2], 'base64').toString('utf8')) - if (!Array.isArray(argv) || argv.length === 0 || argv.some(value => typeof value !== 'string')) throw new Error('invalid argv') -} catch (error) { - process.stderr.write('lsp-e2b proxy argv error: ' + String(error) + '\n') - process.exitCode = 125 - process.stdin.destroy() -} - -if (argv) { - const child = spawn(argv[0], argv.slice(1), { stdio: ['pipe', 'pipe', 'pipe'], env: process.env }) - const input = createInterface({ input: process.stdin, crlfDelay: Infinity }) - child.stdout.on('data', data => { emit({ type: 'stdout', data: data.toString('base64') }) }) - child.stderr.on('data', data => { emit({ type: 'stderr', data: data.toString('base64') }) }) - child.on('error', error => { - emit({ type: 'stderr', data: Buffer.from('language server spawn failed: ' + error.message).toString('base64') }) - }) - child.on('close', (code, signal) => { - emit({ type: 'exit', code, signal }) - input.close() - process.stdin.destroy() - process.exitCode = code === null ? 1 : code - }) - input.on('line', line => { - input.pause() - try { - const message = JSON.parse(Buffer.from(line, 'base64').toString('utf8')) - if (!message || message.type !== 'stdin' || typeof message.data !== 'string') throw new Error('invalid stdin frame') - const data = Buffer.from(message.data, 'base64') - if (data.toString('base64') !== message.data) throw new Error('invalid stdin base64') - if (child.stdin.write(data)) input.resume() - else child.stdin.once('drain', () => { input.resume() }) - } catch (error) { - process.stderr.write('lsp-e2b proxy stdin error: ' + String(error) + '\n') - child.kill('SIGTERM') - } - }) - input.on('close', () => { child.stdin.end() }) -} -` diff --git a/packages/e2b/lsp-e2b/src/transport.ts b/packages/e2b/lsp-e2b/src/transport.ts deleted file mode 100644 index ddd3445db8..0000000000 --- a/packages/e2b/lsp-e2b/src/transport.ts +++ /dev/null @@ -1,183 +0,0 @@ -/** Byte-faithful stdio transport over an E2B subprocess and ASCII/base64 frames. */ - -import { Buffer } from 'node:buffer' -import { PassThrough, Writable } from 'node:stream' -import { E2BFrameDecoder, encodeE2BFrame } from '@deepseek-ai/dsh-e2b' -import type { - SubprocessHandle, - SubprocessOutcome, - SubprocessOutputRead, - SubprocessOutputReader, -} from '@deepseek-ai/dsh-subprocess' - -function asError(error: unknown): Error { - return error instanceof Error ? error : new Error(String(error)) -} - -class ByteTailReader implements SubprocessOutputReader { - private chunks: Buffer[] = [] - private totalBytes = 0 - private retainedBytes = 0 - private dropped = false - - constructor(private readonly maxBytes: number) {} - - append(data: Buffer): void { - if (data.length === 0) return - this.chunks.push(data) - this.totalBytes += data.length - this.retainedBytes += data.length - while (this.retainedBytes > this.maxBytes && this.chunks.length > 0) { - const first = this.chunks[0] as Buffer - const excess = this.retainedBytes - this.maxBytes - if (first.length <= excess) { - this.chunks.shift() - this.retainedBytes -= first.length - } else { - this.chunks[0] = first.subarray(excess) - this.retainedBytes -= excess - } - this.dropped = true - } - } - - readFrom(fromByte: number): SubprocessOutputRead { - if (!Number.isSafeInteger(fromByte) || fromByte < 0) { - throw new Error('subprocess output offset must be a non-negative safe integer') - } - const retainedStart = this.totalBytes - this.retainedBytes - const lossy = fromByte < retainedStart - const start = lossy ? 0 : Math.min(this.retainedBytes, fromByte - retainedStart) - const bytes = Buffer.concat(this.chunks).subarray(start) - return { text: bytes.toString('utf8'), nextOffset: this.totalBytes, lossy: lossy || this.dropped && fromByte === 0 } - } -} - -class FramedInput extends Writable { - constructor(private readonly target: Writable) { - super() - target.on('error', (error: Error) => { this.destroy(error) }) - } - - override _write(chunk: Buffer, _encoding: BufferEncoding, callback: (error?: Error | null) => void): void { - this.target.write(encodeE2BFrame({ type: 'stdin', data: chunk.toString('base64') }), callback) - } - - override _final(callback: (error?: Error | null) => void): void { - this.target.end(callback) - } -} - -/** Subprocess handle that decodes a remote proxy's stdout/stderr byte frames. */ -export class E2BLspTransport implements SubprocessHandle { - readonly stdin: Writable - readonly stdout = new PassThrough() - readonly stderr = undefined - readonly collected: SubprocessHandle['collected'] - readonly done: Promise - private readonly stderrTail: ByteTailReader - private readonly decoder: E2BFrameDecoder - private failed: Error | undefined - - /** - * @param inner - E2B subprocess running the installed proxy. - * @param maxFrameBytes - Maximum decoded proxy frame size. - * @param maxStderrBytes - Retained raw language-server stderr tail. - */ - constructor( - private readonly inner: SubprocessHandle, - maxFrameBytes: number, - maxStderrBytes: number, - ) { - if (inner.stdin === undefined || inner.stdout === undefined) { - inner.terminate() - throw new Error('lsp-e2b: proxy subprocess dropped a piped stream') - } - this.stdin = new FramedInput(inner.stdin) - this.stderrTail = new ByteTailReader(maxStderrBytes) - this.collected = { stderr: this.stderrTail } - this.decoder = new E2BFrameDecoder(maxFrameBytes) - inner.stdout.on('data', (chunk: Buffer) => { this.onProxyData(chunk) }) - inner.stdout.on('error', (error: Error) => { this.fail(error) }) - this.done = inner.done.then( - (outcome) => { - this.finishFrames() - this.captureProxyStderr() - this.stdout.end() - if (this.failed !== undefined) throw this.failed - return outcome - }, - (error: unknown) => { - this.captureProxyStderr() - this.stdout.end() - throw error - }, - ) - void this.done.catch(() => {}) - } - - get pid(): number { - return this.inner.pid - } - - terminate(): void { - this.inner.terminate() - } - - async waitForExit(signal?: AbortSignal): Promise { - return await this.inner.waitForExit(signal) - } - - private onProxyData(chunk: Buffer): void { - if (this.failed !== undefined) return - let frames: unknown[] - try { - frames = this.decoder.push(chunk.toString('utf8')) - } catch (error: unknown) { - this.fail(asError(error)) - return - } - for (const frame of frames) this.dispatch(frame) - } - - private dispatch(frame: unknown): void { - if (typeof frame !== 'object' || frame === null) { - this.fail(new Error('lsp-e2b: proxy emitted a malformed frame')) - return - } - const record = frame as Record - if (record.type === 'exit' && (record.code === null || typeof record.code === 'number') && (record.signal === null || typeof record.signal === 'string')) return - if ((record.type !== 'stdout' && record.type !== 'stderr') || typeof record.data !== 'string') { - this.fail(new Error('lsp-e2b: proxy emitted a malformed frame')) - return - } - const data = Buffer.from(record.data, 'base64') - if (data.toString('base64') !== record.data) { - this.fail(new Error('lsp-e2b: proxy emitted invalid base64')) - return - } - if (record.type === 'stdout') this.stdout.write(data) - else this.stderrTail.append(data) - } - - private finishFrames(): void { - if (this.failed !== undefined) return - try { - this.decoder.finish() - } catch (error: unknown) { - this.fail(asError(error)) - } - } - - private captureProxyStderr(): void { - const diagnostic = this.inner.collected.stderr?.readFrom(0).text - if (diagnostic !== undefined && diagnostic.length > 0) this.stderrTail.append(Buffer.from(diagnostic)) - } - - private fail(error: Error): void { - if (this.failed !== undefined) return - this.failed = error - this.inner.terminate() - this.stdout.end() - } -} diff --git a/packages/e2b/lsp-e2b/tests/provider.spec.ts b/packages/e2b/lsp-e2b/tests/provider.spec.ts deleted file mode 100644 index dc92053e4a..0000000000 --- a/packages/e2b/lsp-e2b/tests/provider.spec.ts +++ /dev/null @@ -1,418 +0,0 @@ -import { PassThrough } from 'node:stream' -import { Context } from 'cordis' -import { beforeEach, describe, expect, it, vi } from 'vitest' -import { - FileType, - type Sandbox, -} from '@deepseek-ai/dsh-e2b' -import type { SubprocessHandle, SubprocessSpawnSpec } from '@deepseek-ai/dsh-subprocess' -import E2BSubprocessService from '@deepseek-ai/dsh-subprocess-e2b' - -const mockedLsp = vi.hoisted(() => { - interface Plan { - query?: (...args: unknown[]) => unknown - transportFailure?: unknown - dead?: boolean - deadAfterQuery?: boolean - disposeError?: unknown - } - - class FakeLspInstance { - static readonly instances: FakeLspInstance[] = [] - static readonly plans: Plan[] = [] - readonly plan: Plan - readonly transport: unknown - readonly queries: unknown[][] = [] - dead: boolean - disposals = 0 - - constructor( - readonly spec: Record, - spawner: (spec: SubprocessSpawnSpec) => unknown, - ) { - this.plan = FakeLspInstance.plans.shift() ?? {} - this.dead = this.plan.dead === true - this.transport = spawner({ - argv: [String(spec.command), ...(spec.args as string[])], - cwd: String(spec.cwd), - env: spec.env as Record, - graceMs: Number(spec.killGraceMs), - stdio: { stdin: 'pipe', stdout: 'pipe', stderr: { maxBytes: Number(spec.maxStderrBytes) } }, - }) - FakeLspInstance.instances.push(this) - } - - async query(...args: unknown[]): Promise { - this.queries.push(args) - const result = await Promise.resolve(this.plan.query?.(...args) ?? { kind: 'hover', hover: null }) - if (this.plan.deadAfterQuery === true) this.dead = true - return result - } - - isTransportFailure(error: unknown): boolean { - return error === this.plan.transportFailure - } - - async dispose(): Promise { - this.disposals += 1 - this.dead = true - if (this.plan.disposeError !== undefined) throw this.plan.disposeError - } - } - - return { FakeLspInstance } -}) - -vi.mock('@deepseek-ai/dsh-lsp-local', () => ({ LspInstance: mockedLsp.FakeLspInstance })) - -import { - E2BLspProvider, - apply, - canonicalizeE2BWorkspace, - e2bFileUri, - readE2BSource, -} from '@deepseek-ai/dsh-lsp-e2b' -import type { LspE2BServerConfig } from '@deepseek-ai/dsh-lsp-e2b' -import * as E2BLspInvariant from '../src/invariant.ts' -import InvariantService from '@deepseek-ai/dsh-invariants' - -class FakeInnerHandle implements SubprocessHandle { - readonly pid = 777 - readonly stdin = new PassThrough() - readonly stdout = new PassThrough() - readonly stderr = undefined - readonly collected = { stderr: { readFrom: () => ({ text: '', nextOffset: 0, lossy: false }) } } - readonly done = Promise.resolve({ exitCode: 0, signal: null }) - terminate(): void {} - async waitForExit(): Promise { return true } -} - -class FakeRemote { - readonly writes: Array> = [] - readonly commands: string[] = [] - readonly infos = new Map() - readonly contents = new Map() - readonly realpaths = new Map() - forcedRealpath: string | undefined - readerResponse: unknown - readerOutput: string | undefined - - constructor() { - this.infos.set('/workspace', { type: FileType.DIR, size: 0 }) - this.infos.set('/workspace/file.ts', { type: FileType.FILE, size: 12 }) - this.contents.set('/workspace/file.ts', Buffer.from('const x = 1')) - this.readerResponse = { kind: 'ok', data: Buffer.from('const x = 1').toString('base64') } - } - - readonly sandbox = { - commands: { - run: async (command: string) => { - this.commands.push(command) - if (command.startsWith('realpath ')) { - const match = /'([^']*)'$/.exec(command) - const requested = match?.[1] ?? '' - return { exitCode: 0, stdout: `${this.forcedRealpath ?? this.realpaths.get(requested) ?? requested}\n`, stderr: '' } - } - if (command.includes('dsh-e2b-source-reader')) { - return { exitCode: 0, stdout: this.readerOutput ?? JSON.stringify(this.readerResponse), stderr: '' } - } - if (command.startsWith('command -v')) return { exitCode: 0, stdout: '/usr/bin/node\n', stderr: '' } - return { exitCode: 0, stdout: '', stderr: '' } - }, - }, - files: { - write: async (files: Array<{ path: string; data: string }>) => { - this.writes.push(files) - return files.map(() => ({})) - }, - getInfo: async (path: string) => { - const info = this.infos.get(path) - if (info === undefined) throw new Error(`missing info for ${path}`) - return info - }, - read: async (path: string) => this.contents.get(path) ?? new Uint8Array(), - }, - } as unknown as Sandbox -} - -function subprocess(spawn = vi.fn((_spec: SubprocessSpawnSpec) => new FakeInnerHandle())): E2BSubprocessService { - const service = Object.create(E2BSubprocessService.prototype) as E2BSubprocessService - Object.defineProperty(service, 'spawn', { value: spawn }) - return service -} - -function server(overrides: Partial = {}): Required { - return { - command: '/usr/bin/server', args: ['--stdio'], env: {}, - extensionToLanguage: { '.ts': 'typescript' }, - initializationOptions: null, configuration: null, - maxMessageBytes: 1_024, maxStderrBytes: 128, maxDocumentBytes: 1_024, - shutdownTimeoutMs: 100, killGraceMs: 50, - ...overrides, - } -} - -function provider(remote = new FakeRemote(), service = subprocess()): E2BLspProvider { - return new E2BLspProvider( - 'fixture', remote.sandbox, service, server(), - '/usr/bin/server', '/usr/bin/node', '/workspace/.dsh-e2b/lsp-proxy.mjs', - ) -} - -function query(workspaceRoot = '/workspace') { - return { - operation: 'hover' as const, - filePath: 'file.ts', - position: { line: 0, character: 1 }, - workspaceRoot, - languageId: 'typescript', - } -} - -beforeEach(() => { - mockedLsp.FakeLspInstance.instances.length = 0 - mockedLsp.FakeLspInstance.plans.length = 0 -}) - -describe('E2B LSP filesystem boundary', () => { - it('canonicalizes a directory and reads a contained UTF-8 source', async () => { - const remote = new FakeRemote() - await expect(canonicalizeE2BWorkspace(remote.sandbox, '/workspace')).resolves.toBe('/workspace') - await expect(readE2BSource(remote.sandbox, 'file.ts', '/workspace', 1_024, '/usr/bin/node')).resolves.toEqual({ - canonicalPath: '/workspace/file.ts', - text: 'const x = 1', - }) - await expect(readE2BSource(remote.sandbox, '/workspace/file.ts', '/workspace', 1_024, '/usr/bin/node')).resolves.toMatchObject({ - canonicalPath: '/workspace/file.ts', - }) - const signal = new AbortController().signal - await expect(canonicalizeE2BWorkspace(remote.sandbox, '/workspace', signal)).resolves.toBe('/workspace') - await expect(readE2BSource(remote.sandbox, 'file.ts', '/workspace', 1_024, '/usr/bin/node', signal)).resolves.toMatchObject({ - canonicalPath: '/workspace/file.ts', - }) - }) - - it('rejects malformed workspaces and source containment/type/size/encoding failures', async () => { - const malformed = new FakeRemote() - malformed.forcedRealpath = 'relative' - await expect(canonicalizeE2BWorkspace(malformed.sandbox, '/workspace')).rejects.toThrow('did not resolve canonically') - malformed.forcedRealpath = '/workspace\nother' - await expect(canonicalizeE2BWorkspace(malformed.sandbox, '/workspace')).rejects.toThrow('did not resolve canonically') - - const notDirectory = new FakeRemote() - notDirectory.infos.set('/workspace', { type: FileType.FILE, size: 0 }) - await expect(canonicalizeE2BWorkspace(notDirectory.sandbox, '/workspace')).rejects.toThrow('not a directory') - - const outside = new FakeRemote() - outside.realpaths.set('/workspace/file.ts', '/outside/file.ts') - await expect(readE2BSource(outside.sandbox, 'file.ts', '/workspace', 20, '/usr/bin/node')).rejects.toThrow('outside the workspace') - - const notFile = new FakeRemote() - notFile.readerResponse = { kind: 'not-file' } - await expect(readE2BSource(notFile.sandbox, 'file.ts', '/workspace', 20, '/usr/bin/node')).rejects.toThrow('not a regular file') - - const tooLarge = new FakeRemote() - tooLarge.readerResponse = { kind: 'oversize', size: 21 } - await expect(readE2BSource(tooLarge.sandbox, 'file.ts', '/workspace', 20, '/usr/bin/node')).rejects.toThrow('over the 20-byte limit') - - const grew = new FakeRemote() - grew.readerResponse = { kind: 'grew' } - await expect(readE2BSource(grew.sandbox, 'file.ts', '/workspace', 20, '/usr/bin/node')).rejects.toThrow('grew past') - - const invalid = new FakeRemote() - invalid.readerResponse = { kind: 'ok', data: Buffer.from([0xff]).toString('base64') } - await expect(readE2BSource(invalid.sandbox, 'file.ts', '/workspace', 20, '/usr/bin/node')).rejects.toThrow('not valid UTF-8') - - const swapped = new FakeRemote() - swapped.readerResponse = { kind: 'open-error', message: 'ELOOP: symbolic link encountered' } - await expect(readE2BSource(swapped.sandbox, 'file.ts', '/workspace', 20, '/usr/bin/node')).rejects.toThrow('opened safely') - - const malformedReader = new FakeRemote() - malformedReader.readerResponse = { kind: 'ok', data: '*' } - await expect(readE2BSource(malformedReader.sandbox, 'file.ts', '/workspace', 20, '/usr/bin/node')).rejects.toThrow('invalid bounded bytes') - malformedReader.readerResponse = { kind: 'oversize', size: 'large' } - await expect(readE2BSource(malformedReader.sandbox, 'file.ts', '/workspace', 20, '/usr/bin/node')).rejects.toThrow('invalid response') - malformedReader.readerOutput = '{' - await expect(readE2BSource(malformedReader.sandbox, 'file.ts', '/workspace', 20, '/usr/bin/node')).rejects.toThrow('invalid response') - - await expect(canonicalizeE2BWorkspace(new FakeRemote().sandbox, '/workspace', AbortSignal.abort('stop'))) - .rejects.toBe('stop') - }) -}) - -describe('E2BLspProvider pooling and lifecycle', () => { - it('reuses one canonical-workspace instance and constructs the remote proxy transport', async () => { - const spawn = vi.fn((_spec: SubprocessSpawnSpec) => new FakeInnerHandle()) - const remote = new FakeRemote() - mockedLsp.FakeLspInstance.plans.push({ query: async () => ({ kind: 'hover', hover: { contents: 'ok' } }) }) - const current = provider(remote, subprocess(spawn)) - - await expect(current.query(query())).resolves.toEqual({ kind: 'hover', hover: { contents: 'ok' } }) - await expect(current.query(query())).resolves.toEqual({ kind: 'hover', hover: { contents: 'ok' } }) - expect(mockedLsp.FakeLspInstance.instances).toHaveLength(1) - expect(mockedLsp.FakeLspInstance.instances[0]?.spec).toMatchObject({ clientProcessId: null, cwd: '/workspace' }) - expect(e2bFileUri('/workspace/a b#c.ts')).toBe('file:///workspace/a%20b%23c.ts') - expect(() => e2bFileUri('relative.ts')).toThrow('absolute remote path') - const pathToFileUri = mockedLsp.FakeLspInstance.instances[0]?.spec.pathToFileUri as (path: string) => string - expect(pathToFileUri('/workspace/a b#c.ts')).toBe('file:///workspace/a%20b%23c.ts') - expect(spawn).toHaveBeenCalledWith(expect.objectContaining({ - argv: ['/usr/bin/node', '/workspace/.dsh-e2b/lsp-proxy.mjs', expect.any(String)], - stdio: { stdin: 'pipe', stdout: 'pipe', stderr: { maxBytes: 128 } }, - })) - expect(current.id).toBe('fixture') - expect(current.extensionToLanguage).toEqual({ '.ts': 'typescript' }) - await current.disposeAll() - expect(mockedLsp.FakeLspInstance.instances[0]?.disposals).toBe(1) - }) - - it('replaces one transport failure, but preserves ordinary query errors', async () => { - const transportFailure = new Error('transport failed') - mockedLsp.FakeLspInstance.plans.push( - { transportFailure, query: async () => { throw transportFailure } }, - { query: async () => ({ kind: 'hover', hover: { contents: 'retried' } }) }, - ) - const retried = provider() - await expect(retried.query(query())).resolves.toMatchObject({ hover: { contents: 'retried' } }) - expect(mockedLsp.FakeLspInstance.instances[0]?.disposals).toBe(1) - expect(mockedLsp.FakeLspInstance.instances).toHaveLength(2) - - const ordinary = new Error('ordinary failure') - mockedLsp.FakeLspInstance.plans.push( - { query: async () => { throw ordinary }, dead: true }, - { query: async () => ({ kind: 'hover', hover: null }) }, - ) - const failed = provider() - await expect(failed.query(query())).rejects.toBe(ordinary) - await expect(failed.query(query())).resolves.toMatchObject({ kind: 'hover' }) - }) - - it('evicts a server that dies after a successful query', async () => { - mockedLsp.FakeLspInstance.plans.push( - { deadAfterQuery: true, query: async () => ({ kind: 'hover', hover: null }) }, - { query: async () => ({ kind: 'hover', hover: null }) }, - ) - const current = provider() - await current.query(query()) - await current.query(query()) - expect(mockedLsp.FakeLspInstance.instances).toHaveLength(2) - expect(mockedLsp.FakeLspInstance.instances[0]?.disposals).toBe(1) - }) - - it('serializes a workspace queue, observes queued abort, and awaits work on disposal', async () => { - const first = Promise.withResolvers() - mockedLsp.FakeLspInstance.plans.push({ query: () => first.promise }) - const current = provider() - const running = current.query(query()) - const controller = new AbortController() - const queued = current.query(query(), controller.signal) - await new Promise(resolve => setImmediate(resolve)) - controller.abort('queued stop') - await expect(queued).rejects.toBe('queued stop') - - const disposing = current.disposeAll() - first.resolve({ kind: 'hover', hover: null }) - await expect(running).resolves.toMatchObject({ kind: 'hover' }) - await disposing - await expect(current.query(query())).rejects.toMatchObject({ code: 'LSP_DISPOSED' }) - }) - - it('covers pre-abort, synthetic abort, resolve, and rejection in the queue race', async () => { - const current = provider() - const internal = current as unknown as { - queues: Map> - enqueue(workspace: string, signal: AbortSignal | undefined, run: () => Promise): Promise - } - await expect(internal.enqueue('pre', AbortSignal.abort('pre-stop'), async () => 'unused')).rejects.toBe('pre-stop') - - const signal = new AbortController().signal - await expect(internal.enqueue('resolve', signal, async () => 'ok')).resolves.toBe('ok') - - const failure = new Error('queue failed') - const rejected = Promise.reject(failure) - void rejected.catch(() => {}) - internal.queues.set('reject', rejected) - await expect(internal.enqueue('reject', signal, async () => 'unused')).rejects.toBe(failure) - - // eslint-disable-next-line @typescript-eslint/prefer-promise-reject-errors -- Exercise normalization at the promise boundary. - const opaque = Promise.reject('opaque queue failure') - void opaque.catch(() => {}) - internal.queues.set('opaque', opaque) - await expect(internal.enqueue('opaque', signal, async () => 'unused')).rejects.toEqual(new Error('opaque queue failure')) - - const synthetic = { - aborted: false, - reason: undefined, - throwIfAborted() {}, - addEventListener(_type: string, listener: () => void) { listener() }, - removeEventListener() {}, - } as unknown as AbortSignal - await expect(internal.enqueue('synthetic', synthetic, async () => 'unused')).rejects.toMatchObject({ name: 'AbortError' }) - await current.disposeAll() - }) -}) - -describe('lsp-e2b plugin composition', () => { - function pluginContext( - remote: FakeRemote, - service: E2BSubprocessService, - registerProvider = vi.fn(() => vi.fn()), - ) { - const effects: Array<() => void | Promise> = [] - const ctx = { - subprocess: service, - e2b: { - runtimeRoot: '/workspace/.dsh-e2b', - getSandbox: async () => remote.sandbox, - }, - lsp: { registerProvider }, - effect: (callback: () => (() => void | Promise)) => { effects.push(callback()) }, - } as unknown as Context - return { ctx, effects, registerProvider } - } - - it('installs one proxy, resolves commands, registers providers, and disposes them', async () => { - const remote = new FakeRemote() - const fixture = pluginContext(remote, subprocess()) - await apply(fixture.ctx, { servers: { one: server(), two: server({ command: 'server-two' }) } }) - expect(remote.writes).toHaveLength(1) - expect(remote.writes[0]?.[0]?.path).toBe('/workspace/.dsh-e2b/lsp-stdio-proxy.mjs') - expect(remote.commands).toContain("chmod 600 -- '/workspace/.dsh-e2b/lsp-stdio-proxy.mjs'") - expect(fixture.registerProvider).toHaveBeenCalledTimes(2) - await fixture.effects[0]?.() - }) - - it('rolls back partial registration and rejects invalid composition/configuration', async () => { - const remote = new FakeRemote() - const firstDispose = vi.fn() - const register = vi.fn() - .mockReturnValueOnce(firstDispose) - .mockImplementationOnce(() => { throw new Error('duplicate provider') }) - const rollback = pluginContext(remote, subprocess(), register) - await expect(apply(rollback.ctx, { servers: { one: server(), two: server() } })).rejects.toThrow('duplicate provider') - expect(firstDispose).toHaveBeenCalledOnce() - - const wrong = pluginContext(remote, {} as E2BSubprocessService) - await expect(apply(wrong.ctx, { servers: { one: server() } })).rejects.toThrow('dsh-subprocess-e2b') - - const empty = pluginContext(remote, subprocess()) - await expect(apply(empty.ctx, { servers: {} })).rejects.toThrow('at least one server') - - for (const [id, config] of [ - ['', server()], - ['one', server({ command: '' })], - ['one', server({ maxMessageBytes: 0 })], - ['one', server({ maxStderrBytes: 1.5 })], - ['one', server({ shutdownTimeoutMs: 2_147_483_648 })], - ] as const) { - const fixture = pluginContext(new FakeRemote(), subprocess()) - await expect(apply(fixture.ctx, { servers: { [id]: config } })).rejects.toThrow() - } - }) - - it('registers the package-owned invariant companion', async () => { - const ctx = new Context() - await ctx.plugin(InvariantService, { enabled: true }) - const fiber = await ctx.plugin(E2BLspInvariant).await() - await fiber.dispose() - }) -}) diff --git a/packages/e2b/lsp-e2b/tests/transport.spec.ts b/packages/e2b/lsp-e2b/tests/transport.spec.ts deleted file mode 100644 index ae5106bc09..0000000000 --- a/packages/e2b/lsp-e2b/tests/transport.spec.ts +++ /dev/null @@ -1,181 +0,0 @@ -import { once } from 'node:events' -import { PassThrough } from 'node:stream' -import { describe, expect, it } from 'vitest' -import { encodeE2BFrame } from '@deepseek-ai/dsh-e2b' -import type { SubprocessHandle, SubprocessOutcome } from '@deepseek-ai/dsh-subprocess' -import { E2BLspTransport } from '@deepseek-ai/dsh-lsp-e2b' - -class FakeHandle implements SubprocessHandle { - readonly pid = 321 - readonly stdin: PassThrough | undefined - readonly stdout: PassThrough | undefined - readonly stderr = undefined - readonly collected: SubprocessHandle['collected'] - readonly result = Promise.withResolvers() - readonly done = this.result.promise - terminated = 0 - waitResult = true - - constructor(options: { stdin?: boolean; stdout?: boolean; diagnostic?: string } = {}) { - this.stdin = options.stdin === false ? undefined : new PassThrough() - this.stdout = options.stdout === false ? undefined : new PassThrough() - this.collected = options.diagnostic === undefined - ? {} - : { stderr: { readFrom: () => ({ text: options.diagnostic as string, nextOffset: 0, lossy: false }) } } - } - - terminate(): void { - this.terminated += 1 - } - - async waitForExit(): Promise { - return this.waitResult - } - - resolve(outcome: SubprocessOutcome = { exitCode: 0, signal: null }): void { - this.stdout?.end() - this.result.resolve(outcome) - } - - reject(error: unknown): void { - this.stdout?.end() - this.result.reject(error) - } -} - -function proxyFrame(type: 'stdout' | 'stderr', data: string | Buffer): string { - return encodeE2BFrame({ type, data: Buffer.from(data).toString('base64') }) -} - -describe('E2BLspTransport', () => { - it('frames stdin, decodes split byte output, and exposes handle lifecycle', async () => { - const inner = new FakeHandle({ diagnostic: 'proxy diagnostic' }) - const transport = new E2BLspTransport(inner, 1_024, 64) - let stdin = '' - inner.stdin?.on('data', (chunk) => { stdin += String(chunk) }) - let stdout = Buffer.alloc(0) - transport.stdout.on('data', (chunk) => { stdout = Buffer.concat([stdout, chunk]) }) - - transport.stdin.write(Buffer.from([0, 0xff])) - await new Promise(resolve => setImmediate(resolve)) - const encodedInput = stdin.trim() - const input = JSON.parse(Buffer.from(encodedInput, 'base64').toString('utf8')) as Record - expect(input).toEqual({ type: 'stdin', data: 'AP8=' }) - - const frames = proxyFrame('stdout', Buffer.from([0, 0xff])) - + proxyFrame('stderr', 'server diagnostic') - + encodeE2BFrame({ type: 'exit', code: 0, signal: null }) - inner.stdout?.write(frames.slice(0, 7)) - inner.stdout?.write(frames.slice(7)) - inner.stdout?.write(proxyFrame('stderr', '')) - inner.stdout?.write(encodeE2BFrame({ type: 'exit', code: null, signal: 'SIGTERM' })) - inner.resolve() - - await expect(transport.done).resolves.toEqual({ exitCode: 0, signal: null }) - expect(stdout).toEqual(Buffer.from([0, 0xff])) - expect(transport.collected.stderr?.readFrom(0).text).toBe('server diagnosticproxy diagnostic') - expect(transport.pid).toBe(321) - inner.waitResult = false - await expect(transport.waitForExit()).resolves.toBe(false) - transport.terminate() - expect(inner.terminated).toBe(1) - }) - - it('ends the inner stdin and retains a bounded byte tail with independent offsets', async () => { - const inner = new FakeHandle() - const transport = new E2BLspTransport(inner, 1_024, 4) - const finished = once(inner.stdin!, 'finish') - transport.stdin.end() - await finished - - inner.stdout?.write(proxyFrame('stderr', 'ab')) - inner.stdout?.write(proxyFrame('stderr', 'cdef')) - const reader = transport.collected.stderr! - expect(reader.readFrom(0)).toEqual({ text: 'cdef', nextOffset: 6, lossy: true }) - expect(reader.readFrom(2)).toEqual({ text: 'cdef', nextOffset: 6, lossy: false }) - expect(reader.readFrom(5)).toEqual({ text: 'f', nextOffset: 6, lossy: false }) - expect(reader.readFrom(99)).toEqual({ text: '', nextOffset: 6, lossy: false }) - expect(() => reader.readFrom(-1)).toThrow('non-negative safe integer') - expect(() => reader.readFrom(1.5)).toThrow('non-negative safe integer') - inner.resolve() - await transport.done - - const partialInner = new FakeHandle() - const partial = new E2BLspTransport(partialInner, 1_024, 4) - partialInner.stdout?.write(proxyFrame('stderr', 'abcdef')) - expect(partial.collected.stderr?.readFrom(0)).toEqual({ text: 'cdef', nextOffset: 6, lossy: true }) - partialInner.resolve() - await partial.done - }) - - it.each([ - ['non-object', encodeE2BFrame(null), 'malformed frame'], - ['wrong type', encodeE2BFrame({ type: 'other', data: '' }), 'malformed frame'], - ['invalid exit', encodeE2BFrame({ type: 'exit', code: 'zero', signal: null }), 'malformed frame'], - ['missing data', encodeE2BFrame({ type: 'stdout' }), 'malformed frame'], - ['invalid base64', encodeE2BFrame({ type: 'stdout', data: 'abc' }), 'invalid base64'], - ['invalid outer frame', 'not-base64\n', 'invalid base64'], - ['non-ASCII outer frame', 'é', 'non-ASCII'], - ])('fails %s proxy traffic and terminates the group', async (_name, frame, message) => { - const inner = new FakeHandle() - const transport = new E2BLspTransport(inner, 1_024, 32) - inner.stdout?.write(frame) - inner.stdout?.write(proxyFrame('stdout', 'ignored')) - inner.stdout?.emit('error', new Error('second failure')) - inner.resolve() - await expect(transport.done).rejects.toThrow(message) - expect(inner.terminated).toBe(1) - }) - - it('fails truncated frames and preserves inner spawn rejection', async () => { - const truncatedInner = new FakeHandle() - const truncated = new E2BLspTransport(truncatedInner, 1_024, 32) - truncatedInner.stdout?.write('YQ==') - truncatedInner.resolve() - await expect(truncated.done).rejects.toThrow('mid-frame') - - const rejectedInner = new FakeHandle({ diagnostic: 'tail' }) - const rejected = new E2BLspTransport(rejectedInner, 1_024, 32) - rejectedInner.reject(new Error('spawn failed')) - await expect(rejected.done).rejects.toThrow('spawn failed') - expect(rejected.collected.stderr?.readFrom(0).text).toBe('tail') - }) - - it('forwards output and input stream errors without an unhandled inner error', async () => { - const outputInner = new FakeHandle() - const output = new E2BLspTransport(outputInner, 1_024, 32) - outputInner.stdout?.emit('error', new Error('proxy stdout failed')) - outputInner.resolve() - await expect(output.done).rejects.toThrow('proxy stdout failed') - - const inputInner = new FakeHandle() - const input = new E2BLspTransport(inputInner, 1_024, 32) - const outerError = once(input.stdin, 'error') - inputInner.stdin?.emit('error', new Error('proxy stdin failed')) - await expect(outerError).resolves.toMatchObject([{ message: 'proxy stdin failed' }]) - inputInner.resolve() - await input.done - }) - - it('normalizes a non-Error decoder throw', async () => { - const inner = new FakeHandle() - const transport = new E2BLspTransport(inner, 1_024, 32) - const internal = transport as unknown as { - decoder: { push(chunk: string): unknown[] } - onProxyData(chunk: Buffer): void - } - internal.decoder = { push: () => { throw 'raw decoder failure' } } - internal.onProxyData(Buffer.from('x')) - inner.resolve() - await expect(transport.done).rejects.toThrow('raw decoder failure') - }) - - it('rejects a subprocess that drops either required pipe', () => { - const missingStdin = new FakeHandle({ stdin: false }) - const missingStdout = new FakeHandle({ stdout: false }) - expect(() => new E2BLspTransport(missingStdin, 10, 10)).toThrow('dropped a piped stream') - expect(() => new E2BLspTransport(missingStdout, 10, 10)).toThrow('dropped a piped stream') - expect(missingStdin.terminated).toBe(1) - expect(missingStdout.terminated).toBe(1) - }) -}) diff --git a/packages/e2b/lsp-e2b/tsconfig.json b/packages/e2b/lsp-e2b/tsconfig.json deleted file mode 100644 index a6c0fd8ece..0000000000 --- a/packages/e2b/lsp-e2b/tsconfig.json +++ /dev/null @@ -1,20 +0,0 @@ -{ - "extends": "../../../tsconfig.base.json", - "compilerOptions": { - "rootDir": "src", - "outDir": "lib/types" - }, - "include": ["src"], - "references": [ - { "path": "../../../vendor/cosmokit" }, - { "path": "../../../vendor/cordis" }, - { "path": "../../../vendor/schemastery" }, - { "path": "../e2b" }, - { "path": "../../lsp/lsp" }, - { "path": "../../lsp/lsp-local" }, - { "path": "../../subprocess/subprocess" }, - { "path": "../subprocess-e2b" }, - { "path": "../../util/timeout" }, - { "path": "../../support/invariants" } - ] -} diff --git a/packages/e2b/pty-e2b/README.i18n.yaml b/packages/e2b/pty-e2b/README.i18n.yaml index ddae4c21db..580f874e62 100644 --- a/packages/e2b/pty-e2b/README.i18n.yaml +++ b/packages/e2b/pty-e2b/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/e2b/pty-e2b/README.md -README.md: d6d8e05f13701b85cee9a9646bfdd82ac07769da -README.zh.md: 265257c72e7bcbd2dfd805b343db5bab7173ab51 +README.md: d1b1731cd7b65577a18814a90363acdc66d7d262 +README.zh.md: 4aa2cc8d661e61e80ef61693422a7b6f73c2cb67 diff --git a/packages/e2b/pty-e2b/README.md b/packages/e2b/pty-e2b/README.md index d6d8e05f13..d1b1731cd7 100644 --- a/packages/e2b/pty-e2b/README.md +++ b/packages/e2b/pty-e2b/README.md @@ -26,7 +26,7 @@ Numeric values are positive safe integers, `backendType` is non-empty, and `maxR The backend uses E2B's byte-oriented PTY callback with a streaming fatal UTF-8 decoder, then the backend-neutral line sanitizer and bounded buffers from `dsh-pty`. It installs a controlled Bash prompt marker and waits for printable prompt text; when that marker is unavailable, observed output plus the configured silence bound yields `inferred_idle`. Startup with no output reaches the absolute timeout and fails instead of publishing an empty session. -Each send writes UTF-8 bytes and an optional carriage-return submit sequence. Cancellation and explicit signals resolve the remote terminal's foreground process group through `ps`, then signal that group; `SIGKILL` refuses to target the shell itself. The backend records the terminal's POSIX session id at startup. Close sends `SIGTERM` to every process group still in that session, escalates survivors to `SIGKILL`, verifies that the session is empty, and does not resolve until the SDK handle reports exit. A startup failure closes the unpublished PTY, and `PtyBackendCleanupError` preserves a concurrent cleanup failure. +Each send writes UTF-8 bytes and an optional carriage-return submit sequence. Cancellation and explicit signals resolve the remote terminal's foreground process group through `ps`, then signal that group; cancellation rechecks the originating send after lookup so a settled operation cannot signal or fail its successor, and `SIGKILL` refuses to target the shell itself. The backend records the terminal's POSIX session id at startup. Close sends `SIGTERM` to every process group still in that session, escalates survivors to `SIGKILL`, verifies that the session is empty, and does not resolve until the SDK handle reports exit. A startup failure closes the unpublished PTY, and `PtyBackendCleanupError` preserves a concurrent cleanup failure. The remote PTY process and its child processes live in E2B. Prompt/readiness state, scrollback, operation handles, owner authority, and SDK event delivery remain in host memory. diff --git a/packages/e2b/pty-e2b/README.zh.md b/packages/e2b/pty-e2b/README.zh.md index 265257c72e..4aa2cc8d66 100644 --- a/packages/e2b/pty-e2b/README.zh.md +++ b/packages/e2b/pty-e2b/README.zh.md @@ -26,7 +26,7 @@ 该后端为 E2B 面向字节的 PTY 回调配备流式、遇到无效序列即失败的 UTF-8 解码器,随后使用 `dsh-pty` 提供的后端无关行清理器与有界缓冲区。它会安装受控的 Bash 提示符标记,并等待可打印的提示符文本;若该标记不可用,系统会在已经观察到输出且达到已配置的静默上限时得出 `inferred_idle`。零输出的启动过程会达到绝对超时并失败,不会发布空会话。 -每次发送都会写入 UTF-8 字节,并可选写入回车提交序列。取消与显式信号会通过 `ps` 确定远程终端的前台进程组,再向该组发送信号;发送 `SIGKILL` 时拒绝以 shell 本身为目标。后端会在启动时记录终端的 POSIX 会话 id。关闭操作会向该会话内仍存在的每个进程组发送 `SIGTERM`,对存活者升级为 `SIGKILL`,验证会话已经清空,并且直到 SDK 句柄报告退出才结算。如果启动失败,系统会关闭尚未发布的 PTY;若清理同时失败,`PtyBackendCleanupError` 会保留这项失败。 +每次发送都会写入 UTF-8 字节,并可选写入回车提交序列。取消与显式信号会通过 `ps` 确定远程终端的前台进程组,再向该组发送信号;取消处理会在查找后重新检查原发送操作是否仍为当前操作,以免已结算的操作向后继操作发送信号或令其失败;发送 `SIGKILL` 时拒绝以 shell 本身为目标。后端会在启动时记录终端的 POSIX 会话 id。关闭操作会向该会话内仍存在的每个进程组发送 `SIGTERM`,对存活者升级为 `SIGKILL`,验证会话已经清空,并且直到 SDK 句柄报告退出才结算。如果启动失败,系统会关闭尚未发布的 PTY;若清理同时失败,`PtyBackendCleanupError` 会保留这项失败。 远程 PTY 进程及其子进程位于 E2B。提示符/就绪状态、scrollback、操作句柄、所有者权限和 SDK 事件交付仍保留在宿主内存中。 diff --git a/packages/e2b/pty-e2b/src/session.ts b/packages/e2b/pty-e2b/src/session.ts index bf8780103d..7742cd44f0 100644 --- a/packages/e2b/pty-e2b/src/session.ts +++ b/packages/e2b/pty-e2b/src/session.ts @@ -226,6 +226,10 @@ export class E2BPtySession implements PtyBackendSession { /* jscpd:ignore-start -- Signal, status, and close methods preserve the seam shape around remote identities. */ async signal(signal: PtySignal): Promise { const pgid = await this.foregroundPgid() + return await this.deliverSignal(signal, pgid) + } + + private async deliverSignal(signal: PtySignal, pgid: number): Promise { if (signal === 'SIGKILL' && pgid === this.pid) { throw new Error('refusing to SIGKILL the E2B PTY shell; use terminal_close') } @@ -302,7 +306,15 @@ export class E2BPtySession implements PtyBackendSession { private interrupt(operation: E2BSendOperation): void { if (this.active !== operation) return - void this.signal('SIGINT').catch((error: unknown) => { this.failActive(error) }) + void this.interruptActive(operation).catch((error: unknown) => { + if (this.active === operation) this.failActive(error) + }) + } + + private async interruptActive(operation: E2BSendOperation): Promise { + const pgid = await this.foregroundPgid() + if (this.active !== operation) return + await this.deliverSignal('SIGINT', pgid) } private async foregroundPgid(): Promise { diff --git a/packages/e2b/pty-e2b/tests/session.spec.ts b/packages/e2b/pty-e2b/tests/session.spec.ts index 554a4a3590..1fedf6bbac 100644 --- a/packages/e2b/pty-e2b/tests/session.spec.ts +++ b/packages/e2b/pty-e2b/tests/session.spec.ts @@ -68,6 +68,7 @@ class FakeSandbox { sendError: unknown signalError: unknown killError: unknown + foregroundLookup: Promise | undefined onTerm: (() => void) | undefined onGroupKill: (() => void) | undefined onKill: (() => void) | undefined @@ -88,7 +89,9 @@ class FakeSandbox { commands: { run: async (command: string): Promise => { this.commands.push(command) - if (command.startsWith('ps -o tpgid')) return { exitCode: 0, stdout: this.pgid, stderr: '' } + if (command.startsWith('ps -o tpgid')) { + return await (this.foregroundLookup ?? Promise.resolve({ exitCode: 0, stdout: this.pgid, stderr: '' })) + } if (command.startsWith('ps -eo sid=')) { return { exitCode: 0, stdout: this.sessionGroups.map(value => `${value}\n`).join(''), stderr: '' } } @@ -273,6 +276,45 @@ describe('E2BPtySession readiness, output, and signals', () => { expect(sendInput).toHaveBeenCalled() }) + it('does not let a stale interrupt signal or fail a successor send', async () => { + vi.useFakeTimers() + const fake = new FakeSandbox() + const handle = new FakePtyHandle() + const session = new E2BPtySession(fake.sandbox, handle.asHandle(), 123, config()) + await initialize(session) + + const lookup = Promise.withResolvers() + fake.foregroundLookup = lookup.promise + const stale = session.startSend({ text: 'old', submit: true }) + expect(stale.cancel()).toBe(true) + session.onData(Buffer.from('\x1b]133;D;0\x07dsh> ')) + await vi.advanceTimersByTimeAsync(10) + await stale.done + + const successor = session.startSend({ text: 'new', submit: true }) + fake.signalError = new Error('late interrupt failure') + lookup.resolve({ exitCode: 0, stdout: '789\n', stderr: '' }) + await vi.advanceTimersByTimeAsync(0) + expect(fake.commands).not.toContain('kill -INT -- -789') + session.onData(Buffer.from('\x1b]133;D;0\x07dsh> ')) + await vi.advanceTimersByTimeAsync(10) + await expect(successor.done).resolves.toMatchObject({ waitReason: 'stdin_read' }) + + const failedLookup = Promise.withResolvers() + fake.foregroundLookup = failedLookup.promise + const staleFailure = session.startSend({ text: 'old failure', submit: true }) + expect(staleFailure.cancel()).toBe(true) + session.onData(Buffer.from('\x1b]133;D;0\x07dsh> ')) + await vi.advanceTimersByTimeAsync(10) + await staleFailure.done + const finalSuccessor = session.startSend({ text: 'new after failure', submit: true }) + failedLookup.reject(new Error('late lookup failure')) + await vi.advanceTimersByTimeAsync(0) + session.onData(Buffer.from('\x1b]133;D;0\x07dsh> ')) + await vi.advanceTimersByTimeAsync(10) + await expect(finalSuccessor.done).resolves.toMatchObject({ waitReason: 'stdin_read' }) + }) + it('preserves startup abort reasons and classifies invalid UTF-8 transport failures', async () => { const fake = new FakeSandbox() const abortHandle = new FakePtyHandle() From 2abc9823e761e5abce520799504d264581719930 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 02:51:44 +0800 Subject: [PATCH 072/108] refactor(e2b): compose portable runtime consumers --- ...026-07-27-e2b-remote-runtime-poc.i18n.yaml | 4 +- .../2026-07-27-e2b-remote-runtime-poc.md | 28 +- .../2026-07-27-e2b-remote-runtime-poc.zh.md | 28 +- examples/headless-agent/e2b.cordis.yml | 23 +- .../tests/fixtures/e2b/e2b/bin.ts | 19 +- .../tests/fixtures/e2b/e2b/cordis.yml | 23 +- examples/package.json | 2 - packages/README.i18n.yaml | 4 +- packages/README.zh.md | 2 +- .../code-runtime-worker/README.i18n.yaml | 4 +- .../code-runtime-worker/README.md | 2 +- .../code-runtime-worker/README.zh.md | 2 +- packages/e2b/README.i18n.yaml | 4 +- packages/e2b/README.md | 11 +- packages/e2b/README.zh.md | 11 +- .../e2b/code-runtime-e2b/README.i18n.yaml | 6 - packages/e2b/code-runtime-e2b/README.md | 44 - packages/e2b/code-runtime-e2b/README.zh.md | 44 - packages/e2b/code-runtime-e2b/package.json | 52 -- packages/e2b/code-runtime-e2b/src/index.ts | 494 ----------- .../e2b/code-runtime-e2b/src/invariant.ts | 20 - .../e2b/code-runtime-e2b/src/runner-source.ts | 649 --------------- .../code-runtime-e2b/tests/runtime.spec.ts | 765 ------------------ packages/e2b/code-runtime-e2b/tsconfig.json | 20 - packages/e2b/e2b/README.i18n.yaml | 4 +- packages/e2b/e2b/README.md | 2 +- packages/e2b/e2b/README.zh.md | 2 +- packages/e2b/e2b/src/frame.ts | 96 --- packages/e2b/e2b/src/index.ts | 22 - packages/e2b/e2b/tests/composition.e2e.ts | 36 +- packages/e2b/e2b/tests/e2b.spec.ts | 55 -- packages/e2b/fs-e2b/README.i18n.yaml | 4 +- packages/e2b/fs-e2b/README.md | 4 +- packages/e2b/fs-e2b/README.zh.md | 4 +- packages/e2b/fs-e2b/src/index.ts | 99 +++ packages/e2b/fs-e2b/src/source-reader.ts | 63 ++ packages/e2b/fs-e2b/tests/filesystem.spec.ts | 107 ++- packages/e2b/pty-e2b/README.i18n.yaml | 6 - packages/e2b/pty-e2b/README.md | 55 -- packages/e2b/pty-e2b/README.zh.md | 55 -- packages/e2b/pty-e2b/package.json | 46 -- packages/e2b/pty-e2b/src/config.ts | 64 -- packages/e2b/pty-e2b/src/index.ts | 130 --- packages/e2b/pty-e2b/src/invariant.ts | 20 - packages/e2b/pty-e2b/src/session.ts | 415 ---------- packages/e2b/pty-e2b/tests/index.spec.ts | 248 ------ packages/e2b/pty-e2b/tests/session.spec.ts | 485 ----------- packages/e2b/pty-e2b/tsconfig.json | 16 - packages/e2b/subprocess-e2b/README.i18n.yaml | 4 +- packages/e2b/subprocess-e2b/README.md | 11 +- packages/e2b/subprocess-e2b/README.zh.md | 11 +- packages/e2b/subprocess-e2b/src/index.ts | 93 ++- packages/e2b/subprocess-e2b/src/terminal.ts | 386 +++++++++ .../e2b/subprocess-e2b/tests/terminal.spec.ts | 541 +++++++++++++ packages/fs/README.i18n.yaml | 4 +- .../lsp/lsp-local/tests/fixture-server.ts | 6 - packages/lsp/lsp-local/tests/instance.spec.ts | 5 - packages/lsp/lsp/README.i18n.yaml | 6 +- packages/lsp/lsp/README.md | 1 - packages/lsp/lsp/README.zh.md | 1 - packages/pty/pty/src/index.ts | 9 - packages/pty/pty/src/terminal.ts | 273 ------- .../verify-package-readme-model-experience.ts | 2 - tsconfig.host.json | 3 - 64 files changed, 1426 insertions(+), 4229 deletions(-) delete mode 100644 packages/e2b/code-runtime-e2b/README.i18n.yaml delete mode 100644 packages/e2b/code-runtime-e2b/README.md delete mode 100644 packages/e2b/code-runtime-e2b/README.zh.md delete mode 100644 packages/e2b/code-runtime-e2b/package.json delete mode 100644 packages/e2b/code-runtime-e2b/src/index.ts delete mode 100644 packages/e2b/code-runtime-e2b/src/invariant.ts delete mode 100644 packages/e2b/code-runtime-e2b/src/runner-source.ts delete mode 100644 packages/e2b/code-runtime-e2b/tests/runtime.spec.ts delete mode 100644 packages/e2b/code-runtime-e2b/tsconfig.json delete mode 100644 packages/e2b/e2b/src/frame.ts create mode 100644 packages/e2b/fs-e2b/src/source-reader.ts delete mode 100644 packages/e2b/pty-e2b/README.i18n.yaml delete mode 100644 packages/e2b/pty-e2b/README.md delete mode 100644 packages/e2b/pty-e2b/README.zh.md delete mode 100644 packages/e2b/pty-e2b/package.json delete mode 100644 packages/e2b/pty-e2b/src/config.ts delete mode 100644 packages/e2b/pty-e2b/src/index.ts delete mode 100644 packages/e2b/pty-e2b/src/invariant.ts delete mode 100644 packages/e2b/pty-e2b/src/session.ts delete mode 100644 packages/e2b/pty-e2b/tests/index.spec.ts delete mode 100644 packages/e2b/pty-e2b/tests/session.spec.ts delete mode 100644 packages/e2b/pty-e2b/tsconfig.json create mode 100644 packages/e2b/subprocess-e2b/src/terminal.ts create mode 100644 packages/e2b/subprocess-e2b/tests/terminal.spec.ts delete mode 100644 packages/pty/pty/src/terminal.ts diff --git a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.i18n.yaml b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.i18n.yaml index 04ebe420ac..fc187ec47b 100644 --- a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.i18n.yaml +++ b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md -2026-07-27-e2b-remote-runtime-poc.md: 4e6414b7d919d765712ac9314ce73a69c5c6344e -2026-07-27-e2b-remote-runtime-poc.zh.md: 4d39ed7eb07cdd17c5bfa9155bc7efabaa798117 +2026-07-27-e2b-remote-runtime-poc.md: df0d6be97c502a9afe1a7aff3656a7567d302704 +2026-07-27-e2b-remote-runtime-poc.zh.md: 111d7aa63f8f7e51a82cfa0e42c489901f2b88e4 diff --git a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md index 4e6414b7d9..df0d6be97c 100644 --- a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md +++ b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md @@ -12,27 +12,23 @@ Moving the complete harness into a remote VM would unify that state but also cou ## Decision -The E2B integration is an opt-in provider-composition POC. Its six E2B-specific packages live under `packages/e2b/` while retaining seam-specific npm names: +The E2B integration is an opt-in provider-composition POC. Its three E2B-specific packages live together under `packages/e2b/`: - `@deepseek-ai/dsh-e2b` creates or reconnects one secure E2B sandbox, creates its working and private runtime directories, and owns kill/pause/leave disposal. - `@deepseek-ai/dsh-fs-e2b` implements `ctx.fs` over that sandbox's Filesystem API. -- `@deepseek-ai/dsh-subprocess-e2b` implements `ctx.subprocess` over E2B Commands and remote Linux process groups. -- `@deepseek-ai/dsh-pty-e2b` registers an E2B byte-PTY backend on `ctx.pty` while the existing registry retains exact-Agent ownership. -- `@deepseek-ai/dsh-lsp-e2b` registers configured remote language servers on `ctx.lsp`, reads source through a bounded no-follow helper in E2B, and runs servers through `dsh-subprocess-e2b`. -- `@deepseek-ai/dsh-code-runtime-e2b` registers `ctx.codeRuntime`, runs each model program in a fresh remote worker, and dispatches binding functions in the host process. -- The existing `@deepseek-ai/dsh-bash-local` remains the Bash implementation because it delegates all process mechanics to `ctx.subprocess`. +- `@deepseek-ai/dsh-subprocess-e2b` implements `ctx.subprocess` over E2B Commands, byte PTYs, and remote Linux process groups. -The owner is the sole source of sandbox identity. Providers inject it and never create private sandboxes, so filesystem tools, Bash, interactive shells, language servers, and code workers share one remote cwd, process namespace, and adapter-private directory while preserving the existing capability interfaces and model-facing tools. +The higher capabilities use provider-neutral implementations. `dsh-bash-local` delegates command mechanics to `ctx.subprocess`; `dsh-pty-local` delegates terminal allocation and signalling to `ctx.subprocess.spawnTerminal()`; `dsh-lsp-local` reads through `ctx.fs` and launches through `ctx.subprocess`; `dsh-code-runtime-subprocess` materializes its runner through `ctx.fs` and starts it through `ctx.subprocess`. The [portable execution-world decision](../architecture/2026-07-28-portable-execution-world-consumers.md) owns those generic interfaces and consumers. -The providers reuse the PTY, LSP, Code Runtime, and subprocess seams without changing their model-facing consumers or the agent loop. Backend-neutral PTY text handling lives in `dsh-pty`; the LSP protocol engine accepts `processId: null` for a server in another process namespace; Code Runtime exports its output-ledger and lossless-JSON helpers for backend parity. +The E2B owner is the sole source of sandbox identity. Its two adapters never create private sandboxes, so filesystem tools, Bash, interactive shells, language servers, and code workers share one remote cwd, process namespace, and adapter-private directory while preserving the existing capability interfaces, generic implementations, model-facing tools, and agent loop. ## POC boundary -E2B owns the mutable filesystem, command and Bash processes, PTY shell and terminal-session process groups, language-server processes and source reads, the Code Runtime launcher, controller, and worker, and adapter-private files under `.dsh-e2b`. +E2B owns the mutable filesystem, managed command and Bash processes, terminal allocation and terminal-session process groups, language-server processes and source reads, the Code Runtime launcher, controller, and worker, and adapter-private files under `.dsh-e2b`. The host owns Cordis and plugin objects, the agent loop, agent/session/goal state, session logs and persistence, LLM calls, prompts and tools, authority decisions, skills, subagent orchestration, PTY buffers and readiness state, LSP JSON-RPC ids/queues/protocol state, Code Runtime type stripping/output accounting/binding dispatch, and E2B SDK/network orchestration. The overlay does not upload, mount, or synchronize the host workspace; identical cwd strings name independent host and remote directories. -Byte-sensitive protocols use the narrowest adapter required by E2B's callback shapes. PTY consumes the SDK's byte callback directly and carries send identity across asynchronous foreground-group lookup. LSP installs a bounded remote source reader that walks no-follow directory descriptors beneath the canonical workspace. Code Runtime keeps framed stdout in a launcher process isolated from the controller and worker descriptors, and gives each controller a process group that is killed before its inherited pipes drain. Their dependency-free helpers encode protocol payloads as validated newline-delimited base64 JSON, keeping E2B's decoded command callbacks on an ASCII transport. +The fundamental adapters carry the substrate-specific mechanics. `dsh-subprocess-e2b` consumes E2B's byte PTY callback directly, retains terminal send identity across asynchronous foreground-group lookup, and owns whole-session cleanup. `dsh-fs-e2b` performs bounded source reads through a dependency-free helper that walks no-follow directory descriptors beneath the canonical target. Generic Code Runtime keeps its controller/worker protocol on validated ASCII/base64 frames and kills the provider-owned process group before inherited pipes drain. Generic LSP uses UTF-8 JSON over command pipes; E2B's decoded callback transport is not an arbitrary binary channel. Retaining a sandbox preserves remote files and unmanaged remote state only. Reconnect does not reconstruct host PTY sessions, buffers, process handles, LSP connections or requests, code workers, binding calls, timers, output cursors, or locks. Managed groups terminate and join when their provider disposes before the shared owner pauses, leaves, or kills the sandbox. @@ -40,7 +36,7 @@ The POC has no session-persistence backend, template builder, volume, snapshot, ## Verification -Focused package suites pin owner lifecycle cleanup, filesystem semantics and commit metadata, subprocess process groups, configuration and verified publication rollback, byte framing and multibyte boundaries, PTY readiness/signal identity/default-environment scrubbing/terminal-session cleanup, descriptor-walked bounded LSP source reads, Code Runtime binding and descriptor isolation, worker and descendant-held pipe draining, hostile traffic, output limits, timeout/abort ordering, disposal to quiescence, and package-owned invariant registrations. Adjacent local-backend suites pin the shared PTY utilities and the LSP cross-namespace `processId` behavior. +Focused package suites pin owner lifecycle cleanup, filesystem paths/containment/bounded descriptor reads and commit metadata, subprocess executable lookup/process groups/publication rollback, terminal byte I/O/signal identity/default-environment scrubbing/session cleanup, output limits, abort ordering, disposal to quiescence, and package-owned invariant registrations. The generic PTY, LSP, and subprocess Code Runtime suites pin their provider-neutral readiness, cross-namespace `processId`, binding bridge, descriptor isolation, hostile traffic, and worker/descendant cleanup behavior. A credential-gated Loader composition creates real E2B sandboxes and exercises FS-to-Bash and Bash-to-FS visibility, process-publication rollback, bounded spill output, PTY default-secret scrubbing, stale-interrupt identity, and process-tree cleanup, parent-swap-safe bounded LSP source reads, Code Runtime host bindings, descriptor-isolated output accounting, descendant-held pipe cleanup, wall timeout, abort, runner cleanup, host-workspace isolation, and final sandbox deletion. The same composition runs through source imports and built package exports. @@ -50,13 +46,13 @@ A credential-gated Loader composition creates real E2B sandboxes and exercises F **Run the entire harness process inside E2B** — rejected because it changes deployment, credential flow, model transport, session durability, plugin loading, and supervision at once. Those questions are independent of proving the provider seams. -**Put every E2B capability in the shared owner package** — rejected because lifecycle identity is the owner's only concern. Filesystem, subprocess, PTY, LSP, and Code Runtime retain separate provider contracts, configuration, tests, and consumers; Bash continues to reuse its subprocess seam. +**Put every E2B operation in the shared owner package** — rejected because lifecycle identity is the owner's only concern. Filesystem and subprocess retain separate provider contracts, tests, and consumers; the owner exposes one shared SDK handle without becoming a capability grab bag. **Implement filesystem operations through shell commands only** — rejected because that bypasses `ctx.fs` identity, structured errors, version guards, streaming reads, and atomic mutation semantics already consumed by the file tools. -**Use the host PTY, LSP, and worker backends unchanged** — rejected because they use host process and filesystem APIs; sharing an absolute cwd string does not share state across machines. +**Keep E2B-specific PTY, LSP, and Code Runtime packages** — rejected because their domain behavior does not vary with E2B. They were shallow adapters that duplicated existing consumers to replace filesystem and process operations; moving those operations behind the fundamental seams gives every provider one implementation of readiness, protocol, binding, and presentation behavior. -**Expose E2B Commands as one generic transport and bypass capability providers** — rejected because PTY needs byte callbacks and foreground signaling, LSP needs byte-faithful stdio plus remote source containment, and Code Runtime needs bidirectional host binding calls and hostile-peer validation. Bypassing their registries would also fork model-facing behavior. +**Call E2B Filesystem, Commands, or PTY APIs directly from higher capabilities** — rejected because it bypasses the `ctx.fs` and `ctx.subprocess` contracts, duplicates execution-world policy in each consumer, and forks model-facing behavior. The subprocess seam includes the irreducible terminal primitive because ordinary pipes cannot supply foreground groups or whole-session cleanup. **Add a generic distributed-runtime abstraction first** — rejected because the existing capability seams already carry the required contracts. A new cross-cutting interface would speculate about persistence, synchronization, and reconnect semantics beyond the POC. @@ -64,6 +60,6 @@ A credential-gated Loader composition creates real E2B sandboxes and exercises F ## Consequences -The small composition demonstrates that existing capability seams can move an agent's mutable coding world off-host without changing the loop or model-facing tool packages. `sandboxId` plus pause/leave permits manual remote-file retention for experiments, while kill remains the demo's cleanup policy. +The three-package composition demonstrates that filesystem and subprocess are the sufficient provider seams for moving an agent's mutable coding world off-host without changing the loop, higher capability implementations, or model-facing tool packages. Fixes to Bash, PTY, LSP, and Code Runtime remain provider-neutral. `sandboxId` plus pause/leave permits manual remote-file retention for experiments, while kill remains the demo's cleanup policy. -The providers are not interchangeable with local backends for every consumer: remote startup cannot synchronously expose a PID, E2B retains complete command output in SDK memory, ordinary command callbacks are not byte-faithful, signal attribution is partly inferred, and reconnect cannot restore handles or protocol state. PTY uses E2B's byte API; LSP and Code Runtime add validated ASCII framing where protocol bytes matter. Remote process/spill artifacts accumulate in a retained sandbox, Code programs share a JavaScript realm with Node worker internals, and a process that deliberately escapes a managed process group or PTY session does not become reconnectable or owned. These gaps remain documented POC constraints rather than compatibility shims or new cross-cutting abstractions. +The adapters are not interchangeable with local backends for every consumer: remote startup cannot synchronously expose a PID, E2B retains complete command output in SDK memory, command callbacks are text-decoded rather than arbitrary binary streams, exact terminal stdin-wait inspection is unavailable, signal attribution is partly inferred, and reconnect cannot restore handles or protocol state. PTY uses E2B's byte API; Code Runtime uses validated ASCII/base64 framing; the exercised LSP path carries valid UTF-8 JSON. Remote process/spill artifacts accumulate in a retained sandbox, Code programs share a JavaScript realm with Node worker internals, and a process that deliberately escapes a managed process group or terminal session does not become reconnectable or owned. These gaps remain documented POC constraints rather than compatibility shims or new cross-cutting abstractions. diff --git a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.zh.md b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.zh.md index 4d39ed7eb0..111d7aa63f 100644 --- a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.zh.md +++ b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.zh.md @@ -12,27 +12,23 @@ Status: implemented ## 决策 -E2B 集成是一个选择性启用的提供方组合 POC。它的 6 个 E2B 专用包(package)位于 `packages/e2b/` 下,同时保留按 seam 区分的 npm 名称: +E2B 集成是一个选择性启用的提供方组合 POC。它的 3 个 E2B 专用包(package)集中位于 `packages/e2b/` 下: - `@deepseek-ai/dsh-e2b` 创建或重新连接一个安全的 E2B 沙箱,创建其工作目录与私有运行时目录,并拥有 kill/pause/leave 资源释放操作。 - `@deepseek-ai/dsh-fs-e2b` 在该沙箱的 Filesystem API 之上实现 `ctx.fs`。 -- `@deepseek-ai/dsh-subprocess-e2b` 在 E2B Commands 和远程 Linux 进程组之上实现 `ctx.subprocess`。 -- `@deepseek-ai/dsh-pty-e2b` 在 `ctx.pty` 上注册 E2B 字节 PTY 后端,并把精确的 Agent 所有权保留在现有注册表中。 -- `@deepseek-ai/dsh-lsp-e2b` 在 `ctx.lsp` 上注册已配置的远程语言服务器,通过 E2B 内有界且不跟随链接的辅助程序读取源代码,并通过 `dsh-subprocess-e2b` 运行服务器。 -- `@deepseek-ai/dsh-code-runtime-e2b` 注册 `ctx.codeRuntime`,在全新的远程 worker 中运行每个模型程序,并在宿主进程中分发绑定函数。 -- 现有的 `@deepseek-ai/dsh-bash-local` 继续作为 Bash 实现,因为它把所有进程机制委托给 `ctx.subprocess`。 +- `@deepseek-ai/dsh-subprocess-e2b` 在 E2B Commands、字节 PTY 和远程 Linux 进程组之上实现 `ctx.subprocess`。 -该所有者是沙箱身份的唯一真源。提供方会注入该所有者,绝不创建私有沙箱,因此文件系统工具、Bash、交互式 shell、语言服务器和代码 worker 会共享一个远程 cwd、进程命名空间和适配器私有目录,同时保留现有功能接口与面向模型的工具。 +上层功能使用提供方无关的实现。`dsh-bash-local` 把命令机制委托给 `ctx.subprocess`;`dsh-pty-local` 把终端分配与信号发送委托给 `ctx.subprocess.spawnTerminal()`;`dsh-lsp-local` 通过 `ctx.fs` 读取,并通过 `ctx.subprocess` 启动;`dsh-code-runtime-subprocess` 通过 `ctx.fs` 物化 runner,再通过 `ctx.subprocess` 启动它。这些通用接口与消费方由[可移植执行环境决策](../architecture/2026-07-28-portable-execution-world-consumers.md)负责定义。 -这些提供方复用 PTY、LSP、Code Runtime 与进程管理 seam,不更改面向模型的消费方或 agent loop(智能体循环)。后端无关的 PTY 文本处理位于 `dsh-pty`;LSP 协议引擎允许位于另一个进程命名空间的服务器使用 `processId: null`;Code Runtime 导出输出账本与无损 JSON 辅助函数,以保持各后端一致。 +E2B 所有者是沙箱身份的唯一真源。其两个适配器绝不创建私有沙箱,因此文件系统工具、Bash、交互式 shell、语言服务器和代码 worker 会共享一个远程 cwd、进程命名空间和适配器私有目录,同时保留现有功能接口、通用实现、面向模型的工具与 agent loop(智能体循环)。 ## POC 边界 -E2B 拥有可变文件系统、命令和 Bash 进程、PTY shell 与终端会话进程组、语言服务器进程及源码读取、Code Runtime launcher、controller 和 worker,以及 `.dsh-e2b` 下的适配器私有文件。 +E2B 拥有可变文件系统、受管命令与 Bash 进程、终端分配与终端会话进程组、语言服务器进程及源码读取、Code Runtime launcher、controller 和 worker,以及 `.dsh-e2b` 下的适配器私有文件。 宿主拥有 Cordis 与插件对象、agent loop、agent/会话/goal 状态、会话日志及持久化、LLM(大语言模型)调用、提示词与工具、权限决策、skill(技能)、subagent 编排、PTY 缓冲与就绪状态、LSP JSON-RPC id/队列/协议状态、Code Runtime 类型剥离/输出计量/绑定分发,以及 E2B SDK/网络编排。该 overlay 不会上传、挂载或同步宿主工作区;拼写相同的 cwd 字符串分别指向彼此独立的宿主与远程目录。 -对字节敏感的协议只使用适配 E2B 回调形状所需的最窄适配器。PTY 直接消费 SDK 的字节回调,并在异步查找前台进程组的过程中保留发送操作身份。LSP 会安装一个有界的远程源码读取器,通过不跟随符号链接打开的目录描述符在规范化工作区下逐级遍历。Code Runtime 则把分帧 stdout 保留在与 controller 和 worker 描述符隔离的 launcher 进程内,并为每个 controller 分配一个进程组,在 controller 继承的管道排空前终止该组。它们的无依赖辅助程序会把协议载荷编码为经过验证、以换行分隔的 base64 JSON,并通过 ASCII 传输承载 E2B 已解码的命令回调。 +基础适配器承载基底专用机制。`dsh-subprocess-e2b` 直接消费 E2B 的字节 PTY 回调,在异步查找前台进程组的过程中保留终端发送身份,并负责全会话清理。`dsh-fs-e2b` 通过无依赖辅助程序执行有界源码读取,该程序会在规范化目标下逐级遍历不跟随符号链接的目录描述符。通用 Code Runtime 通过经过验证的 ASCII/base64 帧承载 controller/worker 协议,并在继承的管道排空前终止提供方拥有的进程组。通用 LSP 通过命令管道使用 UTF-8 JSON;E2B 的已解码回调传输并非任意二进制通道。 保留沙箱只会保存远程文件与未受管的远程状态。重新连接不会重建宿主 PTY 会话、缓冲、进程句柄、LSP 连接或请求、代码 worker、绑定调用、定时器、输出游标或锁。受管进程组会在所属提供方 dispose(资源释放)时终止并等待退出,之后共享所有者才会暂停、脱离或终止沙箱。 @@ -40,7 +36,7 @@ E2B 拥有可变文件系统、命令和 Bash 进程、PTY shell 与终端会话 ## 验证 -聚焦包测试套件固定所有者生命周期清理、文件系统语义与提交元数据、进程管理的进程组、配置与经过验证的发布回滚、字节分帧与多字节边界、PTY 就绪状态/信号身份/默认环境清理/终端会话清理、基于描述符逐级遍历的有界 LSP 源码读取、Code Runtime 绑定与描述符隔离、worker 管道及后代进程所持管道的排空、恶意通信、输出上限、超时/中止顺序、等待完全停稳的资源释放,以及包自有不变式注册。相邻本地后端测试套件固定共享 PTY 工具函数,以及 LSP 跨命名空间 `processId` 行为。 +聚焦包测试套件固定所有者生命周期清理、文件系统路径/containment/有界描述符读取与提交元数据、子进程可执行文件查找/进程组/发布回滚、终端字节 I/O/信号身份/默认环境清理/会话清理、输出上限、中止顺序、等待完全停稳的资源释放,以及包自有不变式注册。通用 PTY、LSP 与子进程 Code Runtime 测试套件固定其提供方无关的就绪判定、跨命名空间 `processId`、绑定桥接、描述符隔离、恶意通信,以及 worker/后代进程清理行为。 凭据门控的 Loader 组合会创建真实 E2B 沙箱,并演练 FS-to-Bash 与 Bash-to-FS 可见性、进程发布回滚、有界 spill 输出、PTY 默认秘密清理、陈旧中断身份与进程树清理、可抵御父目录替换的有界 LSP 源码读取、Code Runtime 宿主绑定、描述符隔离的输出记账、后代进程所持管道的清理、墙钟超时、中止、runner 清理、宿主工作区隔离,以及最终删除沙箱。同一组合分别通过源代码导入与已构建包导出运行。 @@ -50,13 +46,13 @@ E2B 拥有可变文件系统、命令和 Bash 进程、PTY shell 与终端会话 **在 E2B 内运行完整 harness 进程。** 不予采纳,因为这会同时改变部署、凭据流、模型传输、会话持久性、插件加载和监管方式。要证明提供方 seam,并不需要同时回答这些彼此独立的问题。 -**把所有 E2B 功能放入共享所有者包。** 不予采纳,因为生命周期身份是该所有者唯一负责的事项。文件系统、进程管理、PTY、LSP 与 Code Runtime 各自保留独立的提供方契约、配置、测试和消费方;Bash 继续复用其进程管理 seam。 +**把所有 E2B 操作放入共享所有者包。** 不予采纳,因为生命周期身份是该所有者唯一负责的事项。文件系统与进程管理各自保留独立的提供方契约、测试和消费方;所有者只公开一个共享 SDK 句柄,不会因此包揽各类功能。 **仅通过 shell 命令实现文件系统操作。** 不予采纳,因为这会绕过文件工具已经使用的 `ctx.fs` 身份、结构化错误、版本防护、流式读取和原子变更语义。 -**原样使用宿主 PTY、LSP 与 worker 后端。** 不予采纳,因为它们使用宿主的进程与文件系统 API;在不同机器上复用同一个绝对 cwd 字符串并不会共享状态。 +**保留 E2B 专用的 PTY、LSP 与 Code Runtime 包。** 不予采纳,因为它们的领域行为不会随 E2B 改变。这些浅层适配器为了替换文件系统与进程操作而重复现有消费方;把这些操作移到基础 seam 之后,可让所有提供方共享同一套就绪判定、协议、绑定与呈现行为实现。 -**把 E2B Commands 公开为通用传输并绕过功能提供方。** 不予采纳,因为 PTY 需要字节回调和前台信号,LSP 需要字节保真的 stdio 与远程源码路径约束,Code Runtime 则需要双向宿主绑定调用与不可信对等方验证。绕过其注册表还会使面向模型的行为产生分叉。 +**从上层功能直接调用 E2B Filesystem、Commands 或 PTY API。** 不予采纳,因为这会绕过 `ctx.fs` 与 `ctx.subprocess` 契约,在每个消费方中重复执行环境策略,并使面向模型的行为产生分叉。进程管理 seam 纳入不可约简的终端原语,因为普通管道无法提供前台进程组或全会话清理。 **先添加通用分布式运行时抽象。** 不予采纳,因为现有功能 seam 已承载所需契约。新的跨领域接口会预先假定 POC 范围之外的持久化、同步与重连语义。 @@ -64,6 +60,6 @@ E2B 拥有可变文件系统、命令和 Bash 进程、PTY shell 与终端会话 ## 后果 -这个小型组合证明,现有功能 seam 可以把 agent 的可变 coding 环境移出宿主,而无需改变循环或面向模型的工具包。`sandboxId` 与 `pause`/`leave` 允许实验手动保留远程文件,演示仍以 `kill` 作为清理策略。 +这个由 3 个包组成的组合证明,文件系统与进程管理这两个提供方 seam 足以把 agent 的可变 coding 环境移出宿主,而无需改变循环、上层功能实现或面向模型的工具包。Bash、PTY、LSP 与 Code Runtime 的修复仍与提供方无关。`sandboxId` 与 `pause`/`leave` 允许实验手动保留远程文件,演示仍以 `kill` 作为清理策略。 -这些提供方并不能对所有消费方与本地后端互换:远程启动无法同步公开 PID,E2B 会在 SDK 内存中保留完整命令输出,普通命令回调并非字节保真,信号归因部分依靠推断,重新连接也无法恢复句柄或协议状态。PTY 使用 E2B 的字节 API;LSP 与 Code Runtime 则在必须保真处理协议字节之处增加经过验证的 ASCII 分帧。保留沙箱后会累积远程进程/spill 产物,模型程序与 Node worker 内部机制共享一个 JavaScript realm,有意逃离受管理进程组或 PTY 会话的进程也不会因此变得可重新连接或由该组合管理。这些缺口作为 POC 约束明确记录,而不会引入兼容垫片或新的跨领域抽象。 +这些适配器并不能对所有消费方与本地后端互换:远程启动无法同步公开 PID,E2B 会在 SDK 内存中保留完整命令输出,命令回调传递的是已解码文本而非任意二进制流,无法精确检查终端 stdin 等待状态,信号归因部分依靠推断,重新连接也无法恢复句柄或协议状态。PTY 使用 E2B 的字节 API;Code Runtime 使用经过验证的 ASCII/base64 分帧;已演练的 LSP 路径承载有效的 UTF-8 JSON。保留沙箱后会累积远程进程/spill 产物,模型程序与 Node worker 内部机制共享一个 JavaScript realm,有意逃离受管理进程组或终端会话的进程也不会因此变得可重新连接或由该组合管理。这些缺口作为 POC 约束明确记录,而不会引入兼容垫片或新的跨领域抽象。 diff --git a/examples/headless-agent/e2b.cordis.yml b/examples/headless-agent/e2b.cordis.yml index 877172c343..32d7049c5d 100644 --- a/examples/headless-agent/e2b.cordis.yml +++ b/examples/headless-agent/e2b.cordis.yml @@ -1,6 +1,6 @@ # POC overlay: keep the advanced headless agent and model-facing tools, but -# place its filesystem, processes, terminals, language servers, and Code Mode -# execution in one short-lived E2B sandbox. +# place its filesystem and process substrate in one short-lived E2B sandbox; +# the generic Bash, PTY, LSP, and Code Runtime consumers compose above them. - id: base name: '@cordisjs/plugin-include' config: @@ -32,18 +32,25 @@ name: '@deepseek-ai/dsh-subprocess-e2b' - id: fs-e2b name: '@deepseek-ai/dsh-fs-e2b' - - id: code-runtime-e2b - name: '@deepseek-ai/dsh-code-runtime-e2b' + - id: code-runtime-subprocess + name: '@deepseek-ai/dsh-code-runtime-subprocess' + - id: sandbox + name: '@deepseek-ai/dsh-sandbox-local' + - id: sandbox-policy + name: '@deepseek-ai/dsh-sandbox-policy' + config: + mode: danger-full-access + workspaceRoot: !!js process.cwd() - id: pty name: '@deepseek-ai/dsh-pty' - - id: pty-e2b - name: '@deepseek-ai/dsh-pty-e2b' + - id: pty-local + name: '@deepseek-ai/dsh-pty-local' - id: tool-pty name: '@deepseek-ai/dsh-tool-pty' - id: lsp name: '@deepseek-ai/dsh-lsp' - - id: lsp-e2b - name: '@deepseek-ai/dsh-lsp-e2b' + - id: lsp-local + name: '@deepseek-ai/dsh-lsp-local' config: servers: typescript: diff --git a/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts b/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts index a997752711..aeb447b0e8 100644 --- a/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts +++ b/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts @@ -1,15 +1,14 @@ import { readFile } from 'node:fs/promises' import { posix, resolve } from 'node:path' import { boot } from '@deepseek-ai/dsh-app-boot' -import { AgentMessageId } from '@deepseek-ai/dsh-agent' import type { Agent } from '@deepseek-ai/dsh-agent' import { Session, SessionId } from '@deepseek-ai/dsh-session' -import type {} from '@deepseek-ai/dsh-code-runtime-e2b' +import type {} from '@deepseek-ai/dsh-code-runtime-subprocess' import { quoteE2BShellArg } from '@deepseek-ai/dsh-e2b' import type {} from '@deepseek-ai/dsh-fs-e2b' import type {} from '@deepseek-ai/dsh-bash-local' -import type {} from '@deepseek-ai/dsh-lsp-e2b' -import type {} from '@deepseek-ai/dsh-pty-e2b' +import type {} from '@deepseek-ai/dsh-lsp-local' +import type {} from '@deepseek-ai/dsh-pty-local' const configPath = process.argv[2] if (configPath === undefined) throw new Error('usage: bin.ts ') @@ -24,10 +23,10 @@ const owner: Agent = { status: 'idle', acceptsNextStep: false, ctx: ownerFiber.ctx, - followup: () => AgentMessageId('unused'), - steer: () => AgentMessageId('unused'), - inject: () => AgentMessageId('unused'), - send: () => AgentMessageId('unused'), + followup() {}, + steer() {}, + inject() {}, + send() {}, cancel() {}, whenIdle: () => Promise.resolve(), } @@ -159,7 +158,7 @@ try { const runRemoteCommand = remoteCommands.run.bind(sandbox.commands) let containmentFaultInjected = false remoteCommands.run = async (command, options) => { - if (!containmentFaultInjected && command.includes('dsh-e2b-source-reader') && command.includes('swapped-parent/source.ts')) { + if (!containmentFaultInjected && command.includes('dsh-e2b-bounded-reader') && command.includes('swapped-parent/source.ts')) { containmentFaultInjected = true await runRemoteCommand( `rm -rf -- ${quoteE2BShellArg(swappedParentPath)} && ln -s -- ${quoteE2BShellArg(swappedOutsidePath)} ${quoteE2BShellArg(swappedParentPath)}`, @@ -194,7 +193,7 @@ try { workspaceRoot: process.cwd(), }) } catch (error: unknown) { - lspDocumentBound = String(error).includes('over the 4000000-byte limit') + lspDocumentBound = String(error).includes('exceeds the 4000000-byte limit') if (!lspDocumentBound) throw error } if (!lspDocumentBound) throw new Error('E2B LSP accepted an oversized remote source') diff --git a/examples/headless-agent/tests/fixtures/e2b/e2b/cordis.yml b/examples/headless-agent/tests/fixtures/e2b/e2b/cordis.yml index 65bd3d5246..93bb6b65e5 100644 --- a/examples/headless-agent/tests/fixtures/e2b/e2b/cordis.yml +++ b/examples/headless-agent/tests/fixtures/e2b/e2b/cordis.yml @@ -21,22 +21,33 @@ - id: agents name: '@deepseek-ai/dsh-agent' +- id: sandbox + name: '@deepseek-ai/dsh-sandbox-local' + +- id: sandbox-policy + name: '@deepseek-ai/dsh-sandbox-policy' + config: + mode: danger-full-access + workspaceRoot: !!js process.cwd() + - id: pty name: '@deepseek-ai/dsh-pty' -- id: pty-e2b - name: '@deepseek-ai/dsh-pty-e2b' +- id: pty-local + name: '@deepseek-ai/dsh-pty-local' config: pollIntervalMs: 25 + exactProbeAfterMs: 150 idleSilenceMs: 2000 + handoffGraceMs: 500 timeoutMs: 5000 disposeGraceMs: 1000 - id: lsp name: '@deepseek-ai/dsh-lsp' -- id: lsp-e2b - name: '@deepseek-ai/dsh-lsp-e2b' +- id: lsp-local + name: '@deepseek-ai/dsh-lsp-local' config: servers: fixture: @@ -48,8 +59,8 @@ shutdownTimeoutMs: 1000 killGraceMs: 500 -- id: code-runtime-e2b - name: '@deepseek-ai/dsh-code-runtime-e2b' +- id: code-runtime-subprocess + name: '@deepseek-ai/dsh-code-runtime-subprocess' config: computeMs: 500 maxWallMs: 5000 diff --git a/examples/package.json b/examples/package.json index 8dd631e760..e4251c9475 100644 --- a/examples/package.json +++ b/examples/package.json @@ -43,12 +43,10 @@ "@deepseek-ai/dsh-llm-replay": "workspace:*", "@deepseek-ai/dsh-loader-smoke": "workspace:*", "@deepseek-ai/dsh-lsp": "workspace:*", - "@deepseek-ai/dsh-lsp-e2b": "workspace:*", "@deepseek-ai/dsh-lsp-local": "workspace:*", "@deepseek-ai/dsh-permission": "workspace:*", "@deepseek-ai/dsh-plan-mode": "workspace:*", "@deepseek-ai/dsh-pty": "workspace:*", - "@deepseek-ai/dsh-pty-e2b": "workspace:*", "@deepseek-ai/dsh-pty-local": "workspace:*", "@deepseek-ai/dsh-pwsh-local": "workspace:*", "@deepseek-ai/dsh-repeat-tool-guard": "workspace:*", diff --git a/packages/README.i18n.yaml b/packages/README.i18n.yaml index a0827779db..f101375bc6 100644 --- a/packages/README.i18n.yaml +++ b/packages/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/README.md -README.md: 7dc52943e488a14e0c055ac72be0b2ad6f752455 -README.zh.md: bc3dbf0b4533d1d6adf6d49b0376a3dcdd6bfdb5 +README.md: 1fd0c9c2385f3169791a28f3ef50bb42792760af +README.zh.md: f9f7d5ae8839ea7033a349ac7a0083a97780a21c diff --git a/packages/README.zh.md b/packages/README.zh.md index 1667c88273..86a316dca7 100644 --- a/packages/README.zh.md +++ b/packages/README.zh.md @@ -6,7 +6,7 @@ ## 层级结构 -组在 `packages///` 容纳包;包名仍为 `@deepseek-ai/dsh-`。**组 README 是规范的包/ctx 键映射。** +包按组置于 `packages///`;包名仍为 `@deepseek-ai/dsh-`。**组 README 负责包/ctx 键映射。** | 组 | 职责 | 发布预期 | |---|---|---| diff --git a/packages/code-runtime/code-runtime-worker/README.i18n.yaml b/packages/code-runtime/code-runtime-worker/README.i18n.yaml index d3472a697e..10c721446f 100644 --- a/packages/code-runtime/code-runtime-worker/README.i18n.yaml +++ b/packages/code-runtime/code-runtime-worker/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/code-runtime/code-runtime-worker/README.md -README.md: 4b14c6fad5d1e491faeb54c9cb0e4403c8e2d8dd -README.zh.md: c522917835b562cf7648d8bc78f0315b7c52d2d2 +README.md: 35196a0b4fba5cd0388246a70354308ece08b39f +README.zh.md: 49871c38c540addd06f5d24793ae00b45e2f8bb0 diff --git a/packages/code-runtime/code-runtime-worker/README.md b/packages/code-runtime/code-runtime-worker/README.md index 4b14c6fad5..fbe387e4ea 100644 --- a/packages/code-runtime/code-runtime-worker/README.md +++ b/packages/code-runtime/code-runtime-worker/README.md @@ -46,7 +46,7 @@ No direct invalidation; the named consumer owns any request-prefix changes. ## Known Limitations and Deferred Work -- **OS processes a program spawns survive this backend's termination** — `worker.terminate()` ends only the thread; deployments needing remote process-group cleanup can select the E2B backend, whose separate limitations still apply. +- **OS processes a program spawns survive termination** — `worker.terminate()` ends the thread only; deployments requiring process-tree cleanup select `dsh-code-runtime-subprocess`, whose mounted subprocess provider owns that cleanup. - **Type-strip rides Node's experimental `stripTypeScriptTypes` API** — the relied-on behavior is pinned by unit tests, with amaro/sucrase as named drop-in replacements if it shifts. - **`computeMs` expiry can overshoot by up to one poll interval** — busy time is sampled every 25 ms (an internal constant, deliberately not config). - **Programs get a five-method `console` shim** (`log`/`info`/`warn`/`error`/`debug`) — deliberately not Node's full console surface. diff --git a/packages/code-runtime/code-runtime-worker/README.zh.md b/packages/code-runtime/code-runtime-worker/README.zh.md index c522917835..57f2a1145a 100644 --- a/packages/code-runtime/code-runtime-worker/README.zh.md +++ b/packages/code-runtime/code-runtime-worker/README.zh.md @@ -46,7 +46,7 @@ SDK 接口是默认/具名 `WorkerCodeRuntime` 类与 `Config`。可操作的 ## 已知限制与暂缓工作 -- **程序 spawn 的 OS 进程在该后端终止后仍会存活**:`worker.terminate()` 只结束线程;需要清理远程进程组的部署可以选择 E2B 后端,但该后端自身的限制仍然适用。 +- **程序 spawn 的 OS 进程在终止后仍会存活**:`worker.terminate()` 只结束线程;需要清理进程树的部署应选择 `dsh-code-runtime-subprocess`,由其挂载的子进程提供方负责该清理。 - **类型剥离依赖 Node 的实验性 `stripTypeScriptTypes` API**:依赖的行为由单元测试固定;如其发生变化,amaro/sucrase 是已经点名的直接替代品。 - **`computeMs` 到期最多可能超过一个轮询间隔**:系统每 25 ms 采样一次忙碌时间(内部常量,有意不做成配置)。 - **程序获得一个含 5 种方法的 `console` shim**(`log`/`info`/`warn`/`error`/`debug`):有意不提供 Node 的完整 console 接口。 diff --git a/packages/e2b/README.i18n.yaml b/packages/e2b/README.i18n.yaml index 1ded2a1d34..9b56a3ef55 100644 --- a/packages/e2b/README.i18n.yaml +++ b/packages/e2b/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/e2b/README.md -README.md: b25f00fb5f32643008127f0cee4f3d758018404a -README.zh.md: fc3d6901f3cdf6c645cfde49dc46108759959aca +README.md: ef2e5ef49056e6688630e785a61c01fa53febbb1 +README.zh.md: d76b05f9b60bc471c1793decd1def1d15615c852 diff --git a/packages/e2b/README.md b/packages/e2b/README.md index b25f00fb5f..ef2e5ef490 100644 --- a/packages/e2b/README.md +++ b/packages/e2b/README.md @@ -2,15 +2,14 @@ English | [中文](README.zh.md) -An experimental provider-composition POC that places the mutable coding world in one E2B Linux sandbox. The shared owner is separate from capability adapters so every remote provider awaits the same sandbox identity and lifecycle. +An experimental provider-composition POC that places one filesystem/process execution world in an E2B Linux sandbox. E2B supplies only sandbox lifecycle and the two fundamental OS adapters; provider-neutral consumers build higher capabilities above them. | Package | ctx key | Role | |---|---|---| | [`e2b`](e2b/README.md) (`@deepseek-ai/dsh-e2b`) | `ctx.e2b` | Create or reconnect one sandbox, create its working/runtime directories, expose the shared SDK handle, and apply the configured kill/pause/leave disposition | | [`fs-e2b`](fs-e2b/README.md) (`@deepseek-ai/dsh-fs-e2b`) | `ctx.fs` | Implement the filesystem seam over E2B Filesystem APIs | -| [`subprocess-e2b`](subprocess-e2b/README.md) (`@deepseek-ai/dsh-subprocess-e2b`) | `ctx.subprocess` | Implement managed process groups, stdio projection, and remote spill files over E2B Commands | -| [`pty-e2b`](pty-e2b/README.md) (`@deepseek-ai/dsh-pty-e2b`) | `ctx.pty` backend | Run persistent interactive shells through E2B's byte PTY API | -| [`lsp-e2b`](lsp-e2b/README.md) (`@deepseek-ai/dsh-lsp-e2b`) | `ctx.lsp` provider | Run configured language servers and read query sources inside E2B | -| [`code-runtime-e2b`](code-runtime-e2b/README.md) (`@deepseek-ai/dsh-code-runtime-e2b`) | `ctx.codeRuntime` | Run model-written programs remotely while bridging bindings to the host | +| [`subprocess-e2b`](subprocess-e2b/README.md) (`@deepseek-ai/dsh-subprocess-e2b`) | `ctx.subprocess` | Implement executable lookup, managed process groups and stdio, remote spill files, and terminal sessions over E2B Commands and PTY APIs | -The existing [`dsh-bash-local`](../bash/bash-local/README.md) needs no E2B-specific fork: it delegates process mechanics to `ctx.subprocess`, so replacing that provider places Bash in the same remote world. This boundary does not move the harness process, Cordis objects, model calls, agent/session state, session persistence, skills, protocol state, or E2B SDK buffers. The [shared-runtime decision](../../.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md) owns the POC boundary. +The existing [`dsh-bash-local`](../bash/bash-local/README.md), [`dsh-pty-local`](../pty/pty-local/README.md), [`dsh-lsp-local`](../lsp/lsp-local/README.md), and [`dsh-code-runtime-subprocess`](../code-runtime/code-runtime-subprocess/README.md) need no E2B-specific forks. They delegate every execution-world operation to `ctx.fs` and `ctx.subprocess`, so mounting the two E2B adapters places their mutable work in the same sandbox. + +This boundary does not move the harness process, Cordis objects, model calls, agent/session state, session persistence, skills, higher-level protocol state, or E2B SDK buffers. The [shared-runtime decision](../../.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md) owns the POC boundary; the [portable-consumer decision](../../.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md) owns the generic composition. diff --git a/packages/e2b/README.zh.md b/packages/e2b/README.zh.md index fc3d6901f3..d76b05f9b6 100644 --- a/packages/e2b/README.zh.md +++ b/packages/e2b/README.zh.md @@ -2,15 +2,14 @@ [English](README.md) | 中文 -这是一个实验性提供方组合 POC,把可变的编码环境放进同一个 E2B Linux 沙箱。共享所有者与功能适配器分离,使每个远程提供方都等待同一个沙箱身份和生命周期。 +这是一个实验性提供方组合 POC,把一个文件系统/进程执行环境放进 E2B Linux 沙箱。E2B 只提供沙箱生命周期与两个基础 OS 适配器;提供方无关的消费方在其上构建更高层能力。 | 包(package) | ctx 键 | 职责 | |---|---|---| | [`e2b`](e2b/README.md)(`@deepseek-ai/dsh-e2b`) | `ctx.e2b` | 创建或重新连接一个沙箱,创建其工作目录与运行时目录,公开共享 SDK 句柄,并应用配置的 kill/pause/leave 处置方式 | | [`fs-e2b`](fs-e2b/README.md)(`@deepseek-ai/dsh-fs-e2b`) | `ctx.fs` | 通过 E2B Filesystem API 实现文件系统 seam | -| [`subprocess-e2b`](subprocess-e2b/README.md)(`@deepseek-ai/dsh-subprocess-e2b`) | `ctx.subprocess` | 通过 E2B Commands 实现受管进程组、stdio 投影与远程 spill 文件 | -| [`pty-e2b`](pty-e2b/README.md)(`@deepseek-ai/dsh-pty-e2b`) | `ctx.pty` 后端 | 通过 E2B 的字节 PTY API 运行持久交互式 shell | -| [`lsp-e2b`](lsp-e2b/README.md)(`@deepseek-ai/dsh-lsp-e2b`) | `ctx.lsp` 提供方 | 在 E2B 内运行已配置的语言服务器并读取查询源代码 | -| [`code-runtime-e2b`](code-runtime-e2b/README.md)(`@deepseek-ai/dsh-code-runtime-e2b`) | `ctx.codeRuntime` | 远程运行模型编写的程序,同时把绑定桥接到宿主 | +| [`subprocess-e2b`](subprocess-e2b/README.md)(`@deepseek-ai/dsh-subprocess-e2b`) | `ctx.subprocess` | 通过 E2B Commands 与 PTY API 实现可执行文件查找、受管进程组与 stdio、远程 spill 文件及终端会话 | -现有的 [`dsh-bash-local`](../bash/bash-local/README.md) 无需 E2B 专用 fork:它把进程机制委托给 `ctx.subprocess`,因此替换该提供方即可让 Bash 进入同一个远程环境。该边界不会迁移 harness 进程、Cordis 对象、模型调用、agent(智能体)/会话状态、会话持久化、skill(技能)、协议状态或 E2B SDK 缓冲。[共享运行时决策](../../.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md)界定 POC 边界。 +现有的 [`dsh-bash-local`](../bash/bash-local/README.md)、[`dsh-pty-local`](../pty/pty-local/README.md)、[`dsh-lsp-local`](../lsp/lsp-local/README.md) 及 [`dsh-code-runtime-subprocess`](../code-runtime/code-runtime-subprocess/README.md) 无需 E2B 专用 fork。它们把执行环境中的所有操作委托给 `ctx.fs` 和 `ctx.subprocess`,因此挂载这两个 E2B 适配器后,它们执行的可变操作都发生在同一个沙箱内。 + +该边界不会迁移 harness 进程、Cordis 对象、模型调用、agent(智能体)/会话状态、会话持久化、skill(技能)、更高层协议状态或 E2B SDK 缓冲。[共享运行时决策](../../.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md)界定 POC 边界;[可移植消费方决策](../../.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md)界定通用组合。 diff --git a/packages/e2b/code-runtime-e2b/README.i18n.yaml b/packages/e2b/code-runtime-e2b/README.i18n.yaml deleted file mode 100644 index eb4e75f297..0000000000 --- a/packages/e2b/code-runtime-e2b/README.i18n.yaml +++ /dev/null @@ -1,6 +0,0 @@ -# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each -# side as of the last confirmed-consistent state. Both languages carry equal authority; -# after editing either side, bring the other along and re-record with: -# pnpm run verify-translation-pairing --write packages/e2b/code-runtime-e2b/README.md -README.md: a8623f95d16b54b29e53bb9cf2c528b36f121283 -README.zh.md: 2b4a37864e3c6755a74f0d2ef6ce38aa39aae24b diff --git a/packages/e2b/code-runtime-e2b/README.md b/packages/e2b/code-runtime-e2b/README.md deleted file mode 100644 index a8623f95d1..0000000000 --- a/packages/e2b/code-runtime-e2b/README.md +++ /dev/null @@ -1,44 +0,0 @@ -# @deepseek-ai/dsh-code-runtime-e2b - -English | [中文](README.zh.md) - -E2B implementation of [`ctx.codeRuntime`](../../code-runtime/code-runtime/README.md). Each run executes one model-written TypeScript program in a fresh remote Node worker while binding functions, type stripping, output accounting, and lifecycle orchestration remain on the host. - -## Configuration - -| Key | Default | Meaning | -|---|---|---| -| `computeMs` | `60000` | Remote worker event-loop busy-time budget. | -| `maxWallMs` | `600000` | Host-observed wall-clock ceiling. | -| `maxOutputBytes` | `67108864` | Combined serialized outer logs/value/diagnostic cap. | -| `maxOldGenerationSizeMb` | `512` | Remote worker old-generation heap cap in MiB. | -| `maxFrameBytes` | `268435456` | Largest decoded bridge frame, including binding traffic. | -| `killGraceMs` | `2000` | Remote process-group TERM-to-KILL grace. | - -Every value is a positive safe integer. `maxOutputBytes` is at least four bytes, `maxWallMs` cannot exceed Node's maximum timer delay, and `maxFrameBytes` cannot be smaller than `maxOutputBytes`. The service requires the concrete `dsh-subprocess-e2b` backend so run cleanup has remote process-group semantics. - -## Execution and bridge contract - -Setup uploads one dependency-free runner under `ctx.e2b.runtimeRoot` and resolves remote Node. For each run, the host wraps and type-strips erasable TypeScript with Node's `stripTypeScriptTypes`, then starts the runner in `ctx.e2b.cwd`. The runner keeps the framed host protocol in a launcher process, forks a controller process group whose stdout and stderr are bounded data pipes, and creates a fresh worker thread with an empty environment and heap limit. Model writes to native descriptors and inherited child output therefore cannot enter the frame stream; completion kills the controller group before draining its pipes and emitting the terminal frame. The worker measures active event-loop time and is destroyed after one completion. The enclosing E2B subprocess group is terminated and awaited after every result, timeout, abort, or disposal, so ordinary child processes in either managed group stop with the run. - -The bridge uses validated newline-delimited base64 JSON frames because E2B subprocess callbacks expose decoded text. Binding arguments and resolutions use the worker runtime's iterative lossless-JSON wire shape; binding functions execute on the host and typed rejection classes are materialized inside the remote worker. The worker captures the JavaScript intrinsics that its adapter boundary invokes before model code runs, hardening binding transport, output accounting, and completion validation against mutation of those references. The host repeats message validation, call-id deduplication, lossless-JSON checks, and the outer-output ledger. - -Program failures resolve as `CodeRunResult.error`; only seam misuse rejects. `isolation` is reported as `container`, which is a deployment descriptor rather than a security claim. - -## Model Experience - -Indirectly, through Code Mode in `dsh-tools`, which returns program logs, values, or typed failures through the existing `run_code` result contract. - -#### KV Cache effect - -No direct invalidation; Code Mode owns request-prefix changes. - -## Known Limitations and Deferred Work - -- **Not a whole-agent runtime** — Cordis, sessions, LLM calls, binding dispatch, TypeScript stripping, output ledgers, and E2B SDK state remain on the host. -- **No reconnectable runs** — retaining a sandbox preserves files but not worker/subprocess handles, binding calls, timers, or output cursors. -- **Node worker internals share the model realm** — mutating realm-wide globals or prototypes that Node itself uses can terminate the worker; captured adapter intrinsics are not a separate JavaScript realm or a security boundary. -- **Deliberate process-group escape is not captured** — model code can create a new POSIX session; that unmanaged process is outside this backend's cleanup identity. -- **Intermediate binding traffic is memory-bounded only per frame** — it does not enter model context or the outer-output ledger, but aggregate host/remote process memory remains the limit. -- **Experimental type stripping** — the backend shares the worker implementation's reliance on Node's experimental erasable-syntax API. -- **Sandbox policy is template-owned** — this package adds no network, volume, snapshot, or workspace-synchronization policy. diff --git a/packages/e2b/code-runtime-e2b/README.zh.md b/packages/e2b/code-runtime-e2b/README.zh.md deleted file mode 100644 index 2b4a37864e..0000000000 --- a/packages/e2b/code-runtime-e2b/README.zh.md +++ /dev/null @@ -1,44 +0,0 @@ -# @deepseek-ai/dsh-code-runtime-e2b - -[English](README.md) | 中文 - -[`ctx.codeRuntime`](../../code-runtime/code-runtime/README.md) 的 E2B 实现。每次运行都会在全新的远程 Node worker 中执行一段模型编写的 TypeScript 程序;绑定函数、类型剥离、输出记账和生命周期编排仍保留在宿主侧。 - -## 配置 - -| 配置键 | 默认值 | 含义 | -|---|---|---| -| `computeMs` | `60000` | 远程 worker 的事件循环忙碌时间预算。 | -| `maxWallMs` | `600000` | 宿主观测到的墙钟时间上限。 | -| `maxOutputBytes` | `67108864` | 外层日志、值和诊断合计的序列化上限。 | -| `maxOldGenerationSizeMb` | `512` | 远程 worker 的老生代堆上限(MiB)。 | -| `maxFrameBytes` | `268435456` | 已解码桥接帧的最大大小,包括绑定流量。 | -| `killGraceMs` | `2000` | 远程进程组 TERM 到 KILL 的宽限期。 | - -每个值都必须是正的安全整数。`maxOutputBytes` 必须至少为 4 字节,`maxWallMs` 不得超过 Node 的最大定时器延迟,且 `maxFrameBytes` 不得小于 `maxOutputBytes`。本服务要求使用具体的 `dsh-subprocess-e2b` 后端,使运行清理具备远程进程组语义。 - -## 执行与桥接契约 - -设置阶段会在 `ctx.e2b.runtimeRoot` 下上传一个无依赖的 runner,并解析远程 Node。每次运行时,宿主会包装仅使用可擦除语法的 TypeScript,再用 Node 的 `stripTypeScriptTypes` 剥离类型,然后在 `ctx.e2b.cwd` 中启动 runner。runner 会把面向宿主的分帧协议保留在 launcher 进程内,派生一个以 stdout 和 stderr 作为有界数据管道的 controller 进程组,再创建一个具有空环境与堆上限的全新 worker 线程。因此,模型对原生描述符的写入和继承的子进程输出无法进入分帧流;运行结算会先终止 controller 进程组,再排空其管道并发出终结帧。worker 会测量事件循环活跃时间,并在一次运行结算后销毁。每当运行返回结果、超时、中止或因资源释放终止时,系统都会终止外围的 E2B 进程组并等待其退出,因此任一受管组内的普通子进程会随本次运行一同停止。 - -由于 E2B 进程管理回调公开的是已解码文本,桥接层使用经过验证、以换行分隔的 base64 JSON 帧。绑定参数与 resolve 值使用 worker 运行时的迭代式无损 JSON wire 形状;绑定函数在宿主执行,类型化的 reject 类则在远程 worker 内物化。worker 会在模型代码运行前捕获其适配器边界调用的 JavaScript intrinsic,从而增强绑定传输、输出记账与完成值验证对这些引用修改的抵御能力。宿主会再次执行消息验证、调用 id 去重和无损 JSON 检查,并用外层输出账本再次计量。 - -程序失败会 resolve 为 `CodeRunResult.error`;只有 seam 误用才会 reject。`isolation` 报告为 `container`;这是部署描述符,不构成安全声明。 - -## 模型体验 - -通过 `dsh-tools` 中的 Code Mode 间接影响模型;它会通过现有 `run_code` 结果契约返回程序日志、值或类型化失败。 - -#### KV Cache 影响 - -不会直接失效;请求前缀变更由 Code Mode 负责。 - -## 已知限制与暂缓工作 - -- **并非完整的 agent(智能体)运行时**:Cordis、会话、LLM(大语言模型)调用、绑定分发、TypeScript 类型剥离、输出账本和 E2B SDK 状态仍保留在宿主侧。 -- **运行不可重连**:保留沙箱会保留文件,但不会保留 worker/进程管理句柄、绑定调用、定时器或输出游标。 -- **Node worker 内部机制与模型共享同一 realm**:修改 Node 自身使用、影响整个 realm 的全局对象或原型可能会终止 worker;已捕获的适配器 intrinsic 并不构成独立的 JavaScript realm 或安全边界。 -- **不会捕获有意逃逸进程组的行为**:模型代码可以创建新的 POSIX 会话;该非受管进程不属于此后端的清理身份范围。 -- **中间绑定流量的内存边界仅适用于单帧**:它不会进入模型上下文或外层输出账本,但其总量仍只受宿主/远程进程内存限制。 -- **实验性类型剥离**:该后端与 worker 实现一样,依赖 Node 的实验性可擦除语法 API。 -- **沙箱策略归模板负责**:本包不会额外增加网络、卷、快照或工作区同步策略。 diff --git a/packages/e2b/code-runtime-e2b/package.json b/packages/e2b/code-runtime-e2b/package.json deleted file mode 100644 index 222ae71bef..0000000000 --- a/packages/e2b/code-runtime-e2b/package.json +++ /dev/null @@ -1,52 +0,0 @@ -{ - "name": "@deepseek-ai/dsh-code-runtime-e2b", - "description": "E2B code-runtime implementation for DeepSeek Harness", - "version": "0.0.1", - "private": true, - "type": "module", - "main": "lib/index.js", - "types": "lib/types/index.d.ts", - "exports": { - ".": { - "types": "./lib/types/index.d.ts", - "default": "./lib/index.js" - }, - "./invariant": { - "types": "./lib/types/invariant.d.ts", - "default": "./lib/invariant.js" - }, - "./src/*": "./src/*", - "./package.json": "./package.json" - }, - "files": [ - "lib/index.js", - "lib/invariant.js", - "lib/types/**/*.d.ts", - "lib/types/**/*.d.ts.map", - "src" - ], - "license": "BSD-3-Clause", - "peerDependencies": { - "@deepseek-ai/dsh-code-runtime": "^0.0.1", - "@deepseek-ai/dsh-code-runtime-worker": "^0.0.1", - "@deepseek-ai/dsh-e2b": "^0.0.1", - "@deepseek-ai/dsh-invariants": "^0.0.1", - "@deepseek-ai/dsh-session": "^0.0.1", - "@deepseek-ai/dsh-subprocess-e2b": "^0.0.1", - "@deepseek-ai/dsh-timeout": "^0.0.1", - "cordis": "^4.0.0-rc.7" - }, - "dependencies": { - "schemastery": "^3.18.0" - }, - "devDependencies": { - "@deepseek-ai/dsh-code-runtime": "workspace:^", - "@deepseek-ai/dsh-code-runtime-worker": "workspace:^", - "@deepseek-ai/dsh-e2b": "workspace:^", - "@deepseek-ai/dsh-invariants": "workspace:^", - "@deepseek-ai/dsh-session": "workspace:^", - "@deepseek-ai/dsh-subprocess-e2b": "workspace:^", - "@deepseek-ai/dsh-timeout": "workspace:^", - "cordis": "^4.0.0-rc.7" - } -} diff --git a/packages/e2b/code-runtime-e2b/src/index.ts b/packages/e2b/code-runtime-e2b/src/index.ts deleted file mode 100644 index c37cb3cc64..0000000000 --- a/packages/e2b/code-runtime-e2b/src/index.ts +++ /dev/null @@ -1,494 +0,0 @@ -/** E2B process/worker implementation of the harness code-runtime seam. */ - -import { posix } from 'node:path' -import { stripTypeScriptTypes } from 'node:module' -import type { Context } from 'cordis' -import z from 'schemastery' -import { CodeRuntime } from '@deepseek-ai/dsh-code-runtime' -import type { - CodeBindingNamespace, - CodeJsonValue, - CodeRunFailure, - CodeRunRequest, - CodeRunResult, -} from '@deepseek-ai/dsh-code-runtime' -import { - E2BFrameDecoder, - encodeBoundedE2BFrame, - quoteE2BShellArg, - resolveE2BExecutable, -} from '@deepseek-ai/dsh-e2b' -import { - decodeWorkerJson, - encodeWorkerJson, - OutputLedger, -} from '@deepseek-ai/dsh-code-runtime-worker' -import type { WorkerJsonWire } from '@deepseek-ai/dsh-code-runtime-worker' -import { snapshotJsonValue } from '@deepseek-ai/dsh-session' -import type { SubprocessHandle } from '@deepseek-ai/dsh-subprocess' -import E2BSubprocessService from '@deepseek-ai/dsh-subprocess-e2b' -import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout' -import { CODE_RUNNER_SOURCE } from './runner-source.ts' - -/** Runtime configuration; every execution and bridge bound is deployment-tunable. */ -export interface Config { - /** Remote worker measured event-loop busy-time budget. */ - computeMs?: number - /** Host-observed wall-clock ceiling. */ - maxWallMs?: number - /** Combined serialized outer logs/value/diagnostic cap. */ - maxOutputBytes?: number - /** Remote worker old-generation heap cap in MiB. */ - maxOldGenerationSizeMb?: number - /** Largest decoded bridge frame, including binding traffic. */ - maxFrameBytes?: number - /** Remote process-group TERM-to-KILL grace. */ - killGraceMs?: number -} - -type ResolvedConfig = Required -type PreparedRuntime = { node: string; runner: string } - -interface LiveRun { - settle(failure: CodeRunFailure): void - finished: Promise -} - -interface CallMessage { - type: 'call' - id: number - global: string - name: string - args: WorkerJsonWire -} - -interface LogMessage { - type: 'log' - text: string -} - -interface DoneMessage { - type: 'done' - value?: WorkerJsonWire - error?: CodeRunFailure -} - -type RunnerMessage = CallMessage | LogMessage | DoneMessage | { type: 'output-limit' } - -const STRIP_WRAP = { prefix: 'async function __dsh_program__() {\n', suffix: '\n}' } as const -const MIN_OUTPUT_BYTES = 4 -const IDENTIFIER = /^[A-Za-z_$][A-Za-z0-9_$]*$/ -/* jscpd:ignore-start -- Backends enforce the same injected-global vocabulary without coupling lifecycle implementations. */ -const RESERVED_WORDS = new Set([ - 'await', 'break', 'case', 'catch', 'class', 'const', 'continue', 'debugger', 'default', 'delete', 'do', - 'else', 'enum', 'export', 'extends', 'false', 'finally', 'for', 'function', 'if', 'import', 'in', - 'instanceof', 'new', 'null', 'return', 'super', 'switch', 'this', 'throw', 'true', 'try', 'typeof', - 'var', 'void', 'while', 'with', 'yield', 'let', 'static', 'implements', 'interface', 'package', - 'private', 'protected', 'public', 'arguments', 'eval', -]) -const RESERVED_ERROR_PROPERTIES = new Set(['name', 'message', 'stack']) -/* jscpd:ignore-end */ -const FAILURE_KINDS = new Set([ - 'exception', 'timeout', 'abort', 'worker-exit', 'invalid-output', 'output-limit', -]) - -function messageOf(error: unknown): string { - return error instanceof Error ? error.message : String(error) -} - -function parseRunnerMessage(raw: unknown): RunnerMessage | undefined { - if (typeof raw !== 'object' || raw === null) return undefined - const record = raw as Record - if (record.type === 'output-limit') return { type: 'output-limit' } - if (record.type === 'log') return typeof record.text === 'string' ? { type: 'log', text: record.text } : undefined - if (record.type === 'call') { - if (!Number.isSafeInteger(record.id) || (record.id as number) < 1 || typeof record.global !== 'string' || typeof record.name !== 'string' || !Array.isArray(record.args)) return undefined - return { type: 'call', id: record.id as number, global: record.global, name: record.name, args: record.args as WorkerJsonWire } - } - if (record.type !== 'done') return undefined - if (record.error === undefined) { - return { type: 'done', ...record.value === undefined ? {} : { value: record.value as WorkerJsonWire } } - } - if (typeof record.error !== 'object' || record.error === null) return undefined - const error = record.error as Record - if (typeof error.kind !== 'string' || !FAILURE_KINDS.has(error.kind as CodeRunFailure['kind']) || typeof error.message !== 'string') return undefined - return { type: 'done', error: { kind: error.kind as CodeRunFailure['kind'], message: error.message } } -} - -/** E2B-backed runtime: host-side type stripping, remote worker execution, host binding dispatch. */ -export class E2BCodeRuntime extends CodeRuntime { - static inject = ['e2b', 'subprocess'] - - static Config: z = z.object({ - computeMs: z.number().default(60_000), - maxWallMs: z.number().default(600_000), - maxOutputBytes: z.number().default(67_108_864), - maxOldGenerationSizeMb: z.number().default(512), - maxFrameBytes: z.number().default(268_435_456), - killGraceMs: z.number().default(2_000), - }) - - readonly language = 'typescript' - readonly isolation = 'container' - - private readonly config: ResolvedConfig - private readonly ready: Promise - private readonly live = new Set() - private readonly subprocess: E2BSubprocessService - private disposed = false - - constructor(ctx: Context, config: Config) { - super(ctx) - if (!(ctx.subprocess instanceof E2BSubprocessService)) { - throw new Error('code-runtime-e2b requires @deepseek-ai/dsh-subprocess-e2b as ctx.subprocess') - } - this.subprocess = ctx.subprocess - this.config = config as ResolvedConfig - for (const [key, value] of Object.entries(this.config)) { - if (!Number.isSafeInteger(value) || value <= 0) { - throw new Error(`code-runtime-e2b: config.${key} must be a positive safe integer`) - } - } - if (this.config.maxOutputBytes < MIN_OUTPUT_BYTES) { - throw new Error(`code-runtime-e2b: config.maxOutputBytes must be at least ${MIN_OUTPUT_BYTES}`) - } - if (this.config.maxWallMs > MAX_TIMER_DELAY_MS) { - throw new Error(`code-runtime-e2b: config.maxWallMs must be at most ${MAX_TIMER_DELAY_MS}`) - } - if (this.config.maxFrameBytes < this.config.maxOutputBytes) { - throw new Error('code-runtime-e2b: config.maxFrameBytes must be at least maxOutputBytes') - } - this.ready = this.prepare() - void this.ready.catch(() => {}) - ctx.effect(() => () => this.teardown(), 'E2B code-runtime teardown') - } - - /* jscpd:ignore-start -- Seam-level abort and type-strip results remain identical across execution substrates. */ - /** Execute one type-stripped program in a fresh E2B worker process. */ - async run(request: CodeRunRequest): Promise { - if (this.disposed) throw new Error('code-runtime-e2b: run() after disposal') - const bindings = this.validateBindings(request) - if (request.signal?.aborted === true) { - return this.failure({ kind: 'abort', message: String(request.signal.reason) }) - } - let code: string - try { - const stripped = stripTypeScriptTypes(STRIP_WRAP.prefix + request.program + STRIP_WRAP.suffix) - code = stripped.slice(STRIP_WRAP.prefix.length, stripped.length - STRIP_WRAP.suffix.length) - } catch (error: unknown) { - return this.failure({ kind: 'exception', message: messageOf(error) }) - } - let runtime: PreparedRuntime | undefined - try { - runtime = await this.awaitPreparation(request.signal) - } catch (error: unknown) { - // Disposal can race the awaited setup despite the synchronous precheck. - // eslint-disable-next-line @typescript-eslint/no-unnecessary-condition - if (this.disposed) return this.failure({ kind: 'abort', message: 'runtime disposed' }) - return this.failure({ kind: 'worker-exit', message: `E2B runtime setup failed: ${messageOf(error)}` }) - } - if (runtime === undefined) { - return this.failure({ kind: 'abort', message: String(request.signal?.reason) }) - } - // Disposal can race the awaited remote setup after the pre-await check. - /* v8 ignore start -- requires disposal between promise resolution and its awaiting continuation. */ - // eslint-disable-next-line @typescript-eslint/no-unnecessary-condition - if (this.disposed) return this.failure({ kind: 'abort', message: 'runtime disposed' }) - /* v8 ignore stop */ - return await this.execute(request, code, bindings, runtime) - } - /* jscpd:ignore-end */ - - private async awaitPreparation(signal: AbortSignal | undefined): Promise { - if (signal === undefined) return await this.ready - const aborted = Promise.withResolvers() - const onAbort = (): void => { aborted.resolve(undefined) } - signal.addEventListener('abort', onAbort, { once: true }) - if (signal.aborted) { - signal.removeEventListener('abort', onAbort) - return undefined - } - try { - return await Promise.race([this.ready, aborted.promise]) - } finally { - signal.removeEventListener('abort', onAbort) - } - } - - private assertPreparationActive(): void { - if (this.disposed) throw new Error('code-runtime-e2b: runtime disposed during setup') - } - - private async prepare(): Promise { - const sandbox = await this.ctx.e2b.getSandbox() - this.assertPreparationActive() - const runner = posix.join(this.ctx.e2b.runtimeRoot, 'code-runtime-runner.mjs') - await sandbox.files.write([{ path: runner, data: CODE_RUNNER_SOURCE }]) - this.assertPreparationActive() - await sandbox.commands.run(`chmod 600 -- ${quoteE2BShellArg(runner)}`) - this.assertPreparationActive() - const node = await resolveE2BExecutable(sandbox, 'node') - this.assertPreparationActive() - return { node, runner } - } - - private failure(error: CodeRunFailure): CodeRunResult { - return new OutputLedger(this.config.maxOutputBytes).failure([], error) - } - - /* jscpd:ignore-start -- Binding names have one seam contract while dispatch and teardown remain backend-owned. */ - private validateBindings(request: CodeRunRequest): Map { - const bindings = new Map() - for (const namespace of request.bindings) { - if (!IDENTIFIER.test(namespace.global) || RESERVED_WORDS.has(namespace.global)) { - throw new Error(`code-runtime-e2b: binding global ${JSON.stringify(namespace.global)} is not a usable identifier`) - } - if (namespace.global === 'console' || bindings.has(namespace.global)) { - throw new Error(`code-runtime-e2b: duplicate binding global ${JSON.stringify(namespace.global)}`) - } - bindings.set(namespace.global, namespace) - } - const errorClassNames = new Set() - for (const namespace of request.bindings) { - const descriptor = namespace.errorClass - if (descriptor === undefined) continue - if (!IDENTIFIER.test(descriptor.name) || RESERVED_WORDS.has(descriptor.name)) { - throw new Error(`code-runtime-e2b: binding error class ${JSON.stringify(descriptor.name)} is not a usable identifier`) - } - if (descriptor.name === 'console' || bindings.has(descriptor.name) || errorClassNames.has(descriptor.name)) { - throw new Error(`code-runtime-e2b: duplicate injected global ${JSON.stringify(descriptor.name)}`) - } - if (descriptor.memberNameProperty.length === 0 || RESERVED_ERROR_PROPERTIES.has(descriptor.memberNameProperty)) { - throw new Error(`code-runtime-e2b: binding error member property ${JSON.stringify(descriptor.memberNameProperty)} is not usable`) - } - errorClassNames.add(descriptor.name) - } - return bindings - } - /* jscpd:ignore-end */ - - private async execute( - request: CodeRunRequest, - code: string, - bindings: Map, - runtime: PreparedRuntime, - ): Promise { - let handle: SubprocessHandle - try { - handle = this.subprocess.spawn({ - argv: [runtime.node, runtime.runner], - cwd: this.ctx.e2b.cwd, - stdio: { stdin: 'pipe', stdout: 'pipe', stderr: { maxBytes: this.config.maxOutputBytes } }, - graceMs: this.config.killGraceMs, - ...request.signal === undefined ? {} : { signal: request.signal }, - env: {}, - }) - } catch (error: unknown) { - if (this.disposed) return this.failure({ kind: 'abort', message: 'runtime disposed' }) - if (request.signal?.aborted === true) { - return this.failure({ kind: 'abort', message: String(request.signal.reason) }) - } - return this.failure({ kind: 'worker-exit', message: `E2B runtime spawn failed: ${messageOf(error)}` }) - } - if (handle.stdin === undefined || handle.stdout === undefined) { - handle.terminate() - await Promise.allSettled([handle.done]) - try { - await handle.waitForExit() - } catch (error: unknown) { - return this.failure({ kind: 'worker-exit', message: `E2B runtime cleanup failed: ${messageOf(error)}` }) - } - return this.failure({ kind: 'worker-exit', message: 'E2B subprocess dropped a piped runtime stream' }) - } - const stdin = handle.stdin - const stdout = handle.stdout - - return new Promise((resolve) => { - const output = new OutputLedger(this.config.maxOutputBytes) - const logs: string[] = [] - const answered = new Set() - const decoder = new E2BFrameDecoder(this.config.maxFrameBytes) - let settled = false - let finishResolve!: () => void - const finished = new Promise((done) => { finishResolve = done }) - const wallTimer: { current: NodeJS.Timeout | undefined } = { current: undefined } - const live: LiveRun = { - finished, - settle: (failure) => { finish(() => output.failure(logs, failure)) }, - } - - const finish = (result: CodeRunResult | (() => CodeRunResult)): void => { - if (settled) return - settled = true - clearTimeout(wallTimer.current) - request.signal?.removeEventListener('abort', onAbort) - void new Promise((resume) => { setImmediate(resume) }).then(async () => { - handle.terminate() - await handle.done.catch(() => {}) - let cleanupError: unknown - try { - await handle.waitForExit() - } catch (error: unknown) { - cleanupError = error - } - try { - decoder.finish() - } catch (error: unknown) { - result = output.failure(logs, { kind: 'worker-exit', message: messageOf(error) }) - } - if (cleanupError !== undefined) { - result = output.failure(logs, { kind: 'worker-exit', message: `E2B runtime cleanup failed: ${messageOf(cleanupError)}` }) - } - const final = typeof result === 'function' ? result() : result - this.live.delete(live) - finishResolve() - resolve(final) - }) - } - - const sendReply = (message: unknown): void => { - if (settled) return - let frame: string - try { - frame = encodeBoundedE2BFrame(message, this.config.maxFrameBytes) - } catch (error: unknown) { - finish(() => output.failure(logs, { kind: 'worker-exit', message: `E2B runtime bridge failed: ${messageOf(error)}` })) - return - } - stdin.write(frame, (error?: Error | null) => { - if (error !== undefined && error !== null) { - finish(() => output.failure(logs, { kind: 'worker-exit', message: `E2B runtime bridge write failed: ${error.message}` })) - } - }) - } - - /* jscpd:ignore-start -- Host binding resolution mirrors worker semantics over a different transport. */ - const onCall = (message: CallMessage): void => { - if (answered.has(message.id)) return - answered.add(message.id) - const functions = bindings.get(message.global)?.functions - const fn = functions !== undefined && Object.hasOwn(functions, message.name) ? functions[message.name] : undefined - if (typeof fn !== 'function') { - sendReply({ type: 'reply', id: message.id, ok: false, message: `unknown binding ${JSON.stringify(`${message.global}.${message.name}`)}` }) - return - } - const args = decodeWorkerJson(message.args) - if (args === undefined) { - sendReply({ type: 'reply', id: message.id, ok: false, message: 'binding arguments must be lossless JSON' }) - return - } - void (async () => { - try { - const resolved = await fn(args) - let value: CodeJsonValue | undefined - try { - value = snapshotJsonValue(resolved) - } catch { - value = undefined - } - if (value === undefined) { - sendReply({ type: 'reply', id: message.id, ok: false, message: 'binding resolution must be lossless JSON' }) - } else { - sendReply({ type: 'reply', id: message.id, ok: true, value: encodeWorkerJson(value) }) - } - } catch (error: unknown) { - sendReply({ type: 'reply', id: message.id, ok: false, message: messageOf(error) }) - } - })() - } - /* jscpd:ignore-end */ - - const onMessage = (raw: unknown): void => { - if (settled) return - const message = parseRunnerMessage(raw) - if (message === undefined) return - if (message.type === 'log') { - if (!output.admit(message.text, logs)) finish(output.limit([...logs, message.text])) - return - } - if (message.type === 'output-limit') { - finish(output.limit(logs)) - return - } - if (message.type === 'call') { - onCall(message) - return - } - if (message.error !== undefined) { - finish(() => output.failure(logs, message.error as CodeRunFailure)) - } else if (message.value === undefined) { - finish(() => output.success(logs)) - } else { - const value = decodeWorkerJson(message.value) - if (value === undefined) finish(() => output.failure(logs, { kind: 'invalid-output', message: 'program completion must be lossless JSON' })) - else finish(() => output.success(logs, value)) - } - } - - stdout.on('data', (chunk: Buffer) => { - if (settled) return - try { - for (const frame of decoder.push(chunk.toString('utf8'))) onMessage(frame) - } catch (error: unknown) { - finish(() => output.failure(logs, { kind: 'worker-exit', message: `E2B runtime bridge failed: ${messageOf(error)}` })) - } - }) - stdout.on('error', (error: Error) => { - finish(() => output.failure(logs, { kind: 'worker-exit', message: `E2B runtime stdout failed: ${error.message}` })) - }) - stdin.on('error', (error: Error) => { - finish(() => output.failure(logs, { kind: 'worker-exit', message: `E2B runtime stdin failed: ${error.message}` })) - }) - void handle.done.then( - () => { - if (!settled) { - const stderr = handle.collected.stderr?.readFrom(0).text.trim() - finish(() => output.failure(logs, { kind: 'worker-exit', message: stderr === undefined || stderr === '' ? 'E2B runtime exited before completing' : `E2B runtime exited before completing: ${stderr}` })) - } - }, - (error: unknown) => { - finish(() => output.failure(logs, { kind: 'worker-exit', message: `E2B runtime spawn failed: ${messageOf(error)}` })) - }, - ) - - const onAbort = (): void => { - finish(() => output.failure(logs, { kind: 'abort', message: String(request.signal?.reason) })) - } - request.signal?.addEventListener('abort', onAbort, { once: true }) - wallTimer.current = setTimeout(() => { - finish(() => output.failure(logs, { kind: 'timeout', message: `wall-clock ceiling reached (${this.config.maxWallMs}ms)` })) - }, this.config.maxWallMs) - this.live.add(live) - if (request.signal?.aborted === true) { - onAbort() - return - } - sendReply({ - type: 'boot', - code, - namespaces: [...bindings].map(([global, namespace]) => ({ - global, - names: Object.keys(namespace.functions), - ...namespace.errorClass === undefined ? {} : { errorClass: namespace.errorClass }, - })), - computeMs: this.config.computeMs, - maxOutputBytes: this.config.maxOutputBytes, - maxOldGenerationSizeMb: this.config.maxOldGenerationSizeMb, - }) - }) - } - - /* jscpd:ignore-start -- Code-runtime backends share the service lifecycle but own different child identities. */ - private async teardown(): Promise { - this.disposed = true - const runs = [...this.live] - for (const run of runs) run.settle({ kind: 'abort', message: 'runtime disposed' }) - await Promise.all([ - this.ready.then(() => {}, () => {}), - ...runs.map(run => run.finished), - ]) - } - /* jscpd:ignore-end */ -} - -export default E2BCodeRuntime diff --git a/packages/e2b/code-runtime-e2b/src/invariant.ts b/packages/e2b/code-runtime-e2b/src/invariant.ts deleted file mode 100644 index 4112cdc76b..0000000000 --- a/packages/e2b/code-runtime-e2b/src/invariant.ts +++ /dev/null @@ -1,20 +0,0 @@ -/** Package-owned invariant companion for `@deepseek-ai/dsh-code-runtime-e2b`. */ - -/* jscpd:ignore-start */ -import type { Context } from 'cordis' -import type { InvariantInstaller } from '@deepseek-ai/dsh-invariants' - -const PACKAGE_NAME = '@deepseek-ai/dsh-code-runtime-e2b' - -/** Cordis companion plugin name. */ -export const name = 'code-runtime-e2b-invariant' -/** Service required before the companion can reserve package ownership. */ -export const inject = ['invariants'] - -/** No runtime invariant: the service owns every one-shot remote run. */ -const install: InvariantInstaller = () => {} - -/** Register this package's invariant companion. */ -export const apply = (ctx: Context): Promise<() => void> => - Promise.resolve(ctx.invariants.register(PACKAGE_NAME, install)) -/* jscpd:ignore-end */ diff --git a/packages/e2b/code-runtime-e2b/src/runner-source.ts b/packages/e2b/code-runtime-e2b/src/runner-source.ts deleted file mode 100644 index 527f21114e..0000000000 --- a/packages/e2b/code-runtime-e2b/src/runner-source.ts +++ /dev/null @@ -1,649 +0,0 @@ -/** Dependency-free remote code runner installed inside the E2B sandbox. */ - -/** Node program that runs one model program in a fresh remote worker thread. */ -export const CODE_RUNNER_SOURCE = String.raw`import { Buffer } from 'node:buffer' -import { fork } from 'node:child_process' -import { inspect } from 'node:util' -import { Worker, isMainThread, parentPort, workerData } from 'node:worker_threads' -import { createInterface } from 'node:readline' -import { fileURLToPath } from 'node:url' - -const emitFrame = message => { - process.stdout.write(Buffer.from(JSON.stringify(message)).toString('base64') + '\n') -} - -const parseFrame = line => JSON.parse(Buffer.from(line, 'base64').toString('utf8')) - -const waitForPipeDrain = stream => { - if (stream.readableEnded || stream.destroyed) return Promise.resolve() - return new Promise(resolve => { - const done = () => { - stream.off('end', done) - stream.off('close', done) - stream.off('error', done) - resolve() - } - stream.once('end', done) - stream.once('close', done) - stream.once('error', done) - if (stream.readableEnded || stream.destroyed) done() - }) -} - -const waitForChildExit = child => { - if (child.exitCode !== null || child.signalCode !== null) return Promise.resolve() - return new Promise(resolve => { child.once('exit', resolve) }) -} - -const killControllerGroup = child => { - if (process.platform !== 'win32' && Number.isSafeInteger(child.pid)) { - try { - process.kill(-child.pid, 'SIGKILL') - } catch (error) { - if (!error || typeof error !== 'object' || error.code !== 'ESRCH') throw error - } - return - } - child.kill('SIGKILL') -} - -const jsonStringBytes = text => Buffer.byteLength(JSON.stringify(text)) - -const truncateLog = (text, available) => { - if (available < 2) return '' - let result = '' - let bytes = 2 - for (const character of text) { - const cost = jsonStringBytes(character) - 2 - if (bytes + cost > available) break - bytes += cost - result += character - } - return result -} - -const runLauncher = () => { - const input = createInterface({ input: process.stdin, crlfDelay: Infinity }) - let controller - let maxOutputBytes = 0 - let logBytes = 2 - let logEntries = 0 - let settling = false - let closed = false - let terminal - - const finish = message => { - if (settling) { - if (message.type === 'output-limit') terminal = message - return - } - settling = true - terminal = message - const current = controller - controller = undefined - const drain = current - ? new Promise(resolve => { setImmediate(resolve) }).then(async () => { - const stdoutDrained = waitForPipeDrain(current.stdout) - const stderrDrained = waitForPipeDrain(current.stderr) - const exited = waitForChildExit(current) - killControllerGroup(current) - await Promise.all([exited, stdoutDrained, stderrDrained]) - }) - : Promise.resolve() - void drain.catch(error => { - process.stderr.write('code-runtime-e2b controller cleanup error: ' + String(error) + '\n') - }).then(() => { - closed = true - emitFrame(terminal) - input.close() - process.stdin.destroy() - }) - } - - const forwardLog = text => { - if (closed || terminal?.type === 'output-limit') return - const separator = logEntries > 0 ? 1 : 0 - const available = maxOutputBytes - logBytes - separator - const cost = jsonStringBytes(text) - if (cost > available) { - const prefix = truncateLog(text, available) - if (prefix) { - logBytes += jsonStringBytes(prefix) + separator - logEntries += 1 - emitFrame({ type: 'log', text: prefix }) - } - finish({ type: 'output-limit' }) - return - } - logBytes += cost + separator - logEntries += 1 - emitFrame({ type: 'log', text }) - } - - const startController = message => { - maxOutputBytes = message.maxOutputBytes - controller = fork(fileURLToPath(import.meta.url), [], { - env: { DSH_CODE_RUNTIME_CONTROLLER: '1' }, - detached: process.platform !== 'win32', - execArgv: [], - stdio: ['ignore', 'pipe', 'pipe', 'ipc'], - }) - const current = controller - current.stdout.on('data', data => { forwardLog(data.toString('utf8')) }) - current.stderr.on('data', data => { forwardLog(data.toString('utf8')) }) - current.on('message', raw => { - if (!raw || typeof raw !== 'object') return - if (raw.type === 'log' && typeof raw.text === 'string') { - forwardLog(raw.text) - return - } - if (settling) return - if (raw.type === 'call' && typeof raw.id === 'number' && typeof raw.global === 'string' && typeof raw.name === 'string' && Array.isArray(raw.args)) { - emitFrame({ type: 'call', id: raw.id, global: raw.global, name: raw.name, args: raw.args }) - } else if (raw.type === 'output-limit') { - finish({ type: 'output-limit' }) - } else if (raw.type === 'done') { - if (raw.error && typeof raw.error === 'object' && typeof raw.error.kind === 'string' && typeof raw.error.message === 'string') { - finish({ type: 'done', error: { kind: raw.error.kind, message: raw.error.message } }) - } else if (raw.value === undefined || Array.isArray(raw.value)) { - finish({ type: 'done', ...(raw.value === undefined ? {} : { value: raw.value }) }) - } - } - }) - current.on('error', error => { - finish({ type: 'done', error: { kind: 'worker-exit', message: 'remote controller error: ' + error.message } }) - }) - current.on('exit', code => { - if (!settling) finish({ type: 'done', error: { kind: 'worker-exit', message: 'remote controller exited with code ' + code + ' before completing' } }) - }) - current.send(message, error => { - if (error) finish({ type: 'done', error: { kind: 'worker-exit', message: 'remote controller boot failed: ' + error.message } }) - }) - } - - input.on('line', line => { - let message - try { - message = parseFrame(line) - } catch (error) { - process.stderr.write('code-runtime-e2b frame error: ' + String(error) + '\n') - finish({ type: 'done', error: { kind: 'worker-exit', message: 'remote runner received a malformed frame' } }) - return - } - if (!controller) { - if (!message || message.type !== 'boot' || typeof message.code !== 'string' || !Array.isArray(message.namespaces) || !Number.isSafeInteger(message.maxOutputBytes) || message.maxOutputBytes < 4) { - finish({ type: 'done', error: { kind: 'worker-exit', message: 'remote runner received an invalid boot frame' } }) - return - } - startController(message) - return - } - if (message && message.type === 'reply' && typeof message.id === 'number' && typeof message.ok === 'boolean') { - controller.send(message.ok - ? { type: 'reply', id: message.id, ok: true, value: message.value } - : { type: 'reply', id: message.id, ok: false, message: String(message.message) }, error => { - if (error) finish({ type: 'done', error: { kind: 'worker-exit', message: 'remote controller reply failed: ' + error.message } }) - }) - } - }) - input.on('close', () => { if (controller && !settling) killControllerGroup(controller) }) -} - -const runController = () => { - let worker - let finished = false - let computeTimer - const send = message => { - if (process.send) process.send(message) - } - const finish = message => { - if (finished) return - finished = true - clearInterval(computeTimer) - const current = worker - worker = undefined - const drain = current - ? new Promise(resolve => { setImmediate(resolve) }).then(async () => { - const stdoutDrained = waitForPipeDrain(current.stdout) - const stderrDrained = waitForPipeDrain(current.stderr) - await Promise.all([current.terminate(), stdoutDrained, stderrDrained]) - }) - : Promise.resolve() - void drain.catch(error => { - send({ type: 'log', text: 'code-runtime-e2b worker cleanup error: ' + String(error) + '\n' }) - }).then(() => { - if (!process.send) { - process.exitCode = 1 - return - } - process.send(message, () => { if (process.connected) process.disconnect() }) - }) - } - process.on('message', message => { - if (!worker) { - if (!message || message.type !== 'boot' || typeof message.code !== 'string' || !Array.isArray(message.namespaces)) { - finish({ type: 'done', error: { kind: 'worker-exit', message: 'remote controller received an invalid boot frame' } }) - return - } - worker = new Worker(new URL(import.meta.url), { - workerData: message, - env: {}, - execArgv: [], - stdout: true, - stderr: true, - resourceLimits: { maxOldGenerationSizeMb: message.maxOldGenerationSizeMb }, - }) - worker.stdout.on('data', data => { send({ type: 'log', text: data.toString('utf8') }) }) - worker.stderr.on('data', data => { send({ type: 'log', text: data.toString('utf8') }) }) - worker.on('message', raw => { - if (!raw || typeof raw !== 'object') return - if (raw.type === 'call' && typeof raw.id === 'number' && typeof raw.global === 'string' && typeof raw.name === 'string' && Array.isArray(raw.args)) { - send({ type: 'call', id: raw.id, global: raw.global, name: raw.name, args: raw.args }) - } else if (raw.type === 'log' && typeof raw.text === 'string') { - send({ type: 'log', text: raw.text }) - } else if (raw.type === 'output-limit') { - finish({ type: 'output-limit' }) - } else if (raw.type === 'done') { - if (raw.error && typeof raw.error === 'object' && typeof raw.error.kind === 'string' && typeof raw.error.message === 'string') { - finish({ type: 'done', error: { kind: raw.error.kind, message: raw.error.message } }) - } else if (raw.value === undefined || Array.isArray(raw.value)) { - finish({ type: 'done', ...(raw.value === undefined ? {} : { value: raw.value }) }) - } - } - }) - worker.on('error', error => { - finish({ type: 'done', error: { kind: 'worker-exit', message: 'remote worker error: ' + error.message } }) - }) - worker.on('exit', code => { - if (!finished) finish({ type: 'done', error: { kind: 'worker-exit', message: 'remote worker exited with code ' + code + ' before completing' } }) - }) - computeTimer = setInterval(() => { - if (!worker) return - if (worker.performance.eventLoopUtilization().active > message.computeMs) { - finish({ type: 'done', error: { kind: 'timeout', message: 'compute budget exhausted (' + message.computeMs + 'ms busy)' } }) - } - }, 25) - return - } - if (message && message.type === 'reply' && typeof message.id === 'number' && typeof message.ok === 'boolean') { - worker.postMessage(message.ok - ? { type: 'reply', id: message.id, ok: true, value: message.value } - : { type: 'reply', id: message.id, ok: false, message: String(message.message) }) - } - }) - process.on('disconnect', () => { if (worker && !finished) void worker.terminate() }) -} - -if (!isMainThread) { - const port = parentPort - if (!port) throw new Error('remote worker requires parentPort') - - const CapturedError = Error - const ArrayIsArray = Array.isArray - const ArrayPrototype = Array.prototype - const ObjectPrototype = Object.prototype - const ObjectCreate = Object.create - const ObjectDefineProperty = Object.defineProperty - const ObjectGetPrototypeOf = Object.getPrototypeOf - const ObjectHasOwn = Object.hasOwn - const ObjectKeys = Object.keys - const ObjectIs = Object.is - const ObjectPropertyIsEnumerable = Object.prototype.propertyIsEnumerable - const ReflectOwnKeys = Reflect.ownKeys - const ReflectApply = Reflect.apply - const NumberIsFinite = Number.isFinite - const NumberIsSafeInteger = Number.isSafeInteger - const PromiseCtor = Promise - const PromiseReject = Promise.reject - const QueueMicrotask = queueMicrotask - const BufferByteLength = Buffer.byteLength - const SetCtor = Set - const SetAdd = Set.prototype.add - const SetDelete = Set.prototype.delete - const SetHas = Set.prototype.has - const MapDelete = Map.prototype.delete - const MapGet = Map.prototype.get - const MapSet = Map.prototype.set - const ArrayJoin = Array.prototype.join - const ArrayPop = Array.prototype.pop - const StringCharCodeAt = String.prototype.charCodeAt - const StringSlice = String.prototype.slice - const JSONStringify = JSON.stringify - const StringValue = String - - const define = (target, key, value) => { - const descriptor = ObjectCreate(null) - descriptor.value = value - descriptor.enumerable = true - descriptor.configurable = true - descriptor.writable = true - ObjectDefineProperty(target, key, descriptor) - } - const append = (target, value) => { define(target, target.length, value) } - const pop = target => ReflectApply(ArrayPop, target, []) - const setAdd = (target, value) => { ReflectApply(SetAdd, target, [value]) } - const setDelete = (target, value) => { ReflectApply(SetDelete, target, [value]) } - const setHas = (target, value) => ReflectApply(SetHas, target, [value]) - const mapDelete = (target, key) => { ReflectApply(MapDelete, target, [key]) } - const mapGet = (target, key) => ReflectApply(MapGet, target, [key]) - const mapSet = (target, key, value) => { ReflectApply(MapSet, target, [key, value]) } - const plainObject = value => { - const prototype = ObjectGetPrototypeOf(value) - return prototype === null || prototype === ObjectPrototype - } - const ownEnumerableStringKeys = value => { - const keys = ReflectOwnKeys(value) - for (let index = 0; index < keys.length; index++) { - const key = keys[index] - if (typeof key !== 'string' || !ReflectApply(ObjectPropertyIsEnumerable, value, [key])) return undefined - } - return keys - } - const assign = (destination, value) => { - if (destination.kind === 'root') destination.holder.value = value - else define(destination.target, destination.key, value) - } - const snapshot = input => { - const active = new SetCtor() - const holder = ObjectCreate(null) - const tasks = [{ kind: 'visit', value: input, destination: { kind: 'root', holder } }] - while (tasks.length) { - const task = pop(tasks) - if (task.kind === 'leave') { setDelete(active, task.source); continue } - const candidate = task.value - if (candidate === null || typeof candidate === 'boolean' || typeof candidate === 'string') { - assign(task.destination, candidate); continue - } - if (typeof candidate === 'number') { - if (!NumberIsFinite(candidate) || ObjectIs(candidate, -0)) return undefined - assign(task.destination, candidate); continue - } - if (typeof candidate !== 'object' || setHas(active, candidate)) return undefined - if (ArrayIsArray(candidate)) { - if (ObjectGetPrototypeOf(candidate) !== ArrayPrototype || ReflectOwnKeys(candidate).length !== candidate.length + 1) return undefined - const target = [] - assign(task.destination, target) - setAdd(active, candidate) - append(tasks, { kind: 'leave', source: candidate }) - for (let index = candidate.length - 1; index >= 0; index--) { - if (!ObjectHasOwn(candidate, index)) return undefined - append(tasks, { kind: 'visit', value: candidate[index], destination: { kind: 'slot', target, key: index } }) - } - continue - } - if (!plainObject(candidate)) return undefined - const keys = ownEnumerableStringKeys(candidate) - if (!keys) return undefined - const target = {} - assign(task.destination, target) - setAdd(active, candidate) - append(tasks, { kind: 'leave', source: candidate }) - for (let index = keys.length - 1; index >= 0; index--) { - const key = keys[index] - append(tasks, { kind: 'visit', value: candidate[key], destination: { kind: 'slot', target, key } }) - } - } - return holder.value - } - const encodeWire = value => { - const wire = [] - const pending = [value] - while (pending.length) { - const current = pop(pending) - if (current === null || typeof current === 'boolean' || typeof current === 'number' || typeof current === 'string') { - append(wire, current); continue - } - if (ArrayIsArray(current)) { - append(wire, { kind: 'array', length: current.length }) - for (let index = current.length - 1; index >= 0; index--) append(pending, current[index]) - } else { - const keys = ObjectKeys(current) - append(wire, { kind: 'object', keys }) - for (let index = keys.length - 1; index >= 0; index--) append(pending, current[keys[index]]) - } - } - return wire - } - const decodeWire = wire => { - if (!ArrayIsArray(wire) || wire.length === 0) return undefined - const frames = [] - let root - let assigned = false - const attach = value => { - const parent = frames[frames.length - 1] - if (!parent) { - if (assigned) return false - root = value; assigned = true; return true - } - if (parent.kind === 'array') append(parent.target, value) - else define(parent.target, parent.keys[parent.index], value) - parent.index += 1 - return true - } - for (let tokenIndex = 0; tokenIndex < wire.length; tokenIndex++) { - const token = wire[tokenIndex] - let value - let frame - if (token === null || typeof token === 'boolean' || typeof token === 'string') value = token - else if (typeof token === 'number') { - if (!NumberIsFinite(token) || ObjectIs(token, -0)) return undefined - value = token - } else { - if (!plainObject(token)) return undefined - const keys = ownEnumerableStringKeys(token) - if (!keys || keys.length !== 2 || keys[0] !== 'kind') return undefined - if (token.kind === 'array' && keys[1] === 'length' && NumberIsSafeInteger(token.length) && token.length >= 0) { - value = [] - if (token.length > wire.length - tokenIndex - 1) return undefined - if (token.length) frame = { kind: 'array', target: value, length: token.length, index: 0 } - } else if (token.kind === 'object' && keys[1] === 'keys' && ArrayIsArray(token.keys)) { - const unique = new SetCtor() - const objectKeys = [] - for (const key of token.keys) { - if (typeof key !== 'string' || setHas(unique, key)) return undefined - setAdd(unique, key); append(objectKeys, key) - } - if (objectKeys.length > wire.length - tokenIndex - 1) return undefined - value = {} - if (objectKeys.length) frame = { kind: 'object', target: value, keys: objectKeys, index: 0 } - } else return undefined - } - if (!attach(value)) return undefined - if (frame) append(frames, frame) - while (frames.length) { - const current = frames[frames.length - 1] - const length = current.kind === 'array' ? current.length : current.keys.length - if (current.index < length) break - pop(frames) - } - } - return frames.length === 0 ? root : undefined - } - const byteLength = text => ReflectApply(BufferByteLength, Buffer, [text]) - const jsonStringBytes = text => byteLength(JSONStringify(text)) - const jsonValueBytes = value => { - let bytes = 0 - const tasks = [{ kind: 'value', value }] - while (tasks.length) { - const task = pop(tasks) - if (task.kind === 'separator') { bytes += 1; continue } - if (task.kind === 'key') { bytes += jsonStringBytes(task.value) + 1; continue } - const current = task.value - if (current === null) bytes += 4 - else if (typeof current === 'string') bytes += jsonStringBytes(current) - else if (typeof current === 'number' || typeof current === 'boolean') bytes += byteLength(StringValue(current)) - else if (ArrayIsArray(current)) { - bytes += 2 - for (let index = current.length - 1; index >= 0; index--) { - append(tasks, { kind: 'value', value: current[index] }) - if (index > 0) append(tasks, { kind: 'separator' }) - } - } else { - bytes += 2 - const keys = ObjectKeys(current) - for (let index = keys.length - 1; index >= 0; index--) { - const key = keys[index] - append(tasks, { kind: 'value', value: current[key] }) - append(tasks, { kind: 'key', value: key }) - if (index > 0) append(tasks, { kind: 'separator' }) - } - } - } - return bytes - } - const truncate = (text, available) => { - if (available < 2) return '' - let result = '' - let bytes = 2 - let index = 0 - while (index < text.length) { - const first = ReflectApply(StringCharCodeAt, text, [index]) - let end = index + 1 - if (first >= 0xd800 && first <= 0xdbff && end < text.length) { - const second = ReflectApply(StringCharCodeAt, text, [end]) - if (second >= 0xdc00 && second <= 0xdfff) end += 1 - } - const character = ReflectApply(StringSlice, text, [index, end]) - const cost = jsonStringBytes(character) - 2 - if (bytes + cost > available) break - bytes += cost - result += character - index = end - } - return result - } - let logBytes = 2 - let logEntries = 0 - let limited = false - const pushLog = text => { - if (limited) return - const separator = logEntries > 0 ? 1 : 0 - const available = workerData.maxOutputBytes - logBytes - separator - const cost = jsonStringBytes(text) - if (cost > available) { - const prefix = truncate(text, available) - if (prefix) { - logBytes += jsonStringBytes(prefix) + separator - logEntries += 1 - port.postMessage({ type: 'log', text: prefix }) - } - limited = true - port.postMessage({ type: 'output-limit' }) - return - } - logBytes += cost + separator - logEntries += 1 - port.postMessage({ type: 'log', text }) - } - const originalStdout = process.stdout.write - const originalStderr = process.stderr.write - process.stdout.write = (chunk, ...rest) => { - pushLog(typeof chunk === 'string' ? chunk : StringValue(chunk)) - let callback - for (let index = 0; index < rest.length; index++) { - if (typeof rest[index] === 'function') { callback = rest[index]; break } - } - if (callback) QueueMicrotask(() => { callback(null) }) - return true - } - process.stderr.write = process.stdout.write - const consoleShim = ObjectCreate(null) - for (const level of ['log', 'info', 'warn', 'error', 'debug']) { - define(consoleShim, level, (...args) => { - const rendered = [] - for (let index = 0; index < args.length; index++) { - const value = args[index] - append(rendered, typeof value === 'string' ? value : inspect(value, { depth: 4, maxArrayLength: 100, maxStringLength: 10000 })) - } - pushLog(ReflectApply(ArrayJoin, rendered, [' '])) - }) - } - const pending = new Map() - let nextId = 1 - const errorClasses = new Map() - for (const namespace of workerData.namespaces) { - if (!namespace.errorClass) continue - const descriptor = namespace.errorClass - mapSet(errorClasses, namespace.global, class BindingCallError extends CapturedError { - constructor(memberName, message) { - super(message) - ObjectDefineProperty(this, 'name', { value: descriptor.name, enumerable: true }) - ObjectDefineProperty(this, descriptor.memberNameProperty, { value: memberName, enumerable: true }) - } - }) - } - port.on('message', message => { - if (!message || message.type !== 'reply' || typeof message.id !== 'number') return - const entry = mapGet(pending, message.id) - if (!entry) return - mapDelete(pending, message.id) - if (!message.ok) { entry.reject(new CapturedError(StringValue(message.message))); return } - const value = decodeWire(message.value) - if (value === undefined) entry.reject(new CapturedError('binding resolution must be lossless JSON')) - else entry.resolve(value) - }) - const namespaces = workerData.namespaces.map(namespace => { - const target = ObjectCreate(null) - const ErrorClass = mapGet(errorClasses, namespace.global) - for (const name of namespace.names) { - define(target, name, args => { - const detached = snapshot(args) - if (detached === undefined) { - return ReflectApply(PromiseReject, PromiseCtor, [ErrorClass ? new ErrorClass(name, 'binding arguments must be lossless JSON') : new CapturedError('binding arguments must be lossless JSON')]) - } - return new PromiseCtor((resolve, reject) => { - const id = nextId++ - mapSet(pending, id, { - resolve, - reject: error => { reject(ErrorClass ? new ErrorClass(name, error.message) : error) }, - }) - port.postMessage({ type: 'call', id, global: namespace.global, name, args: encodeWire(detached) }) - }) - }) - } - return target - }) - const errorClassNames = [] - const errorClassValues = [] - for (const namespace of workerData.namespaces) { - if (!namespace.errorClass) continue - append(errorClassNames, namespace.errorClass.name) - append(errorClassValues, mapGet(errorClasses, namespace.global)) - } - const AsyncFunction = ObjectGetPrototypeOf(async function () {}).constructor - try { - const fn = new AsyncFunction(...workerData.namespaces.map(value => value.global), ...errorClassNames, 'console', '"use strict";\n' + workerData.code) - const value = await fn(...namespaces, ...errorClassValues, consoleShim) - if (!limited) { - if (value === undefined) port.postMessage({ type: 'done' }) - else { - const detached = snapshot(value) - if (detached === undefined) { - const message = 'program completion must be lossless JSON' - if (jsonStringBytes(message) > workerData.maxOutputBytes - logBytes) port.postMessage({ type: 'output-limit' }) - else port.postMessage({ type: 'done', error: { kind: 'invalid-output', message } }) - } else if (jsonValueBytes(detached) > workerData.maxOutputBytes - logBytes) { - port.postMessage({ type: 'output-limit' }) - } else { - port.postMessage({ type: 'done', value: encodeWire(detached) }) - } - } - } - } catch (error) { - if (!limited) { - let message - try { message = error instanceof CapturedError ? error.stack || error.message : StringValue(error) } - catch { message = 'program threw an unrenderable value' } - if (jsonStringBytes(message) > workerData.maxOutputBytes - logBytes) port.postMessage({ type: 'output-limit' }) - else port.postMessage({ type: 'done', error: { kind: 'exception', message } }) - } - } finally { - process.stdout.write = originalStdout - process.stderr.write = originalStderr - } -} else if (process.env.DSH_CODE_RUNTIME_CONTROLLER === '1') { - runController() -} else { - runLauncher() -} -` diff --git a/packages/e2b/code-runtime-e2b/tests/runtime.spec.ts b/packages/e2b/code-runtime-e2b/tests/runtime.spec.ts deleted file mode 100644 index f99485a929..0000000000 --- a/packages/e2b/code-runtime-e2b/tests/runtime.spec.ts +++ /dev/null @@ -1,765 +0,0 @@ -import { spawn } from 'node:child_process' -import { access, mkdtemp, rm, writeFile } from 'node:fs/promises' -import { tmpdir } from 'node:os' -import { join } from 'node:path' -import { PassThrough, Writable } from 'node:stream' -import { setTimeout as delay } from 'node:timers/promises' -import { Context } from 'cordis' -import { describe, expect, it, vi } from 'vitest' -import type { Sandbox } from '@deepseek-ai/dsh-e2b' -import { - E2BFrameDecoder, - encodeE2BFrame, -} from '@deepseek-ai/dsh-e2b' -import type E2BSandboxService from '@deepseek-ai/dsh-e2b' -import type { - SubprocessHandle, - SubprocessOutcome, - SubprocessSpawnSpec, -} from '@deepseek-ai/dsh-subprocess' -import E2BSubprocessService from '@deepseek-ai/dsh-subprocess-e2b' -import { - encodeWorkerJson, -} from '@deepseek-ai/dsh-code-runtime-worker' -import E2BCodeRuntime from '@deepseek-ai/dsh-code-runtime-e2b' -import * as E2BCodeRuntimeInvariant from '../src/invariant.ts' -import { CODE_RUNNER_SOURCE } from '../src/runner-source.ts' -import InvariantService from '@deepseek-ai/dsh-invariants' - -class FakeHandle implements SubprocessHandle { - readonly pid = 123 - readonly stdin: Writable | undefined - readonly stdout: PassThrough | undefined - readonly stderr = undefined - readonly collected: SubprocessHandle['collected'] - readonly done: Promise - readonly writes: unknown[] = [] - readonly result = Promise.withResolvers() - terminated = 0 - waitCalls = 0 - private readonly decoder = new E2BFrameDecoder(10_000_000) - private readonly waitError: Error | undefined - private readonly waitResult: Promise | undefined - private settled = false - - constructor( - private readonly onMessage: (message: unknown, handle: FakeHandle) => void = () => {}, - options: { - stdin?: boolean - stdout?: boolean - stderr?: string - writeError?: Error - waitError?: Error - waitResult?: Promise - } = {}, - ) { - this.waitError = options.waitError - this.waitResult = options.waitResult - this.stdin = options.stdin === false - ? undefined - : options.writeError === undefined - ? new PassThrough() - : new Writable({ write: (_chunk, _encoding, callback) => { callback(options.writeError) } }) - this.stdout = options.stdout === false ? undefined : new PassThrough() - this.collected = options.stderr === undefined - ? {} - : { stderr: { readFrom: () => ({ text: options.stderr as string, nextOffset: 0, lossy: false }) } } - this.done = this.result.promise - this.stdin?.on('data', (chunk: Buffer) => { - for (const message of this.decoder.push(chunk.toString('ascii'))) { - this.writes.push(message) - this.onMessage(message, this) - } - }) - } - - emit(message: unknown): void { - this.stdout?.write(encodeE2BFrame(message)) - } - - emitRaw(text: string): void { - this.stdout?.write(text) - } - - exit(outcome: SubprocessOutcome = { exitCode: 0, signal: null }): void { - if (this.settled) return - this.settled = true - this.stdout?.end() - this.result.resolve(outcome) - } - - crash(error: unknown): void { - if (this.settled) return - this.settled = true - this.stdout?.end() - this.result.reject(error) - } - - terminate(): void { - this.terminated += 1 - this.exit({ exitCode: null, signal: 'SIGTERM' }) - } - - async waitForExit(): Promise { - this.waitCalls += 1 - if (this.waitError !== undefined) throw this.waitError - if (this.waitResult !== undefined) return await this.waitResult - return true - } -} - -interface RuntimeFixture { - ctx: Context - fiber: Awaited> - runtime: E2BCodeRuntime - sandbox: Sandbox - spawn: ReturnType SubprocessHandle>> - write: ReturnType - run: ReturnType -} - -async function setup( - handles: FakeHandle[] = [], - config: Record = {}, - sandboxOverrides: Partial = {}, - getSandbox?: () => Promise, -): Promise { - const write = vi.fn().mockResolvedValue([]) - const run = vi.fn().mockImplementation(async (command: string) => ({ - exitCode: 0, - stdout: command.startsWith('command -v') ? '/usr/bin/node\n' : '', - stderr: '', - })) - const sandbox = { - files: { write }, - commands: { run }, - ...sandboxOverrides, - } as unknown as Sandbox - const e2b = { - cwd: '/workspace', - runtimeRoot: '/workspace/.dsh-e2b', - getSandbox: getSandbox ?? (async () => sandbox), - } as unknown as E2BSandboxService - const spawn = vi.fn<(spec: SubprocessSpawnSpec) => SubprocessHandle>(() => { - const handle = handles.shift() - if (handle === undefined) throw new Error('no fake handle queued') - return handle - }) - const subprocess = Object.create(E2BSubprocessService.prototype) as E2BSubprocessService - Object.defineProperty(subprocess, 'spawn', { value: spawn }) - const ctx = new Context() - ctx.provide('e2b', e2b) - ctx.provide('subprocess', subprocess) - const fiber = await ctx.plugin(E2BCodeRuntime, config) - return { ctx, fiber, runtime: ctx.codeRuntime as E2BCodeRuntime, sandbox, spawn, write, run } -} - -function request(program = 'return 1') { - return { program, bindings: [] } -} - -async function runInstalledRunner( - code: string, - maxOutputBytes = 2_000_000, -): Promise<{ messages: unknown[]; stderr: string }> { - const directory = await mkdtemp(join(tmpdir(), 'dsh-e2b-code-runner-')) - const runner = join(directory, 'runner.mjs') - await writeFile(runner, CODE_RUNNER_SOURCE) - const child = spawn(process.execPath, [runner], { stdio: ['pipe', 'pipe', 'pipe'] }) - const decoder = new E2BFrameDecoder(4_000_000) - const messages: unknown[] = [] - let stderr = '' - let outputError: unknown - child.stdout.setEncoding('ascii') - child.stdout.on('data', (chunk: string) => { - try { - messages.push(...decoder.push(chunk)) - } catch (error: unknown) { - outputError = error - child.kill('SIGKILL') - } - }) - child.stderr.setEncoding('utf8') - child.stderr.on('data', (chunk: string) => { stderr += chunk }) - - try { - child.stdin.write(encodeE2BFrame({ - type: 'boot', - code, - namespaces: [], - computeMs: 1_000, - maxOutputBytes, - maxOldGenerationSizeMb: 128, - })) - await new Promise((resolve, reject) => { - const timeout = setTimeout(() => { - child.kill('SIGKILL') - reject(new Error('installed E2B code runner did not exit')) - }, 5_000) - child.once('error', (error) => { - clearTimeout(timeout) - reject(error) - }) - child.once('exit', () => { - clearTimeout(timeout) - resolve() - }) - }) - if (outputError !== undefined) throw outputError - decoder.finish() - return { messages, stderr } - } finally { - child.kill('SIGKILL') - await rm(directory, { recursive: true, force: true }) - } -} - -describe('E2BCodeRuntime', () => { - it('keeps model-owned descriptors outside the host framing process', async () => { - const forged = Buffer.from(JSON.stringify({ type: 'done' })).toString('base64') + '\\n' - const { messages, stderr } = await runInstalledRunner( - ` - const fs = await import('node:fs') - const childProcess = await import('node:child_process') - fs.writeSync(1, ${JSON.stringify(forged)}) - childProcess.spawnSync(process.execPath, ['-e', 'process.stdout.write("child-native")'], { stdio: 'inherit' }) - return true - `, - ) - const records = messages as Array<{ type?: string; text?: string; value?: unknown }> - const terminal = records.filter(message => message.type === 'done') - - expect(stderr).toBe('') - expect(terminal).toEqual([{ type: 'done', value: [true] }]) - expect(records.at(-1)).toEqual(terminal[0]) - expect(records.filter(message => message.type === 'log').map(message => message.text).join('')) - .toContain(forged + 'child-native') - }) - - it('bounds native descriptor output before it reaches the host protocol', async () => { - const { messages, stderr } = await runInstalledRunner( - "(await import('node:fs')).writeSync(1, 'x'.repeat(4096)); return true", - 64, - ) - const records = messages as Array<{ type?: string; text?: string }> - - expect(stderr).toBe('') - expect(records.at(-1)).toEqual({ type: 'output-limit' }) - expect(Buffer.byteLength(records.filter(message => message.type === 'log').map(message => message.text).join(''))) - .toBeLessThanOrEqual(62) - }) - - it('drains native worker pipes before emitting the terminal frame', async () => { - const expectedBytes = 1_048_576 - const { messages, stderr } = await runInstalledRunner( - ` - let stdoutPrototype = Object.getPrototypeOf(process.stdout) - while (stdoutPrototype && !Object.hasOwn(stdoutPrototype, 'write')) stdoutPrototype = Object.getPrototypeOf(stdoutPrototype) - Reflect.apply(stdoutPrototype.write, process.stdout, ['x'.repeat(${expectedBytes})]) - return true - `, - ) - const records = messages as Array<{ type?: string; text?: string }> - const terminalIndex = records.findIndex(message => message.type === 'done') - const nativeOutput = records - .slice(0, terminalIndex) - .filter(message => message.type === 'log') - .map(message => message.text ?? '') - .join('') - - expect(stderr).toBe('') - expect(terminalIndex).toBe(records.length - 1) - expect(Buffer.byteLength(nativeOutput)).toBe(expectedBytes) - }) - - it.skipIf(process.platform === 'win32')('reaps descendant-held controller pipes before completion', async () => { - const directory = await mkdtemp(join(tmpdir(), 'dsh-e2b-code-descendant-')) - const marker = join(directory, 'started') - const release = join(directory, 'release') - const childSource = ` - const fs = require('node:fs') - fs.writeFileSync(${JSON.stringify(marker)}, 'started') - const timer = setInterval(() => { - if (fs.existsSync(${JSON.stringify(release)})) clearInterval(timer) - }, 10) - ` - let running: ReturnType | undefined - try { - running = runInstalledRunner(` - const fs = await import('node:fs') - const childProcess = await import('node:child_process') - childProcess.spawn(process.execPath, ['-e', ${JSON.stringify(childSource)}], { - stdio: ['ignore', 'inherit', 'inherit'], - }) - while (!fs.existsSync(${JSON.stringify(marker)})) await new Promise(resolve => setTimeout(resolve, 5)) - return true - `) - const deadline = Date.now() + 2_000 - for (;;) { - try { - await access(marker) - break - } catch (error: unknown) { - if (Date.now() >= deadline) throw error - await delay(10) - } - } - const completed = await Promise.race([ - running.then(() => true), - delay(500).then(() => false), - ]) - await writeFile(release, '') - const { messages, stderr } = await running - - expect(completed).toBe(true) - expect(stderr).toBe('') - expect(messages.at(-1)).toEqual({ type: 'done', value: [true] }) - } finally { - await writeFile(release, '').catch(() => undefined) - await running?.catch(() => undefined) - await rm(directory, { recursive: true, force: true }) - } - }) - - it('prepares the remote runner and returns logs and a lossless completion', async () => { - const handle = new FakeHandle((message, current) => { - if ((message as { type?: string }).type !== 'boot') return - current.emit({ type: 'log', text: 'remote 你好' }) - current.emitRaw( - encodeE2BFrame({ type: 'done', value: encodeWorkerJson({ answer: 42 }) }) - + encodeE2BFrame({ type: 'log', text: 'ignored after done' }), - ) - current.emit({ type: 'log', text: 'also ignored after done' }) - }) - const fixture = await setup([handle]) - - await expect(fixture.runtime.run(request('const answer: number = 42; return { answer }'))) - .resolves.toEqual({ logs: ['remote 你好'], value: { answer: 42 } }) - expect(fixture.runtime.language).toBe('typescript') - expect(fixture.runtime.isolation).toBe('container') - expect(fixture.write).toHaveBeenCalledWith([{ path: '/workspace/.dsh-e2b/code-runtime-runner.mjs', data: CODE_RUNNER_SOURCE }]) - expect(fixture.run).toHaveBeenCalledWith("chmod 600 -- '/workspace/.dsh-e2b/code-runtime-runner.mjs'") - expect(fixture.spawn).toHaveBeenCalledWith(expect.objectContaining({ - argv: ['/usr/bin/node', '/workspace/.dsh-e2b/code-runtime-runner.mjs'], - cwd: '/workspace', - env: {}, - })) - expect(handle.terminated).toBe(1) - expect(handle.waitCalls).toBe(1) - await fixture.fiber.dispose() - }) - - it('bridges binding success, host rejection, unknown members, and invalid values', async () => { - const replies: unknown[] = [] - const handle = new FakeHandle((message, current) => { - const record = message as { type?: string; id?: number; ok?: boolean } - if (record.type === 'boot') { - current.emit({ type: 'call', id: 1, global: 'bridge', name: 'double', args: encodeWorkerJson({ value: 4 }) }) - current.emit({ type: 'call', id: 2, global: 'bridge', name: 'fail', args: encodeWorkerJson(null) }) - current.emit({ type: 'call', id: 3, global: 'bridge', name: 'missing', args: encodeWorkerJson(null) }) - current.emit({ type: 'call', id: 4, global: 'bridge', name: 'double', args: [] }) - current.emit({ type: 'call', id: 5, global: 'bridge', name: 'invalid', args: encodeWorkerJson(null) }) - current.emit({ type: 'call', id: 6, global: 'bridge', name: 'throwing', args: encodeWorkerJson(null) }) - current.emit({ type: 'call', id: 1, global: 'bridge', name: 'double', args: encodeWorkerJson({ value: 99 }) }) - return - } - if (record.type === 'reply') { - replies.push(message) - if (replies.length === 6) current.emit({ type: 'done', value: encodeWorkerJson('done') }) - } - }) - const fixture = await setup([handle]) - const result = await fixture.runtime.run({ - program: 'return await bridge.double({ value: 4 })', - bindings: [ - { - global: 'bridge', - errorClass: { name: 'BridgeError', memberNameProperty: 'member' }, - functions: { - double: async args => (args as { value: number }).value * 2, - fail: async () => { throw 'nope' }, - invalid: (async () => undefined) as never, - throwing: async () => Object.defineProperty({}, 'value', { - enumerable: true, - get: () => { throw new Error('getter failed') }, - }), - }, - }, - { global: 'plain', functions: {} }, - ], - }) - - expect(result).toEqual({ logs: [], value: 'done' }) - expect(replies.sort((left, right) => (left as { id: number }).id - (right as { id: number }).id)).toEqual([ - { type: 'reply', id: 1, ok: true, value: encodeWorkerJson(8) }, - { type: 'reply', id: 2, ok: false, message: 'nope' }, - { type: 'reply', id: 3, ok: false, message: 'unknown binding "bridge.missing"' }, - { type: 'reply', id: 4, ok: false, message: 'binding arguments must be lossless JSON' }, - { type: 'reply', id: 5, ok: false, message: 'binding resolution must be lossless JSON' }, - { type: 'reply', id: 6, ok: false, message: 'binding resolution must be lossless JSON' }, - ]) - await fixture.fiber.dispose() - }) - - it('ignores malformed runner traffic and classifies terminal runner messages', async () => { - const ignored = [ - null, 1, {}, { type: 'log' }, { type: 'call' }, - { type: 'call', id: 0, global: 'x', name: 'y', args: [] }, - { type: 'call', id: 1, global: 1, name: 'y', args: [] }, - { type: 'call', id: 1, global: 'x', name: 1, args: [] }, - { type: 'call', id: 1, global: 'x', name: 'y', args: {} }, - { type: 'done', error: null }, - { type: 'done', error: { kind: 'invented', message: 'x' } }, - { type: 'done', error: { kind: 'exception', message: 1 } }, - ] - const handles = [ - new FakeHandle((message, current) => { - if ((message as { type?: string }).type !== 'boot') return - for (const item of ignored) current.emit(item) - current.emit({ type: 'done' }) - }), - new FakeHandle((message, current) => { - if ((message as { type?: string }).type === 'boot') current.emit({ type: 'done', error: { kind: 'exception', message: 'boom' } }) - }), - new FakeHandle((message, current) => { - if ((message as { type?: string }).type === 'boot') current.emit({ type: 'done', value: [] }) - }), - new FakeHandle((message, current) => { - if ((message as { type?: string }).type === 'boot') current.emit({ type: 'output-limit' }) - }), - ] - const fixture = await setup(handles, { maxOutputBytes: 64, maxFrameBytes: 128 }) - - await expect(fixture.runtime.run(request())).resolves.toEqual({ logs: [] }) - await expect(fixture.runtime.run(request())).resolves.toEqual({ logs: [], error: { kind: 'exception', message: 'boom' } }) - await expect(fixture.runtime.run(request())).resolves.toEqual({ logs: [], error: { kind: 'invalid-output', message: 'program completion must be lossless JSON' } }) - await expect(fixture.runtime.run(request())).resolves.toEqual({ logs: [], error: { kind: 'output-limit', message: 'outer output exceeded 64 bytes' } }) - await fixture.fiber.dispose() - }) - - it('enforces the host output ledger and catches malformed bridge output', async () => { - const handles = [ - new FakeHandle((message, current) => { - if ((message as { type?: string }).type === 'boot') current.emit({ type: 'log', text: 'x'.repeat(1_000) }) - }), - new FakeHandle((message, current) => { - if ((message as { type?: string }).type === 'boot') current.emitRaw('not-base64\n') - }), - new FakeHandle((message, current) => { - if ((message as { type?: string }).type === 'boot') current.emitRaw('é') - }), - new FakeHandle((message, current) => { - if ((message as { type?: string }).type === 'boot') current.stdout?.emit('error', new Error('stdout broke')) - }), - ] - const fixture = await setup(handles, { maxOutputBytes: 128, maxFrameBytes: 4_096 }) - - expect((await fixture.runtime.run(request())).error?.kind).toBe('output-limit') - const malformed = (await fixture.runtime.run(request())).error - expect(malformed?.kind).toBe('worker-exit') - expect(malformed?.message).toContain('bridge failed') - expect((await fixture.runtime.run(request())).error?.message).toContain('non-ASCII') - expect((await fixture.runtime.run(request())).error).toEqual({ kind: 'worker-exit', message: 'E2B runtime stdout failed: stdout broke' }) - await fixture.fiber.dispose() - }) - - it('enforces the outbound frame bound on boot and binding replies', async () => { - const oversizedBoot = new FakeHandle() - const oversizedReply = new FakeHandle((message, current) => { - if ((message as { type?: string }).type === 'boot') { - current.emit({ type: 'call', id: 1, global: 'bridge', name: 'large', args: encodeWorkerJson(null) }) - } - }) - const fixture = await setup([oversizedBoot, oversizedReply], { maxOutputBytes: 128, maxFrameBytes: 512 }) - - const bootResult = await fixture.runtime.run(request(`return ${JSON.stringify('x'.repeat(1_000))}`)) - expect(bootResult.error).toMatchObject({ kind: 'worker-exit' }) - expect(bootResult.error?.message).toContain('frame exceeded its byte limit') - expect(oversizedBoot.writes).toHaveLength(0) - - const replyResult = await fixture.runtime.run({ - program: 'return await bridge.large(null)', - bindings: [{ global: 'bridge', functions: { large: async () => 'x'.repeat(1_000) } }], - }) - expect(replyResult.error).toMatchObject({ kind: 'worker-exit' }) - expect(replyResult.error?.message).toContain('frame exceeded its byte limit') - expect(oversizedReply.writes).toHaveLength(1) - await fixture.fiber.dispose() - }) - - it('contains stdin errors, process exits, spawn failures, and missing pipes', async () => { - const writeError = new FakeHandle(() => {}, { writeError: new Error('write callback broke') }) - const stdinError = new FakeHandle((message, current) => { - if ((message as { type?: string }).type === 'boot') current.stdin?.emit('error', new Error('stdin broke')) - }) - const earlyExit = new FakeHandle(() => {}, { stderr: 'remote diagnostic' }) - const quietExit = new FakeHandle() - const emptyStderrExit = new FakeHandle(() => {}, { stderr: '' }) - const spawnFailure = new FakeHandle() - const missingStdin = new FakeHandle(() => {}, { stdin: false }) - const missingStdout = new FakeHandle(() => {}, { stdout: false, waitError: new Error('missing-stream process query failed') }) - const truncated = new FakeHandle((message, current) => { - if ((message as { type?: string }).type === 'boot') { - current.emitRaw('YQ==') - setImmediate(() => { current.exit() }) - } - }) - const cleanupFailure = new FakeHandle((message, current) => { - if ((message as { type?: string }).type === 'boot') current.emit({ type: 'done' }) - }, { waitError: new Error('process query failed') }) - const fixture = await setup([ - writeError, stdinError, earlyExit, quietExit, emptyStderrExit, - spawnFailure, missingStdin, missingStdout, truncated, cleanupFailure, - ]) - - expect((await fixture.runtime.run(request())).error).toEqual({ kind: 'worker-exit', message: 'E2B runtime bridge write failed: write callback broke' }) - expect((await fixture.runtime.run(request())).error).toEqual({ kind: 'worker-exit', message: 'E2B runtime stdin failed: stdin broke' }) - setImmediate(() => { earlyExit.exit() }) - expect((await fixture.runtime.run(request())).error).toEqual({ kind: 'worker-exit', message: 'E2B runtime exited before completing: remote diagnostic' }) - setImmediate(() => { quietExit.exit() }) - expect((await fixture.runtime.run(request())).error).toEqual({ kind: 'worker-exit', message: 'E2B runtime exited before completing' }) - setImmediate(() => { emptyStderrExit.exit() }) - expect((await fixture.runtime.run(request())).error).toEqual({ kind: 'worker-exit', message: 'E2B runtime exited before completing' }) - setImmediate(() => { spawnFailure.crash('spawn rejected') }) - expect((await fixture.runtime.run(request())).error).toEqual({ kind: 'worker-exit', message: 'E2B runtime spawn failed: spawn rejected' }) - expect((await fixture.runtime.run(request())).error?.message).toContain('dropped a piped runtime stream') - expect((await fixture.runtime.run(request())).error).toEqual({ kind: 'worker-exit', message: 'E2B runtime cleanup failed: missing-stream process query failed' }) - expect(missingStdin.terminated).toBe(1) - expect(missingStdin.waitCalls).toBe(1) - expect(missingStdout.terminated).toBe(1) - expect(missingStdout.waitCalls).toBe(1) - expect((await fixture.runtime.run(request())).error).toEqual({ kind: 'worker-exit', message: 'E2B frame stream ended mid-frame' }) - expect((await fixture.runtime.run(request())).error).toEqual({ kind: 'worker-exit', message: 'E2B runtime cleanup failed: process query failed' }) - await fixture.fiber.dispose() - }) - - it('reports wall timeout, abort, pre-abort, type-strip failure, and disposal', async () => { - const timeout = new FakeHandle() - const abort = new FakeHandle() - const disposing = new FakeHandle() - const fixture = await setup([timeout, abort, disposing], { maxWallMs: 20 }) - - expect((await fixture.runtime.run(request())).error).toEqual({ kind: 'timeout', message: 'wall-clock ceiling reached (20ms)' }) - const controller = new AbortController() - const aborting = fixture.runtime.run({ ...request(), signal: controller.signal }) - controller.abort('stop') - expect((await aborting).error).toEqual({ kind: 'abort', message: 'stop' }) - expect((await fixture.runtime.run({ ...request(), signal: AbortSignal.abort('already') })).error) - .toEqual({ kind: 'abort', message: 'already' }) - expect((await fixture.runtime.run(request('enum E { A }'))).error?.kind).toBe('exception') - - const live = fixture.runtime.run(request()) - await new Promise(resolve => setImmediate(resolve)) - await fixture.fiber.dispose() - expect((await live).error).toEqual({ kind: 'abort', message: 'runtime disposed' }) - await expect(fixture.runtime.run(request())).rejects.toThrow('after disposal') - }) - - it('drops binding replies that settle after abort', async () => { - const controller = new AbortController() - const resolution = Promise.withResolvers() - const invoked = Promise.withResolvers() - const handle = new FakeHandle((message, current) => { - if ((message as { type?: string }).type === 'boot') { - current.emit({ type: 'call', id: 1, global: 'bridge', name: 'late', args: encodeWorkerJson(null) }) - } - }) - const fixture = await setup([handle]) - const running = fixture.runtime.run({ - program: 'return await bridge.late(null)', - bindings: [{ - global: 'bridge', - functions: { - late: async () => { - invoked.resolve(undefined) - return await resolution.promise - }, - }, - }], - signal: controller.signal, - }) - await invoked.promise - controller.abort('stop') - expect((await running).error).toEqual({ kind: 'abort', message: 'stop' }) - resolution.resolve('late') - await new Promise(resolve => setImmediate(resolve)) - expect(handle.writes).toHaveLength(1) - await fixture.fiber.dispose() - }) - - it('validates binding and runtime configuration before remote execution', async () => { - const fixture = await setup([]) - const invalidRequests = [ - { global: 'not-valid!', functions: {} }, - { global: 'await', functions: {} }, - { global: 'console', functions: {} }, - { global: 'same', functions: {} }, - { global: 'same', functions: {} }, - { global: 'ok', functions: {}, errorClass: { name: 'not-valid!', memberNameProperty: 'member' } }, - { global: 'ok', functions: {}, errorClass: { name: 'await', memberNameProperty: 'member' } }, - { global: 'Clash', functions: {}, errorClass: { name: 'Clash', memberNameProperty: 'member' } }, - { global: 'one', functions: {}, errorClass: { name: 'Err', memberNameProperty: 'member' } }, - { global: 'two', functions: {}, errorClass: { name: 'Err', memberNameProperty: 'member' } }, - { global: 'ok', functions: {}, errorClass: { name: 'Err', memberNameProperty: '' } }, - { global: 'ok', functions: {}, errorClass: { name: 'Err', memberNameProperty: 'message' } }, - ] - for (const bindings of [ - [invalidRequests[0]], [invalidRequests[1]], [invalidRequests[2]], - invalidRequests.slice(3, 5), [invalidRequests[5]], [invalidRequests[6]], - [invalidRequests[7]], invalidRequests.slice(8, 10), [invalidRequests[10]], [invalidRequests[11]], - ]) { - await expect(fixture.runtime.run({ program: 'return 1', bindings: bindings as never })).rejects.toThrow() - } - await fixture.fiber.dispose() - - for (const config of [ - { computeMs: 0 }, { computeMs: 1.5 }, { maxOutputBytes: 3 }, - { maxWallMs: 2_147_483_648 }, { maxFrameBytes: 10, maxOutputBytes: 20 }, - ]) { - const ctx = new Context() - const subprocess = Object.create(E2BSubprocessService.prototype) as E2BSubprocessService - ctx.provide('e2b', { getSandbox: async () => ({}) } as never) - ctx.provide('subprocess', subprocess) - await expect(ctx.plugin(E2BCodeRuntime, config)).rejects.toThrow() - } - - const wrong = new Context() - wrong.provide('e2b', { getSandbox: async () => ({}) } as never) - wrong.provide('subprocess', {} as never) - await expect(wrong.plugin(E2BCodeRuntime, {})).rejects.toThrow('dsh-subprocess-e2b') - }) - - it('turns asynchronous runtime preparation failure into a run result', async () => { - const sandbox = { - files: { write: vi.fn().mockRejectedValue(new Error('upload failed')) }, - commands: { run: vi.fn() }, - } as unknown as Sandbox - const fixture = await setup([], {}, sandbox) - expect((await fixture.runtime.run(request())).error).toEqual({ - kind: 'worker-exit', - message: 'E2B runtime setup failed: upload failed', - }) - await fixture.fiber.dispose() - }) - - it('returns disposal when remote preparation completes after teardown', async () => { - const gate = Promise.withResolvers() - const fixture = await setup([], {}, {}, () => gate.promise) - const running = fixture.runtime.run(request()) - const disposing = fixture.fiber.dispose() - let disposed = false - void disposing.then(() => { disposed = true }) - await new Promise(resolve => setImmediate(resolve)) - const disposedBeforeSetup = disposed - gate.resolve(fixture.sandbox) - await disposing - expect(disposedBeforeSetup).toBe(false) - expect((await running).error).toEqual({ kind: 'abort', message: 'runtime disposed' }) - expect(fixture.write).not.toHaveBeenCalled() - }) - - it('observes abort while runtime preparation is pending', async () => { - const gate = Promise.withResolvers() - const fixture = await setup([], {}, {}, () => gate.promise) - const controller = new AbortController() - const running = fixture.runtime.run({ ...request(), signal: controller.signal }) - - controller.abort('stop during setup') - const early = await Promise.race([ - running.then(result => ({ kind: 'result' as const, result })), - new Promise<{ kind: 'pending' }>((resolve) => { setImmediate(() => { resolve({ kind: 'pending' }) }) }), - ]) - expect(fixture.spawn).not.toHaveBeenCalled() - - gate.resolve(fixture.sandbox) - expect(early).toMatchObject({ kind: 'result', result: { error: { kind: 'abort', message: 'stop during setup' } } }) - await running - await fixture.fiber.dispose() - }) - - it('classifies an abort that races synchronous subprocess spawn', async () => { - const fixture = await setup() - const controller = new AbortController() - fixture.spawn.mockImplementationOnce(() => { - controller.abort('stop at spawn') - throw new Error('aborted before spawn') - }) - - expect((await fixture.runtime.run({ ...request(), signal: controller.signal })).error) - .toEqual({ kind: 'abort', message: 'stop at spawn' }) - - fixture.spawn.mockImplementationOnce(() => { throw new Error('synchronous spawn failure') }) - expect((await fixture.runtime.run(request())).error).toEqual({ - kind: 'worker-exit', - message: 'E2B runtime spawn failed: synchronous spawn failure', - }) - await fixture.fiber.dispose() - - const disposingFixture = await setup() - disposingFixture.spawn.mockImplementationOnce(() => { - void (disposingFixture.runtime as unknown as { teardown(): Promise }).teardown() - throw new Error('spawn raced disposal') - }) - expect((await disposingFixture.runtime.run(request())).error) - .toEqual({ kind: 'abort', message: 'runtime disposed' }) - await disposingFixture.fiber.dispose() - }) - - it('closes both abort races around runtime readiness and live-run publication', async () => { - let preparationAborted = false - const preparationSignal = { - get aborted() { return preparationAborted }, - reason: 'preparation race', - addEventListener() { preparationAborted = true }, - removeEventListener() {}, - } as unknown as AbortSignal - const liveHandle = new FakeHandle() - const fixture = await setup([liveHandle]) - expect((await fixture.runtime.run({ ...request(), signal: preparationSignal })).error) - .toEqual({ kind: 'abort', message: 'preparation race' }) - expect(fixture.spawn).not.toHaveBeenCalled() - - let liveAborted = false - let registrations = 0 - const liveSignal = { - get aborted() { return liveAborted }, - reason: 'live publication race', - addEventListener() { - registrations += 1 - if (registrations === 2) liveAborted = true - }, - removeEventListener() {}, - } as unknown as AbortSignal - expect((await fixture.runtime.run({ ...request(), signal: liveSignal })).error) - .toEqual({ kind: 'abort', message: 'live publication race' }) - await fixture.fiber.dispose() - }) - - it('retains a live run until remote cleanup reaches quiescence', async () => { - const cleanup = Promise.withResolvers() - const handle = new FakeHandle((message, current) => { - if ((message as { type?: string }).type === 'boot') current.emit({ type: 'done' }) - }, { waitResult: cleanup.promise }) - const fixture = await setup([handle]) - const running = fixture.runtime.run(request()) - await vi.waitFor(() => { expect(handle.waitCalls).toBe(1) }) - - const disposing = fixture.fiber.dispose() - let disposed = false - void disposing.then(() => { disposed = true }) - await new Promise(resolve => setImmediate(resolve)) - const disposedBeforeCleanup = disposed - - cleanup.resolve(true) - await expect(running).resolves.toEqual({ logs: [] }) - await expect(disposing).resolves.toBeUndefined() - expect(disposedBeforeCleanup).toBe(false) - }) - - it('registers the package-owned invariant companion', async () => { - const ctx = new Context() - await ctx.plugin(InvariantService, { enabled: true }) - const fiber = await ctx.plugin(E2BCodeRuntimeInvariant).await() - await fiber.dispose() - }) -}) diff --git a/packages/e2b/code-runtime-e2b/tsconfig.json b/packages/e2b/code-runtime-e2b/tsconfig.json deleted file mode 100644 index 3bc2e21873..0000000000 --- a/packages/e2b/code-runtime-e2b/tsconfig.json +++ /dev/null @@ -1,20 +0,0 @@ -{ - "extends": "../../../tsconfig.base.json", - "compilerOptions": { - "rootDir": "src", - "outDir": "lib/types" - }, - "include": ["src"], - "references": [ - { "path": "../../../vendor/cosmokit" }, - { "path": "../../../vendor/cordis" }, - { "path": "../../../vendor/schemastery" }, - { "path": "../../code-runtime/code-runtime" }, - { "path": "../../code-runtime/code-runtime-worker" }, - { "path": "../e2b" }, - { "path": "../../core/session" }, - { "path": "../subprocess-e2b" }, - { "path": "../../util/timeout" }, - { "path": "../../support/invariants" } - ] -} diff --git a/packages/e2b/e2b/README.i18n.yaml b/packages/e2b/e2b/README.i18n.yaml index 164bb47784..8ffb6e981f 100644 --- a/packages/e2b/e2b/README.i18n.yaml +++ b/packages/e2b/e2b/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/e2b/e2b/README.md -README.md: 5f89a4bcffdfd11fef8929d2a2ceecb41af319e2 -README.zh.md: b4956033bae131bb8aa236276323ecba30f00115 +README.md: 00264b8f0b03e4af8512025322fe3e457e7b6b9b +README.zh.md: 93fad661ded446e78e3addc0c8b2b8fdc39bd994 diff --git a/packages/e2b/e2b/README.md b/packages/e2b/e2b/README.md index 5f89a4bcff..00264b8f0b 100644 --- a/packages/e2b/e2b/README.md +++ b/packages/e2b/e2b/README.md @@ -2,7 +2,7 @@ English | [中文](README.zh.md) -Shared lifecycle owner for one E2B sandbox. Capability adapters inject `ctx.e2b`, await its single SDK handle, and therefore inhabit the same remote Linux working tree and process world. The package pins `e2b@2.29.1`; the [family map](../README.md) lists the opt-in adapters. +Shared lifecycle owner for one E2B sandbox. The filesystem and subprocess adapters inject `ctx.e2b`, await its single SDK handle, and therefore inhabit the same remote Linux working tree and process world. The package pins `e2b@2.29.1`; the [family map](../README.md) lists the opt-in composition. ## Configuration diff --git a/packages/e2b/e2b/README.zh.md b/packages/e2b/e2b/README.zh.md index b4956033ba..93fad661de 100644 --- a/packages/e2b/e2b/README.zh.md +++ b/packages/e2b/e2b/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -一个 E2B 沙箱的共享生命周期所有者。功能适配器注入 `ctx.e2b`,等待其唯一的 SDK 句柄,因此处于同一个远程 Linux 工作树与进程环境中。本包固定使用 `e2b@2.29.1`;可选适配器见[包族索引](../README.md)。 +一个 E2B 沙箱的共享生命周期所有者。文件系统与进程管理适配器注入 `ctx.e2b`,等待其唯一的 SDK 句柄,因此处于同一个远程 Linux 工作树与进程环境中。本包固定使用 `e2b@2.29.1`;可选组合见[包族索引](../README.md)。 ## 配置 diff --git a/packages/e2b/e2b/src/frame.ts b/packages/e2b/e2b/src/frame.ts deleted file mode 100644 index 5b6fe536c0..0000000000 --- a/packages/e2b/e2b/src/frame.ts +++ /dev/null @@ -1,96 +0,0 @@ -/** ASCII/base64 JSON framing for byte-faithful protocols over E2B text callbacks. */ - -import { Buffer } from 'node:buffer' - -const BASE64_LINE = /^(?:[A-Za-z0-9+/]{4})*(?:[A-Za-z0-9+/]{2}==|[A-Za-z0-9+/]{3}=)?$/ - -/** - * Encode one JSON-compatible value as a newline-delimited ASCII frame. - * @param value - Value accepted by `JSON.stringify`. - * @returns Base64-encoded UTF-8 JSON followed by one newline. - */ -export function encodeE2BFrame(value: unknown): string { - return encodeFrame(value) -} - -/** - * Encode one JSON-compatible value while enforcing the decoded frame bound. - * @param value - Value accepted by `JSON.stringify`. - * @param maxFrameBytes - Maximum UTF-8 JSON bytes in the encoded frame. - * @returns Base64-encoded UTF-8 JSON followed by one newline. - */ -export function encodeBoundedE2BFrame(value: unknown, maxFrameBytes: number): string { - if (!Number.isSafeInteger(maxFrameBytes) || maxFrameBytes <= 0) { - throw new Error('E2B frame maxFrameBytes must be a positive safe integer') - } - return encodeFrame(value, maxFrameBytes) -} - -function encodeFrame(value: unknown, maxFrameBytes?: number): string { - const json: unknown = JSON.stringify(value) - if (typeof json !== 'string') throw new Error('E2B frame value is not JSON-serializable') - const bytes = Buffer.from(json) - if (maxFrameBytes !== undefined && bytes.length > maxFrameBytes) { - throw new Error('E2B frame exceeded its byte limit') - } - return `${bytes.toString('base64')}\n` -} - -/** Incremental decoder for newline-delimited base64 JSON frames. */ -export class E2BFrameDecoder { - private pending = '' - private readonly maxEncodedChars: number - - /** @param maxFrameBytes - Maximum decoded UTF-8 JSON bytes in one frame. */ - constructor(private readonly maxFrameBytes: number) { - if (!Number.isSafeInteger(maxFrameBytes) || maxFrameBytes <= 0) { - throw new Error('E2B frame maxFrameBytes must be a positive safe integer') - } - this.maxEncodedChars = Math.ceil(maxFrameBytes / 3) * 4 - } - - /** - * Consume one E2B callback chunk. - * @param chunk - ASCII text received from the remote helper. - * @returns Every complete decoded JSON value, in order. - */ - push(chunk: string): unknown[] { - if (/[^\x0a\x20-\x7e]/.test(chunk)) throw new Error('E2B frame stream contained non-ASCII data') - this.pending += chunk - const values: unknown[] = [] - for (;;) { - const newline = this.pending.indexOf('\n') - if (newline < 0) { - if (this.pending.length > this.maxEncodedChars) throw new Error('E2B frame exceeded its byte limit') - return values - } - const line = this.pending.slice(0, newline) - this.pending = this.pending.slice(newline + 1) - values.push(this.decode(line)) - } - } - - /** Reject a truncated final frame. */ - finish(): void { - if (this.pending.length !== 0) throw new Error('E2B frame stream ended mid-frame') - } - - private decode(line: string): unknown { - if (line.length === 0 || line.length > this.maxEncodedChars || !BASE64_LINE.test(line)) { - throw new Error('E2B frame contained invalid base64 or exceeded its byte limit') - } - const bytes = Buffer.from(line, 'base64') - if (bytes.length > this.maxFrameBytes) throw new Error('E2B frame exceeded its byte limit') - let json: string - try { - json = new TextDecoder('utf-8', { fatal: true }).decode(bytes) - } catch (error: unknown) { - throw new Error('E2B frame contained invalid UTF-8', { cause: error }) - } - try { - return JSON.parse(json) as unknown - } catch (error: unknown) { - throw new Error('E2B frame contained invalid JSON', { cause: error }) - } - } -} diff --git a/packages/e2b/e2b/src/index.ts b/packages/e2b/e2b/src/index.ts index 11625b3b21..184c49f811 100644 --- a/packages/e2b/e2b/src/index.ts +++ b/packages/e2b/e2b/src/index.ts @@ -10,8 +10,6 @@ import z from 'schemastery' import { Sandbox } from 'e2b' import type { Branded } from '@deepseek-ai/dsh-brand' -export { E2BFrameDecoder, encodeBoundedE2BFrame, encodeE2BFrame } from './frame.ts' - export { CommandExitError, FileNotFoundError, @@ -44,26 +42,6 @@ export function quoteE2BShellArg(value: string): string { return `'${value.replaceAll('\'', "'\"'\"'")}'` } -/** - * Resolve one executable inside an E2B sandbox and require an absolute result. - * @param sandbox - Sandbox whose PATH and filesystem own the executable. - * @param command - Absolute path or bare executable name. - * @returns Verified absolute remote executable path. - */ -export async function resolveE2BExecutable(sandbox: Sandbox, command: string): Promise { - if (command.length === 0) throw new Error('E2B executable name must be non-empty') - if (posix.isAbsolute(command)) { - await sandbox.commands.run(`test -f ${quoteE2BShellArg(command)} -a -x ${quoteE2BShellArg(command)}`) - return command - } - const result = await sandbox.commands.run(`command -v -- ${quoteE2BShellArg(command)}`) - const executable = result.stdout.trim() - if (!posix.isAbsolute(executable) || executable.includes('\n')) { - throw new Error(`E2B executable ${JSON.stringify(command)} did not resolve to one absolute path`) - } - return executable -} - /** Action taken on the owned sandbox when the Cordis service is disposed. */ export type E2BDisposeMode = 'kill' | 'pause' | 'leave' diff --git a/packages/e2b/e2b/tests/composition.e2e.ts b/packages/e2b/e2b/tests/composition.e2e.ts index 1bfbab1e8e..98fcdeae7f 100644 --- a/packages/e2b/e2b/tests/composition.e2e.ts +++ b/packages/e2b/e2b/tests/composition.e2e.ts @@ -3,13 +3,13 @@ import { join } from 'node:path' import { fileURLToPath } from 'node:url' import { Context } from 'cordis' import { describe, expect, it } from 'vitest' -import { AgentMessageId } from '@deepseek-ai/dsh-agent' import type { Agent } from '@deepseek-ai/dsh-agent' import { runLoaderSmoke } from '@deepseek-ai/dsh-loader-smoke' import { Sandbox, SandboxNotFoundError } from '@deepseek-ai/dsh-e2b' -import { E2BPtyBackend } from '@deepseek-ai/dsh-pty-e2b' -import { PtySessionId } from '@deepseek-ai/dsh-pty' +import PtyService, { PtySessionId } from '@deepseek-ai/dsh-pty' +import { LocalPtyBackend } from '@deepseek-ai/dsh-pty-local' import { Session, SessionId } from '@deepseek-ai/dsh-session' +import E2BSubprocessService from '@deepseek-ai/dsh-subprocess-e2b' const fixtureRoot = fileURLToPath(new URL('../../../../examples/headless-agent/tests/fixtures/e2b/e2b/', import.meta.url)) const binScript = join(fixtureRoot, 'bin.ts') @@ -29,7 +29,17 @@ describe.skipIf(!process.env.E2B_API_KEY)('E2B live Loader composition', () => { }) try { const ctx = new Context() - ctx.provide('e2b', { cwd: '/home/user', getSandbox: async () => sandbox } as never) + ctx.provide('e2b', { + cwd: '/home/user', + runtimeRoot: '/home/user/.dsh-e2b', + getSandbox: async () => sandbox, + } as never) + ctx.provide('sandboxPolicy', { + defaultMode: 'danger-full-access', + workspaceRoot: '/home/user', + } as never) + const ptyFiber = await ctx.plugin(PtyService) + const subprocessFiber = await ctx.plugin(E2BSubprocessService) const ownerId = SessionId('e2b-pty-env-owner') const owner: Agent = { id: ownerId, @@ -38,17 +48,19 @@ describe.skipIf(!process.env.E2B_API_KEY)('E2B live Loader composition', () => { status: 'idle', acceptsNextStep: false, ctx, - followup: () => AgentMessageId('unused'), - steer: () => AgentMessageId('unused'), - inject: () => AgentMessageId('unused'), - send: () => AgentMessageId('unused'), + followup() {}, + steer() {}, + inject() {}, + send() {}, cancel() {}, whenIdle: () => Promise.resolve(), } - const backend = new E2BPtyBackend(ctx, { - backendType: 'shell', rows: 24, cols: 80, + const backend = new LocalPtyBackend(ctx, { + backendType: 'shell', shellPath: '/bin/bash', shellArgs: ['--noprofile', '--norc', '-i'], + rows: 24, cols: 80, scrollbackLines: 100, scrollbackMaxBytes: 65_536, maxReadBytes: 16_384, - pollIntervalMs: 25, idleSilenceMs: 1_000, timeoutMs: 5_000, disposeGraceMs: 1_000, + pollIntervalMs: 25, exactProbeAfterMs: 150, idleSilenceMs: 1_000, + handoffGraceMs: 500, timeoutMs: 5_000, disposeGraceMs: 1_000, }) const session = await backend.spawn({ sessionId: PtySessionId('env'), owner, type: 'shell' }) const result = await session.startSend({ @@ -59,6 +71,8 @@ describe.skipIf(!process.env.E2B_API_KEY)('E2B live Loader composition', () => { expect(result.viewport).not.toContain('sentinel-secret') expect(result.viewport).not.toContain('sentinel-stale') await session.close('environment test complete') + await subprocessFiber.dispose() + await ptyFiber.dispose() } finally { await sandbox.kill().catch(() => false) } diff --git a/packages/e2b/e2b/tests/e2b.spec.ts b/packages/e2b/e2b/tests/e2b.spec.ts index 58d2ffbd7a..c0770f5917 100644 --- a/packages/e2b/e2b/tests/e2b.spec.ts +++ b/packages/e2b/e2b/tests/e2b.spec.ts @@ -2,12 +2,8 @@ import { beforeEach, describe, expect, it, vi } from 'vitest' import { Context } from 'cordis' import type { Sandbox as SandboxType } from 'e2b' import E2BSandboxService, { - E2BFrameDecoder, E2BSandboxId, - encodeBoundedE2BFrame, - encodeE2BFrame, quoteE2BShellArg, - resolveE2BExecutable, } from '@deepseek-ai/dsh-e2b' import * as E2BInvariant from '../src/invariant.ts' import InvariantService from '@deepseek-ai/dsh-invariants' @@ -229,57 +225,6 @@ describe('E2B helpers and invariant companion', () => { expect(quoteE2BShellArg("a'b $HOME")).toBe("'a'\"'\"'b $HOME'") }) - it('resolves absolute and PATH executables inside the sandbox', async () => { - const run = vi.fn() - .mockResolvedValueOnce({ exitCode: 0, stdout: '', stderr: '' }) - .mockResolvedValueOnce({ exitCode: 0, stdout: '/usr/bin/node\n', stderr: '' }) - const sandbox = { commands: { run } } as unknown as SandboxType - await expect(resolveE2BExecutable(sandbox, '/bin/bash')).resolves.toBe('/bin/bash') - await expect(resolveE2BExecutable(sandbox, 'node')).resolves.toBe('/usr/bin/node') - expect(run).toHaveBeenNthCalledWith(1, "test -f '/bin/bash' -a -x '/bin/bash'") - expect(run).toHaveBeenNthCalledWith(2, "command -v -- 'node'") - }) - - it('rejects empty or non-absolute executable resolutions', async () => { - const sandbox = { - commands: { run: vi.fn().mockResolvedValue({ exitCode: 0, stdout: 'relative\npath\n', stderr: '' }) }, - } as unknown as SandboxType - await expect(resolveE2BExecutable(sandbox, '')).rejects.toThrow('non-empty') - await expect(resolveE2BExecutable(sandbox, 'tool')).rejects.toThrow('did not resolve') - }) - - it('round-trips split and adjacent ASCII/base64 JSON frames', () => { - const decoder = new E2BFrameDecoder(128) - const encoded = encodeE2BFrame({ text: '你好' }) + encodeE2BFrame([1, true]) - expect(decoder.push(encoded.slice(0, 5))).toEqual([]) - expect(decoder.push(encoded.slice(5))).toEqual([{ text: '你好' }, [1, true]]) - expect(() => { decoder.finish() }).not.toThrow() - expect(() => encodeE2BFrame(undefined)).toThrow('not JSON-serializable') - }) - - it('bounds outbound frames by decoded UTF-8 bytes', () => { - const exact = encodeBoundedE2BFrame({ text: '你' }, 14) - expect(new E2BFrameDecoder(14).push(exact)).toEqual([{ text: '你' }]) - expect(() => encodeBoundedE2BFrame({ text: '你' }, 13)).toThrow('byte limit') - expect(() => encodeBoundedE2BFrame(null, 0)).toThrow('positive safe integer') - expect(() => encodeBoundedE2BFrame(null, 1.5)).toThrow('positive safe integer') - }) - - it('rejects malformed, oversized, and truncated frame streams', () => { - expect(() => new E2BFrameDecoder(0)).toThrow('positive safe integer') - expect(() => new E2BFrameDecoder(1.5)).toThrow('positive safe integer') - expect(() => new E2BFrameDecoder(4).push('é')).toThrow('non-ASCII') - expect(() => new E2BFrameDecoder(3).push('AAAAA')).toThrow('byte limit') - expect(() => new E2BFrameDecoder(8).push('\n')).toThrow('invalid base64') - expect(() => new E2BFrameDecoder(8).push('abc!\n')).toThrow('invalid base64') - expect(() => new E2BFrameDecoder(2).push(`${Buffer.from('abc').toString('base64')}\n`)).toThrow('byte limit') - expect(() => new E2BFrameDecoder(8).push('/w==\n')).toThrow('invalid UTF-8') - expect(() => new E2BFrameDecoder(16).push(`${Buffer.from('not-json').toString('base64')}\n`)).toThrow('invalid JSON') - const truncated = new E2BFrameDecoder(8) - truncated.push('YQ==') - expect(() => { truncated.finish() }).toThrow('mid-frame') - }) - it('registers the package-owned empty invariant installer', async () => { const ctx = new Context() await ctx.plugin(InvariantService, { enabled: true }) diff --git a/packages/e2b/fs-e2b/README.i18n.yaml b/packages/e2b/fs-e2b/README.i18n.yaml index 95a3a9eea6..6e45ccaaf6 100644 --- a/packages/e2b/fs-e2b/README.i18n.yaml +++ b/packages/e2b/fs-e2b/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/e2b/fs-e2b/README.md -README.md: a505703fc764f8441fa54d2b1d922762eb3fafdf -README.zh.md: 626fdb52979d29f12d1bc1b67b0ea13730003af7 +README.md: 86ad8720d3e4c7ce70ee0ac8c41713af03ed297a +README.zh.md: 90dff57a5b9fe786aea3315633125536277ecc20 diff --git a/packages/e2b/fs-e2b/README.md b/packages/e2b/fs-e2b/README.md index a505703fc7..86ad8720d3 100644 --- a/packages/e2b/fs-e2b/README.md +++ b/packages/e2b/fs-e2b/README.md @@ -7,7 +7,9 @@ E2B implementation of the [`@deepseek-ai/dsh-fs`](../../fs/fs/README.md) provide ## Behavior - **Remote identity and metadata** — relative paths resolve as POSIX paths against the caller cwd or `ctx.e2b.cwd`; `realpath -m` supplies canonical target identity without requiring the final file to exist. `stat`, no-follow `lstat`, and stable one-level directory listings project E2B metadata into the filesystem seam. Versions are opaque hashes of E2B metadata plus a per-write extended attribute. +- **Execution-world paths** — canonical targets expose absolute POSIX process paths, percent-encoded `file:` URIs, and provider-owned containment checks, so generic subprocess consumers never parse E2B target ids or apply host path rules. - **UTF-8 reads** — whole reads and streamed reads preserve cross-chunk decoding, reject invalid UTF-8, and use the seam's 8192-byte NUL sample for binary detection. The model-facing tool still owns size selection and line windowing. +- **Stable bounded reads** — a dependency-free Node helper walks directory descriptors with no-follow opens and reads one held regular-file descriptor through the byte cap. Generic LSP queries therefore reject parent swaps, non-files, invalid UTF-8, and growth past the configured document limit before server startup. - **Atomic mutations** — writes upload a mode-`0600` temporary sibling, preserve an existing file's POSIX mode, and publish through E2B's same-directory atomic rename. The rename response supplies the committed version, so no fallible metadata request follows the commit point. E2B creates missing parent directories. Literal edits LF-normalize for matching, restore dominant CRLF storage, and serialize mutations per canonical target within the host process. Optional create/version guards keep the base seam's observed-state semantics. - **Failures and cancellation** — E2B not-found, permission, abort, and other controller failures map to the existing `FsError` vocabulary. Cancellation is best-effort at SDK request boundaries; a successful rename is the commit point. @@ -26,4 +28,4 @@ No direct invalidation; the named consumer owns any request-prefix changes. - **No host synchronization** — an empty E2B cwd stays empty until a tool, command, template, or external process populates it; local files are neither uploaded nor reflected back. - **Mutation coordination is host-process-local** — another harness connection or remote command can race the adapter; version guards detect only metadata changes represented by E2B. - **Whole-file mutation costs remain** — overwrite diffs and literal edits read complete files into host memory, and every operation incurs E2B controller latency. -- **Custom templates must support the used Linux and envd features** — `realpath`, `chmod`, `mv`, same-filesystem POSIX rename, streaming reads, and file metadata extended attributes are required; unsupported templates fail rather than degrade silently. +- **Custom templates must support the used Linux, Node, procfs, and envd features** — `realpath`, `chmod`, `mv`, same-filesystem POSIX rename, streaming reads, file metadata extended attributes, `/proc/self/fd`, and no-follow descriptor opens are required; unsupported templates fail rather than degrade silently. diff --git a/packages/e2b/fs-e2b/README.zh.md b/packages/e2b/fs-e2b/README.zh.md index 626fdb5297..90dff57a5b 100644 --- a/packages/e2b/fs-e2b/README.zh.md +++ b/packages/e2b/fs-e2b/README.zh.md @@ -7,7 +7,9 @@ ## 行为 - **远程身份与元数据**:相对路径以调用方 cwd 或 `ctx.e2b.cwd` 为基准,按照 POSIX 路径解析;`realpath -m` 提供规范化目标身份,且不要求最终文件存在。`stat`、不跟随链接的 `lstat` 和稳定的单层目录列表会把 E2B 元数据投影到文件系统 seam。版本是 E2B 元数据与每次写入设置的扩展属性所组成的不透明哈希。 +- **执行世界路径**:规范化目标公开绝对 POSIX 进程路径、百分号编码的 `file:` URI,以及由提供方负责的包含关系检查,因此通用进程管理消费方无需解析 E2B 目标 ID,也不会套用宿主路径规则。 - **UTF-8 读取**:完整读取和流式读取会保留跨分片解码、拒绝无效 UTF-8,并使用 seam 的 8192 字节 NUL 样本检测二进制内容。面向模型的工具仍负责选择大小和行窗口。 +- **稳定的有界读取**:一个零依赖 Node 辅助程序会以不跟随链接的方式逐级打开目录描述符,并通过一个持续持有的常规文件描述符读取至字节上限。因此,通用 LSP 查询会在服务器启动前拒绝父目录交换、非文件、无效 UTF-8,以及增长后超出所配置文档上限的文件。 - **原子变更**:写入会上传 mode 为 `0600` 的同级临时文件,保留现有文件的 POSIX mode,并通过 E2B 的同目录原子重命名发布。重命名响应会提供已提交的版本,因此提交点之后不会再进行可能失败的元数据请求。E2B 会创建缺失的父目录。字面量编辑匹配时会规范化为 LF,存储时恢复占主导的 CRLF,并在宿主进程内按规范化目标串行执行变更。可选的创建/版本防护会保留基础 seam 的已观察状态语义。 - **失败与取消**:E2B 的未找到、权限、中止及其他控制器故障会映射到现有 `FsError` 词汇。取消在 SDK 请求边界上采用尽力而为语义;成功 rename 是提交点。 @@ -26,4 +28,4 @@ - **不提供宿主同步**:空的 E2B cwd 会一直为空,直到工具、命令、模板或外部进程填充它;本地文件既不会上传,也不会同步回本地。 - **变更协调仅限宿主进程内**:另一个 harness 连接或远程命令可能与适配器发生竞态;版本防护只能检测 E2B 元数据所体现的变更。 - **仍需承担完整文件变更成本**:覆盖差异和字面量编辑会把完整文件读入宿主内存,每项操作也都会产生 E2B 控制器延迟。 -- **自定义模板必须支持所用的 Linux 与 envd 功能**:必须支持 `realpath`、`chmod`、`mv`、同一文件系统内的 POSIX rename、流式读取和文件元数据扩展属性;不支持的模板会失败,而不会静默降级。 +- **自定义模板必须支持所用的 Linux、Node、procfs 与 envd 功能**:必须支持 `realpath`、`chmod`、`mv`、同一文件系统内的 POSIX rename、流式读取、文件元数据扩展属性、`/proc/self/fd` 和不跟随链接的描述符打开操作;不支持的模板会失败,而不会静默降级。 diff --git a/packages/e2b/fs-e2b/src/index.ts b/packages/e2b/fs-e2b/src/index.ts index 2a7fd1275f..2452e856ca 100644 --- a/packages/e2b/fs-e2b/src/index.ts +++ b/packages/e2b/fs-e2b/src/index.ts @@ -5,6 +5,7 @@ */ import { createHash, randomUUID } from 'node:crypto' +import { Buffer } from 'node:buffer' import { posix } from 'node:path' import { FileSystem, FsError, FsTargetKey, FsVersion } from '@deepseek-ai/dsh-fs' import type { @@ -24,10 +25,18 @@ import { quoteE2BShellArg, } from '@deepseek-ai/dsh-e2b' import type { EntryInfo, Sandbox } from '@deepseek-ai/dsh-e2b' +import { BOUNDED_READER_SOURCE } from './source-reader.ts' const VERSION_METADATA_KEY = 'dsh-version' const BINARY_SAMPLE_BYTES = 8192 +type BoundedReadResponse = + | { kind: 'ok'; data: string } + | { kind: 'not-file' } + | { kind: 'oversize'; size: number } + | { kind: 'grew' } + | { kind: 'open-error'; message: string } + function assertNotAborted(signal: AbortSignal | undefined, operation: string): void { if (signal?.aborted === true) throw new FsError(`${operation} aborted`, 'FS_ABORTED') } @@ -141,6 +150,21 @@ export class E2BFileSystem extends FileSystem { } } + override processPath(target: FsTarget): string { + return String(target.targetKey) + } + + override fileUrl(target: FsTarget): string { + const path = this.processPath(target) + if (!posix.isAbsolute(path)) throw new Error(`fs-e2b: expected an absolute process path: ${JSON.stringify(path)}`) + return `file://${path.split('/').map(segment => encodeURIComponent(segment)).join('/')}` + } + + override contains(parent: FsTarget, child: FsTarget): boolean { + const relative = posix.relative(this.processPath(parent), this.processPath(child)) + return relative === '' || (relative !== '..' && !relative.startsWith('../') && !posix.isAbsolute(relative)) + } + override async stat(target: FsTarget, signal?: AbortSignal): Promise { assertNotAborted(signal, 'stat') const entry = await this.probe(String(target.targetKey), target.displayPath, signal) @@ -184,6 +208,69 @@ export class E2BFileSystem extends FileSystem { } } + override async readTextBounded(target: FsTarget, maxBytes: number, signal?: AbortSignal): Promise { + if (!Number.isSafeInteger(maxBytes) || maxBytes <= 0) { + throw new Error('bounded read maxBytes must be a positive safe integer') + } + assertNotAborted(signal, 'read') + const sandbox = await this.ctx.e2b.getSandbox() + try { + const node = await sandbox.commands.run('command -v -- node', signalOpts(signal)) + const executable = node.stdout.trim() + if (!posix.isAbsolute(executable) || executable.includes('\n')) { + throw new Error('fs-e2b: bounded reader requires one absolute Node executable') + } + const command = [ + quoteE2BShellArg(executable), + '--input-type=commonjs', + '-e', + quoteE2BShellArg(BOUNDED_READER_SOURCE), + quoteE2BShellArg(this.processPath(target)), + String(maxBytes), + ].join(' ') + const result = await sandbox.commands.run(command, signalOpts(signal)) + assertNotAborted(signal, 'read') + const response = this.parseBoundedRead(result.stdout, target) + if (response.kind === 'not-file') { + throw new FsError(`cannot read "${target.displayPath}": not a regular file`, 'FS_NOT_REGULAR_FILE') + } + if (response.kind === 'oversize' && Number.isSafeInteger(response.size)) { + throw new FsError( + `cannot read "${target.displayPath}": ${response.size} bytes exceeds the ${maxBytes}-byte limit`, + 'FS_IO_ERROR', + ) + } + if (response.kind === 'grew') { + throw new FsError( + `cannot read "${target.displayPath}": file grew past the ${maxBytes}-byte limit while reading`, + 'FS_IO_ERROR', + ) + } + if (response.kind === 'open-error' && typeof response.message === 'string') { + if (/ENOENT|no such file/i.test(response.message)) { + throw new FsError(`cannot read "${target.displayPath}": not found`, 'FS_NOT_FOUND') + } + if (/EACCES|EPERM|permission denied|operation not permitted/i.test(response.message)) { + throw new FsError(`cannot read "${target.displayPath}": permission denied`, 'FS_PERMISSION_DENIED') + } + throw new FsError( + `cannot read "${target.displayPath}" safely: ${response.message}`, + 'FS_IO_ERROR', + ) + } + if (response.kind !== 'ok' || typeof response.data !== 'string') { + throw new FsError(`cannot read "${target.displayPath}": bounded reader returned an invalid response`, 'FS_IO_ERROR') + } + const bytes = Buffer.from(response.data, 'base64') + if (bytes.toString('base64') !== response.data || bytes.length > maxBytes) { + throw new FsError(`cannot read "${target.displayPath}": bounded reader returned invalid bytes`, 'FS_IO_ERROR') + } + return decodeText(bytes, target.displayPath, BINARY_SAMPLE_BYTES) + } catch (error: unknown) { + throw mapError(error, 'read', target.displayPath, signal) + } + } + override async streamText(target: FsTarget, signal?: AbortSignal): Promise> { const sandbox = await this.ctx.e2b.getSandbox() await this.requireRegular(target, signal) @@ -336,6 +423,18 @@ export class E2BFileSystem extends FileSystem { } } + private parseBoundedRead(stdout: string, target: FsTarget): BoundedReadResponse { + try { + return JSON.parse(stdout) as BoundedReadResponse + } catch (error: unknown) { + throw new FsError( + `cannot read "${target.displayPath}": bounded reader returned invalid JSON`, + 'FS_IO_ERROR', + { cause: error }, + ) + } + } + private async probe(path: string, displayPath: string, signal?: AbortSignal): Promise { assertNotAborted(signal, 'stat') try { diff --git a/packages/e2b/fs-e2b/src/source-reader.ts b/packages/e2b/fs-e2b/src/source-reader.ts new file mode 100644 index 0000000000..9cb522ecbb --- /dev/null +++ b/packages/e2b/fs-e2b/src/source-reader.ts @@ -0,0 +1,63 @@ +/** Dependency-free stable-handle bounded reader installed inside E2B. */ +export const BOUNDED_READER_SOURCE = String.raw` +/* dsh-e2b-bounded-reader */ +const fs = require('node:fs') +const target = process.argv[1] +const maxBytes = Number(process.argv[2]) +const directoryFlags = fs.constants.O_RDONLY | fs.constants.O_DIRECTORY | fs.constants.O_NOFOLLOW | fs.constants.O_NONBLOCK +const fileFlags = fs.constants.O_RDONLY | fs.constants.O_NOFOLLOW | fs.constants.O_NONBLOCK +let directory +let descriptor +let response + +const openChild = (parent, component, flags) => fs.openSync('/proc/self/fd/' + parent + '/' + component, flags) +const invalidComponent = component => component === '' || component === '.' || component === '..' + +try { + if (typeof target !== 'string' || !target.startsWith('/') || !Number.isSafeInteger(maxBytes) || maxBytes <= 0) { + throw new Error('bounded reader requires an absolute target and positive byte limit') + } + const components = target === '/' ? [] : target.slice(1).split('/') + if (components.length === 0 || components.some(invalidComponent)) { + throw new Error('bounded reader received a non-canonical file path') + } + + directory = fs.openSync('/', directoryFlags) + for (const component of components.slice(0, -1)) { + const child = openChild(directory, component, directoryFlags) + fs.closeSync(directory) + directory = child + } + descriptor = openChild(directory, components.at(-1), fileFlags) + + const info = fs.fstatSync(descriptor) + if (!info.isFile()) response = { kind: 'not-file' } + else if (info.size > maxBytes) response = { kind: 'oversize', size: info.size } + else { + const chunks = [] + let total = 0 + while (total <= maxBytes) { + const chunk = Buffer.allocUnsafe(Math.min(65536, maxBytes - total + 1)) + const bytesRead = fs.readSync(descriptor, chunk, 0, chunk.length, null) + if (bytesRead === 0) break + chunks.push(chunk.subarray(0, bytesRead)) + total += bytesRead + } + response = total > maxBytes + ? { kind: 'grew' } + : { kind: 'ok', data: Buffer.concat(chunks, total).toString('base64') } + } +} catch (error) { + response = { kind: 'open-error', message: error instanceof Error ? error.message : String(error) } +} finally { + for (const openDescriptor of [descriptor, directory]) { + if (openDescriptor === undefined) continue + try { + fs.closeSync(openDescriptor) + } catch (error) { + response = { kind: 'open-error', message: error instanceof Error ? error.message : String(error) } + } + } +} +process.stdout.write(JSON.stringify(response)) +` diff --git a/packages/e2b/fs-e2b/tests/filesystem.spec.ts b/packages/e2b/fs-e2b/tests/filesystem.spec.ts index 13d94e70c7..648631c7ee 100644 --- a/packages/e2b/fs-e2b/tests/filesystem.spec.ts +++ b/packages/e2b/fs-e2b/tests/filesystem.spec.ts @@ -1,3 +1,4 @@ +import { Buffer } from 'node:buffer' import { dirname, posix } from 'node:path' import { Context } from 'cordis' import { @@ -8,7 +9,7 @@ import { type Sandbox, } from '@deepseek-ai/dsh-e2b' import type E2BSandboxService from '@deepseek-ai/dsh-e2b' -import { FsVersion } from '@deepseek-ai/dsh-fs' +import { FsTargetKey, FsVersion } from '@deepseek-ai/dsh-fs' import E2BFileSystem from '@deepseek-ai/dsh-fs-e2b' import * as E2BFsInvariant from '../src/invariant.ts' import InvariantService from '@deepseek-ai/dsh-invariants' @@ -46,6 +47,10 @@ class FakeRemote { nextReadError: unknown nextRenameError: unknown nextRemoveError: unknown + boundedOutput: string | undefined + boundedError: unknown + nodeExecutable = '/usr/bin/node\n' + abortAfterBoundedCommand: AbortController | undefined abortAfterRename: AbortController | undefined disappearOnInfo = new Set() private clock = 1 @@ -221,6 +226,14 @@ class FakeRemote { const node = this.nodes.get(input) return { exitCode: 0, stdout: `${node?.symlinkTarget ?? input}\n`, stderr: '' } } + if (command === 'command -v -- node') { + return { exitCode: 0, stdout: this.nodeExecutable, stderr: '' } + } + if (command.includes('dsh-e2b-bounded-reader')) { + if (this.boundedError !== undefined) throw this.boundedError + this.abortAfterBoundedCommand?.abort('after bounded read') + return { exitCode: 0, stdout: this.boundedOutput ?? '{"kind":"ok","data":""}', stderr: '' } + } const chmod = /^chmod ([0-7]+) -- '([^']+)'$/.exec(command) if (chmod !== null) this.required(chmod[2]!).mode = Number.parseInt(chmod[1]!, 8) const move = /^mv -f -- '([^']+)' '([^']+)'$/.exec(command) @@ -289,6 +302,26 @@ describe('E2BFileSystem identity, metadata, and reads', () => { expect(listed.some(entry => entry.name === 'nested.txt')).toBe(false) }) + it('projects canonical process paths, file URLs, and containment', async () => { + const remote = new FakeRemote() + remote.dir('/workspace/nested') + remote.file('/workspace/nested/multibyte # file.ts', 'text') + remote.file('/outside.ts', 'outside') + const { fs } = await setup(remote) + const workspace = await fs.resolve('/workspace') + const nested = await fs.resolve('/workspace/nested/multibyte # file.ts') + const outside = await fs.resolve('/outside.ts') + + expect(fs.processPath(nested)).toBe('/workspace/nested/multibyte # file.ts') + expect(fs.fileUrl(nested)).toBe('file:///workspace/nested/multibyte%20%23%20file.ts') + expect(fs.contains(workspace, workspace)).toBe(true) + expect(fs.contains(workspace, nested)).toBe(true) + expect(fs.contains(nested, workspace)).toBe(false) + expect(fs.contains(workspace, outside)).toBe(false) + expect(() => fs.fileUrl({ targetKey: FsTargetKey('relative'), displayPath: 'relative' })) + .toThrow('expected an absolute process path') + }) + it('reads whole and streamed UTF-8 across chunk boundaries', async () => { const remote = new FakeRemote() remote.file('/workspace/text.txt', 'A€B') @@ -373,6 +406,78 @@ describe('E2BFileSystem identity, metadata, and reads', () => { await expectCode(fs.readText(await fs.resolve('a')), 'FS_ABORTED') }) + it('performs stable bounded reads through the remote no-follow reader', async () => { + const remote = new FakeRemote() + remote.file('/workspace/a', 'unused') + const { fs } = await setup(remote) + const target = await fs.resolve('a') + remote.boundedOutput = JSON.stringify({ kind: 'ok', data: Buffer.from('hello 你好').toString('base64') }) + await expect(fs.readTextBounded(target, 64)).resolves.toBe('hello 你好') + expect(remote.commands.some(command => command.includes('dsh-e2b-bounded-reader'))).toBe(true) + + await expect(fs.readTextBounded(target, 0)).rejects.toThrow('positive safe integer') + await expect(fs.readTextBounded(target, 1.5)).rejects.toThrow('positive safe integer') + await expect(fs.readTextBounded(target, 64, AbortSignal.abort())).rejects.toMatchObject({ code: 'FS_ABORTED' }) + }) + + it('maps bounded-reader file, size, and open failures', async () => { + const remote = new FakeRemote() + remote.file('/workspace/a', 'unused') + const { fs } = await setup(remote) + const target = await fs.resolve('a') + const cases: Array<[unknown, string]> = [ + [{ kind: 'not-file' }, 'FS_NOT_REGULAR_FILE'], + [{ kind: 'oversize', size: 65 }, 'FS_IO_ERROR'], + [{ kind: 'grew' }, 'FS_IO_ERROR'], + [{ kind: 'open-error', message: 'ENOENT: no such file' }, 'FS_NOT_FOUND'], + [{ kind: 'open-error', message: 'EACCES: permission denied' }, 'FS_PERMISSION_DENIED'], + [{ kind: 'open-error', message: 'ELOOP: symbolic link' }, 'FS_IO_ERROR'], + [{ kind: 'oversize', size: 'large' }, 'FS_IO_ERROR'], + [{ kind: 'open-error', message: 7 }, 'FS_IO_ERROR'], + [{ kind: 'unknown' }, 'FS_IO_ERROR'], + ] + for (const [response, code] of cases) { + remote.boundedOutput = JSON.stringify(response) + await expectCode(fs.readTextBounded(target, 64), code) + } + }) + + it('rejects malformed bounded-reader transports and bytes', async () => { + const remote = new FakeRemote() + remote.file('/workspace/a', 'unused') + const { fs } = await setup(remote) + const target = await fs.resolve('a') + + remote.boundedOutput = 'not-json' + await expectCode(fs.readTextBounded(target, 4), 'FS_IO_ERROR') + remote.boundedOutput = JSON.stringify({ kind: 'ok', data: '!!!!' }) + await expectCode(fs.readTextBounded(target, 4), 'FS_IO_ERROR') + remote.boundedOutput = JSON.stringify({ kind: 'ok', data: Buffer.from('12345').toString('base64') }) + await expectCode(fs.readTextBounded(target, 4), 'FS_IO_ERROR') + remote.boundedOutput = JSON.stringify({ kind: 'ok', data: Buffer.from([0]).toString('base64') }) + await expectCode(fs.readTextBounded(target, 4), 'FS_NOT_TEXT') + remote.boundedOutput = JSON.stringify({ kind: 'ok', data: Buffer.from([0xff]).toString('base64') }) + await expectCode(fs.readTextBounded(target, 4), 'FS_NOT_TEXT') + + remote.nodeExecutable = 'node\n' + await expectCode(fs.readTextBounded(target, 4), 'FS_IO_ERROR') + remote.nodeExecutable = '/usr/bin/node\n/other\n' + await expectCode(fs.readTextBounded(target, 4), 'FS_IO_ERROR') + remote.nodeExecutable = '/usr/bin/node\n' + remote.boundedError = new Error('reader transport failed') + await expectCode(fs.readTextBounded(target, 4), 'FS_IO_ERROR') + }) + + it('does not turn a post-read abort into successful source text', async () => { + const remote = new FakeRemote() + remote.file('/workspace/a', 'unused') + const controller = new AbortController() + remote.abortAfterBoundedCommand = controller + remote.boundedOutput = JSON.stringify({ kind: 'ok', data: Buffer.from('text').toString('base64') }) + const { fs } = await setup(remote) + await expectCode(fs.readTextBounded(await fs.resolve('a'), 4, controller.signal), 'FS_ABORTED') + }) + it('rejects empty paths and directory-listing type errors', async () => { const remote = new FakeRemote() remote.file('/workspace/file', 'x') diff --git a/packages/e2b/pty-e2b/README.i18n.yaml b/packages/e2b/pty-e2b/README.i18n.yaml deleted file mode 100644 index 580f874e62..0000000000 --- a/packages/e2b/pty-e2b/README.i18n.yaml +++ /dev/null @@ -1,6 +0,0 @@ -# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each -# side as of the last confirmed-consistent state. Both languages carry equal authority; -# after editing either side, bring the other along and re-record with: -# pnpm run verify-translation-pairing --write packages/e2b/pty-e2b/README.md -README.md: d1b1731cd7b65577a18814a90363acdc66d7d262 -README.zh.md: 4aa2cc8d661e61e80ef61693422a7b6f73c2cb67 diff --git a/packages/e2b/pty-e2b/README.md b/packages/e2b/pty-e2b/README.md deleted file mode 100644 index d1b1731cd7..0000000000 --- a/packages/e2b/pty-e2b/README.md +++ /dev/null @@ -1,55 +0,0 @@ -# @deepseek-ai/dsh-pty-e2b - -English | [中文](README.zh.md) - -E2B byte-PTY backend for [`ctx.pty`](../../pty/pty/README.md). It creates persistent interactive shells inside the shared `ctx.e2b` sandbox while the PTY registry keeps session identity, exact-Agent ownership, and cleanup policy on the host. - -## Plugin and configuration - -The `pty-e2b` plugin injects `e2b` and `pty`, then registers one backend under `backendType`. - -| Key | Default | Meaning | -|---|---|---| -| `backendType` | `shell` | Registry type selected by `terminal_open`. | -| `rows` / `cols` | `40` / `160` | Initial remote PTY size. | -| `scrollbackLines` | `10000` | Maximum retained logical lines. | -| `scrollbackMaxBytes` | `4194304` | Maximum retained UTF-8 scrollback bytes. | -| `maxReadBytes` | `262144` | Maximum bytes returned by one read or settled send. | -| `pollIntervalMs` | `50` | Host readiness-poll interval. | -| `idleSilenceMs` | `3000` | Output silence that yields `inferred_idle`. | -| `timeoutMs` | `30000` | Absolute startup and send wait bound. | -| `disposeGraceMs` | `3000` | TERM-to-KILL cleanup grace. | - -Numeric values are positive safe integers, `backendType` is non-empty, and `maxReadBytes` cannot exceed `scrollbackMaxBytes`. A relative spawn cwd resolves against `ctx.e2b.cwd`; an absolute remote path remains absolute. Before launch, the backend enumerates sandbox-default environment names, blanks `DSH_*` and credential-shaped names, then overlays its controlled terminal values and explicit `spec.env` entries. - -## Runtime contract - -The backend uses E2B's byte-oriented PTY callback with a streaming fatal UTF-8 decoder, then the backend-neutral line sanitizer and bounded buffers from `dsh-pty`. It installs a controlled Bash prompt marker and waits for printable prompt text; when that marker is unavailable, observed output plus the configured silence bound yields `inferred_idle`. Startup with no output reaches the absolute timeout and fails instead of publishing an empty session. - -Each send writes UTF-8 bytes and an optional carriage-return submit sequence. Cancellation and explicit signals resolve the remote terminal's foreground process group through `ps`, then signal that group; cancellation rechecks the originating send after lookup so a settled operation cannot signal or fail its successor, and `SIGKILL` refuses to target the shell itself. The backend records the terminal's POSIX session id at startup. Close sends `SIGTERM` to every process group still in that session, escalates survivors to `SIGKILL`, verifies that the session is empty, and does not resolve until the SDK handle reports exit. A startup failure closes the unpublished PTY, and `PtyBackendCleanupError` preserves a concurrent cleanup failure. - -The remote PTY process and its child processes live in E2B. Prompt/readiness state, scrollback, operation handles, owner authority, and SDK event delivery remain in host memory. - -## Model Experience - -### Indirect consumer - -#### What the model sees - -Nothing directly. Through `@deepseek-ai/dsh-tool-pty`, the model may receive bounded MOTD, send deltas, scrollback pages, readiness reasons, signal results, and cleanup failures. - -#### Token effect - -None until a consumer returns bounded backend output. Retained host PTY scrollback is not placed in model history by this package. - -#### KV Cache effect - -No direct invalidation; the consumer owns prompts, schemas, and appended results. - -## Known Limitations and Deferred Work - -- **Line-oriented terminal model** — CSI/OSC control sequences are removed; alternate-screen and full terminal emulation remain unsupported. -- **Readiness is marker-or-silence based** — E2B exposes foreground process groups but not the local backend's Linux syscall inspection, so `inferred_idle` is deliberately possible. -- **UTF-8 only** — invalid byte sequences fail the session instead of returning lossy text. -- **Deliberate session escape is unmanaged** — a process that calls `setsid` leaves the terminal session and is outside this backend's cleanup identity. -- **No reconnectable terminal handles** — retaining an E2B sandbox preserves remote files, not host ownership, buffers, callbacks, or live PTY sessions. diff --git a/packages/e2b/pty-e2b/README.zh.md b/packages/e2b/pty-e2b/README.zh.md deleted file mode 100644 index 4aa2cc8d66..0000000000 --- a/packages/e2b/pty-e2b/README.zh.md +++ /dev/null @@ -1,55 +0,0 @@ -# @deepseek-ai/dsh-pty-e2b - -[English](README.md) | 中文 - -用于 [`ctx.pty`](../../pty/pty/README.md) 的 E2B 字节 PTY 后端。它在共享的 `ctx.e2b` 沙箱内创建持久交互式 shell;PTY 注册表则在宿主侧维护会话身份、精确的 Agent 所有权和清理策略。 - -## 插件与配置 - -`pty-e2b` 插件注入 `e2b` 和 `pty`,然后以 `backendType` 注册一个后端。 - -| 配置键 | 默认值 | 含义 | -|---|---|---| -| `backendType` | `shell` | `terminal_open` 选择的注册表类型。 | -| `rows` / `cols` | `40` / `160` | 远程 PTY 的初始尺寸。 | -| `scrollbackLines` | `10000` | 保留的逻辑行数上限。 | -| `scrollbackMaxBytes` | `4194304` | 保留的 UTF-8 scrollback 字节数上限。 | -| `maxReadBytes` | `262144` | 单次读取或发送结算时返回的字节数上限。 | -| `pollIntervalMs` | `50` | 宿主就绪轮询间隔。 | -| `idleSilenceMs` | `3000` | 触发 `inferred_idle` 的输出静默时长。 | -| `timeoutMs` | `30000` | 启动与发送等待的绝对上限。 | -| `disposeGraceMs` | `3000` | TERM 到 KILL 的清理宽限期。 | - -数值必须是正的安全整数,`backendType` 必须非空,且 `maxReadBytes` 不得超过 `scrollbackMaxBytes`。相对的 spawn cwd 以 `ctx.e2b.cwd` 为基准解析;绝对远程路径保持不变。启动前,后端会枚举沙箱默认环境变量名,清空 `DSH_*` 和形似凭据的名称,再覆盖其受控终端值与显式 `spec.env` 条目。 - -## 运行时契约 - -该后端为 E2B 面向字节的 PTY 回调配备流式、遇到无效序列即失败的 UTF-8 解码器,随后使用 `dsh-pty` 提供的后端无关行清理器与有界缓冲区。它会安装受控的 Bash 提示符标记,并等待可打印的提示符文本;若该标记不可用,系统会在已经观察到输出且达到已配置的静默上限时得出 `inferred_idle`。零输出的启动过程会达到绝对超时并失败,不会发布空会话。 - -每次发送都会写入 UTF-8 字节,并可选写入回车提交序列。取消与显式信号会通过 `ps` 确定远程终端的前台进程组,再向该组发送信号;取消处理会在查找后重新检查原发送操作是否仍为当前操作,以免已结算的操作向后继操作发送信号或令其失败;发送 `SIGKILL` 时拒绝以 shell 本身为目标。后端会在启动时记录终端的 POSIX 会话 id。关闭操作会向该会话内仍存在的每个进程组发送 `SIGTERM`,对存活者升级为 `SIGKILL`,验证会话已经清空,并且直到 SDK 句柄报告退出才结算。如果启动失败,系统会关闭尚未发布的 PTY;若清理同时失败,`PtyBackendCleanupError` 会保留这项失败。 - -远程 PTY 进程及其子进程位于 E2B。提示符/就绪状态、scrollback、操作句柄、所有者权限和 SDK 事件交付仍保留在宿主内存中。 - -## 模型体验 - -### 间接消费方 - -#### 模型看到的内容 - -没有直接可见内容。模型通过 `@deepseek-ai/dsh-tool-pty` 可能收到有界的 MOTD、发送增量、scrollback 页、就绪原因、信号结果和清理失败。 - -#### Token 影响 - -消费方返回有界的后端输出前没有影响。本包不会把宿主保留的 PTY scrollback 放入模型历史。 - -#### KV Cache 影响 - -不会直接失效;提示词、schema 和追加结果由消费方负责。 - -## 已知限制与暂缓工作 - -- **面向行的终端模型**:CSI/OSC 控制序列会被移除;备用屏幕与完整终端仿真仍不受支持。 -- **就绪判断基于标记或静默**:E2B 会公开前台进程组,但不提供本地后端使用的 Linux syscall 检查,因此系统有意保留返回 `inferred_idle` 的可能性。 -- **仅支持 UTF-8**:无效字节序列会使会话失败,而不是返回有损文本。 -- **主动逃离会话的进程不受管理**:调用 `setsid` 的进程会离开终端会话,因而不属于本后端的清理身份。 -- **没有可重连的终端句柄**:保留 E2B 沙箱会保留远程文件,但不会保留宿主所有权、缓冲区、回调或实时 PTY 会话。 diff --git a/packages/e2b/pty-e2b/package.json b/packages/e2b/pty-e2b/package.json deleted file mode 100644 index cc31699ab1..0000000000 --- a/packages/e2b/pty-e2b/package.json +++ /dev/null @@ -1,46 +0,0 @@ -{ - "name": "@deepseek-ai/dsh-pty-e2b", - "description": "E2B PTY provider for DeepSeek Harness", - "version": "0.0.1", - "private": true, - "type": "module", - "main": "lib/index.js", - "types": "lib/types/index.d.ts", - "exports": { - ".": { - "types": "./lib/types/index.d.ts", - "default": "./lib/index.js" - }, - "./invariant": { - "types": "./lib/types/invariant.d.ts", - "default": "./lib/invariant.js" - }, - "./src/*": "./src/*", - "./package.json": "./package.json" - }, - "files": [ - "lib/index.js", - "lib/invariant.js", - "lib/types/**/*.d.ts", - "lib/types/**/*.d.ts.map", - "src" - ], - "license": "BSD-3-Clause", - "peerDependencies": { - "@deepseek-ai/dsh-e2b": "^0.0.1", - "@deepseek-ai/dsh-invariants": "^0.0.1", - "@deepseek-ai/dsh-pty": "^0.0.1", - "cordis": "^4.0.0-rc.7" - }, - "dependencies": { - "schemastery": "^3.18.0" - }, - "devDependencies": { - "@deepseek-ai/dsh-agent": "workspace:^", - "@deepseek-ai/dsh-e2b": "workspace:^", - "@deepseek-ai/dsh-invariants": "workspace:^", - "@deepseek-ai/dsh-pty": "workspace:^", - "@deepseek-ai/dsh-session": "workspace:^", - "cordis": "^4.0.0-rc.7" - } -} diff --git a/packages/e2b/pty-e2b/src/config.ts b/packages/e2b/pty-e2b/src/config.ts deleted file mode 100644 index e6849bf54d..0000000000 --- a/packages/e2b/pty-e2b/src/config.ts +++ /dev/null @@ -1,64 +0,0 @@ -/** Validated configuration for the E2B PTY backend. */ - -import z from 'schemastery' - -/** Public plugin configuration. */ -export interface Config { - /** Backend registry type. */ - backendType?: string - /** Initial terminal rows. */ - rows?: number - /** Initial terminal columns. */ - cols?: number - /** Maximum retained logical lines. */ - scrollbackLines?: number - /** Maximum retained UTF-8 bytes. */ - scrollbackMaxBytes?: number - /** Maximum bytes returned by one read or settled viewport. */ - maxReadBytes?: number - /** Readiness polling interval. */ - pollIntervalMs?: number - /** Output silence duration that yields `inferred_idle`. */ - idleSilenceMs?: number - /** Absolute send and startup wait bound. */ - timeoutMs?: number - /** Grace before PTY teardown escalates from TERM to KILL. */ - disposeGraceMs?: number -} - -/** Configuration after Schemastery defaults. */ -export type ResolvedConfig = Required - -/* jscpd:ignore-start -- Loader requires a backend-local schema and load-time diagnostics. */ -/** Schemastery config exposed by the plugin. */ -export const Config: z = z.object({ - backendType: z.string().default('shell'), - rows: z.number().default(40), - cols: z.number().default(160), - scrollbackLines: z.number().default(10_000), - scrollbackMaxBytes: z.number().default(4 * 1024 * 1024), - maxReadBytes: z.number().default(256 * 1024), - pollIntervalMs: z.number().default(50), - idleSilenceMs: z.number().default(3_000), - timeoutMs: z.number().default(30_000), - disposeGraceMs: z.number().default(3_000), -}) - -/** - * Validate the resolved configuration before publishing the backend. - * @param config - Schemastery-resolved plugin configuration. - * @returns Nothing; success narrows every optional field to its resolved value. - */ -export function validateConfig(config: Config): asserts config is ResolvedConfig { - const resolved = config as ResolvedConfig - if (resolved.backendType.length === 0) throw new Error('pty-e2b: backendType must be non-empty') - for (const [name, value] of Object.entries(resolved)) { - if (typeof value === 'number' && (!Number.isSafeInteger(value) || value <= 0)) { - throw new Error(`pty-e2b: ${name} must be a positive safe integer`) - } - } - if (resolved.maxReadBytes > resolved.scrollbackMaxBytes) { - throw new Error('pty-e2b: maxReadBytes must not exceed scrollbackMaxBytes') - } -} -/* jscpd:ignore-end */ diff --git a/packages/e2b/pty-e2b/src/index.ts b/packages/e2b/pty-e2b/src/index.ts deleted file mode 100644 index d665a734b7..0000000000 --- a/packages/e2b/pty-e2b/src/index.ts +++ /dev/null @@ -1,130 +0,0 @@ -/** E2B byte-PTY backend for persistent interactive terminal sessions. */ - -import { posix } from 'node:path' -import type { Context } from 'cordis' -import type { CommandHandle, Sandbox } from '@deepseek-ai/dsh-e2b' -import { PtyBackendCleanupError } from '@deepseek-ai/dsh-pty' -import type { PtyBackend, PtyBackendSpawnSpec } from '@deepseek-ai/dsh-pty' -import { type Config, type ResolvedConfig, validateConfig } from './config.ts' -import { E2BPtySession } from './session.ts' - -export { Config } from './config.ts' -export type { Config as PtyE2BConfig } from './config.ts' -export { E2BPtySession } from './session.ts' - -/** Cordis plugin name. */ -export const name = 'pty-e2b' -/** Required shared sandbox owner and PTY registry. */ -export const inject = ['e2b', 'pty'] - -const SENSITIVE_ENV_NAME = /KEY|SECRET|TOKEN/i - -async function terminalEnvironment( - sandbox: Sandbox, - spec: PtyBackendSpawnSpec, -): Promise> { - const discovered = await sandbox.commands.run( - 'env -0 | cut -z -d= -f1', - spec.signal === undefined ? {} : { signal: spec.signal }, - ) - spec.signal?.throwIfAborted() - const scrubbed = Object.fromEntries(discovered.stdout.split('\0') - .filter(name => name.startsWith('DSH_') || SENSITIVE_ENV_NAME.test(name)) - .map(name => [name, ''])) - return { - ...scrubbed, - TERM: 'dumb', - PAGER: 'cat', - GIT_PAGER: 'cat', - PS1: 'dsh> ', - PROMPT_COMMAND: 'printf "\\033]133;D;%s\\007" "$?"', - BASH_SILENCE_DEPRECATION_WARNING: '1', - DSH_SHELL: '1', - DSH_SESSION_ID: spec.owner.id, - DSH_PTY_SESSION_ID: spec.sessionId, - } -} - -async function resolveTerminalSessionId(sandbox: Sandbox, pid: number, signal?: AbortSignal): Promise { - const result = await sandbox.commands.run( - `ps -o sid= -p ${pid}`, - signal === undefined ? {} : { signal }, - ) - signal?.throwIfAborted() - const raw = result.stdout.trim() - const sessionId = Number(raw) - if (!/^[1-9][0-9]*$/.test(raw) || !Number.isSafeInteger(sessionId)) { - throw new Error(`pty-e2b: cannot resolve process session for E2B PTY ${pid}`) - } - return sessionId -} - -/** E2B backend registered under the configured terminal type. */ -export class E2BPtyBackend implements PtyBackend { - readonly type: string - - constructor( - private readonly ctx: Context, - private readonly config: ResolvedConfig, - private readonly createPty: ( - sandbox: Sandbox, - options: Parameters[0], - ) => Promise = (sandbox, options) => sandbox.pty.create(options), - ) { - this.type = config.backendType - } - - /** Create, initialize, and publish one remote PTY session. */ - async spawn(spec: PtyBackendSpawnSpec): Promise { - spec.signal?.throwIfAborted() - const sandbox = await this.ctx.e2b.getSandbox() - spec.signal?.throwIfAborted() - const pending: Uint8Array[] = [] - const created: { session?: E2BPtySession } = {} - const handle = await this.createPty(sandbox, { - rows: this.config.rows, - cols: this.config.cols, - cwd: posix.resolve(this.ctx.e2b.cwd, spec.cwd ?? this.ctx.e2b.cwd), - envs: await terminalEnvironment(sandbox, spec), - timeoutMs: 0, - ...spec.signal === undefined ? {} : { signal: spec.signal }, - onData: (data) => { - if (created.session === undefined) pending.push(Uint8Array.from(data)) - else created.session.onData(data) - }, - }) - if (!Number.isSafeInteger(handle.pid) || handle.pid <= 0) { - await handle.kill().catch(() => false) - throw new Error(`pty-e2b: E2B returned invalid PTY pid ${handle.pid}`) - } - let terminalSessionId: number - try { - terminalSessionId = await resolveTerminalSessionId(sandbox, handle.pid, spec.signal) - } catch (error: unknown) { - await handle.kill().catch(() => false) - await Promise.allSettled([handle.wait()]) - throw error - } - const session = new E2BPtySession(sandbox, handle, terminalSessionId, this.config) - created.session = session - try { - const initializing = session.initialize(spec.signal) - for (const data of pending) session.onData(data) - await initializing - return session - } catch (error: unknown) { - try { - await session.close('E2B PTY startup failed') - } catch (cleanupError: unknown) { - throw new PtyBackendCleanupError(error, cleanupError) - } - throw error - } - } -} - -/** Register the E2B PTY backend. */ -export function apply(ctx: Context, config: Config): void { - validateConfig(config) - ctx.pty.registerBackend(new E2BPtyBackend(ctx, config)) -} diff --git a/packages/e2b/pty-e2b/src/invariant.ts b/packages/e2b/pty-e2b/src/invariant.ts deleted file mode 100644 index b86fdffc5d..0000000000 --- a/packages/e2b/pty-e2b/src/invariant.ts +++ /dev/null @@ -1,20 +0,0 @@ -/** Package-owned invariant companion for `@deepseek-ai/dsh-pty-e2b`. */ - -/* jscpd:ignore-start */ -import type { Context } from 'cordis' -import type { InvariantInstaller } from '@deepseek-ai/dsh-invariants' - -const PACKAGE_NAME = '@deepseek-ai/dsh-pty-e2b' - -/** Cordis companion plugin name. */ -export const name = 'pty-e2b-invariant' -/** Service required before the companion can reserve package ownership. */ -export const inject = ['invariants'] - -/** No runtime invariant: the PTY registry owns publication and cleanup. */ -const install: InvariantInstaller = () => {} - -/** Register this package's invariant companion. */ -export const apply = (ctx: Context): Promise<() => void> => - Promise.resolve(ctx.invariants.register(PACKAGE_NAME, install)) -/* jscpd:ignore-end */ diff --git a/packages/e2b/pty-e2b/src/session.ts b/packages/e2b/pty-e2b/src/session.ts deleted file mode 100644 index 7742cd44f0..0000000000 --- a/packages/e2b/pty-e2b/src/session.ts +++ /dev/null @@ -1,415 +0,0 @@ -/** One byte-oriented E2B PTY session projected onto the harness PTY seam. */ - -import { Buffer } from 'node:buffer' -import type { CommandHandle, Sandbox } from '@deepseek-ai/dsh-e2b' -import { CommandExitError } from '@deepseek-ai/dsh-e2b' -import { - PtyTerminalSanitizer, - PtyTextBuffer, - ptySignalName, - ptyUtf8Tail, -} from '@deepseek-ai/dsh-pty' -import type { - PtyBackendSession, - PtyReadRequest, - PtyReadResult, - PtySendOperation, - PtySendRead, - PtySendRequest, - PtySendResult, - PtySessionStatus, - PtySignal, - PtySignalResult, - PtyWaitReason, -} from '@deepseek-ai/dsh-pty' -import type { ResolvedConfig } from './config.ts' - -function delay(ms: number): Promise { - return new Promise(resolve => setTimeout(resolve, ms)) -} - -/* jscpd:ignore-start -- Operation state stays backend-local because process readiness and cleanup identities diverge. */ -class E2BSendOperation implements PtySendOperation { - private readonly output: PtyTextBuffer - private readonly result = Promise.withResolvers() - private finished = false - - constructor( - maxBytes: number, - readonly startedAt: number, - private readonly onCancel: () => void, - ) { - this.output = new PtyTextBuffer(maxBytes) - } - - get done(): Promise { - return this.result.promise - } - - append(text: string): void { - if (!this.finished) this.output.append(text) - } - - settle(waitReason: PtyWaitReason, sessionStatus: PtySessionStatus, inheritedTruncation: boolean): void { - if (this.finished) return - this.finished = true - const read = this.output.snapshot() - this.result.resolve({ - viewport: read.text, - waitReason, - sessionStatus, - truncated: read.truncated || inheritedTruncation, - }) - } - - fail(error: unknown): void { - if (this.finished) return - this.finished = true - this.result.reject(error) - } - - readOutput(): PtySendRead { - return this.output.consume() - } - - cancel(): boolean { - if (this.finished) return false - this.onCancel() - return true - } -} -/* jscpd:ignore-end */ - -/** Live session around one E2B SDK PTY handle. */ -export class E2BPtySession implements PtyBackendSession { - motd = '' - readonly pid: number - private readonly decoder = new TextDecoder('utf-8', { fatal: true }) - private readonly sanitizer: PtyTerminalSanitizer - private readonly scrollback: PtyTextBuffer - private readonly exited = Promise.withResolvers() - private statusValue: PtySessionStatus = { kind: 'running' } - private active: E2BSendOperation | undefined - private activeTimer: NodeJS.Timeout | undefined - private activeAbort: (() => void) | undefined - private promptSeen = false - private promptTextSeen = false - private initializing = false - private lastOutputAt = Date.now() - private closing = false - private closePromise: Promise | undefined - private closeSignal: NodeJS.Signals | null = null - private transportFailure: Error | undefined - private remoteExited = false - - constructor( - private readonly sandbox: Sandbox, - private readonly handle: CommandHandle, - private readonly terminalSessionId: number, - private readonly config: ResolvedConfig, - ) { - this.pid = handle.pid - this.sanitizer = new PtyTerminalSanitizer(config.maxReadBytes) - this.scrollback = new PtyTextBuffer(config.scrollbackMaxBytes, config.scrollbackLines) - const completion = handle.wait() - void completion.then( - (result) => { this.onExit(result.exitCode) }, - (error: unknown) => { - if (error instanceof CommandExitError) this.onExit(error.exitCode) - else this.onTransportFailure(error) - }, - ) - } - - /** - * Consume bytes received by the SDK's PTY callback. - * @param data - Exact callback bytes in delivery order. - */ - onData(data: Uint8Array): void { - let decoded: string - try { - decoded = this.decoder.decode(data, { stream: true }) - } catch (error: unknown) { - this.onTransportFailure(new Error('pty-e2b: PTY emitted invalid UTF-8', { cause: error })) - return - } - const sanitized = this.sanitizer.push(decoded) - this.appendOutput(sanitized.text) - if (sanitized.prompt) { - this.promptSeen = true - this.promptTextSeen = sanitized.promptText === true - this.lastOutputAt = Date.now() - } else if (this.promptSeen && sanitized.promptText === true) { - this.promptTextSeen = true - } - } - - /** - * Await the first prompt or bounded startup fallback. - * @param signal - Optional startup cancellation signal. - */ - async initialize(signal?: AbortSignal): Promise { - this.initializing = true - try { - const operation = this.startSend({ text: '', submit: false, ...signal === undefined ? {} : { signal } }) - const result = await operation.done - if (result.waitReason === 'session_exit') throw new Error('E2B PTY shell exited during startup') - if (result.waitReason === 'timeout') throw new Error('E2B PTY shell did not reach readiness before startup timeout') - this.motd = result.viewport - } catch (error: unknown) { - signal?.throwIfAborted() - throw error - } finally { - this.initializing = false - } - } - - /* jscpd:ignore-start -- PTY backends share request admission while owning distinct input and readiness transports. */ - startSend(request: PtySendRequest): PtySendOperation { - if (this.closing) throw new Error('E2B PTY session is closing') - if (this.statusValue.kind === 'exited') throw new Error('E2B PTY session has exited') - if (this.active !== undefined) throw new Error('E2B PTY session already has an active send') - if (request.signal?.aborted === true) throw new Error('E2B PTY send aborted before write') - - const operation = new E2BSendOperation( - this.config.maxReadBytes, - Date.now(), - () => { this.interrupt(operation) }, - ) - this.active = operation - this.lastOutputAt = Date.now() - this.promptSeen = false - this.promptTextSeen = false - if (request.signal !== undefined) { - const onAbort = (): void => { operation.cancel() } - request.signal.addEventListener('abort', onAbort, { once: true }) - this.activeAbort = () => request.signal?.removeEventListener('abort', onAbort) - } - - const input = `${request.text}${request.submit ? '\r' : ''}` - if (input.length > 0) { - void this.sandbox.pty.sendInput(this.pid, Buffer.from(input)).catch((error: unknown) => { - if (this.active === operation) this.failActive(error) - }) - } - this.activeTimer = setInterval(() => { this.pollReadiness(operation) }, this.config.pollIntervalMs) - return operation - } - /* jscpd:ignore-end */ - - /* jscpd:ignore-start -- The seam requires identical bounded-read coordinates across backend buffers. */ - read(request: PtyReadRequest): PtyReadResult { - const snapshot = this.scrollback.snapshot() - const lines = snapshot.text.split('\n') - const totalLines = snapshot.text.length === 0 ? 0 : lines.length - const offset = request.offset ?? 0 - const count = request.count ?? 500 - if (!Number.isSafeInteger(offset) || offset < 0) throw new Error('PTY read offset must be a non-negative safe integer') - if (!Number.isSafeInteger(count) || count <= 0) throw new Error('PTY read count must be a positive safe integer') - if (offset >= totalLines) { - return { text: '', totalLines, lineBegin: offset, lineEnd: offset, truncated: snapshot.truncated } - } - const end = totalLines - offset - const start = Math.max(0, end - count) - const bounded = ptyUtf8Tail(lines.slice(start, end).join('\n'), this.config.maxReadBytes) - const returnedLines = bounded.text.length === 0 ? 0 : bounded.text.split('\n').length - return { - text: bounded.text, - totalLines, - lineBegin: offset, - lineEnd: offset + returnedLines, - truncated: snapshot.truncated || bounded.truncated, - } - } - /* jscpd:ignore-end */ - - /* jscpd:ignore-start -- Signal, status, and close methods preserve the seam shape around remote identities. */ - async signal(signal: PtySignal): Promise { - const pgid = await this.foregroundPgid() - return await this.deliverSignal(signal, pgid) - } - - private async deliverSignal(signal: PtySignal, pgid: number): Promise { - if (signal === 'SIGKILL' && pgid === this.pid) { - throw new Error('refusing to SIGKILL the E2B PTY shell; use terminal_close') - } - await this.sandbox.commands.run(`kill -${signal.slice(3)} -- -${pgid}`) - return { delivered: true, targetPgid: pgid } - } - - status(): PtySessionStatus { - return this.statusValue - } - - close(reason: string): Promise { - this.closing = true - if (this.closePromise !== undefined) return this.closePromise - const closing = this.closeOnce(reason).catch((error: unknown) => { - this.closePromise = undefined - this.failActive(error) - throw error - }) - this.closePromise = closing - return closing - } - /* jscpd:ignore-end */ - - private appendOutput(text: string): void { - if (text.length === 0) return - this.lastOutputAt = Date.now() - this.scrollback.append(text) - this.active?.append(text) - } - - private pollReadiness(operation: E2BSendOperation): void { - if (this.active !== operation) return - if (this.statusValue.kind === 'exited') { - this.settleActive('session_exit') - return - } - const elapsed = Date.now() - operation.startedAt - const idleFor = Date.now() - this.lastOutputAt - if (this.promptSeen && this.promptTextSeen && idleFor >= this.config.pollIntervalMs) { - this.settleActive('stdin_read') - return - } - const startupHasOutput = !this.initializing || this.scrollback.snapshot().text.length > 0 - if (startupHasOutput && idleFor >= this.config.idleSilenceMs) { - this.settleActive('inferred_idle') - return - } - if (elapsed >= this.config.timeoutMs) this.settleActive('timeout') - } - - private settleActive(waitReason: PtyWaitReason): void { - const operation = this.active - if (operation === undefined) return - const inherited = this.scrollback.snapshot().truncated - this.clearActive() - operation.settle(waitReason, this.statusValue, inherited) - } - - private clearActive(): void { - if (this.activeTimer !== undefined) clearInterval(this.activeTimer) - this.activeTimer = undefined - this.activeAbort?.() - this.activeAbort = undefined - this.active = undefined - } - - private failActive(error: unknown): void { - const operation = this.active - if (operation === undefined) return - this.clearActive() - operation.fail(error) - } - - private interrupt(operation: E2BSendOperation): void { - if (this.active !== operation) return - void this.interruptActive(operation).catch((error: unknown) => { - if (this.active === operation) this.failActive(error) - }) - } - - private async interruptActive(operation: E2BSendOperation): Promise { - const pgid = await this.foregroundPgid() - if (this.active !== operation) return - await this.deliverSignal('SIGINT', pgid) - } - - private async foregroundPgid(): Promise { - const result = await this.sandbox.commands.run(`ps -o tpgid= -p ${this.pid}`) - const raw = result.stdout.trim() - const pgid = Number(raw) - if (!/^[1-9][0-9]*$/.test(raw) || !Number.isSafeInteger(pgid)) { - throw new Error(`cannot resolve foreground process group for E2B PTY ${this.pid}`) - } - return pgid - } - - private async sessionProcessGroups(): Promise { - const result = await this.sandbox.commands.run( - `ps -eo sid=,pgid= | awk '$1 == ${this.terminalSessionId} { print $2 }'`, - ) - const groups = new Set() - for (const raw of result.stdout.trim().split(/\s+/)) { - if (raw.length === 0) continue - const pgid = Number(raw) - if (!/^[1-9][0-9]*$/.test(raw) || !Number.isSafeInteger(pgid) || pgid <= 1) { - throw new Error(`pty-e2b: invalid process group ${JSON.stringify(raw)} in terminal session ${this.terminalSessionId}`) - } - groups.add(pgid) - } - return [...groups] - } - - private async signalProcessGroups(groups: number[], signal: 'TERM' | 'KILL'): Promise { - try { - await this.sandbox.commands.run(`kill -${signal} -- ${groups.map(pgid => `-${pgid}`).join(' ')}`) - } catch (error: unknown) { - if (!(error instanceof CommandExitError)) throw error - } - } - - private async awaitSessionEmpty(timeoutMs: number, signal?: 'KILL'): Promise { - const deadline = Date.now() + timeoutMs - for (;;) { - const groups = await this.sessionProcessGroups() - if (groups.length === 0 || Date.now() >= deadline) return groups - if (signal !== undefined) await this.signalProcessGroups(groups, signal) - await delay(Math.min(this.config.pollIntervalMs, deadline - Date.now())) - } - } - - private onExit(exitCode: number): void { - this.remoteExited = true - let tail = '' - try { - tail = this.decoder.decode() - } catch (error: unknown) { - this.transportFailure ??= new Error('pty-e2b: PTY ended with invalid UTF-8', { cause: error }) - } - this.appendOutput(this.sanitizer.push(tail).text) - this.appendOutput(this.sanitizer.flush()) - const inferredSignal = this.closeSignal ?? (exitCode > 128 ? ptySignalName(exitCode - 128) : null) - this.statusValue = { - kind: 'exited', - exitCode: inferredSignal === null ? exitCode : null, - signal: inferredSignal, - } - if (this.transportFailure === undefined) this.settleActive('session_exit') - else this.failActive(this.transportFailure) - this.exited.resolve() - } - - private onTransportFailure(error: unknown): void { - const failure = error instanceof Error ? error : new Error(String(error)) - this.transportFailure ??= failure - this.statusValue = { kind: 'exited', exitCode: null, signal: null } - this.failActive(failure) - } - - private async closeOnce(reason: string): Promise { - let survivingGroups = await this.sessionProcessGroups() - if (survivingGroups.length > 0) { - this.closeSignal = 'SIGTERM' - await this.signalProcessGroups(survivingGroups, 'TERM') - survivingGroups = await this.awaitSessionEmpty(this.config.disposeGraceMs) - } - if (survivingGroups.length > 0 || !this.remoteExited) { - this.closeSignal = 'SIGKILL' - if (!this.remoteExited) await this.sandbox.pty.kill(this.pid) - survivingGroups = await this.awaitSessionEmpty(this.config.disposeGraceMs, 'KILL') - if (!this.remoteExited) await Promise.race([this.exited.promise, delay(this.config.disposeGraceMs)]) - } - if (survivingGroups.length > 0) { - throw new Error(`E2B PTY cleanup failed (${reason}); surviving process groups: ${survivingGroups.join(', ')}`) - } - if (!this.remoteExited) { - throw new Error(`E2B PTY cleanup failed (${reason}); surviving pid: ${this.pid}`) - } - this.settleActive('session_exit') - await this.handle.disconnect().catch(() => {}) - if (this.transportFailure !== undefined) throw this.transportFailure - } -} diff --git a/packages/e2b/pty-e2b/tests/index.spec.ts b/packages/e2b/pty-e2b/tests/index.spec.ts deleted file mode 100644 index 476ddc38df..0000000000 --- a/packages/e2b/pty-e2b/tests/index.spec.ts +++ /dev/null @@ -1,248 +0,0 @@ -import { Context } from 'cordis' -import { describe, expect, it, vi } from 'vitest' -import type { CommandHandle, Sandbox } from '@deepseek-ai/dsh-e2b' -import type E2BSandboxService from '@deepseek-ai/dsh-e2b' -import PtyService, { PtyBackendCleanupError, PtySessionId } from '@deepseek-ai/dsh-pty' -import { E2BPtyBackend, apply } from '@deepseek-ai/dsh-pty-e2b' -import { validateConfig } from '@deepseek-ai/dsh-pty-e2b/src/config.ts' -import * as E2BPtyInvariant from '../src/invariant.ts' -import InvariantService from '@deepseek-ai/dsh-invariants' -import { AgentMessageId, type Agent } from '@deepseek-ai/dsh-agent' -import { Session, SessionId } from '@deepseek-ai/dsh-session' - -function config() { - return { - backendType: 'shell', rows: 24, cols: 80, - scrollbackLines: 10, scrollbackMaxBytes: 128, maxReadBytes: 64, - pollIntervalMs: 1, idleSilenceMs: 2, timeoutMs: 5, disposeGraceMs: 1, - } -} - -function owner(ctx: Context): Agent { - const id = SessionId('owner') - return { - id, options: {}, session: new Session(id), status: 'idle', acceptsNextStep: false, ctx, - followup: () => AgentMessageId('unused'), steer: () => AgentMessageId('unused'), - inject: () => AgentMessageId('unused'), send: () => AgentMessageId('unused'), - cancel() {}, whenIdle: () => Promise.resolve(), - } -} - -function handle(pid = 123, kill = vi.fn().mockResolvedValue(true)): CommandHandle { - const result = Promise.withResolvers<{ exitCode: number; stdout: string; stderr: string }>() - return { - pid, - wait: () => result.promise, - kill, - disconnect: vi.fn().mockResolvedValue(undefined), - } as unknown as CommandHandle -} - -describe('E2BPtyBackend and plugin', () => { - it('creates a remote PTY with isolated environment and initializes the session', async () => { - vi.useFakeTimers() - const ctx = new Context() - const run = vi.fn(async (command: string) => command.startsWith('env -0') - ? { exitCode: 0, stdout: 'NPM_TOKEN\0DSH_STALE\0KEEP\0', stderr: '' } - : { exitCode: 0, stdout: '123\n', stderr: '' }) - const sandbox = { commands: { run } } as unknown as Sandbox - ctx.provide('e2b', { - cwd: '/workspace', - getSandbox: async () => sandbox, - } as E2BSandboxService) - const created = handle() - let options: Parameters[0] | undefined - const backend = new E2BPtyBackend(ctx, config(), async (_sandbox, received) => { - options = received - void received.onData(Buffer.from('banner\n')) - void received.onData(Buffer.from('\x1b]133;D;0\x07dsh> ')) - return created - }) - const pending = backend.spawn({ - sessionId: PtySessionId('pty-1'), owner: owner(ctx), type: 'shell', cwd: 'project', - signal: new AbortController().signal, - }) - await vi.advanceTimersByTimeAsync(2) - const session = await pending - - expect(session.motd).toBe('banner\ndsh> ') - expect(options).toMatchObject({ rows: 24, cols: 80, cwd: '/workspace/project', timeoutMs: 0 }) - expect(options?.envs).toMatchObject({ - NPM_TOKEN: '', DSH_STALE: '', - TERM: 'dumb', PAGER: 'cat', GIT_PAGER: 'cat', PS1: 'dsh> ', - DSH_SHELL: '1', DSH_SESSION_ID: 'owner', DSH_PTY_SESSION_ID: 'pty-1', - }) - expect(options?.envs).not.toHaveProperty('KEEP') - vi.useRealTimers() - }) - - it('uses the SDK PTY create method and the shared cwd by default', async () => { - vi.useFakeTimers() - const ctx = new Context() - const created = handle() - const create = vi.fn(async (received: Parameters[0]) => { - setTimeout(() => { void received.onData(Buffer.from('\x1b]133;D;0\x07dsh> ')) }, 0) - return created - }) - const sandbox = { - commands: { run: async (command: string) => ({ exitCode: 0, stdout: command.startsWith('env -0') ? '' : '123\n', stderr: '' }) }, - pty: { create }, - } as unknown as Sandbox - ctx.provide('e2b', { cwd: '/workspace', getSandbox: async () => sandbox } as unknown as E2BSandboxService) - const backend = new E2BPtyBackend(ctx, config()) - const pending = backend.spawn({ sessionId: PtySessionId('default'), owner: owner(ctx), type: 'shell' }) - await vi.advanceTimersByTimeAsync(2) - await pending - expect(create).toHaveBeenCalledWith(expect.objectContaining({ cwd: '/workspace' })) - vi.useRealTimers() - }) - - it('rejects aborts and invalid pids, killing a malformed SDK handle', async () => { - const ctx = new Context() - const sandbox = { - commands: { run: async () => ({ exitCode: 0, stdout: '', stderr: '' }) }, - } as unknown as Sandbox - ctx.provide('e2b', { cwd: '/workspace', getSandbox: async () => sandbox } as E2BSandboxService) - const create = vi.fn().mockResolvedValue(handle(0)) - const backend = new E2BPtyBackend(ctx, config(), create) - const aborted = AbortSignal.abort(new Error('stop')) - await expect(backend.spawn({ sessionId: PtySessionId('one'), owner: owner(ctx), type: 'shell', signal: aborted })).rejects.toThrow('stop') - expect(create).not.toHaveBeenCalled() - - const malformedKill = vi.fn().mockResolvedValue(true) - const malformed = handle(0, malformedKill) - const invalid = new E2BPtyBackend(ctx, config(), async () => malformed) - await expect(invalid.spawn({ sessionId: PtySessionId('two'), owner: owner(ctx), type: 'shell' })).rejects.toThrow('invalid PTY pid') - expect(malformedKill).toHaveBeenCalledOnce() - - const killFailureKill = vi.fn().mockRejectedValue(new Error('already gone')) - const killFailure = handle(0, killFailureKill) - const raced = new E2BPtyBackend(ctx, config(), async () => killFailure) - await expect(raced.spawn({ sessionId: PtySessionId('three'), owner: owner(ctx), type: 'shell' })).rejects.toThrow('invalid PTY pid') - - const invalidSessionKill = vi.fn().mockRejectedValue(new Error('kill raced')) - const invalidSessionHandle = { - pid: 123, - wait: vi.fn().mockRejectedValue(new Error('already exited')), - kill: invalidSessionKill, - disconnect: vi.fn(), - } as unknown as CommandHandle - const invalidSessionSandbox = { - commands: { - run: async (command: string) => ({ - exitCode: 0, - stdout: command.startsWith('env -0') ? '' : '9007199254740992\n', - stderr: '', - }), - }, - } as unknown as Sandbox - const invalidSessionContext = new Context() - invalidSessionContext.provide('e2b', { - cwd: '/workspace', - getSandbox: async () => invalidSessionSandbox, - } as E2BSandboxService) - const invalidSession = new E2BPtyBackend(invalidSessionContext, config(), async () => invalidSessionHandle) - await expect(invalidSession.spawn({ - sessionId: PtySessionId('four'), owner: owner(invalidSessionContext), type: 'shell', - })) - .rejects.toThrow('cannot resolve process session') - expect(invalidSessionKill).toHaveBeenCalledOnce() - }) - - it('cleans failed startup and aggregates a cleanup failure', async () => { - vi.useFakeTimers() - const ctx = new Context() - const sandbox = { - commands: { - run: vi.fn(async (command: string) => ({ - exitCode: 0, - stdout: command.startsWith('ps -o sid=') || command.startsWith('ps -eo sid=') ? '123\n' : '', - stderr: '', - })), - }, - pty: { kill: vi.fn().mockRejectedValue(new Error('cleanup failed')) }, - } as unknown as Sandbox - ctx.provide('e2b', { cwd: '/workspace', getSandbox: async () => sandbox } as E2BSandboxService) - const failedHandle = handle() - const backend = new E2BPtyBackend(ctx, config(), async () => failedHandle) - const pending = backend.spawn({ sessionId: PtySessionId('failed'), owner: owner(ctx), type: 'shell' }) - const rejected = expect(pending).rejects.toMatchObject({ - name: 'PtyBackendCleanupError', - cleanupError: expect.objectContaining({ message: 'cleanup failed' }), - } satisfies Partial) - await vi.advanceTimersByTimeAsync(6) - await vi.advanceTimersByTimeAsync(2) - await rejected - vi.useRealTimers() - }) - - it('preserves startup failure when cleanup succeeds', async () => { - vi.useFakeTimers() - const ctx = new Context() - const completion = Promise.withResolvers<{ exitCode: number; stdout: string; stderr: string }>() - const created = { - pid: 123, - wait: () => completion.promise, - disconnect: vi.fn().mockResolvedValue(undefined), - } as unknown as CommandHandle - let sessionRunning = true - const sandbox = { - commands: { - run: vi.fn(async (command: string) => { - if (command.startsWith('env -0')) return { exitCode: 0, stdout: '', stderr: '' } - if (command.startsWith('ps -o sid=')) return { exitCode: 0, stdout: '123\n', stderr: '' } - if (command.startsWith('ps -eo sid=')) { - return { exitCode: 0, stdout: sessionRunning ? '123\n' : '', stderr: '' } - } - if (command.startsWith('kill -TERM')) { - sessionRunning = false - completion.resolve({ exitCode: 143, stdout: '', stderr: '' }) - } - return { exitCode: 0, stdout: '', stderr: '' } - }), - }, - pty: { kill: vi.fn().mockResolvedValue(true) }, - } as unknown as Sandbox - ctx.provide('e2b', { cwd: '/workspace', getSandbox: async () => sandbox } as unknown as E2BSandboxService) - const backend = new E2BPtyBackend(ctx, config(), async () => created) - const rejected = expect(backend.spawn({ sessionId: PtySessionId('failed-clean'), owner: owner(ctx), type: 'shell' })) - .rejects.toThrow('startup timeout') - await vi.advanceTimersByTimeAsync(6) - await rejected - vi.useRealTimers() - }) - - it('validates configuration and registers the selected backend type', async () => { - const valid = config() - expect(() => { validateConfig(valid) }).not.toThrow() - for (const invalid of [ - { ...valid, backendType: '' }, - { ...valid, rows: 0 }, - { ...valid, rows: 1.5 }, - { ...valid, maxReadBytes: 129 }, - ]) { - expect(() => { validateConfig(invalid) }).toThrow() - } - - const registerBackend = vi.fn() - apply({ pty: { registerBackend } } as unknown as Context, valid) - expect(registerBackend).toHaveBeenCalledWith(expect.objectContaining({ type: 'shell' })) - - const ctx = new Context() - await ctx.plugin(PtyService) - ctx.provide('e2b', { cwd: '/workspace', getSandbox: async () => ({}) } as never) - const fiber = await ctx.plugin({ - inject: ['pty', 'e2b'], - apply: (pluginCtx: Context) => { apply(pluginCtx, valid) }, - }) - expect(ctx.pty.listBackends()).toEqual(['shell']) - await fiber.dispose() - }) - - it('registers the package-owned invariant companion', async () => { - const ctx = new Context() - await ctx.plugin(InvariantService, { enabled: true }) - const fiber = await ctx.plugin(E2BPtyInvariant).await() - await fiber.dispose() - }) -}) diff --git a/packages/e2b/pty-e2b/tests/session.spec.ts b/packages/e2b/pty-e2b/tests/session.spec.ts deleted file mode 100644 index 1fedf6bbac..0000000000 --- a/packages/e2b/pty-e2b/tests/session.spec.ts +++ /dev/null @@ -1,485 +0,0 @@ -import { Buffer } from 'node:buffer' -import { afterEach, describe, expect, it, vi } from 'vitest' -import { - CommandExitError, - type CommandHandle, - type CommandResult, - type Sandbox, -} from '@deepseek-ai/dsh-e2b' -import type { PtySendOperation, PtySessionStatus } from '@deepseek-ai/dsh-pty' -import { E2BPtySession } from '@deepseek-ai/dsh-pty-e2b' -import type { ResolvedConfig } from '@deepseek-ai/dsh-pty-e2b/src/config.ts' - -function commandError(exitCode: number): CommandExitError { - return new CommandExitError({ exitCode, stdout: '', stderr: '', error: `exit ${exitCode}` }) -} - -class FakePtyHandle { - pid = 123 - readonly result = Promise.withResolvers() - disconnects = 0 - kills = 0 - disconnectError: unknown - private settled = false - - wait(): Promise { - return this.result.promise - } - - async disconnect(): Promise { - this.disconnects += 1 - if (this.disconnectError !== undefined) throw this.disconnectError - } - - async kill(): Promise { - this.kills += 1 - return true - } - - exit(exitCode = 0): void { - if (this.settled) return - this.settled = true - this.result.resolve({ exitCode, stdout: '', stderr: '' }) - } - - failExit(exitCode: number): void { - if (this.settled) return - this.settled = true - this.result.reject(commandError(exitCode)) - } - - crash(error: unknown): void { - if (this.settled) return - this.settled = true - this.result.reject(error) - } - - asHandle(): CommandHandle { - return this as unknown as CommandHandle - } -} - -class FakeSandbox { - readonly sent: Array<{ pid: number; data: Buffer }> = [] - readonly commands: string[] = [] - readonly killed: number[] = [] - pgid = '456\n' - sessionGroups = [123] - sendError: unknown - signalError: unknown - killError: unknown - foregroundLookup: Promise | undefined - onTerm: (() => void) | undefined - onGroupKill: (() => void) | undefined - onKill: (() => void) | undefined - - readonly sandbox = { - pty: { - sendInput: async (pid: number, data: Uint8Array): Promise => { - this.sent.push({ pid, data: Buffer.from(data) }) - if (this.sendError !== undefined) throw this.sendError - }, - kill: async (pid: number): Promise => { - this.killed.push(pid) - if (this.killError !== undefined) throw this.killError - this.onKill?.() - return true - }, - }, - commands: { - run: async (command: string): Promise => { - this.commands.push(command) - if (command.startsWith('ps -o tpgid')) { - return await (this.foregroundLookup ?? Promise.resolve({ exitCode: 0, stdout: this.pgid, stderr: '' })) - } - if (command.startsWith('ps -eo sid=')) { - return { exitCode: 0, stdout: this.sessionGroups.map(value => `${value}\n`).join(''), stderr: '' } - } - if (command.startsWith('kill -')) { - if (this.signalError !== undefined) { - const error = this.signalError - this.signalError = undefined - throw error - } - if (command.startsWith('kill -TERM')) this.onTerm?.() - if (command.startsWith('kill -KILL')) this.onGroupKill?.() - } - return { exitCode: 0, stdout: '', stderr: '' } - }, - }, - } as unknown as Sandbox -} - -function config(overrides: Partial = {}): ResolvedConfig { - return { - backendType: 'shell', rows: 24, cols: 80, - scrollbackLines: 10, scrollbackMaxBytes: 128, maxReadBytes: 64, - pollIntervalMs: 10, idleSilenceMs: 40, timeoutMs: 100, disposeGraceMs: 20, - ...overrides, - } -} - -async function initialize(session: E2BPtySession): Promise { - const pending = session.initialize() - session.onData(Buffer.from('\x1b]133;D;0\x07dsh> ')) - await vi.advanceTimersByTimeAsync(10) - await pending -} - -afterEach(() => { vi.useRealTimers() }) - -describe('E2BPtySession readiness, output, and signals', () => { - it('initializes, sends UTF-8 input, settles at a prompt, and reads bounded scrollback', async () => { - vi.useFakeTimers() - const fake = new FakeSandbox() - const handle = new FakePtyHandle() - const session = new E2BPtySession(fake.sandbox, handle.asHandle(), 123, config({ maxReadBytes: 12 })) - expect(session.read({})).toMatchObject({ text: '', totalLines: 0 }) - await initialize(session) - expect(session.motd).toBe('dsh> ') - - const operation = session.startSend({ text: 'printf 你好', submit: true }) - expect(fake.sent).toEqual([{ pid: 123, data: Buffer.from('printf 你好\r') }]) - session.onData(Buffer.from('一\n二\n三\x1b]133;D;0\x07dsh> ')) - const bounded = operation.readOutput() - expect(bounded.delta).toContain('三') - expect(bounded.truncated).toBe(true) - await vi.advanceTimersByTimeAsync(10) - expect(await operation.done).toMatchObject({ waitReason: 'stdin_read', sessionStatus: { kind: 'running' } }) - expect(operation.cancel()).toBe(false) - expect(session.read({ count: 2 }).text).toContain('dsh>') - expect(session.read({ offset: 99 })).toMatchObject({ text: '', lineBegin: 99, lineEnd: 99 }) - expect(() => session.read({ offset: -1 })).toThrow('non-negative safe integer') - expect(() => session.read({ offset: 1.5 })).toThrow('non-negative safe integer') - expect(() => session.read({ count: 0 })).toThrow('positive safe integer') - expect(() => session.read({ count: 1.5 })).toThrow('positive safe integer') - - await expect(session.signal('SIGTERM')).resolves.toEqual({ delivered: true, targetPgid: 456 }) - expect(fake.commands).toContain('kill -TERM -- -456') - expect(session.status()).toEqual({ kind: 'running' }) - }) - - it('distinguishes inferred idle, timeout, session exit, and no-output startup timeout', async () => { - vi.useFakeTimers() - const fake = new FakeSandbox() - const handle = new FakePtyHandle() - const session = new E2BPtySession(fake.sandbox, handle.asHandle(), 123, config()) - await initialize(session) - - const inferred = session.startSend({ text: '', submit: false }) - await vi.advanceTimersByTimeAsync(40) - expect((await inferred.done).waitReason).toBe('inferred_idle') - - const timeout = session.startSend({ text: '', submit: false }) - for (let index = 0; index < 3; index += 1) { - await vi.advanceTimersByTimeAsync(30) - session.onData(Buffer.from('.')) - } - await vi.advanceTimersByTimeAsync(10) - expect((await timeout.done).waitReason).toBe('timeout') - - const exiting = session.startSend({ text: '', submit: false }) - handle.failExit(143) - expect(await exiting.done).toMatchObject({ - waitReason: 'session_exit', - sessionStatus: { kind: 'exited', exitCode: null, signal: 'SIGTERM' }, - }) - expect(() => session.startSend({ text: '', submit: false })).toThrow('has exited') - - const startupHandle = new FakePtyHandle() - const startup = new E2BPtySession(fake.sandbox, startupHandle.asHandle(), 123, config()) - const timedOut = expect(startup.initialize()).rejects.toThrow('startup timeout') - await vi.advanceTimersByTimeAsync(100) - await timedOut - }) - - it('handles split prompt text, stale operations, and explicit cancellation', async () => { - vi.useFakeTimers() - const fake = new FakeSandbox() - const handle = new FakePtyHandle() - const session = new E2BPtySession(fake.sandbox, handle.asHandle(), 123, config()) - const initializing = session.initialize() - session.onData(Buffer.from('\x1b]133;D;0\x07')) - await vi.advanceTimersByTimeAsync(20) - session.onData(Buffer.from('dsh> ')) - await vi.advanceTimersByTimeAsync(10) - await initializing - - const operation = session.startSend({ text: 'sleep', submit: true }) - const internal = session as unknown as { - pollReadiness(operation: PtySendOperation): void - interrupt(operation: PtySendOperation): void - settleActive(reason: 'timeout'): void - failActive(error: unknown): void - appendOutput(text: string): void - statusValue: PtySessionStatus - } - internal.pollReadiness({} as PtySendOperation) - internal.interrupt({} as PtySendOperation) - internal.appendOutput('') - fake.pgid = '789\n' - expect(operation.cancel()).toBe(true) - await vi.advanceTimersByTimeAsync(0) - expect(fake.commands).toContain('kill -INT -- -789') - session.onData(Buffer.from('\x1b]133;D;130\x07dsh> ')) - await vi.advanceTimersByTimeAsync(10) - await operation.done - - internal.settleActive('timeout') - internal.failActive(new Error('ignored')) - const operationInternal = operation as unknown as { - append(text: string): void - settle(reason: 'timeout', status: PtySessionStatus, inherited: boolean): void - fail(error: unknown): void - } - operationInternal.append('ignored') - operationInternal.settle('timeout', { kind: 'running' }, false) - operationInternal.fail(new Error('ignored')) - }) - - it('observes AbortSignal and contains send or foreground lookup failures', async () => { - vi.useFakeTimers() - const fake = new FakeSandbox() - const handle = new FakePtyHandle() - const session = new E2BPtySession(fake.sandbox, handle.asHandle(), 123, config()) - await initialize(session) - - const controller = new AbortController() - const aborting = session.startSend({ text: '', submit: false, signal: controller.signal }) - expect(() => session.startSend({ text: '', submit: false })).toThrow('active send') - fake.pgid = 'not-a-pgid\n' - controller.abort() - await expect(aborting.done).rejects.toThrow('cannot resolve foreground process group') - - const already = new AbortController() - already.abort() - expect(() => session.startSend({ text: '', submit: false, signal: already.signal })).toThrow('aborted before write') - - fake.sendError = new Error('send failed') - const failed = session.startSend({ text: 'x', submit: false }) - await expect(failed.done).rejects.toThrow('send failed') - - fake.pgid = '123\n' - await expect(session.signal('SIGKILL')).rejects.toThrow('refusing to SIGKILL') - fake.pgid = '0\n' - await expect(session.signal('SIGINT')).rejects.toThrow('cannot resolve') - - const deferred = Promise.withResolvers() - fake.sendError = undefined - const sendInput = vi.spyOn(fake.sandbox.pty, 'sendInput').mockReturnValueOnce(deferred.promise) - const late = session.startSend({ text: 'late', submit: false }) - session.onData(Buffer.from('\x1b]133;D;0\x07dsh> ')) - await vi.advanceTimersByTimeAsync(10) - await late.done - deferred.reject(new Error('late failure')) - await vi.advanceTimersByTimeAsync(0) - expect(sendInput).toHaveBeenCalled() - }) - - it('does not let a stale interrupt signal or fail a successor send', async () => { - vi.useFakeTimers() - const fake = new FakeSandbox() - const handle = new FakePtyHandle() - const session = new E2BPtySession(fake.sandbox, handle.asHandle(), 123, config()) - await initialize(session) - - const lookup = Promise.withResolvers() - fake.foregroundLookup = lookup.promise - const stale = session.startSend({ text: 'old', submit: true }) - expect(stale.cancel()).toBe(true) - session.onData(Buffer.from('\x1b]133;D;0\x07dsh> ')) - await vi.advanceTimersByTimeAsync(10) - await stale.done - - const successor = session.startSend({ text: 'new', submit: true }) - fake.signalError = new Error('late interrupt failure') - lookup.resolve({ exitCode: 0, stdout: '789\n', stderr: '' }) - await vi.advanceTimersByTimeAsync(0) - expect(fake.commands).not.toContain('kill -INT -- -789') - session.onData(Buffer.from('\x1b]133;D;0\x07dsh> ')) - await vi.advanceTimersByTimeAsync(10) - await expect(successor.done).resolves.toMatchObject({ waitReason: 'stdin_read' }) - - const failedLookup = Promise.withResolvers() - fake.foregroundLookup = failedLookup.promise - const staleFailure = session.startSend({ text: 'old failure', submit: true }) - expect(staleFailure.cancel()).toBe(true) - session.onData(Buffer.from('\x1b]133;D;0\x07dsh> ')) - await vi.advanceTimersByTimeAsync(10) - await staleFailure.done - const finalSuccessor = session.startSend({ text: 'new after failure', submit: true }) - failedLookup.reject(new Error('late lookup failure')) - await vi.advanceTimersByTimeAsync(0) - session.onData(Buffer.from('\x1b]133;D;0\x07dsh> ')) - await vi.advanceTimersByTimeAsync(10) - await expect(finalSuccessor.done).resolves.toMatchObject({ waitReason: 'stdin_read' }) - }) - - it('preserves startup abort reasons and classifies invalid UTF-8 transport failures', async () => { - const fake = new FakeSandbox() - const abortHandle = new FakePtyHandle() - const abortSession = new E2BPtySession(fake.sandbox, abortHandle.asHandle(), 123, config()) - const controller = new AbortController() - const reason = new Error('startup cancelled') - const initializing = abortSession.initialize(controller.signal) - const rejected = expect(initializing).rejects.toBe(reason) - controller.abort(reason) - await rejected - - const invalidHandle = new FakePtyHandle() - const invalid = new E2BPtySession(fake.sandbox, invalidHandle.asHandle(), 123, config()) - const pending = invalid.startSend({ text: '', submit: false }) - invalid.onData(Uint8Array.from([0xff])) - await expect(pending.done).rejects.toThrow('invalid UTF-8') - expect(invalid.status()).toEqual({ kind: 'exited', exitCode: null, signal: null }) - - const crashHandle = new FakePtyHandle() - const crashed = new E2BPtySession(fake.sandbox, crashHandle.asHandle(), 123, config()) - const active = crashed.startSend({ text: '', submit: false }) - crashHandle.crash('transport gone') - await expect(active.done).rejects.toEqual(new Error('transport gone')) - - const startupExitHandle = new FakePtyHandle() - const startupExit = new E2BPtySession(fake.sandbox, startupExitHandle.asHandle(), 123, config()) - const exitedDuringStartup = expect(startupExit.initialize()).rejects.toThrow('exited during startup') - startupExitHandle.exit(7) - await exitedDuringStartup - }) - - it('covers empty bounded reads and polling an exited active session', async () => { - vi.useFakeTimers() - const fake = new FakeSandbox() - const tinyHandle = new FakePtyHandle() - const tiny = new E2BPtySession(fake.sandbox, tinyHandle.asHandle(), 123, config({ maxReadBytes: 1 })) - tiny.onData(Buffer.from('你')) - expect(tiny.read({ count: 1 })).toMatchObject({ text: '', lineEnd: 0 }) - - const handle = new FakePtyHandle() - const session = new E2BPtySession(fake.sandbox, handle.asHandle(), 123, config()) - const operation = session.startSend({ text: '', submit: false }) - const internal = session as unknown as { - pollReadiness(operation: PtySendOperation): void - clearActive(): void - statusValue: PtySessionStatus - } - internal.statusValue = { kind: 'exited', exitCode: 7, signal: null } - internal.pollReadiness(operation) - expect((await operation.done).waitReason).toBe('session_exit') - internal.clearActive() - }) -}) - -describe('E2BPtySession teardown', () => { - it('terminates the process group once, awaits exit, and disconnects', async () => { - vi.useFakeTimers() - const fake = new FakeSandbox() - const handle = new FakePtyHandle() - fake.onTerm = () => { fake.sessionGroups = []; handle.failExit(143) } - const session = new E2BPtySession(fake.sandbox, handle.asHandle(), 123, config()) - const first = session.close('done') - expect(session.close('again')).toBe(first) - await first - expect(session.status()).toEqual({ kind: 'exited', exitCode: null, signal: 'SIGTERM' }) - expect(handle.disconnects).toBe(1) - expect(() => session.startSend({ text: '', submit: false })).toThrow('closing') - }) - - it('escalates every job-control group that survives shell exit', async () => { - vi.useFakeTimers() - const fake = new FakeSandbox() - fake.sessionGroups = [123, 456] - const handle = new FakePtyHandle() - fake.onTerm = () => { fake.sessionGroups = [456]; handle.failExit(143) } - fake.onGroupKill = () => { fake.sessionGroups = [] } - const session = new E2BPtySession(fake.sandbox, handle.asHandle(), 123, config()) - - const closing = session.close('tree cleanup') - await vi.advanceTimersByTimeAsync(100) - await closing - - expect(fake.commands).toContain('kill -TERM -- -123 -456') - expect(fake.commands).toContain('kill -KILL -- -456') - }) - - it('contains an already-gone TERM, escalates to KILL, and reports a survivor', async () => { - vi.useFakeTimers() - const gone = new FakeSandbox() - const goneHandle = new FakePtyHandle() - gone.signalError = commandError(1) - gone.onGroupKill = () => { gone.sessionGroups = [] } - gone.onKill = () => { goneHandle.failExit(137) } - const goneSession = new E2BPtySession(gone.sandbox, goneHandle.asHandle(), 123, config()) - const closingGone = goneSession.close('gone') - await vi.advanceTimersByTimeAsync(100) - await closingGone - expect(gone.killed).toEqual([123]) - expect(goneSession.status()).toEqual({ kind: 'exited', exitCode: null, signal: 'SIGKILL' }) - - const survivor = new FakeSandbox() - const survivorHandle = new FakePtyHandle() - const survivorSession = new E2BPtySession(survivor.sandbox, survivorHandle.asHandle(), 123, config()) - const failed = expect(survivorSession.close('still alive')).rejects.toThrow('surviving process groups: 123') - await vi.advanceTimersByTimeAsync(100) - await failed - survivorHandle.exit() - survivor.sessionGroups = [] - await expect(survivorSession.close('retry')).resolves.toBeUndefined() - }) - - it('propagates cleanup transport failures and lets close retry', async () => { - const fake = new FakeSandbox() - const handle = new FakePtyHandle() - fake.signalError = new Error('TERM transport failed') - const session = new E2BPtySession(fake.sandbox, handle.asHandle(), 123, config()) - await expect(session.close('failure')).rejects.toThrow('TERM transport failed') - handle.exit() - fake.sessionGroups = [] - await expect(session.close('retry')).resolves.toBeUndefined() - - const invalidTailHandle = new FakePtyHandle() - const invalidTail = new E2BPtySession(fake.sandbox, invalidTailHandle.asHandle(), 123, config()) - invalidTail.onData(Uint8Array.from([0xe2])) - invalidTailHandle.exit() - await expect(invalidTail.close('invalid tail')).rejects.toThrow('invalid UTF-8') - - const normalHandle = new FakePtyHandle() - normalHandle.disconnectError = new Error('disconnect raced') - const normal = new E2BPtySession(fake.sandbox, normalHandle.asHandle(), 123, config()) - normalHandle.exit(7) - await Promise.resolve() - expect(normal.status()).toEqual({ kind: 'exited', exitCode: 7, signal: null }) - await expect(normal.close('already exited')).resolves.toBeUndefined() - }) - - it('rejects invalid session groups and a shell handle that survives SDK kill', async () => { - const invalid = new FakeSandbox() - invalid.sessionGroups = [1] - const invalidSession = new E2BPtySession(invalid.sandbox, new FakePtyHandle().asHandle(), 123, config()) - await expect(invalidSession.close('invalid group')).rejects.toThrow('invalid process group') - - vi.useFakeTimers() - const survivor = new FakeSandbox() - survivor.sessionGroups = [] - const survivorHandle = new FakePtyHandle() - const survivorSession = new E2BPtySession(survivor.sandbox, survivorHandle.asHandle(), 123, config()) - const failed = expect(survivorSession.close('shell survived')).rejects.toThrow('surviving pid: 123') - await vi.advanceTimersByTimeAsync(100) - await failed - expect(survivor.killed).toEqual([123]) - }) - - it('kills a remotely live PTY after its host transport fails', async () => { - const fake = new FakeSandbox() - const handle = new FakePtyHandle() - const session = new E2BPtySession(fake.sandbox, handle.asHandle(), 123, config()) - const active = session.startSend({ text: '', submit: false }) - session.onData(Uint8Array.from([0xff])) - await expect(active.done).rejects.toThrow('invalid UTF-8') - expect(session.status()).toEqual({ kind: 'exited', exitCode: null, signal: null }) - - fake.onTerm = () => { fake.sessionGroups = []; handle.failExit(143) } - await expect(session.close('transport failed')).rejects.toThrow('invalid UTF-8') - expect(fake.commands).toContain('kill -TERM -- -123') - expect(handle.disconnects).toBe(1) - }) -}) diff --git a/packages/e2b/pty-e2b/tsconfig.json b/packages/e2b/pty-e2b/tsconfig.json deleted file mode 100644 index 06bf0cd589..0000000000 --- a/packages/e2b/pty-e2b/tsconfig.json +++ /dev/null @@ -1,16 +0,0 @@ -{ - "extends": "../../../tsconfig.base.json", - "compilerOptions": { - "rootDir": "src", - "outDir": "lib/types" - }, - "include": ["src"], - "references": [ - { "path": "../../../vendor/cosmokit" }, - { "path": "../../../vendor/cordis" }, - { "path": "../../../vendor/schemastery" }, - { "path": "../e2b" }, - { "path": "../../pty/pty" }, - { "path": "../../support/invariants" } - ] -} diff --git a/packages/e2b/subprocess-e2b/README.i18n.yaml b/packages/e2b/subprocess-e2b/README.i18n.yaml index f05191a45b..dca7b016c2 100644 --- a/packages/e2b/subprocess-e2b/README.i18n.yaml +++ b/packages/e2b/subprocess-e2b/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/e2b/subprocess-e2b/README.md -README.md: 066d35a099f560fe40fe629ede632c37535129f7 -README.zh.md: d3af14e8cc58c7d86331156358f78a0abbb24c39 +README.md: e0805345e708c67c6de721d95641e2366a7f990e +README.zh.md: f83969c794e50f0658aa86218a0f5f3c89d53b56 diff --git a/packages/e2b/subprocess-e2b/README.md b/packages/e2b/subprocess-e2b/README.md index 066d35a099..e0805345e7 100644 --- a/packages/e2b/subprocess-e2b/README.md +++ b/packages/e2b/subprocess-e2b/README.md @@ -2,16 +2,18 @@ English | [中文](README.zh.md) -E2B implementation of the [`@deepseek-ai/dsh-subprocess`](../../subprocess/subprocess/README.md) seam. It has no config: load [`@deepseek-ai/dsh-e2b`](../e2b/README.md) first, then this service in place of `dsh-subprocess-local`. Existing consumers such as [`dsh-bash-local`](../../bash/bash-local/README.md) then execute in the shared remote sandbox without an E2B-specific Bash adapter. +E2B implementation of the [`@deepseek-ai/dsh-subprocess`](../../subprocess/subprocess/README.md) seam. It has no config: load [`@deepseek-ai/dsh-e2b`](../e2b/README.md) first, then this service in place of `dsh-subprocess-local`. Existing Bash, PTY, LSP, and subprocess Code Runtime consumers then execute in the shared remote sandbox without E2B-specific capability packages. ## Behavior - **Asynchronous remote start** — the synchronous seam returns a handle immediately while `Sandbox.commands.run(..., { background: true })` starts remotely. `pid` is `-1` until the wrapper publishes and the adapter validates its process-group id; `done`, stdin, termination, and `waitForExit()` wait for readiness internally. +- **Execution-world coordinates** — `cwd` and private `runtimeRoot` come from the shared owner; executable lookup verifies absolute paths or resolves a bare name against the sandbox PATH plus explicit overrides. - **Linux process groups** — a quoted wrapper starts each argv under `exec setsid --wait` and records its actual process-group id plus private status files beneath `ctx.e2b.runtimeRoot/processes`. The handle waits for that file instead of treating the SDK command PID as its published identity. Termination signals the negative recorded id with `SIGTERM`, waits the caller's `graceMs`, then escalates to `SIGKILL` and the SDK kill fallback. If publication fails, the SDK PID remains the provisional `exec setsid` group id; rollback kills and verifies that group before startup rejects. Service disposal terminates and joins every retained handle before the sandbox owner disposes. - **Environment boundary** — the wrapper starts from the sandbox command environment, removes ambient `DSH_*` and credential-shaped (`*KEY*`, `*SECRET*`, `*TOKEN*`) names, then restores every `spec.env` entry as an explicit caller opt-in. Host ambient variables never enter the sandbox implicitly. - **Stdio projection** — pipe mode forwards E2B callbacks into host Node streams; inherit mode forwards them to the harness process streams; collect mode retains a bounded host tail with offset reads. Optional complete spill files are written remotely and advertised only while within their cap. Batch and streaming stdin use the SDK handle. +- **Terminal sessions** — `spawnTerminal()` uses E2B's byte PTY API, installs the exact argv and scrubbed environment through private mode-`0600` files, reports the foreground process group, sends real signals, and tears down every group in the remote terminal session before settlement. Prompt detection, scrollback, readiness, and owner policy remain in `dsh-pty-local`. -The base E2B image supplies the Bash/GNU utilities this adapter invokes: `bash`, `setsid`, `ps`, `tr`, `env`, `chmod`, `tee`, `head`, and `kill`. A custom template must retain compatible commands. +The base E2B image supplies the Bash/GNU utilities this adapter invokes: `bash`, `setsid`, `ps`, `awk`, `tr`, `env`, `chmod`, `tee`, `head`, and `kill`. A custom template must retain compatible commands and E2B PTY support. ## Model Experience @@ -24,9 +26,10 @@ No direct invalidation; the named consumers own any request-prefix changes. ## Known Limitations and Deferred Work - **The SDK still retains complete command output in host memory** — E2B `CommandHandle.stdout` and `.stderr` accumulate even when this adapter exposes bounded tails, so the subprocess seam's normal host-memory bound is not achieved. -- **Pipe output is not byte-faithful** — E2B delivers separately decoded strings rather than raw bytes, so split multibyte sequences and arbitrary binary protocols can be corrupted; LSP and other framed byte-stream consumers are unsupported. +- **Command-pipe output is text-decoded by the SDK** — valid UTF-8 protocol traffic, including the exercised LSP composition and Code Runtime's ASCII/base64 frames, is supported; arbitrary binary protocols and invalid UTF-8 are not byte-faithful. - **Synchronous-PID consumers are unsupported** — `pid` remains `-1` during remote startup; consumers that require a positive PID immediately, including the ACP child backend, cannot use this provider unchanged. - **Reconnect does not reconstruct handles** — remote PID/status/spill files survive a retained sandbox, but a new harness process does not rebuild live `SubprocessHandle` objects or output cursors from them. - **Remote state accumulates when retained** — process directories and valid spill files remain under `.dsh-e2b`; this POC supplies no retention sweep. - **Signal attribution is inferred** — when termination was requested and E2B reports a nonzero exit code, the adapter reports the last requested signal because the SDK result does not identify the terminating signal. -- **Linux utility and E2B transport semantics are assumed** — there is no PTY, Windows, arbitrary-template, or network-partition fidelity layer. +- **Exact terminal stdin-wait inspection is unavailable** — E2B exposes the foreground process group but not the syscall evidence needed to prove it is waiting on fd 0, so the generic PTY backend falls back to controlled prompt markers and bounded silence. +- **Linux utility and E2B transport semantics are assumed** — there is no Windows, arbitrary-template, escaped-session recovery, or network-partition fidelity layer. diff --git a/packages/e2b/subprocess-e2b/README.zh.md b/packages/e2b/subprocess-e2b/README.zh.md index d3af14e8cc..f83969c794 100644 --- a/packages/e2b/subprocess-e2b/README.zh.md +++ b/packages/e2b/subprocess-e2b/README.zh.md @@ -2,16 +2,18 @@ [English](README.md) | 中文 -[`@deepseek-ai/dsh-subprocess`](../../subprocess/subprocess/README.md) seam 的 E2B 实现。它没有配置:先加载 [`@deepseek-ai/dsh-e2b`](../e2b/README.md),再用本服务取代 `dsh-subprocess-local`。随后,[`dsh-bash-local`](../../bash/bash-local/README.md) 等现有消费方会在共享远程沙箱中执行,无需 E2B 专用 Bash 适配器。 +[`@deepseek-ai/dsh-subprocess`](../../subprocess/subprocess/README.md) seam 的 E2B 实现。它没有配置:先加载 [`@deepseek-ai/dsh-e2b`](../e2b/README.md),再用本服务取代 `dsh-subprocess-local`。现有的 Bash、PTY、LSP 以及使用 subprocess 的 Code Runtime 消费方随后会在共享远程沙箱中执行,无需 E2B 专用的功能包(package)。 ## 行为 - **异步远程启动**:同步 seam 会立即返回一个句柄,同时由 `Sandbox.commands.run(..., { background: true })` 在远程启动进程。包装层发布进程组 ID 并由适配器完成验证之前,`pid` 为 `-1`;`done`、stdin、终止和 `waitForExit()` 会在内部等待就绪。 +- **执行世界坐标**:`cwd` 和私有 `runtimeRoot` 来自共享所有者;可执行文件查找会验证绝对路径,或根据沙箱 PATH 加显式覆盖来解析裸名称。 - **Linux 进程组**:带引号保护的包装层会在 `exec setsid --wait` 下启动每组 argv,并在 `ctx.e2b.runtimeRoot/processes` 下记录实际进程组 ID 和私有状态文件。句柄会等待该文件,而不会把 SDK 命令 PID 当作已发布的身份。终止操作以记录的负数 ID 发送 `SIGTERM`,等待调用方的 `graceMs`,再升级到 `SIGKILL` 和 SDK kill 回退。如果发布失败,SDK PID 仍为临时的 `exec setsid` 进程组 ID;回滚会终止并验证该进程组,随后启动操作才会以拒绝结束。服务 dispose(资源释放)会在沙箱所有者释放前终止并等待每个保留句柄退出。 - **环境边界**:包装层从沙箱命令环境开始,移除环境中的 `DSH_*` 和形似凭据的名称(`*KEY*`、`*SECRET*`、`*TOKEN*`),再把每个 `spec.env` 条目恢复为调用方显式选择。宿主环境变量绝不会隐式进入沙箱。 - **stdio 投影**:pipe 模式把 E2B 回调转发到宿主 Node 流;inherit 模式把回调转发到 harness 进程流;collect 模式保留有界的宿主尾部,并支持基于偏移量读取。可选的完整 spill 文件写在远程,并且只有未超过其上限时才会对外公布。批量 stdin 和流式 stdin 都使用 SDK 句柄。 +- **终端会话**:`spawnTerminal()` 使用 E2B 的字节 PTY API,以 mode 为 `0600` 的私有文件传入原样 argv 与清理后的环境,报告前台进程组,发送真实信号,并在结算前清理远程终端会话中的每个进程组。提示符检测、scrollback、就绪状态与所有者策略仍归 `dsh-pty-local` 所有。 -基础 E2B 镜像提供该适配器调用的 Bash/GNU 工具:`bash`、`setsid`、`ps`、`tr`、`env`、`chmod`、`tee`、`head` 和 `kill`。自定义模板必须保留兼容的命令。 +基础 E2B 镜像提供该适配器调用的 Bash/GNU 工具:`bash`、`setsid`、`ps`、`awk`、`tr`、`env`、`chmod`、`tee`、`head` 和 `kill`。自定义模板必须保留兼容的命令和 E2B PTY 支持。 ## 模型体验 @@ -24,9 +26,10 @@ ## 已知限制与延后工作 - **SDK 仍会在宿主内存中保留完整命令输出**:即使本适配器公开的是有界尾部,E2B `CommandHandle.stdout` 和 `.stderr` 仍会持续累积,因此无法达到进程管理 seam 通常提供的宿主内存边界。 -- **Pipe 输出并非字节保真**:E2B 交付的是分别解码后的字符串,而不是原始字节,因此拆分的多字节序列和任意二进制协议可能损坏;不支持 LSP 及其他带帧字节流消费方。 +- **命令管道输出由 SDK 解码为文本**:支持有效的 UTF-8 协议流量,包括已经过测试的 LSP 组合与 Code Runtime 的 ASCII/base64 帧;任意二进制协议和无效 UTF-8 不具备字节保真。 - **不支持需要同步 PID 的消费方**:远程启动期间,`pid` 保持为 `-1`;包括 ACP 子进程后端在内,要求立即获得正 PID 的消费方无法原样使用本提供方。 - **重新连接不会重建句柄**:保留沙箱后,远程 PID/状态/spill 文件仍然存在,但新的 harness 进程不会据此重建实时 `SubprocessHandle` 对象或输出游标。 - **保留沙箱时会累积远程状态**:进程目录和有效的 spill 文件会留在 `.dsh-e2b` 下;本 POC 不提供保留清理。 - **信号归因依靠推断**:如果已经请求终止,而 E2B 报告非零退出码,适配器会报告最后请求的信号,因为 SDK 结果不标识终止信号。 -- **依赖 Linux 工具与 E2B 传输语义**:没有 PTY、Windows、任意模板或网络分区的保真层。 +- **无法精确检查终端 stdin 等待状态**:E2B 会公开前台进程组,但不提供证明其正在等待 fd 0 所需的 syscall 证据,因此通用 PTY 后端会回退到受控提示符标记与有界静默机制。 +- **依赖 Linux 工具与 E2B 传输语义**:没有 Windows、任意模板、逃逸会话恢复或网络分区的保真层。 diff --git a/packages/e2b/subprocess-e2b/src/index.ts b/packages/e2b/subprocess-e2b/src/index.ts index 2856df67bd..1547e1b46e 100644 --- a/packages/e2b/subprocess-e2b/src/index.ts +++ b/packages/e2b/subprocess-e2b/src/index.ts @@ -8,29 +8,87 @@ import { randomUUID } from 'node:crypto' import { posix } from 'node:path' import { Context } from 'cordis' import { SubprocessService } from '@deepseek-ai/dsh-subprocess' -import type { SubprocessHandle, SubprocessSpawnSpec } from '@deepseek-ai/dsh-subprocess' +import type { + SubprocessHandle, + SubprocessSpawnSpec, + SubprocessTerminalHandle, + SubprocessTerminalSpawnSpec, +} from '@deepseek-ai/dsh-subprocess' +import { quoteE2BShellArg } from '@deepseek-ai/dsh-e2b' import { E2BSubprocessHandle } from './process.ts' +import { spawnE2BTerminal } from './terminal.ts' + +function signalOpts(signal: AbortSignal | undefined): { signal?: AbortSignal } { + return signal === undefined ? {} : { signal } +} /** E2B command manager registered as `ctx.subprocess`. */ export class E2BSubprocessService extends SubprocessService { static inject = ['e2b'] private readonly live = new Set() + private readonly terminals = new Set() + + /** @inheritdoc */ + readonly cwd: string + + /** @inheritdoc */ + readonly runtimeRoot: string /** Create the E2B subprocess service and bind its disposal policy. */ constructor(ctx: Context) { super(ctx) + this.cwd = ctx.e2b.cwd + this.runtimeRoot = ctx.e2b.runtimeRoot ctx.effect(() => async () => { const handles = [...this.live] - for (const handle of handles) handle.terminate() - await Promise.all(handles.map(async (handle) => { - await handle.done.catch(() => {}) - await handle.waitForExit() - })) + const terminals = [...this.terminals] + const pending: Promise[] = [] + for (const handle of handles) { + handle.terminate() + pending.push(handle.done.catch(() => {}).then(() => handle.waitForExit())) + } + for (const terminal of terminals) { + terminal.terminate() + pending.push(terminal.waitForExit()) + } this.live.clear() + this.terminals.clear() + await Promise.all(pending) }, 'e2b subprocess teardown') } + /** @inheritdoc */ + async resolveExecutable( + command: string, + env?: Readonly>, + signal?: AbortSignal, + ): Promise { + if (command.length === 0) throw new Error('subprocess-e2b: executable name must be non-empty') + signal?.throwIfAborted() + const sandbox = await this.ctx.e2b.getSandbox() + if (posix.isAbsolute(command)) { + await sandbox.commands.run( + `test -f ${quoteE2BShellArg(command)} -a -x ${quoteE2BShellArg(command)}`, + signalOpts(signal), + ) + signal?.throwIfAborted() + return command + } + const path = env?.PATH + const prefix = path === undefined ? '' : `PATH=${quoteE2BShellArg(path)} ` + const result = await sandbox.commands.run( + `${prefix}command -v -- ${quoteE2BShellArg(command)}`, + signalOpts(signal), + ) + signal?.throwIfAborted() + const executable = result.stdout.trim() + if (!posix.isAbsolute(executable) || executable.includes('\n')) { + throw new Error(`subprocess-e2b: executable ${JSON.stringify(command)} did not resolve to one absolute path`) + } + return executable + } + /** @inheritdoc */ spawn(spec: SubprocessSpawnSpec): SubprocessHandle { const program = spec.argv[0] @@ -53,6 +111,29 @@ export class E2BSubprocessService extends SubprocessService { void handle.done.then(release, release).catch(() => {}) return handle } + + /** @inheritdoc */ + async spawnTerminal(spec: SubprocessTerminalSpawnSpec): Promise { + const program = spec.argv[0] + if (program === undefined || program.length === 0) { + throw new Error('subprocess-e2b: terminal argv must contain a program') + } + for (const [name, value] of [['rows', spec.rows], ['cols', spec.cols], ['graceMs', spec.graceMs]] as const) { + if (!Number.isSafeInteger(value) || value <= 0) { + throw new Error(`subprocess-e2b: terminal ${name} must be a positive safe integer`) + } + } + spec.signal?.throwIfAborted() + const stateDir = posix.join(this.runtimeRoot, 'terminals', randomUUID()) + const terminal = await spawnE2BTerminal(this.ctx.e2b, spec, stateDir) + this.terminals.add(terminal) + const release = async (): Promise => { + await terminal.waitForExit() + this.terminals.delete(terminal) + } + void terminal.done.then(release, release).catch(() => {}) + return terminal + } } export default E2BSubprocessService diff --git a/packages/e2b/subprocess-e2b/src/terminal.ts b/packages/e2b/subprocess-e2b/src/terminal.ts new file mode 100644 index 0000000000..88bea44fde --- /dev/null +++ b/packages/e2b/subprocess-e2b/src/terminal.ts @@ -0,0 +1,386 @@ +/** E2B PTY allocation and process-session ownership for the subprocess seam. */ + +import { Buffer } from 'node:buffer' +import { constants } from 'node:os' +import { PassThrough } from 'node:stream' +import { posix } from 'node:path' +import { + CommandExitError, + FileNotFoundError, + quoteE2BShellArg, +} from '@deepseek-ai/dsh-e2b' +import type { CommandHandle, CommandResult, Sandbox } from '@deepseek-ai/dsh-e2b' +import { SENSITIVE_ENV_PATTERN } from '@deepseek-ai/dsh-subprocess' +import type { + SubprocessOutcome, + SubprocessTerminalForeground, + SubprocessTerminalHandle, + SubprocessTerminalSignal, + SubprocessTerminalSpawnSpec, +} from '@deepseek-ai/dsh-subprocess' +import type E2BSandboxService from '@deepseek-ai/dsh-e2b' + +const POLL_MS = 20 + +const TERMINAL_RUNNER_SOURCE = [ + '#!/bin/bash', + 'set -euo pipefail', + 'dsh_state=$1', + 'mapfile -d \'\' -t dsh_env < "$dsh_state/environment"', + 'mapfile -d \'\' -t dsh_argv < "$dsh_state/argv"', + 'rm -f -- "$dsh_state/environment" "$dsh_state/argv" "$dsh_state/runner.bash"', + 'if (( ${#dsh_argv[@]} == 0 )); then', + " printf 'terminal runner received empty argv\\n' >&2", + ' exit 125', + 'fi', + "printf 'ready\\n' > \"$dsh_state/ready\"", + 'exec env -i "${dsh_env[@]}" "${dsh_argv[@]}"', + '', +].join('\n') + +interface TerminalPaths { + runner: string + environment: string + argv: string + ready: string +} + +function signalOpts(signal: AbortSignal | undefined): { signal?: AbortSignal } { + return signal === undefined ? {} : { signal } +} + +function delay(ms: number): Promise { + return new Promise(resolve => setTimeout(resolve, ms)) +} + +function commandSignal(exitCode: number): NodeJS.Signals | null { + const number = exitCode - 128 + if (number <= 0) return null + for (const [name, value] of Object.entries(constants.signals)) { + if (value === number) return name as NodeJS.Signals + } + return null +} + +function parsePositiveId(value: string, message: string): number { + const raw = value.trim() + const id = Number(raw) + if (!/^[1-9][0-9]*$/.test(raw) || !Number.isSafeInteger(id)) throw new Error(message) + return id +} + +function serializeValues(values: readonly string[], kind: string): string { + for (const value of values) { + if (value.includes('\0')) throw new Error(`subprocess-e2b: terminal ${kind} must not contain NUL bytes`) + } + return values.map(value => `${value}\0`).join('') +} + +function remoteEnvironment(raw: string, explicit: Readonly> | undefined): string { + const environment = new Map() + for (const entry of raw.split('\0')) { + if (entry.length === 0) continue + const separator = entry.indexOf('=') + if (separator <= 0) continue + const name = entry.slice(0, separator) + if (name.startsWith('DSH_') || SENSITIVE_ENV_PATTERN.test(name)) continue + environment.set(name, entry.slice(separator + 1)) + } + for (const [name, value] of Object.entries(explicit ?? {})) { + if (name.length === 0 || name.includes('=') || name.includes('\0') || value.includes('\0')) { + throw new Error('subprocess-e2b: terminal environment entries require non-empty NUL-free names without = and NUL-free values') + } + environment.set(name, value) + } + return serializeValues([...environment].map(([name, value]) => `${name}=${value}`), 'environment') +} + +async function terminalSessionId(sandbox: Sandbox, pid: number, signal?: AbortSignal): Promise { + const result = await sandbox.commands.run(`ps -o sid= -p ${pid}`, signalOpts(signal)) + signal?.throwIfAborted() + return parsePositiveId(result.stdout, `subprocess-e2b: cannot resolve process session for terminal ${pid}`) +} + +async function waitUntilReady( + sandbox: Sandbox, + paths: TerminalPaths, + completion: Promise, + signal?: AbortSignal, +): Promise { + const settled = completion.then(() => true, () => true) + for (;;) { + signal?.throwIfAborted() + try { + if ((await sandbox.files.read(paths.ready, signalOpts(signal))).trim() === 'ready') return + } catch (error: unknown) { + if (!(error instanceof FileNotFoundError)) throw error + } + if (await Promise.race([settled, delay(POLL_MS).then(() => false)])) { + throw new Error('subprocess-e2b: terminal exited before publishing readiness') + } + } +} + +/** One E2B PTY and all process groups in its remote process session. */ +export class E2BTerminalHandle implements SubprocessTerminalHandle { + readonly pid: number + readonly done: Promise + + private topLevelExited = false + private termination: Promise | undefined + private terminationSignal: NodeJS.Signals | null = null + private removeAbort: (() => void) | undefined + + constructor( + private readonly sandbox: Sandbox, + private readonly handle: CommandHandle, + readonly output: PassThrough, + private readonly completion: Promise, + private readonly sessionId: number, + private readonly stateDir: string, + private readonly graceMs: number, + signal?: AbortSignal, + ) { + this.pid = handle.pid + this.done = this.waitForCommand() + void this.done.then(() => { this.terminate() }, () => { this.terminate() }) + if (signal !== undefined) { + const onAbort = (): void => { this.terminate() } + signal.addEventListener('abort', onAbort, { once: true }) + this.removeAbort = () => { signal.removeEventListener('abort', onAbort) } + if (signal.aborted) this.terminate() + } + } + + /** @inheritdoc */ + async write(data: Uint8Array): Promise { + if (this.topLevelExited) throw new Error('terminal process has exited') + await this.sandbox.pty.sendInput(this.pid, data) + } + + /** @inheritdoc */ + async inspectForeground(): Promise { + try { + const result = await this.sandbox.commands.run(`ps -o tpgid= -p ${this.pid}`) + return { + processGroupId: parsePositiveId( + result.stdout, + `subprocess-e2b: cannot resolve foreground process group for terminal ${this.pid}`, + ), + // E2B exposes process-table commands but not the /proc memory access + // needed to prove a specific syscall is waiting on fd 0. + inputWaiting: false, + } + } catch (error: unknown) { + if (error instanceof CommandExitError && this.topLevelExited) return undefined + throw error + } + } + + /** @inheritdoc */ + async signalForeground(signal: SubprocessTerminalSignal): Promise { + const foreground = await this.inspectForeground() + if (foreground === undefined) { + throw new Error(`subprocess-e2b: cannot resolve foreground process group for terminal ${this.pid}`) + } + if (signal === 'SIGKILL' && foreground.processGroupId === this.pid) { + throw new Error('refusing to SIGKILL the terminal shell; terminate the terminal session instead') + } + await this.sandbox.commands.run(`kill -${signal.slice(3)} -- -${foreground.processGroupId}`) + return foreground.processGroupId + } + + /** @inheritdoc */ + terminate(): void { + this.termination ??= this.closeOnce().catch((error: unknown) => { + this.termination = undefined + throw error + }) + void this.termination.catch(() => {}) + } + + /** @inheritdoc */ + async waitForExit(signal?: AbortSignal): Promise { + const quiescence = this.termination ?? this.done.then( + () => { this.terminate(); return this.termination }, + () => { this.terminate(); return this.termination }, + ) + if (signal === undefined) { + await quiescence + return true + } + if (signal.aborted) return false + return await new Promise((resolve, reject) => { + const onAbort = (): void => { cleanup(); resolve(false) } + const cleanup = (): void => { signal.removeEventListener('abort', onAbort) } + signal.addEventListener('abort', onAbort, { once: true }) + void quiescence.then( + () => { cleanup(); resolve(true) }, + (error: unknown) => { + cleanup() + reject(error instanceof Error ? error : new Error(String(error))) + }, + ) + }) + } + + private async waitForCommand(): Promise { + try { + const result = await this.completion + return { exitCode: result.exitCode, signal: null } + } catch (error: unknown) { + if (error instanceof CommandExitError) { + const signal = this.terminationSignal ?? commandSignal(error.exitCode) + return signal === null ? { exitCode: error.exitCode, signal: null } : { exitCode: null, signal } + } + this.output.destroy(error instanceof Error ? error : new Error(String(error))) + throw error + } finally { + this.topLevelExited = true + if (!this.output.destroyed) this.output.end() + } + } + + private async sessionProcessGroups(): Promise { + const result = await this.sandbox.commands.run( + `ps -eo sid=,pgid= | awk '$1 == ${this.sessionId} { print $2 }'`, + ) + const groups = new Set() + for (const raw of result.stdout.trim().split(/\s+/)) { + if (raw.length === 0) continue + const group = parsePositiveId( + raw, + `subprocess-e2b: invalid process group ${JSON.stringify(raw)} in terminal session ${this.sessionId}`, + ) + if (group <= 1) { + throw new Error(`subprocess-e2b: unsafe process group ${group} in terminal session ${this.sessionId}`) + } + groups.add(group) + } + return [...groups] + } + + private async signalGroups(groups: number[], signal: 'TERM' | 'KILL'): Promise { + if (groups.length === 0) return + try { + await this.sandbox.commands.run(`kill -${signal} -- ${groups.map(group => `-${group}`).join(' ')}`) + } catch (error: unknown) { + if (!(error instanceof CommandExitError)) throw error + } + } + + private async awaitSessionEmpty(kill = false): Promise { + const deadline = Date.now() + this.graceMs + for (;;) { + const groups = await this.sessionProcessGroups() + if (groups.length === 0 || Date.now() >= deadline) return groups + if (kill) await this.signalGroups(groups, 'KILL') + await delay(Math.min(POLL_MS, Math.max(1, deadline - Date.now()))) + } + } + + private async closeOnce(): Promise { + let groups = await this.sessionProcessGroups() + if (groups.length > 0) { + this.terminationSignal = 'SIGTERM' + await this.signalGroups(groups, 'TERM') + groups = await this.awaitSessionEmpty() + } + if (groups.length === 0 && !this.topLevelExited) { + await Promise.race([this.done.catch(() => undefined), delay(this.graceMs)]) + } + if (groups.length > 0 || !this.topLevelExited) { + this.terminationSignal = 'SIGKILL' + if (!this.topLevelExited) await this.sandbox.pty.kill(this.pid) + groups = await this.awaitSessionEmpty(true) + if (!this.topLevelExited) await Promise.race([this.done.catch(() => undefined), delay(this.graceMs)]) + } + if (groups.length > 0) { + throw new Error(`subprocess-e2b: terminal cleanup failed; surviving process groups: ${groups.join(', ')}`) + } + if (!this.topLevelExited) { + throw new Error(`subprocess-e2b: terminal cleanup failed; surviving pid: ${this.pid}`) + } + this.removeAbort?.() + this.removeAbort = undefined + await this.handle.disconnect() + await this.sandbox.files.remove(this.stateDir).catch(() => {}) + } +} + +/** + * Allocate an E2B PTY, replace its bootstrap shell with the requested argv, + * and return only after the private runner has published readiness. + * @param runtime - Shared E2B sandbox owner. + * @param spec - Fully specified terminal-process request. + * @param stateDir - Private remote directory for one startup transaction. + * @returns The live subprocess terminal handle. + */ +export async function spawnE2BTerminal( + runtime: E2BSandboxService, + spec: SubprocessTerminalSpawnSpec, + stateDir: string, +): Promise { + const sandbox = await runtime.getSandbox() + spec.signal?.throwIfAborted() + const paths: TerminalPaths = { + runner: posix.join(stateDir, 'runner.bash'), + environment: posix.join(stateDir, 'environment'), + argv: posix.join(stateDir, 'argv'), + ready: posix.join(stateDir, 'ready'), + } + const ambient = await sandbox.commands.run('env -0', signalOpts(spec.signal)) + const environment = remoteEnvironment(ambient.stdout, spec.env) + const argv = serializeValues(spec.argv, 'argv') + await sandbox.files.makeDir(stateDir) + await sandbox.commands.run(`chmod 700 -- ${quoteE2BShellArg(stateDir)}`, signalOpts(spec.signal)) + await sandbox.files.write([ + { path: paths.runner, data: TERMINAL_RUNNER_SOURCE }, + { path: paths.environment, data: environment }, + { path: paths.argv, data: argv }, + ], signalOpts(spec.signal)) + await sandbox.commands.run( + `chmod 600 -- ${quoteE2BShellArg(paths.runner)} ${quoteE2BShellArg(paths.environment)} ${quoteE2BShellArg(paths.argv)}`, + signalOpts(spec.signal), + ) + + const output = new PassThrough() + let handle: CommandHandle | undefined + let completion: Promise | undefined + try { + handle = await sandbox.pty.create({ + rows: spec.rows, + cols: spec.cols, + cwd: spec.cwd, + envs: { TERM: 'dumb' }, + timeoutMs: 0, + ...signalOpts(spec.signal), + onData: (data) => { if (!output.destroyed) output.write(Buffer.from(data)) }, + }) + completion = handle.wait() + void completion.catch(() => {}) + if (!Number.isSafeInteger(handle.pid) || handle.pid <= 0) { + throw new Error(`subprocess-e2b: E2B returned invalid terminal pid ${handle.pid}`) + } + const command = `exec /bin/bash ${quoteE2BShellArg(paths.runner)} ${quoteE2BShellArg(stateDir)}\r` + await sandbox.pty.sendInput(handle.pid, Buffer.from(command), signalOpts(spec.signal)) + await waitUntilReady(sandbox, paths, completion, spec.signal) + const sessionId = await terminalSessionId(sandbox, handle.pid, spec.signal) + return new E2BTerminalHandle( + sandbox, + handle, + output, + completion, + sessionId, + stateDir, + spec.graceMs, + spec.signal, + ) + } catch (error: unknown) { + output.destroy() + if (handle !== undefined) await handle.kill().catch(() => false) + if (completion !== undefined) await completion.catch(() => {}) + await sandbox.files.remove(stateDir).catch(() => {}) + throw error + } +} diff --git a/packages/e2b/subprocess-e2b/tests/terminal.spec.ts b/packages/e2b/subprocess-e2b/tests/terminal.spec.ts new file mode 100644 index 0000000000..9022e3d213 --- /dev/null +++ b/packages/e2b/subprocess-e2b/tests/terminal.spec.ts @@ -0,0 +1,541 @@ +import { Buffer } from 'node:buffer' +import { once } from 'node:events' +import { Context } from 'cordis' +import { describe, expect, it } from 'vitest' +import { + CommandExitError, + FileNotFoundError, + type CommandHandle, + type CommandResult, + type Sandbox, +} from '@deepseek-ai/dsh-e2b' +import type E2BSandboxService from '@deepseek-ai/dsh-e2b' +import type { SubprocessTerminalSpawnSpec } from '@deepseek-ai/dsh-subprocess' +import E2BSubprocessService from '@deepseek-ai/dsh-subprocess-e2b' +import { spawnE2BTerminal } from '../src/terminal.ts' + +function commandError(exitCode: number): CommandExitError { + return new CommandExitError({ exitCode, stdout: '', stderr: '', error: `exit ${exitCode}` }) +} + +class FakeTerminalCommandHandle { + pid = 123 + disconnects = 0 + sdkKills = 0 + disconnectError: unknown + sdkKillError: unknown + private readonly result = Promise.withResolvers() + private settled = false + + wait(): Promise { + return this.result.promise + } + + async disconnect(): Promise { + this.disconnects += 1 + if (this.disconnectError !== undefined) throw this.disconnectError + } + + async kill(): Promise { + this.sdkKills += 1 + if (this.sdkKillError !== undefined) { + const error = this.sdkKillError + this.fail(137) + throw error + } + this.fail(137) + return true + } + + succeed(exitCode = 0): void { + if (this.settled) return + this.settled = true + this.result.resolve({ exitCode, stdout: '', stderr: '' }) + } + + fail(exitCode: number): void { + if (this.settled) return + this.settled = true + this.result.reject(commandError(exitCode)) + } + + crash(error: unknown): void { + if (this.settled) return + this.settled = true + this.result.reject(error) + } + + asHandle(): CommandHandle { + return this as unknown as CommandHandle + } +} + +class FakeTerminalSandbox { + readonly handle = new FakeTerminalCommandHandle() + readonly commands: string[] = [] + readonly inputs: Array<{ pid: number; data: Buffer }> = [] + readonly removed: string[] = [] + readonly directories: string[] = [] + readonly writes = new Map() + createOptions: Parameters[0] | undefined + ambient = 'KEEP=visible\0NPM_TOKEN=secret\0DSH_STALE=old\0BROKEN\0=bad\0' + ready: string | Error = 'ready\n' + sessionId = '123\n' + foreground = '456\n' + groups = [123] + createError: unknown + sendError: unknown + commandFailure: unknown + foregroundFailure: unknown + termFailure: unknown + removeError: unknown + clearOnTerm = true + clearOnKill = true + settleOnPtyKill = true + ptyKills = 0 + resolvedExecutable = '/usr/bin/node\n' + + readonly sandbox = { + files: { + makeDir: async (path: string): Promise => { + this.directories.push(path) + return true + }, + write: async (files: Array<{ path: string; data: string }>): Promise => { + for (const file of files) this.writes.set(file.path, file.data) + return files.map(() => ({})) + }, + read: async (): Promise => { + if (this.ready instanceof Error) throw this.ready + return this.ready + }, + remove: async (path: string): Promise => { + this.removed.push(path) + if (this.removeError !== undefined) throw this.removeError + }, + }, + commands: { + run: async (command: string, options?: { signal?: AbortSignal }): Promise => { + this.commands.push(command) + options?.signal?.throwIfAborted() + if (this.commandFailure !== undefined) { + const error = this.commandFailure + this.commandFailure = undefined + throw error + } + if (command === 'env -0') return { exitCode: 0, stdout: this.ambient, stderr: '' } + if (command.includes('command -v -- ')) { + return { exitCode: 0, stdout: this.resolvedExecutable, stderr: '' } + } + if (command.startsWith('ps -o sid=')) return { exitCode: 0, stdout: this.sessionId, stderr: '' } + if (command.startsWith('ps -o tpgid=')) { + if (this.foregroundFailure !== undefined) throw this.foregroundFailure + return { exitCode: 0, stdout: this.foreground, stderr: '' } + } + if (command.startsWith('ps -eo sid=')) { + return { exitCode: 0, stdout: this.groups.map(group => `${group}\n`).join(''), stderr: '' } + } + if (command.startsWith('kill -TERM -- ')) { + if (this.termFailure !== undefined) throw this.termFailure + if (this.clearOnTerm) { + this.groups = [] + this.handle.fail(143) + } + } + if (command.startsWith('kill -KILL -- ') && this.clearOnKill) this.groups = [] + return { exitCode: 0, stdout: '', stderr: '' } + }, + }, + pty: { + create: async (options: Parameters[0]): Promise => { + this.createOptions = options + if (this.createError !== undefined) throw this.createError + await options.onData(Buffer.from('buffered banner\n')) + return this.handle.asHandle() + }, + sendInput: async (pid: number, data: Uint8Array, options?: { signal?: AbortSignal }): Promise => { + options?.signal?.throwIfAborted() + this.inputs.push({ pid, data: Buffer.from(data) }) + if (this.sendError !== undefined) throw this.sendError + }, + kill: async (pid: number): Promise => { + this.ptyKills += 1 + if (this.settleOnPtyKill) this.handle.fail(137) + return pid === this.handle.pid + }, + }, + } as unknown as Sandbox +} + +function runtime(fake: FakeTerminalSandbox): E2BSandboxService { + return { + cwd: '/workspace', + runtimeRoot: '/workspace/.dsh-e2b', + disposeMode: 'kill', + getSandbox: async () => fake.sandbox, + } as unknown as E2BSandboxService +} + +function spec(overrides: Partial = {}): SubprocessTerminalSpawnSpec { + return { + argv: ['/bin/bash', '--noprofile', '--norc'], + cwd: '/workspace', + rows: 24, + cols: 80, + graceMs: 5, + env: { TERM: 'dumb', DSH_SESSION_ID: 'owner', TOKEN_EXPLICIT: 'kept' }, + ...overrides, + } +} + +describe('E2B terminal allocation', () => { + it('boots the requested argv through a private runner and preserves buffered bytes', async () => { + const fake = new FakeTerminalSandbox() + const terminal = await spawnE2BTerminal(runtime(fake), spec(), '/runtime/terminal-one') + let output = '' + terminal.output.on('data', (chunk) => { output += String(chunk) }) + await new Promise(resolve => setTimeout(resolve, 0)) + + expect(output).toBe('buffered banner\n') + expect(fake.createOptions).toMatchObject({ rows: 24, cols: 80, cwd: '/workspace', timeoutMs: 0, envs: { TERM: 'dumb' } }) + expect(fake.inputs[0]?.data.toString()).toContain("exec /bin/bash '/runtime/terminal-one/runner.bash'") + expect(fake.writes.get('/runtime/terminal-one/environment')).toContain('KEEP=visible\0') + expect(fake.writes.get('/runtime/terminal-one/environment')).toContain('TOKEN_EXPLICIT=kept\0') + expect(fake.writes.get('/runtime/terminal-one/environment')).not.toContain('secret') + expect(fake.writes.get('/runtime/terminal-one/environment')).not.toContain('DSH_STALE') + expect(fake.writes.get('/runtime/terminal-one/argv')).toBe('/bin/bash\0--noprofile\0--norc\0') + const runner = fake.writes.get('/runtime/terminal-one/runner.bash') ?? '' + expect(runner).toContain('if (( ${#dsh_argv[@]} == 0 )); then') + expect(runner).toContain('exec env -i "${dsh_env[@]}" "${dsh_argv[@]}"') + expect(runner).not.toContain('\u007f') + + await terminal.write(Buffer.from('echo ok\r')) + expect(fake.inputs.at(-1)?.data.toString()).toBe('echo ok\r') + await expect(terminal.inspectForeground()).resolves.toEqual({ processGroupId: 456, inputWaiting: false }) + await expect(terminal.signalForeground('SIGINT')).resolves.toBe(456) + expect(fake.commands).toContain('kill -INT -- -456') + + terminal.terminate() + await expect(terminal.done).resolves.toEqual({ exitCode: null, signal: 'SIGTERM' }) + await expect(terminal.waitForExit()).resolves.toBe(true) + expect(fake.handle.disconnects).toBe(1) + expect(fake.removed).toContain('/runtime/terminal-one') + }) + + it('inherits only safe ambient values and binds live abort to terminal cleanup', async () => { + const fake = new FakeTerminalSandbox() + const controller = new AbortController() + const terminal = await spawnE2BTerminal( + runtime(fake), + spec({ env: undefined, signal: controller.signal }), + '/runtime/abort-live', + ) + const environment = fake.writes.get('/runtime/abort-live/environment') ?? '' + expect(environment).toContain('KEEP=visible\0') + expect(environment).not.toContain('secret') + expect(environment).not.toContain('DSH_STALE') + + controller.abort(new Error('stop')) + await expect(terminal.done).resolves.toEqual({ exitCode: null, signal: 'SIGTERM' }) + await expect(terminal.waitForExit(controller.signal)).resolves.toBe(false) + await expect(terminal.waitForExit()).resolves.toBe(true) + }) + + it('rejects malformed environment and argv values before PTY allocation', async () => { + const invalidName = new FakeTerminalSandbox() + await expect(spawnE2BTerminal(runtime(invalidName), spec({ env: { 'BAD=NAME': 'x' } }), '/runtime/name')) + .rejects.toThrow('environment entries') + expect(invalidName.createOptions).toBeUndefined() + + const invalidValue = new FakeTerminalSandbox() + await expect(spawnE2BTerminal(runtime(invalidValue), spec({ env: { BAD: 'x\0y' } }), '/runtime/value')) + .rejects.toThrow('environment entries') + + const invalidArg = new FakeTerminalSandbox() + await expect(spawnE2BTerminal(runtime(invalidArg), spec({ argv: ['/bin/bash', 'x\0y'] }), '/runtime/argv')) + .rejects.toThrow('argv must not contain NUL') + }) + + it('cleans malformed handles, bootstrap failures, and readiness failures', async () => { + const invalidPid = new FakeTerminalSandbox() + invalidPid.handle.pid = 0 + await expect(spawnE2BTerminal(runtime(invalidPid), spec(), '/runtime/invalid-pid')) + .rejects.toThrow('invalid terminal pid 0') + expect(invalidPid.handle.sdkKills).toBe(1) + expect(invalidPid.removed).toContain('/runtime/invalid-pid') + + const failedInput = new FakeTerminalSandbox() + failedInput.sendError = new Error('bootstrap failed') + await expect(spawnE2BTerminal(runtime(failedInput), spec(), '/runtime/input')) + .rejects.toThrow('bootstrap failed') + expect(failedInput.handle.sdkKills).toBe(1) + + const exited = new FakeTerminalSandbox() + exited.ready = new FileNotFoundError('not ready') + queueMicrotask(() => { exited.handle.succeed(0) }) + await expect(spawnE2BTerminal(runtime(exited), spec(), '/runtime/exited')) + .rejects.toThrow('exited before publishing readiness') + + const invalidSession = new FakeTerminalSandbox() + invalidSession.sessionId = 'not-a-session\n' + await expect(spawnE2BTerminal(runtime(invalidSession), spec(), '/runtime/session')) + .rejects.toThrow('cannot resolve process session') + expect(invalidSession.handle.sdkKills).toBe(1) + + const cleanupFailed = new FakeTerminalSandbox() + cleanupFailed.handle.pid = 0 + cleanupFailed.handle.sdkKillError = new Error('kill transport failed') + cleanupFailed.removeError = new Error('remove transport failed') + await expect(spawnE2BTerminal(runtime(cleanupFailed), spec(), '/runtime/cleanup-failed')) + .rejects.toThrow('invalid terminal pid 0') + }) + + it('propagates setup cancellation and provider failures', async () => { + const aborted = new FakeTerminalSandbox() + await expect(spawnE2BTerminal(runtime(aborted), spec({ signal: AbortSignal.abort(new Error('stop')) }), '/runtime/abort')) + .rejects.toThrow('stop') + + const createFailed = new FakeTerminalSandbox() + createFailed.createError = new Error('create failed') + await expect(spawnE2BTerminal(runtime(createFailed), spec(), '/runtime/create')) + .rejects.toThrow('create failed') + + const readFailed = new FakeTerminalSandbox() + readFailed.ready = new Error('ready transport failed') + await expect(spawnE2BTerminal(runtime(readFailed), spec(), '/runtime/read')) + .rejects.toThrow('ready transport failed') + }) +}) + +describe('E2B terminal lifecycle', () => { + it('maps ordinary exits, closes output, and reports an absent foreground after exit', async () => { + const fake = new FakeTerminalSandbox() + fake.groups = [] + const terminal = await spawnE2BTerminal(runtime(fake), spec(), '/runtime/natural') + terminal.output.resume() + const ended = once(terminal.output, 'end') + fake.handle.succeed(7) + await expect(terminal.done).resolves.toEqual({ exitCode: 7, signal: null }) + await ended + await expect(terminal.waitForExit()).resolves.toBe(true) + await expect(terminal.write(Buffer.from('late'))).rejects.toThrow('exited') + fake.foregroundFailure = commandError(1) + await expect(terminal.inspectForeground()).resolves.toBeUndefined() + }) + + it.each([ + [7, { exitCode: 7, signal: null }], + [143, { exitCode: null, signal: 'SIGTERM' }], + [255, { exitCode: 255, signal: null }], + ] as const)('classifies an unrequested command exit %i', async (exitCode, expected) => { + const fake = new FakeTerminalSandbox() + fake.groups = [] + const terminal = await spawnE2BTerminal(runtime(fake), spec(), `/runtime/exit-${exitCode}`) + fake.handle.fail(exitCode) + await expect(terminal.done).resolves.toEqual(expected) + await expect(terminal.waitForExit(new AbortController().signal)).resolves.toBe(true) + }) + + it('lets an early quiescence observer follow a transport rejection', async () => { + const fake = new FakeTerminalSandbox() + fake.groups = [] + const terminal = await spawnE2BTerminal(runtime(fake), spec(), '/runtime/early-observer') + terminal.output.on('error', () => {}) + const quiescence = terminal.waitForExit() + fake.handle.crash(new Error('transport failed')) + await expect(terminal.done).rejects.toThrow('transport failed') + await expect(quiescence).resolves.toBe(true) + }) + + it('rejects killing the terminal shell and propagates live foreground failures', async () => { + const fake = new FakeTerminalSandbox() + fake.foreground = '123\n' + const terminal = await spawnE2BTerminal(runtime(fake), spec(), '/runtime/signal') + await expect(terminal.signalForeground('SIGKILL')).rejects.toThrow('refusing to SIGKILL') + fake.foreground = 'invalid\n' + await expect(terminal.inspectForeground()).rejects.toThrow('cannot resolve foreground') + fake.foregroundFailure = commandError(1) + await expect(terminal.inspectForeground()).rejects.toBeInstanceOf(CommandExitError) + fake.clearOnTerm = true + terminal.terminate() + await terminal.waitForExit() + }) + + it('escalates surviving process groups and bounds an observing wait', async () => { + const fake = new FakeTerminalSandbox() + fake.groups = [123, 456] + fake.clearOnTerm = false + const terminal = await spawnE2BTerminal(runtime(fake), spec({ graceMs: 1 }), '/runtime/escalate') + const controller = new AbortController() + const observing = terminal.waitForExit(controller.signal) + controller.abort() + await expect(observing).resolves.toBe(false) + + terminal.terminate() + await expect(terminal.done).resolves.toEqual({ exitCode: null, signal: 'SIGKILL' }) + await expect(terminal.waitForExit()).resolves.toBe(true) + expect(fake.commands).toContain('kill -TERM -- -123 -456') + expect(fake.commands).toContain('kill -KILL -- -123 -456') + }) + + it('surfaces cleanup failures and allows a later retry', async () => { + const fake = new FakeTerminalSandbox() + fake.groups = [1] + const terminal = await spawnE2BTerminal(runtime(fake), spec({ graceMs: 1 }), '/runtime/retry') + terminal.terminate() + await expect(terminal.waitForExit(new AbortController().signal)).rejects.toThrow('unsafe process group 1') + + fake.groups = [] + fake.handle.succeed(0) + await terminal.done + terminal.terminate() + await expect(terminal.waitForExit()).resolves.toBe(true) + }) + + it('propagates a process-group signalling transport failure before retry', async () => { + const fake = new FakeTerminalSandbox() + fake.termFailure = new Error('signal transport failed') + const terminal = await spawnE2BTerminal(runtime(fake), spec({ graceMs: 1 }), '/runtime/signal-failure') + terminal.terminate() + await expect(terminal.waitForExit()).rejects.toThrow('signal transport failed') + + fake.groups = [] + fake.handle.succeed(0) + await terminal.done + terminal.terminate() + await expect(terminal.waitForExit()).resolves.toBe(true) + }) + + it('keeps command rejection authoritative while cleanup is already waiting', async () => { + const fake = new FakeTerminalSandbox() + fake.groups = [] + fake.removeError = new Error('private state already gone') + const terminal = await spawnE2BTerminal(runtime(fake), spec(), '/runtime/reject-during-cleanup') + terminal.output.on('error', () => {}) + terminal.terminate() + await Promise.resolve() + fake.handle.crash(new Error('command transport failed')) + await expect(terminal.done).rejects.toThrow('command transport failed') + await expect(terminal.waitForExit()).resolves.toBe(true) + }) + + it('keeps a late command rejection authoritative after PTY kill', async () => { + const fake = new FakeTerminalSandbox() + fake.groups = [] + fake.settleOnPtyKill = false + const terminal = await spawnE2BTerminal(runtime(fake), spec({ graceMs: 1 }), '/runtime/reject-after-kill') + terminal.output.on('error', () => {}) + terminal.terminate() + while (fake.ptyKills === 0) await new Promise(resolve => setTimeout(resolve, 0)) + await Promise.resolve() + fake.handle.crash(new Error('late command transport failed')) + await expect(terminal.done).rejects.toThrow('late command transport failed') + await expect(terminal.waitForExit()).resolves.toBe(true) + }) + + it('reports surviving groups, a surviving top-level pid, and transport failure', async () => { + const survivor = new FakeTerminalSandbox() + survivor.clearOnTerm = false + survivor.clearOnKill = false + const terminal = await spawnE2BTerminal(runtime(survivor), spec({ graceMs: 1 }), '/runtime/survivor') + terminal.terminate() + await expect(terminal.waitForExit()).rejects.toThrow('surviving process groups: 123') + + const livePid = new FakeTerminalSandbox() + livePid.groups = [] + livePid.settleOnPtyKill = false + const live = await spawnE2BTerminal(runtime(livePid), spec({ graceMs: 1 }), '/runtime/live-pid') + live.terminate() + await expect(live.waitForExit()).rejects.toThrow('surviving pid: 123') + livePid.handle.succeed(0) + await live.done + + const crashed = new FakeTerminalSandbox() + crashed.groups = [] + const failed = await spawnE2BTerminal(runtime(crashed), spec(), '/runtime/crashed') + const outputError = once(failed.output, 'error') + crashed.handle.crash('transport gone') + await expect(failed.done).rejects.toEqual('transport gone') + await expect(outputError).resolves.toMatchObject([{ message: 'transport gone' }]) + await expect(failed.waitForExit()).resolves.toBe(true) + }) +}) + +describe('E2B subprocess terminal service', () => { + async function service(fake = new FakeTerminalSandbox()): Promise<{ + ctx: Context + fiber: Awaited> + fake: FakeTerminalSandbox + }> { + const ctx = new Context() + ctx.provide('e2b', runtime(fake)) + const fiber = await ctx.plugin(E2BSubprocessService) + return { ctx, fiber, fake } + } + + it('publishes execution-world coordinates and resolves remote executables', async () => { + const { ctx } = await service() + expect(ctx.subprocess.cwd).toBe('/workspace') + expect(ctx.subprocess.runtimeRoot).toBe('/workspace/.dsh-e2b') + await expect(ctx.subprocess.resolveExecutable('/bin/bash')).resolves.toBe('/bin/bash') + await expect(ctx.subprocess.resolveExecutable('node', { PATH: '/custom/bin' }, new AbortController().signal)) + .resolves.toBe('/usr/bin/node') + expect((ctx.e2b)).toBeDefined() + }) + + it('rejects invalid executable lookup inputs and results', async () => { + const { ctx, fake } = await service() + await expect(ctx.subprocess.resolveExecutable('')).rejects.toThrow('non-empty') + await expect(ctx.subprocess.resolveExecutable('node', undefined, AbortSignal.abort(new Error('stop')))) + .rejects.toThrow('stop') + fake.resolvedExecutable = 'relative/node\n' + await expect(ctx.subprocess.resolveExecutable('node')).rejects.toThrow('did not resolve') + fake.resolvedExecutable = '/one\n/two\n' + await expect(ctx.subprocess.resolveExecutable('node')).rejects.toThrow('did not resolve') + }) + + it('owns live terminals through service disposal', async () => { + const { ctx, fiber, fake } = await service() + const terminal = await ctx.subprocess.spawnTerminal(spec()) + await fiber.dispose() + await expect(terminal.done).resolves.toEqual({ exitCode: null, signal: 'SIGTERM' }) + expect(fake.handle.disconnects).toBe(1) + }) + + it('releases naturally settled terminals and validates terminal requests', async () => { + const { ctx, fiber, fake } = await service() + for (const request of [ + spec({ argv: [] }), + spec({ rows: 0 }), + spec({ cols: 1.5 }), + spec({ graceMs: 0 }), + spec({ signal: AbortSignal.abort(new Error('cancelled')) }), + ]) { + await expect(ctx.subprocess.spawnTerminal(request)).rejects.toThrow() + } + + fake.groups = [] + const terminal = await ctx.subprocess.spawnTerminal(spec()) + fake.handle.succeed(0) + await terminal.done + await terminal.waitForExit() + const signals = fake.commands.filter(command => command.startsWith('kill -')).length + await fiber.dispose() + expect(fake.commands.filter(command => command.startsWith('kill -'))).toHaveLength(signals) + }) + + it('contains a failed automatic terminal release until service disposal retries it', async () => { + const { fiber, fake } = await service() + fake.clearOnTerm = false + fake.clearOnKill = false + const terminal = await (fiber.ctx).subprocess.spawnTerminal(spec({ graceMs: 1 })) + fake.handle.succeed(0) + await terminal.done + await new Promise(resolve => setTimeout(resolve, 10)) + expect(fake.commands).toContain('kill -KILL -- -123') + + fake.groups = [] + await fiber.dispose() + await expect(terminal.waitForExit()).resolves.toBe(true) + }) +}) diff --git a/packages/fs/README.i18n.yaml b/packages/fs/README.i18n.yaml index 6b5a5d8b76..ae44721735 100644 --- a/packages/fs/README.i18n.yaml +++ b/packages/fs/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/fs/README.md -README.md: 0b9a244d0c42542909460edf7109dfb9cee34503 -README.zh.md: 77da9b73f866fd88fcb81b28cbb622b0389bf5f0 +README.md: 96bbe6c95bd2aca7e66cf2d57abb3f056cd19390 +README.zh.md: 7f091b4f0955b847d21b8b3423dab421e610cda3 diff --git a/packages/lsp/lsp-local/tests/fixture-server.ts b/packages/lsp/lsp-local/tests/fixture-server.ts index 89053894ee..9399b5a1a7 100644 --- a/packages/lsp/lsp-local/tests/fixture-server.ts +++ b/packages/lsp/lsp-local/tests/fixture-server.ts @@ -15,7 +15,6 @@ * - LSP_FAKE_REPLY_DELAY_MS: delays each textDocument/* response by this many milliseconds. * - LSP_FAKE_OPEN_MARKER: appends each didOpen document text as one JSON line to this path. * - LSP_FAKE_INITIALIZED_MARKER: records when the initialized notification is received. - * - LSP_FAKE_EXPECT_PROCESS_ID: expected JSON `initialize.processId`; mismatch exits nonzero. * - LSP_FAKE_PAUSE_STDIN_AFTER_INITIALIZED: "1" stops consuming stdin after initialized. * - LSP_FAKE_EXIT_DELAY_MS / LSP_FAKE_EXIT_MARKER: delay protocol exit and record exit/termination. * - LSP_FAKE_NO_SHUTDOWN: "1" ignores the shutdown request (forces kill escalation). @@ -38,7 +37,6 @@ const exitAfterReply = process.env.LSP_FAKE_EXIT_AFTER_REPLY === '1' const replyDelayMs = Number(process.env.LSP_FAKE_REPLY_DELAY_MS ?? 0) const openMarker = process.env.LSP_FAKE_OPEN_MARKER const initializedMarker = process.env.LSP_FAKE_INITIALIZED_MARKER -const expectedProcessId = process.env.LSP_FAKE_EXPECT_PROCESS_ID const pauseStdinAfterInitialized = process.env.LSP_FAKE_PAUSE_STDIN_AFTER_INITIALIZED === '1' const exitDelayMs = Number(process.env.LSP_FAKE_EXIT_DELAY_MS ?? 0) const exitMarker = process.env.LSP_FAKE_EXIT_MARKER @@ -104,10 +102,6 @@ function handle(message: { id?: number; method?: string; params?: unknown; resul return } if (method === 'initialize') { - if (expectedProcessId !== undefined) { - const params = message.params as { processId?: unknown } | undefined - if (JSON.stringify(params?.processId) !== expectedProcessId) process.exit(2) - } if (garbage) process.stdout.write('this is not a framed message\r\n') send({ id, diff --git a/packages/lsp/lsp-local/tests/instance.spec.ts b/packages/lsp/lsp-local/tests/instance.spec.ts index fc1d05843c..22877a94c8 100644 --- a/packages/lsp/lsp-local/tests/instance.spec.ts +++ b/packages/lsp/lsp-local/tests/instance.spec.ts @@ -111,11 +111,6 @@ const RESPONDING_SERVER = const locJson = () => JSON.stringify({ uri: pathToFileURL(join(ws, 'a.ts')).href, range: { start: { line: 0, character: 0 }, end: { line: 0, character: 3 } } }) describe('LspInstance server-request handling', () => { - it('advertises a null process id across process namespaces', async () => { - const instance = makeInstance({ LSP_FAKE_EXPECT_PROCESS_ID: 'null', LSP_FAKE_DEF: 'null' }, { clientProcessId: null }) - await expect(run(instance, 'goToDefinition')).resolves.toMatchObject({ kind: 'locations' }) - }) - it('answers workspace/configuration with the static config per item', async () => { const instance = makeInstance({ LSP_FAKE_ON_OPEN: 'configuration', LSP_FAKE_DEF: locJson() }) // The query drives didOpen, which makes the fake emit workspace/configuration; a healthy answer diff --git a/packages/lsp/lsp/README.i18n.yaml b/packages/lsp/lsp/README.i18n.yaml index b672a405de..e63252c7f8 100644 --- a/packages/lsp/lsp/README.i18n.yaml +++ b/packages/lsp/lsp/README.i18n.yaml @@ -1,6 +1,6 @@ # Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: -# pnpm run verify-translation-pairing --write packages/lsp/lsp/README.md -README.md: bcffd460f192b0e6ac214ad0bc0d43da4c81d68c -README.zh.md: e9df352e999834235a66ef7a500e300dadc2029c +# pnpm run verify-translation-pairing --write +README.md: f96fc67ec8cb95f423eff9b312b7b591ec9d3008 +README.zh.md: 13ae9700e284ff238147538a571622066efc5747 diff --git a/packages/lsp/lsp/README.md b/packages/lsp/lsp/README.md index 4dc0436bbb..5c1044be50 100644 --- a/packages/lsp/lsp/README.md +++ b/packages/lsp/lsp/README.md @@ -10,7 +10,6 @@ This package is the interface third of the LSP capability: |---|---| | `@deepseek-ai/dsh-lsp` (this) | the interface: the service, provider registry keyed by branded id + extension mapping, per-query selection, request/result vocabulary, the `LspError` taxonomy | | `@deepseek-ai/dsh-lsp-local` | a generic local backend that registers configured stdio language-server providers | -| `@deepseek-ai/dsh-lsp-e2b` | a generic E2B backend that registers configured remote stdio providers | | `@deepseek-ai/dsh-tool-lsp` | the model-facing `lsp` tool over `ctx.lsp` | The seam exposes exactly four semantic operations — `goToDefinition`, `findReferences`, `goToImplementation`, `hover` — and no generic JSON-RPC escape hatch, so no protocol payload or unreviewed command/mutation reaches a provider through `ctx.lsp`. diff --git a/packages/lsp/lsp/README.zh.md b/packages/lsp/lsp/README.zh.md index fdaedc9a96..cc412333e6 100644 --- a/packages/lsp/lsp/README.zh.md +++ b/packages/lsp/lsp/README.zh.md @@ -10,7 +10,6 @@ |---|---| | `@deepseek-ai/dsh-lsp`(本包) | 接口:服务、以品牌化 id + 扩展名映射为 key 的提供方注册表、逐查询选择、请求/结果词汇、`LspError` 分类体系 | | `@deepseek-ai/dsh-lsp-local` | 通用本地后端,注册已配置的 stdio 语言服务器提供方 | -| `@deepseek-ai/dsh-lsp-e2b` | 通用 E2B 后端,注册已配置的远程 stdio 提供方 | | `@deepseek-ai/dsh-tool-lsp` | 面向模型的 `lsp` 工具,基于 `ctx.lsp` | 该 seam 恰好公开四种语义操作:`goToDefinition`、`findReferences`、`goToImplementation`、`hover`,且没有通用 JSON-RPC 逃生口,因此任何协议载荷或未经评审的命令/修改都无法通过 `ctx.lsp` 到达提供方。 diff --git a/packages/pty/pty/src/index.ts b/packages/pty/pty/src/index.ts index 5cb375fbca..8ff4168e1a 100644 --- a/packages/pty/pty/src/index.ts +++ b/packages/pty/pty/src/index.ts @@ -41,15 +41,6 @@ export type { PtyWaitReason, } from './types.ts' export { PtyBackendCleanupError } from './types.ts' -export { - normalizePtyTerminalText, - PTY_PROMPT_MARKER_PREFIX, - PtyTerminalSanitizer, - PtyTextBuffer, - ptySignalName, - ptyUtf8Tail, -} from './terminal.ts' -export type { PtySanitizedChunk } from './terminal.ts' /** Opaque identity minted by {@link PtyService} for one live PTY session. */ export type PtySessionId = PtySessionIdValue diff --git a/packages/pty/pty/src/terminal.ts b/packages/pty/pty/src/terminal.ts deleted file mode 100644 index ed005dda41..0000000000 --- a/packages/pty/pty/src/terminal.ts +++ /dev/null @@ -1,273 +0,0 @@ -/** Backend-neutral line-oriented terminal buffering and control-sequence sanitization. */ - -import { Buffer } from 'node:buffer' -import { constants } from 'node:os' -import type { PtySendRead } from './types.ts' - -/** OSC marker emitted by a controlled bash before each prompt. */ -export const PTY_PROMPT_MARKER_PREFIX = '133;D;' - -/** One sanitized chunk plus whether it contained the controlled prompt marker. */ -export interface PtySanitizedChunk { - /** Printable, line-normalized terminal text. */ - text: string - /** Whether the chunk completed the controlled prompt marker. */ - prompt: boolean - /** Present when printable text followed the latest controlled prompt marker. */ - promptText?: true -} - -/** - * Return the largest code-point-aligned UTF-8 tail within a byte cap. - * @param text - Candidate terminal text. - * @param maxBytes - Maximum retained UTF-8 bytes. - * @returns The retained tail and whether its head was dropped. - */ -export function ptyUtf8Tail(text: string, maxBytes: number): { text: string; truncated: boolean } { - if (Buffer.byteLength(text) <= maxBytes) return { text, truncated: false } - const chars = Array.from(text) - let bytes = 0 - let start = chars.length - while (start > 0) { - const next = Buffer.byteLength(chars[start - 1] as string) - if (bytes + next > maxBytes) break - bytes += next - start -= 1 - } - return { text: chars.slice(start).join(''), truncated: true } -} - -/** UTF-8 and optionally line-bounded terminal text buffer. */ -export class PtyTextBuffer { - private value = '' - private dropped = false - - /** - * @param maxBytes - Maximum retained UTF-8 bytes. - * @param maxLines - Optional maximum retained logical lines. - */ - constructor( - private readonly maxBytes: number, - private readonly maxLines?: number, - ) {} - - /** - * Append terminal text and drop the oldest excess. - * @param text - Decoded and sanitized terminal text. - */ - append(text: string): void { - if (text.length === 0) return - this.value += text - if (this.maxLines !== undefined) { - const lines = this.value.split('\n') - if (lines.length > this.maxLines) { - this.value = lines.slice(lines.length - this.maxLines).join('\n') - this.dropped = true - } - } - const tail = ptyUtf8Tail(this.value, this.maxBytes) - this.value = tail.text - this.dropped ||= tail.truncated - } - - /** - * Consume all currently retained operation text. - * @returns The delta and whether older text was dropped. - */ - consume(): PtySendRead { - const delta = this.value - const truncated = this.dropped - this.value = '' - this.dropped = false - return { delta, truncated } - } - - /** - * Return the retained text without consuming it. - * @returns The retained text and whether its head was dropped. - */ - snapshot(): { text: string; truncated: boolean } { - return { text: this.value, truncated: this.dropped } - } -} - -/** - * Streaming terminal-control sanitizer for line-oriented PTY backends. - * Full terminal emulation is deliberately outside the PTY seam. - */ -export class PtyTerminalSanitizer { - private pending = '' - private discardMode: 'osc' | 'csi' | undefined - private discardOscEscape = false - private trailingCarriageReturn = false - private awaitingPromptText = false - - /** @param maxPendingBytes - Bound for an incomplete terminal-control sequence. */ - constructor(private readonly maxPendingBytes: number) {} - - /** - * Consume one decoded PTY data chunk. - * @param chunk - Decoded terminal data. - * @returns Printable text and prompt-marker facts. - */ - push(chunk: string): PtySanitizedChunk { - this.pending += this.discardPrefix(chunk) - let text = '' - let prompt = false - let promptText = false - let index = 0 - const appendText = (value: string): boolean => { - text += value - if (this.awaitingPromptText && value.replace(/[\r\n\x07]/g, '').length > 0) { - this.awaitingPromptText = false - return true - } - return false - } - while (index < this.pending.length) { - const escape = this.pending.indexOf('\x1b', index) - if (escape < 0) { - promptText = appendText(this.pending.slice(index)) || promptText - index = this.pending.length - break - } - promptText = appendText(this.pending.slice(index, escape)) || promptText - if (escape + 1 >= this.pending.length) { - index = escape - break - } - const kind = this.pending[escape + 1] - if (kind === ']') { - const bel = this.pending.indexOf('\x07', escape + 2) - const stringTerminator = this.pending.indexOf('\x1b\\', escape + 2) - let end = -1 - if (bel >= 0 && stringTerminator >= 0) end = Math.min(bel + 1, stringTerminator + 2) - else if (bel >= 0) end = bel + 1 - else if (stringTerminator >= 0) end = stringTerminator + 2 - if (end < 0) { - index = escape - break - } - const terminatorBytes = this.pending[end - 1] === '\x07' ? 1 : 2 - const content = this.pending.slice(escape + 2, end - terminatorBytes) - if (content.startsWith(PTY_PROMPT_MARKER_PREFIX)) { - prompt = true - promptText = false - this.awaitingPromptText = true - } - index = end - continue - } - if (kind === '[') { - let end = escape + 2 - while (end < this.pending.length) { - const code = this.pending.charCodeAt(end) - if (code >= 0x40 && code <= 0x7e) break - end += 1 - } - if (end >= this.pending.length) { - index = escape - break - } - index = end + 1 - continue - } - index = escape + 2 - } - this.pending = this.pending.slice(index) - this.enforcePendingBound() - return { text: this.normalizeText(text), prompt, ...promptText ? { promptText: true } : {} } - } - - /** - * Flush printable trailing data and discard incomplete controls. - * @returns Remaining normalized printable text. - */ - flush(): string { - const text = this.pending.startsWith('\x1b') ? '' : this.pending - this.pending = '' - this.discardMode = undefined - this.discardOscEscape = false - this.awaitingPromptText = false - const normalized = this.normalizeText(text) - if (!this.trailingCarriageReturn) return normalized - this.trailingCarriageReturn = false - return `${normalized}\n` - } - - private normalizeText(text: string): string { - let complete = this.trailingCarriageReturn ? `\r${text}` : text - this.trailingCarriageReturn = false - if (complete.endsWith('\r')) { - complete = complete.slice(0, -1) - this.trailingCarriageReturn = true - } - return normalizePtyTerminalText(complete) - } - - private enforcePendingBound(): void { - if (Buffer.byteLength(this.pending) <= this.maxPendingBytes) return - this.discardMode = this.pending[1] === ']' ? 'osc' : 'csi' - this.pending = '' - } - - private discardPrefix(chunk: string): string { - if (this.discardMode === undefined) return chunk - if (this.discardMode === 'csi') { - for (let index = 0; index < chunk.length; index += 1) { - const code = chunk.charCodeAt(index) - if (code >= 0x40 && code <= 0x7e) { - this.discardMode = undefined - return chunk.slice(index + 1) - } - } - return '' - } - - let index = 0 - if (this.discardOscEscape) { - this.discardOscEscape = false - if (chunk.startsWith('\\')) { - this.discardMode = undefined - return chunk.slice(1) - } - } - while (index < chunk.length) { - if (chunk[index] === '\x07') { - this.discardMode = undefined - return chunk.slice(index + 1) - } - if (chunk[index] === '\x1b') { - if (chunk[index + 1] === '\\') { - this.discardMode = undefined - return chunk.slice(index + 2) - } - if (index + 1 === chunk.length) this.discardOscEscape = true - } - index += 1 - } - return '' - } -} - -/** - * Normalize CRLF and standalone carriage returns for line-oriented rendering. - * @param text - Sanitized terminal text. - * @returns Line-normalized text with BEL removed. - */ -export function normalizePtyTerminalText(text: string): string { - return text.replaceAll('\r\n', '\n').replaceAll('\r', '\n').replaceAll('\x07', '') -} - -/** - * Convert a platform signal number into the seam's signal-name vocabulary. - * @param number - Platform signal number, zero, or an absent signal. - * @returns The matching Node signal name, or `null` when unknown or absent. - */ -export function ptySignalName(number: number | undefined): NodeJS.Signals | null { - if (number === undefined || number === 0) return null - for (const [name, value] of Object.entries(constants.signals)) { - if (value === number) return name as NodeJS.Signals - } - return null -} diff --git a/scripts/verify-package-readme-model-experience.ts b/scripts/verify-package-readme-model-experience.ts index 5ad70c0e68..6fda3901b6 100644 --- a/scripts/verify-package-readme-model-experience.ts +++ b/scripts/verify-package-readme-model-experience.ts @@ -47,7 +47,6 @@ const SENTENCE_MODEL_EXPERIENCE: Readonly> = { 'packages/bash/bash-local': { kind: 'indirect', reason: 'The executor backend delegates model rendering to dsh-tool-bash.' }, 'packages/bash/pwsh-local': { kind: 'indirect', reason: 'The executor backend delegates model rendering to dsh-tool-pwsh.' }, 'packages/code-runtime/code-runtime': { kind: 'indirect', reason: 'The service interface delegates model rendering to Code Mode in dsh-tools.' }, - 'packages/e2b/code-runtime-e2b': { kind: 'indirect', reason: 'The E2B backend delegates model rendering to Code Mode in dsh-tools.' }, 'packages/code-runtime/code-runtime-worker': { kind: 'indirect', reason: 'The worker backend delegates model rendering to Code Mode in dsh-tools.' }, 'packages/typert/registry': { kind: 'none', reason: 'Runtime type registry; consumers (cordis_inspect, wire faces, gates) own any model-visible projection of registry contents.' }, 'packages/typert/loader': { kind: 'none', reason: 'Loader integration only registers generated artifacts; consumers own any model-visible projection.' }, @@ -99,7 +98,6 @@ const SENTENCE_MODEL_EXPERIENCE: Readonly> = { 'packages/llm/llm': { kind: 'none', reason: 'The adapter registry forwards already-assembled requests unchanged.' }, 'packages/llm/token-meter': { kind: 'indirect', reason: 'The measurement service leaves model-visible changes to its consumers.' }, 'packages/lsp/lsp': { kind: 'indirect', reason: 'The provider registry delegates model rendering to dsh-tool-lsp.' }, - 'packages/e2b/lsp-e2b': { kind: 'indirect', reason: 'The E2B provider backend delegates model rendering to dsh-tool-lsp.' }, 'packages/lsp/lsp-local': { kind: 'indirect', reason: 'The provider backend delegates model rendering to dsh-tool-lsp.' }, 'packages/subprocess/subprocess': { kind: 'indirect', reason: 'The seam delegates all model rendering to consumer seams such as the bash executor family.' }, 'packages/e2b/subprocess-e2b': { kind: 'indirect', reason: 'The remote spawn backend delegates model rendering to consumer seams such as the bash executor family.' }, diff --git a/tsconfig.host.json b/tsconfig.host.json index 526d4ef4b8..101c810c91 100644 --- a/tsconfig.host.json +++ b/tsconfig.host.json @@ -166,12 +166,10 @@ { "path": "./packages/e2b/subprocess-e2b" }, { "path": "./packages/bash/bash" }, { "path": "./packages/pty/pty" }, - { "path": "./packages/e2b/pty-e2b" }, { "path": "./packages/pty/pty-local" }, { "path": "./packages/pty/tool-bash-persistent" }, { "path": "./packages/pty/tool-pty" }, { "path": "./packages/code-runtime/code-runtime" }, - { "path": "./packages/e2b/code-runtime-e2b" }, { "path": "./packages/code-runtime/code-runtime-worker" }, { "path": "./packages/llm/llm-deepseek" }, { "path": "./packages/llm/llm-pi-ai" }, @@ -268,7 +266,6 @@ { "path": "./packages/sdk/create-sdk" }, { "path": "./packages/sdk/telemetry" }, { "path": "./packages/lsp/lsp" }, - { "path": "./packages/e2b/lsp-e2b" }, { "path": "./packages/lsp/lsp-local" }, { "path": "./packages/lsp/tool-lsp" }, { "path": "./apps/cli" } From 2188cb908c5f9e71efd7ad1333f3f46b8d4b44ae Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 03:32:29 +0800 Subject: [PATCH 073/108] refactor(e2b): delegate terminal lifecycle --- docs/cordis-catalog/services.md | 4 +- packages/e2b/subprocess-e2b/src/terminal.ts | 48 ++++------------- .../e2b/subprocess-e2b/tests/terminal.spec.ts | 53 ++++++++++++++++++- 3 files changed, 63 insertions(+), 42 deletions(-) diff --git a/docs/cordis-catalog/services.md b/docs/cordis-catalog/services.md index 33f357d2bf..7ce45c6817 100644 --- a/docs/cordis-catalog/services.md +++ b/docs/cordis-catalog/services.md @@ -591,7 +591,7 @@ Owns one lazily consumable E2B SDK handle and its final kill/pause/leave decisio async getSandbox(): Promise ``` -Source: [`packages/e2b/e2b/src/index.ts:119`](../../packages/e2b/e2b/src/index.ts) +Source: [`packages/e2b/e2b/src/index.ts:97`](../../packages/e2b/e2b/src/index.ts) ## `ctx.fs` — `FileSystem` (abstract seam) @@ -1180,7 +1180,7 @@ list(owner: Agent): PtySessionSnapshot[] Types: [Agent](../core-data-structures/core.md) · [PtyBackend](../core-data-structures/pty.md) · [PtyReadRequest](../core-data-structures/pty.md) · [PtyReadResult](../core-data-structures/pty.md) · [PtySendOperation](../core-data-structures/pty.md) · [PtySendRequest](../core-data-structures/pty.md) · [PtySessionId](../core-data-structures/pty.md) · [PtySessionSnapshot](../core-data-structures/pty.md) · [PtySignal](../core-data-structures/pty.md) · [PtySignalResult](../core-data-structures/pty.md) · [PtySpawnRequest](../core-data-structures/pty.md) · [PtySpawnResult](../core-data-structures/pty.md) -Source: [`packages/pty/pty/src/index.ts:114`](../../packages/pty/pty/src/index.ts) +Source: [`packages/pty/pty/src/index.ts:105`](../../packages/pty/pty/src/index.ts) ## `ctx.sandbox` — `SandboxProvider` (abstract seam) diff --git a/packages/e2b/subprocess-e2b/src/terminal.ts b/packages/e2b/subprocess-e2b/src/terminal.ts index 88bea44fde..1a916c0fd5 100644 --- a/packages/e2b/subprocess-e2b/src/terminal.ts +++ b/packages/e2b/subprocess-e2b/src/terminal.ts @@ -10,7 +10,7 @@ import { quoteE2BShellArg, } from '@deepseek-ai/dsh-e2b' import type { CommandHandle, CommandResult, Sandbox } from '@deepseek-ai/dsh-e2b' -import { SENSITIVE_ENV_PATTERN } from '@deepseek-ai/dsh-subprocess' +import { SENSITIVE_ENV_PATTERN, SubprocessTerminalLifecycle } from '@deepseek-ai/dsh-subprocess' import type { SubprocessOutcome, SubprocessTerminalForeground, @@ -127,9 +127,8 @@ export class E2BTerminalHandle implements SubprocessTerminalHandle { readonly done: Promise private topLevelExited = false - private termination: Promise | undefined + private readonly lifecycle: SubprocessTerminalLifecycle private terminationSignal: NodeJS.Signals | null = null - private removeAbort: (() => void) | undefined constructor( private readonly sandbox: Sandbox, @@ -143,13 +142,11 @@ export class E2BTerminalHandle implements SubprocessTerminalHandle { ) { this.pid = handle.pid this.done = this.waitForCommand() - void this.done.then(() => { this.terminate() }, () => { this.terminate() }) - if (signal !== undefined) { - const onAbort = (): void => { this.terminate() } - signal.addEventListener('abort', onAbort, { once: true }) - this.removeAbort = () => { signal.removeEventListener('abort', onAbort) } - if (signal.aborted) this.terminate() - } + this.lifecycle = new SubprocessTerminalLifecycle({ + done: this.done, + cleanup: () => this.closeOnce(), + signal, + }) } /** @inheritdoc */ @@ -192,36 +189,12 @@ export class E2BTerminalHandle implements SubprocessTerminalHandle { /** @inheritdoc */ terminate(): void { - this.termination ??= this.closeOnce().catch((error: unknown) => { - this.termination = undefined - throw error - }) - void this.termination.catch(() => {}) + this.lifecycle.terminate() } /** @inheritdoc */ async waitForExit(signal?: AbortSignal): Promise { - const quiescence = this.termination ?? this.done.then( - () => { this.terminate(); return this.termination }, - () => { this.terminate(); return this.termination }, - ) - if (signal === undefined) { - await quiescence - return true - } - if (signal.aborted) return false - return await new Promise((resolve, reject) => { - const onAbort = (): void => { cleanup(); resolve(false) } - const cleanup = (): void => { signal.removeEventListener('abort', onAbort) } - signal.addEventListener('abort', onAbort, { once: true }) - void quiescence.then( - () => { cleanup(); resolve(true) }, - (error: unknown) => { - cleanup() - reject(error instanceof Error ? error : new Error(String(error))) - }, - ) - }) + return await this.lifecycle.waitForExit(signal) } private async waitForCommand(): Promise { @@ -261,7 +234,6 @@ export class E2BTerminalHandle implements SubprocessTerminalHandle { } private async signalGroups(groups: number[], signal: 'TERM' | 'KILL'): Promise { - if (groups.length === 0) return try { await this.sandbox.commands.run(`kill -${signal} -- ${groups.map(group => `-${group}`).join(' ')}`) } catch (error: unknown) { @@ -301,8 +273,6 @@ export class E2BTerminalHandle implements SubprocessTerminalHandle { if (!this.topLevelExited) { throw new Error(`subprocess-e2b: terminal cleanup failed; surviving pid: ${this.pid}`) } - this.removeAbort?.() - this.removeAbort = undefined await this.handle.disconnect() await this.sandbox.files.remove(this.stateDir).catch(() => {}) } diff --git a/packages/e2b/subprocess-e2b/tests/terminal.spec.ts b/packages/e2b/subprocess-e2b/tests/terminal.spec.ts index 9022e3d213..43cb88ea87 100644 --- a/packages/e2b/subprocess-e2b/tests/terminal.spec.ts +++ b/packages/e2b/subprocess-e2b/tests/terminal.spec.ts @@ -1,5 +1,6 @@ import { Buffer } from 'node:buffer' import { once } from 'node:events' +import { PassThrough } from 'node:stream' import { Context } from 'cordis' import { describe, expect, it } from 'vitest' import { @@ -12,7 +13,7 @@ import { import type E2BSandboxService from '@deepseek-ai/dsh-e2b' import type { SubprocessTerminalSpawnSpec } from '@deepseek-ai/dsh-subprocess' import E2BSubprocessService from '@deepseek-ai/dsh-subprocess-e2b' -import { spawnE2BTerminal } from '../src/terminal.ts' +import { E2BTerminalHandle, spawnE2BTerminal } from '../src/terminal.ts' function commandError(exitCode: number): CommandExitError { return new CommandExitError({ exitCode, stdout: '', stderr: '', error: `exit ${exitCode}` }) @@ -80,6 +81,7 @@ class FakeTerminalSandbox { createOptions: Parameters[0] | undefined ambient = 'KEEP=visible\0NPM_TOKEN=secret\0DSH_STALE=old\0BROKEN\0=bad\0' ready: string | Error = 'ready\n' + readyMisses = 0 sessionId = '123\n' foreground = '456\n' groups = [123] @@ -106,6 +108,10 @@ class FakeTerminalSandbox { return files.map(() => ({})) }, read: async (): Promise => { + if (this.readyMisses > 0) { + this.readyMisses -= 1 + throw new FileNotFoundError('not ready') + } if (this.ready instanceof Error) throw this.ready return this.ready }, @@ -191,6 +197,7 @@ function spec(overrides: Partial = {}): SubprocessT describe('E2B terminal allocation', () => { it('boots the requested argv through a private runner and preserves buffered bytes', async () => { const fake = new FakeTerminalSandbox() + fake.readyMisses = 1 const terminal = await spawnE2BTerminal(runtime(fake), spec(), '/runtime/terminal-one') let output = '' terminal.output.on('data', (chunk) => { output += String(chunk) }) @@ -281,6 +288,9 @@ describe('E2B terminal allocation', () => { await expect(spawnE2BTerminal(runtime(invalidSession), spec(), '/runtime/session')) .rejects.toThrow('cannot resolve process session') expect(invalidSession.handle.sdkKills).toBe(1) + const lateData = invalidSession.createOptions?.onData + if (lateData === undefined) throw new Error('missing captured terminal callback') + expect(lateData(Buffer.from('late bytes'))).toBeUndefined() const cleanupFailed = new FakeTerminalSandbox() cleanupFailed.handle.pid = 0 @@ -321,6 +331,26 @@ describe('E2B terminal lifecycle', () => { await expect(terminal.write(Buffer.from('late'))).rejects.toThrow('exited') fake.foregroundFailure = commandError(1) await expect(terminal.inspectForeground()).resolves.toBeUndefined() + await expect(terminal.signalForeground('SIGINT')).rejects.toThrow('cannot resolve foreground process group') + }) + + it('starts cleanup when the lifetime signal is already aborted at handle publication', async () => { + const fake = new FakeTerminalSandbox() + const controller = new AbortController() + controller.abort(new Error('publication cancelled')) + const terminal = new E2BTerminalHandle( + fake.sandbox, + fake.handle.asHandle(), + new PassThrough(), + fake.handle.wait(), + 123, + '/runtime/pre-aborted', + 1, + controller.signal, + ) + + await expect(terminal.done).resolves.toEqual({ exitCode: null, signal: 'SIGTERM' }) + await expect(terminal.waitForExit()).resolves.toBe(true) }) it.each([ @@ -404,6 +434,27 @@ describe('E2B terminal lifecycle', () => { await terminal.done terminal.terminate() await expect(terminal.waitForExit()).resolves.toBe(true) + + const alreadyExited = new FakeTerminalSandbox() + alreadyExited.termFailure = commandError(1) + const tolerant = await spawnE2BTerminal(runtime(alreadyExited), spec({ graceMs: 1 }), '/runtime/group-exited') + tolerant.terminate() + await expect(tolerant.done).resolves.toEqual({ exitCode: null, signal: 'SIGKILL' }) + await expect(tolerant.waitForExit()).resolves.toBe(true) + }) + + it('normalizes a non-Error cleanup rejection for an observing wait', async () => { + const fake = new FakeTerminalSandbox() + const terminal = await spawnE2BTerminal(runtime(fake), spec(), '/runtime/non-error-cleanup') + fake.commandFailure = 'cleanup transport gone' + terminal.terminate() + await expect(terminal.waitForExit(new AbortController().signal)).rejects.toThrow('cleanup transport gone') + + fake.groups = [] + fake.handle.succeed(0) + await terminal.done + terminal.terminate() + await expect(terminal.waitForExit()).resolves.toBe(true) }) it('keeps command rejection authoritative while cleanup is already waiting', async () => { From 8877f5d58239806d28173410275e50f9ea477384 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 03:55:03 +0800 Subject: [PATCH 074/108] fix(e2b): normalize relative executable paths --- packages/e2b/e2b/tests/composition.e2e.ts | 5 ++++- packages/e2b/subprocess-e2b/src/index.ts | 6 +++--- .../e2b/subprocess-e2b/tests/terminal.spec.ts | 17 ++++++++++++++--- 3 files changed, 21 insertions(+), 7 deletions(-) diff --git a/packages/e2b/e2b/tests/composition.e2e.ts b/packages/e2b/e2b/tests/composition.e2e.ts index 98fcdeae7f..e87627af8e 100644 --- a/packages/e2b/e2b/tests/composition.e2e.ts +++ b/packages/e2b/e2b/tests/composition.e2e.ts @@ -1,5 +1,5 @@ import { access } from 'node:fs/promises' -import { join } from 'node:path' +import { join, posix } from 'node:path' import { fileURLToPath } from 'node:url' import { Context } from 'cordis' import { describe, expect, it } from 'vitest' @@ -40,6 +40,9 @@ describe.skipIf(!process.env.E2B_API_KEY)('E2B live Loader composition', () => { } as never) const ptyFiber = await ctx.plugin(PtyService) const subprocessFiber = await ctx.plugin(E2BSubprocessService) + const node = await ctx.subprocess.resolveExecutable('node') + const relativeNodePath = posix.relative(ctx.subprocess.cwd, posix.dirname(node)) || '.' + await expect(ctx.subprocess.resolveExecutable('node', { PATH: relativeNodePath })).resolves.toBe(node) const ownerId = SessionId('e2b-pty-env-owner') const owner: Agent = { id: ownerId, diff --git a/packages/e2b/subprocess-e2b/src/index.ts b/packages/e2b/subprocess-e2b/src/index.ts index 1547e1b46e..86ca5ceada 100644 --- a/packages/e2b/subprocess-e2b/src/index.ts +++ b/packages/e2b/subprocess-e2b/src/index.ts @@ -79,14 +79,14 @@ export class E2BSubprocessService extends SubprocessService { const prefix = path === undefined ? '' : `PATH=${quoteE2BShellArg(path)} ` const result = await sandbox.commands.run( `${prefix}command -v -- ${quoteE2BShellArg(command)}`, - signalOpts(signal), + { cwd: this.cwd, ...signalOpts(signal) }, ) signal?.throwIfAborted() const executable = result.stdout.trim() - if (!posix.isAbsolute(executable) || executable.includes('\n')) { + if (executable.includes('\n') || (!posix.isAbsolute(executable) && !executable.includes('/'))) { throw new Error(`subprocess-e2b: executable ${JSON.stringify(command)} did not resolve to one absolute path`) } - return executable + return posix.resolve(this.cwd, executable) } /** @inheritdoc */ diff --git a/packages/e2b/subprocess-e2b/tests/terminal.spec.ts b/packages/e2b/subprocess-e2b/tests/terminal.spec.ts index 43cb88ea87..81716c3d52 100644 --- a/packages/e2b/subprocess-e2b/tests/terminal.spec.ts +++ b/packages/e2b/subprocess-e2b/tests/terminal.spec.ts @@ -19,6 +19,11 @@ function commandError(exitCode: number): CommandExitError { return new CommandExitError({ exitCode, stdout: '', stderr: '', error: `exit ${exitCode}` }) } +interface CommandOptions { + signal?: AbortSignal + cwd?: string +} + class FakeTerminalCommandHandle { pid = 123 disconnects = 0 @@ -74,6 +79,7 @@ class FakeTerminalCommandHandle { class FakeTerminalSandbox { readonly handle = new FakeTerminalCommandHandle() readonly commands: string[] = [] + readonly commandOptions: CommandOptions[] = [] readonly inputs: Array<{ pid: number; data: Buffer }> = [] readonly removed: string[] = [] readonly directories: string[] = [] @@ -121,8 +127,9 @@ class FakeTerminalSandbox { }, }, commands: { - run: async (command: string, options?: { signal?: AbortSignal }): Promise => { + run: async (command: string, options?: CommandOptions): Promise => { this.commands.push(command) + if (options !== undefined) this.commandOptions.push(options) options?.signal?.throwIfAborted() if (this.commandFailure !== undefined) { const error = this.commandFailure @@ -525,12 +532,16 @@ describe('E2B subprocess terminal service', () => { } it('publishes execution-world coordinates and resolves remote executables', async () => { - const { ctx } = await service() + const { ctx, fake } = await service() expect(ctx.subprocess.cwd).toBe('/workspace') expect(ctx.subprocess.runtimeRoot).toBe('/workspace/.dsh-e2b') await expect(ctx.subprocess.resolveExecutable('/bin/bash')).resolves.toBe('/bin/bash') await expect(ctx.subprocess.resolveExecutable('node', { PATH: '/custom/bin' }, new AbortController().signal)) .resolves.toBe('/usr/bin/node') + fake.resolvedExecutable = 'tools/bin/node\n' + await expect(ctx.subprocess.resolveExecutable('node', { PATH: 'tools/bin' })) + .resolves.toBe('/workspace/tools/bin/node') + expect(fake.commandOptions.at(-1)).toMatchObject({ cwd: '/workspace' }) expect((ctx.e2b)).toBeDefined() }) @@ -539,7 +550,7 @@ describe('E2B subprocess terminal service', () => { await expect(ctx.subprocess.resolveExecutable('')).rejects.toThrow('non-empty') await expect(ctx.subprocess.resolveExecutable('node', undefined, AbortSignal.abort(new Error('stop')))) .rejects.toThrow('stop') - fake.resolvedExecutable = 'relative/node\n' + fake.resolvedExecutable = 'node\n' await expect(ctx.subprocess.resolveExecutable('node')).rejects.toThrow('did not resolve') fake.resolvedExecutable = '/one\n/two\n' await expect(ctx.subprocess.resolveExecutable('node')).rejects.toThrow('did not resolve') From fd78d9bc58cc324c38fe1ac8b882558da152edd9 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 05:31:22 +0800 Subject: [PATCH 075/108] fix(e2b): harden remote process lifecycle --- ...026-07-27-e2b-remote-runtime-poc.i18n.yaml | 4 +- .../2026-07-27-e2b-remote-runtime-poc.md | 6 +- .../2026-07-27-e2b-remote-runtime-poc.zh.md | 6 +- .../tests/fixtures/e2b/e2b/bin.ts | 14 ++ packages/e2b/e2b/README.i18n.yaml | 4 +- packages/e2b/e2b/README.md | 2 +- packages/e2b/e2b/README.zh.md | 2 +- packages/e2b/e2b/src/index.ts | 26 ++- packages/e2b/e2b/tests/composition.e2e.ts | 17 ++ packages/e2b/e2b/tests/e2b.spec.ts | 27 +++ packages/e2b/subprocess-e2b/README.i18n.yaml | 4 +- packages/e2b/subprocess-e2b/README.md | 9 +- packages/e2b/subprocess-e2b/README.zh.md | 9 +- packages/e2b/subprocess-e2b/src/output.ts | 68 +++++- packages/e2b/subprocess-e2b/src/process.ts | 137 ++++++++---- packages/e2b/subprocess-e2b/src/terminal.ts | 201 +++++++++++++----- .../subprocess-e2b/tests/subprocess.spec.ts | 136 ++++++++++-- .../e2b/subprocess-e2b/tests/terminal.spec.ts | 67 +++++- 18 files changed, 586 insertions(+), 153 deletions(-) diff --git a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.i18n.yaml b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.i18n.yaml index fc187ec47b..b5e8e6e47c 100644 --- a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.i18n.yaml +++ b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md -2026-07-27-e2b-remote-runtime-poc.md: df0d6be97c502a9afe1a7aff3656a7567d302704 -2026-07-27-e2b-remote-runtime-poc.zh.md: 111d7aa63f8f7e51a82cfa0e42c489901f2b88e4 +2026-07-27-e2b-remote-runtime-poc.md: 029950cb2dae47bcf1ce819156135e58d46a4bb6 +2026-07-27-e2b-remote-runtime-poc.zh.md: 4029cc85d206e0dd0b7dbd91ec03674062df8531 diff --git a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md index df0d6be97c..029950cb2d 100644 --- a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md +++ b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md @@ -28,7 +28,7 @@ E2B owns the mutable filesystem, managed command and Bash processes, terminal al The host owns Cordis and plugin objects, the agent loop, agent/session/goal state, session logs and persistence, LLM calls, prompts and tools, authority decisions, skills, subagent orchestration, PTY buffers and readiness state, LSP JSON-RPC ids/queues/protocol state, Code Runtime type stripping/output accounting/binding dispatch, and E2B SDK/network orchestration. The overlay does not upload, mount, or synchronize the host workspace; identical cwd strings name independent host and remote directories. -The fundamental adapters carry the substrate-specific mechanics. `dsh-subprocess-e2b` consumes E2B's byte PTY callback directly, retains terminal send identity across asynchronous foreground-group lookup, and owns whole-session cleanup. `dsh-fs-e2b` performs bounded source reads through a dependency-free helper that walks no-follow directory descriptors beneath the canonical target. Generic Code Runtime keeps its controller/worker protocol on validated ASCII/base64 frames and kills the provider-owned process group before inherited pipes drain. Generic LSP uses UTF-8 JSON over command pipes; E2B's decoded callback transport is not an arbitrary binary channel. +The fundamental adapters carry the substrate-specific mechanics. `dsh-subprocess-e2b` consumes E2B's byte PTY callback directly, retains terminal send identity across asynchronous foreground-group lookup, and owns whole-session cleanup. For ordinary command pipes, its remote wrapper branches raw spill bytes before framing each live chunk as newline-delimited base64 ASCII; the host restores raw bytes across arbitrary decoded SDK callback boundaries. `dsh-fs-e2b` performs bounded source reads through a dependency-free helper that walks no-follow directory descriptors beneath the canonical target. Generic Code Runtime keeps its controller/worker protocol on validated ASCII/base64 frames and kills the provider-owned process group before inherited pipes drain. Generic LSP uses byte-faithful UTF-8 JSON over command pipes. Retaining a sandbox preserves remote files and unmanaged remote state only. Reconnect does not reconstruct host PTY sessions, buffers, process handles, LSP connections or requests, code workers, binding calls, timers, output cursors, or locks. Managed groups terminate and join when their provider disposes before the shared owner pauses, leaves, or kills the sandbox. @@ -38,7 +38,7 @@ The POC has no session-persistence backend, template builder, volume, snapshot, Focused package suites pin owner lifecycle cleanup, filesystem paths/containment/bounded descriptor reads and commit metadata, subprocess executable lookup/process groups/publication rollback, terminal byte I/O/signal identity/default-environment scrubbing/session cleanup, output limits, abort ordering, disposal to quiescence, and package-owned invariant registrations. The generic PTY, LSP, and subprocess Code Runtime suites pin their provider-neutral readiness, cross-namespace `processId`, binding bridge, descriptor isolation, hostile traffic, and worker/descendant cleanup behavior. -A credential-gated Loader composition creates real E2B sandboxes and exercises FS-to-Bash and Bash-to-FS visibility, process-publication rollback, bounded spill output, PTY default-secret scrubbing, stale-interrupt identity, and process-tree cleanup, parent-swap-safe bounded LSP source reads, Code Runtime host bindings, descriptor-isolated output accounting, descendant-held pipe cleanup, wall timeout, abort, runner cleanup, host-workspace isolation, and final sandbox deletion. The same composition runs through source imports and built package exports. +A credential-gated Loader composition creates real E2B sandboxes and exercises FS-to-Bash and Bash-to-FS visibility, process-publication rollback, byte-split UTF-8 command output, bounded raw spill output, PTY default-secret scrubbing, stale-interrupt identity, and process-tree cleanup, parent-swap-safe bounded LSP source reads, Code Runtime host bindings, descriptor-isolated output accounting, descendant-held pipe cleanup, wall timeout, abort, runner cleanup, host-workspace isolation, and final sandbox deletion. The same composition runs through source imports and built package exports. ## Alternatives considered @@ -62,4 +62,4 @@ A credential-gated Loader composition creates real E2B sandboxes and exercises F The three-package composition demonstrates that filesystem and subprocess are the sufficient provider seams for moving an agent's mutable coding world off-host without changing the loop, higher capability implementations, or model-facing tool packages. Fixes to Bash, PTY, LSP, and Code Runtime remain provider-neutral. `sandboxId` plus pause/leave permits manual remote-file retention for experiments, while kill remains the demo's cleanup policy. -The adapters are not interchangeable with local backends for every consumer: remote startup cannot synchronously expose a PID, E2B retains complete command output in SDK memory, command callbacks are text-decoded rather than arbitrary binary streams, exact terminal stdin-wait inspection is unavailable, signal attribution is partly inferred, and reconnect cannot restore handles or protocol state. PTY uses E2B's byte API; Code Runtime uses validated ASCII/base64 framing; the exercised LSP path carries valid UTF-8 JSON. Remote process/spill artifacts accumulate in a retained sandbox, Code programs share a JavaScript realm with Node worker internals, and a process that deliberately escapes a managed process group or terminal session does not become reconnectable or owned. These gaps remain documented POC constraints rather than compatibility shims or new cross-cutting abstractions. +The adapters are not interchangeable with local backends for every consumer: remote startup cannot synchronously expose a PID, E2B retains the complete base64 command transport in SDK memory, exact terminal stdin-wait inspection is unavailable, E2B supplies no independent signal fact, and reconnect cannot restore handles or protocol state. The adapter reports only its own requested TERM/KILL as signals and preserves every unrequested SDK exit as an exit code. Remote process/spill artifacts accumulate in a retained sandbox, Code programs share a JavaScript realm with Node worker internals, and a process that deliberately escapes a managed process group or terminal session does not become reconnectable or owned. These gaps remain documented POC constraints rather than compatibility shims or new cross-cutting abstractions. diff --git a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.zh.md b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.zh.md index 111d7aa63f..4029cc85d2 100644 --- a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.zh.md +++ b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.zh.md @@ -28,7 +28,7 @@ E2B 拥有可变文件系统、受管命令与 Bash 进程、终端分配与终 宿主拥有 Cordis 与插件对象、agent loop、agent/会话/goal 状态、会话日志及持久化、LLM(大语言模型)调用、提示词与工具、权限决策、skill(技能)、subagent 编排、PTY 缓冲与就绪状态、LSP JSON-RPC id/队列/协议状态、Code Runtime 类型剥离/输出计量/绑定分发,以及 E2B SDK/网络编排。该 overlay 不会上传、挂载或同步宿主工作区;拼写相同的 cwd 字符串分别指向彼此独立的宿主与远程目录。 -基础适配器承载基底专用机制。`dsh-subprocess-e2b` 直接消费 E2B 的字节 PTY 回调,在异步查找前台进程组的过程中保留终端发送身份,并负责全会话清理。`dsh-fs-e2b` 通过无依赖辅助程序执行有界源码读取,该程序会在规范化目标下逐级遍历不跟随符号链接的目录描述符。通用 Code Runtime 通过经过验证的 ASCII/base64 帧承载 controller/worker 协议,并在继承的管道排空前终止提供方拥有的进程组。通用 LSP 通过命令管道使用 UTF-8 JSON;E2B 的已解码回调传输并非任意二进制通道。 +基础适配器承载基底专用机制。`dsh-subprocess-e2b` 直接消费 E2B 的字节 PTY 回调,在异步查找前台进程组的过程中保留终端发送身份,并负责全会话清理。对于普通命令管道,其远程包装层会先分流原始 spill 字节,再把每个实时分片编码为换行分隔的 base64 ASCII 帧;宿主会跨任意已解码 SDK 回调边界恢复原始字节。`dsh-fs-e2b` 通过无依赖辅助程序执行有界源码读取,该程序会在规范化目标下逐级遍历不跟随符号链接的目录描述符。通用 Code Runtime 通过经过验证的 ASCII/base64 帧承载 controller/worker 协议,并在继承的管道排空前终止提供方拥有的进程组。通用 LSP 通过命令管道使用字节保真的 UTF-8 JSON。 保留沙箱只会保存远程文件与未受管的远程状态。重新连接不会重建宿主 PTY 会话、缓冲、进程句柄、LSP 连接或请求、代码 worker、绑定调用、定时器、输出游标或锁。受管进程组会在所属提供方 dispose(资源释放)时终止并等待退出,之后共享所有者才会暂停、脱离或终止沙箱。 @@ -38,7 +38,7 @@ E2B 拥有可变文件系统、受管命令与 Bash 进程、终端分配与终 聚焦包测试套件固定所有者生命周期清理、文件系统路径/containment/有界描述符读取与提交元数据、子进程可执行文件查找/进程组/发布回滚、终端字节 I/O/信号身份/默认环境清理/会话清理、输出上限、中止顺序、等待完全停稳的资源释放,以及包自有不变式注册。通用 PTY、LSP 与子进程 Code Runtime 测试套件固定其提供方无关的就绪判定、跨命名空间 `processId`、绑定桥接、描述符隔离、恶意通信,以及 worker/后代进程清理行为。 -凭据门控的 Loader 组合会创建真实 E2B 沙箱,并演练 FS-to-Bash 与 Bash-to-FS 可见性、进程发布回滚、有界 spill 输出、PTY 默认秘密清理、陈旧中断身份与进程树清理、可抵御父目录替换的有界 LSP 源码读取、Code Runtime 宿主绑定、描述符隔离的输出记账、后代进程所持管道的清理、墙钟超时、中止、runner 清理、宿主工作区隔离,以及最终删除沙箱。同一组合分别通过源代码导入与已构建包导出运行。 +凭据门控的 Loader 组合会创建真实 E2B 沙箱,并演练 FS-to-Bash 与 Bash-to-FS 可见性、进程发布回滚、按字节切分的 UTF-8 命令输出、有界原始 spill 输出、PTY 默认秘密清理、陈旧中断身份与进程树清理、可抵御父目录替换的有界 LSP 源码读取、Code Runtime 宿主绑定、描述符隔离的输出记账、后代进程所持管道的清理、墙钟超时、中止、runner 清理、宿主工作区隔离,以及最终删除沙箱。同一组合分别通过源代码导入与已构建包导出运行。 ## 曾考虑的替代方案 @@ -62,4 +62,4 @@ E2B 拥有可变文件系统、受管命令与 Bash 进程、终端分配与终 这个由 3 个包组成的组合证明,文件系统与进程管理这两个提供方 seam 足以把 agent 的可变 coding 环境移出宿主,而无需改变循环、上层功能实现或面向模型的工具包。Bash、PTY、LSP 与 Code Runtime 的修复仍与提供方无关。`sandboxId` 与 `pause`/`leave` 允许实验手动保留远程文件,演示仍以 `kill` 作为清理策略。 -这些适配器并不能对所有消费方与本地后端互换:远程启动无法同步公开 PID,E2B 会在 SDK 内存中保留完整命令输出,命令回调传递的是已解码文本而非任意二进制流,无法精确检查终端 stdin 等待状态,信号归因部分依靠推断,重新连接也无法恢复句柄或协议状态。PTY 使用 E2B 的字节 API;Code Runtime 使用经过验证的 ASCII/base64 分帧;已演练的 LSP 路径承载有效的 UTF-8 JSON。保留沙箱后会累积远程进程/spill 产物,模型程序与 Node worker 内部机制共享一个 JavaScript realm,有意逃离受管理进程组或终端会话的进程也不会因此变得可重新连接或由该组合管理。这些缺口作为 POC 约束明确记录,而不会引入兼容垫片或新的跨领域抽象。 +这些适配器并不能对所有消费方与本地后端互换:远程启动无法同步公开 PID,E2B 会在 SDK 内存中保留完整的 base64 命令传输,无法精确检查终端 stdin 等待状态,E2B 不提供独立的信号事实,重新连接也无法恢复句柄或协议状态。适配器只会把自己请求的 TERM/KILL 报告为信号,其他未请求的 SDK 退出都保留为退出码。保留沙箱后会累积远程进程/spill 产物,模型程序与 Node worker 内部机制共享一个 JavaScript realm,有意逃离受管理进程组或终端会话的进程也不会因此变得可重新连接或由该组合管理。这些缺口作为 POC 约束明确记录,而不会引入兼容垫片或新的跨领域抽象。 diff --git a/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts b/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts index aeb447b0e8..7c5fce5535 100644 --- a/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts +++ b/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts @@ -72,6 +72,19 @@ try { ].every(entry => environmentLines.has(entry)) if (!explicitEnvironment) throw new Error(`E2B subprocess dropped an explicit environment entry: ${environmentText}`) + const splitUtf8Handle = ctx.subprocess.spawn({ + argv: ['bash', '-c', "printf '\\344'; sleep 0.05; printf '\\275'; sleep 0.05; printf '\\240'; sleep 0.05; printf '\\345'; sleep 0.05; printf '\\245'; sleep 0.05; printf '\\275'"], + cwd: process.cwd(), + stdio: { stdin: 'ignore', stdout: { maxBytes: 32 }, stderr: { maxBytes: 4_096 } }, + graceMs: 500, + env: {}, + }) + const splitUtf8Outcome = await splitUtf8Handle.done + const splitUtf8Output = splitUtf8Handle.collected.stdout?.readFrom(0).text + if (splitUtf8Outcome.exitCode !== 0 || splitUtf8Output !== '你好') { + throw new Error(`E2B subprocess corrupted split UTF-8 output: ${JSON.stringify({ splitUtf8Outcome, splitUtf8Output })}`) + } + const remoteFiles = sandbox.files as unknown as { read(path: string, options?: unknown): Promise } @@ -370,6 +383,7 @@ try { bashRead: bashRead.stdout.text, fsRead, explicitEnvironment, + splitUtf8Output, publicationRollback, spill: { liveBytes: liveSpillBytes, outcome: spillOutcome, read: spillRead }, hover, diff --git a/packages/e2b/e2b/README.i18n.yaml b/packages/e2b/e2b/README.i18n.yaml index 8ffb6e981f..229a9e9343 100644 --- a/packages/e2b/e2b/README.i18n.yaml +++ b/packages/e2b/e2b/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/e2b/e2b/README.md -README.md: 00264b8f0b03e4af8512025322fe3e457e7b6b9b -README.zh.md: 93fad661ded446e78e3addc0c8b2b8fdc39bd994 +README.md: 56606a22c36e65bcc53f2b8cce27f0739da1b8e2 +README.zh.md: 2e3213e431f15f1b22d2b8a429b27ab8a6a78671 diff --git a/packages/e2b/e2b/README.md b/packages/e2b/e2b/README.md index 00264b8f0b..56606a22c3 100644 --- a/packages/e2b/e2b/README.md +++ b/packages/e2b/e2b/README.md @@ -30,7 +30,7 @@ Set `sandboxId` to reconnect a running or paused sandbox instead of creating one Construction starts one create/connect operation. Before resolving `getSandbox()`, the service creates `cwd` and the private `cwd/.dsh-e2b` adapter-state directory, then sets that directory to mode `0700`. `sandboxId` resolves to a branded `E2BSandboxId` after setup. -Disposal first prevents new handle acquisition, then awaits setup and applies exactly one configured disposition. A newly created sandbox is killed when initial directory setup fails; a reconnected sandbox is not killed on setup failure because the service did not create it. Provider plugins must load after this owner and dispose before it. +Disposal first prevents new handle acquisition, then awaits setup and applies exactly one configured disposition. A `SandboxNotFoundError` means a kill-on-timeout sandbox is already quiescent; every other disposition failure rejects teardown. A newly created sandbox is killed when initial directory setup fails; a reconnected sandbox is not killed on setup failure because the service did not create it. Provider plugins must load after this owner and dispose before it. `pause` and `leave` retain remote filesystem and adapter artifacts for a later `sandboxId` connection, but a later harness process receives only a new SDK handle. The subprocess service still fulfills its seam contract by terminating managed groups before owner disposal; neither disposition recovers prior process objects, output cursors, or in-memory adapter locks. diff --git a/packages/e2b/e2b/README.zh.md b/packages/e2b/e2b/README.zh.md index 93fad661de..2e3213e431 100644 --- a/packages/e2b/e2b/README.zh.md +++ b/packages/e2b/e2b/README.zh.md @@ -30,7 +30,7 @@ 构造阶段会启动一次 create/connect 操作。服务在 `getSandbox()` 结算前创建 `cwd` 和私有的 `cwd/.dsh-e2b` 适配器状态目录,再把该目录的 mode 设为 `0700`。初始化完成后,`sandboxId` 会结算为品牌类型 `E2BSandboxId`。 -资源释放会先阻止继续获取新句柄,再等待初始化完成,并且只应用一种已配置的处置方式。新建沙箱的初始目录设置失败时,服务会终止该沙箱;重新连接的沙箱设置失败时不会被终止,因为它不是由本服务创建的。提供方插件必须在该所有者之后加载,并在其之前 dispose(资源释放)。 +资源释放会先阻止继续获取新句柄,再等待初始化完成,并且只应用一种已配置的处置方式。`SandboxNotFoundError` 表示因超时终止的沙箱已经完全停稳;其他处置失败都会使 teardown 拒绝。新建沙箱的初始目录设置失败时,服务会终止该沙箱;重新连接的沙箱设置失败时不会被终止,因为它不是由本服务创建的。提供方插件必须在该所有者之后加载,并在其之前 dispose(资源释放)。 `pause` 和 `leave` 会保留远程文件系统及适配器产物,供稍后的 `sandboxId` 连接使用,但后续 harness 进程只会获得新的 SDK 句柄。进程管理服务仍会履行其 seam 契约,在所有者释放前终止受管进程组;这两种处置方式都不会恢复先前的进程对象、输出游标或内存中的适配器锁。 diff --git a/packages/e2b/e2b/src/index.ts b/packages/e2b/e2b/src/index.ts index 184c49f811..96da912033 100644 --- a/packages/e2b/e2b/src/index.ts +++ b/packages/e2b/e2b/src/index.ts @@ -7,7 +7,7 @@ import { posix } from 'node:path' import { Context, Service } from 'cordis' import z from 'schemastery' -import { Sandbox } from 'e2b' +import { Sandbox, SandboxNotFoundError } from 'e2b' import type { Branded } from '@deepseek-ai/dsh-brand' export { @@ -159,16 +159,22 @@ export class E2BSandboxService extends Service { // there is no remote resource for teardown to own. return } - switch (this.config.onDispose) { - case 'kill': - await sandbox.kill() - return - case 'pause': { - await sandbox.pause() - return + try { + switch (this.config.onDispose) { + case 'kill': + await sandbox.kill() + return + case 'pause': { + await sandbox.pause() + return + } + case 'leave': + return } - case 'leave': - return + } catch (error: unknown) { + // A kill-on-timeout sandbox is already quiescent; every other disposal + // failure still reports that the configured final disposition is unknown. + if (!(error instanceof SandboxNotFoundError)) throw error } }, 'e2b sandbox teardown') } diff --git a/packages/e2b/e2b/tests/composition.e2e.ts b/packages/e2b/e2b/tests/composition.e2e.ts index e87627af8e..4e7be57c45 100644 --- a/packages/e2b/e2b/tests/composition.e2e.ts +++ b/packages/e2b/e2b/tests/composition.e2e.ts @@ -43,6 +43,22 @@ describe.skipIf(!process.env.E2B_API_KEY)('E2B live Loader composition', () => { const node = await ctx.subprocess.resolveExecutable('node') const relativeNodePath = posix.relative(ctx.subprocess.cwd, posix.dirname(node)) || '.' await expect(ctx.subprocess.resolveExecutable('node', { PATH: relativeNodePath })).resolves.toBe(node) + const environmentProbe = ctx.subprocess.spawn({ + argv: ['/bin/bash', '-c', [ + 'dsh_leak=0', + 'for dsh_pid in "$PPID" $(ps -o pid= --ppid "$PPID"); do', + ' [[ "$dsh_pid" == "$$" ]] && continue', + ' if tr "\\0" "\\n" < "/proc/$dsh_pid/environ" 2>/dev/null | grep -Fqx "NPM_TOKEN=sentinel-secret"; then dsh_leak=1; fi', + 'done', + 'printf "DIRECT=<%s> LEAK=<%s>\\n" "${NPM_TOKEN-}" "$dsh_leak"', + ].join('\n')], + cwd: '/home/user', + stdio: { stdin: 'ignore', stdout: { maxBytes: 1_024 }, stderr: { maxBytes: 1_024 } }, + graceMs: 500, + env: {}, + }) + await expect(environmentProbe.done).resolves.toEqual({ exitCode: 0, signal: null }) + expect(environmentProbe.collected.stdout?.readFrom(0).text).toBe('DIRECT=<> LEAK=<0>\n') const ownerId = SessionId('e2b-pty-env-owner') const owner: Agent = { id: ownerId, @@ -106,6 +122,7 @@ describe.skipIf(!process.env.E2B_API_KEY)('E2B live Loader composition', () => { bashRead: 'written-by-fs\n', fsRead: 'written-by-bash\n', explicitEnvironment: true, + splitUtf8Output: '你好', publicationRollback: true, spill: { liveBytes: 6, diff --git a/packages/e2b/e2b/tests/e2b.spec.ts b/packages/e2b/e2b/tests/e2b.spec.ts index c0770f5917..bb96ca806a 100644 --- a/packages/e2b/e2b/tests/e2b.spec.ts +++ b/packages/e2b/e2b/tests/e2b.spec.ts @@ -3,6 +3,7 @@ import { Context } from 'cordis' import type { Sandbox as SandboxType } from 'e2b' import E2BSandboxService, { E2BSandboxId, + SandboxNotFoundError, quoteE2BShellArg, } from '@deepseek-ai/dsh-e2b' import * as E2BInvariant from '../src/invariant.ts' @@ -140,6 +141,32 @@ describe('E2BSandboxService', () => { expect(fixture.pause).toHaveBeenCalledOnce() }) + it('treats a timeout-killed sandbox as already quiescent during disposal', async () => { + const fixture = fakeSandbox() + fixture.pause.mockRejectedValue(new SandboxNotFoundError('sandbox expired')) + sdk.create.mockResolvedValue(fixture.sandbox) + const ctx = new Context() + const fiber = await ctx.plugin(E2BSandboxService, { + apiKey: 'test-key', + onTimeout: 'kill', + onDispose: 'pause', + }) + await ctx.e2b.getSandbox() + await expect(fiber.dispose()).resolves.toBeUndefined() + expect(fixture.pause).toHaveBeenCalledOnce() + }) + + it('does not classify other disposal failures as an already-gone sandbox', async () => { + const fixture = fakeSandbox() + fixture.kill.mockRejectedValue(new Error('disposition unknown')) + sdk.create.mockResolvedValue(fixture.sandbox) + const ctx = new Context() + const fiber = await ctx.plugin(E2BSandboxService, { apiKey: 'test-key' }) + await ctx.e2b.getSandbox() + await expect(fiber.dispose()).resolves.toBeUndefined() + expect(fixture.kill).toHaveBeenCalledOnce() + }) + it('reconnects without applying creation lifecycle options and can leave state running', async () => { const fixture = fakeSandbox('existing') sdk.connect.mockResolvedValue(fixture.sandbox) diff --git a/packages/e2b/subprocess-e2b/README.i18n.yaml b/packages/e2b/subprocess-e2b/README.i18n.yaml index dca7b016c2..33a36141d0 100644 --- a/packages/e2b/subprocess-e2b/README.i18n.yaml +++ b/packages/e2b/subprocess-e2b/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/e2b/subprocess-e2b/README.md -README.md: e0805345e708c67c6de721d95641e2366a7f990e -README.zh.md: f83969c794e50f0658aa86218a0f5f3c89d53b56 +README.md: 58f13b264324d2168d4bc264b60100cdafa0a366 +README.zh.md: d547e4e8d66f629edcef753651180fe1806ad9bf diff --git a/packages/e2b/subprocess-e2b/README.md b/packages/e2b/subprocess-e2b/README.md index e0805345e7..58f13b2643 100644 --- a/packages/e2b/subprocess-e2b/README.md +++ b/packages/e2b/subprocess-e2b/README.md @@ -10,10 +10,10 @@ E2B implementation of the [`@deepseek-ai/dsh-subprocess`](../../subprocess/subpr - **Execution-world coordinates** — `cwd` and private `runtimeRoot` come from the shared owner; executable lookup verifies absolute paths or resolves a bare name against the sandbox PATH plus explicit overrides. - **Linux process groups** — a quoted wrapper starts each argv under `exec setsid --wait` and records its actual process-group id plus private status files beneath `ctx.e2b.runtimeRoot/processes`. The handle waits for that file instead of treating the SDK command PID as its published identity. Termination signals the negative recorded id with `SIGTERM`, waits the caller's `graceMs`, then escalates to `SIGKILL` and the SDK kill fallback. If publication fails, the SDK PID remains the provisional `exec setsid` group id; rollback kills and verifies that group before startup rejects. Service disposal terminates and joins every retained handle before the sandbox owner disposes. - **Environment boundary** — the wrapper starts from the sandbox command environment, removes ambient `DSH_*` and credential-shaped (`*KEY*`, `*SECRET*`, `*TOKEN*`) names, then restores every `spec.env` entry as an explicit caller opt-in. Host ambient variables never enter the sandbox implicitly. -- **Stdio projection** — pipe mode forwards E2B callbacks into host Node streams; inherit mode forwards them to the harness process streams; collect mode retains a bounded host tail with offset reads. Optional complete spill files are written remotely and advertised only while within their cap. Batch and streaming stdin use the SDK handle. +- **Stdio projection** — the remote wrapper branches raw bytes into optional bounded spill files, frames each live chunk as newline-delimited base64 ASCII, and the host incrementally restores bytes across arbitrary SDK callback boundaries. Pipe mode writes those bytes to host Node streams; inherit mode writes them to the harness process streams; collect mode retains a bounded host tail with offset reads. Batch and streaming stdin use the SDK handle. - **Terminal sessions** — `spawnTerminal()` uses E2B's byte PTY API, installs the exact argv and scrubbed environment through private mode-`0600` files, reports the foreground process group, sends real signals, and tears down every group in the remote terminal session before settlement. Prompt detection, scrollback, readiness, and owner policy remain in `dsh-pty-local`. -The base E2B image supplies the Bash/GNU utilities this adapter invokes: `bash`, `setsid`, `ps`, `awk`, `tr`, `env`, `chmod`, `tee`, `head`, and `kill`. A custom template must retain compatible commands and E2B PTY support. +The base E2B image supplies the runtime and Bash/GNU utilities this adapter invokes: `node`, `bash`, `setsid`, `ps`, `awk`, `tr`, `env`, `chmod`, `tee`, `head`, and `kill`. A custom template must retain compatible commands and E2B PTY support. ## Model Experience @@ -25,11 +25,10 @@ No direct invalidation; the named consumers own any request-prefix changes. ## Known Limitations and Deferred Work -- **The SDK still retains complete command output in host memory** — E2B `CommandHandle.stdout` and `.stderr` accumulate even when this adapter exposes bounded tails, so the subprocess seam's normal host-memory bound is not achieved. -- **Command-pipe output is text-decoded by the SDK** — valid UTF-8 protocol traffic, including the exercised LSP composition and Code Runtime's ASCII/base64 frames, is supported; arbitrary binary protocols and invalid UTF-8 are not byte-faithful. +- **The SDK still retains complete command output in host memory** — E2B `CommandHandle.stdout` and `.stderr` accumulate the base64 transport even when this adapter exposes bounded raw-byte tails, so the subprocess seam's normal host-memory bound is not achieved and transport retention is larger than the source stream. - **Synchronous-PID consumers are unsupported** — `pid` remains `-1` during remote startup; consumers that require a positive PID immediately, including the ACP child backend, cannot use this provider unchanged. - **Reconnect does not reconstruct handles** — remote PID/status/spill files survive a retained sandbox, but a new harness process does not rebuild live `SubprocessHandle` objects or output cursors from them. - **Remote state accumulates when retained** — process directories and valid spill files remain under `.dsh-e2b`; this POC supplies no retention sweep. -- **Signal attribution is inferred** — when termination was requested and E2B reports a nonzero exit code, the adapter reports the last requested signal because the SDK result does not identify the terminating signal. +- **E2B exposes no signal fact** — only an adapter-requested `SIGTERM` or `SIGKILL` is reported as a signal; every unrequested SDK exit remains an exit code, including values shaped like `128 + signal`. - **Exact terminal stdin-wait inspection is unavailable** — E2B exposes the foreground process group but not the syscall evidence needed to prove it is waiting on fd 0, so the generic PTY backend falls back to controlled prompt markers and bounded silence. - **Linux utility and E2B transport semantics are assumed** — there is no Windows, arbitrary-template, escaped-session recovery, or network-partition fidelity layer. diff --git a/packages/e2b/subprocess-e2b/README.zh.md b/packages/e2b/subprocess-e2b/README.zh.md index f83969c794..d547e4e8d6 100644 --- a/packages/e2b/subprocess-e2b/README.zh.md +++ b/packages/e2b/subprocess-e2b/README.zh.md @@ -10,10 +10,10 @@ - **执行世界坐标**:`cwd` 和私有 `runtimeRoot` 来自共享所有者;可执行文件查找会验证绝对路径,或根据沙箱 PATH 加显式覆盖来解析裸名称。 - **Linux 进程组**:带引号保护的包装层会在 `exec setsid --wait` 下启动每组 argv,并在 `ctx.e2b.runtimeRoot/processes` 下记录实际进程组 ID 和私有状态文件。句柄会等待该文件,而不会把 SDK 命令 PID 当作已发布的身份。终止操作以记录的负数 ID 发送 `SIGTERM`,等待调用方的 `graceMs`,再升级到 `SIGKILL` 和 SDK kill 回退。如果发布失败,SDK PID 仍为临时的 `exec setsid` 进程组 ID;回滚会终止并验证该进程组,随后启动操作才会以拒绝结束。服务 dispose(资源释放)会在沙箱所有者释放前终止并等待每个保留句柄退出。 - **环境边界**:包装层从沙箱命令环境开始,移除环境中的 `DSH_*` 和形似凭据的名称(`*KEY*`、`*SECRET*`、`*TOKEN*`),再把每个 `spec.env` 条目恢复为调用方显式选择。宿主环境变量绝不会隐式进入沙箱。 -- **stdio 投影**:pipe 模式把 E2B 回调转发到宿主 Node 流;inherit 模式把回调转发到 harness 进程流;collect 模式保留有界的宿主尾部,并支持基于偏移量读取。可选的完整 spill 文件写在远程,并且只有未超过其上限时才会对外公布。批量 stdin 和流式 stdin 都使用 SDK 句柄。 +- **stdio 投影**:远程包装层先把原始字节分流到可选的有界 spill 文件,再把每个实时分片编码为换行分隔的 base64 ASCII 帧;宿主会跨任意 SDK 回调边界增量恢复字节。pipe 模式把这些字节写入宿主 Node 流;inherit 模式把字节写入 harness 进程流;collect 模式保留有界的宿主尾部,并支持基于偏移量读取。批量 stdin 和流式 stdin 都使用 SDK 句柄。 - **终端会话**:`spawnTerminal()` 使用 E2B 的字节 PTY API,以 mode 为 `0600` 的私有文件传入原样 argv 与清理后的环境,报告前台进程组,发送真实信号,并在结算前清理远程终端会话中的每个进程组。提示符检测、scrollback、就绪状态与所有者策略仍归 `dsh-pty-local` 所有。 -基础 E2B 镜像提供该适配器调用的 Bash/GNU 工具:`bash`、`setsid`、`ps`、`awk`、`tr`、`env`、`chmod`、`tee`、`head` 和 `kill`。自定义模板必须保留兼容的命令和 E2B PTY 支持。 +基础 E2B 镜像提供该适配器调用的运行时和 Bash/GNU 工具:`node`、`bash`、`setsid`、`ps`、`awk`、`tr`、`env`、`chmod`、`tee`、`head` 和 `kill`。自定义模板必须保留兼容的命令和 E2B PTY 支持。 ## 模型体验 @@ -25,11 +25,10 @@ ## 已知限制与延后工作 -- **SDK 仍会在宿主内存中保留完整命令输出**:即使本适配器公开的是有界尾部,E2B `CommandHandle.stdout` 和 `.stderr` 仍会持续累积,因此无法达到进程管理 seam 通常提供的宿主内存边界。 -- **命令管道输出由 SDK 解码为文本**:支持有效的 UTF-8 协议流量,包括已经过测试的 LSP 组合与 Code Runtime 的 ASCII/base64 帧;任意二进制协议和无效 UTF-8 不具备字节保真。 +- **SDK 仍会在宿主内存中保留完整命令输出**:即使本适配器公开的是有界原始字节尾部,E2B `CommandHandle.stdout` 和 `.stderr` 仍会累积 base64 传输内容,因此无法达到进程管理 seam 通常提供的宿主内存边界,而且传输保留量大于源数据流。 - **不支持需要同步 PID 的消费方**:远程启动期间,`pid` 保持为 `-1`;包括 ACP 子进程后端在内,要求立即获得正 PID 的消费方无法原样使用本提供方。 - **重新连接不会重建句柄**:保留沙箱后,远程 PID/状态/spill 文件仍然存在,但新的 harness 进程不会据此重建实时 `SubprocessHandle` 对象或输出游标。 - **保留沙箱时会累积远程状态**:进程目录和有效的 spill 文件会留在 `.dsh-e2b` 下;本 POC 不提供保留清理。 -- **信号归因依靠推断**:如果已经请求终止,而 E2B 报告非零退出码,适配器会报告最后请求的信号,因为 SDK 结果不标识终止信号。 +- **E2B 不公开信号事实**:只有适配器请求的 `SIGTERM` 或 `SIGKILL` 会报告为信号;其他未请求的 SDK 退出始终保留为退出码,包括形似 `128 + signal` 的值。 - **无法精确检查终端 stdin 等待状态**:E2B 会公开前台进程组,但不提供证明其正在等待 fd 0 所需的 syscall 证据,因此通用 PTY 后端会回退到受控提示符标记与有界静默机制。 - **依赖 Linux 工具与 E2B 传输语义**:没有 Windows、任意模板、逃逸会话恢复或网络分区的保真层。 diff --git a/packages/e2b/subprocess-e2b/src/output.ts b/packages/e2b/subprocess-e2b/src/output.ts index b6551983a0..6cb61e4f09 100644 --- a/packages/e2b/subprocess-e2b/src/output.ts +++ b/packages/e2b/subprocess-e2b/src/output.ts @@ -3,6 +3,64 @@ import { Buffer } from 'node:buffer' import type { SubprocessOutputRead, SubprocessOutputReader } from '@deepseek-ai/dsh-subprocess' +const BASE64_TEXT = /^[A-Za-z0-9+/]+={0,2}$/u + +/** Reserved non-base64 frame proving that one remote encoder reached clean EOF. */ +export const E2B_OUTPUT_COMPLETE_FRAME = '!dsh-e2b-output-complete!' + +/** Incrementally decode newline-delimited base64 frames emitted by one remote encoder. */ +export class E2BBase64Decoder { + private pending = '' + private complete = false + + /** + * Decode every complete newline-delimited frame in one arbitrarily split SDK callback. + * @param text - ASCII base64 frames from E2B's decoded callback. + * @returns the complete raw bytes made available by this callback. + */ + push(text: string): Buffer { + if (text.length === 0) return Buffer.alloc(0) + this.pending += text + const decoded: Buffer[] = [] + for (;;) { + const boundary = this.pending.indexOf('\n') + if (boundary < 0) break + const frame = this.pending.slice(0, boundary) + this.pending = this.pending.slice(boundary + 1) + if (frame === E2B_OUTPUT_COMPLETE_FRAME) { + if (this.complete) throw new Error('subprocess-e2b: duplicate output transport completion') + this.complete = true + continue + } + if (this.complete) throw new Error('subprocess-e2b: output transport continued after completion') + if (!BASE64_TEXT.test(frame)) { + throw new Error('subprocess-e2b: invalid base64 output transport') + } + const bytes = Buffer.from(frame, 'base64') + if (bytes.toString('base64') !== frame) { + throw new Error('subprocess-e2b: invalid base64 output transport') + } + decoded.push(bytes) + } + return Buffer.concat(decoded) + } + + /** + * Validate clean encoder completion, or discard an interrupted trailing frame after requested termination. + * @param requireComplete - Whether natural completion requires the reserved EOF frame. + */ + finish(requireComplete = true): void { + if (!requireComplete) { + this.pending = '' + return + } + if (this.pending.length > 0) { + throw new Error('subprocess-e2b: truncated base64 output transport') + } + if (!this.complete) throw new Error('subprocess-e2b: incomplete output transport') + } +} + /** Offset reader used for one collect-mode E2B stream. */ export class E2BOutputReader implements SubprocessOutputReader { private chunks: Buffer[] = [] @@ -27,12 +85,12 @@ export class E2BOutputReader implements SubprocessOutputReader { } /** - * Append one decoded SDK output event. - * @param text - Event text delivered by E2B. + * Append one byte-faithful decoded transport event. + * @param bytes - Raw command bytes recovered from the ASCII SDK transport. */ - push(text: string): void { - if (text.length === 0) return - const chunk = Buffer.from(text) + push(bytes: Uint8Array): void { + if (bytes.length === 0) return + const chunk = Buffer.from(bytes) this.totalBytes += chunk.length this.chunks.push(chunk) this.retainedBytes += chunk.length diff --git a/packages/e2b/subprocess-e2b/src/process.ts b/packages/e2b/subprocess-e2b/src/process.ts index 3b9594a752..9eeee5d877 100644 --- a/packages/e2b/subprocess-e2b/src/process.ts +++ b/packages/e2b/subprocess-e2b/src/process.ts @@ -8,6 +8,7 @@ import { quoteE2BShellArg, } from '@deepseek-ai/dsh-e2b' import type { CommandHandle, CommandResult, Sandbox } from '@deepseek-ai/dsh-e2b' +import { SENSITIVE_ENV_PATTERN } from '@deepseek-ai/dsh-subprocess' import type { SubprocessCollect, SubprocessHandle, @@ -16,9 +17,21 @@ import type { SubprocessSpawnSpec, } from '@deepseek-ai/dsh-subprocess' import type E2BSandboxService from '@deepseek-ai/dsh-e2b' -import { E2BOutputReader } from './output.ts' +import { E2BBase64Decoder, E2B_OUTPUT_COMPLETE_FRAME, E2BOutputReader } from './output.ts' const GROUP_POLL_MS = 20 +const OUTPUT_ENCODER_SOURCE = [ + '(async () => {', + ' for await (const chunk of process.stdin) {', + " if (!process.stdout.write(chunk.toString('base64') + '\\n')) {", + " await new Promise(resolve => process.stdout.once('drain', resolve))", + ' }', + ' }', + ` if (!process.stdout.write(${JSON.stringify(E2B_OUTPUT_COMPLETE_FRAME)} + '\\n')) {`, + " await new Promise(resolve => process.stdout.once('drain', resolve))", + ' }', + '})().catch(() => { process.exitCode = 1 })', +].join('\n') function isCollect(mode: SubprocessOutputMode): mode is SubprocessCollect { return mode !== 'pipe' && mode !== 'inherit' @@ -60,40 +73,65 @@ interface RemotePaths { stderr: string } -function explicitEnvironment(env: Readonly> | undefined): string { - return Object.entries(env ?? {}) - .map(([name, value]) => `${name}=${value}\0`) - .join('') +function remoteEnvironment(raw: string, explicit: Readonly> | undefined): string { + const environment = new Map() + for (const entry of raw.split('\0')) { + if (entry.length === 0) continue + const separator = entry.indexOf('=') + if (separator <= 0) continue + const name = entry.slice(0, separator) + if (name.startsWith('DSH_') || SENSITIVE_ENV_PATTERN.test(name)) continue + environment.set(name, entry.slice(separator + 1)) + } + for (const [name, value] of Object.entries(explicit ?? {})) environment.set(name, value) + return [...environment].map(([name, value]) => `${name}=${value}\0`).join('') } function commandText(spec: SubprocessSpawnSpec, paths: RemotePaths): string { + const encoder = `"$dsh_e2b_env_bin" -i "$dsh_e2b_node" -e ${quoteE2BShellArg(OUTPUT_ENCODER_SOURCE)}` const stdoutRedirect = hasSpill(spec.stdio.stdout) - ? `> >(tee --output-error=warn-nopipe >(head -c ${spec.stdio.stdout.spill.maxBytes} > ${quoteE2BShellArg(paths.stdout)}))` - : '' + ? `> >("$dsh_e2b_tee" --output-error=warn-nopipe >("$dsh_e2b_head" -c ${spec.stdio.stdout.spill.maxBytes} > ${quoteE2BShellArg(paths.stdout)}) | ${encoder} 2>/dev/null)` + : `> >(${encoder} 2>/dev/null)` const stderrRedirect = hasSpill(spec.stdio.stderr) - ? `2> >(tee --output-error=warn-nopipe >(head -c ${spec.stdio.stderr.spill.maxBytes} > ${quoteE2BShellArg(paths.stderr)}) >&2)` - : '' + ? `2> >("$dsh_e2b_tee" --output-error=warn-nopipe >("$dsh_e2b_head" -c ${spec.stdio.stderr.spill.maxBytes} > ${quoteE2BShellArg(paths.stderr)}) | ${encoder} >&2 2>/dev/null)` + : `2> >(${encoder} >&2 2>/dev/null)` const inner = [ 'set +e', 'umask 077', - 'dsh_e2b_pgid="$(ps -o pgid= -p "$$" | tr -d " ")"', + 'dsh_e2b_env_bin=$1', + 'dsh_e2b_node=$2', + 'dsh_e2b_ps=$3', + 'dsh_e2b_tr=$4', + 'dsh_e2b_tee=$5', + 'dsh_e2b_head=$6', + 'shift 6', + 'dsh_e2b_pgid="$("$dsh_e2b_ps" -o pgid= -p "$$" | "$dsh_e2b_tr" -d " ")"', `printf '%s\\n' "$dsh_e2b_pgid" > ${quoteE2BShellArg(paths.pid)}`, - `mapfile -d '' -t dsh_e2b_explicit < ${quoteE2BShellArg(paths.environment)}`, + `mapfile -d '' -t dsh_e2b_env < ${quoteE2BShellArg(paths.environment)}`, `: > ${quoteE2BShellArg(paths.environment)}`, - 'dsh_e2b_env=()', - "while IFS= read -r -d '' dsh_e2b_entry; do", - ' dsh_e2b_name="${dsh_e2b_entry%%=*}"', - ' case "${dsh_e2b_name^^}" in DSH_*|*KEY*|*SECRET*|*TOKEN*) continue ;; esac', - ' dsh_e2b_env+=("$dsh_e2b_entry")', - 'done < <(env -0)', - `env -i "\${dsh_e2b_env[@]}" "\${dsh_e2b_explicit[@]}" "$@" ${stdoutRedirect} ${stderrRedirect}`.trimEnd(), + `"$dsh_e2b_env_bin" -i "\${dsh_e2b_env[@]}" "$@" ${stdoutRedirect} ${stderrRedirect}`.trimEnd(), 'dsh_e2b_status=$?', 'wait', `printf '%s\\n' "$dsh_e2b_status" > ${quoteE2BShellArg(paths.status)}`, 'exit "$dsh_e2b_status"', ].join('\n') const argv = spec.argv.map(quoteE2BShellArg).join(' ') - return `exec setsid --wait -- bash -c ${quoteE2BShellArg(inner)} dsh-e2b ${argv}` + const bootstrap = [ + `mapfile -d '' -t dsh_e2b_env < ${quoteE2BShellArg(paths.environment)}`, + 'dsh_e2b_env_bin="$(command -v env)"', + 'dsh_e2b_setsid="$(command -v setsid)"', + 'dsh_e2b_bash="$(command -v bash)"', + 'dsh_e2b_node="$(command -v node)"', + 'dsh_e2b_ps="$(command -v ps)"', + 'dsh_e2b_tr="$(command -v tr)"', + 'dsh_e2b_tee="$(command -v tee)"', + 'dsh_e2b_head="$(command -v head)"', + 'for dsh_e2b_tool in "$dsh_e2b_env_bin" "$dsh_e2b_setsid" "$dsh_e2b_bash" "$dsh_e2b_node" "$dsh_e2b_ps" "$dsh_e2b_tr" "$dsh_e2b_tee" "$dsh_e2b_head"; do', + ' [[ "$dsh_e2b_tool" == /* && -x "$dsh_e2b_tool" ]] || exit 125', + 'done', + `exec "$dsh_e2b_env_bin" -i "\${dsh_e2b_env[@]}" "$dsh_e2b_setsid" --wait -- "$dsh_e2b_bash" -c ${quoteE2BShellArg(inner)} dsh-e2b "$dsh_e2b_env_bin" "$dsh_e2b_node" "$dsh_e2b_ps" "$dsh_e2b_tr" "$dsh_e2b_tee" "$dsh_e2b_head" ${argv}`, + ].join('\n') + return bootstrap } function signalOpts(signal: AbortSignal | undefined): { signal?: AbortSignal } { @@ -128,10 +166,14 @@ export class E2BSubprocessHandle implements SubprocessHandle { readonly done: Promise private readonly readyState = Promise.withResolvers() + private readonly stdoutDecoder = new E2BBase64Decoder() + private readonly stderrDecoder = new E2BBase64Decoder() private readonly stdoutReader: E2BOutputReader | undefined private readonly stderrReader: E2BOutputReader | undefined private readonly paths: RemotePaths private remotePid = -1 + private commandHandle: CommandHandle | undefined + private outputTransportError: Error | undefined private terminationRequested = false private terminationSignal: NodeJS.Signals | null = null private termination: Promise | undefined @@ -195,7 +237,8 @@ export class E2BSubprocessHandle implements SubprocessHandle { try { handle = await this.readyForWait(signal) } catch { - return true + handle = this.commandHandle + if (handle === undefined) return true } if (handle === undefined) return false let sandbox: Sandbox @@ -205,7 +248,8 @@ export class E2BSubprocessHandle implements SubprocessHandle { if (isAborted(signal)) return false throw error } - while (await this.groupAlive(sandbox, this.remotePid, signal)) { + const processGroupId = this.remotePid > 0 ? this.remotePid : handle.pid + while (await this.groupAlive(sandbox, processGroupId, signal)) { if (!await waitTick(signal)) return false } return !isAborted(signal) @@ -248,6 +292,7 @@ export class E2BSubprocessHandle implements SubprocessHandle { if (!Number.isSafeInteger(handle.pid) || handle.pid <= 0) { throw new Error(`subprocess-e2b: E2B returned invalid command pid ${handle.pid}`) } + this.commandHandle = handle const completion = handle.wait() void completion.catch(() => {}) try { @@ -266,6 +311,10 @@ export class E2BSubprocessHandle implements SubprocessHandle { this.readyState.resolve(handle) await this.writeBatchStdin(handle) const outcome = await this.waitForCommand(completion) + if (this.outputTransportError !== undefined) throw this.outputTransportError + const requireCompleteOutput = this.terminationSignal === null + this.stdoutDecoder.finish(requireCompleteOutput) + this.stderrDecoder.finish(requireCompleteOutput) await this.finalizeSpills(sandbox) return outcome } catch (error: unknown) { @@ -279,12 +328,13 @@ export class E2BSubprocessHandle implements SubprocessHandle { } private async prepareState(sandbox: Sandbox): Promise { + const ambient = await sandbox.commands.run('env -0') await sandbox.files.makeDir(this.stateDir) await sandbox.commands.run(`chmod 700 -- ${quoteE2BShellArg(this.stateDir)}`) const files = [ { path: this.paths.pid, data: '' }, { path: this.paths.status, data: '' }, - { path: this.paths.environment, data: explicitEnvironment(this.spec.env) }, + { path: this.paths.environment, data: remoteEnvironment(ambient.stdout, this.spec.env) }, ...(hasSpill(this.spec.stdio.stdout) ? [{ path: this.paths.stdout, data: '' }] : []), ...(hasSpill(this.spec.stdio.stderr) ? [{ path: this.paths.stderr, data: '' }] : []), ] @@ -303,25 +353,34 @@ export class E2BSubprocessHandle implements SubprocessHandle { } private async dispatchOutput(stream: 'stdout' | 'stderr', data: string): Promise { + let bytes: Buffer + try { + bytes = stream === 'stdout' ? this.stdoutDecoder.push(data) : this.stderrDecoder.push(data) + } catch (error: unknown) { + this.outputTransportError ??= asError(error) + const target = stream === 'stdout' ? this.stdout : this.stderr + target?.destroy(this.outputTransportError) + return + } try { if (stream === 'stdout') { - this.stdoutReader?.push(data) - await this.writeOutput(this.stdout, this.spec.stdio.stdout === 'inherit' ? process.stdout : undefined, data) + this.stdoutReader?.push(bytes) + await this.writeOutput(this.stdout, this.spec.stdio.stdout === 'inherit' ? process.stdout : undefined, bytes) return } - this.stderrReader?.push(data) - await this.writeOutput(this.stderr, this.spec.stdio.stderr === 'inherit' ? process.stderr : undefined, data) + this.stderrReader?.push(bytes) + await this.writeOutput(this.stderr, this.spec.stdio.stderr === 'inherit' ? process.stderr : undefined, bytes) } catch (error: unknown) { const target = stream === 'stdout' ? this.stdout : this.stderr target?.destroy(asError(error)) } } - private async writeOutput(pipe: PassThrough | undefined, inherited: NodeJS.WriteStream | undefined, data: string): Promise { + private async writeOutput(pipe: PassThrough | undefined, inherited: NodeJS.WriteStream | undefined, data: Uint8Array): Promise { const target = pipe ?? inherited if (target === undefined || data.length === 0) return if (target.destroyed) throw new Error('subprocess output stream is closed') - if (target.write(Buffer.from(data))) return + if (target.write(data)) return await new Promise((resolve, reject) => { const onDrain = (): void => { cleanup(); resolve() } const onError = (error: Error): void => { cleanup(); reject(error) } @@ -369,10 +428,10 @@ export class E2BSubprocessHandle implements SubprocessHandle { } private async rollbackUnpublishedGroup(sandbox: Sandbox, handle: CommandHandle): Promise { - // The background command begins with `exec setsid`, so E2B's command PID is - // the provisional group id even before the private publication file can be - // trusted. Kill that group before the SDK-PID fallback, then prove no group - // member survived before rejecting startup. + // The bootstrap ends in an exec chain through the scrubbed environment and + // `setsid`, so E2B's command PID is the provisional group id even before the + // private publication file can be trusted. Kill that group before the SDK-PID + // fallback, then prove no group member survived before rejecting startup. try { await this.signalGroup(sandbox, handle.pid, 'KILL') } finally { @@ -382,23 +441,25 @@ export class E2BSubprocessHandle implements SubprocessHandle { } private async terminateRemote(): Promise { - let handle: CommandHandle + let handle: CommandHandle | undefined try { handle = await this.readyState.promise } catch { - return + handle = this.commandHandle } + if (handle === undefined) return const sandbox = await this.runtime.getSandbox() + const processGroupId = this.remotePid > 0 ? this.remotePid : handle.pid this.terminationSignal = 'SIGTERM' - await this.signalGroup(sandbox, this.remotePid, 'TERM') + await this.signalGroup(sandbox, processGroupId, 'TERM') const deadline = Date.now() + this.spec.graceMs - while (Date.now() < deadline && await this.groupAlive(sandbox, this.remotePid)) { + while (Date.now() < deadline && await this.groupAlive(sandbox, processGroupId)) { await waitTick() } - if (!await this.groupAlive(sandbox, this.remotePid)) return + if (!await this.groupAlive(sandbox, processGroupId)) return this.terminationSignal = 'SIGKILL' try { - await this.signalGroup(sandbox, this.remotePid, 'KILL') + await this.signalGroup(sandbox, processGroupId, 'KILL') } finally { await handle.kill().catch(() => false) } diff --git a/packages/e2b/subprocess-e2b/src/terminal.ts b/packages/e2b/subprocess-e2b/src/terminal.ts index 1a916c0fd5..565e406b97 100644 --- a/packages/e2b/subprocess-e2b/src/terminal.ts +++ b/packages/e2b/subprocess-e2b/src/terminal.ts @@ -1,7 +1,6 @@ /** E2B PTY allocation and process-session ownership for the subprocess seam. */ import { Buffer } from 'node:buffer' -import { constants } from 'node:os' import { PassThrough } from 'node:stream' import { posix } from 'node:path' import { @@ -53,13 +52,8 @@ function delay(ms: number): Promise { return new Promise(resolve => setTimeout(resolve, ms)) } -function commandSignal(exitCode: number): NodeJS.Signals | null { - const number = exitCode - 128 - if (number <= 0) return null - for (const [name, value] of Object.entries(constants.signals)) { - if (value === number) return name as NodeJS.Signals - } - return null +function asError(error: unknown): Error { + return error instanceof Error ? error : new Error(String(error)) } function parsePositiveId(value: string, message: string): number { @@ -121,6 +115,132 @@ async function waitUntilReady( } } +async function sessionProcessGroups(sandbox: Sandbox, sessionId: number): Promise { + const result = await sandbox.commands.run( + `ps -eo sid=,pgid= | awk '$1 == ${sessionId} { print $2 }'`, + ) + const groups = new Set() + for (const raw of result.stdout.trim().split(/\s+/)) { + if (raw.length === 0) continue + const group = parsePositiveId( + raw, + `subprocess-e2b: invalid process group ${JSON.stringify(raw)} in terminal session ${sessionId}`, + ) + if (group <= 1) { + throw new Error(`subprocess-e2b: unsafe process group ${group} in terminal session ${sessionId}`) + } + groups.add(group) + } + return [...groups] +} + +async function signalGroups(sandbox: Sandbox, groups: number[], signal: 'TERM' | 'KILL'): Promise { + try { + await sandbox.commands.run(`kill -${signal} -- ${groups.map(group => `-${group}`).join(' ')}`) + } catch (error: unknown) { + if (!(error instanceof CommandExitError)) throw error + } +} + +async function awaitSessionEmpty( + sandbox: Sandbox, + sessionId: number, + graceMs: number, + kill = false, +): Promise { + const deadline = Date.now() + graceMs + for (;;) { + const groups = await sessionProcessGroups(sandbox, sessionId) + if (groups.length === 0 || Date.now() >= deadline) return groups + if (kill) await signalGroups(sandbox, groups, 'KILL') + await delay(Math.min(POLL_MS, Math.max(1, deadline - Date.now()))) + } +} + +async function rollbackUnpublishedTerminal( + sandbox: Sandbox, + handle: CommandHandle, + completion: Promise, + graceMs: number, +): Promise { + let topLevelExited = false + void completion.then( + () => { topLevelExited = true }, + () => { topLevelExited = true }, + ) + const validPid = Number.isSafeInteger(handle.pid) && handle.pid > 1 + const attemptFailures: Error[] = [] + let sessionId: number | undefined + if (validPid) { + sessionId = handle.pid + try { + sessionId = await terminalSessionId(sandbox, handle.pid) + } catch (_sessionLookupFailure) { + // E2B's PTY leader is also the provisional POSIX session leader, so its + // PID remains usable after the setup lookup itself fails or is canceled. + } + try { + let groups = await sessionProcessGroups(sandbox, sessionId) + if (groups.length > 0) { + await signalGroups(sandbox, groups, 'TERM') + groups = await awaitSessionEmpty(sandbox, sessionId, graceMs) + } + if (groups.length > 0) { + await signalGroups(sandbox, groups, 'KILL') + await awaitSessionEmpty(sandbox, sessionId, graceMs, true) + } + } catch (error: unknown) { + attemptFailures.push(asError(error)) + } + } + // Completion can settle while any awaited provider cleanup above is running. + // eslint-disable-next-line @typescript-eslint/no-unnecessary-condition + if (!topLevelExited) { + if (validPid) { + try { + await sandbox.pty.kill(handle.pid) + } catch (error: unknown) { + attemptFailures.push(asError(error)) + } + } + // The awaited PTY fallback can settle completion before the SDK fallback. + // eslint-disable-next-line @typescript-eslint/no-unnecessary-condition + if (!topLevelExited) { + try { + await handle.kill() + } catch (error: unknown) { + attemptFailures.push(asError(error)) + } + } + await Promise.race([completion.catch(() => undefined), delay(graceMs)]) + } + const proofFailures: Error[] = [] + if (sessionId !== undefined) { + try { + const groups = await awaitSessionEmpty(sandbox, sessionId, graceMs, true) + if (groups.length > 0) { + proofFailures.push(new Error( + `subprocess-e2b: terminal setup rollback failed; surviving process groups: ${groups.join(', ')}`, + )) + } + } catch (error: unknown) { + proofFailures.push(asError(error)) + } + } + // The bounded completion race above updates this callback-owned state. + // eslint-disable-next-line @typescript-eslint/no-unnecessary-condition + if (!topLevelExited) { + proofFailures.push(new Error(`subprocess-e2b: terminal setup rollback failed; surviving pid: ${handle.pid}`)) + } + if (proofFailures.length > 0) { + throw new AggregateError( + [...attemptFailures, ...proofFailures], + 'subprocess-e2b: terminal setup rollback did not reach quiescence', + ) + } + await handle.disconnect() +} + /** One E2B PTY and all process groups in its remote process session. */ export class E2BTerminalHandle implements SubprocessTerminalHandle { readonly pid: number @@ -203,8 +323,9 @@ export class E2BTerminalHandle implements SubprocessTerminalHandle { return { exitCode: result.exitCode, signal: null } } catch (error: unknown) { if (error instanceof CommandExitError) { - const signal = this.terminationSignal ?? commandSignal(error.exitCode) - return signal === null ? { exitCode: error.exitCode, signal: null } : { exitCode: null, signal } + return this.terminationSignal === null + ? { exitCode: error.exitCode, signal: null } + : { exitCode: null, signal: this.terminationSignal } } this.output.destroy(error instanceof Error ? error : new Error(String(error))) throw error @@ -214,49 +335,12 @@ export class E2BTerminalHandle implements SubprocessTerminalHandle { } } - private async sessionProcessGroups(): Promise { - const result = await this.sandbox.commands.run( - `ps -eo sid=,pgid= | awk '$1 == ${this.sessionId} { print $2 }'`, - ) - const groups = new Set() - for (const raw of result.stdout.trim().split(/\s+/)) { - if (raw.length === 0) continue - const group = parsePositiveId( - raw, - `subprocess-e2b: invalid process group ${JSON.stringify(raw)} in terminal session ${this.sessionId}`, - ) - if (group <= 1) { - throw new Error(`subprocess-e2b: unsafe process group ${group} in terminal session ${this.sessionId}`) - } - groups.add(group) - } - return [...groups] - } - - private async signalGroups(groups: number[], signal: 'TERM' | 'KILL'): Promise { - try { - await this.sandbox.commands.run(`kill -${signal} -- ${groups.map(group => `-${group}`).join(' ')}`) - } catch (error: unknown) { - if (!(error instanceof CommandExitError)) throw error - } - } - - private async awaitSessionEmpty(kill = false): Promise { - const deadline = Date.now() + this.graceMs - for (;;) { - const groups = await this.sessionProcessGroups() - if (groups.length === 0 || Date.now() >= deadline) return groups - if (kill) await this.signalGroups(groups, 'KILL') - await delay(Math.min(POLL_MS, Math.max(1, deadline - Date.now()))) - } - } - private async closeOnce(): Promise { - let groups = await this.sessionProcessGroups() + let groups = await sessionProcessGroups(this.sandbox, this.sessionId) if (groups.length > 0) { this.terminationSignal = 'SIGTERM' - await this.signalGroups(groups, 'TERM') - groups = await this.awaitSessionEmpty() + await signalGroups(this.sandbox, groups, 'TERM') + groups = await awaitSessionEmpty(this.sandbox, this.sessionId, this.graceMs) } if (groups.length === 0 && !this.topLevelExited) { await Promise.race([this.done.catch(() => undefined), delay(this.graceMs)]) @@ -264,7 +348,7 @@ export class E2BTerminalHandle implements SubprocessTerminalHandle { if (groups.length > 0 || !this.topLevelExited) { this.terminationSignal = 'SIGKILL' if (!this.topLevelExited) await this.sandbox.pty.kill(this.pid) - groups = await this.awaitSessionEmpty(true) + groups = await awaitSessionEmpty(this.sandbox, this.sessionId, this.graceMs, true) if (!this.topLevelExited) await Promise.race([this.done.catch(() => undefined), delay(this.graceMs)]) } if (groups.length > 0) { @@ -348,9 +432,20 @@ export async function spawnE2BTerminal( ) } catch (error: unknown) { output.destroy() - if (handle !== undefined) await handle.kill().catch(() => false) - if (completion !== undefined) await completion.catch(() => {}) + let cleanupError: Error | undefined + if (handle !== undefined && completion !== undefined) { + try { + await rollbackUnpublishedTerminal(sandbox, handle, completion, spec.graceMs) + } catch (rollbackError: unknown) { + cleanupError = asError(rollbackError) + } + } else if (handle !== undefined) { + await handle.kill().catch(() => false) + } await sandbox.files.remove(stateDir).catch(() => {}) + if (cleanupError !== undefined) { + throw new AggregateError([asError(error), cleanupError], asError(error).message) + } throw error } } diff --git a/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts b/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts index fd4cc97a83..32adf872b3 100644 --- a/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts +++ b/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts @@ -10,7 +10,7 @@ import type E2BSandboxService from '@deepseek-ai/dsh-e2b' import type { SubprocessSpawnSpec } from '@deepseek-ai/dsh-subprocess' import E2BSubprocessService from '@deepseek-ai/dsh-subprocess-e2b' import * as E2BSubprocessInvariant from '../src/invariant.ts' -import { E2BOutputReader } from '../src/output.ts' +import { E2BBase64Decoder, E2B_OUTPUT_COMPLETE_FRAME, E2BOutputReader } from '../src/output.ts' import { E2BSubprocessHandle } from '../src/process.ts' import InvariantService from '@deepseek-ai/dsh-invariants' import { describe, expect, it, vi } from 'vitest' @@ -91,6 +91,7 @@ class FakeSandbox { trapsTerm = false delaysKill = false alive = true + ambient = 'PATH=/ambient/bin\0KEEP=safe\0NPM_TOKEN=secret\0DSH_STALE=old\0BROKEN\0=bad\0' processGroupId = '4242\n' readonly processGroupReads: string[] = [] beforeProbe: (() => void) | undefined @@ -110,15 +111,35 @@ class FakeSandbox { finish(exitCode = 0): void { this.alive = false - if (exitCode === 0) this.handle.succeed(0) - else this.handle.fail(exitCode) + void this.completeOutput().then( + () => { + if (exitCode === 0) this.handle.succeed(0) + else this.handle.fail(exitCode) + }, + (error: unknown) => { this.handle.crash(error) }, + ) + } + + async completeOutput(): Promise { + await Promise.all([ + this.stdoutWire(`${E2B_OUTPUT_COMPLETE_FRAME}\n`), + this.stderrWire(`${E2B_OUTPUT_COMPLETE_FRAME}\n`), + ]) } async stdout(data: string): Promise { - await this.startOptions?.onStdout?.(data) + await this.stdoutWire(data.length === 0 ? '' : `${Buffer.from(data).toString('base64')}\n`) } async stderr(data: string): Promise { + await this.stderrWire(data.length === 0 ? '' : `${Buffer.from(data).toString('base64')}\n`) + } + + async stdoutWire(data: string): Promise { + await this.startOptions?.onStdout?.(data) + } + + async stderrWire(data: string): Promise { await this.startOptions?.onStderr?.(data) } @@ -147,6 +168,7 @@ class FakeSandbox { commands: { run: async (command: string, options?: StartOptions | { signal?: AbortSignal }): Promise => { this.commandsSeen.push(command) + if (command === 'env -0') return { exitCode: 0, stdout: this.ambient, stderr: '' } if (command.startsWith('kill -0 ')) { this.beforeProbe?.() if (options?.signal?.aborted === true) throw new DOMException('aborted', 'AbortError') @@ -221,11 +243,35 @@ async function flush(): Promise { } describe('E2BOutputReader', () => { + it('decodes base64 across arbitrary callback boundaries and rejects malformed framing', () => { + const decoder = new E2BBase64Decoder() + expect(decoder.push('')).toEqual(Buffer.alloc(0)) + expect(decoder.push('5')).toEqual(Buffer.alloc(0)) + expect(decoder.push('L2')).toEqual(Buffer.alloc(0)) + expect(decoder.push('g\n').toString()).toBe('你') + expect(decoder.push('YQ==\nYg==\n').toString()).toBe('ab') + expect(decoder.push(`${Buffer.from([0, 255]).toString('base64')}\n`)).toEqual(Buffer.from([0, 255])) + expect(decoder.push(`${E2B_OUTPUT_COMPLETE_FRAME}\n`)).toEqual(Buffer.alloc(0)) + decoder.finish() + + expect(() => new E2BBase64Decoder().push('%\n')).toThrow('invalid base64') + expect(() => new E2BBase64Decoder().push('AB==\n')).toThrow('invalid base64') + expect(() => decoder.push(`${E2B_OUTPUT_COMPLETE_FRAME}\n`)).toThrow('duplicate output transport completion') + expect(() => decoder.push('YQ==\n')).toThrow('continued after completion') + const truncated = new E2BBase64Decoder() + truncated.push('YQ') + expect(() => { truncated.finish() }).toThrow('truncated base64') + expect(() => { new E2BBase64Decoder().finish() }).toThrow('incomplete output transport') + const interrupted = new E2BBase64Decoder() + interrupted.push('YQ') + expect(() => { interrupted.finish(false) }).not.toThrow() + }) + it('keeps a byte-exact tail with independent whole-stream cursors', () => { const reader = new E2BOutputReader(4, 10, '/remote/spill') - reader.push('') - reader.push('ab') - reader.push('cdef') + reader.push(Buffer.alloc(0)) + reader.push(Buffer.from('ab')) + reader.push(Buffer.from('cdef')) expect(reader.size).toBe(6) expect(reader.readFrom(0)).toEqual({ text: 'cdef', nextOffset: 6, lossy: true, spillPath: '/remote/spill' }) expect(reader.readFrom(2)).toEqual({ text: 'cdef', nextOffset: 6, lossy: false }) @@ -235,11 +281,11 @@ describe('E2BOutputReader', () => { it('drops whole head chunks and withholds absent or over-cap spills', () => { const withoutSpill = new E2BOutputReader(2, undefined, '/unused') - withoutSpill.push('ab') - withoutSpill.push('cd') + withoutSpill.push(Buffer.from('ab')) + withoutSpill.push(Buffer.from('cd')) expect(withoutSpill.readFrom(0)).toEqual({ text: 'cd', nextOffset: 4, lossy: true }) const overCap = new E2BOutputReader(2, 3, '/too-small') - overCap.push('abcd') + overCap.push(Buffer.from('abcd')) expect(overCap.readFrom(0)).toEqual({ text: 'cd', nextOffset: 4, lossy: true }) expect(() => overCap.readFrom(-1)).toThrow(/non-negative safe integer/) expect(() => overCap.readFrom(1.5)).toThrow(/non-negative safe integer/) @@ -265,16 +311,22 @@ describe('E2BSubprocessHandle', () => { expect(fake.handle.sent.map(value => String(value))).toEqual(['hello']) expect(fake.handle.closes).toBe(1) expect(fake.startOptions?.envs).toBeUndefined() - const command = fake.commandsSeen.find(value => value.startsWith('exec setsid'))! - expect(command).toContain('exec setsid --wait -- bash -c') + const command = fake.commandsSeen.find(value => value.includes('exec "$dsh_e2b_env_bin" -i'))! + expect(command).toContain('"$dsh_e2b_setsid" --wait -- "$dsh_e2b_bash" -c') expect(command).not.toContain('DEEPSEEK_API_KEY') expect(command).not.toContain('DSH_MODE') expect(command).not.toContain('FOO-BAR') expect(command).not.toContain('explicit-secret') expect(command).not.toContain('hyphen-value') expect(command).not.toContain('${!dsh_e2b_name}') - expect(command).toContain('env -0') + expect(fake.commandsSeen).toContain('env -0') expect(command).toContain('mapfile -d') + expect(command).toContain('dsh_e2b_node="$(command -v node)"') + expect(command).toContain('"$dsh_e2b_env_bin" -i "$dsh_e2b_node" -e') + expect(command).toContain('exec "$dsh_e2b_env_bin" -i "${dsh_e2b_env[@]}"') + expect(command).toContain('>&2 2>/dev/null') + expect(command).not.toContain('2>/dev/null >&2') + expect(command).toContain('base64') expect(fake.writtenFiles[0]).toEqual([ '/workspace/.dsh-e2b/processes/one/pid', '/workspace/.dsh-e2b/processes/one/exit-code', @@ -282,7 +334,7 @@ describe('E2BSubprocessHandle', () => { '/workspace/.dsh-e2b/processes/one/stderr.log', ]) expect(fake.writtenFileData.get('/workspace/.dsh-e2b/processes/one/environment')).toBe( - 'PATH=/bin\0FOO-BAR=hyphen-value\0DEEPSEEK_API_KEY=explicit-secret\0DSH_MODE=test\0', + 'PATH=/bin\0KEEP=safe\0FOO-BAR=hyphen-value\0DEEPSEEK_API_KEY=explicit-secret\0DSH_MODE=test\0', ) let piped = '' @@ -297,6 +349,47 @@ describe('E2BSubprocessHandle', () => { await expect(handle.waitForExit()).resolves.toBe(true) }) + it('preserves UTF-8 bytes when the ASCII transport is split across callbacks', async () => { + const fake = new FakeSandbox() + const handle = new E2BSubprocessHandle(runtime(fake), spec({ + stdio: { stdin: 'ignore', stdout: 'pipe', stderr: { maxBytes: 4 } }, + }), '/runtime/split-utf8') + await flush() + const chunks: Buffer[] = [] + handle.stdout!.on('data', (chunk: Buffer) => { chunks.push(chunk) }) + for (const character of `${Buffer.from('A你好B').toString('base64')}\n`) { + await fake.stdoutWire(character) + } + fake.finish() + await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) + expect(Buffer.concat(chunks).toString('utf8')).toBe('A你好B') + }) + + it('rejects malformed output transport without confusing it with a consumer sink failure', async () => { + const fake = new FakeSandbox() + const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/malformed-output') + await flush() + await fake.stdoutWire('%\n') + fake.finish() + await expect(handle.done).rejects.toThrow('invalid base64 output transport') + + const stderrFake = new FakeSandbox() + const stderrHandle = new E2BSubprocessHandle(runtime(stderrFake), spec(), '/runtime/malformed-stderr') + await flush() + await stderrFake.stderrWire('%\n') + stderrFake.finish() + await expect(stderrHandle.done).rejects.toThrow('invalid base64 output transport') + }) + + it('rejects a naturally completed command whose encoder omits its completion frame', async () => { + const fake = new FakeSandbox() + const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/incomplete-output') + await flush() + fake.alive = false + fake.handle.succeed(0) + await expect(handle.done).rejects.toThrow('incomplete output transport') + }) + it('surfaces deferred piped-stdin write and close failures as stream errors', async () => { const writeFake = new FakeSandbox() writeFake.deferStart() @@ -362,10 +455,10 @@ describe('E2BSubprocessHandle', () => { await handle.done expect(handle.collected.stdout!.readFrom(0)).toEqual({ text: 'cd', nextOffset: 4, lossy: true }) expect(fake.removed).toContain('/runtime/oversize/stdout.log') - const command = fake.commandsSeen.find(value => value.startsWith('exec setsid'))! - expect(command).toContain('head -c 3') + const command = fake.commandsSeen.find(value => value.includes('dsh_e2b_tee='))! + expect(command).toContain('"$dsh_e2b_head" -c 3') expect(command).toContain('/runtime/oversize/stdout.log') - expect(command).toContain('tee --output-error=warn-nopipe') + expect(command).toContain('"$dsh_e2b_tee" --output-error=warn-nopipe') expect(command).not.toContain('tee -a') }) @@ -436,6 +529,7 @@ describe('E2BSubprocessHandle', () => { const fake = new FakeSandbox() const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/surviving-group') await flush() + await fake.completeOutput() fake.handle.succeed(0) await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) expect(fake.alive).toBe(true) @@ -640,7 +734,13 @@ describe('E2BSubprocessHandle', () => { expect(failures[0].message).toContain('invalid process-group id') expect(failures[1].message).toBe('rollback signal failed') expect(fake.handle.kills).toBe(1) - fake.finish() + const bounded = new AbortController() + const waiting = handle.waitForExit(bounded.signal) + bounded.abort() + await expect(waiting).resolves.toBe(false) + handle.terminate() + await expect(handle.waitForExit()).resolves.toBe(true) + expect(fake.commandsSeen).toContain('kill -TERM -- -4242') }) it('waits for delayed process-group publication', async () => { diff --git a/packages/e2b/subprocess-e2b/tests/terminal.spec.ts b/packages/e2b/subprocess-e2b/tests/terminal.spec.ts index 81716c3d52..cefa432acd 100644 --- a/packages/e2b/subprocess-e2b/tests/terminal.spec.ts +++ b/packages/e2b/subprocess-e2b/tests/terminal.spec.ts @@ -30,10 +30,13 @@ class FakeTerminalCommandHandle { sdkKills = 0 disconnectError: unknown sdkKillError: unknown + waitError: unknown + settleOnSdkKill = true private readonly result = Promise.withResolvers() private settled = false wait(): Promise { + if (this.waitError !== undefined) throw this.waitError return this.result.promise } @@ -46,10 +49,10 @@ class FakeTerminalCommandHandle { this.sdkKills += 1 if (this.sdkKillError !== undefined) { const error = this.sdkKillError - this.fail(137) + if (this.settleOnSdkKill) this.fail(137) throw error } - this.fail(137) + if (this.settleOnSdkKill) this.fail(137) return true } @@ -94,8 +97,10 @@ class FakeTerminalSandbox { createError: unknown sendError: unknown commandFailure: unknown + sessionGroupsFailure: unknown foregroundFailure: unknown termFailure: unknown + ptyKillError: unknown removeError: unknown clearOnTerm = true clearOnKill = true @@ -146,6 +151,7 @@ class FakeTerminalSandbox { return { exitCode: 0, stdout: this.foreground, stderr: '' } } if (command.startsWith('ps -eo sid=')) { + if (this.sessionGroupsFailure !== undefined) throw this.sessionGroupsFailure return { exitCode: 0, stdout: this.groups.map(group => `${group}\n`).join(''), stderr: '' } } if (command.startsWith('kill -TERM -- ')) { @@ -173,6 +179,7 @@ class FakeTerminalSandbox { }, kill: async (pid: number): Promise => { this.ptyKills += 1 + if (this.ptyKillError !== undefined) throw this.ptyKillError if (this.settleOnPtyKill) this.handle.fail(137) return pid === this.handle.pid }, @@ -282,7 +289,8 @@ describe('E2B terminal allocation', () => { failedInput.sendError = new Error('bootstrap failed') await expect(spawnE2BTerminal(runtime(failedInput), spec(), '/runtime/input')) .rejects.toThrow('bootstrap failed') - expect(failedInput.handle.sdkKills).toBe(1) + expect(failedInput.commands).toContain('kill -TERM -- -123') + expect(failedInput.groups).toEqual([]) const exited = new FakeTerminalSandbox() exited.ready = new FileNotFoundError('not ready') @@ -292,13 +300,62 @@ describe('E2B terminal allocation', () => { const invalidSession = new FakeTerminalSandbox() invalidSession.sessionId = 'not-a-session\n' + invalidSession.clearOnTerm = false await expect(spawnE2BTerminal(runtime(invalidSession), spec(), '/runtime/session')) .rejects.toThrow('cannot resolve process session') - expect(invalidSession.handle.sdkKills).toBe(1) + expect(invalidSession.commands).toContain('kill -TERM -- -123') + expect(invalidSession.commands).toContain('kill -KILL -- -123') + expect(invalidSession.groups).toEqual([]) + expect(invalidSession.ptyKills).toBe(1) const lateData = invalidSession.createOptions?.onData if (lateData === undefined) throw new Error('missing captured terminal callback') expect(lateData(Buffer.from('late bytes'))).toBeUndefined() + const termFailed = new FakeTerminalSandbox() + termFailed.sendError = new Error('bootstrap failed') + termFailed.termFailure = new Error('TERM transport failed') + await expect(spawnE2BTerminal(runtime(termFailed), spec(), '/runtime/term-failed')) + .rejects.toThrow('bootstrap failed') + expect(termFailed.commands).toContain('kill -KILL -- -123') + expect(termFailed.ptyKills).toBe(1) + + const uninspectable = new FakeTerminalSandbox() + uninspectable.sendError = new Error('bootstrap failed') + uninspectable.sessionGroupsFailure = 'session enumeration failed' + uninspectable.ptyKillError = new Error('PTY kill failed') + let uninspectableFailure: unknown + try { + await spawnE2BTerminal(runtime(uninspectable), spec(), '/runtime/uninspectable') + } catch (error: unknown) { + uninspectableFailure = error + } + expect(uninspectableFailure).toBeInstanceOf(AggregateError) + expect(uninspectable.ptyKills).toBe(1) + expect(uninspectable.handle.sdkKills).toBe(1) + + const survivingGroups = new FakeTerminalSandbox() + survivingGroups.sendError = new Error('bootstrap failed') + survivingGroups.clearOnTerm = false + survivingGroups.clearOnKill = false + await expect(spawnE2BTerminal(runtime(survivingGroups), spec({ graceMs: 1 }), '/runtime/surviving-groups')) + .rejects.toThrow('bootstrap failed') + + const survivingPid = new FakeTerminalSandbox() + survivingPid.sendError = new Error('bootstrap failed') + survivingPid.groups = [] + survivingPid.settleOnPtyKill = false + survivingPid.handle.settleOnSdkKill = false + await expect(spawnE2BTerminal(runtime(survivingPid), spec({ graceMs: 1 }), '/runtime/surviving-pid')) + .rejects.toThrow('bootstrap failed') + + const waitFailed = new FakeTerminalSandbox() + waitFailed.handle.waitError = new Error('wait failed') + waitFailed.handle.settleOnSdkKill = false + waitFailed.handle.sdkKillError = new Error('kill failed') + await expect(spawnE2BTerminal(runtime(waitFailed), spec(), '/runtime/wait-failed')) + .rejects.toThrow('wait failed') + expect(waitFailed.handle.sdkKills).toBe(1) + const cleanupFailed = new FakeTerminalSandbox() cleanupFailed.handle.pid = 0 cleanupFailed.handle.sdkKillError = new Error('kill transport failed') @@ -362,7 +419,7 @@ describe('E2B terminal lifecycle', () => { it.each([ [7, { exitCode: 7, signal: null }], - [143, { exitCode: null, signal: 'SIGTERM' }], + [143, { exitCode: 143, signal: null }], [255, { exitCode: 255, signal: null }], ] as const)('classifies an unrequested command exit %i', async (exitCode, expected) => { const fake = new FakeTerminalSandbox() From 62df6d0c0977a71c689e91abb4b27c6f4c9a3262 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 05:45:55 +0800 Subject: [PATCH 076/108] refactor(e2b): share environment scrubbing --- .../e2b/subprocess-e2b/src/environment.ts | 21 +++++++++++++++++++ packages/e2b/subprocess-e2b/src/process.ts | 12 ++--------- packages/e2b/subprocess-e2b/src/terminal.ts | 13 +++--------- 3 files changed, 26 insertions(+), 20 deletions(-) create mode 100644 packages/e2b/subprocess-e2b/src/environment.ts diff --git a/packages/e2b/subprocess-e2b/src/environment.ts b/packages/e2b/subprocess-e2b/src/environment.ts new file mode 100644 index 0000000000..cc9b41296a --- /dev/null +++ b/packages/e2b/subprocess-e2b/src/environment.ts @@ -0,0 +1,21 @@ +/** Shared remote-environment scrubbing for E2B process and terminal launchers. */ + +import { SENSITIVE_ENV_PATTERN } from '@deepseek-ai/dsh-subprocess' + +/** + * Parse an E2B NUL-delimited environment while removing harness-private and credential-shaped names. + * @param raw - The complete NUL-delimited remote environment. + * @returns Mutable retained entries for the caller to overlay and serialize. + */ +export function scrubRemoteEnvironment(raw: string): Map { + const environment = new Map() + for (const entry of raw.split('\0')) { + if (entry.length === 0) continue + const separator = entry.indexOf('=') + if (separator <= 0) continue + const name = entry.slice(0, separator) + if (name.startsWith('DSH_') || SENSITIVE_ENV_PATTERN.test(name)) continue + environment.set(name, entry.slice(separator + 1)) + } + return environment +} diff --git a/packages/e2b/subprocess-e2b/src/process.ts b/packages/e2b/subprocess-e2b/src/process.ts index 9eeee5d877..5f8a107cf6 100644 --- a/packages/e2b/subprocess-e2b/src/process.ts +++ b/packages/e2b/subprocess-e2b/src/process.ts @@ -8,7 +8,6 @@ import { quoteE2BShellArg, } from '@deepseek-ai/dsh-e2b' import type { CommandHandle, CommandResult, Sandbox } from '@deepseek-ai/dsh-e2b' -import { SENSITIVE_ENV_PATTERN } from '@deepseek-ai/dsh-subprocess' import type { SubprocessCollect, SubprocessHandle, @@ -17,6 +16,7 @@ import type { SubprocessSpawnSpec, } from '@deepseek-ai/dsh-subprocess' import type E2BSandboxService from '@deepseek-ai/dsh-e2b' +import { scrubRemoteEnvironment } from './environment.ts' import { E2BBase64Decoder, E2B_OUTPUT_COMPLETE_FRAME, E2BOutputReader } from './output.ts' const GROUP_POLL_MS = 20 @@ -74,15 +74,7 @@ interface RemotePaths { } function remoteEnvironment(raw: string, explicit: Readonly> | undefined): string { - const environment = new Map() - for (const entry of raw.split('\0')) { - if (entry.length === 0) continue - const separator = entry.indexOf('=') - if (separator <= 0) continue - const name = entry.slice(0, separator) - if (name.startsWith('DSH_') || SENSITIVE_ENV_PATTERN.test(name)) continue - environment.set(name, entry.slice(separator + 1)) - } + const environment = scrubRemoteEnvironment(raw) for (const [name, value] of Object.entries(explicit ?? {})) environment.set(name, value) return [...environment].map(([name, value]) => `${name}=${value}\0`).join('') } diff --git a/packages/e2b/subprocess-e2b/src/terminal.ts b/packages/e2b/subprocess-e2b/src/terminal.ts index 565e406b97..7185899f79 100644 --- a/packages/e2b/subprocess-e2b/src/terminal.ts +++ b/packages/e2b/subprocess-e2b/src/terminal.ts @@ -9,7 +9,7 @@ import { quoteE2BShellArg, } from '@deepseek-ai/dsh-e2b' import type { CommandHandle, CommandResult, Sandbox } from '@deepseek-ai/dsh-e2b' -import { SENSITIVE_ENV_PATTERN, SubprocessTerminalLifecycle } from '@deepseek-ai/dsh-subprocess' +import { SubprocessTerminalLifecycle } from '@deepseek-ai/dsh-subprocess' import type { SubprocessOutcome, SubprocessTerminalForeground, @@ -18,6 +18,7 @@ import type { SubprocessTerminalSpawnSpec, } from '@deepseek-ai/dsh-subprocess' import type E2BSandboxService from '@deepseek-ai/dsh-e2b' +import { scrubRemoteEnvironment } from './environment.ts' const POLL_MS = 20 @@ -71,15 +72,7 @@ function serializeValues(values: readonly string[], kind: string): string { } function remoteEnvironment(raw: string, explicit: Readonly> | undefined): string { - const environment = new Map() - for (const entry of raw.split('\0')) { - if (entry.length === 0) continue - const separator = entry.indexOf('=') - if (separator <= 0) continue - const name = entry.slice(0, separator) - if (name.startsWith('DSH_') || SENSITIVE_ENV_PATTERN.test(name)) continue - environment.set(name, entry.slice(separator + 1)) - } + const environment = scrubRemoteEnvironment(raw) for (const [name, value] of Object.entries(explicit ?? {})) { if (name.length === 0 || name.includes('=') || name.includes('\0') || value.includes('\0')) { throw new Error('subprocess-e2b: terminal environment entries require non-empty NUL-free names without = and NUL-free values') From ec0310ca1b5eca56424c617af7360d40b329ce59 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 07:03:04 +0800 Subject: [PATCH 077/108] fix(e2b): harden remote teardown transactions --- ...026-07-27-e2b-remote-runtime-poc.i18n.yaml | 4 +- .../2026-07-27-e2b-remote-runtime-poc.md | 2 +- .../2026-07-27-e2b-remote-runtime-poc.zh.md | 2 +- .../tests/fixtures/e2b/e2b/bin.ts | 25 ++ packages/e2b/e2b/README.i18n.yaml | 4 +- packages/e2b/e2b/README.md | 2 +- packages/e2b/e2b/README.zh.md | 2 +- packages/e2b/e2b/src/index.ts | 6 +- packages/e2b/e2b/tests/composition.e2e.ts | 29 +- packages/e2b/e2b/tests/e2b.spec.ts | 23 +- packages/e2b/subprocess-e2b/README.i18n.yaml | 4 +- packages/e2b/subprocess-e2b/README.md | 12 +- packages/e2b/subprocess-e2b/README.zh.md | 12 +- packages/e2b/subprocess-e2b/src/index.ts | 6 +- packages/e2b/subprocess-e2b/src/output.ts | 8 +- packages/e2b/subprocess-e2b/src/process.ts | 332 ++++++++++++++---- packages/e2b/subprocess-e2b/src/terminal.ts | 47 +-- .../subprocess-e2b/tests/subprocess.spec.ts | 311 +++++++++++++++- .../e2b/subprocess-e2b/tests/terminal.spec.ts | 16 + 19 files changed, 712 insertions(+), 135 deletions(-) diff --git a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.i18n.yaml b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.i18n.yaml index b5e8e6e47c..079924ac74 100644 --- a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.i18n.yaml +++ b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md -2026-07-27-e2b-remote-runtime-poc.md: 029950cb2dae47bcf1ce819156135e58d46a4bb6 -2026-07-27-e2b-remote-runtime-poc.zh.md: 4029cc85d206e0dd0b7dbd91ec03674062df8531 +2026-07-27-e2b-remote-runtime-poc.md: 02de8df021094fa326431a996cd183d13b9c0c8c +2026-07-27-e2b-remote-runtime-poc.zh.md: 3b7ddf761b2cbf950898a0083cf9da653d20e723 diff --git a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md index 029950cb2d..02de8df021 100644 --- a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md +++ b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md @@ -38,7 +38,7 @@ The POC has no session-persistence backend, template builder, volume, snapshot, Focused package suites pin owner lifecycle cleanup, filesystem paths/containment/bounded descriptor reads and commit metadata, subprocess executable lookup/process groups/publication rollback, terminal byte I/O/signal identity/default-environment scrubbing/session cleanup, output limits, abort ordering, disposal to quiescence, and package-owned invariant registrations. The generic PTY, LSP, and subprocess Code Runtime suites pin their provider-neutral readiness, cross-namespace `processId`, binding bridge, descriptor isolation, hostile traffic, and worker/descendant cleanup behavior. -A credential-gated Loader composition creates real E2B sandboxes and exercises FS-to-Bash and Bash-to-FS visibility, process-publication rollback, byte-split UTF-8 command output, bounded raw spill output, PTY default-secret scrubbing, stale-interrupt identity, and process-tree cleanup, parent-swap-safe bounded LSP source reads, Code Runtime host bindings, descriptor-isolated output accounting, descendant-held pipe cleanup, wall timeout, abort, runner cleanup, host-workspace isolation, and final sandbox deletion. The same composition runs through source imports and built package exports. +A credential-gated Loader composition creates real E2B sandboxes and exercises FS-to-Bash and Bash-to-FS visibility, real-directory runtime-state setup, process-publication rollback, byte-split UTF-8 command output, bounded raw spill and inherited-output draining, PTY default-secret scrubbing, stale-interrupt identity, and process-tree cleanup, parent-swap-safe bounded LSP source reads, Code Runtime host bindings, descriptor-isolated output accounting, descendant-held pipe cleanup, wall timeout, abort, runner cleanup, host-workspace isolation, and final sandbox deletion. The same composition runs through source imports and built package exports. ## Alternatives considered diff --git a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.zh.md b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.zh.md index 4029cc85d2..3b7ddf761b 100644 --- a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.zh.md +++ b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.zh.md @@ -38,7 +38,7 @@ E2B 拥有可变文件系统、受管命令与 Bash 进程、终端分配与终 聚焦包测试套件固定所有者生命周期清理、文件系统路径/containment/有界描述符读取与提交元数据、子进程可执行文件查找/进程组/发布回滚、终端字节 I/O/信号身份/默认环境清理/会话清理、输出上限、中止顺序、等待完全停稳的资源释放,以及包自有不变式注册。通用 PTY、LSP 与子进程 Code Runtime 测试套件固定其提供方无关的就绪判定、跨命名空间 `processId`、绑定桥接、描述符隔离、恶意通信,以及 worker/后代进程清理行为。 -凭据门控的 Loader 组合会创建真实 E2B 沙箱,并演练 FS-to-Bash 与 Bash-to-FS 可见性、进程发布回滚、按字节切分的 UTF-8 命令输出、有界原始 spill 输出、PTY 默认秘密清理、陈旧中断身份与进程树清理、可抵御父目录替换的有界 LSP 源码读取、Code Runtime 宿主绑定、描述符隔离的输出记账、后代进程所持管道的清理、墙钟超时、中止、runner 清理、宿主工作区隔离,以及最终删除沙箱。同一组合分别通过源代码导入与已构建包导出运行。 +凭据门控的 Loader 组合会创建真实 E2B 沙箱,并演练 FS-to-Bash 与 Bash-to-FS 可见性、真实目录形式的运行时状态设置、进程发布回滚、按字节切分的 UTF-8 命令输出、有界原始 spill 与继承输出排空、PTY 默认秘密清理、陈旧中断身份与进程树清理、可抵御父目录替换的有界 LSP 源码读取、Code Runtime 宿主绑定、描述符隔离的输出记账、后代进程所持管道的清理、墙钟超时、中止、runner 清理、宿主工作区隔离,以及最终删除沙箱。同一组合分别通过源代码导入与已构建包导出运行。 ## 曾考虑的替代方案 diff --git a/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts b/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts index 7c5fce5535..6fa95c24b6 100644 --- a/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts +++ b/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts @@ -85,6 +85,30 @@ try { throw new Error(`E2B subprocess corrupted split UTF-8 output: ${JSON.stringify({ splitUtf8Outcome, splitUtf8Output })}`) } + const outputDrainStarted = Date.now() + const outputDrainHandle = ctx.subprocess.spawn({ + argv: ['bash', '-c', "bash -c 'exec -a dsh-output-drain-descendant sleep 30' & printf 'leader-done\\n'"], + cwd: process.cwd(), + stdio: { stdin: 'ignore', stdout: { maxBytes: 64 }, stderr: { maxBytes: 4_096 } }, + graceMs: 250, + env: {}, + }) + const outputDrainOutcome = await outputDrainHandle.done + const outputDrainText = outputDrainHandle.collected.stdout?.readFrom(0).text + const outputDrainElapsedMs = Date.now() - outputDrainStarted + outputDrainHandle.terminate() + const outputDrainExited = await outputDrainHandle.waitForExit(AbortSignal.timeout(5_000)) + const outputDrainProcesses = await sandbox.commands.list() + const outputDrainClean = !outputDrainProcesses.some(processInfo => + JSON.stringify([processInfo.cmd, processInfo.args]).includes('dsh-output-drain-descendant'), + ) + if (outputDrainOutcome.exitCode !== 0 || outputDrainText !== 'leader-done\n' + || outputDrainElapsedMs >= 10_000 || !outputDrainExited || !outputDrainClean) { + throw new Error(`E2B subprocess output drain was not bounded: ${JSON.stringify({ + outputDrainOutcome, outputDrainText, outputDrainElapsedMs, outputDrainExited, outputDrainClean, + })}`) + } + const remoteFiles = sandbox.files as unknown as { read(path: string, options?: unknown): Promise } @@ -384,6 +408,7 @@ try { fsRead, explicitEnvironment, splitUtf8Output, + outputDrain: { outcome: outputDrainOutcome, text: outputDrainText, exited: outputDrainExited, clean: outputDrainClean }, publicationRollback, spill: { liveBytes: liveSpillBytes, outcome: spillOutcome, read: spillRead }, hover, diff --git a/packages/e2b/e2b/README.i18n.yaml b/packages/e2b/e2b/README.i18n.yaml index 229a9e9343..d8f1dcdc3f 100644 --- a/packages/e2b/e2b/README.i18n.yaml +++ b/packages/e2b/e2b/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/e2b/e2b/README.md -README.md: 56606a22c36e65bcc53f2b8cce27f0739da1b8e2 -README.zh.md: 2e3213e431f15f1b22d2b8a429b27ab8a6a78671 +README.md: 01c7ade2f5f31e34091ad0ae910287e659f15549 +README.zh.md: fb95a12cbbbd431ccb3283a14bb63b324f9bdea6 diff --git a/packages/e2b/e2b/README.md b/packages/e2b/e2b/README.md index 56606a22c3..01c7ade2f5 100644 --- a/packages/e2b/e2b/README.md +++ b/packages/e2b/e2b/README.md @@ -28,7 +28,7 @@ Set `sandboxId` to reconnect a running or paused sandbox instead of creating one ## Lifecycle and ownership -Construction starts one create/connect operation. Before resolving `getSandbox()`, the service creates `cwd` and the private `cwd/.dsh-e2b` adapter-state directory, then sets that directory to mode `0700`. `sandboxId` resolves to a branded `E2BSandboxId` after setup. +Construction starts one create/connect operation. Before resolving `getSandbox()`, the service creates `cwd` and the private `cwd/.dsh-e2b` adapter-state directory, verifies that the reserved path is a real directory rather than a symlink or another file type, then sets it to mode `0700`. `sandboxId` resolves to a branded `E2BSandboxId` after setup. Disposal first prevents new handle acquisition, then awaits setup and applies exactly one configured disposition. A `SandboxNotFoundError` means a kill-on-timeout sandbox is already quiescent; every other disposition failure rejects teardown. A newly created sandbox is killed when initial directory setup fails; a reconnected sandbox is not killed on setup failure because the service did not create it. Provider plugins must load after this owner and dispose before it. diff --git a/packages/e2b/e2b/README.zh.md b/packages/e2b/e2b/README.zh.md index 2e3213e431..fb95a12cbb 100644 --- a/packages/e2b/e2b/README.zh.md +++ b/packages/e2b/e2b/README.zh.md @@ -28,7 +28,7 @@ ## 生命周期与所有权 -构造阶段会启动一次 create/connect 操作。服务在 `getSandbox()` 结算前创建 `cwd` 和私有的 `cwd/.dsh-e2b` 适配器状态目录,再把该目录的 mode 设为 `0700`。初始化完成后,`sandboxId` 会结算为品牌类型 `E2BSandboxId`。 +构造阶段会启动一次 create/connect 操作。服务在 `getSandbox()` 结算前创建 `cwd` 和私有的 `cwd/.dsh-e2b` 适配器状态目录,验证该预留路径是真实目录而非符号链接或其他文件类型,再把该目录的 mode 设为 `0700`。初始化完成后,`sandboxId` 会结算为品牌类型 `E2BSandboxId`。 资源释放会先阻止继续获取新句柄,再等待初始化完成,并且只应用一种已配置的处置方式。`SandboxNotFoundError` 表示因超时终止的沙箱已经完全停稳;其他处置失败都会使 teardown 拒绝。新建沙箱的初始目录设置失败时,服务会终止该沙箱;重新连接的沙箱设置失败时不会被终止,因为它不是由本服务创建的。提供方插件必须在该所有者之后加载,并在其之前 dispose(资源释放)。 diff --git a/packages/e2b/e2b/src/index.ts b/packages/e2b/e2b/src/index.ts index 96da912033..2c0b3c635c 100644 --- a/packages/e2b/e2b/src/index.ts +++ b/packages/e2b/e2b/src/index.ts @@ -7,7 +7,7 @@ import { posix } from 'node:path' import { Context, Service } from 'cordis' import z from 'schemastery' -import { Sandbox, SandboxNotFoundError } from 'e2b' +import { FileType, Sandbox, SandboxNotFoundError } from 'e2b' import type { Branded } from '@deepseek-ai/dsh-brand' export { @@ -232,6 +232,10 @@ export class E2BSandboxService extends Service { try { await sandbox.files.makeDir(this.cwd) await sandbox.files.makeDir(this.runtimeRoot) + const runtimeRoot = await sandbox.files.getInfo(this.runtimeRoot) + if (runtimeRoot.type !== FileType.DIR || runtimeRoot.symlinkTarget !== undefined) { + throw new Error(`dsh-e2b: runtime root must be a real directory: ${this.runtimeRoot}`) + } await sandbox.commands.run(`chmod 700 -- ${quoteE2BShellArg(this.runtimeRoot)}`) return sandbox } catch (error: unknown) { diff --git a/packages/e2b/e2b/tests/composition.e2e.ts b/packages/e2b/e2b/tests/composition.e2e.ts index 4e7be57c45..1ece44bb6a 100644 --- a/packages/e2b/e2b/tests/composition.e2e.ts +++ b/packages/e2b/e2b/tests/composition.e2e.ts @@ -5,7 +5,7 @@ import { Context } from 'cordis' import { describe, expect, it } from 'vitest' import type { Agent } from '@deepseek-ai/dsh-agent' import { runLoaderSmoke } from '@deepseek-ai/dsh-loader-smoke' -import { Sandbox, SandboxNotFoundError } from '@deepseek-ai/dsh-e2b' +import E2BSandboxService, { Sandbox, SandboxNotFoundError } from '@deepseek-ai/dsh-e2b' import PtyService, { PtySessionId } from '@deepseek-ai/dsh-pty' import { LocalPtyBackend } from '@deepseek-ai/dsh-pty-local' import { Session, SessionId } from '@deepseek-ai/dsh-session' @@ -92,6 +92,27 @@ describe.skipIf(!process.env.E2B_API_KEY)('E2B live Loader composition', () => { await session.close('environment test complete') await subprocessFiber.dispose() await ptyFiber.dispose() + + await sandbox.commands.run([ + 'rm -rf -- /home/user/.dsh-e2b /home/user/dsh-e2b-runtime-target', + 'mkdir -p -- /home/user/dsh-e2b-runtime-target', + 'chmod 755 -- /home/user/dsh-e2b-runtime-target', + 'ln -s -- /home/user/dsh-e2b-runtime-target /home/user/.dsh-e2b', + ].join('\n')) + const linkedCtx = new Context() + const linkedFiber = await linkedCtx.plugin(E2BSandboxService, { + apiKey, + sandboxId: sandbox.sandboxId, + cwd: '/home/user', + onDispose: 'leave', + }) + try { + await expect(linkedCtx.e2b.getSandbox()).rejects.toThrow('runtime root must be a real directory') + const target = await sandbox.files.getInfo('/home/user/dsh-e2b-runtime-target') + expect(target.mode & 0o777).toBe(0o755) + } finally { + await linkedFiber.dispose() + } } finally { await sandbox.kill().catch(() => false) } @@ -123,6 +144,12 @@ describe.skipIf(!process.env.E2B_API_KEY)('E2B live Loader composition', () => { fsRead: 'written-by-bash\n', explicitEnvironment: true, splitUtf8Output: '你好', + outputDrain: { + outcome: { exitCode: 0, signal: null }, + text: 'leader-done\n', + exited: true, + clean: true, + }, publicationRollback: true, spill: { liveBytes: 6, diff --git a/packages/e2b/e2b/tests/e2b.spec.ts b/packages/e2b/e2b/tests/e2b.spec.ts index bb96ca806a..ff90c37b3a 100644 --- a/packages/e2b/e2b/tests/e2b.spec.ts +++ b/packages/e2b/e2b/tests/e2b.spec.ts @@ -3,6 +3,7 @@ import { Context } from 'cordis' import type { Sandbox as SandboxType } from 'e2b' import E2BSandboxService, { E2BSandboxId, + FileType, SandboxNotFoundError, quoteE2BShellArg, } from '@deepseek-ai/dsh-e2b' @@ -33,6 +34,7 @@ vi.mock('e2b', async (importOriginal) => { interface SandboxFixture { sandbox: SandboxType makeDir: ReturnType + getInfo: ReturnType run: ReturnType kill: ReturnType pause: ReturnType @@ -40,17 +42,18 @@ interface SandboxFixture { function fakeSandbox(id = 'sandbox-1'): SandboxFixture { const makeDir = vi.fn().mockResolvedValue(true) + const getInfo = vi.fn().mockResolvedValue({ type: FileType.DIR }) const run = vi.fn().mockResolvedValue({ exitCode: 0, stdout: '', stderr: '' }) const kill = vi.fn().mockResolvedValue(undefined) const pause = vi.fn().mockResolvedValue(true) const sandbox = { sandboxId: id, - files: { makeDir }, + files: { makeDir, getInfo }, commands: { run }, kill, pause, } as unknown as SandboxType - return { sandbox, makeDir, run, kill, pause } + return { sandbox, makeDir, getInfo, run, kill, pause } } beforeEach(() => { @@ -82,6 +85,7 @@ describe('E2BSandboxService', () => { }) expect(fixture.makeDir).toHaveBeenNthCalledWith(1, '/home/user/workspace') expect(fixture.makeDir).toHaveBeenNthCalledWith(2, '/home/user/workspace/.dsh-e2b') + expect(fixture.getInfo).toHaveBeenCalledWith('/home/user/workspace/.dsh-e2b') expect(fixture.run).toHaveBeenCalledWith("chmod 700 -- '/home/user/workspace/.dsh-e2b'") await fiber.dispose() @@ -220,6 +224,21 @@ describe('E2BSandboxService', () => { expect(fixture.kill).not.toHaveBeenCalled() }) + it.each([ + ['symbolic link', { type: FileType.DIR, symlinkTarget: '/tmp/redirected' }], + ['regular file', { type: FileType.FILE }], + ])('rejects a reserved runtime root that is a %s', async (_label, info) => { + const fixture = fakeSandbox() + fixture.getInfo.mockResolvedValueOnce(info) + sdk.connect.mockResolvedValue(fixture.sandbox) + const ctx = new Context() + await ctx.plugin(E2BSandboxService, { apiKey: 'test-key', sandboxId: 'existing' }) + + await expect(ctx.e2b.getSandbox()).rejects.toThrow('runtime root must be a real directory') + expect(fixture.run).not.toHaveBeenCalled() + expect(fixture.kill).not.toHaveBeenCalled() + }) + it.each([ [{ apiKey: '' }, /configure apiKey/], [{ apiKey: 'x', cwd: 'relative' }, /absolute Linux path/], diff --git a/packages/e2b/subprocess-e2b/README.i18n.yaml b/packages/e2b/subprocess-e2b/README.i18n.yaml index 33a36141d0..091a6972cf 100644 --- a/packages/e2b/subprocess-e2b/README.i18n.yaml +++ b/packages/e2b/subprocess-e2b/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/e2b/subprocess-e2b/README.md -README.md: 58f13b264324d2168d4bc264b60100cdafa0a366 -README.zh.md: d547e4e8d66f629edcef753651180fe1806ad9bf +README.md: b1aa480305fef9ca485340b6bdae50f9fcffd88f +README.zh.md: b4824c90e7fb908d9c12e77c19d2b952055dcb40 diff --git a/packages/e2b/subprocess-e2b/README.md b/packages/e2b/subprocess-e2b/README.md index 58f13b2643..b1aa480305 100644 --- a/packages/e2b/subprocess-e2b/README.md +++ b/packages/e2b/subprocess-e2b/README.md @@ -6,14 +6,14 @@ E2B implementation of the [`@deepseek-ai/dsh-subprocess`](../../subprocess/subpr ## Behavior -- **Asynchronous remote start** — the synchronous seam returns a handle immediately while `Sandbox.commands.run(..., { background: true })` starts remotely. `pid` is `-1` until the wrapper publishes and the adapter validates its process-group id; `done`, stdin, termination, and `waitForExit()` wait for readiness internally. +- **Asynchronous remote start** — the synchronous seam returns a handle immediately while `Sandbox.commands.run(..., { background: true })` starts remotely. `pid` is `-1` until the wrapper publishes and the adapter validates its process-group id; stdin and ordinary observation wait for that publication, while cancellation can stop the provisional SDK handle first. - **Execution-world coordinates** — `cwd` and private `runtimeRoot` come from the shared owner; executable lookup verifies absolute paths or resolves a bare name against the sandbox PATH plus explicit overrides. -- **Linux process groups** — a quoted wrapper starts each argv under `exec setsid --wait` and records its actual process-group id plus private status files beneath `ctx.e2b.runtimeRoot/processes`. The handle waits for that file instead of treating the SDK command PID as its published identity. Termination signals the negative recorded id with `SIGTERM`, waits the caller's `graceMs`, then escalates to `SIGKILL` and the SDK kill fallback. If publication fails, the SDK PID remains the provisional `exec setsid` group id; rollback kills and verifies that group before startup rejects. Service disposal terminates and joins every retained handle before the sandbox owner disposes. -- **Environment boundary** — the wrapper starts from the sandbox command environment, removes ambient `DSH_*` and credential-shaped (`*KEY*`, `*SECRET*`, `*TOKEN*`) names, then restores every `spec.env` entry as an explicit caller opt-in. Host ambient variables never enter the sandbox implicitly. -- **Stdio projection** — the remote wrapper branches raw bytes into optional bounded spill files, frames each live chunk as newline-delimited base64 ASCII, and the host incrementally restores bytes across arbitrary SDK callback boundaries. Pipe mode writes those bytes to host Node streams; inherit mode writes them to the harness process streams; collect mode retains a bounded host tail with offset reads. Batch and streaming stdin use the SDK handle. -- **Terminal sessions** — `spawnTerminal()` uses E2B's byte PTY API, installs the exact argv and scrubbed environment through private mode-`0600` files, reports the foreground process group, sends real signals, and tears down every group in the remote terminal session before settlement. Prompt detection, scrollback, readiness, and owner policy remain in `dsh-pty-local`. +- **Linux process groups** — a quoted wrapper starts each argv under `exec setsid --wait` and records its actual process-group id plus private status files beneath `ctx.e2b.runtimeRoot/processes`. The handle waits for that file instead of treating the SDK command PID as its published identity. Termination signals the negative recorded id with `SIGTERM`, waits the caller's `graceMs`, then escalates to `SIGKILL` and the SDK kill fallback; TERM delivery or probe failures also force that escalation. A failed transaction is observable through `waitForExit()` and may be retried. Before publication, cancellation uses the provisional SDK handle; if publication fails, rollback kills and verifies the provisional group before startup rejects. Service disposal terminates and joins every retained handle before the sandbox owner disposes. +- **Environment boundary** — the wrapper starts from the sandbox command environment, removes ambient `DSH_*` and credential-shaped (`*KEY*`, `*SECRET*`, `*TOKEN*`) names, then restores every `spec.env` entry as an explicit caller opt-in. Host ambient variables never enter the sandbox implicitly. Private environment files are removed after consumption, and failed command or terminal setup removes its private state before rejecting. +- **Stdio projection** — the remote wrapper branches raw bytes into optional bounded spill files, frames each live chunk as newline-delimited base64 ASCII, and the host incrementally restores bytes across arbitrary SDK callback boundaries. Pipe mode writes those bytes to host Node streams; inherit mode writes them to the harness process streams; collect mode retains a bounded host tail with offset reads. The wrapper publishes the direct command status before waiting for inherited writers; after `graceMs`, the adapter disconnects an incomplete SDK stream, withholds its partial spill, and returns that status while retaining the remote group for `waitForExit()` and termination. Batch and streaming stdin use the SDK handle. +- **Terminal sessions** — `spawnTerminal()` uses E2B's byte PTY API, installs the exact argv and scrubbed environment through private mode-`0600` files, reports the foreground process group, sends real signals, and tears down every group in the remote terminal session before settlement. Setup and teardown own the private state transaction, including failure cleanup. Prompt detection, scrollback, readiness, and owner policy remain in `dsh-pty-local`. -The base E2B image supplies the runtime and Bash/GNU utilities this adapter invokes: `node`, `bash`, `setsid`, `ps`, `awk`, `tr`, `env`, `chmod`, `tee`, `head`, and `kill`. A custom template must retain compatible commands and E2B PTY support. +The base E2B image supplies the runtime and Bash/GNU utilities this adapter invokes: `node`, `bash`, `setsid`, `ps`, `awk`, `tr`, `env`, `chmod`, `tee`, `head`, `rm`, and `kill`. A custom template must retain compatible commands and E2B PTY support. ## Model Experience diff --git a/packages/e2b/subprocess-e2b/README.zh.md b/packages/e2b/subprocess-e2b/README.zh.md index d547e4e8d6..b4824c90e7 100644 --- a/packages/e2b/subprocess-e2b/README.zh.md +++ b/packages/e2b/subprocess-e2b/README.zh.md @@ -6,14 +6,14 @@ ## 行为 -- **异步远程启动**:同步 seam 会立即返回一个句柄,同时由 `Sandbox.commands.run(..., { background: true })` 在远程启动进程。包装层发布进程组 ID 并由适配器完成验证之前,`pid` 为 `-1`;`done`、stdin、终止和 `waitForExit()` 会在内部等待就绪。 +- **异步远程启动**:同步 seam 会立即返回一个句柄,同时由 `Sandbox.commands.run(..., { background: true })` 在远程启动进程。包装层发布进程组 ID 并由适配器完成验证之前,`pid` 为 `-1`;stdin 和常规观察会等待该发布,而取消操作可以先停止临时 SDK 句柄。 - **执行世界坐标**:`cwd` 和私有 `runtimeRoot` 来自共享所有者;可执行文件查找会验证绝对路径,或根据沙箱 PATH 加显式覆盖来解析裸名称。 -- **Linux 进程组**:带引号保护的包装层会在 `exec setsid --wait` 下启动每组 argv,并在 `ctx.e2b.runtimeRoot/processes` 下记录实际进程组 ID 和私有状态文件。句柄会等待该文件,而不会把 SDK 命令 PID 当作已发布的身份。终止操作以记录的负数 ID 发送 `SIGTERM`,等待调用方的 `graceMs`,再升级到 `SIGKILL` 和 SDK kill 回退。如果发布失败,SDK PID 仍为临时的 `exec setsid` 进程组 ID;回滚会终止并验证该进程组,随后启动操作才会以拒绝结束。服务 dispose(资源释放)会在沙箱所有者释放前终止并等待每个保留句柄退出。 -- **环境边界**:包装层从沙箱命令环境开始,移除环境中的 `DSH_*` 和形似凭据的名称(`*KEY*`、`*SECRET*`、`*TOKEN*`),再把每个 `spec.env` 条目恢复为调用方显式选择。宿主环境变量绝不会隐式进入沙箱。 -- **stdio 投影**:远程包装层先把原始字节分流到可选的有界 spill 文件,再把每个实时分片编码为换行分隔的 base64 ASCII 帧;宿主会跨任意 SDK 回调边界增量恢复字节。pipe 模式把这些字节写入宿主 Node 流;inherit 模式把字节写入 harness 进程流;collect 模式保留有界的宿主尾部,并支持基于偏移量读取。批量 stdin 和流式 stdin 都使用 SDK 句柄。 -- **终端会话**:`spawnTerminal()` 使用 E2B 的字节 PTY API,以 mode 为 `0600` 的私有文件传入原样 argv 与清理后的环境,报告前台进程组,发送真实信号,并在结算前清理远程终端会话中的每个进程组。提示符检测、scrollback、就绪状态与所有者策略仍归 `dsh-pty-local` 所有。 +- **Linux 进程组**:带引号保护的包装层会在 `exec setsid --wait` 下启动每组 argv,并在 `ctx.e2b.runtimeRoot/processes` 下记录实际进程组 ID 和私有状态文件。句柄会等待该文件,而不会把 SDK 命令 PID 当作已发布的身份。终止操作以记录的负数 ID 发送 `SIGTERM`,等待调用方的 `graceMs`,再升级到 `SIGKILL` 和 SDK kill 回退;TERM 信号发送或探测失败也会强制触发该升级。失败的事务可通过 `waitForExit()` 观察,并可重试。发布前,取消操作使用临时 SDK 句柄;如果发布失败,回滚会终止并验证临时进程组,随后启动操作才会拒绝。服务 dispose(资源释放)会在沙箱所有者释放前终止并等待每个保留句柄退出。 +- **环境边界**:包装层从沙箱命令环境开始,移除环境中的 `DSH_*` 和形似凭据的名称(`*KEY*`、`*SECRET*`、`*TOKEN*`),再把每个 `spec.env` 条目恢复为调用方显式选择。宿主环境变量绝不会隐式进入沙箱。私有环境文件在使用后会被删除;命令或终端设置失败时,会先删除其私有状态再拒绝。 +- **stdio 投影**:远程包装层先把原始字节分流到可选的有界 spill 文件,再把每个实时分片编码为换行分隔的 base64 ASCII 帧;宿主会跨任意 SDK 回调边界增量恢复字节。pipe 模式把这些字节写入宿主 Node 流;inherit 模式把字节写入 harness 进程流;collect 模式保留有界的宿主尾部,并支持基于偏移量读取。包装层会在等待继承管道的写入方之前发布直接命令状态;超过 `graceMs` 后,适配器会断开未完成的 SDK 流,不公开其中不完整的 spill,并返回该状态,同时保留远程进程组供 `waitForExit()` 和终止操作使用。批量 stdin 和流式 stdin 都使用 SDK 句柄。 +- **终端会话**:`spawnTerminal()` 使用 E2B 的字节 PTY API,以 mode 为 `0600` 的私有文件传入原样 argv 与清理后的环境,报告前台进程组,发送真实信号,并在结算前清理远程终端会话中的每个进程组。setup 与 teardown 负责私有状态事务,包括失败清理。提示符检测、scrollback、就绪状态与所有者策略仍归 `dsh-pty-local` 所有。 -基础 E2B 镜像提供该适配器调用的运行时和 Bash/GNU 工具:`node`、`bash`、`setsid`、`ps`、`awk`、`tr`、`env`、`chmod`、`tee`、`head` 和 `kill`。自定义模板必须保留兼容的命令和 E2B PTY 支持。 +基础 E2B 镜像提供该适配器调用的运行时和 Bash/GNU 工具:`node`、`bash`、`setsid`、`ps`、`awk`、`tr`、`env`、`chmod`、`tee`、`head`、`rm` 和 `kill`。自定义模板必须保留兼容的命令和 E2B PTY 支持。 ## 模型体验 diff --git a/packages/e2b/subprocess-e2b/src/index.ts b/packages/e2b/subprocess-e2b/src/index.ts index 86ca5ceada..27efb216da 100644 --- a/packages/e2b/subprocess-e2b/src/index.ts +++ b/packages/e2b/subprocess-e2b/src/index.ts @@ -46,14 +46,12 @@ export class E2BSubprocessService extends SubprocessService { const pending: Promise[] = [] for (const handle of handles) { handle.terminate() - pending.push(handle.done.catch(() => {}).then(() => handle.waitForExit())) + pending.push(handle.waitForExit().then(() => { this.live.delete(handle) })) } for (const terminal of terminals) { terminal.terminate() - pending.push(terminal.waitForExit()) + pending.push(terminal.waitForExit().then(() => { this.terminals.delete(terminal) })) } - this.live.clear() - this.terminals.clear() await Promise.all(pending) }, 'e2b subprocess teardown') } diff --git a/packages/e2b/subprocess-e2b/src/output.ts b/packages/e2b/subprocess-e2b/src/output.ts index 6cb61e4f09..8afcd138dc 100644 --- a/packages/e2b/subprocess-e2b/src/output.ts +++ b/packages/e2b/subprocess-e2b/src/output.ts @@ -66,6 +66,7 @@ export class E2BOutputReader implements SubprocessOutputReader { private chunks: Buffer[] = [] private retainedBytes = 0 private totalBytes = 0 + private spillValid = true /** * Create a bounded reader over one remote spill path. @@ -84,6 +85,11 @@ export class E2BOutputReader implements SubprocessOutputReader { return this.totalBytes } + /** Stop advertising a remote spill whose writer did not reach clean EOF. */ + invalidateSpill(): void { + this.spillValid = false + } + /** * Append one byte-faithful decoded transport event. * @param bytes - Raw command bytes recovered from the ASCII SDK transport. @@ -120,7 +126,7 @@ export class E2BOutputReader implements SubprocessOutputReader { text: retained.subarray(start).toString('utf8'), nextOffset: this.totalBytes, lossy, - ...(lossy && this.maxSpillBytes !== undefined && this.totalBytes <= this.maxSpillBytes + ...(lossy && this.spillValid && this.maxSpillBytes !== undefined && this.totalBytes <= this.maxSpillBytes ? { spillPath: this.spillPath } : {}), } diff --git a/packages/e2b/subprocess-e2b/src/process.ts b/packages/e2b/subprocess-e2b/src/process.ts index 5f8a107cf6..3a8047eb54 100644 --- a/packages/e2b/subprocess-e2b/src/process.ts +++ b/packages/e2b/subprocess-e2b/src/process.ts @@ -5,6 +5,7 @@ import { PassThrough, Writable } from 'node:stream' import { posix } from 'node:path' import { CommandExitError, + FileNotFoundError, quoteE2BShellArg, } from '@deepseek-ai/dsh-e2b' import type { CommandHandle, CommandResult, Sandbox } from '@deepseek-ai/dsh-e2b' @@ -41,6 +42,10 @@ function hasSpill(mode: SubprocessOutputMode): mode is SubprocessCollect & { spi return isCollect(mode) && mode.spill !== undefined } +function isValidProcessId(value: number): boolean { + return Number.isSafeInteger(value) && value > 0 +} + function asError(error: unknown): Error { return error instanceof Error ? error : new Error(String(error)) } @@ -73,6 +78,20 @@ interface RemotePaths { stderr: string } +type CommandSettlement = + | { kind: 'result'; result: CommandResult } + | { kind: 'error'; error: unknown } + +function withinMs(promise: Promise, timeoutMs: number): Promise { + return new Promise((resolve) => { + const timer = setTimeout(() => { resolve(undefined) }, timeoutMs) + void promise.then((value) => { + clearTimeout(timer) + resolve(value) + }) + }) +} + function remoteEnvironment(raw: string, explicit: Readonly> | undefined): string { const environment = scrubRemoteEnvironment(raw) for (const [name, value] of Object.entries(explicit ?? {})) environment.set(name, value) @@ -96,15 +115,16 @@ function commandText(spec: SubprocessSpawnSpec, paths: RemotePaths): string { 'dsh_e2b_tr=$4', 'dsh_e2b_tee=$5', 'dsh_e2b_head=$6', - 'shift 6', + 'dsh_e2b_rm=$7', + 'shift 7', 'dsh_e2b_pgid="$("$dsh_e2b_ps" -o pgid= -p "$$" | "$dsh_e2b_tr" -d " ")"', `printf '%s\\n' "$dsh_e2b_pgid" > ${quoteE2BShellArg(paths.pid)}`, `mapfile -d '' -t dsh_e2b_env < ${quoteE2BShellArg(paths.environment)}`, - `: > ${quoteE2BShellArg(paths.environment)}`, + `"$dsh_e2b_rm" -f -- ${quoteE2BShellArg(paths.environment)}`, `"$dsh_e2b_env_bin" -i "\${dsh_e2b_env[@]}" "$@" ${stdoutRedirect} ${stderrRedirect}`.trimEnd(), 'dsh_e2b_status=$?', - 'wait', `printf '%s\\n' "$dsh_e2b_status" > ${quoteE2BShellArg(paths.status)}`, + 'wait', 'exit "$dsh_e2b_status"', ].join('\n') const argv = spec.argv.map(quoteE2BShellArg).join(' ') @@ -118,10 +138,11 @@ function commandText(spec: SubprocessSpawnSpec, paths: RemotePaths): string { 'dsh_e2b_tr="$(command -v tr)"', 'dsh_e2b_tee="$(command -v tee)"', 'dsh_e2b_head="$(command -v head)"', - 'for dsh_e2b_tool in "$dsh_e2b_env_bin" "$dsh_e2b_setsid" "$dsh_e2b_bash" "$dsh_e2b_node" "$dsh_e2b_ps" "$dsh_e2b_tr" "$dsh_e2b_tee" "$dsh_e2b_head"; do', + 'dsh_e2b_rm="$(command -v rm)"', + 'for dsh_e2b_tool in "$dsh_e2b_env_bin" "$dsh_e2b_setsid" "$dsh_e2b_bash" "$dsh_e2b_node" "$dsh_e2b_ps" "$dsh_e2b_tr" "$dsh_e2b_tee" "$dsh_e2b_head" "$dsh_e2b_rm"; do', ' [[ "$dsh_e2b_tool" == /* && -x "$dsh_e2b_tool" ]] || exit 125', 'done', - `exec "$dsh_e2b_env_bin" -i "\${dsh_e2b_env[@]}" "$dsh_e2b_setsid" --wait -- "$dsh_e2b_bash" -c ${quoteE2BShellArg(inner)} dsh-e2b "$dsh_e2b_env_bin" "$dsh_e2b_node" "$dsh_e2b_ps" "$dsh_e2b_tr" "$dsh_e2b_tee" "$dsh_e2b_head" ${argv}`, + `exec "$dsh_e2b_env_bin" -i "\${dsh_e2b_env[@]}" "$dsh_e2b_setsid" --wait -- "$dsh_e2b_bash" -c ${quoteE2BShellArg(inner)} dsh-e2b "$dsh_e2b_env_bin" "$dsh_e2b_node" "$dsh_e2b_ps" "$dsh_e2b_tr" "$dsh_e2b_tee" "$dsh_e2b_head" "$dsh_e2b_rm" ${argv}`, ].join('\n') return bootstrap } @@ -149,6 +170,26 @@ function waitTick(signal?: AbortSignal): Promise { }) } +const WAIT_ABORTED = Symbol('wait aborted') + +function waitWithSignal(promise: Promise, signal: AbortSignal | undefined): Promise { + if (signal === undefined) return promise + if (signal.aborted) return Promise.resolve(WAIT_ABORTED) + return new Promise((resolve, reject) => { + const onAbort = (): void => { cleanup(); resolve(WAIT_ABORTED) } + const cleanup = (): void => { signal.removeEventListener('abort', onAbort) } + signal.addEventListener('abort', onAbort, { once: true }) + if (signal.aborted) { + onAbort() + return + } + void promise.then( + (value) => { cleanup(); resolve(value) }, + (error: unknown) => { cleanup(); reject(asError(error)) }, + ) + }) +} + /** E2B-backed subprocess handle with deferred remote PID acquisition. */ export class E2BSubprocessHandle implements SubprocessHandle { readonly stdin: Writable | undefined @@ -157,6 +198,7 @@ export class E2BSubprocessHandle implements SubprocessHandle { readonly collected: SubprocessHandle['collected'] readonly done: Promise + private readonly commandState = Promise.withResolvers() private readonly readyState = Promise.withResolvers() private readonly stdoutDecoder = new E2BBase64Decoder() private readonly stderrDecoder = new E2BBase64Decoder() @@ -166,9 +208,14 @@ export class E2BSubprocessHandle implements SubprocessHandle { private remotePid = -1 private commandHandle: CommandHandle | undefined private outputTransportError: Error | undefined - private terminationRequested = false + private outputDrainExpired = false + private stateDirectoryCreated = false + private invalidHandleQuiescent = false + private provisionalHandleQuiescent = false + private terminationStarted = false + private terminationAttempt: Promise | undefined + private terminationFailure: Error | undefined private terminationSignal: NodeJS.Signals | null = null - private termination: Promise | undefined /** * Begin an E2B command without blocking the synchronous subprocess spawn seam. @@ -217,22 +264,47 @@ export class E2BSubprocessHandle implements SubprocessHandle { /** @inheritdoc */ terminate(): void { - if (this.terminationRequested) return - this.terminationRequested = true - this.termination = this.terminateRemote() - void this.termination.catch(() => {}) + if (this.terminationAttempt !== undefined) return + this.terminationStarted = true + this.terminationFailure = undefined + const attempt = this.terminateRemote() + this.terminationAttempt = attempt + void attempt.then( + () => { + this.terminationAttempt = undefined + }, + (error: unknown) => { + this.terminationFailure = asError(error) + this.terminationAttempt = undefined + }, + ) } /** @inheritdoc */ async waitForExit(signal?: AbortSignal): Promise { let handle: CommandHandle | undefined - try { - handle = await this.readyForWait(signal) - } catch { - handle = this.commandHandle + if (this.terminationStarted) { + const observed = await waitWithSignal(this.commandState.promise, signal) + if (observed === WAIT_ABORTED) return false + handle = observed if (handle === undefined) return true + if (this.remotePid <= 0) { + const attempt = this.terminationAttempt + if (attempt !== undefined && await waitWithSignal(attempt, signal) === WAIT_ABORTED) return false + this.throwTerminationFailure() + if (this.invalidHandleQuiescent || this.provisionalHandleQuiescent) return true + } + } else { + try { + const observed = await waitWithSignal(this.readyState.promise, signal) + if (observed === WAIT_ABORTED) return false + handle = observed + } catch { + handle = this.commandHandle + if (handle === undefined) return true + } } - if (handle === undefined) return false + this.throwTerminationFailure() let sandbox: Sandbox try { sandbox = await this.runtime.getSandbox() @@ -242,33 +314,19 @@ export class E2BSubprocessHandle implements SubprocessHandle { } const processGroupId = this.remotePid > 0 ? this.remotePid : handle.pid while (await this.groupAlive(sandbox, processGroupId, signal)) { + this.throwTerminationFailure() if (!await waitTick(signal)) return false } + this.throwTerminationFailure() return !isAborted(signal) } - private readyForWait(signal: AbortSignal | undefined): Promise { - if (signal === undefined) return this.readyState.promise - return new Promise((resolve, reject) => { - const onAbort = (): void => { cleanup(); resolve(undefined) } - const cleanup = (): void => { signal.removeEventListener('abort', onAbort) } - signal.addEventListener('abort', onAbort, { once: true }) - if (signal.aborted) { - onAbort() - return - } - void this.readyState.promise.then( - (handle) => { cleanup(); resolve(handle) }, - (error: unknown) => { cleanup(); reject(asError(error)) }, - ) - }) - } - private readonly onAbort = (): void => { this.terminate() } private async run(): Promise { + let sandbox: Sandbox | undefined try { - const sandbox = await this.runtime.getSandbox() + sandbox = await this.runtime.getSandbox() await this.prepareState(sandbox) const handle = await sandbox.commands.run( commandText(this.spec, this.paths), @@ -281,12 +339,26 @@ export class E2BSubprocessHandle implements SubprocessHandle { onStderr: async (data) => { await this.dispatchOutput('stderr', data) }, }, ) - if (!Number.isSafeInteger(handle.pid) || handle.pid <= 0) { - throw new Error(`subprocess-e2b: E2B returned invalid command pid ${handle.pid}`) - } this.commandHandle = handle const completion = handle.wait() void completion.catch(() => {}) + if (!isValidProcessId(handle.pid)) { + const invalidPid = new Error(`subprocess-e2b: E2B returned invalid command pid ${handle.pid}`) + try { + await handle.kill() + this.invalidHandleQuiescent = true + this.commandHandle = undefined + } catch (cleanupError: unknown) { + this.terminationFailure = asError(cleanupError) + this.commandState.resolve(handle) + throw new AggregateError( + [invalidPid, cleanupError], + 'subprocess-e2b: invalid command pid rollback did not reach quiescence', + ) + } + throw invalidPid + } + this.commandState.resolve(handle) try { this.remotePid = await this.waitForProcessGroupId(sandbox, completion) } catch (error: unknown) { @@ -302,16 +374,28 @@ export class E2BSubprocessHandle implements SubprocessHandle { } this.readyState.resolve(handle) await this.writeBatchStdin(handle) - const outcome = await this.waitForCommand(completion) + const outcome = await this.waitForCommand(sandbox, handle, completion) if (this.outputTransportError !== undefined) throw this.outputTransportError - const requireCompleteOutput = this.terminationSignal === null + const requireCompleteOutput = this.terminationSignal === null && !this.outputDrainExpired this.stdoutDecoder.finish(requireCompleteOutput) this.stderrDecoder.finish(requireCompleteOutput) await this.finalizeSpills(sandbox) return outcome } catch (error: unknown) { - this.readyState.reject(error) - throw error + this.commandState.resolve(undefined) + let failure = error + if (sandbox !== undefined && this.stateDirectoryCreated) { + try { + await this.removeFailedState(sandbox) + } catch (cleanupError: unknown) { + failure = new AggregateError( + [error, cleanupError], + 'subprocess-e2b: command failed and private state cleanup failed', + ) + } + } + this.readyState.reject(failure) + throw failure } finally { this.spec.signal?.removeEventListener('abort', this.onAbort) this.stdout?.end() @@ -322,6 +406,7 @@ export class E2BSubprocessHandle implements SubprocessHandle { private async prepareState(sandbox: Sandbox): Promise { const ambient = await sandbox.commands.run('env -0') await sandbox.files.makeDir(this.stateDir) + this.stateDirectoryCreated = true await sandbox.commands.run(`chmod 700 -- ${quoteE2BShellArg(this.stateDir)}`) const files = [ { path: this.paths.pid, data: '' }, @@ -405,20 +490,45 @@ export class E2BSubprocessHandle implements SubprocessHandle { } } - private async waitForCommand(completion: Promise): Promise { - try { - const result = await completion - return { exitCode: result.exitCode, signal: null } - } catch (error: unknown) { - if (error instanceof CommandExitError) { - return this.terminationSignal === null - ? { exitCode: error.exitCode, signal: null } - : { exitCode: null, signal: this.terminationSignal } + private async waitForCommand( + sandbox: Sandbox, + handle: CommandHandle, + completion: Promise, + ): Promise { + const settlement = completion.then( + result => ({ kind: 'result', result }), + (error: unknown) => ({ kind: 'error', error }), + ) + while (true) { + const rawStatus = (await sandbox.files.read(this.paths.status)).trim() + if (rawStatus.length > 0) { + const exitCode = Number(rawStatus) + if (!/^(?:0|[1-9][0-9]*)$/.test(rawStatus) || !Number.isSafeInteger(exitCode) || exitCode > 255) { + throw new Error(`subprocess-e2b: remote wrapper published invalid exit code ${JSON.stringify(rawStatus)}`) + } + const completed = await withinMs(settlement, this.spec.graceMs) + if (completed !== undefined) return this.commandOutcome(completed) + this.outputDrainExpired = true + this.stdoutReader?.invalidateSpill() + this.stderrReader?.invalidateSpill() + await handle.disconnect() + return { exitCode, signal: null } } - throw error + const completed = await Promise.race([settlement, waitTick().then(() => undefined)]) + if (completed !== undefined) return this.commandOutcome(completed) } } + private commandOutcome(settlement: CommandSettlement): SubprocessOutcome { + if (settlement.kind === 'result') return { exitCode: settlement.result.exitCode, signal: null } + if (settlement.error instanceof CommandExitError) { + return this.terminationSignal === null + ? { exitCode: settlement.error.exitCode, signal: null } + : { exitCode: null, signal: this.terminationSignal } + } + throw settlement.error + } + private async rollbackUnpublishedGroup(sandbox: Sandbox, handle: CommandHandle): Promise { // The bootstrap ends in an exec chain through the scrubbed environment and // `setsid`, so E2B's command PID is the provisional group id even before the @@ -433,35 +543,99 @@ export class E2BSubprocessHandle implements SubprocessHandle { } private async terminateRemote(): Promise { - let handle: CommandHandle | undefined - try { - handle = await this.readyState.promise - } catch { - handle = this.commandHandle - } + const handle = await this.commandState.promise if (handle === undefined) return - const sandbox = await this.runtime.getSandbox() - const processGroupId = this.remotePid > 0 ? this.remotePid : handle.pid - this.terminationSignal = 'SIGTERM' - await this.signalGroup(sandbox, processGroupId, 'TERM') - const deadline = Date.now() + this.spec.graceMs - while (Date.now() < deadline && await this.groupAlive(sandbox, processGroupId)) { - await waitTick() + if (!isValidProcessId(handle.pid) && this.remotePid <= 0) { + await handle.kill() + this.invalidHandleQuiescent = true + this.commandHandle = undefined + return } - if (!await this.groupAlive(sandbox, processGroupId)) return - this.terminationSignal = 'SIGKILL' + if (this.remotePid <= 0) { + const sandbox = await this.runtime.getSandbox() + this.terminationSignal = 'SIGTERM' + try { + const delivered = await this.signalGroup(sandbox, handle.pid, 'TERM') + if (delivered) { + const deadline = Date.now() + this.spec.graceMs + while (Date.now() < deadline && await this.groupAlive(sandbox, handle.pid)) await waitTick() + if (!await this.groupAlive(sandbox, handle.pid)) { + this.provisionalHandleQuiescent = true + return + } + } + } catch (_gracefulTerminationFailure) { + // A missing or unobservable provisional group still has the SDK handle fallback. + } + this.terminationSignal = 'SIGKILL' + let groupDelivered = false + let groupFailure: unknown + try { + groupDelivered = await this.signalGroup(sandbox, handle.pid, 'KILL') + } catch (error: unknown) { + groupFailure = error + } + let handleFailure: unknown + try { + await handle.kill() + this.provisionalHandleQuiescent = true + } catch (error: unknown) { + handleFailure = error + } + if (!groupDelivered && handleFailure !== undefined) { + throw new AggregateError( + [...(groupFailure === undefined ? [] : [groupFailure]), handleFailure], + 'subprocess-e2b: force termination failed through both process-group and SDK transports', + ) + } + return + } + const sandbox = await this.runtime.getSandbox() + const processGroupId = this.remotePid + this.terminationSignal = 'SIGTERM' try { - await this.signalGroup(sandbox, processGroupId, 'KILL') - } finally { - await handle.kill().catch(() => false) + await this.signalGroup(sandbox, processGroupId, 'TERM') + const deadline = Date.now() + this.spec.graceMs + while (Date.now() < deadline && await this.groupAlive(sandbox, processGroupId)) { + await waitTick() + } + if (!await this.groupAlive(sandbox, processGroupId)) return + } catch (_gracefulTerminationFailure) { + // Failed TERM delivery or observation cannot prove exit; force cleanup still owns the group. + } + this.terminationSignal = 'SIGKILL' + let groupFailure: unknown + let groupDelivered = false + try { + groupDelivered = await this.signalGroup(sandbox, processGroupId, 'KILL') + } catch (error: unknown) { + groupFailure = error + } + let handleFailure: unknown + try { + await handle.kill() + } catch (error: unknown) { + handleFailure = error + } + if (!groupDelivered && handleFailure !== undefined && await this.groupAlive(sandbox, processGroupId)) { + throw new AggregateError( + [...(groupFailure === undefined ? [] : [groupFailure]), handleFailure], + 'subprocess-e2b: force termination failed through both process-group and SDK transports', + ) } } - private async signalGroup(sandbox: Sandbox, pid: number, signal: 'TERM' | 'KILL'): Promise { + private throwTerminationFailure(): void { + if (this.terminationFailure !== undefined) throw this.terminationFailure + } + + private async signalGroup(sandbox: Sandbox, pid: number, signal: 'TERM' | 'KILL'): Promise { try { await sandbox.commands.run(`kill -${signal} -- -${pid}`) + return true } catch (error: unknown) { - if (!(error instanceof CommandExitError)) throw error + if (error instanceof CommandExitError) return false + throw error } } @@ -482,7 +656,7 @@ export class E2BSubprocessHandle implements SubprocessHandle { if (!hasSpill(mode)) return // A spill mode is a collect mode, so construction always created its reader. const size = (reader as E2BOutputReader).size - if (size <= mode.maxBytes || size > mode.spill.maxBytes) { + if (this.outputDrainExpired || size <= mode.maxBytes || size > mode.spill.maxBytes) { removals.push(sandbox.files.remove(path).catch(() => {})) } } @@ -490,4 +664,18 @@ export class E2BSubprocessHandle implements SubprocessHandle { collect(this.spec.stdio.stderr, this.stderrReader, this.paths.stderr) await Promise.all(removals) } + + private async removeFailedState(sandbox: Sandbox): Promise { + const failures: Error[] = [] + for (const path of [this.paths.environment, this.stateDir]) { + try { + await sandbox.files.remove(path) + } catch (error: unknown) { + if (!(error instanceof FileNotFoundError)) failures.push(asError(error)) + } + } + if (failures.length > 0) { + throw new AggregateError(failures, 'subprocess-e2b: failed to remove private command state') + } + } } diff --git a/packages/e2b/subprocess-e2b/src/terminal.ts b/packages/e2b/subprocess-e2b/src/terminal.ts index 7185899f79..fb8dac6f49 100644 --- a/packages/e2b/subprocess-e2b/src/terminal.ts +++ b/packages/e2b/subprocess-e2b/src/terminal.ts @@ -376,25 +376,26 @@ export async function spawnE2BTerminal( argv: posix.join(stateDir, 'argv'), ready: posix.join(stateDir, 'ready'), } - const ambient = await sandbox.commands.run('env -0', signalOpts(spec.signal)) - const environment = remoteEnvironment(ambient.stdout, spec.env) - const argv = serializeValues(spec.argv, 'argv') - await sandbox.files.makeDir(stateDir) - await sandbox.commands.run(`chmod 700 -- ${quoteE2BShellArg(stateDir)}`, signalOpts(spec.signal)) - await sandbox.files.write([ - { path: paths.runner, data: TERMINAL_RUNNER_SOURCE }, - { path: paths.environment, data: environment }, - { path: paths.argv, data: argv }, - ], signalOpts(spec.signal)) - await sandbox.commands.run( - `chmod 600 -- ${quoteE2BShellArg(paths.runner)} ${quoteE2BShellArg(paths.environment)} ${quoteE2BShellArg(paths.argv)}`, - signalOpts(spec.signal), - ) - const output = new PassThrough() let handle: CommandHandle | undefined let completion: Promise | undefined + let stateDirectoryCreated = false try { + const ambient = await sandbox.commands.run('env -0', signalOpts(spec.signal)) + const environment = remoteEnvironment(ambient.stdout, spec.env) + const argv = serializeValues(spec.argv, 'argv') + await sandbox.files.makeDir(stateDir) + stateDirectoryCreated = true + await sandbox.commands.run(`chmod 700 -- ${quoteE2BShellArg(stateDir)}`, signalOpts(spec.signal)) + await sandbox.files.write([ + { path: paths.runner, data: TERMINAL_RUNNER_SOURCE }, + { path: paths.environment, data: environment }, + { path: paths.argv, data: argv }, + ], signalOpts(spec.signal)) + await sandbox.commands.run( + `chmod 600 -- ${quoteE2BShellArg(paths.runner)} ${quoteE2BShellArg(paths.environment)} ${quoteE2BShellArg(paths.argv)}`, + signalOpts(spec.signal), + ) handle = await sandbox.pty.create({ rows: spec.rows, cols: spec.cols, @@ -425,19 +426,25 @@ export async function spawnE2BTerminal( ) } catch (error: unknown) { output.destroy() - let cleanupError: Error | undefined + const cleanupErrors: Error[] = [] if (handle !== undefined && completion !== undefined) { try { await rollbackUnpublishedTerminal(sandbox, handle, completion, spec.graceMs) } catch (rollbackError: unknown) { - cleanupError = asError(rollbackError) + cleanupErrors.push(asError(rollbackError)) } } else if (handle !== undefined) { await handle.kill().catch(() => false) } - await sandbox.files.remove(stateDir).catch(() => {}) - if (cleanupError !== undefined) { - throw new AggregateError([asError(error), cleanupError], asError(error).message) + if (stateDirectoryCreated) { + try { + await sandbox.files.remove(stateDir) + } catch (stateError: unknown) { + if (!(stateError instanceof FileNotFoundError)) cleanupErrors.push(asError(stateError)) + } + } + if (cleanupErrors.length > 0) { + throw new AggregateError([asError(error), ...cleanupErrors], asError(error).message) } throw error } diff --git a/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts b/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts index 32adf872b3..d81b49a600 100644 --- a/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts +++ b/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts @@ -2,6 +2,7 @@ import { once } from 'node:events' import { Context } from 'cordis' import { CommandExitError, + FileNotFoundError, type CommandHandle, type CommandResult, type Sandbox, @@ -35,10 +36,14 @@ class FakeCommandHandle { readonly sent: Array = [] closes = 0 kills = 0 + disconnects = 0 killError: unknown + disconnectError: unknown private readonly result = Promise.withResolvers() private settled = false + constructor(private readonly onKill: () => void = () => {}) {} + wait(): Promise { return this.result.promise } @@ -54,9 +59,15 @@ class FakeCommandHandle { async kill(): Promise { this.kills += 1 if (this.killError !== undefined) throw this.killError + this.onKill() return true } + async disconnect(): Promise { + this.disconnects += 1 + if (this.disconnectError !== undefined) throw this.disconnectError + } + succeed(exitCode = 0): void { if (this.settled) return this.settled = true @@ -77,7 +88,7 @@ class FakeCommandHandle { } class FakeSandbox { - readonly handle = new FakeCommandHandle() + readonly handle: FakeCommandHandle readonly commandsSeen: string[] = [] readonly writtenFiles: string[][] = [] readonly writtenFileData = new Map() @@ -85,19 +96,35 @@ class FakeSandbox { readonly directories: string[] = [] startOptions: StartOptions | undefined backgroundError: unknown + envError: unknown nextRemoveError: unknown probeError: unknown signalError: unknown + readonly signalErrors: unknown[] = [] trapsTerm = false delaysKill = false + sdkKillStops = true alive = true ambient = 'PATH=/ambient/bin\0KEEP=safe\0NPM_TOKEN=secret\0DSH_STALE=old\0BROKEN\0=bad\0' processGroupId = '4242\n' + exitStatus = '' readonly processGroupReads: string[] = [] + afterStatusRead: (() => void) | undefined beforeProbe: (() => void) | undefined afterProbe: (() => void) | undefined private startGate: Promise | undefined private openStart: (() => void) | undefined + private processGroupReadGate: Promise | undefined + private openProcessGroupRead: (() => void) | undefined + + constructor() { + this.handle = new FakeCommandHandle(() => { + if (this.sdkKillStops) { + this.alive = false + this.handle.fail(137) + } + }) + } deferStart(): void { const gate = Promise.withResolvers() @@ -109,6 +136,16 @@ class FakeSandbox { this.openStart?.() } + deferProcessGroupRead(): void { + const gate = Promise.withResolvers() + this.processGroupReadGate = gate.promise + this.openProcessGroupRead = () => { gate.resolve(undefined) } + } + + releaseProcessGroupRead(): void { + this.openProcessGroupRead?.() + } + finish(exitCode = 0): void { this.alive = false void this.completeOutput().then( @@ -155,7 +192,14 @@ class FakeSandbox { for (const file of files) this.writtenFileData.set(file.path, file.data) return files.map(() => ({})) }, - read: async (): Promise => this.processGroupReads.shift() ?? this.processGroupId, + read: async (path: string): Promise => { + if (!path.endsWith('/exit-code')) { + await this.processGroupReadGate + return this.processGroupReads.shift() ?? this.processGroupId + } + this.afterStatusRead?.() + return this.exitStatus + }, remove: async (path: string): Promise => { this.removed.push(path) if (this.nextRemoveError !== undefined) { @@ -168,7 +212,10 @@ class FakeSandbox { commands: { run: async (command: string, options?: StartOptions | { signal?: AbortSignal }): Promise => { this.commandsSeen.push(command) - if (command === 'env -0') return { exitCode: 0, stdout: this.ambient, stderr: '' } + if (command === 'env -0') { + if (this.envError !== undefined) throw this.envError + return { exitCode: 0, stdout: this.ambient, stderr: '' } + } if (command.startsWith('kill -0 ')) { this.beforeProbe?.() if (options?.signal?.aborted === true) throw new DOMException('aborted', 'AbortError') @@ -182,9 +229,9 @@ class FakeSandbox { return { exitCode: 0, stdout: '', stderr: '' } } if (command.startsWith('kill -TERM ')) { - if (this.signalError !== undefined) { - const error = this.signalError - this.signalError = undefined + const error = this.signalErrors.shift() ?? this.signalError + if (error !== undefined) { + if (this.signalErrors.length === 0) this.signalError = undefined throw error } if (!this.trapsTerm) { @@ -194,9 +241,9 @@ class FakeSandbox { return { exitCode: 0, stdout: '', stderr: '' } } if (command.startsWith('kill -KILL ')) { - if (this.signalError !== undefined) { - const error = this.signalError - this.signalError = undefined + const error = this.signalErrors.shift() ?? this.signalError + if (error !== undefined) { + if (this.signalErrors.length === 0) this.signalError = undefined throw error } if (!this.delaysKill) this.alive = false @@ -277,6 +324,8 @@ describe('E2BOutputReader', () => { expect(reader.readFrom(2)).toEqual({ text: 'cdef', nextOffset: 6, lossy: false }) expect(reader.readFrom(5)).toEqual({ text: 'f', nextOffset: 6, lossy: false }) expect(reader.readFrom(99)).toEqual({ text: '', nextOffset: 6, lossy: false }) + reader.invalidateSpill() + expect(reader.readFrom(0)).toEqual({ text: 'cdef', nextOffset: 6, lossy: true }) }) it('drops whole head chunks and withholds absent or over-cap spills', () => { @@ -390,6 +439,50 @@ describe('E2BSubprocessHandle', () => { await expect(handle.done).rejects.toThrow('incomplete output transport') }) + it('bounds descendant-held output draining and withholds the incomplete spill', async () => { + const fake = new FakeSandbox() + const handle = new E2BSubprocessHandle(runtime(fake), spec({ graceMs: 5 }), '/runtime/drain-bound') + await flush() + await fake.stdout('leader-output') + fake.exitStatus = '0\n' + + await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) + expect(fake.handle.disconnects).toBe(1) + expect(handle.collected.stdout?.readFrom(0)).toEqual({ + text: 'tput', + nextOffset: 13, + lossy: true, + }) + expect(fake.removed).toContain('/runtime/drain-bound/stdout.log') + + handle.terminate() + await expect(handle.waitForExit()).resolves.toBe(true) + }) + + it('accepts clean encoder completion inside the output-drain grace', async () => { + const fake = new FakeSandbox() + const handle = new E2BSubprocessHandle(runtime(fake), spec({ graceMs: 100 }), '/runtime/drain-complete') + await flush() + fake.exitStatus = '0\n' + fake.afterStatusRead = () => { + fake.afterStatusRead = undefined + setTimeout(() => { fake.finish() }, 0) + } + + await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) + expect(fake.handle.disconnects).toBe(0) + }) + + it('rejects an invalid direct-command exit status', async () => { + const fake = new FakeSandbox() + const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/invalid-status') + await flush() + fake.exitStatus = '999\n' + await expect(handle.done).rejects.toThrow('invalid exit code') + handle.terminate() + await expect(handle.waitForExit()).resolves.toBe(true) + }) + it('surfaces deferred piped-stdin write and close failures as stream errors', async () => { const writeFake = new FakeSandbox() writeFake.deferStart() @@ -510,6 +603,97 @@ describe('E2BSubprocessHandle', () => { await expect(handle.done).resolves.toEqual({ exitCode: null, signal: 'SIGTERM' }) }) + it('kills through the provisional SDK handle before process-group publication', async () => { + const fake = new FakeSandbox() + fake.deferProcessGroupRead() + fake.signalErrors.push(commandError(1), commandError(1)) + const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/pre-publication-kill') + await vi.waitFor(() => { expect(fake.startOptions).toBeDefined() }) + + handle.terminate() + await vi.waitFor(() => { expect(fake.handle.kills).toBe(1) }) + expect(fake.alive).toBe(false) + await expect(handle.waitForExit()).resolves.toBe(true) + + fake.releaseProcessGroupRead() + await expect(handle.done).resolves.toEqual({ exitCode: null, signal: 'SIGKILL' }) + }) + + it('bounds a quiescence observer while provisional termination is awaiting the controller', async () => { + const fake = new FakeSandbox() + fake.deferProcessGroupRead() + const reconnect = Promise.withResolvers() + let calls = 0 + const delayedRuntime = runtime(fake, async () => { + calls += 1 + return calls === 1 ? fake.sandbox : await reconnect.promise + }) + const handle = new E2BSubprocessHandle(delayedRuntime, spec(), '/runtime/pre-publication-observer') + await vi.waitFor(() => { expect(fake.startOptions).toBeDefined() }) + handle.terminate() + + const controller = new AbortController() + const waiting = handle.waitForExit(controller.signal) + await flush() + controller.abort() + await expect(waiting).resolves.toBe(false) + + reconnect.resolve(fake.sandbox) + await expect(handle.waitForExit()).resolves.toBe(true) + fake.releaseProcessGroupRead() + await handle.done + }) + + it('proves a provisional group exit when the SDK kill fallback fails', async () => { + const fake = new FakeSandbox() + fake.deferProcessGroupRead() + fake.trapsTerm = true + fake.handle.killError = new Error('SDK kill unavailable') + const handle = new E2BSubprocessHandle(runtime(fake), spec({ graceMs: 1 }), '/runtime/pre-publication-group-kill') + await vi.waitFor(() => { expect(fake.startOptions).toBeDefined() }) + + handle.terminate() + await expect(handle.waitForExit()).resolves.toBe(true) + fake.releaseProcessGroupRead() + await expect(handle.done).resolves.toEqual({ exitCode: null, signal: 'SIGKILL' }) + }) + + it('reports failed provisional group and SDK force transports', async () => { + const fake = new FakeSandbox() + fake.deferProcessGroupRead() + fake.signalErrors.push(new Error('TERM transport failed'), new Error('KILL transport failed')) + fake.handle.killError = new Error('SDK kill failed') + const handle = new E2BSubprocessHandle(runtime(fake), spec({ graceMs: 1 }), '/runtime/pre-publication-failure') + await vi.waitFor(() => { expect(fake.startOptions).toBeDefined() }) + + handle.terminate() + await expect(handle.waitForExit()).rejects.toThrow('force termination failed through both') + + fake.handle.killError = undefined + handle.terminate() + await expect(handle.waitForExit()).resolves.toBe(true) + fake.releaseProcessGroupRead() + await handle.done + + const absentGroup = new FakeSandbox() + absentGroup.deferProcessGroupRead() + absentGroup.signalErrors.push(commandError(1), commandError(1)) + absentGroup.handle.killError = new Error('SDK kill failed without a provisional group') + const absentHandle = new E2BSubprocessHandle( + runtime(absentGroup), + spec({ graceMs: 1 }), + '/runtime/pre-publication-absent-group', + ) + await vi.waitFor(() => { expect(absentGroup.startOptions).toBeDefined() }) + absentHandle.terminate() + await expect(absentHandle.waitForExit()).rejects.toThrow('force termination failed through both') + absentGroup.handle.killError = undefined + absentHandle.terminate() + await expect(absentHandle.waitForExit()).resolves.toBe(true) + absentGroup.releaseProcessGroupRead() + await absentHandle.done + }) + it('honors an already-aborted signal when constructing the asynchronous handle directly', async () => { const fake = new FakeSandbox() const handle = new E2BSubprocessHandle(runtime(fake), spec({ signal: AbortSignal.abort('stop') }), '/runtime/pre-aborted') @@ -559,6 +743,17 @@ describe('E2BSubprocessHandle', () => { await expect(liveWait).resolves.toBe(false) fake.finish() await handle.done + + const terminatingFake = new FakeSandbox() + terminatingFake.deferStart() + const terminating = new E2BSubprocessHandle(runtime(terminatingFake), spec(), '/runtime/wait-termination-start') + terminating.terminate() + const beforeHandle = new AbortController() + const handlePending = terminating.waitForExit(beforeHandle.signal) + beforeHandle.abort() + await expect(handlePending).resolves.toBe(false) + terminatingFake.releaseStart() + await terminating.done }) it('bounds both sides of the liveness-poll abort race', async () => { @@ -580,6 +775,14 @@ describe('E2BSubprocessHandle', () => { const duringProbe = new AbortController() fake.beforeProbe = () => { duringProbe.abort(); fake.beforeProbe = undefined } await expect(handle.waitForExit(duringProbe.signal)).resolves.toBe(false) + + let racedAbort = false + const raceSignal = { + get aborted() { return racedAbort }, + addEventListener: () => { racedAbort = true }, + removeEventListener: () => {}, + } as unknown as AbortSignal + await expect(handle.waitForExit(raceSignal)).resolves.toBe(false) fake.finish() await handle.done }) @@ -599,8 +802,36 @@ describe('E2BSubprocessHandle', () => { const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/fail') await expect(handle.done).rejects.toThrow('start failed') expect(handle.pid).toBe(-1) + expect(fake.removed).toContain('/runtime/fail/environment') + expect(fake.removed).toContain('/runtime/fail') await expect(handle.waitForExit()).resolves.toBe(true) handle.terminate() + + const unavailableHandle = new E2BSubprocessHandle( + runtime(new FakeSandbox(), async () => { throw new Error('sandbox unavailable') }), + spec(), + '/runtime/unavailable-start', + ) + await expect(unavailableHandle.done).rejects.toThrow('sandbox unavailable') + await expect(unavailableHandle.waitForExit()).resolves.toBe(true) + + const envFailure = new FakeSandbox() + envFailure.envError = new Error('ambient lookup failed') + const envHandle = new E2BSubprocessHandle(runtime(envFailure), spec(), '/runtime/env-failure') + await expect(envHandle.done).rejects.toThrow('ambient lookup failed') + expect(envFailure.removed).toEqual([]) + + const cleanupFailure = new FakeSandbox() + cleanupFailure.backgroundError = new Error('start failed before credential consumption') + cleanupFailure.nextRemoveError = new Error('credential cleanup failed') + const cleanupHandle = new E2BSubprocessHandle(runtime(cleanupFailure), spec(), '/runtime/cleanup-failure') + await expect(cleanupHandle.done).rejects.toThrow('command failed and private state cleanup failed') + + const absentState = new FakeSandbox() + absentState.backgroundError = new Error('start failed after external cleanup') + absentState.nextRemoveError = new FileNotFoundError('already removed') + const absentHandle = new E2BSubprocessHandle(runtime(absentState), spec(), '/runtime/absent-state') + await expect(absentHandle.done).rejects.toThrow('start failed after external cleanup') }) it('bounds a readiness rejection with a still-live caller signal', async () => { @@ -678,8 +909,20 @@ describe('E2BSubprocessHandle', () => { invalidPid.handle.pid = 0 const invalid = new E2BSubprocessHandle(runtime(invalidPid), spec(), '/runtime/invalid-pid') await expect(invalid.done).rejects.toThrow(/invalid command pid 0/) + expect(invalidPid.handle.kills).toBe(1) + expect(invalidPid.removed).toContain('/runtime/invalid-pid/environment') await expect(invalid.waitForExit()).resolves.toBe(true) + const failedRollback = new FakeSandbox() + failedRollback.handle.pid = 0 + failedRollback.handle.killError = new Error('invalid handle kill failed') + const retained = new E2BSubprocessHandle(runtime(failedRollback), spec(), '/runtime/invalid-pid-retained') + await expect(retained.done).rejects.toThrow('invalid command pid rollback did not reach quiescence') + await expect(retained.waitForExit()).rejects.toThrow('invalid handle kill failed') + failedRollback.handle.killError = undefined + retained.terminate() + await expect(retained.waitForExit()).resolves.toBe(true) + const crashedFake = new FakeSandbox() const crashed = new E2BSubprocessHandle(runtime(crashedFake), spec(), '/runtime/crashed') await flush() @@ -692,6 +935,7 @@ describe('E2BSubprocessHandle', () => { const invalidGroup = new FakeSandbox() invalidGroup.processGroupId = 'not-a-pid\n' invalidGroup.delaysKill = true + invalidGroup.sdkKillStops = false invalidGroup.afterProbe = () => { invalidGroup.alive = false } const invalid = new E2BSubprocessHandle(runtime(invalidGroup), spec(), '/runtime/invalid-group') await expect(invalid.done).rejects.toThrow(/invalid process-group id/) @@ -791,7 +1035,7 @@ describe('E2BSubprocessHandle', () => { await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) }) - it('contains an already-gone group signal and observes non-command signal failures', async () => { + it('contains an already-gone group signal and escalates after a TERM transport failure', async () => { const gone = new FakeSandbox() gone.trapsTerm = true gone.signalError = commandError(1) @@ -805,9 +1049,35 @@ describe('E2BSubprocessHandle', () => { const failedHandle = new E2BSubprocessHandle(runtime(failed), spec(), '/runtime/failed-signal') await flush() failedHandle.terminate() + await expect(failedHandle.done).resolves.toEqual({ exitCode: null, signal: 'SIGKILL' }) + expect(failed.commandsSeen).toContain('kill -KILL -- -4242') + }) + + it('allows termination retry after both force transports fail', async () => { + const fake = new FakeSandbox() + fake.signalErrors.push(new Error('TERM transport failed'), new Error('KILL transport failed')) + fake.handle.killError = new Error('SDK kill failed') + const handle = new E2BSubprocessHandle(runtime(fake), spec({ graceMs: 1 }), '/runtime/retry-signal') await flush() - failed.finish() - await expect(failedHandle.done).resolves.toEqual({ exitCode: 0, signal: null }) + + handle.terminate() + await expect(handle.waitForExit()).rejects.toThrow('force termination failed through both') + fake.handle.killError = undefined + handle.terminate() + await expect(handle.done).resolves.toEqual({ exitCode: null, signal: 'SIGTERM' }) + expect(fake.commandsSeen.filter(command => command.startsWith('kill -TERM '))).toHaveLength(2) + + const missingGroup = new FakeSandbox() + missingGroup.trapsTerm = true + missingGroup.signalErrors.push(undefined, commandError(1)) + missingGroup.handle.killError = new Error('SDK kill failed after group exit race') + const raced = new E2BSubprocessHandle(runtime(missingGroup), spec({ graceMs: 1 }), '/runtime/group-exit-race') + await flush() + raced.terminate() + await expect(raced.waitForExit()).rejects.toThrow('force termination failed through both') + missingGroup.handle.killError = undefined + raced.terminate() + await expect(raced.waitForExit()).resolves.toBe(true) }) }) @@ -833,6 +1103,23 @@ describe('E2BSubprocessService', () => { expect(fake.alive).toBe(false) }) + it('reports a failed termination transaction from disposal instead of waiting on done', async () => { + const fake = new FakeSandbox() + fake.signalErrors.push(new Error('TERM transport failed'), new Error('KILL transport failed')) + fake.handle.killError = new Error('SDK kill failed') + const { ctx, fiber } = await service(fake) + const handle = ctx.subprocess.spawn(spec({ graceMs: 1 })) + await flush() + + await expect(fiber.dispose()).resolves.toBeUndefined() + await expect(handle.waitForExit()).rejects.toThrow('force termination failed through both') + + fake.handle.killError = undefined + handle.terminate() + await expect(handle.waitForExit()).resolves.toBe(true) + await expect(handle.done).resolves.toEqual({ exitCode: null, signal: 'SIGTERM' }) + }) + it('releases naturally settled handles before later service disposal', async () => { const fake = new FakeSandbox() const { ctx, fiber } = await service(fake) diff --git a/packages/e2b/subprocess-e2b/tests/terminal.spec.ts b/packages/e2b/subprocess-e2b/tests/terminal.spec.ts index cefa432acd..bda01a5a61 100644 --- a/packages/e2b/subprocess-e2b/tests/terminal.spec.ts +++ b/packages/e2b/subprocess-e2b/tests/terminal.spec.ts @@ -95,6 +95,7 @@ class FakeTerminalSandbox { foreground = '456\n' groups = [123] createError: unknown + writeError: unknown sendError: unknown commandFailure: unknown sessionGroupsFailure: unknown @@ -116,6 +117,7 @@ class FakeTerminalSandbox { }, write: async (files: Array<{ path: string; data: string }>): Promise => { for (const file of files) this.writes.set(file.path, file.data) + if (this.writeError !== undefined) throw this.writeError return files.map(() => ({})) }, read: async (): Promise => { @@ -278,6 +280,20 @@ describe('E2B terminal allocation', () => { }) it('cleans malformed handles, bootstrap failures, and readiness failures', async () => { + const failedState = new FakeTerminalSandbox() + failedState.writeError = new Error('state write failed') + await expect(spawnE2BTerminal(runtime(failedState), spec(), '/runtime/state-write')) + .rejects.toThrow('state write failed') + expect(failedState.writes.get('/runtime/state-write/environment')).toContain('KEEP=visible\0') + expect(failedState.removed).toContain('/runtime/state-write') + expect(failedState.createOptions).toBeUndefined() + + const stateAlreadyGone = new FakeTerminalSandbox() + stateAlreadyGone.writeError = new Error('state write failed after external cleanup') + stateAlreadyGone.removeError = new FileNotFoundError('state already gone') + await expect(spawnE2BTerminal(runtime(stateAlreadyGone), spec(), '/runtime/state-gone')) + .rejects.toThrow('state write failed after external cleanup') + const invalidPid = new FakeTerminalSandbox() invalidPid.handle.pid = 0 await expect(spawnE2BTerminal(runtime(invalidPid), spec(), '/runtime/invalid-pid')) From e9bef2967dcd6c399637482f8cb1739cf90bf6e0 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 07:07:31 +0800 Subject: [PATCH 078/108] fix(e2b): preserve signals after output drain --- packages/e2b/subprocess-e2b/src/process.ts | 4 +++- .../subprocess-e2b/tests/subprocess.spec.ts | 22 ++++++++++++++++++- 2 files changed, 24 insertions(+), 2 deletions(-) diff --git a/packages/e2b/subprocess-e2b/src/process.ts b/packages/e2b/subprocess-e2b/src/process.ts index 3a8047eb54..473c1a0af4 100644 --- a/packages/e2b/subprocess-e2b/src/process.ts +++ b/packages/e2b/subprocess-e2b/src/process.ts @@ -512,7 +512,9 @@ export class E2BSubprocessHandle implements SubprocessHandle { this.stdoutReader?.invalidateSpill() this.stderrReader?.invalidateSpill() await handle.disconnect() - return { exitCode, signal: null } + return this.terminationSignal === null + ? { exitCode, signal: null } + : { exitCode: null, signal: this.terminationSignal } } const completed = await Promise.race([settlement, waitTick().then(() => undefined)]) if (completed !== undefined) return this.commandOutcome(completed) diff --git a/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts b/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts index d81b49a600..457e68f24d 100644 --- a/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts +++ b/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts @@ -103,6 +103,7 @@ class FakeSandbox { readonly signalErrors: unknown[] = [] trapsTerm = false delaysKill = false + delaysKillCompletion = false sdkKillStops = true alive = true ambient = 'PATH=/ambient/bin\0KEEP=safe\0NPM_TOKEN=secret\0DSH_STALE=old\0BROKEN\0=bad\0' @@ -247,7 +248,7 @@ class FakeSandbox { throw error } if (!this.delaysKill) this.alive = false - this.handle.fail(137) + if (!this.delaysKillCompletion) this.handle.fail(137) return { exitCode: 0, stdout: '', stderr: '' } } if ((options as StartOptions | undefined)?.background === true) { @@ -473,6 +474,25 @@ describe('E2BSubprocessHandle', () => { expect(fake.handle.disconnects).toBe(0) }) + it('preserves a requested signal when output draining expires', async () => { + const fake = new FakeSandbox() + fake.trapsTerm = true + fake.delaysKill = true + fake.delaysKillCompletion = true + fake.sdkKillStops = false + const handle = new E2BSubprocessHandle(runtime(fake), spec({ graceMs: 5 }), '/runtime/drain-signal') + await flush() + + handle.terminate() + await vi.waitFor(() => { expect(fake.commandsSeen).toContain('kill -KILL -- -4242') }) + fake.exitStatus = '143\n' + + await expect(handle.done).resolves.toEqual({ exitCode: null, signal: 'SIGKILL' }) + expect(fake.handle.disconnects).toBe(1) + fake.alive = false + await expect(handle.waitForExit()).resolves.toBe(true) + }) + it('rejects an invalid direct-command exit status', async () => { const fake = new FakeSandbox() const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/invalid-status') From 022841027fb05a18c5567095d6fed88aebb9ad00 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 07:42:25 +0800 Subject: [PATCH 079/108] fix(e2b): fence teardown and terminal publication --- packages/e2b/e2b/tests/composition.e2e.ts | 5 +- packages/e2b/subprocess-e2b/README.i18n.yaml | 4 +- packages/e2b/subprocess-e2b/README.md | 4 +- packages/e2b/subprocess-e2b/README.zh.md | 4 +- packages/e2b/subprocess-e2b/src/index.ts | 42 +++++++-- packages/e2b/subprocess-e2b/src/process.ts | 4 +- packages/e2b/subprocess-e2b/src/terminal.ts | 82 ++++++++++++++++- .../subprocess-e2b/tests/subprocess.spec.ts | 25 ++++++ .../e2b/subprocess-e2b/tests/terminal.spec.ts | 89 ++++++++++++++++++- 9 files changed, 237 insertions(+), 22 deletions(-) diff --git a/packages/e2b/e2b/tests/composition.e2e.ts b/packages/e2b/e2b/tests/composition.e2e.ts index 1ece44bb6a..34875d5153 100644 --- a/packages/e2b/e2b/tests/composition.e2e.ts +++ b/packages/e2b/e2b/tests/composition.e2e.ts @@ -185,7 +185,10 @@ describe.skipIf(!process.env.E2B_API_KEY)('E2B live Loader composition', () => { oversizedReply: { error: { kind: 'worker-exit' } }, lingeringCodeRunners: 0, }) - expect((output.terminal as { motd: string }).motd.length).toBeGreaterThan(0) + const terminalMotd = (output.terminal as { motd: string }).motd + expect(terminalMotd.length).toBeGreaterThan(0) + expect(terminalMotd).not.toContain('exec /bin/bash') + expect(terminalMotd).not.toContain('.dsh-e2b/terminals/') expect((output.terminal as { echo: { viewport: string } }).echo.viewport).toContain('PTY-你好') expect((output.terminal as { scrollback: string }).scrollback).toContain('PTY-你好') expect((output.terminal as { signal: { targetPgid: number } }).signal.targetPgid).toBeGreaterThan(0) diff --git a/packages/e2b/subprocess-e2b/README.i18n.yaml b/packages/e2b/subprocess-e2b/README.i18n.yaml index 091a6972cf..2d20e4ad4c 100644 --- a/packages/e2b/subprocess-e2b/README.i18n.yaml +++ b/packages/e2b/subprocess-e2b/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/e2b/subprocess-e2b/README.md -README.md: b1aa480305fef9ca485340b6bdae50f9fcffd88f -README.zh.md: b4824c90e7fb908d9c12e77c19d2b952055dcb40 +README.md: 03146e02cc76b78fbb93faec7d36d1865418b670 +README.zh.md: 6ed49a355586c6df2a594f83b687e6a565d07cf0 diff --git a/packages/e2b/subprocess-e2b/README.md b/packages/e2b/subprocess-e2b/README.md index b1aa480305..fb3d79433f 100644 --- a/packages/e2b/subprocess-e2b/README.md +++ b/packages/e2b/subprocess-e2b/README.md @@ -8,10 +8,10 @@ E2B implementation of the [`@deepseek-ai/dsh-subprocess`](../../subprocess/subpr - **Asynchronous remote start** — the synchronous seam returns a handle immediately while `Sandbox.commands.run(..., { background: true })` starts remotely. `pid` is `-1` until the wrapper publishes and the adapter validates its process-group id; stdin and ordinary observation wait for that publication, while cancellation can stop the provisional SDK handle first. - **Execution-world coordinates** — `cwd` and private `runtimeRoot` come from the shared owner; executable lookup verifies absolute paths or resolves a bare name against the sandbox PATH plus explicit overrides. -- **Linux process groups** — a quoted wrapper starts each argv under `exec setsid --wait` and records its actual process-group id plus private status files beneath `ctx.e2b.runtimeRoot/processes`. The handle waits for that file instead of treating the SDK command PID as its published identity. Termination signals the negative recorded id with `SIGTERM`, waits the caller's `graceMs`, then escalates to `SIGKILL` and the SDK kill fallback; TERM delivery or probe failures also force that escalation. A failed transaction is observable through `waitForExit()` and may be retried. Before publication, cancellation uses the provisional SDK handle; if publication fails, rollback kills and verifies the provisional group before startup rejects. Service disposal terminates and joins every retained handle before the sandbox owner disposes. +- **Linux process groups** — a quoted wrapper starts each argv under `exec setsid --wait` and records its actual process-group id plus private status files beneath `ctx.e2b.runtimeRoot/processes`. The handle waits for that file instead of treating the SDK command PID as its published identity. Termination signals the negative recorded id with `SIGTERM`, waits the caller's `graceMs`, then escalates to `SIGKILL` and the SDK kill fallback; TERM delivery or probe failures also force that escalation. A failed transaction is observable through `waitForExit()` and may be retried, while successful termination remains a permanent no-op. Before publication, cancellation uses the provisional SDK handle; if publication fails, rollback kills and verifies the provisional group before startup rejects. Service disposal rejects new starts, terminates and joins every retained process group, then awaits SDK settlement and private cleanup before the sandbox owner disposes. - **Environment boundary** — the wrapper starts from the sandbox command environment, removes ambient `DSH_*` and credential-shaped (`*KEY*`, `*SECRET*`, `*TOKEN*`) names, then restores every `spec.env` entry as an explicit caller opt-in. Host ambient variables never enter the sandbox implicitly. Private environment files are removed after consumption, and failed command or terminal setup removes its private state before rejecting. - **Stdio projection** — the remote wrapper branches raw bytes into optional bounded spill files, frames each live chunk as newline-delimited base64 ASCII, and the host incrementally restores bytes across arbitrary SDK callback boundaries. Pipe mode writes those bytes to host Node streams; inherit mode writes them to the harness process streams; collect mode retains a bounded host tail with offset reads. The wrapper publishes the direct command status before waiting for inherited writers; after `graceMs`, the adapter disconnects an incomplete SDK stream, withholds its partial spill, and returns that status while retaining the remote group for `waitForExit()` and termination. Batch and streaming stdin use the SDK handle. -- **Terminal sessions** — `spawnTerminal()` uses E2B's byte PTY API, installs the exact argv and scrubbed environment through private mode-`0600` files, reports the foreground process group, sends real signals, and tears down every group in the remote terminal session before settlement. Setup and teardown own the private state transaction, including failure cleanup. Prompt detection, scrollback, readiness, and owner policy remain in `dsh-pty-local`. +- **Terminal sessions** — `spawnTerminal()` uses E2B's byte PTY API, installs the exact argv and scrubbed environment through private mode-`0600` files, reports the foreground process group, sends real signals, and tears down every group in the remote terminal session before settlement. A private random output boundary discards the E2B bootstrap shell's prompt and echoed runner command while preserving every requested-process byte, including its first prompt. Setup and teardown own the private state transaction, fence publication during service disposal, and include failure cleanup. Prompt detection, scrollback, readiness, and owner policy remain in `dsh-pty-local`. The base E2B image supplies the runtime and Bash/GNU utilities this adapter invokes: `node`, `bash`, `setsid`, `ps`, `awk`, `tr`, `env`, `chmod`, `tee`, `head`, `rm`, and `kill`. A custom template must retain compatible commands and E2B PTY support. diff --git a/packages/e2b/subprocess-e2b/README.zh.md b/packages/e2b/subprocess-e2b/README.zh.md index b4824c90e7..ec012b6a05 100644 --- a/packages/e2b/subprocess-e2b/README.zh.md +++ b/packages/e2b/subprocess-e2b/README.zh.md @@ -8,10 +8,10 @@ - **异步远程启动**:同步 seam 会立即返回一个句柄,同时由 `Sandbox.commands.run(..., { background: true })` 在远程启动进程。包装层发布进程组 ID 并由适配器完成验证之前,`pid` 为 `-1`;stdin 和常规观察会等待该发布,而取消操作可以先停止临时 SDK 句柄。 - **执行世界坐标**:`cwd` 和私有 `runtimeRoot` 来自共享所有者;可执行文件查找会验证绝对路径,或根据沙箱 PATH 加显式覆盖来解析裸名称。 -- **Linux 进程组**:带引号保护的包装层会在 `exec setsid --wait` 下启动每组 argv,并在 `ctx.e2b.runtimeRoot/processes` 下记录实际进程组 ID 和私有状态文件。句柄会等待该文件,而不会把 SDK 命令 PID 当作已发布的身份。终止操作以记录的负数 ID 发送 `SIGTERM`,等待调用方的 `graceMs`,再升级到 `SIGKILL` 和 SDK kill 回退;TERM 信号发送或探测失败也会强制触发该升级。失败的事务可通过 `waitForExit()` 观察,并可重试。发布前,取消操作使用临时 SDK 句柄;如果发布失败,回滚会终止并验证临时进程组,随后启动操作才会拒绝。服务 dispose(资源释放)会在沙箱所有者释放前终止并等待每个保留句柄退出。 +- **Linux 进程组**:带引号保护的包装层会在 `exec setsid --wait` 下启动每组 argv,并在 `ctx.e2b.runtimeRoot/processes` 下记录实际进程组 ID 和私有状态文件。句柄会等待该文件,而不会把 SDK 命令 PID 当作已发布的身份。终止操作以记录的负数 ID 发送 `SIGTERM`,等待调用方的 `graceMs`,再升级到 `SIGKILL` 和 SDK kill 回退;TERM 信号发送或探测失败也会强制触发该升级。失败的事务可通过 `waitForExit()` 观察,并可重试;成功终止后,重复终止将永久为空操作。发布前,取消操作使用临时 SDK 句柄;如果发布失败,回滚会终止并验证临时进程组,随后启动操作才会拒绝。服务 dispose(资源释放)会拒绝新的启动请求、终止并等待每个保留进程组退出,再等待 SDK 结算和私有清理完成,之后沙箱所有者才会释放。 - **环境边界**:包装层从沙箱命令环境开始,移除环境中的 `DSH_*` 和形似凭据的名称(`*KEY*`、`*SECRET*`、`*TOKEN*`),再把每个 `spec.env` 条目恢复为调用方显式选择。宿主环境变量绝不会隐式进入沙箱。私有环境文件在使用后会被删除;命令或终端设置失败时,会先删除其私有状态再拒绝。 - **stdio 投影**:远程包装层先把原始字节分流到可选的有界 spill 文件,再把每个实时分片编码为换行分隔的 base64 ASCII 帧;宿主会跨任意 SDK 回调边界增量恢复字节。pipe 模式把这些字节写入宿主 Node 流;inherit 模式把字节写入 harness 进程流;collect 模式保留有界的宿主尾部,并支持基于偏移量读取。包装层会在等待继承管道的写入方之前发布直接命令状态;超过 `graceMs` 后,适配器会断开未完成的 SDK 流,不公开其中不完整的 spill,并返回该状态,同时保留远程进程组供 `waitForExit()` 和终止操作使用。批量 stdin 和流式 stdin 都使用 SDK 句柄。 -- **终端会话**:`spawnTerminal()` 使用 E2B 的字节 PTY API,以 mode 为 `0600` 的私有文件传入原样 argv 与清理后的环境,报告前台进程组,发送真实信号,并在结算前清理远程终端会话中的每个进程组。setup 与 teardown 负责私有状态事务,包括失败清理。提示符检测、scrollback、就绪状态与所有者策略仍归 `dsh-pty-local` 所有。 +- **终端会话**:`spawnTerminal()` 使用 E2B 的字节 PTY API,以 mode 为 `0600` 的私有文件传入原样 argv 与清理后的环境,报告前台进程组,发送真实信号,并在结算前清理远程终端会话中的每个进程组。私有随机输出边界会丢弃 E2B 引导 shell 的提示符和回显的 runner 命令,同时保留请求进程的每个字节,包括其第一个提示符。setup 与 teardown 负责私有状态事务,在服务 dispose 期间阻止发布,并包括失败清理。提示符检测、scrollback、就绪状态与所有者策略仍归 `dsh-pty-local` 所有。 基础 E2B 镜像提供该适配器调用的运行时和 Bash/GNU 工具:`node`、`bash`、`setsid`、`ps`、`awk`、`tr`、`env`、`chmod`、`tee`、`head`、`rm` 和 `kill`。自定义模板必须保留兼容的命令和 E2B PTY 支持。 diff --git a/packages/e2b/subprocess-e2b/src/index.ts b/packages/e2b/subprocess-e2b/src/index.ts index 27efb216da..fcf2c6f0a4 100644 --- a/packages/e2b/subprocess-e2b/src/index.ts +++ b/packages/e2b/subprocess-e2b/src/index.ts @@ -28,6 +28,8 @@ export class E2BSubprocessService extends SubprocessService { private readonly live = new Set() private readonly terminals = new Set() + private readonly terminalSetups = new Set>() + private disposing = false /** @inheritdoc */ readonly cwd: string @@ -41,12 +43,17 @@ export class E2BSubprocessService extends SubprocessService { this.cwd = ctx.e2b.cwd this.runtimeRoot = ctx.e2b.runtimeRoot ctx.effect(() => async () => { + this.disposing = true + await Promise.all([...this.terminalSetups]) const handles = [...this.live] const terminals = [...this.terminals] const pending: Promise[] = [] for (const handle of handles) { handle.terminate() - pending.push(handle.waitForExit().then(() => { this.live.delete(handle) })) + pending.push(handle.waitForExit().then(async () => { + await handle.done.catch(() => undefined) + this.live.delete(handle) + })) } for (const terminal of terminals) { terminal.terminate() @@ -89,6 +96,7 @@ export class E2BSubprocessService extends SubprocessService { /** @inheritdoc */ spawn(spec: SubprocessSpawnSpec): SubprocessHandle { + if (this.isDisposing()) throw new Error('subprocess-e2b: service is disposing') const program = spec.argv[0] if (program === undefined || program.length === 0) { throw new Error('invalid argv: expected a non-empty program name at argv[0]') @@ -112,6 +120,7 @@ export class E2BSubprocessService extends SubprocessService { /** @inheritdoc */ async spawnTerminal(spec: SubprocessTerminalSpawnSpec): Promise { + if (this.isDisposing()) throw new Error('subprocess-e2b: service is disposing') const program = spec.argv[0] if (program === undefined || program.length === 0) { throw new Error('subprocess-e2b: terminal argv must contain a program') @@ -123,14 +132,31 @@ export class E2BSubprocessService extends SubprocessService { } spec.signal?.throwIfAborted() const stateDir = posix.join(this.runtimeRoot, 'terminals', randomUUID()) - const terminal = await spawnE2BTerminal(this.ctx.e2b, spec, stateDir) - this.terminals.add(terminal) - const release = async (): Promise => { - await terminal.waitForExit() - this.terminals.delete(terminal) + const setup = Promise.withResolvers() + this.terminalSetups.add(setup.promise) + try { + const terminal = await spawnE2BTerminal(this.ctx.e2b, spec, stateDir) + this.terminals.add(terminal) + if (this.isDisposing()) { + terminal.terminate() + await terminal.waitForExit() + this.terminals.delete(terminal) + throw new Error('subprocess-e2b: service disposed during terminal setup') + } + const release = async (): Promise => { + await terminal.waitForExit() + this.terminals.delete(terminal) + } + void terminal.done.then(release, release).catch(() => {}) + return terminal + } finally { + this.terminalSetups.delete(setup.promise) + setup.resolve() } - void terminal.done.then(release, release).catch(() => {}) - return terminal + } + + private isDisposing(): boolean { + return this.disposing } } diff --git a/packages/e2b/subprocess-e2b/src/process.ts b/packages/e2b/subprocess-e2b/src/process.ts index 473c1a0af4..3038f312dc 100644 --- a/packages/e2b/subprocess-e2b/src/process.ts +++ b/packages/e2b/subprocess-e2b/src/process.ts @@ -213,6 +213,7 @@ export class E2BSubprocessHandle implements SubprocessHandle { private invalidHandleQuiescent = false private provisionalHandleQuiescent = false private terminationStarted = false + private terminationSucceeded = false private terminationAttempt: Promise | undefined private terminationFailure: Error | undefined private terminationSignal: NodeJS.Signals | null = null @@ -264,13 +265,14 @@ export class E2BSubprocessHandle implements SubprocessHandle { /** @inheritdoc */ terminate(): void { - if (this.terminationAttempt !== undefined) return + if (this.terminationSucceeded || this.terminationAttempt !== undefined) return this.terminationStarted = true this.terminationFailure = undefined const attempt = this.terminateRemote() this.terminationAttempt = attempt void attempt.then( () => { + this.terminationSucceeded = true this.terminationAttempt = undefined }, (error: unknown) => { diff --git a/packages/e2b/subprocess-e2b/src/terminal.ts b/packages/e2b/subprocess-e2b/src/terminal.ts index fb8dac6f49..78668490d7 100644 --- a/packages/e2b/subprocess-e2b/src/terminal.ts +++ b/packages/e2b/subprocess-e2b/src/terminal.ts @@ -1,6 +1,7 @@ /** E2B PTY allocation and process-session ownership for the subprocess seam. */ import { Buffer } from 'node:buffer' +import { randomUUID } from 'node:crypto' import { PassThrough } from 'node:stream' import { posix } from 'node:path' import { @@ -28,11 +29,13 @@ const TERMINAL_RUNNER_SOURCE = [ 'dsh_state=$1', 'mapfile -d \'\' -t dsh_env < "$dsh_state/environment"', 'mapfile -d \'\' -t dsh_argv < "$dsh_state/argv"', - 'rm -f -- "$dsh_state/environment" "$dsh_state/argv" "$dsh_state/runner.bash"', + 'dsh_output_marker=$(<"$dsh_state/output-marker")', + 'rm -f -- "$dsh_state/environment" "$dsh_state/argv" "$dsh_state/output-marker" "$dsh_state/runner.bash"', 'if (( ${#dsh_argv[@]} == 0 )); then', " printf 'terminal runner received empty argv\\n' >&2", ' exit 125', 'fi', + 'printf \'%s\' "$dsh_output_marker"', "printf 'ready\\n' > \"$dsh_state/ready\"", 'exec env -i "${dsh_env[@]}" "${dsh_argv[@]}"', '', @@ -42,6 +45,7 @@ interface TerminalPaths { runner: string environment: string argv: string + outputMarker: string ready: string } @@ -53,6 +57,73 @@ function delay(ms: number): Promise { return new Promise(resolve => setTimeout(resolve, ms)) } +class BootstrapOutputFilter { + readonly ready: Promise + + private readonly readyState = Promise.withResolvers() + private pending = Buffer.alloc(0) + private published = false + + constructor( + private readonly marker: Buffer, + private readonly output: PassThrough, + ) { + this.ready = this.readyState.promise + } + + push(data: Uint8Array): void { + if (this.published) { + this.write(data) + return + } + const combined = Buffer.concat([this.pending, Buffer.from(data)]) + const markerOffset = combined.indexOf(this.marker) + if (markerOffset < 0) { + const retained = Math.min(combined.length, this.marker.length - 1) + this.pending = Buffer.from(combined.subarray(combined.length - retained)) + return + } + this.published = true + this.pending = Buffer.alloc(0) + this.readyState.resolve() + this.write(combined.subarray(markerOffset + this.marker.length)) + } + + private write(data: Uint8Array): void { + if (data.length > 0 && !this.output.destroyed) this.output.write(data) + } +} + +async function waitForBootstrapOutput( + ready: Promise, + completion: Promise, + signal?: AbortSignal, +): Promise { + signal?.throwIfAborted() + await new Promise((resolve, reject) => { + let settled = false + let removeAbort: (() => void) | undefined + const finish = (complete: () => void): void => { + if (settled) return + settled = true + removeAbort?.() + complete() + } + const onExit = (): void => { + finish(() => { reject(new Error('subprocess-e2b: terminal exited before publishing its output boundary')) }) + } + if (signal !== undefined) { + const onAbort = (): void => { + finish(() => { reject(asError(signal.reason)) }) + } + signal.addEventListener('abort', onAbort, { once: true }) + removeAbort = () => { signal.removeEventListener('abort', onAbort) } + } + void ready.then(() => { finish(resolve) }) + void completion.then(onExit, onExit) + }) +} + function asError(error: unknown): Error { return error instanceof Error ? error : new Error(String(error)) } @@ -374,9 +445,12 @@ export async function spawnE2BTerminal( runner: posix.join(stateDir, 'runner.bash'), environment: posix.join(stateDir, 'environment'), argv: posix.join(stateDir, 'argv'), + outputMarker: posix.join(stateDir, 'output-marker'), ready: posix.join(stateDir, 'ready'), } + const outputMarker = Buffer.from(`dsh-e2b-bootstrap:${randomUUID()}`) const output = new PassThrough() + const outputFilter = new BootstrapOutputFilter(outputMarker, output) let handle: CommandHandle | undefined let completion: Promise | undefined let stateDirectoryCreated = false @@ -391,9 +465,10 @@ export async function spawnE2BTerminal( { path: paths.runner, data: TERMINAL_RUNNER_SOURCE }, { path: paths.environment, data: environment }, { path: paths.argv, data: argv }, + { path: paths.outputMarker, data: outputMarker.toString('utf8') }, ], signalOpts(spec.signal)) await sandbox.commands.run( - `chmod 600 -- ${quoteE2BShellArg(paths.runner)} ${quoteE2BShellArg(paths.environment)} ${quoteE2BShellArg(paths.argv)}`, + `chmod 600 -- ${quoteE2BShellArg(paths.runner)} ${quoteE2BShellArg(paths.environment)} ${quoteE2BShellArg(paths.argv)} ${quoteE2BShellArg(paths.outputMarker)}`, signalOpts(spec.signal), ) handle = await sandbox.pty.create({ @@ -403,7 +478,7 @@ export async function spawnE2BTerminal( envs: { TERM: 'dumb' }, timeoutMs: 0, ...signalOpts(spec.signal), - onData: (data) => { if (!output.destroyed) output.write(Buffer.from(data)) }, + onData: (data) => { outputFilter.push(data) }, }) completion = handle.wait() void completion.catch(() => {}) @@ -413,6 +488,7 @@ export async function spawnE2BTerminal( const command = `exec /bin/bash ${quoteE2BShellArg(paths.runner)} ${quoteE2BShellArg(stateDir)}\r` await sandbox.pty.sendInput(handle.pid, Buffer.from(command), signalOpts(spec.signal)) await waitUntilReady(sandbox, paths, completion, spec.signal) + await waitForBootstrapOutput(outputFilter.ready, completion, spec.signal) const sessionId = await terminalSessionId(sandbox, handle.pid, spec.signal) return new E2BTerminalHandle( sandbox, diff --git a/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts b/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts index 457e68f24d..be8850d905 100644 --- a/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts +++ b/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts @@ -599,6 +599,12 @@ describe('E2BSubprocessHandle', () => { await expect(handle.waitForExit()).resolves.toBe(true) expect(fake.commandsSeen).toContain('kill -TERM -- -4242') expect(fake.commandsSeen).not.toContain('kill -KILL -- -4242') + const signals = fake.commandsSeen.filter(command => command.startsWith('kill -')).length + fake.alive = true + handle.terminate() + await flush() + expect(fake.alive).toBe(true) + expect(fake.commandsSeen.filter(command => command.startsWith('kill -'))).toHaveLength(signals) }) it('escalates a TERM-trapping process group to KILL and uses the SDK kill as fallback', async () => { @@ -1123,6 +1129,24 @@ describe('E2BSubprocessService', () => { expect(fake.alive).toBe(false) }) + it('awaits SDK settlement after the remote process group becomes quiescent', async () => { + const fake = new FakeSandbox() + fake.trapsTerm = true + const { ctx, fiber } = await service(fake) + const handle = ctx.subprocess.spawn(spec()) + await flush() + fake.alive = false + + let disposed = false + const disposing = fiber.dispose().then(() => { disposed = true }) + await flush() + expect(disposed).toBe(false) + + fake.finish() + await disposing + await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) + }) + it('reports a failed termination transaction from disposal instead of waiting on done', async () => { const fake = new FakeSandbox() fake.signalErrors.push(new Error('TERM transport failed'), new Error('KILL transport failed')) @@ -1178,6 +1202,7 @@ describe('E2BSubprocessService', () => { const { ctx, fiber } = await service(fake) const handle = ctx.subprocess.spawn(spec()) const disposing = fiber.dispose() + expect(() => ctx.subprocess.spawn(spec())).toThrow('service is disposing') fake.releaseStart() await expect(disposing).resolves.toBeUndefined() await expect(handle.done).rejects.toThrow('start failed during disposal') diff --git a/packages/e2b/subprocess-e2b/tests/terminal.spec.ts b/packages/e2b/subprocess-e2b/tests/terminal.spec.ts index bda01a5a61..1eac9a5780 100644 --- a/packages/e2b/subprocess-e2b/tests/terminal.spec.ts +++ b/packages/e2b/subprocess-e2b/tests/terminal.spec.ts @@ -2,7 +2,7 @@ import { Buffer } from 'node:buffer' import { once } from 'node:events' import { PassThrough } from 'node:stream' import { Context } from 'cordis' -import { describe, expect, it } from 'vitest' +import { describe, expect, it, vi } from 'vitest' import { CommandExitError, FileNotFoundError, @@ -91,6 +91,7 @@ class FakeTerminalSandbox { ambient = 'KEEP=visible\0NPM_TOKEN=secret\0DSH_STALE=old\0BROKEN\0=bad\0' ready: string | Error = 'ready\n' readyMisses = 0 + readyReads = 0 sessionId = '123\n' foreground = '456\n' groups = [123] @@ -108,6 +109,20 @@ class FakeTerminalSandbox { settleOnPtyKill = true ptyKills = 0 resolvedExecutable = '/usr/bin/node\n' + requestedOutput = 'requested-shell$ ' + emitOutputMarker = true + private createGate: Promise | undefined + private releaseCreateGate: (() => void) | undefined + + deferCreate(): void { + const gate = Promise.withResolvers() + this.createGate = gate.promise + this.releaseCreateGate = () => { gate.resolve(undefined) } + } + + releaseCreate(): void { + this.releaseCreateGate?.() + } readonly sandbox = { files: { @@ -121,6 +136,7 @@ class FakeTerminalSandbox { return files.map(() => ({})) }, read: async (): Promise => { + this.readyReads += 1 if (this.readyMisses > 0) { this.readyMisses -= 1 throw new FileNotFoundError('not ready') @@ -170,6 +186,7 @@ class FakeTerminalSandbox { pty: { create: async (options: Parameters[0]): Promise => { this.createOptions = options + await this.createGate if (this.createError !== undefined) throw this.createError await options.onData(Buffer.from('buffered banner\n')) return this.handle.asHandle() @@ -178,6 +195,17 @@ class FakeTerminalSandbox { options?.signal?.throwIfAborted() this.inputs.push({ pid, data: Buffer.from(data) }) if (this.sendError !== undefined) throw this.sendError + if (this.emitOutputMarker && Buffer.from(data).includes(Buffer.from('runner.bash'))) { + const marker = [...this.writes].find(([path]) => path.endsWith('/output-marker'))?.[1] + const onData = this.createOptions?.onData + if (marker !== undefined && onData !== undefined) { + await onData(Buffer.from(Buffer.from(data).toString().replace(/\r$/, '\r\n'))) + const split = Math.floor(marker.length / 2) + await onData(Buffer.from(marker.slice(0, split))) + await onData(Buffer.from(marker.slice(split))) + await onData(Buffer.from(this.requestedOutput)) + } + } }, kill: async (pid: number): Promise => { this.ptyKills += 1 @@ -211,7 +239,7 @@ function spec(overrides: Partial = {}): SubprocessT } describe('E2B terminal allocation', () => { - it('boots the requested argv through a private runner and preserves buffered bytes', async () => { + it('hides bootstrap-shell bytes and preserves requested-shell bytes across the output boundary', async () => { const fake = new FakeTerminalSandbox() fake.readyMisses = 1 const terminal = await spawnE2BTerminal(runtime(fake), spec(), '/runtime/terminal-one') @@ -219,7 +247,9 @@ describe('E2B terminal allocation', () => { terminal.output.on('data', (chunk) => { output += String(chunk) }) await new Promise(resolve => setTimeout(resolve, 0)) - expect(output).toBe('buffered banner\n') + expect(output).toBe('requested-shell$ ') + expect(output).not.toContain('buffered banner') + expect(output).not.toContain('runner.bash') expect(fake.createOptions).toMatchObject({ rows: 24, cols: 80, cwd: '/workspace', timeoutMs: 0, envs: { TERM: 'dumb' } }) expect(fake.inputs[0]?.data.toString()).toContain("exec /bin/bash '/runtime/terminal-one/runner.bash'") expect(fake.writes.get('/runtime/terminal-one/environment')).toContain('KEEP=visible\0') @@ -227,10 +257,17 @@ describe('E2B terminal allocation', () => { expect(fake.writes.get('/runtime/terminal-one/environment')).not.toContain('secret') expect(fake.writes.get('/runtime/terminal-one/environment')).not.toContain('DSH_STALE') expect(fake.writes.get('/runtime/terminal-one/argv')).toBe('/bin/bash\0--noprofile\0--norc\0') + const marker = fake.writes.get('/runtime/terminal-one/output-marker') ?? '' + expect(marker).toMatch(/^dsh-e2b-bootstrap:/) + expect(fake.inputs[0]?.data.toString()).not.toContain(marker) const runner = fake.writes.get('/runtime/terminal-one/runner.bash') ?? '' expect(runner).toContain('if (( ${#dsh_argv[@]} == 0 )); then') + expect(runner).toContain('printf \'%s\' "$dsh_output_marker"') expect(runner).toContain('exec env -i "${dsh_env[@]}" "${dsh_argv[@]}"') expect(runner).not.toContain('\u007f') + terminal.output.destroy() + await fake.createOptions?.onData(Buffer.from('late bootstrap callback')) + expect(output).toBe('requested-shell$ ') await terminal.write(Buffer.from('echo ok\r')) expect(fake.inputs.at(-1)?.data.toString()).toBe('echo ok\r') @@ -395,6 +432,29 @@ describe('E2B terminal allocation', () => { await expect(spawnE2BTerminal(runtime(readFailed), spec(), '/runtime/read')) .rejects.toThrow('ready transport failed') }) + + it('bounds a missing bootstrap-output boundary by process exit or cancellation', async () => { + const exited = new FakeTerminalSandbox() + exited.emitOutputMarker = false + const exiting = spawnE2BTerminal(runtime(exited), spec(), '/runtime/missing-output-boundary') + await vi.waitFor(() => { expect(exited.inputs).toHaveLength(1) }) + exited.handle.succeed(0) + await expect(exiting).rejects.toThrow('terminal exited before publishing its output boundary') + + const cancelled = new FakeTerminalSandbox() + cancelled.emitOutputMarker = false + const controller = new AbortController() + const cancelling = spawnE2BTerminal( + runtime(cancelled), + spec({ signal: controller.signal }), + '/runtime/cancel-output-boundary', + ) + await vi.waitFor(() => { expect(cancelled.inputs).toHaveLength(1) }) + await vi.waitFor(() => { expect(cancelled.readyReads).toBeGreaterThan(0) }) + await new Promise(resolve => setTimeout(resolve, 0)) + controller.abort(new Error('cancel output boundary')) + await expect(cancelling).rejects.toThrow('cancel output boundary') + }) }) describe('E2B terminal lifecycle', () => { @@ -637,6 +697,29 @@ describe('E2B subprocess terminal service', () => { expect(fake.handle.disconnects).toBe(1) }) + it('joins and rejects terminal setup that completes during service disposal', async () => { + const fake = new FakeTerminalSandbox() + fake.deferCreate() + const { ctx, fiber } = await service(fake) + const spawning = ctx.subprocess.spawnTerminal(spec()) + const rejected = expect(spawning).rejects.toThrow('service disposed during terminal setup') + await vi.waitFor(() => { expect(fake.createOptions).toBeDefined() }) + + let disposed = false + const subprocess = ctx.subprocess + const disposing = fiber.dispose().then(() => { disposed = true }) + await new Promise(resolve => setTimeout(resolve, 0)) + expect(disposed).toBe(false) + await expect(subprocess.spawnTerminal(spec())).rejects.toThrow('service is disposing') + fake.releaseCreate() + + await rejected + await disposing + expect(fake.groups).toEqual([]) + expect(fake.handle.disconnects).toBe(1) + expect(fake.removed.some(path => path.includes('/terminals/'))).toBe(true) + }) + it('releases naturally settled terminals and validates terminal requests', async () => { const { ctx, fiber, fake } = await service() for (const request of [ From ed5d21277faa85ad97cf4fbe17ee62e3ca1bfdb8 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 08:05:37 +0800 Subject: [PATCH 080/108] test(e2b): observe disposal before spawn fence --- packages/e2b/subprocess-e2b/tests/subprocess.spec.ts | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts b/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts index be8850d905..8e6e420bcc 100644 --- a/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts +++ b/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts @@ -1200,9 +1200,11 @@ describe('E2BSubprocessService', () => { fake.deferStart() fake.backgroundError = new Error('start failed during disposal') const { ctx, fiber } = await service(fake) - const handle = ctx.subprocess.spawn(spec()) + const subprocess = ctx.subprocess + const handle = subprocess.spawn(spec()) const disposing = fiber.dispose() - expect(() => ctx.subprocess.spawn(spec())).toThrow('service is disposing') + await flush() + expect(() => subprocess.spawn(spec())).toThrow('service is disposing') fake.releaseStart() await expect(disposing).resolves.toBeUndefined() await expect(handle.done).rejects.toThrow('start failed during disposal') From f5306f6e898c797900c66d1441ce0f7729007732 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 08:06:02 +0800 Subject: [PATCH 081/108] docs(e2b): record subprocess teardown contract --- packages/e2b/subprocess-e2b/README.i18n.yaml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/packages/e2b/subprocess-e2b/README.i18n.yaml b/packages/e2b/subprocess-e2b/README.i18n.yaml index 2d20e4ad4c..3a577227f8 100644 --- a/packages/e2b/subprocess-e2b/README.i18n.yaml +++ b/packages/e2b/subprocess-e2b/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/e2b/subprocess-e2b/README.md -README.md: 03146e02cc76b78fbb93faec7d36d1865418b670 -README.zh.md: 6ed49a355586c6df2a594f83b687e6a565d07cf0 +README.md: fb3d79433fba8bb86097cffb09a257ad5756cc2b +README.zh.md: ec012b6a05c1a5f911d5ede9730f0f7207d5a645 From 219999cd2f3e0abf3878d354de3bf42cfc2e6134 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 09:19:46 +0800 Subject: [PATCH 082/108] fix(e2b): close adapter lifecycle review gaps --- packages/e2b/fs-e2b/README.i18n.yaml | 4 +- packages/e2b/fs-e2b/README.md | 2 +- packages/e2b/fs-e2b/README.zh.md | 2 +- packages/e2b/fs-e2b/src/index.ts | 24 ++- packages/e2b/fs-e2b/tests/filesystem.spec.ts | 33 +++- packages/e2b/subprocess-e2b/README.i18n.yaml | 4 +- packages/e2b/subprocess-e2b/README.md | 10 +- packages/e2b/subprocess-e2b/README.zh.md | 10 +- .../e2b/subprocess-e2b/src/environment.ts | 20 +++ packages/e2b/subprocess-e2b/src/index.ts | 25 ++- packages/e2b/subprocess-e2b/src/process.ts | 67 +++++--- packages/e2b/subprocess-e2b/src/terminal.ts | 61 ++++---- .../subprocess-e2b/tests/subprocess.spec.ts | 146 +++++++++++++++++- .../e2b/subprocess-e2b/tests/terminal.spec.ts | 70 ++++++--- 14 files changed, 375 insertions(+), 103 deletions(-) diff --git a/packages/e2b/fs-e2b/README.i18n.yaml b/packages/e2b/fs-e2b/README.i18n.yaml index 6e45ccaaf6..3c2b30ad15 100644 --- a/packages/e2b/fs-e2b/README.i18n.yaml +++ b/packages/e2b/fs-e2b/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/e2b/fs-e2b/README.md -README.md: 86ad8720d3e4c7ce70ee0ac8c41713af03ed297a -README.zh.md: 90dff57a5b9fe786aea3315633125536277ecc20 +README.md: 8abf130ce16e2b79cda5fc858182159442f9e3c3 +README.zh.md: 943d9603046ad9b2a65f41d5801cac882040a0ff diff --git a/packages/e2b/fs-e2b/README.md b/packages/e2b/fs-e2b/README.md index 86ad8720d3..8abf130ce1 100644 --- a/packages/e2b/fs-e2b/README.md +++ b/packages/e2b/fs-e2b/README.md @@ -10,7 +10,7 @@ E2B implementation of the [`@deepseek-ai/dsh-fs`](../../fs/fs/README.md) provide - **Execution-world paths** — canonical targets expose absolute POSIX process paths, percent-encoded `file:` URIs, and provider-owned containment checks, so generic subprocess consumers never parse E2B target ids or apply host path rules. - **UTF-8 reads** — whole reads and streamed reads preserve cross-chunk decoding, reject invalid UTF-8, and use the seam's 8192-byte NUL sample for binary detection. The model-facing tool still owns size selection and line windowing. - **Stable bounded reads** — a dependency-free Node helper walks directory descriptors with no-follow opens and reads one held regular-file descriptor through the byte cap. Generic LSP queries therefore reject parent swaps, non-files, invalid UTF-8, and growth past the configured document limit before server startup. -- **Atomic mutations** — writes upload a mode-`0600` temporary sibling, preserve an existing file's POSIX mode, and publish through E2B's same-directory atomic rename. The rename response supplies the committed version, so no fallible metadata request follows the commit point. E2B creates missing parent directories. Literal edits LF-normalize for matching, restore dominant CRLF storage, and serialize mutations per canonical target within the host process. Optional create/version guards keep the base seam's observed-state semantics. +- **Atomic mutations** — writes create a random sibling staging directory, change it to mode `0700` before uploading content, preserve an existing file's POSIX mode, and publish the staged file through E2B's same-filesystem atomic rename. The rename response supplies the committed version, so no fallible metadata request follows the commit point. E2B creates missing parent directories. Literal edits LF-normalize for matching, restore dominant CRLF storage, and serialize mutations per canonical target within the host process. Optional create/version guards keep the base seam's observed-state semantics. - **Failures and cancellation** — E2B not-found, permission, abort, and other controller failures map to the existing `FsError` vocabulary. Cancellation is best-effort at SDK request boundaries; a successful rename is the commit point. The provider does not copy, mount, or reconcile the host workspace. Giving it a host path as `cwd` creates a remote directory with the same spelling only. diff --git a/packages/e2b/fs-e2b/README.zh.md b/packages/e2b/fs-e2b/README.zh.md index 90dff57a5b..943d960304 100644 --- a/packages/e2b/fs-e2b/README.zh.md +++ b/packages/e2b/fs-e2b/README.zh.md @@ -10,7 +10,7 @@ - **执行世界路径**:规范化目标公开绝对 POSIX 进程路径、百分号编码的 `file:` URI,以及由提供方负责的包含关系检查,因此通用进程管理消费方无需解析 E2B 目标 ID,也不会套用宿主路径规则。 - **UTF-8 读取**:完整读取和流式读取会保留跨分片解码、拒绝无效 UTF-8,并使用 seam 的 8192 字节 NUL 样本检测二进制内容。面向模型的工具仍负责选择大小和行窗口。 - **稳定的有界读取**:一个零依赖 Node 辅助程序会以不跟随链接的方式逐级打开目录描述符,并通过一个持续持有的常规文件描述符读取至字节上限。因此,通用 LSP 查询会在服务器启动前拒绝父目录交换、非文件、无效 UTF-8,以及增长后超出所配置文档上限的文件。 -- **原子变更**:写入会上传 mode 为 `0600` 的同级临时文件,保留现有文件的 POSIX mode,并通过 E2B 的同目录原子重命名发布。重命名响应会提供已提交的版本,因此提交点之后不会再进行可能失败的元数据请求。E2B 会创建缺失的父目录。字面量编辑匹配时会规范化为 LF,存储时恢复占主导的 CRLF,并在宿主进程内按规范化目标串行执行变更。可选的创建/版本防护会保留基础 seam 的已观察状态语义。 +- **原子变更**:写入会创建随机的同级暂存目录,在上传内容前将其 mode 改为 `0700`,保留现有文件的 POSIX mode,并通过 E2B 的同一文件系统原子重命名发布暂存文件。重命名响应会提供已提交的版本,因此提交点之后不会再进行可能失败的元数据请求。E2B 会创建缺失的父目录。字面量编辑匹配时会规范化为 LF,存储时恢复占主导的 CRLF,并在宿主进程内按规范化目标串行执行变更。可选的创建/版本防护会保留基础 seam 的已观察状态语义。 - **失败与取消**:E2B 的未找到、权限、中止及其他控制器故障会映射到现有 `FsError` 词汇。取消在 SDK 请求边界上采用尽力而为语义;成功 rename 是提交点。 该提供方不会复制、挂载或协调宿主工作区。把宿主路径用作 `cwd`,只会在远程创建一个拼写相同的目录。 diff --git a/packages/e2b/fs-e2b/src/index.ts b/packages/e2b/fs-e2b/src/index.ts index 2452e856ca..0824c3f87c 100644 --- a/packages/e2b/fs-e2b/src/index.ts +++ b/packages/e2b/fs-e2b/src/index.ts @@ -498,8 +498,15 @@ export class E2BFileSystem extends FileSystem { const sandbox = await this.ctx.e2b.getSandbox() const targetPath = String(target.targetKey) const versionId = randomUUID() - const temporary = posix.join(posix.dirname(targetPath), `.${posix.basename(targetPath)}.dsh-${randomUUID()}.tmp`) + const stagingDirectory = posix.join(posix.dirname(targetPath), `.dsh-${randomUUID()}.tmp`) + const temporary = posix.join(stagingDirectory, 'content') + let stagingDirectoryCreated = false try { + const created = await sandbox.files.makeDir(stagingDirectory, signalOpts(signal)) + if (!created) throw new Error('private staging directory already exists') + stagingDirectoryCreated = true + await sandbox.commands.run(`chmod 700 -- ${quoteE2BShellArg(stagingDirectory)}`, signalOpts(signal)) + assertNotAborted(signal, 'write') await sandbox.files.write(temporary, content, { metadata: { [VERSION_METADATA_KEY]: versionId }, ...signalOpts(signal), @@ -512,12 +519,19 @@ export class E2BFileSystem extends FileSystem { ) assertNotAborted(signal, 'write') const committed = await sandbox.files.rename(temporary, targetPath, signalOpts(signal)) + try { + await sandbox.files.remove(stagingDirectory) + } catch (_committedStagingCleanupFailure) { + // The target is already committed; an empty private directory cannot turn that write into a failure. + } return entryVersion(committed) } catch (error: unknown) { - try { - await sandbox.files.remove(temporary) - } catch (_temporaryAlreadyAbsent) { - // Only the private staging path is swallowed; the original failure owns the operation. + if (stagingDirectoryCreated) { + try { + await sandbox.files.remove(stagingDirectory) + } catch (_stagingDirectoryAlreadyAbsentOrCleanupFailed) { + // Only the private staging directory is swallowed; the original failure owns the operation. + } } throw mapError(error, 'write', target.displayPath, signal) } diff --git a/packages/e2b/fs-e2b/tests/filesystem.spec.ts b/packages/e2b/fs-e2b/tests/filesystem.spec.ts index 648631c7ee..303f99c1e8 100644 --- a/packages/e2b/fs-e2b/tests/filesystem.spec.ts +++ b/packages/e2b/fs-e2b/tests/filesystem.spec.ts @@ -35,6 +35,7 @@ function commandError(exitCode: number, stderr = ''): CommandExitError { class FakeRemote { readonly nodes = new Map() readonly writes: Array<{ path: string; data: string; metadata?: Record }> = [] + readonly writeParentModes: number[] = [] readonly renames: Array<{ from: string; to: string }> = [] readonly removals: string[] = [] readonly commands: string[] = [] @@ -42,6 +43,7 @@ class FakeRemote { streamKeepOpen = false readonly streamCancel = vi.fn() nextCommandError: unknown + nextMakeDirResult: boolean | undefined nextInfoError: unknown nextListError: unknown nextReadError: unknown @@ -130,7 +132,13 @@ class FakeRemote { readonly sandbox = { sandboxId: 'fake', files: { - makeDir: async (path: string): Promise => { + makeDir: async (path: string, options?: { signal?: AbortSignal }): Promise => { + this.checkAbort(options) + if (this.nextMakeDirResult !== undefined) { + const result = this.nextMakeDirResult + this.nextMakeDirResult = undefined + return result + } if (this.nodes.has(path)) return false this.dir(path) return true @@ -178,6 +186,7 @@ class FakeRemote { this.checkAbort(options) const parent = dirname(path) if (!this.nodes.has(parent)) this.dir(parent) + this.writeParentModes.push(this.required(parent).mode) this.nodes.set(path, { type: FileType.FILE, data: bytes(data), @@ -209,7 +218,9 @@ class FakeRemote { this.nextRemoveError = undefined throw error } - this.nodes.delete(path) + for (const candidate of this.nodes.keys()) { + if (candidate === path || candidate.startsWith(`${path}/`)) this.nodes.delete(candidate) + } }, }, commands: { @@ -499,6 +510,10 @@ describe('E2BFileSystem atomic writes and edits', () => { expect(outcome).toMatchObject({ operation: 'create', before: null, after: 'one\ntwo\rthree' }) expect(remote.nodes.get('/workspace/new.txt')?.mode).toBe(0o600) expect(remote.nodes.get('/workspace/new.txt')?.metadata?.['dsh-version']).toBeDefined() + expect(remote.writeParentModes).toEqual([0o700]) + const stagingDirectory = posix.dirname(remote.writes[0]!.path) + expect(posix.dirname(stagingDirectory)).toBe('/workspace') + expect(remote.removals).toContain(stagingDirectory) await expect(fs.stat(target)).resolves.toMatchObject({ version: outcome.version, size: 14 }) }) @@ -558,6 +573,15 @@ describe('E2BFileSystem atomic writes and edits', () => { expect(controller.signal.aborted).toBe(true) }) + it('does not turn post-commit staging cleanup failure into a failed write', async () => { + const remote = new FakeRemote() + remote.nextRemoveError = new Error('empty staging cleanup failed') + const { fs } = await setup(remote) + await expect(fs.writeText(await fs.resolve('committed'), 'yes')) + .resolves.toMatchObject({ operation: 'create' }) + expect(new TextDecoder().decode(remote.nodes.get('/workspace/committed')?.data)).toBe('yes') + }) + it('returns committed rename metadata without a fallible post-commit lookup', async () => { const remote = new FakeRemote() const getInfo = vi.spyOn(remote.sandbox.files, 'getInfo') @@ -582,6 +606,11 @@ describe('E2BFileSystem atomic writes and edits', () => { remote.nextRemoveError = new Error('cleanup also failed') remote.nextRenameError = new DOMException('aborted', 'AbortError') await expectCode(fs.writeText(await fs.resolve('abort'), 'x'), 'FS_ABORTED') + + const removalsBeforeCollision = remote.removals.length + remote.nextMakeDirResult = false + await expectCode(fs.writeText(await fs.resolve('collision'), 'x'), 'FS_IO_ERROR') + expect(remote.removals).toHaveLength(removalsBeforeCollision) }) it('applies literal edits atomically and restores the detected CRLF style', async () => { diff --git a/packages/e2b/subprocess-e2b/README.i18n.yaml b/packages/e2b/subprocess-e2b/README.i18n.yaml index 3a577227f8..41d964a58a 100644 --- a/packages/e2b/subprocess-e2b/README.i18n.yaml +++ b/packages/e2b/subprocess-e2b/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/e2b/subprocess-e2b/README.md -README.md: fb3d79433fba8bb86097cffb09a257ad5756cc2b -README.zh.md: ec012b6a05c1a5f911d5ede9730f0f7207d5a645 +README.md: 42cd4951edb101dc75980e114e704c4aab50ae25 +README.zh.md: c4b318184cb7616a378e3336f4f4a7c366a98807 diff --git a/packages/e2b/subprocess-e2b/README.md b/packages/e2b/subprocess-e2b/README.md index fb3d79433f..42cd4951ed 100644 --- a/packages/e2b/subprocess-e2b/README.md +++ b/packages/e2b/subprocess-e2b/README.md @@ -8,10 +8,10 @@ E2B implementation of the [`@deepseek-ai/dsh-subprocess`](../../subprocess/subpr - **Asynchronous remote start** — the synchronous seam returns a handle immediately while `Sandbox.commands.run(..., { background: true })` starts remotely. `pid` is `-1` until the wrapper publishes and the adapter validates its process-group id; stdin and ordinary observation wait for that publication, while cancellation can stop the provisional SDK handle first. - **Execution-world coordinates** — `cwd` and private `runtimeRoot` come from the shared owner; executable lookup verifies absolute paths or resolves a bare name against the sandbox PATH plus explicit overrides. -- **Linux process groups** — a quoted wrapper starts each argv under `exec setsid --wait` and records its actual process-group id plus private status files beneath `ctx.e2b.runtimeRoot/processes`. The handle waits for that file instead of treating the SDK command PID as its published identity. Termination signals the negative recorded id with `SIGTERM`, waits the caller's `graceMs`, then escalates to `SIGKILL` and the SDK kill fallback; TERM delivery or probe failures also force that escalation. A failed transaction is observable through `waitForExit()` and may be retried, while successful termination remains a permanent no-op. Before publication, cancellation uses the provisional SDK handle; if publication fails, rollback kills and verifies the provisional group before startup rejects. Service disposal rejects new starts, terminates and joins every retained process group, then awaits SDK settlement and private cleanup before the sandbox owner disposes. -- **Environment boundary** — the wrapper starts from the sandbox command environment, removes ambient `DSH_*` and credential-shaped (`*KEY*`, `*SECRET*`, `*TOKEN*`) names, then restores every `spec.env` entry as an explicit caller opt-in. Host ambient variables never enter the sandbox implicitly. Private environment files are removed after consumption, and failed command or terminal setup removes its private state before rejecting. -- **Stdio projection** — the remote wrapper branches raw bytes into optional bounded spill files, frames each live chunk as newline-delimited base64 ASCII, and the host incrementally restores bytes across arbitrary SDK callback boundaries. Pipe mode writes those bytes to host Node streams; inherit mode writes them to the harness process streams; collect mode retains a bounded host tail with offset reads. The wrapper publishes the direct command status before waiting for inherited writers; after `graceMs`, the adapter disconnects an incomplete SDK stream, withholds its partial spill, and returns that status while retaining the remote group for `waitForExit()` and termination. Batch and streaming stdin use the SDK handle. -- **Terminal sessions** — `spawnTerminal()` uses E2B's byte PTY API, installs the exact argv and scrubbed environment through private mode-`0600` files, reports the foreground process group, sends real signals, and tears down every group in the remote terminal session before settlement. A private random output boundary discards the E2B bootstrap shell's prompt and echoed runner command while preserving every requested-process byte, including its first prompt. Setup and teardown own the private state transaction, fence publication during service disposal, and include failure cleanup. Prompt detection, scrollback, readiness, and owner policy remain in `dsh-pty-local`. +- **Linux process groups** — a quoted wrapper starts each argv under `exec setsid --wait` and records its actual process-group id plus private status files beneath `ctx.e2b.runtimeRoot/processes`. The handle waits for that file instead of treating the SDK command PID as its published identity. Termination signals the negative recorded id with `SIGTERM`, waits the caller's `graceMs`, then escalates to `SIGKILL` and the SDK kill fallback; TERM delivery or probe failures also force that escalation. A failed transaction is observable through `waitForExit()` and may be retried, while any proven quiescence permanently fences later termination against PID reuse. Before publication, cancellation uses the provisional SDK handle; if publication fails, rollback kills and verifies the provisional group before startup rejects. After publication, a monitoring failure also rolls back the group before rejecting. Service disposal rejects new starts, terminates and joins every retained process group, then awaits SDK settlement and private cleanup before the sandbox owner disposes. +- **Environment boundary** — the wrapper starts from the sandbox command environment, removes ambient `DSH_*` and credential-shaped (`*KEY*`, `*SECRET*`, `*TOKEN*`) names, then restores every valid `spec.env` entry as an explicit caller opt-in; empty names, `=`, and NUL framing violations reject before launch. Host ambient variables never enter the sandbox implicitly. Private environment files are removed after consumption, and failed command or terminal setup removes its private state before rejecting. +- **Stdio projection** — the remote wrapper branches raw bytes into optional bounded spill files, frames each live chunk as newline-delimited base64 ASCII, and the host incrementally restores bytes across arbitrary SDK callback boundaries. Pipe mode writes those bytes to host Node streams; inherit mode writes them to the harness process streams; collect mode retains a bounded host tail with offset reads. The wrapper publishes the direct command status before waiting for inherited writers. For collect or inherit output, the adapter disconnects an incomplete SDK stream after `graceMs`, withholds its partial spill, and returns that status while retaining the remote group for `waitForExit()` and termination; raw pipes instead await lossless transport completion and preserve backpressure. Batch and streaming stdin use the SDK handle. +- **Terminal sessions** — `spawnTerminal()` uses E2B's byte PTY API, installs the exact argv and scrubbed environment through private mode-`0600` files, reports the foreground process group, sends real signals, and tears down every group in the remote terminal session before settlement. A private random output boundary discards the E2B bootstrap shell's prompt and echoed runner command while preserving every requested-process byte, including its first prompt. Setup and teardown own the private state transaction, abort pending setup during service disposal, fence publication, and retain an unproven setup cleanup for disposal retry. Prompt detection, scrollback, readiness, and owner policy remain in `dsh-pty-local`. The base E2B image supplies the runtime and Bash/GNU utilities this adapter invokes: `node`, `bash`, `setsid`, `ps`, `awk`, `tr`, `env`, `chmod`, `tee`, `head`, `rm`, and `kill`. A custom template must retain compatible commands and E2B PTY support. @@ -29,6 +29,6 @@ No direct invalidation; the named consumers own any request-prefix changes. - **Synchronous-PID consumers are unsupported** — `pid` remains `-1` during remote startup; consumers that require a positive PID immediately, including the ACP child backend, cannot use this provider unchanged. - **Reconnect does not reconstruct handles** — remote PID/status/spill files survive a retained sandbox, but a new harness process does not rebuild live `SubprocessHandle` objects or output cursors from them. - **Remote state accumulates when retained** — process directories and valid spill files remain under `.dsh-e2b`; this POC supplies no retention sweep. -- **E2B exposes no signal fact** — only an adapter-requested `SIGTERM` or `SIGKILL` is reported as a signal; every unrequested SDK exit remains an exit code, including values shaped like `128 + signal`. +- **E2B exposes no signal fact** — an adapter-requested `SIGTERM` or `SIGKILL` is reported only when no wrapper-published direct exit code wins; every unrequested SDK exit remains an exit code, including values shaped like `128 + signal`. - **Exact terminal stdin-wait inspection is unavailable** — E2B exposes the foreground process group but not the syscall evidence needed to prove it is waiting on fd 0, so the generic PTY backend falls back to controlled prompt markers and bounded silence. - **Linux utility and E2B transport semantics are assumed** — there is no Windows, arbitrary-template, escaped-session recovery, or network-partition fidelity layer. diff --git a/packages/e2b/subprocess-e2b/README.zh.md b/packages/e2b/subprocess-e2b/README.zh.md index ec012b6a05..c4b318184c 100644 --- a/packages/e2b/subprocess-e2b/README.zh.md +++ b/packages/e2b/subprocess-e2b/README.zh.md @@ -8,10 +8,10 @@ - **异步远程启动**:同步 seam 会立即返回一个句柄,同时由 `Sandbox.commands.run(..., { background: true })` 在远程启动进程。包装层发布进程组 ID 并由适配器完成验证之前,`pid` 为 `-1`;stdin 和常规观察会等待该发布,而取消操作可以先停止临时 SDK 句柄。 - **执行世界坐标**:`cwd` 和私有 `runtimeRoot` 来自共享所有者;可执行文件查找会验证绝对路径,或根据沙箱 PATH 加显式覆盖来解析裸名称。 -- **Linux 进程组**:带引号保护的包装层会在 `exec setsid --wait` 下启动每组 argv,并在 `ctx.e2b.runtimeRoot/processes` 下记录实际进程组 ID 和私有状态文件。句柄会等待该文件,而不会把 SDK 命令 PID 当作已发布的身份。终止操作以记录的负数 ID 发送 `SIGTERM`,等待调用方的 `graceMs`,再升级到 `SIGKILL` 和 SDK kill 回退;TERM 信号发送或探测失败也会强制触发该升级。失败的事务可通过 `waitForExit()` 观察,并可重试;成功终止后,重复终止将永久为空操作。发布前,取消操作使用临时 SDK 句柄;如果发布失败,回滚会终止并验证临时进程组,随后启动操作才会拒绝。服务 dispose(资源释放)会拒绝新的启动请求、终止并等待每个保留进程组退出,再等待 SDK 结算和私有清理完成,之后沙箱所有者才会释放。 -- **环境边界**:包装层从沙箱命令环境开始,移除环境中的 `DSH_*` 和形似凭据的名称(`*KEY*`、`*SECRET*`、`*TOKEN*`),再把每个 `spec.env` 条目恢复为调用方显式选择。宿主环境变量绝不会隐式进入沙箱。私有环境文件在使用后会被删除;命令或终端设置失败时,会先删除其私有状态再拒绝。 -- **stdio 投影**:远程包装层先把原始字节分流到可选的有界 spill 文件,再把每个实时分片编码为换行分隔的 base64 ASCII 帧;宿主会跨任意 SDK 回调边界增量恢复字节。pipe 模式把这些字节写入宿主 Node 流;inherit 模式把字节写入 harness 进程流;collect 模式保留有界的宿主尾部,并支持基于偏移量读取。包装层会在等待继承管道的写入方之前发布直接命令状态;超过 `graceMs` 后,适配器会断开未完成的 SDK 流,不公开其中不完整的 spill,并返回该状态,同时保留远程进程组供 `waitForExit()` 和终止操作使用。批量 stdin 和流式 stdin 都使用 SDK 句柄。 -- **终端会话**:`spawnTerminal()` 使用 E2B 的字节 PTY API,以 mode 为 `0600` 的私有文件传入原样 argv 与清理后的环境,报告前台进程组,发送真实信号,并在结算前清理远程终端会话中的每个进程组。私有随机输出边界会丢弃 E2B 引导 shell 的提示符和回显的 runner 命令,同时保留请求进程的每个字节,包括其第一个提示符。setup 与 teardown 负责私有状态事务,在服务 dispose 期间阻止发布,并包括失败清理。提示符检测、scrollback、就绪状态与所有者策略仍归 `dsh-pty-local` 所有。 +- **Linux 进程组**:带引号保护的包装层会在 `exec setsid --wait` 下启动每组 argv,并在 `ctx.e2b.runtimeRoot/processes` 下记录实际进程组 ID 和私有状态文件。句柄会等待该文件,而不会把 SDK 命令 PID 当作已发布的身份。终止操作以记录的负数 ID 发送 `SIGTERM`,等待调用方的 `graceMs`,再升级到 `SIGKILL` 和 SDK kill 回退;TERM 信号发送或探测失败也会强制触发该升级。失败的事务可通过 `waitForExit()` 观察,并可重试;任何已证明的完全停稳都会永久防止后续终止操作命中复用的 PID。发布前,取消操作使用临时 SDK 句柄;如果发布失败,回滚会终止并验证临时进程组,随后启动操作才会拒绝。发布后,监控失败也会在拒绝前回滚进程组。服务 dispose(资源释放)会拒绝新的启动请求、终止并等待每个保留进程组退出,再等待 SDK 结算和私有清理完成,之后沙箱所有者才会释放。 +- **环境边界**:包装层从沙箱命令环境开始,移除环境中的 `DSH_*` 和形似凭据的名称(`*KEY*`、`*SECRET*`、`*TOKEN*`),再把每个有效的 `spec.env` 条目恢复为调用方显式选择;空名称、`=` 和违反 NUL 分帧规则的条目会在启动前被拒绝。宿主环境变量绝不会隐式进入沙箱。私有环境文件在使用后会被删除;命令或终端设置失败时,会先删除其私有状态再拒绝。 +- **stdio 投影**:远程包装层先把原始字节分流到可选的有界 spill 文件,再把每个实时分片编码为换行分隔的 base64 ASCII 帧;宿主会跨任意 SDK 回调边界增量恢复字节。pipe 模式把这些字节写入宿主 Node 流;inherit 模式把字节写入 harness 进程流;collect 模式保留有界的宿主尾部,并支持基于偏移量读取。包装层会在等待继承管道的写入方之前发布直接命令状态。对于 collect 或 inherit 输出,超过 `graceMs` 后,适配器会断开未完成的 SDK 流,不公开其中不完整的 spill,并返回该状态,同时保留远程进程组供 `waitForExit()` 和终止操作使用;原始 pipe 则会等待无损传输完成并保留背压。批量 stdin 和流式 stdin 都使用 SDK 句柄。 +- **终端会话**:`spawnTerminal()` 使用 E2B 的字节 PTY API,以 mode 为 `0600` 的私有文件传入原样 argv 与清理后的环境,报告前台进程组,发送真实信号,并在结算前清理远程终端会话中的每个进程组。私有随机输出边界会丢弃 E2B 引导 shell 的提示符和回显的 runner 命令,同时保留请求进程的每个字节,包括其第一个提示符。setup 与 teardown 负责私有状态事务,在服务 dispose 期间中止待处理的 setup、阻止发布,并保留未证明已完成的 setup 清理事务,供 dispose 重试。提示符检测、scrollback、就绪状态与所有者策略仍归 `dsh-pty-local` 所有。 基础 E2B 镜像提供该适配器调用的运行时和 Bash/GNU 工具:`node`、`bash`、`setsid`、`ps`、`awk`、`tr`、`env`、`chmod`、`tee`、`head`、`rm` 和 `kill`。自定义模板必须保留兼容的命令和 E2B PTY 支持。 @@ -29,6 +29,6 @@ - **不支持需要同步 PID 的消费方**:远程启动期间,`pid` 保持为 `-1`;包括 ACP 子进程后端在内,要求立即获得正 PID 的消费方无法原样使用本提供方。 - **重新连接不会重建句柄**:保留沙箱后,远程 PID/状态/spill 文件仍然存在,但新的 harness 进程不会据此重建实时 `SubprocessHandle` 对象或输出游标。 - **保留沙箱时会累积远程状态**:进程目录和有效的 spill 文件会留在 `.dsh-e2b` 下;本 POC 不提供保留清理。 -- **E2B 不公开信号事实**:只有适配器请求的 `SIGTERM` 或 `SIGKILL` 会报告为信号;其他未请求的 SDK 退出始终保留为退出码,包括形似 `128 + signal` 的值。 +- **E2B 不公开信号事实**:适配器请求的 `SIGTERM` 或 `SIGKILL` 只有在包装层发布的直接退出码没有胜出时才报告为信号;其他未请求的 SDK 退出始终保留为退出码,包括形似 `128 + signal` 的值。 - **无法精确检查终端 stdin 等待状态**:E2B 会公开前台进程组,但不提供证明其正在等待 fd 0 所需的 syscall 证据,因此通用 PTY 后端会回退到受控提示符标记与有界静默机制。 - **依赖 Linux 工具与 E2B 传输语义**:没有 Windows、任意模板、逃逸会话恢复或网络分区的保真层。 diff --git a/packages/e2b/subprocess-e2b/src/environment.ts b/packages/e2b/subprocess-e2b/src/environment.ts index cc9b41296a..f9c30d378f 100644 --- a/packages/e2b/subprocess-e2b/src/environment.ts +++ b/packages/e2b/subprocess-e2b/src/environment.ts @@ -19,3 +19,23 @@ export function scrubRemoteEnvironment(raw: string): Map { } return environment } + +/** + * Overlay explicit entries and serialize one validated E2B environment. + * @param raw - The complete NUL-delimited remote environment. + * @param explicit - Deliberate caller overrides applied after ambient scrubbing. + * @returns NUL-delimited `name=value` entries accepted by `env -i`. + */ +export function serializeRemoteEnvironment( + raw: string, + explicit: Readonly> | undefined, +): string { + const environment = scrubRemoteEnvironment(raw) + for (const [name, value] of Object.entries(explicit ?? {})) { + if (name.length === 0 || name.includes('=') || name.includes('\0') || value.includes('\0')) { + throw new Error('subprocess-e2b: environment entries require non-empty NUL-free names without = and NUL-free values') + } + environment.set(name, value) + } + return [...environment].map(([name, value]) => `${name}=${value}\0`).join('') +} diff --git a/packages/e2b/subprocess-e2b/src/index.ts b/packages/e2b/subprocess-e2b/src/index.ts index fcf2c6f0a4..bf7edc41a1 100644 --- a/packages/e2b/subprocess-e2b/src/index.ts +++ b/packages/e2b/subprocess-e2b/src/index.ts @@ -28,7 +28,8 @@ export class E2BSubprocessService extends SubprocessService { private readonly live = new Set() private readonly terminals = new Set() - private readonly terminalSetups = new Set>() + private readonly terminalSetups = new Map, AbortController>() + private readonly failedTerminalSetupCleanups = new Set<() => Promise>() private disposing = false /** @inheritdoc */ @@ -44,9 +45,13 @@ export class E2BSubprocessService extends SubprocessService { this.runtimeRoot = ctx.e2b.runtimeRoot ctx.effect(() => async () => { this.disposing = true - await Promise.all([...this.terminalSetups]) + for (const controller of this.terminalSetups.values()) { + controller.abort(new Error('subprocess-e2b: service disposed during terminal setup')) + } + await Promise.all([...this.terminalSetups.keys()]) const handles = [...this.live] const terminals = [...this.terminals] + const failedTerminalSetupCleanups = [...this.failedTerminalSetupCleanups] const pending: Promise[] = [] for (const handle of handles) { handle.terminate() @@ -59,6 +64,9 @@ export class E2BSubprocessService extends SubprocessService { terminal.terminate() pending.push(terminal.waitForExit().then(() => { this.terminals.delete(terminal) })) } + for (const cleanup of failedTerminalSetupCleanups) { + pending.push(cleanup().then(() => { this.failedTerminalSetupCleanups.delete(cleanup) })) + } await Promise.all(pending) }, 'e2b subprocess teardown') } @@ -133,9 +141,18 @@ export class E2BSubprocessService extends SubprocessService { spec.signal?.throwIfAborted() const stateDir = posix.join(this.runtimeRoot, 'terminals', randomUUID()) const setup = Promise.withResolvers() - this.terminalSetups.add(setup.promise) + const setupController = new AbortController() + const setupSignal = spec.signal === undefined + ? setupController.signal + : AbortSignal.any([spec.signal, setupController.signal]) + this.terminalSetups.set(setup.promise, setupController) try { - const terminal = await spawnE2BTerminal(this.ctx.e2b, spec, stateDir) + const terminal = await spawnE2BTerminal( + this.ctx.e2b, + { ...spec, signal: setupSignal }, + stateDir, + (cleanup) => { this.failedTerminalSetupCleanups.add(cleanup) }, + ) this.terminals.add(terminal) if (this.isDisposing()) { terminal.terminate() diff --git a/packages/e2b/subprocess-e2b/src/process.ts b/packages/e2b/subprocess-e2b/src/process.ts index 3038f312dc..4855b27b81 100644 --- a/packages/e2b/subprocess-e2b/src/process.ts +++ b/packages/e2b/subprocess-e2b/src/process.ts @@ -17,7 +17,7 @@ import type { SubprocessSpawnSpec, } from '@deepseek-ai/dsh-subprocess' import type E2BSandboxService from '@deepseek-ai/dsh-e2b' -import { scrubRemoteEnvironment } from './environment.ts' +import { serializeRemoteEnvironment } from './environment.ts' import { E2BBase64Decoder, E2B_OUTPUT_COMPLETE_FRAME, E2BOutputReader } from './output.ts' const GROUP_POLL_MS = 20 @@ -92,12 +92,6 @@ function withinMs(promise: Promise, timeoutMs: number): Promise> | undefined): string { - const environment = scrubRemoteEnvironment(raw) - for (const [name, value] of Object.entries(explicit ?? {})) environment.set(name, value) - return [...environment].map(([name, value]) => `${name}=${value}\0`).join('') -} - function commandText(spec: SubprocessSpawnSpec, paths: RemotePaths): string { const encoder = `"$dsh_e2b_env_bin" -i "$dsh_e2b_node" -e ${quoteE2BShellArg(OUTPUT_ENCODER_SOURCE)}` const stdoutRedirect = hasSpill(spec.stdio.stdout) @@ -213,7 +207,8 @@ export class E2BSubprocessHandle implements SubprocessHandle { private invalidHandleQuiescent = false private provisionalHandleQuiescent = false private terminationStarted = false - private terminationSucceeded = false + private terminationFenced = false + private quiescenceProven = false private terminationAttempt: Promise | undefined private terminationFailure: Error | undefined private terminationSignal: NodeJS.Signals | null = null @@ -265,18 +260,18 @@ export class E2BSubprocessHandle implements SubprocessHandle { /** @inheritdoc */ terminate(): void { - if (this.terminationSucceeded || this.terminationAttempt !== undefined) return + if (this.terminationFenced || this.quiescenceProven || this.terminationAttempt !== undefined) return this.terminationStarted = true this.terminationFailure = undefined const attempt = this.terminateRemote() this.terminationAttempt = attempt void attempt.then( () => { - this.terminationSucceeded = true + this.terminationFenced = true this.terminationAttempt = undefined }, (error: unknown) => { - this.terminationFailure = asError(error) + if (!this.quiescenceProven) this.terminationFailure = asError(error) this.terminationAttempt = undefined }, ) @@ -284,17 +279,24 @@ export class E2BSubprocessHandle implements SubprocessHandle { /** @inheritdoc */ async waitForExit(signal?: AbortSignal): Promise { + if (this.quiescenceProven) return true let handle: CommandHandle | undefined if (this.terminationStarted) { const observed = await waitWithSignal(this.commandState.promise, signal) if (observed === WAIT_ABORTED) return false handle = observed - if (handle === undefined) return true + if (handle === undefined) { + this.markQuiescent() + return true + } if (this.remotePid <= 0) { const attempt = this.terminationAttempt if (attempt !== undefined && await waitWithSignal(attempt, signal) === WAIT_ABORTED) return false this.throwTerminationFailure() - if (this.invalidHandleQuiescent || this.provisionalHandleQuiescent) return true + if (this.invalidHandleQuiescent || this.provisionalHandleQuiescent) { + this.markQuiescent() + return true + } } } else { try { @@ -303,7 +305,10 @@ export class E2BSubprocessHandle implements SubprocessHandle { handle = observed } catch { handle = this.commandHandle - if (handle === undefined) return true + if (handle === undefined) { + this.markQuiescent() + return true + } } } this.throwTerminationFailure() @@ -320,11 +325,18 @@ export class E2BSubprocessHandle implements SubprocessHandle { if (!await waitTick(signal)) return false } this.throwTerminationFailure() - return !isAborted(signal) + if (isAborted(signal)) return false + this.markQuiescent() + return true } private readonly onAbort = (): void => { this.terminate() } + private markQuiescent(): void { + this.quiescenceProven = true + this.terminationFailure = undefined + } + private async run(): Promise { let sandbox: Sandbox | undefined try { @@ -385,7 +397,7 @@ export class E2BSubprocessHandle implements SubprocessHandle { return outcome } catch (error: unknown) { this.commandState.resolve(undefined) - let failure = error + let failure = await this.rollbackPublishedFailure(error) if (sandbox !== undefined && this.stateDirectoryCreated) { try { await this.removeFailedState(sandbox) @@ -413,7 +425,7 @@ export class E2BSubprocessHandle implements SubprocessHandle { const files = [ { path: this.paths.pid, data: '' }, { path: this.paths.status, data: '' }, - { path: this.paths.environment, data: remoteEnvironment(ambient.stdout, this.spec.env) }, + { path: this.paths.environment, data: serializeRemoteEnvironment(ambient.stdout, this.spec.env) }, ...(hasSpill(this.spec.stdio.stdout) ? [{ path: this.paths.stdout, data: '' }] : []), ...(hasSpill(this.spec.stdio.stderr) ? [{ path: this.paths.stderr, data: '' }] : []), ] @@ -508,15 +520,16 @@ export class E2BSubprocessHandle implements SubprocessHandle { if (!/^(?:0|[1-9][0-9]*)$/.test(rawStatus) || !Number.isSafeInteger(exitCode) || exitCode > 255) { throw new Error(`subprocess-e2b: remote wrapper published invalid exit code ${JSON.stringify(rawStatus)}`) } + if (this.spec.stdio.stdout === 'pipe' || this.spec.stdio.stderr === 'pipe') { + return this.commandOutcome(await settlement) + } const completed = await withinMs(settlement, this.spec.graceMs) if (completed !== undefined) return this.commandOutcome(completed) this.outputDrainExpired = true this.stdoutReader?.invalidateSpill() this.stderrReader?.invalidateSpill() await handle.disconnect() - return this.terminationSignal === null - ? { exitCode, signal: null } - : { exitCode: null, signal: this.terminationSignal } + return { exitCode, signal: null } } const completed = await Promise.race([settlement, waitTick().then(() => undefined)]) if (completed !== undefined) return this.commandOutcome(completed) @@ -533,6 +546,20 @@ export class E2BSubprocessHandle implements SubprocessHandle { throw settlement.error } + private async rollbackPublishedFailure(error: unknown): Promise { + if (this.remotePid <= 0 || this.commandHandle === undefined || this.quiescenceProven) return error + this.terminate() + try { + await this.waitForExit() + return error + } catch (cleanupError: unknown) { + return new AggregateError( + [asError(error), asError(cleanupError)], + 'subprocess-e2b: command monitoring failed and process-group rollback did not reach quiescence', + ) + } + } + private async rollbackUnpublishedGroup(sandbox: Sandbox, handle: CommandHandle): Promise { // The bootstrap ends in an exec chain through the scrubbed environment and // `setsid`, so E2B's command PID is the provisional group id even before the diff --git a/packages/e2b/subprocess-e2b/src/terminal.ts b/packages/e2b/subprocess-e2b/src/terminal.ts index 78668490d7..0f5809fa44 100644 --- a/packages/e2b/subprocess-e2b/src/terminal.ts +++ b/packages/e2b/subprocess-e2b/src/terminal.ts @@ -19,7 +19,7 @@ import type { SubprocessTerminalSpawnSpec, } from '@deepseek-ai/dsh-subprocess' import type E2BSandboxService from '@deepseek-ai/dsh-e2b' -import { scrubRemoteEnvironment } from './environment.ts' +import { serializeRemoteEnvironment } from './environment.ts' const POLL_MS = 20 @@ -142,17 +142,6 @@ function serializeValues(values: readonly string[], kind: string): string { return values.map(value => `${value}\0`).join('') } -function remoteEnvironment(raw: string, explicit: Readonly> | undefined): string { - const environment = scrubRemoteEnvironment(raw) - for (const [name, value] of Object.entries(explicit ?? {})) { - if (name.length === 0 || name.includes('=') || name.includes('\0') || value.includes('\0')) { - throw new Error('subprocess-e2b: terminal environment entries require non-empty NUL-free names without = and NUL-free values') - } - environment.set(name, value) - } - return serializeValues([...environment].map(([name, value]) => `${name}=${value}`), 'environment') -} - async function terminalSessionId(sandbox: Sandbox, pid: number, signal?: AbortSignal): Promise { const result = await sandbox.commands.run(`ps -o sid= -p ${pid}`, signalOpts(signal)) signal?.throwIfAborted() @@ -432,12 +421,14 @@ export class E2BTerminalHandle implements SubprocessTerminalHandle { * @param runtime - Shared E2B sandbox owner. * @param spec - Fully specified terminal-process request. * @param stateDir - Private remote directory for one startup transaction. + * @param retainFailedCleanup - Optional owner for retrying a cleanup transaction that could not prove quiescence. * @returns The live subprocess terminal handle. */ export async function spawnE2BTerminal( runtime: E2BSandboxService, spec: SubprocessTerminalSpawnSpec, stateDir: string, + retainFailedCleanup?: (cleanup: () => Promise) => void, ): Promise { const sandbox = await runtime.getSandbox() spec.signal?.throwIfAborted() @@ -456,7 +447,7 @@ export async function spawnE2BTerminal( let stateDirectoryCreated = false try { const ambient = await sandbox.commands.run('env -0', signalOpts(spec.signal)) - const environment = remoteEnvironment(ambient.stdout, spec.env) + const environment = serializeRemoteEnvironment(ambient.stdout, spec.env) const argv = serializeValues(spec.argv, 'argv') await sandbox.files.makeDir(stateDir) stateDirectoryCreated = true @@ -502,25 +493,37 @@ export async function spawnE2BTerminal( ) } catch (error: unknown) { output.destroy() - const cleanupErrors: Error[] = [] - if (handle !== undefined && completion !== undefined) { - try { - await rollbackUnpublishedTerminal(sandbox, handle, completion, spec.graceMs) - } catch (rollbackError: unknown) { - cleanupErrors.push(asError(rollbackError)) + let terminalQuiescent = handle === undefined + let stateRemoved = !stateDirectoryCreated + const retryCleanup = async (): Promise => { + const failures: Error[] = [] + if (!terminalQuiescent && handle !== undefined) { + try { + if (completion === undefined) await handle.kill() + else await rollbackUnpublishedTerminal(sandbox, handle, completion, spec.graceMs) + terminalQuiescent = true + } catch (cleanupError: unknown) { + failures.push(asError(cleanupError)) + } } - } else if (handle !== undefined) { - await handle.kill().catch(() => false) - } - if (stateDirectoryCreated) { - try { - await sandbox.files.remove(stateDir) - } catch (stateError: unknown) { - if (!(stateError instanceof FileNotFoundError)) cleanupErrors.push(asError(stateError)) + if (!stateRemoved) { + try { + await sandbox.files.remove(stateDir) + stateRemoved = true + } catch (stateError: unknown) { + if (stateError instanceof FileNotFoundError) stateRemoved = true + else failures.push(asError(stateError)) + } + } + if (failures.length > 0) { + throw new AggregateError(failures, 'subprocess-e2b: terminal setup cleanup did not complete') } } - if (cleanupErrors.length > 0) { - throw new AggregateError([asError(error), ...cleanupErrors], asError(error).message) + try { + await retryCleanup() + } catch (cleanupError: unknown) { + retainFailedCleanup?.(retryCleanup) + throw new AggregateError([asError(error), asError(cleanupError)], asError(error).message) } throw error } diff --git a/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts b/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts index 8e6e420bcc..462e943a1b 100644 --- a/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts +++ b/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts @@ -97,6 +97,7 @@ class FakeSandbox { startOptions: StartOptions | undefined backgroundError: unknown envError: unknown + statusError: unknown nextRemoveError: unknown probeError: unknown signalError: unknown @@ -117,6 +118,8 @@ class FakeSandbox { private openStart: (() => void) | undefined private processGroupReadGate: Promise | undefined private openProcessGroupRead: (() => void) | undefined + private signalGate: Promise | undefined + private openSignal: (() => void) | undefined constructor() { this.handle = new FakeCommandHandle(() => { @@ -147,6 +150,16 @@ class FakeSandbox { this.openProcessGroupRead?.() } + deferSignals(): void { + const gate = Promise.withResolvers() + this.signalGate = gate.promise + this.openSignal = () => { gate.resolve(undefined) } + } + + releaseSignals(): void { + this.openSignal?.() + } + finish(exitCode = 0): void { this.alive = false void this.completeOutput().then( @@ -198,6 +211,11 @@ class FakeSandbox { await this.processGroupReadGate return this.processGroupReads.shift() ?? this.processGroupId } + if (this.statusError !== undefined) { + const error = this.statusError + this.statusError = undefined + throw error + } this.afterStatusRead?.() return this.exitStatus }, @@ -230,6 +248,7 @@ class FakeSandbox { return { exitCode: 0, stdout: '', stderr: '' } } if (command.startsWith('kill -TERM ')) { + await this.signalGate const error = this.signalErrors.shift() ?? this.signalError if (error !== undefined) { if (this.signalErrors.length === 0) this.signalError = undefined @@ -242,6 +261,7 @@ class FakeSandbox { return { exitCode: 0, stdout: '', stderr: '' } } if (command.startsWith('kill -KILL ')) { + await this.signalGate const error = this.signalErrors.shift() ?? this.signalError if (error !== undefined) { if (this.signalErrors.length === 0) this.signalError = undefined @@ -399,6 +419,16 @@ describe('E2BSubprocessHandle', () => { await expect(handle.waitForExit()).resolves.toBe(true) }) + it('rejects malformed environment entries before command start', async () => { + for (const env of [{ 'BAD=NAME': 'x' }, { BAD: 'x\0INJECTED=1' }]) { + const fake = new FakeSandbox() + const handle = new E2BSubprocessHandle(runtime(fake), spec({ env }), '/runtime/invalid-environment') + await expect(handle.done).rejects.toThrow('environment entries') + expect(fake.startOptions).toBeUndefined() + expect(fake.removed).toContain('/runtime/invalid-environment') + } + }) + it('preserves UTF-8 bytes when the ASCII transport is split across callbacks', async () => { const fake = new FakeSandbox() const handle = new E2BSubprocessHandle(runtime(fake), spec({ @@ -460,6 +490,29 @@ describe('E2BSubprocessHandle', () => { await expect(handle.waitForExit()).resolves.toBe(true) }) + it('waits for lossless raw-pipe output after the direct status is published', async () => { + const fake = new FakeSandbox() + const handle = new E2BSubprocessHandle(runtime(fake), spec({ + graceMs: 1, + stdio: { stdin: 'ignore', stdout: 'pipe', stderr: { maxBytes: 4 } }, + }), '/runtime/pipe-drain') + let output = '' + handle.stdout!.on('data', (chunk) => { output += String(chunk) }) + await flush() + fake.exitStatus = '0\n' + + let settled = false + void handle.done.then(() => { settled = true }) + await new Promise(resolve => setTimeout(resolve, 50)) + expect(settled).toBe(false) + expect(fake.handle.disconnects).toBe(0) + + await fake.stdout('complete protocol frame') + fake.finish() + await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) + expect(output).toBe('complete protocol frame') + }) + it('accepts clean encoder completion inside the output-drain grace', async () => { const fake = new FakeSandbox() const handle = new E2BSubprocessHandle(runtime(fake), spec({ graceMs: 100 }), '/runtime/drain-complete') @@ -474,7 +527,7 @@ describe('E2BSubprocessHandle', () => { expect(fake.handle.disconnects).toBe(0) }) - it('preserves a requested signal when output draining expires', async () => { + it('preserves a published exit code when requested termination outlives output draining', async () => { const fake = new FakeSandbox() fake.trapsTerm = true fake.delaysKill = true @@ -485,9 +538,9 @@ describe('E2BSubprocessHandle', () => { handle.terminate() await vi.waitFor(() => { expect(fake.commandsSeen).toContain('kill -KILL -- -4242') }) - fake.exitStatus = '143\n' + fake.exitStatus = '0\n' - await expect(handle.done).resolves.toEqual({ exitCode: null, signal: 'SIGKILL' }) + await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) expect(fake.handle.disconnects).toBe(1) fake.alive = false await expect(handle.waitForExit()).resolves.toBe(true) @@ -503,6 +556,31 @@ describe('E2BSubprocessHandle', () => { await expect(handle.waitForExit()).resolves.toBe(true) }) + it('rolls back a published process group before rejecting a monitoring failure', async () => { + const fake = new FakeSandbox() + fake.statusError = new Error('status transport failed') + const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/status-failure') + + await expect(handle.done).rejects.toThrow('status transport failed') + expect(fake.commandsSeen).toContain('kill -TERM -- -4242') + expect(fake.alive).toBe(false) + await expect(handle.waitForExit()).resolves.toBe(true) + + const failed = new FakeSandbox() + failed.statusError = new Error('status transport failed') + failed.signalErrors.push(new Error('TERM transport failed'), new Error('KILL transport failed')) + failed.handle.killError = new Error('SDK kill failed') + const retained = new E2BSubprocessHandle(runtime(failed), spec({ graceMs: 1 }), '/runtime/status-cleanup-failure') + + await expect(retained.done).rejects.toThrow( + 'command monitoring failed and process-group rollback did not reach quiescence', + ) + expect(failed.alive).toBe(true) + failed.handle.killError = undefined + retained.terminate() + await expect(retained.waitForExit()).resolves.toBe(true) + }) + it('surfaces deferred piped-stdin write and close failures as stream errors', async () => { const writeFake = new FakeSandbox() writeFake.deferStart() @@ -607,6 +685,46 @@ describe('E2BSubprocessHandle', () => { expect(fake.commandsSeen.filter(command => command.startsWith('kill -'))).toHaveLength(signals) }) + it('makes termination a permanent no-op after natural quiescence is observed', async () => { + const fake = new FakeSandbox() + const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/natural-quiescence') + await flush() + fake.finish() + await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) + await expect(handle.waitForExit()).resolves.toBe(true) + + const signals = fake.commandsSeen.filter(command => command.startsWith('kill -')).length + fake.alive = true + handle.terminate() + await flush() + expect(fake.alive).toBe(true) + expect(fake.commandsSeen.filter(command => command.startsWith('kill -'))).toHaveLength(signals) + }) + + it('keeps proven quiescence after a concurrent termination transport fails', async () => { + const fake = new FakeSandbox() + fake.signalErrors.push(new Error('TERM transport failed'), new Error('KILL transport failed')) + fake.handle.killError = new Error('SDK kill failed') + fake.deferSignals() + const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/quiescent-race') + await flush() + + handle.terminate() + await vi.waitFor(() => { expect(fake.commandsSeen).toContain('kill -TERM -- -4242') }) + fake.alive = false + await expect(handle.waitForExit()).resolves.toBe(true) + + fake.probeError = new Error('post-quiescence probe failed') + fake.releaseSignals() + await vi.waitFor(() => { expect(fake.handle.kills).toBe(1) }) + fake.finish() + await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) + const signals = fake.commandsSeen.filter(command => command.startsWith('kill -')).length + handle.terminate() + await expect(handle.waitForExit()).resolves.toBe(true) + expect(fake.commandsSeen.filter(command => command.startsWith('kill -'))).toHaveLength(signals) + }) + it('escalates a TERM-trapping process group to KILL and uses the SDK kill as fallback', async () => { const fake = new FakeSandbox() fake.trapsTerm = true @@ -620,6 +738,28 @@ describe('E2BSubprocessHandle', () => { expect(fake.handle.kills).toBe(1) }) + it('does not treat successful termination transport as observed quiescence', async () => { + const fake = new FakeSandbox() + fake.trapsTerm = true + fake.delaysKill = true + fake.delaysKillCompletion = true + fake.sdkKillStops = false + const handle = new E2BSubprocessHandle(runtime(fake), spec({ graceMs: 1 }), '/runtime/termination-fence') + await flush() + handle.terminate() + await vi.waitFor(() => { expect(fake.handle.kills).toBe(1) }) + + let quiescent = false + const waiting = handle.waitForExit().then((value) => { quiescent = value }) + await new Promise(resolve => setTimeout(resolve, 10)) + expect(quiescent).toBe(false) + + fake.alive = false + fake.finish() + await waiting + expect(quiescent).toBe(true) + }) + it('honors termination requested before asynchronous startup finishes', async () => { const fake = new FakeSandbox() fake.deferStart() diff --git a/packages/e2b/subprocess-e2b/tests/terminal.spec.ts b/packages/e2b/subprocess-e2b/tests/terminal.spec.ts index 1eac9a5780..e77646e586 100644 --- a/packages/e2b/subprocess-e2b/tests/terminal.spec.ts +++ b/packages/e2b/subprocess-e2b/tests/terminal.spec.ts @@ -111,18 +111,7 @@ class FakeTerminalSandbox { resolvedExecutable = '/usr/bin/node\n' requestedOutput = 'requested-shell$ ' emitOutputMarker = true - private createGate: Promise | undefined - private releaseCreateGate: (() => void) | undefined - - deferCreate(): void { - const gate = Promise.withResolvers() - this.createGate = gate.promise - this.releaseCreateGate = () => { gate.resolve(undefined) } - } - - releaseCreate(): void { - this.releaseCreateGate?.() - } + afterSessionLookup: (() => void) | undefined readonly sandbox = { files: { @@ -163,7 +152,10 @@ class FakeTerminalSandbox { if (command.includes('command -v -- ')) { return { exitCode: 0, stdout: this.resolvedExecutable, stderr: '' } } - if (command.startsWith('ps -o sid=')) return { exitCode: 0, stdout: this.sessionId, stderr: '' } + if (command.startsWith('ps -o sid=')) { + this.afterSessionLookup?.() + return { exitCode: 0, stdout: this.sessionId, stderr: '' } + } if (command.startsWith('ps -o tpgid=')) { if (this.foregroundFailure !== undefined) throw this.foregroundFailure return { exitCode: 0, stdout: this.foreground, stderr: '' } @@ -186,7 +178,6 @@ class FakeTerminalSandbox { pty: { create: async (options: Parameters[0]): Promise => { this.createOptions = options - await this.createGate if (this.createError !== undefined) throw this.createError await options.onData(Buffer.from('buffered banner\n')) return this.handle.asHandle() @@ -691,7 +682,7 @@ describe('E2B subprocess terminal service', () => { it('owns live terminals through service disposal', async () => { const { ctx, fiber, fake } = await service() - const terminal = await ctx.subprocess.spawnTerminal(spec()) + const terminal = await ctx.subprocess.spawnTerminal(spec({ signal: new AbortController().signal })) await fiber.dispose() await expect(terminal.done).resolves.toEqual({ exitCode: null, signal: 'SIGTERM' }) expect(fake.handle.disconnects).toBe(1) @@ -699,19 +690,19 @@ describe('E2B subprocess terminal service', () => { it('joins and rejects terminal setup that completes during service disposal', async () => { const fake = new FakeTerminalSandbox() - fake.deferCreate() const { ctx, fiber } = await service(fake) + let disposing: Promise | undefined + fake.afterSessionLookup = () => { + fake.afterSessionLookup = undefined + queueMicrotask(() => { + queueMicrotask(() => { disposing = fiber.dispose() }) + }) + } + const subprocess = ctx.subprocess const spawning = ctx.subprocess.spawnTerminal(spec()) const rejected = expect(spawning).rejects.toThrow('service disposed during terminal setup') - await vi.waitFor(() => { expect(fake.createOptions).toBeDefined() }) - - let disposed = false - const subprocess = ctx.subprocess - const disposing = fiber.dispose().then(() => { disposed = true }) - await new Promise(resolve => setTimeout(resolve, 0)) - expect(disposed).toBe(false) + await vi.waitFor(() => { expect(disposing).toBeDefined() }) await expect(subprocess.spawnTerminal(spec())).rejects.toThrow('service is disposing') - fake.releaseCreate() await rejected await disposing @@ -720,6 +711,37 @@ describe('E2B subprocess terminal service', () => { expect(fake.removed.some(path => path.includes('/terminals/'))).toBe(true) }) + it('aborts and rolls back terminal setup that cannot publish readiness during disposal', async () => { + const fake = new FakeTerminalSandbox() + fake.ready = new FileNotFoundError('not ready') + const { ctx, fiber } = await service(fake) + const spawning = ctx.subprocess.spawnTerminal(spec()) + const rejected = expect(spawning).rejects.toThrow('service disposed during terminal setup') + await vi.waitFor(() => { expect(fake.readyReads).toBeGreaterThan(0) }) + + await fiber.dispose() + await rejected + expect(fake.groups).toEqual([]) + expect(fake.handle.disconnects).toBe(1) + }) + + it('retains failed terminal setup cleanup for disposal retry', async () => { + const fake = new FakeTerminalSandbox() + fake.sendError = new Error('bootstrap failed') + fake.clearOnTerm = false + fake.clearOnKill = false + const { ctx, fiber } = await service(fake) + + await expect(ctx.subprocess.spawnTerminal(spec({ graceMs: 1 }))).rejects.toThrow('bootstrap failed') + expect(fake.groups).toEqual([123]) + expect(fake.handle.disconnects).toBe(0) + + fake.clearOnKill = true + await fiber.dispose() + expect(fake.groups).toEqual([]) + expect(fake.handle.disconnects).toBe(1) + }) + it('releases naturally settled terminals and validates terminal requests', async () => { const { ctx, fiber, fake } = await service() for (const request of [ From d0648446c97e426c53deefe7d41cc25b1016bbaa Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 09:54:14 +0800 Subject: [PATCH 083/108] fix(e2b): close final lifecycle gaps --- packages/e2b/e2b/README.i18n.yaml | 4 +- packages/e2b/e2b/README.md | 2 +- packages/e2b/e2b/README.zh.md | 2 +- packages/e2b/e2b/src/index.ts | 17 +++++++-- packages/e2b/e2b/tests/e2b.spec.ts | 22 ++++++++++- packages/e2b/subprocess-e2b/README.i18n.yaml | 4 +- packages/e2b/subprocess-e2b/README.md | 4 +- packages/e2b/subprocess-e2b/README.zh.md | 4 +- packages/e2b/subprocess-e2b/src/process.ts | 17 +++++---- packages/e2b/subprocess-e2b/src/terminal.ts | 2 +- .../subprocess-e2b/tests/subprocess.spec.ts | 38 +++++++++++++++++-- .../e2b/subprocess-e2b/tests/terminal.spec.ts | 22 ++++++++++- 12 files changed, 111 insertions(+), 27 deletions(-) diff --git a/packages/e2b/e2b/README.i18n.yaml b/packages/e2b/e2b/README.i18n.yaml index d8f1dcdc3f..46ff37b222 100644 --- a/packages/e2b/e2b/README.i18n.yaml +++ b/packages/e2b/e2b/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/e2b/e2b/README.md -README.md: 01c7ade2f5f31e34091ad0ae910287e659f15549 -README.zh.md: fb95a12cbbbd431ccb3283a14bb63b324f9bdea6 +README.md: ccafa9df480e3812d3d3b6d25b513e7b5d2afc9f +README.zh.md: 60eefd46a0a9a017f4db57e4dc5313d533eeba04 diff --git a/packages/e2b/e2b/README.md b/packages/e2b/e2b/README.md index 01c7ade2f5..ccafa9df48 100644 --- a/packages/e2b/e2b/README.md +++ b/packages/e2b/e2b/README.md @@ -30,7 +30,7 @@ Set `sandboxId` to reconnect a running or paused sandbox instead of creating one Construction starts one create/connect operation. Before resolving `getSandbox()`, the service creates `cwd` and the private `cwd/.dsh-e2b` adapter-state directory, verifies that the reserved path is a real directory rather than a symlink or another file type, then sets it to mode `0700`. `sandboxId` resolves to a branded `E2BSandboxId` after setup. -Disposal first prevents new handle acquisition, then awaits setup and applies exactly one configured disposition. A `SandboxNotFoundError` means a kill-on-timeout sandbox is already quiescent; every other disposition failure rejects teardown. A newly created sandbox is killed when initial directory setup fails; a reconnected sandbox is not killed on setup failure because the service did not create it. Provider plugins must load after this owner and dispose before it. +Disposal first prevents new handle acquisition, then awaits setup and applies exactly one configured disposition. A `SandboxNotFoundError` means a kill-on-timeout sandbox is already quiescent; every other disposition failure rejects teardown. A newly created sandbox is killed when initial directory setup fails; if that rollback fails, disposal retries it before releasing ownership. A reconnected sandbox is not killed on setup failure because the service did not create it. Provider plugins must load after this owner and dispose before it. `pause` and `leave` retain remote filesystem and adapter artifacts for a later `sandboxId` connection, but a later harness process receives only a new SDK handle. The subprocess service still fulfills its seam contract by terminating managed groups before owner disposal; neither disposition recovers prior process objects, output cursors, or in-memory adapter locks. diff --git a/packages/e2b/e2b/README.zh.md b/packages/e2b/e2b/README.zh.md index fb95a12cbb..60eefd46a0 100644 --- a/packages/e2b/e2b/README.zh.md +++ b/packages/e2b/e2b/README.zh.md @@ -30,7 +30,7 @@ 构造阶段会启动一次 create/connect 操作。服务在 `getSandbox()` 结算前创建 `cwd` 和私有的 `cwd/.dsh-e2b` 适配器状态目录,验证该预留路径是真实目录而非符号链接或其他文件类型,再把该目录的 mode 设为 `0700`。初始化完成后,`sandboxId` 会结算为品牌类型 `E2BSandboxId`。 -资源释放会先阻止继续获取新句柄,再等待初始化完成,并且只应用一种已配置的处置方式。`SandboxNotFoundError` 表示因超时终止的沙箱已经完全停稳;其他处置失败都会使 teardown 拒绝。新建沙箱的初始目录设置失败时,服务会终止该沙箱;重新连接的沙箱设置失败时不会被终止,因为它不是由本服务创建的。提供方插件必须在该所有者之后加载,并在其之前 dispose(资源释放)。 +资源释放会先阻止继续获取新句柄,再等待初始化完成,并且只应用一种已配置的处置方式。`SandboxNotFoundError` 表示因超时终止的沙箱已经完全停稳;其他处置失败都会使 teardown 拒绝。新建沙箱的初始目录设置失败时,服务会终止该沙箱;如果该回滚失败,资源释放会在解除所有权前重试。重新连接的沙箱设置失败时不会被终止,因为它不是由本服务创建的。提供方插件必须在该所有者之后加载,并在其之前 dispose(资源释放)。 `pause` 和 `leave` 会保留远程文件系统及适配器产物,供稍后的 `sandboxId` 连接使用,但后续 harness 进程只会获得新的 SDK 句柄。进程管理服务仍会履行其 seam 契约,在所有者释放前终止受管进程组;这两种处置方式都不会恢复先前的进程对象、输出游标或内存中的适配器锁。 diff --git a/packages/e2b/e2b/src/index.ts b/packages/e2b/e2b/src/index.ts index 2c0b3c635c..31e4f911a2 100644 --- a/packages/e2b/e2b/src/index.ts +++ b/packages/e2b/e2b/src/index.ts @@ -120,6 +120,7 @@ export class E2BSandboxService extends Service { private readonly config: ResolvedConfig private readonly ready: Promise + private failedSetupSandbox: Sandbox | undefined private disposed = false constructor(ctx: Context, config: Config) { @@ -155,8 +156,16 @@ export class E2BSandboxService extends Service { try { sandbox = await this.ready } catch { - // Connection creation already failed and is exposed by getSandbox(); - // there is no remote resource for teardown to own. + const failedSetupSandbox = this.failedSetupSandbox + if (failedSetupSandbox === undefined) return + sandbox = failedSetupSandbox + try { + await sandbox.kill() + this.failedSetupSandbox = undefined + } catch (error: unknown) { + if (!(error instanceof SandboxNotFoundError)) throw error + this.failedSetupSandbox = undefined + } return } try { @@ -243,7 +252,9 @@ export class E2BSandboxService extends Service { try { await sandbox.kill() } catch (_cleanupFailure) { - // The setup failure remains authoritative; E2B will still apply the configured lifetime. + // Preserve the setup failure as the public error while retaining the + // created handle for the service disposer to retry this rollback. + this.failedSetupSandbox = sandbox } } throw error diff --git a/packages/e2b/e2b/tests/e2b.spec.ts b/packages/e2b/e2b/tests/e2b.spec.ts index ff90c37b3a..fea24fa1cd 100644 --- a/packages/e2b/e2b/tests/e2b.spec.ts +++ b/packages/e2b/e2b/tests/e2b.spec.ts @@ -210,8 +210,28 @@ describe('E2BSandboxService', () => { fixture.kill.mockRejectedValueOnce(new Error('cleanup failed')) sdk.create.mockResolvedValue(fixture.sandbox) const ctx = new Context() - await ctx.plugin(E2BSandboxService, { apiKey: 'test-key' }) + const fiber = await ctx.plugin(E2BSandboxService, { apiKey: 'test-key' }) await expect(ctx.e2b.getSandbox()).rejects.toThrow('chmod failed') + expect(fixture.kill).toHaveBeenCalledOnce() + + await fiber.dispose() + expect(fixture.kill).toHaveBeenCalledTimes(2) + }) + + it.each([ + ['retries a still-failing rollback', new Error('retry failed')], + ['accepts a setup sandbox that expired before retry', new SandboxNotFoundError('sandbox expired')], + ])('%s during disposal', async (_label, retryError) => { + const fixture = fakeSandbox() + fixture.run.mockRejectedValueOnce(new Error('chmod failed')) + fixture.kill.mockRejectedValueOnce(new Error('cleanup failed')).mockRejectedValueOnce(retryError) + sdk.create.mockResolvedValue(fixture.sandbox) + const ctx = new Context() + const fiber = await ctx.plugin(E2BSandboxService, { apiKey: 'test-key' }) + + await expect(ctx.e2b.getSandbox()).rejects.toThrow('chmod failed') + await fiber.dispose() + expect(fixture.kill).toHaveBeenCalledTimes(2) }) it('does not kill a reconnected sandbox when setup fails', async () => { diff --git a/packages/e2b/subprocess-e2b/README.i18n.yaml b/packages/e2b/subprocess-e2b/README.i18n.yaml index 41d964a58a..0a9baf6d27 100644 --- a/packages/e2b/subprocess-e2b/README.i18n.yaml +++ b/packages/e2b/subprocess-e2b/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/e2b/subprocess-e2b/README.md -README.md: 42cd4951edb101dc75980e114e704c4aab50ae25 -README.zh.md: c4b318184cb7616a378e3336f4f4a7c366a98807 +README.md: 1c05509308e4f8b8b07cffd85e289b3ebee70318 +README.zh.md: b536d4e9d083076eccef4ca238dda1b2a503bee1 diff --git a/packages/e2b/subprocess-e2b/README.md b/packages/e2b/subprocess-e2b/README.md index 42cd4951ed..1c05509308 100644 --- a/packages/e2b/subprocess-e2b/README.md +++ b/packages/e2b/subprocess-e2b/README.md @@ -8,10 +8,10 @@ E2B implementation of the [`@deepseek-ai/dsh-subprocess`](../../subprocess/subpr - **Asynchronous remote start** — the synchronous seam returns a handle immediately while `Sandbox.commands.run(..., { background: true })` starts remotely. `pid` is `-1` until the wrapper publishes and the adapter validates its process-group id; stdin and ordinary observation wait for that publication, while cancellation can stop the provisional SDK handle first. - **Execution-world coordinates** — `cwd` and private `runtimeRoot` come from the shared owner; executable lookup verifies absolute paths or resolves a bare name against the sandbox PATH plus explicit overrides. -- **Linux process groups** — a quoted wrapper starts each argv under `exec setsid --wait` and records its actual process-group id plus private status files beneath `ctx.e2b.runtimeRoot/processes`. The handle waits for that file instead of treating the SDK command PID as its published identity. Termination signals the negative recorded id with `SIGTERM`, waits the caller's `graceMs`, then escalates to `SIGKILL` and the SDK kill fallback; TERM delivery or probe failures also force that escalation. A failed transaction is observable through `waitForExit()` and may be retried, while any proven quiescence permanently fences later termination against PID reuse. Before publication, cancellation uses the provisional SDK handle; if publication fails, rollback kills and verifies the provisional group before startup rejects. After publication, a monitoring failure also rolls back the group before rejecting. Service disposal rejects new starts, terminates and joins every retained process group, then awaits SDK settlement and private cleanup before the sandbox owner disposes. +- **Linux process groups** — a quoted wrapper starts each argv under `exec setsid --wait` and records its actual process-group id plus private status files beneath `ctx.e2b.runtimeRoot/processes`. The handle waits for that file instead of treating the SDK command PID as its published identity. Termination signals the negative recorded id with `SIGTERM`, waits the caller's `graceMs`, then escalates to `SIGKILL` and the SDK kill fallback; TERM delivery or probe failures also force that escalation. Process-table probes treat groups containing only zombie or dead entries as quiescent. A failed transaction is observable through `waitForExit()` and may be retried, while any proven quiescence permanently fences later termination against PID reuse. Before publication, cancellation uses the provisional SDK handle; if publication fails, rollback kills and verifies the provisional group before startup rejects. After publication, a monitoring failure also rolls back the group before rejecting. Service disposal rejects new starts, terminates and joins every retained process group, then awaits SDK settlement and private cleanup before the sandbox owner disposes. - **Environment boundary** — the wrapper starts from the sandbox command environment, removes ambient `DSH_*` and credential-shaped (`*KEY*`, `*SECRET*`, `*TOKEN*`) names, then restores every valid `spec.env` entry as an explicit caller opt-in; empty names, `=`, and NUL framing violations reject before launch. Host ambient variables never enter the sandbox implicitly. Private environment files are removed after consumption, and failed command or terminal setup removes its private state before rejecting. - **Stdio projection** — the remote wrapper branches raw bytes into optional bounded spill files, frames each live chunk as newline-delimited base64 ASCII, and the host incrementally restores bytes across arbitrary SDK callback boundaries. Pipe mode writes those bytes to host Node streams; inherit mode writes them to the harness process streams; collect mode retains a bounded host tail with offset reads. The wrapper publishes the direct command status before waiting for inherited writers. For collect or inherit output, the adapter disconnects an incomplete SDK stream after `graceMs`, withholds its partial spill, and returns that status while retaining the remote group for `waitForExit()` and termination; raw pipes instead await lossless transport completion and preserve backpressure. Batch and streaming stdin use the SDK handle. -- **Terminal sessions** — `spawnTerminal()` uses E2B's byte PTY API, installs the exact argv and scrubbed environment through private mode-`0600` files, reports the foreground process group, sends real signals, and tears down every group in the remote terminal session before settlement. A private random output boundary discards the E2B bootstrap shell's prompt and echoed runner command while preserving every requested-process byte, including its first prompt. Setup and teardown own the private state transaction, abort pending setup during service disposal, fence publication, and retain an unproven setup cleanup for disposal retry. Prompt detection, scrollback, readiness, and owner policy remain in `dsh-pty-local`. +- **Terminal sessions** — `spawnTerminal()` uses E2B's byte PTY API, installs the exact argv and scrubbed environment through private mode-`0600` files, reports the foreground process group, sends real signals, and tears down every live group in the remote terminal session before settlement; zombie-only groups are already quiescent. A private random output boundary discards the E2B bootstrap shell's prompt and echoed runner command while preserving every requested-process byte, including its first prompt. Setup and teardown own the private state transaction, abort pending setup during service disposal, fence publication, and retain an unproven setup cleanup for disposal retry. Prompt detection, scrollback, readiness, and owner policy remain in `dsh-pty-local`. The base E2B image supplies the runtime and Bash/GNU utilities this adapter invokes: `node`, `bash`, `setsid`, `ps`, `awk`, `tr`, `env`, `chmod`, `tee`, `head`, `rm`, and `kill`. A custom template must retain compatible commands and E2B PTY support. diff --git a/packages/e2b/subprocess-e2b/README.zh.md b/packages/e2b/subprocess-e2b/README.zh.md index c4b318184c..b536d4e9d0 100644 --- a/packages/e2b/subprocess-e2b/README.zh.md +++ b/packages/e2b/subprocess-e2b/README.zh.md @@ -8,10 +8,10 @@ - **异步远程启动**:同步 seam 会立即返回一个句柄,同时由 `Sandbox.commands.run(..., { background: true })` 在远程启动进程。包装层发布进程组 ID 并由适配器完成验证之前,`pid` 为 `-1`;stdin 和常规观察会等待该发布,而取消操作可以先停止临时 SDK 句柄。 - **执行世界坐标**:`cwd` 和私有 `runtimeRoot` 来自共享所有者;可执行文件查找会验证绝对路径,或根据沙箱 PATH 加显式覆盖来解析裸名称。 -- **Linux 进程组**:带引号保护的包装层会在 `exec setsid --wait` 下启动每组 argv,并在 `ctx.e2b.runtimeRoot/processes` 下记录实际进程组 ID 和私有状态文件。句柄会等待该文件,而不会把 SDK 命令 PID 当作已发布的身份。终止操作以记录的负数 ID 发送 `SIGTERM`,等待调用方的 `graceMs`,再升级到 `SIGKILL` 和 SDK kill 回退;TERM 信号发送或探测失败也会强制触发该升级。失败的事务可通过 `waitForExit()` 观察,并可重试;任何已证明的完全停稳都会永久防止后续终止操作命中复用的 PID。发布前,取消操作使用临时 SDK 句柄;如果发布失败,回滚会终止并验证临时进程组,随后启动操作才会拒绝。发布后,监控失败也会在拒绝前回滚进程组。服务 dispose(资源释放)会拒绝新的启动请求、终止并等待每个保留进程组退出,再等待 SDK 结算和私有清理完成,之后沙箱所有者才会释放。 +- **Linux 进程组**:带引号保护的包装层会在 `exec setsid --wait` 下启动每组 argv,并在 `ctx.e2b.runtimeRoot/processes` 下记录实际进程组 ID 和私有状态文件。句柄会等待该文件,而不会把 SDK 命令 PID 当作已发布的身份。终止操作以记录的负数 ID 发送 `SIGTERM`,等待调用方的 `graceMs`,再升级到 `SIGKILL` 和 SDK kill 回退;TERM 信号发送或探测失败也会强制触发该升级。进程表探测会把仅含僵尸或已死亡条目的进程组视为完全停稳。失败的事务可通过 `waitForExit()` 观察,并可重试;任何已证明的完全停稳都会永久防止后续终止操作命中复用的 PID。发布前,取消操作使用临时 SDK 句柄;如果发布失败,回滚会终止并验证临时进程组,随后启动操作才会拒绝。发布后,监控失败也会在拒绝前回滚进程组。服务 dispose(资源释放)会拒绝新的启动请求、终止并等待每个保留进程组退出,再等待 SDK 结算和私有清理完成,之后沙箱所有者才会释放。 - **环境边界**:包装层从沙箱命令环境开始,移除环境中的 `DSH_*` 和形似凭据的名称(`*KEY*`、`*SECRET*`、`*TOKEN*`),再把每个有效的 `spec.env` 条目恢复为调用方显式选择;空名称、`=` 和违反 NUL 分帧规则的条目会在启动前被拒绝。宿主环境变量绝不会隐式进入沙箱。私有环境文件在使用后会被删除;命令或终端设置失败时,会先删除其私有状态再拒绝。 - **stdio 投影**:远程包装层先把原始字节分流到可选的有界 spill 文件,再把每个实时分片编码为换行分隔的 base64 ASCII 帧;宿主会跨任意 SDK 回调边界增量恢复字节。pipe 模式把这些字节写入宿主 Node 流;inherit 模式把字节写入 harness 进程流;collect 模式保留有界的宿主尾部,并支持基于偏移量读取。包装层会在等待继承管道的写入方之前发布直接命令状态。对于 collect 或 inherit 输出,超过 `graceMs` 后,适配器会断开未完成的 SDK 流,不公开其中不完整的 spill,并返回该状态,同时保留远程进程组供 `waitForExit()` 和终止操作使用;原始 pipe 则会等待无损传输完成并保留背压。批量 stdin 和流式 stdin 都使用 SDK 句柄。 -- **终端会话**:`spawnTerminal()` 使用 E2B 的字节 PTY API,以 mode 为 `0600` 的私有文件传入原样 argv 与清理后的环境,报告前台进程组,发送真实信号,并在结算前清理远程终端会话中的每个进程组。私有随机输出边界会丢弃 E2B 引导 shell 的提示符和回显的 runner 命令,同时保留请求进程的每个字节,包括其第一个提示符。setup 与 teardown 负责私有状态事务,在服务 dispose 期间中止待处理的 setup、阻止发布,并保留未证明已完成的 setup 清理事务,供 dispose 重试。提示符检测、scrollback、就绪状态与所有者策略仍归 `dsh-pty-local` 所有。 +- **终端会话**:`spawnTerminal()` 使用 E2B 的字节 PTY API,以 mode 为 `0600` 的私有文件传入原样 argv 与清理后的环境,报告前台进程组,发送真实信号,并在结算前清理远程终端会话中仍存活的每个进程组;仅含僵尸进程的进程组已经完全停稳。私有随机输出边界会丢弃 E2B 引导 shell 的提示符和回显的 runner 命令,同时保留请求进程的每个字节,包括其第一个提示符。setup 与 teardown 负责私有状态事务,在服务 dispose 期间中止待处理的 setup、阻止发布,并保留未证明已完成的 setup 清理事务,供 dispose 重试。提示符检测、scrollback、就绪状态与所有者策略仍归 `dsh-pty-local` 所有。 基础 E2B 镜像提供该适配器调用的运行时和 Bash/GNU 工具:`node`、`bash`、`setsid`、`ps`、`awk`、`tr`、`env`、`chmod`、`tee`、`head`、`rm` 和 `kill`。自定义模板必须保留兼容的命令和 E2B PTY 支持。 diff --git a/packages/e2b/subprocess-e2b/src/process.ts b/packages/e2b/subprocess-e2b/src/process.ts index 4855b27b81..5387966578 100644 --- a/packages/e2b/subprocess-e2b/src/process.ts +++ b/packages/e2b/subprocess-e2b/src/process.ts @@ -474,12 +474,15 @@ export class E2BSubprocessHandle implements SubprocessHandle { if (target.write(data)) return await new Promise((resolve, reject) => { const onDrain = (): void => { cleanup(); resolve() } + const onClose = (): void => { cleanup(); resolve() } const onError = (error: Error): void => { cleanup(); reject(error) } const cleanup = (): void => { target.removeListener('drain', onDrain) + target.removeListener('close', onClose) target.removeListener('error', onError) } target.once('drain', onDrain) + target.once('close', onClose) target.once('error', onError) }) } @@ -671,14 +674,14 @@ export class E2BSubprocessHandle implements SubprocessHandle { } private async groupAlive(sandbox: Sandbox, pid: number, signal?: AbortSignal): Promise { - try { - await sandbox.commands.run(`kill -0 -- -${pid}`, signalOpts(signal)) - return true - } catch (error: unknown) { - if (signal?.aborted === true) return false - if (error instanceof CommandExitError) return false + const result = await sandbox.commands.run( + `set -o pipefail; ps -eo pgid=,stat= | awk '$1 == ${pid} && $2 !~ /^[ZXx]/ { live=1 } END { if (live) print "live" }'`, + signalOpts(signal), + ).catch((error: unknown) => { + if (signal?.aborted === true) return undefined throw error - } + }) + return result?.stdout.trim() === 'live' } private async finalizeSpills(sandbox: Sandbox): Promise { diff --git a/packages/e2b/subprocess-e2b/src/terminal.ts b/packages/e2b/subprocess-e2b/src/terminal.ts index 0f5809fa44..be79199b58 100644 --- a/packages/e2b/subprocess-e2b/src/terminal.ts +++ b/packages/e2b/subprocess-e2b/src/terminal.ts @@ -170,7 +170,7 @@ async function waitUntilReady( async function sessionProcessGroups(sandbox: Sandbox, sessionId: number): Promise { const result = await sandbox.commands.run( - `ps -eo sid=,pgid= | awk '$1 == ${sessionId} { print $2 }'`, + `set -o pipefail; ps -eo sid=,pgid=,stat= | awk '$1 == ${sessionId} && $3 !~ /^[ZXx]/ { print $2 }'`, ) const groups = new Set() for (const raw of result.stdout.trim().split(/\s+/)) { diff --git a/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts b/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts index 462e943a1b..b7d1642bd6 100644 --- a/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts +++ b/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts @@ -107,6 +107,7 @@ class FakeSandbox { delaysKillCompletion = false sdkKillStops = true alive = true + zombieOnly = false ambient = 'PATH=/ambient/bin\0KEEP=safe\0NPM_TOKEN=secret\0DSH_STALE=old\0BROKEN\0=bad\0' processGroupId = '4242\n' exitStatus = '' @@ -235,7 +236,7 @@ class FakeSandbox { if (this.envError !== undefined) throw this.envError return { exitCode: 0, stdout: this.ambient, stderr: '' } } - if (command.startsWith('kill -0 ')) { + if (command.startsWith('set -o pipefail; ps -eo pgid=,stat=')) { this.beforeProbe?.() if (options?.signal?.aborted === true) throw new DOMException('aborted', 'AbortError') if (this.probeError !== undefined) { @@ -243,9 +244,9 @@ class FakeSandbox { this.probeError = undefined throw error } - if (!this.alive) throw commandError(1) + const stdout = this.alive && !this.zombieOnly ? 'live\n' : '' this.afterProbe?.() - return { exitCode: 0, stdout: '', stderr: '' } + return { exitCode: 0, stdout, stderr: '' } } if (command.startsWith('kill -TERM ')) { await this.signalGate @@ -701,6 +702,20 @@ describe('E2BSubprocessHandle', () => { expect(fake.commandsSeen.filter(command => command.startsWith('kill -'))).toHaveLength(signals) }) + it('treats a zombie-only process group as quiescent', async () => { + const fake = new FakeSandbox() + fake.zombieOnly = true + const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/zombie-quiescence') + await flush() + + await expect(handle.waitForExit()).resolves.toBe(true) + expect(fake.commandsSeen).toContain( + 'set -o pipefail; ps -eo pgid=,stat= | awk \'$1 == 4242 && $2 !~ /^[ZXx]/ { live=1 } END { if (live) print "live" }\'', + ) + fake.finish() + await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) + }) + it('keeps proven quiescence after a concurrent termination transport fails', async () => { const fake = new FakeSandbox() fake.signalErrors.push(new Error('TERM transport failed'), new Error('KILL transport failed')) @@ -1187,6 +1202,23 @@ describe('E2BSubprocessHandle', () => { await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) }) + it('settles output backpressure when the consumer closes the pipe', async () => { + const fake = new FakeSandbox() + const handle = new E2BSubprocessHandle(runtime(fake), spec({ + stdio: { stdin: 'ignore', stdout: 'pipe', stderr: { maxBytes: 4 } }, + }), '/runtime/backpressure-close') + await flush() + + const stdoutWrite = vi.spyOn(handle.stdout!, 'write').mockReturnValueOnce(false) + const pending = fake.stdout('discarded') + queueMicrotask(() => { handle.stdout!.destroy() }) + await pending + stdoutWrite.mockRestore() + + fake.finish() + await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) + }) + it('contains a pipe callback failure instead of rejecting command settlement', async () => { const fake = new FakeSandbox() const handle = new E2BSubprocessHandle(runtime(fake), spec({ diff --git a/packages/e2b/subprocess-e2b/tests/terminal.spec.ts b/packages/e2b/subprocess-e2b/tests/terminal.spec.ts index e77646e586..b47cb8ffb1 100644 --- a/packages/e2b/subprocess-e2b/tests/terminal.spec.ts +++ b/packages/e2b/subprocess-e2b/tests/terminal.spec.ts @@ -95,6 +95,7 @@ class FakeTerminalSandbox { sessionId = '123\n' foreground = '456\n' groups = [123] + zombieGroups: number[] = [] createError: unknown writeError: unknown sendError: unknown @@ -160,9 +161,12 @@ class FakeTerminalSandbox { if (this.foregroundFailure !== undefined) throw this.foregroundFailure return { exitCode: 0, stdout: this.foreground, stderr: '' } } - if (command.startsWith('ps -eo sid=')) { + if (command.startsWith('set -o pipefail; ps -eo sid=')) { if (this.sessionGroupsFailure !== undefined) throw this.sessionGroupsFailure - return { exitCode: 0, stdout: this.groups.map(group => `${group}\n`).join(''), stderr: '' } + const groups = command.includes('stat=') && command.includes('$3 !~ /^[ZXx]/') + ? this.groups + : [...this.groups, ...this.zombieGroups] + return { exitCode: 0, stdout: groups.map(group => `${group}\n`).join(''), stderr: '' } } if (command.startsWith('kill -TERM -- ')) { if (this.termFailure !== undefined) throw this.termFailure @@ -508,6 +512,20 @@ describe('E2B terminal lifecycle', () => { await expect(quiescence).resolves.toBe(true) }) + it('treats a terminal session containing only zombies as quiescent', async () => { + const fake = new FakeTerminalSandbox() + fake.groups = [] + fake.zombieGroups = [123] + const terminal = await spawnE2BTerminal(runtime(fake), spec(), '/runtime/zombie-session') + + fake.handle.succeed(0) + await expect(terminal.done).resolves.toEqual({ exitCode: 0, signal: null }) + await expect(terminal.waitForExit()).resolves.toBe(true) + expect(fake.commands).toContain( + "set -o pipefail; ps -eo sid=,pgid=,stat= | awk '$1 == 123 && $3 !~ /^[ZXx]/ { print $2 }'", + ) + }) + it('rejects killing the terminal shell and propagates live foreground failures', async () => { const fake = new FakeTerminalSandbox() fake.foreground = '123\n' From 896bcd64a10d74f75a674e278bd816738a31062e Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 10:14:06 +0800 Subject: [PATCH 084/108] fix(e2b): await teardown and honor published status --- packages/e2b/subprocess-e2b/src/index.ts | 5 ++- packages/e2b/subprocess-e2b/src/process.ts | 11 +++-- .../subprocess-e2b/tests/subprocess.spec.ts | 43 +++++++++++++++++++ 3 files changed, 54 insertions(+), 5 deletions(-) diff --git a/packages/e2b/subprocess-e2b/src/index.ts b/packages/e2b/subprocess-e2b/src/index.ts index bf7edc41a1..4516bffa84 100644 --- a/packages/e2b/subprocess-e2b/src/index.ts +++ b/packages/e2b/subprocess-e2b/src/index.ts @@ -67,7 +67,10 @@ export class E2BSubprocessService extends SubprocessService { for (const cleanup of failedTerminalSetupCleanups) { pending.push(cleanup().then(() => { this.failedTerminalSetupCleanups.delete(cleanup) })) } - await Promise.all(pending) + const outcomes = await Promise.allSettled(pending) + for (const outcome of outcomes) { + if (outcome.status === 'rejected') throw outcome.reason + } }, 'e2b subprocess teardown') } diff --git a/packages/e2b/subprocess-e2b/src/process.ts b/packages/e2b/subprocess-e2b/src/process.ts index 5387966578..478dc96c30 100644 --- a/packages/e2b/subprocess-e2b/src/process.ts +++ b/packages/e2b/subprocess-e2b/src/process.ts @@ -524,10 +524,10 @@ export class E2BSubprocessHandle implements SubprocessHandle { throw new Error(`subprocess-e2b: remote wrapper published invalid exit code ${JSON.stringify(rawStatus)}`) } if (this.spec.stdio.stdout === 'pipe' || this.spec.stdio.stderr === 'pipe') { - return this.commandOutcome(await settlement) + return this.commandOutcome(await settlement, exitCode) } const completed = await withinMs(settlement, this.spec.graceMs) - if (completed !== undefined) return this.commandOutcome(completed) + if (completed !== undefined) return this.commandOutcome(completed, exitCode) this.outputDrainExpired = true this.stdoutReader?.invalidateSpill() this.stderrReader?.invalidateSpill() @@ -539,9 +539,12 @@ export class E2BSubprocessHandle implements SubprocessHandle { } } - private commandOutcome(settlement: CommandSettlement): SubprocessOutcome { - if (settlement.kind === 'result') return { exitCode: settlement.result.exitCode, signal: null } + private commandOutcome(settlement: CommandSettlement, publishedExitCode?: number): SubprocessOutcome { + if (settlement.kind === 'result') { + return { exitCode: publishedExitCode ?? settlement.result.exitCode, signal: null } + } if (settlement.error instanceof CommandExitError) { + if (publishedExitCode !== undefined) return { exitCode: publishedExitCode, signal: null } return this.terminationSignal === null ? { exitCode: settlement.error.exitCode, signal: null } : { exitCode: null, signal: this.terminationSignal } diff --git a/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts b/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts index b7d1642bd6..6d20abd3c1 100644 --- a/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts +++ b/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts @@ -547,6 +547,20 @@ describe('E2BSubprocessHandle', () => { await expect(handle.waitForExit()).resolves.toBe(true) }) + it('preserves a published nonzero exit code when termination settles the SDK inside the drain grace', async () => { + const fake = new FakeSandbox() + const handle = new E2BSubprocessHandle(runtime(fake), spec({ graceMs: 100 }), '/runtime/drain-signal-settled') + await flush() + fake.exitStatus = '7\n' + fake.afterStatusRead = () => { + fake.afterStatusRead = undefined + handle.terminate() + } + + await expect(handle.done).resolves.toEqual({ exitCode: 7, signal: null }) + await expect(handle.waitForExit()).resolves.toBe(true) + }) + it('rejects an invalid direct-command exit status', async () => { const fake = new FakeSandbox() const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/invalid-status') @@ -1336,6 +1350,35 @@ describe('E2BSubprocessService', () => { await expect(handle.done).resolves.toEqual({ exitCode: null, signal: 'SIGTERM' }) }) + it('waits for every owned cleanup before reporting a disposal failure', async () => { + const { ctx, fiber } = await service() + const failed = { + terminate: vi.fn(), + waitForExit: vi.fn(async () => { throw new Error('cleanup failed') }), + done: Promise.resolve({ exitCode: 0, signal: null }), + } as unknown as E2BSubprocessHandle + let finishCleanup!: () => void + const cleanup = new Promise((resolve) => { + finishCleanup = () => { resolve(true) } + }) + const draining = { + terminate: vi.fn(), + waitForExit: vi.fn(() => cleanup), + done: Promise.resolve({ exitCode: 0, signal: null }), + } as unknown as E2BSubprocessHandle + const live = (ctx.subprocess as unknown as { live: Set }).live + live.add(failed) + live.add(draining) + + let disposed = false + const disposing = fiber.dispose().then(() => { disposed = true }) + await flush() + expect(disposed).toBe(false) + finishCleanup() + await disposing + expect(live).toEqual(new Set([failed])) + }) + it('releases naturally settled handles before later service disposal', async () => { const fake = new FakeSandbox() const { ctx, fiber } = await service(fake) From cf4b721a8d7a3df38a1152295e098ce0ed3e20f4 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 10:27:56 +0800 Subject: [PATCH 085/108] test(e2b): snapshot provider-neutral overlay surface --- .../headless-agent/e2b.cordis.snapshot.yml | 58 +++++++++++++++++++ .../headless-agent/tests/headless.snapshot.ts | 56 ++++++++++++++++++ .../tests/snapshots/e2b-overlay/input.json | 8 +++ .../e2b-overlay/replay.override.json | 12 ++++ .../e2b-overlay/stream-json.expected.jsonl | 14 +++++ 5 files changed, 148 insertions(+) create mode 100644 examples/headless-agent/e2b.cordis.snapshot.yml create mode 100644 examples/headless-agent/tests/snapshots/e2b-overlay/input.json create mode 100644 examples/headless-agent/tests/snapshots/e2b-overlay/replay.override.json create mode 100644 examples/headless-agent/tests/snapshots/e2b-overlay/stream-json.expected.jsonl diff --git a/examples/headless-agent/e2b.cordis.snapshot.yml b/examples/headless-agent/e2b.cordis.snapshot.yml new file mode 100644 index 0000000000..3f39f0ffe4 --- /dev/null +++ b/examples/headless-agent/e2b.cordis.snapshot.yml @@ -0,0 +1,58 @@ +# Keyless counterpart to e2b.cordis.yml. It keeps the same generic PTY, LSP, +# and subprocess Code Runtime consumers, but supplies their fundamental +# filesystem/process world locally and replaces the live model with replay. +- id: base + name: '@cordisjs/plugin-include' + config: + path: ./cordis.yml + patches: + - id: llm-deepseek + name: '@deepseek-ai/dsh-llm-deepseek' + disabled: true + - id: cli-agent + name: '@deepseek-ai/dsh-cli-demo' + config: + provider: deepseek + model: deepseek-v4-pro + persistenceRoot: './.sessions' + persistenceCompression: none + workspaceContext: + maxBytes: 65536 + tools: + mode: both + persona: | + You are headless-agent, a coding assistant powered by the {{model}} model. Your working directory is {{cwd}}. + + Verify your work by running the code or tests. Keep answers brief and factual. + - insert: + - id: code-runtime-subprocess + name: '@deepseek-ai/dsh-code-runtime-subprocess' + - id: tool-cordis + name: '@deepseek-ai/dsh-tool-cordis' + - id: sandbox + name: '@deepseek-ai/dsh-sandbox-local' + - id: sandbox-policy + name: '@deepseek-ai/dsh-sandbox-policy' + config: + mode: danger-full-access + workspaceRoot: !!js process.cwd() + - id: pty + name: '@deepseek-ai/dsh-pty' + - id: pty-local + name: '@deepseek-ai/dsh-pty-local' + - id: tool-pty + name: '@deepseek-ai/dsh-tool-pty' + - id: lsp + name: '@deepseek-ai/dsh-lsp' + - id: lsp-local + name: '@deepseek-ai/dsh-lsp-local' + config: + servers: + typescript: + command: node + extensionToLanguage: + .ts: typescript + - id: tool-lsp + name: '@deepseek-ai/dsh-tool-lsp' + - id: llm-replay + name: '@deepseek-ai/dsh-llm-replay' diff --git a/examples/headless-agent/tests/headless.snapshot.ts b/examples/headless-agent/tests/headless.snapshot.ts index 87261439da..ca8e93724c 100644 --- a/examples/headless-agent/tests/headless.snapshot.ts +++ b/examples/headless-agent/tests/headless.snapshot.ts @@ -25,6 +25,8 @@ const advancedScenarioDir = join(snapshotsDir, 'advanced-toolchain') const advancedSessionFixture = join(advancedScenarioDir, 'session.jsonl') const advancedStreamExpected = join(advancedScenarioDir, 'stream-json.expected.jsonl') const advancedConfigPath = fileURLToPath(new URL('../advanced.cordis.snapshot.yml', import.meta.url)) +const e2bScenarioDir = join(snapshotsDir, 'e2b-overlay') +const e2bConfigPath = fileURLToPath(new URL('../e2b.cordis.snapshot.yml', import.meta.url)) const ptyScenarioDir = join(snapshotsDir, 'pty-tools') const ptySessionFixture = join(ptyScenarioDir, 'session.jsonl') const ptyStreamExpected = join(ptyScenarioDir, 'stream-json.expected.jsonl') @@ -608,6 +610,60 @@ describe('headless stream-json snapshots', () => { expect(normalized).toBe(await readFile(advancedStreamExpected, 'utf8')) }, LOADER_SMOKE_TEST_TIMEOUT_MS) + it('pins the E2B overlay provider-neutral tool surface', async () => { + const prompt = await scenarioPrompt(e2bScenarioDir, 'e2b-overlay') + const streamExpected = join(e2bScenarioDir, 'stream-json.expected.jsonl') + let runCwd = '' + const result = await runLoaderSmoke({ + label: 'E2B overlay headless stream-json snapshot', + tempDirPrefix: 'headless-snapshot-e2b-overlay-', + binScript, + configPath: e2bConfigPath, + binArgs: ['--config', e2bConfigPath, '--output-format', 'stream-json', prompt], + tsconfigPath, + env: { + DSH_SNAPSHOT: 'replay', + DSH_SNAPSHOT_FILE: join(e2bScenarioDir, 'session.jsonl'), + DSH_SNAPSHOT_OVERRIDE: join(e2bScenarioDir, 'replay.override.json'), + NODE_OPTIONS: [process.env.NODE_OPTIONS, '--disable-warning=ExperimentalWarning'].filter(Boolean).join(' '), + }, + prepare: (cwd) => { runCwd = cwd }, + inspect: async (cwd) => { + const logs = await persistedLogs(cwd) + expect(logs).toHaveLength(1) + const headers = parseJsonl(logs[0]?.content ?? '').filter(record => record.type === 'request/header') + expect(headers).toHaveLength(1) + const data = headers[0]?.data as JsonObject | undefined + const header = data?.header as JsonObject | undefined + if (!Array.isArray(header?.tools)) throw new Error('E2B overlay snapshot request has no tool schemas') + const toolNames = header.tools.map((tool, index) => { + if (tool === null || typeof tool !== 'object' || Array.isArray(tool)) { + throw new Error(`E2B overlay snapshot tool schema ${index} is not an object`) + } + const name = (tool as JsonObject).name + if (typeof name !== 'string') throw new Error(`E2B overlay snapshot tool schema ${index} has no name`) + return name + }) + expect(toolNames.filter(name => name === 'lsp' || name === 'run_code' || name.startsWith('terminal_')).sort()) + .toEqual([ + 'lsp', + 'run_code', + 'terminal_close', + 'terminal_list', + 'terminal_open', + 'terminal_read', + 'terminal_send', + 'terminal_signal', + ]) + }, + }) + + expect(result.stderr).toBe('') + const normalized = normalizeHeadlessStream(result.stdout, runCwd) + if (refreshing) await writeFile(streamExpected, normalized) + expect(normalized).toBe(await readFile(streamExpected, 'utf8')) + }, LOADER_SMOKE_TEST_TIMEOUT_MS) + it('replays persisted goal tools through the one-shot app', async () => { const prompt = await scenarioPrompt(goalScenarioDir, 'goal-tools') const streamExpected = join(goalScenarioDir, 'stream-json.expected.jsonl') diff --git a/examples/headless-agent/tests/snapshots/e2b-overlay/input.json b/examples/headless-agent/tests/snapshots/e2b-overlay/input.json new file mode 100644 index 0000000000..00fd27ce2e --- /dev/null +++ b/examples/headless-agent/tests/snapshots/e2b-overlay/input.json @@ -0,0 +1,8 @@ +{ + "steps": [ + { + "op": "prompt", + "text": "Report the assembled E2B overlay tool surface." + } + ] +} diff --git a/examples/headless-agent/tests/snapshots/e2b-overlay/replay.override.json b/examples/headless-agent/tests/snapshots/e2b-overlay/replay.override.json new file mode 100644 index 0000000000..d1a71ce58c --- /dev/null +++ b/examples/headless-agent/tests/snapshots/e2b-overlay/replay.override.json @@ -0,0 +1,12 @@ +[ + { + "kind": "chunks", + "chunks": [ + { "type": "block-start", "index": 0, "blockType": "text" }, + { "type": "text-delta", "index": 0, "text": "E2B_SURFACE_OK" }, + { "type": "block-end", "index": 0, "block": { "type": "text", "text": "E2B_SURFACE_OK" } }, + { "type": "usage", "usage": { "inputTokens": 8, "outputTokens": 3 } }, + { "type": "finish", "reason": { "kind": "stop" } } + ] + } +] diff --git a/examples/headless-agent/tests/snapshots/e2b-overlay/stream-json.expected.jsonl b/examples/headless-agent/tests/snapshots/e2b-overlay/stream-json.expected.jsonl new file mode 100644 index 0000000000..2fb74415cb --- /dev/null +++ b/examples/headless-agent/tests/snapshots/e2b-overlay/stream-json.expected.jsonl @@ -0,0 +1,14 @@ +{"type":"session_event","sessionId":"{{sessionId}}","event":{"type":"turn/start","seq":0,"time":0,"data":{"turn":1,"trigger":{"kind":"message","source":{"kind":"user"}}}}} +{"type":"session_event","sessionId":"{{sessionId}}","event":{"type":"user/message","seq":1,"time":0,"data":{"content":[{"type":"text","text":"Report the assembled E2B overlay tool surface."}],"source":{"kind":"user"},"role":"user","id":"{{sessionId}}"},"surfaceOp":"append"}} +{"type":"session_event","sessionId":"{{sessionId}}","event":{"type":"session/title","seq":2,"time":0,"data":{"title":"Report the assembled E2B overlay","messageSeqs":[1],"source":{"kind":"fallback"}}}} +{"type":"session_event","sessionId":"{{sessionId}}","event":{"type":"step/start","seq":3,"time":0,"data":{"turn":1,"step":1}}} +{"type":"session_event","sessionId":"{{sessionId}}","event":{"type":"request/header","seq":4,"time":0,"data":{"header":{"config":{"provider":"deepseek","model":"deepseek-v4-pro"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}}} +{"type":"session_event","sessionId":"{{sessionId}}","event":{"type":"assistant/chunk","seq":5,"time":0,"data":{"turn":1,"step":1,"chunk":{"type":"block-start","index":0,"blockType":"text"}}}} +{"type":"session_event","sessionId":"{{sessionId}}","event":{"type":"assistant/chunk","seq":6,"time":0,"data":{"turn":1,"step":1,"chunk":{"type":"text-delta","index":0,"text":"E2B_SURFACE_OK"}}}} +{"type":"session_event","sessionId":"{{sessionId}}","event":{"type":"assistant/chunk","seq":7,"time":0,"data":{"turn":1,"step":1,"chunk":{"type":"block-end","index":0,"block":{"type":"text","text":"E2B_SURFACE_OK"}}}}} +{"type":"session_event","sessionId":"{{sessionId}}","event":{"type":"assistant/chunk","seq":8,"time":0,"data":{"turn":1,"step":1,"chunk":{"type":"usage","usage":{"inputTokens":8,"outputTokens":3}}}}} +{"type":"session_event","sessionId":"{{sessionId}}","event":{"type":"assistant/chunk","seq":9,"time":0,"data":{"turn":1,"step":1,"chunk":{"type":"finish","reason":{"kind":"stop"}}}}} +{"type":"session_event","sessionId":"{{sessionId}}","event":{"type":"assistant/message","seq":10,"time":0,"data":{"turn":1,"step":1,"message":{"role":"assistant","content":[{"type":"text","text":"E2B_SURFACE_OK"}],"source":{"kind":"model","provider":"deepseek","model":"deepseek-v4-pro"},"id":"{{sessionId}}"},"usage":{"inputTokens":8,"outputTokens":3}},"sourceEventSeqs":[5,6,7,8,9],"surfaceOp":"append"}} +{"type":"session_event","sessionId":"{{sessionId}}","event":{"type":"step/end","seq":11,"time":0,"data":{"turn":1,"step":1}}} +{"type":"session_event","sessionId":"{{sessionId}}","event":{"type":"turn/end","seq":12,"time":0,"data":{"turn":1,"reason":{"kind":"completed"}}}} +{"type":"result","success":true,"sessionId":"{{sessionId}}","turn":1,"result":"E2B_SURFACE_OK","reason":{"kind":"completed"},"usage":{"inputTokens":8,"outputTokens":3}} From c122d984c4905d2d5d404fbe7686deded7737b07 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 11:23:01 +0800 Subject: [PATCH 086/108] fix(e2b): harden cancellation and teardown boundaries --- packages/e2b/e2b/README.i18n.yaml | 4 +- packages/e2b/e2b/README.md | 2 +- packages/e2b/e2b/README.zh.md | 2 +- packages/e2b/e2b/src/index.ts | 8 +- packages/e2b/e2b/tests/e2b.spec.ts | 48 ++++++- packages/e2b/fs-e2b/README.i18n.yaml | 4 +- packages/e2b/fs-e2b/README.md | 2 +- packages/e2b/fs-e2b/README.zh.md | 2 +- packages/e2b/fs-e2b/src/index.ts | 2 +- packages/e2b/fs-e2b/tests/filesystem.spec.ts | 1 + packages/e2b/subprocess-e2b/README.i18n.yaml | 4 +- packages/e2b/subprocess-e2b/README.md | 5 +- packages/e2b/subprocess-e2b/README.zh.md | 5 +- packages/e2b/subprocess-e2b/src/process.ts | 34 ++++- packages/e2b/subprocess-e2b/src/terminal.ts | 47 +++++-- .../subprocess-e2b/tests/subprocess.spec.ts | 98 +++++++++++++- .../e2b/subprocess-e2b/tests/terminal.spec.ts | 121 +++++++++++++++++- 17 files changed, 351 insertions(+), 38 deletions(-) diff --git a/packages/e2b/e2b/README.i18n.yaml b/packages/e2b/e2b/README.i18n.yaml index 46ff37b222..2fc8d403d1 100644 --- a/packages/e2b/e2b/README.i18n.yaml +++ b/packages/e2b/e2b/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/e2b/e2b/README.md -README.md: ccafa9df480e3812d3d3b6d25b513e7b5d2afc9f -README.zh.md: 60eefd46a0a9a017f4db57e4dc5313d533eeba04 +README.md: 6eb7d69dd6355de870db6bf05540f2629f464bc8 +README.zh.md: b2e2616cf779bce659fdef58840aa51f095a73cf diff --git a/packages/e2b/e2b/README.md b/packages/e2b/e2b/README.md index ccafa9df48..6eb7d69dd6 100644 --- a/packages/e2b/e2b/README.md +++ b/packages/e2b/e2b/README.md @@ -30,7 +30,7 @@ Set `sandboxId` to reconnect a running or paused sandbox instead of creating one Construction starts one create/connect operation. Before resolving `getSandbox()`, the service creates `cwd` and the private `cwd/.dsh-e2b` adapter-state directory, verifies that the reserved path is a real directory rather than a symlink or another file type, then sets it to mode `0700`. `sandboxId` resolves to a branded `E2BSandboxId` after setup. -Disposal first prevents new handle acquisition, then awaits setup and applies exactly one configured disposition. A `SandboxNotFoundError` means a kill-on-timeout sandbox is already quiescent; every other disposition failure rejects teardown. A newly created sandbox is killed when initial directory setup fails; if that rollback fails, disposal retries it before releasing ownership. A reconnected sandbox is not killed on setup failure because the service did not create it. Provider plugins must load after this owner and dispose before it. +Disposal first prevents new handle acquisition, then awaits setup and applies exactly one configured disposition. A `SandboxNotFoundError` is accepted when disposal requests `kill`, or when this service created a sandbox with `onTimeout: kill`; otherwise, a not-found error from a requested `pause` rejects teardown because retention was not proved. A newly created sandbox is killed when initial directory setup fails; if that rollback fails, disposal retries it before releasing ownership. A reconnected sandbox is not killed on setup failure because the service did not create it. Provider plugins must load after this owner and dispose before it. `pause` and `leave` retain remote filesystem and adapter artifacts for a later `sandboxId` connection, but a later harness process receives only a new SDK handle. The subprocess service still fulfills its seam contract by terminating managed groups before owner disposal; neither disposition recovers prior process objects, output cursors, or in-memory adapter locks. diff --git a/packages/e2b/e2b/README.zh.md b/packages/e2b/e2b/README.zh.md index 60eefd46a0..b2e2616cf7 100644 --- a/packages/e2b/e2b/README.zh.md +++ b/packages/e2b/e2b/README.zh.md @@ -30,7 +30,7 @@ 构造阶段会启动一次 create/connect 操作。服务在 `getSandbox()` 结算前创建 `cwd` 和私有的 `cwd/.dsh-e2b` 适配器状态目录,验证该预留路径是真实目录而非符号链接或其他文件类型,再把该目录的 mode 设为 `0700`。初始化完成后,`sandboxId` 会结算为品牌类型 `E2BSandboxId`。 -资源释放会先阻止继续获取新句柄,再等待初始化完成,并且只应用一种已配置的处置方式。`SandboxNotFoundError` 表示因超时终止的沙箱已经完全停稳;其他处置失败都会使 teardown 拒绝。新建沙箱的初始目录设置失败时,服务会终止该沙箱;如果该回滚失败,资源释放会在解除所有权前重试。重新连接的沙箱设置失败时不会被终止,因为它不是由本服务创建的。提供方插件必须在该所有者之后加载,并在其之前 dispose(资源释放)。 +资源释放会先阻止继续获取新句柄,再等待初始化完成,并且只应用一种已配置的处置方式。`SandboxNotFoundError` 仅在资源释放请求 `kill`,或本服务创建了配置为 `onTimeout: kill` 的沙箱时才可接受;否则,`pause` 请求返回的未找到错误会导致 teardown 拒绝,因为无法证明保留成功。新建沙箱的初始目录设置失败时,服务会终止该沙箱;如果该回滚失败,资源释放会在解除所有权前重试。重新连接的沙箱设置失败时不会被终止,因为它不是由本服务创建的。提供方插件必须在该所有者之后加载,并在其之前 dispose(资源释放)。 `pause` 和 `leave` 会保留远程文件系统及适配器产物,供稍后的 `sandboxId` 连接使用,但后续 harness 进程只会获得新的 SDK 句柄。进程管理服务仍会履行其 seam 契约,在所有者释放前终止受管进程组;这两种处置方式都不会恢复先前的进程对象、输出游标或内存中的适配器锁。 diff --git a/packages/e2b/e2b/src/index.ts b/packages/e2b/e2b/src/index.ts index 31e4f911a2..150961a897 100644 --- a/packages/e2b/e2b/src/index.ts +++ b/packages/e2b/e2b/src/index.ts @@ -181,9 +181,13 @@ export class E2BSandboxService extends Service { return } } catch (error: unknown) { - // A kill-on-timeout sandbox is already quiescent; every other disposal - // failure still reports that the configured final disposition is unknown. if (!(error instanceof SandboxNotFoundError)) throw error + // Missing proves the requested disposition only when this owner asked + // for deletion or created the sandbox with timeout deletion. A + // reconnected sandbox's creation lifecycle is unknown. + if (this.config.onDispose === 'kill') return + if (this.created && this.config.onTimeout === 'kill') return + throw error } }, 'e2b sandbox teardown') } diff --git a/packages/e2b/e2b/tests/e2b.spec.ts b/packages/e2b/e2b/tests/e2b.spec.ts index fea24fa1cd..a4eba5ebc9 100644 --- a/packages/e2b/e2b/tests/e2b.spec.ts +++ b/packages/e2b/e2b/tests/e2b.spec.ts @@ -160,15 +160,59 @@ describe('E2BSandboxService', () => { expect(fixture.pause).toHaveBeenCalledOnce() }) - it('does not classify other disposal failures as an already-gone sandbox', async () => { + it('accepts a missing sandbox when disposal itself requests deletion', async () => { const fixture = fakeSandbox() - fixture.kill.mockRejectedValue(new Error('disposition unknown')) + fixture.kill.mockRejectedValue(new SandboxNotFoundError('already deleted')) sdk.create.mockResolvedValue(fixture.sandbox) const ctx = new Context() + const errors: unknown[] = [] + ctx.logger.error = ((error: unknown) => { errors.push(error) }) as typeof ctx.logger.error + const fiber = await ctx.plugin(E2BSandboxService, { apiKey: 'test-key' }) + await ctx.e2b.getSandbox() + + await fiber.dispose() + expect(fixture.kill).toHaveBeenCalledOnce() + expect(errors).toEqual([]) + }) + + it('does not classify other disposal failures as an already-gone sandbox', async () => { + const fixture = fakeSandbox() + const failure = new Error('disposition unknown') + fixture.kill.mockRejectedValue(failure) + sdk.create.mockResolvedValue(fixture.sandbox) + const ctx = new Context() + const errors: unknown[] = [] + ctx.logger.error = ((error: unknown) => { errors.push(error) }) as typeof ctx.logger.error const fiber = await ctx.plugin(E2BSandboxService, { apiKey: 'test-key' }) await ctx.e2b.getSandbox() await expect(fiber.dispose()).resolves.toBeUndefined() expect(fixture.kill).toHaveBeenCalledOnce() + expect(errors).toContain(failure) + }) + + it.each([ + ['a created pause-on-timeout sandbox', false], + ['a reconnected sandbox with unknown creation policy', true], + ] as const)('reports missing during pause disposal for %s', async (_label, reconnect) => { + const fixture = fakeSandbox() + const failure = new SandboxNotFoundError('sandbox unexpectedly missing') + fixture.pause.mockRejectedValue(failure) + if (reconnect) sdk.connect.mockResolvedValue(fixture.sandbox) + else sdk.create.mockResolvedValue(fixture.sandbox) + const ctx = new Context() + const errors: unknown[] = [] + ctx.logger.error = ((error: unknown) => { errors.push(error) }) as typeof ctx.logger.error + const fiber = await ctx.plugin(E2BSandboxService, { + apiKey: 'test-key', + onTimeout: reconnect ? 'kill' : 'pause', + onDispose: 'pause', + ...(reconnect ? { sandboxId: 'existing' } : {}), + }) + await ctx.e2b.getSandbox() + + await expect(fiber.dispose()).resolves.toBeUndefined() + expect(fixture.pause).toHaveBeenCalledOnce() + expect(errors).toContain(failure) }) it('reconnects without applying creation lifecycle options and can leave state running', async () => { diff --git a/packages/e2b/fs-e2b/README.i18n.yaml b/packages/e2b/fs-e2b/README.i18n.yaml index 3c2b30ad15..87f1c7c69b 100644 --- a/packages/e2b/fs-e2b/README.i18n.yaml +++ b/packages/e2b/fs-e2b/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/e2b/fs-e2b/README.md -README.md: 8abf130ce16e2b79cda5fc858182159442f9e3c3 -README.zh.md: 943d9603046ad9b2a65f41d5801cac882040a0ff +README.md: bb92b5785383e9703a382fddefcd1cff9b2644cb +README.zh.md: 57ebfdbb92799660e74078fcd0affcc8d5bc120a diff --git a/packages/e2b/fs-e2b/README.md b/packages/e2b/fs-e2b/README.md index 8abf130ce1..bb92b57853 100644 --- a/packages/e2b/fs-e2b/README.md +++ b/packages/e2b/fs-e2b/README.md @@ -11,7 +11,7 @@ E2B implementation of the [`@deepseek-ai/dsh-fs`](../../fs/fs/README.md) provide - **UTF-8 reads** — whole reads and streamed reads preserve cross-chunk decoding, reject invalid UTF-8, and use the seam's 8192-byte NUL sample for binary detection. The model-facing tool still owns size selection and line windowing. - **Stable bounded reads** — a dependency-free Node helper walks directory descriptors with no-follow opens and reads one held regular-file descriptor through the byte cap. Generic LSP queries therefore reject parent swaps, non-files, invalid UTF-8, and growth past the configured document limit before server startup. - **Atomic mutations** — writes create a random sibling staging directory, change it to mode `0700` before uploading content, preserve an existing file's POSIX mode, and publish the staged file through E2B's same-filesystem atomic rename. The rename response supplies the committed version, so no fallible metadata request follows the commit point. E2B creates missing parent directories. Literal edits LF-normalize for matching, restore dominant CRLF storage, and serialize mutations per canonical target within the host process. Optional create/version guards keep the base seam's observed-state semantics. -- **Failures and cancellation** — E2B not-found, permission, abort, and other controller failures map to the existing `FsError` vocabulary. Cancellation is best-effort at SDK request boundaries; a successful rename is the commit point. +- **Failures and cancellation** — E2B not-found, permission, abort, and other controller failures map to the existing `FsError` vocabulary. Cancellation is best-effort at earlier SDK request boundaries and checked immediately before rename. The signal is not forwarded into the rename RPC, so cancellation cannot interrupt the atomic commit; a successful rename is the commit point. The provider does not copy, mount, or reconcile the host workspace. Giving it a host path as `cwd` creates a remote directory with the same spelling only. diff --git a/packages/e2b/fs-e2b/README.zh.md b/packages/e2b/fs-e2b/README.zh.md index 943d960304..57ebfdbb92 100644 --- a/packages/e2b/fs-e2b/README.zh.md +++ b/packages/e2b/fs-e2b/README.zh.md @@ -11,7 +11,7 @@ - **UTF-8 读取**:完整读取和流式读取会保留跨分片解码、拒绝无效 UTF-8,并使用 seam 的 8192 字节 NUL 样本检测二进制内容。面向模型的工具仍负责选择大小和行窗口。 - **稳定的有界读取**:一个零依赖 Node 辅助程序会以不跟随链接的方式逐级打开目录描述符,并通过一个持续持有的常规文件描述符读取至字节上限。因此,通用 LSP 查询会在服务器启动前拒绝父目录交换、非文件、无效 UTF-8,以及增长后超出所配置文档上限的文件。 - **原子变更**:写入会创建随机的同级暂存目录,在上传内容前将其 mode 改为 `0700`,保留现有文件的 POSIX mode,并通过 E2B 的同一文件系统原子重命名发布暂存文件。重命名响应会提供已提交的版本,因此提交点之后不会再进行可能失败的元数据请求。E2B 会创建缺失的父目录。字面量编辑匹配时会规范化为 LF,存储时恢复占主导的 CRLF,并在宿主进程内按规范化目标串行执行变更。可选的创建/版本防护会保留基础 seam 的已观察状态语义。 -- **失败与取消**:E2B 的未找到、权限、中止及其他控制器故障会映射到现有 `FsError` 词汇。取消在 SDK 请求边界上采用尽力而为语义;成功 rename 是提交点。 +- **失败与取消**:E2B 的未找到、权限、中止及其他控制器故障会映射到现有 `FsError` 词汇。取消在更早的 SDK 请求边界上采用尽力而为语义,并在 rename 前立即检查。信号不会传入 rename RPC,因此取消无法中断原子提交;成功 rename 是提交点。 该提供方不会复制、挂载或协调宿主工作区。把宿主路径用作 `cwd`,只会在远程创建一个拼写相同的目录。 diff --git a/packages/e2b/fs-e2b/src/index.ts b/packages/e2b/fs-e2b/src/index.ts index 0824c3f87c..d41c2e3d32 100644 --- a/packages/e2b/fs-e2b/src/index.ts +++ b/packages/e2b/fs-e2b/src/index.ts @@ -518,7 +518,7 @@ export class E2BFileSystem extends FileSystem { signalOpts(signal), ) assertNotAborted(signal, 'write') - const committed = await sandbox.files.rename(temporary, targetPath, signalOpts(signal)) + const committed = await sandbox.files.rename(temporary, targetPath) try { await sandbox.files.remove(stagingDirectory) } catch (_committedStagingCleanupFailure) { diff --git a/packages/e2b/fs-e2b/tests/filesystem.spec.ts b/packages/e2b/fs-e2b/tests/filesystem.spec.ts index 303f99c1e8..1689fb4023 100644 --- a/packages/e2b/fs-e2b/tests/filesystem.spec.ts +++ b/packages/e2b/fs-e2b/tests/filesystem.spec.ts @@ -209,6 +209,7 @@ class FakeRemote { this.nodes.set(to, node) this.renames.push({ from, to }) this.abortAfterRename?.abort('after commit') + this.checkAbort(options) return this.info(to) }, remove: async (path: string): Promise => { diff --git a/packages/e2b/subprocess-e2b/README.i18n.yaml b/packages/e2b/subprocess-e2b/README.i18n.yaml index 0a9baf6d27..8378cfa114 100644 --- a/packages/e2b/subprocess-e2b/README.i18n.yaml +++ b/packages/e2b/subprocess-e2b/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/e2b/subprocess-e2b/README.md -README.md: 1c05509308e4f8b8b07cffd85e289b3ebee70318 -README.zh.md: b536d4e9d083076eccef4ca238dda1b2a503bee1 +README.md: 66481fb1c5edd4c124c2c58aaa5c86a1683e5a3b +README.zh.md: 5e4b8bbe401cf4a565ccd53270df9fb6b5de9315 diff --git a/packages/e2b/subprocess-e2b/README.md b/packages/e2b/subprocess-e2b/README.md index 1c05509308..66481fb1c5 100644 --- a/packages/e2b/subprocess-e2b/README.md +++ b/packages/e2b/subprocess-e2b/README.md @@ -10,8 +10,9 @@ E2B implementation of the [`@deepseek-ai/dsh-subprocess`](../../subprocess/subpr - **Execution-world coordinates** — `cwd` and private `runtimeRoot` come from the shared owner; executable lookup verifies absolute paths or resolves a bare name against the sandbox PATH plus explicit overrides. - **Linux process groups** — a quoted wrapper starts each argv under `exec setsid --wait` and records its actual process-group id plus private status files beneath `ctx.e2b.runtimeRoot/processes`. The handle waits for that file instead of treating the SDK command PID as its published identity. Termination signals the negative recorded id with `SIGTERM`, waits the caller's `graceMs`, then escalates to `SIGKILL` and the SDK kill fallback; TERM delivery or probe failures also force that escalation. Process-table probes treat groups containing only zombie or dead entries as quiescent. A failed transaction is observable through `waitForExit()` and may be retried, while any proven quiescence permanently fences later termination against PID reuse. Before publication, cancellation uses the provisional SDK handle; if publication fails, rollback kills and verifies the provisional group before startup rejects. After publication, a monitoring failure also rolls back the group before rejecting. Service disposal rejects new starts, terminates and joins every retained process group, then awaits SDK settlement and private cleanup before the sandbox owner disposes. - **Environment boundary** — the wrapper starts from the sandbox command environment, removes ambient `DSH_*` and credential-shaped (`*KEY*`, `*SECRET*`, `*TOKEN*`) names, then restores every valid `spec.env` entry as an explicit caller opt-in; empty names, `=`, and NUL framing violations reject before launch. Host ambient variables never enter the sandbox implicitly. Private environment files are removed after consumption, and failed command or terminal setup removes its private state before rejecting. -- **Stdio projection** — the remote wrapper branches raw bytes into optional bounded spill files, frames each live chunk as newline-delimited base64 ASCII, and the host incrementally restores bytes across arbitrary SDK callback boundaries. Pipe mode writes those bytes to host Node streams; inherit mode writes them to the harness process streams; collect mode retains a bounded host tail with offset reads. The wrapper publishes the direct command status before waiting for inherited writers. For collect or inherit output, the adapter disconnects an incomplete SDK stream after `graceMs`, withholds its partial spill, and returns that status while retaining the remote group for `waitForExit()` and termination; raw pipes instead await lossless transport completion and preserve backpressure. Batch and streaming stdin use the SDK handle. -- **Terminal sessions** — `spawnTerminal()` uses E2B's byte PTY API, installs the exact argv and scrubbed environment through private mode-`0600` files, reports the foreground process group, sends real signals, and tears down every live group in the remote terminal session before settlement; zombie-only groups are already quiescent. A private random output boundary discards the E2B bootstrap shell's prompt and echoed runner command while preserving every requested-process byte, including its first prompt. Setup and teardown own the private state transaction, abort pending setup during service disposal, fence publication, and retain an unproven setup cleanup for disposal retry. Prompt detection, scrollback, readiness, and owner policy remain in `dsh-pty-local`. +- **Stdio projection** — the remote wrapper branches raw bytes into optional bounded spill files, frames each live chunk as newline-delimited base64 ASCII, and the host incrementally restores bytes across arbitrary SDK callback boundaries. Pipe mode writes those bytes to host Node streams; inherit mode writes them to the harness process streams; collect mode retains a bounded host tail with offset reads. The wrapper publishes the direct command status before waiting for inherited writers. For collect or inherit output, the adapter disconnects an incomplete SDK stream after `graceMs`, withholds its partial spill, and returns that status while retaining the remote group for `waitForExit()` and termination. Natural raw-pipe completion instead awaits lossless transport and preserves backpressure; explicit termination destroys the host pipes and releases blocked output before remote cleanup. Batch and streaming stdin use the SDK handle. +- **Terminal sessions** — `spawnTerminal()` uses E2B's byte PTY API, installs the exact argv and scrubbed environment through private mode-`0600` files, reports the foreground process group, sends real signals, and tears down every live group in the remote terminal session before settlement; zombie-only groups are already quiescent. A private random output boundary discards the E2B bootstrap shell's prompt and echoed runner command while preserving every requested-process byte, including its first prompt. PTY allocation is awaited through handle publication before cancellation is observed, so owned rollback can clean the published handle. Setup and teardown own the private state transaction, abort pending setup during service disposal, fence publication, and retain an unproven setup cleanup for disposal retry. Prompt detection, scrollback, readiness, and owner policy remain in `dsh-pty-local`. +- **Sandbox disappearance** — `SandboxNotFoundError` during process or terminal liveness, termination, rollback, or disconnect proves the remote execution world cannot retain work, so cleanup treats it as quiescent; unrelated failures remain observable. The base E2B image supplies the runtime and Bash/GNU utilities this adapter invokes: `node`, `bash`, `setsid`, `ps`, `awk`, `tr`, `env`, `chmod`, `tee`, `head`, `rm`, and `kill`. A custom template must retain compatible commands and E2B PTY support. diff --git a/packages/e2b/subprocess-e2b/README.zh.md b/packages/e2b/subprocess-e2b/README.zh.md index b536d4e9d0..5e4b8bbe40 100644 --- a/packages/e2b/subprocess-e2b/README.zh.md +++ b/packages/e2b/subprocess-e2b/README.zh.md @@ -10,8 +10,9 @@ - **执行世界坐标**:`cwd` 和私有 `runtimeRoot` 来自共享所有者;可执行文件查找会验证绝对路径,或根据沙箱 PATH 加显式覆盖来解析裸名称。 - **Linux 进程组**:带引号保护的包装层会在 `exec setsid --wait` 下启动每组 argv,并在 `ctx.e2b.runtimeRoot/processes` 下记录实际进程组 ID 和私有状态文件。句柄会等待该文件,而不会把 SDK 命令 PID 当作已发布的身份。终止操作以记录的负数 ID 发送 `SIGTERM`,等待调用方的 `graceMs`,再升级到 `SIGKILL` 和 SDK kill 回退;TERM 信号发送或探测失败也会强制触发该升级。进程表探测会把仅含僵尸或已死亡条目的进程组视为完全停稳。失败的事务可通过 `waitForExit()` 观察,并可重试;任何已证明的完全停稳都会永久防止后续终止操作命中复用的 PID。发布前,取消操作使用临时 SDK 句柄;如果发布失败,回滚会终止并验证临时进程组,随后启动操作才会拒绝。发布后,监控失败也会在拒绝前回滚进程组。服务 dispose(资源释放)会拒绝新的启动请求、终止并等待每个保留进程组退出,再等待 SDK 结算和私有清理完成,之后沙箱所有者才会释放。 - **环境边界**:包装层从沙箱命令环境开始,移除环境中的 `DSH_*` 和形似凭据的名称(`*KEY*`、`*SECRET*`、`*TOKEN*`),再把每个有效的 `spec.env` 条目恢复为调用方显式选择;空名称、`=` 和违反 NUL 分帧规则的条目会在启动前被拒绝。宿主环境变量绝不会隐式进入沙箱。私有环境文件在使用后会被删除;命令或终端设置失败时,会先删除其私有状态再拒绝。 -- **stdio 投影**:远程包装层先把原始字节分流到可选的有界 spill 文件,再把每个实时分片编码为换行分隔的 base64 ASCII 帧;宿主会跨任意 SDK 回调边界增量恢复字节。pipe 模式把这些字节写入宿主 Node 流;inherit 模式把字节写入 harness 进程流;collect 模式保留有界的宿主尾部,并支持基于偏移量读取。包装层会在等待继承管道的写入方之前发布直接命令状态。对于 collect 或 inherit 输出,超过 `graceMs` 后,适配器会断开未完成的 SDK 流,不公开其中不完整的 spill,并返回该状态,同时保留远程进程组供 `waitForExit()` 和终止操作使用;原始 pipe 则会等待无损传输完成并保留背压。批量 stdin 和流式 stdin 都使用 SDK 句柄。 -- **终端会话**:`spawnTerminal()` 使用 E2B 的字节 PTY API,以 mode 为 `0600` 的私有文件传入原样 argv 与清理后的环境,报告前台进程组,发送真实信号,并在结算前清理远程终端会话中仍存活的每个进程组;仅含僵尸进程的进程组已经完全停稳。私有随机输出边界会丢弃 E2B 引导 shell 的提示符和回显的 runner 命令,同时保留请求进程的每个字节,包括其第一个提示符。setup 与 teardown 负责私有状态事务,在服务 dispose 期间中止待处理的 setup、阻止发布,并保留未证明已完成的 setup 清理事务,供 dispose 重试。提示符检测、scrollback、就绪状态与所有者策略仍归 `dsh-pty-local` 所有。 +- **stdio 投影**:远程包装层先把原始字节分流到可选的有界 spill 文件,再把每个实时分片编码为换行分隔的 base64 ASCII 帧;宿主会跨任意 SDK 回调边界增量恢复字节。pipe 模式把这些字节写入宿主 Node 流;inherit 模式把字节写入 harness 进程流;collect 模式保留有界的宿主尾部,并支持基于偏移量读取。包装层会在等待继承管道的写入方之前发布直接命令状态。对于 collect 或 inherit 输出,超过 `graceMs` 后,适配器会断开未完成的 SDK 流,不公开其中不完整的 spill,并返回该状态,同时保留远程进程组供 `waitForExit()` 和终止操作使用。原始 pipe 自然完成时,会等待无损传输完成并保留背压;显式终止则会销毁宿主 pipe,并在远程清理前释放受阻的输出写入。批量 stdin 和流式 stdin 都使用 SDK 句柄。 +- **终端会话**:`spawnTerminal()` 使用 E2B 的字节 PTY API,以 mode 为 `0600` 的私有文件传入原样 argv 与清理后的环境,报告前台进程组,发送真实信号,并在结算前清理远程终端会话中仍存活的每个进程组;仅含僵尸进程的进程组已经完全停稳。私有随机输出边界会丢弃 E2B 引导 shell 的提示符和回显的 runner 命令,同时保留请求进程的每个字节,包括其第一个提示符。PTY 分配会一直等待到句柄发布后才观察取消,以便由承担清理责任的回滚清理已发布句柄。setup 与 teardown 负责私有状态事务,在服务 dispose 期间中止待处理的 setup、阻止发布,并保留未证明已完成的 setup 清理事务,供 dispose 重试。提示符检测、scrollback、就绪状态与所有者策略仍归 `dsh-pty-local` 所有。 +- **沙箱消失**:在进程或终端的存活探测、终止、回滚或断开连接期间出现 `SandboxNotFoundError`,证明远程执行环境无法保留工作,因此清理会将其视为完全停稳;其他故障仍可观察。 基础 E2B 镜像提供该适配器调用的运行时和 Bash/GNU 工具:`node`、`bash`、`setsid`、`ps`、`awk`、`tr`、`env`、`chmod`、`tee`、`head`、`rm` 和 `kill`。自定义模板必须保留兼容的命令和 E2B PTY 支持。 diff --git a/packages/e2b/subprocess-e2b/src/process.ts b/packages/e2b/subprocess-e2b/src/process.ts index 478dc96c30..c7d19e0ab8 100644 --- a/packages/e2b/subprocess-e2b/src/process.ts +++ b/packages/e2b/subprocess-e2b/src/process.ts @@ -6,6 +6,7 @@ import { posix } from 'node:path' import { CommandExitError, FileNotFoundError, + SandboxNotFoundError, quoteE2BShellArg, } from '@deepseek-ai/dsh-e2b' import type { CommandHandle, CommandResult, Sandbox } from '@deepseek-ai/dsh-e2b' @@ -115,7 +116,7 @@ function commandText(spec: SubprocessSpawnSpec, paths: RemotePaths): string { `printf '%s\\n' "$dsh_e2b_pgid" > ${quoteE2BShellArg(paths.pid)}`, `mapfile -d '' -t dsh_e2b_env < ${quoteE2BShellArg(paths.environment)}`, `"$dsh_e2b_rm" -f -- ${quoteE2BShellArg(paths.environment)}`, - `"$dsh_e2b_env_bin" -i "\${dsh_e2b_env[@]}" "$@" ${stdoutRedirect} ${stderrRedirect}`.trimEnd(), + `"$dsh_e2b_env_bin" -i -- "\${dsh_e2b_env[@]}" "$@" ${stdoutRedirect} ${stderrRedirect}`.trimEnd(), 'dsh_e2b_status=$?', `printf '%s\\n' "$dsh_e2b_status" > ${quoteE2BShellArg(paths.status)}`, 'wait', @@ -136,7 +137,7 @@ function commandText(spec: SubprocessSpawnSpec, paths: RemotePaths): string { 'for dsh_e2b_tool in "$dsh_e2b_env_bin" "$dsh_e2b_setsid" "$dsh_e2b_bash" "$dsh_e2b_node" "$dsh_e2b_ps" "$dsh_e2b_tr" "$dsh_e2b_tee" "$dsh_e2b_head" "$dsh_e2b_rm"; do', ' [[ "$dsh_e2b_tool" == /* && -x "$dsh_e2b_tool" ]] || exit 125', 'done', - `exec "$dsh_e2b_env_bin" -i "\${dsh_e2b_env[@]}" "$dsh_e2b_setsid" --wait -- "$dsh_e2b_bash" -c ${quoteE2BShellArg(inner)} dsh-e2b "$dsh_e2b_env_bin" "$dsh_e2b_node" "$dsh_e2b_ps" "$dsh_e2b_tr" "$dsh_e2b_tee" "$dsh_e2b_head" "$dsh_e2b_rm" ${argv}`, + `exec "$dsh_e2b_env_bin" -i -- "\${dsh_e2b_env[@]}" "$dsh_e2b_setsid" --wait -- "$dsh_e2b_bash" -c ${quoteE2BShellArg(inner)} dsh-e2b "$dsh_e2b_env_bin" "$dsh_e2b_node" "$dsh_e2b_ps" "$dsh_e2b_tr" "$dsh_e2b_tee" "$dsh_e2b_head" "$dsh_e2b_rm" ${argv}`, ].join('\n') return bootstrap } @@ -196,6 +197,7 @@ export class E2BSubprocessHandle implements SubprocessHandle { private readonly readyState = Promise.withResolvers() private readonly stdoutDecoder = new E2BBase64Decoder() private readonly stderrDecoder = new E2BBase64Decoder() + private readonly outputTermination = new AbortController() private readonly stdoutReader: E2BOutputReader | undefined private readonly stderrReader: E2BOutputReader | undefined private readonly paths: RemotePaths @@ -262,6 +264,9 @@ export class E2BSubprocessHandle implements SubprocessHandle { terminate(): void { if (this.terminationFenced || this.quiescenceProven || this.terminationAttempt !== undefined) return this.terminationStarted = true + this.outputTermination.abort() + this.stdout?.destroy() + this.stderr?.destroy() this.terminationFailure = undefined const attempt = this.terminateRemote() this.terminationAttempt = attempt @@ -317,6 +322,10 @@ export class E2BSubprocessHandle implements SubprocessHandle { sandbox = await this.runtime.getSandbox() } catch (error: unknown) { if (isAborted(signal)) return false + if (error instanceof SandboxNotFoundError) { + this.markQuiescent() + return true + } throw error } const processGroupId = this.remotePid > 0 ? this.remotePid : handle.pid @@ -469,21 +478,25 @@ export class E2BSubprocessHandle implements SubprocessHandle { private async writeOutput(pipe: PassThrough | undefined, inherited: NodeJS.WriteStream | undefined, data: Uint8Array): Promise { const target = pipe ?? inherited - if (target === undefined || data.length === 0) return + if (target === undefined || data.length === 0 || this.outputTermination.signal.aborted) return if (target.destroyed) throw new Error('subprocess output stream is closed') if (target.write(data)) return await new Promise((resolve, reject) => { const onDrain = (): void => { cleanup(); resolve() } const onClose = (): void => { cleanup(); resolve() } + const onTermination = (): void => { cleanup(); resolve() } const onError = (error: Error): void => { cleanup(); reject(error) } const cleanup = (): void => { target.removeListener('drain', onDrain) target.removeListener('close', onClose) target.removeListener('error', onError) + this.outputTermination.signal.removeEventListener('abort', onTermination) } target.once('drain', onDrain) target.once('close', onClose) target.once('error', onError) + this.outputTermination.signal.addEventListener('abort', onTermination, { once: true }) + if (this.outputTermination.signal.aborted) onTermination() }) } @@ -580,6 +593,18 @@ export class E2BSubprocessHandle implements SubprocessHandle { } private async terminateRemote(): Promise { + try { + await this.terminateRemoteInSandbox() + } catch (error: unknown) { + if (error instanceof SandboxNotFoundError) { + this.markQuiescent() + return + } + throw error + } + } + + private async terminateRemoteInSandbox(): Promise { const handle = await this.commandState.promise if (handle === undefined) return if (!isValidProcessId(handle.pid) && this.remotePid <= 0) { @@ -671,7 +696,7 @@ export class E2BSubprocessHandle implements SubprocessHandle { await sandbox.commands.run(`kill -${signal} -- -${pid}`) return true } catch (error: unknown) { - if (error instanceof CommandExitError) return false + if (error instanceof CommandExitError || error instanceof SandboxNotFoundError) return false throw error } } @@ -682,6 +707,7 @@ export class E2BSubprocessHandle implements SubprocessHandle { signalOpts(signal), ).catch((error: unknown) => { if (signal?.aborted === true) return undefined + if (error instanceof SandboxNotFoundError) return { exitCode: 0, stdout: '', stderr: '' } throw error }) return result?.stdout.trim() === 'live' diff --git a/packages/e2b/subprocess-e2b/src/terminal.ts b/packages/e2b/subprocess-e2b/src/terminal.ts index be79199b58..f918f802ba 100644 --- a/packages/e2b/subprocess-e2b/src/terminal.ts +++ b/packages/e2b/subprocess-e2b/src/terminal.ts @@ -7,6 +7,7 @@ import { posix } from 'node:path' import { CommandExitError, FileNotFoundError, + SandboxNotFoundError, quoteE2BShellArg, } from '@deepseek-ai/dsh-e2b' import type { CommandHandle, CommandResult, Sandbox } from '@deepseek-ai/dsh-e2b' @@ -37,7 +38,7 @@ const TERMINAL_RUNNER_SOURCE = [ 'fi', 'printf \'%s\' "$dsh_output_marker"', "printf 'ready\\n' > \"$dsh_state/ready\"", - 'exec env -i "${dsh_env[@]}" "${dsh_argv[@]}"', + 'exec env -i -- "${dsh_env[@]}" "${dsh_argv[@]}"', '', ].join('\n') @@ -169,9 +170,15 @@ async function waitUntilReady( } async function sessionProcessGroups(sandbox: Sandbox, sessionId: number): Promise { - const result = await sandbox.commands.run( - `set -o pipefail; ps -eo sid=,pgid=,stat= | awk '$1 == ${sessionId} && $3 !~ /^[ZXx]/ { print $2 }'`, - ) + let result: CommandResult + try { + result = await sandbox.commands.run( + `set -o pipefail; ps -eo sid=,pgid=,stat= | awk '$1 == ${sessionId} && $3 !~ /^[ZXx]/ { print $2 }'`, + ) + } catch (error: unknown) { + if (error instanceof SandboxNotFoundError) return [] + throw error + } const groups = new Set() for (const raw of result.stdout.trim().split(/\s+/)) { if (raw.length === 0) continue @@ -191,7 +198,7 @@ async function signalGroups(sandbox: Sandbox, groups: number[], signal: 'TERM' | try { await sandbox.commands.run(`kill -${signal} -- ${groups.map(group => `-${group}`).join(' ')}`) } catch (error: unknown) { - if (!(error instanceof CommandExitError)) throw error + if (!(error instanceof CommandExitError) && !(error instanceof SandboxNotFoundError)) throw error } } @@ -253,6 +260,7 @@ async function rollbackUnpublishedTerminal( try { await sandbox.pty.kill(handle.pid) } catch (error: unknown) { + if (error instanceof SandboxNotFoundError) return attemptFailures.push(asError(error)) } } @@ -262,6 +270,7 @@ async function rollbackUnpublishedTerminal( try { await handle.kill() } catch (error: unknown) { + if (error instanceof SandboxNotFoundError) return attemptFailures.push(asError(error)) } } @@ -291,7 +300,11 @@ async function rollbackUnpublishedTerminal( 'subprocess-e2b: terminal setup rollback did not reach quiescence', ) } - await handle.disconnect() + try { + await handle.disconnect() + } catch (error: unknown) { + if (!(error instanceof SandboxNotFoundError)) throw error + } } /** One E2B PTY and all process groups in its remote process session. */ @@ -400,7 +413,14 @@ export class E2BTerminalHandle implements SubprocessTerminalHandle { } if (groups.length > 0 || !this.topLevelExited) { this.terminationSignal = 'SIGKILL' - if (!this.topLevelExited) await this.sandbox.pty.kill(this.pid) + if (!this.topLevelExited) { + try { + await this.sandbox.pty.kill(this.pid) + } catch (error: unknown) { + if (error instanceof SandboxNotFoundError) return + throw error + } + } groups = await awaitSessionEmpty(this.sandbox, this.sessionId, this.graceMs, true) if (!this.topLevelExited) await Promise.race([this.done.catch(() => undefined), delay(this.graceMs)]) } @@ -410,7 +430,11 @@ export class E2BTerminalHandle implements SubprocessTerminalHandle { if (!this.topLevelExited) { throw new Error(`subprocess-e2b: terminal cleanup failed; surviving pid: ${this.pid}`) } - await this.handle.disconnect() + try { + await this.handle.disconnect() + } catch (error: unknown) { + if (!(error instanceof SandboxNotFoundError)) throw error + } await this.sandbox.files.remove(this.stateDir).catch(() => {}) } } @@ -468,11 +492,11 @@ export async function spawnE2BTerminal( cwd: spec.cwd, envs: { TERM: 'dumb' }, timeoutMs: 0, - ...signalOpts(spec.signal), onData: (data) => { outputFilter.push(data) }, }) completion = handle.wait() void completion.catch(() => {}) + spec.signal?.throwIfAborted() if (!Number.isSafeInteger(handle.pid) || handle.pid <= 0) { throw new Error(`subprocess-e2b: E2B returned invalid terminal pid ${handle.pid}`) } @@ -503,7 +527,8 @@ export async function spawnE2BTerminal( else await rollbackUnpublishedTerminal(sandbox, handle, completion, spec.graceMs) terminalQuiescent = true } catch (cleanupError: unknown) { - failures.push(asError(cleanupError)) + if (cleanupError instanceof SandboxNotFoundError) terminalQuiescent = true + else failures.push(asError(cleanupError)) } } if (!stateRemoved) { @@ -511,7 +536,7 @@ export async function spawnE2BTerminal( await sandbox.files.remove(stateDir) stateRemoved = true } catch (stateError: unknown) { - if (stateError instanceof FileNotFoundError) stateRemoved = true + if (stateError instanceof FileNotFoundError || stateError instanceof SandboxNotFoundError) stateRemoved = true else failures.push(asError(stateError)) } } diff --git a/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts b/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts index 6d20abd3c1..680d309bbb 100644 --- a/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts +++ b/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts @@ -3,6 +3,7 @@ import { Context } from 'cordis' import { CommandExitError, FileNotFoundError, + SandboxNotFoundError, type CommandHandle, type CommandResult, type Sandbox, @@ -371,7 +372,13 @@ describe('E2BSubprocessHandle', () => { const handle = new E2BSubprocessHandle(runtime(fake), spec({ argv: ['tool', 'argument with spaces'], stdio: { stdin: 'pipe', stdout: 'pipe', stderr: { maxBytes: 8, spill: { maxBytes: 32 } } }, - env: { PATH: '/bin', 'FOO-BAR': 'hyphen-value', DEEPSEEK_API_KEY: 'explicit-secret', DSH_MODE: 'test' }, + env: { + PATH: '/bin', + 'FOO-BAR': 'hyphen-value', + '--split-string': 'literal-value', + DEEPSEEK_API_KEY: 'explicit-secret', + DSH_MODE: 'test', + }, }), '/workspace/.dsh-e2b/processes/one') expect(handle.pid).toBe(-1) handle.stdin!.write('hello') @@ -394,7 +401,8 @@ describe('E2BSubprocessHandle', () => { expect(command).toContain('mapfile -d') expect(command).toContain('dsh_e2b_node="$(command -v node)"') expect(command).toContain('"$dsh_e2b_env_bin" -i "$dsh_e2b_node" -e') - expect(command).toContain('exec "$dsh_e2b_env_bin" -i "${dsh_e2b_env[@]}"') + expect(command).toContain('"$dsh_e2b_env_bin" -i -- "${dsh_e2b_env[@]}" "$@"') + expect(command).toContain('exec "$dsh_e2b_env_bin" -i -- "${dsh_e2b_env[@]}"') expect(command).toContain('>&2 2>/dev/null') expect(command).not.toContain('2>/dev/null >&2') expect(command).toContain('base64') @@ -405,7 +413,7 @@ describe('E2BSubprocessHandle', () => { '/workspace/.dsh-e2b/processes/one/stderr.log', ]) expect(fake.writtenFileData.get('/workspace/.dsh-e2b/processes/one/environment')).toBe( - 'PATH=/bin\0KEEP=safe\0FOO-BAR=hyphen-value\0DEEPSEEK_API_KEY=explicit-secret\0DSH_MODE=test\0', + 'PATH=/bin\0KEEP=safe\0FOO-BAR=hyphen-value\0--split-string=literal-value\0DEEPSEEK_API_KEY=explicit-secret\0DSH_MODE=test\0', ) let piped = '' @@ -1088,6 +1096,52 @@ describe('E2BSubprocessHandle', () => { await handle.done }) + it('treats a timeout-killed sandbox as quiescent during liveness probing', async () => { + const fake = new FakeSandbox() + const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/expired-sandbox') + await flush() + fake.finish() + await handle.done + fake.probeError = new SandboxNotFoundError('sandbox expired') + + await expect(handle.waitForExit()).resolves.toBe(true) + }) + + it('treats a missing sandbox handle as quiescent during liveness acquisition', async () => { + const fake = new FakeSandbox() + let calls = 0 + const handle = new E2BSubprocessHandle(runtime(fake, async () => { + calls += 1 + if (calls === 1) return fake.sandbox + throw new SandboxNotFoundError('sandbox expired') + }), spec(), '/runtime/expired-acquisition') + await flush() + + await expect(handle.waitForExit()).resolves.toBe(true) + await fake.completeOutput() + fake.alive = false + fake.handle.succeed(0) + await handle.done + }) + + it('treats sandbox loss during termination as quiescent', async () => { + const fake = new FakeSandbox() + let calls = 0 + const handle = new E2BSubprocessHandle(runtime(fake, async () => { + calls += 1 + if (calls === 1) return fake.sandbox + throw new SandboxNotFoundError('sandbox expired') + }), spec(), '/runtime/expired-termination') + await flush() + await fake.completeOutput() + + handle.terminate() + await expect(handle.waitForExit()).resolves.toBe(true) + fake.alive = false + fake.handle.succeed(0) + await handle.done + }) + it('makes batch stdin close failures best-effort', async () => { const fake = new FakeSandbox() vi.spyOn(fake.handle, 'sendStdin').mockRejectedValueOnce(new Error('closed')) @@ -1233,6 +1287,44 @@ describe('E2BSubprocessHandle', () => { await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) }) + it('breaks output backpressure when termination owns the command', async () => { + const fake = new FakeSandbox() + const handle = new E2BSubprocessHandle(runtime(fake), spec({ + stdio: { stdin: 'ignore', stdout: 'pipe', stderr: { maxBytes: 4 } }, + }), '/runtime/backpressure-termination') + await flush() + + const stdoutWrite = vi.spyOn(handle.stdout!, 'write').mockReturnValueOnce(false) + let released = false + const pending = fake.stdout('blocked').then(() => { released = true }) + await Promise.resolve() + handle.terminate() + await flush() + const releasedByTermination = released + if (!released) handle.stdout!.emit('drain') + await pending + stdoutWrite.mockRestore() + await handle.done + + expect(releasedByTermination).toBe(true) + }) + + it('settles backpressure when a synchronous pipe write starts termination', async () => { + const fake = new FakeSandbox() + const handle = new E2BSubprocessHandle(runtime(fake), spec({ + stdio: { stdin: 'ignore', stdout: 'pipe', stderr: { maxBytes: 4 } }, + }), '/runtime/backpressure-synchronous-termination') + await flush() + + const stdoutWrite = vi.spyOn(handle.stdout!, 'write').mockImplementationOnce(() => { + handle.terminate() + return false + }) + await expect(fake.stdout('blocked')).resolves.toBeUndefined() + stdoutWrite.mockRestore() + await handle.done + }) + it('contains a pipe callback failure instead of rejecting command settlement', async () => { const fake = new FakeSandbox() const handle = new E2BSubprocessHandle(runtime(fake), spec({ diff --git a/packages/e2b/subprocess-e2b/tests/terminal.spec.ts b/packages/e2b/subprocess-e2b/tests/terminal.spec.ts index b47cb8ffb1..c6ec5bfaad 100644 --- a/packages/e2b/subprocess-e2b/tests/terminal.spec.ts +++ b/packages/e2b/subprocess-e2b/tests/terminal.spec.ts @@ -6,6 +6,7 @@ import { describe, expect, it, vi } from 'vitest' import { CommandExitError, FileNotFoundError, + SandboxNotFoundError, type CommandHandle, type CommandResult, type Sandbox, @@ -113,6 +114,18 @@ class FakeTerminalSandbox { requestedOutput = 'requested-shell$ ' emitOutputMarker = true afterSessionLookup: (() => void) | undefined + private createGate: Promise | undefined + private releaseCreateGate: (() => void) | undefined + + deferCreate(): void { + const gate = Promise.withResolvers() + this.createGate = gate.promise + this.releaseCreateGate = () => { gate.resolve(undefined) } + } + + releaseCreate(): void { + this.releaseCreateGate?.() + } readonly sandbox = { files: { @@ -183,6 +196,8 @@ class FakeTerminalSandbox { create: async (options: Parameters[0]): Promise => { this.createOptions = options if (this.createError !== undefined) throw this.createError + await this.createGate + options.signal?.throwIfAborted() await options.onData(Buffer.from('buffered banner\n')) return this.handle.asHandle() }, @@ -258,7 +273,7 @@ describe('E2B terminal allocation', () => { const runner = fake.writes.get('/runtime/terminal-one/runner.bash') ?? '' expect(runner).toContain('if (( ${#dsh_argv[@]} == 0 )); then') expect(runner).toContain('printf \'%s\' "$dsh_output_marker"') - expect(runner).toContain('exec env -i "${dsh_env[@]}" "${dsh_argv[@]}"') + expect(runner).toContain('exec env -i -- "${dsh_env[@]}" "${dsh_argv[@]}"') expect(runner).not.toContain('\u007f') terminal.output.destroy() await fake.createOptions?.onData(Buffer.from('late bootstrap callback')) @@ -296,6 +311,25 @@ describe('E2B terminal allocation', () => { await expect(terminal.waitForExit()).resolves.toBe(true) }) + it('publishes the PTY handle before honoring allocation cancellation', async () => { + const fake = new FakeTerminalSandbox() + fake.deferCreate() + const controller = new AbortController() + const spawning = spawnE2BTerminal( + runtime(fake), + spec({ signal: controller.signal }), + '/runtime/allocation-cancel', + ) + await vi.waitFor(() => { expect(fake.createOptions).toBeDefined() }) + + controller.abort(new Error('allocation cancelled')) + fake.releaseCreate() + await expect(spawning).rejects.toThrow('allocation cancelled') + expect(fake.createOptions?.signal).toBeUndefined() + expect(fake.groups).toEqual([]) + expect(fake.handle.disconnects).toBe(1) + }) + it('rejects malformed environment and argv values before PTY allocation', async () => { const invalidName = new FakeTerminalSandbox() await expect(spawnE2BTerminal(runtime(invalidName), spec({ env: { 'BAD=NAME': 'x' } }), '/runtime/name')) @@ -410,6 +444,44 @@ describe('E2B terminal allocation', () => { cleanupFailed.removeError = new Error('remove transport failed') await expect(spawnE2BTerminal(runtime(cleanupFailed), spec(), '/runtime/cleanup-failed')) .rejects.toThrow('invalid terminal pid 0') + + const expiredDuringRollback = new FakeTerminalSandbox() + expiredDuringRollback.sendError = new Error('bootstrap failed before timeout') + expiredDuringRollback.groups = [] + expiredDuringRollback.settleOnPtyKill = false + expiredDuringRollback.ptyKillError = new SandboxNotFoundError('sandbox expired') + expiredDuringRollback.removeError = new SandboxNotFoundError('sandbox expired') + await expect(spawnE2BTerminal(runtime(expiredDuringRollback), spec(), '/runtime/expired-rollback')) + .rejects.toThrow('bootstrap failed before timeout') + expect(expiredDuringRollback.ptyKills).toBe(1) + + const expiredBeforeSdkRollback = new FakeTerminalSandbox() + expiredBeforeSdkRollback.handle.waitError = new Error('wait failed after timeout') + expiredBeforeSdkRollback.handle.sdkKillError = new SandboxNotFoundError('sandbox expired') + expiredBeforeSdkRollback.handle.settleOnSdkKill = false + await expect(spawnE2BTerminal(runtime(expiredBeforeSdkRollback), spec(), '/runtime/expired-sdk-rollback')) + .rejects.toThrow('wait failed after timeout') + + const expiredDuringSdkFallback = new FakeTerminalSandbox() + expiredDuringSdkFallback.sendError = new Error('bootstrap failed before SDK fallback') + expiredDuringSdkFallback.groups = [] + expiredDuringSdkFallback.settleOnPtyKill = false + expiredDuringSdkFallback.handle.sdkKillError = new SandboxNotFoundError('sandbox expired') + expiredDuringSdkFallback.handle.settleOnSdkKill = false + await expect(spawnE2BTerminal(runtime(expiredDuringSdkFallback), spec(), '/runtime/expired-sdk-fallback')) + .rejects.toThrow('bootstrap failed before SDK fallback') + + const missingDuringDisconnect = new FakeTerminalSandbox() + missingDuringDisconnect.sendError = new Error('bootstrap failed before disconnect') + missingDuringDisconnect.handle.disconnectError = new SandboxNotFoundError('sandbox expired') + await expect(spawnE2BTerminal(runtime(missingDuringDisconnect), spec(), '/runtime/missing-disconnect')) + .rejects.toThrow('bootstrap failed before disconnect') + + const failedDisconnect = new FakeTerminalSandbox() + failedDisconnect.sendError = new Error('bootstrap failed with disconnect failure') + failedDisconnect.handle.disconnectError = new Error('disconnect transport failed') + await expect(spawnE2BTerminal(runtime(failedDisconnect), spec(), '/runtime/failed-disconnect')) + .rejects.toThrow('bootstrap failed with disconnect failure') }) it('propagates setup cancellation and provider failures', async () => { @@ -526,6 +598,53 @@ describe('E2B terminal lifecycle', () => { ) }) + it('treats a timeout-killed sandbox as quiescent during terminal cleanup', async () => { + const fake = new FakeTerminalSandbox() + const terminal = await spawnE2BTerminal(runtime(fake), spec(), '/runtime/expired-sandbox') + fake.sessionGroupsFailure = new SandboxNotFoundError('sandbox expired') + fake.handle.succeed(0) + + await expect(terminal.done).resolves.toEqual({ exitCode: 0, signal: null }) + await expect(terminal.waitForExit()).resolves.toBe(true) + }) + + it('treats sandbox disappearance during PTY kill as quiescent', async () => { + const fake = new FakeTerminalSandbox() + fake.groups = [] + fake.settleOnPtyKill = false + fake.ptyKillError = new SandboxNotFoundError('sandbox expired') + const terminal = await spawnE2BTerminal(runtime(fake), spec({ graceMs: 1 }), '/runtime/expired-pty-kill') + + terminal.terminate() + await expect(terminal.waitForExit()).resolves.toBe(true) + expect(fake.ptyKills).toBe(1) + }) + + it('propagates a non-missing PTY kill failure', async () => { + const fake = new FakeTerminalSandbox() + fake.groups = [] + fake.settleOnPtyKill = false + fake.ptyKillError = new Error('PTY kill transport failed') + const terminal = await spawnE2BTerminal(runtime(fake), spec({ graceMs: 1 }), '/runtime/failed-pty-kill') + + terminal.terminate() + await expect(terminal.waitForExit()).rejects.toThrow('PTY kill transport failed') + }) + + it.each([ + ['accepts sandbox loss', new SandboxNotFoundError('sandbox expired'), true], + ['propagates another failure', new Error('disconnect failed'), false], + ] as const)('%s while disconnecting a settled terminal', async (_label, failure, accepted) => { + const fake = new FakeTerminalSandbox() + const terminal = await spawnE2BTerminal(runtime(fake), spec(), `/runtime/disconnect-${accepted}`) + fake.handle.disconnectError = failure + fake.groups = [] + fake.handle.succeed(0) + + if (accepted) await expect(terminal.waitForExit()).resolves.toBe(true) + else await expect(terminal.waitForExit()).rejects.toThrow('disconnect failed') + }) + it('rejects killing the terminal shell and propagates live foreground failures', async () => { const fake = new FakeTerminalSandbox() fake.foreground = '123\n' From ca79f4364c63c3f0d7a9ace8c127fc2f696e5fdf Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 12:15:18 +0800 Subject: [PATCH 087/108] test(e2b): prove stale interrupt isolation --- .../tests/fixtures/e2b/e2b/bin.ts | 30 +++++++++++++++++-- 1 file changed, 27 insertions(+), 3 deletions(-) diff --git a/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts b/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts index 6fa95c24b6..b4acfd1e5a 100644 --- a/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts +++ b/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts @@ -252,14 +252,38 @@ try { } const staleInterrupt = ctx.pty.startSend(owner, terminal.sessionId, { text: 'sleep 0.2', submit: true }) if (!staleInterrupt.cancel()) throw new Error('E2B PTY refused the stale-interrupt probe cancellation') - await staleInterrupt.done - const sleeping = ctx.pty.startSend(owner, terminal.sessionId, { text: 'sleep 30', submit: true }) + let canceledSendRetained = false + try { + ctx.pty.startSend(owner, terminal.sessionId, { text: 'sleep 30', submit: true }) + } catch (error: unknown) { + canceledSendRetained = String(error).includes('active send') + } foregroundLookup.resolve(undefined) + await staleInterrupt.done + remoteCommands.run = runRemoteCommand + if (!canceledSendRetained) throw new Error('E2B PTY released a canceled send before foreground signalling settled') + const sleeping = ctx.pty.startSend(owner, terminal.sessionId, { + text: "printf 'DSH_SLEEP_%s\\n' READY; sleep 30", + submit: true, + }) + let sleepReadyOutput = '' + const sleepReadyDeadline = Date.now() + 5_000 + while (!sleepReadyOutput.includes('DSH_SLEEP_READY\n')) { + sleepReadyOutput += sleeping.readOutput().delta + if (sleepReadyOutput.includes('DSH_SLEEP_READY\n')) break + const settled = await Promise.race([ + sleeping.done.then(result => ({ result })), + new Promise(resolveDelay => setTimeout(() => { resolveDelay(undefined) }, 25)), + ]) + if (settled !== undefined) { + throw new Error(`E2B PTY successor settled before executing: ${JSON.stringify(settled.result)}`) + } + if (Date.now() >= sleepReadyDeadline) throw new Error(`E2B PTY successor did not execute: ${sleepReadyOutput}`) + } const interruptIdentitySafe = await Promise.race([ sleeping.done.then(() => false), new Promise(resolveDelay => setTimeout(() => { resolveDelay(true) }, 300)), ]) - remoteCommands.run = runRemoteCommand if (!delayedForegroundLookup || !interruptIdentitySafe) { throw new Error('E2B PTY stale interrupt affected its successor send') } From 091af03a81319ccadde487a981933558de3c7c60 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 13:01:37 +0800 Subject: [PATCH 088/108] fix(e2b): harden remote startup and teardown --- ...026-07-27-e2b-remote-runtime-poc.i18n.yaml | 4 +- .../2026-07-27-e2b-remote-runtime-poc.md | 2 +- .../2026-07-27-e2b-remote-runtime-poc.zh.md | 2 +- packages/e2b/subprocess-e2b/README.i18n.yaml | 4 +- packages/e2b/subprocess-e2b/README.md | 8 +- packages/e2b/subprocess-e2b/README.zh.md | 8 +- .../e2b/subprocess-e2b/src/environment.ts | 24 ++++ packages/e2b/subprocess-e2b/src/process.ts | 42 +++++-- packages/e2b/subprocess-e2b/src/terminal.ts | 8 +- .../subprocess-e2b/tests/subprocess.spec.ts | 109 +++++++++++++++++- .../e2b/subprocess-e2b/tests/terminal.spec.ts | 9 +- 11 files changed, 182 insertions(+), 38 deletions(-) diff --git a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.i18n.yaml b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.i18n.yaml index 079924ac74..1343d33032 100644 --- a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.i18n.yaml +++ b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md -2026-07-27-e2b-remote-runtime-poc.md: 02de8df021094fa326431a996cd183d13b9c0c8c -2026-07-27-e2b-remote-runtime-poc.zh.md: 3b7ddf761b2cbf950898a0083cf9da653d20e723 +2026-07-27-e2b-remote-runtime-poc.md: cccee91998e832cc9cf53e554e104ccc5caada38 +2026-07-27-e2b-remote-runtime-poc.zh.md: b1a30c92e313b1395006ebd4fd17eacda2546ad5 diff --git a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md index 02de8df021..cccee91998 100644 --- a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md +++ b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md @@ -28,7 +28,7 @@ E2B owns the mutable filesystem, managed command and Bash processes, terminal al The host owns Cordis and plugin objects, the agent loop, agent/session/goal state, session logs and persistence, LLM calls, prompts and tools, authority decisions, skills, subagent orchestration, PTY buffers and readiness state, LSP JSON-RPC ids/queues/protocol state, Code Runtime type stripping/output accounting/binding dispatch, and E2B SDK/network orchestration. The overlay does not upload, mount, or synchronize the host workspace; identical cwd strings name independent host and remote directories. -The fundamental adapters carry the substrate-specific mechanics. `dsh-subprocess-e2b` consumes E2B's byte PTY callback directly, retains terminal send identity across asynchronous foreground-group lookup, and owns whole-session cleanup. For ordinary command pipes, its remote wrapper branches raw spill bytes before framing each live chunk as newline-delimited base64 ASCII; the host restores raw bytes across arbitrary decoded SDK callback boundaries. `dsh-fs-e2b` performs bounded source reads through a dependency-free helper that walks no-follow directory descriptors beneath the canonical target. Generic Code Runtime keeps its controller/worker protocol on validated ASCII/base64 frames and kills the provider-owned process group before inherited pipes drain. Generic LSP uses byte-faithful UTF-8 JSON over command pipes. +The fundamental adapters carry the substrate-specific mechanics. `dsh-subprocess-e2b` consumes E2B's byte PTY callback directly, retains terminal send identity across asynchronous foreground-group lookup, and owns whole-session cleanup. Ambient environments and ordinary command pipes cross decoded SDK callback boundaries as ASCII/base64 before strict UTF-8 or raw-byte restoration; startup preparation carries the handle's owned cancellation signal, and provisional termination proves the remote group empty after SDK fallback. `dsh-fs-e2b` performs bounded source reads through a dependency-free helper that walks no-follow directory descriptors beneath the canonical target. Generic Code Runtime keeps its controller/worker protocol on validated ASCII/base64 frames and kills the provider-owned process group before inherited pipes drain. Generic LSP uses byte-faithful UTF-8 JSON over command pipes. Retaining a sandbox preserves remote files and unmanaged remote state only. Reconnect does not reconstruct host PTY sessions, buffers, process handles, LSP connections or requests, code workers, binding calls, timers, output cursors, or locks. Managed groups terminate and join when their provider disposes before the shared owner pauses, leaves, or kills the sandbox. diff --git a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.zh.md b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.zh.md index 3b7ddf761b..b1a30c92e3 100644 --- a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.zh.md +++ b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.zh.md @@ -28,7 +28,7 @@ E2B 拥有可变文件系统、受管命令与 Bash 进程、终端分配与终 宿主拥有 Cordis 与插件对象、agent loop、agent/会话/goal 状态、会话日志及持久化、LLM(大语言模型)调用、提示词与工具、权限决策、skill(技能)、subagent 编排、PTY 缓冲与就绪状态、LSP JSON-RPC id/队列/协议状态、Code Runtime 类型剥离/输出计量/绑定分发,以及 E2B SDK/网络编排。该 overlay 不会上传、挂载或同步宿主工作区;拼写相同的 cwd 字符串分别指向彼此独立的宿主与远程目录。 -基础适配器承载基底专用机制。`dsh-subprocess-e2b` 直接消费 E2B 的字节 PTY 回调,在异步查找前台进程组的过程中保留终端发送身份,并负责全会话清理。对于普通命令管道,其远程包装层会先分流原始 spill 字节,再把每个实时分片编码为换行分隔的 base64 ASCII 帧;宿主会跨任意已解码 SDK 回调边界恢复原始字节。`dsh-fs-e2b` 通过无依赖辅助程序执行有界源码读取,该程序会在规范化目标下逐级遍历不跟随符号链接的目录描述符。通用 Code Runtime 通过经过验证的 ASCII/base64 帧承载 controller/worker 协议,并在继承的管道排空前终止提供方拥有的进程组。通用 LSP 通过命令管道使用字节保真的 UTF-8 JSON。 +基础适配器承载基底专用机制。`dsh-subprocess-e2b` 直接消费 E2B 的字节 PTY 回调,在异步查找前台进程组的过程中保留终端发送身份,并负责全会话清理。环境快照与普通命令管道会先以 ASCII/base64 跨越已解码的 SDK 回调边界,再执行严格 UTF-8 解码或恢复原始字节;启动准备会携带句柄自有的取消信号,临时终止则会在 SDK 回退后证明远程进程组为空。`dsh-fs-e2b` 通过无依赖辅助程序执行有界源码读取,该程序会在规范化目标下逐级遍历不跟随符号链接的目录描述符。通用 Code Runtime 通过经过验证的 ASCII/base64 帧承载 controller/worker 协议,并在继承的管道排空前终止提供方拥有的进程组。通用 LSP 通过命令管道使用字节保真的 UTF-8 JSON。 保留沙箱只会保存远程文件与未受管的远程状态。重新连接不会重建宿主 PTY 会话、缓冲、进程句柄、LSP 连接或请求、代码 worker、绑定调用、定时器、输出游标或锁。受管进程组会在所属提供方 dispose(资源释放)时终止并等待退出,之后共享所有者才会暂停、脱离或终止沙箱。 diff --git a/packages/e2b/subprocess-e2b/README.i18n.yaml b/packages/e2b/subprocess-e2b/README.i18n.yaml index 8378cfa114..3f64c67b7e 100644 --- a/packages/e2b/subprocess-e2b/README.i18n.yaml +++ b/packages/e2b/subprocess-e2b/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/e2b/subprocess-e2b/README.md -README.md: 66481fb1c5edd4c124c2c58aaa5c86a1683e5a3b -README.zh.md: 5e4b8bbe401cf4a565ccd53270df9fb6b5de9315 +README.md: aaac559e60162c3051cc6f06cbcfac011f5eb59a +README.zh.md: 904e4fd1d9a13082ddfc8183c0e42abc25bd1caf diff --git a/packages/e2b/subprocess-e2b/README.md b/packages/e2b/subprocess-e2b/README.md index 66481fb1c5..aaac559e60 100644 --- a/packages/e2b/subprocess-e2b/README.md +++ b/packages/e2b/subprocess-e2b/README.md @@ -6,15 +6,15 @@ E2B implementation of the [`@deepseek-ai/dsh-subprocess`](../../subprocess/subpr ## Behavior -- **Asynchronous remote start** — the synchronous seam returns a handle immediately while `Sandbox.commands.run(..., { background: true })` starts remotely. `pid` is `-1` until the wrapper publishes and the adapter validates its process-group id; stdin and ordinary observation wait for that publication, while cancellation can stop the provisional SDK handle first. +- **Asynchronous remote start** — the synchronous seam returns a handle immediately while `Sandbox.commands.run(..., { background: true })` starts remotely. `pid` is `-1` until the wrapper publishes and the adapter validates its process-group id; stdin and ordinary observation wait for that publication. An owned startup signal aborts environment and private-state preparation before allocation; once allocation begins, cancellation waits for a provisional SDK handle it can clean. - **Execution-world coordinates** — `cwd` and private `runtimeRoot` come from the shared owner; executable lookup verifies absolute paths or resolves a bare name against the sandbox PATH plus explicit overrides. -- **Linux process groups** — a quoted wrapper starts each argv under `exec setsid --wait` and records its actual process-group id plus private status files beneath `ctx.e2b.runtimeRoot/processes`. The handle waits for that file instead of treating the SDK command PID as its published identity. Termination signals the negative recorded id with `SIGTERM`, waits the caller's `graceMs`, then escalates to `SIGKILL` and the SDK kill fallback; TERM delivery or probe failures also force that escalation. Process-table probes treat groups containing only zombie or dead entries as quiescent. A failed transaction is observable through `waitForExit()` and may be retried, while any proven quiescence permanently fences later termination against PID reuse. Before publication, cancellation uses the provisional SDK handle; if publication fails, rollback kills and verifies the provisional group before startup rejects. After publication, a monitoring failure also rolls back the group before rejecting. Service disposal rejects new starts, terminates and joins every retained process group, then awaits SDK settlement and private cleanup before the sandbox owner disposes. -- **Environment boundary** — the wrapper starts from the sandbox command environment, removes ambient `DSH_*` and credential-shaped (`*KEY*`, `*SECRET*`, `*TOKEN*`) names, then restores every valid `spec.env` entry as an explicit caller opt-in; empty names, `=`, and NUL framing violations reject before launch. Host ambient variables never enter the sandbox implicitly. Private environment files are removed after consumption, and failed command or terminal setup removes its private state before rejecting. +- **Linux process groups** — a quoted wrapper starts each argv under `exec setsid --wait` and records its actual process-group id plus private status files beneath `ctx.e2b.runtimeRoot/processes`. The handle waits for that file instead of treating the SDK command PID as its published identity. Termination signals the negative recorded id with `SIGTERM`, waits the caller's `graceMs`, then escalates to `SIGKILL` and the SDK kill fallback; TERM delivery or probe failures also force that escalation. Process-table probes treat groups containing only zombie or dead entries as quiescent. A failed transaction is observable through `waitForExit()` and may be retried, while any proven quiescence permanently fences later termination against PID reuse. Before publication, cancellation uses both the provisional group and SDK handle and proves the group empty before reporting quiescence; if publication fails, rollback applies the same proof before startup rejects. After publication, a monitoring failure also rolls back the group before rejecting. Service disposal rejects new starts, terminates and joins every retained process group, then awaits SDK settlement and private cleanup before the sandbox owner disposes. +- **Environment boundary** — the sandbox command environment crosses the SDK callback boundary as base64 ASCII before one strict UTF-8 decode, then the wrapper removes ambient `DSH_*` and credential-shaped (`*KEY*`, `*SECRET*`, `*TOKEN*`) names and restores every valid `spec.env` entry as an explicit caller opt-in; empty names, `=`, and NUL framing violations reject before launch. Host ambient variables never enter the sandbox implicitly. Private environment files are removed after consumption, and failed command or terminal setup removes its private state before rejecting. - **Stdio projection** — the remote wrapper branches raw bytes into optional bounded spill files, frames each live chunk as newline-delimited base64 ASCII, and the host incrementally restores bytes across arbitrary SDK callback boundaries. Pipe mode writes those bytes to host Node streams; inherit mode writes them to the harness process streams; collect mode retains a bounded host tail with offset reads. The wrapper publishes the direct command status before waiting for inherited writers. For collect or inherit output, the adapter disconnects an incomplete SDK stream after `graceMs`, withholds its partial spill, and returns that status while retaining the remote group for `waitForExit()` and termination. Natural raw-pipe completion instead awaits lossless transport and preserves backpressure; explicit termination destroys the host pipes and releases blocked output before remote cleanup. Batch and streaming stdin use the SDK handle. - **Terminal sessions** — `spawnTerminal()` uses E2B's byte PTY API, installs the exact argv and scrubbed environment through private mode-`0600` files, reports the foreground process group, sends real signals, and tears down every live group in the remote terminal session before settlement; zombie-only groups are already quiescent. A private random output boundary discards the E2B bootstrap shell's prompt and echoed runner command while preserving every requested-process byte, including its first prompt. PTY allocation is awaited through handle publication before cancellation is observed, so owned rollback can clean the published handle. Setup and teardown own the private state transaction, abort pending setup during service disposal, fence publication, and retain an unproven setup cleanup for disposal retry. Prompt detection, scrollback, readiness, and owner policy remain in `dsh-pty-local`. - **Sandbox disappearance** — `SandboxNotFoundError` during process or terminal liveness, termination, rollback, or disconnect proves the remote execution world cannot retain work, so cleanup treats it as quiescent; unrelated failures remain observable. -The base E2B image supplies the runtime and Bash/GNU utilities this adapter invokes: `node`, `bash`, `setsid`, `ps`, `awk`, `tr`, `env`, `chmod`, `tee`, `head`, `rm`, and `kill`. A custom template must retain compatible commands and E2B PTY support. +The base E2B image supplies the runtime and Bash/GNU utilities this adapter invokes: `node`, `bash`, `setsid`, `ps`, `awk`, `tr`, `env`, `base64`, `chmod`, `tee`, `head`, `rm`, and `kill`. A custom template must retain compatible commands and E2B PTY support. ## Model Experience diff --git a/packages/e2b/subprocess-e2b/README.zh.md b/packages/e2b/subprocess-e2b/README.zh.md index 5e4b8bbe40..904e4fd1d9 100644 --- a/packages/e2b/subprocess-e2b/README.zh.md +++ b/packages/e2b/subprocess-e2b/README.zh.md @@ -6,15 +6,15 @@ ## 行为 -- **异步远程启动**:同步 seam 会立即返回一个句柄,同时由 `Sandbox.commands.run(..., { background: true })` 在远程启动进程。包装层发布进程组 ID 并由适配器完成验证之前,`pid` 为 `-1`;stdin 和常规观察会等待该发布,而取消操作可以先停止临时 SDK 句柄。 +- **异步远程启动**:同步 seam 会立即返回一个句柄,同时由 `Sandbox.commands.run(..., { background: true })` 在远程启动进程。包装层发布进程组 ID 并由适配器完成验证之前,`pid` 为 `-1`;stdin 和常规观察会等待该发布。自有启动信号会在分配前中止环境和私有状态准备;分配开始后,取消会等待可清理的临时 SDK 句柄。 - **执行世界坐标**:`cwd` 和私有 `runtimeRoot` 来自共享所有者;可执行文件查找会验证绝对路径,或根据沙箱 PATH 加显式覆盖来解析裸名称。 -- **Linux 进程组**:带引号保护的包装层会在 `exec setsid --wait` 下启动每组 argv,并在 `ctx.e2b.runtimeRoot/processes` 下记录实际进程组 ID 和私有状态文件。句柄会等待该文件,而不会把 SDK 命令 PID 当作已发布的身份。终止操作以记录的负数 ID 发送 `SIGTERM`,等待调用方的 `graceMs`,再升级到 `SIGKILL` 和 SDK kill 回退;TERM 信号发送或探测失败也会强制触发该升级。进程表探测会把仅含僵尸或已死亡条目的进程组视为完全停稳。失败的事务可通过 `waitForExit()` 观察,并可重试;任何已证明的完全停稳都会永久防止后续终止操作命中复用的 PID。发布前,取消操作使用临时 SDK 句柄;如果发布失败,回滚会终止并验证临时进程组,随后启动操作才会拒绝。发布后,监控失败也会在拒绝前回滚进程组。服务 dispose(资源释放)会拒绝新的启动请求、终止并等待每个保留进程组退出,再等待 SDK 结算和私有清理完成,之后沙箱所有者才会释放。 -- **环境边界**:包装层从沙箱命令环境开始,移除环境中的 `DSH_*` 和形似凭据的名称(`*KEY*`、`*SECRET*`、`*TOKEN*`),再把每个有效的 `spec.env` 条目恢复为调用方显式选择;空名称、`=` 和违反 NUL 分帧规则的条目会在启动前被拒绝。宿主环境变量绝不会隐式进入沙箱。私有环境文件在使用后会被删除;命令或终端设置失败时,会先删除其私有状态再拒绝。 +- **Linux 进程组**:带引号保护的包装层会在 `exec setsid --wait` 下启动每组 argv,并在 `ctx.e2b.runtimeRoot/processes` 下记录实际进程组 ID 和私有状态文件。句柄会等待该文件,而不会把 SDK 命令 PID 当作已发布的身份。终止操作以记录的负数 ID 发送 `SIGTERM`,等待调用方的 `graceMs`,再升级到 `SIGKILL` 和 SDK kill 回退;TERM 信号发送或探测失败也会强制触发该升级。进程表探测会把仅含僵尸或已死亡条目的进程组视为完全停稳。失败的事务可通过 `waitForExit()` 观察,并可重试;任何已证明的完全停稳都会永久防止后续终止操作命中复用的 PID。发布前,取消操作会同时使用临时进程组与 SDK 句柄,并在报告完全停稳前证明进程组为空;如果发布失败,回滚会在启动拒绝前执行相同证明。发布后,监控失败也会在拒绝前回滚进程组。服务 dispose(资源释放)会拒绝新的启动请求、终止并等待每个保留进程组退出,再等待 SDK 结算和私有清理完成,之后沙箱所有者才会释放。 +- **环境边界**:沙箱命令环境会先以 base64 ASCII 跨越 SDK 回调边界,再进行一次严格 UTF-8 解码;随后包装层移除环境中的 `DSH_*` 和形似凭据的名称(`*KEY*`、`*SECRET*`、`*TOKEN*`),并把每个有效的 `spec.env` 条目恢复为调用方显式选择。空名称、`=` 和违反 NUL 分帧规则的条目会在启动前被拒绝。宿主环境变量绝不会隐式进入沙箱。私有环境文件在使用后会被删除;命令或终端设置失败时,会先删除其私有状态再拒绝。 - **stdio 投影**:远程包装层先把原始字节分流到可选的有界 spill 文件,再把每个实时分片编码为换行分隔的 base64 ASCII 帧;宿主会跨任意 SDK 回调边界增量恢复字节。pipe 模式把这些字节写入宿主 Node 流;inherit 模式把字节写入 harness 进程流;collect 模式保留有界的宿主尾部,并支持基于偏移量读取。包装层会在等待继承管道的写入方之前发布直接命令状态。对于 collect 或 inherit 输出,超过 `graceMs` 后,适配器会断开未完成的 SDK 流,不公开其中不完整的 spill,并返回该状态,同时保留远程进程组供 `waitForExit()` 和终止操作使用。原始 pipe 自然完成时,会等待无损传输完成并保留背压;显式终止则会销毁宿主 pipe,并在远程清理前释放受阻的输出写入。批量 stdin 和流式 stdin 都使用 SDK 句柄。 - **终端会话**:`spawnTerminal()` 使用 E2B 的字节 PTY API,以 mode 为 `0600` 的私有文件传入原样 argv 与清理后的环境,报告前台进程组,发送真实信号,并在结算前清理远程终端会话中仍存活的每个进程组;仅含僵尸进程的进程组已经完全停稳。私有随机输出边界会丢弃 E2B 引导 shell 的提示符和回显的 runner 命令,同时保留请求进程的每个字节,包括其第一个提示符。PTY 分配会一直等待到句柄发布后才观察取消,以便由承担清理责任的回滚清理已发布句柄。setup 与 teardown 负责私有状态事务,在服务 dispose 期间中止待处理的 setup、阻止发布,并保留未证明已完成的 setup 清理事务,供 dispose 重试。提示符检测、scrollback、就绪状态与所有者策略仍归 `dsh-pty-local` 所有。 - **沙箱消失**:在进程或终端的存活探测、终止、回滚或断开连接期间出现 `SandboxNotFoundError`,证明远程执行环境无法保留工作,因此清理会将其视为完全停稳;其他故障仍可观察。 -基础 E2B 镜像提供该适配器调用的运行时和 Bash/GNU 工具:`node`、`bash`、`setsid`、`ps`、`awk`、`tr`、`env`、`chmod`、`tee`、`head`、`rm` 和 `kill`。自定义模板必须保留兼容的命令和 E2B PTY 支持。 +基础 E2B 镜像提供该适配器调用的运行时和 Bash/GNU 工具:`node`、`bash`、`setsid`、`ps`、`awk`、`tr`、`env`、`base64`、`chmod`、`tee`、`head`、`rm` 和 `kill`。自定义模板必须保留兼容的命令和 E2B PTY 支持。 ## 模型体验 diff --git a/packages/e2b/subprocess-e2b/src/environment.ts b/packages/e2b/subprocess-e2b/src/environment.ts index f9c30d378f..2b2516a706 100644 --- a/packages/e2b/subprocess-e2b/src/environment.ts +++ b/packages/e2b/subprocess-e2b/src/environment.ts @@ -1,7 +1,31 @@ /** Shared remote-environment scrubbing for E2B process and terminal launchers. */ +import { Buffer } from 'node:buffer' +import type { Sandbox } from '@deepseek-ai/dsh-e2b' import { SENSITIVE_ENV_PATTERN } from '@deepseek-ai/dsh-subprocess' +const BASE64 = /^(?:[A-Za-z0-9+/]{4})*(?:[A-Za-z0-9+/]{2}==|[A-Za-z0-9+/]{3}=)?$/ + +/** + * Read the remote environment through ASCII base64 so SDK callback chunking cannot corrupt UTF-8. + * @param sandbox - shared E2B execution world. + * @param signal - optional cancellation for the control-plane request. + * @returns the complete NUL-delimited UTF-8 environment. + */ +export async function readRemoteEnvironment(sandbox: Sandbox, signal?: AbortSignal): Promise { + const result = await sandbox.commands.run( + 'set -o pipefail; env -0 | base64 -w 0', + signal === undefined ? {} : { signal }, + ) + const encoded = result.stdout.trim() + if (!BASE64.test(encoded)) throw new Error('subprocess-e2b: remote environment transport returned invalid base64') + try { + return new TextDecoder('utf-8', { fatal: true }).decode(Buffer.from(encoded, 'base64')) + } catch (error: unknown) { + throw new Error('subprocess-e2b: remote environment is not valid UTF-8', { cause: error }) + } +} + /** * Parse an E2B NUL-delimited environment while removing harness-private and credential-shaped names. * @param raw - The complete NUL-delimited remote environment. diff --git a/packages/e2b/subprocess-e2b/src/process.ts b/packages/e2b/subprocess-e2b/src/process.ts index c7d19e0ab8..8c83c4b84b 100644 --- a/packages/e2b/subprocess-e2b/src/process.ts +++ b/packages/e2b/subprocess-e2b/src/process.ts @@ -18,7 +18,7 @@ import type { SubprocessSpawnSpec, } from '@deepseek-ai/dsh-subprocess' import type E2BSandboxService from '@deepseek-ai/dsh-e2b' -import { serializeRemoteEnvironment } from './environment.ts' +import { readRemoteEnvironment, serializeRemoteEnvironment } from './environment.ts' import { E2BBase64Decoder, E2B_OUTPUT_COMPLETE_FRAME, E2BOutputReader } from './output.ts' const GROUP_POLL_MS = 20 @@ -198,6 +198,7 @@ export class E2BSubprocessHandle implements SubprocessHandle { private readonly stdoutDecoder = new E2BBase64Decoder() private readonly stderrDecoder = new E2BBase64Decoder() private readonly outputTermination = new AbortController() + private readonly startupController = new AbortController() private readonly stdoutReader: E2BOutputReader | undefined private readonly stderrReader: E2BOutputReader | undefined private readonly paths: RemotePaths @@ -206,6 +207,7 @@ export class E2BSubprocessHandle implements SubprocessHandle { private outputTransportError: Error | undefined private outputDrainExpired = false private stateDirectoryCreated = false + private preparing = true private invalidHandleQuiescent = false private provisionalHandleQuiescent = false private terminationStarted = false @@ -264,6 +266,7 @@ export class E2BSubprocessHandle implements SubprocessHandle { terminate(): void { if (this.terminationFenced || this.quiescenceProven || this.terminationAttempt !== undefined) return this.terminationStarted = true + if (this.preparing) this.startupController.abort(new Error('subprocess-e2b: command terminated during startup')) this.outputTermination.abort() this.stdout?.destroy() this.stderr?.destroy() @@ -298,6 +301,7 @@ export class E2BSubprocessHandle implements SubprocessHandle { const attempt = this.terminationAttempt if (attempt !== undefined && await waitWithSignal(attempt, signal) === WAIT_ABORTED) return false this.throwTerminationFailure() + /* v8 ignore else -- successful provisional cleanup always records one proof; failures throw above. */ if (this.invalidHandleQuiescent || this.provisionalHandleQuiescent) { this.markQuiescent() return true @@ -351,6 +355,7 @@ export class E2BSubprocessHandle implements SubprocessHandle { try { sandbox = await this.runtime.getSandbox() await this.prepareState(sandbox) + this.preparing = false const handle = await sandbox.commands.run( commandText(this.spec, this.paths), { @@ -405,7 +410,9 @@ export class E2BSubprocessHandle implements SubprocessHandle { await this.finalizeSpills(sandbox) return outcome } catch (error: unknown) { - this.commandState.resolve(undefined) + const canceledPreparation = this.preparing + && this.terminationStarted + && this.startupController.signal.aborted let failure = await this.rollbackPublishedFailure(error) if (sandbox !== undefined && this.stateDirectoryCreated) { try { @@ -417,9 +424,12 @@ export class E2BSubprocessHandle implements SubprocessHandle { ) } } + this.commandState.resolve(undefined) this.readyState.reject(failure) + if (canceledPreparation && failure === error) return { exitCode: null, signal: 'SIGTERM' } throw failure } finally { + this.preparing = false this.spec.signal?.removeEventListener('abort', this.onAbort) this.stdout?.end() this.stderr?.end() @@ -427,19 +437,21 @@ export class E2BSubprocessHandle implements SubprocessHandle { } private async prepareState(sandbox: Sandbox): Promise { - const ambient = await sandbox.commands.run('env -0') - await sandbox.files.makeDir(this.stateDir) + const signal = this.startupController.signal + const ambient = await readRemoteEnvironment(sandbox, signal) + await sandbox.files.makeDir(this.stateDir, { signal }) this.stateDirectoryCreated = true - await sandbox.commands.run(`chmod 700 -- ${quoteE2BShellArg(this.stateDir)}`) + await sandbox.commands.run(`chmod 700 -- ${quoteE2BShellArg(this.stateDir)}`, { signal }) const files = [ { path: this.paths.pid, data: '' }, { path: this.paths.status, data: '' }, - { path: this.paths.environment, data: serializeRemoteEnvironment(ambient.stdout, this.spec.env) }, + { path: this.paths.environment, data: serializeRemoteEnvironment(ambient, this.spec.env) }, ...(hasSpill(this.spec.stdio.stdout) ? [{ path: this.paths.stdout, data: '' }] : []), ...(hasSpill(this.spec.stdio.stderr) ? [{ path: this.paths.stderr, data: '' }] : []), ] - await sandbox.files.write(files) - await sandbox.commands.run(`chmod 600 -- ${files.map(file => quoteE2BShellArg(file.path)).join(' ')}`) + await sandbox.files.write(files, { signal }) + await sandbox.commands.run(`chmod 600 -- ${files.map(file => quoteE2BShellArg(file.path)).join(' ')}`, { signal }) + signal.throwIfAborted() } private async writeBatchStdin(handle: CommandHandle): Promise { @@ -639,17 +651,23 @@ export class E2BSubprocessHandle implements SubprocessHandle { } let handleFailure: unknown try { - await handle.kill() - this.provisionalHandleQuiescent = true + if (!await handle.kill()) handleFailure = new Error('E2B SDK kill did not report command termination') } catch (error: unknown) { handleFailure = error } - if (!groupDelivered && handleFailure !== undefined) { + if (!groupDelivered && await this.groupAlive(sandbox, handle.pid)) { throw new AggregateError( - [...(groupFailure === undefined ? [] : [groupFailure]), handleFailure], + [ + ...(groupFailure === undefined ? [] : [groupFailure]), + ...(handleFailure === undefined + ? [new Error('E2B SDK kill did not quiesce the provisional process group')] + : [handleFailure]), + ], 'subprocess-e2b: force termination failed through both process-group and SDK transports', ) } + while (await this.groupAlive(sandbox, handle.pid)) await waitTick() + this.provisionalHandleQuiescent = true return } const sandbox = await this.runtime.getSandbox() diff --git a/packages/e2b/subprocess-e2b/src/terminal.ts b/packages/e2b/subprocess-e2b/src/terminal.ts index f918f802ba..9e0258c10d 100644 --- a/packages/e2b/subprocess-e2b/src/terminal.ts +++ b/packages/e2b/subprocess-e2b/src/terminal.ts @@ -20,7 +20,7 @@ import type { SubprocessTerminalSpawnSpec, } from '@deepseek-ai/dsh-subprocess' import type E2BSandboxService from '@deepseek-ai/dsh-e2b' -import { serializeRemoteEnvironment } from './environment.ts' +import { readRemoteEnvironment, serializeRemoteEnvironment } from './environment.ts' const POLL_MS = 20 @@ -355,7 +355,7 @@ export class E2BTerminalHandle implements SubprocessTerminalHandle { inputWaiting: false, } } catch (error: unknown) { - if (error instanceof CommandExitError && this.topLevelExited) return undefined + if (error instanceof CommandExitError && (error.exitCode === 1 || this.topLevelExited)) return undefined throw error } } @@ -470,8 +470,8 @@ export async function spawnE2BTerminal( let completion: Promise | undefined let stateDirectoryCreated = false try { - const ambient = await sandbox.commands.run('env -0', signalOpts(spec.signal)) - const environment = serializeRemoteEnvironment(ambient.stdout, spec.env) + const ambient = await readRemoteEnvironment(sandbox, spec.signal) + const environment = serializeRemoteEnvironment(ambient, spec.env) const argv = serializeValues(spec.argv, 'argv') await sandbox.files.makeDir(stateDir) stateDirectoryCreated = true diff --git a/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts b/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts index 680d309bbb..d3c96cf2fc 100644 --- a/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts +++ b/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts @@ -39,6 +39,7 @@ class FakeCommandHandle { kills = 0 disconnects = 0 killError: unknown + killResult = true disconnectError: unknown private readonly result = Promise.withResolvers() private settled = false @@ -61,7 +62,7 @@ class FakeCommandHandle { this.kills += 1 if (this.killError !== undefined) throw this.killError this.onKill() - return true + return this.killResult } async disconnect(): Promise { @@ -109,7 +110,9 @@ class FakeSandbox { sdkKillStops = true alive = true zombieOnly = false - ambient = 'PATH=/ambient/bin\0KEEP=safe\0NPM_TOKEN=secret\0DSH_STALE=old\0BROKEN\0=bad\0' + ambient = 'PATH=/ambient/bin\0KEEP=safe\0UNICODE=你好\0NPM_TOKEN=secret\0DSH_STALE=old\0BROKEN\0=bad\0' + environmentWire: string | undefined + environmentRequest: ((signal: AbortSignal | undefined) => Promise) | undefined processGroupId = '4242\n' exitStatus = '' readonly processGroupReads: string[] = [] @@ -233,9 +236,14 @@ class FakeSandbox { commands: { run: async (command: string, options?: StartOptions | { signal?: AbortSignal }): Promise => { this.commandsSeen.push(command) - if (command === 'env -0') { + if (command.includes('env -0 | base64')) { + await this.environmentRequest?.(options?.signal) if (this.envError !== undefined) throw this.envError - return { exitCode: 0, stdout: this.ambient, stderr: '' } + return { + exitCode: 0, + stdout: this.environmentWire ?? Buffer.from(this.ambient).toString('base64'), + stderr: '', + } } if (command.startsWith('set -o pipefail; ps -eo pgid=,stat=')) { this.beforeProbe?.() @@ -397,7 +405,7 @@ describe('E2BSubprocessHandle', () => { expect(command).not.toContain('explicit-secret') expect(command).not.toContain('hyphen-value') expect(command).not.toContain('${!dsh_e2b_name}') - expect(fake.commandsSeen).toContain('env -0') + expect(fake.commandsSeen).toContain('set -o pipefail; env -0 | base64 -w 0') expect(command).toContain('mapfile -d') expect(command).toContain('dsh_e2b_node="$(command -v node)"') expect(command).toContain('"$dsh_e2b_env_bin" -i "$dsh_e2b_node" -e') @@ -413,7 +421,7 @@ describe('E2BSubprocessHandle', () => { '/workspace/.dsh-e2b/processes/one/stderr.log', ]) expect(fake.writtenFileData.get('/workspace/.dsh-e2b/processes/one/environment')).toBe( - 'PATH=/bin\0KEEP=safe\0FOO-BAR=hyphen-value\0--split-string=literal-value\0DEEPSEEK_API_KEY=explicit-secret\0DSH_MODE=test\0', + 'PATH=/bin\0KEEP=safe\0UNICODE=你好\0FOO-BAR=hyphen-value\0--split-string=literal-value\0DEEPSEEK_API_KEY=explicit-secret\0DSH_MODE=test\0', ) let piped = '' @@ -806,6 +814,33 @@ describe('E2BSubprocessHandle', () => { await expect(handle.done).resolves.toEqual({ exitCode: null, signal: 'SIGTERM' }) }) + it('aborts a stalled preparation request before reporting startup quiescence', async () => { + const fake = new FakeSandbox() + let preparationSignal: AbortSignal | undefined + fake.environmentRequest = async (signal) => { + preparationSignal = signal + await new Promise((_resolve, reject) => { + const rejectAbort = (): void => { + const reason: unknown = signal?.reason + reject(reason instanceof Error ? reason : new Error(String(reason))) + } + if (signal?.aborted === true) { + rejectAbort() + return + } + signal?.addEventListener('abort', rejectAbort, { once: true }) + }) + } + const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/stalled-preparation') + await vi.waitFor(() => { expect(preparationSignal).toBeDefined() }) + + handle.terminate() + await expect(handle.done).resolves.toEqual({ exitCode: null, signal: 'SIGTERM' }) + await expect(handle.waitForExit()).resolves.toBe(true) + expect(preparationSignal?.aborted).toBe(true) + expect(fake.startOptions).toBeUndefined() + }) + it('kills through the provisional SDK handle before process-group publication', async () => { const fake = new FakeSandbox() fake.deferProcessGroupRead() @@ -822,6 +857,32 @@ describe('E2BSubprocessHandle', () => { await expect(handle.done).resolves.toEqual({ exitCode: null, signal: 'SIGKILL' }) }) + it('does not treat an unsuccessful SDK fallback as provisional group quiescence', async () => { + const fake = new FakeSandbox() + fake.deferProcessGroupRead() + fake.trapsTerm = true + fake.delaysKill = true + fake.delaysKillCompletion = true + fake.sdkKillStops = false + fake.handle.killResult = false + const handle = new E2BSubprocessHandle(runtime(fake), spec({ graceMs: 1 }), '/runtime/provisional-sdk-false') + await vi.waitFor(() => { expect(fake.startOptions).toBeDefined() }) + + handle.terminate() + await vi.waitFor(() => { expect(fake.handle.kills).toBe(1) }) + let quiescent = false + const waiting = handle.waitForExit().then((value) => { quiescent = value }) + await new Promise(resolve => setTimeout(resolve, 10)) + expect(quiescent).toBe(false) + + fake.alive = false + await waiting + expect(quiescent).toBe(true) + fake.releaseProcessGroupRead() + fake.finish() + await handle.done + }) + it('bounds a quiescence observer while provisional termination is awaiting the controller', async () => { const fake = new FakeSandbox() fake.deferProcessGroupRead() @@ -895,6 +956,23 @@ describe('E2BSubprocessHandle', () => { await expect(absentHandle.waitForExit()).resolves.toBe(true) absentGroup.releaseProcessGroupRead() await absentHandle.done + + const optimisticSdk = new FakeSandbox() + optimisticSdk.deferProcessGroupRead() + optimisticSdk.signalErrors.push(commandError(1), commandError(1)) + optimisticSdk.sdkKillStops = false + const optimisticHandle = new E2BSubprocessHandle( + runtime(optimisticSdk), + spec({ graceMs: 1 }), + '/runtime/pre-publication-optimistic-sdk', + ) + await vi.waitFor(() => { expect(optimisticSdk.startOptions).toBeDefined() }) + optimisticHandle.terminate() + await expect(optimisticHandle.waitForExit()).rejects.toThrow('force termination failed through both') + optimisticHandle.terminate() + await expect(optimisticHandle.waitForExit()).resolves.toBe(true) + optimisticSdk.releaseProcessGroupRead() + await optimisticHandle.done }) it('honors an already-aborted signal when constructing the asynchronous handle directly', async () => { @@ -1024,6 +1102,24 @@ describe('E2BSubprocessHandle', () => { await expect(envHandle.done).rejects.toThrow('ambient lookup failed') expect(envFailure.removed).toEqual([]) + const malformedEnvironment = new FakeSandbox() + malformedEnvironment.environmentWire = '%' + const malformedEnvironmentHandle = new E2BSubprocessHandle( + runtime(malformedEnvironment), + spec(), + '/runtime/malformed-environment', + ) + await expect(malformedEnvironmentHandle.done).rejects.toThrow('invalid base64') + + const invalidUtf8Environment = new FakeSandbox() + invalidUtf8Environment.environmentWire = Buffer.from([0xff]).toString('base64') + const invalidUtf8EnvironmentHandle = new E2BSubprocessHandle( + runtime(invalidUtf8Environment), + spec(), + '/runtime/invalid-utf8-environment', + ) + await expect(invalidUtf8EnvironmentHandle.done).rejects.toThrow('not valid UTF-8') + const cleanupFailure = new FakeSandbox() cleanupFailure.backgroundError = new Error('start failed before credential consumption') cleanupFailure.nextRemoveError = new Error('credential cleanup failed') @@ -1509,6 +1605,7 @@ describe('E2BSubprocessService', () => { const { ctx, fiber } = await service(fake) const subprocess = ctx.subprocess const handle = subprocess.spawn(spec()) + await vi.waitFor(() => { expect(fake.startOptions).toBeDefined() }) const disposing = fiber.dispose() await flush() expect(() => subprocess.spawn(spec())).toThrow('service is disposing') diff --git a/packages/e2b/subprocess-e2b/tests/terminal.spec.ts b/packages/e2b/subprocess-e2b/tests/terminal.spec.ts index c6ec5bfaad..00b9fc41fc 100644 --- a/packages/e2b/subprocess-e2b/tests/terminal.spec.ts +++ b/packages/e2b/subprocess-e2b/tests/terminal.spec.ts @@ -89,7 +89,7 @@ class FakeTerminalSandbox { readonly directories: string[] = [] readonly writes = new Map() createOptions: Parameters[0] | undefined - ambient = 'KEEP=visible\0NPM_TOKEN=secret\0DSH_STALE=old\0BROKEN\0=bad\0' + ambient = 'KEEP=visible\0UNICODE=你好\0NPM_TOKEN=secret\0DSH_STALE=old\0BROKEN\0=bad\0' ready: string | Error = 'ready\n' readyMisses = 0 readyReads = 0 @@ -162,7 +162,9 @@ class FakeTerminalSandbox { this.commandFailure = undefined throw error } - if (command === 'env -0') return { exitCode: 0, stdout: this.ambient, stderr: '' } + if (command.includes('env -0 | base64')) { + return { exitCode: 0, stdout: Buffer.from(this.ambient).toString('base64'), stderr: '' } + } if (command.includes('command -v -- ')) { return { exitCode: 0, stdout: this.resolvedExecutable, stderr: '' } } @@ -263,6 +265,7 @@ describe('E2B terminal allocation', () => { expect(fake.createOptions).toMatchObject({ rows: 24, cols: 80, cwd: '/workspace', timeoutMs: 0, envs: { TERM: 'dumb' } }) expect(fake.inputs[0]?.data.toString()).toContain("exec /bin/bash '/runtime/terminal-one/runner.bash'") expect(fake.writes.get('/runtime/terminal-one/environment')).toContain('KEEP=visible\0') + expect(fake.writes.get('/runtime/terminal-one/environment')).toContain('UNICODE=你好\0') expect(fake.writes.get('/runtime/terminal-one/environment')).toContain('TOKEN_EXPLICIT=kept\0') expect(fake.writes.get('/runtime/terminal-one/environment')).not.toContain('secret') expect(fake.writes.get('/runtime/terminal-one/environment')).not.toContain('DSH_STALE') @@ -653,6 +656,8 @@ describe('E2B terminal lifecycle', () => { fake.foreground = 'invalid\n' await expect(terminal.inspectForeground()).rejects.toThrow('cannot resolve foreground') fake.foregroundFailure = commandError(1) + await expect(terminal.inspectForeground()).resolves.toBeUndefined() + fake.foregroundFailure = commandError(2) await expect(terminal.inspectForeground()).rejects.toBeInstanceOf(CommandExitError) fake.clearOnTerm = true terminal.terminate() From 81e2e1f647b07b20ec7974176770fdd9a7764c64 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 15:30:32 +0800 Subject: [PATCH 089/108] fix(e2b): harden SDK shell and cleanup boundaries E2B starts command and PTY requests through login shells, so isolate each control shell behind a fresh randomized HOME and blank sandbox credential names before mutable profiles can run. Preserve the real remote HOME only for the requested argv. Collapse duplicate termination state, keep failed force cleanup retryable until quiescence is observed, and make terminal state allocation cancellable. Leave numeric PGID reuse as an explicit provider-level TODO because a userspace precheck would remain TOCTOU. --- ...026-07-27-e2b-remote-runtime-poc.i18n.yaml | 4 +- .../2026-07-27-e2b-remote-runtime-poc.md | 4 +- .../2026-07-27-e2b-remote-runtime-poc.zh.md | 4 +- docs/cordis-catalog/services.md | 2 +- packages/e2b/e2b/README.i18n.yaml | 4 +- packages/e2b/e2b/README.md | 2 +- packages/e2b/e2b/README.zh.md | 2 +- packages/e2b/e2b/src/index.ts | 17 +- packages/e2b/e2b/tests/composition.e2e.ts | 26 ++- packages/e2b/e2b/tests/e2b.spec.ts | 27 ++- packages/e2b/fs-e2b/src/index.ts | 15 +- packages/e2b/fs-e2b/tests/filesystem.spec.ts | 8 +- packages/e2b/subprocess-e2b/README.i18n.yaml | 4 +- packages/e2b/subprocess-e2b/README.md | 4 +- packages/e2b/subprocess-e2b/README.zh.md | 4 +- .../e2b/subprocess-e2b/src/environment.ts | 57 ++++-- packages/e2b/subprocess-e2b/src/index.ts | 6 +- packages/e2b/subprocess-e2b/src/process.ts | 182 ++++++++---------- packages/e2b/subprocess-e2b/src/terminal.ts | 99 +++++++--- .../subprocess-e2b/tests/subprocess.spec.ts | 127 ++++++++---- .../e2b/subprocess-e2b/tests/terminal.spec.ts | 51 ++++- 21 files changed, 438 insertions(+), 211 deletions(-) diff --git a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.i18n.yaml b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.i18n.yaml index 1343d33032..2c3db3fad4 100644 --- a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.i18n.yaml +++ b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md -2026-07-27-e2b-remote-runtime-poc.md: cccee91998e832cc9cf53e554e104ccc5caada38 -2026-07-27-e2b-remote-runtime-poc.zh.md: b1a30c92e313b1395006ebd4fd17eacda2546ad5 +2026-07-27-e2b-remote-runtime-poc.md: 79c5bfe2dfa11cd695c7134e6ceffb639b449688 +2026-07-27-e2b-remote-runtime-poc.zh.md: 729a10ec7efa4dd55f041dc804d2252102b13c01 diff --git a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md index cccee91998..79c5bfe2df 100644 --- a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md +++ b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md @@ -28,7 +28,7 @@ E2B owns the mutable filesystem, managed command and Bash processes, terminal al The host owns Cordis and plugin objects, the agent loop, agent/session/goal state, session logs and persistence, LLM calls, prompts and tools, authority decisions, skills, subagent orchestration, PTY buffers and readiness state, LSP JSON-RPC ids/queues/protocol state, Code Runtime type stripping/output accounting/binding dispatch, and E2B SDK/network orchestration. The overlay does not upload, mount, or synchronize the host workspace; identical cwd strings name independent host and remote directories. -The fundamental adapters carry the substrate-specific mechanics. `dsh-subprocess-e2b` consumes E2B's byte PTY callback directly, retains terminal send identity across asynchronous foreground-group lookup, and owns whole-session cleanup. Ambient environments and ordinary command pipes cross decoded SDK callback boundaries as ASCII/base64 before strict UTF-8 or raw-byte restoration; startup preparation carries the handle's owned cancellation signal, and provisional termination proves the remote group empty after SDK fallback. `dsh-fs-e2b` performs bounded source reads through a dependency-free helper that walks no-follow directory descriptors beneath the canonical target. Generic Code Runtime keeps its controller/worker protocol on validated ASCII/base64 frames and kills the provider-owned process group before inherited pipes drain. Generic LSP uses byte-faithful UTF-8 JSON over command pipes. +The fundamental adapters carry the substrate-specific mechanics. Adapter-internal E2B command and PTY login shells use a fresh randomized root-level `HOME`; `dsh-subprocess-e2b` also gives them empty overrides for scrubbed credential names before user profiles run. The subprocess adapter consumes E2B's byte PTY callback, transports environment snapshots and command bytes as ASCII/base64 across decoded SDK callbacks, and uses one cancellation controller plus one retryable group-cleanup transaction that reports success only after proving quiescence. `dsh-fs-e2b` performs bounded source reads through a dependency-free helper that walks no-follow directory descriptors beneath the canonical target. Generic Code Runtime keeps its controller/worker protocol on validated newline-delimited UTF-8 JSON and kills the provider-owned process group before inherited pipes drain. Generic LSP uses byte-faithful UTF-8 JSON over command pipes. Retaining a sandbox preserves remote files and unmanaged remote state only. Reconnect does not reconstruct host PTY sessions, buffers, process handles, LSP connections or requests, code workers, binding calls, timers, output cursors, or locks. Managed groups terminate and join when their provider disposes before the shared owner pauses, leaves, or kills the sandbox. @@ -38,7 +38,7 @@ The POC has no session-persistence backend, template builder, volume, snapshot, Focused package suites pin owner lifecycle cleanup, filesystem paths/containment/bounded descriptor reads and commit metadata, subprocess executable lookup/process groups/publication rollback, terminal byte I/O/signal identity/default-environment scrubbing/session cleanup, output limits, abort ordering, disposal to quiescence, and package-owned invariant registrations. The generic PTY, LSP, and subprocess Code Runtime suites pin their provider-neutral readiness, cross-namespace `processId`, binding bridge, descriptor isolation, hostile traffic, and worker/descendant cleanup behavior. -A credential-gated Loader composition creates real E2B sandboxes and exercises FS-to-Bash and Bash-to-FS visibility, real-directory runtime-state setup, process-publication rollback, byte-split UTF-8 command output, bounded raw spill and inherited-output draining, PTY default-secret scrubbing, stale-interrupt identity, and process-tree cleanup, parent-swap-safe bounded LSP source reads, Code Runtime host bindings, descriptor-isolated output accounting, descendant-held pipe cleanup, wall timeout, abort, runner cleanup, host-workspace isolation, and final sandbox deletion. The same composition runs through source imports and built package exports. +A credential-gated Loader composition creates real E2B sandboxes and exercises FS-to-Bash and Bash-to-FS visibility, real-directory runtime-state setup, process-publication rollback, byte-split UTF-8 command output, bounded raw spill and inherited-output draining, hostile command and PTY login-profile isolation, default-secret scrubbing, stale-interrupt identity, and process-tree cleanup, parent-swap-safe bounded LSP source reads, Code Runtime host bindings, descriptor-isolated output accounting, descendant-held pipe cleanup, wall timeout, abort, runner cleanup, host-workspace isolation, and final sandbox deletion. The same composition runs through source imports and built package exports. ## Alternatives considered diff --git a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.zh.md b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.zh.md index b1a30c92e3..729a10ec7e 100644 --- a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.zh.md +++ b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.zh.md @@ -28,7 +28,7 @@ E2B 拥有可变文件系统、受管命令与 Bash 进程、终端分配与终 宿主拥有 Cordis 与插件对象、agent loop、agent/会话/goal 状态、会话日志及持久化、LLM(大语言模型)调用、提示词与工具、权限决策、skill(技能)、subagent 编排、PTY 缓冲与就绪状态、LSP JSON-RPC id/队列/协议状态、Code Runtime 类型剥离/输出计量/绑定分发,以及 E2B SDK/网络编排。该 overlay 不会上传、挂载或同步宿主工作区;拼写相同的 cwd 字符串分别指向彼此独立的宿主与远程目录。 -基础适配器承载基底专用机制。`dsh-subprocess-e2b` 直接消费 E2B 的字节 PTY 回调,在异步查找前台进程组的过程中保留终端发送身份,并负责全会话清理。环境快照与普通命令管道会先以 ASCII/base64 跨越已解码的 SDK 回调边界,再执行严格 UTF-8 解码或恢复原始字节;启动准备会携带句柄自有的取消信号,临时终止则会在 SDK 回退后证明远程进程组为空。`dsh-fs-e2b` 通过无依赖辅助程序执行有界源码读取,该程序会在规范化目标下逐级遍历不跟随符号链接的目录描述符。通用 Code Runtime 通过经过验证的 ASCII/base64 帧承载 controller/worker 协议,并在继承的管道排空前终止提供方拥有的进程组。通用 LSP 通过命令管道使用字节保真的 UTF-8 JSON。 +基础适配器承载基底专用机制。适配器内部的 E2B 命令与 PTY 登录 shell 使用位于根目录下、全新随机生成的 `HOME`;在用户 profile 脚本运行前,`dsh-subprocess-e2b` 还会为它们设置已清理凭据名称的空值覆盖。该子进程适配器消费 E2B 的字节 PTY 回调,以 ASCII/base64 跨越已解码的 SDK 回调传输环境快照与命令字节,并使用一个取消控制器与一个可重试的进程组清理事务,后者只有在证明完全停稳后才报告成功。`dsh-fs-e2b` 通过无依赖辅助程序执行有界源码读取,该程序会在规范化目标下逐级遍历不跟随符号链接的目录描述符。通用 Code Runtime 以经过验证、由换行符分隔的 UTF-8 JSON 承载 controller/worker 协议,并在继承的管道排空前终止提供方拥有的进程组。通用 LSP 通过命令管道使用字节保真的 UTF-8 JSON。 保留沙箱只会保存远程文件与未受管的远程状态。重新连接不会重建宿主 PTY 会话、缓冲、进程句柄、LSP 连接或请求、代码 worker、绑定调用、定时器、输出游标或锁。受管进程组会在所属提供方 dispose(资源释放)时终止并等待退出,之后共享所有者才会暂停、脱离或终止沙箱。 @@ -38,7 +38,7 @@ E2B 拥有可变文件系统、受管命令与 Bash 进程、终端分配与终 聚焦包测试套件固定所有者生命周期清理、文件系统路径/containment/有界描述符读取与提交元数据、子进程可执行文件查找/进程组/发布回滚、终端字节 I/O/信号身份/默认环境清理/会话清理、输出上限、中止顺序、等待完全停稳的资源释放,以及包自有不变式注册。通用 PTY、LSP 与子进程 Code Runtime 测试套件固定其提供方无关的就绪判定、跨命名空间 `processId`、绑定桥接、描述符隔离、恶意通信,以及 worker/后代进程清理行为。 -凭据门控的 Loader 组合会创建真实 E2B 沙箱,并演练 FS-to-Bash 与 Bash-to-FS 可见性、真实目录形式的运行时状态设置、进程发布回滚、按字节切分的 UTF-8 命令输出、有界原始 spill 与继承输出排空、PTY 默认秘密清理、陈旧中断身份与进程树清理、可抵御父目录替换的有界 LSP 源码读取、Code Runtime 宿主绑定、描述符隔离的输出记账、后代进程所持管道的清理、墙钟超时、中止、runner 清理、宿主工作区隔离,以及最终删除沙箱。同一组合分别通过源代码导入与已构建包导出运行。 +凭据门控的 Loader 组合会创建真实 E2B 沙箱,并演练 FS-to-Bash 与 Bash-to-FS 可见性、真实目录形式的运行时状态设置、进程发布回滚、按字节切分的 UTF-8 命令输出、有界原始 spill 与继承输出排空、恶意命令与 PTY 登录 profile 隔离、默认秘密清理、陈旧中断身份与进程树清理、可抵御父目录替换的有界 LSP 源码读取、Code Runtime 宿主绑定、描述符隔离的输出记账、后代进程所持管道的清理、墙钟超时、中止、runner 清理、宿主工作区隔离,以及最终删除沙箱。同一组合分别通过源代码导入与已构建包导出运行。 ## 曾考虑的替代方案 diff --git a/docs/cordis-catalog/services.md b/docs/cordis-catalog/services.md index 7ce45c6817..cdd2d87435 100644 --- a/docs/cordis-catalog/services.md +++ b/docs/cordis-catalog/services.md @@ -591,7 +591,7 @@ Owns one lazily consumable E2B SDK handle and its final kill/pause/leave decisio async getSandbox(): Promise ``` -Source: [`packages/e2b/e2b/src/index.ts:97`](../../packages/e2b/e2b/src/index.ts) +Source: [`packages/e2b/e2b/src/index.ts:109`](../../packages/e2b/e2b/src/index.ts) ## `ctx.fs` — `FileSystem` (abstract seam) diff --git a/packages/e2b/e2b/README.i18n.yaml b/packages/e2b/e2b/README.i18n.yaml index 2fc8d403d1..fb645c2e36 100644 --- a/packages/e2b/e2b/README.i18n.yaml +++ b/packages/e2b/e2b/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/e2b/e2b/README.md -README.md: 6eb7d69dd6355de870db6bf05540f2629f464bc8 -README.zh.md: b2e2616cf779bce659fdef58840aa51f095a73cf +README.md: 402903184934903eceb36a04d670e4490879ac65 +README.zh.md: 1fec6ab66858f04a47c8a2a7cb5f6a8907ab6cac diff --git a/packages/e2b/e2b/README.md b/packages/e2b/e2b/README.md index 6eb7d69dd6..4029031849 100644 --- a/packages/e2b/e2b/README.md +++ b/packages/e2b/e2b/README.md @@ -28,7 +28,7 @@ Set `sandboxId` to reconnect a running or paused sandbox instead of creating one ## Lifecycle and ownership -Construction starts one create/connect operation. Before resolving `getSandbox()`, the service creates `cwd` and the private `cwd/.dsh-e2b` adapter-state directory, verifies that the reserved path is a real directory rather than a symlink or another file type, then sets it to mode `0700`. `sandboxId` resolves to a branded `E2BSandboxId` after setup. +Construction starts one create/connect operation. Before resolving `getSandbox()`, the service creates `cwd` and the private `cwd/.dsh-e2b` adapter-state directory, verifies that the reserved path is a real directory rather than a symlink or another file type, then sets it to mode `0700`. Each adapter-internal E2B command shell receives a fresh randomized root-level `HOME`, so the SDK's fixed login shell does not resolve profile files from the mutable user home before the control command. `sandboxId` resolves to a branded `E2BSandboxId` after setup. Disposal first prevents new handle acquisition, then awaits setup and applies exactly one configured disposition. A `SandboxNotFoundError` is accepted when disposal requests `kill`, or when this service created a sandbox with `onTimeout: kill`; otherwise, a not-found error from a requested `pause` rejects teardown because retention was not proved. A newly created sandbox is killed when initial directory setup fails; if that rollback fails, disposal retries it before releasing ownership. A reconnected sandbox is not killed on setup failure because the service did not create it. Provider plugins must load after this owner and dispose before it. diff --git a/packages/e2b/e2b/README.zh.md b/packages/e2b/e2b/README.zh.md index b2e2616cf7..1fec6ab668 100644 --- a/packages/e2b/e2b/README.zh.md +++ b/packages/e2b/e2b/README.zh.md @@ -28,7 +28,7 @@ ## 生命周期与所有权 -构造阶段会启动一次 create/connect 操作。服务在 `getSandbox()` 结算前创建 `cwd` 和私有的 `cwd/.dsh-e2b` 适配器状态目录,验证该预留路径是真实目录而非符号链接或其他文件类型,再把该目录的 mode 设为 `0700`。初始化完成后,`sandboxId` 会结算为品牌类型 `E2BSandboxId`。 +构造阶段会启动一次 create/connect 操作。服务在 `getSandbox()` 结算前创建 `cwd` 和私有的 `cwd/.dsh-e2b` 适配器状态目录,验证该预留路径是真实目录而非符号链接或其他文件类型,再把该目录的 mode 设为 `0700`。每个适配器内部的 E2B 命令 shell 都会获得一个位于根目录下、全新随机生成的 `HOME`,因此 SDK 固定使用的登录 shell 不会在控制命令之前解析可变用户主目录中的配置文件。初始化完成后,`sandboxId` 会结算为品牌类型 `E2BSandboxId`。 资源释放会先阻止继续获取新句柄,再等待初始化完成,并且只应用一种已配置的处置方式。`SandboxNotFoundError` 仅在资源释放请求 `kill`,或本服务创建了配置为 `onTimeout: kill` 的沙箱时才可接受;否则,`pause` 请求返回的未找到错误会导致 teardown 拒绝,因为无法证明保留成功。新建沙箱的初始目录设置失败时,服务会终止该沙箱;如果该回滚失败,资源释放会在解除所有权前重试。重新连接的沙箱设置失败时不会被终止,因为它不是由本服务创建的。提供方插件必须在该所有者之后加载,并在其之前 dispose(资源释放)。 diff --git a/packages/e2b/e2b/src/index.ts b/packages/e2b/e2b/src/index.ts index 150961a897..9e8960a685 100644 --- a/packages/e2b/e2b/src/index.ts +++ b/packages/e2b/e2b/src/index.ts @@ -4,6 +4,7 @@ * @module @deepseek-ai/dsh-e2b */ +import { randomUUID } from 'node:crypto' import { posix } from 'node:path' import { Context, Service } from 'cordis' import z from 'schemastery' @@ -42,6 +43,17 @@ export function quoteE2BShellArg(value: string): string { return `'${value.replaceAll('\'', "'\"'\"'")}'` } +/** + * Isolate E2B's hard-coded login shell behind a fresh randomized home path. + * @param overrides - Additional environment entries for the internal command. + * @returns A fresh mutable map that the E2B SDK may extend. + */ +export function e2bControlEnvs( + overrides: Readonly> = {}, +): Record { + return { ...overrides, HOME: `/.dsh-e2b-control-${randomUUID()}` } +} + /** Action taken on the owned sandbox when the Cordis service is disposed. */ export type E2BDisposeMode = 'kill' | 'pause' | 'leave' @@ -249,7 +261,10 @@ export class E2BSandboxService extends Service { if (runtimeRoot.type !== FileType.DIR || runtimeRoot.symlinkTarget !== undefined) { throw new Error(`dsh-e2b: runtime root must be a real directory: ${this.runtimeRoot}`) } - await sandbox.commands.run(`chmod 700 -- ${quoteE2BShellArg(this.runtimeRoot)}`) + await sandbox.commands.run( + `chmod 700 -- ${quoteE2BShellArg(this.runtimeRoot)}`, + { envs: e2bControlEnvs() }, + ) return sandbox } catch (error: unknown) { if (this.created) { diff --git a/packages/e2b/e2b/tests/composition.e2e.ts b/packages/e2b/e2b/tests/composition.e2e.ts index 34875d5153..f9fcbfae8a 100644 --- a/packages/e2b/e2b/tests/composition.e2e.ts +++ b/packages/e2b/e2b/tests/composition.e2e.ts @@ -5,7 +5,12 @@ import { Context } from 'cordis' import { describe, expect, it } from 'vitest' import type { Agent } from '@deepseek-ai/dsh-agent' import { runLoaderSmoke } from '@deepseek-ai/dsh-loader-smoke' -import E2BSandboxService, { Sandbox, SandboxNotFoundError } from '@deepseek-ai/dsh-e2b' +import E2BSandboxService, { + e2bControlEnvs, + FileNotFoundError, + Sandbox, + SandboxNotFoundError, +} from '@deepseek-ai/dsh-e2b' import PtyService, { PtySessionId } from '@deepseek-ai/dsh-pty' import { LocalPtyBackend } from '@deepseek-ai/dsh-pty-local' import { Session, SessionId } from '@deepseek-ai/dsh-session' @@ -17,7 +22,7 @@ const configPath = join(fixtureRoot, 'cordis.yml') const tsconfigPath = fileURLToPath(new URL('../../../../tsconfig.json', import.meta.url)) describe.skipIf(!process.env.E2B_API_KEY)('E2B live Loader composition', () => { - it('scrubs sandbox-default credentials from an actual E2B PTY', async () => { + it('scrubs credentials before actual E2B command and PTY login shells', async () => { const apiKey = process.env.E2B_API_KEY if (apiKey === undefined) throw new Error('E2B_API_KEY disappeared before the PTY environment test') const sandbox = await Sandbox.create({ @@ -28,6 +33,18 @@ describe.skipIf(!process.env.E2B_API_KEY)('E2B live Loader composition', () => { lifecycle: { onTimeout: 'kill' }, }) try { + const profileLeakPath = '/home/user/dsh-e2b-bootstrap-profile-leak' + const hostileProfile = [ + 'if [[ "${NPM_TOKEN-}" == "sentinel-secret" ]]; then', + ` printf leaked > ${profileLeakPath}`, + 'fi', + '', + ].join('\n') + await sandbox.files.write([ + { path: '/home/user/.bash_profile', data: hostileProfile }, + { path: '/home/user/.profile', data: hostileProfile }, + { path: '/home/user/.bashrc', data: hostileProfile }, + ]) const ctx = new Context() ctx.provide('e2b', { cwd: '/home/user', @@ -43,6 +60,7 @@ describe.skipIf(!process.env.E2B_API_KEY)('E2B live Loader composition', () => { const node = await ctx.subprocess.resolveExecutable('node') const relativeNodePath = posix.relative(ctx.subprocess.cwd, posix.dirname(node)) || '.' await expect(ctx.subprocess.resolveExecutable('node', { PATH: relativeNodePath })).resolves.toBe(node) + await expect(sandbox.files.read(profileLeakPath)).rejects.toBeInstanceOf(FileNotFoundError) const environmentProbe = ctx.subprocess.spawn({ argv: ['/bin/bash', '-c', [ 'dsh_leak=0', @@ -59,6 +77,7 @@ describe.skipIf(!process.env.E2B_API_KEY)('E2B live Loader composition', () => { }) await expect(environmentProbe.done).resolves.toEqual({ exitCode: 0, signal: null }) expect(environmentProbe.collected.stdout?.readFrom(0).text).toBe('DIRECT=<> LEAK=<0>\n') + await expect(sandbox.files.read(profileLeakPath)).rejects.toBeInstanceOf(FileNotFoundError) const ownerId = SessionId('e2b-pty-env-owner') const owner: Agent = { id: ownerId, @@ -89,6 +108,7 @@ describe.skipIf(!process.env.E2B_API_KEY)('E2B live Loader composition', () => { expect(result.viewport).toContain('NPM=<> DSH=<> KEEP=') expect(result.viewport).not.toContain('sentinel-secret') expect(result.viewport).not.toContain('sentinel-stale') + await expect(sandbox.files.read(profileLeakPath)).rejects.toBeInstanceOf(FileNotFoundError) await session.close('environment test complete') await subprocessFiber.dispose() await ptyFiber.dispose() @@ -98,7 +118,7 @@ describe.skipIf(!process.env.E2B_API_KEY)('E2B live Loader composition', () => { 'mkdir -p -- /home/user/dsh-e2b-runtime-target', 'chmod 755 -- /home/user/dsh-e2b-runtime-target', 'ln -s -- /home/user/dsh-e2b-runtime-target /home/user/.dsh-e2b', - ].join('\n')) + ].join('\n'), { envs: e2bControlEnvs({ NPM_TOKEN: '' }) }) const linkedCtx = new Context() const linkedFiber = await linkedCtx.plugin(E2BSandboxService, { apiKey, diff --git a/packages/e2b/e2b/tests/e2b.spec.ts b/packages/e2b/e2b/tests/e2b.spec.ts index a4eba5ebc9..46ba452210 100644 --- a/packages/e2b/e2b/tests/e2b.spec.ts +++ b/packages/e2b/e2b/tests/e2b.spec.ts @@ -1,7 +1,9 @@ import { beforeEach, describe, expect, it, vi } from 'vitest' +import type { Mock } from 'vitest' import { Context } from 'cordis' import type { Sandbox as SandboxType } from 'e2b' import E2BSandboxService, { + e2bControlEnvs, E2BSandboxId, FileType, SandboxNotFoundError, @@ -35,15 +37,20 @@ interface SandboxFixture { sandbox: SandboxType makeDir: ReturnType getInfo: ReturnType - run: ReturnType + run: Mock kill: ReturnType pause: ReturnType } +type RunCommand = ( + command: string, + options?: { envs?: Record }, +) => Promise<{ exitCode: number; stdout: string; stderr: string }> + function fakeSandbox(id = 'sandbox-1'): SandboxFixture { const makeDir = vi.fn().mockResolvedValue(true) const getInfo = vi.fn().mockResolvedValue({ type: FileType.DIR }) - const run = vi.fn().mockResolvedValue({ exitCode: 0, stdout: '', stderr: '' }) + const run = vi.fn().mockResolvedValue({ exitCode: 0, stdout: '', stderr: '' }) const kill = vi.fn().mockResolvedValue(undefined) const pause = vi.fn().mockResolvedValue(true) const sandbox = { @@ -63,6 +70,15 @@ beforeEach(() => { }) describe('E2BSandboxService', () => { + it('gives each SDK login shell a fresh non-overridable control home', () => { + const first = e2bControlEnvs({ HOME: '/hostile', NPM_TOKEN: '' }) + const second = e2bControlEnvs() + + expect(first.HOME).toMatch(/^\/\.dsh-e2b-control-/) + expect(first).toEqual({ HOME: first.HOME, NPM_TOKEN: '' }) + expect(first.HOME).not.toBe(second.HOME) + }) + it('creates one protected shared sandbox and kills it on default disposal', async () => { const fixture = fakeSandbox() sdk.create.mockResolvedValue(fixture.sandbox) @@ -86,7 +102,12 @@ describe('E2BSandboxService', () => { expect(fixture.makeDir).toHaveBeenNthCalledWith(1, '/home/user/workspace') expect(fixture.makeDir).toHaveBeenNthCalledWith(2, '/home/user/workspace/.dsh-e2b') expect(fixture.getInfo).toHaveBeenCalledWith('/home/user/workspace/.dsh-e2b') - expect(fixture.run).toHaveBeenCalledWith("chmod 700 -- '/home/user/workspace/.dsh-e2b'") + const runOptions = fixture.run.mock.calls[0]?.[1] + expect(runOptions?.envs?.HOME).toMatch(/^\/\.dsh-e2b-control-/) + expect(fixture.run).toHaveBeenCalledWith( + "chmod 700 -- '/home/user/workspace/.dsh-e2b'", + { envs: { HOME: runOptions?.envs?.HOME } }, + ) await fiber.dispose() expect(fixture.kill).toHaveBeenCalledOnce() diff --git a/packages/e2b/fs-e2b/src/index.ts b/packages/e2b/fs-e2b/src/index.ts index d41c2e3d32..7c81791d55 100644 --- a/packages/e2b/fs-e2b/src/index.ts +++ b/packages/e2b/fs-e2b/src/index.ts @@ -20,6 +20,7 @@ import type { } from '@deepseek-ai/dsh-fs' import { CommandExitError, + e2bControlEnvs, FileNotFoundError, FileType, quoteE2BShellArg, @@ -71,6 +72,10 @@ function signalOpts(signal: AbortSignal | undefined): { signal?: AbortSignal } { return signal === undefined ? {} : { signal } } +function commandOpts(signal: AbortSignal | undefined): { envs: Record; signal?: AbortSignal } { + return { envs: e2bControlEnvs(), ...signalOpts(signal) } +} + function entryType(entry: EntryInfo): FsInfo['type'] { switch (entry.type) { case FileType.FILE: @@ -215,7 +220,7 @@ export class E2BFileSystem extends FileSystem { assertNotAborted(signal, 'read') const sandbox = await this.ctx.e2b.getSandbox() try { - const node = await sandbox.commands.run('command -v -- node', signalOpts(signal)) + const node = await sandbox.commands.run('command -v -- node', commandOpts(signal)) const executable = node.stdout.trim() if (!posix.isAbsolute(executable) || executable.includes('\n')) { throw new Error('fs-e2b: bounded reader requires one absolute Node executable') @@ -228,7 +233,7 @@ export class E2BFileSystem extends FileSystem { quoteE2BShellArg(this.processPath(target)), String(maxBytes), ].join(' ') - const result = await sandbox.commands.run(command, signalOpts(signal)) + const result = await sandbox.commands.run(command, commandOpts(signal)) assertNotAborted(signal, 'read') const response = this.parseBoundedRead(result.stdout, target) if (response.kind === 'not-file') { @@ -415,7 +420,7 @@ export class E2BFileSystem extends FileSystem { private async canonicalPath(sandbox: Sandbox, path: string, signal?: AbortSignal): Promise { try { - const result = await sandbox.commands.run(`realpath -m -- ${quoteE2BShellArg(path)}`, signalOpts(signal)) + const result = await sandbox.commands.run(`realpath -m -- ${quoteE2BShellArg(path)}`, commandOpts(signal)) return result.stdout.replace(/\n$/, '') } catch (error: unknown) { if (error instanceof CommandExitError) throw new Error(error.stderr || error.message, { cause: error }) @@ -505,7 +510,7 @@ export class E2BFileSystem extends FileSystem { const created = await sandbox.files.makeDir(stagingDirectory, signalOpts(signal)) if (!created) throw new Error('private staging directory already exists') stagingDirectoryCreated = true - await sandbox.commands.run(`chmod 700 -- ${quoteE2BShellArg(stagingDirectory)}`, signalOpts(signal)) + await sandbox.commands.run(`chmod 700 -- ${quoteE2BShellArg(stagingDirectory)}`, commandOpts(signal)) assertNotAborted(signal, 'write') await sandbox.files.write(temporary, content, { metadata: { [VERSION_METADATA_KEY]: versionId }, @@ -515,7 +520,7 @@ export class E2BFileSystem extends FileSystem { const mode = existing === undefined ? 0o600 : existing.mode & 0o777 await sandbox.commands.run( `chmod ${mode.toString(8)} -- ${quoteE2BShellArg(temporary)}`, - signalOpts(signal), + commandOpts(signal), ) assertNotAborted(signal, 'write') const committed = await sandbox.files.rename(temporary, targetPath) diff --git a/packages/e2b/fs-e2b/tests/filesystem.spec.ts b/packages/e2b/fs-e2b/tests/filesystem.spec.ts index 1689fb4023..23835e9967 100644 --- a/packages/e2b/fs-e2b/tests/filesystem.spec.ts +++ b/packages/e2b/fs-e2b/tests/filesystem.spec.ts @@ -225,8 +225,14 @@ class FakeRemote { }, }, commands: { - run: async (command: string, options?: { signal?: AbortSignal }): Promise<{ exitCode: number; stdout: string; stderr: string }> => { + run: async ( + command: string, + options?: { envs?: Record; signal?: AbortSignal }, + ): Promise<{ exitCode: number; stdout: string; stderr: string }> => { this.checkAbort(options) + const home = options?.envs?.HOME + expect(home).toMatch(/^\/\.dsh-e2b-control-/) + expect(options?.envs).toEqual({ HOME: home }) this.commands.push(command) if (this.nextCommandError !== undefined) { const error = this.nextCommandError diff --git a/packages/e2b/subprocess-e2b/README.i18n.yaml b/packages/e2b/subprocess-e2b/README.i18n.yaml index 3f64c67b7e..ac7b8d42a5 100644 --- a/packages/e2b/subprocess-e2b/README.i18n.yaml +++ b/packages/e2b/subprocess-e2b/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/e2b/subprocess-e2b/README.md -README.md: aaac559e60162c3051cc6f06cbcfac011f5eb59a -README.zh.md: 904e4fd1d9a13082ddfc8183c0e42abc25bd1caf +README.md: 9b86af428533ebcc2c0da56339e6ea2a28170fd3 +README.zh.md: e3e8368534266e35ec04dcf5a2d6718829b4a016 diff --git a/packages/e2b/subprocess-e2b/README.md b/packages/e2b/subprocess-e2b/README.md index aaac559e60..9b86af4285 100644 --- a/packages/e2b/subprocess-e2b/README.md +++ b/packages/e2b/subprocess-e2b/README.md @@ -8,8 +8,8 @@ E2B implementation of the [`@deepseek-ai/dsh-subprocess`](../../subprocess/subpr - **Asynchronous remote start** — the synchronous seam returns a handle immediately while `Sandbox.commands.run(..., { background: true })` starts remotely. `pid` is `-1` until the wrapper publishes and the adapter validates its process-group id; stdin and ordinary observation wait for that publication. An owned startup signal aborts environment and private-state preparation before allocation; once allocation begins, cancellation waits for a provisional SDK handle it can clean. - **Execution-world coordinates** — `cwd` and private `runtimeRoot` come from the shared owner; executable lookup verifies absolute paths or resolves a bare name against the sandbox PATH plus explicit overrides. -- **Linux process groups** — a quoted wrapper starts each argv under `exec setsid --wait` and records its actual process-group id plus private status files beneath `ctx.e2b.runtimeRoot/processes`. The handle waits for that file instead of treating the SDK command PID as its published identity. Termination signals the negative recorded id with `SIGTERM`, waits the caller's `graceMs`, then escalates to `SIGKILL` and the SDK kill fallback; TERM delivery or probe failures also force that escalation. Process-table probes treat groups containing only zombie or dead entries as quiescent. A failed transaction is observable through `waitForExit()` and may be retried, while any proven quiescence permanently fences later termination against PID reuse. Before publication, cancellation uses both the provisional group and SDK handle and proves the group empty before reporting quiescence; if publication fails, rollback applies the same proof before startup rejects. After publication, a monitoring failure also rolls back the group before rejecting. Service disposal rejects new starts, terminates and joins every retained process group, then awaits SDK settlement and private cleanup before the sandbox owner disposes. -- **Environment boundary** — the sandbox command environment crosses the SDK callback boundary as base64 ASCII before one strict UTF-8 decode, then the wrapper removes ambient `DSH_*` and credential-shaped (`*KEY*`, `*SECRET*`, `*TOKEN*`) names and restores every valid `spec.env` entry as an explicit caller opt-in; empty names, `=`, and NUL framing violations reject before launch. Host ambient variables never enter the sandbox implicitly. Private environment files are removed after consumption, and failed command or terminal setup removes its private state before rejecting. +- **Linux process groups** — a quoted wrapper starts each argv under `exec setsid --wait` and records its actual process-group id plus private status files beneath `ctx.e2b.runtimeRoot/processes`. The handle waits for that file instead of treating the SDK command PID as its published identity. Termination signals the negative recorded id with `SIGTERM`, waits the caller's `graceMs`, then escalates to `SIGKILL` and the SDK kill fallback; TERM delivery or probe failures also force that escalation. Process-table probes treat groups containing only zombie or dead entries as quiescent. Force cleanup succeeds only after a bounded probe finds the group empty; otherwise `waitForExit()` exposes a retryable failure, while proven quiescence makes later termination a no-op. Publication and monitoring failures apply the same cleanup transaction before rejecting. Service disposal rejects new starts, terminates and joins every retained process group, then awaits SDK settlement and private cleanup before the sandbox owner disposes. +- **Environment boundary** — the sandbox command environment crosses the SDK callback boundary as base64 ASCII before one strict UTF-8 decode, then the wrapper removes ambient `DSH_*` and credential-shaped (`*KEY*`, `*SECRET*`, `*TOKEN*`) names and restores every valid `spec.env` entry as an explicit caller opt-in; empty names, `=`, and NUL framing violations reject before launch. E2B's fixed command and PTY login shells receive a fresh randomized root-level `HOME` plus empty overrides for every scrubbed ambient name before user profiles can run; the requested argv receives the serialized environment afterward. Host ambient variables never enter the sandbox implicitly. Private environment files are removed after consumption, and failed command or terminal setup removes its private state before rejecting. - **Stdio projection** — the remote wrapper branches raw bytes into optional bounded spill files, frames each live chunk as newline-delimited base64 ASCII, and the host incrementally restores bytes across arbitrary SDK callback boundaries. Pipe mode writes those bytes to host Node streams; inherit mode writes them to the harness process streams; collect mode retains a bounded host tail with offset reads. The wrapper publishes the direct command status before waiting for inherited writers. For collect or inherit output, the adapter disconnects an incomplete SDK stream after `graceMs`, withholds its partial spill, and returns that status while retaining the remote group for `waitForExit()` and termination. Natural raw-pipe completion instead awaits lossless transport and preserves backpressure; explicit termination destroys the host pipes and releases blocked output before remote cleanup. Batch and streaming stdin use the SDK handle. - **Terminal sessions** — `spawnTerminal()` uses E2B's byte PTY API, installs the exact argv and scrubbed environment through private mode-`0600` files, reports the foreground process group, sends real signals, and tears down every live group in the remote terminal session before settlement; zombie-only groups are already quiescent. A private random output boundary discards the E2B bootstrap shell's prompt and echoed runner command while preserving every requested-process byte, including its first prompt. PTY allocation is awaited through handle publication before cancellation is observed, so owned rollback can clean the published handle. Setup and teardown own the private state transaction, abort pending setup during service disposal, fence publication, and retain an unproven setup cleanup for disposal retry. Prompt detection, scrollback, readiness, and owner policy remain in `dsh-pty-local`. - **Sandbox disappearance** — `SandboxNotFoundError` during process or terminal liveness, termination, rollback, or disconnect proves the remote execution world cannot retain work, so cleanup treats it as quiescent; unrelated failures remain observable. diff --git a/packages/e2b/subprocess-e2b/README.zh.md b/packages/e2b/subprocess-e2b/README.zh.md index 904e4fd1d9..e3e8368534 100644 --- a/packages/e2b/subprocess-e2b/README.zh.md +++ b/packages/e2b/subprocess-e2b/README.zh.md @@ -8,8 +8,8 @@ - **异步远程启动**:同步 seam 会立即返回一个句柄,同时由 `Sandbox.commands.run(..., { background: true })` 在远程启动进程。包装层发布进程组 ID 并由适配器完成验证之前,`pid` 为 `-1`;stdin 和常规观察会等待该发布。自有启动信号会在分配前中止环境和私有状态准备;分配开始后,取消会等待可清理的临时 SDK 句柄。 - **执行世界坐标**:`cwd` 和私有 `runtimeRoot` 来自共享所有者;可执行文件查找会验证绝对路径,或根据沙箱 PATH 加显式覆盖来解析裸名称。 -- **Linux 进程组**:带引号保护的包装层会在 `exec setsid --wait` 下启动每组 argv,并在 `ctx.e2b.runtimeRoot/processes` 下记录实际进程组 ID 和私有状态文件。句柄会等待该文件,而不会把 SDK 命令 PID 当作已发布的身份。终止操作以记录的负数 ID 发送 `SIGTERM`,等待调用方的 `graceMs`,再升级到 `SIGKILL` 和 SDK kill 回退;TERM 信号发送或探测失败也会强制触发该升级。进程表探测会把仅含僵尸或已死亡条目的进程组视为完全停稳。失败的事务可通过 `waitForExit()` 观察,并可重试;任何已证明的完全停稳都会永久防止后续终止操作命中复用的 PID。发布前,取消操作会同时使用临时进程组与 SDK 句柄,并在报告完全停稳前证明进程组为空;如果发布失败,回滚会在启动拒绝前执行相同证明。发布后,监控失败也会在拒绝前回滚进程组。服务 dispose(资源释放)会拒绝新的启动请求、终止并等待每个保留进程组退出,再等待 SDK 结算和私有清理完成,之后沙箱所有者才会释放。 -- **环境边界**:沙箱命令环境会先以 base64 ASCII 跨越 SDK 回调边界,再进行一次严格 UTF-8 解码;随后包装层移除环境中的 `DSH_*` 和形似凭据的名称(`*KEY*`、`*SECRET*`、`*TOKEN*`),并把每个有效的 `spec.env` 条目恢复为调用方显式选择。空名称、`=` 和违反 NUL 分帧规则的条目会在启动前被拒绝。宿主环境变量绝不会隐式进入沙箱。私有环境文件在使用后会被删除;命令或终端设置失败时,会先删除其私有状态再拒绝。 +- **Linux 进程组**:带引号保护的包装层会在 `exec setsid --wait` 下启动每组 argv,并在 `ctx.e2b.runtimeRoot/processes` 下记录实际进程组 ID 和私有状态文件。句柄会等待该文件,而不会把 SDK 命令 PID 当作已发布的身份。终止操作以记录的负数 ID 发送 `SIGTERM`,等待调用方的 `graceMs`,再升级到 `SIGKILL` 和 SDK kill 回退;TERM 信号发送或探测失败也会强制触发该升级。进程表探测会把仅含僵尸或已死亡条目的进程组视为完全停稳。强制清理只有在有界探测发现进程组为空后才算成功;否则 `waitForExit()` 会公开可重试的失败,而已证明的完全停稳会让后续终止操作不再执行任何动作。发布失败与监控失败都会在拒绝前执行同一清理事务。服务 dispose(资源释放)会拒绝新的启动请求、终止并等待每个保留进程组退出,再等待 SDK 结算和私有清理完成,之后沙箱所有者才会释放。 +- **环境边界**:沙箱命令环境会先以 base64 ASCII 跨越 SDK 回调边界,再进行一次严格 UTF-8 解码;随后包装层移除环境中的 `DSH_*` 和形似凭据的名称(`*KEY*`、`*SECRET*`、`*TOKEN*`),并把每个有效的 `spec.env` 条目恢复为调用方显式选择。空名称、`=` 和违反 NUL 分帧规则的条目会在启动前被拒绝。在用户 profile 脚本运行前,E2B 固定的命令 shell 与 PTY 登录 shell 会获得位于根目录下、全新随机生成的 `HOME`,并为每个被清理的环境变量名设置空值覆盖;之后,请求的 argv 才会接收序列化环境。宿主环境变量绝不会隐式进入沙箱。私有环境文件在使用后会被删除;命令或终端设置失败时,会先删除其私有状态再拒绝。 - **stdio 投影**:远程包装层先把原始字节分流到可选的有界 spill 文件,再把每个实时分片编码为换行分隔的 base64 ASCII 帧;宿主会跨任意 SDK 回调边界增量恢复字节。pipe 模式把这些字节写入宿主 Node 流;inherit 模式把字节写入 harness 进程流;collect 模式保留有界的宿主尾部,并支持基于偏移量读取。包装层会在等待继承管道的写入方之前发布直接命令状态。对于 collect 或 inherit 输出,超过 `graceMs` 后,适配器会断开未完成的 SDK 流,不公开其中不完整的 spill,并返回该状态,同时保留远程进程组供 `waitForExit()` 和终止操作使用。原始 pipe 自然完成时,会等待无损传输完成并保留背压;显式终止则会销毁宿主 pipe,并在远程清理前释放受阻的输出写入。批量 stdin 和流式 stdin 都使用 SDK 句柄。 - **终端会话**:`spawnTerminal()` 使用 E2B 的字节 PTY API,以 mode 为 `0600` 的私有文件传入原样 argv 与清理后的环境,报告前台进程组,发送真实信号,并在结算前清理远程终端会话中仍存活的每个进程组;仅含僵尸进程的进程组已经完全停稳。私有随机输出边界会丢弃 E2B 引导 shell 的提示符和回显的 runner 命令,同时保留请求进程的每个字节,包括其第一个提示符。PTY 分配会一直等待到句柄发布后才观察取消,以便由承担清理责任的回滚清理已发布句柄。setup 与 teardown 负责私有状态事务,在服务 dispose 期间中止待处理的 setup、阻止发布,并保留未证明已完成的 setup 清理事务,供 dispose 重试。提示符检测、scrollback、就绪状态与所有者策略仍归 `dsh-pty-local` 所有。 - **沙箱消失**:在进程或终端的存活探测、终止、回滚或断开连接期间出现 `SandboxNotFoundError`,证明远程执行环境无法保留工作,因此清理会将其视为完全停稳;其他故障仍可观察。 diff --git a/packages/e2b/subprocess-e2b/src/environment.ts b/packages/e2b/subprocess-e2b/src/environment.ts index 2b2516a706..5322491261 100644 --- a/packages/e2b/subprocess-e2b/src/environment.ts +++ b/packages/e2b/subprocess-e2b/src/environment.ts @@ -1,11 +1,24 @@ /** Shared remote-environment scrubbing for E2B process and terminal launchers. */ import { Buffer } from 'node:buffer' +import { posix } from 'node:path' +import { e2bControlEnvs } from '@deepseek-ai/dsh-e2b' import type { Sandbox } from '@deepseek-ai/dsh-e2b' import { SENSITIVE_ENV_PATTERN } from '@deepseek-ai/dsh-subprocess' const BASE64 = /^(?:[A-Za-z0-9+/]{4})*(?:[A-Za-z0-9+/]{2}==|[A-Za-z0-9+/]{3}=)?$/ +function remoteEnvironmentEntries(raw: string): Array { + const entries: Array = [] + for (const entry of raw.split('\0')) { + if (entry.length === 0) continue + const separator = entry.indexOf('=') + if (separator <= 0) continue + entries.push([entry.slice(0, separator), entry.slice(separator + 1)]) + } + return entries +} + /** * Read the remote environment through ASCII base64 so SDK callback chunking cannot corrupt UTF-8. * @param sandbox - shared E2B execution world. @@ -14,16 +27,29 @@ const BASE64 = /^(?:[A-Za-z0-9+/]{4})*(?:[A-Za-z0-9+/]{2}==|[A-Za-z0-9+/]{3}=)?$ */ export async function readRemoteEnvironment(sandbox: Sandbox, signal?: AbortSignal): Promise { const result = await sandbox.commands.run( - 'set -o pipefail; env -0 | base64 -w 0', - signal === undefined ? {} : { signal }, + 'set -o pipefail; printf \'%s\' "$PWD" | base64 -w 0; printf \'\\n\'; env -0 | base64 -w 0', + { envs: e2bControlEnvs(), ...(signal === undefined ? {} : { signal }) }, ) - const encoded = result.stdout.trim() - if (!BASE64.test(encoded)) throw new Error('subprocess-e2b: remote environment transport returned invalid base64') + const lines = result.stdout.trim().split('\n') + if (lines.length !== 2 || !lines.every(line => BASE64.test(line))) { + throw new Error('subprocess-e2b: remote environment transport returned invalid base64') + } + const [encodedHome, encodedEnvironment] = lines as [string, string] + let home: string + let raw: string try { - return new TextDecoder('utf-8', { fatal: true }).decode(Buffer.from(encoded, 'base64')) + const decoder = new TextDecoder('utf-8', { fatal: true }) + home = decoder.decode(Buffer.from(encodedHome, 'base64')) + raw = decoder.decode(Buffer.from(encodedEnvironment, 'base64')) } catch (error: unknown) { throw new Error('subprocess-e2b: remote environment is not valid UTF-8', { cause: error }) } + if (!posix.isAbsolute(home) || home.includes('\0')) { + throw new Error(`subprocess-e2b: remote login home is invalid: ${JSON.stringify(home)}`) + } + const environment = new Map(remoteEnvironmentEntries(raw)) + environment.set('HOME', home) + return [...environment].map(([name, value]) => `${name}=${value}\0`).join('') } /** @@ -33,13 +59,22 @@ export async function readRemoteEnvironment(sandbox: Sandbox, signal?: AbortSign */ export function scrubRemoteEnvironment(raw: string): Map { const environment = new Map() - for (const entry of raw.split('\0')) { - if (entry.length === 0) continue - const separator = entry.indexOf('=') - if (separator <= 0) continue - const name = entry.slice(0, separator) + for (const [name, value] of remoteEnvironmentEntries(raw)) { if (name.startsWith('DSH_') || SENSITIVE_ENV_PATTERN.test(name)) continue - environment.set(name, entry.slice(separator + 1)) + environment.set(name, value) + } + return environment +} + +/** + * Isolate E2B's fixed login-shell bootstrap from user profiles and ambient credentials. + * @param raw - The complete NUL-delimited remote environment. + * @returns Explicit E2B command or PTY overrides for bootstrap-shell startup. + */ +export function bootstrapEnvironment(raw: string): Record { + const environment: Record = { TERM: 'dumb' } + for (const [name] of remoteEnvironmentEntries(raw)) { + if (name.startsWith('DSH_') || SENSITIVE_ENV_PATTERN.test(name)) environment[name] = '' } return environment } diff --git a/packages/e2b/subprocess-e2b/src/index.ts b/packages/e2b/subprocess-e2b/src/index.ts index 4516bffa84..7318dfde95 100644 --- a/packages/e2b/subprocess-e2b/src/index.ts +++ b/packages/e2b/subprocess-e2b/src/index.ts @@ -14,7 +14,7 @@ import type { SubprocessTerminalHandle, SubprocessTerminalSpawnSpec, } from '@deepseek-ai/dsh-subprocess' -import { quoteE2BShellArg } from '@deepseek-ai/dsh-e2b' +import { e2bControlEnvs, quoteE2BShellArg } from '@deepseek-ai/dsh-e2b' import { E2BSubprocessHandle } from './process.ts' import { spawnE2BTerminal } from './terminal.ts' @@ -86,7 +86,7 @@ export class E2BSubprocessService extends SubprocessService { if (posix.isAbsolute(command)) { await sandbox.commands.run( `test -f ${quoteE2BShellArg(command)} -a -x ${quoteE2BShellArg(command)}`, - signalOpts(signal), + { envs: e2bControlEnvs(), ...signalOpts(signal) }, ) signal?.throwIfAborted() return command @@ -95,7 +95,7 @@ export class E2BSubprocessService extends SubprocessService { const prefix = path === undefined ? '' : `PATH=${quoteE2BShellArg(path)} ` const result = await sandbox.commands.run( `${prefix}command -v -- ${quoteE2BShellArg(command)}`, - { cwd: this.cwd, ...signalOpts(signal) }, + { cwd: this.cwd, envs: e2bControlEnvs(), ...signalOpts(signal) }, ) signal?.throwIfAborted() const executable = result.stdout.trim() diff --git a/packages/e2b/subprocess-e2b/src/process.ts b/packages/e2b/subprocess-e2b/src/process.ts index 8c83c4b84b..6b17f3b682 100644 --- a/packages/e2b/subprocess-e2b/src/process.ts +++ b/packages/e2b/subprocess-e2b/src/process.ts @@ -5,6 +5,7 @@ import { PassThrough, Writable } from 'node:stream' import { posix } from 'node:path' import { CommandExitError, + e2bControlEnvs, FileNotFoundError, SandboxNotFoundError, quoteE2BShellArg, @@ -18,7 +19,7 @@ import type { SubprocessSpawnSpec, } from '@deepseek-ai/dsh-subprocess' import type E2BSandboxService from '@deepseek-ai/dsh-e2b' -import { readRemoteEnvironment, serializeRemoteEnvironment } from './environment.ts' +import { bootstrapEnvironment, readRemoteEnvironment, serializeRemoteEnvironment } from './environment.ts' import { E2BBase64Decoder, E2B_OUTPUT_COMPLETE_FRAME, E2BOutputReader } from './output.ts' const GROUP_POLL_MS = 20 @@ -142,8 +143,11 @@ function commandText(spec: SubprocessSpawnSpec, paths: RemotePaths): string { return bootstrap } -function signalOpts(signal: AbortSignal | undefined): { signal?: AbortSignal } { - return signal === undefined ? {} : { signal } +function commandOpts( + envs: Record, + signal: AbortSignal | undefined, +): { envs: Record; signal?: AbortSignal } { + return { envs: e2bControlEnvs(envs), ...(signal === undefined ? {} : { signal }) } } function isAborted(signal: AbortSignal | undefined): boolean { @@ -197,21 +201,18 @@ export class E2BSubprocessHandle implements SubprocessHandle { private readonly readyState = Promise.withResolvers() private readonly stdoutDecoder = new E2BBase64Decoder() private readonly stderrDecoder = new E2BBase64Decoder() - private readonly outputTermination = new AbortController() - private readonly startupController = new AbortController() + private readonly terminationController = new AbortController() private readonly stdoutReader: E2BOutputReader | undefined private readonly stderrReader: E2BOutputReader | undefined private readonly paths: RemotePaths + private controlEnvs: Record = {} private remotePid = -1 private commandHandle: CommandHandle | undefined private outputTransportError: Error | undefined private outputDrainExpired = false private stateDirectoryCreated = false private preparing = true - private invalidHandleQuiescent = false - private provisionalHandleQuiescent = false private terminationStarted = false - private terminationFenced = false private quiescenceProven = false private terminationAttempt: Promise | undefined private terminationFailure: Error | undefined @@ -264,20 +265,16 @@ export class E2BSubprocessHandle implements SubprocessHandle { /** @inheritdoc */ terminate(): void { - if (this.terminationFenced || this.quiescenceProven || this.terminationAttempt !== undefined) return + if (this.quiescenceProven || this.terminationAttempt !== undefined) return this.terminationStarted = true - if (this.preparing) this.startupController.abort(new Error('subprocess-e2b: command terminated during startup')) - this.outputTermination.abort() + this.terminationController.abort(new Error('subprocess-e2b: command terminated')) this.stdout?.destroy() this.stderr?.destroy() this.terminationFailure = undefined const attempt = this.terminateRemote() this.terminationAttempt = attempt void attempt.then( - () => { - this.terminationFenced = true - this.terminationAttempt = undefined - }, + () => { this.terminationAttempt = undefined }, (error: unknown) => { if (!this.quiescenceProven) this.terminationFailure = asError(error) this.terminationAttempt = undefined @@ -301,11 +298,8 @@ export class E2BSubprocessHandle implements SubprocessHandle { const attempt = this.terminationAttempt if (attempt !== undefined && await waitWithSignal(attempt, signal) === WAIT_ABORTED) return false this.throwTerminationFailure() - /* v8 ignore else -- successful provisional cleanup always records one proof; failures throw above. */ - if (this.invalidHandleQuiescent || this.provisionalHandleQuiescent) { - this.markQuiescent() - return true - } + // Successful pre-publication termination records quiescence; its only other outcome is the failure above. + return true } } else { try { @@ -361,6 +355,7 @@ export class E2BSubprocessHandle implements SubprocessHandle { { background: true, cwd: this.spec.cwd, + envs: e2bControlEnvs(this.controlEnvs), stdin: this.spec.stdio.stdin !== 'ignore', timeoutMs: 0, onStdout: async (data) => { await this.dispatchOutput('stdout', data) }, @@ -374,7 +369,7 @@ export class E2BSubprocessHandle implements SubprocessHandle { const invalidPid = new Error(`subprocess-e2b: E2B returned invalid command pid ${handle.pid}`) try { await handle.kill() - this.invalidHandleQuiescent = true + this.markQuiescent() this.commandHandle = undefined } catch (cleanupError: unknown) { this.terminationFailure = asError(cleanupError) @@ -412,7 +407,7 @@ export class E2BSubprocessHandle implements SubprocessHandle { } catch (error: unknown) { const canceledPreparation = this.preparing && this.terminationStarted - && this.startupController.signal.aborted + && this.terminationController.signal.aborted let failure = await this.rollbackPublishedFailure(error) if (sandbox !== undefined && this.stateDirectoryCreated) { try { @@ -437,11 +432,15 @@ export class E2BSubprocessHandle implements SubprocessHandle { } private async prepareState(sandbox: Sandbox): Promise { - const signal = this.startupController.signal + const signal = this.terminationController.signal const ambient = await readRemoteEnvironment(sandbox, signal) + this.controlEnvs = bootstrapEnvironment(ambient) await sandbox.files.makeDir(this.stateDir, { signal }) this.stateDirectoryCreated = true - await sandbox.commands.run(`chmod 700 -- ${quoteE2BShellArg(this.stateDir)}`, { signal }) + await sandbox.commands.run( + `chmod 700 -- ${quoteE2BShellArg(this.stateDir)}`, + commandOpts(this.controlEnvs, signal), + ) const files = [ { path: this.paths.pid, data: '' }, { path: this.paths.status, data: '' }, @@ -450,7 +449,10 @@ export class E2BSubprocessHandle implements SubprocessHandle { ...(hasSpill(this.spec.stdio.stderr) ? [{ path: this.paths.stderr, data: '' }] : []), ] await sandbox.files.write(files, { signal }) - await sandbox.commands.run(`chmod 600 -- ${files.map(file => quoteE2BShellArg(file.path)).join(' ')}`, { signal }) + await sandbox.commands.run( + `chmod 600 -- ${files.map(file => quoteE2BShellArg(file.path)).join(' ')}`, + commandOpts(this.controlEnvs, signal), + ) signal.throwIfAborted() } @@ -490,7 +492,7 @@ export class E2BSubprocessHandle implements SubprocessHandle { private async writeOutput(pipe: PassThrough | undefined, inherited: NodeJS.WriteStream | undefined, data: Uint8Array): Promise { const target = pipe ?? inherited - if (target === undefined || data.length === 0 || this.outputTermination.signal.aborted) return + if (target === undefined || data.length === 0 || this.terminationController.signal.aborted) return if (target.destroyed) throw new Error('subprocess output stream is closed') if (target.write(data)) return await new Promise((resolve, reject) => { @@ -502,13 +504,13 @@ export class E2BSubprocessHandle implements SubprocessHandle { target.removeListener('drain', onDrain) target.removeListener('close', onClose) target.removeListener('error', onError) - this.outputTermination.signal.removeEventListener('abort', onTermination) + this.terminationController.signal.removeEventListener('abort', onTermination) } target.once('drain', onDrain) target.once('close', onClose) target.once('error', onError) - this.outputTermination.signal.addEventListener('abort', onTermination, { once: true }) - if (this.outputTermination.signal.aborted) onTermination() + this.terminationController.signal.addEventListener('abort', onTermination, { once: true }) + if (this.terminationController.signal.aborted) onTermination() }) } @@ -596,12 +598,8 @@ export class E2BSubprocessHandle implements SubprocessHandle { // `setsid`, so E2B's command PID is the provisional group id even before the // private publication file can be trusted. Kill that group before the SDK-PID // fallback, then prove no group member survived before rejecting startup. - try { - await this.signalGroup(sandbox, handle.pid, 'KILL') - } finally { - await handle.kill().catch(() => false) - } - while (await this.groupAlive(sandbox, handle.pid)) await waitTick() + await this.forceKillGroup(sandbox, handle, handle.pid) + this.markQuiescent() } private async terminateRemote(): Promise { @@ -618,91 +616,76 @@ export class E2BSubprocessHandle implements SubprocessHandle { private async terminateRemoteInSandbox(): Promise { const handle = await this.commandState.promise - if (handle === undefined) return + if (handle === undefined) { + this.markQuiescent() + return + } if (!isValidProcessId(handle.pid) && this.remotePid <= 0) { await handle.kill() - this.invalidHandleQuiescent = true + this.markQuiescent() this.commandHandle = undefined return } - if (this.remotePid <= 0) { - const sandbox = await this.runtime.getSandbox() - this.terminationSignal = 'SIGTERM' - try { - const delivered = await this.signalGroup(sandbox, handle.pid, 'TERM') - if (delivered) { - const deadline = Date.now() + this.spec.graceMs - while (Date.now() < deadline && await this.groupAlive(sandbox, handle.pid)) await waitTick() - if (!await this.groupAlive(sandbox, handle.pid)) { - this.provisionalHandleQuiescent = true - return - } - } - } catch (_gracefulTerminationFailure) { - // A missing or unobservable provisional group still has the SDK handle fallback. - } - this.terminationSignal = 'SIGKILL' - let groupDelivered = false - let groupFailure: unknown - try { - groupDelivered = await this.signalGroup(sandbox, handle.pid, 'KILL') - } catch (error: unknown) { - groupFailure = error - } - let handleFailure: unknown - try { - if (!await handle.kill()) handleFailure = new Error('E2B SDK kill did not report command termination') - } catch (error: unknown) { - handleFailure = error - } - if (!groupDelivered && await this.groupAlive(sandbox, handle.pid)) { - throw new AggregateError( - [ - ...(groupFailure === undefined ? [] : [groupFailure]), - ...(handleFailure === undefined - ? [new Error('E2B SDK kill did not quiesce the provisional process group')] - : [handleFailure]), - ], - 'subprocess-e2b: force termination failed through both process-group and SDK transports', - ) - } - while (await this.groupAlive(sandbox, handle.pid)) await waitTick() - this.provisionalHandleQuiescent = true - return - } const sandbox = await this.runtime.getSandbox() - const processGroupId = this.remotePid + const processGroupId = this.remotePid > 0 ? this.remotePid : handle.pid + await this.terminateGroup(sandbox, handle, processGroupId) + } + + private async terminateGroup(sandbox: Sandbox, handle: CommandHandle, processGroupId: number): Promise { this.terminationSignal = 'SIGTERM' try { await this.signalGroup(sandbox, processGroupId, 'TERM') - const deadline = Date.now() + this.spec.graceMs - while (Date.now() < deadline && await this.groupAlive(sandbox, processGroupId)) { - await waitTick() + if (await this.waitForGroupExit(sandbox, processGroupId)) { + this.markQuiescent() + return } - if (!await this.groupAlive(sandbox, processGroupId)) return } catch (_gracefulTerminationFailure) { // Failed TERM delivery or observation cannot prove exit; force cleanup still owns the group. } this.terminationSignal = 'SIGKILL' + await this.forceKillGroup(sandbox, handle, processGroupId) + this.markQuiescent() + } + + private async forceKillGroup(sandbox: Sandbox, handle: CommandHandle, processGroupId: number): Promise { let groupFailure: unknown - let groupDelivered = false try { - groupDelivered = await this.signalGroup(sandbox, processGroupId, 'KILL') + if (!await this.signalGroup(sandbox, processGroupId, 'KILL')) { + groupFailure = new Error('process-group KILL did not report delivery') + } } catch (error: unknown) { groupFailure = error } let handleFailure: unknown try { - await handle.kill() + if (!await handle.kill()) handleFailure = new Error('E2B SDK kill did not report command termination') } catch (error: unknown) { handleFailure = error } - if (!groupDelivered && handleFailure !== undefined && await this.groupAlive(sandbox, processGroupId)) { - throw new AggregateError( - [...(groupFailure === undefined ? [] : [groupFailure]), handleFailure], - 'subprocess-e2b: force termination failed through both process-group and SDK transports', - ) + let proofFailure: unknown + try { + if (await this.waitForGroupExit(sandbox, processGroupId)) return + proofFailure = new Error(`remote process group ${processGroupId} remained live after force termination`) + } catch (error: unknown) { + proofFailure = error } + throw new AggregateError( + [ + ...(groupFailure === undefined ? [] : [groupFailure]), + ...(handleFailure === undefined ? [] : [handleFailure]), + proofFailure, + ], + 'subprocess-e2b: force termination failed through both process-group and SDK transports', + ) + } + + private async waitForGroupExit(sandbox: Sandbox, processGroupId: number): Promise { + const deadline = Date.now() + this.spec.graceMs + while (await this.groupAlive(sandbox, processGroupId)) { + if (Date.now() >= deadline) return false + await waitTick() + } + return true } private throwTerminationFailure(): void { @@ -710,8 +693,13 @@ export class E2BSubprocessHandle implements SubprocessHandle { } private async signalGroup(sandbox: Sandbox, pid: number, signal: 'TERM' | 'KILL'): Promise { + // TODO(e2b-pgid-identity): Prefer an atomic identity-bound group signal if E2B adds one; + // a userspace identity precheck cannot close the numeric-PGID reuse race. try { - await sandbox.commands.run(`kill -${signal} -- -${pid}`) + await sandbox.commands.run( + `kill -${signal} -- -${pid}`, + commandOpts(this.controlEnvs, undefined), + ) return true } catch (error: unknown) { if (error instanceof CommandExitError || error instanceof SandboxNotFoundError) return false @@ -722,7 +710,7 @@ export class E2BSubprocessHandle implements SubprocessHandle { private async groupAlive(sandbox: Sandbox, pid: number, signal?: AbortSignal): Promise { const result = await sandbox.commands.run( `set -o pipefail; ps -eo pgid=,stat= | awk '$1 == ${pid} && $2 !~ /^[ZXx]/ { live=1 } END { if (live) print "live" }'`, - signalOpts(signal), + commandOpts(this.controlEnvs, signal), ).catch((error: unknown) => { if (signal?.aborted === true) return undefined if (error instanceof SandboxNotFoundError) return { exitCode: 0, stdout: '', stderr: '' } diff --git a/packages/e2b/subprocess-e2b/src/terminal.ts b/packages/e2b/subprocess-e2b/src/terminal.ts index 9e0258c10d..ff8af60da9 100644 --- a/packages/e2b/subprocess-e2b/src/terminal.ts +++ b/packages/e2b/subprocess-e2b/src/terminal.ts @@ -6,6 +6,7 @@ import { PassThrough } from 'node:stream' import { posix } from 'node:path' import { CommandExitError, + e2bControlEnvs, FileNotFoundError, SandboxNotFoundError, quoteE2BShellArg, @@ -20,7 +21,11 @@ import type { SubprocessTerminalSpawnSpec, } from '@deepseek-ai/dsh-subprocess' import type E2BSandboxService from '@deepseek-ai/dsh-e2b' -import { readRemoteEnvironment, serializeRemoteEnvironment } from './environment.ts' +import { + bootstrapEnvironment, + readRemoteEnvironment, + serializeRemoteEnvironment, +} from './environment.ts' const POLL_MS = 20 @@ -54,6 +59,13 @@ function signalOpts(signal: AbortSignal | undefined): { signal?: AbortSignal } { return signal === undefined ? {} : { signal } } +function commandOpts( + envs: Record, + signal?: AbortSignal, +): { envs: Record; signal?: AbortSignal } { + return { envs: e2bControlEnvs(envs), ...signalOpts(signal) } +} + function delay(ms: number): Promise { return new Promise(resolve => setTimeout(resolve, ms)) } @@ -143,8 +155,13 @@ function serializeValues(values: readonly string[], kind: string): string { return values.map(value => `${value}\0`).join('') } -async function terminalSessionId(sandbox: Sandbox, pid: number, signal?: AbortSignal): Promise { - const result = await sandbox.commands.run(`ps -o sid= -p ${pid}`, signalOpts(signal)) +async function terminalSessionId( + sandbox: Sandbox, + pid: number, + envs: Record, + signal?: AbortSignal, +): Promise { + const result = await sandbox.commands.run(`ps -o sid= -p ${pid}`, commandOpts(envs, signal)) signal?.throwIfAborted() return parsePositiveId(result.stdout, `subprocess-e2b: cannot resolve process session for terminal ${pid}`) } @@ -169,11 +186,16 @@ async function waitUntilReady( } } -async function sessionProcessGroups(sandbox: Sandbox, sessionId: number): Promise { +async function sessionProcessGroups( + sandbox: Sandbox, + sessionId: number, + envs: Record, +): Promise { let result: CommandResult try { result = await sandbox.commands.run( `set -o pipefail; ps -eo sid=,pgid=,stat= | awk '$1 == ${sessionId} && $3 !~ /^[ZXx]/ { print $2 }'`, + commandOpts(envs), ) } catch (error: unknown) { if (error instanceof SandboxNotFoundError) return [] @@ -194,9 +216,17 @@ async function sessionProcessGroups(sandbox: Sandbox, sessionId: number): Promis return [...groups] } -async function signalGroups(sandbox: Sandbox, groups: number[], signal: 'TERM' | 'KILL'): Promise { +async function signalGroups( + sandbox: Sandbox, + groups: number[], + signal: 'TERM' | 'KILL', + envs: Record, +): Promise { try { - await sandbox.commands.run(`kill -${signal} -- ${groups.map(group => `-${group}`).join(' ')}`) + await sandbox.commands.run( + `kill -${signal} -- ${groups.map(group => `-${group}`).join(' ')}`, + commandOpts(envs), + ) } catch (error: unknown) { if (!(error instanceof CommandExitError) && !(error instanceof SandboxNotFoundError)) throw error } @@ -205,14 +235,15 @@ async function signalGroups(sandbox: Sandbox, groups: number[], signal: 'TERM' | async function awaitSessionEmpty( sandbox: Sandbox, sessionId: number, + envs: Record, graceMs: number, kill = false, ): Promise { const deadline = Date.now() + graceMs for (;;) { - const groups = await sessionProcessGroups(sandbox, sessionId) + const groups = await sessionProcessGroups(sandbox, sessionId, envs) if (groups.length === 0 || Date.now() >= deadline) return groups - if (kill) await signalGroups(sandbox, groups, 'KILL') + if (kill) await signalGroups(sandbox, groups, 'KILL', envs) await delay(Math.min(POLL_MS, Math.max(1, deadline - Date.now()))) } } @@ -221,6 +252,7 @@ async function rollbackUnpublishedTerminal( sandbox: Sandbox, handle: CommandHandle, completion: Promise, + envs: Record, graceMs: number, ): Promise { let topLevelExited = false @@ -234,20 +266,20 @@ async function rollbackUnpublishedTerminal( if (validPid) { sessionId = handle.pid try { - sessionId = await terminalSessionId(sandbox, handle.pid) + sessionId = await terminalSessionId(sandbox, handle.pid, envs) } catch (_sessionLookupFailure) { // E2B's PTY leader is also the provisional POSIX session leader, so its // PID remains usable after the setup lookup itself fails or is canceled. } try { - let groups = await sessionProcessGroups(sandbox, sessionId) + let groups = await sessionProcessGroups(sandbox, sessionId, envs) if (groups.length > 0) { - await signalGroups(sandbox, groups, 'TERM') - groups = await awaitSessionEmpty(sandbox, sessionId, graceMs) + await signalGroups(sandbox, groups, 'TERM', envs) + groups = await awaitSessionEmpty(sandbox, sessionId, envs, graceMs) } if (groups.length > 0) { - await signalGroups(sandbox, groups, 'KILL') - await awaitSessionEmpty(sandbox, sessionId, graceMs, true) + await signalGroups(sandbox, groups, 'KILL', envs) + await awaitSessionEmpty(sandbox, sessionId, envs, graceMs, true) } } catch (error: unknown) { attemptFailures.push(asError(error)) @@ -279,7 +311,7 @@ async function rollbackUnpublishedTerminal( const proofFailures: Error[] = [] if (sessionId !== undefined) { try { - const groups = await awaitSessionEmpty(sandbox, sessionId, graceMs, true) + const groups = await awaitSessionEmpty(sandbox, sessionId, envs, graceMs, true) if (groups.length > 0) { proofFailures.push(new Error( `subprocess-e2b: terminal setup rollback failed; surviving process groups: ${groups.join(', ')}`, @@ -322,6 +354,7 @@ export class E2BTerminalHandle implements SubprocessTerminalHandle { readonly output: PassThrough, private readonly completion: Promise, private readonly sessionId: number, + private readonly controlEnvs: Record, private readonly stateDir: string, private readonly graceMs: number, signal?: AbortSignal, @@ -344,7 +377,10 @@ export class E2BTerminalHandle implements SubprocessTerminalHandle { /** @inheritdoc */ async inspectForeground(): Promise { try { - const result = await this.sandbox.commands.run(`ps -o tpgid= -p ${this.pid}`) + const result = await this.sandbox.commands.run( + `ps -o tpgid= -p ${this.pid}`, + commandOpts(this.controlEnvs), + ) return { processGroupId: parsePositiveId( result.stdout, @@ -369,7 +405,10 @@ export class E2BTerminalHandle implements SubprocessTerminalHandle { if (signal === 'SIGKILL' && foreground.processGroupId === this.pid) { throw new Error('refusing to SIGKILL the terminal shell; terminate the terminal session instead') } - await this.sandbox.commands.run(`kill -${signal.slice(3)} -- -${foreground.processGroupId}`) + await this.sandbox.commands.run( + `kill -${signal.slice(3)} -- -${foreground.processGroupId}`, + commandOpts(this.controlEnvs), + ) return foreground.processGroupId } @@ -402,11 +441,11 @@ export class E2BTerminalHandle implements SubprocessTerminalHandle { } private async closeOnce(): Promise { - let groups = await sessionProcessGroups(this.sandbox, this.sessionId) + let groups = await sessionProcessGroups(this.sandbox, this.sessionId, this.controlEnvs) if (groups.length > 0) { this.terminationSignal = 'SIGTERM' - await signalGroups(this.sandbox, groups, 'TERM') - groups = await awaitSessionEmpty(this.sandbox, this.sessionId, this.graceMs) + await signalGroups(this.sandbox, groups, 'TERM', this.controlEnvs) + groups = await awaitSessionEmpty(this.sandbox, this.sessionId, this.controlEnvs, this.graceMs) } if (groups.length === 0 && !this.topLevelExited) { await Promise.race([this.done.catch(() => undefined), delay(this.graceMs)]) @@ -421,7 +460,7 @@ export class E2BTerminalHandle implements SubprocessTerminalHandle { throw error } } - groups = await awaitSessionEmpty(this.sandbox, this.sessionId, this.graceMs, true) + groups = await awaitSessionEmpty(this.sandbox, this.sessionId, this.controlEnvs, this.graceMs, true) if (!this.topLevelExited) await Promise.race([this.done.catch(() => undefined), delay(this.graceMs)]) } if (groups.length > 0) { @@ -469,13 +508,18 @@ export async function spawnE2BTerminal( let handle: CommandHandle | undefined let completion: Promise | undefined let stateDirectoryCreated = false + let controlEnvs: Record = {} try { const ambient = await readRemoteEnvironment(sandbox, spec.signal) + controlEnvs = bootstrapEnvironment(ambient) const environment = serializeRemoteEnvironment(ambient, spec.env) const argv = serializeValues(spec.argv, 'argv') - await sandbox.files.makeDir(stateDir) stateDirectoryCreated = true - await sandbox.commands.run(`chmod 700 -- ${quoteE2BShellArg(stateDir)}`, signalOpts(spec.signal)) + await sandbox.files.makeDir(stateDir, signalOpts(spec.signal)) + await sandbox.commands.run( + `chmod 700 -- ${quoteE2BShellArg(stateDir)}`, + commandOpts(controlEnvs, spec.signal), + ) await sandbox.files.write([ { path: paths.runner, data: TERMINAL_RUNNER_SOURCE }, { path: paths.environment, data: environment }, @@ -484,13 +528,13 @@ export async function spawnE2BTerminal( ], signalOpts(spec.signal)) await sandbox.commands.run( `chmod 600 -- ${quoteE2BShellArg(paths.runner)} ${quoteE2BShellArg(paths.environment)} ${quoteE2BShellArg(paths.argv)} ${quoteE2BShellArg(paths.outputMarker)}`, - signalOpts(spec.signal), + commandOpts(controlEnvs, spec.signal), ) handle = await sandbox.pty.create({ rows: spec.rows, cols: spec.cols, cwd: spec.cwd, - envs: { TERM: 'dumb' }, + envs: e2bControlEnvs(controlEnvs), timeoutMs: 0, onData: (data) => { outputFilter.push(data) }, }) @@ -504,13 +548,14 @@ export async function spawnE2BTerminal( await sandbox.pty.sendInput(handle.pid, Buffer.from(command), signalOpts(spec.signal)) await waitUntilReady(sandbox, paths, completion, spec.signal) await waitForBootstrapOutput(outputFilter.ready, completion, spec.signal) - const sessionId = await terminalSessionId(sandbox, handle.pid, spec.signal) + const sessionId = await terminalSessionId(sandbox, handle.pid, controlEnvs, spec.signal) return new E2BTerminalHandle( sandbox, handle, output, completion, sessionId, + controlEnvs, stateDir, spec.graceMs, spec.signal, @@ -524,7 +569,7 @@ export async function spawnE2BTerminal( if (!terminalQuiescent && handle !== undefined) { try { if (completion === undefined) await handle.kill() - else await rollbackUnpublishedTerminal(sandbox, handle, completion, spec.graceMs) + else await rollbackUnpublishedTerminal(sandbox, handle, completion, controlEnvs, spec.graceMs) terminalQuiescent = true } catch (cleanupError: unknown) { if (cleanupError instanceof SandboxNotFoundError) terminalQuiescent = true diff --git a/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts b/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts index d3c96cf2fc..0e906b1cfb 100644 --- a/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts +++ b/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts @@ -111,6 +111,7 @@ class FakeSandbox { alive = true zombieOnly = false ambient = 'PATH=/ambient/bin\0KEEP=safe\0UNICODE=你好\0NPM_TOKEN=secret\0DSH_STALE=old\0BROKEN\0=bad\0' + environmentHome = '/home/user' environmentWire: string | undefined environmentRequest: ((signal: AbortSignal | undefined) => Promise) | undefined processGroupId = '4242\n' @@ -241,7 +242,9 @@ class FakeSandbox { if (this.envError !== undefined) throw this.envError return { exitCode: 0, - stdout: this.environmentWire ?? Buffer.from(this.ambient).toString('base64'), + stdout: this.environmentWire ?? [this.environmentHome, this.ambient] + .map(value => Buffer.from(value).toString('base64')) + .join('\n'), stderr: '', } } @@ -396,7 +399,14 @@ describe('E2BSubprocessHandle', () => { expect(handle.pid).toBe(4343) expect(fake.handle.sent.map(value => String(value))).toEqual(['hello']) expect(fake.handle.closes).toBe(1) - expect(fake.startOptions?.envs).toBeUndefined() + const controlEnvs = fake.startOptions?.envs + expect(controlEnvs?.HOME).toMatch(/^\/\.dsh-e2b-control-/) + expect(controlEnvs).toEqual({ + TERM: 'dumb', + NPM_TOKEN: '', + DSH_STALE: '', + HOME: controlEnvs?.HOME, + }) const command = fake.commandsSeen.find(value => value.includes('exec "$dsh_e2b_env_bin" -i'))! expect(command).toContain('"$dsh_e2b_setsid" --wait -- "$dsh_e2b_bash" -c') expect(command).not.toContain('DEEPSEEK_API_KEY') @@ -405,7 +415,9 @@ describe('E2BSubprocessHandle', () => { expect(command).not.toContain('explicit-secret') expect(command).not.toContain('hyphen-value') expect(command).not.toContain('${!dsh_e2b_name}') - expect(fake.commandsSeen).toContain('set -o pipefail; env -0 | base64 -w 0') + expect(fake.commandsSeen).toContain( + 'set -o pipefail; printf \'%s\' "$PWD" | base64 -w 0; printf \'\\n\'; env -0 | base64 -w 0', + ) expect(command).toContain('mapfile -d') expect(command).toContain('dsh_e2b_node="$(command -v node)"') expect(command).toContain('"$dsh_e2b_env_bin" -i "$dsh_e2b_node" -e') @@ -421,7 +433,7 @@ describe('E2BSubprocessHandle', () => { '/workspace/.dsh-e2b/processes/one/stderr.log', ]) expect(fake.writtenFileData.get('/workspace/.dsh-e2b/processes/one/environment')).toBe( - 'PATH=/bin\0KEEP=safe\0UNICODE=你好\0FOO-BAR=hyphen-value\0--split-string=literal-value\0DEEPSEEK_API_KEY=explicit-secret\0DSH_MODE=test\0', + 'PATH=/bin\0KEEP=safe\0UNICODE=你好\0HOME=/home/user\0FOO-BAR=hyphen-value\0--split-string=literal-value\0DEEPSEEK_API_KEY=explicit-secret\0DSH_MODE=test\0', ) let piped = '' @@ -560,6 +572,7 @@ describe('E2BSubprocessHandle', () => { await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) expect(fake.handle.disconnects).toBe(1) fake.alive = false + handle.terminate() await expect(handle.waitForExit()).resolves.toBe(true) }) @@ -783,7 +796,7 @@ describe('E2BSubprocessHandle', () => { expect(fake.handle.kills).toBe(1) }) - it('does not treat successful termination transport as observed quiescence', async () => { + it('keeps force cleanup retryable until quiescence is proven', async () => { const fake = new FakeSandbox() fake.trapsTerm = true fake.delaysKill = true @@ -793,16 +806,15 @@ describe('E2BSubprocessHandle', () => { await flush() handle.terminate() await vi.waitFor(() => { expect(fake.handle.kills).toBe(1) }) + await expect(handle.waitForExit()).rejects.toThrow('force termination failed through both') + expect(fake.alive).toBe(true) - let quiescent = false - const waiting = handle.waitForExit().then((value) => { quiescent = value }) - await new Promise(resolve => setTimeout(resolve, 10)) - expect(quiescent).toBe(false) - - fake.alive = false - fake.finish() - await waiting - expect(quiescent).toBe(true) + fake.delaysKill = false + fake.delaysKillCompletion = false + fake.sdkKillStops = true + handle.terminate() + await expect(handle.waitForExit()).resolves.toBe(true) + await expect(handle.done).resolves.toEqual({ exitCode: null, signal: 'SIGKILL' }) }) it('honors termination requested before asynchronous startup finishes', async () => { @@ -870,14 +882,10 @@ describe('E2BSubprocessHandle', () => { handle.terminate() await vi.waitFor(() => { expect(fake.handle.kills).toBe(1) }) - let quiescent = false - const waiting = handle.waitForExit().then((value) => { quiescent = value }) - await new Promise(resolve => setTimeout(resolve, 10)) - expect(quiescent).toBe(false) - + await expect(handle.waitForExit()).rejects.toThrow('force termination failed through both') fake.alive = false - await waiting - expect(quiescent).toBe(true) + handle.terminate() + await expect(handle.waitForExit()).resolves.toBe(true) fake.releaseProcessGroupRead() fake.finish() await handle.done @@ -1102,23 +1110,36 @@ describe('E2BSubprocessHandle', () => { await expect(envHandle.done).rejects.toThrow('ambient lookup failed') expect(envFailure.removed).toEqual([]) - const malformedEnvironment = new FakeSandbox() - malformedEnvironment.environmentWire = '%' - const malformedEnvironmentHandle = new E2BSubprocessHandle( - runtime(malformedEnvironment), - spec(), - '/runtime/malformed-environment', + const expectEnvironmentFailure = async (name: string, wire: string, message: string): Promise => { + const fake = new FakeSandbox() + fake.environmentWire = wire + const failed = new E2BSubprocessHandle(runtime(fake), spec(), `/runtime/${name}`) + await expect(failed.done).rejects.toThrow(message) + } + const encodedEnvironment = Buffer.from('PATH=/bin\0').toString('base64') + const encodedHome = Buffer.from('/home/user').toString('base64') + await expectEnvironmentFailure('malformed-frame', '%', 'invalid base64') + await expectEnvironmentFailure('malformed-base64', `${encodedHome}\n%`, 'invalid base64') + await expectEnvironmentFailure( + 'invalid-utf8-home', + `${Buffer.from([0xff]).toString('base64')}\n${encodedEnvironment}`, + 'not valid UTF-8', ) - await expect(malformedEnvironmentHandle.done).rejects.toThrow('invalid base64') - - const invalidUtf8Environment = new FakeSandbox() - invalidUtf8Environment.environmentWire = Buffer.from([0xff]).toString('base64') - const invalidUtf8EnvironmentHandle = new E2BSubprocessHandle( - runtime(invalidUtf8Environment), - spec(), - '/runtime/invalid-utf8-environment', + await expectEnvironmentFailure( + 'invalid-utf8-environment', + `${encodedHome}\n${Buffer.from([0xff]).toString('base64')}`, + 'not valid UTF-8', + ) + await expectEnvironmentFailure( + 'relative-home', + `${Buffer.from('home/user').toString('base64')}\n${encodedEnvironment}`, + 'remote login home is invalid', + ) + await expectEnvironmentFailure( + 'nul-home', + `${Buffer.from('/home/user\0tail').toString('base64')}\n${encodedEnvironment}`, + 'remote login home is invalid', ) - await expect(invalidUtf8EnvironmentHandle.done).rejects.toThrow('not valid UTF-8') const cleanupFailure = new FakeSandbox() cleanupFailure.backgroundError = new Error('start failed before credential consumption') @@ -1318,10 +1339,11 @@ describe('E2BSubprocessHandle', () => { const failures = Array.from(failure.errors as Iterable) expect(failures).toHaveLength(2) expect(failures[0]).toBeInstanceOf(Error) - expect(failures[1]).toBeInstanceOf(Error) - if (!(failures[0] instanceof Error) || !(failures[1] instanceof Error)) throw new Error('expected nested errors') + expect(failures[1]).toBeInstanceOf(AggregateError) + if (!(failures[0] instanceof Error) || !(failures[1] instanceof AggregateError)) throw new Error('expected nested errors') expect(failures[0].message).toContain('invalid process-group id') - expect(failures[1].message).toBe('rollback signal failed') + expect(failures[1].message).toBe('subprocess-e2b: force termination failed through both process-group and SDK transports') + expect(Array.from(failures[1].errors as Iterable)).toContainEqual(new Error('rollback signal failed')) expect(fake.handle.kills).toBe(1) const bounded = new AbortController() const waiting = handle.waitForExit(bounded.signal) @@ -1330,6 +1352,15 @@ describe('E2BSubprocessHandle', () => { handle.terminate() await expect(handle.waitForExit()).resolves.toBe(true) expect(fake.commandsSeen).toContain('kill -TERM -- -4242') + + const naturallyGone = new FakeSandbox() + naturallyGone.processGroupId = 'not-a-pid\n' + naturallyGone.signalError = new Error('rollback signal failed') + naturallyGone.handle.killError = new Error('SDK kill failed') + const observed = new E2BSubprocessHandle(runtime(naturallyGone), spec(), '/runtime/failed-rollback-observed') + await expect(observed.done).rejects.toThrow('process-group publication failed') + naturallyGone.alive = false + await expect(observed.waitForExit()).resolves.toBe(true) }) it('waits for delayed process-group publication', async () => { @@ -1479,6 +1510,24 @@ describe('E2BSubprocessHandle', () => { raced.terminate() await expect(raced.waitForExit()).resolves.toBe(true) }) + + it('rejects an optimistic SDK kill while descendants survive a failed group KILL', async () => { + const fake = new FakeSandbox() + fake.trapsTerm = true + fake.sdkKillStops = false + fake.signalErrors.push(undefined, new Error('KILL transport failed')) + const handle = new E2BSubprocessHandle(runtime(fake), spec({ graceMs: 1 }), '/runtime/optimistic-sdk-kill') + await flush() + + handle.terminate() + await expect(handle.waitForExit()).rejects.toThrow('force termination failed through both') + expect(fake.alive).toBe(true) + + fake.sdkKillStops = true + handle.terminate() + await expect(handle.waitForExit()).resolves.toBe(true) + await expect(handle.done).resolves.toEqual({ exitCode: null, signal: 'SIGKILL' }) + }) }) describe('E2BSubprocessService', () => { diff --git a/packages/e2b/subprocess-e2b/tests/terminal.spec.ts b/packages/e2b/subprocess-e2b/tests/terminal.spec.ts index 00b9fc41fc..8ad0eb1dde 100644 --- a/packages/e2b/subprocess-e2b/tests/terminal.spec.ts +++ b/packages/e2b/subprocess-e2b/tests/terminal.spec.ts @@ -23,6 +23,7 @@ function commandError(exitCode: number): CommandExitError { interface CommandOptions { signal?: AbortSignal cwd?: string + envs?: Record } class FakeTerminalCommandHandle { @@ -101,6 +102,7 @@ class FakeTerminalSandbox { writeError: unknown sendError: unknown commandFailure: unknown + makeDirRequest: ((signal: AbortSignal | undefined) => Promise) | undefined sessionGroupsFailure: unknown foregroundFailure: unknown termFailure: unknown @@ -129,8 +131,10 @@ class FakeTerminalSandbox { readonly sandbox = { files: { - makeDir: async (path: string): Promise => { + makeDir: async (path: string, options?: CommandOptions): Promise => { this.directories.push(path) + await this.makeDirRequest?.(options?.signal) + options?.signal?.throwIfAborted() return true }, write: async (files: Array<{ path: string; data: string }>): Promise => { @@ -163,7 +167,11 @@ class FakeTerminalSandbox { throw error } if (command.includes('env -0 | base64')) { - return { exitCode: 0, stdout: Buffer.from(this.ambient).toString('base64'), stderr: '' } + return { + exitCode: 0, + stdout: ['/home/user', this.ambient].map(value => Buffer.from(value).toString('base64')).join('\n'), + stderr: '', + } } if (command.includes('command -v -- ')) { return { exitCode: 0, stdout: this.resolvedExecutable, stderr: '' } @@ -262,7 +270,15 @@ describe('E2B terminal allocation', () => { expect(output).toBe('requested-shell$ ') expect(output).not.toContain('buffered banner') expect(output).not.toContain('runner.bash') - expect(fake.createOptions).toMatchObject({ rows: 24, cols: 80, cwd: '/workspace', timeoutMs: 0, envs: { TERM: 'dumb' } }) + expect(fake.createOptions).toMatchObject({ rows: 24, cols: 80, cwd: '/workspace', timeoutMs: 0 }) + const controlEnvs = fake.createOptions?.envs + expect(controlEnvs?.HOME).toMatch(/^\/\.dsh-e2b-control-/) + expect(controlEnvs).toEqual({ + TERM: 'dumb', + NPM_TOKEN: '', + DSH_STALE: '', + HOME: controlEnvs?.HOME, + }) expect(fake.inputs[0]?.data.toString()).toContain("exec /bin/bash '/runtime/terminal-one/runner.bash'") expect(fake.writes.get('/runtime/terminal-one/environment')).toContain('KEEP=visible\0') expect(fake.writes.get('/runtime/terminal-one/environment')).toContain('UNICODE=你好\0') @@ -554,6 +570,7 @@ describe('E2B terminal lifecycle', () => { new PassThrough(), fake.handle.wait(), 123, + { TERM: 'dumb' }, '/runtime/pre-aborted', 1, controller.signal, @@ -807,7 +824,10 @@ describe('E2B subprocess terminal service', () => { fake.resolvedExecutable = 'tools/bin/node\n' await expect(ctx.subprocess.resolveExecutable('node', { PATH: 'tools/bin' })) .resolves.toBe('/workspace/tools/bin/node') - expect(fake.commandOptions.at(-1)).toMatchObject({ cwd: '/workspace' }) + const commandOptions = fake.commandOptions.at(-1) + expect(commandOptions).toMatchObject({ cwd: '/workspace' }) + expect(commandOptions?.envs?.HOME).toMatch(/^\/\.dsh-e2b-control-/) + expect(commandOptions?.envs).toEqual({ HOME: commandOptions?.envs?.HOME }) expect((ctx.e2b)).toBeDefined() }) @@ -867,6 +887,29 @@ describe('E2B subprocess terminal service', () => { expect(fake.handle.disconnects).toBe(1) }) + it('owns and cancels terminal state-directory creation during disposal', async () => { + const fake = new FakeTerminalSandbox() + fake.makeDirRequest = async (signal) => { + await new Promise((_resolve, reject) => { + const onAbort = (): void => { + const reason: unknown = signal?.reason + reject(reason instanceof Error ? reason : new Error(String(reason))) + } + signal?.addEventListener('abort', onAbort, { once: true }) + if (signal?.aborted === true) onAbort() + }) + } + const { ctx, fiber } = await service(fake) + const spawning = ctx.subprocess.spawnTerminal(spec()) + const rejected = expect(spawning).rejects.toThrow('service disposed during terminal setup') + await vi.waitFor(() => { expect(fake.directories.some(path => path.includes('/terminals/'))).toBe(true) }) + + await fiber.dispose() + await rejected + expect(fake.removed.some(path => path.includes('/terminals/'))).toBe(true) + expect(fake.createOptions).toBeUndefined() + }) + it('retains failed terminal setup cleanup for disposal retry', async () => { const fake = new FakeTerminalSandbox() fake.sendError = new Error('bootstrap failed') From 65b345c96a77a40628ad6bbdee9c065894d03957 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 16:17:56 +0800 Subject: [PATCH 090/108] test(e2b): tolerate real-provider runtime latency --- examples/headless-agent/tests/fixtures/e2b/e2b/cordis.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/examples/headless-agent/tests/fixtures/e2b/e2b/cordis.yml b/examples/headless-agent/tests/fixtures/e2b/e2b/cordis.yml index 93bb6b65e5..d2b73c1077 100644 --- a/examples/headless-agent/tests/fixtures/e2b/e2b/cordis.yml +++ b/examples/headless-agent/tests/fixtures/e2b/e2b/cordis.yml @@ -63,7 +63,7 @@ name: '@deepseek-ai/dsh-code-runtime-subprocess' config: computeMs: 500 - maxWallMs: 5000 + maxWallMs: 15000 maxOutputBytes: 4096 maxOldGenerationSizeMb: 128 maxFrameBytes: 32768 From 917a7493f7ba0e6e2ab59a458d08566aa14669cc Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 19:07:11 +0800 Subject: [PATCH 091/108] refactor(e2b): keep provider layer substrate-only --- ...rtable-execution-world-consumers.i18n.yaml | 4 +- ...7-28-portable-execution-world-consumers.md | 26 +++ ...8-portable-execution-world-consumers.zh.md | 44 ++++- ...026-07-27-e2b-remote-runtime-poc.i18n.yaml | 6 - .../2026-07-27-e2b-remote-runtime-poc.md | 65 -------- .../2026-07-27-e2b-remote-runtime-poc.zh.md | 65 -------- .../tests/fixtures/e2b/e2b/bin.ts | 58 +++---- packages/e2b/README.i18n.yaml | 4 +- packages/e2b/README.md | 2 +- packages/e2b/README.zh.md | 2 +- packages/e2b/e2b/tests/composition.e2e.ts | 4 +- packages/e2b/fs-e2b/README.i18n.yaml | 4 +- packages/e2b/fs-e2b/README.md | 6 +- packages/e2b/fs-e2b/README.zh.md | 6 +- packages/e2b/fs-e2b/src/index.ts | 129 +++++---------- packages/e2b/fs-e2b/src/source-reader.ts | 63 -------- packages/e2b/fs-e2b/tests/filesystem.spec.ts | 152 +++++++----------- packages/e2b/subprocess-e2b/README.i18n.yaml | 4 +- packages/e2b/subprocess-e2b/README.md | 6 +- packages/e2b/subprocess-e2b/README.zh.md | 6 +- .../e2b/subprocess-e2b/src/environment.ts | 2 + packages/e2b/subprocess-e2b/src/index.ts | 16 +- packages/e2b/subprocess-e2b/src/process.ts | 4 +- packages/e2b/subprocess-e2b/src/terminal.ts | 37 +++-- .../e2b/subprocess-e2b/tests/terminal.spec.ts | 135 +++++----------- packages/fs/README.i18n.yaml | 4 +- packages/fs/README.md | 2 +- packages/fs/README.zh.md | 2 +- 28 files changed, 280 insertions(+), 578 deletions(-) delete mode 100644 .agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.i18n.yaml delete mode 100644 .agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md delete mode 100644 .agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.zh.md delete mode 100644 packages/e2b/fs-e2b/src/source-reader.ts diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml index 06fd47f329..fc3b5b012f 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md -2026-07-28-portable-execution-world-consumers.md: 402aa580255a5bd6aa0d046e3bcc16f712da520d -2026-07-28-portable-execution-world-consumers.zh.md: 26d493a79f8adb9e729ff69b0673f2f2775868b0 +2026-07-28-portable-execution-world-consumers.md: 250445cfed6a4fe2dd188562813d586df29992c1 +2026-07-28-portable-execution-world-consumers.zh.md: 361c4de4e813aa7f57a7eb5acc75087b2630db43 diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md index 402aa58025..4c593604f4 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md @@ -8,6 +8,8 @@ English | [中文](2026-07-28-portable-execution-world-consumers.zh.md) The filesystem and subprocess seams made file and ordinary process access replaceable, but PTY and LSP still reached host Node APIs directly. A remote execution provider therefore appeared to need separate PTY and LSP packages even though their domain behavior did not change. Those packages would be shallow adapters: each would duplicate an existing consumer merely to replace its file and process operations. +A remote coding world is useful only when file operations, commands, terminals, language servers, and model-written programs share one sandbox identity. Moving the complete harness into that sandbox would also entangle provider experimentation with plugin loading, credentials, model transport, session durability, supervision, and deployment. + Ordinary pipes do not cover one requirement. A persistent terminal needs PTY allocation, foreground-process-group inspection and signalling, and cleanup of the complete terminal session. Pretending those operations can be rebuilt in `dsh-pty-local` from an ordinary `spawn()` handle would either leak provider internals or weaken its lifecycle contract. ## Decision @@ -24,10 +26,26 @@ Generic consumers use that execution world: - `dsh-lsp-local` reads and contains source through `ctx.fs`, resolves and launches language servers through `ctx.subprocess`, and carries provider-owned file URIs through initialization and result rendering. One provider-lifetime signal aborts filesystem and protocol work during disposal, including workspace lookup before queue ownership; its JSON-RPC, pooling, synchronization, and normalization stay unchanged. - `dsh-pty-local` maps persistent-shell semantics onto `ctx.subprocess.spawnTerminal()`. The local `node-pty` and process-inspection implementation moves into `dsh-subprocess-local`; another subprocess provider supplies the same primitive. `danger-full-access` needs no `ctx.sandbox`; a confined mode requires a same-world sandbox provider and fails before spawn when none is mounted. Prompt and silence evidence collected during asynchronous pre-write inspection is discarded when the provider write begins. Cancellation retains the send reservation while an in-flight write settles and then signals the foreground group, so late bytes or the signal cannot target a successor; an in-flight readiness poll cannot release that reservation, and a rejected write sends no signal. The absolute deadline remains armed throughout cancellation. A signal failure becomes terminal transport failure. Completion of a stale inspection resumes polling for the current send. Startup cancellation begins terminal rollback without waiting for a stalled readiness or signalling call. Close rejects new public signals and delegates provider-observable session quiescence to the handle's awaited termination operation. +## E2B POC boundary + +The opt-in E2B realization has exactly three provider-specific packages under `packages/e2b/`: `dsh-e2b` creates or reconnects one sandbox and owns kill/pause/leave disposal, `dsh-fs-e2b` implements `ctx.fs`, and `dsh-subprocess-e2b` implements `ctx.subprocess` over E2B Commands, PTYs, and remote Linux process groups. The two adapters obtain the sole sandbox identity from the owner and never create private sandboxes. + +E2B owns the mutable filesystem, managed command and Bash processes, terminal allocation and terminal-session groups, language-server processes and source reads, subprocess Code Runtime processes, and adapter-private files under `.dsh-e2b`. The host owns Cordis and plugin objects, the agent loop, agent/session/goal state, session logs and persistence, LLM calls, prompts and tools, authority, skills, subagent orchestration, PTY buffers and readiness, LSP protocol state, Code Runtime program/binding/output policy, and E2B SDK/network buffers. The overlay neither uploads nor synchronizes the host workspace. + +The adapters retain only substrate mechanics. Filesystem canonicalization crosses the SDK's decoded command transport as strict base64-encoded NUL framing; streamed reads leave byte ceilings with consumers. Subprocess command output and environment snapshots use ASCII/base64 where SDK chunk decoding would otherwise lose bytes, while private control shells isolate profiles and later launches blank discovered credential-shaped names. Process and terminal cleanup uses remote groups and proves quiescence before settlement. + +Retaining a sandbox preserves remote files and unmanaged state only. Reconnect does not reconstruct host process or terminal handles, protocol connections, pending calls, output cursors, timers, or locks. The POC adds no session-persistence backend, template builder, volume, snapshot, network-policy layer, sandbox catalog, workspace synchronization, durable remote handles, or whole-harness execution. + +## Verification + +Focused package suites pin sandbox lifecycle, canonical path framing, filesystem metadata and atomic versions, subprocess publication/rollback, terminal text I/O and session cleanup, output limits, cancellation, disposal, and invariant registration. A credential-gated Loader composition exercises the same three-package provider through source imports and built exports, including FS/Bash visibility, post-rename version reread plus guarded edit, hostile login profiles, byte-split UTF-8 output, process and terminal cleanup, LSP document bounds, Code Runtime bindings/limits/cleanup, host-workspace isolation, and final sandbox deletion. + ## Alternatives considered **Keep one PTY and LSP package per remote provider.** Rejected because provider mechanics would be repeated above the existing seams. The deletion test exposes the problem: deleting those adapters should not scatter domain behavior into the remote provider; the generic consumers already own it. +**Create a separate sandbox per capability or tool.** Rejected because file and process operations would not share identity or state, defeating the coding use case and multiplying lifecycle owners. + **Model a terminal as an ordinary piped subprocess.** Rejected because pipes cannot allocate a controlling terminal, resolve the current foreground process group, or prove complete terminal-session cleanup. One terminal primitive is smaller and more honest than exposing substrate-specific escape hatches. **Move PTY readiness and session policy into the subprocess service.** Rejected because those are persistent-terminal consumer semantics, not OS process mechanics. A subprocess provider owns what only its substrate can do; `dsh-pty-local` owns what a Harness terminal means. @@ -38,6 +56,12 @@ Generic consumers use that execution world: **Run the whole harness inside the remote environment.** Rejected as a different deployment model. Making execution capabilities portable does not move model calls, session state, plugin state, or the agent loop. +**Put every provider operation in one shared owner package.** Rejected because sandbox identity and lifecycle are the owner's only concerns. Filesystem and subprocess retain distinct contracts, tests, and consumers without turning the owner into a capability grab bag. + +**Implement remote filesystem operations only through shell commands.** Rejected because that discards structured filesystem identity, errors, streaming, version guards, and atomic mutation semantics already consumed by the file tools. + +**Add a generic distributed-runtime abstraction or reconnect live handles.** Rejected because the existing capability seams carry the demonstrated contracts, while remote identity alone cannot reconstruct callbacks, pending promises, authority, protocol state, or output cursors. A new layer would speculate about persistence and synchronization beyond the POC. + ## Consequences A remote execution provider implements only its shared sandbox owner plus filesystem and subprocess adapters. Bash, PTY, and LSP compose above them, so fixes to those capabilities remain provider-neutral. @@ -45,3 +69,5 @@ A remote execution provider implements only its shared sandbox owner plus filesy The fundamental interfaces are wider, and a filesystem/subprocess pair must agree on one execution world. The added operations are limited to facts and lifecycle mechanics that current generic consumers require; model schemas, protocol framing, readiness policy, and presentation do not leak into the providers. The local implementation absorbs `node-pty` and platform process inspection because it owns local terminal mechanics. This moves code without weakening terminal teardown: disposal sweeps descendants before and after terminating the top-level shell, waits for exact PID-identity-fenced descendants retained during foreground inspection, and retains Linux session members that survive top-level exit. macOS cannot enumerate a POSIX session after its leader exits, so a child that reparents between inspection snapshots remains an explicit local-provider limitation rather than a reason to move process mechanics back into the PTY consumer. + +The E2B composition demonstrates that a shared sandbox owner plus filesystem and subprocess adapters are sufficient to move the mutable coding world off-host while leaving higher capabilities provider-neutral. Its POC limits remain explicit: the SDK retains complete command transport in host memory, remote startup cannot publish a PID synchronously, exact terminal stdin-wait and independent signal facts are unavailable, numeric PID/PGID operations are not identity-fenced, the initial environment probe cannot hide unknown sandbox-default secrets from already-running same-UID processes, retained artifacts accumulate, and escaped processes or reconnect state are not recovered. These are provider constraints, not justification for compatibility shims or more E2B packages. diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md index 26d493a79f..361c4de4e8 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md @@ -6,7 +6,9 @@ Status: implemented ## 问题 -文件系统与进程管理 seam 使文件访问和普通进程访问具备可替换性,但 PTY 和 LSP 仍直接调用宿主 Node API。因此,即使领域行为没有变化,远程执行提供方看起来仍需要独立的 PTY 与 LSP 包(package)。这些包只会成为浅层适配器:每个包都仅为替换文件与进程操作而复制一个现有消费方。 +文件系统与进程管理 seam 使文件访问和普通进程访问具备可替换性,但若干上层能力仍直接调用宿主 Node API。因此,即使领域行为没有变化,远程执行提供方看起来仍需要独立的 PTY、LSP 与代码运行时包(package)。这些包只会成为浅层适配器:每个包都仅为替换文件与进程操作而复制一个现有消费方。 + +只有文件操作、命令、终端、语言服务器和模型编写的程序共享同一个沙箱身份时,远程编码世界才有用。若把完整 harness 移入该沙箱,还会把提供方实验与插件加载、凭据、模型传输、会话持久性、监督和部署纠缠在一起。 普通管道无法满足其中一项要求。持久终端需要分配 PTY、检查前台进程组并发送信号,以及清理完整的终端会话。如果假设可以在 `dsh-pty-local` 中基于普通 `spawn()` 句柄重建这些操作,最终不是泄漏提供方内部细节,就是削弱其生命周期契约。 @@ -16,16 +18,36 @@ Status: implemented 文件系统接口负责其他能力需要的路径事实,同时不公开其不透明目标身份:规范化进程路径、规范化 `file:` URI 和包含关系。现有完整文本与流式文本操作仍归文件系统负责;协议消费方在消费流时执行各自的保留上限。 -进程管理接口负责可执行文件查找与进程原语:以原始或收集模式 spawn 普通进程,以及 `spawnTerminal()`。终端操作是一项深层原语,其句柄负责文本 I/O、前台进程组、信号发送,以及一项须等待的 TERM→KILL 操作;该操作会结算所有在途句柄调用,并使提供方仍可观察到的每个会话成员完全停稳。其信号只取消分配;句柄一经发布,便负责自身生命周期。提示符检测、空闲推断、scrollback、沙箱策略和所有者生命周期仍由 PTY 消费方负责。 +进程管理接口负责进程运行坐标与原语:规范化 cwd、私有运行时存储、可执行文件查找、以原始或收集模式 spawn 普通进程,以及 `spawnTerminal()`。终端操作是一项深层原语,其句柄负责文本 I/O、前台进程组、信号发送,以及一项须等待的 TERM→KILL 操作;该操作会结算所有在途句柄调用,并使整个会话完全停稳。其信号只取消分配;句柄一经发布,便负责自身生命周期。提示符检测、空闲推断、scrollback、沙箱策略和所有者生命周期仍由 PTY 消费方负责。 通用消费方使用该执行世界: - `dsh-bash-local` 继续把 Bash 语义映射到普通的 `ctx.subprocess.spawn()`。 - `dsh-lsp-local` 通过 `ctx.fs` 读取源文件并验证包含关系,通过 `ctx.subprocess` 解析和启动语言服务器,并让由提供方负责的文件 URI 贯穿初始化与结果渲染。一个提供方生命周期信号会在资源释放期间中止文件系统与协议操作,包括取得队列所有权之前的工作区查找;其 JSON-RPC、池化、同步和规范化保持不变。 -- `dsh-pty-local` 把持久 shell 语义映射到 `ctx.subprocess.spawnTerminal()`。本地 `node-pty` 与进程检查实现移入 `dsh-subprocess-local`;其他进程管理提供方则提供相同原语。`danger-full-access` 不需要 `ctx.sandbox`;受限模式要求同一执行世界中存在沙箱提供方,未挂载时会在 spawn 前失败。提供方开始写入时,系统会丢弃异步写入前检查期间收集的提示符与静默证据。取消会在在途写入结算期间保留发送预留,随后向前台进程组发送信号,因此延迟字节和该信号都无法落到后续发送;在途就绪检查无法释放该预留,写入被拒绝时也不会发送信号。绝对截止时间会在整个取消期间保持启用。信号发送失败会成为终结性传输失败。陈旧检查完成后,会针对当前发送恢复轮询。启动取消会立即开始终端回滚,而不等待停滞的就绪检查或信号发送调用。关闭操作会拒绝新的公开信号,并把提供方可观察会话成员的完全停稳委托给句柄上须等待的终止操作。 +- `dsh-pty-local` 把持久 shell 语义映射到 `ctx.subprocess.spawnTerminal()`。本地 `node-pty` 与进程检查实现移入 `dsh-subprocess-local`;其他进程管理提供方则提供相同原语。提供方开始写入时,系统会丢弃异步写入前检查期间收集的提示符与静默证据。取消会在在途写入结算期间保留发送预留,随后向前台进程组发送信号,因此延迟字节和该信号都无法落到后续发送;写入被拒绝时不会发送信号。绝对截止时间会在整个取消期间保持启用。信号发送失败会成为终结性传输失败。陈旧检查完成后,会针对当前发送恢复轮询。关闭操作会拒绝新的公开信号,并把完整会话的完全停稳委托给句柄上须等待的终止操作。 +- `dsh-code-runtime-subprocess` 通过 `ctx.fs` 物化无依赖 runner,并通过 `ctx.subprocess` 启动它,从而在本地或远程执行世界中保留代码运行时的绑定与输出契约。固定 runner 是位于 `ctx.subprocess.runtimeRoot` 下的适配器自有基础设施,因此其写入携带显式 `danger-full-access` 策略,而不继承面向模型的文件系统模式。它通过非插件子路径 `dsh-code-runtime-worker/runtime-host` 共享宿主侧 worker 机制,而不是复制这些机制。准备阶段让同一个生命周期信号贯穿文件系统解析、物化和可执行文件查找,使资源释放能够中止停滞的提供方操作。受堆上限约束的 worker 会在传输前拒绝过大的绑定帧;每个外层转发环节都会在转发前执行相同的上限检查;原始子进程管道承载以换行符分隔的 UTF-8 JSON,无需冗余的 base64 表示;launcher 会在回收 controller 前发布已接纳的终态帧,使继承 controller 管道的后代进程无法阻止完成;宿主仍会等待进程组完全停稳。 + +`dsh-code-runtime-worker` 仍是独立实现。它是较小的进程内后端,可用于无法假定已安装 Node 可执行文件的单文件分发。远程文件系统/进程组合选择 `dsh-code-runtime-subprocess`;它们不需要提供方专用的代码运行时包。 + +## E2B POC 边界 + +可选启用的 E2B 实现在 `packages/e2b/` 下恰好只有三个提供方专用包:`dsh-e2b` 创建或重新连接一个沙箱,并负责 kill、pause 或 leave 资源释放;`dsh-fs-e2b` 实现 `ctx.fs`;`dsh-subprocess-e2b` 基于 E2B Commands、PTY 和远程 Linux 进程组实现 `ctx.subprocess`。两个适配器都从所有者取得唯一的沙箱身份,绝不创建私有沙箱。 + +E2B 负责可变文件系统、受管命令与 Bash 进程、终端分配与终端会话组、语言服务器进程与源文件读取、子进程代码运行时进程,以及 `.dsh-e2b` 下的适配器私有文件。宿主负责 Cordis 与插件对象、agent loop(智能体循环)、agent(智能体)状态、会话状态与目标状态、会话日志与持久化、LLM(大语言模型)调用、提示词与工具、权限、skill(技能)、subagent 编排、PTY 缓冲区与就绪状态、LSP 协议状态、代码运行时程序/绑定/输出策略,以及 E2B SDK/网络缓冲区。该叠加层既不上传,也不同步宿主工作区。 + +适配器只保留执行基底机制。文件系统规范化以严格的 base64 加 NUL 分帧穿过 SDK 已解码的命令传输;流式读取把字节上限留给消费方执行。进程管理命令输出与环境快照采用 ASCII/base64,避免 SDK 分片解码丢失字节;私有控制 shell 隔离 profile,后续启动会把已发现且名称呈凭据特征的环境变量置空。进程与终端清理使用远程进程组,并在结算前证明完全停稳。 + +保留沙箱只会保留远程文件与非托管状态。重新连接不会重建宿主进程或终端句柄、协议连接、待处理调用、输出游标、计时器或锁。该 POC 不会新增会话持久化后端、模板构建器、卷、快照、网络策略层、沙箱目录、工作区同步、持久远程句柄,也不会在其中运行整个 harness。 + +## 验证 + +聚焦的包测试套件锁定了沙箱生命周期、规范化路径分帧、文件系统元数据与原子版本、进程管理发布/回滚、终端文本 I/O 与会话清理、输出上限、取消、资源释放和不变式注册。一项受凭据门控的 Loader 组合通过源代码导入与构建后导出运行同一套三包提供方组合,其中包括 FS/Bash 可见性、重命名后的版本重读与带保护编辑、恶意登录 profile、跨字节边界拆分的 UTF-8 输出、进程与终端清理、LSP 文档上限、代码运行时绑定/上限/清理、宿主工作区隔离,以及最终沙箱删除。 + ## 考虑过的替代方案 -**为每个远程提供方分别保留 PTY 与 LSP 包。** 不予采纳,因为这会在现有 seam 之上重复实现提供方机制。删除检验揭示了这一问题:删除这些适配器不应使领域行为散落到远程提供方中;通用消费方本已负责这些行为。 +**为每个远程提供方分别保留 PTY、LSP 和代码运行时包。** 不予采纳,因为这会在现有 seam 之上重复实现提供方机制。删除检验揭示了这一问题:删除这些适配器不应使领域行为散落到远程提供方中;通用消费方本已负责这些行为。 + +**为每项能力或工具创建独立沙箱。** 不予采纳,因为文件与进程操作将无法共享身份或状态,从而破坏编码用例,并增加生命周期所有者的数量。 **把终端建模为普通的管道子进程。** 不予采纳,因为管道无法分配控制终端、确定当前前台进程组或证明完整终端会话已清理。一项终端原语比公开特定于执行基底的逃生口更小,也更能如实表达契约。 @@ -35,12 +57,22 @@ Status: implemented **在文件系统 seam 中新增稳定的有界读取原语。** 不予采纳,因为只有 LSP 需要完整文档字节上限,而它可以在消费现有文本流时执行该上限。第二项原语会迫使每个提供方实现稳定句柄和不跟随符号链接的机制,远程提供方甚至需要辅助协议,却没有已观察到的并发替换缺陷。 -**在远程环境中运行整个 harness。** 不予采纳,因为这是另一种部署模型。让执行能力可移植,并不意味着移动模型调用、会话状态、插件状态或 agent loop(智能体循环)。 +**删除 worker 线程代码运行时。** 不予采纳,因为可移植性不会消除其当前部署需求。进程管理后端需要 Node 可执行文件和文件系统物化,而 worker 后端两者都不需要,并且仍是受支持的单进程路径。 + +**在远程环境中运行整个 harness。** 不予采纳,因为这是另一种部署模型。让执行能力可移植,并不意味着移动模型调用、会话状态、插件状态或 agent loop。 + +**把所有提供方操作都放进一个共享所有者包。** 不予采纳,因为沙箱身份与生命周期是所有者唯一的关注点。文件系统与进程管理保留各自独立的契约、测试和消费方,同时避免把所有者变成无边界的能力集合。 + +**只通过 shell 命令实现远程文件系统操作。** 不予采纳,因为这会丢弃现有文件工具已消费的结构化文件系统身份、错误、流式输出、版本保护和原子变更语义。 + +**新增通用分布式运行时抽象,或重新连接活跃句柄。** 不予采纳,因为现有能力 seam 已承载经证实的契约,而仅凭远程身份无法重建回调、待处理 promise、权限、协议状态或输出游标。新增一层只会推测 POC 边界之外的持久化与同步问题。 ## 后果 -远程执行提供方只需实现共享沙箱所有者,以及文件系统与进程管理适配器。Bash、PTY 与 LSP 组合在这些适配器之上,因此对这些能力的修复仍与提供方无关。 +远程执行提供方只需实现共享沙箱所有者,以及文件系统与进程管理适配器。Bash、PTY、LSP 和基于进程管理的代码运行时组合在这些适配器之上,因此对这些能力的修复仍与提供方无关。 基础接口更宽,一对文件系统/进程管理提供方必须在同一个执行世界上保持一致。新增操作仅限当前通用消费方所需的事实与生命周期机制;模型 schema、协议分帧、就绪策略和呈现不会渗入提供方。 本地实现承接 `node-pty` 和平台进程检查,因为它负责本地终端机制。这种代码迁移不会削弱终端拆卸:dispose(资源释放)会在终止顶层 shell 前后清理后代进程,等待前台检查期间保留下来且受精确 PID 身份围栏保护的后代进程,并继续追踪在顶层进程退出后仍存活的 Linux 会话成员。macOS 无法在 POSIX 会话 leader 退出后枚举该会话,因此在两次检查快照之间重新设定父进程的子进程仍是明确的本地提供方限制,而不是把进程机制移回 PTY 消费方的理由。 + +E2B 组合证明,共享沙箱所有者加上文件系统与进程管理适配器,就足以在保持上层能力与提供方无关的同时,把可变编码世界移出宿主。其 POC 限制仍明确在案:SDK 会把完整命令传输内容保留在宿主内存中;远程启动无法同步发布 PID;无法获得精确的终端 stdin 等待状态与独立信号事实;基于数值 PID/PGID 的操作没有身份围栏;初始环境探测无法向已在运行的同 UID 进程隐藏未知的沙箱默认 secret;保留的产物会累积;也不会恢复逃逸进程或重连状态。这些是提供方限制,不是引入兼容性 shim 或更多 E2B 包的理由。 diff --git a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.i18n.yaml b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.i18n.yaml deleted file mode 100644 index 2c3db3fad4..0000000000 --- a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.i18n.yaml +++ /dev/null @@ -1,6 +0,0 @@ -# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each -# side as of the last confirmed-consistent state. Both languages carry equal authority; -# after editing either side, bring the other along and re-record with: -# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md -2026-07-27-e2b-remote-runtime-poc.md: 79c5bfe2dfa11cd695c7134e6ceffb639b449688 -2026-07-27-e2b-remote-runtime-poc.zh.md: 729a10ec7efa4dd55f041dc804d2252102b13c01 diff --git a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md deleted file mode 100644 index 79c5bfe2df..0000000000 --- a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md +++ /dev/null @@ -1,65 +0,0 @@ -# Agent Note: Shared E2B remote runtime POC - -Status: implemented - -English | [中文](2026-07-27-e2b-remote-runtime-poc.zh.md) - -## Problem - -A remote coding-agent backend is useful only when filesystem operations, one-shot commands, persistent terminals, language servers, and model-written programs observe one coherent world. Attaching E2B independently at individual tools would let those capabilities address different sandboxes, while retaining host PTY, LSP, or worker backends would split state across machines even when the cwd strings match. - -Moving the complete harness into a remote VM would unify that state but also couple provider experimentation to plugin loading, credentials, model transport, agent/session durability, supervision, and deployment. The POC needs to test the existing capability boundaries without taking on those independent concerns. - -## Decision - -The E2B integration is an opt-in provider-composition POC. Its three E2B-specific packages live together under `packages/e2b/`: - -- `@deepseek-ai/dsh-e2b` creates or reconnects one secure E2B sandbox, creates its working and private runtime directories, and owns kill/pause/leave disposal. -- `@deepseek-ai/dsh-fs-e2b` implements `ctx.fs` over that sandbox's Filesystem API. -- `@deepseek-ai/dsh-subprocess-e2b` implements `ctx.subprocess` over E2B Commands, byte PTYs, and remote Linux process groups. - -The higher capabilities use provider-neutral implementations. `dsh-bash-local` delegates command mechanics to `ctx.subprocess`; `dsh-pty-local` delegates terminal allocation and signalling to `ctx.subprocess.spawnTerminal()`; `dsh-lsp-local` reads through `ctx.fs` and launches through `ctx.subprocess`; `dsh-code-runtime-subprocess` materializes its runner through `ctx.fs` and starts it through `ctx.subprocess`. The [portable execution-world decision](../architecture/2026-07-28-portable-execution-world-consumers.md) owns those generic interfaces and consumers. - -The E2B owner is the sole source of sandbox identity. Its two adapters never create private sandboxes, so filesystem tools, Bash, interactive shells, language servers, and code workers share one remote cwd, process namespace, and adapter-private directory while preserving the existing capability interfaces, generic implementations, model-facing tools, and agent loop. - -## POC boundary - -E2B owns the mutable filesystem, managed command and Bash processes, terminal allocation and terminal-session process groups, language-server processes and source reads, the Code Runtime launcher, controller, and worker, and adapter-private files under `.dsh-e2b`. - -The host owns Cordis and plugin objects, the agent loop, agent/session/goal state, session logs and persistence, LLM calls, prompts and tools, authority decisions, skills, subagent orchestration, PTY buffers and readiness state, LSP JSON-RPC ids/queues/protocol state, Code Runtime type stripping/output accounting/binding dispatch, and E2B SDK/network orchestration. The overlay does not upload, mount, or synchronize the host workspace; identical cwd strings name independent host and remote directories. - -The fundamental adapters carry the substrate-specific mechanics. Adapter-internal E2B command and PTY login shells use a fresh randomized root-level `HOME`; `dsh-subprocess-e2b` also gives them empty overrides for scrubbed credential names before user profiles run. The subprocess adapter consumes E2B's byte PTY callback, transports environment snapshots and command bytes as ASCII/base64 across decoded SDK callbacks, and uses one cancellation controller plus one retryable group-cleanup transaction that reports success only after proving quiescence. `dsh-fs-e2b` performs bounded source reads through a dependency-free helper that walks no-follow directory descriptors beneath the canonical target. Generic Code Runtime keeps its controller/worker protocol on validated newline-delimited UTF-8 JSON and kills the provider-owned process group before inherited pipes drain. Generic LSP uses byte-faithful UTF-8 JSON over command pipes. - -Retaining a sandbox preserves remote files and unmanaged remote state only. Reconnect does not reconstruct host PTY sessions, buffers, process handles, LSP connections or requests, code workers, binding calls, timers, output cursors, or locks. Managed groups terminate and join when their provider disposes before the shared owner pauses, leaves, or kills the sandbox. - -The POC has no session-persistence backend, template builder, volume, snapshot, network-policy layer, sandbox catalog, workspace synchronization, durable remote handles, or whole-harness execution. - -## Verification - -Focused package suites pin owner lifecycle cleanup, filesystem paths/containment/bounded descriptor reads and commit metadata, subprocess executable lookup/process groups/publication rollback, terminal byte I/O/signal identity/default-environment scrubbing/session cleanup, output limits, abort ordering, disposal to quiescence, and package-owned invariant registrations. The generic PTY, LSP, and subprocess Code Runtime suites pin their provider-neutral readiness, cross-namespace `processId`, binding bridge, descriptor isolation, hostile traffic, and worker/descendant cleanup behavior. - -A credential-gated Loader composition creates real E2B sandboxes and exercises FS-to-Bash and Bash-to-FS visibility, real-directory runtime-state setup, process-publication rollback, byte-split UTF-8 command output, bounded raw spill and inherited-output draining, hostile command and PTY login-profile isolation, default-secret scrubbing, stale-interrupt identity, and process-tree cleanup, parent-swap-safe bounded LSP source reads, Code Runtime host bindings, descriptor-isolated output accounting, descendant-held pipe cleanup, wall timeout, abort, runner cleanup, host-workspace isolation, and final sandbox deletion. The same composition runs through source imports and built package exports. - -## Alternatives considered - -**A separate E2B sandbox per capability or tool** — rejected because file and command operations would not share identity or state, defeating the coding-agent use case and multiplying lifecycle ownership. - -**Run the entire harness process inside E2B** — rejected because it changes deployment, credential flow, model transport, session durability, plugin loading, and supervision at once. Those questions are independent of proving the provider seams. - -**Put every E2B operation in the shared owner package** — rejected because lifecycle identity is the owner's only concern. Filesystem and subprocess retain separate provider contracts, tests, and consumers; the owner exposes one shared SDK handle without becoming a capability grab bag. - -**Implement filesystem operations through shell commands only** — rejected because that bypasses `ctx.fs` identity, structured errors, version guards, streaming reads, and atomic mutation semantics already consumed by the file tools. - -**Keep E2B-specific PTY, LSP, and Code Runtime packages** — rejected because their domain behavior does not vary with E2B. They were shallow adapters that duplicated existing consumers to replace filesystem and process operations; moving those operations behind the fundamental seams gives every provider one implementation of readiness, protocol, binding, and presentation behavior. - -**Call E2B Filesystem, Commands, or PTY APIs directly from higher capabilities** — rejected because it bypasses the `ctx.fs` and `ctx.subprocess` contracts, duplicates execution-world policy in each consumer, and forks model-facing behavior. The subprocess seam includes the irreducible terminal primitive because ordinary pipes cannot supply foreground groups or whole-session cleanup. - -**Add a generic distributed-runtime abstraction first** — rejected because the existing capability seams already carry the required contracts. A new cross-cutting interface would speculate about persistence, synchronization, and reconnect semantics beyond the POC. - -**Restore live capability handles after `sandboxId` reconnect** — rejected because remote identity alone cannot reconstruct host callbacks, pending promises, authority, protocol state, or output cursors. Claiming continuity would make stale remote processes appear managed when they are not. - -## Consequences - -The three-package composition demonstrates that filesystem and subprocess are the sufficient provider seams for moving an agent's mutable coding world off-host without changing the loop, higher capability implementations, or model-facing tool packages. Fixes to Bash, PTY, LSP, and Code Runtime remain provider-neutral. `sandboxId` plus pause/leave permits manual remote-file retention for experiments, while kill remains the demo's cleanup policy. - -The adapters are not interchangeable with local backends for every consumer: remote startup cannot synchronously expose a PID, E2B retains the complete base64 command transport in SDK memory, exact terminal stdin-wait inspection is unavailable, E2B supplies no independent signal fact, and reconnect cannot restore handles or protocol state. The adapter reports only its own requested TERM/KILL as signals and preserves every unrequested SDK exit as an exit code. Remote process/spill artifacts accumulate in a retained sandbox, Code programs share a JavaScript realm with Node worker internals, and a process that deliberately escapes a managed process group or terminal session does not become reconnectable or owned. These gaps remain documented POC constraints rather than compatibility shims or new cross-cutting abstractions. diff --git a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.zh.md b/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.zh.md deleted file mode 100644 index 729a10ec7e..0000000000 --- a/.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.zh.md +++ /dev/null @@ -1,65 +0,0 @@ -# Agent Note: 共享 E2B 远程运行时 POC - -Status: implemented - -[English](2026-07-27-e2b-remote-runtime-poc.md) | 中文 - -## 问题 - -远程 coding agent(智能体)后端只有在文件系统操作、一次性命令、持久终端、语言服务器和模型编写的程序观察到同一个一致环境时才有用。若在各工具上分别接入 E2B,这些功能可能访问不同的沙箱;即使 cwd 字符串相同,保留宿主 PTY、LSP 或 worker 后端也会让状态分散在不同机器上。 - -把完整 harness 迁入远程 VM 可以统一这些状态,但也会把提供方实验与插件加载、凭据、模型传输、agent/会话持久性、监管和部署耦合在一起。这个 POC 只需测试现有功能边界,不应把这些彼此独立的问题纳入范围。 - -## 决策 - -E2B 集成是一个选择性启用的提供方组合 POC。它的 3 个 E2B 专用包(package)集中位于 `packages/e2b/` 下: - -- `@deepseek-ai/dsh-e2b` 创建或重新连接一个安全的 E2B 沙箱,创建其工作目录与私有运行时目录,并拥有 kill/pause/leave 资源释放操作。 -- `@deepseek-ai/dsh-fs-e2b` 在该沙箱的 Filesystem API 之上实现 `ctx.fs`。 -- `@deepseek-ai/dsh-subprocess-e2b` 在 E2B Commands、字节 PTY 和远程 Linux 进程组之上实现 `ctx.subprocess`。 - -上层功能使用提供方无关的实现。`dsh-bash-local` 把命令机制委托给 `ctx.subprocess`;`dsh-pty-local` 把终端分配与信号发送委托给 `ctx.subprocess.spawnTerminal()`;`dsh-lsp-local` 通过 `ctx.fs` 读取,并通过 `ctx.subprocess` 启动;`dsh-code-runtime-subprocess` 通过 `ctx.fs` 物化 runner,再通过 `ctx.subprocess` 启动它。这些通用接口与消费方由[可移植执行环境决策](../architecture/2026-07-28-portable-execution-world-consumers.md)负责定义。 - -E2B 所有者是沙箱身份的唯一真源。其两个适配器绝不创建私有沙箱,因此文件系统工具、Bash、交互式 shell、语言服务器和代码 worker 会共享一个远程 cwd、进程命名空间和适配器私有目录,同时保留现有功能接口、通用实现、面向模型的工具与 agent loop(智能体循环)。 - -## POC 边界 - -E2B 拥有可变文件系统、受管命令与 Bash 进程、终端分配与终端会话进程组、语言服务器进程及源码读取、Code Runtime launcher、controller 和 worker,以及 `.dsh-e2b` 下的适配器私有文件。 - -宿主拥有 Cordis 与插件对象、agent loop、agent/会话/goal 状态、会话日志及持久化、LLM(大语言模型)调用、提示词与工具、权限决策、skill(技能)、subagent 编排、PTY 缓冲与就绪状态、LSP JSON-RPC id/队列/协议状态、Code Runtime 类型剥离/输出计量/绑定分发,以及 E2B SDK/网络编排。该 overlay 不会上传、挂载或同步宿主工作区;拼写相同的 cwd 字符串分别指向彼此独立的宿主与远程目录。 - -基础适配器承载基底专用机制。适配器内部的 E2B 命令与 PTY 登录 shell 使用位于根目录下、全新随机生成的 `HOME`;在用户 profile 脚本运行前,`dsh-subprocess-e2b` 还会为它们设置已清理凭据名称的空值覆盖。该子进程适配器消费 E2B 的字节 PTY 回调,以 ASCII/base64 跨越已解码的 SDK 回调传输环境快照与命令字节,并使用一个取消控制器与一个可重试的进程组清理事务,后者只有在证明完全停稳后才报告成功。`dsh-fs-e2b` 通过无依赖辅助程序执行有界源码读取,该程序会在规范化目标下逐级遍历不跟随符号链接的目录描述符。通用 Code Runtime 以经过验证、由换行符分隔的 UTF-8 JSON 承载 controller/worker 协议,并在继承的管道排空前终止提供方拥有的进程组。通用 LSP 通过命令管道使用字节保真的 UTF-8 JSON。 - -保留沙箱只会保存远程文件与未受管的远程状态。重新连接不会重建宿主 PTY 会话、缓冲、进程句柄、LSP 连接或请求、代码 worker、绑定调用、定时器、输出游标或锁。受管进程组会在所属提供方 dispose(资源释放)时终止并等待退出,之后共享所有者才会暂停、脱离或终止沙箱。 - -本 POC 没有会话持久化后端、模板构建器、卷、快照、网络策略层、沙箱目录、工作区同步、持久远程句柄或完整 harness 执行。 - -## 验证 - -聚焦包测试套件固定所有者生命周期清理、文件系统路径/containment/有界描述符读取与提交元数据、子进程可执行文件查找/进程组/发布回滚、终端字节 I/O/信号身份/默认环境清理/会话清理、输出上限、中止顺序、等待完全停稳的资源释放,以及包自有不变式注册。通用 PTY、LSP 与子进程 Code Runtime 测试套件固定其提供方无关的就绪判定、跨命名空间 `processId`、绑定桥接、描述符隔离、恶意通信,以及 worker/后代进程清理行为。 - -凭据门控的 Loader 组合会创建真实 E2B 沙箱,并演练 FS-to-Bash 与 Bash-to-FS 可见性、真实目录形式的运行时状态设置、进程发布回滚、按字节切分的 UTF-8 命令输出、有界原始 spill 与继承输出排空、恶意命令与 PTY 登录 profile 隔离、默认秘密清理、陈旧中断身份与进程树清理、可抵御父目录替换的有界 LSP 源码读取、Code Runtime 宿主绑定、描述符隔离的输出记账、后代进程所持管道的清理、墙钟超时、中止、runner 清理、宿主工作区隔离,以及最终删除沙箱。同一组合分别通过源代码导入与已构建包导出运行。 - -## 曾考虑的替代方案 - -**每项功能或每个工具使用独立的 E2B 沙箱。** 不予采纳,因为文件操作和命令操作将无法共享身份或状态,既违背 coding agent 用例,也会增加生命周期所有者的数量。 - -**在 E2B 内运行完整 harness 进程。** 不予采纳,因为这会同时改变部署、凭据流、模型传输、会话持久性、插件加载和监管方式。要证明提供方 seam,并不需要同时回答这些彼此独立的问题。 - -**把所有 E2B 操作放入共享所有者包。** 不予采纳,因为生命周期身份是该所有者唯一负责的事项。文件系统与进程管理各自保留独立的提供方契约、测试和消费方;所有者只公开一个共享 SDK 句柄,不会因此包揽各类功能。 - -**仅通过 shell 命令实现文件系统操作。** 不予采纳,因为这会绕过文件工具已经使用的 `ctx.fs` 身份、结构化错误、版本防护、流式读取和原子变更语义。 - -**保留 E2B 专用的 PTY、LSP 与 Code Runtime 包。** 不予采纳,因为它们的领域行为不会随 E2B 改变。这些浅层适配器为了替换文件系统与进程操作而重复现有消费方;把这些操作移到基础 seam 之后,可让所有提供方共享同一套就绪判定、协议、绑定与呈现行为实现。 - -**从上层功能直接调用 E2B Filesystem、Commands 或 PTY API。** 不予采纳,因为这会绕过 `ctx.fs` 与 `ctx.subprocess` 契约,在每个消费方中重复执行环境策略,并使面向模型的行为产生分叉。进程管理 seam 纳入不可约简的终端原语,因为普通管道无法提供前台进程组或全会话清理。 - -**先添加通用分布式运行时抽象。** 不予采纳,因为现有功能 seam 已承载所需契约。新的跨领域接口会预先假定 POC 范围之外的持久化、同步与重连语义。 - -**在 `sandboxId` 重连后恢复活动功能句柄。** 不予采纳,因为只有远程身份,无法重建宿主回调、待处理 promise、权限、协议状态或输出游标。若声称保持连续性,就会让陈旧的远程进程看似仍受管理,实际并非如此。 - -## 后果 - -这个由 3 个包组成的组合证明,文件系统与进程管理这两个提供方 seam 足以把 agent 的可变 coding 环境移出宿主,而无需改变循环、上层功能实现或面向模型的工具包。Bash、PTY、LSP 与 Code Runtime 的修复仍与提供方无关。`sandboxId` 与 `pause`/`leave` 允许实验手动保留远程文件,演示仍以 `kill` 作为清理策略。 - -这些适配器并不能对所有消费方与本地后端互换:远程启动无法同步公开 PID,E2B 会在 SDK 内存中保留完整的 base64 命令传输,无法精确检查终端 stdin 等待状态,E2B 不提供独立的信号事实,重新连接也无法恢复句柄或协议状态。适配器只会把自己请求的 TERM/KILL 报告为信号,其他未请求的 SDK 退出都保留为退出码。保留沙箱后会累积远程进程/spill 产物,模型程序与 Node worker 内部机制共享一个 JavaScript realm,有意逃离受管理进程组或终端会话的进程也不会因此变得可重新连接或由该组合管理。这些缺口作为 POC 约束明确记录,而不会引入兼容垫片或新的跨领域抽象。 diff --git a/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts b/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts index b4acfd1e5a..856a26ed78 100644 --- a/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts +++ b/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts @@ -35,9 +35,19 @@ let terminalId: Awaited>['sessionId'] | undefin try { const sandbox = await ctx.e2b.getSandbox() const fromFs = await ctx.fs.resolve('from-fs.txt') - await ctx.fs.writeText(fromFs, 'written-by-fs\n', { kind: 'createIfAbsent' }) + const written = await ctx.fs.writeText(fromFs, 'written-by-fs\n', { kind: 'createIfAbsent' }) + const reread = await ctx.fs.stat(fromFs) + if (reread?.version !== written.version) { + throw new Error(`E2B rename did not preserve version metadata: ${JSON.stringify({ written, reread })}`) + } + await ctx.fs.editText( + fromFs, + { oldString: 'written-by-fs', newString: 'written-by-fs-versioned', replaceAll: false }, + { version: reread.version }, + ) + const fsVersionGuard = true const bashRead = await ctx.bash.run(ctx.bash.resolve({ command: 'cat from-fs.txt' })) - if (bashRead.exitCode !== 0 || bashRead.stdout.text !== 'written-by-fs\n') { + if (bashRead.exitCode !== 0 || bashRead.stdout.text !== 'written-by-fs-versioned\n') { throw new Error(`E2B Bash could not read the FS write: ${JSON.stringify(bashRead)}`) } @@ -183,44 +193,8 @@ try { workspaceRoot: process.cwd(), }) - const swappedParentPath = posix.join(process.cwd(), 'swapped-parent') - const swappedSourcePath = posix.join(swappedParentPath, 'source.ts') - const swappedOutsidePath = '/tmp/dsh-e2b-lsp-outside' - await sandbox.commands.run( - `mkdir -p -- ${quoteE2BShellArg(swappedParentPath)} ${quoteE2BShellArg(swappedOutsidePath)} && printf 'const safe = true\\n' > ${quoteE2BShellArg(swappedSourcePath)} && printf 'const outside = true\\n' > ${quoteE2BShellArg(posix.join(swappedOutsidePath, 'source.ts'))}`, - ) - const remoteCommands = sandbox.commands as unknown as { - run(command: string, options?: unknown): Promise<{ exitCode: number; stdout: string; stderr: string }> - } - const runRemoteCommand = remoteCommands.run.bind(sandbox.commands) - let containmentFaultInjected = false - remoteCommands.run = async (command, options) => { - if (!containmentFaultInjected && command.includes('dsh-e2b-bounded-reader') && command.includes('swapped-parent/source.ts')) { - containmentFaultInjected = true - await runRemoteCommand( - `rm -rf -- ${quoteE2BShellArg(swappedParentPath)} && ln -s -- ${quoteE2BShellArg(swappedOutsidePath)} ${quoteE2BShellArg(swappedParentPath)}`, - ) - } - return await runRemoteCommand(command, options) - } - let lspContainment = false - try { - await ctx.lsp.query({ - operation: 'hover', - filePath: 'swapped-parent/source.ts', - position: { line: 0, character: 1 }, - workspaceRoot: process.cwd(), - }) - } catch (error: unknown) { - lspContainment = containmentFaultInjected && String(error).includes('opened safely') - if (!lspContainment) throw error - } finally { - remoteCommands.run = runRemoteCommand - } - if (!lspContainment) throw new Error('E2B LSP source swap was not rejected') - const oversizedSourcePath = posix.join(process.cwd(), 'oversized-source.ts') - await sandbox.commands.run(`head -c 4000001 /dev/zero > ${quoteE2BShellArg(oversizedSourcePath)}`) + await sandbox.commands.run(`head -c 4000001 /dev/zero | tr '\\0' x > ${quoteE2BShellArg(oversizedSourcePath)}`) let lspDocumentBound = false try { await ctx.lsp.query({ @@ -235,6 +209,10 @@ try { } if (!lspDocumentBound) throw new Error('E2B LSP accepted an oversized remote source') + const remoteCommands = sandbox.commands as unknown as { + run(command: string, options?: unknown): Promise<{ exitCode: number; stdout: string; stderr: string }> + } + const runRemoteCommand = remoteCommands.run.bind(sandbox.commands) const terminal = await ctx.pty.spawn(owner, { type: 'shell' }) terminalId = terminal.sessionId const terminalEcho = await ctx.pty.startSend(owner, terminal.sessionId, { @@ -429,6 +407,7 @@ try { process.stdout.write(`${JSON.stringify({ sandboxId: await ctx.e2b.sandboxId, bashRead: bashRead.stdout.text, + fsVersionGuard, fsRead, explicitEnvironment, splitUtf8Output, @@ -437,7 +416,6 @@ try { spill: { liveBytes: liveSpillBytes, outcome: spillOutcome, read: spillRead }, hover, definition, - lspContainment, lspDocumentBound, terminal: { motd: terminal.motd, diff --git a/packages/e2b/README.i18n.yaml b/packages/e2b/README.i18n.yaml index 9b56a3ef55..2b6a8efb69 100644 --- a/packages/e2b/README.i18n.yaml +++ b/packages/e2b/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/e2b/README.md -README.md: ef2e5ef49056e6688630e785a61c01fa53febbb1 -README.zh.md: d76b05f9b60bc471c1793decd1def1d15615c852 +README.md: cb733ba00d21070773ff9b381e4a72d9d3b9a8df +README.zh.md: 29f8e58a0cdc59098cf8d74120cb360d11cfbcb3 diff --git a/packages/e2b/README.md b/packages/e2b/README.md index ef2e5ef490..cb733ba00d 100644 --- a/packages/e2b/README.md +++ b/packages/e2b/README.md @@ -12,4 +12,4 @@ An experimental provider-composition POC that places one filesystem/process exec The existing [`dsh-bash-local`](../bash/bash-local/README.md), [`dsh-pty-local`](../pty/pty-local/README.md), [`dsh-lsp-local`](../lsp/lsp-local/README.md), and [`dsh-code-runtime-subprocess`](../code-runtime/code-runtime-subprocess/README.md) need no E2B-specific forks. They delegate every execution-world operation to `ctx.fs` and `ctx.subprocess`, so mounting the two E2B adapters places their mutable work in the same sandbox. -This boundary does not move the harness process, Cordis objects, model calls, agent/session state, session persistence, skills, higher-level protocol state, or E2B SDK buffers. The [shared-runtime decision](../../.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md) owns the POC boundary; the [portable-consumer decision](../../.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md) owns the generic composition. +This boundary does not move the harness process, Cordis objects, model calls, agent/session state, session persistence, skills, higher-level protocol state, or E2B SDK buffers. The [portable execution-world decision](../../.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md) owns both the generic composition and this POC boundary. diff --git a/packages/e2b/README.zh.md b/packages/e2b/README.zh.md index d76b05f9b6..29f8e58a0c 100644 --- a/packages/e2b/README.zh.md +++ b/packages/e2b/README.zh.md @@ -12,4 +12,4 @@ 现有的 [`dsh-bash-local`](../bash/bash-local/README.md)、[`dsh-pty-local`](../pty/pty-local/README.md)、[`dsh-lsp-local`](../lsp/lsp-local/README.md) 及 [`dsh-code-runtime-subprocess`](../code-runtime/code-runtime-subprocess/README.md) 无需 E2B 专用 fork。它们把执行环境中的所有操作委托给 `ctx.fs` 和 `ctx.subprocess`,因此挂载这两个 E2B 适配器后,它们执行的可变操作都发生在同一个沙箱内。 -该边界不会迁移 harness 进程、Cordis 对象、模型调用、agent(智能体)/会话状态、会话持久化、skill(技能)、更高层协议状态或 E2B SDK 缓冲。[共享运行时决策](../../.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md)界定 POC 边界;[可移植消费方决策](../../.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md)界定通用组合。 +该边界不会迁移 harness 进程、Cordis 对象、模型调用、agent(智能体)/会话状态、会话持久化、skill(技能)、更高层协议状态或 E2B SDK 缓冲。[可移植执行世界决策](../../.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md)同时界定通用组合和此 POC 边界。 diff --git a/packages/e2b/e2b/tests/composition.e2e.ts b/packages/e2b/e2b/tests/composition.e2e.ts index f9fcbfae8a..6600f1046d 100644 --- a/packages/e2b/e2b/tests/composition.e2e.ts +++ b/packages/e2b/e2b/tests/composition.e2e.ts @@ -160,7 +160,8 @@ describe.skipIf(!process.env.E2B_API_KEY)('E2B live Loader composition', () => { expect(stderr).toBe('') const output = JSON.parse(stdout) as Record expect(output).toMatchObject({ - bashRead: 'written-by-fs\n', + bashRead: 'written-by-fs-versioned\n', + fsVersionGuard: true, fsRead: 'written-by-bash\n', explicitEnvironment: true, splitUtf8Output: '你好', @@ -184,7 +185,6 @@ describe.skipIf(!process.env.E2B_API_KEY)('E2B live Loader composition', () => { kind: 'locations', locations: [{ range: { start: { line: 0, character: 6 }, end: { line: 0, character: 10 } } }], }, - lspContainment: true, lspDocumentBound: true, terminal: { echo: { waitReason: 'stdin_read', sessionStatus: { kind: 'running' } }, diff --git a/packages/e2b/fs-e2b/README.i18n.yaml b/packages/e2b/fs-e2b/README.i18n.yaml index 87f1c7c69b..3f756bf2f9 100644 --- a/packages/e2b/fs-e2b/README.i18n.yaml +++ b/packages/e2b/fs-e2b/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/e2b/fs-e2b/README.md -README.md: bb92b5785383e9703a382fddefcd1cff9b2644cb -README.zh.md: 57ebfdbb92799660e74078fcd0affcc8d5bc120a +README.md: 6827e16e9c45532590ee1aa986c18a353d175fdc +README.zh.md: 21b067829ea95c4581d7e89f3d225f9c90e630ef diff --git a/packages/e2b/fs-e2b/README.md b/packages/e2b/fs-e2b/README.md index bb92b57853..6827e16e9c 100644 --- a/packages/e2b/fs-e2b/README.md +++ b/packages/e2b/fs-e2b/README.md @@ -6,10 +6,9 @@ E2B implementation of the [`@deepseek-ai/dsh-fs`](../../fs/fs/README.md) provide ## Behavior -- **Remote identity and metadata** — relative paths resolve as POSIX paths against the caller cwd or `ctx.e2b.cwd`; `realpath -m` supplies canonical target identity without requiring the final file to exist. `stat`, no-follow `lstat`, and stable one-level directory listings project E2B metadata into the filesystem seam. Versions are opaque hashes of E2B metadata plus a per-write extended attribute. +- **Remote identity and metadata** — relative paths resolve as POSIX paths against the caller cwd or `ctx.e2b.cwd`; GNU `realpath -mz` supplies canonical target identity without requiring the final file to exist, and ASCII/base64 plus strict NUL framing preserves newline and multibyte paths across the decoded SDK transport. `stat`, no-follow `lstat`, and stable one-level directory listings project E2B metadata into the filesystem seam; listings reuse returned metadata and resolve symbolic-link entries sequentially. Versions are opaque hashes of E2B metadata plus a per-write extended attribute. - **Execution-world paths** — canonical targets expose absolute POSIX process paths, percent-encoded `file:` URIs, and provider-owned containment checks, so generic subprocess consumers never parse E2B target ids or apply host path rules. - **UTF-8 reads** — whole reads and streamed reads preserve cross-chunk decoding, reject invalid UTF-8, and use the seam's 8192-byte NUL sample for binary detection. The model-facing tool still owns size selection and line windowing. -- **Stable bounded reads** — a dependency-free Node helper walks directory descriptors with no-follow opens and reads one held regular-file descriptor through the byte cap. Generic LSP queries therefore reject parent swaps, non-files, invalid UTF-8, and growth past the configured document limit before server startup. - **Atomic mutations** — writes create a random sibling staging directory, change it to mode `0700` before uploading content, preserve an existing file's POSIX mode, and publish the staged file through E2B's same-filesystem atomic rename. The rename response supplies the committed version, so no fallible metadata request follows the commit point. E2B creates missing parent directories. Literal edits LF-normalize for matching, restore dominant CRLF storage, and serialize mutations per canonical target within the host process. Optional create/version guards keep the base seam's observed-state semantics. - **Failures and cancellation** — E2B not-found, permission, abort, and other controller failures map to the existing `FsError` vocabulary. Cancellation is best-effort at earlier SDK request boundaries and checked immediately before rename. The signal is not forwarded into the rename RPC, so cancellation cannot interrupt the atomic commit; a successful rename is the commit point. @@ -27,5 +26,6 @@ No direct invalidation; the named consumer owns any request-prefix changes. - **No host synchronization** — an empty E2B cwd stays empty until a tool, command, template, or external process populates it; local files are neither uploaded nor reflected back. - **Mutation coordination is host-process-local** — another harness connection or remote command can race the adapter; version guards detect only metadata changes represented by E2B. +- **Reads reopen canonical targets by path** — a concurrent remote path replacement between resolution and stream opening is not fenced by a stable file handle; no observed product defect justifies a provider-specific bounded-read protocol in this POC. - **Whole-file mutation costs remain** — overwrite diffs and literal edits read complete files into host memory, and every operation incurs E2B controller latency. -- **Custom templates must support the used Linux, Node, procfs, and envd features** — `realpath`, `chmod`, `mv`, same-filesystem POSIX rename, streaming reads, file metadata extended attributes, `/proc/self/fd`, and no-follow descriptor opens are required; unsupported templates fail rather than degrade silently. +- **Custom templates must support the used Linux/GNU and E2B filesystem features** — `realpath -mz`, `base64 -w0`, `chmod`, same-filesystem rename, streaming reads, and file metadata extended attributes are required; unsupported templates fail rather than degrade silently. diff --git a/packages/e2b/fs-e2b/README.zh.md b/packages/e2b/fs-e2b/README.zh.md index 57ebfdbb92..21b067829e 100644 --- a/packages/e2b/fs-e2b/README.zh.md +++ b/packages/e2b/fs-e2b/README.zh.md @@ -6,10 +6,9 @@ ## 行为 -- **远程身份与元数据**:相对路径以调用方 cwd 或 `ctx.e2b.cwd` 为基准,按照 POSIX 路径解析;`realpath -m` 提供规范化目标身份,且不要求最终文件存在。`stat`、不跟随链接的 `lstat` 和稳定的单层目录列表会把 E2B 元数据投影到文件系统 seam。版本是 E2B 元数据与每次写入设置的扩展属性所组成的不透明哈希。 +- **远程身份与元数据**:相对路径以调用方 cwd 或 `ctx.e2b.cwd` 为基准,按照 POSIX 路径解析;GNU `realpath -mz` 提供规范化目标身份,且不要求最终文件存在;ASCII/base64 加严格 NUL 分帧会在已解码的 SDK 传输中保留含换行符和多字节字符的路径。`stat`、不跟随链接的 `lstat` 和稳定的单层目录列表会把 E2B 元数据投影到文件系统 seam;目录列表会复用已返回的元数据,并依次解析符号链接条目。版本是 E2B 元数据与每次写入设置的扩展属性所组成的不透明哈希。 - **执行世界路径**:规范化目标公开绝对 POSIX 进程路径、百分号编码的 `file:` URI,以及由提供方负责的包含关系检查,因此通用进程管理消费方无需解析 E2B 目标 ID,也不会套用宿主路径规则。 - **UTF-8 读取**:完整读取和流式读取会保留跨分片解码、拒绝无效 UTF-8,并使用 seam 的 8192 字节 NUL 样本检测二进制内容。面向模型的工具仍负责选择大小和行窗口。 -- **稳定的有界读取**:一个零依赖 Node 辅助程序会以不跟随链接的方式逐级打开目录描述符,并通过一个持续持有的常规文件描述符读取至字节上限。因此,通用 LSP 查询会在服务器启动前拒绝父目录交换、非文件、无效 UTF-8,以及增长后超出所配置文档上限的文件。 - **原子变更**:写入会创建随机的同级暂存目录,在上传内容前将其 mode 改为 `0700`,保留现有文件的 POSIX mode,并通过 E2B 的同一文件系统原子重命名发布暂存文件。重命名响应会提供已提交的版本,因此提交点之后不会再进行可能失败的元数据请求。E2B 会创建缺失的父目录。字面量编辑匹配时会规范化为 LF,存储时恢复占主导的 CRLF,并在宿主进程内按规范化目标串行执行变更。可选的创建/版本防护会保留基础 seam 的已观察状态语义。 - **失败与取消**:E2B 的未找到、权限、中止及其他控制器故障会映射到现有 `FsError` 词汇。取消在更早的 SDK 请求边界上采用尽力而为语义,并在 rename 前立即检查。信号不会传入 rename RPC,因此取消无法中断原子提交;成功 rename 是提交点。 @@ -27,5 +26,6 @@ - **不提供宿主同步**:空的 E2B cwd 会一直为空,直到工具、命令、模板或外部进程填充它;本地文件既不会上传,也不会同步回本地。 - **变更协调仅限宿主进程内**:另一个 harness 连接或远程命令可能与适配器发生竞态;版本防护只能检测 E2B 元数据所体现的变更。 +- **读取会按路径重新打开规范化目标**:在解析与打开流之间若并发替换远程路径,该操作没有稳定文件句柄提供围栏;在该 POC 中,没有已观察到的产品缺陷能够证明提供方专用的有界读取协议值得引入。 - **仍需承担完整文件变更成本**:覆盖差异和字面量编辑会把完整文件读入宿主内存,每项操作也都会产生 E2B 控制器延迟。 -- **自定义模板必须支持所用的 Linux、Node、procfs 与 envd 功能**:必须支持 `realpath`、`chmod`、`mv`、同一文件系统内的 POSIX rename、流式读取、文件元数据扩展属性、`/proc/self/fd` 和不跟随链接的描述符打开操作;不支持的模板会失败,而不会静默降级。 +- **自定义模板必须支持所用的 Linux/GNU 与 E2B 文件系统功能**:必须支持 `realpath -mz`、`base64 -w0`、`chmod`、同一文件系统内的 rename、流式读取和文件元数据扩展属性;不支持的模板会失败,而不会静默降级。 diff --git a/packages/e2b/fs-e2b/src/index.ts b/packages/e2b/fs-e2b/src/index.ts index 7c81791d55..3b0c835d8c 100644 --- a/packages/e2b/fs-e2b/src/index.ts +++ b/packages/e2b/fs-e2b/src/index.ts @@ -26,17 +26,10 @@ import { quoteE2BShellArg, } from '@deepseek-ai/dsh-e2b' import type { EntryInfo, Sandbox } from '@deepseek-ai/dsh-e2b' -import { BOUNDED_READER_SOURCE } from './source-reader.ts' const VERSION_METADATA_KEY = 'dsh-version' const BINARY_SAMPLE_BYTES = 8192 - -type BoundedReadResponse = - | { kind: 'ok'; data: string } - | { kind: 'not-file' } - | { kind: 'oversize'; size: number } - | { kind: 'grew' } - | { kind: 'open-error'; message: string } +const BASE64 = /^(?:[A-Za-z0-9+/]{4})*(?:[A-Za-z0-9+/]{2}==|[A-Za-z0-9+/]{3}=)?$/ function assertNotAborted(signal: AbortSignal | undefined, operation: string): void { if (signal?.aborted === true) throw new FsError(`${operation} aborted`, 'FS_ABORTED') @@ -68,6 +61,27 @@ function decodeText(bytes: Uint8Array, displayPath: string, binarySampleBytes: n } } +function decodeCanonicalPath(encoded: string): string { + if (encoded.length === 0 || !BASE64.test(encoded)) { + throw new Error('fs-e2b: canonical path transport returned invalid base64') + } + const framed = Buffer.from(encoded, 'base64') + if (framed.toString('base64') !== encoded + || framed.length < 2 + || framed.at(-1) !== 0 + || framed.subarray(0, -1).includes(0)) { + throw new Error('fs-e2b: canonical path transport returned invalid NUL framing') + } + let path: string + try { + path = new TextDecoder('utf-8', { fatal: true }).decode(framed.subarray(0, -1)) + } catch (error: unknown) { + throw new Error('fs-e2b: canonical path is not valid UTF-8', { cause: error }) + } + if (!posix.isAbsolute(path)) throw new Error('fs-e2b: canonical path is not absolute') + return path +} + function signalOpts(signal: AbortSignal | undefined): { signal?: AbortSignal } { return signal === undefined ? {} : { signal } } @@ -213,69 +227,6 @@ export class E2BFileSystem extends FileSystem { } } - override async readTextBounded(target: FsTarget, maxBytes: number, signal?: AbortSignal): Promise { - if (!Number.isSafeInteger(maxBytes) || maxBytes <= 0) { - throw new Error('bounded read maxBytes must be a positive safe integer') - } - assertNotAborted(signal, 'read') - const sandbox = await this.ctx.e2b.getSandbox() - try { - const node = await sandbox.commands.run('command -v -- node', commandOpts(signal)) - const executable = node.stdout.trim() - if (!posix.isAbsolute(executable) || executable.includes('\n')) { - throw new Error('fs-e2b: bounded reader requires one absolute Node executable') - } - const command = [ - quoteE2BShellArg(executable), - '--input-type=commonjs', - '-e', - quoteE2BShellArg(BOUNDED_READER_SOURCE), - quoteE2BShellArg(this.processPath(target)), - String(maxBytes), - ].join(' ') - const result = await sandbox.commands.run(command, commandOpts(signal)) - assertNotAborted(signal, 'read') - const response = this.parseBoundedRead(result.stdout, target) - if (response.kind === 'not-file') { - throw new FsError(`cannot read "${target.displayPath}": not a regular file`, 'FS_NOT_REGULAR_FILE') - } - if (response.kind === 'oversize' && Number.isSafeInteger(response.size)) { - throw new FsError( - `cannot read "${target.displayPath}": ${response.size} bytes exceeds the ${maxBytes}-byte limit`, - 'FS_IO_ERROR', - ) - } - if (response.kind === 'grew') { - throw new FsError( - `cannot read "${target.displayPath}": file grew past the ${maxBytes}-byte limit while reading`, - 'FS_IO_ERROR', - ) - } - if (response.kind === 'open-error' && typeof response.message === 'string') { - if (/ENOENT|no such file/i.test(response.message)) { - throw new FsError(`cannot read "${target.displayPath}": not found`, 'FS_NOT_FOUND') - } - if (/EACCES|EPERM|permission denied|operation not permitted/i.test(response.message)) { - throw new FsError(`cannot read "${target.displayPath}": permission denied`, 'FS_PERMISSION_DENIED') - } - throw new FsError( - `cannot read "${target.displayPath}" safely: ${response.message}`, - 'FS_IO_ERROR', - ) - } - if (response.kind !== 'ok' || typeof response.data !== 'string') { - throw new FsError(`cannot read "${target.displayPath}": bounded reader returned an invalid response`, 'FS_IO_ERROR') - } - const bytes = Buffer.from(response.data, 'base64') - if (bytes.toString('base64') !== response.data || bytes.length > maxBytes) { - throw new FsError(`cannot read "${target.displayPath}": bounded reader returned invalid bytes`, 'FS_IO_ERROR') - } - return decodeText(bytes, target.displayPath, BINARY_SAMPLE_BYTES) - } catch (error: unknown) { - throw mapError(error, 'read', target.displayPath, signal) - } - } - override async streamText(target: FsTarget, signal?: AbortSignal): Promise> { const sandbox = await this.ctx.e2b.getSandbox() await this.requireRegular(target, signal) @@ -339,18 +290,23 @@ export class E2BFileSystem extends FileSystem { try { const sandbox = await this.ctx.e2b.getSandbox() const listed = await sandbox.files.list(String(target.targetKey), { depth: 1, ...signalOpts(signal) }) - const entries = await Promise.all(listed.map(async (entry): Promise => { + const entries: FsDirEntry[] = [] + for (const entry of listed) { const displayPath = posix.join(target.displayPath, entry.name) - const canonical = await this.canonicalPath(sandbox, entry.path, signal) - const resolved = await this.probe(canonical, displayPath, signal) - return { + const canonical = entry.symlinkTarget === undefined + ? entry.path + : await this.canonicalPath(sandbox, entry.path, signal) + const resolved = entry.symlinkTarget === undefined + ? entry + : await this.probe(canonical, displayPath, signal) + entries.push({ name: entry.name, type: resolved === undefined ? 'other' : entryType(resolved), target: { targetKey: FsTargetKey(canonical), displayPath }, ...(resolved !== undefined ? { version: entryVersion(resolved) } : {}), ...(resolved?.type === FileType.FILE ? { size: resolved.size } : {}), - } - })) + }) + } return entries.sort((left, right) => left.name.localeCompare(right.name)) } catch (error: unknown) { throw mapError(error, 'list', target.displayPath, signal) @@ -420,26 +376,17 @@ export class E2BFileSystem extends FileSystem { private async canonicalPath(sandbox: Sandbox, path: string, signal?: AbortSignal): Promise { try { - const result = await sandbox.commands.run(`realpath -m -- ${quoteE2BShellArg(path)}`, commandOpts(signal)) - return result.stdout.replace(/\n$/, '') + const result = await sandbox.commands.run( + `set -o pipefail; realpath -mz -- ${quoteE2BShellArg(path)} | base64 -w0`, + commandOpts(signal), + ) + return decodeCanonicalPath(result.stdout) } catch (error: unknown) { if (error instanceof CommandExitError) throw new Error(error.stderr || error.message, { cause: error }) throw error } } - private parseBoundedRead(stdout: string, target: FsTarget): BoundedReadResponse { - try { - return JSON.parse(stdout) as BoundedReadResponse - } catch (error: unknown) { - throw new FsError( - `cannot read "${target.displayPath}": bounded reader returned invalid JSON`, - 'FS_IO_ERROR', - { cause: error }, - ) - } - } - private async probe(path: string, displayPath: string, signal?: AbortSignal): Promise { assertNotAborted(signal, 'stat') try { diff --git a/packages/e2b/fs-e2b/src/source-reader.ts b/packages/e2b/fs-e2b/src/source-reader.ts deleted file mode 100644 index 9cb522ecbb..0000000000 --- a/packages/e2b/fs-e2b/src/source-reader.ts +++ /dev/null @@ -1,63 +0,0 @@ -/** Dependency-free stable-handle bounded reader installed inside E2B. */ -export const BOUNDED_READER_SOURCE = String.raw` -/* dsh-e2b-bounded-reader */ -const fs = require('node:fs') -const target = process.argv[1] -const maxBytes = Number(process.argv[2]) -const directoryFlags = fs.constants.O_RDONLY | fs.constants.O_DIRECTORY | fs.constants.O_NOFOLLOW | fs.constants.O_NONBLOCK -const fileFlags = fs.constants.O_RDONLY | fs.constants.O_NOFOLLOW | fs.constants.O_NONBLOCK -let directory -let descriptor -let response - -const openChild = (parent, component, flags) => fs.openSync('/proc/self/fd/' + parent + '/' + component, flags) -const invalidComponent = component => component === '' || component === '.' || component === '..' - -try { - if (typeof target !== 'string' || !target.startsWith('/') || !Number.isSafeInteger(maxBytes) || maxBytes <= 0) { - throw new Error('bounded reader requires an absolute target and positive byte limit') - } - const components = target === '/' ? [] : target.slice(1).split('/') - if (components.length === 0 || components.some(invalidComponent)) { - throw new Error('bounded reader received a non-canonical file path') - } - - directory = fs.openSync('/', directoryFlags) - for (const component of components.slice(0, -1)) { - const child = openChild(directory, component, directoryFlags) - fs.closeSync(directory) - directory = child - } - descriptor = openChild(directory, components.at(-1), fileFlags) - - const info = fs.fstatSync(descriptor) - if (!info.isFile()) response = { kind: 'not-file' } - else if (info.size > maxBytes) response = { kind: 'oversize', size: info.size } - else { - const chunks = [] - let total = 0 - while (total <= maxBytes) { - const chunk = Buffer.allocUnsafe(Math.min(65536, maxBytes - total + 1)) - const bytesRead = fs.readSync(descriptor, chunk, 0, chunk.length, null) - if (bytesRead === 0) break - chunks.push(chunk.subarray(0, bytesRead)) - total += bytesRead - } - response = total > maxBytes - ? { kind: 'grew' } - : { kind: 'ok', data: Buffer.concat(chunks, total).toString('base64') } - } -} catch (error) { - response = { kind: 'open-error', message: error instanceof Error ? error.message : String(error) } -} finally { - for (const openDescriptor of [descriptor, directory]) { - if (openDescriptor === undefined) continue - try { - fs.closeSync(openDescriptor) - } catch (error) { - response = { kind: 'open-error', message: error instanceof Error ? error.message : String(error) } - } - } -} -process.stdout.write(JSON.stringify(response)) -` diff --git a/packages/e2b/fs-e2b/tests/filesystem.spec.ts b/packages/e2b/fs-e2b/tests/filesystem.spec.ts index 23835e9967..91f9a278fd 100644 --- a/packages/e2b/fs-e2b/tests/filesystem.spec.ts +++ b/packages/e2b/fs-e2b/tests/filesystem.spec.ts @@ -49,10 +49,7 @@ class FakeRemote { nextReadError: unknown nextRenameError: unknown nextRemoveError: unknown - boundedOutput: string | undefined - boundedError: unknown - nodeExecutable = '/usr/bin/node\n' - abortAfterBoundedCommand: AbortController | undefined + canonicalOutput: string | undefined abortAfterRename: AbortController | undefined disappearOnInfo = new Set() private clock = 1 @@ -239,18 +236,18 @@ class FakeRemote { this.nextCommandError = undefined throw error } - if (command.startsWith('realpath -m -- ')) { - const input = command.slice('realpath -m -- '.length).slice(1, -1) + const realpathPrefix = 'set -o pipefail; realpath -mz -- ' + const realpathSuffix = ' | base64 -w0' + if (command.startsWith(realpathPrefix) && command.endsWith(realpathSuffix)) { + const quoted = command.slice(realpathPrefix.length, -realpathSuffix.length) + const input = quoted.slice(1, -1).replaceAll(String.raw`'"'"'`, '\'') const node = this.nodes.get(input) - return { exitCode: 0, stdout: `${node?.symlinkTarget ?? input}\n`, stderr: '' } - } - if (command === 'command -v -- node') { - return { exitCode: 0, stdout: this.nodeExecutable, stderr: '' } - } - if (command.includes('dsh-e2b-bounded-reader')) { - if (this.boundedError !== undefined) throw this.boundedError - this.abortAfterBoundedCommand?.abort('after bounded read') - return { exitCode: 0, stdout: this.boundedOutput ?? '{"kind":"ok","data":""}', stderr: '' } + const canonical = `${node?.symlinkTarget ?? input}\0` + return { + exitCode: 0, + stdout: this.canonicalOutput ?? Buffer.from(canonical).toString('base64'), + stderr: '', + } } const chmod = /^chmod ([0-7]+) -- '([^']+)'$/.exec(command) if (chmod !== null) this.required(chmod[2]!).mode = Number.parseInt(chmod[1]!, 8) @@ -340,6 +337,28 @@ describe('E2BFileSystem identity, metadata, and reads', () => { .toThrow('expected an absolute process path') }) + it('preserves newline and multibyte canonical paths through strict ASCII framing', async () => { + const remote = new FakeRemote() + const path = '/workspace/你好\nfile.ts' + remote.file(path, 'text') + const { fs } = await setup(remote) + + await expect(fs.resolve(path)).resolves.toEqual({ targetKey: path, displayPath: path }) + }) + + it.each([ + ['invalid base64', '!!!!'], + ['missing terminator', Buffer.from('/workspace/file').toString('base64')], + ['multiple records', Buffer.from('/workspace/file\0/other\0').toString('base64')], + ['invalid UTF-8', Buffer.from([47, 0xff, 0]).toString('base64')], + ['relative path', Buffer.from('workspace/file\0').toString('base64')], + ])('rejects %s from canonical path transport', async (_label, output) => { + const remote = new FakeRemote() + remote.canonicalOutput = output + const { fs } = await setup(remote) + await expectCode(fs.resolve('file'), 'FS_IO_ERROR') + }) + it('reads whole and streamed UTF-8 across chunk boundaries', async () => { const remote = new FakeRemote() remote.file('/workspace/text.txt', 'A€B') @@ -424,78 +443,6 @@ describe('E2BFileSystem identity, metadata, and reads', () => { await expectCode(fs.readText(await fs.resolve('a')), 'FS_ABORTED') }) - it('performs stable bounded reads through the remote no-follow reader', async () => { - const remote = new FakeRemote() - remote.file('/workspace/a', 'unused') - const { fs } = await setup(remote) - const target = await fs.resolve('a') - remote.boundedOutput = JSON.stringify({ kind: 'ok', data: Buffer.from('hello 你好').toString('base64') }) - await expect(fs.readTextBounded(target, 64)).resolves.toBe('hello 你好') - expect(remote.commands.some(command => command.includes('dsh-e2b-bounded-reader'))).toBe(true) - - await expect(fs.readTextBounded(target, 0)).rejects.toThrow('positive safe integer') - await expect(fs.readTextBounded(target, 1.5)).rejects.toThrow('positive safe integer') - await expect(fs.readTextBounded(target, 64, AbortSignal.abort())).rejects.toMatchObject({ code: 'FS_ABORTED' }) - }) - - it('maps bounded-reader file, size, and open failures', async () => { - const remote = new FakeRemote() - remote.file('/workspace/a', 'unused') - const { fs } = await setup(remote) - const target = await fs.resolve('a') - const cases: Array<[unknown, string]> = [ - [{ kind: 'not-file' }, 'FS_NOT_REGULAR_FILE'], - [{ kind: 'oversize', size: 65 }, 'FS_IO_ERROR'], - [{ kind: 'grew' }, 'FS_IO_ERROR'], - [{ kind: 'open-error', message: 'ENOENT: no such file' }, 'FS_NOT_FOUND'], - [{ kind: 'open-error', message: 'EACCES: permission denied' }, 'FS_PERMISSION_DENIED'], - [{ kind: 'open-error', message: 'ELOOP: symbolic link' }, 'FS_IO_ERROR'], - [{ kind: 'oversize', size: 'large' }, 'FS_IO_ERROR'], - [{ kind: 'open-error', message: 7 }, 'FS_IO_ERROR'], - [{ kind: 'unknown' }, 'FS_IO_ERROR'], - ] - for (const [response, code] of cases) { - remote.boundedOutput = JSON.stringify(response) - await expectCode(fs.readTextBounded(target, 64), code) - } - }) - - it('rejects malformed bounded-reader transports and bytes', async () => { - const remote = new FakeRemote() - remote.file('/workspace/a', 'unused') - const { fs } = await setup(remote) - const target = await fs.resolve('a') - - remote.boundedOutput = 'not-json' - await expectCode(fs.readTextBounded(target, 4), 'FS_IO_ERROR') - remote.boundedOutput = JSON.stringify({ kind: 'ok', data: '!!!!' }) - await expectCode(fs.readTextBounded(target, 4), 'FS_IO_ERROR') - remote.boundedOutput = JSON.stringify({ kind: 'ok', data: Buffer.from('12345').toString('base64') }) - await expectCode(fs.readTextBounded(target, 4), 'FS_IO_ERROR') - remote.boundedOutput = JSON.stringify({ kind: 'ok', data: Buffer.from([0]).toString('base64') }) - await expectCode(fs.readTextBounded(target, 4), 'FS_NOT_TEXT') - remote.boundedOutput = JSON.stringify({ kind: 'ok', data: Buffer.from([0xff]).toString('base64') }) - await expectCode(fs.readTextBounded(target, 4), 'FS_NOT_TEXT') - - remote.nodeExecutable = 'node\n' - await expectCode(fs.readTextBounded(target, 4), 'FS_IO_ERROR') - remote.nodeExecutable = '/usr/bin/node\n/other\n' - await expectCode(fs.readTextBounded(target, 4), 'FS_IO_ERROR') - remote.nodeExecutable = '/usr/bin/node\n' - remote.boundedError = new Error('reader transport failed') - await expectCode(fs.readTextBounded(target, 4), 'FS_IO_ERROR') - }) - - it('does not turn a post-read abort into successful source text', async () => { - const remote = new FakeRemote() - remote.file('/workspace/a', 'unused') - const controller = new AbortController() - remote.abortAfterBoundedCommand = controller - remote.boundedOutput = JSON.stringify({ kind: 'ok', data: Buffer.from('text').toString('base64') }) - const { fs } = await setup(remote) - await expectCode(fs.readTextBounded(await fs.resolve('a'), 4, controller.signal), 'FS_ABORTED') - }) - it('rejects empty paths and directory-listing type errors', async () => { const remote = new FakeRemote() remote.file('/workspace/file', 'x') @@ -686,17 +633,34 @@ describe('E2B filesystem adapter integration edges', () => { await expectCode(fs.readText(target), 'FS_IO_ERROR') }) - it('keeps a listed child whose metadata disappears as an other entry', async () => { + it('uses listing metadata directly and canonicalizes only symbolic links', async () => { const remote = new FakeRemote() remote.file('/workspace/a', 'a') - remote.disappearOnInfo.add('/workspace/a') + remote.file('/workspace/target', 'target') + remote.file('/workspace/gone', 'gone') + remote.symlink('/workspace/link', '/workspace/target') + remote.symlink('/workspace/vanished-link', '/workspace/gone') + remote.disappearOnInfo.add('/workspace/gone') const { fs } = await setup(remote) - const listed = await fs.listDir(await fs.resolve('/workspace')) - expect(listed).toEqual([{ - name: 'a', + const directory = await fs.resolve('/workspace') + const commandsBefore = remote.commands.length + const getInfo = vi.spyOn(remote.sandbox.files, 'getInfo') + + const listed = await fs.listDir(directory) + + expect(listed.find(entry => entry.name === 'a')).toMatchObject({ + type: 'file', target: { targetKey: '/workspace/a' }, size: 1, + }) + expect(listed.find(entry => entry.name === 'link')).toMatchObject({ + type: 'file', target: { targetKey: '/workspace/target' }, size: 6, + }) + expect(listed.find(entry => entry.name === 'vanished-link')).toEqual({ + name: 'vanished-link', type: 'other', - target: { targetKey: '/workspace/a', displayPath: '/workspace/a' }, - }]) + target: { targetKey: '/workspace/gone', displayPath: '/workspace/vanished-link' }, + }) + expect(remote.commands.slice(commandsBefore)).toHaveLength(2) + expect(getInfo).toHaveBeenCalledTimes(3) }) it('registers the package-owned empty invariant installer', async () => { diff --git a/packages/e2b/subprocess-e2b/README.i18n.yaml b/packages/e2b/subprocess-e2b/README.i18n.yaml index ac7b8d42a5..20ba9b597e 100644 --- a/packages/e2b/subprocess-e2b/README.i18n.yaml +++ b/packages/e2b/subprocess-e2b/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/e2b/subprocess-e2b/README.md -README.md: 9b86af428533ebcc2c0da56339e6ea2a28170fd3 -README.zh.md: e3e8368534266e35ec04dcf5a2d6718829b4a016 +README.md: 9e044c13b673461a876e016ec560db4897a27c66 +README.zh.md: b9180c0c0679915fae7a1db22353613708392a8c diff --git a/packages/e2b/subprocess-e2b/README.md b/packages/e2b/subprocess-e2b/README.md index 9b86af4285..9e044c13b6 100644 --- a/packages/e2b/subprocess-e2b/README.md +++ b/packages/e2b/subprocess-e2b/README.md @@ -9,9 +9,9 @@ E2B implementation of the [`@deepseek-ai/dsh-subprocess`](../../subprocess/subpr - **Asynchronous remote start** — the synchronous seam returns a handle immediately while `Sandbox.commands.run(..., { background: true })` starts remotely. `pid` is `-1` until the wrapper publishes and the adapter validates its process-group id; stdin and ordinary observation wait for that publication. An owned startup signal aborts environment and private-state preparation before allocation; once allocation begins, cancellation waits for a provisional SDK handle it can clean. - **Execution-world coordinates** — `cwd` and private `runtimeRoot` come from the shared owner; executable lookup verifies absolute paths or resolves a bare name against the sandbox PATH plus explicit overrides. - **Linux process groups** — a quoted wrapper starts each argv under `exec setsid --wait` and records its actual process-group id plus private status files beneath `ctx.e2b.runtimeRoot/processes`. The handle waits for that file instead of treating the SDK command PID as its published identity. Termination signals the negative recorded id with `SIGTERM`, waits the caller's `graceMs`, then escalates to `SIGKILL` and the SDK kill fallback; TERM delivery or probe failures also force that escalation. Process-table probes treat groups containing only zombie or dead entries as quiescent. Force cleanup succeeds only after a bounded probe finds the group empty; otherwise `waitForExit()` exposes a retryable failure, while proven quiescence makes later termination a no-op. Publication and monitoring failures apply the same cleanup transaction before rejecting. Service disposal rejects new starts, terminates and joins every retained process group, then awaits SDK settlement and private cleanup before the sandbox owner disposes. -- **Environment boundary** — the sandbox command environment crosses the SDK callback boundary as base64 ASCII before one strict UTF-8 decode, then the wrapper removes ambient `DSH_*` and credential-shaped (`*KEY*`, `*SECRET*`, `*TOKEN*`) names and restores every valid `spec.env` entry as an explicit caller opt-in; empty names, `=`, and NUL framing violations reject before launch. E2B's fixed command and PTY login shells receive a fresh randomized root-level `HOME` plus empty overrides for every scrubbed ambient name before user profiles can run; the requested argv receives the serialized environment afterward. Host ambient variables never enter the sandbox implicitly. Private environment files are removed after consumption, and failed command or terminal setup removes its private state before rejecting. +- **Environment boundary** — one trusted control-shell probe transports the sandbox environment as base64 ASCII for one strict UTF-8 decode; the wrapper then removes ambient `DSH_*` and credential-shaped (`*KEY*`, `*SECRET*`, `*TOKEN*`) names and restores every valid `spec.env` entry as an explicit caller opt-in. Empty names, `=`, and NUL framing violations reject before launch. Subsequent E2B command and PTY login shells receive a fresh randomized root-level `HOME` plus empty overrides for every scrubbed ambient name before user profiles can run; the requested argv receives the serialized environment afterward. Host ambient variables never enter the sandbox implicitly. Private environment files are removed after consumption, and failed command or terminal setup removes its private state before rejecting. - **Stdio projection** — the remote wrapper branches raw bytes into optional bounded spill files, frames each live chunk as newline-delimited base64 ASCII, and the host incrementally restores bytes across arbitrary SDK callback boundaries. Pipe mode writes those bytes to host Node streams; inherit mode writes them to the harness process streams; collect mode retains a bounded host tail with offset reads. The wrapper publishes the direct command status before waiting for inherited writers. For collect or inherit output, the adapter disconnects an incomplete SDK stream after `graceMs`, withholds its partial spill, and returns that status while retaining the remote group for `waitForExit()` and termination. Natural raw-pipe completion instead awaits lossless transport and preserves backpressure; explicit termination destroys the host pipes and releases blocked output before remote cleanup. Batch and streaming stdin use the SDK handle. -- **Terminal sessions** — `spawnTerminal()` uses E2B's byte PTY API, installs the exact argv and scrubbed environment through private mode-`0600` files, reports the foreground process group, sends real signals, and tears down every live group in the remote terminal session before settlement; zombie-only groups are already quiescent. A private random output boundary discards the E2B bootstrap shell's prompt and echoed runner command while preserving every requested-process byte, including its first prompt. PTY allocation is awaited through handle publication before cancellation is observed, so owned rollback can clean the published handle. Setup and teardown own the private state transaction, abort pending setup during service disposal, fence publication, and retain an unproven setup cleanup for disposal retry. Prompt detection, scrollback, readiness, and owner policy remain in `dsh-pty-local`. +- **Terminal sessions** — `spawnTerminal()` uses E2B's byte PTY API, installs the exact argv and scrubbed environment through private mode-`0600` files, reports the foreground process group, sends real signals, and tears down every live group in the remote terminal session through one retryable awaited `terminate()`; zombie-only groups are already quiescent. A private random output boundary discards the E2B bootstrap shell's prompt and echoed runner command while preserving every requested-process byte, including its first prompt. PTY allocation is awaited through handle publication before cancellation is observed, so owned rollback can clean the published handle. Setup and teardown own the private state transaction, abort pending setup during service disposal, fence publication, and retain an unproven setup cleanup for disposal retry. Prompt detection, scrollback, readiness, and owner policy remain in `dsh-pty-local`. - **Sandbox disappearance** — `SandboxNotFoundError` during process or terminal liveness, termination, rollback, or disconnect proves the remote execution world cannot retain work, so cleanup treats it as quiescent; unrelated failures remain observable. The base E2B image supplies the runtime and Bash/GNU utilities this adapter invokes: `node`, `bash`, `setsid`, `ps`, `awk`, `tr`, `env`, `base64`, `chmod`, `tee`, `head`, `rm`, and `kill`. A custom template must retain compatible commands and E2B PTY support. @@ -30,6 +30,8 @@ No direct invalidation; the named consumers own any request-prefix changes. - **Synchronous-PID consumers are unsupported** — `pid` remains `-1` during remote startup; consumers that require a positive PID immediately, including the ACP child backend, cannot use this provider unchanged. - **Reconnect does not reconstruct handles** — remote PID/status/spill files survive a retained sandbox, but a new harness process does not rebuild live `SubprocessHandle` objects or output cursors from them. - **Remote state accumulates when retained** — process directories and valid spill files remain under `.dsh-e2b`; this POC supplies no retention sweep. +- **Numeric process identities are not reuse-fenced** — E2B exposes numeric PID/PGID PTY input, signalling, and cleanup operations but no atomic identity-bound alternative. The adapter minimizes host round trips and live coverage exercises the reproducible stale-interrupt overlap; replacement is deferred until E2B adds an identity primitive or a failure demonstrates a narrower protocol. +- **The initial environment probe inherits sandbox defaults** — E2B merges command overrides with default environment entries, so the probe cannot blank unknown credential-shaped names before enumerating them. In a reconnected sandbox, a same-UID untrusted process could inspect that short-lived control shell; this POC therefore does not support secrets in sandbox-default environment variables and requires an E2B replacement-environment primitive or a hardened template to close the gap. - **E2B exposes no signal fact** — an adapter-requested `SIGTERM` or `SIGKILL` is reported only when no wrapper-published direct exit code wins; every unrequested SDK exit remains an exit code, including values shaped like `128 + signal`. - **Exact terminal stdin-wait inspection is unavailable** — E2B exposes the foreground process group but not the syscall evidence needed to prove it is waiting on fd 0, so the generic PTY backend falls back to controlled prompt markers and bounded silence. - **Linux utility and E2B transport semantics are assumed** — there is no Windows, arbitrary-template, escaped-session recovery, or network-partition fidelity layer. diff --git a/packages/e2b/subprocess-e2b/README.zh.md b/packages/e2b/subprocess-e2b/README.zh.md index e3e8368534..b9180c0c06 100644 --- a/packages/e2b/subprocess-e2b/README.zh.md +++ b/packages/e2b/subprocess-e2b/README.zh.md @@ -9,9 +9,9 @@ - **异步远程启动**:同步 seam 会立即返回一个句柄,同时由 `Sandbox.commands.run(..., { background: true })` 在远程启动进程。包装层发布进程组 ID 并由适配器完成验证之前,`pid` 为 `-1`;stdin 和常规观察会等待该发布。自有启动信号会在分配前中止环境和私有状态准备;分配开始后,取消会等待可清理的临时 SDK 句柄。 - **执行世界坐标**:`cwd` 和私有 `runtimeRoot` 来自共享所有者;可执行文件查找会验证绝对路径,或根据沙箱 PATH 加显式覆盖来解析裸名称。 - **Linux 进程组**:带引号保护的包装层会在 `exec setsid --wait` 下启动每组 argv,并在 `ctx.e2b.runtimeRoot/processes` 下记录实际进程组 ID 和私有状态文件。句柄会等待该文件,而不会把 SDK 命令 PID 当作已发布的身份。终止操作以记录的负数 ID 发送 `SIGTERM`,等待调用方的 `graceMs`,再升级到 `SIGKILL` 和 SDK kill 回退;TERM 信号发送或探测失败也会强制触发该升级。进程表探测会把仅含僵尸或已死亡条目的进程组视为完全停稳。强制清理只有在有界探测发现进程组为空后才算成功;否则 `waitForExit()` 会公开可重试的失败,而已证明的完全停稳会让后续终止操作不再执行任何动作。发布失败与监控失败都会在拒绝前执行同一清理事务。服务 dispose(资源释放)会拒绝新的启动请求、终止并等待每个保留进程组退出,再等待 SDK 结算和私有清理完成,之后沙箱所有者才会释放。 -- **环境边界**:沙箱命令环境会先以 base64 ASCII 跨越 SDK 回调边界,再进行一次严格 UTF-8 解码;随后包装层移除环境中的 `DSH_*` 和形似凭据的名称(`*KEY*`、`*SECRET*`、`*TOKEN*`),并把每个有效的 `spec.env` 条目恢复为调用方显式选择。空名称、`=` 和违反 NUL 分帧规则的条目会在启动前被拒绝。在用户 profile 脚本运行前,E2B 固定的命令 shell 与 PTY 登录 shell 会获得位于根目录下、全新随机生成的 `HOME`,并为每个被清理的环境变量名设置空值覆盖;之后,请求的 argv 才会接收序列化环境。宿主环境变量绝不会隐式进入沙箱。私有环境文件在使用后会被删除;命令或终端设置失败时,会先删除其私有状态再拒绝。 +- **环境边界**:一次受信任的控制 shell 探测会以 base64 ASCII 传输沙箱环境,再进行一次严格 UTF-8 解码;随后包装层移除环境中的 `DSH_*` 和形似凭据的名称(`*KEY*`、`*SECRET*`、`*TOKEN*`),并把每个有效的 `spec.env` 条目恢复为调用方显式选择。空名称、`=` 和违反 NUL 分帧规则的条目会在启动前被拒绝。在用户 profile 脚本运行前,此后的 E2B 命令 shell 与 PTY 登录 shell 会获得位于根目录下、全新随机生成的 `HOME`,并为每个被清理的环境变量名设置空值覆盖;之后,请求的 argv 才会接收序列化环境。宿主环境变量绝不会隐式进入沙箱。私有环境文件在使用后会被删除;命令或终端设置失败时,会先删除其私有状态再拒绝。 - **stdio 投影**:远程包装层先把原始字节分流到可选的有界 spill 文件,再把每个实时分片编码为换行分隔的 base64 ASCII 帧;宿主会跨任意 SDK 回调边界增量恢复字节。pipe 模式把这些字节写入宿主 Node 流;inherit 模式把字节写入 harness 进程流;collect 模式保留有界的宿主尾部,并支持基于偏移量读取。包装层会在等待继承管道的写入方之前发布直接命令状态。对于 collect 或 inherit 输出,超过 `graceMs` 后,适配器会断开未完成的 SDK 流,不公开其中不完整的 spill,并返回该状态,同时保留远程进程组供 `waitForExit()` 和终止操作使用。原始 pipe 自然完成时,会等待无损传输完成并保留背压;显式终止则会销毁宿主 pipe,并在远程清理前释放受阻的输出写入。批量 stdin 和流式 stdin 都使用 SDK 句柄。 -- **终端会话**:`spawnTerminal()` 使用 E2B 的字节 PTY API,以 mode 为 `0600` 的私有文件传入原样 argv 与清理后的环境,报告前台进程组,发送真实信号,并在结算前清理远程终端会话中仍存活的每个进程组;仅含僵尸进程的进程组已经完全停稳。私有随机输出边界会丢弃 E2B 引导 shell 的提示符和回显的 runner 命令,同时保留请求进程的每个字节,包括其第一个提示符。PTY 分配会一直等待到句柄发布后才观察取消,以便由承担清理责任的回滚清理已发布句柄。setup 与 teardown 负责私有状态事务,在服务 dispose 期间中止待处理的 setup、阻止发布,并保留未证明已完成的 setup 清理事务,供 dispose 重试。提示符检测、scrollback、就绪状态与所有者策略仍归 `dsh-pty-local` 所有。 +- **终端会话**:`spawnTerminal()` 使用 E2B 的字节 PTY API,以 mode 为 `0600` 的私有文件传入原样 argv 与清理后的环境,报告前台进程组,发送真实信号,并通过一项可重试且须等待的 `terminate()` 清理远程终端会话中仍存活的每个进程组;仅含僵尸进程的进程组已经完全停稳。私有随机输出边界会丢弃 E2B 引导 shell 的提示符和回显的 runner 命令,同时保留请求进程的每个字节,包括其第一个提示符。PTY 分配会一直等待到句柄发布后才观察取消,以便由承担清理责任的回滚清理已发布句柄。setup 与 teardown 负责私有状态事务,在服务 dispose 期间中止待处理的 setup、阻止发布,并保留未证明已完成的 setup 清理事务,供 dispose 重试。提示符检测、scrollback、就绪状态与所有者策略仍归 `dsh-pty-local` 所有。 - **沙箱消失**:在进程或终端的存活探测、终止、回滚或断开连接期间出现 `SandboxNotFoundError`,证明远程执行环境无法保留工作,因此清理会将其视为完全停稳;其他故障仍可观察。 基础 E2B 镜像提供该适配器调用的运行时和 Bash/GNU 工具:`node`、`bash`、`setsid`、`ps`、`awk`、`tr`、`env`、`base64`、`chmod`、`tee`、`head`、`rm` 和 `kill`。自定义模板必须保留兼容的命令和 E2B PTY 支持。 @@ -30,6 +30,8 @@ - **不支持需要同步 PID 的消费方**:远程启动期间,`pid` 保持为 `-1`;包括 ACP 子进程后端在内,要求立即获得正 PID 的消费方无法原样使用本提供方。 - **重新连接不会重建句柄**:保留沙箱后,远程 PID/状态/spill 文件仍然存在,但新的 harness 进程不会据此重建实时 `SubprocessHandle` 对象或输出游标。 - **保留沙箱时会累积远程状态**:进程目录和有效的 spill 文件会留在 `.dsh-e2b` 下;本 POC 不提供保留清理。 +- **数值进程身份没有复用围栏**:E2B 公开基于数值 PID/PGID 的 PTY 输入、信号发送和清理操作,却没有与身份原子绑定的替代方案。适配器会尽量减少宿主往返,真实环境测试会覆盖可复现的陈旧中断重叠;在 E2B 新增身份原语,或实际故障证明需要更窄的协议之前,替代方案会继续延后。 +- **初始环境探测会继承沙箱默认值**:E2B 会把命令覆盖与默认环境条目合并,因此探测无法在枚举未知且形似凭据的名称之前将它们置空。在重新连接的沙箱中,一个同 UID 的不可信进程可以检查该短时存在的控制 shell;因此,该 POC 不支持把 secret 放入沙箱默认环境变量,需要 E2B 的替换环境原语或经加固的模板才能弥合该缺口。 - **E2B 不公开信号事实**:适配器请求的 `SIGTERM` 或 `SIGKILL` 只有在包装层发布的直接退出码没有胜出时才报告为信号;其他未请求的 SDK 退出始终保留为退出码,包括形似 `128 + signal` 的值。 - **无法精确检查终端 stdin 等待状态**:E2B 会公开前台进程组,但不提供证明其正在等待 fd 0 所需的 syscall 证据,因此通用 PTY 后端会回退到受控提示符标记与有界静默机制。 - **依赖 Linux 工具与 E2B 传输语义**:没有 Windows、任意模板、逃逸会话恢复或网络分区的保真层。 diff --git a/packages/e2b/subprocess-e2b/src/environment.ts b/packages/e2b/subprocess-e2b/src/environment.ts index 5322491261..907810a1d7 100644 --- a/packages/e2b/subprocess-e2b/src/environment.ts +++ b/packages/e2b/subprocess-e2b/src/environment.ts @@ -26,6 +26,8 @@ function remoteEnvironmentEntries(raw: string): Array * @returns the complete NUL-delimited UTF-8 environment. */ export async function readRemoteEnvironment(sandbox: Sandbox, signal?: AbortSignal): Promise { + // TODO(e2b-replace-environment): Remove this ambient probe when E2B can start + // a command with a replacement environment instead of merged overrides. const result = await sandbox.commands.run( 'set -o pipefail; printf \'%s\' "$PWD" | base64 -w 0; printf \'\\n\'; env -0 | base64 -w 0', { envs: e2bControlEnvs(), ...(signal === undefined ? {} : { signal }) }, diff --git a/packages/e2b/subprocess-e2b/src/index.ts b/packages/e2b/subprocess-e2b/src/index.ts index 7318dfde95..990ab20a62 100644 --- a/packages/e2b/subprocess-e2b/src/index.ts +++ b/packages/e2b/subprocess-e2b/src/index.ts @@ -61,8 +61,7 @@ export class E2BSubprocessService extends SubprocessService { })) } for (const terminal of terminals) { - terminal.terminate() - pending.push(terminal.waitForExit().then(() => { this.terminals.delete(terminal) })) + pending.push(terminal.terminate().then(() => { this.terminals.delete(terminal) })) } for (const cleanup of failedTerminalSetupCleanups) { pending.push(cleanup().then(() => { this.failedTerminalSetupCleanups.delete(cleanup) })) @@ -125,7 +124,9 @@ export class E2BSubprocessService extends SubprocessService { await handle.waitForExit() this.live.delete(handle) } - void handle.done.then(release, release).catch(() => {}) + void handle.done.then(release, release).catch((_automaticReleaseFailure: unknown) => { + // Retain the handle so service disposal can retry its cleanup transaction. + }) return handle } @@ -158,16 +159,17 @@ export class E2BSubprocessService extends SubprocessService { ) this.terminals.add(terminal) if (this.isDisposing()) { - terminal.terminate() - await terminal.waitForExit() + await terminal.terminate() this.terminals.delete(terminal) throw new Error('subprocess-e2b: service disposed during terminal setup') } const release = async (): Promise => { - await terminal.waitForExit() + await terminal.terminate() this.terminals.delete(terminal) } - void terminal.done.then(release, release).catch(() => {}) + void terminal.done.then(release, release).catch((_automaticReleaseFailure: unknown) => { + // Retain the terminal so service disposal can retry its cleanup transaction. + }) return terminal } finally { this.terminalSetups.delete(setup.promise) diff --git a/packages/e2b/subprocess-e2b/src/process.ts b/packages/e2b/subprocess-e2b/src/process.ts index 6b17f3b682..02e1583e56 100644 --- a/packages/e2b/subprocess-e2b/src/process.ts +++ b/packages/e2b/subprocess-e2b/src/process.ts @@ -726,7 +726,9 @@ export class E2BSubprocessHandle implements SubprocessHandle { // A spill mode is a collect mode, so construction always created its reader. const size = (reader as E2BOutputReader).size if (this.outputDrainExpired || size <= mode.maxBytes || size > mode.spill.maxBytes) { - removals.push(sandbox.files.remove(path).catch(() => {})) + removals.push(sandbox.files.remove(path).catch((_adapterPrivateSpillRemovalFailure: unknown) => { + // The command outcome is authoritative; a retained sandbox tolerates private residue. + })) } } collect(this.spec.stdio.stdout, this.stdoutReader, this.paths.stdout) diff --git a/packages/e2b/subprocess-e2b/src/terminal.ts b/packages/e2b/subprocess-e2b/src/terminal.ts index ff8af60da9..6c4b23d1c1 100644 --- a/packages/e2b/subprocess-e2b/src/terminal.ts +++ b/packages/e2b/subprocess-e2b/src/terminal.ts @@ -12,7 +12,6 @@ import { quoteE2BShellArg, } from '@deepseek-ai/dsh-e2b' import type { CommandHandle, CommandResult, Sandbox } from '@deepseek-ai/dsh-e2b' -import { SubprocessTerminalLifecycle } from '@deepseek-ai/dsh-subprocess' import type { SubprocessOutcome, SubprocessTerminalForeground, @@ -345,7 +344,7 @@ export class E2BTerminalHandle implements SubprocessTerminalHandle { readonly done: Promise private topLevelExited = false - private readonly lifecycle: SubprocessTerminalLifecycle + private cleanup: Promise | undefined private terminationSignal: NodeJS.Signals | null = null constructor( @@ -357,21 +356,17 @@ export class E2BTerminalHandle implements SubprocessTerminalHandle { private readonly controlEnvs: Record, private readonly stateDir: string, private readonly graceMs: number, - signal?: AbortSignal, ) { this.pid = handle.pid this.done = this.waitForCommand() - this.lifecycle = new SubprocessTerminalLifecycle({ - done: this.done, - cleanup: () => this.closeOnce(), - signal, - }) } + // TODO(e2b-pgid-identity): Replace retained numeric PTY/session ids when E2B + // exposes identity-bound input, foreground-signal, and cleanup operations. /** @inheritdoc */ - async write(data: Uint8Array): Promise { + async write(data: string): Promise { if (this.topLevelExited) throw new Error('terminal process has exited') - await this.sandbox.pty.sendInput(this.pid, data) + await this.sandbox.pty.sendInput(this.pid, Buffer.from(data, 'utf8')) } /** @inheritdoc */ @@ -413,13 +408,14 @@ export class E2BTerminalHandle implements SubprocessTerminalHandle { } /** @inheritdoc */ - terminate(): void { - this.lifecycle.terminate() - } - - /** @inheritdoc */ - async waitForExit(signal?: AbortSignal): Promise { - return await this.lifecycle.waitForExit(signal) + terminate(): Promise { + if (this.cleanup !== undefined) return this.cleanup + const cleanup = this.closeOnce() + this.cleanup = cleanup + void cleanup.catch((_cleanupFailure: unknown) => { + this.cleanup = undefined + }) + return cleanup } private async waitForCommand(): Promise { @@ -474,7 +470,11 @@ export class E2BTerminalHandle implements SubprocessTerminalHandle { } catch (error: unknown) { if (!(error instanceof SandboxNotFoundError)) throw error } - await this.sandbox.files.remove(this.stateDir).catch(() => {}) + try { + await this.sandbox.files.remove(this.stateDir) + } catch (_adapterPrivateStateRemovalFailure) { + // The terminal is quiescent; a retained sandbox tolerates private residue. + } } } @@ -558,7 +558,6 @@ export async function spawnE2BTerminal( controlEnvs, stateDir, spec.graceMs, - spec.signal, ) } catch (error: unknown) { output.destroy() diff --git a/packages/e2b/subprocess-e2b/tests/terminal.spec.ts b/packages/e2b/subprocess-e2b/tests/terminal.spec.ts index 8ad0eb1dde..6c3ef477af 100644 --- a/packages/e2b/subprocess-e2b/tests/terminal.spec.ts +++ b/packages/e2b/subprocess-e2b/tests/terminal.spec.ts @@ -1,6 +1,5 @@ import { Buffer } from 'node:buffer' import { once } from 'node:events' -import { PassThrough } from 'node:stream' import { Context } from 'cordis' import { describe, expect, it, vi } from 'vitest' import { @@ -14,7 +13,7 @@ import { import type E2BSandboxService from '@deepseek-ai/dsh-e2b' import type { SubprocessTerminalSpawnSpec } from '@deepseek-ai/dsh-subprocess' import E2BSubprocessService from '@deepseek-ai/dsh-subprocess-e2b' -import { E2BTerminalHandle, spawnE2BTerminal } from '../src/terminal.ts' +import { spawnE2BTerminal } from '../src/terminal.ts' function commandError(exitCode: number): CommandExitError { return new CommandExitError({ exitCode, stdout: '', stderr: '', error: `exit ${exitCode}` }) @@ -298,20 +297,20 @@ describe('E2B terminal allocation', () => { await fake.createOptions?.onData(Buffer.from('late bootstrap callback')) expect(output).toBe('requested-shell$ ') - await terminal.write(Buffer.from('echo ok\r')) + await terminal.write('echo ok\r') expect(fake.inputs.at(-1)?.data.toString()).toBe('echo ok\r') await expect(terminal.inspectForeground()).resolves.toEqual({ processGroupId: 456, inputWaiting: false }) await expect(terminal.signalForeground('SIGINT')).resolves.toBe(456) expect(fake.commands).toContain('kill -INT -- -456') - terminal.terminate() + const terminated = terminal.terminate() await expect(terminal.done).resolves.toEqual({ exitCode: null, signal: 'SIGTERM' }) - await expect(terminal.waitForExit()).resolves.toBe(true) + await terminated expect(fake.handle.disconnects).toBe(1) expect(fake.removed).toContain('/runtime/terminal-one') }) - it('inherits only safe ambient values and binds live abort to terminal cleanup', async () => { + it('inherits only safe ambient values and limits the allocation signal to setup', async () => { const fake = new FakeTerminalSandbox() const controller = new AbortController() const terminal = await spawnE2BTerminal( @@ -325,9 +324,10 @@ describe('E2B terminal allocation', () => { expect(environment).not.toContain('DSH_STALE') controller.abort(new Error('stop')) + await terminal.write('still live\r') + expect(fake.inputs.at(-1)?.data.toString()).toBe('still live\r') + await terminal.terminate() await expect(terminal.done).resolves.toEqual({ exitCode: null, signal: 'SIGTERM' }) - await expect(terminal.waitForExit(controller.signal)).resolves.toBe(false) - await expect(terminal.waitForExit()).resolves.toBe(true) }) it('publishes the PTY handle before honoring allocation cancellation', async () => { @@ -553,31 +553,11 @@ describe('E2B terminal lifecycle', () => { fake.handle.succeed(7) await expect(terminal.done).resolves.toEqual({ exitCode: 7, signal: null }) await ended - await expect(terminal.waitForExit()).resolves.toBe(true) - await expect(terminal.write(Buffer.from('late'))).rejects.toThrow('exited') + await expect(terminal.write('late')).rejects.toThrow('exited') fake.foregroundFailure = commandError(1) await expect(terminal.inspectForeground()).resolves.toBeUndefined() await expect(terminal.signalForeground('SIGINT')).rejects.toThrow('cannot resolve foreground process group') - }) - - it('starts cleanup when the lifetime signal is already aborted at handle publication', async () => { - const fake = new FakeTerminalSandbox() - const controller = new AbortController() - controller.abort(new Error('publication cancelled')) - const terminal = new E2BTerminalHandle( - fake.sandbox, - fake.handle.asHandle(), - new PassThrough(), - fake.handle.wait(), - 123, - { TERM: 'dumb' }, - '/runtime/pre-aborted', - 1, - controller.signal, - ) - - await expect(terminal.done).resolves.toEqual({ exitCode: null, signal: 'SIGTERM' }) - await expect(terminal.waitForExit()).resolves.toBe(true) + await terminal.terminate() }) it.each([ @@ -590,18 +570,7 @@ describe('E2B terminal lifecycle', () => { const terminal = await spawnE2BTerminal(runtime(fake), spec(), `/runtime/exit-${exitCode}`) fake.handle.fail(exitCode) await expect(terminal.done).resolves.toEqual(expected) - await expect(terminal.waitForExit(new AbortController().signal)).resolves.toBe(true) - }) - - it('lets an early quiescence observer follow a transport rejection', async () => { - const fake = new FakeTerminalSandbox() - fake.groups = [] - const terminal = await spawnE2BTerminal(runtime(fake), spec(), '/runtime/early-observer') - terminal.output.on('error', () => {}) - const quiescence = terminal.waitForExit() - fake.handle.crash(new Error('transport failed')) - await expect(terminal.done).rejects.toThrow('transport failed') - await expect(quiescence).resolves.toBe(true) + await terminal.terminate() }) it('treats a terminal session containing only zombies as quiescent', async () => { @@ -612,7 +581,7 @@ describe('E2B terminal lifecycle', () => { fake.handle.succeed(0) await expect(terminal.done).resolves.toEqual({ exitCode: 0, signal: null }) - await expect(terminal.waitForExit()).resolves.toBe(true) + await terminal.terminate() expect(fake.commands).toContain( "set -o pipefail; ps -eo sid=,pgid=,stat= | awk '$1 == 123 && $3 !~ /^[ZXx]/ { print $2 }'", ) @@ -625,7 +594,7 @@ describe('E2B terminal lifecycle', () => { fake.handle.succeed(0) await expect(terminal.done).resolves.toEqual({ exitCode: 0, signal: null }) - await expect(terminal.waitForExit()).resolves.toBe(true) + await terminal.terminate() }) it('treats sandbox disappearance during PTY kill as quiescent', async () => { @@ -635,8 +604,7 @@ describe('E2B terminal lifecycle', () => { fake.ptyKillError = new SandboxNotFoundError('sandbox expired') const terminal = await spawnE2BTerminal(runtime(fake), spec({ graceMs: 1 }), '/runtime/expired-pty-kill') - terminal.terminate() - await expect(terminal.waitForExit()).resolves.toBe(true) + await terminal.terminate() expect(fake.ptyKills).toBe(1) }) @@ -647,8 +615,11 @@ describe('E2B terminal lifecycle', () => { fake.ptyKillError = new Error('PTY kill transport failed') const terminal = await spawnE2BTerminal(runtime(fake), spec({ graceMs: 1 }), '/runtime/failed-pty-kill') - terminal.terminate() - await expect(terminal.waitForExit()).rejects.toThrow('PTY kill transport failed') + await expect(terminal.terminate()).rejects.toThrow('PTY kill transport failed') + fake.ptyKillError = undefined + fake.handle.succeed(0) + await terminal.done + await terminal.terminate() }) it.each([ @@ -661,8 +632,8 @@ describe('E2B terminal lifecycle', () => { fake.groups = [] fake.handle.succeed(0) - if (accepted) await expect(terminal.waitForExit()).resolves.toBe(true) - else await expect(terminal.waitForExit()).rejects.toThrow('disconnect failed') + if (accepted) await expect(terminal.terminate()).resolves.toBeUndefined() + else await expect(terminal.terminate()).rejects.toThrow('disconnect failed') }) it('rejects killing the terminal shell and propagates live foreground failures', async () => { @@ -677,23 +648,17 @@ describe('E2B terminal lifecycle', () => { fake.foregroundFailure = commandError(2) await expect(terminal.inspectForeground()).rejects.toBeInstanceOf(CommandExitError) fake.clearOnTerm = true - terminal.terminate() - await terminal.waitForExit() + await terminal.terminate() }) - it('escalates surviving process groups and bounds an observing wait', async () => { + it('escalates surviving process groups', async () => { const fake = new FakeTerminalSandbox() fake.groups = [123, 456] fake.clearOnTerm = false const terminal = await spawnE2BTerminal(runtime(fake), spec({ graceMs: 1 }), '/runtime/escalate') - const controller = new AbortController() - const observing = terminal.waitForExit(controller.signal) - controller.abort() - await expect(observing).resolves.toBe(false) - - terminal.terminate() + const terminating = terminal.terminate() await expect(terminal.done).resolves.toEqual({ exitCode: null, signal: 'SIGKILL' }) - await expect(terminal.waitForExit()).resolves.toBe(true) + await terminating expect(fake.commands).toContain('kill -TERM -- -123 -456') expect(fake.commands).toContain('kill -KILL -- -123 -456') }) @@ -702,49 +667,31 @@ describe('E2B terminal lifecycle', () => { const fake = new FakeTerminalSandbox() fake.groups = [1] const terminal = await spawnE2BTerminal(runtime(fake), spec({ graceMs: 1 }), '/runtime/retry') - terminal.terminate() - await expect(terminal.waitForExit(new AbortController().signal)).rejects.toThrow('unsafe process group 1') + await expect(terminal.terminate()).rejects.toThrow('unsafe process group 1') fake.groups = [] fake.handle.succeed(0) await terminal.done - terminal.terminate() - await expect(terminal.waitForExit()).resolves.toBe(true) + await terminal.terminate() }) it('propagates a process-group signalling transport failure before retry', async () => { const fake = new FakeTerminalSandbox() fake.termFailure = new Error('signal transport failed') const terminal = await spawnE2BTerminal(runtime(fake), spec({ graceMs: 1 }), '/runtime/signal-failure') - terminal.terminate() - await expect(terminal.waitForExit()).rejects.toThrow('signal transport failed') + await expect(terminal.terminate()).rejects.toThrow('signal transport failed') fake.groups = [] fake.handle.succeed(0) await terminal.done - terminal.terminate() - await expect(terminal.waitForExit()).resolves.toBe(true) + await terminal.terminate() const alreadyExited = new FakeTerminalSandbox() alreadyExited.termFailure = commandError(1) const tolerant = await spawnE2BTerminal(runtime(alreadyExited), spec({ graceMs: 1 }), '/runtime/group-exited') - tolerant.terminate() + const tolerantTermination = tolerant.terminate() await expect(tolerant.done).resolves.toEqual({ exitCode: null, signal: 'SIGKILL' }) - await expect(tolerant.waitForExit()).resolves.toBe(true) - }) - - it('normalizes a non-Error cleanup rejection for an observing wait', async () => { - const fake = new FakeTerminalSandbox() - const terminal = await spawnE2BTerminal(runtime(fake), spec(), '/runtime/non-error-cleanup') - fake.commandFailure = 'cleanup transport gone' - terminal.terminate() - await expect(terminal.waitForExit(new AbortController().signal)).rejects.toThrow('cleanup transport gone') - - fake.groups = [] - fake.handle.succeed(0) - await terminal.done - terminal.terminate() - await expect(terminal.waitForExit()).resolves.toBe(true) + await tolerantTermination }) it('keeps command rejection authoritative while cleanup is already waiting', async () => { @@ -753,11 +700,11 @@ describe('E2B terminal lifecycle', () => { fake.removeError = new Error('private state already gone') const terminal = await spawnE2BTerminal(runtime(fake), spec(), '/runtime/reject-during-cleanup') terminal.output.on('error', () => {}) - terminal.terminate() + const cleanup = terminal.terminate() await Promise.resolve() fake.handle.crash(new Error('command transport failed')) await expect(terminal.done).rejects.toThrow('command transport failed') - await expect(terminal.waitForExit()).resolves.toBe(true) + await cleanup }) it('keeps a late command rejection authoritative after PTY kill', async () => { @@ -766,12 +713,12 @@ describe('E2B terminal lifecycle', () => { fake.settleOnPtyKill = false const terminal = await spawnE2BTerminal(runtime(fake), spec({ graceMs: 1 }), '/runtime/reject-after-kill') terminal.output.on('error', () => {}) - terminal.terminate() + const cleanup = terminal.terminate() while (fake.ptyKills === 0) await new Promise(resolve => setTimeout(resolve, 0)) await Promise.resolve() fake.handle.crash(new Error('late command transport failed')) await expect(terminal.done).rejects.toThrow('late command transport failed') - await expect(terminal.waitForExit()).resolves.toBe(true) + await cleanup }) it('reports surviving groups, a surviving top-level pid, and transport failure', async () => { @@ -779,15 +726,13 @@ describe('E2B terminal lifecycle', () => { survivor.clearOnTerm = false survivor.clearOnKill = false const terminal = await spawnE2BTerminal(runtime(survivor), spec({ graceMs: 1 }), '/runtime/survivor') - terminal.terminate() - await expect(terminal.waitForExit()).rejects.toThrow('surviving process groups: 123') + await expect(terminal.terminate()).rejects.toThrow('surviving process groups: 123') const livePid = new FakeTerminalSandbox() livePid.groups = [] livePid.settleOnPtyKill = false const live = await spawnE2BTerminal(runtime(livePid), spec({ graceMs: 1 }), '/runtime/live-pid') - live.terminate() - await expect(live.waitForExit()).rejects.toThrow('surviving pid: 123') + await expect(live.terminate()).rejects.toThrow('surviving pid: 123') livePid.handle.succeed(0) await live.done @@ -798,7 +743,7 @@ describe('E2B terminal lifecycle', () => { crashed.handle.crash('transport gone') await expect(failed.done).rejects.toEqual('transport gone') await expect(outputError).resolves.toMatchObject([{ message: 'transport gone' }]) - await expect(failed.waitForExit()).resolves.toBe(true) + await failed.terminate() }) }) @@ -943,7 +888,7 @@ describe('E2B subprocess terminal service', () => { const terminal = await ctx.subprocess.spawnTerminal(spec()) fake.handle.succeed(0) await terminal.done - await terminal.waitForExit() + await terminal.terminate() const signals = fake.commands.filter(command => command.startsWith('kill -')).length await fiber.dispose() expect(fake.commands.filter(command => command.startsWith('kill -'))).toHaveLength(signals) @@ -961,6 +906,6 @@ describe('E2B subprocess terminal service', () => { fake.groups = [] await fiber.dispose() - await expect(terminal.waitForExit()).resolves.toBe(true) + await expect(terminal.terminate()).resolves.toBeUndefined() }) }) diff --git a/packages/fs/README.i18n.yaml b/packages/fs/README.i18n.yaml index ae44721735..f4719447aa 100644 --- a/packages/fs/README.i18n.yaml +++ b/packages/fs/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/fs/README.md -README.md: 96bbe6c95bd2aca7e66cf2d57abb3f056cd19390 -README.zh.md: 7f091b4f0955b847d21b8b3423dab421e610cda3 +README.md: b15012e882b60847e1ad22edf08d1202ba64fe5b +README.zh.md: 628f6c74894bc67559d49f7cf5d1378d0ece2382 diff --git a/packages/fs/README.md b/packages/fs/README.md index 96bbe6c95b..b15012e882 100644 --- a/packages/fs/README.md +++ b/packages/fs/README.md @@ -14,7 +14,7 @@ The filesystem stack: a provider seam (execution-world paths, bounded text IO, a | `tool-fs/` | Model-facing `read`/`write`/`edit` tools AND the executor (reads via `ctx.fs`, owns read windowing, dispatches `fs/*`); preserves filesystem semantics for session-cwd-relative paths and advertises sandbox escalation fields when the mounted `ctx.fs` confines | (registers on `ctx.tools`) | | `tool-fs-search/` | Model-facing `glob`/`grep` discovery tools when `rg` is available on the bash executor `PATH`, backed by fixed ripgrep commands through `ctx.bash`, NOT by `ctx.fs` provider methods | (registers on `ctx.tools`) | -The interface lives at `fs/fs/`. A sandboxed, remote, or project-scoped filesystem backend can replace `fs-local` without touching the seam, the policy gate, or the model-facing tool schemas: `fs-sandbox` provides an in-process path fence over the shared sandbox mode ([decision](../../.agents/notes/implemented/feature/2026-07-14-cross-family-fs-sandbox.md)), while `fs-e2b` places file state in the remote runtime shared with the E2B subprocess provider ([POC decision](../../.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md)). The policy (`fs-policy/`) is a plugin that participates only through the `fs/*` event gate, not a service the tool injects — so dropping it gracefully loses the policy and leaves the unconstrained bare provider rather than breaking the tool. A deployment that loads `tool-fs/` is expected to also load it. The mode fence and the read-before-edit gate are orthogonal and compose. Discovery (`tool-fs-search/`) deliberately does NOT extend the provider seam: search is a process-backed `rg` workflow on the bash executor, so filesystem backends stay free of a universal search contract; its tools register only when that executor can find `rg`, and its results are follow-up-readable when the bash workdir and the `read` root are the same workspace (the co-located deployment its README documents). +The interface lives at `fs/fs/`. A sandboxed, remote, or project-scoped filesystem backend can replace `fs-local` without touching the seam, the policy gate, or the model-facing tool schemas: `fs-sandbox` provides an in-process path fence over the shared sandbox mode ([decision](../../.agents/notes/implemented/feature/2026-07-14-cross-family-fs-sandbox.md)), while `fs-e2b` places file state in the remote execution world shared with the E2B subprocess provider ([decision](../../.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md)). The policy (`fs-policy/`) is a plugin that participates only through the `fs/*` event gate, not a service the tool injects — so dropping it gracefully loses the policy and leaves the unconstrained bare provider rather than breaking the tool. A deployment that loads `tool-fs/` is expected to also load it. The mode fence and the read-before-edit gate are orthogonal and compose. Discovery (`tool-fs-search/`) deliberately does NOT extend the provider seam: search is a process-backed `rg` workflow on the bash executor, so filesystem backends stay free of a universal search contract; its tools register only when that executor can find `rg`, and its results are follow-up-readable when the bash workdir and the `read` root are the same workspace (the co-located deployment its README documents). ## No timeouts on file IO diff --git a/packages/fs/README.zh.md b/packages/fs/README.zh.md index 7f091b4f09..628f6c7489 100644 --- a/packages/fs/README.zh.md +++ b/packages/fs/README.zh.md @@ -14,7 +14,7 @@ | `tool-fs/` | 面向模型的 `read`/`write`/`edit` 工具以及执行器(通过 `ctx.fs` 读取,拥有读取窗口逻辑,分派 `fs/*`);为会话 cwd 相对路径保留文件系统语义,并在已挂载的 `ctx.fs` 实施约束时声明沙箱升级字段 | (注册到 `ctx.tools`) | | `tool-fs-search/` | 面向模型的 `glob`/`grep` 发现工具;当 `rg` 位于 bash 执行器 `PATH` 上时注册,通过 `ctx.bash` 运行固定 ripgrep 命令,而不是使用 `ctx.fs` 提供方方法 | (注册到 `ctx.tools`) | -接口位于 `fs/fs/`。沙箱化、远程或限定项目作用域的文件系统后端可以替换 `fs-local`,而无需更改 seam、政策门禁或面向模型的工具 schema:`fs-sandbox` 基于共享沙箱模式提供进程内路径围栏([决策](../../.agents/notes/implemented/feature/2026-07-14-cross-family-fs-sandbox.md)),而 `fs-e2b` 则把文件状态置于与 E2B 进程管理提供方共享的远程运行时中([POC 决策](../../.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md))。政策(`fs-policy/`)是一个只通过 `fs/*` 事件门禁参与的插件,不是工具注入的服务;因此移除它会平稳失去政策,留下不受约束的裸提供方,而不会破坏工具。加载 `tool-fs/` 的部署也应加载该插件。模式围栏与编辑前读取门禁彼此正交,可以组合。发现(`tool-fs-search/`)有意不扩展提供方 seam:搜索是在 bash 执行器上运行 `rg`、由进程支持的工作流,因此文件系统后端无需承担通用搜索契约;只有当执行器能找到 `rg` 时,其工具才会注册。如果 bash 工作目录与 `read` 根目录是同一工作区,结果就能继续读取,这也是其 README 所述的共置部署。 +接口位于 `fs/fs/`。沙箱化、远程或限定项目作用域的文件系统后端可以替换 `fs-local`,而无需更改 seam、政策门禁或面向模型的工具 schema:`fs-sandbox` 基于共享沙箱模式提供进程内路径围栏([决策](../../.agents/notes/implemented/feature/2026-07-14-cross-family-fs-sandbox.md)),而 `fs-e2b` 则把文件状态置于与 E2B 进程管理提供方共享的远程执行世界中([决策](../../.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md))。政策(`fs-policy/`)是一个只通过 `fs/*` 事件门禁参与的插件,不是工具注入的服务;因此移除它会平稳失去政策,留下不受约束的裸提供方,而不会破坏工具。加载 `tool-fs/` 的部署也应加载该插件。模式围栏与编辑前读取门禁彼此正交,可以组合。发现(`tool-fs-search/`)有意不扩展提供方 seam:搜索是在 bash 执行器上运行 `rg`、由进程支持的工作流,因此文件系统后端无需承担通用搜索契约;只有当执行器能找到 `rg` 时,其工具才会注册。如果 bash 工作目录与 `read` 根目录是同一工作区,结果就能继续读取,这也是其 README 所述的共置部署。 ## 文件 I/O 不设超时 From 677541d1230a78fe7af64fd28f0afdaba00225ab Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 22:04:55 +0800 Subject: [PATCH 092/108] fix(e2b): close proven provider boundary gaps --- docs/cordis-catalog/services.md | 2 +- packages/e2b/e2b/README.i18n.yaml | 4 +- packages/e2b/e2b/README.md | 2 +- packages/e2b/e2b/README.zh.md | 2 +- packages/e2b/e2b/src/index.ts | 21 ++--- packages/e2b/e2b/tests/e2b.spec.ts | 6 +- packages/e2b/fs-e2b/src/invariant.ts | 5 +- packages/e2b/subprocess-e2b/README.i18n.yaml | 4 +- packages/e2b/subprocess-e2b/README.md | 6 +- packages/e2b/subprocess-e2b/README.zh.md | 6 +- .../e2b/subprocess-e2b/src/environment.ts | 2 +- packages/e2b/subprocess-e2b/src/invariant.ts | 5 +- packages/e2b/subprocess-e2b/src/process.ts | 15 ++-- packages/e2b/subprocess-e2b/src/terminal.ts | 85 +++++++++++++------ .../subprocess-e2b/tests/subprocess.spec.ts | 11 ++- .../e2b/subprocess-e2b/tests/terminal.spec.ts | 61 +++++++++++++ 16 files changed, 167 insertions(+), 70 deletions(-) diff --git a/docs/cordis-catalog/services.md b/docs/cordis-catalog/services.md index cdd2d87435..3942c0696d 100644 --- a/docs/cordis-catalog/services.md +++ b/docs/cordis-catalog/services.md @@ -591,7 +591,7 @@ Owns one lazily consumable E2B SDK handle and its final kill/pause/leave decisio async getSandbox(): Promise ``` -Source: [`packages/e2b/e2b/src/index.ts:109`](../../packages/e2b/e2b/src/index.ts) +Source: [`packages/e2b/e2b/src/index.ts:108`](../../packages/e2b/e2b/src/index.ts) ## `ctx.fs` — `FileSystem` (abstract seam) diff --git a/packages/e2b/e2b/README.i18n.yaml b/packages/e2b/e2b/README.i18n.yaml index fb645c2e36..7d6b503ef4 100644 --- a/packages/e2b/e2b/README.i18n.yaml +++ b/packages/e2b/e2b/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/e2b/e2b/README.md -README.md: 402903184934903eceb36a04d670e4490879ac65 -README.zh.md: 1fec6ab66858f04a47c8a2a7cb5f6a8907ab6cac +README.md: 6881556dc18497956966aff74adc085d8e8620d3 +README.zh.md: aea13f341f903e6c47b7484f477bf54cebf101c6 diff --git a/packages/e2b/e2b/README.md b/packages/e2b/e2b/README.md index 4029031849..6881556dc1 100644 --- a/packages/e2b/e2b/README.md +++ b/packages/e2b/e2b/README.md @@ -24,7 +24,7 @@ Shared lifecycle owner for one E2B sandbox. The filesystem and subprocess adapte `apiKey` is optional and otherwise reads `E2B_API_KEY`; the key configures the host SDK connection and is never installed in the sandbox. `cwd` defaults to `/home/user/workspace` and must be an absolute POSIX path. `timeoutMs` defaults to five minutes. `onTimeout` is `pause` by default and accepts `pause | kill`; it applies only when this service creates a sandbox. Pause-on-timeout enables E2B auto-resume so the shared SDK handle wakes on its next operation. `onDispose` defaults to `kill` and accepts `kill | pause | leave`. -Set `sandboxId` to reconnect a running or paused sandbox instead of creating one. E2B resumes a paused sandbox during connect; `template` is creation-only and cannot accompany `sandboxId`. Omitting `template` uses E2B's default base template. +Set `sandboxId` to reconnect a running or paused sandbox instead of creating one. E2B resumes a paused sandbox during connect; `template` and `onTimeout` are creation-only and cannot accompany `sandboxId`. Omitting `template` uses E2B's default base template. ## Lifecycle and ownership diff --git a/packages/e2b/e2b/README.zh.md b/packages/e2b/e2b/README.zh.md index 1fec6ab668..aea13f341f 100644 --- a/packages/e2b/e2b/README.zh.md +++ b/packages/e2b/e2b/README.zh.md @@ -24,7 +24,7 @@ `apiKey` 可省略;省略时读取 `E2B_API_KEY`。该密钥只配置宿主 SDK 连接,绝不会安装进沙箱。`cwd` 默认为 `/home/user/workspace`,并且必须是绝对 POSIX 路径。`timeoutMs` 默认为 5 分钟。`onTimeout` 默认为 `pause`,接受 `pause | kill`;它只在本服务创建沙箱时生效。超时时 pause 会启用 E2B 自动恢复,使共享 SDK 句柄在下一次操作时唤醒。`onDispose` 默认为 `kill`,接受 `kill | pause | leave`。 -设置 `sandboxId` 可重新连接正在运行或已经暂停的沙箱,而不是创建新沙箱。连接时,E2B 会恢复已经暂停的沙箱;`template` 仅用于创建,不能与 `sandboxId` 同时使用。省略 `template` 时使用 E2B 的默认基础模板。 +设置 `sandboxId` 可重新连接正在运行或已经暂停的沙箱,而不是创建新沙箱。连接时,E2B 会恢复已经暂停的沙箱;`template` 和 `onTimeout` 仅用于创建,不能与 `sandboxId` 同时使用。省略 `template` 时使用 E2B 的默认基础模板。 ## 生命周期与所有权 diff --git a/packages/e2b/e2b/src/index.ts b/packages/e2b/e2b/src/index.ts index 9e8960a685..3285d82395 100644 --- a/packages/e2b/e2b/src/index.ts +++ b/packages/e2b/e2b/src/index.ts @@ -91,7 +91,6 @@ interface ResolvedConfig { interface SchemaResolvedConfig extends Config { cwd: string timeoutMs: number - onTimeout: E2BTimeoutMode onDispose: E2BDisposeMode } @@ -113,7 +112,7 @@ export class E2BSandboxService extends Service { template: z.string(), cwd: z.string().default('/home/user/workspace'), timeoutMs: z.number().default(300_000), - onTimeout: z.union(['kill', 'pause'] as const).default('pause'), + onTimeout: z.union(['kill', 'pause'] as const), onDispose: z.union(['kill', 'pause', 'leave'] as const).default('kill'), }) @@ -121,17 +120,12 @@ export class E2BSandboxService extends Service { readonly cwd: string /** Remote directory reserved for adapter-owned process and terminal state. */ readonly runtimeRoot: string - /** Whether this service creates a sandbox rather than reconnecting one. */ - readonly created: boolean - /** Configured action when a newly created sandbox reaches its lifetime. */ - readonly timeoutMode: E2BTimeoutMode - /** Configured final sandbox disposition. */ - readonly disposeMode: E2BDisposeMode /** Sandbox id once E2B has created or resolved the remote runtime. */ readonly sandboxId: Promise private readonly config: ResolvedConfig private readonly ready: Promise + private readonly created: boolean private failedSetupSandbox: Sandbox | undefined private disposed = false @@ -144,17 +138,15 @@ export class E2BSandboxService extends Service { apiKey: apiKey ?? '', cwd: resolved.cwd, timeoutMs: resolved.timeoutMs, - onTimeout: resolved.onTimeout, + onTimeout: config.onTimeout ?? 'pause', onDispose: resolved.onDispose, ...(config.sandboxId !== undefined ? { sandboxId: config.sandboxId } : {}), ...(config.template !== undefined ? { template: config.template } : {}), } - this.validate() + this.validate(config) this.cwd = this.config.cwd this.runtimeRoot = posix.join(this.cwd, '.dsh-e2b') this.created = this.config.sandboxId === undefined - this.timeoutMode = this.config.onTimeout - this.disposeMode = this.config.onDispose this.ready = this.open() // A deployment may load the owner before any adapter uses it. Keep a // failed eager connection observed; getSandbox() still returns the error. @@ -218,7 +210,7 @@ export class E2BSandboxService extends Service { return sandbox } - private validate(): void { + private validate(input: Config): void { if (this.config.apiKey.length === 0) { throw new Error('dsh-e2b: configure apiKey or set E2B_API_KEY') } @@ -234,6 +226,9 @@ export class E2BSandboxService extends Service { if (this.config.sandboxId !== undefined && this.config.template !== undefined) { throw new Error('dsh-e2b: template applies only when creating; omit it when sandboxId reconnects') } + if (this.config.sandboxId !== undefined && input.onTimeout !== undefined) { + throw new Error('dsh-e2b: onTimeout applies only when creating; omit it when sandboxId reconnects') + } } private async open(): Promise { diff --git a/packages/e2b/e2b/tests/e2b.spec.ts b/packages/e2b/e2b/tests/e2b.spec.ts index 46ba452210..af55a2419a 100644 --- a/packages/e2b/e2b/tests/e2b.spec.ts +++ b/packages/e2b/e2b/tests/e2b.spec.ts @@ -90,9 +90,6 @@ describe('E2BSandboxService', () => { await expect(service.sandboxId).resolves.toBe(E2BSandboxId('sandbox-1')) expect(service.cwd).toBe('/home/user/workspace') expect(service.runtimeRoot).toBe('/home/user/workspace/.dsh-e2b') - expect(service.created).toBe(true) - expect(service.timeoutMode).toBe('pause') - expect(service.disposeMode).toBe('kill') expect(sdk.create).toHaveBeenCalledWith({ apiKey: 'test-key', timeoutMs: 300_000, @@ -225,7 +222,6 @@ describe('E2BSandboxService', () => { ctx.logger.error = ((error: unknown) => { errors.push(error) }) as typeof ctx.logger.error const fiber = await ctx.plugin(E2BSandboxService, { apiKey: 'test-key', - onTimeout: reconnect ? 'kill' : 'pause', onDispose: 'pause', ...(reconnect ? { sandboxId: 'existing' } : {}), }) @@ -248,7 +244,6 @@ describe('E2BSandboxService', () => { }) await ctx.e2b.getSandbox() - expect(ctx.e2b.created).toBe(false) expect(sdk.connect).toHaveBeenCalledWith('existing', { apiKey: 'test-key', timeoutMs: 90_000 }) expect(sdk.create).not.toHaveBeenCalled() await fiber.dispose() @@ -330,6 +325,7 @@ describe('E2BSandboxService', () => { [{ apiKey: 'x', timeoutMs: 0 }, /positive finite/], [{ apiKey: 'x', sandboxId: '' }, /sandboxId must be non-empty/], [{ apiKey: 'x', sandboxId: 'one', template: 'two' }, /template applies only/], + [{ apiKey: 'x', sandboxId: 'one', onTimeout: 'kill' }, /onTimeout applies only/], ] as const)('fails self-contained configuration before opening E2B: %j', async (config, message) => { vi.stubEnv('E2B_API_KEY', '') const ctx = new Context() diff --git a/packages/e2b/fs-e2b/src/invariant.ts b/packages/e2b/fs-e2b/src/invariant.ts index 6294d66ace..9f14bb37a6 100644 --- a/packages/e2b/fs-e2b/src/invariant.ts +++ b/packages/e2b/fs-e2b/src/invariant.ts @@ -1,4 +1,7 @@ -/** Package-owned invariant companion for `@deepseek-ai/dsh-fs-e2b`. */ +/** + * Package-owned invariant companion for `@deepseek-ai/dsh-fs-e2b`. + * @module @deepseek-ai/dsh-fs-e2b/invariant + */ /* jscpd:ignore-start */ import type { Context } from 'cordis' diff --git a/packages/e2b/subprocess-e2b/README.i18n.yaml b/packages/e2b/subprocess-e2b/README.i18n.yaml index 20ba9b597e..9ac46f1bcb 100644 --- a/packages/e2b/subprocess-e2b/README.i18n.yaml +++ b/packages/e2b/subprocess-e2b/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/e2b/subprocess-e2b/README.md -README.md: 9e044c13b673461a876e016ec560db4897a27c66 -README.zh.md: b9180c0c0679915fae7a1db22353613708392a8c +README.md: 9d563c91c149d28718159af1b1f8ddbd6652dc44 +README.zh.md: be189f41818c66aeff2bed7b1851b82ca08f08f4 diff --git a/packages/e2b/subprocess-e2b/README.md b/packages/e2b/subprocess-e2b/README.md index 9e044c13b6..9d563c91c1 100644 --- a/packages/e2b/subprocess-e2b/README.md +++ b/packages/e2b/subprocess-e2b/README.md @@ -9,12 +9,12 @@ E2B implementation of the [`@deepseek-ai/dsh-subprocess`](../../subprocess/subpr - **Asynchronous remote start** — the synchronous seam returns a handle immediately while `Sandbox.commands.run(..., { background: true })` starts remotely. `pid` is `-1` until the wrapper publishes and the adapter validates its process-group id; stdin and ordinary observation wait for that publication. An owned startup signal aborts environment and private-state preparation before allocation; once allocation begins, cancellation waits for a provisional SDK handle it can clean. - **Execution-world coordinates** — `cwd` and private `runtimeRoot` come from the shared owner; executable lookup verifies absolute paths or resolves a bare name against the sandbox PATH plus explicit overrides. - **Linux process groups** — a quoted wrapper starts each argv under `exec setsid --wait` and records its actual process-group id plus private status files beneath `ctx.e2b.runtimeRoot/processes`. The handle waits for that file instead of treating the SDK command PID as its published identity. Termination signals the negative recorded id with `SIGTERM`, waits the caller's `graceMs`, then escalates to `SIGKILL` and the SDK kill fallback; TERM delivery or probe failures also force that escalation. Process-table probes treat groups containing only zombie or dead entries as quiescent. Force cleanup succeeds only after a bounded probe finds the group empty; otherwise `waitForExit()` exposes a retryable failure, while proven quiescence makes later termination a no-op. Publication and monitoring failures apply the same cleanup transaction before rejecting. Service disposal rejects new starts, terminates and joins every retained process group, then awaits SDK settlement and private cleanup before the sandbox owner disposes. -- **Environment boundary** — one trusted control-shell probe transports the sandbox environment as base64 ASCII for one strict UTF-8 decode; the wrapper then removes ambient `DSH_*` and credential-shaped (`*KEY*`, `*SECRET*`, `*TOKEN*`) names and restores every valid `spec.env` entry as an explicit caller opt-in. Empty names, `=`, and NUL framing violations reject before launch. Subsequent E2B command and PTY login shells receive a fresh randomized root-level `HOME` plus empty overrides for every scrubbed ambient name before user profiles can run; the requested argv receives the serialized environment afterward. Host ambient variables never enter the sandbox implicitly. Private environment files are removed after consumption, and failed command or terminal setup removes its private state before rejecting. +- **Environment boundary** — one trusted control-shell probe resolves the sandbox user's login home from its passwd entry and transports the sandbox environment as base64 ASCII for one strict UTF-8 decode; the wrapper then removes ambient `DSH_*` and credential-shaped (`*KEY*`, `*SECRET*`, `*TOKEN*`) names and restores every valid `spec.env` entry as an explicit caller opt-in. Empty names, `=`, and NUL framing violations reject before launch. Subsequent E2B command and PTY login shells receive a fresh randomized root-level `HOME` plus empty overrides for every scrubbed ambient name before user profiles can run; the requested argv receives the serialized environment afterward without changing the template's umask. Host ambient variables never enter the sandbox implicitly. Private environment files are removed after consumption, and failed command or terminal setup removes its private state before rejecting. - **Stdio projection** — the remote wrapper branches raw bytes into optional bounded spill files, frames each live chunk as newline-delimited base64 ASCII, and the host incrementally restores bytes across arbitrary SDK callback boundaries. Pipe mode writes those bytes to host Node streams; inherit mode writes them to the harness process streams; collect mode retains a bounded host tail with offset reads. The wrapper publishes the direct command status before waiting for inherited writers. For collect or inherit output, the adapter disconnects an incomplete SDK stream after `graceMs`, withholds its partial spill, and returns that status while retaining the remote group for `waitForExit()` and termination. Natural raw-pipe completion instead awaits lossless transport and preserves backpressure; explicit termination destroys the host pipes and releases blocked output before remote cleanup. Batch and streaming stdin use the SDK handle. -- **Terminal sessions** — `spawnTerminal()` uses E2B's byte PTY API, installs the exact argv and scrubbed environment through private mode-`0600` files, reports the foreground process group, sends real signals, and tears down every live group in the remote terminal session through one retryable awaited `terminate()`; zombie-only groups are already quiescent. A private random output boundary discards the E2B bootstrap shell's prompt and echoed runner command while preserving every requested-process byte, including its first prompt. PTY allocation is awaited through handle publication before cancellation is observed, so owned rollback can clean the published handle. Setup and teardown own the private state transaction, abort pending setup during service disposal, fence publication, and retain an unproven setup cleanup for disposal retry. Prompt detection, scrollback, readiness, and owner policy remain in `dsh-pty-local`. +- **Terminal sessions** — `spawnTerminal()` uses E2B's byte PTY API, installs the exact argv and scrubbed environment through private mode-`0600` files, reports the foreground process group, sends real signals, and tears down every live group in the remote terminal session through one retryable awaited `terminate()`; termination rejects new handle operations, aborts and joins in-flight writes, inspections, and signals, and treats zombie-only groups as quiescent. A private random output boundary discards the E2B bootstrap shell's prompt and echoed runner command while preserving every requested-process byte, including its first prompt. PTY allocation is awaited through handle publication before cancellation is observed, so owned rollback can clean the published handle. Setup and teardown own the private state transaction, abort pending setup during service disposal, fence publication, and retain an unproven setup cleanup for disposal retry. Prompt detection, scrollback, readiness, and owner policy remain in `dsh-pty-local`. - **Sandbox disappearance** — `SandboxNotFoundError` during process or terminal liveness, termination, rollback, or disconnect proves the remote execution world cannot retain work, so cleanup treats it as quiescent; unrelated failures remain observable. -The base E2B image supplies the runtime and Bash/GNU utilities this adapter invokes: `node`, `bash`, `setsid`, `ps`, `awk`, `tr`, `env`, `base64`, `chmod`, `tee`, `head`, `rm`, and `kill`. A custom template must retain compatible commands and E2B PTY support. +The base E2B image supplies the runtime and Bash/GNU utilities this adapter invokes: `node`, `bash`, `setsid`, `ps`, `awk`, `tr`, `env`, `base64`, `chmod`, `tee`, `head`, `rm`, `kill`, `id`, and `getent`. A custom template must retain compatible commands and E2B PTY support. ## Model Experience diff --git a/packages/e2b/subprocess-e2b/README.zh.md b/packages/e2b/subprocess-e2b/README.zh.md index b9180c0c06..be189f4181 100644 --- a/packages/e2b/subprocess-e2b/README.zh.md +++ b/packages/e2b/subprocess-e2b/README.zh.md @@ -9,12 +9,12 @@ - **异步远程启动**:同步 seam 会立即返回一个句柄,同时由 `Sandbox.commands.run(..., { background: true })` 在远程启动进程。包装层发布进程组 ID 并由适配器完成验证之前,`pid` 为 `-1`;stdin 和常规观察会等待该发布。自有启动信号会在分配前中止环境和私有状态准备;分配开始后,取消会等待可清理的临时 SDK 句柄。 - **执行世界坐标**:`cwd` 和私有 `runtimeRoot` 来自共享所有者;可执行文件查找会验证绝对路径,或根据沙箱 PATH 加显式覆盖来解析裸名称。 - **Linux 进程组**:带引号保护的包装层会在 `exec setsid --wait` 下启动每组 argv,并在 `ctx.e2b.runtimeRoot/processes` 下记录实际进程组 ID 和私有状态文件。句柄会等待该文件,而不会把 SDK 命令 PID 当作已发布的身份。终止操作以记录的负数 ID 发送 `SIGTERM`,等待调用方的 `graceMs`,再升级到 `SIGKILL` 和 SDK kill 回退;TERM 信号发送或探测失败也会强制触发该升级。进程表探测会把仅含僵尸或已死亡条目的进程组视为完全停稳。强制清理只有在有界探测发现进程组为空后才算成功;否则 `waitForExit()` 会公开可重试的失败,而已证明的完全停稳会让后续终止操作不再执行任何动作。发布失败与监控失败都会在拒绝前执行同一清理事务。服务 dispose(资源释放)会拒绝新的启动请求、终止并等待每个保留进程组退出,再等待 SDK 结算和私有清理完成,之后沙箱所有者才会释放。 -- **环境边界**:一次受信任的控制 shell 探测会以 base64 ASCII 传输沙箱环境,再进行一次严格 UTF-8 解码;随后包装层移除环境中的 `DSH_*` 和形似凭据的名称(`*KEY*`、`*SECRET*`、`*TOKEN*`),并把每个有效的 `spec.env` 条目恢复为调用方显式选择。空名称、`=` 和违反 NUL 分帧规则的条目会在启动前被拒绝。在用户 profile 脚本运行前,此后的 E2B 命令 shell 与 PTY 登录 shell 会获得位于根目录下、全新随机生成的 `HOME`,并为每个被清理的环境变量名设置空值覆盖;之后,请求的 argv 才会接收序列化环境。宿主环境变量绝不会隐式进入沙箱。私有环境文件在使用后会被删除;命令或终端设置失败时,会先删除其私有状态再拒绝。 +- **环境边界**:一次受信任的控制 shell 探测会从 passwd 条目解析沙箱用户的登录主目录,以 base64 ASCII 传输沙箱环境,再进行一次严格 UTF-8 解码;随后包装层移除环境中的 `DSH_*` 和形似凭据的名称(`*KEY*`、`*SECRET*`、`*TOKEN*`),并把每个有效的 `spec.env` 条目恢复为调用方显式选择。空名称、`=` 和违反 NUL 分帧规则的条目会在启动前被拒绝。在用户 profile 脚本运行前,此后的 E2B 命令 shell 与 PTY 登录 shell 会获得位于根目录下、全新随机生成的 `HOME`,并为每个被清理的环境变量名设置空值覆盖;之后,请求的 argv 会在不改变模板 umask 的前提下接收序列化环境。宿主环境变量绝不会隐式进入沙箱。私有环境文件在使用后会被删除;命令或终端设置失败时,会先删除其私有状态再拒绝。 - **stdio 投影**:远程包装层先把原始字节分流到可选的有界 spill 文件,再把每个实时分片编码为换行分隔的 base64 ASCII 帧;宿主会跨任意 SDK 回调边界增量恢复字节。pipe 模式把这些字节写入宿主 Node 流;inherit 模式把字节写入 harness 进程流;collect 模式保留有界的宿主尾部,并支持基于偏移量读取。包装层会在等待继承管道的写入方之前发布直接命令状态。对于 collect 或 inherit 输出,超过 `graceMs` 后,适配器会断开未完成的 SDK 流,不公开其中不完整的 spill,并返回该状态,同时保留远程进程组供 `waitForExit()` 和终止操作使用。原始 pipe 自然完成时,会等待无损传输完成并保留背压;显式终止则会销毁宿主 pipe,并在远程清理前释放受阻的输出写入。批量 stdin 和流式 stdin 都使用 SDK 句柄。 -- **终端会话**:`spawnTerminal()` 使用 E2B 的字节 PTY API,以 mode 为 `0600` 的私有文件传入原样 argv 与清理后的环境,报告前台进程组,发送真实信号,并通过一项可重试且须等待的 `terminate()` 清理远程终端会话中仍存活的每个进程组;仅含僵尸进程的进程组已经完全停稳。私有随机输出边界会丢弃 E2B 引导 shell 的提示符和回显的 runner 命令,同时保留请求进程的每个字节,包括其第一个提示符。PTY 分配会一直等待到句柄发布后才观察取消,以便由承担清理责任的回滚清理已发布句柄。setup 与 teardown 负责私有状态事务,在服务 dispose 期间中止待处理的 setup、阻止发布,并保留未证明已完成的 setup 清理事务,供 dispose 重试。提示符检测、scrollback、就绪状态与所有者策略仍归 `dsh-pty-local` 所有。 +- **终端会话**:`spawnTerminal()` 使用 E2B 的字节 PTY API,以 mode 为 `0600` 的私有文件传入原样 argv 与清理后的环境,报告前台进程组,发送真实信号,并通过一项可重试且须等待的 `terminate()` 清理远程终端会话中仍存活的每个进程组;终止会拒绝新的句柄操作,中止并等待在途写入、检查和信号操作结算,并把仅含僵尸进程的进程组视为已经完全停稳。私有随机输出边界会丢弃 E2B 引导 shell 的提示符和回显的 runner 命令,同时保留请求进程的每个字节,包括其第一个提示符。PTY 分配会一直等待到句柄发布后才观察取消,以便由承担清理责任的回滚清理已发布句柄。setup 与 teardown 负责私有状态事务,在服务 dispose 期间中止待处理的 setup、阻止发布,并保留未证明已完成的 setup 清理事务,供 dispose 重试。提示符检测、scrollback、就绪状态与所有者策略仍归 `dsh-pty-local` 所有。 - **沙箱消失**:在进程或终端的存活探测、终止、回滚或断开连接期间出现 `SandboxNotFoundError`,证明远程执行环境无法保留工作,因此清理会将其视为完全停稳;其他故障仍可观察。 -基础 E2B 镜像提供该适配器调用的运行时和 Bash/GNU 工具:`node`、`bash`、`setsid`、`ps`、`awk`、`tr`、`env`、`base64`、`chmod`、`tee`、`head`、`rm` 和 `kill`。自定义模板必须保留兼容的命令和 E2B PTY 支持。 +基础 E2B 镜像提供该适配器调用的运行时和 Bash/GNU 工具:`node`、`bash`、`setsid`、`ps`、`awk`、`tr`、`env`、`base64`、`chmod`、`tee`、`head`、`rm`、`kill`、`id` 和 `getent`。自定义模板必须保留兼容的命令和 E2B PTY 支持。 ## 模型体验 diff --git a/packages/e2b/subprocess-e2b/src/environment.ts b/packages/e2b/subprocess-e2b/src/environment.ts index 907810a1d7..8daae9ccfd 100644 --- a/packages/e2b/subprocess-e2b/src/environment.ts +++ b/packages/e2b/subprocess-e2b/src/environment.ts @@ -29,7 +29,7 @@ export async function readRemoteEnvironment(sandbox: Sandbox, signal?: AbortSign // TODO(e2b-replace-environment): Remove this ambient probe when E2B can start // a command with a replacement environment instead of merged overrides. const result = await sandbox.commands.run( - 'set -o pipefail; printf \'%s\' "$PWD" | base64 -w 0; printf \'\\n\'; env -0 | base64 -w 0', + 'set -o pipefail; dsh_e2b_passwd="$(getent passwd "$(id -u)")"; IFS=: read -r _ _ _ _ _ dsh_e2b_home _ <<<"$dsh_e2b_passwd"; test -n "$dsh_e2b_home" -a -d "$dsh_e2b_home"; printf \'%s\' "$dsh_e2b_home" | base64 -w 0; printf \'\\n\'; env -0 | base64 -w 0', { envs: e2bControlEnvs(), ...(signal === undefined ? {} : { signal }) }, ) const lines = result.stdout.trim().split('\n') diff --git a/packages/e2b/subprocess-e2b/src/invariant.ts b/packages/e2b/subprocess-e2b/src/invariant.ts index 4416175b1b..9f8b8fb739 100644 --- a/packages/e2b/subprocess-e2b/src/invariant.ts +++ b/packages/e2b/subprocess-e2b/src/invariant.ts @@ -1,4 +1,7 @@ -/** Package-owned invariant companion for `@deepseek-ai/dsh-subprocess-e2b`. */ +/** + * Package-owned invariant companion for `@deepseek-ai/dsh-subprocess-e2b`. + * @module @deepseek-ai/dsh-subprocess-e2b/invariant + */ /* jscpd:ignore-start */ import type { Context } from 'cordis' diff --git a/packages/e2b/subprocess-e2b/src/process.ts b/packages/e2b/subprocess-e2b/src/process.ts index 02e1583e56..83dedd4e97 100644 --- a/packages/e2b/subprocess-e2b/src/process.ts +++ b/packages/e2b/subprocess-e2b/src/process.ts @@ -104,7 +104,6 @@ function commandText(spec: SubprocessSpawnSpec, paths: RemotePaths): string { : `2> >(${encoder} >&2 2>/dev/null)` const inner = [ 'set +e', - 'umask 077', 'dsh_e2b_env_bin=$1', 'dsh_e2b_node=$2', 'dsh_e2b_ps=$3', @@ -520,6 +519,8 @@ export class E2BSubprocessHandle implements SubprocessHandle { () => true, ) while (true) { + // TODO(e2b-publication-cancel): Join cancellation to the existing + // termination transaction before aborting an in-flight SDK file read. const raw = await sandbox.files.read(this.paths.pid) const value = raw.trim() if (value.length > 0) { @@ -543,6 +544,8 @@ export class E2BSubprocessHandle implements SubprocessHandle { result => ({ kind: 'result', result }), (error: unknown) => ({ kind: 'error', error }), ) + const hasPipeOutput = this.spec.stdio.stdout === 'pipe' || this.spec.stdio.stderr === 'pipe' + let completed = hasPipeOutput ? await settlement : undefined while (true) { const rawStatus = (await sandbox.files.read(this.paths.status)).trim() if (rawStatus.length > 0) { @@ -550,19 +553,19 @@ export class E2BSubprocessHandle implements SubprocessHandle { if (!/^(?:0|[1-9][0-9]*)$/.test(rawStatus) || !Number.isSafeInteger(exitCode) || exitCode > 255) { throw new Error(`subprocess-e2b: remote wrapper published invalid exit code ${JSON.stringify(rawStatus)}`) } - if (this.spec.stdio.stdout === 'pipe' || this.spec.stdio.stderr === 'pipe') { - return this.commandOutcome(await settlement, exitCode) - } - const completed = await withinMs(settlement, this.spec.graceMs) if (completed !== undefined) return this.commandOutcome(completed, exitCode) + const drained = await withinMs(settlement, this.spec.graceMs) + if (drained !== undefined) return this.commandOutcome(drained, exitCode) this.outputDrainExpired = true this.stdoutReader?.invalidateSpill() this.stderrReader?.invalidateSpill() await handle.disconnect() return { exitCode, signal: null } } - const completed = await Promise.race([settlement, waitTick().then(() => undefined)]) if (completed !== undefined) return this.commandOutcome(completed) + // TODO(e2b-status-watch): Replace collect/inherit polling when E2B can + // observe direct-command exit independently of descendant-held output. + completed = await Promise.race([settlement, waitTick().then(() => undefined)]) } } diff --git a/packages/e2b/subprocess-e2b/src/terminal.ts b/packages/e2b/subprocess-e2b/src/terminal.ts index 6c4b23d1c1..8d003dfc35 100644 --- a/packages/e2b/subprocess-e2b/src/terminal.ts +++ b/packages/e2b/subprocess-e2b/src/terminal.ts @@ -345,6 +345,9 @@ export class E2BTerminalHandle implements SubprocessTerminalHandle { private topLevelExited = false private cleanup: Promise | undefined + private readonly operationController = new AbortController() + private readonly operations = new Set>() + private terminating = false private terminationSignal: NodeJS.Signals | null = null constructor( @@ -364,17 +367,56 @@ export class E2BTerminalHandle implements SubprocessTerminalHandle { // TODO(e2b-pgid-identity): Replace retained numeric PTY/session ids when E2B // exposes identity-bound input, foreground-signal, and cleanup operations. /** @inheritdoc */ - async write(data: string): Promise { - if (this.topLevelExited) throw new Error('terminal process has exited') - await this.sandbox.pty.sendInput(this.pid, Buffer.from(data, 'utf8')) + write(data: string): Promise { + return this.trackOperation(async (signal) => { + if (this.topLevelExited) throw new Error('terminal process has exited') + await this.sandbox.pty.sendInput(this.pid, Buffer.from(data, 'utf8'), { signal }) + }) } /** @inheritdoc */ - async inspectForeground(): Promise { + inspectForeground(): Promise { + return this.trackOperation(signal => this.inspectForegroundOnce(signal)) + } + + /** @inheritdoc */ + signalForeground(signal: SubprocessTerminalSignal): Promise { + return this.trackOperation(async (operationSignal) => { + const foreground = await this.inspectForegroundOnce(operationSignal) + if (foreground === undefined) { + throw new Error(`subprocess-e2b: cannot resolve foreground process group for terminal ${this.pid}`) + } + if (signal === 'SIGKILL' && foreground.processGroupId === this.pid) { + throw new Error('refusing to SIGKILL the terminal shell; terminate the terminal session instead') + } + await this.sandbox.commands.run( + `kill -${signal.slice(3)} -- -${foreground.processGroupId}`, + commandOpts(this.controlEnvs, operationSignal), + ) + return foreground.processGroupId + }) + } + + /** @inheritdoc */ + terminate(): Promise { + if (this.cleanup !== undefined) return this.cleanup + this.terminating = true + this.operationController.abort(new Error('subprocess-e2b: terminal is terminating')) + const cleanup = this.closeAfterOperations() + this.cleanup = cleanup + void cleanup.catch((_cleanupFailure: unknown) => { + this.cleanup = undefined + }) + return cleanup + } + + private async inspectForegroundOnce( + signal: AbortSignal, + ): Promise { try { const result = await this.sandbox.commands.run( `ps -o tpgid= -p ${this.pid}`, - commandOpts(this.controlEnvs), + commandOpts(this.controlEnvs, signal), ) return { processGroupId: parsePositiveId( @@ -391,31 +433,20 @@ export class E2BTerminalHandle implements SubprocessTerminalHandle { } } - /** @inheritdoc */ - async signalForeground(signal: SubprocessTerminalSignal): Promise { - const foreground = await this.inspectForeground() - if (foreground === undefined) { - throw new Error(`subprocess-e2b: cannot resolve foreground process group for terminal ${this.pid}`) - } - if (signal === 'SIGKILL' && foreground.processGroupId === this.pid) { - throw new Error('refusing to SIGKILL the terminal shell; terminate the terminal session instead') - } - await this.sandbox.commands.run( - `kill -${signal.slice(3)} -- -${foreground.processGroupId}`, - commandOpts(this.controlEnvs), + private trackOperation(operation: (signal: AbortSignal) => Promise): Promise { + if (this.terminating) return Promise.reject(new Error('subprocess-e2b: terminal is terminating')) + const pending = operation(this.operationController.signal) + this.operations.add(pending) + void pending.then( + () => { this.operations.delete(pending) }, + () => { this.operations.delete(pending) }, ) - return foreground.processGroupId + return pending } - /** @inheritdoc */ - terminate(): Promise { - if (this.cleanup !== undefined) return this.cleanup - const cleanup = this.closeOnce() - this.cleanup = cleanup - void cleanup.catch((_cleanupFailure: unknown) => { - this.cleanup = undefined - }) - return cleanup + private async closeAfterOperations(): Promise { + if (this.operations.size > 0) await Promise.allSettled(this.operations) + await this.closeOnce() } private async waitForCommand(): Promise { diff --git a/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts b/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts index 0e906b1cfb..83e86892d4 100644 --- a/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts +++ b/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts @@ -116,6 +116,7 @@ class FakeSandbox { environmentRequest: ((signal: AbortSignal | undefined) => Promise) | undefined processGroupId = '4242\n' exitStatus = '' + statusReads = 0 readonly processGroupReads: string[] = [] afterStatusRead: (() => void) | undefined beforeProbe: (() => void) | undefined @@ -222,6 +223,7 @@ class FakeSandbox { this.statusError = undefined throw error } + this.statusReads += 1 this.afterStatusRead?.() return this.exitStatus }, @@ -415,9 +417,10 @@ describe('E2BSubprocessHandle', () => { expect(command).not.toContain('explicit-secret') expect(command).not.toContain('hyphen-value') expect(command).not.toContain('${!dsh_e2b_name}') - expect(fake.commandsSeen).toContain( - 'set -o pipefail; printf \'%s\' "$PWD" | base64 -w 0; printf \'\\n\'; env -0 | base64 -w 0', - ) + const environmentProbe = fake.commandsSeen.find(value => value.includes('env -0 | base64')) + expect(environmentProbe).toContain('getent passwd "$(id -u)"') + expect(environmentProbe).toContain('test -n "$dsh_e2b_home" -a -d "$dsh_e2b_home"') + expect(environmentProbe).not.toContain('"$PWD"') expect(command).toContain('mapfile -d') expect(command).toContain('dsh_e2b_node="$(command -v node)"') expect(command).toContain('"$dsh_e2b_env_bin" -i "$dsh_e2b_node" -e') @@ -535,11 +538,13 @@ describe('E2BSubprocessHandle', () => { await new Promise(resolve => setTimeout(resolve, 50)) expect(settled).toBe(false) expect(fake.handle.disconnects).toBe(0) + expect(fake.statusReads).toBe(0) await fake.stdout('complete protocol frame') fake.finish() await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) expect(output).toBe('complete protocol frame') + expect(fake.statusReads).toBe(1) }) it('accepts clean encoder completion inside the output-drain grace', async () => { diff --git a/packages/e2b/subprocess-e2b/tests/terminal.spec.ts b/packages/e2b/subprocess-e2b/tests/terminal.spec.ts index 6c3ef477af..a4f4008a56 100644 --- a/packages/e2b/subprocess-e2b/tests/terminal.spec.ts +++ b/packages/e2b/subprocess-e2b/tests/terminal.spec.ts @@ -102,6 +102,9 @@ class FakeTerminalSandbox { sendError: unknown commandFailure: unknown makeDirRequest: ((signal: AbortSignal | undefined) => Promise) | undefined + sendInputRequest: ((signal: AbortSignal | undefined) => Promise) | undefined + foregroundRequest: ((signal: AbortSignal | undefined) => Promise) | undefined + signalRequest: ((signal: AbortSignal | undefined) => Promise) | undefined sessionGroupsFailure: unknown foregroundFailure: unknown termFailure: unknown @@ -180,6 +183,8 @@ class FakeTerminalSandbox { return { exitCode: 0, stdout: this.sessionId, stderr: '' } } if (command.startsWith('ps -o tpgid=')) { + await this.foregroundRequest?.(options?.signal) + options?.signal?.throwIfAborted() if (this.foregroundFailure !== undefined) throw this.foregroundFailure return { exitCode: 0, stdout: this.foreground, stderr: '' } } @@ -197,6 +202,10 @@ class FakeTerminalSandbox { this.handle.fail(143) } } + if (command.startsWith('kill -INT -- ')) { + await this.signalRequest?.(options?.signal) + options?.signal?.throwIfAborted() + } if (command.startsWith('kill -KILL -- ') && this.clearOnKill) this.groups = [] return { exitCode: 0, stdout: '', stderr: '' } }, @@ -211,6 +220,8 @@ class FakeTerminalSandbox { return this.handle.asHandle() }, sendInput: async (pid: number, data: Uint8Array, options?: { signal?: AbortSignal }): Promise => { + options?.signal?.throwIfAborted() + await this.sendInputRequest?.(options?.signal) options?.signal?.throwIfAborted() this.inputs.push({ pid, data: Buffer.from(data) }) if (this.sendError !== undefined) throw this.sendError @@ -257,6 +268,19 @@ function spec(overrides: Partial = {}): SubprocessT } } +function holdRequestUntilAbort(started: PromiseWithResolvers) { + return async (signal: AbortSignal | undefined): Promise => { + if (signal === undefined) throw new Error('expected an operation signal') + signal.throwIfAborted() + started.resolve(signal) + await new Promise((_resolve, reject) => { + signal.addEventListener('abort', () => { + reject(signal.reason instanceof Error ? signal.reason : new Error(String(signal.reason))) + }, { once: true }) + }) + } +} + describe('E2B terminal allocation', () => { it('hides bootstrap-shell bytes and preserves requested-shell bytes across the output boundary', async () => { const fake = new FakeTerminalSandbox() @@ -544,6 +568,43 @@ describe('E2B terminal allocation', () => { }) describe('E2B terminal lifecycle', () => { + it('aborts and joins in-flight terminal operations before cleanup', async () => { + const fake = new FakeTerminalSandbox() + const terminal = await spawnE2BTerminal(runtime(fake), spec(), '/runtime/in-flight-operations') + const writeStarted = Promise.withResolvers() + const inspectStarted = Promise.withResolvers() + const signalStarted = Promise.withResolvers() + fake.sendInputRequest = holdRequestUntilAbort(writeStarted) + let foregroundRequests = 0 + fake.foregroundRequest = async (signal) => { + foregroundRequests += 1 + if (foregroundRequests === 1) await holdRequestUntilAbort(inspectStarted)(signal) + } + let signalCompleted = false + fake.signalRequest = async (operationSignal) => { + await holdRequestUntilAbort(signalStarted)(operationSignal) + signalCompleted = true + } + const write = terminal.write('late input') + const inspect = terminal.inspectForeground() + await Promise.all([writeStarted.promise, inspectStarted.promise]) + const signal = terminal.signalForeground('SIGINT') + await signalStarted.promise + + const terminating = terminal.terminate() + await expect(write).rejects.toThrow('terminal is terminating') + await expect(inspect).rejects.toThrow('terminal is terminating') + await expect(signal).rejects.toThrow('terminal is terminating') + await terminating + expect(signalCompleted).toBe(false) + expect(fake.inputs).toHaveLength(1) + const commandCount = fake.commands.length + await expect(terminal.write('after termination')).rejects.toThrow('terminal is terminating') + await expect(terminal.inspectForeground()).rejects.toThrow('terminal is terminating') + await expect(terminal.signalForeground('SIGINT')).rejects.toThrow('terminal is terminating') + expect(fake.commands).toHaveLength(commandCount) + }) + it('maps ordinary exits, closes output, and reports an absent foreground after exit', async () => { const fake = new FakeTerminalSandbox() fake.groups = [] From e8d55999456164af0a9842d0f103c2a9b77916ec Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 23:53:20 +0800 Subject: [PATCH 093/108] refactor(e2b): narrow settlement polling helper --- packages/e2b/subprocess-e2b/src/process.ts | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/packages/e2b/subprocess-e2b/src/process.ts b/packages/e2b/subprocess-e2b/src/process.ts index 83dedd4e97..5e383a50e2 100644 --- a/packages/e2b/subprocess-e2b/src/process.ts +++ b/packages/e2b/subprocess-e2b/src/process.ts @@ -84,10 +84,10 @@ type CommandSettlement = | { kind: 'result'; result: CommandResult } | { kind: 'error'; error: unknown } -function withinMs(promise: Promise, timeoutMs: number): Promise { - return new Promise((resolve) => { +function withinMs(settlement: Promise, timeoutMs: number): Promise { + return new Promise((resolve) => { const timer = setTimeout(() => { resolve(undefined) }, timeoutMs) - void promise.then((value) => { + void settlement.then((value) => { clearTimeout(timer) resolve(value) }) @@ -563,8 +563,8 @@ export class E2BSubprocessHandle implements SubprocessHandle { return { exitCode, signal: null } } if (completed !== undefined) return this.commandOutcome(completed) - // TODO(e2b-status-watch): Replace collect/inherit polling when E2B can - // observe direct-command exit independently of descendant-held output. + // TODO(e2b-status-watch): Replace collect/inherit control-plane polling + // when E2B can observe direct-command exit independently of descendant-held output. completed = await Promise.race([settlement, waitTick().then(() => undefined)]) } } From bb0be75d8549473996bbea3761897465e5a92141 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Wed, 29 Jul 2026 23:56:02 +0800 Subject: [PATCH 094/108] refactor(e2b): drop obsolete host sandbox overlay --- examples/headless-agent/e2b.cordis.snapshot.yml | 2 -- examples/headless-agent/e2b.cordis.yml | 2 -- examples/headless-agent/tests/fixtures/e2b/e2b/cordis.yml | 3 --- 3 files changed, 7 deletions(-) diff --git a/examples/headless-agent/e2b.cordis.snapshot.yml b/examples/headless-agent/e2b.cordis.snapshot.yml index 3f39f0ffe4..936aab290d 100644 --- a/examples/headless-agent/e2b.cordis.snapshot.yml +++ b/examples/headless-agent/e2b.cordis.snapshot.yml @@ -29,8 +29,6 @@ name: '@deepseek-ai/dsh-code-runtime-subprocess' - id: tool-cordis name: '@deepseek-ai/dsh-tool-cordis' - - id: sandbox - name: '@deepseek-ai/dsh-sandbox-local' - id: sandbox-policy name: '@deepseek-ai/dsh-sandbox-policy' config: diff --git a/examples/headless-agent/e2b.cordis.yml b/examples/headless-agent/e2b.cordis.yml index 32d7049c5d..683bb7a0d4 100644 --- a/examples/headless-agent/e2b.cordis.yml +++ b/examples/headless-agent/e2b.cordis.yml @@ -34,8 +34,6 @@ name: '@deepseek-ai/dsh-fs-e2b' - id: code-runtime-subprocess name: '@deepseek-ai/dsh-code-runtime-subprocess' - - id: sandbox - name: '@deepseek-ai/dsh-sandbox-local' - id: sandbox-policy name: '@deepseek-ai/dsh-sandbox-policy' config: diff --git a/examples/headless-agent/tests/fixtures/e2b/e2b/cordis.yml b/examples/headless-agent/tests/fixtures/e2b/e2b/cordis.yml index d2b73c1077..4132f4d1d6 100644 --- a/examples/headless-agent/tests/fixtures/e2b/e2b/cordis.yml +++ b/examples/headless-agent/tests/fixtures/e2b/e2b/cordis.yml @@ -21,9 +21,6 @@ - id: agents name: '@deepseek-ai/dsh-agent' -- id: sandbox - name: '@deepseek-ai/dsh-sandbox-local' - - id: sandbox-policy name: '@deepseek-ai/dsh-sandbox-policy' config: From de77310c6f68bafc6593dd37f134fd282bcbdd7b Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Thu, 30 Jul 2026 04:25:47 +0800 Subject: [PATCH 095/108] refactor(e2b): narrow the sandbox POC --- ...rtable-execution-world-consumers.i18n.yaml | 4 +- ...7-28-portable-execution-world-consumers.md | 8 +- ...8-portable-execution-world-consumers.zh.md | 8 +- AGENTS.md | 7 +- docs/cordis-catalog/services.md | 8 +- .../headless-agent/e2b.cordis.snapshot.yml | 56 ----- examples/headless-agent/e2b.cordis.yml | 7 - .../tests/fixtures/e2b/e2b/bin.ts | 215 +----------------- .../tests/fixtures/e2b/e2b/cordis.yml | 4 - .../headless-agent/tests/headless.snapshot.ts | 56 ----- .../tests/snapshots/e2b-overlay/input.json | 8 - .../e2b-overlay/replay.override.json | 12 - .../e2b-overlay/stream-json.expected.jsonl | 14 -- packages/e2b/README.i18n.yaml | 4 +- packages/e2b/README.md | 2 +- packages/e2b/README.zh.md | 2 +- packages/e2b/e2b/README.i18n.yaml | 4 +- packages/e2b/e2b/README.md | 18 +- packages/e2b/e2b/README.zh.md | 18 +- packages/e2b/e2b/package.json | 2 - packages/e2b/e2b/src/index.ts | 146 +++--------- packages/e2b/e2b/tests/composition.e2e.ts | 51 +---- packages/e2b/e2b/tests/e2b.spec.ts | 122 +--------- packages/e2b/fs-e2b/README.i18n.yaml | 4 +- packages/e2b/fs-e2b/README.md | 4 +- packages/e2b/fs-e2b/README.zh.md | 4 +- packages/e2b/fs-e2b/tests/filesystem.spec.ts | 1 - packages/e2b/subprocess-e2b/README.i18n.yaml | 4 +- packages/e2b/subprocess-e2b/README.md | 11 +- packages/e2b/subprocess-e2b/README.zh.md | 11 +- packages/e2b/subprocess-e2b/src/index.ts | 13 +- packages/e2b/subprocess-e2b/src/output.ts | 3 - packages/e2b/subprocess-e2b/src/process.ts | 59 ++--- packages/e2b/subprocess-e2b/src/terminal.ts | 72 ++---- .../subprocess-e2b/tests/subprocess.spec.ts | 4 - .../e2b/subprocess-e2b/tests/terminal.spec.ts | 85 ++----- pnpm-lock.yaml | 6 +- 37 files changed, 176 insertions(+), 881 deletions(-) delete mode 100644 examples/headless-agent/e2b.cordis.snapshot.yml delete mode 100644 examples/headless-agent/tests/snapshots/e2b-overlay/input.json delete mode 100644 examples/headless-agent/tests/snapshots/e2b-overlay/replay.override.json delete mode 100644 examples/headless-agent/tests/snapshots/e2b-overlay/stream-json.expected.jsonl diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml index fc3b5b012f..c7ce82595d 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md -2026-07-28-portable-execution-world-consumers.md: 250445cfed6a4fe2dd188562813d586df29992c1 -2026-07-28-portable-execution-world-consumers.zh.md: 361c4de4e813aa7f57a7eb5acc75087b2630db43 +2026-07-28-portable-execution-world-consumers.md: 57c3a8a8b211279593359b02def865644052f549 +2026-07-28-portable-execution-world-consumers.zh.md: 8231913c26f75361add7a28bc40f7723c9b18728 diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md index 4c593604f4..6f296bfa74 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md @@ -28,17 +28,17 @@ Generic consumers use that execution world: ## E2B POC boundary -The opt-in E2B realization has exactly three provider-specific packages under `packages/e2b/`: `dsh-e2b` creates or reconnects one sandbox and owns kill/pause/leave disposal, `dsh-fs-e2b` implements `ctx.fs`, and `dsh-subprocess-e2b` implements `ctx.subprocess` over E2B Commands, PTYs, and remote Linux process groups. The two adapters obtain the sole sandbox identity from the owner and never create private sandboxes. +The opt-in E2B realization has exactly three provider-specific packages under `packages/e2b/`: `dsh-e2b` creates one sandbox and deletes it on timeout or disposal, `dsh-fs-e2b` implements `ctx.fs`, and `dsh-subprocess-e2b` implements `ctx.subprocess` over E2B Commands, PTYs, and remote Linux process groups. The two adapters obtain the sole SDK handle from the owner and never create private sandboxes. E2B owns the mutable filesystem, managed command and Bash processes, terminal allocation and terminal-session groups, language-server processes and source reads, subprocess Code Runtime processes, and adapter-private files under `.dsh-e2b`. The host owns Cordis and plugin objects, the agent loop, agent/session/goal state, session logs and persistence, LLM calls, prompts and tools, authority, skills, subagent orchestration, PTY buffers and readiness, LSP protocol state, Code Runtime program/binding/output policy, and E2B SDK/network buffers. The overlay neither uploads nor synchronizes the host workspace. The adapters retain only substrate mechanics. Filesystem canonicalization crosses the SDK's decoded command transport as strict base64-encoded NUL framing; streamed reads leave byte ceilings with consumers. Subprocess command output and environment snapshots use ASCII/base64 where SDK chunk decoding would otherwise lose bytes, while private control shells isolate profiles and later launches blank discovered credential-shaped names. Process and terminal cleanup uses remote groups and proves quiescence before settlement. -Retaining a sandbox preserves remote files and unmanaged state only. Reconnect does not reconstruct host process or terminal handles, protocol connections, pending calls, output cursors, timers, or locks. The POC adds no session-persistence backend, template builder, volume, snapshot, network-policy layer, sandbox catalog, workspace synchronization, durable remote handles, or whole-harness execution. +Sandbox state is deliberately ephemeral: timeout and disposal delete the remote files and unmanaged state. The POC adds no reconnect or pause/leave retention, session-persistence backend, template builder, volume, snapshot, network-policy layer, sandbox catalog, workspace synchronization, durable remote handles, or whole-harness execution. ## Verification -Focused package suites pin sandbox lifecycle, canonical path framing, filesystem metadata and atomic versions, subprocess publication/rollback, terminal text I/O and session cleanup, output limits, cancellation, disposal, and invariant registration. A credential-gated Loader composition exercises the same three-package provider through source imports and built exports, including FS/Bash visibility, post-rename version reread plus guarded edit, hostile login profiles, byte-split UTF-8 output, process and terminal cleanup, LSP document bounds, Code Runtime bindings/limits/cleanup, host-workspace isolation, and final sandbox deletion. +Focused package suites pin sandbox lifecycle, canonical path framing, filesystem metadata and atomic versions, subprocess publication/rollback, terminal text I/O and session cleanup, output limits, cancellation, disposal, and invariant registration. A credential-gated Loader composition exercises the same three-package provider through source imports and built exports, including FS/Bash visibility, hostile login profiles, byte-split UTF-8 output, process and terminal cleanup, LSP queries, Code Runtime bindings and cleanup, host-workspace isolation, and final sandbox deletion. ## Alternatives considered @@ -70,4 +70,4 @@ The fundamental interfaces are wider, and a filesystem/subprocess pair must agre The local implementation absorbs `node-pty` and platform process inspection because it owns local terminal mechanics. This moves code without weakening terminal teardown: disposal sweeps descendants before and after terminating the top-level shell, waits for exact PID-identity-fenced descendants retained during foreground inspection, and retains Linux session members that survive top-level exit. macOS cannot enumerate a POSIX session after its leader exits, so a child that reparents between inspection snapshots remains an explicit local-provider limitation rather than a reason to move process mechanics back into the PTY consumer. -The E2B composition demonstrates that a shared sandbox owner plus filesystem and subprocess adapters are sufficient to move the mutable coding world off-host while leaving higher capabilities provider-neutral. Its POC limits remain explicit: the SDK retains complete command transport in host memory, remote startup cannot publish a PID synchronously, exact terminal stdin-wait and independent signal facts are unavailable, numeric PID/PGID operations are not identity-fenced, the initial environment probe cannot hide unknown sandbox-default secrets from already-running same-UID processes, retained artifacts accumulate, and escaped processes or reconnect state are not recovered. These are provider constraints, not justification for compatibility shims or more E2B packages. +The E2B composition demonstrates that a shared sandbox owner plus filesystem and subprocess adapters are sufficient to move the mutable coding world off-host while leaving higher capabilities provider-neutral. Its POC limits remain explicit: the SDK retains complete command transport in host memory, remote startup cannot publish a PID synchronously, exact terminal stdin-wait and independent signal facts are unavailable, numeric PID/PGID operations are not identity-fenced, the initial environment probe cannot hide unknown sandbox-default secrets from already-running same-UID processes, and adapter artifacts remain until sandbox deletion. These are provider constraints, not justification for compatibility shims or more E2B packages. diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md index 361c4de4e8..471753573d 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md @@ -31,17 +31,17 @@ Status: implemented ## E2B POC 边界 -可选启用的 E2B 实现在 `packages/e2b/` 下恰好只有三个提供方专用包:`dsh-e2b` 创建或重新连接一个沙箱,并负责 kill、pause 或 leave 资源释放;`dsh-fs-e2b` 实现 `ctx.fs`;`dsh-subprocess-e2b` 基于 E2B Commands、PTY 和远程 Linux 进程组实现 `ctx.subprocess`。两个适配器都从所有者取得唯一的沙箱身份,绝不创建私有沙箱。 +可选启用的 E2B 实现在 `packages/e2b/` 下恰好只有三个提供方专用包:`dsh-e2b` 创建一个沙箱,并在超时或资源释放时将其删除;`dsh-fs-e2b` 实现 `ctx.fs`;`dsh-subprocess-e2b` 基于 E2B Commands、PTY 和远程 Linux 进程组实现 `ctx.subprocess`。两个适配器都从所有者取得唯一的 SDK 句柄,绝不创建私有沙箱。 E2B 负责可变文件系统、受管命令与 Bash 进程、终端分配与终端会话组、语言服务器进程与源文件读取、子进程代码运行时进程,以及 `.dsh-e2b` 下的适配器私有文件。宿主负责 Cordis 与插件对象、agent loop(智能体循环)、agent(智能体)状态、会话状态与目标状态、会话日志与持久化、LLM(大语言模型)调用、提示词与工具、权限、skill(技能)、subagent 编排、PTY 缓冲区与就绪状态、LSP 协议状态、代码运行时程序/绑定/输出策略,以及 E2B SDK/网络缓冲区。该叠加层既不上传,也不同步宿主工作区。 适配器只保留执行基底机制。文件系统规范化以严格的 base64 加 NUL 分帧穿过 SDK 已解码的命令传输;流式读取把字节上限留给消费方执行。进程管理命令输出与环境快照采用 ASCII/base64,避免 SDK 分片解码丢失字节;私有控制 shell 隔离 profile,后续启动会把已发现且名称呈凭据特征的环境变量置空。进程与终端清理使用远程进程组,并在结算前证明完全停稳。 -保留沙箱只会保留远程文件与非托管状态。重新连接不会重建宿主进程或终端句柄、协议连接、待处理调用、输出游标、计时器或锁。该 POC 不会新增会话持久化后端、模板构建器、卷、快照、网络策略层、沙箱目录、工作区同步、持久远程句柄,也不会在其中运行整个 harness。 +沙箱状态有意保持短暂:超时与资源释放会删除远程文件和非托管状态。该 POC 不提供重新连接、pause/leave 保留、会话持久化后端、模板构建器、卷、快照、网络策略层、沙箱目录、工作区同步、持久远程句柄,也不会在其中运行整个 harness。 ## 验证 -聚焦的包测试套件锁定了沙箱生命周期、规范化路径分帧、文件系统元数据与原子版本、进程管理发布/回滚、终端文本 I/O 与会话清理、输出上限、取消、资源释放和不变式注册。一项受凭据门控的 Loader 组合通过源代码导入与构建后导出运行同一套三包提供方组合,其中包括 FS/Bash 可见性、重命名后的版本重读与带保护编辑、恶意登录 profile、跨字节边界拆分的 UTF-8 输出、进程与终端清理、LSP 文档上限、代码运行时绑定/上限/清理、宿主工作区隔离,以及最终沙箱删除。 +聚焦的包测试套件锁定了沙箱生命周期、规范化路径分帧、文件系统元数据与原子版本、进程管理发布/回滚、终端文本 I/O 与会话清理、输出上限、取消、资源释放和不变式注册。一项受凭据门控的 Loader 组合通过源代码导入与构建后导出运行同一套三包提供方组合,其中包括 FS/Bash 可见性、恶意登录 profile、跨字节边界拆分的 UTF-8 输出、进程与终端清理、LSP 查询、代码运行时绑定与清理、宿主工作区隔离,以及最终沙箱删除。 ## 考虑过的替代方案 @@ -75,4 +75,4 @@ E2B 负责可变文件系统、受管命令与 Bash 进程、终端分配与终 本地实现承接 `node-pty` 和平台进程检查,因为它负责本地终端机制。这种代码迁移不会削弱终端拆卸:dispose(资源释放)会在终止顶层 shell 前后清理后代进程,等待前台检查期间保留下来且受精确 PID 身份围栏保护的后代进程,并继续追踪在顶层进程退出后仍存活的 Linux 会话成员。macOS 无法在 POSIX 会话 leader 退出后枚举该会话,因此在两次检查快照之间重新设定父进程的子进程仍是明确的本地提供方限制,而不是把进程机制移回 PTY 消费方的理由。 -E2B 组合证明,共享沙箱所有者加上文件系统与进程管理适配器,就足以在保持上层能力与提供方无关的同时,把可变编码世界移出宿主。其 POC 限制仍明确在案:SDK 会把完整命令传输内容保留在宿主内存中;远程启动无法同步发布 PID;无法获得精确的终端 stdin 等待状态与独立信号事实;基于数值 PID/PGID 的操作没有身份围栏;初始环境探测无法向已在运行的同 UID 进程隐藏未知的沙箱默认 secret;保留的产物会累积;也不会恢复逃逸进程或重连状态。这些是提供方限制,不是引入兼容性 shim 或更多 E2B 包的理由。 +E2B 组合证明,共享沙箱所有者加上文件系统与进程管理适配器,就足以在保持上层能力与提供方无关的同时,把可变编码世界移出宿主。其 POC 限制仍明确在案:SDK 会把完整命令传输内容保留在宿主内存中;远程启动无法同步发布 PID;无法获得精确的终端 stdin 等待状态与独立信号事实;基于数值 PID/PGID 的操作没有身份围栏;初始环境探测无法向已在运行的同 UID 进程隐藏未知的沙箱默认 secret;适配器产物会一直保留到沙箱删除。这些是提供方限制,不是引入兼容性 shim 或更多 E2B 包的理由。 diff --git a/AGENTS.md b/AGENTS.md index c663a5e099..1d2a6d977c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -12,8 +12,9 @@ DeepSeek Harness SDK is a plugin-based agent harness on vendored Cordis: **every vendor/ Vendored Cordis source — manifest + sync procedure in vendor/README.md packages/ @deepseek-ai/dsh- workspaces at packages/// core/ product API spine: session, system-prompt, tools, agent, agent-loop + typert/ type graph generator, loader, and runtime registry llm/ LLM seam + DeepSeek adapters (direct-fetch + pi-ai design twin) - e2b/ remote-runtime POC + e2b/ E2B POC: sandbox owner + FS/subprocess adapters bash/ bash executor seam + local impl + model-facing bash tools subprocess/ subprocess seam + local process-tree impl pty/ persistent PTY seam/backend/tools @@ -90,7 +91,7 @@ Real-API tests and demos read `DEEPSEEK_API_KEY`, optional `DEEPSEEK_BASE_URL`, ## Conventions - Every npm package is `@deepseek-ai/dsh-`; vendored packages keep upstream names and are `private: true`. `cordis` is a peerDependency (+ dev) of every harness package. -- ESM everywhere (`"type": "module"`). Cross-package imports use package names; in-package relative imports include `.ts`. CI subprocesses that boot examples or Cordis configs run built `lib/` under plain Node; only explicit source-path regressions use tsx ([testing policy](docs/testing.md#test-subprocess-launch-modes)). +- ESM everywhere (`"type": "module"`). Cross-package imports use package names; in-package relative imports include `.ts`. Config subprocesses run built `lib/` under plain Node; source regressions use their declared launcher ([testing policy](docs/testing.md#test-subprocess-launch-modes)). The `dsh` CLI source launch runs through tsx's ESM-only hook (`node --import tsx/esm`); modules it reaches must stay ESM (no CJS-only shapes) — Node's native TypeScript modes are unavailable across the engines range ([source-launch contract](.agents/notes/implemented/architecture/2026-07-29-dsh-source-launch-tsx-esm.md)). TUI/Web `cordis.yml` bare plugins must appear in their resolver manifest's `dependencies`; `verify-cordis-config` enforces it. - **Registrations are effects**: every contribution goes through `ctx.effect()` / `ctx.on()`; a registry's `register()` returns the disposer. - **Runtime invariants assert owned relationships.** Check authoritative event streams or mutable data, not service or method presence, plugin metadata or effects, or fixed pure examples. If a package has no plausible relationship, an explained empty companion is correct ([package contract](packages/AGENTS.md)). - **Typed events use declaration merging** and merge-extensible maps. Event JSDoc needs `@mode` and payload `@param`; scoped keys absent from payloads need `@dshScopeScan unsupported`. Public service methods document parameters and non-void returns. @@ -114,7 +115,7 @@ Real-API tests and demos read `DEEPSEEK_API_KEY`, optional `DEEPSEEK_BASE_URL`, - **Testing policy** — [docs/testing.md](docs/testing.md). Every non-trivial model- or product-user-visible behavior change adds or updates a keyless snapshot through a real runnable example in the same PR; package tests, e2e-only assertions, and mock-only fixtures do not substitute for the assembled application transcript. Fixtures must replay on macOS/Linux; fix fixtures, not normalizers. - **A tool's UI render intent is part of its design**, decided up front (`generic`/`terminal`/`diff`, `locations`); presentation methods are pure functions of `args` ([cookbook](docs/cookbook/adding-a-tool.md)). - **Plan unit, e2e, and snapshot coverage** for new seams, lifecycle shapes, and transcript surfaces; missing snapshot-harness support is part of the implementation, not deferred follow-up. -- **Use incremental merge commits.** Split independent changes; never squash, rebase, or rewrite pushed history. Fix the introducing PR before merging down-stack. If the base advances mid-merge, never restart: finish the checkpoint, push when authorized, then merge the newer tip separately ([rationale](.agents/notes/implemented/process/2026-07-26-incremental-pr-base-retargeting.md)). +- **Use incremental merge commits.** Split independent changes. Pushed history may be rewritten before review; afterward prefer new commits. Fix the introducing PR before merging down-stack. If the base advances mid-merge, finish the checkpoint, push when authorized, then merge the newer tip separately ([rationale](.agents/notes/implemented/process/2026-07-26-incremental-pr-base-retargeting.md)). - **Label PRs:** one kind (`feature`/`bug-fix`/`doc`/`testing`/`cleanup`), each matching area; the [taxonomy](.agents/notes/implemented/process/2026-07-25-semantic-pr-label-taxonomy.md) is extensible. - TODO markers: `FIXME`/`TODO`/`XXX` by urgency ([semantics](docs/development.md)). - Files end with exactly one trailing newline; `git diff --cached --check` (pre-commit) gates it. diff --git a/docs/cordis-catalog/services.md b/docs/cordis-catalog/services.md index 3942c0696d..db66ca5799 100644 --- a/docs/cordis-catalog/services.md +++ b/docs/cordis-catalog/services.md @@ -580,18 +580,18 @@ Source: [`packages/host/directory-picker/src/index.ts:131`](../../packages/host/ ## `ctx.e2b` — `E2BSandboxService` -Owns one lazily consumable E2B SDK handle and its final kill/pause/leave decision. The connection begins at plugin construction; adapters await getSandbox before their first operation. +Creates one lazily consumable E2B SDK handle and deletes the sandbox at timeout or disposal. Creation begins at plugin construction; adapters await getSandbox before their first operation. ```ts cordis-catalog /** * Return the shared live SDK handle. - * @returns the created or reconnected sandbox after the configured cwd exists. - * @throws when E2B rejects creation/reconnection or the service is disposing. + * @returns the created sandbox after the configured cwd exists. + * @throws when E2B rejects creation or the service is disposing. */ async getSandbox(): Promise ``` -Source: [`packages/e2b/e2b/src/index.ts:108`](../../packages/e2b/e2b/src/index.ts) +Source: [`packages/e2b/e2b/src/index.ts:76`](../../packages/e2b/e2b/src/index.ts) ## `ctx.fs` — `FileSystem` (abstract seam) diff --git a/examples/headless-agent/e2b.cordis.snapshot.yml b/examples/headless-agent/e2b.cordis.snapshot.yml deleted file mode 100644 index 936aab290d..0000000000 --- a/examples/headless-agent/e2b.cordis.snapshot.yml +++ /dev/null @@ -1,56 +0,0 @@ -# Keyless counterpart to e2b.cordis.yml. It keeps the same generic PTY, LSP, -# and subprocess Code Runtime consumers, but supplies their fundamental -# filesystem/process world locally and replaces the live model with replay. -- id: base - name: '@cordisjs/plugin-include' - config: - path: ./cordis.yml - patches: - - id: llm-deepseek - name: '@deepseek-ai/dsh-llm-deepseek' - disabled: true - - id: cli-agent - name: '@deepseek-ai/dsh-cli-demo' - config: - provider: deepseek - model: deepseek-v4-pro - persistenceRoot: './.sessions' - persistenceCompression: none - workspaceContext: - maxBytes: 65536 - tools: - mode: both - persona: | - You are headless-agent, a coding assistant powered by the {{model}} model. Your working directory is {{cwd}}. - - Verify your work by running the code or tests. Keep answers brief and factual. - - insert: - - id: code-runtime-subprocess - name: '@deepseek-ai/dsh-code-runtime-subprocess' - - id: tool-cordis - name: '@deepseek-ai/dsh-tool-cordis' - - id: sandbox-policy - name: '@deepseek-ai/dsh-sandbox-policy' - config: - mode: danger-full-access - workspaceRoot: !!js process.cwd() - - id: pty - name: '@deepseek-ai/dsh-pty' - - id: pty-local - name: '@deepseek-ai/dsh-pty-local' - - id: tool-pty - name: '@deepseek-ai/dsh-tool-pty' - - id: lsp - name: '@deepseek-ai/dsh-lsp' - - id: lsp-local - name: '@deepseek-ai/dsh-lsp-local' - config: - servers: - typescript: - command: node - extensionToLanguage: - .ts: typescript - - id: tool-lsp - name: '@deepseek-ai/dsh-tool-lsp' - - id: llm-replay - name: '@deepseek-ai/dsh-llm-replay' diff --git a/examples/headless-agent/e2b.cordis.yml b/examples/headless-agent/e2b.cordis.yml index 683bb7a0d4..724f30240d 100644 --- a/examples/headless-agent/e2b.cordis.yml +++ b/examples/headless-agent/e2b.cordis.yml @@ -15,19 +15,12 @@ - id: code-runtime name: '@deepseek-ai/dsh-code-runtime-worker' disabled: true - - id: bash - name: '@deepseek-ai/dsh-bash-local' - config: - cwd: !!js process.cwd() - timeoutMs: 60000 - insert: - id: e2b name: '@deepseek-ai/dsh-e2b' config: cwd: !!js process.cwd() timeoutMs: 300000 - onTimeout: kill - onDispose: kill - id: subprocess-e2b name: '@deepseek-ai/dsh-subprocess-e2b' - id: fs-e2b diff --git a/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts b/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts index 856a26ed78..570a244e88 100644 --- a/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts +++ b/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts @@ -1,10 +1,9 @@ import { readFile } from 'node:fs/promises' -import { posix, resolve } from 'node:path' +import { resolve } from 'node:path' import { boot } from '@deepseek-ai/dsh-app-boot' import type { Agent } from '@deepseek-ai/dsh-agent' import { Session, SessionId } from '@deepseek-ai/dsh-session' import type {} from '@deepseek-ai/dsh-code-runtime-subprocess' -import { quoteE2BShellArg } from '@deepseek-ai/dsh-e2b' import type {} from '@deepseek-ai/dsh-fs-e2b' import type {} from '@deepseek-ai/dsh-bash-local' import type {} from '@deepseek-ai/dsh-lsp-local' @@ -36,18 +35,17 @@ try { const sandbox = await ctx.e2b.getSandbox() const fromFs = await ctx.fs.resolve('from-fs.txt') const written = await ctx.fs.writeText(fromFs, 'written-by-fs\n', { kind: 'createIfAbsent' }) - const reread = await ctx.fs.stat(fromFs) - if (reread?.version !== written.version) { - throw new Error(`E2B rename did not preserve version metadata: ${JSON.stringify({ written, reread })}`) + const observed = await ctx.fs.stat(fromFs) + if (observed?.version !== written.version) { + throw new Error(`E2B rename did not preserve version metadata: ${JSON.stringify({ written, observed })}`) } await ctx.fs.editText( fromFs, - { oldString: 'written-by-fs', newString: 'written-by-fs-versioned', replaceAll: false }, - { version: reread.version }, + { oldString: 'written-by-fs', newString: 'versioned-by-fs', replaceAll: false }, + { version: observed.version }, ) - const fsVersionGuard = true const bashRead = await ctx.bash.run(ctx.bash.resolve({ command: 'cat from-fs.txt' })) - if (bashRead.exitCode !== 0 || bashRead.stdout.text !== 'written-by-fs-versioned\n') { + if (bashRead.exitCode !== 0 || bashRead.stdout.text !== 'versioned-by-fs\n') { throw new Error(`E2B Bash could not read the FS write: ${JSON.stringify(bashRead)}`) } @@ -114,67 +112,11 @@ try { ) if (outputDrainOutcome.exitCode !== 0 || outputDrainText !== 'leader-done\n' || outputDrainElapsedMs >= 10_000 || !outputDrainExited || !outputDrainClean) { - throw new Error(`E2B subprocess output drain was not bounded: ${JSON.stringify({ + throw new Error(`E2B subprocess did not bound descendant-held output: ${JSON.stringify({ outputDrainOutcome, outputDrainText, outputDrainElapsedMs, outputDrainExited, outputDrainClean, })}`) } - const remoteFiles = sandbox.files as unknown as { - read(path: string, options?: unknown): Promise - } - const readRemoteFile = remoteFiles.read.bind(sandbox.files) - let publicationFaultInjected = false - remoteFiles.read = async (path, options) => { - if (!publicationFaultInjected && path.includes('/processes/') && path.endsWith('/pid')) { - publicationFaultInjected = true - throw new Error('injected process-group publication read failure') - } - return await readRemoteFile(path, options) - } - let publicationRollback = false - try { - const unpublished = ctx.subprocess.spawn({ - argv: ['bash', '-c', 'exec -a dsh-publication-survivor sleep 30 & wait'], - cwd: process.cwd(), - stdio: { stdin: 'ignore', stdout: { maxBytes: 4_096 }, stderr: { maxBytes: 4_096 } }, - graceMs: 500, - env: {}, - }) - await unpublished.done - throw new Error('E2B subprocess unexpectedly survived an injected publication failure') - } catch (error: unknown) { - if (!String(error).includes('injected process-group publication read failure')) throw error - const processes = await sandbox.commands.run('ps -eo args=') - publicationRollback = publicationFaultInjected && !processes.stdout.includes('dsh-publication-survivor') - if (!publicationRollback) throw new Error('E2B subprocess publication rollback left its remote process group alive') - } finally { - remoteFiles.read = readRemoteFile - } - - const spillHandle = ctx.subprocess.spawn({ - argv: ['bash', '-c', "printf '0123456789'; sleep 30"], - cwd: process.cwd(), - stdio: { stdin: 'ignore', stdout: { maxBytes: 4, spill: { maxBytes: 6 } }, stderr: { maxBytes: 4_096 } }, - graceMs: 500, - env: {}, - }) - const spillReader = spillHandle.collected.stdout - if (spillReader === undefined) throw new Error('E2B subprocess omitted its configured stdout collector') - const spillDeadline = Date.now() + 15_000 - while (spillReader.readFrom(0).nextOffset < 10) { - if (Date.now() >= spillDeadline) throw new Error('E2B subprocess did not stream the spill probe output') - await new Promise(resolveDelay => setTimeout(resolveDelay, 20)) - } - const spillPath = posix.join((spillHandle as unknown as { stateDir: string }).stateDir, 'stdout.log') - const liveSpillBytes = (await (await ctx.e2b.getSandbox()).files.getInfo(spillPath)).size - spillHandle.terminate() - const spillOutcome = await spillHandle.done - const spillExited = await spillHandle.waitForExit(AbortSignal.timeout(5_000)) - const spillRead = spillReader.readFrom(0) - if (liveSpillBytes !== 6 || !spillExited || spillRead.spillPath !== undefined) { - throw new Error(`E2B subprocess spill bound failed: ${JSON.stringify({ liveSpillBytes, spillExited, spillRead })}`) - } - const lspFixture = await readFile(new URL('./fixture-lsp.mjs', import.meta.url), 'utf8') const remoteLspFixture = await ctx.fs.resolve('fixture-lsp.mjs') await ctx.fs.writeText(remoteLspFixture, lspFixture, { kind: 'createIfAbsent' }) @@ -193,53 +135,12 @@ try { workspaceRoot: process.cwd(), }) - const oversizedSourcePath = posix.join(process.cwd(), 'oversized-source.ts') - await sandbox.commands.run(`head -c 4000001 /dev/zero | tr '\\0' x > ${quoteE2BShellArg(oversizedSourcePath)}`) - let lspDocumentBound = false - try { - await ctx.lsp.query({ - operation: 'hover', - filePath: 'oversized-source.ts', - position: { line: 0, character: 0 }, - workspaceRoot: process.cwd(), - }) - } catch (error: unknown) { - lspDocumentBound = String(error).includes('exceeds the 4000000-byte limit') - if (!lspDocumentBound) throw error - } - if (!lspDocumentBound) throw new Error('E2B LSP accepted an oversized remote source') - - const remoteCommands = sandbox.commands as unknown as { - run(command: string, options?: unknown): Promise<{ exitCode: number; stdout: string; stderr: string }> - } - const runRemoteCommand = remoteCommands.run.bind(sandbox.commands) const terminal = await ctx.pty.spawn(owner, { type: 'shell' }) terminalId = terminal.sessionId const terminalEcho = await ctx.pty.startSend(owner, terminal.sessionId, { text: "printf 'PTY-你好\\n'", submit: true, }).done - const foregroundLookup = Promise.withResolvers() - let delayedForegroundLookup = false - remoteCommands.run = async (command, options) => { - if (!delayedForegroundLookup && command.startsWith('ps -o tpgid=')) { - delayedForegroundLookup = true - await foregroundLookup.promise - } - return await runRemoteCommand(command, options) - } - const staleInterrupt = ctx.pty.startSend(owner, terminal.sessionId, { text: 'sleep 0.2', submit: true }) - if (!staleInterrupt.cancel()) throw new Error('E2B PTY refused the stale-interrupt probe cancellation') - let canceledSendRetained = false - try { - ctx.pty.startSend(owner, terminal.sessionId, { text: 'sleep 30', submit: true }) - } catch (error: unknown) { - canceledSendRetained = String(error).includes('active send') - } - foregroundLookup.resolve(undefined) - await staleInterrupt.done - remoteCommands.run = runRemoteCommand - if (!canceledSendRetained) throw new Error('E2B PTY released a canceled send before foreground signalling settled') const sleeping = ctx.pty.startSend(owner, terminal.sessionId, { text: "printf 'DSH_SLEEP_%s\\n' READY; sleep 30", submit: true, @@ -258,13 +159,6 @@ try { } if (Date.now() >= sleepReadyDeadline) throw new Error(`E2B PTY successor did not execute: ${sleepReadyOutput}`) } - const interruptIdentitySafe = await Promise.race([ - sleeping.done.then(() => false), - new Promise(resolveDelay => setTimeout(() => { resolveDelay(true) }, 300)), - ]) - if (!delayedForegroundLookup || !interruptIdentitySafe) { - throw new Error('E2B PTY stale interrupt affected its successor send') - } const terminalSignal = await ctx.pty.signal(owner, terminal.sessionId, 'SIGINT') const interrupted = await sleeping.done const stubborn = await ctx.pty.startSend(owner, terminal.sessionId, { @@ -284,24 +178,6 @@ try { const code = await ctx.codeRuntime.run({ program: ` console.log('remote-log 你好', 42) - const arrayPrototype = Array.prototype - const objectPrototype = Object.prototype - const setPrototype = Set.prototype - const stringPrototype = String.prototype - Array.isArray = () => false - Object.defineProperty = Object.getPrototypeOf = Object.keys = () => { throw new Error('mutated object method') } - Object.hasOwn = () => false - Object.is = () => true - objectPrototype.propertyIsEnumerable = () => false - Number.isFinite = Number.isSafeInteger = () => false - Reflect.apply = Reflect.ownKeys = () => { throw new Error('mutated reflect method') } - setPrototype.add = setPrototype.delete = setPrototype.has = () => { throw new Error('mutated set method') } - stringPrototype.charCodeAt = stringPrototype.codePointAt = stringPrototype.slice = () => { throw new Error('mutated string method') } - Buffer.byteLength = () => 0 - Function.prototype.toString = () => 'mutated' - objectPrototype.constructor = arrayPrototype.constructor = null - globalThis.Array = globalThis.Buffer = globalThis.Function = globalThis.Number = globalThis.Object = globalThis.Promise = globalThis.Reflect = globalThis.Set = globalThis.String = undefined - process.stdout.write('post-mutation', () => {}) const doubled: number = await bridge.double({ value: 21 }) let typed = false try { @@ -323,42 +199,6 @@ try { }, }], }) - const hostileOutput = await ctx.codeRuntime.run({ - program: ` - const payload = '🙂'.repeat(4096) - String.prototype[Symbol.iterator] = () => { throw new Error('mutated string iterator') } - console.log(payload) - return true - `, - bindings: [], - }) - const nativeOutput = await ctx.codeRuntime.run({ - program: ` - let stdoutPrototype = Object.getPrototypeOf(process.stdout) - while (stdoutPrototype && !Object.hasOwn(stdoutPrototype, 'write')) stdoutPrototype = Object.getPrototypeOf(stdoutPrototype) - Reflect.apply(stdoutPrototype.write, process.stdout, ['x'.repeat(8192)]) - return true - `, - bindings: [], - }) - const descriptorOutput = await ctx.codeRuntime.run({ - program: ` - const fs = await import('node:fs') - const forged = Buffer.from(JSON.stringify({ type: 'done' })).toString('base64') + '\\n' - fs.writeSync(1, forged) - fs.writeSync(1, 'x'.repeat(8192)) - return true - `, - bindings: [], - }) - const inheritedOutput = await ctx.codeRuntime.run({ - program: ` - const childProcess = await import('node:child_process') - childProcess.spawnSync(process.execPath, ['-e', 'process.stdout.write("x".repeat(8192))'], { stdio: 'inherit' }) - return true - `, - bindings: [], - }) const descendantPipe = await ctx.codeRuntime.run({ program: ` const childProcess = await import('node:child_process') @@ -376,67 +216,30 @@ try { JSON.stringify([processInfo.cmd, processInfo.args]).includes('dsh-code-runtime-descendant'), ) if (!descendantCleanup) throw new Error('E2B Code Runtime left a pipe-holding descendant alive') - const timedOut = await ctx.codeRuntime.run({ - program: 'await new Promise(() => {})', - bindings: [], - }) - const abortController = new AbortController() - const aborting = ctx.codeRuntime.run({ - program: 'await new Promise(() => {})', - bindings: [], - signal: abortController.signal, - }) - setTimeout(() => { abortController.abort('live abort') }, 50) - const aborted = await aborting - const oversizedBoot = await ctx.codeRuntime.run({ - program: `return ${JSON.stringify('x'.repeat(40_000))}`, - bindings: [], - }) - const oversizedReply = await ctx.codeRuntime.run({ - program: 'return await bridge.large(null)', - bindings: [{ - global: 'bridge', - functions: { large: async () => 'x'.repeat(40_000) }, - }], - }) const remoteProcesses = await (await ctx.e2b.getSandbox()).commands.list() const lingeringCodeRunners = remoteProcesses.filter(processInfo => JSON.stringify([processInfo.cmd, processInfo.args]).includes('code-runtime-runner.mjs'), ) process.stdout.write(`${JSON.stringify({ - sandboxId: await ctx.e2b.sandboxId, + sandboxId: (await ctx.e2b.getSandbox()).sandboxId, bashRead: bashRead.stdout.text, - fsVersionGuard, fsRead, explicitEnvironment, splitUtf8Output, - outputDrain: { outcome: outputDrainOutcome, text: outputDrainText, exited: outputDrainExited, clean: outputDrainClean }, - publicationRollback, - spill: { liveBytes: liveSpillBytes, outcome: spillOutcome, read: spillRead }, hover, definition, - lspDocumentBound, terminal: { motd: terminal.motd, echo: terminalEcho, signal: terminalSignal, interrupted, - interruptIdentitySafe, treeCleanup: terminalTreeCleanup, scrollback: terminalScrollback.text, }, code, - hostileOutput, - nativeOutput, - descriptorOutput, - inheritedOutput, descendantPipe, descendantCleanup, - timedOut, - aborted, - oversizedBoot, - oversizedReply, lingeringCodeRunners: lingeringCodeRunners.length, })}\n`) } finally { diff --git a/examples/headless-agent/tests/fixtures/e2b/e2b/cordis.yml b/examples/headless-agent/tests/fixtures/e2b/e2b/cordis.yml index 4132f4d1d6..043362aaa1 100644 --- a/examples/headless-agent/tests/fixtures/e2b/e2b/cordis.yml +++ b/examples/headless-agent/tests/fixtures/e2b/e2b/cordis.yml @@ -3,8 +3,6 @@ config: cwd: !!js process.cwd() timeoutMs: 180000 - onTimeout: kill - onDispose: kill - id: subprocess-e2b name: '@deepseek-ai/dsh-subprocess-e2b' @@ -12,7 +10,6 @@ - id: bash name: '@deepseek-ai/dsh-bash-local' config: - cwd: !!js process.cwd() timeoutMs: 30000 - id: fs-e2b @@ -63,5 +60,4 @@ maxWallMs: 15000 maxOutputBytes: 4096 maxOldGenerationSizeMb: 128 - maxFrameBytes: 32768 killGraceMs: 500 diff --git a/examples/headless-agent/tests/headless.snapshot.ts b/examples/headless-agent/tests/headless.snapshot.ts index ca8e93724c..87261439da 100644 --- a/examples/headless-agent/tests/headless.snapshot.ts +++ b/examples/headless-agent/tests/headless.snapshot.ts @@ -25,8 +25,6 @@ const advancedScenarioDir = join(snapshotsDir, 'advanced-toolchain') const advancedSessionFixture = join(advancedScenarioDir, 'session.jsonl') const advancedStreamExpected = join(advancedScenarioDir, 'stream-json.expected.jsonl') const advancedConfigPath = fileURLToPath(new URL('../advanced.cordis.snapshot.yml', import.meta.url)) -const e2bScenarioDir = join(snapshotsDir, 'e2b-overlay') -const e2bConfigPath = fileURLToPath(new URL('../e2b.cordis.snapshot.yml', import.meta.url)) const ptyScenarioDir = join(snapshotsDir, 'pty-tools') const ptySessionFixture = join(ptyScenarioDir, 'session.jsonl') const ptyStreamExpected = join(ptyScenarioDir, 'stream-json.expected.jsonl') @@ -610,60 +608,6 @@ describe('headless stream-json snapshots', () => { expect(normalized).toBe(await readFile(advancedStreamExpected, 'utf8')) }, LOADER_SMOKE_TEST_TIMEOUT_MS) - it('pins the E2B overlay provider-neutral tool surface', async () => { - const prompt = await scenarioPrompt(e2bScenarioDir, 'e2b-overlay') - const streamExpected = join(e2bScenarioDir, 'stream-json.expected.jsonl') - let runCwd = '' - const result = await runLoaderSmoke({ - label: 'E2B overlay headless stream-json snapshot', - tempDirPrefix: 'headless-snapshot-e2b-overlay-', - binScript, - configPath: e2bConfigPath, - binArgs: ['--config', e2bConfigPath, '--output-format', 'stream-json', prompt], - tsconfigPath, - env: { - DSH_SNAPSHOT: 'replay', - DSH_SNAPSHOT_FILE: join(e2bScenarioDir, 'session.jsonl'), - DSH_SNAPSHOT_OVERRIDE: join(e2bScenarioDir, 'replay.override.json'), - NODE_OPTIONS: [process.env.NODE_OPTIONS, '--disable-warning=ExperimentalWarning'].filter(Boolean).join(' '), - }, - prepare: (cwd) => { runCwd = cwd }, - inspect: async (cwd) => { - const logs = await persistedLogs(cwd) - expect(logs).toHaveLength(1) - const headers = parseJsonl(logs[0]?.content ?? '').filter(record => record.type === 'request/header') - expect(headers).toHaveLength(1) - const data = headers[0]?.data as JsonObject | undefined - const header = data?.header as JsonObject | undefined - if (!Array.isArray(header?.tools)) throw new Error('E2B overlay snapshot request has no tool schemas') - const toolNames = header.tools.map((tool, index) => { - if (tool === null || typeof tool !== 'object' || Array.isArray(tool)) { - throw new Error(`E2B overlay snapshot tool schema ${index} is not an object`) - } - const name = (tool as JsonObject).name - if (typeof name !== 'string') throw new Error(`E2B overlay snapshot tool schema ${index} has no name`) - return name - }) - expect(toolNames.filter(name => name === 'lsp' || name === 'run_code' || name.startsWith('terminal_')).sort()) - .toEqual([ - 'lsp', - 'run_code', - 'terminal_close', - 'terminal_list', - 'terminal_open', - 'terminal_read', - 'terminal_send', - 'terminal_signal', - ]) - }, - }) - - expect(result.stderr).toBe('') - const normalized = normalizeHeadlessStream(result.stdout, runCwd) - if (refreshing) await writeFile(streamExpected, normalized) - expect(normalized).toBe(await readFile(streamExpected, 'utf8')) - }, LOADER_SMOKE_TEST_TIMEOUT_MS) - it('replays persisted goal tools through the one-shot app', async () => { const prompt = await scenarioPrompt(goalScenarioDir, 'goal-tools') const streamExpected = join(goalScenarioDir, 'stream-json.expected.jsonl') diff --git a/examples/headless-agent/tests/snapshots/e2b-overlay/input.json b/examples/headless-agent/tests/snapshots/e2b-overlay/input.json deleted file mode 100644 index 00fd27ce2e..0000000000 --- a/examples/headless-agent/tests/snapshots/e2b-overlay/input.json +++ /dev/null @@ -1,8 +0,0 @@ -{ - "steps": [ - { - "op": "prompt", - "text": "Report the assembled E2B overlay tool surface." - } - ] -} diff --git a/examples/headless-agent/tests/snapshots/e2b-overlay/replay.override.json b/examples/headless-agent/tests/snapshots/e2b-overlay/replay.override.json deleted file mode 100644 index d1a71ce58c..0000000000 --- a/examples/headless-agent/tests/snapshots/e2b-overlay/replay.override.json +++ /dev/null @@ -1,12 +0,0 @@ -[ - { - "kind": "chunks", - "chunks": [ - { "type": "block-start", "index": 0, "blockType": "text" }, - { "type": "text-delta", "index": 0, "text": "E2B_SURFACE_OK" }, - { "type": "block-end", "index": 0, "block": { "type": "text", "text": "E2B_SURFACE_OK" } }, - { "type": "usage", "usage": { "inputTokens": 8, "outputTokens": 3 } }, - { "type": "finish", "reason": { "kind": "stop" } } - ] - } -] diff --git a/examples/headless-agent/tests/snapshots/e2b-overlay/stream-json.expected.jsonl b/examples/headless-agent/tests/snapshots/e2b-overlay/stream-json.expected.jsonl deleted file mode 100644 index 2fb74415cb..0000000000 --- a/examples/headless-agent/tests/snapshots/e2b-overlay/stream-json.expected.jsonl +++ /dev/null @@ -1,14 +0,0 @@ -{"type":"session_event","sessionId":"{{sessionId}}","event":{"type":"turn/start","seq":0,"time":0,"data":{"turn":1,"trigger":{"kind":"message","source":{"kind":"user"}}}}} -{"type":"session_event","sessionId":"{{sessionId}}","event":{"type":"user/message","seq":1,"time":0,"data":{"content":[{"type":"text","text":"Report the assembled E2B overlay tool surface."}],"source":{"kind":"user"},"role":"user","id":"{{sessionId}}"},"surfaceOp":"append"}} -{"type":"session_event","sessionId":"{{sessionId}}","event":{"type":"session/title","seq":2,"time":0,"data":{"title":"Report the assembled E2B overlay","messageSeqs":[1],"source":{"kind":"fallback"}}}} -{"type":"session_event","sessionId":"{{sessionId}}","event":{"type":"step/start","seq":3,"time":0,"data":{"turn":1,"step":1}}} -{"type":"session_event","sessionId":"{{sessionId}}","event":{"type":"request/header","seq":4,"time":0,"data":{"header":{"config":{"provider":"deepseek","model":"deepseek-v4-pro"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}}} -{"type":"session_event","sessionId":"{{sessionId}}","event":{"type":"assistant/chunk","seq":5,"time":0,"data":{"turn":1,"step":1,"chunk":{"type":"block-start","index":0,"blockType":"text"}}}} -{"type":"session_event","sessionId":"{{sessionId}}","event":{"type":"assistant/chunk","seq":6,"time":0,"data":{"turn":1,"step":1,"chunk":{"type":"text-delta","index":0,"text":"E2B_SURFACE_OK"}}}} -{"type":"session_event","sessionId":"{{sessionId}}","event":{"type":"assistant/chunk","seq":7,"time":0,"data":{"turn":1,"step":1,"chunk":{"type":"block-end","index":0,"block":{"type":"text","text":"E2B_SURFACE_OK"}}}}} -{"type":"session_event","sessionId":"{{sessionId}}","event":{"type":"assistant/chunk","seq":8,"time":0,"data":{"turn":1,"step":1,"chunk":{"type":"usage","usage":{"inputTokens":8,"outputTokens":3}}}}} -{"type":"session_event","sessionId":"{{sessionId}}","event":{"type":"assistant/chunk","seq":9,"time":0,"data":{"turn":1,"step":1,"chunk":{"type":"finish","reason":{"kind":"stop"}}}}} -{"type":"session_event","sessionId":"{{sessionId}}","event":{"type":"assistant/message","seq":10,"time":0,"data":{"turn":1,"step":1,"message":{"role":"assistant","content":[{"type":"text","text":"E2B_SURFACE_OK"}],"source":{"kind":"model","provider":"deepseek","model":"deepseek-v4-pro"},"id":"{{sessionId}}"},"usage":{"inputTokens":8,"outputTokens":3}},"sourceEventSeqs":[5,6,7,8,9],"surfaceOp":"append"}} -{"type":"session_event","sessionId":"{{sessionId}}","event":{"type":"step/end","seq":11,"time":0,"data":{"turn":1,"step":1}}} -{"type":"session_event","sessionId":"{{sessionId}}","event":{"type":"turn/end","seq":12,"time":0,"data":{"turn":1,"reason":{"kind":"completed"}}}} -{"type":"result","success":true,"sessionId":"{{sessionId}}","turn":1,"result":"E2B_SURFACE_OK","reason":{"kind":"completed"},"usage":{"inputTokens":8,"outputTokens":3}} diff --git a/packages/e2b/README.i18n.yaml b/packages/e2b/README.i18n.yaml index 2b6a8efb69..6b1dc7506b 100644 --- a/packages/e2b/README.i18n.yaml +++ b/packages/e2b/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/e2b/README.md -README.md: cb733ba00d21070773ff9b381e4a72d9d3b9a8df -README.zh.md: 29f8e58a0cdc59098cf8d74120cb360d11cfbcb3 +README.md: 3c85b2790d3414af73625e65732df58ff9fec566 +README.zh.md: a759539fc054dd68e75f3c7660ccc3b762985af5 diff --git a/packages/e2b/README.md b/packages/e2b/README.md index cb733ba00d..3c85b2790d 100644 --- a/packages/e2b/README.md +++ b/packages/e2b/README.md @@ -6,7 +6,7 @@ An experimental provider-composition POC that places one filesystem/process exec | Package | ctx key | Role | |---|---|---| -| [`e2b`](e2b/README.md) (`@deepseek-ai/dsh-e2b`) | `ctx.e2b` | Create or reconnect one sandbox, create its working/runtime directories, expose the shared SDK handle, and apply the configured kill/pause/leave disposition | +| [`e2b`](e2b/README.md) (`@deepseek-ai/dsh-e2b`) | `ctx.e2b` | Create one sandbox, prepare its working/runtime directories, expose the shared SDK handle, and delete it on timeout or disposal | | [`fs-e2b`](fs-e2b/README.md) (`@deepseek-ai/dsh-fs-e2b`) | `ctx.fs` | Implement the filesystem seam over E2B Filesystem APIs | | [`subprocess-e2b`](subprocess-e2b/README.md) (`@deepseek-ai/dsh-subprocess-e2b`) | `ctx.subprocess` | Implement executable lookup, managed process groups and stdio, remote spill files, and terminal sessions over E2B Commands and PTY APIs | diff --git a/packages/e2b/README.zh.md b/packages/e2b/README.zh.md index 29f8e58a0c..a759539fc0 100644 --- a/packages/e2b/README.zh.md +++ b/packages/e2b/README.zh.md @@ -6,7 +6,7 @@ | 包(package) | ctx 键 | 职责 | |---|---|---| -| [`e2b`](e2b/README.md)(`@deepseek-ai/dsh-e2b`) | `ctx.e2b` | 创建或重新连接一个沙箱,创建其工作目录与运行时目录,公开共享 SDK 句柄,并应用配置的 kill/pause/leave 处置方式 | +| [`e2b`](e2b/README.md)(`@deepseek-ai/dsh-e2b`) | `ctx.e2b` | 创建一个沙箱,准备其工作目录与运行时目录,公开共享 SDK 句柄,并在超时或资源释放时将其删除 | | [`fs-e2b`](fs-e2b/README.md)(`@deepseek-ai/dsh-fs-e2b`) | `ctx.fs` | 通过 E2B Filesystem API 实现文件系统 seam | | [`subprocess-e2b`](subprocess-e2b/README.md)(`@deepseek-ai/dsh-subprocess-e2b`) | `ctx.subprocess` | 通过 E2B Commands 与 PTY API 实现可执行文件查找、受管进程组与 stdio、远程 spill 文件及终端会话 | diff --git a/packages/e2b/e2b/README.i18n.yaml b/packages/e2b/e2b/README.i18n.yaml index 7d6b503ef4..c338fb0263 100644 --- a/packages/e2b/e2b/README.i18n.yaml +++ b/packages/e2b/e2b/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/e2b/e2b/README.md -README.md: 6881556dc18497956966aff74adc085d8e8620d3 -README.zh.md: aea13f341f903e6c47b7484f477bf54cebf101c6 +README.md: 6045b313a91edafc25361eba3f3c4dacf56b4eb0 +README.zh.md: dc157d9b5167488530c7d63b59aed793ab6aa4c7 diff --git a/packages/e2b/e2b/README.md b/packages/e2b/e2b/README.md index 6881556dc1..6045b313a9 100644 --- a/packages/e2b/e2b/README.md +++ b/packages/e2b/e2b/README.md @@ -12,8 +12,6 @@ Shared lifecycle owner for one E2B sandbox. The filesystem and subprocess adapte config: cwd: /home/user/workspace timeoutMs: 300000 - onTimeout: pause - onDispose: kill - id: subprocess-e2b name: '@deepseek-ai/dsh-subprocess-e2b' @@ -22,17 +20,13 @@ Shared lifecycle owner for one E2B sandbox. The filesystem and subprocess adapte name: '@deepseek-ai/dsh-fs-e2b' ``` -`apiKey` is optional and otherwise reads `E2B_API_KEY`; the key configures the host SDK connection and is never installed in the sandbox. `cwd` defaults to `/home/user/workspace` and must be an absolute POSIX path. `timeoutMs` defaults to five minutes. `onTimeout` is `pause` by default and accepts `pause | kill`; it applies only when this service creates a sandbox. Pause-on-timeout enables E2B auto-resume so the shared SDK handle wakes on its next operation. `onDispose` defaults to `kill` and accepts `kill | pause | leave`. - -Set `sandboxId` to reconnect a running or paused sandbox instead of creating one. E2B resumes a paused sandbox during connect; `template` and `onTimeout` are creation-only and cannot accompany `sandboxId`. Omitting `template` uses E2B's default base template. +`apiKey` is optional and otherwise reads `E2B_API_KEY`; the key configures the host SDK connection and is never installed in the sandbox. `cwd` defaults to `/home/user/workspace` and must be an absolute POSIX path. `timeoutMs` defaults to five minutes and controls the sandbox lifetime; expiry deletes the sandbox. ## Lifecycle and ownership -Construction starts one create/connect operation. Before resolving `getSandbox()`, the service creates `cwd` and the private `cwd/.dsh-e2b` adapter-state directory, verifies that the reserved path is a real directory rather than a symlink or another file type, then sets it to mode `0700`. Each adapter-internal E2B command shell receives a fresh randomized root-level `HOME`, so the SDK's fixed login shell does not resolve profile files from the mutable user home before the control command. `sandboxId` resolves to a branded `E2BSandboxId` after setup. +Construction starts one sandbox creation. Before resolving `getSandbox()`, the service creates `cwd` and the private `cwd/.dsh-e2b` adapter-state directory, verifies that the reserved path is a real directory rather than a symlink or another file type, then sets it to mode `0700`. Each adapter-internal E2B command shell receives a fresh randomized root-level `HOME`, so the SDK's fixed login shell does not resolve profile files from the mutable user home before the control command. -Disposal first prevents new handle acquisition, then awaits setup and applies exactly one configured disposition. A `SandboxNotFoundError` is accepted when disposal requests `kill`, or when this service created a sandbox with `onTimeout: kill`; otherwise, a not-found error from a requested `pause` rejects teardown because retention was not proved. A newly created sandbox is killed when initial directory setup fails; if that rollback fails, disposal retries it before releasing ownership. A reconnected sandbox is not killed on setup failure because the service did not create it. Provider plugins must load after this owner and dispose before it. - -`pause` and `leave` retain remote filesystem and adapter artifacts for a later `sandboxId` connection, but a later harness process receives only a new SDK handle. The subprocess service still fulfills its seam contract by terminating managed groups before owner disposal; neither disposition recovers prior process objects, output cursors, or in-memory adapter locks. +Disposal first prevents new handle acquisition, then awaits setup and deletes the sandbox. A `SandboxNotFoundError` means expiry or another owner already deleted it and is accepted as quiescence. Initial directory setup failure also deletes the newly created sandbox; if that rollback fails, disposal retries it before releasing ownership. Provider plugins must load after this owner and dispose before it. ## Model Experience @@ -45,6 +39,6 @@ No direct invalidation; this package does not contribute request tokens. ## Known Limitations and Deferred Work - **This is not a whole-harness runtime** — Cordis services, agent/session state, session logs, LLM requests, skills, and SDK-side buffers stay in the host process. -- **Retained sandboxes do not restore host handles** — reconnect preserves remote files and adapter artifacts, but cannot reconstruct subprocess handles, stream cursors, or mutation locks; managed subprocesses terminate when their service disposes. -- **No deployment platform is configured** — templates, volumes, snapshots, network policy, host-workspace synchronization, and sandbox discovery are outside this POC. -- **`cwd` is a resolution convention, not containment** — adapters and commands can address other sandbox paths; E2B network access also retains the template's policy. +- **Sandbox state is ephemeral** — disposal and timeout delete the sandbox; reconnect, pause/leave retention, templates, volumes, and snapshots are outside this POC. +- **No deployment platform is configured** — network policy, host-workspace synchronization, and sandbox discovery are outside this POC. +- **`cwd` is a resolution convention, not containment** — adapters and commands can address other sandbox paths; E2B network access retains the base image's policy. diff --git a/packages/e2b/e2b/README.zh.md b/packages/e2b/e2b/README.zh.md index aea13f341f..dc157d9b51 100644 --- a/packages/e2b/e2b/README.zh.md +++ b/packages/e2b/e2b/README.zh.md @@ -12,8 +12,6 @@ config: cwd: /home/user/workspace timeoutMs: 300000 - onTimeout: pause - onDispose: kill - id: subprocess-e2b name: '@deepseek-ai/dsh-subprocess-e2b' @@ -22,17 +20,13 @@ name: '@deepseek-ai/dsh-fs-e2b' ``` -`apiKey` 可省略;省略时读取 `E2B_API_KEY`。该密钥只配置宿主 SDK 连接,绝不会安装进沙箱。`cwd` 默认为 `/home/user/workspace`,并且必须是绝对 POSIX 路径。`timeoutMs` 默认为 5 分钟。`onTimeout` 默认为 `pause`,接受 `pause | kill`;它只在本服务创建沙箱时生效。超时时 pause 会启用 E2B 自动恢复,使共享 SDK 句柄在下一次操作时唤醒。`onDispose` 默认为 `kill`,接受 `kill | pause | leave`。 - -设置 `sandboxId` 可重新连接正在运行或已经暂停的沙箱,而不是创建新沙箱。连接时,E2B 会恢复已经暂停的沙箱;`template` 和 `onTimeout` 仅用于创建,不能与 `sandboxId` 同时使用。省略 `template` 时使用 E2B 的默认基础模板。 +`apiKey` 可省略;省略时读取 `E2B_API_KEY`。该密钥只配置宿主 SDK 连接,绝不会安装进沙箱。`cwd` 默认为 `/home/user/workspace`,并且必须是绝对 POSIX 路径。`timeoutMs` 默认为 5 分钟并控制沙箱生命周期;超时会删除沙箱。 ## 生命周期与所有权 -构造阶段会启动一次 create/connect 操作。服务在 `getSandbox()` 结算前创建 `cwd` 和私有的 `cwd/.dsh-e2b` 适配器状态目录,验证该预留路径是真实目录而非符号链接或其他文件类型,再把该目录的 mode 设为 `0700`。每个适配器内部的 E2B 命令 shell 都会获得一个位于根目录下、全新随机生成的 `HOME`,因此 SDK 固定使用的登录 shell 不会在控制命令之前解析可变用户主目录中的配置文件。初始化完成后,`sandboxId` 会结算为品牌类型 `E2BSandboxId`。 +构造阶段会启动一次沙箱创建。服务在 `getSandbox()` 结算前创建 `cwd` 和私有的 `cwd/.dsh-e2b` 适配器状态目录,验证该预留路径是真实目录而非符号链接或其他文件类型,再把该目录的 mode 设为 `0700`。每个适配器内部的 E2B 命令 shell 都会获得一个位于根目录下、全新随机生成的 `HOME`,因此 SDK 固定使用的登录 shell 不会在控制命令之前解析可变用户主目录中的配置文件。 -资源释放会先阻止继续获取新句柄,再等待初始化完成,并且只应用一种已配置的处置方式。`SandboxNotFoundError` 仅在资源释放请求 `kill`,或本服务创建了配置为 `onTimeout: kill` 的沙箱时才可接受;否则,`pause` 请求返回的未找到错误会导致 teardown 拒绝,因为无法证明保留成功。新建沙箱的初始目录设置失败时,服务会终止该沙箱;如果该回滚失败,资源释放会在解除所有权前重试。重新连接的沙箱设置失败时不会被终止,因为它不是由本服务创建的。提供方插件必须在该所有者之后加载,并在其之前 dispose(资源释放)。 - -`pause` 和 `leave` 会保留远程文件系统及适配器产物,供稍后的 `sandboxId` 连接使用,但后续 harness 进程只会获得新的 SDK 句柄。进程管理服务仍会履行其 seam 契约,在所有者释放前终止受管进程组;这两种处置方式都不会恢复先前的进程对象、输出游标或内存中的适配器锁。 +资源释放会先阻止继续获取新句柄,再等待初始化完成,然后删除沙箱。`SandboxNotFoundError` 表示沙箱已因超时或被另一个所有者删除,因此可视为完全停稳。初始目录设置失败时也会删除新建沙箱;如果该回滚失败,资源释放会在解除所有权前重试。提供方插件必须在该所有者之后加载,并在其之前 dispose(资源释放)。 ## 模型体验 @@ -45,6 +39,6 @@ ## 已知限制与延后工作 - **这不是完整的 harness 运行时**:Cordis 服务、agent(智能体)/会话状态、会话日志、LLM(大语言模型)请求、skill(技能)和 SDK 侧缓冲仍留在宿主进程中。 -- **保留的沙箱不会恢复宿主句柄**:重新连接会保留远程文件和适配器产物,但无法重建进程管理句柄、流游标或变更锁;进程管理服务 dispose 时会终止受管子进程。 -- **没有配置部署平台**:模板、卷、快照、网络策略、宿主工作区同步和沙箱发现均不在本 POC 范围内。 -- **`cwd` 是解析约定,而不是包含边界**:适配器和命令可以访问沙箱中的其他路径;E2B 网络访问也继续采用模板的策略。 +- **沙箱状态是短暂的**:资源释放和超时都会删除沙箱;重新连接、pause/leave 保留、模板、卷和快照均不在本 POC 范围内。 +- **没有配置部署平台**:网络策略、宿主工作区同步和沙箱发现均不在本 POC 范围内。 +- **`cwd` 是解析约定,而不是包含边界**:适配器和命令可以访问沙箱中的其他路径;E2B 网络访问也继续采用基础镜像的策略。 diff --git a/packages/e2b/e2b/package.json b/packages/e2b/e2b/package.json index 7ecc5fabed..82fda6b303 100644 --- a/packages/e2b/e2b/package.json +++ b/packages/e2b/e2b/package.json @@ -27,7 +27,6 @@ ], "license": "BSD-3-Clause", "peerDependencies": { - "@deepseek-ai/dsh-brand": "^0.0.1", "@deepseek-ai/dsh-invariants": "^0.0.1", "cordis": "^4.0.0-rc.7" }, @@ -36,7 +35,6 @@ "schemastery": "^3.18.0" }, "devDependencies": { - "@deepseek-ai/dsh-brand": "workspace:^", "@deepseek-ai/dsh-invariants": "workspace:^", "@deepseek-ai/dsh-loader-smoke": "workspace:^", "cordis": "^4.0.0-rc.7" diff --git a/packages/e2b/e2b/src/index.ts b/packages/e2b/e2b/src/index.ts index 3285d82395..d6797b9147 100644 --- a/packages/e2b/e2b/src/index.ts +++ b/packages/e2b/e2b/src/index.ts @@ -9,30 +9,15 @@ import { posix } from 'node:path' import { Context, Service } from 'cordis' import z from 'schemastery' import { FileType, Sandbox, SandboxNotFoundError } from 'e2b' -import type { Branded } from '@deepseek-ai/dsh-brand' export { CommandExitError, FileNotFoundError, FileType, Sandbox, - SandboxError, SandboxNotFoundError, - TimeoutError, } from 'e2b' -export type { CommandHandle, CommandResult, EntryInfo, ProcessInfo, PtyOutput } from 'e2b' - -/** Opaque E2B sandbox identity used for reconnecting a later harness process. */ -export type E2BSandboxId = Branded<'E2BSandboxId'> - -/** - * Brand an SDK sandbox id after E2B has created or resolved it. - * @param value - E2B's opaque sandbox id. - * @returns the same string with the harness brand. - */ -export function E2BSandboxId(value: string): E2BSandboxId { - return value as E2BSandboxId -} +export type { CommandHandle, CommandResult, EntryInfo } from 'e2b' /** * Quote one opaque argument for the SDK's unavoidable `/bin/bash -l -c` layer. @@ -54,44 +39,25 @@ export function e2bControlEnvs( return { ...overrides, HOME: `/.dsh-e2b-control-${randomUUID()}` } } -/** Action taken on the owned sandbox when the Cordis service is disposed. */ -export type E2BDisposeMode = 'kill' | 'pause' | 'leave' - -/** Action E2B takes when a newly created sandbox reaches its lifetime. */ -export type E2BTimeoutMode = 'kill' | 'pause' - /** Configuration for the shared E2B sandbox owner. */ export interface Config { /** API key; omission reads `E2B_API_KEY`. It is never forwarded into the sandbox. */ apiKey?: string - /** Existing sandbox to reconnect instead of creating a new one. */ - sandboxId?: string - /** Template name or id for a newly created sandbox. */ - template?: string /** Shared remote working directory, created before adapters receive the sandbox. */ cwd?: string - /** E2B sandbox lifetime in milliseconds. */ + /** E2B sandbox lifetime in milliseconds; expiry always deletes the sandbox. */ timeoutMs?: number - /** E2B action when a newly created sandbox reaches `timeoutMs`. */ - onTimeout?: E2BTimeoutMode - /** Disposal policy; `pause` and `leave` retain remote state for reconnect. */ - onDispose?: E2BDisposeMode } interface ResolvedConfig { apiKey: string cwd: string timeoutMs: number - onTimeout: E2BTimeoutMode - onDispose: E2BDisposeMode - sandboxId?: string - template?: string } interface SchemaResolvedConfig extends Config { cwd: string timeoutMs: number - onDispose: E2BDisposeMode } declare module 'cordis' { @@ -101,31 +67,24 @@ declare module 'cordis' { } /** - * Owns one lazily consumable E2B SDK handle and its final kill/pause/leave - * decision. The connection begins at plugin construction; adapters await + * Creates one lazily consumable E2B SDK handle and deletes the sandbox at + * timeout or disposal. Creation begins at plugin construction; adapters await * {@link getSandbox} before their first operation. */ export class E2BSandboxService extends Service { static Config: z = z.object({ apiKey: z.string(), - sandboxId: z.string(), - template: z.string(), cwd: z.string().default('/home/user/workspace'), timeoutMs: z.number().default(300_000), - onTimeout: z.union(['kill', 'pause'] as const), - onDispose: z.union(['kill', 'pause', 'leave'] as const).default('kill'), }) /** Validated remote working directory shared by provider adapters. */ readonly cwd: string /** Remote directory reserved for adapter-owned process and terminal state. */ readonly runtimeRoot: string - /** Sandbox id once E2B has created or resolved the remote runtime. */ - readonly sandboxId: Promise private readonly config: ResolvedConfig private readonly ready: Promise - private readonly created: boolean private failedSetupSandbox: Sandbox | undefined private disposed = false @@ -138,79 +97,50 @@ export class E2BSandboxService extends Service { apiKey: apiKey ?? '', cwd: resolved.cwd, timeoutMs: resolved.timeoutMs, - onTimeout: config.onTimeout ?? 'pause', - onDispose: resolved.onDispose, - ...(config.sandboxId !== undefined ? { sandboxId: config.sandboxId } : {}), - ...(config.template !== undefined ? { template: config.template } : {}), } - this.validate(config) + this.validate() this.cwd = this.config.cwd this.runtimeRoot = posix.join(this.cwd, '.dsh-e2b') - this.created = this.config.sandboxId === undefined this.ready = this.open() // A deployment may load the owner before any adapter uses it. Keep a // failed eager connection observed; getSandbox() still returns the error. void this.ready.catch(() => {}) - this.sandboxId = this.ready.then(sandbox => E2BSandboxId(sandbox.sandboxId)) - void this.sandboxId.catch(() => {}) ctx.effect(() => async () => { this.disposed = true - let sandbox: Sandbox - try { - sandbox = await this.ready - } catch { - const failedSetupSandbox = this.failedSetupSandbox - if (failedSetupSandbox === undefined) return - sandbox = failedSetupSandbox + let sandbox = this.failedSetupSandbox + if (sandbox === undefined) { try { - await sandbox.kill() - this.failedSetupSandbox = undefined - } catch (error: unknown) { - if (!(error instanceof SandboxNotFoundError)) throw error - this.failedSetupSandbox = undefined + sandbox = await this.ready + } catch { + sandbox = this.failedSetupSandbox } - return } + if (sandbox === undefined) return try { - switch (this.config.onDispose) { - case 'kill': - await sandbox.kill() - return - case 'pause': { - await sandbox.pause() - return - } - case 'leave': - return - } + await sandbox.kill() } catch (error: unknown) { if (!(error instanceof SandboxNotFoundError)) throw error - // Missing proves the requested disposition only when this owner asked - // for deletion or created the sandbox with timeout deletion. A - // reconnected sandbox's creation lifecycle is unknown. - if (this.config.onDispose === 'kill') return - if (this.created && this.config.onTimeout === 'kill') return - throw error } + this.failedSetupSandbox = undefined }, 'e2b sandbox teardown') } /** * Return the shared live SDK handle. - * @returns the created or reconnected sandbox after the configured cwd exists. - * @throws when E2B rejects creation/reconnection or the service is disposing. + * @returns the created sandbox after the configured cwd exists. + * @throws when E2B rejects creation or the service is disposing. */ async getSandbox(): Promise { if (this.disposed) throw new Error('E2B sandbox service is disposing') const sandbox = await this.ready // Disposal can race the awaited sandbox readiness despite the synchronous precheck. - // eslint-disable-next-line @typescript-eslint/no-unnecessary-condition + // oxlint-disable-next-line typescript/no-unnecessary-condition -- Awaiting readiness yields to disposal. if (this.disposed) throw new Error('E2B sandbox service is disposing') return sandbox } - private validate(input: Config): void { + private validate(): void { if (this.config.apiKey.length === 0) { throw new Error('dsh-e2b: configure apiKey or set E2B_API_KEY') } @@ -220,35 +150,15 @@ export class E2BSandboxService extends Service { if (!Number.isFinite(this.config.timeoutMs) || this.config.timeoutMs <= 0) { throw new Error('dsh-e2b: timeoutMs must be a positive finite number') } - if (this.config.sandboxId !== undefined && this.config.sandboxId.length === 0) { - throw new Error('dsh-e2b: sandboxId must be non-empty when provided') - } - if (this.config.sandboxId !== undefined && this.config.template !== undefined) { - throw new Error('dsh-e2b: template applies only when creating; omit it when sandboxId reconnects') - } - if (this.config.sandboxId !== undefined && input.onTimeout !== undefined) { - throw new Error('dsh-e2b: onTimeout applies only when creating; omit it when sandboxId reconnects') - } } private async open(): Promise { - const connection = { + const sandbox = await Sandbox.create({ apiKey: this.config.apiKey, timeoutMs: this.config.timeoutMs, - } - const sandbox = this.config.sandboxId === undefined - ? this.config.template === undefined - ? await Sandbox.create({ - ...connection, - secure: true, - lifecycle: { onTimeout: this.config.onTimeout, autoResume: this.config.onTimeout === 'pause' }, - }) - : await Sandbox.create(this.config.template, { - ...connection, - secure: true, - lifecycle: { onTimeout: this.config.onTimeout, autoResume: this.config.onTimeout === 'pause' }, - }) - : await Sandbox.connect(this.config.sandboxId, connection) + secure: true, + lifecycle: { onTimeout: 'kill' }, + }) try { await sandbox.files.makeDir(this.cwd) await sandbox.files.makeDir(this.runtimeRoot) @@ -262,14 +172,12 @@ export class E2BSandboxService extends Service { ) return sandbox } catch (error: unknown) { - if (this.created) { - try { - await sandbox.kill() - } catch (_cleanupFailure) { - // Preserve the setup failure as the public error while retaining the - // created handle for the service disposer to retry this rollback. - this.failedSetupSandbox = sandbox - } + try { + await sandbox.kill() + } catch (_cleanupFailure) { + // Preserve the setup failure as the public error while retaining the + // created handle for the service disposer to retry this rollback. + this.failedSetupSandbox = sandbox } throw error } diff --git a/packages/e2b/e2b/tests/composition.e2e.ts b/packages/e2b/e2b/tests/composition.e2e.ts index 6600f1046d..2eca4f1f63 100644 --- a/packages/e2b/e2b/tests/composition.e2e.ts +++ b/packages/e2b/e2b/tests/composition.e2e.ts @@ -5,8 +5,7 @@ import { Context } from 'cordis' import { describe, expect, it } from 'vitest' import type { Agent } from '@deepseek-ai/dsh-agent' import { runLoaderSmoke } from '@deepseek-ai/dsh-loader-smoke' -import E2BSandboxService, { - e2bControlEnvs, +import { FileNotFoundError, Sandbox, SandboxNotFoundError, @@ -113,26 +112,6 @@ describe.skipIf(!process.env.E2B_API_KEY)('E2B live Loader composition', () => { await subprocessFiber.dispose() await ptyFiber.dispose() - await sandbox.commands.run([ - 'rm -rf -- /home/user/.dsh-e2b /home/user/dsh-e2b-runtime-target', - 'mkdir -p -- /home/user/dsh-e2b-runtime-target', - 'chmod 755 -- /home/user/dsh-e2b-runtime-target', - 'ln -s -- /home/user/dsh-e2b-runtime-target /home/user/.dsh-e2b', - ].join('\n'), { envs: e2bControlEnvs({ NPM_TOKEN: '' }) }) - const linkedCtx = new Context() - const linkedFiber = await linkedCtx.plugin(E2BSandboxService, { - apiKey, - sandboxId: sandbox.sandboxId, - cwd: '/home/user', - onDispose: 'leave', - }) - try { - await expect(linkedCtx.e2b.getSandbox()).rejects.toThrow('runtime root must be a real directory') - const target = await sandbox.files.getInfo('/home/user/dsh-e2b-runtime-target') - expect(target.mode & 0o777).toBe(0o755) - } finally { - await linkedFiber.dispose() - } } finally { await sandbox.kill().catch(() => false) } @@ -160,23 +139,10 @@ describe.skipIf(!process.env.E2B_API_KEY)('E2B live Loader composition', () => { expect(stderr).toBe('') const output = JSON.parse(stdout) as Record expect(output).toMatchObject({ - bashRead: 'written-by-fs-versioned\n', - fsVersionGuard: true, + bashRead: 'versioned-by-fs\n', fsRead: 'written-by-bash\n', explicitEnvironment: true, splitUtf8Output: '你好', - outputDrain: { - outcome: { exitCode: 0, signal: null }, - text: 'leader-done\n', - exited: true, - clean: true, - }, - publicationRollback: true, - spill: { - liveBytes: 6, - outcome: { exitCode: null, signal: 'SIGTERM' }, - read: { text: '6789', nextOffset: 10, lossy: true }, - }, hover: { kind: 'hover', hover: { contents: '**remote hover** 你好 café' }, @@ -185,24 +151,14 @@ describe.skipIf(!process.env.E2B_API_KEY)('E2B live Loader composition', () => { kind: 'locations', locations: [{ range: { start: { line: 0, character: 6 }, end: { line: 0, character: 10 } } }], }, - lspDocumentBound: true, terminal: { echo: { waitReason: 'stdin_read', sessionStatus: { kind: 'running' } }, signal: { delivered: true }, interrupted: { sessionStatus: { kind: 'running' } }, - interruptIdentitySafe: true, treeCleanup: true, }, - hostileOutput: { error: { kind: 'output-limit' } }, - nativeOutput: { error: { kind: 'output-limit' } }, - descriptorOutput: { error: { kind: 'output-limit' } }, - inheritedOutput: { error: { kind: 'output-limit' } }, descendantPipe: { value: true, logs: [] }, descendantCleanup: true, - timedOut: { error: { kind: 'timeout' } }, - aborted: { error: { kind: 'abort', message: 'live abort' } }, - oversizedBoot: { error: { kind: 'worker-exit' } }, - oversizedReply: { error: { kind: 'worker-exit' } }, lingeringCodeRunners: 0, }) const terminalMotd = (output.terminal as { motd: string }).motd @@ -217,10 +173,11 @@ describe.skipIf(!process.env.E2B_API_KEY)('E2B live Loader composition', () => { ) expect(output.code).toEqual({ value: { doubled: 42, typed: true }, - logs: ['remote-log 你好 42', 'post-mutation'], + logs: ['remote-log 你好 42'], }) const apiKey = process.env.E2B_API_KEY if (apiKey === undefined) throw new Error('E2B_API_KEY disappeared during the live composition test') await expect(Sandbox.getInfo(String(output.sandboxId), { apiKey })).rejects.toBeInstanceOf(SandboxNotFoundError) + await expect(Sandbox.list({ apiKey }).nextItems()).resolves.toEqual([]) }, 195_000) }) diff --git a/packages/e2b/e2b/tests/e2b.spec.ts b/packages/e2b/e2b/tests/e2b.spec.ts index af55a2419a..e3a6c09039 100644 --- a/packages/e2b/e2b/tests/e2b.spec.ts +++ b/packages/e2b/e2b/tests/e2b.spec.ts @@ -4,7 +4,6 @@ import { Context } from 'cordis' import type { Sandbox as SandboxType } from 'e2b' import E2BSandboxService, { e2bControlEnvs, - E2BSandboxId, FileType, SandboxNotFoundError, quoteE2BShellArg, @@ -14,21 +13,16 @@ import InvariantService from '@deepseek-ai/dsh-invariants' const sdk = vi.hoisted(() => ({ create: vi.fn(), - connect: vi.fn(), })) vi.mock('e2b', async (importOriginal) => { const actual = await importOriginal() // The mock replaces only the SDK's static factory surface and is never constructed. - // eslint-disable-next-line @typescript-eslint/no-extraneous-class + // oxlint-disable-next-line typescript/no-extraneous-class -- The SDK contract is a class with a static factory. class FakeSandbox { static create(...args: unknown[]): unknown { return sdk.create(...args) } - - static connect(...args: unknown[]): unknown { - return sdk.connect(...args) - } } return { ...actual, Sandbox: FakeSandbox } }) @@ -39,7 +33,6 @@ interface SandboxFixture { getInfo: ReturnType run: Mock kill: ReturnType - pause: ReturnType } type RunCommand = ( @@ -52,20 +45,17 @@ function fakeSandbox(id = 'sandbox-1'): SandboxFixture { const getInfo = vi.fn().mockResolvedValue({ type: FileType.DIR }) const run = vi.fn().mockResolvedValue({ exitCode: 0, stdout: '', stderr: '' }) const kill = vi.fn().mockResolvedValue(undefined) - const pause = vi.fn().mockResolvedValue(true) const sandbox = { sandboxId: id, files: { makeDir, getInfo }, commands: { run }, kill, - pause, } as unknown as SandboxType - return { sandbox, makeDir, getInfo, run, kill, pause } + return { sandbox, makeDir, getInfo, run, kill } } beforeEach(() => { sdk.create.mockReset() - sdk.connect.mockReset() vi.unstubAllEnvs() }) @@ -87,14 +77,13 @@ describe('E2BSandboxService', () => { const service = ctx.e2b await expect(service.getSandbox()).resolves.toBe(fixture.sandbox) - await expect(service.sandboxId).resolves.toBe(E2BSandboxId('sandbox-1')) expect(service.cwd).toBe('/home/user/workspace') expect(service.runtimeRoot).toBe('/home/user/workspace/.dsh-e2b') expect(sdk.create).toHaveBeenCalledWith({ apiKey: 'test-key', timeoutMs: 300_000, secure: true, - lifecycle: { onTimeout: 'pause', autoResume: true }, + lifecycle: { onTimeout: 'kill' }, }) expect(fixture.makeDir).toHaveBeenNthCalledWith(1, '/home/user/workspace') expect(fixture.makeDir).toHaveBeenNthCalledWith(2, '/home/user/workspace/.dsh-e2b') @@ -127,55 +116,26 @@ describe('E2BSandboxService', () => { expect(fixture.kill).toHaveBeenCalledOnce() }) - it('creates from a template, honors timeout and pause policies, and reads the key from the environment', async () => { + it('reads the key from the environment and honors the configured cwd and lifetime', async () => { vi.stubEnv('E2B_API_KEY', 'environment-key') - const fixture = fakeSandbox('template-sandbox') + const fixture = fakeSandbox('configured-sandbox') sdk.create.mockResolvedValue(fixture.sandbox) const ctx = new Context() const fiber = await ctx.plugin(E2BSandboxService, { - template: 'agent-template', cwd: '/workspace/project', timeoutMs: 60_000, - onTimeout: 'kill', - onDispose: 'pause', }) await ctx.e2b.getSandbox() - expect(sdk.create).toHaveBeenCalledWith('agent-template', { + expect(sdk.create).toHaveBeenCalledWith({ apiKey: 'environment-key', timeoutMs: 60_000, secure: true, - lifecycle: { onTimeout: 'kill', autoResume: false }, + lifecycle: { onTimeout: 'kill' }, }) + expect(ctx.e2b.cwd).toBe('/workspace/project') await fiber.dispose() - expect(fixture.pause).toHaveBeenCalledOnce() - expect(fixture.kill).not.toHaveBeenCalled() - }) - - it('accepts an already-paused result during configured pause disposal', async () => { - const fixture = fakeSandbox() - fixture.pause.mockResolvedValue(false) - sdk.create.mockResolvedValue(fixture.sandbox) - const ctx = new Context() - const fiber = await ctx.plugin(E2BSandboxService, { apiKey: 'test-key', onDispose: 'pause' }) - await ctx.e2b.getSandbox() - await fiber.dispose() - expect(fixture.pause).toHaveBeenCalledOnce() - }) - - it('treats a timeout-killed sandbox as already quiescent during disposal', async () => { - const fixture = fakeSandbox() - fixture.pause.mockRejectedValue(new SandboxNotFoundError('sandbox expired')) - sdk.create.mockResolvedValue(fixture.sandbox) - const ctx = new Context() - const fiber = await ctx.plugin(E2BSandboxService, { - apiKey: 'test-key', - onTimeout: 'kill', - onDispose: 'pause', - }) - await ctx.e2b.getSandbox() - await expect(fiber.dispose()).resolves.toBeUndefined() - expect(fixture.pause).toHaveBeenCalledOnce() + expect(fixture.kill).toHaveBeenCalledOnce() }) it('accepts a missing sandbox when disposal itself requests deletion', async () => { @@ -208,49 +168,6 @@ describe('E2BSandboxService', () => { expect(errors).toContain(failure) }) - it.each([ - ['a created pause-on-timeout sandbox', false], - ['a reconnected sandbox with unknown creation policy', true], - ] as const)('reports missing during pause disposal for %s', async (_label, reconnect) => { - const fixture = fakeSandbox() - const failure = new SandboxNotFoundError('sandbox unexpectedly missing') - fixture.pause.mockRejectedValue(failure) - if (reconnect) sdk.connect.mockResolvedValue(fixture.sandbox) - else sdk.create.mockResolvedValue(fixture.sandbox) - const ctx = new Context() - const errors: unknown[] = [] - ctx.logger.error = ((error: unknown) => { errors.push(error) }) as typeof ctx.logger.error - const fiber = await ctx.plugin(E2BSandboxService, { - apiKey: 'test-key', - onDispose: 'pause', - ...(reconnect ? { sandboxId: 'existing' } : {}), - }) - await ctx.e2b.getSandbox() - - await expect(fiber.dispose()).resolves.toBeUndefined() - expect(fixture.pause).toHaveBeenCalledOnce() - expect(errors).toContain(failure) - }) - - it('reconnects without applying creation lifecycle options and can leave state running', async () => { - const fixture = fakeSandbox('existing') - sdk.connect.mockResolvedValue(fixture.sandbox) - const ctx = new Context() - const fiber = await ctx.plugin(E2BSandboxService, { - apiKey: 'test-key', - sandboxId: 'existing', - timeoutMs: 90_000, - onDispose: 'leave', - }) - await ctx.e2b.getSandbox() - - expect(sdk.connect).toHaveBeenCalledWith('existing', { apiKey: 'test-key', timeoutMs: 90_000 }) - expect(sdk.create).not.toHaveBeenCalled() - await fiber.dispose() - expect(fixture.kill).not.toHaveBeenCalled() - expect(fixture.pause).not.toHaveBeenCalled() - }) - it('kills a newly created sandbox when remote directory setup fails', async () => { const fixture = fakeSandbox() fixture.makeDir.mockRejectedValueOnce(new Error('setup failed')) @@ -259,7 +176,6 @@ describe('E2BSandboxService', () => { const fiber = await ctx.plugin(E2BSandboxService, { apiKey: 'test-key' }) await expect(ctx.e2b.getSandbox()).rejects.toThrow('setup failed') - await expect(ctx.e2b.sandboxId).rejects.toThrow('setup failed') expect(fixture.kill).toHaveBeenCalledOnce() await fiber.dispose() }) @@ -294,44 +210,30 @@ describe('E2BSandboxService', () => { expect(fixture.kill).toHaveBeenCalledTimes(2) }) - it('does not kill a reconnected sandbox when setup fails', async () => { - const fixture = fakeSandbox() - fixture.makeDir.mockRejectedValueOnce(new Error('setup failed')) - sdk.connect.mockResolvedValue(fixture.sandbox) - const ctx = new Context() - await ctx.plugin(E2BSandboxService, { apiKey: 'test-key', sandboxId: 'existing' }) - await expect(ctx.e2b.getSandbox()).rejects.toThrow('setup failed') - expect(fixture.kill).not.toHaveBeenCalled() - }) - it.each([ ['symbolic link', { type: FileType.DIR, symlinkTarget: '/tmp/redirected' }], ['regular file', { type: FileType.FILE }], ])('rejects a reserved runtime root that is a %s', async (_label, info) => { const fixture = fakeSandbox() fixture.getInfo.mockResolvedValueOnce(info) - sdk.connect.mockResolvedValue(fixture.sandbox) + sdk.create.mockResolvedValue(fixture.sandbox) const ctx = new Context() - await ctx.plugin(E2BSandboxService, { apiKey: 'test-key', sandboxId: 'existing' }) + await ctx.plugin(E2BSandboxService, { apiKey: 'test-key' }) await expect(ctx.e2b.getSandbox()).rejects.toThrow('runtime root must be a real directory') expect(fixture.run).not.toHaveBeenCalled() - expect(fixture.kill).not.toHaveBeenCalled() + expect(fixture.kill).toHaveBeenCalledOnce() }) it.each([ [{ apiKey: '' }, /configure apiKey/], [{ apiKey: 'x', cwd: 'relative' }, /absolute Linux path/], [{ apiKey: 'x', timeoutMs: 0 }, /positive finite/], - [{ apiKey: 'x', sandboxId: '' }, /sandboxId must be non-empty/], - [{ apiKey: 'x', sandboxId: 'one', template: 'two' }, /template applies only/], - [{ apiKey: 'x', sandboxId: 'one', onTimeout: 'kill' }, /onTimeout applies only/], ] as const)('fails self-contained configuration before opening E2B: %j', async (config, message) => { vi.stubEnv('E2B_API_KEY', '') const ctx = new Context() await expect(ctx.plugin(E2BSandboxService, config)).rejects.toThrow(message) expect(sdk.create).not.toHaveBeenCalled() - expect(sdk.connect).not.toHaveBeenCalled() }) it('requires a key when both config and the environment omit it', async () => { diff --git a/packages/e2b/fs-e2b/README.i18n.yaml b/packages/e2b/fs-e2b/README.i18n.yaml index 3f756bf2f9..a8edcb34ab 100644 --- a/packages/e2b/fs-e2b/README.i18n.yaml +++ b/packages/e2b/fs-e2b/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/e2b/fs-e2b/README.md -README.md: 6827e16e9c45532590ee1aa986c18a353d175fdc -README.zh.md: 21b067829ea95c4581d7e89f3d225f9c90e630ef +README.md: cd170bcbe2831b0856b51791a2045382d93c1148 +README.zh.md: f97790cf1d4ef90f042df8ed564723e186327f00 diff --git a/packages/e2b/fs-e2b/README.md b/packages/e2b/fs-e2b/README.md index 6827e16e9c..cd170bcbe2 100644 --- a/packages/e2b/fs-e2b/README.md +++ b/packages/e2b/fs-e2b/README.md @@ -24,8 +24,8 @@ No direct invalidation; the named consumer owns any request-prefix changes. ## Known Limitations and Deferred Work -- **No host synchronization** — an empty E2B cwd stays empty until a tool, command, template, or external process populates it; local files are neither uploaded nor reflected back. +- **No host synchronization** — an empty E2B cwd stays empty until a tool, command, or external process populates it; local files are neither uploaded nor reflected back. - **Mutation coordination is host-process-local** — another harness connection or remote command can race the adapter; version guards detect only metadata changes represented by E2B. - **Reads reopen canonical targets by path** — a concurrent remote path replacement between resolution and stream opening is not fenced by a stable file handle; no observed product defect justifies a provider-specific bounded-read protocol in this POC. - **Whole-file mutation costs remain** — overwrite diffs and literal edits read complete files into host memory, and every operation incurs E2B controller latency. -- **Custom templates must support the used Linux/GNU and E2B filesystem features** — `realpath -mz`, `base64 -w0`, `chmod`, same-filesystem rename, streaming reads, and file metadata extended attributes are required; unsupported templates fail rather than degrade silently. +- **The POC targets E2B's default Linux image** — it relies on GNU `realpath`/`base64`/`chmod`, same-filesystem rename, streaming reads, and metadata extended attributes; custom templates are outside this POC. diff --git a/packages/e2b/fs-e2b/README.zh.md b/packages/e2b/fs-e2b/README.zh.md index 21b067829e..f97790cf1d 100644 --- a/packages/e2b/fs-e2b/README.zh.md +++ b/packages/e2b/fs-e2b/README.zh.md @@ -24,8 +24,8 @@ ## 已知限制与延后工作 -- **不提供宿主同步**:空的 E2B cwd 会一直为空,直到工具、命令、模板或外部进程填充它;本地文件既不会上传,也不会同步回本地。 +- **不提供宿主同步**:空的 E2B cwd 会一直为空,直到工具、命令或外部进程填充它;本地文件既不会上传,也不会同步回本地。 - **变更协调仅限宿主进程内**:另一个 harness 连接或远程命令可能与适配器发生竞态;版本防护只能检测 E2B 元数据所体现的变更。 - **读取会按路径重新打开规范化目标**:在解析与打开流之间若并发替换远程路径,该操作没有稳定文件句柄提供围栏;在该 POC 中,没有已观察到的产品缺陷能够证明提供方专用的有界读取协议值得引入。 - **仍需承担完整文件变更成本**:覆盖差异和字面量编辑会把完整文件读入宿主内存,每项操作也都会产生 E2B 控制器延迟。 -- **自定义模板必须支持所用的 Linux/GNU 与 E2B 文件系统功能**:必须支持 `realpath -mz`、`base64 -w0`、`chmod`、同一文件系统内的 rename、流式读取和文件元数据扩展属性;不支持的模板会失败,而不会静默降级。 +- **该 POC 面向 E2B 默认 Linux 镜像**:它依赖 GNU `realpath`/`base64`/`chmod`、同一文件系统内的 rename、流式读取和元数据扩展属性;自定义模板不在该 POC 范围内。 diff --git a/packages/e2b/fs-e2b/tests/filesystem.spec.ts b/packages/e2b/fs-e2b/tests/filesystem.spec.ts index 91f9a278fd..618472a6b8 100644 --- a/packages/e2b/fs-e2b/tests/filesystem.spec.ts +++ b/packages/e2b/fs-e2b/tests/filesystem.spec.ts @@ -275,7 +275,6 @@ async function setup(remote = new FakeRemote()): Promise<{ ctx: Context; fs: E2B const runtime = { cwd: '/workspace', runtimeRoot: '/workspace/.dsh-e2b', - disposeMode: 'kill', getSandbox: async () => remote.sandbox, } as unknown as E2BSandboxService ctx.provide('e2b', runtime) diff --git a/packages/e2b/subprocess-e2b/README.i18n.yaml b/packages/e2b/subprocess-e2b/README.i18n.yaml index 9ac46f1bcb..96fb24e21b 100644 --- a/packages/e2b/subprocess-e2b/README.i18n.yaml +++ b/packages/e2b/subprocess-e2b/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/e2b/subprocess-e2b/README.md -README.md: 9d563c91c149d28718159af1b1f8ddbd6652dc44 -README.zh.md: be189f41818c66aeff2bed7b1851b82ca08f08f4 +README.md: 4e47566d55cf9400459993185f03482761c92096 +README.zh.md: 21ef231af700f72c2c5ec1fc651297ebdd17d49b diff --git a/packages/e2b/subprocess-e2b/README.md b/packages/e2b/subprocess-e2b/README.md index 9d563c91c1..4e47566d55 100644 --- a/packages/e2b/subprocess-e2b/README.md +++ b/packages/e2b/subprocess-e2b/README.md @@ -9,12 +9,12 @@ E2B implementation of the [`@deepseek-ai/dsh-subprocess`](../../subprocess/subpr - **Asynchronous remote start** — the synchronous seam returns a handle immediately while `Sandbox.commands.run(..., { background: true })` starts remotely. `pid` is `-1` until the wrapper publishes and the adapter validates its process-group id; stdin and ordinary observation wait for that publication. An owned startup signal aborts environment and private-state preparation before allocation; once allocation begins, cancellation waits for a provisional SDK handle it can clean. - **Execution-world coordinates** — `cwd` and private `runtimeRoot` come from the shared owner; executable lookup verifies absolute paths or resolves a bare name against the sandbox PATH plus explicit overrides. - **Linux process groups** — a quoted wrapper starts each argv under `exec setsid --wait` and records its actual process-group id plus private status files beneath `ctx.e2b.runtimeRoot/processes`. The handle waits for that file instead of treating the SDK command PID as its published identity. Termination signals the negative recorded id with `SIGTERM`, waits the caller's `graceMs`, then escalates to `SIGKILL` and the SDK kill fallback; TERM delivery or probe failures also force that escalation. Process-table probes treat groups containing only zombie or dead entries as quiescent. Force cleanup succeeds only after a bounded probe finds the group empty; otherwise `waitForExit()` exposes a retryable failure, while proven quiescence makes later termination a no-op. Publication and monitoring failures apply the same cleanup transaction before rejecting. Service disposal rejects new starts, terminates and joins every retained process group, then awaits SDK settlement and private cleanup before the sandbox owner disposes. -- **Environment boundary** — one trusted control-shell probe resolves the sandbox user's login home from its passwd entry and transports the sandbox environment as base64 ASCII for one strict UTF-8 decode; the wrapper then removes ambient `DSH_*` and credential-shaped (`*KEY*`, `*SECRET*`, `*TOKEN*`) names and restores every valid `spec.env` entry as an explicit caller opt-in. Empty names, `=`, and NUL framing violations reject before launch. Subsequent E2B command and PTY login shells receive a fresh randomized root-level `HOME` plus empty overrides for every scrubbed ambient name before user profiles can run; the requested argv receives the serialized environment afterward without changing the template's umask. Host ambient variables never enter the sandbox implicitly. Private environment files are removed after consumption, and failed command or terminal setup removes its private state before rejecting. +- **Environment boundary** — one trusted control-shell probe resolves the sandbox user's login home from its passwd entry and transports the sandbox environment as base64 ASCII for one strict UTF-8 decode; the wrapper then removes ambient `DSH_*` and credential-shaped (`*KEY*`, `*SECRET*`, `*TOKEN*`) names and restores every valid `spec.env` entry as an explicit caller opt-in. Empty names, `=`, and NUL framing violations reject before launch. Subsequent E2B command and PTY login shells receive a fresh randomized root-level `HOME` plus empty overrides for every scrubbed ambient name before user profiles can run; the requested argv receives the serialized environment afterward without changing the sandbox user's umask. Host ambient variables never enter the sandbox implicitly. Private environment files are removed after consumption, and failed command or terminal setup removes its private state before rejecting. - **Stdio projection** — the remote wrapper branches raw bytes into optional bounded spill files, frames each live chunk as newline-delimited base64 ASCII, and the host incrementally restores bytes across arbitrary SDK callback boundaries. Pipe mode writes those bytes to host Node streams; inherit mode writes them to the harness process streams; collect mode retains a bounded host tail with offset reads. The wrapper publishes the direct command status before waiting for inherited writers. For collect or inherit output, the adapter disconnects an incomplete SDK stream after `graceMs`, withholds its partial spill, and returns that status while retaining the remote group for `waitForExit()` and termination. Natural raw-pipe completion instead awaits lossless transport and preserves backpressure; explicit termination destroys the host pipes and releases blocked output before remote cleanup. Batch and streaming stdin use the SDK handle. - **Terminal sessions** — `spawnTerminal()` uses E2B's byte PTY API, installs the exact argv and scrubbed environment through private mode-`0600` files, reports the foreground process group, sends real signals, and tears down every live group in the remote terminal session through one retryable awaited `terminate()`; termination rejects new handle operations, aborts and joins in-flight writes, inspections, and signals, and treats zombie-only groups as quiescent. A private random output boundary discards the E2B bootstrap shell's prompt and echoed runner command while preserving every requested-process byte, including its first prompt. PTY allocation is awaited through handle publication before cancellation is observed, so owned rollback can clean the published handle. Setup and teardown own the private state transaction, abort pending setup during service disposal, fence publication, and retain an unproven setup cleanup for disposal retry. Prompt detection, scrollback, readiness, and owner policy remain in `dsh-pty-local`. - **Sandbox disappearance** — `SandboxNotFoundError` during process or terminal liveness, termination, rollback, or disconnect proves the remote execution world cannot retain work, so cleanup treats it as quiescent; unrelated failures remain observable. -The base E2B image supplies the runtime and Bash/GNU utilities this adapter invokes: `node`, `bash`, `setsid`, `ps`, `awk`, `tr`, `env`, `base64`, `chmod`, `tee`, `head`, `rm`, `kill`, `id`, and `getent`. A custom template must retain compatible commands and E2B PTY support. +The default E2B base image supplies the runtime and Bash/GNU utilities this adapter invokes: `node`, `bash`, `setsid`, `ps`, `awk`, `tr`, `env`, `base64`, `chmod`, `tee`, `head`, `rm`, `kill`, `id`, and `getent`. ## Model Experience @@ -28,10 +28,9 @@ No direct invalidation; the named consumers own any request-prefix changes. - **The SDK still retains complete command output in host memory** — E2B `CommandHandle.stdout` and `.stderr` accumulate the base64 transport even when this adapter exposes bounded raw-byte tails, so the subprocess seam's normal host-memory bound is not achieved and transport retention is larger than the source stream. - **Synchronous-PID consumers are unsupported** — `pid` remains `-1` during remote startup; consumers that require a positive PID immediately, including the ACP child backend, cannot use this provider unchanged. -- **Reconnect does not reconstruct handles** — remote PID/status/spill files survive a retained sandbox, but a new harness process does not rebuild live `SubprocessHandle` objects or output cursors from them. -- **Remote state accumulates when retained** — process directories and valid spill files remain under `.dsh-e2b`; this POC supplies no retention sweep. +- **Private state lives for the sandbox lifetime** — process directories and valid spill files remain under `.dsh-e2b` until the owner deletes the sandbox; this POC supplies no in-sandbox sweep. - **Numeric process identities are not reuse-fenced** — E2B exposes numeric PID/PGID PTY input, signalling, and cleanup operations but no atomic identity-bound alternative. The adapter minimizes host round trips and live coverage exercises the reproducible stale-interrupt overlap; replacement is deferred until E2B adds an identity primitive or a failure demonstrates a narrower protocol. -- **The initial environment probe inherits sandbox defaults** — E2B merges command overrides with default environment entries, so the probe cannot blank unknown credential-shaped names before enumerating them. In a reconnected sandbox, a same-UID untrusted process could inspect that short-lived control shell; this POC therefore does not support secrets in sandbox-default environment variables and requires an E2B replacement-environment primitive or a hardened template to close the gap. +- **The initial environment probe inherits sandbox defaults** — E2B merges command overrides with default environment entries, so the probe cannot blank unknown credential-shaped names before enumerating them. A same-UID untrusted process already in the sandbox could inspect that short-lived control shell; this POC therefore does not support secrets in sandbox-default environment variables and requires an E2B replacement-environment primitive to close the gap. - **E2B exposes no signal fact** — an adapter-requested `SIGTERM` or `SIGKILL` is reported only when no wrapper-published direct exit code wins; every unrequested SDK exit remains an exit code, including values shaped like `128 + signal`. - **Exact terminal stdin-wait inspection is unavailable** — E2B exposes the foreground process group but not the syscall evidence needed to prove it is waiting on fd 0, so the generic PTY backend falls back to controlled prompt markers and bounded silence. -- **Linux utility and E2B transport semantics are assumed** — there is no Windows, arbitrary-template, escaped-session recovery, or network-partition fidelity layer. +- **Linux utility and E2B transport semantics are assumed** — there is no Windows, escaped-session recovery, or network-partition fidelity layer. diff --git a/packages/e2b/subprocess-e2b/README.zh.md b/packages/e2b/subprocess-e2b/README.zh.md index be189f4181..21ef231af7 100644 --- a/packages/e2b/subprocess-e2b/README.zh.md +++ b/packages/e2b/subprocess-e2b/README.zh.md @@ -9,12 +9,12 @@ - **异步远程启动**:同步 seam 会立即返回一个句柄,同时由 `Sandbox.commands.run(..., { background: true })` 在远程启动进程。包装层发布进程组 ID 并由适配器完成验证之前,`pid` 为 `-1`;stdin 和常规观察会等待该发布。自有启动信号会在分配前中止环境和私有状态准备;分配开始后,取消会等待可清理的临时 SDK 句柄。 - **执行世界坐标**:`cwd` 和私有 `runtimeRoot` 来自共享所有者;可执行文件查找会验证绝对路径,或根据沙箱 PATH 加显式覆盖来解析裸名称。 - **Linux 进程组**:带引号保护的包装层会在 `exec setsid --wait` 下启动每组 argv,并在 `ctx.e2b.runtimeRoot/processes` 下记录实际进程组 ID 和私有状态文件。句柄会等待该文件,而不会把 SDK 命令 PID 当作已发布的身份。终止操作以记录的负数 ID 发送 `SIGTERM`,等待调用方的 `graceMs`,再升级到 `SIGKILL` 和 SDK kill 回退;TERM 信号发送或探测失败也会强制触发该升级。进程表探测会把仅含僵尸或已死亡条目的进程组视为完全停稳。强制清理只有在有界探测发现进程组为空后才算成功;否则 `waitForExit()` 会公开可重试的失败,而已证明的完全停稳会让后续终止操作不再执行任何动作。发布失败与监控失败都会在拒绝前执行同一清理事务。服务 dispose(资源释放)会拒绝新的启动请求、终止并等待每个保留进程组退出,再等待 SDK 结算和私有清理完成,之后沙箱所有者才会释放。 -- **环境边界**:一次受信任的控制 shell 探测会从 passwd 条目解析沙箱用户的登录主目录,以 base64 ASCII 传输沙箱环境,再进行一次严格 UTF-8 解码;随后包装层移除环境中的 `DSH_*` 和形似凭据的名称(`*KEY*`、`*SECRET*`、`*TOKEN*`),并把每个有效的 `spec.env` 条目恢复为调用方显式选择。空名称、`=` 和违反 NUL 分帧规则的条目会在启动前被拒绝。在用户 profile 脚本运行前,此后的 E2B 命令 shell 与 PTY 登录 shell 会获得位于根目录下、全新随机生成的 `HOME`,并为每个被清理的环境变量名设置空值覆盖;之后,请求的 argv 会在不改变模板 umask 的前提下接收序列化环境。宿主环境变量绝不会隐式进入沙箱。私有环境文件在使用后会被删除;命令或终端设置失败时,会先删除其私有状态再拒绝。 +- **环境边界**:一次受信任的控制 shell 探测会从 passwd 条目解析沙箱用户的登录主目录,以 base64 ASCII 传输沙箱环境,再进行一次严格 UTF-8 解码;随后包装层移除环境中的 `DSH_*` 和形似凭据的名称(`*KEY*`、`*SECRET*`、`*TOKEN*`),并把每个有效的 `spec.env` 条目恢复为调用方显式选择。空名称、`=` 和违反 NUL 分帧规则的条目会在启动前被拒绝。在用户 profile 脚本运行前,此后的 E2B 命令 shell 与 PTY 登录 shell 会获得位于根目录下、全新随机生成的 `HOME`,并为每个被清理的环境变量名设置空值覆盖;之后,请求的 argv 会在不改变沙箱用户 umask 的前提下接收序列化环境。宿主环境变量绝不会隐式进入沙箱。私有环境文件在使用后会被删除;命令或终端设置失败时,会先删除其私有状态再拒绝。 - **stdio 投影**:远程包装层先把原始字节分流到可选的有界 spill 文件,再把每个实时分片编码为换行分隔的 base64 ASCII 帧;宿主会跨任意 SDK 回调边界增量恢复字节。pipe 模式把这些字节写入宿主 Node 流;inherit 模式把字节写入 harness 进程流;collect 模式保留有界的宿主尾部,并支持基于偏移量读取。包装层会在等待继承管道的写入方之前发布直接命令状态。对于 collect 或 inherit 输出,超过 `graceMs` 后,适配器会断开未完成的 SDK 流,不公开其中不完整的 spill,并返回该状态,同时保留远程进程组供 `waitForExit()` 和终止操作使用。原始 pipe 自然完成时,会等待无损传输完成并保留背压;显式终止则会销毁宿主 pipe,并在远程清理前释放受阻的输出写入。批量 stdin 和流式 stdin 都使用 SDK 句柄。 - **终端会话**:`spawnTerminal()` 使用 E2B 的字节 PTY API,以 mode 为 `0600` 的私有文件传入原样 argv 与清理后的环境,报告前台进程组,发送真实信号,并通过一项可重试且须等待的 `terminate()` 清理远程终端会话中仍存活的每个进程组;终止会拒绝新的句柄操作,中止并等待在途写入、检查和信号操作结算,并把仅含僵尸进程的进程组视为已经完全停稳。私有随机输出边界会丢弃 E2B 引导 shell 的提示符和回显的 runner 命令,同时保留请求进程的每个字节,包括其第一个提示符。PTY 分配会一直等待到句柄发布后才观察取消,以便由承担清理责任的回滚清理已发布句柄。setup 与 teardown 负责私有状态事务,在服务 dispose 期间中止待处理的 setup、阻止发布,并保留未证明已完成的 setup 清理事务,供 dispose 重试。提示符检测、scrollback、就绪状态与所有者策略仍归 `dsh-pty-local` 所有。 - **沙箱消失**:在进程或终端的存活探测、终止、回滚或断开连接期间出现 `SandboxNotFoundError`,证明远程执行环境无法保留工作,因此清理会将其视为完全停稳;其他故障仍可观察。 -基础 E2B 镜像提供该适配器调用的运行时和 Bash/GNU 工具:`node`、`bash`、`setsid`、`ps`、`awk`、`tr`、`env`、`base64`、`chmod`、`tee`、`head`、`rm`、`kill`、`id` 和 `getent`。自定义模板必须保留兼容的命令和 E2B PTY 支持。 +E2B 默认基础镜像提供该适配器调用的运行时和 Bash/GNU 工具:`node`、`bash`、`setsid`、`ps`、`awk`、`tr`、`env`、`base64`、`chmod`、`tee`、`head`、`rm`、`kill`、`id` 和 `getent`。 ## 模型体验 @@ -28,10 +28,9 @@ - **SDK 仍会在宿主内存中保留完整命令输出**:即使本适配器公开的是有界原始字节尾部,E2B `CommandHandle.stdout` 和 `.stderr` 仍会累积 base64 传输内容,因此无法达到进程管理 seam 通常提供的宿主内存边界,而且传输保留量大于源数据流。 - **不支持需要同步 PID 的消费方**:远程启动期间,`pid` 保持为 `-1`;包括 ACP 子进程后端在内,要求立即获得正 PID 的消费方无法原样使用本提供方。 -- **重新连接不会重建句柄**:保留沙箱后,远程 PID/状态/spill 文件仍然存在,但新的 harness 进程不会据此重建实时 `SubprocessHandle` 对象或输出游标。 -- **保留沙箱时会累积远程状态**:进程目录和有效的 spill 文件会留在 `.dsh-e2b` 下;本 POC 不提供保留清理。 +- **私有状态随沙箱生命周期存在**:进程目录和有效的 spill 文件会留在 `.dsh-e2b` 下,直到所有者删除沙箱;本 POC 不提供沙箱内清理。 - **数值进程身份没有复用围栏**:E2B 公开基于数值 PID/PGID 的 PTY 输入、信号发送和清理操作,却没有与身份原子绑定的替代方案。适配器会尽量减少宿主往返,真实环境测试会覆盖可复现的陈旧中断重叠;在 E2B 新增身份原语,或实际故障证明需要更窄的协议之前,替代方案会继续延后。 -- **初始环境探测会继承沙箱默认值**:E2B 会把命令覆盖与默认环境条目合并,因此探测无法在枚举未知且形似凭据的名称之前将它们置空。在重新连接的沙箱中,一个同 UID 的不可信进程可以检查该短时存在的控制 shell;因此,该 POC 不支持把 secret 放入沙箱默认环境变量,需要 E2B 的替换环境原语或经加固的模板才能弥合该缺口。 +- **初始环境探测会继承沙箱默认值**:E2B 会把命令覆盖与默认环境条目合并,因此探测无法在枚举未知且形似凭据的名称之前将它们置空。一个已在沙箱内运行的同 UID 不可信进程可以检查该短时存在的控制 shell;因此,该 POC 不支持把 secret 放入沙箱默认环境变量,需要 E2B 的替换环境原语才能弥合该缺口。 - **E2B 不公开信号事实**:适配器请求的 `SIGTERM` 或 `SIGKILL` 只有在包装层发布的直接退出码没有胜出时才报告为信号;其他未请求的 SDK 退出始终保留为退出码,包括形似 `128 + signal` 的值。 - **无法精确检查终端 stdin 等待状态**:E2B 会公开前台进程组,但不提供证明其正在等待 fd 0 所需的 syscall 证据,因此通用 PTY 后端会回退到受控提示符标记与有界静默机制。 -- **依赖 Linux 工具与 E2B 传输语义**:没有 Windows、任意模板、逃逸会话恢复或网络分区的保真层。 +- **依赖 Linux 工具与 E2B 传输语义**:没有 Windows、逃逸会话恢复或网络分区的保真层。 diff --git a/packages/e2b/subprocess-e2b/src/index.ts b/packages/e2b/subprocess-e2b/src/index.ts index 990ab20a62..63cafcbfe0 100644 --- a/packages/e2b/subprocess-e2b/src/index.ts +++ b/packages/e2b/subprocess-e2b/src/index.ts @@ -106,7 +106,7 @@ export class E2BSubprocessService extends SubprocessService { /** @inheritdoc */ spawn(spec: SubprocessSpawnSpec): SubprocessHandle { - if (this.isDisposing()) throw new Error('subprocess-e2b: service is disposing') + if (this.disposing) throw new Error('subprocess-e2b: service is disposing') const program = spec.argv[0] if (program === undefined || program.length === 0) { throw new Error('invalid argv: expected a non-empty program name at argv[0]') @@ -115,7 +115,7 @@ export class E2BSubprocessService extends SubprocessService { throw new Error('subprocess-e2b: graceMs must be a positive finite number') } if (spec.signal?.aborted === true) { - throw new Error(`aborted before spawn: ${String(spec.signal.reason ?? 'aborted')}`) + throw new Error(`aborted before spawn: ${String(spec.signal.reason)}`) } const stateDir = posix.join(this.ctx.e2b.runtimeRoot, 'processes', randomUUID()) const handle = new E2BSubprocessHandle(this.ctx.e2b, spec, stateDir) @@ -132,7 +132,7 @@ export class E2BSubprocessService extends SubprocessService { /** @inheritdoc */ async spawnTerminal(spec: SubprocessTerminalSpawnSpec): Promise { - if (this.isDisposing()) throw new Error('subprocess-e2b: service is disposing') + if (this.disposing) throw new Error('subprocess-e2b: service is disposing') const program = spec.argv[0] if (program === undefined || program.length === 0) { throw new Error('subprocess-e2b: terminal argv must contain a program') @@ -158,7 +158,8 @@ export class E2BSubprocessService extends SubprocessService { (cleanup) => { this.failedTerminalSetupCleanups.add(cleanup) }, ) this.terminals.add(terminal) - if (this.isDisposing()) { + // oxlint-disable-next-line typescript/no-unnecessary-condition -- Remote allocation yields to disposal. + if (this.disposing) { await terminal.terminate() this.terminals.delete(terminal) throw new Error('subprocess-e2b: service disposed during terminal setup') @@ -176,10 +177,6 @@ export class E2BSubprocessService extends SubprocessService { setup.resolve() } } - - private isDisposing(): boolean { - return this.disposing - } } export default E2BSubprocessService diff --git a/packages/e2b/subprocess-e2b/src/output.ts b/packages/e2b/subprocess-e2b/src/output.ts index 8afcd138dc..ba1dc1a716 100644 --- a/packages/e2b/subprocess-e2b/src/output.ts +++ b/packages/e2b/subprocess-e2b/src/output.ts @@ -115,9 +115,6 @@ export class E2BOutputReader implements SubprocessOutputReader { /** @inheritdoc */ readFrom(fromByte: number): SubprocessOutputRead { - if (!Number.isSafeInteger(fromByte) || fromByte < 0) { - throw new Error('subprocess output offset must be a non-negative safe integer') - } const retained = Buffer.concat(this.chunks, this.retainedBytes) const firstRetained = this.totalBytes - this.retainedBytes const lossy = fromByte < firstRetained diff --git a/packages/e2b/subprocess-e2b/src/process.ts b/packages/e2b/subprocess-e2b/src/process.ts index 5e383a50e2..fe964a7395 100644 --- a/packages/e2b/subprocess-e2b/src/process.ts +++ b/packages/e2b/subprocess-e2b/src/process.ts @@ -149,10 +149,6 @@ function commandOpts( return { envs: e2bControlEnvs(envs), ...(signal === undefined ? {} : { signal }) } } -function isAborted(signal: AbortSignal | undefined): boolean { - return signal?.aborted === true -} - function waitTick(signal?: AbortSignal): Promise { if (signal?.aborted === true) return Promise.resolve(false) return new Promise((resolve) => { @@ -173,7 +169,7 @@ const WAIT_ABORTED = Symbol('wait aborted') function waitWithSignal(promise: Promise, signal: AbortSignal | undefined): Promise { if (signal === undefined) return promise if (signal.aborted) return Promise.resolve(WAIT_ABORTED) - return new Promise((resolve, reject) => { + return new Promise((resolve) => { const onAbort = (): void => { cleanup(); resolve(WAIT_ABORTED) } const cleanup = (): void => { signal.removeEventListener('abort', onAbort) } signal.addEventListener('abort', onAbort, { once: true }) @@ -181,10 +177,7 @@ function waitWithSignal(promise: Promise, signal: AbortSignal | undefined) onAbort() return } - void promise.then( - (value) => { cleanup(); resolve(value) }, - (error: unknown) => { cleanup(); reject(asError(error)) }, - ) + void promise.then((value) => { cleanup(); resolve(value) }) }) } @@ -206,12 +199,9 @@ export class E2BSubprocessHandle implements SubprocessHandle { private readonly paths: RemotePaths private controlEnvs: Record = {} private remotePid = -1 - private commandHandle: CommandHandle | undefined private outputTransportError: Error | undefined private outputDrainExpired = false private stateDirectoryCreated = false - private preparing = true - private terminationStarted = false private quiescenceProven = false private terminationAttempt: Promise | undefined private terminationFailure: Error | undefined @@ -265,7 +255,6 @@ export class E2BSubprocessHandle implements SubprocessHandle { /** @inheritdoc */ terminate(): void { if (this.quiescenceProven || this.terminationAttempt !== undefined) return - this.terminationStarted = true this.terminationController.abort(new Error('subprocess-e2b: command terminated')) this.stdout?.destroy() this.stderr?.destroy() @@ -285,7 +274,7 @@ export class E2BSubprocessHandle implements SubprocessHandle { async waitForExit(signal?: AbortSignal): Promise { if (this.quiescenceProven) return true let handle: CommandHandle | undefined - if (this.terminationStarted) { + if (this.terminationController.signal.aborted) { const observed = await waitWithSignal(this.commandState.promise, signal) if (observed === WAIT_ABORTED) return false handle = observed @@ -295,22 +284,23 @@ export class E2BSubprocessHandle implements SubprocessHandle { } if (this.remotePid <= 0) { const attempt = this.terminationAttempt - if (attempt !== undefined && await waitWithSignal(attempt, signal) === WAIT_ABORTED) return false + if (attempt !== undefined && await waitWithSignal(attempt.catch(() => undefined), signal) === WAIT_ABORTED) { + return false + } this.throwTerminationFailure() // Successful pre-publication termination records quiescence; its only other outcome is the failure above. return true } } else { - try { - const observed = await waitWithSignal(this.readyState.promise, signal) - if (observed === WAIT_ABORTED) return false - handle = observed - } catch { - handle = this.commandHandle - if (handle === undefined) { - this.markQuiescent() - return true - } + const observed = await waitWithSignal( + this.readyState.promise.catch(() => this.commandState.promise), + signal, + ) + if (observed === WAIT_ABORTED) return false + handle = observed + if (handle === undefined) { + this.markQuiescent() + return true } } this.throwTerminationFailure() @@ -318,7 +308,7 @@ export class E2BSubprocessHandle implements SubprocessHandle { try { sandbox = await this.runtime.getSandbox() } catch (error: unknown) { - if (isAborted(signal)) return false + if (signal?.aborted === true) return false if (error instanceof SandboxNotFoundError) { this.markQuiescent() return true @@ -331,7 +321,7 @@ export class E2BSubprocessHandle implements SubprocessHandle { if (!await waitTick(signal)) return false } this.throwTerminationFailure() - if (isAborted(signal)) return false + if (signal?.aborted === true) return false this.markQuiescent() return true } @@ -345,10 +335,11 @@ export class E2BSubprocessHandle implements SubprocessHandle { private async run(): Promise { let sandbox: Sandbox | undefined + let preparing = true try { sandbox = await this.runtime.getSandbox() await this.prepareState(sandbox) - this.preparing = false + preparing = false const handle = await sandbox.commands.run( commandText(this.spec, this.paths), { @@ -361,7 +352,6 @@ export class E2BSubprocessHandle implements SubprocessHandle { onStderr: async (data) => { await this.dispatchOutput('stderr', data) }, }, ) - this.commandHandle = handle const completion = handle.wait() void completion.catch(() => {}) if (!isValidProcessId(handle.pid)) { @@ -369,7 +359,6 @@ export class E2BSubprocessHandle implements SubprocessHandle { try { await handle.kill() this.markQuiescent() - this.commandHandle = undefined } catch (cleanupError: unknown) { this.terminationFailure = asError(cleanupError) this.commandState.resolve(handle) @@ -404,9 +393,7 @@ export class E2BSubprocessHandle implements SubprocessHandle { await this.finalizeSpills(sandbox) return outcome } catch (error: unknown) { - const canceledPreparation = this.preparing - && this.terminationStarted - && this.terminationController.signal.aborted + const canceledPreparation = preparing && this.terminationController.signal.aborted let failure = await this.rollbackPublishedFailure(error) if (sandbox !== undefined && this.stateDirectoryCreated) { try { @@ -423,7 +410,6 @@ export class E2BSubprocessHandle implements SubprocessHandle { if (canceledPreparation && failure === error) return { exitCode: null, signal: 'SIGTERM' } throw failure } finally { - this.preparing = false this.spec.signal?.removeEventListener('abort', this.onAbort) this.stdout?.end() this.stderr?.end() @@ -583,7 +569,7 @@ export class E2BSubprocessHandle implements SubprocessHandle { } private async rollbackPublishedFailure(error: unknown): Promise { - if (this.remotePid <= 0 || this.commandHandle === undefined || this.quiescenceProven) return error + if (this.remotePid <= 0 || this.quiescenceProven) return error this.terminate() try { await this.waitForExit() @@ -626,7 +612,6 @@ export class E2BSubprocessHandle implements SubprocessHandle { if (!isValidProcessId(handle.pid) && this.remotePid <= 0) { await handle.kill() this.markQuiescent() - this.commandHandle = undefined return } const sandbox = await this.runtime.getSandbox() @@ -730,7 +715,7 @@ export class E2BSubprocessHandle implements SubprocessHandle { const size = (reader as E2BOutputReader).size if (this.outputDrainExpired || size <= mode.maxBytes || size > mode.spill.maxBytes) { removals.push(sandbox.files.remove(path).catch((_adapterPrivateSpillRemovalFailure: unknown) => { - // The command outcome is authoritative; a retained sandbox tolerates private residue. + // The command outcome is authoritative; owner teardown bounds private residue. })) } } diff --git a/packages/e2b/subprocess-e2b/src/terminal.ts b/packages/e2b/subprocess-e2b/src/terminal.ts index 8d003dfc35..3876191759 100644 --- a/packages/e2b/subprocess-e2b/src/terminal.ts +++ b/packages/e2b/subprocess-e2b/src/terminal.ts @@ -41,7 +41,6 @@ const TERMINAL_RUNNER_SOURCE = [ ' exit 125', 'fi', 'printf \'%s\' "$dsh_output_marker"', - "printf 'ready\\n' > \"$dsh_state/ready\"", 'exec env -i -- "${dsh_env[@]}" "${dsh_argv[@]}"', '', ].join('\n') @@ -51,7 +50,6 @@ interface TerminalPaths { environment: string argv: string outputMarker: string - ready: string } function signalOpts(signal: AbortSignal | undefined): { signal?: AbortSignal } { @@ -165,26 +163,6 @@ async function terminalSessionId( return parsePositiveId(result.stdout, `subprocess-e2b: cannot resolve process session for terminal ${pid}`) } -async function waitUntilReady( - sandbox: Sandbox, - paths: TerminalPaths, - completion: Promise, - signal?: AbortSignal, -): Promise { - const settled = completion.then(() => true, () => true) - for (;;) { - signal?.throwIfAborted() - try { - if ((await sandbox.files.read(paths.ready, signalOpts(signal))).trim() === 'ready') return - } catch (error: unknown) { - if (!(error instanceof FileNotFoundError)) throw error - } - if (await Promise.race([settled, delay(POLL_MS).then(() => false)])) { - throw new Error('subprocess-e2b: terminal exited before publishing readiness') - } - } -} - async function sessionProcessGroups( sandbox: Sandbox, sessionId: number, @@ -241,8 +219,13 @@ async function awaitSessionEmpty( const deadline = Date.now() + graceMs for (;;) { const groups = await sessionProcessGroups(sandbox, sessionId, envs) - if (groups.length === 0 || Date.now() >= deadline) return groups - if (kill) await signalGroups(sandbox, groups, 'KILL', envs) + if (groups.length === 0) return groups + if (kill) { + await signalGroups(sandbox, groups, 'KILL', envs) + if (Date.now() >= deadline) return await sessionProcessGroups(sandbox, sessionId, envs) + } else if (Date.now() >= deadline) { + return groups + } await delay(Math.min(POLL_MS, Math.max(1, deadline - Date.now()))) } } @@ -277,7 +260,6 @@ async function rollbackUnpublishedTerminal( groups = await awaitSessionEmpty(sandbox, sessionId, envs, graceMs) } if (groups.length > 0) { - await signalGroups(sandbox, groups, 'KILL', envs) await awaitSessionEmpty(sandbox, sessionId, envs, graceMs, true) } } catch (error: unknown) { @@ -285,25 +267,13 @@ async function rollbackUnpublishedTerminal( } } // Completion can settle while any awaited provider cleanup above is running. - // eslint-disable-next-line @typescript-eslint/no-unnecessary-condition + // oxlint-disable-next-line typescript/no-unnecessary-condition -- Provider cleanup yields to completion. if (!topLevelExited) { - if (validPid) { - try { - await sandbox.pty.kill(handle.pid) - } catch (error: unknown) { - if (error instanceof SandboxNotFoundError) return - attemptFailures.push(asError(error)) - } - } - // The awaited PTY fallback can settle completion before the SDK fallback. - // eslint-disable-next-line @typescript-eslint/no-unnecessary-condition - if (!topLevelExited) { - try { - await handle.kill() - } catch (error: unknown) { - if (error instanceof SandboxNotFoundError) return - attemptFailures.push(asError(error)) - } + try { + await handle.kill() + } catch (error: unknown) { + if (error instanceof SandboxNotFoundError) return + attemptFailures.push(asError(error)) } await Promise.race([completion.catch(() => undefined), delay(graceMs)]) } @@ -321,7 +291,7 @@ async function rollbackUnpublishedTerminal( } } // The bounded completion race above updates this callback-owned state. - // eslint-disable-next-line @typescript-eslint/no-unnecessary-condition + // oxlint-disable-next-line typescript/no-unnecessary-condition -- The callback mutates this after a race. if (!topLevelExited) { proofFailures.push(new Error(`subprocess-e2b: terminal setup rollback failed; surviving pid: ${handle.pid}`)) } @@ -347,7 +317,6 @@ export class E2BTerminalHandle implements SubprocessTerminalHandle { private cleanup: Promise | undefined private readonly operationController = new AbortController() private readonly operations = new Set>() - private terminating = false private terminationSignal: NodeJS.Signals | null = null constructor( @@ -400,7 +369,6 @@ export class E2BTerminalHandle implements SubprocessTerminalHandle { /** @inheritdoc */ terminate(): Promise { if (this.cleanup !== undefined) return this.cleanup - this.terminating = true this.operationController.abort(new Error('subprocess-e2b: terminal is terminating')) const cleanup = this.closeAfterOperations() this.cleanup = cleanup @@ -434,7 +402,9 @@ export class E2BTerminalHandle implements SubprocessTerminalHandle { } private trackOperation(operation: (signal: AbortSignal) => Promise): Promise { - if (this.terminating) return Promise.reject(new Error('subprocess-e2b: terminal is terminating')) + if (this.operationController.signal.aborted) { + return Promise.reject(new Error('subprocess-e2b: terminal is terminating')) + } const pending = operation(this.operationController.signal) this.operations.add(pending) void pending.then( @@ -445,7 +415,7 @@ export class E2BTerminalHandle implements SubprocessTerminalHandle { } private async closeAfterOperations(): Promise { - if (this.operations.size > 0) await Promise.allSettled(this.operations) + await Promise.allSettled(this.operations) await this.closeOnce() } @@ -481,7 +451,7 @@ export class E2BTerminalHandle implements SubprocessTerminalHandle { this.terminationSignal = 'SIGKILL' if (!this.topLevelExited) { try { - await this.sandbox.pty.kill(this.pid) + await this.handle.kill() } catch (error: unknown) { if (error instanceof SandboxNotFoundError) return throw error @@ -504,7 +474,7 @@ export class E2BTerminalHandle implements SubprocessTerminalHandle { try { await this.sandbox.files.remove(this.stateDir) } catch (_adapterPrivateStateRemovalFailure) { - // The terminal is quiescent; a retained sandbox tolerates private residue. + // The terminal is quiescent; owner teardown bounds private residue. } } } @@ -531,7 +501,6 @@ export async function spawnE2BTerminal( environment: posix.join(stateDir, 'environment'), argv: posix.join(stateDir, 'argv'), outputMarker: posix.join(stateDir, 'output-marker'), - ready: posix.join(stateDir, 'ready'), } const outputMarker = Buffer.from(`dsh-e2b-bootstrap:${randomUUID()}`) const output = new PassThrough() @@ -577,7 +546,6 @@ export async function spawnE2BTerminal( } const command = `exec /bin/bash ${quoteE2BShellArg(paths.runner)} ${quoteE2BShellArg(stateDir)}\r` await sandbox.pty.sendInput(handle.pid, Buffer.from(command), signalOpts(spec.signal)) - await waitUntilReady(sandbox, paths, completion, spec.signal) await waitForBootstrapOutput(outputFilter.ready, completion, spec.signal) const sessionId = await terminalSessionId(sandbox, handle.pid, controlEnvs, spec.signal) return new E2BTerminalHandle( diff --git a/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts b/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts index 83e86892d4..f7adf8183c 100644 --- a/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts +++ b/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts @@ -316,7 +316,6 @@ function runtime(fake: FakeSandbox, getSandbox: () => Promise = async ( return { cwd: '/workspace', runtimeRoot: '/workspace/.dsh-e2b', - disposeMode: 'kill', getSandbox, } as unknown as E2BSandboxService } @@ -372,8 +371,6 @@ describe('E2BOutputReader', () => { const overCap = new E2BOutputReader(2, 3, '/too-small') overCap.push(Buffer.from('abcd')) expect(overCap.readFrom(0)).toEqual({ text: 'cd', nextOffset: 4, lossy: true }) - expect(() => overCap.readFrom(-1)).toThrow(/non-negative safe integer/) - expect(() => overCap.readFrom(1.5)).toThrow(/non-negative safe integer/) }) }) @@ -1673,7 +1670,6 @@ describe('E2BSubprocessService', () => { expect(() => ctx.subprocess.spawn(spec({ argv: [] }))).toThrow(/non-empty program/) expect(() => ctx.subprocess.spawn(spec({ graceMs: 0 }))).toThrow(/positive finite/) expect(() => ctx.subprocess.spawn(spec({ signal: AbortSignal.abort('stop') }))).toThrow(/aborted before spawn/) - expect(() => ctx.subprocess.spawn(spec({ signal: { aborted: true, reason: undefined } as AbortSignal }))).toThrow(/aborted$/) }) it('registers the package-owned empty invariant installer', async () => { diff --git a/packages/e2b/subprocess-e2b/tests/terminal.spec.ts b/packages/e2b/subprocess-e2b/tests/terminal.spec.ts index a4f4008a56..08d49bb87b 100644 --- a/packages/e2b/subprocess-e2b/tests/terminal.spec.ts +++ b/packages/e2b/subprocess-e2b/tests/terminal.spec.ts @@ -90,9 +90,6 @@ class FakeTerminalSandbox { readonly writes = new Map() createOptions: Parameters[0] | undefined ambient = 'KEEP=visible\0UNICODE=你好\0NPM_TOKEN=secret\0DSH_STALE=old\0BROKEN\0=bad\0' - ready: string | Error = 'ready\n' - readyMisses = 0 - readyReads = 0 sessionId = '123\n' foreground = '456\n' groups = [123] @@ -108,12 +105,9 @@ class FakeTerminalSandbox { sessionGroupsFailure: unknown foregroundFailure: unknown termFailure: unknown - ptyKillError: unknown removeError: unknown clearOnTerm = true clearOnKill = true - settleOnPtyKill = true - ptyKills = 0 resolvedExecutable = '/usr/bin/node\n' requestedOutput = 'requested-shell$ ' emitOutputMarker = true @@ -144,15 +138,6 @@ class FakeTerminalSandbox { if (this.writeError !== undefined) throw this.writeError return files.map(() => ({})) }, - read: async (): Promise => { - this.readyReads += 1 - if (this.readyMisses > 0) { - this.readyMisses -= 1 - throw new FileNotFoundError('not ready') - } - if (this.ready instanceof Error) throw this.ready - return this.ready - }, remove: async (path: string): Promise => { this.removed.push(path) if (this.removeError !== undefined) throw this.removeError @@ -237,12 +222,6 @@ class FakeTerminalSandbox { } } }, - kill: async (pid: number): Promise => { - this.ptyKills += 1 - if (this.ptyKillError !== undefined) throw this.ptyKillError - if (this.settleOnPtyKill) this.handle.fail(137) - return pid === this.handle.pid - }, }, } as unknown as Sandbox } @@ -251,7 +230,6 @@ function runtime(fake: FakeTerminalSandbox): E2BSandboxService { return { cwd: '/workspace', runtimeRoot: '/workspace/.dsh-e2b', - disposeMode: 'kill', getSandbox: async () => fake.sandbox, } as unknown as E2BSandboxService } @@ -284,7 +262,6 @@ function holdRequestUntilAbort(started: PromiseWithResolvers) { describe('E2B terminal allocation', () => { it('hides bootstrap-shell bytes and preserves requested-shell bytes across the output boundary', async () => { const fake = new FakeTerminalSandbox() - fake.readyMisses = 1 const terminal = await spawnE2BTerminal(runtime(fake), spec(), '/runtime/terminal-one') let output = '' terminal.output.on('data', (chunk) => { output += String(chunk) }) @@ -417,12 +394,6 @@ describe('E2B terminal allocation', () => { expect(failedInput.commands).toContain('kill -TERM -- -123') expect(failedInput.groups).toEqual([]) - const exited = new FakeTerminalSandbox() - exited.ready = new FileNotFoundError('not ready') - queueMicrotask(() => { exited.handle.succeed(0) }) - await expect(spawnE2BTerminal(runtime(exited), spec(), '/runtime/exited')) - .rejects.toThrow('exited before publishing readiness') - const invalidSession = new FakeTerminalSandbox() invalidSession.sessionId = 'not-a-session\n' invalidSession.clearOnTerm = false @@ -431,7 +402,7 @@ describe('E2B terminal allocation', () => { expect(invalidSession.commands).toContain('kill -TERM -- -123') expect(invalidSession.commands).toContain('kill -KILL -- -123') expect(invalidSession.groups).toEqual([]) - expect(invalidSession.ptyKills).toBe(1) + expect(invalidSession.handle.sdkKills).toBe(1) const lateData = invalidSession.createOptions?.onData if (lateData === undefined) throw new Error('missing captured terminal callback') expect(lateData(Buffer.from('late bytes'))).toBeUndefined() @@ -442,12 +413,12 @@ describe('E2B terminal allocation', () => { await expect(spawnE2BTerminal(runtime(termFailed), spec(), '/runtime/term-failed')) .rejects.toThrow('bootstrap failed') expect(termFailed.commands).toContain('kill -KILL -- -123') - expect(termFailed.ptyKills).toBe(1) + expect(termFailed.handle.sdkKills).toBe(1) const uninspectable = new FakeTerminalSandbox() uninspectable.sendError = new Error('bootstrap failed') uninspectable.sessionGroupsFailure = 'session enumeration failed' - uninspectable.ptyKillError = new Error('PTY kill failed') + uninspectable.handle.sdkKillError = new Error('PTY kill failed') let uninspectableFailure: unknown try { await spawnE2BTerminal(runtime(uninspectable), spec(), '/runtime/uninspectable') @@ -455,7 +426,6 @@ describe('E2B terminal allocation', () => { uninspectableFailure = error } expect(uninspectableFailure).toBeInstanceOf(AggregateError) - expect(uninspectable.ptyKills).toBe(1) expect(uninspectable.handle.sdkKills).toBe(1) const survivingGroups = new FakeTerminalSandbox() @@ -468,7 +438,6 @@ describe('E2B terminal allocation', () => { const survivingPid = new FakeTerminalSandbox() survivingPid.sendError = new Error('bootstrap failed') survivingPid.groups = [] - survivingPid.settleOnPtyKill = false survivingPid.handle.settleOnSdkKill = false await expect(spawnE2BTerminal(runtime(survivingPid), spec({ graceMs: 1 }), '/runtime/surviving-pid')) .rejects.toThrow('bootstrap failed') @@ -491,12 +460,12 @@ describe('E2B terminal allocation', () => { const expiredDuringRollback = new FakeTerminalSandbox() expiredDuringRollback.sendError = new Error('bootstrap failed before timeout') expiredDuringRollback.groups = [] - expiredDuringRollback.settleOnPtyKill = false - expiredDuringRollback.ptyKillError = new SandboxNotFoundError('sandbox expired') + expiredDuringRollback.handle.settleOnSdkKill = false + expiredDuringRollback.handle.sdkKillError = new SandboxNotFoundError('sandbox expired') expiredDuringRollback.removeError = new SandboxNotFoundError('sandbox expired') await expect(spawnE2BTerminal(runtime(expiredDuringRollback), spec(), '/runtime/expired-rollback')) .rejects.toThrow('bootstrap failed before timeout') - expect(expiredDuringRollback.ptyKills).toBe(1) + expect(expiredDuringRollback.handle.sdkKills).toBe(1) const expiredBeforeSdkRollback = new FakeTerminalSandbox() expiredBeforeSdkRollback.handle.waitError = new Error('wait failed after timeout') @@ -505,15 +474,6 @@ describe('E2B terminal allocation', () => { await expect(spawnE2BTerminal(runtime(expiredBeforeSdkRollback), spec(), '/runtime/expired-sdk-rollback')) .rejects.toThrow('wait failed after timeout') - const expiredDuringSdkFallback = new FakeTerminalSandbox() - expiredDuringSdkFallback.sendError = new Error('bootstrap failed before SDK fallback') - expiredDuringSdkFallback.groups = [] - expiredDuringSdkFallback.settleOnPtyKill = false - expiredDuringSdkFallback.handle.sdkKillError = new SandboxNotFoundError('sandbox expired') - expiredDuringSdkFallback.handle.settleOnSdkKill = false - await expect(spawnE2BTerminal(runtime(expiredDuringSdkFallback), spec(), '/runtime/expired-sdk-fallback')) - .rejects.toThrow('bootstrap failed before SDK fallback') - const missingDuringDisconnect = new FakeTerminalSandbox() missingDuringDisconnect.sendError = new Error('bootstrap failed before disconnect') missingDuringDisconnect.handle.disconnectError = new SandboxNotFoundError('sandbox expired') @@ -537,10 +497,6 @@ describe('E2B terminal allocation', () => { await expect(spawnE2BTerminal(runtime(createFailed), spec(), '/runtime/create')) .rejects.toThrow('create failed') - const readFailed = new FakeTerminalSandbox() - readFailed.ready = new Error('ready transport failed') - await expect(spawnE2BTerminal(runtime(readFailed), spec(), '/runtime/read')) - .rejects.toThrow('ready transport failed') }) it('bounds a missing bootstrap-output boundary by process exit or cancellation', async () => { @@ -560,7 +516,6 @@ describe('E2B terminal allocation', () => { '/runtime/cancel-output-boundary', ) await vi.waitFor(() => { expect(cancelled.inputs).toHaveLength(1) }) - await vi.waitFor(() => { expect(cancelled.readyReads).toBeGreaterThan(0) }) await new Promise(resolve => setTimeout(resolve, 0)) controller.abort(new Error('cancel output boundary')) await expect(cancelling).rejects.toThrow('cancel output boundary') @@ -661,23 +616,23 @@ describe('E2B terminal lifecycle', () => { it('treats sandbox disappearance during PTY kill as quiescent', async () => { const fake = new FakeTerminalSandbox() fake.groups = [] - fake.settleOnPtyKill = false - fake.ptyKillError = new SandboxNotFoundError('sandbox expired') + fake.handle.settleOnSdkKill = false + fake.handle.sdkKillError = new SandboxNotFoundError('sandbox expired') const terminal = await spawnE2BTerminal(runtime(fake), spec({ graceMs: 1 }), '/runtime/expired-pty-kill') await terminal.terminate() - expect(fake.ptyKills).toBe(1) + expect(fake.handle.sdkKills).toBe(1) }) it('propagates a non-missing PTY kill failure', async () => { const fake = new FakeTerminalSandbox() fake.groups = [] - fake.settleOnPtyKill = false - fake.ptyKillError = new Error('PTY kill transport failed') + fake.handle.settleOnSdkKill = false + fake.handle.sdkKillError = new Error('PTY kill transport failed') const terminal = await spawnE2BTerminal(runtime(fake), spec({ graceMs: 1 }), '/runtime/failed-pty-kill') await expect(terminal.terminate()).rejects.toThrow('PTY kill transport failed') - fake.ptyKillError = undefined + fake.handle.sdkKillError = undefined fake.handle.succeed(0) await terminal.done await terminal.terminate() @@ -712,11 +667,11 @@ describe('E2B terminal lifecycle', () => { await terminal.terminate() }) - it('escalates surviving process groups', async () => { + it('sends KILL before checking an expired force-cleanup deadline', async () => { const fake = new FakeTerminalSandbox() fake.groups = [123, 456] fake.clearOnTerm = false - const terminal = await spawnE2BTerminal(runtime(fake), spec({ graceMs: 1 }), '/runtime/escalate') + const terminal = await spawnE2BTerminal(runtime(fake), spec({ graceMs: 0 }), '/runtime/escalate') const terminating = terminal.terminate() await expect(terminal.done).resolves.toEqual({ exitCode: null, signal: 'SIGKILL' }) await terminating @@ -771,11 +726,11 @@ describe('E2B terminal lifecycle', () => { it('keeps a late command rejection authoritative after PTY kill', async () => { const fake = new FakeTerminalSandbox() fake.groups = [] - fake.settleOnPtyKill = false + fake.handle.settleOnSdkKill = false const terminal = await spawnE2BTerminal(runtime(fake), spec({ graceMs: 1 }), '/runtime/reject-after-kill') terminal.output.on('error', () => {}) const cleanup = terminal.terminate() - while (fake.ptyKills === 0) await new Promise(resolve => setTimeout(resolve, 0)) + while (fake.handle.sdkKills === 0) await new Promise(resolve => setTimeout(resolve, 0)) await Promise.resolve() fake.handle.crash(new Error('late command transport failed')) await expect(terminal.done).rejects.toThrow('late command transport failed') @@ -791,7 +746,7 @@ describe('E2B terminal lifecycle', () => { const livePid = new FakeTerminalSandbox() livePid.groups = [] - livePid.settleOnPtyKill = false + livePid.handle.settleOnSdkKill = false const live = await spawnE2BTerminal(runtime(livePid), spec({ graceMs: 1 }), '/runtime/live-pid') await expect(live.terminate()).rejects.toThrow('surviving pid: 123') livePid.handle.succeed(0) @@ -879,13 +834,13 @@ describe('E2B subprocess terminal service', () => { expect(fake.removed.some(path => path.includes('/terminals/'))).toBe(true) }) - it('aborts and rolls back terminal setup that cannot publish readiness during disposal', async () => { + it('aborts and rolls back terminal setup that cannot publish its output boundary during disposal', async () => { const fake = new FakeTerminalSandbox() - fake.ready = new FileNotFoundError('not ready') + fake.emitOutputMarker = false const { ctx, fiber } = await service(fake) const spawning = ctx.subprocess.spawnTerminal(spec()) const rejected = expect(spawning).rejects.toThrow('service disposed during terminal setup') - await vi.waitFor(() => { expect(fake.readyReads).toBeGreaterThan(0) }) + await vi.waitFor(() => { expect(fake.inputs).toHaveLength(1) }) await fiber.dispose() await rejected diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 87648e01e1..f682bc6847 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -716,6 +716,9 @@ importers: packages/bash/bash: devDependencies: + '@deepseek-ai/dsh-brand': + specifier: workspace:^ + version: link:../../util/brand '@deepseek-ai/dsh-invariants': specifier: workspace:^ version: link:../../support/invariants @@ -3091,9 +3094,6 @@ importers: packages/credentials/credentials: devDependencies: - '@deepseek-ai/dsh-brand': - specifier: workspace:^ - version: link:../../util/brand '@deepseek-ai/dsh-invariants': specifier: workspace:^ version: link:../../support/invariants From b1b2d63785a8b3a5341d2420401f7576ce12ff50 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Thu, 30 Jul 2026 06:02:10 +0800 Subject: [PATCH 096/108] refactor(e2b): drop speculative cleanup retry state --- AGENTS.md | 6 ++-- docs/cordis-catalog/services.md | 2 +- packages/e2b/e2b/README.i18n.yaml | 4 +-- packages/e2b/e2b/README.md | 2 +- packages/e2b/e2b/README.zh.md | 2 +- packages/e2b/e2b/src/index.ts | 23 +++++-------- packages/e2b/e2b/tests/e2b.spec.ts | 20 ++--------- packages/e2b/subprocess-e2b/README.i18n.yaml | 4 +-- packages/e2b/subprocess-e2b/README.md | 2 +- packages/e2b/subprocess-e2b/README.zh.md | 2 +- packages/e2b/subprocess-e2b/src/index.ts | 6 ---- packages/e2b/subprocess-e2b/src/process.ts | 33 +++++-------------- packages/e2b/subprocess-e2b/src/terminal.ts | 9 +++-- .../subprocess-e2b/tests/subprocess.spec.ts | 27 ++++++++------- .../e2b/subprocess-e2b/tests/terminal.spec.ts | 17 ---------- 15 files changed, 48 insertions(+), 111 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 1d2a6d977c..87f545ee86 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -14,12 +14,12 @@ packages/ @deepseek-ai/dsh- workspaces at packages/// core/ product API spine: session, system-prompt, tools, agent, agent-loop typert/ type graph generator, loader, and runtime registry llm/ LLM seam + DeepSeek adapters (direct-fetch + pi-ai design twin) - e2b/ E2B POC: sandbox owner + FS/subprocess adapters + e2b/ E2B POC: sandbox + FS/subprocess adapters bash/ bash executor seam + local impl + model-facing bash tools subprocess/ subprocess seam + local process-tree impl pty/ persistent PTY seam/backend/tools - fs/ filesystem seam + local impl + policy gate + read/write/edit tools - lsp/ language-server seam + local stdio provider + model-facing lsp tool + fs/ filesystem seam/backends/policy/tools + lsp/ language-server seam/local backend/tool skill/ skill provider registry + local impl + catalog/loader tool web/ web seam + search/fetch providers + model-facing web tools compact/ compaction seam + basic backend diff --git a/docs/cordis-catalog/services.md b/docs/cordis-catalog/services.md index db66ca5799..8af0e6ca30 100644 --- a/docs/cordis-catalog/services.md +++ b/docs/cordis-catalog/services.md @@ -591,7 +591,7 @@ Creates one lazily consumable E2B SDK handle and deletes the sandbox at timeout async getSandbox(): Promise ``` -Source: [`packages/e2b/e2b/src/index.ts:76`](../../packages/e2b/e2b/src/index.ts) +Source: [`packages/e2b/e2b/src/index.ts:74`](../../packages/e2b/e2b/src/index.ts) ## `ctx.fs` — `FileSystem` (abstract seam) diff --git a/packages/e2b/e2b/README.i18n.yaml b/packages/e2b/e2b/README.i18n.yaml index c338fb0263..3587067d31 100644 --- a/packages/e2b/e2b/README.i18n.yaml +++ b/packages/e2b/e2b/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/e2b/e2b/README.md -README.md: 6045b313a91edafc25361eba3f3c4dacf56b4eb0 -README.zh.md: dc157d9b5167488530c7d63b59aed793ab6aa4c7 +README.md: 7ade7c3d6522d8fa6d54d7011766238451b17c9a +README.zh.md: b683f33d2211106cf422e780b2b37904b0c640ad diff --git a/packages/e2b/e2b/README.md b/packages/e2b/e2b/README.md index 6045b313a9..7ade7c3d65 100644 --- a/packages/e2b/e2b/README.md +++ b/packages/e2b/e2b/README.md @@ -26,7 +26,7 @@ Shared lifecycle owner for one E2B sandbox. The filesystem and subprocess adapte Construction starts one sandbox creation. Before resolving `getSandbox()`, the service creates `cwd` and the private `cwd/.dsh-e2b` adapter-state directory, verifies that the reserved path is a real directory rather than a symlink or another file type, then sets it to mode `0700`. Each adapter-internal E2B command shell receives a fresh randomized root-level `HOME`, so the SDK's fixed login shell does not resolve profile files from the mutable user home before the control command. -Disposal first prevents new handle acquisition, then awaits setup and deletes the sandbox. A `SandboxNotFoundError` means expiry or another owner already deleted it and is accepted as quiescence. Initial directory setup failure also deletes the newly created sandbox; if that rollback fails, disposal retries it before releasing ownership. Provider plugins must load after this owner and dispose before it. +Disposal first prevents new handle acquisition, then awaits setup and deletes the sandbox. A `SandboxNotFoundError` means expiry or another owner already deleted it and is accepted as quiescence. Initial directory setup failure makes one deletion attempt; the configured E2B timeout bounds a second failure. Provider plugins must load after this owner and dispose before it. ## Model Experience diff --git a/packages/e2b/e2b/README.zh.md b/packages/e2b/e2b/README.zh.md index dc157d9b51..b683f33d22 100644 --- a/packages/e2b/e2b/README.zh.md +++ b/packages/e2b/e2b/README.zh.md @@ -26,7 +26,7 @@ 构造阶段会启动一次沙箱创建。服务在 `getSandbox()` 结算前创建 `cwd` 和私有的 `cwd/.dsh-e2b` 适配器状态目录,验证该预留路径是真实目录而非符号链接或其他文件类型,再把该目录的 mode 设为 `0700`。每个适配器内部的 E2B 命令 shell 都会获得一个位于根目录下、全新随机生成的 `HOME`,因此 SDK 固定使用的登录 shell 不会在控制命令之前解析可变用户主目录中的配置文件。 -资源释放会先阻止继续获取新句柄,再等待初始化完成,然后删除沙箱。`SandboxNotFoundError` 表示沙箱已因超时或被另一个所有者删除,因此可视为完全停稳。初始目录设置失败时也会删除新建沙箱;如果该回滚失败,资源释放会在解除所有权前重试。提供方插件必须在该所有者之后加载,并在其之前 dispose(资源释放)。 +资源释放会先阻止继续获取新句柄,再等待初始化完成,然后删除沙箱。`SandboxNotFoundError` 表示沙箱已因超时或被另一个所有者删除,因此可视为完全停稳。初始目录设置失败时会尝试删除一次;若该尝试也失败,则由已配置的 E2B 超时约束沙箱的存活时间。提供方插件必须在该所有者之后加载,并在其之前 dispose(资源释放)。 ## 模型体验 diff --git a/packages/e2b/e2b/src/index.ts b/packages/e2b/e2b/src/index.ts index d6797b9147..906941d421 100644 --- a/packages/e2b/e2b/src/index.ts +++ b/packages/e2b/e2b/src/index.ts @@ -85,7 +85,6 @@ export class E2BSandboxService extends Service { private readonly config: ResolvedConfig private readonly ready: Promise - private failedSetupSandbox: Sandbox | undefined private disposed = false constructor(ctx: Context, config: Config) { @@ -108,21 +107,18 @@ export class E2BSandboxService extends Service { ctx.effect(() => async () => { this.disposed = true - let sandbox = this.failedSetupSandbox - if (sandbox === undefined) { - try { - sandbox = await this.ready - } catch { - sandbox = this.failedSetupSandbox - } + let sandbox: Sandbox + try { + sandbox = await this.ready + } catch (_sandboxSetupFailure) { + // open() either acquired no sandbox or already made the POC's one rollback attempt. + return } - if (sandbox === undefined) return try { await sandbox.kill() } catch (error: unknown) { if (!(error instanceof SandboxNotFoundError)) throw error } - this.failedSetupSandbox = undefined }, 'e2b sandbox teardown') } @@ -174,10 +170,9 @@ export class E2BSandboxService extends Service { } catch (error: unknown) { try { await sandbox.kill() - } catch (_cleanupFailure) { - // Preserve the setup failure as the public error while retaining the - // created handle for the service disposer to retry this rollback. - this.failedSetupSandbox = sandbox + } catch (_sandboxSetupRollbackFailure) { + // TODO(e2b-setup-rollback): Add retry state only if a real double failure + // outlives E2B's configured sandbox timeout. } throw error } diff --git a/packages/e2b/e2b/tests/e2b.spec.ts b/packages/e2b/e2b/tests/e2b.spec.ts index e3a6c09039..b108bc68b0 100644 --- a/packages/e2b/e2b/tests/e2b.spec.ts +++ b/packages/e2b/e2b/tests/e2b.spec.ts @@ -180,7 +180,7 @@ describe('E2BSandboxService', () => { await fiber.dispose() }) - it('preserves the setup failure even when cleanup also fails', async () => { + it('preserves the setup failure after its one rollback attempt fails', async () => { const fixture = fakeSandbox() fixture.run.mockRejectedValueOnce(new Error('chmod failed')) fixture.kill.mockRejectedValueOnce(new Error('cleanup failed')) @@ -191,23 +191,7 @@ describe('E2BSandboxService', () => { expect(fixture.kill).toHaveBeenCalledOnce() await fiber.dispose() - expect(fixture.kill).toHaveBeenCalledTimes(2) - }) - - it.each([ - ['retries a still-failing rollback', new Error('retry failed')], - ['accepts a setup sandbox that expired before retry', new SandboxNotFoundError('sandbox expired')], - ])('%s during disposal', async (_label, retryError) => { - const fixture = fakeSandbox() - fixture.run.mockRejectedValueOnce(new Error('chmod failed')) - fixture.kill.mockRejectedValueOnce(new Error('cleanup failed')).mockRejectedValueOnce(retryError) - sdk.create.mockResolvedValue(fixture.sandbox) - const ctx = new Context() - const fiber = await ctx.plugin(E2BSandboxService, { apiKey: 'test-key' }) - - await expect(ctx.e2b.getSandbox()).rejects.toThrow('chmod failed') - await fiber.dispose() - expect(fixture.kill).toHaveBeenCalledTimes(2) + expect(fixture.kill).toHaveBeenCalledOnce() }) it.each([ diff --git a/packages/e2b/subprocess-e2b/README.i18n.yaml b/packages/e2b/subprocess-e2b/README.i18n.yaml index 96fb24e21b..381ada5198 100644 --- a/packages/e2b/subprocess-e2b/README.i18n.yaml +++ b/packages/e2b/subprocess-e2b/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/e2b/subprocess-e2b/README.md -README.md: 4e47566d55cf9400459993185f03482761c92096 -README.zh.md: 21ef231af700f72c2c5ec1fc651297ebdd17d49b +README.md: 3ddfe5bc47309ba56d3f8af7720fcbe71ec8a302 +README.zh.md: c4db7bf8bb8a0f4b2f147dd22868978be5d1fcf4 diff --git a/packages/e2b/subprocess-e2b/README.md b/packages/e2b/subprocess-e2b/README.md index 4e47566d55..3ddfe5bc47 100644 --- a/packages/e2b/subprocess-e2b/README.md +++ b/packages/e2b/subprocess-e2b/README.md @@ -11,7 +11,7 @@ E2B implementation of the [`@deepseek-ai/dsh-subprocess`](../../subprocess/subpr - **Linux process groups** — a quoted wrapper starts each argv under `exec setsid --wait` and records its actual process-group id plus private status files beneath `ctx.e2b.runtimeRoot/processes`. The handle waits for that file instead of treating the SDK command PID as its published identity. Termination signals the negative recorded id with `SIGTERM`, waits the caller's `graceMs`, then escalates to `SIGKILL` and the SDK kill fallback; TERM delivery or probe failures also force that escalation. Process-table probes treat groups containing only zombie or dead entries as quiescent. Force cleanup succeeds only after a bounded probe finds the group empty; otherwise `waitForExit()` exposes a retryable failure, while proven quiescence makes later termination a no-op. Publication and monitoring failures apply the same cleanup transaction before rejecting. Service disposal rejects new starts, terminates and joins every retained process group, then awaits SDK settlement and private cleanup before the sandbox owner disposes. - **Environment boundary** — one trusted control-shell probe resolves the sandbox user's login home from its passwd entry and transports the sandbox environment as base64 ASCII for one strict UTF-8 decode; the wrapper then removes ambient `DSH_*` and credential-shaped (`*KEY*`, `*SECRET*`, `*TOKEN*`) names and restores every valid `spec.env` entry as an explicit caller opt-in. Empty names, `=`, and NUL framing violations reject before launch. Subsequent E2B command and PTY login shells receive a fresh randomized root-level `HOME` plus empty overrides for every scrubbed ambient name before user profiles can run; the requested argv receives the serialized environment afterward without changing the sandbox user's umask. Host ambient variables never enter the sandbox implicitly. Private environment files are removed after consumption, and failed command or terminal setup removes its private state before rejecting. - **Stdio projection** — the remote wrapper branches raw bytes into optional bounded spill files, frames each live chunk as newline-delimited base64 ASCII, and the host incrementally restores bytes across arbitrary SDK callback boundaries. Pipe mode writes those bytes to host Node streams; inherit mode writes them to the harness process streams; collect mode retains a bounded host tail with offset reads. The wrapper publishes the direct command status before waiting for inherited writers. For collect or inherit output, the adapter disconnects an incomplete SDK stream after `graceMs`, withholds its partial spill, and returns that status while retaining the remote group for `waitForExit()` and termination. Natural raw-pipe completion instead awaits lossless transport and preserves backpressure; explicit termination destroys the host pipes and releases blocked output before remote cleanup. Batch and streaming stdin use the SDK handle. -- **Terminal sessions** — `spawnTerminal()` uses E2B's byte PTY API, installs the exact argv and scrubbed environment through private mode-`0600` files, reports the foreground process group, sends real signals, and tears down every live group in the remote terminal session through one retryable awaited `terminate()`; termination rejects new handle operations, aborts and joins in-flight writes, inspections, and signals, and treats zombie-only groups as quiescent. A private random output boundary discards the E2B bootstrap shell's prompt and echoed runner command while preserving every requested-process byte, including its first prompt. PTY allocation is awaited through handle publication before cancellation is observed, so owned rollback can clean the published handle. Setup and teardown own the private state transaction, abort pending setup during service disposal, fence publication, and retain an unproven setup cleanup for disposal retry. Prompt detection, scrollback, readiness, and owner policy remain in `dsh-pty-local`. +- **Terminal sessions** — `spawnTerminal()` uses E2B's byte PTY API, installs the exact argv and scrubbed environment through private mode-`0600` files, reports the foreground process group, sends real signals, and tears down every live group in the remote terminal session through one retryable awaited `terminate()`; termination rejects new handle operations, aborts and joins in-flight writes, inspections, and signals, and treats zombie-only groups as quiescent. A private random output boundary discards the E2B bootstrap shell's prompt and echoed runner command while preserving every requested-process byte, including its first prompt. PTY allocation is awaited through handle publication before cancellation is observed, so owned rollback can clean the published handle. Setup and teardown own the private state transaction, abort pending setup during service disposal, and fence publication; sandbox disposal or timeout bounds a setup rollback that also fails. Prompt detection, scrollback, readiness, and owner policy remain in `dsh-pty-local`. - **Sandbox disappearance** — `SandboxNotFoundError` during process or terminal liveness, termination, rollback, or disconnect proves the remote execution world cannot retain work, so cleanup treats it as quiescent; unrelated failures remain observable. The default E2B base image supplies the runtime and Bash/GNU utilities this adapter invokes: `node`, `bash`, `setsid`, `ps`, `awk`, `tr`, `env`, `base64`, `chmod`, `tee`, `head`, `rm`, `kill`, `id`, and `getent`. diff --git a/packages/e2b/subprocess-e2b/README.zh.md b/packages/e2b/subprocess-e2b/README.zh.md index 21ef231af7..c4db7bf8bb 100644 --- a/packages/e2b/subprocess-e2b/README.zh.md +++ b/packages/e2b/subprocess-e2b/README.zh.md @@ -11,7 +11,7 @@ - **Linux 进程组**:带引号保护的包装层会在 `exec setsid --wait` 下启动每组 argv,并在 `ctx.e2b.runtimeRoot/processes` 下记录实际进程组 ID 和私有状态文件。句柄会等待该文件,而不会把 SDK 命令 PID 当作已发布的身份。终止操作以记录的负数 ID 发送 `SIGTERM`,等待调用方的 `graceMs`,再升级到 `SIGKILL` 和 SDK kill 回退;TERM 信号发送或探测失败也会强制触发该升级。进程表探测会把仅含僵尸或已死亡条目的进程组视为完全停稳。强制清理只有在有界探测发现进程组为空后才算成功;否则 `waitForExit()` 会公开可重试的失败,而已证明的完全停稳会让后续终止操作不再执行任何动作。发布失败与监控失败都会在拒绝前执行同一清理事务。服务 dispose(资源释放)会拒绝新的启动请求、终止并等待每个保留进程组退出,再等待 SDK 结算和私有清理完成,之后沙箱所有者才会释放。 - **环境边界**:一次受信任的控制 shell 探测会从 passwd 条目解析沙箱用户的登录主目录,以 base64 ASCII 传输沙箱环境,再进行一次严格 UTF-8 解码;随后包装层移除环境中的 `DSH_*` 和形似凭据的名称(`*KEY*`、`*SECRET*`、`*TOKEN*`),并把每个有效的 `spec.env` 条目恢复为调用方显式选择。空名称、`=` 和违反 NUL 分帧规则的条目会在启动前被拒绝。在用户 profile 脚本运行前,此后的 E2B 命令 shell 与 PTY 登录 shell 会获得位于根目录下、全新随机生成的 `HOME`,并为每个被清理的环境变量名设置空值覆盖;之后,请求的 argv 会在不改变沙箱用户 umask 的前提下接收序列化环境。宿主环境变量绝不会隐式进入沙箱。私有环境文件在使用后会被删除;命令或终端设置失败时,会先删除其私有状态再拒绝。 - **stdio 投影**:远程包装层先把原始字节分流到可选的有界 spill 文件,再把每个实时分片编码为换行分隔的 base64 ASCII 帧;宿主会跨任意 SDK 回调边界增量恢复字节。pipe 模式把这些字节写入宿主 Node 流;inherit 模式把字节写入 harness 进程流;collect 模式保留有界的宿主尾部,并支持基于偏移量读取。包装层会在等待继承管道的写入方之前发布直接命令状态。对于 collect 或 inherit 输出,超过 `graceMs` 后,适配器会断开未完成的 SDK 流,不公开其中不完整的 spill,并返回该状态,同时保留远程进程组供 `waitForExit()` 和终止操作使用。原始 pipe 自然完成时,会等待无损传输完成并保留背压;显式终止则会销毁宿主 pipe,并在远程清理前释放受阻的输出写入。批量 stdin 和流式 stdin 都使用 SDK 句柄。 -- **终端会话**:`spawnTerminal()` 使用 E2B 的字节 PTY API,以 mode 为 `0600` 的私有文件传入原样 argv 与清理后的环境,报告前台进程组,发送真实信号,并通过一项可重试且须等待的 `terminate()` 清理远程终端会话中仍存活的每个进程组;终止会拒绝新的句柄操作,中止并等待在途写入、检查和信号操作结算,并把仅含僵尸进程的进程组视为已经完全停稳。私有随机输出边界会丢弃 E2B 引导 shell 的提示符和回显的 runner 命令,同时保留请求进程的每个字节,包括其第一个提示符。PTY 分配会一直等待到句柄发布后才观察取消,以便由承担清理责任的回滚清理已发布句柄。setup 与 teardown 负责私有状态事务,在服务 dispose 期间中止待处理的 setup、阻止发布,并保留未证明已完成的 setup 清理事务,供 dispose 重试。提示符检测、scrollback、就绪状态与所有者策略仍归 `dsh-pty-local` 所有。 +- **终端会话**:`spawnTerminal()` 使用 E2B 的字节 PTY API,以 mode 为 `0600` 的私有文件传入原样 argv 与清理后的环境,报告前台进程组,发送真实信号,并通过一项可重试且须等待的 `terminate()` 清理远程终端会话中仍存活的每个进程组;终止会拒绝新的句柄操作,中止并等待在途写入、检查和信号操作结算,并把仅含僵尸进程的进程组视为已经完全停稳。私有随机输出边界会丢弃 E2B 引导 shell 的提示符和回显的 runner 命令,同时保留请求进程的每个字节,包括其第一个提示符。PTY 分配会一直等待到句柄发布后才观察取消,以便由承担清理责任的回滚清理已发布句柄。setup 与 teardown 负责私有状态事务,在服务 dispose 期间中止待处理的 setup 并阻止发布;若 setup 回滚也失败,则由沙箱 dispose 或超时约束其存活时间。提示符检测、scrollback、就绪状态与所有者策略仍归 `dsh-pty-local` 所有。 - **沙箱消失**:在进程或终端的存活探测、终止、回滚或断开连接期间出现 `SandboxNotFoundError`,证明远程执行环境无法保留工作,因此清理会将其视为完全停稳;其他故障仍可观察。 E2B 默认基础镜像提供该适配器调用的运行时和 Bash/GNU 工具:`node`、`bash`、`setsid`、`ps`、`awk`、`tr`、`env`、`base64`、`chmod`、`tee`、`head`、`rm`、`kill`、`id` 和 `getent`。 diff --git a/packages/e2b/subprocess-e2b/src/index.ts b/packages/e2b/subprocess-e2b/src/index.ts index 63cafcbfe0..976880ab01 100644 --- a/packages/e2b/subprocess-e2b/src/index.ts +++ b/packages/e2b/subprocess-e2b/src/index.ts @@ -29,7 +29,6 @@ export class E2BSubprocessService extends SubprocessService { private readonly live = new Set() private readonly terminals = new Set() private readonly terminalSetups = new Map, AbortController>() - private readonly failedTerminalSetupCleanups = new Set<() => Promise>() private disposing = false /** @inheritdoc */ @@ -51,7 +50,6 @@ export class E2BSubprocessService extends SubprocessService { await Promise.all([...this.terminalSetups.keys()]) const handles = [...this.live] const terminals = [...this.terminals] - const failedTerminalSetupCleanups = [...this.failedTerminalSetupCleanups] const pending: Promise[] = [] for (const handle of handles) { handle.terminate() @@ -63,9 +61,6 @@ export class E2BSubprocessService extends SubprocessService { for (const terminal of terminals) { pending.push(terminal.terminate().then(() => { this.terminals.delete(terminal) })) } - for (const cleanup of failedTerminalSetupCleanups) { - pending.push(cleanup().then(() => { this.failedTerminalSetupCleanups.delete(cleanup) })) - } const outcomes = await Promise.allSettled(pending) for (const outcome of outcomes) { if (outcome.status === 'rejected') throw outcome.reason @@ -155,7 +150,6 @@ export class E2BSubprocessService extends SubprocessService { this.ctx.e2b, { ...spec, signal: setupSignal }, stateDir, - (cleanup) => { this.failedTerminalSetupCleanups.add(cleanup) }, ) this.terminals.add(terminal) // oxlint-disable-next-line typescript/no-unnecessary-condition -- Remote allocation yields to disposal. diff --git a/packages/e2b/subprocess-e2b/src/process.ts b/packages/e2b/subprocess-e2b/src/process.ts index fe964a7395..c0aceda334 100644 --- a/packages/e2b/subprocess-e2b/src/process.ts +++ b/packages/e2b/subprocess-e2b/src/process.ts @@ -636,35 +636,18 @@ export class E2BSubprocessHandle implements SubprocessHandle { } private async forceKillGroup(sandbox: Sandbox, handle: CommandHandle, processGroupId: number): Promise { - let groupFailure: unknown try { - if (!await this.signalGroup(sandbox, processGroupId, 'KILL')) { - groupFailure = new Error('process-group KILL did not report delivery') - } - } catch (error: unknown) { - groupFailure = error + await this.signalGroup(sandbox, processGroupId, 'KILL') + } catch (_processGroupKillFailure) { + // SDK kill and the final liveness probe remain independent cleanup paths. } - let handleFailure: unknown try { - if (!await handle.kill()) handleFailure = new Error('E2B SDK kill did not report command termination') - } catch (error: unknown) { - handleFailure = error + await handle.kill() + } catch (_sdkKillFailure) { + // The final liveness probe, not either transport's self-report, proves cleanup. } - let proofFailure: unknown - try { - if (await this.waitForGroupExit(sandbox, processGroupId)) return - proofFailure = new Error(`remote process group ${processGroupId} remained live after force termination`) - } catch (error: unknown) { - proofFailure = error - } - throw new AggregateError( - [ - ...(groupFailure === undefined ? [] : [groupFailure]), - ...(handleFailure === undefined ? [] : [handleFailure]), - proofFailure, - ], - 'subprocess-e2b: force termination failed through both process-group and SDK transports', - ) + if (await this.waitForGroupExit(sandbox, processGroupId)) return + throw new Error(`subprocess-e2b: remote process group ${processGroupId} remained live after force termination`) } private async waitForGroupExit(sandbox: Sandbox, processGroupId: number): Promise { diff --git a/packages/e2b/subprocess-e2b/src/terminal.ts b/packages/e2b/subprocess-e2b/src/terminal.ts index 3876191759..3108a4a60a 100644 --- a/packages/e2b/subprocess-e2b/src/terminal.ts +++ b/packages/e2b/subprocess-e2b/src/terminal.ts @@ -485,14 +485,12 @@ export class E2BTerminalHandle implements SubprocessTerminalHandle { * @param runtime - Shared E2B sandbox owner. * @param spec - Fully specified terminal-process request. * @param stateDir - Private remote directory for one startup transaction. - * @param retainFailedCleanup - Optional owner for retrying a cleanup transaction that could not prove quiescence. * @returns The live subprocess terminal handle. */ export async function spawnE2BTerminal( runtime: E2BSandboxService, spec: SubprocessTerminalSpawnSpec, stateDir: string, - retainFailedCleanup?: (cleanup: () => Promise) => void, ): Promise { const sandbox = await runtime.getSandbox() spec.signal?.throwIfAborted() @@ -562,7 +560,7 @@ export async function spawnE2BTerminal( output.destroy() let terminalQuiescent = handle === undefined let stateRemoved = !stateDirectoryCreated - const retryCleanup = async (): Promise => { + const cleanup = async (): Promise => { const failures: Error[] = [] if (!terminalQuiescent && handle !== undefined) { try { @@ -588,9 +586,10 @@ export async function spawnE2BTerminal( } } try { - await retryCleanup() + await cleanup() } catch (cleanupError: unknown) { - retainFailedCleanup?.(retryCleanup) + // TODO(e2b-terminal-setup-rollback): Retain retry state only if a real + // double failure must be recovered before sandbox disposal or timeout. throw new AggregateError([asError(error), asError(cleanupError)], asError(error).message) } throw error diff --git a/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts b/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts index f7adf8183c..471716ef1a 100644 --- a/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts +++ b/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts @@ -808,7 +808,7 @@ describe('E2BSubprocessHandle', () => { await flush() handle.terminate() await vi.waitFor(() => { expect(fake.handle.kills).toBe(1) }) - await expect(handle.waitForExit()).rejects.toThrow('force termination failed through both') + await expect(handle.waitForExit()).rejects.toThrow('remained live after force termination') expect(fake.alive).toBe(true) fake.delaysKill = false @@ -884,7 +884,7 @@ describe('E2BSubprocessHandle', () => { handle.terminate() await vi.waitFor(() => { expect(fake.handle.kills).toBe(1) }) - await expect(handle.waitForExit()).rejects.toThrow('force termination failed through both') + await expect(handle.waitForExit()).rejects.toThrow('remained live after force termination') fake.alive = false handle.terminate() await expect(handle.waitForExit()).resolves.toBe(true) @@ -941,7 +941,7 @@ describe('E2BSubprocessHandle', () => { await vi.waitFor(() => { expect(fake.startOptions).toBeDefined() }) handle.terminate() - await expect(handle.waitForExit()).rejects.toThrow('force termination failed through both') + await expect(handle.waitForExit()).rejects.toThrow('remained live after force termination') fake.handle.killError = undefined handle.terminate() @@ -960,7 +960,7 @@ describe('E2BSubprocessHandle', () => { ) await vi.waitFor(() => { expect(absentGroup.startOptions).toBeDefined() }) absentHandle.terminate() - await expect(absentHandle.waitForExit()).rejects.toThrow('force termination failed through both') + await expect(absentHandle.waitForExit()).rejects.toThrow('remained live after force termination') absentGroup.handle.killError = undefined absentHandle.terminate() await expect(absentHandle.waitForExit()).resolves.toBe(true) @@ -978,7 +978,7 @@ describe('E2BSubprocessHandle', () => { ) await vi.waitFor(() => { expect(optimisticSdk.startOptions).toBeDefined() }) optimisticHandle.terminate() - await expect(optimisticHandle.waitForExit()).rejects.toThrow('force termination failed through both') + await expect(optimisticHandle.waitForExit()).rejects.toThrow('remained live after force termination') optimisticHandle.terminate() await expect(optimisticHandle.waitForExit()).resolves.toBe(true) optimisticSdk.releaseProcessGroupRead() @@ -1322,7 +1322,7 @@ describe('E2BSubprocessHandle', () => { await expect(absent.waitForExit()).resolves.toBe(true) }) - it('preserves publication and rollback failures when cleanup cannot be verified', async () => { + it('preserves publication failure and reports cleanup that cannot be verified', async () => { const fake = new FakeSandbox() fake.processGroupId = 'not-a-pid\n' fake.signalError = new Error('rollback signal failed') @@ -1341,11 +1341,10 @@ describe('E2BSubprocessHandle', () => { const failures = Array.from(failure.errors as Iterable) expect(failures).toHaveLength(2) expect(failures[0]).toBeInstanceOf(Error) - expect(failures[1]).toBeInstanceOf(AggregateError) - if (!(failures[0] instanceof Error) || !(failures[1] instanceof AggregateError)) throw new Error('expected nested errors') + expect(failures[1]).toBeInstanceOf(Error) + if (!(failures[0] instanceof Error) || !(failures[1] instanceof Error)) throw new Error('expected nested errors') expect(failures[0].message).toContain('invalid process-group id') - expect(failures[1].message).toBe('subprocess-e2b: force termination failed through both process-group and SDK transports') - expect(Array.from(failures[1].errors as Iterable)).toContainEqual(new Error('rollback signal failed')) + expect(failures[1].message).toContain('remained live after force termination') expect(fake.handle.kills).toBe(1) const bounded = new AbortController() const waiting = handle.waitForExit(bounded.signal) @@ -1494,7 +1493,7 @@ describe('E2BSubprocessHandle', () => { await flush() handle.terminate() - await expect(handle.waitForExit()).rejects.toThrow('force termination failed through both') + await expect(handle.waitForExit()).rejects.toThrow('remained live after force termination') fake.handle.killError = undefined handle.terminate() await expect(handle.done).resolves.toEqual({ exitCode: null, signal: 'SIGTERM' }) @@ -1507,7 +1506,7 @@ describe('E2BSubprocessHandle', () => { const raced = new E2BSubprocessHandle(runtime(missingGroup), spec({ graceMs: 1 }), '/runtime/group-exit-race') await flush() raced.terminate() - await expect(raced.waitForExit()).rejects.toThrow('force termination failed through both') + await expect(raced.waitForExit()).rejects.toThrow('remained live after force termination') missingGroup.handle.killError = undefined raced.terminate() await expect(raced.waitForExit()).resolves.toBe(true) @@ -1522,7 +1521,7 @@ describe('E2BSubprocessHandle', () => { await flush() handle.terminate() - await expect(handle.waitForExit()).rejects.toThrow('force termination failed through both') + await expect(handle.waitForExit()).rejects.toThrow('remained live after force termination') expect(fake.alive).toBe(true) fake.sdkKillStops = true @@ -1581,7 +1580,7 @@ describe('E2BSubprocessService', () => { await flush() await expect(fiber.dispose()).resolves.toBeUndefined() - await expect(handle.waitForExit()).rejects.toThrow('force termination failed through both') + await expect(handle.waitForExit()).rejects.toThrow('remained live after force termination') fake.handle.killError = undefined handle.terminate() diff --git a/packages/e2b/subprocess-e2b/tests/terminal.spec.ts b/packages/e2b/subprocess-e2b/tests/terminal.spec.ts index 08d49bb87b..067910a10b 100644 --- a/packages/e2b/subprocess-e2b/tests/terminal.spec.ts +++ b/packages/e2b/subprocess-e2b/tests/terminal.spec.ts @@ -871,23 +871,6 @@ describe('E2B subprocess terminal service', () => { expect(fake.createOptions).toBeUndefined() }) - it('retains failed terminal setup cleanup for disposal retry', async () => { - const fake = new FakeTerminalSandbox() - fake.sendError = new Error('bootstrap failed') - fake.clearOnTerm = false - fake.clearOnKill = false - const { ctx, fiber } = await service(fake) - - await expect(ctx.subprocess.spawnTerminal(spec({ graceMs: 1 }))).rejects.toThrow('bootstrap failed') - expect(fake.groups).toEqual([123]) - expect(fake.handle.disconnects).toBe(0) - - fake.clearOnKill = true - await fiber.dispose() - expect(fake.groups).toEqual([]) - expect(fake.handle.disconnects).toBe(1) - }) - it('releases naturally settled terminals and validates terminal requests', async () => { const { ctx, fiber, fake } = await service() for (const request of [ From 258bd4456b21bb037583d696b351a2fbb6ceb6e9 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Thu, 30 Jul 2026 06:02:22 +0800 Subject: [PATCH 097/108] test(e2b): poll sandbox list convergence --- packages/e2b/e2b/tests/composition.e2e.ts | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/packages/e2b/e2b/tests/composition.e2e.ts b/packages/e2b/e2b/tests/composition.e2e.ts index 2eca4f1f63..bf53ba7bb9 100644 --- a/packages/e2b/e2b/tests/composition.e2e.ts +++ b/packages/e2b/e2b/tests/composition.e2e.ts @@ -178,6 +178,9 @@ describe.skipIf(!process.env.E2B_API_KEY)('E2B live Loader composition', () => { const apiKey = process.env.E2B_API_KEY if (apiKey === undefined) throw new Error('E2B_API_KEY disappeared during the live composition test') await expect(Sandbox.getInfo(String(output.sandboxId), { apiKey })).rejects.toBeInstanceOf(SandboxNotFoundError) - await expect(Sandbox.list({ apiKey }).nextItems()).resolves.toEqual([]) + await expect.poll(async () => { + const sandboxes = await Sandbox.list({ apiKey }).nextItems() + return sandboxes.some(sandbox => sandbox.sandboxId === output.sandboxId) + }, { interval: 250, timeout: 5_000 }).toBe(false) }, 195_000) }) From 26b18ba189715c8a4ff40cb785c34daf83957e53 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Thu, 30 Jul 2026 14:50:47 +0800 Subject: [PATCH 098/108] refactor(e2b): keep code runtime host-local --- ...rtable-execution-world-consumers.i18n.yaml | 4 +- ...7-28-portable-execution-world-consumers.md | 6 +-- ...8-portable-execution-world-consumers.zh.md | 6 +-- examples/headless-agent/e2b.cordis.yml | 7 +-- .../tests/fixtures/e2b/e2b/bin.ts | 51 ------------------- .../tests/fixtures/e2b/e2b/cordis.yml | 9 ---- packages/README.i18n.yaml | 4 +- packages/e2b/README.i18n.yaml | 4 +- packages/e2b/README.md | 2 +- packages/e2b/README.zh.md | 2 +- packages/e2b/e2b/README.i18n.yaml | 4 +- packages/e2b/e2b/README.md | 2 +- packages/e2b/e2b/README.zh.md | 2 +- packages/e2b/e2b/tests/composition.e2e.ts | 11 +--- packages/e2b/subprocess-e2b/README.i18n.yaml | 4 +- packages/e2b/subprocess-e2b/README.md | 2 +- packages/e2b/subprocess-e2b/README.zh.md | 2 +- packages/e2b/subprocess-e2b/src/index.ts | 14 ++--- .../e2b/subprocess-e2b/tests/terminal.spec.ts | 4 +- 19 files changed, 29 insertions(+), 111 deletions(-) diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml index c7ce82595d..fb8a3344c1 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md -2026-07-28-portable-execution-world-consumers.md: 57c3a8a8b211279593359b02def865644052f549 -2026-07-28-portable-execution-world-consumers.zh.md: 8231913c26f75361add7a28bc40f7723c9b18728 +2026-07-28-portable-execution-world-consumers.md: fcc30d553f07a28b44227f614f62c2027f8f3dd3 +2026-07-28-portable-execution-world-consumers.zh.md: 5e1e73aee7b96ffe1ae9862f72114a48152ad218 diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md index 6f296bfa74..fcc30d553f 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md @@ -8,7 +8,7 @@ English | [中文](2026-07-28-portable-execution-world-consumers.zh.md) The filesystem and subprocess seams made file and ordinary process access replaceable, but PTY and LSP still reached host Node APIs directly. A remote execution provider therefore appeared to need separate PTY and LSP packages even though their domain behavior did not change. Those packages would be shallow adapters: each would duplicate an existing consumer merely to replace its file and process operations. -A remote coding world is useful only when file operations, commands, terminals, language servers, and model-written programs share one sandbox identity. Moving the complete harness into that sandbox would also entangle provider experimentation with plugin loading, credentials, model transport, session durability, supervision, and deployment. +A remote coding world is useful only when file operations, commands, terminals, and language servers share one sandbox identity. Moving the complete harness into that sandbox would also entangle provider experimentation with plugin loading, credentials, model transport, session durability, supervision, and deployment. Ordinary pipes do not cover one requirement. A persistent terminal needs PTY allocation, foreground-process-group inspection and signalling, and cleanup of the complete terminal session. Pretending those operations can be rebuilt in `dsh-pty-local` from an ordinary `spawn()` handle would either leak provider internals or weaken its lifecycle contract. @@ -30,7 +30,7 @@ Generic consumers use that execution world: The opt-in E2B realization has exactly three provider-specific packages under `packages/e2b/`: `dsh-e2b` creates one sandbox and deletes it on timeout or disposal, `dsh-fs-e2b` implements `ctx.fs`, and `dsh-subprocess-e2b` implements `ctx.subprocess` over E2B Commands, PTYs, and remote Linux process groups. The two adapters obtain the sole SDK handle from the owner and never create private sandboxes. -E2B owns the mutable filesystem, managed command and Bash processes, terminal allocation and terminal-session groups, language-server processes and source reads, subprocess Code Runtime processes, and adapter-private files under `.dsh-e2b`. The host owns Cordis and plugin objects, the agent loop, agent/session/goal state, session logs and persistence, LLM calls, prompts and tools, authority, skills, subagent orchestration, PTY buffers and readiness, LSP protocol state, Code Runtime program/binding/output policy, and E2B SDK/network buffers. The overlay neither uploads nor synchronizes the host workspace. +E2B owns the mutable filesystem, managed command and Bash processes, terminal allocation and terminal-session groups, language-server processes and source reads, and adapter-private files under `.dsh-e2b`. The host owns Cordis and plugin objects, the agent loop, agent/session/goal state, session logs and persistence, LLM calls, prompts and tools, authority, skills, subagent orchestration, PTY buffers and readiness, LSP protocol state, the worker-backed Code Runtime and its program/binding/output policy, and E2B SDK/network buffers. The overlay neither uploads nor synchronizes the host workspace. The adapters retain only substrate mechanics. Filesystem canonicalization crosses the SDK's decoded command transport as strict base64-encoded NUL framing; streamed reads leave byte ceilings with consumers. Subprocess command output and environment snapshots use ASCII/base64 where SDK chunk decoding would otherwise lose bytes, while private control shells isolate profiles and later launches blank discovered credential-shaped names. Process and terminal cleanup uses remote groups and proves quiescence before settlement. @@ -38,7 +38,7 @@ Sandbox state is deliberately ephemeral: timeout and disposal delete the remote ## Verification -Focused package suites pin sandbox lifecycle, canonical path framing, filesystem metadata and atomic versions, subprocess publication/rollback, terminal text I/O and session cleanup, output limits, cancellation, disposal, and invariant registration. A credential-gated Loader composition exercises the same three-package provider through source imports and built exports, including FS/Bash visibility, hostile login profiles, byte-split UTF-8 output, process and terminal cleanup, LSP queries, Code Runtime bindings and cleanup, host-workspace isolation, and final sandbox deletion. +Focused package suites pin sandbox lifecycle, canonical path framing, filesystem metadata and atomic versions, subprocess publication/rollback, terminal text I/O and session cleanup, output limits, cancellation, disposal, and invariant registration. A credential-gated Loader composition exercises the same three-package provider through source imports and built exports, including FS/Bash visibility, hostile login profiles, byte-split UTF-8 output, process and terminal cleanup, LSP queries, host-workspace isolation, and final sandbox deletion. ## Alternatives considered diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md index 471753573d..2ab0685a1d 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md @@ -8,7 +8,7 @@ Status: implemented 文件系统与进程管理 seam 使文件访问和普通进程访问具备可替换性,但若干上层能力仍直接调用宿主 Node API。因此,即使领域行为没有变化,远程执行提供方看起来仍需要独立的 PTY、LSP 与代码运行时包(package)。这些包只会成为浅层适配器:每个包都仅为替换文件与进程操作而复制一个现有消费方。 -只有文件操作、命令、终端、语言服务器和模型编写的程序共享同一个沙箱身份时,远程编码世界才有用。若把完整 harness 移入该沙箱,还会把提供方实验与插件加载、凭据、模型传输、会话持久性、监督和部署纠缠在一起。 +只有文件操作、命令、终端和语言服务器共享同一个沙箱身份时,远程编码世界才有用。若把完整 harness 移入该沙箱,还会把提供方实验与插件加载、凭据、模型传输、会话持久性、监督和部署纠缠在一起。 普通管道无法满足其中一项要求。持久终端需要分配 PTY、检查前台进程组并发送信号,以及清理完整的终端会话。如果假设可以在 `dsh-pty-local` 中基于普通 `spawn()` 句柄重建这些操作,最终不是泄漏提供方内部细节,就是削弱其生命周期契约。 @@ -33,7 +33,7 @@ Status: implemented 可选启用的 E2B 实现在 `packages/e2b/` 下恰好只有三个提供方专用包:`dsh-e2b` 创建一个沙箱,并在超时或资源释放时将其删除;`dsh-fs-e2b` 实现 `ctx.fs`;`dsh-subprocess-e2b` 基于 E2B Commands、PTY 和远程 Linux 进程组实现 `ctx.subprocess`。两个适配器都从所有者取得唯一的 SDK 句柄,绝不创建私有沙箱。 -E2B 负责可变文件系统、受管命令与 Bash 进程、终端分配与终端会话组、语言服务器进程与源文件读取、子进程代码运行时进程,以及 `.dsh-e2b` 下的适配器私有文件。宿主负责 Cordis 与插件对象、agent loop(智能体循环)、agent(智能体)状态、会话状态与目标状态、会话日志与持久化、LLM(大语言模型)调用、提示词与工具、权限、skill(技能)、subagent 编排、PTY 缓冲区与就绪状态、LSP 协议状态、代码运行时程序/绑定/输出策略,以及 E2B SDK/网络缓冲区。该叠加层既不上传,也不同步宿主工作区。 +E2B 负责可变文件系统、受管命令与 Bash 进程、终端分配与终端会话组、语言服务器进程与源文件读取,以及 `.dsh-e2b` 下的适配器私有文件。宿主负责 Cordis 与插件对象、agent loop(智能体循环)、agent(智能体)状态、会话状态与目标状态、会话日志与持久化、LLM(大语言模型)调用、提示词与工具、权限、skill(技能)、subagent 编排、PTY 缓冲区与就绪状态、LSP 协议状态、基于 worker 的代码运行时及其程序/绑定/输出策略,以及 E2B SDK/网络缓冲区。该叠加层既不上传,也不同步宿主工作区。 适配器只保留执行基底机制。文件系统规范化以严格的 base64 加 NUL 分帧穿过 SDK 已解码的命令传输;流式读取把字节上限留给消费方执行。进程管理命令输出与环境快照采用 ASCII/base64,避免 SDK 分片解码丢失字节;私有控制 shell 隔离 profile,后续启动会把已发现且名称呈凭据特征的环境变量置空。进程与终端清理使用远程进程组,并在结算前证明完全停稳。 @@ -41,7 +41,7 @@ E2B 负责可变文件系统、受管命令与 Bash 进程、终端分配与终 ## 验证 -聚焦的包测试套件锁定了沙箱生命周期、规范化路径分帧、文件系统元数据与原子版本、进程管理发布/回滚、终端文本 I/O 与会话清理、输出上限、取消、资源释放和不变式注册。一项受凭据门控的 Loader 组合通过源代码导入与构建后导出运行同一套三包提供方组合,其中包括 FS/Bash 可见性、恶意登录 profile、跨字节边界拆分的 UTF-8 输出、进程与终端清理、LSP 查询、代码运行时绑定与清理、宿主工作区隔离,以及最终沙箱删除。 +聚焦的包测试套件锁定了沙箱生命周期、规范化路径分帧、文件系统元数据与原子版本、进程管理发布/回滚、终端文本 I/O 与会话清理、输出上限、取消、资源释放和不变式注册。一项受凭据门控的 Loader 组合通过源代码导入与构建后导出运行同一套三包提供方组合,其中包括 FS/Bash 可见性、恶意登录 profile、跨字节边界拆分的 UTF-8 输出、进程与终端清理、LSP 查询、宿主工作区隔离,以及最终沙箱删除。 ## 考虑过的替代方案 diff --git a/examples/headless-agent/e2b.cordis.yml b/examples/headless-agent/e2b.cordis.yml index 724f30240d..a65b0baf81 100644 --- a/examples/headless-agent/e2b.cordis.yml +++ b/examples/headless-agent/e2b.cordis.yml @@ -1,6 +1,6 @@ # POC overlay: keep the advanced headless agent and model-facing tools, but # place its filesystem and process substrate in one short-lived E2B sandbox; -# the generic Bash, PTY, LSP, and Code Runtime consumers compose above them. +# the generic Bash, PTY, and LSP consumers compose above them. - id: base name: '@cordisjs/plugin-include' config: @@ -12,9 +12,6 @@ - id: fs-local name: '@deepseek-ai/dsh-fs-local' disabled: true - - id: code-runtime - name: '@deepseek-ai/dsh-code-runtime-worker' - disabled: true - insert: - id: e2b name: '@deepseek-ai/dsh-e2b' @@ -25,8 +22,6 @@ name: '@deepseek-ai/dsh-subprocess-e2b' - id: fs-e2b name: '@deepseek-ai/dsh-fs-e2b' - - id: code-runtime-subprocess - name: '@deepseek-ai/dsh-code-runtime-subprocess' - id: sandbox-policy name: '@deepseek-ai/dsh-sandbox-policy' config: diff --git a/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts b/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts index 570a244e88..7348d1d181 100644 --- a/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts +++ b/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts @@ -3,7 +3,6 @@ import { resolve } from 'node:path' import { boot } from '@deepseek-ai/dsh-app-boot' import type { Agent } from '@deepseek-ai/dsh-agent' import { Session, SessionId } from '@deepseek-ai/dsh-session' -import type {} from '@deepseek-ai/dsh-code-runtime-subprocess' import type {} from '@deepseek-ai/dsh-fs-e2b' import type {} from '@deepseek-ai/dsh-bash-local' import type {} from '@deepseek-ai/dsh-lsp-local' @@ -175,52 +174,6 @@ try { const terminalTreeCleanup = stubbornProbe.stdout === 'gone' if (!terminalTreeCleanup) throw new Error(`E2B PTY left process ${stubbornPid} alive after close`) - const code = await ctx.codeRuntime.run({ - program: ` - console.log('remote-log 你好', 42) - const doubled: number = await bridge.double({ value: 21 }) - let typed = false - try { - await bridge.fail({ reason: 'expected' }) - } catch (error) { - typed = error instanceof BridgeError && (error as { member: string }).member === 'fail' - } - return { doubled, typed } - `, - bindings: [{ - global: 'bridge', - errorClass: { name: 'BridgeError', memberNameProperty: 'member' }, - functions: { - double: async (args) => { - const value = (args as { value: number }).value - return value * 2 - }, - fail: async () => { throw new Error('binding rejected') }, - }, - }], - }) - const descendantPipe = await ctx.codeRuntime.run({ - program: ` - const childProcess = await import('node:child_process') - const child = childProcess.spawn( - process.execPath, - ['-e', 'setInterval(() => {}, 1000)', 'dsh-code-runtime-descendant'], - { stdio: ['ignore', 'inherit', 'inherit'] }, - ) - return child.pid > 0 - `, - bindings: [], - }) - const descendantProcesses = await sandbox.commands.list() - const descendantCleanup = !descendantProcesses.some(processInfo => - JSON.stringify([processInfo.cmd, processInfo.args]).includes('dsh-code-runtime-descendant'), - ) - if (!descendantCleanup) throw new Error('E2B Code Runtime left a pipe-holding descendant alive') - const remoteProcesses = await (await ctx.e2b.getSandbox()).commands.list() - const lingeringCodeRunners = remoteProcesses.filter(processInfo => - JSON.stringify([processInfo.cmd, processInfo.args]).includes('code-runtime-runner.mjs'), - ) - process.stdout.write(`${JSON.stringify({ sandboxId: (await ctx.e2b.getSandbox()).sandboxId, bashRead: bashRead.stdout.text, @@ -237,10 +190,6 @@ try { treeCleanup: terminalTreeCleanup, scrollback: terminalScrollback.text, }, - code, - descendantPipe, - descendantCleanup, - lingeringCodeRunners: lingeringCodeRunners.length, })}\n`) } finally { if (terminalId !== undefined) await ctx.pty.kill(owner, terminalId, 'fixture cleanup').catch(() => false) diff --git a/examples/headless-agent/tests/fixtures/e2b/e2b/cordis.yml b/examples/headless-agent/tests/fixtures/e2b/e2b/cordis.yml index 043362aaa1..3a6deb7e49 100644 --- a/examples/headless-agent/tests/fixtures/e2b/e2b/cordis.yml +++ b/examples/headless-agent/tests/fixtures/e2b/e2b/cordis.yml @@ -52,12 +52,3 @@ .ts: typescript shutdownTimeoutMs: 1000 killGraceMs: 500 - -- id: code-runtime-subprocess - name: '@deepseek-ai/dsh-code-runtime-subprocess' - config: - computeMs: 500 - maxWallMs: 15000 - maxOutputBytes: 4096 - maxOldGenerationSizeMb: 128 - killGraceMs: 500 diff --git a/packages/README.i18n.yaml b/packages/README.i18n.yaml index f101375bc6..ce95627688 100644 --- a/packages/README.i18n.yaml +++ b/packages/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/README.md -README.md: 1fd0c9c2385f3169791a28f3ef50bb42792760af -README.zh.md: f9f7d5ae8839ea7033a349ac7a0083a97780a21c +README.md: a9a121d0cb13cd5e2148da8d754967359be06bab +README.zh.md: c6500ebb2abce14fabf68b79ec20f5c23cf5d5f7 diff --git a/packages/e2b/README.i18n.yaml b/packages/e2b/README.i18n.yaml index 6b1dc7506b..b93f48d8cd 100644 --- a/packages/e2b/README.i18n.yaml +++ b/packages/e2b/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/e2b/README.md -README.md: 3c85b2790d3414af73625e65732df58ff9fec566 -README.zh.md: a759539fc054dd68e75f3c7660ccc3b762985af5 +README.md: adc002649e44faefd2f6bd84b7b45fb9fc6fa41d +README.zh.md: d941de8da2cb831a69a165c86a825def68c2b319 diff --git a/packages/e2b/README.md b/packages/e2b/README.md index 3c85b2790d..adc002649e 100644 --- a/packages/e2b/README.md +++ b/packages/e2b/README.md @@ -10,6 +10,6 @@ An experimental provider-composition POC that places one filesystem/process exec | [`fs-e2b`](fs-e2b/README.md) (`@deepseek-ai/dsh-fs-e2b`) | `ctx.fs` | Implement the filesystem seam over E2B Filesystem APIs | | [`subprocess-e2b`](subprocess-e2b/README.md) (`@deepseek-ai/dsh-subprocess-e2b`) | `ctx.subprocess` | Implement executable lookup, managed process groups and stdio, remote spill files, and terminal sessions over E2B Commands and PTY APIs | -The existing [`dsh-bash-local`](../bash/bash-local/README.md), [`dsh-pty-local`](../pty/pty-local/README.md), [`dsh-lsp-local`](../lsp/lsp-local/README.md), and [`dsh-code-runtime-subprocess`](../code-runtime/code-runtime-subprocess/README.md) need no E2B-specific forks. They delegate every execution-world operation to `ctx.fs` and `ctx.subprocess`, so mounting the two E2B adapters places their mutable work in the same sandbox. +The existing [`dsh-bash-local`](../bash/bash-local/README.md), [`dsh-pty-local`](../pty/pty-local/README.md), and [`dsh-lsp-local`](../lsp/lsp-local/README.md) need no E2B-specific forks. They delegate every execution-world operation to `ctx.fs` and `ctx.subprocess`, so mounting the two E2B adapters places their mutable work in the same sandbox. The worker-backed Code Runtime remains host-local and reaches that world only through the tools its programs call. This boundary does not move the harness process, Cordis objects, model calls, agent/session state, session persistence, skills, higher-level protocol state, or E2B SDK buffers. The [portable execution-world decision](../../.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md) owns both the generic composition and this POC boundary. diff --git a/packages/e2b/README.zh.md b/packages/e2b/README.zh.md index a759539fc0..d941de8da2 100644 --- a/packages/e2b/README.zh.md +++ b/packages/e2b/README.zh.md @@ -10,6 +10,6 @@ | [`fs-e2b`](fs-e2b/README.md)(`@deepseek-ai/dsh-fs-e2b`) | `ctx.fs` | 通过 E2B Filesystem API 实现文件系统 seam | | [`subprocess-e2b`](subprocess-e2b/README.md)(`@deepseek-ai/dsh-subprocess-e2b`) | `ctx.subprocess` | 通过 E2B Commands 与 PTY API 实现可执行文件查找、受管进程组与 stdio、远程 spill 文件及终端会话 | -现有的 [`dsh-bash-local`](../bash/bash-local/README.md)、[`dsh-pty-local`](../pty/pty-local/README.md)、[`dsh-lsp-local`](../lsp/lsp-local/README.md) 及 [`dsh-code-runtime-subprocess`](../code-runtime/code-runtime-subprocess/README.md) 无需 E2B 专用 fork。它们把执行环境中的所有操作委托给 `ctx.fs` 和 `ctx.subprocess`,因此挂载这两个 E2B 适配器后,它们执行的可变操作都发生在同一个沙箱内。 +现有的 [`dsh-bash-local`](../bash/bash-local/README.md)、[`dsh-pty-local`](../pty/pty-local/README.md) 和 [`dsh-lsp-local`](../lsp/lsp-local/README.md) 无需 E2B 专用 fork。它们把执行环境中的所有操作委托给 `ctx.fs` 和 `ctx.subprocess`,因此挂载这两个 E2B 适配器后,它们执行的可变操作都发生在同一个沙箱内。基于 worker 的代码运行时仍在宿主本地运行,只会通过其程序调用的工具触达该执行世界。 该边界不会迁移 harness 进程、Cordis 对象、模型调用、agent(智能体)/会话状态、会话持久化、skill(技能)、更高层协议状态或 E2B SDK 缓冲。[可移植执行世界决策](../../.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md)同时界定通用组合和此 POC 边界。 diff --git a/packages/e2b/e2b/README.i18n.yaml b/packages/e2b/e2b/README.i18n.yaml index 3587067d31..0256b3ccfe 100644 --- a/packages/e2b/e2b/README.i18n.yaml +++ b/packages/e2b/e2b/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/e2b/e2b/README.md -README.md: 7ade7c3d6522d8fa6d54d7011766238451b17c9a -README.zh.md: b683f33d2211106cf422e780b2b37904b0c640ad +README.md: 52f99178ce8c06e61c13ea9364769ac887db4410 +README.zh.md: 98ba7e79841f548ff6465fda4bc37170c17bc6e0 diff --git a/packages/e2b/e2b/README.md b/packages/e2b/e2b/README.md index 7ade7c3d65..52f99178ce 100644 --- a/packages/e2b/e2b/README.md +++ b/packages/e2b/e2b/README.md @@ -38,7 +38,7 @@ No direct invalidation; this package does not contribute request tokens. ## Known Limitations and Deferred Work -- **This is not a whole-harness runtime** — Cordis services, agent/session state, session logs, LLM requests, skills, and SDK-side buffers stay in the host process. +- **This is not a whole-harness runtime** — Cordis services, the worker-backed Code Runtime, agent/session state, session logs, LLM requests, skills, and SDK-side buffers stay in the host process. - **Sandbox state is ephemeral** — disposal and timeout delete the sandbox; reconnect, pause/leave retention, templates, volumes, and snapshots are outside this POC. - **No deployment platform is configured** — network policy, host-workspace synchronization, and sandbox discovery are outside this POC. - **`cwd` is a resolution convention, not containment** — adapters and commands can address other sandbox paths; E2B network access retains the base image's policy. diff --git a/packages/e2b/e2b/README.zh.md b/packages/e2b/e2b/README.zh.md index b683f33d22..98ba7e7984 100644 --- a/packages/e2b/e2b/README.zh.md +++ b/packages/e2b/e2b/README.zh.md @@ -38,7 +38,7 @@ ## 已知限制与延后工作 -- **这不是完整的 harness 运行时**:Cordis 服务、agent(智能体)/会话状态、会话日志、LLM(大语言模型)请求、skill(技能)和 SDK 侧缓冲仍留在宿主进程中。 +- **这不是完整的 harness 运行时**:Cordis 服务、基于 worker 的代码运行时、agent(智能体)/会话状态、会话日志、LLM(大语言模型)请求、skill(技能)和 SDK 侧缓冲仍留在宿主进程中。 - **沙箱状态是短暂的**:资源释放和超时都会删除沙箱;重新连接、pause/leave 保留、模板、卷和快照均不在本 POC 范围内。 - **没有配置部署平台**:网络策略、宿主工作区同步和沙箱发现均不在本 POC 范围内。 - **`cwd` 是解析约定,而不是包含边界**:适配器和命令可以访问沙箱中的其他路径;E2B 网络访问也继续采用基础镜像的策略。 diff --git a/packages/e2b/e2b/tests/composition.e2e.ts b/packages/e2b/e2b/tests/composition.e2e.ts index bf53ba7bb9..3f0b69668d 100644 --- a/packages/e2b/e2b/tests/composition.e2e.ts +++ b/packages/e2b/e2b/tests/composition.e2e.ts @@ -57,7 +57,7 @@ describe.skipIf(!process.env.E2B_API_KEY)('E2B live Loader composition', () => { const ptyFiber = await ctx.plugin(PtyService) const subprocessFiber = await ctx.plugin(E2BSubprocessService) const node = await ctx.subprocess.resolveExecutable('node') - const relativeNodePath = posix.relative(ctx.subprocess.cwd, posix.dirname(node)) || '.' + const relativeNodePath = posix.relative(ctx.e2b.cwd, posix.dirname(node)) || '.' await expect(ctx.subprocess.resolveExecutable('node', { PATH: relativeNodePath })).resolves.toBe(node) await expect(sandbox.files.read(profileLeakPath)).rejects.toBeInstanceOf(FileNotFoundError) const environmentProbe = ctx.subprocess.spawn({ @@ -117,7 +117,7 @@ describe.skipIf(!process.env.E2B_API_KEY)('E2B live Loader composition', () => { } }, 70_000) - it('runs FS, Bash, PTY, LSP, and Code Runtime in one sandbox and deletes it', async () => { + it('runs FS, Bash, PTY, and LSP in one sandbox and deletes it', async () => { const { stdout, stderr } = await runLoaderSmoke({ label: 'E2B composition', tempDirPrefix: 'dsh-e2b-composition-', @@ -157,9 +157,6 @@ describe.skipIf(!process.env.E2B_API_KEY)('E2B live Loader composition', () => { interrupted: { sessionStatus: { kind: 'running' } }, treeCleanup: true, }, - descendantPipe: { value: true, logs: [] }, - descendantCleanup: true, - lingeringCodeRunners: 0, }) const terminalMotd = (output.terminal as { motd: string }).motd expect(terminalMotd.length).toBeGreaterThan(0) @@ -171,10 +168,6 @@ describe.skipIf(!process.env.E2B_API_KEY)('E2B live Loader composition', () => { expect(['stdin_read', 'inferred_idle']).toContain( (output.terminal as { interrupted: { waitReason: string } }).interrupted.waitReason, ) - expect(output.code).toEqual({ - value: { doubled: 42, typed: true }, - logs: ['remote-log 你好 42'], - }) const apiKey = process.env.E2B_API_KEY if (apiKey === undefined) throw new Error('E2B_API_KEY disappeared during the live composition test') await expect(Sandbox.getInfo(String(output.sandboxId), { apiKey })).rejects.toBeInstanceOf(SandboxNotFoundError) diff --git a/packages/e2b/subprocess-e2b/README.i18n.yaml b/packages/e2b/subprocess-e2b/README.i18n.yaml index 381ada5198..0a18bf24a1 100644 --- a/packages/e2b/subprocess-e2b/README.i18n.yaml +++ b/packages/e2b/subprocess-e2b/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/e2b/subprocess-e2b/README.md -README.md: 3ddfe5bc47309ba56d3f8af7720fcbe71ec8a302 -README.zh.md: c4db7bf8bb8a0f4b2f147dd22868978be5d1fcf4 +README.md: 3bb9802c3afa41f3949fceeb73e76ea5cebce744 +README.zh.md: 520f0ea1cde18ed523ba80015d3d4700730aba7a diff --git a/packages/e2b/subprocess-e2b/README.md b/packages/e2b/subprocess-e2b/README.md index 3ddfe5bc47..3bb9802c3a 100644 --- a/packages/e2b/subprocess-e2b/README.md +++ b/packages/e2b/subprocess-e2b/README.md @@ -2,7 +2,7 @@ English | [中文](README.zh.md) -E2B implementation of the [`@deepseek-ai/dsh-subprocess`](../../subprocess/subprocess/README.md) seam. It has no config: load [`@deepseek-ai/dsh-e2b`](../e2b/README.md) first, then this service in place of `dsh-subprocess-local`. Existing Bash, PTY, LSP, and subprocess Code Runtime consumers then execute in the shared remote sandbox without E2B-specific capability packages. +E2B implementation of the [`@deepseek-ai/dsh-subprocess`](../../subprocess/subprocess/README.md) seam. It has no config: load [`@deepseek-ai/dsh-e2b`](../e2b/README.md) first, then this service in place of `dsh-subprocess-local`. Existing Bash, PTY, and LSP consumers then execute in the shared remote sandbox without E2B-specific capability packages. ## Behavior diff --git a/packages/e2b/subprocess-e2b/README.zh.md b/packages/e2b/subprocess-e2b/README.zh.md index c4db7bf8bb..520f0ea1cd 100644 --- a/packages/e2b/subprocess-e2b/README.zh.md +++ b/packages/e2b/subprocess-e2b/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -[`@deepseek-ai/dsh-subprocess`](../../subprocess/subprocess/README.md) seam 的 E2B 实现。它没有配置:先加载 [`@deepseek-ai/dsh-e2b`](../e2b/README.md),再用本服务取代 `dsh-subprocess-local`。现有的 Bash、PTY、LSP 以及使用 subprocess 的 Code Runtime 消费方随后会在共享远程沙箱中执行,无需 E2B 专用的功能包(package)。 +[`@deepseek-ai/dsh-subprocess`](../../subprocess/subprocess/README.md) seam 的 E2B 实现。它没有配置:先加载 [`@deepseek-ai/dsh-e2b`](../e2b/README.md),再用本服务取代 `dsh-subprocess-local`。现有的 Bash、PTY 和 LSP 消费方随后会在共享远程沙箱中执行,无需 E2B 专用的功能包(package)。 ## 行为 diff --git a/packages/e2b/subprocess-e2b/src/index.ts b/packages/e2b/subprocess-e2b/src/index.ts index 976880ab01..15e7feccdd 100644 --- a/packages/e2b/subprocess-e2b/src/index.ts +++ b/packages/e2b/subprocess-e2b/src/index.ts @@ -31,17 +31,9 @@ export class E2BSubprocessService extends SubprocessService { private readonly terminalSetups = new Map, AbortController>() private disposing = false - /** @inheritdoc */ - readonly cwd: string - - /** @inheritdoc */ - readonly runtimeRoot: string - /** Create the E2B subprocess service and bind its disposal policy. */ constructor(ctx: Context) { super(ctx) - this.cwd = ctx.e2b.cwd - this.runtimeRoot = ctx.e2b.runtimeRoot ctx.effect(() => async () => { this.disposing = true for (const controller of this.terminalSetups.values()) { @@ -89,14 +81,14 @@ export class E2BSubprocessService extends SubprocessService { const prefix = path === undefined ? '' : `PATH=${quoteE2BShellArg(path)} ` const result = await sandbox.commands.run( `${prefix}command -v -- ${quoteE2BShellArg(command)}`, - { cwd: this.cwd, envs: e2bControlEnvs(), ...signalOpts(signal) }, + { cwd: this.ctx.e2b.cwd, envs: e2bControlEnvs(), ...signalOpts(signal) }, ) signal?.throwIfAborted() const executable = result.stdout.trim() if (executable.includes('\n') || (!posix.isAbsolute(executable) && !executable.includes('/'))) { throw new Error(`subprocess-e2b: executable ${JSON.stringify(command)} did not resolve to one absolute path`) } - return posix.resolve(this.cwd, executable) + return posix.resolve(this.ctx.e2b.cwd, executable) } /** @inheritdoc */ @@ -138,7 +130,7 @@ export class E2BSubprocessService extends SubprocessService { } } spec.signal?.throwIfAborted() - const stateDir = posix.join(this.runtimeRoot, 'terminals', randomUUID()) + const stateDir = posix.join(this.ctx.e2b.runtimeRoot, 'terminals', randomUUID()) const setup = Promise.withResolvers() const setupController = new AbortController() const setupSignal = spec.signal === undefined diff --git a/packages/e2b/subprocess-e2b/tests/terminal.spec.ts b/packages/e2b/subprocess-e2b/tests/terminal.spec.ts index 067910a10b..9973b5151a 100644 --- a/packages/e2b/subprocess-e2b/tests/terminal.spec.ts +++ b/packages/e2b/subprocess-e2b/tests/terminal.spec.ts @@ -775,10 +775,8 @@ describe('E2B subprocess terminal service', () => { return { ctx, fiber, fake } } - it('publishes execution-world coordinates and resolves remote executables', async () => { + it('resolves remote executables', async () => { const { ctx, fake } = await service() - expect(ctx.subprocess.cwd).toBe('/workspace') - expect(ctx.subprocess.runtimeRoot).toBe('/workspace/.dsh-e2b') await expect(ctx.subprocess.resolveExecutable('/bin/bash')).resolves.toBe('/bin/bash') await expect(ctx.subprocess.resolveExecutable('node', { PATH: '/custom/bin' }, new AbortController().signal)) .resolves.toBe('/usr/bin/node') From e6b3afbee9078429f18a11ce30b2c869e5532545 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Thu, 30 Jul 2026 20:02:38 +0800 Subject: [PATCH 099/108] docs(e2b): remove code runtime residue --- .../2026-07-28-portable-execution-world-consumers.i18n.yaml | 4 ++-- .../2026-07-28-portable-execution-world-consumers.md | 2 +- .../2026-07-28-portable-execution-world-consumers.zh.md | 2 +- packages/e2b/README.i18n.yaml | 4 ++-- packages/e2b/README.md | 2 +- packages/e2b/README.zh.md | 2 +- packages/e2b/e2b/README.i18n.yaml | 4 ++-- packages/e2b/e2b/README.md | 2 +- packages/e2b/e2b/README.zh.md | 2 +- 9 files changed, 12 insertions(+), 12 deletions(-) diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml index fb8a3344c1..8cd6ad419a 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md -2026-07-28-portable-execution-world-consumers.md: fcc30d553f07a28b44227f614f62c2027f8f3dd3 -2026-07-28-portable-execution-world-consumers.zh.md: 5e1e73aee7b96ffe1ae9862f72114a48152ad218 +2026-07-28-portable-execution-world-consumers.md: 2cfdf08ac369048994ebc64d498caaa0847b77de +2026-07-28-portable-execution-world-consumers.zh.md: ce6650a284e7d64fb1277c758bb77c2394e942c8 diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md index fcc30d553f..2cfdf08ac3 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md @@ -30,7 +30,7 @@ Generic consumers use that execution world: The opt-in E2B realization has exactly three provider-specific packages under `packages/e2b/`: `dsh-e2b` creates one sandbox and deletes it on timeout or disposal, `dsh-fs-e2b` implements `ctx.fs`, and `dsh-subprocess-e2b` implements `ctx.subprocess` over E2B Commands, PTYs, and remote Linux process groups. The two adapters obtain the sole SDK handle from the owner and never create private sandboxes. -E2B owns the mutable filesystem, managed command and Bash processes, terminal allocation and terminal-session groups, language-server processes and source reads, and adapter-private files under `.dsh-e2b`. The host owns Cordis and plugin objects, the agent loop, agent/session/goal state, session logs and persistence, LLM calls, prompts and tools, authority, skills, subagent orchestration, PTY buffers and readiness, LSP protocol state, the worker-backed Code Runtime and its program/binding/output policy, and E2B SDK/network buffers. The overlay neither uploads nor synchronizes the host workspace. +E2B owns the mutable filesystem, managed command and Bash processes, terminal allocation and terminal-session groups, language-server processes and source reads, and adapter-private files under `.dsh-e2b`. The host owns Cordis and plugin objects, the agent loop, agent/session/goal state, session logs and persistence, LLM calls, prompts and tools, authority, skills, subagent orchestration, PTY buffers and readiness, LSP protocol state, and E2B SDK/network buffers. The overlay neither uploads nor synchronizes the host workspace. The adapters retain only substrate mechanics. Filesystem canonicalization crosses the SDK's decoded command transport as strict base64-encoded NUL framing; streamed reads leave byte ceilings with consumers. Subprocess command output and environment snapshots use ASCII/base64 where SDK chunk decoding would otherwise lose bytes, while private control shells isolate profiles and later launches blank discovered credential-shaped names. Process and terminal cleanup uses remote groups and proves quiescence before settlement. diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md index 2ab0685a1d..68d022216c 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md @@ -33,7 +33,7 @@ Status: implemented 可选启用的 E2B 实现在 `packages/e2b/` 下恰好只有三个提供方专用包:`dsh-e2b` 创建一个沙箱,并在超时或资源释放时将其删除;`dsh-fs-e2b` 实现 `ctx.fs`;`dsh-subprocess-e2b` 基于 E2B Commands、PTY 和远程 Linux 进程组实现 `ctx.subprocess`。两个适配器都从所有者取得唯一的 SDK 句柄,绝不创建私有沙箱。 -E2B 负责可变文件系统、受管命令与 Bash 进程、终端分配与终端会话组、语言服务器进程与源文件读取,以及 `.dsh-e2b` 下的适配器私有文件。宿主负责 Cordis 与插件对象、agent loop(智能体循环)、agent(智能体)状态、会话状态与目标状态、会话日志与持久化、LLM(大语言模型)调用、提示词与工具、权限、skill(技能)、subagent 编排、PTY 缓冲区与就绪状态、LSP 协议状态、基于 worker 的代码运行时及其程序/绑定/输出策略,以及 E2B SDK/网络缓冲区。该叠加层既不上传,也不同步宿主工作区。 +E2B 负责可变文件系统、受管命令与 Bash 进程、终端分配与终端会话组、语言服务器进程与源文件读取,以及 `.dsh-e2b` 下的适配器私有文件。宿主负责 Cordis 与插件对象、agent loop(智能体循环)、agent(智能体)状态、会话状态与目标状态、会话日志与持久化、LLM(大语言模型)调用、提示词与工具、权限、skill(技能)、subagent 编排、PTY 缓冲区与就绪状态、LSP 协议状态,以及 E2B SDK/网络缓冲区。该叠加层既不上传,也不同步宿主工作区。 适配器只保留执行基底机制。文件系统规范化以严格的 base64 加 NUL 分帧穿过 SDK 已解码的命令传输;流式读取把字节上限留给消费方执行。进程管理命令输出与环境快照采用 ASCII/base64,避免 SDK 分片解码丢失字节;私有控制 shell 隔离 profile,后续启动会把已发现且名称呈凭据特征的环境变量置空。进程与终端清理使用远程进程组,并在结算前证明完全停稳。 diff --git a/packages/e2b/README.i18n.yaml b/packages/e2b/README.i18n.yaml index b93f48d8cd..4125487253 100644 --- a/packages/e2b/README.i18n.yaml +++ b/packages/e2b/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/e2b/README.md -README.md: adc002649e44faefd2f6bd84b7b45fb9fc6fa41d -README.zh.md: d941de8da2cb831a69a165c86a825def68c2b319 +README.md: f9758e2748aaa2acffb0e928752b2b0fb70cd0a4 +README.zh.md: de068c16d4309499f5dcaa2d08cd9b6cc3023d94 diff --git a/packages/e2b/README.md b/packages/e2b/README.md index adc002649e..f9758e2748 100644 --- a/packages/e2b/README.md +++ b/packages/e2b/README.md @@ -10,6 +10,6 @@ An experimental provider-composition POC that places one filesystem/process exec | [`fs-e2b`](fs-e2b/README.md) (`@deepseek-ai/dsh-fs-e2b`) | `ctx.fs` | Implement the filesystem seam over E2B Filesystem APIs | | [`subprocess-e2b`](subprocess-e2b/README.md) (`@deepseek-ai/dsh-subprocess-e2b`) | `ctx.subprocess` | Implement executable lookup, managed process groups and stdio, remote spill files, and terminal sessions over E2B Commands and PTY APIs | -The existing [`dsh-bash-local`](../bash/bash-local/README.md), [`dsh-pty-local`](../pty/pty-local/README.md), and [`dsh-lsp-local`](../lsp/lsp-local/README.md) need no E2B-specific forks. They delegate every execution-world operation to `ctx.fs` and `ctx.subprocess`, so mounting the two E2B adapters places their mutable work in the same sandbox. The worker-backed Code Runtime remains host-local and reaches that world only through the tools its programs call. +The existing [`dsh-bash-local`](../bash/bash-local/README.md), [`dsh-pty-local`](../pty/pty-local/README.md), and [`dsh-lsp-local`](../lsp/lsp-local/README.md) need no E2B-specific forks. They delegate every execution-world operation to `ctx.fs` and `ctx.subprocess`, so mounting the two E2B adapters places their mutable work in the same sandbox. This boundary does not move the harness process, Cordis objects, model calls, agent/session state, session persistence, skills, higher-level protocol state, or E2B SDK buffers. The [portable execution-world decision](../../.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md) owns both the generic composition and this POC boundary. diff --git a/packages/e2b/README.zh.md b/packages/e2b/README.zh.md index d941de8da2..de068c16d4 100644 --- a/packages/e2b/README.zh.md +++ b/packages/e2b/README.zh.md @@ -10,6 +10,6 @@ | [`fs-e2b`](fs-e2b/README.md)(`@deepseek-ai/dsh-fs-e2b`) | `ctx.fs` | 通过 E2B Filesystem API 实现文件系统 seam | | [`subprocess-e2b`](subprocess-e2b/README.md)(`@deepseek-ai/dsh-subprocess-e2b`) | `ctx.subprocess` | 通过 E2B Commands 与 PTY API 实现可执行文件查找、受管进程组与 stdio、远程 spill 文件及终端会话 | -现有的 [`dsh-bash-local`](../bash/bash-local/README.md)、[`dsh-pty-local`](../pty/pty-local/README.md) 和 [`dsh-lsp-local`](../lsp/lsp-local/README.md) 无需 E2B 专用 fork。它们把执行环境中的所有操作委托给 `ctx.fs` 和 `ctx.subprocess`,因此挂载这两个 E2B 适配器后,它们执行的可变操作都发生在同一个沙箱内。基于 worker 的代码运行时仍在宿主本地运行,只会通过其程序调用的工具触达该执行世界。 +现有的 [`dsh-bash-local`](../bash/bash-local/README.md)、[`dsh-pty-local`](../pty/pty-local/README.md) 和 [`dsh-lsp-local`](../lsp/lsp-local/README.md) 无需 E2B 专用 fork。它们把执行环境中的所有操作委托给 `ctx.fs` 和 `ctx.subprocess`,因此挂载这两个 E2B 适配器后,它们执行的可变操作都发生在同一个沙箱内。 该边界不会迁移 harness 进程、Cordis 对象、模型调用、agent(智能体)/会话状态、会话持久化、skill(技能)、更高层协议状态或 E2B SDK 缓冲。[可移植执行世界决策](../../.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md)同时界定通用组合和此 POC 边界。 diff --git a/packages/e2b/e2b/README.i18n.yaml b/packages/e2b/e2b/README.i18n.yaml index 0256b3ccfe..3587067d31 100644 --- a/packages/e2b/e2b/README.i18n.yaml +++ b/packages/e2b/e2b/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/e2b/e2b/README.md -README.md: 52f99178ce8c06e61c13ea9364769ac887db4410 -README.zh.md: 98ba7e79841f548ff6465fda4bc37170c17bc6e0 +README.md: 7ade7c3d6522d8fa6d54d7011766238451b17c9a +README.zh.md: b683f33d2211106cf422e780b2b37904b0c640ad diff --git a/packages/e2b/e2b/README.md b/packages/e2b/e2b/README.md index 52f99178ce..7ade7c3d65 100644 --- a/packages/e2b/e2b/README.md +++ b/packages/e2b/e2b/README.md @@ -38,7 +38,7 @@ No direct invalidation; this package does not contribute request tokens. ## Known Limitations and Deferred Work -- **This is not a whole-harness runtime** — Cordis services, the worker-backed Code Runtime, agent/session state, session logs, LLM requests, skills, and SDK-side buffers stay in the host process. +- **This is not a whole-harness runtime** — Cordis services, agent/session state, session logs, LLM requests, skills, and SDK-side buffers stay in the host process. - **Sandbox state is ephemeral** — disposal and timeout delete the sandbox; reconnect, pause/leave retention, templates, volumes, and snapshots are outside this POC. - **No deployment platform is configured** — network policy, host-workspace synchronization, and sandbox discovery are outside this POC. - **`cwd` is a resolution convention, not containment** — adapters and commands can address other sandbox paths; E2B network access retains the base image's policy. diff --git a/packages/e2b/e2b/README.zh.md b/packages/e2b/e2b/README.zh.md index 98ba7e7984..b683f33d22 100644 --- a/packages/e2b/e2b/README.zh.md +++ b/packages/e2b/e2b/README.zh.md @@ -38,7 +38,7 @@ ## 已知限制与延后工作 -- **这不是完整的 harness 运行时**:Cordis 服务、基于 worker 的代码运行时、agent(智能体)/会话状态、会话日志、LLM(大语言模型)请求、skill(技能)和 SDK 侧缓冲仍留在宿主进程中。 +- **这不是完整的 harness 运行时**:Cordis 服务、agent(智能体)/会话状态、会话日志、LLM(大语言模型)请求、skill(技能)和 SDK 侧缓冲仍留在宿主进程中。 - **沙箱状态是短暂的**:资源释放和超时都会删除沙箱;重新连接、pause/leave 保留、模板、卷和快照均不在本 POC 范围内。 - **没有配置部署平台**:网络策略、宿主工作区同步和沙箱发现均不在本 POC 范围内。 - **`cwd` 是解析约定,而不是包含边界**:适配器和命令可以访问沙箱中的其他路径;E2B 网络访问也继续采用基础镜像的策略。 From f5866fc2023c2c190f61229a8da4aaf917e55a4a Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Sun, 2 Aug 2026 15:38:00 +0800 Subject: [PATCH 100/108] fix(e2b): address review round on cadence config, disposal, and SDK edge cases - subprocess-e2b: the 20 ms remote poll cadence becomes a validated pollMs Config field (each tick is one control-plane request); the README documents the latency-versus-request-count trade. - subprocess-e2b: extract src/remote.ts owning asError, signalOpts, commandOpts, delay, waitTick, and one tolerant signalRemoteGroups shared by the pgid-keyed process ladder and sid-keyed terminal ladder, so the two teardown paths keep identical error tolerance. - subprocess-e2b: service disposal aggregates sibling cleanup failures into one AggregateError instead of discarding all but the first. - subprocess-e2b: waitForProcessGroupId refuses published group ids <= 1, so a same-UID rewrite of the pid file cannot aim termination at kill -- -1; README documents the same-UID control-state limitation. - subprocess-e2b: drain-grace expiry now releases an inherited-output E2B callback blocked on host backpressure before disconnecting, so the SDK settlement cannot stay pinned behind an unread host stream. - subprocess-e2b: spawn/spawnTerminal stop validating typed spec fields (trust-TypeScript rule; pty-local validates its config before specs exist); resolveExecutable rejects separator-containing relative paths per the seam contract; terminal setups tracked as a Set of records. - subprocess-e2b: PTY output push-without-backpressure is a documented contract (flowing consumer folds bytes; paused consumer buffers). - fs-e2b: streamText normalizes the pinned SDK's empty-file '' return into an empty stream instead of throwing on getReader(). - e2b overlays: comment the one-world cwd invariant across e2b.cwd, workspaceRoot, and bash-local's implicit default workdir. --- docs/config-catalog.md | 14 +++ examples/headless-agent/e2b.cordis.yml | 6 ++ .../tests/fixtures/e2b/e2b/cordis.yml | 3 + packages/e2b/fs-e2b/src/index.ts | 8 +- packages/e2b/fs-e2b/tests/filesystem.spec.ts | 13 ++- packages/e2b/subprocess-e2b/README.i18n.yaml | 4 +- packages/e2b/subprocess-e2b/README.md | 13 ++- packages/e2b/subprocess-e2b/README.zh.md | 13 ++- packages/e2b/subprocess-e2b/package.json | 3 + packages/e2b/subprocess-e2b/src/index.ts | 77 ++++++++++----- packages/e2b/subprocess-e2b/src/process.ts | 77 +++++---------- packages/e2b/subprocess-e2b/src/remote.ts | 97 +++++++++++++++++++ packages/e2b/subprocess-e2b/src/terminal.ts | 64 ++++-------- .../subprocess-e2b/tests/subprocess.spec.ts | 72 +++++++++++++- .../e2b/subprocess-e2b/tests/terminal.spec.ts | 14 ++- 15 files changed, 339 insertions(+), 139 deletions(-) create mode 100644 packages/e2b/subprocess-e2b/src/remote.ts diff --git a/docs/config-catalog.md b/docs/config-catalog.md index b8a49a7ba2..977921db07 100644 --- a/docs/config-catalog.md +++ b/docs/config-catalog.md @@ -1813,6 +1813,20 @@ export interface Config { Source: [`packages/subagent/subagent-spawn/src/index.ts:25`](../packages/subagent/subagent-spawn/src/index.ts) +## `@deepseek-ai/dsh-subprocess-e2b` + +Requires: `e2b` + +```ts config-catalog +/** Configuration for the E2B subprocess adapter. */ +export interface Config { + /** Remote status/liveness poll cadence in milliseconds; each tick is one control-plane request. */ + pollMs?: number +} +``` + +Source: [`packages/e2b/subprocess-e2b/src/index.ts:24`](../packages/e2b/subprocess-e2b/src/index.ts) + ## `@deepseek-ai/dsh-system-prompt` ```ts config-catalog diff --git a/examples/headless-agent/e2b.cordis.yml b/examples/headless-agent/e2b.cordis.yml index a65b0baf81..61bdbc8aa7 100644 --- a/examples/headless-agent/e2b.cordis.yml +++ b/examples/headless-agent/e2b.cordis.yml @@ -1,6 +1,12 @@ # POC overlay: keep the advanced headless agent and model-facing tools, but # place its filesystem and process substrate in one short-lived E2B sandbox; # the generic Bash, PTY, and LSP consumers compose above them. +# +# One-world invariant: e2b.cwd, sandbox-policy.workspaceRoot, and bash-local's +# default workdir (implicit host process.cwd()) must all name the same remote +# directory. Only e2b.cwd is created at sandbox open; dropping its !!js line +# falls back to /home/user/workspace while Bash and PTY keep targeting the +# host path, so every tool call fails with a remote spawn error. - id: base name: '@cordisjs/plugin-include' config: diff --git a/examples/headless-agent/tests/fixtures/e2b/e2b/cordis.yml b/examples/headless-agent/tests/fixtures/e2b/e2b/cordis.yml index 3a6deb7e49..e6fd968779 100644 --- a/examples/headless-agent/tests/fixtures/e2b/e2b/cordis.yml +++ b/examples/headless-agent/tests/fixtures/e2b/e2b/cordis.yml @@ -1,3 +1,6 @@ +# One-world invariant (same pairing as examples/headless-agent/e2b.cordis.yml): +# e2b.cwd and sandbox-policy.workspaceRoot must name the same remote directory, +# which is also bash-local's implicit default workdir. - id: e2b name: '@deepseek-ai/dsh-e2b' config: diff --git a/packages/e2b/fs-e2b/src/index.ts b/packages/e2b/fs-e2b/src/index.ts index 3b0c835d8c..84c9ac7868 100644 --- a/packages/e2b/fs-e2b/src/index.ts +++ b/packages/e2b/fs-e2b/src/index.ts @@ -232,7 +232,13 @@ export class E2BFileSystem extends FileSystem { await this.requireRegular(target, signal) let stream: ReadableStream try { - stream = await sandbox.files.read(String(target.targetKey), { format: 'stream', ...signalOpts(signal) }) + // The pinned SDK's stream overload lies for empty files: content-length 0 + // returns '' instead of a ReadableStream. + const read = await sandbox.files.read(String(target.targetKey), { format: 'stream', ...signalOpts(signal) }) as + ReadableStream | string + stream = typeof read === 'string' + ? new ReadableStream({ start(controller) { controller.close() } }) + : read } catch (error: unknown) { throw mapError(error, 'read', target.displayPath, signal) } diff --git a/packages/e2b/fs-e2b/tests/filesystem.spec.ts b/packages/e2b/fs-e2b/tests/filesystem.spec.ts index 618472a6b8..9cad00b6eb 100644 --- a/packages/e2b/fs-e2b/tests/filesystem.spec.ts +++ b/packages/e2b/fs-e2b/tests/filesystem.spec.ts @@ -149,7 +149,7 @@ class FakeRemote { } return this.info(path) }, - read: async (path: string, options: { format: 'bytes' | 'stream'; signal?: AbortSignal }): Promise> => { + read: async (path: string, options: { format: 'bytes' | 'stream'; signal?: AbortSignal }): Promise | string> => { this.checkAbort(options) if (this.nextReadError !== undefined) { const error = this.nextReadError @@ -158,6 +158,8 @@ class FakeRemote { } const data = this.followed(path).node.data if (options.format === 'bytes') return data.slice() + // Pinned-SDK fidelity: a content-length-0 response returns '' even in stream format. + if (data.length === 0 && this.streamChunks === undefined) return '' const chunks = this.streamChunks ?? [data.slice()] return new ReadableStream({ start: (controller) => { @@ -375,6 +377,15 @@ describe('E2BFileSystem identity, metadata, and reads', () => { expect(initiallyBuffered).toBe('€') }) + it('streams an empty file even though the pinned SDK returns a non-stream value', async () => { + const remote = new FakeRemote() + remote.file('/workspace/empty.txt', '') + const { fs } = await setup(remote) + let streamed = '' + for await (const chunk of await fs.streamText(await fs.resolve('empty.txt'))) streamed += chunk + expect(streamed).toBe('') + }) + it('cancels a remote stream when its consumer stops early', async () => { const remote = new FakeRemote() remote.file('/workspace/text.txt', 'ab') diff --git a/packages/e2b/subprocess-e2b/README.i18n.yaml b/packages/e2b/subprocess-e2b/README.i18n.yaml index 0a18bf24a1..d0ce148746 100644 --- a/packages/e2b/subprocess-e2b/README.i18n.yaml +++ b/packages/e2b/subprocess-e2b/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/e2b/subprocess-e2b/README.md -README.md: 3bb9802c3afa41f3949fceeb73e76ea5cebce744 -README.zh.md: 520f0ea1cde18ed523ba80015d3d4700730aba7a +README.md: 926d4f22f8e96daa103c236121d35e461290221a +README.zh.md: d1634b6d1a701f7f4815135b4ec6bb86ad1a8c87 diff --git a/packages/e2b/subprocess-e2b/README.md b/packages/e2b/subprocess-e2b/README.md index 3bb9802c3a..926d4f22f8 100644 --- a/packages/e2b/subprocess-e2b/README.md +++ b/packages/e2b/subprocess-e2b/README.md @@ -2,16 +2,22 @@ English | [中文](README.zh.md) -E2B implementation of the [`@deepseek-ai/dsh-subprocess`](../../subprocess/subprocess/README.md) seam. It has no config: load [`@deepseek-ai/dsh-e2b`](../e2b/README.md) first, then this service in place of `dsh-subprocess-local`. Existing Bash, PTY, and LSP consumers then execute in the shared remote sandbox without E2B-specific capability packages. +E2B implementation of the [`@deepseek-ai/dsh-subprocess`](../../subprocess/subprocess/README.md) seam. Load [`@deepseek-ai/dsh-e2b`](../e2b/README.md) first, then this service in place of `dsh-subprocess-local`. Existing Bash, PTY, and LSP consumers then execute in the shared remote sandbox without E2B-specific capability packages. + +## Configuration + +| Key | Default | Meaning | +| --- | --- | --- | +| `pollMs` | `20` | Remote status/liveness poll cadence in milliseconds; each tick is one control-plane request, so a larger value trades exit-observation latency for fewer requests. | ## Behavior - **Asynchronous remote start** — the synchronous seam returns a handle immediately while `Sandbox.commands.run(..., { background: true })` starts remotely. `pid` is `-1` until the wrapper publishes and the adapter validates its process-group id; stdin and ordinary observation wait for that publication. An owned startup signal aborts environment and private-state preparation before allocation; once allocation begins, cancellation waits for a provisional SDK handle it can clean. -- **Execution-world coordinates** — `cwd` and private `runtimeRoot` come from the shared owner; executable lookup verifies absolute paths or resolves a bare name against the sandbox PATH plus explicit overrides. +- **Execution-world coordinates** — `cwd` and private `runtimeRoot` come from the shared owner; executable lookup verifies absolute paths or resolves a bare name against the sandbox PATH plus explicit overrides, and rejects relative paths containing separators like every subprocess provider. - **Linux process groups** — a quoted wrapper starts each argv under `exec setsid --wait` and records its actual process-group id plus private status files beneath `ctx.e2b.runtimeRoot/processes`. The handle waits for that file instead of treating the SDK command PID as its published identity. Termination signals the negative recorded id with `SIGTERM`, waits the caller's `graceMs`, then escalates to `SIGKILL` and the SDK kill fallback; TERM delivery or probe failures also force that escalation. Process-table probes treat groups containing only zombie or dead entries as quiescent. Force cleanup succeeds only after a bounded probe finds the group empty; otherwise `waitForExit()` exposes a retryable failure, while proven quiescence makes later termination a no-op. Publication and monitoring failures apply the same cleanup transaction before rejecting. Service disposal rejects new starts, terminates and joins every retained process group, then awaits SDK settlement and private cleanup before the sandbox owner disposes. - **Environment boundary** — one trusted control-shell probe resolves the sandbox user's login home from its passwd entry and transports the sandbox environment as base64 ASCII for one strict UTF-8 decode; the wrapper then removes ambient `DSH_*` and credential-shaped (`*KEY*`, `*SECRET*`, `*TOKEN*`) names and restores every valid `spec.env` entry as an explicit caller opt-in. Empty names, `=`, and NUL framing violations reject before launch. Subsequent E2B command and PTY login shells receive a fresh randomized root-level `HOME` plus empty overrides for every scrubbed ambient name before user profiles can run; the requested argv receives the serialized environment afterward without changing the sandbox user's umask. Host ambient variables never enter the sandbox implicitly. Private environment files are removed after consumption, and failed command or terminal setup removes its private state before rejecting. - **Stdio projection** — the remote wrapper branches raw bytes into optional bounded spill files, frames each live chunk as newline-delimited base64 ASCII, and the host incrementally restores bytes across arbitrary SDK callback boundaries. Pipe mode writes those bytes to host Node streams; inherit mode writes them to the harness process streams; collect mode retains a bounded host tail with offset reads. The wrapper publishes the direct command status before waiting for inherited writers. For collect or inherit output, the adapter disconnects an incomplete SDK stream after `graceMs`, withholds its partial spill, and returns that status while retaining the remote group for `waitForExit()` and termination. Natural raw-pipe completion instead awaits lossless transport and preserves backpressure; explicit termination destroys the host pipes and releases blocked output before remote cleanup. Batch and streaming stdin use the SDK handle. -- **Terminal sessions** — `spawnTerminal()` uses E2B's byte PTY API, installs the exact argv and scrubbed environment through private mode-`0600` files, reports the foreground process group, sends real signals, and tears down every live group in the remote terminal session through one retryable awaited `terminate()`; termination rejects new handle operations, aborts and joins in-flight writes, inspections, and signals, and treats zombie-only groups as quiescent. A private random output boundary discards the E2B bootstrap shell's prompt and echoed runner command while preserving every requested-process byte, including its first prompt. PTY allocation is awaited through handle publication before cancellation is observed, so owned rollback can clean the published handle. Setup and teardown own the private state transaction, abort pending setup during service disposal, and fence publication; sandbox disposal or timeout bounds a setup rollback that also fails. Prompt detection, scrollback, readiness, and owner policy remain in `dsh-pty-local`. +- **Terminal sessions** — `spawnTerminal()` uses E2B's byte PTY API, installs the exact argv and scrubbed environment through private mode-`0600` files, reports the foreground process group, sends real signals, and tears down every live group in the remote terminal session through one retryable awaited `terminate()`; termination rejects new handle operations, aborts and joins in-flight writes, inspections, and signals, and treats zombie-only groups as quiescent. A private random output boundary discards the E2B bootstrap shell's prompt and echoed runner command while preserving every requested-process byte, including its first prompt. Terminal output is pushed to the handle's stream without awaiting host backpressure: a flowing consumer (the PTY backend attaches one at construction) folds bytes into its own bounded state, while a paused consumer buffers in host memory. PTY allocation is awaited through handle publication before cancellation is observed, so owned rollback can clean the published handle. Setup and teardown own the private state transaction, abort pending setup during service disposal, and fence publication; sandbox disposal or timeout bounds a setup rollback that also fails. Prompt detection, scrollback, readiness, and owner policy remain in `dsh-pty-local`. - **Sandbox disappearance** — `SandboxNotFoundError` during process or terminal liveness, termination, rollback, or disconnect proves the remote execution world cannot retain work, so cleanup treats it as quiescent; unrelated failures remain observable. The default E2B base image supplies the runtime and Bash/GNU utilities this adapter invokes: `node`, `bash`, `setsid`, `ps`, `awk`, `tr`, `env`, `base64`, `chmod`, `tee`, `head`, `rm`, `kill`, `id`, and `getent`. @@ -29,6 +35,7 @@ No direct invalidation; the named consumers own any request-prefix changes. - **The SDK still retains complete command output in host memory** — E2B `CommandHandle.stdout` and `.stderr` accumulate the base64 transport even when this adapter exposes bounded raw-byte tails, so the subprocess seam's normal host-memory bound is not achieved and transport retention is larger than the source stream. - **Synchronous-PID consumers are unsupported** — `pid` remains `-1` during remote startup; consumers that require a positive PID immediately, including the ACP child backend, cannot use this provider unchanged. - **Private state lives for the sandbox lifetime** — process directories and valid spill files remain under `.dsh-e2b` until the owner deletes the sandbox; this POC supplies no in-sandbox sweep. +- **Control state shares the sandbox user's UID** — E2B runs every command as the same default user, so `0700`/`0600` modes cannot isolate `.dsh-e2b` control files from concurrently running sandbox processes. A background process could rewrite `pid`/`exit-code` or read a not-yet-consumed `environment` file. The adapter validates published values and refuses group ids whose negative form is unsafe to signal (`<= 1`), but real isolation needs an E2B per-command user or an out-of-band control channel. - **Numeric process identities are not reuse-fenced** — E2B exposes numeric PID/PGID PTY input, signalling, and cleanup operations but no atomic identity-bound alternative. The adapter minimizes host round trips and live coverage exercises the reproducible stale-interrupt overlap; replacement is deferred until E2B adds an identity primitive or a failure demonstrates a narrower protocol. - **The initial environment probe inherits sandbox defaults** — E2B merges command overrides with default environment entries, so the probe cannot blank unknown credential-shaped names before enumerating them. A same-UID untrusted process already in the sandbox could inspect that short-lived control shell; this POC therefore does not support secrets in sandbox-default environment variables and requires an E2B replacement-environment primitive to close the gap. - **E2B exposes no signal fact** — an adapter-requested `SIGTERM` or `SIGKILL` is reported only when no wrapper-published direct exit code wins; every unrequested SDK exit remains an exit code, including values shaped like `128 + signal`. diff --git a/packages/e2b/subprocess-e2b/README.zh.md b/packages/e2b/subprocess-e2b/README.zh.md index 520f0ea1cd..d1634b6d1a 100644 --- a/packages/e2b/subprocess-e2b/README.zh.md +++ b/packages/e2b/subprocess-e2b/README.zh.md @@ -2,16 +2,22 @@ [English](README.md) | 中文 -[`@deepseek-ai/dsh-subprocess`](../../subprocess/subprocess/README.md) seam 的 E2B 实现。它没有配置:先加载 [`@deepseek-ai/dsh-e2b`](../e2b/README.md),再用本服务取代 `dsh-subprocess-local`。现有的 Bash、PTY 和 LSP 消费方随后会在共享远程沙箱中执行,无需 E2B 专用的功能包(package)。 +[`@deepseek-ai/dsh-subprocess`](../../subprocess/subprocess/README.md) seam 的 E2B 实现。先加载 [`@deepseek-ai/dsh-e2b`](../e2b/README.md),再用本服务取代 `dsh-subprocess-local`。现有的 Bash、PTY 和 LSP 消费方随后会在共享远程沙箱中执行,无需 E2B 专用的功能包(package)。 + +## 配置 + +| 键 | 默认值 | 含义 | +| --- | --- | --- | +| `pollMs` | `20` | 远程状态/存活轮询节奏(毫秒);每个 tick 是一次控制面请求,调大该值以牺牲退出观察延迟换取更少的请求。 | ## 行为 - **异步远程启动**:同步 seam 会立即返回一个句柄,同时由 `Sandbox.commands.run(..., { background: true })` 在远程启动进程。包装层发布进程组 ID 并由适配器完成验证之前,`pid` 为 `-1`;stdin 和常规观察会等待该发布。自有启动信号会在分配前中止环境和私有状态准备;分配开始后,取消会等待可清理的临时 SDK 句柄。 -- **执行世界坐标**:`cwd` 和私有 `runtimeRoot` 来自共享所有者;可执行文件查找会验证绝对路径,或根据沙箱 PATH 加显式覆盖来解析裸名称。 +- **执行世界坐标**:`cwd` 和私有 `runtimeRoot` 来自共享所有者;可执行文件查找会验证绝对路径,或根据沙箱 PATH 加显式覆盖来解析裸名称,并与所有 subprocess 提供方一致地拒绝含分隔符的相对路径。 - **Linux 进程组**:带引号保护的包装层会在 `exec setsid --wait` 下启动每组 argv,并在 `ctx.e2b.runtimeRoot/processes` 下记录实际进程组 ID 和私有状态文件。句柄会等待该文件,而不会把 SDK 命令 PID 当作已发布的身份。终止操作以记录的负数 ID 发送 `SIGTERM`,等待调用方的 `graceMs`,再升级到 `SIGKILL` 和 SDK kill 回退;TERM 信号发送或探测失败也会强制触发该升级。进程表探测会把仅含僵尸或已死亡条目的进程组视为完全停稳。强制清理只有在有界探测发现进程组为空后才算成功;否则 `waitForExit()` 会公开可重试的失败,而已证明的完全停稳会让后续终止操作不再执行任何动作。发布失败与监控失败都会在拒绝前执行同一清理事务。服务 dispose(资源释放)会拒绝新的启动请求、终止并等待每个保留进程组退出,再等待 SDK 结算和私有清理完成,之后沙箱所有者才会释放。 - **环境边界**:一次受信任的控制 shell 探测会从 passwd 条目解析沙箱用户的登录主目录,以 base64 ASCII 传输沙箱环境,再进行一次严格 UTF-8 解码;随后包装层移除环境中的 `DSH_*` 和形似凭据的名称(`*KEY*`、`*SECRET*`、`*TOKEN*`),并把每个有效的 `spec.env` 条目恢复为调用方显式选择。空名称、`=` 和违反 NUL 分帧规则的条目会在启动前被拒绝。在用户 profile 脚本运行前,此后的 E2B 命令 shell 与 PTY 登录 shell 会获得位于根目录下、全新随机生成的 `HOME`,并为每个被清理的环境变量名设置空值覆盖;之后,请求的 argv 会在不改变沙箱用户 umask 的前提下接收序列化环境。宿主环境变量绝不会隐式进入沙箱。私有环境文件在使用后会被删除;命令或终端设置失败时,会先删除其私有状态再拒绝。 - **stdio 投影**:远程包装层先把原始字节分流到可选的有界 spill 文件,再把每个实时分片编码为换行分隔的 base64 ASCII 帧;宿主会跨任意 SDK 回调边界增量恢复字节。pipe 模式把这些字节写入宿主 Node 流;inherit 模式把字节写入 harness 进程流;collect 模式保留有界的宿主尾部,并支持基于偏移量读取。包装层会在等待继承管道的写入方之前发布直接命令状态。对于 collect 或 inherit 输出,超过 `graceMs` 后,适配器会断开未完成的 SDK 流,不公开其中不完整的 spill,并返回该状态,同时保留远程进程组供 `waitForExit()` 和终止操作使用。原始 pipe 自然完成时,会等待无损传输完成并保留背压;显式终止则会销毁宿主 pipe,并在远程清理前释放受阻的输出写入。批量 stdin 和流式 stdin 都使用 SDK 句柄。 -- **终端会话**:`spawnTerminal()` 使用 E2B 的字节 PTY API,以 mode 为 `0600` 的私有文件传入原样 argv 与清理后的环境,报告前台进程组,发送真实信号,并通过一项可重试且须等待的 `terminate()` 清理远程终端会话中仍存活的每个进程组;终止会拒绝新的句柄操作,中止并等待在途写入、检查和信号操作结算,并把仅含僵尸进程的进程组视为已经完全停稳。私有随机输出边界会丢弃 E2B 引导 shell 的提示符和回显的 runner 命令,同时保留请求进程的每个字节,包括其第一个提示符。PTY 分配会一直等待到句柄发布后才观察取消,以便由承担清理责任的回滚清理已发布句柄。setup 与 teardown 负责私有状态事务,在服务 dispose 期间中止待处理的 setup 并阻止发布;若 setup 回滚也失败,则由沙箱 dispose 或超时约束其存活时间。提示符检测、scrollback、就绪状态与所有者策略仍归 `dsh-pty-local` 所有。 +- **终端会话**:`spawnTerminal()` 使用 E2B 的字节 PTY API,以 mode 为 `0600` 的私有文件传入原样 argv 与清理后的环境,报告前台进程组,发送真实信号,并通过一项可重试且须等待的 `terminate()` 清理远程终端会话中仍存活的每个进程组;终止会拒绝新的句柄操作,中止并等待在途写入、检查和信号操作结算,并把仅含僵尸进程的进程组视为已经完全停稳。私有随机输出边界会丢弃 E2B 引导 shell 的提示符和回显的 runner 命令,同时保留请求进程的每个字节,包括其第一个提示符。终端输出推入句柄流时不等待宿主背压:流动的消费方(PTY 后端在构造时就挂上一个)把字节折叠进自身的有界状态,而暂停的消费方会在宿主内存中缓冲。PTY 分配会一直等待到句柄发布后才观察取消,以便由承担清理责任的回滚清理已发布句柄。setup 与 teardown 负责私有状态事务,在服务 dispose 期间中止待处理的 setup 并阻止发布;若 setup 回滚也失败,则由沙箱 dispose 或超时约束其存活时间。提示符检测、scrollback、就绪状态与所有者策略仍归 `dsh-pty-local` 所有。 - **沙箱消失**:在进程或终端的存活探测、终止、回滚或断开连接期间出现 `SandboxNotFoundError`,证明远程执行环境无法保留工作,因此清理会将其视为完全停稳;其他故障仍可观察。 E2B 默认基础镜像提供该适配器调用的运行时和 Bash/GNU 工具:`node`、`bash`、`setsid`、`ps`、`awk`、`tr`、`env`、`base64`、`chmod`、`tee`、`head`、`rm`、`kill`、`id` 和 `getent`。 @@ -29,6 +35,7 @@ E2B 默认基础镜像提供该适配器调用的运行时和 Bash/GNU 工具: - **SDK 仍会在宿主内存中保留完整命令输出**:即使本适配器公开的是有界原始字节尾部,E2B `CommandHandle.stdout` 和 `.stderr` 仍会累积 base64 传输内容,因此无法达到进程管理 seam 通常提供的宿主内存边界,而且传输保留量大于源数据流。 - **不支持需要同步 PID 的消费方**:远程启动期间,`pid` 保持为 `-1`;包括 ACP 子进程后端在内,要求立即获得正 PID 的消费方无法原样使用本提供方。 - **私有状态随沙箱生命周期存在**:进程目录和有效的 spill 文件会留在 `.dsh-e2b` 下,直到所有者删除沙箱;本 POC 不提供沙箱内清理。 +- **控制状态与沙箱用户同 UID**:E2B 以同一默认用户运行每条命令,因此 `0700`/`0600` 权限无法把 `.dsh-e2b` 控制文件与并发运行的沙箱进程隔离开。后台进程可以改写 `pid`/`exit-code`,或读取尚未被消费的 `environment` 文件。适配器会验证已发布的值,并拒绝取负后不安全的进程组 ID(`<= 1`),但真正的隔离需要 E2B 提供按命令用户或带外控制通道。 - **数值进程身份没有复用围栏**:E2B 公开基于数值 PID/PGID 的 PTY 输入、信号发送和清理操作,却没有与身份原子绑定的替代方案。适配器会尽量减少宿主往返,真实环境测试会覆盖可复现的陈旧中断重叠;在 E2B 新增身份原语,或实际故障证明需要更窄的协议之前,替代方案会继续延后。 - **初始环境探测会继承沙箱默认值**:E2B 会把命令覆盖与默认环境条目合并,因此探测无法在枚举未知且形似凭据的名称之前将它们置空。一个已在沙箱内运行的同 UID 不可信进程可以检查该短时存在的控制 shell;因此,该 POC 不支持把 secret 放入沙箱默认环境变量,需要 E2B 的替换环境原语才能弥合该缺口。 - **E2B 不公开信号事实**:适配器请求的 `SIGTERM` 或 `SIGKILL` 只有在包装层发布的直接退出码没有胜出时才报告为信号;其他未请求的 SDK 退出始终保留为退出码,包括形似 `128 + signal` 的值。 diff --git a/packages/e2b/subprocess-e2b/package.json b/packages/e2b/subprocess-e2b/package.json index d73607d1f3..fa96e17293 100644 --- a/packages/e2b/subprocess-e2b/package.json +++ b/packages/e2b/subprocess-e2b/package.json @@ -32,6 +32,9 @@ "@deepseek-ai/dsh-subprocess": "^0.0.1", "cordis": "^4.0.0-rc.7" }, + "dependencies": { + "schemastery": "^3.18.0" + }, "devDependencies": { "@deepseek-ai/dsh-e2b": "workspace:^", "@deepseek-ai/dsh-invariants": "workspace:^", diff --git a/packages/e2b/subprocess-e2b/src/index.ts b/packages/e2b/subprocess-e2b/src/index.ts index 15e7feccdd..38b1d32465 100644 --- a/packages/e2b/subprocess-e2b/src/index.ts +++ b/packages/e2b/subprocess-e2b/src/index.ts @@ -7,6 +7,7 @@ import { randomUUID } from 'node:crypto' import { posix } from 'node:path' import { Context } from 'cordis' +import z from 'schemastery' import { SubprocessService } from '@deepseek-ai/dsh-subprocess' import type { SubprocessHandle, @@ -16,30 +17,53 @@ import type { } from '@deepseek-ai/dsh-subprocess' import { e2bControlEnvs, quoteE2BShellArg } from '@deepseek-ai/dsh-e2b' import { E2BSubprocessHandle } from './process.ts' +import { asError, signalOpts } from './remote.ts' import { spawnE2BTerminal } from './terminal.ts' -function signalOpts(signal: AbortSignal | undefined): { signal?: AbortSignal } { - return signal === undefined ? {} : { signal } +/** Configuration for the E2B subprocess adapter. */ +export interface Config { + /** Remote status/liveness poll cadence in milliseconds; each tick is one control-plane request. */ + pollMs?: number +} + +interface SchemaResolvedConfig extends Config { + pollMs: number +} + +interface TerminalSetup { + done: Promise + controller: AbortController } /** E2B command manager registered as `ctx.subprocess`. */ export class E2BSubprocessService extends SubprocessService { static inject = ['e2b'] + static Config: z = z.object({ + pollMs: z.number().default(20), + }) + private readonly live = new Set() private readonly terminals = new Set() - private readonly terminalSetups = new Map, AbortController>() + private readonly terminalSetups = new Set() + private readonly pollMs: number private disposing = false /** Create the E2B subprocess service and bind its disposal policy. */ - constructor(ctx: Context) { + constructor(ctx: Context, config: Config) { super(ctx) + // Schemastery fills pollMs before construction; the type does not encode that step. + const { pollMs } = config as SchemaResolvedConfig + if (!Number.isSafeInteger(pollMs) || pollMs <= 0) { + throw new Error('subprocess-e2b: pollMs must be a positive safe integer') + } + this.pollMs = pollMs ctx.effect(() => async () => { this.disposing = true - for (const controller of this.terminalSetups.values()) { - controller.abort(new Error('subprocess-e2b: service disposed during terminal setup')) + for (const setup of this.terminalSetups) { + setup.controller.abort(new Error('subprocess-e2b: service disposed during terminal setup')) } - await Promise.all([...this.terminalSetups.keys()]) + await Promise.all([...this.terminalSetups].map(setup => setup.done)) const handles = [...this.live] const terminals = [...this.terminals] const pending: Promise[] = [] @@ -54,9 +78,11 @@ export class E2BSubprocessService extends SubprocessService { pending.push(terminal.terminate().then(() => { this.terminals.delete(terminal) })) } const outcomes = await Promise.allSettled(pending) - for (const outcome of outcomes) { - if (outcome.status === 'rejected') throw outcome.reason - } + const failures = outcomes.flatMap(outcome => outcome.status === 'rejected' + ? [outcome.reason as unknown] + : []) + if (failures.length === 1) throw asError(failures[0]) + if (failures.length > 1) throw new AggregateError(failures, 'subprocess-e2b: teardown failed') }, 'e2b subprocess teardown') } @@ -77,6 +103,11 @@ export class E2BSubprocessService extends SubprocessService { signal?.throwIfAborted() return command } + if (command.includes('/')) { + throw new Error( + `subprocess-e2b: command ${JSON.stringify(command)} is a relative path; use an absolute path or a bare PATH name`, + ) + } const path = env?.PATH const prefix = path === undefined ? '' : `PATH=${quoteE2BShellArg(path)} ` const result = await sandbox.commands.run( @@ -88,6 +119,7 @@ export class E2BSubprocessService extends SubprocessService { if (executable.includes('\n') || (!posix.isAbsolute(executable) && !executable.includes('/'))) { throw new Error(`subprocess-e2b: executable ${JSON.stringify(command)} did not resolve to one absolute path`) } + // A relative result comes from a relative PATH entry; the lookup ran with the shared cwd. return posix.resolve(this.ctx.e2b.cwd, executable) } @@ -98,14 +130,11 @@ export class E2BSubprocessService extends SubprocessService { if (program === undefined || program.length === 0) { throw new Error('invalid argv: expected a non-empty program name at argv[0]') } - if (!Number.isFinite(spec.graceMs) || spec.graceMs <= 0) { - throw new Error('subprocess-e2b: graceMs must be a positive finite number') - } if (spec.signal?.aborted === true) { throw new Error(`aborted before spawn: ${String(spec.signal.reason)}`) } const stateDir = posix.join(this.ctx.e2b.runtimeRoot, 'processes', randomUUID()) - const handle = new E2BSubprocessHandle(this.ctx.e2b, spec, stateDir) + const handle = new E2BSubprocessHandle(this.ctx.e2b, spec, stateDir, this.pollMs) this.live.add(handle) const release = async (): Promise => { await handle.waitForExit() @@ -124,24 +153,20 @@ export class E2BSubprocessService extends SubprocessService { if (program === undefined || program.length === 0) { throw new Error('subprocess-e2b: terminal argv must contain a program') } - for (const [name, value] of [['rows', spec.rows], ['cols', spec.cols], ['graceMs', spec.graceMs]] as const) { - if (!Number.isSafeInteger(value) || value <= 0) { - throw new Error(`subprocess-e2b: terminal ${name} must be a positive safe integer`) - } - } spec.signal?.throwIfAborted() const stateDir = posix.join(this.ctx.e2b.runtimeRoot, 'terminals', randomUUID()) - const setup = Promise.withResolvers() - const setupController = new AbortController() + const done = Promise.withResolvers() + const setup: TerminalSetup = { done: done.promise, controller: new AbortController() } const setupSignal = spec.signal === undefined - ? setupController.signal - : AbortSignal.any([spec.signal, setupController.signal]) - this.terminalSetups.set(setup.promise, setupController) + ? setup.controller.signal + : AbortSignal.any([spec.signal, setup.controller.signal]) + this.terminalSetups.add(setup) try { const terminal = await spawnE2BTerminal( this.ctx.e2b, { ...spec, signal: setupSignal }, stateDir, + this.pollMs, ) this.terminals.add(terminal) // oxlint-disable-next-line typescript/no-unnecessary-condition -- Remote allocation yields to disposal. @@ -159,8 +184,8 @@ export class E2BSubprocessService extends SubprocessService { }) return terminal } finally { - this.terminalSetups.delete(setup.promise) - setup.resolve() + this.terminalSetups.delete(setup) + done.resolve() } } } diff --git a/packages/e2b/subprocess-e2b/src/process.ts b/packages/e2b/subprocess-e2b/src/process.ts index c0aceda334..feb21a70bc 100644 --- a/packages/e2b/subprocess-e2b/src/process.ts +++ b/packages/e2b/subprocess-e2b/src/process.ts @@ -21,8 +21,8 @@ import type { import type E2BSandboxService from '@deepseek-ai/dsh-e2b' import { bootstrapEnvironment, readRemoteEnvironment, serializeRemoteEnvironment } from './environment.ts' import { E2BBase64Decoder, E2B_OUTPUT_COMPLETE_FRAME, E2BOutputReader } from './output.ts' +import { asError, commandOpts, signalRemoteGroups, waitTick } from './remote.ts' -const GROUP_POLL_MS = 20 const OUTPUT_ENCODER_SOURCE = [ '(async () => {', ' for await (const chunk of process.stdin) {', @@ -48,10 +48,6 @@ function isValidProcessId(value: number): boolean { return Number.isSafeInteger(value) && value > 0 } -function asError(error: unknown): Error { - return error instanceof Error ? error : new Error(String(error)) -} - class DeferredStdin extends Writable { constructor(private readonly ready: Promise) { super({ decodeStrings: false }) @@ -142,28 +138,6 @@ function commandText(spec: SubprocessSpawnSpec, paths: RemotePaths): string { return bootstrap } -function commandOpts( - envs: Record, - signal: AbortSignal | undefined, -): { envs: Record; signal?: AbortSignal } { - return { envs: e2bControlEnvs(envs), ...(signal === undefined ? {} : { signal }) } -} - -function waitTick(signal?: AbortSignal): Promise { - if (signal?.aborted === true) return Promise.resolve(false) - return new Promise((resolve) => { - const timer = setTimeout(() => { - signal?.removeEventListener('abort', onAbort) - resolve(true) - }, GROUP_POLL_MS) - const onAbort = (): void => { - clearTimeout(timer) - resolve(false) - } - signal?.addEventListener('abort', onAbort, { once: true }) - }) -} - const WAIT_ABORTED = Symbol('wait aborted') function waitWithSignal(promise: Promise, signal: AbortSignal | undefined): Promise { @@ -194,6 +168,8 @@ export class E2BSubprocessHandle implements SubprocessHandle { private readonly stdoutDecoder = new E2BBase64Decoder() private readonly stderrDecoder = new E2BBase64Decoder() private readonly terminationController = new AbortController() + /** Releases output waits that survive the command outcome, so blocked SDK callbacks settle. */ + private readonly outputReleased = new AbortController() private readonly stdoutReader: E2BOutputReader | undefined private readonly stderrReader: E2BOutputReader | undefined private readonly paths: RemotePaths @@ -212,11 +188,13 @@ export class E2BSubprocessHandle implements SubprocessHandle { * @param runtime - Shared E2B sandbox owner. * @param spec - Fully resolved subprocess request. * @param stateDir - Remote directory retaining process identity, status, and valid spills. + * @param pollMs - Remote status/liveness poll cadence. */ constructor( private readonly runtime: E2BSandboxService, private readonly spec: SubprocessSpawnSpec, readonly stateDir: string, + private readonly pollMs = 20, ) { this.paths = { pid: posix.join(stateDir, 'pid'), @@ -318,7 +296,7 @@ export class E2BSubprocessHandle implements SubprocessHandle { const processGroupId = this.remotePid > 0 ? this.remotePid : handle.pid while (await this.groupAlive(sandbox, processGroupId, signal)) { this.throwTerminationFailure() - if (!await waitTick(signal)) return false + if (!await waitTick(this.pollMs, signal)) return false } this.throwTerminationFailure() if (signal?.aborted === true) return false @@ -483,19 +461,21 @@ export class E2BSubprocessHandle implements SubprocessHandle { await new Promise((resolve, reject) => { const onDrain = (): void => { cleanup(); resolve() } const onClose = (): void => { cleanup(); resolve() } - const onTermination = (): void => { cleanup(); resolve() } + const onRelease = (): void => { cleanup(); resolve() } const onError = (error: Error): void => { cleanup(); reject(error) } const cleanup = (): void => { target.removeListener('drain', onDrain) target.removeListener('close', onClose) target.removeListener('error', onError) - this.terminationController.signal.removeEventListener('abort', onTermination) + this.terminationController.signal.removeEventListener('abort', onRelease) + this.outputReleased.signal.removeEventListener('abort', onRelease) } target.once('drain', onDrain) target.once('close', onClose) target.once('error', onError) - this.terminationController.signal.addEventListener('abort', onTermination, { once: true }) - if (this.terminationController.signal.aborted) onTermination() + this.terminationController.signal.addEventListener('abort', onRelease, { once: true }) + this.outputReleased.signal.addEventListener('abort', onRelease, { once: true }) + if (this.terminationController.signal.aborted || this.outputReleased.signal.aborted) onRelease() }) } @@ -514,9 +494,14 @@ export class E2BSubprocessHandle implements SubprocessHandle { if (!/^[1-9][0-9]*$/.test(value) || !Number.isSafeInteger(pid)) { throw new Error(`subprocess-e2b: remote wrapper published invalid process-group id ${JSON.stringify(value)}`) } + // A same-UID sandbox process can rewrite this file; refuse ids whose + // negative form addresses every process (`kill -- -1`) or init's group. + if (pid <= 1) { + throw new Error(`subprocess-e2b: unsafe published process-group id ${pid}`) + } return pid } - const settled = await Promise.race([commandSettled, waitTick().then(() => false)]) + const settled = await Promise.race([commandSettled, waitTick(this.pollMs).then(() => false)]) if (settled) throw new Error('subprocess-e2b: remote command exited before publishing its process-group id') } } @@ -545,13 +530,16 @@ export class E2BSubprocessHandle implements SubprocessHandle { this.outputDrainExpired = true this.stdoutReader?.invalidateSpill() this.stderrReader?.invalidateSpill() + // Release inherited-output waits so a callback blocked on host + // backpressure cannot keep the disconnected SDK settlement pending. + this.outputReleased.abort(new Error('subprocess-e2b: output drain grace expired')) await handle.disconnect() return { exitCode, signal: null } } if (completed !== undefined) return this.commandOutcome(completed) // TODO(e2b-status-watch): Replace collect/inherit control-plane polling // when E2B can observe direct-command exit independently of descendant-held output. - completed = await Promise.race([settlement, waitTick().then(() => undefined)]) + completed = await Promise.race([settlement, waitTick(this.pollMs).then(() => undefined)]) } } @@ -622,7 +610,7 @@ export class E2BSubprocessHandle implements SubprocessHandle { private async terminateGroup(sandbox: Sandbox, handle: CommandHandle, processGroupId: number): Promise { this.terminationSignal = 'SIGTERM' try { - await this.signalGroup(sandbox, processGroupId, 'TERM') + await signalRemoteGroups(sandbox, this.controlEnvs, [processGroupId], 'TERM') if (await this.waitForGroupExit(sandbox, processGroupId)) { this.markQuiescent() return @@ -637,7 +625,7 @@ export class E2BSubprocessHandle implements SubprocessHandle { private async forceKillGroup(sandbox: Sandbox, handle: CommandHandle, processGroupId: number): Promise { try { - await this.signalGroup(sandbox, processGroupId, 'KILL') + await signalRemoteGroups(sandbox, this.controlEnvs, [processGroupId], 'KILL') } catch (_processGroupKillFailure) { // SDK kill and the final liveness probe remain independent cleanup paths. } @@ -654,7 +642,7 @@ export class E2BSubprocessHandle implements SubprocessHandle { const deadline = Date.now() + this.spec.graceMs while (await this.groupAlive(sandbox, processGroupId)) { if (Date.now() >= deadline) return false - await waitTick() + await waitTick(this.pollMs) } return true } @@ -663,21 +651,6 @@ export class E2BSubprocessHandle implements SubprocessHandle { if (this.terminationFailure !== undefined) throw this.terminationFailure } - private async signalGroup(sandbox: Sandbox, pid: number, signal: 'TERM' | 'KILL'): Promise { - // TODO(e2b-pgid-identity): Prefer an atomic identity-bound group signal if E2B adds one; - // a userspace identity precheck cannot close the numeric-PGID reuse race. - try { - await sandbox.commands.run( - `kill -${signal} -- -${pid}`, - commandOpts(this.controlEnvs, undefined), - ) - return true - } catch (error: unknown) { - if (error instanceof CommandExitError || error instanceof SandboxNotFoundError) return false - throw error - } - } - private async groupAlive(sandbox: Sandbox, pid: number, signal?: AbortSignal): Promise { const result = await sandbox.commands.run( `set -o pipefail; ps -eo pgid=,stat= | awk '$1 == ${pid} && $2 !~ /^[ZXx]/ { live=1 } END { if (live) print "live" }'`, diff --git a/packages/e2b/subprocess-e2b/src/remote.ts b/packages/e2b/subprocess-e2b/src/remote.ts new file mode 100644 index 0000000000..1937f54bf0 --- /dev/null +++ b/packages/e2b/subprocess-e2b/src/remote.ts @@ -0,0 +1,97 @@ +/** + * Shared remote-control helpers for the E2B subprocess adapter: SDK option + * shaping, poll ticks, and the one tolerant process-group signal used by both + * the ordinary-process and terminal teardown ladders. + */ + +import { CommandExitError, e2bControlEnvs, SandboxNotFoundError } from '@deepseek-ai/dsh-e2b' +import type { Sandbox } from '@deepseek-ai/dsh-e2b' + +/** + * Normalize an unknown rejection into an Error. + * @param error - Any thrown or rejected value. + * @returns The value itself when already an Error, else a stringified wrapper. + */ +export function asError(error: unknown): Error { + return error instanceof Error ? error : new Error(String(error)) +} + +/** + * Shape the optional-signal SDK options object. + * @param signal - Optional cancellation for one SDK request. + * @returns An options fragment that omits an undefined signal. + */ +export function signalOpts(signal: AbortSignal | undefined): { signal?: AbortSignal } { + return signal === undefined ? {} : { signal } +} + +/** + * Shape control-shell command options with the isolated HOME override. + * @param envs - Explicit environment entries for the control command. + * @param signal - Optional cancellation for the SDK request. + * @returns Options for `sandbox.commands.run` control invocations. + */ +export function commandOpts( + envs: Record, + signal?: AbortSignal, +): { envs: Record; signal?: AbortSignal } { + return { envs: e2bControlEnvs(envs), ...signalOpts(signal) } +} + +/** + * Resolve after one duration. + * @param ms - Milliseconds to wait. + * @returns Settles after the timeout. + */ +export function delay(ms: number): Promise { + return new Promise(resolve => setTimeout(resolve, ms)) +} + +/** + * Wait one poll interval or until the signal aborts. + * @param pollMs - Poll cadence in milliseconds. + * @param signal - Optional abort that ends the wait early. + * @returns `true` after a full tick, `false` when aborted first. + */ +export function waitTick(pollMs: number, signal?: AbortSignal): Promise { + if (signal?.aborted === true) return Promise.resolve(false) + return new Promise((resolve) => { + const timer = setTimeout(() => { + signal?.removeEventListener('abort', onAbort) + resolve(true) + }, pollMs) + const onAbort = (): void => { + clearTimeout(timer) + resolve(false) + } + signal?.addEventListener('abort', onAbort, { once: true }) + }) +} + +/** + * Signal remote process groups, tolerating the shared teardown outcomes: a + * nonzero `kill` (groups already gone) and a disappeared sandbox. Both the + * pgid-keyed process ladder and the sid-keyed terminal ladder deliver signals + * through this single tolerance so they cannot drift apart. + * @param sandbox - Live SDK handle. + * @param envs - Control-shell environment entries. + * @param groups - Positive process-group ids to signal. + * @param signal - `TERM` or `KILL`. + */ +export async function signalRemoteGroups( + sandbox: Sandbox, + envs: Record, + groups: readonly number[], + signal: 'TERM' | 'KILL', +): Promise { + // TODO(e2b-pgid-identity): Prefer an atomic identity-bound group signal if E2B adds one; + // a userspace identity precheck cannot close the numeric-PGID reuse race. + try { + await sandbox.commands.run( + `kill -${signal} -- ${groups.map(group => `-${group}`).join(' ')}`, + commandOpts(envs), + ) + } catch (error: unknown) { + if (!(error instanceof CommandExitError) && !(error instanceof SandboxNotFoundError)) throw error + } +} diff --git a/packages/e2b/subprocess-e2b/src/terminal.ts b/packages/e2b/subprocess-e2b/src/terminal.ts index 3108a4a60a..fd10ecdd8e 100644 --- a/packages/e2b/subprocess-e2b/src/terminal.ts +++ b/packages/e2b/subprocess-e2b/src/terminal.ts @@ -25,8 +25,7 @@ import { readRemoteEnvironment, serializeRemoteEnvironment, } from './environment.ts' - -const POLL_MS = 20 +import { asError, commandOpts, delay, signalOpts, signalRemoteGroups } from './remote.ts' const TERMINAL_RUNNER_SOURCE = [ '#!/bin/bash', @@ -52,21 +51,6 @@ interface TerminalPaths { outputMarker: string } -function signalOpts(signal: AbortSignal | undefined): { signal?: AbortSignal } { - return signal === undefined ? {} : { signal } -} - -function commandOpts( - envs: Record, - signal?: AbortSignal, -): { envs: Record; signal?: AbortSignal } { - return { envs: e2bControlEnvs(envs), ...signalOpts(signal) } -} - -function delay(ms: number): Promise { - return new Promise(resolve => setTimeout(resolve, ms)) -} - class BootstrapOutputFilter { readonly ready: Promise @@ -134,10 +118,6 @@ async function waitForBootstrapOutput( }) } -function asError(error: unknown): Error { - return error instanceof Error ? error : new Error(String(error)) -} - function parsePositiveId(value: string, message: string): number { const raw = value.trim() const id = Number(raw) @@ -193,27 +173,12 @@ async function sessionProcessGroups( return [...groups] } -async function signalGroups( - sandbox: Sandbox, - groups: number[], - signal: 'TERM' | 'KILL', - envs: Record, -): Promise { - try { - await sandbox.commands.run( - `kill -${signal} -- ${groups.map(group => `-${group}`).join(' ')}`, - commandOpts(envs), - ) - } catch (error: unknown) { - if (!(error instanceof CommandExitError) && !(error instanceof SandboxNotFoundError)) throw error - } -} - async function awaitSessionEmpty( sandbox: Sandbox, sessionId: number, envs: Record, graceMs: number, + pollMs: number, kill = false, ): Promise { const deadline = Date.now() + graceMs @@ -221,12 +186,12 @@ async function awaitSessionEmpty( const groups = await sessionProcessGroups(sandbox, sessionId, envs) if (groups.length === 0) return groups if (kill) { - await signalGroups(sandbox, groups, 'KILL', envs) + await signalRemoteGroups(sandbox, envs, groups, 'KILL') if (Date.now() >= deadline) return await sessionProcessGroups(sandbox, sessionId, envs) } else if (Date.now() >= deadline) { return groups } - await delay(Math.min(POLL_MS, Math.max(1, deadline - Date.now()))) + await delay(Math.min(pollMs, Math.max(1, deadline - Date.now()))) } } @@ -236,6 +201,7 @@ async function rollbackUnpublishedTerminal( completion: Promise, envs: Record, graceMs: number, + pollMs: number, ): Promise { let topLevelExited = false void completion.then( @@ -256,11 +222,11 @@ async function rollbackUnpublishedTerminal( try { let groups = await sessionProcessGroups(sandbox, sessionId, envs) if (groups.length > 0) { - await signalGroups(sandbox, groups, 'TERM', envs) - groups = await awaitSessionEmpty(sandbox, sessionId, envs, graceMs) + await signalRemoteGroups(sandbox, envs, groups, 'TERM') + groups = await awaitSessionEmpty(sandbox, sessionId, envs, graceMs, pollMs) } if (groups.length > 0) { - await awaitSessionEmpty(sandbox, sessionId, envs, graceMs, true) + await awaitSessionEmpty(sandbox, sessionId, envs, graceMs, pollMs, true) } } catch (error: unknown) { attemptFailures.push(asError(error)) @@ -280,7 +246,7 @@ async function rollbackUnpublishedTerminal( const proofFailures: Error[] = [] if (sessionId !== undefined) { try { - const groups = await awaitSessionEmpty(sandbox, sessionId, envs, graceMs, true) + const groups = await awaitSessionEmpty(sandbox, sessionId, envs, graceMs, pollMs, true) if (groups.length > 0) { proofFailures.push(new Error( `subprocess-e2b: terminal setup rollback failed; surviving process groups: ${groups.join(', ')}`, @@ -328,6 +294,7 @@ export class E2BTerminalHandle implements SubprocessTerminalHandle { private readonly controlEnvs: Record, private readonly stateDir: string, private readonly graceMs: number, + private readonly pollMs: number, ) { this.pid = handle.pid this.done = this.waitForCommand() @@ -441,8 +408,8 @@ export class E2BTerminalHandle implements SubprocessTerminalHandle { let groups = await sessionProcessGroups(this.sandbox, this.sessionId, this.controlEnvs) if (groups.length > 0) { this.terminationSignal = 'SIGTERM' - await signalGroups(this.sandbox, groups, 'TERM', this.controlEnvs) - groups = await awaitSessionEmpty(this.sandbox, this.sessionId, this.controlEnvs, this.graceMs) + await signalRemoteGroups(this.sandbox, this.controlEnvs, groups, 'TERM') + groups = await awaitSessionEmpty(this.sandbox, this.sessionId, this.controlEnvs, this.graceMs, this.pollMs) } if (groups.length === 0 && !this.topLevelExited) { await Promise.race([this.done.catch(() => undefined), delay(this.graceMs)]) @@ -457,7 +424,7 @@ export class E2BTerminalHandle implements SubprocessTerminalHandle { throw error } } - groups = await awaitSessionEmpty(this.sandbox, this.sessionId, this.controlEnvs, this.graceMs, true) + groups = await awaitSessionEmpty(this.sandbox, this.sessionId, this.controlEnvs, this.graceMs, this.pollMs, true) if (!this.topLevelExited) await Promise.race([this.done.catch(() => undefined), delay(this.graceMs)]) } if (groups.length > 0) { @@ -485,12 +452,14 @@ export class E2BTerminalHandle implements SubprocessTerminalHandle { * @param runtime - Shared E2B sandbox owner. * @param spec - Fully specified terminal-process request. * @param stateDir - Private remote directory for one startup transaction. + * @param pollMs - Remote session liveness poll cadence. * @returns The live subprocess terminal handle. */ export async function spawnE2BTerminal( runtime: E2BSandboxService, spec: SubprocessTerminalSpawnSpec, stateDir: string, + pollMs = 20, ): Promise { const sandbox = await runtime.getSandbox() spec.signal?.throwIfAborted() @@ -555,6 +524,7 @@ export async function spawnE2BTerminal( controlEnvs, stateDir, spec.graceMs, + pollMs, ) } catch (error: unknown) { output.destroy() @@ -565,7 +535,7 @@ export async function spawnE2BTerminal( if (!terminalQuiescent && handle !== undefined) { try { if (completion === undefined) await handle.kill() - else await rollbackUnpublishedTerminal(sandbox, handle, completion, controlEnvs, spec.graceMs) + else await rollbackUnpublishedTerminal(sandbox, handle, completion, controlEnvs, spec.graceMs, pollMs) terminalQuiescent = true } catch (cleanupError: unknown) { if (cleanupError instanceof SandboxNotFoundError) terminalQuiescent = true diff --git a/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts b/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts index 471716ef1a..747dc936c6 100644 --- a/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts +++ b/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts @@ -519,6 +519,38 @@ describe('E2BSubprocessHandle', () => { await expect(handle.waitForExit()).resolves.toBe(true) }) + it('releases an inherited-output callback blocked on host backpressure at drain expiry', async () => { + const fake = new FakeSandbox() + const written: string[] = [] + const stdoutWrite = vi.spyOn(process.stdout, 'write').mockImplementation(((chunk: Uint8Array) => { + written.push(Buffer.from(chunk).toString()) + return false + }) as typeof process.stdout.write) + try { + const handle = new E2BSubprocessHandle(runtime(fake), spec({ + graceMs: 5, + stdio: { stdin: 'ignore', stdout: 'inherit', stderr: { maxBytes: 4 } }, + }), '/runtime/inherit-backpressure') + await flush() + let callbackSettled = false + const blocked = fake.stdout('blocked bytes').then(() => { callbackSettled = true }) + await flush() + expect(callbackSettled).toBe(false) + fake.exitStatus = '0\n' + + await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) + await blocked + expect(callbackSettled).toBe(true) + expect(written.join('')).toBe('blocked bytes') + expect(fake.handle.disconnects).toBe(1) + + handle.terminate() + await expect(handle.waitForExit()).resolves.toBe(true) + } finally { + stdoutWrite.mockRestore() + } + }) + it('waits for lossless raw-pipe output after the direct status is published', async () => { const fake = new FakeSandbox() const handle = new E2BSubprocessHandle(runtime(fake), spec({ @@ -1311,6 +1343,17 @@ describe('E2BSubprocessHandle', () => { expect(invalidGroup.commandsSeen).toContain('kill -KILL -- -4242') await expect(invalid.waitForExit()).resolves.toBe(true) + // A rewritten pid file must not aim the kill at every process (`-- -1`). + const unsafeGroup = new FakeSandbox() + unsafeGroup.processGroupId = '1\n' + unsafeGroup.delaysKill = true + unsafeGroup.sdkKillStops = false + unsafeGroup.afterProbe = () => { unsafeGroup.alive = false } + const unsafe = new E2BSubprocessHandle(runtime(unsafeGroup), spec(), '/runtime/unsafe-group') + await expect(unsafe.done).rejects.toThrow(/unsafe published process-group id 1/) + expect(unsafeGroup.commandsSeen).not.toContain('kill -KILL -- -1') + await expect(unsafe.waitForExit()).resolves.toBe(true) + const absentGroup = new FakeSandbox() absentGroup.processGroupId = '' const absent = new E2BSubprocessHandle(runtime(absentGroup), spec(), '/runtime/absent-group') @@ -1588,6 +1631,34 @@ describe('E2BSubprocessService', () => { await expect(handle.done).resolves.toEqual({ exitCode: null, signal: 'SIGTERM' }) }) + it('aggregates sibling cleanup failures instead of reporting only the first', async () => { + const { ctx, fiber } = await service() + const disposalErrors: unknown[] = [] + ctx.logger.error = ((error: unknown) => { disposalErrors.push(error) }) as typeof ctx.logger.error + const first = { + terminate: vi.fn(), + waitForExit: vi.fn(async () => { throw new Error('first cleanup failed') }), + done: Promise.resolve({ exitCode: 0, signal: null }), + } as unknown as E2BSubprocessHandle + const second = { + terminate: vi.fn(), + waitForExit: vi.fn(async () => { throw new Error('second cleanup failed') }), + done: Promise.resolve({ exitCode: 0, signal: null }), + } as unknown as E2BSubprocessHandle + const live = (ctx.subprocess as unknown as { live: Set }).live + live.add(first) + live.add(second) + + await fiber.dispose() + const failure = disposalErrors[0] + expect(failure).toBeInstanceOf(AggregateError) + if (!(failure instanceof AggregateError)) throw new Error('expected AggregateError') + expect(failure.errors.map(error => (error as Error).message).sort()).toEqual([ + 'first cleanup failed', + 'second cleanup failed', + ]) + }) + it('waits for every owned cleanup before reporting a disposal failure', async () => { const { ctx, fiber } = await service() const failed = { @@ -1667,7 +1738,6 @@ describe('E2BSubprocessService', () => { it('validates synchronous spawn preconditions', async () => { const { ctx } = await service() expect(() => ctx.subprocess.spawn(spec({ argv: [] }))).toThrow(/non-empty program/) - expect(() => ctx.subprocess.spawn(spec({ graceMs: 0 }))).toThrow(/positive finite/) expect(() => ctx.subprocess.spawn(spec({ signal: AbortSignal.abort('stop') }))).toThrow(/aborted before spawn/) }) diff --git a/packages/e2b/subprocess-e2b/tests/terminal.spec.ts b/packages/e2b/subprocess-e2b/tests/terminal.spec.ts index 9973b5151a..826732f057 100644 --- a/packages/e2b/subprocess-e2b/tests/terminal.spec.ts +++ b/packages/e2b/subprocess-e2b/tests/terminal.spec.ts @@ -793,6 +793,8 @@ describe('E2B subprocess terminal service', () => { it('rejects invalid executable lookup inputs and results', async () => { const { ctx, fake } = await service() await expect(ctx.subprocess.resolveExecutable('')).rejects.toThrow('non-empty') + await expect(ctx.subprocess.resolveExecutable('./bin/server')).rejects.toThrow('is a relative path') + await expect(ctx.subprocess.resolveExecutable('node_modules/.bin/server')).rejects.toThrow('is a relative path') await expect(ctx.subprocess.resolveExecutable('node', undefined, AbortSignal.abort(new Error('stop')))) .rejects.toThrow('stop') fake.resolvedExecutable = 'node\n' @@ -801,6 +803,15 @@ describe('E2B subprocess terminal service', () => { await expect(ctx.subprocess.resolveExecutable('node')).rejects.toThrow('did not resolve') }) + it('rejects a non-positive poll cadence at load', async () => { + const ctx = new Context() + ctx.provide('e2b', runtime(new FakeTerminalSandbox())) + await expect(ctx.plugin(E2BSubprocessService, { pollMs: 0 })) + .rejects.toThrow('pollMs must be a positive safe integer') + const explicit = await ctx.plugin(E2BSubprocessService, { pollMs: 5 }) + await explicit.dispose() + }) + it('owns live terminals through service disposal', async () => { const { ctx, fiber, fake } = await service() const terminal = await ctx.subprocess.spawnTerminal(spec({ signal: new AbortController().signal })) @@ -873,9 +884,6 @@ describe('E2B subprocess terminal service', () => { const { ctx, fiber, fake } = await service() for (const request of [ spec({ argv: [] }), - spec({ rows: 0 }), - spec({ cols: 1.5 }), - spec({ graceMs: 0 }), spec({ signal: AbortSignal.abort(new Error('cancelled')) }), ]) { await expect(ctx.subprocess.spawnTerminal(request)).rejects.toThrow() From 6e1ae76c9bbf001b68dfdeac93f628f755fcdbbd Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Mon, 3 Aug 2026 02:17:04 +0800 Subject: [PATCH 101/108] test(e2b): satisfy the steer() receipt signature in Agent stubs Master's steering-receipt API (agent.steer returns SteeringReceipt) reached this branch through the parent sync; the two E2B live-fixture Agent stubs return the same rejected-outcome receipt the pty test stubs use. --- examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts | 2 +- packages/e2b/e2b/tests/composition.e2e.ts | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts b/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts index 7348d1d181..985b0a6121 100644 --- a/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts +++ b/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts @@ -22,7 +22,7 @@ const owner: Agent = { acceptsNextStep: false, ctx: ownerFiber.ctx, followup() {}, - steer() {}, + steer: () => ({ outcome: Promise.resolve({ status: 'rejected' as const }) }), inject() {}, send() {}, cancel() {}, diff --git a/packages/e2b/e2b/tests/composition.e2e.ts b/packages/e2b/e2b/tests/composition.e2e.ts index 3f0b69668d..c09243183e 100644 --- a/packages/e2b/e2b/tests/composition.e2e.ts +++ b/packages/e2b/e2b/tests/composition.e2e.ts @@ -86,7 +86,7 @@ describe.skipIf(!process.env.E2B_API_KEY)('E2B live Loader composition', () => { acceptsNextStep: false, ctx, followup() {}, - steer() {}, + steer: () => ({ outcome: Promise.resolve({ status: 'rejected' as const }) }), inject() {}, send() {}, cancel() {}, From d488330ba4890d12de3373718b243d4ac27b5783 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Fri, 7 Aug 2026 21:04:33 +0800 Subject: [PATCH 102/108] fix(rebase): migrate the replayed E2B branch onto the rebased parent The linear replay carried old-lineage content over parent-owned files; this checkpoint restores them and adapts the branch to the parent's post-rebase seam: - restore all pty/lsp/subprocess/code-runtime surfaces to the parent's exact content (this branch claims none of them) and drop the net-zero code-runtime-e2b/pty-e2b/lsp-e2b residue and its registrations - widen serializeRemoteEnvironment to the seam's NodeJS.ProcessEnv tombstone contract: an explicit undefined removes an ambient entry - migrate the two E2B fixture Agent stubs to the Inbox-model interface and Session.create - re-apply the branch's gen-doc-graphs roles, THIRD_PARTY_NOTICES e2b row, and packages/README group row (trimmed to the doc budget); regenerate catalogs and re-record bilingual pairings --- ...8-portable-execution-world-consumers.zh.md | 15 +- THIRD_PARTY_NOTICES.md | 2 + docs/architecture.zh.md | 1 - docs/capability-seams.md | 14 +- docs/config-catalog.md | 17 + .../tests/fixtures/e2b/e2b/bin.ts | 13 +- examples/package.json | 3 - packages/README.i18n.yaml | 4 +- packages/README.md | 2 +- packages/code-runtime/README.i18n.yaml | 4 +- packages/code-runtime/README.md | 9 +- packages/code-runtime/README.zh.md | 13 +- .../code-runtime-worker/README.i18n.yaml | 4 +- .../code-runtime-worker/README.md | 6 +- .../code-runtime-worker/README.zh.md | 26 +- .../code-runtime-worker/src/index.ts | 35 +- .../code-runtime/README.i18n.yaml | 4 +- packages/code-runtime/code-runtime/README.md | 2 +- .../code-runtime/code-runtime/README.zh.md | 20 +- .../cordis/tool-cordis/src/api-catalog.ts | 10 + packages/e2b/e2b/tests/composition.e2e.ts | 13 +- .../e2b/subprocess-e2b/src/environment.ts | 10 +- packages/lsp/README.i18n.yaml | 4 +- packages/lsp/README.md | 5 +- packages/lsp/README.zh.md | 5 +- packages/lsp/lsp-local/src/index.ts | 199 ++++--- packages/lsp/lsp-local/src/instance.ts | 27 +- packages/lsp/lsp-local/tests/instance.spec.ts | 2 - packages/lsp/lsp/README.i18n.yaml | 6 +- packages/pty/README.i18n.yaml | 4 +- packages/pty/README.md | 5 +- packages/pty/README.zh.md | 5 +- packages/pty/pty-local/src/sanitize.ts | 188 +++++++ packages/pty/pty-local/src/session.ts | 513 ++++++++++++------ packages/pty/pty-local/tests/sanitize.spec.ts | 34 +- packages/subprocess/README.i18n.yaml | 4 +- packages/subprocess/README.md | 7 +- packages/subprocess/README.zh.md | 11 +- pnpm-lock.yaml | 230 +++++++- scripts/gen-doc-graphs.ts | 13 +- 40 files changed, 1043 insertions(+), 446 deletions(-) create mode 100644 packages/pty/pty-local/src/sanitize.ts diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md index 68d022216c..ce6650a284 100644 --- a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md @@ -6,7 +6,7 @@ Status: implemented ## 问题 -文件系统与进程管理 seam 使文件访问和普通进程访问具备可替换性,但若干上层能力仍直接调用宿主 Node API。因此,即使领域行为没有变化,远程执行提供方看起来仍需要独立的 PTY、LSP 与代码运行时包(package)。这些包只会成为浅层适配器:每个包都仅为替换文件与进程操作而复制一个现有消费方。 +文件系统与进程管理 seam 使文件访问和普通进程访问具备可替换性,但 PTY 和 LSP 仍直接调用宿主 Node API。因此,即使领域行为没有变化,远程执行提供方看起来仍需要独立的 PTY 与 LSP 包(package)。这些包只会成为浅层适配器:每个包都仅为替换文件与进程操作而复制一个现有消费方。 只有文件操作、命令、终端和语言服务器共享同一个沙箱身份时,远程编码世界才有用。若把完整 harness 移入该沙箱,还会把提供方实验与插件加载、凭据、模型传输、会话持久性、监督和部署纠缠在一起。 @@ -18,16 +18,13 @@ Status: implemented 文件系统接口负责其他能力需要的路径事实,同时不公开其不透明目标身份:规范化进程路径、规范化 `file:` URI 和包含关系。现有完整文本与流式文本操作仍归文件系统负责;协议消费方在消费流时执行各自的保留上限。 -进程管理接口负责进程运行坐标与原语:规范化 cwd、私有运行时存储、可执行文件查找、以原始或收集模式 spawn 普通进程,以及 `spawnTerminal()`。终端操作是一项深层原语,其句柄负责文本 I/O、前台进程组、信号发送,以及一项须等待的 TERM→KILL 操作;该操作会结算所有在途句柄调用,并使整个会话完全停稳。其信号只取消分配;句柄一经发布,便负责自身生命周期。提示符检测、空闲推断、scrollback、沙箱策略和所有者生命周期仍由 PTY 消费方负责。 +进程管理接口负责可执行文件查找与进程原语:以原始或收集模式 spawn 普通进程,以及 `spawnTerminal()`。终端操作是一项深层原语,其句柄负责文本 I/O、前台进程组、信号发送,以及一项须等待的 TERM→KILL 操作;该操作会结算所有在途句柄调用,并使提供方仍可观察到的每个会话成员完全停稳。其信号只取消分配;句柄一经发布,便负责自身生命周期。提示符检测、空闲推断、scrollback、沙箱策略和所有者生命周期仍由 PTY 消费方负责。 通用消费方使用该执行世界: - `dsh-bash-local` 继续把 Bash 语义映射到普通的 `ctx.subprocess.spawn()`。 - `dsh-lsp-local` 通过 `ctx.fs` 读取源文件并验证包含关系,通过 `ctx.subprocess` 解析和启动语言服务器,并让由提供方负责的文件 URI 贯穿初始化与结果渲染。一个提供方生命周期信号会在资源释放期间中止文件系统与协议操作,包括取得队列所有权之前的工作区查找;其 JSON-RPC、池化、同步和规范化保持不变。 -- `dsh-pty-local` 把持久 shell 语义映射到 `ctx.subprocess.spawnTerminal()`。本地 `node-pty` 与进程检查实现移入 `dsh-subprocess-local`;其他进程管理提供方则提供相同原语。提供方开始写入时,系统会丢弃异步写入前检查期间收集的提示符与静默证据。取消会在在途写入结算期间保留发送预留,随后向前台进程组发送信号,因此延迟字节和该信号都无法落到后续发送;写入被拒绝时不会发送信号。绝对截止时间会在整个取消期间保持启用。信号发送失败会成为终结性传输失败。陈旧检查完成后,会针对当前发送恢复轮询。关闭操作会拒绝新的公开信号,并把完整会话的完全停稳委托给句柄上须等待的终止操作。 -- `dsh-code-runtime-subprocess` 通过 `ctx.fs` 物化无依赖 runner,并通过 `ctx.subprocess` 启动它,从而在本地或远程执行世界中保留代码运行时的绑定与输出契约。固定 runner 是位于 `ctx.subprocess.runtimeRoot` 下的适配器自有基础设施,因此其写入携带显式 `danger-full-access` 策略,而不继承面向模型的文件系统模式。它通过非插件子路径 `dsh-code-runtime-worker/runtime-host` 共享宿主侧 worker 机制,而不是复制这些机制。准备阶段让同一个生命周期信号贯穿文件系统解析、物化和可执行文件查找,使资源释放能够中止停滞的提供方操作。受堆上限约束的 worker 会在传输前拒绝过大的绑定帧;每个外层转发环节都会在转发前执行相同的上限检查;原始子进程管道承载以换行符分隔的 UTF-8 JSON,无需冗余的 base64 表示;launcher 会在回收 controller 前发布已接纳的终态帧,使继承 controller 管道的后代进程无法阻止完成;宿主仍会等待进程组完全停稳。 - -`dsh-code-runtime-worker` 仍是独立实现。它是较小的进程内后端,可用于无法假定已安装 Node 可执行文件的单文件分发。远程文件系统/进程组合选择 `dsh-code-runtime-subprocess`;它们不需要提供方专用的代码运行时包。 +- `dsh-pty-local` 把持久 shell 语义映射到 `ctx.subprocess.spawnTerminal()`。本地 `node-pty` 与进程检查实现移入 `dsh-subprocess-local`;其他进程管理提供方则提供相同原语。`danger-full-access` 不需要 `ctx.sandbox`;受限模式要求同一执行世界中存在沙箱提供方,未挂载时会在 spawn 前失败。提供方开始写入时,系统会丢弃异步写入前检查期间收集的提示符与静默证据。取消会在在途写入结算期间保留发送预留,随后向前台进程组发送信号,因此延迟字节和该信号都无法落到后续发送;在途就绪检查无法释放该预留,写入被拒绝时也不会发送信号。绝对截止时间会在整个取消期间保持启用。信号发送失败会成为终结性传输失败。陈旧检查完成后,会针对当前发送恢复轮询。启动取消会立即开始终端回滚,而不等待停滞的就绪检查或信号发送调用。关闭操作会拒绝新的公开信号,并把提供方可观察会话成员的完全停稳委托给句柄上须等待的终止操作。 ## E2B POC 边界 @@ -45,7 +42,7 @@ E2B 负责可变文件系统、受管命令与 Bash 进程、终端分配与终 ## 考虑过的替代方案 -**为每个远程提供方分别保留 PTY、LSP 和代码运行时包。** 不予采纳,因为这会在现有 seam 之上重复实现提供方机制。删除检验揭示了这一问题:删除这些适配器不应使领域行为散落到远程提供方中;通用消费方本已负责这些行为。 +**为每个远程提供方分别保留 PTY 与 LSP 包。** 不予采纳,因为这会在现有 seam 之上重复实现提供方机制。删除检验揭示了这一问题:删除这些适配器不应使领域行为散落到远程提供方中;通用消费方本已负责这些行为。 **为每项能力或工具创建独立沙箱。** 不予采纳,因为文件与进程操作将无法共享身份或状态,从而破坏编码用例,并增加生命周期所有者的数量。 @@ -57,8 +54,6 @@ E2B 负责可变文件系统、受管命令与 Bash 进程、终端分配与终 **在文件系统 seam 中新增稳定的有界读取原语。** 不予采纳,因为只有 LSP 需要完整文档字节上限,而它可以在消费现有文本流时执行该上限。第二项原语会迫使每个提供方实现稳定句柄和不跟随符号链接的机制,远程提供方甚至需要辅助协议,却没有已观察到的并发替换缺陷。 -**删除 worker 线程代码运行时。** 不予采纳,因为可移植性不会消除其当前部署需求。进程管理后端需要 Node 可执行文件和文件系统物化,而 worker 后端两者都不需要,并且仍是受支持的单进程路径。 - **在远程环境中运行整个 harness。** 不予采纳,因为这是另一种部署模型。让执行能力可移植,并不意味着移动模型调用、会话状态、插件状态或 agent loop。 **把所有提供方操作都放进一个共享所有者包。** 不予采纳,因为沙箱身份与生命周期是所有者唯一的关注点。文件系统与进程管理保留各自独立的契约、测试和消费方,同时避免把所有者变成无边界的能力集合。 @@ -69,7 +64,7 @@ E2B 负责可变文件系统、受管命令与 Bash 进程、终端分配与终 ## 后果 -远程执行提供方只需实现共享沙箱所有者,以及文件系统与进程管理适配器。Bash、PTY、LSP 和基于进程管理的代码运行时组合在这些适配器之上,因此对这些能力的修复仍与提供方无关。 +远程执行提供方只需实现共享沙箱所有者,以及文件系统与进程管理适配器。Bash、PTY 与 LSP 组合在这些适配器之上,因此对这些能力的修复仍与提供方无关。 基础接口更宽,一对文件系统/进程管理提供方必须在同一个执行世界上保持一致。新增操作仅限当前通用消费方所需的事实与生命周期机制;模型 schema、协议分帧、就绪策略和呈现不会渗入提供方。 diff --git a/THIRD_PARTY_NOTICES.md b/THIRD_PARTY_NOTICES.md index 1d169819b4..9060950bea 100644 --- a/THIRD_PARTY_NOTICES.md +++ b/THIRD_PARTY_NOTICES.md @@ -57,6 +57,8 @@ External packages that a workspace package resolves at runtime. `scripts/install | [`clsx`](https://github.com/lukeed/clsx) | MIT | | [`commander`](https://github.com/tj/commander.js) | MIT | | [`diff`](https://github.com/kpdecker/jsdiff) | BSD-3-Clause | +| [`dotenv`](https://github.com/motdotla/dotenv) | BSD-2-Clause | +| [`e2b`](https://github.com/e2b-dev/e2b) | MIT | | [`eventsource-parser`](https://github.com/rexxars/eventsource-parser) | MIT | | [`handlebars`](https://github.com/handlebars-lang/handlebars.js) | MIT | | [`immer`](https://github.com/immerjs/immer) | MIT | diff --git a/docs/architecture.zh.md b/docs/architecture.zh.md index 2d1a0b43d5..9314a3fbfc 100644 --- a/docs/architecture.zh.md +++ b/docs/architecture.zh.md @@ -25,7 +25,6 @@ |---|---|---| | `ctx.llm` | [`llm/`](../packages/llm/README.md) | 适配器注册表和模型流式调用 | | `ctx.tokenMeter` | [`llm/token-meter`](../packages/llm/token-meter/README.md) | 感知回放的单实例请求压力与表面压力 | -| `ctx.e2b` | [`e2b/`](../packages/e2b/README.md) | 共享 E2B 沙箱 | | `ctx.bash` | [`bash/`](../packages/bash/README.md) | 前台和后台命令执行 | | `ctx.subprocess` | [`subprocess/`](../packages/subprocess/README.md) | 可执行文件查找、受管进程树、终端 | | `ctx.pty` | [`pty/`](../packages/pty/README.md) | 按 owner 隔离的持久化终端会话 | diff --git a/docs/capability-seams.md b/docs/capability-seams.md index 50ebe88d4f..4ed8a260d4 100644 --- a/docs/capability-seams.md +++ b/docs/capability-seams.md @@ -98,6 +98,10 @@ flowchart LR pkg_agent_spine_demo["agent-spine-demo"] pkg_goal["goal"] svc_goals["ctx.goals
Same-session goal domain"] + pkg_e2b["e2b"] + svc_e2b["ctx.e2b
E2B sandbox lifecycle owner"] + pkg_fs_e2b["fs-e2b"] + pkg_subprocess_e2b["subprocess-e2b"] pkg_subprocess["subprocess"] svc_subprocess["ctx.subprocess
Subprocess seam"] pkg_subprocess_local["subprocess-local"] @@ -190,7 +194,9 @@ flowchart LR pkg_directory_picker --> svc_directoryPicker pkg_directory_picker_browse --> svc_directoryPicker pkg_directory_picker_native --> svc_directoryPicker + pkg_e2b --> svc_e2b pkg_fs --> svc_fs + pkg_fs_e2b --> svc_fs pkg_fs_local --> svc_fs pkg_fs_sandbox --> svc_fs pkg_goal --> svc_goals @@ -241,6 +247,7 @@ flowchart LR pkg_subagent_fork --> svc_subagents pkg_subagent_spawn --> svc_subagents pkg_subprocess --> svc_subprocess + pkg_subprocess_e2b --> svc_subprocess pkg_subprocess_local --> svc_subprocess pkg_system_prompt --> svc_systemPrompt pkg_tasks --> svc_tasks @@ -277,6 +284,8 @@ flowchart LR svc_credentials --> pkg_llm_deepseek svc_credentials --> pkg_llm_pi_ai svc_directoryPicker --> pkg_apiproxy + svc_e2b --> pkg_fs_e2b + svc_e2b --> pkg_subprocess_e2b svc_fs --> pkg_tool_fs svc_httpServer --> pkg_connection svc_httpServer --> pkg_hmr @@ -390,7 +399,8 @@ flowchart LR | `ctx.agents` | `core` | [`agent`](../packages/core/agent) | - | [`agent-loop`](../packages/core/agent-loop), [`acp`](../packages/acp/acp), [`subagent-inprocess`](../packages/subagent/subagent-inprocess) | - | Owns live Agent handles, the create/resume factory seam, and process-local initiator propagation. | | `ctx.agentLoop` | `bundle` | [`agent-loop`](../packages/core/agent-loop) | - | [`agent-spine-demo`](../packages/examples/agent-spine-demo) | - | The one concrete loop plugin; extension packages depend on dsh-agent events and services, not on this package. | | `ctx.goals` | `core` | [`goal`](../packages/goal/goal) | - | - | - | Folds revisioned objective state from the session log and keeps live continuation activation process-local. | -| `ctx.subprocess` | `seam` | [`subprocess`](../packages/subprocess/subprocess) | [`subprocess-local`](../packages/subprocess/subprocess-local) | [`bash-local`](../packages/bash/bash-local), [`bash-sandbox`](../packages/bash/bash-sandbox), [`pty-local`](../packages/pty/pty-local), [`lsp-local`](../packages/lsp/lsp-local), [`subagent-acp`](../packages/subagent/subagent-acp), [`subagent-codex`](../packages/subagent/subagent-codex), [`subagent-claude-code`](../packages/subagent/subagent-claude-code) | - | The bash executors, the PTY shell backend, the LSP host, and the out-of-process ACP, Codex, and Claude Code subagent backends spawn through ctx.subprocess; the service owns process coordinates, tree/session lifetime, stdio dispositions, terminal mechanics, and kill escalation. | +| `ctx.e2b` | `core` | [`e2b`](../packages/e2b/e2b) | - | [`fs-e2b`](../packages/e2b/fs-e2b), [`subprocess-e2b`](../packages/e2b/subprocess-e2b) | - | Owns one shared E2B SDK handle, remote working directory, and final sandbox disposition so both fundamental E2B providers inhabit the same Linux runtime. | +| `ctx.subprocess` | `seam` | [`subprocess`](../packages/subprocess/subprocess) | [`subprocess-local`](../packages/subprocess/subprocess-local), [`subprocess-e2b`](../packages/e2b/subprocess-e2b) | [`bash-local`](../packages/bash/bash-local), [`bash-sandbox`](../packages/bash/bash-sandbox), [`pty-local`](../packages/pty/pty-local), [`lsp-local`](../packages/lsp/lsp-local), [`subagent-acp`](../packages/subagent/subagent-acp), [`subagent-codex`](../packages/subagent/subagent-codex), [`subagent-claude-code`](../packages/subagent/subagent-claude-code) | - | The bash executors, the PTY shell backend, the LSP host, and the out-of-process ACP, Codex, and Claude Code subagent backends spawn through ctx.subprocess; the service owns process coordinates, tree/session lifetime, stdio dispositions, terminal mechanics, and kill escalation. | | `ctx.bash` | `seam` | [`bash`](../packages/bash/bash) | [`bash-local`](../packages/bash/bash-local), [`bash-sandbox`](../packages/bash/bash-sandbox), [`pwsh-local`](../packages/bash/pwsh-local) | [`tool-bash`](../packages/bash/tool-bash), [`tool-pwsh`](../packages/bash/tool-pwsh), [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex) | - | The model-facing shell tools and hook bridges consume this seam; sandboxed, remote, or PowerShell executors replace bash-local without touching them. | | `ctx.bashEnv` | `core` | [`bash-env`](../packages/bash/bash-env) | - | [`tool-bash`](../packages/bash/tool-bash), [`tool-pwsh`](../packages/bash/tool-pwsh) | - | Plugins declare effect-scoped DSH_* facts; each shell tool collects one trusted snapshot per execution and its executor rebuilds the namespace. | | `ctx.pty` | `seam` | [`pty`](../packages/pty/pty) | [`pty-local`](../packages/pty/pty-local) | [`tool-pty`](../packages/pty/tool-pty) | - | The registry owns exact-Agent session identity and cleanup; backends own terminal mechanics, while tool-pty exposes the owner-scoped model surface. | @@ -399,7 +409,7 @@ flowchart LR | `ctx.approval` | `seam` | `approval` | [`acp`](../packages/acp/acp) | [`tools`](../packages/core/tools), [`tool-bash`](../packages/bash/tool-bash) | - | One-shot permission decisions dispatched over the `approval/request` waterfall; answerers are listeners (the ACP bridge for its own agents), absence fails closed to `unavailable`. | | `ctx.permission` | `core` | [`permission`](../packages/ui/permission) | - | - | - | User-facing preset table (`workspace-write`/`danger-full-access`) bundling the sandbox-mode and approval-policy knobs; a switch writes one `permission/preset` event through to both knob events. | | `ctx.codeRuntime` | `seam` | [`code-runtime`](../packages/code-runtime/code-runtime) | [`code-runtime-worker`](../packages/code-runtime/code-runtime-worker) | [`tools`](../packages/core/tools) | - | Runs one model-written program against host-provided async bindings; backends differ by substrate and language (the tool registry consumes it for Code Mode). | -| `ctx.fs` | `seam` | [`fs`](../packages/fs/fs) | [`fs-local`](../packages/fs/fs-local), [`fs-sandbox`](../packages/fs/fs-sandbox) | [`tool-fs`](../packages/fs/tool-fs) | [`fs-policy`](../packages/fs/fs-policy) | tool-fs executes read/write/edit through ctx.fs; fs-sandbox fences mutations by the shared sandbox mode; fs-policy contributes observed-state checks through the fs/* event gate. | +| `ctx.fs` | `seam` | [`fs`](../packages/fs/fs) | [`fs-local`](../packages/fs/fs-local), [`fs-sandbox`](../packages/fs/fs-sandbox), [`fs-e2b`](../packages/e2b/fs-e2b) | [`tool-fs`](../packages/fs/tool-fs) | [`fs-policy`](../packages/fs/fs-policy) | tool-fs executes read/write/edit through ctx.fs; fs-sandbox fences mutations by the shared sandbox mode; fs-policy contributes observed-state checks through the fs/* event gate. | | `ctx.compact` | `seam` | [`compact`](../packages/compact/compact) | [`compact-basic`](../packages/compact/compact-basic) | [`compact-basic`](../packages/compact/compact-basic) | - | The basic backend consumes post-step pressure and request-error recovery events; a model-facing compact tool remains deferred. | | `ctx.subagents` | `seam` | [`subagent`](../packages/subagent/subagent) | [`subagent-spawn`](../packages/subagent/subagent-spawn), [`subagent-fork`](../packages/subagent/subagent-fork), [`subagent-acp`](../packages/subagent/subagent-acp), [`subagent-codex`](../packages/subagent/subagent-codex), [`subagent-claude-code`](../packages/subagent/subagent-claude-code), [`subagent-dsh-sdk`](../packages/subagent/subagent-dsh-sdk) | [`tool-subagent`](../packages/subagent/tool-subagent), [`tool-subagent-control`](../packages/subagent/tool-subagent-control), [`tool-ralph`](../packages/workflow/tool-ralph) | - | Providers implement transports; the service also owns optional Activation-based continuation orchestration, tool-subagent selects one-shot or continuable delegation, tool-subagent-control delivers follow-ups, and tool-ralph requires one fresh structured-output route. | | `ctx.tasks` | `seam` | [`tasks`](../packages/tasks/tasks) | [`tasks-local`](../packages/tasks/tasks-local) | [`tool-bash`](../packages/bash/tool-bash), [`tool-pty`](../packages/pty/tool-pty), [`tool-subagent`](../packages/subagent/tool-subagent), [`tool-tasks`](../packages/tasks/tool-tasks) | - | Producers (background bash, PTY sends, and subagent delegations) register running work; tool-tasks is the model-facing control surface that reads, lists, and kills it; tasks-local is the process-local registry. | diff --git a/docs/config-catalog.md b/docs/config-catalog.md index 977921db07..58b319d662 100644 --- a/docs/config-catalog.md +++ b/docs/config-catalog.md @@ -399,6 +399,22 @@ export interface Config { Source: [`packages/credentials/credentials-local/src/index.ts:54`](../packages/credentials/credentials-local/src/index.ts) +## `@deepseek-ai/dsh-e2b` + +```ts config-catalog +/** Configuration for the shared E2B sandbox owner. */ +export interface Config { + /** API key; omission reads `E2B_API_KEY`. It is never forwarded into the sandbox. */ + apiKey?: string + /** Shared remote working directory, created before adapters receive the sandbox. */ + cwd?: string + /** E2B sandbox lifetime in milliseconds; expiry always deletes the sandbox. */ + timeoutMs?: number +} +``` + +Source: [`packages/e2b/e2b/src/index.ts:43`](../packages/e2b/e2b/src/index.ts) + ## `@deepseek-ai/dsh-frontend-static` Requires: `httpServer` @@ -2587,6 +2603,7 @@ These load from a `cordis.yml` entry with no `config:` block; they declare no co - `@deepseek-ai/dsh-command-feedback` — requires `commands` ([`packages/feedback/command-feedback/src/index.ts`](../packages/feedback/command-feedback/src/index.ts)) - `@deepseek-ai/dsh-command-goal` — requires `commands` · `goals` ([`packages/goal/command-goal/src/index.ts`](../packages/goal/command-goal/src/index.ts)) - `@deepseek-ai/dsh-commands` ([`packages/ui/commands/src/index.ts`](../packages/ui/commands/src/index.ts)) +- `@deepseek-ai/dsh-fs-e2b` — requires `e2b` ([`packages/e2b/fs-e2b/src/index.ts`](../packages/e2b/fs-e2b/src/index.ts)) - `@deepseek-ai/dsh-fs-policy` ([`packages/fs/fs-policy/src/index.ts`](../packages/fs/fs-policy/src/index.ts)) - `@deepseek-ai/dsh-goal-session` — requires `agents` · `goals` · `sessions` ([`packages/goal/goal-session/src/index.ts`](../packages/goal/goal-session/src/index.ts)) - `@deepseek-ai/dsh-host-directory-picker-auto` — requires `httpServer` · `loader` ([`packages/host/directory-picker-auto/src/index.ts`](../packages/host/directory-picker-auto/src/index.ts)) diff --git a/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts b/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts index 985b0a6121..7a787c735b 100644 --- a/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts +++ b/examples/headless-agent/tests/fixtures/e2b/e2b/bin.ts @@ -1,6 +1,7 @@ import { readFile } from 'node:fs/promises' import { resolve } from 'node:path' import { boot } from '@deepseek-ai/dsh-app-boot' +import { Inbox } from '@deepseek-ai/dsh-agent' import type { Agent } from '@deepseek-ai/dsh-agent' import { Session, SessionId } from '@deepseek-ai/dsh-session' import type {} from '@deepseek-ai/dsh-fs-e2b' @@ -14,18 +15,20 @@ if (configPath === undefined) throw new Error('usage: bin.ts ') const ctx = await boot('e2b-composition', resolve(configPath)) const ownerFiber = ctx.plugin(() => {}) const ownerId = SessionId('e2b-live-owner') +const session = Session.create(ownerId) const owner: Agent = { id: ownerId, options: {}, - session: new Session(ownerId), + session, + inbox: new Inbox(session, { inserted: () => {}, discarded: () => {}, claimed: () => {} }), status: 'idle', - acceptsNextStep: false, ctx: ownerFiber.ctx, - followup() {}, - steer: () => ({ outcome: Promise.resolve({ status: 'rejected' as const }) }), - inject() {}, send() {}, + followup() {}, + steer() {}, + inject() {}, cancel() {}, + runMaintenance: task => task(new AbortController().signal), whenIdle: () => Promise.resolve(), } const unregisterOwner = ctx.agents.register(owner) diff --git a/examples/package.json b/examples/package.json index e4251c9475..48ac39f5ca 100644 --- a/examples/package.json +++ b/examples/package.json @@ -54,10 +54,7 @@ "@deepseek-ai/dsh-sandbox": "workspace:*", "@deepseek-ai/dsh-sandbox-local": "workspace:*", "@deepseek-ai/dsh-sandbox-policy": "workspace:^", -<<<<<<< HEAD "@deepseek-ai/dsh-scope": "workspace:*", -======= ->>>>>>> 3f6c905d5e (Add E2B PTY, LSP, and code runtime providers) "@deepseek-ai/dsh-session": "workspace:*", "@deepseek-ai/dsh-session-checkpoint-policy": "workspace:*", "@deepseek-ai/dsh-session-persistence-jsonl": "workspace:*", diff --git a/packages/README.i18n.yaml b/packages/README.i18n.yaml index ce95627688..7e74d63f31 100644 --- a/packages/README.i18n.yaml +++ b/packages/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/README.md -README.md: a9a121d0cb13cd5e2148da8d754967359be06bab -README.zh.md: c6500ebb2abce14fabf68b79ec20f5c23cf5d5f7 +README.md: 1516c20ff55f366fe6af6b0be5495958708f29e3 +README.zh.md: 42f4d2b8d924b19fd3cde863dafb8c59d9c078cf diff --git a/packages/README.md b/packages/README.md index 125b466d72..98fc87a6b2 100644 --- a/packages/README.md +++ b/packages/README.md @@ -6,7 +6,7 @@ Packages use the `@deepseek-ai/dsh-*` scope. Cordis `Service` subclasses and fun ## Hierarchy -Groups contain packages at `packages///`; names remain `@deepseek-ai/dsh-`. **Group READMEs own package/ctx-key maps.** +Groups hold `packages///`; names stay `@deepseek-ai/dsh-`. **Group READMEs own package/ctx-key maps.** | Group | Role | Release expectation | |---|---|---| diff --git a/packages/code-runtime/README.i18n.yaml b/packages/code-runtime/README.i18n.yaml index 30c5ca2572..3eb462d513 100644 --- a/packages/code-runtime/README.i18n.yaml +++ b/packages/code-runtime/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/code-runtime/README.md -README.md: 34eecc4874afc5a576ed129e72eeeb944b95f67f -README.zh.md: 85d7a95a37cd1f4090062f3036a6634fde5011ee +README.md: f20a287419b94b1a9dc1d8da7303fc4d3032cfd3 +README.zh.md: f5cd4c9949f2bd7a7d6d7cd078144910712a3819 diff --git a/packages/code-runtime/README.md b/packages/code-runtime/README.md index 34eecc4874..f20a287419 100644 --- a/packages/code-runtime/README.md +++ b/packages/code-runtime/README.md @@ -2,12 +2,11 @@ English | [中文](README.zh.md) -The code-execution capability seam (see [capability seams](../../.agents/notes/implemented/architecture/2026-06-13-capability-seams.md)): an abstract runtime interface for executing one model-written program against host-provided async bindings, capturing what it printed and returned. The consumer is the tool registry's [Code Mode](../core/tools/README.md) (`tools: { mode: code }` — the `run_code` tool and the generated TypeScript SDK); design in the [Code Mode Agent Note](../../.agents/notes/implemented/feature/2026-06-15-code-mode.md). **Product** packages. +The code-execution capability seam (see [capability seams](../../.agents/notes/implemented/architecture/2026-06-13-capability-seams.md)): an abstract runtime interface for executing one model-written program against host-provided async bindings, capturing what it printed and returned. The consumer is the tool registry's [Code Mode](../core/tools/README.md) (`tools: { mode: code }` — the `run_code` tool and the SDK generated in the loaded runtime's `language`); design in the [Code Mode Agent Note](../../.agents/notes/implemented/feature/2026-06-15-code-mode.md). **Product** packages. | Package | Role | ctx key | |---|---|---| -| `code-runtime/` | Abstract code-execution seam (interface + vocabulary) | `ctx.codeRuntime` | -| [`code-runtime-worker/`](code-runtime-worker/README.md) | Worker-thread backend: fresh worker per run, TypeScript via host-side type-strip (annotations advisory, never type-checked), port-bridged bindings, budget/heap containment | registers `ctx.codeRuntime` | -| [`e2b/code-runtime-e2b`](../e2b/code-runtime-e2b/README.md) | E2B backend: host type-strip and bindings, fresh remote runner/worker, framed bridge, remote process-group cleanup | registers `ctx.codeRuntime` | +| [`code-runtime/`](code-runtime/README.md) | Code-execution seam and shared vocabulary | `ctx.codeRuntime` | +| [`code-runtime-worker/`](code-runtime-worker/README.md) | Worker-thread backend | registers `ctx.codeRuntime` | -Backends differ by execution substrate and source language—both readonly descriptors on the service—and register `ctx.codeRuntime` without touching the interface or its consumer. The E2B ownership split is recorded in the [shared E2B runtime note](../../.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md). +Backends register the seam without changing its consumer. The child READMEs own language, isolation, and execution-budget details. diff --git a/packages/code-runtime/README.zh.md b/packages/code-runtime/README.zh.md index 85d7a95a37..f5cd4c9949 100644 --- a/packages/code-runtime/README.zh.md +++ b/packages/code-runtime/README.zh.md @@ -1,13 +1,12 @@ -# code-runtime/:代码执行能力家族 +# code-runtime/ — 代码执行能力家族 [English](README.md) | 中文 -代码执行能力 seam(参见[能力 seam](../../.agents/notes/implemented/architecture/2026-06-13-capability-seams.md)):一个抽象运行时接口,用于针对宿主提供的异步绑定执行一段模型编写的程序,并捕获程序打印和返回的内容。消费方是工具注册表的 [Code Mode](../core/tools/README.md)(`tools: { mode: code }`,即 `run_code` 工具与生成的 TypeScript SDK);设计记录在 [Code Mode Agent Note](../../.agents/notes/implemented/feature/2026-06-15-code-mode.md) 中。这些都是**产品** 包。 +代码执行能力 seam(参见[能力 seam](../../.agents/notes/implemented/architecture/2026-06-13-capability-seams.md)):一个抽象运行时接口,用于对宿主提供的异步绑定执行模型编写的程序,并捕获它打印和返回的内容。消费方是工具注册表的 [Code Mode](../core/tools/README.md)(`tools: { mode: code }`,即 `run_code` 工具和按所加载运行时 `language` 生成的 SDK);设计见 [Code Mode Agent Note](../../.agents/notes/implemented/feature/2026-06-15-code-mode.md)。这些全是**产品**包。 -| 包 | 职责 | ctx 键 | +| 包 | 职责 | ctx key | |---|---|---| -| `code-runtime/` | 抽象代码执行 seam(接口 + 词汇) | `ctx.codeRuntime` | -| [`code-runtime-worker/`](code-runtime-worker/README.md) | worker 线程后端:每次运行使用全新 worker,由宿主侧剥离 TypeScript 类型(类型注解仅供参考,绝不执行类型检查)、端口桥接绑定、预算/堆隔离 | 注册 `ctx.codeRuntime` | -| [`e2b/code-runtime-e2b`](../e2b/code-runtime-e2b/README.md) | E2B 后端:宿主侧类型剥离与绑定、全新远程 runner/worker、分帧桥、远程进程组清理 | 注册 `ctx.codeRuntime` | +| [`code-runtime/`](code-runtime/README.md) | 代码执行 seam 与共享词汇 | `ctx.codeRuntime` | +| [`code-runtime-worker/`](code-runtime-worker/README.md) | Worker 线程后端 | 注册 `ctx.codeRuntime` | -不同后端的执行基底和源语言各异,二者都是服务上的只读描述符;后端注册 `ctx.codeRuntime`,无需修改接口或消费方。E2B 所有权拆分记录在[共享 E2B 运行时 Agent Note](../../.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md) 中。 +后端在不改变消费方的情况下注册该 seam。子 README 负责语言、隔离和执行预算细节。 diff --git a/packages/code-runtime/code-runtime-worker/README.i18n.yaml b/packages/code-runtime/code-runtime-worker/README.i18n.yaml index 10c721446f..8d3568e82e 100644 --- a/packages/code-runtime/code-runtime-worker/README.i18n.yaml +++ b/packages/code-runtime/code-runtime-worker/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/code-runtime/code-runtime-worker/README.md -README.md: 35196a0b4fba5cd0388246a70354308ece08b39f -README.zh.md: 49871c38c540addd06f5d24793ae00b45e2f8bb0 +README.md: 590b79dcd1bc322350060b55767b09c6305edacc +README.zh.md: 12c25f892bd20892cb47e593f16b6aadd9ffa84c diff --git a/packages/code-runtime/code-runtime-worker/README.md b/packages/code-runtime/code-runtime-worker/README.md index fbe387e4ea..590b79dcd1 100644 --- a/packages/code-runtime/code-runtime-worker/README.md +++ b/packages/code-runtime/code-runtime-worker/README.md @@ -32,7 +32,7 @@ Every field is validated and defaulted; `maxOutputBytes` is a safe integer of at ## The worker entry, unbuilt and built -Source mode loads erasable-only `src/worker.ts` through Node's native type stripping. Its transitive runtime closure contains only Node built-ins and relative source modules, so a fresh checkout never requires a sibling workspace package's unbuilt `lib/` export. The worker-local JSON snapshotter is parity-tested against the session-owned canonical boundary; both sides flatten and rebuild validated values around the message port so application nesting never reaches structured clone. Built mode passes the sibling `lib/worker.cjs` as a filesystem path because pkg's VFS Worker hook expects CommonJS; the same path works under ordinary Node. `tests/built-lib.e2e.ts` pins the real load path required by [docs/testing.md](../../../docs/testing.md). +Source mode loads erasable-only `src/worker.ts` through Node's native type stripping. Its transitive runtime closure contains only Node built-ins and relative source modules, so a fresh checkout never requires a sibling workspace package's unbuilt `lib/` export. The worker-local and session-owned JSON boundaries both flatten and rebuild validated values around the message port so application nesting never reaches structured clone. Built mode passes the sibling `lib/worker.cjs` as a filesystem path because pkg's VFS Worker hook expects CommonJS; the same path works under ordinary Node. The repository-wide requirement to exercise this published entry path belongs to the [testing policy](../../../docs/testing.md). The SDK surface is the default/named `WorkerCodeRuntime` class plus `Config`. The operational `./worker` subpath exists only as the packaged spawn entry; the wire protocol and bootstrap helpers are source-private implementation details. @@ -46,8 +46,8 @@ No direct invalidation; the named consumer owns any request-prefix changes. ## Known Limitations and Deferred Work -- **OS processes a program spawns survive termination** — `worker.terminate()` ends the thread only; deployments requiring process-tree cleanup select `dsh-code-runtime-subprocess`, whose mounted subprocess provider owns that cleanup. -- **Type-strip rides Node's experimental `stripTypeScriptTypes` API** — the relied-on behavior is pinned by unit tests, with amaro/sucrase as named drop-in replacements if it shifts. +- **OS processes a program spawns survive termination** — `worker.terminate()` ends the thread only, weaker than bash-local's process-group kill; orphan cleanup is a deployment concern until a container backend exists. +- **Type-strip rides Node's experimental `stripTypeScriptTypes` API** — amaro or sucrase are the named drop-in replacements if the relied-on behavior shifts. - **`computeMs` expiry can overshoot by up to one poll interval** — busy time is sampled every 25 ms (an internal constant, deliberately not config). - **Programs get a five-method `console` shim** (`log`/`info`/`warn`/`error`/`debug`) — deliberately not Node's full console surface. - **Intermediate binding values have no byte cap** — a program can exhaust process or worker memory with a value that never becomes outer output. diff --git a/packages/code-runtime/code-runtime-worker/README.zh.md b/packages/code-runtime/code-runtime-worker/README.zh.md index 57f2a1145a..12c25f892b 100644 --- a/packages/code-runtime/code-runtime-worker/README.zh.md +++ b/packages/code-runtime/code-runtime-worker/README.zh.md @@ -23,18 +23,18 @@ - **每次运行使用一个全新 worker,不设池化**:程序所在的世界会随 worker 一同终止,不会留下需要记录的跨运行状态,也无法发生状态泄漏;仅凭会话日志即可重建运行。 - **在执行上下文中,由宿主侧剥离类型**:程序会包裹在异步函数外壳中,通过 `node:module` 的 `stripTypeScriptTypes` 剥离类型(只支持可擦除语法;`enum`/namespace 会作为程序 `exception` 被拒绝,且不会启动 worker),再按字节位置切回原内容。之后程序作为 `AsyncFunction` 的函数体执行,因此顶层 `await`/`return` 可用。 - **端口把对端视为不可信**:模型代码能够访问 `parentPort` 并伪造通信,因此任何代码读取入站消息前,系统都会验证其形状并重新构建(`null`、原始值、无效类型和格式错误的载荷会被静默丢弃;伪造的额外字段绝不会被带入);宿主对每个调用 id 最多响应一次,只将绑定名称解析为自有属性(伪造的 `constructor` 无法沿原型链访问),丢弃结算后的回复,并验证每个绑定 resolve 值与完成值是否为无损 JSON。伪造的 `log`/`done` 消息无法绕过外层上限:宿主会再次验证,并统计每条获准日志以及完成值或诊断。worker 侧命名空间使用 null-prototype 和 `defineProperty`,因此形似 `__proto__` 的绑定名称只是普通键。 -- **绑定 reject 类属于请求数据**:可选命名空间描述符会指定构造器全局变量,以及用于接收失败成员名称的自有属性。worker 会创建并注入该真实类,使 `instanceof` 生效,同时无需硬编码 `tools` 或 `ToolCallError`;全局变量无效或冲突的声明会在启动 worker 前失败。失败路径使用模块捕获的错误与属性定义 intrinsic,以及 null-prototype 描述符,因此模型之后的修改无法把被拒绝的绑定变成 worker 崩溃。 -- **两个独立预算,因为对端不可信**:`computeMs` 统计 worker 实际测得的忙碌时间(轮询 `worker.performance.eventLoopUtilization()`);热循环无法借助待完成的诱饵 dispatch 隐藏,程序等待慢工具时则不累计。`maxWallMs` 为忙碌时间无法观测的情况兜底(例如等待永远不会 resolve 的 promise)。二者最终都会调用 `worker.terminate()`,连同步热循环也能终止;堆溢出会表现为 worker 的 OOM 退出(`kind: 'worker-exit'`)。`maxWallMs` 在加载时会对照 `MAX_TIMER_DELAY_MS` 做范围校验:`setTimeout` 会把更长的延迟夹到 1 ms,仅有正数校验会放行一个在第一个 tick 就到期的上限。`computeMs` 不需要这道上界,因为它对照的是实测占用率,而不是喂给定时器。 -- **中间绑定值是完整 JSON**:绑定参数与 resolve 值会接受迭代式无损 JSON 验证。程序执行前,worker 会捕获自己 realm 中的普通容器原型身份,以及只用于外部 realm 的原生函数源码检查,因此构造器槽修改和用户编写的仿冒对象都无法改变容器分类。它还会捕获该 JSON 边界使用的每一个结构与计量 intrinsic,以无原型对象创建属性描述符,并绕过可变集合原型管理私有遍历状态;因此,模型对全局对象、原型方法或 `Object.prototype` 上形似描述符字段的修改,都无法改变验证、wire 传输或字节计量。值会展平为有深度上限的前序 wire 值,供 structured clone 使用,并在另一侧迭代式重建。它们没有字节、JavaScript 调用栈或嵌套 structured-clone 深度上限,绝不会进入外层输出账本或模型上下文;上限仍来自提供方/执行器获取限制与进程/worker 内存。 -- **日志主动流入一个外层账本**:console/stdout/stderr 文本按发送顺序穿过端口,因此超时或被终止的程序仍会显示已经打印的内容。worker 会按 JSON 字符串精确计费,并在发送完成值和异常诊断前,根据组合预算的剩余量预检;因此,抛出的百万字节 stack 会在 worker 边界变成固定的 `output-limit` 诊断。绕过补丁 stream 槽的原生写入会到达独立于完成端口的 pipe,因此宿主会针对这些字节和不可信伪造通信再次执行账本统计;在物化结果前,结算过程会持续进行有界 pipe 捕获,直到 worker 完成终止。`maxOutputBytes` 统计外层 `logs` 数组加完成值或失败消息载荷的 JSON 序列化;固定的 `CodeRunResult` 字段名、花括号、有界错误 kind 标签,以及后续呈现空白不计入这份可变载荷账本。未超过上限时会返回精确值;有损完成值属于 `invalid-output`,组合溢出属于 `output-limit`,不会用 inspected string 代替。失败会保留能容纳的已捕获前缀,之后按普通外层 `run_code` 落盘策略处理。 -- **空环境**:worker 使用 `env: {}` 和 `execArgv: []`,既没有环境凭据(比 spawn 命令的清理环境规则更严格),也不会继承 loader 标志。 -- **释放资源时等待完全停稳**:清理会把进行中的运行标记为 `abort`,并在 resolve 前等待每个 worker 退出。 +- **绑定调用被拒绝时使用的异常类属于请求数据**:可选命名空间描述符会指定构造器全局变量,以及用于接收调用失败的成员名称的自有属性。worker 会创建并注入该真实类,使 `instanceof` 生效,同时无需硬编码 `tools` 或 `ToolCallError`;全局变量无效或冲突的声明会在启动 worker 前失败。失败路径使用模块捕获的错误 intrinsic 与属性定义 intrinsic,以及 null-prototype 描述符,因此模型之后的修改无法把被拒绝的绑定变成 worker 崩溃。 +- **两个独立预算,因为对端不可信**:`computeMs` 统计 worker 实际测得的忙碌时间(轮询 `worker.performance.eventLoopUtilization()`);热循环无法借助待完成的诱饵 dispatch 隐藏,程序等待慢工具时则不累计。`maxWallMs` 为忙碌时间无法观测的情况兜底(例如等待永远不会 resolve 的 promise)。二者最终都会调用 `worker.terminate()`,连同步热循环也能终止;堆溢出会表现为 worker 的 OOM 退出(`kind: 'worker-exit'`)。`maxWallMs` 在加载时会对照 `MAX_TIMER_DELAY_MS` 做范围校验:`setTimeout` 会把更长的延迟限制为 1 ms,仅有正数校验会放行一个在第一个 tick 就到期的上限。`computeMs` 不需要这道上界,因为它对照的是实测占用率,而不是喂给定时器。 +- **中间绑定值是完整 JSON**:绑定参数与 resolve 值会接受迭代式无损 JSON 验证。程序执行前,worker 会捕获自己 realm 中的普通容器原型身份,以及只用于外部 realm 的原生函数源码检查,因此构造器槽修改和用户编写的仿冒对象都无法改变容器分类。它还会捕获该 JSON 边界使用的每一个结构与计量 intrinsic,以无原型对象创建属性描述符,并绕过可变集合原型管理私有遍历状态;因此,模型对全局对象、原型方法或 `Object.prototype` 上形似描述符字段的修改,都无法改变验证、wire 传输或字节计量。值会展平为自身嵌套深度有界的前序 wire 值,供 structured clone 使用,并在另一侧迭代式重建。它们没有字节、JavaScript 调用栈或嵌套 structured-clone 深度上限,绝不会进入外层输出账本或模型上下文;上限仍来自提供方/执行器获取限制与进程/worker 内存。 +- **日志主动流入一个外层账本**:console/stdout/stderr 文本按产生顺序经端口传输,因此超时或被终止的程序仍会显示已经打印的内容。worker 会精确统计 JSON 字符串的字节数,并在发送完成值和异常诊断前,根据组合预算的剩余量预检;因此,抛出的百万字节 stack 会在 worker 边界变成固定的 `output-limit` 诊断。绕过补丁 stream 槽的原生写入会到达独立于完成端口的 pipe,因此宿主会针对这些字节和不可信伪造通信再次执行账本统计;在物化结果前,结算过程会持续进行有界 pipe 捕获,直到 worker 完成终止。`maxOutputBytes` 统计外层 `logs` 数组加完成值或失败消息载荷的 JSON 序列化;固定的 `CodeRunResult` 字段名、花括号、有界错误 kind 标签,以及后续呈现空白不计入这份可变载荷账本。未超过上限时会返回精确值;有损完成值属于 `invalid-output`,组合溢出属于 `output-limit`,不会用 inspected string 代替。失败会保留能容纳的已捕获前缀,之后按普通外层 `run_code` 落盘策略处理。 +- **空环境**:worker 使用 `env: {}` 和 `execArgv: []`,既不会获得环境变量中的凭据(比 spawn 命令的清理环境规则更严格),也不会继承 loader 标志。 +- **dispose(资源释放)时等待完全停稳**:清理会使进行中的运行以 `abort` 失败,并会等待每个 worker 退出后再完成。 ## 未构建与已构建的 worker 入口 -源代码模式通过 Node 原生类型剥离加载只包含可擦除语法的 `src/worker.ts`。其传递运行时闭包只包含 Node 内置模块和相对源模块,因此全新 checkout 绝不需要兄弟工作区包尚未构建的 `lib/` 导出。worker 本地 JSON 快照器会与会话自有的规范边界执行一致性测试;消息端口两侧都会展平并重建已验证值,使应用嵌套永远不会进入 structured clone。构建模式会把兄弟文件 `lib/worker.cjs` 作为文件系统路径传入,因为 pkg 的 VFS Worker hook 要求 CommonJS;同一路径也可在普通 Node 下使用。`tests/built-lib.e2e.ts` 固定了 [docs/testing.md](../../../docs/testing.md) 要求的真实加载路径。 +源代码模式通过 Node 原生类型剥离加载只包含可擦除语法的 `src/worker.ts`。其传递运行时闭包只包含 Node 内置模块和相对源模块,因此全新 checkout 绝不需要兄弟工作区包尚未构建的 `lib/` 导出。worker 本地和会话自有的 JSON 边界都会在消息端口周围展平并重建已验证值,使应用嵌套永远不会进入 structured clone。构建模式会把兄弟文件 `lib/worker.cjs` 作为文件系统路径传入,因为 pkg 的虚拟文件系统(VFS)Worker hook 要求 CommonJS;同一路径也可在普通 Node 下使用。演练这个已发布入口路径的仓库级要求由[测试策略](../../../docs/testing.md)规定。 -SDK 接口是默认/具名 `WorkerCodeRuntime` 类与 `Config`。可操作的 `./worker` 子路径仅作为打包后的 spawn 入口存在;wire 协议与启动辅助模块是源代码私有的实现细节。 +SDK 对外提供默认及具名导出的 `WorkerCodeRuntime` 类,以及 `Config`。运行所用的 `./worker` 子路径仅作为打包后的 spawn 入口存在;wire 协议与启动辅助模块是源代码私有的实现细节。 ## 模型体验 @@ -42,13 +42,13 @@ SDK 接口是默认/具名 `WorkerCodeRuntime` 类与 `Config`。可操作的 #### KV Cache 影响 -不会直接失效;由具名消费方负责请求前缀变更。 +不会直接失效;由上述消费方负责请求前缀变更。 -## 已知限制与暂缓工作 +## 已知限制与暂缓事项 -- **程序 spawn 的 OS 进程在终止后仍会存活**:`worker.terminate()` 只结束线程;需要清理进程树的部署应选择 `dsh-code-runtime-subprocess`,由其挂载的子进程提供方负责该清理。 -- **类型剥离依赖 Node 的实验性 `stripTypeScriptTypes` API**:依赖的行为由单元测试固定;如其发生变化,amaro/sucrase 是已经点名的直接替代品。 +- **程序派生的 OS 进程在程序终止后仍会存活**:`worker.terminate()` 只结束线程,比 bash-local 的进程组终止更弱;在容器后端出现前,孤儿进程清理属于部署职责。 +- **类型剥离依赖 Node 的实验性 `stripTypeScriptTypes` API**:如依赖的行为发生变化,amaro 或 sucrase 是已经点名的直接替代品。 - **`computeMs` 到期最多可能超过一个轮询间隔**:系统每 25 ms 采样一次忙碌时间(内部常量,有意不做成配置)。 -- **程序获得一个含 5 种方法的 `console` shim**(`log`/`info`/`warn`/`error`/`debug`):有意不提供 Node 的完整 console 接口。 +- **程序获得一个含 5 个方法的 `console` shim**(`log`/`info`/`warn`/`error`/`debug`):有意不提供 Node 的完整 console 接口。 - **中间绑定值没有字节上限**:程序可以用永远不会成为外层输出的值耗尽进程或 worker 内存。 - **默认 64 MiB 是拒绝边界,不是可恢复存储**:外层落盘只能保存发生 `output-limit` 后返回的有界日志和诊断;在运行时上限之外被拒绝的字节永远不会到达落盘层。 diff --git a/packages/code-runtime/code-runtime-worker/src/index.ts b/packages/code-runtime/code-runtime-worker/src/index.ts index 2ece0b5c39..43801899a2 100644 --- a/packages/code-runtime/code-runtime-worker/src/index.ts +++ b/packages/code-runtime/code-runtime-worker/src/index.ts @@ -21,10 +21,6 @@ import { jsonStringBytesUpTo, jsonValueBytesUpTo, truncateJsonStringBytes } from import { decodeWorkerJson, encodeWorkerJson } from './worker-json.ts' import type { WorkerJsonWire } from './worker-json.ts' -export { jsonStringBytesUpTo, jsonValueBytesUpTo, truncateJsonStringBytes } from './output-json.ts' -export { decodeWorkerJson, encodeWorkerJson } from './worker-json.ts' -export type { WorkerJsonWire } from './worker-json.ts' - /** Plugin config: every execution cap, changeable from `cordis.yml` (no hardcoded tunables). */ export interface Config { /** @@ -169,19 +165,14 @@ function parseWorkerMessage(raw: unknown): WorkerToHost | undefined { } -/** Shared outer-output accounting for one isolated run; binding values never enter it. */ -export class OutputLedger { +/** One run's combined outer-output ledger; binding values never enter it. */ +class OutputLedger { private bytes = 2 // JSON serialization of the empty logs array: [] private entries = 0 constructor(private readonly maxBytes: number) {} - /** - * Admit one exact log entry, or report that the hard cap was crossed. - * @param text - Candidate log entry. - * @param sink - Accepted log entries for the current run. - * @returns Whether the complete entry fits the remaining outer-output budget. - */ + /** Admit one exact log entry, or report that the hard cap was crossed. */ admit(text: string, sink: string[]): boolean { const separatorBytes = this.entries > 0 ? 1 : 0 const stringBytes = jsonStringBytesUpTo(text, this.maxBytes - this.bytes - separatorBytes) @@ -192,33 +183,19 @@ export class OutputLedger { return true } - /** - * Finalize a successful absent-or-JSON completion against the combined cap. - * @param logs - Already accepted log entries. - * @param value - Optional lossless-JSON completion value. - * @returns A success result or an output-limit failure. - */ + /** Finalize a successful absent-or-JSON completion against the combined cap. */ success(logs: string[], value?: CodeJsonValue): CodeRunResult { if (value !== undefined && jsonValueBytesUpTo(value, this.maxBytes - this.bytes) === undefined) return this.limit(logs) return { logs, ...value !== undefined ? { value } : {} } } - /** - * Finalize a failure diagnostic, with output-limit taking precedence when combined bytes exceed the cap. - * @param logs - Already accepted log entries. - * @param error - Candidate failure diagnostic. - * @returns The diagnostic result or an output-limit failure. - */ + /** Finalize a failure diagnostic, with output-limit taking precedence when combined bytes exceed the cap. */ failure(logs: string[], error: CodeRunFailure): CodeRunResult { if (jsonStringBytesUpTo(error.message, this.maxBytes - this.bytes) === undefined) return this.limit(logs) return { logs, error } } - /** - * Build the explicit output-limit failure while retaining a fitting prefix of the final log. - * @param logs - Candidate log entries in original order. - * @returns A capped output-limit result. - */ + /** Build the explicit output-limit failure while retaining a fitting prefix of the final log. */ limit(logs: string[]): CodeRunResult { const fullMessage = `outer output exceeded ${this.maxBytes} bytes` // The fixed diagnostic is ASCII, so every character is one byte plus the quotes. diff --git a/packages/code-runtime/code-runtime/README.i18n.yaml b/packages/code-runtime/code-runtime/README.i18n.yaml index 51c1466872..a6bd209c7b 100644 --- a/packages/code-runtime/code-runtime/README.i18n.yaml +++ b/packages/code-runtime/code-runtime/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/code-runtime/code-runtime/README.md -README.md: c7690412d0f1bc8556fc758da4e94c6ce5a08d8f -README.zh.md: ecfa48a97c46113c52f13c7a6edfc5d6fbbc2285 +README.md: bb1c20d00a260f643f601c42c6e48722437d5aab +README.zh.md: 15fbcecf77b2318acf3b09101802cd032ae426d2 diff --git a/packages/code-runtime/code-runtime/README.md b/packages/code-runtime/code-runtime/README.md index 4018cdf508..bb1c20d00a 100644 --- a/packages/code-runtime/code-runtime/README.md +++ b/packages/code-runtime/code-runtime/README.md @@ -34,5 +34,5 @@ No direct invalidation; the named consumer owns any request-prefix changes. - **`run()` is one-shot** — `logs` arrive only on the resolved `CodeRunResult`; the seam exposes no streaming-log or progress surface for a live program's output. - **A persistent REPL-style kernel is recorded future work** — the no-state-between-runs contract stands until a persistent-kernel backend brings its own logging story ([Code Mode Agent Note](../../../.agents/notes/implemented/feature/2026-06-15-code-mode.md)). -- **Isolation is backend-specific** — the worker backend is process-local, while the E2B backend reports `container` and keeps orchestration and bindings on the host; the descriptor remains informational rather than a security claim. +- **Only the worker-thread backend ships** — `'process'`/`'container'` are declared well-known `isolation` values with no implementation; a hard security boundary awaits a container backend. - **Intermediate binding values have no byte cap** — implementations remain subject to structured-clone cost and process memory, while a provider or executor may already have imposed its own acquisition bound. diff --git a/packages/code-runtime/code-runtime/README.zh.md b/packages/code-runtime/code-runtime/README.zh.md index ecfa48a97c..15fbcecf77 100644 --- a/packages/code-runtime/code-runtime/README.zh.md +++ b/packages/code-runtime/code-runtime/README.zh.md @@ -4,21 +4,23 @@ 这是**代码执行 seam**:抽象的 `CodeRuntime` 服务(`ctx.codeRuntime`)只定义代码运行时做什么,即针对宿主提供的一组异步绑定运行一段模型编写的程序,并报告 `{ value, logs, error? }`,而不规定如何实现。 -此包是该能力的接口层(以 bash 三包结构为模板,参见[能力 seam](../../../.agents/notes/implemented/architecture/2026-06-13-capability-seams.md)):实现通过继承 `CodeRuntime` 并注册服务接入;消费方是工具注册表的 Code Mode,它生成面向模型的 SDK,并桥接工具分发。两者都由 [Code Mode Agent Note](../../../.agents/notes/implemented/feature/2026-06-15-code-mode.md) 规定,首个实现是 Node worker 线程后端。运行时不了解工具或会话:调用方只向它提供具名异步函数与程序字符串;所有工具形状的内容都留在消费方。 +此包承担该能力三个组成部分中的接口职责(以 bash 三包结构为模板,参见[能力 seam](../../../.agents/notes/implemented/architecture/2026-06-13-capability-seams.md)):实现通过继承 `CodeRuntime` 并注册服务接入;消费方是工具注册表的 Code Mode,它生成面向模型的 SDK,并桥接工具分发。这两项职责均由 [Code Mode Agent Note](../../../.agents/notes/implemented/feature/2026-06-15-code-mode.md) 规定,首个实现是 Node worker 线程后端。运行时不了解工具或会话:调用方只向它提供具名异步函数与程序字符串;所有与工具有关的内容都留在消费方。 ## 服务 API(`ctx.codeRuntime`) | 成员 | 语义 | |---|---| -| `run(request)` | 针对请求的绑定执行一段程序。**每一种程序结果都通过 error 字段完成 resolve**:包括解析/转换失败、抛出异常、无效完成值、输出溢出、预算到期、中止或执行基底死亡(由 `CodeRunFailure` 的正交 `kind` 分类表示);只有调用方误用 seam 本身时才 reject(例如资源释放后仍提交运行)。程序作为异步函数的函数体运行,因此顶层 `await`/`return` 可用,无损 JSON 完成值会成为 `result.value`。 | -| `language` | 只读描述符:`run` 期望的源语言(已知值为 `'typescript'`)。仅供参考,不作门禁;生成语言专用呈现的消费方会对该值执行分支,遇到无法呈现的语言时明确失败。 | +| `run(request)` | 针对请求的绑定执行一段程序。**所有程序失败结果都通过 resolve 结果中的 error 字段报告**:包括解析/转换失败、抛出异常、无效完成值、输出溢出、预算到期、中止或执行基底终止(由 `CodeRunFailure` 的正交 `kind` 分类表示);只有调用方误用 seam 本身时才 reject(例如 dispose(资源释放)后仍提交运行)。程序作为异步函数的函数体运行,因此顶层 `await`/`return` 可用,无损 JSON 完成值会成为 `result.value`。 | +| `language` | 只读描述符:`run` 期望的源语言。已知值为 `'typescript'` 与 `'python'`——`dsh-tools` 能呈现的那些;其中只有 `'typescript'` 有已发布的后端。仅供参考,不作门禁;生成语言专用呈现的消费方会根据该值选择分支,遇到无法呈现的语言时明确失败。 | | `isolation` | 只读描述符:执行基底(`'worker-thread'`、`'process'`、`'container'`)。供部署与诊断使用,**不构成安全声明**。 | -每个实现都必须遵守以下语义(完整契约见类 JSDoc):绑定调用会桥接完整的无损 JSON 参数与 resolve 值,seam 层不设字节上限;程序被视为不可信对等方(任意绑定名称都是自有属性,格式错误的通信绝不能使宿主崩溃);不同运行之间不保留任何状态;资源释放会终止进行中的运行,并且在完成前等待其退出。 +每个实现都必须遵守以下语义(完整契约见类 JSDoc):绑定调用会桥接完整的无损 JSON 参数与 resolve 值,seam 层不设字节上限;程序被视为敌对对等方(任意绑定名称都会成为自有属性,格式错误的通信绝不能使宿主崩溃);不同运行之间不保留任何状态;dispose 会终止进行中的运行,并且在完成前等待其退出。 ## 词汇 -`CodeRunRequest`(`program`、`bindings`、`signal?`)携带运行时操作所需的全部内容;默认值解析(时间预算与外层输出上限)属于实现的已验证配置,绝不能是隐藏的 `??`,更不能藏在 `run()` 内部。`bindings` 是 `CodeBindingNamespace` 列表(`global` + `functions` + 可选 `errorClass`);每个命名空间会作为一个由异步可调用函数组成的全局对象公开给程序,这些函数返回 `CodeJsonValue`。后者是 seam 本地、与规范 `JsonValue` 结构等价的类型,使接口包保持独立于会话。`errorClass` 描述符点名真实的程序全局构造器,以及用于接收 reject 成员名称的自有属性;运行时不依赖 `ToolCallError` 等消费方术语。`CodeRunResult` 报告无损 JSON 完成值 `value?`、有序的 `logs: string[]` 和 `error?`(`CodeRunFailure`:`kind` + 可反馈给模型的 `message`)。完整契约见 `src/types.ts`。 +`CodeRunRequest`(`program`、`bindings`、`signal?`)携带运行时操作所需的全部内容;默认值解析(时间预算与外层输出上限)属于实现的已验证配置,绝不能是隐藏的 `??`,更不能藏在 `run()` 内部。`bindings` 是 `CodeBindingNamespace` 列表(`global` + `functions` + 可选 `errorClass`);每个命名空间会作为一个由异步可调用函数组成的全局对象公开给程序,这些函数返回 `CodeJsonValue`。后者是 seam 本地、与规范 `JsonValue` 结构等价的类型,使接口包保持独立于会话。`errorClass` 描述符点名真实的程序全局构造器,以及用于接收被拒绝成员名称的自有属性;运行时不依赖 `ToolCallError` 等消费方术语。`CodeRunResult` 报告无损 JSON 完成值 `value?`、有序的 `logs: string[]` 和 `error?`(`CodeRunFailure`:`kind` + 可反馈给模型的 `message`)。完整契约见 `src/types.ts`。 + +binding-global 与 error-class 名称是**语言可移植**的:必须匹配标识符子集 `[A-Za-z_][A-Za-z0-9_]*`(不含 JS 专有的 `$`)并通过 seam 导出的排除集,因此同一份 `bindings` 列表对每个后端都有效,无论其 `language` 为何。本包导出每个后端都执行的契约——`PORTABLE_RESERVED_WORDS`(ECMAScript ∪ Python 保留字)、`RESERVED_BINDING_GLOBALS`(如 `console` 等后端拥有的 global)、`RESERVED_ERROR_MEMBERS` 与 `DUNDER_MEMBER`(error-member 排除)——因此 `$tools`、`lambda`、`__dsh_main__` 之类的名称会让 `run()` 在任何后端上作为 seam 误用而 reject,而非只在某些后端。确切集合与理由见 `src/index.ts`。 ## 模型体验 @@ -26,11 +28,11 @@ #### KV Cache 影响 -不会直接失效;由具名消费方负责请求前缀变更。 +不会直接失效;由上述消费方负责请求前缀变更。 -## 已知限制与暂缓工作 +## 已知限制与暂缓事项 -- **`run()` 是一次性的**:`logs` 只有在 `CodeRunResult` resolve 后才能获得;seam 不提供活跃程序输出的流式日志或进度接口。 +- **`run()` 是一次性的**:`logs` 只有在 `CodeRunResult` resolve 后才能获得;seam 不提供正在运行的程序所产生输出的流式日志或进度接口。 - **持久 REPL 风格内核已记录为未来工作**:在持久内核后端带来自己的日志方案前,运行之间不保留状态的契约继续有效(参见 [Code Mode Agent Note](../../../.agents/notes/implemented/feature/2026-06-15-code-mode.md))。 -- **隔离方式由后端决定**:worker 后端位于宿主进程内,而 E2B 后端报告 `container`,并把编排与绑定留在宿主;该描述符仍只提供信息,不构成安全声明。 +- **目前只提供 worker 线程后端**:`'process'`/`'container'` 是已经声明但没有实现的已知 `isolation` 值;强安全边界需要等待容器后端。 - **中间绑定值没有字节上限**:实现仍受 structured-clone 成本与进程内存约束,而提供方或执行器可能已经应用自己的获取上限。 diff --git a/packages/cordis/tool-cordis/src/api-catalog.ts b/packages/cordis/tool-cordis/src/api-catalog.ts index 21848e55e2..5bd34a8642 100644 --- a/packages/cordis/tool-cordis/src/api-catalog.ts +++ b/packages/cordis/tool-cordis/src/api-catalog.ts @@ -304,6 +304,16 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [ }, ], }, + { + key: 'e2b', + summary: 'Creates one lazily consumable E2B SDK handle and deletes the sandbox at timeout or disposal.', + methods: [ + { + signature: 'async getSandbox(): Promise', + jsDoc: '/**\n * Return the shared live SDK handle.\n * @returns the created sandbox after the configured cwd exists.\n * @throws when E2B rejects creation or the service is disposing.\n */', + }, + ], + }, { key: 'fs', summary: 'Abstract filesystem provider.', diff --git a/packages/e2b/e2b/tests/composition.e2e.ts b/packages/e2b/e2b/tests/composition.e2e.ts index c09243183e..01316aeb2d 100644 --- a/packages/e2b/e2b/tests/composition.e2e.ts +++ b/packages/e2b/e2b/tests/composition.e2e.ts @@ -3,6 +3,7 @@ import { join, posix } from 'node:path' import { fileURLToPath } from 'node:url' import { Context } from 'cordis' import { describe, expect, it } from 'vitest' +import { Inbox } from '@deepseek-ai/dsh-agent' import type { Agent } from '@deepseek-ai/dsh-agent' import { runLoaderSmoke } from '@deepseek-ai/dsh-loader-smoke' import { @@ -78,18 +79,20 @@ describe.skipIf(!process.env.E2B_API_KEY)('E2B live Loader composition', () => { expect(environmentProbe.collected.stdout?.readFrom(0).text).toBe('DIRECT=<> LEAK=<0>\n') await expect(sandbox.files.read(profileLeakPath)).rejects.toBeInstanceOf(FileNotFoundError) const ownerId = SessionId('e2b-pty-env-owner') + const ownerSession = Session.create(ownerId) const owner: Agent = { id: ownerId, options: {}, - session: new Session(ownerId), + session: ownerSession, + inbox: new Inbox(ownerSession, { inserted: () => {}, discarded: () => {}, claimed: () => {} }), status: 'idle', - acceptsNextStep: false, ctx, - followup() {}, - steer: () => ({ outcome: Promise.resolve({ status: 'rejected' as const }) }), - inject() {}, send() {}, + followup() {}, + steer() {}, + inject() {}, cancel() {}, + runMaintenance: task => task(new AbortController().signal), whenIdle: () => Promise.resolve(), } const backend = new LocalPtyBackend(ctx, { diff --git a/packages/e2b/subprocess-e2b/src/environment.ts b/packages/e2b/subprocess-e2b/src/environment.ts index 8daae9ccfd..7cdb1b7fbf 100644 --- a/packages/e2b/subprocess-e2b/src/environment.ts +++ b/packages/e2b/subprocess-e2b/src/environment.ts @@ -84,19 +84,21 @@ export function bootstrapEnvironment(raw: string): Record { /** * Overlay explicit entries and serialize one validated E2B environment. * @param raw - The complete NUL-delimited remote environment. - * @param explicit - Deliberate caller overrides applied after ambient scrubbing. + * @param explicit - Deliberate caller overrides applied after ambient scrubbing; an `undefined` tombstone removes an ambient entry. * @returns NUL-delimited `name=value` entries accepted by `env -i`. */ export function serializeRemoteEnvironment( raw: string, - explicit: Readonly> | undefined, + explicit: Readonly | undefined, ): string { const environment = scrubRemoteEnvironment(raw) for (const [name, value] of Object.entries(explicit ?? {})) { - if (name.length === 0 || name.includes('=') || name.includes('\0') || value.includes('\0')) { + if (name.length === 0 || name.includes('=') || name.includes('\0') || value?.includes('\0') === true) { throw new Error('subprocess-e2b: environment entries require non-empty NUL-free names without = and NUL-free values') } - environment.set(name, value) + // An explicit undefined is the seam's tombstone: remove the ambient entry. + if (value === undefined) environment.delete(name) + else environment.set(name, value) } return [...environment].map(([name, value]) => `${name}=${value}\0`).join('') } diff --git a/packages/lsp/README.i18n.yaml b/packages/lsp/README.i18n.yaml index be241ada6e..9919cdaba5 100644 --- a/packages/lsp/README.i18n.yaml +++ b/packages/lsp/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/lsp/README.md -README.md: d8bc6047e7eadeb2993f1114564b41913b2970dc -README.zh.md: d838af346626bb96299a3b702df0dabb6231c15a +README.md: 7fbdf071735673fb0158f6fa66148be1c644a433 +README.zh.md: e059dbd80b7e38c0e447e54178162316dfd127c7 diff --git a/packages/lsp/README.md b/packages/lsp/README.md index d8bc6047e7..7fbdf07173 100644 --- a/packages/lsp/README.md +++ b/packages/lsp/README.md @@ -7,10 +7,9 @@ The language-server capability seam: an abstract LSP interface, a generic stdio | Package | Role | ctx key | |---|---|---| | `lsp/` | Abstract LSP seam (provider registry by branded id + extension mapping, per-query selection, vocabulary, `LspError`) | `ctx.lsp` | -| [`lsp-local/`](lsp-local/README.md) | Generic multi-server local backend (spawn, JSON-RPC, transient-open queries) | (registers providers on `ctx.lsp`) | -| [`e2b/lsp-e2b`](../e2b/lsp-e2b/README.md) | Remote E2B backend (remote source reads and servers, byte-framed stdio bridge) | (registers providers on `ctx.lsp`) | +| `lsp-local/` | Generic multi-server stdio backend over `ctx.fs` and `ctx.subprocess` (JSON-RPC, transient-open queries) | (registers providers on `ctx.lsp`) | | `tool-lsp/` | Model-facing `lsp` tool (four operations, one-based UTF-16 cursor coordinates) | (registers on `ctx.tools`) | The interface lives at `lsp/lsp/`. The seam exposes exactly four semantic operations — `goToDefinition`, `findReferences`, `goToImplementation`, `hover` — and no generic JSON-RPC escape hatch, so a provider swap does not change how the model asks for navigation and no protocol payload or unreviewed mutation reaches the model contract. Providers register **capabilities**, not tools; `tool-lsp` is the only owner of the model-facing name, schema, prompt guidance, and presentation. -See the [LSP capability seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md) for the protocol design and the [shared E2B runtime note](../../.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md) for the remote process/filesystem boundary. +See the [LSP capability seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md) for the design rationale, including why documents open transiently per query, why the stdio host consumes the shared filesystem/subprocess execution world, and why extension ownership is exclusive within one runtime. diff --git a/packages/lsp/README.zh.md b/packages/lsp/README.zh.md index d838af3466..e059dbd80b 100644 --- a/packages/lsp/README.zh.md +++ b/packages/lsp/README.zh.md @@ -7,10 +7,9 @@ | 包 | 职责 | ctx key | |---|---|---| | `lsp/` | 抽象 LSP seam(按品牌化 id + 扩展名映射组织的提供方注册表、逐查询选择、词汇、`LspError`) | `ctx.lsp` | -| [`lsp-local/`](lsp-local/README.md) | 通用多服务器本地后端(spawn、JSON-RPC、临时打开查询) | (在 `ctx.lsp` 上注册提供方) | -| [`e2b/lsp-e2b`](../e2b/lsp-e2b/README.md) | 远程 E2B 后端(在远程读取源文件并运行服务器、采用字节分帧的 stdio 桥) | (在 `ctx.lsp` 上注册提供方) | +| `lsp-local/` | 基于 `ctx.fs` 与 `ctx.subprocess` 的通用多服务器 stdio 后端(JSON-RPC、临时打开查询) | (在 `ctx.lsp` 上注册提供方) | | `tool-lsp/` | 面向模型的 `lsp` 工具(四种操作、从 1 开始的 UTF-16 光标坐标) | (注册到 `ctx.tools`) | 接口位于 `lsp/lsp/`。该 seam 恰好公开四种语义操作:`goToDefinition`、`findReferences`、`goToImplementation`、`hover`,且不提供通用 JSON-RPC 逃生口;因此,替换提供方不会改变模型请求导航的方式,也不会让协议载荷或未经评审的修改进入模型契约。提供方注册的是**能力** 而非工具;`tool-lsp` 是面向模型名称、schema、提示词指引和呈现的唯一 owner。 -协议设计见 [LSP 能力 seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md),远程进程/文件系统边界见 [共享 E2B 运行时 Agent Note](../../.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md)。 +设计原理见 [LSP 能力 seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md),其中也解释了文档为何在每次查询时临时打开、stdio 主机为何使用共享的文件系统/子进程执行环境,以及扩展名归属为何在同一运行时内互斥。 diff --git a/packages/lsp/lsp-local/src/index.ts b/packages/lsp/lsp-local/src/index.ts index 3a2baa0c55..fc7e50787f 100644 --- a/packages/lsp/lsp-local/src/index.ts +++ b/packages/lsp/lsp-local/src/index.ts @@ -1,19 +1,16 @@ /** * Generic stdio language-server backend for `ctx.lsp`. One plugin instance configures a named table * of server commands and registers one isolated provider for each entry. Every provider lazily - * single-flights one server process per canonical workspace realpath, serves transient-open queries + * single-flights one server process per canonical workspace target, serves transient-open queries * through it, and replaces a selected transport that fails before or during the next read-only - * query. Providers read sources through Node APIs in the host namespace (not `ctx.fs`) - * and trust their configured servers — no sandbox confinement. + * query. Providers read sources through `ctx.fs` and launch servers through + * `ctx.subprocess`, so both local and remote implementations share one host. * * Namespace plugin (named exports, no default export). Lifecycle is effect-scoped: disposal * unregisters from `ctx.lsp` and tears down every live server. * @module @deepseek-ai/dsh-lsp-local */ -import { accessSync, constants, statSync } from 'node:fs' -import { delimiter, isAbsolute, join } from 'node:path' -import { pathToFileURL } from 'node:url' import type { Context } from 'cordis' import z from 'schemastery' import { LspError, LspProviderId } from '@deepseek-ai/dsh-lsp' @@ -25,9 +22,9 @@ import type { import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout' import { abortable, abortError } from './abort.ts' import { canonicalizeWorkspace, readHostSource } from './host.ts' +import type { HostWorkspace } from './host.ts' import { LspInstance } from './instance.ts' import type { ConnectionSpawner } from './connection.ts' -import { scrubbedParentEnv } from '@deepseek-ai/dsh-subprocess' import type { InstanceSpec } from './instance.ts' export { canonicalizeWorkspace, readHostSource } from './host.ts' @@ -47,10 +44,7 @@ export { LspConnection } from './connection.ts' export const name = 'lsp-local' /** Services required by this plugin. */ -export const inject = ['lsp', 'subprocess'] - -/** Credential-shaped ambient env vars are NOT forwarded to the child by default. */ - +export const inject = ['fs', 'lsp', 'subprocess'] const DEFAULT_MAX_MESSAGE_BYTES = 16_000_000 const DEFAULT_MAX_STDERR_BYTES = 1_000_000 @@ -92,6 +86,7 @@ export interface Config { /** One server config after schemastery fills every default. */ type ResolvedServerConfig = Required +type WorkspaceKey = HostWorkspace['target']['targetKey'] const LspLocalServerConfig: z = z.object({ command: z.string().required(), @@ -111,27 +106,67 @@ export const Config: z = z.object({ servers: z.dict(LspLocalServerConfig).required(), }) +/** Propagate teardown failures only after every sibling has settled. */ +function throwTeardownFailures(results: readonly PromiseSettledResult[], message: string): void { + const failures: unknown[] = [] + for (const result of results) { + if (result.status === 'rejected') failures.push(result.reason) + } + if (failures.length === 1) throw failures[0] + if (failures.length > 1) throw new AggregateError(failures, message) +} + /** * Register the configured stdio LSP providers. Resolves every executable at load (after credential * scrubbing) before publishing any provider; each process launches lazily on its first matching * query. - * @param ctx - the plugin context (must inject `lsp`). + * @param ctx - the plugin context carrying `fs`, `lsp`, and `subprocess`. * @param config - the resolved plugin configuration (schemastery has filled every default). */ -export function apply(ctx: Context, config: Config): void { +export async function apply(ctx: Context, config: Config): Promise { const entries = Object.entries(config.servers) if (entries.length === 0) throw new Error('lsp-local: servers must contain at least one server') + const setupAbort = new AbortController() + const stopSetupCancellation = ctx.on('internal/plugin', (fiber) => { + // An async plugin callback must observe its own disposal before Cordis can + // run effect cleanup, because unload otherwise waits for this callback. + if (fiber === ctx.fiber && fiber.uid === null) { + setupAbort.abort(new Error('lsp-local setup disposed')) + } + }) + // Resolve every server-local setting before registration so a bad later command or bound cannot // publish an earlier provider. Registry-level mapping conflicts are rolled back below. - const providers = entries.map(([providerId, rawConfig]) => { - if (providerId.trim() === '') throw new Error('lsp-local: server ids must be non-empty strings') - const resolved = rawConfig as ResolvedServerConfig - validateServerConfig(providerId, resolved) - const childEnv = buildChildEnv(resolved.env) - const executable = resolveExecutable(resolved.command, childEnv) - return new LocalLspProvider(providerId, resolved, childEnv, executable, spec => ctx.subprocess.spawn(spec)) - }) + const providers = await (async () => { + const lookups = entries.map(async ([providerId, rawConfig]) => { + if (providerId.trim() === '') throw new Error('lsp-local: server ids must be non-empty strings') + const resolved = rawConfig as ResolvedServerConfig + validateServerConfig(providerId, resolved) + const executable = await ctx.subprocess.resolveExecutable( + resolved.command, + resolved.env, + setupAbort.signal, + ) + setupAbort.signal.throwIfAborted() + return new LocalLspProvider( + providerId, + ctx.fs, + resolved, + executable, + spec => ctx.subprocess.spawn(spec), + ) + }) + try { + return await Promise.all(lookups) + } catch (error: unknown) { + setupAbort.abort(error) + await Promise.allSettled(lookups) + throw error + } finally { + stopSetupCancellation() + } + })() ctx.effect(() => { const disposers: Array<() => void> = [] @@ -144,7 +179,8 @@ export function apply(ctx: Context, config: Config): void { return async () => { // Remove every route before process teardown so no new query can enter a draining provider. for (const dispose of disposers.reverse()) dispose() - await Promise.all(providers.map(provider => provider.disposeAll())) + const results = await Promise.allSettled(providers.map(provider => provider.disposeAll())) + throwTeardownFailures(results, 'lsp-local provider teardown failed') } }, 'lsp-local.registerProviders') } @@ -181,16 +217,19 @@ function assertPositiveInteger(providerId: string, name: string, value: number): class LocalLspProvider implements LspProvider { readonly id: LspProviderId readonly extensionToLanguage: Readonly> - /** One live instance per canonical workspace realpath. */ - private readonly instances = new Map() + /** One live instance per stable canonical workspace identity. */ + private readonly instances = new Map() /** One complete source-read→open→query→close serialization tail per canonical workspace. */ - private readonly queues = new Map>() + private readonly queues = new Map>() + /** Workspace canonicalizations that have not entered a provider-owned queue yet. */ + private readonly workspaceLookups = new Set>() + private readonly lifetime = new AbortController() private disposed = false constructor( providerId: string, + private readonly fs: Context['fs'], private readonly config: ResolvedServerConfig, - private readonly childEnv: Record, private readonly executable: string, private readonly spawner: ConnectionSpawner, ) { @@ -211,43 +250,60 @@ class LocalLspProvider implements LspProvider { if (signal?.aborted) throw abortError(signal) } + /** Fuse caller cancellation with provider disposal for every filesystem and protocol await. */ + private querySignal(signal?: AbortSignal): AbortSignal { + return signal === undefined + ? this.lifetime.signal + : AbortSignal.any([signal, this.lifetime.signal]) + } + async query(request: LspProviderQuery, signal?: AbortSignal): Promise { - // Honor an already-aborted signal before host I/O so a canceled request never starts a server. + // Honor an already-aborted signal before provider I/O so a canceled request never starts a server. this.assertActive(signal) - const workspace = await canonicalizeWorkspace(request.workspaceRoot, signal) - this.assertActive(signal) - return this.enqueue(workspace, signal, async () => { - this.assertActive(signal) + const querySignal = this.querySignal(signal) + const workspaceResult = canonicalizeWorkspace(this.fs, request.workspaceRoot, querySignal) + const workspaceLookup = workspaceResult.then(() => undefined, () => undefined) + this.workspaceLookups.add(workspaceLookup) + let workspace: HostWorkspace + try { + workspace = await workspaceResult + } finally { + this.workspaceLookups.delete(workspaceLookup) + } + this.assertActive(querySignal) + const workspaceKey = workspace.target.targetKey + return this.enqueue(workspaceKey, querySignal, async () => { + this.assertActive(querySignal) // Read inside the workspace queue but before spawning: a queued query sees current bytes when // its turn starts, while an invalid source still cannot leave an idle process pooled. - const source = await readHostSource(request.filePath, workspace, this.config.maxDocumentBytes, signal) + const source = await readHostSource(this.fs, request.filePath, workspace, this.config.maxDocumentBytes, querySignal) // Disposal may have snapshotted the instance map while host I/O was pending. Re-check before a // synchronous get-or-create so every spawned process remains owned by teardown. - this.assertActive(signal) - let instance = this.instanceFor(workspace) + this.assertActive(querySignal) + let instance = this.instanceFor(workspaceKey, workspace) try { - return await instance.query(request, source, signal) + return await instance.query(request, source, querySignal) } catch (error) { // A selected child can have died while idle or fail during the next write. Queries are // read-only, so replace that transport once and retry transparently. if (!instance.isTransportFailure(error)) throw error await instance.dispose() - this.evictIfCurrent(workspace, instance) - this.assertActive(signal) - instance = this.instanceFor(workspace) - return await instance.query(request, source, signal) + this.evictIfCurrent(workspaceKey, instance) + this.assertActive(querySignal) + instance = this.instanceFor(workspaceKey, workspace) + return await instance.query(request, source, querySignal) } finally { // Reach quiescence before dropping a dead slot; a replacement must survive this ownership check. if (instance.dead) { await instance.dispose() - this.evictIfCurrent(workspace, instance) + this.evictIfCurrent(workspaceKey, instance) } } }) } /** Serialize one complete query lifecycle for a canonical workspace. */ - private enqueue(workspace: string, signal: AbortSignal | undefined, run: () => Promise): Promise { + private enqueue(workspace: WorkspaceKey, signal: AbortSignal | undefined, run: () => Promise): Promise { const previous = this.queues.get(workspace) ?? Promise.resolve() const result = abortable(previous, signal).then(run) // The tail follows the actual prior work even when this caller aborts its wait. It never rejects, @@ -261,34 +317,34 @@ class LocalLspProvider implements LspProvider { } /** Return or synchronously publish the one instance for a canonical workspace. */ - private instanceFor(workspace: string): LspInstance { + private instanceFor(workspaceKey: WorkspaceKey, workspace: HostWorkspace): LspInstance { this.assertActive() - const existing = this.instances.get(workspace) + const existing = this.instances.get(workspaceKey) if (existing !== undefined) return existing const created = this.createInstance(workspace) - this.instances.set(workspace, created) + this.instances.set(workspaceKey, created) return created } /** Drop the slot iff it still contains this instance. */ - private evictIfCurrent(workspace: string, instance: LspInstance): void { + private evictIfCurrent(workspace: WorkspaceKey, instance: LspInstance): void { /* v8 ignore next -- mismatch requires another query to replace the slot before this finally runs. */ if (this.instances.get(workspace) === instance) this.instances.delete(workspace) } - private createInstance(workspace: string): LspInstance { + private createInstance(workspace: HostWorkspace): LspInstance { const spec: InstanceSpec = { command: this.executable, args: this.config.args, - cwd: workspace, - env: this.childEnv, + cwd: workspace.canonicalPath, + workspaceUri: workspace.fileUrl, + env: this.config.env, configuration: this.config.configuration, initializationOptions: this.config.initializationOptions, maxMessageBytes: this.config.maxMessageBytes, maxStderrBytes: this.config.maxStderrBytes, shutdownTimeoutMs: this.config.shutdownTimeoutMs, killGraceMs: this.config.killGraceMs, - pathToFileUri: path => pathToFileURL(path).href, } return new LspInstance(spec, this.spawner) } @@ -296,51 +352,18 @@ class LocalLspProvider implements LspProvider { /** Dispose every live instance and block further queries. */ async disposeAll(): Promise { this.disposed = true + this.lifetime.abort(new LspError('lsp-local provider is disposed', 'LSP_DISPOSED')) const live = [...this.instances.values()] const draining = [...this.queues.values()] + const resolving = [...this.workspaceLookups] this.instances.clear() - await Promise.all([ + const results = await Promise.allSettled([ ...live.map(instance => instance.dispose()), ...draining, + ...resolving, ]) this.queues.clear() - } -} - -/** The seam's scrubbed parent env (credential-shaped and DSH_* names dropped), plus the config's explicit env. */ -function buildChildEnv(extra: Record): Record { - return { ...scrubbedParentEnv(), ...extra } -} - -/** - * Resolve the server executable to an absolute path: an absolute command is verified directly; a - * bare command is looked up on the child's PATH. Fails loudly when nothing is executable. - */ -function resolveExecutable(command: string, childEnv: Record): string { - if (isAbsolute(command)) { - // Verify an absolute command too, so an unavailable one fails at load, not on the first query. - if (!isExecutableFileSync(command)) { - throw new Error(`lsp-local: command "${command}" is not an executable file`) - } - return command - } - /* v8 ignore next -- buildChildEnv always sets PATH from the ambient env; the further fallbacks are defensive. */ - const pathValue = childEnv.PATH ?? process.env.PATH ?? '' - for (const dir of pathValue.split(delimiter)) { - if (dir === '') continue - const candidate = join(dir, command) - if (isExecutableFileSync(candidate)) return candidate - } - throw new Error(`lsp-local: command "${command}" was not found on PATH`) -} - -/** Synchronous regular-file and executable check used only at load-time resolution. */ -function isExecutableFileSync(path: string): boolean { - try { - if (!statSync(path).isFile()) return false - accessSync(path, constants.X_OK) - return true - } catch { - return false + this.workspaceLookups.clear() + throwTeardownFailures(results, 'lsp-local instance teardown failed') } } diff --git a/packages/lsp/lsp-local/src/instance.ts b/packages/lsp/lsp-local/src/instance.ts index 5a12192c62..318704b9fc 100644 --- a/packages/lsp/lsp-local/src/instance.ts +++ b/packages/lsp/lsp-local/src/instance.ts @@ -30,18 +30,12 @@ import { /** Everything an instance needs beyond the connection spec. */ export interface InstanceSpec extends ConnectionSpec { + /** Canonical workspace file URI supplied by the filesystem provider. */ + readonly workspaceUri: string /** Static `initialize` options forwarded to the server. */ readonly initializationOptions: unknown /** Graceful `shutdown`/`exit` budget before escalation (ms). */ readonly shutdownTimeoutMs: number - /** PID advertised to the server; `null` when client and server do not share a process namespace. */ - readonly clientProcessId?: number | null - /** - * Encode one implementation-native absolute path as a file URI. - * @param path - Canonical workspace or source path. - * @returns A file URI interpreted in the server's filesystem namespace. - */ - readonly pathToFileUri: (path: string) => string } /** @@ -115,9 +109,11 @@ export class LspInstance { private async initialize(): Promise { const initializeResult = await this.connection.request('initialize', { - processId: this.spec.clientProcessId === undefined ? process.pid : this.spec.clientProcessId, - rootUri: this.spec.pathToFileUri(this.spec.cwd), - workspaceFolders: [{ uri: this.spec.pathToFileUri(this.spec.cwd), name: 'workspace' }], + // A subprocess provider may run in another PID namespace or machine; + // the host PID would let the server monitor an unrelated process. + processId: null, + rootUri: this.spec.workspaceUri, + workspaceFolders: [{ uri: this.spec.workspaceUri, name: 'workspace' }], capabilities: CLIENT_CAPABILITIES, initializationOptions: this.spec.initializationOptions, }) as WireInitializeResult @@ -154,7 +150,7 @@ export class LspInstance { throw new LspError('server does not support the transient textDocument/didOpen this host requires', 'LSP_UNSUPPORTED_OPERATION') } - const uri = this.spec.pathToFileUri(source.canonicalPath) + const uri = source.fileUrl let opened = false try { /* v8 ignore next -- guards an abort landing between the ready wait and didOpen; not deterministically reproducible. */ @@ -248,10 +244,9 @@ export class LspInstance { if (operation === 'hover') { return { kind: 'hover', hover: normalizeHover(payload) } } - // `spec.cwd` is the canonical workspace realpath (the provider canonicalizes before spawning), - // and every `file:` location URI is relative to it — so it is the root a caller must relativize - // display paths against, not the request's possibly-symlinked workspaceRoot. - return { kind: 'locations', locations: normalizeLocations(payload), resolvedWorkspaceRoot: this.spec.cwd } + // The filesystem provider owns URI syntax for the execution platform, which may differ from the + // harness host. Preserve that coordinate through rendering instead of reparsing `spec.cwd` there. + return { kind: 'locations', locations: normalizeLocations(payload), resolvedWorkspaceUri: this.spec.workspaceUri } } private answerServerRequest(method: string, params: unknown): Promise { diff --git a/packages/lsp/lsp-local/tests/instance.spec.ts b/packages/lsp/lsp-local/tests/instance.spec.ts index 22877a94c8..08bad67ae2 100644 --- a/packages/lsp/lsp-local/tests/instance.spec.ts +++ b/packages/lsp/lsp-local/tests/instance.spec.ts @@ -56,7 +56,6 @@ function makeInstance( maxStderrBytes: 100_000, shutdownTimeoutMs: 200, killGraceMs: 200, - pathToFileUri: path => pathToFileURL(path).href, ...overrides, }, spawnSubprocess, writer) live.push(instance) @@ -92,7 +91,6 @@ function scriptInstance(script: string, overrides: Partial = {}): maxStderrBytes: 100_000, shutdownTimeoutMs: 150, killGraceMs: 150, - pathToFileUri: path => pathToFileURL(path).href, ...overrides, }, spawnSubprocess) live.push(instance) diff --git a/packages/lsp/lsp/README.i18n.yaml b/packages/lsp/lsp/README.i18n.yaml index e63252c7f8..02649a8d39 100644 --- a/packages/lsp/lsp/README.i18n.yaml +++ b/packages/lsp/lsp/README.i18n.yaml @@ -1,6 +1,6 @@ # Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: -# pnpm run verify-translation-pairing --write -README.md: f96fc67ec8cb95f423eff9b312b7b591ec9d3008 -README.zh.md: 13ae9700e284ff238147538a571622066efc5747 +# pnpm run verify-translation-pairing --write packages/lsp/lsp/README.md +README.md: 5c1044be50368acf13d8c36a15d5b2bd99d02701 +README.zh.md: cc412333e63b9469319240d67269bf0192ad3858 diff --git a/packages/pty/README.i18n.yaml b/packages/pty/README.i18n.yaml index 2dff12e3da..65d541a7c1 100644 --- a/packages/pty/README.i18n.yaml +++ b/packages/pty/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/pty/README.md -README.md: a6706ed653bc60909a23b3598c41bb10ef499cbe -README.zh.md: 01fecf8d57d933168e91331c4d3c3e4666c13cdc +README.md: a4f743056b4a524be9623b0f700f37e0534b463f +README.zh.md: c84ad3f1b59afcdbbd111f1b82c57c56aa24fdcf diff --git a/packages/pty/README.md b/packages/pty/README.md index a6706ed653..a4f743056b 100644 --- a/packages/pty/README.md +++ b/packages/pty/README.md @@ -7,8 +7,7 @@ English | [中文](README.zh.md) | Package | Role | ctx key | |---|---|---| | [`pty`](pty/README.md) (`@deepseek-ai/dsh-pty`) | Backend registry, branded ids, exact-Agent ownership, session operations, and awaited cleanup | `ctx.pty` | -| [`pty-local`](pty-local/README.md) (`@deepseek-ai/dsh-pty-local`) | Local `node-pty` backend, readiness detection, bounded terminal state, sandboxing, and process-session supervision | registers on `ctx.pty` | -| [`e2b/pty-e2b`](../e2b/pty-e2b/README.md) (`@deepseek-ai/dsh-pty-e2b`) | E2B byte-PTY backend, remote foreground signaling, bounded terminal state, and awaited remote cleanup | registers on `ctx.pty` | +| `pty-local` (`@deepseek-ai/dsh-pty-local`) | Shell backend over `ctx.subprocess.spawnTerminal`: readiness detection, bounded terminal state, sandbox policy, and session operations | registers on `ctx.pty` | | `tool-pty` (`@deepseek-ai/dsh-tool-pty`) | Six model-facing tools and generic task integration for background sends | registers on `ctx.tools` | -The core design lives in the [persistent PTY Agent Note](../../.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md); the remote ownership boundary lives in the [shared E2B runtime note](../../.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md). +The design and deferred boundaries live in the [persistent PTY Agent Note](../../.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md). diff --git a/packages/pty/README.zh.md b/packages/pty/README.zh.md index 01fecf8d57..c84ad3f1b5 100644 --- a/packages/pty/README.zh.md +++ b/packages/pty/README.zh.md @@ -7,8 +7,7 @@ | 包 | 职责 | ctx 键 | |---|---|---| | [`pty`](pty/README.md)(`@deepseek-ai/dsh-pty`) | 后端注册表、品牌化 id、精确的 Agent 所有权、会话操作与等待完成的清理 | `ctx.pty` | -| [`pty-local`](pty-local/README.md)(`@deepseek-ai/dsh-pty-local`) | 本地 `node-pty` 后端、就绪检测、有界终端状态、沙箱与进程会话监管 | 注册到 `ctx.pty` | -| [`e2b/pty-e2b`](../e2b/pty-e2b/README.md)(`@deepseek-ai/dsh-pty-e2b`) | E2B 字节 PTY 后端、远程前台信号传递、有界终端状态与等待完成的远程清理 | 注册到 `ctx.pty` | +| `pty-local`(`@deepseek-ai/dsh-pty-local`) | `ctx.subprocess.spawnTerminal` 之上的 shell 后端:就绪检测、有界终端状态、沙箱策略与会话操作 | 注册到 `ctx.pty` | | `tool-pty`(`@deepseek-ai/dsh-tool-pty`) | 6 个面向模型的工具,并为后台发送集成通用任务 | 注册到 `ctx.tools` | -核心设计记录在[持久 PTY Agent Note](../../.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md) 中;远程所有权边界记录在 [共享 E2B 运行时 Agent Note](../../.agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md) 中。 +设计与暂缓边界记录在[持久 PTY Agent Note](../../.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md) 中。 diff --git a/packages/pty/pty-local/src/sanitize.ts b/packages/pty/pty-local/src/sanitize.ts new file mode 100644 index 0000000000..1f28315fb1 --- /dev/null +++ b/packages/pty/pty-local/src/sanitize.ts @@ -0,0 +1,188 @@ +/** Streaming terminal-control sanitizer for the line-oriented first release. */ + +import { Buffer } from 'node:buffer' + +/** OSC marker emitted by the controlled bash before each prompt. */ +export const PROMPT_MARKER_PREFIX = '133;D;' + +/** Exact printable prompt emitted after the private marker. */ +export const CONTROLLED_PROMPT = 'dsh> ' + +/** One sanitized chunk plus whether it contained the owned prompt marker. */ +export interface SanitizedChunk { + text: string + prompt: boolean + /** Printable text after the latest owned marker in this chunk. */ + promptTail?: string +} + +/** + * Remove CSI/OSC/short escape sequences while preserving split-sequence carry. + * Full terminal emulation is deliberately deferred; ordinary line output and + * the private prompt marker are the supported contract. + */ +export class TerminalSanitizer { + private pending = '' + private discardMode: 'osc' | 'csi' | undefined + private discardOscEscape = false + private trailingCarriageReturn = false + private trackingPromptTail = false + + constructor(private readonly maxPendingBytes: number) {} + + /** + * Consume one decoded `node-pty` data chunk. + * @param chunk - decoded terminal data. + * @returns Printable text and whether the private prompt marker completed. + */ + push(chunk: string): SanitizedChunk { + this.pending += this.discardPrefix(chunk) + let text = '' + let prompt = false + let includePromptTail = this.trackingPromptTail + let promptTail = '' + let index = 0 + const appendText = (value: string): void => { + text += value + if (this.trackingPromptTail) promptTail += value + } + while (index < this.pending.length) { + const escape = this.pending.indexOf('\x1b', index) + if (escape < 0) { + appendText(this.pending.slice(index)) + index = this.pending.length + break + } + appendText(this.pending.slice(index, escape)) + if (escape + 1 >= this.pending.length) { + index = escape + break + } + const kind = this.pending[escape + 1] + if (kind === ']') { + const bel = this.pending.indexOf('\x07', escape + 2) + const stringTerminator = this.pending.indexOf('\x1b\\', escape + 2) + let end = -1 + if (bel >= 0 && stringTerminator >= 0) end = Math.min(bel + 1, stringTerminator + 2) + else if (bel >= 0) end = bel + 1 + else if (stringTerminator >= 0) end = stringTerminator + 2 + if (end < 0) { + index = escape + break + } + const terminatorBytes = this.pending[end - 1] === '\x07' ? 1 : 2 + const content = this.pending.slice(escape + 2, end - terminatorBytes) + if (content.startsWith(PROMPT_MARKER_PREFIX)) { + prompt = true + this.trackingPromptTail = true + includePromptTail = true + promptTail = '' + } + index = end + continue + } + if (kind === '[') { + let end = escape + 2 + while (end < this.pending.length) { + const code = this.pending.charCodeAt(end) + if (code >= 0x40 && code <= 0x7e) break + end += 1 + } + if (end >= this.pending.length) { + index = escape + break + } + index = end + 1 + continue + } + // Two-byte escape family (save/restore cursor and similar). + index = escape + 2 + } + this.pending = this.pending.slice(index) + this.enforcePendingBound() + return { + text: this.normalizeText(text), + prompt, + ...includePromptTail ? { promptTail } : {}, + } + } + + /** + * Flush a trailing printable fragment when the PTY exits. + * @returns Remaining printable text; incomplete escapes are discarded. + */ + flush(): string { + const text = this.pending.startsWith('\x1b') ? '' : this.pending + this.pending = '' + this.discardMode = undefined + this.discardOscEscape = false + this.trackingPromptTail = false + const normalized = this.normalizeText(text) + if (!this.trailingCarriageReturn) return normalized + this.trailingCarriageReturn = false + return `${normalized}\n` + } + + private normalizeText(text: string): string { + let complete = this.trailingCarriageReturn ? `\r${text}` : text + this.trailingCarriageReturn = false + if (complete.endsWith('\r')) { + complete = complete.slice(0, -1) + this.trailingCarriageReturn = true + } + return normalizeTerminalText(complete) + } + + private enforcePendingBound(): void { + if (Buffer.byteLength(this.pending) <= this.maxPendingBytes) return + this.discardMode = this.pending[1] === ']' ? 'osc' : 'csi' + this.pending = '' + } + + private discardPrefix(chunk: string): string { + if (this.discardMode === undefined) return chunk + if (this.discardMode === 'csi') { + for (let index = 0; index < chunk.length; index += 1) { + const code = chunk.charCodeAt(index) + if (code >= 0x40 && code <= 0x7e) { + this.discardMode = undefined + return chunk.slice(index + 1) + } + } + return '' + } + + let index = 0 + if (this.discardOscEscape) { + this.discardOscEscape = false + if (chunk.startsWith('\\')) { + this.discardMode = undefined + return chunk.slice(1) + } + } + while (index < chunk.length) { + if (chunk[index] === '\x07') { + this.discardMode = undefined + return chunk.slice(index + 1) + } + if (chunk[index] === '\x1b') { + if (chunk[index + 1] === '\\') { + this.discardMode = undefined + return chunk.slice(index + 2) + } + if (index + 1 === chunk.length) this.discardOscEscape = true + } + index += 1 + } + return '' + } +} + +/** + * Normalize CRLF and standalone carriage returns for line-oriented rendering. + * @param text - sanitized terminal text. + * @returns Line-normalized text with BEL removed. + */ +export function normalizeTerminalText(text: string): string { + return text.replaceAll('\r\n', '\n').replaceAll('\r', '\n').replaceAll('\x07', '') +} diff --git a/packages/pty/pty-local/src/session.ts b/packages/pty/pty-local/src/session.ts index ebf6b559d1..25ed8fe586 100644 --- a/packages/pty/pty-local/src/session.ts +++ b/packages/pty/pty-local/src/session.ts @@ -1,7 +1,12 @@ -/** Local `node-pty` session: bounded output, readiness, signals, and teardown. */ +/** Persistent PTY session over the subprocess seam's terminal primitive. */ -import type { IDisposable, IPty } from 'node-pty' -import { PtyTerminalSanitizer, PtyTextBuffer, ptySignalName, ptyUtf8Tail } from '@deepseek-ai/dsh-pty' +import { Buffer } from 'node:buffer' +import type { + SubprocessOutcome, + SubprocessTerminalForeground, + SubprocessTerminalHandle, +} from '@deepseek-ai/dsh-subprocess' +import { PtyError } from '@deepseek-ai/dsh-pty' import type { PtyBackendSession, PtyReadRequest, @@ -16,30 +21,89 @@ import type { PtyWaitReason, } from '@deepseek-ai/dsh-pty' import type { ResolvedConfig } from './config.ts' -import type { ProcessIdentity, ProcessInspector } from './process-inspector.ts' +import { CONTROLLED_PROMPT, TerminalSanitizer } from './sanitize.ts' -function delay(ms: number): Promise { - return new Promise(resolve => setTimeout(resolve, ms)) +function utf8Tail(text: string, maxBytes: number): { text: string; truncated: boolean } { + if (Buffer.byteLength(text) <= maxBytes) return { text, truncated: false } + const chars = Array.from(text) + let bytes = 0 + let start = chars.length + while (start > 0) { + const next = Buffer.byteLength(chars[start - 1] as string) + if (bytes + next > maxBytes) break + bytes += next + start -= 1 + } + return { text: chars.slice(start).join(''), truncated: true } +} + +class BoundedTextBuffer { + private value = '' + private dropped = false + + constructor( + private readonly maxBytes: number, + private readonly maxLines?: number, + ) {} + + append(text: string): void { + if (text.length === 0) return + this.value += text + if (this.maxLines !== undefined) { + const lines = this.value.split('\n') + if (lines.length > this.maxLines) { + this.value = lines.slice(lines.length - this.maxLines).join('\n') + this.dropped = true + } + } + const tail = utf8Tail(this.value, this.maxBytes) + this.value = tail.text + this.dropped ||= tail.truncated + } + + consume(): PtySendRead { + const delta = this.value + const truncated = this.dropped + this.value = '' + this.dropped = false + return { delta, truncated } + } + + snapshot(): { text: string; truncated: boolean } { + return { text: this.value, truncated: this.dropped } + } } class LocalSendOperation implements PtySendOperation { - private readonly output: PtyTextBuffer + private readonly output: BoundedTextBuffer private readonly promise: PromiseWithResolvers private finished = false + private cancellationRequested = false + private initialForegroundLeftWait: boolean + private initialForegroundPgid: number | undefined constructor( maxBytes: number, readonly startedAt: number, private readonly onCancel: () => void, ) { - this.output = new PtyTextBuffer(maxBytes) + this.output = new BoundedTextBuffer(maxBytes) this.promise = Promise.withResolvers() + this.initialForegroundLeftWait = true } get done(): Promise { return this.promise.promise } + get settled(): boolean { + return this.finished + } + + get cancelRequested(): boolean { + return this.cancellationRequested + } + append(text: string): void { if (!this.finished) this.output.append(text) } @@ -66,50 +130,75 @@ class LocalSendOperation implements PtySendOperation { return this.output.consume() } + setInitialForeground(foreground: SubprocessTerminalForeground | undefined): void { + this.initialForegroundPgid = foreground?.processGroupId + this.initialForegroundLeftWait = foreground?.inputWaiting !== true + } + + acceptsStdinWait(pgid: number, waiting: boolean): boolean { + // The same group may still expose the wait that existed before terminal.write. + // Observe every poll so a departure before the exact-settlement threshold + // still makes a later return to that wait post-write evidence. + if (pgid !== this.initialForegroundPgid) return waiting + if (!waiting) this.initialForegroundLeftWait = true + return waiting && this.initialForegroundLeftWait + } + cancel(): boolean { if (this.finished) return false + this.cancellationRequested = true this.onCancel() return true } } -/** Backend session wrapping one `node-pty` process and its captured process tree. */ +/** Backend session wrapping one provider-owned terminal process. */ export class LocalPtySession implements PtyBackendSession { motd = '' readonly pid: number - private readonly sanitizer: PtyTerminalSanitizer - private readonly scrollback: PtyTextBuffer - private readonly exitPromise: PromiseWithResolvers = Promise.withResolvers() - private readonly dataDisposable: IDisposable - private readonly exitDisposable: IDisposable + private readonly decoder = new TextDecoder() + private readonly sanitizer: TerminalSanitizer + private readonly scrollback: BoundedTextBuffer + private readonly outputEnded = Promise.withResolvers() + private readonly completion: Promise private statusValue: PtySessionStatus = { kind: 'running' } + // TODO(pty-send-state-consolidation): Fold the per-send fields below + // (active/activeTimer/activeDeadlineTimer/activeAbort/interrupting/ + // activeWrite/pollingReady/polling) into one send-lifecycle owner; the + // cancellation/readiness interplay now has enough pinned tests to carry + // that refactor safely. private active: LocalSendOperation | undefined private activeTimer: NodeJS.Timeout | undefined + private activeDeadlineTimer: NodeJS.Timeout | undefined private activeAbort: (() => void) | undefined + private interrupting: LocalSendOperation | undefined + private activeWrite: Promise | undefined + private pollingReady: LocalSendOperation | undefined + private polling = false private promptSeen = false private promptTextSeen = false + private promptTail = '' private shellPgid: number | undefined private initializing = false private lastOutputAt = Date.now() private closing = false private closePromise: Promise | undefined + private transportFailure: Error | undefined constructor( - private readonly terminal: IPty, - private readonly inspector: ProcessInspector, + private readonly terminal: SubprocessTerminalHandle, private readonly config: ResolvedConfig, ) { this.pid = terminal.pid - this.sanitizer = new PtyTerminalSanitizer(config.maxReadBytes) - this.scrollback = new PtyTextBuffer(config.scrollbackMaxBytes, config.scrollbackLines) - this.dataDisposable = terminal.onData((data) => { this.onData(data) }) - this.exitDisposable = terminal.onExit(({ exitCode, signal }) => { - const tail = this.sanitizer.flush() - this.appendOutput(tail) - this.statusValue = { kind: 'exited', exitCode, signal: ptySignalName(signal) } - this.settleActive('session_exit') - this.exitPromise.resolve() - }) + this.sanitizer = new TerminalSanitizer(config.maxReadBytes) + this.scrollback = new BoundedTextBuffer(config.scrollbackMaxBytes, config.scrollbackLines) + terminal.output.on('data', this.onTerminalData) + terminal.output.once('end', this.onTerminalEnd) + terminal.output.once('error', this.onTerminalError) + this.completion = terminal.done.then( + outcome => this.onExit(outcome), + (error: unknown) => { this.onTransportFailure(error) }, + ) } /** @@ -136,7 +225,14 @@ export class LocalPtySession implements PtyBackendSession { startSend(request: PtySendRequest): PtySendOperation { if (this.closing) throw new Error('PTY session is closing') if (this.statusValue.kind === 'exited') throw new Error('PTY session has exited') - if (this.active !== undefined) throw new Error('PTY session already has an active send') + if (this.active !== undefined) { + const draining = this.activeWrite !== undefined + ? ' or draining provider write' + : this.interrupting !== undefined + ? ' or draining foreground interrupt' + : '' + throw new PtyError(`PTY session already has an active send${draining}`, 'SEND_ACTIVE') + } if (request.signal?.aborted === true) throw new Error('PTY send aborted before write') const operation = new LocalSendOperation( @@ -145,29 +241,79 @@ export class LocalPtySession implements PtyBackendSession { () => { this.interrupt(operation) }, ) this.active = operation - this.lastOutputAt = Date.now() - this.promptSeen = false - this.promptTextSeen = false + this.resetReadinessEvidence() if (request.signal !== undefined) { const onAbort = (): void => { operation.cancel() } request.signal.addEventListener('abort', onAbort, { once: true }) this.activeAbort = () => request.signal?.removeEventListener('abort', onAbort) } - - try { - if (request.text.length > 0) this.terminal.write(request.text) - if (request.submit) this.terminal.write('\r') - } catch (error: unknown) { - this.clearActive() - operation.fail(error) - return operation - } - - this.activeTimer = setInterval(() => { this.pollReadiness(operation) }, this.config.pollIntervalMs) + this.activeDeadlineTimer = setTimeout(() => { + if (this.active === operation) { + this.settleActive('timeout', this.activeWrite !== undefined || this.interrupting === operation) + } + }, this.config.timeoutMs) + void this.beginSend(operation, request) return operation } + private async beginSend(operation: LocalSendOperation, request: PtySendRequest): Promise { + let foreground: SubprocessTerminalForeground | undefined + try { + foreground = await this.terminal.inspectForeground() + } catch (error: unknown) { + // A pre-write inspection failure while cancellation owns the slot must not + // release it: interruptOnce's in-flight foreground signal could land on a + // successor's foreground group. The interrupt path's post-signal tail + // resumes polling, whose guarded catch propagates a persistent failure. + // A retained settled operation implies that same in-flight interrupt, so + // this guard admits only an unsettled active send. + if (this.active === operation && !this.closing && this.interrupting !== operation) { + this.failActive(error) + } + return + } + try { + if (this.active !== operation || this.closing || this.interrupting === operation) return + operation.setInitialForeground(foreground) + const input = `${request.text}${request.submit ? '\r' : ''}` + if (input.length > 0 && !operation.cancelRequested) { + this.resetReadinessEvidence() + const write = this.terminal.write(input) + this.activeWrite = write.then(() => true, () => false) + try { + await write + } finally { + this.activeWrite = undefined + } + } + // Cancellation owns post-write signalling and reservation release. + if (operation.cancelRequested) return + if (this.active === operation && operation.settled) { + this.clearActive() + return + } + // Closing can race the awaited provider write even though static analysis sees only local assignments. + // oxlint-disable-next-line typescript/no-unnecessary-condition -- awaited provider writes can close the session. + if (this.active === operation && !this.closing) { + this.pollingReady = operation + this.schedulePoll(operation) + } + } catch (error: unknown) { + if (this.active === operation && !this.closing) { + if (operation.settled) this.clearActive() + else this.failActive(error) + } + } + } + + private resetReadinessEvidence(): void { + this.lastOutputAt = Date.now() + this.promptSeen = false + this.promptTextSeen = false + this.promptTail = '' + } + read(request: PtyReadRequest): PtyReadResult { const snapshot = this.scrollback.snapshot() const lines = snapshot.text.split('\n') @@ -182,7 +328,7 @@ export class LocalPtySession implements PtyBackendSession { const end = totalLines - offset const start = Math.max(0, end - count) const requested = lines.slice(start, end).join('\n') - const bounded = ptyUtf8Tail(requested, this.config.maxReadBytes) + const bounded = utf8Tail(requested, this.config.maxReadBytes) const returnedLines = bounded.text.length === 0 ? 0 : bounded.text.split('\n').length return { text: bounded.text, @@ -193,16 +339,10 @@ export class LocalPtySession implements PtyBackendSession { } } - signal(signal: PtySignal): Promise { - return Promise.resolve().then(() => { - const pgid = this.inspector.foregroundPgid(this.pid) - if (pgid === undefined) throw new Error(`cannot resolve foreground process group for PTY ${this.pid}`) - if (signal === 'SIGKILL' && pgid === this.pid) { - throw new Error('refusing to SIGKILL the PTY shell; use terminal_close') - } - this.inspector.signalGroup(pgid, signal) - return { delivered: true, targetPgid: pgid } - }) + async signal(signal: PtySignal): Promise { + if (this.closing) throw new Error('PTY session is closing') + const targetPgid = await this.terminal.signalForeground(signal) + return { delivered: true, targetPgid } } status(): PtySessionStatus { @@ -221,21 +361,56 @@ export class LocalPtySession implements PtyBackendSession { return closing } + private readonly onTerminalData = (chunk: Buffer | Uint8Array | string): void => { + const bytes = typeof chunk === 'string' ? Buffer.from(chunk, 'utf8') : chunk + this.onData(this.decoder.decode(bytes, { stream: true })) + } + + private readonly onTerminalEnd = (): void => { + this.onData(this.decoder.decode()) + this.appendOutput(this.sanitizer.flush()) + this.outputEnded.resolve() + } + + private readonly onTerminalError = (error: Error): void => { + this.onTransportFailure(error) + this.outputEnded.resolve() + } + private onData(data: string): void { const sanitized = this.sanitizer.push(data) this.appendOutput(sanitized.text) if (sanitized.prompt) { - const foregroundPgid = this.inspector.foregroundPgid(this.pid) - if (this.shellPgid === undefined) this.shellPgid = foregroundPgid + // TODO(pty-delayed-signal-prompt): With a reproducer, define a marker-generation boundary + // before attributing a signal-delayed prompt to a later send. // Bash can print PROMPT_COMMAND before the kernel publishes its return // to the foreground process group. Retain the marker; polling below is // the authority that accepts it only after bash owns the foreground. this.promptSeen = true - this.promptTextSeen = sanitized.promptText === true + this.promptTail = '' this.lastOutputAt = Date.now() - } else if (this.promptSeen && sanitized.promptText === true) { - this.promptTextSeen = true } + if (this.promptSeen && sanitized.promptTail !== undefined) { + const remaining = Math.max(0, CONTROLLED_PROMPT.length + 1 - this.promptTail.length) + this.promptTail += sanitized.promptTail.slice(0, remaining) + if (sanitized.promptTail.length > remaining) this.promptTail = `${CONTROLLED_PROMPT}\0` + this.promptTextSeen = this.promptTail === CONTROLLED_PROMPT + } + } + + private async onExit(outcome: SubprocessOutcome): Promise { + await this.outputEnded.promise + if (this.transportFailure !== undefined) return + this.statusValue = { kind: 'exited', exitCode: outcome.exitCode, signal: outcome.signal } + this.settleActive('session_exit') + } + + private onTransportFailure(error: unknown): void { + const failure = error instanceof Error ? error : new Error(String(error)) + this.transportFailure ??= failure + this.statusValue = { kind: 'exited', exitCode: null, signal: null } + this.failActive(failure) + void this.terminal.terminate().catch(() => {}) } private appendOutput(text: string): void { @@ -245,60 +420,94 @@ export class LocalPtySession implements PtyBackendSession { this.active?.append(text) } - private pollReadiness(operation: LocalSendOperation): void { - if (this.active !== operation) return - if (this.statusValue.kind === 'exited') { - this.settleActive('session_exit') - return - } - if (this.promptSeen && this.promptTextSeen && Date.now() - this.lastOutputAt >= this.config.pollIntervalMs) { - const pgid = this.inspector.foregroundPgid(this.pid) - if (this.shellPgid !== undefined && pgid === this.shellPgid) { - this.settleActive('stdin_read') - return - } - } - const elapsed = Date.now() - operation.startedAt - const startupHasOutput = !this.initializing || this.scrollback.snapshot().text.length > 0 - if (startupHasOutput && elapsed >= this.config.exactProbeAfterMs) { - const pgid = this.inspector.foregroundPgid(this.pid) - if (pgid !== undefined && this.inspector.isStdinWaiting(pgid)) { - this.settleActive('stdin_read') - return - } - } - // A prompt candidate can race bash's foreground handoff, but an interactive - // child also inherits PROMPT_COMMAND. Silence therefore remains the bound - // on waiting for shell ownership instead of letting a child marker suppress - // readiness until the absolute timeout. When a prompt marker was seen, the - // configured grace holds the fallback past the silence bound so polls in - // that window can observe the foreground handoff and settle as stdin_read. - const idleFor = Date.now() - this.lastOutputAt - const handoffGrace = this.promptSeen ? this.config.handoffGraceMs : 0 - if (startupHasOutput && idleFor >= this.config.idleSilenceMs + handoffGrace) { - this.settleActive('inferred_idle') - return - } - if (elapsed >= this.config.timeoutMs) this.settleActive('timeout') + private schedulePoll(operation: LocalSendOperation, delayMs = this.config.pollIntervalMs): void { + if (this.active !== operation || this.interrupting === operation || this.polling) return + if (this.activeTimer !== undefined) clearTimeout(this.activeTimer) + this.activeTimer = setTimeout(() => { + this.activeTimer = undefined + void this.pollReadiness(operation) + }, delayMs) } - private settleActive(waitReason: PtyWaitReason): void { + private async pollReadiness(operation: LocalSendOperation): Promise { + if (this.active !== operation || this.polling) return + this.polling = true + try { + if (this.statusValue.kind === 'exited') { + this.settleActive('session_exit') + return + } + const foreground = await this.terminal.inspectForeground() + if (this.active !== operation || this.closing || this.interrupting === operation) return + const idleFor = Date.now() - this.lastOutputAt + if (this.promptSeen && foreground !== undefined && this.shellPgid === undefined) { + this.shellPgid = foreground.processGroupId + } + if (this.promptSeen && this.promptTextSeen && idleFor >= this.config.pollIntervalMs + && foreground?.processGroupId === this.shellPgid) { + this.settleActive('stdin_read') + return + } + const elapsed = Date.now() - operation.startedAt + const startupHasOutput = !this.initializing || this.scrollback.snapshot().text.length > 0 + const acceptsStdinWait = startupHasOutput && foreground !== undefined + && operation.acceptsStdinWait(foreground.processGroupId, foreground.inputWaiting) + if (elapsed >= this.config.exactProbeAfterMs && acceptsStdinWait) { + this.settleActive('stdin_read') + return + } + // A prompt candidate can race bash's foreground handoff, but an interactive + // child also inherits PROMPT_COMMAND. Silence therefore remains the bound + // on waiting for shell ownership instead of letting a child marker suppress + // readiness until the absolute timeout. + const handoffGrace = this.promptSeen ? this.config.handoffGraceMs : 0 + if (startupHasOutput && idleFor >= this.config.idleSilenceMs + handoffGrace) { + this.settleActive('inferred_idle') + } + } catch (error: unknown) { + if (this.active === operation && !this.closing && this.interrupting !== operation) this.failActive(error) + } finally { + this.polling = false + const active = this.active + // Awaited provider inspection can clear or replace the active send despite static analysis. + // oxlint-disable-next-line typescript/no-unnecessary-condition -- awaited inspection can replace the active send. + if (active !== undefined && this.pollingReady === active) this.schedulePoll(active) + } + } + + private settleActive(waitReason: PtyWaitReason, retainOwnership = false): void { const operation = this.active if (operation === undefined) return const scrollbackTruncated = this.scrollback.snapshot().truncated - this.clearActive() + if (retainOwnership) { + this.stopPolling() + this.activeAbort?.() + this.activeAbort = undefined + } else { + this.clearActive() + } operation.settle(waitReason, this.statusValue, scrollbackTruncated) } private stopPolling(): void { - if (this.activeTimer !== undefined) clearInterval(this.activeTimer) + this.stopReadinessPolling() + if (this.activeDeadlineTimer !== undefined) clearTimeout(this.activeDeadlineTimer) + this.activeDeadlineTimer = undefined + } + + private stopReadinessPolling(): void { + if (this.activeTimer !== undefined) clearTimeout(this.activeTimer) this.activeTimer = undefined + this.pollingReady = undefined } private clearActive(): void { + const operation = this.active this.stopPolling() this.activeAbort?.() this.activeAbort = undefined + if (this.interrupting === operation) this.interrupting = undefined + this.pollingReady = undefined this.active = undefined } @@ -311,104 +520,46 @@ export class LocalPtySession implements PtyBackendSession { private interrupt(operation: LocalSendOperation): void { if (this.active !== operation) return + this.interrupting = operation + this.stopReadinessPolling() + void this.interruptOnce(operation) + } + + private async interruptOnce(operation: LocalSendOperation): Promise { try { - const pgid = this.inspector.foregroundPgid(this.pid) - if (pgid === undefined) throw new Error(`cannot resolve foreground process group for PTY ${this.pid}`) - this.inspector.signalGroup(pgid, 'SIGINT') + const activeWrite = this.activeWrite + if (activeWrite !== undefined && !await activeWrite) return + await this.terminal.signalForeground('SIGINT') } catch (error: unknown) { - this.failActive(error) + if (this.active === operation && !this.closing) this.onTransportFailure(error) + return + } finally { + if (this.interrupting === operation) this.interrupting = undefined } - } - - private survivors(members: ProcessIdentity[]): ProcessIdentity[] { - return members.filter(member => this.inspector.isAlive(member)) - } - - private descendants(): ProcessIdentity[] { - return this.inspector.processTree(this.pid).filter(member => member.pid !== this.pid) - } - - private async waitForExit(members: ProcessIdentity[]): Promise { - const deadline = Date.now() + this.config.disposeGraceMs - let survivors = this.survivors(members) - while (survivors.length > 0 && Date.now() < deadline) { - await delay(Math.min(25, Math.max(1, deadline - Date.now()))) - survivors = this.survivors(members) - } - return survivors - } - - private signalMembers(members: ProcessIdentity[], signal: 'SIGTERM' | 'SIGKILL'): void { - for (const member of members) { - try { - this.inspector.signalProcess(member, signal) - } catch (_alreadyExitedDuringSignal) { - // Identity is rechecked by the inspector; a same-tick exit is success. - } - } - } - - private unionMembers(...groups: ProcessIdentity[][]): ProcessIdentity[] { - const members: ProcessIdentity[] = [] - const seen = new Set() - for (const group of groups) { - for (const member of group) { - const key = JSON.stringify([member.pid, member.started]) - if (seen.has(key)) continue - seen.add(key) - members.push(member) - } - } - return members - } - - private async stopDescendants(): Promise { - const captured = this.descendants() - this.signalMembers(captured, 'SIGTERM') - const capturedSurvivors = await this.waitForExit(captured) - // A TERM-handling descendant may have forked while winding down. Rescan - // while the shell can still reap every member, then kill both the fresh - // tree and captured survivors that were reparented out of that tree. - const members = this.unionMembers(capturedSurvivors, this.descendants()) - this.signalMembers(members, 'SIGKILL') - const survivors = await this.waitForExit(members) - return this.survivors(this.unionMembers(survivors, this.descendants())) - } - - private async stopShell(): Promise { - try { - this.terminal.kill('SIGTERM') - } catch (_topLevelAlreadyExitedDuringTerm) { - // The exit notification remains authoritative. - } - if (this.statusValue.kind === 'running') { - await Promise.race([this.exitPromise.promise, delay(this.config.disposeGraceMs)]) - } - if (this.statusValue.kind === 'running') { - try { - this.terminal.kill('SIGKILL') - } catch (_topLevelAlreadyExitedDuringKill) { - // The exit notification remains authoritative. - } - await Promise.race([this.exitPromise.promise, delay(this.config.disposeGraceMs)]) - } - if (this.statusValue.kind === 'running') { - throw new Error(`PTY cleanup failed; surviving pids: ${this.pid}`) + if (this.active === operation && operation.settled) { + this.clearActive() + } else if (this.active === operation && !this.closing) { + this.pollingReady = operation + this.schedulePoll(operation, 0) } } private async closeOnce(reason: string): Promise { - this.dataDisposable.dispose() // Stop readiness polling but retain the active operation: teardown settles // it as session_exit below, so an in-flight send is never mis-settled as // stdin_read/inferred_idle/timeout during the grace period. this.stopPolling() - const survivors = await this.stopDescendants() - if (survivors.length > 0) { - throw new Error(`PTY cleanup failed (${reason}); surviving pids: ${survivors.map(member => member.pid).join(', ')}`) + try { + await this.terminal.terminate() + } catch (error: unknown) { + throw new Error(`PTY cleanup failed (${reason})`, { cause: error }) } - await this.stopShell() + // Quiescence is the active send's terminal outcome. this.settleActive('session_exit') - this.exitDisposable.dispose() + await this.completion + this.terminal.output.off('data', this.onTerminalData) + this.terminal.output.off('end', this.onTerminalEnd) + this.terminal.output.off('error', this.onTerminalError) + if (this.transportFailure !== undefined) throw this.transportFailure } } diff --git a/packages/pty/pty-local/tests/sanitize.spec.ts b/packages/pty/pty-local/tests/sanitize.spec.ts index 06ec076358..f6649b3c6f 100644 --- a/packages/pty/pty-local/tests/sanitize.spec.ts +++ b/packages/pty/pty-local/tests/sanitize.spec.ts @@ -1,17 +1,17 @@ import { describe, expect, it } from 'vitest' -import { normalizePtyTerminalText, PtyTerminalSanitizer } from '@deepseek-ai/dsh-pty' +import { normalizeTerminalText, TerminalSanitizer } from '@deepseek-ai/dsh-pty-local/src/sanitize.ts' -describe('PtyTerminalSanitizer', () => { +describe('TerminalSanitizer', () => { it('removes split CSI and owned OSC prompt markers', () => { - const sanitizer = new PtyTerminalSanitizer(64) + const sanitizer = new TerminalSanitizer(64) expect(sanitizer.push('red\x1b[3')).toEqual({ text: 'red', prompt: false }) expect(sanitizer.push('1m text\x1b[0m\r\n')).toEqual({ text: ' text\n', prompt: false }) expect(sanitizer.push('\x1b]133;')).toEqual({ text: '', prompt: false }) - expect(sanitizer.push('D;0\x07dsh> ')).toEqual({ text: 'dsh> ', prompt: true, promptText: true }) + expect(sanitizer.push('D;0\x07dsh> ')).toEqual({ text: 'dsh> ', prompt: true, promptTail: 'dsh> ' }) }) it('drops unrelated OSC, short escapes, BEL, and incomplete trailing escape', () => { - const sanitizer = new PtyTerminalSanitizer(64) + const sanitizer = new TerminalSanitizer(64) expect(sanitizer.push('a\x1b]0;title\x1b\\b\x1b7c\x07')).toEqual({ text: 'abc', prompt: false }) expect(sanitizer.push('tail\x1b')).toEqual({ text: 'tail', prompt: false }) expect(sanitizer.flush()).toBe('') @@ -22,11 +22,11 @@ describe('PtyTerminalSanitizer', () => { }) it('normalizes CRLF and standalone carriage returns', () => { - expect(normalizePtyTerminalText('a\r\nb\rc\x07')).toBe('a\nb\nc') + expect(normalizeTerminalText('a\r\nb\rc\x07')).toBe('a\nb\nc') }) it('carries a trailing carriage return across data chunks and flushes standalone CR', () => { - const sanitizer = new PtyTerminalSanitizer(64) + const sanitizer = new TerminalSanitizer(64) expect(sanitizer.push('a\r')).toEqual({ text: 'a', prompt: false }) expect(sanitizer.push('\nb')).toEqual({ text: '\nb', prompt: false }) expect(sanitizer.push('\r')).toEqual({ text: '', prompt: false }) @@ -34,41 +34,41 @@ describe('PtyTerminalSanitizer', () => { }) it('reports printable prompt text that follows a marker in a later chunk', () => { - const sanitizer = new PtyTerminalSanitizer(64) - expect(sanitizer.push('\x1b]133;D;0\x07')).toEqual({ text: '', prompt: true }) - expect(sanitizer.push('dsh> ')).toEqual({ text: 'dsh> ', prompt: false, promptText: true }) + const sanitizer = new TerminalSanitizer(64) + expect(sanitizer.push('\x1b]133;D;0\x07')).toEqual({ text: '', prompt: true, promptTail: '' }) + expect(sanitizer.push('dsh> ')).toEqual({ text: 'dsh> ', prompt: false, promptTail: 'dsh> ' }) }) it('bounds and discards unterminated control sequences through their terminators', () => { - const oscBel = new PtyTerminalSanitizer(8) + const oscBel = new TerminalSanitizer(8) expect(oscBel.push(`\x1b]0;${'x'.repeat(16)}`)).toEqual({ text: '', prompt: false }) expect(oscBel.push('more\x07tail')).toEqual({ text: 'tail', prompt: false }) - const oscSt = new PtyTerminalSanitizer(8) + const oscSt = new TerminalSanitizer(8) oscSt.push(`\x1b]0;${'x'.repeat(16)}`) expect(oscSt.push('more\x1b')).toEqual({ text: '', prompt: false }) expect(oscSt.push('\\tail')).toEqual({ text: 'tail', prompt: false }) - const oscDirectSt = new PtyTerminalSanitizer(8) + const oscDirectSt = new TerminalSanitizer(8) oscDirectSt.push(`\x1b]0;${'x'.repeat(16)}`) expect(oscDirectSt.push('more\x1b\\tail')).toEqual({ text: 'tail', prompt: false }) - const oscFalseSt = new PtyTerminalSanitizer(8) + const oscFalseSt = new TerminalSanitizer(8) oscFalseSt.push(`\x1b]0;${'x'.repeat(16)}`) oscFalseSt.push('\x1b') expect(oscFalseSt.push('more')).toEqual({ text: '', prompt: false }) expect(oscFalseSt.push('\x07tail')).toEqual({ text: 'tail', prompt: false }) - const oscNonTerminatingEscape = new PtyTerminalSanitizer(8) + const oscNonTerminatingEscape = new TerminalSanitizer(8) oscNonTerminatingEscape.push(`\x1b]0;${'x'.repeat(16)}`) expect(oscNonTerminatingEscape.push('more\x1bxmore\x07tail')).toEqual({ text: 'tail', prompt: false }) - const csi = new PtyTerminalSanitizer(8) + const csi = new TerminalSanitizer(8) expect(csi.push(`\x1b[${'1'.repeat(16)}`)).toEqual({ text: '', prompt: false }) expect(csi.push('123')).toEqual({ text: '', prompt: false }) expect(csi.push('mtext')).toEqual({ text: 'text', prompt: false }) - const flushed = new PtyTerminalSanitizer(8) + const flushed = new TerminalSanitizer(8) flushed.push(`\x1b]0;${'x'.repeat(16)}`) expect(flushed.flush()).toBe('') expect(flushed.push('text')).toEqual({ text: 'text', prompt: false }) diff --git a/packages/subprocess/README.i18n.yaml b/packages/subprocess/README.i18n.yaml index 4097420382..0ebb5bd4af 100644 --- a/packages/subprocess/README.i18n.yaml +++ b/packages/subprocess/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/subprocess/README.md -README.md: cdea3803e903b31e895658745bc6dbf5a3c04c71 -README.zh.md: 95b02b6491077750f23ff12ffd14c1aa174609be +README.md: f2b19436da40feb14d067e2cfc706222625680b5 +README.zh.md: 938312448dd5c0a691ed07ddc9843cf2c4445637 diff --git a/packages/subprocess/README.md b/packages/subprocess/README.md index cdea3803e9..f2b19436da 100644 --- a/packages/subprocess/README.md +++ b/packages/subprocess/README.md @@ -2,12 +2,11 @@ English | [中文](README.zh.md) -The shared home for spawning managed child-process trees: fully-specified spawn specs with Node-shaped per-stream stdio dispositions (raw pipes, inherit, bounded tail-keep collection with spill files), the one credential scrub every harness spawner uses, offset-based incremental reads, tree-scoped signalling with SIGTERM→grace→SIGKILL escalation, and the cooperative dispose ladder. Command defaulting, shell semantics, deadlines, protocol framing, and presentation stay with consumers — the [bash executors](../bash/README.md), the [LSP host](../lsp/README.md), and the [ACP subagent backend](../subagent/README.md). See the [subprocess seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md). +The shared process substrate for one execution world: executable lookup, fully-specified managed child-process trees with raw or collected stdio, and one deep terminal-process primitive that owns PTY allocation, foreground groups, and provider-observable session cleanup. Command defaulting, shell semantics, deadlines, protocol framing, readiness, and presentation stay with consumers — the [bash executors](../bash/README.md), [LSP host](../lsp/README.md), [PTY shell backend](../pty/README.md), and [ACP subagent backend](../subagent/README.md). See the [subprocess seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md). | Package | ctx key | Role | |---|---|---| -| [`subprocess`](subprocess/README.md) (`@deepseek-ai/dsh-subprocess`) | `ctx.subprocess` | The seam: abstract `SubprocessService.spawn(spec)`, the fully-explicit `SubprocessSpawnSpec` with per-stream stdio dispositions, `SubprocessHandle` (streams, offset-based readers, terminate/waitForExit/dispose), and the shared scrub + `DSH_*`/`CollectedOutput` vocabulary | -| [`subprocess-local`](subprocess-local/README.md) (`@deepseek-ai/dsh-subprocess-local`) | — | The local implementation: detached process trees, per-disposition stream wiring, tail-keep truncation with bounded private spill files, the `DSH_*` merge order, tree signalling with escalation, the dispose ladder, and terminate-and-join disposal | -| [`e2b/subprocess-e2b`](../e2b/subprocess-e2b/README.md) (`@deepseek-ai/dsh-subprocess-e2b`) | — | Experimental E2B implementation: remote Linux process groups and spill state in the shared `ctx.e2b` sandbox, with asynchronous PID acquisition and SDK buffering limitations | +| [`subprocess`](subprocess/README.md) (`@deepseek-ai/dsh-subprocess`) | `ctx.subprocess` | The seam: executable lookup, ordinary managed spawns, the terminal-process primitive, handle lifecycles, and shared environment/output vocabulary | +| [`subprocess-local`](subprocess-local/README.md) (`@deepseek-ai/dsh-subprocess-local`) | — | The local implementation: detached process trees, bounded collection/spill, `node-pty`, foreground/session inspection, tree signalling, and terminate-and-join disposal | The service owns process lifetime across consumer reloads; consumers own what a process means (a bash command, a future non-shell runner) and every default that shapes one. diff --git a/packages/subprocess/README.zh.md b/packages/subprocess/README.zh.md index 95b02b6491..938312448d 100644 --- a/packages/subprocess/README.zh.md +++ b/packages/subprocess/README.zh.md @@ -1,13 +1,12 @@ -# subprocess/:进程管理能力家族 +# subprocess/:子进程能力家族 [English](README.md) | 中文 -spawn 受管子进程树的共用归属位置:完全显式的 spawn spec,其 stdio 处置方式(disposition)为 Node 形状、按流划分(原始管道、inherit、附带 spill 文件的有界尾部保留收集);harness 中所有 spawn 调用方共用的那一份凭据清除;基于偏移量的增量读取;以进程树为范围、带 SIGTERM→宽限期→SIGKILL 升级的信号发送;以及协作式 dispose(资源释放)阶梯。命令默认值补全、shell 语义、deadline、协议分帧与呈现留在消费方:[bash 执行器](../bash/README.md)、[LSP 主机](../lsp/README.md)与 [ACP(Agent Client Protocol)subagent 后端](../subagent/README.md)。参见[进程管理器 seam Agent Note(agent 决策记录)](../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md)。 +这里集中提供一个执行世界的共享进程基底:可执行文件查找、具有原始或收集式 stdio 的完整指定受管子进程树,以及一项深层终端进程原语,负责 PTY 分配、前台进程组和提供方仍可观察到的会话成员清理。命令默认值补全、shell 语义、时限、协议分帧、就绪状态与呈现留在消费方:[bash 执行器](../bash/README.md)、[LSP 主机](../lsp/README.md)、[PTY shell 后端](../pty/README.md)与 [ACP(Agent Client Protocol)subagent 后端](../subagent/README.md)。参见[subprocess seam Agent Note(agent 决策记录)](../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md)。 | 包(package) | ctx 键 | 角色 | |---|---|---| -| [`subprocess`](subprocess/README.md)(`@deepseek-ai/dsh-subprocess`) | `ctx.subprocess` | seam 本体:抽象的 `SubprocessService.spawn(spec)`、完全显式且带按流划分 stdio 处置方式的 `SubprocessSpawnSpec`、`SubprocessHandle`(流、基于偏移量的读取器、terminate/waitForExit/dispose),以及共享的凭据清除 + `DSH_*`/`CollectedOutput` 词汇 | -| [`subprocess-local`](subprocess-local/README.md)(`@deepseek-ai/dsh-subprocess-local`) | 无 | 本地实现:detached 进程树、按处置方式接线的流、附带有界私有 spill 文件的尾部保留截断、`DSH_*` 合并次序、带升级的进程树信号发送、dispose 阶梯,以及先终止再等待退出的 dispose | -| [`e2b/subprocess-e2b`](../e2b/subprocess-e2b/README.md)(`@deepseek-ai/dsh-subprocess-e2b`) | 无 | 实验性 E2B 实现:远程 Linux 进程组和共享 `ctx.e2b` 沙箱中的 spill 状态,但 PID 异步获取,且受 SDK 缓冲限制 | +| [`subprocess`](subprocess/README.md)(`@deepseek-ai/dsh-subprocess`) | `ctx.subprocess` | seam 本体:可执行文件查找、普通受管 spawn、终端进程原语、句柄生命周期,以及共享的环境/输出词汇 | +| [`subprocess-local`](subprocess-local/README.md)(`@deepseek-ai/dsh-subprocess-local`) | 无 | 本地实现:detached 进程树、有界收集/spill、`node-pty`、前台/会话检查、进程树信号发送,以及先终止再等待退出的资源释放 | -服务拥有跨消费方重载的进程存续期;消费方拥有一个进程的含义(一条 bash 命令、未来的非 shell 运行器)以及塑造它的每一项默认值。 +即使消费方重载,进程生命周期仍由服务负责管理;消费方负责定义进程的含义(一条 bash 命令、未来的非 shell 运行器),以及决定塑造该进程的每一项默认值。 diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index f682bc6847..d2e7d0ec1c 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -346,6 +346,12 @@ importers: '@deepseek-ai/dsh-credentials-local': specifier: workspace:* version: link:../packages/credentials/credentials-local + '@deepseek-ai/dsh-e2b': + specifier: workspace:* + version: link:../packages/e2b/e2b + '@deepseek-ai/dsh-fs-e2b': + specifier: workspace:* + version: link:../packages/e2b/fs-e2b '@deepseek-ai/dsh-fs-local': specifier: workspace:* version: link:../packages/fs/fs-local @@ -493,6 +499,9 @@ importers: '@deepseek-ai/dsh-subagent-spawn': specifier: workspace:* version: link:../packages/subagent/subagent-spawn + '@deepseek-ai/dsh-subprocess-e2b': + specifier: workspace:* + version: link:../packages/e2b/subprocess-e2b '@deepseek-ai/dsh-subprocess-local': specifier: workspace:* version: link:../packages/subprocess/subprocess-local @@ -716,9 +725,6 @@ importers: packages/bash/bash: devDependencies: - '@deepseek-ai/dsh-brand': - specifier: workspace:^ - version: link:../../util/brand '@deepseek-ai/dsh-invariants': specifier: workspace:^ version: link:../../support/invariants @@ -3094,6 +3100,9 @@ importers: packages/credentials/credentials: devDependencies: + '@deepseek-ai/dsh-brand': + specifier: workspace:^ + version: link:../../util/brand '@deepseek-ai/dsh-invariants': specifier: workspace:^ version: link:../../support/invariants @@ -3132,6 +3141,59 @@ importers: specifier: ^4.0.0-rc.7 version: link:../../../vendor/cordis + packages/e2b/e2b: + dependencies: + e2b: + specifier: 2.29.1 + version: 2.29.1 + schemastery: + specifier: ^3.18.0 + version: link:../../../vendor/schemastery + devDependencies: + '@deepseek-ai/dsh-invariants': + specifier: workspace:^ + version: link:../../support/invariants + '@deepseek-ai/dsh-loader-smoke': + specifier: workspace:^ + version: link:../../support/loader-smoke + cordis: + specifier: ^4.0.0-rc.7 + version: link:../../../vendor/cordis + + packages/e2b/fs-e2b: + devDependencies: + '@deepseek-ai/dsh-e2b': + specifier: workspace:^ + version: link:../e2b + '@deepseek-ai/dsh-fs': + specifier: workspace:^ + version: link:../../fs/fs + '@deepseek-ai/dsh-invariants': + specifier: workspace:^ + version: link:../../support/invariants + cordis: + specifier: ^4.0.0-rc.7 + version: link:../../../vendor/cordis + + packages/e2b/subprocess-e2b: + dependencies: + schemastery: + specifier: ^3.18.0 + version: link:../../../vendor/schemastery + devDependencies: + '@deepseek-ai/dsh-e2b': + specifier: workspace:^ + version: link:../e2b + '@deepseek-ai/dsh-invariants': + specifier: workspace:^ + version: link:../../support/invariants + '@deepseek-ai/dsh-subprocess': + specifier: workspace:^ + version: link:../../subprocess/subprocess + cordis: + specifier: ^4.0.0-rc.7 + version: link:../../../vendor/cordis + packages/examples/acp-demo: devDependencies: '@cordisjs/plugin-include': @@ -7848,6 +7910,9 @@ packages: resolution: {integrity: sha512-ctxtJ/eA+t+6q2++vj5j7FYX3nRu311q1wfYH3xjlLOsczhlhxAg2FWNUXhpGvAw3BWo1xBcvOV6/YLc2r5FJw==} hasBin: true + '@bufbuild/protobuf@2.13.0': + resolution: {integrity: sha512-acq7c49vxfm1ggJ95P70TX7ABDM0vxr1SYD3BB0o0jnBLB4OAqeHyKuN+cD3w80gXEDQ2zxHpR6CUeA+O/aU9g==} + '@chevrotain/types@11.1.2': resolution: {integrity: sha512-U+HFai5+zmJCkK86QsaJtoITlboZHBqrVketcO2ROv865xfCMSFpELQoz1GkX5GzME8pTa+3kbKrZHQtI0gdbw==} @@ -7859,6 +7924,17 @@ packages: resolution: {integrity: sha512-y7/yvZ2TPAnR9+jnc00klvNNLkJiXFFrQA/hlLCcxA9a2A4zQIOimyFQ9XfwYKiGD1fb5GY8vbKIIgO8d5Tb2A==} engines: {node: '>= 20.12.0'} + '@connectrpc/connect-web@2.0.0-rc.3': + resolution: {integrity: sha512-w88P8Lsn5CCsA7MFRl2e6oLY4J/5toiNtJns/YJrlyQaWOy3RO8pDgkz+iIkG98RPMhj2thuBvsd3Cn4DKKCkw==} + peerDependencies: + '@bufbuild/protobuf': ^2.2.0 + '@connectrpc/connect': 2.0.0-rc.3 + + '@connectrpc/connect@2.0.0-rc.3': + resolution: {integrity: sha512-ARBt64yEyKbanyRETTjcjJuHr2YXorzQo0etyS5+P6oSeW8xEuzajA9g+zDnMcj1hlX2dQE93foIWQGfpru7gQ==} + peerDependencies: + '@bufbuild/protobuf': ^2.2.0 + '@csstools/color-helpers@6.1.0': resolution: {integrity: sha512-064IFJdjTfUqnjpCVpMOdbr8FLQBhinbZj6yRv2An2E41O/pLEXqfFRWqGq/SxlE5PEUYTlvWsG2r8MswAVvkg==} engines: {node: '>=20.19.0'} @@ -8478,6 +8554,14 @@ packages: resolution: {integrity: sha512-O8jcjabXaleOG9DQ0+ARXWZBTfnP4WNAqzuiJK7ll44AmxGKv/J2M4TPjxjY3znBCfvBXFzucm1twdyFybFqEA==} engines: {node: '>=12'} + '@isaacs/cliui@9.0.0': + resolution: {integrity: sha512-AokJm4tuBHillT+FpMtxQ60n8ObyXBatq7jD2/JA9dxbDDokKQm8KMht5ibGzLVU9IJDIKK4TPKgMHEYMn3lMg==} + engines: {node: '>=18'} + + '@isaacs/fs-minipass@4.0.1': + resolution: {integrity: sha512-wgm9Ehl2jpeqP3zw/7mo3kRHFp5MEDhqAdwy1fTGkHAwnkGOVsgpvQhL8B5n1qlb01jV3n/bI0ZfZp5lWA1k4w==} + engines: {node: '>=18.0.0'} + '@joplin/turndown-plugin-gfm@1.0.67': resolution: {integrity: sha512-FZfW5EZfidhzd1IaY1uxHnIZPTVOxAdleMZ4/1U6Nt5b7+Qj5JThDnaIomuJtetnUBzuRNbe9FWMuqD4B3dlWA==} @@ -10263,6 +10347,10 @@ packages: resolution: {integrity: sha512-NUPRluOfOiTKBKvWPtSD4PhFvWCqOi0BGStNWs57X9js7XGTprSmFoz5F0tWhR4WPjNeR9jXqdC7/UpSJTnlRg==} engines: {node: '>=18'} + chalk@5.6.2: + resolution: {integrity: sha512-7NzBL0rN6fMUW+f7A6Io4h40qQlG+xGmtMxfbnH/K7TAtt8JQWVQK+6g0UXKMeVJoyV5EkkNsErQ8pVD3bLHbA==} + engines: {node: ^12.17.0 || ^14.13 || >=16.0.0} + character-entities-html4@2.1.0: resolution: {integrity: sha512-1v7fgQRj6hnSwFpq1Eu0ynr/CDEw0rXo2B61qXrLNdHZmPKgb7fqS1a2JwF0rISo9q77jDI8VMEHoApn8qDoZA==} @@ -10280,6 +10368,10 @@ packages: resolution: {integrity: sha512-TQMmc3w+5AxjpL8iIiwebF73dRDF4fBIieAqGn9RGCWaEVwQ6Fb2cGe31Yns0RRIzii5goJ1Y7xbMwo1TxMplw==} engines: {node: '>= 20.19.0'} + chownr@3.0.0: + resolution: {integrity: sha512-+IxzY9BZOQd/XuYPRmrvEVjF/nqj5kgT4kEq7VofrDoM1MxoRjEWkrCC3EtLi59TVawxTAn+orJwFQcrqEN1+g==} + engines: {node: '>=18'} + clsx@2.1.1: resolution: {integrity: sha512-eYm0QWBtUrBWZWG0d386OGAw16Z995PiOVo2B7bjWSbHedGl5e0ZWaq65kOGgUSNesEIDkB9ISbTg/JK9dhCZA==} engines: {node: '>=6'} @@ -10306,6 +10398,9 @@ packages: resolution: {integrity: sha512-OkTL9umf+He2DZkUq8f8J9of7yL6RJKI24dVITBmNfZBmri9zYZQrKkuXiKhyfPSu8tUhnVBB1iKXevvnlR4Ww==} engines: {node: '>= 12'} + compare-versions@6.1.1: + resolution: {integrity: sha512-4hm4VPpIecmlg59CHXnRDnqGplJFrbLG4aFEl5vl6cK1u76ws3LLvX7ikFnTDl5vo39sjWD6AaDPYodJp/NNHg==} + content-disposition@1.1.0: resolution: {integrity: sha512-5jRCH9Z/+DRP7rkvY83B+yGIGX96OYdJmzngqnw2SBSxqCFPd0w2km3s5iawpGX8krnwSGmF0FW5Nhr0Hfai3g==} engines: {node: '>=18'} @@ -10571,6 +10666,9 @@ packages: resolution: {integrity: sha512-svtcdpS8CgJyqAjEQIXdb3OjhFVVYjzGAPO8WGCmRbrml64SPw/jJD4GoE98aR7r25A0XcgrK3F02yw9R/vhQw==} engines: {node: '>=0.3.1'} + dockerfile-ast@0.7.1: + resolution: {integrity: sha512-oX/A4I0EhSkGqrFv0YuvPkBUSYp1XiY8O8zAKc8Djglx8ocz+JfOr8gP0ryRMC2myqvDLagmnZaU9ot1vG2ijw==} + dom-accessibility-api@0.5.16: resolution: {integrity: sha512-X7BJ2yElsnOJ30pZF4uIIDfBEVgF4XEBxL9Bxhy6dnrm5hkzqmsWHGTiHqRiITNhMyFLyAiWndIJP7Z1NTteDg==} @@ -10590,6 +10688,10 @@ packages: resolution: {integrity: sha512-KIN/nDJBQRcXw0MLVhZE9iQHmG68qAVIBg9CqmUYjmQIhgij9U5MFvrqkUL5FbtyyzZuOeOt0zdeRe4UY7ct+A==} engines: {node: '>= 0.4'} + e2b@2.29.1: + resolution: {integrity: sha512-n4aGNwRKTj2oct7BrOWfR4T+xGO834vbsrzfSlWUNJrhz615Lp+ad9hc8KtRaaHULKr/W/14Z6v4c7fqk3y0pg==} + engines: {node: '>=20.18.1'} + eastasianwidth@0.2.0: resolution: {integrity: sha512-I88TYZWc9XiYHRQ4/3c5rjjfgkjhLyW2luGIheGERbNQ6OY7yTybanSpDXZa8y7VUP9YmDcYa+eyq4ca7iLqWA==} @@ -10920,6 +11022,12 @@ packages: deprecated: Old versions of glob are not supported, and contain widely publicized security vulnerabilities, which have been fixed in the current version. Please update. Support for old versions may be purchased (at exorbitant rates) by contacting i@izs.me hasBin: true + glob@11.1.0: + resolution: {integrity: sha512-vuNwKSaKiqm7g0THUBu2x7ckSs3XJLXE+2ssL7/MfTGPLLcrJQ/4Uq1CjPTtO5cCIiRxqvN6Twy1qOwhL0Xjcw==} + engines: {node: 20 || >=22} + deprecated: Old versions of glob are not supported, and contain widely publicized security vulnerabilities, which have been fixed in the current version. Please update. Support for old versions may be purchased (at exorbitant rates) by contacting i@izs.me + hasBin: true + globals@17.7.0: resolution: {integrity: sha512-Czmyns5dUsq4seFBR/Kdydhmo8y9kC79hiSkPn0YcGtNnYWnrgt0vjrSjx9tspoDGWm2CMarffRuLjM4xUz8xg==} engines: {node: '>=18'} @@ -11103,6 +11211,10 @@ packages: jackspeak@3.4.3: resolution: {integrity: sha512-OGlZQpz2yfahA/Rd1Y8Cd9SIEsqvXkLVoSw/cgwhnhFMDbsQFeZYoJJ7bIZBS9BcamUW96asq/npPWugM+RQBw==} + jackspeak@4.2.3: + resolution: {integrity: sha512-ykkVRwrYvFm1nb2AJfKKYPr0emF6IiXDYUaFx4Zn9ZuIH7MrzEZ3sD5RlqGXNRpHtvUHJyOnCEFxOlNDtGo7wg==} + engines: {node: 20 || >=22} + jiti@2.7.0: resolution: {integrity: sha512-AC/7JofJvZGrrneWNaEnJeOLUx+JlGt7tNa0wZiRPT4MY1wmfKjt2+6O2p2uz2+skll8OZZmJMNqeke7kKbNgQ==} hasBin: true @@ -11599,6 +11711,10 @@ packages: minisearch@7.2.0: resolution: {integrity: sha512-dqT2XBYUOZOiC5t2HRnwADjhNS2cecp9u+TJRiJ1Qp/f5qjkeT5APcGPjHw+bz89Ms8Jp+cG4AlE+QZ/QnDglg==} + minizlib@3.1.0: + resolution: {integrity: sha512-KZxYo1BUkWD2TVFLr0MQoM8vUUigWD3LlD83a/75BqC+4qE0Hb1Vo5v1FgcfaNXvfXzr+5EhQ6ing/CaBijTlw==} + engines: {node: '>= 18'} + mitt@3.0.1: resolution: {integrity: sha512-vKivATfr97l2/QBCYAkXYDbrIWPM2IIKEl7YPhjCvKlG3kE2gm+uBo6nEXK3M5/Ffh/FLpKExzOQ3JJoJGFKBw==} @@ -11742,6 +11858,12 @@ packages: zod: optional: true + openapi-fetch@0.14.1: + resolution: {integrity: sha512-l7RarRHxlEZYjMLd/PR0slfMVse2/vvIAGm75/F7J6MlQ8/b9uUQmUF2kCPrQhJqMXSxmYWObVgeYXbFYzZR+A==} + + openapi-typescript-helpers@0.0.15: + resolution: {integrity: sha512-opyTPaunsklCBpTK8JGef6mfPhLSnyy5a0IN9vKtx3+4aExf+KxEqYwIy3hqkedXIB97u357uLMJsOnm3GVjsw==} + optionator@0.9.4: resolution: {integrity: sha512-6IpQ7mKUxRcZNLIObR0hz7lxsapSSIYNZJwXPGeF0mTVqGKFIXj1DQcMoT22S3ROcLyY/rz0PWaWZ9ayWmad9g==} engines: {node: '>= 0.8.0'} @@ -11828,6 +11950,10 @@ packages: resolution: {integrity: sha512-Xa4Nw17FS9ApQFJ9umLiJS4orGjm7ZzwUrwamcGQuHSzDyth9boKDaycYdDcZDuqYATXw4HFXgaqWTctW/v1HA==} engines: {node: '>=16 || 14 >=14.18'} + path-scurry@2.0.2: + resolution: {integrity: sha512-3O/iVVsJAPsOnpwWIeD+d6z/7PmqApyQePUtCndjatj/9I5LylHvt5qluFaBT3I5h3r1ejfR056c+FCv+NnNXg==} + engines: {node: 18 || 20 || >=22} + path-to-regexp@8.4.2: resolution: {integrity: sha512-qRcuIdP69NPm4qbACK+aDogI5CBDMi1jKe0ry5rSQJz8JVLsC7jV8XpiJjGRLLol3N+R5ihGYcrPLTno6pAdBA==} @@ -11848,6 +11974,9 @@ packages: resolution: {integrity: sha512-wQ0b/W4Fr01qtpHlqSqspcj3EhBvimsdh0KlHhH8HRZnMsEa0ea2fTULOXOS9ccQr3om+GcGRk4e+isrZWV8qQ==} engines: {node: '>=16.20.0'} + platform@1.3.6: + resolution: {integrity: sha512-fnWVljUchTro6RiCFvCXBbNhJc2NijN7oIQxbwsyL0buWJPG85v81ehlHI9fXrJsMNgTofEoWIQeClKpgxFLrg==} + playwright-core@1.61.1: resolution: {integrity: sha512-h7Qlt6m4REp25qvIdvbDtVmD4LqVXfpRxhORv9L0jzETM05p4fuPJ3dKyuSXQxDSbXnmS79HAgi9589lGSpLkg==} engines: {node: '>=18'} @@ -12226,6 +12355,10 @@ packages: tabbable@6.5.0: resolution: {integrity: sha512-wieBHXygIm7OyQOu5hQlkk62/WyCFYGlWg7L6/ZCUZwx0o398Zkn4pVmMyfYhfMG8kGrj/Krt8eIk6UKC6VzwA==} + tar@7.5.22: + resolution: {integrity: sha512-MFO/QzvtAOmJbkhOaCTvbGcFN9L9b+JunIsDwaKljSOdcLMea3NJ1k9Usz/rjdfSXTq4dfzfeS7W4p4YOAAHeA==} + engines: {node: '>=18'} + tinybench@2.9.0: resolution: {integrity: sha512-0+DUvqWMValLmha6lr4kD8iAMK1HzV0/aKnCtWb9v9641TnP/MFb7Pc2bxoxQjTXAErryXVgUOfv2YqNllqGeg==} @@ -12629,6 +12762,9 @@ packages: vscode-languageserver-protocol@3.18.2: resolution: {integrity: sha512-XRyDbT0Pp3sSNti3JmxVEUMySWCSi1hhM+/KUlCy1hV1zmrqpM1OwO12EAki8blhmLuIMpaJrYbo0OzGVfK2Qg==} + vscode-languageserver-textdocument@1.0.12: + resolution: {integrity: sha512-cxWNPesCnQCcMPeenjKKsOCKQZ/L6Tv19DTRIGuLWe32lyzWhihGVJ/rcckZXJxfdKCFvRLS3fpBIsV/ZGX4zA==} + vscode-languageserver-types@3.18.0: resolution: {integrity: sha512-8TsGPNMIMiiBdkORgRSvLjuiEIiAFtO+KssmYWxQ+uSVvlf7RjK8YKCOjPzZ+YA04jXEV7+7LvkSmHkhpNS99g==} @@ -12723,6 +12859,10 @@ packages: yallist@3.1.1: resolution: {integrity: sha512-a4UGQaWPH59mOXUYnAG2ewncQS4i4F43Tv3JoAM+s2VDAmS9NsK8GpDMLrCHPksFT7h3K6TOoUNn2pb7RoXx4g==} + yallist@5.0.0: + resolution: {integrity: sha512-YgvUTfwqyc7UXVMrB+SImsVYSmTS8X/tSrtdNZMImM+n7+QTriRXyXim0mBrTXNeqzVF0KWGgHPeiyViFFrNDw==} + engines: {node: '>=18'} + yaml@2.9.0: resolution: {integrity: sha512-2AvhNX3mb8zd6Zy7INTtSpl1F15HW6Wnqj0srWlkKLcpYl/gMIMJiyuGq2KeI2YFxUPjdlB+3Lc10seMLtL4cA==} engines: {node: '>= 14.6'} @@ -13326,6 +13466,8 @@ snapshots: dependencies: css-tree: 3.2.1 + '@bufbuild/protobuf@2.13.0': {} + '@chevrotain/types@11.1.2': {} '@clack/core@1.4.3': @@ -13340,6 +13482,15 @@ snapshots: fast-wrap-ansi: 0.2.2 sisteransi: 1.0.5 + '@connectrpc/connect-web@2.0.0-rc.3(@bufbuild/protobuf@2.13.0)(@connectrpc/connect@2.0.0-rc.3(@bufbuild/protobuf@2.13.0))': + dependencies: + '@bufbuild/protobuf': 2.13.0 + '@connectrpc/connect': 2.0.0-rc.3(@bufbuild/protobuf@2.13.0) + + '@connectrpc/connect@2.0.0-rc.3(@bufbuild/protobuf@2.13.0)': + dependencies: + '@bufbuild/protobuf': 2.13.0 + '@csstools/color-helpers@6.1.0': {} '@csstools/css-calc@3.2.1(@csstools/css-parser-algorithms@4.0.0(@csstools/css-tokenizer@4.0.0))(@csstools/css-tokenizer@4.0.0)': @@ -13756,6 +13907,12 @@ snapshots: wrap-ansi: 8.1.0 wrap-ansi-cjs: wrap-ansi@7.0.0 + '@isaacs/cliui@9.0.0': {} + + '@isaacs/fs-minipass@4.0.1': + dependencies: + minipass: 7.1.3 + '@joplin/turndown-plugin-gfm@1.0.67': {} '@jridgewell/gen-mapping@0.3.13': @@ -15310,6 +15467,8 @@ snapshots: chai@6.2.2: {} + chalk@5.6.2: {} + character-entities-html4@2.1.0: {} character-entities-legacy@3.0.0: {} @@ -15324,6 +15483,8 @@ snapshots: dependencies: readdirp: 5.0.0 + chownr@3.0.0: {} + clsx@2.1.1: {} color-convert@2.0.1: @@ -15340,6 +15501,8 @@ snapshots: commander@8.3.0: {} + compare-versions@6.1.1: {} + content-disposition@1.1.0: {} content-type@1.0.5: {} @@ -15611,6 +15774,11 @@ snapshots: diff@9.0.0: {} + dockerfile-ast@0.7.1: + dependencies: + vscode-languageserver-textdocument: 1.0.12 + vscode-languageserver-types: 3.18.0 + dom-accessibility-api@0.5.16: {} dompurify@3.4.11: @@ -15627,6 +15795,20 @@ snapshots: es-errors: 1.3.0 gopd: 1.2.0 + e2b@2.29.1: + dependencies: + '@bufbuild/protobuf': 2.13.0 + '@connectrpc/connect': 2.0.0-rc.3(@bufbuild/protobuf@2.13.0) + '@connectrpc/connect-web': 2.0.0-rc.3(@bufbuild/protobuf@2.13.0)(@connectrpc/connect@2.0.0-rc.3(@bufbuild/protobuf@2.13.0)) + chalk: 5.6.2 + compare-versions: 6.1.1 + dockerfile-ast: 0.7.1 + glob: 11.1.0 + openapi-fetch: 0.14.1 + platform: 1.3.6 + tar: 7.5.22 + undici: 7.28.0 + eastasianwidth@0.2.0: {} ecdsa-sig-formatter@1.0.11: @@ -16089,6 +16271,15 @@ snapshots: package-json-from-dist: 1.0.1 path-scurry: 1.11.1 + glob@11.1.0: + dependencies: + foreground-child: 3.3.1 + jackspeak: 4.2.3 + minimatch: 10.2.5 + minipass: 7.1.3 + package-json-from-dist: 1.0.1 + path-scurry: 2.0.2 + globals@17.7.0: {} globrex@0.1.2: {} @@ -16258,6 +16449,10 @@ snapshots: optionalDependencies: '@pkgjs/parseargs': 0.11.0 + jackspeak@4.2.3: + dependencies: + '@isaacs/cliui': 9.0.0 + jiti@2.7.0: {} jose@6.2.3: {} @@ -16942,6 +17137,10 @@ snapshots: minisearch@7.2.0: {} + minizlib@3.1.0: + dependencies: + minipass: 7.1.3 + mitt@3.0.1: {} mri@1.2.0: {} @@ -17062,6 +17261,12 @@ snapshots: ws: 8.21.0 zod: 4.4.3 + openapi-fetch@0.14.1: + dependencies: + openapi-typescript-helpers: 0.0.15 + + openapi-typescript-helpers@0.0.15: {} + optionator@0.9.4: dependencies: deep-is: 0.1.4 @@ -17194,6 +17399,11 @@ snapshots: lru-cache: 10.4.3 minipass: 7.1.3 + path-scurry@2.0.2: + dependencies: + lru-cache: 11.5.1 + minipass: 7.1.3 + path-to-regexp@8.4.2: {} pathe@2.0.3: {} @@ -17206,6 +17416,8 @@ snapshots: pkce-challenge@5.0.1: {} + platform@1.3.6: {} + playwright-core@1.61.1: {} playwright@1.61.1: @@ -17659,6 +17871,14 @@ snapshots: tabbable@6.5.0: {} + tar@7.5.22: + dependencies: + '@isaacs/fs-minipass': 4.0.1 + chownr: 3.0.0 + minipass: 7.1.3 + minizlib: 3.1.0 + yallist: 5.0.0 + tinybench@2.9.0: {} tinyexec@1.2.4: {} @@ -18092,6 +18312,8 @@ snapshots: vscode-jsonrpc: 9.0.1 vscode-languageserver-types: 3.18.0 + vscode-languageserver-textdocument@1.0.12: {} + vscode-languageserver-types@3.18.0: {} vue@3.5.39(typescript@6.0.3): @@ -18163,6 +18385,8 @@ snapshots: yallist@3.1.1: {} + yallist@5.0.0: {} + yaml@2.9.0: {} yocto-queue@0.1.0: {} diff --git a/scripts/gen-doc-graphs.ts b/scripts/gen-doc-graphs.ts index e254696491..e6d6acbdf2 100644 --- a/scripts/gen-doc-graphs.ts +++ b/scripts/gen-doc-graphs.ts @@ -70,6 +70,7 @@ const GROUP_ORDER = [ 'bash', 'pty', 'sandbox', + 'e2b', 'fs', 'skill', 'compact', @@ -321,12 +322,20 @@ const SERVICE_ROLES: ServiceRole[] = [ mode: 'core', note: 'Folds revisioned objective state from the session log and keeps live continuation activation process-local.', }, + { + key: 'e2b', + pkg: 'e2b', + title: 'E2B sandbox lifecycle owner', + mode: 'core', + consumers: ['fs-e2b', 'subprocess-e2b'], + note: 'Owns one shared E2B SDK handle, remote working directory, and final sandbox disposition so both fundamental E2B providers inhabit the same Linux runtime.', + }, { key: 'subprocess', pkg: 'subprocess', title: 'Subprocess seam', mode: 'seam', - implementations: ['subprocess-local'], + implementations: ['subprocess-local', 'subprocess-e2b'], consumers: ['bash-local', 'bash-sandbox', 'pty-local', 'lsp-local', 'subagent-acp', 'subagent-codex', 'subagent-claude-code'], note: 'The bash executors, the PTY shell backend, the LSP host, and the out-of-process ACP, Codex, and Claude Code subagent backends spawn through ctx.subprocess; the service owns process coordinates, tree/session lifetime, stdio dispositions, terminal mechanics, and kill escalation.', }, @@ -405,7 +414,7 @@ const SERVICE_ROLES: ServiceRole[] = [ pkg: 'fs', title: 'Filesystem provider seam', mode: 'seam', - implementations: ['fs-local', 'fs-sandbox'], + implementations: ['fs-local', 'fs-sandbox', 'fs-e2b'], consumers: ['tool-fs'], companions: ['fs-policy'], note: 'tool-fs executes read/write/edit through ctx.fs; fs-sandbox fences mutations by the shared sandbox mode; fs-policy contributes observed-state checks through the fs/* event gate.', From 55c0a44bd97113a3437b562df6acfb87a619582d Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Fri, 7 Aug 2026 21:11:31 +0800 Subject: [PATCH 103/108] fix(e2b): trim published files to the hardened allowlist Master's publication hardening enforces the exact package files list; the three E2B manifests still published src and d.ts maps. --- packages/e2b/e2b/package.json | 4 +--- packages/e2b/fs-e2b/package.json | 4 +--- packages/e2b/subprocess-e2b/package.json | 4 +--- 3 files changed, 3 insertions(+), 9 deletions(-) diff --git a/packages/e2b/e2b/package.json b/packages/e2b/e2b/package.json index 82fda6b303..a77e3d9464 100644 --- a/packages/e2b/e2b/package.json +++ b/packages/e2b/e2b/package.json @@ -21,9 +21,7 @@ "files": [ "lib/index.js", "lib/invariant.js", - "lib/types/**/*.d.ts", - "lib/types/**/*.d.ts.map", - "src" + "lib/types/**/*.d.ts" ], "license": "BSD-3-Clause", "peerDependencies": { diff --git a/packages/e2b/fs-e2b/package.json b/packages/e2b/fs-e2b/package.json index cc96cba6c9..02925ee7c4 100644 --- a/packages/e2b/fs-e2b/package.json +++ b/packages/e2b/fs-e2b/package.json @@ -21,9 +21,7 @@ "files": [ "lib/index.js", "lib/invariant.js", - "lib/types/**/*.d.ts", - "lib/types/**/*.d.ts.map", - "src" + "lib/types/**/*.d.ts" ], "license": "BSD-3-Clause", "peerDependencies": { diff --git a/packages/e2b/subprocess-e2b/package.json b/packages/e2b/subprocess-e2b/package.json index fa96e17293..abf3d5973c 100644 --- a/packages/e2b/subprocess-e2b/package.json +++ b/packages/e2b/subprocess-e2b/package.json @@ -21,9 +21,7 @@ "files": [ "lib/index.js", "lib/invariant.js", - "lib/types/**/*.d.ts", - "lib/types/**/*.d.ts.map", - "src" + "lib/types/**/*.d.ts" ], "license": "BSD-3-Clause", "peerDependencies": { From 72419bc33351b659874a7be32b3b74c2053e1815 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Fri, 7 Aug 2026 21:29:14 +0800 Subject: [PATCH 104/108] test(e2b): cover the environment tombstone removal The serializer's undefined-tombstone branch (from the seam's NodeJS.ProcessEnv migration) removes an ambient entry; the composition env test now exercises it against the scrubbed ambient KEEP entry. --- packages/e2b/subprocess-e2b/tests/subprocess.spec.ts | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts b/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts index 747dc936c6..d40c69df89 100644 --- a/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts +++ b/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts @@ -388,6 +388,8 @@ describe('E2BSubprocessHandle', () => { '--split-string': 'literal-value', DEEPSEEK_API_KEY: 'explicit-secret', DSH_MODE: 'test', + // The seam's tombstone: an explicit undefined removes the ambient entry. + KEEP: undefined, }, }), '/workspace/.dsh-e2b/processes/one') expect(handle.pid).toBe(-1) @@ -433,7 +435,7 @@ describe('E2BSubprocessHandle', () => { '/workspace/.dsh-e2b/processes/one/stderr.log', ]) expect(fake.writtenFileData.get('/workspace/.dsh-e2b/processes/one/environment')).toBe( - 'PATH=/bin\0KEEP=safe\0UNICODE=你好\0HOME=/home/user\0FOO-BAR=hyphen-value\0--split-string=literal-value\0DEEPSEEK_API_KEY=explicit-secret\0DSH_MODE=test\0', + 'PATH=/bin\0UNICODE=你好\0HOME=/home/user\0FOO-BAR=hyphen-value\0--split-string=literal-value\0DEEPSEEK_API_KEY=explicit-secret\0DSH_MODE=test\0', ) let piped = '' From e0a31475fc0767f1f081f86de237592ebd5e915d Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Sat, 8 Aug 2026 00:27:21 +0800 Subject: [PATCH 105/108] chore(rebase): re-record the packages-README pairing after the cascade The parent's round-two replay restored master's pairing record while this branch's e2b group row changes both sides; re-record the pair. --- packages/README.i18n.yaml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/packages/README.i18n.yaml b/packages/README.i18n.yaml index 7e74d63f31..143e7704c2 100644 --- a/packages/README.i18n.yaml +++ b/packages/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/README.md -README.md: 1516c20ff55f366fe6af6b0be5495958708f29e3 -README.zh.md: 42f4d2b8d924b19fd3cde863dafb8c59d9c078cf +README.md: 98fc87a6b26a42c58bb63cba96cd6af384f3ec71 +README.zh.md: 86a316dca77efd5e5eb59e3fca7d53890a34901d From 0569437ff3219757001c3d7501d88f24172a7575 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Sat, 8 Aug 2026 01:16:24 +0800 Subject: [PATCH 106/108] chore(rebase): regenerate third-party notices and the module graph Master turned THIRD_PARTY_NOTICES.md into generator output (the branch's hand-added e2b row carried a stale dotenv entry), and the cascade's ours-side resolution flattened the module graph to the parent's content; both regenerate from the current tree. --- THIRD_PARTY_NOTICES.md | 1 - docs/module-graph.md | 15 +++++++++++++++ 2 files changed, 15 insertions(+), 1 deletion(-) diff --git a/THIRD_PARTY_NOTICES.md b/THIRD_PARTY_NOTICES.md index 9060950bea..48f04c9799 100644 --- a/THIRD_PARTY_NOTICES.md +++ b/THIRD_PARTY_NOTICES.md @@ -57,7 +57,6 @@ External packages that a workspace package resolves at runtime. `scripts/install | [`clsx`](https://github.com/lukeed/clsx) | MIT | | [`commander`](https://github.com/tj/commander.js) | MIT | | [`diff`](https://github.com/kpdecker/jsdiff) | BSD-3-Clause | -| [`dotenv`](https://github.com/motdotla/dotenv) | BSD-2-Clause | | [`e2b`](https://github.com/e2b-dev/e2b) | MIT | | [`eventsource-parser`](https://github.com/rexxars/eventsource-parser) | MIT | | [`handlebars`](https://github.com/handlebars-lang/handlebars.js) | MIT | diff --git a/docs/module-graph.md b/docs/module-graph.md index 38a85a508a..91d584466b 100644 --- a/docs/module-graph.md +++ b/docs/module-graph.md @@ -204,6 +204,11 @@ flowchart TD pkg_credentials["credentials"] pkg_credentials_local["credentials-local"] end + subgraph group_e2b["packages/e2b"] + pkg_e2b["e2b"] + pkg_fs_e2b["fs-e2b"] + pkg_subprocess_e2b["subprocess-e2b"] + end subgraph group_examples["packages/examples"] pkg_acp_demo["acp-demo"] pkg_agent_spine_demo["agent-spine-demo"] @@ -309,6 +314,7 @@ flowchart TD pkg_client_web --> pkg_invariants pkg_client_web_react --> pkg_invariants pkg_code_runtime --> pkg_invariants + pkg_e2b --> pkg_invariants pkg_jsonrpc_demo --> pkg_invariants pkg_host_apiproxy --> pkg_invariants pkg_host_directory_picker --> pkg_invariants @@ -331,6 +337,9 @@ flowchart TD pkg_client_runtime --> pkg_typert_registry pkg_credentials --> pkg_brand pkg_credentials --> pkg_invariants + pkg_subprocess_e2b --> pkg_e2b + pkg_subprocess_e2b --> pkg_invariants + pkg_subprocess_e2b --> pkg_subprocess pkg_frontend_static --> pkg_host_webserver pkg_frontend_static --> pkg_invariants pkg_helper --> pkg_brand @@ -591,6 +600,9 @@ flowchart TD pkg_tmux_context --> pkg_bash pkg_tmux_context --> pkg_invariants pkg_tmux_context --> pkg_session + pkg_fs_e2b --> pkg_e2b + pkg_fs_e2b --> pkg_fs + pkg_fs_e2b --> pkg_invariants pkg_host_directory_picker_browse --> pkg_client_locale pkg_host_directory_picker_browse --> pkg_client_runtime pkg_host_directory_picker_browse --> pkg_client_ui_primitives @@ -1181,6 +1193,7 @@ flowchart TD | [`client-web`](../packages/client/web) | `client` | [`invariants`](../packages/support/invariants) | | [`client-web-react`](../packages/client/web-react) | `client` | [`invariants`](../packages/support/invariants) | | [`code-runtime`](../packages/code-runtime/code-runtime) | `code-runtime` | [`invariants`](../packages/support/invariants) | +| [`e2b`](../packages/e2b/e2b) | `e2b` | [`invariants`](../packages/support/invariants) | | [`jsonrpc-demo`](../packages/examples/jsonrpc-demo) | `examples` | [`invariants`](../packages/support/invariants) | | [`host-apiproxy`](../packages/host/apiproxy) | `host` | [`invariants`](../packages/support/invariants) | | [`host-directory-picker`](../packages/host/directory-picker) | `host` | [`invariants`](../packages/support/invariants) | @@ -1195,6 +1208,7 @@ flowchart TD | [`client-hmr`](../packages/client/hmr) | `client` | [`client-modules`](../packages/client/modules), [`host-webserver`](../packages/host/webserver), [`invariants`](../packages/support/invariants) | | [`client-runtime`](../packages/client/runtime) | `client` | [`invariants`](../packages/support/invariants), [`type-meta`](../packages/typert/type-meta), [`typert-registry`](../packages/typert/registry) | | [`credentials`](../packages/credentials/credentials) | `credentials` | [`brand`](../packages/util/brand), [`invariants`](../packages/support/invariants) | +| [`subprocess-e2b`](../packages/e2b/subprocess-e2b) | `e2b` | [`e2b`](../packages/e2b/e2b), [`invariants`](../packages/support/invariants), [`subprocess`](../packages/subprocess/subprocess) | | [`frontend-static`](../packages/host/frontend-static) | `host` | [`host-webserver`](../packages/host/webserver), [`invariants`](../packages/support/invariants) | | [`helper`](../packages/sdk/helper) | `sdk` | [`brand`](../packages/util/brand), [`invariants`](../packages/support/invariants), [`subprocess`](../packages/subprocess/subprocess) | | [`telemetry`](../packages/sdk/telemetry) | `sdk` | [`brand`](../packages/util/brand), [`invariants`](../packages/support/invariants), [`paths`](../packages/util/paths) | @@ -1264,6 +1278,7 @@ flowchart TD | [`client-ui-layout`](../packages/client/ui-layout) | `client` | [`client-runtime`](../packages/client/runtime), [`client-ui-slots`](../packages/client/ui-slots), [`client-ui-theme`](../packages/client/ui-theme), [`invariants`](../packages/support/invariants) | | [`time-context`](../packages/context/time-context) | `context` | [`agent`](../packages/core/agent), [`invariants`](../packages/support/invariants), [`session`](../packages/core/session) | | [`tmux-context`](../packages/context/tmux-context) | `context` | [`agent`](../packages/core/agent), [`bash`](../packages/bash/bash), [`invariants`](../packages/support/invariants), [`session`](../packages/core/session) | +| [`fs-e2b`](../packages/e2b/fs-e2b) | `e2b` | [`e2b`](../packages/e2b/e2b), [`fs`](../packages/fs/fs), [`invariants`](../packages/support/invariants) | | [`host-directory-picker-browse`](../packages/host/directory-picker-browse) | `host` | [`client-locale`](../packages/client/locale), [`client-runtime`](../packages/client/runtime), [`client-ui-primitives`](../packages/client/ui-primitives), [`client-ui-slots`](../packages/client/ui-slots), [`client-ui-workspace`](../packages/client/ui-workspace), [`invariants`](../packages/support/invariants) | | [`host-directory-picker-native`](../packages/host/directory-picker-native) | `host` | [`client-runtime`](../packages/client/runtime), [`client-ui-slots`](../packages/client/ui-slots), [`client-ui-workspace`](../packages/client/ui-workspace), [`invariants`](../packages/support/invariants) | | [`lsp-local`](../packages/lsp/lsp-local) | `lsp` | [`brand`](../packages/util/brand), [`fs`](../packages/fs/fs), [`invariants`](../packages/support/invariants), [`llm`](../packages/llm/llm), [`lsp`](../packages/lsp/lsp), [`subprocess`](../packages/subprocess/subprocess), [`timeout`](../packages/util/timeout) | From 10fc9405c772bbb75868d62d04487f3f529dabb4 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Sat, 8 Aug 2026 17:29:10 +0800 Subject: [PATCH 107/108] docs(e2b): re-home the overlay section on master's README Master rewrote the headless README around the profile CLI and removed cli-demo; the replay had resurrected the old body. The branch's claim is the E2B overlay section alone, now inserted into master's text with the credential-gated live composition e2e as the runnable instruction (the config-file demo runner no longer exists). --- examples/headless-agent/README.i18n.yaml | 4 ++-- examples/headless-agent/README.md | 22 +++++++++------------- examples/headless-agent/README.zh.md | 22 +++++++++------------- 3 files changed, 20 insertions(+), 28 deletions(-) diff --git a/examples/headless-agent/README.i18n.yaml b/examples/headless-agent/README.i18n.yaml index 56c02d3e0a..4a71194e5f 100644 --- a/examples/headless-agent/README.i18n.yaml +++ b/examples/headless-agent/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write examples/headless-agent/README.md -README.md: 16c45d304eb0f06a2b062518b053db683cd402c0 -README.zh.md: 72d2813d174e74a3bf93c363b268499a44f1cd4f +README.md: 6e80e56dec70c2be341ae5dfbad70e13a5109715 +README.zh.md: ea8c41b9ae75f0cee3edd78e59408f37c19182d3 diff --git a/examples/headless-agent/README.md b/examples/headless-agent/README.md index 16c45d304e..6e80e56dec 100644 --- a/examples/headless-agent/README.md +++ b/examples/headless-agent/README.md @@ -2,7 +2,7 @@ English | [中文](README.zh.md) -Headless one-shot agent wiring: DeepSeek V4 + local bash and filesystem tools + subagent delegation + workflows and fresh-agent Ralph iteration + `todo_write` + JSONL persistence, with [`@deepseek-ai/dsh-cli-demo`](../../packages/examples/cli-demo) as the app front door. +This directory owns the replay and real-model test composition for a headless coding agent: DeepSeek V4 + local bash and filesystem tools + subagent delegation + workflows and fresh-agent Ralph iteration + `todo_write` + JSONL persistence. It explicitly mounts the shared agent spine, one root agent, persistence, and checkpoint policy; it is not a second product front door. ## Run it @@ -10,27 +10,23 @@ Headless one-shot agent wiring: DeepSeek V4 + local bash and filesystem tools + # repo root .env (gitignored) or exported env: # DEEPSEEK_API_KEY=sk-… # DEEPSEEK_BASE_URL=https://… # optional; defaults to the public API -pnpm run demo:headless "fix the failing test in this workspace" -pnpm run demo:headless --output-format json -- "summarize the implementation" -pnpm run demo:headless --output-format stream-json -- "run the focused tests" +pnpm run dsh run "fix the failing test in this workspace" ``` -Exactly one nonblank positional task is required; quote tasks containing spaces. There is no `-p` flag. `text` prints the last text-bearing assistant message, `json` prints one DSH-native result record, and `stream-json` emits the top-level session's canonical task-turn events before that record. Child sessions surface only through parent tool events and results. +The product command is [`dsh run`](../../apps/cli/README.md): it accepts one nonblank task, creates and persists a fresh session, prints the final assistant text, and exits. The root `demo:headless` script is only an alias of that command. -Each invocation creates and persists a fresh session, runs all model and tool steps in one turn, flushes, disposes, and exits. This is non-interactive automation: there is no prompt, approval, resume, second turn, or stdin context. The configured tools can mutate the launch workspace, run commands, spawn child agents, and consume provider tokens. +Snapshot suites run this directory's configuration through [`tests/fixtures/headless-driver.ts`](tests/fixtures/headless-driver.ts), an unexported test-only process that emits canonical session events as JSONL before its result record. That stream is test infrastructure, not a supported CLI output format. Child sessions surface only through parent tool events and results. ## E2B POC overlay -[`e2b.cordis.yml`](e2b.cordis.yml) replaces the local filesystem and subprocess providers with one shared E2B sandbox while retaining `dsh-bash-local` and the same model-facing tools. Put `E2B_API_KEY` beside `DEEPSEEK_API_KEY` in the gitignored root `.env`, then run: +[`e2b.cordis.yml`](e2b.cordis.yml) replaces the local filesystem and subprocess providers with one shared E2B sandbox while retaining `dsh-bash-local` and the same model-facing tools. Put `E2B_API_KEY` beside `DEEPSEEK_API_KEY` in the gitignored root `.env`, then run the credential-gated live composition, which drives FS, Bash, PTY, and LSP in one sandbox and proves final deletion: ```sh -pnpm run demo:e2b "create hello.txt, read it back, and run pwd" +pnpm exec vitest run --config vitest.e2e.config.ts packages/e2b/e2b/tests/composition.e2e.ts ``` -The overlay creates the same absolute cwd inside the sandbox, but it does not upload or mount the host workspace. File and Bash mutations exist only in E2B; Cordis, model calls, agent/session state, session logs, skills, and SDK buffers remain on the host. The demo kills its sandbox on timeout and disposal. It is a provider-composition POC, not a whole-harness migration or a workspace-sync feature. +The overlay creates the same absolute cwd inside the sandbox, but it does not upload or mount the host workspace. File and Bash mutations exist only in E2B; Cordis, model calls, agent/session state, session logs, skills, and SDK buffers remain on the host. The composition kills its sandbox on timeout and disposal. It is a provider-composition POC, not a whole-harness migration or a workspace-sync feature. -## Advanced and snapshot wiring +## Advanced configuration -[`advanced.cordis.yml`](advanced.cordis.yml) adds Code Mode and the Cordis tools to the shipped leaf. [`advanced.cordis.snapshot.yml`](advanced.cordis.snapshot.yml) replaces only the live LLM with replay. The tests under [`tests/`](tests/) own the keyless real-Loader smoke, key-gated world-verified smoke, and the `stream-json` replay snapshot with its parent and child session fixtures. - -The package-level [CLI contract](../../packages/examples/cli-demo/README.md) documents output records, exit status, cancellation, persistence, and model/token effects. +[`advanced.cordis.yml`](advanced.cordis.yml) adds Code Mode and the Cordis tools to the test composition. diff --git a/examples/headless-agent/README.zh.md b/examples/headless-agent/README.zh.md index 72d2813d17..ea8c41b9ae 100644 --- a/examples/headless-agent/README.zh.md +++ b/examples/headless-agent/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -无头单次 agent(智能体)接线:DeepSeek V4 + 本地 bash 与文件系统工具 + subagent 委托 + 工作流与新 agent Ralph 迭代 + `todo_write` + JSONL 持久化,并以 [`@deepseek-ai/dsh-cli-demo`](../../packages/examples/cli-demo) 作为应用入口。 +本目录负责 headless coding agent(智能体)的回放和真实模型测试组装:DeepSeek V4 + 本地 bash 与文件系统工具 + subagent 委托 + 工作流与全新 agent Ralph 迭代 + `todo_write` + JSONL 持久化。本目录显式挂载共享 agent 主干、一个根 agent、持久化和检查点策略;它不是第二个产品入口。 ## 运行 @@ -10,27 +10,23 @@ # repo root .env (gitignored) or exported env: # DEEPSEEK_API_KEY=sk-… # DEEPSEEK_BASE_URL=https://… # optional; defaults to the public API -pnpm run demo:headless "fix the failing test in this workspace" -pnpm run demo:headless --output-format json -- "summarize the implementation" -pnpm run demo:headless --output-format stream-json -- "run the focused tests" +pnpm run dsh run "fix the failing test in this workspace" ``` -必须提供且只能提供一个非空位置任务;含空格的任务需要加引号。没有 `-p` 标志。`text` 打印最后一条包含文本的 assistant 消息,`json` 打印一条 DSH 原生结果记录,`stream-json` 则在该记录之前发出顶层会话的规范任务轮次事件。子会话只通过父工具事件和结果对外显示。 +产品命令是 [`dsh run`](../../apps/cli/README.md):它接受一项非空任务,创建并持久化新会话,打印最终 assistant 文本,然后退出。根目录的 `demo:headless` 脚本只是该命令的别名。 -每次调用都会创建并持久化新会话,在一个轮次中运行所有模型和工具步骤,然后刷新、释放并退出。这是非交互式自动化:没有提示符、批准、恢复、第二轮次或 stdin 上下文。已配置工具可以修改启动 workspace、运行命令、spawn 子 agent,并消耗提供方 token。 +快照套件通过 [`tests/fixtures/headless-driver.ts`](tests/fixtures/headless-driver.ts) 运行本目录的配置。这个未导出且仅供测试使用的进程会在结果记录之前,以 JSONL 发出规范会话事件。该事件流属于测试基础设施,不是受支持的 CLI(命令行界面)输出格式。子会话只通过父会话的工具事件和结果对外显示。 ## E2B POC overlay -[`e2b.cordis.yml`](e2b.cordis.yml) 使用一个共享 E2B 沙箱替换本地文件系统与进程管理提供方,同时保留 `dsh-bash-local` 和相同的面向模型工具。请在 git 忽略的根目录 `.env` 中,将 `E2B_API_KEY` 与 `DEEPSEEK_API_KEY` 放在一起,然后运行: +[`e2b.cordis.yml`](e2b.cordis.yml) 使用一个共享 E2B 沙箱替换本地文件系统与进程管理提供方,同时保留 `dsh-bash-local` 和相同的面向模型工具。请在 git 忽略的根目录 `.env` 中,将 `E2B_API_KEY` 与 `DEEPSEEK_API_KEY` 放在一起,然后运行凭据门控的实机组合测试;它在同一个沙箱中驱动 FS、Bash、PTY 和 LSP,并证明沙箱最终被删除: ```sh -pnpm run demo:e2b "create hello.txt, read it back, and run pwd" +pnpm exec vitest run --config vitest.e2e.config.ts packages/e2b/e2b/tests/composition.e2e.ts ``` -该 overlay 会在沙箱中创建拼写相同的绝对 cwd,但不会上传或挂载宿主工作区。文件与 Bash 变更只存在于 E2B;Cordis、模型调用、agent/会话状态、会话日志、skill(技能)和 SDK 缓冲仍在宿主上。演示会在超时和资源释放时终止其沙箱。它是提供方组合 POC,而不是完整 harness 迁移或工作区同步功能。 +该 overlay 会在沙箱中创建拼写相同的绝对 cwd,但不会上传或挂载宿主工作区。文件与 Bash 变更只存在于 E2B;Cordis、模型调用、agent/会话状态、会话日志、skill(技能)和 SDK 缓冲仍在宿主上。该组合会在超时和资源释放时终止其沙箱。它是提供方组合 POC,而不是完整 harness 迁移或工作区同步功能。 -## 高级与快照接线 +## 高级配置 -[`advanced.cordis.yml`](advanced.cordis.yml) 在已交付叶节点上添加 Code Mode 和 Cordis 工具。[`advanced.cordis.snapshot.yml`](advanced.cordis.snapshot.yml) 只将实时 LLM(大语言模型)替换为回放。[`tests/`](tests/) 下的测试拥有无密钥真实 Loader 冒烟测试、密钥门控的外部状态验证冒烟测试,以及带父子会话 fixture(测试前置数据)的 `stream-json` 回放快照。 - -包级 [CLI 契约](../../packages/examples/cli-demo/README.md)记录输出记录、退出状态、取消、持久化以及模型/token 影响。 +[`advanced.cordis.yml`](advanced.cordis.yml) 在测试组装中添加 Code Mode 和 Cordis 工具。 From 450ec99325ebef5d21fdcae1a0ef62b98027dd68 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Sat, 8 Aug 2026 22:37:38 +0800 Subject: [PATCH 108/108] fix(e2b): address the v7 review round MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - restore the parent's AGENTS.md body around this branch's e2b row (the replay had resurrected the pre-profile-CLI layout, deleting the api/, bundle/, settings/, and credentials/ groups); the fs/lsp row condensations stay for the word budget - aggregate the composite rollback failure, not the original error, when private state cleanup also fails — a surviving remote group is no longer hidden by a later cleanup error (triple-failure regression) - own the state directory before the makeDir RPC, matching the terminal transaction, so a cancellation racing a committed creation still enters cleanup - drop the hidden pollMs parameter defaults; the schemastery default is the one home, and tests pass the cadence explicitly - restore spawn/spawnTerminal graceMs validation to the seam's documented bound (the earlier removal cited subprocess-local as not validating; it does), with rejection regressions --- AGENTS.md | 19 +- docs/config-catalog.md | 2 +- docs/module-graph.md | 3 +- packages/e2b/subprocess-e2b/package.json | 2 + packages/e2b/subprocess-e2b/src/index.ts | 15 ++ packages/e2b/subprocess-e2b/src/process.ts | 8 +- packages/e2b/subprocess-e2b/src/terminal.ts | 2 +- .../subprocess-e2b/tests/subprocess.spec.ts | 190 +++++++++++------- .../e2b/subprocess-e2b/tests/terminal.spec.ts | 96 +++++---- packages/e2b/subprocess-e2b/tsconfig.json | 15 +- pnpm-lock.yaml | 3 + 11 files changed, 218 insertions(+), 137 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 87f545ee86..61812e586c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -12,10 +12,11 @@ DeepSeek Harness SDK is a plugin-based agent harness on vendored Cordis: **every vendor/ Vendored Cordis source — manifest + sync procedure in vendor/README.md packages/ @deepseek-ai/dsh- workspaces at packages/// core/ product API spine: session, system-prompt, tools, agent, agent-loop + api/ Remote BFF assembly and TypeRT RPC gateway typert/ type graph generator, loader, and runtime registry llm/ LLM seam + DeepSeek adapters (direct-fetch + pi-ai design twin) e2b/ E2B POC: sandbox + FS/subprocess adapters - bash/ bash executor seam + local impl + model-facing bash tools + bash/ bash executor seam + local/pwsh impls + model-facing shell tools subprocess/ subprocess seam + local process-tree impl pty/ persistent PTY seam/backend/tools fs/ filesystem seam/backends/policy/tools @@ -25,6 +26,7 @@ packages/ @deepseek-ai/dsh- workspaces at packages/// compact/ compaction seam + basic backend context/ request-context plugins subagent/ subagent seam + spawn/fork/ACP backends + delegation tool + bundle/ profile plugin bundles: installable patch layers for dsh --profile workflow/ workflow seam + worker-thread engine + workflow tool todo/ todo_write tool plan/ plan mode as logged per-agent collaboration state @@ -32,9 +34,11 @@ packages/ @deepseek-ai/dsh- workspaces at packages/// cordis/ self-referential toolset: the agent inspects/mounts plugins in its own runtime hooks/ Claude Code/Codex hook bridges + shared wire-protocol library session-persistence/ persistence seam + JSONL/SQLite backends + settings/ user-settings seam + file-backed provider + credentials/ credential-reference seam + env-over-.env provider acp/ automation-only Agent Client Protocol server - ui/ TUI/JSON-RPC bridges; boot, approval, interaction plugins - examples/ demo bundles (agent-spine + TUI/CLI/ACP/JSON-RPC bins) leaves load + ui/ JSON-RPC bridge; boot, approval, interaction plugins + examples/ demo bundles (agent-spine + CLI/ACP/JSON-RPC bins) leaves load support/ dev/test infrastructure util/ zero-dependency utilities python/ Python SDK and bundled runtime (see python/README.md) @@ -56,7 +60,7 @@ pnpm run clean # remove build outputs and safe residue from deleted pa pnpm run test # vitest unit tests pnpm run test:coverage # CI coverage gate: per-file 100% on packages/*/*/src pnpm run test:e2e # real-API tests; self-skip without DEEPSEEK_API_KEY -pnpm run test:snapshot # keyless ACP/headless/TUI replay vs expected outputs; filter: -t +pnpm run test:snapshot # keyless ACP/headless replay vs expected outputs; filter: -t pnpm run test:snapshot:record # re-record expected outputs (needs key) pnpm run typecheck pnpm run lint @@ -67,7 +71,6 @@ pnpm run hygiene # knip + publint + workspace constraints + NodeNext cons pnpm run doc-sync # all documentation gates; leaf list in scripts/run-gates.ts pnpm run website:build # VitePress build (doubles as dead-link check) pnpm run demo:headless "task" # one-shot agent (needs DEEPSEEK_API_KEY) -pnpm run demo:tui # full-screen TUI coding agent (needs DEEPSEEK_API_KEY) pnpm run demo:cordis # the agent modifies its own runtime (needs key) pnpm run demo:acp # ACP automation server (needs DEEPSEEK_API_KEY) ``` @@ -78,7 +81,7 @@ When required `gh`, `pnpm`, build, test, or generator commands fail because the ### Run relevant checks locally -Agents MUST run relevant tests and checks before pushing; select them with [dsh-pre-push-checks](.agents/skills/dsh-pre-push-checks/SKILL.md) and report only commands run. +Run checks before pushes via [dsh-pre-push-checks](.agents/skills/dsh-pre-push-checks/SKILL.md); report only commands run. After `gh stack sync`, validate immediately; do not merge before checks pass. - Match evidence to the surface: focused tests for behavior, snapshots for model or user output, `doc-sync` for docs, build/hygiene and built smokes for published paths, and real-API e2e for provider behavior. - Never default to the full suite or repeat a passing check for commit or push. CI owns exhaustive coverage and the platform matrix; rehearse all locally only by explicit request, for CI diagnosis, or for an irreducibly repository-wide change. @@ -91,7 +94,7 @@ Real-API tests and demos read `DEEPSEEK_API_KEY`, optional `DEEPSEEK_BASE_URL`, ## Conventions - Every npm package is `@deepseek-ai/dsh-`; vendored packages keep upstream names and are `private: true`. `cordis` is a peerDependency (+ dev) of every harness package. -- ESM everywhere (`"type": "module"`). Cross-package imports use package names; in-package relative imports include `.ts`. Config subprocesses run built `lib/` under plain Node; source regressions use their declared launcher ([testing policy](docs/testing.md#test-subprocess-launch-modes)). The `dsh` CLI source launch runs through tsx's ESM-only hook (`node --import tsx/esm`); modules it reaches must stay ESM (no CJS-only shapes) — Node's native TypeScript modes are unavailable across the engines range ([source-launch contract](.agents/notes/implemented/architecture/2026-07-29-dsh-source-launch-tsx-esm.md)). TUI/Web `cordis.yml` bare plugins must appear in their resolver manifest's `dependencies`; `verify-cordis-config` enforces it. +- ESM everywhere (`"type": "module"`). Cross-package imports use package names; in-package relative imports include `.ts`. Config subprocesses run built `lib/` under plain Node; source regressions use their declared launcher ([testing policy](docs/testing.md#test-subprocess-launch-modes)). The `dsh` CLI source launch runs through tsx's ESM-only hook (`node --import tsx/esm`); modules it reaches must stay ESM (no CJS-only shapes) — Node's native TypeScript modes are unavailable across the engines range ([source-launch contract](.agents/notes/implemented/architecture/2026-07-29-dsh-source-launch-tsx-esm.md)). Raw/Web `cordis.yml` bare plugins must appear in their resolver manifest's `dependencies`; `verify-cordis-config` enforces it. - **Registrations are effects**: every contribution goes through `ctx.effect()` / `ctx.on()`; a registry's `register()` returns the disposer. - **Runtime invariants assert owned relationships.** Check authoritative event streams or mutable data, not service or method presence, plugin metadata or effects, or fixed pure examples. If a package has no plausible relationship, an explained empty companion is correct ([package contract](packages/AGENTS.md)). - **Typed events use declaration merging** and merge-extensible maps. Event JSDoc needs `@mode` and payload `@param`; scoped keys absent from payloads need `@dshScopeScan unsupported`. Public service methods document parameters and non-void returns. @@ -115,7 +118,7 @@ Real-API tests and demos read `DEEPSEEK_API_KEY`, optional `DEEPSEEK_BASE_URL`, - **Testing policy** — [docs/testing.md](docs/testing.md). Every non-trivial model- or product-user-visible behavior change adds or updates a keyless snapshot through a real runnable example in the same PR; package tests, e2e-only assertions, and mock-only fixtures do not substitute for the assembled application transcript. Fixtures must replay on macOS/Linux; fix fixtures, not normalizers. - **A tool's UI render intent is part of its design**, decided up front (`generic`/`terminal`/`diff`, `locations`); presentation methods are pure functions of `args` ([cookbook](docs/cookbook/adding-a-tool.md)). - **Plan unit, e2e, and snapshot coverage** for new seams, lifecycle shapes, and transcript surfaces; missing snapshot-harness support is part of the implementation, not deferred follow-up. -- **Use incremental merge commits.** Split independent changes. Pushed history may be rewritten before review; afterward prefer new commits. Fix the introducing PR before merging down-stack. If the base advances mid-merge, finish the checkpoint, push when authorized, then merge the newer tip separately ([rationale](.agents/notes/implemented/process/2026-07-26-incremental-pr-base-retargeting.md)). +- **Choose PR history deliberately.** Split independent changes; fix the introducing PR before propagation. Standalone PRs and official stacks may merge-forward or rebase after review. Rewrites use `--force-with-lease`, abort on remote movement, never raw `--force`; an in-progress merge-forward preserves its checkpoint before taking a newer base ([rationale](.agents/notes/implemented/process/2026-08-02-native-github-stacks-and-optional-rebases.md)). - **Label PRs:** one kind (`feature`/`bug-fix`/`doc`/`testing`/`cleanup`), each matching area; the [taxonomy](.agents/notes/implemented/process/2026-07-25-semantic-pr-label-taxonomy.md) is extensible. - TODO markers: `FIXME`/`TODO`/`XXX` by urgency ([semantics](docs/development.md)). - Files end with exactly one trailing newline; `git diff --cached --check` (pre-commit) gates it. diff --git a/docs/config-catalog.md b/docs/config-catalog.md index 58b319d662..c74358fd39 100644 --- a/docs/config-catalog.md +++ b/docs/config-catalog.md @@ -1841,7 +1841,7 @@ export interface Config { } ``` -Source: [`packages/e2b/subprocess-e2b/src/index.ts:24`](../packages/e2b/subprocess-e2b/src/index.ts) +Source: [`packages/e2b/subprocess-e2b/src/index.ts:25`](../packages/e2b/subprocess-e2b/src/index.ts) ## `@deepseek-ai/dsh-system-prompt` diff --git a/docs/module-graph.md b/docs/module-graph.md index 91d584466b..2e71473379 100644 --- a/docs/module-graph.md +++ b/docs/module-graph.md @@ -340,6 +340,7 @@ flowchart TD pkg_subprocess_e2b --> pkg_e2b pkg_subprocess_e2b --> pkg_invariants pkg_subprocess_e2b --> pkg_subprocess + pkg_subprocess_e2b --> pkg_timeout pkg_frontend_static --> pkg_host_webserver pkg_frontend_static --> pkg_invariants pkg_helper --> pkg_brand @@ -1208,7 +1209,7 @@ flowchart TD | [`client-hmr`](../packages/client/hmr) | `client` | [`client-modules`](../packages/client/modules), [`host-webserver`](../packages/host/webserver), [`invariants`](../packages/support/invariants) | | [`client-runtime`](../packages/client/runtime) | `client` | [`invariants`](../packages/support/invariants), [`type-meta`](../packages/typert/type-meta), [`typert-registry`](../packages/typert/registry) | | [`credentials`](../packages/credentials/credentials) | `credentials` | [`brand`](../packages/util/brand), [`invariants`](../packages/support/invariants) | -| [`subprocess-e2b`](../packages/e2b/subprocess-e2b) | `e2b` | [`e2b`](../packages/e2b/e2b), [`invariants`](../packages/support/invariants), [`subprocess`](../packages/subprocess/subprocess) | +| [`subprocess-e2b`](../packages/e2b/subprocess-e2b) | `e2b` | [`e2b`](../packages/e2b/e2b), [`invariants`](../packages/support/invariants), [`subprocess`](../packages/subprocess/subprocess), [`timeout`](../packages/util/timeout) | | [`frontend-static`](../packages/host/frontend-static) | `host` | [`host-webserver`](../packages/host/webserver), [`invariants`](../packages/support/invariants) | | [`helper`](../packages/sdk/helper) | `sdk` | [`brand`](../packages/util/brand), [`invariants`](../packages/support/invariants), [`subprocess`](../packages/subprocess/subprocess) | | [`telemetry`](../packages/sdk/telemetry) | `sdk` | [`brand`](../packages/util/brand), [`invariants`](../packages/support/invariants), [`paths`](../packages/util/paths) | diff --git a/packages/e2b/subprocess-e2b/package.json b/packages/e2b/subprocess-e2b/package.json index abf3d5973c..007b88f1c9 100644 --- a/packages/e2b/subprocess-e2b/package.json +++ b/packages/e2b/subprocess-e2b/package.json @@ -28,6 +28,7 @@ "@deepseek-ai/dsh-e2b": "^0.0.1", "@deepseek-ai/dsh-invariants": "^0.0.1", "@deepseek-ai/dsh-subprocess": "^0.0.1", + "@deepseek-ai/dsh-timeout": "^0.0.1", "cordis": "^4.0.0-rc.7" }, "dependencies": { @@ -37,6 +38,7 @@ "@deepseek-ai/dsh-e2b": "workspace:^", "@deepseek-ai/dsh-invariants": "workspace:^", "@deepseek-ai/dsh-subprocess": "workspace:^", + "@deepseek-ai/dsh-timeout": "workspace:^", "cordis": "^4.0.0-rc.7" } } diff --git a/packages/e2b/subprocess-e2b/src/index.ts b/packages/e2b/subprocess-e2b/src/index.ts index 38b1d32465..c6d2dbd4eb 100644 --- a/packages/e2b/subprocess-e2b/src/index.ts +++ b/packages/e2b/subprocess-e2b/src/index.ts @@ -9,6 +9,7 @@ import { posix } from 'node:path' import { Context } from 'cordis' import z from 'schemastery' import { SubprocessService } from '@deepseek-ai/dsh-subprocess' +import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout' import type { SubprocessHandle, SubprocessSpawnSpec, @@ -35,6 +36,18 @@ interface TerminalSetup { controller: AbortController } +/** + * Enforce the seam's documented grace bound (positive, finite, one Node timer), + * matching subprocess-local's spawn-time check; an unbounded grace would make + * the remote force-escalation deadline unreachable. + * @param graceMs - The spec's cleanup grace in milliseconds. + */ +function requireRepresentableGrace(graceMs: number): void { + if (!Number.isFinite(graceMs) || graceMs <= 0 || graceMs > MAX_TIMER_DELAY_MS) { + throw new Error(`subprocess graceMs must be a positive finite number no greater than ${MAX_TIMER_DELAY_MS}`) + } +} + /** E2B command manager registered as `ctx.subprocess`. */ export class E2BSubprocessService extends SubprocessService { static inject = ['e2b'] @@ -130,6 +143,7 @@ export class E2BSubprocessService extends SubprocessService { if (program === undefined || program.length === 0) { throw new Error('invalid argv: expected a non-empty program name at argv[0]') } + requireRepresentableGrace(spec.graceMs) if (spec.signal?.aborted === true) { throw new Error(`aborted before spawn: ${String(spec.signal.reason)}`) } @@ -153,6 +167,7 @@ export class E2BSubprocessService extends SubprocessService { if (program === undefined || program.length === 0) { throw new Error('subprocess-e2b: terminal argv must contain a program') } + requireRepresentableGrace(spec.graceMs) spec.signal?.throwIfAborted() const stateDir = posix.join(this.ctx.e2b.runtimeRoot, 'terminals', randomUUID()) const done = Promise.withResolvers() diff --git a/packages/e2b/subprocess-e2b/src/process.ts b/packages/e2b/subprocess-e2b/src/process.ts index feb21a70bc..c80a2f5bd9 100644 --- a/packages/e2b/subprocess-e2b/src/process.ts +++ b/packages/e2b/subprocess-e2b/src/process.ts @@ -194,7 +194,7 @@ export class E2BSubprocessHandle implements SubprocessHandle { private readonly runtime: E2BSandboxService, private readonly spec: SubprocessSpawnSpec, readonly stateDir: string, - private readonly pollMs = 20, + private readonly pollMs: number, ) { this.paths = { pid: posix.join(stateDir, 'pid'), @@ -378,7 +378,7 @@ export class E2BSubprocessHandle implements SubprocessHandle { await this.removeFailedState(sandbox) } catch (cleanupError: unknown) { failure = new AggregateError( - [error, cleanupError], + [failure, cleanupError], 'subprocess-e2b: command failed and private state cleanup failed', ) } @@ -398,8 +398,10 @@ export class E2BSubprocessHandle implements SubprocessHandle { const signal = this.terminationController.signal const ambient = await readRemoteEnvironment(sandbox, signal) this.controlEnvs = bootstrapEnvironment(ambient) - await sandbox.files.makeDir(this.stateDir, { signal }) + // Own the directory before the request: a cancellation racing a committed + // creation must still enter cleanup (removal tolerates an absent path). this.stateDirectoryCreated = true + await sandbox.files.makeDir(this.stateDir, { signal }) await sandbox.commands.run( `chmod 700 -- ${quoteE2BShellArg(this.stateDir)}`, commandOpts(this.controlEnvs, signal), diff --git a/packages/e2b/subprocess-e2b/src/terminal.ts b/packages/e2b/subprocess-e2b/src/terminal.ts index fd10ecdd8e..c8f3bb57b7 100644 --- a/packages/e2b/subprocess-e2b/src/terminal.ts +++ b/packages/e2b/subprocess-e2b/src/terminal.ts @@ -459,7 +459,7 @@ export async function spawnE2BTerminal( runtime: E2BSandboxService, spec: SubprocessTerminalSpawnSpec, stateDir: string, - pollMs = 20, + pollMs: number, ): Promise { const sandbox = await runtime.getSandbox() spec.signal?.throwIfAborted() diff --git a/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts b/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts index d40c69df89..153b144ccd 100644 --- a/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts +++ b/packages/e2b/subprocess-e2b/tests/subprocess.spec.ts @@ -324,6 +324,16 @@ async function flush(): Promise { await new Promise(resolve => setTimeout(resolve, 0)) } +/** Construct the handle under test with the config default the service would pass. */ +function testHandle( + runtime: ConstructorParameters[0], + spec: ConstructorParameters[1], + stateDir: string, + pollMs = 20, +): E2BSubprocessHandle { + return new E2BSubprocessHandle(runtime, spec, stateDir, pollMs) +} + describe('E2BOutputReader', () => { it('decodes base64 across arbitrary callback boundaries and rejects malformed framing', () => { const decoder = new E2BBase64Decoder() @@ -379,7 +389,7 @@ describe('E2BSubprocessHandle', () => { const fake = new FakeSandbox() fake.processGroupId = '4343\n' fake.deferStart() - const handle = new E2BSubprocessHandle(runtime(fake), spec({ + const handle = testHandle(runtime(fake), spec({ argv: ['tool', 'argument with spaces'], stdio: { stdin: 'pipe', stdout: 'pipe', stderr: { maxBytes: 8, spill: { maxBytes: 32 } } }, env: { @@ -450,10 +460,23 @@ describe('E2BSubprocessHandle', () => { await expect(handle.waitForExit()).resolves.toBe(true) }) + it('rejects an unrepresentable graceMs before any remote work', () => { + const ctx = new Context() + const service = Object.create(E2BSubprocessService.prototype) as E2BSubprocessService + Reflect.set(service, 'disposing', false) + Reflect.set(service, 'ctx', ctx) + for (const graceMs of [0, -1, Number.NaN, Number.POSITIVE_INFINITY]) { + expect(() => service.spawn(spec({ graceMs }))).toThrow('graceMs must be a positive finite number') + void expect(service.spawnTerminal({ + argv: ['bash'], cwd: '/w', rows: 24, cols: 80, graceMs, + })).rejects.toThrow('graceMs must be a positive finite number') + } + }) + it('rejects malformed environment entries before command start', async () => { for (const env of [{ 'BAD=NAME': 'x' }, { BAD: 'x\0INJECTED=1' }]) { const fake = new FakeSandbox() - const handle = new E2BSubprocessHandle(runtime(fake), spec({ env }), '/runtime/invalid-environment') + const handle = testHandle(runtime(fake), spec({ env }), '/runtime/invalid-environment') await expect(handle.done).rejects.toThrow('environment entries') expect(fake.startOptions).toBeUndefined() expect(fake.removed).toContain('/runtime/invalid-environment') @@ -462,7 +485,7 @@ describe('E2BSubprocessHandle', () => { it('preserves UTF-8 bytes when the ASCII transport is split across callbacks', async () => { const fake = new FakeSandbox() - const handle = new E2BSubprocessHandle(runtime(fake), spec({ + const handle = testHandle(runtime(fake), spec({ stdio: { stdin: 'ignore', stdout: 'pipe', stderr: { maxBytes: 4 } }, }), '/runtime/split-utf8') await flush() @@ -478,14 +501,14 @@ describe('E2BSubprocessHandle', () => { it('rejects malformed output transport without confusing it with a consumer sink failure', async () => { const fake = new FakeSandbox() - const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/malformed-output') + const handle = testHandle(runtime(fake), spec(), '/runtime/malformed-output') await flush() await fake.stdoutWire('%\n') fake.finish() await expect(handle.done).rejects.toThrow('invalid base64 output transport') const stderrFake = new FakeSandbox() - const stderrHandle = new E2BSubprocessHandle(runtime(stderrFake), spec(), '/runtime/malformed-stderr') + const stderrHandle = testHandle(runtime(stderrFake), spec(), '/runtime/malformed-stderr') await flush() await stderrFake.stderrWire('%\n') stderrFake.finish() @@ -494,7 +517,7 @@ describe('E2BSubprocessHandle', () => { it('rejects a naturally completed command whose encoder omits its completion frame', async () => { const fake = new FakeSandbox() - const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/incomplete-output') + const handle = testHandle(runtime(fake), spec(), '/runtime/incomplete-output') await flush() fake.alive = false fake.handle.succeed(0) @@ -503,7 +526,7 @@ describe('E2BSubprocessHandle', () => { it('bounds descendant-held output draining and withholds the incomplete spill', async () => { const fake = new FakeSandbox() - const handle = new E2BSubprocessHandle(runtime(fake), spec({ graceMs: 5 }), '/runtime/drain-bound') + const handle = testHandle(runtime(fake), spec({ graceMs: 5 }), '/runtime/drain-bound') await flush() await fake.stdout('leader-output') fake.exitStatus = '0\n' @@ -529,7 +552,7 @@ describe('E2BSubprocessHandle', () => { return false }) as typeof process.stdout.write) try { - const handle = new E2BSubprocessHandle(runtime(fake), spec({ + const handle = testHandle(runtime(fake), spec({ graceMs: 5, stdio: { stdin: 'ignore', stdout: 'inherit', stderr: { maxBytes: 4 } }, }), '/runtime/inherit-backpressure') @@ -555,7 +578,7 @@ describe('E2BSubprocessHandle', () => { it('waits for lossless raw-pipe output after the direct status is published', async () => { const fake = new FakeSandbox() - const handle = new E2BSubprocessHandle(runtime(fake), spec({ + const handle = testHandle(runtime(fake), spec({ graceMs: 1, stdio: { stdin: 'ignore', stdout: 'pipe', stderr: { maxBytes: 4 } }, }), '/runtime/pipe-drain') @@ -580,7 +603,7 @@ describe('E2BSubprocessHandle', () => { it('accepts clean encoder completion inside the output-drain grace', async () => { const fake = new FakeSandbox() - const handle = new E2BSubprocessHandle(runtime(fake), spec({ graceMs: 100 }), '/runtime/drain-complete') + const handle = testHandle(runtime(fake), spec({ graceMs: 100 }), '/runtime/drain-complete') await flush() fake.exitStatus = '0\n' fake.afterStatusRead = () => { @@ -598,7 +621,7 @@ describe('E2BSubprocessHandle', () => { fake.delaysKill = true fake.delaysKillCompletion = true fake.sdkKillStops = false - const handle = new E2BSubprocessHandle(runtime(fake), spec({ graceMs: 5 }), '/runtime/drain-signal') + const handle = testHandle(runtime(fake), spec({ graceMs: 5 }), '/runtime/drain-signal') await flush() handle.terminate() @@ -614,7 +637,7 @@ describe('E2BSubprocessHandle', () => { it('preserves a published nonzero exit code when termination settles the SDK inside the drain grace', async () => { const fake = new FakeSandbox() - const handle = new E2BSubprocessHandle(runtime(fake), spec({ graceMs: 100 }), '/runtime/drain-signal-settled') + const handle = testHandle(runtime(fake), spec({ graceMs: 100 }), '/runtime/drain-signal-settled') await flush() fake.exitStatus = '7\n' fake.afterStatusRead = () => { @@ -628,7 +651,7 @@ describe('E2BSubprocessHandle', () => { it('rejects an invalid direct-command exit status', async () => { const fake = new FakeSandbox() - const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/invalid-status') + const handle = testHandle(runtime(fake), spec(), '/runtime/invalid-status') await flush() fake.exitStatus = '999\n' await expect(handle.done).rejects.toThrow('invalid exit code') @@ -639,7 +662,7 @@ describe('E2BSubprocessHandle', () => { it('rolls back a published process group before rejecting a monitoring failure', async () => { const fake = new FakeSandbox() fake.statusError = new Error('status transport failed') - const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/status-failure') + const handle = testHandle(runtime(fake), spec(), '/runtime/status-failure') await expect(handle.done).rejects.toThrow('status transport failed') expect(fake.commandsSeen).toContain('kill -TERM -- -4242') @@ -650,7 +673,7 @@ describe('E2BSubprocessHandle', () => { failed.statusError = new Error('status transport failed') failed.signalErrors.push(new Error('TERM transport failed'), new Error('KILL transport failed')) failed.handle.killError = new Error('SDK kill failed') - const retained = new E2BSubprocessHandle(runtime(failed), spec({ graceMs: 1 }), '/runtime/status-cleanup-failure') + const retained = testHandle(runtime(failed), spec({ graceMs: 1 }), '/runtime/status-cleanup-failure') await expect(retained.done).rejects.toThrow( 'command monitoring failed and process-group rollback did not reach quiescence', @@ -659,13 +682,30 @@ describe('E2BSubprocessHandle', () => { failed.handle.killError = undefined retained.terminate() await expect(retained.waitForExit()).resolves.toBe(true) + + // A state-cleanup failure on top preserves the rollback failure instead of + // re-aggregating only the original monitoring error. + const triple = new FakeSandbox() + triple.statusError = new Error('status transport failed') + triple.signalErrors.push(new Error('TERM transport failed'), new Error('KILL transport failed')) + triple.handle.killError = new Error('SDK kill failed') + triple.nextRemoveError = new Error('state cleanup failed') + const tripleHandle = testHandle(runtime(triple), spec({ graceMs: 1 }), '/runtime/triple-failure') + const failure = await tripleHandle.done.catch((error: unknown) => error as AggregateError) + expect(failure).toBeInstanceOf(AggregateError) + expect((failure as AggregateError).message).toContain('private state cleanup failed') + const nested = (failure as AggregateError).errors[0] as AggregateError + expect(nested.message).toContain('rollback did not reach quiescence') + triple.handle.killError = undefined + tripleHandle.terminate() + await expect(tripleHandle.waitForExit()).resolves.toBe(true) }) it('surfaces deferred piped-stdin write and close failures as stream errors', async () => { const writeFake = new FakeSandbox() writeFake.deferStart() vi.spyOn(writeFake.handle, 'sendStdin').mockRejectedValueOnce('stdin rejected') - const writeHandle = new E2BSubprocessHandle(runtime(writeFake), spec({ + const writeHandle = testHandle(runtime(writeFake), spec({ stdio: { stdin: 'pipe', stdout: { maxBytes: 4 }, stderr: { maxBytes: 4 } }, }), '/runtime/stdin-write-error') const writeError = once(writeHandle.stdin!, 'error') @@ -677,7 +717,7 @@ describe('E2BSubprocessHandle', () => { const closeFake = new FakeSandbox() vi.spyOn(closeFake.handle, 'closeStdin').mockRejectedValueOnce(new Error('close rejected')) - const closeHandle = new E2BSubprocessHandle(runtime(closeFake), spec({ + const closeHandle = testHandle(runtime(closeFake), spec({ stdio: { stdin: 'pipe', stdout: { maxBytes: 4 }, stderr: { maxBytes: 4 } }, }), '/runtime/stdin-close-error') await flush() @@ -690,7 +730,7 @@ describe('E2BSubprocessHandle', () => { it('collects bounded tails, retains valid spills, and maps natural nonzero exits', async () => { const fake = new FakeSandbox() - const handle = new E2BSubprocessHandle(runtime(fake), spec({ + const handle = testHandle(runtime(fake), spec({ stdio: { stdin: { data: 'batch' }, stdout: { maxBytes: 4, spill: { maxBytes: 16 } }, @@ -716,7 +756,7 @@ describe('E2BSubprocessHandle', () => { it('removes a spill once the complete stream exceeds its cap', async () => { const fake = new FakeSandbox() - const handle = new E2BSubprocessHandle(runtime(fake), spec({ + const handle = testHandle(runtime(fake), spec({ stdio: { stdin: 'ignore', stdout: { maxBytes: 2, spill: { maxBytes: 3 } }, stderr: 'inherit' }, }), '/runtime/oversize') await flush() @@ -736,7 +776,7 @@ describe('E2BSubprocessHandle', () => { it('contains remote spill-removal failures and routes empty inherited output', async () => { const fake = new FakeSandbox() fake.nextRemoveError = new Error('already removed') - const handle = new E2BSubprocessHandle(runtime(fake), spec({ + const handle = testHandle(runtime(fake), spec({ stdio: { stdin: 'ignore', stdout: 'inherit', stderr: { maxBytes: 4, spill: { maxBytes: 8 } } }, }), '/runtime/remove-error') await flush() @@ -749,7 +789,7 @@ describe('E2BSubprocessHandle', () => { it('terminates a process group with TERM and reports the signal outcome', async () => { const fake = new FakeSandbox() - const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/term') + const handle = testHandle(runtime(fake), spec(), '/runtime/term') await flush() handle.terminate() handle.terminate() @@ -767,7 +807,7 @@ describe('E2BSubprocessHandle', () => { it('makes termination a permanent no-op after natural quiescence is observed', async () => { const fake = new FakeSandbox() - const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/natural-quiescence') + const handle = testHandle(runtime(fake), spec(), '/runtime/natural-quiescence') await flush() fake.finish() await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) @@ -784,7 +824,7 @@ describe('E2BSubprocessHandle', () => { it('treats a zombie-only process group as quiescent', async () => { const fake = new FakeSandbox() fake.zombieOnly = true - const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/zombie-quiescence') + const handle = testHandle(runtime(fake), spec(), '/runtime/zombie-quiescence') await flush() await expect(handle.waitForExit()).resolves.toBe(true) @@ -800,7 +840,7 @@ describe('E2BSubprocessHandle', () => { fake.signalErrors.push(new Error('TERM transport failed'), new Error('KILL transport failed')) fake.handle.killError = new Error('SDK kill failed') fake.deferSignals() - const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/quiescent-race') + const handle = testHandle(runtime(fake), spec(), '/runtime/quiescent-race') await flush() handle.terminate() @@ -823,7 +863,7 @@ describe('E2BSubprocessHandle', () => { const fake = new FakeSandbox() fake.trapsTerm = true fake.handle.killError = new Error('already gone') - const handle = new E2BSubprocessHandle(runtime(fake), spec({ graceMs: 1 }), '/runtime/kill') + const handle = testHandle(runtime(fake), spec({ graceMs: 1 }), '/runtime/kill') await flush() handle.terminate() await expect(handle.done).resolves.toEqual({ exitCode: null, signal: 'SIGKILL' }) @@ -838,7 +878,7 @@ describe('E2BSubprocessHandle', () => { fake.delaysKill = true fake.delaysKillCompletion = true fake.sdkKillStops = false - const handle = new E2BSubprocessHandle(runtime(fake), spec({ graceMs: 1 }), '/runtime/termination-fence') + const handle = testHandle(runtime(fake), spec({ graceMs: 1 }), '/runtime/termination-fence') await flush() handle.terminate() await vi.waitFor(() => { expect(fake.handle.kills).toBe(1) }) @@ -856,7 +896,7 @@ describe('E2BSubprocessHandle', () => { it('honors termination requested before asynchronous startup finishes', async () => { const fake = new FakeSandbox() fake.deferStart() - const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/deferred-kill') + const handle = testHandle(runtime(fake), spec(), '/runtime/deferred-kill') handle.terminate() fake.releaseStart() await expect(handle.done).resolves.toEqual({ exitCode: null, signal: 'SIGTERM' }) @@ -879,7 +919,7 @@ describe('E2BSubprocessHandle', () => { signal?.addEventListener('abort', rejectAbort, { once: true }) }) } - const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/stalled-preparation') + const handle = testHandle(runtime(fake), spec(), '/runtime/stalled-preparation') await vi.waitFor(() => { expect(preparationSignal).toBeDefined() }) handle.terminate() @@ -893,7 +933,7 @@ describe('E2BSubprocessHandle', () => { const fake = new FakeSandbox() fake.deferProcessGroupRead() fake.signalErrors.push(commandError(1), commandError(1)) - const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/pre-publication-kill') + const handle = testHandle(runtime(fake), spec(), '/runtime/pre-publication-kill') await vi.waitFor(() => { expect(fake.startOptions).toBeDefined() }) handle.terminate() @@ -913,7 +953,7 @@ describe('E2BSubprocessHandle', () => { fake.delaysKillCompletion = true fake.sdkKillStops = false fake.handle.killResult = false - const handle = new E2BSubprocessHandle(runtime(fake), spec({ graceMs: 1 }), '/runtime/provisional-sdk-false') + const handle = testHandle(runtime(fake), spec({ graceMs: 1 }), '/runtime/provisional-sdk-false') await vi.waitFor(() => { expect(fake.startOptions).toBeDefined() }) handle.terminate() @@ -936,7 +976,7 @@ describe('E2BSubprocessHandle', () => { calls += 1 return calls === 1 ? fake.sandbox : await reconnect.promise }) - const handle = new E2BSubprocessHandle(delayedRuntime, spec(), '/runtime/pre-publication-observer') + const handle = testHandle(delayedRuntime, spec(), '/runtime/pre-publication-observer') await vi.waitFor(() => { expect(fake.startOptions).toBeDefined() }) handle.terminate() @@ -957,7 +997,7 @@ describe('E2BSubprocessHandle', () => { fake.deferProcessGroupRead() fake.trapsTerm = true fake.handle.killError = new Error('SDK kill unavailable') - const handle = new E2BSubprocessHandle(runtime(fake), spec({ graceMs: 1 }), '/runtime/pre-publication-group-kill') + const handle = testHandle(runtime(fake), spec({ graceMs: 1 }), '/runtime/pre-publication-group-kill') await vi.waitFor(() => { expect(fake.startOptions).toBeDefined() }) handle.terminate() @@ -971,7 +1011,7 @@ describe('E2BSubprocessHandle', () => { fake.deferProcessGroupRead() fake.signalErrors.push(new Error('TERM transport failed'), new Error('KILL transport failed')) fake.handle.killError = new Error('SDK kill failed') - const handle = new E2BSubprocessHandle(runtime(fake), spec({ graceMs: 1 }), '/runtime/pre-publication-failure') + const handle = testHandle(runtime(fake), spec({ graceMs: 1 }), '/runtime/pre-publication-failure') await vi.waitFor(() => { expect(fake.startOptions).toBeDefined() }) handle.terminate() @@ -987,7 +1027,7 @@ describe('E2BSubprocessHandle', () => { absentGroup.deferProcessGroupRead() absentGroup.signalErrors.push(commandError(1), commandError(1)) absentGroup.handle.killError = new Error('SDK kill failed without a provisional group') - const absentHandle = new E2BSubprocessHandle( + const absentHandle = testHandle( runtime(absentGroup), spec({ graceMs: 1 }), '/runtime/pre-publication-absent-group', @@ -1005,7 +1045,7 @@ describe('E2BSubprocessHandle', () => { optimisticSdk.deferProcessGroupRead() optimisticSdk.signalErrors.push(commandError(1), commandError(1)) optimisticSdk.sdkKillStops = false - const optimisticHandle = new E2BSubprocessHandle( + const optimisticHandle = testHandle( runtime(optimisticSdk), spec({ graceMs: 1 }), '/runtime/pre-publication-optimistic-sdk', @@ -1021,14 +1061,14 @@ describe('E2BSubprocessHandle', () => { it('honors an already-aborted signal when constructing the asynchronous handle directly', async () => { const fake = new FakeSandbox() - const handle = new E2BSubprocessHandle(runtime(fake), spec({ signal: AbortSignal.abort('stop') }), '/runtime/pre-aborted') + const handle = testHandle(runtime(fake), spec({ signal: AbortSignal.abort('stop') }), '/runtime/pre-aborted') await expect(handle.done).resolves.toEqual({ exitCode: null, signal: 'SIGTERM' }) }) it('reacts to a signal that aborts after the remote command has started', async () => { const fake = new FakeSandbox() const controller = new AbortController() - const handle = new E2BSubprocessHandle(runtime(fake), spec({ signal: controller.signal }), '/runtime/live-abort') + const handle = testHandle(runtime(fake), spec({ signal: controller.signal }), '/runtime/live-abort') await flush() controller.abort('stop') await expect(handle.done).resolves.toEqual({ exitCode: null, signal: 'SIGTERM' }) @@ -1036,7 +1076,7 @@ describe('E2BSubprocessHandle', () => { it('can terminate a surviving process group after the command leader settles', async () => { const fake = new FakeSandbox() - const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/surviving-group') + const handle = testHandle(runtime(fake), spec(), '/runtime/surviving-group') await flush() await fake.completeOutput() fake.handle.succeed(0) @@ -1054,7 +1094,7 @@ describe('E2BSubprocessHandle', () => { it('bounds waitForExit while startup or a live group is pending', async () => { const fake = new FakeSandbox() fake.deferStart() - const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/wait') + const handle = testHandle(runtime(fake), spec(), '/runtime/wait') const beforeStart = new AbortController() const pending = handle.waitForExit(beforeStart.signal) beforeStart.abort() @@ -1071,7 +1111,7 @@ describe('E2BSubprocessHandle', () => { const terminatingFake = new FakeSandbox() terminatingFake.deferStart() - const terminating = new E2BSubprocessHandle(runtime(terminatingFake), spec(), '/runtime/wait-termination-start') + const terminating = testHandle(runtime(terminatingFake), spec(), '/runtime/wait-termination-start') terminating.terminate() const beforeHandle = new AbortController() const handlePending = terminating.waitForExit(beforeHandle.signal) @@ -1083,7 +1123,7 @@ describe('E2BSubprocessHandle', () => { it('bounds both sides of the liveness-poll abort race', async () => { const fake = new FakeSandbox() - const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/poll-abort') + const handle = testHandle(runtime(fake), spec(), '/runtime/poll-abort') await flush() const beforeTick = new AbortController() @@ -1114,7 +1154,7 @@ describe('E2BSubprocessHandle', () => { it('observes a live group across one successful bounded poll', async () => { const fake = new FakeSandbox() - const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/poll-success') + const handle = testHandle(runtime(fake), spec(), '/runtime/poll-success') await flush() setTimeout(() => { fake.finish() }, 1) await expect(handle.waitForExit(new AbortController().signal)).resolves.toBe(true) @@ -1124,7 +1164,7 @@ describe('E2BSubprocessHandle', () => { it('treats startup failure as no live tree and contains readiness rejection', async () => { const fake = new FakeSandbox() fake.backgroundError = new Error('start failed') - const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/fail') + const handle = testHandle(runtime(fake), spec(), '/runtime/fail') await expect(handle.done).rejects.toThrow('start failed') expect(handle.pid).toBe(-1) expect(fake.removed).toContain('/runtime/fail/environment') @@ -1132,7 +1172,7 @@ describe('E2BSubprocessHandle', () => { await expect(handle.waitForExit()).resolves.toBe(true) handle.terminate() - const unavailableHandle = new E2BSubprocessHandle( + const unavailableHandle = testHandle( runtime(new FakeSandbox(), async () => { throw new Error('sandbox unavailable') }), spec(), '/runtime/unavailable-start', @@ -1142,14 +1182,14 @@ describe('E2BSubprocessHandle', () => { const envFailure = new FakeSandbox() envFailure.envError = new Error('ambient lookup failed') - const envHandle = new E2BSubprocessHandle(runtime(envFailure), spec(), '/runtime/env-failure') + const envHandle = testHandle(runtime(envFailure), spec(), '/runtime/env-failure') await expect(envHandle.done).rejects.toThrow('ambient lookup failed') expect(envFailure.removed).toEqual([]) const expectEnvironmentFailure = async (name: string, wire: string, message: string): Promise => { const fake = new FakeSandbox() fake.environmentWire = wire - const failed = new E2BSubprocessHandle(runtime(fake), spec(), `/runtime/${name}`) + const failed = testHandle(runtime(fake), spec(), `/runtime/${name}`) await expect(failed.done).rejects.toThrow(message) } const encodedEnvironment = Buffer.from('PATH=/bin\0').toString('base64') @@ -1180,13 +1220,13 @@ describe('E2BSubprocessHandle', () => { const cleanupFailure = new FakeSandbox() cleanupFailure.backgroundError = new Error('start failed before credential consumption') cleanupFailure.nextRemoveError = new Error('credential cleanup failed') - const cleanupHandle = new E2BSubprocessHandle(runtime(cleanupFailure), spec(), '/runtime/cleanup-failure') + const cleanupHandle = testHandle(runtime(cleanupFailure), spec(), '/runtime/cleanup-failure') await expect(cleanupHandle.done).rejects.toThrow('command failed and private state cleanup failed') const absentState = new FakeSandbox() absentState.backgroundError = new Error('start failed after external cleanup') absentState.nextRemoveError = new FileNotFoundError('already removed') - const absentHandle = new E2BSubprocessHandle(runtime(absentState), spec(), '/runtime/absent-state') + const absentHandle = testHandle(runtime(absentState), spec(), '/runtime/absent-state') await expect(absentHandle.done).rejects.toThrow('start failed after external cleanup') }) @@ -1194,7 +1234,7 @@ describe('E2BSubprocessHandle', () => { const fake = new FakeSandbox() fake.deferStart() fake.backgroundError = new Error('start failed') - const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/fail-with-signal') + const handle = testHandle(runtime(fake), spec(), '/runtime/fail-with-signal') const waiting = handle.waitForExit(new AbortController().signal) fake.releaseStart() await expect(handle.done).rejects.toThrow('start failed') @@ -1209,7 +1249,7 @@ describe('E2BSubprocessHandle', () => { if (calls === 1) return fake.sandbox throw new Error('connection unavailable') }) - const handle = new E2BSubprocessHandle(unavailable, spec(), '/runtime/unavailable') + const handle = testHandle(unavailable, spec(), '/runtime/unavailable') await flush() await expect(handle.waitForExit()).rejects.toThrow('connection unavailable') fake.finish() @@ -1224,7 +1264,7 @@ describe('E2BSubprocessHandle', () => { calls += 1 return calls === 1 ? fake.sandbox : await reconnect.promise }) - const handle = new E2BSubprocessHandle(unavailable, spec(), '/runtime/reconnect-abort') + const handle = testHandle(unavailable, spec(), '/runtime/reconnect-abort') await flush() const controller = new AbortController() const waiting = handle.waitForExit(controller.signal) @@ -1238,7 +1278,7 @@ describe('E2BSubprocessHandle', () => { it('returns false when a liveness request itself is aborted and surfaces other probe failures', async () => { const fake = new FakeSandbox() - const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/probe') + const handle = testHandle(runtime(fake), spec(), '/runtime/probe') await flush() const controller = new AbortController() controller.abort() @@ -1251,7 +1291,7 @@ describe('E2BSubprocessHandle', () => { it('treats a timeout-killed sandbox as quiescent during liveness probing', async () => { const fake = new FakeSandbox() - const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/expired-sandbox') + const handle = testHandle(runtime(fake), spec(), '/runtime/expired-sandbox') await flush() fake.finish() await handle.done @@ -1263,7 +1303,7 @@ describe('E2BSubprocessHandle', () => { it('treats a missing sandbox handle as quiescent during liveness acquisition', async () => { const fake = new FakeSandbox() let calls = 0 - const handle = new E2BSubprocessHandle(runtime(fake, async () => { + const handle = testHandle(runtime(fake, async () => { calls += 1 if (calls === 1) return fake.sandbox throw new SandboxNotFoundError('sandbox expired') @@ -1280,7 +1320,7 @@ describe('E2BSubprocessHandle', () => { it('treats sandbox loss during termination as quiescent', async () => { const fake = new FakeSandbox() let calls = 0 - const handle = new E2BSubprocessHandle(runtime(fake, async () => { + const handle = testHandle(runtime(fake, async () => { calls += 1 if (calls === 1) return fake.sandbox throw new SandboxNotFoundError('sandbox expired') @@ -1298,7 +1338,7 @@ describe('E2BSubprocessHandle', () => { it('makes batch stdin close failures best-effort', async () => { const fake = new FakeSandbox() vi.spyOn(fake.handle, 'sendStdin').mockRejectedValueOnce(new Error('closed')) - const handle = new E2BSubprocessHandle(runtime(fake), spec({ + const handle = testHandle(runtime(fake), spec({ stdio: { stdin: { data: 'ignored' }, stdout: { maxBytes: 4 }, stderr: { maxBytes: 4 } }, }), '/runtime/stdin-closed') await flush() @@ -1309,7 +1349,7 @@ describe('E2BSubprocessHandle', () => { it('rejects malformed SDK process ids and non-command settlement failures', async () => { const invalidPid = new FakeSandbox() invalidPid.handle.pid = 0 - const invalid = new E2BSubprocessHandle(runtime(invalidPid), spec(), '/runtime/invalid-pid') + const invalid = testHandle(runtime(invalidPid), spec(), '/runtime/invalid-pid') await expect(invalid.done).rejects.toThrow(/invalid command pid 0/) expect(invalidPid.handle.kills).toBe(1) expect(invalidPid.removed).toContain('/runtime/invalid-pid/environment') @@ -1318,7 +1358,7 @@ describe('E2BSubprocessHandle', () => { const failedRollback = new FakeSandbox() failedRollback.handle.pid = 0 failedRollback.handle.killError = new Error('invalid handle kill failed') - const retained = new E2BSubprocessHandle(runtime(failedRollback), spec(), '/runtime/invalid-pid-retained') + const retained = testHandle(runtime(failedRollback), spec(), '/runtime/invalid-pid-retained') await expect(retained.done).rejects.toThrow('invalid command pid rollback did not reach quiescence') await expect(retained.waitForExit()).rejects.toThrow('invalid handle kill failed') failedRollback.handle.killError = undefined @@ -1326,7 +1366,7 @@ describe('E2BSubprocessHandle', () => { await expect(retained.waitForExit()).resolves.toBe(true) const crashedFake = new FakeSandbox() - const crashed = new E2BSubprocessHandle(runtime(crashedFake), spec(), '/runtime/crashed') + const crashed = testHandle(runtime(crashedFake), spec(), '/runtime/crashed') await flush() crashedFake.alive = false crashedFake.handle.crash(new Error('command transport failed')) @@ -1339,7 +1379,7 @@ describe('E2BSubprocessHandle', () => { invalidGroup.delaysKill = true invalidGroup.sdkKillStops = false invalidGroup.afterProbe = () => { invalidGroup.alive = false } - const invalid = new E2BSubprocessHandle(runtime(invalidGroup), spec(), '/runtime/invalid-group') + const invalid = testHandle(runtime(invalidGroup), spec(), '/runtime/invalid-group') await expect(invalid.done).rejects.toThrow(/invalid process-group id/) expect(invalidGroup.handle.kills).toBe(1) expect(invalidGroup.commandsSeen).toContain('kill -KILL -- -4242') @@ -1351,14 +1391,14 @@ describe('E2BSubprocessHandle', () => { unsafeGroup.delaysKill = true unsafeGroup.sdkKillStops = false unsafeGroup.afterProbe = () => { unsafeGroup.alive = false } - const unsafe = new E2BSubprocessHandle(runtime(unsafeGroup), spec(), '/runtime/unsafe-group') + const unsafe = testHandle(runtime(unsafeGroup), spec(), '/runtime/unsafe-group') await expect(unsafe.done).rejects.toThrow(/unsafe published process-group id 1/) expect(unsafeGroup.commandsSeen).not.toContain('kill -KILL -- -1') await expect(unsafe.waitForExit()).resolves.toBe(true) const absentGroup = new FakeSandbox() absentGroup.processGroupId = '' - const absent = new E2BSubprocessHandle(runtime(absentGroup), spec(), '/runtime/absent-group') + const absent = testHandle(runtime(absentGroup), spec(), '/runtime/absent-group') await flush() absentGroup.finish() await expect(absent.done).rejects.toThrow(/exited before publishing/) @@ -1372,7 +1412,7 @@ describe('E2BSubprocessHandle', () => { fake.processGroupId = 'not-a-pid\n' fake.signalError = new Error('rollback signal failed') fake.handle.killError = new Error('SDK kill failed') - const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/failed-rollback') + const handle = testHandle(runtime(fake), spec(), '/runtime/failed-rollback') let failure: unknown try { @@ -1403,7 +1443,7 @@ describe('E2BSubprocessHandle', () => { naturallyGone.processGroupId = 'not-a-pid\n' naturallyGone.signalError = new Error('rollback signal failed') naturallyGone.handle.killError = new Error('SDK kill failed') - const observed = new E2BSubprocessHandle(runtime(naturallyGone), spec(), '/runtime/failed-rollback-observed') + const observed = testHandle(runtime(naturallyGone), spec(), '/runtime/failed-rollback-observed') await expect(observed.done).rejects.toThrow('process-group publication failed') naturallyGone.alive = false await expect(observed.waitForExit()).resolves.toBe(true) @@ -1412,7 +1452,7 @@ describe('E2BSubprocessHandle', () => { it('waits for delayed process-group publication', async () => { const fake = new FakeSandbox() fake.processGroupReads.push('', '4242\n') - const handle = new E2BSubprocessHandle(runtime(fake), spec(), '/runtime/delayed-group') + const handle = testHandle(runtime(fake), spec(), '/runtime/delayed-group') await vi.waitFor(() => { expect(handle.pid).toBe(4242) }) fake.finish() await expect(handle.done).resolves.toEqual({ exitCode: 0, signal: null }) @@ -1420,7 +1460,7 @@ describe('E2BSubprocessHandle', () => { it('handles output backpressure and contains a stderr sink failure', async () => { const fake = new FakeSandbox() - const handle = new E2BSubprocessHandle(runtime(fake), spec({ + const handle = testHandle(runtime(fake), spec({ stdio: { stdin: 'ignore', stdout: 'pipe', stderr: 'pipe' }, }), '/runtime/backpressure') await flush() @@ -1445,7 +1485,7 @@ describe('E2BSubprocessHandle', () => { it('settles output backpressure when the consumer closes the pipe', async () => { const fake = new FakeSandbox() - const handle = new E2BSubprocessHandle(runtime(fake), spec({ + const handle = testHandle(runtime(fake), spec({ stdio: { stdin: 'ignore', stdout: 'pipe', stderr: { maxBytes: 4 } }, }), '/runtime/backpressure-close') await flush() @@ -1462,7 +1502,7 @@ describe('E2BSubprocessHandle', () => { it('breaks output backpressure when termination owns the command', async () => { const fake = new FakeSandbox() - const handle = new E2BSubprocessHandle(runtime(fake), spec({ + const handle = testHandle(runtime(fake), spec({ stdio: { stdin: 'ignore', stdout: 'pipe', stderr: { maxBytes: 4 } }, }), '/runtime/backpressure-termination') await flush() @@ -1484,7 +1524,7 @@ describe('E2BSubprocessHandle', () => { it('settles backpressure when a synchronous pipe write starts termination', async () => { const fake = new FakeSandbox() - const handle = new E2BSubprocessHandle(runtime(fake), spec({ + const handle = testHandle(runtime(fake), spec({ stdio: { stdin: 'ignore', stdout: 'pipe', stderr: { maxBytes: 4 } }, }), '/runtime/backpressure-synchronous-termination') await flush() @@ -1500,7 +1540,7 @@ describe('E2BSubprocessHandle', () => { it('contains a pipe callback failure instead of rejecting command settlement', async () => { const fake = new FakeSandbox() - const handle = new E2BSubprocessHandle(runtime(fake), spec({ + const handle = testHandle(runtime(fake), spec({ stdio: { stdin: 'ignore', stdout: 'pipe', stderr: { maxBytes: 4 } }, }), '/runtime/pipe-error') await flush() @@ -1516,14 +1556,14 @@ describe('E2BSubprocessHandle', () => { const gone = new FakeSandbox() gone.trapsTerm = true gone.signalError = commandError(1) - const goneHandle = new E2BSubprocessHandle(runtime(gone), spec({ graceMs: 1 }), '/runtime/gone-signal') + const goneHandle = testHandle(runtime(gone), spec({ graceMs: 1 }), '/runtime/gone-signal') await flush() goneHandle.terminate() await expect(goneHandle.done).resolves.toEqual({ exitCode: null, signal: 'SIGKILL' }) const failed = new FakeSandbox() failed.signalError = new Error('signal transport failed') - const failedHandle = new E2BSubprocessHandle(runtime(failed), spec(), '/runtime/failed-signal') + const failedHandle = testHandle(runtime(failed), spec(), '/runtime/failed-signal') await flush() failedHandle.terminate() await expect(failedHandle.done).resolves.toEqual({ exitCode: null, signal: 'SIGKILL' }) @@ -1534,7 +1574,7 @@ describe('E2BSubprocessHandle', () => { const fake = new FakeSandbox() fake.signalErrors.push(new Error('TERM transport failed'), new Error('KILL transport failed')) fake.handle.killError = new Error('SDK kill failed') - const handle = new E2BSubprocessHandle(runtime(fake), spec({ graceMs: 1 }), '/runtime/retry-signal') + const handle = testHandle(runtime(fake), spec({ graceMs: 1 }), '/runtime/retry-signal') await flush() handle.terminate() @@ -1548,7 +1588,7 @@ describe('E2BSubprocessHandle', () => { missingGroup.trapsTerm = true missingGroup.signalErrors.push(undefined, commandError(1)) missingGroup.handle.killError = new Error('SDK kill failed after group exit race') - const raced = new E2BSubprocessHandle(runtime(missingGroup), spec({ graceMs: 1 }), '/runtime/group-exit-race') + const raced = testHandle(runtime(missingGroup), spec({ graceMs: 1 }), '/runtime/group-exit-race') await flush() raced.terminate() await expect(raced.waitForExit()).rejects.toThrow('remained live after force termination') @@ -1562,7 +1602,7 @@ describe('E2BSubprocessHandle', () => { fake.trapsTerm = true fake.sdkKillStops = false fake.signalErrors.push(undefined, new Error('KILL transport failed')) - const handle = new E2BSubprocessHandle(runtime(fake), spec({ graceMs: 1 }), '/runtime/optimistic-sdk-kill') + const handle = testHandle(runtime(fake), spec({ graceMs: 1 }), '/runtime/optimistic-sdk-kill') await flush() handle.terminate() diff --git a/packages/e2b/subprocess-e2b/tests/terminal.spec.ts b/packages/e2b/subprocess-e2b/tests/terminal.spec.ts index 826732f057..5472b2cfa7 100644 --- a/packages/e2b/subprocess-e2b/tests/terminal.spec.ts +++ b/packages/e2b/subprocess-e2b/tests/terminal.spec.ts @@ -259,10 +259,20 @@ function holdRequestUntilAbort(started: PromiseWithResolvers) { } } +/** Spawn the terminal under test with the config default the service would pass. */ +function testSpawn( + runtime: Parameters[0], + spec: Parameters[1], + stateDir: string, + pollMs = 20, +): ReturnType { + return spawnE2BTerminal(runtime, spec, stateDir, pollMs) +} + describe('E2B terminal allocation', () => { it('hides bootstrap-shell bytes and preserves requested-shell bytes across the output boundary', async () => { const fake = new FakeTerminalSandbox() - const terminal = await spawnE2BTerminal(runtime(fake), spec(), '/runtime/terminal-one') + const terminal = await testSpawn(runtime(fake), spec(), '/runtime/terminal-one') let output = '' terminal.output.on('data', (chunk) => { output += String(chunk) }) await new Promise(resolve => setTimeout(resolve, 0)) @@ -314,7 +324,7 @@ describe('E2B terminal allocation', () => { it('inherits only safe ambient values and limits the allocation signal to setup', async () => { const fake = new FakeTerminalSandbox() const controller = new AbortController() - const terminal = await spawnE2BTerminal( + const terminal = await testSpawn( runtime(fake), spec({ env: undefined, signal: controller.signal }), '/runtime/abort-live', @@ -335,7 +345,7 @@ describe('E2B terminal allocation', () => { const fake = new FakeTerminalSandbox() fake.deferCreate() const controller = new AbortController() - const spawning = spawnE2BTerminal( + const spawning = testSpawn( runtime(fake), spec({ signal: controller.signal }), '/runtime/allocation-cancel', @@ -352,23 +362,23 @@ describe('E2B terminal allocation', () => { it('rejects malformed environment and argv values before PTY allocation', async () => { const invalidName = new FakeTerminalSandbox() - await expect(spawnE2BTerminal(runtime(invalidName), spec({ env: { 'BAD=NAME': 'x' } }), '/runtime/name')) + await expect(testSpawn(runtime(invalidName), spec({ env: { 'BAD=NAME': 'x' } }), '/runtime/name')) .rejects.toThrow('environment entries') expect(invalidName.createOptions).toBeUndefined() const invalidValue = new FakeTerminalSandbox() - await expect(spawnE2BTerminal(runtime(invalidValue), spec({ env: { BAD: 'x\0y' } }), '/runtime/value')) + await expect(testSpawn(runtime(invalidValue), spec({ env: { BAD: 'x\0y' } }), '/runtime/value')) .rejects.toThrow('environment entries') const invalidArg = new FakeTerminalSandbox() - await expect(spawnE2BTerminal(runtime(invalidArg), spec({ argv: ['/bin/bash', 'x\0y'] }), '/runtime/argv')) + await expect(testSpawn(runtime(invalidArg), spec({ argv: ['/bin/bash', 'x\0y'] }), '/runtime/argv')) .rejects.toThrow('argv must not contain NUL') }) it('cleans malformed handles, bootstrap failures, and readiness failures', async () => { const failedState = new FakeTerminalSandbox() failedState.writeError = new Error('state write failed') - await expect(spawnE2BTerminal(runtime(failedState), spec(), '/runtime/state-write')) + await expect(testSpawn(runtime(failedState), spec(), '/runtime/state-write')) .rejects.toThrow('state write failed') expect(failedState.writes.get('/runtime/state-write/environment')).toContain('KEEP=visible\0') expect(failedState.removed).toContain('/runtime/state-write') @@ -377,19 +387,19 @@ describe('E2B terminal allocation', () => { const stateAlreadyGone = new FakeTerminalSandbox() stateAlreadyGone.writeError = new Error('state write failed after external cleanup') stateAlreadyGone.removeError = new FileNotFoundError('state already gone') - await expect(spawnE2BTerminal(runtime(stateAlreadyGone), spec(), '/runtime/state-gone')) + await expect(testSpawn(runtime(stateAlreadyGone), spec(), '/runtime/state-gone')) .rejects.toThrow('state write failed after external cleanup') const invalidPid = new FakeTerminalSandbox() invalidPid.handle.pid = 0 - await expect(spawnE2BTerminal(runtime(invalidPid), spec(), '/runtime/invalid-pid')) + await expect(testSpawn(runtime(invalidPid), spec(), '/runtime/invalid-pid')) .rejects.toThrow('invalid terminal pid 0') expect(invalidPid.handle.sdkKills).toBe(1) expect(invalidPid.removed).toContain('/runtime/invalid-pid') const failedInput = new FakeTerminalSandbox() failedInput.sendError = new Error('bootstrap failed') - await expect(spawnE2BTerminal(runtime(failedInput), spec(), '/runtime/input')) + await expect(testSpawn(runtime(failedInput), spec(), '/runtime/input')) .rejects.toThrow('bootstrap failed') expect(failedInput.commands).toContain('kill -TERM -- -123') expect(failedInput.groups).toEqual([]) @@ -397,7 +407,7 @@ describe('E2B terminal allocation', () => { const invalidSession = new FakeTerminalSandbox() invalidSession.sessionId = 'not-a-session\n' invalidSession.clearOnTerm = false - await expect(spawnE2BTerminal(runtime(invalidSession), spec(), '/runtime/session')) + await expect(testSpawn(runtime(invalidSession), spec(), '/runtime/session')) .rejects.toThrow('cannot resolve process session') expect(invalidSession.commands).toContain('kill -TERM -- -123') expect(invalidSession.commands).toContain('kill -KILL -- -123') @@ -410,7 +420,7 @@ describe('E2B terminal allocation', () => { const termFailed = new FakeTerminalSandbox() termFailed.sendError = new Error('bootstrap failed') termFailed.termFailure = new Error('TERM transport failed') - await expect(spawnE2BTerminal(runtime(termFailed), spec(), '/runtime/term-failed')) + await expect(testSpawn(runtime(termFailed), spec(), '/runtime/term-failed')) .rejects.toThrow('bootstrap failed') expect(termFailed.commands).toContain('kill -KILL -- -123') expect(termFailed.handle.sdkKills).toBe(1) @@ -421,7 +431,7 @@ describe('E2B terminal allocation', () => { uninspectable.handle.sdkKillError = new Error('PTY kill failed') let uninspectableFailure: unknown try { - await spawnE2BTerminal(runtime(uninspectable), spec(), '/runtime/uninspectable') + await testSpawn(runtime(uninspectable), spec(), '/runtime/uninspectable') } catch (error: unknown) { uninspectableFailure = error } @@ -432,21 +442,21 @@ describe('E2B terminal allocation', () => { survivingGroups.sendError = new Error('bootstrap failed') survivingGroups.clearOnTerm = false survivingGroups.clearOnKill = false - await expect(spawnE2BTerminal(runtime(survivingGroups), spec({ graceMs: 1 }), '/runtime/surviving-groups')) + await expect(testSpawn(runtime(survivingGroups), spec({ graceMs: 1 }), '/runtime/surviving-groups')) .rejects.toThrow('bootstrap failed') const survivingPid = new FakeTerminalSandbox() survivingPid.sendError = new Error('bootstrap failed') survivingPid.groups = [] survivingPid.handle.settleOnSdkKill = false - await expect(spawnE2BTerminal(runtime(survivingPid), spec({ graceMs: 1 }), '/runtime/surviving-pid')) + await expect(testSpawn(runtime(survivingPid), spec({ graceMs: 1 }), '/runtime/surviving-pid')) .rejects.toThrow('bootstrap failed') const waitFailed = new FakeTerminalSandbox() waitFailed.handle.waitError = new Error('wait failed') waitFailed.handle.settleOnSdkKill = false waitFailed.handle.sdkKillError = new Error('kill failed') - await expect(spawnE2BTerminal(runtime(waitFailed), spec(), '/runtime/wait-failed')) + await expect(testSpawn(runtime(waitFailed), spec(), '/runtime/wait-failed')) .rejects.toThrow('wait failed') expect(waitFailed.handle.sdkKills).toBe(1) @@ -454,7 +464,7 @@ describe('E2B terminal allocation', () => { cleanupFailed.handle.pid = 0 cleanupFailed.handle.sdkKillError = new Error('kill transport failed') cleanupFailed.removeError = new Error('remove transport failed') - await expect(spawnE2BTerminal(runtime(cleanupFailed), spec(), '/runtime/cleanup-failed')) + await expect(testSpawn(runtime(cleanupFailed), spec(), '/runtime/cleanup-failed')) .rejects.toThrow('invalid terminal pid 0') const expiredDuringRollback = new FakeTerminalSandbox() @@ -463,7 +473,7 @@ describe('E2B terminal allocation', () => { expiredDuringRollback.handle.settleOnSdkKill = false expiredDuringRollback.handle.sdkKillError = new SandboxNotFoundError('sandbox expired') expiredDuringRollback.removeError = new SandboxNotFoundError('sandbox expired') - await expect(spawnE2BTerminal(runtime(expiredDuringRollback), spec(), '/runtime/expired-rollback')) + await expect(testSpawn(runtime(expiredDuringRollback), spec(), '/runtime/expired-rollback')) .rejects.toThrow('bootstrap failed before timeout') expect(expiredDuringRollback.handle.sdkKills).toBe(1) @@ -471,30 +481,30 @@ describe('E2B terminal allocation', () => { expiredBeforeSdkRollback.handle.waitError = new Error('wait failed after timeout') expiredBeforeSdkRollback.handle.sdkKillError = new SandboxNotFoundError('sandbox expired') expiredBeforeSdkRollback.handle.settleOnSdkKill = false - await expect(spawnE2BTerminal(runtime(expiredBeforeSdkRollback), spec(), '/runtime/expired-sdk-rollback')) + await expect(testSpawn(runtime(expiredBeforeSdkRollback), spec(), '/runtime/expired-sdk-rollback')) .rejects.toThrow('wait failed after timeout') const missingDuringDisconnect = new FakeTerminalSandbox() missingDuringDisconnect.sendError = new Error('bootstrap failed before disconnect') missingDuringDisconnect.handle.disconnectError = new SandboxNotFoundError('sandbox expired') - await expect(spawnE2BTerminal(runtime(missingDuringDisconnect), spec(), '/runtime/missing-disconnect')) + await expect(testSpawn(runtime(missingDuringDisconnect), spec(), '/runtime/missing-disconnect')) .rejects.toThrow('bootstrap failed before disconnect') const failedDisconnect = new FakeTerminalSandbox() failedDisconnect.sendError = new Error('bootstrap failed with disconnect failure') failedDisconnect.handle.disconnectError = new Error('disconnect transport failed') - await expect(spawnE2BTerminal(runtime(failedDisconnect), spec(), '/runtime/failed-disconnect')) + await expect(testSpawn(runtime(failedDisconnect), spec(), '/runtime/failed-disconnect')) .rejects.toThrow('bootstrap failed with disconnect failure') }) it('propagates setup cancellation and provider failures', async () => { const aborted = new FakeTerminalSandbox() - await expect(spawnE2BTerminal(runtime(aborted), spec({ signal: AbortSignal.abort(new Error('stop')) }), '/runtime/abort')) + await expect(testSpawn(runtime(aborted), spec({ signal: AbortSignal.abort(new Error('stop')) }), '/runtime/abort')) .rejects.toThrow('stop') const createFailed = new FakeTerminalSandbox() createFailed.createError = new Error('create failed') - await expect(spawnE2BTerminal(runtime(createFailed), spec(), '/runtime/create')) + await expect(testSpawn(runtime(createFailed), spec(), '/runtime/create')) .rejects.toThrow('create failed') }) @@ -502,7 +512,7 @@ describe('E2B terminal allocation', () => { it('bounds a missing bootstrap-output boundary by process exit or cancellation', async () => { const exited = new FakeTerminalSandbox() exited.emitOutputMarker = false - const exiting = spawnE2BTerminal(runtime(exited), spec(), '/runtime/missing-output-boundary') + const exiting = testSpawn(runtime(exited), spec(), '/runtime/missing-output-boundary') await vi.waitFor(() => { expect(exited.inputs).toHaveLength(1) }) exited.handle.succeed(0) await expect(exiting).rejects.toThrow('terminal exited before publishing its output boundary') @@ -510,7 +520,7 @@ describe('E2B terminal allocation', () => { const cancelled = new FakeTerminalSandbox() cancelled.emitOutputMarker = false const controller = new AbortController() - const cancelling = spawnE2BTerminal( + const cancelling = testSpawn( runtime(cancelled), spec({ signal: controller.signal }), '/runtime/cancel-output-boundary', @@ -525,7 +535,7 @@ describe('E2B terminal allocation', () => { describe('E2B terminal lifecycle', () => { it('aborts and joins in-flight terminal operations before cleanup', async () => { const fake = new FakeTerminalSandbox() - const terminal = await spawnE2BTerminal(runtime(fake), spec(), '/runtime/in-flight-operations') + const terminal = await testSpawn(runtime(fake), spec(), '/runtime/in-flight-operations') const writeStarted = Promise.withResolvers() const inspectStarted = Promise.withResolvers() const signalStarted = Promise.withResolvers() @@ -563,7 +573,7 @@ describe('E2B terminal lifecycle', () => { it('maps ordinary exits, closes output, and reports an absent foreground after exit', async () => { const fake = new FakeTerminalSandbox() fake.groups = [] - const terminal = await spawnE2BTerminal(runtime(fake), spec(), '/runtime/natural') + const terminal = await testSpawn(runtime(fake), spec(), '/runtime/natural') terminal.output.resume() const ended = once(terminal.output, 'end') fake.handle.succeed(7) @@ -583,7 +593,7 @@ describe('E2B terminal lifecycle', () => { ] as const)('classifies an unrequested command exit %i', async (exitCode, expected) => { const fake = new FakeTerminalSandbox() fake.groups = [] - const terminal = await spawnE2BTerminal(runtime(fake), spec(), `/runtime/exit-${exitCode}`) + const terminal = await testSpawn(runtime(fake), spec(), `/runtime/exit-${exitCode}`) fake.handle.fail(exitCode) await expect(terminal.done).resolves.toEqual(expected) await terminal.terminate() @@ -593,7 +603,7 @@ describe('E2B terminal lifecycle', () => { const fake = new FakeTerminalSandbox() fake.groups = [] fake.zombieGroups = [123] - const terminal = await spawnE2BTerminal(runtime(fake), spec(), '/runtime/zombie-session') + const terminal = await testSpawn(runtime(fake), spec(), '/runtime/zombie-session') fake.handle.succeed(0) await expect(terminal.done).resolves.toEqual({ exitCode: 0, signal: null }) @@ -605,7 +615,7 @@ describe('E2B terminal lifecycle', () => { it('treats a timeout-killed sandbox as quiescent during terminal cleanup', async () => { const fake = new FakeTerminalSandbox() - const terminal = await spawnE2BTerminal(runtime(fake), spec(), '/runtime/expired-sandbox') + const terminal = await testSpawn(runtime(fake), spec(), '/runtime/expired-sandbox') fake.sessionGroupsFailure = new SandboxNotFoundError('sandbox expired') fake.handle.succeed(0) @@ -618,7 +628,7 @@ describe('E2B terminal lifecycle', () => { fake.groups = [] fake.handle.settleOnSdkKill = false fake.handle.sdkKillError = new SandboxNotFoundError('sandbox expired') - const terminal = await spawnE2BTerminal(runtime(fake), spec({ graceMs: 1 }), '/runtime/expired-pty-kill') + const terminal = await testSpawn(runtime(fake), spec({ graceMs: 1 }), '/runtime/expired-pty-kill') await terminal.terminate() expect(fake.handle.sdkKills).toBe(1) @@ -629,7 +639,7 @@ describe('E2B terminal lifecycle', () => { fake.groups = [] fake.handle.settleOnSdkKill = false fake.handle.sdkKillError = new Error('PTY kill transport failed') - const terminal = await spawnE2BTerminal(runtime(fake), spec({ graceMs: 1 }), '/runtime/failed-pty-kill') + const terminal = await testSpawn(runtime(fake), spec({ graceMs: 1 }), '/runtime/failed-pty-kill') await expect(terminal.terminate()).rejects.toThrow('PTY kill transport failed') fake.handle.sdkKillError = undefined @@ -643,7 +653,7 @@ describe('E2B terminal lifecycle', () => { ['propagates another failure', new Error('disconnect failed'), false], ] as const)('%s while disconnecting a settled terminal', async (_label, failure, accepted) => { const fake = new FakeTerminalSandbox() - const terminal = await spawnE2BTerminal(runtime(fake), spec(), `/runtime/disconnect-${accepted}`) + const terminal = await testSpawn(runtime(fake), spec(), `/runtime/disconnect-${accepted}`) fake.handle.disconnectError = failure fake.groups = [] fake.handle.succeed(0) @@ -655,7 +665,7 @@ describe('E2B terminal lifecycle', () => { it('rejects killing the terminal shell and propagates live foreground failures', async () => { const fake = new FakeTerminalSandbox() fake.foreground = '123\n' - const terminal = await spawnE2BTerminal(runtime(fake), spec(), '/runtime/signal') + const terminal = await testSpawn(runtime(fake), spec(), '/runtime/signal') await expect(terminal.signalForeground('SIGKILL')).rejects.toThrow('refusing to SIGKILL') fake.foreground = 'invalid\n' await expect(terminal.inspectForeground()).rejects.toThrow('cannot resolve foreground') @@ -671,7 +681,7 @@ describe('E2B terminal lifecycle', () => { const fake = new FakeTerminalSandbox() fake.groups = [123, 456] fake.clearOnTerm = false - const terminal = await spawnE2BTerminal(runtime(fake), spec({ graceMs: 0 }), '/runtime/escalate') + const terminal = await testSpawn(runtime(fake), spec({ graceMs: 0 }), '/runtime/escalate') const terminating = terminal.terminate() await expect(terminal.done).resolves.toEqual({ exitCode: null, signal: 'SIGKILL' }) await terminating @@ -682,7 +692,7 @@ describe('E2B terminal lifecycle', () => { it('surfaces cleanup failures and allows a later retry', async () => { const fake = new FakeTerminalSandbox() fake.groups = [1] - const terminal = await spawnE2BTerminal(runtime(fake), spec({ graceMs: 1 }), '/runtime/retry') + const terminal = await testSpawn(runtime(fake), spec({ graceMs: 1 }), '/runtime/retry') await expect(terminal.terminate()).rejects.toThrow('unsafe process group 1') fake.groups = [] @@ -694,7 +704,7 @@ describe('E2B terminal lifecycle', () => { it('propagates a process-group signalling transport failure before retry', async () => { const fake = new FakeTerminalSandbox() fake.termFailure = new Error('signal transport failed') - const terminal = await spawnE2BTerminal(runtime(fake), spec({ graceMs: 1 }), '/runtime/signal-failure') + const terminal = await testSpawn(runtime(fake), spec({ graceMs: 1 }), '/runtime/signal-failure') await expect(terminal.terminate()).rejects.toThrow('signal transport failed') fake.groups = [] @@ -704,7 +714,7 @@ describe('E2B terminal lifecycle', () => { const alreadyExited = new FakeTerminalSandbox() alreadyExited.termFailure = commandError(1) - const tolerant = await spawnE2BTerminal(runtime(alreadyExited), spec({ graceMs: 1 }), '/runtime/group-exited') + const tolerant = await testSpawn(runtime(alreadyExited), spec({ graceMs: 1 }), '/runtime/group-exited') const tolerantTermination = tolerant.terminate() await expect(tolerant.done).resolves.toEqual({ exitCode: null, signal: 'SIGKILL' }) await tolerantTermination @@ -714,7 +724,7 @@ describe('E2B terminal lifecycle', () => { const fake = new FakeTerminalSandbox() fake.groups = [] fake.removeError = new Error('private state already gone') - const terminal = await spawnE2BTerminal(runtime(fake), spec(), '/runtime/reject-during-cleanup') + const terminal = await testSpawn(runtime(fake), spec(), '/runtime/reject-during-cleanup') terminal.output.on('error', () => {}) const cleanup = terminal.terminate() await Promise.resolve() @@ -727,7 +737,7 @@ describe('E2B terminal lifecycle', () => { const fake = new FakeTerminalSandbox() fake.groups = [] fake.handle.settleOnSdkKill = false - const terminal = await spawnE2BTerminal(runtime(fake), spec({ graceMs: 1 }), '/runtime/reject-after-kill') + const terminal = await testSpawn(runtime(fake), spec({ graceMs: 1 }), '/runtime/reject-after-kill') terminal.output.on('error', () => {}) const cleanup = terminal.terminate() while (fake.handle.sdkKills === 0) await new Promise(resolve => setTimeout(resolve, 0)) @@ -741,20 +751,20 @@ describe('E2B terminal lifecycle', () => { const survivor = new FakeTerminalSandbox() survivor.clearOnTerm = false survivor.clearOnKill = false - const terminal = await spawnE2BTerminal(runtime(survivor), spec({ graceMs: 1 }), '/runtime/survivor') + const terminal = await testSpawn(runtime(survivor), spec({ graceMs: 1 }), '/runtime/survivor') await expect(terminal.terminate()).rejects.toThrow('surviving process groups: 123') const livePid = new FakeTerminalSandbox() livePid.groups = [] livePid.handle.settleOnSdkKill = false - const live = await spawnE2BTerminal(runtime(livePid), spec({ graceMs: 1 }), '/runtime/live-pid') + const live = await testSpawn(runtime(livePid), spec({ graceMs: 1 }), '/runtime/live-pid') await expect(live.terminate()).rejects.toThrow('surviving pid: 123') livePid.handle.succeed(0) await live.done const crashed = new FakeTerminalSandbox() crashed.groups = [] - const failed = await spawnE2BTerminal(runtime(crashed), spec(), '/runtime/crashed') + const failed = await testSpawn(runtime(crashed), spec(), '/runtime/crashed') const outputError = once(failed.output, 'error') crashed.handle.crash('transport gone') await expect(failed.done).rejects.toEqual('transport gone') diff --git a/packages/e2b/subprocess-e2b/tsconfig.json b/packages/e2b/subprocess-e2b/tsconfig.json index fda7cfc667..1bae0ef012 100644 --- a/packages/e2b/subprocess-e2b/tsconfig.json +++ b/packages/e2b/subprocess-e2b/tsconfig.json @@ -4,22 +4,27 @@ "rootDir": "src", "outDir": "lib/types" }, - "include": ["src"], + "include": [ + "src" + ], "references": [ - { - "path": "../../../vendor/cosmokit" - }, { "path": "../../../vendor/cordis" }, { - "path": "../e2b" + "path": "../../../vendor/cosmokit" }, { "path": "../../subprocess/subprocess" }, { "path": "../../support/invariants" + }, + { + "path": "../../util/timeout" + }, + { + "path": "../e2b" } ] } diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index d2e7d0ec1c..fab00ba9cb 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -3190,6 +3190,9 @@ importers: '@deepseek-ai/dsh-subprocess': specifier: workspace:^ version: link:../../subprocess/subprocess + '@deepseek-ai/dsh-timeout': + specifier: workspace:^ + version: link:../../util/timeout cordis: specifier: ^4.0.0-rc.7 version: link:../../../vendor/cordis