diff --git a/.agents/notes/implemented/feature/2026-06-15-code-mode.i18n.yaml b/.agents/notes/implemented/feature/2026-06-15-code-mode.i18n.yaml index 04a4c91430..bc05e497c5 100644 --- a/.agents/notes/implemented/feature/2026-06-15-code-mode.i18n.yaml +++ b/.agents/notes/implemented/feature/2026-06-15-code-mode.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-06-15-code-mode.md -2026-06-15-code-mode.md: b6a24ecd9700e32912b8112b59cbd8b6ab131eb5 -2026-06-15-code-mode.zh.md: a00a43ece1e581190de6096be8138df25a23f07f +2026-06-15-code-mode.md: 99bbed3edab32512f88ece9694d6519a1f89c2dd +2026-06-15-code-mode.zh.md: ca1bbe9ed3e412186763d1ed4fca9ed06669d4c3 diff --git a/.agents/notes/implemented/feature/2026-06-15-code-mode.md b/.agents/notes/implemented/feature/2026-06-15-code-mode.md index b6a24ecd97..99bbed3eda 100644 --- a/.agents/notes/implemented/feature/2026-06-15-code-mode.md +++ b/.agents/notes/implemented/feature/2026-06-15-code-mode.md @@ -6,7 +6,7 @@ English | [中文](2026-06-15-code-mode.zh.md) ## Problem -In the registry's native presentation, the agent loop advertises every visible capability as a JSON-schema function definition. `ToolRegistry` contributes its schemas to the system-prompt assembly, the assembly's `tools` land on the wire (and in the logged request header), the model invokes one `tool-call` block per step, and the loop dispatches each call through `ctx.tools.execute()` **sequentially** (parallel tool execution is an explicit open TODO in `dsh-tools` and [docs/architecture.md](../../../../docs/architecture.md)), with **every** intermediate `tool-result` re-entering the model's context on the next request. +In the registry's native presentation, the agent loop advertises every visible capability as a JSON-schema function definition. `ToolRegistry` contributes its schemas to the system-prompt assembly, the assembly's `tools` land on the wire (and in the logged request header), the model invokes one `tool-call` block per step, and at the time of this note the loop dispatched each call through `ctx.tools.execute()` **sequentially** (parallel tool execution was an open TODO then; bounded parallel dispatch has since shipped — the [parallel tool-call note](2026-07-10-parallel-tool-call-execution.md), the rolling pool in [docs/architecture.md](../../../../docs/architecture.md)) — with **every** intermediate `tool-result` re-entering the model's context on the next request. For multi-step tool work this is token-heavy and serial. The model cannot compose tools — loop over a result set, branch on an intermediate value, fan out, post-process — without a full model round-trip per call, and each round-trip drags the entire intermediate result back into context whether the model needs it or not. @@ -32,7 +32,7 @@ This note owns Code Mode's presentation, composition, isolation, and settlement **Interaction with `toolOrder`, stated up front:** a configured `systemPrompt.toolOrder` naming native capabilities rejects every assembly under `mode: 'code'`, because those names are outside that mode's wire-validation universe. This is correct behavior, not a bug: a deployment using Code Mode updates its order config or drops it. -**SDK prompt section.** In `'code'` and `'both'`, the lazy `tools:sdk` section in the tool-guidance order band renders TypeScript declarations plus fixed usage instructions for the scope's visible capabilities. It shares lookup and execution visibility, excludes `run_code`, and sorts tools lexicographically for byte-stable output. +**SDK prompt section.** In `'code'` and `'both'`, the lazy `tools:sdk` section in the tool-guidance order band renders the loaded runtime's language declarations plus fixed usage instructions for the scope's visible capabilities (TypeScript by default; the [language-dispatch note](2026-07-31-code-mode-language-dispatch.md) added Python and the `ctx.codeRuntime.language` renderer table). It shares lookup and execution visibility, excludes `run_code`, and sorts tools lexicographically for byte-stable output. **Assembly ownership.** `run_code` and `tools:sdk` enter the trusted `system-prompt/assemble` waterfall as normal assembly inputs. A scoped `tools:sdk` section may shadow the global default before dispatch, and a listener may remove or replace either contribution. The waterfall's returned assembly is final, so whoever changes these inputs owns preserving a viable Code Mode protocol when the deployment expects Code Mode to remain usable; no restoration pass overrides deliberate composition. @@ -48,7 +48,7 @@ Under `'code'` and `'both'` the registry owns `run_code` as a reserved presentat **Sub-call contexts are deferred through the parent.** Injecting inside `run_code` would break parent call/result adjacency, so `ToolRunContext.deferContext()` collects every sub-result `additionalContexts` entry in dispatch order. The registry carries that array even when the program later throws, and the loop appends each entry only after the outer result and every sibling result in the step. An outer post-execute block discards tool-deferred entries and exposes only contexts explicitly attached by the blocking decision. -**Concurrency is serialized.** Each run owns a dispatch queue, so even `Promise.all` executes tool calls in submission order. Settlement abandons queued calls that have not started. Parallelism requires per-tool concurrency-safety metadata. +**Concurrency is bounded, not serialized.** Each run owns a dispatch queue that starts calls strictly in submission order and classifies each one through `registry.executionMode`, the same fail-closed `isConcurrencySafe` contract the native loop uses. Consecutive parallel-classified calls overlap up to `maxParallelSubCalls` (default 10; `1` restores serial dispatch); an exclusive call drains the pool and runs alone. Settlement abandons queued calls that have not started. This note shipped the serialized placeholder; the [live-parallel Agent Note](2026-07-26-code-mode-live-parallel-dispatch.md) owns the scheduler that replaced it. **Presentation.** `run_code`'s render intent is decided here per the [render-intent Agent Note](../architecture/2026-07-02-tool-render-intent-union.md): `presentCall` creates a `generic` card with `kind: 'execute'`, the program text as its title, and the same program text as `rawInput`; `run_code` intentionally declares no `presentResult`, so the TUI and host/client runtime (Web) complete that card through their generic raw-content fallback using the final durable `tool/result.content`, including captured logs plus the returned value, failure, or post-policy spill preview. This is not a `terminal` card: that card's semantics are "a shell command in a working directory", which a program is not. See the [result-card completeness note](../../archived/bug-fix/2026-07-20-code-mode-result-card-completeness.md). @@ -64,7 +64,7 @@ Each sub-dispatch appends a log-only `tool/code-dispatch-start` event at pool en - `CodeBindingNamespace = { global: string; functions: Record Promise>; errorClass?: { name: string; memberNameProperty: string } }` — the runtime exposes each namespace as a global object of async functions inside the program; the optional descriptor asks the runtime to inject a real program-visible rejection class without teaching the seam consumer-specific names. `CodeJsonValue` is this dependency-light seam's structural lossless-JSON type, so binding arguments and resolutions cross the implementation's serialization boundary whole. - `CodeRunResult = { value?: CodeJsonValue; logs: string[]; error?: CodeRunFailure }` — program execution outcomes resolve as the `error` field. `run()` may reject only for caller/seam misuse (for example a duplicate binding namespace); consumers still contain a non-conforming backend rejection at their own error boundary. - `CodeRunFailure = { kind: 'exception' | 'timeout' | 'abort' | 'worker-exit' | 'invalid-output' | 'output-limit'; message: string }` — orthogonal outcomes reported independently per [defensive patterns](../../../../docs/defensive-patterns.md); a timed-out run is not an exception, an abort is not a timeout, a lossy completion is not an overflow, and a substrate exit is none of them. -- Two readonly backend descriptors, informational not gating: `language` (what the program must be written in — `'typescript'` for the shipped backend; a Python backend would say so, and pair with its own SDK generator on the presentation side) and `isolation` (`'worker-thread'` for the shipped backend; `'process'`, `'container'`, … for future ones). `dsh-tools` requires `language === 'typescript'` in the MVP — its codegen emits TS — and fails the assembly loudly otherwise, the same misconfiguration idiom as `toolOrder` violations (as when `mode` is non-native with no `ctx.codeRuntime` loaded at all). +- Two readonly backend descriptors, informational not gating: `language` (what the program must be written in — `'typescript'` for the first backend; a Python backend says `'python'` and pairs with its own SDK generator on the presentation side) and `isolation` (`'worker-thread'` for the shipped backend; `'process'`, `'container'`, … for future ones). `dsh-tools` accepts any `language` with a registered SDK renderer and `run_code` flavor (TypeScript and Python ship; see the [language-dispatch note](2026-07-31-code-mode-language-dispatch.md)) and fails the assembly loudly otherwise, the same misconfiguration idiom as `toolOrder` violations (as when `mode` is non-native with no `ctx.codeRuntime` loaded at all). Requests contain every runtime input; implementations own validated timeout and cap defaults. The registry looks up the optional runtime only when Code Mode is assembled, so native mode does not depend on one. Missing or language-incompatible runtimes fail loudly. Alternate substrates or languages can replace the implementation behind the same seam, paired with the appropriate SDK generator. @@ -85,11 +85,11 @@ The worker runtime provides containment, not a security boundary: model code can ### What the model sees -The SDK instructs the model to write an async erasable-TypeScript body, call tools through `await tools.name(args)`, catch rejected tool calls when needed, and return or log only the output that should re-enter context. Calls remain sequential even under `Promise.all`. The declaration prefix can be as large as native schemas, especially in `'both'`, but remains stable for provider caching. +The SDK instructs the model to write an async body in the loaded runtime's language (an erasable-TypeScript body by default; a Python `async` body under a Python runtime — see the [language-dispatch note](2026-07-31-code-mode-language-dispatch.md)), call tools through `await tools.name(args)`, catch rejected tool calls when needed, and return or log only the output that should re-enter context. Both flavors state the same contract in their own primitive: independent read-only calls MAY overlap under `Promise.all` (TypeScript) or `asyncio.gather` (Python), mutating calls run alone in submission order, and dependent work sequences with `await`. The declaration prefix can be as large as native schemas, especially in `'both'`, but remains stable for provider caching. ## Consequences -Deployments switching to `'code'` must update any native-only `toolOrder`. Assembly listeners own the integrity of any rewritten protocol surface. Sub-dispatch remains serialized, while per-call contexts retain their source, envelope, and metadata through the outer result. +Deployments switching to `'code'` must update any native-only `toolOrder`. Assembly listeners own the integrity of any rewritten protocol surface. Sub-dispatch starts in submission order under a bounded overlap pool, while per-call contexts retain their source, envelope, and metadata through the outer result. ## Testing @@ -106,7 +106,7 @@ Deployments switching to `'code'` must update any native-only `toolOrder`. Assem **Result elision / summarization over native tool-calling.** Addresses only the context-bloat half of the problem: trimming old `tool-result`s is cheap to add as a logged surface replacement under reconstructable requests, but still pays one model round-trip per call and cannot express loops, branches, or joins. Complementary, not competing; it can layer under Code Mode for residual native calls. -**Parallel native dispatch in the loop.** The other answer to round-trip cost; still valid future work (the open TODO), still blocked on concurrency-safety metadata, and still no composition — it parallelizes calls the model already decided on in one step. Code Mode's serialized-queue decision keeps the two compatible: when the metadata lands, both native parallel dispatch and per-tool binding parallelism unlock together. +**Parallel native dispatch in the loop.** The other answer to round-trip cost at decision time; it was blocked on concurrency-safety metadata and offers no composition either way — it parallelizes calls the model already decided on in one step. Code Mode's queue decision kept the two compatible, and that is how it played out: the metadata landed as `isConcurrencySafe` (the [parallel tool-call note](2026-07-10-parallel-tool-call-execution.md)), and native rolling-pool dispatch and per-tool binding parallelism unlocked on the same classifier. **Always-exclusive (Cloudflare-faithful, no mode).** Rejected for this SDK's primary consumer: a coding agent's bread-and-butter single calls (`bash`, `read`, `edit`) are already ideal as native calls, and forcing every edit through a program taxes the common case. The mode config keeps the faithful form (`'code'`) one line away without imposing it. @@ -128,6 +128,6 @@ Deployments switching to `'code'` must update any native-only `toolOrder`. Assem **Large lossless JSON values can exhaust memory.** Tool bindings snapshot lossless JSON before dispatch and return canonical JSON resolutions whole. The runtime validates both sides of the worker port and applies no per-binding byte cap; structured-clone cost and process or worker memory are the practical bounds. The combined outer-output ledger for logs, the completion value, and a failure diagnostic is the only byte-capped boundary. -**Serialized-only sub-dispatch.** `Promise.all` gains no wall-clock parallelism yet, only fewer round-trips; models may over-expect. The instructions state it; lifting it is tied to the same concurrency-safety metadata the native parallel-dispatch TODO needs. +**Sub-dispatch overlap is bounded by tool safety claims, not by the caller.** A program's `Promise.all` or `asyncio.gather` buys wall-clock parallelism only across calls the tool itself classifies concurrency-safe; a run of exclusive calls still costs its round-trips in sequence, and models may over-expect. Both flavors' SDK instructions state the real contract. This note shipped the serialized placeholder that made the risk absolute; the [live-parallel Agent Note](2026-07-26-code-mode-live-parallel-dispatch.md) owns the scheduler and its overlap cap. **Budget metering reads the event loop, not a flag.** Busy-time polling (`eventLoopUtilization()`) is coarser than an exact CPU meter — a budget expires up to one poll interval late — and its correctness claim ("a pending dispatch cannot pause it") is load-bearing against a hostile program. Both sides are unit-tested (hot loop with a pending decoy dispatch dies at `computeMs`; idle-on-slow-binding survives to `maxWallMs`), and the poll interval is an internal constant, not config — nothing a deployment could mis-tune into a bypass. `maxWallMs` is config, and it reaches `setTimeout`, which clamps a delay above `MAX_TIMER_DELAY_MS` (2^31-1 ms) to 1 ms; a positivity check alone therefore accepts a 25-day ceiling that expires on the first tick and times out every run. The worker runtime range-checks the field at load for that reason. `computeMs` needs no upper bound because it is compared against measured utilization instead of being handed to a timer. diff --git a/.agents/notes/implemented/feature/2026-06-15-code-mode.zh.md b/.agents/notes/implemented/feature/2026-06-15-code-mode.zh.md index a00a43ece1..ca1bbe9ed3 100644 --- a/.agents/notes/implemented/feature/2026-06-15-code-mode.zh.md +++ b/.agents/notes/implemented/feature/2026-06-15-code-mode.zh.md @@ -6,7 +6,7 @@ Status: implemented ## 问题 -在注册表的原生呈现方式下,agent loop(智能体循环)将每个可见能力以 JSON Schema 函数定义的形式通告给模型。`ToolRegistry` 将其 schema 贡献给系统提示词组装,组装结果中的 `tools` 落到协议格式(wire format)上(也记录在请求头日志中),模型每步调用一个 `tool-call` 块,循环通过 `ctx.tools.execute()` **逐个**分发每次调用(并行工具执行是 `dsh-tools` 和 [docs/architecture.md](../../../../docs/architecture.md) 中明确标注的 open TODO),且**每一个**中间 `tool-result` 都会在下一次请求时重新进入模型上下文。 +在注册表的原生呈现方式下,agent loop(智能体循环)将每个可见能力以 JSON Schema 函数定义的形式通告给模型。`ToolRegistry` 将其 schema 贡献给系统提示词组装,组装结果中的 `tools` 落到协议格式(wire format)上(也记录在请求头日志中),模型每步调用一个 `tool-call` 块,而在本 note 写作时,循环通过 `ctx.tools.execute()` **逐个**分发每次调用(并行工具执行当时还是 open TODO;此后有界的并行分发已经交付——见[并行工具调用 note](2026-07-10-parallel-tool-call-execution.md),以及 [docs/architecture.md](../../../../docs/architecture.md) 中的 rolling pool)——且**每一个**中间 `tool-result` 都会在下一次请求时重新进入模型上下文。 对于多步工具操作,这种方式 token 开销大且串行。模型无法组合工具——遍历结果集、根据中间值分支、扇出、后处理——每次调用都需要一次完整的模型往返,而每次往返都会把完整的中间结果拖回上下文,不管模型是否需要。 @@ -32,7 +32,7 @@ Cloudflare 的 [Code Mode](https://blog.cloudflare.com/code-mode/) 提出了一 **与 `toolOrder` 的交互,预先说明:** 如果配置的 `systemPrompt.toolOrder` 引用了原生能力名称,在 `mode: 'code'` 下会拒绝所有组装,因为那些名称不在该模式的协议校验范围内。这是正确行为而非 bug:使用 Code Mode 的部署需要更新其 order 配置或移除它。 -**SDK 提示词段。** 在 `'code'` 和 `'both'` 下,tool-guidance order band 中的惰性 `tools:sdk` 段为当前 scope 的可见能力渲染 TypeScript 声明加固定的使用说明。它共享查找和执行可见性,排除 `run_code`,并按字典序排列工具以获得字节稳定的输出。 +**SDK 提示词段。** 在 `'code'` 和 `'both'` 下,tool-guidance order band 中的惰性 `tools:sdk` 段为当前 scope 的可见能力渲染所加载运行时语言的声明加固定的使用说明(默认 TypeScript;[语言分发 note](2026-07-31-code-mode-language-dispatch.md) 加入了 Python 与按 `ctx.codeRuntime.language` 选择的渲染器表)。它共享查找和执行可见性,排除 `run_code`,并按字典序排列工具以获得字节稳定的输出。 **组装所有权。** `run_code` 和 `tools:sdk` 作为正常的组装输入进入受信任的 `system-prompt/assemble` waterfall。一个 scoped 的 `tools:sdk` 段可以在分发前遮蔽全局默认值,监听器也可以移除或替换任一贡献。waterfall 返回的组装结果是最终的,因此修改这些输入的人有责任在部署期望 Code Mode 可用时保持协议面的完整性;没有恢复 pass 会覆盖有意的组合。 @@ -48,7 +48,7 @@ Cloudflare 的 [Code Mode](https://blog.cloudflare.com/code-mode/) 提出了一 **子调用上下文通过父调用延后。** 在 `run_code` 内部注入会破坏父调用/结果的相邻性,因此 `ToolRunContext.deferContext()` 按分发顺序收集每个子结果的 `additionalContexts` 条目。即使程序后来抛出异常,注册表仍携带该数组;循环只在外层结果与步骤中所有兄弟结果之后追加每个条目。外层 post-execute 阻止会丢弃工具延后的条目,只暴露阻止 decision 显式附加的上下文。 -**并发被序列化。** 每次 run 拥有一个分发队列,因此即使 `Promise.all` 也按提交顺序执行工具调用。结算时放弃尚未开始的排队调用。并行化需要每个工具的并发安全元数据。 +**并发是有界的,而非被序列化。** 每次 run 拥有一个分发队列,严格按提交顺序启动调用,并通过 `registry.executionMode` 对每个调用分类——与原生循环所用的 fail-closed `isConcurrencySafe` 契约相同。连续的 parallel 类调用最多重叠 `maxParallelSubCalls` 个(默认 10;设为 `1` 恢复串行分发);exclusive 类调用会排空池并单独运行。结算时放弃尚未开始的排队调用。本 note 交付的是被序列化的占位实现;取代它的调度器由[实时并行 Agent Note](2026-07-26-code-mode-live-parallel-dispatch.md) 负责。 **呈现。** `run_code` 的 render intent 按[呈现意图 Agent Note](../architecture/2026-07-02-tool-render-intent-union.md)在此决定:`presentCall` 创建一个 `generic` 卡片,`kind: 'execute'`,以程序文本作为标题,并将同一程序文本作为 `rawInput`;`run_code` 有意不声明 `presentResult`,因此 TUI 和宿主/客户端运行时(Web)会通过通用原始内容回退机制,使用最终持久化的 `tool/result.content` 补全该卡片,其中包括捕获的日志,以及返回值、失败信息或 post-policy 输出落盘预览。这不是 `terminal` 卡片:该卡片的语义是「工作目录中的 shell 命令」,程序不是。参见[结果卡片完整性说明](../../archived/bug-fix/2026-07-20-code-mode-result-card-completeness.md)。 @@ -64,7 +64,7 @@ Cloudflare 的 [Code Mode](https://blog.cloudflare.com/code-mode/) 提出了一 - `CodeBindingNamespace = { global: string; functions: Record Promise>; errorClass?: { name: string; memberNameProperty: string } }`——运行时将每个命名空间作为程序内部的全局异步函数对象暴露;可选描述符要求运行时注入真正的、程序可见的 reject 类,而无需让 seam 获知消费方专用名称。`CodeJsonValue` 是这个低依赖 seam 的结构化无损 JSON 类型,因此绑定参数与返回值可以完整跨越实现的序列化边界。 - `CodeRunResult = { value?: CodeJsonValue; logs: string[]; error?: CodeRunFailure }`——程序执行失败时,执行 promise 仍会 fulfill,并通过 `error` 字段返回失败结果。只有调用方/seam 误用(例如重复的绑定命名空间)时,`run()` 才会 reject;消费方仍在自己的错误边界处理不合规后端的拒绝。 - `CodeRunFailure = { kind: 'exception' | 'timeout' | 'abort' | 'worker-exit' | 'invalid-output' | 'output-limit'; message: string }`——按[防御性模式](../../../../docs/defensive-patterns.md)独立报告的正交结果;超时的 run 不是异常,abort 不是超时,有损完成值不是溢出,基底退出也与上述情况相互独立。 -- 两个只读的后端描述符,仅供信息参考而非门禁判定:`language`(程序必须使用的语言——交付的后端为 `'typescript'`;Python 后端会声明自己,并在呈现侧配对自己的 SDK 生成器)和 `isolation`(交付的后端为 `'worker-thread'`;未来可为 `'process'`、`'container'` 等)。`dsh-tools` 在 MVP 中要求 `language === 'typescript'`——其代码生成输出 TS——否则组装会大声失败,与 `toolOrder` 违规时的配置错误惯用法相同(如 `mode` 为非 native 但根本没有加载 `ctx.codeRuntime`)。 +- 两个只读的后端描述符,仅供信息参考而非门禁判定:`language`(程序必须使用的语言——首个后端为 `'typescript'`;Python 后端声明 `'python'`,并在呈现侧配对自己的 SDK 生成器)和 `isolation`(交付的后端为 `'worker-thread'`;未来可为 `'process'`、`'container'` 等)。`dsh-tools` 接受任何注册了 SDK 渲染器与 `run_code` flavor 的 `language`(TypeScript 与 Python 已交付;见[语言分发 note](2026-07-31-code-mode-language-dispatch.md)),否则组装会大声失败,与 `toolOrder` 违规时的配置错误惯用法相同(如 `mode` 为非 native 但根本没有加载 `ctx.codeRuntime`)。 请求包含所有运行时输入;实现方拥有经校验的超时和上限默认值。注册表仅在组装 Code Mode 时查找可选的运行时,因此 native 模式不依赖它。缺失或语言不兼容的运行时会大声失败。替代基底或语言可以在同一 seam 背后替换实现,配对相应的 SDK 生成器。 @@ -85,11 +85,11 @@ worker 运行时只能约束程序的运行,而不构成安全边界:模型 ### 模型看到的内容 -SDK 指示模型编写一个异步的可擦除 TypeScript 函数体,通过 `await tools.name(args)` 调用工具,在需要时捕获被拒绝的工具调用,并仅 return 或 log 应重新进入上下文的输出。即使在 `Promise.all` 下调用仍保持顺序。声明前缀可能与原生 schema 一样大,尤其在 `'both'` 下,但对提供方缓存保持稳定。 +SDK 指示模型编写一个所加载运行时语言的异步函数体(默认可擦除 TypeScript;Python 运行时下为 Python `async` 函数体——见[语言分发 note](2026-07-31-code-mode-language-dispatch.md)),通过 `await tools.name(args)` 调用工具,在需要时捕获被拒绝的工具调用,并仅 return 或 log 应重新进入上下文的输出。两种 flavor 用各自的原语陈述同一契约:相互独立的只读调用可以在 `Promise.all`(TypeScript)或 `asyncio.gather`(Python)下重叠,有副作用的调用按提交顺序单独运行,有依赖的工作用 `await` 排序。声明前缀可能与原生 schema 一样大,尤其在 `'both'` 下,但对提供方缓存保持稳定。 ## 后果 -切换到 `'code'` 的部署必须更新任何仅限 native 的 `toolOrder`。组装监听器有责任维护任何被重写的协议面的完整性。子分发保持序列化,而每次调用的上下文会通过外层结果保留其 source、信封与元数据。 +切换到 `'code'` 的部署必须更新任何仅限 native 的 `toolOrder`。组装监听器有责任维护任何被重写的协议面的完整性。子分发在有界的重叠池下按提交顺序启动,而每次调用的上下文会通过外层结果保留其 source、信封与元数据。 ## 测试 @@ -106,7 +106,7 @@ SDK 指示模型编写一个异步的可擦除 TypeScript 函数体,通过 `aw **在原生工具调用上做结果省略/摘要。** 仅解决问题中上下文膨胀这一半:裁剪旧 `tool-result` 作为可重建请求下的日志化表面替换成本低,但仍需每次调用一次模型往返,且无法表达循环、分支或汇合。互补而非竞争;它可以在 Code Mode 下为残余的原生调用分层。 -**循环中的并行原生分发。** 往返成本的另一个答案;仍是有效的未来工作(open TODO),仍被并发安全元数据阻塞,且仍无组合能力——它并行化的是模型在一步中已经决定的调用。Code Mode 的序列化队列决策保持两者兼容:当元数据就绪时,原生并行分发和每工具绑定并行化一起解锁。 +**循环中的并行原生分发。** 决策当时对往返成本的另一个答案;它被并发安全元数据阻塞,且无论如何都不提供组合能力——它并行化的是模型在一步中已经决定的调用。Code Mode 的队列决策保持了两者兼容,后续也正是这样落地的:元数据以 `isConcurrencySafe` 的形式就绪(见[并行工具调用 note](2026-07-10-parallel-tool-call-execution.md)),原生 rolling-pool 分发与每工具绑定并行化基于同一个分类器一起解锁。 **始终排他(忠于 Cloudflare,无模式)。** 否决,因为本 SDK 的主要消费方是编码 agent:其日常的单次调用(`bash`、`read`、`edit`)作为原生调用已经是最优的,强制每次编辑都通过程序会给常见场景增加负担。mode 配置让忠实形式(`'code'`)只需一行配置即可启用,而不强加于人。 @@ -128,6 +128,6 @@ SDK 指示模型编写一个异步的可擦除 TypeScript 函数体,通过 `aw **大型无损 JSON 值可能耗尽内存。** 工具绑定会在分发前对无损 JSON 创建快照,并完整返回规范 JSON 返回值。运行时会校验 worker 端口两侧,但不对单次绑定设置字节数上限;结构化克隆成本以及进程或 worker 内存构成实际边界。只有包含日志、完成值和失败诊断的组合外层输出账本受字节数上限约束。 -**仅序列化的子分发。** `Promise.all` 尚未获得挂钟并行性,仅减少往返次数;模型可能过度期望。说明中已声明;解除此限制与原生并行分发 TODO 所需的并发安全元数据绑定。 +**子分发的重叠由工具自身的安全声明限定,而非由调用方决定。** 程序里的 `Promise.all` 或 `asyncio.gather` 只在工具自己分类为并发安全的调用之间换来挂钟并行性;一串 exclusive 调用仍要按顺序付出各自的往返开销,模型可能过度期望。两种 flavor 的 SDK 说明都陈述了真实契约。本 note 交付的是使该风险绝对化的序列化占位实现;调度器及其重叠上限由[实时并行 Agent Note](2026-07-26-code-mode-live-parallel-dispatch.md) 负责。 **预算计量读取事件循环,而非 flag。** 忙碌时间轮询(`eventLoopUtilization()`)比精确 CPU 计量更粗糙——预算到期最多延迟一个轮询间隔——且其正确性声明(「pending 的分发不能暂停它」)是抵御恶意程序的关键。两种情况均有单元测试(带 pending 诱饵分发的热循环会在耗尽 `computeMs` 预算时终止;等待慢速绑定的空闲程序则会持续运行至 `maxWallMs`),轮询间隔是内部常量而非配置——部署无法将其误调为绕过手段。`maxWallMs` 是配置项,且会传入 `setTimeout`,后者会把超过 `MAX_TIMER_DELAY_MS`(2^31-1 ms)的延迟夹到 1 ms;因此仅有正数校验会放行一个 25 天的上限,它在第一个 tick 就到期,使每次运行都超时。worker 运行时正因如此在加载时对该字段做范围校验。`computeMs` 不需要上界,因为它对照的是实测占用率,而不是交给定时器。 diff --git a/.agents/notes/implemented/feature/2026-07-31-code-mode-language-dispatch.i18n.yaml b/.agents/notes/implemented/feature/2026-07-31-code-mode-language-dispatch.i18n.yaml new file mode 100644 index 0000000000..66ac99d37c --- /dev/null +++ b/.agents/notes/implemented/feature/2026-07-31-code-mode-language-dispatch.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-31-code-mode-language-dispatch.md +2026-07-31-code-mode-language-dispatch.md: 96001252d6494d058a8df9974fb5a0d59e7d7112 +2026-07-31-code-mode-language-dispatch.zh.md: aa7eb2a6b4b9117f1d707b37afcdbe12b814bad2 diff --git a/.agents/notes/implemented/feature/2026-07-31-code-mode-language-dispatch.md b/.agents/notes/implemented/feature/2026-07-31-code-mode-language-dispatch.md new file mode 100644 index 0000000000..96001252d6 --- /dev/null +++ b/.agents/notes/implemented/feature/2026-07-31-code-mode-language-dispatch.md @@ -0,0 +1,46 @@ +# Agent Note: Code Mode language dispatch and the Python SDK renderer + +Status: implemented + +English | [中文](2026-07-31-code-mode-language-dispatch.zh.md) + +## Problem + +Code Mode generated one SDK flavor: TypeScript. `ToolRegistry` hard-coded `renderToolsSdk` for the `tools:sdk` section and `requireCodeRuntime` rejected any `ctx.codeRuntime.language !== 'typescript'`. Adding a CPython backend means a program's source language is no longer fixed: the same visible tool registry must project a Python SDK when a Python runtime is loaded, and the model-facing `run_code` schema strings ("Execute a Python program …") must match the SDK section's language so the model never sees a TypeScript instruction over a Python runtime. + +This is the tool-facing half of the multi-language Code Mode split; the [code-runtime seam](../../../../packages/code-runtime/code-runtime/README.md) already carries `CodeRuntime.language`. This note owns only how `dsh-tools` dispatches on that field. The backend that implements `language: 'python'` is owned by its own note, delivered separately. + +## Decision + +Language selection is a lookup on `ctx.codeRuntime.language`, resolved lazily at prompt assembly, against two parallel tables in `dsh-tools`: + +- `SDK_RENDERERS` (index.ts) maps a language to its `tools:sdk` renderer — `typescript → renderToolsSdk`, `python → renderToolsSdkPy`. The `tools:sdk` section reads the loaded runtime's language and picks the renderer; `requireCodeRuntime` rejects a `mode: code`/`both` runtime whose language is absent from the table, naming the known languages. +- `RUN_CODE_FLAVORS` (code-mode.ts) maps a language to its two model-facing `run_code` strings (tool `description` and the `code` parameter description), so a language's SDK section and its transport schema always agree. + +Both tables are read with `Object.hasOwn` before use so a language named `toString`/`constructor` cannot resolve an inherited `Object.prototype` member as a renderer. The two guards differ in reachability: `SDK_RENDERERS`' in-callback guard is unreachable because `requireCodeRuntime` validated the same `const` table earlier in the same callback (it carries a `/* v8 ignore */`), while `RUN_CODE_FLAVORS`' guard is the primary, publicly reachable rejection — any language absent from the flavor table hits it through `run_code`'s language-aware getters, which the public `schemas()` reaches without passing `requireCodeRuntime` first; the test reads one of those getters off the definition directly, under a language absent from both tables. A language present in `SDK_RENDERERS` but not `RUN_CODE_FLAVORS` is drift the shared `CodeSdkLanguage` `satisfies` pins reject at `typecheck`, so it is not an input either guard can see; what the guards still own is a mounted runtime reporting a language absent from both tables. Schema emission reads the runtime through `peekRuntime()` rather than `requireRuntime()`: `undefined` (no runtime mounted, reached by definition readers and `schemas()`, of which the doc-catalog harvest is the only shipped one and none of which feeds a model because assembly passes `requireCodeRuntime` first) degrades to the TypeScript flavor, whereas a mounted unknown language fails loud — this is NOT the silent fallback rejected below, which concerns emitting a wrong-language SDK for a real runtime. Adding a backend language is three parallel edits — a `CodeSdkLanguage` member and the two table entries — plus its renderer and the prose that names the well-known values instead of deriving them (the seam's `dsh-code-runtime` README pair, its `CodeRuntime.language` JSDoc, and the `docs/core-data-structures/code-runtime.md` pair; this package's own README pair and its `Config.mode` JSDoc — no gate checks any of it), with no `agent-loop` or registry-structure change. + +`code-mode.ts` depends only on the runtime seam (`@deepseek-ai/dsh-code-runtime`), never on a concrete backend; dispatch is by `runtime.language` at run time. The tool layer therefore lands independently of the protocol and backend PRs — it needs only the seam's `language` field, which is already on master. + +### The Python SDK renderer + +`py-types.ts` renders the same unified tool-schema vocabulary `jsonSchemaToTs` covers, targeting Python: `jsonSchemaToPy` emits a type expression per JSON-schema node, and `renderToolsSdkPy` assembles named `TypedDict`s for each visible tool's arguments and canonical output plus a `tools` object with usage instructions equivalent to the TypeScript flavor. Unsupported raw constructs degrade rather than throwing during assembly, matching the TypeScript renderer's contract. The output is deterministic — lexicographic tool order, byte-identical text for an unchanged tool set — so the prompt stays prefix-cache-friendly. Lexicographic means one ordered member stream: a tool whose name is not a legal attribute is listed as a `tools[name]` comment in its sorted position rather than partitioned to the end, matching how the TypeScript flavor quotes an exotic key in place. That stream forces one thing directly — comment lines are not statements, so a tool set that emits no method at all still needs an explicit `pass`. Three further rules are Python-specific rather than consequences of the ordering. The usage contract states that the declarations are static stubs and arguments are plain `dict`/`list` values: a `TypedDict` reads as a constructible class, so a model that writes `FooArgs(field=1)` gets a `NameError` — TypeScript's `interface` is visibly a type, and the TS flavor's "runs type-stripped" clause already covers it. A description becomes the method's docstring emitted as the FIRST statement of its body: above the `async def` the first one would document the `Tools` class and the rest would be dead expressions, leaving every method undocumented. And a `list[…]` chain degrades to `Any` past `MAX_LIST_NESTING`, because CPython's tokenizer rejects a line with more than 200 open brackets and the block must stay parseable Python — the same reason `docLines` escapes quotes and backslashes. `ts-types` needs neither: TypeScript attaches a leading `/** … */` to the member that follows it and bounds nesting nowhere in its grammar. + +The standard that cap serves is grammatical validity, and the boundary is deliberate: a long `A | B | …` union is valid Python at any length and is left uncapped, even though CPython's `compile()` exhausts its C recursion walking the left-nested `BinOp` spine (measured on 3.9: 1,000 branches compile, 5,000 raise `RecursionError`). Nothing compiles this block — it is prompt text — so that limit costs nothing, whereas capping union length would retire the deep-chain tests that pin the walk's linear time and the class-name propagation cap. A future renderer that does need compilable output should flatten unions rather than truncate them. + +`renderType` validates the whole schema once (`assertSupportedJsonSchema`) and then trusts it, wrapping the walk in one `try/catch` that degrades to `Any` — the same trusted-after-validation stance the sibling `ts-types` renderer takes at this typed same-process seam ([Trust TypeScript at typed same-process seams](../../../../AGENTS.md)). It deliberately carries NO defenses against a schema whose accessors mutate between reads (post-validation cycles, TOCTOU on `const`/`enum`, self-referential functions): the input is a first-party registration (a `defineTool` literal or a raw registration) or a wire-derived plain JSON schema — the former is trusted per AGENTS.md, the latter is a `JSON.parse` product that physically cannot carry accessors, and `renderType` re-validates the whole tree on every call regardless — so such inputs are unreachable, and adding per-shape guards here would break symmetry with `ts-types` (which has none) for values the static interface forbids. `jsonSchemaToPy(schema: unknown)` accepts `unknown` and returns `Any` on a malformed schema — the Python counterpart of the TS flavor's `unknown` — but its contract is "degrade an unsupported schema", not "survive an adversarial mutating one". + +## Alternatives considered + +- **A `language` config field on `ToolRegistry`.** Deployment would then have two places to name the language (the loaded runtime and the tools config) that can disagree; the loaded runtime is the single source of truth, so the registry reads it rather than duplicating it. +- **Importing the Python backend into `code-mode.ts` to detect it.** That would couple the tool layer to a concrete backend and force the protocol/backend PRs to land first. Runtime dispatch on `language` keeps the layer backend-agnostic and independently shippable. +- **A default renderer for an unknown language.** A silent fallback would emit a TypeScript SDK over, e.g., a Ruby runtime — the model would see instructions in the wrong language. Failing loud at assembly is the repository's misconfiguration stance. + +## Consequences + +Adding a backend language is three parallel edits — a `CodeSdkLanguage` member, an `SDK_RENDERERS` entry, and a `RUN_CODE_FLAVORS` entry — plus the renderer function the second points at, with no change to `agent-loop` or the registry structure. The two tables (`SDK_RENDERERS`, `RUN_CODE_FLAVORS`) must stay in step, and that invariant is checked statically rather than left to review: both are `satisfies`-checked against that one union, so a language added to one and not the other fails `typecheck`. This is the mechanical form the drift risk deserves — the runtime `Object.hasOwn` guards would catch it too, but only once a backend reporting that language ships: one PR after the drift, at the consumer's integration point rather than where it was introduced, and on this base never, since no second backend exists. The tables keep their `Record` declared type because `CodeRuntime.language` is an unconstrained `string`; the union pins what the harness ships, the guards reject what a runtime reports. What stays outside that check is the prose that names the well-known values instead of deriving them: `dsh-code-runtime`'s README pair, its `CodeRuntime.language` JSDoc, and the `docs/core-data-structures/code-runtime.md` pair at the seam, plus this package's own README pair and its `Config.mode` JSDoc. Earlier notes name the values as the state at their own PR and are not on that list. Two separate reasons keep it ungated. Prose is not type-checked at all, wherever the union lives. And no type-level pin can stand in for it here: the interface package must not import its consumer's table, and `CodeRuntime.language` stays an unconstrained `string` by design, so moving the union into the seam would not apply it either. A unit test pinning the two key sets equal was rejected in favor of this: it would buy the same check at the cost of a test-only export of two private tables, and would run later than the compiler does. Which of the two runtime failures surfaces depends on the entry point, for a language absent from both tables: assembly reports the missing renderer, because `wireSchemas` calls `requireCodeRuntime` before projecting, while the public `schemas()` reaches `run_code`'s language-aware getters first and reports the missing flavor. The tool layer stays free of any concrete backend dependency, so it lands and is testable on master ahead of the Python protocol and backend. + +The cost is that the Python branch of both tables is unreachable on this base: `CodeRuntime.language` is set by the loaded backend, the only published backend is `dsh-code-runtime-worker` (`'typescript'`), and the registry reads the loaded runtime rather than a config field, so no assembled application can select `renderToolsSdkPy` or `PYTHON_FLAVOR`. The model-visible surface is therefore unchanged by this note's work until a backend reporting `'python'` is published, and this PR's coverage is unit-level — the renderer output plus the dispatch and rejection paths. The keyless snapshot for the Python model interface belongs to the PR that publishes that backend, because only there does a real `cordis.yml` over published plugins produce a Python assembly; a snapshot example that mounted a fixture runtime here would assert against a test double, which [docs/testing.md](../../../../docs/testing.md) rejects as a substitute for the assembled application transcript. + +Two runtime contracts the Python SDK text asserts are owed by that same backend PR. First, the instructions tell the model that exactly `tools` and `ToolCallError` are bound and that the declared `TypedDict` classes are not, so the backend must inject those two names — with `ToolCallError.toolName` populated per the seam's `errorClass` contract — and must NOT bind the declared class names into the program's globals; injecting them "helpfully" would make the SDK text false. Second, the language has to be bound to the request: `requireCodeRuntime` resolves `ctx.codeRuntime` separately at assembly and at `run_code` execution, so a reload that swapped the runtime between those two points would hand a program written against one flavor to the other. The split is finer than those two points — `run_code`'s `description` and `parameters` getters each call `resolveFlavor(peekRuntime())`, and `schemaOf` destructures both, so one projection reads the runtime twice; both reads are for `run_code`'s own schema, since the getters are installed on that one definition and every other definition carries plain data properties. A reload between those two reads yields a single schema whose two halves name different languages. Neither is reachable here — one published backend means both reads return the same flavor and no program ever runs against this renderer's output — and the cross-language rejection is not testable until a second language exists. + +Third, that PR owns the CPython floor, and with it the renderer's Unicode-table skew. Four expressions read the running engine's tables (Node 22.23.1: Unicode 17.0) while the interpreter uses its own (CPython 3.9.6: 13.0.0): `isBareIdentifier`'s `IDENTIFIER`, and `camelCase`'s split set, head test, and `toUpperCase()`. An interpreter older than the engine is the failing direction — the engine emits a character its tokenizer refuses, taking the whole block down — and it arrives by three independent paths. Through the predicate, a bare method or field name carrying a character added between the two versions — to `XID_Start` at its head, or to `XID_Continue` in any tail position, the middle of a name included. Through `camelCase`'s XID reads, a class name, which reaches emitted text whenever any object shape in the tool's schema declares a `TypedDict`, and which the predicate's verdict on the tool name does not gate: `zz-` plus U+1E4D0 never reaches the predicate's skew, since the `-` rejects it outright, yet it still declares `class Zz𞓐xArgs`. Through the case mapping, a class name derived from a tool the predicate accepted — a different table and a wider window than XID membership: U+019B is XID_Start and NFKC-stable, so `async def ƛ` compiles on 3.9.6, but Node uppercases it to U+A7DC (unassigned there; CPython's own `.upper()` is the identity) and `class ꟜArgs` fails with `invalid non-printable character U+A7DC`. The exposure window is the characters and mappings that changed between the two versions, so the PR that names a supported CPython range must decide explicitly between accepting it and pinning all four read points to tables for that floor — pinning the predicate alone leaves both class-name paths open. Nothing here can decide it: the floor does not exist yet, and a table pinned to a guess would be a deployment-varying constant with no configurability behind it. A second axis rides along with the floor and is not one of the four: the names and syntax the block would evaluate at definition time. `TypedDict` needs 3.8, the PEP 585 builtin generics `dict[str, Any]` and `list[…]` need 3.9, an `A | B` annotation 3.10, and `NotRequired` 3.11. These are not parse failures — the block parses on any version, which is the standard the `MAX_LIST_NESTING` cap serves — but definition-time evaluation failures, and nothing in the product evaluates this text. Recording them with the read points keeps "parseable on the supported range" from being read as "executable on it". diff --git a/.agents/notes/implemented/feature/2026-07-31-code-mode-language-dispatch.zh.md b/.agents/notes/implemented/feature/2026-07-31-code-mode-language-dispatch.zh.md new file mode 100644 index 0000000000..aa7eb2a6b4 --- /dev/null +++ b/.agents/notes/implemented/feature/2026-07-31-code-mode-language-dispatch.zh.md @@ -0,0 +1,46 @@ +# Agent Note: Code Mode 语言分发与 Python SDK 渲染器 + +Status: implemented + +[English](2026-07-31-code-mode-language-dispatch.md) | 中文 + +## 问题 + +Code Mode 只生成一种 SDK 形态:TypeScript。`ToolRegistry` 为 `tools:sdk` 段硬编码了 `renderToolsSdk`,且 `requireCodeRuntime` 会拒绝任何 `ctx.codeRuntime.language !== 'typescript'`。引入 CPython 后端后,程序的源语言不再固定:同一个可见工具注册表在加载 Python 运行时时必须投射出 Python SDK,而面向模型的 `run_code` schema 字符串("Execute a Python program …")也必须与 SDK 段的语言一致,模型才不会在 Python 运行时下看到 TypeScript 指令。 + +这是多语言 Code Mode 拆分中面向工具的那一半;[代码运行时 seam](../../../../packages/code-runtime/code-runtime/README.md) 已经携带 `CodeRuntime.language`。本 Note 只负责 `dsh-tools` 如何在该字段上分发。实现 `language: 'python'` 的后端由它自己的 Note 负责,单独交付。 + +## 决策 + +语言选择就是对 `ctx.codeRuntime.language` 的查表,在 prompt 装配时惰性解析,查 `dsh-tools` 里两张平行的表: + +- `SDK_RENDERERS`(index.ts)把语言映射到它的 `tools:sdk` 渲染器——`typescript → renderToolsSdk`、`python → renderToolsSdkPy`。`tools:sdk` 段读取所加载运行时的语言并选出渲染器;`requireCodeRuntime` 拒绝其语言不在表中的 `mode: code`/`both` 运行时,并列出已知语言。 +- `RUN_CODE_FLAVORS`(code-mode.ts)把语言映射到它那两条面向模型的 `run_code` 字符串(工具 `description` 与 `code` 参数描述),使一种语言的 SDK 段与它的传输 schema 始终一致。 + +两张表在使用前都以 `Object.hasOwn` 读取,这样名为 `toString`/`constructor` 的语言不会把继承自 `Object.prototype` 的成员解析成渲染器。两个守卫的可达性不同:`SDK_RENDERERS` 的段内守卫不可达,因为 `requireCodeRuntime` 已在同一回调更早处校验过同一张 `const` 表(它带 `/* v8 ignore */`);而 `RUN_CODE_FLAVORS` 的守卫是主要的、可公开到达的拒绝路径——任何缺席 flavor 表的语言都经 `run_code` 的语言感知 getter 到达它,而公共 `schemas()` 抵达那些 getter 时并未先过 `requireCodeRuntime`;测试直读 definition 上的其中一个 getter,用的是对两张表都缺席的语言。「在 `SDK_RENDERERS` 里却不在 `RUN_CODE_FLAVORS` 里」这种漂移已由共享的 `CodeSdkLanguage` `satisfies` 在 `typecheck` 处拒绝,两个守卫都看不到这种输入;它们如今负责的是所挂载运行时报告了一门两张表都缺席的语言。schema 发射通过 `peekRuntime()` 而非 `requireRuntime()` 读取运行时:`undefined`(无运行时,由直读 definition 的读者与 `schemas()` 到达,其中 doc-catalog 采集是唯一已交付的一个,而它们都不会喂给模型,因为组装路径先过 `requireCodeRuntime`)降级到 TypeScript flavor,而挂载了未知语言则 fail loud——这不是下方被否决的静默回退,那指的是为真实运行时发出错误语言的 SDK。新增一门后端语言是三处并列编辑——一个 `CodeSdkLanguage` 成员加两条表项——再加它的渲染器,以及点名已知值而非从中派生的散文(seam 侧的 `dsh-code-runtime` README 双语对、它的 `CodeRuntime.language` JSDoc 与 `docs/core-data-structures/code-runtime.md` 双语对;本包自己的 README 双语对与它的 `Config.mode` JSDoc,无任何 gate 检查其中任何一处),不动 `agent-loop`,也不动注册表结构。 + +`code-mode.ts` 只依赖运行时 seam(`@deepseek-ai/dsh-code-runtime`),绝不依赖具体后端;分发在运行时按 `runtime.language` 进行。因此工具层独立于协议和后端 PR 落地——它只需要 seam 的 `language` 字段,而该字段已在 master 上。 + +### Python SDK 渲染器 + +`py-types.ts` 渲染 `jsonSchemaToTs` 所覆盖的同一套统一工具 schema 词汇,目标为 Python:`jsonSchemaToPy` 为每个 JSON-schema 节点发出一个类型表达式,`renderToolsSdkPy` 为每个可见工具的参数与规范输出装配具名 `TypedDict`,再加一个带用法说明的 `tools` 对象,与 TypeScript 形态等价。不支持的原始构造在装配时降级而非抛错,与 TypeScript 渲染器的契约一致。输出是确定性的——工具按字典序排列,工具集不变时文本逐字节相同——因此 prompt 保持 prefix-cache 友好。字典序意味着单一有序的成员流:名字不是合法属性的工具以 `tools[name]` 注释出现在它排序后的位置上,而不是被分拣到末尾,与 TypeScript 形态就地为异常键加引号的做法一致。这个成员流直接决定了一件事:注释行不是语句,所以一个不发出任何方法的工具集仍需显式 `pass`。另有三条规则并非源自排序,而是 Python 特有。其一,用法约定声明这些声明只是静态存根、参数为普通 `dict`/`list` 值:`TypedDict` 读起来像一个可构造的类,模型若写 `FooArgs(field=1)` 会得到 `NameError`——TypeScript 的 `interface` 一眼就是类型,且 TS 形态的「runs type-stripped」一句已经覆盖了它。其二,描述会成为方法的 docstring,且必须作为方法体的**第一条语句**发出:放在 `async def` 之上,第一条会变成 `Tools` 的类文档、其余都是无效果表达式,导致每个方法都没有文档。其三,`list[…]` 链超过 `MAX_LIST_NESTING` 后降级为 `Any`,因为 CPython 的 tokenizer 拒绝一行中超过 200 个同时未闭合的括号,而这个块必须是可解析的 Python——与 `docLines` 转义引号和反斜杠是同一个理由。`ts-types` 两者都不需要:TypeScript 会把前置的 `/** … */` 附着到其后的成员上,其语法也不对嵌套设限。 + +该上限服务的标准是**语法合法性**,这条边界是有意划定的:长的 `A | B | …` union 在任何长度下都是合法 Python,故不设上限——尽管 CPython 的 `compile()` 在沿左嵌套 `BinOp` 脊柱下降时会耗尽 C 递归(在 3.9 上实测:1,000 个分支可编译,5,000 个抛 `RecursionError`)。没有任何东西会编译这个块——它是提示词文本——所以那条限制在这里没有代价;而给 union 长度封顶会作废那几个钉住 walk 线性时间与类名传播上限的深链测试。将来若有渲染器确实需要可编译的输出,应当把 union 拍平,而不是截断。 + +`renderType` 先用 `assertSupportedJsonSchema` 整树校验一次、随后信任它,用单个 `try/catch` 把整个遍历兜住并降级为 `Any`——与姊妹渲染器 `ts-types` 在这个 typed 同进程 seam 上采取的「校验后信任」姿态一致([Trust TypeScript at typed same-process seams](../../../../AGENTS.md))。它有意不设任何针对「访问器在多次读取间变值」的防御(校验后成环、`const`/`enum` 的 TOCTOU、自引用函数):输入是第一方注册(`defineTool` 字面量或 raw 注册)或从 wire 桥接而来的纯 JSON——前者按 AGENTS.md 受信任,后者是 `JSON.parse` 产物、物理上不可能携带访问器,且每次调用 `renderType` 都会整树重新校验——这类输入不可达,而在此加逐形态守卫会为静态接口所禁止的值破坏与 `ts-types`(没有这类守卫)的对称。`jsonSchemaToPy(schema: unknown)` 接受 `unknown` 并对畸形 schema 返回 `Any`——TypeScript 形态 `unknown` 的对应物——但它的契约是「降级不支持的 schema」,而非「扛住对抗性的可变 schema」。 + +## Alternatives considered + +- **在 `ToolRegistry` 上加一个 `language` 配置字段。** 那样部署方就会有两处命名语言(所加载的运行时与 tools 配置)且可能相互矛盾;所加载的运行时是唯一真相来源,故注册表读取它而不复制它。 +- **把 Python 后端 import 进 `code-mode.ts` 来检测它。** 那会把工具层耦合到具体后端,并迫使协议/后端 PR 先落地。按 `language` 运行时分发使该层保持后端无关、可独立发布。 +- **为未知语言提供默认渲染器。** 静默回退会在比如 Ruby 运行时上发出 TypeScript SDK——模型会看到错误语言的指令。在装配处 fail loud 是本仓库对错误配置的立场。 + +## Consequences + +新增一门后端语言是三处并列编辑——一个 `CodeSdkLanguage` 成员、一个 `SDK_RENDERERS` 表项、一个 `RUN_CODE_FLAVORS` 表项——再加第二处所指向的渲染器函数,不动 `agent-loop`,也不动注册表结构。两张表(`SDK_RENDERERS`、`RUN_CODE_FLAVORS`)必须同步,且这条不变式由静态检查把关,而非交给 review:两张表都以 `satisfies` 对上述同一个 union 校验,因此只加其一而漏掉另一会在 `typecheck` 处失败。这正是该漂移风险应有的机械形式——运行期的 `Object.hasOwn` 守卫同样能捕获,但要等到有后端报告该语言之后:晚于漂移引入一个 PR,且触发点在消费方的集成处而非漂移引入处;在当前 base 上则永远不会触发,因为不存在第二个后端。两张表的声明类型仍是 `Record`,因为 `CodeRuntime.language` 是不受约束的 `string`:union 钉住本仓库交付了什么,守卫拒绝运行时报告了什么。落在这条检查之外的是点名已知值而非从中派生的散文:seam 侧的 `dsh-code-runtime` README 双语对、它的 `CodeRuntime.language` JSDoc 与 `docs/core-data-structures/code-runtime.md` 双语对,再加本包自己的 README 双语对与它的 `Config.mode` JSDoc。更早的 note 点名这些值时记的是其自身 PR 当时的状态,不在此列。让它无 gate 的是两条独立理由。其一,散文根本不受类型检查,union 放在哪里都一样。其二,类型级替代在这里也不可用:接口包不得 import 其消费方的表,而 `CodeRuntime.language` 按设计保持不受约束的 `string`,即便把 union 迁进 seam 也不会作用到它。用一个断言两张表键集相等的 unit test 的方案被否决:它买到的是同一条检查,代价却是把两张私有表做测试专用导出,且运行时机晚于编译器。对两张表都缺席的语言,两种运行期失败中报出哪一条随入口而异:组装路径报缺渲染器,因为 `wireSchemas` 在投影前先调 `requireCodeRuntime`;而公共 `schemas()` 先经过 `run_code` 的语言感知 getter,报的是缺 flavor 表项。工具层不依赖任何具体后端,因此它能先于 Python 协议和后端在 master 上落地并可测。 + +代价是两张表的 Python 分支在当前 base 上不可达:`CodeRuntime.language` 由所加载的后端设定,已发布的后端只有 `dsh-code-runtime-worker`(`'typescript'`),而注册表读取的是所加载的运行时而非某个配置字段,因此没有任何一份组装好的应用能选中 `renderToolsSdkPy` 或 `PYTHON_FLAVOR`。也就是说,在报告 `'python'` 的后端发布之前,本 note 的工作不改变模型可见表面,本 PR 的覆盖因此是 unit 级——渲染器输出加分发与拒绝路径。Python 模型界面的 keyless snapshot 归属于发布该后端的那个 PR,因为只有在那里,一份基于已发布插件的真实 `cordis.yml` 才会产出 Python 组装;在此处挂载 fixture 运行时的快照示例断言的是测试替身,而 [docs/testing.md](../../../../docs/testing.md) 明确拒绝以此替代组装好的应用 transcript。 + +Python SDK 文本断言的两条运行时契约同样归属那个 backend PR。其一,说明文字告诉模型运行时恰好绑定 `tools` 与 `ToolCallError` 两个名字、所声明的 `TypedDict` 类不绑定,因此后端必须注入这两个名字(并按 seam 的 `errorClass` 契约填充 `ToolCallError.toolName`),且**不得**把所声明的类名绑进程序全局——「好心」注入会使这段 SDK 文本变成假话。其二,语言必须绑定到请求上:`requireCodeRuntime` 在组装时与 `run_code` 执行时分别解析 `ctx.codeRuntime`,若在这两点之间发生重载并换掉运行时,就会把针对一种形态写成的程序交给另一种形态执行。分裂比这两点更细——`run_code` 的 `description` 与 `parameters` 两个 getter 各自调用 `resolveFlavor(peekRuntime())`,而 `schemaOf` 会解构这两个字段,因此一次投影读两次运行时;两次都属于 `run_code` 自己的 schema,因为这两个 getter 只装在那一个 definition 上,其余 definition 携带的都是普通数据属性。在这两次读取之间重载会产出单个 schema 的两半分属不同语言。两者在此处都不可达——只有一个已发布后端意味着两次读取返回同一形态,且没有任何程序会针对本渲染器的输出运行——而跨语言拒绝在第二门语言存在之前也无法测试。 + +其三,那个 PR 拥有 CPython 版本下限,连带拥有本渲染器的 Unicode 表偏斜。有四处表达式读所运行引擎的表(Node 22.23.1:Unicode 17.0),而解释器用它自己的表(CPython 3.9.6:13.0.0):`isBareIdentifier` 的 `IDENTIFIER`,以及 `camelCase` 的切分集、头部测试与 `toUpperCase()`。解释器旧于引擎是会失败的那个方向——引擎发出的字符被其 tokenizer 拒收,整个块随之不可解析——而它经三条独立路径抵达。经判据抵达的是裸发的方法名或字段名,其中带有一个在两个版本之间新增的字符——首位加进 `XID_Start`,或尾部任意位置(含名字中部)加进 `XID_Continue`。经 `camelCase` 的 XID 读取抵达的是类名:只要工具 schema 中有任一对象形态声明 `TypedDict`,该类名就进入发出的文本,且判据对工具名的裁决并不对它设闸——工具名 `zz-` 加 U+1E4D0 因 `-` 被判据直接拒绝、从不触及那里的偏斜,却照样声明 `class Zz𞓐xArgs`。经大写映射抵达的是由判据已接受的工具派生出的类名——这是另一张表,窗口也比 XID 归属更宽:U+019B 既是 XID_Start 又 NFKC 稳定,故 `async def ƛ` 在 3.9.6 上可编译,但 Node 将其大写为 U+A7DC(在那里未分配;CPython 自己的 `.upper()` 在此是恒等),于是 `class ꟜArgs` 以 `invalid non-printable character U+A7DC` 失败。暴露窗口是两个版本之间发生变化的那些字符与映射,所以宣布支持某个 CPython 范围的那个 PR 必须在「接受该暴露」与「按该下限的表钉住全部四个读取点」之间显式作出决定——只钉判据会同时留下两条类名路径。此处无法决定:下限尚不存在,而按猜测钉死一张表会成为一个随部署而变、却没有可配置性支撑的常量。还有第二条轴随该下限一同确定,且不属于那四个读取点:本块在定义期会被求值的那些名字与语法。`TypedDict` 需要 3.8,PEP 585 的内建泛型 `dict[str, Any]` 与 `list[…]` 需要 3.9,`A | B` 形式的注解需要 3.10,`NotRequired` 需要 3.11。这些不是解析失败——本块在任何版本上都能解析,这正是 `MAX_LIST_NESTING` 上限所服务的标准——而是定义期求值失败,且产品中没有任何东西会求值这段文本。把它们与那四个读取点记在一起,可避免把「在所支持范围上可解析」读成「在其上可执行」。 diff --git a/docs/config-catalog.md b/docs/config-catalog.md index ad27aac3b9..c1d69445e0 100644 --- a/docs/config-catalog.md +++ b/docs/config-catalog.md @@ -2218,8 +2218,9 @@ export interface Config { /** * Model presentation. `native` (default) sends every visible schema; `code` * sends only `run_code` plus a generated SDK prompt; `both` sends both forms. - * Code modes require a TypeScript runtime and fail prompt assembly when it is - * absent or mismatched. Under `code`, native names in `toolOrder` are invalid. + * Code modes require a `ctx.codeRuntime` whose `language` has a registered + * SDK renderer (TypeScript or Python) and fail prompt assembly when it is + * absent or has no renderer. Under `code`, native names in `toolOrder` are invalid. */ mode?: ToolPresentationMode /** @@ -2236,7 +2237,7 @@ export interface Config { export type ToolPresentationMode = 'native' | 'code' | 'both' ``` -Source: [`packages/core/tools/src/index.ts:592`](../packages/core/tools/src/index.ts) +Source: [`packages/core/tools/src/index.ts:616`](../packages/core/tools/src/index.ts) ## `@deepseek-ai/dsh-typert-loader` diff --git a/docs/cordis-catalog/events.md b/docs/cordis-catalog/events.md index 44e0727101..793b9d142c 100644 --- a/docs/cordis-catalog/events.md +++ b/docs/cordis-catalog/events.md @@ -842,7 +842,7 @@ A tool was registered or unregistered, or a scoped restriction changed (the avai 'tools/change'(): void ``` -Source: [`packages/core/tools/src/index.ts:167`](../../packages/core/tools/src/index.ts) +Source: [`packages/core/tools/src/index.ts:191`](../../packages/core/tools/src/index.ts) ### `tools/code-dispatch-log` — waterfall @@ -866,7 +866,7 @@ Shape the DURABLE LOG COPY of one `run_code` sub-dispatch outcome before the bri Types: [CodeDispatchLog](../core-data-structures/tools.md) · [ContentBlock](../core-data-structures/core.md) · [Scoped](../core-data-structures/scope.md) · [ToolRegistry](../core-data-structures/tools.md) -Source: [`packages/core/tools/src/index.ts:149`](../../packages/core/tools/src/index.ts) +Source: [`packages/core/tools/src/index.ts:173`](../../packages/core/tools/src/index.ts) ### `tools/execute` — waterfall @@ -888,7 +888,7 @@ Around-dispatch waterfall for timeout, retry, or metrics. `next()` returns a nor Types: [Scoped](../core-data-structures/scope.md) · [ToolDispatchExecution](../core-data-structures/tools.md) · [ToolExecutionResult](../core-data-structures/tools.md) · [ToolRegistry](../core-data-structures/tools.md) -Source: [`packages/core/tools/src/index.ts:124`](../../packages/core/tools/src/index.ts) +Source: [`packages/core/tools/src/index.ts:148`](../../packages/core/tools/src/index.ts) ### `tools/post-execute` — waterfall @@ -911,7 +911,7 @@ Accept, replace, enrich, or block a normalized dispatch result. `next()` accepts Types: [PostToolDecision](../core-data-structures/tools.md) · [Scoped](../core-data-structures/scope.md) · [ToolExecution](../core-data-structures/tools.md) · [ToolExecutionResult](../core-data-structures/tools.md) · [ToolRegistry](../core-data-structures/tools.md) -Source: [`packages/core/tools/src/index.ts:136`](../../packages/core/tools/src/index.ts) +Source: [`packages/core/tools/src/index.ts:160`](../../packages/core/tools/src/index.ts) ### `tools/pre-execute` — waterfall @@ -932,7 +932,7 @@ Allow, deny, or ask before dispatch. `next()` delegates to allow; missing approv Types: [PreToolDecision](../core-data-structures/tools.md) · [Scoped](../core-data-structures/scope.md) · [ToolExecution](../core-data-structures/tools.md) · [ToolRegistry](../core-data-structures/tools.md) -Source: [`packages/core/tools/src/index.ts:113`](../../packages/core/tools/src/index.ts) +Source: [`packages/core/tools/src/index.ts:137`](../../packages/core/tools/src/index.ts) ### `tools/result` — emit @@ -951,7 +951,7 @@ Observe the frozen, lossless-JSON final outcome. Listener failures are contained Types: [Scoped](../core-data-structures/scope.md) · [ToolExecution](../core-data-structures/tools.md) · [ToolExecutionResult](../core-data-structures/tools.md) · [ToolRegistry](../core-data-structures/tools.md) -Source: [`packages/core/tools/src/index.ts:157`](../../packages/core/tools/src/index.ts) +Source: [`packages/core/tools/src/index.ts:181`](../../packages/core/tools/src/index.ts) ## `workflow/*` diff --git a/docs/cordis-catalog/services.md b/docs/cordis-catalog/services.md index 2ded3e4f07..f614295063 100644 --- a/docs/cordis-catalog/services.md +++ b/docs/cordis-catalog/services.md @@ -2523,7 +2523,7 @@ async execute(exec: ToolExecutionInput): Promise Types: [ScopeKey](../core-data-structures/scope.md) · [ToolDefinition](../core-data-structures/tools.md) · [ToolExecutionInput](../core-data-structures/tools.md) · [ToolExecutionMode](../core-data-structures/tools.md) · [ToolExecutionResult](../core-data-structures/tools.md) · [ToolGuard](../core-data-structures/tools.md) · [ToolRestriction](../core-data-structures/tools.md) · [ToolSchema](../core-data-structures/tools.md) -Source: [`packages/core/tools/src/index.ts:714`](../../packages/core/tools/src/index.ts) +Source: [`packages/core/tools/src/index.ts:739`](../../packages/core/tools/src/index.ts) ## `ctx.typert` — `TypertRegistry` diff --git a/docs/core-data-structures/code-runtime.i18n.yaml b/docs/core-data-structures/code-runtime.i18n.yaml index fbdee4c938..686ba7b940 100644 --- a/docs/core-data-structures/code-runtime.i18n.yaml +++ b/docs/core-data-structures/code-runtime.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write docs/core-data-structures/code-runtime.md -code-runtime.md: 64de3c45d4f1d1d981daa6c6f074abb667e0aa52 -code-runtime.zh.md: daf07aaf613852a6c4a7b1aff152fcc61052fbca +code-runtime.md: 24127dafbd4a202b6764b55319ec404e77391929 +code-runtime.zh.md: 35f06f2b48bfd3af6ccac6d9a4dd366ea9ec0c92 diff --git a/docs/core-data-structures/code-runtime.md b/docs/core-data-structures/code-runtime.md index 64de3c45d4..24127dafbd 100644 --- a/docs/core-data-structures/code-runtime.md +++ b/docs/core-data-structures/code-runtime.md @@ -144,4 +144,4 @@ interface CodeRunFailure { ## The service -`CodeRuntime` (`ctx.codeRuntime`, abstract — defined in [`packages/code-runtime/code-runtime/src/index.ts`](../../packages/code-runtime/code-runtime/src/index.ts)) is `run(request)` plus two readonly descriptors: `language` (what the program must be written in — `'typescript'` is the well-known value; a consumer generating language-specific presentation switches on it and fails loud on one it cannot present) and `isolation` (the execution substrate — `'worker-thread'`, `'process'`, `'container'`; a diagnostic label, **not a security claim**). Implementations must keep runs isolated from each other (no cross-run state) and dispose to quiescence: in-flight runs are terminated and awaited before teardown completes. +`CodeRuntime` (`ctx.codeRuntime`, abstract — defined in [`packages/code-runtime/code-runtime/src/index.ts`](../../packages/code-runtime/code-runtime/src/index.ts)) is `run(request)` plus two readonly descriptors: `language` (what the program must be written in — `'typescript'` and `'python'` are the well-known values, those `dsh-tools` presents, and only `'typescript'` has a published backend; a consumer generating language-specific presentation switches on it and fails loud on one it cannot present) and `isolation` (the execution substrate — `'worker-thread'`, `'process'`, `'container'`; a diagnostic label, **not a security claim**). Implementations must keep runs isolated from each other (no cross-run state) and dispose to quiescence: in-flight runs are terminated and awaited before teardown completes. diff --git a/docs/core-data-structures/code-runtime.zh.md b/docs/core-data-structures/code-runtime.zh.md index daf07aaf61..35f06f2b48 100644 --- a/docs/core-data-structures/code-runtime.zh.md +++ b/docs/core-data-structures/code-runtime.zh.md @@ -144,4 +144,4 @@ interface CodeRunFailure { ## 服务 -`CodeRuntime`(`ctx.codeRuntime`,抽象服务,定义于 [`packages/code-runtime/code-runtime/src/index.ts`](../../packages/code-runtime/code-runtime/src/index.ts))由 `run(request)` 加两个只读描述符组成:`language`(程序必须使用的语言,`'typescript'` 是已知值;生成语言相关展示的消费方据此切换,遇到无法展示的语言时应显式报错)和 `isolation`(执行基底,`'worker-thread'`、`'process'`、`'container'`;仅为诊断标签,**不构成安全承诺**)。实现必须保证各次运行彼此隔离(无跨运行状态),并在 dispose(资源释放)时等待系统完全停稳:teardown 要等到所有进行中的运行均已终止并结算后才完成。 +`CodeRuntime`(`ctx.codeRuntime`,抽象服务,定义于 [`packages/code-runtime/code-runtime/src/index.ts`](../../packages/code-runtime/code-runtime/src/index.ts))由 `run(request)` 加两个只读描述符组成:`language`(程序必须使用的语言,已知值为 `'typescript'` 与 `'python'`,即 `dsh-tools` 能呈现的那些,其中只有 `'typescript'` 有已发布的后端;生成语言相关展示的消费方据此切换,遇到无法展示的语言时应显式报错)和 `isolation`(执行基底,`'worker-thread'`、`'process'`、`'container'`;仅为诊断标签,**不构成安全承诺**)。实现必须保证各次运行彼此隔离(无跨运行状态),并在 dispose(资源释放)时等待系统完全停稳:teardown 要等到所有进行中的运行均已终止并结算后才完成。 diff --git a/docs/event-producer-consumer.md b/docs/event-producer-consumer.md index 23794b5c9d..6790684a8c 100644 --- a/docs/event-producer-consumer.md +++ b/docs/event-producer-consumer.md @@ -44,12 +44,12 @@ This matrix shows which packages dispatch each harness-owned event and which pac | `system-prompt/assemble` | `waterfall` | [`packages/core/system-prompt/src/index.ts:29`](../packages/core/system-prompt/src/index.ts) | [`system-prompt`](../packages/core/system-prompt) (`waterfall`) | [`agent`](../packages/core/agent), [`system-prompt`](../packages/core/system-prompt) | | `system-prompt/change` | `emit` | [`packages/core/system-prompt/src/index.ts:35`](../packages/core/system-prompt/src/index.ts) | [`system-prompt`](../packages/core/system-prompt) (`emit`) | - | | `telemetry/record` | `waterfall` | [`packages/telemetry/session-telemetry/src/index.ts:41`](../packages/telemetry/session-telemetry/src/index.ts) | [`session-telemetry`](../packages/telemetry/session-telemetry) (`waterfall`) | - | -| `tools/change` | `emit` | [`packages/core/tools/src/index.ts:167`](../packages/core/tools/src/index.ts) | [`tools`](../packages/core/tools) (`emit`) | - | -| `tools/code-dispatch-log` | `waterfall` | [`packages/core/tools/src/index.ts:149`](../packages/core/tools/src/index.ts) | [`tools`](../packages/core/tools) (`waterfall`) | [`spill-policy`](../packages/spill/spill-policy) | -| `tools/execute` | `waterfall` | [`packages/core/tools/src/index.ts:124`](../packages/core/tools/src/index.ts) | [`tools`](../packages/core/tools) (`waterfall`) | [`session-checkpoint-policy`](../packages/session-persistence/session-checkpoint-policy), [`timeout-policy`](../packages/timeout/timeout-policy) | -| `tools/post-execute` | `waterfall` | [`packages/core/tools/src/index.ts:136`](../packages/core/tools/src/index.ts) | [`tools`](../packages/core/tools) (`waterfall`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex), [`repeat-tool-guard`](../packages/guard/repeat-tool-guard), [`spill-policy`](../packages/spill/spill-policy), [`tool-fs-search`](../packages/fs/tool-fs-search) | -| `tools/pre-execute` | `waterfall` | [`packages/core/tools/src/index.ts:113`](../packages/core/tools/src/index.ts) | [`tools`](../packages/core/tools) (`waterfall`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex), [`tool-tasks`](../packages/tasks/tool-tasks) | -| `tools/result` | `emit` | [`packages/core/tools/src/index.ts:157`](../packages/core/tools/src/index.ts) | [`tools`](../packages/core/tools) (`events.dispatch`) | [`subagent-inprocess`](../packages/subagent/subagent-inprocess), [`workspace-context`](../packages/context/workspace-context) | +| `tools/change` | `emit` | [`packages/core/tools/src/index.ts:191`](../packages/core/tools/src/index.ts) | [`tools`](../packages/core/tools) (`emit`) | - | +| `tools/code-dispatch-log` | `waterfall` | [`packages/core/tools/src/index.ts:173`](../packages/core/tools/src/index.ts) | [`tools`](../packages/core/tools) (`waterfall`) | [`spill-policy`](../packages/spill/spill-policy) | +| `tools/execute` | `waterfall` | [`packages/core/tools/src/index.ts:148`](../packages/core/tools/src/index.ts) | [`tools`](../packages/core/tools) (`waterfall`) | [`session-checkpoint-policy`](../packages/session-persistence/session-checkpoint-policy), [`timeout-policy`](../packages/timeout/timeout-policy) | +| `tools/post-execute` | `waterfall` | [`packages/core/tools/src/index.ts:160`](../packages/core/tools/src/index.ts) | [`tools`](../packages/core/tools) (`waterfall`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex), [`repeat-tool-guard`](../packages/guard/repeat-tool-guard), [`spill-policy`](../packages/spill/spill-policy), [`tool-fs-search`](../packages/fs/tool-fs-search) | +| `tools/pre-execute` | `waterfall` | [`packages/core/tools/src/index.ts:137`](../packages/core/tools/src/index.ts) | [`tools`](../packages/core/tools) (`waterfall`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex), [`tool-tasks`](../packages/tasks/tool-tasks) | +| `tools/result` | `emit` | [`packages/core/tools/src/index.ts:181`](../packages/core/tools/src/index.ts) | [`tools`](../packages/core/tools) (`events.dispatch`) | [`subagent-inprocess`](../packages/subagent/subagent-inprocess), [`workspace-context`](../packages/context/workspace-context) | | `workflow/agent-end` | `emit` | [`packages/workflow/workflow/src/index.ts:81`](../packages/workflow/workflow/src/index.ts) | [`workflow`](../packages/workflow/workflow) (`events.dispatch`) | [`workflow`](../packages/workflow/workflow) | | `workflow/agent-start` | `emit` | [`packages/workflow/workflow/src/index.ts:70`](../packages/workflow/workflow/src/index.ts) | [`workflow`](../packages/workflow/workflow) (`events.dispatch`) | [`workflow`](../packages/workflow/workflow) | | `workflow/end` | `emit` | [`packages/workflow/workflow/src/index.ts:91`](../packages/workflow/workflow/src/index.ts) | [`workflow`](../packages/workflow/workflow) (`events.dispatch`) | [`workflow`](../packages/workflow/workflow) | diff --git a/docs/tool-catalog.md b/docs/tool-catalog.md index d3dfa55ccb..42b68bb6e8 100644 --- a/docs/tool-catalog.md +++ b/docs/tool-catalog.md @@ -16,7 +16,7 @@ This table connects model-visible tool names to the plugin package and service s | Tool package | Model-visible names | Requires | Writes / affects | Shipped aliases | Deployment note | | --- | --- | --- | --- | --- | --- | | `@deepseek-ai/dsh-tool-ask-user` | `ask_user_question` | `ctx.tools`, `ctx.userInteraction` | `tool/call`, `tool/result after a UI/provider answers the question` | - | ask_user_question pauses the tool call until the active UI provider returns a human answer. | -| `@deepseek-ai/dsh-tools` | `run_code` | `ctx.tools`, `ctx.codeRuntime (execution time)`, `ctx.systemPrompt` | `tool/call`, `one tool/code-dispatch-start + tool/code-dispatch pair per bridged sub-call`, `tool/result` | - | Owned by the tool registry as a reserved transport outside filterable capability layers under `mode: code` / `mode: both` (see the Code Mode Agent Note). Under `code` it is the registry's only wire contribution; the other visible capabilities are declared in a generated TypeScript SDK section, and a program calls them through bindings scheduled under the native concurrency contract (submission-ordered starts and policy; concurrency-safe bodies overlap up to `maxParallelSubCalls`) that re-enter the complete guarded tool pipeline and link each nested execution to this outer result. | +| `@deepseek-ai/dsh-tools` | `run_code` | `ctx.tools`, `ctx.codeRuntime (execution time)`, `ctx.systemPrompt` | `tool/call`, `one tool/code-dispatch-start + tool/code-dispatch pair per bridged sub-call`, `tool/result` | - | Owned by the tool registry as a reserved transport outside filterable capability layers under `mode: code` / `mode: both` (see the Code Mode Agent Note). Under `code` it is the registry's only wire contribution; the other visible capabilities are declared in a generated SDK section in the loaded runtime's language, and a program calls them through bindings scheduled under the native concurrency contract (submission-ordered starts and policy; concurrency-safe bodies overlap up to `maxParallelSubCalls`) that re-enter the complete guarded tool pipeline and link each nested execution to this outer result. | | `@deepseek-ai/dsh-plan-mode` | `exit_plan_mode` | `ctx.tools`, `ctx.systemPrompt`, `ctx.userInteraction (execution time, opportunistic)` | `tool/call`, `plan/mode inactive on an approved review`, `tool/result` | - | exit_plan_mode stays in the model-facing schema while planning is inactive so transitions add no tool-catalog churn on top of the plan-policy change. Its execute path rejects calls outside plan mode; in plan mode it presents the plan over the user-interaction seam (approve / keep planning with feedback), and approval logs plan mode inactive at the step boundary. | | `@deepseek-ai/dsh-tool-bash` | `bash` | `ctx.tools`, `ctx.bash`, `ctx.systemPrompt`, `ctx.bashEnv`, `ctx.tasks at call time for run_in_background` | `tool/call`, `tool/result` | - | The bash tool is the model-facing consumer of the bash executor seam. A `run_in_background` run registers with the generic `ctx.tasks` runtime and is collected/stopped through the `task_*` tools from `@deepseek-ai/dsh-tool-tasks`; the `enableRunInBackground` config (default true) removes the parameter entirely when disabled. | | `@deepseek-ai/dsh-tool-pwsh` | `pwsh` | `ctx.tools`, `ctx.bash`, `ctx.systemPrompt`, `ctx.bashEnv`, `ctx.tasks at call time for run_in_background` | `tool/call`, `tool/result` | - | The pwsh tool is the PowerShell-dialect consumer of the bash executor seam for Windows compositions (a PowerShell executor such as `@deepseek-ai/dsh-pwsh-local` backs `ctx.bash`); it mirrors the bash tool call-for-call minus the sandbox surface — `run_in_background` runs register with the generic `ctx.tasks` runtime and are collected/stopped through the `task_*` tools, and the managed `DSH_*` environment comes from `@deepseek-ai/dsh-bash-env`. Each call runs in a fresh process (no persistent PTY session; ConPTY is roadmap work), with native `C:\...` paths and `$env:NAME` variables. | @@ -139,7 +139,7 @@ Execute a TypeScript program against the available tools. Write the BODY of an a Source: [`packages/core/tools/src/code-mode.ts`](../packages/core/tools/src/code-mode.ts) -Owned by the tool registry as a reserved transport outside filterable capability layers under `mode: code` / `mode: both` (see the Code Mode Agent Note). Under `code` it is the registry's only wire contribution; the other visible capabilities are declared in a generated TypeScript SDK section, and a program calls them through bindings scheduled under the native concurrency contract (submission-ordered starts and policy; concurrency-safe bodies overlap up to `maxParallelSubCalls`) that re-enter the complete guarded tool pipeline and link each nested execution to this outer result. +Owned by the tool registry as a reserved transport outside filterable capability layers under `mode: code` / `mode: both` (see the Code Mode Agent Note). Under `code` it is the registry's only wire contribution; the other visible capabilities are declared in a generated SDK section in the loaded runtime's language, and a program calls them through bindings scheduled under the native concurrency contract (submission-ordered starts and policy; concurrency-safe bodies overlap up to `maxParallelSubCalls`) that re-enter the complete guarded tool pipeline and link each nested execution to this outer result. ## `@deepseek-ai/dsh-plan-mode` diff --git a/packages/code-runtime/README.i18n.yaml b/packages/code-runtime/README.i18n.yaml index a04f1985c2..3eb462d513 100644 --- a/packages/code-runtime/README.i18n.yaml +++ b/packages/code-runtime/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/code-runtime/README.md -README.md: 2d32a05071efdfa05c336211196bb769ed5a5fc7 -README.zh.md: 62c9a395ac3cf2b5cd55455ab5a273a1e276f6b7 +README.md: f20a287419b94b1a9dc1d8da7303fc4d3032cfd3 +README.zh.md: f5cd4c9949f2bd7a7d6d7cd078144910712a3819 diff --git a/packages/code-runtime/README.md b/packages/code-runtime/README.md index 2d32a05071..f20a287419 100644 --- a/packages/code-runtime/README.md +++ b/packages/code-runtime/README.md @@ -2,7 +2,7 @@ English | [中文](README.zh.md) -The code-execution capability seam (see [capability seams](../../.agents/notes/implemented/architecture/2026-06-13-capability-seams.md)): an abstract runtime interface for executing one model-written program against host-provided async bindings, capturing what it printed and returned. The consumer is the tool registry's [Code Mode](../core/tools/README.md) (`tools: { mode: code }` — the `run_code` tool and the generated TypeScript SDK); design in the [Code Mode Agent Note](../../.agents/notes/implemented/feature/2026-06-15-code-mode.md). **Product** packages. +The code-execution capability seam (see [capability seams](../../.agents/notes/implemented/architecture/2026-06-13-capability-seams.md)): an abstract runtime interface for executing one model-written program against host-provided async bindings, capturing what it printed and returned. The consumer is the tool registry's [Code Mode](../core/tools/README.md) (`tools: { mode: code }` — the `run_code` tool and the SDK generated in the loaded runtime's `language`); design in the [Code Mode Agent Note](../../.agents/notes/implemented/feature/2026-06-15-code-mode.md). **Product** packages. | Package | Role | ctx key | |---|---|---| diff --git a/packages/code-runtime/README.zh.md b/packages/code-runtime/README.zh.md index 62c9a395ac..f5cd4c9949 100644 --- a/packages/code-runtime/README.zh.md +++ b/packages/code-runtime/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -代码执行能力 seam(参见[能力 seam](../../.agents/notes/implemented/architecture/2026-06-13-capability-seams.md)):一个抽象运行时接口,用于对宿主提供的异步绑定执行模型编写的程序,并捕获它打印和返回的内容。消费方是工具注册表的 [Code Mode](../core/tools/README.md)(`tools: { mode: code }`,即 `run_code` 工具和生成的 TypeScript SDK);设计见 [Code Mode Agent Note](../../.agents/notes/implemented/feature/2026-06-15-code-mode.md)。这些全是**产品**包。 +代码执行能力 seam(参见[能力 seam](../../.agents/notes/implemented/architecture/2026-06-13-capability-seams.md)):一个抽象运行时接口,用于对宿主提供的异步绑定执行模型编写的程序,并捕获它打印和返回的内容。消费方是工具注册表的 [Code Mode](../core/tools/README.md)(`tools: { mode: code }`,即 `run_code` 工具和按所加载运行时 `language` 生成的 SDK);设计见 [Code Mode Agent Note](../../.agents/notes/implemented/feature/2026-06-15-code-mode.md)。这些全是**产品**包。 | 包 | 职责 | ctx key | |---|---|---| diff --git a/packages/code-runtime/code-runtime/README.i18n.yaml b/packages/code-runtime/code-runtime/README.i18n.yaml index 8e45c6265b..6da4ec0ca9 100644 --- a/packages/code-runtime/code-runtime/README.i18n.yaml +++ b/packages/code-runtime/code-runtime/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/code-runtime/code-runtime/README.md -README.md: c7a2d519e47d160f5ab123bfc887e7e9f24ec602 -README.zh.md: 22d0b120d7cea50b578a184b3e40d77707ebc489 +README.md: e9641041af76b60606f999783f29224d8d79c743 +README.zh.md: cc97b6b6cf7c8c5aedb58e40d06ee6dd962ac3b2 diff --git a/packages/code-runtime/code-runtime/README.md b/packages/code-runtime/code-runtime/README.md index c7a2d519e4..e9641041af 100644 --- a/packages/code-runtime/code-runtime/README.md +++ b/packages/code-runtime/code-runtime/README.md @@ -11,7 +11,7 @@ This package is the interface third of the capability (the bash trio is the temp | Member | Semantics | |---|---| | `run(request)` | Execute one program against the request's bindings. **Resolves with an error FIELD for every program outcome** — parse/transform failure, thrown exception, invalid completion, output overflow, budget expiry, abort, or substrate death (`CodeRunFailure`'s orthogonal `kind` taxonomy); it rejects only for caller misuse of the seam itself (e.g. a run submitted after disposal). The program runs as the body of an async function: top-level `await`/`return` work, and a lossless JSON completion becomes `result.value`. | -| `language` | Readonly descriptor: the source language `run` expects (`'typescript'` is the well-known value). Informational, not gating — a consumer that generates language-specific presentation switches on it and fails loud on a language it cannot present. | +| `language` | Readonly descriptor: the source language `run` expects. `'typescript'` and `'python'` are the well-known values — those `dsh-tools` presents; only `'typescript'` has a published backend. Informational, not gating — a consumer that generates language-specific presentation switches on it and fails loud on a language it cannot present. | | `isolation` | Readonly descriptor: the execution substrate (`'worker-thread'`, `'process'`, `'container'`). A label for deployments and diagnostics, **not a security claim**. | Semantics every implementation must honor (contract details in the class JSDoc): binding calls bridge complete lossless-JSON arguments and resolutions with no seam-level byte cap; the program is treated as a hostile peer (arbitrary binding names are own properties, malformed traffic never crashes the host); no state survives between runs; disposal terminates in-flight runs AND awaits their exit before completing. diff --git a/packages/code-runtime/code-runtime/README.zh.md b/packages/code-runtime/code-runtime/README.zh.md index 22d0b120d7..cc97b6b6cf 100644 --- a/packages/code-runtime/code-runtime/README.zh.md +++ b/packages/code-runtime/code-runtime/README.zh.md @@ -11,7 +11,7 @@ | 成员 | 语义 | |---|---| | `run(request)` | 针对请求的绑定执行一段程序。**所有程序失败结果都通过 resolve 结果中的 error 字段报告**:包括解析/转换失败、抛出异常、无效完成值、输出溢出、预算到期、中止或执行基底终止(由 `CodeRunFailure` 的正交 `kind` 分类表示);只有调用方误用 seam 本身时才 reject(例如 dispose(资源释放)后仍提交运行)。程序作为异步函数的函数体运行,因此顶层 `await`/`return` 可用,无损 JSON 完成值会成为 `result.value`。 | -| `language` | 只读描述符:`run` 期望的源语言(已知值为 `'typescript'`)。仅供参考,不作门禁;生成语言专用呈现的消费方会根据该值选择分支,遇到无法呈现的语言时明确失败。 | +| `language` | 只读描述符:`run` 期望的源语言。已知值为 `'typescript'` 与 `'python'`——`dsh-tools` 能呈现的那些;其中只有 `'typescript'` 有已发布的后端。仅供参考,不作门禁;生成语言专用呈现的消费方会根据该值选择分支,遇到无法呈现的语言时明确失败。 | | `isolation` | 只读描述符:执行基底(`'worker-thread'`、`'process'`、`'container'`)。供部署与诊断使用,**不构成安全声明**。 | 每个实现都必须遵守以下语义(完整契约见类 JSDoc):绑定调用会桥接完整的无损 JSON 参数与 resolve 值,seam 层不设字节上限;程序被视为敌对对等方(任意绑定名称都会成为自有属性,格式错误的通信绝不能使宿主崩溃);不同运行之间不保留任何状态;dispose 会终止进行中的运行,并且在完成前等待其退出。 diff --git a/packages/code-runtime/code-runtime/src/index.ts b/packages/code-runtime/code-runtime/src/index.ts index bd52b9ed29..033a5f238e 100644 --- a/packages/code-runtime/code-runtime/src/index.ts +++ b/packages/code-runtime/code-runtime/src/index.ts @@ -36,7 +36,8 @@ export abstract class CodeRuntime extends Service { * lowercase identifier. Informational, not gating — a consumer that * generates language-specific presentation (typed SDK stubs, usage * instructions) switches on it and fails loud on a language it cannot - * present. Well-known value: `'typescript'`. + * present. Well-known values: `'typescript'` and `'python'`, those + * `dsh-tools` presents; only `'typescript'` has a published backend. */ abstract readonly language: string diff --git a/packages/core/tools/README.i18n.yaml b/packages/core/tools/README.i18n.yaml index 61bfcd005b..c1bd91ce2b 100644 --- a/packages/core/tools/README.i18n.yaml +++ b/packages/core/tools/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/core/tools/README.md -README.md: 80ea3cc93437d48a7ea0ffba0ff4d2ef2407755f -README.zh.md: 691d2f2fcccdaa1bcab5343b2fce661d9c99e8ad +README.md: 81cc57983d83fd19468017b217d4db9978f4e228 +README.zh.md: 9f875bd80a03d1d0f78625ee98eeaad9d118f871 diff --git a/packages/core/tools/README.md b/packages/core/tools/README.md index 80ea3cc934..81cc57983d 100644 --- a/packages/core/tools/README.md +++ b/packages/core/tools/README.md @@ -13,7 +13,7 @@ tools: mode: native # native (default) | code | both ``` -`native` contributes visible tools as function definitions. `code` contributes the reserved `run_code` transport and generated `tools:sdk` section; `both` contributes both forms. The reserved transport cannot be registered, shadowed, restricted, or removed. Non-native modes require a TypeScript `ctx.codeRuntime`, and a `systemPrompt.toolOrder` entry for a tool the mode does not contribute rejects prompt assembly. A `system-prompt/assemble` listener may replace the registry's contributions; its returned assembly is authoritative, so that listener owns preserving a usable Code Mode protocol. +`native` contributes visible tools as function definitions. `code` contributes the reserved `run_code` transport and generated `tools:sdk` section; `both` contributes both forms. The reserved transport cannot be registered, shadowed, restricted, or removed. Non-native modes require a `ctx.codeRuntime` whose `language` has a registered SDK renderer — TypeScript ships via [`dsh-code-runtime-worker`](../../code-runtime/code-runtime-worker/README.md); a Python renderer is built in and drives any runtime that reports `language: 'python'` (a first-party `dsh-code-runtime-python` backend is delivered separately). A runtime language with no renderer fails prompt assembly loudly, and a `systemPrompt.toolOrder` entry for a tool the mode does not contribute rejects prompt assembly. A `system-prompt/assemble` listener may replace the registry's contributions; its returned assembly is authoritative, so that listener owns preserving a usable Code Mode protocol. ### Public API @@ -114,9 +114,9 @@ Returning `undefined` selects generic fallback. Presenters depend only on their ### Code Mode -Under `code` or `both`, the registry exposes the reserved `run_code` transport and a deterministic TypeScript SDK for the current scope; only the program's outer logs and return value re-enter model context. The SDK declares exact `ToolArgsMap` and `ToolOutputMap` entries for every visible tool, and each binding resolves to the tool's canonical JSON value. Each lossless-JSON binding call re-enters the complete tool pipeline under the native scheduling contract (concurrency-safe calls may overlap up to `maxParallelSubCalls`; exclusive calls run alone as ordering barriers) with logged correlation to the outer call. Denials and other failed results reject with the real program-visible `ToolCallError` carrying only `toolName` and `message`; Native content and internal error codes stay outside the Code contract. Ordinary side effects are not rolled back, and sub-call `additionalContexts` are deferred through the parent result to preserve call/result adjacency. Run settlement aborts and drains outstanding bindings; runtime failures surface as `CodeRunFailedError`. See the [Code Mode foundation](../../../.agents/notes/implemented/feature/2026-06-15-code-mode.md), [typed-return contract](../../../.agents/notes/implemented/feature/2026-07-20-code-mode-typed-tool-returns.md), and [code-runtime seam](../../code-runtime/README.md). Try `pnpm run demo:code-mode`. +Under `code` or `both`, the registry exposes the reserved `run_code` transport and a deterministic SDK for the current scope, generated in the loaded runtime's language — the registry selects the renderer by `ctx.codeRuntime.language` (`typescript` → the TypeScript SDK below, `python` → the Python SDK). Only the program's outer logs and return value re-enter model context. The SDK declares exact per-tool argument and canonical-output types for every visible tool (`ToolArgsMap`/`ToolOutputMap` in TypeScript, named `TypedDict`s in Python), and each binding resolves to the tool's canonical JSON value. Each lossless-JSON binding call re-enters the complete tool pipeline under the native scheduling contract (concurrency-safe calls may overlap up to `maxParallelSubCalls`; exclusive calls run alone as ordering barriers) with logged correlation to the outer call. Denials and other failed results reject with the real program-visible `ToolCallError` carrying only `toolName` and `message`; Native content and internal error codes stay outside the Code contract. Ordinary side effects are not rolled back, and sub-call `additionalContexts` are deferred through the parent result to preserve call/result adjacency. Run settlement aborts and drains outstanding bindings; runtime failures surface as `CodeRunFailedError`. See the [Code Mode foundation](../../../.agents/notes/implemented/feature/2026-06-15-code-mode.md), [typed-return contract](../../../.agents/notes/implemented/feature/2026-07-20-code-mode-typed-tool-returns.md), and [code-runtime seam](../../code-runtime/README.md). Try `pnpm run demo:code-mode`. -- **The SDK section** (`tools:sdk`, order 150): a lazy prompt section regenerating, at each assembly, `JsonValue`, exact `ToolArgsMap` / `ToolOutputMap`, `ToolName`, the `ToolCallError` declaration, and a mapped `tools` namespace for the calling scope's visible end capabilities (exotic names via quoted keys), plus fixed usage instructions. Deterministic — lexicographic tool order, byte-identical text for an unchanged tool set (prefix-cache-friendly). The codegen (`jsonSchemaToTs`, exported) handles every unified schema construct and degrades unsupported raw constructs to `unknown`, never throwing during prompt assembly. +- **The SDK section** (`tools:sdk`, order 150): a lazy prompt section regenerating the language-appropriate SDK text at each assembly. In the TypeScript flavor it emits `JsonValue`, exact `ToolArgsMap` / `ToolOutputMap`, `ToolName`, the `ToolCallError` declaration, and a mapped `tools` namespace for the calling scope's visible end capabilities (exotic names via quoted keys), plus fixed usage instructions; the Python flavor (`ctx.codeRuntime.language === 'python'`) emits the equivalent named `TypedDict`s and a `tools` object with matching usage instructions. Deterministic — lexicographic tool order, byte-identical text for an unchanged tool set (prefix-cache-friendly). Both codegens are exported and never throw during prompt assembly: `jsonSchemaToTs` handles every unified schema construct and degrades unsupported raw constructs to `unknown`; `jsonSchemaToPy` does the same, degrading to `Any` (and a whole object to `dict[str, Any]` when a field name is not a legal `TypedDict` attribute, or whenever it is called outside the SDK render, which supplies the naming context a `TypedDict` declaration needs). - **The dispatch bridge** (`run_code`'s execute): every binding call is snapshotted as lossless JSON before dispatch (`undefined`, `BigInt`, cycles, sparse arrays, `-0`, and exotic objects reject that one call), scheduled through a per-run pool that reuses the native concurrency contract — calls start strictly in submission order, consecutive `isConcurrencySafe` calls overlap up to the validated `maxParallelSubCalls` config (default 10; `1` restores serial dispatch), and an exclusive-classified call drains the pool, runs alone, and bars later calls — given the outer execution's opaque token as `parent`, and run through the complete pre-execute → guards → execute → post-execute → result pipeline. A success returns the final canonical value after policy; a failure reaches the worker as one message and becomes `ToolCallError(toolName, message)`. Each started sub-call logs a `tool/code-dispatch-start` event (deterministic id `:code:`, numbered by submission) at pipeline entry and settles with one `tool/code-dispatch` event carrying the complete model-facing `content`/`isError` outcome (the `tool/result` vocabulary, so UIs render sub-calls through the native path — the pair's `time` fields carry per-sub-call timing); a queued call abandoned by run settlement logs neither. `deriveMessages()` surfaces neither event nor persists the canonical value. Token correlation lets commit-style observers defer an inner success until the final `run_code` result without exposing the live outer execution; ordinary tool side effects are not rolled back. Every sub-call `additionalContexts` entry is deferred through the outer `ToolRunContext` in dispatch order; the loop appends those contexts only after the parent `run_code` result, preserving adjacency and retaining each source/meta even when the program later fails. - **Settlement discipline**: the bridge owns a run-scoped abort that follows the outer signal in and fires when the run settles for any reason, so a budget expiry aborts an in-flight sub-tool instead of orphaning it; the bridge then drains its queue BEFORE returning, so every `tool/code-dispatch` lands inside the open turn. A failed run throws `CodeRunFailedError` (`code: 'CODE_RUN_FAILED'`, message = the failure kind + captured logs), which the pipeline converts to a structured `isError` the model self-corrects from. - **Result boundary**: intermediate binding values cross the worker boundary whole and have no per-binding byte cap. `run_code` returns canonical `{ logs: string[], result?: JsonValue }`; strings render raw, every other present JSON root renders through a stack-safe pretty JSON traversal whose total indentation is capped at ten characters (deeper subtrees stay compact), `null` remains explicit, and absent `result` means the program returned `undefined`. The worker's configurable `maxOutputBytes` (default 64 MiB) applies only to the combined serialized outer log-array, completion-value, or failure-message payloads; fixed result-envelope syntax and presentation whitespace are outside that ledger. Invalid and over-limit completions fail explicitly, and only this outer result is eligible for ordinary spill. @@ -145,7 +145,7 @@ Prefix-stable while visible definitions and their order are unchanged. Registrat #### What the model sees -Code Mode exposes the generated [`run_code` schema](../../../docs/tool-catalog.md#deepseek-aidsh-tools), the SDK instructions below, and the generated exact `declare const tools` block. `both` exposes normal schemas and this Code Mode surface. +Code Mode exposes the generated [`run_code` schema](../../../docs/tool-catalog.md#deepseek-aidsh-tools), the SDK instructions below, and the generated exact SDK block for the loaded runtime's language (the TypeScript `declare const tools` block, or the Python `tools` declaration). `both` exposes normal schemas and this Code Mode surface. The instructions and SDK block match the loaded runtime's language; the TypeScript flavor (via [`dsh-code-runtime-worker`](../../code-runtime/code-runtime-worker/README.md)) is shown below, and the Python flavor (for any runtime reporting `language: 'python'`) is the same shape with Python syntax (`await tools.name(args)`, subscript access for exotic names, `print(...)` and top-level `return`). ##### Code Mode SDK instructions @@ -190,6 +190,6 @@ Append-only; newly visible content follows the reusable request prefix and does - **`tools/pre-execute` deliberately cannot rewrite `exec.arguments`** — logged and rendered args would desync from what ran; the rewrite design is [a proposed Agent Note](../../../.agents/notes/proposed/feature/2026-06-30-pre-tool-input-rewrite.md). - **Caller-defined subagent and workflow structured outputs remain object-rooted** — this is a consumer-level guard; the shared schema vocabulary and tool outputs support every JSON root. - **`timeoutMs` on a definition is declarative only** — the registry never enforces deadlines; enforcement requires the `@deepseek-ai/dsh-timeout-policy` wrapper. -- **Code Mode is TypeScript-only and the presentation mode is service-wide** — `mode: code`/`both` rejects prompt assembly unless `ctx.codeRuntime.language === 'typescript'`; scoped restrictions/shadows still choose each agent's visible bindings, but one tool cannot be native-only while another is code-only. +- **Code Mode's SDK language follows the one loaded runtime and the presentation mode is service-wide** — `mode: code`/`both` rejects prompt assembly unless `ctx.codeRuntime.language` has a registered SDK renderer (`typescript` via the worker backend, `python` for any runtime reporting that language); scoped restrictions/shadows still choose each agent's visible bindings, but one tool cannot be native-only while another is code-only, and a single runtime fixes the language service-wide (the [language-dispatch Agent Note](../../../.agents/notes/implemented/feature/2026-07-31-code-mode-language-dispatch.md) owns the lookup, and why the registry reads the loaded runtime instead of carrying a language field of its own). - **Code Mode intermediate values are execution-local and unbounded by bytes** — the canonical typed values cannot be reconstructed from session replay and may exhaust process or worker memory; only the outer `run_code` output has the worker's configurable hard cap. The durable log copy of each sub-call IS bounded: the `tools/code-dispatch-log` waterfall lets the spill policy replace an oversized `tool/code-dispatch` content with a preview + locator ([rationale](../../../.agents/notes/implemented/feature/2026-07-26-code-dispatch-log-spill.md)). - **`run_code` state is fresh per run** — a persistent REPL-style kernel is rejected for the MVP (cross-call state would be invisible to the log); see [the Code Mode Agent Note](../../../.agents/notes/implemented/feature/2026-06-15-code-mode.md). diff --git a/packages/core/tools/README.zh.md b/packages/core/tools/README.zh.md index 691d2f2fcc..9f875bd80a 100644 --- a/packages/core/tools/README.zh.md +++ b/packages/core/tools/README.zh.md @@ -13,7 +13,7 @@ tools: mode: native # native (default) | code | both ``` -`native` 以函数定义的形式贡献可见工具。`code` 贡献保留的 `run_code` 传输和生成的 `tools:sdk` 段;`both` 同时贡献两种形式。不能注册、遮蔽、限制或移除该保留传输。非原生模式要求存在 TypeScript `ctx.codeRuntime`;如果 `systemPrompt.toolOrder` 条目指向当前模式未贡献的工具,系统会拒绝组装提示词。`system-prompt/assemble` 监听器可以替换注册表贡献;它返回的组装结果具有权威性,因此该监听器负责保留可用的 Code Mode 协议。 +`native` 以函数定义的形式贡献可见工具。`code` 贡献保留的 `run_code` 传输和生成的 `tools:sdk` 段;`both` 同时贡献两种形式。不能注册、遮蔽、限制或移除该保留传输。非原生模式要求所加载 `ctx.codeRuntime` 的 `language` 有已注册的 SDK 渲染器——TypeScript 经 [`dsh-code-runtime-worker`](../../code-runtime/code-runtime-worker/README.md) 交付;Python 渲染器内置,驱动任何报告 `language: 'python'` 的运行时(第一方 `dsh-code-runtime-python` 后端另行交付)。没有渲染器的运行时语言会让提示词组装响亮失败;如果 `systemPrompt.toolOrder` 条目指向当前模式未贡献的工具,系统会拒绝组装提示词。`system-prompt/assemble` 监听器可以替换注册表贡献;它返回的组装结果具有权威性,因此该监听器负责保留可用的 Code Mode 协议。 ### 公开 API @@ -114,9 +114,9 @@ ctx.tools.register(defineTool({ ### Code Mode -在 `code` 或 `both` 模式下,注册表为当前作用域公开保留的 `run_code` 传输和确定性的 TypeScript SDK;只有程序的外层日志与返回值会重新进入模型上下文。SDK 为每个可见工具声明精确的 `ToolArgsMap` 和 `ToolOutputMap` 条目,每个绑定都会解析为该工具的规范 JSON 值。每个无损 JSON 绑定调用都会在原生调度契约下重新进入完整工具流水线(并发安全的调用最多可重叠 `maxParallelSubCalls` 个;独占调用单独运行并构成排序屏障),并在日志中与外层调用建立关联。拒绝及其他失败结果会以程序实际可见的 `ToolCallError` 形式拒绝,且只携带 `toolName` 和 `message`;Native 内容和内部错误码留在 Code 契约之外。普通副作用不会回滚,子调用的 `additionalContexts` 会通过父结果延迟,以保持调用/结果相邻。运行结算会中止并排空尚未完成的绑定;运行时失败以 `CodeRunFailedError` 形式出现。参见 [Code Mode 基础](../../../.agents/notes/implemented/feature/2026-06-15-code-mode.md)、[类型化返回契约](../../../.agents/notes/implemented/feature/2026-07-20-code-mode-typed-tool-returns.md)和[代码运行时 seam](../../code-runtime/README.md)。可以运行 `pnpm run demo:code-mode` 试用。 +在 `code` 或 `both` 模式下,注册表为当前作用域公开保留的 `run_code` 传输和按所加载运行时语言生成的确定性 SDK——注册表按 `ctx.codeRuntime.language` 选择渲染器(`typescript` → 下方的 TypeScript SDK,`python` → Python SDK)。只有程序的外层日志与返回值会重新进入模型上下文。SDK 为每个可见工具声明精确的参数与规范输出类型(TypeScript 为 `ToolArgsMap`/`ToolOutputMap`,Python 为具名 `TypedDict`),每个绑定都会解析为该工具的规范 JSON 值。每个无损 JSON 绑定调用都会在原生调度契约下重新进入完整工具流水线(并发安全的调用最多可重叠 `maxParallelSubCalls` 个;独占调用单独运行并构成排序屏障),并在日志中与外层调用建立关联。拒绝及其他失败结果会以程序实际可见的 `ToolCallError` 形式拒绝,且只携带 `toolName` 和 `message`;Native 内容和内部错误码留在 Code 契约之外。普通副作用不会回滚,子调用的 `additionalContexts` 会通过父结果延迟,以保持调用/结果相邻。运行结算会中止并排空尚未完成的绑定;运行时失败以 `CodeRunFailedError` 形式出现。参见 [Code Mode 基础](../../../.agents/notes/implemented/feature/2026-06-15-code-mode.md)、[类型化返回契约](../../../.agents/notes/implemented/feature/2026-07-20-code-mode-typed-tool-returns.md)和[代码运行时 seam](../../code-runtime/README.md)。可以运行 `pnpm run demo:code-mode` 试用。 -- **SDK 段**(`tools:sdk`,顺序 150):一个惰性提示词段,每次组装时都会重新生成 `JsonValue`、精确的 `ToolArgsMap` / `ToolOutputMap`、`ToolName`、`ToolCallError` 声明、面向调用作用域可见最终能力的映射 `tools` 命名空间(特殊名称使用带引号的键),以及固定用法说明。其输出具有确定性:工具按字典序排列;工具集合不变时,文本逐字节相同(有利于前缀 cache)。导出的代码生成器 `jsonSchemaToTs` 会处理统一 schema 的每种构造,并将不受支持的原始构造降级为 `unknown`,绝不会在提示词组装期间抛出。 +- **SDK 段**(`tools:sdk`,顺序 150):一个惰性提示词段,每次组装时都会重新生成与所加载运行时语言相符的 SDK 文本。TypeScript 形态发出 `JsonValue`、精确的 `ToolArgsMap` / `ToolOutputMap`、`ToolName`、`ToolCallError` 声明、面向调用作用域可见最终能力的映射 `tools` 命名空间(特殊名称使用带引号的键),以及固定用法说明;Python 形态(`ctx.codeRuntime.language === 'python'`)发出等价的具名 `TypedDict` 与一个带相同用法说明的 `tools` 对象。其输出具有确定性:工具按字典序排列;工具集合不变时,文本逐字节相同(有利于前缀 cache)。两个代码生成器都已导出,且绝不会在提示词组装期间抛出:`jsonSchemaToTs` 处理统一 schema 的每种构造并将不受支持的原始构造降级为 `unknown`;`jsonSchemaToPy` 同理,降级为 `Any`(当某字段名不是合法的 `TypedDict` 属性时,或在 SDK 渲染之外被调用时——`TypedDict` 声明所需的命名上下文由该渲染提供——整个对象降级为 `dict[str, Any]`)。 - **分发桥接层**(`run_code` 的 execute):每个绑定调用都会在分发前快照为无损 JSON(`undefined`、`BigInt`、循环、稀疏数组、`-0` 和特殊对象会使该次调用被拒绝),经由每次运行独有、复用原生并发契约的池调度——调用严格按提交顺序启动,连续的 `isConcurrencySafe` 调用最多可重叠经校验的 `maxParallelSubCalls` 配置个(默认 10;设为 `1` 即恢复串行分发),被分类为独占的调用先排空池、单独运行并阻挡其后的调用——以外层执行的不透明 token 作为 `parent`,并经过完整的 pre-execute → guards → execute → post-execute → result 流水线。成功会返回策略处理后的最终规范值;失败以一条消息到达 worker,并成为 `ToolCallError(toolName, message)`。每个已启动的子调用在进入流水线时记录一条 `tool/code-dispatch-start` 事件(确定性 id `:code:`,按提交顺序编号),并以一条携带完整模型可见 `content`/`isError` 结果的 `tool/code-dispatch` 事件完结(采用 `tool/result` 词汇,因此 UI 会沿原生路径呈现子调用——这对事件的 `time` 字段承载每个子调用的计时);因 run 结算而被放弃的排队调用两者都不记录。`deriveMessages()` 既不公开这两个事件,也不持久化规范值。token 关联让以提交为语义的观察器能够把内部成功延迟到最终 `run_code` 结果,而无需公开实时外层执行;普通工具副作用不会回滚。每个子调用的 `additionalContexts` 条目都会按分发顺序通过外层 `ToolRunContext` 延迟;循环只在父级 `run_code` 结果之后追加这些上下文,从而保持相邻关系,并且即使程序后来失败,也会保留各自的来源/元数据。 - **结算纪律**:桥接层拥有一个运行作用域的中止机制;该中止会跟随传入的外层信号,并在运行因任何原因结算时触发,因此预算耗尽会中止正在运行的子工具,而不会将其遗留。桥接层随后会在返回之前排空队列,使每个 `tool/code-dispatch` 都落在仍打开的轮次内。失败的运行会抛出 `CodeRunFailedError`(`code: 'CODE_RUN_FAILED'`,message = 失败类型 + 已捕获日志),流水线会将其转换为模型可据以自我修正的结构化 `isError`。 - **结果边界**:中间绑定值会完整跨越 worker 边界,且没有逐绑定字节上限。`run_code` 返回规范的 `{ logs: string[], result?: JsonValue }`;字符串原样呈现,其他所有存在的 JSON 根都通过栈安全的美化 JSON 遍历呈现,总缩进最多为 10 个字符(更深的子树保持紧凑),`null` 保持显式,而缺少 `result` 表示程序返回 `undefined`。worker 可配置的 `maxOutputBytes`(默认 64 MiB)只应用于组合序列化后的外层日志数组、完成值或失败消息载荷;固定的结果 envelope 语法和呈现空白不计入该账本。无效和超限的完成会明确失败,只有此外层结果可以使用普通 spill。 @@ -145,7 +145,7 @@ agent loop 将连续的 `parallel` 调用归入有界滚动池,并把每个 `e #### 模型看到的内容 -Code Mode 会公开生成的 [`run_code` schema](../../../docs/tool-catalog.md#deepseek-aidsh-tools)、下方 SDK 说明,以及生成的精确 `declare const tools` 块。`both` 会同时公开普通 schema 与此 Code Mode 接口。 +Code Mode 会公开生成的 [`run_code` schema](../../../docs/tool-catalog.md#deepseek-aidsh-tools)、下方 SDK 说明,以及按所加载运行时语言生成的精确 SDK 块(TypeScript 的 `declare const tools` 块,或 Python 的 `tools` 声明)。`both` 会同时公开普通 schema 与此 Code Mode 接口。说明与 SDK 块随所加载运行时的语言切换;下方展示 TypeScript 风格(经 [`dsh-code-runtime-worker`](../../code-runtime/code-runtime-worker/README.md)),Python 风格(用于任何报告 `language: 'python'` 的运行时)形状相同,只是换成 Python 语法(`await tools.name(args)`、特殊名称用下标访问、`print(...)` 与顶层 `return`)。 ##### Code Mode SDK 说明 @@ -190,6 +190,6 @@ The available tools: - **`tools/pre-execute` 有意不允许改写 `exec.arguments`**:否则日志记录和呈现的参数会与实际运行内容失去同步;改写设计记录在[拟议的 Agent Note](../../../.agents/notes/proposed/feature/2026-06-30-pre-tool-input-rewrite.md)中。 - **调用方定义的 subagent 与工作流结构化输出仍要求对象根**:这是消费方层面的守卫;共享 schema 词汇和工具输出支持任意 JSON 根。 - **定义上的 `timeoutMs` 仅为声明**:注册表绝不会强制执行截止时间;要强制执行,必须使用 `@deepseek-ai/dsh-timeout-policy` 包装层。 -- **Code Mode 只支持 TypeScript,且呈现模式在服务内统一**:`mode: code`/`both` 会拒绝组装提示词,除非 `ctx.codeRuntime.language === 'typescript'`;作用域限制/遮蔽仍会选择每个 agent 的可见绑定,但不能让一个工具仅使用 Native,而另一个仅使用 Code。 +- **Code Mode 的 SDK 语言跟随唯一加载的运行时,且呈现模式在服务内统一**:`mode: code`/`both` 会拒绝组装提示词,除非 `ctx.codeRuntime.language` 有已注册的 SDK 渲染器(`typescript` 经 worker 后端,`python` 用于任何报告该语言的运行时);作用域限制/遮蔽仍会选择每个 agent 的可见绑定,但不能让一个工具仅使用 Native、另一个仅使用 Code,且单个运行时把语言固定为服务级([语言分发 Agent Note](../../../.agents/notes/implemented/feature/2026-07-31-code-mode-language-dispatch.md) 负责这次查表,以及注册表为何读取所加载的运行时而不自带 language 字段)。 - **Code Mode 中间值只存在于执行局部,且没有字节上限**:这些规范的类型化值无法从会话回放重建,并可能耗尽进程或 worker 内存;只有外层 `run_code` 输出受 worker 可配置的硬上限约束。每个子调用的持久日志副本则确实有上限:`tools/code-dispatch-log` waterfall 允许 spill 策略把过大的 `tool/code-dispatch` 内容替换为预览加定位符([原理](../../../.agents/notes/implemented/feature/2026-07-26-code-dispatch-log-spill.md))。 - **每次运行都会获得全新的 `run_code` 状态**:MVP 不采用持久 REPL 风格内核(跨调用状态不会出现在日志中);参见 [Code Mode Agent Note](../../../.agents/notes/implemented/feature/2026-06-15-code-mode.md)。 diff --git a/packages/core/tools/src/code-mode.ts b/packages/core/tools/src/code-mode.ts index 244934de3f..aa4a1f027a 100644 --- a/packages/core/tools/src/code-mode.ts +++ b/packages/core/tools/src/code-mode.ts @@ -11,7 +11,7 @@ import type { ContentBlock } from '@deepseek-ai/dsh-llm' import type { CodeBindingFunction, CodeRunResult, CodeRuntime } from '@deepseek-ai/dsh-code-runtime' import { snapshotJsonValue } from '@deepseek-ai/dsh-session' import type { JsonValue } from '@deepseek-ai/dsh-session' -import { defineTool } from './schema.ts' +import { defineTool, parameterSchemaSpecToJsonSchema } from './schema.ts' import { TOOL_REGISTRY_SCHEDULER } from './index.ts' import type { CodeDispatchLog, ToolDefinition, ToolExecutionResult, ToolRegistry, ToolRunContext } from './index.ts' @@ -56,6 +56,111 @@ export const RUN_CODE_NAME = 'run_code' /** The `tools:sdk` section order: inside the 100–199 tool-guidance band, after per-tool guidance sections. */ export const SDK_SECTION_ORDER = 150 +/** + * The language-specific `run_code` schema text: the tool `description` and its + * `code` parameter description, kept together so a language's two model-facing + * strings share one source of truth. Keyed by `CodeRuntime.language`, mirroring + * `SDK_RENDERERS` in {@link ./index.ts}. The emitted flavor MUST match the + * semantics the same language's SDK instructions promise, so the model never + * receives a TypeScript-shaped schema beside a Python SDK (or vice versa). + */ +interface RunCodeFlavor { + /** The tool `description` the model sees for this language. */ + readonly description: string + /** The `code` parameter's description for this language. */ + readonly codeDescription: string +} + +/** + * The TypeScript flavor: the historical default, and the fallback for a schema + * read with no runtime mounted ({@link resolveFlavor} owns which readers reach + * that). A real assembly always resolves a runtime first, so the model never + * sees this fallback outside its own language. + */ +const TYPESCRIPT_FLAVOR: RunCodeFlavor = { + description: + 'Execute a TypeScript program against the available tools. Write the BODY of an ' + + 'async function (erasable syntax only; top-level `await` and `return` work) and ' + + 'call tools as `await tools.name(args)` per the declarations in the system prompt. ' + + 'Only what you print or return comes back — curate it.', + codeDescription: 'The program: the body of an async TypeScript function.', +} + +/** + * The Python flavor: the body of an async function, top-level `await` and + * `return`, answer via `print` and/or the returned value, matching + * {@link ./py-types.ts}'s SDK instructions. + */ +const PYTHON_FLAVOR: RunCodeFlavor = { + description: + 'Execute a Python program against the available tools. Write the BODY of an ' + + 'async function (top-level `await` and `return` work) and call tools as ' + + '`await tools.name(args)` per the declarations in the system prompt. Answer ' + + 'with `print(...)` and/or `return ` — only that comes back, so curate it.', + codeDescription: 'The program: the body of an async Python function.', +} + +/** + * The languages Code Mode ships a presentation for. Both per-language tables — + * {@link RUN_CODE_FLAVORS} here and `SDK_RENDERERS` in {@link ./index.ts} — are + * checked against this union with `satisfies`, so a language added to one and + * not the other fails `typecheck` instead of waiting for a runtime that reports + * it. The tables stay declared `Record` because `CodeRuntime.language` + * is an unconstrained `string`: this union pins what the harness ships, while the + * `Object.hasOwn` guards reject what a mounted runtime may report. + */ +export type CodeSdkLanguage = 'typescript' | 'python' + +/** Per-language `run_code` schema flavors (see {@link RunCodeFlavor}); one entry per {@link CodeSdkLanguage}. */ +const RUN_CODE_FLAVORS: Record = { + typescript: TYPESCRIPT_FLAVOR, + python: PYTHON_FLAVOR, +} satisfies Record + +/** + * The `description` parameter's model-facing description: language-independent + * (the UI label contract is the same for every runtime), shared between the + * static spec and the language-aware `parameters` getter so the two emissions + * can never drift. + */ +const RUN_CODE_DESCRIPTION_PARAM_DESCRIPTION + = 'Clear, concise description of what this program does in active voice, ' + + '5-10 words (shown in the UI). Examples: "Count TODO markers across packages"; ' + + '"Read failing test and its fixture"; "Rename config key in every cordis.yml".' + +/** + * Resolve the {@link RunCodeFlavor} for the loaded runtime's language, read at + * schema-emission time so the model-visible `run_code` schema always matches + * the SDK section's language. `peekRuntime` returns `undefined` only when no + * runtime is mounted, which reaches this function through definition readers + * and `schemas()` — the doc-catalog harvest is the only shipped one, and none + * of them feeds a model, because `wireSchemas` calls `requireCodeRuntime` + * before projecting — so that path degrades to {@link TYPESCRIPT_FLAVOR}. A + * mounted runtime whose language has no flavor entry fails loud, exactly as + * `requireCodeRuntime` rejects it at assembly. Keeping this table in step with + * `SDK_RENDERERS` is the compiler's job ({@link CodeSdkLanguage}); what this + * guard owns is the runtime-supplied language neither table knows, which never + * yields a wrong-language schema for a real runtime. + */ +function resolveFlavor(peekRuntime: () => CodeRuntime | undefined): RunCodeFlavor { + const runtime = peekRuntime() + if (runtime === undefined) { + // No runtime mounted: reached by definition readers and `schemas()`, of + // which the doc-catalog harvest is the only shipped one. None feeds a + // model — `wireSchemas` calls `requireCodeRuntime` before projecting, so + // the assembly path never arrives here. Degrade to the TS default. + return TYPESCRIPT_FLAVOR + } + // Own-property read: a language like `toString`/`constructor` would otherwise + // resolve an inherited Object.prototype member as a flavor. + const flavor = RUN_CODE_FLAVORS[runtime.language] + if (!Object.hasOwn(RUN_CODE_FLAVORS, runtime.language) || flavor === undefined) { + const known = Object.keys(RUN_CODE_FLAVORS).map(name => JSON.stringify(name)).join(', ') + throw new Error(`dsh-tools: no run_code schema flavor registered for runtime language ${JSON.stringify(runtime.language)} (known: ${known})`) + } + return flavor +} + /** * Thrown by `run_code` when the program run itself failed — a program * exception, a budget expiry, an abort, or substrate death. Extends @@ -194,6 +299,13 @@ type RunCodeOutput = { logs: string[]; result?: JsonValue } export interface RunCodeBridgeOptions { /** Resolves `ctx.codeRuntime` or throws the loud misconfiguration error (shared with the registry's assembly-time checks). */ requireRuntime: () => CodeRuntime + /** + * Reads `ctx.codeRuntime` without throwing: `undefined` when none is mounted. + * Lets schema emission tell "no runtime" (degrade to TS; the readers that + * reach it are {@link resolveFlavor}'s) apart from "unknown language" (fail + * loud). + */ + peekRuntime: () => CodeRuntime | undefined /** The run's overlap cap for parallel-classified sub-calls (the registry passes its validated `maxParallelSubCalls`). */ maxParallel: number /** Runs the contained `tools/code-dispatch-log` waterfall over one settled sub-dispatch (the registry's private invoker). */ @@ -212,22 +324,22 @@ export interface RunCodeBridgeOptions { * @returns the registry-ready definition. */ export function createRunCodeTool(registry: ToolRegistry, options: RunCodeBridgeOptions): ToolDefinition { - const { requireRuntime, maxParallel, shapeDispatchLog } = options - return defineTool({ + const { requireRuntime, peekRuntime, maxParallel, shapeDispatchLog } = options + const definition = defineTool({ name: RUN_CODE_NAME, - description: - 'Execute a TypeScript program against the available tools. Write the BODY of an ' - + 'async function (erasable syntax only; top-level `await` and `return` work) and ' - + 'call tools as `await tools.name(args)` per the declarations in the system prompt. ' - + 'Only what you print or return comes back — curate it.', + // The description and `code` parameter description are placeholders here: + // the language-aware getters installed below replace both, resolving the + // loaded runtime's flavor at schema-emission time so the schema the MODEL + // sees matches the SDK section's language. Argument VALIDATION still keys + // off this static spec (defineTool closes over it), which is language- + // independent (one required string `code`). + description: TYPESCRIPT_FLAVOR.description, parameters: { - code: { type: 'string', required: true, description: 'The program: the body of an async TypeScript function.' }, + code: { type: 'string', required: true, description: TYPESCRIPT_FLAVOR.codeDescription }, description: { type: 'string', required: true, - description: 'Clear, concise description of what this program does in active voice, ' - + '5-10 words (shown in the UI). Examples: "Count TODO markers across packages"; ' - + '"Read failing test and its fixture"; "Rename config key in every cordis.yml".', + description: RUN_CODE_DESCRIPTION_PARAM_DESCRIPTION, }, }, output: { @@ -569,4 +681,22 @@ export function createRunCodeTool(registry: ToolRegistry, options: RunCodeBridge // title and reads durable result content without duplicating a large raw // result into the host view payload. }) + // Resolve the language flavor lazily, at the moment the registry projects the + // schema (`schemaOf` destructures `description`/`parameters`). The definition + // is minted once at registration, before a runtime is known; deferring here + // is the least invasive point that still emits the loaded runtime's language. + Object.defineProperty(definition, 'description', { + enumerable: true, + get: () => resolveFlavor(peekRuntime).description, + }) + Object.defineProperty(definition, 'parameters', { + enumerable: true, + // Recompile through the same spec→schema projection defineTool used, so + // the emitted shape can never drift from the validated one. + get: () => parameterSchemaSpecToJsonSchema({ + code: { type: 'string', required: true, description: resolveFlavor(peekRuntime).codeDescription }, + description: { type: 'string', required: true, description: RUN_CODE_DESCRIPTION_PARAM_DESCRIPTION }, + }) as unknown as Record, + }) + return definition } diff --git a/packages/core/tools/src/index.ts b/packages/core/tools/src/index.ts index 72254e2abd..b49350c1a3 100644 --- a/packages/core/tools/src/index.ts +++ b/packages/core/tools/src/index.ts @@ -22,8 +22,31 @@ import type { ToolCallView, ToolResultView } from './presentation.ts' import { assertSupportedJsonSchema, validateJsonSchemaValue } from './json-schema.ts' import type { JsonSchemaNode } from './json-schema.ts' import { createRunCodeTool, RUN_CODE_NAME, SDK_SECTION_ORDER } from './code-mode.ts' +import type { CodeSdkLanguage } from './code-mode.ts' import { renderToolsSdk } from './ts-types.ts' import type { ToolSdkSchema } from './ts-types.ts' +import { renderToolsSdkPy } from './py-types.ts' + +/** + * Language → SDK-section renderer. The registry looks up the loaded + * `ctx.codeRuntime.language` in this table when assembling the `tools:sdk` + * section under a non-native mode; a runtime whose language is not a key + * fails the assembly loudly (same idiom as `toolOrder` violations). Adding a + * new backend language is three parallel edits — a {@link CodeSdkLanguage} + * member, an entry here, and a `RUN_CODE_FLAVORS` entry in `code-mode.ts` for + * its `run_code` schema strings — plus the renderer function this table points + * at. The `satisfies` clause pins this table's key set to that union, which + * the flavor table is checked against too, so any of the three left out is a + * typecheck failure. What no check reaches is the prose that names the values + * instead of deriving them: the seam's `dsh-code-runtime` README pair, its + * `CodeRuntime.language` JSDoc, and `docs/core-data-structures/code-runtime.md` + * with its zh pair, plus this package's own README pair and the + * {@link Config.mode} JSDoc. + */ +const SDK_RENDERERS: Record string> = { + typescript: renderToolsSdk, + python: renderToolsSdkPy, +} satisfies Record string> export { defineTool, @@ -65,6 +88,7 @@ export type { JsonValue } from '@deepseek-ai/dsh-session' export { CodeRunFailedError, RUN_CODE_NAME } from './code-mode.ts' export { jsonSchemaToTs, renderToolsSdk } from './ts-types.ts' +export { jsonSchemaToPy, renderToolsSdkPy } from './py-types.ts' export { defineContentToolFixture, type ContentToolFixtureOptions } from './testing.ts' // The render-intent vocabulary a tool declares via `presentCall`/`presentResult` @@ -593,8 +617,9 @@ export interface Config { /** * Model presentation. `native` (default) sends every visible schema; `code` * sends only `run_code` plus a generated SDK prompt; `both` sends both forms. - * Code modes require a TypeScript runtime and fail prompt assembly when it is - * absent or mismatched. Under `code`, native names in `toolOrder` are invalid. + * Code modes require a `ctx.codeRuntime` whose `language` has a registered + * SDK renderer (TypeScript or Python) and fail prompt assembly when it is + * absent or has no renderer. Under `code`, native names in `toolOrder` are invalid. */ mode?: ToolPresentationMode /** @@ -757,6 +782,7 @@ export class ToolRegistry extends Service { ? undefined : createRunCodeTool(this, { requireRuntime: () => this.requireCodeRuntime(), + peekRuntime: () => this.ctx.get('codeRuntime'), maxParallel: resolveMaxParallelSubCalls(config.maxParallelSubCalls), shapeDispatchLog: dispatch => this.shapeDispatchLog(dispatch), }) @@ -765,10 +791,21 @@ export class ToolRegistry extends Service { ctx.systemPrompt.section({ name: 'tools:sdk', order: SDK_SECTION_ORDER, - // Regenerate from the calling scope's visible tools in stable order. + // Regenerate from the calling scope's visible tools in stable order, + // picking the renderer that matches the loaded runtime's language. + // `requireCodeRuntime` already validated the language is in the table, + // so the guard below is defense-in-depth against a caller that bypassed + // it (impossible under normal composition). text: (context) => { - this.requireCodeRuntime() - return renderToolsSdk(this.sdkSchemas(context.scope)) + const runtime = this.requireCodeRuntime() + // Own-property read: a language like `toString`/`constructor` would + // otherwise resolve an inherited Object.prototype member as a renderer. + const render = SDK_RENDERERS[runtime.language] + /* v8 ignore next 3 -- requireCodeRuntime rejects an unknown language before this ever runs. */ + if (!Object.hasOwn(SDK_RENDERERS, runtime.language) || render === undefined) { + throw new Error(`dsh-tools: no SDK renderer registered for runtime language ${JSON.stringify(runtime.language)} (known: ${Object.keys(SDK_RENDERERS).map(name => JSON.stringify(name)).join(', ')})`) + } + return render(this.sdkSchemas(context.scope)) }, }) } @@ -780,11 +817,17 @@ export class ToolRegistry extends Service { */ private wireSchemas(scope?: ScopeKey): ToolProviderResult { const view = this.view(scope) - const schemas = [...view.visible.values()].map(definition => this.schemaOf(definition, false)) if (this.mode === 'native') { + const schemas = [...view.visible.values()].map(definition => this.schemaOf(definition, false)) return { schemas, knownNames: [...view.knownNames] } } + // Validate the runtime language BEFORE projecting schemas: schemaOf reads + // run_code's language-aware description/parameters getters, whose own + // flavor-table guard would otherwise surface first. This keeps the + // renderer-table rejection the canonical assembly-time error for a + // language with no SDK renderer. this.requireCodeRuntime() + const schemas = [...view.visible.values()].map(definition => this.schemaOf(definition, false)) if (this.mode === 'code') { return { schemas: schemas.filter(schema => schema.name === RUN_CODE_NAME), @@ -801,14 +844,23 @@ export class ToolRegistry extends Service { * behind it — hostage to a code runtime existing even under `mode: * 'native'` (the loop's optional-backend idiom, same as * `sessionPersistence`). + * + * Assembly and `run_code` execution read separately, so the language is not + * bound to a request. Harmless while one published backend exists — both + * reads return the same flavor — but a reload that swapped in a second + * language between them would hand a program written against one SDK to the + * other. Binding it belongs to the PR that publishes that backend, which is + * also the first point it can be tested; recorded in the + * [language-dispatch note](../../../../.agents/notes/implemented/feature/2026-07-31-code-mode-language-dispatch.md). */ private requireCodeRuntime(): CodeRuntime { const runtime = this.ctx.get('codeRuntime') if (!runtime) { throw new Error(`dsh-tools: mode "${this.mode}" requires a code runtime — load a ctx.codeRuntime implementation (e.g. @deepseek-ai/dsh-code-runtime-worker) or set tools mode to "native"`) } - if (runtime.language !== 'typescript') { - throw new Error(`dsh-tools: mode "${this.mode}" generates a TypeScript SDK, but the loaded code runtime's language is "${runtime.language}"`) + if (!Object.hasOwn(SDK_RENDERERS, runtime.language)) { + const known = Object.keys(SDK_RENDERERS).map(name => JSON.stringify(name)).join(', ') + throw new Error(`dsh-tools: no SDK renderer registered for runtime language ${JSON.stringify(runtime.language)} (known: ${known})`) } return runtime } diff --git a/packages/core/tools/src/py-types.ts b/packages/core/tools/src/py-types.ts new file mode 100644 index 0000000000..d1358124c3 --- /dev/null +++ b/packages/core/tools/src/py-types.ts @@ -0,0 +1,818 @@ +/** + * Code Mode codegen — Python flavor. The pure projection from registered tool schemas to the + * Python SDK text the model programs against under `runtime.language === 'python'`. Sibling of + * {@link ./ts-types.ts | ts-types.ts}; the two files are two projections of the same registry + * store, keyed by the loaded {@link @deepseek-ai/dsh-code-runtime#CodeRuntime.language | code + * runtime's language}. + * + * Under `mode: 'code'` the native tool schemas are omitted from the request, so this generated + * SDK is the model's ONLY source for each tool's argument names, required fields, types, + * descriptions, and canonical output shapes; under `mode: 'both'` the native schemas ship + * alongside it and it is one of two. Object-shaped arguments and outputs therefore render as one + * named `TypedDict` per tool (and per nested object), not an opaque `dict[str, Any]`, so the + * shape survives into the program under the mode that has nothing else to carry it. + * @module @deepseek-ai/dsh-tools/src/py-types + */ + +import { assertSupportedJsonSchema } from './json-schema.ts' +import type { JsonSchemaNode, JsonSchemaScalar } from './json-schema.ts' +import type { ToolSdkSchema } from './ts-types.ts' + +/** + * The reference grammar's `xid_start xid_continue*` — the set + * `str.isidentifier()` accepts on a CPython whose Unicode tables match the + * engine's. See {@link isBareIdentifier} for what a version skew does. + */ +const IDENTIFIER = /^[\p{XID_Start}_]\p{XID_Continue}*$/u + +/** + * Whether a name can be emitted as a bare Python identifier rather than + * routed to the subscript/`dict[str, Any]` path. + * + * Python identifiers are not ASCII: `路径` is as legal a field name as `path`, + * and rejecting it would degrade the whole enclosing object, dropping every + * field's name, requiredness, and type — information whose only source under + * `mode: 'code'` is this generated text. + * + * NFKC stability is a second and separate condition, because CPython + * normalizes identifiers at compile time while JSON keys are compared as + * written: `field` would be declared and reachable as `field`, so the SDK would + * advertise a key under a spelling the harness never accepts, and two keys + * that normalize together would collapse into one declaration. Those names + * take the subscript path, which carries their exact bytes. + * + * `IDENTIFIER`'s equivalence to `str.isidentifier()` was measured across 21 + * samples with zero divergence, on Node 22.23.1 against CPython 3.9.6 — every + * sample sits inside the two versions' shared tables, and the skew characters + * below are exactly where that pair diverges. The predicate as a whole is + * deliberately stricter than `isidentifier()`, which does not test NFKC + * stability: `'field'.isidentifier()` is True and this returns false. + * + * Both conditions are evaluated against the ENGINE's Unicode tables, and the + * two sides are versioned independently — `\p{XID_Start}`/`\p{XID_Continue}` + * follow the running engine (Node 22.23.1 reports Unicode 17.0) while CPython + * follows its own (3.9.6 reports 13.0.0). The skew is not symmetric. A CPython + * older than the engine is the dangerous direction: a character added to either + * property since its tables (U+10570 Vithkuqi and U+1E290 Toto, 14.0; U+1E4D0 + * Nag Mundari, 15.0; U+1C89 Cyrillic TJE, 16.0 — ages per `DerivedAge.txt`; all + * four are NFKC-stable and accepted here, and all four are `Cn` on that 3.9.6, + * which rejects them) is emitted bare and its tokenizer refuses the character, + * taking the whole SDK block down — the same parseability invariant + * {@link UNPRINTABLE}, {@link LONE_SURROGATE} and {@link MAX_LIST_NESTING} + * exist for. Both properties carry it: a character added only to `XID_Continue` + * passes the trailing `\p{XID_Continue}*` in a tail position and fails the same + * way — U+200C ZWNJ and U+200D ZWJ are that case, gaining `XID_Continue` in UCD + * 15.1 and absent from it in 13.0.0, 14.0.0 and 15.0.0, so `a\u{200C}b` is + * emitted bare here while `isidentifier()` is False on 3.9.6 and on 3.12.13 + * (15.0.0). A CPython newer than the engine only routes a legal name to the + * subscript/`dict[str, Any]` path: less readable, still correct. The NFKC + * condition reduces to the same skew, since normalization stability guarantees + * an assigned character's normalization never changes afterwards. + * + * This predicate is not the only reader of engine tables. {@link camelCase} + * reads them at three further points — its split set, its head test, and its + * `toUpperCase()` case mapping — and this predicate's verdict gates none of + * them: a class name derived there reaches emitted text whenever any object + * shape in the tool's schema declares a `TypedDict`, including for a tool this + * predicate rejected. A tool named `zz-\u{1E4D0}x` with such parameters never + * reaches the skew here (the `-` rejects it outright) yet emits `class + * Zz\u{1E4D0}xArgs`, which that same 3.9.6 refuses — Nag Mundari arrived two + * releases after its tables. The case mapping is a separate table rather than + * an XID membership test, and it fails on names both conditions above accept: + * `\u{019B}` is XID_Start and NFKC-stable, so this predicate accepts it and + * `async def \u{019B}` compiles on 3.9.6, but Node uppercases it to + * `\u{A7DC}` — unassigned in that CPython, whose own `.upper()` is the identity + * here — and the declared `class \u{A7DC}Args` fails with `invalid + * non-printable character U+A7DC`. Closing the exposure therefore covers all + * four read points, not this predicate alone; it needs the target interpreter's + * version, which the backend reporting `language: 'python'` owns and which is + * unpublished on this base, so the note records it as that PR's decision. + * + * The `ts-types` sibling keeps its own ASCII rule rather than sharing this + * one: ECMAScript identifiers are a different set (`$`) and are never + * normalized, so one predicate cannot be correct for both. ZWJ/ZWNJ are not + * part of that difference — both sets carry them on the engine's tables; what + * separates the two there is the CPython table version above. + * @param name - the raw schema field or tool name. + * @returns whether the name can be emitted bare. + */ +function isBareIdentifier(name: string): boolean { + return IDENTIFIER.test(name) && name.normalize('NFKC') === name +} + +/** + * Python hard keywords: reserved everywhere, so a tool or field named + * ``class`` or ``lambda`` is legal on the wire but not as an attribute + * (``tools.class`` would be a SyntaxError in the model program) and not as a + * class-syntax `TypedDict` field. Such a tool renders under subscript access + * and such an object degrades to ``dict[str, Any]`` — the model still reaches + * every tool and field without collisions. + * Soft keywords (``match``, ``case``, ``type``, ``_`` — the language + * reference's whole set) are deliberately ABSENT: each is special in exactly + * one syntactic position — a statement head (``match``, ``type``), a ``match`` + * statement's clause head (``case``), or a pattern (``_``) — so ``match: str`` + * as a field and ``async def match(...)`` as a method are both legal, and + * including them would needlessly degrade common search/regex tool fields to + * ``dict[str, Any]``. Underscore-leading names are handled separately, not + * here: a non-dunder ``__token`` name-mangles, a dunder present on + * ``object``/``type`` resolves before the proxy hook, and implicit + * special-method lookup bypasses the hook. + */ +const RESERVED = new Set([ + 'False', 'None', 'True', 'and', 'as', 'assert', 'async', 'await', 'break', 'class', + 'continue', 'def', 'del', 'elif', 'else', 'except', 'finally', 'for', 'from', 'global', + 'if', 'import', 'in', 'is', 'lambda', 'nonlocal', 'not', 'or', 'pass', 'raise', + 'return', 'try', 'while', 'with', 'yield', + // Not a keyword, but CPython refuses to ASSIGN it at compile time + // (`SyntaxError: cannot assign to __debug__`), which is what a TypedDict + // field, a parameter name, and a keyword argument all are. + '__debug__', +]) + +/** `typing` symbols this module may emit, in the deterministic import order. */ +const TYPING_ORDER = ['Any', 'Literal', 'NotRequired', 'Protocol', 'TypedDict'] as const + +/** `indent`-deep line prefix (four spaces per level to match PEP 8 output). */ +function pad(indent: number): string { + return ' '.repeat(indent) +} + +/** + * Collector threaded through {@link renderType}: the emitted `TypedDict` class + * declarations (nested classes precede the parent that references them), the + * class names already taken (for collision suffixing), a per-base collision + * counter, and the `typing` symbols the render actually used. + */ +interface RenderState { + readonly classes: string[] + readonly usedClassNames: Set + /** Next collision counter per capped base, so allocation is amortized O(1) instead of rescanning from `2`. */ + readonly nextClassCounter: Map + readonly typing: Set +} + +/** + * The `Cc` code points that survive the whitespace collapse in {@link describe} + * and have no printable form: the C0 controls, DEL, and the C1 controls. Only + * U+0009 to U+000D are absent, because ECMAScript `\s` already collapsed them — + * `\s` is TAB/VT/FF/SP/NBSP/ZWNBSP/Zs plus LF/CR/LS/PS, so no C1 code point is + * in it and the whole U+0080 to U+009F block reaches this rule intact. Those + * are not hypothetical input: they are what Windows-1252 bytes 0x80 to 0x9F + * (smart quotes, em dash) become when decoded as Latin-1. + * CPython rejects source containing a NUL outright + * (`SyntaxError: source code string cannot contain null bytes`), whether it + * sits in a docstring or in a comment, so one such byte anywhere in a schema + * description would make the whole generated SDK unparseable — under + * `mode: 'code'`, the model's only declaration of the tools. The rest are + * legal but invisible; escaping them with the same rule keeps the emitted text + * readable and the treatment uniform. + * + * The boundary is the category, not per-code-point addressability: `\xNN` + * addresses U+0000 to U+00FF, so one escape form covers `Cc` exactly. The + * invisible `Cf` formatting characters pass through by design — of them only + * U+00AD soft hyphen would fit `\xNN` at all, and escaping that one while + * U+200B ZWSP, U+200E/U+200F bidi marks, and U+2060 word joiner passed through + * would leave a rule that is neither category- nor addressability-shaped. The + * whole family is legal in both consumers, since only LF and CR terminate a + * Python string literal or a `#` comment. That set is the tokenizer's, not + * `str.splitlines()`': NEL (U+0085), LS (U+2028), and PS (U+2029) split a + * string at run time but do not end a physical line in source — measured on + * CPython 3.9.6 and 3.12.13, each accepted in both positions with the value + * round-tripping — so they are safe raw wherever they reach emitted text + * unescaped, which for all three is `JSON.stringify`, at two call sites: + * {@link pyScalar}'s literal path, and the subscript tool-name comment's own + * call, which a name carrying any of them always reaches, none being + * `XID_Continue`. The `description` path escapes NEL under the class above and + * folds LS and PS in {@link describe}'s `\s+` collapse, both being `\s`. + */ +const UNPRINTABLE = /[\u0000-\u0008\u000e-\u001f\u007f-\u009f]/g + +/** + * Unpaired surrogate code points, escaped by {@link describe} as `\uNNNN` — + * its own form, since `\xNN` stops at U+00FF. The `u` flag is what makes this + * the LONE ones: in Unicode mode a well-formed pair is a single astral code + * point outside D800 to DFFF, so an emoji in a description survives untouched. + * + * This is the NUL case from {@link UNPRINTABLE}, not the invisible-character + * case. Python source must be UTF-8-encodable and a lone surrogate is not, so + * `compile()` raises `UnicodeEncodeError: surrogates not allowed` for one + * anywhere in the text — measured on 3.9 for a string literal and for a `#` + * comment alike. A raw or MCP tool description reaches this: `JSON.parse` on a + * wire `"\ud800"` escape yields exactly such a code point. + */ +const LONE_SURROGATE = /[\ud800-\udfff]/gu + +/** + * The collapsed one-line `description` of a schema node (byte-stable across + * formatting churn), or `undefined` when the node carries none. Every caller + * passes an object — a validated property node, the `ToolSdkSchema` itself, or + * the `{ description }` wrapper {@link docLines} synthesizes — so only the + * description field needs guarding. A description that collapses + * to nothing (empty, or whitespace only) is `undefined` too: it documents the + * node no better than an absent one, and emitting it would leave an empty + * `"""` docstring or a bare `# ` line in the SDK. Only ECMAScript whitespace + * folds, so a description of whitespace plus one surviving control character is + * NOT absent: it collapses to that character's visible escape. + * + * Control characters left over after the whitespace collapse are rendered as + * their `\xNN` escapes (see {@link UNPRINTABLE}) and unpaired surrogates as + * their `\uNNNN` escapes (see {@link LONE_SURROGATE}); the escape's own backslash is + * emitted literally by both consumers, since {@link docLines} doubles it into a + * Python source escape and a `#` comment carries it verbatim. + */ +function describe(schema: object): string | undefined { + const description = (schema as Record).description + if (typeof description !== 'string') return undefined + const collapsed = description + .replace(/\s+/g, ' ') + .replace(UNPRINTABLE, char => `\\x${char.charCodeAt(0).toString(16).padStart(2, '0')}`) + .replace(LONE_SURROGATE, char => `\\u${char.charCodeAt(0).toString(16).padStart(4, '0')}`) + .trim() + return collapsed.length === 0 ? undefined : collapsed +} + +/** + * One-line docstring for a tool `description`, or no lines when there is none. + * Backslashes are doubled first, every quote is escaped, and a trailing + * backslash cannot survive: a description ending in `"` or an odd backslash + * would otherwise merge with (or escape) the closing triple quote and make + * the generated block — Code Mode's only SDK — syntactically invalid Python. + */ +function docLines(description: unknown, indent: number): string[] { + const collapsed = describe({ description }) + if (collapsed === undefined) return [] + const escaped = collapsed.replaceAll('\\', '\\\\').replaceAll('"', '\\"') + return [`${pad(indent)}"""${escaped}"""`] +} + +/** + * CamelCase a name into a Python type identifier: non-identifier characters + * split words, `_` splits too (it is `XID_Continue`, so the split set names it + * explicitly), and a head that cannot start an identifier takes a `Tool` + * prefix. Unicode survives, so a `路径` field yields `路径`-based class names + * instead of collapsing to the bare prefix. A character that is not + * `XID_Continue` splits even when it is a letter, so a name whose NFKC folding + * would leave the identifier set is not carried through — the split set is the + * grammar's, not an ASCII approximation of it. + * + * The result is NFKC-normalized: these names are generated, never matched + * against a JSON key, so normalizing is free here and keeps what CPython + * compiles identical to what is emitted — unlike {@link isBareIdentifier}, + * which must reject unstable names outright. Normalizing AFTER the prefix + * decision is what makes that hold at the seam the prefix creates: `Tool` + + * a combining-mark head composes there (`U+0301` gives `Tooĺ`, U+013A), so + * normalizing only the un-prefixed part would emit a name CPython compiles to + * a different symbol. The second call is idempotent on the un-prefixed arm. + * + * The split set, the head test, and `toUpperCase()` all read the engine's + * Unicode tables, so this function carries the same version skew + * {@link isBareIdentifier} documents, by paths independent of it: a class name + * derived here reaches emitted text whenever any object shape in the tool's + * schema declares a `TypedDict`, and the predicate's verdict on the tool name + * does not gate that. The case mapping is the one that can fail on a name the + * predicate accepted; the worked example is there. + * @param raw - the schema field or tool name to derive from. + * @returns a class-name segment safe to emit. + */ +function camelCase(raw: string): string { + const joined = raw + .split(/[^\p{XID_Continue}]+|_+/u) + .filter(part => part.length > 0) + .map(part => `${part.charAt(0).toUpperCase()}${part.slice(1)}`) + .join('') + .normalize('NFKC') + return (/^\p{XID_Start}/u.test(joined) ? joined : `Tool${joined}`).normalize('NFKC') +} + +/** Class-name base cap keeping each emitted name — and total text — linear in schema depth. */ +const MAX_CLASS_NAME_BASE = 120 + +/** + * Deepest `list[…]` nesting emitted into one annotation before the item type + * degrades to `Any`. CPython's tokenizer rejects a logical line holding more + * than 200 simultaneously-open brackets (`MAXLEVEL`, `SyntaxError: too many + * nested parentheses`), so an array chain deeper than that would render an SDK + * block that is not valid Python at all — the same failure the docstring + * escaping in {@link docLines} exists to prevent. 180 leaves headroom for the + * few brackets an annotation can add around the chain, all of which count + * toward the same limit. Per emission site, counting brackets open at the + * chain's innermost point: + * + * - Return annotation, `async def f(self, args: X) -> chain:` — 180 `list[` + * plus an innermost `Literal[`. The parameter list's `(` closed at the `)` + * before the `->`, so it is NOT open here: 181. + * - TypedDict field, `field: NotRequired[chain]` — a class-body line with no + * other open bracket, and its children start at `listDepth: 1` to reserve + * the `NotRequired[`, so 179 `list[` plus `Literal[`: 181. Required fields + * share that start for uniformity, spending one level of representable depth + * on a bracket they never emit. + * - Argument annotation, `async def f(self, args: chain) -> Y:` — the `(` IS + * still open around it: 180 `list[` plus `Literal[` plus the paren, 182, the + * worst case. Reachable only through a raw `register()` whose `parameters` + * is an array reached from the root through `oneOf` arms alone — the root + * array itself, or one nested under any depth of unions, since an arm + * inherits the enclosing depth unchanged (`A | B` opens no bracket). An + * object ancestor takes it out of this case: its fields restart the chain at + * the 181 site. `defineTool` compiles an object root, so the annotation is a + * bare TypedDict class name or a one-bracket `dict[str, Any]` when that + * object degrades — never a chain. + * + * A CPython grammar limit, not a deployment choice, so it is fixed rather than + * configurable. The sibling `ts-types` renderer needs no counterpart: nothing + * in the TypeScript grammar bounds nesting, and its SDK block is never type- + * checked. Only bracket nesting counts — a `oneOf` renders as a flat `A | B` + * chain and nested objects render as separate `class` statements, so neither + * accumulates open brackets at any depth. The invariant this cap serves is + * grammatical validity; see the `oneOf` arm in {@link renderType} for the one + * interpreter limit deliberately left uncapped. + */ +const MAX_LIST_NESTING = 180 + +/** + * Cap a class-name base at {@link MAX_CLASS_NAME_BASE} (see the callers for + * why capping keeps the render linear). `slice` counts UTF-16 code units, so + * an astral character straddling the boundary would be cut in half and leave a + * lone surrogate — not an identifier character, and not even well-formed text; + * drop it rather than emit it. + */ +function capClassNameBase(base: string): string { + if (base.length <= MAX_CLASS_NAME_BASE) return base + const capped = base.slice(0, MAX_CLASS_NAME_BASE) + return /[\uD800-\uDBFF]$/.test(capped) ? capped.slice(0, -1) : capped +} + +/** + * Reserve a unique class name from a base, suffixing `2`, `3`, … on collision. + * The base is capped at {@link MAX_CLASS_NAME_BASE} first: child class names + * derive from their parent's allocated name (`ParentChild`), so an unbounded + * schema of single-field objects would otherwise grow each name by one field + * per level and the sum of all names to Θ(depth²). Capping the base keeps each + * name — and the total emitted text — linear in depth. Collisions resume from + * the per-base counter in `state.nextClassCounter` rather than rescanning from + * `2`, so a deep chain sharing one capped base stays O(1) per allocation + * (amortized) instead of Θ(depth²) in time. + */ +function allocateClassName(base: string, state: RenderState): string { + const capped = capClassNameBase(base) + let name = capped + if (state.usedClassNames.has(name)) { + let n = state.nextClassCounter.get(capped) ?? 2 + while (state.usedClassNames.has(`${capped}${n}`)) n++ + name = `${capped}${n}` + state.nextClassCounter.set(capped, n + 1) + } + state.usedClassNames.add(name) + return name +} + +/** + * Append a child-name segment to a parent class-name base, capping the result + * at {@link MAX_CLASS_NAME_BASE}. Capping AT PROPAGATION (not only inside + * {@link allocateClassName}) keeps each level O(1): a deep `oneOf`- or + * object-chain would otherwise carry an ever-growing ConsString down the tree + * and re-materialize it (via `.length`/`.slice`) at every level — Θ(depth²). + * The bounded base plus the collision counter still yields unique names. + * + * The join is NFKC-normalized because both sides are separately normalized yet + * their concatenation need not be: a base ending in a Hangul L jamo or LV + * syllable composes with a following V or T jamo head (`가` + `ᆨ` gives `각`), + * so the emitted class name would differ from the symbol CPython compiles, and + * two byte-distinct names could fold onto one — `usedClassNames` dedupes by the + * raw bytes, so the collision counter would not see it. Normalizing costs + * O(cap + segment) per level, the same order as the `slice` it feeds. The other + * two join points need no counterpart: `Args`/`Output` start with `A`/`O` and + * {@link allocateClassName}'s suffix is digits, none of which compose backwards. + */ +function childClassName(base: string, segment: string): string { + return capClassNameBase(`${base}${segment}`.normalize('NFKC')) +} + +/** + * Render one validated scalar as Python literal text (`True`/`False`, + * JSON-quoted strings, bare numbers). `null` cannot reach here: the `null` + * type renders directly as `None`, and the unified validator rejects a null + * `const`/`enum` entry on every other scalar type. + * + * A beyond-safe-range integral number takes `BigInt` digits rather than + * `String`: Python integers are arbitrary-precision, so the emitted digits ARE + * the value the model programs against, and `String` can give a different + * integer than the double holds (`2 ** 60` prints the rounded `...847000`, not + * the exact `...846976`) or no integer literal at all (`1e21` prints `1e+21`). + * `String`'s rounding is not a bug in it: `Number::toString` emits the shortest + * decimal string that re-reads to the same double, then pads to the exponent + * with zeros (1 significant digit for `1e20`, 16 for `2 ** 60`) — and when the + * shortest string is shorter than the double's exact value, those padded digits + * name an integer no double holds. Passing one back would have to cross the + * argument boundary as a JSON number — a double again — so the SDK would + * document a value no program can pass. `BigInt` needs no case split: where + * `String` is already exact (`2 ** 53`, `1e20`) the two agree byte for byte, + * and where it is not, `BigInt` is the exact one. The TS flavor needs no + * counterpart at all: its literal is re-read by a JS parser back into the same + * double. + * + * `JSON.stringify` is also what keeps this path's output parseable, and it is + * the only thing that does. It covers both classes of hazard: the two kinds of + * code point CPython refuses anywhere in source — NUL among the C0 controls, + * and the whole D800–DFFF unpaired-surrogate block, escaped under ES2019 + * well-formed stringification, which the engines range guarantees — and the + * ones that break this line in particular, a bare `"` closing the literal + * early, a trailing odd backslash eating the closing quote, and a bare LF/CR + * ending it before its terminator. The `description` path carries + * {@link UNPRINTABLE} and {@link LONE_SURROGATE} because nothing quotes it, + * and folds newlines in {@link describe}. + * + * That leans on a coincidence worth naming: every escape `JSON.stringify` can + * emit (`\"`, `\\`, `\b`, `\f`, `\n`, `\r`, `\t`, `\uXXXX`) is also a Python + * escape denoting the same character, so the emitted `Literal[...]` both + * parses and decodes back to the value the schema declared. DEL, the C1 + * controls (NEL among them), and LS/PS (U+2028/U+2029) do reach it raw — + * legal but invisible, byte-for-byte as in the TS flavor; escaping them is a + * both-flavors change. Those last three are legal here for the reason + * {@link UNPRINTABLE} records: they are `str.splitlines()` boundaries, not + * tokenizer line terminators. The subscript tool-name comment quotes its name + * through its own call to the same `JSON.stringify`, never through this + * function, and inherits both halves — escapes and pass-throughs alike. + */ +function pyScalar(value: JsonSchemaScalar): string { + if (value === true) return 'True' + if (value === false) return 'False' + if (typeof value === 'string') return JSON.stringify(value) + if (typeof value === 'number' && Number.isInteger(value) && !Number.isSafeInteger(value)) { + return BigInt(value).toString() + } + return String(value) +} + +/** + * Render a validated scalar `const`/`enum` as `Literal[...]`, falling back to + * the broad type. Deliberately deviates from PEP 586, which restricts `Literal` + * parameters to int/bool/str/bytes/enum/None: a non-integral number + * `const`/`enum` emits a float literal (`Literal[1.5]`) a strict checker would + * reject. An integral one does not deviate — {@link pyScalar} emits int digits, + * including for the beyond-safe-range values it widens through `BigInt`, and + * PEP 586 admits int parameters. Harmless either way — the stub is advisory + * prompt text, only required to parse — and keeping the exact value + * communicates the constraint to the model. + */ +function renderConstrainedScalar(node: JsonSchemaNode, broad: string, state: RenderState): string { + if (node.const !== undefined) { + state.typing.add('Literal') + return `Literal[${pyScalar(node.const)}]` + } + if (node.enum !== undefined) { + state.typing.add('Literal') + return `Literal[${node.enum.map(pyScalar).join(', ')}]` + } + return broad +} + +/** + * Map one JSON-Schema node to a Python type expression, threading `state` to + * collect the `TypedDict` declarations and `typing` symbols a full render + * needs. `className` is the name to give an object node with properties (and + * the prefix for its nested objects). Handles every unified schema construct — + * `oneOf` (→ `X | Y`), `const`/`enum` (→ `Literal[...]`), `integer` (→ `int`), + * `null` (→ `None`) — and degrades an unsupported or malformed schema to `Any` + * without throwing, the same trusted-after-validation stance as the sibling + * {@link ./ts-types.ts | ts-types} renderer. {@link jsonSchemaToPy} is the + * context-free entry point; this is the collecting core. + */ +function renderType(schema: unknown, className: string, state: RenderState): string { + interface Frame { + // A validated JSON-schema node past the root `assertSupportedJsonSchema` + // (the root frame's schema is asserted before any frame is built), so the + // walk reads its fields without casts — the same typed-frame shape as the + // sibling ts-types renderer. + schema: JsonSchemaNode + className: string + phase: 'start' | 'children' + kind?: 'oneOf' | 'array' | 'typeddict' + node?: JsonSchemaNode + /** Open `list[` brackets enclosing this node in the annotation being built ({@link MAX_LIST_NESTING}). */ + listDepth: number + children: { schema: JsonSchemaNode; className: string; listDepth: number }[] + childIndex: number + childTypes: string[] + entries: [string, JsonSchemaNode][] + allocated?: string + } + const newFrame = (schema: JsonSchemaNode, className: string, listDepth: number): Frame => + ({ schema, className, phase: 'start', listDepth, children: [], childIndex: 0, childTypes: [], entries: [] }) + try { + // Validate the WHOLE tree once, then trust it — the same contract the + // sibling ts-types renderer follows at a typed same-process seam. Every + // node past this point is a validated JSON-schema node, so the walk reads + // its fields without re-checking. An unsupported or malformed schema throws + // here (before anything is emitted) and degrades to `Any`, the Python + // counterpart of the TS flavor's `unknown`. + assertSupportedJsonSchema(schema) + const frames: Frame[] = [newFrame(schema, className, 0)] + let result: string | undefined + /* jscpd:ignore-start -- the explicit-stack walk skeleton deliberately parallels + ts-types.ts's renderSupportedSchema; the two sibling renderers keep symmetric shapes. */ + const finish = (type: string): void => { + frames.pop() + const parent = frames.at(-1) + if (parent === undefined) result = type + else parent.childTypes.push(type) + } + + while (frames.length > 0) { + const frame = frames.at(-1) + /* v8 ignore next -- the loop condition guarantees a current frame. */ + if (frame === undefined) break + + if (frame.phase === 'children') { + if (frame.childIndex < frame.children.length) { + const child = frame.children[frame.childIndex] + /* v8 ignore next -- childIndex is bounded by children.length. */ + if (child === undefined) throw new Error('missing python render child') + frame.childIndex++ + frames.push(newFrame(child.schema, child.className, child.listDepth)) + continue + } + if (frame.kind === 'oneOf') { + // Concatenate incrementally (template literal, not `Array.join`): V8 + // builds a lazy ConsString, so a deep oneOf chain materializes once + // at the root instead of re-materializing the accumulated string at + // every level (which `join` would, making it Θ(depth²)). This matches + // the array arm's template-literal laziness and ts-types' composable- + // document approach — the whole walk stays linear in schema depth. + let union = '' + for (const [index, childType] of frame.childTypes.entries()) { + union = index === 0 ? childType : `${union} | ${childType}` + } + finish(union) + continue + } + /* jscpd:ignore-end */ + if (frame.kind === 'array') { + // `list[A | B]` needs no parentheses in Python. Array frames always + // schedule exactly one child, so its type is present. + /* v8 ignore next -- the ?? arm needs a childless array frame, which start never builds. */ + finish(`list[${frame.childTypes[0] ?? 'Any'}]`) + continue + } + // typeddict: assemble AFTER the children so any nested class this one + // references is already declared (declaration order = reference order). + const node = frame.node + const name = frame.allocated + /* v8 ignore next -- typeddict frames always set node and allocated at start. */ + if (node === undefined || name === undefined) throw new Error('missing typeddict frame state') + const required = new Set(node.required) + const lines = [`class ${name}(TypedDict):`] + for (let index = 0; index < frame.entries.length; index++) { + const entry = frame.entries[index] + const fieldType = frame.childTypes[index] + /* v8 ignore next -- entries and childTypes correspond one-to-one. */ + if (entry === undefined || fieldType === undefined) throw new Error('missing typeddict field type') + const [field, fieldSchema] = entry + // The parent node passed assertSupportedJsonSchema, so every property + // value is a validated schema node. + const description = describe(fieldSchema) + if (description !== undefined) lines.push(`${pad(1)}# ${description}`) + if (required.has(field)) { + lines.push(`${pad(1)}${field}: ${fieldType}`) + } else { + state.typing.add('NotRequired') + lines.push(`${pad(1)}${field}: NotRequired[${fieldType}]`) + } + } + // TypedDict syntax cannot express openness, so an open object states it + // in-band: the annotation is advisory either way, and `mode: 'code'` + // omits the native schemas, making this line the model's only signal + // that extra keys are accepted. + if (node.additionalProperties !== false) { + lines.push(`${pad(1)}# Additional keys beyond those declared are allowed.`) + } + // A closed empty object still needs a class body (`pass`) to be valid + // Python; the declared emptiness is the information. + if (lines.length === 1) lines.push(`${pad(1)}pass`) + state.classes.push(lines.join('\n')) + finish(name) + continue + } + + frame.phase = 'children' + const node = frame.schema + if (node.oneOf !== undefined) { + frame.kind = 'oneOf' + // A union renders as `A | B` — no brackets of its own, so the branches + // inherit the enclosing depth unchanged. + // + // Union LENGTH is deliberately uncapped, unlike list nesting. The two + // limits are different in kind: >200 open brackets is a SyntaxError + // from the tokenizer, so the text is not Python; a long `A | B | …` + // chain is grammatically valid at any length and only defeats CPython's + // C-recursion when `compile()` walks the left-nested BinOp spine + // (measured: 1,000 branches compile, 5,000 raise RecursionError). This + // block is prompt text — nothing compiles it — so that limit costs + // nothing here, while capping would retire the deep-chain tests that + // pin the walk's linear time and the class-name propagation cap. The + // standard this renderer holds is grammatical validity, not + // compilability under one interpreter's stack. + frame.children = node.oneOf.map((branch, index) => ({ schema: branch, className: childClassName(frame.className, `${index + 1}`), listDepth: frame.listDepth })) + continue + } + if (node.type === undefined) { + state.typing.add('Any') + finish('Any') + continue + } + switch (node.type) { + case 'string': finish(renderConstrainedScalar(node, 'str', state)); break + case 'number': finish(renderConstrainedScalar(node, 'float', state)); break + case 'integer': finish(renderConstrainedScalar(node, 'int', state)); break + case 'boolean': finish(renderConstrainedScalar(node, 'bool', state)); break + case 'null': finish('None'); break + case 'array': { + if (node.items === undefined) { + state.typing.add('Any') + finish('list[Any]') + break + } + // Past MAX_LIST_NESTING another `list[` would push the annotation + // beyond CPython's open-bracket limit and make the whole SDK block + // unparseable, so the chain degrades here instead — an unusable + // annotation either way, and this one is valid Python. + if (frame.listDepth >= MAX_LIST_NESTING) { + state.typing.add('Any') + finish('Any') + break + } + // An array of objects names its item type after the array field. + frame.kind = 'array' + frame.children = [{ schema: node.items, className: frame.className, listDepth: frame.listDepth + 1 }] + break + } + case 'object': { + // A missing `properties` is an empty property map, exactly as the + // unified validator and the TS renderer read it — NOT an unknown + // shape. The openness of the resulting empty object is decided below, + // so a closed empty object still declares an empty TypedDict rather + // than a permissive `dict[str, Any]`. + const entries = Object.entries(node.properties ?? {}) + // An empty `className` marks the context-free `jsonSchemaToPy` entry: + // there is no naming context to declare into, so degrade. This reads + // the CALL's className, not `frame.className`: the marker belongs to + // the whole walk, and frames propagate a derived name (a `oneOf` + // branch of the context-free root gets the index-derived name `1` — + // `childClassName` concatenates and caps, it does not go through + // `camelCase`), so a per-frame read would declare classes the caller + // has no way to receive, under a name that is not even a legal + // identifier: `class 1(TypedDict):`. A field + // name that is not a legal Python attribute is inexpressible as a + // class-syntax `TypedDict` field, so such an object degrades whole. + // A leading-double-underscore non-dunder field (`__token`) would be + // NAME-MANGLED inside class syntax (`_ClassName__token`), describing a + // different JSON key than the registered schema — degrade like any + // other inexpressible field name. + if (className === '' || !entries.every(([name]) => isBareIdentifier(name) && !RESERVED.has(name) && !(name.startsWith('__') && !name.endsWith('__')))) { + state.typing.add('Any') + finish('dict[str, Any]') + break + } + // An OPEN empty object is any dict; a CLOSED empty object declares an + // empty TypedDict so "no keys accepted" survives into the SDK. + if (entries.length === 0 && node.additionalProperties !== false) { + state.typing.add('Any') + finish('dict[str, Any]') + break + } + frame.kind = 'typeddict' + frame.node = node + frame.allocated = allocateClassName(frame.className, state) + state.typing.add('TypedDict') + frame.entries = entries + // A field annotation is its own logical line, so nesting restarts — + // at 1, reserving the bracket an optional field's `NotRequired[…]` + // wraps around it. frame.allocated was assigned three statements up; + // the ?? arm is for the type system only. + /* v8 ignore next -- allocated is always set before children are built. */ + frame.children = entries.map(([field, child]) => ({ schema: child, className: childClassName(frame.allocated ?? '', camelCase(field)), listDepth: 1 })) + break + } + /* v8 ignore next 4 -- assertSupportedJsonSchema narrowed this closed type union. */ + default: { + state.typing.add('Any') + finish('Any') + } + } + } + /* v8 ignore next -- every root frame produces one expression. */ + return result ?? 'Any' + } catch { + // An unsupported or malformed schema failed validation (before any + // emission), or an unreachable internal invariant tripped. Either degrades + // the node to `Any` rather than crashing prompt assembly — the Python + // counterpart of the TS flavor's `unknown` fallback. + state.typing.add('Any') + return 'Any' + } +} + +/** + * Map one JSON-Schema node to a context-free Python type expression from the + * `typing` module. Handles every unified schema construct — `object` (degraded + * to `dict[str, Any]`: naming a `TypedDict` requires the render context that + * {@link renderToolsSdkPy} supplies), `const`/`enum` (→ `Literal[...]`), + * `oneOf` (→ union), `string`/`number`/`integer`/`boolean`/`null`, `array` + * (`items` → `list[T]`) — and returns `Any` for an unsupported or malformed + * schema, matching the TS flavor's `unknown` fallback. Type annotations in the + * emitted SDK are advisory: Python does not enforce them at runtime. + * @param schema - the JSON-Schema node. + * @returns the Python type text. + */ +export function jsonSchemaToPy(schema: unknown): string { + // A throwaway state whose class collector never escapes: an object with + // properties has nowhere to declare its TypedDict and degrades to + // dict[str, Any]. renderToolsSdkPy drives the named-TypedDict path. + return renderType(schema, '', { classes: [], usedClassNames: new Set(), nextClassCounter: new Map(), typing: new Set() }) +} + +/** The fixed model-facing usage contract rendered above the declarations. */ +const SDK_INSTRUCTIONS = `## Writing code for run_code + +Pass \`run_code\` the body of an async Python function (top-level \`await\` and \`return\` both work). At run time exactly two of the names declared below are bound: \`tools\` and \`ToolCallError\`. Everything else is a STATIC STUB describing shapes — in particular the \`TypedDict\` classes do NOT exist at run time, so build arguments as plain \`dict\`/\`list\` JSON values: \`await tools.name({"field": 1})\`, never \`FooArgs(field=1)\`, which raises \`NameError\`. Inside the program: + +- Call tools as \`await tools.name(args)\` — subscript access for exotic, reserved, or underscore-leading names: \`await tools["my-tool"](args)\`. Every call resolves to the tool's typed canonical JSON value (each method's return type below). Tool arguments must be lossless JSON. +- A FAILED tool call raises \`ToolCallError\`, whose \`toolName\` identifies the failed tool and whose message is human-readable — wrap in \`try/except\` to handle and continue. +- Independent read-only calls MAY overlap under \`asyncio.gather\` (safe calls run concurrently; mutating calls run alone, in submission order). Sequence dependent work with \`await\`. +- Emit the run's answer with \`print(...)\` and/or a top-level \`return \`; the returned value must be lossless JSON. ONLY what you print and the returned value come back — intermediate tool results never enter the conversation, so extract just what you need. + +The available tools:` + +/** + * Render the full `tools:sdk` prompt section under `runtime.language === + * 'python'`: the Python-flavored usage instructions plus one named `TypedDict` + * per tool argument or output object (and per nested object) and one awaitable + * method per visible tool on a `Tools` protocol — typed args in, the tool's + * canonical output value out — with a `tools: Tools` singleton the model calls + * into. The `typing` import line lists exactly the symbols the render used. + * Deterministic — tools are emitted in lexicographic name order, and class + * declarations precede the protocol in that same order (nested classes before + * the parent that references them), so an unchanged tool set produces + * byte-identical text across assemblies. The sort is not a total order on + * byte-equal names, so two schemas sharing a name would render in argument + * order; the caller's visible-capability map is keyed by name, so the input + * never carries a duplicate. + * @param schemas - the tool schemas plus canonical output schemas to declare + * (the caller excludes `run_code` itself). + * @returns the complete section text. + */ +export function renderToolsSdkPy(schemas: ToolSdkSchema[]): string { + const sorted = [...schemas].sort((a, b) => a.name < b.name ? -1 : a.name > b.name ? 1 : 0) + const state: RenderState = { classes: [], usedClassNames: new Set(), nextClassCounter: new Map(), typing: new Set(['Protocol']) } + // ONE ordered member stream, matching the documented lexicographic contract + // and the TypeScript flavor (which quotes exotic keys in place rather than + // partitioning them out). Interleaving is free here: a comment line between + // two `async def` lines is not a statement, so it changes nothing about how + // the class body parses. + const members: string[] = [] + let statements = 0 + for (const schema of sorted) { + const argType = renderType(schema.parameters, `${camelCase(schema.name)}Args`, state) + const outputType = renderType(schema.output, `${camelCase(schema.name)}Output`, state) + if (isBareIdentifier(schema.name) && !RESERVED.has(schema.name) && !schema.name.startsWith('_')) { + // A docstring only documents its method when it is the FIRST statement + // of that method's body. Emitted before the `async def` it would instead + // become the `Tools` class docstring (for the first tool) or a dead + // expression (for every later one), leaving every method undocumented — + // and under `mode: 'code'` this SDK is the model's only description of + // what a tool does. A docstring is a complete body, so the `...` stub is + // only for the description-less case. + const doc = docLines(schema.description, 2) + members.push(doc.length > 0 + ? `${pad(1)}async def ${schema.name}(self, args: ${argType}) -> ${outputType}:` + : `${pad(1)}async def ${schema.name}(self, args: ${argType}) -> ${outputType}: ...`) + members.push(...doc) + statements += 1 + } else { + // Not reachable as ``tools.name`` — the model reaches it via + // ``tools[name]``. Exotic names and hard keywords are not legal + // attributes at all; an underscore-leading name (``_foo``) IS a legal + // attribute and is routed here anyway, because the forms that break + // split three ways — a non-dunder ``__token`` name-mangles at the CALL + // site, a dunder that exists on ``object``/``type`` (``__class__``, + // ``__doc__``) resolves before ``__getattr__`` ever runs, and implicit + // special-method lookup skips the hook entirely — and one rule over the + // whole family costs nothing while a per-form rule would have to + // enumerate them (see {@link RESERVED}). The stub lists it as a subscript comment + // (referencing the named TypedDicts too) so a reader sees what is + // accessible; runtime resolution goes through the proxy's __getitem__. + members.push(`${pad(1)}# tools[${JSON.stringify(schema.name)}](args: ${argType}) -> ${outputType}`) + const description = describe(schema) + if (description !== undefined) members.push(`${pad(1)}# ${description}`) + } + } + // Subscript entries are COMMENTS, not statements: a class body of only + // comments fails to parse, so `pass` is required whenever no method was + // emitted — including the subscript-only tool set. + const bodyLines = statements > 0 ? members : [`${pad(1)}pass`, ...members] + const body = bodyLines.join('\n') + const imports = TYPING_ORDER.filter(symbol => state.typing.has(symbol)) + const classBlock = state.classes.length > 0 ? `${state.classes.join('\n\n')}\n\n` : '' + const errorDeclaration = 'class ToolCallError(Exception):\n toolName: str' + const declaration = `from typing import ${imports.join(', ')}\n\n${errorDeclaration}\n\n${classBlock}class Tools(Protocol):\n${body}\n\ntools: Tools` + return `${SDK_INSTRUCTIONS}\n\n\`\`\`python\n${declaration}\n\`\`\`` +} diff --git a/packages/core/tools/src/ts-types.ts b/packages/core/tools/src/ts-types.ts index 26566d9548..1d33aa3514 100644 --- a/packages/core/tools/src/ts-types.ts +++ b/packages/core/tools/src/ts-types.ts @@ -262,7 +262,10 @@ The available tools:` * Render the full `tools:sdk` prompt section: the fixed usage instructions * plus one `declare const tools` interface covering every given tool. * Deterministic — tools are emitted in lexicographic name order, so an - * unchanged tool set produces byte-identical text across assemblies. + * unchanged tool set produces byte-identical text across assemblies. The sort + * is not a total order on byte-equal names, so two schemas sharing a name + * would render in argument order; the caller's visible-capability map is keyed + * by name, so the input never carries a duplicate. * @param schemas - the tool schemas to declare (the caller excludes * `run_code` itself). * @returns the complete section text. diff --git a/packages/core/tools/tests/code-mode.spec.ts b/packages/core/tools/tests/code-mode.spec.ts index 7b037d266a..e2c2c8be7e 100644 --- a/packages/core/tools/tests/code-mode.spec.ts +++ b/packages/core/tools/tests/code-mode.spec.ts @@ -335,9 +335,89 @@ describe('mode-aware wire contribution', () => { await expect(systemPrompt.assemble()).rejects.toThrow(/requires a code runtime/) }) - it("rejects every assembly when the runtime's language is not typescript", async () => { - const { systemPrompt } = await setup({ mode: 'code', runtime: { language: 'python' } }) - await expect(systemPrompt.assemble()).rejects.toThrow(/language is "python"/) + it('rejects every assembly when the runtime language has no registered SDK renderer', async () => { + const { systemPrompt } = await setup({ mode: 'code', runtime: { language: 'ruby' } }) + await expect(systemPrompt.assemble()).rejects.toThrow(/no SDK renderer registered for runtime language "ruby"/) + }) + + it('assembles under a python runtime by picking the Python SDK renderer', async () => { + const { ctx, systemPrompt } = await setup({ mode: 'code', runtime: { language: 'python' } }) + registerEcho(ctx) + const assembly = await systemPrompt.assemble() + const sdk = assembly.sections.find(section => section.name === 'tools:sdk') + expect(sdk?.text).toContain('class Tools(Protocol):') + expect(sdk?.text).toContain('async def echo(self, args:') + expect(sdk?.text).toContain('top-level `await`') + }) + + it("assembles under a python runtime in mode 'both' as well, SDK and schema together", async () => { + // `both` reaches the same wireSchemas/requireCodeRuntime/SDK-section code + // as `code`, so this pins the mode-by-language matrix rather than a + // separate path — including that the `wireSchemas` projection behind + // `assembly.tools` picks the Python flavor under `both` instead of hitting + // the flavor-table guard. + const { ctx, systemPrompt } = await setup({ mode: 'both', runtime: { language: 'python' } }) + registerEcho(ctx) + const assembly = await systemPrompt.assemble() + expect(assembly.sections.find(section => section.name === 'tools:sdk')?.text).toContain('class Tools(Protocol):') + const runCodeSchema = assembly.tools.find(tool => tool.name === RUN_CODE_NAME) + expect(runCodeSchema?.description).toContain('Execute a Python program') + // `both` keeps the native tools alongside run_code; `code` does not. + expect(assembly.tools.map(tool => tool.name)).toContain('echo') + }) + + it('emits a TypeScript-flavored run_code schema under a typescript runtime', async () => { + const { ctx, systemPrompt } = await setup({ mode: 'code', runtime: { language: 'typescript' } }) + registerEcho(ctx) + const assembly = await systemPrompt.assemble() + const runCodeSchema = assembly.tools.find(tool => tool.name === RUN_CODE_NAME) + expect(runCodeSchema?.description).toContain('Execute a TypeScript program') + expect(runCodeSchema?.description).toContain('BODY of an') + const codeParam = (runCodeSchema?.parameters as { properties: { code: { description: string } } }).properties.code + expect(codeParam.description).toBe('The program: the body of an async TypeScript function.') + }) + + it('emits a Python-flavored run_code schema under a python runtime (matches the SDK language)', async () => { + const { ctx, systemPrompt } = await setup({ mode: 'code', runtime: { language: 'python' } }) + registerEcho(ctx) + const assembly = await systemPrompt.assemble() + const runCodeSchema = assembly.tools.find(tool => tool.name === RUN_CODE_NAME) + expect(runCodeSchema?.description).toContain('Execute a Python program') + expect(runCodeSchema?.description).toContain('`return `') + expect(runCodeSchema?.description).not.toContain('TypeScript') + const codeParam = (runCodeSchema?.parameters as { properties: { code: { description: string } } }).properties.code + expect(codeParam.description).toBe('The program: the body of an async Python function.') + }) + + it('resolves the run_code schema flavor lazily and fails loud on a language absent from the flavor table', async () => { + // The flavor getter reads the runtime directly (peekRuntime), so it — not + // requireCodeRuntime — owns the flavor-table guard. Keeping + // RUN_CODE_FLAVORS in step with SDK_RENDERERS is the compiler's job (both + // are `satisfies`-checked against CodeSdkLanguage), so what the guard + // covers is a mounted runtime naming a language absent from both tables, + // which throws when the schema is projected. Assembly's + // requireCodeRuntime rejects such a language earlier; this reaches the + // guard on its own. + const { ctx } = await setup({ mode: 'code', runtime: { language: 'ruby' } }) + const definition = ctx.tools.get(RUN_CODE_NAME) + // Names the known languages, symmetric with the SDK_RENDERERS guard: this + // is the reachable rejection, so it must be at least as diagnosable. + expect(() => definition?.description) + .toThrow(/no run_code schema flavor registered for runtime language "ruby" \(known: "typescript", "python"\)/) + }) + + it('degrades the run_code flavor to TypeScript when no runtime is mounted', async () => { + // Any reader of the definition without a mounted runtime lands here; the + // shipped one is the tool-catalog generator, which boots the registry under + // `mode: code` and reads run_code's schema WITHOUT a runtime. peekRuntime + // returns undefined there, so the flavor getter degrades to the TS default + // rather than throwing. None of those readers feeds a model: assembly goes + // through wireSchemas, which requires a runtime first. + const { ctx } = await setup({ mode: 'code', runtime: false }) + const definition = ctx.tools.get(RUN_CODE_NAME) + expect(definition?.description).toContain('Execute a TypeScript program') + const params = definition?.parameters as { properties: { code: { description: string } } } + expect(params.properties.code.description).toBe('The program: the body of an async TypeScript function.') }) it("rejects the assembly when toolOrder names a native tool that mode 'code' no longer contributes", async () => { diff --git a/packages/core/tools/tests/py-types.spec.ts b/packages/core/tools/tests/py-types.spec.ts new file mode 100644 index 0000000000..3439cb6a37 --- /dev/null +++ b/packages/core/tools/tests/py-types.spec.ts @@ -0,0 +1,1163 @@ +import { describe, expect, it } from 'vitest' +import { jsonSchemaToPy, renderToolsSdkPy } from '@deepseek-ai/dsh-tools/src/py-types.ts' +import { parameterSchemaSpecToJsonSchema } from '@deepseek-ai/dsh-tools' +import type { ToolSdkSchema } from '@deepseek-ai/dsh-tools/src/ts-types.ts' + +describe('jsonSchemaToPy', () => { + it('maps the defineTool DSL subset', () => { + const cases: [unknown, string][] = [ + [{ type: 'string' }, 'str'], + [{ type: 'number' }, 'float'], + [{ type: 'boolean' }, 'bool'], + [{ type: 'string', enum: ['a', 'b'] }, 'Literal["a", "b"]'], + [{ type: 'array', items: { type: 'number' } }, 'list[float]'], + [{ type: 'array', items: { type: 'string', enum: ['x', 'y'] } }, 'list[Literal["x", "y"]]'], + [{ type: 'array' }, 'list[Any]'], + [{ type: 'object' }, 'dict[str, Any]'], + [{ type: 'object', properties: {} }, 'dict[str, Any]'], + [{ type: 'object', properties: { x: { type: 'string' } } }, 'dict[str, Any]'], + ] + for (const [schema, expected] of cases) { + expect(jsonSchemaToPy(schema), JSON.stringify(schema)).toBe(expected) + } + }) + + it('is total: unsupported or hostile constructs degrade to Any, never throw', () => { + const cases: unknown[] = [ + undefined, + null, + 42, + 'string-schema', + {}, + { oneOf: 7 }, + { $ref: '#/defs/x' }, + { type: 'object', properties: 7 }, + { type: 'string', enum: [1, 2] }, + { type: 'string', enum: [] }, + ] + for (const schema of cases) { + expect(() => jsonSchemaToPy(schema), JSON.stringify(schema)).not.toThrow() + } + expect(jsonSchemaToPy({ type: 'integer' })).toBe('int') + expect(jsonSchemaToPy({ type: 'string', const: 'fixed' })).toBe('Literal["fixed"]') + expect(jsonSchemaToPy({ type: 'boolean', const: true })).toBe('Literal[True]') + expect(jsonSchemaToPy({ type: 'number', const: 1.5 })).toBe('Literal[1.5]') + expect(jsonSchemaToPy({ type: 'boolean', enum: [false] })).toBe('Literal[False]') + expect(jsonSchemaToPy({ type: 'null' })).toBe('None') + expect(jsonSchemaToPy({ oneOf: [{ type: 'string' }, { type: 'null' }] })).toBe('str | None') + expect(jsonSchemaToPy({ oneOf: [] })).toBe('Any') + expect(jsonSchemaToPy({ type: 'object', properties: 7 })).toBe('Any') + expect(jsonSchemaToPy({ type: 'string', enum: [1, 2] })).toBe('Any') + expect(jsonSchemaToPy({ type: 'string', enum: [] })).toBe('Any') + }) + + it('leans on JSON.stringify to keep a Literal parseable', () => { + // Nothing here escapes anything itself; `JSON.stringify` carries both + // classes of hazard. The two kinds of code point CPython refuses anywhere + // in source: NUL, and the D800–DFFF unpaired-surrogate block under ES2019 + // well-formed stringification. + expect(jsonSchemaToPy({ type: 'string', const: 'a\u0000b' })).toBe(String.raw`Literal["a\u0000b"]`) + expect(jsonSchemaToPy({ type: 'string', enum: ['a\ud800b'] })).toBe(String.raw`Literal["a\ud800b"]`) + // And the ones that break this line in particular: a bare quote closing + // the literal early, a trailing ODD backslash eating the closing quote (an + // even run does not), a bare newline ending it before its terminator. + // Every escape it emits is also a Python escape for the same character, so + // the value round-trips. + expect(jsonSchemaToPy({ type: 'string', const: 'say "hi"\n' })).toBe(String.raw`Literal["say \"hi\"\n"]`) + expect(jsonSchemaToPy({ type: 'string', const: 'ends\\' })).toBe(String.raw`Literal["ends\\"]`) + }) + + it('passes NEL and the line/paragraph separators through raw, which CPython does not treat as line terminators', () => { + // `JSON.stringify` escapes LF and CR but not NEL (U+0085), LS (U+2028), or + // PS (U+2029), which is safe here and not by accident: those three are + // `str.splitlines()` boundaries, not tokenizer line terminators, so they + // end neither a string literal nor a `#` comment — measured on CPython + // 3.9.6 and 3.12.13. Pinning the raw form keeps a later "escape them for + // symmetry with LF" change from landing as a silent both-flavors + // divergence from `ts-types`. Escapes below — the two forms denote the + // same bytes, and none of the three has a visible width. + expect(jsonSchemaToPy({ type: 'string', const: 'a\u2028b' })).toBe('Literal["a\u2028b"]') + expect(jsonSchemaToPy({ type: 'string', enum: ['a\u2029b'] })).toBe('Literal["a\u2029b"]') + // NEL is inside `UNPRINTABLE`'s class, so the description path escapes it. + // This is one of the two routes that carry it raw; the other is the + // subscript tool-name comment's own `JSON.stringify` call. + expect(jsonSchemaToPy({ type: 'string', const: 'a\u0085b' })).toBe('Literal["a\u0085b"]') + }) + + it('emits exact digits for a beyond-safe-range integer literal', () => { + // Python integers are arbitrary-precision, so the emitted digits ARE the + // value the model programs against. `String(2 ** 60)` prints the rounded + // ...847000, which is a DIFFERENT integer from the double's exact + // ...846976: `Number::toString` is shortest round-trip, so it emits the 16 + // digits that re-read to the same double and pads with zeros, and those + // padded digits name an integer no double holds. Passing one back would + // have to cross the argument boundary as a JSON number, so the SDK would + // document a value no program can pass. This assertion is what separates + // the two spellings; the 1e21 case below separates them again on the other + // failure mode, where `String` gives no integer literal at all. + expect(jsonSchemaToPy({ type: 'integer', const: 2 ** 60 })).toBe('Literal[1152921504606846976]') + expect(jsonSchemaToPy({ type: 'integer', enum: [2 ** 60, -(2 ** 60)] })) + .toBe('Literal[1152921504606846976, -1152921504606846976]') + // `String(1e21)` prints `1e+21`, not a Python integer literal at all. The + // rule keys off the VALUE, not the declared type, so a `number` const that + // happens to be an integral double is spelled the same exact way (both + // spellings denote the same double, and only the digits also denote the + // same Python integer). + expect(jsonSchemaToPy({ type: 'integer', const: 1e21 })).toBe('Literal[1000000000000000000000]') + expect(jsonSchemaToPy({ type: 'number', const: 1e21 })).toBe('Literal[1000000000000000000000]') + // Within the safe range, and for non-integral numbers, the plain spelling + // is already exact and stays unchanged. + expect(jsonSchemaToPy({ type: 'integer', const: 2 ** 53 - 1 })).toBe('Literal[9007199254740991]') + expect(jsonSchemaToPy({ type: 'number', const: 1e-7 })).toBe('Literal[1e-7]') + }) +}) + +describe('renderToolsSdkPy', () => { + const bash: ToolSdkSchema = { + name: 'bash', + description: 'Run a shell command.', + parameters: parameterSchemaSpecToJsonSchema({ command: { type: 'string', required: true } }) as unknown as Record, + output: { type: 'string' }, + } + const exotic: ToolSdkSchema = { + name: 'my-mcp.tool', + description: 'Exotic name.', + parameters: parameterSchemaSpecToJsonSchema({}) as unknown as Record, + output: { type: 'string' }, + } + const reserved: ToolSdkSchema = { + name: 'class', + description: 'Uses a reserved Python word.', + parameters: parameterSchemaSpecToJsonSchema({}) as unknown as Record, + output: { type: 'string' }, + } + /** One tool carrying `description` at both emission sites: the method docstring and the field comment. */ + const described = (description: string): ToolSdkSchema => ({ + name: 'weird', + description, + parameters: parameterSchemaSpecToJsonSchema({ + field: { type: 'string', required: true, description }, + }) as unknown as Record, + output: { type: 'string' }, + }) + + it('declares identifier tools as async methods and lists exotic/reserved names as subscript comments', () => { + const text = renderToolsSdkPy([exotic, bash, reserved]) + expect(text).toContain('class Tools(Protocol):') + // The argument object is a named TypedDict, not an opaque dict. + expect(text).toContain('class BashArgs(TypedDict):') + expect(text).toContain('async def bash(self, args: BashArgs) -> str:') + // Empty-property tools keep the opaque dict (nothing to name). + expect(text).toContain('# tools["my-mcp.tool"](args: dict[str, Any]) -> str') + expect(text).toContain('# tools["class"](args: dict[str, Any]) -> str') + // Fixed instruction lines the model relies on. + expect(text).toContain('top-level `await`') + // The binding boundary: `tools`/`ToolCallError` are bound, the TypedDicts + // are not. Both halves are pinned — dropping either one turns a correct + // contract into a wrong one (a model that reads only "STATIC STUB" would + // stop catching `ToolCallError`). + expect(text).toContain('exactly two of the names declared below are bound: `tools` and `ToolCallError`') + expect(text).toContain('never `FooArgs(field=1)`, which raises `NameError`') + expect(text).toContain('ToolCallError') + expect(text).toContain('class ToolCallError(Exception):') + expect(text).toContain('MAY overlap under `asyncio.gather`') + expect(text).toContain('lossless JSON') + expect(text).toContain('```python') + expect(text).toContain('tools: Tools') + }) + + it('renders required as plain fields and optional as NotRequired, with per-field description comments', () => { + const tool: ToolSdkSchema = { + name: 'search', + description: 'Search for text.', + parameters: parameterSchemaSpecToJsonSchema({ + query: { type: 'string', required: true, description: 'What to search for.' }, + limit: { type: 'number', description: 'Max results.' }, + }) as unknown as Record, + output: { type: 'string' }, + } + const text = renderToolsSdkPy([tool]) + expect(text).toContain('class SearchArgs(TypedDict):') + expect(text).toContain(' # What to search for.') + expect(text).toContain(' query: str') + expect(text).toContain(' # Max results.') + expect(text).toContain(' limit: NotRequired[float]') + expect(text).toContain('async def search(self, args: SearchArgs) -> str:') + // NotRequired is imported because an optional field used it; Any is NOT, + // since every type here is concrete — the import line lists only what ran. + expect(text).toContain('from typing import NotRequired, Protocol, TypedDict') + }) + + it('prefixes Tool when a name CamelCases to a non-letter head, and degrades a malformed schema to Any', () => { + const tool: ToolSdkSchema = { + name: '1st-tool', // subscript path; CamelCases to "1stTool" → prefixed "Tool1stTool" + description: 'Hostile-shape probe.', + // Malformed node: the unified schema validator rejects it whole, so the + // args position degrades to Any (registration would refuse this schema; + // the renderer just must not throw on it). + parameters: { type: 'object', properties: { field: { type: 'string', description: 42 } } }, + output: { type: 'object', additionalProperties: false, properties: { ok: { type: 'boolean' } }, required: ['ok'] }, + } + const text = renderToolsSdkPy([tool]) + expect(text).toContain('# tools["1st-tool"](args: Any) -> Tool1stToolOutput') + expect(text).toContain('class Tool1stToolOutput(TypedDict):') + expect(text).toContain(' ok: bool') + }) + + it('treats every field as optional when the object carries no required array', () => { + const tool: ToolSdkSchema = { + name: 'all_optional', + description: 'No required array.', + parameters: { type: 'object', properties: { flag: { type: 'boolean' } } }, + output: { type: 'string' }, + } + const text = renderToolsSdkPy([tool]) + expect(text).toContain(' flag: NotRequired[bool]') + }) + + it('renders an enum inside an object property as a Literal field', () => { + const tool: ToolSdkSchema = { + name: 'mode_tool', + description: 'Pick a mode.', + parameters: parameterSchemaSpecToJsonSchema({ + mode: { type: 'string', required: true, enum: ['fast', 'slow'] }, + }) as unknown as Record, + output: { type: 'string' }, + } + const text = renderToolsSdkPy([tool]) + expect(text).toContain('class ModeToolArgs(TypedDict):') + expect(text).toContain(' mode: Literal["fast", "slow"]') + expect(text).toContain('from typing import Literal, Protocol, TypedDict') + }) + + it('renders one level of nested object as its own named TypedDict declared before the parent', () => { + const tool: ToolSdkSchema = { + name: 'workflow', + description: 'Run a workflow.', + parameters: parameterSchemaSpecToJsonSchema({ + meta: { + type: 'object', + required: true, + additionalProperties: false, + description: 'Identity block.', + properties: { + name: { type: 'string', required: true, description: 'Short name.' }, + phases: { + type: 'array', + items: { + type: 'object', + additionalProperties: false, + properties: { title: { type: 'string', required: true, description: 'Phase title.' } }, + }, + }, + }, + }, + }) as unknown as Record, + output: { type: 'string' }, + } + const text = renderToolsSdkPy([tool]) + // Nested class for the `meta` object, and a further nested class for the + // array item object, each named after its field path. + expect(text).toContain('class WorkflowArgsMeta(TypedDict):') + expect(text).toContain('class WorkflowArgsMetaPhases(TypedDict):') + expect(text).toContain(' meta: WorkflowArgsMeta') + expect(text).toContain(' phases: NotRequired[list[WorkflowArgsMetaPhases]]') + // Dependency-before-dependent: the item class precedes its container, + // which precedes the top-level args class, which precedes the protocol. + expect(text.indexOf('class WorkflowArgsMetaPhases')).toBeLessThan(text.indexOf('class WorkflowArgsMeta(TypedDict):')) + expect(text.indexOf('class WorkflowArgsMeta(TypedDict):')).toBeLessThan(text.indexOf('class WorkflowArgs(TypedDict):')) + expect(text.indexOf('class WorkflowArgs(TypedDict):')).toBeLessThan(text.indexOf('class Tools(Protocol):')) + }) + + it('renders a oneOf of object branches as a union of named TypedDicts declared before the parent', () => { + const tool: ToolSdkSchema = { + name: 'act', + description: 'Union output.', + parameters: { type: 'object', additionalProperties: false, properties: {} }, + output: { + oneOf: [ + { type: 'object', additionalProperties: false, properties: { ok: { type: 'boolean' } }, required: ['ok'] }, + { type: 'object', additionalProperties: false, properties: { err: { type: 'string' } }, required: ['err'] }, + ], + }, + } + const text = renderToolsSdkPy([tool]) + // Each object branch becomes its own named class (`${base}Output1/2`), + // declared before the protocol references the union. + expect(text).toContain('class ActOutput1(TypedDict):') + expect(text).toContain('class ActOutput2(TypedDict):') + expect(text).toContain('-> ActOutput1 | ActOutput2') + expect(text.indexOf('class ActOutput1(TypedDict):')).toBeLessThan(text.indexOf('class Tools(Protocol):')) + expect(text.indexOf('class ActOutput2(TypedDict):')).toBeLessThan(text.indexOf('class Tools(Protocol):')) + }) + + it('degrades a context-free oneOf of object branches to a union of dict[str, Any]', () => { + // jsonSchemaToPy has no naming context, so each object branch degrades + // rather than declaring a class. + const type = jsonSchemaToPy({ + oneOf: [ + { type: 'object', additionalProperties: false, properties: { ok: { type: 'boolean' } }, required: ['ok'] }, + { type: 'string' }, + ], + }) + expect(type).toBe('dict[str, Any] | str') + // Both branches objects, and the same shape reached through an array: the + // marker is the CALL's className, so a propagated frame name (`1`, the + // index-derived branch name) does not revive class declaration on a walk + // that has nowhere to declare into. + const object = { type: 'object', additionalProperties: false, properties: { ok: { type: 'boolean' } }, required: ['ok'] } + expect(jsonSchemaToPy({ oneOf: [object, object] })).toBe('dict[str, Any] | dict[str, Any]') + expect(jsonSchemaToPy({ type: 'array', items: { oneOf: [object, { type: 'string' }] } })).toBe('list[dict[str, Any] | str]') + }) + + it('suffixes a counter when two tools CamelCase to the same class base', () => { + const a: ToolSdkSchema = { + name: 'my-tool', + description: 'Dash form.', + parameters: parameterSchemaSpecToJsonSchema({ x: { type: 'string', required: true } }) as unknown as Record, + output: { type: 'string' }, + } + const b: ToolSdkSchema = { + name: 'my.tool', + description: 'Dot form.', + parameters: parameterSchemaSpecToJsonSchema({ y: { type: 'string', required: true } }) as unknown as Record, + output: { type: 'string' }, + } + const text = renderToolsSdkPy([a, b]) + // Both sanitize to `MyToolArgs`; the second collides and gets a suffix. + expect(text).toContain('class MyToolArgs(TypedDict):') + expect(text).toContain('class MyToolArgs2(TypedDict):') + }) + + it('caps class-name length so a deep single-field chain stays linear', () => { + // Child class names derive from their parent's, so without a cap the sum of + // names would be Theta(depth^2). MAX_CLASS_NAME_BASE (120) bounds each name. + const depth = 4000 + let schema: Record = { type: 'string' } + for (let i = 0; i < depth; i++) { + schema = { type: 'object', additionalProperties: false, properties: { inner: schema }, required: ['inner'] } + } + const tool: ToolSdkSchema = { name: 'deep', description: 'Deep chain.', parameters: schema, output: { type: 'string' } } + const text = renderToolsSdkPy([tool]) + const longestClassName = [...text.matchAll(/^class (\w+)\(TypedDict\):/gm)].reduce((max, m) => Math.max(max, m[1]?.length ?? 0), 0) + expect(longestClassName).toBeLessThanOrEqual(140) + expect(text.length).toBeLessThan(depth * 400) + }) + + it('skips an already-taken counter suffix when a sibling object occupies it', () => { + // `phase` and `Phase` both CamelCase to base `FooArgsPhase`; `phase2` + // independently takes `FooArgsPhase2`, so `Phase`'s collision scan must + // advance to `FooArgsPhase3` (exercises the collision-skip loop). + const obj = (field: string) => ({ type: 'object' as const, additionalProperties: false, properties: { [field]: { type: 'string' } } }) + const tool: ToolSdkSchema = { + name: 'foo', + description: 'Sibling objects with colliding class bases.', + parameters: { + type: 'object', + additionalProperties: false, + properties: { phase: obj('a'), phase2: obj('b'), Phase: obj('c') }, + required: ['phase', 'phase2', 'Phase'], + }, + output: { type: 'string' }, + } + const text = renderToolsSdkPy([tool]) + expect(text).toContain('class FooArgsPhase(TypedDict):') + expect(text).toContain('class FooArgsPhase2(TypedDict):') + expect(text).toContain('class FooArgsPhase3(TypedDict):') + }) + + it('references the named TypedDict from a reserved/subscript tool too', () => { + const tool: ToolSdkSchema = { + name: 'class', + description: 'Reserved word tool.', + parameters: parameterSchemaSpecToJsonSchema({ value: { type: 'string', required: true } }) as unknown as Record, + output: { type: 'string' }, + } + const text = renderToolsSdkPy([tool]) + expect(text).toContain('class ClassArgs(TypedDict):') + expect(text).toContain('# tools["class"](args: ClassArgs) -> str') + }) + + it('degrades an object to dict[str, Any] when a field name is not a legal Python attribute', () => { + const tool: ToolSdkSchema = { + name: 'weird_fields', + description: 'Has an illegal field name.', + parameters: { type: 'object', properties: { 'a-b': { type: 'string' } } }, + output: { type: 'string' }, + } + const text = renderToolsSdkPy([tool]) + expect(text).toContain('async def weird_fields(self, args: dict[str, Any]) -> str:') + expect(text).not.toContain('WeirdFieldsArgs') + }) + + it('keeps soft-keyword field names as TypedDict fields (each is special in exactly one syntactic position)', () => { + const tool: ToolSdkSchema = { + name: 'search', + description: 'Soft keywords as fields.', + parameters: { + type: 'object', + additionalProperties: false, + properties: { + match: { type: 'string' }, + case: { type: 'boolean' }, + type: { type: 'string' }, + }, + required: ['match'], + }, + output: { type: 'string' }, + } + const text = renderToolsSdkPy([tool]) + // The object keeps its shape rather than degrading to dict[str, Any]. + expect(text).toContain('class SearchArgs(TypedDict):') + expect(text).toContain('match: str') + expect(text).toContain('case: NotRequired[bool]') + expect(text).toContain('type: NotRequired[str]') + expect(text).not.toContain('dict[str, Any]') + }) + + it('keeps a non-ASCII field name as a TypedDict field and derives its class name from it', () => { + // `路径` satisfies `xid_start xid_continue*`, so CPython accepts it as an + // attribute and as the `TypedDict` key. Rejecting it would degrade the + // whole object, dropping every SIBLING field's name, requiredness and type + // too — and under `mode: 'code'` the native schemas are omitted, so + // nothing else carries them. The nested class name is from the field, so + // `camelCase` has to pass the same characters through instead of splitting + // on them. + const tool: ToolSdkSchema = { + name: '搜索', + description: 'Unicode identifiers.', + parameters: { + type: 'object', + additionalProperties: false, + properties: { + 路径: { type: 'string' }, + opts: { type: 'object', additionalProperties: false, properties: { 深度: { type: 'number' } } }, + }, + required: ['路径'], + }, + output: { type: 'string' }, + } + const text = renderToolsSdkPy([tool]) + expect(text).toContain('async def 搜索(self, args: 搜索Args) -> str:') + expect(text).toContain('class 搜索Args(TypedDict):') + expect(text).toContain(' 路径: str') + expect(text).toContain('class 搜索ArgsOpts(TypedDict):') + expect(text).toContain(' 深度: NotRequired[float]') + expect(text).not.toContain('dict[str, Any]') + }) + + it('degrades a field name that NFKC-normalizes to something else, which would be declared under another spelling', () => { + // U+FB01 LATIN SMALL LIGATURE FI passes the identifier grammar, but CPython + // normalizes identifiers at compile time while the harness compares the + // JSON key as written: `field: str` would declare and be reachable as + // `field`, a key the tool never accepts. Two keys that normalize together + // would additionally collapse into one declaration. The subscript path + // carries the exact bytes instead. + const text = renderToolsSdkPy([ + { + name: 'ligature', + description: 'Normalizing field name.', + parameters: { type: 'object', additionalProperties: false, properties: { field: { type: 'string' } } }, + output: { type: 'string' }, + }, + ]) + expect(text).toContain('async def ligature(self, args: dict[str, Any]) -> str:') + expect(text).not.toContain('field:') + expect(text).not.toContain('field:') + }) + + it('keeps U+200C in a name tail while rejecting it at a name head, per the two XID properties', () => { + // ZWNJ carries `XID_Continue` and not `XID_Start`, so the predicate splits + // on position: bare in a tail, subscripted at a head. Both verdicts are + // stable across the supported engines — the property arrives in Unicode + // 15.1 and the floor (Node 22.19.0, Unicode 16.0) is past it. + // + // The interpreter side is where this one skews, and it is the same skew the + // docstring's four other characters record, reached in a tail position + // instead of at a head: CPython reads XID_Continue out of the + // `DerivedCoreProperties.txt` of the UCD it was built against (13.0.0 on + // 3.9.6 and 15.0.0 on 3.12.13 both lack the row, and + // `'a\u200Cb'.isidentifier()` is False on both, measured). What then needs + // 15.1 tables or newer is the bare field, once in each class, and the + // `Tool\u200CbArgs` class name. The subscript comment quoting the tool name + // is not one of them: it is not parsed as an identifier. + const of = (name: string): ToolSdkSchema => ({ + name, + description: `Tool ${name}.`, + parameters: { type: 'object', additionalProperties: false, properties: { 'a\u200Cb': { type: 'string' } } }, + output: { type: 'string' }, + }) + const text = renderToolsSdkPy([of('ping'), of('\u200Cb')]) + expect(text).toContain('async def ping(self, args: PingArgs) -> str:') + expect(text).toContain(' a\u200Cb: NotRequired[str]') + // A head that is XID_Continue but not XID_Start takes the subscript path, + // and `camelCase` prefixes `Tool` to make the class name start legally. + expect(text).toContain('# tools["\u200Cb"](args: Tool\u200CbArgs) -> str') + expect(text).toContain('class Tool\u200CbArgs(TypedDict):') + expect(text).not.toContain('async def \u200Cb') + }) + + it('subscripts a tool name that NFKC-normalizes to something else, while declaring a plain Unicode one', () => { + // Same split at the tool-name site: `路径` becomes an `async def`, the + // ligature name cannot, because `async def find` would define `find`. The + // subscript comment quotes the name, so its exact bytes survive, and its + // TypedDict is still named and referenced — the name is only unusable as a + // method, not as a class-name source. The `FInd` spelling comes from `fi`'s + // multi-character full case mapping (`'fi'.toUpperCase()` is `'FI'`), not + // from `camelCase`'s NFKC step, which is the identity on `FInd`: the + // ligature is XID_Start, so the split set keeps it and only the + // capitalization of the head transforms it. + const of = (name: string): ToolSdkSchema => ({ + name, + description: `Tool ${name}.`, + parameters: { type: 'object', additionalProperties: false, properties: { q: { type: 'string' } }, required: ['q'] }, + output: { type: 'string' }, + }) + const text = renderToolsSdkPy([of('路径'), of('find')]) + expect(text).toContain('async def 路径(self, args: 路径Args) -> str:') + expect(text).toContain('# tools["find"](args: FIndArgs) -> str') + expect(text).toContain('class FIndArgs(TypedDict):') + expect(text).not.toContain('async def find') + expect(text).not.toContain('async def find') + }) + + it('derives a class name through the case-mapping table, independently of the bare-name predicate', () => { + // The head capitalization reads a table `isBareIdentifier` never consults, + // so the class-name path can carry a character the predicate cleared. ƛ + // (U+019B) is XID_Start and NFKC-stable, so the method is emitted bare; + // the head maps to Ƛ (U+A7DC), a code point the engine's tables assign and + // an older interpreter's do not. This pins which table produced the name, + // so a change to the mapping step shows up here rather than only in a + // downstream Python parse. + // + // Unlike the other Unicode cases in this file, the table row is recent: + // U+A7DC and the U+019B uppercase mapping to it both arrive in Unicode + // 16.0 (`DerivedAge.txt`; CPython 3.12.13's 15.0.0 has neither). The + // engines floor sits exactly there with no margin — Node 22.19.0 reports + // Unicode 16.0 (ICU 77.1) and maps U+019B to U+A7DC, measured — so an + // engine below the floor fails here as a renderer regression whose real + // cause is the table version. + const text = renderToolsSdkPy([ + { + name: 'ƛ', + description: 'Lambda with stroke.', + parameters: { type: 'object', additionalProperties: false, properties: { q: { type: 'string' } }, required: ['q'] }, + output: { type: 'string' }, + }, + ]) + expect(text).toContain('async def ƛ(self, args: ꟜArgs) -> str:') + expect(text).toContain('class ꟜArgs(TypedDict):') + expect(text).not.toContain('class ƛArgs') + }) + + it('drops a surrogate half rather than cutting a pair when capping an astral class-name base', () => { + // Class-name bases are capped by `slice`, which counts UTF-16 code units, + // so a boundary landing inside an astral pair would leave a lone high + // surrogate — not an identifier character, and not encodable text. Padding + // with one ASCII character shifts the boundary onto the pair. + // U+10330 GOTHIC LETTER AHSA: XID_Start and NFKC-stable, unlike `𝕏`, which + // NFKC-folds to ASCII `X` and so never reaches the boundary at all. + const AHSA = String.fromCodePoint(0x10330) + const className = (pad: string): string => { + const text = renderToolsSdkPy([ + { + name: `${pad}${AHSA.repeat(200)}`, + description: 'Astral name.', + parameters: { type: 'object', additionalProperties: false, properties: { a: { type: 'string' } } }, + output: { type: 'string' }, + }, + ]) + // The base is `${camelCase(name)}Args` capped to 120 code units, so the + // `Args` suffix itself is cut off here; match the declaration instead. + return /^class (.+)\(TypedDict\):$/mu.exec(text)![1]! + } + // Each character is 2 code units, so an unpadded name fills the cap with 60 + // whole characters; one ASCII character of padding puts the boundary inside + // the 60th pair, and that half is dropped rather than emitted. + expect(className('')).toBe(AHSA.repeat(60)) + const padded = className('x') + expect(padded).toBe(`X${AHSA.repeat(59)}`) + expect(padded).toHaveLength(119) + }) + + it('normalizes the seam the Tool prefix creates, which the prefixed part alone does not cover', () => { + // U+0301 COMBINING ACUTE ACCENT is XID_Continue but not XID_Start, so a name + // headed by it takes the `Tool` prefix — and `Tool` ends in `l`, which + // composes with it. Normalizing only the part being prefixed would emit + // `Tool` + U+0301, which CPython compiles as `Too` + U+013A: the class + // the SDK declares would not be the class the interpreter defines. Every + // code point below is an escape — the two forms render identically. + const text = renderToolsSdkPy([ + { + name: '\u0301abc', + description: 'Combining-mark head.', + parameters: { type: 'object', additionalProperties: false, properties: { q: { type: 'string' } }, required: ['q'] }, + output: { type: 'string' }, + }, + ]) + expect(text).toContain('class Too\u013AabcArgs(TypedDict):') + expect(text).toContain('# tools["\u0301abc"](args: Too\u013AabcArgs) -> str') + expect(text).not.toContain('Tool\u0301') + }) + + it('normalizes a class-name join where two separately stable segments compose', () => { + // Hangul jamo compose ACROSS the join `childClassName` makes: the parent + // base ends in U+1100 (L jamo) and the child segment starts with U+1161 (V + // jamo), each NFKC-stable alone, together U+AC00. Unnormalized, the declared + // name differs from the compiled symbol, and two byte-distinct names can + // fold onto one — `usedClassNames` dedupes by raw bytes, so the collision + // counter never sees it and the later declaration shadows the earlier one + // under CPython. Escapes again, for the same reason as above. + const text = renderToolsSdkPy([ + { + name: 'x', + description: 'Jamo field names.', + parameters: { + type: 'object', + additionalProperties: false, + required: ['\uAC00\u1100'], + properties: { + '\uAC00\u1100': { + type: 'object', + additionalProperties: false, + required: ['\u1161x'], + properties: { + '\u1161x': { type: 'object', additionalProperties: false, properties: { q: { type: 'string' } } }, + }, + }, + }, + }, + output: { type: 'string' }, + }, + ]) + // The join is `XArgs` + U+AC00 U+1100 followed by U+1161 `x`, whose + // trailing L+V pair composes into a second U+AC00. + expect(text).toContain('class XArgs\uAC00\uAC00x(TypedDict):') + expect(text).toContain(' \u1161x: XArgs\uAC00\uAC00x') + expect(text).not.toContain('\u1100\u1161') + // The level above it is a join that composes nothing (LV + L), so it stays + // byte-identical — normalizing is not silently rewriting every name. + expect(text).toContain('class XArgs\uAC00\u1100(TypedDict):') + }) + + it('routes a fold collision through the counter that raw-byte dedup would miss', () => { + // The other half of the `childClassName` normalization: two joins that are + // byte-distinct before NFKC and identical after. Field `\uAC00` allocates + // `XArgs\uAC00`; the sibling `\u1100` allocates `XArgs\u1100`, and ITS child + // `\u1161` joins to `XArgs\u1100\u1161` — the same `XArgs\uAC00` once composed. + // Normalizing at the join is what lets `usedClassNames`, which dedupes by raw + // bytes, see the collision at all; unnormalized, both would be declared and + // CPython would compile the second as a shadow of the first. + const text = renderToolsSdkPy([ + { + name: 'x', + description: 'Colliding jamo joins.', + parameters: { + type: 'object', + additionalProperties: false, + required: ['\uAC00', '\u1100'], + properties: { + '\uAC00': { type: 'object', additionalProperties: false, required: ['q'], properties: { q: { type: 'string' } } }, + '\u1100': { + type: 'object', + additionalProperties: false, + required: ['\u1161'], + properties: { + '\u1161': { type: 'object', additionalProperties: false, required: ['q'], properties: { q: { type: 'string' } } }, + }, + }, + }, + }, + output: { type: 'string' }, + }, + ]) + expect(text).toContain('class XArgs\uAC00(TypedDict):') + expect(text).toContain('class XArgs\uAC002(TypedDict):') + expect(text).toContain(' \u1161: XArgs\uAC002') + }) + + it('names both branches of a oneOf of objects on the argument side', () => { + // The output side is pinned elsewhere; arguments reach the same + // `childClassName(frame.className, index + 1)` path, and the annotation is + // the union of the two derived names rather than a degraded dict. + const text = renderToolsSdkPy([ + { + name: 'x', + description: 'Union arguments.', + parameters: { + oneOf: [ + { type: 'object', additionalProperties: false, required: ['a'], properties: { a: { type: 'string' } } }, + { type: 'object', additionalProperties: false, required: ['b'], properties: { b: { type: 'number' } } }, + ], + }, + output: { type: 'string' }, + }, + ]) + expect(text).toContain('class XArgs1(TypedDict):') + expect(text).toContain('class XArgs2(TypedDict):') + expect(text).toContain('async def x(self, args: XArgs1 | XArgs2) -> str:') + }) + + it('declares a closed empty object with omitted properties as an empty TypedDict, not dict[str, Any]', () => { + // `{ type: 'object', additionalProperties: false }` with no `properties` + // is a closed empty object — no key accepted — exactly as the validator + // and the TS renderer read it. It must not degrade to a permissive dict. + const tool: ToolSdkSchema = { + name: 'closed', + description: 'Closed empty object with omitted properties.', + parameters: { + type: 'object', + additionalProperties: false, + properties: { inner: { type: 'object', additionalProperties: false } }, + required: ['inner'], + }, + output: { type: 'string' }, + } + const text = renderToolsSdkPy([tool]) + expect(text).toMatch(/class ClosedArgsInner\(TypedDict\):\n pass/) + expect(text).toContain('inner: ClosedArgsInner') + expect(text).not.toContain('dict[str, Any]') + }) + + it('degrades an open object with omitted properties to dict[str, Any]', () => { + // An OPEN empty object (default additionalProperties) is any dict. + const type = jsonSchemaToPy({ type: 'object', properties: {} }) + expect(type).toBe('dict[str, Any]') + expect(jsonSchemaToPy({ type: 'object' })).toBe('dict[str, Any]') + }) + + it('renders docstrings for descriptions and orders emissions lexicographically', () => { + const text = renderToolsSdkPy([bash, exotic]) + expect(text).toContain('"""Run a shell command."""') + // Descriptions on subscript names ride as a comment beside their entry. + expect(text).toContain('# tools["my-mcp.tool"]') + expect(text).toContain('# Exotic name.') + // Lexicographic: `bash` before `my-mcp.tool`. + expect(text.indexOf('async def bash')).toBeLessThan(text.indexOf('# tools["my-mcp.tool"]')) + }) + + it('places a docstring as the first statement of its own method body', () => { + // Python attaches a docstring to a function only when it is that + // function's first statement. Above the `async def` the first one would + // document the `Tools` class and every later one would be a dead + // expression, so each method must open its body with its own docstring. + const second: ToolSdkSchema = { + name: 'zzz', + description: 'Second by name.', + parameters: parameterSchemaSpecToJsonSchema({}) as unknown as Record, + output: { type: 'string' }, + } + const lines = renderToolsSdkPy([bash, second]).split('\n') + for (const [name, doc] of [['bash', 'Run a shell command.'], ['zzz', 'Second by name.']]) { + const signature = lines.findIndex(line => line.startsWith(`${' '.repeat(4)}async def ${name}(`)) + expect(signature).toBeGreaterThan(-1) + // Ends in `:`, not the `: ...` stub — a docstring IS the whole body. + expect(lines[signature]?.endsWith(':')).toBe(true) + expect(lines[signature + 1]).toBe(`${' '.repeat(8)}"""${doc}"""`) + } + // No docstring is left floating at class-body indentation. + expect(lines.filter(line => line.startsWith(`${' '.repeat(4)}"""`))).toEqual([]) + }) + + it('orders subscript entries against methods by name, not by member kind', () => { + // `a-tool` sorts before `z`, so the subscript comment must precede the + // method: one ordered stream, not methods-then-comments. + const noArgs = parameterSchemaSpecToJsonSchema({}) as unknown as Record + const text = renderToolsSdkPy([ + { name: 'z', description: 'Last by name.', parameters: noArgs, output: { type: 'string' } }, + { name: 'a-tool', description: 'First by name.', parameters: noArgs, output: { type: 'string' } }, + ]) + expect(text.indexOf('# tools["a-tool"]')).toBeLessThan(text.indexOf('async def z')) + // The interleaved comment does not disturb the class body: `z` still parses + // as the statement that keeps `pass` out. + expect(text).not.toContain(`${' '.repeat(4)}pass`) + }) + + it('is deterministic: byte-identical output regardless of input order or duplication', () => { + expect(renderToolsSdkPy([bash, exotic])).toBe(renderToolsSdkPy([exotic, bash])) + expect(renderToolsSdkPy([bash, bash])).toBe(renderToolsSdkPy([bash, bash])) + }) + + it('renders a pass body and a minimal import for an empty tool set', () => { + const text = renderToolsSdkPy([]) + expect(text).toContain('class Tools(Protocol):') + expect(text).toContain(' pass') + // Nothing but the protocol is used, so the import line is just Protocol. + expect(text).toContain('from typing import Protocol') + }) + + it('omits the docstring/comment when a schema has no description', () => { + const undescribedIdentifier: ToolSdkSchema = { + name: 'plain', + description: '', + parameters: parameterSchemaSpecToJsonSchema({}) as unknown as Record, + output: { type: 'string' }, + } + const undescribedExotic: ToolSdkSchema = { + name: 'weird-name', + description: '', + parameters: parameterSchemaSpecToJsonSchema({}) as unknown as Record, + output: { type: 'string' }, + } + const text = renderToolsSdkPy([undescribedIdentifier, undescribedExotic]) + // Identifier method appears without a docstring in its body — hence the + // `: ...` stub, which a documented method replaces with the docstring. + expect(text).toContain('async def plain(self, args: dict[str, Any]) -> str: ...') + expect(text).not.toContain('"""') + // Subscript entry appears without the "# ..." description follow-up. + expect(text).toContain('# tools["weird-name"]') + expect(text.split('\n').every(line => !line.startsWith(' # '))).toBe(true) + // A whitespace-only description collapses to nothing and is treated as + // absent: no empty `""""""` docstring, no bare `# ` line. + const blank = renderToolsSdkPy([ + { ...undescribedIdentifier, description: ' \t\n ' }, + { ...undescribedExotic, description: ' ' }, + ]) + expect(blank).toBe(text) + }) + + it('marks an open object TypedDict and declares a closed empty object', () => { + const t: ToolSdkSchema = { + name: 'openness', + description: '', + parameters: { + type: 'object', + additionalProperties: false, + properties: { + open: { type: 'object', additionalProperties: true, properties: { x: { type: 'string' } }, required: ['x'] }, + closedEmpty: { type: 'object', additionalProperties: false, properties: {} }, + }, + required: ['open', 'closedEmpty'], + }, + output: { type: 'string' }, + } + const text = renderToolsSdkPy([t]) + // The open nested object carries the in-band openness note... + expect(text).toContain('class OpennessArgsOpen(TypedDict):') + expect(text).toMatch(/class OpennessArgsOpen\(TypedDict\):\n x: str\n # Additional keys beyond those declared are allowed\./) + // ...the closed root does not... + expect(text).toMatch(/class OpennessArgs\(TypedDict\):\n open: OpennessArgsOpen\n closedEmpty: OpennessArgsClosedEmpty\n\n/) + // ...and a closed EMPTY object declares an empty TypedDict rather than + // degrading to dict[str, Any] (which would falsely accept any keys). + expect(text).toMatch(/class OpennessArgsClosedEmpty\(TypedDict\):\n pass/) + expect(text).toContain('closedEmpty: OpennessArgsClosedEmpty') + }) + + it('renders a deeply nested array schema without exhausting the call stack, capped at CPython\'s bracket limit', () => { + // The registry supports depth-unbounded schemas; the renderer must not + // reintroduce a recursion limit during prompt assembly. It must also not + // emit more open brackets than CPython's tokenizer accepts (200), so the + // chain degrades to `Any` at MAX_LIST_NESTING instead of rendering an SDK + // block that is not valid Python. + let deep: Record = { type: 'string' } + for (let i = 0; i < 20000; i++) deep = { type: 'array', items: deep } + const type = jsonSchemaToPy(deep) + expect(type.startsWith('list[list[')).toBe(true) + expect(type.endsWith(']]')).toBe(true) + // 180 `list[` levels around `Any`, not 20000 around `str`. + expect(type).toBe(`${'list['.repeat(180)}Any${']'.repeat(180)}`) + expect(type.split('[').length - 1).toBeLessThan(200) + }) + + it('keeps a chain just under the nesting cap exact, and restarts nesting per TypedDict field', () => { + // 179 levels still render the real item type: the cap degrades only what + // would not parse. + let under: Record = { type: 'string' } + for (let i = 0; i < 179; i++) under = { type: 'array', items: under } + expect(jsonSchemaToPy(under)).toBe(`${'list['.repeat(179)}str${']'.repeat(179)}`) + // A field annotation is a fresh logical line, so a 179-deep chain reached + // THROUGH an object field is unaffected by the depth spent on the object. + const tool: ToolSdkSchema = { + name: 'deep_field', + description: 'Deep array under a field.', + parameters: { type: 'object', additionalProperties: false, properties: { rows: under }, required: ['rows'] }, + output: { type: 'string' }, + } + expect(renderToolsSdkPy([tool])).toContain(` rows: ${'list['.repeat(179)}str${']'.repeat(179)}`) + }) + + it('caps the argument annotation, the site whose enclosing paren stays open', () => { + // The worst of the three emission sites: the parameter list's `(` is still + // open around this annotation, so 180 `list[` plus the innermost bracket + // plus that paren is 182 of CPython's 200. Only a raw `register()` whose + // `parameters` is an array reached from the root through `oneOf` arms + // alone gets there — the root array itself, or one under any depth of + // unions, since an arm inherits the enclosing depth unchanged. An object + // ancestor takes it out of this case: its fields restart at the 181 site. + // `defineTool` compiles an object root, whose annotation is a bare + // TypedDict name or a one-bracket `dict[str, Any]`, never a chain. + const rooted = (depth: number): ToolSdkSchema => { + let schema: Record = { type: 'string', const: 'x' } + for (let i = 0; i < depth; i++) schema = { type: 'array', items: schema } + return { name: 'rooted', description: 'Array-rooted parameters.', parameters: schema, output: { type: 'string' } } + } + // Exactly at the cap with a scalar underneath is the worst case itself: the + // chain's root frame starts at `listDepth: 0` here, so all 180 `list[` + // still emit and the innermost `Literal[` is reached rather than degraded. + const worst = renderToolsSdkPy([rooted(180)]) + expect(worst).toContain(`async def rooted(self, args: ${'list['.repeat(180)}Literal["x"]${']'.repeat(180)}) -> str:`) + const annotation = worst.split('async def rooted(self, args: ')[1]!.split(') -> str:')[0]! + // 181 brackets on the annotation plus the still-open parameter-list paren, + // the 182 the cap is chosen against. + expect(annotation.split('[').length - 1).toBe(181) + // One array deeper is where the degradation lands, and it lands on the item + // rather than on another `list[`, so the count cannot grow past that. + expect(renderToolsSdkPy([rooted(181)])) + .toContain(`async def rooted(self, args: ${'list['.repeat(180)}Any${']'.repeat(180)}) -> str:`) + // A union spine reaches the same 182, at any number of arms deep: each arm + // inherits the enclosing depth because `A | B` opens nothing, so the chain + // under the innermost one still starts at 0. Three unions here, to pin that + // it is the whole `oneOf`-only path and not just a single root union. + let spine: Record = rooted(180).parameters + for (let i = 0; i < 3; i++) spine = { oneOf: [spine, { type: 'string' }] } + const text = renderToolsSdkPy([{ ...rooted(180), parameters: spine }]) + const chain = `${'list['.repeat(180)}Literal["x"]${']'.repeat(180)}` + expect(text).toContain(`args: ${chain} | str | str | str) -> str:`) + expect(text.split('async def rooted(self, args: ')[1]!.split(') -> str:')[0]!.split('[').length - 1).toBe(181) + // An object ancestor is the boundary of that path: the field it declares is + // a class-body line, so the same chain lands on the 181 site instead. + const boxed = renderToolsSdkPy([ + { + ...rooted(180), + parameters: { type: 'object', properties: { rows: rooted(180).parameters }, required: ['rows'] }, + }, + ]) + expect(boxed).toContain(` rows: ${'list['.repeat(179)}Any${']'.repeat(179)}`) + }) + + it('renders a deeply nested oneOf chain in linear time (no per-level re-materialization)', () => { + // Each level is a two-branch oneOf whose first branch recurses; joining the + // accumulated union string at every level would be Theta(depth^2). At this + // depth the quadratic path (~100,000^2 char copies) blows past vitest's 5s + // default, so this fails loud on a regression; the `+`/ConsString path is + // milliseconds. (Guard the depth explicitly so the assertions stay exact.) + // The resulting chain is intentionally uncapped, unlike list nesting: it is + // grammatically valid Python at any length, and only CPython's `compile()` + // recursion would reject it — see the `oneOf` arm in py-types.ts. + const depth = 100000 + let deep: Record = { type: 'string' } + for (let i = 0; i < depth; i++) deep = { oneOf: [deep, { type: 'null' }] } + const type = jsonSchemaToPy(deep) + expect(type.startsWith('str | None')).toBe(true) + expect(type.endsWith(' | None')).toBe(true) + expect(type.length).toBe('str'.length + ' | None'.length * depth) + }) + + it('names a deep oneOf-of-object chain in linear time (bounded propagated class names)', () => { + // Every level is a oneOf whose SECOND branch is a named object (a closed + // empty TypedDict) and whose first branch recurses — so every level has an + // object node, each propagating a class name one segment longer. Without a + // propagation cap, allocateClassName slices an ever-longer rope at every + // level → Theta(depth^2) (~9.5s at this depth, past the 5s default); + // childClassName caps the base so it stays linear (~ms). Assertions are + // shape-based but the depth is the tripwire: a regression times out. + const depth = 60000 + let deep: Record = { type: 'object', additionalProperties: false, properties: {} } + for (let i = 0; i < depth; i++) { + deep = { oneOf: [deep, { type: 'object', additionalProperties: false, properties: {} }] } + } + const tool: ToolSdkSchema = { name: 'deep', description: 'Deep oneOf-object chain.', parameters: { type: 'object', additionalProperties: false, properties: { root: deep }, required: ['root'] }, output: { type: 'string' } } + const text = renderToolsSdkPy([tool]) + // No emitted class name exceeds the cap (plus a short collision suffix). + const longest = [...text.matchAll(/^class (\w+)\(TypedDict\):/gm)].reduce((max, m) => Math.max(max, m[1]?.length ?? 0), 0) + expect(longest).toBeLessThanOrEqual(140) + expect(text).toContain('class Tools(Protocol):') + }) + + it('caps the class name for a tool whose name exceeds the base length limit', () => { + // The root class base is `${CamelCase(name)}Args`; a very long tool name + // makes it exceed MAX_CLASS_NAME_BASE, so allocateClassName caps it. + const longName = `x_${'a'.repeat(200)}` + const tool: ToolSdkSchema = { + name: longName, + description: 'Long name.', + parameters: { type: 'object', additionalProperties: false, properties: { f: { type: 'string' } }, required: ['f'] }, + output: { type: 'string' }, + } + const text = renderToolsSdkPy([tool]) + const longest = [...text.matchAll(/^class (\w+)\(TypedDict\):/gm)].reduce((max, m) => Math.max(max, m[1]?.length ?? 0), 0) + expect(longest).toBeLessThanOrEqual(140) + expect(text).toContain('class Tools(Protocol):') + }) + + it('emits pass for a subscript-only tool set (comments are not statements)', () => { + const t: ToolSdkSchema = { + name: 'my-exotic.tool', + description: '', + parameters: parameterSchemaSpecToJsonSchema({}) as unknown as Record, + output: { type: 'string' }, + } + const text = renderToolsSdkPy([t]) + // The class body must contain a statement before the subscript comments. + expect(text).toMatch(/class Tools\(Protocol\):\n pass\n # tools\["my-exotic\.tool"\]/) + }) + + it('degrades an object whose field would be name-mangled (__token) to dict[str, Any]', () => { + // Class-syntax TypedDict mangles a leading-double-underscore non-dunder + // annotation to _ClassName__token — a different JSON key than the schema. + const t: ToolSdkSchema = { + name: 'mangler', + description: '', + parameters: { + type: 'object', + additionalProperties: false, + properties: { __token: { type: 'string' } }, + required: ['__token'], + }, + output: { type: 'string' }, + } + const text = renderToolsSdkPy([t]) + expect(text).toContain('async def mangler(self, args: dict[str, Any]) -> str: ...') + expect(text).not.toContain('__token:') + // Dunder-form fields (__meta__) are NOT mangled and stay expressible. + const dunder: ToolSdkSchema = { + name: 'dunder', + description: '', + parameters: { + type: 'object', + additionalProperties: false, + properties: { __meta__: { type: 'string' } }, + required: ['__meta__'], + }, + output: { type: 'string' }, + } + expect(renderToolsSdkPy([dunder])).toContain('__meta__: str') + }) + + it('degrades an object with a __debug__ field, which CPython refuses to assign', () => { + // `__debug__` is a legal identifier and dunder-form, so it clears both the + // identifier rule and the name-mangling rule, but CPython rejects the + // annotation at COMPILE time (`SyntaxError: cannot assign to __debug__`) — + // and this block is Code Mode's only SDK, so it must always parse. + const t: ToolSdkSchema = { + name: 'debugger', + description: '', + parameters: { + type: 'object', + additionalProperties: false, + properties: { __debug__: { type: 'string' } }, + required: ['__debug__'], + }, + output: { type: 'string' }, + } + const text = renderToolsSdkPy([t]) + expect(text).toContain('async def debugger(self, args: dict[str, Any]) -> str: ...') + expect(text).not.toContain('__debug__') + }) + + it('routes every underscore-leading tool name to subscript access', () => { + // `_foo` and `__meta__` are both legal Python attributes, unlike an exotic + // name or a hard keyword, yet the whole underscore family goes to + // `tools[name]` under one rule. Only some forms actually break — `__token` + // name-mangles at the CALL SITE inside the model's own class, and a dunder + // that exists on `object` (`__class__`) resolves before the proxy's + // __getattr__ runs — so the family rule is what routes `_foo` and + // `__meta__`, not a defect in those two names. + const make = (name: string): ToolSdkSchema => ({ + name, + description: 'Leading underscore.', + parameters: parameterSchemaSpecToJsonSchema({}) as unknown as Record, + output: { type: 'string' }, + }) + const text = renderToolsSdkPy([make('_foo'), make('__meta__'), make('__token')]) + for (const name of ['_foo', '__meta__', '__token']) { + expect(text).toContain(`# tools[${JSON.stringify(name)}](args: dict[str, Any]) -> str`) + expect(text).not.toContain(`async def ${name}(`) + } + // No method emitted at all, so the class body needs the explicit `pass`. + expect(text).toContain(' pass\n') + }) + + it('quotes a tool name through the same JSON.stringify the Literal path depends on', () => { + // A lone surrogate is reachable in a name — `"\ud800"` survives + // `JSON.parse` of MCP wire JSON — and this path has no UNPRINTABLE / + // LONE_SURROGATE fallback behind it, only ES2019 well-formed + // stringification. Raw, it would make the whole SDK block uncompilable, + // exactly as on the `Literal[...]` path. + const text = renderToolsSdkPy([ + { + name: 'a\ud800b', + description: 'Lone surrogate in the name.', + parameters: parameterSchemaSpecToJsonSchema({}) as unknown as Record, + output: { type: 'string' }, + }, + ]) + expect(text).toContain(String.raw`# tools["a\ud800b"](args: dict[str, Any]) -> str`) + expect(text).not.toContain('\ud800') + }) + + it('escapes quotes and backslashes in descriptions so the docstring stays valid Python', () => { + // A description ending in `"` or an odd backslash would otherwise merge + // with (or escape) the closing triple quote — and this block is Code + // Mode's only SDK, so it must always parse. + const trailingQuote = renderToolsSdkPy([described('ends in a quote"')]) + expect(trailingQuote).toContain(String.raw`"""ends in a quote\""""`) + const trailingBackslash = renderToolsSdkPy([described('ends in a backslash\\')]) + expect(trailingBackslash).toContain(String.raw`"""ends in a backslash\\"""`) + const tripleQuote = renderToolsSdkPy([described('contains """ triple quote')]) + expect(tripleQuote).toContain(String.raw`"""contains \"\"\" triple quote"""`) + }) + + it('escapes unprintable control characters, which CPython refuses inside source at all', () => { + // `compile()` raises `SyntaxError: source code string cannot contain null + // bytes` for a NUL ANYWHERE in the source text, including inside a string + // literal or a comment, so a NUL that survives normalization into a + // docstring or a `#` field comment stops this block — Code Mode's only SDK — + // from parsing at all. The whitespace collapse does not remove it (a NUL is + // not whitespace). Rendering it as a visible escape keeps the source + // parseable and still shows the model what the schema said. + const nul = renderToolsSdkPy([described('before\u0000after')]) + // Both emission sites: the method docstring and the `#` field comment. The + // docstring's backslash is doubled by the same escaping that keeps a literal + // backslash from escaping the closing triple quote, so Python parses it back + // to the visible `\x00` the comment shows directly. Neither carries the byte. + expect(nul).not.toContain('\u0000') + expect(nul).toContain(String.raw`"""before\\x00after"""`) + expect(nul).toContain(String.raw`# before\x00after`) + // The other C0 controls and DEL escape on the same path. Tab, newline and + // carriage return never reach it: the whitespace collapse folds them to a + // space first. + const others = renderToolsSdkPy([described('bell\u0007esc\u001bdel\u007f')]) + expect(others).toContain(String.raw`bell\x07esc\x1bdel\x7f`) + expect(renderToolsSdkPy([described('tab\tnewline\ncr\r')])).toContain('"""tab newline cr"""') + // No C1 control is ECMAScript whitespace (TAB/VT/FF/SP/NBSP/ZWNBSP/Zs plus + // LF/CR/LS/PS), so the collapse folds none of U+0080 to U+009F and the + // escape is what keeps them out of the docstring, where they would be + // invisible. NBSP, which IS whitespace, folds instead. Windows-1252 bytes + // 0x80 to 0x9F decoded as Latin-1 land exactly here. + const nel = renderToolsSdkPy([described('a\u0085b')]) + expect(nel).not.toContain('\u0085') + expect(nel).toContain(String.raw`# a\x85b`) + const c1 = renderToolsSdkPy([described('csi\u009bst\u009cend\u009f')]) + expect(c1).toContain(String.raw`csi\x9bst\x9cend\x9f`) + expect(renderToolsSdkPy([described('nb\u00a0sp')])).toContain('"""nb sp"""') + // `Cf` formatting characters pass through by category, not by + // addressability — U+00AD would fit `\xNN`, the rest would need a second + // form. They terminate neither a Python string literal nor a `#` comment, + // so the block stays parseable with the code point intact. + expect(renderToolsSdkPy([described('zero\u200bwidth')])).toContain('"""zero\u200bwidth"""') + // Whitespace around a surviving control character is not an absent + // description. The escape's output is non-whitespace ASCII and the escaped + // sets are disjoint from what `trim()` strips, so the two operations touch + // different characters and their order is unobservable. + expect(renderToolsSdkPy([described(' \u0085 ')])).toContain(String.raw`# \x85`) + }) + + it('escapes unpaired surrogates, which make the source impossible to encode', () => { + // This is the NUL case, not the invisible-character case: Python source + // must be UTF-8-encodable, and `compile()` raises `UnicodeEncodeError: + // surrogates not allowed` for a lone surrogate in a string literal and in a + // `#` comment alike, so one would stop this block — Code Mode's only SDK — + // from parsing. A wire description reaches it: `JSON.parse` on a `"\ud800"` + // escape yields exactly this code point. + const high = renderToolsSdkPy([described('a\ud800b')]) + expect(high).not.toContain('\ud800') + expect(high).toContain(String.raw`# a\ud800b`) + expect(high).toContain(String.raw`"""a\\ud800b"""`) + // A lone LOW surrogate is just as unencodable, and `\xNN` reaches neither. + expect(renderToolsSdkPy([described('a\udfffb')])).toContain(String.raw`# a\udfffb`) + // A well-formed pair is ONE astral code point, not two surrogates — the + // regex's `u` flag is what draws that line, so an emoji survives intact. + expect(renderToolsSdkPy([described('emoji \u{1f600} ok')])).toContain('"""emoji \u{1f600} ok"""') + }) +}) diff --git a/scripts/gen-tool-catalog.ts b/scripts/gen-tool-catalog.ts index 38d2f406ff..74042fdf12 100644 --- a/scripts/gen-tool-catalog.ts +++ b/scripts/gen-tool-catalog.ts @@ -176,7 +176,7 @@ const TOOL_PACKAGES: ToolPackage[] = [ toolsConfig: { mode: 'code' }, async mount() {}, note: - 'Owned by the tool registry as a reserved transport outside filterable capability layers under `mode: code` / `mode: both` (see the Code Mode Agent Note). Under `code` it is the registry\'s only wire contribution; the other visible capabilities are declared in a generated TypeScript SDK section, and a program calls them through bindings scheduled under the native concurrency contract (submission-ordered starts and policy; concurrency-safe bodies overlap up to `maxParallelSubCalls`) that re-enter the complete guarded tool pipeline and link each nested execution to this outer result.', + 'Owned by the tool registry as a reserved transport outside filterable capability layers under `mode: code` / `mode: both` (see the Code Mode Agent Note). Under `code` it is the registry\'s only wire contribution; the other visible capabilities are declared in a generated SDK section in the loaded runtime\'s language, and a program calls them through bindings scheduled under the native concurrency contract (submission-ordered starts and policy; concurrency-safe bodies overlap up to `maxParallelSubCalls`) that re-enter the complete guarded tool pipeline and link each nested execution to this outer result.', }, { pkg: '@deepseek-ai/dsh-plan-mode',