diff --git a/.agents/notes/implemented/feature/2026-07-31-code-mode-language-dispatch.i18n.yaml b/.agents/notes/implemented/feature/2026-07-31-code-mode-language-dispatch.i18n.yaml new file mode 100644 index 0000000000..eb6fe2f9ae --- /dev/null +++ b/.agents/notes/implemented/feature/2026-07-31-code-mode-language-dispatch.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-31-code-mode-language-dispatch.md +2026-07-31-code-mode-language-dispatch.md: 6726842741988d71f5bce1885ebe7e8e7b3564d3 +2026-07-31-code-mode-language-dispatch.zh.md: 0bb381c410fbbb468d65076518861923021a4453 diff --git a/.agents/notes/implemented/feature/2026-07-31-code-mode-language-dispatch.md b/.agents/notes/implemented/feature/2026-07-31-code-mode-language-dispatch.md new file mode 100644 index 0000000000..6726842741 --- /dev/null +++ b/.agents/notes/implemented/feature/2026-07-31-code-mode-language-dispatch.md @@ -0,0 +1,32 @@ +# Agent Note: Code Mode language dispatch and the Python SDK renderer + +Status: implemented + +English | [中文](2026-07-31-code-mode-language-dispatch.zh.md) + +## Problem + +Code Mode generated one SDK flavor: TypeScript. `ToolRegistry` hard-coded `renderToolsSdk` for the `tools:sdk` section and `requireCodeRuntime` rejected any `ctx.codeRuntime.language !== 'typescript'`. Adding a CPython backend means a program's source language is no longer fixed: the same visible tool registry must project a Python SDK when a Python runtime is loaded, and the model-facing `run_code` schema strings ("Execute a Python program …") must match the SDK section's language so the model never sees a TypeScript instruction over a Python runtime. + +This is the tool-facing half of the multi-language Code Mode split; the [code-runtime seam](../../../../packages/code-runtime/code-runtime/README.md) already carries `CodeRuntime.language`. This note owns only how `dsh-tools` dispatches on that field. The backend that implements `language: 'python'` is owned by its own note, delivered separately. + +## Decision + +Language selection is a lookup on `ctx.codeRuntime.language`, resolved lazily at prompt assembly, against two parallel tables in `dsh-tools`: + +- `SDK_RENDERERS` (index.ts) maps a language to its `tools:sdk` renderer — `typescript → renderToolsSdk`, `python → renderToolsSdkPy`. The `tools:sdk` section reads the loaded runtime's language and picks the renderer; `requireCodeRuntime` rejects a `mode: code`/`both` runtime whose language is absent from the table, naming the known languages. +- `RUN_CODE_FLAVORS` (code-mode.ts) maps a language to its two model-facing `run_code` strings (tool `description` and the `code` parameter description), so a language's SDK section and its transport schema always agree. + +Both tables are read with `Object.hasOwn` before use so a language named `toString`/`constructor` cannot resolve an inherited `Object.prototype` member as a renderer; a language present on neither table but reaching the read fails loud (defense-in-depth against a caller bypassing the guard). Adding a backend language is two table entries plus its renderer — no `agent-loop` or registry-structure change. + +`code-mode.ts` depends only on the runtime seam (`@deepseek-ai/dsh-code-runtime`), never on a concrete backend; dispatch is by `runtime.language` at run time. The tool layer therefore lands independently of the protocol and backend PRs — it needs only the seam's `language` field, which is already on master. + +### The Python SDK renderer + +`py-types.ts` renders the same unified tool-schema vocabulary `jsonSchemaToTs` covers, targeting Python: `jsonSchemaToPy` emits a type expression per JSON-schema node, and `renderToolsSdkPy` assembles named `TypedDict`s for each visible tool's arguments and canonical output plus a `tools` object with usage instructions equivalent to the TypeScript flavor. Unsupported raw constructs degrade rather than throwing during assembly, matching the TypeScript renderer's contract. The output is deterministic — lexicographic tool order, byte-identical text for an unchanged tool set — so the prompt stays prefix-cache-friendly. + +## Rejected alternatives + +- **A `language` config field on `ToolRegistry`.** Deployment would then have two places to name the language (the loaded runtime and the tools config) that can disagree; the loaded runtime is the single source of truth, so the registry reads it rather than duplicating it. +- **Importing the Python backend into `code-mode.ts` to detect it.** That would couple the tool layer to a concrete backend and force the protocol/backend PRs to land first. Runtime dispatch on `language` keeps the layer backend-agnostic and independently shippable. +- **A default renderer for an unknown language.** A silent fallback would emit a TypeScript SDK over, e.g., a Ruby runtime — the model would see instructions in the wrong language. Failing loud at assembly is the repository's misconfiguration stance. diff --git a/.agents/notes/implemented/feature/2026-07-31-code-mode-language-dispatch.zh.md b/.agents/notes/implemented/feature/2026-07-31-code-mode-language-dispatch.zh.md new file mode 100644 index 0000000000..0bb381c410 --- /dev/null +++ b/.agents/notes/implemented/feature/2026-07-31-code-mode-language-dispatch.zh.md @@ -0,0 +1,32 @@ +# Agent Note: Code Mode 语言分发与 Python SDK 渲染器 + +Status: implemented + +[English](2026-07-31-code-mode-language-dispatch.md) | 中文 + +## 问题 + +Code Mode 只生成一种 SDK 形态：TypeScript。`ToolRegistry` 为 `tools:sdk` 段硬编码了 `renderToolsSdk`，且 `requireCodeRuntime` 会拒绝任何 `ctx.codeRuntime.language !== 'typescript'`。引入 CPython 后端后，程序的源语言不再固定：同一个可见工具注册表在加载 Python 运行时时必须投射出 Python SDK，而面向模型的 `run_code` schema 字符串（"Execute a Python program …"）也必须与 SDK 段的语言一致，模型才不会在 Python 运行时下看到 TypeScript 指令。 + +这是多语言 Code Mode 拆分中面向工具的那一半；[代码运行时 seam](../../../../packages/code-runtime/code-runtime/README.md) 已经携带 `CodeRuntime.language`。本 Note 只负责 `dsh-tools` 如何在该字段上分发。实现 `language: 'python'` 的后端由它自己的 Note 负责，单独交付。 + +## 决策 + +语言选择就是对 `ctx.codeRuntime.language` 的查表，在 prompt 装配时惰性解析，查 `dsh-tools` 里两张平行的表： + +- `SDK_RENDERERS`（index.ts）把语言映射到它的 `tools:sdk` 渲染器——`typescript → renderToolsSdk`、`python → renderToolsSdkPy`。`tools:sdk` 段读取所加载运行时的语言并选出渲染器；`requireCodeRuntime` 拒绝其语言不在表中的 `mode: code`/`both` 运行时，并列出已知语言。 +- `RUN_CODE_FLAVORS`（code-mode.ts）把语言映射到它那两条面向模型的 `run_code` 字符串（工具 `description` 与 `code` 参数描述），使一种语言的 SDK 段与它的传输 schema 始终一致。 + +两张表在使用前都以 `Object.hasOwn` 读取，这样名为 `toString`/`constructor` 的语言不会把继承自 `Object.prototype` 的成员解析成渲染器；一个两张表都没有、却仍走到读取处的语言会 fail loud（对绕过守卫的调用方的纵深防御）。新增一门后端语言就是两条表项加它的渲染器——不动 `agent-loop`，也不动注册表结构。 + +`code-mode.ts` 只依赖运行时 seam（`@deepseek-ai/dsh-code-runtime`），绝不依赖具体后端；分发在运行时按 `runtime.language` 进行。因此工具层独立于协议和后端 PR 落地——它只需要 seam 的 `language` 字段，而该字段已在 master 上。 + +### Python SDK 渲染器 + +`py-types.ts` 渲染 `jsonSchemaToTs` 所覆盖的同一套统一工具 schema 词汇，目标为 Python：`jsonSchemaToPy` 为每个 JSON-schema 节点发出一个类型表达式，`renderToolsSdkPy` 为每个可见工具的参数与规范输出装配具名 `TypedDict`，再加一个带用法说明的 `tools` 对象，与 TypeScript 形态等价。不支持的原始构造在装配时降级而非抛错，与 TypeScript 渲染器的契约一致。输出是确定性的——工具按字典序排列，工具集不变时文本逐字节相同——因此 prompt 保持 prefix-cache 友好。 + +## 被否决的备选方案 + +- **在 `ToolRegistry` 上加一个 `language` 配置字段。** 那样部署方就会有两处命名语言（所加载的运行时与 tools 配置）且可能相互矛盾；所加载的运行时是唯一真相来源，故注册表读取它而不复制它。 +- **把 Python 后端 import 进 `code-mode.ts` 来检测它。** 那会把工具层耦合到具体后端，并迫使协议/后端 PR 先落地。按 `language` 运行时分发使该层保持后端无关、可独立发布。 +- **为未知语言提供默认渲染器。** 静默回退会在比如 Ruby 运行时上发出 TypeScript SDK——模型会看到错误语言的指令。在装配处 fail loud 是本仓库对错误配置的立场。 diff --git a/packages/core/tools/README.i18n.yaml b/packages/core/tools/README.i18n.yaml index d26f891f91..e413309f32 100644 --- a/packages/core/tools/README.i18n.yaml +++ b/packages/core/tools/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/core/tools/README.md -README.md: 15fc5839a3b0e3fa2d20c5a9cc50577e9807ffda -README.zh.md: 8547ee4a796dcd93945dfa40373c14c10d7d0c8a +README.md: 0c6b5ec5bc213e8a568592f3aca7c79b52d73907 +README.zh.md: 80397e37c6e92053d825d4aa7d61e20455cd881a diff --git a/packages/core/tools/README.md b/packages/core/tools/README.md index 15fc5839a3..0c6b5ec5bc 100644 --- a/packages/core/tools/README.md +++ b/packages/core/tools/README.md @@ -13,7 +13,7 @@ tools: mode: native # native (default) | code | both ``` -`native` contributes visible tools as function definitions. `code` contributes the reserved `run_code` transport and generated `tools:sdk` section; `both` contributes both forms. The reserved transport cannot be registered, shadowed, restricted, or removed. Non-native modes require a TypeScript `ctx.codeRuntime`, and a `systemPrompt.toolOrder` entry for a tool the mode does not contribute rejects prompt assembly. A `system-prompt/assemble` listener may replace the registry's contributions; its returned assembly is authoritative, so that listener owns preserving a usable Code Mode protocol. +`native` contributes visible tools as function definitions. `code` contributes the reserved `run_code` transport and generated `tools:sdk` section; `both` contributes both forms. The reserved transport cannot be registered, shadowed, restricted, or removed. Non-native modes require a `ctx.codeRuntime` whose `language` has a registered SDK renderer (TypeScript via [`dsh-code-runtime-worker`](../../code-runtime/code-runtime-worker/README.md), Python via [`dsh-code-runtime-python`](../../code-runtime/code-runtime-python/README.md)); a runtime language with no renderer fails prompt assembly loudly, and a `systemPrompt.toolOrder` entry for a tool the mode does not contribute rejects prompt assembly. A `system-prompt/assemble` listener may replace the registry's contributions; its returned assembly is authoritative, so that listener owns preserving a usable Code Mode protocol. ### Public API @@ -114,9 +114,9 @@ Returning `undefined` selects generic fallback. Presenters depend only on their ### Code Mode -Under `code` or `both`, the registry exposes the reserved `run_code` transport and a deterministic TypeScript SDK for the current scope; only the program's outer logs and return value re-enter model context. The SDK declares exact `ToolArgsMap` and `ToolOutputMap` entries for every visible tool, and each binding resolves to the tool's canonical JSON value. Each lossless-JSON binding call re-enters the complete tool pipeline under the native scheduling contract (concurrency-safe calls may overlap up to `maxParallelSubCalls`; exclusive calls run alone as ordering barriers) with logged correlation to the outer call. Denials and other failed results reject with the real program-visible `ToolCallError` carrying only `toolName` and `message`; Native content and internal error codes stay outside the Code contract. Ordinary side effects are not rolled back, and sub-call `additionalContexts` are deferred through the parent result to preserve call/result adjacency. Run settlement aborts and drains outstanding bindings; runtime failures surface as `CodeRunFailedError`. See the [Code Mode foundation](../../../.agents/notes/implemented/feature/2026-06-15-code-mode.md), [typed-return contract](../../../.agents/notes/implemented/feature/2026-07-20-code-mode-typed-tool-returns.md), and [code-runtime seam](../../code-runtime/README.md). Try `pnpm run demo:code-mode`. +Under `code` or `both`, the registry exposes the reserved `run_code` transport and a deterministic SDK for the current scope, generated in the loaded runtime's language — the registry selects the renderer by `ctx.codeRuntime.language` (`typescript` → the TypeScript SDK below, `python` → the Python SDK). Only the program's outer logs and return value re-enter model context. The SDK declares exact per-tool argument and canonical-output types for every visible tool (`ToolArgsMap`/`ToolOutputMap` in TypeScript, named `TypedDict`s in Python), and each binding resolves to the tool's canonical JSON value. Each lossless-JSON binding call re-enters the complete tool pipeline under the native scheduling contract (concurrency-safe calls may overlap up to `maxParallelSubCalls`; exclusive calls run alone as ordering barriers) with logged correlation to the outer call. Denials and other failed results reject with the real program-visible `ToolCallError` carrying only `toolName` and `message`; Native content and internal error codes stay outside the Code contract. Ordinary side effects are not rolled back, and sub-call `additionalContexts` are deferred through the parent result to preserve call/result adjacency. Run settlement aborts and drains outstanding bindings; runtime failures surface as `CodeRunFailedError`. See the [Code Mode foundation](../../../.agents/notes/implemented/feature/2026-06-15-code-mode.md), [typed-return contract](../../../.agents/notes/implemented/feature/2026-07-20-code-mode-typed-tool-returns.md), and [code-runtime seam](../../code-runtime/README.md). Try `pnpm run demo:code-mode`. -- **The SDK section** (`tools:sdk`, order 150): a lazy prompt section regenerating, at each assembly, `JsonValue`, exact `ToolArgsMap` / `ToolOutputMap`, `ToolName`, the `ToolCallError` declaration, and a mapped `tools` namespace for the calling scope's visible end capabilities (exotic names via quoted keys), plus fixed usage instructions. Deterministic — lexicographic tool order, byte-identical text for an unchanged tool set (prefix-cache-friendly). The codegen (`jsonSchemaToTs`, exported) handles every unified schema construct and degrades unsupported raw constructs to `unknown`, never throwing during prompt assembly. +- **The SDK section** (`tools:sdk`, order 150): a lazy prompt section regenerating the language-appropriate SDK text at each assembly. In the TypeScript flavor it emits `JsonValue`, exact `ToolArgsMap` / `ToolOutputMap`, `ToolName`, the `ToolCallError` declaration, and a mapped `tools` namespace for the calling scope's visible end capabilities (exotic names via quoted keys), plus fixed usage instructions; the Python flavor (`ctx.codeRuntime.language === 'python'`) emits the equivalent named `TypedDict`s and a `tools` object with matching usage instructions. Deterministic — lexicographic tool order, byte-identical text for an unchanged tool set (prefix-cache-friendly). The TypeScript codegen (`jsonSchemaToTs`, exported) handles every unified schema construct and degrades unsupported raw constructs to `unknown`, never throwing during prompt assembly. - **The dispatch bridge** (`run_code`'s execute): every binding call is snapshotted as lossless JSON before dispatch (`undefined`, `BigInt`, cycles, sparse arrays, `-0`, and exotic objects reject that one call), scheduled through a per-run pool that reuses the native concurrency contract — calls start strictly in submission order, consecutive `isConcurrencySafe` calls overlap up to the validated `maxParallelSubCalls` config (default 10; `1` restores serial dispatch), and an exclusive-classified call drains the pool, runs alone, and bars later calls — given the outer execution's opaque token as `parent`, and run through the complete pre-execute → guards → execute → post-execute → result pipeline. A success returns the final canonical value after policy; a failure reaches the worker as one message and becomes `ToolCallError(toolName, message)`. Each started sub-call logs a `tool/code-dispatch-start` event (deterministic id `:code:`, numbered by submission) at pipeline entry and settles with one `tool/code-dispatch` event carrying the complete model-facing `content`/`isError` outcome (the `tool/result` vocabulary, so UIs render sub-calls through the native path — the pair's `time` fields carry per-sub-call timing); a queued call abandoned by run settlement logs neither. `deriveMessages()` surfaces neither event nor persists the canonical value. Token correlation lets commit-style observers defer an inner success until the final `run_code` result without exposing the live outer execution; ordinary tool side effects are not rolled back. Every sub-call `additionalContexts` entry is deferred through the outer `ToolRunContext` in dispatch order; the loop appends those contexts only after the parent `run_code` result, preserving adjacency and retaining each source/meta even when the program later fails. - **Settlement discipline**: the bridge owns a run-scoped abort that follows the outer signal in and fires when the run settles for any reason, so a budget expiry aborts an in-flight sub-tool instead of orphaning it; the bridge then drains its queue BEFORE returning, so every `tool/code-dispatch` lands inside the open turn. A failed run throws `CodeRunFailedError` (`code: 'CODE_RUN_FAILED'`, message = the failure kind + captured logs), which the pipeline converts to a structured `isError` the model self-corrects from. - **Result boundary**: intermediate binding values cross the worker boundary whole and have no per-binding byte cap. `run_code` returns canonical `{ logs: string[], result?: JsonValue }`; strings render raw, every other present JSON root renders through a stack-safe pretty JSON traversal whose total indentation is capped at ten characters (deeper subtrees stay compact), `null` remains explicit, and absent `result` means the program returned `undefined`. The worker's configurable `maxOutputBytes` (default 64 MiB) applies only to the combined serialized outer log-array, completion-value, or failure-message payloads; fixed result-envelope syntax and presentation whitespace are outside that ledger. Invalid and over-limit completions fail explicitly, and only this outer result is eligible for ordinary spill. @@ -145,7 +145,7 @@ Prefix-stable while visible definitions and their order are unchanged. Registrat #### What the model sees -Code Mode exposes the generated [`run_code` schema](../../../docs/tool-catalog.md#deepseek-aidsh-tools), the SDK instructions below, and the generated exact `declare const tools` block. `both` exposes normal schemas and this Code Mode surface. +Code Mode exposes the generated [`run_code` schema](../../../docs/tool-catalog.md#deepseek-aidsh-tools), the SDK instructions below, and the generated exact SDK block for the loaded runtime's language (the TypeScript `declare const tools` block, or the Python `tools` declaration). `both` exposes normal schemas and this Code Mode surface. The instructions and SDK block match the loaded runtime's language; the TypeScript flavor (via [`dsh-code-runtime-worker`](../../code-runtime/code-runtime-worker/README.md)) is shown below, and the Python flavor (via [`dsh-code-runtime-python`](../../code-runtime/code-runtime-python/README.md)) is the same shape with Python syntax (`await tools.name(args)`, subscript access for exotic names, `print(...)` and top-level `return`). ##### Code Mode SDK instructions @@ -190,6 +190,6 @@ Append-only; newly visible content follows the reusable request prefix and does - **`tools/pre-execute` deliberately cannot rewrite `exec.arguments`** — logged and rendered args would desync from what ran; the rewrite design is [a proposed Agent Note](../../../.agents/notes/proposed/feature/2026-06-30-pre-tool-input-rewrite.md). - **Caller-defined subagent and workflow structured outputs remain object-rooted** — this is a consumer-level guard; the shared schema vocabulary and tool outputs support every JSON root. - **`timeoutMs` on a definition is declarative only** — the registry never enforces deadlines; enforcement requires the `@deepseek-ai/dsh-timeout-policy` wrapper. -- **Code Mode is TypeScript-only and the presentation mode is service-wide** — `mode: code`/`both` rejects prompt assembly unless `ctx.codeRuntime.language === 'typescript'`; scoped restrictions/shadows still choose each agent's visible bindings, but one tool cannot be native-only while another is code-only. +- **Code Mode's SDK language follows the one loaded runtime and the presentation mode is service-wide** — `mode: code`/`both` rejects prompt assembly unless `ctx.codeRuntime.language` has a registered SDK renderer (`typescript` via the worker backend, `python` via the python backend); scoped restrictions/shadows still choose each agent's visible bindings, but one tool cannot be native-only while another is code-only, and a single runtime fixes the language service-wide. - **Code Mode intermediate values are execution-local and unbounded by bytes** — the canonical typed values cannot be reconstructed from session replay and may exhaust process or worker memory; only the outer `run_code` output has the worker's configurable hard cap. The durable log copy of each sub-call IS bounded: the `tools/code-dispatch-log` waterfall lets the spill policy replace an oversized `tool/code-dispatch` content with a preview + locator ([rationale](../../../.agents/notes/implemented/feature/2026-07-26-code-dispatch-log-spill.md)). - **`run_code` state is fresh per run** — a persistent REPL-style kernel is rejected for the MVP (cross-call state would be invisible to the log); see [the Code Mode Agent Note](../../../.agents/notes/implemented/feature/2026-06-15-code-mode.md). diff --git a/packages/core/tools/README.zh.md b/packages/core/tools/README.zh.md index 8547ee4a79..80397e37c6 100644 --- a/packages/core/tools/README.zh.md +++ b/packages/core/tools/README.zh.md @@ -13,7 +13,7 @@ tools: mode: native # native (default) | code | both ``` -`native` 以函数定义的形式贡献可见工具。`code` 贡献保留的 `run_code` 传输和生成的 `tools:sdk` 段；`both` 同时贡献两种形式。不能注册、遮蔽、限制或移除该保留传输。非原生模式要求存在 TypeScript `ctx.codeRuntime`；如果 `systemPrompt.toolOrder` 条目指向当前模式未贡献的工具，系统会拒绝组装提示词。`system-prompt/assemble` 监听器可以替换注册表贡献；它返回的组装结果具有权威性，因此该监听器负责保留可用的 Code Mode 协议。 +`native` 以函数定义的形式贡献可见工具。`code` 贡献保留的 `run_code` 传输和生成的 `tools:sdk` 段；`both` 同时贡献两种形式。不能注册、遮蔽、限制或移除该保留传输。非原生模式要求所加载 `ctx.codeRuntime` 的 `language` 有已注册的 SDK 渲染器（TypeScript 经 [`dsh-code-runtime-worker`](../../code-runtime/code-runtime-worker/README.md)，Python 经 [`dsh-code-runtime-python`](../../code-runtime/code-runtime-python/README.md)）；没有渲染器的运行时语言会让提示词组装响亮失败；如果 `systemPrompt.toolOrder` 条目指向当前模式未贡献的工具，系统会拒绝组装提示词。`system-prompt/assemble` 监听器可以替换注册表贡献；它返回的组装结果具有权威性，因此该监听器负责保留可用的 Code Mode 协议。 ### 公开 API @@ -114,7 +114,7 @@ ctx.tools.register(defineTool({ ### Code Mode -在 `code` 或 `both` 模式下，注册表为当前作用域公开保留的 `run_code` 传输和确定性的 TypeScript SDK；只有程序的外层日志与返回值会重新进入模型上下文。SDK 为每个可见工具声明精确的 `ToolArgsMap` 和 `ToolOutputMap` 条目，每个绑定都会解析为该工具的规范 JSON 值。每个无损 JSON 绑定调用都会在原生调度契约下重新进入完整工具流水线（并发安全的调用最多可重叠 `maxParallelSubCalls` 个；独占调用单独运行并构成排序屏障），并在日志中与外层调用建立关联。拒绝及其他失败结果会以程序实际可见的 `ToolCallError` 形式拒绝，且只携带 `toolName` 和 `message`；Native 内容和内部错误码留在 Code 契约之外。普通副作用不会回滚，子调用的 `additionalContexts` 会通过父结果延迟，以保持调用／结果相邻。运行结算会中止并排空尚未完成的绑定；运行时失败以 `CodeRunFailedError` 形式出现。参见 [Code Mode 基础](../../../.agents/notes/implemented/feature/2026-06-15-code-mode.md)、[类型化返回契约](../../../.agents/notes/implemented/feature/2026-07-20-code-mode-typed-tool-returns.md)和[代码运行时 seam](../../code-runtime/README.md)。可以运行 `pnpm run demo:code-mode` 试用。 +在 `code` 或 `both` 模式下，注册表为当前作用域公开保留的 `run_code` 传输和按所加载运行时语言生成的确定性 SDK——注册表按 `ctx.codeRuntime.language` 选择渲染器（`typescript` → 下方的 TypeScript SDK，`python` → Python SDK）。只有程序的外层日志与返回值会重新进入模型上下文。SDK 为每个可见工具声明精确的参数与规范输出类型（TypeScript 为 `ToolArgsMap`/`ToolOutputMap`，Python 为具名 `TypedDict`），每个绑定都会解析为该工具的规范 JSON 值。每个无损 JSON 绑定调用都会在原生调度契约下重新进入完整工具流水线（并发安全的调用最多可重叠 `maxParallelSubCalls` 个；独占调用单独运行并构成排序屏障），并在日志中与外层调用建立关联。拒绝及其他失败结果会以程序实际可见的 `ToolCallError` 形式拒绝，且只携带 `toolName` 和 `message`；Native 内容和内部错误码留在 Code 契约之外。普通副作用不会回滚，子调用的 `additionalContexts` 会通过父结果延迟，以保持调用／结果相邻。运行结算会中止并排空尚未完成的绑定；运行时失败以 `CodeRunFailedError` 形式出现。参见 [Code Mode 基础](../../../.agents/notes/implemented/feature/2026-06-15-code-mode.md)、[类型化返回契约](../../../.agents/notes/implemented/feature/2026-07-20-code-mode-typed-tool-returns.md)和[代码运行时 seam](../../code-runtime/README.md)。可以运行 `pnpm run demo:code-mode` 试用。 - **SDK 段**（`tools:sdk`，顺序 150）：一个惰性提示词段，每次组装时都会重新生成 `JsonValue`、精确的 `ToolArgsMap` / `ToolOutputMap`、`ToolName`、`ToolCallError` 声明、面向调用作用域可见最终能力的映射 `tools` 命名空间（特殊名称使用带引号的键），以及固定用法说明。其输出具有确定性：工具按字典序排列；工具集合不变时，文本逐字节相同（有利于前缀 cache）。导出的代码生成器 `jsonSchemaToTs` 会处理统一 schema 的每种构造，并将不受支持的原始构造降级为 `unknown`，绝不会在提示词组装期间抛出。 - **分发桥接层**（`run_code` 的 execute）：每个绑定调用都会在分发前快照为无损 JSON（`undefined`、`BigInt`、循环、稀疏数组、`-0` 和特殊对象会使该次调用被拒绝），经由每次运行独有、复用原生并发契约的池调度——调用严格按提交顺序启动，连续的 `isConcurrencySafe` 调用最多可重叠经校验的 `maxParallelSubCalls` 配置个（默认 10；设为 `1` 即恢复串行分发），被分类为独占的调用先排空池、单独运行并阻挡其后的调用——以外层执行的不透明 token 作为 `parent`，并经过完整的 pre-execute → guards → execute → post-execute → result 流水线。成功会返回策略处理后的最终规范值；失败以一条消息到达 worker，并成为 `ToolCallError(toolName, message)`。每个已启动的子调用在进入流水线时记录一条 `tool/code-dispatch-start` 事件（确定性 id `:code:`，按提交顺序编号），并以一条携带完整模型可见 `content`/`isError` 结果的 `tool/code-dispatch` 事件完结（采用 `tool/result` 词汇，因此 UI 会沿原生路径呈现子调用——这对事件的 `time` 字段承载每个子调用的计时）；因 run 结算而被放弃的排队调用两者都不记录。`deriveMessages()` 既不公开这两个事件，也不持久化规范值。token 关联让以提交为语义的观察器能够把内部成功延迟到最终 `run_code` 结果，而无需公开实时外层执行；普通工具副作用不会回滚。每个子调用的 `additionalContexts` 条目都会按分发顺序通过外层 `ToolRunContext` 延迟；循环只在父级 `run_code` 结果之后追加这些上下文，从而保持相邻关系，并且即使程序后来失败，也会保留各自的来源／元数据。 @@ -145,7 +145,7 @@ agent loop 将连续的 `parallel` 调用归入有界滚动池，并把每个 `e #### 模型看到的内容 -Code Mode 会公开生成的 [`run_code` schema](../../../docs/tool-catalog.md#deepseek-aidsh-tools)、下方 SDK 说明，以及生成的精确 `declare const tools` 块。`both` 会同时公开普通 schema 与此 Code Mode 接口。 +Code Mode 会公开生成的 [`run_code` schema](../../../docs/tool-catalog.md#deepseek-aidsh-tools)、下方 SDK 说明，以及按所加载运行时语言生成的精确 SDK 块（TypeScript 的 `declare const tools` 块，或 Python 的 `tools` 声明）。`both` 会同时公开普通 schema 与此 Code Mode 接口。说明与 SDK 块随所加载运行时的语言切换；下方展示 TypeScript 风格（经 [`dsh-code-runtime-worker`](../../code-runtime/code-runtime-worker/README.md)），Python 风格（经 [`dsh-code-runtime-python`](../../code-runtime/code-runtime-python/README.md)）形状相同，只是换成 Python 语法（`await tools.name(args)`、异体名用下标访问、`print(...)` 与顶层 `return`）。 ##### Code Mode SDK 说明 diff --git a/packages/core/tools/src/code-mode.ts b/packages/core/tools/src/code-mode.ts index 244934de3f..38a0654ced 100644 --- a/packages/core/tools/src/code-mode.ts +++ b/packages/core/tools/src/code-mode.ts @@ -11,7 +11,7 @@ import type { ContentBlock } from '@deepseek-ai/dsh-llm' import type { CodeBindingFunction, CodeRunResult, CodeRuntime } from '@deepseek-ai/dsh-code-runtime' import { snapshotJsonValue } from '@deepseek-ai/dsh-session' import type { JsonValue } from '@deepseek-ai/dsh-session' -import { defineTool } from './schema.ts' +import { defineTool, parameterSchemaSpecToJsonSchema } from './schema.ts' import { TOOL_REGISTRY_SCHEDULER } from './index.ts' import type { CodeDispatchLog, ToolDefinition, ToolExecutionResult, ToolRegistry, ToolRunContext } from './index.ts' @@ -56,6 +56,95 @@ export const RUN_CODE_NAME = 'run_code' /** The `tools:sdk` section order: inside the 100–199 tool-guidance band, after per-tool guidance sections. */ export const SDK_SECTION_ORDER = 150 +/** + * The language-specific `run_code` schema text: the tool `description` and its + * `code` parameter description, kept together so a language's two model-facing + * strings share one source of truth. Keyed by `CodeRuntime.language`, mirroring + * `SDK_RENDERERS` in {@link ./index.ts}. The emitted flavor MUST match the + * semantics the same language's SDK instructions promise, so the model never + * receives a TypeScript-shaped schema beside a Python SDK (or vice versa). + */ +interface RunCodeFlavor { + /** The tool `description` the model sees for this language. */ + readonly description: string + /** The `code` parameter's description for this language. */ + readonly codeDescription: string +} + +/** + * The TypeScript flavor: the historical default, and the fallback the schema + * harvest degrades to when no runtime is mounted (the doc-catalog generator + * reads `schemas()` without one). A real assembly always resolves a runtime + * first, so the model never sees this fallback outside its own language. + */ +const TYPESCRIPT_FLAVOR: RunCodeFlavor = { + description: + 'Execute a TypeScript program against the available tools. Write the BODY of an ' + + 'async function (erasable syntax only; top-level `await` and `return` work) and ' + + 'call tools as `await tools.name(args)` per the declarations in the system prompt. ' + + 'Only what you print or return comes back — curate it.', + codeDescription: 'The program: the body of an async TypeScript function.', +} + +/** + * The Python flavor: the body of an async function, top-level `await` and + * `return`, answer via `print` and/or the returned value, matching + * {@link ./py-types.ts}'s SDK instructions. + */ +const PYTHON_FLAVOR: RunCodeFlavor = { + description: + 'Execute a Python program against the available tools. Write the BODY of an ' + + 'async function (top-level `await` and `return` work) and call tools as ' + + '`await tools.name(args)` per the declarations in the system prompt. Answer ' + + 'with `print(...)` and/or `return ` — only that comes back, so curate it.', + codeDescription: 'The program: the body of an async Python function.', +} + +/** Per-language `run_code` schema flavors (see {@link RunCodeFlavor}); one entry per `SDK_RENDERERS` language. */ +const RUN_CODE_FLAVORS: Record = { + typescript: TYPESCRIPT_FLAVOR, + python: PYTHON_FLAVOR, +} + +/** + * The `description` parameter's model-facing description: language-independent + * (the UI label contract is the same for every runtime), shared between the + * static spec and the language-aware `parameters` getter so the two emissions + * can never drift. + */ +const RUN_CODE_DESCRIPTION_PARAM_DESCRIPTION + = 'Clear, concise description of what this program does in active voice, ' + + '5-10 words (shown in the UI). Examples: "Count TODO markers across packages"; ' + + '"Read failing test and its fixture"; "Rename config key in every cordis.yml".' + +/** + * Resolve the {@link RunCodeFlavor} for the loaded runtime's language, read at + * schema-emission time so the model-visible `run_code` schema always matches + * the SDK section's language. When no runtime is mounted the schema harvest + * degrades to {@link TYPESCRIPT_FLAVOR} (a doc-only path — an assembly always + * has one). A mounted runtime whose language has no flavor entry fails loud, + * keeping this table coupled to `SDK_RENDERERS`. + */ +function resolveFlavor(requireRuntime: () => CodeRuntime): RunCodeFlavor { + let runtime: CodeRuntime + try { + runtime = requireRuntime() + } catch { + // No runtime mounted: the only reader here is the static schema harvest + // (doc catalog), which never reaches a model — degrade to the TS default. + return TYPESCRIPT_FLAVOR + } + // Own-property read: a language like `toString`/`constructor` would otherwise + // resolve an inherited Object.prototype member as a flavor. + const flavor = RUN_CODE_FLAVORS[runtime.language] + /* v8 ignore next 3 -- requireRuntime rejects a language absent from SDK_RENDERERS, whose keys + mirror RUN_CODE_FLAVORS; the guard is defense-in-depth against the two tables drifting. */ + if (!Object.hasOwn(RUN_CODE_FLAVORS, runtime.language) || flavor === undefined) { + throw new Error(`dsh-tools: no run_code schema flavor registered for runtime language ${JSON.stringify(runtime.language)}`) + } + return flavor +} + /** * Thrown by `run_code` when the program run itself failed — a program * exception, a budget expiry, an abort, or substrate death. Extends @@ -213,21 +302,21 @@ export interface RunCodeBridgeOptions { */ export function createRunCodeTool(registry: ToolRegistry, options: RunCodeBridgeOptions): ToolDefinition { const { requireRuntime, maxParallel, shapeDispatchLog } = options - return defineTool({ + const definition = defineTool({ name: RUN_CODE_NAME, - description: - 'Execute a TypeScript program against the available tools. Write the BODY of an ' - + 'async function (erasable syntax only; top-level `await` and `return` work) and ' - + 'call tools as `await tools.name(args)` per the declarations in the system prompt. ' - + 'Only what you print or return comes back — curate it.', + // The description and `code` parameter description are placeholders here: + // the language-aware getters installed below replace both, resolving the + // loaded runtime's flavor at schema-emission time so the schema the MODEL + // sees matches the SDK section's language. Argument VALIDATION still keys + // off this static spec (defineTool closes over it), which is language- + // independent (one required string `code`). + description: TYPESCRIPT_FLAVOR.description, parameters: { - code: { type: 'string', required: true, description: 'The program: the body of an async TypeScript function.' }, + code: { type: 'string', required: true, description: TYPESCRIPT_FLAVOR.codeDescription }, description: { type: 'string', required: true, - description: 'Clear, concise description of what this program does in active voice, ' - + '5-10 words (shown in the UI). Examples: "Count TODO markers across packages"; ' - + '"Read failing test and its fixture"; "Rename config key in every cordis.yml".', + description: RUN_CODE_DESCRIPTION_PARAM_DESCRIPTION, }, }, output: { @@ -569,4 +658,22 @@ export function createRunCodeTool(registry: ToolRegistry, options: RunCodeBridge // title and reads durable result content without duplicating a large raw // result into the host view payload. }) + // Resolve the language flavor lazily, at the moment the registry projects the + // schema (`schemaOf` destructures `description`/`parameters`). The definition + // is minted once at registration, before a runtime is known; deferring here + // is the least invasive point that still emits the loaded runtime's language. + Object.defineProperty(definition, 'description', { + enumerable: true, + get: () => resolveFlavor(requireRuntime).description, + }) + Object.defineProperty(definition, 'parameters', { + enumerable: true, + // Recompile through the same spec→schema projection defineTool used, so + // the emitted shape can never drift from the validated one. + get: () => parameterSchemaSpecToJsonSchema({ + code: { type: 'string', required: true, description: resolveFlavor(requireRuntime).codeDescription }, + description: { type: 'string', required: true, description: RUN_CODE_DESCRIPTION_PARAM_DESCRIPTION }, + }) as unknown as Record, + }) + return definition } diff --git a/packages/core/tools/src/index.ts b/packages/core/tools/src/index.ts index f30dce6cd0..17d1935cf1 100644 --- a/packages/core/tools/src/index.ts +++ b/packages/core/tools/src/index.ts @@ -24,6 +24,19 @@ import type { JsonSchemaNode } from './json-schema.ts' import { createRunCodeTool, RUN_CODE_NAME, SDK_SECTION_ORDER } from './code-mode.ts' import { renderToolsSdk } from './ts-types.ts' import type { ToolSdkSchema } from './ts-types.ts' +import { renderToolsSdkPy } from './py-types.ts' + +/** + * Language → SDK-section renderer. The registry looks up the loaded + * `ctx.codeRuntime.language` in this table when assembling the `tools:sdk` + * section under a non-native mode; a runtime whose language is not a key + * fails the assembly loudly (same idiom as `toolOrder` violations). Adding a + * new backend language is a table entry plus its renderer, nothing else. + */ +const SDK_RENDERERS: Record string> = { + typescript: renderToolsSdk, + python: renderToolsSdkPy, +} export { defineTool, @@ -65,6 +78,7 @@ export type { JsonValue } from '@deepseek-ai/dsh-session' export { CodeRunFailedError, RUN_CODE_NAME } from './code-mode.ts' export { jsonSchemaToTs, renderToolsSdk } from './ts-types.ts' +export { jsonSchemaToPy, renderToolsSdkPy } from './py-types.ts' export { defineContentToolFixture, type ContentToolFixtureOptions } from './testing.ts' // The render-intent vocabulary a tool declares via `presentCall`/`presentResult` @@ -762,10 +776,21 @@ export class ToolRegistry extends Service { ctx.systemPrompt.section({ name: 'tools:sdk', order: SDK_SECTION_ORDER, - // Regenerate from the calling scope's visible tools in stable order. + // Regenerate from the calling scope's visible tools in stable order, + // picking the renderer that matches the loaded runtime's language. + // `requireCodeRuntime` already validated the language is in the + // table, so the fallback here is defense-in-depth against a caller + // that bypassed the guard (impossible under normal composition). text: (context) => { - this.requireCodeRuntime() - return renderToolsSdk(this.sdkSchemas(context.scope)) + const runtime = this.requireCodeRuntime() + // Own-property read: a language like `toString`/`constructor` would + // otherwise resolve an inherited Object.prototype member as a renderer. + const render = SDK_RENDERERS[runtime.language] + /* v8 ignore next 3 -- requireCodeRuntime rejects an unknown language before this ever runs. */ + if (!Object.hasOwn(SDK_RENDERERS, runtime.language) || render === undefined) { + throw new Error(`dsh-tools: no SDK renderer registered for runtime language "${runtime.language}"`) + } + return render(this.sdkSchemas(context.scope)) }, }) } @@ -804,8 +829,9 @@ export class ToolRegistry extends Service { if (!runtime) { throw new Error(`dsh-tools: mode "${this.mode}" requires a code runtime — load a ctx.codeRuntime implementation (e.g. @deepseek-ai/dsh-code-runtime-worker) or set tools mode to "native"`) } - if (runtime.language !== 'typescript') { - throw new Error(`dsh-tools: mode "${this.mode}" generates a TypeScript SDK, but the loaded code runtime's language is "${runtime.language}"`) + if (!Object.hasOwn(SDK_RENDERERS, runtime.language)) { + const known = Object.keys(SDK_RENDERERS).map(name => JSON.stringify(name)).join(', ') + throw new Error(`dsh-tools: no SDK renderer registered for runtime language ${JSON.stringify(runtime.language)} (known: ${known})`) } return runtime } diff --git a/packages/core/tools/src/py-types.ts b/packages/core/tools/src/py-types.ts new file mode 100644 index 0000000000..80c1ee7dc5 --- /dev/null +++ b/packages/core/tools/src/py-types.ts @@ -0,0 +1,440 @@ +/** + * Code Mode codegen — Python flavor. The pure projection from registered tool schemas to the + * Python SDK text the model programs against under `runtime.language === 'python'`. Sibling of + * {@link ./ts-types.ts | ts-types.ts}; the two files are two projections of the same registry + * store, keyed by the loaded {@link @deepseek-ai/dsh-code-runtime#CodeRuntime.language | code + * runtime's language}. + * + * In Code Mode the native tool schemas are omitted from the request, so this generated SDK is + * the model's ONLY source for each tool's argument names, required fields, types, descriptions, + * and canonical output shapes. Object-shaped arguments and outputs therefore render as one named + * `TypedDict` per tool (and per nested object), not an opaque `dict[str, Any]`, so the shape + * survives into the program. + * @module @deepseek-ai/dsh-tools/src/py-types + */ + +import { assertSupportedJsonSchema } from './json-schema.ts' +import type { JsonSchemaScalar } from './json-schema.ts' +import type { ToolSdkSchema } from './ts-types.ts' + +/** Property names that are valid bare Python identifiers; anything else is subscripted. */ +const IDENTIFIER = /^[A-Za-z_][A-Za-z0-9_]*$/ + +/** + * Python 3.x soft-keyword-inclusive reserved set. A tool named ``class`` or + * ``lambda`` is legal on the wire but not as an attribute (``tools.class`` + * would be a SyntaxError in the model program), so we render it under + * subscript access — the model still reaches every tool without collisions. + * Underscore-leading names (``_x``, ``__class__``) are also subscript-only: + * dunders resolve on ``object`` before the proxy's fallback hook, and the + * subscript path is the one guaranteed bridge route for them. + * The same set rejects an argument field whose name would be an illegal + * class-syntax `TypedDict` attribute, degrading that object to + * ``dict[str, Any]``. + */ +const RESERVED = new Set([ + 'False', 'None', 'True', 'and', 'as', 'assert', 'async', 'await', 'break', 'class', + 'continue', 'def', 'del', 'elif', 'else', 'except', 'finally', 'for', 'from', 'global', + 'if', 'import', 'in', 'is', 'lambda', 'nonlocal', 'not', 'or', 'pass', 'raise', + 'return', 'try', 'while', 'with', 'yield', 'match', 'case', + // Not a keyword, but CPython refuses to ASSIGN it at compile time + // (`SyntaxError: cannot assign to __debug__`), which is what a TypedDict + // field, a parameter name, and a keyword argument all are. + '__debug__', +]) + +/** `typing` symbols this module may emit, in the deterministic import order. */ +const TYPING_ORDER = ['Any', 'Literal', 'NotRequired', 'Protocol', 'TypedDict'] as const + +/** `indent`-deep line prefix (four spaces per level to match PEP 8 output). */ +function pad(indent: number): string { + return ' '.repeat(indent) +} + +/** + * Collector threaded through {@link renderType}: the emitted `TypedDict` class + * declarations (nested classes precede the parent that references them), the + * class names already taken (for collision suffixing), and the `typing` + * symbols the render actually used. + */ +interface RenderState { + readonly classes: string[] + readonly usedClassNames: Set + readonly typing: Set +} + +/** + * Control characters that survive the whitespace collapse in {@link describe} + * and have no printable form. CPython rejects source containing a NUL outright + * (`SyntaxError: source code string cannot contain null bytes`), whether it + * sits in a docstring or in a comment, so one such byte anywhere in a schema + * description would make the whole generated SDK unparseable — the model's only + * declaration of the tools. The rest are legal but invisible; escaping them + * with the same rule keeps the emitted text readable and the treatment uniform. + */ +const UNPRINTABLE = /[\u0000-\u0008\u000e-\u001f\u007f]/g + +/** + * The collapsed one-line `description` of a schema node (byte-stable across + * formatting churn), or `undefined` when the node carries none. Every caller + * passes an object (validated property nodes, or the ToolSdkSchema itself), + * so only the description field needs guarding. + * + * Control characters left over after the whitespace collapse are rendered as + * their `\xNN` escapes (see {@link UNPRINTABLE}); the escape's own backslash is + * emitted literally by both consumers, since {@link docLines} doubles it into a + * Python source escape and a `#` comment carries it verbatim. + */ +function describe(schema: object): string | undefined { + const description = (schema as Record).description + if (typeof description !== 'string' || description.length === 0) return undefined + return description + .replace(/\s+/g, ' ') + .replace(UNPRINTABLE, char => `\\x${char.charCodeAt(0).toString(16).padStart(2, '0')}`) + .trim() +} + +/** + * One-line docstring for a tool `description`, or no lines when there is none. + * Backslashes are doubled first, every quote is escaped, and a trailing + * backslash cannot survive: a description ending in `"` or an odd backslash + * would otherwise merge with (or escape) the closing triple quote and make + * the generated block — Code Mode's only SDK — syntactically invalid Python. + */ +function docLines(description: unknown, indent: number): string[] { + const collapsed = describe({ description }) + if (collapsed === undefined) return [] + const escaped = collapsed.replaceAll('\\', '\\\\').replaceAll('"', '\\"') + return [`${pad(indent)}"""${escaped}"""`] +} + +/** CamelCase a name into a Python type identifier (non-identifier chars split words; a non-letter head is prefixed). */ +function camelCase(raw: string): string { + const joined = raw + .split(/[^A-Za-z0-9]+/) + .filter(part => part.length > 0) + .map(part => `${part.charAt(0).toUpperCase()}${part.slice(1)}`) + .join('') + return /^[A-Za-z]/.test(joined) ? joined : `Tool${joined}` +} + +/** Reserve a unique class name, suffixing a counter on collision after CamelCase sanitization. */ +function allocateClassName(base: string, state: RenderState): string { + let name = base + for (let n = 2; state.usedClassNames.has(name); n++) name = `${base}${n}` + state.usedClassNames.add(name) + return name +} + +/** + * Render one validated scalar as Python literal text (`True`/`False`, + * JSON-quoted strings, bare numbers). `null` cannot reach here: the `null` + * type renders directly as `None`, and the unified validator rejects a null + * `const`/`enum` entry on every other scalar type. + * + * A beyond-safe-range integral number takes `BigInt` digits rather than + * `String`: Python integers are arbitrary-precision, so the emitted digits ARE + * the value the model programs against, and `String` gives a different integer + * than the double holds (`2 ** 60` prints the rounded `...847000`, not the + * exact `...846976`) or no integer literal at all (`1e21` prints `1e+21`). The + * Python runtime then rejects the advertised literal as not exactly + * representable as a JavaScript number, so the SDK would document a value no + * program can pass. The TS flavor needs no counterpart: its literal is re-read + * by a JS parser back into the same double. + */ +function pyScalar(value: JsonSchemaScalar): string { + if (value === true) return 'True' + if (value === false) return 'False' + if (typeof value === 'string') return JSON.stringify(value) + if (typeof value === 'number' && Number.isInteger(value) && !Number.isSafeInteger(value)) { + return BigInt(value).toString() + } + return String(value) +} + +/** Render a validated scalar `const`/`enum` as `Literal[...]`, falling back to the broad type. */ +function renderConstrainedScalar(node: Record, broad: string, state: RenderState): string { + if (Object.hasOwn(node, 'const')) { + state.typing.add('Literal') + return `Literal[${pyScalar(node.const as JsonSchemaScalar)}]` + } + if (Object.hasOwn(node, 'enum')) { + state.typing.add('Literal') + return `Literal[${(node.enum as JsonSchemaScalar[]).map(pyScalar).join(', ')}]` + } + return broad +} + +/** + * Map one JSON-Schema node to a Python type expression, threading `state` to + * collect the `TypedDict` declarations and `typing` symbols a full render + * needs. `className` is the name to give an object node with properties (and + * the prefix for its nested objects). Handles every unified schema construct — + * `oneOf` (→ `X | Y`), `const`/`enum` (→ `Literal[...]`), `integer` (→ `int`), + * `null` (→ `None`) — and degrades malformed or unsupported inputs to `Any` + * without throwing. {@link jsonSchemaToPy} is the context-free entry point; + * this is the collecting core. + */ +function renderType(schema: unknown, className: string, state: RenderState): string { + interface Frame { + schema: unknown + className: string + phase: 'start' | 'children' + kind?: 'oneOf' | 'array' | 'typeddict' + node?: Record + children: { schema: unknown; className: string }[] + childIndex: number + childTypes: string[] + entries: [string, unknown][] + allocated?: string + validated: boolean + } + const newFrame = (schema: unknown, className: string, validated: boolean): Frame => + ({ schema, className, phase: 'start', children: [], childIndex: 0, childTypes: [], entries: [], validated }) + const frames: Frame[] = [newFrame(schema, className, false)] + let result: string | undefined + /* jscpd:ignore-start -- the explicit-stack walk skeleton deliberately parallels + ts-types.ts's renderSupportedSchema; the two sibling renderers keep symmetric shapes. */ + const finish = (type: string): void => { + frames.pop() + const parent = frames.at(-1) + if (parent === undefined) result = type + else parent.childTypes.push(type) + } + + while (frames.length > 0) { + const frame = frames.at(-1) + /* v8 ignore next -- the loop condition guarantees a current frame. */ + if (frame === undefined) break + + if (frame.phase === 'children') { + if (frame.childIndex < frame.children.length) { + const child = frame.children[frame.childIndex] + /* v8 ignore next -- childIndex is bounded by children.length. */ + if (child === undefined) throw new Error('missing python render child') + frame.childIndex++ + frames.push(newFrame(child.schema, child.className, true)) + continue + } + if (frame.kind === 'oneOf') { + finish(frame.childTypes.join(' | ')) + continue + } + /* jscpd:ignore-end */ + if (frame.kind === 'array') { + // `list[A | B]` needs no parentheses in Python. Array frames always + // schedule exactly one child, so its type is present. + /* v8 ignore next -- the ?? arm needs a childless array frame, which start never builds. */ + finish(`list[${frame.childTypes[0] ?? 'Any'}]`) + continue + } + // typeddict: assemble AFTER the children so any nested class this one + // references is already declared (declaration order = reference order). + const node = frame.node + const name = frame.allocated + /* v8 ignore next -- typeddict frames always set node and allocated at start. */ + if (node === undefined || name === undefined) throw new Error('missing typeddict frame state') + const required = new Set(Array.isArray(node.required) ? node.required.filter((n): n is string => typeof n === 'string') : []) + const lines = [`class ${name}(TypedDict):`] + for (let index = 0; index < frame.entries.length; index++) { + const entry = frame.entries[index] + const fieldType = frame.childTypes[index] + /* v8 ignore next -- entries and childTypes correspond one-to-one. */ + if (entry === undefined || fieldType === undefined) throw new Error('missing typeddict field type') + const [field, fieldSchema] = entry + // The parent node passed assertSupportedJsonSchema, so every property + // value is a validated schema node (an object). + const description = describe(fieldSchema as object) + if (description !== undefined) lines.push(`${pad(1)}# ${description}`) + if (required.has(field)) { + lines.push(`${pad(1)}${field}: ${fieldType}`) + } else { + state.typing.add('NotRequired') + lines.push(`${pad(1)}${field}: NotRequired[${fieldType}]`) + } + } + // TypedDict syntax cannot express openness, so an open object states it + // in-band: the annotation is advisory either way, and Code Mode omits + // the native schemas, making this line the model's only signal that + // extra keys are accepted. + if (node.additionalProperties !== false) { + lines.push(`${pad(1)}# Additional keys beyond those declared are allowed.`) + } + // A closed empty object still needs a class body (`pass`) to be valid + // Python; the declared emptiness is the information. + if (lines.length === 1) lines.push(`${pad(1)}pass`) + state.classes.push(lines.join('\n')) + finish(name) + continue + } + + frame.phase = 'children' + // Validate the WHOLE tree once at the root frame (the assertion walks it + // with an explicit stack); child frames are inside that validated tree, so + // re-asserting them would make a deep schema quadratic. + if (!frame.validated) { + try { + assertSupportedJsonSchema(frame.schema) + } catch { + state.typing.add('Any') + finish('Any') + continue + } + } + const node = frame.schema as Record + if (Object.hasOwn(node, 'oneOf')) { + frame.kind = 'oneOf' + frame.children = (node.oneOf as unknown[]).map((branch, index) => ({ schema: branch, className: `${frame.className}${index + 1}` })) + continue + } + if (!Object.hasOwn(node, 'type')) { + state.typing.add('Any') + finish('Any') + continue + } + switch (node.type) { + case 'string': finish(renderConstrainedScalar(node, 'str', state)); break + case 'number': finish(renderConstrainedScalar(node, 'float', state)); break + case 'integer': finish(renderConstrainedScalar(node, 'int', state)); break + case 'boolean': finish(renderConstrainedScalar(node, 'bool', state)); break + case 'null': finish('None'); break + case 'array': { + if (!Object.hasOwn(node, 'items')) { + state.typing.add('Any') + finish('list[Any]') + break + } + // An array of objects names its item type after the array field. + frame.kind = 'array' + frame.children = [{ schema: node.items, className: frame.className }] + break + } + case 'object': { + const properties = node.properties + if (typeof properties !== 'object' || properties === null) { + state.typing.add('Any') + finish('dict[str, Any]') + break + } + const entries = Object.entries(properties as Record) + // An empty `className` marks the context-free `jsonSchemaToPy` entry: + // there is no naming context to declare into, so degrade. A field + // name that is not a legal Python attribute is inexpressible as a + // class-syntax `TypedDict` field, so such an object degrades whole. + // A leading-double-underscore non-dunder field (`__token`) would be + // NAME-MANGLED inside class syntax (`_ClassName__token`), describing a + // different JSON key than the registered schema — degrade like any + // other inexpressible field name. + if (className === '' || !entries.every(([name]) => IDENTIFIER.test(name) && !RESERVED.has(name) && !(name.startsWith('__') && !name.endsWith('__')))) { + state.typing.add('Any') + finish('dict[str, Any]') + break + } + // An OPEN empty object is any dict; a CLOSED empty object declares an + // empty TypedDict so "no keys accepted" survives into the SDK. + if (entries.length === 0 && node.additionalProperties !== false) { + state.typing.add('Any') + finish('dict[str, Any]') + break + } + frame.kind = 'typeddict' + frame.node = node + frame.allocated = allocateClassName(frame.className, state) + state.typing.add('TypedDict') + frame.entries = entries + // frame.allocated was assigned two statements up; the ?? arm is for the type system only. + /* v8 ignore next -- allocated is always set before children are built. */ + frame.children = entries.map(([field, child]) => ({ schema: child, className: `${frame.allocated ?? ''}${camelCase(field)}` })) + break + } + /* v8 ignore next 4 -- assertSupportedJsonSchema narrowed this closed type union. */ + default: { + state.typing.add('Any') + finish('Any') + } + } + } + /* v8 ignore next -- every root frame produces one expression. */ + return result ?? 'Any' +} + +/** + * Map one JSON-Schema node to a context-free Python type expression from the + * `typing` module. Handles every unified schema construct — `object` (degraded + * to `dict[str, Any]`: naming a `TypedDict` requires the render context that + * {@link renderToolsSdkPy} supplies), `const`/`enum` (→ `Literal[...]`), + * `oneOf` (→ union), `string`/`number`/`integer`/`boolean`/`null`, `array` + * (`items` → `list[T]`) — and returns `Any` for anything else, without + * throwing. Type annotations in the emitted SDK are advisory: Python does not + * enforce them at runtime, matching the TS flavor's advisory-type stance. + * @param schema - the JSON-Schema node (any shape; hostile inputs degrade). + * @returns the Python type text. + */ +export function jsonSchemaToPy(schema: unknown): string { + // A throwaway state whose class collector never escapes: an object with + // properties has nowhere to declare its TypedDict and degrades to + // dict[str, Any]. renderToolsSdkPy drives the named-TypedDict path. + return renderType(schema, '', { classes: [], usedClassNames: new Set(), typing: new Set() }) +} + +/** The fixed model-facing usage contract rendered above the declarations. */ +const SDK_INSTRUCTIONS = `## Writing code for run_code + +Pass \`run_code\` the body of an async Python function (top-level \`await\` and \`return\` both work). Inside the program: + +- Call tools as \`await tools.name(args)\` — subscript access for exotic names or reserved words: \`await tools["my-tool"](args)\`. Every call resolves to the tool's typed canonical JSON value (each method's return type below). Tool arguments must be lossless JSON. +- A FAILED tool call raises \`ToolCallError\`, whose \`toolName\` identifies the failed tool and whose message is human-readable — wrap in \`try/except\` to handle and continue. +- Independent read-only calls MAY overlap under \`asyncio.gather\` (safe calls run concurrently; mutating calls run alone, in submission order). Sequence dependent work with \`await\`. +- Emit the run's answer with \`print(...)\` and/or a top-level \`return \`; the returned value must be lossless JSON. ONLY what you print and the returned value come back — intermediate tool results never enter the conversation, so extract just what you need. + +The available tools:` + +/** + * Render the full `tools:sdk` prompt section under `runtime.language === + * 'python'`: the Python-flavored usage instructions plus one named `TypedDict` + * per tool argument or output object (and per nested object) and one awaitable + * method per visible tool on a `Tools` protocol — typed args in, the tool's + * canonical output value out — with a `tools: Tools` singleton the model calls + * into. The `typing` import line lists exactly the symbols the render used. + * Deterministic — tools are emitted in lexicographic name order, and class + * declarations precede the protocol in that same order (nested classes before + * the parent that references them), so an unchanged tool set produces + * byte-identical text across assemblies. + * @param schemas - the tool schemas plus canonical output schemas to declare + * (the caller excludes `run_code` itself). + * @returns the complete section text. + */ +export function renderToolsSdkPy(schemas: ToolSdkSchema[]): string { + const sorted = [...schemas].sort((a, b) => a.name < b.name ? -1 : a.name > b.name ? 1 : 0) + const state: RenderState = { classes: [], usedClassNames: new Set(), typing: new Set(['Protocol']) } + const inlineMembers: string[] = [] + const subscriptMembers: string[] = [] + for (const schema of sorted) { + const argType = renderType(schema.parameters, `${camelCase(schema.name)}Args`, state) + const outputType = renderType(schema.output, `${camelCase(schema.name)}Output`, state) + if (IDENTIFIER.test(schema.name) && !RESERVED.has(schema.name) && !schema.name.startsWith('_')) { + inlineMembers.push(...docLines(schema.description, 1)) + inlineMembers.push(`${pad(1)}async def ${schema.name}(self, args: ${argType}) -> ${outputType}: ...`) + } else { + // Not a legal attribute name — the model reaches it via ``tools[name]``. + // The stub lists it as a subscript comment (referencing the named + // TypedDicts too) so a reader sees what is accessible; runtime resolution + // goes through the proxy's __getitem__. + subscriptMembers.push(`${pad(1)}# tools[${JSON.stringify(schema.name)}](args: ${argType}) -> ${outputType}`) + const description = describe(schema) + if (description !== undefined) subscriptMembers.push(`${pad(1)}# ${description}`) + } + } + // Subscript entries are COMMENTS, not statements: a class body of only + // comments fails to parse, so `pass` is required whenever no inline method + // exists — including the subscript-only tool set. + const bodyLines = inlineMembers.length > 0 + ? [...inlineMembers, ...subscriptMembers] + : [`${pad(1)}pass`, ...subscriptMembers] + const body = bodyLines.join('\n') + const imports = TYPING_ORDER.filter(symbol => state.typing.has(symbol)) + const classBlock = state.classes.length > 0 ? `${state.classes.join('\n\n')}\n\n` : '' + const errorDeclaration = 'class ToolCallError(Exception):\n toolName: str' + const declaration = `from typing import ${imports.join(', ')}\n\n${errorDeclaration}\n\n${classBlock}class Tools(Protocol):\n${body}\n\ntools: Tools` + return `${SDK_INSTRUCTIONS}\n\n\`\`\`python\n${declaration}\n\`\`\`` +} diff --git a/packages/core/tools/tests/code-mode.spec.ts b/packages/core/tools/tests/code-mode.spec.ts index b06e29866d..ca488738ce 100644 --- a/packages/core/tools/tests/code-mode.spec.ts +++ b/packages/core/tools/tests/code-mode.spec.ts @@ -335,9 +335,51 @@ describe('mode-aware wire contribution', () => { await expect(systemPrompt.assemble()).rejects.toThrow(/requires a code runtime/) }) - it("rejects every assembly when the runtime's language is not typescript", async () => { - const { systemPrompt } = await setup({ mode: 'code', runtime: { language: 'python' } }) - await expect(systemPrompt.assemble()).rejects.toThrow(/language is "python"/) + it('rejects every assembly when the runtime language has no registered SDK renderer', async () => { + const { systemPrompt } = await setup({ mode: 'code', runtime: { language: 'ruby' } }) + await expect(systemPrompt.assemble()).rejects.toThrow(/no SDK renderer registered for runtime language "ruby"/) + }) + + it('assembles under a python runtime by picking the Python SDK renderer', async () => { + const { ctx, systemPrompt } = await setup({ mode: 'code', runtime: { language: 'python' } }) + registerEcho(ctx) + const assembly = await systemPrompt.assemble() + const sdk = assembly.sections.find(section => section.name === 'tools:sdk') + expect(sdk?.text).toContain('class Tools(Protocol):') + expect(sdk?.text).toContain('async def echo(self, args:') + expect(sdk?.text).toContain('top-level `await`') + }) + + it('emits a TypeScript-flavored run_code schema under a typescript runtime', async () => { + const { ctx, systemPrompt } = await setup({ mode: 'code', runtime: { language: 'typescript' } }) + registerEcho(ctx) + const assembly = await systemPrompt.assemble() + const runCodeSchema = assembly.tools.find(tool => tool.name === RUN_CODE_NAME) + expect(runCodeSchema?.description).toContain('Execute a TypeScript program') + expect(runCodeSchema?.description).toContain('BODY of an') + const codeParam = (runCodeSchema?.parameters as { properties: { code: { description: string } } }).properties.code + expect(codeParam.description).toBe('The program: the body of an async TypeScript function.') + }) + + it('emits a Python-flavored run_code schema under a python runtime (matches the SDK language)', async () => { + const { ctx, systemPrompt } = await setup({ mode: 'code', runtime: { language: 'python' } }) + registerEcho(ctx) + const assembly = await systemPrompt.assemble() + const runCodeSchema = assembly.tools.find(tool => tool.name === RUN_CODE_NAME) + expect(runCodeSchema?.description).toContain('Execute a Python program') + expect(runCodeSchema?.description).toContain('`return `') + expect(runCodeSchema?.description).not.toContain('TypeScript') + const codeParam = (runCodeSchema?.parameters as { properties: { code: { description: string } } }).properties.code + expect(codeParam.description).toBe('The program: the body of an async Python function.') + }) + + it('fails loud when the runtime language has no run_code schema flavor', async () => { + // A language with an SDK renderer registered but (hypothetically) no schema + // flavor would fail here; a language with neither fails earlier at + // requireCodeRuntime. Both guards keep the two tables coupled. + const { ctx, systemPrompt } = await setup({ mode: 'code', runtime: { language: 'ruby' } }) + registerEcho(ctx) + await expect(systemPrompt.assemble()).rejects.toThrow(/no SDK renderer registered for runtime language "ruby"/) }) it("rejects the assembly when toolOrder names a native tool that mode 'code' no longer contributes", async () => { diff --git a/packages/core/tools/tests/py-types.spec.ts b/packages/core/tools/tests/py-types.spec.ts new file mode 100644 index 0000000000..93b9990ac3 --- /dev/null +++ b/packages/core/tools/tests/py-types.spec.ts @@ -0,0 +1,462 @@ +import { describe, expect, it } from 'vitest' +import { jsonSchemaToPy, renderToolsSdkPy } from '@deepseek-ai/dsh-tools/src/py-types.ts' +import { parameterSchemaSpecToJsonSchema } from '@deepseek-ai/dsh-tools' +import type { ToolSdkSchema } from '@deepseek-ai/dsh-tools/src/ts-types.ts' + +describe('jsonSchemaToPy', () => { + it('maps the defineTool DSL subset', () => { + const cases: [unknown, string][] = [ + [{ type: 'string' }, 'str'], + [{ type: 'number' }, 'float'], + [{ type: 'boolean' }, 'bool'], + [{ type: 'string', enum: ['a', 'b'] }, 'Literal["a", "b"]'], + [{ type: 'array', items: { type: 'number' } }, 'list[float]'], + [{ type: 'array', items: { type: 'string', enum: ['x', 'y'] } }, 'list[Literal["x", "y"]]'], + [{ type: 'array' }, 'list[Any]'], + [{ type: 'object' }, 'dict[str, Any]'], + [{ type: 'object', properties: {} }, 'dict[str, Any]'], + [{ type: 'object', properties: { x: { type: 'string' } } }, 'dict[str, Any]'], + ] + for (const [schema, expected] of cases) { + expect(jsonSchemaToPy(schema), JSON.stringify(schema)).toBe(expected) + } + }) + + it('is total: unsupported or hostile constructs degrade to Any, never throw', () => { + const cases: unknown[] = [ + undefined, + null, + 42, + 'string-schema', + {}, + { oneOf: 7 }, + { $ref: '#/defs/x' }, + { type: 'object', properties: 7 }, + { type: 'string', enum: [1, 2] }, + { type: 'string', enum: [] }, + ] + for (const schema of cases) { + expect(() => jsonSchemaToPy(schema), JSON.stringify(schema)).not.toThrow() + } + expect(jsonSchemaToPy({ type: 'integer' })).toBe('int') + expect(jsonSchemaToPy({ type: 'string', const: 'fixed' })).toBe('Literal["fixed"]') + expect(jsonSchemaToPy({ type: 'boolean', const: true })).toBe('Literal[True]') + expect(jsonSchemaToPy({ type: 'number', const: 1.5 })).toBe('Literal[1.5]') + expect(jsonSchemaToPy({ type: 'boolean', enum: [false] })).toBe('Literal[False]') + expect(jsonSchemaToPy({ type: 'null' })).toBe('None') + expect(jsonSchemaToPy({ oneOf: [{ type: 'string' }, { type: 'null' }] })).toBe('str | None') + expect(jsonSchemaToPy({ oneOf: [] })).toBe('Any') + expect(jsonSchemaToPy({ type: 'object', properties: 7 })).toBe('Any') + expect(jsonSchemaToPy({ type: 'string', enum: [1, 2] })).toBe('Any') + expect(jsonSchemaToPy({ type: 'string', enum: [] })).toBe('Any') + }) + + it('emits exact digits for a beyond-safe-range integer literal', () => { + // Python integers are arbitrary-precision, so the emitted digits ARE the + // value the model programs against. `String(2 ** 60)` prints the rounded + // ...847000, which is a DIFFERENT integer from the double's exact + // ...846976 — the Python runtime would reject the advertised literal as + // not exactly representable as a JavaScript number, so the SDK would + // document a value no program can pass. + expect(jsonSchemaToPy({ type: 'integer', const: 2 ** 60 })).toBe('Literal[1152921504606846976]') + expect(jsonSchemaToPy({ type: 'integer', enum: [2 ** 60, -(2 ** 60)] })) + .toBe('Literal[1152921504606846976, -1152921504606846976]') + // `String(1e21)` prints `1e+21`, not a Python integer literal at all. The + // rule keys off the VALUE, not the declared type, so a `number` const that + // happens to be an integral double is spelled the same exact way (both + // spellings denote the same double, and only the digits also denote the + // same Python integer). + expect(jsonSchemaToPy({ type: 'integer', const: 1e21 })).toBe('Literal[1000000000000000000000]') + expect(jsonSchemaToPy({ type: 'number', const: 1e21 })).toBe('Literal[1000000000000000000000]') + // Within the safe range, and for non-integral numbers, the plain spelling + // is already exact and stays unchanged. + expect(jsonSchemaToPy({ type: 'integer', const: 2 ** 53 - 1 })).toBe('Literal[9007199254740991]') + expect(jsonSchemaToPy({ type: 'number', const: 1e-7 })).toBe('Literal[1e-7]') + }) +}) + +describe('renderToolsSdkPy', () => { + const bash: ToolSdkSchema = { + name: 'bash', + description: 'Run a shell command.', + parameters: parameterSchemaSpecToJsonSchema({ command: { type: 'string', required: true } }) as unknown as Record, + output: { type: 'string' }, + } + const exotic: ToolSdkSchema = { + name: 'my-mcp.tool', + description: 'Exotic name.', + parameters: parameterSchemaSpecToJsonSchema({}) as unknown as Record, + output: { type: 'string' }, + } + const reserved: ToolSdkSchema = { + name: 'class', + description: 'Uses a reserved Python word.', + parameters: parameterSchemaSpecToJsonSchema({}) as unknown as Record, + output: { type: 'string' }, + } + + it('declares identifier tools as async methods and lists exotic/reserved names as subscript comments', () => { + const text = renderToolsSdkPy([exotic, bash, reserved]) + expect(text).toContain('class Tools(Protocol):') + // The argument object is a named TypedDict, not an opaque dict. + expect(text).toContain('class BashArgs(TypedDict):') + expect(text).toContain('async def bash(self, args: BashArgs) -> str: ...') + // Empty-property tools keep the opaque dict (nothing to name). + expect(text).toContain('# tools["my-mcp.tool"](args: dict[str, Any]) -> str') + expect(text).toContain('# tools["class"](args: dict[str, Any]) -> str') + // Fixed instruction lines the model relies on. + expect(text).toContain('top-level `await`') + expect(text).toContain('ToolCallError') + expect(text).toContain('class ToolCallError(Exception):') + expect(text).toContain('MAY overlap under `asyncio.gather`') + expect(text).toContain('lossless JSON') + expect(text).toContain('```python') + expect(text).toContain('tools: Tools') + }) + + it('renders required as plain fields and optional as NotRequired, with per-field description comments', () => { + const tool: ToolSdkSchema = { + name: 'search', + description: 'Search for text.', + parameters: parameterSchemaSpecToJsonSchema({ + query: { type: 'string', required: true, description: 'What to search for.' }, + limit: { type: 'number', description: 'Max results.' }, + }) as unknown as Record, + output: { type: 'string' }, + } + const text = renderToolsSdkPy([tool]) + expect(text).toContain('class SearchArgs(TypedDict):') + expect(text).toContain(' # What to search for.') + expect(text).toContain(' query: str') + expect(text).toContain(' # Max results.') + expect(text).toContain(' limit: NotRequired[float]') + expect(text).toContain('async def search(self, args: SearchArgs) -> str: ...') + // NotRequired is imported because an optional field used it; Any is NOT, + // since every type here is concrete — the import line lists only what ran. + expect(text).toContain('from typing import NotRequired, Protocol, TypedDict') + }) + + it('prefixes Tool when a name CamelCases to a non-letter head, and degrades a malformed schema to Any', () => { + const tool: ToolSdkSchema = { + name: '1st-tool', // subscript path; CamelCases to "1stTool" → prefixed "Tool1stTool" + description: 'Hostile-shape probe.', + // Malformed node: the unified schema validator rejects it whole, so the + // args position degrades to Any (registration would refuse this schema; + // the renderer just must not throw on it). + parameters: { type: 'object', properties: { field: { type: 'string', description: 42 } } }, + output: { type: 'object', additionalProperties: false, properties: { ok: { type: 'boolean' } }, required: ['ok'] }, + } + const text = renderToolsSdkPy([tool]) + expect(text).toContain('# tools["1st-tool"](args: Any) -> Tool1stToolOutput') + expect(text).toContain('class Tool1stToolOutput(TypedDict):') + expect(text).toContain(' ok: bool') + }) + + it('treats every field as optional when the object carries no required array', () => { + const tool: ToolSdkSchema = { + name: 'all_optional', + description: 'No required array.', + parameters: { type: 'object', properties: { flag: { type: 'boolean' } } }, + output: { type: 'string' }, + } + const text = renderToolsSdkPy([tool]) + expect(text).toContain(' flag: NotRequired[bool]') + }) + + it('renders an enum inside an object property as a Literal field', () => { + const tool: ToolSdkSchema = { + name: 'mode_tool', + description: 'Pick a mode.', + parameters: parameterSchemaSpecToJsonSchema({ + mode: { type: 'string', required: true, enum: ['fast', 'slow'] }, + }) as unknown as Record, + output: { type: 'string' }, + } + const text = renderToolsSdkPy([tool]) + expect(text).toContain('class ModeToolArgs(TypedDict):') + expect(text).toContain(' mode: Literal["fast", "slow"]') + expect(text).toContain('from typing import Literal, Protocol, TypedDict') + }) + + it('renders one level of nested object as its own named TypedDict declared before the parent', () => { + const tool: ToolSdkSchema = { + name: 'workflow', + description: 'Run a workflow.', + parameters: parameterSchemaSpecToJsonSchema({ + meta: { + type: 'object', + required: true, + additionalProperties: false, + description: 'Identity block.', + properties: { + name: { type: 'string', required: true, description: 'Short name.' }, + phases: { + type: 'array', + items: { + type: 'object', + additionalProperties: false, + properties: { title: { type: 'string', required: true, description: 'Phase title.' } }, + }, + }, + }, + }, + }) as unknown as Record, + output: { type: 'string' }, + } + const text = renderToolsSdkPy([tool]) + // Nested class for the `meta` object, and a further nested class for the + // array item object, each named after its field path. + expect(text).toContain('class WorkflowArgsMeta(TypedDict):') + expect(text).toContain('class WorkflowArgsMetaPhases(TypedDict):') + expect(text).toContain(' meta: WorkflowArgsMeta') + expect(text).toContain(' phases: NotRequired[list[WorkflowArgsMetaPhases]]') + // Dependency-before-dependent: the item class precedes its container, + // which precedes the top-level args class, which precedes the protocol. + expect(text.indexOf('class WorkflowArgsMetaPhases')).toBeLessThan(text.indexOf('class WorkflowArgsMeta(TypedDict):')) + expect(text.indexOf('class WorkflowArgsMeta(TypedDict):')).toBeLessThan(text.indexOf('class WorkflowArgs(TypedDict):')) + expect(text.indexOf('class WorkflowArgs(TypedDict):')).toBeLessThan(text.indexOf('class Tools(Protocol):')) + }) + + it('suffixes a counter when two tools CamelCase to the same class base', () => { + const a: ToolSdkSchema = { + name: 'my-tool', + description: 'Dash form.', + parameters: parameterSchemaSpecToJsonSchema({ x: { type: 'string', required: true } }) as unknown as Record, + output: { type: 'string' }, + } + const b: ToolSdkSchema = { + name: 'my.tool', + description: 'Dot form.', + parameters: parameterSchemaSpecToJsonSchema({ y: { type: 'string', required: true } }) as unknown as Record, + output: { type: 'string' }, + } + const text = renderToolsSdkPy([a, b]) + // Both sanitize to `MyToolArgs`; the second collides and gets a suffix. + expect(text).toContain('class MyToolArgs(TypedDict):') + expect(text).toContain('class MyToolArgs2(TypedDict):') + }) + + it('references the named TypedDict from a reserved/subscript tool too', () => { + const tool: ToolSdkSchema = { + name: 'class', + description: 'Reserved word tool.', + parameters: parameterSchemaSpecToJsonSchema({ value: { type: 'string', required: true } }) as unknown as Record, + output: { type: 'string' }, + } + const text = renderToolsSdkPy([tool]) + expect(text).toContain('class ClassArgs(TypedDict):') + expect(text).toContain('# tools["class"](args: ClassArgs) -> str') + }) + + it('degrades an object to dict[str, Any] when a field name is not a legal Python attribute', () => { + const tool: ToolSdkSchema = { + name: 'weird_fields', + description: 'Has an illegal field name.', + parameters: { type: 'object', properties: { 'a-b': { type: 'string' } } }, + output: { type: 'string' }, + } + const text = renderToolsSdkPy([tool]) + expect(text).toContain('async def weird_fields(self, args: dict[str, Any]) -> str: ...') + expect(text).not.toContain('WeirdFieldsArgs') + }) + + it('renders docstrings for descriptions and orders emissions lexicographically', () => { + const text = renderToolsSdkPy([bash, exotic]) + expect(text).toContain('"""Run a shell command."""') + // Descriptions on subscript names ride as a comment beside their entry. + expect(text).toContain('# tools["my-mcp.tool"]') + expect(text).toContain('# Exotic name.') + // Lexicographic: `bash` before `my-mcp.tool` (identifier methods first, + // then subscript comments — the emitter partitions). + expect(text.indexOf('async def bash')).toBeLessThan(text.indexOf('# tools["my-mcp.tool"]')) + }) + + it('is deterministic: byte-identical output regardless of input order or duplication', () => { + expect(renderToolsSdkPy([bash, exotic])).toBe(renderToolsSdkPy([exotic, bash])) + expect(renderToolsSdkPy([bash, bash])).toBe(renderToolsSdkPy([bash, bash])) + }) + + it('renders a pass body and a minimal import for an empty tool set', () => { + const text = renderToolsSdkPy([]) + expect(text).toContain('class Tools(Protocol):') + expect(text).toContain(' pass') + // Nothing but the protocol is used, so the import line is just Protocol. + expect(text).toContain('from typing import Protocol') + }) + + it('omits the docstring/comment when a schema has no description', () => { + const undescribedIdentifier: ToolSdkSchema = { + name: 'plain', + description: '', + parameters: parameterSchemaSpecToJsonSchema({}) as unknown as Record, + output: { type: 'string' }, + } + const undescribedExotic: ToolSdkSchema = { + name: 'weird-name', + description: '', + parameters: parameterSchemaSpecToJsonSchema({}) as unknown as Record, + output: { type: 'string' }, + } + const text = renderToolsSdkPy([undescribedIdentifier, undescribedExotic]) + // Identifier method appears without a docstring line above it. + expect(text).toContain('async def plain(self, args: dict[str, Any]) -> str: ...') + expect(text).not.toContain('"""') + // Subscript entry appears without the "# ..." description follow-up. + expect(text).toContain('# tools["weird-name"]') + expect(text.split('\n').every(line => !line.startsWith(' # '))).toBe(true) + }) + + it('marks an open object TypedDict and declares a closed empty object', () => { + const t: ToolSdkSchema = { + name: 'openness', + description: '', + parameters: { + type: 'object', + additionalProperties: false, + properties: { + open: { type: 'object', additionalProperties: true, properties: { x: { type: 'string' } }, required: ['x'] }, + closedEmpty: { type: 'object', additionalProperties: false, properties: {} }, + }, + required: ['open', 'closedEmpty'], + }, + output: { type: 'string' }, + } + const text = renderToolsSdkPy([t]) + // The open nested object carries the in-band openness note... + expect(text).toContain('class OpennessArgsOpen(TypedDict):') + expect(text).toMatch(/class OpennessArgsOpen\(TypedDict\):\n x: str\n # Additional keys beyond those declared are allowed\./) + // ...the closed root does not... + expect(text).toMatch(/class OpennessArgs\(TypedDict\):\n open: OpennessArgsOpen\n closedEmpty: OpennessArgsClosedEmpty\n\n/) + // ...and a closed EMPTY object declares an empty TypedDict rather than + // degrading to dict[str, Any] (which would falsely accept any keys). + expect(text).toMatch(/class OpennessArgsClosedEmpty\(TypedDict\):\n pass/) + expect(text).toContain('closedEmpty: OpennessArgsClosedEmpty') + }) + + it('renders a deeply nested array schema without exhausting the call stack', () => { + // The registry supports depth-unbounded schemas; the renderer must not + // reintroduce a recursion limit during prompt assembly. + let deep: Record = { type: 'string' } + for (let i = 0; i < 20000; i++) deep = { type: 'array', items: deep } + const type = jsonSchemaToPy(deep) + expect(type.startsWith('list[list[')).toBe(true) + expect(type.endsWith(']]')).toBe(true) + expect(type).toContain('str') + expect(type.length).toBe('list['.length * 20000 + 'str'.length + ']'.repeat(20000).length) + }) + + it('emits pass for a subscript-only tool set (comments are not statements)', () => { + const t: ToolSdkSchema = { + name: 'my-exotic.tool', + description: '', + parameters: parameterSchemaSpecToJsonSchema({}) as unknown as Record, + output: { type: 'string' }, + } + const text = renderToolsSdkPy([t]) + // The class body must contain a statement before the subscript comments. + expect(text).toMatch(/class Tools\(Protocol\):\n pass\n # tools\["my-exotic\.tool"\]/) + }) + + it('degrades an object whose field would be name-mangled (__token) to dict[str, Any]', () => { + // Class-syntax TypedDict mangles a leading-double-underscore non-dunder + // annotation to _ClassName__token — a different JSON key than the schema. + const t: ToolSdkSchema = { + name: 'mangler', + description: '', + parameters: { + type: 'object', + additionalProperties: false, + properties: { __token: { type: 'string' } }, + required: ['__token'], + }, + output: { type: 'string' }, + } + const text = renderToolsSdkPy([t]) + expect(text).toContain('async def mangler(self, args: dict[str, Any]) -> str: ...') + expect(text).not.toContain('__token:') + // Dunder-form fields (__meta__) are NOT mangled and stay expressible. + const dunder: ToolSdkSchema = { + name: 'dunder', + description: '', + parameters: { + type: 'object', + additionalProperties: false, + properties: { __meta__: { type: 'string' } }, + required: ['__meta__'], + }, + output: { type: 'string' }, + } + expect(renderToolsSdkPy([dunder])).toContain('__meta__: str') + }) + + it('degrades an object with a __debug__ field, which CPython refuses to assign', () => { + // `__debug__` is a legal identifier and dunder-form, so it clears both the + // identifier rule and the name-mangling rule, but CPython rejects the + // annotation at COMPILE time (`SyntaxError: cannot assign to __debug__`) — + // and this block is Code Mode's only SDK, so it must always parse. + const t: ToolSdkSchema = { + name: 'debugger', + description: '', + parameters: { + type: 'object', + additionalProperties: false, + properties: { __debug__: { type: 'string' } }, + required: ['__debug__'], + }, + output: { type: 'string' }, + } + const text = renderToolsSdkPy([t]) + expect(text).toContain('async def debugger(self, args: dict[str, Any]) -> str: ...') + expect(text).not.toContain('__debug__') + }) + + it('escapes quotes and backslashes in descriptions so the docstring stays valid Python', () => { + // A description ending in `"` or an odd backslash would otherwise merge + // with (or escape) the closing triple quote — and this block is Code + // Mode's only SDK, so it must always parse. + const make = (description: string): ToolSdkSchema => ({ + name: 'weird', + description, + parameters: parameterSchemaSpecToJsonSchema({}) as unknown as Record, + output: { type: 'string' }, + }) + const trailingQuote = renderToolsSdkPy([make('ends in a quote"')]) + expect(trailingQuote).toContain(String.raw`"""ends in a quote\""""`) + const trailingBackslash = renderToolsSdkPy([make('ends in a backslash\\')]) + expect(trailingBackslash).toContain(String.raw`"""ends in a backslash\\"""`) + const tripleQuote = renderToolsSdkPy([make('contains """ triple quote')]) + expect(tripleQuote).toContain(String.raw`"""contains \"\"\" triple quote"""`) + }) + + it('escapes unprintable control characters, which CPython refuses inside source at all', () => { + // `compile()` raises `SyntaxError: source code string cannot contain null + // bytes` for a NUL ANYWHERE in the source text, including inside a string + // literal or a comment, so a NUL that survives normalization into a + // docstring or a `#` field comment stops this block — Code Mode's only SDK — + // from parsing at all. The whitespace collapse does not remove it (a NUL is + // not whitespace). Rendering it as a visible escape keeps the source + // parseable and still shows the model what the schema said. + const make = (description: string): ToolSdkSchema => ({ + name: 'weird', + description, + parameters: parameterSchemaSpecToJsonSchema({ + field: { type: 'string', required: true, description }, + }) as unknown as Record, + output: { type: 'string' }, + }) + const nul = renderToolsSdkPy([make('before\u0000after')]) + // Both emission sites: the class docstring and the `#` field comment. The + // docstring's backslash is doubled by the same escaping that keeps a literal + // backslash from escaping the closing triple quote, so Python parses it back + // to the visible `\x00` the comment shows directly. Neither carries the byte. + expect(nul).not.toContain('\u0000') + expect(nul).toContain(String.raw`"""before\\x00after"""`) + expect(nul).toContain(String.raw`# before\x00after`) + // The other C0 controls and DEL escape on the same path. Tab, newline and + // carriage return never reach it: the whitespace collapse folds them to a + // space first. + const others = renderToolsSdkPy([make('bell\u0007esc\u001bdel\u007f')]) + expect(others).toContain(String.raw`bell\x07esc\x1bdel\x7f`) + expect(renderToolsSdkPy([make('tab\tnewline\ncr\r')])).toContain('"""tab newline cr"""') + }) +})