ci: validate and publish Python runtime wheels
This commit is contained in:
Yichen Jiang
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
README.md: 294cbd1eb0690de17fa70f3049bb4677e1fd135a
|
||||
README.zh.md: f89d1612f6ac454b3a57515b5a1a0f3c59c579d1
|
||||
README.md: a55b3b90bcb2329f58f063edc7edc733bad030d1
|
||||
README.zh.md: f99cf2302d9dbad36fff1616ae33b40b8bb02f4e
|
||||
+11
-5
@@ -49,20 +49,26 @@ Two flavors, both for repo members:
|
||||
|
||||
## Distributing the Python packages
|
||||
|
||||
Build one wheel per package; the runtime wheel embeds whatever executables are present under `runtime/` at build time (and always the default `cordis.yml`), and excludes `runtime/node/`:
|
||||
Python releases use stable tags of the form `python-vX.Y.Z`. The common staging script derives both distribution versions from the tag, pins `deepseek-harness-runtime-bin==X.Y.Z` in the SDK metadata, and rejects any other tag form. Build the pure SDK wheel once and one runtime wheel on each native platform:
|
||||
|
||||
```sh
|
||||
(cd python/sdk-runtime && uv build) # or: hatch build
|
||||
(cd python/sdk && uv build)
|
||||
pip install dist/deepseek_harness_runtime_bin-*.whl dist/deepseek_harness-*.whl
|
||||
python scripts/build-python-release.py --package sdk --tag python-v0.1.0 --output-dir dist-python
|
||||
python scripts/build-python-release.py --package runtime --tag python-v0.1.0 --platform macos-arm64 --runtime-exe dist-exe/dsh-jsonrpc-agent-pkg-macos-arm64 --output-dir dist-python
|
||||
pip install --find-links dist-python deepseek-harness==0.1.0
|
||||
```
|
||||
|
||||
Once the binaries are in place (built or downloaded), installing the two wheels (or `pip install ./python/sdk-runtime ./python/sdk` straight from the directories) gives a working zero-config `DeepSeekHarness()`. Two pre-release caveats: the runtime wheel is currently tagged `py3-none-any` while embedding platform-specific binaries — build it with only the matching platform's exe in place (one wheel per platform), or drop all three exes in for a fat universal wheel (~500 MB); and versioning/publishing policy (`0.0.0-dev`, no registry) is deliberately unset until the first tagged release.
|
||||
The runtime distribution is wheel-only and rejects sdist builds, missing executables, and mixed-platform payloads. Its three wheel tags are `py3-none-manylinux_2_28_x86_64`, `py3-none-manylinux_2_28_aarch64`, and `py3-none-macosx_11_0_arm64`; the SDK remains `py3-none-any`. A tag pipeline builds these four non-conflicting files and publishes them together, so a normal `pip install deepseek-harness==X.Y.Z` selects the matching runtime wheel and `import deepseek_harness` needs no `runtime_bin`.
|
||||
|
||||
## Zero-config semantics
|
||||
|
||||
The runtime binary itself always requires an explicit config (`$DSH_CORDIS_CONFIG`, or a config path as the first argv argument), has no built-in fallback, and boots only what the config lists. Zero-config is SDK wrapper behavior: when the caller uses no explicit channel, the client injects the runtime package's checked-in default configuration ([runtime/cordis.yml](sdk-runtime/src/deepseek_harness_runtime/runtime/cordis.yml)) via `DSH_CORDIS_CONFIG`; any explicit channel wins and disables the injection. The full injection conditions live in the [sdk README](sdk/README.md); the default config's contents and the hard semantic in the [sdk-runtime README](sdk-runtime/README.md).
|
||||
|
||||
The executable is also a supported direct interface; keep stdin open for the NDJSON JSON-RPC exchange and supply a config explicitly:
|
||||
|
||||
```sh
|
||||
DSH_CORDIS_CONFIG=/absolute/path/cordis.yml ./dsh-jsonrpc-agent-pkg-macos-arm64
|
||||
```
|
||||
|
||||
## Test layout
|
||||
|
||||
`test_client.py` is fully keyless (a Python fake runtime is the peer). `test_bundled_runtime.py` boots each bundled carrier and skips per carrier when its artifact is missing. `test_runtime_resolution.py` covers the carrier-resolution rules without spawning anything.
|
||||
+11
-5
@@ -49,20 +49,26 @@ print(DeepSeekHarness().run("say hi").final_response) # auto-resolution picks
|
||||
|
||||
## 分发 Python 包
|
||||
|
||||
每个包各构建一个 wheel；runtime wheel 会内嵌构建时 `runtime/` 下存在的全部可执行文件（以及始终包含的默认 `cordis.yml`），并排除 `runtime/node/`：
|
||||
Python 发布只使用 `python-vX.Y.Z` 形式的稳定 tag。统一暂存脚本从 tag 推导两个分发物的版本，在 SDK 元数据中钉死 `deepseek-harness-runtime-bin==X.Y.Z`，并拒绝其他 tag 形式。纯 SDK wheel 只构建一次，runtime wheel 则在每个原生平台各构建一个：
|
||||
|
||||
```sh
|
||||
(cd python/sdk-runtime && uv build) # or: hatch build
|
||||
(cd python/sdk && uv build)
|
||||
pip install dist/deepseek_harness_runtime_bin-*.whl dist/deepseek_harness-*.whl
|
||||
python scripts/build-python-release.py --package sdk --tag python-v0.1.0 --output-dir dist-python
|
||||
python scripts/build-python-release.py --package runtime --tag python-v0.1.0 --platform macos-arm64 --runtime-exe dist-exe/dsh-jsonrpc-agent-pkg-macos-arm64 --output-dir dist-python
|
||||
pip install --find-links dist-python deepseek-harness==0.1.0
|
||||
```
|
||||
|
||||
二进制就位（构建或下载）后，安装这两个 wheel（或直接 `pip install ./python/sdk-runtime ./python/sdk` 从目录安装），就得到零配置可用的 `DeepSeekHarness()`。两个 pre-release 注意事项：runtime wheel 目前的标签是 `py3-none-any`，内容却是平台相关的二进制——构建时只放匹配平台的 exe（每平台一个 wheel），或把三个 exe 全部放入做成通用胖 wheel（约 500 MB）；版本与发布策略（`0.0.0-dev`、无 registry）刻意留待首个 tagged release 再定。
|
||||
runtime 分发物只提供 wheel，并拒绝 sdist 构建、缺失可执行文件以及混合平台载荷。三个 wheel tag 分别是 `py3-none-manylinux_2_28_x86_64`、`py3-none-manylinux_2_28_aarch64` 与 `py3-none-macosx_11_0_arm64`；SDK 保持 `py3-none-any`。tag 流水线统一构建并发布这 4 个互不冲突的文件，因此常规的 `pip install deepseek-harness==X.Y.Z` 会选中匹配平台的 runtime wheel，`import deepseek_harness` 不需要 `runtime_bin`。
|
||||
|
||||
## 零配置语义
|
||||
|
||||
运行时二进制本身始终要求显式配置（`$DSH_CORDIS_CONFIG`，或作为首个 argv 参数的配置路径），没有内置兜底，也只启动配置里列出的东西。零配置是 SDK 包装层的行为：调用方没有走任何显式通道时，客户端把 runtime 包检入的默认配置（[runtime/cordis.yml](sdk-runtime/src/deepseek_harness_runtime/runtime/cordis.yml)）注入 `DSH_CORDIS_CONFIG`；任一显式通道存在即胜出并禁用注入。注入条件的完整定义见 [sdk README](sdk/README.md)，默认配置的内容与硬语义见 [sdk-runtime README](sdk-runtime/README.md)。
|
||||
|
||||
可执行文件也支持直接调用；在 NDJSON JSON-RPC 交互期间保持 stdin 打开，并显式提供配置：
|
||||
|
||||
```sh
|
||||
DSH_CORDIS_CONFIG=/absolute/path/cordis.yml ./dsh-jsonrpc-agent-pkg-macos-arm64
|
||||
```
|
||||
|
||||
## 测试布局
|
||||
|
||||
`test_client.py` 完全无需 key（对端是 Python 假运行时）。`test_bundled_runtime.py` 逐个启动内置载体，某载体产物缺失时对应用例跳过。`test_runtime_resolution.py` 覆盖载体解析规则，不 spawn 任何进程。
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
README.md: 4cf48c55f40ec1c8632582546dcbfb990bbaa660
|
||||
README.zh.md: 0c82569be45b5cefa2ca025728945be1bdefc541
|
||||
README.md: ee7c8d5254bd1b1c1e9ba06c8c9ba0c6e8dfcacc
|
||||
README.zh.md: d3c44bf34c686e1aee7157b8b5d4d5b4d14bb053
|
||||
@@ -8,13 +8,15 @@ Runtime carrier package for the Python SDK (dist `deepseek-harness-runtime-bin`,
|
||||
|
||||
Two carriers coexist under `src/deepseek_harness_runtime/runtime/`, both injected by the repo's `scripts/build-exe-for-python-sdk.ts` build and both gitignored:
|
||||
|
||||
- **exe (production)** — single-file executables `dsh-jsonrpc-agent-pkg-<platform>-<arch>` (platform: `linux`/`macos`; arch: `x64`/`arm64`). No Node installation needed on the target machine. This is the only carrier that ships in wheel/sdist distributions.
|
||||
- **exe (production)** — single-file executables `dsh-jsonrpc-agent-pkg-<platform>-<arch>` (platform: `linux`/`macos`; arch: `x64`/`arm64`). No Node installation needed on the target machine. This is the only carrier that ships in wheel distributions; this package does not publish sdists.
|
||||
- **node (dev-only)** — the full deploy closure under `runtime/node/` (`package.json` + `node_modules/`), executed as `node runtime/node/node_modules/@deepseek-ai/dsh-jsonrpc-agent/lib/bin.js` on a system Node >= 22.19. It is the current checkout's source build, meant for repo-local development and verification only; it is never selected automatically and is excluded from distributions.
|
||||
|
||||
Both carriers hold the same content, defined once: the [package.json](package.json) at this package's root is the deploy root of the single-exe pipeline — a pure dependency manifest (no code of its own) whose dependency closure IS both the plugin set compiled into the exe and the tree materialized into `runtime/node/`. Adding a plugin to the distribution means adding one dependency line there and rebuilding.
|
||||
|
||||
Missing carriers raise `FileNotFoundError` naming the acquisition routes: build via `scripts/build-exe-for-python-sdk.ts` in a deepseek-harness checkout, or download the platform artifact of the `build-exe-for-python-sdk` CI workflow (a tar.gz — tar preserves the executable bit) and unpack it into the runtime directory. Acquisition strategy is deliberately separate from the lookup interface, so an on-demand download can replace it later without touching callers.
|
||||
|
||||
Each wheel contains exactly one executable. The fixed tags are `py3-none-manylinux_2_28_x86_64`, `py3-none-manylinux_2_28_aarch64`, and `py3-none-macosx_11_0_arm64`; the build hook rejects `py3-none-any`, absent or multiple executables, and unsupported platform tags. Stable `python-vX.Y.Z` releases use the same `X.Y.Z` for this package and the SDK.
|
||||
|
||||
## Resolution API
|
||||
|
||||
- `resolve_bundled_launch_args(mode=None) -> tuple[str, ...]` — the argv tuple that launches the bundled runtime: `(exe_path,)` in exe mode, `(node_path, bin_js_path)` in node mode. Mode selection: explicit argument > `DSH_RUNTIME_MODE` env var (`exe` | `node`) > automatic. Automatic resolution finds the production exe ONLY — the dev-only node carrier must be opted into explicitly so a production deployment can never silently ride on a source build.
|
||||
|
||||
@@ -8,13 +8,15 @@ Python SDK 的运行时载体包（dist 名 `deepseek-harness-runtime-bin`，模
|
||||
|
||||
两种载体并存于 `src/deepseek_harness_runtime/runtime/` 之下，均由仓库的 `scripts/build-exe-for-python-sdk.ts` 构建注入，且均被 gitignore：
|
||||
|
||||
- **exe（生产）**——单文件可执行程序 `dsh-jsonrpc-agent-pkg-<platform>-<arch>`（platform：`linux`/`macos`；arch：`x64`/`arm64`）。目标机器无需安装 Node。这是唯一进入 wheel/sdist 分发物的载体。
|
||||
- **exe（生产）**——单文件可执行程序 `dsh-jsonrpc-agent-pkg-<platform>-<arch>`（platform：`linux`/`macos`；arch：`x64`/`arm64`）。目标机器无需安装 Node。这是唯一进入 wheel 分发物的载体；本包不发布 sdist。
|
||||
- **node（仅限开发）**——`runtime/node/` 下的完整 deploy 闭包（`package.json` + `node_modules/`），在系统 Node >= 22.19 上以 `node runtime/node/node_modules/@deepseek-ai/dsh-jsonrpc-agent/lib/bin.js` 执行。它是当前检出的源码构建，仅用于仓库本地的开发与验证；不会被自动选中，也不进入分发物。
|
||||
|
||||
两种载体承载相同的内容，且只定义一次：本包根目录的 [package.json](package.json) 是 single-exe 流水线的 deploy root——一份零代码的纯依赖 manifest，其依赖闭包既是编译进 exe 的插件集，也是物化到 `runtime/node/` 的文件树。往分发物里加插件，就是在那里加一行依赖再重新构建。
|
||||
|
||||
载体缺失时抛出 `FileNotFoundError` 并写明获取途径：在 deepseek-harness 检出中经 `scripts/build-exe-for-python-sdk.ts` 构建，或下载 `build-exe-for-python-sdk` CI 工作流的对应平台产物（tar.gz——tar 保留可执行位）并解包到 runtime 目录。获取策略与查找接口刻意分离，之后可以换成按需下载而不动任何调用方。
|
||||
|
||||
每个 wheel 只包含一个可执行文件。固定 tag 为 `py3-none-manylinux_2_28_x86_64`、`py3-none-manylinux_2_28_aarch64` 与 `py3-none-macosx_11_0_arm64`；构建钩子会拒绝 `py3-none-any`、可执行文件缺失或重复以及不支持的平台 tag。稳定的 `python-vX.Y.Z` 发布为本包和 SDK 使用同一个 `X.Y.Z`。
|
||||
|
||||
## 解析 API
|
||||
|
||||
- `resolve_bundled_launch_args(mode=None) -> tuple[str, ...]`——启动内置运行时的 argv 元组：exe 模式下为 `(exe_path,)`，node 模式下为 `(node_path, bin_js_path)`。模式选择：显式参数 > `DSH_RUNTIME_MODE` 环境变量（`exe` | `node`）> 自动。自动解析只找生产 exe——仅限开发的 node 载体必须显式选用，从而生产部署绝不会悄悄跑在源码构建上。
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
README.md: ef9f8703f3bdb7b6d2dca28639f3b794083f7905
|
||||
README.zh.md: 31b6e87d7b75cc10cb799862fe6eb0f1f26dccd6
|
||||
README.md: 1916086324e18ce79ea572f3c85115fda3b4ab91
|
||||
README.zh.md: 380ed1726f2b541cd66403edb68a2629071e18f0
|
||||
@@ -8,6 +8,14 @@ runtime inherits normal DeepSeek Harness environment variables such as
|
||||
endpoints directly or point those variables at a local proxy during
|
||||
benchmark runs.
|
||||
|
||||
Installing `deepseek-harness` installs the exact same-version `deepseek-harness-runtime-bin` platform wheel. The normal entry point therefore needs no executable argument:
|
||||
|
||||
```py
|
||||
from deepseek_harness import DeepSeekHarness
|
||||
|
||||
result = DeepSeekHarness().run("Say hi.")
|
||||
```
|
||||
|
||||
By default, the SDK launches the bundled single-file `dsh-jsonrpc-agent` executable from the `deepseek-harness-runtime-bin` package and injects that package's default configuration (the stdio JSON-RPC server, agent core, preloaded DeepSeek adapter, JSONL session persistence, local bash) via `DSH_CORDIS_CONFIG`. To run a plugin composition of your own, keep the `@deepseek-ai/dsh-jsonrpc` entry in the config and pass the Cordis config path.
|
||||
|
||||
```py
|
||||
@@ -25,3 +33,5 @@ with DeepSeekHarness(
|
||||
event stream, including intermediate assistant messages and tool activity.
|
||||
|
||||
The same behavior can be selected for the runtime subprocess with `DSH_CORDIS_CONFIG`. The injection lives in `HarnessClient.start()`, so the low-level client's default launch gets it too: when the launch resolves to the bundled runtime and neither `cordis` nor a non-empty `DSH_CORDIS_CONFIG` is set (the runtime treats an empty value as absent, and so does the injection check), the bundled default configuration is used; an explicit `runtime_bin` or `launch_args_override` disables the injection entirely. See the [sdk-runtime README](../sdk-runtime/README.md) for the runtime carriers (production exe vs dev-only node closure) and how to obtain them.
|
||||
|
||||
`cwd` and `runtime_cwd` are resolved to absolute paths before subprocess launch, environment injection, and the wire handshake. The public API exposes only applied options: deployment persona and persistence belong in `cordis.yml`, while `session_root` remains the high-level convenience that sets `DSH_SESSION_ROOT`.
|
||||
@@ -4,6 +4,14 @@
|
||||
|
||||
通过 JSON-RPC stdio 驱动 DeepSeek Harness 的 Python 子进程 SDK。运行时继承常规的 DeepSeek Harness 环境变量（如 `DEEPSEEK_BASE_URL` 与 `DEEPSEEK_API_KEY`），调用方可以直接用真实模型端点，也可以在跑基准测试时把它们指向本地代理。
|
||||
|
||||
安装 `deepseek-harness` 会同时安装版本完全相同的 `deepseek-harness-runtime-bin` 平台 wheel。因此常规入口不需要传可执行文件参数：
|
||||
|
||||
```py
|
||||
from deepseek_harness import DeepSeekHarness
|
||||
|
||||
result = DeepSeekHarness().run("Say hi.")
|
||||
```
|
||||
|
||||
默认情况下，SDK 启动 `deepseek-harness-runtime-bin` 包内置的单文件 `dsh-jsonrpc-agent` 可执行程序，并通过 `DSH_CORDIS_CONFIG` 注入该包的默认配置（stdio JSON-RPC 服务器、agent core、预载的 DeepSeek 适配器、JSONL 会话持久化、本地 bash）。要运行自己用插件组合需要在配置里保留 `@deepseek-ai/dsh-jsonrpc` 条目，并传入 Cordis 配置路径。
|
||||
|
||||
```py
|
||||
@@ -19,3 +27,5 @@ with DeepSeekHarness(
|
||||
`TurnResult.final_response` 是本轮次最后一个 `assistant/message` 事件的文本内容。完整的事件流（包括中间的助手消息与工具活动）用 `TurnResult.events` 获取。
|
||||
|
||||
同样的行为也可以用 `DSH_CORDIS_CONFIG` 为运行时子进程选定。注入逻辑位于 `HarnessClient.start()`，因此低层客户端的默认启动同样享有它：当启动解析到内置运行时，且 `cordis` 与非空的 `DSH_CORDIS_CONFIG` 均未设置时（运行时把空值当作缺省，注入检查与之一致），使用内置的默认配置；显式给出 `runtime_bin` 或 `launch_args_override` 则完全禁用注入。运行时载体（生产用 exe 与仅限开发的 node 闭包）及其获取方式见 [sdk-runtime README](../sdk-runtime/README.md)。
|
||||
|
||||
`cwd` 与 `runtime_cwd` 会在启动子进程、注入环境变量和协议握手前解析为绝对路径。公开 API 只暴露真正生效的选项：部署 persona 与持久化配置归 `cordis.yml` 管理，而 `session_root` 继续作为设置 `DSH_SESSION_ROOT` 的高层便捷选项。
|
||||
Reference in New Issue
Block a user