diff --git a/.agents/notes/implemented/architecture/2026-07-23-client-plugin-loading-model.md b/.agents/notes/implemented/architecture/2026-07-23-client-plugin-loading-model.md index 58651fd258..9f8b697392 100644 --- a/.agents/notes/implemented/architecture/2026-07-23-client-plugin-loading-model.md +++ b/.agents/notes/implemented/architecture/2026-07-23-client-plugin-loading-model.md @@ -56,11 +56,11 @@ What happens between `dsh web` starting and the UI appearing? Three stages: the **Host side — compose the graph.** -1. The composing app (`apps/cli`) mounts the roster as in-memory Loader entries via `mountWebPlugins`. The roster is one flat list of the plugin packages, plus the `client-hmr` row under `--dev`. A roster package that fails to import throws loud at mount. -2. The registry (`createHostWebPluginRegistry`) scans the mounted entries' package.json `dshClient` declarations and composes `window.__DSH_BOOT__`: `{ rev, entries: [{ id, url, rev, inject?, immediately? }] }`. The `inject` edges and the `immediately` mark come from manifests, never hand-copied. It refuses a declared plugin without a built `./client` bundle, and any malformed declaration field — load-time fail loud. -3. The registry rescans on cordis `internal/plugin`, microtask-debounced; a rescan failure keeps serving the previous graph. Each bundle's content is hashed into its `rev` (cache busting + HMR diff anchor), and the row set into `graph.rev`. Every row is fetch-served: `/plugins//client.js?rev=…`. The graph types are a wire contract dual-held on both sides, because the webserver keeps zero workspace dependencies. +1. The composing app (`apps/cli`) ships the roster as ordinary rows in its `cordis.yml` config tree — client plugin packages are entry rows like every host plugin, and `--dev` appends the `client-hmr` row in code (`AppCLIEntry`) before the settle/sweep so the fail-loud triple covers it. A roster row that fails to import is caught by the boot's `assertEntriesLoaded`. +2. The `dsh-client-modules` node half (the package is dual-face: its browser half is the module table) scans loader entries' package.json `dshClient` declarations and composes `window.__DSH_BOOT__`: `{ rev, entries: [{ id, url, rev, inject?, immediately? }] }`. The `inject` edges and the `immediately` mark come from manifests, never hand-copied. It refuses a declared plugin without a built `./client` bundle, and any malformed declaration field — activation-time fail loud (a FAILED fiber the sweep reports). +3. Scanning is incremental per package — there is no full-rescan code path. Each cordis `internal/plugin` emission marks the fiber's entry name dirty (entry-less fibers drop O(1)); a microtask flush reconciles each dirty name against live loader entries, with package metadata (including the negative "not a client package" verdict) cached per name forever and bundle re-hashing reachable only through `rebuilt(id)`. The activation pass seeds the same dirty set from current entries and flushes synchronously, so first scan and steady state share one implementation. Each bundle's content hash is its `rev` (cache busting + HMR diff anchor), the row set hashes into `graph.rev`, and every row is fetch-served: `/plugins//client.js?rev=…`. The graph types are single-sourced in the modules package's `./impl` export — the webserver knows nothing about the graph (it is a plain route-registration plugin; modules registers the bundle route and taps the index render itself). -Why is the roster a hand-written list and not a scan? Because which plugins compose into a deployment is a composition decision, not a package property — a dshClient package existing in the repo does not mean this deployment mounts it, so discovery-by-scan cannot make that call. The roster lives in `apps/cli/web.ts` rather than cordis.yml only because `dsh web`'s host is a hand-assembled `bootHost` with no Loader config tree yet. +Why is the roster yml rows and not a scan? Because which plugins compose into a deployment is a composition decision, not a package property — a dshClient package existing in the repo does not mean this deployment mounts it, so discovery-by-scan cannot make that call; the node half scans only what the tree actually mounted. **Phase one — the module face.** The shell builds the module system over the graph, then prefetches every `immediately` row in parallel. Prefetch is fetch + execute, which registers factories only. A single row's prefetch failure is swallowed here: phase two's import retries the fetch and owns the loud failure, so one bad row cannot mask the others. `immediately` is a prefetch mark — not a barrier, not an identity. The package declares it, the registry carries it into the row. The infrastructure plugins (connection, runtime, ui-theme, i18n, plus hmr) declare it; UI plugins simply arrive on demand. @@ -74,9 +74,9 @@ Why is the roster a hand-written list and not a scan? Because which plugins comp ### Hot reload: one driver plugin, self-watched bundles -Whether hot reload is active is a composition decision: dev graphs include the `client-hmr` row (a normal plugin package) and turn on bundle watching; prod graphs do neither. +Whether hot reload is active is a composition decision: dev compositions mount the `client-hmr` row (a normal plugin package, appended by `--dev`) whose node half brings the bundle watch and the SSE channel; prod compositions mount nothing and have neither. -How does a rebuilt bundle become a reload signal? The webserver observes it itself — no builder tells it. The registry scan already holds every plugin's bundle path (`clientPath`), so in dev mode the registry stat-polls each scanned bundle file with `fs.watchFile`. Polling is by design: inotify does not fire on the weka network mount, the same reason the build-side watcher needs `--poll`. On a mtime/size change the registry re-hashes that row (`rebuilt(id)`); when the `rev` actually changed, it broadcasts a `rebuilt` frame on `GET /plugins/events` — a system SSE channel that sends the full graph on connect and `rebuilt` frames on change, presentation-only wire that never enters the session log. Watch set membership follows the table: rescans add watches for new rows and drop them for vanished ones, dispose drops all. The poll interval is a validated config field (default 500ms), not a constant. Rebuilding the bundles is any tsdown watch process's business — `scripts/dev-web.ts` remains as the watch-build entry point, its package list dshClient-discovered by scanning `packages/*/*/package.json` at startup — and builder and host share zero protocol. A torn read of a half-written bundle self-heals: the stats keep changing while the write completes, so the next poll tick re-hashes again and broadcasts the final rev. +How does a rebuilt bundle become a reload signal? The hmr node half observes it itself — no builder tells it. It reads the graph's bundle paths from `ctx.clientModuleHost.clientPath(id)` and stat-polls each with `fs.watchFile`, following graph membership through `onGraphChanged` (rows added late in the boot window get watches; vanished rows drop them; all lifecycles ride `ctx.effect`). Polling is by design: inotify does not fire on the weka network mount, the same reason the build-side watcher needs `--poll`. On a mtime/size change it calls `clientModuleHost.rebuilt(id)` — the single re-hash entry point — and when the `rev` actually changed, broadcasts a `rebuilt` frame on `GET /plugins/events` — a system SSE channel that sends the full graph on connect and `rebuilt` frames on change, presentation-only wire that never enters the session log. The poll interval is a validated config field (default 500ms), not a constant. Rebuilding the bundles is any tsdown watch process's business — `scripts/dev-web.ts` remains as the watch-build entry point, its package list dshClient-discovered by scanning `packages/*/*/package.json` at startup — and builder and host share zero protocol. A torn read of a half-written bundle self-heals: the stats keep changing while the write completes, so the next poll tick re-hashes again and broadcasts the final rev. On the browser side, the driver reloads one plugin per frame, serialized: @@ -116,7 +116,7 @@ One governance implementation runs on both sides of the wire; the browser-specif Costs accepted: the vendored Loader carries idle machinery in the browser (EntryTree persistence is a no-op, groups/isolation unused); every plugin edit in dev pays a bundle rebuild plus fiber remount; graph `inject` rows are informational — activation truth is service-level — so a mismatch surfaces at the settled sweep, not at graph validation; and the three not-yet-promoted libraries keep their static-import export surface until their DI conversions land. -Roster endgame: when `dsh web` moves to config-tree boot, the roster lands in cordis.yml — client plugin packages become ordinary config-tree entry rows, `mountWebPlugins` and the `CLIENT_PACKAGES` constant disappear, and recomposing a deployment means swapping the yml/overlay. The registry needs zero changes for that move, since its `internal/plugin` subscription already discovers whatever entries the tree mounts. +Roster endgame (landed 2026-07-25 with the config-tree boot move): the roster lives in `apps/cli/cordis.yml`, `mountWebPlugins` and the `CLIENT_PACKAGES` constant are gone, and recomposing a deployment means swapping the yml/overlay. The graph composer moved from a webserver-side registry into the `dsh-client-modules` node half (the package upgraded to dual-face per this note's promotion rule — its consumer now reaches it through cordis DI), and the transport split landed alongside: the webserver became a plain route-registration plugin, `/api/*` binding moved to the connection node half over the upgraded `api-gateway` plugin (`dsh-host-apiproxy` providing `ctx.apiProxy`), and the dev bundle watch + SSE channel moved to the hmr node half. ## Alternatives considered diff --git a/.agents/notes/implemented/architecture/2026-07-24-web-config-tree-boot-and-transport-layering.md b/.agents/notes/implemented/architecture/2026-07-24-web-config-tree-boot-and-transport-layering.md new file mode 100644 index 0000000000..9e93b828d5 --- /dev/null +++ b/.agents/notes/implemented/architecture/2026-07-24-web-config-tree-boot-and-transport-layering.md @@ -0,0 +1,42 @@ +# Agent Note: dsh web config-tree boot and the web transport layering + +Status: implemented + +English | [中文](2026-07-24-web-config-tree-boot-and-transport-layering.zh.md) + +> Scope: how `dsh web` composes (cordis.yml + pre-cordis boot classes + config sources) and how the web transport splits across packages (gateway / carrier / binding / graph / dev-reload). The [client plugin loading note](2026-07-23-client-plugin-loading-model.md) owns the browser-side loading chain this composition feeds. + +## Problem + +`dsh web` was the only hand-assembled surface left: `bootHost` mounted 32 plugins with configs pinned in code (violating no-hardcoded-tunables), the client roster was a `web.ts` constant, and TUI/headless had long been yml compositions. The transport layer misplaced responsibilities to match: the webserver self-described as a dumb carrier yet knew the `__DSH_BOOT__` graph, owned the SSE channel, and hard-coded the `/api/*` prefix; the dev bundle watch lived inside the prod registry behind a `watch?` flag with no lifecycle owner; the graph registry rescanned everything on every `internal/plugin` emission; per-request errors and fatal server errors shared one sink that always exited the process. One user-visible defect rode along: the web path never loaded `$DSH_HOME/.env`, so `DSH_HOME=… dsh web` could not find an API key living there. + +## Decision + +**Composition is one flat config tree.** `apps/cli/cordis.yml` holds every row — the host runtime (32 rows), the `api-gateway` row, the `webserver` row, and the ten `dshClient` rows (the browser roster; the modules row is simultaneously a host row). No spine bundle: every plugin is one row and every config field is yml-editable. `--dev` appends the `dsh-client-hmr` row in code before the settle sweep — prod and dev differ by exactly that row. Row order carries no load semantics; activation is service-availability driven, and the boot compensates with a fail-loud triple: `assertEntriesLoaded` (import failures), `installFailLoud` (late apply rejections), and an all-ACTIVE sweep (PENDING fibers — cordis inject waiting has no timeout). + +**Boot glue is a class pair.** `AppCLIEntry` (apps/cli) and `AppWebEntry` (the shell kernel) hold only what must exist independently of cordis: argv facts, the composed patch set, the parsed boot manifest, the module system instance, loading-page handles — everything else lives in plugins. `AppCLIEntry.run()` is three stages: layered env (ambient > cwd `.env` > `$DSH_HOME/.env`, closing the defect above) → patch composition → Loader include boot plus the triple. `AppWebEntry.run()` mirrors it browser-side: parse `window.__DSH_BOOT__` into a `BootManifest` (two views: npm-package rows for the module table, cordis-plugin rows for entry composition; malformed wire throws), build the module system, render the loading page, prefetch the `immediately` tier in parallel with Context/Loader setup, **await the prefetch before creating entries** (materialization is `tree.import`'s synchronous require, unprotected by fiber inject waiting; cross-package require edges such as i18n → runtime/client need every immediately-tier factory registered first — an empirically found 10–25% boot race otherwise), adopt the modules entry, create the graph rows, settle, sweep. + +**Config sources have one declaration place each.** yml static values are engineering defaults; the profile json (`./.dsh-tmp-profile/config.json`, read-only, never created, cwd-anchored until the `$DSH_HOME` migration) is user config mapped through a static `PROFILE_MAPPINGS` table onto target rows (`provider`/`model` → the `api-gateway` row, `persistenceRoot` → the jsonl row); CLI flags map onto the `webserver` row with a field set disjoint from the json's; env values enter through yml `!!js` expressions, never through the mapping table. Patches replace a row's config wholesale, so the entry class re-reads the yml row's static values (bypass parse) and merges overrides on top. An unmapped json key fails loud. The resolved frontend `distIndex` rides the same patch channel — an assembly fact, not user config. + +**The transport splits five ways.** `dsh-host-apiproxy` upgraded to the gateway plugin (`api-gateway` row): default-exports `ApiProxyService`, config `{provider, model}`, provides `ctx.apiProxy`, transport-agnostic and registers no routes — `createApiProxy` moved here from runtime (dependency direction allows it; runtime keeps `bootHost`/`startHost` for headless). `dsh-host-webserver` shrank to a plain route-registration plugin: `HttpServerService` provides `ctx.httpServer` (`register(route) → disposer` with duplicate-pattern throw, `tapIndex` transforms applied in registration order, `port`), listens on activation, per-request failures answer 400 and log without exiting, and knows no harness concepts. The connection node half owns the binding: it injects both services and registers `toFetchHandler(ctx.apiProxy)` under the `/api` prefix — future IPC carriers swap connection's transport while the gateway stays untouched. The modules node half (`ClientModuleHostService`, providing `ctx.clientModuleHost`) owns the graph: incremental per-package scanning (no full-rescan code path — `internal/plugin` marks the fiber's entry name dirty, a flush reconciles each name against live entries, package metadata including negative verdicts is cached forever, re-hashing is reachable only through `rebuilt(id)`), the bundle route, the index tap, and `onRebuilt`/`onGraphChanged` notification. The hmr node half owns dev reload: `fs.watchFile` stat-polling driven by `onGraphChanged` membership, and the `/plugins/events` SSE route. + +**Package export discipline.** The modules package exposes exactly `.` (node half) and `./client` (the complete browser half: `ClientModuleSystem`, `parseBootManifest`, the adoption plugin face) — no bespoke subpaths; wire types re-export through the root for host-side consumers. The adoption handshake: the kernel writes the constructed instance to `window.__DSH_MODULES__` before cordis exists; the `./client` apply reads the slot (missing = loud throw) and provides `ctx.modules`. + +## Consequences + +- Recomposing a web deployment is a yml/patch edit; the retired pieces (`mountWebPlugins`, `CLIENT_PACKAGES`, `createHostWebPluginRegistry`, `startWebServer`, the webserver's graph/SSE/api knowledge) are deleted. +- Headless still boots through `bootHost` (unchanged this round); its migration, the profile write path, the `$DSH_HOME` profile relocation, and IPC carriers are recorded deferrals in the design ledger. +- A TypeScript pitfall worth remembering: a `declare module 'cordis'` augmentation in a file with **no cordis import** is demoted to a standalone module declaration and silently shatters the program-wide `Context` merge (`ctx.on`/`ctx.effect` vanish across the program). Anchor with `import type {} from 'cordis'`. + +## Alternatives considered + +| Rejected | One-line reason | +|---|---| +| Dedicated `dsh-host-profile` receiver package | The profile json is consumed at patch time; the only runtime consumer of `{provider, model}` is the gateway itself — its config is the receiver | +| Runtime `assembly` shim plugin providing an `apiHandler` service | Existed only because `createApiProxy` lived in runtime; moving it into apiproxy made the gateway self-hosting, and `toFetchHandler` is a pure function the binding side calls | +| Full-rescan + incremental scan coexisting | Two implementations, two semantics; the single per-package path covers the activation pass too | +| A bespoke `./impl` export on the modules package | Non-uniform export surface; the standard `./client` carries the whole browser half | +| dev overlay / `cordis.dev.yml` | One yml; `!!js` cannot conditionalize row existence, and `--dev` appending one row is the entire difference | +| env vars in the mapping table | The same field would gain env/json double sourcing and need an invented precedence | +| Unbarriered create-after-prefetch (`arrive()` dedup as safety) | Disproved by a 10–25% boot race: in-flight dedup covers same-package double-fetch, not cross-package synchronous require edges | +| json file used directly as loader patches | json keys would couple to yml row structure; profile writers would need cordis knowledge | diff --git a/.agents/notes/implemented/architecture/2026-07-24-web-config-tree-boot-and-transport-layering.zh.md b/.agents/notes/implemented/architecture/2026-07-24-web-config-tree-boot-and-transport-layering.zh.md new file mode 100644 index 0000000000..a7671802df --- /dev/null +++ b/.agents/notes/implemented/architecture/2026-07-24-web-config-tree-boot-and-transport-layering.zh.md @@ -0,0 +1,42 @@ +# Agent Note:dsh web 的 config-tree boot 与 web 传输分层 + +Status: implemented + +[English](2026-07-24-web-config-tree-boot-and-transport-layering.md) | 中文 + +> 范围:`dsh web` 如何组合(cordis.yml + cordis 之前的 boot 类 + 配置源),以及 web 传输如何跨包分层(网关 / 载体 / 绑定 / 图 / 开发期重载)。浏览器侧装载链归 [client 插件装载 note](2026-07-23-client-plugin-loading-model.zh.md) 所有,本组合只是它的供给方。 + +## 问题 + +`dsh web` 曾是仅剩的手工装配面:`bootHost` 逐个挂 32 个插件、config 钉死在代码里(违反 no-hardcoded-tunables),client roster 是 `web.ts` 常量,而 TUI/headless 早已是 yml 组合。传输层的职责错位与之配套:webserver 自称哑载体却认识 `__DSH_BOOT__` 图、拥有 SSE 通道、硬编码 `/api/*` 前缀;dev 的 bundle watch 寄居在 prod registry 里靠 `watch?` 参数开关、生命周期无主;图 registry 对每次 `internal/plugin` 全量重扫;单请求失败与致命 server 错误共用一个一律退进程的 sink。还有一个用户可见缺陷:web 路径不装 `$DSH_HOME/.env`,`DSH_HOME=… dsh web` 读不到自定义 home 下的 API key。 + +## 决策 + +**组合是一棵平铺 config tree。** `apps/cli/cordis.yml` 持有全部行——host runtime(32 行)、`api-gateway` 行、`webserver` 行、十个 `dshClient` 行(浏览器 roster;modules 行同时是 host 行)。不做 spine bundle:每插件一行、每个 config 字段 yml 可改。`--dev` 在 settle sweep 之前由代码追加 `dsh-client-hmr` 行——prod 与 dev 的全部差异就是这一行。行序无装载语义;激活由服务可用性驱动,boot 以 fail-loud 三件套补偿:`assertEntriesLoaded`(import 失败)、`installFailLoud`(迟到的 apply 拒绝)、all-ACTIVE sweep(PENDING fiber——cordis inject 等待没有超时)。 + +**boot 胶水是一对 class。** `AppCLIEntry`(apps/cli)与 `AppWebEntry`(壳内核)只持有独立于 cordis 必须提前存在的东西:argv 事实、合成的 patch 集、解析出的 boot manifest、模块系统实例、loading 页句柄——其余一律进插件。`AppCLIEntry.run()` 三段:分层 env(ambient > cwd `.env` > `$DSH_HOME/.env`,顺手关掉上述缺陷)→ patch 合成 → Loader include boot 加三件套。`AppWebEntry.run()` 在浏览器侧镜像它:把 `window.__DSH_BOOT__` 解析成 `BootManifest`(双视角:npm 包行给模块表、cordis 插件行给 entry 组合;畸形 wire 大声抛)、建模块系统、渲染 loading 页、immediately 层预取与 Context/Loader 准备并行、**create entry 之前等预取齐**(物化是 `tree.import` 的同步 require,不受 fiber inject 等待保护;i18n → runtime/client 这类跨包 require 边要求 immediately 层工厂全部注册完——否则有实测 10–25% 的 boot 竞态)、收编 modules entry、逐图行 create、settle、sweep。 + +**每个配置源有唯一声明位置。** yml 静态值是工程默认;profile json(`./.dsh-tmp-profile/config.json`,只读、绝不创建、暂锚 cwd 直至 `$DSH_HOME` 迁移)是用户配置,经静态 `PROFILE_MAPPINGS` 表映射到目标行(`provider`/`model` → `api-gateway` 行,`persistenceRoot` → jsonl 行);CLI flags 映射到 `webserver` 行、字段集与 json 不相交;env 值经 yml `!!js` 表达式进入,绝不进映射表。patch 整体替换行 config,故 entry 类旁路 parse 重读 yml 行静态值再叠加覆盖。未映射的 json 键 fail loud。解析出的前端 `distIndex` 走同一 patch 通道——装配事实,不是用户配置。 + +**传输五分。** `dsh-host-apiproxy` 升格网关插件(`api-gateway` 行):默认导出 `ApiProxyService`,config `{provider, model}`,provide `ctx.apiProxy`,传输无关、不注册路由——`createApiProxy` 从 runtime 迁入(依赖方向允许;runtime 保留 `bootHost`/`startHost` 供 headless)。`dsh-host-webserver` 缩成朴素路由注册插件:`HttpServerService` provide `ctx.httpServer`(`register(route) → disposer`、重复 pattern 即抛、`tapIndex` 按注册序应用、`port`),激活即 listen,单请求失败答 400 并记日志不退进程,不认识任何 harness 概念。connection node 半拥有绑定:inject 两个服务,把 `toFetchHandler(ctx.apiProxy)` 注册在 `/api` 前缀下——将来 IPC 载体只换 connection 的传输,网关零改动。modules node 半(`ClientModuleHostService`,provide `ctx.clientModuleHost`)拥有图:单包增量扫描(无全量重扫路径——`internal/plugin` 把 fiber 的 entry 名标脏,flush 逐名对账 live entries,包元数据含否定结论永久缓存,重哈希唯一入口 `rebuilt(id)`)、bundle 路由、index tap、`onRebuilt`/`onGraphChanged` 通知。hmr node 半拥有开发期重载:`fs.watchFile` stat 轮询、watch 集合跟随 `onGraphChanged`、`/plugins/events` SSE 路由。 + +**包出口纪律。** modules 包只暴露 `.`(node 半)与 `./client`(完整浏览器半:`ClientModuleSystem`、`parseBootManifest`、收编插件面)——不设特设子路径;wire 类型经根出口 re-export 给 host 侧消费方。收编握手:内核在 cordis 之前把建好的实例写入 `window.__DSH_MODULES__`;`./client` 的 apply 读槽(缺槽大声抛)并 provide `ctx.modules`。 + +## 后果 + +- 重组一个 web 部署 = 改 yml/patch;退役件(`mountWebPlugins`、`CLIENT_PACKAGES`、`createHostWebPluginRegistry`、`startWebServer`、webserver 的图/SSE/api 知识)全部删除。 +- headless 本轮仍走 `bootHost`;它的迁移、profile 写入路径、profile 迁 `$DSH_HOME`、IPC 载体,均为设计台账中的挂账项。 +- 一个值得记住的 TypeScript 坑:`declare module 'cordis'` augmentation 所在文件若**没有任何 cordis import**,会被降级成独立 module declaration,无声打散全程序的 `Context` merge(`ctx.on`/`ctx.effect` 全程序消失)。用 `import type {} from 'cordis'` 锚定。 + +## Alternatives considered + +| 弃案 | 一行理由 | +|---|---| +| 专门的 `dsh-host-profile` 受体包 | profile json 在 patch 阶段消费完;`{provider, model}` 的唯一运行时消费方是网关自己——受体即网关 config | +| runtime 里的 `assembly` 垫层插件(provide `apiHandler`) | 它的存在只因 `createApiProxy` 住 runtime;本体迁入 apiproxy 后网关自持插件身份,且 `toFetchHandler` 是绑定方自己调的纯函数 | +| 全量重扫与增量扫描并存 | 两条实现两份语义;单包路径足以覆盖激活初扫 | +| modules 包特设 `./impl` 出口 | 出口面不统一;标准 `./client` 承载完整浏览器半 | +| dev overlay / `cordis.dev.yml` | 一套 yml;`!!js` 无法条件化行存在性,`--dev` 追加一行就是全部差异 | +| env 进映射表 | 同一字段将出现 env/json 双源,需再发明优先级 | +| create 不等预取(以 `arrive()` 去重为安全依据) | 被 10–25% boot 竞态证伪:在途去重只覆盖同包双拉,不覆盖跨包同步 require 边 | +| json 直接当 loader patches 文件 | json 键名将耦合 yml 行结构,写入方要懂 cordis |