Merge pull request #1054 from deepseek-harness/worktree-infdev01
Anchor storage-json root to $DSH_HOME/storages
This commit is contained in:
Turtle
GitHub
+2
-2
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/proposed/architecture/2026-07-28-storage-root-and-derived-medium-recovery.md
|
||||
2026-07-28-storage-root-and-derived-medium-recovery.md: 3937b17502d0bf640625e73822b758b1862b5391
|
||||
2026-07-28-storage-root-and-derived-medium-recovery.zh.md: ff2f196a911986d23b7a102e6336f353c6aef278
|
||||
2026-07-28-storage-root-and-derived-medium-recovery.md: 7759d0b6a8d522d6f853be0cd31a0b3909292446
|
||||
2026-07-28-storage-root-and-derived-medium-recovery.zh.md: 893b91c2ad3943cce9f64702e52afbb8075b907c
|
||||
+9
-7
@@ -8,7 +8,7 @@ English | [中文](2026-07-28-storage-root-and-derived-medium-recovery.zh.md)
|
||||
|
||||
The persisted projection cache ([RFC](2026-07-27-session-projection-and-command-log.md), shipped as `dsh-session-projection-cache`) surfaced two gaps in the storage substrate it landed on. Both are properties of the domain-KV stack ([design](2026-07-24-domain-kv-storage-and-workspace.md)), not of the cache itself, and both bite the cache first because it is the first *derived* medium on that stack.
|
||||
|
||||
**Where the files actually live.** The shipped Web overlay gives the json backend a relative root — `root: './.storages'` (`apps/cli/config/web.cordis.yml`) — while the shared base defaults the session store to the global harness home (`$DSH_HOME/sessions`, default `~/.dsh/sessions`); no equivalent global root exists for `storage-json`. `JsonStorageBackend` never resolves its root either — each unit open joins the still-relative path against whatever `process.cwd()` is at that moment (packages/storage/storage-json/src/index.ts) — the exact hazard the JSONL session backend resolves-once to prevent ("later process.cwd() changes cannot split one backend across roots", packages/session-persistence/session-persistence-jsonl/src/index.ts). Net effect: session logs are global across launch directories, but `workspace.json` and `session_projcache.json` land under `<launch dir>/.storages/`. Two launches from different directories share their sessions yet see different workspace registries and different projection caches — and the cache exists precisely to serve the cross-session cold listing, which now misses for every session last cached under another launch directory.
|
||||
**Where the files actually live (root mismatch closed; resolve-once residual still open).** The shared base defaults the session store to the global harness home (`$DSH_HOME/sessions`, default `~/.dsh/sessions`), while the shipped Web overlay used to give the json backend the relative root `./.storages`: `workspace.json` and `session_projcache.json` landed under `<launch dir>/.storages/` — two launches from different directories shared their sessions yet saw different workspace registries and different projection caches, and the cache exists precisely to serve the cross-session cold listing, which missed for every session last cached under another launch directory. That mismatch is now closed: the overlay anchors `storage-json.root` to `$DSH_HOME/storages` with the same `!!js` expression the session root uses (`apps/cli/config/web.cordis.yml`). The residual hazard: `JsonStorageBackend` still never resolves its root — each unit open joins the path against whatever `process.cwd()` is at that moment (packages/storage/storage-json/src/index.ts); the shipped overlay root is already absolute and unaffected, but any relative root (bare Loader boots, tests) still splits on a later cwd change — the exact hazard the JSONL session backend resolves-once to prevent ("later process.cwd() changes cannot split one backend across roots", packages/session-persistence/session-persistence-jsonl/src/index.ts).
|
||||
|
||||
**How recovery works today.** Inside a healthy medium the cache is fully self-healing by design: a `stateVersion`-mismatched row is discarded and refolded, a log shrunk below a row's watermark is detected by the anchored restore floor and answered with one full re-read, and every background write is fail-soft. But at the *medium* level there is no recovery at all: a truncated, hand-edited, or version-bumped `session_projcache.json` fails `openJsonUnit` with `malformed-medium`/`version-mismatch` (packages/storage/storage-json/src/format.ts), a schema-drifted record fails domain open with `invalid-record` (packages/storage/storage-domain/src/index.ts), the rejection propagates through `SessionProjectionCache[Service.init]`, and under the CLI's fail-loud boot the assembly refuses to start. A file whose entire content is rebuildable from session logs can brick boot. This contradicts the cache package's own stated stance ("a stale or unreadable cache costs a longer tail replay, never a wrong value") and the cache domain spec's JSDoc ("version bumps discard the whole medium"), which today describes an aspiration, not the implementation. The same fail-loud path is *correct* for `workspace.json` — workspace records are authoritative, not derivable — so the missing concept is a per-domain declaration of authority, not a global behavior change.
|
||||
|
||||
@@ -16,11 +16,11 @@ The persisted projection cache ([RFC](2026-07-27-session-projection-and-command-
|
||||
|
||||
Two independent changes, one per gap.
|
||||
|
||||
### One global storage root, resolved once
|
||||
### One global storage root (shipped, amended form); resolved once at construction (still open)
|
||||
|
||||
- `AppCLIEntry.composePatches` Source 0 additionally patches `storage-json.root` to `join(resolveDshHome(), 'storages')` — `~/.dsh/storages` by default, beside `~/.dsh/sessions` — and `PROFILE_MAPPINGS` gains `storageRoot` → (`storage-json`, `root`), mirroring `persistenceRoot` exactly. The yml keeps `./.storages` as the raw-composition engineering default (tests and bare Loader boots are unaffected), same layering as the session root today.
|
||||
- `JsonStorageBackend` resolves its configured root once at construction (`resolve(config.root)`), adopting the JSONL backend's recorded rationale verbatim: a later `process.cwd()` change must not split one backend across roots. The SQLite storage backend already resolves its path.
|
||||
- Pre-release stance applies: no migration shim. A deployment that cached under `<cwd>/.storages` re-derives everything (workspace re-bootstraps from the header index; the projection cache refolds lazily) or moves the two json files by hand once.
|
||||
- **Shipped**: the Web overlay anchors `storage-json.root` to `$DSH_HOME/storages` directly in the row, with the same `!!js` IIFE the session root uses (`~/.dsh/storages` by default, beside `~/.dsh/sessions`; no leading dot — the home is already a hidden tree). The user ruled this form in over this section's original launcher-patch + `storageRoot` profile key (see Alternatives); per-row overrides ride the personal `~/.dsh/config.yaml` patch layer. The verbatim duplication of the expression against the session root (`base.cordis.yml`) is a known cost — acceptable at two consumers; a third `$DSH_HOME`-derived root triggers extracting a single source (a launcher variable or a shared expression). The web e2e scaffold already patches the row to an absolute temp root, so tests never touch the user's home.
|
||||
- **Still open**: `JsonStorageBackend` resolves its configured root once at construction (`resolve(config.root)`), adopting the JSONL backend's recorded rationale verbatim: a later `process.cwd()` change must not split one backend across roots. The SQLite storage backend already resolves its path.
|
||||
- Pre-release stance applies (and was executed): no migration shim. A deployment that cached under `<cwd>/.storages` re-derives everything (workspace re-bootstraps from the header index; the projection cache refolds lazily) or moves the two json files by hand once.
|
||||
|
||||
### Declared derived media: reset instead of reject
|
||||
|
||||
@@ -31,7 +31,9 @@ Two independent changes, one per gap.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**Keep per-launch-directory `.storages` (status quo)** — rejected: sessions are global, so every derived-from-sessions medium splits against its own source of truth; the cache's motivating scenario (one listing over all sessions) structurally misses rows, and the workspace registry indexes sessions it cannot see from another launch directory.
|
||||
**Keep per-launch-directory `.storages` (the pre-change status quo)** — rejected: sessions are global, so every derived-from-sessions medium splits against its own source of truth; the cache's motivating scenario (one listing over all sessions) structurally misses rows, and the workspace registry indexes sessions it cannot see from another launch directory.
|
||||
|
||||
**Launcher patch + a `storageRoot` profile key (this section's original form)** — not taken: one `!!js` yml expression reaches the global root with the same layering the session root already has; a launcher patch adds a second rewrite point, and the profile key is an empty seat until a real consumer exists (per-row overrides already have the personal config.yaml patch layer).
|
||||
|
||||
**Patch only the projection cache's route to a global root, leave `workspace.json` per-cwd** — rejected: the workspace registry has the identical global-vs-cwd mismatch, and the user decision that shaped the cache placed it deliberately beside `workspace.json` — one hub root keeps the media co-located and the mental model single.
|
||||
|
||||
@@ -45,7 +47,7 @@ Two independent changes, one per gap.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- `dsh` launched from any directory reads and writes the same `$DSH_HOME/storages/*.json` (default `~/.dsh/storages`); the profile key `storageRoot` overrides it; a raw Loader boot of the yml still lands in `./.storages` relative to the boot cwd, resolved once at backend construction.
|
||||
- `dsh` launched from any directory reads and writes the same `$DSH_HOME/storages/*.json` (default `~/.dsh/storages`) — already satisfied by the overlay expression; per-row overrides ride the personal config.yaml patch layer; the backend resolves a relative root once at construction (still to do).
|
||||
- With a truncated, version-bumped, or schema-drifted `session_projcache.json`, the assembly boots clean: one warning names the discarded medium, the file is gone, the cache rebuilds through normal operation, and the cold listing column reappears as sessions are re-checkpointed.
|
||||
- The same damage to `workspace.json` still fails boot loudly.
|
||||
- Facility tests cover: each damage class resets a `'reset'` domain exactly once; non-damage failures stay loud on a `'reset'` domain; a `'reject'` domain propagates every failure; `destroy` removes the medium on both shipped backends.
|
||||
|
||||
+9
-7
@@ -8,7 +8,7 @@ Status: proposed
|
||||
|
||||
持久投影缓存（[RFC](2026-07-27-session-projection-and-command-log.md)，已作为 `dsh-session-projection-cache` 落地）暴露了它所依托的存储基座的两个缺口。二者都是 domain-KV 栈（[设计](2026-07-24-domain-kv-storage-and-workspace.md)）的属性而非缓存自身的问题，且都首先咬到缓存——因为它是这条栈上第一个*派生*介质。
|
||||
|
||||
**文件到底存在哪。** 出厂 Web overlay 给 json 后端的是相对根目录——`root: './.storages'`（`apps/cli/config/web.cordis.yml`）——而共享 base 将会话存储默认为全局 harness home（`$DSH_HOME/sessions`，默认 `~/.dsh/sessions`）；`storage-json` 没有对应的全局根目录。`JsonStorageBackend` 自己也从不 resolve 根——每次打开 unit 都把仍然相对的路径 join 到当时的 `process.cwd()` 上（packages/storage/storage-json/src/index.ts）——这正是 JSONL 会话后端用「构造时 resolve 一次」防住的那个隐患（"later process.cwd() changes cannot split one backend across roots"，packages/session-persistence/session-persistence-jsonl/src/index.ts）。净效果：会话日志跨启动目录全局共享，但 `workspace.json` 和 `session_projcache.json` 落在 `<启动目录>/.storages/` 下。从两个不同目录启动，会话相同，工作区注册表和投影缓存却各是一份——而缓存存在的意义恰恰是跨会话冷列表，如今凡是上次在别的启动目录下缓存过的会话全部 miss。
|
||||
**文件到底存在哪（根错位已收口，resolve-once 残余仍开放）。** 共享 base 将会话存储默认为全局 harness home（`$DSH_HOME/sessions`，默认 `~/.dsh/sessions`），而出厂 Web overlay 曾给 json 后端相对根 `./.storages`：`workspace.json` 和 `session_projcache.json` 落在 `<启动目录>/.storages/` 下——从两个不同目录启动，会话相同，工作区注册表和投影缓存却各是一份，而缓存存在的意义恰恰是跨会话冷列表，凡上次在别的启动目录下缓存过的会话全部 miss。这一错位已消除：overlay 现以与会话根同一段 `!!js` 表达式把 `storage-json.root` 锚定到 `$DSH_HOME/storages`（`apps/cli/config/web.cordis.yml`）。残余隐患：`JsonStorageBackend` 仍从不 resolve 根——每次打开 unit 都把路径 join 到当时的 `process.cwd()` 上（packages/storage/storage-json/src/index.ts）；出厂 overlay 的根已是绝对路径不受影响，但任何相对根（裸 Loader 启动、测试）仍会被后续 cwd 变化劈开，JSONL 会话后端用「构造时 resolve 一次」防住的正是它（"later process.cwd() changes cannot split one backend across roots"，packages/session-persistence/session-persistence-jsonl/src/index.ts）。
|
||||
|
||||
**现在是怎么恢复的。** 在健康介质内部，缓存按设计完全自愈：`stateVersion` 不匹配的行被丢弃重折，日志缩短到行水位以下由带锚的 restore floor 检出并以一次全量重读回答，每次后台写都是 fail-soft。但在*介质*层面完全没有恢复：被截断、被手改或版本被 bump 的 `session_projcache.json` 会让 `openJsonUnit` 以 `malformed-medium`/`version-mismatch` 失败（packages/storage/storage-json/src/format.ts），schema 漂移的记录让域 open 以 `invalid-record` 失败（packages/storage/storage-domain/src/index.ts），拒绝一路穿过 `SessionProjectionCache[Service.init]`，在 CLI 的 fail-loud 启动下整个组装拒绝启动。一个内容完全可从会话日志重建的文件能把启动搞死。这与缓存包自己声明的立场（"a stale or unreadable cache costs a longer tail replay, never a wrong value"）和缓存域 spec 的 JSDoc（"version bumps discard the whole medium"）相矛盾——后者今天描述的是愿望而非实现。同一条 fail-loud 路径对 `workspace.json` 却是*正确*的——工作区记录是权威数据，不可派生——所以缺的概念是按域声明权威性，而不是全局改行为。
|
||||
|
||||
@@ -16,11 +16,11 @@ Status: proposed
|
||||
|
||||
两个独立改动，一个缺口一个。
|
||||
|
||||
### 全局唯一存储根，构造时 resolve 一次
|
||||
### 全局唯一存储根（已落地，形态修正）；构造时 resolve 一次（仍开放）
|
||||
|
||||
- `AppCLIEntry.composePatches` 的 Source 0 追加把 `storage-json.root` patch 到 `join(resolveDshHome(), 'storages')`——默认 `~/.dsh/storages`，与 `~/.dsh/sessions` 并肩——并且 `PROFILE_MAPPINGS` 增加 `storageRoot` →（`storage-json`，`root`），与 `persistenceRoot` 完全镜像。yml 保留 `./.storages` 作为裸组合的工程默认（测试和裸 Loader 启动不受影响），分层方式与今天的会话根相同。
|
||||
- `JsonStorageBackend` 在构造时对配置根 `resolve` 一次，原样采纳 JSONL 后端已记录的理由：后续 `process.cwd()` 变化不得把一个后端劈到多个根下。SQLite 存储后端已经 resolve 其路径。
|
||||
- 适用 pre-release 立场：不做迁移垫片。曾在 `<cwd>/.storages` 下缓存过的部署要么全部重新派生（工作区从 header 索引重新 bootstrap；投影缓存惰性重折），要么手动把两个 json 文件挪一次。
|
||||
- **已落地**：出厂 Web overlay 直接在 `storage-json` 行内用与会话根同一段 `!!js` IIFE 把 `root` 锚定到 `$DSH_HOME/storages`（默认 `~/.dsh/storages`，与 `~/.dsh/sessions` 并肩；目录名不带点——home 本身已是隐藏树）。用户拍板采用此形态取代本节初版的「launcher patch + `storageRoot` profile 键」（见 Alternatives）；按行覆盖仍走个人 `~/.dsh/config.yaml` patch 层。该表达式与 sessions 根（`base.cordis.yml`）逐字重复是已知代价——两个消费者尚可接受；出现第三个 `$DSH_HOME` 派生根时提取单一来源（launcher 变量或共享表达式）。web e2e scaffold 本就把该行 patch 到临时绝对根，测试不触用户 home。
|
||||
- **仍开放**：`JsonStorageBackend` 在构造时对配置根 `resolve` 一次，原样采纳 JSONL 后端已记录的理由：后续 `process.cwd()` 变化不得把一个后端劈到多个根下。SQLite 存储后端已经 resolve 其路径。
|
||||
- 适用 pre-release 立场（已按此执行）：不做迁移垫片。曾在 `<cwd>/.storages` 下缓存过的部署要么全部重新派生（工作区从 header 索引重新 bootstrap；投影缓存惰性重折），要么手动把两个 json 文件挪一次。
|
||||
|
||||
### 声明派生介质：损坏时重置而非拒绝
|
||||
|
||||
@@ -31,7 +31,9 @@ Status: proposed
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**保持按启动目录的 `.storages`（现状）**——拒绝：会话是全局的，所以每个从会话派生的介质都与自己的真源劈叉；缓存的动机场景（一次列出全部会话）结构性丢行，工作区注册表索引着从另一个启动目录看不见的会话。
|
||||
**保持按启动目录的 `.storages`（改动前现状）**——拒绝：会话是全局的，所以每个从会话派生的介质都与自己的真源劈叉；缓存的动机场景（一次列出全部会话）结构性丢行，工作区注册表索引着从另一个启动目录看不见的会话。
|
||||
|
||||
**launcher patch + `storageRoot` profile 键（本节初版提案形态）**——未采：一行 yml `!!js` 表达式即达全局根，与会话根的既有分层完全一致；launcher patch 多引入一个改写点，profile 键在有真实消费者前是空席（按行覆盖已有个人 config.yaml patch 层可用）。
|
||||
|
||||
**只把投影缓存的 route 指到全局根，`workspace.json` 留在 per-cwd**——拒绝：工作区注册表有一模一样的全局 vs per-cwd 错位，而且塑造缓存的用户决策就是刻意把它放在 `workspace.json` 旁边——一个 hub 根让介质同址、心智模型单一。
|
||||
|
||||
@@ -45,7 +47,7 @@ Status: proposed
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- 从任意目录启动 `dsh` 都读写同一份 `$DSH_HOME/storages/*.json`（默认 `~/.dsh/storages`）；profile 键 `storageRoot` 可覆盖；裸 Loader 启动 yml 仍落在相对启动 cwd 的 `./.storages`,并在后端构造时 resolve 一次。
|
||||
- 从任意目录启动 `dsh` 都读写同一份 `$DSH_HOME/storages/*.json`（默认 `~/.dsh/storages`）——已由 overlay 表达式满足，按行覆盖走个人 config.yaml patch 层；后端对相对根在构造时 resolve 一次（待做）。
|
||||
- `session_projcache.json` 被截断、版本 bump 或 schema 漂移时，组装干净启动：一条警告命名被丢弃的介质，文件消失，缓存经正常运转重建，冷列表列随会话重新 checkpoint 逐步回归。
|
||||
- 同样的损坏发生在 `workspace.json` 上仍大声拒绝启动。
|
||||
- facility 测试覆盖：每个损坏类恰好重置一次 `'reset'` 域；非损坏失败在 `'reset'` 域上保持大声；`'reject'` 域传播一切失败；`destroy` 在两个出厂后端上都移除介质。
|
||||
|
||||
@@ -97,7 +97,8 @@
|
||||
- id: storage-json
|
||||
name: '@deepseek-ai/dsh-storage-json'
|
||||
config:
|
||||
root: './.storages'
|
||||
root: !!js >-
|
||||
(() => { const path = process.getBuiltinModule('node:path'); const home = process.getBuiltinModule('node:os').homedir(); const configured = process.env.DSH_HOME; const selected = configured !== undefined && configured.trim().length > 0 ? configured : path.join(home, '.dsh'); const expanded = selected === '~' ? home : selected.startsWith('~/') || selected.startsWith('~\\') ? path.join(home, selected.slice(2)) : selected; return path.join(path.resolve(expanded), 'storages') })()
|
||||
|
||||
- id: storage-domain
|
||||
name: '@deepseek-ai/dsh-storage-domain'
|
||||
|
||||
@@ -206,9 +206,9 @@ export async function launchWebScaffold(options: LaunchOptions = {}): Promise<We
|
||||
...surfacePatches,
|
||||
{ id: 'session-persistence-jsonl', config: { root: persistenceRoot } },
|
||||
{ id: 'session-query-sqlite', config: { path: ':memory:', openAt: 'first-search' } },
|
||||
// storage-json's './.storages' yml default is cwd-relative and resolves
|
||||
// per write; the scaffold restores the original cwd after boot, so the
|
||||
// row gets an absolute temp root (removed with the workspace at close).
|
||||
// storage-json's yml root is anchored to the real $DSH_HOME; pin the row
|
||||
// to an absolute temp root (removed with the workspace at close) so tests
|
||||
// never write the user's harness home.
|
||||
{ id: 'storage-json', config: { root: join(workspaceCwd, '.dsh-storages') } },
|
||||
// Skill discovery is model-visible input. Pin every host-level root inside
|
||||
// the owned temp world so ~/.dsh, ~/.agents, and a bundled-root env setting
|
||||
|
||||
Reference in New Issue
Block a user