diff --git a/docs/architecture.md b/docs/architecture.md index 796026bb10..1afa766f63 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -25,8 +25,8 @@ For a catalog of the **data structures** this architecture moves around — the │ @deepseek-ai/dsh-bash-local (bash impl) │ │ @deepseek-ai/dsh-tool-bash (bash tool schemas) │ │ @deepseek-ai/dsh-fs-local (filesystem impl) │ -│ @deepseek-ai/dsh-file-context (filesystem policy) │ -│ @deepseek-ai/dsh-tool-fs (filesystem tool schemas) │ +│ @deepseek-ai/dsh-file-context (filesystem policy gate) │ +│ @deepseek-ai/dsh-tool-fs (filesystem tools+executor)│ │ @deepseek-ai/dsh-session-persistence-jsonl (persistence impl)│ ├─────────────────────────────────────────────────────────────┤ │ @deepseek-ai/dsh-agent (vocabulary + registry) │ @@ -57,8 +57,7 @@ Dependency rule: **extension** plugins depend on interface packages, never on `d | `ctx.agents` | `AgentRegistry` | dsh-agent | live `Agent` handles + the create/resume factory seam (returns an `AgentHandle` = `{ agent, dispose() }` for owned per-agent teardown) | | `ctx.agentLoop` | `AgentLoop` | dsh-agent-loop | creates `ReactLoopAgent`s and drives their loops | | `ctx.bash` | `BashExecutor` (abstract) | dsh-bash | bash execution seam: foreground runs + background tasks | -| `ctx.fs` | `FileSystem` (abstract) | dsh-fs | filesystem provider seam: path resolution, stat, text read/stream, guarded writes/edits | -| `ctx.fileContext` | `FileContext` | dsh-file-context | filesystem policy: read windowing, observed-state, write/edit freshness over `ctx.fs` | +| `ctx.fs` | `FileSystem` (abstract) | dsh-fs | filesystem provider seam: path resolution, stat, text read/stream, atomic writes/edits (optional version guard); owns the `fs/*` policy events | All registrations (`registerAdapter`, `section`, `tools`, `register`, …) go through `ctx.effect()` and return disposers, so plugin hot-reload (vendored HMR) and fiber disposal clean up automatically. @@ -74,7 +73,7 @@ Swappable capabilities are split into **three packages** so each part evolves in The LLM seam has the same topology folded differently: `dsh-llm` carries the interface (`LlmAdapter`) AND the consumer surface (`ctx.llm.stream()`), with adapters as implementation packages — there the consumer is the loop itself, not a swappable schema surface. Use the full three-package split when the consumer is independently replaceable; keep interface + consumer together when they are one concern. Don't split preemptively: a capability with one conceivable implementation and one consumer stays one package until proven otherwise. -The filesystem capability follows the bash topology with a fourth layer: `dsh-fs` owns the abstract `ctx.fs` provider seam (text IO + guarded mutation primitives), `dsh-fs-local` provides the local backend, `dsh-file-context` is a concrete `ctx.fileContext` policy service (read windowing + observed-state + write/edit freshness, injecting `fs`), and `dsh-tool-fs` exposes the model-facing `read`/`write`/`edit` schemas over `ctx.fileContext`. The policy layer is a concrete service, not a second swappable seam — it owns the model-facing observation policy a sandboxed/remote backend has no business carrying. +The filesystem capability follows the bash topology with a fourth layer, but the policy is contributed through an **event gate**, not a method service: `dsh-fs` owns the abstract `ctx.fs` provider seam (text IO + atomic mutation primitives whose version guard is optional) and the `fs/*` policy event vocabulary, `dsh-fs-local` provides the local backend, `dsh-tool-fs` is the model-facing `read`/`write`/`edit` tools AND the executor (it reads/writes/edits through `ctx.fs` directly, owns read windowing, dispatches the `fs/*` events), and `dsh-file-context` is a policy PLUGIN (no service) that decides the `fs/write-expectation`/`fs/edit-expectation` waterfalls and records on `fs/observed` to add observed-state + read-before-edit + version-guarded write/edit. Because the tool is not method-coupled to the policy, dropping `dsh-file-context` gracefully loses the policy and leaves the unconstrained bare provider rather than breaking the tool at a service-injection boundary. The default product config loads `dsh-file-context`, so the default behavior remains read-before-write/edit. See [the file-context event-gate RFC](rfc/implemented/architecture/2026-06-26-file-context-as-event-gate.md). > **"Capability" — two unrelated meanings.** (1) The *seam pattern* above ("one plugin provides a capability, another needs it") is realized by plain Cordis **services + `inject`**: a provider registers a service (`ctx.bash`, declared in `interface Context`); a consumer declares `inject: ['bash']` and its fiber stays pending until the service exists, tearing down via HMR if it later vanishes. No extra library is needed. (2) `@cordisjs/plugin-capability` is a different axis entirely — a **permission/capability-security** service (named permissions with inheritance/dependency, tested against a session via `ctx.capability.test`). It is a candidate for the deferred permissions/sandbox work (the `tools/execute` veto seam), NOT a mechanism for swapping implementations. diff --git a/docs/cordis-catalog/events-and-services.md b/docs/cordis-catalog/events-and-services.md index 78f9f44324..b1114ce4c8 100644 --- a/docs/cordis-catalog/events-and-services.md +++ b/docs/cordis-catalog/events-and-services.md @@ -11,7 +11,7 @@ The **harness tier** below (the `@deepseek-ai/dsh-*` packages) is the vocabulary ## Events -Dispatch modes: **emit** (fire-and-forget), **waterfall** (each listener gets `next()` and may transform or veto — see [waterfall semantics](../architecture.md#cordis-waterfall-semantics-important)), **parallel** (awaited fan-out, no veto). The harness declares 22 events across 5 scopes. +Dispatch modes: **emit** (fire-and-forget), **waterfall** (each listener gets `next()` and may transform or veto — see [waterfall semantics](../architecture.md#cordis-waterfall-semantics-important)), **parallel** (awaited fan-out, no veto). The harness declares 25 events across 6 scopes. ### `agent/*` @@ -183,6 +183,44 @@ Types: [Agent](../core-data-structures/core.md) Source: [`packages/core/agent/src/types.ts:162`](../../packages/core/agent/src/types.ts) +### `fs/*` + +#### `fs/edit-expectation` — waterfall + +Single-slot decision: produce the optional version guard for the next FileSystem.editText. The tool dispatches this as an unbound waterfall and supplies a default thunk returning `undefined` (unconditional edit of the current content — the bare provider; no `stat`). The `@deepseek-ai/dsh-file-context` policy listener returns `{ version: vObserved }`, or throws `FS_NOT_OBSERVED` if the actor is unset or has not observed the target. Does NOT call `next()`: one decision, first-wins (see Events.'fs/write-expectation'). + +```ts cordis-catalog +'fs/edit-expectation'(target: FsTarget, actor: object | undefined, next: () => { version: FsVersion } | undefined | Promise<{ version: FsVersion } | undefined>): Promise<{ version: FsVersion } | undefined> +``` + +Types: [FsTarget](../core-data-structures/filesystem.md) · [FsVersion](../core-data-structures/filesystem.md) + +Source: [`packages/fs/fs/src/index.ts:117`](../../packages/fs/fs/src/index.ts) + +#### `fs/observed` — emit + +Record that an actor observed a target at a version, after a successful read/write/edit. Fire-and-forget. A listener MUST be a synchronous, side-effect-only recorder (`@deepseek-ai/dsh-file-context`'s is a `WeakMap.set`); the tool wraps the emit in a try/catch so a synchronous listener bug is logged and swallowed, never failing the already-completed mutation. cordis `emit` does not await listener promises, so this is not an async-error containment seam — async audit/telemetry does not belong here. No listener ⇒ nothing recorded. `actor` is the opaque tool-execution context. + +```ts cordis-catalog +'fs/observed'(target: FsTarget, version: FsVersion, actor: object | undefined): void +``` + +Types: [FsTarget](../core-data-structures/filesystem.md) · [FsVersion](../core-data-structures/filesystem.md) + +Source: [`packages/fs/fs/src/index.ts:129`](../../packages/fs/fs/src/index.ts) + +#### `fs/write-expectation` — waterfall + +Single-slot decision: produce the write expectation for the next FileSystem.writeText. The tool dispatches this as an unbound waterfall (no `this`) and supplies a default thunk returning `undefined` (unconditional create-or-overwrite — the bare provider). The `@deepseek-ai/dsh-file-context` policy listener returns `createIfAbsent` (unobserved actor) or `{ kind: 'replaceIfVersion', version: vObserved }` (observed) and does NOT call `next()` — one decision, not a composable chain. The slot is first-wins: the first non-`next()` decider (registration order, or `prepend`) occupies it; a second decider is a misconfiguration, not layering. `actor` is the opaque tool-execution context, never read here. + +```ts cordis-catalog +'fs/write-expectation'(target: FsTarget, actor: object | undefined, next: () => FsWriteExpectation | undefined | Promise): Promise +``` + +Types: [FsTarget](../core-data-structures/filesystem.md) · [FsWriteExpectation](../core-data-structures/filesystem.md) + +Source: [`packages/fs/fs/src/index.ts:105`](../../packages/fs/fs/src/index.ts) + ### `llm/*` #### `llm/stream` — waterfall @@ -279,7 +317,7 @@ Source: [`packages/core/tools/src/index.ts:43`](../../packages/core/tools/src/in ## Services -The 10 `ctx.` services the harness provides. An abstract seam (e.g. `ctx.bash`) is implemented by a separate package; the interface is what consumers code against. +The 9 `ctx.` services the harness provides. An abstract seam (e.g. `ctx.bash`) is implemented by a separate package; the interface is what consumers code against. ### `ctx.agentLoop` — `AgentLoop` @@ -339,22 +377,6 @@ Types: [BashExecRequest](../core-data-structures/bash.md) · [BashExecSpec](../c Source: [`packages/bash/bash/src/index.ts:59`](../../packages/bash/bash/src/index.ts) -### `ctx.fileContext` — `FileContext` - -The file-context policy service. Injects `fs`, registers as `ctx.fileContext`, and is the only read/write/edit path the model-facing tools use. - -```ts cordis-catalog -owner(exec?: FileContextExec): object | undefined -async resolve(path: string): Promise -async read(target: FsTarget, request: FileReadRequest, exec?: FileContextExec, signal?: AbortSignal): Promise -async write(target: FsTarget, content: string, exec?: FileContextExec, signal?: AbortSignal): Promise -async edit(target: FsTarget, edit: FsEditRequest, exec?: FileContextExec, signal?: AbortSignal): Promise -``` - -Types: [FileContextExec](../core-data-structures/filesystem.md) · [FileReadOutcome](../core-data-structures/filesystem.md) · [FileReadRequest](../core-data-structures/filesystem.md) · [FsEditOutcome](../core-data-structures/filesystem.md) · [FsEditRequest](../core-data-structures/filesystem.md) · [FsTarget](../core-data-structures/filesystem.md) · [FsWriteOutcome](../core-data-structures/filesystem.md) - -Source: [`packages/fs/file-context/src/index.ts:65`](../../packages/fs/file-context/src/index.ts) - ### `ctx.fs` — `FileSystem` (abstract seam) Abstract filesystem provider service. Subclass, implement the six text-storage primitives, and load the subclass as a plugin — it registers as `ctx.fs` (one implementation per context; loading a second throws, cordis' standard duplicate-service behavior). @@ -364,21 +386,21 @@ Semantics every backend must honor: - resolve returns a stable FsTarget; the same underlying file reached by different input paths must yield the same `targetKey` so stale guards and target lookup agree across paths (e.g. through symlinks). - stat returns FsInfo metadata (never content) or `undefined` when the target is absent. - readText/streamText read the whole regular text file (the stream for large files); both own regular-file checks, UTF-8 decoding, binary/NUL rejection, and `FS_NOT_TEXT`. -- writeText is atomic temp-file + rename honoring the FsWriteExpectation. -- editText verifies `expected.version` BEFORE literal matching (so a stale edit reports `FS_STALE_VERSION`, not `FS_EDIT_NOT_FOUND`/ `FS_AMBIGUOUS_EDIT` against newer content), then applies literal replacement and writes atomically — all inside one mutation critical section. +- writeText is atomic temp-file + rename. `expected` is OPTIONAL: omit it for an unconditional create-or-overwrite (the bare-provider default), or supply a FsWriteExpectation to guard the write. +- editText verifies `expected.version` BEFORE literal matching (so a stale edit reports `FS_STALE_VERSION`, not `FS_EDIT_NOT_FOUND`/ `FS_AMBIGUOUS_EDIT` against newer content), then applies literal replacement and writes atomically — all inside one mutation critical section. `expected` is OPTIONAL: omit it for an unconditional edit of the current content (a missing target still reports `FS_STALE_VERSION`). ```ts cordis-catalog abstract resolve(path: string): Promise abstract stat(target: FsTarget, signal?: AbortSignal): Promise abstract readText(target: FsTarget, signal?: AbortSignal): Promise abstract streamText(target: FsTarget, signal?: AbortSignal): Promise> -abstract writeText(target: FsTarget, content: string, expected: FsWriteExpectation, signal?: AbortSignal): Promise -abstract editText(target: FsTarget, edit: FsEditRequest, expected: { version: FsVersion }, signal?: AbortSignal): Promise +abstract writeText(target: FsTarget, content: string, expected?: FsWriteExpectation, signal?: AbortSignal): Promise +abstract editText(target: FsTarget, edit: FsEditRequest, expected?: { version: FsVersion }, signal?: AbortSignal): Promise ``` Types: [FsEditOutcome](../core-data-structures/filesystem.md) · [FsEditRequest](../core-data-structures/filesystem.md) · [FsInfo](../core-data-structures/filesystem.md) · [FsTarget](../core-data-structures/filesystem.md) · [FsVersion](../core-data-structures/filesystem.md) · [FsWriteExpectation](../core-data-structures/filesystem.md) · [FsWriteOutcome](../core-data-structures/filesystem.md) -Source: [`packages/fs/fs/src/index.ts:90`](../../packages/fs/fs/src/index.ts) +Source: [`packages/fs/fs/src/index.ts:158`](../../packages/fs/fs/src/index.ts) ### `ctx.llm` — `LlmService` diff --git a/docs/core-data-structures/filesystem.md b/docs/core-data-structures/filesystem.md index 99d073c6d4..e114c409d8 100644 --- a/docs/core-data-structures/filesystem.md +++ b/docs/core-data-structures/filesystem.md @@ -1,8 +1,10 @@ # Filesystem -The filesystem stack is split across four packages: a provider seam ([dsh-fs](../../packages/fs/fs), `ctx.fs`, text IO + guarded mutation), a local implementation ([dsh-fs-local](../../packages/fs/fs-local), local disk), a policy layer ([dsh-file-context](../../packages/fs/file-context), `ctx.fileContext`, read windowing + write/edit freshness), and a consumer ([dsh-tool-fs](../../packages/fs/tool-fs), the model-facing `read`/`write`/`edit` tools). Filesystem access is an optional capability, not part of the agent-loop spine, so its vocabulary lives here rather than in [core.md](core.md). A sandboxed, remote, virtual, or project-scoped backend can implement the same `FileSystem` service without changing the policy layer or the tool schemas. +The filesystem stack is split across four packages: a provider seam ([dsh-fs](../../packages/fs/fs), `ctx.fs`, text IO + atomic mutation primitives whose version guard is optional), a local implementation ([dsh-fs-local](../../packages/fs/fs-local), local disk), a policy plugin ([dsh-file-context](../../packages/fs/file-context), observed-state + read-before-edit + version-guarded write/edit, contributed through the `fs/*` event gate — NO service), and a consumer ([dsh-tool-fs](../../packages/fs/tool-fs), the model-facing `read`/`write`/`edit` tools, which is also the EXECUTOR — it reads/writes/edits through `ctx.fs` directly and owns read windowing). Filesystem access is an optional capability, not part of the agent-loop spine, so its vocabulary lives here rather than in [core.md](core.md). A sandboxed, remote, virtual, or project-scoped backend can implement the same `FileSystem` service without changing the policy plugin or the tool schemas. -Provider source: [`packages/fs/fs/src/types.ts`](../../packages/fs/fs/src/types.ts) and [`packages/fs/fs/src/index.ts`](../../packages/fs/fs/src/index.ts). Policy source: [`packages/fs/file-context/src/types.ts`](../../packages/fs/file-context/src/types.ts). +The model is **additive, not subtractive**: `ctx.fs` alone is a complete, unconstrained text-storage seam (`write` unconditionally creates-or-overwrites, `edit` unconditionally replaces literal text). `dsh-file-context` is a plugin that *adds* policy on top by deciding the `fs/*` waterfalls; removing it leaves the bare provider rather than breaking the tool, because the tool is not method-coupled to the policy. The default product config still loads it, so the default behavior remains read-before-write/edit. + +Provider source: [`packages/fs/fs/src/types.ts`](../../packages/fs/fs/src/types.ts) and [`packages/fs/fs/src/index.ts`](../../packages/fs/fs/src/index.ts). Policy source: [`packages/fs/file-context/src/types.ts`](../../packages/fs/file-context/src/types.ts). Read-rendering source: [`packages/fs/tool-fs/src/types.ts`](../../packages/fs/tool-fs/src/types.ts). ## Target identity and metadata (provider seam) @@ -16,7 +18,7 @@ interface FsTarget { } ``` -The backend owns file-version tokens — the freshness token a write/edit guards against. The policy layer stores them for stale checks; consumers do not interpret them. Both ids are branded opaque strings. +The backend owns file-version tokens — the freshness token a write/edit guards against. The policy plugin stores them for stale checks; consumers do not interpret them. Both ids are branded opaque strings. ```ts type-equiv type FsTargetKey = Branded<'FsTargetKey'> @@ -26,7 +28,7 @@ type FsTargetKey = Branded<'FsTargetKey'> type FsVersion = Branded<'FsVersion'> ``` -`stat` returns metadata (never content), or `undefined` when the target is absent. `type` lets the policy layer reject directories/special files before reading, and `size` lets it choose `readText` vs `streamText` without probing by failure. +`stat` returns metadata (never content), or `undefined` when the target is absent. `type` lets the tool reject directories/special files before reading, and `size` lets it choose `readText` vs `streamText` without probing by failure. ```ts type-equiv interface FsInfo { @@ -38,7 +40,7 @@ interface FsInfo { ## Write and edit guards (provider seam) -`writeText` takes an explicit write expectation rather than inferring intent. `createIfAbsent` creates a missing target and rejects an existing one with `FS_NOT_OBSERVED`; `replaceIfVersion` replaces only when the target exists at the observed version, else `FS_STALE_VERSION`. +Both `writeText` and `editText` take their version guard OPTIONALLY: omit it for an unconditional (bare-provider) mutation, supply it to guard. `writeText`'s guard is an `FsWriteExpectation` — `createIfAbsent` creates a missing target and rejects an existing one with `FS_NOT_OBSERVED`; `replaceIfVersion` replaces only when the target exists at the observed version, else `FS_STALE_VERSION`. Omitting `expected` unconditionally creates-or-overwrites. The union itself carries only the two guarded intents; "no guard" is expressed by omission, so write and edit share one symmetric `expected?` shape. ```ts type-equiv type FsWriteExpectation = @@ -53,7 +55,7 @@ interface FsWriteOutcome { } ``` -`editText` is a provider-level guarded mutation, not a `read` plus `write` composed in the policy layer. It verifies the expected version BEFORE literal matching (so a stale edit reports `FS_STALE_VERSION`, not a match failure against newer content), then applies the replacement and writes atomically — keeping matching, line-ending handling, stale checks, and atomic replacement inside one mutation critical section. +`editText` is a provider-level mutation, not a `read` plus `write` composed elsewhere. When guarded it verifies the expected version BEFORE literal matching (so a stale edit reports `FS_STALE_VERSION`, not a match failure against newer content); unguarded it edits the current content. Either way it applies the replacement and writes atomically — keeping matching, line-ending handling, the stale check, and atomic replacement inside one mutation critical section — and a missing target reports `FS_STALE_VERSION` on both paths. ```ts type-equiv interface FsEditRequest { @@ -71,9 +73,15 @@ interface FsEditOutcome { } ``` -## Execution context and read outcome (policy layer) +## The fs policy events (provider-seam vocabulary) -The policy layer needs just enough execution context to derive the observed-state owner. `ToolExecution` satisfies this shape, so `dsh-tool-fs` passes its execution object through without making `dsh-file-context` import the tool, agent, or session packages. +`dsh-fs` owns three events the tool dispatches and the policy plugin listens for, so the emitter (`dsh-tool-fs`) and the listener (`dsh-file-context`) share a vocabulary without the emitter depending on the policy plugin. They carry only `dsh-fs` vocabulary plus an opaque `object` actor — no model-facing concepts and no agent/session owner structure. + +`fs/write-expectation` and `fs/edit-expectation` are **single-slot decision waterfalls**: the tool dispatches each with a default thunk returning `undefined` (the bare provider), and a listener fully decides without calling `next()`. The slot is first-wins by registration order — the policy plugin owning it is a deployment convention, not an enforced invariant. `fs/observed` is a fire-and-forget recording event whose listener must be synchronous and side-effect-only; the tool contains a throw so a recording bug never fails the already-completed mutation. The generated catalog shows the exact signatures on [events-and-services.md](../cordis-catalog/events-and-services.md). + +## Execution context (policy plugin) + +The policy plugin needs just enough execution context to derive the observed-state owner by narrowing the opaque `object` actor the `fs/*` events carry. `ToolExecution` satisfies this shape, so `dsh-tool-fs` passes its execution object through as the actor without making `dsh-file-context` import the tool, agent, or session packages. ```ts type-equiv interface FileContextExec { @@ -83,14 +91,9 @@ interface FileContextExec { } ``` -A text read is bounded by line window, byte cap, and backend limits. The outcome the model-facing `read` tool renders carries the file's version at read time; there is no `full`/`partial` view — authorization is freshness-based, so any windowed read can authorize a later write/edit when the file is unchanged. +## Read outcome (consumer / read rendering) -```ts type-equiv -interface FileReadRequest { - offset: number - limit: number -} -``` +A text read is bounded by line window, byte cap, and backend limits. The outcome the model-facing `read` tool renders carries the file's version at read time; there is no `full`/`partial` view — authorization is freshness-based, so any windowed read can authorize a later write/edit when the file is unchanged. Read windowing and this outcome shape live in `dsh-tool-fs` (the executor that owns the read), not in the policy plugin. ```ts type-equiv interface FileReadOutcome { @@ -103,9 +106,9 @@ interface FileReadOutcome { } ``` -## Observed-file state (policy layer) +## Observed-file state (policy plugin) -Observed state is a `WeakMap>` inside `ctx.fileContext`. An entry exists **iff** the owner has read that target through `ctx.fileContext.read`, so its presence *is* the read record — there is no separate `hasRead` flag and no view distinction. The owner is normally `exec.agent.session`, but the policy layer treats it as opaque and never reads its fields. A successful read/write/edit refreshes the recorded version for that owner; disposal drops everything (HMR safety). +Observed state is a `WeakMap>` held inside the `dsh-file-context` plugin. An entry exists **iff** the owner has read, written, OR edited that target (every success emits `fs/observed`), so its presence is the prior-observation record — there is no separate `hasRead` flag and no view distinction. The owner is derived from the event actor (normally `exec.agent.session`), treated as opaque and never read. A successful read/write/edit refreshes the recorded version for that owner; disposal drops everything (HMR safety). ## Error taxonomy (provider seam) @@ -123,8 +126,8 @@ type FsErrorCode = | 'FS_ABORTED' ``` -`FS_NOT_OBSERVED` means no recorded read exists for this owner (or a `createIfAbsent` hit an existing file). `FS_STALE_VERSION` means the backend version no longer matches the observed one. Freshness authorization has no partial/full distinction, so there is no `FS_PARTIAL_OBSERVATION`. +`FS_NOT_OBSERVED` means the policy plugin has no prior-observation record for this owner (or a `createIfAbsent` hit an existing file). `FS_STALE_VERSION` means the backend version no longer matches the observed one (or an edit hit a missing target). Freshness authorization has no partial/full distinction, so there is no `FS_PARTIAL_OBSERVATION`. -## The services +## The service and the plugin -`FileSystem` (`ctx.fs`, abstract) owns the provider primitives: `resolve`, `stat`, `readText`, `streamText`, `writeText`, and `editText`. `FileContext` (`ctx.fileContext`, concrete) injects `fs` and owns the model-facing policy: `read` windows text and records observed state, `write`/`edit` derive the freshness expectation and refresh state. The generated wiring catalog shows the exact service signatures on [events-and-services.md](../cordis-catalog/events-and-services.md#ctxfs--filesystem-abstract-seam). +`FileSystem` (`ctx.fs`, abstract) owns the provider primitives: `resolve`, `stat`, `readText`, `streamText`, `writeText`, and `editText`. `dsh-file-context` registers **no service** — it is a plugin that adds policy through the `fs/*` event gate: it decides the write/edit expectation waterfalls (supplying `createIfAbsent`/`replaceIfVersion`/`{ version }` or throwing `FS_NOT_OBSERVED`) and records on `fs/observed`. The executor is `dsh-tool-fs`: it reads/writes/edits through `ctx.fs`, dispatches the waterfalls, and emits the recording event. The generated wiring catalog shows the exact `ctx.fs` signatures on [events-and-services.md](../cordis-catalog/events-and-services.md#ctxfs--filesystem-abstract-seam). diff --git a/docs/module-graph.md b/docs/module-graph.md index ad3d7f7b89..dfa1c3a3ed 100644 --- a/docs/module-graph.md +++ b/docs/module-graph.md @@ -53,7 +53,6 @@ graph TD tool-bash --> bash tool-bash --> llm tool-bash --> tools - tool-fs --> file-context tool-fs --> fs tool-fs --> llm tool-fs --> system-prompt @@ -100,7 +99,7 @@ graph TD | `acp` | `agent`, `llm`, `session`, `session-persistence`, `tools` | | `agent-loop` | `agent`, `llm`, `session`, `session-persistence`, `system-prompt`, `tools` | | `tool-bash` | `agent`, `bash`, `llm`, `tools` | -| `tool-fs` | `file-context`, `fs`, `llm`, `system-prompt`, `tools` | +| `tool-fs` | `fs`, `llm`, `system-prompt`, `tools` | | `agent-core` | `agent`, `agent-loop`, `invariants`, `llm`, `session`, `system-prompt`, `tool-bash`, `tools` | | `acp-agent` | `acp`, `agent-core`, `session-persistence-jsonl` | | `stdio-agent` | `agent`, `agent-core`, `session`, `session-persistence-jsonl`, `ui-stdio` | diff --git a/docs/rfc/README.md b/docs/rfc/README.md index cf9c343206..64f9060acf 100644 --- a/docs/rfc/README.md +++ b/docs/rfc/README.md @@ -118,6 +118,7 @@ Do NOT write one for a mechanical or local choice (a variable name, a one-file r | [Reorganize packages into a modular hierarchy](implemented/architecture/2026-06-20-package-hierarchy.md) | 2026-06-20 | | [Branded IDs everywhere they belong](implemented/architecture/2026-06-20-branded-ids.md) | 2026-06-20 | | [Extract example apps into packages](implemented/architecture/2026-06-20-extract-example-app-packages.md) | 2026-06-20 | +| [Make `dsh-file-context` an event-gate plugin, not a method interface](implemented/architecture/2026-06-26-file-context-as-event-gate.md) | 2026-06-26 | ### Process diff --git a/docs/rfc/implemented/architecture/2026-06-26-file-context-as-event-gate.md b/docs/rfc/implemented/architecture/2026-06-26-file-context-as-event-gate.md new file mode 100644 index 0000000000..efb6667d4e --- /dev/null +++ b/docs/rfc/implemented/architecture/2026-06-26-file-context-as-event-gate.md @@ -0,0 +1,175 @@ +# RFC: Make `dsh-file-context` an event-gate plugin, not a method interface + +Status: implemented + +## Problem + +[The split-fs-seam RFC](../simplification/2026-06-26-fsspec-style-fs-seam.md) put `ctx.fileContext` between the model-facing tools and the `ctx.fs` provider: `dsh-tool-fs` injects `fileContext` and routes every `read`/`write`/`edit` through its methods. That makes `fileContext` **in-path and mandatory**. The tool cannot reach `ctx.fs` without it, the policy layer owns the fs I/O and the read windowing, and a deployment that does not want observed-state policy cannot simply drop the package — `dsh-tool-fs` would fail to resolve `ctx.fileContext`. + +This couples three things that should be separable: + +1. **What the tool does** — resolve a path, read a window, write/edit a file. This is the tool's job and needs only `ctx.fs`. +2. **The freshness/observation policy** — "edit requires a prior read", "write/edit must be based on the version you read". This is the `dsh-file-context` plugin's job. +3. **The recording of observed state** — a side effect that should never block the tool from functioning. + +Because the tool calls `fileContext` methods, removing the policy layer is a breaking change rather than a graceful loss of an *add-on*. The policy is load-bearing for the tool to even run, not an opt-in tightening. + +## Decision + +Invert the control flow. **`dsh-tool-fs` becomes the executor and calls `ctx.fs` directly**; **`dsh-file-context` becomes a gate + recorder plugin** that participates through events, never through a method the tool calls and never by registering a `ctx.fileContext` service. + +```text +tool dsh-tool-fs executor: resolves, reads windows, writes/edits via ctx.fs; + emits fs policy events; renders results +policy dsh-file-context plugin: listens to fs/write-expectation + + fs/edit-expectation (single-slot waterfall) and fs/observed + (emit) events; adds observed-state + freshness. +provider seam dsh-fs ctx.fs: text IO + ATOMIC mutation primitives whose version + guard is OPTIONAL; owns the fs policy event vocabulary +provider dsh-fs-local local implementation of ctx.fs +``` + +The model is **additive, not subtractive**: `ctx.fs` on its own is a complete, unconstrained text-storage seam — `read` reads, `write` unconditionally creates-or-overwrites, `edit` unconditionally replaces literal text in the current content. There is no "先读后写", no version check, nothing to remove; the bare provider just does the I/O atomically. `dsh-file-context` is a plugin that *adds* constraints on top: observed-state, read-before-edit, and "write/edit must be based on the version you read". So removing `dsh-file-context` does not break `dsh-tool-fs` at the service-injection boundary; it removes the policy gate and leaves the bare provider behavior. The product default still loads `dsh-file-context`, so the default user-facing behavior and prompt discipline remain read-before-write/edit. The bare-provider mode exists because the tool should not be method-coupled to the policy plugin, not because an unconstrained filesystem is the normal product stance. + +`dsh-tool-fs` no longer injects `fileContext`. It injects `fs` and `tools`/`systemPrompt`. + +## The policy is enforced by provider CAS, not by `dsh-file-context` stat + +`dsh-file-context` enforces "you must write/edit based on the version you read" **without ever calling `stat` or comparing versions itself**. It supplies the observed version as the CAS basis and lets the provider's mutation critical section detect staleness: + +- "Have you read this file?" is the one thing `dsh-file-context` decides locally — a `WeakMap` lookup, no I/O. No record ⇒ `FS_NOT_OBSERVED`. +- "Is the version you read still current?" is decided **inside `ctx.fs.editText`/`writeText`**, in the same atomic lock that performs the read-match-rename. `dsh-file-context` passes `vObserved` as the expectation; the provider raises `FS_STALE_VERSION` if the file has moved on. + +This is deliberate. If `dsh-file-context` stat-ed and compared versions in its waterfall handler, there would be a TOCTOU gap between that check and the tool's actual write — the file could change in between, so the check would be a false guarantee that the provider's lock has to back up anyway. Putting the version check in the provider's critical section is both race-free and zero extra `stat`. So `dsh-file-context` does **no** filesystem I/O; the "must be based on the latest read" guarantee is *realized* by CAS, and `dsh-file-context` only chooses the basis (`vObserved`) and gates on prior observation. + +## Provider contract change: the version guard is optional + +For the bare provider to be unconstrained, the version guard on its two mutations becomes **optional** — present ⇒ guarded, absent ⇒ unconditional: + +```ts ignore-check +// writeText: expected is now optional. The FsWriteExpectation union is UNCHANGED. +writeText(target: FsTarget, content: string, expected?: FsWriteExpectation, signal?: AbortSignal): Promise +// undefined → unconditionally create-or-overwrite (bare default) +// createIfAbsent → create only, reject an existing file (dsh-file-context, unobserved) [unchanged] +// replaceIfVersion → overwrite only at the observed version, else FS_STALE_VERSION [unchanged] + +// editText: expected becomes optional (was the required { version: FsVersion }). +editText(target: FsTarget, edit: FsEditRequest, expected?: { version: FsVersion }, signal?: AbortSignal): Promise +// undefined → unconditionally replace literal text in the current content (bare default); +// a missing target still reports FS_STALE_VERSION +// { version } → edit only at that version, else FS_STALE_VERSION (the current behavior) +``` + +The `FsWriteExpectation` union itself does not change — the third "unconditional" state is expressed by *omitting* `expected`, so both mutations share one symmetric shape (`expected?`: omit = no guard, present = guarded). This keeps full backward compatibility for the guarded paths `dsh-file-context` uses; only the previously-impossible "no guard" case is new, and it is the bare-provider default. The mutation still runs inside the backend's per-target lock either way, so an unconditional write/edit is still atomic (no torn files); "unconditional" drops the *version* precondition, not the atomicity. `editText` reports a missing target as `FS_STALE_VERSION` on both guarded and unguarded paths, preserving one edit failure code for "the target cannot be edited at this moment". + +## Event vocabulary (owned by `dsh-fs`) + +The events live in `@deepseek-ai/dsh-fs`, not in `dsh-file-context`. This is forced by the decoupling contract: `dsh-tool-fs` is the emitter, so it must reference the event types, and it must keep compiling even though `dsh-file-context` no longer provides a method service. `dsh-fs` is the package both `dsh-tool-fs` and `dsh-file-context` already depend on, so it is the only home that lets the emitter and the policy listener share a vocabulary without the emitter depending on the policy plugin. + +These events carry existing `dsh-fs` vocabulary (`FsTarget`, `FsVersion`, `FsWriteExpectation`) plus an opaque actor — not model-facing concepts (no line windows, numbered lines, or rendered footers leak down). + +**The two `fs/*` decision events are single-slot decision points, NOT a composable interception chain.** A waterfall listener that does not call `next()` short-circuits the rest of the chain (verified in [vendor/cordis/src/events.ts](../../../../vendor/cordis/src/events.ts) — `waterfall` runs listeners around the final `next` thunk, and a listener that returns without calling `next()` reaches neither later listeners nor the tool's default thunk). `dsh-file-context` fully decides the write/edit expectation and does not call `next()`, so it occupies that one decision slot in the default deployment. This is deliberate: "what version basis does this mutation guard against" is a single decision, not an accumulation. The names (`fs/write-expectation`, `fs/edit-expectation`) say "produce the value", not "authorize", so they do not imply a stackable authorization chain. Genuinely composable interception (permission, audit, sandbox) belongs on the existing `tools/execute` waterfall, which every tool call already flows through — not on this fs version-decision slot. + +**The occupant is decided by registration order — first-registered (or `prepend`ed) wins.** cordis dispatches waterfall listeners in registration order (`push`, or `unshift` for `prepend` — [vendor/cordis/src/events.ts](../../../../vendor/cordis/src/events.ts)), and the first non-`next()` decider short-circuits the rest. So the slot is **first-wins**, and `dsh-file-context` owning it rests on the default deployment convention: it is the decider registered for these events. The event shape does NOT itself guarantee "an unread edit is rejected" — a plugin that registers a looser `fs/edit-expectation` decider BEFORE `dsh-file-context` (or with `prepend`) would decide first and bypass the `FS_NOT_OBSERVED` gate. That is the inherent property of a first-wins single slot, stated here so it is not mistaken for an enforced invariant. This RFC does not add a multi-policy composition mechanism; the implementation requirement is that the shipped `dsh-tool-fs` dispatches these waterfalls on every write/edit path and the shipped default config loads `dsh-file-context` as the policy decider. + +The actor is typed `object` in `dsh-fs` — a pure opaque carrier the provider seam never reads or narrows. The owner-derivation (`actor.agent?.session`) and the `{ agent?: { session? } }` structural shape stay entirely inside `dsh-file-context`, which narrows the `object` actor to that shape in its listeners. `dsh-fs` owns the event names and the fs vocabulary; it does NOT own the policy layer's runtime owner structure. + +```ts +import type { FsTarget, FsVersion, FsWriteExpectation } from '@deepseek-ai/dsh-fs' + +interface Events { + /** + * Single-slot decision: produce the write expectation for the next + * ctx.fs.writeText. The default returns undefined (unconditional create-or- + * overwrite — the bare provider). The policy listener returns createIfAbsent + * (unobserved) or { kind: 'replaceIfVersion', version: vObserved } (observed). + * The listener does NOT call next(): one decision, not a composable chain. @mode waterfall + */ + 'fs/write-expectation'(target: FsTarget, actor: object | undefined, next: () => FsWriteExpectation | undefined | Promise): Promise + /** + * Single-slot decision: produce the optional version guard for the next + * ctx.fs.editText. The default returns undefined (unconditional edit of the + * current content — the bare provider; no stat). The policy listener returns + * { version: vObserved }, or throws FS_NOT_OBSERVED if the actor is unset or + * has not observed the target. Does NOT call next(): one decision. @mode waterfall + */ + 'fs/edit-expectation'(target: FsTarget, actor: object | undefined, next: () => { version: FsVersion } | undefined | Promise<{ version: FsVersion } | undefined>): Promise<{ version: FsVersion } | undefined> + /** + * Record that an actor observed a target at a version, after a successful + * read/write/edit. Fire-and-forget. Listeners MUST be synchronous, side-effect- + * only recorders (`dsh-file-context`'s is a WeakMap write); the tool wraps the + * emit in a try/catch so a synchronous listener bug is logged and swallowed, + * never failing the already-completed mutation. No listener ⇒ nothing recorded. + * @mode emit + */ + 'fs/observed'(target: FsTarget, version: FsVersion, actor: object | undefined): void +} +``` + +The `fs/*` decision events are **unbound waterfalls dispatched by the tool** (like `agent/request`, which the loop dispatches with no `this`), not service-bound waterfalls (like `llm/stream`). The dispatcher is the `dsh-tool-fs` plugin, which is not a service. + +## Tool contract (`dsh-tool-fs`) + +The tool keeps its model-facing schemas (`read`/`write`/`edit`, byte-for-byte unchanged) and prompt sections. The prompt guidance stays policy-first because the default product config loads `dsh-file-context`: the model is still told to read before overwriting or editing, and any wording that says the "backend" requires that should be corrected to say the default file-context policy requires it. The bare-provider fallback does not change the default prompt stance. + +`dsh-tool-fs` gains the executor responsibilities relocated from the old `fileContext` method service, including **read windowing** (`window.ts`, `READ_MAX_BYTES`, `READ_MAX_LINE_LENGTH`, `FileReadRequest`/`FileReadOutcome`/`FileTextLine`, `STREAM_MIN_SIZE`), which is the tool's rendering detail now that the tool owns the read. Those read-windowing types and helpers move into `dsh-tool-fs`; the policy plugin must not remain a type dependency for the tool. + +`dsh-tool-fs` exposes each tool as a first-class **subpath plugin** (`/read`, `/write`, `/edit`) for focused deployments, plus a root plugin that composes all three. The `inject` change applies to **all four**: each of `read.ts`, `write.ts`, `edit.ts`, and `index.ts` drops `fileContext` from `inject` and adds `fs` (keeping `tools`/`systemPrompt`). Updating only the root plugin would leave a focused deployment that loads just `@deepseek-ai/dsh-tool-fs/edit` still coupled to the old method service, silently breaking the decoupling contract for exactly the deployments subpaths exist to serve. + +`stat` budget is minimized by letting the waterfall produce the expectation lazily — the bare default returns `undefined` (no guard) and never stats: + +- **read** — one `stat` (type + size routing + version), then `readText`/`streamText`, then `buildWindow`, then a contained `emit('fs/observed', target, info.version, exec)`. The post-read confirming `stat` from the old `fileContext.read` is dropped; a writer racing between the routing stat and the read can at worst make a *later* guarded edit spuriously `FS_STALE_VERSION` (fail-closed: the model re-reads, never writes against the wrong version, since `editText` re-checks in its lock). +- **write** — `expectation = await ctx.waterfall('fs/write-expectation', target, exec, () => undefined)`, then `ctx.fs.writeText(target, content, expectation)`, then a contained `emit('fs/observed', target, outcome.version, exec)`. **Zero stat in the tool** with or without `dsh-file-context`. +- **edit** — `expectation = await ctx.waterfall('fs/edit-expectation', target, exec, () => undefined)`, then `ctx.fs.editText(target, edit, expectation)`, then a contained `emit('fs/observed', target, outcome.version, exec)`. **Zero stat in the tool** in both cases: the bare default is `undefined` (unconditional edit), so the tool never stats to manufacture a basis. If the target is absent, the provider reports `FS_STALE_VERSION` even on the unguarded path. + +The tool passes `exec` (the tool-execution context) as the `actor` argument on every dispatch, so `dsh-file-context` can derive its observed-state owner. The tool does not know whether the policy plugin is present: it always provides the bare default behavior in the `next` thunk, and `dsh-file-context` short-circuits the thunk before it runs in the default deployment. + +**`fs/observed` recording must never fail the tool, because it fires AFTER the mutation already succeeded** — a throw there becomes an `isError` result ([tools/index.ts](../../../../packages/core/tools/src/index.ts) — `ToolRegistry.execute` catches a tool throw into an error result), reporting failure for a write/edit that actually happened. The tool therefore wraps the dispatch in a try/catch that logs and swallows synchronous listener bugs (the established fire-and-forget pattern in [agent.ts](../../../../packages/core/agent-loop/src/agent.ts)). The event contract is intentionally narrower than "arbitrary observers": an `fs/observed` listener MUST be synchronous and side-effect-only — `dsh-file-context`'s listener is a `WeakMap.set`, which cannot throw under normal operation and returns no promise. Cordis `emit` does not await listener promises, so the try/catch is NOT an async-error containment mechanism; async audit/telemetry/listener work does not belong on this event. If layered or async observation is ever wanted, that is a new event with its own dispatch story. + +## Policy plugin contract (`dsh-file-context`) + +`dsh-file-context` is a plugin, not a service. It does not register `ctx.fileContext`, has no public method surface, and exposes no `read`/`write`/`edit`/`resolve` methods. It attaches three listeners via `ctx.on()` registrations (each returning a disposer for HMR). It keeps the observed-state `WeakMap>` and the structural owner derivation (narrowing the event's opaque `object` actor to its own `{ agent?: { session? } }` shape), but does not inject `fs` — every handler operates only on its own `WeakMap`, never on `ctx.fs`. + +- `fs/write-expectation` listener: `prior = getObserved(owner, key)`; return `prior ? { kind: 'replaceIfVersion', version: prior.version } : { kind: 'createIfAbsent' }`. It does NOT call `next()`: it fully owns the single decision slot. +- `fs/edit-expectation` listener: `prior = getObserved(owner, key)`; if no `owner` or no `prior`, throw `FS_NOT_OBSERVED`; else return `{ version: prior.version }`. Also does not call `next()`. +- `fs/observed` listener: `record(owner, key, version)`. + +An observed-state entry is the **prior-observation record**: a successful `read`, `write`, OR `edit` all emit `fs/observed` and record `{ version }`, so the entry's presence means "this owner has observed this target at this version", not narrowly "has read it". This is what lets a create-then-edit or edit-then-edit sequence work without an intervening re-read: the mutation refreshes the recorded version to its own result, so the next edit's basis is the version it just produced. `FS_NOT_OBSERVED` rejects only an edit with NO prior observation of any kind. The owner is derived structurally from `{ agent?: { session? } }`; disposal drops all state (HMR safety). + +`dsh-file-context` is now a pure policy/recording plugin with no service surface — it influences the world only through the event seam. That is what removes the method coupling from `dsh-tool-fs`. + +## Bare-provider behavior (no `dsh-file-context`) + +This is not the default product mode — the default product config loads `dsh-file-context`. It is the unconstrained provider floor that exists once the tool is no longer coupled to a policy method service. With `dsh-file-context` absent, every `fs/*` waterfall falls through to its `undefined` default and `fs/observed` has no listener: + +- **read** is identical (it never needed policy; it only emits a now-unheard `fs/observed`). +- **write** unconditionally creates-or-overwrites: `expected` is `undefined`, so `writeText` writes whether or not the file exists and whatever its current version. No read-first requirement, no version check. +- **edit** unconditionally replaces literal text in the file's current content: `expected` is `undefined`, so `editText` matches and rewrites without a version guard or a read-first requirement (`FS_EDIT_NOT_FOUND`/`FS_AMBIGUOUS_EDIT` still apply — those are about the literal match, not freshness). A missing target still reports `FS_STALE_VERSION`, matching the guarded edit path's "cannot edit this target now" code. + +Both mutations are still atomic (the backend's per-target lock is unconditional). What is simply *absent*, not lost, is the policy `dsh-file-context` would add: observed-state, read-before-edit, and version-guarded write/edit. Loading `dsh-file-context` layers those constraints on by having its listeners return guarded `expected` values instead of `undefined`; nothing in the bare provider changes. + +## Supersedes + +This amends — does not reverse — [the split-fs-seam RFC](../simplification/2026-06-26-fsspec-style-fs-seam.md). The four-layer split, the provider contract, and the freshness *policy* are all kept. What changes is the **coupling between the tool and the policy layer**: a mandatory method service became a plugin-owned event gate, and the fs I/O + read windowing moved from `fileContext` up into `dsh-tool-fs`. The split-fs-seam RFC's description of `dsh-tool-fs` injecting `fileContext` and of `fileContext` owning `read`/`write`/`edit` was updated to match in the same change. + +## Acceptance Criteria + +- All four `dsh-tool-fs` injection points — the root plugin AND the `/read`, `/write`, `/edit` subpath plugins — inject `fs` (+ `tools`/`systemPrompt`), not `fileContext`; each calls `ctx.fs` directly and dispatches the `fs/write-expectation`/`fs/edit-expectation` waterfalls (passing `exec` as the actor) and the contained `fs/observed` emit. Read windowing lives in `dsh-tool-fs`. +- `dsh-fs` declares the three events with `@mode` tags and an opaque `object` actor argument (no agent/session structure leaks into the provider vocabulary); the generated cordis catalog is regenerated. +- `dsh-file-context` is a plugin, not a service: it does not register `ctx.fileContext`, has no public `read`/`write`/`edit`/`resolve` methods, and does not inject `fs`; it registers the three listeners, keeps observed-state, and has HMR/disposal coverage (dispose the fiber, assert the gate no longer rewrites). +- **Bare-provider test**: a config WITHOUT `dsh-file-context` that loads a **subpath plugin** (e.g. just `@deepseek-ai/dsh-tool-fs/edit`, plus `/read`/`/write` as the scenario needs) boots, and `read`/`write`(create AND overwrite)/`edit` work through `dsh-tool-fs` against the real `dsh-fs-local`; an `edit` of an unread existing file and an overwrite of an existing unread file both succeed (unconditional bare-provider behavior), proving the subpath plugins — not just the root — carry no `fileContext` dependency. A bare-provider edit of a missing target reports `FS_STALE_VERSION`. With `dsh-file-context` present, the same unread `edit` is rejected `FS_NOT_OBSERVED` and the same unread overwrite uses `createIfAbsent` (rejected on an existing file). +- **Single-slot semantics**: a test registers a second `fs/edit-expectation` listener AFTER `dsh-file-context` and asserts it is NOT reached (first-wins short-circuit), and documents in a comment that a decider registered before/`prepend`ed would instead win — the slot is first-wins by convention, not an enforced invariant. +- **Contained observed recording**: a test with a synchronously throwing `fs/observed` listener performs a write/edit and asserts the tool result is still success (the completed mutation is not turned into an `isError`). The event contract requires synchronous side-effect-only listeners; the try/catch is the synchronous backstop, not async rejection handling. +- `dsh-fs` `writeText`/`editText` make `expected` optional (omit ⇒ unconditional); the `FsWriteExpectation` union is unchanged, and `dsh-file-context`'s guarded paths (`createIfAbsent`/`replaceIfVersion`/`{ version }`) behave exactly as today. A bare-provider test exercises an unconditional overwrite, an unconditional edit, and a missing-target edit reporting `FS_STALE_VERSION`. +- Freshness is enforced by provider CAS when guarded: an edit after a stale read reports `FS_STALE_VERSION` (regression test); `dsh-file-context` performs no `stat`. +- `stat` budget: read = 1, write = 0, edit = 0 — in the tool, with or without `dsh-file-context` (the bare default returns `undefined`, never stats). A test asserts neither write nor edit stats in the tool on either path. +- Model-facing schemas stay byte-for-byte unchanged; snapshot transcript goldens are unaffected (or the diff is reviewed and re-recorded with justification). +- Docs/artifacts updated in the same change: `docs/architecture.md`, fs package READMEs, `docs/core-data-structures/filesystem.md`, the split-fs-seam RFC's now-amended description, type-equiv blocks + manifest, cordis catalog, module graph. Gates green: `doc-sync`, `knip`, `test:coverage` (100% per-file). + +## Risks + +- **Event indirection over a method call.** A waterfall + emit is less direct than `await ctx.fileContext.edit(...)`. The payoff is removing the tool-to-policy method dependency while keeping the default policy plugin; the cost is one more event vocabulary to learn. Mitigated by keeping the three events narrow and documenting the default-thunk semantics on each. +- **Policy events in the storage seam.** `dsh-fs` gains two version-decision events plus a recording event though it is "just storage". This is the price of decoupling (the emitter cannot depend on the policy plugin). The events carry only `dsh-fs` vocabulary plus an opaque `object` actor and no model-facing concepts, so the seam stays free of line-window/observation policy types and of the agent/session owner structure. +- **Single policy occupant, first-wins by convention.** The `fs/write-expectation`/`fs/edit-expectation` slots hold exactly one decider; the first-registered (or `prepend`ed) listener wins and the rest are short-circuited. `dsh-file-context` owning the slot is a deployment convention, not an event-enforced invariant — a second decider registered first would bypass it. This is acceptable because a second fs-version-policy decider is a misconfiguration, not a feature. If a future need for *layered* fs version policy appears, it is a new RFC (a composable value-passing seam), not a silent second listener on these events. Layered permission/audit/sandbox interception already has its home on `tools/execute`. +- **Dropping the post-read confirming stat** makes a follow-up *guarded* edit occasionally fail-closed (`FS_STALE_VERSION` → re-read) under a read/write race. This is a UX nicety lost, never a correctness hole; the provider lock still prevents wrong-version writes. +- **The bare provider does no read-before-write/edit and no version check.** A deployment without `dsh-file-context` lets the model overwrite or edit any existing file unconditionally. This is the deliberate meaning of keeping the tool independent of a policy service: the safety disciplines live in the default `dsh-file-context` plugin. A deployment that omits it is opting into an unconstrained filesystem on purpose; that is not the default product stance. diff --git a/docs/rfc/implemented/simplification/2026-06-26-fsspec-style-fs-seam.md b/docs/rfc/implemented/simplification/2026-06-26-fsspec-style-fs-seam.md index 25ff66816f..7f2e5d7091 100644 --- a/docs/rfc/implemented/simplification/2026-06-26-fsspec-style-fs-seam.md +++ b/docs/rfc/implemented/simplification/2026-06-26-fsspec-style-fs-seam.md @@ -20,13 +20,15 @@ The old RFC already deferred a separate `@deepseek-ai/dsh-file-context` package. Split the stack into four layers: ```text -tool dsh-tool-fs model-facing schemas + text rendering -policy dsh-file-context ctx.fileContext (concrete service): observed-state, read windowing, write/edit freshness -provider seam dsh-fs ctx.fs: text IO + guarded mutation primitives +tool dsh-tool-fs model-facing schemas + read windowing + text rendering; the EXECUTOR (reads/writes/edits via ctx.fs, dispatches the fs/* events) +policy dsh-file-context observed-state + read-before-edit + write/edit freshness, contributed through the fs/* event gate (no service) +provider seam dsh-fs ctx.fs: text IO + atomic mutation primitives (optional version guard) provider dsh-fs-local local implementation of ctx.fs ``` -`dsh-tool-fs` keeps the same model-facing `read`/`write`/`edit` schemas. It injects `fileContext`, not `fs`, and never reaches around the policy layer for model reads/writes/edits. +`dsh-tool-fs` keeps the same model-facing `read`/`write`/`edit` schemas. It injects `fs` (not a policy service) and reaches `ctx.fs` directly, dispatching the `fs/*` policy events so `dsh-file-context` can gate and record. + +The tool↔policy COUPLING below was reworked by [the file-context event-gate RFC](../architecture/2026-06-26-file-context-as-event-gate.md): `dsh-file-context` is now a gate PLUGIN that participates through the `fs/*` events (no `ctx.fileContext` service), and read windowing + the fs I/O moved up into `dsh-tool-fs`. The four-layer split, the provider contract, and the freshness *policy* this RFC decided are unchanged. Read the "`ctx.fileContext.read`/`write`/`edit`" method descriptions below as the policy DECISIONS the gate plugin now makes on the `fs/*` events, and the provider's version guard as optional (omit = unconditional bare provider). ## Provider Contract diff --git a/packages/README.md b/packages/README.md index 265fdd8676..85e802a5ab 100644 --- a/packages/README.md +++ b/packages/README.md @@ -31,10 +31,10 @@ dsh-agent ← dsh-llm, dsh-session, dsh-brand dsh-tools ← dsh-llm, dsh-system-prompt, dsh-agent dsh-bash-local ← dsh-bash (BashExecutor impl) dsh-tool-bash ← dsh-bash, dsh-tools (bash tool schemas) -dsh-fs ← dsh-llm, dsh-brand (filesystem provider seam) +dsh-fs ← dsh-llm, dsh-brand (filesystem provider seam + fs/* events) dsh-fs-local ← dsh-fs (FileSystem impl) -dsh-file-context ← dsh-fs (read windowing + write/edit freshness policy) -dsh-tool-fs ← dsh-file-context, dsh-fs, dsh-tools (file tool schemas) +dsh-file-context ← dsh-fs (observed-state + freshness policy gate, no service) +dsh-tool-fs ← dsh-fs, dsh-tools (file tools + executor) dsh-llm-deepseek ← dsh-llm (DeepSeek adapter) dsh-llm-pi-ai ← dsh-llm (pi-ai-backed adapter) dsh-agent-loop ← dsh-llm, dsh-session, dsh-system-prompt, dsh-tools, dsh-agent @@ -63,10 +63,10 @@ The rule: **extension** plugins depend on interfaces, never on the concrete loop | `bash/` | `bash` | Abstract bash executor seam (interface + vocabulary) | `ctx.bash` | | `bash-local/` | `bash` | Local-subprocess `BashExecutor` implementation | (registers `ctx.bash`) | | `tool-bash/` | `bash` | Model-facing `bash`/`bash_output`/`bash_kill` tool schemas | (registers on `ctx.tools`) | -| `fs/` | `fs` | Filesystem provider seam: text IO + guarded mutation primitives | `ctx.fs` | +| `fs/` | `fs` | Filesystem provider seam: text IO + atomic mutation primitives (optional version guard); owns the `fs/*` events | `ctx.fs` | | `fs-local/` | `fs` | Local-filesystem `FileSystem` implementation | (registers `ctx.fs`) | -| `file-context/` | `fs` | Policy layer: read windowing, observed-state, write/edit freshness | `ctx.fileContext` | -| `tool-fs/` | `fs` | Model-facing `read`/`write`/`edit` tool schemas | (registers on `ctx.tools`) | +| `file-context/` | `fs` | Policy gate plugin: observed-state + read-before-edit + version-guarded write/edit via the `fs/*` event gate | (no service — `fs/*` listeners) | +| `tool-fs/` | `fs` | Model-facing `read`/`write`/`edit` tools + executor (reads via `ctx.fs`, owns read windowing, dispatches `fs/*`) | (registers on `ctx.tools`) | | `llm-deepseek/` | `llm` | DeepSeek API adapter (hand-rolled fetch/SSE) | (registers on `ctx.llm`) | | `llm-pi-ai/` | `llm` | DeepSeek adapter via `@earendil-works/pi-ai` (design twin) | (registers on `ctx.llm`) | | `session-persistence/` | `session-persistence` | Persistence seam + write coordinator | `ctx.sessionPersistence` | diff --git a/packages/fs/README.md b/packages/fs/README.md index a793a94b4b..79e1d5c0f6 100644 --- a/packages/fs/README.md +++ b/packages/fs/README.md @@ -1,12 +1,12 @@ # fs/ - filesystem capability family -The filesystem stack: a provider seam (text IO + guarded mutation), a local implementation, a policy layer (read windowing + write/edit freshness), and the model-facing file tools. All **product** packages. +The filesystem stack: a provider seam (text IO + atomic mutation with an optional version guard), a local implementation, a policy gate plugin (observed-state + read-before-edit + version-guarded write/edit), and the model-facing file tools + executor. All **product** packages. | Package | Role | ctx key | |---|---|---| -| `fs/` | Provider seam: text IO + guarded mutation primitives | `ctx.fs` | +| `fs/` | Provider seam: text IO + atomic mutation primitives (optional version guard); owns the `fs/*` policy events | `ctx.fs` | | `fs-local/` | Local-filesystem `FileSystem` implementation | (registers `ctx.fs`) | -| `file-context/` | Policy layer: observed-state, read windowing, write/edit freshness | `ctx.fileContext` | -| `tool-fs/` | Model-facing `read`/`write`/`edit` tool schemas | (registers on `ctx.tools`) | +| `file-context/` | Policy gate plugin: observed-state + read-before-edit + version-guarded write/edit, via the `fs/*` event gate | (no service — `fs/*` listeners) | +| `tool-fs/` | Model-facing `read`/`write`/`edit` tools AND the executor (reads via `ctx.fs`, owns read windowing, dispatches `fs/*`) | (registers on `ctx.tools`) | -The interface lives at `fs/fs/`. A sandboxed, remote, or project-scoped filesystem backend can replace `fs-local` without touching the seam, the policy layer, or the model-facing tool schemas. The policy layer (`file-context/`) is a concrete service, not a swappable seam — it owns the model-facing observation policy that does not belong on a provider backend. +The interface lives at `fs/fs/`. A sandboxed, remote, or project-scoped filesystem backend can replace `fs-local` without touching the seam, the policy gate, or the model-facing tool schemas. The policy (`file-context/`) is a plugin that participates only through the `fs/*` event gate, not a service the tool injects — so dropping it gracefully loses the policy and leaves the unconstrained bare provider rather than breaking the tool. The default product config loads it. diff --git a/packages/fs/file-context/README.md b/packages/fs/file-context/README.md index 373fb2759b..b543722055 100644 --- a/packages/fs/file-context/README.md +++ b/packages/fs/file-context/README.md @@ -1,16 +1,18 @@ # @deepseek-ai/dsh-file-context -The **file-context policy layer**: a concrete `ctx.fileContext` service that owns model-facing read windowing and write/edit freshness on top of the `ctx.fs` provider seam ([`@deepseek-ai/dsh-fs`](../fs)). This is the policy third of the filesystem stack — it is **not** a swappable seam, but the deferred policy layer that does not belong on the `FileSystem` provider base class. +The **file-context policy plugin**: it adds observed-state, read-before-edit, and version-guarded write/edit on top of the `ctx.fs` provider seam ([`@deepseek-ai/dsh-fs`](../fs)) — through the `fs/*` event gate, **NOT** through a method service. This plugin registers **no** `ctx.fileContext` service and has no public `read`/`write`/`edit`/`resolve` methods. It is the policy third of the filesystem stack: not a swappable seam, but the policy that does not belong on the `FileSystem` provider base class. ```ts import type { Context } from 'cordis' -import FileContext from '@deepseek-ai/dsh-file-context' +import * as FileContext from '@deepseek-ai/dsh-file-context' declare const ctx: Context -// A ctx.fs provider must already be loaded (e.g. @deepseek-ai/dsh-fs-local); -// FileContext injects `fs` and registers ctx.fileContext. Load -// @deepseek-ai/dsh-tool-fs afterwards to expose read/write/edit to the model. +// No service to inject — this plugin only registers the three fs/* listeners. +// Load it alongside a ctx.fs provider (e.g. @deepseek-ai/dsh-fs-local) and the +// @deepseek-ai/dsh-tool-fs tools; the tools dispatch the fs/* events this plugin +// decides. Order does not matter for resolution (no inject), but the policy +// listener should be the first decider registered for the fs/*-expectation slots. await ctx.plugin(FileContext) ``` @@ -18,26 +20,29 @@ await ctx.plugin(FileContext) | Layer | Package | Role | |---|---|---| -| tool | `@deepseek-ai/dsh-tool-fs` | model-facing schemas + text rendering | -| policy | `@deepseek-ai/dsh-file-context` (this) | `ctx.fileContext`: observed-state, read windowing, write/edit freshness | -| provider seam | `@deepseek-ai/dsh-fs` | `ctx.fs`: text IO + guarded mutation primitives | +| tool / executor | `@deepseek-ai/dsh-tool-fs` | model-facing schemas + read windowing + text rendering; reads/writes/edits via `ctx.fs`, dispatches the `fs/*` events | +| policy | `@deepseek-ai/dsh-file-context` (this) | observed-state + read-before-edit + version-guarded write/edit, contributed through the `fs/*` event gate (no service) | +| provider seam | `@deepseek-ai/dsh-fs` | `ctx.fs`: text IO + atomic mutation primitives (optional version guard); owns the `fs/*` event vocabulary | | provider | `@deepseek-ai/dsh-fs-local` | local implementation of `ctx.fs` | -## Service API (`ctx.fileContext`) +## How the gate participates -| Member | Semantics | +Three `fs/*` events (declared by `@deepseek-ai/dsh-fs`, dispatched by `@deepseek-ai/dsh-tool-fs`): + +| Event | This plugin's listener | |---|---| -| `read(target, request, exec?, signal?)` | Stats the target, rejects absent/non-regular targets, chooses `readText`/`streamText` by size, builds the requested line window, records the version, and returns the `FileReadOutcome` the tool renders. | -| `write(target, content, exec?, signal?)` | No recorded read → `writeText({ kind: 'createIfAbsent' })` (only new files create blindly); a recorded read → `writeText({ kind: 'replaceIfVersion', version })`. Refreshes recorded state on success. | -| `edit(target, edit, exec?, signal?)` | Requires a recorded read by this owner (else `FS_NOT_OBSERVED`); passes the observed version to `ctx.fs.editText` as the stale guard and refreshes recorded state. | -| `owner(exec?)` | Derives the observed-state owner (`exec.agent.session`) — `undefined` when there is none. | +| `fs/write-expectation` | No prior observation → `{ kind: 'createIfAbsent' }`; a prior observation → `{ kind: 'replaceIfVersion', version: vObserved }`. Single-slot decision; does NOT call `next()`. | +| `fs/edit-expectation` | Requires a prior observation by this owner (else throws `FS_NOT_OBSERVED`); returns `{ version: vObserved }` as the CAS basis. Single-slot decision; does NOT call `next()`. | +| `fs/observed` | Records `{ version }` for this owner+target. Synchronous, side-effect-only `WeakMap.set`. | -## Observed state is the read record, freshness is the authorization +## Observed state is the prior-observation record; freshness is provider CAS -Observed state is a `WeakMap>`. An entry exists **iff** the owner has read that target through `read`, so its presence *is* the read record — there is no `hasRead` flag and no `full`/`partial` view. Authorization is based on version freshness only: a windowed read of lines 100-150 records the file's version, and a later edit of line 120 is authorized as long as the file is unchanged (the provider's stale guard enforces it). State is held weakly and dropped on disposal (HMR safety); persistence across sessions is deferred. +Observed state is a `WeakMap>`. An entry exists **iff** the owner has read, written, OR edited that target (every success emits `fs/observed`), so its presence is the prior-observation record — there is no `hasRead` flag and no `full`/`partial` view. This plugin does **no** filesystem I/O: "have you observed this file?" is a `WeakMap` lookup, and "is the version you read still current?" is decided inside `ctx.fs.editText`/`writeText` in the same atomic lock that performs the mutation — this plugin only supplies `vObserved` as the basis. A windowed read of lines 100-150 records the file's version, and a later edit of line 120 is authorized as long as the file is unchanged. State is held weakly and dropped on disposal (HMR safety); persistence across sessions is deferred. -## The no-bypass contract +## Single-slot, first-wins -A model-facing read MUST go through `ctx.fileContext.read`, never `ctx.fs.readText`/`streamText`, so every successful read records observed state before the tool renders. Direct `ctx.fs` calls remain an explicit escape hatch for non-tool consumers: a direct `ctx.fs.readText` records nothing, so a later `ctx.fileContext.edit` rejects with `FS_NOT_OBSERVED` until the file is read through `ctx.fileContext`. +The `fs/write-expectation`/`fs/edit-expectation` slots hold exactly one decider — this plugin fully decides and does not call `next()`. The slot is first-wins by registration order; this plugin owning it is the default-deployment convention, not an event-enforced invariant (a decider registered before / `prepend`ed would win instead). This is not a composable authorization chain — layered permission/audit/sandbox interception belongs on `tools/execute`. -The line-windowing mechanics live in `src/window.ts` (Cordis-free, independently unit-tested); `src/index.ts` is the service wiring and policy. +## No method coupling + +Because the plugin influences the world only through events, removing it does not break `@deepseek-ai/dsh-tool-fs` at a service-injection boundary: the tool falls through to the bare `ctx.fs` provider (unconditional write/edit, no observed-state). Loading it back layers the policy on. That graceful add/remove is the whole point of the event gate over a mandatory method service. diff --git a/packages/fs/file-context/package.json b/packages/fs/file-context/package.json index 77c905703b..577b650aac 100644 --- a/packages/fs/file-context/package.json +++ b/packages/fs/file-context/package.json @@ -1,6 +1,6 @@ { "name": "@deepseek-ai/dsh-file-context", - "description": "File-context policy layer (ctx.fileContext) for the DeepSeek Harness — read windowing and write/edit freshness over the ctx.fs provider seam", + "description": "File-context policy plugin for the DeepSeek Harness — observed-state, read-before-edit, and version-guarded write/edit added over the ctx.fs provider seam through the fs/* event gate (no service surface)", "version": "0.0.1", "private": true, "type": "module", diff --git a/packages/fs/file-context/src/index.ts b/packages/fs/file-context/src/index.ts index c8ed44dde1..5e0488ea0e 100644 --- a/packages/fs/file-context/src/index.ts +++ b/packages/fs/file-context/src/index.ts @@ -1,193 +1,159 @@ /** - * The file-context policy layer (`ctx.fileContext`): a concrete service that - * owns model-facing read windowing and write/edit freshness on top of the - * `ctx.fs` provider seam. It is NOT a swappable seam — it is the previously - * deferred policy layer that does not belong on the `FileSystem` provider base - * class (where a sandboxed/remote backend would otherwise inherit model-facing - * observation policy it has no business carrying). + * The file-context policy PLUGIN: observed-state, read-before-edit, and + * "write/edit must be based on the version you read" — added on top of the + * `ctx.fs` provider seam through the `fs/*` event gate, NOT through a method + * service. This plugin registers NO `ctx.fileContext` service and exposes no + * `read`/`write`/`edit`/`resolve` methods; it influences the world only by + * deciding the `fs/write-expectation`/`fs/edit-expectation` waterfalls and + * recording on `fs/observed`. That is what keeps `@deepseek-ai/dsh-tool-fs` + * (the executor) free of any method coupling to the policy layer — removing + * this plugin gracefully loses the policy and leaves the unconstrained bare + * provider, rather than breaking the tool at a service-injection boundary. * - * ## Observed state IS the read record + * ## Observed state IS the prior-observation record * - * Observed state lives here as `WeakMap>`. An - * entry exists iff the owner has read that target through {@link read}, so its - * presence *is* the read record — there is no separate `hasRead` flag. The owner - * is derived structurally from `{ agent?: { session? } }` and held weakly, so a - * collected session frees its state; disposal drops everything (HMR safety). + * State lives here as `WeakMap>`. An entry + * exists iff the owner has read, written, OR edited that target (every success + * emits `fs/observed`), so its presence means "this owner has observed this + * target at this version". This is what lets a create-then-edit or + * edit-then-edit sequence work without an intervening re-read: the mutation + * refreshes the recorded version to its own result. The owner is derived + * structurally from `{ agent?: { session? } }` and held weakly, so a collected + * session frees its state; disposal drops everything (HMR safety). * - * ## Freshness, not full/partial views + * ## Freshness via provider CAS, not stat * - * Authorization is based on version freshness only. A windowed read records the - * file's version, and any later write/edit at that version is authorized — a - * model that read lines 100-150 of a large file can still edit line 120 as long - * as the file is unchanged. There is no `full`/`partial` distinction: the bytes - * the edit matches must merely come from the version the model read, which the - * provider's stale guard enforces. + * This plugin does NO filesystem I/O. "Have you observed this file?" is a + * `WeakMap` lookup (no record ⇒ `FS_NOT_OBSERVED`). "Is the version you read + * still current?" is decided INSIDE `ctx.fs.editText`/`writeText`, in the same + * atomic lock that performs the mutation — this plugin only supplies the + * observed version as the CAS basis. Stat-ing and comparing here would open a + * TOCTOU gap the provider lock has to back up anyway, so it is deliberately + * avoided. * - * ## The no-bypass contract + * ## Single-slot, first-wins * - * A model-facing read MUST go through {@link read} (never `ctx.fs.readText`/ - * `streamText` directly), so every successful read records observed state before - * the tool renders. Direct `ctx.fs` calls are allowed for non-tool consumers but - * record nothing, so a later {@link edit} rejects with `FS_NOT_OBSERVED` until - * the file is read through `ctx.fileContext`. + * The `fs/write-expectation`/`fs/edit-expectation` listeners do NOT call + * `next()`: each fully decides its single slot. The slot is first-wins by + * registration order — this plugin owning it is the default-deployment + * convention, not an event-enforced invariant (a decider registered before / + * `prepend`ed would win instead). This is not a composable authorization chain; + * layered permission/audit/sandbox interception belongs on `tools/execute`. * * @module @deepseek-ai/dsh-file-context */ -import { Context, Service } from 'cordis' +import type { Context } from 'cordis' import { FsError } from '@deepseek-ai/dsh-fs' -import type { FsTarget, FsVersion, FsEditRequest, FsEditOutcome, FsWriteOutcome } from '@deepseek-ai/dsh-fs' -import { buildWindow } from './window.ts' -import type { FileContextExec, FileReadRequest, FileReadOutcome } from './types.ts' +import type { FsTarget, FsVersion, FsWriteExpectation } from '@deepseek-ai/dsh-fs' +import type { FileContextExec } from './types.ts' -export type { FileTextLine, ReadWindow, WindowResult } from './window.ts' -export { READ_MAX_BYTES, READ_MAX_LINE_LENGTH, buildWindow } from './window.ts' -export type { FileContextExec, FileReadRequest, FileReadOutcome } from './types.ts' - -/** Files at or above this size stream; smaller files read whole into memory. */ -export const STREAM_MIN_SIZE = 10 * 1024 * 1024 - -declare module 'cordis' { - interface Context { - fileContext: FileContext - } -} - -/** What an owner has observed about one target: just the version it last saw. */ -interface ObservedState { - version: FsVersion -} +export type { FileContextExec } from './types.ts' /** - * The file-context policy service. Injects `fs`, registers as `ctx.fileContext`, - * and is the only read/write/edit path the model-facing tools use. + * Per-context observed-file state and the three `fs/*` decisions over it. One + * instance is created per `apply()` so disposal can drop all state for HMR. */ -export class FileContext extends Service { - static inject = ['fs'] - +class ObservedStateGate { /** * Observed-file state, keyed first by the owner object (weakly held, so a * collected session frees its state), then by {@link FsTarget.targetKey}. An - * entry's PRESENCE is the read record. + * entry's PRESENCE is the prior-observation record. */ - private observed = new WeakMap>() - - constructor(ctx: Context) { - super(ctx, 'fileContext') - ctx.effect(() => () => { - // Drop all recorded state on disposal so a reloaded service starts clean - // (HMR safety). The WeakMap itself would be GC'd, but replacing it makes - // the release observable and immediate for tests. - this.observed = new WeakMap() - }, 'fileContext observed-state teardown') - } + private observed = new WeakMap>() /** - * Derive the observed-state owner from an execution context — normally the + * Derive the observed-state owner from the opaque event actor — normally the * active agent session. `undefined` when no owner can be derived (e.g. a * direct tool call with no agent); such calls read freely but cannot satisfy * the write/edit prior-observation policy. */ - owner(exec?: FileContextExec): object | undefined { - return exec?.agent?.session + private owner(actor: object | undefined): object | undefined { + return (actor as FileContextExec | undefined)?.agent?.session } - private getObserved(owner: object, targetKey: string): ObservedState | undefined { + private get(owner: object, targetKey: string): FsVersion | undefined { return this.observed.get(owner)?.get(targetKey) } - private record(owner: object, targetKey: string, version: FsVersion): void { + private set(owner: object, targetKey: string, version: FsVersion): void { let byTarget = this.observed.get(owner) if (!byTarget) { byTarget = new Map() this.observed.set(owner, byTarget) } - byTarget.set(targetKey, { version }) + byTarget.set(targetKey, version) + } + + /** Drop all recorded state (HMR safety / disposal). */ + clear(): void { + this.observed = new WeakMap() } /** - * Resolve a path into a stable {@link FsTarget}, delegating to the provider. - * Exposed here so the model-facing tools never need to inject `ctx.fs` - * directly — they resolve and then read/write/edit entirely through - * `ctx.fileContext`. + * Decide the write expectation: no prior observation ⇒ `createIfAbsent` (only + * new files can be created blindly); a prior observation ⇒ `replaceIfVersion` + * at the observed version (existing files replaced only if unchanged). */ - async resolve(path: string): Promise { - return this.ctx.fs.resolve(path) + writeExpectation(target: FsTarget, actor: object | undefined): FsWriteExpectation { + const owner = this.owner(actor) + const prior = owner ? this.get(owner, target.targetKey) : undefined + return prior ? { kind: 'replaceIfVersion', version: prior } : { kind: 'createIfAbsent' } } /** - * Read a bounded line window from a target. Stats first (rejecting an absent - * target with `FS_NOT_FOUND` and a non-regular one with `FS_NOT_REGULAR_FILE`), - * chooses `readText` vs `streamText` by size — streaming when the size is - * large OR unknown so a size-less backend never buffers an arbitrarily large - * file — builds the window, then records the version observed AFTER the read - * so the recorded freshness token corresponds to the bytes actually returned - * (a writer racing between the routing stat and the read can't make a - * follow-up edit spuriously stale against a pre-read version). + * Decide the edit version guard: requires a prior observation by this owner + * (else `FS_NOT_OBSERVED`); returns the observed version as the CAS basis. */ - async read(target: FsTarget, request: FileReadRequest, exec?: FileContextExec, signal?: AbortSignal): Promise { - const info = await this.ctx.fs.stat(target, signal) - if (!info) throw new FsError(`cannot read "${target.displayPath}": not found`, 'FS_NOT_FOUND') - if (info.type !== 'file') throw new FsError(`cannot read "${target.displayPath}": not a regular file`, 'FS_NOT_REGULAR_FILE') - - const chunks = info.size === undefined || info.size >= STREAM_MIN_SIZE - ? await this.ctx.fs.streamText(target, signal) - : [await this.ctx.fs.readText(target, signal)] - const window = await buildWindow(chunks, request, target.displayPath) - - // The version that matches the bytes just read: a stat taken after the read - // (falling back to the routing stat if the file vanished in the interim). - const after = await this.ctx.fs.stat(target, signal) - const version = after?.version ?? info.version - - const owner = this.owner(exec) - if (owner) this.record(owner, target.targetKey, version) - return { - offset: request.offset, - limit: request.limit, - lines: window.lines, - totalLines: window.totalLines, - version, - ...window.truncatedByBytes ? { truncatedByBytes: true } : {}, - } - } - - /** - * Create or fully replace a file. With no recorded read, writes - * `createIfAbsent` (only new files can be created blindly); with a recorded - * read, writes `replaceIfVersion` at the observed version (existing files are - * replaced only if unchanged since the read). Refreshes recorded state from - * the returned version on success. - */ - async write(target: FsTarget, content: string, exec?: FileContextExec, signal?: AbortSignal): Promise { - const owner = this.owner(exec) - const prior = owner ? this.getObserved(owner, target.targetKey) : undefined - const outcome = await this.ctx.fs.writeText( - target, - content, - prior ? { kind: 'replaceIfVersion', version: prior.version } : { kind: 'createIfAbsent' }, - signal, - ) - if (owner) this.record(owner, target.targetKey, outcome.version) - return outcome - } - - /** - * Apply a literal edit. Requires a recorded read by this owner (else - * `FS_NOT_OBSERVED`); passes the observed version to `ctx.fs.editText` as the - * stale guard and refreshes recorded state from the returned version. The - * provider owns the mutation critical section and the literal match. - */ - async edit(target: FsTarget, edit: FsEditRequest, exec?: FileContextExec, signal?: AbortSignal): Promise { - const owner = this.owner(exec) - const prior = owner ? this.getObserved(owner, target.targetKey) : undefined + editExpectation(target: FsTarget, actor: object | undefined): { version: FsVersion } { + const owner = this.owner(actor) + const prior = owner ? this.get(owner, target.targetKey) : undefined if (!owner || !prior) { throw new FsError(`edit requires reading "${target.displayPath}" first`, 'FS_NOT_OBSERVED') } - const outcome = await this.ctx.fs.editText(target, edit, { version: prior.version }, signal) - this.record(owner, target.targetKey, outcome.version) - return outcome + return { version: prior } + } + + /** Record a successful read/write/edit: this owner observed this target at this version. */ + observe(target: FsTarget, version: FsVersion, actor: object | undefined): void { + const owner = this.owner(actor) + if (owner) this.set(owner, target.targetKey, version) } } -export default FileContext +/** Cordis plugin name used by loader diagnostics. */ +export const name = 'file-context' + +/** + * Register the three `fs/*` listeners. No `inject` — this plugin reads no + * services; it operates only on its own `WeakMap`. The waterfalls are unbound + * (the tool dispatches them with no `this`), so the listeners take the raw + * `(target, actor, next)` arguments. + */ +export function apply(ctx: Context): void { + const gate = new ObservedStateGate() + + ctx.effect(() => () => { + // Drop all recorded state on disposal so a reloaded plugin starts clean + // (HMR safety). The WeakMap itself would be GC'd, but replacing it makes the + // release observable and immediate for tests. + gate.clear() + }, 'file-context observed-state teardown') + + // fs/write-expectation: occupy the single decision slot — do NOT call next(). + // Deferred through Promise.resolve().then so the declared Promise return type + // holds (a throw rejects, never escapes synchronously through the waterfall). + ctx.on('fs/write-expectation', (target, actor) => Promise.resolve().then(() => gate.writeExpectation(target, actor))) + + // fs/edit-expectation: occupy the single decision slot — do NOT call next(). + // Deferred the same way so an FS_NOT_OBSERVED throw becomes a rejected promise + // the edit tool's `await ctx.waterfall(...)` surfaces as its isError result. + ctx.on('fs/edit-expectation', (target, actor) => Promise.resolve().then(() => gate.editExpectation(target, actor))) + + // fs/observed: synchronous, side-effect-only WeakMap write (cannot throw under + // normal operation); the tool contains any throw so a record bug never fails + // the already-completed mutation. + ctx.on('fs/observed', (target, version, actor) => { + gate.observe(target, version, actor) + }) +} diff --git a/packages/fs/file-context/src/types.ts b/packages/fs/file-context/src/types.ts index 842d9a08c7..b3157cc2e9 100644 --- a/packages/fs/file-context/src/types.ts +++ b/packages/fs/file-context/src/types.ts @@ -1,24 +1,21 @@ /** - * Vocabulary for the file-context policy layer (`ctx.fileContext`): the - * minimal execution-context shape used to derive an observed-state owner, the - * resolved read window, and the structured read outcome the model-facing `read` - * tool renders. + * Vocabulary for the file-context policy plugin: the minimal execution-context + * shape used to derive an observed-state owner by narrowing the opaque `object` + * actor the `fs/*` events carry. * * The provider vocabulary (`FsTarget`, `FsVersion`, write/edit shapes) is - * re-used from `@deepseek-ai/dsh-fs` — this package owns only the model-facing - * read-windowing and observation policy on top of it. + * re-used from `@deepseek-ai/dsh-fs`; this package owns only the observed-state + * owner structure on top of it. * * @module @deepseek-ai/dsh-file-context/types */ -import type { FsVersion } from '@deepseek-ai/dsh-fs' -import type { FileTextLine } from './window.ts' - /** - * Minimal structural view of a tool execution the policy layer needs to derive + * Minimal structural view of a tool execution the policy plugin needs to derive * an observed-state owner. `@deepseek-ai/dsh-tools`' `ToolExecution` satisfies - * this shape, so the consumer passes its `exec` straight through without - * `dsh-file-context` importing `dsh-tools`, `dsh-agent`, or `dsh-session`. + * this shape, so the tool passes its `exec` straight through as the opaque + * `object` actor on the `fs/*` events; this plugin narrows that actor to this + * shape without importing `dsh-tools`, `dsh-agent`, or `dsh-session`. * * The owner is `agent.session` when present. It is treated as an opaque object * identity (a `WeakMap` key); this package never reads any of its fields. @@ -30,27 +27,3 @@ export interface FileContextExec { session?: object } } - -/** Resolved read window. The consumer applies its defaults/caps before calling. */ -export interface FileReadRequest { - /** 1-based first line to return. */ - offset: number - /** Maximum number of lines to return. */ - limit: number -} - -/** Outcome of a bounded text read — what the model-facing `read` tool renders. */ -export interface FileReadOutcome { - /** 1-based first line requested. */ - offset: number - /** Maximum number of lines requested. */ - limit: number - /** Returned lines, already numbered. */ - lines: FileTextLine[] - /** Total line count in the file, unless `truncatedByBytes` stopped scanning early. */ - totalLines: number - /** Whether selected output hit the byte cap before EOF or the requested limit. */ - truncatedByBytes?: true - /** Opaque version of the file at read time. */ - version: FsVersion -} diff --git a/packages/fs/file-context/tests/policy.spec.ts b/packages/fs/file-context/tests/policy.spec.ts index 317776d5b3..63808f1610 100644 --- a/packages/fs/file-context/tests/policy.spec.ts +++ b/packages/fs/file-context/tests/policy.spec.ts @@ -1,349 +1,192 @@ /** - * Tests for the file-context policy layer: registration/disposal/HMR, owner - * derivation, observed-state-as-read-record, read windowing over a fake - * provider, freshness-based write/edit authorization (including the key - * windowed-read-authorizes-edit behavior), the read→streamText size routing, - * and multi-owner isolation. The provider is a fake `ctx.fs` recording the - * expectations it was handed. + * Tests for the file-context policy PLUGIN: it registers no service, only the + * three `fs/*` listeners. We dispatch those events directly (the unbound + * waterfalls the tool would dispatch, and the `fs/observed` emit) and assert the + * decisions: createIfAbsent vs replaceIfVersion, FS_NOT_OBSERVED for an unread + * edit, observed-state-as-prior-observation (read/write/edit all record), + * multi-owner isolation, single-slot first-wins, and disposal/HMR release. + * + * No `ctx.fs` provider is needed — the plugin does no filesystem I/O; it only + * decides expectations and records versions on its own WeakMap. */ import { describe, expect, it } from 'vitest' import { Context } from 'cordis' -import { FileSystem, FsError, FsTargetKey, FsVersion } from '@deepseek-ai/dsh-fs' -import type { - FsEditOutcome, - FsEditRequest, - FsInfo, - FsTarget, - FsWriteExpectation, - FsWriteOutcome, -} from '@deepseek-ai/dsh-fs' -import FileContext, { STREAM_MIN_SIZE } from '@deepseek-ai/dsh-file-context' -import type { FileContextExec, FileReadRequest } from '@deepseek-ai/dsh-file-context' +import { FsTargetKey, FsVersion } from '@deepseek-ai/dsh-fs' +import type { FsTarget, FsWriteExpectation } from '@deepseek-ai/dsh-fs' +import * as FileContext from '@deepseek-ai/dsh-file-context' +import type { FileContextExec } from '@deepseek-ai/dsh-file-context' -/** A fake provider: in-memory files, recording every expectation/version it is handed. */ -class FakeFs extends FileSystem { - files = new Map() - versions = new Map() - /** Size to report from stat (lets a test push read onto the streaming path). */ - reportSize?: number - /** When true, stat omits `size` entirely (a size-less backend). */ - omitSize = false - /** Whether streamText was used for the last read (vs readText). */ - lastReadStreamed = false - writeExpectations: FsWriteExpectation[] = [] - editExpectedVersions: string[] = [] +function target(path: string): FsTarget { + return { inputPath: path, targetKey: FsTargetKey(path), displayPath: path } +} +const ownerExec = (session: object): FileContextExec => ({ agent: { session } }) - private ver(key: string): FsVersion { - return FsVersion(`v${this.versions.get(key) ?? 0}`) - } - private bump(key: string): FsVersion { - const next = (this.versions.get(key) ?? 0) + 1 - this.versions.set(key, next) - return FsVersion(`v${next}`) - } - - override async resolve(path: string): Promise { - return { inputPath: path, targetKey: FsTargetKey(path), displayPath: path } - } - override async stat(target: FsTarget): Promise { - const content = this.files.get(target.targetKey) - if (content === undefined) return undefined - return { version: this.ver(target.targetKey), type: 'file', ...this.omitSize ? {} : { size: this.reportSize ?? content.length } } - } - override async readText(target: FsTarget): Promise { - this.lastReadStreamed = false - return this.files.get(target.targetKey) ?? '' - } - override async streamText(target: FsTarget): Promise> { - this.lastReadStreamed = true - const content = this.files.get(target.targetKey) ?? '' - return (async function* () { yield content })() - } - override async writeText(target: FsTarget, content: string, expected: FsWriteExpectation): Promise { - this.writeExpectations.push(expected) - const existed = this.files.has(target.targetKey) - this.files.set(target.targetKey, content) - return { operation: existed ? 'update' : 'create', version: this.bump(target.targetKey) } - } - override async editText(target: FsTarget, edit: FsEditRequest, expected: { version: FsVersion }): Promise { - this.editExpectedVersions.push(expected.version) - const content = this.files.get(target.targetKey) ?? '' - this.files.set(target.targetKey, content.split(edit.oldString).join(edit.newString)) - return { replacements: 1, replaceAll: edit.replaceAll, version: this.bump(target.targetKey) } - } +/** Dispatch the write-expectation waterfall with the bare default thunk. */ +function writeExpectation(ctx: Context, t: FsTarget, actor: object | undefined): Promise { + return ctx.waterfall('fs/write-expectation', t, actor, () => undefined) +} +/** Dispatch the edit-expectation waterfall with the bare default thunk. */ +function editExpectation(ctx: Context, t: FsTarget, actor: object | undefined): Promise<{ version: FsVersion } | undefined> { + return ctx.waterfall('fs/edit-expectation', t, actor, () => undefined) } async function setup() { const ctx = new Context() - await ctx.plugin(FakeFs) - await ctx.plugin(FileContext) - const fs = ctx.fs as FakeFs - const fileContext = ctx.fileContext - return { ctx, fs, fileContext } + const fiber = await ctx.plugin(FileContext) + return { ctx, fiber } } -const READ_ALL: FileReadRequest = { offset: 1, limit: 2000 } -const ownerExec = (session: object): FileContextExec => ({ agent: { session } }) - describe('registration / disposal', () => { - it('registers as ctx.fileContext and injects fs', async () => { - const { fileContext } = await setup() - expect(fileContext).toBeDefined() + it('registers no service surface (it is a plugin, not ctx.fileContext)', async () => { + const { ctx } = await setup() + expect((ctx as Context & { fileContext?: unknown }).fileContext).toBeUndefined() }) - it('stays pending until ctx.fs exists', async () => { + it('mounts with no inject (reads no services)', async () => { + // It mounts immediately even with nothing else in the context. const ctx = new Context() - await ctx.plugin(FileContext) // no fs provider - expect(ctx.fileContext).toBeUndefined() - }) - - it('withdraws ctx.fileContext when its fiber is disposed (HMR safety)', async () => { - const ctx = new Context() - await ctx.plugin(FakeFs) - const fiber = await ctx.plugin(FileContext) - expect(ctx.fileContext).toBeDefined() - await fiber.dispose() - expect(ctx.fileContext).toBeUndefined() + await ctx.plugin(FileContext) + // The listener is live: an unobserved write decides createIfAbsent. + expect(await writeExpectation(ctx, target('a.txt'), undefined)).toEqual({ kind: 'createIfAbsent' }) }) }) -describe('owner derivation', () => { - it('derives the owner from exec.agent.session', async () => { - const { fileContext } = await setup() - const session = {} - expect(fileContext.owner(ownerExec(session))).toBe(session) +describe('write-expectation decision', () => { + it('an unobserved target decides createIfAbsent', async () => { + const { ctx } = await setup() + expect(await writeExpectation(ctx, target('a.txt'), ownerExec({}))).toEqual({ kind: 'createIfAbsent' }) }) - it('returns undefined with no exec, no agent, or no session', async () => { - const { fileContext } = await setup() - expect(fileContext.owner()).toBeUndefined() - expect(fileContext.owner({})).toBeUndefined() - expect(fileContext.owner({ agent: {} })).toBeUndefined() + it('a no-owner actor decides createIfAbsent', async () => { + const { ctx } = await setup() + expect(await writeExpectation(ctx, target('a.txt'), undefined)).toEqual({ kind: 'createIfAbsent' }) + expect(await writeExpectation(ctx, target('a.txt'), {})).toEqual({ kind: 'createIfAbsent' }) + }) + + it('an observed target decides replaceIfVersion at the observed version', async () => { + const { ctx } = await setup() + const exec = ownerExec({}) + ctx.emit('fs/observed', target('a.txt'), FsVersion('v7'), exec) + expect(await writeExpectation(ctx, target('a.txt'), exec)).toEqual({ kind: 'replaceIfVersion', version: 'v7' }) }) }) -describe('read', () => { - it('returns a windowed outcome and rejects an absent target', async () => { - const { fs, fileContext } = await setup() - fs.files.set('a.txt', 'one\ntwo') - const outcome = await fileContext.read(await fs.resolve('a.txt'), READ_ALL) - expect(outcome.lines).toEqual([{ number: 1, text: 'one' }, { number: 2, text: 'two' }]) - expect(outcome.version).toBe('v0') - - await expect(fileContext.read(await fs.resolve('missing.txt'), READ_ALL)) - .rejects.toMatchObject({ code: 'FS_NOT_FOUND' }) +describe('edit-expectation decision', () => { + it('rejects an unread edit with FS_NOT_OBSERVED', async () => { + const { ctx } = await setup() + await expect(editExpectation(ctx, target('a.txt'), ownerExec({}))).rejects.toMatchObject({ code: 'FS_NOT_OBSERVED' }) }) - it('rejects a non-regular target', async () => { - const { fs, fileContext } = await setup() - fs.files.set('d', '') - const target = await fs.resolve('d') - // Force stat to report a directory. - fs.stat = async () => ({ version: FsVersion('v0'), type: 'directory' }) - await expect(fileContext.read(target, READ_ALL)).rejects.toMatchObject({ code: 'FS_NOT_REGULAR_FILE' }) + it('rejects an edit with no owner (cannot prove prior observation)', async () => { + const { ctx } = await setup() + await expect(editExpectation(ctx, target('a.txt'), undefined)).rejects.toMatchObject({ code: 'FS_NOT_OBSERVED' }) }) - it('reads small files whole and large files via streamText', async () => { - const { fs, fileContext } = await setup() - fs.files.set('a.txt', 'one\ntwo') - - await fileContext.read(await fs.resolve('a.txt'), READ_ALL) - expect(fs.lastReadStreamed).toBe(false) - - fs.reportSize = STREAM_MIN_SIZE - await fileContext.read(await fs.resolve('a.txt'), READ_ALL) - expect(fs.lastReadStreamed).toBe(true) - }) - - it('streams when the backend reports no size (never buffers a size-less file)', async () => { - const { fs, fileContext } = await setup() - fs.files.set('a.txt', 'one\ntwo') - fs.omitSize = true - await fileContext.read(await fs.resolve('a.txt'), READ_ALL) - expect(fs.lastReadStreamed).toBe(true) - }) - - it('records the version observed after the read, not the routing stat', async () => { - const { fs, fileContext } = await setup() + it('returns the observed version as the CAS basis after an observation', async () => { + const { ctx } = await setup() const exec = ownerExec({}) - fs.files.set('a.txt', 'hello') - fs.versions.set('a.txt', 1) - const target = await fs.resolve('a.txt') - // A writer bumps the version after the routing stat but before the post-read stat. - const realReadText = fs.readText.bind(fs) - fs.readText = async (t) => { - const text = await realReadText(t) - fs.versions.set('a.txt', 5) // file changed during the read - return text - } - const outcome = await fileContext.read(target, READ_ALL, exec) - expect(outcome.version).toBe('v5') - // The recorded (post-read) version authorizes an edit without going stale. - await fileContext.edit(target, { oldString: 'hello', newString: 'bye', replaceAll: false }, exec) - expect(fs.editExpectedVersions).toEqual(['v5']) - }) - - it('falls back to the routing-stat version if the file vanishes after the read', async () => { - const { fs, fileContext } = await setup() - fs.files.set('a.txt', 'hello') - const target = await fs.resolve('a.txt') - const realReadText = fs.readText.bind(fs) - fs.readText = async (t) => { - const text = await realReadText(t) - fs.files.delete('a.txt') // vanishes → post-read stat returns undefined - return text - } - const outcome = await fileContext.read(target, READ_ALL) - expect(outcome.version).toBe('v0') // the routing-stat version - }) - - it('surfaces truncatedByBytes when the window hits the byte cap', async () => { - const { fs, fileContext } = await setup() - fs.files.set('big.txt', Array.from({ length: 2000 }, () => 'y'.repeat(100)).join('\n')) - const outcome = await fileContext.read(await fs.resolve('big.txt'), READ_ALL) - expect(outcome.truncatedByBytes).toBe(true) + ctx.emit('fs/observed', target('a.txt'), FsVersion('v3'), exec) + expect(await editExpectation(ctx, target('a.txt'), exec)).toEqual({ version: 'v3' }) }) }) -describe('observed-state is the read record', () => { - it('a read authorizes a later in-place write at the observed version', async () => { - const { fs, fileContext } = await setup() +describe('observed-state is the prior-observation record', () => { + it('a read observation authorizes an in-place write at that version', async () => { + const { ctx } = await setup() const exec = ownerExec({}) - fs.files.set('a.txt', 'hello') - const target = await fs.resolve('a.txt') - - await fileContext.read(target, READ_ALL, exec) - await fileContext.write(target, 'goodbye', exec) - - expect(fs.writeExpectations).toEqual([{ kind: 'replaceIfVersion', version: 'v0' }]) + ctx.emit('fs/observed', target('a.txt'), FsVersion('v0'), exec) // a read + expect(await writeExpectation(ctx, target('a.txt'), exec)).toEqual({ kind: 'replaceIfVersion', version: 'v0' }) }) - it('a windowed (partial) read still authorizes edit — freshness, not full/partial', async () => { - const { fs, fileContext } = await setup() + it('a write/edit observation refreshes the basis, so the next edit needs no re-read', async () => { + const { ctx } = await setup() const exec = ownerExec({}) - fs.files.set('a.txt', 'one\ntwo\nthree\nfour') - const target = await fs.resolve('a.txt') - - // Read only lines 2-3 — a partial window. - const outcome = await fileContext.read(target, { offset: 2, limit: 2 }, exec) - expect(outcome.lines.map(l => l.number)).toEqual([2, 3]) - - // Edit is authorized anyway: the file is unchanged since the read. - await fileContext.edit(target, { oldString: 'one', newString: 'X', replaceAll: false }, exec) - expect(fs.editExpectedVersions).toEqual(['v0']) + // A create records v1; the follow-up edit guards against v1 with no read. + ctx.emit('fs/observed', target('a.txt'), FsVersion('v1'), exec) + expect(await editExpectation(ctx, target('a.txt'), exec)).toEqual({ version: 'v1' }) + // The edit records v2; a second edit guards against v2. + ctx.emit('fs/observed', target('a.txt'), FsVersion('v2'), exec) + expect(await editExpectation(ctx, target('a.txt'), exec)).toEqual({ version: 'v2' }) }) - it('skips recording when there is no owner, so write is createIfAbsent', async () => { - const { fs, fileContext } = await setup() - fs.files.set('a.txt', 'hello') - const target = await fs.resolve('a.txt') - - await fileContext.read(target, READ_ALL) // no exec - // No recorded read → createIfAbsent → the provider rejects an existing target. - fs.writeText = async () => { throw new FsError('exists', 'FS_NOT_OBSERVED') } - await expect(fileContext.write(target, 'x')).rejects.toMatchObject({ code: 'FS_NOT_OBSERVED' }) - }) -}) - -describe('write policy', () => { - it('a create (no prior read) uses createIfAbsent', async () => { - const { fs, fileContext } = await setup() - const exec = ownerExec({}) - const target = await fs.resolve('new.txt') - const outcome = await fileContext.write(target, 'fresh', exec) - expect(outcome.operation).toBe('create') - expect(fs.writeExpectations).toEqual([{ kind: 'createIfAbsent' }]) - }) - - it('refreshes state after a write, so a follow-up edit needs no re-read', async () => { - const { fs, fileContext } = await setup() - const exec = ownerExec({}) - const target = await fs.resolve('a.txt') - await fileContext.write(target, 'one', exec) // create → state now at v1 - await fileContext.edit(target, { oldString: 'one', newString: 'two', replaceAll: false }, exec) - expect(fs.editExpectedVersions).toEqual(['v1']) - }) -}) - -describe('edit policy', () => { - it('rejects with FS_NOT_OBSERVED when the file was never read', async () => { - const { fs, fileContext } = await setup() - const exec = ownerExec({}) - fs.files.set('a.txt', 'hello') - const target = await fs.resolve('a.txt') - await expect(fileContext.edit(target, { oldString: 'hello', newString: 'bye', replaceAll: false }, exec)) - .rejects.toMatchObject({ code: 'FS_NOT_OBSERVED' }) - }) - - it('rejects when there is no owner (cannot prove prior observation)', async () => { - const { fs, fileContext } = await setup() - fs.files.set('a.txt', 'hello') - const target = await fs.resolve('a.txt') - await expect(fileContext.edit(target, { oldString: 'hello', newString: 'bye', replaceAll: false })) - .rejects.toMatchObject({ code: 'FS_NOT_OBSERVED' }) - }) - - it('passes the recorded version as the stale guard after a read', async () => { - const { fs, fileContext } = await setup() - const exec = ownerExec({}) - fs.files.set('a.txt', 'hello') - fs.versions.set('a.txt', 7) - const target = await fs.resolve('a.txt') - await fileContext.read(target, READ_ALL, exec) - await fileContext.edit(target, { oldString: 'hello', newString: 'bye', replaceAll: false }, exec) - expect(fs.editExpectedVersions).toEqual(['v7']) + it('a no-owner observation records nothing', async () => { + const { ctx } = await setup() + ctx.emit('fs/observed', target('a.txt'), FsVersion('v0'), undefined) + // Still unobserved for any owner. + await expect(editExpectation(ctx, target('a.txt'), ownerExec({}))).rejects.toMatchObject({ code: 'FS_NOT_OBSERVED' }) }) }) describe('multi-owner isolation', () => { - it('owner A reading does not grant owner B edit authority', async () => { - const { fs, fileContext } = await setup() + it('owner A observing does not grant owner B edit authority', async () => { + const { ctx } = await setup() const a = ownerExec({}) const b = ownerExec({}) - fs.files.set('a.txt', 'hello') - const target = await fs.resolve('a.txt') - - await fileContext.read(target, READ_ALL, a) - await expect(fileContext.edit(target, { oldString: 'hello', newString: 'bye', replaceAll: false }, b)) - .rejects.toMatchObject({ code: 'FS_NOT_OBSERVED' }) - await expect(fileContext.edit(target, { oldString: 'hello', newString: 'bye', replaceAll: false }, a)) - .resolves.toMatchObject({ replacements: 1 }) + ctx.emit('fs/observed', target('a.txt'), FsVersion('v0'), a) + await expect(editExpectation(ctx, target('a.txt'), b)).rejects.toMatchObject({ code: 'FS_NOT_OBSERVED' }) + expect(await editExpectation(ctx, target('a.txt'), a)).toEqual({ version: 'v0' }) }) it('each owner records its own observed version independently', async () => { - const { fs, fileContext } = await setup() + const { ctx } = await setup() const a = ownerExec({}) const b = ownerExec({}) - fs.files.set('a.txt', 'hello') - const target = await fs.resolve('a.txt') - - await fileContext.read(target, READ_ALL, a) // A sees v0 - await fileContext.write(target, 'mid', b) // B has no read → createIfAbsent - await fileContext.write(target, 'late', a) // A still holds its v0 observation - - expect(fs.writeExpectations).toEqual([ - { kind: 'createIfAbsent' }, - { kind: 'replaceIfVersion', version: 'v0' }, - ]) + ctx.emit('fs/observed', target('a.txt'), FsVersion('v0'), a) // A observed v0 + // B never observed → createIfAbsent; A still holds v0 → replaceIfVersion. + expect(await writeExpectation(ctx, target('a.txt'), b)).toEqual({ kind: 'createIfAbsent' }) + expect(await writeExpectation(ctx, target('a.txt'), a)).toEqual({ kind: 'replaceIfVersion', version: 'v0' }) }) }) -describe('disposal releases recorded state', () => { - it('a fresh service after disposal starts with no inherited state', async () => { - const ctx = new Context() - await ctx.plugin(FakeFs) - const fs = ctx.fs as FakeFs - const fiber = await ctx.plugin(FileContext) +describe('single-slot, first-wins', () => { + it('fully decides the slot without calling next() (the bare default is unreached)', async () => { + const { ctx } = await setup() + let defaultRan = false + const expectation = await ctx.waterfall('fs/write-expectation', target('a.txt'), ownerExec({}), () => { + defaultRan = true + return undefined + }) + expect(expectation).toEqual({ kind: 'createIfAbsent' }) + expect(defaultRan).toBe(false) + }) + + it('a SECOND decider registered AFTER file-context is not reached (first-wins short-circuit)', async () => { + const { ctx } = await setup() + let secondRan = false + // Registered after file-context, so it dispatches second; file-context does + // not call next(), so this never runs. (A decider registered BEFORE — or with + // prepend — would instead win: first-wins is by convention, not enforced.) + ctx.on('fs/edit-expectation', () => { + secondRan = true + return Promise.resolve(undefined) + }) const exec = ownerExec({}) - fs.files.set('a.txt', 'hello') - await ctx.fileContext.read(await fs.resolve('a.txt'), READ_ALL, exec) + ctx.emit('fs/observed', target('a.txt'), FsVersion('v0'), exec) + await editExpectation(ctx, target('a.txt'), exec) + expect(secondRan).toBe(false) + }) +}) + +describe('disposal releases recorded state (HMR safety)', () => { + it('a fresh plugin after disposal starts with no inherited state', async () => { + const ctx = new Context() + const exec = ownerExec({}) + const fiber = await ctx.plugin(FileContext) + ctx.emit('fs/observed', target('a.txt'), FsVersion('v0'), exec) + expect(await editExpectation(ctx, target('a.txt'), exec)).toEqual({ version: 'v0' }) await fiber.dispose() await ctx.plugin(FileContext) - const target = await fs.resolve('a.txt') // Same owner object, but state was released on disposal. - await expect(ctx.fileContext.edit(target, { oldString: 'hello', newString: 'bye', replaceAll: false }, exec)) - .rejects.toMatchObject({ code: 'FS_NOT_OBSERVED' }) + await expect(editExpectation(ctx, target('a.txt'), exec)).rejects.toMatchObject({ code: 'FS_NOT_OBSERVED' }) + }) + + it('no listeners remain after disposal (the gate no longer decides)', async () => { + const ctx = new Context() + const fiber = await ctx.plugin(FileContext) + await fiber.dispose() + // With no listener, the waterfall falls through to the bare default. + expect(await writeExpectation(ctx, target('a.txt'), ownerExec({}))).toBeUndefined() }) }) diff --git a/packages/fs/fs-local/README.md b/packages/fs/fs-local/README.md index a4bb087aea..60ad805938 100644 --- a/packages/fs/fs-local/README.md +++ b/packages/fs/fs-local/README.md @@ -6,17 +6,17 @@ The **local-filesystem implementation** of the `ctx.fs` provider seam ([`@deepse import { LocalFileSystem } from '@deepseek-ai/dsh-fs-local' await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) -// ctx.fs is now the local backend; load @deepseek-ai/dsh-file-context for policy -// and @deepseek-ai/dsh-tool-fs to expose read/write/edit to the model. +// ctx.fs is now the local backend; load @deepseek-ai/dsh-file-context for the +// freshness policy gate and @deepseek-ai/dsh-tool-fs to expose read/write/edit. ``` ## Behavior - **`resolve(path)`** — relative paths resolve from `config.cwd` (default `process.cwd()`). The `targetKey` is the file's `realpath`, so two input paths reaching the same file through symlinks share one identity, and writes/edits land on the link target (preserving the link). A not-yet-existing path uses the realpathed parent directory plus basename when the parent exists; only an unresolvable parent falls back to the absolute path. `displayPath` is the absolute (un-resolved) path. - **`stat`** — returns `FsInfo` (`version` = `mtimeMs:size`, `type` of `file`/`directory`/`other`, byte `size`) or `undefined` when the target is absent. -- **`readText` / `streamText`** — UTF-8 only. `readText` reads the whole file; `streamText` streams it in chunks (cross-chunk decoding) so a huge file never has to be held whole in memory. Both reject invalid UTF-8 and NUL-byte binary samples (`FS_NOT_TEXT`) and non-regular targets. The policy layer (`ctx.fileContext`) decides which to call by size and owns the line windowing. -- **`writeText`** — atomic: writes to a temp file opened exclusively (`wx`, `0o600`) inside a randomly-named private staging dir (`0o700`) next to the target, fsyncs, then renames over the target. An existing file's mode is preserved, while new files default to `0o600`. Honors the `FsWriteExpectation`: `createIfAbsent` creates a missing target and rejects an existing one (`FS_NOT_OBSERVED`); `replaceIfVersion` replaces only at the observed version (a missing target or mismatch is `FS_STALE_VERSION`). -- **`editText`** — atomic literal read-modify-write over the same primitive, serialized per target by a mutation lock. Verifies the expected version BEFORE literal matching (a stale edit reports `FS_STALE_VERSION`, never `FS_EDIT_NOT_FOUND`/`FS_AMBIGUOUS_EDIT` against newer content), LF-normalizes for matching, restores the file's dominant CRLF/LF style, and rejects empty `oldString` / zero matches (`FS_EDIT_NOT_FOUND`) or ambiguous multi-matches without `replace_all` (`FS_AMBIGUOUS_EDIT`). +- **`readText` / `streamText`** — UTF-8 only. `readText` reads the whole file; `streamText` streams it in chunks (cross-chunk decoding) so a huge file never has to be held whole in memory. Both reject invalid UTF-8 and NUL-byte binary samples (`FS_NOT_TEXT`) and non-regular targets. The `read` tool (`@deepseek-ai/dsh-tool-fs`) decides which to call by size and owns the line windowing. +- **`writeText`** — atomic: writes to a temp file opened exclusively (`wx`, `0o600`) inside a randomly-named private staging dir (`0o700`) next to the target, fsyncs, then renames over the target. An existing file's mode is preserved, while new files default to `0o600`. The `expected` guard is OPTIONAL: omitting it unconditionally creates-or-overwrites; `createIfAbsent` creates a missing target and rejects an existing one (`FS_NOT_OBSERVED`); `replaceIfVersion` replaces only at the observed version (a missing target or mismatch is `FS_STALE_VERSION`). +- **`editText`** — atomic literal read-modify-write over the same primitive, serialized per target by a mutation lock. The `expected` guard is OPTIONAL: when supplied it verifies the version BEFORE literal matching (a stale edit reports `FS_STALE_VERSION`, never `FS_EDIT_NOT_FOUND`/`FS_AMBIGUOUS_EDIT` against newer content); omitting it edits the current content unconditionally. A missing target reports `FS_STALE_VERSION` either way. LF-normalizes for matching, restores the file's dominant CRLF/LF style, and rejects empty `oldString` / zero matches (`FS_EDIT_NOT_FOUND`) or ambiguous multi-matches without `replace_all` (`FS_AMBIGUOUS_EDIT`). ## `cwd` is not a sandbox diff --git a/packages/fs/fs-local/src/index.ts b/packages/fs/fs-local/src/index.ts index 3c8ba61f78..9f769bc7cf 100644 --- a/packages/fs/fs-local/src/index.ts +++ b/packages/fs/fs-local/src/index.ts @@ -120,7 +120,7 @@ export class LocalFileSystem extends FileSystem { override async writeText( target: FsTarget, content: string, - expected: FsWriteExpectation, + expected?: FsWriteExpectation, signal?: AbortSignal, ): Promise { return this.withLock(target.targetKey, async () => { @@ -129,16 +129,19 @@ export class LocalFileSystem extends FileSystem { throw new FsError(`cannot write "${target.displayPath}": not a regular file`, 'FS_NOT_REGULAR_FILE') } - if (expected.kind === 'replaceIfVersion') { + if (expected?.kind === 'replaceIfVersion') { // Stale guard: the file must still exist at the version the owner observed. if (!existing) throw new FsError(`cannot write "${target.displayPath}": file no longer exists`, 'FS_STALE_VERSION') if (existing.version !== expected.version) { throw new FsError(`cannot write "${target.displayPath}": file changed since it was read`, 'FS_STALE_VERSION') } - } else if (existing) { + } else if (expected?.kind === 'createIfAbsent' && existing) { // createIfAbsent onto an existing file: a blind overwrite — require a read first. throw new FsError(`cannot overwrite existing "${target.displayPath}" without reading it first`, 'FS_NOT_OBSERVED') } + // expected === undefined: unconditional create-or-overwrite (the bare + // provider) — no version guard, no read-first requirement. Still atomic + // (the per-target lock is unconditional), so the write is never torn. await writeFileAtomic(target.targetKey, content, existing?.mode, signal, this.internals) const after = await probe(target.targetKey) @@ -152,16 +155,21 @@ export class LocalFileSystem extends FileSystem { override async editText( target: FsTarget, edit: FsEditRequest, - expected: { version: FsVersion }, + expected?: { version: FsVersion }, signal?: AbortSignal, ): Promise { return this.withLock(target.targetKey, async () => { const existing = await probe(target.targetKey) // Stale guard BEFORE literal matching: an edit based on an old read reports // FS_STALE_VERSION, not FS_EDIT_NOT_FOUND/FS_AMBIGUOUS_EDIT against newer content. + // A missing target reports FS_STALE_VERSION on BOTH paths (guarded and + // unconditional) — one "cannot edit this target now" code. if (!existing) throw new FsError(`cannot edit "${target.displayPath}": file changed since it was read`, 'FS_STALE_VERSION') if (existing.type !== 'file') throw new FsError(`cannot edit "${target.displayPath}": not a regular file`, 'FS_NOT_REGULAR_FILE') - if (existing.version !== expected.version) { + // expected === undefined: unconditional edit of the current content — no + // version guard. Still inside the per-target lock, so the read→match→write + // window is serialized and atomic. + if (expected && existing.version !== expected.version) { throw new FsError(`cannot edit "${target.displayPath}": file changed since it was read`, 'FS_STALE_VERSION') } diff --git a/packages/fs/fs-local/tests/filesystem.spec.ts b/packages/fs/fs-local/tests/filesystem.spec.ts index 20c5cfa21f..4119188f1b 100644 --- a/packages/fs/fs-local/tests/filesystem.spec.ts +++ b/packages/fs/fs-local/tests/filesystem.spec.ts @@ -144,6 +144,26 @@ describe('writeText', () => { .rejects.toMatchObject({ code: 'FS_NOT_REGULAR_FILE' }) }) + it('unconditionally creates a new file with no expectation (bare provider)', async () => { + const target = await fs.resolve('new.txt') + const outcome = await fs.writeText(target, 'fresh') + expect(outcome.operation).toBe('create') + expect(await readFile(join(dir, 'new.txt'), 'utf8')).toBe('fresh') + }) + + it('unconditionally OVERWRITES an existing file with no expectation (bare provider)', async () => { + await writeFile(join(dir, 'a.txt'), 'old') + const target = await fs.resolve('a.txt') + const outcome = await fs.writeText(target, 'clobbered') + expect(outcome.operation).toBe('update') + expect(await readFile(join(dir, 'a.txt'), 'utf8')).toBe('clobbered') + }) + + it('rejects writing onto a directory even with no expectation', async () => { + const target = await fs.resolve('.') + await expect(fs.writeText(target, 'x')).rejects.toMatchObject({ code: 'FS_NOT_REGULAR_FILE' }) + }) + it('releases per-target mutation locks after success and failure', async () => { const target = await fs.resolve('a.txt') await fs.writeText(target, 'created', { kind: 'createIfAbsent' }) @@ -173,6 +193,28 @@ describe('editText', () => { .rejects.toMatchObject({ code: 'FS_STALE_VERSION' }) }) + it('unconditionally edits the current content with no expectation (bare provider)', async () => { + await writeFile(join(dir, 'a.txt'), 'hello world') + const target = await fs.resolve('a.txt') + // No version guard: any current content is edited, regardless of version. + const outcome = await fs.editText(target, { oldString: 'world', newString: 'there', replaceAll: false }) + expect(outcome.replacements).toBe(1) + expect(await readFile(join(dir, 'a.txt'), 'utf8')).toBe('hello there') + }) + + it('reports a missing target as FS_STALE_VERSION even with no expectation (bare provider)', async () => { + const target = await fs.resolve('missing.txt') + await expect(fs.editText(target, { oldString: 'a', newString: 'b', replaceAll: false })) + .rejects.toMatchObject({ code: 'FS_STALE_VERSION' }) + }) + + it('still reports literal-match codes with no expectation (FS_EDIT_NOT_FOUND, unrelated to freshness)', async () => { + await writeFile(join(dir, 'a.txt'), 'hello world') + const target = await fs.resolve('a.txt') + await expect(fs.editText(target, { oldString: 'absent', newString: 'x', replaceAll: false })) + .rejects.toMatchObject({ code: 'FS_EDIT_NOT_FOUND' }) + }) + it('rejects a deleted target as stale (before matching)', async () => { await writeFile(join(dir, 'a.txt'), 'hello') const target = await fs.resolve('a.txt') diff --git a/packages/fs/fs/README.md b/packages/fs/fs/README.md index 1599ac25cc..fd2307dec5 100644 --- a/packages/fs/fs/README.md +++ b/packages/fs/fs/README.md @@ -1,14 +1,14 @@ # @deepseek-ai/dsh-fs -The **filesystem provider seam**: an abstract `FileSystem` service (`ctx.fs`) defining the text-storage primitives a backend provides — resolve a path, stat metadata, read/stream text, write atomically, and apply a guarded literal edit — without saying HOW. +The **filesystem provider seam**: an abstract `FileSystem` service (`ctx.fs`) defining the text-storage primitives a backend provides — resolve a path, stat metadata, read/stream text, write atomically, and apply a literal edit — without saying HOW. Both mutations take their version guard **optionally**, so `ctx.fs` on its own is a complete, unconstrained text-storage seam. This package also owns the `fs/*` policy event vocabulary the tool dispatches and the policy plugin listens for. -This package is the provider-seam layer of the four-layer filesystem stack, split so each concern can evolve (and be swapped) independently (see [the capability-seam RFC](../../../docs/rfc/implemented/architecture/2026-06-13-capability-seams.md), [the filesystem capability-seam RFC](../../../docs/rfc/implemented/architecture/2026-06-17-filesystem-capability-seam.md), and [the split-the-filesystem-seam RFC](../../../docs/rfc/implemented/simplification/2026-06-26-fsspec-style-fs-seam.md)): +This package is the provider-seam layer of the four-layer filesystem stack, split so each concern can evolve (and be swapped) independently (see [the capability-seam RFC](../../../docs/rfc/implemented/architecture/2026-06-13-capability-seams.md), [the filesystem capability-seam RFC](../../../docs/rfc/implemented/architecture/2026-06-17-filesystem-capability-seam.md), [the split-the-filesystem-seam RFC](../../../docs/rfc/implemented/simplification/2026-06-26-fsspec-style-fs-seam.md), and [the file-context event-gate RFC](../../../docs/rfc/implemented/architecture/2026-06-26-file-context-as-event-gate.md)): | Layer | Package | Role | |---|---|---| -| tool | `@deepseek-ai/dsh-tool-fs` | model-facing `read`/`write`/`edit` schemas + text rendering | -| policy | `@deepseek-ai/dsh-file-context` | `ctx.fileContext`: observed-state, read windowing, write/edit freshness | -| provider seam | `@deepseek-ai/dsh-fs` (this) | `ctx.fs`: text IO + guarded mutation primitives | +| tool / executor | `@deepseek-ai/dsh-tool-fs` | model-facing `read`/`write`/`edit` schemas + read windowing + text rendering; reads/writes/edits via `ctx.fs`, dispatches the `fs/*` events | +| policy | `@deepseek-ai/dsh-file-context` | observed-state + read-before-edit + version-guarded write/edit, contributed through the `fs/*` event gate (no service) | +| provider seam | `@deepseek-ai/dsh-fs` (this) | `ctx.fs`: text IO + atomic mutation primitives (optional version guard); owns the `fs/*` event vocabulary | | provider | `@deepseek-ai/dsh-fs-local` | the host-filesystem implementation | A future sandboxed, virtual, or remote backend implements this interface and the policy/tool layers don't change. @@ -23,15 +23,22 @@ A backend subclasses `FileSystem` and implements six primitives. | `stat(target, signal?)` | Return `FsInfo` metadata (`version`, `type`, optional `size`), or `undefined` when the target is absent. Never content. | | `readText(target, signal?)` | Read the whole regular text file as one decoded string. Owns regular-file checks, UTF-8 decoding, binary/NUL rejection (`FS_NOT_TEXT`). | | `streamText(target, signal?)` | Stream the same text as decoded chunks for large files (cross-chunk UTF-8 decoding stays here). | -| `writeText(target, content, expected, signal?)` | Atomic create/replace honoring the `FsWriteExpectation` (`createIfAbsent` or `replaceIfVersion`). | -| `editText(target, edit, expected, signal?)` | Version-guarded literal edit. Verifies `expected.version` BEFORE matching, then applies the replacement and writes atomically — one mutation critical section. | +| `writeText(target, content, expected?, signal?)` | Atomic create/replace. `expected` is OPTIONAL: omit ⇒ unconditional create-or-overwrite; supply an `FsWriteExpectation` (`createIfAbsent`/`replaceIfVersion`) to guard. | +| `editText(target, edit, expected?, signal?)` | Literal edit. `expected` is OPTIONAL: omit ⇒ unconditional edit of the current content; supply `{ version }` to guard (verified BEFORE matching). A missing target reports `FS_STALE_VERSION` either way. Applies and writes atomically — one mutation critical section. | + +The mutation runs inside the backend's per-target lock either way, so an unconditional write/edit is still atomic — "unconditional" drops the *version* precondition, not the atomicity. + +## The `fs/*` policy events + +This package declares three events (see the generated [catalog](../../../docs/cordis-catalog/events-and-services.md)) so the emitter (`@deepseek-ai/dsh-tool-fs`) and the policy listener (`@deepseek-ai/dsh-file-context`) share a vocabulary without the emitter depending on the policy plugin. `fs/write-expectation` and `fs/edit-expectation` are single-slot decision waterfalls (the listener fully decides, never calling `next()`); `fs/observed` is a fire-and-forget recording event. They carry only `dsh-fs` vocabulary plus an opaque `object` actor — no model-facing concepts and no agent/session owner structure. ## A provider seam, not the policy layer -`ctx.fs` is deliberately close to fsspec-style storage primitives — half a level above byte-level `cat`/`open`, because it decodes text and rejects binaries so the policy layer never touches raw bytes. It owns UTF-8 decoding, binary rejection, atomic writes, and the version-guarded literal-edit critical section. It does **not** own line windows, numbered lines, rendered footers, or observed-state — those model-facing read-windowing and read-before-write/edit policies live one layer up in `ctx.fileContext` ([`@deepseek-ai/dsh-file-context`](../file-context)), so a sandboxed/remote backend inherits no model-facing observation policy. +`ctx.fs` is deliberately close to fsspec-style storage primitives — half a level above byte-level `cat`/`open`, because it decodes text and rejects binaries so the policy layer never touches raw bytes. It owns UTF-8 decoding, binary rejection, atomic writes, and the literal-edit critical section. It does **not** own line windows, numbered lines, rendered footers, or observed-state. Observed-state, read-before-edit, and version-guarded write/edit are policy a plugin (`@deepseek-ai/dsh-file-context`) ADDS by supplying the optional guard — not provider behavior — so a sandboxed/remote backend inherits no model-facing observation policy. `editText` stays on this seam (not composed in the policy layer from a read plus a write) because version guard + literal match + atomic rewrite must stay inside one critical section for correct error attribution and one-wins/one-stale concurrency, and a remote backend may implement it as a native compare-and-edit. ## Vocabulary -`FsTargetKey` / `FsVersion` are branded opaque ids ([the branded-ids RFC](../../../docs/rfc/implemented/architecture/2026-06-20-branded-ids.md)) — consumers must not parse `targetKey` or interpret `version`; only `displayPath` is for model/UI output. `FsWriteExpectation` is the explicit write intent (`createIfAbsent` creates a missing target and rejects an existing one with `FS_NOT_OBSERVED`; `replaceIfVersion` replaces only at the observed version, else `FS_STALE_VERSION`). Failures throw `FsError` (extends `HarnessError`, [the structured error taxonomy RFC](../../../docs/rfc/implemented/architecture/2026-06-11-structured-error-taxonomy.md)) carrying a stable `FsErrorCode` (`FS_NOT_FOUND`, `FS_NOT_TEXT`, `FS_NOT_REGULAR_FILE`, `FS_STALE_VERSION`, `FS_NOT_OBSERVED`, `FS_AMBIGUOUS_EDIT`, `FS_EDIT_NOT_FOUND`, `FS_ABORTED`); the tool registry surfaces `{ name, code }` on `isError` results. See `src/types.ts` for the full contracts. +`FsTargetKey` / `FsVersion` are branded opaque ids ([the branded-ids RFC](../../../docs/rfc/implemented/architecture/2026-06-20-branded-ids.md)) — consumers must not parse `targetKey` or interpret `version`; only `displayPath` is for model/UI output. `FsWriteExpectation` is the explicit GUARDED write intent (`createIfAbsent` creates a missing target and rejects an existing one with `FS_NOT_OBSERVED`; `replaceIfVersion` replaces only at the observed version, else `FS_STALE_VERSION`); omitting it from `writeText` is the third, unconditional state. Failures throw `FsError` (extends `HarnessError`, [the structured error taxonomy RFC](../../../docs/rfc/implemented/architecture/2026-06-11-structured-error-taxonomy.md)) carrying a stable `FsErrorCode` (`FS_NOT_FOUND`, `FS_NOT_TEXT`, `FS_NOT_REGULAR_FILE`, `FS_STALE_VERSION`, `FS_NOT_OBSERVED`, `FS_AMBIGUOUS_EDIT`, `FS_EDIT_NOT_FOUND`, `FS_ABORTED`); the tool registry surfaces `{ name, code }` on `isError` results. See `src/types.ts` for the full contracts. + diff --git a/packages/fs/fs/src/index.ts b/packages/fs/fs/src/index.ts index 70675b01c2..f23eae98fb 100644 --- a/packages/fs/fs/src/index.ts +++ b/packages/fs/fs/src/index.ts @@ -17,12 +17,12 @@ * * `ctx.fs` is deliberately close to fsspec-style storage primitives. It owns * UTF-8 decoding, binary/NUL rejection, atomic full-file writes, and the - * version-guarded literal-edit critical section — but NOT line windows, - * numbered lines, rendered footers, or observed-state. Those model-facing - * read-windowing and read-before-write/edit policies live one layer up in the - * concrete `ctx.fileContext` service (`@deepseek-ai/dsh-file-context`), so a - * sandboxed/remote backend inherits no model-facing observation policy it has - * no business carrying. + * literal-edit critical section — but NOT line windows, numbered lines, + * rendered footers, or observed-state. Read windowing lives in the model-facing + * tool (`@deepseek-ai/dsh-tool-fs`); observed-state and read-before-write/edit + * are policy a plugin (`@deepseek-ai/dsh-file-context`) adds through the `fs/*` + * event gate. So a sandboxed/remote backend inherits no model-facing observation + * policy it has no business carrying. * * `editText` stays on this seam (not composed in the policy layer from a read * plus a write) because version guard + literal match + atomic rewrite must @@ -30,6 +30,30 @@ * one-wins/one-stale concurrency, and a remote backend may implement it as a * native compare-and-edit. * + * ## The version guard is OPTIONAL — additive policy, not subtractive + * + * `ctx.fs` on its own is a complete, unconstrained text-storage seam: `read` + * reads, `write` unconditionally creates-or-overwrites, `edit` unconditionally + * replaces literal text in the current content. Both mutations take their + * version guard as an OPTIONAL argument — omit it for the unconstrained + * bare-provider behavior, supply it to guard against a concurrent change. The + * mutation runs inside the backend's per-target lock either way, so an + * unconditional write/edit is still atomic; "unconditional" drops the *version* + * precondition, not the atomicity. Observed-state, read-before-edit, and + * version-guarded write/edit are NOT provider behavior — they are policy a + * plugin (`@deepseek-ai/dsh-file-context`) adds on top by supplying the guard. + * + * ## The fs policy events live here, not in the policy plugin + * + * This package owns the `fs/write-expectation`, `fs/edit-expectation`, and + * `fs/observed` event vocabulary (see {@link Events}). The emitter is + * `@deepseek-ai/dsh-tool-fs` and the default listener is + * `@deepseek-ai/dsh-file-context`; the events live in the one package both + * already depend on, so the emitter shares a vocabulary with the policy listener + * without depending on the policy plugin. The events carry only `dsh-fs` + * vocabulary plus an opaque `object` actor — no model-facing concepts (line + * windows, numbered lines) and no agent/session owner structure leak down. + * * @module @deepseek-ai/dsh-fs */ @@ -63,6 +87,47 @@ declare module 'cordis' { interface Context { fs: FileSystem } + + interface Events { + /** + * Single-slot decision: produce the write expectation for the next + * {@link FileSystem.writeText}. The tool dispatches this as an unbound + * waterfall (no `this`) and supplies a default thunk returning `undefined` + * (unconditional create-or-overwrite — the bare provider). The + * `@deepseek-ai/dsh-file-context` policy listener returns `createIfAbsent` + * (unobserved actor) or `{ kind: 'replaceIfVersion', version: vObserved }` + * (observed) and does NOT call `next()` — one decision, not a composable + * chain. The slot is first-wins: the first non-`next()` decider (registration + * order, or `prepend`) occupies it; a second decider is a misconfiguration, + * not layering. `actor` is the opaque tool-execution context, never read here. + * @mode waterfall + */ + 'fs/write-expectation'(target: FsTarget, actor: object | undefined, next: () => FsWriteExpectation | undefined | Promise): Promise + /** + * Single-slot decision: produce the optional version guard for the next + * {@link FileSystem.editText}. The tool dispatches this as an unbound + * waterfall and supplies a default thunk returning `undefined` (unconditional + * edit of the current content — the bare provider; no `stat`). The + * `@deepseek-ai/dsh-file-context` policy listener returns + * `{ version: vObserved }`, or throws `FS_NOT_OBSERVED` if the actor is unset + * or has not observed the target. Does NOT call `next()`: one decision, + * first-wins (see {@link Events.'fs/write-expectation'}). + * @mode waterfall + */ + 'fs/edit-expectation'(target: FsTarget, actor: object | undefined, next: () => { version: FsVersion } | undefined | Promise<{ version: FsVersion } | undefined>): Promise<{ version: FsVersion } | undefined> + /** + * Record that an actor observed a target at a version, after a successful + * read/write/edit. Fire-and-forget. A listener MUST be a synchronous, + * side-effect-only recorder (`@deepseek-ai/dsh-file-context`'s is a + * `WeakMap.set`); the tool wraps the emit in a try/catch so a synchronous + * listener bug is logged and swallowed, never failing the already-completed + * mutation. cordis `emit` does not await listener promises, so this is not an + * async-error containment seam — async audit/telemetry does not belong here. + * No listener ⇒ nothing recorded. `actor` is the opaque tool-execution context. + * @mode emit + */ + 'fs/observed'(target: FsTarget, version: FsVersion, actor: object | undefined): void + } } /** @@ -80,12 +145,15 @@ declare module 'cordis' { * - {@link readText}/{@link streamText} read the whole regular text file (the * stream for large files); both own regular-file checks, UTF-8 decoding, * binary/NUL rejection, and `FS_NOT_TEXT`. - * - {@link writeText} is atomic temp-file + rename honoring the - * {@link FsWriteExpectation}. + * - {@link writeText} is atomic temp-file + rename. `expected` is OPTIONAL: + * omit it for an unconditional create-or-overwrite (the bare-provider default), + * or supply a {@link FsWriteExpectation} to guard the write. * - {@link editText} verifies `expected.version` BEFORE literal matching (so a * stale edit reports `FS_STALE_VERSION`, not `FS_EDIT_NOT_FOUND`/ * `FS_AMBIGUOUS_EDIT` against newer content), then applies literal replacement - * and writes atomically — all inside one mutation critical section. + * and writes atomically — all inside one mutation critical section. `expected` + * is OPTIONAL: omit it for an unconditional edit of the current content (a + * missing target still reports `FS_STALE_VERSION`). */ export abstract class FileSystem extends Service { constructor(ctx: Context) { @@ -115,17 +183,21 @@ export abstract class FileSystem extends Service { abstract streamText(target: FsTarget, signal?: AbortSignal): Promise> /** - * Create or fully replace a UTF-8 text file atomically, honoring `expected` - * as the create-vs-replace decision and stale guard. + * Create or fully replace a UTF-8 text file atomically. `expected` is the + * create-vs-replace decision and stale guard when supplied; OMITTING it is an + * unconditional create-or-overwrite (the bare provider — no version guard, no + * read-first requirement). Atomic either way. */ - abstract writeText(target: FsTarget, content: string, expected: FsWriteExpectation, signal?: AbortSignal): Promise + abstract writeText(target: FsTarget, content: string, expected?: FsWriteExpectation, signal?: AbortSignal): Promise /** - * Apply a literal edit to an existing UTF-8 text file. Verifies - * `expected.version` as the stale guard BEFORE literal matching, then applies - * the replacement and writes atomically — one mutation critical section. + * Apply a literal edit to an existing UTF-8 text file. When `expected` is + * supplied, verifies `expected.version` as the stale guard BEFORE literal + * matching; OMITTING it edits the current content unconditionally (no version + * guard). Either way applies the replacement and writes atomically — one + * mutation critical section — and a missing target reports `FS_STALE_VERSION`. */ - abstract editText(target: FsTarget, edit: FsEditRequest, expected: { version: FsVersion }, signal?: AbortSignal): Promise + abstract editText(target: FsTarget, edit: FsEditRequest, expected?: { version: FsVersion }, signal?: AbortSignal): Promise } export default FileSystem diff --git a/packages/fs/fs/src/types.ts b/packages/fs/fs/src/types.ts index 62ba52b52f..258c7a1e8b 100644 --- a/packages/fs/fs/src/types.ts +++ b/packages/fs/fs/src/types.ts @@ -13,7 +13,8 @@ * consumer may show. * * Model-facing concepts (line windows, numbered lines, observed-state) do NOT - * live here; they belong to the policy layer (`ctx.fileContext`). + * live here; they belong to the consumer tool and the policy plugin + * (`@deepseek-ai/dsh-tool-fs` / `@deepseek-ai/dsh-file-context`). * * @module @deepseek-ai/dsh-fs/types */ @@ -78,11 +79,17 @@ export interface FsInfo { } /** - * The explicit intent of a {@link FileSystem.writeText} call. `createIfAbsent` - * creates a missing target and rejects an existing one with `FS_NOT_OBSERVED` - * (the path used when the owner has no prior read). `replaceIfVersion` replaces - * only when the target exists at the observed version; a missing target or a - * version mismatch throws `FS_STALE_VERSION`. + * The explicit intent of a guarded {@link FileSystem.writeText} call. + * `createIfAbsent` creates a missing target and rejects an existing one with + * `FS_NOT_OBSERVED` (the path the policy plugin uses when the owner has no prior + * read). `replaceIfVersion` replaces only when the target exists at the observed + * version; a missing target or a version mismatch throws `FS_STALE_VERSION`. + * + * `writeText` takes this OPTIONALLY: omitting `expected` is the third, + * unconstrained state — an unconditional create-or-overwrite (the bare + * provider). The union itself carries only the two GUARDED intents; "no guard" + * is expressed by omission, so the write and edit mutations share one symmetric + * shape (`expected?`: omit = unconditional, present = guarded). */ export type FsWriteExpectation = | { kind: 'createIfAbsent' } diff --git a/packages/fs/fs/tests/service.spec.ts b/packages/fs/fs/tests/service.spec.ts index e06cfa9ee6..7091746dc3 100644 --- a/packages/fs/fs/tests/service.spec.ts +++ b/packages/fs/fs/tests/service.spec.ts @@ -38,7 +38,7 @@ class FakeFileSystem extends FileSystem { const content = await this.readText(target) return (async function* () { yield content })() } - override async writeText(target: FsTarget, content: string, _expected: FsWriteExpectation): Promise { + override async writeText(target: FsTarget, content: string, _expected?: FsWriteExpectation): Promise { const existed = this.files.has(target.targetKey) this.files.set(target.targetKey, content) return { operation: existed ? 'update' : 'create', version: FsVersion('v2') } diff --git a/packages/fs/tool-fs/README.md b/packages/fs/tool-fs/README.md index f751ecb051..e3e8a1cdff 100644 --- a/packages/fs/tool-fs/README.md +++ b/packages/fs/tool-fs/README.md @@ -1,15 +1,17 @@ # @deepseek-ai/dsh-tool-fs -The **model-facing filesystem tools** — `read`, `write`, `edit` — over the `ctx.fileContext` policy layer ([`@deepseek-ai/dsh-file-context`](../file-context)). This is the consumer layer of the filesystem stack; it owns tool names, JSON schemas, argument validation, prompt sections, and result formatting, and **never** touches filesystem I/O (no `node:fs`/`node:path`, no implementation import) or reaches around the policy layer to `ctx.fs`. +The **model-facing filesystem tools** — `read`, `write`, `edit` — and their **executor**. This is the consumer layer of the filesystem stack: it owns tool names, JSON schemas, argument validation, prompt sections, **read windowing**, and result formatting. It reads/writes/edits through the `ctx.fs` provider seam ([`@deepseek-ai/dsh-fs`](../fs)) **directly** — it injects `fs` (plus `tools`/`systemPrompt`), **not** a policy service. The freshness/observation policy is contributed by a separate plugin ([`@deepseek-ai/dsh-file-context`](../file-context)) through the `fs/*` event gate; the tool is not method-coupled to it. ```ts ignore-check -// Load a ctx.fs provider, the policy layer, then the tools. +// Default deployment: a ctx.fs provider, the policy plugin, then the tools. await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) // @deepseek-ai/dsh-fs-local -await ctx.plugin(FileContext) // @deepseek-ai/dsh-file-context +await ctx.plugin(FileContext) // @deepseek-ai/dsh-file-context (policy gate) await ctx.plugin(ToolFs) // this package — registers read/write/edit ``` -Each tool also ships as a subpath plugin for focused deployments: +`@deepseek-ai/dsh-file-context` is **optional**: omit it and the tools run against the bare provider (unconditional write/overwrite/edit, no observed-state). The default product config loads it, so the default behavior stays read-before-write/edit. + +Each tool also ships as a subpath plugin for focused deployments (each injects `fs`, not a policy service): ```ts ignore-check import * as readPlugin from '@deepseek-ai/dsh-tool-fs/read' @@ -22,17 +24,23 @@ import * as editPlugin from '@deepseek-ai/dsh-tool-fs/edit' | Tool | Arguments | Behavior | |---|---|---| | `read` | `file_path`, `offset?`, `limit?` | Line-numbered UTF-8 content with a pagination footer. `offset` is 1-based; `limit` defaults to and caps at 2000 lines. | -| `write` | `file_path`, `content` | Create or fully replace a file. Overwriting an existing file requires a prior `read` at the unchanged version; creating a new file does not. | -| `edit` | `file_path`, non-empty `old_string`, `new_string`, `replace_all?` | Literal replacement; unique match required unless `replace_all` is true. Requires a prior `read` (any window) and the file unchanged since. | +| `write` | `file_path`, `content` | Create or fully replace a file. With the policy plugin: overwriting an existing file requires a prior `read` at the unchanged version; creating a new file does not. Without it: unconditional. | +| `edit` | `file_path`, non-empty `old_string`, `new_string`, `replace_all?` | Literal replacement; unique match required unless `replace_all` is true. With the policy plugin: requires a prior `read` (any window) and the file unchanged since. Without it: unconditional. | Field names are snake_case to match Claude Code and existing harness tool schemas. -## How the read-before-write/edit policy is enforced +## The tool is the executor; policy is an event gate -The tools do **not** check whether a `read` ran or inspect any cache. Each tool resolves the path via `ctx.fileContext.resolve()`, then calls `ctx.fileContext.read/write/edit(target, …, exec)` — passing the current tool execution context straight through. `ctx.fileContext` derives the observed-state owner (normally the agent session) from that context and owns the freshness policy: a recorded read at the file's current version authorizes a write/edit, and any windowed read counts (authorization is freshness, not a full-view requirement). Backend errors (`FsError`) flow through `ToolRegistry.execute()` and become `isError` tool results with their `{ name, code }` attached. +The tools do **not** inject a policy service or inspect any cache. Each tool resolves the path via `ctx.fs.resolve()`, then: -## The no-bypass contract +- **read** — one `ctx.fs.stat` (type + size routing + version), then `readText`/`streamText`, then builds the line window, then emits a contained `fs/observed`. (1 stat.) +- **write** — `ctx.waterfall('fs/write-expectation', target, exec, () => undefined)` for the optional guard, then `ctx.fs.writeText(target, content, expectation)`, then `fs/observed`. (0 stat.) +- **edit** — `ctx.waterfall('fs/edit-expectation', target, exec, () => undefined)` for the optional guard, then `ctx.fs.editText(target, edit, expectation)`, then `fs/observed`. (0 stat.) -A model-facing read MUST go through `ctx.fileContext.read`, never `ctx.fs.readText`/`streamText`, so every successful read records observed-state before rendering — which is why the tools inject `fileContext`, not `fs`. Direct `ctx.fs` calls remain an explicit escape hatch for non-tool consumers: a direct `ctx.fs.readText` records nothing, so a later `edit` rejects with `FS_NOT_OBSERVED` until the file is read through `ctx.fileContext`. +The tool passes `exec` (the tool-execution context) as the opaque `actor` on every dispatch. The default thunks return `undefined` (the unconstrained bare provider). When `@deepseek-ai/dsh-file-context` is loaded it occupies the single decision slot — returning `createIfAbsent`/`replaceIfVersion`/`{ version }` or throwing `FS_NOT_OBSERVED` — and records on `fs/observed`. Backend errors (`FsError`) and a thrown `FS_NOT_OBSERVED` flow through `ToolRegistry.execute()` and become `isError` tool results with their `{ name, code }` attached. -Tool schemas reach the system prompt automatically via the tool registry; this package additionally registers short prose guidance through `ctx.systemPrompt.section(...)`. +## `fs/observed` never fails the tool + +`fs/observed` fires AFTER the read/write/edit already succeeded, so the tool wraps the emit in a try/catch (`src/observe.ts`) that logs and swallows a synchronous listener bug — otherwise a recording failure would turn a completed mutation into an `isError`. The event contract requires synchronous, side-effect-only listeners; this is the synchronous backstop, not async-error handling. + +The line-windowing mechanics live in `src/window.ts` (Cordis-free, independently unit-tested); `src/read.ts`/`write.ts`/`edit.ts` are the tool executors and `src/index.ts` composes them. diff --git a/packages/fs/tool-fs/package.json b/packages/fs/tool-fs/package.json index f158142fca..801712bf1c 100644 --- a/packages/fs/tool-fs/package.json +++ b/packages/fs/tool-fs/package.json @@ -32,7 +32,6 @@ ], "license": "BSD-3-Clause", "peerDependencies": { - "@deepseek-ai/dsh-file-context": "^0.0.1", "@deepseek-ai/dsh-fs": "^0.0.1", "@deepseek-ai/dsh-llm": "^0.0.1", "@deepseek-ai/dsh-system-prompt": "^0.0.1", diff --git a/packages/fs/tool-fs/src/edit.ts b/packages/fs/tool-fs/src/edit.ts index d54fe3045b..5d70310502 100644 --- a/packages/fs/tool-fs/src/edit.ts +++ b/packages/fs/tool-fs/src/edit.ts @@ -1,8 +1,14 @@ /** * The model-facing `edit` tool: update an existing UTF-8 text file by replacing - * literal text, requiring a unique match by default. Execution goes through - * `ctx.fileContext`, which enforces prior observation (the freshness policy) - * and delegates the literal-match + stale-guard critical section to `ctx.fs`. + * literal text, requiring a unique match by default. The tool is the executor: + * it dispatches the `fs/edit-expectation` waterfall to obtain the optional + * version guard, calls `ctx.fs.editText` directly, and emits a contained + * `fs/observed`. The default thunk returns `undefined` (unconditional edit of + * the current content — the bare provider); a policy plugin + * (`@deepseek-ai/dsh-file-context`) occupies the single decision slot, returning + * `{ version: vObserved }` or throwing `FS_NOT_OBSERVED` for an unread file. The + * tool stats ZERO times either way; a missing target is reported by the provider + * as `FS_STALE_VERSION`. * * @module @deepseek-ai/dsh-tool-fs/edit */ @@ -11,7 +17,9 @@ import type { Context } from 'cordis' import { defineTool } from '@deepseek-ai/dsh-tools' import type { ContentBlock } from '@deepseek-ai/dsh-llm' import type { FsEditOutcome } from '@deepseek-ai/dsh-fs' +import type {} from '@deepseek-ai/dsh-fs' import type {} from '@deepseek-ai/dsh-system-prompt' +import { emitObserved } from './observe.ts' /** Validated `edit` arguments after defaulting. */ interface EditInput { @@ -60,13 +68,18 @@ export function apply(ctx: Context): void { }, async execute(args, exec): Promise { const input = parseEditArgs(args) - const target = await ctx.fileContext.resolve(input.filePath) - const outcome = await ctx.fileContext.edit( + const target = await ctx.fs.resolve(input.filePath) + // Single-slot decision: the policy plugin returns { version: vObserved } or + // throws FS_NOT_OBSERVED; the bare default is undefined (unconditional edit). + // No stat — the bare default never manufactures a version basis. + const expectation = await ctx.waterfall('fs/edit-expectation', target, exec, () => undefined) + const outcome = await ctx.fs.editText( target, { oldString: input.oldString, newString: input.newString, replaceAll: input.replaceAll }, - exec, + expectation, exec.signal, ) + emitObserved(ctx, target, outcome.version, exec) return [{ type: 'text', text: formatEditOutput(target.displayPath, outcome) }] }, })) @@ -76,7 +89,7 @@ export function apply(ctx: Context): void { export const name = 'fs-edit' /** Services required by the `edit` tool plugin. */ -export const inject = ['tools', 'fileContext', 'systemPrompt'] +export const inject = ['tools', 'fs', 'systemPrompt'] /** Named helper for direct registration in the root plugin and tests. */ export const applyEditTool = apply diff --git a/packages/fs/tool-fs/src/index.ts b/packages/fs/tool-fs/src/index.ts index 5509c7980b..b57810185e 100644 --- a/packages/fs/tool-fs/src/index.ts +++ b/packages/fs/tool-fs/src/index.ts @@ -1,15 +1,24 @@ /** * The model-facing filesystem tool suite (`read`, `write`, `edit`) over the - * `ctx.fileContext` policy layer. This root plugin registers all three tools by + * `ctx.fs` provider seam. This root plugin registers all three tools by * composing the per-tool registration helpers; each tool is also exposed as a * subpath plugin (`@deepseek-ai/dsh-tool-fs/read`, `/write`, `/edit`) for focused * deployments. * - * The package owns model-facing concerns only — tool names, JSON schemas, - * argument validation, prompt sections, result formatting. All filesystem - * execution goes through `ctx.fileContext` (never directly around it to - * `ctx.fs`), so every model read records observed-state before rendering; this - * package never imports `node:fs`, `node:path`, or an + * ## The tool is the executor; policy is an event gate + * + * The tool reads/writes/edits through `ctx.fs` DIRECTLY and owns model-facing + * concerns only — tool names, JSON schemas, argument validation, prompt + * sections, read windowing, result formatting. It does NOT inject a policy + * service. Instead, on each write/edit it dispatches a single-slot waterfall + * (`fs/write-expectation`/`fs/edit-expectation`) to obtain the OPTIONAL version + * guard, and after every read/write/edit it emits a contained `fs/observed`. A + * policy plugin (`@deepseek-ai/dsh-file-context`, loaded by the default product + * config) occupies the decision slot and listens for `fs/observed` to add + * observed-state + read-before-edit + version-guarded write/edit. With no policy + * plugin the waterfalls fall through to their `undefined` default (the + * unconstrained bare provider) and `fs/observed` is unheard — the tool still + * functions. This package never imports `node:fs`, `node:path`, or an * `@deepseek-ai/dsh-fs-local` implementation. * * @module @deepseek-ai/dsh-tool-fs @@ -20,15 +29,19 @@ import { applyReadTool } from './read.ts' import { applyWriteTool } from './write.ts' import { applyEditTool } from './edit.ts' -export { READ_LIMIT, applyReadTool, formatReadOutput, parseReadArgs } from './read.ts' +export { READ_LIMIT, STREAM_MIN_SIZE, applyReadTool, formatReadOutput, parseReadArgs } from './read.ts' export { applyWriteTool, formatWriteOutput, parseWriteArgs } from './write.ts' export { applyEditTool, formatEditOutput, parseEditArgs } from './edit.ts' +export { emitObserved } from './observe.ts' +export type { FileTextLine, ReadWindow, WindowResult } from './window.ts' +export { READ_MAX_BYTES, READ_MAX_LINE_LENGTH, buildWindow } from './window.ts' +export type { FileReadOutcome } from './types.ts' /** Cordis plugin name used by loader diagnostics. */ export const name = 'tool-fs' /** Services required by the filesystem tool suite. */ -export const inject = ['tools', 'fileContext', 'systemPrompt'] +export const inject = ['tools', 'fs', 'systemPrompt'] /** Register the full `read`/`write`/`edit` filesystem tool suite. */ export function apply(ctx: Context): void { diff --git a/packages/fs/tool-fs/src/observe.ts b/packages/fs/tool-fs/src/observe.ts new file mode 100644 index 0000000000..407dc66fbb --- /dev/null +++ b/packages/fs/tool-fs/src/observe.ts @@ -0,0 +1,34 @@ +/** + * The contained `fs/observed` emit shared by the `read`/`write`/`edit` tools. + * + * `fs/observed` fires AFTER a mutation/read already succeeded, so a throwing + * listener must never turn the completed operation into an `isError` result + * (the tool registry catches a tool throw into an error result). The event + * contract requires a synchronous, side-effect-only listener (the policy + * plugin's is a `WeakMap.set`); this try/catch is the synchronous backstop — + * it logs and swallows a listener bug, mirroring the fire-and-forget pattern in + * the agent loop. It is NOT async-error containment: cordis `emit` does not + * await listener promises, so async observation does not belong on this event. + * + * @module @deepseek-ai/dsh-tool-fs/observe + */ + +import type { Context } from 'cordis' +import type { FsTarget, FsVersion } from '@deepseek-ai/dsh-fs' +import type {} from '@deepseek-ai/dsh-fs' + +/** + * Emit `fs/observed` for a just-completed read/write/edit, containing any + * synchronous listener throw so the already-successful operation still reports + * success. + */ +export function emitObserved(ctx: Context, target: FsTarget, version: FsVersion, actor: object | undefined): void { + try { + ctx.emit('fs/observed', target, version, actor) + } catch (error: unknown) { + // Contained: the read/write/edit already succeeded. An `fs/observed` listener + // MUST be synchronous and side-effect-only; a synchronous bug is logged and + // swallowed so a recording failure never fails the completed operation. + ctx.logger.warn(`fs/observed listener threw for "${target.displayPath}": ${String(error)}`) + } +} diff --git a/packages/fs/tool-fs/src/read.ts b/packages/fs/tool-fs/src/read.ts index 8a6ae8d609..bc12068553 100644 --- a/packages/fs/tool-fs/src/read.ts +++ b/packages/fs/tool-fs/src/read.ts @@ -1,9 +1,12 @@ /** * The model-facing `read` tool: inspect a UTF-8 text file and return - * line-numbered content with pagination guidance. Execution goes through - * `ctx.fileContext` (which records observed state and owns read windowing) — - * this module owns only the model-facing schema, argument validation, and - * result formatting, never filesystem I/O. + * line-numbered content with pagination guidance. The tool is the executor — it + * stats and reads through `ctx.fs` directly, builds the line window + * ({@link module:@deepseek-ai/dsh-tool-fs/window}), and emits a contained + * `fs/observed` so a policy plugin (`@deepseek-ai/dsh-file-context`) can record + * the read. With no policy plugin the emit is simply unheard. This module owns + * the model-facing schema, argument validation, read windowing, and result + * formatting; the freshness/observation policy is not its concern. * * @module @deepseek-ai/dsh-tool-fs/read */ @@ -11,12 +14,19 @@ import type { Context } from 'cordis' import { defineTool } from '@deepseek-ai/dsh-tools' import type { ContentBlock } from '@deepseek-ai/dsh-llm' -import type { FileReadOutcome } from '@deepseek-ai/dsh-file-context' +import { FsError } from '@deepseek-ai/dsh-fs' +import type {} from '@deepseek-ai/dsh-fs' import type {} from '@deepseek-ai/dsh-system-prompt' +import { buildWindow } from './window.ts' +import { emitObserved } from './observe.ts' +import type { FileReadOutcome } from './types.ts' /** Default and maximum number of lines returned by one `read` call. */ export const READ_LIMIT = 2000 +/** Files at or above this size stream; smaller files read whole into memory. */ +export const STREAM_MIN_SIZE = 10 * 1024 * 1024 + /** Validated `read` arguments after defaulting. */ interface ReadInput { filePath: string @@ -79,8 +89,32 @@ export function apply(ctx: Context): void { }, async execute(args, exec): Promise { const input = parseReadArgs(args) - const target = await ctx.fileContext.resolve(input.filePath) - const outcome = await ctx.fileContext.read(target, { offset: input.offset, limit: input.limit }, exec, exec.signal) + const target = await ctx.fs.resolve(input.filePath) + + // One stat: type check + size routing + the version recorded as observed. + // A writer racing between this stat and the read can at worst make a LATER + // guarded edit spuriously FS_STALE_VERSION (fail-closed: re-read; editText + // re-checks the version in its lock). + const info = await ctx.fs.stat(target, exec.signal) + if (!info) throw new FsError(`cannot read "${target.displayPath}": not found`, 'FS_NOT_FOUND') + if (info.type !== 'file') throw new FsError(`cannot read "${target.displayPath}": not a regular file`, 'FS_NOT_REGULAR_FILE') + + // Stream when the file is large OR size is unknown, so a size-less backend + // never buffers an arbitrarily large file. + const chunks = info.size === undefined || info.size >= STREAM_MIN_SIZE + ? await ctx.fs.streamText(target, exec.signal) + : [await ctx.fs.readText(target, exec.signal)] + const window = await buildWindow(chunks, { offset: input.offset, limit: input.limit }, target.displayPath) + + const outcome: FileReadOutcome = { + offset: input.offset, + limit: input.limit, + lines: window.lines, + totalLines: window.totalLines, + version: info.version, + ...window.truncatedByBytes ? { truncatedByBytes: true } : {}, + } + emitObserved(ctx, target, info.version, exec) return [{ type: 'text', text: formatReadOutput(target.displayPath, outcome) }] }, })) @@ -90,7 +124,7 @@ export function apply(ctx: Context): void { export const name = 'fs-read' /** Services required by the `read` tool plugin. */ -export const inject = ['tools', 'fileContext', 'systemPrompt'] +export const inject = ['tools', 'fs', 'systemPrompt'] /** Named helper for direct registration in the root plugin and tests. */ export const applyReadTool = apply diff --git a/packages/fs/tool-fs/src/types.ts b/packages/fs/tool-fs/src/types.ts new file mode 100644 index 0000000000..48a0592abb --- /dev/null +++ b/packages/fs/tool-fs/src/types.ts @@ -0,0 +1,32 @@ +/** + * Vocabulary for the model-facing filesystem tools (`@deepseek-ai/dsh-tool-fs`): + * the structured read outcome the `read` tool renders. The read window + * (`offset`/`limit`) and per-line shape live in + * {@link module:@deepseek-ai/dsh-tool-fs/window}; this file owns the assembled + * outcome the tool formats. + * + * The provider vocabulary (`FsTarget`, `FsVersion`, write/edit shapes) is + * re-used from `@deepseek-ai/dsh-fs` — this package owns only the model-facing + * read-rendering shape on top of it. + * + * @module @deepseek-ai/dsh-tool-fs/types + */ + +import type { FsVersion } from '@deepseek-ai/dsh-fs' +import type { FileTextLine } from './window.ts' + +/** Outcome of a bounded text read — what the model-facing `read` tool renders. */ +export interface FileReadOutcome { + /** 1-based first line requested. */ + offset: number + /** Maximum number of lines requested. */ + limit: number + /** Returned lines, already numbered. */ + lines: FileTextLine[] + /** Total line count in the file, unless `truncatedByBytes` stopped scanning early. */ + totalLines: number + /** Whether selected output hit the byte cap before EOF or the requested limit. */ + truncatedByBytes?: true + /** Opaque version of the file at read time. */ + version: FsVersion +} diff --git a/packages/fs/file-context/src/window.ts b/packages/fs/tool-fs/src/window.ts similarity index 92% rename from packages/fs/file-context/src/window.ts rename to packages/fs/tool-fs/src/window.ts index 97e51e2ee4..fb33907710 100644 --- a/packages/fs/file-context/src/window.ts +++ b/packages/fs/tool-fs/src/window.ts @@ -1,16 +1,16 @@ /** - * Cordis-free line-windowing for `@deepseek-ai/dsh-file-context`. Relocated - * from the local backend: turning a file's decoded text into a bounded, - * line-numbered window (offset/limit, byte cap, per-line truncation) is - * model-facing READ POLICY, not a storage primitive, so it lives in the policy - * layer rather than in every `ctx.fs` backend. + * Cordis-free line-windowing for `@deepseek-ai/dsh-tool-fs`. Turning a file's + * decoded text into a bounded, line-numbered window (offset/limit, byte cap, + * per-line truncation) is the model-facing READ-RENDERING detail the tool owns + * now that the tool reads through `ctx.fs` directly — it is not a storage + * primitive and not freshness policy. * * The provider (`ctx.fs.readText`/`streamText`) hands back already-decoded text * (UTF-8 validated, binary rejected); this module only scans that text for * newlines and builds the requested window. A capped line buffer means a * newline-free giant line can never balloon memory even when streamed. * - * @module @deepseek-ai/dsh-file-context/window + * @module @deepseek-ai/dsh-tool-fs/window */ import { FsError } from '@deepseek-ai/dsh-fs' diff --git a/packages/fs/tool-fs/src/write.ts b/packages/fs/tool-fs/src/write.ts index 8c242c5256..e2b44ee78a 100644 --- a/packages/fs/tool-fs/src/write.ts +++ b/packages/fs/tool-fs/src/write.ts @@ -1,8 +1,12 @@ /** - * The model-facing `write` tool: create or fully replace a UTF-8 text file. - * Execution goes through `ctx.fileContext`, which enforces the freshness policy - * (creating a new file needs no prior read; replacing an existing file requires - * a prior read in the same execution context at the unchanged version). + * The model-facing `write` tool: create or fully replace a UTF-8 text file. The + * tool is the executor: it dispatches the `fs/write-expectation` waterfall to + * obtain the optional version guard, calls `ctx.fs.writeText` directly, and + * emits a contained `fs/observed`. The default thunk returns `undefined` + * (unconditional create-or-overwrite — the bare provider); a policy plugin + * (`@deepseek-ai/dsh-file-context`) occupies the single decision slot and + * returns `createIfAbsent`/`replaceIfVersion` instead. The tool stats ZERO + * times either way. * * @module @deepseek-ai/dsh-tool-fs/write */ @@ -11,7 +15,9 @@ import type { Context } from 'cordis' import { defineTool } from '@deepseek-ai/dsh-tools' import type { ContentBlock } from '@deepseek-ai/dsh-llm' import type { FsWriteOutcome } from '@deepseek-ai/dsh-fs' +import type {} from '@deepseek-ai/dsh-fs' import type {} from '@deepseek-ai/dsh-system-prompt' +import { emitObserved } from './observe.ts' /** Validate value constraints the schema DSL can't express. */ export function parseWriteArgs(args: { file_path: string; content: string }): { filePath: string; content: string } { @@ -34,7 +40,7 @@ export function apply(ctx: Context): void { ctx.systemPrompt.section({ name: 'tool:write', order: 101, - text: 'Use the write tool to create files or completely replace file contents. Existing files are overwritten, so read an existing file first (the backend requires it) and prefer edit for targeted changes.', + text: 'Use the write tool to create files or completely replace file contents. Existing files are overwritten, so read an existing file first (the default file-context policy requires it) and prefer edit for targeted changes.', }) ctx.tools.register(defineTool({ @@ -46,8 +52,12 @@ export function apply(ctx: Context): void { }, async execute(args, exec): Promise { const input = parseWriteArgs(args) - const target = await ctx.fileContext.resolve(input.filePath) - const outcome = await ctx.fileContext.write(target, input.content, exec, exec.signal) + const target = await ctx.fs.resolve(input.filePath) + // Single-slot decision: the policy plugin produces createIfAbsent/ + // replaceIfVersion; the bare default is undefined (unconditional). No stat. + const expectation = await ctx.waterfall('fs/write-expectation', target, exec, () => undefined) + const outcome = await ctx.fs.writeText(target, input.content, expectation, exec.signal) + emitObserved(ctx, target, outcome.version, exec) return [{ type: 'text', text: formatWriteOutput(target.displayPath, outcome) }] }, })) @@ -57,7 +67,7 @@ export function apply(ctx: Context): void { export const name = 'fs-write' /** Services required by the `write` tool plugin. */ -export const inject = ['tools', 'fileContext', 'systemPrompt'] +export const inject = ['tools', 'fs', 'systemPrompt'] /** Named helper for direct registration in the root plugin and tests. */ export const applyWriteTool = apply diff --git a/packages/fs/tool-fs/tests/integration.spec.ts b/packages/fs/tool-fs/tests/integration.spec.ts index d61bdb5cac..696741c334 100644 --- a/packages/fs/tool-fs/tests/integration.spec.ts +++ b/packages/fs/tool-fs/tests/integration.spec.ts @@ -1,12 +1,20 @@ /** - * Integration tests: the real local backend (`dsh-fs-local`) plus the real - * policy layer (`dsh-file-context`) plus the model tools (`dsh-tool-fs`), - * exercised through `ctx.tools.execute()` so nothing bypasses the tool registry. + * Integration tests: the real local backend (`dsh-fs-local`) plus the model + * tools (`dsh-tool-fs`) as the executor, exercised through `ctx.tools.execute()` + * so nothing bypasses the tool registry. Two deployments: + * + * - DEFAULT — with the real `dsh-file-context` policy gate plugin: read-before- + * write/edit, version-guarded mutation, FS_NOT_OBSERVED for unread edits. + * - BARE — WITHOUT the policy plugin, loading only SUBPATH plugins: every + * `fs/*` waterfall falls through to its undefined default, so write/edit are + * unconditional. This proves the subpaths (not just the root) carry no policy + * dependency. + * * These verify the WORLD — files are read back from disk and asserted * byte-for-byte — not the tool's self-report. */ -import { afterEach, beforeEach, describe, expect, it } from 'vitest' +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' import { mkdtemp, readFile, rm, writeFile } from 'node:fs/promises' import { tmpdir } from 'node:os' import { join } from 'node:path' @@ -15,8 +23,11 @@ import { CallId } from '@deepseek-ai/dsh-llm' import SystemPrompt from '@deepseek-ai/dsh-system-prompt' import ToolRegistry from '@deepseek-ai/dsh-tools' import { LocalFileSystem } from '@deepseek-ai/dsh-fs-local' -import FileContext from '@deepseek-ai/dsh-file-context' +import * as FileContext from '@deepseek-ai/dsh-file-context' import * as ToolFs from '@deepseek-ai/dsh-tool-fs' +import * as readPlugin from '@deepseek-ai/dsh-tool-fs/read' +import * as writePlugin from '@deepseek-ai/dsh-tool-fs/write' +import * as editPlugin from '@deepseek-ai/dsh-tool-fs/edit' let dir: string let ctx: Context @@ -24,20 +35,6 @@ let fiber: Awaited> // A stable session object stands in for an agent session (the file-state owner). const session = {} -beforeEach(async () => { - dir = await mkdtemp(join(tmpdir(), 'dsh-tool-fs-')) - ctx = new Context() - await ctx.plugin(SystemPrompt) - await ctx.plugin(ToolRegistry) - await ctx.plugin(LocalFileSystem, { cwd: dir }) - await ctx.plugin(FileContext) - fiber = await ctx.plugin(ToolFs) -}) -afterEach(async () => { - await fiber.dispose() - await rm(dir, { recursive: true, force: true }) -}) - let callCounter = 0 function call(name: string, args: unknown) { return ctx.tools.execute({ @@ -52,137 +49,260 @@ function text(result: { content: { type: string; text?: string }[] }): string { return result.content.filter(b => b.type === 'text').map(b => b.text).join('') } -describe('write → disk', () => { - it('creates a file with exactly the requested bytes', async () => { - const result = await call('write', { file_path: 'new.txt', content: 'line one\nline two\n' }) - expect(result.isError).toBe(false) - expect(await readFile(join(dir, 'new.txt'), 'utf8')).toBe('line one\nline two\n') +afterEach(async () => { + await fiber.dispose() + await rm(dir, { recursive: true, force: true }) +}) + +// -------------------------------------------------------------------------- +// DEFAULT deployment: the policy gate plugin is loaded. +// -------------------------------------------------------------------------- +describe('default deployment (with dsh-file-context)', () => { + beforeEach(async () => { + dir = await mkdtemp(join(tmpdir(), 'dsh-tool-fs-')) + ctx = new Context() + await ctx.plugin(SystemPrompt) + await ctx.plugin(ToolRegistry) + await ctx.plugin(LocalFileSystem, { cwd: dir }) + await ctx.plugin(FileContext) + fiber = await ctx.plugin(ToolFs) }) - it('rejects overwriting an existing file without reading it first', async () => { - await writeFile(join(dir, 'a.txt'), 'original') - const result = await call('write', { file_path: 'a.txt', content: 'clobber' }) - expect(result.isError).toBe(true) - expect(result.error).toMatchObject({ code: 'FS_NOT_OBSERVED' }) - expect(await readFile(join(dir, 'a.txt'), 'utf8')).toBe('original') + describe('write → disk', () => { + it('creates a file with exactly the requested bytes', async () => { + const result = await call('write', { file_path: 'new.txt', content: 'line one\nline two\n' }) + expect(result.isError).toBe(false) + expect(await readFile(join(dir, 'new.txt'), 'utf8')).toBe('line one\nline two\n') + }) + + it('rejects overwriting an existing file without reading it first', async () => { + await writeFile(join(dir, 'a.txt'), 'original') + const result = await call('write', { file_path: 'a.txt', content: 'clobber' }) + expect(result.isError).toBe(true) + expect(result.error).toMatchObject({ code: 'FS_NOT_OBSERVED' }) + expect(await readFile(join(dir, 'a.txt'), 'utf8')).toBe('original') + }) + + it('allows overwriting after a read', async () => { + await writeFile(join(dir, 'a.txt'), 'original') + expect((await call('read', { file_path: 'a.txt' })).isError).toBe(false) + const result = await call('write', { file_path: 'a.txt', content: 'replaced' }) + expect(result.isError).toBe(false) + expect(await readFile(join(dir, 'a.txt'), 'utf8')).toBe('replaced') + }) + + it('rejects a full overwrite when the file changed since the read (stale)', async () => { + await writeFile(join(dir, 'a.txt'), 'original') + await call('read', { file_path: 'a.txt' }) + await writeFile(join(dir, 'a.txt'), 'changed-externally') // out-of-band change + const result = await call('write', { file_path: 'a.txt', content: 'replaced' }) + expect(result.isError).toBe(true) + expect(result.error).toMatchObject({ code: 'FS_STALE_VERSION' }) + }) }) - it('allows overwriting after a read', async () => { - await writeFile(join(dir, 'a.txt'), 'original') - expect((await call('read', { file_path: 'a.txt' })).isError).toBe(false) - const result = await call('write', { file_path: 'a.txt', content: 'replaced' }) - expect(result.isError).toBe(false) - expect(await readFile(join(dir, 'a.txt'), 'utf8')).toBe('replaced') + describe('read', () => { + it('returns line-numbered content', async () => { + await writeFile(join(dir, 'a.txt'), 'alpha\nbeta') + const result = await call('read', { file_path: 'a.txt' }) + expect(text(result)).toContain('1: alpha') + expect(text(result)).toContain('2: beta') + expect(text(result)).toContain('(End of file - total 2 lines)') + }) + + it('reports a binary file as an error', async () => { + await writeFile(join(dir, 'bin'), Buffer.from([0x00, 0x01, 0x02])) + const result = await call('read', { file_path: 'bin' }) + expect(result.isError).toBe(true) + expect(result.error).toMatchObject({ code: 'FS_NOT_TEXT' }) + }) + + it('paginates a multi-line file with offset/limit', async () => { + await writeFile(join(dir, 'a.txt'), 'one\ntwo\nthree\nfour') + const result = await call('read', { file_path: 'a.txt', offset: 2, limit: 2 }) + expect(text(result)).toContain('2: two') + expect(text(result)).toContain('3: three') + expect(text(result)).toContain('(Showing lines 2-3 of 4. Use offset=4 to continue.)') + }) }) - it('rejects a full overwrite when the file changed since the read (stale)', async () => { - await writeFile(join(dir, 'a.txt'), 'original') - await call('read', { file_path: 'a.txt' }) - await writeFile(join(dir, 'a.txt'), 'changed-externally') // out-of-band change - const result = await call('write', { file_path: 'a.txt', content: 'replaced' }) - expect(result.isError).toBe(true) - expect(result.error).toMatchObject({ code: 'FS_STALE_VERSION' }) + describe('edit → disk', () => { + it('applies a unique literal replacement after a read', async () => { + await writeFile(join(dir, 'a.txt'), 'hello world') + await call('read', { file_path: 'a.txt' }) + const result = await call('edit', { file_path: 'a.txt', old_string: 'world', new_string: 'there' }) + expect(result.isError).toBe(false) + expect(await readFile(join(dir, 'a.txt'), 'utf8')).toBe('hello there') + }) + + it('rejects an edit before any read, leaving the file untouched', async () => { + await writeFile(join(dir, 'a.txt'), 'hello world') + const result = await call('edit', { file_path: 'a.txt', old_string: 'world', new_string: 'there' }) + expect(result.isError).toBe(true) + expect(result.error).toMatchObject({ code: 'FS_NOT_OBSERVED' }) + expect(await readFile(join(dir, 'a.txt'), 'utf8')).toBe('hello world') + }) + + it('lets a WINDOWED read authorize an edit when the file is unchanged (freshness, not full-view)', async () => { + // A file with more lines than the read window; read only the first line. + const lines = Array.from({ length: 20 }, (_, i) => `line ${i + 1}`) + await writeFile(join(dir, 'a.txt'), lines.join('\n')) + const read = await call('read', { file_path: 'a.txt', offset: 1, limit: 1 }) + expect(read.isError).toBe(false) + expect(text(read)).toContain('(Showing lines 1-1 of 20') + + // Editing a line OUTSIDE the window is authorized because the file is unchanged. + const result = await call('edit', { file_path: 'a.txt', old_string: 'line 12', new_string: 'LINE 12' }) + expect(result.isError).toBe(false) + expect(await readFile(join(dir, 'a.txt'), 'utf8')).toBe(lines.map(l => l === 'line 12' ? 'LINE 12' : l).join('\n')) + }) + + it('rejects an edit when the file changed since the windowed read (stale before matching)', async () => { + await writeFile(join(dir, 'a.txt'), 'hello world') + await call('read', { file_path: 'a.txt', offset: 1, limit: 1 }) + await writeFile(join(dir, 'a.txt'), 'goodbye') // out-of-band change removes 'world' + const result = await call('edit', { file_path: 'a.txt', old_string: 'world', new_string: 'there' }) + expect(result.isError).toBe(true) + expect(result.error).toMatchObject({ code: 'FS_STALE_VERSION' }) + }) + + it('rejects an ambiguous match without replace_all', async () => { + await writeFile(join(dir, 'a.txt'), 'a a a') + await call('read', { file_path: 'a.txt' }) + const result = await call('edit', { file_path: 'a.txt', old_string: 'a', new_string: 'b' }) + expect(result.isError).toBe(true) + expect(result.error).toMatchObject({ code: 'FS_AMBIGUOUS_EDIT' }) + expect(await readFile(join(dir, 'a.txt'), 'utf8')).toBe('a a a') + }) + + it('replaces all matches with replace_all', async () => { + await writeFile(join(dir, 'a.txt'), 'a a a') + await call('read', { file_path: 'a.txt' }) + const result = await call('edit', { file_path: 'a.txt', old_string: 'a', new_string: 'b', replace_all: true }) + expect(result.isError).toBe(false) + expect(await readFile(join(dir, 'a.txt'), 'utf8')).toBe('b b b') + }) + + it('supports a full write→edit cycle without an intervening read', async () => { + await call('write', { file_path: 'a.txt', content: 'one two' }) + const result = await call('edit', { file_path: 'a.txt', old_string: 'two', new_string: 'three' }) + expect(result.isError).toBe(false) + expect(await readFile(join(dir, 'a.txt'), 'utf8')).toBe('one three') + }) + }) + + describe('the gate records only through the events (no method coupling)', () => { + it('a direct ctx.fs.readText records no observed-state, so a later edit rejects', async () => { + await writeFile(join(dir, 'a.txt'), 'hello world') + // Reach AROUND the tool — an explicit escape hatch for non-tool consumers. + await ctx.fs.readText(await ctx.fs.resolve('a.txt')) + // The model-facing edit still rejects: the read did not emit fs/observed. + const result = await call('edit', { file_path: 'a.txt', old_string: 'world', new_string: 'there' }) + expect(result.isError).toBe(true) + expect(result.error).toMatchObject({ code: 'FS_NOT_OBSERVED' }) + }) + }) + + describe('stat budget', () => { + it('read stats once; write and edit never stat in the tool (the gate stats zero too)', async () => { + await writeFile(join(dir, 'a.txt'), 'hello world') + const statSpy = vi.spyOn(ctx.fs, 'stat') + + // read: exactly one stat (type + size routing + observed version). + await call('read', { file_path: 'a.txt' }) + expect(statSpy).toHaveBeenCalledTimes(1) + + // edit (guarded, after the read): the gate supplies vObserved; the tool + // does not stat to manufacture a basis. CAS happens in editText's lock. + statSpy.mockClear() + const edited = await call('edit', { file_path: 'a.txt', old_string: 'world', new_string: 'there' }) + expect(edited.isError).toBe(false) + expect(statSpy).not.toHaveBeenCalled() + + // write (guarded replace, after the edit refreshed observed state): zero stat. + statSpy.mockClear() + const written = await call('write', { file_path: 'a.txt', content: 'fresh' }) + expect(written.isError).toBe(false) + expect(statSpy).not.toHaveBeenCalled() + statSpy.mockRestore() + }) + }) + + describe('contained fs/observed recording', () => { + it('a synchronously throwing fs/observed listener does not fail the completed write', async () => { + ctx.on('fs/observed', () => { throw new Error('listener boom') }) + const warn = vi.spyOn(ctx.logger, 'warn').mockImplementation(() => {}) + const result = await call('write', { file_path: 'a.txt', content: 'hi' }) + // The write succeeded on disk; the listener throw was logged and swallowed. + expect(result.isError).toBe(false) + expect(await readFile(join(dir, 'a.txt'), 'utf8')).toBe('hi') + expect(warn).toHaveBeenCalled() + warn.mockRestore() + }) }) }) -describe('read', () => { - it('returns line-numbered content', async () => { +// -------------------------------------------------------------------------- +// BARE deployment: SUBPATH plugins only, NO policy gate. +// -------------------------------------------------------------------------- +describe('bare provider (subpath plugins, no dsh-file-context)', () => { + beforeEach(async () => { + dir = await mkdtemp(join(tmpdir(), 'dsh-tool-fs-bare-')) + ctx = new Context() + await ctx.plugin(SystemPrompt) + await ctx.plugin(ToolRegistry) + await ctx.plugin(LocalFileSystem, { cwd: dir }) + await ctx.plugin(readPlugin) + await ctx.plugin(writePlugin) + fiber = await ctx.plugin(editPlugin) + }) + + it('read works (it never needed policy)', async () => { await writeFile(join(dir, 'a.txt'), 'alpha\nbeta') const result = await call('read', { file_path: 'a.txt' }) + expect(result.isError).toBe(false) expect(text(result)).toContain('1: alpha') - expect(text(result)).toContain('2: beta') - expect(text(result)).toContain('(End of file - total 2 lines)') }) - it('reports a binary file as an error', async () => { - await writeFile(join(dir, 'bin'), Buffer.from([0x00, 0x01, 0x02])) - const result = await call('read', { file_path: 'bin' }) - expect(result.isError).toBe(true) - expect(result.error).toMatchObject({ code: 'FS_NOT_TEXT' }) + it('write unconditionally creates a new file', async () => { + const result = await call('write', { file_path: 'new.txt', content: 'fresh' }) + expect(result.isError).toBe(false) + expect(await readFile(join(dir, 'new.txt'), 'utf8')).toBe('fresh') }) - it('paginates a multi-line file with offset/limit', async () => { - await writeFile(join(dir, 'a.txt'), 'one\ntwo\nthree\nfour') - const result = await call('read', { file_path: 'a.txt', offset: 2, limit: 2 }) - expect(text(result)).toContain('2: two') - expect(text(result)).toContain('3: three') - expect(text(result)).toContain('(Showing lines 2-3 of 4. Use offset=4 to continue.)') + it('write unconditionally OVERWRITES an existing unread file', async () => { + await writeFile(join(dir, 'a.txt'), 'original') + const result = await call('write', { file_path: 'a.txt', content: 'clobbered' }) + expect(result.isError).toBe(false) + expect(await readFile(join(dir, 'a.txt'), 'utf8')).toBe('clobbered') }) -}) -describe('edit → disk', () => { - it('applies a unique literal replacement after a read', async () => { + it('edit unconditionally edits an UNREAD existing file', async () => { await writeFile(join(dir, 'a.txt'), 'hello world') - await call('read', { file_path: 'a.txt' }) const result = await call('edit', { file_path: 'a.txt', old_string: 'world', new_string: 'there' }) expect(result.isError).toBe(false) expect(await readFile(join(dir, 'a.txt'), 'utf8')).toBe('hello there') }) - it('rejects an edit before any read, leaving the file untouched', async () => { - await writeFile(join(dir, 'a.txt'), 'hello world') - const result = await call('edit', { file_path: 'a.txt', old_string: 'world', new_string: 'there' }) - expect(result.isError).toBe(true) - expect(result.error).toMatchObject({ code: 'FS_NOT_OBSERVED' }) - expect(await readFile(join(dir, 'a.txt'), 'utf8')).toBe('hello world') - }) - - it('lets a WINDOWED read authorize an edit when the file is unchanged (freshness, not full-view)', async () => { - // A file with more lines than the read window; read only the first line. - const lines = Array.from({ length: 20 }, (_, i) => `line ${i + 1}`) - await writeFile(join(dir, 'a.txt'), lines.join('\n')) - const read = await call('read', { file_path: 'a.txt', offset: 1, limit: 1 }) - expect(read.isError).toBe(false) - expect(text(read)).toContain('(Showing lines 1-1 of 20') - - // Editing a line OUTSIDE the window is authorized because the file is unchanged. - const result = await call('edit', { file_path: 'a.txt', old_string: 'line 12', new_string: 'LINE 12' }) - expect(result.isError).toBe(false) - expect(await readFile(join(dir, 'a.txt'), 'utf8')).toBe(lines.map(l => l === 'line 12' ? 'LINE 12' : l).join('\n')) - }) - - it('rejects an edit when the file changed since the windowed read (stale before matching)', async () => { - await writeFile(join(dir, 'a.txt'), 'hello world') - await call('read', { file_path: 'a.txt', offset: 1, limit: 1 }) - await writeFile(join(dir, 'a.txt'), 'goodbye') // out-of-band change removes 'world' - const result = await call('edit', { file_path: 'a.txt', old_string: 'world', new_string: 'there' }) + it('edit of a MISSING target reports FS_STALE_VERSION even on the unguarded path', async () => { + const result = await call('edit', { file_path: 'missing.txt', old_string: 'a', new_string: 'b' }) expect(result.isError).toBe(true) expect(result.error).toMatchObject({ code: 'FS_STALE_VERSION' }) }) - it('rejects an ambiguous match without replace_all', async () => { - await writeFile(join(dir, 'a.txt'), 'a a a') - await call('read', { file_path: 'a.txt' }) - const result = await call('edit', { file_path: 'a.txt', old_string: 'a', new_string: 'b' }) - expect(result.isError).toBe(true) - expect(result.error).toMatchObject({ code: 'FS_AMBIGUOUS_EDIT' }) - expect(await readFile(join(dir, 'a.txt'), 'utf8')).toBe('a a a') - }) - - it('replaces all matches with replace_all', async () => { - await writeFile(join(dir, 'a.txt'), 'a a a') - await call('read', { file_path: 'a.txt' }) - const result = await call('edit', { file_path: 'a.txt', old_string: 'a', new_string: 'b', replace_all: true }) - expect(result.isError).toBe(false) - expect(await readFile(join(dir, 'a.txt'), 'utf8')).toBe('b b b') - }) - - it('supports a full write→edit cycle without an intervening read', async () => { - await call('write', { file_path: 'a.txt', content: 'one two' }) - const result = await call('edit', { file_path: 'a.txt', old_string: 'two', new_string: 'three' }) - expect(result.isError).toBe(false) - expect(await readFile(join(dir, 'a.txt'), 'utf8')).toBe('one three') - }) -}) - -describe('no-bypass / escape-hatch contract', () => { - it('a direct ctx.fs.readText records no observed-state, so a later edit rejects', async () => { + it('edit still enforces literal-match codes (FS_EDIT_NOT_FOUND), unrelated to freshness', async () => { await writeFile(join(dir, 'a.txt'), 'hello world') - // Reach AROUND the policy layer — an explicit escape hatch for non-tool consumers. - await ctx.fs.readText(await ctx.fs.resolve('a.txt')) - // The model-facing edit still rejects: the read was not through ctx.fileContext. - const result = await call('edit', { file_path: 'a.txt', old_string: 'world', new_string: 'there' }) + const result = await call('edit', { file_path: 'a.txt', old_string: 'absent', new_string: 'x' }) expect(result.isError).toBe(true) - expect(result.error).toMatchObject({ code: 'FS_NOT_OBSERVED' }) + expect(result.error).toMatchObject({ code: 'FS_EDIT_NOT_FOUND' }) + }) + + it('neither write nor edit stats in the tool on the bare path', async () => { + await writeFile(join(dir, 'a.txt'), 'hello world') + const statSpy = vi.spyOn(ctx.fs, 'stat') + expect((await call('write', { file_path: 'a.txt', content: 'x y' })).isError).toBe(false) + expect((await call('edit', { file_path: 'a.txt', old_string: 'y', new_string: 'z' })).isError).toBe(false) + expect(statSpy).not.toHaveBeenCalled() + statSpy.mockRestore() }) }) diff --git a/packages/fs/tool-fs/tests/subpaths.spec.ts b/packages/fs/tool-fs/tests/subpaths.spec.ts index 7243955969..32a276ca25 100644 --- a/packages/fs/tool-fs/tests/subpaths.spec.ts +++ b/packages/fs/tool-fs/tests/subpaths.spec.ts @@ -1,7 +1,10 @@ /** * Tests for the per-tool subpath plugins (`@deepseek-ai/dsh-tool-fs/read`, - * `/write`, `/edit`): each registers exactly one tool, injects the same - * services (`tools`, `fileContext`, `systemPrompt`), and cleans up on disposal. + * `/write`, `/edit`): each registers exactly one tool, injects the same services + * (`tools`, `fs`, `systemPrompt`) — NOT a policy service — and cleans up on + * disposal. They boot over the bare `ctx.fs` provider with NO + * `@deepseek-ai/dsh-file-context`, proving each subpath carries no policy-plugin + * dependency. */ import { describe, expect, it } from 'vitest' @@ -15,7 +18,6 @@ import type { FsTarget, FsWriteOutcome, } from '@deepseek-ai/dsh-fs' -import FileContext from '@deepseek-ai/dsh-file-context' import * as readPlugin from '@deepseek-ai/dsh-tool-fs/read' import * as writePlugin from '@deepseek-ai/dsh-tool-fs/write' import * as editPlugin from '@deepseek-ai/dsh-tool-fs/edit' @@ -46,12 +48,11 @@ async function base() { await ctx.plugin(SystemPrompt) await ctx.plugin(ToolRegistry) await ctx.plugin(StubFs) - await ctx.plugin(FileContext) return ctx } describe('subpath plugins', () => { - it('each registers exactly its one tool', async () => { + it('each registers exactly its one tool (over the bare provider, no policy plugin)', async () => { const cases: Array<[unknown, string]> = [ [readPlugin, 'read'], [writePlugin, 'write'], @@ -72,7 +73,7 @@ describe('subpath plugins', () => { expect(ctx.tools.schemas()).toHaveLength(0) }) - it('stays pending without a ctx.fileContext provider', async () => { + it('stays pending without a ctx.fs provider', async () => { const ctx = new Context() await ctx.plugin(SystemPrompt) await ctx.plugin(ToolRegistry) diff --git a/packages/fs/tool-fs/tests/tools.spec.ts b/packages/fs/tool-fs/tests/tools.spec.ts index ef1490b5ce..5ca6947354 100644 --- a/packages/fs/tool-fs/tests/tools.spec.ts +++ b/packages/fs/tool-fs/tests/tools.spec.ts @@ -1,13 +1,14 @@ /** - * Consumer-surface tests for the filesystem tools. They run the REAL - * `ctx.fileContext` policy service over a fake `ctx.fs` provider (the genuine - * collaborator, per the prefer-the-real-implementation rule), so they verify - * schemas, argument validation, result formatting, FsError→isError propagation, - * and that each tool records observed-state through `ctx.fileContext` (the - * no-bypass contract) — not just that it moved bytes. + * Consumer-surface tests for the filesystem tools as the EXECUTOR. They run the + * REAL `@deepseek-ai/dsh-file-context` gate plugin (the genuine policy + * collaborator, per the prefer-the-real-implementation rule) over a fake + * `ctx.fs` provider, so they verify schemas, argument validation, result + * formatting, FsError→isError propagation, and that each tool dispatches the + * `fs/*` waterfalls + records observed-state through the gate (read authorizes a + * later edit) — not just that it moved bytes. */ -import { describe, expect, it } from 'vitest' +import { describe, expect, it, vi } from 'vitest' import { Context } from 'cordis' import { CallId } from '@deepseek-ai/dsh-llm' import SystemPrompt, { renderPrompt } from '@deepseek-ai/dsh-system-prompt' @@ -21,15 +22,17 @@ import type { FsWriteExpectation, FsWriteOutcome, } from '@deepseek-ai/dsh-fs' -import FileContext from '@deepseek-ai/dsh-file-context' -import type { FileReadOutcome } from '@deepseek-ai/dsh-file-context' +import * as FileContext from '@deepseek-ai/dsh-file-context' import * as ToolFs from '@deepseek-ai/dsh-tool-fs' -import { formatReadOutput } from '@deepseek-ai/dsh-tool-fs' +import { formatReadOutput, STREAM_MIN_SIZE } from '@deepseek-ai/dsh-tool-fs' +import type { FileReadOutcome } from '@deepseek-ai/dsh-tool-fs' /** An in-memory fake provider; a test can arm a rejection on any primitive. */ class FakeFs extends FileSystem { files = new Map() rejectWith?: FsError + writeExpectations: (FsWriteExpectation | undefined)[] = [] + editExpectations: ({ version: FsVersion } | undefined)[] = [] private throwIfArmed(): void { if (this.rejectWith) throw this.rejectWith @@ -51,14 +54,16 @@ class FakeFs extends FileSystem { const content = this.files.get(target.targetKey) ?? '' return (async function* () { yield content })() } - override async writeText(target: FsTarget, content: string, _expected: FsWriteExpectation): Promise { + override async writeText(target: FsTarget, content: string, expected?: FsWriteExpectation): Promise { this.throwIfArmed() + this.writeExpectations.push(expected) const existed = this.files.has(target.targetKey) this.files.set(target.targetKey, content) return { operation: existed ? 'update' : 'create', version: FsVersion('v2') } } - override async editText(target: FsTarget, edit: FsEditRequest): Promise { + override async editText(target: FsTarget, edit: FsEditRequest, expected?: { version: FsVersion }): Promise { this.throwIfArmed() + this.editExpectations.push(expected) const content = this.files.get(target.targetKey) ?? '' this.files.set(target.targetKey, content.split(edit.oldString).join(edit.newString)) return { replacements: 1, replaceAll: edit.replaceAll, version: FsVersion('v3') } @@ -104,11 +109,11 @@ describe('registration', () => { expect(prompt).toContain('Use the edit tool') }) - it('stays pending until ctx.fileContext exists (inject)', async () => { + it('stays pending until ctx.fs exists (inject)', async () => { const ctx = new Context() await ctx.plugin(SystemPrompt) await ctx.plugin(ToolRegistry) - await ctx.plugin(ToolFs) // no fileContext provider + await ctx.plugin(ToolFs) // no fs provider expect(ctx.tools.schemas()).toHaveLength(0) }) @@ -169,6 +174,7 @@ describe('read tool', () => { expect((await call(ctx, 'read', { file_path: 'a.txt' }, { session })).isError).toBe(false) const edited = await call(ctx, 'edit', { file_path: 'a.txt', old_string: 'hello', new_string: 'bye' }, { session }) expect(edited.isError).toBe(false) + expect(fs.editExpectations).toEqual([{ version: 'v1' }]) }) it('propagates FS_NOT_FOUND for an absent file', async () => { @@ -177,6 +183,48 @@ describe('read tool', () => { expect(result.isError).toBe(true) expect(result.error).toMatchObject({ code: 'FS_NOT_FOUND' }) }) + + it('rejects a non-regular target', async () => { + const { ctx, fs } = await setup() + fs.files.set('key:d', '') + fs.stat = async () => ({ version: FsVersion('v1'), type: 'directory' }) + const result = await call(ctx, 'read', { file_path: 'd' }) + expect(result.isError).toBe(true) + expect(result.error).toMatchObject({ code: 'FS_NOT_REGULAR_FILE' }) + }) + + it('streams a large file (size at/above the cap) instead of reading whole', async () => { + const { ctx, fs } = await setup() + fs.files.set('key:big.txt', 'alpha\nbeta') + const readSpy = vi.spyOn(fs, 'readText') + const streamSpy = vi.spyOn(fs, 'streamText') + fs.stat = async () => ({ version: FsVersion('v1'), type: 'file', size: STREAM_MIN_SIZE }) + const result = await call(ctx, 'read', { file_path: 'big.txt' }) + expect(result.isError).toBe(false) + expect(text(result)).toContain('1: alpha') + expect(streamSpy).toHaveBeenCalled() + expect(readSpy).not.toHaveBeenCalled() + }) + + it('streams when the backend reports no size (never buffers a size-less file)', async () => { + const { ctx, fs } = await setup() + fs.files.set('key:a.txt', 'alpha') + const streamSpy = vi.spyOn(fs, 'streamText') + fs.stat = async () => ({ version: FsVersion('v1'), type: 'file' }) // no size + const result = await call(ctx, 'read', { file_path: 'a.txt' }) + expect(result.isError).toBe(false) + expect(streamSpy).toHaveBeenCalled() + }) + + it('surfaces a byte-capped read as a truncated footer', async () => { + const { ctx, fs } = await setup() + // Many long lines so the window hits the byte cap before EOF. + fs.files.set('key:big.txt', Array.from({ length: 2000 }, () => 'y'.repeat(100)).join('\n')) + const result = await call(ctx, 'read', { file_path: 'big.txt' }) + expect(result.isError).toBe(false) + expect(text(result)).toContain('Output capped.') + }) + }) describe('formatReadOutput footer variants', () => { @@ -204,11 +252,12 @@ describe('formatReadOutput footer variants', () => { }) describe('write tool', () => { - it('formats a create result', async () => { - const { ctx } = await setup() - const result = await call(ctx, 'write', { file_path: 'a.txt', content: 'hi' }) + it('formats a create result and uses createIfAbsent (unobserved, with the gate)', async () => { + const { ctx, fs } = await setup() + const result = await call(ctx, 'write', { file_path: 'a.txt', content: 'hi' }, { session: {} }) expect(result.isError).toBe(false) expect(text(result)).toContain('Created file') + expect(fs.writeExpectations).toEqual([{ kind: 'createIfAbsent' }]) }) it('rejects a blank file_path', async () => { @@ -258,7 +307,7 @@ describe('edit tool', () => { expect(text(result)).toContain('file_path must be a non-empty string') }) - it('propagates FS_NOT_OBSERVED when the file was never read', async () => { + it('propagates FS_NOT_OBSERVED when the file was never read (the gate decides)', async () => { const { ctx, fs } = await setup() fs.files.set('key:a.txt', 'hello') const result = await call(ctx, 'edit', { file_path: 'a.txt', old_string: 'a', new_string: 'b' }, { session: {} }) diff --git a/packages/fs/file-context/tests/window.spec.ts b/packages/fs/tool-fs/tests/window.spec.ts similarity index 98% rename from packages/fs/file-context/tests/window.spec.ts rename to packages/fs/tool-fs/tests/window.spec.ts index 6b1a8b5b93..b596a47465 100644 --- a/packages/fs/file-context/tests/window.spec.ts +++ b/packages/fs/tool-fs/tests/window.spec.ts @@ -6,8 +6,8 @@ */ import { describe, expect, it } from 'vitest' -import { buildWindow, READ_MAX_LINE_LENGTH } from '@deepseek-ai/dsh-file-context' -import type { ReadWindow } from '@deepseek-ai/dsh-file-context' +import { buildWindow, READ_MAX_LINE_LENGTH } from '@deepseek-ai/dsh-tool-fs' +import type { ReadWindow } from '@deepseek-ai/dsh-tool-fs' const READ_ALL: ReadWindow = { offset: 1, limit: 2000 } diff --git a/scripts/type-equiv.manifest.json b/scripts/type-equiv.manifest.json index 1c5ec2430e..f471723679 100644 --- a/scripts/type-equiv.manifest.json +++ b/scripts/type-equiv.manifest.json @@ -47,7 +47,6 @@ { "doc": "docs/core-data-structures/filesystem.md", "symbol": "FsEditOutcome", "source": "packages/fs/fs/src/types.ts" }, { "doc": "docs/core-data-structures/filesystem.md", "symbol": "FsErrorCode", "source": "packages/fs/fs/src/types.ts" }, { "doc": "docs/core-data-structures/filesystem.md", "symbol": "FileContextExec", "source": "packages/fs/file-context/src/types.ts" }, - { "doc": "docs/core-data-structures/filesystem.md", "symbol": "FileReadRequest", "source": "packages/fs/file-context/src/types.ts" }, - { "doc": "docs/core-data-structures/filesystem.md", "symbol": "FileReadOutcome", "source": "packages/fs/file-context/src/types.ts" } + { "doc": "docs/core-data-structures/filesystem.md", "symbol": "FileReadOutcome", "source": "packages/fs/tool-fs/src/types.ts" } ] }