From ef37ce3b9dea48a63e41706fe82a2ffb12086914 Mon Sep 17 00:00:00 2001 From: Dudu-0223 Date: Fri, 26 Jun 2026 17:23:18 +0800 Subject: [PATCH] refactor(fs): split filesystem seam into provider ctx.fs + policy ctx.fileContext Implements the split-the-filesystem-seam RFC. ctx.fs shrinks to a text-storage provider seam (resolve/stat/readText/streamText/writeText/editText with branded FsTargetKey/FsVersion and an explicit FsWriteExpectation); the new dsh-file-context package owns the model-facing policy (read windowing, observed-state, write/edit freshness) as the concrete ctx.fileContext service. Authorization is now freshness-based rather than full/partial view: a windowed read records the file version and authorizes a later edit when the file is unchanged, removing the dead-end where reading lines 100-150 of a large file could not edit line 120. editText stays a provider primitive so version guard + literal match + atomic rewrite remain one critical section, and the stale check runs before matching so a stale edit reports FS_STALE_VERSION. tool-fs injects fileContext, never reaching around to ctx.fs (the no-bypass contract). --- AGENTS.md | 1 + docs/architecture.md | 8 +- docs/cordis-catalog/events-and-services.md | 47 ++- docs/core-data-structures/filesystem.md | 119 +++--- docs/module-graph.md | 8 +- docs/rfc/README.md | 1 + .../2026-06-26-fsspec-style-fs-seam.md | 120 ++++++ examples/coding-agent/cordis.yml | 2 +- packages/README.md | 8 +- packages/fs/README.md | 7 +- packages/fs/file-context/README.md | 43 +++ packages/fs/file-context/package.json | 31 ++ packages/fs/file-context/src/index.ts | 184 ++++++++++ packages/fs/file-context/src/types.ts | 56 +++ packages/fs/file-context/src/window.ts | 139 +++++++ packages/fs/file-context/tests/policy.spec.ts | 305 +++++++++++++++ packages/fs/file-context/tests/window.spec.ts | 102 ++++++ packages/fs/file-context/tsconfig.json | 14 + packages/fs/fs-local/README.md | 12 +- packages/fs/fs-local/src/fsio.ts | 301 ++++----------- packages/fs/fs-local/src/index.ts | 91 +++-- packages/fs/fs-local/tests/filesystem.spec.ts | 346 ++++++++---------- packages/fs/fs-local/tests/fsio.spec.ts | 310 ++++++---------- packages/fs/fs/README.md | 45 ++- packages/fs/fs/package.json | 2 + packages/fs/fs/src/index.ts | 255 ++++--------- packages/fs/fs/src/types.ts | 157 +++----- packages/fs/fs/tests/service.spec.ts | 312 +++------------- packages/fs/fs/tsconfig.json | 1 + packages/fs/tool-fs/README.md | 19 +- packages/fs/tool-fs/package.json | 2 + packages/fs/tool-fs/src/edit.ts | 10 +- packages/fs/tool-fs/src/index.ts | 15 +- packages/fs/tool-fs/src/read.ts | 15 +- packages/fs/tool-fs/src/write.ts | 12 +- packages/fs/tool-fs/tests/integration.spec.ts | 63 +++- packages/fs/tool-fs/tests/subpaths.spec.ts | 30 +- packages/fs/tool-fs/tests/tools.spec.ts | 134 ++++--- packages/fs/tool-fs/tsconfig.json | 3 +- pnpm-lock.yaml | 18 + scripts/type-equiv.manifest.json | 16 +- tsconfig.build.json | 1 + 42 files changed, 1899 insertions(+), 1466 deletions(-) create mode 100644 docs/rfc/implemented/simplification/2026-06-26-fsspec-style-fs-seam.md create mode 100644 packages/fs/file-context/README.md create mode 100644 packages/fs/file-context/package.json create mode 100644 packages/fs/file-context/src/index.ts create mode 100644 packages/fs/file-context/src/types.ts create mode 100644 packages/fs/file-context/src/window.ts create mode 100644 packages/fs/file-context/tests/policy.spec.ts create mode 100644 packages/fs/file-context/tests/window.spec.ts create mode 100644 packages/fs/file-context/tsconfig.json diff --git a/AGENTS.md b/AGENTS.md index a68e13ccef..04ce3b49fc 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -199,6 +199,7 @@ Dev/test/demo run **unbuilt** via tsx + the `paths` map in the root `tsconfig.js - **Plugins, not loop changes**: new behavior goes into a plugin on the documented extension seams (see the plugin sanity checklist in docs/architecture.md). Changing `agent-loop` requires updating that doc. - **Capability seams are three packages**: when adding a swappable capability (an execution backend, a provider integration, …), split it into *interface* (abstract service + vocabulary types, e.g. `bash/`), *implementation* (a concrete subclass, e.g. `bash-local/`), and *consumer* (what the model/plugins see, e.g. `tool-bash/`). Implementations and consumers then evolve independently — a sandboxed executor replaces `bash-local` without touching tool schemas. The LLM seam follows the same shape (`llm/` is interface + consumer surface; adapters are implementations). See docs/architecture.md § "Capability seams" for when NOT to split. - **Explicit > implicit at package seams**: interface/vocabulary types spell out every field a consumer must supply — no optional field that the implementation silently fills with a hidden `?? default`. Put defaulting in the owning implementation as an explicit step (a `resolve(request): Spec` method that turns the optional-field request into the required-field spec), not smuggled inside `run()`/`start()`. Example: `dsh-bash` splits `BashExecRequest` (optional `workdir`/`timeoutMs`, model-facing) from `BashExecSpec` (required, what `run`/`start` act on); the tool layer calls `ctx.bash.resolve()` between them. The reader of a `BashExecSpec` never has to wonder where the working directory came from. +- **Opaque cross-boundary ids are branded, never bare `string`**: an identity that crosses a package seam and that a consumer must store-and-return but never parse (a backend-defined version token, a target key, a task/session/call id) is a `Branded` from `@deepseek-ai/dsh-brand` with a same-named cast factory in the owning package — a zero-cost compile-time guard so semantically-distinct strings stop being interchangeable. Not every string needs it: author-readable names (`ToolName`) and closed code unions (`ErrorCode`) don't. See [Branded IDs everywhere they belong](docs/rfc/implemented/architecture/2026-06-20-branded-ids.md). - **An empty `catch` must name what it swallows and why nothing else can hit it**: a bare `catch {}` hides bugs. When you deliberately ignore a throw, the comment must (a) name the single expected failure, (b) say why ignoring it is correct — usually because the useful state was already captured *before* the `try` — and (c) make clear nothing else of consequence can reach the catch (ideally the `try` wraps a single statement). Example: the error-body `response.json()` parse in `dsh-llm-deepseek`'s adapter sets `code` + HTTP `status` from the status line before the `try`, so a malformed provider body can only cost a richer message, never the real error. - **Symmetry is usually more correct**: when two related values play parallel roles (a test fixture and its expected output, a request shape and its response shape, a buggy input and the test that checks the fix), give them parallel form — both named consts, or both inline, not one each way. Asymmetry is a smell that usually points at a missed extraction. - **Merging PRs**: always merge with a **merge commit** (`gh pr merge --merge`), never squash or rebase. The per-PR commit history is intentional — review-fix commits, regression-test commits, and the reasoning in each message are part of the record — and squashing flattens it away. diff --git a/docs/architecture.md b/docs/architecture.md index 8f78a0b086..796026bb10 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -25,6 +25,7 @@ For a catalog of the **data structures** this architecture moves around — the │ @deepseek-ai/dsh-bash-local (bash impl) │ │ @deepseek-ai/dsh-tool-bash (bash tool schemas) │ │ @deepseek-ai/dsh-fs-local (filesystem impl) │ +│ @deepseek-ai/dsh-file-context (filesystem policy) │ │ @deepseek-ai/dsh-tool-fs (filesystem tool schemas) │ │ @deepseek-ai/dsh-session-persistence-jsonl (persistence impl)│ ├─────────────────────────────────────────────────────────────┤ @@ -35,7 +36,7 @@ For a catalog of the **data structures** this architecture moves around — the │ @deepseek-ai/dsh-session-persistence (persistence seam) │ │ @deepseek-ai/dsh-llm (abstract model service) │ │ @deepseek-ai/dsh-bash (abstract bash executor) │ -│ @deepseek-ai/dsh-fs (abstract filesystem) │ +│ @deepseek-ai/dsh-fs (filesystem provider seam) │ ├─────────────────────────────────────────────────────────────┤ │ vendor/: cordis, loader, include, group, timer, hmr, │ │ logger-console, cosmokit, schemastery │ @@ -56,7 +57,8 @@ Dependency rule: **extension** plugins depend on interface packages, never on `d | `ctx.agents` | `AgentRegistry` | dsh-agent | live `Agent` handles + the create/resume factory seam (returns an `AgentHandle` = `{ agent, dispose() }` for owned per-agent teardown) | | `ctx.agentLoop` | `AgentLoop` | dsh-agent-loop | creates `ReactLoopAgent`s and drives their loops | | `ctx.bash` | `BashExecutor` (abstract) | dsh-bash | bash execution seam: foreground runs + background tasks | -| `ctx.fs` | `FileSystem` (abstract) | dsh-fs | filesystem seam: path resolution, text reads, writes, edits, and observed-file policy | +| `ctx.fs` | `FileSystem` (abstract) | dsh-fs | filesystem provider seam: path resolution, stat, text read/stream, guarded writes/edits | +| `ctx.fileContext` | `FileContext` | dsh-file-context | filesystem policy: read windowing, observed-state, write/edit freshness over `ctx.fs` | All registrations (`registerAdapter`, `section`, `tools`, `register`, …) go through `ctx.effect()` and return disposers, so plugin hot-reload (vendored HMR) and fiber disposal clean up automatically. @@ -72,7 +74,7 @@ Swappable capabilities are split into **three packages** so each part evolves in The LLM seam has the same topology folded differently: `dsh-llm` carries the interface (`LlmAdapter`) AND the consumer surface (`ctx.llm.stream()`), with adapters as implementation packages — there the consumer is the loop itself, not a swappable schema surface. Use the full three-package split when the consumer is independently replaceable; keep interface + consumer together when they are one concern. Don't split preemptively: a capability with one conceivable implementation and one consumer stays one package until proven otherwise. -The filesystem capability follows the bash topology: `dsh-fs` owns the abstract `ctx.fs` service and observed-file policy, `dsh-fs-local` provides the local backend, and `dsh-tool-fs` exposes the model-facing `read`/`write`/`edit` schemas over the interface. +The filesystem capability follows the bash topology with a fourth layer: `dsh-fs` owns the abstract `ctx.fs` provider seam (text IO + guarded mutation primitives), `dsh-fs-local` provides the local backend, `dsh-file-context` is a concrete `ctx.fileContext` policy service (read windowing + observed-state + write/edit freshness, injecting `fs`), and `dsh-tool-fs` exposes the model-facing `read`/`write`/`edit` schemas over `ctx.fileContext`. The policy layer is a concrete service, not a second swappable seam — it owns the model-facing observation policy a sandboxed/remote backend has no business carrying. > **"Capability" — two unrelated meanings.** (1) The *seam pattern* above ("one plugin provides a capability, another needs it") is realized by plain Cordis **services + `inject`**: a provider registers a service (`ctx.bash`, declared in `interface Context`); a consumer declares `inject: ['bash']` and its fiber stays pending until the service exists, tearing down via HMR if it later vanishes. No extra library is needed. (2) `@cordisjs/plugin-capability` is a different axis entirely — a **permission/capability-security** service (named permissions with inheritance/dependency, tested against a session via `ctx.capability.test`). It is a candidate for the deferred permissions/sandbox work (the `tools/execute` veto seam), NOT a mechanism for swapping implementations. diff --git a/docs/cordis-catalog/events-and-services.md b/docs/cordis-catalog/events-and-services.md index 0d0cd7b90b..c4896ec0ad 100644 --- a/docs/cordis-catalog/events-and-services.md +++ b/docs/cordis-catalog/events-and-services.md @@ -279,7 +279,7 @@ Source: [`packages/core/tools/src/index.ts:43`](../../packages/core/tools/src/in ## Services -The 9 `ctx.` services the harness provides. An abstract seam (e.g. `ctx.bash`) is implemented by a separate package; the interface is what consumers code against. +The 10 `ctx.` services the harness provides. An abstract seam (e.g. `ctx.bash`) is implemented by a separate package; the interface is what consumers code against. ### `ctx.agentLoop` — `AgentLoop` @@ -339,33 +339,46 @@ Types: [BashExecRequest](../core-data-structures/bash.md) · [BashExecSpec](../c Source: [`packages/bash/bash/src/index.ts:59`](../../packages/bash/bash/src/index.ts) +### `ctx.fileContext` — `FileContext` + +The file-context policy service. Injects `fs`, registers as `ctx.fileContext`, and is the only read/write/edit path the model-facing tools use. + +```ts cordis-catalog +owner(exec?: FileContextExec): object | undefined +async resolve(path: string): Promise +async read(target: FsTarget, request: FileReadRequest, exec?: FileContextExec, signal?: AbortSignal): Promise +async write(target: FsTarget, content: string, exec?: FileContextExec, signal?: AbortSignal): Promise +async edit(target: FsTarget, edit: FsEditRequest, exec?: FileContextExec, signal?: AbortSignal): Promise +``` + +Types: [FsEditOutcome](../core-data-structures/filesystem.md) · [FsEditRequest](../core-data-structures/filesystem.md) · [FsTarget](../core-data-structures/filesystem.md) · [FsWriteOutcome](../core-data-structures/filesystem.md) + +Source: [`packages/fs/file-context/src/index.ts:65`](../../packages/fs/file-context/src/index.ts) + ### `ctx.fs` — `FileSystem` (abstract seam) -Abstract filesystem service. Subclass, implement the four backend primitives (resolve, readPage, createOrReplace, applyEdit), and load the subclass as a plugin — it registers as `ctx.fs` (one implementation per context; loading a second throws, cordis' standard duplicate-service behavior). - -Consumers call the concrete public API (read/write/ edit), which derives the file-state owner, enforces the read-before-write/edit policy, and refreshes recorded state — then delegates the actual I/O to the backend primitives. +Abstract filesystem provider service. Subclass, implement the six text-storage primitives, and load the subclass as a plugin — it registers as `ctx.fs` (one implementation per context; loading a second throws, cordis' standard duplicate-service behavior). Semantics every backend must honor: -- resolve returns a stable FsTarget; the same underlying file reached by different input paths must yield the same `targetKey` so stale guards and file-state lookup agree across paths (e.g. through symlinks). -- readPage returns line-numbered UTF-8 content with a `version` and a `view` (`full` only when the page covered the whole file). -- createOrReplace honors the FsExpectation: `observed` rejects with `FS_STALE_VERSION` if the file changed since `version`; `partial` rejects existing targets because the owner saw only a non-editable view; `unobserved` creates iff the target is absent and otherwise rejects. -- applyEdit verifies the expected version (stale guard) and is atomic (read-modify-write must not interleave with a concurrent edit). +- resolve returns a stable FsTarget; the same underlying file reached by different input paths must yield the same `targetKey` so stale guards and target lookup agree across paths (e.g. through symlinks). +- stat returns FsInfo metadata (never content) or `undefined` when the target is absent. +- readText/streamText read the whole regular text file (the stream for large files); both own regular-file checks, UTF-8 decoding, binary/NUL rejection, and `FS_NOT_TEXT`. +- writeText is atomic temp-file + rename honoring the FsWriteExpectation. +- editText verifies `expected.version` BEFORE literal matching (so a stale edit reports `FS_STALE_VERSION`, not `FS_EDIT_NOT_FOUND`/ `FS_AMBIGUOUS_EDIT` against newer content), then applies literal replacement and writes atomically — all inside one mutation critical section. ```ts cordis-catalog abstract resolve(path: string): Promise -abstract readPage(target: FsTarget, request: FsReadRequest, signal?: AbortSignal): Promise -abstract createOrReplace(target: FsTarget, content: string, expected: FsExpectation, signal?: AbortSignal): Promise -abstract applyEdit(target: FsTarget, edit: FsEditRequest, expected: { version: FsVersion }, signal?: AbortSignal): Promise -owner(exec?: FsExecContext): object | undefined -async read(target: FsTarget, request: FsReadRequest, exec?: FsExecContext, signal?: AbortSignal): Promise -async write(target: FsTarget, content: string, exec?: FsExecContext, signal?: AbortSignal): Promise -async edit(target: FsTarget, edit: FsEditRequest, exec?: FsExecContext, signal?: AbortSignal): Promise +abstract stat(target: FsTarget, signal?: AbortSignal): Promise +abstract readText(target: FsTarget, signal?: AbortSignal): Promise +abstract streamText(target: FsTarget, signal?: AbortSignal): Promise> +abstract writeText(target: FsTarget, content: string, expected: FsWriteExpectation, signal?: AbortSignal): Promise +abstract editText(target: FsTarget, edit: FsEditRequest, expected: { version: FsVersion }, signal?: AbortSignal): Promise ``` -Types: [FsEditOutcome](../core-data-structures/filesystem.md) · [FsEditRequest](../core-data-structures/filesystem.md) · [FsExecContext](../core-data-structures/filesystem.md) · [FsExpectation](../core-data-structures/filesystem.md) · [FsReadOutcome](../core-data-structures/filesystem.md) · [FsReadRequest](../core-data-structures/filesystem.md) · [FsTarget](../core-data-structures/filesystem.md) · [FsVersion](../core-data-structures/filesystem.md) · [FsWriteOutcome](../core-data-structures/filesystem.md) +Types: [FsEditOutcome](../core-data-structures/filesystem.md) · [FsEditRequest](../core-data-structures/filesystem.md) · [FsTarget](../core-data-structures/filesystem.md) · [FsVersion](../core-data-structures/filesystem.md) · [FsWriteOutcome](../core-data-structures/filesystem.md) -Source: [`packages/fs/fs/src/index.ts:94`](../../packages/fs/fs/src/index.ts) +Source: [`packages/fs/fs/src/index.ts:90`](../../packages/fs/fs/src/index.ts) ### `ctx.llm` — `LlmService` diff --git a/docs/core-data-structures/filesystem.md b/docs/core-data-structures/filesystem.md index fc80891ef6..99d073c6d4 100644 --- a/docs/core-data-structures/filesystem.md +++ b/docs/core-data-structures/filesystem.md @@ -1,80 +1,49 @@ # Filesystem -The filesystem execution seam is split across three packages: interface ([dsh-fs](../../packages/fs/fs), `ctx.fs`), implementation ([dsh-fs-local](../../packages/fs/fs-local), local disk), and consumer ([dsh-tool-fs](../../packages/fs/tool-fs), the model-facing `read`/`write`/`edit` tools). Filesystem access is an optional capability, not part of the agent-loop spine, so its vocabulary lives here rather than in [core.md](core.md). A sandboxed, remote, virtual, or project-scoped backend can implement the same `FileSystem` service without changing the tool schemas. +The filesystem stack is split across four packages: a provider seam ([dsh-fs](../../packages/fs/fs), `ctx.fs`, text IO + guarded mutation), a local implementation ([dsh-fs-local](../../packages/fs/fs-local), local disk), a policy layer ([dsh-file-context](../../packages/fs/file-context), `ctx.fileContext`, read windowing + write/edit freshness), and a consumer ([dsh-tool-fs](../../packages/fs/tool-fs), the model-facing `read`/`write`/`edit` tools). Filesystem access is an optional capability, not part of the agent-loop spine, so its vocabulary lives here rather than in [core.md](core.md). A sandboxed, remote, virtual, or project-scoped backend can implement the same `FileSystem` service without changing the policy layer or the tool schemas. -Source: [`packages/fs/fs/src/types.ts`](../../packages/fs/fs/src/types.ts) and [`packages/fs/fs/src/index.ts`](../../packages/fs/fs/src/index.ts) +Provider source: [`packages/fs/fs/src/types.ts`](../../packages/fs/fs/src/types.ts) and [`packages/fs/fs/src/index.ts`](../../packages/fs/fs/src/index.ts). Policy source: [`packages/fs/file-context/src/types.ts`](../../packages/fs/file-context/src/types.ts). -## Execution context and target identity +## Target identity and metadata (provider seam) -The filesystem seam needs just enough execution context to derive the observed-file owner. `ToolExecution` satisfies this shape, so `dsh-tool-fs` passes its execution object through without making `dsh-fs` import the tool, agent, or session packages. - -```ts type-equiv -interface FsExecContext { - agent?: { - session?: object - } -} -``` - -Every operation resolves a user-supplied path to an opaque backend target first. Consumers may display `displayPath`, but must not parse `targetKey` or assume it is a local absolute path. +Every operation resolves a user-supplied path to an opaque backend target first. Consumers may display `displayPath`, but must not parse `targetKey` (a branded opaque id) or assume it is a local absolute path. ```ts type-equiv interface FsTarget { inputPath: string - targetKey: string + targetKey: FsTargetKey displayPath: string } ``` -The backend also owns file-version tokens. `ctx.fs` stores them for stale checks; consumers do not interpret them. +The backend owns file-version tokens — the freshness token a write/edit guards against. The policy layer stores them for stale checks; consumers do not interpret them. Both ids are branded opaque strings. ```ts type-equiv -type FsVersion = string -``` - -## Reads and editable views - -A text read is bounded by line window, byte cap, and backend limits. The returned view records whether the owner saw the whole file or only a partial page; only a `full` view authorizes later write/edit. - -```ts type-equiv -interface FsReadRequest { - offset: number - limit: number -} +type FsTargetKey = Branded<'FsTargetKey'> ``` ```ts type-equiv -interface FsTextLine { - number: number - text: string -} +type FsVersion = Branded<'FsVersion'> ``` -```ts type-equiv -type FsView = 'full' | 'partial' -``` +`stat` returns metadata (never content), or `undefined` when the target is absent. `type` lets the policy layer reject directories/special files before reading, and `size` lets it choose `readText` vs `streamText` without probing by failure. ```ts type-equiv -interface FsReadOutcome { - offset: number - limit: number - lines: FsTextLine[] - totalLines: number - truncatedByBytes?: true +interface FsInfo { version: FsVersion - view: FsView + type: 'file' | 'directory' | 'other' + size?: number } ``` -## Write and edit guards +## Write and edit guards (provider seam) -The base `FileSystem` service converts recorded state into an `FsExpectation` before calling the backend. `observed` carries the stale guard, `partial` means the owner saw a non-editable view, and `unobserved` allows create-if-absent but rejects blind overwrite. +`writeText` takes an explicit write expectation rather than inferring intent. `createIfAbsent` creates a missing target and rejects an existing one with `FS_NOT_OBSERVED`; `replaceIfVersion` replaces only when the target exists at the observed version, else `FS_STALE_VERSION`. ```ts type-equiv -type FsExpectation = - | { kind: 'observed'; version: FsVersion } - | { kind: 'partial'; version: FsVersion } - | { kind: 'unobserved' } +type FsWriteExpectation = + | { kind: 'createIfAbsent' } + | { kind: 'replaceIfVersion'; version: FsVersion } ``` ```ts type-equiv @@ -84,7 +53,7 @@ interface FsWriteOutcome { } ``` -Literal edit is a backend operation, not a `read` plus `write` composed in the tool wrapper. That keeps matching, line-ending handling, stale checks, and atomic replacement inside the filesystem seam. +`editText` is a provider-level guarded mutation, not a `read` plus `write` composed in the policy layer. It verifies the expected version BEFORE literal matching (so a stale edit reports `FS_STALE_VERSION`, not a match failure against newer content), then applies the replacement and writes atomically — keeping matching, line-ending handling, stale checks, and atomic replacement inside one mutation critical section. ```ts type-equiv interface FsEditRequest { @@ -102,26 +71,43 @@ interface FsEditOutcome { } ``` -## Observed-file state +## Execution context and read outcome (policy layer) -Observed state is keyed inside the service by owner object and `FsTarget.targetKey`. The owner is normally `exec.agent.session`, but `dsh-fs` treats it as opaque and never reads its fields. A successful read/write/edit refreshes this state for that owner. +The policy layer needs just enough execution context to derive the observed-state owner. `ToolExecution` satisfies this shape, so `dsh-tool-fs` passes its execution object through without making `dsh-file-context` import the tool, agent, or session packages. ```ts type-equiv -type FsStateSource = 'read' | 'write' | 'edit' -``` - -```ts type-equiv -interface FileState { - targetKey: string - displayPath: string - version: FsVersion - view: FsView - updatedAt: number - source: FsStateSource +interface FileContextExec { + agent?: { + session?: object + } } ``` -## Error taxonomy +A text read is bounded by line window, byte cap, and backend limits. The outcome the model-facing `read` tool renders carries the file's version at read time; there is no `full`/`partial` view — authorization is freshness-based, so any windowed read can authorize a later write/edit when the file is unchanged. + +```ts type-equiv +interface FileReadRequest { + offset: number + limit: number +} +``` + +```ts type-equiv +interface FileReadOutcome { + offset: number + limit: number + lines: FileTextLine[] + totalLines: number + truncatedByBytes?: true + version: FsVersion +} +``` + +## Observed-file state (policy layer) + +Observed state is a `WeakMap>` inside `ctx.fileContext`. An entry exists **iff** the owner has read that target through `ctx.fileContext.read`, so its presence *is* the read record — there is no separate `hasRead` flag and no view distinction. The owner is normally `exec.agent.session`, but the policy layer treats it as opaque and never reads its fields. A successful read/write/edit refreshes the recorded version for that owner; disposal drops everything (HMR safety). + +## Error taxonomy (provider seam) Filesystem failures use stable `FsErrorCode` strings carried by `FsError` (`HarnessError`). The tool registry preserves `{ name, code }` on error results, so retry, permission, and UI layers can branch without parsing text. @@ -132,14 +118,13 @@ type FsErrorCode = | 'FS_NOT_REGULAR_FILE' | 'FS_STALE_VERSION' | 'FS_NOT_OBSERVED' - | 'FS_PARTIAL_OBSERVATION' | 'FS_AMBIGUOUS_EDIT' | 'FS_EDIT_NOT_FOUND' | 'FS_ABORTED' ``` -`FS_NOT_OBSERVED` means no usable prior observation exists. `FS_PARTIAL_OBSERVATION` means the owner saw only a partial read. `FS_STALE_VERSION` means there was a prior full observation, but the backend version no longer matches. +`FS_NOT_OBSERVED` means no recorded read exists for this owner (or a `createIfAbsent` hit an existing file). `FS_STALE_VERSION` means the backend version no longer matches the observed one. Freshness authorization has no partial/full distinction, so there is no `FS_PARTIAL_OBSERVATION`. -## The service +## The services -`FileSystem` (`ctx.fs`, abstract) owns the shared orchestration: `resolve`, `readPage`, `createOrReplace`, and `applyEdit` are backend primitives; public `read`, `write`, and `edit` derive/record owner state and enforce the read-before-write/edit policy before delegating to the backend. The generated wiring catalog shows the exact service signatures on [events-and-services.md](../cordis-catalog/events-and-services.md#ctxfs--filesystem-abstract-seam). +`FileSystem` (`ctx.fs`, abstract) owns the provider primitives: `resolve`, `stat`, `readText`, `streamText`, `writeText`, and `editText`. `FileContext` (`ctx.fileContext`, concrete) injects `fs` and owns the model-facing policy: `read` windows text and records observed state, `write`/`edit` derive the freshness expectation and refresh state. The generated wiring catalog shows the exact service signatures on [events-and-services.md](../cordis-catalog/events-and-services.md#ctxfs--filesystem-abstract-seam). diff --git a/docs/module-graph.md b/docs/module-graph.md index 5a69585ef8..ad3d7f7b89 100644 --- a/docs/module-graph.md +++ b/docs/module-graph.md @@ -10,6 +10,7 @@ graph TD bash --> brand llm --> brand bash-local --> bash + fs --> brand fs --> llm llm-deepseek --> llm llm-pi-ai --> llm @@ -19,6 +20,7 @@ graph TD agent --> brand agent --> llm agent --> session + file-context --> fs fs-local --> fs llm-replay --> llm llm-replay --> session @@ -51,6 +53,7 @@ graph TD tool-bash --> bash tool-bash --> llm tool-bash --> tools + tool-fs --> file-context tool-fs --> fs tool-fs --> llm tool-fs --> system-prompt @@ -79,12 +82,13 @@ graph TD | `bash` | `brand` | | `llm` | `brand` | | `bash-local` | `bash` | -| `fs` | `llm` | +| `fs` | `brand`, `llm` | | `llm-deepseek` | `llm` | | `llm-pi-ai` | `llm` | | `session` | `brand`, `llm` | | `system-prompt` | `llm` | | `agent` | `brand`, `llm`, `session` | +| `file-context` | `fs` | | `fs-local` | `fs` | | `llm-replay` | `llm`, `session` | | `session-persistence` | `session` | @@ -96,7 +100,7 @@ graph TD | `acp` | `agent`, `llm`, `session`, `session-persistence`, `tools` | | `agent-loop` | `agent`, `llm`, `session`, `session-persistence`, `system-prompt`, `tools` | | `tool-bash` | `agent`, `bash`, `llm`, `tools` | -| `tool-fs` | `fs`, `llm`, `system-prompt`, `tools` | +| `tool-fs` | `file-context`, `fs`, `llm`, `system-prompt`, `tools` | | `agent-core` | `agent`, `agent-loop`, `invariants`, `llm`, `session`, `system-prompt`, `tool-bash`, `tools` | | `acp-agent` | `acp`, `agent-core`, `session-persistence-jsonl` | | `stdio-agent` | `agent`, `agent-core`, `session`, `session-persistence-jsonl`, `ui-stdio` | diff --git a/docs/rfc/README.md b/docs/rfc/README.md index 7f56757be0..cf9c343206 100644 --- a/docs/rfc/README.md +++ b/docs/rfc/README.md @@ -94,6 +94,7 @@ Do NOT write one for a mechanical or local choice (a variable name, a one-file r | [Prune dead methods from the persistence seam](implemented/simplification/2026-06-20-prune-dead-seam-methods.md) | 2026-06-20 | | [Keep one public stop primitive](implemented/simplification/2026-06-20-public-agent-stop-surface.md) | 2026-06-20 | | [Fold trace-only session facts into load-bearing events](implemented/simplification/2026-06-20-collapse-trace-only-session-events.md) | 2026-06-20 | +| [Split the filesystem seam — provider text mutations plus policy `ctx.fileContext`](implemented/simplification/2026-06-26-fsspec-style-fs-seam.md) | 2026-06-26 | ### Architecture diff --git a/docs/rfc/implemented/simplification/2026-06-26-fsspec-style-fs-seam.md b/docs/rfc/implemented/simplification/2026-06-26-fsspec-style-fs-seam.md new file mode 100644 index 0000000000..25ff66816f --- /dev/null +++ b/docs/rfc/implemented/simplification/2026-06-26-fsspec-style-fs-seam.md @@ -0,0 +1,120 @@ +# RFC: Split the filesystem seam — provider text mutations plus policy `ctx.fileContext` + +Status: implemented + +## Problem + +The filesystem capability from [filesystem-capability-seam](../../implemented/architecture/2026-06-17-filesystem-capability-seam.md) currently makes one abstract `FileSystem` service own two different jobs: + +1. **Provider operations** — resolving targets, stat/version metadata, text reads/streams, atomic writes, and guarded literal edits. +2. **Agent-facing policy** — line windows, literal edit semantics, and read-before-write/edit observed-state. + +That makes every future backend reimplement model-facing read semantics and observation policy. `readPage` returns numbered lines and view metadata; the base service stores per-owner file state and distinguishes `full` from `partial` reads. Those are useful policies, but they are not filesystem-provider primitives. Literal text mutation is different: version guard, literal match, ambiguity detection, and atomic rewrite must stay together inside the provider mutation boundary, but the current `applyEdit` name and surrounding seam tie that provider operation to the old read-before-edit policy shape. + +This also creates a real UX dead-end: a windowed read records `view: partial`, and partial views cannot authorize `edit`. A model that reads lines 100-150 of a large file therefore cannot edit line 120 unless it first gets a `full` read, which may be impossible for a file past the read cap. Literal edit only needs freshness: the bytes being matched must still be from the version the model read. + +The old RFC already deferred a separate `@deepseek-ai/dsh-file-context` package. This RFC builds that layer and keeps `ctx.fs` close to fsspec-style storage primitives (`info`/`cat`/`open`), without turning it into full fsspec. + +## Decision + +Split the stack into four layers: + +```text +tool dsh-tool-fs model-facing schemas + text rendering +policy dsh-file-context ctx.fileContext (concrete service): observed-state, read windowing, write/edit freshness +provider seam dsh-fs ctx.fs: text IO + guarded mutation primitives +provider dsh-fs-local local implementation of ctx.fs +``` + +`dsh-tool-fs` keeps the same model-facing `read`/`write`/`edit` schemas. It injects `fileContext`, not `fs`, and never reaches around the policy layer for model reads/writes/edits. + +## Provider Contract + +`@deepseek-ai/dsh-fs` shrinks to provider text IO plus guarded text mutation: + +```ts ignore-check +abstract resolve(path: string): Promise +abstract stat(target: FsTarget, signal?: AbortSignal): Promise +abstract readText(target: FsTarget, signal?: AbortSignal): Promise +abstract streamText(target: FsTarget, signal?: AbortSignal): Promise> +abstract writeText(target: FsTarget, content: string, expected: FsWriteExpectation, signal?: AbortSignal): Promise +abstract editText(target: FsTarget, edit: FsEditRequest, expected: { version: FsVersion }, signal?: AbortSignal): Promise + +interface FsInfo { + version: FsVersion + type: 'file' | 'directory' | 'other' + size?: number +} + +type FsWriteExpectation = + | { kind: 'createIfAbsent' } + | { kind: 'replaceIfVersion'; version: FsVersion } +``` + +`stat` returns metadata, not content. `version` is the freshness token; `type` lets the policy reject directories/special files before reading; `size` lets `ctx.fileContext.read` choose `readText` vs `streamText` without probing by failure. `undefined` means absent. + +`readText` reads the whole regular text file. `streamText` streams the same text semantics for large files. Both provider primitives own regular-file checks, UTF-8 decoding, binary/NUL rejection, and `FS_NOT_TEXT`; the policy layer never handles raw bytes or reimplements cross-chunk decoding. `readText` is the small-file/direct whole-file primitive, while large model-facing reads use `streamText`. + +`writeText` is atomic temp-file + rename with an explicit write expectation. `createIfAbsent` creates a missing target and rejects an existing target with `FS_NOT_OBSERVED`; it is the path used when the owner has no prior read. `replaceIfVersion` replaces only when the target exists at the observed version; a missing target or version mismatch throws `FS_STALE_VERSION`. + +`editText` is a provider-level guarded text mutation. It first verifies the target still exists at `expected.version`, then reads the current text, applies literal replacement, and writes atomically. The stale check must happen before literal matching so an edit based on an old read reports `FS_STALE_VERSION`, not `FS_EDIT_NOT_FOUND` or `FS_AMBIGUOUS_EDIT` from matching against newer content. Keeping this primitive on the provider seam also preserves backend-local locking and lets a future remote backend implement native compare-and-edit without forcing `ctx.fileContext` to pull the whole file through the policy layer. + +This is a *text-storage* seam, deliberately half a level above byte-level fsspec (`cat`/`open` hand back raw bytes). UTF-8 decoding, binary/NUL rejection, guarded full-file writes, and guarded literal text edits live in the provider so the policy layer never touches raw bytes, reimplements cross-chunk decoding, or separates stale checks from the mutation critical section. Model-facing concepts still stay out of the provider: no line windows, numbered lines, rendered footers, or observed-state store leak down. + +Deleted from `dsh-fs`: `readPage`, `FsExpectation`, `FsView`, `FsStateSource`, `FsReadRequest`, `FsTextLine`, line/window constants, `formatReadBody`, and the observed-state `WeakMap`. `applyEdit` is replaced by the narrower provider primitive `editText`, whose contract is version-guarded literal text mutation rather than policy-layer read authorization. The `FS_PARTIAL_OBSERVATION` code also leaves the `FsErrorCode` taxonomy: freshness authorization has no partial/full distinction, so nothing can raise it. `FsTargetKey` and `FsVersion` become branded opaque ids under the existing [branded-ids RFC](../../implemented/architecture/2026-06-20-branded-ids.md). + +## Policy Contract + +`@deepseek-ai/dsh-file-context` registers concrete service `ctx.fileContext` and injects `fs`. It is a concrete service, not a seam: it owns the read-windowing and write/edit freshness policy that does not belong on the `FileSystem` provider base class (where a sandboxed/remote backend would otherwise inherit model-facing observation policy it has no business carrying). + +Observed state lives here as `WeakMap>`. An entry exists iff the owner has read that target through `ctx.fileContext.read`, so its presence *is* the read record — there is no separate `hasRead` flag. The owner is still derived structurally from `{ agent?: { session? } }`, but that shape no longer belongs to `dsh-fs`. + +`read(target, request, exec?, signal?)` is the only read path used by the model-facing `read` tool. It stats the target, rejects absent/non-regular targets, chooses `readText` or `streamText` from `FsInfo`, builds the requested line window from text chunks, records `{ version: info.version }`, and returns the structured outcome that the tool renders. + +`write(target, content, exec?, signal?)` uses freshness policy: no recorded read calls `writeText({ kind: 'createIfAbsent' })`, so only new files can be created blindly; a recorded read calls `writeText({ kind: 'replaceIfVersion', version: vObserved })`, so existing files are replaced only if they are unchanged since the read. A successful write refreshes recorded state from the returned outcome or a post-write `stat`. + +`edit(target, edit, exec?, signal?)` requires a recorded read at `vObserved`, then calls `ctx.fs.editText(target, edit, { version: vObserved })` and refreshes recorded state from the returned version. `ctx.fileContext` does not implement literal replacement itself; it authorizes the operation and passes the observed version to the provider. The provider owns the mutation critical section, so concurrent edits based on the same observed version remain one-wins/one-stale rather than being merged or re-applied. If a backend needs a defensive whole-file edit cap, it should surface that as the same filesystem error taxonomy, but large model-facing reads should stream instead of failing just because the file is large. + +## Tool Contract + +`dsh-tool-fs` keeps the same schemas and prompt surface. `read` still exposes `file_path`, `offset`, and `limit`; `write` and `edit` are unchanged. + +The tool package only validates model args, calls `ctx.fileContext`, and renders results (`N: text`, footer, `/` envelope). The no-bypass rule is part of the contract: a model-facing `read` must call `ctx.fileContext.read`, never `ctx.fs.readText` or `ctx.fs.streamText`, so every successful read records observed-state before rendering. + +Direct `ctx.fs` calls are still allowed for non-tool consumers. They are explicit escape hatches: a direct `ctx.fs.readText` records no observed-state, so a later `ctx.fileContext.edit` rejects with `FS_NOT_OBSERVED` until the file is read through `ctx.fileContext`. + +## Concurrency Boundary + +In-process updates are safe: the local backend keeps the existing per-target mutation lock, so version-check-then-rename is serialized and a losing update sees `FS_STALE_VERSION`. + +In-process creates are guarded by the same per-target mutation lock: two callers racing with `createIfAbsent` serialize, one creates, and the next sees the target exists and receives `FS_NOT_OBSERVED`. Cross-process creates are best-effort only; a local stat-then-rename guard cannot make portable create-exclusive guarantees across all future backends. + +Cross-process writes are best-effort freshness plus atomic replacement: `mtime:size` usually catches editor saves, but same-tick same-size writes can miss; atomic temp+rename prevents torn files but not every lost update. + +## Supersedes + +This RFC reverses two decisions from [filesystem-capability-seam](../../implemented/architecture/2026-06-17-filesystem-capability-seam.md) and narrows a third: + +- Read-before-write/edit policy moves out of `ctx.fs` and into `ctx.fileContext`. +- Text reads no longer return backend-numbered line records or `full`/`partial` views; authorization is based on version freshness, so a windowed read can authorize edit when the file is unchanged. +- Literal edit no longer sits behind the old `applyEdit` API that mixed backend mutation with seam-owned observation policy. It remains a provider primitive as `editText`, because version guard + literal match + atomic rewrite must stay inside the provider's mutation critical section. + +It keeps the interface/implementation/consumer discipline, consumer-never-imports-backend rule, backend-defined target/version/display metadata, atomic local writes, and the shared `FsError` taxonomy. + +## Acceptance Criteria + +- `dsh-fs` exposes exactly `resolve`/`stat`/`readText`/`streamText`/`writeText`/`editText`; `stat` returns `FsInfo | undefined`; `writeText` uses `FsWriteExpectation` (`createIfAbsent` or `replaceIfVersion`); removed types/primitives are gone, and the old `applyEdit` API is replaced by `editText`. +- `dsh-file-context` registers `ctx.fileContext`, owns observed-state plus `read`/`write`/`edit` policy, injects `fs`, and has HMR/disposal coverage. +- `dsh-tool-fs` injects `fileContext`; model-facing schemas stay byte-for-byte unchanged; the no-bypass contract and escape-hatch contract are documented and tested. +- Windowed read authorizing edit is shown to fail on the pre-refit code and pass after the refit. Existing version-CAS behavior is preserved with a regression test; it is not claimed as a pre-refit failure. An edit based on a stale read must report `FS_STALE_VERSION` before attempting literal matching. +- `dsh-fs-local` carries no line, view, or `formatReadBody` logic; it does carry provider-level `editText` logic. +- Docs and generated artifacts are updated: `docs/architecture.md`, `packages/README.md`, fs package READMEs, `docs/core-data-structures/filesystem.md`, affected `type-equiv` blocks and `scripts/type-equiv.manifest.json`, Cordis catalog, module graph, and doc references. +- Gates stay green: normal `doc-sync`, `pnpm run knip`, and `pnpm run test:coverage` with 100% per-file coverage. + +## Risks + +- Adds a fourth fs package and a new service. This is intentional: it is the previously deferred policy layer, not a second abstract backend seam. +- Direct `ctx.fs` use can surprise callers who later use `ctx.fileContext`. The failure is explicit (`FS_NOT_OBSERVED`) and documented. +- Large-file line windowing moves from the backend to `ctx.fileContext.read`; text decoding and binary rejection stay in `ctx.fs.streamText`, so this is relocation of windowing only, not a second text-IO implementation. +- Keeping `editText` in the provider seam means every backend must implement the literal replacement contract. This is intentional: the operation is not pure storage, but stale guard + literal match + atomic rewrite is the unit that must stay together for correct error attribution and concurrency behavior. The contract should stay narrow and text-only so future backends can implement it natively or by whole-file rewrite. +- Freshness permits full-file `write` after a windowed read. That is weaker than the old view check, but avoids making large files impossible to edit; prompt guidance should still discourage blind full replaces. diff --git a/examples/coding-agent/cordis.yml b/examples/coding-agent/cordis.yml index 497aa8896a..57c1e3dd2f 100644 --- a/examples/coding-agent/cordis.yml +++ b/examples/coding-agent/cordis.yml @@ -25,8 +25,8 @@ apiKey: !!js process.env.DEEPSEEK_API_KEY baseURL: !!js process.env.DEEPSEEK_BASE_URL models: - - deepseek-v4-flash - deepseek-v4-pro + - deepseek-v4-flash # Local bash executor (the model's only tool, via agent-core's tool-bash schema). - id: bash diff --git a/packages/README.md b/packages/README.md index dd418f97ca..265fdd8676 100644 --- a/packages/README.md +++ b/packages/README.md @@ -31,9 +31,10 @@ dsh-agent ← dsh-llm, dsh-session, dsh-brand dsh-tools ← dsh-llm, dsh-system-prompt, dsh-agent dsh-bash-local ← dsh-bash (BashExecutor impl) dsh-tool-bash ← dsh-bash, dsh-tools (bash tool schemas) -dsh-fs ← dsh-llm (abstract filesystem seam) +dsh-fs ← dsh-llm, dsh-brand (filesystem provider seam) dsh-fs-local ← dsh-fs (FileSystem impl) -dsh-tool-fs ← dsh-fs, dsh-tools (file tool schemas) +dsh-file-context ← dsh-fs (read windowing + write/edit freshness policy) +dsh-tool-fs ← dsh-file-context, dsh-fs, dsh-tools (file tool schemas) dsh-llm-deepseek ← dsh-llm (DeepSeek adapter) dsh-llm-pi-ai ← dsh-llm (pi-ai-backed adapter) dsh-agent-loop ← dsh-llm, dsh-session, dsh-system-prompt, dsh-tools, dsh-agent @@ -62,8 +63,9 @@ The rule: **extension** plugins depend on interfaces, never on the concrete loop | `bash/` | `bash` | Abstract bash executor seam (interface + vocabulary) | `ctx.bash` | | `bash-local/` | `bash` | Local-subprocess `BashExecutor` implementation | (registers `ctx.bash`) | | `tool-bash/` | `bash` | Model-facing `bash`/`bash_output`/`bash_kill` tool schemas | (registers on `ctx.tools`) | -| `fs/` | `fs` | Abstract filesystem seam (interface + vocabulary + observed-file policy) | `ctx.fs` | +| `fs/` | `fs` | Filesystem provider seam: text IO + guarded mutation primitives | `ctx.fs` | | `fs-local/` | `fs` | Local-filesystem `FileSystem` implementation | (registers `ctx.fs`) | +| `file-context/` | `fs` | Policy layer: read windowing, observed-state, write/edit freshness | `ctx.fileContext` | | `tool-fs/` | `fs` | Model-facing `read`/`write`/`edit` tool schemas | (registers on `ctx.tools`) | | `llm-deepseek/` | `llm` | DeepSeek API adapter (hand-rolled fetch/SSE) | (registers on `ctx.llm`) | | `llm-pi-ai/` | `llm` | DeepSeek adapter via `@earendil-works/pi-ai` (design twin) | (registers on `ctx.llm`) | diff --git a/packages/fs/README.md b/packages/fs/README.md index 15757698b2..a793a94b4b 100644 --- a/packages/fs/README.md +++ b/packages/fs/README.md @@ -1,11 +1,12 @@ # fs/ - filesystem capability family -The filesystem capability seam: an abstract filesystem interface, a local implementation, and the model-facing file tools. All **product** packages. +The filesystem stack: a provider seam (text IO + guarded mutation), a local implementation, a policy layer (read windowing + write/edit freshness), and the model-facing file tools. All **product** packages. | Package | Role | ctx key | |---|---|---| -| `fs/` | Abstract filesystem seam (interface + vocabulary + observed-file policy) | `ctx.fs` | +| `fs/` | Provider seam: text IO + guarded mutation primitives | `ctx.fs` | | `fs-local/` | Local-filesystem `FileSystem` implementation | (registers `ctx.fs`) | +| `file-context/` | Policy layer: observed-state, read windowing, write/edit freshness | `ctx.fileContext` | | `tool-fs/` | Model-facing `read`/`write`/`edit` tool schemas | (registers on `ctx.tools`) | -The interface lives at `fs/fs/`. A sandboxed, remote, or project-scoped filesystem backend can replace `fs-local` without touching the interface or model-facing tool schemas. +The interface lives at `fs/fs/`. A sandboxed, remote, or project-scoped filesystem backend can replace `fs-local` without touching the seam, the policy layer, or the model-facing tool schemas. The policy layer (`file-context/`) is a concrete service, not a swappable seam — it owns the model-facing observation policy that does not belong on a provider backend. diff --git a/packages/fs/file-context/README.md b/packages/fs/file-context/README.md new file mode 100644 index 0000000000..373fb2759b --- /dev/null +++ b/packages/fs/file-context/README.md @@ -0,0 +1,43 @@ +# @deepseek-ai/dsh-file-context + +The **file-context policy layer**: a concrete `ctx.fileContext` service that owns model-facing read windowing and write/edit freshness on top of the `ctx.fs` provider seam ([`@deepseek-ai/dsh-fs`](../fs)). This is the policy third of the filesystem stack — it is **not** a swappable seam, but the deferred policy layer that does not belong on the `FileSystem` provider base class. + +```ts +import type { Context } from 'cordis' +import FileContext from '@deepseek-ai/dsh-file-context' + +declare const ctx: Context + +// A ctx.fs provider must already be loaded (e.g. @deepseek-ai/dsh-fs-local); +// FileContext injects `fs` and registers ctx.fileContext. Load +// @deepseek-ai/dsh-tool-fs afterwards to expose read/write/edit to the model. +await ctx.plugin(FileContext) +``` + +## The four-layer split + +| Layer | Package | Role | +|---|---|---| +| tool | `@deepseek-ai/dsh-tool-fs` | model-facing schemas + text rendering | +| policy | `@deepseek-ai/dsh-file-context` (this) | `ctx.fileContext`: observed-state, read windowing, write/edit freshness | +| provider seam | `@deepseek-ai/dsh-fs` | `ctx.fs`: text IO + guarded mutation primitives | +| provider | `@deepseek-ai/dsh-fs-local` | local implementation of `ctx.fs` | + +## Service API (`ctx.fileContext`) + +| Member | Semantics | +|---|---| +| `read(target, request, exec?, signal?)` | Stats the target, rejects absent/non-regular targets, chooses `readText`/`streamText` by size, builds the requested line window, records the version, and returns the `FileReadOutcome` the tool renders. | +| `write(target, content, exec?, signal?)` | No recorded read → `writeText({ kind: 'createIfAbsent' })` (only new files create blindly); a recorded read → `writeText({ kind: 'replaceIfVersion', version })`. Refreshes recorded state on success. | +| `edit(target, edit, exec?, signal?)` | Requires a recorded read by this owner (else `FS_NOT_OBSERVED`); passes the observed version to `ctx.fs.editText` as the stale guard and refreshes recorded state. | +| `owner(exec?)` | Derives the observed-state owner (`exec.agent.session`) — `undefined` when there is none. | + +## Observed state is the read record, freshness is the authorization + +Observed state is a `WeakMap>`. An entry exists **iff** the owner has read that target through `read`, so its presence *is* the read record — there is no `hasRead` flag and no `full`/`partial` view. Authorization is based on version freshness only: a windowed read of lines 100-150 records the file's version, and a later edit of line 120 is authorized as long as the file is unchanged (the provider's stale guard enforces it). State is held weakly and dropped on disposal (HMR safety); persistence across sessions is deferred. + +## The no-bypass contract + +A model-facing read MUST go through `ctx.fileContext.read`, never `ctx.fs.readText`/`streamText`, so every successful read records observed state before the tool renders. Direct `ctx.fs` calls remain an explicit escape hatch for non-tool consumers: a direct `ctx.fs.readText` records nothing, so a later `ctx.fileContext.edit` rejects with `FS_NOT_OBSERVED` until the file is read through `ctx.fileContext`. + +The line-windowing mechanics live in `src/window.ts` (Cordis-free, independently unit-tested); `src/index.ts` is the service wiring and policy. diff --git a/packages/fs/file-context/package.json b/packages/fs/file-context/package.json new file mode 100644 index 0000000000..77c905703b --- /dev/null +++ b/packages/fs/file-context/package.json @@ -0,0 +1,31 @@ +{ + "name": "@deepseek-ai/dsh-file-context", + "description": "File-context policy layer (ctx.fileContext) for the DeepSeek Harness — read windowing and write/edit freshness over the ctx.fs provider seam", + "version": "0.0.1", + "private": true, + "type": "module", + "main": "lib/index.js", + "types": "lib/index.d.ts", + "exports": { + ".": { + "types": "./lib/index.d.ts", + "default": "./lib/index.js" + }, + "./src/*": "./src/*", + "./package.json": "./package.json" + }, + "files": [ + "lib", + "src" + ], + "license": "BSD-3-Clause", + "peerDependencies": { + "@deepseek-ai/dsh-fs": "^0.0.1", + "cordis": "^4.0.0-rc.6" + }, + "devDependencies": { + "@deepseek-ai/dsh-fs": "workspace:^", + "@deepseek-ai/dsh-llm": "workspace:^", + "cordis": "^4.0.0-rc.6" + } +} diff --git a/packages/fs/file-context/src/index.ts b/packages/fs/file-context/src/index.ts new file mode 100644 index 0000000000..6dbb33d1d2 --- /dev/null +++ b/packages/fs/file-context/src/index.ts @@ -0,0 +1,184 @@ +/** + * The file-context policy layer (`ctx.fileContext`): a concrete service that + * owns model-facing read windowing and write/edit freshness on top of the + * `ctx.fs` provider seam. It is NOT a swappable seam — it is the previously + * deferred policy layer that does not belong on the `FileSystem` provider base + * class (where a sandboxed/remote backend would otherwise inherit model-facing + * observation policy it has no business carrying). + * + * ## Observed state IS the read record + * + * Observed state lives here as `WeakMap>`. An + * entry exists iff the owner has read that target through {@link read}, so its + * presence *is* the read record — there is no separate `hasRead` flag. The owner + * is derived structurally from `{ agent?: { session? } }` and held weakly, so a + * collected session frees its state; disposal drops everything (HMR safety). + * + * ## Freshness, not full/partial views + * + * Authorization is based on version freshness only. A windowed read records the + * file's version, and any later write/edit at that version is authorized — a + * model that read lines 100-150 of a large file can still edit line 120 as long + * as the file is unchanged. There is no `full`/`partial` distinction: the bytes + * the edit matches must merely come from the version the model read, which the + * provider's stale guard enforces. + * + * ## The no-bypass contract + * + * A model-facing read MUST go through {@link read} (never `ctx.fs.readText`/ + * `streamText` directly), so every successful read records observed state before + * the tool renders. Direct `ctx.fs` calls are allowed for non-tool consumers but + * record nothing, so a later {@link edit} rejects with `FS_NOT_OBSERVED` until + * the file is read through `ctx.fileContext`. + * + * @module @deepseek-ai/dsh-file-context + */ + +import { Context, Service } from 'cordis' +import { FsError } from '@deepseek-ai/dsh-fs' +import type { FsTarget, FsVersion, FsEditRequest, FsEditOutcome, FsWriteOutcome } from '@deepseek-ai/dsh-fs' +import { buildWindow } from './window.ts' +import type { FileContextExec, FileReadRequest, FileReadOutcome } from './types.ts' + +export type { FileTextLine, ReadWindow, WindowResult } from './window.ts' +export { READ_MAX_BYTES, READ_MAX_LINE_LENGTH, buildWindow } from './window.ts' +export type { FileContextExec, FileReadRequest, FileReadOutcome } from './types.ts' + +/** Files at or above this size stream; smaller files read whole into memory. */ +export const STREAM_MIN_SIZE = 10 * 1024 * 1024 + +declare module 'cordis' { + interface Context { + fileContext: FileContext + } +} + +/** What an owner has observed about one target: just the version it last saw. */ +interface ObservedState { + version: FsVersion +} + +/** + * The file-context policy service. Injects `fs`, registers as `ctx.fileContext`, + * and is the only read/write/edit path the model-facing tools use. + */ +export class FileContext extends Service { + static inject = ['fs'] + + /** + * Observed-file state, keyed first by the owner object (weakly held, so a + * collected session frees its state), then by {@link FsTarget.targetKey}. An + * entry's PRESENCE is the read record. + */ + private observed = new WeakMap>() + + constructor(ctx: Context) { + super(ctx, 'fileContext') + ctx.effect(() => () => { + // Drop all recorded state on disposal so a reloaded service starts clean + // (HMR safety). The WeakMap itself would be GC'd, but replacing it makes + // the release observable and immediate for tests. + this.observed = new WeakMap() + }, 'fileContext observed-state teardown') + } + + /** + * Derive the observed-state owner from an execution context — normally the + * active agent session. `undefined` when no owner can be derived (e.g. a + * direct tool call with no agent); such calls read freely but cannot satisfy + * the write/edit prior-observation policy. + */ + owner(exec?: FileContextExec): object | undefined { + return exec?.agent?.session + } + + private getObserved(owner: object, targetKey: string): ObservedState | undefined { + return this.observed.get(owner)?.get(targetKey) + } + + private record(owner: object, targetKey: string, version: FsVersion): void { + let byTarget = this.observed.get(owner) + if (!byTarget) { + byTarget = new Map() + this.observed.set(owner, byTarget) + } + byTarget.set(targetKey, { version }) + } + + /** + * Resolve a path into a stable {@link FsTarget}, delegating to the provider. + * Exposed here so the model-facing tools never need to inject `ctx.fs` + * directly — they resolve and then read/write/edit entirely through + * `ctx.fileContext`. + */ + async resolve(path: string): Promise { + return this.ctx.fs.resolve(path) + } + + /** + * Read a bounded line window from a target. Stats first (rejecting an absent + * target with `FS_NOT_FOUND` and a non-regular one with `FS_NOT_REGULAR_FILE`), + * chooses `readText` vs `streamText` by size, builds the window, and — when an + * owner is derivable — records the version so a later write/edit is authorized. + */ + async read(target: FsTarget, request: FileReadRequest, exec?: FileContextExec, signal?: AbortSignal): Promise { + const info = await this.ctx.fs.stat(target, signal) + if (!info) throw new FsError(`cannot read "${target.displayPath}": not found`, 'FS_NOT_FOUND') + if (info.type !== 'file') throw new FsError(`cannot read "${target.displayPath}": not a regular file`, 'FS_NOT_REGULAR_FILE') + + const chunks = info.size !== undefined && info.size >= STREAM_MIN_SIZE + ? await this.ctx.fs.streamText(target, signal) + : [await this.ctx.fs.readText(target, signal)] + const window = await buildWindow(chunks, request, target.displayPath) + + const owner = this.owner(exec) + if (owner) this.record(owner, target.targetKey, info.version) + return { + offset: request.offset, + limit: request.limit, + lines: window.lines, + totalLines: window.totalLines, + version: info.version, + ...window.truncatedByBytes ? { truncatedByBytes: true } : {}, + } + } + + /** + * Create or fully replace a file. With no recorded read, writes + * `createIfAbsent` (only new files can be created blindly); with a recorded + * read, writes `replaceIfVersion` at the observed version (existing files are + * replaced only if unchanged since the read). Refreshes recorded state from + * the returned version on success. + */ + async write(target: FsTarget, content: string, exec?: FileContextExec, signal?: AbortSignal): Promise { + const owner = this.owner(exec) + const prior = owner ? this.getObserved(owner, target.targetKey) : undefined + const outcome = await this.ctx.fs.writeText( + target, + content, + prior ? { kind: 'replaceIfVersion', version: prior.version } : { kind: 'createIfAbsent' }, + signal, + ) + if (owner) this.record(owner, target.targetKey, outcome.version) + return outcome + } + + /** + * Apply a literal edit. Requires a recorded read by this owner (else + * `FS_NOT_OBSERVED`); passes the observed version to `ctx.fs.editText` as the + * stale guard and refreshes recorded state from the returned version. The + * provider owns the mutation critical section and the literal match. + */ + async edit(target: FsTarget, edit: FsEditRequest, exec?: FileContextExec, signal?: AbortSignal): Promise { + const owner = this.owner(exec) + const prior = owner ? this.getObserved(owner, target.targetKey) : undefined + if (!owner || !prior) { + throw new FsError(`edit requires reading "${target.displayPath}" first`, 'FS_NOT_OBSERVED') + } + const outcome = await this.ctx.fs.editText(target, edit, { version: prior.version }, signal) + this.record(owner, target.targetKey, outcome.version) + return outcome + } +} + +export default FileContext diff --git a/packages/fs/file-context/src/types.ts b/packages/fs/file-context/src/types.ts new file mode 100644 index 0000000000..842d9a08c7 --- /dev/null +++ b/packages/fs/file-context/src/types.ts @@ -0,0 +1,56 @@ +/** + * Vocabulary for the file-context policy layer (`ctx.fileContext`): the + * minimal execution-context shape used to derive an observed-state owner, the + * resolved read window, and the structured read outcome the model-facing `read` + * tool renders. + * + * The provider vocabulary (`FsTarget`, `FsVersion`, write/edit shapes) is + * re-used from `@deepseek-ai/dsh-fs` — this package owns only the model-facing + * read-windowing and observation policy on top of it. + * + * @module @deepseek-ai/dsh-file-context/types + */ + +import type { FsVersion } from '@deepseek-ai/dsh-fs' +import type { FileTextLine } from './window.ts' + +/** + * Minimal structural view of a tool execution the policy layer needs to derive + * an observed-state owner. `@deepseek-ai/dsh-tools`' `ToolExecution` satisfies + * this shape, so the consumer passes its `exec` straight through without + * `dsh-file-context` importing `dsh-tools`, `dsh-agent`, or `dsh-session`. + * + * The owner is `agent.session` when present. It is treated as an opaque object + * identity (a `WeakMap` key); this package never reads any of its fields. + */ +export interface FileContextExec { + /** The agent on whose behalf the call runs, when there is one. */ + agent?: { + /** The session that owns observed-file state, used as an opaque key. */ + session?: object + } +} + +/** Resolved read window. The consumer applies its defaults/caps before calling. */ +export interface FileReadRequest { + /** 1-based first line to return. */ + offset: number + /** Maximum number of lines to return. */ + limit: number +} + +/** Outcome of a bounded text read — what the model-facing `read` tool renders. */ +export interface FileReadOutcome { + /** 1-based first line requested. */ + offset: number + /** Maximum number of lines requested. */ + limit: number + /** Returned lines, already numbered. */ + lines: FileTextLine[] + /** Total line count in the file, unless `truncatedByBytes` stopped scanning early. */ + totalLines: number + /** Whether selected output hit the byte cap before EOF or the requested limit. */ + truncatedByBytes?: true + /** Opaque version of the file at read time. */ + version: FsVersion +} diff --git a/packages/fs/file-context/src/window.ts b/packages/fs/file-context/src/window.ts new file mode 100644 index 0000000000..97e51e2ee4 --- /dev/null +++ b/packages/fs/file-context/src/window.ts @@ -0,0 +1,139 @@ +/** + * Cordis-free line-windowing for `@deepseek-ai/dsh-file-context`. Relocated + * from the local backend: turning a file's decoded text into a bounded, + * line-numbered window (offset/limit, byte cap, per-line truncation) is + * model-facing READ POLICY, not a storage primitive, so it lives in the policy + * layer rather than in every `ctx.fs` backend. + * + * The provider (`ctx.fs.readText`/`streamText`) hands back already-decoded text + * (UTF-8 validated, binary rejected); this module only scans that text for + * newlines and builds the requested window. A capped line buffer means a + * newline-free giant line can never balloon memory even when streamed. + * + * @module @deepseek-ai/dsh-file-context/window + */ + +import { FsError } from '@deepseek-ai/dsh-fs' + +/** Maximum characters returned for a single line. */ +export const READ_MAX_LINE_LENGTH = 2000 + +/** Maximum bytes returned for selected file lines. */ +export const READ_MAX_BYTES = 50 * 1024 + +const READ_MAX_LINE_SUFFIX = `... (line truncated to ${READ_MAX_LINE_LENGTH} chars)` +const LINE_BUFFER_CAP = READ_MAX_LINE_LENGTH + 1 + +/** Resolved read window. The consumer applies its defaults/caps before calling. */ +export interface ReadWindow { + /** 1-based first line to return. */ + offset: number + /** Maximum number of lines to return. */ + limit: number +} + +/** One line returned from a text file. */ +export interface FileTextLine { + /** 1-based line number in the file. */ + number: number + /** Line text without its trailing newline. */ + text: string +} + +/** The windowed result this module builds from a file's decoded text. */ +export interface WindowResult { + /** Returned lines, already numbered. */ + lines: FileTextLine[] + /** Total line count in the file, unless `truncatedByBytes` stopped scanning early. */ + totalLines: number + /** Whether selected output hit the byte cap before EOF or the requested limit. */ + truncatedByBytes: boolean +} + +interface WindowAccumulator { + lines: FileTextLine[] + totalLines: number + outputBytes: number + truncatedByBytes: boolean + done: boolean +} + +function newAccumulator(): WindowAccumulator { + return { lines: [], totalLines: 0, outputBytes: 0, truncatedByBytes: false, done: false } +} + +function truncateLine(line: string): string { + return line.length > READ_MAX_LINE_LENGTH ? `${line.substring(0, READ_MAX_LINE_LENGTH)}${READ_MAX_LINE_SUFFIX}` : line +} + +function lineByteSize(line: string, currentLineCount: number): number { + return Buffer.byteLength(line, 'utf8') + (currentLineCount > 0 ? 1 : 0) +} + +function consumeLine(acc: WindowAccumulator, rawLine: string, request: ReadWindow): void { + acc.totalLines += 1 + if (acc.totalLines < request.offset || acc.lines.length >= request.limit) return + + const text = truncateLine(rawLine) + const bytes = lineByteSize(text, acc.lines.length) + if (acc.outputBytes + bytes > READ_MAX_BYTES) { + acc.truncatedByBytes = true + acc.done = true + return + } + acc.outputBytes += bytes + acc.lines.push({ number: acc.totalLines, text }) +} + +function stripCarriageReturn(line: string): string { + return line.endsWith('\r') ? line.slice(0, -1) : line +} + +function finish(acc: WindowAccumulator, request: ReadWindow, displayPath: string): WindowResult { + if (!acc.truncatedByBytes && request.offset > acc.totalLines && !(acc.totalLines === 0 && request.offset === 1)) { + throw new FsError(`offset ${request.offset} is out of range for "${displayPath}" (${acc.totalLines} lines)`, 'FS_NOT_FOUND') + } + return { lines: acc.lines, totalLines: acc.totalLines, truncatedByBytes: acc.truncatedByBytes } +} + +/** + * Build a bounded, line-numbered window from a file's decoded text chunks. + * Accepts an `AsyncIterable` (a chunked `streamText`) or an + * `Iterable` (a whole-file `readText` wrapped as `[text]`), so one code + * path serves both. Scans for newlines with a capped line buffer (a newline-free + * giant line is truncated, never buffered past {@link READ_MAX_LINE_LENGTH}), + * enforces the byte cap, and throws `FS_NOT_FOUND` for an offset past EOF. + */ +export async function buildWindow( + chunks: AsyncIterable | Iterable, + request: ReadWindow, + displayPath: string, +): Promise { + const acc = newAccumulator() + let lineBuffer = '' + + function appendToLineBuffer(segment: string): void { + if (lineBuffer.length >= LINE_BUFFER_CAP) return + lineBuffer += segment + if (lineBuffer.length > LINE_BUFFER_CAP) lineBuffer = lineBuffer.slice(0, LINE_BUFFER_CAP) + } + + function flushLine(): void { + consumeLine(acc, stripCarriageReturn(lineBuffer), request) + lineBuffer = '' + } + + for await (const chunk of chunks) { + let startPos = 0 + let newlinePos: number + while ((newlinePos = chunk.indexOf('\n', startPos)) !== -1) { + appendToLineBuffer(chunk.slice(startPos, newlinePos)) + flushLine() + startPos = newlinePos + 1 + if (acc.done) return finish(acc, request, displayPath) + } + appendToLineBuffer(chunk.slice(startPos)) + } + if (lineBuffer.length > 0) flushLine() + return finish(acc, request, displayPath) +} diff --git a/packages/fs/file-context/tests/policy.spec.ts b/packages/fs/file-context/tests/policy.spec.ts new file mode 100644 index 0000000000..e15398eb54 --- /dev/null +++ b/packages/fs/file-context/tests/policy.spec.ts @@ -0,0 +1,305 @@ +/** + * Tests for the file-context policy layer: registration/disposal/HMR, owner + * derivation, observed-state-as-read-record, read windowing over a fake + * provider, freshness-based write/edit authorization (including the key + * windowed-read-authorizes-edit behavior), the read→streamText size routing, + * and multi-owner isolation. The provider is a fake `ctx.fs` recording the + * expectations it was handed. + */ + +import { describe, expect, it } from 'vitest' +import { Context } from 'cordis' +import { FileSystem, FsError, FsTargetKey, FsVersion } from '@deepseek-ai/dsh-fs' +import type { + FsEditOutcome, + FsEditRequest, + FsInfo, + FsTarget, + FsWriteExpectation, + FsWriteOutcome, +} from '@deepseek-ai/dsh-fs' +import FileContext, { STREAM_MIN_SIZE } from '@deepseek-ai/dsh-file-context' +import type { FileContextExec, FileReadRequest } from '@deepseek-ai/dsh-file-context' + +/** A fake provider: in-memory files, recording every expectation/version it is handed. */ +class FakeFs extends FileSystem { + files = new Map() + versions = new Map() + /** Size to report from stat (lets a test push read onto the streaming path). */ + reportSize?: number + /** Whether streamText was used for the last read (vs readText). */ + lastReadStreamed = false + writeExpectations: FsWriteExpectation[] = [] + editExpectedVersions: string[] = [] + + private ver(key: string): FsVersion { + return FsVersion(`v${this.versions.get(key) ?? 0}`) + } + private bump(key: string): FsVersion { + const next = (this.versions.get(key) ?? 0) + 1 + this.versions.set(key, next) + return FsVersion(`v${next}`) + } + + override async resolve(path: string): Promise { + return { inputPath: path, targetKey: FsTargetKey(path), displayPath: path } + } + override async stat(target: FsTarget): Promise { + const content = this.files.get(target.targetKey) + if (content === undefined) return undefined + return { version: this.ver(target.targetKey), type: 'file', size: this.reportSize ?? content.length } + } + override async readText(target: FsTarget): Promise { + this.lastReadStreamed = false + return this.files.get(target.targetKey) ?? '' + } + override async streamText(target: FsTarget): Promise> { + this.lastReadStreamed = true + const content = this.files.get(target.targetKey) ?? '' + return (async function* () { yield content })() + } + override async writeText(target: FsTarget, content: string, expected: FsWriteExpectation): Promise { + this.writeExpectations.push(expected) + const existed = this.files.has(target.targetKey) + this.files.set(target.targetKey, content) + return { operation: existed ? 'update' : 'create', version: this.bump(target.targetKey) } + } + override async editText(target: FsTarget, edit: FsEditRequest, expected: { version: FsVersion }): Promise { + this.editExpectedVersions.push(expected.version) + const content = this.files.get(target.targetKey) ?? '' + this.files.set(target.targetKey, content.split(edit.oldString).join(edit.newString)) + return { replacements: 1, replaceAll: edit.replaceAll, version: this.bump(target.targetKey) } + } +} + +async function setup() { + const ctx = new Context() + await ctx.plugin(FakeFs) + await ctx.plugin(FileContext) + const fs = ctx.fs as FakeFs + const fileContext = ctx.fileContext + return { ctx, fs, fileContext } +} + +const READ_ALL: FileReadRequest = { offset: 1, limit: 2000 } +const ownerExec = (session: object): FileContextExec => ({ agent: { session } }) + +describe('registration / disposal', () => { + it('registers as ctx.fileContext and injects fs', async () => { + const { fileContext } = await setup() + expect(fileContext).toBeDefined() + }) + + it('stays pending until ctx.fs exists', async () => { + const ctx = new Context() + await ctx.plugin(FileContext) // no fs provider + expect(ctx.fileContext).toBeUndefined() + }) + + it('withdraws ctx.fileContext when its fiber is disposed (HMR safety)', async () => { + const ctx = new Context() + await ctx.plugin(FakeFs) + const fiber = await ctx.plugin(FileContext) + expect(ctx.fileContext).toBeDefined() + await fiber.dispose() + expect(ctx.fileContext).toBeUndefined() + }) +}) + +describe('owner derivation', () => { + it('derives the owner from exec.agent.session', async () => { + const { fileContext } = await setup() + const session = {} + expect(fileContext.owner(ownerExec(session))).toBe(session) + }) + + it('returns undefined with no exec, no agent, or no session', async () => { + const { fileContext } = await setup() + expect(fileContext.owner()).toBeUndefined() + expect(fileContext.owner({})).toBeUndefined() + expect(fileContext.owner({ agent: {} })).toBeUndefined() + }) +}) + +describe('read', () => { + it('returns a windowed outcome and rejects an absent target', async () => { + const { fs, fileContext } = await setup() + fs.files.set('a.txt', 'one\ntwo') + const outcome = await fileContext.read(await fs.resolve('a.txt'), READ_ALL) + expect(outcome.lines).toEqual([{ number: 1, text: 'one' }, { number: 2, text: 'two' }]) + expect(outcome.version).toBe('v0') + + await expect(fileContext.read(await fs.resolve('missing.txt'), READ_ALL)) + .rejects.toMatchObject({ code: 'FS_NOT_FOUND' }) + }) + + it('rejects a non-regular target', async () => { + const { fs, fileContext } = await setup() + fs.files.set('d', '') + const target = await fs.resolve('d') + // Force stat to report a directory. + fs.stat = async () => ({ version: FsVersion('v0'), type: 'directory' }) + await expect(fileContext.read(target, READ_ALL)).rejects.toMatchObject({ code: 'FS_NOT_REGULAR_FILE' }) + }) + + it('reads small files whole and large files via streamText', async () => { + const { fs, fileContext } = await setup() + fs.files.set('a.txt', 'one\ntwo') + + await fileContext.read(await fs.resolve('a.txt'), READ_ALL) + expect(fs.lastReadStreamed).toBe(false) + + fs.reportSize = STREAM_MIN_SIZE + await fileContext.read(await fs.resolve('a.txt'), READ_ALL) + expect(fs.lastReadStreamed).toBe(true) + }) + + it('surfaces truncatedByBytes when the window hits the byte cap', async () => { + const { fs, fileContext } = await setup() + fs.files.set('big.txt', Array.from({ length: 2000 }, () => 'y'.repeat(100)).join('\n')) + const outcome = await fileContext.read(await fs.resolve('big.txt'), READ_ALL) + expect(outcome.truncatedByBytes).toBe(true) + }) +}) + +describe('observed-state is the read record', () => { + it('a read authorizes a later in-place write at the observed version', async () => { + const { fs, fileContext } = await setup() + const exec = ownerExec({}) + fs.files.set('a.txt', 'hello') + const target = await fs.resolve('a.txt') + + await fileContext.read(target, READ_ALL, exec) + await fileContext.write(target, 'goodbye', exec) + + expect(fs.writeExpectations).toEqual([{ kind: 'replaceIfVersion', version: 'v0' }]) + }) + + it('a windowed (partial) read still authorizes edit — freshness, not full/partial', async () => { + const { fs, fileContext } = await setup() + const exec = ownerExec({}) + fs.files.set('a.txt', 'one\ntwo\nthree\nfour') + const target = await fs.resolve('a.txt') + + // Read only lines 2-3 — a partial window. + const outcome = await fileContext.read(target, { offset: 2, limit: 2 }, exec) + expect(outcome.lines.map(l => l.number)).toEqual([2, 3]) + + // Edit is authorized anyway: the file is unchanged since the read. + await fileContext.edit(target, { oldString: 'one', newString: 'X', replaceAll: false }, exec) + expect(fs.editExpectedVersions).toEqual(['v0']) + }) + + it('skips recording when there is no owner, so write is createIfAbsent', async () => { + const { fs, fileContext } = await setup() + fs.files.set('a.txt', 'hello') + const target = await fs.resolve('a.txt') + + await fileContext.read(target, READ_ALL) // no exec + // No recorded read → createIfAbsent → the provider rejects an existing target. + fs.writeText = async () => { throw new FsError('exists', 'FS_NOT_OBSERVED') } + await expect(fileContext.write(target, 'x')).rejects.toMatchObject({ code: 'FS_NOT_OBSERVED' }) + }) +}) + +describe('write policy', () => { + it('a create (no prior read) uses createIfAbsent', async () => { + const { fs, fileContext } = await setup() + const exec = ownerExec({}) + const target = await fs.resolve('new.txt') + const outcome = await fileContext.write(target, 'fresh', exec) + expect(outcome.operation).toBe('create') + expect(fs.writeExpectations).toEqual([{ kind: 'createIfAbsent' }]) + }) + + it('refreshes state after a write, so a follow-up edit needs no re-read', async () => { + const { fs, fileContext } = await setup() + const exec = ownerExec({}) + const target = await fs.resolve('a.txt') + await fileContext.write(target, 'one', exec) // create → state now at v1 + await fileContext.edit(target, { oldString: 'one', newString: 'two', replaceAll: false }, exec) + expect(fs.editExpectedVersions).toEqual(['v1']) + }) +}) + +describe('edit policy', () => { + it('rejects with FS_NOT_OBSERVED when the file was never read', async () => { + const { fs, fileContext } = await setup() + const exec = ownerExec({}) + fs.files.set('a.txt', 'hello') + const target = await fs.resolve('a.txt') + await expect(fileContext.edit(target, { oldString: 'hello', newString: 'bye', replaceAll: false }, exec)) + .rejects.toMatchObject({ code: 'FS_NOT_OBSERVED' }) + }) + + it('rejects when there is no owner (cannot prove prior observation)', async () => { + const { fs, fileContext } = await setup() + fs.files.set('a.txt', 'hello') + const target = await fs.resolve('a.txt') + await expect(fileContext.edit(target, { oldString: 'hello', newString: 'bye', replaceAll: false })) + .rejects.toMatchObject({ code: 'FS_NOT_OBSERVED' }) + }) + + it('passes the recorded version as the stale guard after a read', async () => { + const { fs, fileContext } = await setup() + const exec = ownerExec({}) + fs.files.set('a.txt', 'hello') + fs.versions.set('a.txt', 7) + const target = await fs.resolve('a.txt') + await fileContext.read(target, READ_ALL, exec) + await fileContext.edit(target, { oldString: 'hello', newString: 'bye', replaceAll: false }, exec) + expect(fs.editExpectedVersions).toEqual(['v7']) + }) +}) + +describe('multi-owner isolation', () => { + it('owner A reading does not grant owner B edit authority', async () => { + const { fs, fileContext } = await setup() + const a = ownerExec({}) + const b = ownerExec({}) + fs.files.set('a.txt', 'hello') + const target = await fs.resolve('a.txt') + + await fileContext.read(target, READ_ALL, a) + await expect(fileContext.edit(target, { oldString: 'hello', newString: 'bye', replaceAll: false }, b)) + .rejects.toMatchObject({ code: 'FS_NOT_OBSERVED' }) + await expect(fileContext.edit(target, { oldString: 'hello', newString: 'bye', replaceAll: false }, a)) + .resolves.toMatchObject({ replacements: 1 }) + }) + + it('each owner records its own observed version independently', async () => { + const { fs, fileContext } = await setup() + const a = ownerExec({}) + const b = ownerExec({}) + fs.files.set('a.txt', 'hello') + const target = await fs.resolve('a.txt') + + await fileContext.read(target, READ_ALL, a) // A sees v0 + await fileContext.write(target, 'mid', b) // B has no read → createIfAbsent + await fileContext.write(target, 'late', a) // A still holds its v0 observation + + expect(fs.writeExpectations).toEqual([ + { kind: 'createIfAbsent' }, + { kind: 'replaceIfVersion', version: 'v0' }, + ]) + }) +}) + +describe('disposal releases recorded state', () => { + it('a fresh service after disposal starts with no inherited state', async () => { + const ctx = new Context() + await ctx.plugin(FakeFs) + const fs = ctx.fs as FakeFs + const fiber = await ctx.plugin(FileContext) + const exec = ownerExec({}) + fs.files.set('a.txt', 'hello') + await ctx.fileContext.read(await fs.resolve('a.txt'), READ_ALL, exec) + await fiber.dispose() + + await ctx.plugin(FileContext) + const target = await fs.resolve('a.txt') + // Same owner object, but state was released on disposal. + await expect(ctx.fileContext.edit(target, { oldString: 'hello', newString: 'bye', replaceAll: false }, exec)) + .rejects.toMatchObject({ code: 'FS_NOT_OBSERVED' }) + }) +}) diff --git a/packages/fs/file-context/tests/window.spec.ts b/packages/fs/file-context/tests/window.spec.ts new file mode 100644 index 0000000000..6b1a8b5b93 --- /dev/null +++ b/packages/fs/file-context/tests/window.spec.ts @@ -0,0 +1,102 @@ +/** + * Cordis-free tests for the line-windowing module: offset/limit windows, byte + * caps, per-line truncation, CRLF stripping, offset-past-EOF rejection, and the + * capped line buffer for newline-free giant lines — all over an async-iterable + * of decoded text chunks (so one code path serves whole-file and streamed reads). + */ + +import { describe, expect, it } from 'vitest' +import { buildWindow, READ_MAX_LINE_LENGTH } from '@deepseek-ai/dsh-file-context' +import type { ReadWindow } from '@deepseek-ai/dsh-file-context' + +const READ_ALL: ReadWindow = { offset: 1, limit: 2000 } + +/** Yield `text` as one chunk (whole-file read shape). */ +async function* whole(text: string): AsyncIterable { + yield text +} + +/** Yield `text` split into fixed-size chunks (streamed read shape). */ +async function* chunked(text: string, size: number): AsyncIterable { + for (let i = 0; i < text.length; i += size) yield text.slice(i, i + size) +} + +describe('buildWindow', () => { + it('numbers lines and reports total for a whole-file read', async () => { + const result = await buildWindow(whole('one\ntwo\nthree'), READ_ALL, 'f') + expect(result.lines).toEqual([ + { number: 1, text: 'one' }, + { number: 2, text: 'two' }, + { number: 3, text: 'three' }, + ]) + expect(result.totalLines).toBe(3) + expect(result.truncatedByBytes).toBe(false) + }) + + it('applies offset/limit', async () => { + const result = await buildWindow(whole('one\ntwo\nthree\nfour'), { offset: 2, limit: 2 }, 'f') + expect(result.lines.map(l => l.number)).toEqual([2, 3]) + expect(result.totalLines).toBe(4) + }) + + it('strips CRLF', async () => { + const result = await buildWindow(whole('one\r\ntwo\r\n'), READ_ALL, 'f') + expect(result.lines.map(l => l.text)).toEqual(['one', 'two']) + }) + + it('truncates an over-long line', async () => { + const result = await buildWindow(whole('x'.repeat(3000)), READ_ALL, 'f') + expect(result.lines[0]?.text).toContain(`... (line truncated to ${READ_MAX_LINE_LENGTH} chars)`) + }) + + it('caps output bytes and reports truncatedByBytes', async () => { + const big = Array.from({ length: 2000 }, () => 'y'.repeat(100)).join('\n') + const result = await buildWindow(whole(big), READ_ALL, 'f') + expect(result.truncatedByBytes).toBe(true) + }) + + it('reads an empty file at offset 1 as zero lines', async () => { + const result = await buildWindow(whole(''), READ_ALL, 'f') + expect(result.lines).toEqual([]) + expect(result.totalLines).toBe(0) + }) + + it('rejects an offset past EOF', async () => { + await expect(buildWindow(whole('one\ntwo'), { offset: 9, limit: 1 }, 'f')).rejects.toMatchObject({ code: 'FS_NOT_FOUND' }) + }) + + it('flushes a final line with no trailing newline', async () => { + const result = await buildWindow(whole('one\ntwo'), READ_ALL, 'f') + expect(result.lines.map(l => l.text)).toEqual(['one', 'two']) + }) + + it('handles a trailing newline (no dangling empty line)', async () => { + const result = await buildWindow(whole('one\ntwo\n'), READ_ALL, 'f') + expect(result.lines.map(l => l.text)).toEqual(['one', 'two']) + expect(result.totalLines).toBe(2) + }) + + describe('chunked input (streamed read shape)', () => { + it('windows identically when text arrives in small chunks', async () => { + const result = await buildWindow(chunked('one\ntwo\nthree', 2), { offset: 2, limit: 1 }, 'f') + expect(result.lines).toEqual([{ number: 2, text: 'two' }]) + expect(result.totalLines).toBe(3) + }) + + it('caps a newline-free giant line split across chunks without unbounded buffering', async () => { + const result = await buildWindow(chunked('z'.repeat(5000), 256), READ_ALL, 'f') + expect(result.lines[0]?.text).toContain(`... (line truncated to ${READ_MAX_LINE_LENGTH} chars)`) + }) + + it('caps output bytes mid-stream', async () => { + const big = Array.from({ length: 2000 }, () => 'y'.repeat(100)).join('\n') + const result = await buildWindow(chunked(big, 512), READ_ALL, 'f') + expect(result.truncatedByBytes).toBe(true) + }) + + it('flushes a final newline-terminated line across a chunk boundary', async () => { + const result = await buildWindow(chunked('one\ntwo\n', 3), READ_ALL, 'f') + expect(result.lines.map(l => l.text)).toEqual(['one', 'two']) + }) + }) +}) diff --git a/packages/fs/file-context/tsconfig.json b/packages/fs/file-context/tsconfig.json new file mode 100644 index 0000000000..dc4518f7f0 --- /dev/null +++ b/packages/fs/file-context/tsconfig.json @@ -0,0 +1,14 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib" + }, + "include": ["src"], + "references": [ + { "path": "../../../vendor/cosmokit" }, + { "path": "../../../vendor/cordis" }, + { "path": "../../llm/llm" }, + { "path": "../fs" } + ] +} diff --git a/packages/fs/fs-local/README.md b/packages/fs/fs-local/README.md index 794239ed2d..a4bb087aea 100644 --- a/packages/fs/fs-local/README.md +++ b/packages/fs/fs-local/README.md @@ -1,20 +1,22 @@ # @deepseek-ai/dsh-fs-local -The **local-filesystem implementation** of the `ctx.fs` seam ([`@deepseek-ai/dsh-fs`](../fs)). Backs the four `FileSystem` primitives with the host filesystem; loading it as a plugin populates `ctx.fs`. +The **local-filesystem implementation** of the `ctx.fs` provider seam ([`@deepseek-ai/dsh-fs`](../fs)). Backs the six `FileSystem` primitives with the host filesystem; loading it as a plugin populates `ctx.fs`. ```ts ignore-check import { LocalFileSystem } from '@deepseek-ai/dsh-fs-local' await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) -// ctx.fs is now the local backend; load @deepseek-ai/dsh-tool-fs to expose read/write/edit to the model. +// ctx.fs is now the local backend; load @deepseek-ai/dsh-file-context for policy +// and @deepseek-ai/dsh-tool-fs to expose read/write/edit to the model. ``` ## Behavior - **`resolve(path)`** — relative paths resolve from `config.cwd` (default `process.cwd()`). The `targetKey` is the file's `realpath`, so two input paths reaching the same file through symlinks share one identity, and writes/edits land on the link target (preserving the link). A not-yet-existing path uses the realpathed parent directory plus basename when the parent exists; only an unresolvable parent falls back to the absolute path. `displayPath` is the absolute (un-resolved) path. -- **`readPage`** — UTF-8 only. A fast path (`readFile`) handles files under `FAST_PATH_MAX_SIZE` (10 MB); larger files stream with a capped line buffer so a newline-free giant file can't exhaust memory. Invalid UTF-8 and NUL-byte samples are rejected (`FS_NOT_TEXT`). Output is bounded to `READ_LIMIT` (2000) lines, `READ_MAX_BYTES` (50 KB), and `READ_MAX_LINE_LENGTH` (2000) chars per line; hitting any bound records a `partial` view. The `version` is `mtimeMs:size`. -- **`createOrReplace`** — atomic: writes to a temp file opened exclusively (`wx`, `0o600`) inside a randomly-named private staging dir (`0o700`) next to the target, fsyncs, then renames over the target. An existing file's mode is preserved, while new files default to `0o600`. Honors the `FsExpectation`: an `observed` write must match the recorded version (else `FS_STALE_VERSION`); a `partial` write onto an existing file is rejected (`FS_PARTIAL_OBSERVATION`); an `unobserved` write onto an existing file is rejected (`FS_NOT_OBSERVED`). -- **`applyEdit`** — atomic literal read-modify-write over the same primitive. Verifies the expected version, LF-normalizes for matching, restores the file's dominant CRLF/LF style, and rejects empty `oldString` / zero matches (`FS_EDIT_NOT_FOUND`) or ambiguous multi-matches without `replace_all` (`FS_AMBIGUOUS_EDIT`). +- **`stat`** — returns `FsInfo` (`version` = `mtimeMs:size`, `type` of `file`/`directory`/`other`, byte `size`) or `undefined` when the target is absent. +- **`readText` / `streamText`** — UTF-8 only. `readText` reads the whole file; `streamText` streams it in chunks (cross-chunk decoding) so a huge file never has to be held whole in memory. Both reject invalid UTF-8 and NUL-byte binary samples (`FS_NOT_TEXT`) and non-regular targets. The policy layer (`ctx.fileContext`) decides which to call by size and owns the line windowing. +- **`writeText`** — atomic: writes to a temp file opened exclusively (`wx`, `0o600`) inside a randomly-named private staging dir (`0o700`) next to the target, fsyncs, then renames over the target. An existing file's mode is preserved, while new files default to `0o600`. Honors the `FsWriteExpectation`: `createIfAbsent` creates a missing target and rejects an existing one (`FS_NOT_OBSERVED`); `replaceIfVersion` replaces only at the observed version (a missing target or mismatch is `FS_STALE_VERSION`). +- **`editText`** — atomic literal read-modify-write over the same primitive, serialized per target by a mutation lock. Verifies the expected version BEFORE literal matching (a stale edit reports `FS_STALE_VERSION`, never `FS_EDIT_NOT_FOUND`/`FS_AMBIGUOUS_EDIT` against newer content), LF-normalizes for matching, restores the file's dominant CRLF/LF style, and rejects empty `oldString` / zero matches (`FS_EDIT_NOT_FOUND`) or ambiguous multi-matches without `replace_all` (`FS_AMBIGUOUS_EDIT`). ## `cwd` is not a sandbox diff --git a/packages/fs/fs-local/src/fsio.ts b/packages/fs/fs-local/src/fsio.ts index a7e124db15..6a2b5a28cc 100644 --- a/packages/fs/fs-local/src/fsio.ts +++ b/packages/fs/fs-local/src/fsio.ts @@ -1,13 +1,13 @@ /** * Cordis-free local-filesystem I/O for `@deepseek-ai/dsh-fs-local`. Kept * separate from the service class (mirroring `dsh-bash-local`'s `run.ts`) so - * the raw read/write/edit mechanics can be unit-tested without a Context. + * the raw stat/read/write/edit mechanics can be unit-tested without a Context. * - * The reader uses two code paths so a single huge line can never balloon - * memory: a **fast path** (`readFile` + in-memory split) for files under - * {@link FAST_PATH_MAX_SIZE}, and a **streaming path** (manual newline scan - * with a capped line buffer) for larger files. Both reject invalid UTF-8 and - * NUL-byte binary samples, and keep only the requested page in memory. + * This is the PROVIDER layer: it hands back decoded whole-file text (validated + * UTF-8, binary rejected) — never line windows or numbered lines, which are + * model-facing read policy owned by `@deepseek-ai/dsh-file-context`. Large files + * stream their text in chunks so a huge file never has to be held whole in + * memory; the binary/NUL sample and cross-chunk UTF-8 decoding stay here. * * Writes are atomic: content goes to a temp file opened exclusively (`wx`, * `0o600`, so a pre-existing path can never be clobbered and write-in-progress @@ -24,56 +24,12 @@ import { chmod, mkdir, open, readFile, realpath, rename, rm, stat } from 'node:f import type { Stats } from 'node:fs' import { basename, dirname, join, resolve } from 'node:path' import { TextDecoder } from 'node:util' -import { FsError } from '@deepseek-ai/dsh-fs' -import type { FsReadRequest, FsTextLine, FsView } from '@deepseek-ai/dsh-fs' +import { FsError, FsTargetKey, FsVersion } from '@deepseek-ai/dsh-fs' -/** Default and maximum number of lines returned by one read. */ -export const READ_LIMIT = 2000 +/** Files at or above this size stream their text; smaller files read whole. */ +export const STREAM_MIN_SIZE = 10 * 1024 * 1024 -/** Maximum characters returned for a single line. */ -export const READ_MAX_LINE_LENGTH = 2000 - -/** Maximum bytes returned for selected file lines. */ -export const READ_MAX_BYTES = 50 * 1024 - -/** Files smaller than this use the in-memory fast path; larger files stream. */ -export const FAST_PATH_MAX_SIZE = 10 * 1024 * 1024 - -const READ_MAX_BYTES_LABEL = `${READ_MAX_BYTES / 1024} KB` -const READ_MAX_LINE_SUFFIX = `... (line truncated to ${READ_MAX_LINE_LENGTH} chars)` const BINARY_SAMPLE_BYTES = 8192 -const LINE_BUFFER_CAP = READ_MAX_LINE_LENGTH + 1 - -/** - * Test seam: lets specs force the streaming path (via a small - * `fastPathMaxSize`) and pin the temp-file name (to prove exclusive-open - * behavior) without a 10 MB fixture or a name race. - */ -export interface FsIoInternals { - /** Override {@link FAST_PATH_MAX_SIZE} for routing. */ - fastPathMaxSize?: number - /** Override the generated private staging-dir name (relative to the target dir). */ - tempDirName?: (writePath: string) => string - /** Override the generated temp-file name (relative to the private staging dir). */ - tempName?: (writePath: string) => string - /** Test hook after the temp file is written/synced but before final chmod+rename. */ - inspectTemp?: (paths: { stagingDir: string; tempPath: string }) => void | Promise -} - -/** A resolved local path: the absolute path shown to callers and its realpath identity. */ -export interface LocalTarget { - /** Absolute path (symlinks not resolved) — used for display. */ - displayPath: string - /** Realpath identity — used as the stable target key and the I/O path. */ - targetKey: string -} - -/** Result of probing a path: null when it does not exist. */ -export interface PathInfo { - version: string - mode: number - isFile: boolean -} function isENOENT(error: unknown): boolean { return error instanceof Error && 'code' in error && error.code === 'ENOENT' @@ -94,8 +50,40 @@ function throwIfAborted(signal: AbortSignal | undefined, verb: string): void { } /** Opaque version token from a stat: mtime (ns precision) + size. */ -function versionOf(info: Stats): string { - return `${info.mtimeMs}:${info.size}` +function versionOf(info: Stats): FsVersion { + return FsVersion(`${info.mtimeMs}:${info.size}`) +} + +/** + * Test seam: lets specs force the streaming read path (via a small + * `streamMinSize`) and pin the temp-file name (to prove exclusive-open + * behavior) without a 10 MB fixture or a name race. + */ +export interface FsIoInternals { + /** Override {@link STREAM_MIN_SIZE} for read routing. */ + streamMinSize?: number + /** Override the generated private staging-dir name (relative to the target dir). */ + tempDirName?: (writePath: string) => string + /** Override the generated temp-file name (relative to the private staging dir). */ + tempName?: (writePath: string) => string + /** Test hook after the temp file is written/synced but before final chmod+rename. */ + inspectTemp?: (paths: { stagingDir: string; tempPath: string }) => void | Promise +} + +/** A resolved local path: the absolute path shown to callers and its realpath identity. */ +export interface LocalTarget { + /** Absolute path (symlinks not resolved) — used for display. */ + displayPath: string + /** Realpath identity — used as the stable target key and the I/O path. */ + targetKey: FsTargetKey +} + +/** Result of probing a path: null when it does not exist. */ +export interface PathInfo { + version: FsVersion + mode: number + type: 'file' | 'directory' | 'other' + size: number } /** @@ -111,26 +99,27 @@ export async function resolveLocalTarget(cwd: string, path: string): Promise { try { const info = await stat(absolutePath) - return { version: versionOf(info), mode: info.mode & 0o777, isFile: info.isFile() } + const type = info.isFile() ? 'file' : info.isDirectory() ? 'directory' : 'other' + return { version: versionOf(info), mode: info.mode & 0o777, type, size: info.size } } catch (error: unknown) { /* v8 ignore next 2 -- a non-ENOENT stat failure needs a permission/IO fault; surface it. */ if (!isENOENT(error)) throw error @@ -140,67 +129,6 @@ export async function probe(absolutePath: string): Promise { // --- Reading --- -interface PageAccumulator { - lines: FsTextLine[] - totalLines: number - outputBytes: number - truncatedByBytes: boolean - truncatedByLine: boolean - done: boolean -} - -function newAccumulator(): PageAccumulator { - return { lines: [], totalLines: 0, outputBytes: 0, truncatedByBytes: false, truncatedByLine: false, done: false } -} - -function truncateReadLine(line: string): { text: string; truncated: boolean } { - return line.length > READ_MAX_LINE_LENGTH - ? { text: `${line.substring(0, READ_MAX_LINE_LENGTH)}${READ_MAX_LINE_SUFFIX}`, truncated: true } - : { text: line, truncated: false } -} - -function lineByteSize(line: string, currentLineCount: number): number { - return Buffer.byteLength(line, 'utf8') + (currentLineCount > 0 ? 1 : 0) -} - -function consumeLine(acc: PageAccumulator, rawLine: string, request: FsReadRequest): void { - acc.totalLines += 1 - if (acc.totalLines < request.offset || acc.lines.length >= request.limit) return - - const { text, truncated } = truncateReadLine(rawLine) - if (truncated) acc.truncatedByLine = true - const bytes = lineByteSize(text, acc.lines.length) - if (acc.outputBytes + bytes > READ_MAX_BYTES) { - acc.truncatedByBytes = true - acc.done = true - return - } - acc.outputBytes += bytes - acc.lines.push({ number: acc.totalLines, text }) -} - -function stripCarriageReturn(line: string): string { - return line.endsWith('\r') ? line.slice(0, -1) : line -} - -/** The outcome shape `readTextPage` returns (minus the offset/limit echo, which the caller adds). */ -export interface ReadPageResult { - lines: FsTextLine[] - totalLines: number - truncatedByBytes: boolean - view: FsView - version: string -} - -function buildResult(acc: PageAccumulator, request: FsReadRequest, version: string, displayPath: string): ReadPageResult { - if (!acc.truncatedByBytes && request.offset > acc.totalLines && !(acc.totalLines === 0 && request.offset === 1)) { - throw new FsError(`offset ${request.offset} is out of range for "${displayPath}" (${acc.totalLines} lines)`, 'FS_NOT_FOUND') - } - const endLine = acc.lines.at(-1)?.number ?? Math.max(0, request.offset - 1) - const view: FsView = request.offset === 1 && !acc.truncatedByBytes && !acc.truncatedByLine && endLine >= acc.totalLines ? 'full' : 'partial' - return { lines: acc.lines, totalLines: acc.totalLines, truncatedByBytes: acc.truncatedByBytes, view, version } -} - function notTextError(verb: 'read' | 'edit', displayPath: string): FsError { return new FsError(`cannot ${verb} "${displayPath}": invalid UTF-8 text`, 'FS_NOT_TEXT') } @@ -209,8 +137,9 @@ function decodeUtf8(buffer: Uint8Array, verb: 'read' | 'edit', displayPath: stri try { return new TextDecoder('utf-8', { fatal: true }).decode(buffer) } catch (error: unknown) { - if (error instanceof TypeError) throw notTextError(verb, displayPath) - throw error + /* v8 ignore next 2 -- TextDecoder({fatal}) only throws TypeError on invalid bytes; any other throw is an unreachable runtime fault. */ + if (!(error instanceof TypeError)) throw error + throw notTextError(verb, displayPath) } } @@ -223,90 +152,50 @@ function decodeUtf8Stream( try { return chunk ? decoder.decode(chunk, { stream: true }) : decoder.decode() } catch (error: unknown) { - if (error instanceof TypeError) throw notTextError(verb, displayPath) - throw error + /* v8 ignore next 2 -- TextDecoder({fatal}) only throws TypeError on invalid bytes; any other throw is an unreachable runtime fault. */ + if (!(error instanceof TypeError)) throw error + throw notTextError(verb, displayPath) } } -/** - * Read a bounded UTF-8 text-file page. Rejects non-regular files, invalid - * UTF-8, and NUL-byte binary samples; dispatches to the fast or streaming path - * by file size. - */ -export async function readTextPage( - target: LocalTarget, - request: FsReadRequest, - signal?: AbortSignal, - internals: FsIoInternals = {}, -): Promise { - throwIfAborted(signal, 'read') - const absolutePath = target.targetKey +async function statRegularFile(target: LocalTarget, verb: 'read', signal?: AbortSignal): Promise { + throwIfAborted(signal, verb) let info: Stats try { - info = await stat(absolutePath) + info = await stat(target.targetKey) } catch (error: unknown) { /* v8 ignore next 2 -- a non-ENOENT stat failure needs a permission/IO fault; only the not-found path is reachable in tests. */ if (!isENOENT(error)) throw error - throw new FsError(`cannot read "${target.displayPath}": not found`, 'FS_NOT_FOUND') + throw new FsError(`cannot ${verb} "${target.displayPath}": not found`, 'FS_NOT_FOUND') } - if (!info.isFile()) throw new FsError(`cannot read "${target.displayPath}": not a regular file`, 'FS_NOT_REGULAR_FILE') - - const version = versionOf(info) - const fastPathMax = internals.fastPathMaxSize ?? FAST_PATH_MAX_SIZE - return info.size < fastPathMax - ? readTextPageFast(target, request, version, signal) - : readTextPageStreaming(target, request, version, signal) + if (!info.isFile()) throw new FsError(`cannot ${verb} "${target.displayPath}": not a regular file`, 'FS_NOT_REGULAR_FILE') + return info } -async function readTextPageFast( - target: LocalTarget, - request: FsReadRequest, - version: string, - signal?: AbortSignal, -): Promise { +/** + * Read a whole regular UTF-8 text file into a single decoded string. Rejects + * non-regular files, invalid UTF-8, and NUL-byte binary samples. + */ +export async function readWholeText(target: LocalTarget, signal?: AbortSignal): Promise { + await statRegularFile(target, 'read', signal) const raw = await readFile(target.targetKey, signal ? { signal } : {}) throwIfAborted(signal, 'read') if (raw.subarray(0, BINARY_SAMPLE_BYTES).includes(0)) { throw new FsError(`cannot read "${target.displayPath}": binary file`, 'FS_NOT_TEXT') } - - const text = decodeUtf8(raw, 'read', target.displayPath) - const acc = newAccumulator() - let startPos = 0 - let newlinePos: number - while ((newlinePos = text.indexOf('\n', startPos)) !== -1) { - consumeLine(acc, stripCarriageReturn(text.slice(startPos, newlinePos)), request) - if (acc.done) break - startPos = newlinePos + 1 - } - if (!acc.done && startPos < text.length) { - consumeLine(acc, stripCarriageReturn(text.slice(startPos)), request) - } - return buildResult(acc, request, version, target.displayPath) + return decodeUtf8(raw, 'read', target.displayPath) } -async function readTextPageStreaming( - target: LocalTarget, - request: FsReadRequest, - version: string, - signal?: AbortSignal, -): Promise { +/** + * Stream a whole regular UTF-8 text file as decoded text chunks. Same text + * semantics as {@link readWholeText} (regular-file check, binary/NUL rejection, + * cross-chunk UTF-8 decoding), but never holds the whole file in memory. + */ +export async function* streamWholeText(target: LocalTarget, signal?: AbortSignal): AsyncIterable { + await statRegularFile(target, 'read', signal) const stream = createReadStream(target.targetKey, signal ? { signal } : {}) - const acc = newAccumulator() - let lineBuffer = '' - let sampledBytes = 0 const decoder = new TextDecoder('utf-8', { fatal: true }) - - function appendToLineBuffer(segment: string): void { - if (lineBuffer.length >= LINE_BUFFER_CAP) return - lineBuffer += segment - if (lineBuffer.length > LINE_BUFFER_CAP) lineBuffer = lineBuffer.slice(0, LINE_BUFFER_CAP) - } - - function flushLine(): void { - consumeLine(acc, stripCarriageReturn(lineBuffer), request) - lineBuffer = '' - } + let sampledBytes = 0 function scanBinarySample(chunk: Buffer): void { if (sampledBytes >= BINARY_SAMPLE_BYTES) return @@ -317,51 +206,17 @@ async function readTextPageStreaming( sampledBytes += sample.length } - function consumeChunk(chunk: string): ReadPageResult | undefined { - let startPos = 0 - let newlinePos: number - while ((newlinePos = chunk.indexOf('\n', startPos)) !== -1) { - appendToLineBuffer(chunk.slice(startPos, newlinePos)) - flushLine() - startPos = newlinePos + 1 - if (acc.done) return buildResult(acc, request, version, target.displayPath) - } - appendToLineBuffer(chunk.slice(startPos)) - return undefined - } - try { for await (const chunk of stream as AsyncIterable) { scanBinarySample(chunk) - const result = consumeChunk(decodeUtf8Stream(decoder, chunk, 'read', target.displayPath)) - if (result) return result + yield decodeUtf8Stream(decoder, chunk, 'read', target.displayPath) } - const finalResult = consumeChunk(decodeUtf8Stream(decoder, undefined, 'read', target.displayPath)) - if (finalResult) return finalResult + yield decodeUtf8Stream(decoder, undefined, 'read', target.displayPath) } catch (error: unknown) { /* v8 ignore next 4 -- mid-stream errors need an abort/IO fault racing the loop; pre-abort is caught by throwIfAborted. */ if (isAbortError(error)) throw new FsError('read aborted', 'FS_ABORTED') throw error } - - if (lineBuffer.length > 0) flushLine() - return buildResult(acc, request, version, target.displayPath) -} - -/** Format the line-numbered body + pagination footer for a read page. */ -export function formatReadBody(result: ReadPageResult, offset: number): string { - const endLine = result.lines.at(-1)?.number ?? Math.max(0, offset - 1) - let footer: string - if (result.truncatedByBytes) { - footer = `(Output capped at ${READ_MAX_BYTES_LABEL}. Showing lines ${offset}-${endLine}. Use offset=${endLine + 1} to continue.)` - } else if (endLine < result.totalLines) { - footer = `(Showing lines ${offset}-${endLine} of ${result.totalLines}. Use offset=${endLine + 1} to continue.)` - } else { - footer = `(End of file - total ${result.totalLines} lines)` - } - return result.lines.length > 0 - ? `${result.lines.map(line => `${line.number}: ${line.text}`).join('\n')}\n\n${footer}` - : footer } // --- Writing --- diff --git a/packages/fs/fs-local/src/index.ts b/packages/fs/fs-local/src/index.ts index 0184a8a323..3c8ba61f78 100644 --- a/packages/fs/fs-local/src/index.ts +++ b/packages/fs/fs-local/src/index.ts @@ -1,10 +1,11 @@ /** - * Local-filesystem implementation of the `ctx.fs` seam. {@link LocalFileSystem} - * subclasses {@link FileSystem} and backs the four primitives with the host - * filesystem via {@link module:@deepseek-ai/dsh-fs-local/fsio}. Path resolution - * uses `realpath`, so the stable `targetKey` is the real file identity (two - * input paths reaching the same file through symlinks share one key, and writes - * land on the link target — preserving the link). + * Local-filesystem implementation of the `ctx.fs` provider seam. + * {@link LocalFileSystem} subclasses {@link FileSystem} and backs the six + * text-storage primitives with the host filesystem via + * {@link module:@deepseek-ai/dsh-fs-local/fsio}. Path resolution uses + * `realpath`, so the stable `targetKey` is the real file identity (two input + * paths reaching the same file through symlinks share one key, and writes land + * on the link target — preserving the link). * * Future sandboxed/remote/virtual backends are sibling packages implementing * the same interface; loading this one populates `ctx.fs`. @@ -14,43 +15,39 @@ import { Context } from 'cordis' import z from 'schemastery' -import { FileSystem, FsError } from '@deepseek-ai/dsh-fs' +import { FileSystem, FsError, FsVersion } from '@deepseek-ai/dsh-fs' import type { FsEditOutcome, FsEditRequest, - FsExpectation, - FsReadOutcome, - FsReadRequest, + FsInfo, FsTarget, - FsVersion, + FsWriteExpectation, FsWriteOutcome, } from '@deepseek-ai/dsh-fs' import { applyLiteralEdit, probe, readForEdit, - readTextPage, + readWholeText, resolveLocalTarget, restoreLineEndings, + streamWholeText, writeFileAtomic, } from './fsio.ts' import type { FsIoInternals } from './fsio.ts' export { - FAST_PATH_MAX_SIZE, - READ_LIMIT, - READ_MAX_BYTES, - READ_MAX_LINE_LENGTH, + STREAM_MIN_SIZE, applyLiteralEdit, - formatReadBody, probe, readForEdit, - readTextPage, + readWholeText, resolveLocalTarget, restoreLineEndings, + streamWholeText, writeFileAtomic, } from './fsio.ts' -export type { FsIoInternals, LineEndings, LocalTarget, PathInfo, ReadPageResult } from './fsio.ts' +export type { FsIoInternals, LineEndings, LocalTarget, PathInfo } from './fsio.ts' /** Configuration for the local filesystem backend. */ export interface Config { @@ -105,47 +102,41 @@ export class LocalFileSystem extends FileSystem { return { inputPath: path, targetKey: local.targetKey, displayPath: local.displayPath } } - override async readPage(target: FsTarget, request: FsReadRequest, signal?: AbortSignal): Promise { - const result = await readTextPage( - { displayPath: target.displayPath, targetKey: target.targetKey }, - request, - signal, - this.internals, - ) - return { - offset: request.offset, - limit: request.limit, - lines: result.lines, - totalLines: result.totalLines, - version: result.version, - view: result.view, - ...result.truncatedByBytes ? { truncatedByBytes: true } : {}, - } + override async stat(target: FsTarget, signal?: AbortSignal): Promise { + if (signal?.aborted) throw new FsError('stat aborted', 'FS_ABORTED') + const info = await probe(target.targetKey) + if (!info) return undefined + return { version: info.version, type: info.type, size: info.size } } - override async createOrReplace( + override async readText(target: FsTarget, signal?: AbortSignal): Promise { + return readWholeText({ displayPath: target.displayPath, targetKey: target.targetKey }, signal) + } + + override streamText(target: FsTarget, signal?: AbortSignal): Promise> { + return Promise.resolve(streamWholeText({ displayPath: target.displayPath, targetKey: target.targetKey }, signal)) + } + + override async writeText( target: FsTarget, content: string, - expected: FsExpectation, + expected: FsWriteExpectation, signal?: AbortSignal, ): Promise { return this.withLock(target.targetKey, async () => { const existing = await probe(target.targetKey) - if (existing && !existing.isFile) { + if (existing && existing.type !== 'file') { throw new FsError(`cannot write "${target.displayPath}": not a regular file`, 'FS_NOT_REGULAR_FILE') } - if (expected.kind === 'observed') { - // Stale guard: the file must still be at the version the owner observed. + if (expected.kind === 'replaceIfVersion') { + // Stale guard: the file must still exist at the version the owner observed. if (!existing) throw new FsError(`cannot write "${target.displayPath}": file no longer exists`, 'FS_STALE_VERSION') if (existing.version !== expected.version) { throw new FsError(`cannot write "${target.displayPath}": file changed since it was read`, 'FS_STALE_VERSION') } - } else if (expected.kind === 'partial') { - if (!existing) throw new FsError(`cannot write "${target.displayPath}": file no longer exists`, 'FS_STALE_VERSION') - throw new FsError(`cannot overwrite existing "${target.displayPath}" after only a partial read`, 'FS_PARTIAL_OBSERVATION') } else if (existing) { - // Unobserved write onto an existing file: a blind overwrite — require a read first. + // createIfAbsent onto an existing file: a blind overwrite — require a read first. throw new FsError(`cannot overwrite existing "${target.displayPath}" without reading it first`, 'FS_NOT_OBSERVED') } @@ -158,7 +149,7 @@ export class LocalFileSystem extends FileSystem { }) } - override async applyEdit( + override async editText( target: FsTarget, edit: FsEditRequest, expected: { version: FsVersion }, @@ -166,8 +157,10 @@ export class LocalFileSystem extends FileSystem { ): Promise { return this.withLock(target.targetKey, async () => { const existing = await probe(target.targetKey) - if (!existing) throw new FsError(`cannot edit "${target.displayPath}": not found`, 'FS_NOT_FOUND') - if (!existing.isFile) throw new FsError(`cannot edit "${target.displayPath}": not a regular file`, 'FS_NOT_REGULAR_FILE') + // Stale guard BEFORE literal matching: an edit based on an old read reports + // FS_STALE_VERSION, not FS_EDIT_NOT_FOUND/FS_AMBIGUOUS_EDIT against newer content. + if (!existing) throw new FsError(`cannot edit "${target.displayPath}": file changed since it was read`, 'FS_STALE_VERSION') + if (existing.type !== 'file') throw new FsError(`cannot edit "${target.displayPath}": not a regular file`, 'FS_NOT_REGULAR_FILE') if (existing.version !== expected.version) { throw new FsError(`cannot edit "${target.displayPath}": file changed since it was read`, 'FS_STALE_VERSION') } @@ -188,9 +181,9 @@ export class LocalFileSystem extends FileSystem { /* v8 ignore next 5 -- the post-write probe finding the file absent requires a * concurrent unlink between rename and stat; fall back to a sentinel version. */ - private versionAfterWrite(after: { version: string } | null, target: FsTarget): string { + private versionAfterWrite(after: { version: FsVersion } | null, target: FsTarget): FsVersion { if (after) return after.version - return `missing:${target.targetKey}` + return FsVersion(`missing:${target.targetKey}`) } } diff --git a/packages/fs/fs-local/tests/filesystem.spec.ts b/packages/fs/fs-local/tests/filesystem.spec.ts index eb675472af..20c5cfa21f 100644 --- a/packages/fs/fs-local/tests/filesystem.spec.ts +++ b/packages/fs/fs-local/tests/filesystem.spec.ts @@ -1,7 +1,9 @@ /** - * Tests for the local backend through the `ctx.fs` service: the full - * read→write→edit lifecycle with the read-before-write policy, stale-version - * guards, concurrency races, symlink identity, and HMR/disposal. + * Tests for the local backend through the `ctx.fs` provider seam: stat, whole- + * file/streamed text reads, atomic guarded writes (createIfAbsent / + * replaceIfVersion), version-guarded literal edits, concurrency races, symlink + * identity, and HMR/disposal. Read WINDOWING is policy and lives in + * `dsh-file-context`, so it is not exercised here. */ import { afterEach, beforeEach, describe, expect, it } from 'vitest' @@ -9,8 +11,9 @@ import { mkdtemp, readFile, rm, stat, symlink, writeFile, unlink } from 'node:fs import { tmpdir } from 'node:os' import { join } from 'node:path' import { Context } from 'cordis' -import { LocalFileSystem, probe } from '@deepseek-ai/dsh-fs-local' -import type { FsExecContext } from '@deepseek-ai/dsh-fs' +import { LocalFileSystem } from '@deepseek-ai/dsh-fs-local' +import { FsVersion } from '@deepseek-ai/dsh-fs' +import type { FsTarget } from '@deepseek-ai/dsh-fs' let dir: string let ctx: Context @@ -28,12 +31,17 @@ afterEach(async () => { await rm(dir, { recursive: true, force: true }) }) -const READ_ALL = { offset: 1, limit: 2000 } -const exec = (): FsExecContext => ({ agent: { session: {} } }) function lockCount(localFs: LocalFileSystem): number { return (localFs as unknown as { locks: Map> }).locks.size } +/** The version the backend currently reports for a resolved target. */ +async function versionOf(target: FsTarget): Promise { + const info = await fs.stat(target) + if (!info) throw new Error('expected target to exist') + return info.version +} + describe('registration', () => { it('registers LocalFileSystem as ctx.fs with a default cwd', async () => { const bare = new Context() @@ -43,255 +51,219 @@ describe('registration', () => { }) }) -describe('read → write → edit lifecycle', () => { - it('creates a new file without a prior read', async () => { +describe('stat', () => { + it('returns file metadata, directory type, and undefined for absent', async () => { + await writeFile(join(dir, 'a.txt'), 'hello') + const fileInfo = await fs.stat(await fs.resolve('a.txt')) + expect(fileInfo?.type).toBe('file') + expect(fileInfo?.size).toBe(5) + expect(typeof fileInfo?.version).toBe('string') + + expect((await fs.stat(await fs.resolve('.')))?.type).toBe('directory') + expect(await fs.stat(await fs.resolve('missing.txt'))).toBeUndefined() + }) + + it('honors a pre-aborted signal', async () => { + await expect(fs.stat(await fs.resolve('a.txt'), AbortSignal.abort())).rejects.toMatchObject({ code: 'FS_ABORTED' }) + }) +}) + +describe('readText / streamText', () => { + it('reads whole-file text', async () => { + await writeFile(join(dir, 'a.txt'), 'one\ntwo\nthree') + expect(await fs.readText(await fs.resolve('a.txt'))).toBe('one\ntwo\nthree') + }) + + it('streams the same text', async () => { + await writeFile(join(dir, 'a.txt'), 'one\ntwo\nthree') + const target = await fs.resolve('a.txt') + let streamed = '' + for await (const chunk of await fs.streamText(target)) streamed += chunk + expect(streamed).toBe('one\ntwo\nthree') + }) + + it('rejects a missing file, a directory, binary, and invalid UTF-8', async () => { + await expect(fs.readText(await fs.resolve('nope'))).rejects.toMatchObject({ code: 'FS_NOT_FOUND' }) + await expect(fs.readText(await fs.resolve('.'))).rejects.toMatchObject({ code: 'FS_NOT_REGULAR_FILE' }) + + await writeFile(join(dir, 'bin'), Buffer.from([0x68, 0x00, 0x69])) + await expect(fs.readText(await fs.resolve('bin'))).rejects.toMatchObject({ code: 'FS_NOT_TEXT' }) + + await writeFile(join(dir, 'bad'), Buffer.from([0x68, 0xff, 0x69])) + await expect(fs.readText(await fs.resolve('bad'))).rejects.toMatchObject({ code: 'FS_NOT_TEXT' }) + }) +}) + +describe('writeText', () => { + it('createIfAbsent creates a new file', async () => { const target = await fs.resolve('new.txt') - const outcome = await fs.write(target, 'fresh', exec()) + const outcome = await fs.writeText(target, 'fresh', { kind: 'createIfAbsent' }) expect(outcome.operation).toBe('create') expect(await readFile(join(dir, 'new.txt'), 'utf8')).toBe('fresh') }) - it('updates an existing file after reading it', async () => { + it('createIfAbsent rejects an existing file as FS_NOT_OBSERVED', async () => { await writeFile(join(dir, 'a.txt'), 'old') - const owner = exec() const target = await fs.resolve('a.txt') - await fs.read(target, READ_ALL, owner) - const outcome = await fs.write(target, 'new', owner) + await expect(fs.writeText(target, 'new', { kind: 'createIfAbsent' })) + .rejects.toMatchObject({ code: 'FS_NOT_OBSERVED' }) + expect(await readFile(join(dir, 'a.txt'), 'utf8')).toBe('old') + }) + + it('replaceIfVersion replaces when the version matches', async () => { + await writeFile(join(dir, 'a.txt'), 'old') + const target = await fs.resolve('a.txt') + const outcome = await fs.writeText(target, 'new', { kind: 'replaceIfVersion', version: await versionOf(target) }) expect(outcome.operation).toBe('update') expect(await readFile(join(dir, 'a.txt'), 'utf8')).toBe('new') }) - it('edits an existing file after reading it', async () => { - await writeFile(join(dir, 'a.txt'), 'hello world') - const owner = exec() + it('replaceIfVersion rejects a stale version', async () => { + await writeFile(join(dir, 'a.txt'), 'v1') const target = await fs.resolve('a.txt') - await fs.read(target, READ_ALL, owner) - const outcome = await fs.edit(target, { oldString: 'world', newString: 'there', replaceAll: false }, owner) - expect(outcome.replacements).toBe(1) - expect(await readFile(join(dir, 'a.txt'), 'utf8')).toBe('hello there') + const stale = await versionOf(target) + await writeFile(join(dir, 'a.txt'), 'changed-externally') + await expect(fs.writeText(target, 'v2', { kind: 'replaceIfVersion', version: stale })) + .rejects.toMatchObject({ code: 'FS_STALE_VERSION' }) }) - it('rejects an empty edit oldString through ctx.fs without hanging or changing the file', async () => { - await writeFile(join(dir, 'a.txt'), 'hello world') - const owner = exec() + it('replaceIfVersion rejects a deleted target as stale, without recreating it', async () => { + const path = join(dir, 'a.txt') + await writeFile(path, 'v1') const target = await fs.resolve('a.txt') - await fs.read(target, READ_ALL, owner) - - await expect(fs.edit(target, { oldString: '', newString: 'boom', replaceAll: false }, owner)) - .rejects.toMatchObject({ code: 'FS_EDIT_NOT_FOUND' }) - expect(await readFile(join(dir, 'a.txt'), 'utf8')).toBe('hello world') + const version = await versionOf(target) + await unlink(path) + await expect(fs.writeText(target, 'v2', { kind: 'replaceIfVersion', version })) + .rejects.toMatchObject({ code: 'FS_STALE_VERSION' }) + await expect(stat(path)).rejects.toMatchObject({ code: 'ENOENT' }) }) - it('propagates truncatedByBytes from a byte-capped read', async () => { - await writeFile(join(dir, 'big.txt'), Array.from({ length: 2000 }, () => 'y'.repeat(100)).join('\n')) - const outcome = await fs.read(await fs.resolve('big.txt'), READ_ALL, exec()) - expect(outcome.truncatedByBytes).toBe(true) - expect(outcome.view).toBe('partial') - }) - - it('records an over-long-line read as partial, so write/edit stay blocked', async () => { - await writeFile(join(dir, 'long.txt'), 'x'.repeat(3000)) - const owner = exec() - const target = await fs.resolve('long.txt') - const outcome = await fs.read(target, READ_ALL, owner) - - expect(outcome.view).toBe('partial') - await expect(fs.write(target, 'new', owner)).rejects.toMatchObject({ code: 'FS_PARTIAL_OBSERVATION' }) - await expect( - fs.edit(target, { oldString: 'x', newString: 'y', replaceAll: false }, owner), - ).rejects.toMatchObject({ code: 'FS_PARTIAL_OBSERVATION' }) - }) - - it('allows a follow-up edit without re-reading (write/edit refresh state)', async () => { - await writeFile(join(dir, 'a.txt'), 'a b') - const owner = exec() - const target = await fs.resolve('a.txt') - await fs.read(target, READ_ALL, owner) - await fs.edit(target, { oldString: 'a', newString: 'X', replaceAll: false }, owner) - await fs.edit(target, { oldString: 'b', newString: 'Y', replaceAll: false }, owner) - expect(await readFile(join(dir, 'a.txt'), 'utf8')).toBe('X Y') + it('rejects writing onto a directory', async () => { + const target = await fs.resolve('.') + await expect(fs.writeText(target, 'x', { kind: 'createIfAbsent' })) + .rejects.toMatchObject({ code: 'FS_NOT_REGULAR_FILE' }) }) it('releases per-target mutation locks after success and failure', async () => { const target = await fs.resolve('a.txt') - await fs.write(target, 'created', exec()) + await fs.writeText(target, 'created', { kind: 'createIfAbsent' }) expect(lockCount(fs)).toBe(0) - - await expect(fs.write(target, 'blind overwrite', exec())).rejects.toMatchObject({ code: 'FS_NOT_OBSERVED' }) + await expect(fs.writeText(target, 'again', { kind: 'createIfAbsent' })) + .rejects.toMatchObject({ code: 'FS_NOT_OBSERVED' }) expect(lockCount(fs)).toBe(0) }) }) -describe('read-before-write policy', () => { - it('rejects a blind overwrite of an existing file (no prior read)', async () => { - await writeFile(join(dir, 'a.txt'), 'old') +describe('editText', () => { + it('applies a literal edit at the matching version', async () => { + await writeFile(join(dir, 'a.txt'), 'hello world') const target = await fs.resolve('a.txt') - await expect(fs.write(target, 'new', exec())).rejects.toMatchObject({ code: 'FS_NOT_OBSERVED' }) + const outcome = await fs.editText(target, { oldString: 'world', newString: 'there', replaceAll: false }, { version: await versionOf(target) }) + expect(outcome.replacements).toBe(1) + expect(await readFile(join(dir, 'a.txt'), 'utf8')).toBe('hello there') }) - it('rejects a write after only a partial read', async () => { - await writeFile(join(dir, 'a.txt'), 'one\ntwo') - const owner = exec() + it('checks the stale version BEFORE literal matching', async () => { + await writeFile(join(dir, 'a.txt'), 'hello world') const target = await fs.resolve('a.txt') - await fs.read(target, { offset: 1, limit: 1 }, owner) - await expect(fs.write(target, 'new', owner)).rejects.toMatchObject({ code: 'FS_PARTIAL_OBSERVATION' }) + const stale = await versionOf(target) + // Change the file so 'world' is gone — a stale edit must report STALE, not NOT_FOUND. + await writeFile(join(dir, 'a.txt'), 'goodbye') + await expect(fs.editText(target, { oldString: 'world', newString: 'there', replaceAll: false }, { version: stale })) + .rejects.toMatchObject({ code: 'FS_STALE_VERSION' }) }) - it('rejects a write after a partial read when the file was deleted, without recreating it', async () => { - const path = join(dir, 'a.txt') - await writeFile(path, 'one\ntwo') - const owner = exec() + it('rejects a deleted target as stale (before matching)', async () => { + await writeFile(join(dir, 'a.txt'), 'hello') const target = await fs.resolve('a.txt') - await fs.read(target, { offset: 1, limit: 1 }, owner) - await unlink(path) - - await expect(fs.write(target, 'new', owner)).rejects.toMatchObject({ code: 'FS_STALE_VERSION' }) - await expect(stat(path)).rejects.toMatchObject({ code: 'ENOENT' }) + const version = await versionOf(target) + await unlink(join(dir, 'a.txt')) + await expect(fs.editText(target, { oldString: 'hello', newString: 'bye', replaceAll: false }, { version })) + .rejects.toMatchObject({ code: 'FS_STALE_VERSION' }) }) - it('rejects an edit with no prior read (FS_NOT_OBSERVED)', async () => { - await writeFile(join(dir, 'a.txt'), 'old') - const target = await fs.resolve('a.txt') - await expect(fs.edit(target, { oldString: 'old', newString: 'new', replaceAll: false }, exec())) - .rejects.toMatchObject({ code: 'FS_NOT_OBSERVED' }) + it('rejects a non-regular target', async () => { + const target = await fs.resolve('.') + await expect(fs.editText(target, { oldString: 'a', newString: 'b', replaceAll: false }, { version: FsVersion('v') })) + .rejects.toMatchObject({ code: 'FS_NOT_REGULAR_FILE' }) }) - it('rejects invalid UTF-8 reads and edits without rewriting the file', async () => { - const path = join(dir, 'invalid-utf8.txt') + it('rejects zero matches and ambiguous matches at the right version', async () => { + await writeFile(join(dir, 'a.txt'), 'a a a') + const target = await fs.resolve('a.txt') + const version = await versionOf(target) + await expect(fs.editText(target, { oldString: 'z', newString: 'X', replaceAll: false }, { version })) + .rejects.toMatchObject({ code: 'FS_EDIT_NOT_FOUND' }) + await expect(fs.editText(target, { oldString: 'a', newString: 'X', replaceAll: false }, { version })) + .rejects.toMatchObject({ code: 'FS_AMBIGUOUS_EDIT' }) + }) + + it('replaces all matches with replaceAll', async () => { + await writeFile(join(dir, 'a.txt'), 'a a a') + const target = await fs.resolve('a.txt') + const outcome = await fs.editText(target, { oldString: 'a', newString: 'b', replaceAll: true }, { version: await versionOf(target) }) + expect(outcome.replacements).toBe(3) + expect(await readFile(join(dir, 'a.txt'), 'utf8')).toBe('b b b') + }) + + it('rejects invalid UTF-8 without rewriting the file', async () => { + const path = join(dir, 'bad.txt') const bytes = Buffer.from([0x68, 0xff, 0x69]) await writeFile(path, bytes) - const owner = exec() - const target = await fs.resolve('invalid-utf8.txt') - - await expect(fs.read(target, READ_ALL, owner)).rejects.toMatchObject({ code: 'FS_NOT_TEXT' }) - const existing = await probe(target.targetKey) - if (!existing) throw new Error('expected invalid UTF-8 fixture to exist') - await expect( - fs.applyEdit(target, { oldString: 'h', newString: 'H', replaceAll: false }, { version: existing.version }), - ).rejects.toMatchObject({ code: 'FS_NOT_TEXT' }) + const target = await fs.resolve('bad.txt') + const version = await versionOf(target) + await expect(fs.editText(target, { oldString: 'h', newString: 'H', replaceAll: false }, { version })) + .rejects.toMatchObject({ code: 'FS_NOT_TEXT' }) expect(await readFile(path)).toEqual(bytes) }) -}) - -describe('stale-version guard + concurrency (defensive class B)', () => { - it('rejects a write when the file changed since it was read', async () => { - await writeFile(join(dir, 'a.txt'), 'v1') - const owner = exec() - const target = await fs.resolve('a.txt') - await fs.read(target, READ_ALL, owner) - // An out-of-band change after the read. - await writeFile(join(dir, 'a.txt'), 'changed-externally') - await expect(fs.write(target, 'v2', owner)).rejects.toMatchObject({ code: 'FS_STALE_VERSION' }) - }) - - it('rejects an observed write when the file was deleted after the read', async () => { - await writeFile(join(dir, 'a.txt'), 'v1') - const owner = exec() - const target = await fs.resolve('a.txt') - await fs.read(target, READ_ALL, owner) - await unlink(join(dir, 'a.txt')) // file vanishes; observed write must fail (not silently create) - await expect(fs.write(target, 'v2', owner)).rejects.toMatchObject({ code: 'FS_STALE_VERSION' }) - }) it('two concurrent edits: one wins, the other is rejected as stale', async () => { await writeFile(join(dir, 'a.txt'), 'base') - const owner = exec() const target = await fs.resolve('a.txt') - await fs.read(target, READ_ALL, owner) - // Both edits captured the same recorded version; only one rename can match it. + const version = await versionOf(target) const results = await Promise.allSettled([ - fs.edit(target, { oldString: 'base', newString: 'one', replaceAll: false }, owner), - fs.edit(target, { oldString: 'base', newString: 'two', replaceAll: false }, owner), + fs.editText(target, { oldString: 'base', newString: 'one', replaceAll: false }, { version }), + fs.editText(target, { oldString: 'base', newString: 'two', replaceAll: false }, { version }), ]) - const fulfilled = results.filter(r => r.status === 'fulfilled') + expect(results.filter(r => r.status === 'fulfilled')).toHaveLength(1) const rejected = results.filter(r => r.status === 'rejected') - expect(fulfilled).toHaveLength(1) expect(rejected).toHaveLength(1) expect((rejected[0] as PromiseRejectedResult).reason).toMatchObject({ code: 'FS_STALE_VERSION' }) expect(lockCount(fs)).toBe(0) }) }) -describe('symlink targetKey identity (defensive class F)', () => { - it('a read via the real path authorizes an edit via the symlink path', async () => { +describe('symlink targetKey identity', () => { + it('two paths to the same file via a symlink share one version and write the real target', async () => { await writeFile(join(dir, 'real.txt'), 'hello') await symlink(join(dir, 'real.txt'), join(dir, 'link.txt')) - const owner = exec() - await fs.read(await fs.resolve('real.txt'), READ_ALL, owner) - // Edit through the link: same realpath → same targetKey → prior read counts. - const linkTarget = await fs.resolve('link.txt') - const outcome = await fs.edit(linkTarget, { oldString: 'hello', newString: 'bye', replaceAll: false }, owner) - expect(outcome.replacements).toBe(1) - expect(await readFile(join(dir, 'real.txt'), 'utf8')).toBe('bye') // link preserved, target written - }) + const viaReal = await fs.resolve('real.txt') + const viaLink = await fs.resolve('link.txt') + expect(viaLink.targetKey).toBe(viaReal.targetKey) - it('write through a symlink preserves the link and writes the real target', async () => { - await writeFile(join(dir, 'real.txt'), 'hello') - await symlink(join(dir, 'real.txt'), join(dir, 'link.txt')) - const owner = exec() - const linkTarget = await fs.resolve('link.txt') - await fs.read(linkTarget, READ_ALL, owner) - await fs.write(linkTarget, 'replaced', owner) - expect(await readFile(join(dir, 'real.txt'), 'utf8')).toBe('replaced') + const version = await versionOf(viaReal) + await fs.editText(viaLink, { oldString: 'hello', newString: 'bye', replaceAll: false }, { version }) + expect(await readFile(join(dir, 'real.txt'), 'utf8')).toBe('bye') // link preserved }) it('a stale change is detected across both paths', async () => { await writeFile(join(dir, 'real.txt'), 'hello') await symlink(join(dir, 'real.txt'), join(dir, 'link.txt')) - const owner = exec() - await fs.read(await fs.resolve('real.txt'), READ_ALL, owner) - await writeFile(join(dir, 'real.txt'), 'changed') // out-of-band via real path - const linkTarget = await fs.resolve('link.txt') - await expect(fs.edit(linkTarget, { oldString: 'hello', newString: 'bye', replaceAll: false }, owner)) + const viaReal = await fs.resolve('real.txt') + const stale = await versionOf(viaReal) + await writeFile(join(dir, 'real.txt'), 'changed') + const viaLink = await fs.resolve('link.txt') + await expect(fs.editText(viaLink, { oldString: 'hello', newString: 'bye', replaceAll: false }, { version: stale })) .rejects.toMatchObject({ code: 'FS_STALE_VERSION' }) }) }) -describe('non-regular targets', () => { - it('rejects writing onto a directory', async () => { - const target = await fs.resolve('.') // the cwd dir - await expect(fs.write(target, 'x', exec())).rejects.toMatchObject({ code: 'FS_NOT_REGULAR_FILE' }) - }) - - it('applyEdit rejects a target that vanished after the read', async () => { - await writeFile(join(dir, 'a.txt'), 'hello') - const owner = exec() - const target = await fs.resolve('a.txt') - const version = (await fs.read(target, READ_ALL, owner)).version - await unlink(join(dir, 'a.txt')) - await expect(fs.applyEdit(target, { oldString: 'hello', newString: 'bye', replaceAll: false }, { version })) - .rejects.toMatchObject({ code: 'FS_NOT_FOUND' }) - }) - - it('applyEdit rejects a non-regular target', async () => { - const target = await fs.resolve('.') - await expect(fs.applyEdit(target, { oldString: 'a', newString: 'b', replaceAll: false }, { version: 'v' })) - .rejects.toMatchObject({ code: 'FS_NOT_REGULAR_FILE' }) - }) -}) - -describe('HMR / disposal (defensive class D)', () => { +describe('HMR / disposal', () => { it('disposing the fiber withdraws ctx.fs', async () => { const local = new Context() - const fiber = await local.plugin(LocalFileSystem, { cwd: dir }) + const localFiber = await local.plugin(LocalFileSystem, { cwd: dir }) expect(local.fs).toBeDefined() - await fiber.dispose() + await localFiber.dispose() expect(local.fs).toBeUndefined() }) - - it('a fresh provider does not inherit recorded file state', async () => { - await writeFile(join(dir, 'a.txt'), 'hello') - const local = new Context() - const owner = exec() - const fiber = await local.plugin(LocalFileSystem, { cwd: dir }) - await (local.fs as LocalFileSystem).read(await local.fs.resolve('a.txt'), READ_ALL, owner) - await fiber.dispose() - - await local.plugin(LocalFileSystem, { cwd: dir }) - const fs2 = local.fs as LocalFileSystem - const target = await fs2.resolve('a.txt') - // Same owner object, but state was released on disposal. - await expect(fs2.edit(target, { oldString: 'hello', newString: 'bye', replaceAll: false }, owner)) - .rejects.toMatchObject({ code: 'FS_NOT_OBSERVED' }) - }) }) diff --git a/packages/fs/fs-local/tests/fsio.spec.ts b/packages/fs/fs-local/tests/fsio.spec.ts index f25f9e9a0b..13ab9ed860 100644 --- a/packages/fs/fs-local/tests/fsio.spec.ts +++ b/packages/fs/fs-local/tests/fsio.spec.ts @@ -1,24 +1,27 @@ /** - * Cordis-free tests for the raw local-filesystem I/O: path resolution, - * fast/streaming reads, pagination/caps, binary rejection, atomic-write temp - * safety, literal edit matching, and line-ending handling. + * Cordis-free tests for the raw local-filesystem I/O: path resolution, probe, + * whole-file/streamed text reads, binary/UTF-8 rejection, atomic-write temp + * safety, literal edit matching, and line-ending handling. Line WINDOWING is + * policy and lives in `dsh-file-context`, so it is not tested here. */ import { afterEach, beforeEach, describe, expect, it } from 'vitest' -import { mkdtemp, readFile, rm, stat, symlink, writeFile, mkdir, readdir } from 'node:fs/promises' +import { mkdtemp, readFile, rm, stat, symlink, writeFile, mkdir, readdir, realpath } from 'node:fs/promises' import { tmpdir } from 'node:os' import { join } from 'node:path' +import { createServer } from 'node:net' import { applyLiteralEdit, - formatReadBody, probe, readForEdit, - readTextPage, + readWholeText, resolveLocalTarget, restoreLineEndings, + streamWholeText, writeFileAtomic, } from '@deepseek-ai/dsh-fs-local' import type { LocalTarget } from '@deepseek-ai/dsh-fs-local' +import { FsTargetKey } from '@deepseek-ai/dsh-fs' let dir: string beforeEach(async () => { @@ -28,8 +31,13 @@ afterEach(async () => { await rm(dir, { recursive: true, force: true }) }) -const READ_ALL = { offset: 1, limit: 2000 } -const localTarget = (path: string): LocalTarget => ({ displayPath: path, targetKey: path }) +const localTarget = (path: string): LocalTarget => ({ displayPath: path, targetKey: FsTargetKey(path) }) + +async function collect(chunks: AsyncIterable): Promise { + let out = '' + for await (const chunk of chunks) out += chunk + return out +} describe('resolveLocalTarget', () => { it('resolves a relative path from cwd and realpaths it', async () => { @@ -37,11 +45,10 @@ describe('resolveLocalTarget', () => { await writeFile(file, 'hi') const target = await resolveLocalTarget(dir, 'a.txt') expect(target.displayPath).toBe(file) - expect(target.targetKey).toBe(await (await import('node:fs/promises')).realpath(file)) + expect(target.targetKey).toBe(await realpath(file)) }) it('uses the realpathed parent + basename when the file does not exist (stable across create)', async () => { - const { realpath } = await import('node:fs/promises') const target = await resolveLocalTarget(dir, 'missing.txt') expect(target.targetKey).toBe(join(await realpath(dir), 'missing.txt')) }) @@ -67,186 +74,104 @@ describe('resolveLocalTarget', () => { }) }) -describe('readTextPage', () => { - it('reads a small file with line numbers and full view', async () => { +describe('probe', () => { + it('returns null for a missing path and metadata for a file', async () => { + expect(await probe(join(dir, 'nope'))).toBeNull() + const file = join(dir, 'a.txt') + await writeFile(file, 'hi') + const info = await probe(file) + expect(info?.type).toBe('file') + expect(info?.size).toBe(2) + expect(typeof info?.version).toBe('string') + }) + + it('reports a directory and a non-regular type', async () => { + const sub = join(dir, 'sub') + await mkdir(sub) + expect((await probe(sub))?.type).toBe('directory') + }) + + it('reports a socket/special file as type "other"', async () => { + const sockPath = join(dir, 'sock') + const server = createServer() + await new Promise((resolve) => { server.listen(sockPath, () => { resolve() }) }) + try { + expect((await probe(sockPath))?.type).toBe('other') + } finally { + await new Promise((resolve) => { server.close(() => { resolve() }) }) + } + }) +}) + +describe('readWholeText', () => { + it('reads a small file', async () => { const file = join(dir, 'a.txt') await writeFile(file, 'one\ntwo\nthree') - const result = await readTextPage(localTarget(file), READ_ALL) - expect(result.lines).toEqual([ - { number: 1, text: 'one' }, - { number: 2, text: 'two' }, - { number: 3, text: 'three' }, - ]) - expect(result.totalLines).toBe(3) - expect(result.view).toBe('full') - }) - - it('paginates with offset/limit and reports a partial view', async () => { - const file = join(dir, 'a.txt') - await writeFile(file, 'one\ntwo\nthree\nfour') - const result = await readTextPage(localTarget(file), { offset: 2, limit: 2 }) - expect(result.lines.map(l => l.number)).toEqual([2, 3]) - expect(result.view).toBe('partial') - expect(formatReadBody(result, 2)).toContain('(Showing lines 2-3 of 4. Use offset=4 to continue.)') - }) - - it('a whole-file read from offset 1 is a full view; offset>1 is partial', async () => { - const file = join(dir, 'a.txt') - await writeFile(file, 'one\ntwo') - expect((await readTextPage(localTarget(file), { offset: 1, limit: 10 })).view).toBe('full') - expect((await readTextPage(localTarget(file), { offset: 2, limit: 10 })).view).toBe('partial') - }) - - it('truncates an over-long line', async () => { - const file = join(dir, 'long.txt') - await writeFile(file, 'x'.repeat(3000)) - const result = await readTextPage(localTarget(file), READ_ALL) - expect(result.lines[0]?.text).toContain('... (line truncated to 2000 chars)') - expect(result.view).toBe('partial') - }) - - it('caps output bytes and reports truncatedByBytes', async () => { - const file = join(dir, 'big.txt') - const lines = Array.from({ length: 2000 }, () => 'y'.repeat(100)) - await writeFile(file, lines.join('\n')) - const result = await readTextPage(localTarget(file), READ_ALL) - expect(result.truncatedByBytes).toBe(true) - expect(formatReadBody(result, 1)).toContain('Output capped at 50 KB') - }) - - it('strips CRLF so a Windows file reads like LF', async () => { - const file = join(dir, 'crlf.txt') - await writeFile(file, 'one\r\ntwo\r\n') - const result = await readTextPage(localTarget(file), READ_ALL) - expect(result.lines.map(l => l.text)).toEqual(['one', 'two']) - }) - - it('reads an empty file at offset 1', async () => { - const file = join(dir, 'empty.txt') - await writeFile(file, '') - const result = await readTextPage(localTarget(file), READ_ALL) - expect(result.lines).toEqual([]) - expect(result.totalLines).toBe(0) - expect(formatReadBody(result, 1)).toBe('(End of file - total 0 lines)') - }) - - it('rejects an offset past EOF', async () => { - const file = join(dir, 'a.txt') - await writeFile(file, 'one\ntwo') - await expect(readTextPage(localTarget(file), { offset: 9, limit: 1 })).rejects.toMatchObject({ code: 'FS_NOT_FOUND' }) - }) - - it('rejects a binary file (fast path)', async () => { - const file = join(dir, 'bin') - await writeFile(file, Buffer.from([0x68, 0x00, 0x69])) - await expect(readTextPage(localTarget(file), READ_ALL)).rejects.toMatchObject({ code: 'FS_NOT_TEXT' }) - }) - - it('rejects invalid UTF-8 bytes (fast path)', async () => { - const file = join(dir, 'invalid-utf8.txt') - await writeFile(file, Buffer.from([0x68, 0xff, 0x69])) - await expect(readTextPage(localTarget(file), READ_ALL)).rejects.toMatchObject({ code: 'FS_NOT_TEXT' }) + expect(await readWholeText(localTarget(file))).toBe('one\ntwo\nthree') }) it('rejects a missing file and a directory', async () => { - await expect(readTextPage(localTarget(join(dir, 'nope')), READ_ALL)).rejects.toMatchObject({ code: 'FS_NOT_FOUND' }) - await expect(readTextPage(localTarget(dir), READ_ALL)).rejects.toMatchObject({ code: 'FS_NOT_REGULAR_FILE' }) + await expect(readWholeText(localTarget(join(dir, 'nope')))).rejects.toMatchObject({ code: 'FS_NOT_FOUND' }) + await expect(readWholeText(localTarget(dir))).rejects.toMatchObject({ code: 'FS_NOT_REGULAR_FILE' }) + }) + + it('rejects binary and invalid UTF-8', async () => { + await writeFile(join(dir, 'bin'), Buffer.from([0x68, 0x00, 0x69])) + await expect(readWholeText(localTarget(join(dir, 'bin')))).rejects.toMatchObject({ code: 'FS_NOT_TEXT' }) + await writeFile(join(dir, 'bad'), Buffer.from([0x68, 0xff, 0x69])) + await expect(readWholeText(localTarget(join(dir, 'bad')))).rejects.toMatchObject({ code: 'FS_NOT_TEXT' }) }) it('honors a pre-aborted signal', async () => { const file = join(dir, 'a.txt') await writeFile(file, 'one') - await expect(readTextPage(localTarget(file), READ_ALL, AbortSignal.abort())).rejects.toMatchObject({ code: 'FS_ABORTED' }) + await expect(readWholeText(localTarget(file), AbortSignal.abort())).rejects.toMatchObject({ code: 'FS_ABORTED' }) }) - it('passes a live (non-aborted) signal through the fast path', async () => { + it('passes a live (non-aborted) signal through', async () => { const file = join(dir, 'a.txt') await writeFile(file, 'one\ntwo') - const result = await readTextPage(localTarget(file), READ_ALL, new AbortController().signal) - expect(result.totalLines).toBe(2) - }) - - describe('streaming path (forced via a tiny fastPathMaxSize)', () => { - const stream = { fastPathMaxSize: 1 } - - it('reads and paginates large files the same way', async () => { - const file = join(dir, 'a.txt') - await writeFile(file, 'one\ntwo\nthree') - const result = await readTextPage(localTarget(file), { offset: 2, limit: 1 }, undefined, stream) - expect(result.lines).toEqual([{ number: 2, text: 'two' }]) - expect(result.totalLines).toBe(3) - }) - - it('rejects a binary file on the streaming path', async () => { - const file = join(dir, 'bin') - await writeFile(file, Buffer.from([0x68, 0x00, 0x69])) - await expect(readTextPage(localTarget(file), READ_ALL, undefined, stream)).rejects.toMatchObject({ code: 'FS_NOT_TEXT' }) - }) - - it('caps a newline-free giant line without unbounded buffering', async () => { - const file = join(dir, 'one-line.txt') - await writeFile(file, 'z'.repeat(5000)) - const result = await readTextPage(localTarget(file), READ_ALL, undefined, stream) - expect(result.lines[0]?.text).toContain('... (line truncated to 2000 chars)') - expect(result.view).toBe('partial') - }) - - it('rejects invalid UTF-8 bytes on the streaming path', async () => { - const file = join(dir, 'invalid-utf8.txt') - await writeFile(file, Buffer.from([0x68, 0xff, 0x69])) - await expect(readTextPage(localTarget(file), READ_ALL, undefined, stream)).rejects.toMatchObject({ code: 'FS_NOT_TEXT' }) - }) - - it('honors abort on the streaming path', async () => { - const file = join(dir, 'a.txt') - await writeFile(file, 'one\ntwo') - await expect(readTextPage(localTarget(file), READ_ALL, AbortSignal.abort(), stream)).rejects.toMatchObject({ code: 'FS_ABORTED' }) - }) - - it('caps output bytes mid-stream', async () => { - const file = join(dir, 'big.txt') - await writeFile(file, Array.from({ length: 2000 }, () => 'y'.repeat(100)).join('\n')) - const result = await readTextPage(localTarget(file), READ_ALL, undefined, stream) - expect(result.truncatedByBytes).toBe(true) - }) - - it('flushes a final line with no trailing newline', async () => { - const file = join(dir, 'no-nl.txt') - await writeFile(file, 'one\ntwo') // no trailing \n - const result = await readTextPage(localTarget(file), READ_ALL, undefined, stream) - expect(result.lines.map(l => l.text)).toEqual(['one', 'two']) - }) - - it('handles a trailing newline (no dangling buffer at EOF)', async () => { - const file = join(dir, 'nl.txt') - await writeFile(file, 'one\ntwo\n') // trailing \n → empty buffer at end - const result = await readTextPage(localTarget(file), READ_ALL, undefined, stream) - expect(result.lines.map(l => l.text)).toEqual(['one', 'two']) - expect(result.totalLines).toBe(2) - }) - - it('passes a live (non-aborted) signal through to the stream', async () => { - const file = join(dir, 'a.txt') - await writeFile(file, 'one\ntwo') - const result = await readTextPage(localTarget(file), READ_ALL, new AbortController().signal, stream) - expect(result.totalLines).toBe(2) - }) - - it('scans across multiple stream chunks', async () => { - // A file well past the default 64 KB stream highWaterMark yields multiple chunks, - // exercising the non-first-chunk branch and the line-buffer cap across appends. - const file = join(dir, 'multi.txt') - const lines = Array.from({ length: 50 }, (_, i) => `line ${i}: ${'x'.repeat(3000)}`) - await writeFile(file, lines.join('\n')) - const result = await readTextPage(localTarget(file), { offset: 1, limit: 3 }, undefined, stream) - expect(result.lines[0]?.text.startsWith('line 0:')).toBe(true) - expect(result.lines[0]?.text).toContain('... (line truncated to 2000 chars)') - expect(result.totalLines).toBeGreaterThanOrEqual(3) - }) + expect(await readWholeText(localTarget(file), new AbortController().signal)).toBe('one\ntwo') }) }) -describe('writeFileAtomic — temp-file safety (defensive class A)', () => { +describe('streamWholeText', () => { + it('streams the whole file as decoded text', async () => { + const file = join(dir, 'a.txt') + await writeFile(file, 'one\ntwo\nthree') + expect(await collect(streamWholeText(localTarget(file)))).toBe('one\ntwo\nthree') + }) + + it('streams a large multi-chunk file correctly', async () => { + const file = join(dir, 'big.txt') + const content = Array.from({ length: 50 }, (_, i) => `line ${i}: ${'x'.repeat(3000)}`).join('\n') + await writeFile(file, content) + expect(await collect(streamWholeText(localTarget(file)))).toBe(content) + }) + + it('rejects a missing file, directory, binary, and invalid UTF-8', async () => { + await expect(collect(streamWholeText(localTarget(join(dir, 'nope'))))).rejects.toMatchObject({ code: 'FS_NOT_FOUND' }) + await expect(collect(streamWholeText(localTarget(dir)))).rejects.toMatchObject({ code: 'FS_NOT_REGULAR_FILE' }) + await writeFile(join(dir, 'bin'), Buffer.from([0x68, 0x00, 0x69])) + await expect(collect(streamWholeText(localTarget(join(dir, 'bin'))))).rejects.toMatchObject({ code: 'FS_NOT_TEXT' }) + await writeFile(join(dir, 'bad'), Buffer.from([0x68, 0xff, 0x69])) + await expect(collect(streamWholeText(localTarget(join(dir, 'bad'))))).rejects.toMatchObject({ code: 'FS_NOT_TEXT' }) + }) + + it('honors a pre-aborted signal', async () => { + const file = join(dir, 'a.txt') + await writeFile(file, 'one') + await expect(collect(streamWholeText(localTarget(file), AbortSignal.abort()))).rejects.toMatchObject({ code: 'FS_ABORTED' }) + }) + + it('passes a live (non-aborted) signal through the stream', async () => { + const file = join(dir, 'a.txt') + await writeFile(file, 'one\ntwo') + expect(await collect(streamWholeText(localTarget(file), new AbortController().signal))).toBe('one\ntwo') + }) +}) + +describe('writeFileAtomic — temp-file safety', () => { it('writes through a private staging dir and owner-only temp file', async () => { const file = join(dir, 'a.txt') let inspected = false @@ -259,8 +184,7 @@ describe('writeFileAtomic — temp-file safety (defensive class A)', () => { }) expect(inspected).toBe(true) expect(await readFile(file, 'utf8')).toBe('hello') - const info = await stat(file) - expect(info.mode & 0o777).toBe(0o640) + expect((await stat(file)).mode & 0o777).toBe(0o640) expect((await readdir(dir)).filter(n => n.includes('.tmp'))).toEqual([]) }) @@ -278,7 +202,6 @@ describe('writeFileAtomic — temp-file safety (defensive class A)', () => { await expect( writeFileAtomic(file, 'hello', undefined, undefined, { tempDirName: () => tempDirName }), ).rejects.toMatchObject({ code: 'EEXIST' }) - // The pre-existing staging dir is intact and the target was not created. expect(await readFile(join(dir, tempDirName, 'PRECIOUS'), 'utf8')).toBe('keep') await expect(stat(file)).rejects.toMatchObject({ code: 'ENOENT' }) }) @@ -303,9 +226,8 @@ describe('writeFileAtomic — temp-file safety (defensive class A)', () => { it('cleans up the temp file when the final rename fails', async () => { const sub = join(dir, 'occupied') - await mkdir(sub) // rename(temp, sub) fails because sub is a non-empty/dir target + await mkdir(sub) await expect(writeFileAtomic(sub, 'hi', undefined, undefined)).rejects.toBeInstanceOf(Error) - // No leftover staging dirs in the directory. expect((await readdir(dir)).filter(n => n.includes('.tmp'))).toEqual([]) }) }) @@ -346,16 +268,11 @@ describe('readForEdit + restoreLineEndings', () => { expect(restoreLineEndings(edited.content, original.lineEndings)).toBe('one\r\nTWO\r\n') }) - it('rejects a binary file', async () => { - const file = join(dir, 'bin') - await writeFile(file, Buffer.from([0x00, 0x01])) - await expect(readForEdit(file, file)).rejects.toMatchObject({ code: 'FS_NOT_TEXT' }) - }) - - it('rejects invalid UTF-8 bytes', async () => { - const file = join(dir, 'invalid-utf8.txt') - await writeFile(file, Buffer.from([0x68, 0xff, 0x69])) - await expect(readForEdit(file, file)).rejects.toMatchObject({ code: 'FS_NOT_TEXT' }) + it('rejects a binary file and invalid UTF-8', async () => { + await writeFile(join(dir, 'bin'), Buffer.from([0x00, 0x01])) + await expect(readForEdit(join(dir, 'bin'), join(dir, 'bin'))).rejects.toMatchObject({ code: 'FS_NOT_TEXT' }) + await writeFile(join(dir, 'bad'), Buffer.from([0x68, 0xff, 0x69])) + await expect(readForEdit(join(dir, 'bad'), join(dir, 'bad'))).rejects.toMatchObject({ code: 'FS_NOT_TEXT' }) }) it('passes a live (non-aborted) signal through the read', async () => { @@ -365,20 +282,3 @@ describe('readForEdit + restoreLineEndings', () => { expect(original.content).toBe('one\ntwo') }) }) - -describe('probe', () => { - it('returns null for a missing path and info for a file', async () => { - expect(await probe(join(dir, 'nope'))).toBeNull() - const file = join(dir, 'a.txt') - await writeFile(file, 'hi') - const info = await probe(file) - expect(info?.isFile).toBe(true) - expect(typeof info?.version).toBe('string') - }) - - it('marks a directory as not a regular file', async () => { - const sub = join(dir, 'sub') - await mkdir(sub) - expect((await probe(sub))?.isFile).toBe(false) - }) -}) diff --git a/packages/fs/fs/README.md b/packages/fs/fs/README.md index 856ec95076..1599ac25cc 100644 --- a/packages/fs/fs/README.md +++ b/packages/fs/fs/README.md @@ -1,38 +1,37 @@ # @deepseek-ai/dsh-fs -The **filesystem seam**: an abstract `FileSystem` service (`ctx.fs`) defining WHAT a filesystem backend does — resolve paths, read bounded text pages, create/replace files, apply literal edits — without saying HOW. +The **filesystem provider seam**: an abstract `FileSystem` service (`ctx.fs`) defining the text-storage primitives a backend provides — resolve a path, stat metadata, read/stream text, write atomically, and apply a guarded literal edit — without saying HOW. -This package is one third of the filesystem capability, split so each concern can evolve (and be swapped) independently (see [the capability-seam RFC](../../../docs/rfc/implemented/architecture/2026-06-13-capability-seams.md) and [the filesystem capability-seam RFC](../../../docs/rfc/implemented/architecture/2026-06-17-filesystem-capability-seam.md)): +This package is the provider-seam layer of the four-layer filesystem stack, split so each concern can evolve (and be swapped) independently (see [the capability-seam RFC](../../../docs/rfc/implemented/architecture/2026-06-13-capability-seams.md), [the filesystem capability-seam RFC](../../../docs/rfc/implemented/architecture/2026-06-17-filesystem-capability-seam.md), and [the split-the-filesystem-seam RFC](../../../docs/rfc/implemented/simplification/2026-06-26-fsspec-style-fs-seam.md)): -| Package | Role | -|---|---| -| `@deepseek-ai/dsh-fs` (this) | the interface: abstract service + vocabulary types + read-before-write/edit policy | -| `@deepseek-ai/dsh-fs-local` | an implementation: the host filesystem | -| `@deepseek-ai/dsh-tool-fs` | the model-facing `read`/`write`/`edit` tool schemas over `ctx.fs` | +| Layer | Package | Role | +|---|---|---| +| tool | `@deepseek-ai/dsh-tool-fs` | model-facing `read`/`write`/`edit` schemas + text rendering | +| policy | `@deepseek-ai/dsh-file-context` | `ctx.fileContext`: observed-state, read windowing, write/edit freshness | +| provider seam | `@deepseek-ai/dsh-fs` (this) | `ctx.fs`: text IO + guarded mutation primitives | +| provider | `@deepseek-ai/dsh-fs-local` | the host-filesystem implementation | -A future sandboxed, virtual, or remote backend implements this interface and the tool schemas don't change. +A future sandboxed, virtual, or remote backend implements this interface and the policy/tool layers don't change. ## Service API (`ctx.fs`) -Consumers call the concrete public API; backends implement the four primitives. +A backend subclasses `FileSystem` and implements six primitives. -| Member | Kind | Semantics | -|---|---|---| -| `resolve(path)` | primitive | Resolve a path into a stable `FsTarget` (`inputPath`, opaque `targetKey`, `displayPath`). Async — a remote backend may need I/O. The same file via different paths must yield the same `targetKey`. | -| `readPage(target, request, signal?)` | primitive | Read a bounded UTF-8 text page. Returns line-numbered content, `totalLines`, an opaque `version`, and a `view` (`full` only when the page covered the whole file). | -| `createOrReplace(target, content, expected, signal?)` | primitive | Create/replace a file honoring the `FsExpectation` stale guard. | -| `applyEdit(target, edit, expected, signal?)` | primitive | Atomic literal read-modify-write, verifying the expected version. `oldString` must be non-empty. | -| `read(target, request, exec?, signal?)` | public | Calls `readPage`, then records observed state for the derived owner. | -| `write(target, content, exec?, signal?)` | public | Builds the `FsExpectation` from recorded state, calls `createOrReplace`, refreshes state to `full`. Updating an existing file needs a prior `full` read; a create does not. | -| `edit(target, edit, exec?, signal?)` | public | Requires a prior `full` read by this owner (else `FS_NOT_OBSERVED` / `FS_PARTIAL_OBSERVATION`), rejects empty `oldString`, calls `applyEdit`, refreshes state. | -| `owner(exec?)` | helper | Derives the file-state owner (`exec.agent.session`) — `undefined` when there is none. | +| Member | Semantics | +|---|---| +| `resolve(path)` | Resolve a path into a stable `FsTarget` (`inputPath`, opaque `targetKey`, `displayPath`). Async — a remote backend may need I/O. The same file via different paths must yield the same `targetKey`. | +| `stat(target, signal?)` | Return `FsInfo` metadata (`version`, `type`, optional `size`), or `undefined` when the target is absent. Never content. | +| `readText(target, signal?)` | Read the whole regular text file as one decoded string. Owns regular-file checks, UTF-8 decoding, binary/NUL rejection (`FS_NOT_TEXT`). | +| `streamText(target, signal?)` | Stream the same text as decoded chunks for large files (cross-chunk UTF-8 decoding stays here). | +| `writeText(target, content, expected, signal?)` | Atomic create/replace honoring the `FsWriteExpectation` (`createIfAbsent` or `replaceIfVersion`). | +| `editText(target, edit, expected, signal?)` | Version-guarded literal edit. Verifies `expected.version` BEFORE matching, then applies the replacement and writes atomically — one mutation critical section. | -## Read-before-write/edit lives in the seam +## A provider seam, not the policy layer -Write/edit safety depends on backend-defined target identity and version tokens, so `ctx.fs` — not the tool layer — records what each owner has observed (keyed by an opaque owner object, normally the agent session, then by `targetKey`) and enforces the policy. The base class owns owner derivation, the file-state store, and *which* `FsExpectation` to hand the backend; the backend owns version comparison and I/O. Only a `full` view authorizes write/edit; a `partial` view (paged/truncated read) records context but does not. +`ctx.fs` is deliberately close to fsspec-style storage primitives — half a level above byte-level `cat`/`open`, because it decodes text and rejects binaries so the policy layer never touches raw bytes. It owns UTF-8 decoding, binary rejection, atomic writes, and the version-guarded literal-edit critical section. It does **not** own line windows, numbered lines, rendered footers, or observed-state — those model-facing read-windowing and read-before-write/edit policies live one layer up in `ctx.fileContext` ([`@deepseek-ai/dsh-file-context`](../file-context)), so a sandboxed/remote backend inherits no model-facing observation policy. -State is held in a `WeakMap` keyed by the owner object and dropped on disposal (HMR safety). Persistence across sessions is deferred — a resumed session must read files again before write/edit. +`editText` stays on this seam (not composed in the policy layer from a read plus a write) because version guard + literal match + atomic rewrite must stay inside one critical section for correct error attribution and one-wins/one-stale concurrency, and a remote backend may implement it as a native compare-and-edit. ## Vocabulary -`FsTarget` / `FsVersion` are opaque — consumers must not parse `targetKey` or interpret `version`; only `displayPath` is for model/UI output. Failures throw `FsError` (extends `HarnessError`, [the structured error taxonomy RFC](../../../docs/rfc/implemented/architecture/2026-06-11-structured-error-taxonomy.md)) carrying a stable `FsErrorCode` (`FS_NOT_FOUND`, `FS_NOT_TEXT`, `FS_NOT_REGULAR_FILE`, `FS_STALE_VERSION`, `FS_NOT_OBSERVED`, `FS_PARTIAL_OBSERVATION`, `FS_AMBIGUOUS_EDIT`, `FS_EDIT_NOT_FOUND`, `FS_ABORTED`); the tool registry surfaces `{ name, code }` on `isError` results. See `src/types.ts` for the full contracts. +`FsTargetKey` / `FsVersion` are branded opaque ids ([the branded-ids RFC](../../../docs/rfc/implemented/architecture/2026-06-20-branded-ids.md)) — consumers must not parse `targetKey` or interpret `version`; only `displayPath` is for model/UI output. `FsWriteExpectation` is the explicit write intent (`createIfAbsent` creates a missing target and rejects an existing one with `FS_NOT_OBSERVED`; `replaceIfVersion` replaces only at the observed version, else `FS_STALE_VERSION`). Failures throw `FsError` (extends `HarnessError`, [the structured error taxonomy RFC](../../../docs/rfc/implemented/architecture/2026-06-11-structured-error-taxonomy.md)) carrying a stable `FsErrorCode` (`FS_NOT_FOUND`, `FS_NOT_TEXT`, `FS_NOT_REGULAR_FILE`, `FS_STALE_VERSION`, `FS_NOT_OBSERVED`, `FS_AMBIGUOUS_EDIT`, `FS_EDIT_NOT_FOUND`, `FS_ABORTED`); the tool registry surfaces `{ name, code }` on `isError` results. See `src/types.ts` for the full contracts. diff --git a/packages/fs/fs/package.json b/packages/fs/fs/package.json index a0bce4940a..7520956310 100644 --- a/packages/fs/fs/package.json +++ b/packages/fs/fs/package.json @@ -20,10 +20,12 @@ ], "license": "BSD-3-Clause", "peerDependencies": { + "@deepseek-ai/dsh-brand": "^0.0.1", "@deepseek-ai/dsh-llm": "^0.0.1", "cordis": "^4.0.0-rc.6" }, "devDependencies": { + "@deepseek-ai/dsh-brand": "workspace:^", "@deepseek-ai/dsh-llm": "workspace:^", "cordis": "^4.0.0-rc.6" } diff --git a/packages/fs/fs/src/index.ts b/packages/fs/fs/src/index.ts index b23d7dd90a..70675b01c2 100644 --- a/packages/fs/fs/src/index.ts +++ b/packages/fs/fs/src/index.ts @@ -1,62 +1,62 @@ /** - * The filesystem seam (`ctx.fs`): an abstract service defining WHAT a - * filesystem backend does — resolve paths into stable targets, read bounded - * text pages, create/replace files, and apply literal edits — without saying - * HOW. Implementations subclass {@link FileSystem} and register themselves as - * the `fs` service; `@deepseek-ai/dsh-fs-local` (the host filesystem) is the - * first. Future implementations swap in sandboxed, remote, virtual, or - * project-scoped backends without touching the tool schemas that consume them + * The filesystem provider seam (`ctx.fs`): an abstract service defining the + * text-storage primitives a backend provides — resolve a path into a stable + * target, stat its metadata, read/stream its text, write it atomically with an + * explicit expectation, and apply a guarded literal edit — without saying HOW. + * Implementations subclass {@link FileSystem} and register themselves as the + * `fs` service; `@deepseek-ai/dsh-fs-local` (the host filesystem) is the first. + * Future implementations swap in sandboxed, remote, virtual, or project-scoped + * backends without touching the model-facing tool schemas * (`@deepseek-ai/dsh-tool-fs`). * - * The split mirrors the bash seam (`BashExecutor`/`LocalBashExecutor`). See - * the capability-seam RFC for why a swappable capability is three packages. + * The split mirrors the bash seam (`BashExecutor`/`LocalBashExecutor`). See the + * capability-seam RFC for why a swappable capability is three (here four) + * packages. * - * ## Read-before-write/edit lives here, not in the tools + * ## This is a provider seam, not the policy layer * - * Write/edit safety depends on backend-defined target identity and version - * tokens, so the seam — not the consumer — records what each owner has observed - * and enforces the policy. The base class owns owner derivation, the file-state - * store, and the decision of *which* {@link FsExpectation} to hand a backend; - * the backend owns version comparison and the actual I/O. A consumer passes its - * execution context through {@link read}/{@link write}/{@link edit} and never - * touches the cache, owner key, or version tokens. + * `ctx.fs` is deliberately close to fsspec-style storage primitives. It owns + * UTF-8 decoding, binary/NUL rejection, atomic full-file writes, and the + * version-guarded literal-edit critical section — but NOT line windows, + * numbered lines, rendered footers, or observed-state. Those model-facing + * read-windowing and read-before-write/edit policies live one layer up in the + * concrete `ctx.fileContext` service (`@deepseek-ai/dsh-file-context`), so a + * sandboxed/remote backend inherits no model-facing observation policy it has + * no business carrying. + * + * `editText` stays on this seam (not composed in the policy layer from a read + * plus a write) because version guard + literal match + atomic rewrite must + * stay inside one mutation critical section for correct error attribution and + * one-wins/one-stale concurrency, and a remote backend may implement it as a + * native compare-and-edit. * * @module @deepseek-ai/dsh-fs */ import { Context, Service } from 'cordis' -import { FsError } from './types.ts' import type { FsEditOutcome, FsEditRequest, - FsExecContext, - FsExpectation, - FsReadOutcome, - FsReadRequest, + FsInfo, FsTarget, FsVersion, + FsWriteExpectation, FsWriteOutcome, - FileState, } from './types.ts' export { FsError, + FsTargetKey, + FsVersion, } from './types.ts' export type { FsEditOutcome, FsEditRequest, FsErrorCode, - FsExecContext, - FsExpectation, - FsReadOutcome, - FsReadRequest, - FsStateSource, + FsInfo, FsTarget, - FsTextLine, - FsVersion, - FsView, + FsWriteExpectation, FsWriteOutcome, - FileState, } from './types.ts' declare module 'cordis' { @@ -66,50 +66,32 @@ declare module 'cordis' { } /** - * Abstract filesystem service. Subclass, implement the four backend primitives - * ({@link resolve}, {@link readPage}, {@link createOrReplace}, - * {@link applyEdit}), and load the subclass as a plugin — it registers as - * `ctx.fs` (one implementation per context; loading a second throws, cordis' - * standard duplicate-service behavior). - * - * Consumers call the concrete public API ({@link read}/{@link write}/ - * {@link edit}), which derives the file-state owner, enforces the - * read-before-write/edit policy, and refreshes recorded state — then delegates - * the actual I/O to the backend primitives. + * Abstract filesystem provider service. Subclass, implement the six text-storage + * primitives, and load the subclass as a plugin — it registers as `ctx.fs` (one + * implementation per context; loading a second throws, cordis' standard + * duplicate-service behavior). * * Semantics every backend must honor: * - {@link resolve} returns a stable {@link FsTarget}; the same underlying file * reached by different input paths must yield the same `targetKey` so stale - * guards and file-state lookup agree across paths (e.g. through symlinks). - * - {@link readPage} returns line-numbered UTF-8 content with a `version` and a - * `view` (`full` only when the page covered the whole file). - * - {@link createOrReplace} honors the {@link FsExpectation}: `observed` - * rejects with `FS_STALE_VERSION` if the file changed since `version`; - * `partial` rejects existing targets because the owner saw only a - * non-editable view; `unobserved` creates iff the target is absent and - * otherwise rejects. - * - {@link applyEdit} verifies the expected version (stale guard) and is atomic - * (read-modify-write must not interleave with a concurrent edit). + * guards and target lookup agree across paths (e.g. through symlinks). + * - {@link stat} returns {@link FsInfo} metadata (never content) or `undefined` + * when the target is absent. + * - {@link readText}/{@link streamText} read the whole regular text file (the + * stream for large files); both own regular-file checks, UTF-8 decoding, + * binary/NUL rejection, and `FS_NOT_TEXT`. + * - {@link writeText} is atomic temp-file + rename honoring the + * {@link FsWriteExpectation}. + * - {@link editText} verifies `expected.version` BEFORE literal matching (so a + * stale edit reports `FS_STALE_VERSION`, not `FS_EDIT_NOT_FOUND`/ + * `FS_AMBIGUOUS_EDIT` against newer content), then applies literal replacement + * and writes atomically — all inside one mutation critical section. */ export abstract class FileSystem extends Service { - /** - * Observed-file state, keyed first by the owner object (weakly held, so a - * collected session frees its state), then by {@link FsTarget.targetKey}. - */ - private fileStates = new WeakMap>() - constructor(ctx: Context) { super(ctx, 'fs') - ctx.effect(() => () => { - // Drop all recorded state on disposal so a reloaded backend starts clean - // (HMR safety). The WeakMap itself would be GC'd, but replacing it makes - // the release observable and immediate for tests. - this.fileStates = new WeakMap() - }, 'fs file-state teardown') } - // --- Backend primitives (subclass implements; all backend I/O lives here) --- - /** * Resolve a model/plugin-supplied path into a stable {@link FsTarget}. May * perform I/O (a remote/sandboxed backend may need a round-trip to map a path @@ -118,139 +100,32 @@ export abstract class FileSystem extends Service { */ abstract resolve(path: string): Promise - /** Read a bounded UTF-8 text page from a target. */ - abstract readPage(target: FsTarget, request: FsReadRequest, signal?: AbortSignal): Promise + /** Return target metadata, or `undefined` when the target does not exist. */ + abstract stat(target: FsTarget, signal?: AbortSignal): Promise + + /** Read the whole regular text file as a single decoded string. */ + abstract readText(target: FsTarget, signal?: AbortSignal): Promise /** - * Create or fully replace a UTF-8 text file, honoring `expected` as the - * stale guard / create-vs-update decision. + * Stream the whole regular text file as decoded text chunks (same text + * semantics as {@link readText}, for large files). The backend owns + * cross-chunk UTF-8 decoding and binary rejection so the policy layer never + * touches raw bytes. */ - abstract createOrReplace(target: FsTarget, content: string, expected: FsExpectation, signal?: AbortSignal): Promise + abstract streamText(target: FsTarget, signal?: AbortSignal): Promise> /** - * Apply a literal edit to an existing UTF-8 text file, verifying - * `expected.version` as the stale guard. Atomic read-modify-write. + * Create or fully replace a UTF-8 text file atomically, honoring `expected` + * as the create-vs-replace decision and stale guard. */ - abstract applyEdit(target: FsTarget, edit: FsEditRequest, expected: { version: FsVersion }, signal?: AbortSignal): Promise - - // --- Owner + file-state machinery (shared by all backends) --- + abstract writeText(target: FsTarget, content: string, expected: FsWriteExpectation, signal?: AbortSignal): Promise /** - * Derive the file-state owner from an execution context — normally the active - * agent session. Returns `undefined` when no owner can be derived (e.g. a - * direct tool call with no agent); such calls read freely but cannot satisfy - * the write/edit prior-observation policy. + * Apply a literal edit to an existing UTF-8 text file. Verifies + * `expected.version` as the stale guard BEFORE literal matching, then applies + * the replacement and writes atomically — one mutation critical section. */ - owner(exec?: FsExecContext): object | undefined { - return exec?.agent?.session - } - - /** Look up recorded state for an owner+target, if any. */ - protected getState(owner: object, targetKey: string): FileState | undefined { - return this.fileStates.get(owner)?.get(targetKey) - } - - /** Record (or replace) one owner's observed state for a target. */ - protected recordState(owner: object, state: FileState): void { - let byTarget = this.fileStates.get(owner) - if (!byTarget) { - byTarget = new Map() - this.fileStates.set(owner, byTarget) - } - byTarget.set(state.targetKey, state) - } - - // --- Concrete public API (orchestration; consumers call these) --- - - /** - * Read a bounded text page and, when an owner is derivable, record the - * observed state (a `full` view authorizes later write/edit; a `partial` view - * does not). - */ - async read(target: FsTarget, request: FsReadRequest, exec?: FsExecContext, signal?: AbortSignal): Promise { - const outcome = await this.readPage(target, request, signal) - const owner = this.owner(exec) - if (owner) { - this.recordState(owner, { - targetKey: target.targetKey, - displayPath: target.displayPath, - version: outcome.version, - view: outcome.view, - updatedAt: this.now(), - source: 'read', - }) - } - return outcome - } - - /** - * Create or fully replace a file. Updating an existing file requires a `full` - * prior observation by this owner; a create (no prior state, target absent) - * does not. After a successful write the recorded state refreshes to `full` - * at the new version so a follow-up modification needs no re-read. - */ - async write(target: FsTarget, content: string, exec?: FsExecContext, signal?: AbortSignal): Promise { - const owner = this.owner(exec) - const prior = owner ? this.getState(owner, target.targetKey) : undefined - const expected: FsExpectation = prior - ? prior.view === 'full' - ? { kind: 'observed', version: prior.version } - : { kind: 'partial', version: prior.version } - : { kind: 'unobserved' } - - const outcome = await this.createOrReplace(target, content, expected, signal) - if (owner) { - this.recordState(owner, { - targetKey: target.targetKey, - displayPath: target.displayPath, - version: outcome.version, - view: 'full', - updatedAt: this.now(), - source: 'write', - }) - } - return outcome - } - - /** - * Apply a literal edit. Always requires a `full` prior observation by this - * owner. No owner or absent state rejects with `FS_NOT_OBSERVED`; a partial - * view rejects with `FS_PARTIAL_OBSERVATION`; an empty `oldString` rejects - * before backend I/O. There is no "create via edit". Refreshes recorded - * state to `full` at the new version on success. - */ - async edit(target: FsTarget, edit: FsEditRequest, exec?: FsExecContext, signal?: AbortSignal): Promise { - if (edit.oldString.length === 0) { - throw new FsError('old_string must be a non-empty string', 'FS_EDIT_NOT_FOUND') - } - const owner = this.owner(exec) - const prior = owner ? this.getState(owner, target.targetKey) : undefined - if (!owner || !prior) { - throw new FsError(`edit requires reading "${target.displayPath}" first`, 'FS_NOT_OBSERVED') - } - if (prior.view !== 'full') { - throw new FsError(`edit requires a full read of "${target.displayPath}" first`, 'FS_PARTIAL_OBSERVATION') - } - - const outcome = await this.applyEdit(target, edit, { version: prior.version }, signal) - this.recordState(owner, { - targetKey: target.targetKey, - displayPath: target.displayPath, - version: outcome.version, - view: 'full', - updatedAt: this.now(), - source: 'edit', - }) - return outcome - } - - /** - * Wall-clock now (ms). A protected seam so tests can use deterministic - * timestamps; production uses `Date.now()`. - */ - protected now(): number { - return Date.now() - } + abstract editText(target: FsTarget, edit: FsEditRequest, expected: { version: FsVersion }, signal?: AbortSignal): Promise } export default FileSystem diff --git a/packages/fs/fs/src/types.ts b/packages/fs/fs/src/types.ts index f08723731e..62ba52b52f 100644 --- a/packages/fs/fs/src/types.ts +++ b/packages/fs/fs/src/types.ts @@ -1,34 +1,49 @@ /** - * Vocabulary for the filesystem capability seam (`ctx.fs`): the request/outcome - * shapes backends produce and consumers format, the opaque target/version - * identities, the per-owner file-state record, and the typed error taxonomy. + * Vocabulary for the filesystem provider seam (`ctx.fs`): the opaque + * target/version identities, the metadata `stat` returns, the write-expectation + * and outcome shapes, the literal-edit request/outcome, and the typed error + * taxonomy. * * These types are shared by every backend (`@deepseek-ai/dsh-fs-local` and - * future sandboxed/remote backends) and by the model-facing consumer - * (`@deepseek-ai/dsh-tool-fs`). They deliberately avoid host-path assumptions: - * `targetKey` and `version` are opaque tokens, and `displayPath` is the only - * field a consumer may show. + * future sandboxed/remote backends) and by the policy layer + * (`@deepseek-ai/dsh-file-context`). They are deliberately a *text-storage* + * vocabulary half a level above byte-level fsspec: `readText`/`streamText` hand + * back decoded text, never raw bytes. Host-path assumptions stay out — `targetKey` + * and `version` are opaque branded tokens, and `displayPath` is the only field a + * consumer may show. + * + * Model-facing concepts (line windows, numbered lines, observed-state) do NOT + * live here; they belong to the policy layer (`ctx.fileContext`). * * @module @deepseek-ai/dsh-fs/types */ import { HarnessError } from '@deepseek-ai/dsh-llm' +import type { Branded } from '@deepseek-ai/dsh-brand' /** - * Minimal structural view of a tool execution the filesystem seam needs to - * derive a file-state owner. `@deepseek-ai/dsh-tools`' `ToolExecution` - * satisfies this shape, so the consumer passes its `exec` straight through - * without `dsh-fs` importing `dsh-tools`, `dsh-agent`, or `dsh-session`. - * - * The owner is `agent.session` when present. It is treated as an opaque object - * identity (a `WeakMap` key); `dsh-fs` never reads any of its fields. + * Opaque key for stale guards and target lookup. The local backend uses a + * realpath-like string; a remote backend might use a workspace URI or file id. + * Consumers MUST NOT parse it or assume it is a local absolute path. */ -export interface FsExecContext { - /** The agent on whose behalf the call runs, when there is one. */ - agent?: { - /** The session that owns observed-file state, used as an opaque key. */ - session?: object - } +export type FsTargetKey = Branded<'FsTargetKey'> + +/** Brand a string as an {@link FsTargetKey}. */ +export function FsTargetKey(key: string): FsTargetKey { + return key as FsTargetKey +} + +/** + * Opaque file-version token — the freshness token a write/edit guards against. + * The local backend derives it from mtime+size; a remote backend might use a + * revision id. The policy layer records it for stale checks; consumers may + * display related metadata but MUST NOT interpret this token. + */ +export type FsVersion = Branded<'FsVersion'> + +/** Brand a string as an {@link FsVersion}. */ +export function FsVersion(v: string): FsVersion { + return v as FsVersion } /** @@ -38,12 +53,8 @@ export interface FsExecContext { export interface FsTarget { /** The original model/plugin-supplied path, for diagnostics only. */ inputPath: string - /** - * Opaque key for stale guards and file-state lookup. The local backend uses - * a realpath-like string; a remote backend might use a workspace URI or file - * id. Consumers MUST NOT parse it or assume it is a local absolute path. - */ - targetKey: string + /** Opaque key for stale guards and target lookup. */ + targetKey: FsTargetKey /** * Path for model/UI-facing output. May be a local absolute path, * workspace-relative path, or remote URI depending on the backend. @@ -52,64 +63,30 @@ export interface FsTarget { } /** - * Opaque file-version token. The local backend derives it from mtime+size; a - * remote backend might use a revision id. `ctx.fs` records it for stale checks; - * consumers may display related metadata but MUST NOT interpret this token. + * Metadata about a target — what {@link FileSystem.stat} returns. Lets the + * policy layer reject directories/special files before reading and choose + * `readText` vs `streamText` from `size` without probing by failure. `version` + * is the freshness token. `undefined` from `stat` means the target is absent. */ -export type FsVersion = string - -/** Resolved read window. The consumer applies its defaults/caps before calling. */ -export interface FsReadRequest { - /** 1-based first line to return. */ - offset: number - /** Maximum number of lines to return. */ - limit: number -} - -/** One line returned from a text file. */ -export interface FsTextLine { - /** 1-based line number in the file. */ - number: number - /** Line text without its trailing newline. */ - text: string -} - -/** Whether a recorded/returned view covers the whole file or only part of it. */ -export type FsView = 'full' | 'partial' - -/** Outcome of a bounded text read. */ -export interface FsReadOutcome { - /** 1-based first line requested. */ - offset: number - /** Maximum number of lines requested. */ - limit: number - /** Returned lines, already numbered. */ - lines: FsTextLine[] - /** Total line count in the file, unless `truncatedByBytes` stopped scanning early. */ - totalLines: number - /** Whether selected output hit the byte cap before EOF or the requested limit. */ - truncatedByBytes?: true - /** Opaque version of the file at read time. */ +export interface FsInfo { + /** Opaque freshness token of the target right now. */ version: FsVersion - /** - * Whether this read saw the whole file (`full`) or only part of it - * (`partial`). Only a `full` view authorizes a later write/edit. - */ - view: FsView + /** Whether the target is a regular file, a directory, or something else. */ + type: 'file' | 'directory' | 'other' + /** Byte size of a regular file, when the backend can report it. */ + size?: number } /** - * The read-before-write decision the base service hands to a backend for a - * full-file write. `observed` means the owner has a `full` view recorded at - * `version` (the backend rejects if the file has since changed); `partial` - * means the owner saw only a non-editable view of that target; `unobserved` - * means there is no prior view (the backend may create iff the target is - * absent, else rejects as not observed). + * The explicit intent of a {@link FileSystem.writeText} call. `createIfAbsent` + * creates a missing target and rejects an existing one with `FS_NOT_OBSERVED` + * (the path used when the owner has no prior read). `replaceIfVersion` replaces + * only when the target exists at the observed version; a missing target or a + * version mismatch throws `FS_STALE_VERSION`. */ -export type FsExpectation = - | { kind: 'observed'; version: FsVersion } - | { kind: 'partial'; version: FsVersion } - | { kind: 'unobserved' } +export type FsWriteExpectation = + | { kind: 'createIfAbsent' } + | { kind: 'replaceIfVersion'; version: FsVersion } /** Outcome of a full-file write. */ export interface FsWriteOutcome { @@ -139,29 +116,6 @@ export interface FsEditOutcome { version: FsVersion } -/** Source that last touched a recorded {@link FileState}. */ -export type FsStateSource = 'read' | 'write' | 'edit' - -/** - * What an owner has observed about one target. Keyed (inside the service) first - * by the owner object, then by {@link FsTarget.targetKey}. Only a `full` view - * authorizes write/edit. - */ -export interface FileState { - /** Backend target identity this state describes. */ - targetKey: string - /** Display path captured when the state was recorded. */ - displayPath: string - /** Opaque version the owner last saw. */ - version: FsVersion - /** Whether the owner saw the whole file or only part of it. */ - view: FsView - /** Wall-clock time the state was last updated (ms since epoch). */ - updatedAt: number - /** Operation that produced this state. */ - source: FsStateSource -} - /** * Stable, machine-routable codes for filesystem failures. Carried on * {@link FsError}; the tool registry surfaces `{ name, code }` on `isError` @@ -173,7 +127,6 @@ export type FsErrorCode = | 'FS_NOT_REGULAR_FILE' | 'FS_STALE_VERSION' | 'FS_NOT_OBSERVED' - | 'FS_PARTIAL_OBSERVATION' | 'FS_AMBIGUOUS_EDIT' | 'FS_EDIT_NOT_FOUND' | 'FS_ABORTED' diff --git a/packages/fs/fs/tests/service.spec.ts b/packages/fs/fs/tests/service.spec.ts index 84f84cbe0b..e06cfa9ee6 100644 --- a/packages/fs/fs/tests/service.spec.ts +++ b/packages/fs/fs/tests/service.spec.ts @@ -1,98 +1,69 @@ /** - * Tests for the filesystem service seam itself: registration/disposal, owner - * derivation, and the read-before-write/edit policy the base class enforces - * (which `FsExpectation` it hands the backend, multi-owner isolation, and - * state refresh) — all exercised through a fake in-memory backend that records - * the expectations it received. + * Tests for the filesystem provider seam itself: registration, duplicate-service + * behavior, disposal, and the branded id factories. The provider primitives and + * policy live in `dsh-fs-local` and `dsh-file-context`; this seam owns only the + * abstract service contract, so a minimal fake backend exercises it. */ import { describe, expect, it } from 'vitest' import { Context } from 'cordis' -import { FileSystem, FsError } from '@deepseek-ai/dsh-fs' +import { FileSystem, FsError, FsTargetKey, FsVersion } from '@deepseek-ai/dsh-fs' import type { FsEditOutcome, FsEditRequest, - FsExpectation, - FsReadOutcome, - FsReadRequest, + FsInfo, FsTarget, - FsView, + FsWriteExpectation, FsWriteOutcome, } from '@deepseek-ai/dsh-fs' -/** A fake backend: an in-memory file table, recording every expectation it is handed. */ +/** A minimal in-memory fake implementing the six provider primitives. */ class FakeFileSystem extends FileSystem { files = new Map() - versions = new Map() - /** View the next `readPage` should report (tests flip this for partial reads). */ - nextReadView: FsView = 'full' - /** Expectations handed to `createOrReplace`, in call order. */ - writeExpectations: FsExpectation[] = [] - /** Versions handed to `applyEdit`, in call order. */ - editExpectedVersions: string[] = [] - - private bump(key: string): string { - const next = (this.versions.get(key) ?? 0) + 1 - this.versions.set(key, next) - return `v${next}` - } override async resolve(path: string): Promise { - return { inputPath: path, targetKey: path, displayPath: path } + return { inputPath: path, targetKey: FsTargetKey(path), displayPath: path } } - - override async readPage(target: FsTarget, request: FsReadRequest): Promise { + override async stat(target: FsTarget): Promise { + const content = this.files.get(target.targetKey) + if (content === undefined) return undefined + return { version: FsVersion('v1'), type: 'file', size: content.length } + } + override async readText(target: FsTarget): Promise { const content = this.files.get(target.targetKey) if (content === undefined) throw new FsError(`not found: ${target.displayPath}`, 'FS_NOT_FOUND') - const allLines = content.split('\n') - const lines = allLines - .slice(request.offset - 1, request.offset - 1 + request.limit) - .map((text, i) => ({ number: request.offset + i, text })) - return { - offset: request.offset, - limit: request.limit, - lines, - totalLines: allLines.length, - version: `v${this.versions.get(target.targetKey) ?? 0}`, - view: this.nextReadView, - } + return content } - - override async createOrReplace(target: FsTarget, content: string, expected: FsExpectation): Promise { - this.writeExpectations.push(expected) + override async streamText(target: FsTarget): Promise> { + const content = await this.readText(target) + return (async function* () { yield content })() + } + override async writeText(target: FsTarget, content: string, _expected: FsWriteExpectation): Promise { const existed = this.files.has(target.targetKey) this.files.set(target.targetKey, content) - return { operation: existed ? 'update' : 'create', version: this.bump(target.targetKey) } + return { operation: existed ? 'update' : 'create', version: FsVersion('v2') } } - - override async applyEdit(target: FsTarget, edit: FsEditRequest, expected: { version: string }): Promise { - this.editExpectedVersions.push(expected.version) + override async editText(target: FsTarget, edit: FsEditRequest): Promise { const content = this.files.get(target.targetKey) ?? '' this.files.set(target.targetKey, content.split(edit.oldString).join(edit.newString)) - return { replacements: 1, replaceAll: edit.replaceAll, version: this.bump(target.targetKey) } + return { replacements: 1, replaceAll: edit.replaceAll, version: FsVersion('v3') } } } -async function setup() { - const ctx = new Context() - await ctx.plugin(FakeFileSystem) - const fs = ctx.fs as FakeFileSystem - return { ctx, fs } -} - -const READ_ALL: FsReadRequest = { offset: 1, limit: 2000 } -const ownerExec = (session: object) => ({ agent: { session } }) - -describe('FileSystem service seam', () => { - it('registers as ctx.fs and serves the API', async () => { - const { fs } = await setup() +describe('FileSystem provider seam', () => { + it('registers as ctx.fs and serves the primitives', async () => { + const ctx = new Context() + await ctx.plugin(FakeFileSystem) + const fs = ctx.fs as FakeFileSystem fs.files.set('a.txt', 'hi') - const outcome = await fs.read(await fs.resolve('a.txt'), READ_ALL) - expect(outcome.lines).toEqual([{ number: 1, text: 'hi' }]) + const target = await fs.resolve('a.txt') + expect((await fs.stat(target))?.type).toBe('file') + expect(await fs.readText(target)).toBe('hi') }) it('throws when a second implementation is loaded (duplicate service)', async () => { - const { ctx } = await setup() + const ctx = new Context() + await ctx.plugin(FakeFileSystem) await expect(ctx.plugin(FakeFileSystem)).rejects.toThrow() }) @@ -103,203 +74,30 @@ describe('FileSystem service seam', () => { await fiber.dispose() expect(ctx.fs).toBeUndefined() }) -}) -describe('owner derivation', () => { - it('derives the owner from exec.agent.session', async () => { - const { fs } = await setup() - const session = {} - expect(fs.owner(ownerExec(session))).toBe(session) - }) - - it('returns undefined with no exec, no agent, or no session', async () => { - const { fs } = await setup() - expect(fs.owner()).toBeUndefined() - expect(fs.owner({})).toBeUndefined() - expect(fs.owner({ agent: {} })).toBeUndefined() - }) -}) - -describe('read records observed state', () => { - it('a full read authorizes a later in-place write (observed expectation)', async () => { - const { fs } = await setup() - const exec = ownerExec({}) - fs.files.set('a.txt', 'hello') - const target = await fs.resolve('a.txt') - - await fs.read(target, READ_ALL, exec) - await fs.write(target, 'goodbye', exec) - - expect(fs.writeExpectations).toEqual([{ kind: 'observed', version: 'v0' }]) - }) - - it('a partial read does NOT authorize a write (passes a partial expectation)', async () => { - const { fs } = await setup() - const exec = ownerExec({}) - fs.files.set('a.txt', 'hello') - fs.nextReadView = 'partial' - const target = await fs.resolve('a.txt') - - await fs.read(target, { offset: 1, limit: 1 }, exec) - await fs.write(target, 'goodbye', exec) - - expect(fs.writeExpectations).toEqual([{ kind: 'partial', version: 'v0' }]) - }) - - it('skips recording when there is no owner', async () => { - const { fs } = await setup() - fs.files.set('a.txt', 'hello') - const target = await fs.resolve('a.txt') - - await fs.read(target, READ_ALL) // no exec - await fs.write(target, 'goodbye') // no exec → cannot be observed - - expect(fs.writeExpectations).toEqual([{ kind: 'unobserved' }]) - }) -}) - -describe('write policy', () => { - it('a create (no prior state) is unobserved', async () => { - const { fs } = await setup() - const exec = ownerExec({}) - const target = await fs.resolve('new.txt') - - const outcome = await fs.write(target, 'fresh', exec) - - expect(outcome.operation).toBe('create') - expect(fs.writeExpectations).toEqual([{ kind: 'unobserved' }]) - }) - - it('refreshes state to full after a write, so a follow-up edit needs no re-read', async () => { - const { fs } = await setup() - const exec = ownerExec({}) - const target = await fs.resolve('a.txt') - - await fs.write(target, 'one', exec) // create → state now full at v1 - await fs.edit(target, { oldString: 'one', newString: 'two', replaceAll: false }, exec) - - expect(fs.editExpectedVersions).toEqual(['v1']) - }) -}) - -describe('edit policy', () => { - it('rejects with FS_NOT_OBSERVED when the file was never read', async () => { - const { fs } = await setup() - const exec = ownerExec({}) - fs.files.set('a.txt', 'hello') - const target = await fs.resolve('a.txt') - - await expect( - fs.edit(target, { oldString: 'hello', newString: 'bye', replaceAll: false }, exec), - ).rejects.toMatchObject({ code: 'FS_NOT_OBSERVED' }) - }) - - it('rejects with FS_PARTIAL_OBSERVATION when only a partial view was recorded', async () => { - const { fs } = await setup() - const exec = ownerExec({}) - fs.files.set('a.txt', 'hello') - fs.nextReadView = 'partial' - const target = await fs.resolve('a.txt') - await fs.read(target, { offset: 1, limit: 1 }, exec) - - await expect( - fs.edit(target, { oldString: 'hello', newString: 'bye', replaceAll: false }, exec), - ).rejects.toMatchObject({ code: 'FS_PARTIAL_OBSERVATION' }) - }) - - it('rejects an empty oldString before calling the backend primitive', async () => { - const { fs } = await setup() - const exec = ownerExec({}) - fs.files.set('a.txt', 'hello') - const target = await fs.resolve('a.txt') - await fs.read(target, READ_ALL, exec) - - await expect( - fs.edit(target, { oldString: '', newString: 'bye', replaceAll: false }, exec), - ).rejects.toMatchObject({ code: 'FS_EDIT_NOT_FOUND' }) - expect(fs.editExpectedVersions).toEqual([]) - }) - - it('rejects when there is no owner (cannot prove prior observation)', async () => { - const { fs } = await setup() - fs.files.set('a.txt', 'hello') - const target = await fs.resolve('a.txt') - - await expect( - fs.edit(target, { oldString: 'hello', newString: 'bye', replaceAll: false }), - ).rejects.toMatchObject({ code: 'FS_NOT_OBSERVED' }) - }) - - it('proceeds after a full read, passing the recorded version as the stale guard', async () => { - const { fs } = await setup() - const exec = ownerExec({}) - fs.files.set('a.txt', 'hello') - fs.versions.set('a.txt', 7) // distinguishable version - const target = await fs.resolve('a.txt') - await fs.read(target, READ_ALL, exec) - - await fs.edit(target, { oldString: 'hello', newString: 'bye', replaceAll: false }, exec) - - expect(fs.editExpectedVersions).toEqual(['v7']) - }) -}) - -describe('multi-owner isolation', () => { - it('owner A reading does not grant owner B edit authority', async () => { - const { fs } = await setup() - const a = ownerExec({}) - const b = ownerExec({}) - fs.files.set('a.txt', 'hello') - const target = await fs.resolve('a.txt') - - await fs.read(target, READ_ALL, a) - - // B never read it → B's edit must be rejected. - await expect( - fs.edit(target, { oldString: 'hello', newString: 'bye', replaceAll: false }, b), - ).rejects.toMatchObject({ code: 'FS_NOT_OBSERVED' }) - // A still may edit. - await expect( - fs.edit(target, { oldString: 'hello', newString: 'bye', replaceAll: false }, a), - ).resolves.toMatchObject({ replacements: 1 }) - }) - - it('each owner records its own observed version independently', async () => { - const { fs } = await setup() - const a = ownerExec({}) - const b = ownerExec({}) - fs.files.set('a.txt', 'hello') - const target = await fs.resolve('a.txt') - - await fs.read(target, READ_ALL, a) // A sees v0 - await fs.write(target, 'mid', b) // B writes unobserved → file now v1 - await fs.write(target, 'late', a) // A still holds its v0 observation - - expect(fs.writeExpectations).toEqual([ - { kind: 'unobserved' }, - { kind: 'observed', version: 'v0' }, - ]) - }) -}) - -describe('disposal releases recorded state', () => { - it('a fresh provider after disposal starts with no inherited state', async () => { + it('streamText yields the same text readText returns', async () => { const ctx = new Context() - const fiber = await ctx.plugin(FakeFileSystem) - const fs1 = ctx.fs as FakeFileSystem - const exec = ownerExec({}) - fs1.files.set('a.txt', 'hello') - await fs1.read(await fs1.resolve('a.txt'), READ_ALL, exec) - await fiber.dispose() - await ctx.plugin(FakeFileSystem) - const fs2 = ctx.fs as FakeFileSystem - fs2.files.set('a.txt', 'hello') - const target = await fs2.resolve('a.txt') - // Reusing the same exec/owner object: state must NOT carry over. - await expect( - fs2.edit(target, { oldString: 'hello', newString: 'bye', replaceAll: false }, exec), - ).rejects.toMatchObject({ code: 'FS_NOT_OBSERVED' }) + const fs = ctx.fs as FakeFileSystem + fs.files.set('a.txt', 'one\ntwo') + const target = await fs.resolve('a.txt') + let streamed = '' + for await (const chunk of await fs.streamText(target)) streamed += chunk + expect(streamed).toBe(await fs.readText(target)) + }) + + it('stat returns undefined for an absent target', async () => { + const ctx = new Context() + await ctx.plugin(FakeFileSystem) + const fs = ctx.fs as FakeFileSystem + expect(await fs.stat(await fs.resolve('missing.txt'))).toBeUndefined() + }) +}) + +describe('branded id factories', () => { + it('FsTargetKey and FsVersion brand a string at compile time (identity at runtime)', () => { + expect(FsTargetKey('k')).toBe('k') + expect(FsVersion('v')).toBe('v') }) }) diff --git a/packages/fs/fs/tsconfig.json b/packages/fs/fs/tsconfig.json index 7b250a29c4..1ed5f54447 100644 --- a/packages/fs/fs/tsconfig.json +++ b/packages/fs/fs/tsconfig.json @@ -8,6 +8,7 @@ "references": [ { "path": "../../../vendor/cosmokit" }, { "path": "../../../vendor/cordis" }, + { "path": "../../util/brand" }, { "path": "../../llm/llm" } ] } diff --git a/packages/fs/tool-fs/README.md b/packages/fs/tool-fs/README.md index 2beb45be9c..f751ecb051 100644 --- a/packages/fs/tool-fs/README.md +++ b/packages/fs/tool-fs/README.md @@ -1,11 +1,12 @@ # @deepseek-ai/dsh-tool-fs -The **model-facing filesystem tools** — `read`, `write`, `edit` — over the `ctx.fs` seam ([`@deepseek-ai/dsh-fs`](../fs)). This is the consumer third of the filesystem capability; it owns tool names, JSON schemas, argument validation, prompt sections, and result formatting, and **never** touches filesystem I/O (no `node:fs`/`node:path`, no implementation import). +The **model-facing filesystem tools** — `read`, `write`, `edit` — over the `ctx.fileContext` policy layer ([`@deepseek-ai/dsh-file-context`](../file-context)). This is the consumer layer of the filesystem stack; it owns tool names, JSON schemas, argument validation, prompt sections, and result formatting, and **never** touches filesystem I/O (no `node:fs`/`node:path`, no implementation import) or reaches around the policy layer to `ctx.fs`. ```ts ignore-check -// Load a ctx.fs provider first, then the tools. +// Load a ctx.fs provider, the policy layer, then the tools. await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) // @deepseek-ai/dsh-fs-local -await ctx.plugin(ToolFs) // this package — registers read/write/edit +await ctx.plugin(FileContext) // @deepseek-ai/dsh-file-context +await ctx.plugin(ToolFs) // this package — registers read/write/edit ``` Each tool also ships as a subpath plugin for focused deployments: @@ -21,13 +22,17 @@ import * as editPlugin from '@deepseek-ai/dsh-tool-fs/edit' | Tool | Arguments | Behavior | |---|---|---| | `read` | `file_path`, `offset?`, `limit?` | Line-numbered UTF-8 content with a pagination footer. `offset` is 1-based; `limit` defaults to and caps at 2000 lines. | -| `write` | `file_path`, `content` | Create or fully replace a file. Overwriting an existing file requires a prior `read` (the backend enforces it); creating a new file does not. | -| `edit` | `file_path`, non-empty `old_string`, `new_string`, `replace_all?` | Literal replacement; unique match required unless `replace_all` is true. Requires a prior `read`. | +| `write` | `file_path`, `content` | Create or fully replace a file. Overwriting an existing file requires a prior `read` at the unchanged version; creating a new file does not. | +| `edit` | `file_path`, non-empty `old_string`, `new_string`, `replace_all?` | Literal replacement; unique match required unless `replace_all` is true. Requires a prior `read` (any window) and the file unchanged since. | Field names are snake_case to match Claude Code and existing harness tool schemas. -## How the read-before-write policy is enforced +## How the read-before-write/edit policy is enforced -The tools do **not** check whether a `read` ran or inspect any cache. Each tool resolves the path via `ctx.fs.resolve()`, then calls `ctx.fs.read/write/edit(target, …, exec)` — passing the current tool execution context straight through. `ctx.fs` derives the file-state owner (normally the agent session) from that context and owns the prior-observation and stale-version policy. Backend errors (`FsError`) flow through `ToolRegistry.execute()` and become `isError` tool results with their `{ name, code }` attached. +The tools do **not** check whether a `read` ran or inspect any cache. Each tool resolves the path via `ctx.fileContext.resolve()`, then calls `ctx.fileContext.read/write/edit(target, …, exec)` — passing the current tool execution context straight through. `ctx.fileContext` derives the observed-state owner (normally the agent session) from that context and owns the freshness policy: a recorded read at the file's current version authorizes a write/edit, and any windowed read counts (authorization is freshness, not a full-view requirement). Backend errors (`FsError`) flow through `ToolRegistry.execute()` and become `isError` tool results with their `{ name, code }` attached. + +## The no-bypass contract + +A model-facing read MUST go through `ctx.fileContext.read`, never `ctx.fs.readText`/`streamText`, so every successful read records observed-state before rendering — which is why the tools inject `fileContext`, not `fs`. Direct `ctx.fs` calls remain an explicit escape hatch for non-tool consumers: a direct `ctx.fs.readText` records nothing, so a later `edit` rejects with `FS_NOT_OBSERVED` until the file is read through `ctx.fileContext`. Tool schemas reach the system prompt automatically via the tool registry; this package additionally registers short prose guidance through `ctx.systemPrompt.section(...)`. diff --git a/packages/fs/tool-fs/package.json b/packages/fs/tool-fs/package.json index 744a41736d..f158142fca 100644 --- a/packages/fs/tool-fs/package.json +++ b/packages/fs/tool-fs/package.json @@ -32,6 +32,7 @@ ], "license": "BSD-3-Clause", "peerDependencies": { + "@deepseek-ai/dsh-file-context": "^0.0.1", "@deepseek-ai/dsh-fs": "^0.0.1", "@deepseek-ai/dsh-llm": "^0.0.1", "@deepseek-ai/dsh-system-prompt": "^0.0.1", @@ -40,6 +41,7 @@ }, "devDependencies": { "@deepseek-ai/dsh-agent": "workspace:^", + "@deepseek-ai/dsh-file-context": "workspace:^", "@deepseek-ai/dsh-fs": "workspace:^", "@deepseek-ai/dsh-fs-local": "workspace:^", "@deepseek-ai/dsh-llm": "workspace:^", diff --git a/packages/fs/tool-fs/src/edit.ts b/packages/fs/tool-fs/src/edit.ts index 3f65f5660d..d54fe3045b 100644 --- a/packages/fs/tool-fs/src/edit.ts +++ b/packages/fs/tool-fs/src/edit.ts @@ -1,8 +1,8 @@ /** * The model-facing `edit` tool: update an existing UTF-8 text file by replacing * literal text, requiring a unique match by default. Execution goes through - * `ctx.fs`, which enforces prior observation and the stale-version guard and - * owns the literal-match semantics. + * `ctx.fileContext`, which enforces prior observation (the freshness policy) + * and delegates the literal-match + stale-guard critical section to `ctx.fs`. * * @module @deepseek-ai/dsh-tool-fs/edit */ @@ -60,8 +60,8 @@ export function apply(ctx: Context): void { }, async execute(args, exec): Promise { const input = parseEditArgs(args) - const target = await ctx.fs.resolve(input.filePath) - const outcome = await ctx.fs.edit( + const target = await ctx.fileContext.resolve(input.filePath) + const outcome = await ctx.fileContext.edit( target, { oldString: input.oldString, newString: input.newString, replaceAll: input.replaceAll }, exec, @@ -76,7 +76,7 @@ export function apply(ctx: Context): void { export const name = 'fs-edit' /** Services required by the `edit` tool plugin. */ -export const inject = ['tools', 'fs', 'systemPrompt'] +export const inject = ['tools', 'fileContext', 'systemPrompt'] /** Named helper for direct registration in the root plugin and tests. */ export const applyEditTool = apply diff --git a/packages/fs/tool-fs/src/index.ts b/packages/fs/tool-fs/src/index.ts index 437c16b5dd..5509c7980b 100644 --- a/packages/fs/tool-fs/src/index.ts +++ b/packages/fs/tool-fs/src/index.ts @@ -1,13 +1,16 @@ /** * The model-facing filesystem tool suite (`read`, `write`, `edit`) over the - * `ctx.fs` seam. This root plugin registers all three tools by composing the - * per-tool registration helpers; each tool is also exposed as a subpath plugin - * (`@deepseek-ai/dsh-tool-fs/read`, `/write`, `/edit`) for focused deployments. + * `ctx.fileContext` policy layer. This root plugin registers all three tools by + * composing the per-tool registration helpers; each tool is also exposed as a + * subpath plugin (`@deepseek-ai/dsh-tool-fs/read`, `/write`, `/edit`) for focused + * deployments. * * The package owns model-facing concerns only — tool names, JSON schemas, * argument validation, prompt sections, result formatting. All filesystem - * execution goes through `ctx.fs`; this package never imports `node:fs`, - * `node:path`, or an `@deepseek-ai/dsh-fs-local` implementation. + * execution goes through `ctx.fileContext` (never directly around it to + * `ctx.fs`), so every model read records observed-state before rendering; this + * package never imports `node:fs`, `node:path`, or an + * `@deepseek-ai/dsh-fs-local` implementation. * * @module @deepseek-ai/dsh-tool-fs */ @@ -25,7 +28,7 @@ export { applyEditTool, formatEditOutput, parseEditArgs } from './edit.ts' export const name = 'tool-fs' /** Services required by the filesystem tool suite. */ -export const inject = ['tools', 'fs', 'systemPrompt'] +export const inject = ['tools', 'fileContext', 'systemPrompt'] /** Register the full `read`/`write`/`edit` filesystem tool suite. */ export function apply(ctx: Context): void { diff --git a/packages/fs/tool-fs/src/read.ts b/packages/fs/tool-fs/src/read.ts index bfa67a588f..8a6ae8d609 100644 --- a/packages/fs/tool-fs/src/read.ts +++ b/packages/fs/tool-fs/src/read.ts @@ -1,8 +1,9 @@ /** * The model-facing `read` tool: inspect a UTF-8 text file and return * line-numbered content with pagination guidance. Execution goes through - * `ctx.fs` — this module owns only the model-facing schema, argument - * validation, and result formatting, never filesystem I/O. + * `ctx.fileContext` (which records observed state and owns read windowing) — + * this module owns only the model-facing schema, argument validation, and + * result formatting, never filesystem I/O. * * @module @deepseek-ai/dsh-tool-fs/read */ @@ -10,7 +11,7 @@ import type { Context } from 'cordis' import { defineTool } from '@deepseek-ai/dsh-tools' import type { ContentBlock } from '@deepseek-ai/dsh-llm' -import type { FsReadOutcome } from '@deepseek-ai/dsh-fs' +import type { FileReadOutcome } from '@deepseek-ai/dsh-file-context' import type {} from '@deepseek-ai/dsh-system-prompt' /** Default and maximum number of lines returned by one `read` call. */ @@ -40,7 +41,7 @@ export function parseReadArgs(args: { file_path: string; offset?: number; limit? } /** Format a read outcome as one OpenCode-style line-numbered text block body. */ -export function formatReadOutput(displayPath: string, outcome: FsReadOutcome): string { +export function formatReadOutput(displayPath: string, outcome: FileReadOutcome): string { const endLine = outcome.lines.at(-1)?.number ?? Math.max(0, outcome.offset - 1) let footer: string if (outcome.truncatedByBytes) { @@ -78,8 +79,8 @@ export function apply(ctx: Context): void { }, async execute(args, exec): Promise { const input = parseReadArgs(args) - const target = await ctx.fs.resolve(input.filePath) - const outcome = await ctx.fs.read(target, { offset: input.offset, limit: input.limit }, exec, exec.signal) + const target = await ctx.fileContext.resolve(input.filePath) + const outcome = await ctx.fileContext.read(target, { offset: input.offset, limit: input.limit }, exec, exec.signal) return [{ type: 'text', text: formatReadOutput(target.displayPath, outcome) }] }, })) @@ -89,7 +90,7 @@ export function apply(ctx: Context): void { export const name = 'fs-read' /** Services required by the `read` tool plugin. */ -export const inject = ['tools', 'fs', 'systemPrompt'] +export const inject = ['tools', 'fileContext', 'systemPrompt'] /** Named helper for direct registration in the root plugin and tests. */ export const applyReadTool = apply diff --git a/packages/fs/tool-fs/src/write.ts b/packages/fs/tool-fs/src/write.ts index ff66d10127..8c242c5256 100644 --- a/packages/fs/tool-fs/src/write.ts +++ b/packages/fs/tool-fs/src/write.ts @@ -1,8 +1,8 @@ /** * The model-facing `write` tool: create or fully replace a UTF-8 text file. - * Execution goes through `ctx.fs`, which enforces the read-before-overwrite - * policy (updating an existing file requires a prior read in the same - * execution context; creating a new file does not). + * Execution goes through `ctx.fileContext`, which enforces the freshness policy + * (creating a new file needs no prior read; replacing an existing file requires + * a prior read in the same execution context at the unchanged version). * * @module @deepseek-ai/dsh-tool-fs/write */ @@ -46,8 +46,8 @@ export function apply(ctx: Context): void { }, async execute(args, exec): Promise { const input = parseWriteArgs(args) - const target = await ctx.fs.resolve(input.filePath) - const outcome = await ctx.fs.write(target, input.content, exec, exec.signal) + const target = await ctx.fileContext.resolve(input.filePath) + const outcome = await ctx.fileContext.write(target, input.content, exec, exec.signal) return [{ type: 'text', text: formatWriteOutput(target.displayPath, outcome) }] }, })) @@ -57,7 +57,7 @@ export function apply(ctx: Context): void { export const name = 'fs-write' /** Services required by the `write` tool plugin. */ -export const inject = ['tools', 'fs', 'systemPrompt'] +export const inject = ['tools', 'fileContext', 'systemPrompt'] /** Named helper for direct registration in the root plugin and tests. */ export const applyWriteTool = apply diff --git a/packages/fs/tool-fs/tests/integration.spec.ts b/packages/fs/tool-fs/tests/integration.spec.ts index 6f81763241..d61bdb5cac 100644 --- a/packages/fs/tool-fs/tests/integration.spec.ts +++ b/packages/fs/tool-fs/tests/integration.spec.ts @@ -1,8 +1,9 @@ /** - * Integration tests: the real local backend (`dsh-fs-local`) plus the model - * tools (`dsh-tool-fs`), exercised through `ctx.tools.execute()` so nothing - * bypasses the tool registry. These verify the WORLD — files are read back from - * disk and asserted byte-for-byte — not the tool's self-report. + * Integration tests: the real local backend (`dsh-fs-local`) plus the real + * policy layer (`dsh-file-context`) plus the model tools (`dsh-tool-fs`), + * exercised through `ctx.tools.execute()` so nothing bypasses the tool registry. + * These verify the WORLD — files are read back from disk and asserted + * byte-for-byte — not the tool's self-report. */ import { afterEach, beforeEach, describe, expect, it } from 'vitest' @@ -14,6 +15,7 @@ import { CallId } from '@deepseek-ai/dsh-llm' import SystemPrompt from '@deepseek-ai/dsh-system-prompt' import ToolRegistry from '@deepseek-ai/dsh-tools' import { LocalFileSystem } from '@deepseek-ai/dsh-fs-local' +import FileContext from '@deepseek-ai/dsh-file-context' import * as ToolFs from '@deepseek-ai/dsh-tool-fs' let dir: string @@ -28,6 +30,7 @@ beforeEach(async () => { await ctx.plugin(SystemPrompt) await ctx.plugin(ToolRegistry) await ctx.plugin(LocalFileSystem, { cwd: dir }) + await ctx.plugin(FileContext) fiber = await ctx.plugin(ToolFs) }) afterEach(async () => { @@ -61,7 +64,6 @@ describe('write → disk', () => { const result = await call('write', { file_path: 'a.txt', content: 'clobber' }) expect(result.isError).toBe(true) expect(result.error).toMatchObject({ code: 'FS_NOT_OBSERVED' }) - // The world is unchanged. expect(await readFile(join(dir, 'a.txt'), 'utf8')).toBe('original') }) @@ -72,6 +74,15 @@ describe('write → disk', () => { expect(result.isError).toBe(false) expect(await readFile(join(dir, 'a.txt'), 'utf8')).toBe('replaced') }) + + it('rejects a full overwrite when the file changed since the read (stale)', async () => { + await writeFile(join(dir, 'a.txt'), 'original') + await call('read', { file_path: 'a.txt' }) + await writeFile(join(dir, 'a.txt'), 'changed-externally') // out-of-band change + const result = await call('write', { file_path: 'a.txt', content: 'replaced' }) + expect(result.isError).toBe(true) + expect(result.error).toMatchObject({ code: 'FS_STALE_VERSION' }) + }) }) describe('read', () => { @@ -89,6 +100,14 @@ describe('read', () => { expect(result.isError).toBe(true) expect(result.error).toMatchObject({ code: 'FS_NOT_TEXT' }) }) + + it('paginates a multi-line file with offset/limit', async () => { + await writeFile(join(dir, 'a.txt'), 'one\ntwo\nthree\nfour') + const result = await call('read', { file_path: 'a.txt', offset: 2, limit: 2 }) + expect(text(result)).toContain('2: two') + expect(text(result)).toContain('3: three') + expect(text(result)).toContain('(Showing lines 2-3 of 4. Use offset=4 to continue.)') + }) }) describe('edit → disk', () => { @@ -108,13 +127,27 @@ describe('edit → disk', () => { expect(await readFile(join(dir, 'a.txt'), 'utf8')).toBe('hello world') }) - it('rejects an edit after only a partial read, leaving the file untouched', async () => { - await writeFile(join(dir, 'a.txt'), 'hello\nworld') + it('lets a WINDOWED read authorize an edit when the file is unchanged (freshness, not full-view)', async () => { + // A file with more lines than the read window; read only the first line. + const lines = Array.from({ length: 20 }, (_, i) => `line ${i + 1}`) + await writeFile(join(dir, 'a.txt'), lines.join('\n')) + const read = await call('read', { file_path: 'a.txt', offset: 1, limit: 1 }) + expect(read.isError).toBe(false) + expect(text(read)).toContain('(Showing lines 1-1 of 20') + + // Editing a line OUTSIDE the window is authorized because the file is unchanged. + const result = await call('edit', { file_path: 'a.txt', old_string: 'line 12', new_string: 'LINE 12' }) + expect(result.isError).toBe(false) + expect(await readFile(join(dir, 'a.txt'), 'utf8')).toBe(lines.map(l => l === 'line 12' ? 'LINE 12' : l).join('\n')) + }) + + it('rejects an edit when the file changed since the windowed read (stale before matching)', async () => { + await writeFile(join(dir, 'a.txt'), 'hello world') await call('read', { file_path: 'a.txt', offset: 1, limit: 1 }) + await writeFile(join(dir, 'a.txt'), 'goodbye') // out-of-band change removes 'world' const result = await call('edit', { file_path: 'a.txt', old_string: 'world', new_string: 'there' }) expect(result.isError).toBe(true) - expect(result.error).toMatchObject({ code: 'FS_PARTIAL_OBSERVATION' }) - expect(await readFile(join(dir, 'a.txt'), 'utf8')).toBe('hello\nworld') + expect(result.error).toMatchObject({ code: 'FS_STALE_VERSION' }) }) it('rejects an ambiguous match without replace_all', async () => { @@ -141,3 +174,15 @@ describe('edit → disk', () => { expect(await readFile(join(dir, 'a.txt'), 'utf8')).toBe('one three') }) }) + +describe('no-bypass / escape-hatch contract', () => { + it('a direct ctx.fs.readText records no observed-state, so a later edit rejects', async () => { + await writeFile(join(dir, 'a.txt'), 'hello world') + // Reach AROUND the policy layer — an explicit escape hatch for non-tool consumers. + await ctx.fs.readText(await ctx.fs.resolve('a.txt')) + // The model-facing edit still rejects: the read was not through ctx.fileContext. + const result = await call('edit', { file_path: 'a.txt', old_string: 'world', new_string: 'there' }) + expect(result.isError).toBe(true) + expect(result.error).toMatchObject({ code: 'FS_NOT_OBSERVED' }) + }) +}) diff --git a/packages/fs/tool-fs/tests/subpaths.spec.ts b/packages/fs/tool-fs/tests/subpaths.spec.ts index ac35babe04..7243955969 100644 --- a/packages/fs/tool-fs/tests/subpaths.spec.ts +++ b/packages/fs/tool-fs/tests/subpaths.spec.ts @@ -1,36 +1,43 @@ /** * Tests for the per-tool subpath plugins (`@deepseek-ai/dsh-tool-fs/read`, * `/write`, `/edit`): each registers exactly one tool, injects the same - * services, and cleans up on disposal. + * services (`tools`, `fileContext`, `systemPrompt`), and cleans up on disposal. */ import { describe, expect, it } from 'vitest' import { Context } from 'cordis' import SystemPrompt from '@deepseek-ai/dsh-system-prompt' import ToolRegistry from '@deepseek-ai/dsh-tools' -import { FileSystem } from '@deepseek-ai/dsh-fs' +import { FileSystem, FsTargetKey, FsVersion } from '@deepseek-ai/dsh-fs' import type { FsEditOutcome, - FsReadOutcome, + FsInfo, FsTarget, FsWriteOutcome, } from '@deepseek-ai/dsh-fs' +import FileContext from '@deepseek-ai/dsh-file-context' import * as readPlugin from '@deepseek-ai/dsh-tool-fs/read' import * as writePlugin from '@deepseek-ai/dsh-tool-fs/write' import * as editPlugin from '@deepseek-ai/dsh-tool-fs/edit' class StubFs extends FileSystem { override async resolve(path: string): Promise { - return { inputPath: path, targetKey: path, displayPath: path } + return { inputPath: path, targetKey: FsTargetKey(path), displayPath: path } } - override async readPage(): Promise { - return { offset: 1, limit: 1, lines: [], totalLines: 0, version: 'v', view: 'full' } + override async stat(): Promise { + return { version: FsVersion('v'), type: 'file', size: 0 } } - override async createOrReplace(): Promise { - return { operation: 'create', version: 'v' } + override async readText(): Promise { + return '' } - override async applyEdit(): Promise { - return { replacements: 1, replaceAll: false, version: 'v' } + override async streamText(): Promise> { + return (async function* () { yield '' })() + } + override async writeText(): Promise { + return { operation: 'create', version: FsVersion('v') } + } + override async editText(): Promise { + return { replacements: 1, replaceAll: false, version: FsVersion('v') } } } @@ -39,6 +46,7 @@ async function base() { await ctx.plugin(SystemPrompt) await ctx.plugin(ToolRegistry) await ctx.plugin(StubFs) + await ctx.plugin(FileContext) return ctx } @@ -64,7 +72,7 @@ describe('subpath plugins', () => { expect(ctx.tools.schemas()).toHaveLength(0) }) - it('stays pending without a ctx.fs provider', async () => { + it('stays pending without a ctx.fileContext provider', async () => { const ctx = new Context() await ctx.plugin(SystemPrompt) await ctx.plugin(ToolRegistry) diff --git a/packages/fs/tool-fs/tests/tools.spec.ts b/packages/fs/tool-fs/tests/tools.spec.ts index 594a07dbbd..ef1490b5ce 100644 --- a/packages/fs/tool-fs/tests/tools.spec.ts +++ b/packages/fs/tool-fs/tests/tools.spec.ts @@ -1,8 +1,10 @@ /** - * Consumer-surface tests for the filesystem tools using a fake `ctx.fs` that - * records the execution context it received and returns canned outcomes. These - * verify schemas, argument validation, result formatting, FsError→isError - * propagation, and that each tool passes `exec` straight through to `ctx.fs`. + * Consumer-surface tests for the filesystem tools. They run the REAL + * `ctx.fileContext` policy service over a fake `ctx.fs` provider (the genuine + * collaborator, per the prefer-the-real-implementation rule), so they verify + * schemas, argument validation, result formatting, FsError→isError propagation, + * and that each tool records observed-state through `ctx.fileContext` (the + * no-bypass contract) — not just that it moved bytes. */ import { describe, expect, it } from 'vitest' @@ -10,65 +12,56 @@ import { Context } from 'cordis' import { CallId } from '@deepseek-ai/dsh-llm' import SystemPrompt, { renderPrompt } from '@deepseek-ai/dsh-system-prompt' import ToolRegistry from '@deepseek-ai/dsh-tools' -import { FileSystem, FsError } from '@deepseek-ai/dsh-fs' +import { FileSystem, FsError, FsTargetKey, FsVersion } from '@deepseek-ai/dsh-fs' import type { FsEditOutcome, FsEditRequest, - FsExecContext, - FsReadOutcome, - FsReadRequest, + FsInfo, FsTarget, + FsWriteExpectation, FsWriteOutcome, } from '@deepseek-ai/dsh-fs' +import FileContext from '@deepseek-ai/dsh-file-context' +import type { FileReadOutcome } from '@deepseek-ai/dsh-file-context' import * as ToolFs from '@deepseek-ai/dsh-tool-fs' import { formatReadOutput } from '@deepseek-ai/dsh-tool-fs' -/** - * Records the public-API calls (and the exec each received) and returns canned - * outcomes; lets a test arm a rejection. Overrides the public methods directly - * (not the primitives) so we observe exactly what the tool passed. - */ +/** An in-memory fake provider; a test can arm a rejection on any primitive. */ class FakeFs extends FileSystem { - calls: Array<{ op: string; exec: FsExecContext | undefined; target: FsTarget }> = [] + files = new Map() rejectWith?: FsError + private throwIfArmed(): void { + if (this.rejectWith) throw this.rejectWith + } + override async resolve(path: string): Promise { - return { inputPath: path, targetKey: `key:${path}`, displayPath: `/abs/${path}` } + return { inputPath: path, targetKey: FsTargetKey(`key:${path}`), displayPath: `/abs/${path}` } } - - override async readPage(): Promise { - throw new Error('not used: tool tests override read()') + override async stat(target: FsTarget): Promise { + this.throwIfArmed() + const content = this.files.get(target.targetKey) + if (content === undefined) return undefined + return { version: FsVersion('v1'), type: 'file', size: content.length } } - override async createOrReplace(): Promise { - throw new Error('not used') + override async readText(target: FsTarget): Promise { + return this.files.get(target.targetKey) ?? '' } - override async applyEdit(): Promise { - throw new Error('not used') + override async streamText(target: FsTarget): Promise> { + const content = this.files.get(target.targetKey) ?? '' + return (async function* () { yield content })() } - - override async read(target: FsTarget, _request: FsReadRequest, exec?: FsExecContext): Promise { - this.calls.push({ op: 'read', exec, target }) - if (this.rejectWith) throw this.rejectWith - return { - offset: 1, - limit: 2000, - lines: [{ number: 1, text: 'hello' }, { number: 2, text: 'world' }], - totalLines: 2, - version: 'v1', - view: 'full', - } + override async writeText(target: FsTarget, content: string, _expected: FsWriteExpectation): Promise { + this.throwIfArmed() + const existed = this.files.has(target.targetKey) + this.files.set(target.targetKey, content) + return { operation: existed ? 'update' : 'create', version: FsVersion('v2') } } - - override async write(target: FsTarget, _content: string, exec?: FsExecContext): Promise { - this.calls.push({ op: 'write', exec, target }) - if (this.rejectWith) throw this.rejectWith - return { operation: 'create', version: 'v1' } - } - - override async edit(target: FsTarget, _edit: FsEditRequest, exec?: FsExecContext): Promise { - this.calls.push({ op: 'edit', exec, target }) - if (this.rejectWith) throw this.rejectWith - return { replacements: 1, replaceAll: false, version: 'v1' } + override async editText(target: FsTarget, edit: FsEditRequest): Promise { + this.throwIfArmed() + const content = this.files.get(target.targetKey) ?? '' + this.files.set(target.targetKey, content.split(edit.oldString).join(edit.newString)) + return { replacements: 1, replaceAll: edit.replaceAll, version: FsVersion('v3') } } } @@ -77,6 +70,7 @@ async function setup() { await ctx.plugin(SystemPrompt) await ctx.plugin(ToolRegistry) await ctx.plugin(FakeFs) + await ctx.plugin(FileContext) await ctx.plugin(ToolFs) const fs = ctx.fs as FakeFs return { ctx, fs } @@ -110,11 +104,11 @@ describe('registration', () => { expect(prompt).toContain('Use the edit tool') }) - it('stays pending until ctx.fs exists (inject)', async () => { + it('stays pending until ctx.fileContext exists (inject)', async () => { const ctx = new Context() await ctx.plugin(SystemPrompt) await ctx.plugin(ToolRegistry) - await ctx.plugin(ToolFs) // no fs provider + await ctx.plugin(ToolFs) // no fileContext provider expect(ctx.tools.schemas()).toHaveLength(0) }) @@ -123,6 +117,7 @@ describe('registration', () => { await ctx.plugin(SystemPrompt) await ctx.plugin(ToolRegistry) await ctx.plugin(FakeFs) + await ctx.plugin(FileContext) const fiber = await ctx.plugin(ToolFs) expect(ctx.tools.schemas()).toHaveLength(3) await fiber.dispose() @@ -132,7 +127,8 @@ describe('registration', () => { describe('read tool', () => { it('formats line-numbered content with a footer', async () => { - const { ctx } = await setup() + const { ctx, fs } = await setup() + fs.files.set('key:a.txt', 'hello\nworld') const result = await call(ctx, 'read', { file_path: 'a.txt' }) expect(result.isError).toBe(false) expect(text(result)).toBe(`/abs/a.txt @@ -166,18 +162,25 @@ describe('read tool', () => { expect(text(result)).toContain('file_path must be a non-empty string') }) - it('passes the execution context through to ctx.fs', async () => { + it('records observed state so a follow-up edit by the same session is authorized', async () => { const { ctx, fs } = await setup() const session = {} - await call(ctx, 'read', { file_path: 'a.txt' }, { session }) - expect(fs.calls).toHaveLength(1) - expect(fs.calls[0]?.op).toBe('read') - expect(fs.calls[0]?.exec?.agent?.session).toBe(session) + fs.files.set('key:a.txt', 'hello') + expect((await call(ctx, 'read', { file_path: 'a.txt' }, { session })).isError).toBe(false) + const edited = await call(ctx, 'edit', { file_path: 'a.txt', old_string: 'hello', new_string: 'bye' }, { session }) + expect(edited.isError).toBe(false) + }) + + it('propagates FS_NOT_FOUND for an absent file', async () => { + const { ctx } = await setup() + const result = await call(ctx, 'read', { file_path: 'missing.txt' }) + expect(result.isError).toBe(true) + expect(result.error).toMatchObject({ code: 'FS_NOT_FOUND' }) }) }) describe('formatReadOutput footer variants', () => { - const base = { offset: 1, limit: 2000, lines: [{ number: 1, text: 'x' }], totalLines: 1, version: 'v', view: 'full' as const } + const base: FileReadOutcome = { offset: 1, limit: 2000, lines: [{ number: 1, text: 'x' }], totalLines: 1, version: FsVersion('v') } it('reports a byte-capped read', () => { const out = formatReadOutput('/f', { ...base, totalLines: 99, truncatedByBytes: true }) @@ -225,9 +228,12 @@ describe('write tool', () => { }) describe('edit tool', () => { - it('formats a single-replacement success', async () => { - const { ctx } = await setup() - const result = await call(ctx, 'edit', { file_path: 'a.txt', old_string: 'a', new_string: 'b' }) + it('formats a single-replacement success after a read', async () => { + const { ctx, fs } = await setup() + const session = {} + fs.files.set('key:a.txt', 'a') + await call(ctx, 'read', { file_path: 'a.txt' }, { session }) + const result = await call(ctx, 'edit', { file_path: 'a.txt', old_string: 'a', new_string: 'b' }, { session }) expect(text(result)).toBe('The file /abs/a.txt has been updated successfully.') }) @@ -252,19 +258,11 @@ describe('edit tool', () => { expect(text(result)).toContain('file_path must be a non-empty string') }) - it('propagates FS_NOT_OBSERVED from the backend', async () => { + it('propagates FS_NOT_OBSERVED when the file was never read', async () => { const { ctx, fs } = await setup() - fs.rejectWith = new FsError('read first', 'FS_NOT_OBSERVED') - const result = await call(ctx, 'edit', { file_path: 'a.txt', old_string: 'a', new_string: 'b' }) + fs.files.set('key:a.txt', 'hello') + const result = await call(ctx, 'edit', { file_path: 'a.txt', old_string: 'a', new_string: 'b' }, { session: {} }) expect(result.isError).toBe(true) expect(result.error).toMatchObject({ code: 'FS_NOT_OBSERVED' }) }) - - it('propagates FS_PARTIAL_OBSERVATION from the backend', async () => { - const { ctx, fs } = await setup() - fs.rejectWith = new FsError('read fully first', 'FS_PARTIAL_OBSERVATION') - const result = await call(ctx, 'edit', { file_path: 'a.txt', old_string: 'a', new_string: 'b' }) - expect(result.isError).toBe(true) - expect(result.error).toMatchObject({ code: 'FS_PARTIAL_OBSERVATION' }) - }) }) diff --git a/packages/fs/tool-fs/tsconfig.json b/packages/fs/tool-fs/tsconfig.json index ee5a853c91..b8bd0b2148 100644 --- a/packages/fs/tool-fs/tsconfig.json +++ b/packages/fs/tool-fs/tsconfig.json @@ -11,6 +11,7 @@ { "path": "../../llm/llm" }, { "path": "../../core/tools" }, { "path": "../../core/system-prompt" }, - { "path": "../fs" } + { "path": "../fs" }, + { "path": "../file-context" } ] } diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index cae3a42202..1095777517 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -236,8 +236,23 @@ importers: specifier: ^4.0.0-rc.6 version: 4.0.0-rc.6(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.4) + packages/fs/file-context: + devDependencies: + '@deepseek-ai/dsh-fs': + specifier: workspace:^ + version: link:../fs + '@deepseek-ai/dsh-llm': + specifier: workspace:^ + version: link:../../llm/llm + cordis: + specifier: ^4.0.0-rc.6 + version: 4.0.0-rc.6(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.4) + packages/fs/fs: devDependencies: + '@deepseek-ai/dsh-brand': + specifier: workspace:^ + version: link:../../util/brand '@deepseek-ai/dsh-llm': specifier: workspace:^ version: link:../../llm/llm @@ -266,6 +281,9 @@ importers: '@deepseek-ai/dsh-agent': specifier: workspace:^ version: link:../../core/agent + '@deepseek-ai/dsh-file-context': + specifier: workspace:^ + version: link:../file-context '@deepseek-ai/dsh-fs': specifier: workspace:^ version: link:../fs diff --git a/scripts/type-equiv.manifest.json b/scripts/type-equiv.manifest.json index d9df0302ab..1c5ec2430e 100644 --- a/scripts/type-equiv.manifest.json +++ b/scripts/type-equiv.manifest.json @@ -37,19 +37,17 @@ { "doc": "docs/core-data-structures/bash.md", "symbol": "BashTask", "source": "packages/bash/bash/src/types.ts" }, { "doc": "docs/core-data-structures/bash.md", "symbol": "BashTaskRead", "source": "packages/bash/bash/src/types.ts" }, - { "doc": "docs/core-data-structures/filesystem.md", "symbol": "FsExecContext", "source": "packages/fs/fs/src/types.ts" }, { "doc": "docs/core-data-structures/filesystem.md", "symbol": "FsTarget", "source": "packages/fs/fs/src/types.ts" }, + { "doc": "docs/core-data-structures/filesystem.md", "symbol": "FsTargetKey", "source": "packages/fs/fs/src/types.ts" }, { "doc": "docs/core-data-structures/filesystem.md", "symbol": "FsVersion", "source": "packages/fs/fs/src/types.ts" }, - { "doc": "docs/core-data-structures/filesystem.md", "symbol": "FsReadRequest", "source": "packages/fs/fs/src/types.ts" }, - { "doc": "docs/core-data-structures/filesystem.md", "symbol": "FsTextLine", "source": "packages/fs/fs/src/types.ts" }, - { "doc": "docs/core-data-structures/filesystem.md", "symbol": "FsView", "source": "packages/fs/fs/src/types.ts" }, - { "doc": "docs/core-data-structures/filesystem.md", "symbol": "FsReadOutcome", "source": "packages/fs/fs/src/types.ts" }, - { "doc": "docs/core-data-structures/filesystem.md", "symbol": "FsExpectation", "source": "packages/fs/fs/src/types.ts" }, + { "doc": "docs/core-data-structures/filesystem.md", "symbol": "FsInfo", "source": "packages/fs/fs/src/types.ts" }, + { "doc": "docs/core-data-structures/filesystem.md", "symbol": "FsWriteExpectation", "source": "packages/fs/fs/src/types.ts" }, { "doc": "docs/core-data-structures/filesystem.md", "symbol": "FsWriteOutcome", "source": "packages/fs/fs/src/types.ts" }, { "doc": "docs/core-data-structures/filesystem.md", "symbol": "FsEditRequest", "source": "packages/fs/fs/src/types.ts" }, { "doc": "docs/core-data-structures/filesystem.md", "symbol": "FsEditOutcome", "source": "packages/fs/fs/src/types.ts" }, - { "doc": "docs/core-data-structures/filesystem.md", "symbol": "FsStateSource", "source": "packages/fs/fs/src/types.ts" }, - { "doc": "docs/core-data-structures/filesystem.md", "symbol": "FileState", "source": "packages/fs/fs/src/types.ts" }, - { "doc": "docs/core-data-structures/filesystem.md", "symbol": "FsErrorCode", "source": "packages/fs/fs/src/types.ts" } + { "doc": "docs/core-data-structures/filesystem.md", "symbol": "FsErrorCode", "source": "packages/fs/fs/src/types.ts" }, + { "doc": "docs/core-data-structures/filesystem.md", "symbol": "FileContextExec", "source": "packages/fs/file-context/src/types.ts" }, + { "doc": "docs/core-data-structures/filesystem.md", "symbol": "FileReadRequest", "source": "packages/fs/file-context/src/types.ts" }, + { "doc": "docs/core-data-structures/filesystem.md", "symbol": "FileReadOutcome", "source": "packages/fs/file-context/src/types.ts" } ] } diff --git a/tsconfig.build.json b/tsconfig.build.json index ea3882b873..d32b663b9a 100644 --- a/tsconfig.build.json +++ b/tsconfig.build.json @@ -28,6 +28,7 @@ { "path": "./packages/bash/tool-bash" }, { "path": "./packages/fs/fs" }, { "path": "./packages/fs/fs-local" }, + { "path": "./packages/fs/file-context" }, { "path": "./packages/fs/tool-fs" }, { "path": "./packages/support/invariants" }, { "path": "./packages/ui/acp" },