From 6825918eaddca6bb7113f5a889a95ffc9f386470 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Thu, 16 Jul 2026 11:15:58 +0800 Subject: [PATCH] docs: finish session simplification prose cleanup --- .../2026-07-05-reconstructable-requests.md | 20 +++++----- .../2026-06-18-compaction-capability-seam.md | 14 +++---- .../implemented/feature/2026-07-06-sandbox.md | 14 +++---- packages/compact/compact-basic/src/index.ts | 18 ++------- .../compact-basic/tests/compact-basic.spec.ts | 26 +----------- .../tests/request-reconstruction.spec.ts | 11 ++--- packages/core/session/README.md | 40 +++++++++---------- packages/core/session/tests/surface.spec.ts | 4 -- packages/ui/user-approval/README.md | 12 +++--- 9 files changed, 56 insertions(+), 103 deletions(-) diff --git a/docs/rfc/implemented/architecture/2026-07-05-reconstructable-requests.md b/docs/rfc/implemented/architecture/2026-07-05-reconstructable-requests.md index a5bc946af4..e93b4e0bd9 100644 --- a/docs/rfc/implemented/architecture/2026-07-05-reconstructable-requests.md +++ b/docs/rfc/implemented/architecture/2026-07-05-reconstructable-requests.md @@ -20,35 +20,35 @@ Prefix-cache stability is corollary #1, not the headline: an append-only log pro **Messages.** `Session.deriveMessages()` is cached: each surface entry is projected exactly once, when first seen, through the public per-event function `deriveEventMessage(event)`; a surface rewrite (a compaction `replace` — `SurfaceManager.replaceGeneration`) rebuilds. Callers get a fresh array per call over shared, deep-frozen messages: mutating logged history through a projection is unrepresentable (it throws), replacing the old clone-per-call isolation. External reconstructors fold the same public function over a log prefix, so no two paths can disagree. -**The header.** The request's non-history half — `EpochHeader`: call config (`LlmCallConfig`: model + sampling scalars), rendered system prompt, assembled tool schemas, and the session prefix (`messagePrefix`, below) — is logged session state in canonical form (empty system/tools/prefix ≡ absent). One log-only, turn-enclosed event carries it: `request/header`, always a full snapshot. Each loop instance appends one on its first request (`'initial'` when the log has none, `'resume'` otherwise, even when nothing changed: the boundary itself is a recorded fact and cross-restart drift becomes attributable); a later request whose canonical header differs appends another with reason `'change'`. `foldRequestHeader` reconstructs by selecting the latest snapshot, and the live session tracks that fold with the same lazy cursor as the message cache. Legacy v0 logs containing the removed delta representation are rejected at seed and persistence-load boundaries rather than partially replayed. +`EpochHeader` records the request's non-history state: call config, rendered system prompt, tool schemas, and session prefix, with empty values canonicalized to absence. `request/header` always writes a full snapshot: the first loop instance uses reason `initial`, later instances use `resume`, and an in-instance change uses `change`. `foldRequestHeader` selects the latest snapshot. Legacy `request/header-delta` events and the removed `fallback` reason are rejected when appended or loaded. -**The loop, transmission-stateless.** Per step: render assembly (every step — value comparison needs no change-signal discipline, and a section that varies per step surfaces as a *logged* header event per step instead of a silent bust) → on the instance's FIRST step only, the `agent/session-prefix` waterfall — request-ONLY messages fronting the entire derived history (a frozen empty seed, contributions returned as an extension of `next()`; the home for session-stable openers that must NOT become history — a skills catalog, an AGENTS.md digest), deep-frozen and cached on the instance so reuse is structural and the prefix cannot drift mid-session — → `agent/pre-step`, carrying the composed prefix (compaction's surface mutations land before derivation, and its pressure gate counts the prefix this instance will actually send — never a previous instance's logged one, which could under-gate a resumed/forked instance whose contributor grew) → **messages snapshot, then `step/start` appended as the next operation in the same synchronous frame** → seed the call config (first request of the instance: from `AgentOptions`, so explicit options always beat the logged baseline — fork model-overrides and resume reconfiguration stay correct; afterwards: from the folded header) → the `agent/request` waterfall, re-typed `(agent, turn, step, config: LlmCallConfig, next) → LlmCallConfig` — a frozen seed and a returned replacement are ALL a listener shapes; durable content flows through the log channels (`inject()`, steering, prompt-submit `additionalContext`, sections via `system-prompt/assemble`) — → the header event the request owes the log, carrying the prefix as `messagePrefix` (no session event carries it, so the header is its only durable record; resume = a new instance = a recompose, anchored by its `'resume'` snapshot) → build `GenerateOptions` from `messagePrefix + snapshot` + header, deep-freeze (`deepFreeze` exempts the `AbortSignal`, the one live control channel — freezing one breaks `AbortController.abort()`), dispatch. The loop's per-instance bookkeeping is one boolean plus the cached prefix: whether this instance has logged its anchoring snapshot, and what it composed. +Each step rebuilds prompt assembly. On the instance's first step, `agent/session-prefix` extends a frozen empty seed with request-only opener messages; the result is frozen and cached for that loop instance. `agent/pre-step` then receives the composed prefix before messages are snapshotted immediately ahead of `step/start`. The first call config starts from explicit `AgentOptions`, preserving fork overrides and resume reconfiguration; later calls start from the folded header. `agent/request` may replace only that frozen config seed, while model-visible content enters through logged channels. The loop records the owed header event—the prefix's only durable home—builds `GenerateOptions` from prefix, snapshot, and header, and deep-freezes it while leaving `AbortSignal` live. Per-instance state is only the cached prefix and whether its anchoring snapshot has been written. -**The reconstruction boundary is `step/start`, unconditionally.** A step's messages are the derivation over `events[0..stepStartSeq)`. Because the snapshot precedes the `step/start` append in the same synchronous frame, an `agent.inject()` from an `agent/request` listener or any concurrent task lands after the boundary and joins the NEXT request. `session/event` is observe-only during publication: a reentrant append is rejected until the current callback list drains, preventing nested event delivery from overtaking the event being observed. `agent/pre-step` is the sanctioned seam for content that must affect the CURRENT request. A step's header for reconstruction is the latest `request/header` at or after its `step/start` (before the first response event), or the fold carried forward when the request header is unchanged. +**`step/start` is the reconstruction boundary.** A step derives messages from events before that sequence. Injection after the snapshot joins the next request, and reentrant appends are rejected during event publication. `agent/pre-step` is the seam for content needed by the current request. Header reconstruction selects the step's `request/header`, or carries the prior snapshot when no new header is written. -**Enforcement.** Dev-mode ([dsh-invariants](../../../../packages/support/invariants/src/index.ts)), on `llm/stream`: a frozen request with a live `sessionId` — the loop-built marker; hand-built one-shots are unfrozen and skipped — must carry messages deep-equal to the folded header's `messagePrefix` followed by the boundary derivation — the derivation rebuilt through a FRESH `Session` over `events[0..stepStartSeq)` so the live cache cannot vouch for itself — and header fields equal to `foldRequestHeader` over the log. There is no divergence allowance and nothing to allow: no seam can put unlogged content into a request — the `agent/session-prefix` seam's product enters only because the header event records it first. `prepend: true` only defends against the replay adapter's short-circuit (an append-registered listener); two prepended listeners have no defined mutual order in cordis, so correctness rests on the seq-bounded fold, never on listener timing. Measurement stays lean: the with-key e2e ([request-cache.e2e.ts](../../../../packages/core/agent-loop/tests/request-cache.e2e.ts)) proves `usage.cacheReadTokens > 0` on every request after the first against the live API, and per-step usage in the log is the production observable — a header event or compaction shows up as a cache-read collapse on the next step. +**Enforcement.** In development, `dsh-invariants` independently rebuilds each loop request through a fresh `Session`, so the live cache cannot vouch for itself, then compares messages and folded header fields at `llm/stream`. Loop requests are identified by their frozen shape and session id; direct one-shots are excluded. Correctness depends on sequence-bounded reconstruction rather than listener order. A with-key e2e requires positive cache-read tokens after the first request; per-step usage is the production signal, and a header change or compaction appears as a cache-read drop on the next step. ### The MiniCode shape: adopted, with the provenance arrow inverted -What survives from `LLMClient`: the conversation is maintained, not rebuilt — one projection per message, ever; requests advance append-only; resets happen only for a system-prompt/tool change, a config change, or compaction, each now a *logged* fact. What is deliberately inverted: MiniCode's client is the source of truth and its event stream derives from client appends (`on_event(MessageAdded)`), which suits an advisory event stream. Here the log is contractual — persistence, crash recovery, fork seeding, transcript rendering, and the snapshot harness all replay it — and it carries strictly more than a message list (turn/step boundaries, raw chunk streams, tool-call pairing, provenance, log-only records), so a message-list client cannot generate it. The arrow therefore points log → client: the conversation state IS the log plus two cached folds inside `Session` (messages, header), and the "client" the loop talks to is the session itself. What the inversion buys over the original: the reconstruction is *checkable* against an independent record on every request — MiniCode's client has nothing to check itself against. +Like MiniCode, the conversation advances append-only and resets only when model-visible state changes. Unlike MiniCode, the event log remains the source of truth because it also owns persistence, recovery, boundaries, tool pairing, and provenance. `Session` caches message and header folds derived from that log, making every request independently checkable. ## Alternatives considered - **Client as source of truth** (literal MiniCode): a second operative truth beside the log — the two drift and nothing notices; see the section above. -- **A stateful transmission client mirroring the log** (a `PromptPrefix` class holding committed/open message zones with an append/editTail/reset vocabulary, the log pushed into it per event): behaviorally equivalent on the happy path, but it duplicates conversation state outside the session, needs transactional rollback around listener seams, keeps an unlogged content-shaping surface (`editTail`) whose divergence the invariant must specially allow, and still cannot answer "what header did the model see" from the log. Dissolving it into the session's own caches plus logged header events made every one of those problems unrepresentable instead of guarded. (PR #162 is the archaeology of this alternative, three designs deep.) +- **A stateful transmission client mirroring the log** — duplicates conversation state, needs rollback around listeners, leaves an unlogged edit surface, and still cannot reconstruct request headers. Session-owned caches plus logged headers avoid those split truths. - **Per-call request scalars** (a freely mutable config handed to each `agent/request` dispatch): a listener flips the model per call with zero accounting, silently abandoning the provider cache this design exists to protect. Config is per-conversation logged state; the waterfall proposes, the log records. - **Detect-and-report** (compare consecutive requests, warn on divergence): catches violations after the fact; a violating request is still constructible and ships. Rejected for interface-level unrepresentability. - **Event-driven assembly** (re-render only on change signals): a missed-signal bug class — a tool registered mid-session emits `tools/change`, not `system-prompt/change`, and a third-party provider may emit nothing. Per-step render + value compare is robust with zero signal discipline. -- **A custom header-delta codec** (system line edits, name-keyed tool edits, whole config/prefix replacements): it reduced repeated header bytes but duplicated state across codec types, diff/apply machinery, and fallback handling. Full changed snapshots preserve reconstructability with one representation; compression remains available if measured logs justify it. -- **Narrative changed-field lists on header snapshots**: derivable by diffing consecutive snapshots — one home per fact. Snapshots keep a reason because an instance boundary versus an in-instance change is not derivable from data alone. +- **A custom header-delta codec** (system line edits, name-keyed tool edits, whole config/prefix replacements): reduced repeated bytes but duplicated the representation and its diff/apply/fallback machinery. Full snapshots retain one replay representation. +- **Narrative changed-field lists on header snapshots**: derivable by comparing consecutive snapshots. The `reason` remains because an instance boundary is not derivable from the snapshot values. ## Consequences - A request that is not explained by the log cannot be constructed by accident — not by the loop, not by a listener; mutating a built request throws; every header change is a durable, diffable log event. - Choosing between the advisory channels is a change-frequency decision, and the design makes the stable one structural: an `agent/session-prefix` contribution is composed once per loop instance and reused verbatim, so it extends the cacheable prefix at zero marginal cost and CANNOT bust the provider cache mid-session; content that changes mid-session flows through the append-only history channels — `agent.inject()`, a `tools/post-execute` decision's `additionalContext`, prompt-submit `additionalContext` — each a durable `context/message` paid once and prefix-cached thereafter, at the price of accumulating in history and the log. Route session-frozen openers to the prefix and change notices to the history channels; a per-step request-only tail slot was deliberately dropped (no consumer, and a durable append covers every current update pattern). -- What still costs full price at the provider is inherent and logged: compaction (its `compact/*` events and surface replacement), a real prompt/tool/config change (`request/header` with reason `'change'`), or a process boundary with drift (`'resume'` snapshot differing from its predecessor). The provider's own reasoning-content exclusion is managed server-side. +- What still costs full price at the provider is inherent and logged: compaction (its `compact/*` events and replacement entry), a real prompt/tool/config change (`request/header` with reason `change`), or a process boundary with drift (a differing `resume` snapshot). The provider's own reasoning-content exclusion is managed server-side. - The `step/start`-listener behavior change (above) is the one observable semantics change for plugins; `agent/pre-step` is the current-request seam. - Tool-result trimming (planned) needs no new mechanism: a logged single-entry surface replace (`start === end`) carrying a trimmed `tool/result` under the same `callId` — compaction-family, replay-correct, cache-bust batched by the same pressure logic. -- Session logs grow one `request/header` snapshot per loop instance plus full snapshots on real changes. This spends more bytes than a custom delta codec but stays small beside chunk-heavy logs and leaves one replay representation. `SESSION_FORMAT_VERSION` stays `0`; a legacy v0 delta event is rejected rather than migrated. +- Session logs grow one `request/header` snapshot per loop instance plus snapshots on real changes. This is larger than a delta codec but small beside chunk-heavy logs and retains one replay representation. `SESSION_FORMAT_VERSION` stays `0`; legacy delta events are rejected rather than migrated. - Snapshot goldens changed once (every transcript gains its header events); the fs-writing fixtures are stored in the normalized authored form with cwd-relative tool arguments, because replay only round-trips cwd-independent argument paths. - FIXME(call-config-shape): revisit `LlmCallConfig`'s exact field set — which fields are genuinely epoch-level for cache purposes (`model` certainly; the sampling scalars sit there out of caution), and where provider-specific extras (reasoning options, extra body params) belong when an adapter needs them. diff --git a/docs/rfc/implemented/feature/2026-06-18-compaction-capability-seam.md b/docs/rfc/implemented/feature/2026-06-18-compaction-capability-seam.md index a9efbee2b4..955a39fcf6 100644 --- a/docs/rfc/implemented/feature/2026-06-18-compaction-capability-seam.md +++ b/docs/rfc/implemented/feature/2026-06-18-compaction-capability-seam.md @@ -46,19 +46,17 @@ messages = session.deriveMessages() ⟵ single derive, reflects the compaction request = waterfall agent/request ⟵ pure request transform (hooks, model switch) ``` -This makes the layering correct *by construction*: compaction mutates the surface, the loop derives **once** from the result (no double-derive), and at `pre-step` the assembled `messages` do not yet exist — so a listener structurally *cannot* see or be expected to act on downstream-injected context. `agent/request` reverts to a pure request transformer. Firing the seam **before** `step/start` (not inside the open step) is load-bearing for crash-safety: compaction's log-only `compact/*` records and its replacement node land *outside* any step, so the honest log structure a crash leaves (a dangling `compact/start` sitting before the synthetic `turn/end` that turn-repair appends) holds without a half-open step to reconcile. The seam is `serial` (awaited, in registration order), not `parallel`: a listener mutates the surface as a side effect — there is nothing to transform or return — and serial isolates listeners from each other so two surface-mutating listeners can never interleave their `session.append`s. Cordis `serial` does bail early if a listener returns a bail value, so `agent/pre-step` listeners are typed/documented to return `void` and must not use that bail channel as a semantic veto surface. - -This **amends** the original RFC's claim of "NO changes to `dsh-agent-loop`; compaction is a pure plugin." That claim was load-bearing for a wrong design — reusing `agent/request` was the mistake. Per the pre-release "foundation over blast radius" stance, adding the correct seam (one event declaration in `dsh-agent`, one awaited emit in the loop) beats preserving a no-change boast that locked in the double-derive. +The loop derives messages once after `agent/pre-step`. Running before `step/start` keeps compaction records outside any half-open step, simplifying crash repair. The seam is awaited and serial so surface mutations cannot interleave; listeners return `void` and do not use Cordis bail values as vetoes. ### Retention is turn-agnostic; tool-pairing balance is the only structural guard Auto-compaction fires before **every** step, not once per turn. This is **load-bearing for runaway-turn survival**: a tool-heavy ReAct turn appends an `assistant/message` + a `tool/result` per step, so the surface grows *within* a turn. A single turn can grow past the window on its own (a "runaway turn") — and the only moment to rescue it before the next model call overflows is the next step's `pre-step` checkpoint. Gating compaction to a turn's first step (or, worse, retaining the whole in-flight turn verbatim) re-opens exactly the hole compaction exists to close: the harness would die when compaction is most needed. -So retention does **not** protect the in-flight turn, and turn boundaries play no role in it. `compactIfNeeded` walks the surface entries tail→head, summing per-entry token estimates, and retains the smallest tail-run of **whole units** whose total reaches `retainTokens`; everything older is compacted (head-anchored — see below). A *unit* is either a whole closed step (its `assistant/message` plus its `tool/result`s) or a single no-step entry (a pre-step `user/message`, inter-step `steering/message`, or injection `context/message`). The walk rounds toward retaining *more*: when the raw token cutoff lands mid-step, it extends the retained side head-ward until the cut before the retained entry is **tool-pairing balanced**. The single structural guard is therefore **tool-pairing balance** — a region's edges are balanced cuts on the *surface* (no unanswered `tool-call` crosses either edge), so a compacted region never splits a step's tool-calls from their `tool/result`s (which would produce a transcript every provider rejects). The check is decided over surface order, **not** the log's `step/*` markers: a compaction lands a replacement at a high log seq whose surface position is the head, so a log-position scan misreads its neighbours — `dsh-session` exports `isToolPairingBalanced(nodes, events, beforeSeq)` for the surface-anchored check. `compactRegion` enforces it strictly, throwing on a boundary that would split a step. +`compactIfNeeded` retains the smallest tail of whole surface units whose estimated size reaches `retainTokens` and compacts older nodes. A unit is a complete closed step or one no-step message. If the token cutoff lands inside a step, retention expands until the cut is tool-pairing balanced. Balance is checked on surface order, not log sequence, because replacement summaries have new sequence numbers at old surface positions. `compactRegion` rejects boundaries that split a tool call from its result. The in-flight turn receives no special retention. A runaway turn thus compacts exactly like any other history: its early *closed* steps get summarized while its recent steps stay verbatim. When the only compactable content left is an un-splittable open tail step (its tool-calls have no results yet), compaction declines (`null`) and retries once that step closes. -**Single-unit overflow is out of scope, by design.** If a single retained unit — one closed step, or a large free node such as a pasted `user/message` — *alone* exceeds the budget, compaction cannot help and the next model call may go out over-budget. Bounding an individual unit's size is a separate concern (output truncation), handled elsewhere; compaction makes no promise about it, and the harness without such a mechanism can still break on a single oversized unit. This is named honestly rather than papered over. +**Single-unit overflow is out of scope, by design.** If a single retained unit — one closed step, or a large free entry such as a pasted `user/message` — *alone* exceeds the budget, compaction cannot help and the next model call may go out over-budget. Bounding an individual unit's size is a separate concern (output truncation), handled elsewhere; compaction makes no promise about it, and the harness without such a mechanism can still break on a single oversized unit. This is named honestly rather than papered over. ### Head-anchoring: one auto checkpoint, always at the head @@ -70,7 +68,7 @@ Auto-compaction always starts at the surface head, merging the prior checkpoint ### Surface replacement: `compact/*` events are log-only; one `user/message` carries the summary -Because `SurfaceEventType` is closed, the summary cannot ride on a `compact/*` event. The backend instead appends a **single `user/message`** with `surfaceOp: { op: 'replace', start, end }` whose `content` is the (framed) summary and whose `sourceEventSeqs` covers the shadowed nodes *and* the bookkeeping events. The `compact/*` events are pure log records (lock + provenance). The surface mutation sits **inside** the lock — `compact/end` is the last event appended: +Because `SurfaceEventType` is closed, the summary cannot ride on a `compact/*` event. The backend instead appends a **single `user/message`** with `surfaceOp: { op: 'replace', start, end }` whose `content` is the (framed) summary and whose `sourceEventSeqs` covers the shadowed entries *and* the bookkeeping events. The `compact/*` events are pure log records (lock + provenance). The surface mutation sits **inside** the lock — `compact/end` is the last event appended: ``` compact/start → log-only. Acquires the lock. @@ -81,7 +79,7 @@ user/message → surfaceOp { op:'replace', start, end }. THE surface mutatio compact/end → log-only. Releases the lock (carries `error` on a recoverable failure). ``` -`deriveMessages()` then yields `[summary_as_user_message, ...retained_nodes]`. Reusing `user/message` is honest rather than a workaround: a summary genuinely *is* user-role context. +`deriveMessages()` then yields `[summary_as_user_message, ...retained_entries]`. Reusing `user/message` is honest rather than a workaround: a summary genuinely *is* user-role context. ### Checkpoint framing + incremental merge (backend-private) @@ -116,7 +114,7 @@ Two failure paths, both documented: - **New loop seam**: `agent/pre-step` (`@mode serial`) declared in `dsh-agent` and emitted by `dsh-agent-loop` after system assembly and before `step/start`. This is a documented change to the loop — `docs/architecture.md` records it and the generated cordis catalog carries its signature. - **`SessionEventMap`** gains `compact/start` / `compact/summary` / `compact/end` by declaration merging (merge-extensible); `SurfaceEventType` is **not** touched. These are session events, not cordis `Events`, so the event-taxonomy gate needs no entry. - **`dsh-session`** gains the tool-pairing balance predicate (`isToolPairingBalanced`, in `tool-pairing.ts`, exported from the package index) that `compactRegion`/`compactIfNeeded` use to keep a collapsed region from splitting a step's tool-call/result pair. The surface `replace` op and the surface-metadata runtime guard already existed and are reused. -- **`dsh-invariants`** drops its `surface replace: start must be <= end` assertion: a head-anchored compaction lands a high-seq replacement node at an older range's *position*, so `start > end` numerically is normal and valid (the range is positional, validated by the surface's `indexOf` checks that remain). The turn-enclosure invariant is reused unchanged. +- **`dsh-invariants`** drops its `surface replace: start must be <= end` assertion: a head-anchored compaction lands a high-seq replacement entry at an older range's *position*, so `start > end` numerically is normal and valid (the range is positional, validated by the surface's `indexOf` checks that remain). The turn-enclosure invariant is reused unchanged. - **Wiring**: `dsh-compact-basic` is loaded in `examples/coding-agent`'s `cordis.yml`, so the seam ships in the real demo (it was previously loaded nowhere). ## Testing diff --git a/docs/rfc/implemented/feature/2026-07-06-sandbox.md b/docs/rfc/implemented/feature/2026-07-06-sandbox.md index 3e2180484e..e236154cdc 100644 --- a/docs/rfc/implemented/feature/2026-07-06-sandbox.md +++ b/docs/rfc/implemented/feature/2026-07-06-sandbox.md @@ -105,11 +105,11 @@ interface SessionEventMap { Each owner exports the same three-piece kit: the event declaration, a pure fold (`effectiveSandboxMode(events)` / `effectiveApprovalPolicy(events)` — a `findLast`, typed to the domain's closed union), and THE write path (`setSandboxMode(session, mode)` / `setApprovalPolicy(session, policy)` — a switch IS its event; nothing mutates state out of band). No shared owner service, no generic facts map, no registry: a third knob copies the ~40-line pattern into its own package. Execution follows the fold on both sides — the bash tool's per-call stamp reads it as the middle rung of the § Escalation precedence chain, and the approval seam's `'never'` gate is [the approval RFC](2026-07-06-approval-seam.md)'s side of the same pattern. -**Visibility is deliberately asymmetric between the knobs.** The SANDBOX mode is stated nowhere and its switches are not narrated: a standing "you are read-only" declaration teaches the model to refuse preemptively (observed live: sessions where the model would not even attempt a write it could have escalated), while the denial marker already names the mode the command ran under at exactly the moment the boundary matters — behavior, not belief, carries the state, and a switch simply changes what the next command does. The APPROVAL policy keeps both layers, because its failure mode is the opposite: an auto-rejected ask under `'never'` returns "the user rejected …" wording no behavior can disambiguate, so the prompt states `'never'` (and ONLY `'never'` — an `'ask'` promise is unknowable without asking, and absence under a logged header is how the narrator reads `'ask'` back), and an `agent/pre-step` narrator injects at most one coalesced notice per policy switch: idle flip-flops collapse to one notice at the next turn's first step, a net-zero round trip to none, and a mid-turn change is narrated no later than the next step. Its "last told" is in-memory with a log-derived fallback (the folded header's system text parsed against the closed candidate sentence; LAST occurrence wins, so a persona quoting it cannot shadow the real section), so restarts lose nothing; attribution is positional (a knob event after the log's last `request/header` reads `changed by the user`, a drift with no such event reads `changed by the operator/config`). +Sandbox mode is not narrated in the prompt; denial results report the mode when it matters, avoiding preemptive refusal based on a standing label. Approval policy is different: only `'never'` is stated because automatic rejection otherwise looks like a user decision. Policy-change notices are coalesced and delivered by the next pre-step, with log-derived fallback after restart. The notice source is inferred from event position: a knob event after the last request header is user-driven; unlogged drift is operator or config driven. **The editor surface** is protocol-native [Session Config Options](https://agentclientprotocol.com/protocol/session-config-options) — the spec's replacement for session modes (slated for removal in ACP v2), already SDK-typed. When `ctx.permission` is composed, the bridge advertises one `permission` select (category `mode`) in `session/new` and `session/load`; its options are the deployment's preset table, and its `currentValue` is `PermissionService.current()` over the session log plus composition defaults. The shipped `workspace-write` and `danger-full-access` presets each bundle a sandbox mode with an approval policy and write through to both domain setters; a knob combination outside the table is reported as switch-away-only `custom`. `session/set_config_option` validates and switches through the permission service, then returns the complete refreshed state (the spec contract). -**Anchoring: turn-enclosure is the commit boundary.** The turn-enclosure contract makes a bare between-turns append invalid (the JSONL backend treats a post-`turn/end` tail as crash garbage; dev invariants throw). A switch while a turn is open appends immediately — openness read from the LOG (last boundary event is `turn/start`), not `agent.status`, which stays `running` between queued turns. An idle switch is held on the bridge's session record and anchored at the next turn's `agent/prompt-submit` — inside the turn, before anything in it assembles or executes, last write per knob, and OUTSIDE any `session/event` emit (appending from inside that feed reorders events for later-registered listeners — a bug the dev invariants caught live). Until anchored, the switch exists only in bridge memory: responses overlay it truthfully, and a crash before the next turn reverts it — `session/load` then reports the fold's truth, so the editor UI self-corrects rather than lies. +**Turn enclosure is the commit boundary.** A switch during an open turn appends immediately. An idle switch remains pending on the bridge record and is appended at the next prompt submission, before assembly or execution; last write wins per knob. Openness comes from log boundaries rather than `agent.status`, and setters do not append from inside a `session/event` listener because that would reorder later observers. Until anchoring, responses overlay the pending value. A crash discards it, and reload returns the durable fold. #### In-process tools @@ -119,10 +119,10 @@ FIXME: Revisit this tool-local boundary. The follow-up design needs to determine ### Testing -- Unit tier (no real runner anywhere): profile dialects, per-platform chain selection (sole candidate unprobed, no chain fails closed, multi-candidate probe order), verdict caching, the fail-closed end, probe-report parsing, and the launcher/`sandbox-exec` CLI contracts via fake runner scripts in `dsh-sandbox-local`; wrapping, policy hand-off, fact stamping, and runner-failure-outranks-denial classification (foreground throw, background `runnerFailed` fact) against a fake provider in `dsh-bash-sandbox`; the error's structured identity in `dsh-sandbox`. The escalation matrix spans the three bash packages: verbatim carry-through in `dsh-bash-local`, stamp/branch/per-task-facts in `dsh-bash-sandbox`, and the capability gate, `justification` pairing, fail-closed texts (pinned verbatim), and grant stamping in `dsh-tool-bash`. The switching surface pins the folds, the stamping precedence, the `'never'` gate, per-session section rendering, the full narrator matrix (cold start, coalescing, net-zero, resume drift with operator wording, positional attribution, persona-shadow hardening), and the bridge's advertisement gating, validation rejections, idle-vs-mid-turn anchoring (dev invariants mounted), and `session/load` reporting over a real two-process JSONL round trip. -- Keyless real-runner e2e, split along the seam and per rung: CI's `sandbox-e2e` matrix runs bwrap and Landlock on Linux (the Landlock leg once per architecture, each confining through the registry-installed launcher) and Seatbelt on macOS against real kernels, failing on a silent all-skip. World-proofs live in `dsh-sandbox-local` (denied writes absent on disk, workspace writes landing, temp-area grants pinned, kernel denial text matching the advertised dialect) and `dsh-bash-sandbox` (the through-`ctx.bash` consumer proofs, including denied-then-overridden-write-lands). This package's own publish path is rehearsed without publishing (`packed-install.e2e.ts`): `pnpm pack`, tarballs installed into a throwaway consumer with the launcher family resolving from the registry, plain `node` confining through the INSTALLED launcher — asserted executable apart, so a mode-stripped binary can never masquerade as a non-enforcing kernel. The switching surface has its own keyless e2e (the acp-agent example's `escalation.e2e.ts`): the real default `cordis.yml` tree advertises the one permission option, honors switches end to end, and rejects out-of-vocabulary values. -- With-key e2e (`examples/acp-agent/tests/escalation.e2e.ts`): real model + real runner + the REAL bridge answerer, world-verified — denied under `read-only`, escalates with justification, the scripted editor grants and the retried write lands on disk, while a rejected escalation leaves no write. Self-skips without `DEEPSEEK_API_KEY` or a usable runner (e2e.yml installs bubblewrap so it actually executes in CI). -- Snapshot tier (`examples/acp-agent/tests/acp.snapshot.ts`): the keyless config-option wire; the recorded permission-switching arc as the pinned header of its class — necessarily, since mid-session switches emit the changed `request/header` snapshot the uniformity guard licenses only in the pin — committing one `workspace-write`→`danger-full-access` preset switch (the `permission/preset` event written through to both knobs), the changed approval prompt section and its "changed by the user" notice; and both recorded escalation branches over scripted `permissionAnswers` (grant runs under the granted `danger-full-access`; rejection executes nothing and pins the fail-closed text). Snapshot mode starts the shared example tree at `danger-full-access` so established fixtures remain runner-independent; the switching and escalation inputs explicitly select `workspace-write` before exercising the policy path. Deliberately absent: a fixture carrying a real DENIAL — denial stderr is the backend's dialect and would pin a fixture to its recording platform; the escalation prompts assert the prior denial instead, and the denial→marker path stays on the real-runner tiers above. +- **Unit:** pin platform selection and profiles, fail-closed runner classification, per-call facts, escalation validation and outcomes, permission preset folding and write-through, narrator coalescing, ACP advertisement and validation, and turn-enclosed config writes. +- **Keyless real-runner:** exercise bwrap, Landlock, and Seatbelt against real filesystem effects at provider and bash-consumer layers; packed-install coverage proves the registry launcher remains executable. The real ACP composition pins permission switching and rejects unknown presets. CI rejects a silent all-skip. +- **With-key:** drive a real model, runner, bridge answerer, and disk effect through granted and rejected escalation; unavailable credentials or runners self-skip. +- **Snapshot:** pin the permission config-option wire, preset and knob events, prompt deltas and notices, and both scripted approval branches. Snapshot mode starts unconfined so unrelated fixtures remain platform-independent; policy scenarios switch explicitly. Real denial stderr stays on platform tests because its dialect is runner-specific. ## Deferred phases @@ -155,7 +155,7 @@ Each phase gets its full design when picked up, validated against the code at th - **A generic `env/state` facts map with an owner service** — rejected: approval and sandbox compose independently, so neither's state may drag in a third package; single-key folds are one `findLast` each, dissolving the owner service; no invariant spans the knobs, so atomic multi-key patches bought nothing. - **Narrate via `agent/user-message` + a bus event** — rejected: it presupposes a turn-entry seam that does not exist (the real seam is `agent/prompt-submit`), and pre-step's position serves both the coalesced turn-entry notice and the mid-turn immediacy bound with one listener. - **A standing prompt statement of the sandbox mode (+ a switch narrator)** — shipped first, then removed on live evidence: with `Bash commands run under the "read-only" file sandbox.` in every request, the model refused to ATTEMPT denied-then-escalatable work (five of twelve turns in the first manual session ended with zero tool calls), turning the sandbox into a soft lockout. The denial marker names the mode at the moment it matters and the escalation fields carry the recovery; the approval knob keeps its statement because an auto-rejection is behaviorally indistinguishable from a human "no". -- **Track "last told" with its own bookkeeping events** — rejected: the latest `request/header` already records the exact prompt the model saw; parsing the closed candidate sentences back replaces a second bookkeeping stream — events are needed only where they ARE the store. +- **Track "last told" with its own bookkeeping events** — rejected: the `request/header` fold already records the exact prompt the model saw; parsing the closed candidate sentences back replaces a second bookkeeping stream — events are needed only where they ARE the store. - **ACP session modes instead of config options** — rejected: the preset is already one deployment-defined config-option select, and modes are slated for removal in ACP v2. ## Consequences diff --git a/packages/compact/compact-basic/src/index.ts b/packages/compact/compact-basic/src/index.ts index c879dabfa1..d32264cd2f 100644 --- a/packages/compact/compact-basic/src/index.ts +++ b/packages/compact/compact-basic/src/index.ts @@ -337,13 +337,7 @@ export class BasicCompactService extends CompactService { agent: Agent, signal?: AbortSignal, ): Promise { - // Resolve the range by surface POSITION, not numeric seq interval. A prior - // replace lands a fresh high-seq summary node AT the shadowed range's - // position, so the surface order (head→tail) no longer tracks seq order — - // `[newSummarySeq, olderRetainedSeq, …]` is normal. Indexing into the - // ordered node list and slicing it is the only correct way to read a range; - // a `seq >= start && seq <= end` interval test would mis-collect - // nodes (and `start > end` would falsely reject) once that happens. + // Resolve by surface position: a newer replacement seq may occupy an older slot. const nodes = session.surface.nodes const startIdx = nodes.indexOf(start) const endIdx = nodes.indexOf(end) @@ -510,14 +504,8 @@ export class BasicCompactService extends CompactService { // The whole surface fits the retain budget — nothing to compact. if (keepFromIdx === 0) return null - // Round the cutoff to a tool-pairing boundary: if the cut before - // `nodes[keepFromIdx]` is unbalanced (an unanswered tool-call sits before - // it — i.e. it is mid-step), extend the retained side head-ward until the - // cut is balanced, so the compacted range ends without splitting an - // assistant↔result pair. A node that belongs to no step is already a - // balanced (free) boundary. Decline if no balanced cut exists at or below - // `keepFromIdx` (the compactable range is only an un-splittable open tail - // step — retry once it closes). + // Round the cutoff head-ward to a tool-pairing boundary; decline when no + // safe compactable prefix exists. while (keepFromIdx > 0) { // eslint-disable-next-line @typescript-eslint/no-non-null-assertion if (isToolPairingBalanced(nodes, events, nodes[keepFromIdx]!)) break diff --git a/packages/compact/compact-basic/tests/compact-basic.spec.ts b/packages/compact/compact-basic/tests/compact-basic.spec.ts index 44ae51ccad..140ad5efb0 100644 --- a/packages/compact/compact-basic/tests/compact-basic.spec.ts +++ b/packages/compact/compact-basic/tests/compact-basic.spec.ts @@ -1548,40 +1548,23 @@ describe('BasicCompactService edge cases', () => { describe('BasicCompactService positional range (surface seqs are not monotonic after a replace)', () => { it('compacts a second region after the first replace lands a high-seq summary at the head position', async () => { - // A replace inserts the new summary node (a high seq) AT the shadowed - // range's surface position, so the surface becomes - // [highSeqSummary, …olderRetainedLowerSeqs]. A second compaction over a - // range whose start node has a HIGHER seq than its end node must still - // succeed — the range is positional, not a numeric seq interval. + // Replacement can make surface seqs non-monotonic; ranges remain positional. const svc = createTestService({ auto: false }) const session = multiTurnSession(4, 1) - // First compaction: shadow the two oldest surface nodes. const nodes0 = session.surface.nodes const first = await compactRegion(svc, session, nodes0[0]!, nodes0[1]!, 'm') - // The summary node now sits at the head with a seq HIGHER than the - // retained older nodes that follow it — the non-monotonic surface. (The - // head is the user/message replace node, appended after the compact/summary - // provenance event, so its seq is at least first.summarySeq.) const nodes1 = session.surface.nodes expect(nodes1[0]!).toBeGreaterThanOrEqual(first.summarySeq) expect(nodes1[0]!).toBeGreaterThan(nodes1[1]!) - // Second compaction: shadow [summary(head) … turn-2's step end]. The start - // seq (the head summary node) is GREATER than the end seq (an older retained - // node), so the range is a SURFACE-POSITION span, not a numeric seq interval. - // The end must land on a step boundary (turn-2's assistant message closes - // its step). const startSeq = nodes1[0]! const endSeq = nodes1[2]! expect(startSeq).toBeGreaterThan(endSeq) const second = await compactRegion(svc, session, startSeq, endSeq, 'm') - // Exactly the three nodes at surface positions [0..2] are shadowed, in - // surface order — the positional slice, regardless of their seq values. expect(second.shadowedSeqs).toEqual([nodes1[0]!, nodes1[1]!, nodes1[2]!]) - // The surface still derives cleanly: a new head replace node + the rest. const finalNodes = session.surface.nodes expect(finalNodes[0]!).toBeGreaterThanOrEqual(second.summarySeq) expect(session.deriveMessages().length).toBe(finalNodes.length) @@ -1591,20 +1574,13 @@ describe('BasicCompactService positional range (surface seqs are not monotonic a const svc = createTestService({ auto: false }) const session = multiTurnSession(3, 1) - // First compaction shadows the oldest two surface nodes, landing a high-seq - // summary node at the head. const n0 = session.surface.nodes await compactRegion(svc, session, n0[0]!, n0[1]!, 'm') - // Second compaction spans [head summary … turn-2's step end]. The head's seq - // is higher than the older retained nodes' seqs, so a log-seq-order walk - // would emit the older messages BEFORE the checkpoint. const n1 = session.surface.nodes svc.summarizeCalls = [] await compactRegion(svc, session, n1[0]!, n1[2]!, 'm') - // The extracted transcript follows surface order: the checkpoint (head) - // first, then the older retained messages — matching deriveMessages(). const { text } = svc.summarizeCalls[0]! const checkpointIdx = text.indexOf('compacted-summary') const olderIdx = text.indexOf('turn 2 user') diff --git a/packages/core/agent-loop/tests/request-reconstruction.spec.ts b/packages/core/agent-loop/tests/request-reconstruction.spec.ts index 7054b683cb..721c2a1eff 100644 --- a/packages/core/agent-loop/tests/request-reconstruction.spec.ts +++ b/packages/core/agent-loop/tests/request-reconstruction.spec.ts @@ -1,11 +1,8 @@ /** - * Loop-level reconstructability: every request the loop sends is a pure - * function of the session log — messages are the derivation at the step/start - * boundary, the header is the latest request/header snapshot — and every - * request is an append-extension of its predecessor unless a logged event - * (compaction replace, header change) explains the difference. The requests - * recorded by the mock adapter are the observable; the offline-rebuild test - * at the bottom is the theorem stated end-to-end. + * Loop-level reconstructability: every request the loop sends is a pure function of the + * session log — messages derive at the step/start boundary and the header is the latest + * request/header snapshot. Each request extends its predecessor unless a logged compaction + * replacement or header change explains the difference. */ import { describe, expect, it } from 'vitest' diff --git a/packages/core/session/README.md b/packages/core/session/README.md index 3f11876c3b..a65765df39 100644 --- a/packages/core/session/README.md +++ b/packages/core/session/README.md @@ -1,6 +1,6 @@ # dsh-session -Event-sourced session log and in-memory store. A `Session` is the append-only source of truth for an agent's whole interaction history — the LLM message history is *derived* from it. A **surface** layer (an ordered sequence of message-producing event seqs) is maintained on top of the raw log for efficient derivation and compaction. +Event-sourced session log and in-memory store. A `Session` is the append-only source of truth for an agent's whole interaction history — the LLM message history is *derived* from it. A **surface** layer (an ordered projection of message-producing events) is maintained on top of the raw log for efficient derivation and compaction. ## Service: `SessionStore` (ctx key: `sessions`) @@ -8,35 +8,35 @@ Creates and holds event-sourced `Session` instances. Persistence is intentionall ### Public API -- `ctx.sessions.create(id?: SessionId, options?: { seed?: readonly SessionEvent[]; meta?: { cwd?: string; parentSession?: SessionId; createdAt?: number; seedLength?: number } }): Session` — Create a session. The persistence/replay seed and resulting header are validated, detached, and deep-frozen at this durable boundary. The store fills `version`/`id` and defaults `createdAt` to now; a persisted reconstruction supplies the original `createdAt` and `seedLength`. Disposed with the calling fiber. -- `ctx.sessions.flush(session: Session): Promise` Dispatch the awaited `session/flush` durability checkpoint with the carrier captured at enter — THE flush entry point (the loop's turn-end checkpoint and idle injection call it; never dispatch a raw `ctx.parallel`). Every captured listener starts, the call waits for all of them to settle, and a failure rejects only after the other listeners finish. Rejects a prepared, detached, or stale same-id object instead of inventing a subject-less carrier. +- `ctx.sessions.create(id?, { seed?, meta? }?)` validates and detaches durable seed/header data, fills the version and id, defaults `createdAt` to now, publishes the session, and binds it to the calling fiber. Persisted reconstruction supplies its original `createdAt` and `seedLength`. +- `ctx.sessions.flush(session)` dispatches the awaited parallel durability checkpoint through the session's captured scope. Every listener starts and the call waits for all to settle before reporting failure; unpublished, detached, and stale objects reject. - `ctx.sessions.fork(source, boundary?, childSessionId?): Session` — Resolve a live session object or id, select a seed through the inclusive `boundary` event seq (default: current last event), require that boundary to be `turn/end`, and create a live child session with lineage metadata. - `ctx.sessions.get(id: SessionId): Session | undefined` - `ctx.sessions.list(): Session[]` #### Advanced: ordered-teardown lifecycle primitives -`create()` covers the common case (the session is owned by the calling fiber). When a session must be torn down **in order with another resource** — so a final flush is captured before the store attachment and publication hooks are removed — `create()`'s self-contained effect is wrong, because a fiber unload disposes sibling effects *concurrently*. For that, split the lifecycle and fold it into the owner's single effect: +Use the split lifecycle only when teardown must be ordered with another resource: -- `ctx.sessions.prepare(id?, options?): Session` — validate durable seed/header data and construct the `Session` WITHOUT entering it into the store. Same options as `create`. -- `ctx.sessions.enter(session): () => void` — perform the authoritative ID collision check, install append publication state, and insert the exact session without announcing it. Returns an idempotent detach bound to the captured entry object, so a stale disposer cannot remove a later same-ID replacement. Concurrent same-ID preparation is allowed; only one final entry succeeds. -- `ctx.sessions.announce(session): void` — begin the one allowed `session/created` announcement for an entered session; repeat and reentrant calls reject before dispatch. A detach requested synchronously by a creation listener is deferred until that dispatch unwinds, so another creation listener cannot observe `session/disposed` before its own `session/created` callback. Detach emits `session/disposed` exactly once, including rollback after a partially delivered creation notification; a never-announced entry emits neither edge. +- `prepare(id?, options?)` validates and constructs without publication. +- `enter(session)` performs the collision check, publishes without announcing, and returns an entry-bound idempotent detach. Concurrent same-id preparations are allowed, but only one entry succeeds; a stale detach cannot remove its replacement. +- `announce(session)` emits the single creation edge and rejects repeat or reentrant announcements. Detach during that dispatch is deferred and later emits the paired disposal edge; an unannounced entry emits neither lifecycle edge. -`dsh-agent-loop` is the canonical consumer: after unpublished agent setup it enters both session and agent before announcing either, then nests loop stop, agent removal, session detach, and scope unwind in one ordered lifecycle. The final flush therefore settles before this package detaches the session, whether teardown starts from an `AgentHandle` or owner-fiber unload. +`dsh-agent-loop` uses this split so final loop flush precedes session detach; see the [ownership RFC](../../../docs/rfc/implemented/architecture/2026-06-18-agent-lifecycle-and-ownership-seams.md). ### Live service events -The store pairs announced creation with disposal, publishes each append, and provides an awaited durability checkpoint. Before the log push it resolves the scoped `session/event` callback list. The push is the commit point; callback throws or returned-promise rejections are logged and contained per observer. A committed append therefore returns normally, later observers still run, and detach waits until publication unwinds. Exact `session/*` signatures, modes, and scope-carrier behavior live in the generated [Cordis event catalog](../../../docs/cordis-catalog/events.md); the append-only payload vocabulary is separately generated into the [persistence catalog](../../../docs/persistence-catalog.md). +The store pairs announced creation with disposal, publishes post-commit append notifications with per-listener containment, and provides an awaited durability checkpoint. Exact signatures and scope behavior live in the generated [event catalog](../../../docs/cordis-catalog/events.md); payloads live in the [persistence catalog](../../../docs/persistence-catalog.md). ### Class: `Session` Plain class (not a Cordis Service). Create via `ctx.sessions.create()`. -- `session.append(type, data, opts?): SessionEvent` — synchronous, never blocks on I/O. At this durable boundary, data and surface metadata are lossless-JSON snapshotted and deep-frozen. For an attached session, a reentrant append during dispatch/observer publication rejects, and detach waits for that publication to unwind. Callbacks resolve before the log push; the push is the commit point, after which each observer failure is contained independently. Runtime surface validation covers widened unions and raw seed/load logs. -- `session.deriveMessages(): Message[]` — the LLM message history, CACHED: each surface entry is projected exactly once, when first seen (O(new entries) per call; a surface rewrite rebuilds via `surface.replaceGeneration`). Returns a fresh array per call over shared, deep-frozen `Message` objects. Each projection reuses the already deep-frozen content in its durable log event, so no second deep clone is needed and a consumer still cannot mutate logged data. The surface is the single source of derived history — there is no raw-log fallback. -- `session.deriveEventMessage(event): Message | null` — the per-event projection `deriveMessages()` folds: a fresh message wrapper that reuses the event's already frozen content, or `null` when the event produces none (a non-surface event, or an empty-content `assistant/message` hosting only usage). External reconstructors and the dev invariant fold the same function over a log prefix's surface, so no two paths can disagree about what a request's messages were (the reconstructability RFC). -- `session.surface: SurfaceManager` — the derived surface, lazily folded from `surfaceOp` markers in the log. Processes only new events (delta) on each access — the log is append-only, so prior events never change. `surface.replaceGeneration` is the rewrite signal: bumped by every folded `replace` and never reset, so an incremental consumer comparing generations cannot be fooled. -- `session.events` — a cached, frozen array snapshot over deep-frozen events. Repeated reads without an append return the same array; an append invalidates the cache and the next read returns a new snapshot, while earlier snapshots stay unchanged. Neither a cast nor a retained reference can push into the live log or rewrite an accepted event. +- `session.append(type, data, opts?)` snapshots and freezes durable data and surface metadata, commits synchronously, then notifies observers with independent failure containment. Reentrant attached-session appends reject, and runtime checks cover widened unions and loaded logs. +- `session.deriveMessages()` incrementally projects each new surface entry once and returns a fresh array over shared frozen messages. A surface rewrite rebuilds the projection; there is no raw-log fallback. +- `session.deriveEventMessage(event)` is the canonical per-event projection used by reconstruction and invariants. +- `session.surface` lazily folds only new `surfaceOp` markers; `replaceGeneration` changes on every rewrite. +- `session.events` is a cached frozen snapshot invalidated by append; accepted events remain deeply frozen. - `session.seq`, `session.id` — current sequence and readonly typed identity. - `session.header: SessionHeader` — detached, deep-frozen creation metadata (`version`, `id`, `createdAt`, optional `cwd`/`parentSession`/`seedLength`). Construction validates the durable record and requires its id to match `session.id`. @@ -49,11 +49,11 @@ Durable values need one accepted representation, not a check followed by a secon - `SurfaceOp` — how an event entered the ordered surface: `'append'` (normal tail append) or `{ op: 'replace', start, end }` (replace entries from `start` through `end` inclusive — both must be valid surface seqs; `start === end` replaces one entry). Used by compaction to shadow old events without deleting them. - `SurfaceIntent` — `{ surfaceOp: SurfaceOp; sourceEventSeqs?: number[] }`, the required third parameter to `session.append()` for surface-eligible types. - `foldSurface(events)` — replay the canonical surface transitions into detached current event sequences and actual replacement ranges, rejecting surface-eligible events that lack their mandatory marker. `SurfaceManager` shares the same transitions while retaining only its incremental sequence cache. -- `isSurfaceEvent(event)` / `isSurfaceEligibleType(type)` — the first narrows a `SessionEvent` to a fully formed surface event (type is surface-eligible AND `surfaceOp` present); the second is the type-only check, used to detect a surface-eligible event MISSING its marker when validating a seed or loaded log. +- `isSurfaceEvent(event)` / `isSurfaceEligibleType(type)` — the first narrows a `SessionEvent` to a fully formed surface event; the second is the type-only check used to detect a surface-eligible event missing its marker when validating a seed or loaded log. ### Request-header reconstruction (`request-header.ts`) -The `request/header` event records a full canonical `EpochHeader` snapshot with reason `initial`, `resume`, or `change`, making the request envelope logged session state and every conversation request a pure function of the log. `foldRequestHeader(events)` selects the latest snapshot from a log or prefix; `canonicalHeader` pins the one representation of absence (empty system/tools/messagePrefix ≡ absent fields), and `headerEquals` compares canonical headers. `EpochHeader.messagePrefix` is the durable record of the `agent/session-prefix` waterfall's product — composed once per loop instance, the request is `messagePrefix + derived history`, and `deriveMessages()` never returns it. Legacy v0 seeds containing the removed `request/header-delta` event or its full-snapshot `fallback` reason are rejected rather than partially replayed. +`request/header` records a full canonical snapshot of the non-history request envelope with reason `initial`, `resume`, or `change`. `foldRequestHeader()` selects the latest snapshot; legacy delta events and the removed `fallback` reason are rejected. `messagePrefix` remains separate from derived history. See the [reconstructable-requests RFC](../../../docs/rfc/implemented/architecture/2026-07-05-reconstructable-requests.md). ### Session event vocabulary (`types.ts`) @@ -65,7 +65,7 @@ Also defines `TurnTriggerMap` and `TurnEndReasonMap` (merge-extensible sum types Every `SessionEvent` carries two optional top-level fields (structural metadata): -- `sourceEventSeqs?: number[]` — seq numbers of provenance sources (e.g., the `assistant/chunk` seqs behind an `assistant/message`, or the shadowed nodes behind a compaction replace node). +- `sourceEventSeqs?: number[]` — seq numbers of provenance sources (e.g., the `assistant/chunk` seqs behind an `assistant/message`, or the shadowed entries behind a compaction replacement entry). - `surfaceOp?: SurfaceOp` — how this event entered the surface. Absent for non-surface events (boundaries, chunks, usage, errors). ### Metadata types (`types.ts`) @@ -76,15 +76,15 @@ Every `SessionEvent` carries two optional top-level fields (structural metadata) - Persistence plugins: subscribe to `session/event` (write-behind) and drain on `session/flush` (awaited) and fiber dispose. A durable backend reads the log and reloads it into a live session; the metadata seam (`SessionHeader`, `session.header`) is what such a backend stores beside the log. - Replay/fork: `create(id, { seed })` validates and freezes a contiguous log and rebuilds its surface. `fork(source, boundary?, childSessionId?)` selects a completed-turn prefix and records lineage. -- Compaction: the `dsh-compact-basic` plugin appends a `user/message` with `surfaceOp: { op: 'replace', start, end }` to shadow old surface nodes behind a summary checkpoint. +- Compaction: the `dsh-compact-basic` plugin appends a `user/message` with `surfaceOp: { op: 'replace', start, end }` to shadow old surface entries behind a summary checkpoint. ## Model Experience ### Derived message history -**What the model sees**: The model receives projections of `user/message`, `assistant/message`, and `tool/result` surface nodes verbatim. A `context/message` is a user-role message containing exactly ``, its content blocks, and ``; `steering/message` uses the identical `` / `` wrapper. Tool calls live inside assistant messages. Chunks, boundaries, usage, hook records, todo records, and other log-only events add no message. +**What the model sees**: The model receives projections of `user/message`, `assistant/message`, and `tool/result` surface entries verbatim. A `context/message` is a user-role message containing exactly ``, its content blocks, and ``; `steering/message` uses the identical `` / `` wrapper. Tool calls live inside assistant messages. Chunks, boundaries, usage, hook records, todo records, and other log-only events add no message. -**Token effect**: Appended surface nodes are resent on later steps. A `replace` surface operation removes the shadowed nodes from future inputs without deleting their raw log records. +**Token effect**: Appended surface entries are resent on later steps. A `replace` surface operation removes the shadowed entries from future inputs without deleting their raw log records. ### Crash-repair result diff --git a/packages/core/session/tests/surface.spec.ts b/packages/core/session/tests/surface.spec.ts index 5f09efe089..53c9abfccc 100644 --- a/packages/core/session/tests/surface.spec.ts +++ b/packages/core/session/tests/surface.spec.ts @@ -108,14 +108,10 @@ describe('SurfaceManager', () => { it('rebuild with replace operation splices out shadowed nodes', () => { const s = surfaceSession() - // seq: 0=turn/start, 1=user, 2=assistant, 3=turn/end - // Surface nodes: seq 1 (user), seq 2 (assistant). - // Replace both with a compaction marker. Both 1 and 2 are valid surface seqs. s.append('assistant/message', { turn: 2, step: 1, content: [{ type: 'text', text: 'summary' }] }, { surfaceOp: { op: 'replace', start: 1, end: 2 }, sourceEventSeqs: [1, 2] }, ) - // Now the surface should have just the compaction node. expect(s.surface.nodes).toEqual([4]) }) diff --git a/packages/ui/user-approval/README.md b/packages/ui/user-approval/README.md index b4fe3420ef..cebd25275e 100644 --- a/packages/ui/user-approval/README.md +++ b/packages/ui/user-approval/README.md @@ -1,16 +1,14 @@ # @deepseek-ai/dsh-user-approval -User-approval seam. Owns the `ctx.approval` service ([`ApprovalService`](src/index.ts)) and the one-shot permission vocabulary the harness shares: `ApprovalRequest` (agent + tool identity + reason + abort signal), the closed `ApprovalOutcome` union (`allowed-once` / `rejected` / `cancelled` / `unavailable`), the `ApprovalRequestId` brand pairing the two log-only audit events (`approval/asked` / `approval/decided`), and the `approval/request` waterfall the answerers listen on. It lives in the UI group because its purpose is human permission, while remaining channel-neutral: it depends only on Cordis and core vocabulary packages, never on a concrete UI. +Channel-neutral one-shot approval seam. `ctx.approval.request(req)` returns `allowed-once`, `rejected`, `cancelled`, or `unavailable`; missing or failing answerers fail closed, and a grant applies only to the requested action. Exact event signatures live in the generated [Cordis catalog](../../../docs/cordis-catalog/events.md). -The contract in one line: `ctx.approval.request(req)` puts exactly one question — "may this specific action proceed?" — to whatever answerers the deployment composed, and its answerer phase always produces an outcome: an aborted signal yields `cancelled`, a throwing or missing answerer yields `unavailable`, and `allowed-once` is a grant for the single asked-about action, never a class of future ones. `ApprovalRequest` is a readonly same-process contract: the service borrows the exact request, agent, session, and abort signal rather than cloning or freezing them. The request requires an open turn because the audit pair is turn-enclosed by contract (the turn is the durable log's commit/replay boundary; a bare event between turns is crash-tail garbage on reload), so an idle ask rejects before appending. Either audit append may reject before commit because returning an unlogged decision would violate the pair. Session contains post-commit observer failures, so an authoritative audit append cannot reject the request or suppress its matching event. +Each request must belong to an open agent turn. The service appends a paired `approval/asked` and `approval/decided` audit record, while the model sees only the resulting logged tool outcome. An aborted request resolves `cancelled`; an audit append that fails before commit rejects rather than returning an unlogged decision. -The service is the mechanism, answerers are the policy. Answerers are `approval/request` waterfall listeners occupying a single decision slot: answer for an agent you own by returning an outcome without calling `next()`, or delegate an agent you don't recognize by calling `next()` — the chain's built-in default is `unavailable`, so a deployment with no answerer (headless, CI) fails closed with zero configuration. Dispatch is keyed by `req.agent`: a listener registered through `agent.ctx` receives only that agent's questions, while a plain-context listener receives every agent's. Registration order across sibling plugins is not load-order deterministic; compose one terminal answerer per deployment and use `prepend` listeners only for decide-or-delegate gates. +Answerers are `approval/request` waterfall listeners. Return an outcome to answer for an owned agent or call `next()` to delegate. Agent-scoped listeners receive only that agent's requests; compose one terminal answerer per deployment because sibling listener order is not a policy priority mechanism. The ACP bridge is the shipped human answerer. -The seam also owns the per-session POLICY tier ([the sandbox RFC § Per-session mode switching](../../../docs/rfc/implemented/feature/2026-07-06-sandbox.md)): `ApprovalPolicy` is `'ask'` (delegate to the answerers) or `'never'` (deterministically reject without prompting anyone; the strict CI/unattended stance), with `effective = fold(the session's 'approval/policy' events, last one wins) ?? Config.policy` — the session log is the store, written only through `setApprovalPolicy(session, policy)`, which rejects any value outside that closed vocabulary before appending. The service decides `'never'` inside `request()` itself, before dispatching the waterfall (`'never'` → `'rejected'` with the audit pair still landing; no listener registration, including a later `prepend`, can precede it), states `'never'` — and only `'never'` in prose — in a per-agent prompt section, records either value with a source-owned header marker, and narrates a policy switch to the model in at most one coalesced `agent/pre-step` notice. The restart fallback reads the marker rather than deployment-controlled persona prose; attribution is positional (an override event after the last `request/header` reads `changed by the user`, otherwise `changed by the operator/config`). +`ApprovalPolicy` is `'ask'` or `'never'`. The effective value is the last `approval/policy` event, falling back to config; `setApprovalPolicy()` is the write path. `'never'` rejects before interactive dispatch and is the only policy stated in the prompt. Switches produce at most one coalesced notice, attributed to the user when the override follows the last `request/header` and to operator/config otherwise. -One seam serves both ask paths of [the sandbox RFC](../../../docs/rfc/implemented/feature/2026-07-06-sandbox.md): the `tools/pre-execute` `ask` decision (routed by [`@deepseek-ai/dsh-tools`](../../core/tools/) when this service is mounted; degrading to deny when it is not), and the sandbox post-denial escalated retry (the bash tool's `sandbox_permissions` gate in [`@deepseek-ai/dsh-tool-bash`](../../bash/tool-bash/) — [the sandbox RFC § Escalation](../../../docs/rfc/implemented/feature/2026-07-06-sandbox.md)). The full design: [the approval-seam RFC](../../../docs/rfc/implemented/feature/2026-07-06-approval-seam.md). - -Answerers today: the ACP bridge ([`@deepseek-ai/dsh-acp`](../../ui/acp/)) forwards to the editor's `session/request_permission` prompt for agents it owns. The audit events are log-only session records — the model only ever sees the tool result the asker derives from the outcome. +The tools pipeline routes `ask` decisions through this seam and fails closed when it is absent; the sandboxed bash tool also uses it for escalated retries. The ACP bridge is the shipped human answerer for calls it owns. Audit events remain log-only, so the model sees only the asking consumer's result. See the [approval-seam RFC](../../../docs/rfc/implemented/feature/2026-07-06-approval-seam.md) and [sandbox RFC](../../../docs/rfc/implemented/feature/2026-07-06-sandbox.md). ## Model Experience