Merge remote-tracking branch 'origin/master' into codex/agent-session-jsonl-location
# Conflicts: # docs/config-catalog.md # docs/cordis-catalog/services.md # docs/core-data-structures/bash.md # docs/module-graph.md # docs/rfc/INDEX.md # docs/rfc/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-surface.md # docs/rfc/implemented/feature/2026-06-30-hook-bridges.md # examples/acp-agent/tests/snapshots/advanced-toolchain/session.1.jsonl # examples/acp-agent/tests/snapshots/advanced-toolchain/session.2.jsonl # examples/acp-agent/tests/snapshots/advanced-toolchain/session.jsonl # examples/acp-agent/tests/snapshots/advanced-toolchain/system-prompt.golden.md # examples/acp-agent/tests/snapshots/both-mode-turn/session.jsonl # examples/acp-agent/tests/snapshots/both-mode-turn/system-prompt.golden.md # examples/acp-agent/tests/snapshots/code-mode-turn/system-prompt.golden.md # examples/acp-agent/tests/snapshots/skill-load/session.jsonl # examples/acp-agent/tests/snapshots/text-turn/session.jsonl # examples/sandbox-acp-agent/tests/snapshots/mode-switching/session.jsonl # packages/bash/bash-local/README.md # packages/bash/bash-local/src/run.ts # packages/bash/bash-local/tests/run.spec.ts # packages/bash/bash/README.md # packages/bash/tool-bash/README.md # packages/bash/tool-bash/src/index.ts # packages/bash/tool-bash/tests/tools.spec.ts # packages/cordis/tool-cordis/src/api-catalog.ts # packages/core/agent-core/src/index.ts # packages/session-persistence/session-persistence-jsonl/src/index.ts # packages/session-persistence/session-persistence-sqlite/src/index.ts # packages/session-persistence/session-persistence/README.md # packages/session-persistence/session-persistence/src/index.ts # packages/ui/acp-agent/README.md # packages/ui/stdio-agent/README.md
This commit is contained in:
Yichen Jiang
672 files changed
+10233
-14251
No files matched your search
@@ -1,26 +1,7 @@
|
||||
/**
|
||||
* The backend-agnostic write-path orchestration shared by every first-party
|
||||
* {@link SessionPersistence} backend.
|
||||
*
|
||||
* Every durable backend needs the same orchestration: the in-memory bookkeeping
|
||||
* (the per-id state, the write-behind buffers, the per-id serialization chains,
|
||||
* the per-session init promises), the `session/event` → buffer → `session/flush`
|
||||
* drain, lazy materialization, crash-tail repair on load, the four
|
||||
* `session/created` adoption cases (new / HMR-adopt / collision /
|
||||
* ownerless-claim), and dispose-time quiescence. Only the STORAGE primitives are
|
||||
* backend-specific (file bytes for `dsh-session-persistence-jsonl`, `node:sqlite`
|
||||
* rows for `dsh-session-persistence-sqlite`). {@link PersistenceCoordinator} owns
|
||||
* the orchestration; a backend supplies the storage primitives as a small
|
||||
* {@link PersistenceBackend} hook object.
|
||||
*
|
||||
* The abstract {@link SessionPersistence} service's public API is independent of
|
||||
* this: a backend IS a `SessionPersistence` (its four public methods delegate to
|
||||
* a coordinator it composes), so a third-party backend MAY implement the service
|
||||
* directly without using the coordinator at all.
|
||||
*
|
||||
* See the write-coordinator RFC (docs/rfc/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.md)
|
||||
* for the design rationale (composition over inheritance, the opaque torn marker).
|
||||
*
|
||||
* Shared buffering, serialization, adoption, repair, and disposal orchestration
|
||||
* for first-party backends. Third-party backends may implement the public
|
||||
* persistence seam directly.
|
||||
* @module @deepseek-ai/dsh-session-persistence/coordinator
|
||||
*/
|
||||
|
||||
@@ -30,16 +11,9 @@ import type { Session, SessionEvent, SessionId, SessionHeader } from '@deepseek-
|
||||
import { seedCoversPrefix } from './index.ts'
|
||||
|
||||
/**
|
||||
* A stored session's durable prefix as read back from a backend: its
|
||||
* {@link SessionHeader}, the preserved (seq-contiguous, parseable) event prefix,
|
||||
* and an OPAQUE `tornMarker` that is present iff a never-committed torn tail must
|
||||
* be truncated before further writes.
|
||||
*
|
||||
* The coordinator NEVER inspects `tornMarker`'s value — it only tests
|
||||
* `!== undefined` (is there a tail to repair?) and passes the value back to
|
||||
* {@link PersistenceBackend.commitRepair}. Each backend chooses its own marker
|
||||
* type: the JSONL backend uses the byte offset to truncate to, the SQLite
|
||||
* backend uses the seq to delete from (both happen to be `number`).
|
||||
* A stored session's header, valid contiguous event prefix, and optional opaque
|
||||
* torn-tail marker. The coordinator only checks marker presence and returns its
|
||||
* value to {@link PersistenceBackend.commitRepair}; each backend owns the type.
|
||||
*/
|
||||
export interface StoredPrefix<TornMarker = unknown> {
|
||||
meta: SessionHeader
|
||||
@@ -112,16 +86,9 @@ interface SessionState {
|
||||
/** The next seq the backend expects to append (the stored log length). */
|
||||
cursor: number
|
||||
/**
|
||||
* Whether the backend has physically written this session (a JSONL file /
|
||||
* SQLite row exists). `create()` registers state LAZILY — cursor 0,
|
||||
* materialized false, nothing on disk — so an empty session leaves no
|
||||
* artifact and the FIRST `appendBatch` writes the header + its events in ONE
|
||||
* transaction (the "a row exists ⇔ it has events" invariant `list`
|
||||
* relies on; a separate up-front materialize could crash leaving a row with
|
||||
* zero events). The flag is the only signal that distinguishes a session
|
||||
* registered-but-never-written from one durably present, which the reclaim
|
||||
* path needs (an abandoned id with no artifact AND no buffered events is free
|
||||
* to reuse; a materialized one is a real collision).
|
||||
* Whether lazy creation has produced a durable artifact. The first append
|
||||
* atomically materializes the header with events; reclaim logic uses this to
|
||||
* distinguish an unused id from a persisted collision.
|
||||
*/
|
||||
materialized: boolean
|
||||
/**
|
||||
@@ -166,14 +133,9 @@ export class PersistenceCoordinator<TornMarker = unknown> {
|
||||
*/
|
||||
private chains = new Map<SessionId, Promise<unknown>>()
|
||||
/**
|
||||
* Per-session init promise (onCreated). Keyed by the LIVE Session OBJECT, not
|
||||
* its id: a disposed fiber's session can be replaced by a different live
|
||||
* Session reusing the same id (HMR, an ACP reconnect), and an id-keyed cache
|
||||
* would hand the new object the old object's init promise.
|
||||
*
|
||||
* Public (readonly) so a backend can expose it for white-box tests that await
|
||||
* a specific session's init (there is no public API to await one init); the
|
||||
* coordinator itself only ever mutates it internally.
|
||||
* Init promises keyed by live session object, preventing an id-reusing
|
||||
* replacement from inheriting stale initialization. Readonly access supports
|
||||
* backend white-box tests.
|
||||
*/
|
||||
readonly inits = new Map<Session, Promise<void>>()
|
||||
|
||||
@@ -184,16 +146,11 @@ export class PersistenceCoordinator<TornMarker = unknown> {
|
||||
// --- public surface (the backend's service methods delegate here) ---
|
||||
|
||||
/**
|
||||
* Register a new session's metadata (lazy: no physical write until the first
|
||||
* {@link append}). Rejects if the id is already tracked or already persisted.
|
||||
* @param meta - the header (id, version, cwd, lineage) to record; materialized
|
||||
* as a detached lossless-JSON snapshot at call time.
|
||||
* Register detached session metadata for lazy creation on the first append.
|
||||
* @param meta - header to snapshot; duplicate tracked or persisted ids reject.
|
||||
*/
|
||||
create(meta: SessionHeader): Promise<void> {
|
||||
// Snapshot the metadata at call time: the op runs later (behind the
|
||||
// per-session chain) and the snapshot is stored as the lazy state, so keeping
|
||||
// the caller's object by reference would let a later mutation of `id`/`cwd`
|
||||
// register under one key but materialize under a different path/header.
|
||||
// Snapshot before queueing so caller mutation cannot diverge the key and header.
|
||||
const snapshot = snapshotJsonValue(meta)
|
||||
if (snapshot === undefined) {
|
||||
return Promise.reject(new TypeError('session metadata must be losslessly JSON-serializable'))
|
||||
@@ -274,32 +231,20 @@ export class PersistenceCoordinator<TornMarker = unknown> {
|
||||
const { meta, events, tornMarker } = stored
|
||||
this.assertVersion(meta)
|
||||
|
||||
// Crash-recovery: if the log ended mid-turn (real, preserved events but no
|
||||
// closing turn/end), close it durably DURING load so disk, the returned log,
|
||||
// and the cursor all agree. The interrupted turn's real events are preserved,
|
||||
// never truncated (a turn can be huge — the session-persistence RFC); only a
|
||||
// never-fully-written torn tail fragment is discarded.
|
||||
// Preserve complete interrupted events and synthesize only missing closers.
|
||||
const closers = interruptedTurnClosers(events)
|
||||
const balanced = [...events, ...closers]
|
||||
|
||||
// Make the repair durable (truncate the torn tail + append the synthetic
|
||||
// closers) BEFORE recording state — commitRepair takes `meta` directly, so
|
||||
// there is no state-path ordering dependency (uniform across backends).
|
||||
// Repair storage before publishing coordinator state.
|
||||
if (tornMarker !== undefined || closers.length > 0) {
|
||||
await this.backend.commitRepair(meta, tornMarker, closers)
|
||||
}
|
||||
// The state keeps its OWN copy of the meta; the returned value is separate so
|
||||
// a consumer mutating loaded.meta cannot corrupt the backend's metadata.
|
||||
// Keep coordinator metadata detached from the returned record.
|
||||
this.states.set(id, { meta: { ...meta }, cursor: balanced.length, materialized: true })
|
||||
return { meta, events: balanced }
|
||||
}
|
||||
|
||||
// NOTE: there is deliberately no coordinator `list()`. Listing needs none of
|
||||
// the coordinator's orchestration (no per-id serialization, no cursor, no
|
||||
// in-memory state) — it is a pure read of stored metadata. A backend's public
|
||||
// `list()` IS the {@link PersistenceBackend.list} hook (one method); routing it
|
||||
// through the coordinator would only forward to that same hook, so the
|
||||
// coordinator stays out of the listing path entirely.
|
||||
// Listing is a direct backend read and needs no coordinator state.
|
||||
|
||||
// --- per-id serialization + adoption helpers ---
|
||||
|
||||
|
||||
@@ -1,23 +1,7 @@
|
||||
/**
|
||||
* The durable session-persistence seam (`ctx.sessionPersistence`): an abstract
|
||||
* service defining WHAT a persistence backend does — durably store, reload,
|
||||
* and list sessions — without saying HOW. Implementations subclass
|
||||
* {@link SessionPersistence} and register themselves as the
|
||||
* `sessionPersistence` service; `@deepseek-ai/dsh-session-persistence-jsonl`
|
||||
* (an append-only JSONL log per session) is the first and
|
||||
* `@deepseek-ai/dsh-session-persistence-sqlite` (`node:sqlite`, one row per
|
||||
* event) is a second that validates the seam is backend-agnostic by passing
|
||||
* the same `runPersistenceContract` suite. Further backends swap in an object
|
||||
* store or a remote service without touching the consumers (the write-path
|
||||
* plugin, the agent-loop resume seam).
|
||||
*
|
||||
* The persisted unit IS the existing {@link SessionEvent} — there is no
|
||||
* parallel "persisted message" type the log must be converted to and from
|
||||
* (faithful to the event-sourced model: the log is the single source of
|
||||
* truth). Metadata that is NOT replayable conversation state (format version,
|
||||
* cwd, lineage, seed boundary) travels separately as {@link SessionHeader},
|
||||
* which is owned by `dsh-session` and re-exported here.
|
||||
*
|
||||
* Durable session-persistence seam (`ctx.sessionPersistence`). Backends store
|
||||
* {@link SessionEvent}s as the event-sourced log and carry non-replayable
|
||||
* {@link SessionHeader} metadata separately.
|
||||
* @module @deepseek-ai/dsh-session-persistence
|
||||
*/
|
||||
|
||||
@@ -51,12 +35,8 @@ export interface SessionLocation {
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether a live session's seed reproduces a persisted prefix exactly. Backends
|
||||
* use this collision check to distinguish a legitimate resume/HMR rebind from a
|
||||
* different live session reusing an existing session id.
|
||||
*
|
||||
* The comparison includes the full event payload, not just seq/type/time, so a
|
||||
* mutated seed cannot be grafted onto a durable log with the same envelope.
|
||||
* Check whether a live seed exactly reproduces a durable prefix, including full
|
||||
* payloads. This distinguishes resume/HMR rebinding from an id collision.
|
||||
* @param seed - the live session's creation-time event snapshot.
|
||||
* @param prefix - the persisted prefix the seed must reproduce.
|
||||
* @returns `true` when the prefix fits within the seed and every event matches by JSON text.
|
||||
@@ -84,32 +64,10 @@ export function assertSerializable(events: readonly SessionEvent[]): void {
|
||||
}
|
||||
|
||||
/**
|
||||
* Abstract durable session-persistence service. Subclass, implement the
|
||||
* abstract methods, and load the subclass as a plugin — it registers as
|
||||
* `ctx.sessionPersistence` (one implementation per context; loading a second
|
||||
* throws, cordis' standard duplicate-service behavior).
|
||||
*
|
||||
* Contracts every implementation MUST honor (a DB backend asserts them inside
|
||||
* a transaction; a file backend appends at EOF):
|
||||
*
|
||||
* - **Append-only; a crashed turn is closed, not truncated.** Committed events
|
||||
* — those at or below a flushed `turn/end` — are never rewritten. A crash can
|
||||
* leave an unclosed final turn whose events are real (and possibly large);
|
||||
* {@link load} preserves them and closes the orphaned turn with synthetic
|
||||
* boundary events (see {@link load}). Only a never-fully-written torn tail
|
||||
* fragment is discarded.
|
||||
* - **Contiguous seq.** A persisted log is contiguous: `events[i].seq === i`.
|
||||
* {@link load} rejects a parse error or a `seq` gap in the COMMITTED region
|
||||
* (unloadable); {@link append}'s first event `seq` MUST equal the backend's
|
||||
* stored next-seq (after `load` has balanced any interrupted turn).
|
||||
* - **JSON-serializable events.** `SessionEventMap` is merge-extensible, so
|
||||
* {@link append} materializes each complete batch through the shared
|
||||
* lossless-JSON boundary before buffering it. The public `session.events`
|
||||
* view is immutable, but persistence still snapshots direct/replay callers at
|
||||
* this independent trust boundary.
|
||||
* - **Durability.** {@link append} returns only once the batch is durable
|
||||
* (the file backend fsyncs; a DB commits). {@link create} MAY defer the
|
||||
* physical write until the first {@link append} (lazy materialization).
|
||||
* Durable append-only session storage. Implementations preserve contiguous,
|
||||
* losslessly JSON-serializable events; {@link append} resolves only after
|
||||
* durability, and {@link load} balances a complete interrupted tail without
|
||||
* rewriting committed events.
|
||||
*/
|
||||
export abstract class SessionPersistence extends Service {
|
||||
constructor(ctx: Context) {
|
||||
@@ -146,29 +104,12 @@ export abstract class SessionPersistence extends Service {
|
||||
abstract append(id: SessionId, events: readonly SessionEvent[]): Promise<void>
|
||||
|
||||
/**
|
||||
* Reload a session: its {@link SessionHeader} plus the event log up to the last
|
||||
* durable checkpoint. Returns `meta` AND `events` so the live session is
|
||||
* reconstructed with its `cwd`/lineage, not just its log.
|
||||
*
|
||||
* The loop only flushes at `turn/end`, so a crash can leave a durable log
|
||||
* whose final turn never closed: real, fully-written events sit after the last
|
||||
* `turn/end`. Those events are PRESERVED — a single turn can be huge in a
|
||||
* long-horizon task, so truncating it would destroy real work — and `load`
|
||||
* CLOSES the orphaned turn by durably appending the minimal synthetic boundary
|
||||
* events: an error `tool/result` for every `tool-call` the crash left
|
||||
* unanswered (so the rehydrated history is a valid provider transcript — a
|
||||
* dangling assistant tool-call is otherwise rejected), then a `step/end` if a
|
||||
* step was open, then a `turn/end` carrying the `{ kind: 'interrupted' }`
|
||||
* reason. The returned `events` therefore end on a balanced `turn/end` and are
|
||||
* immediately usable as a session seed. Only a never-fully-written TORN tail
|
||||
* fragment (a half-written final record) is discarded. Returned events are
|
||||
* contiguous (`events[i].seq === i`); a parse error or a `seq` gap in the
|
||||
* COMMITTED region (at or before the last real `turn/end`) makes the session
|
||||
* unloadable (reject). Rejects an unknown format `version`. See the session-persistence RFC for
|
||||
* the crash-recovery contract.
|
||||
* Load a header and balanced contiguous log. A complete interrupted final
|
||||
* turn is preserved and durably closed with missing tool errors plus any open
|
||||
* step and turn boundaries; only a torn final record is discarded. Unknown
|
||||
* versions and corruption in the committed prefix reject.
|
||||
* @param id - the persisted session to reload.
|
||||
* @returns the header plus the event log, ending on a balanced `turn/end` —
|
||||
* immediately usable as a session seed.
|
||||
* @returns the header and a log ending on a balanced `turn/end`.
|
||||
*/
|
||||
abstract load(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }>
|
||||
|
||||
|
||||
Reference in New Issue
Block a user