deepseek-harness/packages/session-persistence/session-persistence

@deepseek-ai/dsh-session-persistence

English | 中文

The abstract durable session-persistence seam (ctx.sessionPersistence). Defines WHAT a persistence backend does — durably store, reload, and list sessions — without saying HOW. Mirrors the dsh-bash capability-seam template (capability seams): an abstract service here, a concrete implementation in a sibling package, consumers that inject the interface.

The persisted unit IS the existing SessionEvent (event-sourced model — the log is the single source of truth), so there is no parallel "persisted message" type. Metadata that is NOT replayable conversation state (format version, cwd, lineage, seed boundary, origin, delegation depth) travels separately as SessionHeader, owned by dsh-session and re-exported here.

Service API (ctx.sessionPersistence)

Method	Contract
locate(meta): SessionLocation | undefined	Resolve an absolute per-session artifact target without I/O or materialization. Backends without an independent local artifact return undefined.
create(meta): Promise<void>	Register a new session's metadata. MAY defer the physical write until the first append (lazy materialization).
append(id, events): Promise<void>	Durably persist a batch. Append-only; first event seq == stored next-seq after any repair; rejects non-JSON-serializable data naming the offending type.
load(id): Promise<{ meta; events }>	Return a stored header plus a balanced contiguous log whose events are detached and validated and whose identified messages are deeply frozen. The coordinator upgrades the supported same-version message and pre-react-loop event shapes into the current read snapshot; all other obsolete or malformed shapes still reject. A live load first flushes its snapshot and rejects while its turn is open; a cold load preserves an interrupted final turn and closes it with synthetic tool/result/step/end?/turn/end {interrupted} events. Only a torn tail fragment is dropped; committed corruption and unknown version reject.
inspect(id, signal?): Promise<{ meta; events }>	Return a detached valid stored prefix with upgraded, validated, deeply frozen identified messages, without truncating a torn tail, synthesizing recovery closers, or publishing coordinator state. Serialized with same-id writes; the optional signal promptly rejects a queued caller, prevents that queued backend read from starting, and cancels active backend read work. Intended for read models and other observers that must never recover a log.
readFrom(id, fromSeq, signal?): Promise<{ meta; events }>	The read-from-seq primitive: return the header plus the valid stored events with seq >= fromSeq, detached and non-mutating like inspect (no truncation, no closers, no coordinator state). A fromSeq at or past the stored end returns an empty event list; a negative or non-safe-integer fromSeq rejects. Seek-capable backends (SQLite) read only the suffix unless a legacy event in that suffix requires prefix context for normalization; sequential backends (JSONL) parse the whole artifact and skip forward. Intended for checkpoint consumers (e.g. the persisted projection cache) that fold only the tail past a watermark.
list(signal?): Promise<SessionHeader[]>	Lightweight listing from metadata, no full-log parse. The optional signal cancels backend listing work. A zero-event lazily-materialized session is absent from list.
listSnapshots(signal?): Promise<SessionPersistenceSnapshot[]>	Lightweight metadata plus an opaque branded per-log revision, without loading event logs. A revision stays equal while that log and its backing store are unchanged, changes after append or mutating load repair, and cannot collide solely because two stores use the same local counter. The optional signal requests cancellation of backend discovery work; first-party backends settle any started listing work before rejecting so an awaited call is quiescent.

Invariants every backend must honor

• Append-only; a crashed turn is closed, not truncated. Flushed events are never rewritten. A crash can leave an unclosed final turn whose events are real and possibly large; load preserves them and durably appends synthetic closers (a risk-classified error tool/result per unanswered assistant call, then step/end?+turn/end {interrupted}) to balance the log and keep the rehydrated history a valid provider transcript. Only a never-fully-written torn tail fragment is discarded.
• Contiguous seq. load rejects a seq gap/parse error in the MIDDLE of the log; append's first seq must equal the stored next-seq.
• JSON-serializable data. append materializes each direct/replay batch through the shared one-pass lossless-JSON boundary. Live Session events are already deep-frozen, but the write coordinator still copies each event into a persistence-owned buffer.
• Durability. append returns only once the batch is durable.

The write coordinator

PersistenceCoordinator owns per-id state and serialization, one eager write controller per live session, lazy materialization, crash-tail repair, session adoption, and quiescent disposal. A first-party backend composes one, implements the small PersistenceBackend storage hook interface, and delegates its stateful methods. JSONL and SQLite therefore share lifecycle correctness while retaining different storage primitives; see the coordinator Agent Note and flush-controller simplification.

Each session/event copies its event into the session controller and starts an eager drain without blocking the producer. Concurrent notifications share the current drain; events admitted during a write remain pending and trigger the next batch. session/flush is an observation barrier that waits until the controller has no current or pending batch. An eager failure is logged and retains the batch; the next explicit flush or backend teardown retries it and surfaces failure to its caller.

Crash repair is cold-only. For a live id, load(id) snapshots the authoritative in-memory log, waits for that snapshot to become durable, and returns it with the coordinator's stored header only when balanced; an open live turn rejects instead of receiving synthetic interruption closers. A cold load reserves its id across backend reads and repair writes, so concurrent publication of a same-id live Session rejects and rolls back. HMR adoption reads through loadStored, applies the coordinator's cwd check, and never closes the active turn.

Backend reads normalize the exact supported same-version shapes before current-shape validation. Pre-identity messages receive the deterministic id legacy-message:<session-id>:<event-seq>; a tool-result content replacement inherits its target's imported id. A pre-react-loop turn/start loses its obsolete trigger, a removed steering/message becomes the same identified user/message, and an older turn/end maps its terminal reason without inventing unavailable cancellation provenance. The coordinator uses the same normalized view for load, inspect, readFrom, ownerless-state claims, and HMR prefix adoption. Storage remains append-only: reads do not rewrite old records, and later appends use the current shape. These are narrow import exceptions from the pre-identity message and pre-react-loop session decisions, not a general v0 migration promise.

When a live session emits session/disposed, the coordinator waits for its controller, serializes a final drain, then releases state owned by that exact Session object. Failed retirement leaves the controller in the live-session map, so backend teardown can retry it. Backend teardown stops event admission first, flushes every remaining controller, awaits per-id operations, and only then closes the storage handle.

The side-effect-free locate and lightweight listSnapshots queries remain backend-owned because they describe storage topology and revision identity rather than write orchestration. listSnapshots(signal?) passes the caller's exact signal into backend discovery so observers can cancel that work without detaching it.

The PersistenceBackend<TornMarker> hooks (the only seam between the coordinator and storage):

Hook	Role
name	Backend label for the dispose-failure AggregateError.
loadStored(id, signal?)	Read a stored prefix by id across every storage scope. Used by resume/load, non-mutating inspect, live adoption, and the create-collision probe. The optional signal belongs to observation-only reads. Returned metadata identifies id; an opaque tornMarker is present iff a torn tail must be truncated.
loadStoredFrom?(id, fromSeq, signal?)	Optional seek-capable suffix read behind the service's readFrom: the header plus stored events with seq >= fromSeq, non-mutating, no torn marker. SQLite implements it (WHERE seq >= ?); a backend that omits it gets the coordinator's fallback — loadStored plus a forward skip.
appendBatch(meta, events, isMaterialized)	Durably append a contiguous batch, lazily materializing ATOMICALLY when not yet materialized.
commitRepair(meta, tornMarker, closers)	Make a crash repair durable: truncate the torn tail (iff tornMarker !== undefined — a marker may be falsy, e.g. seq/offset 0) and append closers. NOT required to be atomic. Used by load (truncate + closers) and live-adoption (truncate only).
list(signal?)	List all stored metadata, observing optional cancellation.
close?()	Optional lifecycle teardown (e.g. close a db handle), awaited after the dispose drain.

The coordinator asserts the stored id and compares stored/live cwd before repair or live adoption. Its inspect() path validates and clones the prefix without calling commitRepair or publishing write state. The tornMarker is fully OPAQUE: the coordinator only tests !== undefined and round-trips it to commitRepair, never inspecting its value (the JSONL backend uses the byte offset to truncate to, the SQLite backend the seq to delete from). A third-party backend MAY implement the abstract service directly without the coordinator, but it must provide the same non-mutating inspection and trustworthy lightweight snapshot revisions. See the write-coordinator Agent Note.

Metadata and location types

Re-exported from dsh-session: SessionHeader (immutable session metadata: version, id, createdAt, cwd?, parentSession?, seedLength?, origin?, delegationDepth?). SessionLocation is { readonly kind: string; readonly path: string }; its path is an absolute backend target, not proof that the artifact exists or contains an unflushed turn.

Model Experience

Resumed conversation history

What the model sees

This seam adds no prompt or schema. Resume restores stored surface events as message history; stored request headers reconstruct earlier calls, while the new loop composes the current system prompt, tools, and session prefix for its next request. Crash repair marks an assistant request without a durable call as TOOL_NOT_STARTED; a durable call without a result becomes TOOL_OUTCOME_UNKNOWN, whose text lets the model retry read-only or idempotent work but directs it to verify side effects or ask the user instead of retrying blindly.

Token effect

Zero tokens during ordinary persistence. Resume restores retained history cost and pays the current request envelope normally; each repaired call adds the quoted retained error text.

KV Cache effect

Persistence does not mutate live request prefixes. A resumed loop can reuse provider cache only when its reconstructed history, current envelope, and model route match; crash-repair results append without rewriting earlier history.

Known Limitations and Deferred Work

• No deletion or retention surface — pruning stored sessions is out-of-band backend maintenance.
• list() is unpaginated and unfiltered — it returns every stored session's header; fine for local stores, unindexed at scale.
• Repair-time synthetic closers are the only crash story — a backend must synthesize tool/result/step/end/turn/end closers on load; there is no partial-turn resume that continues an interrupted turn instead of closing it.