Restack on the carved-out foundation (#192), per review feedback on #170.
The seam files resolve to the carve-out's revision — its prompt-order
neutrality fix (backends no longer inject 'tools'; the structured runtime
gates its own capture-tool registration) restores the subagent tools to
master's front position, so every recorded fixture is re-recorded on the
stacked tree and the authored error-finish/cancel headers re-patched to the
stacked tool list ([subagent, subagent_fork, workflow, todo_write, ...]).
Carved out of #170 per review feedback — the foundation the workflow tool
builds on, now standing alone on master:
- dsh-tools: the structured-output JSON Schema subset (StructuredOutputSchema,
assertSupportedOutputSchema, validateStructuredValue) — rejects loud outside
the enforced subset, listing every violation
- dsh-subagent: SubagentStartRequest.outputSchema / SubagentResult.structured
become a real capability; the service rejects a schema'd request whose
provider lacks it
- dsh-subagent-inprocess: the shared structured runtime — one global
structured_output capture tool, a prepend final-assembly listener that
strips the placeholder for plain agents and swaps in the run's own schema
(plus the calling instruction as a trailing section) for structured
children, an agent/turn-continuation veto once captured, and the
capture/nudge loop in the run driver (structuredNudgeRetries, cancellation
honored mid-nudge); lifetime refcounted by backends and live runs
- subagent-spawn / subagent-fork flip outputSchema: true
One deliberate divergence from the #170 revision: the backends do NOT add
'tools' to their plugin inject. Doing so deferred their apply past the todo
plugin, and the delegation tool mirrors provider lifecycle — so the
model-visible tool order of every existing prompt changed, invalidating every
recorded snapshot fixture. The runtime now gates its capture-tool registration
on tools availability itself (sync when live, a scoped inject fiber when the
Loader starts the backend first), keeping this PR byte-invisible to existing
transcripts: all 35 snapshot scenarios pass against master's fixtures
unchanged.
Master's reconstructable-requests overhaul (#179) meets the workflow tool:
- subagent-inprocess structured-output nudge becomes a system-prompt section
plus logged context (the injected-request waterfall shape is gone upstream)
- snapshot fixtures re-recorded on the merged tree so every request/header
carries the workflow tool; authored error-finish/cancel headers patched to
the merged tool list and system text
- architecture.md condensed back under its word ceiling; module graph regenerated
The review's critical finding: the engine README and the tool
description read as if the missing filesystem/network/Node globals were
enforced, but a script can reach the host Function constructor via
globalThis.constructor.constructor and from it process and every Node
builtin. Per the trust premise this is ACCEPTED (model-written scripts,
bash-equivalent trust; genuine sandboxing is the deferred engine swap
already listed) — but the docs must say so instead of implying a wall.
The trust-premise sections (engine README, module doc, RFC) now name
the escape and its acceptance; the model-facing tool description says
the APIs are not PROVIDED rather than implying they are prevented.
A cancel that lands AFTER a child's clean turn end but before the nudge
continuation runs clears nothing — child.cancel() only kills queued or
running work — so the loop's turn-state check alone let a later
child.send() spend a fresh post-cancellation turn (and even capture a
structured result the caller had already abandoned). The loop condition
now also reads the run's own cancelled flag, re-evaluated after every
whenIdle(), through an accessor because closure assignments are
invisible to control-flow narrowing (an inline read lints always-true).
readResult keeps the outcome honest on this path: a completed-but-
uncaptured turn maps to aborted, not error, when a cancel is why the
nudging stopped — the cancel contract outranks the schema shortfall.
Three review findings on the engine's child seam, one mechanism each:
- Cancellation now bridges to run.cancel() on every in-flight child, not
just the shared request signal — the subagent seam leaves a provider
free to honor either channel, so the consumer drives both (listener
removed in the child finally).
- A child result REJECTION (an infrastructure fault the seam allows) now
emits the paired workflow/agent-end before propagating, and propagates
as a fatal WorkflowError with the new AGENT_RESULT code — previously it
skipped agent-end (permanently open child for seq-matching observers)
and dissolved to a per-item null inside parallel()/pipeline(), letting
a broken provider read as an ordinary failed child. A rejection landing
after cancel stays a cancellation (cancelled outcome + CANCELLED).
- Every hook now guards its entry with a shared throwIfCancelled():
phase()/log() no longer emit observer events after a script caught an
earlier cancelled rejection, and parallel()/pipeline() refuse entry —
cancellation is the next HOOK boundary, not just the next agent().
The two tests that wait for two children IN FLIGHT (the pipeline
no-barrier test and the dropped-promise cancellation test) inherited the
engine's auto-resolved maxConcurrentAgents, which is
min(16, max(1, cores - 2)) — exactly 1 on the 2-core CI runner, so the
second child never started and vi.waitFor timed out. Reproduced locally
under taskset -c 0,1; the full spec passes there with the helper pinning
a fixed ceiling. Tests about the ceiling itself keep their explicit
overrides, and the auto-resolve arm stays covered by the
default-config provider tests.
The compact-basic README's opening line still described summarization
as routed through the agent request pipeline — corrected to the direct
one-shot llm/stream contract the rest of the package documents. The
proposed Code Mode RFC gets a premise-stale note: it names the old
full-request agent/request seam, which now shapes call config only —
re-map onto the log channels and system-prompt/assemble before
implementing from it.
Codex diff review, round 1, two (A) findings:
- The agent/request fallback resolved the RAW seed object — on later
steps the session's cached header fold — so a delegating listener
(await next(), mutate, return) could rewrite the fold in place and
the change would compare as already-baseline: no delta logged, the
persisted log unable to reconstruct the request (the dev invariant
would fire on the divergence, but the log would still lie). One
structuredClone'd, deep-frozen seed now serves both the listener
chain and the fallback — in-place shaping after delegation throws —
and Session.requestHeader() freezes its fold on update, so the leak
class is unrepresentable from either side. Pinned by a loop-level
delegating-mutator test.
- Doc sweep for the old contract: agent README's event row (mutate
GenerateOptions / tool filtering → frozen config seed, replacement
out, logged header), compact-basic's module JSDoc (summarize routed
through agent/request → direct one-shot at llm/stream), and
architecture.md's event-domain line (request mutation → call-config
shaping).
surface.replaceGeneration gets its direct-access test (the getter folds
a pending delta itself, not only via a nodes read); the reconstruction
theorem drives maxTokens and stop through the config waterfall; the
compact-basic envelope drops a dead conditional — config.maxTokens is
required and validated positive, so this backend's envelope always
carries the cap (the return type's optionality exists for overriding
subclasses).
The reconstructability RFC is the principle's home: model-visible ⟺
logged in both forms, the mechanism (boundary derivation + header
fold), the enforcement (write-time round-trip guard, the dev
invariant), the corollaries ranked (prefix-cache stability first), the
MiniCode lineage with the provenance arrow inverted, and the
alternatives it beat — including the stateful transmission client
whose three-design archaeology lives in PR #162.
Placements per the one-home-per-fact taxonomy: a standing-order line in
root AGENTS.md (with displacement trims to stay inside the 1,575-word
ceiling), the principle statement in architecture.md § Session Log and
its Turn Flow lines (condensed to the ratcheted 1,630 ceiling), the
request-envelope section in core-data-structures/core.md with the
LlmCallConfig paste, both review-requested FIXMEs
(FIXME(call-config-shape) beside the type, FIXME(catalog-verbs) at the
catalog's drift-gate note), cookbook rows redirected off agent/request
(tool filtering → system-prompt/assemble, plan-mode prompt → sections/
inject()), and the llm/stream JSDoc stating the frozen-request
contract. RFC index and all generated catalogs regenerated.
One re-record after the header events landed: recorded scenarios
re-harvested against the live API; the three fs-writing fixtures are
relativized (this recording's model happened to emit absolute
file_path arguments, which only round-trip through replay when the
path is cwd-independent) and, with the two never-re-recorded authored
scenarios (error-finish, cancel), rewritten in the normalized
authored-fixture form the harness documents — each now carrying the
request/header snapshot the loop logs before its first dispatch.
Keyless replay verified green across all 35 scenarios.
compact/summary gains { model, maxTokens? } — the envelope the
summarize call actually used, reported by the backend that made the
call: summarize() now returns { summary, model, maxTokens? } instead of
bare blocks, so an overriding backend (template or remote summarizer)
reports its own envelope honestly and compactRegion logs it. 'Which
model wrote this summary' becomes answerable from the log alone, and
the one-shot summarize request — outside the loop's header-event fold
by design — is reconstructable from log + code (the reconstructability
RFC's scope statement).
Beyond the mechanical conflicts (provider capability lines vs master's new
inheritsParentContext field; generated catalogs regenerated rather than
hand-merged; knip/lockfile), three master-side reworks required semantic
adaptation of this branch:
- The persona rework removed AgentOptions.systemPrompt, which was the
structured-output instruction's channel. The instruction now rides the
SAME final-request enforcement listener that injects the schema'd tool:
appended per request to final.system (per-request wire state, not agent
prompt state). Tests assert the wire request (adapter.requests) instead
of child.options; the bare-direct-dispatch test pins the no-system arm.
- Tool guidance moved out of deployment prompts into per-tool prompt
sections; the examples' workflow paragraph became a tool:<toolName>
section contributed by dsh-tool-workflow (explicit-ask-only policy),
and both example personas resolve to master's minimal identity+behavior
form. tool-workflow gains inject: systemPrompt (+ peer dep, tsconfig
ref); the export-shape guard updated.
- The uniform-RFC-format gate: the dynamic-workflows RFC restructured to
the implemented/ skeleton (bare Status line; Proposal -> Decision;
What-was-rejected -> Alternatives considered; new Consequences), and
the overall-run-timeout deferral is now recorded in the RFC's Deferred
list. The doc-graphs atlas classification gains the workflows seam
(workflow-vm implementation, tool-workflow consumer).
Master's harness-identity section made "empty assembled prompt" states
unreachable through the loop, so the instruction-append is a plain
undefined-ternary and the structured tests assert append-not-replace.
All snapshot goldens (including workflow-run) replay unchanged. Full
local CI-equivalent gate sequence green on the merged tree.
The dev-mode cross-check on llm/stream: a frozen request with a live
sessionId (the loop-built marker; hand-built one-shots stay unfrozen
and skipped) must carry messages deep-equal to the derivation over the
log prefix strictly before the in-flight step's step/start, and header
fields equal to the fold of the log's request/header* events. The
messages side rebuilds through a FRESH Session over the boundary
prefix — same projection code, zero shared state — so the live cache
cannot vouch for itself, and the seq-bounded rebuild is boundary-
correct: content appended after step/start (an agent/request-window
inject) legitimately belongs to the next request and cannot false-fire
the check. prepend:true only defends against the replay adapter's
short-circuit (append-registered); correctness never rests on listener
ordering. There is no divergence-allowance caveat: nothing can shape
request content outside the log.
The loop is now transmission-stateless; a request is a pure function of
(session log, this step's rendered assembly, current AgentOptions):
- The reconstruction boundary is step/start: the messages snapshot is
taken in the same synchronous frame immediately before the step/start
append, so the request's messages are exactly the derivation over
events[0..stepStartSeq) — an inject() from an agent/request listener
(or any concurrent task) lands after the boundary and joins the NEXT
request. This changes behavior for a synchronous step/start
session/event listener that appends content (master derived after the
append, so such a listener could reach the current request):
agent/pre-step is the sanctioned seam for current-request content.
- agent/request is re-typed to config-only: (agent, turn, step,
config: LlmCallConfig, next) → LlmCallConfig. The frozen seed comes
from AgentOptions on a loop instance's first request (explicit options
beat the logged baseline — fork overrides and resume reconfiguration
stay correct) and from the log's folded header afterwards; listeners
return a replacement to switch. Content shaping through the request is
no longer expressible — model-visible content flows through the log
channels.
- recordRequestHeader appends whatever header event the request owes the
log before dispatch: an 'initial'/'resume' snapshot anchoring each
loop instance, a round-trip-verified delta on change, a 'fallback'
snapshot when the encoding cannot express it. Session.requestHeader()
is the log's incrementally-folded baseline.
- Requests are deep-frozen before dispatch (deepFreeze exempts the
AbortSignal — freezing one breaks AbortController.abort() outright);
frozen + sessionId is the loop-built marker the dev invariant keys on.
Ported from #162 and re-anchored on the log: the append-extension /
frozen-end-to-end / compaction-resend / prompt-change property tests,
plus new specs for the boundary semantics, resume anchoring, and the
end-to-end theorem (every recorded request rebuilds byte-equal from the
log alone). Live cache-hit e2e (request-cache.e2e.ts) verified against
the real DeepSeek API. Snapshot goldens intentionally stale until the
single re-record after the compact/summary envelope lands.
deriveMessages() no longer re-derives and re-clones the whole surface
per call: each node is projected exactly once, when first seen (O(new
nodes) per access), through the now-public deriveEventMessage — the
per-node pure function external reconstructors and the dev invariant
fold over a log prefix, so no two paths can disagree about a request's
messages. A surface rewrite (replace, invalidate) rebuilds, signalled
by SurfaceManager.replaceGeneration — monotonic, bumped by every folded
replace and by invalidate(), never reset. Callers get a fresh array
snapshot per call over SHARED deep-frozen messages: isolation of the
append-only log moves from per-call cloning to unrepresentability
(mutation throws), which the two isolation tests now pin directly.
Every conversation request's non-content half (system prompt, tool
schemas, call config — the EpochHeader) is now recorded in the session
log: a 'request/header' full snapshot (reason 'initial' | 'resume' |
'fallback') anchors the fold at conversation birth and process
boundaries, and 'request/header-delta' events (system line-trim,
name-keyed tools delta, whole config) encode mid-run changes. The pure
trio — foldRequestHeader / diffHeader / applyHeaderDelta — reconstructs
the header any request was built under from the log alone; the writer
contract round-trip-verifies every delta with a 'fallback' snapshot
when the encoding cannot express a change (pure tool reordering), so a
well-formed log always folds cleanly. Canonical absence: empty system
and empty tools normalize to absent fields, matching request builds.
Persistence and cordis catalogs regenerated; SessionEventMap paste and
EpochHeader added to the core-data-structures session page.
The call configuration (model + sampling scalars) becomes named
vocabulary: per-conversation state that the session log records as part
of the request header (the reconstructability RFC on this branch), with
callConfigEquals as the real-change detector behind logged header deltas
and deepFreeze as the ownership helper the loop applies to every built
request. dsh-llm stays stateless — request in, chunks out; no
conversation object lives here.
The summarization request no longer dispatches agent/request — that seam
shapes the loop's conversation requests; a hand-built one-shot's
interception surface is llm/stream, like every other direct call. The
model comes from summarizationModel falling back to the agent's own. The
turn/step parameters existed on the compact seam only to feed the
agent/request dispatch and leave compactIfNeeded/compactRegion.
Groundwork for making every conversation request a pure function of the
session log (reconstructability RFC, this branch): the seam split keeps
the loop's request path as the single thing the log must explain.
Ported from worktree-prompt-prefix-stability (PR #162) where it was
review-converged; catalog and producer/consumer graphs regenerated.
The per-file branch gate caught it: drive()'s finally always cancels
first, and every first cancel() arms the abandon timer, so the
`!== undefined` guard's false arm was unreachable. clearTimeout
tolerates undefined by contract — call it unguarded.
Codex convergence round 1 on the review-response commits: agent()'s
post-acquire window was real and unguarded. `await acquireSlot()` yields
at least one microtask tick even when a slot is free (and a queued
waiter resumes a tick after its release), so a cancel() landing in that
tick let the continuation start a child carrying an ALREADY-aborted
signal — the in-code comment claimed the window could not exist. A
provider that subscribes only to future abort events (the test stub;
the seam does not promise pre-aborted-signal handling) would never
settle such a child, leaking it until the dispose grace abandoned the
run, and a backend that misses the pre-aborted signal would burn a full
model turn after the user cancelled.
agent() now re-checks isCancelled() immediately after the acquire
(inside the slot-owning try, so the finally still releases), making
every post-cancel path reject before subagents.start. New deterministic
regression: cancel() in the same synchronous frame as start() lands in
the free-slot await tick — the run settles cancelled with ZERO children
started (previously: one leaked child and a grace-delayed settle). The
raced-release test's comment now states what it actually pins (the
queued-waiter rejection path). Also aligns the RFC's auto-concurrency
formula with the code (min(16, max(1, availableParallelism() - 2))).
Tianyi Cui
Wenlu Wang
Hypatia May