The compact-basic README's opening line still described summarization
as routed through the agent request pipeline — corrected to the direct
one-shot llm/stream contract the rest of the package documents. The
proposed Code Mode RFC gets a premise-stale note: it names the old
full-request agent/request seam, which now shapes call config only —
re-map onto the log channels and system-prompt/assemble before
implementing from it.
Codex diff review, round 1, two (A) findings:
- The agent/request fallback resolved the RAW seed object — on later
steps the session's cached header fold — so a delegating listener
(await next(), mutate, return) could rewrite the fold in place and
the change would compare as already-baseline: no delta logged, the
persisted log unable to reconstruct the request (the dev invariant
would fire on the divergence, but the log would still lie). One
structuredClone'd, deep-frozen seed now serves both the listener
chain and the fallback — in-place shaping after delegation throws —
and Session.requestHeader() freezes its fold on update, so the leak
class is unrepresentable from either side. Pinned by a loop-level
delegating-mutator test.
- Doc sweep for the old contract: agent README's event row (mutate
GenerateOptions / tool filtering → frozen config seed, replacement
out, logged header), compact-basic's module JSDoc (summarize routed
through agent/request → direct one-shot at llm/stream), and
architecture.md's event-domain line (request mutation → call-config
shaping).
surface.replaceGeneration gets its direct-access test (the getter folds
a pending delta itself, not only via a nodes read); the reconstruction
theorem drives maxTokens and stop through the config waterfall; the
compact-basic envelope drops a dead conditional — config.maxTokens is
required and validated positive, so this backend's envelope always
carries the cap (the return type's optionality exists for overriding
subclasses).
compact/summary gains { model, maxTokens? } — the envelope the
summarize call actually used, reported by the backend that made the
call: summarize() now returns { summary, model, maxTokens? } instead of
bare blocks, so an overriding backend (template or remote summarizer)
reports its own envelope honestly and compactRegion logs it. 'Which
model wrote this summary' becomes answerable from the log alone, and
the one-shot summarize request — outside the loop's header-event fold
by design — is reconstructable from log + code (the reconstructability
RFC's scope statement).
The summarization request no longer dispatches agent/request — that seam
shapes the loop's conversation requests; a hand-built one-shot's
interception surface is llm/stream, like every other direct call. The
model comes from summarizationModel falling back to the agent's own. The
turn/step parameters existed on the compact seam only to feed the
agent/request dispatch and leave compactIfNeeded/compactRegion.
Groundwork for making every conversation request a pure function of the
session log (reconstructability RFC, this branch): the seam split keeps
the loop's request path as the single thing the log must explain.
Ported from worktree-prompt-prefix-stability (PR #162) where it was
review-converged; catalog and producer/consumer graphs regenerated.
The convergence pass found four summary-level sites still describing
the estimator as fixed char/4: packages/README.md (twice), the compact
group and interface READMEs, and compact-basic's package.json
description. All now say chars-per-token with the charsPerToken
default, matching the authoritative package README/module doc/RFC.
A Codex review pass on the draft caught four real gaps and two solid
suggestions; all addressed except one pushed back on the merits:
- hooks-claude/hooks-codex: stderrSummaryMaxChars was the one new knob
with NO range validation — a negative/NaN cap would silently
misbehave inside slice(). Both bridges now assert a positive integer
at the TOP of apply() (before the config-file parse's early return,
so a bad value fails the load loudly), with rejection tests.
- tool-fs: the read caps count lines/chars/bytes, so positive-FINITE
was too loose (a fractional readLimit would flow into windowing
arithmetic and the schema description). All four now require a
positive integer, matching tool-web's cap.
- Doc drift the gates cannot catch: tool-web's README tools table
still named WEB_SEARCH_MAX_RESULTS as the mechanism; compact-basic's
README/module doc and the compaction-capability-seam RFC still
described estimation as fixed char/4 rather than the charsPerToken
default.
- subagent-acp: the dispose graces were tested only at the
startAcpRun level, so a regression that stopped threading plugin
config into AcpRunSpec would have survived. A provider-path test now
drives the trap-escalation scenario through ctx.subagents.start with
small config graces and bounds dispose at 4s.
Pushed back on: converting compact-basic's charsPerToken to a
schemastery field. The package's whole config is deliberately
hand-rolled (resolveConfig, every threshold REQUIRED with no default —
a documented design posture); one schemastery field beside it would be
incoherent. The knob is cordis.yml-reachable, defaulted, and validated,
which is what the convention requires; migrating the package to
schemastery wholesale is pre-existing config-surface hygiene out of
this change's scope.
The audit swept every packages/*/* plugin for the new AGENTS.md
convention (no hardcoded tunables in plugins) and exposes each finding
as a defaulted, validated Config field. Defaults are the previously
hardcoded values throughout, so no deployment or golden changes.
- tool-fs (had NO Config): readLimit, readMaxLineLength, readMaxBytes,
readStreamMinSize. The caps thread through ReadToolCaps/ReadWindow —
read-render already documented that the consumer applies the caps, so
they become explicit per-request fields.
- tool-web: searchMaxResults (WEB_SEARCH_MAX_RESULTS stays as the
schemastery default). Also fixes the stale GREP_LIMIT references in
search.ts and the web-capability-seam RFC (no such constant exists).
- bash-local: graceMs (SIGTERM->SIGKILL escalation grace). The
RunInternals.graceMs test seam is gone: graceMs is now a required
SpawnSpec field filled from config, so tests exercise the real
config path and the defaults live in exactly one place.
- subagent-acp: disposeEofGraceMs / disposeGraceMs. The AcpRunSpec
fields become required for the same one-defaulting-layer reason.
- session-persistence-sqlite: journalMode ('wal' default; the
rollback-journal modes serve filesystems where WAL's shared-memory
files do not work, e.g. network mounts).
- hooks-claude + hooks-codex: stderrSummaryMaxChars for the persisted
hook/result stderr summary. The duplicated summarize() helpers merge
into hook-protocol's summarizeStderr(stderr, maxChars), beside the
HookResultRecord field it feeds, with the bound parameterized the
same way runHook's defaultTimeoutMs already is.
- compact-basic: charsPerToken for the token estimator (default 4, the
English-text heuristic; CJK-heavy deployments need ~1-2 or compaction
fires far too late). Also corrects the BasicCompactService class doc,
which claimed defaults the required-field config never had.
- fs-local: deletes the dead STREAM_MIN_SIZE constant and the dead
FsIoInternals.streamMinSize seam — the read-routing bound lives in
the consumer (tool-fs), where it is now config. This is item 1 of
the proposed prune-write-only-fs-surface RFC, annotated accordingly.
Every new field gets range validation (following the existing
assertPositiveFinite pattern), a README row, and tests covering the
configured behavior, the schema default, and load-time rejection.
ImageBlock had no production producer and every consumer dropped it:
the deepseek serializer skipped it, the pi-ai converter skipped it as
unrepresentable, the ACP bridge neither advertises image prompt
capability nor forwards image blocks, and compact-basic charged a flat
85-token estimate and rendered an [image] placeholder. A block
constructed today would silently vanish from the wire — the vocabulary
advertised a capability no path honors, the silent-data-loss shape the
defensive patterns warn against. The only constructors were tests
pinning the skip/estimate branches.
Remove ImageBlock and its ContentBlockMap entry (its cache?: CacheHint
field leaves with it; CacheHint itself and the other two cache? fields
are out of scope). compact-basic loses its explicit image estimate and
placeholder arms (the merge-extensible default arms absorb the case);
the deepseek serializer, pi-ai converter, and ACP codec already handled
image in their default arms, so only their image-naming comments
change. The codec's inbound rejection of ACP-protocol image prompt
content stays — that guards wire content a client can send regardless
of our vocabulary.
Tests that constructed harness image blocks to pin the removed branches
are dropped (the 85-token estimate pin) or retargeted onto plugin-added
block types / other non-text blocks, which the surviving default arms
own. Docs, the type-equiv pastes, and the content-block vocabulary
RFC's block list and multimodal-home consequence are updated in the
same change; the RFC moves to implemented/ and the index is
regenerated. A real multimodal feature reintroduces image via
declaration merging together with the adapter mapping, ACP
advertisement, and compaction pricing that honor it.
Address @tianyicui's minor-revision review on PR #110:
- Make every BasicCompactConfig knob required except `auto` (defaults
true): there is no data yet to justify default thresholds/budgets, so
a consumer states each value explicitly. Drop the DEFAULTS export and
the constructor's `= {}` default; example cordis.yml, the compaction
e2e, the README, and every test construction site now pass a complete
config (tests route through a `cfg()` helper).
- Add a TODO on estimateContentTokens: char/4 is coarse; replace with a
real tokenizer or post-response usage feedback in a follow-up.
- Add a TODO on the agent/pre-step `fullSystemPrompt` param flagging it
as a smell on a generic per-step seam (compaction is its sole
consumer); a `//` line comment so it stays out of the generated catalog.
Use maxTokens as the provider generation cap and remove the confusing stored-summary max config.
Strip reasoning blocks before storing compaction summaries, reject non-shrinking summaries, and retry bounded re-compaction when the surface remains over threshold.
Add config validation for numeric and type-shaped knobs plus unit and real-API e2e coverage for reasoning-capable summarization.
Honor cancellation and disposal around async pre-step setup before the loop can open a step or call the model.
Route compaction summarization through agent/request so router agents can select the model, and remove the stale model argument from agent/pre-step.
Document serial events and the approximate convergence bound, regenerate the Cordis catalog, and add regression coverage for router compaction, HMR cleanup, and assembly/pre-step interruption.
Manual review round, two non-blocking findings:
- CBR-005: _extractText's JSDoc claimed it "walks events in log order",
but it walks the seqs in surface order (the inline comment already said
so) — the exact distinction CBR-001 paid for, since after a replace a
high-seq checkpoint heads the surface before lower-seq retained nodes.
Corrected the JSDoc to match.
- CBR-006: the "HMR safety" suite only asserted registration; the actual
dispose-and-confirm-cleanup test lived under "llm inject", so a reader
searching by name could miss it. Added a disposal test to the HMR-safety
suite (mount via the real plugin fiber with LlmService present so inject
resolves, dispose, assert ctx.get('compact') is undefined) and reframed
the llm-inject test's trailing teardown to point at it.
Codex round 1 CBR-003: several docs still described compaction as an
`agent/request` waterfall concern, and the implemented compaction RFC
claimed "No changes to dsh-session or dsh-invariants" while the diff
changed both.
- Package READMEs / JSDoc (agent, agent-loop, system-prompt, compact,
compact-basic): compaction now lives on the serial `agent/pre-step`
seam (fired after turn/start, before step/start); the structural guard
is tool-pairing balance (`isToolPairingBalanced`), not step-alignment;
the convergence bound is strict (`>=` rejects).
- architecture.md / core-data-structures/compaction.md: same seam +
predicate + dispatch-mode updates; regenerated cordis catalog.
- Implemented compaction RFC, updated in place to describe shipped
reality: the seam is `agent/pre-step` (@mode serial) fired before
step/start; alignment is surface tool-pairing balance; the convergence
invariant rejects `>=`; and the "no dsh-session/dsh-invariants changes"
claim is corrected — dsh-session gains the tool-pairing predicate and
dsh-invariants drops its `start <= end` replace assertion (a positional
replace makes start > end normal).
Codex round 1 CBR-002: `resolveConfig` rejected only
`summarizationMaxTokens + retainTokens > threshold` (allowing equality),
but `compactIfNeeded` declines only when the estimate is `< threshold`.
At exact equality the post-compaction history sits at the threshold and
re-triggers on the very next check.
Make the bound strict (`>=` rejects), so post-compaction history is
guaranteed strictly below the threshold. Updated the boundary test (the
sum-equals-threshold case is now rejected, not accepted) and added an
"accepts just below the threshold" case; nudged one unrelated config that
incidentally sat at the equality boundary.
Codex round 1 CBR-001: a head-anchored compaction checkpoint was
mis-classified by the log-position step-alignment scan, so a second
auto-compaction over a checkpoint-headed surface silently failed.
Root cause: `isStepAlignedStart/End` scanned the LOG by seq, but a
`replace` op lands a checkpoint at a high log seq whose SURFACE position
is the head — its log neighbours (the open step's assistant/message) are
not its surface neighbours, so the forward scan wrongly reported mid-step.
Fix, per the agreed direction:
- Replace the two log-position predicates with one surface-anchored
helper `isToolPairingBalanced(nodes, events, beforeSeq)` in
`dsh-session` (renamed step-boundary.ts → tool-pairing.ts). A cut is
balanced when no unanswered tool-call precedes it on the surface; a
region is collapsible iff both edges are balanced cuts. The open-tail
and free-node cases fall out of the same counter. It also throws on a
corrupt surface (a tool/result with no matching call).
- Move compaction off the in-step seam to a new "pre-step" seam fired
after turn/start and before step/start, so a compaction's log-only
compact/* records and its replacement node land cleanly OUTSIDE any
step (the honest structure crash-safety relies on). Renamed the event
agent/pre-request → agent/pre-step and switched its dispatch from
parallel → serial (listeners mutate the surface as a side effect;
serial isolates them so concurrent appends can't interleave). Extended
the catalog generator to accept @mode serial.
Regression coverage: a real-loop test driving an auto-compaction asserts
the landed checkpoint is a balanced cut on both sides; unit tests pin the
checkpoint case, the mid-step injection case, multi-call steps, and the
corrupt-surface guard. Proven red on the old log-position logic.
Reform the compaction blueprint so a runaway turn survives and the design
stops drifting across review rounds:
- Drop in-flight-turn protection ("layer 2"). Retention is a uniform tail→head
whole-unit walk; the only structural guard is step-alignment. A single turn
that alone exceeds the window now compacts its own early closed steps instead
of being retained verbatim (the failure mode that motivated this).
- Move auto-compaction off the agent/request waterfall onto a new awaited
agent/pre-request loop seam, fired before history derivation. Compaction
mutates the surface; the loop derives once from the result — no double-derive,
and a listener structurally cannot act on not-yet-derived messages.
- Tighten compactIfNeeded to required (session, system, model, signal).
- Enforce a single-pass convergence invariant in resolveConfig: reject configs
where summarizationMaxTokens + retainTokens exceeds the threshold, so a
compaction can never immediately re-trigger.
- Document the crash vs recoverable failure taxonomy; core session repair stays
compaction-agnostic (a log-only orphaned compact/start is inert).
- Wire dsh-compact-basic into examples/coding-agent and add a with-key
compaction e2e (compaction's first real-world exercise + runaway net).
- Rewrite the RFC to encode the blueprint and move it to implemented/.
The runaway-turn snapshot is a named deferred follow-up: dsh-llm-replay cannot
yet serve the interleaved summarization model call.
Collapses the per-round review churn of the prior compact-basic branch into a
single clean baseline on top of compact-interface, so the upcoming retention
refactor lands as fresh, well-scoped commits rather than stacking on a history
of fixes that are being superseded.
Tianyi Cui
Hypatia May