Commit Graph
985 Commits
Author SHA1 Message Date
Ziya ac64a0c7b1 docs(rfc): specify stub input layering; add amortized stub drafting follow-up 2026-07-07 02:58:56 -07:00
Ziya 0056238bbd docs(rfc): propose recallable compaction — split checkpoints and in-session history recall 2026-07-07 01:01:24 -07:00
Tianyi Cui dc5656fab3 Merge pull request #194 from deepseek-harness/worktree-snapshot-pin-request-header
Snapshot tests: pin request-header content in one scenario, tokenize the rest
2026-07-07 09:18:58 +08:00
Tianyi Cui 17f89642ae Merge branch 'master' into worktree-snapshot-pin-request-header 2026-07-07 09:16:28 +08:00
Tianyi Cui 09dbc06079 Merge pull request #193 from deepseek-harness/worktree-hooks-dispose-quiescence
fix(hooks): drain detached hook runs on bridge dispose
2026-07-07 09:15:31 +08:00
Tianyi Cui 2d3f75c77b Merge branch 'master' into worktree-hooks-dispose-quiescence 2026-07-07 09:13:00 +08:00
Tianyi Cui 25fc167165 Merge branch 'master' into worktree-snapshot-pin-request-header 2026-07-07 09:09:27 +08:00
Tianyi Cui 515d04339b Address review round 2: preserve delta insert arity; reject unpinned header-deltas
Residuals from the Codex re-review:

1. A system delta's insert was flattened to one token, so deltas differing
   only in inserted-line count compared equal. Now one {{system}} token per
   inserted line — position AND extent survive, content does not.

2. The live uniformity guard folded only request/header snapshots, so a
   mid-run header CHANGE (request/header-delta) could diverge from the pin
   invisibly. Non-pinning runs now assert zero header-delta events: a
   scenario that legitimately changes its header mid-run exists to show
   that change, so it must pin (fail-loud until it does).
2026-07-07 01:26:25 +08:00
Tianyi Cui a0d8f33b29 Address review: structure-preserving delta scrub, live header-uniformity guard, RFC style
Codex review findings on the pinned-header change:

1. scrubRequestHeaders flattened a request/header-delta's whole
   system/tools payload to one token, so two meaningfully different deltas
   compared equal. Now the structural facts survive — keepStart/keepEnd
   line positions, added/removed/changed tool NAMES — and only the bulk
   (inserted prompt lines, schema bodies) is tokenized.

2. The one-pin design rested on an unasserted premise (all sessions
   compose the same header). Every non-pinning scenario now asserts, live,
   that each request/header its run produces equals the pinned fixture's
   header (both sides normalized against their own volatile values), so a
   session-dependent header fails loud until it gets its own pin.
   Verified the guard bites: perturbing the pinned fixture's prompt fails
   a non-pinned scenario with the intended message.

3. RFC de-slopped per docs/AGENTS.md: no PR reference, no SHOULD
   spec-speak; Decision/Verification/Consequences updated for 1 and 2.
2026-07-07 00:51:55 +08:00
Tianyi Cui 81cbc3daa4 Merge pull request #191 from deepseek-harness/catalog-flatten
docs: flatten single-file catalog dirs to docs/tool-catalog.md and docs/persistence-catalog.md
2026-07-07 00:30:36 +08:00
Tianyi Cui 0c568201c9 Merge remote-tracking branch 'origin/master' into catalog-flatten 2026-07-07 00:29:12 +08:00
Tianyi Cui 215ff9a4e6 Merge pull request #188 from deepseek-harness/config-catalog
feat: generated plugin config catalog (docs/config-catalog.md)
2026-07-07 00:27:59 +08:00
Tianyi Cui d45e470192 fix: retarget the i18n pairing exclusions to the flattened catalog paths
The manifest excluded docs/tool-catalog/ and docs/persistence-catalog/ as
directories; the trailing slash is a deliberate path boundary, so the
flattened single files no longer matched and lost their generated-doc
classification — the gate would have accepted a hand-added .zh.md twin it
must reject. The entries now name the files.
2026-07-07 00:08:55 +08:00
Tianyi Cui 4e2da27201 Merge branch 'config-catalog' into catalog-flatten 2026-07-07 00:06:19 +08:00
Tianyi Cui a4152705ef fix review finding: abort tests must prove the hook process is gone
Codex review of the draft: the abort-on-dispose tests asserted only that
dispose() resolved and no warning fired — a regression back to untracked
fire-and-forget runs would pass both. The hook script now records its
shell PID before sleeping, and after dispose the test asserts
process.kill(pid, 0) throws: the drain resolves only after the killed
run settled (the child reaped), so ESRCH-by-dispose-return is
deterministic under the correct implementation, while an alive or
unreaped process fails it. Verified the discriminator by gutting drain()
locally: exactly these two tests fail.
2026-07-07 00:02:10 +08:00
Tianyi Cui f0ef280848 fix: identity-check name resolutions across the pasted closure; exclude the catalog from i18n pairing
Review findings on the config catalog:

The transitive-paste closure deduplicated by bare name, so two DIFFERENT
package-local declarations sharing a name would silently collapse to the
first one pasted — and a reference satisfied by the wrong declaration is a
wrong catalog, not just a thin one. Every resolution is now identity-checked
by source pointer: reaching the same declaration again is benign, while a
name that resolves to two distinct declarations, to conflicting imports, or
to a declaration in one file and an import in another is a violation — a
verbatim fence has a single flat namespace, so the fix is a rename at the
source. A spec case pins the two-declarations collision.

docs/config-catalog.md also joins the translation-pairing exclusions: it is
a generated English-only doc, and without the entry the gate would accept a
hand-added .zh.md twin it must reject.
2026-07-07 00:00:34 +08:00
Tianyi Cui 1fd9e2a9ab Pin request-header content in one snapshot scenario
Every session.jsonl fixture embedded the full composed system prompt and
complete tool-schema list in its request/header event (~8 KB on one line,
identical across the suite), so any prompt or tool-schema edit forced a
re-record or hand-edit of every fixture — see the dynamic-workflows PR for
the churn pattern this removes.

Now exactly one scenario (text-turn, flagged pinsHeader) commits and
compares that content verbatim; every other fixture stores and compares it
as {{system}}/{{tools}} tokens via the new pure scrubRequestHeaders
normalizer (applied to both compare sides and to record-mode writes, so a
re-record cannot reintroduce the content). request/header-delta payloads
are scrubbed the same way; config/reason stay verbatim — a model swap
SHOULD churn every fixture, a prompt edit should not. Replay is unaffected:
script derivation reads only assistant/chunk events.

Fixture meta-guards enforce the split: non-pinning fixtures must be fixed
points of the scrub, the pinning fixture must not be, and exactly one
scenario pins. Committed fixtures migrated through the same function.

Docs: pinned-header RFC (implemented/testing), base snapshot RFC + testing
policy + llm-replay module doc/README updated.
2026-07-06 23:41:20 +08:00
Tianyi Cui 82dfc62620 Merge branch 'config-catalog' into catalog-flatten 2026-07-06 23:38:40 +08:00
Tianyi Cui fe815c9586 Merge remote-tracking branch 'origin/master' into config-catalog
# Conflicts:
#	docs/rfc/INDEX.md
2026-07-06 23:31:58 +08:00
Tianyi Cui 9ca31ab193 fix(hooks): drain detached hook runs on bridge dispose
The emit-shaped hook points (SessionStart, SubagentStart, SubagentStop)
run fire-and-forget: no seam awaits the run chain, so disposing a bridge
could strand a live hook process and let a late continuation inject into
a disposed context. The floating continuation also made the coverage
gate racy: the only coverage of the SubagentStart continuation's
no-context branch arm rode on an un-awaited .then, and on a loaded CI
runner the fork's per-file coverage snapshot beat it — master run
28798191671 failed the 100% branch gate on hooks-claude/src/index.ts at
99.03% (uncovered line 336) with the identical tree passing the PR run
three minutes earlier.

New shared primitive createDetachedRuns() in dsh-hook-protocol: a bridge
tracks each detached run chain, passes the tracker's abort signal to
runHook, and registers drain() as its effect disposer — drain aborts
still-running hook processes (a kill via the bash seam, not a wait out
to the 10-minute default hook timeout), then resolves once every chain
has settled. fiber.dispose() resolving now means the bridge's detached
work is quiescent (docs/defensive-patterns.md).

The subagent marker test disposes the bridge as its sync point, so the
formerly racy branch arm is executed deterministically before the file's
coverage snapshot; new tests pin abort-on-dispose promptness for both
bridges and the tracker's settle/drain contract in hook-protocol.
2026-07-06 23:27:54 +08:00
Tianyi Cui 6d9a4f1258 Merge branch 'config-catalog' into catalog-flatten
# Conflicts:
#	docs/config-catalog.md
#	scripts/gen-config-catalog.ts
2026-07-06 23:16:29 +08:00
Tianyi Cui fb3f3a69ec fix: extend the schema cross-check to nested key paths; scope the catalog framing
Review findings on the config catalog:

The schema-subset check compared only top-level z.object keys against
top-level type members, so a nested loader-accepted key (agents[].id,
agentOptions.model, capabilities.*) missing from the declared type would
pass the gate unseen. The walk now collects nested object/array
compositions as key paths and resolves each against the declared config
type — through interfaces (heritage included), aliases, literals,
intersections, unions, arrays, indexed access, Partial-style wrappers, and
type references across package-local and workspace imports (re-export
chains included). The check stays presence-only and one-directional, and
only a definite miss is a violation: a path crossing a type the walk
cannot enumerate (an external package's) is skipped, never mis-reported.
The recursion guard applies at named declarations only — a structural
first child shares its span start with its parent, so a span-keyed guard
on every node mistakes ordinary descent for a cycle and silently turns
definite misses into unknowns.

The page and RFC framing also overstated the catalog as the exact
cordis.yml-settable surface: the paste is the plugin's full declared
config type, and a field the runtime schema deliberately excludes (the ACP
bridge's test-injected stream) is a runtime-only seam its own JSDoc marks.
Both now say so.

Five spec cases pin the new behavior: nested hidden key, workspace
intersection via star re-export, Partial wrapper, indexed-access
composition, and the external-type unknown path.
2026-07-06 23:06:03 +08:00
Tianyi Cui 116e2c45a8 docs: flatten single-file catalog dirs to docs/tool-catalog.md and docs/persistence-catalog.md
A directory holding exactly one file adds a path level for nothing;
cordis-catalog/ keeps its folder because it holds two sibling pages
(events.md + services.md), matching config-catalog.md which was born flat.

docs/tool-catalog/tools.md -> docs/tool-catalog.md and
docs/persistence-catalog/log-events.md -> docs/persistence-catalog.md; their
generators' OUT paths and in-page relative links drop one directory level,
and every citation repo-wide (docs, RFCs, package READMEs, generator
headers) is updated. graph-atlas.md, config-catalog.md, and the doc graphs
regenerate with the new paths.
2026-07-06 22:26:06 +08:00
Tianyi Cui 0606cd559c Merge pull request #175 from deepseek-harness/codex/parallelize-ci
ci: parallelize GitHub workflows
2026-07-06 22:15:13 +08:00
Tianyi Cui 8458530298 Merge branch 'master' into codex/parallelize-ci 2026-07-06 22:12:44 +08:00
Wenlu Wang 654d192c47 Merge pull request #182 from deepseek-harness/codex/gate-cadence-docs
docs: clarify local gate cadence
2026-07-06 22:10:19 +08:00
Tianyi Cui b0c7eadd23 feat: generated plugin config catalog (docs/config-catalog.md)
scripts/gen-config-catalog.ts walks every packages/<group>/<pkg> entry with
the TypeScript compiler API and emits docs/config-catalog.md: per loadable
plugin, the verbatim config declaration (JSDoc included) its apply/constructor
receives in a ts config-catalog fence, the inject requirements, resolved links
for every referenced type (package-local types pasted transitively, other
plugins' config types as intra-page anchors, LINK_MAP names to
core-data-structures, workspace types to source), and terse classification
lists for config-free plugins, abstract seams, and libraries — classification
is total, so a new package cannot go undocumented.

The walk enforces per-field JSDoc prose on every pasted declaration and
statically cross-checks the schemastery schema (z.object keys, z.intersect
composition across packages): every schema-validated key must be a declared
member of the config type. One violation existed repo-wide — the agents[].id
field in dsh-agent-loop — fixed by adding its JSDoc (which shifts the
cordis-catalog services page's source pointers; regenerated).

verify-config-catalog (--check) joins doc-sync; doc-typecheck learns the
ts config-catalog fence; gen-cordis-catalog exports its JSDoc/pointer helpers
and LINK_MAP for reuse. Negative-path spec in
packages/core/agent-core/tests/gen-config-catalog.spec.ts mirrors the
gen-cordis-catalog spec. Decision record:
docs/rfc/implemented/process/2026-07-06-generated-config-catalog.md (includes
the deliberate acceptance of README ## Config overlap).
2026-07-06 21:57:17 +08:00
Tianyi Cui 9df79534a6 Merge branch 'master' into codex/gate-cadence-docs 2026-07-06 21:55:48 +08:00
Tianyi Cui c57c7527a5 Merge branch 'master' into codex/parallelize-ci 2026-07-06 21:54:07 +08:00
Tianyi Cui 1c999804d8 Merge pull request #184 from deepseek-harness/codex/clarify-doc-budget-guidance
Clarify doc budget guidance
2026-07-06 20:46:31 +08:00
kingwl e48a344f6a docs: keep gate cadence guidance agent-only 2026-07-06 17:08:50 +08:00
Hypatia May 41e1955ace Clarify condensing in doc budget guidance 2026-07-06 16:31:18 +08:00
Hypatia May 72166dad03 Clarify doc budget guidance 2026-07-06 16:11:56 +08:00
kingwl e1b85da182 docs: trim daily command wording 2026-07-06 13:45:17 +08:00
kingwl 7af8f98080 docs: clarify local gate cadence 2026-07-06 13:45:17 +08:00
Tianyi Cui cbc9c26366 Merge pull request #179 from deepseek-harness/worktree-reconstructable-requests
Make every LLM request a pure function of the session log
2026-07-06 13:37:20 +08:00
Tianyi Cui 39f2898502 fix review findings: last stale compact README line; stale-mark the Code Mode proposal
The compact-basic README's opening line still described summarization
as routed through the agent request pipeline — corrected to the direct
one-shot llm/stream contract the rest of the package documents. The
proposed Code Mode RFC gets a premise-stale note: it names the old
full-request agent/request seam, which now shapes call config only —
re-map onto the log channels and system-prompt/assemble before
implementing from it.
2026-07-06 04:22:37 +08:00
Tianyi Cui 8bd80e5e9b fix review findings: one frozen seed through the waterfall; stale agent/request docs
Codex diff review, round 1, two (A) findings:

- The agent/request fallback resolved the RAW seed object — on later
  steps the session's cached header fold — so a delegating listener
  (await next(), mutate, return) could rewrite the fold in place and
  the change would compare as already-baseline: no delta logged, the
  persisted log unable to reconstruct the request (the dev invariant
  would fire on the divergence, but the log would still lie). One
  structuredClone'd, deep-frozen seed now serves both the listener
  chain and the fallback — in-place shaping after delegation throws —
  and Session.requestHeader() freezes its fold on update, so the leak
  class is unrepresentable from either side. Pinned by a loop-level
  delegating-mutator test.
- Doc sweep for the old contract: agent README's event row (mutate
  GenerateOptions / tool filtering → frozen config seed, replacement
  out, logged header), compact-basic's module JSDoc (summarize routed
  through agent/request → direct one-shot at llm/stream), and
  architecture.md's event-domain line (request mutation → call-config
  shaping).
2026-07-06 04:17:29 +08:00
Tianyi Cui 23ed47ba75 chore: knip learns agent-loop's e2e entry (ported from #162 with request-cache.e2e.ts) 2026-07-06 04:01:56 +08:00
Tianyi Cui c58e5adee7 test: close the coverage gaps the gate found
surface.replaceGeneration gets its direct-access test (the getter folds
a pending delta itself, not only via a nodes read); the reconstruction
theorem drives maxTokens and stop through the config waterfall; the
compact-basic envelope drops a dead conditional — config.maxTokens is
required and validated positive, so this backend's envelope always
carries the cap (the return type's optionality exists for overriding
subclasses).
2026-07-06 03:55:30 +08:00
Tianyi Cui c0808d5126 docs: the governing principle — every LLM request is reconstructable from the session log
The reconstructability RFC is the principle's home: model-visible ⟺
logged in both forms, the mechanism (boundary derivation + header
fold), the enforcement (write-time round-trip guard, the dev
invariant), the corollaries ranked (prefix-cache stability first), the
MiniCode lineage with the provenance arrow inverted, and the
alternatives it beat — including the stateful transmission client
whose three-design archaeology lives in PR #162.

Placements per the one-home-per-fact taxonomy: a standing-order line in
root AGENTS.md (with displacement trims to stay inside the 1,575-word
ceiling), the principle statement in architecture.md § Session Log and
its Turn Flow lines (condensed to the ratcheted 1,630 ceiling), the
request-envelope section in core-data-structures/core.md with the
LlmCallConfig paste, both review-requested FIXMEs
(FIXME(call-config-shape) beside the type, FIXME(catalog-verbs) at the
catalog's drift-gate note), cookbook rows redirected off agent/request
(tool filtering → system-prompt/assemble, plan-mode prompt → sections/
inject()), and the llm/stream JSDoc stating the frozen-request
contract. RFC index and all generated catalogs regenerated.
2026-07-06 03:49:35 +08:00
Tianyi Cui 7376346246 test: re-record snapshot goldens — every transcript gains its request/header event
One re-record after the header events landed: recorded scenarios
re-harvested against the live API; the three fs-writing fixtures are
relativized (this recording's model happened to emit absolute
file_path arguments, which only round-trip through replay when the
path is cwd-independent) and, with the two never-re-recorded authored
scenarios (error-finish, cancel), rewritten in the normalized
authored-fixture form the harness documents — each now carrying the
request/header snapshot the loop logs before its first dispatch.
Keyless replay verified green across all 35 scenarios.
2026-07-06 03:39:48 +08:00
Tianyi Cui 7539231eca compact: the summary's provenance records its call envelope
compact/summary gains { model, maxTokens? } — the envelope the
summarize call actually used, reported by the backend that made the
call: summarize() now returns { summary, model, maxTokens? } instead of
bare blocks, so an overriding backend (template or remote summarizer)
reports its own envelope honestly and compactRegion logs it. 'Which
model wrote this summary' becomes answerable from the log alone, and
the one-shot summarize request — outside the loop's header-event fold
by design — is reconstructable from log + code (the reconstructability
RFC's scope statement).
2026-07-06 03:21:29 +08:00
Tianyi Cui 616c2ffba3 ci: keep measured e2e worker cap 2026-07-06 03:20:08 +08:00
Tianyi Cui cc670c0762 ci: use full e2e worker fanout 2026-07-06 03:16:34 +08:00
Tianyi Cui 65d644b161 invariants: a loop-built request must be exactly what the log reconstructs
The dev-mode cross-check on llm/stream: a frozen request with a live
sessionId (the loop-built marker; hand-built one-shots stay unfrozen
and skipped) must carry messages deep-equal to the derivation over the
log prefix strictly before the in-flight step's step/start, and header
fields equal to the fold of the log's request/header* events. The
messages side rebuilds through a FRESH Session over the boundary
prefix — same projection code, zero shared state — so the live cache
cannot vouch for itself, and the seq-bounded rebuild is boundary-
correct: content appended after step/start (an agent/request-window
inject) legitimately belongs to the next request and cannot false-fire
the check. prepend:true only defends against the replay adapter's
short-circuit (append-registered); correctness never rests on listener
ordering. There is no divergence-allowance caveat: nothing can shape
request content outside the log.
2026-07-06 03:14:07 +08:00
Tianyi Cui 3464f88297 test: shorten compaction e2e smoke 2026-07-06 03:12:37 +08:00
Tianyi Cui 1b9408268b ci: cache eslint lane 2026-07-06 03:07:35 +08:00
Tianyi Cui 2093a8898b loop: every request is built from the log — boundary snapshot, header events, config-only waterfall
The loop is now transmission-stateless; a request is a pure function of
(session log, this step's rendered assembly, current AgentOptions):

- The reconstruction boundary is step/start: the messages snapshot is
  taken in the same synchronous frame immediately before the step/start
  append, so the request's messages are exactly the derivation over
  events[0..stepStartSeq) — an inject() from an agent/request listener
  (or any concurrent task) lands after the boundary and joins the NEXT
  request. This changes behavior for a synchronous step/start
  session/event listener that appends content (master derived after the
  append, so such a listener could reach the current request):
  agent/pre-step is the sanctioned seam for current-request content.
- agent/request is re-typed to config-only: (agent, turn, step,
  config: LlmCallConfig, next) → LlmCallConfig. The frozen seed comes
  from AgentOptions on a loop instance's first request (explicit options
  beat the logged baseline — fork overrides and resume reconfiguration
  stay correct) and from the log's folded header afterwards; listeners
  return a replacement to switch. Content shaping through the request is
  no longer expressible — model-visible content flows through the log
  channels.
- recordRequestHeader appends whatever header event the request owes the
  log before dispatch: an 'initial'/'resume' snapshot anchoring each
  loop instance, a round-trip-verified delta on change, a 'fallback'
  snapshot when the encoding cannot express it. Session.requestHeader()
  is the log's incrementally-folded baseline.
- Requests are deep-frozen before dispatch (deepFreeze exempts the
  AbortSignal — freezing one breaks AbortController.abort() outright);
  frozen + sessionId is the loop-built marker the dev invariant keys on.

Ported from #162 and re-anchored on the log: the append-extension /
frozen-end-to-end / compaction-resend / prompt-change property tests,
plus new specs for the boundary semantics, resume anchoring, and the
end-to-end theorem (every recorded request rebuilds byte-equal from the
log alone). Live cache-hit e2e (request-cache.e2e.ts) verified against
the real DeepSeek API. Snapshot goldens intentionally stale until the
single re-record after the compact/summary envelope lands.
2026-07-06 03:07:34 +08:00
Tianyi Cui d71c96a44c ci: keep lint lane serial 2026-07-06 02:59:50 +08:00