Commit Graph
532 Commits
Author SHA1 Message Date
Tianyi Cui b115b2b4ac Merge remote-tracking branch 'origin/master' into compact-basic-refactor 2026-07-01 23:31:41 +08:00
Tianyi Cui 8a8b88b0a7 Merge remote-tracking branch 'origin/master' into compact-basic-refactor
# Conflicts:
#	docs/architecture.md
#	examples/coding-agent/README.md
#	packages/core/session/README.md
2026-07-01 23:29:20 +08:00
Tianyi Cui fd692eff19 Merge pull request #111 from deepseek-harness/docs/chinese-terminology
docs: add Chinese terminology table
2026-07-01 23:24:54 +08:00
Tianyi Cui 57900f9cbd docs(i18n): address round-2 terminology review
- ACP/SSE: show full English name in parens on first occurrence
- fixture: drop 测试夹具 gloss, keep descriptive note
- manifest: keep English, drop 首次出现可写 clause
2026-07-01 23:08:45 +08:00
Tianyi Cui 5252477bc9 fix(compact): make config knobs explicit and flag two review smells
Address @tianyicui's minor-revision review on PR #110:

- Make every BasicCompactConfig knob required except `auto` (defaults
  true): there is no data yet to justify default thresholds/budgets, so
  a consumer states each value explicitly. Drop the DEFAULTS export and
  the constructor's `= {}` default; example cordis.yml, the compaction
  e2e, the README, and every test construction site now pass a complete
  config (tests route through a `cfg()` helper).
- Add a TODO on estimateContentTokens: char/4 is coarse; replace with a
  real tokenizer or post-response usage feedback in a follow-up.
- Add a TODO on the agent/pre-step `fullSystemPrompt` param flagging it
  as a smell on a generic per-step seam (compaction is its sole
  consumer); a `//` line comment so it stays out of the generated catalog.
2026-07-01 22:10:49 +08:00
Ziya c93ed006e2 Merge branch 'master' into docs/chinese-terminology 2026-07-01 03:31:51 -04:00
Tianyi Cui 2fa16c16a4 Merge pull request #126 from deepseek-harness/codex/sync-docs-with-implementation
docs: sync implementation docs and doc gates
2026-07-01 13:18:34 +08:00
Tianyi Cui 0478f5965a docs: address review sync gaps 2026-07-01 12:56:13 +08:00
kingwl 643b77dabf docs: sync implementation docs and doc gates 2026-07-01 10:29:47 +08:00
Hypatia May 1a5302dbcf fix(compact): stamp summarization session ids 2026-06-30 12:07:20 +08:00
Hypatia May ed5d8550ae test(agent-loop): cover step-start disposal 2026-06-30 11:33:40 +08:00
Hypatia May b0eae94fc8 fix pre-step cancellation and compaction convergence 2026-06-30 10:56:34 +08:00
Hypatia May 6ae1e229fd docs(cordis): clarify serial bail semantics 2026-06-30 09:40:51 +08:00
Hypatia May 0c4059fc84 Merge remote-tracking branch 'origin/master' into compact-basic-refactor
# Conflicts:
#	docs/cordis-catalog/events-and-services.md
#	examples/AGENTS.md
#	examples/coding-agent/cordis.yml
#	examples/coding-agent/tests/harness.ts
#	scripts/gen-cordis-catalog.ts
2026-06-30 09:13:15 +08:00
Tianyi Cui 3f85f522ea Merge pull request #117 from deepseek-harness/drop-catalog-count-prose
docs(cordis-catalog): omit exact event/service counts from catalog prose
2026-06-29 22:10:30 +08:00
Tianyi Cui dcf8e10e5a docs(cordis-catalog): drop merge-conflict-prone count sentences
The Events intro carried "The harness declares N events across M scopes."
and the Services intro "The N `ctx.<key>` services the harness provides."
Both embed counts the generator recomputes from source, so every branch
that adds an event or service rewrites that one line — a guaranteed merge
conflict against any sibling branch that also touched the catalog, for
prose that adds nothing a reader can't get by scanning the page.

Remove the count clauses from the generator's render() and regenerate the
catalog. The freshness gate (verify-cordis-catalog) stays green.
2026-06-29 22:04:37 +08:00
Tianyi Cui 08522c2c3f Merge pull request #116 from deepseek-harness/worktree-todo-write-tool
feat: add the todo_write tool (model task list as session state)
2026-06-29 21:46:05 +08:00
Tianyi Cui 9f1caf7c5b fix(todo): address todo_write review feedback 2026-06-29 19:44:38 +08:00
Hypatia May 170643ec9b test(compact-basic): restore coverage gate 2026-06-29 18:04:59 +08:00
Hypatia May 4bda95b589 ci: remove gitlab mirror workflow 2026-06-29 18:04:11 +08:00
Hypatia May 277bbf85fb Merge remote-tracking branch 'origin/compact-basic-refactor' into compact-basic-refactor 2026-06-29 17:04:41 +08:00
Hypatia May 1808570933 fix(compact): harden summarization convergence
Use maxTokens as the provider generation cap and remove the confusing stored-summary max config.

Strip reasoning blocks before storing compaction summaries, reject non-shrinking summaries, and retry bounded re-compaction when the surface remains over threshold.

Add config validation for numeric and type-shaped knobs plus unit and real-API e2e coverage for reasoning-capable summarization.
2026-06-29 16:56:44 +08:00
Hypatia May 1f35a4446d fix(compact): address PR 110 review findings
Honor cancellation and disposal around async pre-step setup before the loop can open a step or call the model.

Route compaction summarization through agent/request so router agents can select the model, and remove the stale model argument from agent/pre-step.

Document serial events and the approximate convergence bound, regenerate the Cordis catalog, and add regression coverage for router compaction, HMR cleanup, and assembly/pre-step interruption.
2026-06-29 16:02:27 +08:00
Tianyi Cui 55088d1cfc docs(AGENTS): require running the CI gates locally before marking a PR ready
Add a "Run the CI gates locally BEFORE marking a PR ready" subsection: the
CI-equivalent local command line, and the rule that `pnpm run test:coverage`
(per-file 100%, CI-enforced) — not `pnpm run test` — is the gating test command,
alongside hygiene/snapshot/doc-sync. A green `test` run can still fail CI on an
uncovered line, which is usually dead code the gate is correctly flagging.
2026-06-29 15:53:34 +08:00
ZiyaZhang f05717e1ac docs: address terminology review comments
Clarity-first revisions from review:
- keep English for ambiguous terms (Cordis, fork, harness, schema, spawn,
  manifest, transcript, compaction, dispose)
- first-use glosses for CLI, Function Calling, HMR, fiber
- inference/reasoning carry the English in parens to disambiguate
- memory distinguishes 记忆 (agent memory) vs 内存 (resource usage)
- plugin -> 插件; add a separate mod -> 模组 row
- 中英搭配 note on translated terms (wire format, adapter contract)
2026-06-29 00:47:29 -07:00
Tianyi Cui 5269d2ac3d fix(tool-todo): drop the unreachable status re-check that broke the coverage gate
The registry's validateArgs rejects a bad `status` enum before execute runs, so
the in-body re-check (`status !== 'pending' && …` → throw) was unreachable dead
code — line 70 was uncovered, failing the per-file 100% coverage gate. Narrow
the registry-guaranteed value with `status as TodoItem['status']` instead of
re-validating it, mirroring tool-bash (which only checks what the DSL can't
express). The malformed-status test still passes — it exercises the registry's
rejection, the actual path. Coverage back to 100%.
2026-06-29 15:38:54 +08:00
Tianyi Cui 5f744e4fa6 test(tool-todo): guard that stored content is trimmed
Codex confirmation review: the trim-the-stored-content fix had no test that
would fail if it regressed (existing assertions use already-trimmed todos).
Add a focused test asserting "  plan the work  " appends content "plan the
work". Verified it fails red against the pre-fix code.
2026-06-29 11:06:41 +08:00
Tianyi Cui 93a6dc6716 test(todo): add the todo-plan ACP snapshot scenario and a with-key e2e
Record the `todo-plan` snapshot scenario: a real prompt drives the model to call
todo_write, and the golden captures the resulting `plan` sessionUpdate (three
entries, priority synthesized as medium, status 1:1) plus the persisted
todo/write event. Registered in SCENARIOS; replays deterministically keyless.

Add a with-key coding-agent e2e that verifies the WORLD — a real model call to
todo_write lands a todo/write event whose snapshot is a valid, one-in-progress
list — not the agent's self-report. Wire tool-todo into the e2e harness.
2026-06-29 10:56:55 +08:00
Tianyi Cui e17c4b748d refactor(tool-todo): store the trimmed content, matching the dedupe key
toTodoList dedupes and length-checks on the trimmed content but stored the raw
item.content, so a todo with leading/trailing whitespace was deduped by its
trimmed form yet persisted untrimmed — the stored value and the uniqueness key
could differ. Store the trimmed content so the persisted list matches what was
validated.
2026-06-29 10:50:31 +08:00
Tianyi Cui 46e31d8481 feat(tool-todo): add the model-facing todo_write tool
Add @deepseek-ai/dsh-tool-todo (a new packages/todo/ group): a model-facing
todo_write(todos: [{content, status}]) tool with whole-list-replace semantics.
Each call appends the full list as a todo/write event to the calling agent's
session log; the current list is the most recent such event (last-write-wins).
Single-owner — a non-agent caller is rejected. Beyond the schema's
type/required/enum checks, execute rejects empty/duplicate content and more than
one in_progress task, narrowing the loosely-typed args into a real TodoItem[].

Both UIs render off the existing session/event: the stdio UI prints a glyphed
checklist; the ACP bridge maps the list to a `plan` sessionUpdate (todosToPlan
synthesizes the priority ACP requires; status maps 1:1). Wired into the
coding-agent, acp-agent, and snapshot example configs with a system-prompt nudge.

Tests: unit (schema, validation, append/replace, no-agent rejection, presentCall,
HMR-safety, Loader export-shape guard), full-loop integration through the agent
loop, the ACP todosToPlan mapping + stream-update arm, the stdio render arm, and
a session/load replay that re-emits the plan. New-group TS wiring added to
tsconfig.base/json/build. RFC + a doc-inventory sweep (architecture, packages
README, AGENTS layout, cookbook group list, example READMEs) ship with it.

The todo-plan ACP snapshot scenario is recorded separately (needs an API key).
2026-06-29 10:30:52 +08:00
Tianyi Cui 4f09157612 docs(session): scope the todo/write JSDoc to the event's own contract
Codex Phase 1 review: the event JSDoc described Phase 2 consumers (the
todo_write tool, stdio printing, ACP plan mapping) as current state, and put an
@mode tag on a SessionEventMap member. @mode is for first-class Cordis
`interface Events` entries the catalog generator reads — this event rides the
existing session/event emit and has no catalog row, so the tag was wrong.

Trim the JSDoc to the event's own contract (snapshot data shape,
last-write-wins, not-a-surface-event) and drop @mode; phrase TodoItem in terms
of its own purpose rather than a not-yet-present tool.
2026-06-29 01:50:35 +08:00
Tianyi Cui 22a89847ac feat(session): add TodoItem + todo/write event vocabulary
Add the TodoItem type and a todo/write SessionEventMap variant carrying the
whole todo list as a snapshot (last-write-wins on replay). It is NOT a
SurfaceEventType: it produces no LLM message and never reaches
deriveMessages(), so it carries no surfaceOp and stays off the surface — it is
durable, replayable UI state that rides the existing session/event emit.

Tests cover the snapshot-clone-on-append contract, last-write-wins, the
not-on-surface guarantee, and a seeded replay round-trip. Docs: session.md
gains the TodoItem type-equiv block + the event member; core.md's variant count
goes to twelve; the type-equiv manifest gains TodoItem.
2026-06-29 01:39:25 +08:00
Tianyi Cui b995b4f788 Merge branch 'master' into compact-basic-refactor
# Conflicts:
#	packages/core/session/tests/surface.spec.ts
2026-06-27 00:06:04 +08:00
Tianyi Cui 185c09d6a5 Merge pull request #115 from deepseek-harness/worktree-ci-lint-heap
ci: fix lint OOM and surface.ts coverage gap (re-enabled CI)
2026-06-27 00:03:15 +08:00
Tianyi Cui 489a26e865 test(session): cover the isSurfaceEvent / isSurfaceEligibleType guards
The per-file 100% coverage gate flagged surface.ts line 46 — the
branch where a surface-eligible event type carries no surfaceOp marker
(isSurfaceEvent returns false). Exercise both guards directly: the
type-only eligibility check, the positive narrowing path, a
non-eligible type, and the markerless-but-eligible branch.
2026-06-26 23:47:54 +08:00
Tianyi Cui 0619ce62b6 ci: raise Node heap ceiling for type-aware ESLint lint step
Type-aware ESLint loads every package tsconfig through the project
service and peaks at ~3.4GB RSS. The default V8 old-space ceiling
(~2GB) OOMs it (FATAL ERROR: Ineffective mark-compacts near heap
limit, exit 134) on both node 24 and 26. Set NODE_OPTIONS with an
8GB ceiling for the Lint step, comfortably above the peak.
2026-06-26 23:38:53 +08:00
Tianyi Cui 3e2f957861 Merge branch 'master' into compact-basic-refactor 2026-06-26 23:28:28 +08:00
Tianyi Cui 8d1f86fcca empty 2026-06-26 23:25:40 +08:00
Hypatia May c13f25586c Merge remote-tracking branch 'origin/master' into compact-basic-refactor 2026-06-26 16:47:21 +08:00
Hypatia May c76b7042b6 fix(compact): size the compaction e2e to actually cross threshold; sync stale docs (CBR-007)
The compaction e2e never exercised compaction: its window/fixture combo
(contextWindow 8000, thresholdRatio 0.5 → threshold 4000; four small files)
peaked at ~1389 estimated tokens, so compactIfNeeded declined every pre-step
and compact/start never landed. Shrink the window (contextWindow 2400 →
threshold 1200; retainTokens 500 + summarizationMaxTokens 300 = 800 < 1200,
convergence holds) and grow the fixture to six files so a couple of bash steps
reliably cross the threshold. Verified compaction fires and the suite passes
across repeated real-API runs.

Sync docs left stale by the landed compaction work: list compaction.e2e.ts and
keyless-smoke.e2e.ts in the coding-agent README (and fix the wrong "Both
self-skip" count), add compaction to the examples with-key inventory, and
replace the hypothetical compaction/marker / "future plugin" naming in the
session README, session types JSDoc, and the core-data-structures catalog with
the real compact/start, compact/summary, compact/end events.
2026-06-26 16:42:51 +08:00
Ni Shentu 05c1bb628c Add GitHub Actions workflow to mirror to GitLab 2026-06-26 16:18:15 +08:00
Hypatia May f4ace25648 fix(compact): correct _extractText surface-order JSDoc; add disposal to the HMR-safety suite (CBR-005, CBR-006)
Manual review round, two non-blocking findings:

- CBR-005: _extractText's JSDoc claimed it "walks events in log order",
  but it walks the seqs in surface order (the inline comment already said
  so) — the exact distinction CBR-001 paid for, since after a replace a
  high-seq checkpoint heads the surface before lower-seq retained nodes.
  Corrected the JSDoc to match.
- CBR-006: the "HMR safety" suite only asserted registration; the actual
  dispose-and-confirm-cleanup test lived under "llm inject", so a reader
  searching by name could miss it. Added a disposal test to the HMR-safety
  suite (mount via the real plugin fiber with LlmService present so inject
  resolves, dispose, assert ctx.get('compact') is undefined) and reframed
  the llm-inject test's trailing teardown to point at it.
2026-06-26 14:40:20 +08:00
Hypatia May 6de5e3a20a docs(compact): fix stale agent/pre-request comment in coding-agent cordis.yml (CBR-004)
codex review round 2 (non-blocking) CBR-004: the example's compaction
wiring comment still named the old `agent/pre-request` seam. Renamed to
`agent/pre-step` to match the shipped seam.
2026-06-26 14:02:02 +08:00
Hypatia May b13586ff8e docs(compact): align seam docs with the pre-step seam and record session/invariants changes (CBR-003)
Codex round 1 CBR-003: several docs still described compaction as an
`agent/request` waterfall concern, and the implemented compaction RFC
claimed "No changes to dsh-session or dsh-invariants" while the diff
changed both.

- Package READMEs / JSDoc (agent, agent-loop, system-prompt, compact,
  compact-basic): compaction now lives on the serial `agent/pre-step`
  seam (fired after turn/start, before step/start); the structural guard
  is tool-pairing balance (`isToolPairingBalanced`), not step-alignment;
  the convergence bound is strict (`>=` rejects).
- architecture.md / core-data-structures/compaction.md: same seam +
  predicate + dispatch-mode updates; regenerated cordis catalog.
- Implemented compaction RFC, updated in place to describe shipped
  reality: the seam is `agent/pre-step` (@mode serial) fired before
  step/start; alignment is surface tool-pairing balance; the convergence
  invariant rejects `>=`; and the "no dsh-session/dsh-invariants changes"
  claim is corrected — dsh-session gains the tool-pairing predicate and
  dsh-invariants drops its `start <= end` replace assertion (a positional
  replace makes start > end normal).
2026-06-26 13:51:45 +08:00
Hypatia May 6cac3e6476 fix(compact): reject threshold-equality config to keep compaction convergent (CBR-002)
Codex round 1 CBR-002: `resolveConfig` rejected only
`summarizationMaxTokens + retainTokens > threshold` (allowing equality),
but `compactIfNeeded` declines only when the estimate is `< threshold`.
At exact equality the post-compaction history sits at the threshold and
re-triggers on the very next check.

Make the bound strict (`>=` rejects), so post-compaction history is
guaranteed strictly below the threshold. Updated the boundary test (the
sum-equals-threshold case is now rejected, not accepted) and added an
"accepts just below the threshold" case; nudged one unrelated config that
incidentally sat at the equality boundary.
2026-06-26 13:51:20 +08:00
Hypatia May d6da8ca29a fix(compact): decide step-alignment from surface tool-pairing, fire compaction pre-step (CBR-001)
Codex round 1 CBR-001: a head-anchored compaction checkpoint was
mis-classified by the log-position step-alignment scan, so a second
auto-compaction over a checkpoint-headed surface silently failed.

Root cause: `isStepAlignedStart/End` scanned the LOG by seq, but a
`replace` op lands a checkpoint at a high log seq whose SURFACE position
is the head — its log neighbours (the open step's assistant/message) are
not its surface neighbours, so the forward scan wrongly reported mid-step.

Fix, per the agreed direction:
- Replace the two log-position predicates with one surface-anchored
  helper `isToolPairingBalanced(nodes, events, beforeSeq)` in
  `dsh-session` (renamed step-boundary.ts → tool-pairing.ts). A cut is
  balanced when no unanswered tool-call precedes it on the surface; a
  region is collapsible iff both edges are balanced cuts. The open-tail
  and free-node cases fall out of the same counter. It also throws on a
  corrupt surface (a tool/result with no matching call).
- Move compaction off the in-step seam to a new "pre-step" seam fired
  after turn/start and before step/start, so a compaction's log-only
  compact/* records and its replacement node land cleanly OUTSIDE any
  step (the honest structure crash-safety relies on). Renamed the event
  agent/pre-request → agent/pre-step and switched its dispatch from
  parallel → serial (listeners mutate the surface as a side effect;
  serial isolates them so concurrent appends can't interleave). Extended
  the catalog generator to accept @mode serial.

Regression coverage: a real-loop test driving an auto-compaction asserts
the landed checkpoint is a balanced cut on both sides; unit tests pin the
checkpoint case, the mid-step injection case, multi-call steps, and the
corrupt-surface guard. Proven red on the old log-position logic.
2026-06-26 13:51:01 +08:00
ZiyaZhang f962fda8c1 docs: add Chinese terminology table 2026-06-25 22:42:48 -07:00
hypatiamay 32462f42bd Merge pull request #97 from deepseek-ai/compact-interface
feat(compact): compaction capability seam — abstract CompactService interface
2026-06-26 10:00:06 +08:00
Hypatia May cec32faa4e refactor(compact): turn-agnostic retention + dedicated agent/pre-request seam
Reform the compaction blueprint so a runaway turn survives and the design
stops drifting across review rounds:

- Drop in-flight-turn protection ("layer 2"). Retention is a uniform tail→head
  whole-unit walk; the only structural guard is step-alignment. A single turn
  that alone exceeds the window now compacts its own early closed steps instead
  of being retained verbatim (the failure mode that motivated this).
- Move auto-compaction off the agent/request waterfall onto a new awaited
  agent/pre-request loop seam, fired before history derivation. Compaction
  mutates the surface; the loop derives once from the result — no double-derive,
  and a listener structurally cannot act on not-yet-derived messages.
- Tighten compactIfNeeded to required (session, system, model, signal).
- Enforce a single-pass convergence invariant in resolveConfig: reject configs
  where summarizationMaxTokens + retainTokens exceeds the threshold, so a
  compaction can never immediately re-trigger.
- Document the crash vs recoverable failure taxonomy; core session repair stays
  compaction-agnostic (a log-only orphaned compact/start is inert).
- Wire dsh-compact-basic into examples/coding-agent and add a with-key
  compaction e2e (compaction's first real-world exercise + runaway net).
- Rewrite the RFC to encode the blueprint and move it to implemented/.

The runaway-turn snapshot is a named deferred follow-up: dsh-llm-replay cannot
yet serve the interleaved summarization model call.
2026-06-26 08:59:33 +08:00
hypatiamay 6cd446da2f Merge pull request #49 from deepseek-ai/session-surface
feat(session): add session surface — a linked list over the event log for message derivation
2026-06-26 08:57:33 +08:00