Commit Graph
59 Commits
Author SHA1 Message Date
Tianyi Cui 9801a0d102 fix(tool-web): bound HTML conversion work 2026-07-27 16:40:02 +08:00
Tianyi Cui 109b469a7e fix(tool-web): bound conversion depth and complete fetch output
Two review findings on the turndown swap, both verified empirically:

- Unclosed-tag nesting makes the synchronous turndown/domino walk
  superlinear (measured: depth 512 ~0.15s, 2k ~2s, 20k ~5s), during
  which the cooperative fetchTimeoutMs timer cannot fire. renderBody
  now preflights nesting depth with a linear tag scan and passes
  bodies past 512 levels through raw; the try/catch stays for markup
  the scan cannot see (comment-hidden tags), simulated in tests via a
  converter throw.
- Markdown escaping can expand converted HTML ~2x (100k underscores
  render as 200k chars), so provider body caps no longer bounded the
  model-visible result. formatFetchOutput now caps the complete output
  (header + body + footer) under new fetchMaxOutputChars config
  (default 200000 = 2x the local provider's default body cap), reusing
  the truncation notice.

README EN+ZH, config catalog, Agent Note EN+ZH updated; the new
web-fetch fixture is migrated to the packed layout master now
requires; tool-web coverage stays 100% per-file.
2026-07-27 12:41:36 +08:00
Tianyi Cui 45a5175e44 feat(tool-web): replace the regex HTML-to-markdown converter with turndown
Implements the turndown Agent Note from the NIH dependency audit (full
variant, not the minimal entities-only fallback): dsh-tool-web's fetch
rendering now converts HTML through turndown + @joplin/turndown-plugin-gfm
(atx headings, fenced code, dash bullets, GFM tables/strikethrough) over
the real domino DOM, with script/style/noscript removed wholesale. The
hand-rolled ~86-line regex converter html.ts and its entity tables are
deleted; renderBody wraps the conversion in try/catch falling back to
the raw HTML body, because turndown's recursive DOM walk overflows with
a RangeError on pathological nesting (measured: 4k levels on the main
thread, 8k in a worker) where the regex version could never throw.

Closure weight, measured: tool-web IS in the single-exe runtime closure,
and the exe asset globs would pack ~7.9 MB of the three new packages —
but ~6 MB of that is domino's test corpus, with runtime lib/ at ~550 KB
against a ~174 MB artifact (<0.5% either way), so the swap wins.

Per testing policy the previously-missing keyless web_fetch snapshot
ships in the same change: the acp-agent `web-fetch` scenario boots a new
web.cordis.yml overlay (web seam + real dsh-web-fetch-local provider +
tool-web fetch-only + a loopback HTTP fixture server on a fixed port
serving deterministic HTML with entities, a GFM table, and nesting), so
recording and keyless replay both drive the real HTTP fetch and real
conversion end to end; the scenario pins the new `web` header class.

The Agent Note moves proposed -> implemented and is rewritten per the
lifecycle contract (Decision/Consequences/Testing, closure verdict and
alternatives recorded); tool-web and acp-agent READMEs updated in both
languages and pairs re-recorded.
2026-07-27 00:22:13 +08:00
Tianyi Cui 226dc7a249 docs: translate remaining READMEs 2026-07-26 05:06:39 +08:00
Tianyi Cui 7d3cb39047 Merge branch 'codex/tool-json-schema-dsl' into codex/canonical-tool-output
# Conflicts:
#	.agents/notes/implemented/feature/2026-06-30-interception-seams.md
#	docs/config-catalog.md
#	docs/cookbook/adding-a-tool.i18n.yaml
#	docs/cookbook/adding-a-tool.md
#	docs/cookbook/adding-a-tool.zh.md
#	docs/cordis-catalog/events.md
#	docs/cordis-catalog/services.md
#	docs/core-data-structures/tools.md
#	docs/event-producer-consumer.md
#	docs/persistence-catalog.md
#	examples/acp-agent/tests/snapshots/cordis-inspect-jsdoc/session.jsonl
#	examples/acp-agent/tests/snapshots/cordis-inspect-jsdoc/stdout.expected.jsonl
#	packages/bash/tool-bash/src/index.ts
#	packages/core/agent-loop/src/tool-calls.ts
#	packages/core/agent-loop/tests/cancel.spec.ts
#	packages/core/agent-loop/tests/contract-regressions.spec.ts
#	packages/core/agent-loop/tests/tool-calls.spec.ts
#	packages/core/tools/README.md
#	packages/core/tools/src/index.ts
#	packages/core/tools/tests/code-mode.spec.ts
#	packages/core/tools/tests/tools.spec.ts
#	packages/fs/tool-fs-search/tests/integration.spec.ts
#	packages/fs/tool-fs-search/tests/tools.spec.ts
#	packages/fs/tool-fs/tests/integration.spec.ts
#	packages/mcp/mcp-client/src/tools.ts
#	packages/timeout/timeout-policy/tests/timeout-policy.spec.ts
#	packages/web/tool-web/tests/integration.spec.ts
#	packages/web/tool-web/tests/tool-web.spec.ts
2026-07-21 23:39:03 +08:00
Tianyi Cui 0004b8675f Merge remote-tracking branch 'origin/master' into codex/enforce-tool-cancellation
# Conflicts:
#	docs/event-producer-consumer.md
#	examples/acp-agent/tests/snapshots/bash-spill/session.jsonl
#	examples/acp-agent/tests/snapshots/escalation-approved/session.jsonl
#	examples/acp-agent/tests/snapshots/escalation-rejected/session.jsonl
#	examples/acp-agent/tests/snapshots/fs-escalation-approved/session.jsonl
#	examples/acp-agent/tests/snapshots/hook-cc-pretool-ask/session.jsonl
#	packages/context/workspace-context/tests/workspace-context.spec.ts
#	packages/core/agent/src/index.ts
#	packages/support/invariants/tests/invariants.spec.ts
#	packages/ui/acp/src/index.ts
#	packages/ui/tui/tests/harness.ts
#	packages/ui/tui/tests/tui.spec.ts
2026-07-21 20:03:00 +08:00
Tianyi Cui 72a48fede6 Merge branch 'codex/tool-json-schema-dsl' into codex/canonical-tool-output
# Conflicts:
#	docs/event-producer-consumer.md
#	packages/context/time-context/tests/time-context.spec.ts
#	packages/context/workspace-context/tests/workspace-context.spec.ts
2026-07-21 19:56:43 +08:00
Tianyi Cui 66c36e7325 feat: add canonical typed tool outputs 2026-07-21 03:22:14 +08:00
Tianyi Cui 1145ee5fc3 fix(invariants): assert runtime relationships, not API shapes 2026-07-20 19:34:19 +08:00
Tianyi Cui 684fcf3357 Merge branch 'codex/invariant-package-registration-gate' into codex/package-invariant-checks
# Conflicts:
#	.agents/notes/implemented/architecture/2026-07-19-package-invariant-runtime-contracts.i18n.yaml
#	.agents/notes/implemented/architecture/2026-07-19-package-invariant-runtime-contracts.md
#	.agents/notes/implemented/architecture/2026-07-19-package-invariant-runtime-contracts.zh.md
#	.agents/notes/implemented/architecture/2026-07-19-package-owned-invariant-service.i18n.yaml
#	docs/rfc/INDEX.md
#	packages/AGENTS.md
2026-07-20 00:54:28 +08:00
Tianyi Cui 941b0411d8 feat(invariants): implement package runtime checks 2026-07-20 00:38:37 +08:00
Tianyi Cui 6520f71f94 Merge remote-tracking branch 'origin/codex/invariant-service-seam' into codex/invariant-package-registration-gate
# Conflicts:
#	.agents/notes/implemented/architecture/2026-07-19-package-owned-invariant-service.i18n.yaml
#	docs/core-data-structures/session.md
#	docs/module-graph.md
#	package.json
#	packages/sdk/scripts/tsconfig.json
2026-07-20 00:35:06 +08:00
Tianyi Cui a9a44fea96 Merge remote-tracking branch 'origin/master' into codex/enforce-tool-cancellation
# Conflicts:
#	.agents/notes/implemented/architecture/2026-07-19-cooperative-tool-cancellation.i18n.yaml
#	.agents/notes/implemented/architecture/2026-07-19-cooperative-tool-cancellation.md
#	.agents/notes/implemented/architecture/2026-07-19-cooperative-tool-cancellation.zh.md
#	.agents/notes/proposed/architecture/2026-07-19-required-cancellation-through-tool-capability-seams.i18n.yaml
#	.agents/notes/proposed/architecture/2026-07-19-required-cancellation-through-tool-capability-seams.md
#	.agents/notes/proposed/architecture/2026-07-19-required-cancellation-through-tool-capability-seams.zh.md
#	docs/cookbook/adding-a-tool.i18n.yaml
#	docs/rfc/INDEX.md
#	packages/AGENTS.md
2026-07-19 23:43:36 +08:00
Tianyi Cui e8b95c8754 feat(tools): require cancellation signal on every invocation 2026-07-19 23:38:54 +08:00
Tianyi Cui e8eddc7ef8 Rename RFCs to Agent Notes 2026-07-19 22:52:03 +08:00
Tianyi Cui 433670a754 feat(invariants): require package-owned companions 2026-07-19 22:13:50 +08:00
Tianyi Cui a278374e8f docs: structure model experience fields 2026-07-19 18:08:42 +08:00
Tianyi Cui bcc920369c docs: document package KV cache effects 2026-07-19 17:39:50 +08:00
Tianyi Cui 21ec178841 docs: tighten parallel tool-call prose 2026-07-18 14:59:26 +08:00
Tianyi Cui 688b95a7e5 Merge origin/master into parallel-tool-call 2026-07-17 22:13:33 +08:00
Dudu-0223 a3f62478e1 Merge origin/master into codex/truncated-design 2026-07-17 18:21:54 +08:00
Dudu-0223 3bed1a644f Merge origin/master into parallel-tool-call 2026-07-16 14:56:02 +08:00
Tianyi Cui 96851da571 Merge remote-tracking branch 'origin/master' into codex/simp-prune-web-seam-fields
# Conflicts:
#	docs/rfc/INDEX.md
2026-07-15 13:28:40 +08:00
imccyu 31f4441fd4 pkg: update vendor dep 2026-07-15 11:28:45 +08:00
Dudu-0223 77be4b891b Merge remote-tracking branch 'origin/master' into pr-265
# Conflicts:
#	docs/config-catalog.md
#	docs/cordis-catalog/services.md
#	docs/rfc/INDEX.md
#	examples/acp-agent/tests/snapshots/both-mode-turn/session.jsonl
#	examples/acp-agent/tests/snapshots/skill-load/session.jsonl
#	examples/acp-agent/tests/snapshots/text-turn/session.jsonl
#	examples/sandbox-acp-agent/tests/snapshots/escalation-approved/session.jsonl
#	examples/sandbox-acp-agent/tests/snapshots/escalation-rejected/session.jsonl
#	packages/core/agent-loop/README.md
#	packages/core/agent-loop/src/loop.ts
#	packages/core/tools/README.md
#	packages/core/tools/src/index.ts
#	packages/core/tools/src/schema.ts
#	packages/ui/acp/src/index.ts
#	packages/ui/stdio-agent/README.md
2026-07-14 20:49:54 +08:00
Tianyi Cui 92c52eec20 Merge remote-tracking branch 'origin/master' into codex/simp-prune-web-seam-fields
# Conflicts:
#	docs/config-catalog.md
#	docs/cordis-catalog/services.md
#	docs/core-data-structures/web.md
#	docs/rfc/implemented/simplification/2026-07-04-drop-unconsumed-web-observation-surface.md
#	packages/web/tool-web/tests/integration.spec.ts
#	packages/web/web-fetch-local/README.md
#	packages/web/web-fetch-local/src/provider.ts
#	packages/web/web/src/types.ts
2026-07-14 17:57:50 +08:00
Tianyi Cui b79520ab87 Merge remote-tracking branch 'origin/master' into codex/simp-prune-web-seam-fields
# Conflicts:
#	packages/web/tool-web/README.md
#	packages/web/web/README.md
2026-07-14 15:56:42 +08:00
Tianyi Cui 8590ba00a4 Merge remote-tracking branch 'origin/master' into codex/trim-ai-prose
# Conflicts:
#	docs/AGENTS.md
#	docs/config-catalog.md
#	packages/bash/bash-sandbox/src/index.ts
#	packages/bash/bash/src/session-mode.ts
#	packages/bash/tool-bash/README.md
#	packages/code-runtime/code-runtime-worker/README.md
#	packages/compact/compact/src/index.ts
#	packages/core/agent-core/README.md
#	packages/hooks/hooks-claude/src/config.ts
#	packages/hooks/hooks-claude/src/index.ts
#	packages/hooks/hooks-codex/src/config.ts
#	packages/hooks/hooks-codex/src/index.ts
#	packages/llm/llm/README.md
#	packages/session-persistence/session-persistence-jsonl/README.md
#	packages/session-persistence/session-persistence/README.md
#	packages/skill/skill-local/README.md
#	packages/support/acp-snapshot/README.md
#	packages/support/invariants/src/index.ts
#	packages/ui/acp/README.md
#	packages/ui/jsonrpc-agent/README.md
#	packages/ui/jsonrpc/README.md
#	packages/ui/permission/README.md
#	packages/ui/user-approval/README.md
#	packages/ui/user-interaction/README.md
#	packages/web/web-search-deepseek/README.md
2026-07-14 14:37:16 +08:00
Tianyi Cui 3ab35de64f refactor: prune unused web seam fields 2026-07-14 04:17:38 +08:00
Tianyi Cui eed43abe66 docs: address final model experience review 2026-07-14 00:22:52 +08:00
Tianyi Cui 148046b9c8 docs: rebalance prose cleanup and add trimming skill 2026-07-13 23:27:00 +08:00
Tianyi Cui d77f797d89 docs: replace model experience tables with sections 2026-07-13 22:26:33 +08:00
Tianyi Cui 646f3d2d0c docs: apply verbatim model experience guidance repo-wide 2026-07-13 21:33:23 +08:00
Tianyi Cui 7b539a0f8b docs: align model experience with runtime visibility 2026-07-13 15:00:47 +08:00
Dudu-0223 7ea1bf119f feat(agent-loop): run safe tool calls in parallel 2026-07-13 11:13:21 +08:00
Dudu-0223 5ac03dde3f fix(review): generalize spill storage locators 2026-07-13 11:07:27 +08:00
Tianyi Cui 75838e10b5 docs: trim generated prose 2026-07-12 03:36:43 +08:00
Tianyi Cui b5cd511f35 docs: address model experience review 2026-07-12 02:55:26 +08:00
Tianyi Cui a08485fc80 Merge remote-tracking branch 'origin/codex/package-readme-limitations-audit-20260712' into codex/model-experience-readmes-20260712
# Conflicts:
#	docs/cookbook/adding-a-package.md
#	docs/rfc/INDEX.md
#	package.json
#	packages/AGENTS.md
#	packages/bash/bash-sandbox/README.md
#	packages/sandbox/sandbox-local/README.md
#	packages/sandbox/sandbox/README.md
#	packages/session-persistence/session-persistence-sqlite/README.md
#	packages/support/acp-snapshot/README.md
#	packages/ui/app-boot/README.md
#	packages/ui/user-approval/README.md
#	packages/workflow/tool-workflow/README.md
2026-07-12 02:48:49 +08:00
Tianyi Cui 027970043c docs: document package model experience 2026-07-12 02:12:36 +08:00
Tianyi Cui 41a7c70cb1 docs: audit package limitations on current stack 2026-07-12 02:08:35 +08:00
Tianyi Cui ecb8aa5b8e Add a gated Known Limitations and Deferred Work section to every package README
Every packages/*/* README now carries a canonical '## Known Limitations and
Deferred Work' section: condensed, evidence-backed bullets for consumer-visible
gaps (unimplemented features, platform caveats, MVP cuts) and consciously
postponed work (TODO/FIXME/XXX markers, RFC deferrals still open). The ten
pre-existing ad-hoc variants ('What is NOT here (TODO)', 'Deferred',
'Limitations (MVP)', 'Known limitations (tracked TODOs)', ...) are normalized
into the canonical heading.

A new doc-sync gate, scripts/verify-readme-limitations.ts, enforces the shape:
exactly one limitations-like heading per package README, byte-equal to the
canonical h2, with at least one bullet; near-miss headings fail so variants
cannot creep back. Packages with genuinely nothing to declare (dsh-brand,
dsh-timeout, dsh-subagent-mock, dsh-app-boot) are whitelisted in the script and
must NOT carry the section; whitelist entries are validated against the scanned
package set so a rename fails loud.

Wired into the doc-sync chain (package.json) and the run-gates doc-sync leaf
set; the standing rule lands in packages/AGENTS.md and the adding-a-package
cookbook; decision record in
docs/rfc/implemented/process/2026-07-10-readme-known-limitations-gate.md
(RFC index regenerated).

Also fixes two stale '(deferred)' markers claiming dsh-compact-basic is
unimplemented (the dsh-compact seam README's package table and the seam's
module doc comment).
2026-07-12 01:46:34 +08:00
Dudu-0223 78318f5d12 Merge remote-tracking branch 'origin/master' into codex/truncated-design
# Conflicts:
#	docs/event-producer-consumer.md
#	docs/module-graph.md
#	docs/rfc/INDEX.md
#	knip.json
#	packages/README.md
#	scripts/gen-module-graph.ts
#	tsconfig.base.json
#	tsconfig.build.json
#	tsconfig.json
2026-07-09 12:54:30 +08:00
Dudu-0223 326b199f25 fix: address codex review round 2
- spill-policy reserves the spill notice's byte cost inside maxInlineBytes, so
  the replacement (preview + notice) never exceeds the documented model-facing
  cap. When the notice alone fills the budget the preview is empty; when even a
  notice-only replacement is not smaller than the original, the inline result is
  kept (spilling would only add bytes).
- retention TextRetainer trims an oversized single suffix chunk to the last
  suffixCap bytes on push, so tail/headTail retention stays bounded by suffixCap
  instead of retaining and re-copying the whole chunk in finish() — this is the
  spill preview path, which pushes the whole result as one chunk.
2026-07-09 09:51:35 +08:00
Dudu-0223 463b72ce96 feat(spill): add tool-output spill seam, local backend, and policy
Oversized plain-text tool results now spill to a session-scoped file and
return a bounded preview plus the spill path, so a verbose result stays
readable via `read` without consuming the next model request in full.

- dsh-spill: minimal SpillFiles seam (saveText → session-scoped SpillPath)
- dsh-spill-local: private 0700 session dirs, traversal-safe names, exclusive
  owner-only writes
- dsh-spill-policy: tools/post-execute transformer; no-op unless maxInlineBytes
  is set; skips read; best-effort on save failure (never turns a success into
  an isError)

web_fetch is the showcase — no tool-specific spill code. The coding-agent
example loads the stack so its keyless Loader smoke guards the namespace-plugin
export shape. Snapshot gap for a transcript-visible web_fetch spill is recorded
in the RFC's Consequences (ACP replay is keyless and cannot hit the web).
2026-07-08 20:41:55 +08:00
Dudu-0223 7a822ee402 feat(tool-web): declare web tool timeout budgets via config
fetchTimeoutMs/searchTimeoutMs (default 30000) resolve to each tool's
ToolDefinition.timeoutMs, moving the budget's declaration home onto the
owning tool plugin and preserving per-tool deployment override without a
mistypable central tool-name map.
2026-07-08 14:40:14 +08:00
Dudu-0223 a76285c4e6 Merge origin/master into timeout-design
Resolve conflicts from master's catalog/doc refactors landing alongside the
tool-call timeout work:
- knip.json: keep both new workspace entries (util/timeout + support/acp-snapshot).
- tool-web/src/fetch.ts: keep the timeout_ms removal, adopt master's richer
  JSDoc @param/@returns style on parseFetchArgs/presentFetchCall.
- tools/README.md: keep the tools/execute pipeline wording, adopt master's
  flattened docs/tool-catalog.md path.
- Regenerate every generated doc (cordis-catalog, tool-catalog, config-catalog,
  doc-graphs, module-graph) so they carry both master's changes and the
  tools/execute event + timeout-policy package.
- Add @param/@returns to toolTimeoutResult for master's new verify-export-jsdoc gate.
2026-07-08 11:18:27 +08:00
Dudu-0223 8190016e2b feat(timeout): add tools/execute seam + tool-timeout policy plugin
Model-facing tool-call budgets were tangled into each capability's schema
(bash timeoutMs, web_fetch timeout_ms) with no shared home. Add a
tools/execute around-dispatch waterfall to dsh-tools whose base next() is
the dispatch-with-normalization thunk, and a new @deepseek-ai/dsh-timeout-policy
plugin (packages/timeout/) that arms a per-tool deadline on exec.signal and
returns a structured TOOL_TIMEOUT when it wins. Migrate web_fetch (drop the
model-facing timeout_ms) and web_search onto it; the fetch provider keeps its
timeout only as a resource backstop for direct callers. bash and hook command
execution keep BASH_TIMEOUT unchanged.

Named the plugin timeout-policy (not the RFC's tool-timeout) so it does not
trip the gen-tool-catalog packages/*/tool-* completeness guard, and replace
exec.signal by in-place mutation before next() since cordis waterfall next()
ignores passed arguments. RFC moved to implemented/ recording both deviations.
2026-07-08 10:10:37 +08:00
Tianyi Cui cd9737d569 Gate JSDoc completeness on every package export
New doc-sync gate verify-export-jsdoc walks every module-level exported
name under packages/*/*/src and requires description prose everywhere,
plus @param per parameter and @returns on non-void annotated returns for
function-like exports, public class methods, properties, and accessors.
The parsing + check helpers move out of gen-cordis-catalog.ts into a
shared scripts/jsdoc.ts so 'documented' means one thing on both gated
surfaces.

Deliberate exemptions (documented in the RFC): heritage-declared class
members (the seam declaration is the doc's one home — the one checker
query in an otherwise pure-AST walk), cordis plugin-protocol slots
(name/inject/reusable/Config/apply, top-level and static), constructors,
overload implementations, declare-module augmentation bodies, and
re-export statements (checked at the defining module).

The 203 under-documented exports the gate found at adoption are filled
in this change, so the gate lands green; generated catalogs/graphs are
regenerated for the shifted line pointers.

RFC: docs/rfc/implemented/process/2026-07-06-export-surface-jsdoc-gate.md
2026-07-06 22:09:30 +08:00
Tianyi Cui f256f3961d feat(system-prompt): prompt variables, persona-as-section, tool-guidance ownership
One principle: every fact in the assembled prompt has exactly one owner.

- dsh-system-prompt: merge-extensible AssembleContext on assemble();
  a variable(name, provider) registry; {{name}} interpolation in
  renderPrompt, strict (unknown/valueless/malformed references throw);
  duplicate section and variable names rejected; assembly carries
  resolved section text + variables through the assemble waterfall.
- dsh-agent declares AssembleContext.agent; dsh-agent-loop registers
  the agent:persona section (order 0 - identity renders before tool
  guidance) and the model/cwd variables, and drops its string join:
  renderPrompt(assembly) IS the full prompt.
- Tool guidance moves to its owners: descriptions carry per-tool
  semantics; sections only cross-call habits (tool:bash exit-code
  habit at order 105; read's not-shell nudge). todo/subagent need no
  section - their descriptions already carry the contract.
- SubagentProvider.inheritsParentContext (spawn/acp false, fork true);
  dsh-tool-subagent derives truthful per-provider wording and resolves
  the provider at load (backend must be listed first).
- Example personas shrink to identity + behavior with {{model}} (and
  {{cwd}} in the ACP tree); the welcome banner stops enumerating tools.

RFC: docs/rfc/implemented/architecture/2026-07-05-prompt-variables-and-tool-guidance-ownership.md
2026-07-05 01:54:46 +08:00