Commit Graph
19 Commits
Author SHA1 Message Date
Tianyi Cui 9801a0d102 fix(tool-web): bound HTML conversion work 2026-07-27 16:40:02 +08:00
Tianyi Cui 109b469a7e fix(tool-web): bound conversion depth and complete fetch output
Two review findings on the turndown swap, both verified empirically:

- Unclosed-tag nesting makes the synchronous turndown/domino walk
  superlinear (measured: depth 512 ~0.15s, 2k ~2s, 20k ~5s), during
  which the cooperative fetchTimeoutMs timer cannot fire. renderBody
  now preflights nesting depth with a linear tag scan and passes
  bodies past 512 levels through raw; the try/catch stays for markup
  the scan cannot see (comment-hidden tags), simulated in tests via a
  converter throw.
- Markdown escaping can expand converted HTML ~2x (100k underscores
  render as 200k chars), so provider body caps no longer bounded the
  model-visible result. formatFetchOutput now caps the complete output
  (header + body + footer) under new fetchMaxOutputChars config
  (default 200000 = 2x the local provider's default body cap), reusing
  the truncation notice.

README EN+ZH, config catalog, Agent Note EN+ZH updated; the new
web-fetch fixture is migrated to the packed layout master now
requires; tool-web coverage stays 100% per-file.
2026-07-27 12:41:36 +08:00
Tianyi Cui 45a5175e44 feat(tool-web): replace the regex HTML-to-markdown converter with turndown
Implements the turndown Agent Note from the NIH dependency audit (full
variant, not the minimal entities-only fallback): dsh-tool-web's fetch
rendering now converts HTML through turndown + @joplin/turndown-plugin-gfm
(atx headings, fenced code, dash bullets, GFM tables/strikethrough) over
the real domino DOM, with script/style/noscript removed wholesale. The
hand-rolled ~86-line regex converter html.ts and its entity tables are
deleted; renderBody wraps the conversion in try/catch falling back to
the raw HTML body, because turndown's recursive DOM walk overflows with
a RangeError on pathological nesting (measured: 4k levels on the main
thread, 8k in a worker) where the regex version could never throw.

Closure weight, measured: tool-web IS in the single-exe runtime closure,
and the exe asset globs would pack ~7.9 MB of the three new packages —
but ~6 MB of that is domino's test corpus, with runtime lib/ at ~550 KB
against a ~174 MB artifact (<0.5% either way), so the swap wins.

Per testing policy the previously-missing keyless web_fetch snapshot
ships in the same change: the acp-agent `web-fetch` scenario boots a new
web.cordis.yml overlay (web seam + real dsh-web-fetch-local provider +
tool-web fetch-only + a loopback HTTP fixture server on a fixed port
serving deterministic HTML with entities, a GFM table, and nesting), so
recording and keyless replay both drive the real HTTP fetch and real
conversion end to end; the scenario pins the new `web` header class.

The Agent Note moves proposed -> implemented and is rewritten per the
lifecycle contract (Decision/Consequences/Testing, closure verdict and
alternatives recorded); tool-web and acp-agent READMEs updated in both
languages and pairs re-recorded.
2026-07-27 00:22:13 +08:00
Tianyi Cui 66c36e7325 feat: add canonical typed tool outputs 2026-07-21 03:22:14 +08:00
Tianyi Cui 21ec178841 docs: tighten parallel tool-call prose 2026-07-18 14:59:26 +08:00
Dudu-0223 3bed1a644f Merge origin/master into parallel-tool-call 2026-07-16 14:56:02 +08:00
Dudu-0223 77be4b891b Merge remote-tracking branch 'origin/master' into pr-265
# Conflicts:
#	docs/config-catalog.md
#	docs/cordis-catalog/services.md
#	docs/rfc/INDEX.md
#	examples/acp-agent/tests/snapshots/both-mode-turn/session.jsonl
#	examples/acp-agent/tests/snapshots/skill-load/session.jsonl
#	examples/acp-agent/tests/snapshots/text-turn/session.jsonl
#	examples/sandbox-acp-agent/tests/snapshots/escalation-approved/session.jsonl
#	examples/sandbox-acp-agent/tests/snapshots/escalation-rejected/session.jsonl
#	packages/core/agent-loop/README.md
#	packages/core/agent-loop/src/loop.ts
#	packages/core/tools/README.md
#	packages/core/tools/src/index.ts
#	packages/core/tools/src/schema.ts
#	packages/ui/acp/src/index.ts
#	packages/ui/stdio-agent/README.md
2026-07-14 20:49:54 +08:00
Tianyi Cui 92c52eec20 Merge remote-tracking branch 'origin/master' into codex/simp-prune-web-seam-fields
# Conflicts:
#	docs/config-catalog.md
#	docs/cordis-catalog/services.md
#	docs/core-data-structures/web.md
#	docs/rfc/implemented/simplification/2026-07-04-drop-unconsumed-web-observation-surface.md
#	packages/web/tool-web/tests/integration.spec.ts
#	packages/web/web-fetch-local/README.md
#	packages/web/web-fetch-local/src/provider.ts
#	packages/web/web/src/types.ts
2026-07-14 17:57:50 +08:00
Tianyi Cui 3ab35de64f refactor: prune unused web seam fields 2026-07-14 04:17:38 +08:00
Tianyi Cui 148046b9c8 docs: rebalance prose cleanup and add trimming skill 2026-07-13 23:27:00 +08:00
Dudu-0223 7ea1bf119f feat(agent-loop): run safe tool calls in parallel 2026-07-13 11:13:21 +08:00
Tianyi Cui 75838e10b5 docs: trim generated prose 2026-07-12 03:36:43 +08:00
Dudu-0223 7a822ee402 feat(tool-web): declare web tool timeout budgets via config
fetchTimeoutMs/searchTimeoutMs (default 30000) resolve to each tool's
ToolDefinition.timeoutMs, moving the budget's declaration home onto the
owning tool plugin and preserving per-tool deployment override without a
mistypable central tool-name map.
2026-07-08 14:40:14 +08:00
Dudu-0223 a76285c4e6 Merge origin/master into timeout-design
Resolve conflicts from master's catalog/doc refactors landing alongside the
tool-call timeout work:
- knip.json: keep both new workspace entries (util/timeout + support/acp-snapshot).
- tool-web/src/fetch.ts: keep the timeout_ms removal, adopt master's richer
  JSDoc @param/@returns style on parseFetchArgs/presentFetchCall.
- tools/README.md: keep the tools/execute pipeline wording, adopt master's
  flattened docs/tool-catalog.md path.
- Regenerate every generated doc (cordis-catalog, tool-catalog, config-catalog,
  doc-graphs, module-graph) so they carry both master's changes and the
  tools/execute event + timeout-policy package.
- Add @param/@returns to toolTimeoutResult for master's new verify-export-jsdoc gate.
2026-07-08 11:18:27 +08:00
Dudu-0223 8190016e2b feat(timeout): add tools/execute seam + tool-timeout policy plugin
Model-facing tool-call budgets were tangled into each capability's schema
(bash timeoutMs, web_fetch timeout_ms) with no shared home. Add a
tools/execute around-dispatch waterfall to dsh-tools whose base next() is
the dispatch-with-normalization thunk, and a new @deepseek-ai/dsh-timeout-policy
plugin (packages/timeout/) that arms a per-tool deadline on exec.signal and
returns a structured TOOL_TIMEOUT when it wins. Migrate web_fetch (drop the
model-facing timeout_ms) and web_search onto it; the fetch provider keeps its
timeout only as a resource backstop for direct callers. bash and hook command
execution keep BASH_TIMEOUT unchanged.

Named the plugin timeout-policy (not the RFC's tool-timeout) so it does not
trip the gen-tool-catalog packages/*/tool-* completeness guard, and replace
exec.signal by in-place mutation before next() since cordis waterfall next()
ignores passed arguments. RFC moved to implemented/ recording both deviations.
2026-07-08 10:10:37 +08:00
Tianyi Cui cd9737d569 Gate JSDoc completeness on every package export
New doc-sync gate verify-export-jsdoc walks every module-level exported
name under packages/*/*/src and requires description prose everywhere,
plus @param per parameter and @returns on non-void annotated returns for
function-like exports, public class methods, properties, and accessors.
The parsing + check helpers move out of gen-cordis-catalog.ts into a
shared scripts/jsdoc.ts so 'documented' means one thing on both gated
surfaces.

Deliberate exemptions (documented in the RFC): heritage-declared class
members (the seam declaration is the doc's one home — the one checker
query in an otherwise pure-AST walk), cordis plugin-protocol slots
(name/inject/reusable/Config/apply, top-level and static), constructors,
overload implementations, declare-module augmentation bodies, and
re-export statements (checked at the defining module).

The 203 under-documented exports the gate found at adoption are filled
in this change, so the gate lands green; generated catalogs/graphs are
regenerated for the shifted line pointers.

RFC: docs/rfc/implemented/process/2026-07-06-export-surface-jsdoc-gate.md
2026-07-06 22:09:30 +08:00
Tianyi Cui 8ca82d03ab refactor(tool-web): port web tools to the render-intent union
master's web_search/web_fetch tools were authored against the old
ToolCallPresentation bag; the render-intent union replaces it with a
card-tagged discriminated union. Both are simple generic cards, so they
declare card:'generic' explicitly.
2026-07-03 22:52:18 +08:00
Dudu-0223 70a8b57738 fix: drop tool-web subpath exports, align with tool-bash single-entry shape
The web tool package exposed ./search and ./fetch as standalone subpath
plugins, but nothing consumed them, the RFC never called for them, and the
sibling dsh-tool-bash (also a multi-tool consumer) ships a single entry and
selects tools via config. The extra entries also tripped the workspace
constraints gate, whose expected `files` list covers single-entry and bin
packages but not a non-bin multi-entry one.

Collapse to a single `.` entry: drop the ./search|./fetch exports and their
lib/*.js from package.json files, delete the per-package tsdown override (the
root config's lib/types/index.js entry now suffices), and remove the
plugin-shaped name/inject exports from search.ts/fetch.ts (renaming each
apply to its applyWeb{Search,Fetch}Tool helper, still composed by the root
plugin and re-exported from the index). Selective enablement stays via the
existing { search?, fetch? } config. Docs updated to match.
2026-06-28 17:02:06 +08:00
Dudu-0223 d01f5f73b7 Add web capability seam: ctx.web, search/fetch providers, web tools
Introduce web access as a first-class capability seam so the model-facing
web tools stay stable while backends change. dsh-web owns ctx.web as a
provider registry with registration-order-independent selection and the
WebError taxonomy; dsh-web-search-exa, dsh-web-search-perplexity, and
dsh-web-fetch-local register capabilities into it; dsh-tool-web is the sole
owner of the model-facing web_search/web_fetch schemas, prompt sections, and
HTML-to-markdown presentation. Search and fetch are deliberately one seam.

Providers ship as namespace plugins that register into ctx.web (like an
LlmAdapter into ctx.llm), not key-owning services, since multiple search
providers cannot each own the key. Tool registration follows product
enablement, not backend availability, so load order/credentials never enter
the model contract; the seam resolves the provider at execution time and
surfaces a structured WebError otherwise.

Moves the RFC to implemented/ amended to match what shipped. Example/app
configs are intentionally not wired yet (RFC migration step 6).
2026-06-26 19:12:13 +08:00