Commit Graph
17 Commits
Author SHA1 Message Date
Tianyi Cui 9801a0d102 fix(tool-web): bound HTML conversion work 2026-07-27 16:40:02 +08:00
Tianyi Cui 109b469a7e fix(tool-web): bound conversion depth and complete fetch output
Two review findings on the turndown swap, both verified empirically:

- Unclosed-tag nesting makes the synchronous turndown/domino walk
  superlinear (measured: depth 512 ~0.15s, 2k ~2s, 20k ~5s), during
  which the cooperative fetchTimeoutMs timer cannot fire. renderBody
  now preflights nesting depth with a linear tag scan and passes
  bodies past 512 levels through raw; the try/catch stays for markup
  the scan cannot see (comment-hidden tags), simulated in tests via a
  converter throw.
- Markdown escaping can expand converted HTML ~2x (100k underscores
  render as 200k chars), so provider body caps no longer bounded the
  model-visible result. formatFetchOutput now caps the complete output
  (header + body + footer) under new fetchMaxOutputChars config
  (default 200000 = 2x the local provider's default body cap), reusing
  the truncation notice.

README EN+ZH, config catalog, Agent Note EN+ZH updated; the new
web-fetch fixture is migrated to the packed layout master now
requires; tool-web coverage stays 100% per-file.
2026-07-27 12:41:36 +08:00
Tianyi Cui 45a5175e44 feat(tool-web): replace the regex HTML-to-markdown converter with turndown
Implements the turndown Agent Note from the NIH dependency audit (full
variant, not the minimal entities-only fallback): dsh-tool-web's fetch
rendering now converts HTML through turndown + @joplin/turndown-plugin-gfm
(atx headings, fenced code, dash bullets, GFM tables/strikethrough) over
the real domino DOM, with script/style/noscript removed wholesale. The
hand-rolled ~86-line regex converter html.ts and its entity tables are
deleted; renderBody wraps the conversion in try/catch falling back to
the raw HTML body, because turndown's recursive DOM walk overflows with
a RangeError on pathological nesting (measured: 4k levels on the main
thread, 8k in a worker) where the regex version could never throw.

Closure weight, measured: tool-web IS in the single-exe runtime closure,
and the exe asset globs would pack ~7.9 MB of the three new packages —
but ~6 MB of that is domino's test corpus, with runtime lib/ at ~550 KB
against a ~174 MB artifact (<0.5% either way), so the swap wins.

Per testing policy the previously-missing keyless web_fetch snapshot
ships in the same change: the acp-agent `web-fetch` scenario boots a new
web.cordis.yml overlay (web seam + real dsh-web-fetch-local provider +
tool-web fetch-only + a loopback HTTP fixture server on a fixed port
serving deterministic HTML with entities, a GFM table, and nesting), so
recording and keyless replay both drive the real HTTP fetch and real
conversion end to end; the scenario pins the new `web` header class.

The Agent Note moves proposed -> implemented and is rewritten per the
lifecycle contract (Decision/Consequences/Testing, closure verdict and
alternatives recorded); tool-web and acp-agent READMEs updated in both
languages and pairs re-recorded.
2026-07-27 00:22:13 +08:00
Tianyi Cui 7d3cb39047 Merge branch 'codex/tool-json-schema-dsl' into codex/canonical-tool-output
# Conflicts:
#	.agents/notes/implemented/feature/2026-06-30-interception-seams.md
#	docs/config-catalog.md
#	docs/cookbook/adding-a-tool.i18n.yaml
#	docs/cookbook/adding-a-tool.md
#	docs/cookbook/adding-a-tool.zh.md
#	docs/cordis-catalog/events.md
#	docs/cordis-catalog/services.md
#	docs/core-data-structures/tools.md
#	docs/event-producer-consumer.md
#	docs/persistence-catalog.md
#	examples/acp-agent/tests/snapshots/cordis-inspect-jsdoc/session.jsonl
#	examples/acp-agent/tests/snapshots/cordis-inspect-jsdoc/stdout.expected.jsonl
#	packages/bash/tool-bash/src/index.ts
#	packages/core/agent-loop/src/tool-calls.ts
#	packages/core/agent-loop/tests/cancel.spec.ts
#	packages/core/agent-loop/tests/contract-regressions.spec.ts
#	packages/core/agent-loop/tests/tool-calls.spec.ts
#	packages/core/tools/README.md
#	packages/core/tools/src/index.ts
#	packages/core/tools/tests/code-mode.spec.ts
#	packages/core/tools/tests/tools.spec.ts
#	packages/fs/tool-fs-search/tests/integration.spec.ts
#	packages/fs/tool-fs-search/tests/tools.spec.ts
#	packages/fs/tool-fs/tests/integration.spec.ts
#	packages/mcp/mcp-client/src/tools.ts
#	packages/timeout/timeout-policy/tests/timeout-policy.spec.ts
#	packages/web/tool-web/tests/integration.spec.ts
#	packages/web/tool-web/tests/tool-web.spec.ts
2026-07-21 23:39:03 +08:00
Tianyi Cui 66c36e7325 feat: add canonical typed tool outputs 2026-07-21 03:22:14 +08:00
Tianyi Cui e8b95c8754 feat(tools): require cancellation signal on every invocation 2026-07-19 23:38:54 +08:00
Dudu-0223 3bed1a644f Merge origin/master into parallel-tool-call 2026-07-16 14:56:02 +08:00
Tianyi Cui 3ab35de64f refactor: prune unused web seam fields 2026-07-14 04:17:38 +08:00
Dudu-0223 7ea1bf119f feat(agent-loop): run safe tool calls in parallel 2026-07-13 11:13:21 +08:00
Dudu-0223 7a822ee402 feat(tool-web): declare web tool timeout budgets via config
fetchTimeoutMs/searchTimeoutMs (default 30000) resolve to each tool's
ToolDefinition.timeoutMs, moving the budget's declaration home onto the
owning tool plugin and preserving per-tool deployment override without a
mistypable central tool-name map.
2026-07-08 14:40:14 +08:00
Dudu-0223 8190016e2b feat(timeout): add tools/execute seam + tool-timeout policy plugin
Model-facing tool-call budgets were tangled into each capability's schema
(bash timeoutMs, web_fetch timeout_ms) with no shared home. Add a
tools/execute around-dispatch waterfall to dsh-tools whose base next() is
the dispatch-with-normalization thunk, and a new @deepseek-ai/dsh-timeout-policy
plugin (packages/timeout/) that arms a per-tool deadline on exec.signal and
returns a structured TOOL_TIMEOUT when it wins. Migrate web_fetch (drop the
model-facing timeout_ms) and web_search onto it; the fetch provider keeps its
timeout only as a resource backstop for direct callers. bash and hook command
execution keep BASH_TIMEOUT unchanged.

Named the plugin timeout-policy (not the RFC's tool-timeout) so it does not
trip the gen-tool-catalog packages/*/tool-* completeness guard, and replace
exec.signal by in-place mutation before next() since cordis waterfall next()
ignores passed arguments. RFC moved to implemented/ recording both deviations.
2026-07-08 10:10:37 +08:00
Tianyi Cui f256f3961d feat(system-prompt): prompt variables, persona-as-section, tool-guidance ownership
One principle: every fact in the assembled prompt has exactly one owner.

- dsh-system-prompt: merge-extensible AssembleContext on assemble();
  a variable(name, provider) registry; {{name}} interpolation in
  renderPrompt, strict (unknown/valueless/malformed references throw);
  duplicate section and variable names rejected; assembly carries
  resolved section text + variables through the assemble waterfall.
- dsh-agent declares AssembleContext.agent; dsh-agent-loop registers
  the agent:persona section (order 0 - identity renders before tool
  guidance) and the model/cwd variables, and drops its string join:
  renderPrompt(assembly) IS the full prompt.
- Tool guidance moves to its owners: descriptions carry per-tool
  semantics; sections only cross-call habits (tool:bash exit-code
  habit at order 105; read's not-shell nudge). todo/subagent need no
  section - their descriptions already carry the contract.
- SubagentProvider.inheritsParentContext (spawn/acp false, fork true);
  dsh-tool-subagent derives truthful per-provider wording and resolves
  the provider at load (backend must be listed first).
- Example personas shrink to identity + behavior with {{model}} (and
  {{cwd}} in the ACP tree); the welcome banner stops enumerating tools.

RFC: docs/rfc/implemented/architecture/2026-07-05-prompt-variables-and-tool-guidance-ownership.md
2026-07-05 01:54:46 +08:00
Tianyi Cui 1905c809e3 Merge remote-tracking branch 'origin/master' into simpl-c-web-observation
# Conflicts:
#	docs/architecture.md
#	packages/web/web/src/index.ts
2026-07-04 20:54:42 +08:00
Tianyi Cui 774d460889 Expose audited hardcoded tunables as plugin config
The audit swept every packages/*/* plugin for the new AGENTS.md
convention (no hardcoded tunables in plugins) and exposes each finding
as a defaulted, validated Config field. Defaults are the previously
hardcoded values throughout, so no deployment or golden changes.

- tool-fs (had NO Config): readLimit, readMaxLineLength, readMaxBytes,
  readStreamMinSize. The caps thread through ReadToolCaps/ReadWindow —
  read-render already documented that the consumer applies the caps, so
  they become explicit per-request fields.
- tool-web: searchMaxResults (WEB_SEARCH_MAX_RESULTS stays as the
  schemastery default). Also fixes the stale GREP_LIMIT references in
  search.ts and the web-capability-seam RFC (no such constant exists).
- bash-local: graceMs (SIGTERM->SIGKILL escalation grace). The
  RunInternals.graceMs test seam is gone: graceMs is now a required
  SpawnSpec field filled from config, so tests exercise the real
  config path and the defaults live in exactly one place.
- subagent-acp: disposeEofGraceMs / disposeGraceMs. The AcpRunSpec
  fields become required for the same one-defaulting-layer reason.
- session-persistence-sqlite: journalMode ('wal' default; the
  rollback-journal modes serve filesystems where WAL's shared-memory
  files do not work, e.g. network mounts).
- hooks-claude + hooks-codex: stderrSummaryMaxChars for the persisted
  hook/result stderr summary. The duplicated summarize() helpers merge
  into hook-protocol's summarizeStderr(stderr, maxChars), beside the
  HookResultRecord field it feeds, with the bound parameterized the
  same way runHook's defaultTimeoutMs already is.
- compact-basic: charsPerToken for the token estimator (default 4, the
  English-text heuristic; CJK-heavy deployments need ~1-2 or compaction
  fires far too late). Also corrects the BasicCompactService class doc,
  which claimed defaults the required-field config never had.
- fs-local: deletes the dead STREAM_MIN_SIZE constant and the dead
  FsIoInternals.streamMinSize seam — the read-routing bound lives in
  the consumer (tool-fs), where it is now config. This is item 1 of
  the proposed prune-write-only-fs-surface RFC, annotated accordingly.

Every new field gets range validation (following the existing
assertPositiveFinite pattern), a README row, and tests covering the
configured behavior, the schema default, and load-time rejection.
2026-07-04 17:37:23 +08:00
Tianyi Cui de79553d4f refactor(web): drop the unconsumed observation surface
WebService exposed an observation surface nothing in production observes:
the web/providers-change event (declared, emitted on every provider
registration/disposal, rollback-yield ordered before the emit solely so a
throwing change listener unwinds the registration) and the aggregated
searchStatus()/fetchStatus() query with its WebCapabilityStatus union.
dsh-tool-web executes through ctx.web.search()/fetch() and routes on the
structured WebError codes selection throws at execution time; tool
registration follows product enablement, not provider availability. The
only listeners/callers were the web packages' own tests, and the
tool-web README / architecture.md prose claiming the tool 'reads only the
aggregated searchStatus()/fetchStatus()' had drifted from the call sites.

Remove the event declaration, both emits, and the rollback-before-emit
machinery (the plain ctx.effect disposer keeps HMR cleanup, matching
LlmService.registerAdapter). Remove searchStatus()/fetchStatus(),
resolveStatus(), and WebCapabilityStatus; the provider-private status()
stays as the execution-time selection input. Delete the listener-throw
rollback test, and rewrite every event/status assertion across the web
packages' tests onto caller-observable behavior: a successful
search()/fetch() or the structured WEB_PROVIDER_* codes. Regenerate the
cordis catalog; update the web/tool-web READMEs, the architecture.md web
paragraph, core-data-structures/web.md, and the type-equiv manifest; amend
the web capability seam RFC's facts to the shipped surface. This follows
the llm/adapter-change precedent: a boot-time backend-registry signal and
an availability probe distinct from executing both sit on the cut side of
its keep/cut criterion.

RFC: docs/rfc/implemented/simplification/2026-07-04-drop-unconsumed-web-observation-surface.md
2026-07-04 15:42:56 +08:00
Tianyi Cui 8ca82d03ab refactor(tool-web): port web tools to the render-intent union
master's web_search/web_fetch tools were authored against the old
ToolCallPresentation bag; the render-intent union replaces it with a
card-tagged discriminated union. Both are simple generic cards, so they
declare card:'generic' explicitly.
2026-07-03 22:52:18 +08:00
Dudu-0223 d01f5f73b7 Add web capability seam: ctx.web, search/fetch providers, web tools
Introduce web access as a first-class capability seam so the model-facing
web tools stay stable while backends change. dsh-web owns ctx.web as a
provider registry with registration-order-independent selection and the
WebError taxonomy; dsh-web-search-exa, dsh-web-search-perplexity, and
dsh-web-fetch-local register capabilities into it; dsh-tool-web is the sole
owner of the model-facing web_search/web_fetch schemas, prompt sections, and
HTML-to-markdown presentation. Search and fetch are deliberately one seam.

Providers ship as namespace plugins that register into ctx.web (like an
LlmAdapter into ctx.llm), not key-owning services, since multiple search
providers cannot each own the key. Tool registration follows product
enablement, not backend availability, so load order/credentials never enter
the model contract; the seam resolves the provider at execution time and
surfaces a structured WebError otherwise.

Moves the RFC to implemented/ amended to match what shipped. Example/app
configs are intentionally not wired yet (RFC migration step 6).
2026-06-26 19:12:13 +08:00