a706ec7966041e468da5bd1a24f05b3fced3f58a
13
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Tianyi Cui
|
4641f1a851 |
chore: absorb the no-nudge seam into the workflow tree
The parent branch dropped structuredNudgeRetries; the two workflow-side test setups stop passing it, and the RFC's foundation paragraph now describes the current design (final-ASSEMBLY enforcement logged via request/header, the post-capture pre-execute deny, the start() schema snapshot, no re-prompt) instead of the retired agent/request + nudge shape. |
||
|
Tianyi Cui
|
db4a39b024 |
workflow: close the review-found cancellation and child-lifecycle gaps
Three review findings on the engine's child seam, one mechanism each: - Cancellation now bridges to run.cancel() on every in-flight child, not just the shared request signal — the subagent seam leaves a provider free to honor either channel, so the consumer drives both (listener removed in the child finally). - A child result REJECTION (an infrastructure fault the seam allows) now emits the paired workflow/agent-end before propagating, and propagates as a fatal WorkflowError with the new AGENT_RESULT code — previously it skipped agent-end (permanently open child for seq-matching observers) and dissolved to a per-item null inside parallel()/pipeline(), letting a broken provider read as an ordinary failed child. A rejection landing after cancel stays a cancellation (cancelled outcome + CANCELLED). - Every hook now guards its entry with a shared throwIfCancelled(): phase()/log() no longer emit observer events after a script caught an earlier cancelled rejection, and parallel()/pipeline() refuse entry — cancellation is the next HOOK boundary, not just the next agent(). |
||
|
Tianyi Cui
|
e70227d581 |
workflow: pin the spec engine's concurrency — the auto default is one slot on small CI runners
The two tests that wait for two children IN FLIGHT (the pipeline no-barrier test and the dropped-promise cancellation test) inherited the engine's auto-resolved maxConcurrentAgents, which is min(16, max(1, cores - 2)) — exactly 1 on the 2-core CI runner, so the second child never started and vi.waitFor timed out. Reproduced locally under taskset -c 0,1; the full spec passes there with the helper pinning a fixed ceiling. Tests about the ceiling itself keep their explicit overrides, and the auto-resolve arm stays covered by the default-config provider tests. |
||
|
Tianyi Cui
|
a8986c2c8a |
Merge remote-tracking branch 'origin/master' into worktree-dynamic-workflows
Beyond the mechanical conflicts (provider capability lines vs master's new inheritsParentContext field; generated catalogs regenerated rather than hand-merged; knip/lockfile), three master-side reworks required semantic adaptation of this branch: - The persona rework removed AgentOptions.systemPrompt, which was the structured-output instruction's channel. The instruction now rides the SAME final-request enforcement listener that injects the schema'd tool: appended per request to final.system (per-request wire state, not agent prompt state). Tests assert the wire request (adapter.requests) instead of child.options; the bare-direct-dispatch test pins the no-system arm. - Tool guidance moved out of deployment prompts into per-tool prompt sections; the examples' workflow paragraph became a tool:<toolName> section contributed by dsh-tool-workflow (explicit-ask-only policy), and both example personas resolve to master's minimal identity+behavior form. tool-workflow gains inject: systemPrompt (+ peer dep, tsconfig ref); the export-shape guard updated. - The uniform-RFC-format gate: the dynamic-workflows RFC restructured to the implemented/ skeleton (bare Status line; Proposal -> Decision; What-was-rejected -> Alternatives considered; new Consequences), and the overall-run-timeout deferral is now recorded in the RFC's Deferred list. The doc-graphs atlas classification gains the workflows seam (workflow-vm implementation, tool-workflow consumer). Master's harness-identity section made "empty assembled prompt" states unreachable through the loop, so the instruction-append is a plain undefined-ternary and the structured tests assert append-not-replace. All snapshot goldens (including workflow-run) replay unchanged. Full local CI-equivalent gate sequence green on the merged tree. |
||
|
Tianyi Cui
|
706691a7df |
workflow: re-check cancellation after the slot acquire
Codex convergence round 1 on the review-response commits: agent()'s post-acquire window was real and unguarded. `await acquireSlot()` yields at least one microtask tick even when a slot is free (and a queued waiter resumes a tick after its release), so a cancel() landing in that tick let the continuation start a child carrying an ALREADY-aborted signal — the in-code comment claimed the window could not exist. A provider that subscribes only to future abort events (the test stub; the seam does not promise pre-aborted-signal handling) would never settle such a child, leaking it until the dispose grace abandoned the run, and a backend that misses the pre-aborted signal would burn a full model turn after the user cancelled. agent() now re-checks isCancelled() immediately after the acquire (inside the slot-owning try, so the finally still releases), making every post-cancel path reject before subagents.start. New deterministic regression: cancel() in the same synchronous frame as start() lands in the free-slot await tick — the run settles cancelled with ZERO children started (previously: one leaked child and a grace-delayed settle). The raced-release test's comment now states what it actually pins (the queued-waiter rejection path). Also aligns the RFC's auto-concurrency formula with the code (min(16, max(1, availableParallelism() - 2))). |
||
|
Tianyi Cui
|
ed3972a9c6 |
workflow: linear meta-prefix scan (the regex backtracked exponentially)
Review finding, measured: the leading-trivia prefix regex (`^\s*(?:comment|comment|\s+)*export …`) partitions a whitespace run ambiguously between its outer `\s*` and the starred `\s+` alternative, so a script that ultimately FAILS the match backtracks exponentially — ~19 ms at 35 leading whitespace characters, ~174 ms at 38, ×2.2 per character; a realistic near-miss (a comment header, blank indented lines, then `const meta` missing its `export`) did not finish in 10 seconds. The regex ran on the HOST stack inside the synchronous `start()`, where no vm timeout applies and no abort can interleave — a benign one-token typo, exactly what SCRIPT_PARSE exists to bounce back to the model, hung the whole process instead of reaching that designed recovery. Replaced with a hand-rolled linear trivia scan (whitespace + `//` and `/* */` comments — the module already scans characters for the literal) followed by an anchored `^export\s+const\s+meta\s*=\s*` on the remainder, whose quantifiers cannot backtrack ambiguously. An unterminated block comment before the statement now gets its own SCRIPT_PARSE message. Regressions: the near-miss shape must reject in under a second (the old regex would trip the suite timeout), plus the unterminated-leading-comment and comment-to-EOF edges. |
||
|
Tianyi Cui
|
2accf85714 |
workflow: simplify to the trust premise; settle result on cancellation
Two review responses that belong together — the same review argued the
engine was defending the wrong threat while a benign-input bug wedged
the product.
1) Drop hostile-value containment; state the trust premise.
Scripts are model-written — the same trust level as the model's bash
access — yet successive pre-push review rounds had ratcheted in defenses
that only matter against an adversarial author: trap-free proxy
rejection, accessor-never-invoked descriptor walks, realm-side
pre-rendering of thrown values, realm-built promises/arrays/error clones
with structural fatal recognition. That same author keeps a documented,
accepted, unkillable event-loop spin, so containing its error VALUES is
cost without a threat model — and the planned hardened engine
(worker/isolated-vm) gets value isolation by serialization and deletes
all of this machinery anyway.
What stays, because benign scripts hit it constantly: result never
rejects; dropped hook promises cannot become unhandled rejections; the
value boundary rejects LOUD everything JSON cannot carry (now a plain
recursive walk — getters are read ordinarily and their result is what
crosses; a throwing read fails loud); a "__proto__" key still copies as
a data property; the fatal-vs-null combinator discipline (now host
instanceof — unforgeable from the realm and simpler than clone-shape
recognition). What changes for scripts (documented in the engine
README): hooks hand back host values and host errors — in-script
`instanceof Error` on a hook failure is false (branch on e.name/e.code)
— and args are host-cloned once so a script cannot mutate the caller's
object. realm.ts drops 289 → 173 lines; the hostile-value test tables go
with it. The premise now leads the engine module doc, the README, and
the RFC's engine section, with the removed machinery recorded under
What was rejected.
2) result settles within the dispose grace of a cancellation.
Review finding (verified through the real registry + tool + engine): a
script parked on a promise no hook owns — `await new Promise(() => {})`,
`await Promise.race([])`, a returned never-settling thenable — could not
be settled by cancel(): hooks reject and children abort, but nothing
touches a promise the engine does not own, so `result` stayed pending
FOREVER (the previous cut even pinned that as intended). The tool awaits
run.result BEFORE its disposing finally, the registry awaits the tool,
the loop awaits the registry — one such script wedged the whole agent
turn past any abort, unrecoverable in-process; the mock engine in the
tool's abort test settles result on cancel, which is exactly the
behavior the real engine lacked, so no existing test could see it.
The seam contract now says it out loud: once a run is cancelled, result
SETTLES within the implementation's bounded grace even if the script
never does. The vm engine arms an abandon channel in cancel(); drive()
races the script against it, force-settling 'cancelled' at the grace
(the abandoned settlement stays contained; a post-slice synchronous spin
remains the documented limitation). dispose()'s outer race now exists
for child quiescence only, and `workflow/end` again fires exactly once
per started run. The old 'result stays pending' pin is FLIPPED to the
new contract (the pinned behavior was the bug); new regressions cover
cancel-then-settle on a parked script, a never-settling returned
thenable, and the full composition through the REAL registry + tool +
vm engine (tool-workflow gains workflow-vm/subagent devDeps for it).
agentsStarted JSDoc clarified while touching the vocabulary (accepted
calls, including ones still queued at cancellation).
|
||
|
Tianyi Cui
|
7234d41b91 |
workflow: hook promises and hook failures are realm-built too
Codex code-review round 5: agent()/parallel()/pipeline() returned HOST Promise
objects into the script realm — Object.getPrototypeOf(agent('x')) reached host
Promise.prototype, contradicting the realm contract (correctness containment,
not the accepted sandbox stance). The rejection channel had the same leak one
hop away: a caught hook failure was a host WorkflowError (host Error.prototype
chain), and phase()/log() threw host errors synchronously.
All three surfaces are realm-built now:
- hook promises: the realm's own Promise.resolve (bound at context setup)
assimilates the host promise, so the script-visible promise carries realm
prototypes; the realm promise gets the same no-op rejection consumer as the
host one (a script may drop it).
- hook failures: rejections and phase/log sync throws are translated at the
boundary into realm-built clones (name/code/message/fatal via an in-realm
factory); non-WorkflowError host failures become generic realm Errors
carrying their describeThrown rendering.
- the combinators recognize FATAL clones structurally
(isFatalWorkflowErrorClone: proxy-guarded descriptor reads), preserving the
fatal-vs-null discipline across the boundary; a script forging the shape
kills only its own run. drive() maps any post-cancel failure to 'cancelled'
by run state (a CANCELLED clone deliberately fails the host instanceof).
Tests: realm-promise identity for all three hooks + host Promise.prototype
pollution unreachable; clone shape (instanceof realm Error, name/code/fatal/
message) with prototype-chain mutation staying realm-side; a rejecting
provider result crossing as a generic clone; phase/log sync-throw clones;
combinator catch branches (string throw, proxy throw, shape-miss forgery →
null; forged fatal → kills own run); existing fatal-propagation, cancellation,
and unhandled-rejection tests as canaries.
|
||
|
Tianyi Cui
|
95c8c878e1 |
workflow: pin thenable-return semantics as documented async-JS behavior
Codex code-review round 4 flagged the return channel: an async IIFE Promise-assimilates a returned thenable, so its then() runs past the sync slice and the RESOLUTION replaces the raw object. Verified against the real engine and judged behavior, not defect: - Assimilation is standard JavaScript (an async function's returned thenable resolves before the caller sees it) and is load-bearing ergonomics: an un-awaited 'return agent(...)' / 'return parallel(...)' resolves to the intended value precisely because of it. Rejecting callable-then returns would break that; intercepting pre-assimilation is spec-impossible (the Get(v,'then') and job enqueue are internal to promise resolution). - The realm-boundary guard applies to the RESOLUTION (a thenable resolving to non-JSON is still RESULT_UNSERIALIZABLE), so nothing crosses unmaterialized. - A spin inside a returned thenable's then() is the same accepted class as any post-slice spin (it runs on the microtask queue, past the vm timeout's reach); the docs previously said 'after the first await', which was too narrow — reworded to 'past the initial synchronous slice (an await continuation, or a thenable's then invoked by promise resolution)'. Pinned with an engine test (un-awaited return agent(); custom thenable resolution as the return value; thenable resolving to non-JSON rejects), and the limitation wording updated in the module doc, README, and RFC. |
||
|
Tianyi Cui
|
fff2e1f33d |
workflow: render thrown script values inside the realm's execution window
Codex code-review round 3: the round-2 'contained stack getter' still let a
script escape the vm sync-slice timeout — throw { get stack() { while(true){} } }
put the spin on the HOST catch path, where no timeout applies (verified: a
direct sync-slice spin dies by the timeout; the getter-hidden one hung the
process). Identity-trusting the native getter is also insufficient: V8 stack
formatting reads script-controllable hooks at format time (Error.prepareStackTrace,
a subclass name getter — both empirically confirmed), so ANY host-side
formatting of a realm error can run realm code.
The fix moves rendering into the realm itself: the compiled body (and the meta
literal) is wrapped in a realm-side catch that pre-renders the thrown value to
a string (REALM_THROWN_RENDERER_SOURCE) — a hostile accessor/toString now runs
as ordinary script code, killed by the sync-slice timeout or falling under the
documented post-await spin limitation; host WorkflowErrors pass through for
the CANCELLED mapping. The host catch descriptor-reads the pre-rendered string
(thrownRendering) or falls back to describeThrown, which invokes no getter
whose identity is not the host realm's own native stack getter.
Tests: hostile-table expectations updated for realm-side rendering; new
regressions for the getter-hidden sync spin dying by the vm timeout (engine +
meta paths) and for a hostile thenable rejection that bypasses the realm
wrapper (renders host-side, proxy labelled, traps never run); describeThrown/
thrownRendering unit tables including the realm-error identity-mismatch case.
|
||
|
Tianyi Cui
|
57b9910339 |
workflow: total, contained rendering of hostile thrown script values
Codex code-review round 2: errorText() read .stack/.message as plain property gets and fell back to String(error) — a script throwing a value with a throwing accessor (or toString/Symbol.toPrimitive) ran realm code in drive()'s catch and made WorkflowRun.result REJECT, which the detached workflow/end hook turned into an unhandledRejection (process death under dsh-app-boot). Replaced with describeThrown in dsh-workflow-vm/realm: total (never throws), proxy-labelling before any inspection, own-descriptor reads, String() only on primitives, and a CONTAINED stack-getter invocation — modern V8 (Node >= 22) makes stack an own ACCESSOR on genuine Errors, so refusing all accessors would lose every real stack and the lineOffset line numbers; a hostile getter's throw is swallowed and rendering falls back to message. The meta-literal eval catch had the same String(error) exposure and now uses the same renderer. Regression tests: a hostile-thrown-values table through the real engine (throwing stack/message getters, data stack, setter-only stack, proxy, Symbol.toPrimitive, function, null) asserting result resolves 'error' with the expected rendering and NO unhandledRejection fires; a meta-path hostile throw mapping to META_INVALID. |
||
|
Tianyi Cui
|
e264a106fd |
workflow, subagent: fix Codex code-review round-1 blockers
Six verified A-findings from the code-stage review, each with a regression test: - parallel()/pipeline() resolved to HOST arrays inside the vm realm, exposing host Array.prototype to scripts; combinator results are now realm-built (in-realm Array.from bound at context setup). - materializeFromRealm ran proxy traps (ownKeys/getOwnPropertyDescriptor/ getPrototypeOf) during the descriptor walk — realm code on the host stack, outside the vm timeout, escaping as raw errors; proxies (root, nested, and in the prototype position) are now rejected trap-free via util.types.isProxy before any inspection. - an already-aborted signal or an immediate cancel() no longer reports 'completed' for a hook-free script: drive() checks cancellation before running the body and again when the script settles. - dispose() now waits (bounded by disposeGraceMs) for stray agent() children to FINISH disposing, not just for the script to settle: every agent() call is tracked and quiesce() drains the in-flight set. - workflow/* event payloads were live mutable aliases shared across emissions; emitWorkflowEvent now hands each listener its own structural clone. - the structured-output turn-continuation veto is now prepend: true, so an earlier-registered force-continue listener cannot short-circuit it. Docs updated in the same change (READMEs, core-data-structures/workflow.md, the dynamic-workflows RFC, regenerated cordis catalogs). |
||
|
Tianyi Cui
|
1d43ea3cd5 |
workflow: dynamic workflows — script-driven multi-agent orchestration
A new capability family at packages/workflow/ in the bash seam shape,
modeled on Claude Code's dynamic workflows: the model writes a JavaScript
orchestration script (export const meta = {...} + plain-JS body), a runtime
executes it, and the script — not the conversation — holds the loop, the
branching, and the intermediate results.
- dsh-workflow (ctx.workflows): abstract WorkflowService + run vocabulary
(WorkflowRun whose result NEVER rejects) + observe-only workflow/* events
carrying data snapshots (id + meta, never the live run), per-listener
contained like subagent/*.
- dsh-workflow-vm: in-process node:vm engine. Meta extraction via a
string/comment-aware scanner (template interpolation rejected; literal
evaluated alone in an empty timed context; statement blanked line-
preservingly so stacks keep script line numbers). Hooks: agent(prompt,
{label, phase, schema, model}) over ctx.subagents, parallel(), pipeline()
(no cross-stage barrier), phase(), log(), args. Fatal-vs-null discipline:
hook misuse (unknown/deferred options, bad arguments, unsupported
schemas, tripped caps, seam start failures, cancellation) throws fatal
WorkflowErrors the combinators RE-THROW — never dissolved into the
per-item null reserved for child failures. Realm boundary: inbound values
materialized by descriptor walks that never invoke accessors (defineProperty
copies, __proto__-safe); outbound values rebuilt in-realm via the
context's own JSON.parse. Determinism bans (Date.now/Math.random/argless
new Date) kept so future resume support cannot break scripts. Caps and
timeouts are validated Config. Every hook promise carries a no-op
rejection consumer (app-boot exits on unhandled rejections).
- dsh-tool-workflow: the model-facing workflow tool, synchronous like
dsh-tool-subagent (start → await → try/finally dispose; abort bridged;
non-completed → isError). Generic render card titled by a textual
meta.name sniff. The tool description carries the authoring contract.
Wired into examples/{coding-agent,acp-agent} with explicit-ask-only
guidance. Coverage at every tier: unit (meta scanner, materializer incl.
counting-getter and __proto__ regressions, combinator semantics,
concurrency ceiling, caps, cancellation, no-unhandled-rejection abandon),
integration over the real spawn stack, with-key e2e (real two-phase run +
the tool through the registry pipeline), and a recorded ACP snapshot
scenario (workflow-run, 1 child session). RFC:
docs/rfc/implemented/feature/2026-07-05-dynamic-workflows.md (deferred
work explicitly listed). AGENTS.md budget 1575 → 1590 for the new group's
layout line.
|