diff --git a/.agents/notes/proposed/feature/2026-07-16-harness-level-loop.i18n.yaml b/.agents/notes/implemented/feature/2026-07-16-harness-level-loop.i18n.yaml similarity index 65% rename from .agents/notes/proposed/feature/2026-07-16-harness-level-loop.i18n.yaml rename to .agents/notes/implemented/feature/2026-07-16-harness-level-loop.i18n.yaml index 743d50cc5e..1a8c03cb52 100644 --- a/.agents/notes/proposed/feature/2026-07-16-harness-level-loop.i18n.yaml +++ b/.agents/notes/implemented/feature/2026-07-16-harness-level-loop.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write -2026-07-16-harness-level-loop.md: 97041b1c08d5fb85a222824ee6d8dfd55e74f4bd -2026-07-16-harness-level-loop.zh.md: 6460254b07b0e185ff2a40c415d34bd25b6ad925 +2026-07-16-harness-level-loop.md: 9a9511b9dcea1b5fdc90f4fc716c4399f2346967 +2026-07-16-harness-level-loop.zh.md: 284e73051eaaa4633b9f56367de9096dadc8184e diff --git a/.agents/notes/implemented/feature/2026-07-16-harness-level-loop.md b/.agents/notes/implemented/feature/2026-07-16-harness-level-loop.md new file mode 100644 index 0000000000..9a9511b9dc --- /dev/null +++ b/.agents/notes/implemented/feature/2026-07-16-harness-level-loop.md @@ -0,0 +1,129 @@ +# Agent Note: Harness-level goal-based execution + +Status: implemented + +English | [中文](2026-07-16-harness-level-loop.zh.md) + +## Problem + +The concrete agent loop owns one turn: it drains admitted input, performs one or more model-and-tool steps, and stops. Substantial objectives often need an outer policy that can begin another turn, retain progress, stop at a budget, and remain intelligible to humans. A timed prompt, a same-session continuation, and a fresh-agent Ralph attempt all repeat work, but they do not share the same state, authority, memory, or lifecycle. + +Treating every repeated action as one generic “loop” obscures those differences. Same-session work must persist the human objective in the existing transcript while preserving conversation context. Ralph work must intentionally discard conversation context and use the workspace plus a bounded handoff. Human-facing status must not imply that reopening a session silently authorizes more work. Completion and blocker claims also need an explicit trust boundary rather than being smuggled into a scheduler abstraction. + +The repository therefore needs goal-based execution above the turn/step loop, but it does not need a speculative universal loop service that combines persistence, evaluation, budgeting, scheduling, handoff, background tasks, and UI. + +## Decision + +This proposal is implemented in amended form as two explicit plugin policies over existing seams: + +1. **Same-session goals** retain one durable objective in the current session and admit goal-attributed continuation turns only while live activation is armed. +2. **Fresh-agent Ralph runs** execute a fixed foreground workflow whose rounds each spawn a new structured child with no conversation seed. + +There is no `packages/loop/` family, `LoopDriver`, `LoopId`, universal `StopCondition`, or model-facing generic `loop` tool. The two policies share the repository's ordinary agent, session, tools, workflow, subagent, and UI extension seams, but they do not pretend that one lifecycle fits both. + +### Vocabulary and policy boundary + +The same-session hierarchy is **Goal → Goal Round → Turn → Step**. A goal round is one continuation cycle admitted for the current goal and materialized as one goal-sourced turn. Human or unrelated turns in the same session do not consume the goal-round cap, and a turn may still contain multiple model/tool steps. + +The fresh-agent hierarchy is **Ralph Run → Ralph Round → fresh child Turn → Step**. One Ralph round creates one child session. The parent transcript and prior child transcripts are not seed context; the shared workspace and one bounded structured report carry cross-round state. + +“Round” is therefore an outer policy iteration, not a synonym for every session turn. The concrete `dsh-agent-loop` remains the turn/step engine. The same-session driver uses public agent and session events; its only core addition is the generic observe-before-cancel `agent/cancel-requested` notification needed by any lifecycle policy that must settle cancellation safely. + +Time-based `/loop` or scheduled execution is a third policy and is not implemented by this decision. It belongs with a scheduler rather than either goal family. + +### Package topology and owning verbs + +| Package | Repository category | Owned structures and verbs | +|---|---|---| +| `@deepseek-ai/dsh-goal` | `packages/goal/goal/`, domain service | Owns `GoalId`, compare-and-set `GoalRef`, `GoalSnapshot`, four-state `GoalPhase`, structured `GoalBlockReason`, process-local `GoalActivation`, replay folding, and `get`, `create`, `edit`, `pause`, `resume`, `complete`, `block`, `clear`, and `disarm` verbs. | +| `@deepseek-ai/dsh-tool-goal` | `packages/goal/tool-goal/`, model-facing consumer | Registers exclusive `get_goal`, `create_goal`, and `update_goal`; authenticates live turn provenance and narrows autonomous-round authority to completion or blocking reports with machine-routable reason codes. | +| `@deepseek-ai/dsh-goal-session` | `packages/goal/goal-session/`, continuation policy | Reserves, fences, admits, attributes, settles, cancels, and quiescently drains same-session goal rounds without importing the concrete loop. | +| `@deepseek-ai/dsh-commands` | `packages/ui/commands/`, UI registry | Owns `CommandDefinition`, discovery, scoped registration, direct dispatch, `CommandResult`, and request cancellation for human-only commands. | +| `@deepseek-ai/dsh-command-goal` | `packages/goal/command-goal/`, human-command producer | Registers `/goal` status, creation, edit, pause, resume, and clear over the goal domain for TUI and ACP. | +| `@deepseek-ai/dsh-tool-ralph` | `packages/workflow/tool-ralph/`, fixed workflow consumer | Registers `ralph({ objective, maxRounds? })`, validates the fresh structured provider and bounded `RalphRoundReport`, and returns `complete`, `blocked`, or `budget-limited`. | + +The detailed contracts live in the [goal-domain](2026-07-19-persisted-same-session-goal-domain.md), [model goal-tools](2026-07-19-model-facing-goal-tools.md), [goal-round driver](2026-07-19-same-session-goal-round-driver.md), [command registry](2026-07-19-plugin-command-registration.md), [human goal-command](2026-07-19-human-goal-command.md), and [Ralph workflow-tool](2026-07-19-fresh-agent-ralph-workflow-tool.md) Agent Notes. + +### Durable goal state and live authority + +One session has at most one current goal. Every non-clear mutation appends a full, versioned, model-visible goal snapshot through `Agent.inject()`; clear appends a revisioned tombstone. The session log is the only durable source of truth, so normal persistence, resume, compaction semantics, and `SessionStore.fork()` carry the goal without a second database or an artificial cancellation record. + +Durable phases are only `active`, `paused`, `blocked`, and `complete`. A blocked goal carries a required `GoalBlockReason` with a stable lower-kebab-case `code` and a non-empty human-readable `message`; usage limits, round exhaustion, model failures, and policy rejection are reason codes rather than extra lifecycle phases. Separate activation is `armed` or `disarmed` and is never persisted. Creation and explicit resume arm a goal; stop transitions, session start, fork replay, driver replacement, and driver teardown leave it disarmed. + +This separation makes session restoration observable and unsurprising. Reopening a session never starts goal work by itself. A later human prompt such as “continue”, “resume the goal”, or an equivalent request in any language gives the runtime-root model a new turn in which it may read the goal and call `update_goal(..., action: 'resume')`. `/goal resume` is the direct human-command path. The runtime authenticates that the request came from a live direct-human turn; prompt policy lets the model interpret whether the wording semantically authorizes creation or resumption. + +Forked sessions inherit the durable goal prefix because that is the natural replay result. The fork starts disarmed, so inheritance does not imply execution authority and no synthetic goal cancellation is inserted into history. + +`defaultMaxGoalRounds` is configurable and defaults to `256`. The cap counts only admitted goal rounds. `blockedAfterConsecutiveRounds` is separately configurable in the model-tool policy and defaults to `3`; it is a mechanical lower bound before an autonomous round may report a repeated blocker, not an evaluator of semantic sameness. + +### Same-session continuation + +The goal-round driver owns at most one pending reservation per exact live agent. It admits a reservation only when the goal is active and armed, the agent is idle, no competing human work exists, pending mutations are durable, the exact goal id/revision/round still matches, and downstream prompt policy accepts it. The prompt-submit fence checks those facts both before and after asynchronous listeners, preventing an edit, pause, human message, or unload race from admitting obsolete work. + +Only the durable goal-sourced `user/message` charges a round. Stale reservations become rejected zero-step turns without consuming the cap. A concurrent goal revision wins over settlement from an older round. + +Normal turn completion schedules another round only while the goal remains active, armed, and below its cap. Cancellation pauses. Rate limiting or quota exhaustion blocks with code `usage-limited`; cap exhaustion blocks with `round-limit`; queue failure uses `queue-failed`; turn errors, max-token stops, policy rejection, and unknown terminal results use their corresponding blocker codes. An independently composed request-recovery plugin may retry transient provider failures within that same turn; the goal driver never invents another round after an abnormal terminal outcome. A human can later authorize resume through ordinary language or `/goal resume`. + +### Human and model surfaces + +The human UX follows the compact Codex shape in the [public OpenAI Codex TUI dispatcher at commit `678157a`](https://github.com/openai/codex/blob/678157acaa819d5510adfe359abb5d0392cfe461/codex-rs/tui/src/chatwidget/slash_dispatch.rs#L750-L805): `/goal` shows status, `/goal ` creates, and `edit`, `pause`, `resume`, or `clear` perform direct lifecycle actions. The commit permalink keeps the researched grammar verifiable as Codex evolves. Status includes durable phase, admitted/capped rounds, and live armed/disarmed activation. Direct status and command output do not enter model history; accepted domain mutations remain reconstructable because the goal service records them. + +The model receives only `get_goal`, `create_goal`, and `update_goal`. It may create a goal when a direct human request clearly asks for substantial multi-round work, and it may infer that intent in any language. It must not turn routine one-turn work into a goal. Direct-human provenance is enforced in code; semantic interpretation remains model judgment. An autonomous goal round may report `complete` or `blocked` for the exact current goal round but cannot edit, pause, resume, or replace the human objective. + +TUI and ACP mount the shared command registry and complete goal stack by default and expose `/goal` through one producer. Every effective registered command is discoverable and invocable through every composed command adapter; a plugin incompatible with an application omits its command producer from that composition rather than relying on registry-level surface masks. The UI-less agent spine is opt-in so one-shot callers do not silently become multi-round operations. The headless CLI and JSON-RPC front doors do not consume the command plane; ordinary human text can still authorize model goal tools when that stack is composed. + +### Fresh-agent Ralph execution + +Ralph is a first-class model tool in its own plugin, demonstrating that a sophisticated fixed execution policy can be composed without a new loop core. The plugin owns a fixed workflow script over `ctx.workflows` and `ctx.subagents`; it does not create session-goal state or add a branch to `dsh-agent-loop`. + +Each round uses an explicit `WorkflowStartRequest.subagentProvider`, defaulting to `spawn`. The provider must exist, support structured output, and declare that it does not inherit parent context. Ralph also passes its resolved round cap as `WorkflowStartRequest.maxTotalAgents`; the worker engine validates both per-run policies before publishing work, so provider misconfiguration or an engine ceiling below the requested Ralph scale fails before a run exists. The child inherits cwd and lineage but receives only the immutable objective, round/cap, workspace-as-authority instruction, and previous normalized report. + +A report contains status, summary, evidence, next steps, and blocker text. Status-specific invariants and serialized size are validated inside the fixed script and again at the consumer boundary. `maxRounds` is configurable, defaults to `256`, and is the ceiling for a call override. `maxHandoffChars` defaults to `16384`; oversized reports fail rather than being silently truncated. `maxResultChars` separately defaults to `16384` and bounds the complete successful parent-facing text, including its envelope and truncation marker. + +An ordinary child failure ends the run without retry. The fixed script reports the failed round and last successful handoff when one exists, and the tool returns that state as an error instead of misclassifying it as a malformed report or budget exhaustion. Fatal workflow infrastructure failures can settle before the script returns that state; richer reason transport and retry policy remain deferred. + +The tool is foreground and process-local. The parent tool call waits for the terminal result, propagates cancellation into the worker engine, and awaits `run.dispose()` so child work is quiescent before return. The model sees one call and one bounded successful terminal result or an error; completion and blocker envelopes explicitly say that a worker reported the outcome rather than presenting it as independent certification. Intermediate child conversations remain outside the parent transcript. + +### External design lineage + +Codex provides the minimal observable goal UX used here: a persistent chat-attached target with set, view, edit, pause, resume, and clear controls. This implementation adopts that discoverability while using this repository's event-sourced goal record, plugin scopes, and runtime authority checks. + +Current [Claude Code goals](https://code.claude.com/docs/en/goal) reinforce the distinction between a goal that starts another turn after the previous turn and a timed `/loop`. Claude Code also uses a separate small-model evaluator after each turn. This implementation adopts the policy distinction but intentionally does not copy that evaluator: evaluator inputs, tool access, deterministic checks, provider choice, isolation, and authority need a separately designed plugin contract rather than an implicit self-certification layer. + +External products are comparators, not compatibility targets. The local source studies informed the boundaries, while the shipped interfaces follow this repository's “everything is a plugin”, model-visible-is-logged, explicit default resolution, and quiescent teardown rules. + +### Verification + +The six owning Agent Notes record unit, integration, process, snapshot, cancellation, replay, and built-runtime coverage. The stack exercises strict goal-record folding, compare-and-set races, session fork inheritance, disarmed restoration, natural-language direct-human authority, configurable caps and blocked thresholds, exact goal-round attribution, adapter-wide command discovery, and transcript isolation. Shipped keyless snapshots cover model goal creation/inspection through the headless app, multi-round same-session lifecycle and cancellation through ACP, direct `/goal` status without a model turn, and two real Ralph rounds through the headless app. The Ralph snapshot boots the worker-thread engine, spawn provider, structured-output runtime, and agent loop, then inspects distinct unseeded child logs and exact one-way bounded handoff while pinning the parent stream. Focused real-stack tests additionally cover completion, blocker and round-limit outcomes, malformed and oversized reports, ordinary child failure with the last good handoff, one phase event, and cancellation to child quiescence. Package sources remain under the repository's per-file 100% coverage gate, and built-binary tests cover installed-artifact resolution. The implementation experience is recorded in the root testing policy: every non-trivial model- or human-visible change must carry a real-example keyless snapshot in the same PR rather than relying on package-only or mock-only fixture coverage. + +## Alternatives considered + +- **Implement the original universal loop capability seam** — rejected because `Evaluator`, `BudgetPolicy`, `RoundHandoff`, `GoalReflector`, background task ownership, persistence, and scheduling do not form one coherent mandatory abstraction. Building all of them before their first concrete consumers would create broad speculative surface and duplicate existing session, workflow, subagent, and task machinery. +- **Implement only same-session goals** — rejected because fresh-context iteration is materially different and is a valuable demonstration of the plugin architecture. Ralph belongs as a fixed workflow consumer with explicit context reset. +- **Put Ralph inside the goal-round driver** — rejected because same-session goals deliberately preserve one conversation while Ralph deliberately removes it. Combining them would make activation, replay, handoff, and UI state ambiguous. +- **Treat a fork as a fresh Ralph child** — rejected because a fork carries a conversation prefix. Fresh children plus workspace state and one explicit report are easier to bound and replay without a synthetic cancel record. +- **Copy Claude Code's evaluator into the first goal implementation** — rejected because a transcript-only model evaluator is one useful policy, not a generally trustworthy completion certificate. Deterministic evaluation and isolation must remain possible, so the evaluator is deferred until its authority and provider seam are designed. +- **Automatically continue after session restore** — rejected because opening a session is observation, not authority to spend resources. Durable state is restored while activation waits for a new human prompt. +- **Route `/goal` through the model** — rejected because status and explicit lifecycle controls should be deterministic, token-free UI actions; ordinary natural-language prompts remain the semantic model path. +- **Modify the concrete agent loop with goal or Ralph modes** — rejected because public queue, prompt, session, cancellation, workflow, and subagent seams already support both policies. The generic cancel-requested observation is the only core coordination addition. + +## Consequences + +- Goal-based execution ships without one overloaded “loop” object: same-session continuation and fresh-agent iteration have explicit, separately testable contracts. +- Durable goal history is replayable and forkable, while process-local activation prevents accidental work on resume. +- Humans receive a small Codex-shaped UX; models receive a compact provenance-checked tool surface; deployments can remove either independently. +- Ralph demonstrates a nontrivial fixed policy entirely as a plugin over existing workflow and subagent primitives. +- Round limits are generous by default but remain deployment-controlled. They bound iterations, not tokens, price, elapsed time, or external side effects. +- The original proposal's evaluator, budget, reflector, background-task, CLI, and generic loop-session architecture is intentionally not part of the implemented public surface. + +## Known limitations and deferred work + +- **Independent evaluation** — same-session completion/blocking and Ralph terminal status are model or worker declarations. A separate evaluator, evaluator-driven feedback round, completion certificate, deterministic checker, adversarial verifier, and criteria/executor/isolation contract remain deferred. +- **Aggregate budgets** — `maxGoalRounds` and Ralph `maxRounds` are the only aggregate effort limits. Token, currency, elapsed-time, provider-usage, and per-round price admission policies are absent. +- **No persistent autonomous runner** — same-session goal facts persist, but activation and scheduling are process-local and deliberately wait for human input after restore. Ralph runs are foreground and cannot resume after process loss. Background collection, restart recovery, and unattended resident execution are deferred. +- **No time scheduler** — interval `/loop`, cron, proactive maintenance, and cloud or desktop scheduling are outside this decision. +- **No generic loop journal or execution-world rewind** — session replay reconstructs model-visible goal history, not prior files, processes, environment, credentials, or external side effects. Ralph treats the current workspace as authority and carries no cross-run journal. +- **No goal reflector** — concern events, automatic no-progress heuristics, goal revision by an independent reflector, stuck-pattern detection, and `loop_split` are not implemented. Humans can edit, pause, clear, or resume the goal directly. +- **Ralph policy remains narrow** — one round creates one fresh child; within-round fan-out, evaluator/worker role separation, dynamic provider/model selection, and structural recursive-Ralph tool denial need separate policy surfaces. Prompt guidance is not enforcement. +- **Ralph does not retry a failed child** — an ordinary failure preserves the failed round and last good handoff, while fatal workflow infrastructure failures can end before that state is available. Retry count, backoff, and richer failure transport need separate policy and seam design. +- **Portable UI remains modest** — TUI and ACP render plain-text goal status and generic Ralph cards. There is no continuous status widget, reconnectable command output, modal goal editor, or command plane in the headless CLI or JSON-RPC front doors. diff --git a/.agents/notes/implemented/feature/2026-07-16-harness-level-loop.zh.md b/.agents/notes/implemented/feature/2026-07-16-harness-level-loop.zh.md new file mode 100644 index 0000000000..284e73051e --- /dev/null +++ b/.agents/notes/implemented/feature/2026-07-16-harness-level-loop.zh.md @@ -0,0 +1,129 @@ +# Agent Note: Harness 层目标式执行 + +Status: implemented + +[English](2026-07-16-harness-level-loop.md) | 中文 + +## 问题 + +具体 agent loop 只拥有一个 Turn：它排空已接纳输入，执行一个或多个模型与工具 Step，然后停止。大型目标通常需要一项外层策略来开始另一个 Turn、保留进度、在预算处停止，并让人类能够理解其状态。定时提示词、同会话续行和全新 agent Ralph 尝试都会重复工作，但它们并不共享相同的状态、权限、记忆或生命周期。 + +若把每种重复动作都称为一个通用“loop”，就会掩盖这些差异。同会话工作必须在现有转录中持久化人类目标，同时保留对话上下文。Ralph 工作必须有意丢弃对话上下文，只使用工作区和一份有界交接。面向人类的状态不能暗示重新打开会话就会静默授权更多工作。完成与阻塞声明也需要显式信任边界，而不能被偷渡进调度器抽象。 + +因此，本仓库需要位于 Turn/Step loop 之上的目标式执行，但不需要一个把持久化、评估、预算、调度、交接、后台任务和 UI 组合在一起的推测性通用 loop 服务。 + +## 决策 + +本提案以修订后的形式实现为构建在现有接缝之上的两项显式插件策略： + +1. **同会话目标**在当前会话中保留一个持久目标，并且只在实时激活态已激活时接纳带目标归属的续行 Turn。 +2. **全新 agent Ralph 运行**执行一个固定前台工作流，其中每个 Round 都生成一个不带对话种子的全新结构化子 agent。 + +系统中没有 `packages/loop/` 包族、`LoopDriver`、`LoopId`、通用 `StopCondition` 或面向模型的通用 `loop` 工具。两项策略共享本仓库普通的 agent、session、tools、workflow、subagent 与 UI 扩展接缝，但不会假装一种生命周期可以同时适配两者。 + +### 词汇与策略边界 + +同会话层级是 **Goal → Goal Round → Turn → Step**。一个 Goal Round 是为当前目标接纳的一次续行周期，并实体化为一个带目标来源的 Turn。同一会话中的人类 Turn 或无关 Turn 不会消耗目标回合上限，而一个 Turn 仍可包含多个模型/工具 Step。 + +全新 agent 层级是 **Ralph Run → Ralph Round → fresh child Turn → Step**。一个 Ralph Round 创建一个子会话。父转录和此前子转录都不是种子上下文；共享工作区与一份有界结构化报告承载跨 Round 状态。 + +因此，“Round”是外层策略迭代，不是每个会话 Turn 的同义词。具体 `dsh-agent-loop` 仍是 Turn/Step 引擎。同会话驱动器使用公开 agent 与 session 事件；它对核心唯一的新增项是通用的取消前观察通知 `agent/cancel-requested`，任何需要安全收敛取消的生命周期策略都可以使用它。 + +基于时间的 `/loop` 或定时执行是第三种策略，本决策不实现它。它应归属于调度器，而不是任一目标包族。 + +### 包拓扑与所属动词 + +| 包 | 仓库类别 | 所属结构与动词 | +|---|---|---| +| `@deepseek-ai/dsh-goal` | `packages/goal/goal/`，领域服务 | 拥有 `GoalId`、比较并交换 `GoalRef`、`GoalSnapshot`、四状态 `GoalPhase`、结构化 `GoalBlockReason`、进程本地 `GoalActivation`、重放折叠，以及 `get`、`create`、`edit`、`pause`、`resume`、`complete`、`block`、`clear` 与 `disarm` 动词。 | +| `@deepseek-ai/dsh-tool-goal` | `packages/goal/tool-goal/`，面向模型消费者 | 注册互斥的 `get_goal`、`create_goal` 与 `update_goal`；认证实时 Turn 来源，并把自治 Round 权限收窄到带机器可路由原因代码的完成或阻塞报告。 | +| `@deepseek-ai/dsh-goal-session` | `packages/goal/goal-session/`，续行策略 | 在不导入具体 loop 的情况下，预留、设围栏、接纳、归属、结算、取消并静止排空同会话目标回合。 | +| `@deepseek-ai/dsh-commands` | `packages/ui/commands/`，UI 注册表 | 拥有面向人类专用命令的 `CommandDefinition`、发现、作用域注册、直接分发、`CommandResult` 与请求取消。 | +| `@deepseek-ai/dsh-command-goal` | `packages/goal/command-goal/`，人类命令生产方 | 为 TUI 和 ACP 注册构建在目标领域之上的 `/goal` 状态、创建、编辑、暂停、恢复与清除。 | +| `@deepseek-ai/dsh-tool-ralph` | `packages/workflow/tool-ralph/`，固定工作流消费者 | 注册 `ralph({ objective, maxRounds? })`，验证全新结构化 provider 与有界 `RalphRoundReport`，并返回 `complete`、`blocked` 或 `budget-limited`。 | + +详细契约分别由[目标领域](2026-07-19-persisted-same-session-goal-domain.md)、[模型目标工具](2026-07-19-model-facing-goal-tools.md)、[目标回合驱动器](2026-07-19-same-session-goal-round-driver.md)、[命令注册表](2026-07-19-plugin-command-registration.md)、[人类目标命令](2026-07-19-human-goal-command.md)与 [Ralph 工作流工具](2026-07-19-fresh-agent-ralph-workflow-tool.md) Agent Note 拥有。 + +### 持久目标状态与实时权限 + +一个会话至多有一个当前目标。每次非清除变更都通过 `Agent.inject()` 追加一份完整、带版本且模型可见的目标快照；清除会追加带修订号的墓碑。会话日志是唯一持久事实来源，因此普通持久化、恢复、压缩语义与 `SessionStore.fork()` 会携带目标，无需第二个数据库或人为取消记录。 + +持久阶段只有 `active`、`paused`、`blocked` 与 `complete`。阻塞目标必须携带 `GoalBlockReason`，其中包含稳定的小写 kebab-case `code` 与非空的人类可读 `message`；用量限制、Round 耗尽、模型失败与策略拒绝都是原因代码，而不是额外生命周期阶段。独立激活态是 `armed` 或 `disarmed`，且永不持久化。创建与显式恢复会激活目标；停止转换、会话启动、fork 重放、驱动器替换和驱动器拆卸都会让目标保持未激活。 + +这种分离让会话恢复可观察且符合直觉。重新打开会话绝不会自行开始目标工作。随后的人类提示词，例如“继续”、“恢复目标”或任何语言中的等价请求，会给运行时根 agent 的模型一个新 Turn；模型可在其中读取目标并调用 `update_goal(..., action: 'resume')`。`/goal resume` 是直接人类命令路径。运行时认证请求来自实时直接人类 Turn；提示策略让模型解释措辞在语义上是否授权创建或恢复。 + +fork 会话会继承持久目标前缀，因为这是自然的重放结果。fork 从未激活状态开始，因此继承不等于执行权限，历史中也不会插入合成目标取消。 + +`defaultMaxGoalRounds` 可配置且默认为 `256`。该上限只计算已接纳目标回合。`blockedAfterConsecutiveRounds` 在模型工具策略中单独配置且默认为 `3`；它只是在自治 Round 报告重复阻塞前的机械下限，不是对语义相同性的评估器。 + +### 同会话续行 + +目标回合驱动器为每个准确实时 agent 至多拥有一个待定预留。只有目标处于活跃且已激活状态、agent 空闲、不存在竞争性人类工作、待定变更已经持久、准确目标 id/修订号/Round 仍匹配，并且下游提示词策略接受时，它才会接纳预留。prompt-submit 围栏在异步监听器前后都检查这些事实，防止编辑、暂停、人类消息或卸载竞争接纳过时工作。 + +只有持久的目标来源 `user/message` 会计入一个 Round。过时预留会成为未消耗上限的零 Step 拒绝 Turn。并发目标修订会胜过旧 Round 的结算。 + +普通 Turn 完成后，只有目标仍活跃、已激活且低于上限时才会安排另一个 Round。取消会暂停。速率限制或配额耗尽以代码 `usage-limited` 阻塞；上限耗尽使用 `round-limit`；队列失败使用 `queue-failed`；Turn 错误、max-token 停止、策略拒绝与未知终止结果使用各自对应的阻塞代码。独立组合的请求恢复插件可以在同一个 Turn 内重试暂时性 provider 失败；目标驱动器绝不会在异常终止结果后凭空发起另一个 Round。人类随后可以通过普通语言或 `/goal resume` 授权恢复。 + +### 人类与模型表面 + +人类 UX 遵循 [OpenAI Codex 在提交 `678157a` 时的公开 TUI 分发器](https://github.com/openai/codex/blob/678157acaa819d5510adfe359abb5d0392cfe461/codex-rs/tui/src/chatwidget/slash_dispatch.rs#L750-L805)中的紧凑形态：`/goal` 显示状态，`/goal ` 创建目标，而 `edit`、`pause`、`resume` 或 `clear` 执行直接生命周期操作。该提交永久链接让研究所得语法在 Codex 演进时仍可验证。状态包含持久阶段、已接纳/上限 Round 数以及实时已激活/未激活状态。直接状态与命令输出不会进入模型历史；已接受领域变更仍可重建，因为目标服务会记录它们。 + +模型只接收 `get_goal`、`create_goal` 和 `update_goal`。当直接人类请求清楚要求大量多 Round 工作时，模型可以创建目标，并且可以从任何语言推断该意图。它不得把日常单 Turn 工作变成目标。直接人类来源由代码强制执行；语义解释仍是模型判断。自治目标 Round 可以为准确当前目标 Round 报告 `complete` 或 `blocked`，但不能编辑、暂停、恢复或替换人类目标。 + +TUI 与 ACP 默认挂载共享命令注册表和完整目标栈，并通过同一个生产方暴露 `/goal`。每条有效已注册命令都能被每个已组合的命令适配器发现和调用；若插件与某应用不兼容，该应用组合会省略其命令生产方，而不是依赖注册表层面的表面掩码。无 UI agent spine 要求显式选择加入，以免单次调用方静默变成多 Round 操作。无头 CLI 与 JSON-RPC 前端不消费命令平面；挂载目标栈后，普通人类文本仍可授权模型目标工具。 + +### 全新 agent Ralph 执行 + +Ralph 是位于自有插件中的一等模型工具，展示了复杂固定执行策略可以在没有新 loop 核心的情况下组合完成。该插件拥有构建在 `ctx.workflows` 与 `ctx.subagents` 之上的固定工作流脚本；它不会创建会话目标状态，也不会为 `dsh-agent-loop` 增加分支。 + +每个 Round 都使用显式 `WorkflowStartRequest.subagentProvider`，默认为 `spawn`。该 provider 必须存在、支持结构化输出，并声明不继承父上下文。Ralph 还会把解析后的 Round 上限作为 `WorkflowStartRequest.maxTotalAgents` 传递；工作线程引擎会在发布工作前验证两项每次运行策略，因此 provider 配置错误或低于所请求 Ralph 规模的引擎上限会在运行存在前失败。子 agent 继承 cwd 与谱系，但只接收不可变目标、当前 Round/上限、以工作区为权威的指令和上一份规范化报告。 + +报告包含状态、摘要、证据、下一步与阻塞文本。固定脚本内部和消费者边界都会验证状态专用不变量与序列化大小。`maxRounds` 可配置，默认为 `256`，并作为调用覆盖值的上限。`maxHandoffChars` 默认为 `16384`；过大报告会失败，而不会被静默截断。`maxResultChars` 单独默认为 `16384`，并限制面向父级的完整成功文本，包括外层文本与截断标记。 + +普通子 agent 失败会结束运行且不重试。固定脚本会报告失败 Round，并在存在时带回上一份成功交接；工具会把该状态作为错误返回，而不会误判为畸形报告或预算耗尽。致命工作流基础设施错误可能在脚本返回该状态前结算；更丰富的原因传输与重试策略均予以延期。 + +该工具位于前台且只存在于进程内。父工具调用等待终止结果，把取消传播到工作线程引擎，并等待 `run.dispose()`，因此返回前子工作已达到静止。模型只看到一次调用，以及一份有界成功终止结果或一个错误；完成与阻塞的外层文本会明确说明结果由工作者报告，而不会呈现为独立认证。中间子 agent 对话不会进入父转录。 + +### 外部设计谱系 + +Codex 提供了这里采用的最小可观察目标 UX：一个附着于聊天的持久目标，以及设置、查看、编辑、暂停、恢复与清除控制。本实现采用这种可发现性，但使用本仓库的事件溯源目标记录、插件作用域与运行时权限检查。 + +当前 [Claude Code goals](https://code.claude.com/docs/en/goal) 进一步验证了“前一 Turn 后启动另一 Turn 的目标”和定时 `/loop` 之间的区别。Claude Code 还会在每个 Turn 后使用独立小模型评估器。本实现采用策略区分，但有意不复制该评估器：评估器输入、工具访问、确定性检查、provider 选择、隔离与权限需要单独设计的插件契约，而不是隐式自我认证层。 + +外部产品只是比较对象，不是兼容目标。本地源码研究帮助确定边界，而交付接口遵循本仓库“一切皆插件”、模型可见即可记录、显式解析默认值与静止拆卸规则。 + +### 验证 + +六份所属 Agent Note 记录了单元、集成、进程、快照、取消、重放与构建后运行时覆盖。该栈验证严格目标记录折叠、比较并交换竞争、会话 fork 继承、恢复后未激活、自然语言直接人类权限、可配置上限与阻塞阈值、准确目标回合归属、适配器范围的命令发现与转录隔离。已发布的无密钥快照覆盖通过无头应用创建/检查模型目标、通过 ACP 执行多 Round 同会话生命周期与取消、无需模型 Turn 的直接 `/goal` 状态，以及通过无头应用执行两个真实 Ralph Round。Ralph 快照会启动工作线程引擎、spawn provider、结构化输出运行时与 agent loop，随后检查互不相同且无种子的子日志和准确单向有界交接，同时固定父级事件流。聚焦的真实栈测试还覆盖完成、阻塞与 Round 上限结果、畸形及过大报告、保留上一份有效交接的普通子 agent 失败、单个阶段事件，以及取消后达到子 agent 静止状态。包源码继续受仓库逐文件 100% 覆盖率门禁约束，构建后二进制测试覆盖已安装产物解析。实现经验已记录进根测试策略：每项非平凡的模型或人类可见变更都必须在同一 PR 中携带真实示例无密钥快照，而不能依赖仅包级或仅模拟夹具的覆盖。 + +## 考虑过的替代方案 + +- **实现原始通用 loop 能力接缝**——不予采纳，因为 `Evaluator`、`BudgetPolicy`、`RoundHandoff`、`GoalReflector`、后台任务所有权、持久化与调度并不构成一项一致的必选抽象。在出现首个具体消费者前全部构建，会产生宽泛推测性表面，并重复现有 session、workflow、subagent 与 task 机制。 +- **只实现同会话目标**——不予采纳，因为全新上下文迭代在实质上不同，也是插件架构的重要示范。Ralph 应作为带显式上下文重置的固定工作流消费者。 +- **把 Ralph 放进目标回合驱动器**——不予采纳，因为同会话目标有意保留一段对话，而 Ralph 有意移除对话。合并两者会让激活、重放、交接与 UI 状态含糊不清。 +- **把 fork 当成全新 Ralph 子 agent**——不予采纳，因为 fork 会携带对话前缀。全新子 agent 加工作区状态与一份显式报告更容易限制和重放，并且无需合成取消记录。 +- **把 Claude Code 评估器复制进首个目标实现**——不予采纳，因为只读取转录的模型评估器是一项有用策略，但不是普遍可信的完成证书。系统必须仍能支持确定性评估与隔离，因此评估器延期到其权限与 provider 接缝完成设计之后。 +- **会话恢复后自动续行**——不予采纳，因为打开会话是观察行为，不是花费资源的权限。系统恢复持久状态，而激活态等待新的人类提示词。 +- **通过模型路由 `/goal`**——不予采纳，因为状态与显式生命周期控制应是确定、零 token 的 UI 操作；普通自然语言提示词仍是语义模型路径。 +- **为具体 agent loop 增加目标或 Ralph 模式**——不予采纳，因为公开队列、提示词、会话、取消、工作流与 subagent 接缝已经支持两项策略。通用 cancel-requested 观察是唯一核心协调新增项。 + +## 后果 + +- 目标式执行在没有单个过载“loop”对象的情况下交付：同会话续行与全新 agent 迭代拥有显式、可独立测试的契约。 +- 持久目标历史可以重放和 fork，而进程本地激活态会防止恢复时意外开始工作。 +- 人类获得小型 Codex 形态 UX；模型获得紧凑、带来源检查的工具表面；部署可以独立移除任一能力。 +- Ralph 展示了非平凡固定策略可以完全作为现有 workflow 与 subagent 原语之上的插件实现。 +- Round 上限默认宽裕，但仍由部署控制。它限制迭代次数，不限制 token、价格、耗时或外部副作用。 +- 原始提案中的评估器、预算、反思器、后台任务、CLI 与通用 loop-session 架构有意不进入已实现公开表面。 + +## 已知限制与延期工作 + +- **独立评估**——同会话完成/阻塞和 Ralph 终止状态都是模型或工作者声明。独立评估器、评估器驱动反馈 Round、完成证书、确定性检查器、对抗式 verifier 与 criteria/executor/isolation 契约均予以延期。 +- **聚合预算**——`maxGoalRounds` 与 Ralph `maxRounds` 是唯一聚合工作量限制。token、货币、耗时、provider 用量与逐 Round 价格准入策略均不存在。 +- **没有持久自治运行器**——同会话目标事实会持久化，但激活与调度只存在于进程内，并且有意在恢复后等待人类输入。Ralph 位于前台，进程丢失后无法恢复。后台收集、重启恢复与无人值守常驻执行均予以延期。 +- **没有时间调度器**——间隔 `/loop`、cron、主动维护以及云端或桌面调度不在本决策范围内。 +- **没有通用 loop 日志或执行世界回退**——会话重放会重建模型可见目标历史，而不会恢复此前文件、进程、环境、凭据或外部副作用。Ralph 把当前工作区作为权威，并且没有跨运行日志。 +- **没有目标反思器**——concern 事件、自动无进展启发式、由独立反思器执行的目标修订、卡住模式检测与 `loop_split` 均未实现。人类可以直接编辑、暂停、清除或恢复目标。 +- **Ralph 策略仍然狭窄**——一个 Round 创建一个全新子 agent；Round 内扇出、评估器/工作者角色分离、动态 provider/模型选择与结构化递归 Ralph 工具禁止都需要独立策略表面。提示词指导不是强制执行。 +- **Ralph 不会重试失败的子 agent**——普通失败会保留失败 Round 与上一份有效交接，而致命工作流基础设施错误可能在该状态可用前结束。重试次数、退避与更丰富的失败传输需要独立的策略与接缝设计。 +- **可移植 UI 仍较朴素**——TUI 与 ACP 渲染纯文本目标状态和通用 Ralph 卡片。系统没有持续状态组件、可重连命令输出、模态目标编辑器，无头 CLI 与 JSON-RPC 前端也没有命令平面。 diff --git a/.agents/notes/proposed/feature/2026-07-16-harness-level-loop.md b/.agents/notes/proposed/feature/2026-07-16-harness-level-loop.md deleted file mode 100644 index 97041b1c08..0000000000 --- a/.agents/notes/proposed/feature/2026-07-16-harness-level-loop.md +++ /dev/null @@ -1,343 +0,0 @@ -# Agent Note: harness-level goal-based loop - -Status: proposed - -English | [中文](2026-07-16-harness-level-loop.zh.md) - -## Problem - -`packages/core/agent-loop` runs only the inner loop: reasoning plus tool calls within one turn, ending when the model returns `end_turn`. Its README explicitly writes "No built-in turn budget"—budget is a gap it acknowledges itself. Cross-round scheduling falls on the harness layer: iterating until tests all pass, revising drafts against a rubric, splitting a PRD into beads and driving them one by one, running unattended for a whole night. None of these tasks has a first-class implementation today. - -The existing code offers three "just enough to run" alternatives, none of them adequate: - -| Alternative | Problem | -|---|---| -| A `packages/workflow` script expressing `while (!done)` | The README explicitly writes "No token-budget vocabulary" and "No journaling or resume"; the parent turn blocks until the script settles. Fine for orchestration lasting minutes, unusable for tasks lasting hours | -| An external shell `while :; do dsh-sdk …; done` | Ralph-style scheduling can be written this way today. It lacks a shared vocabulary for stop condition, budget, and evaluator, so every user reinvents them; the loop itself has no durable object for post-hoc diagnosis or recovery | -| The `sendMessage`/`resume` capabilities on the `packages/subagent` seam | The README explicitly writes "Runtime steering and continuation are seam-only capabilities". There is no model-facing consumer, so the model can only start a fresh subagent | - -Three typical use cases. **Automated fix**: a failing test suite in front of you, and you want a process to keep modifying code, running tests, and modifying again against the failure messages, until everything is green or the budget cap is hit. **Rubric-driven iterative revision**: a document, code, or translation must meet a set of scoring criteria; the loop repeatedly adjusts, an independent evaluator scores, and the loop stops when the criteria are met or the round budget is exhausted. **Unattended long runs**: for example, porting a repository from one tech stack to another overnight, kicked off before leaving work and reviewed the next morning, with the budget as the only safety net. Common shape across all three: minutes to hours, evaluator decides success, budget is a hard constraint, and post-run review and recovery are required. - -## Proposal - -**Loops come in four trigger shapes**, distinguished by who starts a round and when: - -| Shape | Who triggers | When | Existing comparable | This RFC | -|---|---|---|---|---| -| **turn-based** | The user sends a message in the session | Every user reply | `packages/core/agent-loop`'s existing reasoning-plus-tools cycle within one turn | Not covered; already implemented | -| **goal-based** | The user or the agent specifies "run until some condition" | One start, evaluator decides when to stop | Claude Code's `/goal`, Codex's `/goal`, the Ralph family | **This RFC covers it** | -| **time-based** | A scheduler | On cron or fixed interval | Claude Code's `/loop` (periodic), `/schedule` | Deferred to a `dsh-schedule` RFC | -| **proactive** | The agent itself | When the agent realizes during reasoning that a loop is needed | The proactive tier in Anthropic ClaudeDevs's four-way taxonomy | **Naturally included** (an agent calling the `loop` tool is already proactive) | - -This RFC only **adds a capability seam `packages/loop/`** for the goal-based shape. Proactive reuses the same `loop` tool—an agent invocation is a trigger by itself, with no extra machinery. Time-based needs an independent scheduler package and belongs to a separate RFC; this RFC only reserves a hook on the cordis leaf trigger surface for the future `dsh-schedule` integration. - -Three packages: - -- `@deepseek-ai/dsh-loop`: types, the `LoopDriver` service, the `StopCondition` discriminated union, the Phase 1 service definitions (`Evaluator` / `BudgetPolicy` / `RoundHandoff`), and the event schema; `GoalReflector` joins in Phase 2 with its first caller -- `@deepseek-ai/dsh-loop-driver`: the default driver implementation -- `@deepseek-ai/dsh-loop-tool`: the model-facing `loop` tool plus the CLI `dsh-sdk loop` - -The design is organized around four concrete problems, addressed by service seams or explicit driver policies in the phase where each has a caller: - -1. A long-running loop that goes wrong leaves no systematic diagnosis or recovery. **Loop as an independent session** addresses this. -2. Whether the PASS at loop end is trustworthy determines whether hours of work are wasted. An architecture where the same LLM both generates and self-evaluates is not trustworthy on its face. **Making Evaluator and Budget into service seams** addresses this. -3. Short and long tasks need opposite memory strategies; hardcoding one mode makes the other class of scenario unusable. **Making RoundHandoff into a service seam** addresses this. -4. The user's initial goal is not always correct. An agent stubbornly pursuing a wrong goal exhausts the budget doing wrong work. **Goal concern events and policies cover Phase 1; the GoalReflector service arrives with the Phase 2 `reflect` path**. - -Beyond the four seams, one principle threads through the whole document: **one loop handles one atomic goal**. Large goals should be split into several small loops chained in sequence, not stuffed into one loop with the evaluator judging multiple things. A rule of thumb for whether granularity is right: if you cannot say what a finished loop actually accomplished, granularity is too large and should be split. Phase 2 adds a `loop_split` model-facing tool so the agent can split an oversized goal itself. - -Terminology: **inner loop** refers to the existing per-turn reasoning-and-tools cycle in `packages/core/agent-loop`; **harness loop** refers to the outer scheduler introduced by this RFC, iterating around the inner loop. This RFC does not modify `agent-loop`, matching AGENTS.md's "Plugins, not loop changes". - -`StopCondition` is a discriminated union with `assertNever` closing the switch: - -```ts -interface EvaluatorReport { criteria: readonly { name: string; pass: boolean; evidence: readonly string[] }[] } - -type StopCondition = - | { kind: 'goal-met'; evidence: EvaluatorReport } - | { kind: 'budget-cap'; scope: 'usd' | 'tokens' | 'rounds'; observed: number; maximum: number } - | { kind: 'stuck'; pattern: 'repeat-action' | 'no-progress' | 'error-loop' } - | { kind: 'approval-required'; reason: string } - | { kind: 'user-cancel' } - -export {} -``` - -### Loop as an independent session - -Once a long-running loop goes wrong, the user has no systematic diagnostic method. A failure hours in leaves only scattered log files to sift through. Discovering that some middle round went off track and wanting to roll back to re-run means starting over from scratch. An agent wanting to consult its own experience from past loops has no API to reach it. - -The driver opens an independent loop-session (a new session id) for each loop. Every round's inputs, inner-loop results, evaluator reports, and stop decisions are persisted as session events, reusing the SQLite backend from `packages/session-persistence`. This yields three diagnostic and replay capabilities. - -- **Replay conversation from a recorded round**: while the source session is live, discover round 78 went off and fork the round-77 event prefix with a different prompt or evaluator; persisted replay needs a separate trusted load-and-seed path. Both forms replay conversation state against the current workspace, not the files and external side effects that existed at round 77 -- **Post-hoc diagnosis**: through the existing `ctx.sessionQuery` exact-read service, inspect the round where the evaluator started hanging on the same criterion -- **Meta-loop learning**: the proposed [SQLite FTS5 search](2026-07-10-sqlite-session-query-provider.md) can later find related historical loops before a new run—"have I fixed a similar bug before? Which round did it fail on?" - -Claude Code's and Codex's `/goal` are one-off objects: discarded when the run ends, so the agent starts from zero when facing a similar problem again. - -**Storage and recovery boundary**. A few KB of events per round, roughly 100–500 KB per 100-round loop; thousands of loops reach GB scale. Mitigated by the `logDetail: 'summary' | 'full'` config, defaulting to `full` with long-run users able to switch to `summary`. Persisting all intermediate state also writes generated keys, passwords, and similar secrets to disk—the same class of risk as an ordinary session but amplified 10–100×, and the README calls this out clearly. Exact live and persisted reads already exist through `ctx.sessionQuery`; FTS5 is an optional discovery improvement, not a Phase 1 dependency. Exact execution-world restore is not promised: `SessionStore.fork()` accepts a live session, and session events do not restore files, processes, environment, or external side effects. Restoring those requires a separate Git/worktree/checkpoint design. - -### Pluggable Evaluator and Budget - -A loop's value ultimately depends on whether the final PASS is trustworthy. If the evaluator can be hacked or hallucinates PASS, hours of work are wasted. An architecture where the same LLM both generates and self-evaluates is not trustworthy on its face: the model has the means to talk itself into PASS. Even letting an independent subagent be the evaluator only mitigates the problem; as long as the evaluator is still an LLM, it retains a systematic bias for the same class of content—an independent subagent is a mitigation, not a cure. - -Trustworthy evaluation needs both a deterministic judgment mechanism and an isolation boundary appropriate to the threat model: shell exit code, static analysis, or an external service avoids LLM self-judgment, while a separate worktree, read-only mount, container, or remote service prevents the worker from rewriting evaluator inputs. Only the user knows which checks and boundary to use: `pytest` commands differ by project, companies have private compliance checkers, and some teams also run internal lint. No number of built-in evaluators can cover them all. Evaluator therefore must be a seam the user can plug into. - -Budget is the same story: product-level spending guardrails are opaque, and cannot be adjusted for team policy (personal card, team splitting, per-PR settlement). - -`Evaluator` and `BudgetPolicy` are both exposed as cordis service seams, with users injecting them as plugins. `Goal` must carry an `EvaluatorSpec` at an explicit tier; the driver refuses to start a loop without a paired evaluator—vague goals ("write good code") cannot enter the loop system: - -```ts -interface RubricItem { name: string; description: string } -interface EvaluatorContract { readonly name: string } - -type CriteriaSpec = - | { kind: 'single-metric'; name: string } - | { kind: 'rubric'; criteria: RubricItem[] } - | { kind: 'contract'; interface: EvaluatorContract } - -type ExecutorSpec = - | { kind: 'shell'; command: string } - | { kind: 'llm-judge'; rubric: string; model: string } - | { kind: 'provider'; name: string; config?: unknown } - -type IsolationSpec = 'same-workspace' | 'separate-worktree' | 'container' | 'remote' - -interface EvaluatorSpec { - criteria: CriteriaSpec - executor: ExecutorSpec - isolation: IsolationSpec -} - -export {} -``` - -**Why explicit dimensions instead of letting the user pass any function?** The spec forces the user, at start time, to declare what is judged, what executes the judgment, and what isolation boundary protects it. A free function looks flexible, but in practice it lets evaluator strength quietly regress—the user thinks they're doing a deterministic isolated check when they've actually written a same-workspace LLM judgment. In long-run scenarios the cost is hours wasted. - -Criteria shape, executor, and isolation are orthogonal rather than a trust ladder: a rubric may be checked by shell, an LLM, or an external service, and a contract may run in the same workspace or in a container. `llm-judge` remains the weakest executor for soft goals where there truly is no hard signal (revision, translation quality). The documentation explicitly notes "this executor cannot defend against targeted adversarial input; long unattended runs require at least one deterministic evaluator with an isolation boundary appropriate to the threat model". - -The driver enforces four structural constraints, not delegated to Evaluator implementations. Isolation strength remains an explicit property of the configured provider rather than a claim the driver can manufacture. - -**Preventing "the same agent both generates and self-evaluates"**: - -1. **fresh subagent for LLM evaluation**: an LLM evaluator opens a fresh subagent (via `dsh-subagent`'s `spawn`) each round, not inheriting the main loop's context - -**Preventing the evaluator subagent itself from being subverted**: - -2. **scoped tool set**: an LLM evaluator's model-facing tool set is stripped by the driver to reads only, with write tools disabled (reusing `SubagentProvider`'s `toolFilter`). This reduces accidental mutation but is not process isolation: shell, code runtimes, or another capability can still write unless the configured isolation boundary prevents it - -**Preventing the evaluator report itself from deceiving the driver**: - -3. **PASS can only flip via an evaluator report**: the `goal-met` StopCondition can only come from the evaluator; the driver and the main agent cannot construct it directly -4. **Default-FAIL**: the driver maintains each criterion's pass state at `false` internally; only an evaluator report with non-empty evidence is allowed to flip it to `true`. The evaluator cannot get the driver to accept a `{pass: true}` return with no evidence - -Together, the four ensure that evaluator conclusions are structurally evidence-driven rather than confidence-driven. They do not stop the main agent from modifying evaluator inputs in a shared workspace. - -**Phase 1 ships three backends**: - -- `loop-evaluator-shell` implements `single-metric`: runs a shell command, `exit 0` = pass -- `loop-evaluator-rubric-judge` implements `llm-judge`: a prewritten rubric plus LLM scoring, soft goals only -- `loop-budget-preflight`: before each round starts, estimate `(promptTokens + overhead + estOutputTokens) / 1M × pricePerMTok`; refuse to start if the estimate exceeds `perRoundUsd`. The estimation model comes from MartinLoop `policy.ts:551-596` - -A `PricingProvider` service injects the pricing table; a test seam can override it, and it is not hardcoded into the driver (AGENTS.md "No hardcoded tunables in plugins"). A resolved budget carries `maxRounds`, optional `maxTokens` and `maxUsd`, and optional `perRoundUsd`. The driver checks per-round admission before starting work, then accumulates worker, evaluator, compaction, and reflector usage after every request. A token or USD cap may overrun by one in-flight request because usage arrives after completion; the `budget-cap` result reports `observed` and `maximum`. The `rubric` and `contract` criteria shapes get built-in executors in Phase 2; Phase 1 exposes the shapes so third-party plugins can integrate first. - -**Limitation**: `same-workspace` plus a read-only model-facing tool set is not hard isolation. The current `packages/fs` policy enforces read-before-edit and version guards, not path denial, and bash or code runtimes can bypass filesystem tools. Defending against targeted adversarial input requires a boundary across every mutation channel—such as a read-only mount, isolated worktree, container, or remote evaluator. The two-container approach (the evaluator's definition files are entirely inaccessible to the main agent, the route Anthropic patch.py takes) remains a Phase 3 item. See Risks. - -### Pluggable RoundHandoff - -How context passes between rounds is a dilemma. Preserving the full prior conversation (continue) reads more coherently, but the conversation keeps growing and eventually hits the context ceiling, and errors from a prior round contaminate every subsequent round. Starting each round from scratch (fresh) avoids the contamination, but has to re-understand context every time. A 3-round revision loop and an 80-round overnight bug-fix loop need opposite strategies. Claude Code and Codex both hardcode one mode, so users cannot switch by task type. - -Made a service seam: - -```ts -interface ContinuationRun { - readonly id: string - resume?(prompt: string): Promise -} - -interface PreviousRound { - result: unknown - evaluator: { criteria: readonly { name: string; pass: boolean; evidence: readonly string[] }[] } - tokenUsage: number - summary: string - sessionId: string - run?: ContinuationRun -} - -interface RoundContext { loopId: string; round: number; previous: PreviousRound } - -type NextRoundSpec = - | { mode: 'fresh'; prompt: string } - | { mode: 'continue'; run: ContinuationRun; prompt: string } - -interface RoundHandoff { - buildNextRound(prev: RoundContext, signal: AbortSignal): Promise -} - -export {} -``` - -Phase 1 ships the fresh backend; Phase 2 adds the two continuation backends after provider continuation exists: - -| Backend | Phase | Scenario | Mechanism | -|---|---|---|---| -| `handoff-fresh-with-summary` (default) | Phase 1 | Long runs, unattended | Open a fresh subagent each round, injecting only a progress summary as a system prompt append | -| `handoff-continue-with-compaction` (recommended middle) | Phase 2 | Medium length, 5–20 rounds | Retain the full conversation up to a token threshold; over the threshold, reuse [`packages/compact`](../../../../packages/compact/README.md) to compress into a summary, using summary + last K rounds as the starting point | -| `handoff-continue-raw` (advanced) | Phase 2 | ≤5 rounds, short tasks, testing | Plain continuation without truncation | - -**Why default to fresh?** Every long-run loop that actually succeeded (repomirror, Kimi ralph-loop, autoresearch) uses fresh. Placing important loop state outside the context window under driver management is the correct posture for long runs. `handoff-continue-raw` violates this experience, and the README explicitly notes it is not suitable for long runs. - -**Why is only this repo able to build the middle tier?** `handoff-continue-with-compaction` depends on a compaction seam—the competitors don't have one; only this repo's `packages/compact` provides that infrastructure. - -**Why a seam rather than a three-choice flag?** Users can write 20-line plugins expressing hybrid strategies like "continue for the first 5 rounds, then fresh", or "auto-compact once when context hits 50%", without waiting for main-library support. - -**Limitation**: `continue-with-compaction` depends on the compression quality of `packages/compact`; compression itself may write hallucinated information into the summary and propagate it forward. The README recommends fresh for long runs. The three backends' boundaries may confuse new users about which to pick; the `dsh-sdk loop` CLI defaults to fresh, so users don't have to understand the differences before hitting a concrete problem. - -### Pluggable GoalReflector - -The goal the user gives at loop start is not always accurate. It may be based on a wrong assumption (asking the agent to implement a feature with a since-deprecated API), it may not be clear enough (the agent discovers a clarification is needed only mid-work), or it may be invalidated by later information. Current loop-execution frameworks treat the goal as a contract frozen at start; the agent can only push down the original path, and the result is exhausting the budget on the wrong direction. - -Phase 2 makes this a service seam, with responsibility separated from `Evaluator`: the evaluator asks "did we reach the goal", the reflector asks "is the goal still the same goal". Phase 1 carries concern events plus the `stop` and `notify-continue` driver policies without registering an unused `GoalReflector` service. - -```ts -interface RoundContext { loopId: string; round: number } -interface GoalConcern { concern: string; severity: 'low' | 'medium' | 'high' } - -interface GoalReflector { - reflect(ctx: RoundContext, concerns: GoalConcern[]): Promise -} - -type GoalReflection = - | { kind: 'continue' } // goal 仍有效 - | { kind: 'revise'; newGoal: string; why: string } // 建议修正 goal - | { kind: 'stop-for-human'; reason: string } // 需要人拍板 - -export {} -``` - -**Concerns have three sources**. Phase 1 ships the first two; the `GoalReflector` service and periodic source arrive together in Phase 2: - -- **Agent-initiated**: via the model-facing tool `loop_flag_concern({ concern, severity })`. An agent that realizes during investigation that "the library the user assumed has been deprecated" can raise directly -- **Driver heuristic**: when budget passes 50% and zero criteria have passed, the driver auto-raises a `no-progress-toward-goal` concern -- **Periodic reflector subagent** (Phase 2): every N rounds, run an independent read-only subagent to re-audit goal validity, following the same isolation approach as the evaluator - -**Response strategy is controlled by the `onGoalConcern` config**. The four settings correspond to different philosophies about loop use; users choose by their team's collaboration style, and the driver takes no default stance: - -- `'stop'` (Phase 1 default): any concern triggers `StopCondition: approval-required`, and a human decides. A loop should never proceed on its own in the face of uncertainty—suitable for cautious teams and for high-impact loop scenarios -- `'notify-continue'` (Phase 1): record an ordinary `loop/goal-concern` session event, then continue; a human reviews at the end. ACP has no general high-priority marker, so dedicated concern rendering is deferred with the ACP command infrastructure. The loop internal is not interrupted—suitable for unattended long runs -- `'reflect'` (Phase 2): call `GoalReflector` to decide continue, revise, or stop. Delegates the initial judgment to an independent agent in place of a human—suitable for teams with moderate autonomy -- Not registering a `GoalReflector` and leaving `onGoalConcern` unset = the most hands-off tier: the loop stops only on traditional stop conditions - -**Why default to `stop`?** In unattended scenarios, stopping one extra time is safer than running for hours in the wrong direction. Users who explicitly want unattended can switch to `notify-continue`. - -A concern is itself just an ordinary session event, composing naturally with the persistence capability described earlier: a later replay can seed a new conversation from the round where the concern surfaced and swap the goal. This does not roll the workspace back to that round. - -**Abuse and loss protection**. An agent could raise a concern every round; the mitigation is the `severity` field plus a minimal rate limit on the driver side (same-concern dedup within 30 seconds). The cost of that abuse is that the agent stalls itself and cannot make progress, so the incentive is weak. Once a goal has been revised, the original goal is lost; each revise persists a `loop/goal-revised` session event with rationale, and later replay can select any historical goal version without claiming workspace restoration. - -### User surface - -Four trigger surfaces share one driver: - -- **Agent-side tool**: `loop({ goal, evaluator, maxRounds, maxUsd, onGoalConcern })` registers `kind: 'loop'` through `ctx.tasks`, returns the task id immediately, and runs the harness loop in the background. `task_output`, `task_list`, and `task_kill` provide collection and cancellation. Inside a running loop, the internal agent can call `loop_flag_concern({ concern, severity })` to raise a concern proactively. ACP rendering intent is `generic`. An agent-initiated call is proactive triggering with no extra machinery -- **CLI**: `dsh-sdk loop --stop --max-rounds N --max-usd X --handoff fresh` is human-initiated startup, the most typical Ralph-style usage -- **cordis leaf**: declare a resident loop as a leaf in `cordis.yml`, with future `dsh-schedule` RFC integration for periodic triggering -- **ACP slash command**: `/loop ` (and `/loop-flag-concern`) starts directly from within the editor or client's current session. Semantically equivalent to a human typing `dsh-sdk loop` in a shell, but happens within the ongoing ACP session context, letting the loop result inject back into the session - -The ACP slash command depends on: `packages/ui/acp`'s `available_commands_update` surface is currently unbuilt ([acp-feature-support.md](../../../../packages/ui/acp/acp-feature-support.md)). Once the harness's slash-command infrastructure lands, `/loop` and `/loop-flag-concern` only need to be registered against that infrastructure; the driver and tool interfaces do not change. This RFC reserves the names and specifies the argument shape, but does not commit the infrastructure itself—that belongs to a separate ACP catch-up RFC. - -The default system prompt carries two behavioral instructions, distributed with every built-in `loop` tool: - -1. No writing of `TODO`, `FAKE`, or `PLACEHOLDER` placeholders to superficially pass the evaluator -2. No writing of empty `try/except` or `catch(_)` blocks so the evaluator ignores errors - -Neither can be enforced at the seam layer; both are prompt-layer guidance and must not be described as hard constraints. Users may customize the system prompt; evaluators that require these rules must check them explicitly. - -### Relationship with existing code - -Direct reuse without modification: - -- `packages/subagent`'s `spawn` provider, `toolFilter`, and `persona`—the loop spawns a subagent per round; an LLM evaluator gets a scoped model-facing tool set, not a process-isolation guarantee -- `packages/tasks`—the model-facing loop is a `loop` task producer and reuses owner isolation, `task_output`/`task_list`/`task_kill`, completion notices, cancellation, and awaited cleanup -- The SQLite backend from `packages/session-persistence`—the loop-session persists -- `packages/session-query`—exact live and persisted session reads for post-hoc diagnosis -- `packages/compact`—the implementation basis for `handoff-continue-with-compaction` -- `packages/todo`—an optional progress representation in single-session continue mode -- If [ToolExecution.reportProgress](2026-07-13-stream-workflow-progress-through-tool-calls.md) lands first, the loop tool can use it for per-round UI updates - -Not touched: `packages/core/agent-loop` (the inner-loop semantics stay the same); `packages/workflow` (DAG orchestration vs. iterating one goal is an orthogonal relationship; the two READMEs cross-link in their "Related" section to describe the boundary). - -One dependency is not yet landed: - -- The ACP slash-command infrastructure (the `available_commands_update` surface)—see User surface. Before the infrastructure lands, the slash-command trigger is absent while the other three trigger surfaces work as normal - -The proposed [SQLite FTS5 search](2026-07-10-sqlite-session-query-provider.md) is an optional Phase 2 discovery improvement over the existing exact-read query service, not a dependency for Phase 1 event access. - -Continuation work can be deferred to Phase 2: the `SubagentRun.sendMessage` and `resume` methods exist as optional seam capabilities, but the current `subagent-spawn` provider deliberately exposes neither. The two `handoff-continue-*` backends therefore require provider implementations, capability checks, ownership tests, and a consumer surface—not only a new argument on `packages/subagent-tool`. Phase 1 ships only `handoff-fresh-with-summary` and does not touch subagent continuation. - -### Phasing - -**Phase 1** (the scope this RFC commits): the three-package seam; `StopCondition`; the orthogonal criteria/executor/isolation `EvaluatorSpec`, with built-in implementations for shell and LLM-judge execution and rubric/contract criteria shapes open for integration; Default-FAIL enforcement; evaluator and cumulative-budget backends; `handoff-fresh-with-summary`; `ctx.tasks` integration; the `loop_flag_concern` tool; the no-progress heuristic; the `onGoalConcern: 'stop' | 'notify-continue'` pair; the CLI; the tool; and the default system-prompt guidance. **Not included**: the SQLite FTS5 search surface, the ACP slash-command trigger surface (depends on the `available_commands_update` infrastructure), subagent continuation provider/tool work, the `GoalReflector` service, the stuck detector, the Reflector subagent, the `loop_split` tool, and built-in executors for every rubric/contract combination. - -**Phase 2**: the SQLite FTS5 search surface; the stuck detector (reproducing OpenHands's five patterns); subagent continuation provider implementations, capability checks, and consumer surface (unlocking the two continue handoffs); the `GoalReflector` service and Reflector subagent; the `onGoalConcern: 'reflect'` tier; the `loop_split` model-facing tool; and built-in executors for additional rubric/contract combinations. - -**Phase 3**: agent fleet (N parallel loops for the same goal, best result wins); integration with `dsh-schedule`; two-container evaluator isolation (evaluator definition files entirely inaccessible to the main agent, defending against reward hacking). - -## Alternatives considered - -**Extend `packages/core/agent-loop`**: add an "iterate on end_turn until goal" switch to the inner loop. Rejected—AGENTS.md says "new behavior goes on documented extension seams; changing agent-loop requires updating docs/architecture.md". The harness loop needs state across sessions and across agents; stuffing it into the inner loop tangles session semantics into two mixed layers. - -**Ship a single slash command `/loop` (Claude Code clone)**: minimal implementation. Rejected—the slash-command layer does not resolve the harness/inner boundary; the four design points (queryable session, tiered evaluator, pluggable handoff, pluggable goal reflector) have nowhere to sit at the slash-command layer, and every capability this RFC commits is lost. - -**Fully outsource to `packages/workflow`**: express the loop as a workflow node with a back edge. Rejected—workflow lacks first-class semantics for iteration, StopCondition, and Evaluator; forcing it means the evaluator has to masquerade as a phase, violating the architectural-isolation requirement that the evaluator be independent of the producer; the budget guardrail in workflow is phase-level rather than round-level, and the granularities do not match. - -**Hardcode a binary choice between A (fresh) and B (continue)**: the Ralph school and the LoopTroop school each have strong scenarios. Rejected—Pluggable RoundHandoff proposes a seam plus three built-in backends that cover both schools and allow hybrids. - -**Skip the evaluator seam, ship a few built-ins**: lighter. Rejected—the core value of Pluggable Evaluator and Budget is that team-private evaluators can extend the system. Hardcoding leaves long unattended users no option but to modify the main library. - -**Accept a free function that lacks an explicit `EvaluatorSpec`**: allow users to pass any `(result) => boolean`. Rejected—the criteria/executor/isolation dimensions force users to declare at start time what is judged, what runs the judgment, and what boundary protects it, preventing quiet regression to a weaker setup. A free function looks flexible but lets evaluator strength quietly regress, and the cost is heavy in long-run scenarios. - -**Introduce an independent memory engine (Beads / dex-style)**: an established approach to external state. Rejected—`packages/session-persistence` plus the existing exact-read `ctx.sessionQuery` already cover Phase 1 diagnosis, while SQLite FTS5 can add search later; the payoff of a new engine is far smaller than the maintenance cost. - -**Fold goal reflection into the Evaluator seam** (have the evaluator return "criteria are impossible"): rejected—it conflates "was the goal achieved" with "is the goal still correct", which are orthogonal concerns. `Evaluator` should stay independent, read-only, and simple. - -**Only ever add an event for goal-concern, no seam**: lighter. Phase 1 does use the event plus `stop`/`notify` policies; rejected as the final design because the Phase 2 `reflect` path needs a replaceable response strategy. The seam lands with that first caller rather than ahead of it. - -**Ship the full Reflector subagent in Phase 1**: more complete. Rejected—`loop_flag_concern` tool plus no-progress heuristic plus the two-policy `onGoalConcern` already covers 80% of scenarios; running an independent subagent every round is expensive, and introducing it on demand in Phase 2 is more sensible. - -**Do not ship `loop_split`; let users split themselves**: Phase 1 already does. Phase 2 adds it because long-run scenarios reveal that agents receiving an oversized goal will run it directly rather than split it, so explicit tool guidance is needed. - -## Acceptance criteria - -- The three packages `packages/loop/{loop,loop-driver,loop-tool}` are built as a capability seam; `dsh-loop` exports only types and registry -- `StopCondition` discrimination covers all branches (unit); `assertNever` closes the switch at compile time -- The Phase 1 services `Evaluator`, `BudgetPolicy`, and `RoundHandoff` can each be replaced by an external plugin (fixture: inject a mock implementation, driver calls it correctly); no `GoalReflector` service is registered before the Phase 2 `reflect` consumer exists -- `EvaluatorSpec`'s criteria/executor/isolation dimensions converge at compile time; the driver refuses to start a loop without a paired evaluator (fixture: `loop({ goal, evaluator: undefined })` returns a configuration error immediately) -- Default-FAIL fixture: when the evaluator report returns `{criterion, pass: true, evidence: []}`, the driver refuses that criterion flip and records an `evaluator/invalid-report` session event -- `RoundHandoff` receives the previous result, evaluator report, token usage, summary, session id, optional run handle, and cancellation signal; Phase 1's `fresh-with-summary` has unit coverage plus one pass-path e2e, while continuation backend tests wait for Phase 2 provider support -- `dsh-sdk loop` CLI e2e: given a goal plus a 3-round cap plus one shell evaluator, both the pass and exhaustion paths return a structured stop cause with a semantic exit code -- Evaluator scoping fixture: the main agent has fs.write while an LLM evaluator's model-facing tool set does not; the result and documentation still label `same-workspace` as non-isolated, and no `protectedPaths` guarantee is exposed -- Budget fixtures cover `perRoundUsd` admission plus cumulative `maxRounds`, `maxTokens`, and `maxUsd` across worker and evaluator usage; an in-flight overrun emits `budget-cap` with `observed` and `maximum` -- Goal concern fixture: `loop_flag_concern` is callable from the main agent and yields an ordinary `loop/goal-concern` session event; under `onGoalConcern: 'stop'` an `approval-required` StopCondition is emitted; under `'notify-continue'` the loop continues without nonexistent ACP priority metadata; the no-progress heuristic fires once when budget exceeds 50% with zero passes (with rate-limit dedup) -- The default system-prompt guidance (no TODO/FAKE/PLACEHOLDER, no empty catch) is distributed with the built-in `loop` tool, and a snapshot covers the prompt content without treating it as enforcement -- Each round's prompt, inner-loop result, evaluator report, and stop decision appear as session events and are readable through the existing exact-read `ctx.sessionQuery`; FTS5 search remains Phase 2 -- Model-facing loop startup returns a `loop` task id immediately; `task_output`, `task_list`, `task_kill`, parent-agent disposal, cancellation, producer reload, and service disposal cover owner isolation and awaited quiescence -- The "Related" sections in `packages/loop/README.md` and `packages/workflow/README.md` cross-link and describe the "when to use workflow vs. when to use loop" boundary clearly -- Unit 100% / snapshot / e2e / doc-sync / verify-module-graph / build / hygiene all green; the ACP rendering intent (`generic`) of the new tool has a snapshot - -## Risks - -**Conversation replay is not workspace restore**. Exact session reads already exist, and FTS5 improves historical discovery rather than enabling correctness. Replaying a round prefix against the current workspace can diagnose or redirect a run, but reproducing the execution world at that round requires Git/worktree/checkpoint support and an explicit policy for external side effects. - -**The boundary between `packages/workflow` and loop is a recurring FAQ**. "Is multi-round a loop or a workflow?"—both READMEs must state clearly: workflow is "steps known, agent to run undecided, parallel or serial orchestration"; loop is "agent decided, round count undecided, evaluator decides when to stop". Unclear docs cause users to pick the wrong one. - -**Evaluator reverse-optimization (reward hacking)**. In a sufficiently long loop, the agent can identify the evaluator's pattern and optimize against it—for example, discovering that "as long as `assert True` appears in a test file, it PASSes" and bypassing real completion that way. Phase 1's `same-workspace` mode does not prevent the agent from modifying tests or evaluator configuration through bash, code runtimes, or another write channel; the current `packages/fs` policy is not a path-isolation boundary. Users needing adversarial strength must choose an isolated worktree, read-only mount, container, or remote evaluator. Phase 3's two-container approach keeps the evaluator runtime (binary, rubric, dependency libraries) entirely inaccessible to the main agent, matching what Anthropic patch.py does. - -**Placeholder faking and over-defensive code**. Agents sometimes write `# TODO: implement` to sneak through a test, or write large amounts of `try/except: pass` to make the evaluator superficially PASS. These do not belong to the evaluator layer; they are prompt and training issues at the agent-generation stage. The two default system-prompt instructions in User surface are guidance only; users who add "static-check-forbid TODO and empty catch" rules to a custom evaluator get enforceable coverage. This class of problem cannot be cured at the seam layer. - -**Budget estimation drift and in-flight overrun**. Pricing can change, and cumulative token/USD usage becomes exact only after each worker, evaluator, compaction, or reflector request reports usage. Preflight protects a single round; cumulative caps stop the next request and may exceed the configured maximum by one in-flight request. The README reports both observed and maximum values and states that provider billing remains authoritative. - -**Background tasks are process-local**. `ctx.tasks` gives the model-facing loop owner isolation, generic collection/cancellation, completion notices, and awaited cleanup. Parent-agent or service disposal cancels and awaits the loop; a process crash cannot run cleanup, and durable restart remains outside Phase 1. - -**Long-run loop log growth**. A 100-round loop reaches MB scale for one session. `logDetail: 'summary'` is a safety net but Phase 1 defaults to `full`; Phase 2 adds summary semantics. - -**Pre-release allows direct evolution**. `SESSION_FORMAT_VERSION=0`; the `LoopRoundEvent` schema can change at any time. Backends reject old formats rather than maintain compatibility, matching the pre-release stance at the top of AGENTS.md. diff --git a/.agents/notes/proposed/feature/2026-07-16-harness-level-loop.zh.md b/.agents/notes/proposed/feature/2026-07-16-harness-level-loop.zh.md deleted file mode 100644 index 6460254b07..0000000000 --- a/.agents/notes/proposed/feature/2026-07-16-harness-level-loop.zh.md +++ /dev/null @@ -1,343 +0,0 @@ -# Agent Note: harness 层 goal-based loop - -Status: proposed - -[English](2026-07-16-harness-level-loop.md) | 中文 - -## 问题 - -`packages/core/agent-loop` 只跑 inner loop：一次 turn 内推理加工具循环，模型返回 `end_turn` 就结束。其 README 明确写「No built-in turn budget」——预算是它自己承认的 gap。跨轮次调度落在 harness 层：跑到测试全绿、按 rubric 反复改稿、把 PRD 拆成 bead 逐个推进、无人值守跑一整晚。这几类任务今天都没有一等公民的实现。 - -现有代码里有三种「能凑合跑」的替代，都不够用： - -| 替代 | 问题 | -|---|---| -| `packages/workflow` 脚本表达 `while (!done)` | README 明写「No token-budget vocabulary」和「No journaling or resume」；父 turn 阻塞到脚本 settle。能跑几分钟的编排，跑不了几小时的长期任务 | -| 外部 shell `while :; do dsh-sdk …; done` | Ralph 风格的调度今天就能这么写。缺共享的 stop condition、budget、evaluator 词汇，每个使用者各自重发明；循环本身没有持久化对象可供事后诊断或恢复 | -| `packages/subagent` seam 的 `sendMessage`/`resume` | README 明写「Runtime steering and continuation are seam-only capabilities」。没有 model-facing consumer，模型只能起 fresh 子会话 | - -典型使用场景有三类。**自动化修复**：面前一个失败的测试套件，希望一个进程持续修改代码、跑测试、根据失败信息再修改，直到全绿或触达预算上限。**按 rubric 迭代改稿**：一份文档、代码或翻译需要满足打分标准，循环反复调整、独立评估者打分、直到达标或耗尽轮数。**无人值守长跑**：例如通宵把一个仓库从一种技术栈移植到另一种，下班前启动第二天回来看结果，全程只有预算兜底。三类共同的形态：几分钟到几小时、evaluator 决定成败、预算是硬约束、跑完还需要能回看和恢复。 - -## 提案 - -**Loop 有四种触发形态**，按谁在什么时候启动一轮划分： - -| 形态 | 谁触发 | 何时触发 | 现有对标 | 本 RFC | -|---|---|---|---|---| -| **turn-based** | 用户在会话里发一条消息 | 每一轮用户回复 | `packages/core/agent-loop` 现有一次 turn 内的推理与工具循环 | 不覆盖，已有实现 | -| **goal-based** | 用户或 agent 明确指定「跑到某条件为止」 | 一次启动，evaluator 判停 | Claude Code 的 `/goal`、Codex 的 `/goal`、Ralph 家族 | **本 RFC 覆盖** | -| **time-based** | scheduler | 按 cron 或时间间隔 | Claude Code 的 `/loop`（周期性）、`/schedule` | 延后到 `dsh-schedule` RFC | -| **proactive** | agent 自己 | agent 在推理中意识到需要开一个 loop 时 | Anthropic ClaudeDevs 4 类分类里的 proactive 档 | **本 RFC 自然包含**（agent 调 `loop` tool 就是 proactive） | - -本 RFC 只**新增 capability seam `packages/loop/`** 处理 goal-based 一种。proactive 复用同一 `loop` tool，agent 主动调用即触发，无需额外机制。time-based 需要独立的 scheduler package，属于另一份 RFC 的事情；本 RFC 只在 cordis leaf 触发面预留跟未来 `dsh-schedule` 联动的钩子。 - -三个包： - -- `@deepseek-ai/dsh-loop`：类型、`LoopDriver` service、`StopCondition` 判别联合、Phase 1 service 定义（`Evaluator` / `BudgetPolicy` / `RoundHandoff`）、事件 schema；`GoalReflector` 在 Phase 2 与首个调用方一起加入 -- `@deepseek-ai/dsh-loop-driver`：默认 driver 实现 -- `@deepseek-ai/dsh-loop-tool`：model-facing `loop` tool + CLI `dsh-sdk loop` - -设计围绕四个具体问题展开，在每项能力出现调用方的 phase 中通过 service seam 或显式 driver policy 解决： - -1. 长跑 loop 出问题后缺诊断和恢复手段。**loop 作为独立 session** 解决。 -2. loop 结束时的 PASS 是否可信决定几小时工作是否作废。同一个 LLM 既生成又自评的架构本身就不可信。**Evaluator 与 Budget 做成 service seam** 解决。 -3. 短任务和长任务需要的记忆策略相反，硬编一种模式会让另一类场景不可用。**RoundHandoff 做成 service seam** 解决。 -4. 用户初始给的 goal 未必始终正确。agent 沿着错的目标蛮干会耗尽预算做错事。**Phase 1 用 goal concern event 与 policy 处理；GoalReflector service 随 Phase 2 的 `reflect` 路径一起加入**。 - -四条 seam 之外还有一条贯穿全文的原则：**一个 loop 只处理一个原子目标**。大目标拆成若干小 loop 串联，不塞进一个 loop 让 evaluator 判定多件事。判定 granularity 是否合适的经验规则：如果 loop 跑完说不清它到底做完了什么，granularity 就太大，应当拆。Phase 2 补 `loop_split` model-facing tool 让 agent 收到过大 goal 时能自己拆。 - -术语约定：**inner loop** 指 `packages/core/agent-loop` 一次 turn 的推理与工具循环；**harness loop** 指本 RFC 引入的外层调度器，围绕 inner loop 反复迭代。本 RFC 不改 `agent-loop`，符合 AGENTS.md「Plugins, not loop changes」。 - -`StopCondition` 是 discriminated union，`assertNever` 收口： - -```ts -interface EvaluatorReport { criteria: readonly { name: string; pass: boolean; evidence: readonly string[] }[] } - -type StopCondition = - | { kind: 'goal-met'; evidence: EvaluatorReport } - | { kind: 'budget-cap'; scope: 'usd' | 'tokens' | 'rounds'; observed: number; maximum: number } - | { kind: 'stuck'; pattern: 'repeat-action' | 'no-progress' | 'error-loop' } - | { kind: 'approval-required'; reason: string } - | { kind: 'user-cancel' } - -export {} -``` - -### Loop 作为独立 session - -长跑 loop 一旦出错，用户没有系统的诊断手段。跑几小时后失败，只能翻散落的日志文件。发现中间某一轮走偏想倒回去重跑，只能从头开始。agent 想参考自己过去 loop 的经验也没有可用的 API。 - -Driver 为每个 loop 开一个独立的 loop-session（新的 session id）。每轮的输入、inner-loop 结果、evaluator 报告、stop 决策都作为 session event 落盘，复用 `packages/session-persistence` 的 SQLite backend。得到三种诊断与 replay 能力。 - -- **从已记录轮次 replay 对话**：源 session 仍 live 时，发现第 78 轮偏航，可以 fork 第 77 轮的 event prefix，换 prompt 或 evaluator；已持久化 session 的 replay 还需要独立的受信任 load-and-seed 路径。两者都只会基于当前工作区 replay 对话状态，不会恢复第 77 轮的文件与外部副作用 -- **事后诊断**：通过现有 `ctx.sessionQuery` 精确读取 service 检查 evaluator 从哪一轮开始一直挂在同条 criterion 上 -- **元循环学习**：拟议中的 [SQLite FTS5 search](2026-07-10-sqlite-session-query-provider.md) 后续可以在新 loop 启动前找到相关历史 loop——「我以前 fix 过类似的 bug 吗？失败在哪一轮？」 - -Claude Code、Codex 的 `/goal` 是一次性对象：跑完就丢，agent 下次遇到同类问题从零开始。 - -**存储与恢复边界**。每轮几 KB events，100 轮 loop 约 100–500 KB；跑几千个 loop 会到 GB 级。`logDetail: 'summary' | 'full'` 配置缓解，默认 `full`，长跑用户可切 `summary`。中间态全持久化会把生成过的 key、密码一并落盘，跟普通 session 是同一类风险但量放大 10–100 倍，README 明确提示。通过 `ctx.sessionQuery` 的精确 live 与已持久化读取已经存在；FTS5 是可选的发现能力增强，不是 Phase 1 依赖。本 RFC 不承诺精确恢复执行世界：`SessionStore.fork()` 只接受 live session，而 session event 不会恢复文件、进程、环境或外部副作用。这需要单独的 Git/worktree/checkpoint 设计。 - -### 可插拔的 Evaluator 与 Budget - -loop 的价值最终取决于结束时的 PASS 是否可信。如果 evaluator 会被 hack 或幻觉 PASS，前面几小时的工作全部作废。同一个 LLM 既生成又自评的架构本身就不可信：模型有条件说服自己 PASS。即便让独立 subagent 做 evaluator，只要 evaluator 还是 LLM，就仍然对同类内容有系统性偏好——独立 subagent 只是缓解不是根治。 - -可信评估同时需要确定性的判断机制，以及与 threat model 匹配的隔离边界：shell exit code、静态分析或外部服务避免 LLM 自评；独立 worktree、只读 mount、容器或远程服务防止 worker 改写 evaluator 输入。具体检查和边界只有用户知道：不同项目 `pytest` 命令不同、公司有私有合规检查器、有些团队还要跑内部 lint。主库无论内置几种都覆盖不全。所以 evaluator 必须做成用户可以自己接入的 seam。 - -预算方面同理：产品级的花费护栏是黑盒，无法按团队策略调整（个人卡、团队分摊、按 PR 结算）。 - -`Evaluator` 和 `BudgetPolicy` 都作为 cordis service seam 暴露。`Goal` 必须携带一个明确档位的 `EvaluatorSpec`，driver 拒绝启动没有 evaluator 配对的 loop——含糊的目标（"把代码写好"）不能进入 loop 系统： - -```ts -interface RubricItem { name: string; description: string } -interface EvaluatorContract { readonly name: string } - -type CriteriaSpec = - | { kind: 'single-metric'; name: string } - | { kind: 'rubric'; criteria: RubricItem[] } - | { kind: 'contract'; interface: EvaluatorContract } - -type ExecutorSpec = - | { kind: 'shell'; command: string } - | { kind: 'llm-judge'; rubric: string; model: string } - | { kind: 'provider'; name: string; config?: unknown } - -type IsolationSpec = 'same-workspace' | 'separate-worktree' | 'container' | 'remote' - -interface EvaluatorSpec { - criteria: CriteriaSpec - executor: ExecutorSpec - isolation: IsolationSpec -} - -export {} -``` - -**为什么使用显式维度，而不是让用户传自由函数？** spec 强制用户在启动时声明评估什么、由什么执行判断，以及什么隔离边界保护它。自由函数看起来灵活，实际让 evaluator 强度隐性下沉——用户以为在做确定性隔离检查，实际写的是同工作区 LLM 判断。长跑场景下代价是几小时白跑。 - -criteria shape、executor 与 isolation 是三个正交维度，不是可信度阶梯：rubric 可以由 shell、LLM 或外部服务检查，contract 也可以在同一工作区或容器中运行。`llm-judge` 仍是最弱的 executor，仅用于确实无硬信号的软目标（改稿、翻译质量）。文档明确标注「此 executor 不能挡定向对抗，长跑无人值守场景至少需要一个确定性 evaluator，并配合与 threat model 匹配的隔离边界」。 - -Driver 强制四条结构约束，不下放给 Evaluator 实现。隔离强度仍是已配置提供方的显式属性，不是 driver 能凭空制造的保证。 - -**防「同一个 agent 既生成又自评」**： - -1. **LLM 评估使用 fresh subagent**：LLM evaluator 每轮开 fresh subagent（用 `dsh-subagent` 的 `spawn`），不继承主循环 context - -**防 evaluator subagent 自身被 subverted**： - -2. **限制模型可见工具集**：LLM evaluator 的 model-facing tool set 被 driver 剥离到只保留读类工具，写类工具禁用（复用 `SubagentProvider` 的 `toolFilter`）。这会减少意外修改，但不是进程隔离：除非已配置隔离边界拦截，否则 shell、代码运行时或其他 capability 仍能写入 - -**防 evaluator 报告本身欺骗 driver**： - -3. **PASS 只能由 evaluator 报告翻转**：`goal-met` StopCondition 只能来自 evaluator，driver 或主 agent 都不能直接构造 -4. **Default-FAIL**：driver 内部维护每个 criterion 的 pass 状态默认 `false`，只有 evaluator 报告里带非空 evidence 才允许翻 `true`；evaluator 无法通过返回 `{pass: true}` 而不给 evidence 让 driver 接受 - -四条一起保证 evaluator 结论在结构上由证据推动，而不是由自信推动。它们不能阻止主 agent 在共享工作区中修改 evaluator 输入。 - -**Phase 1 内置三个 backend**： - -- `loop-evaluator-shell` 实现 `single-metric`：跑 shell 命令，`exit 0` = pass -- `loop-evaluator-rubric-judge` 实现 `llm-judge`：预写 rubric + LLM 打分，仅软目标 -- `loop-budget-preflight`：每轮启动前估 `(promptTokens + overhead + estOutputTokens) / 1M × pricePerMTok`，超 `perRoundUsd` 拒绝启动。估算模型来自 MartinLoop `policy.ts:551-596` - -`PricingProvider` 服务注入 pricing 表，test seam 可覆盖，不硬编到 driver 里（AGENTS.md「No hardcoded tunables in plugins」）。解析后的 budget 携带 `maxRounds`、可选 `maxTokens` 与 `maxUsd`，以及可选 `perRoundUsd`。driver 在启动工作前检查单轮准入，随后在每次请求后累计 worker、evaluator、compaction 和 reflector 用量。token 或 USD 上限可能被一个在途请求超出，因为 usage 在完成后才到达；`budget-cap` 结果同时报告 `observed` 与 `maximum`。`rubric` 与 `contract` criteria shape 在 Phase 2 补内置 executor，Phase 1 暴露这些 shape 让第三方插件先接。 - -**局限**：`same-workspace` 加只读 model-facing tool set 不是硬隔离。当前 `packages/fs` policy 实施 read-before-edit 与版本保护，不是路径拒写；bash 或代码运行时可以绕过 filesystem tool。挡定向对抗需要覆盖所有写入通道的边界，例如只读 mount、隔离 worktree、容器或远程 evaluator。两容器方案（evaluator 定义文件对主 agent 完全不可访问，Anthropic patch.py 走的就是这条路）仍在 Phase 3。见 风险。 - -### 可插拔的 RoundHandoff - -每轮之间如何传递 context 是一个两难。完整保留之前对话（continue）连续性好，但对话会持续增长最终撞上 context 上限，且上一轮的错误信息会污染后续每一轮。每轮从零开始（fresh）避免污染，但每次需要重新理解上下文。跑 3 轮改稿与跑 80 轮 overnight 修 bug 需要的策略是相反的。Claude Code、Codex 都硬编一种模式，用户没法按任务类型切换。 - -做成 service seam： - -```ts -interface ContinuationRun { - readonly id: string - resume?(prompt: string): Promise -} - -interface PreviousRound { - result: unknown - evaluator: { criteria: readonly { name: string; pass: boolean; evidence: readonly string[] }[] } - tokenUsage: number - summary: string - sessionId: string - run?: ContinuationRun -} - -interface RoundContext { loopId: string; round: number; previous: PreviousRound } - -type NextRoundSpec = - | { mode: 'fresh'; prompt: string } - | { mode: 'continue'; run: ContinuationRun; prompt: string } - -interface RoundHandoff { - buildNextRound(prev: RoundContext, signal: AbortSignal): Promise -} - -export {} -``` - -Phase 1 交付 fresh backend；Phase 2 在 provider continuation 存在后增加两个 continuation backend： - -| Backend | Phase | 场景 | 机制 | -|---|---|---|---| -| `handoff-fresh-with-summary`（默认） | Phase 1 | 长跑、无人值守 | 每轮开 fresh subagent，只注入一段 progress 摘要作 system prompt 附加段 | -| `handoff-continue-with-compaction`（推荐中间档） | Phase 2 | 5–20 轮的中等长度 | 整段对话保留到 token 阈值，超了复用 [`packages/compact`](../../../../packages/compact/README.md) 压缩，摘要 + 最近 K 轮作起点 | -| `handoff-continue-raw`（专业档） | Phase 2 | ≤5 轮短任务、测试 | 纯连续对话不裁剪 | - -**为什么默认 fresh？** 所有实际跑成的长跑 loop（repomirror、Kimi ralph-loop、autoresearch）用的都是 fresh。把重要 loop 状态放在 context window 外由 driver 管理是长跑的正确姿势。`handoff-continue-raw` 违反这条经验，README 明写长跑不适用。 - -**为什么中间档只有我们能做？** `handoff-continue-with-compaction` 依赖 compaction seam——竞品都没有，只有本仓库 `packages/compact` 提供了这个基础设施。 - -**为什么做成 seam 而不是三选一 flag？** 用户可以写 20 行插件表达「前 5 轮 continue、之后 fresh」这类混合策略，或表达「context 到 50% 自动 compact 一次」，不用等主库支持。 - -**局限**：`continue-with-compaction` 依赖 `packages/compact` 的压缩质量，压缩本身可能把幻觉信息写进摘要传下去；README 建议长跑首选 fresh。三个 backend 的边界会让新用户不知道选哪个；`dsh-sdk loop` CLI 默认用 fresh，用户在遇到具体问题前不需要理解这些差别。 - -### 可插拔的 GoalReflector - -用户在启动 loop 时给的目标不一定准确。可能基于错误假设（让 agent 用某个已经废弃的 API 实现功能），可能不够清晰（agent 在做的过程中才发现需要澄清），也可能被后来的信息证伪。现在的循环执行框架把 goal 当作启动时冻结的合约，agent 只能沿着原路蛮干，结果是在错的方向上耗尽预算。 - -Phase 2 把它做成 service seam，与 `Evaluator` 职责分离：evaluator 问「是否达成目标」，reflector 问「目标是否还是那个目标」。Phase 1 只携带 concern event，以及 `stop` 与 `notify-continue` driver policy，不注册没有调用方的 `GoalReflector` service。 - -```ts -interface RoundContext { loopId: string; round: number } -interface GoalConcern { concern: string; severity: 'low' | 'medium' | 'high' } - -interface GoalReflector { - reflect(ctx: RoundContext, concerns: GoalConcern[]): Promise -} - -type GoalReflection = - | { kind: 'continue' } // goal 仍有效 - | { kind: 'revise'; newGoal: string; why: string } // 建议修正 goal - | { kind: 'stop-for-human'; reason: string } // 需要人拍板 - -export {} -``` - -**concern 有三种触发来源**。Phase 1 实现前两种；`GoalReflector` service 与周期性来源一起在 Phase 2 加入： - -- **agent 主动**：通过 model-facing tool `loop_flag_concern({ concern, severity })`。agent 在调研中意识到「用户假设的那个库已经废弃」时可以直接 raise -- **driver 启发式**：预算过 50% 且零 criterion pass 时，driver 自动 raise `no-progress-toward-goal` concern -- **周期性 reflector subagent**（Phase 2）：每 N 轮独立跑一个只读 subagent 复审 goal 有效性，与 evaluator 独立性遵循同一思路 - -**响应策略通过 `onGoalConcern` 配置项**。这四种配置对应不同的 loop 使用哲学，用户按团队协作方式选，driver 不预设立场： - -- `'stop'`（Phase 1 默认）：任何 concern 都触发 `StopCondition: approval-required`，人拍板。loop 在遇到任何不确定性时都不应自己往下走，适合谨慎风格团队与影响面较大的 loop 场景 -- `'notify-continue'`（Phase 1）：记录普通 `loop/goal-concern` session event 后继续跑，人在结束时集中审阅。ACP 没有通用高优先级 marker，因此专用 concern 渲染与 ACP command 基础设施一起后置。loop 内部不打扰，适合无人值守长跑 -- `'reflect'`（Phase 2）：调 `GoalReflector` 决定 continue、revise 还是 stop。委派一个独立 agent 代替人做初步判断，适合中等自主度的团队 -- 不注册 `GoalReflector` 且 `onGoalConcern` 未设 = 最放手档，loop 只在传统 stop condition 触发时停 - -**为什么默认选 `stop`？** 无人值守场景下宁可多停一次也不要在错方向上跑几小时。用户明确要无人值守可切 `notify-continue`。 - -concern 本身就是普通 session event，跟前文的持久化 session 能力天然协同：后续 replay 可以从 concern 出现的轮次为新对话提供 seed，并替换 goal。这不会把工作区回滚到该轮。 - -**滥用与丢失防护**。agent 可能每轮都 raise concern；缓解是 `severity` 字段和 driver 侧的最小 rate limit（同一 concern 30 秒内去重）。这种滥用的代价是 agent 卡住自己无法推进，动机不强。goal 被 revise 后原始 goal 会丢失；每次 revise 落 `loop/goal-revised` session event 带 rationale，后续 replay 可选任意历史 goal 版本，但不承诺恢复工作区。 - -### 用户面 - -四个触发面共享同一个 driver： - -- **agent 侧 tool**：`loop({ goal, evaluator, maxRounds, maxUsd, onGoalConcern })` 通过 `ctx.tasks` 注册 `kind: 'loop'`，立即返回 task id，并在后台运行 harness loop。`task_output`、`task_list` 和 `task_kill` 负责收集与取消。正在跑的 loop 内部 agent 可用 `loop_flag_concern({ concern, severity })` 主动发起 concern。ACP 渲染意图为 `generic`。agent 自主发起就是 proactive 触发，无需额外机制 -- **CLI**：`dsh-sdk loop --stop --max-rounds N --max-usd X --handoff fresh`。人类主导启动，最典型的 Ralph 风格用法 -- **cordis leaf**：`cordis.yml` 里以 leaf 形式声明常驻循环，配合未来的 `dsh-schedule` RFC 可做周期性触发 -- **ACP slash command**：`/loop `（还有 `/loop-flag-concern`）在编辑器/客户端的当前会话里直接启动。语义等价于人类在 CLI 里敲 `dsh-sdk loop`，但发生在正在进行的 ACP session 上下文中，允许 loop 结果直接注入会话 - -ACP slash command 的依赖：`packages/ui/acp` 的 `available_commands_update` 面目前是 unbuilt 状态（[acp-feature-support.md](../../../../packages/ui/acp/acp-feature-support.md)）。等 harness 的 slash command 基础设施落地，`/loop` 与 `/loop-flag-concern` 只需在该基础设施里注册；driver 与 tool 接口不变。本 RFC 保留名字并给出参数 shape，但不承诺基础设施本身——那属于独立的 ACP 补齐 RFC。 - -默认 system prompt 里有两条行为指令，随所有内置 `loop` tool 一起分发： - -1. 不允许写 `TODO`、`FAKE`、`PLACEHOLDER` 占位符让 evaluator 表面通过 -2. 不允许写空的 `try/except` 或 `catch(_)` 让 evaluator 忽略错误 - -这两条无法在 seam 层强制，只是 prompt 层 guidance，不能描述成硬约束。用户可以自定义 system prompt；需要强制这些规则的 evaluator 必须显式检查。 - -### 与仓库现有代码的关系 - -直接复用无需修改： - -- `packages/subagent` 的 `spawn` provider、`toolFilter`、`persona`——loop 每轮起 subagent；LLM evaluator 获得受限的 model-facing tool set，不获得进程隔离保证 -- `packages/tasks`——model-facing loop 是 `loop` task producer，复用 owner isolation、`task_output`/`task_list`/`task_kill`、完成通知、取消和 awaited cleanup -- `packages/session-persistence` 的 SQLite backend——loop-session 落盘 -- `packages/session-query`——精确读取 live 与已持久化 session，用于事后诊断 -- `packages/compact`——`handoff-continue-with-compaction` 的实现基础 -- `packages/todo`——单会话 continue 模式下作为可选 progress 表达 -- 若 [ToolExecution.reportProgress](2026-07-13-stream-workflow-progress-through-tool-calls.md) 先落地，loop tool 可用它逐轮 UI 更新 - -不动：`packages/core/agent-loop`（inner loop 语义保持）；`packages/workflow`（DAG 编排 vs. 迭代同 goal 是 orthogonal 关系，两个 README 在「Related」段互链说明边界）。 - -依赖尚未落地的一处： - -- ACP slash command 基础设施（`available_commands_update` 面）——见 用户面。基础设施落地前，slash command 触发面缺席，其它三个触发面照常工作 - -拟议中的 [SQLite FTS5 search](2026-07-10-sqlite-session-query-provider.md) 是现有 exact-read query service 之上的可选 Phase 2 发现能力增强，不是 Phase 1 event 访问的依赖。 - -Continuation 工作可以延后到 Phase 2：`SubagentRun.sendMessage` 与 `resume` 方法作为可选 seam capability 存在，但当前 `subagent-spawn` provider 明确不暴露这两个方法。因此，两个 `handoff-continue-*` backend 需要 provider 实现、capability check、ownership 测试和 consumer surface，不只是给 `packages/subagent-tool` 增加参数。Phase 1 只交付 `handoff-fresh-with-summary`，不改 subagent continuation。 - -### 分阶段 - -**Phase 1**（本 RFC 承诺范围）：三包 seam；`StopCondition`；criteria／executor／isolation 三个正交维度的 `EvaluatorSpec`，其中 shell 与 LLM-judge execution 有内置实现，rubric／contract criteria shape 开放待接；Default-FAIL 强制；evaluator 与累计 budget backend；`handoff-fresh-with-summary`；`ctx.tasks` 集成；`loop_flag_concern` tool；no-progress 启发式；`onGoalConcern: 'stop' | 'notify-continue'` 二档；CLI；tool；默认 system prompt guidance。**不含**：SQLite FTS5 search 面、ACP slash command 触发面（依赖 `available_commands_update` 基础设施）、subagent continuation provider/tool 工作、`GoalReflector` service、stuck 检测器、Reflector subagent、`loop_split` tool，以及每种 rubric／contract 组合的内置 executor。 - -**Phase 2**：SQLite FTS5 search 面；stuck 检测器（复现 OpenHands 5 种模式）；subagent continuation provider 实现、capability check 与 consumer surface（解锁两个 continue handoff）；`GoalReflector` service 与 Reflector subagent；`onGoalConcern: 'reflect'` 档；`loop_split` model-facing tool；更多 rubric／contract 组合的内置 executor。 - -**Phase 3**：agent fleet（同 goal 派 N 个并行 loop 取最优）；与 `dsh-schedule` 集成；两容器 evaluator 隔离（evaluator 定义文件对主 agent 完全不可访问，防 reward 反向优化）。 - -## 备选方案 - -**扩 `packages/core/agent-loop`**：给 inner loop 加「iterate on end_turn until goal」开关。拒绝——AGENTS.md「新行为走文档化扩展 seam；改 agent-loop 需要更新 docs/architecture.md」。harness loop 需要跨 session、跨 agent 的状态，塞进 inner loop 会把 session 语义拧成两层混合。 - -**只做一个 slash command `/loop`（Claude Code 复刻）**：实现最简。拒绝——slash-command 层不解决 harness/inner 边界；四条设计要点（可查询 session、分档 evaluator、可插拔 handoff、可插拔 goal reflector）在 slash-command 层没有承载点，本 RFC 承诺的能力全部丢失。 - -**全权外包给 `packages/workflow`**：把 loop 表达成带回边的 workflow 节点。拒绝——workflow 缺 iteration、StopCondition、Evaluator 的一等公民语义。硬用会把 evaluator 冒充成一个 phase，违反 evaluator 独立于 producer 的架构隔离要求；预算护栏在 workflow 是 phase-level 而非 round-level，粒度对不上。 - -**A（fresh）vs. B（continue）硬编二选一**：Ralph 派和 LoopTroop 派各自都有强场景。拒绝——可插拔的 RoundHandoff 提出 seam + 三档内置 backend 涵盖两派并允许 hybrid。 - -**不做 evaluator seam，内置几种够用**：更轻。拒绝——可插拔的 Evaluator 与 Budget 的核心价值是团队或私有 evaluator 可扩展。写死后长跑无人值守场景的用户只能改主库。 - -**接受不带显式 `EvaluatorSpec` 的自由函数**：允许用户传任意 `(result) => boolean`。拒绝——criteria／executor／isolation 维度强制用户在启动时声明评估什么、由什么执行判断、由什么边界保护，防止不知不觉滑到更弱的配置。自由函数看起来灵活，实际让 evaluator 强度隐性下沉，长跑场景代价大。 - -**引入独立记忆引擎（Beads / dex-style）**：外部化状态的成熟做法。拒绝——`packages/session-persistence` 加现有 exact-read `ctx.sessionQuery` 已经覆盖 Phase 1 诊断，SQLite FTS5 后续可以补 search；新引擎收益远小于维护成本。 - -**goal reflection 塞进 Evaluator seam**（让 evaluator 返回「criteria 不可能满足」）：拒绝——混淆「是否成功」和「目标是否正确」两个正交问题。`Evaluator` 应保持独立、只读、简单。 - -**goal-concern 永远只做 event 不做 seam**：更轻。Phase 1 确实使用 event 加 `stop`／`notify` policy；作为最终设计仍拒绝，因为 Phase 2 的 `reflect` 路径需要可替换响应策略。seam 与首个调用方一起落地，不提前出现。 - -**Phase 1 就上完整 Reflector subagent**：更全。拒绝——`loop_flag_concern` tool + no-progress 启发式 + 二档 policy 覆盖 80% 场景；每轮跑独立 subagent 成本高，Phase 2 按需引入更合理。 - -**不做 `loop_split`，用户自己拆**：Phase 1 已经如此。Phase 2 加是因为长跑场景发现 agent 收到过大 goal 会直接跑而不是自己拆，需要显式工具引导。 - -## 验收标准 - -- `packages/loop/{loop,loop-driver,loop-tool}` 三包按 capability seam 建成；`dsh-loop` 只导 types 与 registry -- `StopCondition` 判别覆盖所有分支（单元），`assertNever` 编译期收口 -- Phase 1 的 `Evaluator`、`BudgetPolicy`、`RoundHandoff` 三条 service 都能被外部插件替换（fixture：注入 mock 实现，driver 正确调用）；Phase 2 `reflect` consumer 出现前不注册 `GoalReflector` service -- `EvaluatorSpec` 的 criteria／executor／isolation 维度在编译期收敛；driver 拒绝启动没有 evaluator 配对的 loop（fixture：`loop({ goal, evaluator: undefined })` 立即返回配置错误） -- Default-FAIL fixture：evaluator 报告返回 `{criterion, pass: true, evidence: []}` 时 driver 拒绝该 criterion 翻转、记 `evaluator/invalid-report` session event -- `RoundHandoff` 接收上一轮 result、evaluator report、token usage、summary、session id、可选 run handle 和 cancellation signal；Phase 1 的 `fresh-with-summary` 有单元覆盖与一个 pass-path e2e，continuation backend 测试等待 Phase 2 provider 支持 -- `dsh-sdk loop` CLI e2e：给定 goal + 3 轮上限 + 一个 shell evaluator，通过与耗尽两条路径都返回结构化 stop cause 并 exit code 语义化 -- Evaluator scope fixture：主 agent 有 fs.write，LLM evaluator 的 model-facing tool set 没有；结果与文档仍把 `same-workspace` 标记为未隔离，不暴露 `protectedPaths` 保证 -- Budget fixture 覆盖 `perRoundUsd` 准入，以及跨 worker 与 evaluator usage 累计的 `maxRounds`、`maxTokens`、`maxUsd`；在途超限 emit 带 `observed` 与 `maximum` 的 `budget-cap` -- Goal concern fixture：`loop_flag_concern` 可从主 agent 调用并产出普通 `loop/goal-concern` session event；`onGoalConcern: 'stop'` 下 emit `approval-required` StopCondition；`'notify-continue'` 下继续跑且不携带不存在的 ACP priority metadata；no-progress 启发式在预算超 50% 且零 pass 时自动触发一次（rate-limit 去重） -- 默认 system prompt guidance（无 TODO/FAKE/PLACEHOLDER、无空 catch）随内置 `loop` tool 一起分发，snapshot 覆盖 prompt 内容但不把它当作强制机制 -- 每轮的 prompt、inner-loop 结果、evaluator report、stop 决策都以 session event 出现，并可通过现有 exact-read `ctx.sessionQuery` 读取；FTS5 search 留在 Phase 2 -- model-facing loop 启动后立即返回 `loop` task id；`task_output`、`task_list`、`task_kill`、父 agent dispose、取消、producer reload 和 service dispose 覆盖 owner isolation 与 awaited quiescence -- `packages/loop/README.md` 和 `packages/workflow/README.md` 的「Related」段互链清楚「何时用 workflow、何时用 loop」的边界 -- 单元 100% / snapshot / e2e / doc-sync / verify-module-graph / build / hygiene 全绿；新增 tool 的 ACP 渲染意图（`generic`）有 snapshot - -## 风险 - -**对话 replay 不是工作区恢复**。精确 session 读取已经存在，FTS5 改善历史发现能力，不决定正确性。基于当前工作区 replay 某一轮 prefix 可以诊断或改变运行方向，但复现该轮执行世界需要 Git/worktree/checkpoint 支持，以及针对外部副作用的显式 policy。 - -**`packages/workflow` 与 loop 的边界是持续答疑热点**。「多轮是 loop 还是 workflow」两个 README 必须写清楚：workflow 是「步骤已知、agent 未定、并串行编排」；loop 是「agent 已定、轮数未定、evaluator 判停」。文档不清晰会让用户混用错档。 - -**evaluator 反向优化（reward hacking）**。足够长的 loop 里 agent 有条件识别 evaluator 的模式并针对性优化，例如发现「只要测试文件里出现 `assert True` 就 PASS」从而绕过实质完成。Phase 1 的 `same-workspace` 模式不能阻止 agent 通过 bash、代码运行时或其他写入通道修改测试或 evaluator 配置；当前 `packages/fs` policy 不是路径隔离边界。需要对抗强度的用户必须选择隔离 worktree、只读 mount、容器或远程 evaluator。Phase 3 的两容器方案让 evaluator 整个运行时（二进制、rubric、依赖库）对主 agent 完全不可访问，Anthropic patch.py 走的就是这条路。 - -**占位符伪造与过度防御码**。agent 有时会写 `# TODO: implement` 让测试勉强通过，或写大量 `try/except: pass` 让 evaluator 表面 PASS。这些不属于 evaluator 层的问题，而是 agent 生成阶段的 prompt 与训练问题。用户面 段的两条默认 system prompt 指令只是 guidance；用户在自定义 evaluator 中加入「静态检查禁止 TODO 与空 catch」才能获得可强制覆盖。这类问题不是 seam 层能根治的。 - -**预算估算漂移与在途超限**。pricing 可能变化，累计 token／USD usage 只能在每次 worker、evaluator、compaction 或 reflector 请求报告 usage 后精确。preflight 保护单轮；累计上限会停止下一个请求，但可能被一个在途请求超出。README 同时报告 observed 与 maximum，并说明 provider 账单才是权威。 - -**后台 task 只存在于当前进程**。`ctx.tasks` 为 model-facing loop 提供 owner isolation、通用收集／取消、完成通知和 awaited cleanup。父 agent 或 service dispose 会取消并等待 loop；进程 crash 无法执行 cleanup，持久重启不在 Phase 1 范围内。 - -**长跑 loop 日志膨胀**。跑 100 轮 loop 单 session 上 MB 级。`logDetail: 'summary'` 兜底但 Phase 1 默认 `full`，Phase 2 再补 summary 语义。 - -**pre-release 允许直接演进**。`SESSION_FORMAT_VERSION=0`，`LoopRoundEvent` schema 可随时改；后端拒收旧格式而非兼容，与 AGENTS.md 顶部 pre-release stance 一致。 diff --git a/AGENTS.md b/AGENTS.md index 860f60f1e0..6a0e4ca13b 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -112,9 +112,9 @@ Real-API tests and demos read `DEEPSEEK_API_KEY`, optional `DEEPSEEK_BASE_URL`, - **Prefer symmetry for parallel values**; unexplained asymmetry usually signals a missed extraction. - **Tests describe behavior, not correctness.** Change obsolete behavior with its tests; explain why in the PR. - **Every non-trivial change MUST include at least one Agent Note in the same PR.** Update the owning note or add one, validate its premises against code, and exempt only mechanical/local edits ([scope](.agents/notes/README.md#when-to-write-one)). -- **Testing policy** — [docs/testing.md](docs/testing.md). Transcript changes need snapshots or a PR note. Fixtures must replay on macOS/Linux; fix fixtures, not normalizers. +- **Testing policy** — [docs/testing.md](docs/testing.md). Every non-trivial model- or human-visible change adds or updates a keyless snapshot through a real runnable example in the same PR; package tests, e2e-only assertions, and mock-only fixtures do not substitute for the assembled application transcript. Fixtures must replay on macOS/Linux; fix fixtures, not normalizers. - **A tool's ACP render intent is part of its design**, decided up front (`generic`/`terminal`/`diff`, `locations`); presentation methods are pure functions of `args` ([cookbook](docs/cookbook/adding-a-tool.md)). -- **Plan unit, e2e, and snapshot coverage** for new seams, lifecycle shapes, and transcript surfaces, and schedule any missing harness support before implementation. +- **Plan unit, e2e, and snapshot coverage** for new seams, lifecycle shapes, and transcript surfaces; missing snapshot-harness support is part of the implementation, not deferred follow-up. - **Keep PRs coherent and merge with merge commits.** Split an independently meaningful feature or design decision into a separate or stacked PR when combining it obscures ownership, intent, or verification. Never squash/rebase or rewrite pushed branches; put a review fix on its introducing PR, then merge down the stack ([guide](docs/cookbook/responding-to-pr-review-on-a-stack.md)). - TODO markers: `FIXME`/`TODO`/`XXX` by urgency ([semantics](docs/development.md)). - Files end with exactly one trailing newline; `git diff --check` (pre-push) gates it. diff --git a/docs/testing.md b/docs/testing.md index c158c9b922..85601ab6c9 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -37,4 +37,4 @@ An e2e assertion re-runs the command or re-reads the file externally; a keyword ## When a snapshot test is required -Any change affecting an editor-facing transcript, headless event stream, or end-to-end agent UX adds or updates a scenario in the owning snapshot suite, or states in the PR why none applies. ACP surfaces use `examples//tests/snapshots/`, a scenario table over the [`dsh-acp-snapshot`](../packages/support/acp-snapshot/README.md) suite factory (`examples/acp-agent` is primary); `examples/headless-agent` owns the `stream-json` snapshot and replay fixtures. Completed interactive-terminal journeys use JSONL-driven scenarios under `examples/tui-agent/tests/snapshots/`; transient presentation uses the package-local semantic matrix, with a PTY case when input, Loader selection, or terminal teardown changes. New capability seams, lifecycle shapes, or transcript surfaces name their coverage at every tier at plan time and verify the harness can express it — a harness gap is scheduled work, not a mid-build surprise. +Every non-trivial model- or human-visible change adds or updates a keyless scenario in the same PR through a runnable example's owning snapshot suite. Package tests, e2e assertions, mock/test-only compositions, and PR rationale do not replace the assembled transcript; extend the harness when needed. ACP surfaces use `examples//tests/snapshots/`, a scenario table over the [`dsh-acp-snapshot`](../packages/support/acp-snapshot/README.md) suite factory (`examples/acp-agent` is primary); `examples/headless-agent` owns the `stream-json` snapshot and replay fixtures. Completed interactive-terminal journeys use JSONL-driven scenarios under `examples/tui-agent/tests/snapshots/`; transient presentation uses the package-local semantic matrix, with a PTY case when input, Loader selection, or terminal teardown changes. New capability seams, lifecycle shapes, or transcript surfaces name every coverage tier at plan time and verify the harness can express it before implementation.