Merge remote-tracking branch 'origin/master' into codex/enforce-tool-cancellation
This commit is contained in:
@@ -40,7 +40,7 @@ description: Use when reviewing a pull request in the deepseek-harness repo —
|
||||
- **Test strength:** assertions fail on the intended regression and verify external state, logs, events, or disposal rather than restating the implementation or trusting an agent's report. Coverage is necessary but not evidence that the scenario is correct.
|
||||
- **Mechanized invariants and negative controls:** trace each new or changed check through the executed top-level gate and its deliberately invalid case; confirm the real runner fails for the intended rule.
|
||||
- **Implemented RFCs match shipped reality:** when a PR implements a proposed RFC, move and rewrite it as present-tense shipped state in the same diff, then verify paths, names, and mechanisms against the implementation.
|
||||
- **Transcript changes:** editor-visible or model-visible changes update snapshots or explain why no snapshot applies. Review golden diffs as behavior changes, not formatting noise.
|
||||
- **Transcript changes:** editor-visible or model-visible changes update snapshots or explain why no snapshot applies. Review expected-output diffs as behavior changes, not formatting noise.
|
||||
- **Bilingual changes:** compare meaning and terminology on both sides; a green pairing hash does not prove translation quality.
|
||||
|
||||
## Reporting findings
|
||||
|
||||
@@ -24,7 +24,7 @@ A strong simplification removes, folds, or demotes something real and has clear
|
||||
- A seam has methods every implementation must support but no consumer uses.
|
||||
- A package boundary exists only for test/demo/support code and adds publish or dependency overhead.
|
||||
- A feature implements speculative product generality: multi-session/session-load, background task rosters, live registry invalidation, mid-turn steering, tool-owned UI rendering, and similar shapes with no product owner.
|
||||
- An invariant, rollback path, goldens set, or special-case test exists only to protect an unused surface.
|
||||
- An invariant, rollback path, set of expected outputs, or special-case test exists only to protect an unused surface.
|
||||
- The simplified behavior may differ slightly, but the new behavior is still reasonable and easier to explain.
|
||||
|
||||
Thin candidates are usually not enough for an RFC: deleting one typo, running `knip` once, removing an intentionally documented backend/adapter, or flagging "this looks complex" without call-site proof.
|
||||
@@ -37,7 +37,7 @@ Use parallel subagents when the user asks for breadth or many candidates. Give e
|
||||
- ACP and UI surfaces: `session/*` methods, terminal `_meta`, transcript rendering, single vs multi-session state.
|
||||
- LLM/tools/system prompt: stream/generate surfaces, assemblers, registries, tool schema defaults, presentation hooks.
|
||||
- Bash and tool execution: foreground/background split, task ownership, output spill files, executor methods.
|
||||
- Packages/examples/scripts/tests: package boundaries, static inventories, redundant snapshot goldens, support packages.
|
||||
- Packages/examples/scripts/tests: package boundaries, static inventories, redundant snapshot expected outputs, support packages.
|
||||
|
||||
If subagents are unavailable, simulate the same breadth yourself. Do not let the first good candidate stop the survey.
|
||||
|
||||
@@ -54,7 +54,7 @@ For complex asynchronous code, draw the ownership graph and map each sentinel, r
|
||||
For every symbol or behavior, classify consumers before writing:
|
||||
|
||||
- Production corpus: `packages/*/src`, `examples/*/src`, `examples/**/*.yml`, runtime scripts, and loader/config paths.
|
||||
- Non-production corpus: tests, README/docs, RFCs, snapshots, generated goldens, and comments.
|
||||
- Non-production corpus: tests, README/docs, RFCs, snapshots, generated expected outputs, and comments.
|
||||
- Ambiguous corpus: examples and scripts that may be product smoke paths. Inspect usage before classifying.
|
||||
|
||||
Use `rg` first. Good searches include the exact symbol, event name, package name, config key, method name with both `.name(` and `name(`, and any wire strings. Then read the call sites. `knip` can help, but it is not a substitute for understanding public interfaces, dynamic event names, tests, docs, and Cordis loader paths.
|
||||
|
||||
@@ -0,0 +1,21 @@
|
||||
name: Expected filenames
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
paths:
|
||||
- '*[gG][oO][lL][dD][eE][nN]*'
|
||||
- '**/*[gG][oO][lL][dD][eE][nN]*'
|
||||
- '!vendor/**'
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
expected-filenames:
|
||||
name: no golden filenames
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
- name: Check tracked filenames
|
||||
run: scripts/check-expected-filenames.sh
|
||||
@@ -0,0 +1,78 @@
|
||||
name: E2E (pi-ai Azure OpenAI and Anthropic)
|
||||
|
||||
# This suite spends tokens against two external providers and is intentionally
|
||||
# opt-in. It has no push, pull_request, schedule, or workflow_call trigger.
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
azure_openai_model:
|
||||
description: Azure OpenAI model from pi-ai's installed catalog
|
||||
required: true
|
||||
default: gpt-5.5
|
||||
type: string
|
||||
anthropic_model:
|
||||
description: Anthropic model from pi-ai's installed catalog
|
||||
required: true
|
||||
default: claude-opus-4-8
|
||||
type: string
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
e2e:
|
||||
runs-on: ubuntu-latest
|
||||
name: Azure OpenAI Responses + Anthropic Messages
|
||||
timeout-minutes: 20
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
- uses: actions/setup-node@v6
|
||||
with:
|
||||
node-version: 24
|
||||
|
||||
- name: Enable corepack (pnpm)
|
||||
run: corepack enable
|
||||
|
||||
- name: Resolve pnpm store path
|
||||
id: pnpm-store
|
||||
run: echo "path=$(pnpm store path --silent)" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- uses: actions/cache@v4
|
||||
with:
|
||||
path: ${{ steps.pnpm-store.outputs.path }}
|
||||
key: ${{ runner.os }}-node-24-pnpm-${{ hashFiles('pnpm-lock.yaml') }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-node-24-pnpm-
|
||||
|
||||
- name: Install (immutable)
|
||||
run: pnpm install --frozen-lockfile
|
||||
|
||||
# The tests self-skip locally when a credential is absent. A manually
|
||||
# dispatched CI run must fail instead of reporting an all-skipped green.
|
||||
- name: Preflight (require provider API keys)
|
||||
env:
|
||||
AZURE_OPENAI_API_KEY: ${{ secrets.AZURE_OPENAI_API_KEY_EXTERNAL }}
|
||||
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY_EXTERNAL }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
missing=0
|
||||
for name in AZURE_OPENAI_API_KEY ANTHROPIC_API_KEY; do
|
||||
if [ -z "${!name:-}" ]; then
|
||||
echo "::error::${name} is empty. Configure the corresponding *_EXTERNAL repository secret."
|
||||
missing=1
|
||||
fi
|
||||
done
|
||||
exit "$missing"
|
||||
|
||||
- name: E2E tests (real Azure OpenAI and Anthropic APIs)
|
||||
env:
|
||||
AZURE_OPENAI_API_KEY: ${{ secrets.AZURE_OPENAI_API_KEY_EXTERNAL }}
|
||||
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY_EXTERNAL }}
|
||||
DSH_PI_AI_OPENAI_MODEL: ${{ inputs.azure_openai_model }}
|
||||
DSH_PI_AI_OPENAI_BASE_URL: https://openai-routerhub-resource.services.ai.azure.com/api/projects/openai/openai/v1
|
||||
DSH_PI_AI_ANTHROPIC_MODEL: ${{ inputs.anthropic_model }}
|
||||
DSH_E2E_MAX_WORKERS: 2
|
||||
run: >-
|
||||
pnpm exec vitest run --config vitest.e2e.config.ts
|
||||
packages/llm/llm-pi-ai/tests/provider-apis.e2e.ts
|
||||
@@ -47,8 +47,8 @@ pnpm install # pnpm workspaces, node ^22.19 || >=24
|
||||
pnpm run test # vitest unit tests
|
||||
pnpm run test:coverage # THE gating test run: per-file 100% coverage on packages/*/*/src
|
||||
pnpm run test:e2e # real-API tests; self-skip without DEEPSEEK_API_KEY
|
||||
pnpm run test:snapshot # keyless ACP/headless/TUI replay vs goldens; filter: -t <name>
|
||||
pnpm run test:snapshot:record # re-record goldens (needs key)
|
||||
pnpm run test:snapshot # keyless ACP/headless/TUI replay vs expected outputs; filter: -t <name>
|
||||
pnpm run test:snapshot:record # re-record expected outputs (needs key)
|
||||
pnpm run typecheck
|
||||
pnpm run lint
|
||||
pnpm run duplication # cross-file TypeScript clone detection
|
||||
|
||||
+14
-7
@@ -64,7 +64,7 @@ export interface Config {
|
||||
skills?: agentCore.SkillConfig
|
||||
/** Model-facing bash tool config forwarded through agent-core. */
|
||||
toolBash?: NonNullable<agentCore.Config['toolBash']>
|
||||
/** Generic background-task control-tool config forwarded through agent-core. */
|
||||
/** Generic background-task controls forwarded through agent-core; set false to omit their tool surface. */
|
||||
toolTasks?: NonNullable<agentCore.Config['toolTasks']>
|
||||
}
|
||||
```
|
||||
@@ -140,12 +140,14 @@ export interface Config {
|
||||
skills?: SkillConfig
|
||||
/** Model-facing bash tool config, including this producer's background opt-in. */
|
||||
toolBash?: toolBash.Config
|
||||
/** Generic background-task control-tool wait bounds. */
|
||||
toolTasks?: toolTasks.Config
|
||||
/** Generic background-task controls; set false to keep the task service without model-facing task tools. */
|
||||
toolTasks?: toolTasks.Config | false
|
||||
}
|
||||
|
||||
/** Skill bundle config forwarded to the registry, local provider, and model-facing consumer. */
|
||||
export interface SkillConfig {
|
||||
/** Mount the bundled local skill provider and model-facing skill tool (default true). */
|
||||
enabled?: boolean
|
||||
/** Registry-level discovery cache settings. */
|
||||
registry?: SkillRegistryConfig
|
||||
/** Local filesystem skill provider settings. */
|
||||
@@ -157,7 +159,7 @@ export interface SkillConfig {
|
||||
|
||||
Depends on: [`AgentLoopConfig`](#deepseek-aidsh-agent-loop) · [`SkillLocal`](../packages/skill/skill-local/src/index.ts) · [`SkillRegistryConfig`](#deepseek-aidsh-skill) · [`SystemPromptConfig`](#deepseek-aidsh-system-prompt) · [`toolBash`](../packages/bash/tool-bash/src/index.ts) · [`ToolsConfig`](#deepseek-aidsh-tools) · [`toolSkill`](../packages/skill/tool-skill/src/index.ts) · [`toolTasks`](../packages/tasks/tool-tasks/src/index.ts) · [`workspaceContext`](../packages/context/workspace-context/src/index.ts)
|
||||
|
||||
Source: [`packages/examples/agent-spine-demo/src/index.ts:57`](../packages/examples/agent-spine-demo/src/index.ts)
|
||||
Source: [`packages/examples/agent-spine-demo/src/index.ts:59`](../packages/examples/agent-spine-demo/src/index.ts)
|
||||
|
||||
## `@deepseek-ai/dsh-bash-local`
|
||||
|
||||
@@ -172,7 +174,9 @@ export interface Config {
|
||||
maxTimeoutMs?: number
|
||||
/** Per-stream in-memory output cap; overflow spills to a temp file. */
|
||||
maxOutputBytes?: number
|
||||
/** Grace period between the SIGTERM and the SIGKILL escalation on a kill. */
|
||||
/** Per-stream spill-file cap; larger streams retain only their in-memory tail. */
|
||||
maxSpillBytes?: number
|
||||
/** Grace period for kill escalation and for inherited pipes after shell exit. */
|
||||
graceMs?: number
|
||||
}
|
||||
```
|
||||
@@ -384,8 +388,10 @@ Source: [`packages/hooks/hooks-codex/src/index.ts:42`](../packages/hooks/hooks-c
|
||||
Requires: `agents`
|
||||
|
||||
```ts config-catalog
|
||||
/** Runtime-only test seams; no field is configurable from `cordis.yml`. */
|
||||
/** JSON-RPC deployment config plus runtime-only test seams. */
|
||||
export interface JsonRpcConfig {
|
||||
/** Report max-token turn/subagent termination as a successful SDK result. */
|
||||
maxTokensAsSuccess?: boolean
|
||||
/** Transport input override; production uses `process.stdin`. */
|
||||
input?: Readable
|
||||
/** Transport output override; production uses `process.stdout`. */
|
||||
@@ -856,7 +862,7 @@ export interface Config {
|
||||
skills?: agentCore.SkillConfig
|
||||
/** Model-facing bash tool config forwarded through agent-core. */
|
||||
toolBash?: NonNullable<agentCore.Config['toolBash']>
|
||||
/** Generic background-task control-tool config forwarded through agent-core. */
|
||||
/** Generic background-task controls forwarded through agent-core; set false to omit their tool surface. */
|
||||
toolTasks?: NonNullable<agentCore.Config['toolTasks']>
|
||||
/**
|
||||
* If set, the pre-created agent RESUMES this persisted session id instead of
|
||||
@@ -1501,4 +1507,5 @@ Imported as libraries by other packages; a `cordis.yml` cannot load them.
|
||||
- `@deepseek-ai/dsh-scripts` ([`packages/sdk/scripts/src/index.ts`](../packages/sdk/scripts/src/index.ts))
|
||||
- `@deepseek-ai/dsh-subagent-inprocess` ([`packages/subagent/subagent-inprocess/src/index.ts`](../packages/subagent/subagent-inprocess/src/index.ts))
|
||||
- `@deepseek-ai/dsh-subagent-subprocess` ([`packages/subagent/subagent-subprocess/src/index.ts`](../packages/subagent/subagent-subprocess/src/index.ts))
|
||||
- `@deepseek-ai/dsh-telemetry` ([`packages/sdk/telemetry/src/index.ts`](../packages/sdk/telemetry/src/index.ts))
|
||||
- `@deepseek-ai/dsh-timeout` ([`packages/util/timeout/src/index.ts`](../packages/util/timeout/src/index.ts))
|
||||
@@ -106,6 +106,7 @@
|
||||
| event stream | 事件流 | | | |
|
||||
| event-sourced | 事件溯源 | | | 沿用 DDD 社区通行译法 |
|
||||
| executor | 执行器 | | | |
|
||||
| expected output | 预期输出 | | 金标 | 指 snapshot 比较产物;翻译语料的人工校准样例不在此列 |
|
||||
| extension | 扩展 | | | |
|
||||
| extension point | 扩展点 | | | 注意与 `seam` 区分 |
|
||||
| fail-fast | 快速失败 | | | |
|
||||
|
||||
@@ -140,6 +140,7 @@ flowchart TD
|
||||
subgraph group_sdk["packages/sdk"]
|
||||
pkg_helper["helper"]
|
||||
pkg_scripts["scripts"]
|
||||
pkg_telemetry["telemetry"]
|
||||
end
|
||||
subgraph group_tasks["packages/tasks"]
|
||||
pkg_tasks["tasks"]
|
||||
@@ -154,6 +155,7 @@ flowchart TD
|
||||
pkg_code_runtime_worker --> pkg_code_runtime
|
||||
pkg_helper --> pkg_brand
|
||||
pkg_scripts --> pkg_app_boot
|
||||
pkg_telemetry --> pkg_brand
|
||||
pkg_llm_deepseek --> pkg_llm
|
||||
pkg_llm_pi_ai --> pkg_llm
|
||||
pkg_session --> pkg_brand
|
||||
@@ -463,6 +465,7 @@ flowchart TD
|
||||
| [`code-runtime-worker`](../packages/code-runtime/code-runtime-worker) | `code-runtime` | [`code-runtime`](../packages/code-runtime/code-runtime) |
|
||||
| [`helper`](../packages/sdk/helper) | `sdk` | [`brand`](../packages/util/brand) |
|
||||
| [`scripts`](../packages/sdk/scripts) | `sdk` | [`app-boot`](../packages/ui/app-boot) |
|
||||
| [`telemetry`](../packages/sdk/telemetry) | `sdk` | [`brand`](../packages/util/brand) |
|
||||
| [`llm-deepseek`](../packages/llm/llm-deepseek) | `llm` | [`llm`](../packages/llm/llm) |
|
||||
| [`llm-pi-ai`](../packages/llm/llm-pi-ai) | `llm` | [`llm`](../packages/llm/llm) |
|
||||
| [`session`](../packages/core/session) | `core` | [`brand`](../packages/util/brand), [`llm`](../packages/llm/llm), [`scope`](../packages/core/scope) |
|
||||
|
||||
@@ -4,7 +4,7 @@ Status: resolved
|
||||
|
||||
## Executive summary
|
||||
|
||||
The ACP example attempted to enable filesystem plugins conditionally with `disabled: !!js ...`, but Cordis evaluates JavaScript expressions only inside plugin `config`. The raw expression object was truthy, so the filesystem stack was always disabled. Snapshot refresh then accepted `UNKNOWN_TOOL` results as new goldens. The fix uses an explicit filesystem overlay and adds static-config and snapshot-result guards.
|
||||
The ACP example attempted to enable filesystem plugins conditionally with `disabled: !!js ...`, but Cordis evaluates JavaScript expressions only inside plugin `config`. The raw expression object was truthy, so the filesystem stack was always disabled. Snapshot refresh then accepted `UNKNOWN_TOOL` results as new expected outputs. The fix uses an explicit filesystem overlay and adds static-config and snapshot-result guards.
|
||||
|
||||
## Summary
|
||||
|
||||
@@ -22,7 +22,7 @@ The live confined default did not gain unintended filesystem access. A naive int
|
||||
|
||||
- PR #261 consolidated ACP compositions and refreshed the filesystem snapshots while introducing conditional filesystem entries.
|
||||
- All unit, coverage, snapshot, documentation, build, and hygiene checks passed.
|
||||
- Review of the refreshed filesystem goldens found generic failed cards and structured `UNKNOWN_TOOL` results.
|
||||
- Review of the refreshed filesystem expected outputs found generic failed cards and structured `UNKNOWN_TOOL` results.
|
||||
- A real Loader boot confirmed that every `disabled` value remained an expression object and every filesystem fiber was absent.
|
||||
|
||||
## Root cause
|
||||
@@ -36,10 +36,10 @@ The snapshot framework treated any deterministic transcript as valid behavior. H
|
||||
- Filesystem scenarios boot `fs.cordis.yml`, an explicit fixed full-access overlay with a paired replay config and its own request-header class.
|
||||
- [`AGENTS.md`](../../AGENTS.md) and the [Cordis primer](../cordis-primer.md#loader-configuration) state that `!!js` is valid only under plugin `config` and conditional composition uses overlays.
|
||||
- `verify-cordis-config` parses repository Cordis YAML and rejects expression nodes in Loader entry metadata, including include patches and inserted entries.
|
||||
- `dsh-acp-snapshot` rejects structured `UNKNOWN_TOOL` results in fresh runs and committed session fixtures before they can become accepted goldens.
|
||||
- `dsh-acp-snapshot` rejects structured `UNKNOWN_TOOL` results in fresh runs and committed session fixtures before they can be committed as expected outputs.
|
||||
|
||||
## Lessons
|
||||
|
||||
- A syntactically accepted configuration value is not necessarily evaluated at that location; document and verify interpolation boundaries.
|
||||
- A snapshot refresh is fixture production, not correctness review. Semantic impossibilities such as a missing registered tool need assertions independent of the golden.
|
||||
- A snapshot refresh is fixture production, not correctness review. Semantic impossibilities such as a missing registered tool need assertions independent of the expected output.
|
||||
- Permission controls must describe only the capabilities they actually govern. Composition-time filesystem access cannot follow a runtime bash-only preset safely.
|
||||
+3
-2
@@ -15,6 +15,7 @@ Generated by `pnpm run gen-rfc-index` from the RFC tree — never edit by hand;
|
||||
| [SQLite FTS5 session search](proposed/feature/2026-07-10-sqlite-session-query-provider.md) | 2026-07-10 |
|
||||
| [Stream workflow progress through tool calls](proposed/feature/2026-07-13-stream-workflow-progress-through-tool-calls.md) | 2026-07-13 |
|
||||
| [Developer-owned SDK projects](proposed/feature/2026-07-14-sdk-developer-projects.md) | 2026-07-14 |
|
||||
| [SDK follow-up capabilities](proposed/feature/2026-07-17-sdk-follow-up-capabilities.md) | 2026-07-17 |
|
||||
|
||||
### Simplification
|
||||
|
||||
@@ -208,11 +209,11 @@ Generated by `pnpm run gen-rfc-index` from the RFC tree — never edit by hand;
|
||||
| [Property-based testing for protocol-shaped code](implemented/testing/2026-06-11-property-based-testing.md) | 2026-06-11 |
|
||||
| [ACP snapshot tests — record-once / replay-deterministic](implemented/testing/2026-06-19-acp-snapshot-tests.md) | 2026-06-19 |
|
||||
| [Real-API e2e in CI against the external DeepSeek API](implemented/testing/2026-06-19-real-api-e2e-ci.md) | 2026-06-19 |
|
||||
| [Use `session.jsonl` as the only snapshot session-log artifact](implemented/testing/2026-06-20-remove-redundant-snapshot-log-goldens.md) | 2026-06-20 |
|
||||
| [Use `session.jsonl` as the only snapshot session-log artifact](implemented/testing/2026-06-20-remove-redundant-snapshot-log-expected-output.md) | 2026-06-20 |
|
||||
| [Persist the seed boundary so fork-child replay routes correctly](implemented/testing/2026-06-22-fork-child-replay-seed-boundary.md) | 2026-06-22 |
|
||||
| [Record fork and mixed spawn+fork snapshot scenarios](implemented/testing/2026-06-22-fork-snapshot-scenarios.md) | 2026-06-22 |
|
||||
| [Per-session snapshot replay for nested agents](implemented/testing/2026-06-22-subagent-snapshot-replay.md) | 2026-06-22 |
|
||||
| [Hook snapshot matrix — end-to-end goldens for both bridges](implemented/testing/2026-07-04-hook-snapshot-matrix.md) | 2026-07-04 |
|
||||
| [Hook snapshot matrix — end-to-end expected outputs for both bridges](implemented/testing/2026-07-04-hook-snapshot-matrix.md) | 2026-07-04 |
|
||||
| [Single-source the acp-agent replay config](implemented/testing/2026-07-04-single-source-acp-replay-config.md) | 2026-07-04 |
|
||||
| [Pin request-header content in one snapshot scenario](implemented/testing/2026-07-06-pin-request-header-content-in-one-scenario.md) | 2026-07-06 |
|
||||
| [Extract the ACP snapshot suite into a support package](implemented/testing/2026-07-08-shared-acp-snapshot-package.md) | 2026-07-08 |
|
||||
|
||||
@@ -10,7 +10,7 @@ A tool declares how its calls render in a UI (an editor's tool-call card) throug
|
||||
- Which combinations are *valid* is unwritten: a `terminal` call that also sets `content` means "description above the card"; a generic call that sets `terminal` is meaningless but representable. The type permits nonsense.
|
||||
- There is no way to express the one file-tool affordance an editor most wants — a **diff card** (`{path, oldText, newText}`, which Zed renders as an inline diff / new-file preview). `ToolCallPresentation.content` is the *LLM* `ContentBlock[]` vocabulary (text/image), so a tool literally cannot ask for a diff.
|
||||
|
||||
The existing `FIXME(tool-presentation)` in `packages/core/tools/src/index.ts` named the fix: "redesign the type so a tool declares its render INTENT once (e.g. a tagged union over card kinds) rather than a bag of optional fields the bridge stitches together." The rejected RFC [Collapse tool-owned UI presentation](../../rejected/simplification/2026-06-20-generic-tool-rendering.md) deferred it explicitly: rich rendering "should return later as a tagged render-intent union after there are at least two real tools and two real consumers to validate the vocabulary." That bar is now met — two producer families (`dsh-tool-bash`, `dsh-tool-fs`) and two consumers (the ACP bridge live path + the snapshot-golden replay path).
|
||||
The existing `FIXME(tool-presentation)` in `packages/core/tools/src/index.ts` named the fix: "redesign the type so a tool declares its render INTENT once (e.g. a tagged union over card kinds) rather than a bag of optional fields the bridge stitches together." The rejected RFC [Collapse tool-owned UI presentation](../../rejected/simplification/2026-06-20-generic-tool-rendering.md) deferred it explicitly: rich rendering "should return later as a tagged render-intent union after there are at least two real tools and two real consumers to validate the vocabulary." That bar is now met — two producer families (`dsh-tool-bash`, `dsh-tool-fs`) and two consumers (the ACP bridge live path + the snapshot replay path).
|
||||
|
||||
## Decision
|
||||
|
||||
|
||||
@@ -50,5 +50,5 @@ Like MiniCode, the conversation advances append-only and resets only when model-
|
||||
- The `step/start`-listener behavior change (above) is the one observable semantics change for plugins; `agent/pre-step` is the current-request seam.
|
||||
- Tool-result trimming (planned) needs no new mechanism: a logged single-entry surface replace (`start === end`) carrying a trimmed `tool/result` under the same `callId` — compaction-family, replay-correct, cache-bust batched by the same pressure logic.
|
||||
- Session logs grow one `request/header` snapshot per loop instance plus snapshots on real changes. This is larger than a delta codec but small beside chunk-heavy logs and retains one replay representation. `SESSION_FORMAT_VERSION` stays `0`; legacy delta events are rejected rather than migrated.
|
||||
- Snapshot goldens changed once (every transcript gains its header events); the fs-writing fixtures are stored in the normalized authored form with cwd-relative tool arguments, because replay only round-trips cwd-independent argument paths.
|
||||
- Snapshot expected outputs changed once (every transcript gains its header events); the fs-writing fixtures are stored in the normalized authored form with cwd-relative tool arguments, because replay only round-trips cwd-independent argument paths.
|
||||
- FIXME(call-config-shape): revisit `LlmCallConfig`'s exact field set — which fields are genuinely epoch-level for cache purposes (`model` certainly; the sampling scalars sit there out of caution), and where provider-specific extras (reasoning options, extra body params) belong when an adapter needs them.
|
||||
+2
-2
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-10-single-file-executable-sdk-runtime-distribution.md: b177af24e988c6a314db522b8de0d1c09e30464f
|
||||
2026-07-10-single-file-executable-sdk-runtime-distribution.zh.md: 0b964e8a748e4adcc32c017957e5294a3f258365
|
||||
2026-07-10-single-file-executable-sdk-runtime-distribution.md: 80d5ab64682f1b72ca1dfa1f96bc34f2a3db5f2d
|
||||
2026-07-10-single-file-executable-sdk-runtime-distribution.zh.md: aea1fb66136b31e0a75ba51471fec4dadd960e8f
|
||||
+1
-1
@@ -21,7 +21,7 @@ The exe is packaged with the **`--sea` (enhanced SEA) mode** of [@yao-pkg/pkg](h
|
||||
|
||||
`--sea` requires target ≥ node22; the exe uniformly targets node24. One pkg invocation packages exactly one target; multi-platform builds invoke it once per platform.
|
||||
|
||||
Terminology reminder: pkg's `/snapshot` VFS has nothing to do with this repo's testing-system "snapshot" (ACP replay goldens, `$DSH_SNAPSHOT`); this document says "VFS" for the former.
|
||||
Terminology reminder: pkg's `/snapshot` VFS has nothing to do with this repo's testing-system "snapshot" (ACP replay expected outputs, `$DSH_SNAPSHOT`); this document says "VFS" for the former.
|
||||
|
||||
### The serving surface is a plugin: the two packages ui/jsonrpc + examples/jsonrpc-demo
|
||||
|
||||
|
||||
+1
-1
@@ -21,7 +21,7 @@ exe 使用 [@yao-pkg/pkg](https://github.com/yao-pkg/pkg)(vercel/pkg 归档后
|
||||
|
||||
`--sea` 要求构建目标 ≥ node22,exe 统一以 node24 为构建目标;每次 pkg 调用只打包一个构建目标,多平台各调用一次。
|
||||
|
||||
术语提醒:pkg 的 `/snapshot` VFS 与本仓库测试体系的“快照”(ACP 回放 golden、`$DSH_SNAPSHOT`)无关,本文用“VFS”指前者。
|
||||
术语提醒:pkg 的 `/snapshot` VFS 与本仓库测试体系的“快照”(ACP 回放预期输出、`$DSH_SNAPSHOT`)无关,本文用“VFS”指前者。
|
||||
|
||||
### 对外服务接口也是插件:ui/jsonrpc + examples/jsonrpc-demo 两包
|
||||
|
||||
|
||||
@@ -49,7 +49,7 @@ Four tiers, designed up front:
|
||||
- **Real-Loader path** — the plugin run through `Loader.unwrapExports`, asserting the namespace export shape survives (it HAS `inject`, so a stray default would crash at load — postmortem/0001).
|
||||
- **Full-loop integration** — a scripted mock model calls `todo_write` through the real agent loop; the `todo/write` event lands and a second call replaces it.
|
||||
- **`session/load` replay** — a persisted `todo/write` re-emits the `plan` update when a fresh ACP bridge loads the session.
|
||||
- **With-key e2e + snapshot** — a real prompt induces a `todo_write`; the snapshot golden gains the `plan` notification and the log event.
|
||||
- **With-key e2e + snapshot** — a real prompt induces a `todo_write`; the snapshot expected output gains the `plan` notification and the log event.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
|
||||
@@ -199,7 +199,7 @@ Coverage is named per tier; each behavior lives at the cheapest tier that can ex
|
||||
|
||||
- **Unit** (`tests/mcp-client.spec.ts`, `tests/apply.spec.ts`, mocked MCP SDK): the `publicToolName` algorithm (clean, normalize, truncate-and-hash, determinism, distinct-identity separation), raw-vs-public wire discipline, cross-server and native-tool coexistence, duplicate-`serverName` load failure and reservation release, invalid-tool-list rejection, generation swap/rollback, failed-re-sync retention, result mapping, cancellation, config schema validation. 100% per-file coverage gates the package.
|
||||
- **E2E** (`tests/mcp-client.e2e.ts`, keyless): the real MCP protocol against the in-repo fixture server, `@modelcontextprotocol/server-everything`, and `@modelcontextprotocol/server-filesystem` over stdio, and against an in-process `StreamableHTTPServerTransport` server over Streamable HTTP — discovery under the namespace, dotted-name normalization end to end, execution round-trips, duplicate-`serverName` rejection, disposal.
|
||||
- **Snapshot**: deliberately none. MCP tools introduce no new transcript surface — they register as raw `ToolDefinition`s and render through the ACP bridge's generic-card fallback, which the bridge's unit suite already pins (`packages/ui/acp/tests/stream-update.spec.ts`). Adding an MCP server to the snapshot example's `cordis.yml` would mutate the pinned `text-turn` system-prompt fixture (forcing a with-key re-record of every recorded golden) and make every replay depend on spawning an external MCP server process — for zero new rendering behavior. If a later change gives MCP tools their own render intent, that change names its snapshot coverage then.
|
||||
- **Snapshot**: deliberately none. MCP tools introduce no new transcript surface — they register as raw `ToolDefinition`s and render through the ACP bridge's generic-card fallback, which the bridge's unit suite already pins (`packages/ui/acp/tests/stream-update.spec.ts`). Adding an MCP server to the snapshot example's `cordis.yml` would mutate the pinned `text-turn` system-prompt fixture (forcing a with-key re-record of every recorded expected output) and make every replay depend on spawning an external MCP server process — for zero new rendering behavior. If a later change gives MCP tools their own render intent, that change names its snapshot coverage then.
|
||||
|
||||
## Consequences
|
||||
|
||||
|
||||
@@ -144,7 +144,7 @@ If the complete logical result fits under the inline cap, no formatted spill art
|
||||
- The first-party tool-owned spill precedent is covered directly: spill backend present, spill backend absent, `saveText()` failure, and missing spill owner.
|
||||
- The package has real Loader-path coverage for the namespace plugin export shape (`name`, `inject`, `Config`, and `apply`, with no default export).
|
||||
- A real-executor integration suite (`dsh-bash-local` + a real `rg`) verifies the world: hostile patterns stay inert, per-session cwd resolution, VCS-metadata exclusion, modification-time ordering, and real ripgrep stderr classification. It self-skips where `rg` is not on PATH (a CI accommodation mirroring the keyless e2e skip); the fake-executor suite alone carries the per-file 100% coverage gate.
|
||||
- Snapshot gap note for the transcript-visible spill notice: this landed with the gap note, not a snapshot. The snapshot tier replays the acp-agent tree, and adding the search plugin there changes the assembled system prompt — every golden would need re-recording with a real key, which the implementing environment did not hold. The spill notice's exact transcript text is pinned by unit tests (`formatGlobOutput`/`formatGrepOutput` and the through-the-registry spill tests); wiring the plugin into the acp-agent tree plus a `test:snapshot:record` pass is the follow-up for the next key-holding session.
|
||||
- Snapshot gap note for the transcript-visible spill notice: this landed with the gap note, not a snapshot. The snapshot tier replays the acp-agent tree, and adding the search plugin there changes the assembled system prompt — every expected output would need re-recording with a real key, which the implementing environment did not hold. The spill notice's exact transcript text is pinned by unit tests (`formatGlobOutput`/`formatGrepOutput` and the through-the-registry spill tests); wiring the plugin into the acp-agent tree plus a `test:snapshot:record` pass is the follow-up for the next key-holding session.
|
||||
|
||||
## Consequences
|
||||
|
||||
|
||||
+1
-1
@@ -24,7 +24,7 @@ If an LLM adapter browser or dynamic model-picker needs this signal later, reint
|
||||
|
||||
## Verification
|
||||
|
||||
`llm/adapter-change` and its emits are gone and the regenerated cordis catalog is fresh; HMR-safety holds (disposing a contributing fiber removes the adapter); `tools/change` and `system-prompt/change` remain documented and tested; and no production path changed observable behavior — the ACP snapshot goldens and the echo-agent smoke are byte-unchanged.
|
||||
`llm/adapter-change` and its emits are gone and the regenerated cordis catalog is fresh; HMR-safety holds (disposing a contributing fiber removes the adapter); `tools/change` and `system-prompt/change` remain documented and tested; and no production path changed observable behavior — the ACP snapshot expected outputs and the echo-agent smoke are byte-unchanged.
|
||||
|
||||
## Consequences
|
||||
|
||||
|
||||
+1
-1
@@ -26,7 +26,7 @@ This is the [drop-mutable-session-summary](../../implemented/simplification/2026
|
||||
|
||||
## Verification
|
||||
|
||||
`streamBlocks`, `generate`, `llm/generate`, and the assembler helpers they alone required are gone with no new dead exports; both real adapters are exercised through `stream()` and the shared assembler; the loop behaves identically (ACP snapshot goldens unchanged); and the README, architecture doc, and module docs carry no mention of the removed surfaces.
|
||||
`streamBlocks`, `generate`, `llm/generate`, and the assembler helpers they alone required are gone with no new dead exports; both real adapters are exercised through `stream()` and the shared assembler; the loop behaves identically (ACP snapshot expected outputs unchanged); and the README, architecture doc, and module docs carry no mention of the removed surfaces.
|
||||
|
||||
## Consequences
|
||||
|
||||
|
||||
@@ -23,7 +23,7 @@ A future permission/containment layer might want the pre-resolution path for err
|
||||
|
||||
## Verification
|
||||
|
||||
The removed surfaces are gone — `STREAM_MIN_SIZE`/`streamMinSize` in `dsh-fs-local`, `FsTarget.inputPath`, `FsEditOutcome.replacements`/`.replaceAll`, and `FileReadOutcome.limit`/`.version` — while the request-side `replaceAll` (`FsEditRequest`) and the version fields on the other outcome types are untouched; the test fakes shrank with the types. `formatEditOutput`'s emitted text is unchanged for both `replace_all` branches, so no snapshot golden churned.
|
||||
The removed surfaces are gone — `STREAM_MIN_SIZE`/`streamMinSize` in `dsh-fs-local`, `FsTarget.inputPath`, `FsEditOutcome.replacements`/`.replaceAll`, and `FileReadOutcome.limit`/`.version` — while the request-side `replaceAll` (`FsEditRequest`) and the version fields on the other outcome types are untouched; the test fakes shrank with the types. `formatEditOutput`'s emitted text is unchanged for both `replace_all` branches, so no snapshot expected output churned.
|
||||
|
||||
## Consequences
|
||||
|
||||
|
||||
@@ -8,7 +8,7 @@ Status: implemented
|
||||
|
||||
`agent/steering` duplicated the immediately preceding durable `steering/message` with the same payload. `agent/queued` remains the live-only signal because it fires before persistence and covers work that may be cancelled before entering the log.
|
||||
|
||||
Steering carries real production traffic — the hook bridges' turn-continuation decisions inject their reasons through `inbox.steer()`, landing as durable `steering/message` events that the hook-matrix goldens pin — and every one of those consumers observes the durable event. Nothing observed the mirror.
|
||||
Steering carries real production traffic — the hook bridges' turn-continuation decisions inject their reasons through `inbox.steer()`, landing as durable `steering/message` events that the hook-matrix expected outputs pin — and every one of those consumers observes the durable event. Nothing observed the mirror.
|
||||
|
||||
## Decision
|
||||
|
||||
|
||||
@@ -27,4 +27,4 @@ Unsupported vocabulary can return when a real consumer exists. `durationMs` rema
|
||||
|
||||
## Consequences
|
||||
|
||||
The `dialect`, `suppressOutput`, tunables, and semantics changes are invisible on the wire and in the goldens. The cost was churn in `dsh-hook-protocol` and both bridges — cheap under the pre-release stance, and cheaper than letting two copies of a durable event's semantics age apart.
|
||||
The `dialect`, `suppressOutput`, tunables, and semantics changes are invisible on the wire and in the expected outputs. The cost was churn in `dsh-hook-protocol` and both bridges — cheap under the pre-release stance, and cheaper than letting two copies of a durable event's semantics age apart.
|
||||
+1
-1
@@ -6,7 +6,7 @@ Status: implemented
|
||||
|
||||
Two pieces of `dsh-acp` surface were unreachable from any shipped configuration:
|
||||
|
||||
1. **`AcpConfig.agentName` / `agentVersion`** (`packages/ui/acp/src/index.ts`). The shipped app package hands the bridge only `{ model }` (`packages/examples/acp-demo/src/index.ts`), so no leaf `cordis.yml` — the only production config surface — could set the knobs at all; they were settable solely by direct-mounting the bridge, which only a unit test did. Every snapshot golden — the hook-matrix scenarios included — pins the schema defaults (`deepseek-harness-acp` / `0.0.1`). The pair also carried a live `TODO(double-default)`: the literals existed twice (schema `.default(...)` plus `??` fallbacks), with the TODO asking to pick one home.
|
||||
1. **`AcpConfig.agentName` / `agentVersion`** (`packages/ui/acp/src/index.ts`). The shipped app package hands the bridge only `{ model }` (`packages/examples/acp-demo/src/index.ts`), so no leaf `cordis.yml` — the only production config surface — could set the knobs at all; they were settable solely by direct-mounting the bridge, which only a unit test did. Every snapshot expected output — the hook-matrix scenarios included — pins the schema defaults (`deepseek-harness-acp` / `0.0.1`). The pair also carried a live `TODO(double-default)`: the literals existed twice (schema `.default(...)` plus `??` fallbacks), with the TODO asking to pick one home.
|
||||
2. **The `toolKindFor` name heuristic** (same file) special-cased `bash*`/`read*`/`write`/`edit*` tool names in the generic-fallback path. Since the [render-intent union](../architecture/2026-07-02-tool-render-intent-union.md), every first-party tool those arms matched ships its own `presentCall` carrying its kind, and the presenter-less production tools (`subagent`, `subagent_fork`) fell through to `other` anyway. The arms were production-reachable only when a tool declined to present its own call — a `presentCall` that THROWS (the containment fallback), or model arguments that fail the tool's schema so `defineTool`'s `presentCall` wrapper returns `undefined` (e.g. a `bash` call missing the required `description`) — and the bridge's own module doc states the design rule the heuristic violated: "the bridge never special-cases tool names".
|
||||
|
||||
## Decision
|
||||
|
||||
@@ -12,11 +12,11 @@ This RFC records the decision to add a third test tier — **snapshot tests**
|
||||
|
||||
## Decision
|
||||
|
||||
A snapshot test boots the real ACP example, drives its stdio protocol from a deterministic script, and compares normalized output with committed goldens. A session log recorded once from the real API supplies all later model streams. The fixture is the product's ordinary persisted JSONL.
|
||||
A snapshot test boots the real ACP example, drives its stdio protocol from a deterministic script, and compares normalized output with committed expected outputs. A session log recorded once from the real API supplies all later model streams. The fixture is the product's ordinary persisted JSONL.
|
||||
|
||||
### The fixture is the persisted session JSONL
|
||||
|
||||
Each scenario's `session.jsonl` is harvested from a real run. `assistant/chunk` events reproduce the model streams; tool, message, and boundary events capture the harness behavior. One ordinary session artifact therefore serves as both replay source and behavioral golden.
|
||||
Each scenario's `session.jsonl` is harvested from a real run. `assistant/chunk` events reproduce the model streams; tool, message, and boundary events capture the harness behavior. One ordinary session artifact therefore serves as both replay source and behavioral expected output.
|
||||
|
||||
### Replay derives the model script from the log
|
||||
|
||||
@@ -48,12 +48,12 @@ Replay uses a `cordis.snapshot.yml` overlay that replaces the real adapter with
|
||||
|
||||
A snapshot run asserts **two** normalized surfaces, because the harness's external surfaces are distinct:
|
||||
|
||||
1. The **stdout transcript** — the framed `session/update` JSON-RPC the editor sees. Catches regressions in the ACP bridge's event→update translation (`streamSessionEventUpdate`). Compared against a committed `stdout.golden.jsonl`.
|
||||
1. The **stdout transcript** — the framed `session/update` JSON-RPC the editor sees. Catches regressions in the ACP bridge's event→update translation (`streamSessionEventUpdate`). Compared against a committed `stdout.expected.jsonl`.
|
||||
2. The **re-persisted session JSONL**, normalized and compared with `session.jsonl`. The same fixture is both replay source and expected log. Prompt text is scrubbed; one scenario per header class pins readable prompt and tool content as described in the [header-pinning RFC](2026-07-06-pin-request-header-content-in-one-scenario.md). Override scenarios derive model behavior solely from their sidecar.
|
||||
|
||||
The surfaces are complementary: stdout covers bridge projection, while JSONL covers loop, tool, and boundary structure that the projection omits.
|
||||
|
||||
Normalization replaces session, cwd, protocol-id, timestamp, path, and process volatility while preserving deterministic sequence numbers. Scenarios constrain real bash use to stable commands. The stdout golden remains wire-shaped JSONL and every raw line must parse as JSON. Vitest updates only the stdout golden; normalized session equality never overwrites the replay fixture.
|
||||
Normalization replaces session, cwd, protocol-id, timestamp, path, and process volatility while preserving deterministic sequence numbers. Scenarios constrain real bash use to stable commands. The stdout expected output remains wire-shaped JSONL and every raw line must parse as JSON. Vitest updates only the stdout expected output; normalized session equality never overwrites the replay fixture.
|
||||
|
||||
### Isolation: normalization now, sandbox later
|
||||
|
||||
@@ -65,11 +65,11 @@ Tool determinism comes from a temporary cwd, scrubbed environment, fresh non-log
|
||||
|
||||
### Two subcommands, replay in the default gate
|
||||
|
||||
`pnpm run test:snapshot` replays committed fixtures keylessly; `test:snapshot:record` uses the real API and rewrites the harvested session log and stdout golden. Missing fixtures fail loud. Every scenario carries `input.json`, `stdout.golden.jsonl`, and `session.jsonl`; no-model cases use a header-only log. `replay.override.json` is required only for scenarios marked `overridden`, because its presence replaces derived replay. Fixture guards reject missing, mismatched, and orphaned files. Both commands accept scenario filters.
|
||||
`pnpm run test:snapshot` replays committed fixtures keylessly; `test:snapshot:record` uses the real API and rewrites the harvested session log and stdout expected output. Missing fixtures fail loud. Every scenario carries `input.json`, `stdout.expected.jsonl`, and `session.jsonl`; no-model cases use a header-only log. `replay.override.json` is required only for scenarios marked `overridden`, because its presence replaces derived replay. Fixture guards reject missing, mismatched, and orphaned files. Both commands accept scenario filters.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **A hand-authored `llm.json` of model chunks** — the earlier draft; reusing the real session log makes the fixture a genuine product of the system rather than a hand-built mock, and doubles it as a behavioral golden.
|
||||
- **A hand-authored `llm.json` of model chunks** — the earlier draft; reusing the real session log makes the fixture a genuine product of the system rather than a hand-built mock, and doubles it as a behavioral expected output.
|
||||
- **A byte-level HTTP-record library (Polly/nock/MSW)** — rejected: adapter-specific, awkward with streaming SSE, and lower-level than the thing under test.
|
||||
- **Synthesizing throw/cancel entries from `turn/end {kind:'error'|'aborted'}`** — rejected: it couples `llm-replay` to loop-internal turn-closing semantics, and the `turn/end` reason is lossy (it cannot distinguish a thrown 401 from a finish-error); the explicit `replay.override.json` sidecar is the cleaner seam.
|
||||
|
||||
|
||||
+35
@@ -0,0 +1,35 @@
|
||||
# RFC: Use `session.jsonl` as the only snapshot session-log artifact
|
||||
|
||||
Status: implemented
|
||||
|
||||
## Problem
|
||||
|
||||
Model-driving ACP snapshot scenarios ship both `session.jsonl` and `session.expected.jsonl`. For normal recorded scenarios, `session.jsonl` is the replay fixture harvested from a real run, and the replay test normalizes the newly persisted log and compares it to `session.expected.jsonl`. In the current fixtures, the two normalized logs are identical for ordinary recorded scenarios.
|
||||
|
||||
Authored override scenarios (`error-finish`, `cancel`) currently use `replay.override.json` to drive model behavior and keep `session.jsonl` as a minimal dummy fixture, while `session.expected.jsonl` holds the expected persisted log. The override file is a JSON array of `ReplayEntry` objects: `{ "kind": "chunks", "chunks": StreamChunk[] }`, `{ "kind": "throw", "chunks": StreamChunk[], "message": string, "code": string }`, or `{ "kind": "hang" }`. That split is also unnecessary: when an override sidecar exists, `llm-replay` replaces the derived script and does not need `session.jsonl` for model chunks, so `session.jsonl` can still be the expected session-log artifact for the scenario.
|
||||
|
||||
## Decision
|
||||
|
||||
The `session.expected.jsonl` concept is removed entirely. Every scenario has at most one committed session-log artifact, `session.jsonl`:
|
||||
|
||||
- For recorded scenarios, `session.jsonl` remains the raw harvested log. Replay still derives model chunks from it, and the snapshot test compares the replay run's normalized persisted log against normalized `session.jsonl`.
|
||||
- For authored override scenarios, `replay.override.json` drives model behavior and `session.jsonl` holds the expected produced session log. The replay adapter ignores the fixture for model chunks when the override exists, so the same file can be the expected log without affecting replay behavior.
|
||||
- For no-model scenarios, `session.jsonl` can stay as the minimal fixture needed to boot `llm-replay`; no session-log comparison is needed unless the scenario creates a persisted session.
|
||||
|
||||
Stdout expected outputs remain unchanged; they are the editor-facing projection and are not redundant with the session fixture.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**Normalizing both sides against a shared (replay-run) context** — rejected: `normalizeSessionLog` scrubs cwd by exact string match, so the fixture's recorded cwd would survive unscrubbed and every compare would fail. Each side normalizes against its own header-derived context — the implementation note below carries the mechanics.
|
||||
|
||||
## Verification
|
||||
|
||||
`session.expected.jsonl` appears nowhere in the snapshot harness, fixtures, orphan guards, or docs; the snapshot test derives the expected session log from `session.jsonl` for every model scenario; authored sidecar scenarios commit their expected produced log as `session.jsonl` with `replay.override.json` as the model-behavior override; and the orphan-fixture guards know which files each scenario kind requires. The [ACP snapshot tests RFC](../../implemented/testing/2026-06-19-acp-snapshot-tests.md) describes the reduced fixture set.
|
||||
|
||||
## Consequences
|
||||
|
||||
Reviewers lose one artifact name that made the expected persisted log visually separate from the replay fixture. The stdout expected output still protects the editor transcript, and comparing replay output to `session.jsonl` preserves the loop/persistence regression check without duplicating files.
|
||||
|
||||
## Implementation note
|
||||
|
||||
Each side is normalized against its own header values because recording and replay have different ids, paths, and timestamps. `fixtureContext()` derives the fixture context from its header, making already-normalized fixtures idempotent. Session logs use plain equality rather than file-snapshot updates, so comparison never rewrites fixtures.
|
||||
@@ -1,35 +0,0 @@
|
||||
# RFC: Use `session.jsonl` as the only snapshot session-log artifact
|
||||
|
||||
Status: implemented
|
||||
|
||||
## Problem
|
||||
|
||||
Model-driving ACP snapshot scenarios ship both `session.jsonl` and `session.golden.jsonl`. For normal recorded scenarios, `session.jsonl` is the replay fixture harvested from a real run, and the replay test normalizes the newly persisted log and compares it to `session.golden.jsonl`. In the current fixtures, the normalized recorded log and normalized golden are identical for the ordinary recorded scenarios.
|
||||
|
||||
Authored override scenarios (`error-finish`, `cancel`) currently use `replay.override.json` to drive model behavior and keep `session.jsonl` as a minimal dummy fixture, while `session.golden.jsonl` holds the expected persisted log. The override file is a JSON array of `ReplayEntry` objects: `{ "kind": "chunks", "chunks": StreamChunk[] }`, `{ "kind": "throw", "chunks": StreamChunk[], "message": string, "code": string }`, or `{ "kind": "hang" }`. That split is also unnecessary: when an override sidecar exists, `llm-replay` replaces the derived script and does not need `session.jsonl` for model chunks, so `session.jsonl` can still be the expected session-log artifact for the scenario.
|
||||
|
||||
## Decision
|
||||
|
||||
The `session.golden.jsonl` concept is removed entirely. Every scenario has at most one committed session-log artifact, `session.jsonl`:
|
||||
|
||||
- For recorded scenarios, `session.jsonl` remains the raw harvested log. Replay still derives model chunks from it, and the snapshot test compares the replay run's normalized persisted log against normalized `session.jsonl`.
|
||||
- For authored override scenarios, `replay.override.json` drives model behavior and `session.jsonl` holds the expected produced session log. The replay adapter ignores the fixture for model chunks when the override exists, so the same file can be the expected log without affecting replay behavior.
|
||||
- For no-model scenarios, `session.jsonl` can stay as the minimal fixture needed to boot `llm-replay`; no session-log comparison is needed unless the scenario creates a persisted session.
|
||||
|
||||
Stdout goldens remain unchanged; they are the editor-facing projection and are not redundant with the session fixture.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**Normalizing both sides against a shared (replay-run) context** — rejected: `normalizeSessionLog` scrubs cwd by exact string match, so the fixture's recorded cwd would survive unscrubbed and every compare would fail. Each side normalizes against its own header-derived context — the implementation note below carries the mechanics.
|
||||
|
||||
## Verification
|
||||
|
||||
`session.golden.jsonl` appears nowhere in the snapshot harness, fixtures, orphan guards, or docs; the snapshot test derives the expected session log from `session.jsonl` for every model scenario; authored sidecar scenarios commit their expected produced log as `session.jsonl` with `replay.override.json` as the model-behavior override; and the orphan-fixture guards know which files each scenario kind requires. The [ACP snapshot tests RFC](../../implemented/testing/2026-06-19-acp-snapshot-tests.md) describes the reduced fixture set.
|
||||
|
||||
## Consequences
|
||||
|
||||
Reviewers lose one artifact name that made the expected persisted log visually separate from the replay fixture. The stdout golden still protects the editor transcript, and comparing replay output to `session.jsonl` preserves the loop/persistence regression check without duplicating files.
|
||||
|
||||
## Implementation note
|
||||
|
||||
Each side is normalized against its own header values because recording and replay have different ids, paths, and timestamps. `fixtureContext()` derives the fixture context from its header, making already-normalized fixtures idempotent. Session logs use plain equality rather than file-snapshot updates, so comparison never rewrites fixtures.
|
||||
@@ -4,7 +4,7 @@ Status: implemented
|
||||
|
||||
## Problem
|
||||
|
||||
The snapshot tier (`pnpm run test:snapshot`) boots the real `acp-agent` subprocess, replays a recorded session through [`dsh-llm-replay`](../../../../packages/support/llm-replay), and diffs the normalized stdout transcript + re-persisted session log against committed goldens. It is the only tier that exercises the full editor-facing transcript end to end.
|
||||
The snapshot tier (`pnpm run test:snapshot`) boots the real `acp-agent` subprocess, replays a recorded session through [`dsh-llm-replay`](../../../../packages/support/llm-replay), and diffs the normalized stdout transcript + re-persisted session log against committed expected outputs. It is the only tier that exercises the full editor-facing transcript end to end.
|
||||
|
||||
It was built for ONE session per process, and that assumption is wired into two places:
|
||||
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
# RFC: Hook snapshot matrix — end-to-end goldens for both bridges
|
||||
# RFC: Hook snapshot matrix — end-to-end expected outputs for both bridges
|
||||
|
||||
Status: implemented
|
||||
|
||||
## Problem
|
||||
|
||||
The hook bridges — [`dsh-hooks-claude`](../../../../packages/hooks/hooks-claude) (7 Claude Code hook points) and [`dsh-hooks-codex`](../../../../packages/hooks/hooks-codex) (5 Codex points) — map external hook commands onto the harness interception seams. They carry deep unit and coverage-spec coverage (every decision arm, every payload dialect, driven against a mocked seam) plus one key-gated e2e (`hooks.e2e.ts`, a live `PreToolUse` block). But the full-transcript snapshot tier — the one net that boots the real `acp-agent` subprocess, replays a recorded session keyless, and diffs the normalized ACP stdout + re-persisted log against committed goldens — covered exactly ONE hook: a Claude `UserPromptSubmit` block (`hook-cc-promptsubmit-block`).
|
||||
The hook bridges — [`dsh-hooks-claude`](../../../../packages/hooks/hooks-claude) (7 Claude Code hook points) and [`dsh-hooks-codex`](../../../../packages/hooks/hooks-codex) (5 Codex points) — map external hook commands onto the harness interception seams. They carry deep unit and coverage-spec coverage (every decision arm, every payload dialect, driven against a mocked seam) plus one key-gated e2e (`hooks.e2e.ts`, a live `PreToolUse` block). But the full-transcript snapshot tier — the one net that boots the real `acp-agent` subprocess, replays a recorded session keyless, and diffs the normalized ACP stdout + re-persisted log against committed expected outputs — covered exactly ONE hook: a Claude `UserPromptSubmit` block (`hook-cc-promptsubmit-block`).
|
||||
|
||||
That is the tier a mocked unit test structurally cannot be: it exercises the REAL bridge translating a REAL hook process's outcome into the REAL seam decision, then the REAL loop's reaction, rendered exactly as an editor sees it. A bridge-translation or loop-structure regression that left every unit green would still escape it for every hook point but one — and for the Codex bridge, the ACP example did not even LOAD it, so no Codex hook could fire end-to-end at all.
|
||||
|
||||
@@ -29,22 +29,22 @@ Thirteen scenarios under `examples/acp-agent/tests/snapshots/`, naming `hook-<di
|
||||
|
||||
Each hook command emits only FIXED LITERAL strings (no timestamps/pids/`$RANDOM`/cwd echoes); the snapshot normalizer scrubs the one volatile field a `hook/result` carries (`durationMs`). The `Stop` scenarios self-limit with a marker file (`.stop_fired`) so the force-continue does not loop — the `stop_hook_active` loop-guard is still a bridge `TODO`, so an unconditional Stop hook would force-continue every step.
|
||||
|
||||
The `PostToolUse` block scenarios self-limit at the mechanism they prove. The Claude hook persists a workspace marker after its first rejection, so one recovery call is allowed; the Codex prompt makes one call and reports the injected result. Each golden pins one blocked call without repeated block/retry cycles.
|
||||
The `PostToolUse` block scenarios self-limit at the mechanism they prove. The Claude hook persists a workspace marker after its first rejection, so one recovery call is allowed; the Codex prompt makes one call and reports the injected result. Each expected output pins one blocked call without repeated block/retry cycles.
|
||||
|
||||
### Three hook points are deliberately NOT snapshotted
|
||||
|
||||
Discovered while building the matrix, and documented here because the omission is a decision, not an oversight:
|
||||
|
||||
- **`SessionStart` and `SubagentStart`** inject context through a detached, best-effort `void runPoint(...).then(agent.inject())` with NO turn binding. The resulting `context/message` races the work it precedes (the first model request / the child's first turn) and lands at a nondeterministic log position. A recorded golden does not even reproduce on its own replay — a 10× replay stability check failed 10/10 for both. They stay on the bridges' unit coverage, which drives the seam directly without the timing race. (If the injection is ever made turn-bound and deterministic — the direction the `TODO(session-start-gating)` points — these become snapshottable.)
|
||||
- **`SubagentStop`** is observe-only: its `subagent/end` handler passes no turn (so no `hook/*` log events) and does no injection. It writes NOTHING to the transcript, so a golden would be byte-identical to the no-hook run and could never be proven to fail — a guard that cannot bite. It stays on unit coverage (`bridge.spec.ts` already asserts the observe-only call).
|
||||
- **`SessionStart` and `SubagentStart`** inject context through a detached, best-effort `void runPoint(...).then(agent.inject())` with NO turn binding. The resulting `context/message` races the work it precedes (the first model request / the child's first turn) and lands at a nondeterministic log position. A recorded expected output does not even reproduce on its own replay — a 10× replay stability check failed 10/10 for both. They stay on the bridges' unit coverage, which drives the seam directly without the timing race. (If the injection is ever made turn-bound and deterministic — the direction the `TODO(session-start-gating)` points — these become snapshottable.)
|
||||
- **`SubagentStop`** is observe-only: its `subagent/end` handler passes no turn (so no `hook/*` log events) and does no injection. It writes NOTHING to the transcript, so an expected output would be byte-identical to the no-hook run and could never be proven to fail — a guard that cannot bite. It stays on unit coverage (`bridge.spec.ts` already asserts the observe-only call).
|
||||
|
||||
The matrix therefore covers every hook point that has a DETERMINISTIC, OBSERVABLE transcript footprint, for both dialects.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Every bridge seam mapping with an observable transcript is now guarded at the full-transcript tier, in the real app, for both dialects — including the Codex bridge, which had no end-to-end coverage at all. Recorded goldens capture the model's real reaction to a denied/blocked/force-continued turn, which a hand-authored transcript could only guess at.
|
||||
- Every bridge seam mapping with an observable transcript is now guarded at the full-transcript tier, in the real app, for both dialects — including the Codex bridge, which had no end-to-end coverage at all. Recorded expected outputs capture the model's real reaction to a denied/blocked/force-continued turn, which a hand-authored transcript could only guess at.
|
||||
- The `UserPromptSubmit` block scenarios are authored keylessly (no model turn); the rest replay keylessly from recorded fixtures. `pnpm run test:snapshot:record` regenerates the recorded fixtures from the live API and self-skips without a key like every recorded scenario.
|
||||
- The prove-red discipline holds: tampering a hook config's output (e.g. changing a deny reason) turns its scenario red on replay — the hook process runs FOR REAL during replay (only the model is replayed), so the golden guards the actual hook→seam→loop path, not a mock of it.
|
||||
- The prove-red discipline holds: tampering a hook config's output (e.g. changing a deny reason) turns its scenario red on replay — the hook process runs FOR REAL during replay (only the model is replayed), so the expected output guards the actual hook→seam→loop path, not a mock of it.
|
||||
- The `acp-agent` demo now loads a Codex bridge it will usually no-op (no `codex-hooks.json` in a typical project), which is the intended fail-soft behavior, not a cost.
|
||||
|
||||
<!-- rfc-format: alternatives-not-recorded (pre-format RFC) -->
|
||||
@@ -10,7 +10,7 @@ Status: implemented
|
||||
|
||||
`cordis.snapshot.yml` includes the live config, disables the named DeepSeek adapter by id and name, and inserts the replay adapter. Every other entry therefore comes from the shipping tree. Replay selects the overlay; recording still boots `cordis.yml`, and the load guard permits the intentionally disabled entry.
|
||||
|
||||
One vendored-plugin fact the overlay depends on, deliberately: the include applies `patches` when it loads the file — its `refresh()`/`internal/update` paths re-read without re-patching — which is exactly enough for a one-shot replay boot (the replay app loads no `hmr` and nothing rewrites the config mid-run). The snapshot suite is the proof: all scenarios pass unchanged on the overlay, byte-identical goldens included.
|
||||
One vendored-plugin fact the overlay depends on, deliberately: the include applies `patches` when it loads the file — its `refresh()`/`internal/update` paths re-read without re-patching — which is exactly enough for a one-shot replay boot (the replay app loads no `hmr` and nothing rewrites the config mid-run). The snapshot suite is the proof: all scenarios pass unchanged on the overlay, byte-identical expected outputs included.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
|
||||
+1
-1
@@ -8,7 +8,7 @@ An ACP snapshot suite needs to prove the exact composed system prompt and tool-s
|
||||
|
||||
## Decision
|
||||
|
||||
Exactly one scenario per header-composition class is flagged `pinsHeader`. Its directory splits the pin by review format: `system-prompt.golden.md` contains the normalized full prompt sequence as ordinary Markdown, `tool-schemas.golden.json` contains the corresponding complete schema sequence as structured JSON, and `session.jsonl` retains config, reason, and any model-visible prefix while storing `header.system` and `header.tools` as `"{{system}}"` / `"{{tools}}"`. Every other JSONL uses the same prompt and tool tokens and also tokenizes session-prefix content. The pin mechanics live in [`dsh-acp-snapshot`](../../../../packages/support/acp-snapshot/README.md), whose suite factory enforces one pin per class.
|
||||
Exactly one scenario per header-composition class is flagged `pinsHeader`. Its directory splits the pin by review format: `system-prompt.expected.md` contains the normalized full prompt sequence as ordinary Markdown, `tool-schemas.expected.json` contains the corresponding complete schema sequence as structured JSON, and `session.jsonl` retains config, reason, and any model-visible prefix while storing `header.system` and `header.tools` as `"{{system}}"` / `"{{tools}}"`. Every other JSONL uses the same prompt and tool tokens and also tokenizes session-prefix content. The pin mechanics live in [`dsh-acp-snapshot`](../../../../packages/support/acp-snapshot/README.md), whose suite factory enforces one pin per class.
|
||||
|
||||
The pure `scrubSystemPrompts` and `scrubToolSchemas` normalizers independently tokenize every stored full header. `scrubRequestHeaders` also tokenizes session-prefix content for non-pinning scenarios while retaining header count, field presence, config, reason, and prefix message count. Record and refresh write-back apply the appropriate scrub before writing JSONL and regenerate both sidecars from the normalized live full-header sequence, so neither path can reintroduce prompt/schema bulk into JSONL or leave a review artifact stale.
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@ Status: implemented
|
||||
|
||||
## Problem
|
||||
|
||||
The ACP snapshot tier ([snapshot RFC](2026-06-19-acp-snapshot-tests.md)) was built from three modules living inside one example's test directory: `snapshot-harness.ts` (boot the real bin subprocess, drive it over ACP JSON-RPC, harvest the persisted logs), `snapshot-normalize.ts` (the pure golden normalizers), and the ~150-line scenario body plus fixture guards in `acp.snapshot.ts` (record/replay modes, the stdout-golden and log compares, the pinned-header uniformity guard, the orphan/required-file/single-pin meta-tests).
|
||||
The ACP snapshot tier ([snapshot RFC](2026-06-19-acp-snapshot-tests.md)) was built from three modules living inside one example's test directory: `snapshot-harness.ts` (boot the real bin subprocess, drive it over ACP JSON-RPC, harvest the persisted logs), `snapshot-normalize.ts` (the pure expected-output normalizers), and the ~150-line scenario body plus fixture guards in `acp.snapshot.ts` (record/replay modes, the stdout expected-output and log comparisons, the pinned-header uniformity guard, the orphan/required-file/single-pin meta-tests).
|
||||
|
||||
A second ACP example wanting snapshot coverage — the sandbox/approval composition is the immediate consumer — could only copy those modules, forking exactly the logic that must not drift: record write-back, header scrubbing, child-session harvest ordering. The spawn/client glue was also triplicated across `acp.e2e.ts`, `hooks.e2e.ts`, and the harness. Location decided test rigor: the per-file 100% coverage gate measures `packages/*/*/src` only, so none of this machinery was measured — the same gap that had moved `dsh-llm-replay` out of `examples/` into [packages/support](../../../../packages/support/README.md). And the harness's ACP client hardcoded `requestPermission → cancelled`, so an approval round-trip — the headline behavior of the sandbox composition — could not be expressed at the snapshot tier at all.
|
||||
|
||||
@@ -18,7 +18,7 @@ The machinery lives in [`packages/support/acp-snapshot`](../../../../packages/su
|
||||
|
||||
**`src/normalize.ts`** — the pure normalizers, hook-free by policy: when a future event carries a new volatile field (an approval duration, say), the shared normalizer learns it in the same change, keeping one home for what "normalized" means rather than per-suite scrub extensions.
|
||||
|
||||
**`src/suite.ts`** — the `Scenario` type and `defineAcpSnapshotSuite(options)`, registering the per-scenario compares, record/refresh fixture write-back, the header pin with its live uniformity guard, and the fixture guard block (no orphan scenario dirs, required files present, exactly one pin per class, every JSONL a `scrubSystemPrompts` fixed point, non-pinning fixtures also `scrubRequestHeaders` fixed points). A scenario directory's `session.jsonl` plus contiguous `session.<n>.jsonl` siblings are its ordered primary/child inventory, so the scenario table declares policy without duplicating a child count. The pinned-header contract ([pinned-header RFC](2026-07-06-pin-request-header-content-in-one-scenario.md)) is per-suite: each header class flags exactly one `pinsHeader` scenario, whose `system-prompt.golden.md` and JSONL tool list split the composed header into reviewable artifacts; the uniformity guard compares both against every live header in that class. A pinning scenario declares any legitimate changed-header count, and its Markdown artifact records every full changed prompt. The pure helpers (`sessionFixtureNames`, `fixtureContext`, `normalizedHeaders`, `normalizedSystemPrompts`, `formatSystemPromptSnapshot`, `headerChangeCount`) are exported from the module for direct unit coverage.
|
||||
**`src/suite.ts`** — the `Scenario` type and `defineAcpSnapshotSuite(options)`, registering the per-scenario compares, record/refresh fixture write-back, the header pin with its live uniformity guard, and the fixture guard block (no orphan scenario dirs, required files present, exactly one pin per class, every JSONL a `scrubSystemPrompts` fixed point, non-pinning fixtures also `scrubRequestHeaders` fixed points). A scenario directory's `session.jsonl` plus contiguous `session.<n>.jsonl` siblings are its ordered primary/child inventory, so the scenario table declares policy without duplicating a child count. The pinned-header contract ([pinned-header RFC](2026-07-06-pin-request-header-content-in-one-scenario.md)) is per-suite: each header class flags exactly one `pinsHeader` scenario, whose `system-prompt.expected.md` and JSONL tool list split the composed header into reviewable artifacts; the uniformity guard compares both against every live header in that class. A pinning scenario declares any legitimate changed-header count, and its Markdown artifact records every full changed prompt. The pure helpers (`sessionFixtureNames`, `fixtureContext`, `normalizedHeaders`, `normalizedSystemPrompts`, `formatSystemPromptSnapshot`, `headerChangeCount`) are exported from the module for direct unit coverage.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
@@ -26,7 +26,7 @@ The machinery lives in [`packages/support/acp-snapshot`](../../../../packages/su
|
||||
- **A shared module directory under `examples/`** — keeps the code outside the coverage gate and forces relative imports across example boundaries, against the package-name import convention; `examples/` leaves stay thin by design.
|
||||
- **A `/testing` subpath export of `dsh-acp-demo`** — couples test infrastructure into a product package's surface and dependency set; `packages/support/` exists precisely for real-but-lower-compatibility dev/test packages, with `dsh-llm-replay` as the precedent this package completes.
|
||||
- **Export raw test-body functions instead of a suite factory** — each example would re-own the `describe`/`it` skeleton (~80 lines of registration boilerplate per suite) for no flexibility gain; the factory keeps consumers to a scenario table plus one call, and the exported pure helpers preserve unit-testability inside the factory design.
|
||||
- **An injectable ACP `Client` factory instead of declarative `permissionAnswers`** — maximally flexible, but it leaks SDK client construction to every consumer and reopens per-example drift in exactly the layer being unified; a declarative queue keeps `input.json` the single scripting surface and stays golden-normalizable.
|
||||
- **An injectable ACP `Client` factory instead of declarative `permissionAnswers`** — maximally flexible, but it leaks SDK client construction to every consumer and reopens per-example drift in exactly the layer being unified; a declarative queue keeps `input.json` the single scripting surface and compatible with expected-output normalization.
|
||||
- **Generalize beyond ACP (a transport-agnostic snapshot harness)** — no second transport exists; the harness is ACP-shaped end to end (SDK client, JSON-RPC frames, `session/update` waiters), and a speculative abstraction would be a seam split ahead of any consumer.
|
||||
|
||||
## Testing
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-18-tui-terminal-state-snapshots.md: a1363a521372c11dab8b239cad87df5ff5ca8f22
|
||||
2026-07-18-tui-terminal-state-snapshots.zh.md: aa365ba1f2ba17e5bc5409dbdc3736cbc29fe26a
|
||||
2026-07-18-tui-terminal-state-snapshots.md: a609e7c167ddf7ebe594f6793e34a48c555c10a9
|
||||
2026-07-18-tui-terminal-state-snapshots.zh.md: 8690a6f83a827619682b82c2df56d360e104c53b
|
||||
@@ -25,19 +25,19 @@ The runnable TUI has its own `examples/tui-agent` leaf beside the readline `repl
|
||||
|
||||
### Recorded-session replay
|
||||
|
||||
Each example-level scenario directory owns `session.jsonl`, optional child logs `session.<n>.jsonl`, and `terminal.golden.txt`. The primary log supplies user-authored `user/message` prompts and the recorded `assistant/chunk` sequence. `dsh-llm-replay` derives one model-call script per session, binds child logs to fresh child sessions, and is the only mocked boundary. The agent loop, bash and filesystem implementations, Code Mode worker, subagent provider, workflow worker, Cordis tools, presenters, and TUI are production implementations.
|
||||
Each example-level scenario directory owns `session.jsonl`, optional child logs `session.<n>.jsonl`, and `terminal.expected.txt`. The primary log supplies user-authored `user/message` prompts and the recorded `assistant/chunk` sequence. `dsh-llm-replay` derives one model-call script per session, binds child logs to fresh child sessions, and is the only mocked boundary. The agent loop, bash and filesystem implementations, Code Mode worker, subagent provider, workflow worker, Cordis tools, presenters, and TUI are production implementations.
|
||||
|
||||
The suite rejects a journey when its tool-call sequence differs, an expected event count is missing, a tool result is an error, a turn ends in error, a workflow lifecycle is incomplete, or the live child-session count differs from the fixture set. These assertions prevent an attractive terminal golden from hiding a failed or bypassed production path.
|
||||
The suite rejects a journey when its tool-call sequence differs, an expected event count is missing, a tool result is an error, a turn ends in error, a workflow lifecycle is incomplete, or the live child-session count differs from the fixture set. These assertions prevent an attractive terminal expected output from hiding a failed or bypassed production path.
|
||||
|
||||
The live-model fixtures use `DSH_SNAPSHOT=record`; record mode rewrites their primary and child JSONL logs and terminal goldens. The deterministic Cordis toolchain keeps an authored complete JSONL script because reliably coercing a live model through five exact tool boundaries and two children is not a stable recording contract. `DSH_SNAPSHOT=refresh` replays every committed script keylessly and rewrites only derived terminal goldens. Plain replay compares without writing, and unknown mode values fail loud.
|
||||
The live-model fixtures use `DSH_SNAPSHOT=record`; record mode rewrites their primary and child JSONL logs and terminal expected outputs. The deterministic Cordis toolchain keeps an authored complete JSONL script because reliably coercing a live model through five exact tool boundaries and two children is not a stable recording contract. `DSH_SNAPSHOT=refresh` replays every committed script keylessly and rewrites only derived terminal expected outputs. Plain replay compares without writing, and unknown mode values fail loud.
|
||||
|
||||
### Semantic terminal projection
|
||||
|
||||
The package-local `HeadlessTerminal` implements the same pi-tui `Terminal` interface as the process terminal and feeds every ANSI write into the pinned `@xterm/headless` parser. Snapshot code waits for synchronized frames to quiesce before reading state, so a checkpoint represents a completed screen rather than a timer-dependent write prefix.
|
||||
|
||||
Each golden projects dimensions, active-buffer and viewport coordinates, lifecycle and cursor state, rows, wrap markers, and non-default style ranges into text. Scroll-heavy cards capture the used buffer; overlays capture the visible viewport. Text and style remain separate so a reviewer can distinguish content changes from presentation changes without decoding ANSI bytes.
|
||||
Each expected output projects dimensions, active-buffer and viewport coordinates, lifecycle and cursor state, rows, wrap markers, and non-default style ranges into text. Scroll-heavy cards capture the used buffer; overlays capture the visible viewport. Text and style remain separate so a reviewer can distinguish content changes from presentation changes without decoding ANSI bytes.
|
||||
|
||||
Every checkpoint enforces theme independence across the complete terminal state: no RGB colors, no palette entries beyond ANSI 0–15, and no explicit background colors. Reverse video remains valid for selection because it uses terminal defaults. Both suites own closed inventories that reject missing scenarios, missing checkpoints, and orphaned golden files.
|
||||
Every checkpoint enforces theme independence across the complete terminal state: no RGB colors, no palette entries beyond ANSI 0–15, and no explicit background colors. Reverse video remains valid for selection because it uses terminal defaults. Both suites own closed inventories that reject missing scenarios, missing checkpoints, and orphaned expected output files.
|
||||
|
||||
### Required scenario matrix
|
||||
|
||||
@@ -58,7 +58,7 @@ Every checkpoint enforces theme independence across the complete terminal state:
|
||||
- **Snapshot raw terminal writes** — rejected because differential rendering may change write boundaries without changing the screen, while cursor and clear sequences are unreadable in review.
|
||||
- **Snapshot component render lines before terminal output** — rejected because it does not test ANSI parsing, cursor movement, overlays, viewport behavior, or independent components in one frame.
|
||||
- **Build every completed flow by appending session events** — rejected because a hand-authored event sequence can drift from the agent loop, tool execution, child-session binding, or worker behavior while its presentation test stays green. Direct event construction remains limited to transient renderer states.
|
||||
- **Reuse ACP stdout goldens as the TUI oracle** — rejected because a recorded model journey is transport-neutral but its presentation is not. TUI scenarios own terminal goldens while using the same JSONL replay vocabulary.
|
||||
- **Reuse ACP stdout expected outputs as the TUI oracle** — rejected because a recorded model journey is transport-neutral but its presentation is not. TUI scenarios own terminal expected outputs while using the same JSONL replay vocabulary.
|
||||
- **Commit raster screenshots** — rejected because fonts, glyph metrics, antialiasing, and host terminal themes make them platform-sensitive and make semantic style changes difficult to review.
|
||||
- **Use only PTY end-to-end tests** — rejected because raw PTY output is a stream of historical drawing operations, not queryable final state. PTY tests retain the real Loader/input/teardown boundary, while the emulator owns broad state coverage.
|
||||
|
||||
@@ -67,4 +67,4 @@ Every checkpoint enforces theme independence across the complete terminal state:
|
||||
- Completed advanced snapshots now fail when the real Code Mode, workflow, subagent, filesystem, bash, or Cordis path breaks, rather than accepting a fabricated result event.
|
||||
- TUI visual regressions produce readable cell-and-style diffs, while JSONL fixtures retain the exact model chunks that made the production path execute.
|
||||
- The emulator uses xterm's proposed buffer API. An xterm upgrade requires rerunning and reviewing the semantic projection; terminal-specific behavior still needs the PTY smoke.
|
||||
- Goldens deliberately encode wrapping and viewport behavior at fixed sizes. Intentional layout changes use keyless refresh, while model-journey changes use record mode and review both JSONL and terminal diffs.
|
||||
- Expected outputs deliberately encode wrapping and viewport behavior at fixed sizes. Intentional layout changes use keyless refresh, while model-journey changes use record mode and review both JSONL and terminal diffs.
|
||||
@@ -25,19 +25,19 @@ TUI 覆盖分为四个互补层次:
|
||||
|
||||
### 已录制会话回放
|
||||
|
||||
每个示例级场景目录都包含 `session.jsonl`、可选的子会话日志 `session.<n>.jsonl`,以及 `terminal.golden.txt`。主日志提供用户来源的 `user/message` 提示词和已录制的 `assistant/chunk` 序列。`dsh-llm-replay` 为每个会话派生一份模型调用脚本,并将子日志绑定到新建的子会话;这是测试中唯一的 mock 边界。agent loop、bash 与文件系统实现、Code Mode worker、subagent 提供方、工作流 worker、Cordis 工具、呈现器和 TUI 都使用生产实现。
|
||||
每个示例级场景目录都包含 `session.jsonl`、可选的子会话日志 `session.<n>.jsonl`,以及 `terminal.expected.txt`。主日志提供用户来源的 `user/message` 提示词和已录制的 `assistant/chunk` 序列。`dsh-llm-replay` 为每个会话派生一份模型调用脚本,并将子日志绑定到新建的子会话;这是测试中唯一的 mock 边界。agent loop、bash 与文件系统实现、Code Mode worker、subagent 提供方、工作流 worker、Cordis 工具、呈现器和 TUI 都使用生产实现。
|
||||
|
||||
如果工具调用顺序不符、预期事件数量不足、工具结果报错、轮次以错误结束、工作流生命周期不完整,或者实时子会话数量与 fixture(测试前置数据)集合不一致,测试都会失败。即使终端金标表面正确,这些断言也能阻止失败或被绕过的生产路径混入结果。
|
||||
如果工具调用顺序不符、预期事件数量不足、工具结果报错、轮次以错误结束、工作流生命周期不完整,或者实时子会话数量与 fixture(测试前置数据)集合不一致,测试都会失败。即使终端预期输出表面正确,这些断言也能阻止失败或被绕过的生产路径混入结果。
|
||||
|
||||
真实模型 fixture 通过 `DSH_SNAPSHOT=record` 更新;录制模式会重写其主会话与子会话 JSONL 日志以及终端金标。确定性的 Cordis 工具链保留一份人工编写的完整 JSONL 脚本,因为要求真实模型稳定经过五个指定工具边界和两个子会话并不是可靠的录制契约。`DSH_SNAPSHOT=refresh` 会无密钥回放所有已提交脚本,并且只重写派生的终端金标。普通回放只比较而不写入,未知模式值会快速失败。
|
||||
真实模型 fixture 通过 `DSH_SNAPSHOT=record` 更新;录制模式会重写其主会话与子会话 JSONL 日志以及终端预期输出。确定性的 Cordis 工具链保留一份人工编写的完整 JSONL 脚本,因为要求真实模型稳定经过五个指定工具边界和两个子会话并不是可靠的录制契约。`DSH_SNAPSHOT=refresh` 会无密钥回放所有已提交脚本,并且只重写派生的终端预期输出。普通回放只比较而不写入,未知模式值会快速失败。
|
||||
|
||||
### 语义终端投影
|
||||
|
||||
包内的 `HeadlessTerminal` 实现与进程终端相同的 pi-tui `Terminal` 接口,并把每次 ANSI 写入交给固定版本的 `@xterm/headless` 解析器。读取状态前,快照代码会等待同步帧稳定,因此每个检查点表示已经完成的画面,而不是依赖计时的写入前缀。
|
||||
|
||||
每份金标把终端尺寸、活动缓冲区和视口坐标、生命周期与光标状态、各行、换行标记以及非默认样式区间投影为文本。滚动内容较多的卡片捕获已使用缓冲区;浮层捕获可见视口。文本和样式相互分离,评审人无需解码 ANSI 字节即可区分内容变化与呈现变化。
|
||||
每份预期输出把终端尺寸、活动缓冲区和视口坐标、生命周期与光标状态、各行、换行标记以及非默认样式区间投影为文本。滚动内容较多的卡片捕获已使用缓冲区;浮层捕获可见视口。文本和样式相互分离,评审人无需解码 ANSI 字节即可区分内容变化与呈现变化。
|
||||
|
||||
每个检查点还会对完整终端状态强制执行主题无关性:禁止 RGB 颜色、禁止 ANSI 0–15 以外的调色板项,也禁止显式背景色。选择行使用终端默认色进行反显,因此仍然有效。两套测试都拥有封闭清单,会拒绝缺失的场景、缺失的检查点和遗留金标文件。
|
||||
每个检查点还会对完整终端状态强制执行主题无关性:禁止 RGB 颜色、禁止 ANSI 0–15 以外的调色板项,也禁止显式背景色。选择行使用终端默认色进行反显,因此仍然有效。两套测试都拥有封闭清单,会拒绝缺失的场景、缺失的检查点和遗留预期输出文件。
|
||||
|
||||
### 必需场景矩阵
|
||||
|
||||
@@ -58,7 +58,7 @@ TUI 覆盖分为四个互补层次:
|
||||
- **快照原始终端写入**:不予采纳,因为差分渲染可能在画面不变时改变写入边界,而且光标与清屏序列难以评审。
|
||||
- **快照进入终端输出之前的组件渲染行**:不予采纳,因为它无法测试 ANSI 解析、光标移动、浮层、视口行为,也无法测试独立组件在同一帧中的相互作用。
|
||||
- **通过追加会话事件构造所有完整流程**:不予采纳,因为人工编写的事件序列可能与 agent loop、工具执行、子会话绑定或 worker 行为发生偏差,但呈现测试仍然保持绿色。直接构造事件只用于渲染器瞬态。
|
||||
- **复用 ACP stdout 金标作为 TUI 判定依据**:不予采纳,因为已录制模型流程与传输方式无关,其呈现方式却并非如此。TUI 场景使用同一套 JSONL 回放词汇,但拥有独立的终端金标。
|
||||
- **复用 ACP stdout 预期输出作为 TUI 判定依据**:不予采纳,因为已录制模型流程与传输方式无关,其呈现方式却并非如此。TUI 场景使用同一套 JSONL 回放词汇,但拥有独立的终端预期输出。
|
||||
- **提交栅格截图**:不予采纳,因为字体、字形度量、抗锯齿和宿主终端主题会使结果依赖平台,也会增加语义样式变更的评审难度。
|
||||
- **只使用 PTY 端到端测试**:不予采纳,因为原始 PTY 输出是一系列历史绘制操作,而不是可查询的最终状态。PTY 测试保留真实 Loader、输入与清理边界,模拟器负责广泛的状态覆盖。
|
||||
|
||||
@@ -67,4 +67,4 @@ TUI 覆盖分为四个互补层次:
|
||||
- 当真实 Code Mode、工作流、subagent、文件系统、bash 或 Cordis 路径损坏时,已完成高级快照会失败,不会继续接受伪造的结果事件。
|
||||
- TUI 视觉回归会产生便于阅读的单元格和样式 diff,而 JSONL fixture 会保留触发生产路径的确切模型分片。
|
||||
- 模拟器使用 xterm 的拟议缓冲区 API。升级 xterm 时必须重新运行并评审语义投影;终端特有行为仍需由 PTY 冒烟测试覆盖。
|
||||
- 金标有意固定指定尺寸下的换行与视口行为。预期布局变更使用无密钥刷新;模型流程变更使用录制模式,并同时评审 JSONL 与终端 diff。
|
||||
- 预期输出有意固定指定尺寸下的换行与视口行为。预期布局变更使用无密钥刷新;模型流程变更使用录制模式,并同时评审 JSONL 与终端 diff。
|
||||
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-17-sdk-follow-up-capabilities.md: dac859f95e4ee4c628c50be72b3a18720f2b19fb
|
||||
2026-07-17-sdk-follow-up-capabilities.zh.md: 55648a368b4f9aa865129f3e0501e417d15f09ed
|
||||
@@ -0,0 +1,118 @@
|
||||
# RFC: SDK follow-up capabilities
|
||||
|
||||
Status: proposed
|
||||
|
||||
English | [中文](2026-07-17-sdk-follow-up-capabilities.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
The first SDK release creates and edits developer-owned Cordis projects through the shared model defined by the [developer-project RFC](2026-07-14-sdk-developer-projects.md) and the [project-editing architecture](../architecture/2026-07-15-sdk-project-editing-architecture.md). Its create and config workflows are interactive, external Cordis plugins require manual dependency and configuration edits, command-line telemetry has no owning boundary, and interactive branches lack a stable test strategy.
|
||||
|
||||
These gaps are coupled. Create and config already share questions, feature configuration, and `ProjectEditSession`; adding separate automation paths would duplicate that domain logic. External-plugin installation must update both the package manager's files and `cordis.yml`. Telemetry must observe commands such as create and build that do not boot Cordis. Interactive testing must exercise Harness behavior without making terminal rendering a brittle product contract.
|
||||
|
||||
## Proposal
|
||||
|
||||
The SDK extends the existing prompt and project-editing boundaries instead of creating parallel workflows. A non-interactive prompt port and structured feature plan drive create and config, `dsh-sdk create <source>` delegates dependency resolution to the project package manager before mounting the resolved package through `ProjectEditSession`, launcher-side telemetry wraps `create-sdk` and every `dsh-sdk` command, and injected prompt streams provide the primary interactive-test seam.
|
||||
|
||||
| Capability | Product entrypoint | Owning mechanism | Required outcome |
|
||||
|---|---|---|---|
|
||||
| Headless project creation | `create-sdk --config <file>` or `--config-json <json>` with optional `--json` | `HeadlessPromptPort`, structured project answers, and a complete feature plan | No terminal blocking; missing required input is explicit |
|
||||
| External Cordis plugin installation | `dsh-sdk create <source>` | Native package-manager `add` plus `ProjectEditSession` | The dependency and `cordis.yml` entry identify the package manager's resolved package |
|
||||
| Developer-cycle telemetry | `create-sdk` and every `dsh-sdk` command | Launcher-side consent, payload, redaction, anonymous identity, and delivery services | Reporting is best-effort and cannot change the command result |
|
||||
| Interactive regression coverage | Create and config tests | Injected `PromptPort` input/output and filesystem assertions | Tests cover Harness decisions and generated files without snapshotting terminal repainting |
|
||||
|
||||
## Shared headless workflow
|
||||
|
||||
### Structured input and lifecycle events
|
||||
|
||||
Headless create accepts a JSON object either inline through `--config-json` or from a file through `--config`. Scalar fields supply the ordinary create answers, while `features` supplies the complete selected feature set, feature options, secrets, and dedicated values. Defaults remain valid only where the owning question declares one; the headless path never invents an answer for a required prompt.
|
||||
|
||||
With `--json`, stdout is an NDJSON event stream. `done` means creation and any requested setup completed, `action-required` names an unanswered required prompt, and `error` reports another failure. Human-readable progress and package-manager output go to stderr so every stdout line remains parseable as one event. A caller responds to `action-required` by adding the missing value and running the command again.
|
||||
|
||||
Create and config consume the same feature-plan shape. Create exposes it through the command-line inputs above; config uses it at the shared workflow boundary so a later automation entrypoint does not need a second feature-selection model.
|
||||
|
||||
### Prompt and project-editing boundaries
|
||||
|
||||
`PromptPort` remains the only boundary between SDK questions and an interaction implementation. `ClackPromptPort` handles terminals. `HeadlessPromptPort` consumes defaults exposed by the question contract and otherwise fails with the unanswered prompt; prefilled values normally prevent the port from being called.
|
||||
|
||||
Both paths use the same `Question` objects, `FeatureConfigurator`, `SdkProject`, and `ProjectEditSession`. The headless path therefore changes how answers arrive, not how features are interpreted or files are committed.
|
||||
|
||||
### Agent skill
|
||||
|
||||
The repository ships a thin `SKILL.md` that teaches an agent to construct the structured input, request NDJSON, fill an `action-required` value, and retry. The skill invokes the public CLI and does not import an internal SDK API or introduce another project specification.
|
||||
|
||||
## External Cordis plugin installation
|
||||
|
||||
`dsh-sdk create <source>` accepts a package-manager-native npm specifier such as `pkg@version` or a GitHub specifier such as `github:owner/repo#ref`. After confirmation, it asks the project's package manager to add the source, compares the direct dependency names before and after the operation, reopens the project, and mounts each newly resolved package in `cordis.yml` through `ProjectEditSession`.
|
||||
|
||||
The package manager owns source parsing, version or commit resolution, integrity data, lockfile updates, and any build policy. The SDK does not download or unpack a second copy through giget or pacote. An external plugin remains a dependency under `node_modules`; local plugin scaffolding remains a separate project-creation concern.
|
||||
|
||||
## Launcher telemetry
|
||||
|
||||
### Consent and collection
|
||||
|
||||
Telemetry wraps the `create-sdk` initializer and the `dsh-sdk` launcher command lifecycle because project initialization, plugin creation, and build do not reliably boot Cordis. One event records the command name, duration, success, a random per-user anonymous identifier, and redacted `cordis.yml` and `package.json` text when those project files are eligible.
|
||||
|
||||
Reporting is enabled unless a present telemetry config entry is explicitly disabled. `DO_NOT_TRACK` and CI deny reporting regardless of project configuration. A missing `cordis.yml` does not itself deny the event, but `package.json` content is included only when `cordis.yml` establishes that the directory is an SDK project.
|
||||
|
||||
### Safety and delivery
|
||||
|
||||
The payload builder never reads `.env`. It redacts secret-shaped keys and values, known token forms, PEM blocks, URL credentials, and high-entropy opaque strings in the two eligible text files. Redaction is a safety backstop rather than a guarantee; SDK projects must keep credentials in `.env`.
|
||||
|
||||
The reporter uses a fixed endpoint and resolves every send path without throwing. Command dispatch records success or failure in a `finally` path, starts reporting after the command outcome is known, and drains within a bounded interval. Consent parsing, payload construction, storage, or network failures are swallowed only at this telemetry boundary and never alter the command's exit code.
|
||||
|
||||
## Interactive workflow testing
|
||||
|
||||
Create and config tests inject a `PromptPort` and scripted input/output streams into the existing workflows. Parameterized scenarios cover feature selection, feature options, secrets, cancellation, review, and apply behavior, then assert the resulting `cordis.yml` and other project files. The stable product assertion is the generated project state, not clack's ANSI redraw sequence.
|
||||
|
||||
One or two optional real-PTY smoke tests may cover the shipped binary and TTY guard that injection cannot reproduce. Native PTY tooling does not belong on the required path unless it is reliable across the repository's supported Node and host versions.
|
||||
|
||||
## Deferred work
|
||||
|
||||
- Extend the headless create specification to express local `plugin` or `tool` scaffolding instead of defaulting that interactive choice to none.
|
||||
- Expose the telemetry opt-out in create and config while preserving the consent representation in which only a disabled telemetry entry is written.
|
||||
- Define whether GitHub source dependencies must be prebuilt or may run package-manager-controlled preparation scripts, and surface the policy before installation.
|
||||
- Replace the telemetry package's `.invalid` endpoint placeholder with the production endpoint before release.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**Build a separate headless creation engine.** This would duplicate questions, feature requirements, configuration behavior, and project-editing rules. Reusing the prompt and edit-session boundaries keeps one implementation of project semantics.
|
||||
|
||||
**Make a specification file the primary automation interface.** Agents can pass the same typed JSON object inline, while people and CI may still use a file. A file-only protocol adds persistence and cleanup without adding semantics.
|
||||
|
||||
**Use `npx skills add` as the project creator.** The skills CLI installs Markdown skills; it does not create SDK projects or install npm packages. The agent skill therefore drives the SDK initializer instead of replacing it.
|
||||
|
||||
**Fetch GitHub and npm sources through giget or pacote.** A second fetch layer would duplicate package-manager resolution, integrity, lockfile, and lifecycle policy. Native dependency specifiers keep those decisions in the selected package manager.
|
||||
|
||||
**Implement telemetry as a Cordis runtime plugin.** Create and build do not necessarily boot Cordis, so a runtime plugin cannot observe the complete developer command cycle. The launcher is the boundary shared by those commands.
|
||||
|
||||
**Derive the anonymous identifier from git metadata.** Repository remotes can identify a project or organization. A random per-user identifier supports aggregation without encoding repository identity.
|
||||
|
||||
**Collect only aggregate counters.** Aggregate-only events reduce exposure but cannot answer which plugins, dependencies, and configuration shapes developers actually use. This proposal accepts collection of redacted project text and makes that exposure explicit.
|
||||
|
||||
**Use real PTYs and transcript snapshots as the primary test strategy.** Native PTY dependencies and terminal repaint sequences add platform and rendering instability while mostly testing clack. Injected interaction plus generated-file assertions tests the SDK-owned behavior directly.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- Create runs without a TTY from a complete structured input, emits only NDJSON on stdout under `--json`, and reports missing required input as `action-required` without writing a partial project.
|
||||
- Create and config resolve the same feature-plan contract through the shared question, feature-configuration, and project-editing code paths.
|
||||
- `dsh-sdk create <source>` uses the selected project package manager, mounts the dependency name that operation actually added, and fails loudly when no new dependency can be identified.
|
||||
- The initializer and every `dsh-sdk` command reach one best-effort telemetry completion path; an explicit disabled entry, `DO_NOT_TRACK`, or CI prevents delivery, and telemetry failures never change the command result.
|
||||
- Telemetry never reads `.env`, withholds unrelated `package.json` content when no `cordis.yml` exists, redacts both eligible text payloads, and uses an identifier unrelated to git metadata.
|
||||
- Interactive tests cover create and config decisions through injected interaction and assert committed project files; any real-PTY coverage remains a narrow smoke layer.
|
||||
- The agent skill documents the public structured-input and event contracts without depending on private package exports.
|
||||
|
||||
## Risks
|
||||
|
||||
- Full redacted `cordis.yml` and `package.json` text still reveals plugin and dependency names, URLs, paths, and configuration values to the endpoint operator, and heuristic redaction can miss a secret.
|
||||
- Default-on reporting may surprise developers when no telemetry entry exists; the CLI must make the opt-out discoverable before release.
|
||||
- A package-manager add can change `package.json`, the lockfile, and installed files before `ProjectEditSession` mounts the plugin, so a later mount failure can leave dependency changes that require manual recovery.
|
||||
- GitHub dependencies may execute preparation or lifecycle code according to package-manager policy; an unresolved build policy is a supply-chain and reproducibility risk.
|
||||
- Injected prompt tests do not prove raw-mode, signal, or repaint behavior in a real terminal; the optional smoke layer must cover only those residual contracts.
|
||||
|
||||
## References
|
||||
|
||||
- [Vercel Eve](https://github.com/vercel/eve) and [Vercel Labs Skills](https://github.com/vercel-labs/skills) for the distinction between a headless initializer and skill distribution.
|
||||
- [npm package specifications](https://docs.npmjs.com/cli/v11/using-npm/package-spec), [pnpm add](https://pnpm.io/cli/add), and [Yarn add](https://yarnpkg.com/cli/add) for package-manager-native sources.
|
||||
- [`DO_NOT_TRACK`](https://donottrack.sh/) for the environment-level opt-out convention.
|
||||
- [Clack](https://github.com/bombshell-dev/clack) and [Vitest snapshots](https://vitest.dev/guide/snapshot) for injected prompts and generated-file assertions.
|
||||
@@ -0,0 +1,118 @@
|
||||
# RFC: SDK 后续功能
|
||||
|
||||
Status: proposed
|
||||
|
||||
[English](2026-07-17-sdk-follow-up-capabilities.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
首个 SDK 版本通过[开发者工程 RFC](2026-07-14-sdk-developer-projects.md) 和 [SDK 工程编辑架构](../architecture/2026-07-15-sdk-project-editing-architecture.md)定义的共享模型创建和编辑开发者拥有的 Cordis 工程。create 和 config 工作流仅支持交互调用,接入外部 Cordis 插件需要手工修改依赖和配置,命令行遥测没有明确的所属边界,交互分支也缺少稳定的测试策略。
|
||||
|
||||
这些缺口彼此关联。create 和 config 已经共享问题、功能配置和 `ProjectEditSession`;若另建自动化路径,就会复制领域逻辑。安装外部插件必须同时修改包管理器文件和 `cordis.yml`。遥测需要观察 create、build 等不会启动 Cordis 的命令。交互测试需要覆盖 Harness 自身行为,同时避免把终端渲染固化成脆弱的产品契约。
|
||||
|
||||
## 提案
|
||||
|
||||
SDK 扩展现有提示词与工程编辑边界,不另建平行工作流。非交互式 `PromptPort` 实现和结构化功能计划驱动 create 与 config;`dsh-sdk create <source>` 先把依赖解析交给工程的包管理器,再通过 `ProjectEditSession` 挂载解析所得的包;启动器侧遥测包住 `create-sdk` 和每个 `dsh-sdk` 命令;交互测试主要通过注入的提示词输入输出流完成。
|
||||
|
||||
| 功能 | 产品入口 | 所属机制 | 必须达到的结果 |
|
||||
|---|---|---|---|
|
||||
| Headless 工程创建 | `create-sdk --config <file>` 或 `--config-json <json>`,可搭配 `--json` | `HeadlessPromptPort`、结构化工程答案和完整功能计划 | 不阻塞等待终端;明确报告缺失的必答输入 |
|
||||
| 外部 Cordis 插件安装 | `dsh-sdk create <source>` | 包管理器原生 `add` 加 `ProjectEditSession` | 依赖和 `cordis.yml` 配置项指向包管理器解析出的包 |
|
||||
| 开发周期遥测 | `create-sdk` 和每个 `dsh-sdk` 命令 | 启动器侧的上报条件判断、遥测内容构建、脱敏、匿名身份和传输服务 | 上报采用尽力而为语义,不能改变命令结果 |
|
||||
| 交互回归覆盖 | create 和 config 测试 | 注入的 `PromptPort` 输入输出和文件系统断言 | 测试覆盖 Harness 决策与生成文件,不快照终端重绘 |
|
||||
|
||||
## 共享 headless 工作流
|
||||
|
||||
### 结构化输入和生命周期事件
|
||||
|
||||
Headless create 通过 `--config-json` 接收内联 JSON 对象,或通过 `--config` 从文件读取。标量字段提供普通 create 答案,`features` 提供完整的已选功能、功能选项、secret(密钥)和专用值。只有所属问题明确声明的默认值才有效;headless 路径绝不为必答问题臆造答案。
|
||||
|
||||
使用 `--json` 时,stdout 是 NDJSON 事件流。`done` 表示创建及要求执行的安装和构建均已完成,`action-required` 指明一个尚未回答的必答问题,`error` 报告其他失败。面向人的进度信息和包管理器输出写入 stderr,确保 stdout 每一行都能解析成一个事件。调用方收到 `action-required` 后补充缺失值,再次运行命令。
|
||||
|
||||
Create 和 config 使用相同的功能计划形状。create 通过上述命令行输入公开该形状;config 在共享工作流边界使用同一形状,使后续自动化入口无需另建功能选择模型。
|
||||
|
||||
### Prompt 与工程编辑边界
|
||||
|
||||
`PromptPort` 仍是 SDK 问题与交互实现之间的唯一边界。`ClackPromptPort` 负责终端交互。`HeadlessPromptPort` 使用问题契约公开的默认值,否则通过未回答问题快速失败;预填值通常会让流程根本不调用该 port。
|
||||
|
||||
两条路径使用相同的 `Question` 对象、`FeatureConfigurator`、`SdkProject` 和 `ProjectEditSession`。因此,headless 路径只改变答案的到达方式,不改变功能解释或文件提交方式。
|
||||
|
||||
### Agent skill
|
||||
|
||||
仓库提供一份轻量 `SKILL.md`,指导 agent skill(智能体技能)构造结构化输入、请求 NDJSON、补充 `action-required` 指明的值并重试。该 skill 调用公开 CLI,不导入 SDK 内部 API,也不引入另一套工程规格。
|
||||
|
||||
## 外部 Cordis 插件安装
|
||||
|
||||
`dsh-sdk create <source>` 接受包管理器原生的 npm package specifier,例如 `pkg@version`,也接受 `github:owner/repo#ref` 等 GitHub package specifier。用户确认后,命令要求工程包管理器添加来源,对比操作前后的直接依赖名,重新打开工程,再通过 `ProjectEditSession` 把每个新增且已解析的包挂载进 `cordis.yml`。
|
||||
|
||||
包管理器负责来源解析、版本或 commit 解析、`integrity` 数据、lockfile 更新和构建策略。SDK 不再通过 giget 或 pacote 下载、解压第二份副本。外部插件是 `node_modules` 下的依赖;本地插件脚手架仍属于独立的工程创建问题。
|
||||
|
||||
## Launcher 遥测
|
||||
|
||||
### Consent 与采集
|
||||
|
||||
遥测包住 `create-sdk` 初始化命令与 `dsh-sdk` launcher 的命令生命周期,因为工程初始化、插件创建和 build 都不会稳定地启动 Cordis。每个事件记录命令名、时长、成败、随机生成的用户级匿名标识符,以及符合条件时经过脱敏的 `cordis.yml` 与 `package.json` 文本。
|
||||
|
||||
除非当前存在的遥测配置项被明确禁用,否则允许上报。`DO_NOT_TRACK` 和 CI 无论工程配置如何都禁止上报。缺少 `cordis.yml` 本身不会禁止事件,但只有 `cordis.yml` 能证明目录是 SDK 工程时,遥测内容才包含 `package.json` 文本。
|
||||
|
||||
### 安全与传输
|
||||
|
||||
Payload 构建器绝不读取 `.env`。它会脱敏两个符合条件的文本文件中的疑似密钥键和值、已知 token 形式、PEM 块、URL 凭据和高熵不透明字符串。脱敏只是安全兜底,不能提供绝对保证;SDK 工程必须把凭据放进 `.env`。
|
||||
|
||||
`TelemetryReporter` 使用固定 endpoint,每条发送路径都会正常结束且不抛错。命令分发通过 `finally` 路径记录成败,在命令结果已确定后启动上报,并在有界时间内等待传输结束。只有遥测边界会吞掉上报条件解析、遥测内容构建、存储或网络错误,这些错误绝不改变命令退出码。
|
||||
|
||||
## 交互工作流测试
|
||||
|
||||
Create 和 config 测试向现有工作流注入 `PromptPort` 和脚本化输入输出流。参数化场景覆盖功能选择、功能选项、secret、取消、评审和应用行为,再断言最终的 `cordis.yml` 及其他工程文件。稳定的产品断言是生成后的工程状态,不是 clack 的 ANSI 重绘序列。
|
||||
|
||||
可以用一到两个可选的真实 PTY 冒烟测试覆盖注入无法复现的发布二进制和 TTY 检查。除非原生 PTY 工具在仓库支持的 Node 与宿主版本上足够可靠,否则它不进入必跑路径。
|
||||
|
||||
## 延后工作
|
||||
|
||||
- 扩展 headless create 规格,使其能表达本地 `plugin` 或 `tool` 脚手架,而不是把该交互选择默认为 none。
|
||||
- 在 create 和 config 中公开遥测关闭选项,同时保留只有禁用时才写入遥测配置项的上报许可表示。
|
||||
- 明确 GitHub 来源依赖必须预先构建,还是允许运行由包管理器控制的 preparation script(准备脚本),并在安装前向用户展示该策略。
|
||||
- 发布前把遥测包中的 `.invalid` endpoint 占位符替换为生产端点。
|
||||
|
||||
## 曾考虑的替代方案
|
||||
|
||||
**另建 headless 创建引擎。** 该方案会复制问题、功能依赖、配置行为和工程编辑规则。复用提示词与编辑会话边界,可以保证工程语义只有一份实现。
|
||||
|
||||
**把规格文件作为主要自动化接口。** Agent 可以内联传入相同的类型化 JSON 对象,人和 CI 仍可选用文件。文件专用协议会增加持久化与清理工作,却不增加语义。
|
||||
|
||||
**使用 `npx skills add` 创建工程。** Skills CLI 只安装 Markdown skill,不创建 SDK 工程,也不安装 npm 包。因此,agent skill 驱动 SDK 初始化命令,而不是取代它。
|
||||
|
||||
**通过 giget 或 pacote 获取 GitHub 与 npm 来源。** 第二套获取层会复制包管理器的解析、完整性、lockfile 和生命周期策略。原生 package specifier 让这些决策留在所选包管理器中。
|
||||
|
||||
**把遥测实现成 Cordis 运行时插件。** Create 和 build 不一定启动 Cordis,因此运行时插件无法观察完整的开发命令周期。Launcher 是这些命令共用的边界。
|
||||
|
||||
**从 git 元数据派生匿名标识符。** 仓库的 git remote 可能识别工程或组织。随机的用户级标识符能够支持聚合,同时不编码仓库身份。
|
||||
|
||||
**只采集聚合计数。** 仅聚合事件可以降低暴露,但无法回答开发者实际使用哪些插件、依赖和配置形状。本提案接受采集脱敏后的工程文本,并明确记录这项暴露。
|
||||
|
||||
**把真实 PTY 和 transcript(文本记录)快照作为主要测试策略。** 原生 PTY 依赖与终端重绘序列会带来平台和渲染不稳定性,而且主要是在测试 clack。注入交互并断言生成文件,可以直接测试 SDK 拥有的行为。
|
||||
|
||||
## 验收标准
|
||||
|
||||
- Create 能依据完整结构化输入在没有 TTY 时运行;使用 `--json` 时 stdout 只输出 NDJSON;缺少必答输入时通过 `action-required` 报告,且不写入部分工程。
|
||||
- Create 和 config 通过共享的问题、功能配置和工程编辑代码路径解析相同的功能计划契约。
|
||||
- `dsh-sdk create <source>` 使用工程选定的包管理器,挂载该操作实际新增的依赖名;无法识别新增依赖时快速失败。
|
||||
- 初始化命令与每个 `dsh-sdk` 命令都进入同一条尽力而为的遥测收尾路径;明确禁用的配置项、`DO_NOT_TRACK` 或 CI 会阻止传输,遥测失败绝不改变命令结果。
|
||||
- 遥测绝不读取 `.env`;没有 `cordis.yml` 时不发送无关的 `package.json` 内容;两个符合条件的文本都经过脱敏;匿名标识符与 git 元数据无关。
|
||||
- 交互测试通过注入交互覆盖 create 和 config 决策,并断言已提交的工程文件;真实 PTY 覆盖只作为窄范围冒烟层。
|
||||
- Agent skill 说明公开的结构化输入与事件契约,不依赖包的私有导出。
|
||||
|
||||
## 风险
|
||||
|
||||
- 即使经过脱敏,完整的 `cordis.yml` 与 `package.json` 文本仍会向 endpoint 运营方暴露插件名、依赖名、URL、路径和配置值;启发式脱敏也可能漏掉 secret。
|
||||
- 没有遥测配置项时默认上报可能让开发者意外;发布前 CLI 必须让关闭方法易于发现。
|
||||
- 在 `ProjectEditSession` 挂载插件前,包管理器的 add 操作已经可能修改 `package.json`、lockfile 和安装文件;后续挂载失败会留下需要手工恢复的依赖改动。
|
||||
- GitHub 依赖可能按包管理器策略执行 preparation 或 lifecycle script;尚未解决的构建策略会带来供应链与可复现性风险。
|
||||
- 注入提示词交互的测试无法证明真实终端中的 raw mode、signal 或重绘行为;可选冒烟层只应覆盖这些残余契约。
|
||||
|
||||
## 参考资料
|
||||
|
||||
- [Vercel Eve](https://github.com/vercel/eve) 与 [Vercel Labs Skills](https://github.com/vercel-labs/skills) 用于区分 headless 初始化命令与 skill 分发。
|
||||
- [npm package specifications](https://docs.npmjs.com/cli/v11/using-npm/package-spec)、[pnpm add](https://pnpm.io/cli/add)和 [Yarn add](https://yarnpkg.com/cli/add)说明包管理器原生来源。
|
||||
- [`DO_NOT_TRACK`](https://donottrack.sh/)定义环境级关闭约定。
|
||||
- [Clack](https://github.com/bombshell-dev/clack) 和 [Vitest snapshots](https://vitest.dev/guide/snapshot) 说明注入提示词交互与生成文件断言。
|
||||
@@ -4,7 +4,7 @@ Status: rejected — `step/end` is the durable indication that a model step fini
|
||||
|
||||
## Problem
|
||||
|
||||
The session log stores `step/start` and `step/end` events even though every step-scoped event already carries `{ turn, step }`: assistant chunks, assistant messages, tool calls, tool results, usage, and errors. `deriveMessages()` ignores step boundaries, ACP ignores them for UI, and the main consumers are invariants, tests, snapshot goldens, and crash repair.
|
||||
The session log stores `step/start` and `step/end` events even though every step-scoped event already carries `{ turn, step }`: assistant chunks, assistant messages, tool calls, tool results, usage, and errors. `deriveMessages()` ignores step boundaries, ACP ignores them for UI, and the main consumers are invariants, tests, snapshot expected outputs, and crash repair.
|
||||
|
||||
The rejected argument was that boundary events make the log more ceremonial than informative. In practice, `step/end` is concrete information: a reader can tell whether a model request finished, crashed, or is being repaired without deriving that state from the next event. A bare `step/start` is likewise useful for a model request that began but produced no chunks before failing.
|
||||
|
||||
|
||||
@@ -22,7 +22,7 @@ As a smaller alternative, replace the current optional-field bag with one explic
|
||||
- `ToolCallPresentation`, `ToolResultPresentation`, `ToolTerminal`, and `ToolCallKind` disappear unless a minimal generic UI type still needs one.
|
||||
- ACP no longer keeps presenter pending state or calls tool callbacks during live streaming/load replay.
|
||||
- `dsh-tool-bash` no longer parses rendered text to recover exit status for a UI pill.
|
||||
- Snapshot goldens show generic tool cards and text results.
|
||||
- Snapshot expected outputs show generic tool cards and text results.
|
||||
|
||||
## What we give up
|
||||
|
||||
|
||||
+1
-1
@@ -7,7 +7,7 @@ How this repo tests, tier by tier, and the rules that keep a green suite meaning
|
||||
- **Unit** (`pnpm run test`): vitest over `packages|examples/*/tests/**/*.spec.ts`, colocated with what they test. Every registry gets an HMR-safety test (dispose the contributing fiber, assert cleanup). Prefer edge cases, error paths, event ordering, concurrency races, and permanent contract regressions (see `packages/core/agent-loop/tests/contract-regressions.spec.ts`).
|
||||
- **Coverage gate** (`pnpm run test:coverage`): the gating run, per-file 100% on `packages/*/*/src`. An uncovered line is often dead code the gate is correctly flagging for deletion, not a missing test to bolt on. Line coverage is necessary, never sufficient — it proves lines ran, not that the feature works as shipped.
|
||||
- **Real-API e2e** (`pnpm run test:e2e`): with-key tests against live provider APIs — the DeepSeek model plus provider-specific smokes that gate on their own keys (`EXA_API_KEY`, `PERPLEXITY_API_KEY`, …); each suite self-skips without its key so keyless CI stays green ([real-API e2e RFC](rfc/implemented/testing/2026-06-19-real-api-e2e-ci.md)).
|
||||
- **Snapshot** (`pnpm run test:snapshot`): transport-specific keyless goldens cover external presentation. ACP suites boot the real example subprocess, replay a recorded session, and diff normalized JSON-RPC plus the re-persisted log ([ACP snapshot RFC](rfc/implemented/testing/2026-06-19-acp-snapshot-tests.md)); the headless suite independently pins `stream-json` through its real one-shot subprocess. TUI completed journeys replay recorded primary/child JSONL through the real agent loop and tools before projecting ANSI into semantic terminal-state goldens; package-local snapshots retain transient renderer states, and a real PTY conversation covers the process boundary ([TUI snapshot RFC](rfc/implemented/testing/2026-07-18-tui-terminal-state-snapshots.md)). Use `pnpm run test:snapshot:record` when a model transcript must change and `pnpm run test:snapshot:refresh` when committed replay input remains correct; review every JSONL and golden diff. System-prompt/tool-schema content is pinned by one ACP scenario (`text-turn`) and tokenized in every other fixture, so a prompt or schema edit churns one committed line ([pinned-header RFC](rfc/implemented/testing/2026-07-06-pin-request-header-content-in-one-scenario.md)).
|
||||
- **Snapshot** (`pnpm run test:snapshot`): transport-specific keyless expected outputs cover external presentation. ACP suites boot the real example subprocess, replay a recorded session, and diff normalized JSON-RPC plus the re-persisted log ([ACP snapshot RFC](rfc/implemented/testing/2026-06-19-acp-snapshot-tests.md)); the headless suite independently pins `stream-json` through its real one-shot subprocess. TUI completed journeys replay recorded primary/child JSONL through the real agent loop and tools before projecting ANSI into semantic terminal-state expected outputs; package-local snapshots retain transient renderer states, and a real PTY conversation covers the process boundary ([TUI snapshot RFC](rfc/implemented/testing/2026-07-18-tui-terminal-state-snapshots.md)). Use `pnpm run test:snapshot:record` when a model transcript must change and `pnpm run test:snapshot:refresh` when committed replay input remains correct; review every JSONL and expected-output diff. System-prompt/tool-schema content is pinned by one ACP scenario (`text-turn`) and tokenized in every other fixture, so a prompt or schema edit churns one committed line ([pinned-header RFC](rfc/implemented/testing/2026-07-06-pin-request-header-content-in-one-scenario.md)).
|
||||
|
||||
## The with-key policy: inference is cheap here
|
||||
|
||||
|
||||
@@ -33,6 +33,10 @@ The full-screen terminal sibling of `repl-agent`: it reuses the same coding back
|
||||
|
||||
Run with: `pnpm run demo:tui` (needs `DEEPSEEK_API_KEY`). See [tui-agent/README.md](tui-agent/README.md) for controls and composition.
|
||||
|
||||
## jsonrpc-agent
|
||||
|
||||
An unattended coding agent driven through the Python SDK: JSON-RPC stdio, foreground-only `bash`, `read` / `write` / `edit`, one foreground `subagent`, `todo_write`, JSONL persistence, and compaction. It excludes terminal UI, stdout logging, approvals, skills, and background task controls. See [jsonrpc-agent/README.md](jsonrpc-agent/README.md).
|
||||
|
||||
## cordis-agent
|
||||
|
||||
The **self-referential** demo: the coding spine plus [`@deepseek-ai/dsh-tool-cordis`](../packages/cordis/tool-cordis), whose three tools (`cordis_inspect` / `cordis_mount` / `cordis_unmount`) let the agent inspect the live cordis runtime it runs inside, mount model-written plugins into it (an event listener, a brand-new tool for itself, or a service another mount injects), and dispose them again — all dynamic mounts grouped under one `cordis-dynamic` fiber subtree. The `ctx.fs`/`ctx.web` services ride along provider-only, as the capabilities those plugins build on.
|
||||
|
||||
@@ -5,10 +5,10 @@ import { defineAcpSnapshotSuite, type Scenario, type SnapshotSuiteOptions } from
|
||||
/**
|
||||
* The acp-agent example's snapshot suite: the scenario table for
|
||||
* `dsh-acp-snapshot`'s suite factory, which owns every compare/guard mechanic
|
||||
* (golden + re-persisted-log diffs, record/refresh write-back, the pinned-header
|
||||
* (expected-output + re-persisted-log diffs, record/refresh write-back, the pinned-header
|
||||
* uniformity guard, the fixture guards). Fixtures live under `snapshots/<name>/`;
|
||||
* `pnpm run test:snapshot:record` re-records model transcripts against the real
|
||||
* API; `pnpm run test:snapshot:refresh` rewrites current replay goldens keyless.
|
||||
* API; `pnpm run test:snapshot:refresh` rewrites current replay expected outputs keyless.
|
||||
* See the package README (packages/support/acp-snapshot) and the snapshot RFC,
|
||||
* docs/rfc/implemented/testing/2026-06-19-acp-snapshot-tests.md.
|
||||
*/
|
||||
@@ -137,7 +137,7 @@ const SCENARIOS: Scenario[] = [
|
||||
// The mid-turn seams fire during a real model turn, so each is recorded with its hook active
|
||||
// (the model's reaction to a deny/block/force-continue is part of the captured transcript).
|
||||
// SessionStart/SubagentStart are excluded because detached injection races log
|
||||
// order; SubagentStop writes no transcript, so a golden could not prove it ran.
|
||||
// order; SubagentStop writes no transcript, so an expected output could not prove it ran.
|
||||
// Unit tests cover those points; the hook-snapshot-matrix RFC owns the rationale.
|
||||
{ name: 'hook-cc-promptsubmit-context', hasModelTurn: true, recorded: true },
|
||||
{ name: 'hook-cc-pretool-deny', hasModelTurn: true, recorded: true },
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user