docs: address final model experience review
This commit is contained in:
Tianyi Cui
@@ -16,7 +16,7 @@ packages/<group>/<pkg>/
|
||||
README.md # service API, events, extension points, design notes,
|
||||
# + gated Model Experience context blocks or short sentence
|
||||
# + the gated "Known Limitations and Deferred Work" section
|
||||
# (or a whitelist entry in scripts/verify-readme-limitations.ts)
|
||||
# (or a whitelist entry in scripts/verify-package-readme-limitations.ts)
|
||||
```
|
||||
|
||||
Choose an existing group when one matches the package's role (`core`, `llm`, `bash`, `compact`, `subagent`, `todo`, `session-persistence`, `ui`, `util`, or `support`). A new group is allowed, but it is a pure container: no `package.json`, no source files, and packages still sit exactly one level below it.
|
||||
@@ -42,23 +42,21 @@ For a swappable capability, split interface / implementation / consumer into sep
|
||||
|
||||
## 4. Write the package README
|
||||
|
||||
Keep package-specific service API, config, events, extension points, and design notes first. End a package README with this canonical sequence:
|
||||
Keep package-specific service API, config, events, extension points, and design notes first. Document only behavior, limitations, and deferred work owned by this package: a feature already implemented elsewhere is not this package's limitation. An indirect Model Experience sentence may name the consumer that surfaces this package's contribution, but it does not restate that consumer's implementation. End a package README with this canonical sequence:
|
||||
|
||||
````markdown
|
||||
## Model Experience
|
||||
|
||||
### Request surface and condition
|
||||
|
||||
**What the model sees**: Short verbatim text, an exact data-dependent shape, or a link to the long literal below.
|
||||
**What the model sees**: An exact data-dependent shape, an anchored generated-catalog link, or an introduction to the verbatim literal below.
|
||||
|
||||
**Token effect**: Fixed, conditional, retained, replaced, capped, or zero-direct token effect.
|
||||
|
||||
### Verbatim model-visible text
|
||||
|
||||
#### Descriptive title matching the context surface
|
||||
#### Verbatim text for this context surface, when needed
|
||||
|
||||
```markdown
|
||||
Long stable prompt or schema description, copied exactly from source.
|
||||
Stable system-prompt prose of any length, or another long non-generated literal, copied exactly from source.
|
||||
```
|
||||
|
||||
## Known Limitations and Deferred Work
|
||||
@@ -66,9 +64,9 @@ Long stable prompt or schema description, copied exactly from source.
|
||||
- **Consumer-visible gap** — exact boundary or deliberately deferred work.
|
||||
````
|
||||
|
||||
Fill Model Experience from the implementation. Direct, multi-surface, conditional, capped, or lifetime effects use one H3 block per context surface; each block has the exact bold-led `What the model sees` and `Token effect` fields shown above. Quote short stable model-visible source literals verbatim in inline code, using named placeholders such as `<mode>` only for interpolated values. Put a long stable prompt or schema description under the optional exact `### Verbatim model-visible text` heading: give each literal an H4 title and its own `markdown` fence, then link it from the relevant context surface. Summarize only data-dependent payloads, provider-owned text, or schemas too large to reproduce, and identify their exact shape and renderer. Do not infer prompt visibility from tool-schema visibility because independently registered guidance can remain after a scoped tool restriction.
|
||||
Fill Model Experience from the implementation. Direct, multi-surface, conditional, capped, or lifetime effects use one H3 block per context surface; each block has the exact bold-led `What the model sees` and `Token effect` fields shown above. A structured section grounds at least one surface in concrete model-visible text through inline code, a nested `markdown` block, or an anchored tool-catalog link. Put every stable system-prompt paragraph, including a one-liner, in a titled H4 plus `markdown` fence immediately after those fields inside the owning H3 whose title contains `system prompt`; never leave prompt prose in inline code. Quote other short stable model-visible source literals inline, using named placeholders such as `<mode>` only for interpolated values, and attach other long non-generated literals to their owning H3 in the same H4-plus-fence form. Describe an attached literal as the text "below" instead of linking between Model Experience subsections; the physical nesting already records ownership. A tool-schema surface uses `schema` in its H3 and links the relevant anchored package section in the generated [tool schema catalog](../tool-catalog.md) instead of copying its default descriptions or JSON Schema; describe only configuration or composition deltas the catalog does not contain. A runtime-only definition outside the catalog's stated scope links that scope and explains the exception before reproducing its stable text. Summarize data-dependent payloads or provider-owned text by identifying their exact shape and renderer. Do not infer prompt visibility from tool-schema visibility because independently registered guidance can remain after a scoped tool restriction.
|
||||
|
||||
An audited package with no context effect or one simple consumer-owned path belongs in [`SENTENCE_MODEL_EXPERIENCE`](../../scripts/verify-package-readme-model-experience.ts) and uses one line beginning `None, as ` or `Indirectly, through `. A provider backend whose single context path is formatted and inserted entirely by a named consumer uses the indirect sentence even when it caps or filters data before returning it; do not give it a structured block describing the consumer's work. Packages that register model input themselves, invoke a model, or own multiple context paths keep context-surface blocks; the verifier gates their H3 headings, field labels, spacing, and the optional verbatim appendix's H4-plus-`markdown`-fence shape. A package with genuinely no limitations joins the separate allowlist in [`verify-readme-limitations.ts`](../../scripts/verify-readme-limitations.ts) and ends after Model Experience. The [Model Experience RFC](../rfc/implemented/process/2026-07-12-package-model-experience-contract.md) records the rationale.
|
||||
An audited package with no context effect or one simple consumer-owned path belongs in [`SENTENCE_MODEL_EXPERIENCE`](../../scripts/verify-package-readme-model-experience.ts) and uses one line beginning `None, as ` or `Indirectly, through `. Pure routing, transport, and keyless test-support packages use `None, as ` when they create no model-bound content even if consumers use them during composition. A provider backend whose single context path is formatted and inserted entirely by a named consumer uses `Indirectly, through ` even when it caps or filters data before returning it; a wiring bundle whose model effects all belong to named children uses the same form. Do not give these packages a structured block describing another package's work. Packages that own model input, output shaping, multiple context paths, or an auxiliary request keep context-surface blocks; the verifier gates their H3 headings, field labels, spacing, concrete literal evidence, nested H4-plus-`markdown` blocks, absence of local subsection links, system-prompt literals, and schema-surface-to-catalog links. A package with genuinely no limitations joins the separate allowlist in [`verify-package-readme-limitations.ts`](../../scripts/verify-package-readme-limitations.ts) and ends after Model Experience. The [Model Experience RFC](../rfc/implemented/process/2026-07-12-package-model-experience-contract.md) records the rationale.
|
||||
|
||||
## 5. Verify
|
||||
|
||||
|
||||
@@ -8,7 +8,7 @@ The [documentation standard](../../../AGENTS.md) assigns limitations to the pack
|
||||
|
||||
## Decision
|
||||
|
||||
Every package manifest under `packages/<group>/<pkg>/package.json` has a sibling README carrying a canonical `## Known Limitations and Deferred Work` section: a condensed bullet list of consumer-visible gaps (unimplemented features, platform caveats, deliberate MVP cuts) and consciously postponed work (TODO markers, RFC deferrals still open). A `doc-sync` gate, `verify-readme-limitations` ([scripts/verify-readme-limitations.ts](../../../../scripts/verify-readme-limitations.ts)), derives the package set from those manifests, rejects a missing README, and enforces the shape per README: exactly one limitations-like heading, byte-equal to the canonical h2, with at least one top-level bullet. Near-miss headings at any level ("Limitations", "Deferred", "What is NOT here", "Non-goals", …) fail the gate, so variant sections cannot creep back beside — or instead of — the canonical one.
|
||||
Every package manifest under `packages/<group>/<pkg>/package.json` has a sibling README carrying a canonical `## Known Limitations and Deferred Work` section: a condensed bullet list of consumer-visible gaps (unimplemented features, platform caveats, deliberate MVP cuts) and consciously postponed work (TODO markers, RFC deferrals still open). A `doc-sync` gate, `verify-package-readme-limitations` ([scripts/verify-package-readme-limitations.ts](../../../../scripts/verify-package-readme-limitations.ts)), derives the package set from those manifests, rejects a missing README, and enforces the shape per README: exactly one limitations-like heading, byte-equal to the canonical h2, with at least one top-level bullet. Near-miss headings at any level ("Limitations", "Deferred", "What is NOT here", "Non-goals", …) fail the gate, so variant sections cannot creep back beside — or instead of — the canonical one.
|
||||
|
||||
A package with genuinely nothing to declare is whitelisted (`NO_LIMITATIONS` in the script) and must NOT carry the section. The inverted check keeps the whitelist honest in both directions: an empty or boilerplate section cannot satisfy the gate, and giving a whitelisted package real limitations forces the whitelist edit in the same change. Whitelist entries are validated against the scanned package set, so a package rename or removal fails loud instead of silently un-gating a README.
|
||||
|
||||
@@ -22,7 +22,7 @@ The gate checks presence, shape, and the whitelist; the bullets' truthfulness an
|
||||
|
||||
## Consequences
|
||||
|
||||
- A new package cannot ship without either declaring its gaps or explicitly claiming it has none; a missing, drifted, or empty section fails `doc-sync` locally (pre-push) and in CI (`readme-limitations` in the run-gates doc-sync leaf set).
|
||||
- A new package cannot ship without either declaring its gaps or explicitly claiming it has none; a missing, drifted, or empty section fails `doc-sync` locally (pre-push) and in CI (`package-readme-limitations` in the run-gates doc-sync leaf set).
|
||||
- Every package README answers the limitations question through the canonical heading or an explicit no-limitations allowlist entry.
|
||||
- One more fast tsx script in the `doc-sync` chain; no new dependency (plain `node:fs` glob + line scan).
|
||||
- The canonical heading is enforced verbatim, so renaming it later is a mechanical one-script-plus-all-READMEs change guarded by the same gate.
|
||||
@@ -8,11 +8,11 @@ A package README can explain APIs and runtime mechanics without answering the qu
|
||||
|
||||
## Decision
|
||||
|
||||
Every workspace package README ends with the canonical [Model Experience section](../../../cookbook/adding-a-package.md#4-write-the-package-readme), immediately before `## Known Limitations and Deferred Work`; a package on the no-limitations allowlist ends with Model Experience itself. Packages with direct, multi-surface, conditional, capped, or lifetime effects use one H3 block per context surface. Each block says what the relevant model literally receives and when under `**What the model sees**:`, then classifies the token effect under `**Token effect**:`. Short stable source literals are quoted verbatim in the first field, with named placeholders only for interpolated values. Long stable prompts or schema descriptions live in a `### Verbatim model-visible text` appendix after the context surfaces, one titled `markdown` fence per literal, and the relevant surface links to them. Summaries are reserved for data-dependent payloads, provider-owned text, or schemas too large to reproduce. The default subject is the conversation model; a package that invokes an auxiliary model, such as a summarizer or search provider, names that request separately. Agent-scoped visibility is stated where it changes which agent receives a contribution. Prompt text and tool schemas are described separately whenever configuration or scoping can hide one without the other.
|
||||
Every workspace package README ends with the canonical [Model Experience section](../../../cookbook/adding-a-package.md#4-write-the-package-readme), immediately before `## Known Limitations and Deferred Work`; a package on the no-limitations allowlist ends with Model Experience itself. Packages with direct, multi-surface, conditional, capped, or lifetime effects use one H3 block per context surface. Each block says what the relevant model literally receives and when under `**What the model sees**:`, then classifies the token effect under `**Token effect**:`; the structured section grounds at least one surface with inline code, a nested `markdown` block, or an anchored catalog link. Every stable system-prompt paragraph, including a one-liner, follows those fields inside the owning H3 as a titled H4 plus `markdown` fence; the H3 title contains `system prompt`. Other short stable source literals remain inline with named placeholders only for interpolated values; other long non-generated literals use the same nested H4 form. Model Experience subsections do not link to each other because physical nesting owns the literal. Tool-schema surfaces use `schema` in their H3 and link the relevant anchored package section of the generated [tool schema catalog](../../../tool-catalog.md) rather than copying default descriptions or JSON Schema, then state only configuration or composition deltas absent from that catalog. A runtime-only definition outside the catalog's stated scope links that scope and explains the exception before reproducing its stable text. Summaries are reserved for data-dependent payloads and provider-owned text. The default subject is the conversation model; a package that invokes an auxiliary model, such as a summarizer or search provider, names that request separately. Agent-scoped visibility is stated where it changes which agent receives a contribution. Prompt text and tool schemas are described separately whenever configuration or scoping can hide one without the other.
|
||||
|
||||
Every package participates. A package with no model-context effect, or one simple effect rendered entirely by another package, uses the verifier's audited sentence allowlist. It carries exactly one sentence beginning `None, as ` or `Indirectly, through ` instead of expanding a negative fact into a structured block. A provider backend whose single context path is formatted and inserted entirely by a named consumer uses the indirect sentence even when it caps or filters data before returning it. Packages that register model input themselves, invoke a model, or own multiple context paths keep the context-surface blocks even when they add zero direct prompt tokens.
|
||||
Every package participates. A package with no model-context effect, or one simple effect rendered entirely by another package, uses the verifier's audited sentence allowlist. It carries exactly one sentence beginning `None, as ` or `Indirectly, through ` instead of expanding a negative fact into a structured block. Pure routing, transport, and keyless test-support packages use the none form when they create no model-bound content. A provider backend whose single context path is formatted and inserted entirely by a named consumer uses the indirect sentence even when it caps or filters data before returning it; wiring bundles do the same when every model effect belongs to named children. The indirect form names that consumer only to locate this package's contribution and does not restate the consumer's implementation. Structured blocks likewise document only package-owned inputs, transformations, and deltas. Packages that own model input, output shaping, multiple context paths, or an auxiliary request keep context-surface blocks even when they add zero direct prompt tokens.
|
||||
|
||||
`verify-package-readme-model-experience` discovers packages from `packages/*/*/package.json`, requires one sibling README and the canonical final-section order, and validates one of two package-classified bodies. An allowlisted package carries exactly one sentence with its assigned prefix; every other package carries at least one H3 context surface with the two exact, non-empty fields and one blank line between each element. Optional long literals follow the structured blocks under the exact appendix heading, with each H4 title paired to one non-empty `markdown` fence and linked from a model-view field. It runs in `doc-sync` and the parallel gate runner. The check owns package classification, structural presence, block and appendix shape, and order; implementation review owns coverage and the truth of the prose.
|
||||
`verify-package-readme-model-experience` discovers packages from `packages/*/*/package.json`, requires one sibling README and the canonical final-section order, and validates one of two package-classified bodies. An allowlisted package carries exactly one sentence with its assigned prefix; every other package carries at least one H3 context surface with the two exact, non-empty fields and one blank line between each element plus at least one inline literal, nested block, or catalog link across the section. Optional verbatim literals follow those fields inside that surface, each as an H4 title paired to one non-empty `markdown` fence. Local subsection links are rejected, every system-prompt surface requires at least one nested block, and every tool-schema surface links an existing H2 section in the generated tool catalog. The check runs in `doc-sync` and the parallel gate runner. It owns package classification, structural presence, concrete-literal evidence, nested-block shape, catalog-link shape, and order; implementation review owns coverage, link relevance, and the truth of the prose.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
|
||||
Reference in New Issue
Block a user