Commit Graph
34 Commits
Author SHA1 Message Date
imccyu ec601ca13d build(vendor): rescope the vendored Cordis packages into @deepseek-ai
Machine-produced by `pnpm run rescope-vendor --apply` plus the regeneration it
prints: `pnpm install` for the lockfile, `pnpm run gen-third-party-notices`,
`verify-translation-pairing --write` for the touched bilingual pairs,
`gen-doc-graphs`, and one typert snapshot whose ids embed character offsets.
`pnpm run rescope-vendor --check` verifies the result.

Renames nine vendored packages (cordis, cosmokit, schemastery and the six
@cordisjs plugins) and every reference that resolves them: manifest names and
dependency keys, module specifiers including declare-module merges, cordis.yml
plugin names, tsconfig paths, every Markdown fence, and `docs/` prose.
Directory names, upstream versions, and dependency ranges are unchanged, so
vendor/README.md still reads as an upstream snapshot; its manifest table gains
an upstream-name column so THIRD_PARTY_NOTICES keeps MIT attribution pointed
at each fork's origin.

The tutorial tier follows the rename end to end: its yaml fences named plugins
the Loader can no longer resolve, its `ts ignore-check` fences disagreed with
the compiled fences beside them, and its prose quoted both. The contracts that
told readers to keep upstream names — the root convention and the vendoring
cookbook's tree comment and manifest invariant — now say to rescope instead.

Two rules read `@deepseek-ai/` as "another workspace plugin": the client bundle
purity gate now names the vendored libraries a browser bundle inlines, and the
files where a bare `cordis` is an agent-preset id keep that product data.
2026-08-10 22:04:13 +08:00
Turtle a27efdef36 docs: make technical prose concrete 2026-08-10 16:34:20 +08:00
Turtle dda02250f5 docs: reserve seam for complete capabilities 2026-08-09 17:26:57 +08:00
Turtle 9704749b01 docs: replace vague provenance prose with recorded facts 2026-08-09 15:35:02 +08:00
imccyu ed8431cdd0 fix: projection error 2026-08-06 19:36:15 +08:00
imccyu 297edd512d fix(token-meter): drop the dead deep-compare from claim bookkeeping
A defined fold claim is always freshly built and adjacent metering events
cannot repeat, so presence alone decides the claim transition; the value
comparison was unreachable and failed the per-file branch gate.
2026-08-06 03:20:34 +08:00
imccyu 59df683ef1 fix(token-meter): bound projection state via logged shadow prices
The contextBreakdown and contextPressure units carried the full priced
surface, so each session's persisted projection checkpoint grew without
bound. A surface replacement is now priced by the shadow-price event
logged directly before it — compact/summary for compaction, the new
compact/prune from tool-result pruning (priced through the injected
token meter) — and the unit states shrink to a fixed handful of numbers.
Regenerate the persistence/cordis/module/config catalogs.
2026-08-06 02:50:38 +08:00
imccyu a17eebcaca docs(metrics): clarify projection and latency contracts 2026-08-06 00:22:09 +08:00
Yichen Jiang e43b08d7b8 Merge remote-tracking branch 'origin/master' into feat/add-session-data-preview
# Conflicts:
#	docs/core-data-structures/session.i18n.yaml
#	packages/client/ui-conversation/README.i18n.yaml
#	packages/client/ui-conversation/README.md
#	packages/llm/token-meter/README.i18n.yaml
#	packages/llm/token-meter/README.md
#	packages/llm/token-meter/README.zh.md
#	packages/llm/token-meter/src/projection.ts
2026-08-05 17:11:16 +08:00
Yichen Jiang 038699bcb4 fix(web): let the context meter see a compaction
The composer ring, percentage, and `~used / capacity` header read
`contextPressure.pressureTokens`, which moves only when a request reports
usage. Compaction reports none — compact-basic summarizes through a direct
`ctx.llm.stream()` call and appends only its own `compact/*` records plus the
replacement `user/message` — so the meter was frozen across the one action
taken to change it. Driving a real `compactNow` through the agent loop:

    BEFORE compact:  ring=4%  header=~4227/100000  rows=[18, 0, 4365]
    AFTER  compact:  ring=4%  header=~4227/100000  rows=[18, 0,  286]

The composition rows fell 93%; the ring did not move, and would not until an
entire further turn completed. The panel then contradicted itself by more than
an order of magnitude at exactly the moment a reader opens it.

`contextPressure` now also publishes `projectedTokens`: the provider sample
plus the heuristic repricing of everything the surface gained or lost since
that sample, clamped at zero, folded through the shared `surface-fold.ts`. The
sample is stamped before the same event joins the surface, so an
`assistant/message` anchors against the surface its own request carried. Only
the delta is estimated, so the figure stays provider-anchored — the estimator's
CJK and JSON-schema underpricing stays out of the occupancy number — while
reacting the moment content lands or a span is shadowed. Same run after:

    BEFORE compact:  ring=4%  header=~4323/100000  (pressure=4227, projected=4323)
    AFTER  compact:  ring=0%  header=~ 244/100000  (pressure=4227, projected= 244)

`contextOccupancy` prefers the projected figure and falls back to the bare
sample, so a projection restored from a pre-field checkpoint degrades to the
old behavior rather than disappearing. `stateVersion` moves to 3.
2026-08-05 17:00:48 +08:00
Yichen Jiang e62cbe12e4 refactor(token-meter): fold the surface once for the meter and the projection
`contextBreakdown.messageTokens` and `measure().surfaceTokens` answer the same
question in the same heuristic vocabulary, and the panel's composition rows are
only honest while they agree. Each owner carried its own copy of the positional
fold — same pricing, same `{seq, tokens}` node list, same replace-range lookup
and guard, differing only in mutable versus immutable application — so an edit
to either one would have moved the panel away from `measure()` with both sides
still green. The duplication gate caught the shared 62 tokens.

`src/surface-fold.ts` now owns `foldSurfaceTokens`: total, allocation-fresh,
returning the event's price, the next surface, and the signed total delta. The
service assigns that result where it used to prepare a commit closure, which
keeps its validate-before-mutate replay transaction intact — the fold throws
before any state is touched, so a malformed event still fails identically on
every retry. `_prepareSurfaceMutation` and `_estimateSurfaceEvent` go away with
it, and the projection's apply drops to one call.

Covers the identity with a session that appends and then compacts, asserting
the projection figure equals the service surface at each boundary; the test
fails when either side of the fold is perturbed.
2026-08-05 16:29:51 +08:00
Turtle 8af3babaea docs: rescan rebased documentation hierarchy 2026-08-05 16:16:57 +08:00
Yif 0073d6aaa1 feat(web): turn speed metrics and composer context meter
Assistant footers and the stats line gain TTFT/tok-per-second readings
folded from step timings; context occupancy moves off the stats line onto
a composer ring whose panel shows a heuristic system/tools/messages
breakdown from the new token-meter contextBreakdown session projection.
2026-08-05 13:54:46 +08:00
Hypatia May 6d58953f30 fix(token-meter): close projected usage review gaps 2026-07-30 17:22:15 +08:00
Hypatia May 8a8c1965d7 feat(web): show durable token usage and context occupancy in the stats line
The chat stats line took its token totals from the loaded conversation nodes,
so paging changed them and compaction erased the billing behind replaced
content. It also had no way to show context occupancy: the numerator and
capacity never reached the browser.

Both now come from token-meter session projections read through the standard
useProjection seat. Window nodes keep supplying turn and step counts plus LLM
and tool wall times, which are correctly window-scoped facts about what is on
screen; accounting no longer comes from there.

`tokenUsage` supplies billing and cache hit. `contextPressure` supplies
occupancy, pairing the newest provider-reported prompt size with the newest
capacity recorded by `request/context`. Deployments without token-meter drop
the token groups; a route whose adapter advertises no capacity drops the
occupancy group rather than rendering a placeholder.

Occupancy is deliberately approximate: the numerator and capacity are
independent last-wins fields, not one atomic request observation, so switching
models pairs a fresh capacity with the prior route's pressure until the next
request reports usage. It is a user-facing reference figure that nothing in the
harness makes decisions from, and it matches how the TUI status line has always
computed occupancy. The Agent Note and token-meter README state this as a
decision, including why the atomic alternative was implemented and rejected, so
it is not re-litigated as a defect.

Snapshot delta is one added `Context N% of 128K` segment across eight web
goldens; the preceding commit absorbed master's pre-existing golden drift.
2026-07-30 14:48:19 +08:00
Hypatia May 7f30a7e30c Merge remote-tracking branch 'origin/master' into codex/status-bar-token-metrics
Conflict resolution notes:

- StatsLine: master redesigned the row into pipe-separated groups with LLM and
  tool wall times. Kept that design and swapped only the token accounting
  source, so counts and durations stay window-scoped while billing and context
  occupancy read the durable projections.
- Generated artifacts (cordis catalog, module graph, event producer/consumer,
  i18n pairing hashes) and web snapshots took master's side; they are
  regenerated and re-recorded after this merge.
- Web e2e goldens and details-panel/timeline assertions took master's side:
  that evolution is unrelated to this branch.
- ui-conversation package.json: kept master's devDependency ordering, re-adding
  only the token-meter entry this branch needs.
2026-07-30 13:56:39 +08:00
Hypatia May 4819210142 refactor(token-meter): make context occupancy durable projection state
Replace the transient `session/model-request` mux frame with ordinary durable
session state. Occupancy now rides two last-wins projection fields instead of a
non-replayable frame that needed removal tombstones and cross-stream fencing.

The frame was the only non-replayable class on the mux stream. Because host and
mux are independent SSE streams with no cross-stream order, a request emitted
before a removal could arrive after `host/session-removed`, and a legitimate
request for a new lifecycle reusing the same id could be fenced by a late
removal. Fixing that needed a lifecycle generation on every frame; the frame
itself was the problem.

Removed: the `session/model-request` frame and schema, the `agent/model-request`
core event, the ApiProxy measurement point, the client-side telemetry map and
removal tombstone, and the synthetic `cancelled` open error used to signal
reconnect through the error channel.

Added: `request/context`, a log-only session event recording the
registration-bound capacity of the route a request resolved to, appended beside
`request/header` from the lookup that already prepared the call and skipped when
the route is unchanged. Capacity stays out of `EpochHeader` because it is
adapter metadata about a route, not an input the request was built from, so it
must not join request reconstruction or header equality.

The `contextPressure` projection pairs the newest provider-reported prompt size
with the newest recorded capacity. The two are deliberately not one atomic
request observation: switching models can pair a fresh capacity with the prior
route's pressure until the next request reports usage. The figure is a
user-facing reference, and this matches how the TUI status line has always
computed occupancy.
2026-07-30 13:53:08 +08:00
Hypatia May 840e5eaa40 fix(web): address request telemetry review 2026-07-29 16:42:36 +08:00
Hypatia May e37cb23336 feat(token-meter): project durable token usage 2026-07-29 14:35:24 +08:00
Turtle 95a995968b refactor: migrate linting to Oxlint 2026-07-29 14:32:11 +08:00
_Kerman fbf87e660c refactor: identify and freeze messages at creation 2026-07-28 13:55:59 +08:00
_Kerman 009d113e0e refactor(agent): trim obsolete loop surfaces 2026-07-24 21:58:07 +08:00
Tianyi Cui 72c6696e4f Merge branch 'master' into worktree/routed-model-compaction-policy
# Conflicts:
#	docs/event-producer-consumer.md
2026-07-21 19:23:54 +08:00
Yichen Jiang 49a9bca9f6 fix: validate compaction config pairs 2026-07-21 14:49:57 +08:00
Tianyi Cui 1145ee5fc3 fix(invariants): assert runtime relationships, not API shapes 2026-07-20 19:34:19 +08:00
Yichen Jiang cfa180c127 Resolve compaction policy per routed model 2026-07-20 15:34:00 +08:00
Tianyi Cui 941b0411d8 feat(invariants): implement package runtime checks 2026-07-20 00:38:37 +08:00
Tianyi Cui 433670a754 feat(invariants): require package-owned companions 2026-07-19 22:13:50 +08:00
Tianyi Cui 06ea8643ff Merge newest origin/master into token-meter-service 2026-07-17 22:49:54 +08:00
Hypatia May 06e0450b8d fix(token-meter): keep pressure anchors conservative 2026-07-16 15:26:30 +08:00
Hypatia May 6d96b3e4a8 refactor(token-meter): merge measurement snapshots (round 1) 2026-07-16 14:38:24 +08:00
Hypatia May 19a56ec542 fix(token-meter): reject stale config (round 2) 2026-07-16 13:23:20 +08:00
Hypatia May 5c243e8a8d refactor(token-meter): simplify singleton service (round 1) 2026-07-16 12:58:07 +08:00
Hypatia May f038780ff6 feat(llm): add replay token metering (PR2 round 1) 2026-07-15 14:47:29 +08:00