Yichen Jiang
e62cbe12e4
`contextBreakdown.messageTokens` and `measure().surfaceTokens` answer the same
question in the same heuristic vocabulary, and the panel's composition rows are
only honest while they agree. Each owner carried its own copy of the positional
fold — same pricing, same `{seq, tokens}` node list, same replace-range lookup
and guard, differing only in mutable versus immutable application — so an edit
to either one would have moved the panel away from `measure()` with both sides
still green. The duplication gate caught the shared 62 tokens.
`src/surface-fold.ts` now owns `foldSurfaceTokens`: total, allocation-fresh,
returning the event's price, the next surface, and the signed total delta. The
service assigns that result where it used to prepare a commit closure, which
keeps its validate-before-mutate replay transaction intact — the fold throws
before any state is touched, so a malformed event still fails identically on
every retry. `_prepareSurfaceMutation` and `_estimateSurfaceEvent` go away with
it, and the projection's apply drops to one call.
Covers the identity with a session that appends and then compacts, asserting
the projection figure equals the service surface at each boundary; the test
fails when either side of the fold is perturbed.
63 lines
2.5 KiB
TypeScript
63 lines
2.5 KiB
TypeScript
/**
|
|
* Pure fold for the heuristic context-composition projection: system prompt
|
|
* and tool schemas from the newest request envelope, conversation from the
|
|
* live surface. Prices with the same shared estimator as the meter service,
|
|
* so the three figures match `measure()`'s heuristic vocabulary exactly.
|
|
*/
|
|
|
|
import { z } from 'zod'
|
|
import { canonicalHeader, isSurfaceEvent } from '@deepseek-ai/dsh-session'
|
|
import type { ProjectionDefinition } from '@deepseek-ai/dsh-session-projection'
|
|
import type { TokenSurfaceNode } from './types.ts'
|
|
import { estimateSystemTokens, estimateToolsTokens } from './estimate.ts'
|
|
import { foldSurfaceTokens } from './surface-fold.ts'
|
|
// Import for the `contextBreakdown` SessionProjectionMap key merge.
|
|
import type {} from './projection.ts'
|
|
|
|
interface ContextBreakdownState {
|
|
systemTokens: number
|
|
toolsTokens: number
|
|
messageTokens: number
|
|
/** Priced surface nodes (plain JSON for the persisted projection cache). */
|
|
surface: TokenSurfaceNode[]
|
|
}
|
|
|
|
const breakdownSchema = z.object({
|
|
systemTokens: z.number().int().nonnegative(),
|
|
toolsTokens: z.number().int().nonnegative(),
|
|
messageTokens: z.number().int().nonnegative(),
|
|
}).strict()
|
|
|
|
/**
|
|
* Token-meter's context-composition projection unit.
|
|
*
|
|
* Envelope figures are last-wins per `request/header`; the message figure
|
|
* rides {@link foldSurfaceTokens} — the same fold the measurement service
|
|
* replays — so it equals `measure().surfaceTokens` at every event boundary and
|
|
* compaction shrinks it the way it shrinks the next request.
|
|
*/
|
|
export const contextBreakdownProjectionDefinition:
|
|
ProjectionDefinition<'contextBreakdown', ContextBreakdownState> = {
|
|
key: 'contextBreakdown',
|
|
schema: breakdownSchema,
|
|
init: () => ({ systemTokens: 0, toolsTokens: 0, messageTokens: 0, surface: [] }),
|
|
apply: (state, event) => {
|
|
if (event.type === 'request/header') {
|
|
const header = canonicalHeader(event.data.header)
|
|
const systemTokens = estimateSystemTokens(header)
|
|
const toolsTokens = estimateToolsTokens(header)
|
|
if (systemTokens === state.systemTokens && toolsTokens === state.toolsTokens) return state
|
|
return { ...state, systemTokens, toolsTokens }
|
|
}
|
|
if (!isSurfaceEvent(event)) return state
|
|
const fold = foldSurfaceTokens(state.surface, event)
|
|
return {
|
|
...state,
|
|
messageTokens: state.messageTokens + fold.deltaTokens,
|
|
surface: fold.nodes,
|
|
}
|
|
},
|
|
view: ({ systemTokens, toolsTokens, messageTokens }) => ({ systemTokens, toolsTokens, messageTokens }),
|
|
stateVersion: 1,
|
|
}
|