2.5 KiB
ctx.llm
LlmService — provided by @deepseek-ai/dsh-llm.
The abstract llm service: an adapter registry plus a streaming model-call surface, interceptable via the llm/stream waterfall.
ctx.llm.registerAdapter(providers, adapter)
registerAdapter(providers: string[], adapter: LlmAdapter): () => void
Register an adapter for the given provider routes. Throws LlmError with code DUPLICATE_ADAPTER if any provider already has an adapter (all-or-nothing). Disposed with the fiber.
providers— every provider route this adapter should serve.adapter— the adapter that streams calls for those providers.
Returns the disposer that unregisters all of them.
ctx.llm.listProviders()
listProviders(): LlmProviderInfo[]
Describe provider routes with a registered adapter.
Returns detached provider metadata in registration order.
ctx.llm.listModels(provider)
async listModels(provider: string): Promise<LlmModelInfo[]>
Discover models advertised by one registered provider. Catalog membership is advisory and never changes routing or request validation.
provider— registered provider route to inspect.
Returns detached model metadata in adapter-preferred order.
ctx.llm.stream(options)
stream(options: GenerateOptions): AsyncIterable<StreamChunk>
Stream one model call as raw chunks (token-level deltas). Throws LlmError with code NO_ADAPTER if no adapter is registered for options.provider. Replay state is retained only when the same adapter instance owns its historical provider and the target provider. Dispatches through the llm/stream waterfall.
options— the full request;options.providerselects the adapter.
Returns the chunk stream, possibly wrapped by llm/stream listeners.