diff --git a/apps/cli/config/base.cordis.yml b/apps/cli/config/base.cordis.yml index 5e0940d241..041e7939bc 100644 --- a/apps/cli/config/base.cordis.yml +++ b/apps/cli/config/base.cordis.yml @@ -105,7 +105,9 @@ # DSH_TELEMETRY_OTLP_URL overrides the production endpoint, and a non-empty # DSH_TELEMETRY_DISABLED — any value, including '0'/'false' — opts the # process out (the launchers patch the row disabled; config cannot disable -# a row). The exporter/processor values bound the shutdown drain to ~1s +# a row). Exports carry the harness home's anonymous user id ($DSH_HOME/.userid, +# random UUID; delete the file to reset the identity) as the Resource's +# user.id. The exporter/processor values bound the shutdown drain to ~1s # against an unreachable collector: exporter.timeoutMillis is both the # per-attempt socket timeout and the retry deadline (1s effectively # disables the SDK's 5-try backoff), maxExportBatchSize == maxQueueSize diff --git a/docs/config-catalog.md b/docs/config-catalog.md index f3d855bcd5..a918750790 100644 --- a/docs/config-catalog.md +++ b/docs/config-catalog.md @@ -1229,7 +1229,7 @@ export interface Config { Depends on: `BatchLogRecordProcessorOptions` (`@opentelemetry/sdk-logs`) · `OTLPExporterNodeConfigBase` (`@opentelemetry/otlp-exporter-base`) -Source: [`packages/telemetry/session-telemetry-otel/src/index.ts:40`](../packages/telemetry/session-telemetry-otel/src/index.ts) +Source: [`packages/telemetry/session-telemetry-otel/src/index.ts:41`](../packages/telemetry/session-telemetry-otel/src/index.ts) ## `@deepseek-ai/dsh-session-title` diff --git a/packages/telemetry/session-telemetry-otel/README.i18n.yaml b/packages/telemetry/session-telemetry-otel/README.i18n.yaml index b1a2052a3f..db61f28a67 100644 --- a/packages/telemetry/session-telemetry-otel/README.i18n.yaml +++ b/packages/telemetry/session-telemetry-otel/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/telemetry/session-telemetry-otel/README.md -README.md: 9b208e291e77bee50d9d4fd14808268dca75f2db -README.zh.md: 76de1bf1ad58a0239907f3b63c672177874c7966 +README.md: ad4a97868c28dc3873c839490aa506271459e249 +README.zh.md: f1ad73ddf66aacc30a9024a9290df2c44686efe9 diff --git a/packages/telemetry/session-telemetry-otel/README.md b/packages/telemetry/session-telemetry-otel/README.md index 9b208e291e..ad4a97868c 100644 --- a/packages/telemetry/session-telemetry-otel/README.md +++ b/packages/telemetry/session-telemetry-otel/README.md @@ -2,7 +2,7 @@ English | [中文](README.zh.md) -The OpenTelemetry backend for [the telemetry seam](../session-telemetry/) — the only entry a deployment loads. It composes the OTel JS SDK as-is (`LoggerProvider` → `BatchLogRecordProcessor` → OTLP/HTTP log exporter) and maps each record the seam hands over onto `logger.emit()`, under two instrumentation scopes: ledger records on `@deepseek-ai/dsh-session-telemetry-otel`, operational records on `@deepseek-ai/dsh-session-telemetry-otel/ops`. Resource identity (`service.name`/`service.version`) comes from `dsh-llm`'s `APP_IDENTITY`, the same source the attribution headers use. +The OpenTelemetry backend for [the telemetry seam](../session-telemetry/) — the only entry a deployment loads. It composes the OTel JS SDK as-is (`LoggerProvider` → `BatchLogRecordProcessor` → OTLP/HTTP log exporter) and maps each record the seam hands over onto `logger.emit()`, under two instrumentation scopes: ledger records on `@deepseek-ai/dsh-session-telemetry-otel`, operational records on `@deepseek-ai/dsh-session-telemetry-otel/ops`. Resource identity (`service.name`/`service.version`) comes from `dsh-llm`'s `APP_IDENTITY`, the same source the attribution headers use, plus `user.id` — the harness home's anonymous user id this package owns (`src/user-id.ts`: `$DSH_HOME/.userid`, a random UUID minted on first use; deleting the file resets the identity), carried once per export batch on the Resource rather than per record. ## Config diff --git a/packages/telemetry/session-telemetry-otel/README.zh.md b/packages/telemetry/session-telemetry-otel/README.zh.md index 76de1bf1ad..f1ad73ddf6 100644 --- a/packages/telemetry/session-telemetry-otel/README.zh.md +++ b/packages/telemetry/session-telemetry-otel/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -[遥测（telemetry）seam](../session-telemetry/) 的 OpenTelemetry 后端，也是部署方唯一要加载的条目。它原样组合 OTel JS SDK（`LoggerProvider` → `BatchLogRecordProcessor` → OTLP/HTTP 日志导出器），把 seam 交接过来的每条记录映射到 `logger.emit()`，并使用两个插桩作用域（instrumentation scope）：ledger 记录挂在 `@deepseek-ai/dsh-session-telemetry-otel` 下，运维记录挂在 `@deepseek-ai/dsh-session-telemetry-otel/ops` 下。资源身份（`service.name`/`service.version`）来自 `dsh-llm` 的 `APP_IDENTITY`，与归因标头同源。 +[遥测（telemetry）seam](../session-telemetry/) 的 OpenTelemetry 后端，也是部署方唯一要加载的条目。它原样组合 OTel JS SDK（`LoggerProvider` → `BatchLogRecordProcessor` → OTLP/HTTP 日志导出器），把 seam 交接过来的每条记录映射到 `logger.emit()`，并使用两个插桩作用域（instrumentation scope）：ledger 记录挂在 `@deepseek-ai/dsh-session-telemetry-otel` 下，运维记录挂在 `@deepseek-ai/dsh-session-telemetry-otel/ops` 下。资源身份（`service.name`/`service.version`）来自 `dsh-llm` 的 `APP_IDENTITY`，与归因标头同源；另有 `user.id`——本包自有的 harness home 匿名用户 id（`src/user-id.ts`：`$DSH_HOME/.userid`，首用生成随机 UUID；删除该文件即重置身份），随 Resource 每批导出携带一次而非逐条携带。 ## 配置 diff --git a/packages/telemetry/session-telemetry-otel/package.json b/packages/telemetry/session-telemetry-otel/package.json index 7be8c04ce4..49e9ef6cad 100644 --- a/packages/telemetry/session-telemetry-otel/package.json +++ b/packages/telemetry/session-telemetry-otel/package.json @@ -36,16 +36,20 @@ "schemastery": "^3.18.0" }, "peerDependencies": { + "@deepseek-ai/dsh-brand": "^0.0.1", "@deepseek-ai/dsh-invariants": "^0.0.1", "@deepseek-ai/dsh-llm": "^0.0.1", + "@deepseek-ai/dsh-paths": "^0.0.1", "@deepseek-ai/dsh-session": "^0.0.1", "@deepseek-ai/dsh-session-telemetry": "^0.0.1", "cordis": "^4.0.0-rc.7" }, "devDependencies": { "@cordisjs/plugin-loader": "workspace:^", + "@deepseek-ai/dsh-brand": "workspace:^", "@deepseek-ai/dsh-invariants": "workspace:^", "@deepseek-ai/dsh-llm": "workspace:^", + "@deepseek-ai/dsh-paths": "workspace:^", "@deepseek-ai/dsh-session": "workspace:^", "@deepseek-ai/dsh-session-telemetry": "workspace:^", "cordis": "^4.0.0-rc.7" diff --git a/packages/telemetry/session-telemetry-otel/src/index.ts b/packages/telemetry/session-telemetry-otel/src/index.ts index 85dd75f275..55caa1037b 100644 --- a/packages/telemetry/session-telemetry-otel/src/index.ts +++ b/packages/telemetry/session-telemetry-otel/src/index.ts @@ -17,6 +17,7 @@ import z from 'schemastery' import type { Context } from 'cordis' import { Telemetry, TelemetryCoordinator, type TelemetryRecord, type TelemetrySeverity } from '@deepseek-ai/dsh-session-telemetry' import { APP_IDENTITY } from '@deepseek-ai/dsh-llm' +import { getOrCreateAnonymousUserId } from './user-id.ts' import { BatchLogRecordProcessor, LoggerProvider, @@ -118,6 +119,10 @@ export class TelemetryOtel extends Telemetry { resource: resourceFromAttributes({ 'service.name': APP_IDENTITY.product, 'service.version': APP_IDENTITY.version, + // OTel semconv's standard user attribute, carried once per export + // batch on the Resource rather than per record: the collector + // aggregates by Resource, and the id is process-stable anyway. + 'user.id': getOrCreateAnonymousUserId(), }), processors: [ new BatchLogRecordProcessor({ diff --git a/packages/telemetry/session-telemetry-otel/src/user-id.ts b/packages/telemetry/session-telemetry-otel/src/user-id.ts new file mode 100644 index 0000000000..f06d9856fe --- /dev/null +++ b/packages/telemetry/session-telemetry-otel/src/user-id.ts @@ -0,0 +1,103 @@ +/** + * Per-harness-home anonymous user id for the OTel Resource. + * + * The id is a random UUID persisted as a bare line in `.userid` inside the + * harness home resolved by {@link resolveDshHome} (`$DSH_HOME` > `~/.dsh`), + * and never derived from the hostname, network address, git remote, or any + * other identifying source — a derived id would make "anonymous" a fiction. + * The id is scoped to the harness home, not the machine: every process + * sharing one `$DSH_HOME` reports the same id, and deleting the file simply + * mints a fresh identity on the next launch (loss is accepted by design). + * This identity belongs to the OTel feed alone; the dsh-sdk launcher + * telemetry keeps its own separate store. + * + * Reads and writes are synchronous so the backend constructor can call this + * on its boot path, and the result is memoized per resolved file path: one + * process touches the disk once, and a file deleted mid-run keeps the + * process's id until the next launch. + * + * @module @deepseek-ai/dsh-session-telemetry-otel/user-id + */ + +import { randomUUID } from 'node:crypto' +import { mkdirSync, readFileSync, writeFileSync } from 'node:fs' +import { dirname, join } from 'node:path' +import type { Branded } from '@deepseek-ai/dsh-brand' +import { resolveDshHome } from '@deepseek-ai/dsh-paths' + +/** A harness-home-scoped anonymous user id (random UUID v4). */ +export type AnonymousUserId = Branded<'AnonymousUserId'> + +/** File inside the harness home storing the id: a bare UUID line, no wrapper format. */ +export const USER_ID_FILE_NAME = '.userid' + +const UUID_PATTERN = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i + +/** Ambient seams for locating and generating the id; every field has a default. */ +export interface AnonymousUserIdOptions { + /** Environment consulted for `DSH_HOME`; defaults to `process.env`. */ + env?: NodeJS.ProcessEnv + /** UUID generator; defaults to `crypto.randomUUID` (test seam). */ + randomUUID?: () => string +} + +/** Process-lifetime memo keyed by resolved file path, so distinct test homes never share an id. */ +const memo = new Map() + +/** Read a valid persisted id from the file, or `undefined` when absent/corrupt. */ +function readPersistedId(file: string): AnonymousUserId | undefined { + let text: string + try { + text = readFileSync(file, 'utf8') + } catch { + // Absent or unreadable: the caller mints and persists a fresh id. + return undefined + } + const value = text.trim() + return UUID_PATTERN.test(value) ? (value as AnonymousUserId) : undefined +} + +/** + * Return the harness home's anonymous user id, creating and persisting one on + * first use. A concurrent first launch is settled by an exclusive-create + * write: the loser rereads the winner's id. (A reread landing in the winner's + * narrow create-to-write window can still yield two per-process ids for that + * run; the next launch converges on the persisted one.) Persistence is + * best-effort — a write failure (read-only home) still returns a usable id + * for the current run so telemetry is never blocked. + * @param options - home-location and UUID-generation seams. + * @returns the stable per-harness-home anonymous user id. + */ +export function getOrCreateAnonymousUserId(options: AnonymousUserIdOptions = {}): AnonymousUserId { + const file = join(resolveDshHome(undefined, options.env ?? process.env), USER_ID_FILE_NAME) + const cached = memo.get(file) + if (cached !== undefined) return cached + + let id = readPersistedId(file) + if (id === undefined) { + const generate = options.randomUUID ?? randomUUID + const created = generate() as AnonymousUserId + try { + mkdirSync(dirname(file), { recursive: true }) + writeFileSync(file, `${created}\n`, { encoding: 'utf8', flag: 'wx' }) + id = created + } catch { + // A wx refusal (EEXIST) covers both a concurrent winner and a + // pre-existing corrupt file: the reread adopts a valid winner, and an + // invalid reread falls through to the overwrite path. Non-EEXIST + // failures (read-only home) land there too, accepted best-effort below. + id = readPersistedId(file) + if (id === undefined) { + try { + writeFileSync(file, `${created}\n`, 'utf8') + } catch { + // Best-effort persistence: keep the fresh id in memory even when the + // home is unwritable, so this run still reports a consistent id. + } + id = created + } + } + } + memo.set(file, id) + return id +} diff --git a/packages/telemetry/session-telemetry-otel/tests/otel.spec.ts b/packages/telemetry/session-telemetry-otel/tests/otel.spec.ts index cccb90ed43..6be7f81977 100644 --- a/packages/telemetry/session-telemetry-otel/tests/otel.spec.ts +++ b/packages/telemetry/session-telemetry-otel/tests/otel.spec.ts @@ -5,11 +5,15 @@ * for the default-exported Service class. */ -import { afterEach, describe, expect, it } from 'vitest' +import { afterAll, afterEach, beforeAll, describe, expect, it } from 'vitest' import { createServer, type Server } from 'node:http' import { once } from 'node:events' +import { mkdtempSync, rmSync } from 'node:fs' +import { tmpdir } from 'node:os' +import { join } from 'node:path' import { gunzipSync } from 'node:zlib' import { Context } from 'cordis' +import { getOrCreateAnonymousUserId } from '../src/user-id.ts' import Loader from '@cordisjs/plugin-loader' import SessionStore, { SessionId } from '@deepseek-ai/dsh-session' import TelemetryOtel, { Config } from '../src/index.ts' @@ -37,6 +41,21 @@ interface OtlpLogsRequest { const servers: Server[] = [] +// The backend resolves the harness home's anonymous user id at construction; +// pin DSH_HOME to a temp dir so the suite never touches the ambient ~/.dsh. +let tempHome: string +let previousDshHome: string | undefined +beforeAll(() => { + tempHome = mkdtempSync(join(tmpdir(), 'dsh-otel-home-')) + previousDshHome = process.env.DSH_HOME + process.env.DSH_HOME = tempHome +}) +afterAll(() => { + if (previousDshHome === undefined) delete process.env.DSH_HOME + else process.env.DSH_HOME = previousDshHome + rmSync(tempHome, { recursive: true, force: true }) +}) + afterEach(async () => { for (const server of servers.splice(0)) { server.close() @@ -104,6 +123,7 @@ describe('TelemetryOtel wire', () => { const resource = first.body.resourceLogs[0]!.resource.attributes expect(resource).toContainEqual({ key: 'service.name', value: { stringValue: 'deepseek-harness' } }) + expect(resource).toContainEqual({ key: 'user.id', value: { stringValue: getOrCreateAnonymousUserId() } }) const records = allRecords(captures) const ledger = records.filter(r => r.scope === '@deepseek-ai/dsh-session-telemetry-otel') diff --git a/packages/telemetry/session-telemetry-otel/tests/user-id.spec.ts b/packages/telemetry/session-telemetry-otel/tests/user-id.spec.ts new file mode 100644 index 0000000000..f4d9505546 --- /dev/null +++ b/packages/telemetry/session-telemetry-otel/tests/user-id.spec.ts @@ -0,0 +1,107 @@ +import { chmodSync, existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { afterEach, describe, expect, it } from 'vitest' +import { + USER_ID_FILE_NAME, + getOrCreateAnonymousUserId, +} from '../src/user-id.ts' + +const dirs: string[] = [] + +function tempHome(): string { + const dir = mkdtempSync(join(tmpdir(), 'dsh-userid-')) + dirs.push(dir) + return dir +} + +afterEach(() => { + for (const dir of dirs.splice(0)) { + chmodSync(dir, 0o700) + rmSync(dir, { recursive: true, force: true }) + } +}) + +const UUID = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i + +describe('getOrCreateAnonymousUserId', () => { + it('creates, persists, and returns a bare UUID line on first use', () => { + const home = tempHome() + const id = getOrCreateAnonymousUserId({ env: { DSH_HOME: home } }) + expect(id).toMatch(UUID) + expect(readFileSync(join(home, USER_ID_FILE_NAME), 'utf8')).toBe(`${id}\n`) + }) + + it('creates the home directory when missing', () => { + const home = join(tempHome(), 'nested', 'home') + const id = getOrCreateAnonymousUserId({ env: { DSH_HOME: home } }) + expect(readFileSync(join(home, USER_ID_FILE_NAME), 'utf8')).toBe(`${id}\n`) + }) + + it('returns the persisted id on subsequent calls, tolerating surrounding whitespace', () => { + const home = tempHome() + const existing = '01234567-89ab-4cde-8f01-23456789abcd' + writeFileSync(join(home, USER_ID_FILE_NAME), ` ${existing}\n\n`, 'utf8') + expect(getOrCreateAnonymousUserId({ env: { DSH_HOME: home } })).toBe(existing) + }) + + it('overwrites a corrupt file with a fresh id', () => { + const home = tempHome() + writeFileSync(join(home, USER_ID_FILE_NAME), 'not-a-uuid\n', 'utf8') + const id = getOrCreateAnonymousUserId({ env: { DSH_HOME: home } }) + expect(id).toMatch(UUID) + expect(readFileSync(join(home, USER_ID_FILE_NAME), 'utf8')).toBe(`${id}\n`) + }) + + it('adopts a concurrent winner: exclusive create loses to an id written after the initial read', () => { + const home = tempHome() + const winner = 'aaaaaaaa-bbbb-4ccc-8ddd-eeeeeeeeeeee' + const file = join(home, USER_ID_FILE_NAME) + // The generator seam runs between the initial read (absent) and the wx + // write, so planting the winner here simulates the concurrent first launch. + const id = getOrCreateAnonymousUserId({ + env: { DSH_HOME: home }, + randomUUID: () => { + writeFileSync(file, `${winner}\n`, 'utf8') + return 'ffffffff-0000-4000-8000-000000000000' + }, + }) + expect(id).toBe(winner) + }) + + it('returns a usable id when the home is unwritable, without persisting', () => { + const home = tempHome() + const blocked = join(home, 'blocked') + mkdirSync(blocked) + chmodSync(blocked, 0o500) + const id = getOrCreateAnonymousUserId({ env: { DSH_HOME: blocked } }) + expect(id).toMatch(UUID) + expect(existsSync(join(blocked, USER_ID_FILE_NAME))).toBe(false) + }) + + it('memoizes per resolved home for the process lifetime: one read, deletion-proof', () => { + const home = tempHome() + const first = getOrCreateAnonymousUserId({ env: { DSH_HOME: home } }) + rmSync(join(home, USER_ID_FILE_NAME)) + expect(getOrCreateAnonymousUserId({ env: { DSH_HOME: home } })).toBe(first) + }) + + it('keeps distinct homes on distinct ids', () => { + const a = getOrCreateAnonymousUserId({ env: { DSH_HOME: tempHome() } }) + const b = getOrCreateAnonymousUserId({ env: { DSH_HOME: tempHome() } }) + expect(a).not.toBe(b) + }) + + it('reads process.env by default', () => { + const home = tempHome() + const previous = process.env.DSH_HOME + process.env.DSH_HOME = home + try { + const id = getOrCreateAnonymousUserId() + expect(readFileSync(join(home, USER_ID_FILE_NAME), 'utf8')).toBe(`${id}\n`) + } finally { + if (previous === undefined) delete process.env.DSH_HOME + else process.env.DSH_HOME = previous + } + }) +}) diff --git a/packages/telemetry/session-telemetry-otel/tsconfig.json b/packages/telemetry/session-telemetry-otel/tsconfig.json index 9512133cf7..8acfe15b33 100644 --- a/packages/telemetry/session-telemetry-otel/tsconfig.json +++ b/packages/telemetry/session-telemetry-otel/tsconfig.json @@ -26,6 +26,12 @@ { "path": "../session-telemetry" }, + { + "path": "../../util/brand" + }, + { + "path": "../../util/paths" + }, { "path": "../../support/invariants" } diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 96a8df6f8f..b6a697dbb7 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -5199,12 +5199,18 @@ importers: '@cordisjs/plugin-loader': specifier: workspace:^ version: link:../../../vendor/loader + '@deepseek-ai/dsh-brand': + specifier: workspace:^ + version: link:../../util/brand '@deepseek-ai/dsh-invariants': specifier: workspace:^ version: link:../../support/invariants '@deepseek-ai/dsh-llm': specifier: workspace:^ version: link:../../llm/llm + '@deepseek-ai/dsh-paths': + specifier: workspace:^ + version: link:../../util/paths '@deepseek-ai/dsh-session': specifier: workspace:^ version: link:../../core/session