docs: fix notices template claim and close generator omission paths
The template still described the removed doc-sync gate; it now states the pre-commit + test-lane mechanism that actually runs. Read the nested native/landlock-run/packages manifests, accept PEP 508 requirements with no version or with a marker, and reject a vendor/README.md table that stops covering a vendored directory instead of dropping it silently. Extend the pre-commit glob to the generator and the build-time pin source; record the deletion trigger gap the test lane backstops.
This commit is contained in:
+2
-2
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/process/2026-07-30-generated-third-party-notices.md
|
||||
2026-07-30-generated-third-party-notices.md: 2a669be4c8dcf92beb08c1305098e1baa7626ef4
|
||||
2026-07-30-generated-third-party-notices.zh.md: 990d15238a2939c54b4242016f9986e3f58a260b
|
||||
2026-07-30-generated-third-party-notices.md: 53d75d1464b4ddd3cab0eeaa291dc6d41da10503
|
||||
2026-07-30-generated-third-party-notices.zh.md: b55982d94a6d9d9f9b5f6fb98a36ea1471f5eb6c
|
||||
@@ -14,7 +14,9 @@ A hand-written inventory answers none of those durably. Roughly a hundred rows o
|
||||
|
||||
[`THIRD_PARTY_NOTICES.md`](../../../../THIRD_PARTY_NOTICES.md) is generated by [`scripts/gen-third-party-notices.ts`](../../../../scripts/gen-third-party-notices.ts) from the workspace manifests, `vendor/README.md`, the `pyproject.toml` files, and `pnpm-workspace.yaml`. The root README pair links the file from its License section.
|
||||
|
||||
**Freshness is maintained, not merely enforced.** A pre-commit job regenerates the file and stages it whenever a manifest, lock file, `vendor/README.md`, or `pyproject.toml` is staged, so an unrelated dependency edit never has to come back and rerun a generator. The committed bytes are then asserted inside [`scripts/gen-third-party-notices.spec.ts`](../../../../scripts/gen-third-party-notices.spec.ts), which the test lane already runs — the check adds no gate process, no scheduler slot, and no separate CI step. `pnpm run verify-third-party-notices` remains available for a standalone check.
|
||||
**Freshness is maintained, not merely enforced.** A pre-commit job regenerates the file and stages it whenever a generator input is staged — any manifest, either lock file, `vendor/README.md`, a `pyproject.toml`, the generator itself, or the script holding the build-time pin — so an unrelated dependency edit never has to come back and rerun a generator. The committed bytes are then asserted inside [`scripts/gen-third-party-notices.spec.ts`](../../../../scripts/gen-third-party-notices.spec.ts), which the test lane already runs — the check adds no gate process, no scheduler slot, and no separate CI step. `pnpm run verify-third-party-notices` remains available for a standalone check.
|
||||
|
||||
One trigger gap is accepted rather than worked around: lefthook inspects only files present on disk, so **deleting** a manifest runs no job, and removing a package reaches the assertion in the test lane instead. Reconstructing the staged file list to include deletions was tried and does not work — lefthook filters the list against the working tree either way. The assertion is the backstop for exactly this case.
|
||||
|
||||
The file discloses **direct** dependencies only. The complete npm closure with pinned versions already lives in `pnpm-lock.yaml` (`pnpm licenses list` renders it) and the Python closure in `python/sdk/uv.lock`; re-materializing either as prose would be a second, worse copy.
|
||||
|
||||
|
||||
@@ -14,7 +14,9 @@ Status: implemented
|
||||
|
||||
[`THIRD_PARTY_NOTICES.md`](../../../../THIRD_PARTY_NOTICES.md) 由 [`scripts/gen-third-party-notices.ts`](../../../../scripts/gen-third-party-notices.ts) 依据各工作区清单、`vendor/README.md`、`pyproject.toml` 与 `pnpm-workspace.yaml` 生成。根 README 双语两侧都从「许可证」一节链到该文件。
|
||||
|
||||
**新鲜度靠维护而非拦截。** 只要暂存了清单文件、锁文件、`vendor/README.md` 或 `pyproject.toml`,pre-commit 任务就会重新生成并一并入库,改依赖的人不必事后再折返跑一次生成器。已提交的字节随后由 [`scripts/gen-third-party-notices.spec.ts`](../../../../scripts/gen-third-party-notices.spec.ts) 断言,而测试 lane 本就会跑这个文件——这项校验不增加门禁进程、不占调度位、也不新增 CI 步骤。需要单独校验时,`pnpm run verify-third-party-notices` 仍然可用。
|
||||
**新鲜度靠维护而非拦截。** 只要暂存了生成器的任一输入——任何清单文件、两个锁文件、`vendor/README.md`、某个 `pyproject.toml`、生成器自身,或持有构建期 pin 的脚本——pre-commit 任务就会重新生成并一并入库,改依赖的人不必事后再折返跑一次生成器。已提交的字节随后由 [`scripts/gen-third-party-notices.spec.ts`](../../../../scripts/gen-third-party-notices.spec.ts) 断言,而测试 lane 本就会跑这个文件——这项校验不增加门禁进程、不占调度位、也不新增 CI 步骤。需要单独校验时,`pnpm run verify-third-party-notices` 仍然可用。
|
||||
|
||||
有一处触发缺口是接受而非绕过的:lefthook 只检视磁盘上存在的文件,因此**删除**清单文件不会触发任何任务,移除一个包会落到测试 lane 的断言上。重构暂存文件列表以纳入删除的做法试过,不成立——无论怎么给列表,lefthook 都会拿工作树过滤一遍。这个场景正由断言兜底。
|
||||
|
||||
文件只披露**直接**依赖。完整的 npm 闭包连同锁定版本已记录在 `pnpm-lock.yaml`(`pnpm licenses list` 可渲染),Python 闭包记录在 `python/sdk/uv.lock`;再用散文誊一遍只会得到一份更差的副本。
|
||||
|
||||
|
||||
@@ -5,7 +5,9 @@
|
||||
|
||||
DeepSeek Harness is licensed under [BSD 3-Clause](LICENSE). It depends on the third-party open-source software listed below. Each project remains under its own license; nothing in this file changes those terms.
|
||||
|
||||
This file lists **direct** dependencies declared by the workspace, generated from the workspace manifests by `scripts/gen-third-party-notices.ts` and verified fresh by `pnpm run verify-third-party-notices` (part of `doc-sync`). The complete npm transitive closure, with exact pinned versions, is recorded in [`pnpm-lock.yaml`](pnpm-lock.yaml) (inspect it with `pnpm licenses list`); the Python closure is recorded in [`python/sdk/uv.lock`](python/sdk/uv.lock).
|
||||
This file lists **direct** dependencies declared by the workspace. It is generated from the workspace manifests by `scripts/gen-third-party-notices.ts`: a pre-commit hook regenerates it whenever a manifest changes, and `scripts/gen-third-party-notices.spec.ts` asserts in the test lane that the committed bytes match. Run `pnpm run verify-third-party-notices` for the standalone check.
|
||||
|
||||
The complete npm transitive closure, with exact pinned versions, is recorded in [`pnpm-lock.yaml`](pnpm-lock.yaml) — inspect it with `pnpm licenses list`. The Python closure is recorded in [`python/sdk/uv.lock`](python/sdk/uv.lock), and the Landlock launcher workspace keeps its own in [`native/landlock-run/pnpm-lock.yaml`](native/landlock-run/pnpm-lock.yaml).
|
||||
|
||||
## Vendored source (`vendor/`)
|
||||
|
||||
|
||||
+6
-2
@@ -19,9 +19,13 @@ pre-commit:
|
||||
stage_fixed: true
|
||||
|
||||
# Regenerate rather than reject: a dependency edit that forgot the notices
|
||||
# would otherwise fail the test lane long after the commit.
|
||||
# would otherwise fail the test lane long after the commit. The glob matches
|
||||
# every input the generator reads, including the generator itself and the
|
||||
# build-time pin source. Deleting a manifest cannot trigger this job —
|
||||
# lefthook only inspects files present on disk — so that one case still
|
||||
# falls through to the freshness assertion in the test lane.
|
||||
- name: third-party notices (staged)
|
||||
glob: '{package.json,*/package.json,*/*/package.json,*/*/*/package.json,pnpm-workspace.yaml,pnpm-lock.yaml,vendor/README.md,python/*/pyproject.toml}'
|
||||
glob: '{package.json,*/package.json,*/*/package.json,*/*/*/package.json,pnpm-workspace.yaml,pnpm-lock.yaml,vendor/README.md,python/*/pyproject.toml,scripts/gen-third-party-notices.ts,scripts/build-exe-for-python-sdk.ts}'
|
||||
run: node_modules/.bin/tsx scripts/gen-third-party-notices.ts && git add THIRD_PARTY_NOTICES.md
|
||||
|
||||
- name: whitespace (staged)
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import { readFileSync } from 'node:fs'
|
||||
import { readdirSync, readFileSync } from 'node:fs'
|
||||
import { resolve } from 'node:path'
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { type Manifest, parseVendoredRows, render, tierExternalDeps } from './gen-third-party-notices.ts'
|
||||
import { type Manifest, parsePythonRequirements, parseVendoredRows, render, tierExternalDeps } from './gen-third-party-notices.ts'
|
||||
|
||||
const root = resolve(import.meta.dirname, '..')
|
||||
|
||||
@@ -76,4 +76,27 @@ describe('parseVendoredRows', () => {
|
||||
it('yields nothing when the table shape changes, so the generator fails loud', () => {
|
||||
expect(parseVendoredRows('| `cordis/` | cordis | 4.0.0 | https://example.com | `abc123` |\n')).toEqual([])
|
||||
})
|
||||
|
||||
it('covers every vendored directory, so no package can drop out of the notices', () => {
|
||||
const parsed = new Set(parseVendoredRows(readFileSync(resolve(root, 'vendor/README.md'), 'utf8')).map(row => row.npmName))
|
||||
const onDisk = readdirSync(resolve(root, 'vendor'), { withFileTypes: true })
|
||||
.filter(entry => entry.isDirectory())
|
||||
.map(entry => (JSON.parse(readFileSync(resolve(root, 'vendor', entry.name, 'package.json'), 'utf8')) as Manifest).name)
|
||||
|
||||
expect([...onDisk].sort()).toEqual([...parsed].sort())
|
||||
})
|
||||
})
|
||||
|
||||
describe('parsePythonRequirements', () => {
|
||||
it('reads names whether or not the requirement carries a version, extras, or a marker', () => {
|
||||
expect(parsePythonRequirements('"pydantic>=2.12", "requests", "httpx[http2]", "tomli ; python_version < \'3.11\'", "hatchling >= 1.24.0"'))
|
||||
.toEqual(['pydantic', 'requests', 'httpx', 'tomli', 'hatchling'])
|
||||
})
|
||||
|
||||
it('reads the committed manifests', () => {
|
||||
const text = readFileSync(resolve(root, 'python/sdk/pyproject.toml'), 'utf8')
|
||||
const block = /dependencies\s*=\s*\[([^\]]*)\]/.exec(text)?.[1] ?? ''
|
||||
|
||||
expect(parsePythonRequirements(block)).toContain('pydantic')
|
||||
})
|
||||
})
|
||||
@@ -41,7 +41,11 @@ const DEV_ONLY_AREAS = [
|
||||
* First-party packages released from sibling repositories under the project's
|
||||
* own license: reachable from workspace manifests but not third-party.
|
||||
*/
|
||||
const FIRST_PARTY = new Set(['node-addon-landlock-run'])
|
||||
const FIRST_PARTY = new Set([
|
||||
'node-addon-landlock-run',
|
||||
'node-addon-landlock-run-linux-arm64',
|
||||
'node-addon-landlock-run-linux-x64',
|
||||
])
|
||||
|
||||
/**
|
||||
* Metadata overrides where the installed manifest is wrong or unreachable.
|
||||
@@ -108,7 +112,9 @@ function readManifest(rel: string): Manifest {
|
||||
|
||||
/** Every workspace manifest, keyed by path, plus the set of workspace package names. */
|
||||
function loadWorkspaceManifests(): { manifests: Map<string, Manifest>; names: Set<string> } {
|
||||
const patterns = ['package.json', 'vendor/*/package.json', 'packages/*/*/package.json', 'apps/*/package.json', 'website/package.json', 'examples/package.json', 'python/sdk-runtime/package.json', 'native/landlock-run/package.json', 'native/landlock-run/*/package.json']
|
||||
// `native/landlock-run` is a nested workspace with its own lock file; its
|
||||
// leaf manifests live one level deeper than this repository's own tiers.
|
||||
const patterns = ['package.json', 'vendor/*/package.json', 'packages/*/*/package.json', 'apps/*/package.json', 'website/package.json', 'examples/package.json', 'python/sdk-runtime/package.json', 'native/landlock-run/package.json', 'native/landlock-run/packages/*/package.json']
|
||||
const manifests = new Map<string, Manifest>()
|
||||
const names = new Set<string>()
|
||||
for (const pattern of patterns) {
|
||||
@@ -219,27 +225,51 @@ export function parseVendoredRows(text: string): VendoredRow[] {
|
||||
return rows
|
||||
}
|
||||
|
||||
/** Parse the vendored manifest table and confirm every vendored package is MIT. */
|
||||
/**
|
||||
* Parse the vendored manifest table and confirm it accounts for every vendored
|
||||
* directory. The `vendor/` tree — not the table — is the set that must be
|
||||
* disclosed, so a row that stops matching the table format is a hard error
|
||||
* rather than a package that quietly vanishes from the notices.
|
||||
*/
|
||||
function collectVendored(): VendoredRow[] {
|
||||
const rows = parseVendoredRows(readFileSync(resolve(root, 'vendor/README.md'), 'utf8'))
|
||||
if (rows.length === 0) throw new Error('gen-third-party-notices: no vendored rows parsed from vendor/README.md; its table format changed.')
|
||||
const onDisk = new Map<string, string>()
|
||||
for (const entry of readdirSync(resolve(root, 'vendor'), { withFileTypes: true })) {
|
||||
if (!entry.isDirectory()) continue
|
||||
const manifest = readManifest(`vendor/${entry.name}/package.json`)
|
||||
if (manifest.name !== undefined) onDisk.set(manifest.name, entry.name)
|
||||
}
|
||||
|
||||
const parsed = new Set(rows.map(row => row.npmName))
|
||||
const missing = [...onDisk.keys()].filter(name => !parsed.has(name))
|
||||
if (missing.length > 0) {
|
||||
throw new Error(`gen-third-party-notices: vendor/README.md has no manifest-table row for ${missing.join(', ')}; its table format changed or the sync is incomplete.`)
|
||||
}
|
||||
for (const row of rows) {
|
||||
const manifest = readManifest(`vendor/${vendorDir(row.npmName)}/package.json`)
|
||||
if (manifest.license !== 'MIT') {
|
||||
throw new Error(`gen-third-party-notices: vendored ${row.npmName} declares license ${JSON.stringify(manifest.license)}; the vendored section assumes MIT throughout.`)
|
||||
const dir = onDisk.get(row.npmName)
|
||||
if (dir === undefined) throw new Error(`gen-third-party-notices: vendored package ${row.npmName} from vendor/README.md has no vendor/ directory.`)
|
||||
const license = readManifest(`vendor/${dir}/package.json`).license
|
||||
if (license !== 'MIT') {
|
||||
throw new Error(`gen-third-party-notices: vendored ${row.npmName} declares license ${JSON.stringify(license)}; the vendored section assumes MIT throughout.`)
|
||||
}
|
||||
}
|
||||
return rows
|
||||
}
|
||||
|
||||
/** The vendor/ directory of a vendored npm name (manifest table order is authoritative for names). */
|
||||
function vendorDir(npmName: string): string {
|
||||
const dirs = readdirSync(resolve(root, 'vendor'), { withFileTypes: true }).filter(entry => entry.isDirectory()).map(entry => entry.name)
|
||||
for (const dir of dirs) {
|
||||
const manifest = readManifest(`vendor/${dir}/package.json`)
|
||||
if (manifest.name === npmName) return dir
|
||||
/**
|
||||
* Extract the distribution names from one `pyproject.toml` requirement array.
|
||||
* PEP 508 makes every part after the name optional, so a bare `"requests"` and
|
||||
* a marker-only `"requests; python_version < '3.11'"` must both be found.
|
||||
* @param block - the bracketed array text of a requirement list.
|
||||
* @returns each requirement's distribution name, in file order.
|
||||
*/
|
||||
export function parsePythonRequirements(block: string): string[] {
|
||||
const names: string[] = []
|
||||
for (const match of block.matchAll(/"\s*([a-zA-Z][a-zA-Z0-9._-]*)\s*(?:\[[^\]]*\])?\s*(?:[<>=!~;@].*?)?"/g)) {
|
||||
const name = match[1]
|
||||
if (name !== undefined) names.push(name)
|
||||
}
|
||||
throw new Error(`gen-third-party-notices: vendored package ${npmName} from vendor/README.md has no vendor/ directory.`)
|
||||
return names
|
||||
}
|
||||
|
||||
/** Direct Python dependencies named by the `pyproject.toml` manifests under `python/`. */
|
||||
@@ -247,10 +277,15 @@ function collectPython(): { name: string; license: string; repo: string; role: s
|
||||
const found = new Set<string>()
|
||||
for (const path of ['python/sdk/pyproject.toml', 'python/sdk-runtime/pyproject.toml']) {
|
||||
const text = readFileSync(resolve(root, path), 'utf8')
|
||||
for (const match of text.matchAll(/"([a-zA-Z][a-zA-Z0-9._-]*)\s*(?:>=|==|~=|<|>|\[)/g)) {
|
||||
const name = match[1]
|
||||
if (name === undefined || name.startsWith('deepseek')) continue
|
||||
found.add(name)
|
||||
// Requirement arrays only: `[project] name`/`readme` and `[tool.*]` string
|
||||
// values would otherwise read as dependencies.
|
||||
for (const block of text.matchAll(/(?:^|\n)\s*(?:requires|dependencies|test|dev|lint)\s*=\s*\[([^\]]*)\]/g)) {
|
||||
const body = block[1]
|
||||
if (body === undefined) continue
|
||||
for (const name of parsePythonRequirements(body)) {
|
||||
if (name.startsWith('deepseek')) continue
|
||||
found.add(name)
|
||||
}
|
||||
}
|
||||
}
|
||||
return [...found].sort((a, b) => a.localeCompare(b)).map((name) => {
|
||||
@@ -306,7 +341,9 @@ export function render(): string {
|
||||
|
||||
DeepSeek Harness is licensed under [BSD 3-Clause](LICENSE). It depends on the third-party open-source software listed below. Each project remains under its own license; nothing in this file changes those terms.
|
||||
|
||||
This file lists **direct** dependencies declared by the workspace, generated from the workspace manifests by \`scripts/gen-third-party-notices.ts\` and verified fresh by \`pnpm run verify-third-party-notices\` (part of \`doc-sync\`). The complete npm transitive closure, with exact pinned versions, is recorded in [\`pnpm-lock.yaml\`](pnpm-lock.yaml) (inspect it with \`pnpm licenses list\`); the Python closure is recorded in [\`python/sdk/uv.lock\`](python/sdk/uv.lock).
|
||||
This file lists **direct** dependencies declared by the workspace. It is generated from the workspace manifests by \`scripts/gen-third-party-notices.ts\`: a pre-commit hook regenerates it whenever a manifest changes, and \`scripts/gen-third-party-notices.spec.ts\` asserts in the test lane that the committed bytes match. Run \`pnpm run verify-third-party-notices\` for the standalone check.
|
||||
|
||||
The complete npm transitive closure, with exact pinned versions, is recorded in [\`pnpm-lock.yaml\`](pnpm-lock.yaml) — inspect it with \`pnpm licenses list\`. The Python closure is recorded in [\`python/sdk/uv.lock\`](python/sdk/uv.lock), and the Landlock launcher workspace keeps its own in [\`native/landlock-run/pnpm-lock.yaml\`](native/landlock-run/pnpm-lock.yaml).
|
||||
|
||||
## Vendored source (\`vendor/\`)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user