diff --git a/.agents/notes/implemented/process/2026-08-10-npm-release-sequences.i18n.yaml b/.agents/notes/implemented/process/2026-08-10-npm-release-sequences.i18n.yaml index 5ba0aef406..59b51bbe6b 100644 --- a/.agents/notes/implemented/process/2026-08-10-npm-release-sequences.i18n.yaml +++ b/.agents/notes/implemented/process/2026-08-10-npm-release-sequences.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/process/2026-08-10-npm-release-sequences.md -2026-08-10-npm-release-sequences.md: e8138aef923e201cc0883232f48ce6921452ca00 -2026-08-10-npm-release-sequences.zh.md: 7ce9fa01dbe10cbdae5585f64392d0ca6a9fb868 +2026-08-10-npm-release-sequences.md: e74a4ac8f2aadd8665ec0db198c6a317a0c201bc +2026-08-10-npm-release-sequences.zh.md: e152163976f945224f2524fccd7f831ba98e8161 diff --git a/.agents/notes/implemented/process/2026-08-10-npm-release-sequences.md b/.agents/notes/implemented/process/2026-08-10-npm-release-sequences.md index e8138aef92..e74a4ac8f2 100644 --- a/.agents/notes/implemented/process/2026-08-10-npm-release-sequences.md +++ b/.agents/notes/implemented/process/2026-08-10-npm-release-sequences.md @@ -70,6 +70,10 @@ Publication runs only from GitHub Actions; there is no local publication path. P The third state catches code that changed without a version bump. The first two provide idempotence — re-running publish over one artifact republishes nothing and needs no manual selection of packages. The same rule resolves the tension between one vendor release carrying several tags and a workflow that can only run from one ref: the workflow never infers which packages to publish from the tag it ran from. +All three sequences decide this way, including the native one: it publishes through its own script rather than a shell loop, because a loop of bare `npm publish` calls cannot be retried — the registry answers a repeat of an existing version permanently, so one failure partway through left no way forward. + +Two registry behaviours shape how a publish is attempted. Writes are spaced by at least two seconds and retried with a backoff, because publishing several packages back to back outruns the registry's own processing and earns `E409 Failed to save packument`. And every retry re-reads the registry first: a reported failure can answer a write that landed anyway, so a version that now exists with this tarball's integrity counts as published rather than as a version to place again. + ### Workspace-internal references use the `workspace:` protocol Every reference to a workspace member uses `workspace:^`, so `pnpm pack` substitutes a range matching the target version: sibling `peerDependencies` follow the family version, and a reference to a vendored package follows that package's own line. The Landlock platform packages keep `workspace:*`, which publishes the exact version, because a platform package and its entry must agree exactly. diff --git a/.agents/notes/implemented/process/2026-08-10-npm-release-sequences.zh.md b/.agents/notes/implemented/process/2026-08-10-npm-release-sequences.zh.md index 7ce9fa01db..e152163976 100644 --- a/.agents/notes/implemented/process/2026-08-10-npm-release-sequences.zh.md +++ b/.agents/notes/implemented/process/2026-08-10-npm-release-sequences.zh.md @@ -70,6 +70,10 @@ tag 只是 commit 指针,不是发布成功的证明。bump 会向 registry 第三态拦住「改了代码却没 bump 版本」。前两态给出幂等——同一个 artifact 重跑 publish 不会重复发布,也不需要人工挑拣包。同一条规则还解决了「一次 vendor 发布携带多个 tag,而 workflow 只能从一个 ref 触发」的矛盾:workflow 从不从触发它的 tag 去推断该发哪些包。 +三条序列都按这套判定,native 也在内:它通过自己的脚本发布,而不是 shell 循环——一串裸 `npm publish` 无法重试,registry 对「重发已存在的版本」的回答是永久失败,因此中途失败一次就没有前路了。 + +registry 的两个行为决定了「怎么尝试一次发布」。写入之间至少间隔两秒并带退避重试,因为连续背靠背发多个包会超出 registry 自身的处理速度,换来 `E409 Failed to save packument`。而每次重试都先重查 registry:报出来的失败可能对应一次其实已经落地的写入,所以「该版本现在存在且 integrity 与本 tarball 相同」算作已发布,而不是又一个待放置的版本。 + ### workspace 内部引用走 `workspace:` 协议 所有指向 workspace 成员的引用都用 `workspace:^`,由 `pnpm pack` 替换成匹配目标版本的范围:兄弟包的 `peerDependencies` 跟随族版本,指向 vendored 包的引用跟随那个包自己的版本线。Landlock 平台包保留 `workspace:*`(发布成精确版本),因为平台包与它的入口必须版本完全一致。 diff --git a/.github/workflows/landlock-run-release.yml b/.github/workflows/landlock-run-release.yml index 8448af1ad1..76d08e9e4c 100644 --- a/.github/workflows/landlock-run-release.yml +++ b/.github/workflows/landlock-run-release.yml @@ -147,6 +147,12 @@ jobs: contents: read id-token: write steps: + # The publish script is the only repository file this job needs, and it + # imports nothing outside Node's builtins, so there is no install step. + - uses: actions/checkout@v4 + with: + persist-credentials: false + - uses: actions/setup-node@v4 with: node-version: 24 @@ -167,12 +173,8 @@ jobs: fi - name: Publish tarballs - run: | - version="${GITHUB_REF#refs/tags/landlock-run-v}" - tag_args=() - case "$version" in *-*) tag_args=(--tag next);; esac - while IFS= read -r tarball; do - # No --access: publishConfig.access in each manifest decides, and a - # command-line flag would override it. - npm publish "dist/npm/${tarball}" "${tag_args[@]}" - done < dist/npm/publish-order.txt + # Publication is decided per package against the registry, so re-running + # this job over the same artifact skips what already landed instead of + # failing on it. A bare `npm publish` loop could not be retried: the + # registry answers a repeat of an existing version permanently. + run: node ./scripts/publish-release.mjs dist/npm diff --git a/knip.json b/knip.json index e06fa48220..3017292382 100644 --- a/knip.json +++ b/knip.json @@ -9,6 +9,7 @@ "musl-gcc", "python3", "sandbox-exec", + "tar", "taskkill", "where.exe" ], diff --git a/native/landlock-run/package.json b/native/landlock-run/package.json index ab7bb611fd..be0ca9732d 100644 --- a/native/landlock-run/package.json +++ b/native/landlock-run/package.json @@ -1,6 +1,6 @@ { "name": "@deepseek-ai/node-addon-landlock-run-workspace", - "version": "0.1.0", + "version": "0.1.1", "private": true, "type": "module", "license": "BSD-3-Clause", @@ -19,6 +19,7 @@ "release:assemble-prebuilds": "node ./scripts/assemble-prebuilds.mjs", "release:verify": "node ./scripts/verify-release.mjs", "release:pack": "node ./scripts/pack-release.mjs", + "release:publish": "node ./scripts/publish-release.mjs", "release:verify-packed-install": "node ./scripts/verify-packed-install.mjs" }, "devDependencies": { diff --git a/native/landlock-run/packages/entry/package.json b/native/landlock-run/packages/entry/package.json index 8fcab55ee1..5e34cf96f0 100644 --- a/native/landlock-run/packages/entry/package.json +++ b/native/landlock-run/packages/entry/package.json @@ -1,6 +1,6 @@ { "name": "@deepseek-ai/node-addon-landlock-run", - "version": "0.1.0", + "version": "0.1.1", "type": "module", "description": "Landlock self-restrict-then-exec launcher for sandboxing subprocesses on Linux: per-platform prebuilt static binaries plus the JS seam that resolves, probes, and speaks their CLI contract", "repository": { diff --git a/native/landlock-run/packages/linux-arm64/package.json b/native/landlock-run/packages/linux-arm64/package.json index 3af83f6a19..8e6affec6a 100644 --- a/native/landlock-run/packages/linux-arm64/package.json +++ b/native/landlock-run/packages/linux-arm64/package.json @@ -1,6 +1,6 @@ { "name": "@deepseek-ai/node-addon-landlock-run-linux-arm64", - "version": "0.1.0", + "version": "0.1.1", "description": "Prebuilt landlock-run Landlock launcher binary for linux-arm64 (static musl) — resolved as a file path by @deepseek-ai/node-addon-landlock-run, never imported", "repository": { "type": "git", diff --git a/native/landlock-run/packages/linux-x64/package.json b/native/landlock-run/packages/linux-x64/package.json index 10721032b4..7e9cab7a1a 100644 --- a/native/landlock-run/packages/linux-x64/package.json +++ b/native/landlock-run/packages/linux-x64/package.json @@ -1,6 +1,6 @@ { "name": "@deepseek-ai/node-addon-landlock-run-linux-x64", - "version": "0.1.0", + "version": "0.1.1", "description": "Prebuilt landlock-run Landlock launcher binary for linux-x64 (static musl) — resolved as a file path by @deepseek-ai/node-addon-landlock-run, never imported", "repository": { "type": "git", diff --git a/native/landlock-run/scripts/publish-release.mjs b/native/landlock-run/scripts/publish-release.mjs new file mode 100644 index 0000000000..76c875b8d3 --- /dev/null +++ b/native/landlock-run/scripts/publish-release.mjs @@ -0,0 +1,165 @@ +#!/usr/bin/env node +/** + * Publish the packed launcher family from the tarballs `pack-release.mjs` + * produced, in `publish-order.txt` order. + * + * What goes out is decided per package against the registry, never from the + * order file alone: a version the registry lacks is published, a version whose + * published tarball has the same integrity is skipped, and a version whose + * published tarball differs fails the run — that last case means the content + * changed without a version bump. Skipping on identical integrity is what makes + * re-running the publish step over the same artifact safe, which matters here + * because a partial publication used to leave no way forward: republishing an + * existing version fails permanently. + * + * Usage: `node scripts/publish-release.mjs [packed dir]`. + */ + +import fs from 'node:fs'; +import path from 'node:path'; +import crypto from 'node:crypto'; +import { spawnSync } from 'node:child_process'; +import { setTimeout as sleep } from 'node:timers/promises'; +import { root } from './repo.mjs'; + +/** + * Registry codes that answer a write which did not settle, rather than a + * rejection of what was sent. `E409 Failed to save packument` is the one this + * sequence actually hits: publishing the platform packages and the entry back + * to back can outrun the registry's own processing. A rejected payload (`E403` + * over an existing version, a malformed manifest) never clears on a retry. + */ +const TRANSIENT_PUBLISH_CODES = ['E409', 'E429', 'E500', 'E502', 'E503', 'E504', 'ETIMEDOUT', 'ECONNRESET', 'EAI_AGAIN']; + +/** How many times one tarball's publish is attempted before the run fails. */ +const PUBLISH_ATTEMPTS = 4; + +/** + * Shortest gap between two publishes, and the first retry backoff. The registry + * needs a moment to commit a packument before the next write; back to back + * publishes are what produce `E409`. + */ +const PUBLISH_SPACING_MS = 2_000; + +const destination = path.resolve(process.argv.slice(2).find((arg) => !arg.startsWith('--')) || path.join(root, 'dist', 'npm')); + +/** + * @param {string} output Combined npm output. + * @returns {boolean} True when the registry reported a write it did not commit. + */ +function isTransientFailure(output) { + return TRANSIENT_PUBLISH_CODES.some((code) => output.includes(`code ${code}`)); +} + +/** + * @param {string} tarball Absolute tarball path. + * @returns {string} The `sha512-` integrity npm records for it. + */ +function integrityOf(tarball) { + return `sha512-${crypto.createHash('sha512').update(fs.readFileSync(tarball)).digest('base64')}`; +} + +/** + * @param {string} tarball Absolute tarball path. + * @returns {{name: string, version: string}} What the packed manifest declares. + */ +function packedIdentity(tarball) { + const result = spawnSync('tar', ['-xOzf', tarball, 'package/package.json'], { encoding: 'utf8' }); + if (result.status !== 0) throw new Error(`cannot read the manifest inside ${tarball}:\n${result.stderr}`); + const manifest = JSON.parse(result.stdout); + if (typeof manifest.name !== 'string' || typeof manifest.version !== 'string') { + throw new Error(`${tarball} manifest lacks name/version`); + } + return { name: manifest.name, version: manifest.version }; +} + +/** + * Ask the registry whether a version exists, and with what integrity. + * @param {string} name Package name. + * @param {string} version Package version. + * @returns {{kind: 'absent'} | {kind: 'present', integrity: string}} Registry state. + */ +function registryState(name, version) { + const result = spawnSync('npm', ['view', `${name}@${version}`, 'dist.integrity', '--json'], { encoding: 'utf8' }); + if (result.status !== 0) { + const output = `${result.stdout}${result.stderr}`; + if (output.includes('E404') || output.includes('404 Not Found')) return { kind: 'absent' }; + throw new Error(`npm view ${name}@${version} failed:\n${output}`); + } + const parsed = JSON.parse(result.stdout); + if (typeof parsed !== 'string' || parsed === '') { + throw new Error(`registry reported no dist.integrity for ${name}@${version}`); + } + return { kind: 'present', integrity: parsed }; +} + +/** + * Publish one tarball, retrying a registry write that did not settle. + * + * Every retry re-reads the registry first, because `E409` can answer a write + * that landed anyway: republishing a version that now exists fails permanently, + * so the same integrity appearing under the failed attempt counts as success. + * @param {string} tarball Absolute tarball path. + * @param {string} name Package name the tarball declares. + * @param {string} version Package version the tarball declares. + */ +async function publishTarball(tarball, name, version) { + // A prerelease version never takes the latest dist-tag. + const tagArgs = version.includes('-') ? ['--tag', 'next'] : []; + for (let tries = 1; tries <= PUBLISH_ATTEMPTS; tries += 1) { + // No --access: publishConfig.access in each manifest decides, and a + // command-line flag would override it. + const result = spawnSync('npm', ['publish', tarball, ...tagArgs], { encoding: 'utf8' }); + const output = `${result.stdout}${result.stderr}`; + if (result.status === 0) return; + + const settled = registryState(name, version); + if (settled.kind === 'present' && settled.integrity === integrityOf(tarball)) { + console.log(`landlock publish: ${name}@${version} landed despite a reported failure, continuing`); + return; + } + if (tries === PUBLISH_ATTEMPTS || !isTransientFailure(output)) { + throw new Error(`npm publish ${name}@${version} failed:\n${output}`); + } + const backoff = PUBLISH_SPACING_MS * 2 ** (tries - 1); + console.log( + `landlock publish: ${name}@${version} hit a transient registry failure` + + ` (attempt ${tries} of ${PUBLISH_ATTEMPTS}), retrying in ${backoff}ms`, + ); + await sleep(backoff); + } +} + +const order = fs + .readFileSync(path.join(destination, 'publish-order.txt'), 'utf8') + .split('\n') + .filter((line) => line !== ''); + +let published = 0; +let skipped = 0; +for (const filename of order) { + const tarball = path.join(destination, filename); + const { name, version } = packedIdentity(tarball); + const state = registryState(name, version); + if (state.kind === 'present') { + const local = integrityOf(tarball); + if (state.integrity !== local) { + throw new Error( + `${name}@${version} is already published with different content` + + `\n registry: ${state.integrity}\n packed: ${local}` + + '\nBump the version, or investigate why the build is not reproducible.', + ); + } + console.log(`landlock publish: ${name}@${version} already published, skipping`); + skipped += 1; + continue; + } + // Space out the writes: the gap belongs between publishes, so a run that only + // skips does not wait at all. + if (published > 0) await sleep(PUBLISH_SPACING_MS); + await publishTarball(tarball, name, version); + console.log(`landlock publish: ${name}@${version} published`); + published += 1; +} + +console.log(`landlock publish: ${published} published, ${skipped} already present`); diff --git a/scripts/release/publish.ts b/scripts/release/publish.ts index 2c68dc7f9f..11ad01173c 100644 --- a/scripts/release/publish.ts +++ b/scripts/release/publish.ts @@ -15,16 +15,46 @@ import { createHash } from 'node:crypto' import { readFileSync } from 'node:fs' import { join, resolve } from 'node:path' +import { setTimeout as sleep } from 'node:timers/promises' import { parseArgs } from 'node:util' import { releaseFamily } from './families.ts' -import { attempt, isEntry, run } from './process.ts' +import { attempt, isEntry } from './process.ts' import { packedIdentity, readPublishOrder } from './tarball.ts' +/** + * Registry codes that answer a write which did not settle, rather than a + * rejection of what was sent. `E409 Failed to save packument` is the one this + * sequence actually hits: publishing several packages in a row can outrun the + * registry's own processing. A rejected payload (`E403` over an existing + * version, a malformed manifest) never clears on a retry and must surface. + */ +const TRANSIENT_PUBLISH_CODES = ['E409', 'E429', 'E500', 'E502', 'E503', 'E504', 'ETIMEDOUT', 'ECONNRESET', 'EAI_AGAIN'] as const + +/** How many times one tarball's publish is attempted before the run fails. */ +const PUBLISH_ATTEMPTS = 4 + +/** + * Shortest gap between two publishes, and the first retry backoff. + * + * The registry needs a moment to commit a packument before the next write; back + * to back publishes are what produce `E409`. + */ +const PUBLISH_SPACING_MS = 2_000 + /** What the registry knows about one version. */ type RegistryState = | { readonly kind: 'absent' } | { readonly kind: 'present'; readonly integrity: string } +/** + * Whether a failed publish is worth another attempt. + * @param output - combined npm output. + * @returns True when the registry reported a write it did not commit. + */ +function isTransientFailure(output: string): boolean { + return TRANSIENT_PUBLISH_CODES.some(code => output.includes(`code ${code}`)) +} + /** * The subresource integrity string npm records for a tarball. * @param tarball - absolute tarball path. @@ -54,8 +84,47 @@ function registryState(name: string, version: string): RegistryState { return { kind: 'present', integrity: parsed } } +/** + * Publish one tarball, retrying a registry write that did not settle. + * + * Every retry re-reads the registry first, because `E409` can answer a write + * that landed anyway: republishing a version that now exists fails permanently, + * so the same integrity appearing under the failed attempt counts as success. + * @param tarball - absolute tarball path. + * @param name - package name the tarball declares. + * @param version - package version the tarball declares. + */ +async function publishTarball(tarball: string, name: string, version: string): Promise { + // A prerelease version never takes the latest dist-tag. + const tagArgs = version.includes('-') ? ['--tag', 'next'] : [] + for (let tries = 1; tries <= PUBLISH_ATTEMPTS; tries += 1) { + // No --access: the sequences do not share one access level, so a + // command-line flag could not serve both and would override the manifest + // that does. Each packed manifest decides, and + // check-workspace-constraints holds every manifest to its sequence's level. + const result = attempt('npm', ['publish', tarball, ...tagArgs]) + const output = `${result.stdout}${result.stderr}` + if (result.status === 0) return + + const settled = registryState(name, version) + if (settled.kind === 'present' && settled.integrity === integrityOf(tarball)) { + console.log(`release publish: ${name}@${version} landed despite a reported failure, continuing`) + return + } + if (tries === PUBLISH_ATTEMPTS || !isTransientFailure(output)) { + throw new Error(`npm publish ${name}@${version} failed:\n${output}`) + } + const backoff = PUBLISH_SPACING_MS * 2 ** (tries - 1) + console.log( + `release publish: ${name}@${version} hit a transient registry failure` + + ` (attempt ${String(tries)} of ${String(PUBLISH_ATTEMPTS)}), retrying in ${String(backoff)}ms`, + ) + await sleep(backoff) + } +} + /** Publish the family named by `--family` from the directory named by `--from`. */ -function main(): void { +async function main(): Promise { const { values } = parseArgs({ options: { family: { type: 'string' }, from: { type: 'string' } }, allowPositionals: false, @@ -86,17 +155,15 @@ function main(): void { skipped += 1 continue } - // A prerelease version never takes the latest dist-tag. - const tagArgs = version.includes('-') ? ['--tag', 'next'] : [] - // No --access: the sequences do not share one access level, so a - // command-line flag could not serve both and would override the manifest - // that does. Each packed manifest decides, and - // check-workspace-constraints holds every manifest to its sequence's level. - run('npm', ['publish', tarball, ...tagArgs]) + // Space out the writes: the gap belongs between publishes, so a run that + // only skips does not wait at all. + if (published > 0) await sleep(PUBLISH_SPACING_MS) + await publishTarball(tarball, name, version) + console.log(`release publish: ${name}@${version} published`) published += 1 } console.log(`release publish: family ${family.id}, ${String(published)} published, ${String(skipped)} already present`) } -if (isEntry(import.meta.url)) main() +if (isEntry(import.meta.url)) await main()