Merge remote-tracking branch 'origin/master' into worktree/fix-models-settings
This commit is contained in:
+2
-2
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-30-settings-write-path-integrity.md
|
||||
2026-07-30-settings-write-path-integrity.md: 07bd095162879c8e7866846cf562f6a13307e5fc
|
||||
2026-07-30-settings-write-path-integrity.zh.md: 5d02177073d482b61750d7bdfbbd0866bc227a6a
|
||||
2026-07-30-settings-write-path-integrity.md: 7bd50adc8a812759c3ae3f80a50978d75baa712a
|
||||
2026-07-30-settings-write-path-integrity.zh.md: da07745ef1de1b694fc3fd1ee3d04322cdadc992
|
||||
@@ -14,7 +14,7 @@ Review found the provider's write path could destroy state it never observed, an
|
||||
|
||||
**One operation chain, and every write is a read-modify-write.** Watcher refreshes and persists from every namespace queue share a single settled chain, and `persistSection` begins by reconciling the on-disk text into the seam — publishing any unobserved difference first — before rendering against that fresh text. A write can no longer resurrect a stale document, and an on-disk document that turned invalid fails the write loud rather than being overwritten (the reload path keeps its warn-and-keep-last-good policy; the shared `reconcileFromDisk` throws and each caller picks its policy). The watcher's `ready` signal queues one extra reconcile, closing the startup gap between the initial load and the watcher becoming active.
|
||||
|
||||
**Writes hold a `wx`-created `<file>.lock` sibling.** The read-render-rename cycle runs under a cross-process writer lock with exponential backoff, a 2 s acquisition deadline, and stale takeover after 5 s (a crashed holder, broken with a warning). Readers never lock — the rename commit is atomic — so contention is writer-only and resolves in milliseconds. The lock constants are protocol invariants, not config: a holder rewrites one small document, so the deadline and stale age derive from that bound, not from deployment taste.
|
||||
**Writes hold a `wx`-created `<file>.lock` sibling.** The read-render-rename cycle runs under a cross-process writer lock with exponential backoff and a 2 s acquisition deadline. A contender times out without removing the existing lock because age cannot distinguish a crashed owner from a paused live writer; orphan recovery is an operator action. Readers never lock — the rename commit is atomic — so contention is writer-only. The retry and deadline constants are protocol invariants, not deployment config.
|
||||
|
||||
**Observer disposal is quiescent.** Watchers carry an `active` flag checked when a queued invocation would start, so a disposer that ran while the invocation waited prevents the start entirely; started invocations register in a service-level `pendingTails` set that the dispose drain awaits beside the write queues. The `settings/updated` fan-out contains a returned thenable's rejection through the same listener diagnostic as a sync throw, and the event contract now states that the `INVARIANT` rethrow serves synchronous listeners only — invariant companions must stay sync, which the shipped companion already is.
|
||||
|
||||
@@ -24,7 +24,7 @@ Review found the provider's write path could destroy state it never observed, an
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **`proper-lockfile` instead of a hand-rolled lock** — the dependency-over-hand-rolling policy was weighed: the library is barely maintained, its stale/retry policy is broader than this one-file protocol needs, and the shipped lock is ~40 lines with deterministic tests (including injected `EEXIST`/`stat` races). The policy favors dependencies that delete owned code; this one would replace 40 explained lines with an opaque peer.
|
||||
- **`proper-lockfile` instead of a hand-rolled lock** — the dependency-over-hand-rolling policy was weighed: the library is barely maintained, its ownership and retry policy is broader than this one-file protocol needs, and the shipped lock is a small exclusive-create loop with deterministic contention tests. The policy favors dependencies that delete owned code; this one would replace a narrow protocol with an opaque peer.
|
||||
- **Revision/CAS instead of a lock** — rename cannot express compare-and-swap, so a CAS needs a version sidecar or content re-hash and a retry loop in every writer; the lock achieves the same serialization with one primitive and keeps readers free.
|
||||
- **Merging external edits into the in-flight write's own section** — the seam merges patches over the state visible at call time, so a same-namespace external edit racing a write still resolves last-write-wins; folding it in would need three-way merge semantics no consumer has asked for. The write publishes the external state first, so the loser is at least observed before being superseded.
|
||||
- **Declaring async `settings/updated` listeners unsupported** — the typed signature is `void` and lint flags misused promises, but an unlinted JS plugin can still register an async listener; a contract note cannot un-throw an unhandled rejection, so containment is the only defense that holds at runtime.
|
||||
@@ -32,4 +32,4 @@ Review found the provider's write path could destroy state it never observed, an
|
||||
|
||||
## Consequences
|
||||
|
||||
`update()` gained a documented failure mode (lock deadline, invalid on-disk document) and the rejection messages carry `$`-rooted paths. Remaining, documented in the provider README: same-namespace concurrent edits stay last-write-wins (no per-value merge or revision check), a watcher event the OS never delivers leaves the cache stale until the next signal or write, and comments inside replaced arrays or attached inline to changed scalar values go with the value they described. The [user-settings seam note](2026-07-28-user-settings-seam.md)'s deferred-lockfile alternative is superseded by this note. The same defect classes exist in `dsh-credentials-local` (two chains over one `.env`, cached whole-file write-back, post-persist emit) and in the `llm/adapters-updated` fan-out on the stacked branches; those fixes belong to the PRs that introduce the packages and follow this template on merge-up.
|
||||
`update()` has documented failure modes for the lock deadline and an invalid on-disk document, and rejection messages carry `$`-rooted paths. A crashed holder can leave a lock that requires verified operator removal; automatic age-based takeover would permit overlapping writers. Remaining, documented in the provider README: same-namespace concurrent edits stay last-write-wins (no per-value merge or revision check), a watcher event the OS never delivers leaves the cache stale until the next signal or write, and comments inside replaced arrays or attached inline to changed scalar values go with the value they described. The [user-settings seam note](2026-07-28-user-settings-seam.md)'s deferred-lockfile alternative is superseded by this note. The same defect classes exist in `dsh-credentials-local` (two chains over one `.env`, cached whole-file write-back, post-persist emit) and in the `llm/adapters-updated` fan-out on the stacked branches; those fixes belong to the PRs that introduce the packages and follow this template on merge-up.
|
||||
+3
-3
@@ -18,7 +18,7 @@ YAML 写入则整体替换 namespace 节点,把分节内的每条注释都删
|
||||
|
||||
**单一操作链,且每次写入都是读-改-写。**watcher 的刷新与来自各 namespace 队列的持久化共享同一条结算链;`persistSection` 会先把磁盘上的文本对账进 seam——任何未被观察到的差异都先发布出去——然后才对照这份新鲜文本渲染。写入不再可能复活一份陈旧文档;磁盘上已变非法的文档会让写入响亮失败,而不是被覆盖(重载路径保持其“告警并保留最后可用值”策略;共享的 `reconcileFromDisk` 抛错,各调用方自选策略)。watcher 的 `ready` 信号会额外排入一次对账,弥合初始加载与 watcher 生效之间的启动缺口。
|
||||
|
||||
**写入持有以 `wx` 创建的同目录 `<file>.lock`。**读-渲染-rename 循环在一把跨进程写锁下运行:指数退避、2 s 获取截止时间、5 s 后陈旧接管(持有者已崩溃;打破旧锁时给出告警)。读方从不加锁——rename 提交是原子的——因此竞争只发生在写方之间,毫秒级即可化解。锁的各项常量是协议不变式,不是配置:持有者只是重写一份小文档,截止时间与陈旧时限都从这一上界推得,而非出自部署偏好。
|
||||
**写入持有以 `wx` 创建的同目录 `<file>.lock`。**读-渲染-rename 循环在一把跨进程写锁下运行,采用指数退避与 2 s 获取期限。竞争者会超时,但不会移除现有锁,因为锁龄无法区分已经崩溃的所有者与被暂停但仍存活的写入方;遗留锁恢复须由操作者执行。读方从不加锁——rename 提交是原子的——因此竞争只发生在写方之间。重试与期限常量是协议不变式,而非部署配置。
|
||||
|
||||
**观察者 dispose 达到完全停稳。**watcher 携带一个 `active` 标志,排队的调用即将启动时先检查它,因此在调用等待期间已经运行过的释放器能让这次启动彻底不发生;已启动的调用会登记进服务级的 `pendingTails` 集合,dispose 排空除了等待各写队列,还会等待该集合。`settings/updated` 扇出会把监听器返回的 thenable 的 rejection 收容进与同步抛错相同的监听器诊断;事件契约现已写明 `INVARIANT` 重抛只服务同步监听器——不变式配套插件必须保持同步,而已交付的那个配套插件本就是同步的。
|
||||
|
||||
@@ -28,7 +28,7 @@ YAML 写入则整体替换 namespace 节点,把分节内的每条注释都删
|
||||
|
||||
## 曾考虑的替代方案
|
||||
|
||||
- **用 `proper-lockfile` 取代手写锁**——按“依赖优先于手写”政策做过权衡:该库几乎无人维护,其陈旧/重试策略比这个单文件协议所需的更宽泛,而已交付的锁约 40 行并带确定性测试(含注入的 `EEXIST`/`stat` 竞态)。该政策偏向能删除自有代码的依赖;这个依赖只会把 40 行带解释的代码换成一个不透明的等价物。
|
||||
- **用 `proper-lockfile` 取代手写锁**——按“依赖优先于手写”政策做过权衡:该库几乎无人维护,其所有权与重试策略比这个单文件协议所需的更宽泛,而已交付的锁只是一个小型独占创建循环,带确定性的竞争测试。该政策偏向能删除自有代码的依赖;这个依赖只会把一个窄协议换成不透明的等价物。
|
||||
- **用修订号/CAS 取代锁**——rename 表达不了 compare-and-swap,因此 CAS 需要一个版本伴随文件或内容重哈希,外加每个写方里的一个重试循环;锁用一个原语实现同样的串行化,还让读方完全免锁。
|
||||
- **把外部编辑合并进正在进行的写入自身的分节**——seam 是在调用时刻可见的状态之上合并 patch 的,因此与写入竞态的同 namespace 外部编辑仍按后写胜出解决;要把外部编辑并进来,需要三方合并语义,而没有任何消费方提出过这种需求。写入会先发布外部状态,落败一方至少在被取代之前被观察到。
|
||||
- **宣布不支持异步 `settings/updated` 监听器**——类型签名是 `void`,lint 也会标记误用的 promise,但未经 lint 的 JS 插件仍能注册异步监听器;契约里的一句说明无法收回已经抛出的 unhandled rejection,收容是唯一在运行时守得住的防线。
|
||||
@@ -36,6 +36,6 @@ YAML 写入则整体替换 namespace 节点,把分节内的每条注释都删
|
||||
|
||||
## 后果
|
||||
|
||||
`update()` 有了成文的失败模式(锁截止时间到期、磁盘文档非法),rejection 消息携带以 `$` 为根的路径。仍然存在、且已记录在提供方 README 中的有:同 namespace 并发编辑仍是后写胜出(没有逐值合并,也没有修订号检查);OS 从未投递的 watcher 事件会让缓存保持陈旧,直到下一个信号或下一次写入;被替换数组内部的注释、以及行内附着在被改标量值上的注释,会随其描述的值一起消失。
|
||||
`update()` 对锁获取期限与磁盘文档非法都有成文的失败模式,rejection 消息携带以 `$` 为根的路径。持有者崩溃后可能留下锁,需要操作者核实后移除;若按锁龄自动接管,则会允许多个写入方重叠。仍然存在、且已记录在提供方 README 中的有:同 namespace 并发编辑仍是后写胜出(没有逐值合并,也没有修订号检查);OS 从未投递的 watcher 事件会让缓存保持陈旧,直到下一个信号或下一次写入;被替换数组内部的注释、以及行内附着在被改标量值上的注释,会随其描述的值一起消失。
|
||||
|
||||
[用户设置 seam note](2026-07-28-user-settings-seam.md)里“延后锁文件”那条替代方案已被本 note 取代。同类缺陷还存在于 `dsh-credentials-local`(两条链共用一个 `.env`、按缓存整文件写回、持久化之后才发事件)与堆叠分支上的 `llm/adapters-updated` 扇出;这些修复归引入相应包(package)的那些 PR(Pull Request)所有,向上合并时按本模板处理。
|
||||
@@ -364,10 +364,6 @@ export class CredentialsLocal extends Credentials {
|
||||
// After the commit: a broken observer must never make the durable
|
||||
// write look failed (an INVARIANT failure still rethrows).
|
||||
this.notifyUpdated(ref)
|
||||
}, {
|
||||
onStaleBreak: (lockPath) => {
|
||||
this.ctx.logger.warn('credentials-local: breaking a stale writer lock at %s', lockPath)
|
||||
},
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
// editor's multi-line and CRLF discipline.
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import { mkdtemp, readFile, rm, stat, utimes, writeFile } from 'node:fs/promises'
|
||||
import { mkdtemp, readFile, rm, stat, writeFile } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import { credentialRef } from '@deepseek-ai/dsh-credentials'
|
||||
@@ -68,17 +68,6 @@ describe('read-modify-write', () => {
|
||||
expect(await third.credentials.resolve(BETA)).toEqual({ value: '3', source: 'file' })
|
||||
})
|
||||
|
||||
it('breaks a stale writer lock with a warning and writes through', async () => {
|
||||
const dir = await tempDir()
|
||||
const path = join(dir, '.env')
|
||||
const ctx = await boot({ path, watch: false })
|
||||
await writeFile(`${path}.lock`, 'crashed-holder\n')
|
||||
const past = (Date.now() - 60_000) / 1000
|
||||
await utimes(`${path}.lock`, past, past)
|
||||
await ctx.credentials.set(ALPHA, 'nine')
|
||||
expect(await readFile(path, 'utf8')).toContain(`${ALPHA}=nine`)
|
||||
})
|
||||
|
||||
it('creates the credentials directory owner-only', async () => {
|
||||
const dir = await tempDir()
|
||||
const home = join(dir, 'home')
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/settings/settings-local/README.md
|
||||
README.md: 2c0817afd2f2fd35fda2d22cd7f7ef3772fe2257
|
||||
README.zh.md: 547abb035368f07d4478a5c3a1793cdaa6743c68
|
||||
README.md: 344300c33879918e836b6e208b172343cc472faa
|
||||
README.zh.md: 7e4913c0883c48c23de3408a3b0fe0455160984f
|
||||
@@ -19,7 +19,7 @@ Defaulting is one explicit `resolveSpec(config)` step; an unsupported extension
|
||||
|
||||
- **Boot fails loud, reload keeps last-good.** An existing-but-invalid document fails plugin load; once live, an unreadable or unparsable edit warns and keeps the last good sections. A missing document resolves every namespace from defaults and `base`; deleting it publishes the same empty state.
|
||||
- **Every write is a read-modify-write.** A persist first re-reads the document and publishes any difference into the seam — an external edit still inside the watcher debounce window, a change the watcher missed, or another process's write — then renders against that fresh text, so a write can never resurrect a stale document or drop an unobserved sibling section. If the on-disk document turned invalid, the write rejects loud instead of overwriting the user's manual edit.
|
||||
- **Writes hold a cross-process writer lock.** The read-render-rename cycle runs under a `wx`-created `<file>.lock` sibling with exponential backoff, a 2 s acquisition deadline (the write rejects), and stale-lock takeover after 5 s (a crashed holder, broken with a warning). Readers never take the lock: the rename commit is atomic, so reloads are always consistent.
|
||||
- **Writes hold a cross-process writer lock.** The read-render-rename cycle runs under a `wx`-created `<file>.lock` sibling with exponential backoff and a 2 s acquisition deadline. A contender times out without removing the existing lock because age cannot distinguish a crashed owner from a paused live writer; orphan recovery is an operator action. Readers never take the lock: the rename commit is atomic, so reloads are always consistent.
|
||||
- **Write-back is atomic, owner-only, and symlink-proof.** The render exclusive-creates a random-suffix temp sibling with mode `0600` (`wx` refuses to follow a planted symlink) and renames over the target, cleaning the temp up on failure.
|
||||
- **YAML edits are leaf-level diffs.** A write sets only the values that changed and deletes only the keys that were removed, so comments, anchors, and formatting survive on every untouched node and on the key of every changed pair; a changed array (or other non-map value) replaces wholesale, taking comments inside it along. JSON re-serializes without comments.
|
||||
- **Reloads and writes share one operation chain.** Watcher refreshes and persists from every namespace queue run one at a time in queue order; each render sees the text the previous operation committed.
|
||||
|
||||
@@ -19,7 +19,7 @@
|
||||
|
||||
- **启动报错响亮,重载保留最后可用值。** 存在但非法的文档使插件加载失败;运行中不可读或不可解析的编辑只告警并保留最后可用分节。文档缺失时所有 namespace 按默认值与 `base` 解析;删除文档发布同样的空状态。
|
||||
- **每次写入都是一次读-改-写。** persist 先重读文档并把任何差异发布进 seam——无论是仍在 watcher 防抖窗口内的外部编辑、watcher 漏掉的变更,还是另一个进程的写入——再基于这份新鲜文本渲染,因此写入绝不会复活陈旧文档,也不会丢掉未观察到的同级分节。若磁盘上的文档已变为非法,写入响亮拒绝,而不是覆盖用户的手工编辑。
|
||||
- **写入持有跨进程写锁。** 读-渲染-rename 流程在 `wx` 创建的 `<file>.lock` 同级文件下运行,带指数退避、2 s 的获取期限(到期则写入拒绝)与 5 s 后的陈旧锁接管(持有者已崩溃,破锁并告警)。读取方从不取锁:rename 提交是原子的,重载因此始终一致。
|
||||
- **写入持有跨进程写锁。** 读-渲染-rename 流程在 `wx` 创建的 `<file>.lock` 同级文件下运行,带指数退避与 2 s 的获取期限。竞争者会超时,但不会移除现有锁,因为锁龄无法区分已经崩溃的所有者与被暂停但仍存活的写入方;遗留锁恢复须由操作者执行。读取方从不取锁:rename 提交是原子的,重载因此始终一致。
|
||||
- **写回原子、仅属主可读、抗符号链接。** 渲染以 `0600` 权限独占创建随机后缀临时同级文件(`wx` 拒绝跟随预埋符号链接)后 rename 覆盖目标,失败时清理临时文件。
|
||||
- **YAML 编辑是叶子级 diff。** 写入只设置发生变化的值、只删除被移除的键,因此注释、锚点与排版在每个未触碰的节点上以及每个被改键值对的键上都得以保留;被改的数组(或其他非 map 值)整体替换,其中的注释随之一同被换掉。JSON 重新序列化,无注释。
|
||||
- **重载与写入共享一条操作链。** watcher 刷新与来自各 namespace 队列的 persist 按队列顺序逐个执行;每次渲染都基于上一次操作提交后的文本。
|
||||
|
||||
@@ -198,10 +198,6 @@ export class SettingsLocal extends Settings {
|
||||
// 0600: a document that may hold personal values is never world-readable.
|
||||
await writeFileAtomic(this.spec.filename, output, { mode: 0o600, dirMode: 0o700 })
|
||||
this.text = output
|
||||
}, {
|
||||
onStaleBreak: (lockPath) => {
|
||||
this.ctx.logger.warn('settings-local: breaking a stale writer lock at %s', lockPath)
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
|
||||
@@ -70,25 +70,20 @@ describe('writer lock', () => {
|
||||
expect(await readFile(path, 'utf8')).toContain('value: 7')
|
||||
})
|
||||
|
||||
it('breaks a stale writer lock with a warning and writes through', async () => {
|
||||
it('does not steal an old writer lock', async () => {
|
||||
const dir = await tempDir()
|
||||
const path = join(dir, 'settings.yaml')
|
||||
await writeFile(path, 'alpha:\n value: 4\n')
|
||||
const ctx = await boot({ path, watch: false })
|
||||
const scope = ctx.settings.register(settingsNamespace('alpha'), AlphaSchema)
|
||||
await writeFile(`${path}.lock`, 'crashed-holder\n')
|
||||
const lockPath = `${path}.lock`
|
||||
await writeFile(lockPath, 'slow-holder\n')
|
||||
const past = (Date.now() - 60_000) / 1000
|
||||
await utimes(`${path}.lock`, past, past)
|
||||
await scope.update({ value: 9 })
|
||||
expect(await readFile(path, 'utf8')).toContain('value: 9')
|
||||
})
|
||||
await utimes(lockPath, past, past)
|
||||
|
||||
it('times out on a lock a live holder never releases', async () => {
|
||||
const dir = await tempDir()
|
||||
const path = join(dir, 'settings.yaml')
|
||||
const ctx = await boot({ path, watch: false })
|
||||
const scope = ctx.settings.register(settingsNamespace('alpha'), AlphaSchema)
|
||||
await writeFile(`${path}.lock`, 'busy-holder\n')
|
||||
await expect(scope.update({ value: 1 })).rejects.toThrow(/timed out waiting for the writer lock/)
|
||||
await expect(scope.update({ value: 9 })).rejects.toThrow(/timed out waiting for the writer lock/)
|
||||
expect(await readFile(path, 'utf8')).toContain('value: 4')
|
||||
expect(await readFile(lockPath, 'utf8')).toBe('slow-holder\n')
|
||||
}, 10_000)
|
||||
|
||||
it('surfaces a non-contention lock failure as the write error', async () => {
|
||||
|
||||
@@ -1,8 +1,5 @@
|
||||
// Writer-lock races that cannot be timed from outside: a contender whose lock
|
||||
// vanishes between the failed exclusive create and the stat, a stat failing
|
||||
// for a reason other than absence, and a temp-file write failing mid-cycle.
|
||||
// The fs/promises seam is partially mocked to inject exactly one failure at a
|
||||
// chosen path suffix; everything else passes through to the real filesystem.
|
||||
// A temp-file write failure cannot be timed from outside. The fs/promises seam
|
||||
// injects it once so the test can prove that the writer lock still releases.
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import z from 'schemastery'
|
||||
@@ -13,28 +10,20 @@ import { settingsNamespace } from '@deepseek-ai/dsh-settings'
|
||||
import { SettingsLocal } from '../src/index.ts'
|
||||
|
||||
const state = vi.hoisted(() => ({
|
||||
/** One-shot failure injections keyed by operation, matched on a path suffix. */
|
||||
failures: [] as Array<{ op: 'writeFile' | 'stat'; suffix: string; code: string }>,
|
||||
failTempWrite: false,
|
||||
}))
|
||||
|
||||
vi.mock('node:fs/promises', async (importOriginal) => {
|
||||
const actual = await importOriginal<typeof import('node:fs/promises')>()
|
||||
const inject = (op: 'writeFile' | 'stat', path: unknown): void => {
|
||||
const index = state.failures.findIndex(f => f.op === op && String(path).endsWith(f.suffix))
|
||||
if (index === -1) return
|
||||
const [failure] = state.failures.splice(index, 1)
|
||||
throw Object.assign(new Error(`${failure!.code}: injected ${op} failure`), { code: failure!.code })
|
||||
}
|
||||
return {
|
||||
...actual,
|
||||
writeFile: (async (path: unknown, ...rest: never[]) => {
|
||||
inject('writeFile', path)
|
||||
if (state.failTempWrite && String(path).endsWith('.tmp')) {
|
||||
state.failTempWrite = false
|
||||
throw Object.assign(new Error('ENOSPC: injected writeFile failure'), { code: 'ENOSPC' })
|
||||
}
|
||||
return (actual.writeFile as (path: unknown, ...args: never[]) => Promise<void>)(path, ...rest)
|
||||
}) as typeof actual.writeFile,
|
||||
stat: (async (path: unknown, ...rest: never[]) => {
|
||||
inject('stat', path)
|
||||
return (actual.stat as (path: unknown, ...args: never[]) => Promise<unknown>)(path, ...rest)
|
||||
}) as typeof actual.stat,
|
||||
}
|
||||
})
|
||||
|
||||
@@ -43,7 +32,7 @@ const AlphaSchema: z<{ value: number }> = z.object({ value: z.number().default(0
|
||||
const cleanups: Array<() => Promise<void>> = []
|
||||
|
||||
afterEach(async () => {
|
||||
state.failures.length = 0
|
||||
state.failTempWrite = false
|
||||
while (cleanups.length > 0) await cleanups.pop()!()
|
||||
})
|
||||
|
||||
@@ -61,37 +50,14 @@ async function boot(config: ConstructorParameters<typeof SettingsLocal>[1]): Pro
|
||||
return ctx
|
||||
}
|
||||
|
||||
describe('writer-lock races', () => {
|
||||
it('retries immediately when the contending lock vanished before the stat', async () => {
|
||||
const dir = await tempDir()
|
||||
const path = join(dir, 'settings.yaml')
|
||||
const ctx = await boot({ path, watch: false })
|
||||
const scope = ctx.settings.register(settingsNamespace('alpha'), AlphaSchema)
|
||||
// The exclusive create loses to a holder that releases before the stat:
|
||||
// no lock file actually exists, so the stat sees honest absence and the
|
||||
// very next attempt takes the lock.
|
||||
state.failures.push({ op: 'writeFile', suffix: '.lock', code: 'EEXIST' })
|
||||
await scope.update({ value: 3 })
|
||||
expect(await readFile(path, 'utf8')).toContain('value: 3')
|
||||
})
|
||||
|
||||
it('propagates a stat failure that does not mean absence', async () => {
|
||||
const dir = await tempDir()
|
||||
const path = join(dir, 'settings.yaml')
|
||||
const ctx = await boot({ path, watch: false })
|
||||
const scope = ctx.settings.register(settingsNamespace('alpha'), AlphaSchema)
|
||||
state.failures.push({ op: 'writeFile', suffix: '.lock', code: 'EEXIST' })
|
||||
state.failures.push({ op: 'stat', suffix: '.lock', code: 'EACCES' })
|
||||
await expect(scope.update({ value: 3 })).rejects.toThrow(/EACCES/)
|
||||
})
|
||||
|
||||
describe('writer-lock failure cleanup', () => {
|
||||
it('cleans up the temp file and releases the lock when the write fails mid-cycle', async () => {
|
||||
const dir = await tempDir()
|
||||
const path = join(dir, 'settings.yaml')
|
||||
await writeFile(path, 'alpha:\n value: 1\n')
|
||||
const ctx = await boot({ path, watch: false })
|
||||
const scope = ctx.settings.register(settingsNamespace('alpha'), AlphaSchema)
|
||||
state.failures.push({ op: 'writeFile', suffix: '.tmp', code: 'ENOSPC' })
|
||||
state.failTempWrite = true
|
||||
await expect(scope.update({ value: 9 })).rejects.toThrow(/ENOSPC/)
|
||||
// The document is untouched and the writer lock was released on the way out.
|
||||
expect(await readFile(path, 'utf8')).toContain('value: 1')
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/util/atomic-write/README.md
|
||||
README.md: be9f896eb24e28aedc2c04858da8b8da9da548dc
|
||||
README.zh.md: 19a067dc84f12d334e5c31dda58e7cf78dac51f9
|
||||
README.md: 2ff4abb6ac10d8b592ccd2056b4f1f92cc8518b0
|
||||
README.zh.md: bd5d3f1f2583ff1b97a80ce2c1c6ca7989e13d15
|
||||
@@ -28,7 +28,7 @@ await withFileLock('/home/u/.dsh/settings.yaml', async () => {
|
||||
- **Same-directory sibling** keeps the rename on one filesystem, so the swap stays atomic.
|
||||
- Parent directories are created; on any failure the temp is removed and the failure rethrown; readers observe either the old or the new complete content.
|
||||
|
||||
`withFileLock` serializes the writers of one file across processes, for the read-render-commit cycles a bare atomic commit cannot make safe on its own. The lock is a `wx`-created `<filename>.lock` sibling, so readers never contend; waiters back off exponentially and fail with a timeout rather than block forever. A lock older than the stale age is treated as a crashed holder and broken — see [Known Limitations and Deferred Work](#known-limitations-and-deferred-work) for what that costs.
|
||||
`withFileLock` serializes the writers of one file across processes, for the read-render-commit cycles a bare atomic commit cannot make safe on its own. The lock is a `wx`-created `<filename>.lock` sibling, so readers never contend; waiters back off exponentially and fail with a timeout rather than block forever. A contender never removes the existing lock: age cannot distinguish a crashed owner from a paused live writer.
|
||||
|
||||
## Model Experience
|
||||
|
||||
@@ -42,4 +42,4 @@ None; nothing here enters a request prefix.
|
||||
|
||||
- **Atomic, not durable** — no `fsync` of the file or its directory, so after a crash the rename may be observed unwound. The file-backed stores here re-read and republish on boot, keeping durability the caller's policy.
|
||||
- **String content only** — no `Buffer` or stream form until a consumer needs one.
|
||||
- **The lock takes over by age, not by ownership** (`TODO(settings-lock-ownership)`) — a holder slower than the stale age has its lock broken by a waiter, and release unlinks the path unconditionally, so a slow writer can remove a successor's lock. Two writers can then overlap and one cycle's result be lost. The stale age is set well above any write this repo performs, so the exposure is a paused or swapped-out process; ownership-safe acquisition and release is the fix.
|
||||
- **Orphaned locks require operator recovery** — a process that exits while holding the lock can leave the sibling behind. Later writers time out without deleting it; an operator removes it only after verifying that no writer still owns it. File age alone is not safe evidence of abandonment.
|
||||
@@ -28,7 +28,7 @@ await withFileLock('/home/u/.dsh/settings.yaml', async () => {
|
||||
- **同目录兄弟文件**保证 rename 落在同一文件系统上,交换保持原子。
|
||||
- 自动创建父目录;任何失败都会移除临时文件并重新抛出该失败;读取方只会观察到旧内容或完整的新内容。
|
||||
|
||||
`withFileLock` 跨进程串行化同一文件的写入方,服务于单靠原子提交无法保证安全的读-渲染-提交循环。锁是以 `wx` 创建的同目录 `<filename>.lock`,因此读取方从不参与竞争;等待方按指数退避,超时即失败而非无限阻塞。超过陈旧时限的锁被视为持有者已崩溃并被打破——其代价见[Known Limitations and Deferred Work](#known-limitations-and-deferred-work)。
|
||||
`withFileLock` 跨进程串行化同一文件的写入方,服务于单靠原子提交无法保证安全的读-渲染-提交循环。锁是以 `wx` 创建的同目录 `<filename>.lock`,因此读取方从不参与竞争;等待方按指数退避,超时即失败而非无限阻塞。竞争者绝不移除现有锁:锁龄无法区分已经崩溃的所有者与被暂停但仍存活的写入方。
|
||||
|
||||
## Model Experience
|
||||
|
||||
@@ -42,4 +42,4 @@ await withFileLock('/home/u/.dsh/settings.yaml', async () => {
|
||||
|
||||
- **原子但不保证持久**——不对文件或其所在目录做 `fsync`,因此崩溃后可能观察到 rename 被回退。此处的文件型存储在启动时重新读取并重新发布,把持久性留作调用方的策略。
|
||||
- **仅支持字符串内容**——在有消费方需要之前,不提供 `Buffer` 或流式形态。
|
||||
- **锁按时长而非归属接管**(`TODO(settings-lock-ownership)`)——持有者若慢于陈旧时限,其锁会被等待方打破,而释放又无条件删除该路径,因此慢写入方可能删掉后继者的锁。两个写入方随之重叠,一轮循环的结果可能丢失。陈旧时限远高于本仓库的任何一次写入,因此暴露面是被暂停或被换出的进程;修法是按归属安全地获取与释放。
|
||||
- **遗留锁需要操作者恢复**——进程持锁退出时可能留下同级锁文件。后续写入方超时也不会删除它;操作者只有在确认没有写入方仍拥有该锁后才会移除。文件存续时间本身不能安全证明它已无人持有。
|
||||
@@ -11,7 +11,7 @@
|
||||
*/
|
||||
|
||||
import { randomBytes } from 'node:crypto'
|
||||
import { mkdir, rename, rm, stat, writeFile } from 'node:fs/promises'
|
||||
import { mkdir, rename, rm, writeFile } from 'node:fs/promises'
|
||||
import { dirname } from 'node:path'
|
||||
|
||||
/**
|
||||
@@ -68,59 +68,31 @@ function isEEXIST(error: unknown): boolean {
|
||||
return (error as NodeJS.ErrnoException | null)?.code === 'EEXIST'
|
||||
}
|
||||
|
||||
/** Whether a filesystem error means absence. */
|
||||
function isENOENT(error: unknown): boolean {
|
||||
return (error as NodeJS.ErrnoException | null)?.code === 'ENOENT'
|
||||
}
|
||||
|
||||
/**
|
||||
* Writer-lock protocol constants. These are robustness invariants of the
|
||||
* cross-process write protocol, not deployment tunables: a holder rewrites one
|
||||
* small file in milliseconds, so contention resolves well inside the retry
|
||||
* deadline, and a lock older than the stale age can only belong to a crashed
|
||||
* holder.
|
||||
* cross-process write protocol, not deployment tunables: contention normally
|
||||
* resolves within the retry deadline, while expiry fails the contender without
|
||||
* guessing whether the existing lock still has an owner.
|
||||
*/
|
||||
const LOCK_RETRY_INITIAL_MS = 20
|
||||
const LOCK_RETRY_MAX_MS = 200
|
||||
const LOCK_TIMEOUT_MS = 2_000
|
||||
const LOCK_STALE_MS = 5_000
|
||||
|
||||
/** Options for {@link withFileLock}. */
|
||||
export interface WithFileLockOptions {
|
||||
/**
|
||||
* Called once each time a stale (crashed-holder) lock is broken, so the
|
||||
* caller can log the takeover in its own voice.
|
||||
*/
|
||||
onStaleBreak?: (lockPath: string) => void
|
||||
}
|
||||
|
||||
/** Age of the lock file, or `undefined` when it vanished after a failed create. */
|
||||
async function lockAgeMs(lockPath: string): Promise<number | undefined> {
|
||||
try {
|
||||
return Date.now() - (await stat(lockPath)).mtimeMs
|
||||
} catch (error) {
|
||||
if (!isENOENT(error)) throw error
|
||||
return undefined
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Hold the cross-process writer lock for `filename` around one operation. The
|
||||
* lock is a `wx`-created sibling (`<filename>.lock`); paired with the
|
||||
* rename-based commit of {@link writeFileAtomic}, readers stay lock-free and
|
||||
* only writers contend. Contention backs off exponentially; a lock older than
|
||||
* the stale age is a crashed holder and is broken (see
|
||||
* {@link WithFileLockOptions.onStaleBreak}); a live holder past the deadline
|
||||
* fails the operation with a timed-out error. The parent directory must exist.
|
||||
* only writers contend. Contention backs off exponentially and fails with a
|
||||
* timed-out error after the deadline. The contender never removes an existing
|
||||
* lock because file age cannot prove that its owner stopped; orphan recovery
|
||||
* is an operator action. The parent directory must exist.
|
||||
* @param filename - the file whose writers this lock serializes.
|
||||
* @param operation - the read-render-commit cycle to run while holding the lock.
|
||||
* @param options - stale-takeover notification hook.
|
||||
* @returns the operation's result; the lock releases on both outcomes.
|
||||
*/
|
||||
export async function withFileLock<T>(
|
||||
filename: string,
|
||||
operation: () => Promise<T>,
|
||||
options?: WithFileLockOptions,
|
||||
): Promise<T> {
|
||||
const lockPath = `${filename}.lock`
|
||||
const deadline = Date.now() + LOCK_TIMEOUT_MS
|
||||
@@ -132,17 +104,6 @@ export async function withFileLock<T>(
|
||||
} catch (error) {
|
||||
if (!isEEXIST(error)) throw error
|
||||
}
|
||||
const ageMs = await lockAgeMs(lockPath)
|
||||
// The holder released between the failed create and the stat: the lock is
|
||||
// free right now, so retry without burning backoff or deadline.
|
||||
if (ageMs === undefined) continue
|
||||
if (ageMs > LOCK_STALE_MS) {
|
||||
// TODO(settings-lock-ownership): Replace age-only takeover with ownership-safe
|
||||
// acquisition and release so a slow writer cannot remove a successor's lock.
|
||||
options?.onStaleBreak?.(lockPath)
|
||||
await rm(lockPath, { force: true })
|
||||
continue
|
||||
}
|
||||
if (Date.now() >= deadline) {
|
||||
throw new Error(`atomic-write: timed out waiting for the writer lock at ${lockPath}`)
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user