Merge pull request #834 from deepseek-harness/fix/translation-brief-uncommitted-pair
fix(i18n): persist uncommitted translation snapshots
This commit is contained in:
Ziya
GitHub
+2
-2
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/process/2026-07-26-briefed-minimal-translation-updates.md
|
||||
2026-07-26-briefed-minimal-translation-updates.md: 63f25d5c36caecba435e9192534e0b494e1e0105
|
||||
2026-07-26-briefed-minimal-translation-updates.zh.md: 17cf2f895b187fb794e691c2b14d6e0bff78363d
|
||||
2026-07-26-briefed-minimal-translation-updates.md: a47251376771165d0eb229aaa0fb7f63589d7d77
|
||||
2026-07-26-briefed-minimal-translation-updates.zh.md: 5a6787b9bdb3286b71842abec075c5a0ebfc7991
|
||||
@@ -14,7 +14,7 @@ Pair updates run on a generated briefing instead of the guidance corpus; only ne
|
||||
|
||||
- **`pnpm run gen-translation-brief [--apply] [pair...]`** ([scripts/gen-translation-brief.ts](../../../../scripts/gen-translation-brief.ts), assembly in [scripts/translation-brief.ts](../../../../scripts/translation-brief.ts)) prints, per out-of-sync pair, the authored side's diff from its recorded last-confirmed blob to the working tree plus the change mapped at the narrowest safely aligned granularity, deterministically widening on mapping failure: a change confined to the pair's byte-identical code fences is computed outright (`--apply` splices it into the counterpart and validates the result against the pairing gate's structural signature before writing); otherwise changed Markdown units (headings, paragraphs, table rows, list items, code fences, block quotes, HTML blocks, thematic breaks, link definitions — matched by container-scoped kind sequences) each carry their last-confirmed source, current source, and current counterpart text with line numbers; units that do not align fall back to depth-matched heading sections; and when sections do not align either, or both sides drifted, the briefing says so and withholds the mapping instead of guessing. Terminology rows are matched against the changed spans only (word-boundary English matching with plural inflections), and for Chinese targets the briefing tracks each relevant term's document-wide first occurrence — when an edit moves it, the vacated and receiving spans join the briefing with an explanatory note, since the 首次出现 annotation must move with it. The unit mapping, code splice, and first-occurrence mechanics adopt the planner design from the [incremental prompt-pipeline work](https://github.com/deepseek-harness/deepseek-harness/pull/684), whose provider-backed bake-off independently validated the same scope ladder for the automated pipeline. The briefing is the translator's whole working set; the full sources of truth remain the escalation path for decisions the briefing cannot answer.
|
||||
- **The update path in [dsh-translate-docs](../../../skills/dsh-translate-docs/SKILL.md)** consumes the briefing: mechanical (code-fence-only) changes are applied with `--apply`, no subagent; prose diffs go to a subagent whose prompt is the briefing, not the corpus; verification is clause-by-clause on the changed spans, not the whole document.
|
||||
- **The pairing gate takes pair arguments.** `verify-translation-pairing [pair...]` checks just the named pairs (any of a pair's three files, or the bare stem, names it); the corpus-wide sweep remains the no-argument form that `doc-sync` and CI run. `--write` now requires naming the confirmed pairs — bare `--write` refuses, and re-recording everything is an explicit `--write --all` — because the old bare form silently blessed every drifted pair in the tree, including ones the caller never looked at, and a prose-only drift would then stay green forever. Each record's comment names its own scoped command.
|
||||
- **The pairing gate takes pair arguments.** `verify-translation-pairing [pair...]` checks just the named pairs (any of a pair's three files, or the bare stem, names it); the corpus-wide sweep remains the no-argument form that `doc-sync` and CI run. `--write` now requires naming the confirmed pairs — bare `--write` refuses, and re-recording everything is an explicit `--write --all` — because the old bare form silently blessed every drifted pair in the tree, including ones the caller never looked at, and a prose-only drift would then stay green forever. Each record's comment names its own scoped command. Before recording, `--write` stores each side's exact bytes with `git hash-object -w --stdin` and pins the blob under a content-addressed local `refs/dsh/translation-pairing/snapshots/` ref; an uncommitted last-confirmed snapshot is therefore available to the briefing generator's later `git cat-file`, not merely named by a hash that Git cannot resolve or left vulnerable to garbage collection.
|
||||
|
||||
## Benchmark
|
||||
|
||||
@@ -40,6 +40,7 @@ A second head-to-head replay on the same ten examples compared this note's shipp
|
||||
|
||||
- A small prose edit's counterpart update now costs a briefing generation plus one small focused task — no corpus reads, no archaeology, no corpus-wide scans inside the loop — and the same PR obligation holds; the cheap path and the correct path point the same way.
|
||||
- The briefing generator is a second consumer of the consistency records: recorded blob hashes now also drive diff recovery and section mapping, strengthening the incentive to keep records honest.
|
||||
- Each distinct confirmed snapshot retains a content-addressed local ref and object. An abandoned re-record may therefore leave an extra durable pin, but it changes no branch or commit history; this local retention is the tradeoff that prevents garbage collection from invalidating an accepted pairing record.
|
||||
- `--write` without arguments no longer works; muscle-memory callers must name pairs or pass `--all`. That is the point — the bulk bless is now a visible, deliberate act.
|
||||
- Scoped checks mean an update loop can be green while an unrelated pair elsewhere is red; the corpus-wide check in `doc-sync`/CI still owns the tree-level invariant.
|
||||
- Span mapping trusts an alignment only when the kind sequences match across the last-confirmed source, current source, and current counterpart; a mapping failure widens deterministically (units → sections → whole document) rather than guessing, so a restructured document gets an explicit "locate the regions yourself" briefing, never a wrong map.
|
||||
@@ -47,4 +48,4 @@ A second head-to-head replay on the same ten examples compared this note's shipp
|
||||
|
||||
## Testing
|
||||
|
||||
[scripts/translation-brief.spec.ts](../../../../scripts/translation-brief.spec.ts) pins unit and section span extraction (container-scoped kinds, depth-only section alignment so translated heading text still maps, preamble), alignment and changed-index detection, the mechanical code splice and each of its refusal conditions, terminology row matching in both directions with word-boundary and plural-inflection discipline, first-occurrence movement tracking, fence escalation, and the rendered briefing's contract (unit bundles with three-way context, mechanical/sections/document scopes, per-direction digests, scoped finish commands). [scripts/translation-pairing.spec.ts](../../../../scripts/translation-pairing.spec.ts) pins argument normalization (any pair file or bare stem to the anchor) and the CLI matrix: scoped check, bare `--write` refusal, `--write <pair>`, `--write --all`, `--list` exclusivity, unknown flags.
|
||||
[scripts/translation-brief.spec.ts](../../../../scripts/translation-brief.spec.ts) pins unit and section span extraction (container-scoped kinds, depth-only section alignment so translated heading text still maps, preamble), alignment and changed-index detection, the mechanical code splice and each of its refusal conditions, terminology row matching in both directions with word-boundary and plural-inflection discipline, first-occurrence movement tracking, fence escalation, and the rendered briefing's contract (unit bundles with three-way context, mechanical/sections/document scopes, per-direction digests, scoped finish commands). [scripts/translation-pairing.spec.ts](../../../../scripts/translation-pairing.spec.ts) pins exact-byte persistence and recovery for uncommitted snapshots, failure before an unavailable object can enter a sidecar, argument normalization (any pair file or bare stem to the anchor), and the CLI matrix: scoped check, bare `--write` refusal, `--write <pair>`, `--write --all`, `--list` exclusivity, unknown flags.
|
||||
+3
-2
@@ -14,7 +14,7 @@ Status: implemented
|
||||
|
||||
- **`pnpm run gen-translation-brief [--apply] [pair...]`**（[scripts/gen-translation-brief.ts](../../../../scripts/gen-translation-brief.ts)，组装逻辑在 [scripts/translation-brief.ts](../../../../scripts/translation-brief.ts)）针对每个失去同步的配对，打印被改一侧从其记录在案的上次确认 blob 到当前工作区的 diff，并附上以能安全对齐的最窄粒度映射的这次改动，映射失败时粒度确定性地逐级放宽：仅落在配对中逐字节一致的围栏代码块内的改动会直接算出（`--apply` 会把它拼接进对侧文件，并在写入前用配对门禁的结构签名校验所得结果）；否则，每个有改动的 Markdown 单元（标题、段落、表格行、列表项、围栏代码块、块引用、HTML 块、分隔线、链接定义；匹配依据是以容器为作用域的种类序列）都带上各自的上次确认源文、当前源文与当前对侧文本及行号；无法对齐的单元回退到按深度匹配的标题章节；当章节也无法对齐或两侧同时漂移时，简报会明说这一点并省略映射，而不是靠猜。术语表行只与改动块匹配（英文术语按词边界匹配，含复数变形）；当目标侧是中文时，简报还会跟踪每个相关术语在整篇文档中的首次出现：一旦某次编辑使其移位，腾出的与接收的两处区间就会附一条解释性说明加入简报，因为「首次出现」括注必须随之移动。单元映射、代码拼接与首次出现机制采纳了[增量提示词流水线工作](https://github.com/deepseek-harness/deepseek-harness/pull/684)中的规划器设计；该项工作中接入提供方的对比评测，已为自动流水线独立验证了同一套范围阶梯。简报就是译者的全部工作集；简报回答不了的决策，仍以完整的真源文档作为升级求证路径。
|
||||
- **[dsh-translate-docs](../../../skills/dsh-translate-docs/SKILL.md) 中的更新路径**消费这份简报：机械类改动（只涉及围栏代码块）用 `--apply` 应用，不动用 subagent；行文类 diff 交给 subagent，其提示词就是简报本身，而非指导语料；核验只对改动块逐句进行，不覆盖整篇文档。
|
||||
- **配对门禁接受配对参数。**`verify-translation-pairing [pair...]` 只检查被点名的配对（配对三个文件中的任意一个，或其裸词干，都能指代该配对）；全语料扫描仍是 `doc-sync`（文档同步门禁）与 CI 运行的无参数形式。`--write` 现在要求点名已确认的配对：裸 `--write` 会拒绝执行，重新记录全部配对必须显式写 `--write --all`；原因是旧的裸形式会默默为树中每一个漂移的配对背书，包括调用者从未看过的那些，纯行文层面的漂移于是可以永远保持绿灯。每份记录的注释都写明针对该配对自身的按对命令。
|
||||
- **配对门禁接受配对参数。**`verify-translation-pairing [pair...]` 只检查被点名的配对（配对三个文件中的任意一个，或其裸词干，都能指代该配对）；全语料扫描仍是 `doc-sync`（文档同步门禁）与 CI 运行的无参数形式。`--write` 现在要求点名已确认的配对：裸 `--write` 会拒绝执行，重新记录全部配对必须显式写 `--write --all`；原因是旧的裸形式会默默为树中每一个漂移的配对背书，包括调用者从未看过的那些，纯行文层面的漂移于是可以永远保持绿灯。每份记录的注释都写明针对该配对自身的按对命令。写下记录之前，`--write` 用 `git hash-object -w --stdin` 存入每一侧的精确字节，并在内容寻址的本地 `refs/dsh/translation-pairing/snapshots/` ref 下固定该 blob；未提交的上次确认快照因此能被简报生成器之后的 `git cat-file` 取回，而不只是留下一个 Git 无法解析的 hash 名称或暴露于垃圾回收。
|
||||
|
||||
## 基准测试
|
||||
|
||||
@@ -40,6 +40,7 @@ Status: implemented
|
||||
|
||||
- 一次小的行文修改，其对侧更新如今只需生成一份简报，外加一个小而聚焦的任务（不读指导语料、不翻查历史、循环内不做全语料扫描），同一 PR 内完成更新的义务保持不变；低成本的路径与正确的路径指向同一个方向。
|
||||
- 简报生成器成为一致性记录的第二个消费方：记录的 blob hash 如今还驱动 diff 还原与章节映射，这进一步强化了如实维护记录的动机。
|
||||
- 每个不同的已确认快照都会保留一个内容寻址的本地 ref 和对象。因此，中途放弃的重新记录可能留下额外的持久固定项，但它不会改变任何分支或提交历史；这种本地保留正是防止垃圾回收让已接受配对记录失效所付出的代价。
|
||||
- 不带参数的 `--write` 不再可用；靠肌肉记忆的调用者必须点名配对或传 `--all`。这正是目的所在：批量背书如今是一个可见的、有意为之的动作。
|
||||
- 按对检查意味着一个更新循环可以在别处某个无关配对处于红灯时自己保持绿灯；`doc-sync`/CI 中的全语料检查仍然承载树级不变式。
|
||||
- 区间映射只在上次确认源文、当前源文与当前对侧文本三方的种类序列一致时才信任一处对齐；映射失败时粒度确定性地逐级放宽（单元 → 章节 → 整篇文档）而不是靠猜，因此被重构过的文档拿到的是一份明确写着「请自行定位相关区域」的简报，绝不会是一张错误的地图。
|
||||
@@ -47,4 +48,4 @@ Status: implemented
|
||||
|
||||
## 测试
|
||||
|
||||
[scripts/translation-brief.spec.ts](../../../../scripts/translation-brief.spec.ts) 固定单元与章节的区间提取（以容器为作用域的种类、只按深度对齐章节从而让已翻译的标题文字仍能映射、首个标题前的序言）、对齐与改动索引检测、机械代码拼接及其每一个拒绝条件、带词边界与复数变形约束的双向术语行匹配、首次出现移位跟踪、围栏升级，以及渲染后简报的契约（带三方上下文的单元条目、机械／章节／整篇文档三种范围、分方向的规则摘要、按对的收尾命令）。[scripts/translation-pairing.spec.ts](../../../../scripts/translation-pairing.spec.ts) 固定参数归一化（配对的任一文件或裸词干都归一到锚点）与 CLI（命令行界面）用例矩阵：按对检查、裸 `--write` 拒绝执行、`--write <pair>`、`--write --all`、`--list` 的互斥性、未知标志。
|
||||
[scripts/translation-brief.spec.ts](../../../../scripts/translation-brief.spec.ts) 固定单元与章节的区间提取（以容器为作用域的种类、只按深度对齐章节从而让已翻译的标题文字仍能映射、首个标题前的序言）、对齐与改动索引检测、机械代码拼接及其每一个拒绝条件、带词边界与复数变形约束的双向术语行匹配、首次出现移位跟踪、围栏升级，以及渲染后简报的契约（带三方上下文的单元条目、机械／章节／整篇文档三种范围、分方向的规则摘要、按对的收尾命令）。[scripts/translation-pairing.spec.ts](../../../../scripts/translation-pairing.spec.ts) 固定未提交快照的精确字节持久化与取回、在不可用对象进入伴随记录前失败、参数归一化（配对的任一文件或裸词干都归一到锚点），以及 CLI（命令行界面）用例矩阵：按对检查、裸 `--write` 拒绝执行、`--write <pair>`、`--write --all`、`--list` 的互斥性、未知标志。
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write docs/i18n/README.md
|
||||
README.md: 9fe01174909f0400b94c70b95759526aec783d2e
|
||||
README.zh.md: 2a9a20ae4702e205879f0fbfb6d76d3a761d2446
|
||||
README.md: d1fff61a1d3db7fbe93d3c95b144598c2ca41c8b
|
||||
README.zh.md: 9f1d61f1b854ead0ef5851157cce2e9a6de83d89
|
||||
+1
-1
@@ -15,7 +15,7 @@ This repo's documentation is read by people and agents both inside and outside t
|
||||
foo.zh.md: 89e6c98d92887913cadf06b2adb97f26cde4849b
|
||||
```
|
||||
|
||||
Blob hashes, not commit hashes, so the record is computable for files edited in the same PR (`git hash-object foo.md`) and consistency is a pure content comparison. The recorded hashes also recover the exact last-confirmed text of either side, so an out-of-sync pair is updated by patching the counterpart minimally against the edited side's diff — never by re-translating whole files. `pnpm run gen-translation-brief <pair>` assembles that update's working set mechanically at the narrowest safely aligned granularity — changed Markdown units, then heading sections, then whole document — with the edited side's diff since last confirmation, each changed span's three-way text, the terminology rows the change touches, and the binding update rules; a change confined to the pair's byte-identical code fences is computed outright, and `--apply` splices it into the counterpart after structural validation ([briefed-updates Agent Note](../../.agents/notes/implemented/process/2026-07-26-briefed-minimal-translation-updates.md)). After bringing the pair back in line, `pnpm run verify-translation-pairing --write <pair>` re-records both hashes; that yaml diff is the reviewable act of confirming consistency, which is why `--write` requires naming the pairs you confirmed (`--write --all` is the explicit corpus-wide form).
|
||||
Blob hashes, not commit hashes, so the record is computable for files edited in the same PR (`git hash-object foo.md`) and consistency is a pure content comparison. `--write` stores those snapshots in the local Git object database before recording them, including uncommitted working-tree contents, and pins every distinct stored blob under a content-addressed `refs/dsh/translation-pairing/snapshots/` ref so garbage collection cannot invalidate a recorded recovery pointer. The recorded hashes therefore recover the exact last-confirmed text of either side, so an out-of-sync pair is updated by patching the counterpart minimally against the edited side's diff — never by re-translating whole files. `pnpm run gen-translation-brief <pair>` assembles that update's working set mechanically at the narrowest safely aligned granularity — changed Markdown units, then heading sections, then whole document — with the edited side's diff since last confirmation, each changed span's three-way text, the terminology rows the change touches, and the binding update rules; a change confined to the pair's byte-identical code fences is computed outright, and `--apply` splices it into the counterpart after structural validation ([briefed-updates Agent Note](../../.agents/notes/implemented/process/2026-07-26-briefed-minimal-translation-updates.md)). After bringing the pair back in line, `pnpm run verify-translation-pairing --write <pair>` re-records both hashes; that yaml diff is the reviewable act of confirming consistency, which is why `--write` requires naming the pairs you confirmed (`--write --all` is the explicit corpus-wide form).
|
||||
- **Language switcher.** Both files link to each other immediately after their H1 heading: the English file carries `English | [中文](foo.zh.md)` and the Chinese file carries `[English](foo.md) | 中文`.
|
||||
- **Structure mirrors the counterpart.** Heading depths and order, list kinds, ordered-list starts, list item counts, table row and column counts, link targets, and verbatim code blocks match one to one across the pair — see [translation-rules.md](translation-rules.md) for the full preservation rules. Existing Markdown gates apply to `.zh.md` files unchanged (`verify-md-wrap`, `verify-md-links`).
|
||||
|
||||
|
||||
@@ -15,7 +15,7 @@
|
||||
foo.zh.md: 89e6c98d92887913cadf06b2adb97f26cde4849b
|
||||
```
|
||||
|
||||
用 blob hash 而不是 commit hash，这样同一个 PR 里改动的文件也能算出记录（`git hash-object foo.md`），一致性是纯内容比较。记录的 hash 还能还原任一侧上次确认时的确切文本，所以失去同步的配对是「按被改一侧的 diff 最小化地修补另一侧」，从不整篇重译。`pnpm run gen-translation-brief <pair>` 会以能安全对齐的最窄粒度——先是有改动的 Markdown 单元，再是标题小节，最后是整篇文档——机械地汇集这次更新的工作集：被改一侧自上次确认以来的 diff、每个改动块的三方文本、改动触及的术语表行，以及有约束力的更新规则；仅落在配对中逐字节一致的围栏代码块内的改动可以直接算出，`--apply` 则经结构签名校验后把它拼接进对侧文件（[briefed-updates Agent Note](../../.agents/notes/implemented/process/2026-07-26-briefed-minimal-translation-updates.md)）。两侧对齐后，`pnpm run verify-translation-pairing --write <pair>` 重新记录两个 hash；那份 yaml diff 就是「确认一致」这个动作本身，可以被评审，也正因如此，`--write` 要求点名你确认过的配对（`--write --all` 是显式的全语料形式）。
|
||||
用 blob hash 而不是 commit hash，这样同一个 PR 里改动的文件也能算出记录（`git hash-object foo.md`），一致性是纯内容比较。`--write` 会先把这些快照存入本地 Git 对象库再写下记录，未提交的工作树内容也不例外；它还会在内容寻址的 `refs/dsh/translation-pairing/snapshots/` ref 下固定每个不同的已存 blob，使垃圾回收无法让已记录的恢复指针失效。因此记录的 hash 能还原任一侧上次确认时的确切文本，所以失去同步的配对是「按被改一侧的 diff 最小化地修补另一侧」，从不整篇重译。`pnpm run gen-translation-brief <pair>` 会以能安全对齐的最窄粒度——先是有改动的 Markdown 单元，再是标题小节，最后是整篇文档——机械地汇集这次更新的工作集：被改一侧自上次确认以来的 diff、每个改动块的三方文本、改动触及的术语表行，以及有约束力的更新规则；仅落在配对中逐字节一致的围栏代码块内的改动可以直接算出，`--apply` 则经结构签名校验后把它拼接进对侧文件（[briefed-updates Agent Note](../../.agents/notes/implemented/process/2026-07-26-briefed-minimal-translation-updates.md)）。两侧对齐后，`pnpm run verify-translation-pairing --write <pair>` 重新记录两个 hash；那份 yaml diff 就是「确认一致」这个动作本身，可以被评审，也正因如此，`--write` 要求点名你确认过的配对（`--write --all` 是显式的全语料形式）。
|
||||
- **语言切换行。** 两个文件在各自 H1 标题之后立即互链：英文文件带 `English | [中文](foo.zh.md)`，中文文件带 `[English](foo.md) | 中文`。
|
||||
- **结构与另一侧一一对应。** 标题深度与顺序、列表类型、有序列表起始编号、列表项数量、表格行列数、链接目标与逐字节一致的代码块在配对两侧一一对应；完整保持规则见 [translation-rules.md](translation-rules.md)。既有 Markdown 门禁对 `.zh.md` 文件原样生效（`verify-md-wrap`、`verify-md-links`）。
|
||||
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -0,0 +1,50 @@
|
||||
/** Git-blob operations owned by the bilingual pairing workflow. */
|
||||
|
||||
import { spawnSync } from 'node:child_process'
|
||||
import { createHash } from 'node:crypto'
|
||||
|
||||
const SNAPSHOT_REF_PREFIX = 'refs/dsh/translation-pairing/snapshots'
|
||||
|
||||
/** Full SHA-1 Git blob hash (the 40-hex format used by pairing records). */
|
||||
export function gitBlobHash(content: Buffer): string {
|
||||
const hash = createHash('sha1')
|
||||
hash.update(`blob ${content.byteLength}\0`)
|
||||
hash.update(content)
|
||||
return hash.digest('hex')
|
||||
}
|
||||
|
||||
function runGit(root: string, args: string[], operation: string, input?: Buffer): Buffer {
|
||||
const result = spawnSync('git', ['-C', root, ...args], {
|
||||
input,
|
||||
maxBuffer: 1 << 26,
|
||||
})
|
||||
if (result.error) {
|
||||
throw new Error(`${operation} failed: ${result.error.message}`, { cause: result.error })
|
||||
}
|
||||
if (result.status !== 0) {
|
||||
throw new Error(`${operation} failed with status ${String(result.status)}: ${result.stderr.toString('utf8').trim()}`)
|
||||
}
|
||||
return result.stdout
|
||||
}
|
||||
|
||||
/**
|
||||
* Persist exact working-tree bytes so a pairing record can later recover them
|
||||
* with `git cat-file`, even when they have never appeared in the index or a
|
||||
* commit. The returned object ID is checked against the pairing format's own
|
||||
* content hash before the caller writes a sidecar.
|
||||
*/
|
||||
export function storeGitBlob(root: string, content: Buffer): string {
|
||||
const expected = gitBlobHash(content)
|
||||
const stored = runGit(root, ['hash-object', '-w', '--stdin'], 'git hash-object -w --stdin', content)
|
||||
.toString('utf8')
|
||||
.trim()
|
||||
if (stored !== expected) {
|
||||
throw new Error(`git hash-object -w --stdin returned unexpected object ID ${JSON.stringify(stored)}; expected ${expected}`)
|
||||
}
|
||||
runGit(
|
||||
root,
|
||||
['update-ref', `${SNAPSHOT_REF_PREFIX}/${stored}`, stored],
|
||||
'git update-ref for translation snapshot',
|
||||
)
|
||||
return stored
|
||||
}
|
||||
@@ -1,6 +1,11 @@
|
||||
/** Regression tests for the bilingual corpus scope and structural signature. */
|
||||
/** Regression tests for bilingual snapshots, corpus scope, and structure. */
|
||||
|
||||
import { execFileSync, spawnSync } from 'node:child_process'
|
||||
import { mkdtempSync, rmSync } from 'node:fs'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { gitBlobHash, storeGitBlob } from './translation-pairing-git.ts'
|
||||
import {
|
||||
isTranslationScopeFile,
|
||||
pairAnchorOfArgument,
|
||||
@@ -15,6 +20,71 @@ function signature(markdown: string) {
|
||||
return translationStructureSignature(parseTranslationMarkdown(markdown), 'counterpart.zh.md')
|
||||
}
|
||||
|
||||
function gitSupportsObjectFormat(format: 'sha256'): boolean {
|
||||
const root = mkdtempSync(join(tmpdir(), 'dsh-git-object-format-'))
|
||||
try {
|
||||
return spawnSync('git', ['init', '--quiet', `--object-format=${format}`, root], {
|
||||
stdio: 'ignore',
|
||||
}).status === 0
|
||||
} finally {
|
||||
rmSync(root, { recursive: true, force: true })
|
||||
}
|
||||
}
|
||||
|
||||
const supportsSha256ObjectFormat = gitSupportsObjectFormat('sha256')
|
||||
|
||||
describe('translation pairing snapshots', () => {
|
||||
it('stores exact uncommitted bytes for later recovery by object ID', () => {
|
||||
const root = mkdtempSync(join(tmpdir(), 'dsh-translation-pairing-'))
|
||||
try {
|
||||
execFileSync('git', ['init', '--quiet', root], {
|
||||
env: { ...process.env, GIT_DEFAULT_HASH: 'sha1' },
|
||||
})
|
||||
const content = Buffer.from([0x75, 0x6e, 0x63, 0x6f, 0x6d, 0x6d, 0x69, 0x74, 0x74, 0x65, 0x64, 0x0a, 0xff])
|
||||
|
||||
const objectId = storeGitBlob(root, content)
|
||||
|
||||
expect(objectId).toBe(gitBlobHash(content))
|
||||
expect(execFileSync('git', [
|
||||
'-C', root, 'rev-parse', `refs/dsh/translation-pairing/snapshots/${objectId}`,
|
||||
], { encoding: 'utf8' }).trim()).toBe(objectId)
|
||||
execFileSync('git', ['-C', root, 'gc', '--prune=now'])
|
||||
expect(execFileSync('git', ['-C', root, 'cat-file', '-p', objectId])).toEqual(content)
|
||||
} finally {
|
||||
rmSync(root, { recursive: true, force: true })
|
||||
}
|
||||
})
|
||||
|
||||
it('fails before a sidecar can reference an unavailable object', () => {
|
||||
const root = mkdtempSync(join(tmpdir(), 'dsh-translation-pairing-'))
|
||||
try {
|
||||
expect(() => storeGitBlob(root, Buffer.from('snapshot'))).toThrow('git hash-object -w --stdin failed')
|
||||
} finally {
|
||||
rmSync(root, { recursive: true, force: true })
|
||||
}
|
||||
})
|
||||
|
||||
it('fails clearly when Git cannot be started', () => {
|
||||
const previousPath = process.env.PATH
|
||||
try {
|
||||
process.env.PATH = ''
|
||||
expect(() => storeGitBlob('.', Buffer.from('snapshot'))).toThrow('git hash-object -w --stdin failed')
|
||||
} finally {
|
||||
process.env.PATH = previousPath
|
||||
}
|
||||
})
|
||||
|
||||
it.skipIf(!supportsSha256ObjectFormat)('rejects an object format that pairing records cannot represent', () => {
|
||||
const root = mkdtempSync(join(tmpdir(), 'dsh-translation-pairing-'))
|
||||
try {
|
||||
execFileSync('git', ['init', '--quiet', '--object-format=sha256', root])
|
||||
expect(() => storeGitBlob(root, Buffer.from('snapshot'))).toThrow('returned unexpected object ID')
|
||||
} finally {
|
||||
rmSync(root, { recursive: true, force: true })
|
||||
}
|
||||
})
|
||||
})
|
||||
|
||||
describe('translation pairing manifest', () => {
|
||||
it('accepts an exclusions-only manifest', () => {
|
||||
expect(parseTranslationPairingManifest(JSON.stringify({
|
||||
|
||||
@@ -9,9 +9,9 @@
|
||||
* See `docs/i18n/README.md` for the owning contract.
|
||||
*/
|
||||
|
||||
import { createHash } from 'node:crypto'
|
||||
import { existsSync, globSync, readFileSync, writeFileSync } from 'node:fs'
|
||||
import { basename, join, resolve, sep } from 'node:path'
|
||||
import { gitBlobHash, storeGitBlob } from './translation-pairing-git.ts'
|
||||
import {
|
||||
linksTo,
|
||||
parseTranslationMarkdown,
|
||||
@@ -54,14 +54,6 @@ function isExcluded(file: string): boolean {
|
||||
return manifest.excluded.some(entry => (entry.endsWith('/') ? file.startsWith(entry) : file === entry))
|
||||
}
|
||||
|
||||
/** Full git blob hash (what `git hash-object` prints). */
|
||||
function blobHash(content: Buffer): string {
|
||||
const hash = createHash('sha1')
|
||||
hash.update(`blob ${content.byteLength}\0`)
|
||||
hash.update(content)
|
||||
return hash.digest('hex')
|
||||
}
|
||||
|
||||
/** The three paths of a pair, derived from the English-file path. */
|
||||
function pairPaths(source: string): { zh: string; meta: string } {
|
||||
return { zh: source.replace(/\.md$/, '.zh.md'), meta: source.replace(/\.md$/, '.i18n.yaml') }
|
||||
@@ -148,7 +140,12 @@ if (writeMode) {
|
||||
}
|
||||
continue
|
||||
}
|
||||
const record = renderMeta(source, blobHash(readFileSync(join(root, source))), zh, blobHash(readFileSync(join(root, zh))))
|
||||
const sourceContent = readFileSync(join(root, source))
|
||||
const zhContent = readFileSync(join(root, zh))
|
||||
// A consistency record is also a recovery pointer for the briefing
|
||||
// generator. Persist both snapshots even when the sidecar text is already
|
||||
// current, because the bytes may exist only in this working tree.
|
||||
const record = renderMeta(source, storeGitBlob(root, sourceContent), zh, storeGitBlob(root, zhContent))
|
||||
if (existsSync(join(root, meta)) && readFileSync(join(root, meta), 'utf8') === record) continue
|
||||
writeFileSync(join(root, meta), record)
|
||||
console.log(`verify-translation-pairing: recorded ${meta}`)
|
||||
@@ -203,7 +200,7 @@ for (const source of [...pairAnchors].sort()) {
|
||||
|
||||
let consistent = true
|
||||
for (const [file, content] of [[source, sourceContent], [zh, zhContent]] as const) {
|
||||
const current = blobHash(content)
|
||||
const current = gitBlobHash(content)
|
||||
if (record.get(basename(file)) !== current) {
|
||||
errors.push(`${file}: out of sync — content no longer matches the pair's last confirmed-consistent state in ${meta} (bring the other side along, then re-record with --write)`)
|
||||
consistent = false
|
||||
|
||||
Reference in New Issue
Block a user