docs: record the directory-listing and glob-sampling decision

Agent Note for the two changes: what the session log showed, what ordering
can and cannot fix (measured, correcting the first diagnosis), why the page
is sampled only past the cap, why `list` is needed alongside it, and the
alternatives each one beat.
This commit is contained in:
NI0317
2026-07-28 12:06:25 +08:00
parent 9b9b45e65e
commit a9c0e00620
3 changed files with 226 additions and 0 deletions
@@ -0,0 +1,6 @@
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-27-directory-listing-tool.md
2026-07-27-directory-listing-tool.md: 4292c173f07064ff8825462e08096780c20ad9d6
2026-07-27-directory-listing-tool.zh.md: 4033254b08a77ba7ed9b1f3e638213f019abb692
@@ -0,0 +1,110 @@
# Agent Note: Sample over-cap glob results across the tree, and give the model a directory-listing tool
Status: implemented
English | [中文](2026-07-27-directory-listing-tool.zh.md)
## Problem
Asked what a workspace contained, an agent described one subfolder as if it were the whole project.
The session log shows exactly how. The workspace held 22 top-level entries and 11,485 files. The model called `glob {"pattern": "*"}`, which matched 10,030 paths; the tool showed the first 100, and all 100 sat under a single recently-unpacked subdirectory holding 355 of those files. The model never saw the other 21 top-level entries and answered from the one it did see. The session cwd was correct throughout — nothing was misconfigured, and every number the tool printed was true.
Three properties of `glob` compose into that page:
- **A pattern with no `/` matches at any depth.** The pattern goes to ripgrep as `--glob=<pattern>`, where a glob without a separator matches the basename anywhere in the tree. `*` therefore means "every file in the workspace", not "the top level" — the opposite of what it means in a shell. The tool said nothing about this, and every example in its schema was `**/…`, so nothing suggested the plain form was recursive.
- **`--sort=modified` orders oldest first.** Unpacking an archive restores the timestamps stored inside it, which predate everything the user wrote, so a freshly unpacked subtree lands at the very front of any broad match. (Ripgrep sorts ascending; `--sortr` is the descending form and is not used here.)
- **The inline page was the head of that order.** `globMaxResults` (100) paths were kept from the front. Under the first two properties, one subtree takes every slot.
Each property is defensible alone. Together they make the most ordinary request an agent receives — "what is in this directory" — reliably produce a confident wrong answer, because nothing in the result distinguishes "the 100 newest files in this workspace" from "this workspace".
### What ordering can and cannot fix
A directory's name reaches the model only as the prefix of one of its files' paths, since `rg --files` emits files and never directory entries. That is enough for ordering to matter a great deal, and not enough for `glob` to answer the question.
Measured on a reproduction of the failure's shape — 24 top-level entries, 716 files, one recently-written subtree:
| First 100 paths chosen by | Distinct top-level names visible |
| --- | --- |
| modification time, oldest first (the shipped behavior) | 7 |
| round-robin across top-level entries | 21 |
So a differently chosen page does surface most of the missing names, and the original diagnosis that ordering could not have helped was wrong. What no ordering fixes: an entry with no files beneath it never appears at all (the reproduction's empty directory is absent from the complete 716-path output), and nothing in the output says which names are directories or how many entries a directory holds. `glob` can therefore convey a tree's rough shape; it cannot state a directory's contents.
## Decision
Two changes, in the two packages that own the two halves of the failure.
### The inline page of an over-cap `glob` result is sampled, not taken from the head
`@deepseek-ai/dsh-tool-fs-search`. A result within `globMaxResults` is unchanged: shown whole, in modification-time order. Only when the result is larger does the page change — and there, taking the head is what fails.
`sampleAcrossTopLevel` groups the complete result by leading path segment and fills the page round-robin: every top-level entry gets a slot before any entry gets a second, and an entry that runs out of paths drops out so its remaining slots go to the rest. Sort order survives where it still carries meaning — groups are visited in the order ripgrep first emits them, and each group's own paths keep their relative order. The page is emitted grouped by entry rather than interleaved, so the breadth is legible at a glance.
The footer states the basis, because a page that silently stopped being "the newest N" would be a second, quieter version of the same lie:
```
(Showing 100 of 10030 paths, sampled across 22 of the 22 top-level entries this pattern matched
instead of taken in modification-time order. Full sorted result stored at: …)
```
When the page cannot reach every top-level entry — more entries than slots — the footer says so with the shown/total spread and points at `list`. A flat result, where every path is its own top-level entry, keeps the plain `(Showing k of n paths. …)` footer: there the round-robin *is* the sorted head, and naming a spread would only restate the counts. The spill artifact always holds the complete list in modification-time order, so the sorted view is never lost.
The guidance and schema stop misleading in the same change: "not shell find or ls" becomes "not shell find"; both now state that a pattern without `/` matches basenames at any depth, that results are files and never directories, that a fitting result is modification-time ordered while a larger one is sampled, and that `list` is the tool for a directory's contents.
### `list`, in `@deepseek-ai/dsh-tool-fs`
A fourth model-facing filesystem tool over the existing `ctx.fs.listDir` primitive, which until now shipped with skill discovery as its only consumer; [the seam Agent Note](../../archived/architecture/2026-07-03-filesystem-directory-listing-seam.md) deferred the model-facing consumer to a separate decision, which this is.
It takes an optional `path` — defaulting to `.`, the calling agent's session workspace, so the common question needs no argument — and returns the direct children of one directory as `{ path, entries: [{ name, type }] }`, where `type` is `file`, `directory`, or `other`. It calls `listDir` and nothing else: the seam already reports absence as `FS_NOT_FOUND` and a non-directory target as `FS_NOT_DIRECTORY`, so a preceding `stat` would add a round-trip and a second source of truth.
Two presentation rules carry the decision:
- **Directories sort first, then files, then non-regular children, each alphabetically** — in the canonical value as well as the rendered text, so a Code Mode caller and the model see one ordering contract. The seam returns stable name order, which scatters subdirectories through the alphabet; capping such a list can drop every subdirectory and reproduce, inside `list`, the same blindness. Directory-first ordering makes truncation lose leaves, never structure.
- **The footer always states the complete listing's size and composition** — `(22 entries: 18 directories, 4 files)`, and when the view is capped at `listMaxEntries` (default 200, configurable), `(Showing 200 of 5000 entries: 12 directories, 4988 files. …)`. A partial listing therefore cannot read as a whole directory.
Directory entries render with a trailing `/` and non-regular children with `@`, so the model can tell what it may descend into without a second call.
`list` emits no `fs/observed`. Seeing a filename is not reading a file, and a listing must never satisfy the read-before-write gate that `@deepseek-ai/dsh-fs-policy` enforces. It declares `isConcurrencySafe`, because it mutates nothing at all.
### Why both
They answer different questions and neither substitutes for the other. `list` answers "what is here" exactly — entry names, their types, the complete count — which `glob` cannot do at any ordering. Sampling fixes the page a broad `glob` returns for every *other* question, which stays wrong even once a better tool exists, because the model has no reason to abandon a page that looks representative.
## Alternatives considered
**Leave `glob` ordering alone and only warn in the footer.** This is what the first implementation did: keep the recency head and add a clause saying the head covered 1 of 22 top-level entries. Rejected once measured. A warning asks the model to distrust the only data it has and go elsewhere; a better page just is not wrong. The warning also does nothing for a model that stops reading at the paths, which is the failure being fixed.
**Sample always, replacing modification-time order outright.** Rejected. Over a complete result the order answers age questions — what is stale, what was touched last — a genuinely useful and separate purpose, and a complete result is the case where the order costs nothing and means everything. Sampling only past the cap is the point where the order has already stopped describing the result: the head of a 10,030-path list is not "the oldest files worth knowing about", it is an arbitrary 1% of them.
**Sample by a skew threshold — head unless the head is badly concentrated.** Rejected. A threshold is a deployment-varying tunable with no evidence behind any value, and it makes the result's ordering contract conditional on data the model cannot see. "Over the cap" is a boundary the model already knows about from the footer.
**Balance recursively, not just at the top level.** Deferred, and recorded as a Known Limitation. Top-level balance fixes the observed failure and is explainable in one sentence of tool description; per-level balancing needs a policy for how depth trades against breadth, which no current evidence settles.
**Reject `*`, or silently rewrite it to a top-level-anchored pattern.** Rejected. The same basename-at-any-depth rule that makes `*` recursive is what makes `*.ts` mean "every TypeScript file", the overwhelmingly common and correct use; anchoring one and not the other is an arbitrary special case, and rejecting a pattern ripgrep accepts turns a working call into an error. Documenting the rule costs nothing and generalizes.
**Add `list` without touching `glob`.** Rejected, for the reason stated under *Why both* above. The misleading page is reachable from any broad pattern, and a model that believes its sample is representative has no reason to reach for another tool.
**Fix `glob` without adding `list`.** Rejected for the same reason in reverse. A sampled page shows most top-level *names*, but not which are directories, not the empty ones, and not the entry count; "what is in this directory" deserves a tool that answers it rather than a sample the model must infer from.
**Report per-directory child counts in `list`.** Rejected. Counting each entry's children means one `listDir` per child — an N+1 fan-out across a seam that may be remote or sandboxed, paid on every listing, to sharpen a decision the model can settle by listing the one subdirectory it cares about.
**Make `list` recursive with a depth argument.** Rejected for now. One level composes: the model lists what it needs to descend into. Recursion reintroduces the size and truncation problems this note exists to fix, and the provider primitive is deliberately one-level.
**Spill a capped listing through `ctx.spillStore`, as `glob` does.** Rejected for v1. `glob` needs spill because a truncated path list has no cheap successor call; a truncated listing does, and the footer states the complete size and composition, so the model knows both that it is looking at part of a directory and what to do about it.
## Consequences
An over-cap `glob` result no longer returns the most recently modified paths. That is a real contract change on a hot path, and it is why the footer, the prompt section, and the schema all state the sampled basis rather than leaving the model to infer it. A result within the cap is byte-identical to before, so age-ordered reading keeps working wherever it was working.
Balancing is by first path segment only, so a result concentrated deeper — one enormous directory inside an otherwise even tree — is still shown unevenly below the top level. Recorded in the package's Known Limitations.
The shipped tool surface grows by one tool in every deployment that loads `@deepseek-ai/dsh-tool-fs`, which is all of them: a fixed schema and prompt cost on every request, and an invalidated pinned request header in the ACP snapshot scenario that pins full system-prompt and tool-schema content. The gain is that the harness's most common question has a correct answer; the previous state was not a missing convenience but a capability hole that produced confidently wrong answers.
`ctx.fs.listDir` gains its first model-facing consumer, which makes its contract load-bearing for a product surface: a future remote or sandboxed backend must implement direct-child listing well enough for a model to navigate by, not merely well enough for skill discovery. The `other` type stays collapsed at the seam, so `list` cannot distinguish a symlink from a socket and marks both `@`.
## Testing
Package tests pin the model-visible text of both surfaces. For `glob`: `sampleAcrossTopLevel` over a concentrated result, an exhausted group handing its slots on, a page smaller than the top level, absolute paths outside the workdir, and a flat result that must reproduce the sorted head; plus end-to-end assertions that a fitting result is untouched, that an over-cap result returns the sampled page with the sampled-basis footer, that the list hint disappears once the page reaches every entry, and that a flat over-cap result keeps the plain footer. For `list`: the envelope, type markers, singular and plural footers, the empty-directory footer, and the capped footer that keeps the sole directory visible. A registry-level test asserts that a listing emits no `fs/observed` and that a following `edit` still fails `FS_NOT_OBSERVED`, so the tool cannot become an accidental read-before-write bypass. Prompt-section registration, schema registration, and HMR disposal cover the fourth tool alongside the existing three.
The assembled transcript is the `fs-list` ACP scenario: a workspace whose answer is its subdirectories, where the model calls `list` with no arguments and the pinned tool result carries the directory-first envelope and its composition footer — an answer `glob` could not have produced at all.
@@ -0,0 +1,110 @@
# Agent Note: Sample over-cap glob results across the tree, and give the model a directory-listing tool
Status: implemented
[English](2026-07-27-directory-listing-tool.md) | 中文
## Problem
被问到工作区里有什么时,agent(智能体)把其中一个子目录当作整个项目描述了一遍。
会话日志把过程说得很清楚。该工作区有 22 个顶层条目、11485 个文件。模型调用了 `glob {"pattern": "*"}`,匹配到 10030 条路径;工具展示了前 100 条,而这 100 条全部位于同一个新近解包的子目录下,该子目录只占其中 355 个文件。模型从未看到另外 21 个顶层条目,只能依据它看到的那一个作答。整个过程中会话 cwd 都是正确的——没有任何配置错误,工具打印的每个数字也都是真的。
`glob` 的三项性质合成了那个页面:
- **不含 `/` 的 pattern 匹配任意深度。** pattern 直接作为 `--glob=<pattern>` 交给 ripgrep,而不含分隔符的 glob 匹配树中任何位置的基名。因此 `*` 的含义是「工作区里的每个文件」,而不是「顶层」——与它在 shell 里的含义正好相反。工具对此只字未提,schema 里的示例又全是 `**/…`,没有任何线索表明朴素写法是递归的。
- **`--sort=modified` 按从旧到新排序。** 解包压缩档会还原档案内部保存的时间戳,它们早于用户自己写下的一切,因此新近解包的子树会落在任何宽泛匹配的最前面。(ripgrep 按升序排序;降序是 `--sortr`,此处未使用。)
- **内联页面取的是该顺序的头部。** 从最前面保留 `globMaxResults`(100)条路径。在前两项性质之下,一个子树吃掉全部位置。
单看每一项都站得住。合在一起,它们让 agent 收到的最普通的请求——「这个目录里有什么」——稳定地产出一个笃定的错误答案,因为结果里没有任何信息能区分「本工作区最新的 100 个文件」与「本工作区」。
### 排序能修什么,不能修什么
由于 `rg --files` 输出文件、从不输出目录条目,一个目录的名字只能作为其下某个文件路径的前缀抵达模型。这既足以让排序变得非常重要,也不足以让 `glob` 回答那个问题。
在一份复现该失败形态的目录上实测——24 个顶层条目、716 个文件、一个新近写入的子树:
| 前 100 条路径的挑选方式 | 可见的顶层名个数 |
| --- | --- |
| 按修改时间、从旧到新(已交付的行为) | 7 |
| 跨顶层条目轮转 | 21 |
也就是说,换一种页面挑选方式确实能呈现出大部分缺失的名字,最初那句「排序帮不上忙」的诊断是错的。排序修不了的是:没有任何文件的条目根本不会出现(复现目录里的空目录在完整的 716 条输出中一次都没出现),而且输出里没有任何信息说明哪些名字是目录、某个目录有多少条目。因此 `glob` 能传达一棵树的大致形状,却说不出一个目录的内容。
## Decision
两项改动,分别落在承担这次失败两半责任的两个包中。
### 超过上限的 `glob` 结果,内联页面改为取样而非取头部
`@deepseek-ai/dsh-tool-fs-search`。未超过 `globMaxResults` 的结果完全不变:整体展示,按修改时间排序。只有结果更大时页面才改变——而恰恰在这种情况下,取头部是会失败的做法。
`sampleAcrossTopLevel` 按路径首段对完整结果分组,并以轮转方式填充页面:每个顶层条目都先拿到一个位置,任何条目才可能拿到第二个;某个条目的路径用尽后退出,其剩余份额交给其余条目。排序在仍有意义的地方被保留下来——分组按 ripgrep 首次输出它们的顺序访问,而每个分组内部的路径保持原有相对顺序。页面按条目分组输出而非交错输出,使覆盖广度一眼可辨。
footer 会说明取用依据,因为一个悄悄不再是「最新 N 条」的页面,只会成为同一个谎言更安静的版本:
```
(Showing 100 of 10030 paths, sampled across 22 of the 22 top-level entries this pattern matched
instead of taken in modification-time order. Full sorted result stored at: …)
```
当页面无法触达全部顶层条目时——条目数多于位置数——footer 会给出已触达/总计的分布并指向 `list`。扁平结果(每个路径各自构成一个顶层条目)保留朴素的 `(Showing k of n paths. …)` footer:此时轮转结果**就是**排序后的头部,说明分布只会重复计数。spill 产物始终保存按修改时间排序的完整列表,排序视图不会丢失。
同一次改动里,指导与 schema 也不再误导:「not shell find or ls」改为「not shell find」;两处现在都写明:不含 `/` 的 pattern 匹配任意深度的基名、结果只有文件从不包含目录、未超上限的结果按修改时间排序而更大的结果为取样所得、目录内容请用 `list`
### `list`,位于 `@deepseek-ai/dsh-tool-fs`
在既有的 `ctx.fs.listDir` 原语之上新增第四个面向模型的文件系统工具;该原语此前交付时唯一的消费方是 skill(技能)发现,[seam Agent Note](../../archived/architecture/2026-07-03-filesystem-directory-listing-seam.md) 把面向模型的消费方推迟为一项单独的决策,也就是本文。
它接受可选的 `path`,默认为 `.`,即调用 agent 的会话工作区,因此那个最常见的问题不需要任何参数;返回单个目录的直接子项,形如 `{ path, entries: [{ name, type }] }`,其中 `type``file``directory``other`。它只调用 `listDir`seam 已经把不存在报告为 `FS_NOT_FOUND`、把非目录目标报告为 `FS_NOT_DIRECTORY`,前置 `stat` 只会多一次往返并制造第二个真源。
有两条展示规则承载了这个决策:
- **先目录、再文件、最后非常规子项,各组内按字母序** —— 规范值与渲染文本采用同一顺序,使 Code Mode 调用方和模型看到同一份顺序契约。seam 返回的是稳定名称序,会把子目录散落在字母表各处;对这样的列表设上限可能丢掉全部子目录,在 `list` 内部重演同一种盲区。目录优先的顺序让截断只丢叶子,绝不丢结构。
- **footer 始终说明完整列表的规模与构成** —— 例如 `(22 entries: 18 directories, 4 files)`;视图受 `listMaxEntries`(默认 200,可配置)截断时则为 `(Showing 200 of 5000 entries: 12 directories, 4988 files. …)`。因此部分列出结果不可能被读成整个目录。
目录条目渲染带尾部 `/`,非常规子项带 `@`,使模型无需第二次调用就能判断哪些可以继续进入。
`list` 不发出 `fs/observed`。看到文件名不等于读过文件,列出绝不能满足 `@deepseek-ai/dsh-fs-policy` 施加的编辑前读取门禁。它声明 `isConcurrencySafe`,因为它完全不做任何变更。
### 为什么两者都要
它们回答的是不同的问题,彼此无法替代。`list` 精确回答「这里有什么」——条目名、类型、完整计数——这是 `glob` 在任何排序下都做不到的。取样修的是宽泛 `glob` 为**其他**所有问题返回的那个页面;即便有了更好的工具,那个页面依然是错的,因为模型没有理由放弃一个看起来有代表性的页面。
## Alternatives considered
**不动 `glob` 排序,只在 footer 里警告。** 这正是第一版实现的做法:保留新近序头部,加一句「该头部只覆盖 22 个顶层条目中的 1 个」。实测之后否决。一句警告是在要求模型不信任它手上唯一的数据并另寻他处;而一个更好的页面本身就不是错的。对于读到路径就停下的模型,警告也毫无作用——而那正是本次要修的失败。
**一律取样,彻底取消按修改时间排序。** 已否决。在完整结果上,该顺序回答的是与新旧有关的问题——哪些已经陈旧、哪些最后被动过——这是一个确实有用且独立的用途;而未超上限的完整结果恰恰是该顺序毫无代价、意义最大的场景。只在超过上限后取样,正好落在该顺序已经不再描述结果的那个点上:一份 10030 条列表的头部不是「最值得知道的最旧文件」,而是其中任意的 1%。
**按偏斜阈值决定是否取样——头部不太集中时仍取头部。** 已否决。阈值是随部署而变的可调参数,且其取值没有任何证据支撑;它还会让结果的顺序契约取决于模型看不到的数据。而「是否超过上限」是模型本来就能从 footer 得知的边界。
**递归均衡,而不只在顶层均衡。** 已延期,并记入已知限制。顶层均衡修好了已观察到的失败,且能用一句工具描述解释清楚;逐层均衡需要一套关于深度与广度如何权衡的策略,目前没有证据能定下来。
**拒绝 `*`,或悄悄将其改写为锚定顶层的 pattern。** 已否决。让 `*` 递归的那条「任意深度匹配基名」规则,同样让 `*.ts` 意为「每个 TypeScript 文件」,而后者是压倒性常见且正确的用法;只锚定其一是任意的特例,而拒绝一个 ripgrep 本可接受的 pattern 则把可用调用变成错误。把规则写进文档不花任何代价,而且可以推广。
**只加 `list`,不动 `glob`。** 已否决,理由见上文《为什么两者都要》。任何宽泛 pattern 都能触达那个误导性页面,而一个相信自己样本有代表性的模型,没有理由改用别的工具。
**只修 `glob`,不加 `list`。** 出于对称的理由否决。取样页面能展示大部分顶层**名字**,但说不出哪些是目录、看不到空目录、也给不出条目总数;「这个目录里有什么」值得一个直接回答它的工具,而不是一份需要模型去推断的样本。
**在 `list` 中报告每个目录的子项数。** 已否决。统计每个条目的子项意味着对每个子项各调用一次 `listDir`——在一个可能是远程或沙箱的 seam 上做 N+1 扇出,每次列出都要付费,只为了让模型少做一个它本可以通过列出所关心的那一个子目录就完成的判断。
**让 `list` 支持递归和深度参数。** 暂时否决。单层是可组合的:模型列出它需要进入的那一层即可。递归会重新引入本 Agent Note 要解决的规模与截断问题,而且提供方原语本身就有意只做一层。
**像 `glob` 那样,把达到上限的列出结果通过 `ctx.spillStore` 落盘。** v1 已否决。`glob` 需要 spill,是因为被截断的路径列表没有廉价的后继调用;被截断的列出结果有,而且 footer 已说明完整规模与构成,模型既知道自己只看到目录的一部分,也知道该怎么办。
## Consequences
超过上限的 `glob` 结果不再返回修改时间最新的那些路径。这是热路径上一项真实的契约变更,也正因如此,footer、提示词段和 schema 都写明了取样这一依据,而不是留给模型去推断。未超上限的结果与此前逐字节相同,因此按新旧顺序阅读结果的用法在原本能用的地方继续能用。
均衡只按路径首段进行,因此集中在更深层的结果——一棵总体均匀的树里某个特别庞大的目录——在顶层以下仍然分布不均。已记入该包的已知限制。
随产品交付的工具接口在每个加载 `@deepseek-ai/dsh-tool-fs` 的部署中都多出一个工具,而这是全部部署:每个请求都要支付固定的 schema 与提示词成本,并且会使 ACPAgent Client Protocol)快照场景中那份钉住完整系统提示词与工具 schema 内容的请求头失效。换来的是:harness 最常见的问题终于有了正确答案;此前的状态不是缺少一项便利,而是一个会产出笃定错误答案的能力空洞。
`ctx.fs.listDir` 拥有了第一个面向模型的消费方,其契约由此成为产品接口的承重部分:未来的远程或沙箱后端实现直接子项列出时,标准不再是「够 skill 发现用」,而是「够模型据以导航」。`other` 类型在 seam 处仍是合并的,因此 `list` 无法区分符号链接与套接字,两者都标记为 `@`
## Testing
包测试钉住两个接口面向模型的文本。`glob` 方面:`sampleAcrossTopLevel` 在集中结果、某分组用尽后交出份额、页面小于顶层条目数、工作目录之外的绝对路径,以及必须复现排序头部的扁平结果上的行为;另有端到端断言——未超上限的结果原样不动、超上限结果返回取样页面并带取样依据 footer、页面覆盖全部条目后 list 提示消失、扁平的超上限结果保留朴素 footer。`list` 方面:包络、类型标记、单复数 footer、空目录 footer,以及让唯一那个目录留在视野内的截断 footer。一项注册表层级的测试断言列出不会发出 `fs/observed`,且随后的 `edit` 仍以 `FS_NOT_OBSERVED` 失败,使该工具不会变成意外的编辑前读取绕过口。提示词段注册、schema 注册与 HMR(热模块替换)dispose(资源释放)覆盖了这第四个工具与既有三个工具。
组装后的 transcript(文本记录)由 `fs-list` ACP 场景承担:该工作区的答案就是它的子目录,模型不带任何参数调用 `list`,被钉住的工具结果携带目录优先的包络及其构成 footer —— 这个答案 `glob` 根本无法给出。