diff --git a/CHANGES.md b/CHANGES.md index 80b63a3..f9a722d 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -59,6 +59,53 @@ concern - readable here, never shipped as something to parse. --- +## 6.0.1 - 2026-09-16 - docs toc/verify erreichen die .template-Form einer Referenzdatei + +**Author:** Torben Nehmer + + +**High impact** +- docs toc/verify erreichen die .template-Form einer Referenzdatei + + +### docs toc/verify erreichen die .template-Form einer Referenzdatei + +`kb/CONVENTIONS.md.template` war 105 Zeilen lang und trug keine TOC-Region. `toc.target_files()` +berechnete den Dateisatz ueber die *adoptierten* Namen, und eine Datei auf `.md.template` faellt +aus jedem dieser Walks heraus - also hat `docs toc --apply` das Template nie angefasst und +`docs verify` es nie gelesen. Eine Instanz, die es nach `instructions/setup-instance.md` +adoptiert, bekam damit eine `kb/CONVENTIONS.md` ohne Region und fiel am `docs verify` in +Schritt 13 derselben Anleitung um - dem Befehl, mit dem das Setup endet. Ausgeliefert war das in +`6.0.0`. + +Eine in Scope stehende Datei nimmt ihr `.template` jetzt mit hinein: das Template ist +dasselbe Dokument einen Schritt frueher in seinem Leben, und wer es auslaesst, laesst die +adoptierte Kopie den Fehler erben. `docs verify` prueft im Ursprungs-Repo damit 57 statt 56 +Referenzdateien, in einer frisch exportierten Instanz 59. + +Ausgeloest hat es ein Wachstum um sechs Zeilen: `f350999` hat das Template von 99 auf 105 Zeilen +gebracht und damit ueber die Schwelle von 100. Seither war `ci.yml` auf jedem Push rot (Laeufe +279 bis 289) - was als Flackern gelesen wurde, weil jeder Push zusaetzlich einen gruenen +`release.yml`-Lauf erzeugt und die Paare wie Lauf und Wiederholung aussehen. Sie sind zwei +verschiedene Workflows. + +Grenzuebertritt-Frage geprueft und verneint, gegen den dokumentierten Update-Weg: das Template ist +stack-eigen (`ownership.is_stack_owned` - jede `.template` unter einer Content-Stage), steht nicht +in `UPGRADE_PRESERVED_PATHS`, und `dist upgrade` schreibt es damit mit. Eine Instanz bekommt das +reparierte Template also durch den Upgrade selbst, ohne Handarbeit; der Rueckweg funktioniert +ebenso, weil die alte Maschinerie das Template gar nicht erst prueft. Handarbeit faellt nur an, wo +eine Instanz ihr stack-eigenes Template lokal veraendert hat - `dist upgrade` meldet genau das als +`blocked` und verlangt `--keep-local`. + +Verifiziert: `docs verify`/`instructions verify` gruen, 1275 Tests gruen (3 neu: das Template einer +in Scope stehenden Datei steht im Dateisatz, ein `.template` ohne solche Datei daneben nicht +(`USER.md.template`), und ein Template ueber der Schwelle ohne Region ist ein Befund - der letzte +waere am heutigen Stand rot gewesen). Dazu der vollstaendige `setup-instance.md`-Replay gegen einen +frischen `dist export`: `doctor`, `docs verify`, `instructions verify` und `lint` laufen in der +frischen Instanz durch. + +--- + ## 6.0.0 - 2026-09-15 - search: Pfad und Titel vollstaendig, Trunkierung sichtbar **Author:** Torben Nehmer diff --git a/VERSION b/VERSION index 09b254e..5fe6072 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -6.0.0 +6.0.1 diff --git a/instructions/dev/doc-pull-through.md b/instructions/dev/doc-pull-through.md index 047bfc0..f8b1d3c 100644 --- a/instructions/dev/doc-pull-through.md +++ b/instructions/dev/doc-pull-through.md @@ -43,7 +43,8 @@ touched; a row that does not apply needs no action. 3. **A heading you changed means a table of contents to regenerate - by the tool, never by hand.** Every reference file over 100 lines carries one (`AGENTS.md`, the stage contracts, `kb/CONVENTIONS.md`, each `COLLECTION.md`, the flat `instructions/**.md` form, the - type-specs, the `docs/` pages - a `SKILL.md` is the one exception). Adding, renaming, + type-specs, the `docs/` pages - each with the `.template` it ships as, where one + exists, and a `SKILL.md` the one exception). Adding, renaming, reordering or deleting a `##`/`###` heading in one of them makes its region stale, and `docs verify` fails on stale exactly as it fails on missing: diff --git a/kb/CONVENTIONS.md.template b/kb/CONVENTIONS.md.template index 95b4639..4b5236b 100644 --- a/kb/CONVENTIONS.md.template +++ b/kb/CONVENTIONS.md.template @@ -25,6 +25,18 @@ The frontmatter above is the one machine-read part. `sections:` names the headin generated regions render under. Safe to change at any time - each region is located by its marker pair, so a rename re-renders words and nothing else. + +## Contents + +- [Language](#language) + - [Section headings](#section-headings) +- [Naming](#naming) +- [Tone](#tone) +- [Relationship labels](#relationship-labels) +- [Hedging](#hedging) +- [Keeping this file honest](#keeping-this-file-honest) + + ## Language Pages are written in **{language}** - the `language:` in this file's own frontmatter, and the diff --git a/tools/CONTRACT.md b/tools/CONTRACT.md index 5c0dc1e..c908963 100644 --- a/tools/CONTRACT.md +++ b/tools/CONTRACT.md @@ -171,7 +171,7 @@ tools/wikitool --help | `instructions verify` | Check the instruction layer: flat instructions validate against `types/instruction.schema.yaml`, each `SKILL.md` carries the frontmatter its harness reads, no `SKILL.md` carries a relative markdown link (`sync` copies it to a different depth than the source, so a `SKILL.md` references a target as a repo-root-relative plain path instead - see [instructions/CONTRACT.md](../instructions/CONTRACT.md) § "A skill's outbound reference is a plain path, not a link"), every published copy is byte-identical to its source, no instruction is left that nothing references, and nothing under `instructions/dev/` is referenced from outside it (a `` block is exempt - see [instructions/CONTRACT.md](../instructions/CONTRACT.md)). Missing *every* copy is reported as "run sync", not as drift - that is a clean checkout | | `instructions list [--json]` | List the flat instructions with their descriptions. This is how the layer is discovered; `search` deliberately covers `kb/` only | | `docs verify` | Check the docs that mirror the code: every CLI command documented in this file's own § Commands table and, separately, in its § Error contracts table (both directions, checked per table, so a row dropped from one is not hidden by the same name surviving in the other, and only a name's presence in a row is checked, never the rest of that row's text), every directory under `kb/` has a `COLLECTION.md` and no directory outside it does, every collection declaring `profile:` and a `required_by_stack:` that agrees with the stack's own list, `kb/CONVENTIONS.md` naming all three tool-owned section headings if it exists at all, every stage contract present, every file under `types/` declaring `type: types/type-spec.md` validating against `types/type-spec.schema.yaml`, no pre-migration `type: entity` blocks left in the contracts, the `.gitignore` canaries clear in both directions (nothing ignored under `raw/`/`kb/`, `incoming/` ignored, everything ignored under `reports/` and the published skill directories), and no `.md`/`.template` file `dist export` would ship citing an issue number - the tracker exists only in the origin repo, so such a number in a distributed instance is a reference its reader can neither resolve nor recognise as unresolvable (a `` region is exempt: it is already gone from the text the check reads, which is the export plan's, not the working tree's), every reference file `docs toc` covers carrying the current table-of-contents region for its own headings - missing and stale are one check, because the generator is idempotent - and every relative markdown link in one of those same reference files resolving to a file that actually exists (a target's `#anchor` suffix is stripped first; code fences and inline code spans are masked before scanning, so a passage showing link syntax as an example is not mistaken for a real reference). The name is about documentation parity, not about the `docs/` directory - it neither reads nor requires one, the same way `kb/` predates the collection it now checks | -| `docs toc [--apply]` | Create, refresh or remove the generated table-of-contents region (`` ... ``, placed after the title and before the first `##`) on every reference file over 100 lines, in the scope Anthropic's skill-authoring guidance names for a file previewed rather than read in full: `AGENTS.md`, every stage contract, `kb/CONVENTIONS.md`, every `kb/*/COLLECTION.md`, every flat `instructions/**.md` file, every `types/*.md` type-spec, and every `docs/` page. Computed from those categories rather than listed, so a file added later is in scope without a code change. `SKILL.md` is the one exception, and the same guidance is why: it places a skill body on the loading level that is read whole when the skill triggers, and aims its own TOC advice at the bundled reference files a skill points *at*. Human docs (`README.md`, `CHANGES.md`, `EVALS.md`, `INSTALL.md`, `tools/README.md`) are out of scope because AGENTS.md § File naming says no agent loads them as instruction. Dry-run by default (prints which files would change); `--apply` writes. `docs verify` checks the result stays current the same way it checks every other generated-from-code copy | +| `docs toc [--apply]` | Create, refresh or remove the generated table-of-contents region (`` ... ``, placed after the title and before the first `##`) on every reference file over 100 lines, in the scope Anthropic's skill-authoring guidance names for a file previewed rather than read in full: `AGENTS.md`, every stage contract, `kb/CONVENTIONS.md`, every `kb/*/COLLECTION.md`, every flat `instructions/**.md` file, every `types/*.md` type-spec, and every `docs/` page - each together with the `.template` it ships as, where one exists. Computed from those categories rather than listed, so a file added later is in scope without a code change. A template is in scope because it is the same document one step earlier in its life: an instance adopts it by copying it back, so a region missing there is a region missing in the adopted file, which is how `kb/CONVENTIONS.md.template` came to grow past the threshold with no region and left every instance adopting it failing `docs verify` at the end of its own setup. `SKILL.md` is the one exception, and the same guidance is why: it places a skill body on the loading level that is read whole when the skill triggers, and aims its own TOC advice at the bundled reference files a skill points *at*. Human docs (`README.md`, `CHANGES.md`, `EVALS.md`, `INSTALL.md`, `tools/README.md`) are out of scope because AGENTS.md § File naming says no agent loads them as instruction. Dry-run by default (prints which files would change); `--apply` writes. `docs verify` checks the result stays current the same way it checks every other generated-from-code copy | ### Telemetry diff --git a/tools/chemenu/commands/docs_verify.py b/tools/chemenu/commands/docs_verify.py index ecfc1bc..f829b69 100644 --- a/tools/chemenu/commands/docs_verify.py +++ b/tools/chemenu/commands/docs_verify.py @@ -511,11 +511,13 @@ def check_toc_regions() -> list[str]: MARKDOWN_LINK_RE = re.compile(r"\[[^\]]*\]\(([^)\s]+)\)") # The suffix `dist export` re-keys an instance-owned file to, and the one -# `setup-instance.md` renames away again. Spelled here rather than imported -# from `ownership`, whose own `.template` handling answers a different -# question (which side an upstream merge keeps) over a narrower scope -# (paths under a content stage). -TEMPLATE_SUFFIX = ".template" +# `setup-instance.md` renames away again. Imported from `toc` rather than +# spelled again here: that module already decides which files are reference +# material in both their forms, and this check runs over its scope. Not from +# `ownership`, whose own `.template` handling answers a different question +# (which side an upstream merge keeps) over a narrower scope (paths under a +# content stage). +TEMPLATE_SUFFIX = toc.TEMPLATE_SUFFIX def is_external_or_anchor(target: str) -> bool: diff --git a/tools/chemenu/tests/test_docs_verify.py b/tools/chemenu/tests/test_docs_verify.py index 0fd9c56..f15cc91 100644 --- a/tools/chemenu/tests/test_docs_verify.py +++ b/tools/chemenu/tests/test_docs_verify.py @@ -571,6 +571,33 @@ def test_every_reference_files_toc_is_current(): assert docs_verify.check_toc_regions() == [] +def test_a_shipped_template_over_the_threshold_without_a_region_is_reported(tmp_path, monkeypatch): + """Regression guard for the defect this scope extension fixes: the template + an instance adopts was maintained by nothing and checked by nothing, so + `kb/CONVENTIONS.md.template` grew past the threshold carrying no region - + and every instance that adopted it got a `kb/CONVENTIONS.md` that fails + `docs verify` at the end of `setup-instance.md`, the one command that step + ends with. Before the template entered `toc.target_files()`, this check + returned nothing here.""" + from chemenu import config, toc as toc_mod + + monkeypatch.setattr(config, "ROOT", tmp_path) + (tmp_path / "kb").mkdir() + long_body = "# Conventions\n\nIntro.\n" + "".join( + f"\n## Section {i}\n\n" + "Body line.\n" * 12 for i in range(8) + ) + assert toc_mod.needs_toc(long_body) + # The adopted file is current; only the template it was adopted from is not. + (tmp_path / "kb" / "CONVENTIONS.md").write_text(toc_mod.upsert(long_body), encoding="utf-8") + (tmp_path / "kb" / "CONVENTIONS.md.template").write_text(long_body, encoding="utf-8") + + issues = docs_verify.check_toc_regions() + + assert len(issues) == 1 + assert "kb/CONVENTIONS.md.template" in issues[0] + assert "docs toc --apply" in issues[0] + + def test_no_shipped_document_cites_an_issue(): """Forward direction, against the real tree: a `#42` in a file `dist export` ships points at a board only the origin repo has, and the reader of a diff --git a/tools/chemenu/tests/test_toc.py b/tools/chemenu/tests/test_toc.py index e4b1a4c..34ca57d 100644 --- a/tools/chemenu/tests/test_toc.py +++ b/tools/chemenu/tests/test_toc.py @@ -170,6 +170,38 @@ def test_target_files_matches_the_documented_scope(): assert not any(rel.endswith("SKILL.md") for rel in relatives) +def test_target_files_takes_the_shipped_template_of_a_file_in_scope(): + """Integration check against the real repo: `kb/CONVENTIONS.md` is + instance-owned and crosses the distribution boundary as + `kb/CONVENTIONS.md.template`, so the template is the same document one step + earlier in its life and is maintained here too. Left out, nothing wrote the + region into it and nothing checked it - and the adopted copy inherited the + gap, which is how a fresh instance came to fail `docs verify` at the end of + its own setup.""" + from chemenu import config + + relatives = {f.relative_to(config.ROOT).as_posix() for f in toc.target_files()} + + assert "kb/CONVENTIONS.md.template" in relatives + assert "kb/CONVENTIONS.md" in relatives # both forms, not one replacing the other + + +def test_target_files_takes_a_template_only_for_a_file_already_in_scope(tmp_path, monkeypatch): + """The rule is "a file in scope carries its `.template` in with it", not + "every `.template` in the tree" - `USER.md.template` has no business here, + because `USER.md` itself is not reference material this region applies to.""" + from chemenu import config + + monkeypatch.setattr(config, "ROOT", tmp_path) + (tmp_path / "kb").mkdir() + for name in ("kb/CONVENTIONS.md", "kb/CONVENTIONS.md.template", "USER.md.template"): + (tmp_path / name).write_text("# A file\n", encoding="utf-8") + + relatives = {f.relative_to(tmp_path).as_posix() for f in toc.target_files()} + + assert relatives == {"kb/CONVENTIONS.md", "kb/CONVENTIONS.md.template"} + + def test_target_files_lists_type_spec_once_despite_two_sources(): """`types/type-spec.md` is both a stage contract and a `types/*.md` file; the set in `target_files` is what keeps that from being a special case.""" diff --git a/tools/chemenu/toc.py b/tools/chemenu/toc.py index 4b573f3..a9ab3fa 100644 --- a/tools/chemenu/toc.py +++ b/tools/chemenu/toc.py @@ -23,7 +23,8 @@ precedent first. governs `wikitool` itself. `target_files()` walks every file-naming category AGENTS.md's own table calls agent-loaded: `AGENTS.md`, every stage contract, `kb/CONVENTIONS.md`, every `kb/*/COLLECTION.md`, every flat `instructions/**.md` -file, every `types/*.md` type-spec, and every `docs/` page. One rule, and +file, every `types/*.md` type-spec, and every `docs/` page - each together with +the `.template` it ships as, where one exists. One rule, and exactly one exception below it - which is the whole point, because a scope carrying several unexplained absences reads as an accident rather than a decision, and did: `docs/` and the page type-specs sat outside it for no @@ -75,6 +76,14 @@ from chemenu import blocks, config, kb_collections, markdown_code REGION_NAME = "toc" HEADING_TEXT = "Contents" +# The suffix `dist export` re-keys an instance-owned file to, and the one +# `setup-instance.md` adopts away again. Lives here because this module is what +# decides which files are reference material *in both their forms*; +# `docs_verify` imports it rather than keeping a second spelling. `ownership.py` +# keeps its own literal deliberately - it answers a different question (which +# side an upstream merge keeps) over a narrower scope. +TEMPLATE_SUFFIX = ".template" + # The line threshold Anthropic's own guidance names. Measured on the body # with any existing TOC region stripped out first, so inserting or updating # the region can never be what pushes a file over the line. @@ -120,6 +129,18 @@ def target_files() -> list[Path]: `types/type-spec.md` arrives twice - once as a stage contract, once from the `types/` walk - and the set at the bottom is what makes that a non-issue rather than something to special-case. + + **A file in scope carries its `.template` in with it.** An instance-owned + reference file crosses the distribution boundary as `.template` and + is adopted by copying it back (`instructions/setup-instance.md`), so the + template is the same document one step earlier in its life - the shipped + form of a file this scope already covers. Leaving it out meant nothing + maintained it and nothing checked it: `kb/CONVENTIONS.md.template` grew + past the threshold carrying no region at all, and the first thing to notice + was a fresh instance failing `docs verify` at the end of its own setup, + because the adopted copy inherited the gap. `docs toc --apply` writes the + region into the template as into anything else here; the instance's own + later edits stay its business, and its own `docs toc` run answers for them. """ files: list[Path] = [config.ROOT / "AGENTS.md"] files += [config.ROOT / rel for rel in _STAGE_CONTRACTS] @@ -139,6 +160,13 @@ def target_files() -> list[Path]: directory = config.ROOT / subdir if directory.is_dir(): files += sorted(directory.rglob("*.md")) + files += [ + template + for template in ( + path.with_name(path.name + TEMPLATE_SUFFIX) for path in tuple(files) + ) + if template.is_file() + ] return sorted({f for f in files if f.is_file()})