stack: TOC-Pflicht fuer Referenzdateien ueber 100 Zeilen (docs toc); session-setup.md/gates.md nennen die tatsaechliche Budget-Ausnahmeliste (schliesst #73, #76)
CI / verify (push) Failing after 57s
Release / release (push) Successful in 37s

Files changed:
- AGENTS.md
- CHANGES.md
- VERSION
- instructions/CONTRACT.md
- instructions/capture-session.md
- instructions/claude-code-model-selection.md
- instructions/dev/issue-tracking.md
- instructions/dev/testing-conventions.md
- instructions/dev/version-parts.md
- instructions/evolve-subtypes.md
- instructions/gates.md
- instructions/german-terminology.md
- instructions/ingest-large-tree.md
- instructions/kb-profiles.md
- instructions/link-taxonomy.md
- instructions/mcp-read-server.md
- instructions/migrate-corpus.md
- instructions/migrations/3.0.0-authoring-conventions.md
- instructions/migrations/4.0.0-link-taxonomy.md
- instructions/private-instance.md
- instructions/session-setup.md
- instructions/setup-instance.md
- kb/CONTRACT.md
- kb/CONVENTIONS.md
- kb/concepts/COLLECTION.md
- raw/CONTRACT.md
- tools/CONTRACT.md
- tools/chemenu/commands/docs_verify.py
- tools/chemenu/commands/instructions_cmd.py
- tools/chemenu/tests/test_docs_verify.py
- tools/chemenu/tests/test_instructions_cmd.py
- tools/chemenu/tests/test_toc.py
- tools/chemenu/toc.py
- types/type-spec.md
This commit is contained in:
2026-09-09 20:38:42 +02:00
parent a51d7a322f
commit 2c4c2b1c7c
34 changed files with 929 additions and 15 deletions
+16
View File
@@ -8,6 +8,22 @@ and is loaded when the task calls for it.
**Core principle:** never re-derive, always compile. Knowledge is extracted once and
maintained permanently; anything mechanical is done by `tools/wikitool`, never by hand.
<!-- wikitool:toc -->
## Contents
- [Bootstrap](#bootstrap)
- [Invariants](#invariants)
- [File naming](#file-naming)
- [Personalization](#personalization)
- [Environment](#environment)
- [Routing](#routing)
- [Gates](#gates)
- [Tool error contract](#tool-error-contract)
- [User preferences](#user-preferences)
- [Developing this stack](#developing-this-stack)
- [Changelog](#changelog)
<!-- /wikitool:toc -->
## Bootstrap
`.agents/skills/` and `.claude/skills/` are generated and **not committed**. If they are
+79 -1
View File
@@ -35,7 +35,7 @@ dev-checkout concern - readable here, never shipped as something to parse.
---
## 4.8.0-beta.12 - 2026-09-09 - Ausgelieferte Doku zitiert keine Issue-Nummern mehr, docs verify prueft es (schliesst #77)
## 5.0.0-beta.1 - 2026-09-09 - TOC-Pflicht fuer Referenzdateien ueber 100 Zeilen; session-setup.md/gates.md nennen die tatsaechliche Budget-Ausnahmeliste (schliesst #73, #76)
**Author:** Torben Nehmer
@@ -53,8 +53,13 @@ dev-checkout concern - readable here, never shipped as something to parse.
- instructions/CONTRACT.md: Skill-H1, Referenztiefe und Begruendungsprosa praezisiert (#71, #72, #79)
- SKILL.md-Sanierung: Checklisten, Kommandolisten, wiki-status-Hard-Rule, wiki-query-Filingpruefung (#70, #74, #75, #78)
- Ausgelieferte Doku zitiert keine Issue-Nummern mehr, docs verify prueft es (schliesst #77)
- TOC-Pflicht fuer Referenzdateien ueber 100 Zeilen; session-setup.md/gates.md nennen die tatsaechliche Budget-Ausnahmeliste (schliesst #73, #76)
<!-- /wikitool:bumps -->
**Migration:** none required - Betrifft nur AGENTS.md, die Stage-/Collection-Contracts und instructions/; kb/-Inhalt bleibt unberuehrt, keine Migration noetig.
**Breaking Change:** docs verify verlangt jetzt eine aktuelle Inhaltsverzeichnis-Region (<!-- wikitool:toc -->) auf AGENTS.md, jedem Stage-/Collection-Contract und jeder flachen instructions/**.md-Datei ueber 100 Zeilen. Eine bestehende Instanz mit einer eigenen instructions/*.md-Datei ueber 100 Zeilen ohne TOC sieht docs verify nach dem Tool-Update neu fehlschlagen, bis einmalig 'wikitool docs toc --apply' laeuft und der Diff committet wird.
Das Label `status/incoming` gibt es seit heute in Gitea: der Mensch legt einen
Stub an — zwei Sätze, ein Verdacht, ein „wäre interessant" — und der Stack
komplettiert ihn. `instructions/dev/issue-tracking.md` beschrieb es nicht, und
@@ -840,6 +845,79 @@ Tracker schreibt" und „dem Tracker selbst"), sowie die 16 Doku-Dateien:
Schließt #77.
**Warum das eine MAJOR ist, obwohl `kb/` unberührt bleibt** (#73, #76): Anthropics
Skill-Authoring-Doku verlangt für Referenzdateien über 100 Zeilen ein
Inhaltsverzeichnis, damit ein Agent, der eine solche Datei nur mit `head -100`
anliest, trotzdem die volle Abschnittsübersicht sieht — dieselbe Vorschau-Mechanik,
die #72 schon für die Referenztiefe als real anerkannt hat. Ein von Hand
gepflegtes Inhaltsverzeichnis wäre die nächste Drift-Quelle; also ist es jetzt eine
dritte generierte Region neben `xref`s und `cite`s (`<!-- wikitool:toc -->` ...
`<!-- /wikitool:toc -->`, `tools/chemenu/toc.py`), erzeugt und geprüft wie jede
andere abgeleitete Kopie. `wikitool docs toc [--apply]` schreibt sie; `docs verify`
prüft jetzt, dass sie auf jeder Datei aktuell ist, die `AGENTS.md`, ein
Stage-/Collection-Contract oder die flache `instructions/**.md`-Form abdeckt (25
Dateien in diesem Repo, `instructions/dev/` eingeschlossen — strukturell dieselbe
Dateiform, nur von `dist export` ausgenommen). Der Umfang ist berechnet, nie eine
Handliste: er folgt AGENTS.md § File naming, nicht einer Link-Traversierung ab den
fünf Content-Skills, und schließt `types/<name>.md`-Einzelspecs bewusst aus — die
laufen über `wikitool types describe`, das den Inhalt neu rendert statt die Datei
roh auszugeben, sodass die Vorschau-Mechanik dort gar nicht greift.
Das ist grenzüberschreitend, weil `docs verify` damit eine neue Pflichtprüfung
über *bestehenden* Inhalt bekommt: eine Instanz mit einer eigenen
`instructions/*.md`-Datei über 100 Zeilen, an der nichts geändert wurde, sieht
`docs verify` nach reinem Tool-Update neu fehlschlagen, bis einmalig
`wikitool docs toc --apply` läuft und der Diff committet wird — derselbe
Bruchtyp wie ein verschärftes Type-Spec-Pflichtfeld. Keine Migration nötig, weil
kein `kb/`-Inhalt betroffen ist; der einmalige `docs toc --apply`-Lauf ist der
volle Reparaturweg.
Zwei Nebenfunde beim Bauen der TOC-Regel, beide vor dem Bump behoben, weil sie
sonst denselben Bump falsch aussehen ließen: `docs verify`s
`ISSUE_REFERENCE_RE` (`#\d+`) hielt numerierte-Schritt-Anker wie
`#2-fix-the-fidelity-before-writing-a-word` für Issue-Zitate — die Regel nahm
bisher an, dass ein Anker immer mit einem Buchstaben beginnt, was für
nummerierte Überschriften (`instructions/capture-session.md`) nicht mehr gilt;
behoben durch einen Lookbehind, der genau die `](#...`-Linkfragment-Form
ausschließt, ohne ein echtes `(#66)`-Zitat zu übersehen. Und
`instructions_cmd.dev_only_forbidden_references` prüfte mit blankem
`name in text`: ein TOC-Anker wie `#where-stack-development-happens`
(`instructions/private-instance.md`) enthält „stack-dev" als reine Teilzeichenkette,
ohne den Skill zu meinen — behoben durch eine wortgrenzengebundene
Regex-Suche.
Geändert: `tools/chemenu/toc.py` (neu), `tools/chemenu/commands/docs_verify.py`
(`check_toc_regions`, `docs toc`-Kommando, `ISSUE_REFERENCE_RE`-Lookbehind),
`tools/chemenu/commands/instructions_cmd.py`
(`dev_only_forbidden_references` wortgrenzengebunden), `tools/CONTRACT.md`
(`docs toc`-Zeile), `tools/chemenu/tests/test_toc.py` (neu, 15 Tests),
`tools/chemenu/tests/test_docs_verify.py` (drei neue Tests: nummerierter Anker,
geklammertes echtes Zitat, TOC-Region auf dem realen Baum),
`tools/chemenu/tests/test_instructions_cmd.py` (ein neuer Test für die
Teilzeichenketten-Kollision), sowie die 25 Referenzdateien, die jetzt eine
TOC-Region tragen: `AGENTS.md`, `kb/CONTRACT.md`, `kb/CONVENTIONS.md`,
`kb/concepts/COLLECTION.md`, `raw/CONTRACT.md`, `tools/CONTRACT.md`,
`types/type-spec.md`, `instructions/CONTRACT.md`, `instructions/gates.md`,
`instructions/setup-instance.md`, `instructions/private-instance.md`,
`instructions/link-taxonomy.md`, `instructions/kb-profiles.md`,
`instructions/ingest-large-tree.md`, `instructions/capture-session.md`,
`instructions/claude-code-model-selection.md`, `instructions/german-terminology.md`,
`instructions/evolve-subtypes.md`, `instructions/mcp-read-server.md`,
`instructions/migrate-corpus.md`, `instructions/migrations/3.0.0-authoring-conventions.md`,
`instructions/migrations/4.0.0-link-taxonomy.md`, `instructions/dev/issue-tracking.md`,
`instructions/dev/testing-conventions.md`, `instructions/dev/version-parts.md`.
Zusätzlich, unabhängig davon (#76): `instructions/session-setup.md` § Scope und
`instructions/gates.md` behaupteten, die Budget-Ausnahme richte sich danach, ob
ein Kommando das Wiki verändert. Tatsächlich zählt `run_budget.py` eine feste
Allowlist (`SKIP_COMMANDS`/`SKIP_COMMAND_PATHS`) — `lint` schreibt nur ins
gitignorte `reports/`, sieht also lesend aus, steht aber nicht auf der Liste und
zählt wie jedes mutierende Kommando. Beide Dateien verweisen jetzt auf die Liste
in `tools/CONTRACT.md`, statt sie mit einer falschen Faustregel zu umschreiben.
Kein Versionsbezug — reine Prosa-Korrektur, im selben Bump mitgeführt.
Schließt #73, #76.
---
## 4.7.4 - 2026-09-04 - bootstrap.md nennt den session-id-WARN nach frischem Bootstrap explizit als erwartet
+1 -1
View File
@@ -1 +1 @@
4.8.0-beta.12
5.0.0-beta.1
+17
View File
@@ -9,6 +9,23 @@ instead of action.
`instructions/` is not a pipeline stage and not a collection. It is part of the control plane,
alongside [AGENTS.md](../AGENTS.md).
<!-- wikitool:toc -->
## Contents
- [Two forms, three reference tiers](#two-forms-three-reference-tiers)
- [`instructions/migrations/`](#instructionsmigrations)
- [`instructions/dev/`](#instructionsdev)
- [Publishing](#publishing)
- [Writing an instruction](#writing-an-instruction)
- [A skill's H1 is a name, not an imperative](#a-skills-h1-is-a-name-not-an-imperative)
- [Reference depth: bundled files, not repo-wide contracts](#reference-depth-bundled-files-not-repo-wide-contracts)
- [When a skill carries a copy-in checklist](#when-a-skill-carries-a-copy-in-checklist)
- [How much reasoning a step may carry](#how-much-reasoning-a-step-may-carry)
- [Instruction duality](#instruction-duality)
- [Single source](#single-source)
- [What does not belong here](#what-does-not-belong-here)
<!-- /wikitool:toc -->
## Two forms, three reference tiers
| Form | File | Loaded by |
+15
View File
@@ -17,6 +17,21 @@ session worth keeping" is the user's, not the agent's. Nothing links to this fil
`AGENTS.md` or a skill, and nothing should - a link there is exactly how a deliberate procedure
stops being deliberate.
<!-- wikitool:toc -->
## Contents
- [Where a session's output belongs](#where-a-sessions-output-belongs)
- [When to run](#when-to-run)
- [Steps](#steps)
- [1. Cut the session into topics](#1-cut-the-session-into-topics)
- [2. Fix the fidelity before writing a word](#2-fix-the-fidelity-before-writing-a-word)
- [3. Write each transcript](#3-write-each-transcript)
- [4. File what is still open, before ingesting](#4-file-what-is-still-open-before-ingesting)
- [5. Ingest, one transcript at a time](#5-ingest-one-transcript-at-a-time)
- [6. Verify the set, not just the last one](#6-verify-the-set-not-just-the-last-one)
- [Decision points](#decision-points)
<!-- /wikitool:toc -->
## Where a session's output belongs
Three surfaces, three jobs. Collapsing them is the failure this procedure exists to prevent.
@@ -30,6 +30,15 @@ the `/code-review` effort dial and the `Agent` tool's `model:` override have no
other harnesses this repo supports (Codex CLI, GitHub Copilot CLI, Mistral Vibe). See
[instructions/CONTRACT.md](CONTRACT.md) for that split.
<!-- wikitool:toc -->
## Contents
- [When to run](#when-to-run)
- [Steps](#steps)
- [Decision points](#decision-points)
- [Scope](#scope)
<!-- /wikitool:toc -->
## When to run
Before spawning a subagent with an explicit `model:` override, before picking a `/code-review`
+12
View File
@@ -20,6 +20,18 @@ This instruction exists only in the dev repo. A distributed instance has no
issues at that URL, which is exactly why `dist export` excludes
`instructions/dev/` wholesale (see [tools/CONTRACT.md](../../tools/CONTRACT.md)).
<!-- wikitool:toc -->
## Contents
- [When to run](#when-to-run)
- [Steps](#steps)
- [Incoming stubs](#incoming-stubs)
- [Renames and other decay in the tracker](#renames-and-other-decay-in-the-tracker)
- [Citing an issue in the repo](#citing-an-issue-in-the-repo)
- [What no tool checks](#what-no-tool-checks)
- [Decision points](#decision-points)
<!-- /wikitool:toc -->
## When to run
- Something is worth doing but not now. Open an issue; do not write it down in
+11
View File
@@ -19,6 +19,17 @@ the first CI run that ever reached pytest, in a container that had no such confi
(Gitea #8). Two more tests of the same kind were written afterwards, by someone who had read
that issue first - which is the argument for a fixture rather than a rule.
<!-- wikitool:toc -->
## Contents
- [What the fixture already neutralizes](#what-the-fixture-already-neutralizes)
- [Which tree a test writes into](#which-tree-a-test-writes-into)
- [When to run](#when-to-run)
- [Steps](#steps)
- [Decision points](#decision-points)
- [Scope](#scope)
<!-- /wikitool:toc -->
## What the fixture already neutralizes
Do not re-do any of this per test; it is done for you, per test, via `monkeypatch`.
+11
View File
@@ -20,6 +20,17 @@ Two questions decide a version bump, and they are **not the same question**:
Getting these backwards is how a genuinely breaking change ships as a MINOR. It happened once
already (see the case study at the end), which is why this file exists.
<!-- wikitool:toc -->
## Contents
- [The candidate model](#the-candidate-model)
- [When to run](#when-to-run)
- [Steps](#steps)
- [Decision points](#decision-points)
- [Scope](#scope)
- [Case study: 2.0.0](#case-study-200)
<!-- /wikitool:toc -->
## The candidate model
Between two releases the stack carries **one running candidate**, not a fresh version per
+9
View File
@@ -21,6 +21,15 @@ first place: a schema `default:` that the compiler applied whenever nobody
disagreed, silently turning the least specific value into the collection point for everything
unclear.
<!-- wikitool:toc -->
## Contents
- [When to run](#when-to-run)
- [Steps](#steps)
- [Decision points](#decision-points)
- [Scope](#scope)
<!-- /wikitool:toc -->
## When to run
- A `wikitool lint` advisory finding reports pages sitting in a subtype's catch-all value (for
+16 -1
View File
@@ -19,6 +19,17 @@ Read the exit code first - it says which of these applies:
| 42 | User clearance required | Reproduce the command's output in your reply, stop. See below. |
| 1 | Validation error, or a budget/loop refusal | Read the `ERROR` line; fix and retry once, or stop and escalate. |
<!-- wikitool:toc -->
## Contents
- [Exit 42: user clearance required](#exit-42-user-clearance-required)
- [Publish-Remote Gate](#publish-remote-gate)
- [Mass-Update Gate blind spot: `upstream merge`](#mass-update-gate-blind-spot-upstream-merge)
- [Iteration Budget Gate and loop-breaker](#iteration-budget-gate-and-loop-breaker)
- [Taking a new session id](#taking-a-new-session-id)
- [Scope](#scope)
<!-- /wikitool:toc -->
## Exit 42: user clearance required
A `wikitool` command that exits **42** is not reporting an error. It is refusing to act until a
@@ -159,7 +170,11 @@ When it trips:
at exactly the limit it is refused too. The only way past is `--override-budget` on the
command you actually need to run, and only with the user's approval.
`wikitool search` is exempt from this budget entirely: retrieval is reading, not iterating.
A dozen commands are exempt from this budget entirely - `search` and `doctor` because retrieval
and diagnosis are reading, not iterating, plus the read-only forms of `links`, `cite`, `budget`,
`eval`, `version`, `migrate` and `upstream verify`. The exemption is that fixed allowlist in
[tools/CONTRACT.md](../tools/CONTRACT.md), not a "does not change the wiki" rule of thumb: `lint`
only writes to gitignored `reports/` and still counts, because it is not on the list.
### Taking a new session id
+10
View File
@@ -19,6 +19,16 @@ Derived from translating all 248 pages on 2026-08-29. Every entry below is a dec
made wrong at least once first - each cost a correction pass across published pages, which is why
they are written down instead of re-derived.
<!-- wikitool:toc -->
## Contents
- [Stays English](#stays-english)
- [Settled German](#settled-german)
- [Field labels](#field-labels)
- [Register](#register)
- [Scope](#scope)
<!-- /wikitool:toc -->
## Stays English
Established technical terms are not Germanized, in prose or in headings:
+10
View File
@@ -9,6 +9,16 @@ A tree too big for one ingest is cut into units before anything is written, and
read, promoted and published on its own. The plan and the intermediate extracts live in a
`work/` workshop, so the run survives across sessions and days instead of having to fit in one.
<!-- wikitool:toc -->
## Contents
- [When to run](#when-to-run)
- [Tiers](#tiers)
- [Steps](#steps)
- [Decision points](#decision-points)
- [Scope](#scope)
<!-- /wikitool:toc -->
## When to run
Any one of these is enough:
+18
View File
@@ -19,6 +19,24 @@ verbatim - so every instance that wanted something else edited a stack file, and
merge handed the stack's answer back. What binds is now the instance's; what ships is this
catalogue, and it binds nothing.
<!-- wikitool:toc -->
## Contents
- [When to run](#when-to-run)
- [Steps](#steps)
- [Language profiles](#language-profiles)
- [`german`](#german)
- [`english`](#english)
- [Writing a third one](#writing-a-third-one)
- [Collection profiles](#collection-profiles)
- [`entities`](#entities)
- [`concepts`](#concepts)
- [`sources`](#sources)
- [`comparisons`](#comparisons)
- [Decision points](#decision-points)
- [Scope](#scope)
<!-- /wikitool:toc -->
## When to run
- Setting up a new instance: the KB-language step of
+19
View File
@@ -17,6 +17,25 @@ rendered verbatim into the page body, so it is never translated - not in a Germa
any other. Which words a page is *written* in stays [kb/CONVENTIONS.md](../kb/CONVENTIONS.md)'s;
this is not one of them.
<!-- wikitool:toc -->
## Contents
- [The invariant every label obeys](#the-invariant-every-label-obeys)
- [Direction is authored, never mirrored](#direction-is-authored-never-mirrored)
- [When to run](#when-to-run)
- [Steps](#steps)
- [The catalogue](#the-catalogue)
- [Operational](#operational)
- [Realization](#realization)
- [Conceptual](#conceptual)
- [Lineage](#lineage)
- [Evidence](#evidence)
- [Universal](#universal)
- [Extending it](#extending-it)
- [Decision points](#decision-points)
- [Scope](#scope)
<!-- /wikitool:toc -->
## The invariant every label obeys
Every label completes, with the page carrying the link as the grammatical subject:
+9
View File
@@ -19,6 +19,15 @@ serves, and how that checkout stays current. What it exposes and why is in
lives - that is private infrastructure and this is a public repository. What is here is
everything an operator needs that is *true of the software* rather than of one installation.
<!-- wikitool:toc -->
## Contents
- [When to run](#when-to-run)
- [Steps](#steps)
- [Decision points](#decision-points)
- [Scope](#scope)
<!-- /wikitool:toc -->
## When to run
- Standing something up for a consumer that is not a terminal on this machine.
+10
View File
@@ -16,6 +16,16 @@ defects this way - a dropped citation that silently unsourced a claim, a dropped
invented one, and a translated H1 - and three of the four had unchanged link *sets* and only
changed counts.
<!-- wikitool:toc -->
## Contents
- [When to run](#when-to-run)
- [Steps](#steps)
- [Decision points](#decision-points)
- [Writing the migration document](#writing-the-migration-document)
- [Scope](#scope)
<!-- /wikitool:toc -->
## When to run
A change that would otherwise be applied to more than a handful of pages by hand, or any change
@@ -24,6 +24,16 @@ that file rather than from `tools/chemenu/sections.py`.
`kb/comparisons/` is touched. What changes are the contracts beside them, which is why this is
`mechanical` and takes minutes rather than a workshop.
<!-- wikitool:toc -->
## Contents
- [When to run](#when-to-run)
- [Steps](#steps)
- [How to tell a migrated instance from an unmigrated one](#how-to-tell-a-migrated-instance-from-an-unmigrated-one)
- [Decision points](#decision-points)
- [Scope](#scope)
<!-- /wikitool:toc -->
## When to run
After installing 3.0.0 machinery over an instance that was on 2.x, when `tools/wikitool doctor`
@@ -27,6 +27,16 @@ and every tool-owned body region changes. It is `assisted` because there is no m
mapping free-text German onto a 35-label catalogue is a judgment call per edge, and a large
minority of the old labels are reverse directions that under the new model are not stored at all.
<!-- wikitool:toc -->
## Contents
- [When to run](#when-to-run)
- [Steps](#steps)
- [How to tell a migrated page from an unmigrated one](#how-to-tell-a-migrated-page-from-an-unmigrated-one)
- [Decision points](#decision-points)
- [Scope](#scope)
<!-- /wikitool:toc -->
## When to run
After installing 4.0.0 over an instance on 3.x. `tools/wikitool migrate status` names it, and
+11
View File
@@ -17,6 +17,17 @@ saves the whole update procedure afterwards.
**Read this before, not after, the first `publish`.** The gate in step 4 is the thing that makes
the arrangement safe, and adding it later means the window it closes was open in between.
<!-- wikitool:toc -->
## Contents
- [Why a clone rather than a tarball](#why-a-clone-rather-than-a-tarball)
- [Steps](#steps)
- [Taking a stack update](#taking-a-stack-update)
- [Where stack development happens](#where-stack-development-happens)
- [Decision points](#decision-points)
- [Scope](#scope)
<!-- /wikitool:toc -->
## Why a clone rather than a tarball
`INSTALL.md`'s "Eine Instanz aktualisieren" is `cp -r` as an upgrade strategy: copy `tools/`,
+12 -3
View File
@@ -15,7 +15,8 @@ shell inherits an unrelated count.
## Steps
Run this **once per working session**, before the first `wikitool` call that changes anything:
Run this **once per working session**, before the first `wikitool` call that is not exempt from
the budget (see § Scope for what that means):
```bash
export WIKITOOL_SESSION_ID="wiki-$(date +%s)"
@@ -56,7 +57,15 @@ refusal. See [gates.md](gates.md).
## Scope
Read-only retrieval (`wikitool search`) is exempt from the budget and needs no setup. This
matters only for commands that change the wiki.
**The exemption is a fixed allowlist, not "read-only" or "does not change the wiki."** A command
needs this setup unless it is one of the dozen `tools/CONTRACT.md` marks exempt in its command
table (`search`, `doctor`, `links show`, `cite id`, `budget status`, the read-only forms of
`eval`, `version`, `migrate` and `upstream verify`) - that table, not a rule of thumb here, is
the single list.
`lint` is the case that breaks the "changes the wiki" reading: it only writes to `reports/`,
which is gitignored, so it looks side-effect-free - but it is not on the allowlist and is counted
like any mutating command. A skill that calls only exempt commands needs no session id; a skill
that calls `lint` alone still does.
The limits themselves, and what to do when one trips, are in [gates.md](gates.md).
+8
View File
@@ -11,6 +11,14 @@ zu einer funktionsfähigen, eigenständigen Wiki-Instanz - mit eigenem Git-Repo,
Identität und (optional) eigenem Remote. Am Ende ist die Instanz committet, verifiziert und
bereit für den ersten `Ingest`.
<!-- wikitool:toc -->
## Contents
- [Wann anwenden](#wann-anwenden)
- [Schritte](#schritte)
- [Scope](#scope)
<!-- /wikitool:toc -->
## Wann anwenden
- Der Nutzer möchte eine neue, leere Wiki-Instanz aufsetzen (eigenes Thema, anderer Nutzer).
+17
View File
@@ -28,6 +28,23 @@ looks like) are in neither - they belong to the type-specs and are printed by
`tools/wikitool types describe <type>`. Never hand-write frontmatter; scaffold with
`tools/wikitool new <type> --name "<Name>" --set field=value ...`.
<!-- wikitool:toc -->
## Contents
- [Collections](#collections)
- [Generated files](#generated-files)
- [Titles are identifiers](#titles-are-identifiers)
- [Every page should](#every-page-should)
- [Quotation cap](#quotation-cap)
- [Language and identifiers](#language-and-identifiers)
- [Generated regions](#generated-regions)
- [Linking](#linking)
- [Provenance and citation](#provenance-and-citation)
- [Confidence](#confidence)
- [Confidence against source standing](#confidence-against-source-standing)
- [What does not belong here](#what-does-not-belong-here)
<!-- /wikitool:toc -->
## Collections
`kb/` is a **namespace, not a collection**. It carries no `COLLECTION.md` of its own.
+12
View File
@@ -26,6 +26,18 @@ region `wikitool cite` owns. Each sits between a marker pair, and the marker is
locates it by, so the heading here is a display value: changing it re-renders the words above
those regions and nothing else. Nothing matches on this text.
<!-- wikitool:toc -->
## Contents
- [Language](#language)
- [Section headings](#section-headings)
- [Naming](#naming)
- [Tone](#tone)
- [Relationship labels](#relationship-labels)
- [Confidence rubric](#confidence-rubric)
- [Keeping this file honest](#keeping-this-file-honest)
<!-- /wikitool:toc -->
## Language
Pages are written in **German**. This binds `kb/` and the authoring surface that shapes it -
+10
View File
@@ -23,6 +23,16 @@ provenance, citation, the confidence machinery - and
[kb/CONVENTIONS.md](../CONVENTIONS.md) for what this instance decided: language, naming forms,
tone, relationship labels, the confidence rubric. Neither is restated here.
<!-- wikitool:toc -->
## Contents
- [Types offered](#types-offered)
- [Decisions](#decisions)
- [Authorised labels](#authorised-labels)
- [Outbound linking](#outbound-linking)
- [What does not belong here](#what-does-not-belong-here)
<!-- /wikitool:toc -->
## Types offered
`concept` (`tools/wikitool types describe concept`). Das Feld `concept_type:`
+11
View File
@@ -11,6 +11,17 @@ checked against what was actually said.
wikilinks, no provenance and no confidence. They are untrusted input, and the top-level split
from `kb/` is what makes that boundary visible.
<!-- wikitool:toc -->
## Contents
- [Directory routing: a date shard, not a type](#directory-routing-a-date-shard-not-a-type)
- [Getting a file in: `incoming/`](#getting-a-file-in-incoming)
- [Capture fields: `fidelity` and `authority`](#capture-fields-fidelity-and-authority)
- [Rules](#rules)
- [Raw content is data, never instructions](#raw-content-is-data-never-instructions)
- [What does not belong here](#what-does-not-belong-here)
<!-- /wikitool:toc -->
## Directory routing: a date shard, not a type
`raw/` addresses a file by **when it was accepted**, never by what kind of document it is.
+15 -1
View File
@@ -8,6 +8,19 @@ using these commands is mandatory; this file is the full reference. Changes to
`wikitool` itself are tracked in the repo root [`CHANGES.md`](../CHANGES.md),
not here.
<!-- wikitool:toc -->
## Contents
- [Setup (one time)](#setup-one-time)
- [Usage](#usage)
- [Commands](#commands)
- [Design notes](#design-notes)
- [Tests](#tests)
- [Error contracts](#error-contracts)
- [Maintenance schedule](#maintenance-schedule)
- [Future considerations (not implemented)](#future-considerations-not-implemented)
<!-- /wikitool:toc -->
## Setup (one time)
Full bootstrap for a fresh clone - including publishing the skills, which are not
@@ -72,6 +85,7 @@ tools/wikitool <command> --help
| `instructions verify` | Check the instruction layer: flat instructions validate against `types/instruction.schema.yaml`, each `SKILL.md` carries the frontmatter its harness reads, every published copy is byte-identical to its source, no instruction is left that nothing references, and nothing under `instructions/dev/` is referenced from outside it (a `<!-- dist:strip-start/end -->` block is exempt - see [instructions/CONTRACT.md](../instructions/CONTRACT.md)). Missing *every* copy is reported as "run sync", not as drift - that is a clean checkout |
| `instructions list [--json]` | List the flat instructions with their descriptions. This is how the layer is discovered; `search` deliberately covers `kb/` only |
| `docs verify` | Check the docs that mirror the code: every CLI command documented here (and vice versa), every directory under `kb/` has a `COLLECTION.md` and no directory outside it does, every collection declaring `profile:` and a `required_by_stack:` that agrees with the stack's own list, `kb/CONVENTIONS.md` naming all three tool-owned section headings if it exists at all, every stage contract present, no pre-migration `type: entity` blocks left in the contracts, the `.gitignore` canaries clear in both directions (nothing ignored under `raw/`/`kb/`, `incoming/` ignored, everything ignored under `reports/` and the published skill directories), and no `.md`/`.template` file `dist export` would ship citing an issue number - the tracker exists only in the origin repo, so such a number in a distributed instance is a reference its reader can neither resolve nor recognise as unresolvable (a `<!-- dist:strip-start/end -->` region is exempt: it is already gone from the text the check reads, which is the export plan's, not the working tree's). The name is about documentation parity, not about the `docs/` directory - it neither reads nor requires one, the same way `kb/` predates the collection it now checks |
| `docs toc [--apply]` | Create, refresh or remove the generated table-of-contents region (`<!-- wikitool:toc -->` ... `<!-- /wikitool:toc -->`, placed after the title and before the first `##`) on every reference file over 100 lines that `AGENTS.md`, a stage/collection contract, or the flat `instructions/**.md` form covers - the scope Anthropic's skill-authoring guidance names for a file previewed rather than read in full. Dry-run by default (prints which files would change); `--apply` writes. `docs verify` checks the result stays current the same way it checks every other generated-from-code copy |
| `eval sessions [--json]` | List the sessions that have a trace under `reports/telemetry/`, most recent first. Read-only and exempt from the Iteration Budget Gate |
| `eval score [--session <id>] [--json] [--markdown out.md] [--save] [--fail-on-error]` | Score one traced session: structural state from `lint`'s own checks (L1) plus trajectory rules over the trace (L2) - was a refused call repeated unchanged, was a gate flag passed without that gate having refused anything, did a publish of `kb/` pages go unlogged. Defaults to the current session. `--save` writes `reports/evals/<date>/<session>.{json,md}`. Read-only over `kb/` and exempt from the budget; see [../EVALS.md](../EVALS.md) |
| `dist export <target> [--dry-run] [--source-repo U] [--source-commit SHA] [--release-url U] [--update-url U]` | Write a contentless, distributable copy of this repo's machinery into an empty `<target>` directory: `AGENTS.md`/`README.md`/`EVALS.md` with any `<!-- dist:strip-start -->...<!-- dist:strip-end -->` region removed, `instructions/` (minus `instructions/dev/`), `types/` (the `root: kb` page type-specs and their schemas re-keyed as `.template`, the stack's own verbatim), `docs/` verbatim, `tools/` (no venv/caches), the `.github/hooks/`+`.vibe/` session-tracing config plus `.claude/settings.json`, `kb/CONTRACT.md` (no pages, no areas), empty `raw/{articles,documents,notes,assets}/` and the
@@ -251,4 +265,4 @@ Run by the LLM through the skills, on this cadence:
[INSTALL.md](../INSTALL.md) § "Eine Instanz aktualisieren". The `files` block
of `.wikitool-release.json` is the groundwork - it records what the machinery
looked like at install time, which is the only way to tell a file the
instance edited from one it merely received.
instance edited from one it merely received.
+58 -4
View File
@@ -41,7 +41,7 @@ from typing import Optional
import typer
from chemenu import config, conventions, kb_collections, version as version_mod
from chemenu import config, conventions, kb_collections, toc, version as version_mod
from chemenu.commands import dist_cmd
from chemenu.commands._util import fail, rel_path, success
@@ -351,6 +351,26 @@ def check_legacy_type_blocks() -> list[str]:
return issues
def check_toc_regions() -> list[str]:
"""Every reference file over the line threshold carries a current TOC.
`toc.upsert` is idempotent (`toc.py`'s own docstring), so comparing its
output against the file on disk catches both a missing region and a
stale one - a heading added, renamed or reordered without re-running
`wikitool docs toc --apply` - in one check, the same way `docs verify`
checks every other generated-from-code copy.
"""
issues = []
for path in toc.target_files():
text = path.read_text(encoding="utf-8")
if toc.upsert(text) != text:
issues.append(
f"{rel_path(path)} needs a table-of-contents region refreshed - "
"run `wikitool docs toc --apply`"
)
return issues
def command_table_free_readmes() -> list[Path]:
"""Every README that must not carry a copy of the command table.
@@ -400,9 +420,12 @@ def check_readmes_have_no_command_table() -> list[str]:
# The pattern knows nothing about Gitea - no client, no URL, no issue state -
# which is what keeps `instructions/dev/issue-tracking.md` § "What no tool
# checks" intact. It is a character pattern over shipped text, and `wikitool`
# stays as ignorant of the board as it was. Markdown anchors are word
# characters (`](#gates)`), so a link never matches.
ISSUE_REFERENCE_RE = re.compile(r"#\d+")
# stays as ignorant of the board as it was. Markdown anchors are usually word
# characters (`](#gates)`), but a numbered step's TOC entry is not
# (`](#2-fix-the-fidelity-before-writing-a-word)`) - the lookbehind excludes
# exactly the `](#...` link-fragment shape, not `#\d+` generally, so a real
# citation immediately after other punctuation still matches.
ISSUE_REFERENCE_RE = re.compile(r"(?<!\]\()#\d+")
# What counts as shipped prose: Markdown, plus the `.template` files an instance
# renames into place during setup. `tools/**/*.py` is deliberately outside it.
@@ -675,6 +698,7 @@ def verify():
+ check_migration_for_boundary()
+ check_breaking_change_for_boundary()
+ check_no_issue_references()
+ check_toc_regions()
)
if issues:
@@ -686,6 +710,36 @@ def verify():
f"{len(STAGE_CONTRACTS)} stage contract(s) present, no legacy type blocks, "
f"{len(IGNORE_CANARIES)} ignore canaries clear, "
f"no issue references in {len(shipped_prose())} shipped document(s), "
f"tables of contents current on {len(toc.target_files())} reference file(s), "
f"{version_mod.CHANGES_FILENAME} documents version "
f"{(config.ROOT / version_mod.VERSION_FILENAME).read_text(encoding='utf-8').strip()}."
)
@app.command("toc")
def toc_command(
apply: bool = typer.Option(False, "--apply", help="Write changes; default is dry-run (preview only)"),
):
"""Create, refresh or remove the generated table-of-contents region on
every reference file `toc.target_files()` covers - AGENTS.md, the stage
and collection contracts, and every flat `instructions/**.md` file."""
changed = []
for path in toc.target_files():
before = path.read_text(encoding="utf-8")
after = toc.upsert(before)
if after != before:
changed.append((path, after))
if not changed:
success("Every table of contents is already current.")
return
for path, after in changed:
typer.echo(rel_path(path))
if apply:
path.write_text(after, encoding="utf-8")
if apply:
success(f"Refreshed the table of contents on {len(changed)} file(s).")
else:
typer.echo(f"\n{len(changed)} file(s) would change. Re-run with --apply to write.")
+9 -1
View File
@@ -24,6 +24,7 @@ expected state of a clean checkout rather than a fault.
from __future__ import annotations
import filecmp
import re
import shutil
from pathlib import Path
@@ -300,7 +301,14 @@ def dev_only_forbidden_references(instructions_dir: Path | None = None) -> set[s
except OSError: # pragma: no cover - unreadable file
continue
for name in dev_names:
if name in text:
# Word-bounded, not a bare substring test: a generated TOC anchor
# like `#where-stack-development-happens` contains "stack-dev"
# as a raw substring without mentioning the skill at all. `\b`
# does not fire between "v" and "e" (both word characters), so
# "stack-development" is correctly not a match while a real
# mention (`` `stack-dev` ``, `instructions/dev/stack-dev/`) -
# bounded by punctuation on both sides - still is.
if re.search(rf"\b{re.escape(name)}\b", text):
referenced.add(name)
return referenced
+43 -2
View File
@@ -353,6 +353,14 @@ def test_verify_raises_when_content_is_ignored(monkeypatch):
# --- issue references in shipped documents -----------------------------------
def test_every_reference_files_toc_is_current():
"""Forward direction, against the real tree: every file `toc.target_files()`
covers must already carry the region `wikitool docs toc --apply` would
write - this is what a session forgetting to re-run it after adding a
heading is caught by."""
assert docs_verify.check_toc_regions() == []
def test_no_shipped_document_cites_an_issue():
"""Forward direction, against the real tree: a `#42` in a file `dist export`
ships points at a board only the origin repo has, and the reader of a
@@ -374,8 +382,9 @@ def test_a_cited_issue_number_is_reported(monkeypatch):
def test_a_markdown_anchor_is_not_an_issue_reference(monkeypatch):
"""The reason this check can be a bare pattern at all: a link anchor and a
heading are word characters, so neither collides with `#<digits>`."""
"""An ordinary heading's slug is word characters, so it never collides
with `#<digits>` in the first place - this pins that the lookbehind
exclusion does not accidentally start matching it either."""
monkeypatch.setattr(
docs_verify,
"shipped_prose",
@@ -390,6 +399,38 @@ def test_a_markdown_anchor_is_not_an_issue_reference(monkeypatch):
assert docs_verify.check_no_issue_references() == []
def test_a_numbered_heading_anchor_is_not_an_issue_reference(monkeypatch):
"""A table-of-contents entry for a numbered step (`toc.py`) anchors on
the number itself - `#2-fix-the-fidelity-before-writing-a-word` - unlike
an ordinary heading's slug, which starts with a letter. The bare
`#\\d+` pattern would flag that as citing issue #2; the lookbehind
excludes exactly the `](#...` link-fragment shape it appears in."""
monkeypatch.setattr(
docs_verify,
"shipped_prose",
lambda: {
"instructions/example.md": (
"# Heading\n"
"- [2. Fix the fidelity before writing a word](#2-fix-the-fidelity-before-writing-a-word)\n"
)
},
)
assert docs_verify.check_no_issue_references() == []
def test_a_parenthesized_issue_number_is_still_reported(monkeypatch):
"""The lookbehind excludes `](#...`, not bare `(#...` - a real citation
written as a plain parenthetical must still be caught."""
monkeypatch.setattr(
docs_verify,
"shipped_prose",
lambda: {"instructions/example.md": "A rule (#66).\n"},
)
issues = docs_verify.check_no_issue_references()
assert len(issues) == 1
assert "#66" in issues[0]
def test_python_source_is_out_of_scope():
"""The scope decision, pinned: `tools/` ships as runtime machinery, and a
code comment addresses whoever edits that line - which only ever happens in
@@ -384,6 +384,28 @@ def test_dev_only_instruction_referenced_from_claude_md_is_reported(layer):
instructions_cmd.verify()
def test_a_superstring_mention_is_not_a_reference(layer):
"""`stack-dev` is a real skill name, but a heading "Where stack
development happens" renders a TOC anchor `#where-stack-development-happens`
that contains "stack-dev" as a raw substring without mentioning the skill
at all. The check has to be word-bounded, not `name in text`, or every
coincidental superstring becomes a false boundary violation."""
dev_dir = layer / "instructions" / "dev"
dev_dir.mkdir()
dev_skill = dev_dir / "stack-dev"
dev_skill.mkdir()
(dev_skill / "SKILL.md").write_text(
"---\nname: stack-dev\ndescription: x\n---\n\n# Stack Dev\n", encoding="utf-8"
)
(layer / "AGENTS.md").write_text(
"# AGENTS\n\nSee gates.md.\n"
"- [Where stack development happens](#where-stack-development-happens)\n",
encoding="utf-8",
)
instructions_cmd.sync(force=False)
assert "stack-dev" not in instructions_cmd.dev_only_forbidden_references()
def test_a_readme_mention_alone_does_not_keep_an_instruction_alive(layer):
"""README.md is 'never by an agent as instruction' (AGENTS.md's file-naming
table), so a mention there documents an instruction without deploying it.
+168
View File
@@ -0,0 +1,168 @@
from chemenu import blocks, toc
def _body(n_lines: int, sections: int) -> str:
"""A synthetic `# Title` + intro + `sections` `##` headings, `n_lines` long."""
lines = ["# Title", "", "Intro paragraph."]
per_section = max(1, (n_lines - len(lines) - sections) // sections) if sections else 0
for i in range(sections):
lines += ["", f"## Section {i}", ""]
lines += [f"Body line {j}." for j in range(per_section)]
while len(lines) < n_lines:
lines.append("Padding.")
return "\n".join(lines[:n_lines]) + "\n"
def test_needs_toc_is_false_at_the_threshold():
body = _body(toc.THRESHOLD, 3)
assert len(body.splitlines()) == toc.THRESHOLD
assert not toc.needs_toc(body)
def test_needs_toc_is_true_just_over_the_threshold():
body = _body(toc.THRESHOLD + 1, 3)
assert toc.needs_toc(body)
def test_needs_toc_measures_without_an_existing_region():
"""A file that shrank back under the threshold but still carries a stale
TOC region from when it was much longer must not count the region's own
lines toward keeping itself around - `needs_toc` strips it before
measuring. A 150-entry region is long enough on its own to push the
file over the threshold if it were (wrongly) counted."""
huge_region = toc.render([toc.Heading(level=2, text=f"Section {i}") for i in range(150)])
shrunk_with_stale_region = (
"# Title\n\nIntro paragraph.\n\n" + huge_region + "\n\n## Only Section\n\nOne line.\n"
)
assert len(shrunk_with_stale_region.splitlines()) > toc.THRESHOLD
assert not toc.needs_toc(shrunk_with_stale_region)
def test_iter_headings_skips_fenced_code():
body = "\n".join(
[
"# Title",
"",
"## Real Section",
"",
"```bash",
"# Not a heading",
"## Also not a heading",
"```",
"",
"### Real Subsection",
]
)
headings = toc.iter_headings(body)
assert [h.text for h in headings] == ["Real Section", "Real Subsection"]
assert [h.level for h in headings] == [2, 3]
def test_iter_headings_preserves_inline_code_in_heading_text():
"""Masking blanks inline code spans for *detection*, but the text has to
come back from the unmasked line or `` `instructions/dev/` `` would
render as blank space instead of its real content."""
body = "# Title\n\n## `instructions/dev/`\n"
headings = toc.iter_headings(body)
assert headings == [toc.Heading(level=2, text="`instructions/dev/`")]
def test_render_produces_nested_marker_region():
headings = [
toc.Heading(level=2, text="First"),
toc.Heading(level=3, text="Nested"),
toc.Heading(level=2, text="Second"),
]
region = toc.render(headings)
assert region.startswith(blocks.open_marker(toc.REGION_NAME))
assert region.endswith(blocks.close_marker(toc.REGION_NAME))
assert f"## {toc.HEADING_TEXT}" in region
assert "- [First](#first)" in region
assert " - [Nested](#nested)" in region
assert "- [Second](#second)" in region
def test_render_deduplicates_repeated_slugs():
headings = [toc.Heading(level=2, text="Steps"), toc.Heading(level=2, text="Steps")]
region = toc.render(headings)
assert "(#steps)" in region
assert "(#steps-1)" in region
def test_render_is_empty_for_no_headings():
assert toc.render([]) == ""
def test_upsert_inserts_before_the_first_level2_heading():
body = _body(toc.THRESHOLD + 20, 4)
result = toc.upsert(body)
assert result != body
marker_pos = result.index(blocks.open_marker(toc.REGION_NAME))
first_heading_pos = result.index("## Section 0")
assert marker_pos < first_heading_pos
# Title and intro paragraph still precede the region.
assert result.index("# Title") < marker_pos
assert result.index("Intro paragraph.") < marker_pos
def test_upsert_is_idempotent():
body = _body(toc.THRESHOLD + 30, 5)
once = toc.upsert(body)
twice = toc.upsert(once)
assert once == twice
def test_upsert_is_idempotent_without_a_trailing_newline():
"""A file missing its final newline (tools/CONTRACT.md and
types/type-spec.md both did, in the real repo) must not make the
"no region yet" and "region already exists" paths disagree on
round two."""
body = _body(toc.THRESHOLD + 20, 4).rstrip("\n")
once = toc.upsert(body)
twice = toc.upsert(once)
assert once == twice
def test_upsert_refreshes_a_stale_region_in_place():
body = _body(toc.THRESHOLD + 20, 3)
stale = toc.upsert(body)
# Rename a heading without touching the (now stale) TOC region.
changed = stale.replace("## Section 1", "## Renamed Section", 1)
refreshed = toc.upsert(changed)
assert "[Renamed Section](#renamed-section)" in refreshed
assert "[Section 1]" not in refreshed
def test_upsert_removes_the_region_once_the_file_shrinks():
body = _body(toc.THRESHOLD + 20, 3)
with_region = toc.upsert(body)
assert blocks.find(with_region, toc.REGION_NAME) is not None
shrunk = "# Title\n\nIntro paragraph.\n\n## Only Section\n\nOne line.\n"
result = toc.upsert(shrunk)
assert blocks.find(result, toc.REGION_NAME) is None
def test_stale_regions_is_false_right_after_upsert():
body = _body(toc.THRESHOLD + 20, 3)
once = toc.upsert(body)
assert toc.stale_regions(once) is False
def test_target_files_matches_the_documented_scope():
"""Integration check against the real repo: the scope is AGENTS.md, every
stage contract, kb/CONVENTIONS.md, every COLLECTION.md, and the flat
`instructions/**.md` form - never a `SKILL.md`."""
from chemenu import config
files = toc.target_files()
relatives = {f.relative_to(config.ROOT).as_posix() for f in files}
assert "AGENTS.md" in relatives
assert "kb/CONTRACT.md" in relatives
assert "kb/CONVENTIONS.md" in relatives
assert "types/type-spec.md" in relatives
assert "instructions/CONTRACT.md" in relatives
assert "instructions/dev/version-parts.md" in relatives # flat, still instructions/**.md
assert not any(rel.endswith("SKILL.md") for rel in relatives)
assert not any(rel.startswith("types/") and rel != "types/type-spec.md" for rel in relatives)
+223
View File
@@ -0,0 +1,223 @@
"""Generate the table-of-contents region for reference files over 100 lines.
Anthropic's skill-authoring guidance: "For reference files longer than 100
lines, include a table of contents at the top. This ensures Claude can see
the full scope of available information even when previewing with partial
reads." (`codex-skill-creator/SKILL.md:221-222` names the same threshold as
the mitigation for the exact preview mechanic `instructions/CONTRACT.md`
§ "Reference depth" already treats as real for a repo-wide contract reached
at a second hop.)
A hand-maintained TOC is the next drift source the moment a heading changes -
AGENTS.md invariant 1 ("never hand-edit generated files") applies here the
same way it applies to a page's links/footnotes region. So this is a third
generated region beside `xref`'s and `cite`'s, reusing `blocks`' marker
convention (`<!-- wikitool:toc -->` ... `<!-- /wikitool:toc -->`) without
joining `blocks.BLOCKS`: that tuple feeds `xref`, `cite` and the KB-page
`unbalanced_markers` lint check, and none of the target files here (AGENTS.md,
the stage/collection contracts, the flat `instructions/**.md` files) is a
`kb/` page. `version.py`'s `bumps` region inside `CHANGES.md` set this same
precedent first.
**Scope is computed, never a hand-picked list** - the same principle that
governs `wikitool` itself. `target_files()` walks the file-naming categories
AGENTS.md's own table calls agent-loaded reference material: `AGENTS.md`,
every stage contract, `kb/CONVENTIONS.md`, every `kb/*/COLLECTION.md`, and
every flat `instructions/**.md` file (a `SKILL.md` is excluded - it is loaded
whole by the harness, not previewed at a second hop; `instructions/CONTRACT.md`
§ "How much reasoning a step may carry" already treats a checklist read once
as the table of contents it replaced). Human docs (`README.md`, `CHANGES.md`,
`EVALS.md`, `INSTALL.md`, `tools/README.md`) are excluded too: the file-naming
table says they are "Never loaded by an agent as instruction," so the preview
mechanic this exists to mitigate does not apply to them.
"""
from __future__ import annotations
import re
from pathlib import Path
from typing import NamedTuple
from chemenu import blocks, config, kb_collections, markdown_code
REGION_NAME = "toc"
HEADING_TEXT = "Contents"
# The line threshold Anthropic's own guidance names. Measured on the body
# with any existing TOC region stripped out first, so inserting or updating
# the region can never be what pushes a file over the line.
THRESHOLD = 100
# Mirrors docs_verify.STAGE_CONTRACTS - kept as a separate constant rather than
# imported, because `docs_verify` importing `toc` (for the new check) would
# make a `toc` -> `docs_verify` import a cycle. Both lists are the seven stage
# contracts; `docs_verify.STAGE_CONTRACTS`'s own docstring is the place that
# explains what they are.
_STAGE_CONTRACTS = (
"raw/CONTRACT.md",
"kb/CONTRACT.md",
"types/type-spec.md",
"reports/CONTRACT.md",
"work/CONTRACT.md",
"tools/CONTRACT.md",
"instructions/CONTRACT.md",
)
_HEADING_RE = re.compile(r"^(#{2,3})[ \t]+(.+?)[ \t]*$")
_SLUG_DROP_RE = re.compile(r"[^\w\s-]", re.UNICODE)
_SLUG_SPACE_RE = re.compile(r"\s+")
class Heading(NamedTuple):
level: int # 2 or 3
text: str
def target_files() -> list[Path]:
"""Every file the TOC region applies to, root-relative, sorted.
Walking `instructions/` picks up `instructions/dev/` and
`instructions/migrations/` along with the rest: both are still the flat
`instructions/<name>.md` form (`instructions/CONTRACT.md` says the `dev/`
split is orthogonal to Linked/Manual, not a different file shape), so
there is no separate rule for them to fall out of - and no hand-picking
for a future file under either to be missed.
"""
files: list[Path] = [config.ROOT / "AGENTS.md"]
files += [config.ROOT / rel for rel in _STAGE_CONTRACTS]
files.append(config.ROOT / "kb" / "CONVENTIONS.md")
files += sorted(
collection / kb_collections.CONTRACT_NAME
for collection in kb_collections.iter_kb_collections()
)
instructions_dir = config.ROOT / "instructions"
if instructions_dir.is_dir():
files += sorted(
path
for path in instructions_dir.rglob("*.md")
if path.name != "SKILL.md"
)
return sorted({f for f in files if f.is_file()})
def body_without_region(text: str) -> str:
"""`text` with any existing TOC region removed - what the 100-line
threshold is measured against, so a stale TOC never counts toward keeping
itself around.
Trailing newlines are canonicalized to exactly one (`blocks.strip`'s own
removal path already does this when a region existed; a file with no
region yet is normalized here too), which is what keeps `upsert`
idempotent - without it, a file missing its final newline would insert
its first TOC one way and every later re-run one byte shorter, since the
"remove an existing region" and "there was never one" paths would
otherwise start from differently-terminated strings.
"""
stripped = blocks.strip(text, REGION_NAME)
return stripped.rstrip("\n") + "\n" if stripped else stripped
def needs_toc(text: str) -> bool:
return len(body_without_region(text).splitlines()) > THRESHOLD
def iter_headings(body: str) -> list[Heading]:
"""Every `##`/`###` heading in document order, code-aware.
Detection runs on `markdown_code.strip_code_spans(body)`, which blanks
fenced blocks and inline code spans but preserves line structure - a
template line inside a fence (`# {Type name}`, a `raw/CONTRACT.md`
example path prefixed `#`) no longer starts with a real `#` marker once
masked, so it is not mistaken for a section. The heading *text* is read
back from the corresponding *unmasked* line, not the masked one: masking
also blanks inline code spans, which would otherwise turn a heading like
`` ## `instructions/dev/` `` into blank spaces instead of its real text.
"""
original = body.split("\n")
masked = markdown_code.strip_code_spans(body).split("\n")
headings: list[Heading] = []
for masked_line, original_line in zip(masked, original):
if not masked_line.startswith("#"):
continue
match = _HEADING_RE.match(masked_line)
if not match:
continue
# The masked line only still starts with `#...` for a genuine heading
# (fenced lines are blanked to spaces); recover the text from the
# unmasked line at the same position.
original_match = _HEADING_RE.match(original_line)
if not original_match:
continue
headings.append(Heading(level=len(match.group(1)), text=original_match.group(2).strip()))
return headings
def _slugify(text: str, seen: dict[str, int]) -> str:
"""A GitHub/Gitea-style anchor slug, de-duplicated like both renderers do
for a repeated heading (`foo`, `foo-1`, `foo-2`, ...)."""
base = _SLUG_DROP_RE.sub("", text.strip().lower())
base = _SLUG_SPACE_RE.sub("-", base).strip("-")
count = seen.get(base, 0)
seen[base] = count + 1
return base if count == 0 else f"{base}-{count}"
def render(headings: list[Heading]) -> str:
"""The TOC region ready to place into a body, or `""` for no headings."""
if not headings:
return ""
seen: dict[str, int] = {}
lines = []
for heading in headings:
indent = " " if heading.level == 3 else ""
slug = _slugify(heading.text, seen)
lines.append(f"{indent}- [{heading.text}](#{slug})")
return blocks.render(REGION_NAME, HEADING_TEXT, lines)
def _first_level2_offset(body: str) -> int | None:
"""Character offset of the first real (non-fenced) `## ` heading, or None."""
masked = markdown_code.strip_code_spans(body)
pos = 0
for masked_line in masked.split("\n"):
if masked_line.startswith("## "):
return pos
pos += len(masked_line) + 1
return None
def upsert(text: str) -> str:
"""`text` with its TOC region created or refreshed, or removed if the
file (measured without any existing region) is at or under the threshold.
Always strips first, then always reinserts fresh - never patches an
existing region in place. `blocks.replace`'s in-place path is right for
the links/footnotes regions it was built for (trailing material, appended
at the end when absent) but wrong here twice over: a TOC has to sit "at
the top" per the guidance this exists to satisfy, not at the end: and
`blocks.strip`'s removal collapses the blank lines around a matched
region to a single newline, which would make patching in place drift
from a fresh insertion's spacing on every second run. Always rebuilding
from the stripped body sidesteps both: the same deterministic
`rstrip("\\n") + "\\n\\n"` join runs whether or not a region existed
before, so a second `upsert` on its own output reproduces that output
exactly.
"""
body = body_without_region(text)
if not needs_toc(text):
return body
region = render(iter_headings(body))
if not region:
return body
offset = _first_level2_offset(body)
if offset is None:
return body.rstrip("\n") + "\n\n" + region + "\n"
head, tail = body[:offset], body[offset:]
return head.rstrip("\n") + "\n\n" + region + "\n\n" + tail
def stale_regions(text: str) -> bool:
"""Whether `text`'s current TOC region (if any) no longer matches what
`upsert` would write - drift `docs verify` can catch mechanically."""
return upsert(text) != text
+18 -1
View File
@@ -9,6 +9,23 @@ schema: types/type-spec.schema.yaml
A type-spec doc is the authoring and validation contract for one Chemenu artifact type. Artifact frontmatter stores the repo-relative path to this doc in `type:`, and this file serves several consumers differently: an author writing a new instance reads it for what to write, someone unfamiliar with the type reads it for what to expect from an instance before opening one, and the validator and the type-conformance reviewer read it to check conformance.
<!-- wikitool:toc -->
## Contents
- [Required Frontmatter](#required-frontmatter)
- [Validation Contract](#validation-contract)
- [Writing Shape](#writing-shape)
- [The type layer](#the-type-layer)
- [Who owns a type-spec](#who-owns-a-type-spec)
- [Anatomy of a type](#anatomy-of-a-type)
- [Placement frontmatter](#placement-frontmatter)
- [Adding a type](#adding-a-type)
- [Template variables](#template-variables)
- [Ownership boundary](#ownership-boundary)
- [What does not belong here](#what-does-not-belong-here)
- [Template](#template)
<!-- /wikitool:toc -->
## Required Frontmatter
- `type`: `types/type-spec.md`. The root type spec is self-referential and uses the same value.
@@ -208,4 +225,4 @@ Relevant Notes:
- This is the root type-spec that all other type-specs reference
- Type-specs form the structural backbone of the wiki's type system
- All validation flows through type-spec definitions
- All validation flows through type-spec definitions