diff --git a/.wikitool-kb.json b/.wikitool-kb.json index 99fd338..fefc492 100644 --- a/.wikitool-kb.json +++ b/.wikitool-kb.json @@ -1,6 +1,6 @@ { "schema": 1, - "kb_version": "5.0.0", + "kb_version": "9.0.0", "applied": [ { "migration": "3.0.0-authoring-conventions", @@ -16,6 +16,11 @@ "migration": "5.0.0-confidence-removal", "at": "2026-09-10", "pages": 152 + }, + { + "migration": "9.0.0-docs-diataxis", + "at": "2026-10-06", + "pages": 0 } ] } diff --git a/AGENTS.md b/AGENTS.md index 239f5ad..f62a219 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -89,7 +89,6 @@ What a file is called says who it is for and how it is loaded. This is a rule, n |------|-----|--------| | `README.md` | Humans - technical documentation and how to develop the thing in that directory | Never by an agent as instruction | | `EVALS.md` | Humans - how telemetry and evaluation work; routes to the contracts that bind | Never by an agent as instruction | -| `DEVELOPMENT.md` | Humans - the release workflow (`version bump`/`version release`/`publish`/CI), for whoever develops this stack rather than an instance built on it | Never by an agent as instruction. Not shipped: `dist_cmd.ROOT_FILES` excludes it deliberately, the same way `instructions/dev/` (which it may link to, unlike the documents `instructions verify` holds to that rule) is excluded - a distributed instance has no release workflow to document | | `AGENTS.md` | Agents | Always, every session | | `CLAUDE.md` | Agents on Claude Code | Automatically by that harness, which does not load `AGENTS.md` on its own - so it imports this file, carrying no rule of its own. It also links the one remaining Claude-Code-only decision (model/effort selection), per [instructions/CONTRACT.md](instructions/CONTRACT.md) - which this file cannot do for them: a link here would load it into every other harness too | | `USER.md` | Agents | Always, every session | @@ -103,7 +102,8 @@ What a file is called says who it is for and how it is loaded. This is a rule, n | `types/.md` | Agents + validator | Via `tools/wikitool types describe`. Split by `root:`: a page type-spec (`root: kb`) belongs to the instance and ships as `.template`; one describing a stack artifact ships verbatim | | `types/.guidance.md` | Agents + validator | Via `tools/wikitool types describe`, composed with the `types/.md` it documents. Stack-owned regardless of the type-spec's own `root:` - it ships verbatim and is optional, present only where the type-spec declares `guidance:` | | `types/..md` | Agents + `new` | Never as instruction: `wikitool new` copies it as the page skeleton for that one subtype value, in place of the type-spec's `## Template` block. Page material in the KB language, no frontmatter; its name is its only declaration, and `guidance` is reserved. Owned like the type-spec beside it, so it ships as `.template` | -| `docs/.md` | Agents and humans | By link, or on explicit request - never automatically, and never as instruction | +| `docs//.md` | Humans and agents - `` is one of `tutorials`, `how-to`, `reference`, `explanation` (the four kinds of documentation), and each area's `README.md` says what belongs in it | By link, or on explicit request - never automatically, and never as instruction | +| `docs/dev/.md` | Humans - for whoever develops this stack rather than an instance built on it (the release workflow, the development environment) | Never by an agent as instruction. Not shipped: `dist export` excludes `docs/dev/` the same way it excludes `instructions/dev/`, which it may therefore link to, unlike the documents `instructions verify` holds to that rule - a distributed instance has no release workflow to document | | `INDEX.md` | Both | Generated - never hand-edited | A stage may carry both a `README.md` and a `CONTRACT.md`: different readers, different @@ -128,39 +128,50 @@ whether it ever leaves this checkout. a further stage contract. Such a file is instance-owned end to end, which settles who may change it, not who reads it - and the reader is still an agent. There is deliberately no second language value beside `kb/CONVENTIONS.md`'s `language:`, and no instance setting that - moves this rule; [docs/language-boundaries.md](docs/language-boundaries.md) has the reasoning. + moves this rule; + [docs/explanation/language-boundaries.md](docs/explanation/language-boundaries.md) has the + reasoning. 2. **An agent speaks the instance's KB language**, whatever this file is written in. The value lives in `kb/CONVENTIONS.md`'s `language:` and nowhere else; an instruction that models a sentence for the user writes it in English like the rest of the control plane, and the agent says it in that language. +3. **The stack's human documentation is written in English** - `README.md`, `INSTALL*.md`, + `EVALS.md` and everything under `docs/` - whatever an instance's `language:` says. It is the + same text in every instance, and the parts of it that are generated come from English + sources, so there is no instance whose language it could follow. `CHANGES.md` is a log, not + documentation, and is not covered. -Nothing checks either mechanically - a stop-word scan would flag the quoted vocabulary above -and miss a translated paragraph that reads cleanly. They are held up by whoever writes an -instruction, which is why [instructions/CONTRACT.md](instructions/CONTRACT.md) § "Writing an -instruction" names them at the step where that happens. +Nothing checks any of the three mechanically - a stop-word scan would flag the quoted vocabulary +above and miss a translated paragraph that reads cleanly. They are held up by whoever writes the +text, which is why [instructions/CONTRACT.md](instructions/CONTRACT.md) § "Writing an +instruction" names the first two at the step where that happens. -**`docs/` carries no normative sentence.** It holds why the stack is built the way it is - -background consulted in passing, not a rule to follow; anything that would bind belongs in a +**`docs/` carries no normative sentence.** It holds the stack's human documentation - tutorials, +how-to guides and reference, and under `docs/explanation/` why the stack is built the way it is: +background consulted in passing, not a rule to follow. Anything that would bind belongs in a `CONTRACT.md` instead, which is what keeps invariant 8 intact here. It carries no frontmatter, -type, index, lint or provenance; `dist export` ships it verbatim and no other `tools/wikitool` -command touches it. +type, index, lint or provenance; `dist export` ships it verbatim except `docs/dev/`, `docs +verify` checks only that every file sits in one of its five subdirectories, and no other +`tools/wikitool` command touches it. -Six pages are reached from this file, each by link rather than automatically: -[docs/pipeline-rationale.md](docs/pipeline-rationale.md) (why the pipeline has four stages), -[docs/ownership-and-templates.md](docs/ownership-and-templates.md) (why a `.template` split -exists, why silent overwrite is the failure it guards against, and why an instance comes only -from a release), -[docs/language-boundaries.md](docs/language-boundaries.md) (why the control plane is English -everywhere and the KB language is a value, and why the axis is the reader rather than the -owner), [docs/why-gates-are-code.md](docs/why-gates-are-code.md) (why the five gates in -[Gates](#gates) are code rather than instruction), -[docs/version-model.md](docs/version-model.md) (why a version number answers a compatibility -question and a migration question separately), and -[docs/knowledge-and-commitment.md](docs/knowledge-and-commitment.md) (why commitments live in a -task tracker rather than in `kb/`, and why the two are joined at read time instead of synced). A -seventh, `docs/model-and-effort-selection.md`, is deliberately not linked here but from -`CLAUDE.md`: it decides something only that harness has to decide, and a link here would load it -into the other three. +Six explanation pages are reached from this file, each by link rather than automatically: +[docs/explanation/pipeline-rationale.md](docs/explanation/pipeline-rationale.md) (why the +pipeline has four stages), +[docs/explanation/ownership-and-templates.md](docs/explanation/ownership-and-templates.md) (why +a `.template` split exists, why silent overwrite is the failure it guards against, and why an +instance comes only from a release), +[docs/explanation/language-boundaries.md](docs/explanation/language-boundaries.md) (why the +control plane is English everywhere and the KB language is a value, and why the axis is the +reader rather than the owner), +[docs/explanation/why-gates-are-code.md](docs/explanation/why-gates-are-code.md) (why the five +gates in [Gates](#gates) are code rather than instruction), +[docs/explanation/version-model.md](docs/explanation/version-model.md) (why a version number +answers a compatibility question and a migration question separately), and +[docs/explanation/knowledge-and-commitment.md](docs/explanation/knowledge-and-commitment.md) (why +commitments live in a task tracker rather than in `kb/`, and why the two are joined at read time +instead of synced). A seventh, `docs/explanation/model-and-effort-selection.md`, is deliberately +not linked here but from `CLAUDE.md`: it decides something only that harness has to decide, and +a link here would load it into the other three. ## Personalization @@ -175,7 +186,7 @@ Both belong to one instance and one person, so a distribution ships only the `.t the Personalization step of [instructions/setup-instance.md](instructions/setup-instance.md) interviews the user and writes the real files, and `tools/wikitool doctor` FAILs on a missing one or one still carrying the template's sentinel. Why a `.template` rather than an absent -file: [docs/ownership-and-templates.md](docs/ownership-and-templates.md). +file: [docs/explanation/ownership-and-templates.md](docs/explanation/ownership-and-templates.md). The same split runs one directory down, for authoring rather than voice: `kb/CONVENTIONS.md` and each `kb//COLLECTION.md` bind every page, ship as templates, and are filled by the @@ -194,7 +205,7 @@ exists, and prefer what it says over asking the user the same question again. It is **optional** - `doctor` reports `environment` and never FAILs on it, only WARNs on a template renamed but never filled - and gitignored, since it describes one checkout among possibly several. Why an absent `ENVIRONMENT.md` is a lesser failure than a missing -`USER.md`/`SOUL.md`: [docs/ownership-and-templates.md](docs/ownership-and-templates.md). +`USER.md`/`SOUL.md`: [docs/explanation/ownership-and-templates.md](docs/explanation/ownership-and-templates.md). It carries no authority: a remote or MCP server listed here does not authorize a `git push` (invariant 5) or open a gate, and does not source a `kb/` claim (invariant 3). It holds no @@ -355,17 +366,18 @@ READMEs go in [CHANGES.md](CHANGES.md) - never in an inline version-history tabl *content* operations are logged separately via `tools/wikitool log append` into `kb/log.md`. **A stack change is not finished until the human docs describe it.** `README.md`, `EVALS.md`, -`tools/README.md`, `tools/CONTRACT.md` and the touched `/CONTRACT.md` are part of the -change that introduced a stage, a command or a workflow, not follow-up work: nobody comes back -for them, and a document that describes a repo which no longer exists is worse than none. What +`tools/README.md`, the pages under `docs/tutorials/`, `docs/how-to/`, `docs/reference/` and +`docs/dev/`, `tools/CONTRACT.md` and the touched `/CONTRACT.md` are part of the change +that introduced a stage, a command or a workflow, not follow-up work: nobody comes back for them, +and a document that describes a repo which no longer exists is worse than none. What `tools/wikitool docs verify` mechanically checks is exactly what its own `docs verify` row in [tools/CONTRACT.md](tools/CONTRACT.md) lists - no more. **Every prose field is outside that check** - a command's own summary, notes or retry-policy text, a stage contract's prose - and is -therefore session work, the same as the three README-shaped files. +therefore session work, the same as the README-shaped files. -`docs/` pages are held to a different clock than those three. A README goes stale on every new -flag; a `docs/` page goes stale only when the reasoning it wrote down stops holding - a gate -that stops living in code, an ownership line that moves, a boundary redrawn - which is rarer -and not tied to any one commit. Nothing checks this by construction: a page there carries no +`docs/explanation/` pages are held to a different clock than those. A README goes stale on +every new flag; an explanation page goes stale only when the reasoning it wrote down stops +holding - a gate that stops living in code, an ownership line that moves, a boundary redrawn - +which is rarer and not tied to any one commit. Nothing checks this by construction: a page there carries no normative sentence (see [File naming](#file-naming)), so there is no rule for `docs verify` to check, only a rationale for a session to notice has gone stale and to update or retire. diff --git a/CHANGES.md b/CHANGES.md index 0fb82dc..19a2990 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -14,11 +14,18 @@ place only: `instructions/dev/version-parts.md` § The candidate model. --- -## 8.1.0-beta.2 - 2026-10-06 - version bump: Abschlussmeldung nennt Stichpunkt oder Themenabsatz statt pauschal Prosa +## 9.0.0-beta.1 - 2026-10-06 - docs/ sorted into tutorials, how-to, reference and explanation plus an unshipped docs/dev/; explanation pages moved, developer guide moved and translated, human docs English **Author:** Torben Nehmer +**Breaking Change:** The seven pages that sat directly in docs/ now live in docs/explanation/, with nothing left at the old paths. An instance's own kb/CONVENTIONS.md still links ../docs/language-boundaries.md, so docs verify reports a dead link after the upgrade until the required migration document rewrites the path - step 12 of the upgrade. + +**Migration:** required - instructions/migrations/9.0.0-docs-diataxis.md + +**High impact** +- docs/ sorted into tutorials, how-to, reference and explanation plus an unshipped docs/dev/; explanation pages moved, developer guide moved and translated, human docs English + **Medium impact** - Changelog kompakt: ein Absatz pro Thema, Größenbudget, Migrationsverweis @@ -26,6 +33,16 @@ place only: `instructions/dev/version-parts.md` § The candidate model. - version bump: Abschlussmeldung nennt Stichpunkt oder Themenabsatz statt pauschal Prosa +### Doku-Gerüst nach Diátaxis unter `docs/` (#187) + +`docs/` hat vier Bereiche – `tutorials/`, `how-to/`, `reference/`, `explanation/`, je mit einer +`README.md` – und `docs/dev/`, das `dist export` wie `instructions/dev/` weglässt. Die sieben +Erklärseiten liegen jetzt unter `docs/explanation/`, ohne Stub am alten Ort; `DEVELOPMENT.md` ist +als `docs/dev/development.md` englisch. `docs verify` meldet jede Datei, die direkt in `docs/` oder +außerhalb der fünf Bereiche liegt. `AGENTS.md` § File naming legt fest, dass die menschliche Doku +des Stacks englisch ist, unabhängig von `language:`. Das bestehende `kb/CONVENTIONS.md` einer +Instanz verlinkt noch den alten Pfad, deshalb führt das Pflicht-Migrationsdokument ihn nach. + ### Changelog-Einträge mit Größenbudget und Migrationsverweis (#184) Ein Eintrag bekommt `###`-Abschnitte pro Thema statt pro Bump; eine kleinere Änderung steht nur diff --git a/CLAUDE.md b/CLAUDE.md index 7db0bed..6487324 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -15,4 +15,4 @@ Model and effort selection is the one remaining Claude-Code-only decision ([instructions/CONTRACT.md](instructions/CONTRACT.md#two-forms-three-reference-tiers) has the import-vs-link rule in general), and it earns a link rather than an import: a session stops to make this call - spawning a subagent, opening a review - rather than needing it pre-loaded before -it has done anything. See [docs/model-and-effort-selection.md](docs/model-and-effort-selection.md). +it has done anything. See [docs/explanation/model-and-effort-selection.md](docs/explanation/model-and-effort-selection.md). diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md deleted file mode 100644 index d01653b..0000000 --- a/DEVELOPMENT.md +++ /dev/null @@ -1,188 +0,0 @@ -# Entwicklung dieses Stacks - -Dieses Dokument richtet sich an Menschen, die an `tools/wikitool`, dem Type-Schema oder der -Instruction-/Skill-Schicht selbst arbeiten - nicht an den Konsumenten einer Instanz. Für die -Gegenseite (eine Instanz installieren, aktualisieren, betreiben) siehe [INSTALL.md](INSTALL.md). - -**Diese Datei wird nicht ausgeliefert.** Sie ist das menschliche Gegenstück zu -`instructions/dev/`, das `tools/wikitool dist export` vollständig ausschließt: eine -ausgelieferte Instanz hat keinen Release-Workflow, keine CI und kein Issue-Board, also braucht -sie auch keine Anleitung dafür. `dist_cmd.ROOT_FILES` listet sie deshalb bewusst nicht - der -Grund steht dort als Kommentar, damit eine spätere Sitzung die vermeintliche Lücke nicht -"repariert". Und weil sie nicht ausgeliefert wird, darf sie - anders als `README.md`, -`INSTALL.md` oder `EVALS.md`, die `instructions verify` auf genau diesen Punkt prüft - nach -`instructions/dev/` verlinken. - -## Entwicklungsumgebung - -Am Stack wird in einem Klon dieses Repos gearbeitet. Ein solcher Klon ist **keine Instanz** und -wird nie eine. Instanzen entstehen ausschließlich aus Releases, siehe [INSTALL.md](INSTALL.md). - -```bash -git clone https://gitea.nehmer.net/torben/chemenu.git -cd chemenu -``` - -Danach den Agenten `instructions/bootstrap.md` ausführen lassen: Preflight, dann -`tools/wikitool instructions sync`. Die Agenten-Seite dazu - was hier anders ist als in einer -Instanz und wie `dist export` als Testwerkzeug läuft - steht in -[instructions/dev/dev-setup.md](instructions/dev/dev-setup.md). - -Was ein Klon mitbringt und eine Instanz nicht: - -- **Den Demo-Korpus.** Rund 170 Seiten, die den Stack selbst dokumentieren: Gates, Lint, - Versionierung, Suche, das Wiki-Muster. Er ist Testbett und begehbares Beispiel, keine - produktive Wissensbasis. Was an ihm geändert werden darf, regelt - [instructions/dev/corpus-policy.md](instructions/dev/corpus-policy.md). -- **Eine Demo-Persona** in `USER.md`/`SOUL.md`. `doctor` meldet beide als ausgefüllt. Sie - beschreiben den Demo-Betrieb, nicht dich. -- **Telemetrie an.** Ohne `.wikitool-release.json` ist der Klon die Messstation, mit der der - Stack sich selbst bewertet ([EVALS.md](EVALS.md) § „Whether it runs at all“). -- **`instructions/dev/`, `commonplace/`, `.gitea/` und diese Datei.** `dist export` liefert - davon nichts aus. - -`ENVIRONMENT.md` fehlt nach jedem Klon, weil die Datei gitignored ist: Sie beschreibt einen -Checkout, nicht das Repo. Wer sie anlegt (Vorlage `ENVIRONMENT.md.template`, Schritt 5 in -`instructions/bootstrap.md`), erspart jeder Stack-Sitzung die Fragen nach Harness, `gitea-mcp` -und Remote. - -### `dist export` als Build- und Testwerkzeug - -`tools/wikitool dist export ` schreibt genau den Baum, den ein Release -ausliefert: Maschinerie ohne Wiki-Inhalt, ohne Git-Historie, ohne `instructions/dev/`. Damit -prüft man vor einem Release, was ausgeliefert würde (`--dry-run` listet es nur). Und man spielt -den Installationsweg nach, ohne auf ein Release zu warten: Tarball daraus bauen wie -`.gitea/workflows/release.yml`, dann das Preflight-Skript in einem leeren Verzeichnis mit -`--archive ` starten. Der CI-Schritt „The distribution works as a fresh instance“ in -`.gitea/workflows/ci.yml` macht genau das. - -Für eine echte Instanz ist ein solcher Export kein Weg. Ihm fehlen die Release-Herkunft im -Stamp, und `dist upgrade --latest` vergleicht später gegen einen Stand, den es nie als Release -gab. - -### Ein Release-Feed in einem nicht öffentlichen Repo - -Wer den Stack in einem eigenen, nicht öffentlichen Repo betreibt und Instanzen von dort -aktualisiert, lässt `WIKITOOL_UPDATE_URL` auf dessen Feed zeigen. Dabei gibt es eine Eigenheit: -Gitea antwortet anonymen Aufrufern für ein unsichtbares Repo mit demselben `404` wie für ein gar -nicht existierendes. Ein fehlendes Release und ein fehlender Zugriff sehen dann identisch aus – -„kein Update gefunden“ wäre in dem Fall schlicht falsch. Dagegen hilft ein Gitea-Token mit -Lesezugriff in `WIKITOOL_UPDATE_TOKEN`. Es geht nur an Downloads auf demselben Host wie der Feed. - -## Der Release-Ablauf - -Zwischen zwei Releases führt der Stack **einen** laufenden Versionskandidaten statt einer neuen -Nummer pro Bump. Das volle Modell - Zustandsort, Eskalationslogik, warum eine Nummer erst durch -ein Release verbraucht wird - steht in -[instructions/dev/version-parts.md](instructions/dev/version-parts.md) und -[docs/version-model.md](docs/version-model.md). Hier nur der Ablauf, in der Reihenfolge, in der -eine Sitzung ihn tatsächlich durchläuft: - -1. **Bump eröffnet oder eskaliert den Kandidaten, gewichtet mit `--impact`.** - - ```bash - tools/wikitool version bump --minor --title "Was sich geändert hat" --impact medium - ``` - - Schreibt `VERSION` als `X.Y.Z-beta.N` und öffnet (oder aktualisiert) den passenden - `CHANGES.md`-Eintrag. Mehrere Bumps für dieselbe Änderung sind normal - jeder aktualisiert - denselben Eintrag, statt einen neuen zu eröffnen. `--impact high|medium|low` (Default - `medium`) gruppiert den Eintrag; `tools/wikitool version regrade` korrigiert eine Note später, - wenn der Gesamteindruck des Kandidaten den Blick auf einen früheren Bump ändert. - -2. **Der Eintrag bekommt seine Prosa - nur wo sie nötig ist.** `bump` schreibt das Skelett - (Heading, Datum, Autor, die maschinenverwaltete, gruppierte Bump-Titel-Liste, ggf. - Breaking- und Migrationszeile). Für eine kleinere Änderung genügt ihr Bump-Titel; eine größere - bekommt einen Absatz zu ihrem Thema, und vor dem Release kommt eine kurze Zusammenfassung - dazu. Form und Größenbudget, das `docs verify` und `version release` prüfen, stehen in - [instructions/dev/version-parts.md](instructions/dev/version-parts.md) § The candidate model. - -3. **Verify laufen lassen, bevor irgendetwas gepublished wird:** - - ```bash - cd tools && .venv/bin/python -m pytest -q - tools/wikitool docs verify - tools/wikitool instructions verify - ``` - -4. **`version release` fixiert den Kandidaten**, sobald er ausgeliefert werden soll: - - ```bash - tools/wikitool version release --title "Zusammenfassender Titel" - ``` - - Streicht den `-beta.N`-Suffix aus `VERSION` und schließt den Changelog-Eintrag. `--title` ist - optional - ohne ihn bleibt der Titel des letzten Bumps stehen; mit ihm bekommt ein Kandidat, - der mehrere Bump-Titel gesammelt hat, eine zusammenfassende Überschrift. Verweigert, wenn der - Kandidat zwei oder mehr Bumps gesammelt hat und die Zusammenfassung aus Schritt 2 noch fehlt - - ein Kandidat mit genau einem Bump ist davon ausgenommen -, und ebenso, wenn der Eintrag sein - Größenbudget überschreitet. Committet und pusht nichts - (Invariante 5 in [AGENTS.md](AGENTS.md)). - -5. **Publish bewegt `VERSION` auf `main`.** - - ```bash - tools/wikitool publish --message "..." - ``` - - Das Mass-Update-Gate und das Publish-Remote-Gate gelten wie bei jedem anderen Publish - - siehe [instructions/gates.md](instructions/gates.md). - -6. **CI übernimmt den Rest.** `.gitea/workflows/release.yml` reagiert auf jeden Push, der - `VERSION` bewegt: Eine suffixbehaftete `VERSION` (ein Kandidat) lässt den Job sauber - überspringen, bevor er die Releases-API überhaupt anfragt - Betas werden nie veröffentlicht. - Eine suffixfreie `VERSION` baut die Distribution (`dist export`), erzeugt Tag und Release und - lädt Tarball und Prüfsumme hoch, dazu die beiden Preflight-Skripte und die Anleitungen - `setup-instance.md` und `preflight.md`, auf die der Installationssatz in `INSTALL.md` zeigt. - **CI setzt den Tag, nie eine Sitzung** - das hält Invariante 5 intakt. - - Die Release-Notiz ist der `CHANGES.md`-Eintrag; Gitea speichert auf MySQL höchstens 65535 - Bytes, und der Job verweigert ab 60000, bevor er einen Tag anlegt. Das ist nur das letzte - Netz: `docs verify` hält den obersten Eintrag schon während des Kandidaten bei höchstens - 32000 Bytes (Schritt 2). Scheitert der Job, nachdem - `VERSION` schon auf `main` steht, hilft weder ein Push (die Version steigt nicht noch einmal) - noch ein Re-run (er nimmt die Workflow-Datei des gescheiterten Commits): Ursache beheben, - publishen und `release.yml` per `workflow_dispatch` auf `main` starten. Die Prüfung auf ein - schon vorhandenes Release verhindert ein zweites. - -Die drei Verify-Befehle stehen oben in Schritt 3; was jeder von ihnen prüft, steht in -[tools/CONTRACT.md](tools/CONTRACT.md) und wird dort von `docs verify` gegen die tatsächliche -CLI gehalten. Hier steht es bewusst **nicht** noch einmal: eine zweite Beschreibung derselben -Befehle ist genau die Kopie, die driftet (AGENTS.md Invariante 8), und dieses Dokument liegt -außerhalb der Dateien, die der Kommando-Datensatz-Check von `docs verify` abdeckt - hier fällt eine -Drift also niemandem auf. Was `pytest` an dieser Stelle vom Entwickler erwartet, steht in -[instructions/dev/testing-conventions.md](instructions/dev/testing-conventions.md). - -## Die CI-Hälfte - -`.gitea/workflows/ci.yml` läuft auf jeden Push/PR gegen `main` (Content-Pfade ausgenommen) und -führt Testsuite, `docs verify`, `instructions verify` sowie einen vollständigen -`setup-instance.md`-Replay aus: ein lokal gebauter Release-Tarball, das Preflight-Skript mit -`--archive` in einem leeren Verzeichnis, dann die Schritte der Anleitung - derselbe Pfad, den -ein neuer Nutzer tatsächlich geht, nur ohne Download. `.gitea/workflows/nightly.yml` ist der Drift-Check gegen die Zeit statt -gegen einen Commit. `.gitea/workflows/release.yml` ist Schritt 6 oben. - -Drei weitere Workflows tragen die Live-Tests der Tracker-Adapter (Super Productivity, CalDAV): -`ci.yml` führt in einem eigenen Schritt die CalDAV-Hälfte gegen ein Radicale als Prozess aus, -`tracker-live.yml` läuft nachts und deckt beide Anbieter ab, und `sp-live-image.yml` baut täglich -(bei neuer Version) und monatlich (immer) das Image `chemenu-sp-live` mit der jeweils aktuellen -Super-Productivity-Version. Das Image folgt dem Update-Kanal der Desktop-Clients, nicht einer -festen Version. Einmalig nach dem allerersten Push muss das Paket von Hand dem Repo -`torben/chemenu` zugeordnet werden. Was die Suite schreibt, wie man sie gegen einen eigenen -Tracker laufen lässt und was ein roter Lauf bedeutet, steht in -[instructions/dev/tracker-testing.md](instructions/dev/tracker-testing.md). - -## Stack-Entwicklung als eigener Sitzungstyp - -Drei Skills (`instructions/dev/`, nur in diesem Ursprungs-Repo vorhanden) führen eine Sitzung, -die den Stack selbst statt Wiki-Inhalt bearbeitet, durch drei Phasen: `stack-dev` arbeitet das -Issue aus, bis sein Body „ready“ ist, `stack-build` baut, publiziert und wartet auf einen grünen -CI-Lauf, `stack-close` prüft den Endzustand des Bodys und veraltete `docs/`- und Contract-Prosa -und schließt das Issue. Übergeben wird über den Zustand im Tracker, nicht über den Kontext einer -Sitzung: Jeder Phasenwechsel geht in derselben Sitzung oder nach `/clear`. `stack-dev` greift -automatisch; `stack-build` und `stack-close` startet nur der Betreiber per Slash-Kommando -(`/stack-build #N`, `/stack-close`) - an genau dieser Stelle fällt die Wahl, ob es in derselben -Sitzung weitergeht oder in einer neuen, auf welchem Modell. Einen Modellwechsel mitten in der -Sitzung bietet keiner der drei an. Regeln und Phasentabelle stehen in -[instructions/dev/stack-mode.md](instructions/dev/stack-mode.md), der Issue-Tracker selbst in -[instructions/dev/issue-tracking.md](instructions/dev/issue-tracking.md). diff --git a/INSTALL.md b/INSTALL.md index 7aa4843..0f01b4f 100644 --- a/INSTALL.md +++ b/INSTALL.md @@ -13,7 +13,7 @@ Terminal auf dieser Maschine ist. Wer am Stack selbst arbeiten will, klont dieses Repo. Das ist eine Entwicklungsumgebung mit -Demo-Korpus und keine Instanz; sie steht in [DEVELOPMENT.md](DEVELOPMENT.md). +Demo-Korpus und keine Instanz; sie steht in [docs/dev/development.md](docs/dev/development.md). ## Was vorher da sein muss diff --git a/README.md b/README.md index 37cc35f..e6d9337 100644 --- a/README.md +++ b/README.md @@ -53,10 +53,26 @@ Two starting points, depending on what you're doing: - **Working on the stack itself.** A clone of this repository is a development checkout, with the demo corpus described below; it is never an instance. Setting it up, and `dist export` as - the build and test tool it is, are in `DEVELOPMENT.md` (for you) and + the build and test tool it is, are in `docs/dev/development.md` (for you) and `instructions/dev/dev-setup.md` (for the agent). +## Documentation + +The human documentation lives under `docs/`, sorted by what you want from it: + +- [docs/tutorials/](docs/tutorials/README.md) - learning: a guided first walk through an + instance. Installation is still [INSTALL.md](INSTALL.md). +- [docs/how-to/](docs/how-to/README.md) - doing: one task per page, for the operator of an + instance. +- [docs/reference/](docs/reference/README.md) - looking up: commands, exit codes, skills, page + types. The command reference is [tools/CONTRACT.md](tools/CONTRACT.md). +- [docs/explanation/](docs/explanation/README.md) - understanding: why the stack is built the way + it is. + +The areas are filling up gradually; until a page exists there, the sections below still carry +its material. + ## Architecture ``` @@ -76,6 +92,7 @@ chemenu/ │ # incoming/ is the one directory excluded the other way round ├── .github/hooks/ # Copilot CLI hooks - session tracing ├── .vibe/ # Mistral Vibe hooks + the repo's telemetry policy +├── docs/ # Human documentation: tutorials/, how-to/, reference/, explanation/ ├── instructions/ # CONTROL: everything an agent is told to do │ ├── CONTRACT.md # Instruction vs. skill, publishing, writing standard │ ├── bootstrap.md # Prepare a further checkout of an instance @@ -155,7 +172,7 @@ chemenu/ Dev-instance-only (see `tools/CONTRACT.md` for how it got here): ``` -├── DEVELOPMENT.md # Human-readable: the release workflow (version bump/release/publish/CI) +├── docs/dev/ # Human-readable: development environment and release workflow └── commonplace/ # Vendored, read-only knowledge base ``` @@ -294,7 +311,7 @@ way `task new` opened it, or the way the weekly review proposes it for a `waiting_overdue`/`someday_stale` finding once you confirm. Why the split runs this way, rather than syncing the two: -[docs/knowledge-and-commitment.md](docs/knowledge-and-commitment.md). +[docs/explanation/knowledge-and-commitment.md](docs/explanation/knowledge-and-commitment.md). ## Entity Types diff --git a/VERSION b/VERSION index 1ab9b49..93bdcf1 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -8.1.0-beta.2 +9.0.0-beta.1 diff --git a/docs/dev/development.md b/docs/dev/development.md new file mode 100644 index 0000000..1644c3b --- /dev/null +++ b/docs/dev/development.md @@ -0,0 +1,196 @@ +# Developing this stack + +This document is for people working on `tools/wikitool`, the type schema or the +instruction/skill layer itself - not for the operators of an instance. For the other side +(installing, upgrading and running an instance) see [INSTALL.md](../../INSTALL.md). + +**This file is not shipped.** It lives under `docs/dev/`, the human counterpart of +`instructions/dev/`, and `tools/wikitool dist export` excludes both directories wholesale: a +distributed instance has no release workflow, no CI and no issue board, so it needs no guide to +them. The reason is a comment at `dist_cmd.DOCS_DEV_ONLY`, so that a later session does not +"repair" the apparent gap. And because it is not shipped, this file may link into +`instructions/dev/` - unlike a shipped document such as `README.md`, which +`instructions verify` checks on exactly that point. + + +## Contents + +- [Development environment](#development-environment) + - [`dist export` as a build and test tool](#dist-export-as-a-build-and-test-tool) + - [A release feed in a non-public repository](#a-release-feed-in-a-non-public-repository) +- [The release workflow](#the-release-workflow) +- [The CI half](#the-ci-half) +- [Stack development as a session type of its own](#stack-development-as-a-session-type-of-its-own) + + +## Development environment + +Work on the stack happens in a clone of this repository. Such a clone is **not an instance** and +never becomes one. Instances come only from releases - see [INSTALL.md](../../INSTALL.md). + +```bash +git clone https://gitea.nehmer.net/torben/chemenu.git +cd chemenu +``` + +Then have the agent run `instructions/bootstrap.md`: the preflight, then +`tools/wikitool instructions sync`. The agent's side of it - what differs here from an instance, +and how `dist export` runs as a test tool - is +[instructions/dev/dev-setup.md](../../instructions/dev/dev-setup.md). + +What a clone has and an instance does not: + +- **The demo corpus.** Some 170 pages documenting the stack itself: gates, lint, versioning, + search, the wiki pattern. It is a testbed and a walkable example, not a productive knowledge + base. What may be changed in it is governed by + [instructions/dev/corpus-policy.md](../../instructions/dev/corpus-policy.md). +- **A demo persona** in `USER.md`/`SOUL.md`. `doctor` reports both as filled in. They describe + the demo setup, not you. +- **Telemetry on.** Without `.wikitool-release.json` the clone is the measuring station the stack + evaluates itself with ([EVALS.md](../../EVALS.md) § "Whether it runs at all"). +- **`instructions/dev/`, `docs/dev/`, `commonplace/` and `.gitea/`.** `dist export` ships none + of them. + +`ENVIRONMENT.md` is missing after every clone, because the file is gitignored: it describes a +checkout, not the repository. Creating it (template `ENVIRONMENT.md.template`, step 5 in +`instructions/bootstrap.md`) spares every stack session the questions about harness, `gitea-mcp` +and remote. + +### `dist export` as a build and test tool + +`tools/wikitool dist export ` writes exactly the tree a release ships: the +machinery without wiki content, without git history, without `instructions/dev/` or `docs/dev/`. +That is how you check before a release what would ship (`--dry-run` only lists it). And it is +how you replay the installation path without waiting for a release: build a tarball from it the +way `.gitea/workflows/release.yml` does, then start the preflight script in an empty directory +with `--archive `. The CI step "The distribution works as a fresh instance" in +`.gitea/workflows/ci.yml` does exactly that. + +For a real instance such an export is not a way in. It lacks the release origin in its stamp, +and `dist upgrade --latest` later compares against a state that never existed as a release. + +### A release feed in a non-public repository + +Whoever runs the stack from a repository of their own that is not public, and upgrades instances +from there, points `WIKITOOL_UPDATE_URL` at its feed. There is one quirk: Gitea answers an +anonymous caller with the same `404` for an invisible repository as for one that does not exist +at all. A missing release and missing access then look identical - and "no update found" would +simply be wrong. A Gitea token with read access in `WIKITOOL_UPDATE_TOKEN` fixes that. It is sent +only to downloads on the same host as the feed. + +## The release workflow + +Between two releases the stack carries **one** running version candidate instead of a new +number per bump. The full model - where the state lives, how escalation works, why a number is +only consumed by a release - is in +[instructions/dev/version-parts.md](../../instructions/dev/version-parts.md) and +[docs/explanation/version-model.md](../explanation/version-model.md). Here only the sequence, in +the order a session actually goes through it: + +1. **A bump opens or escalates the candidate, graded with `--impact`.** + + ```bash + tools/wikitool version bump --minor --title "What changed" --impact medium + ``` + + Writes `VERSION` as `X.Y.Z-beta.N` and opens (or updates) the matching `CHANGES.md` entry. + Several bumps for the same change are normal - each one updates the same entry instead of + opening a new one. `--impact high|medium|low` (default `medium`) groups the entry; + `tools/wikitool version regrade` corrects a grade later, when the candidate's overall shape + changes the read on an earlier bump. + +2. **The entry gets its prose - only where it is needed.** `bump` writes the skeleton (heading, + date, author, the machine-managed, grouped list of bump titles, and the breaking and migration + lines where they apply). A smaller change needs nothing beyond its bump title; a larger one + gets a paragraph for its topic, and a short summary is added before the release. The form, and + the size budget `docs verify` and `version release` check, are in + [instructions/dev/version-parts.md](../../instructions/dev/version-parts.md) § The candidate + model. + +3. **Run the checks before anything is published:** + + ```bash + cd tools && .venv/bin/python -m pytest -q + tools/wikitool docs verify + tools/wikitool instructions verify + ``` + +4. **`version release` fixes the candidate** once it is meant to ship: + + ```bash + tools/wikitool version release --title "Summarising title" + ``` + + Strips the `-beta.N` suffix from `VERSION` and closes the changelog entry. `--title` is + optional - without it the last bump's title stays; with it, a candidate that collected several + bump titles gets a summarising heading. Refused when the candidate collected two or more bumps + and the summary from step 2 is still missing - a candidate with exactly one bump is exempt - + and likewise when the entry is over its size budget. Commits and pushes nothing (invariant 5 + in [AGENTS.md](../../AGENTS.md)). + +5. **Publish moves `VERSION` on `main`.** + + ```bash + tools/wikitool publish --message "..." + ``` + + The Mass-Update Gate and the Publish-Remote Gate apply as on every other publish - see + [instructions/gates.md](../../instructions/gates.md). + +6. **CI does the rest.** `.gitea/workflows/release.yml` reacts to every push that moves + `VERSION`: a `VERSION` with a suffix (a candidate) makes the job skip cleanly before it even + queries the releases API - betas are never published. A suffix-free `VERSION` builds the + distribution (`dist export`), creates the tag and the release, and uploads the tarball and its + checksum, along with the two preflight scripts and the instructions `setup-instance.md` and + `preflight.md` that the installation sentence in `INSTALL.md` points at. **CI sets the tag, + never a session** - that keeps invariant 5 intact. + + The release note is the `CHANGES.md` entry; Gitea on MySQL stores at most 65535 bytes, and + the job refuses from 60000 on, before it creates a tag. That is only the last net: `docs + verify` already holds the topmost entry to at most 32000 bytes while it is a candidate (step + 2). If the job fails after `VERSION` is already on `main`, neither a push (the version does + not rise again) nor a re-run (it takes the workflow file of the failed commit) helps: fix the + cause, publish, and start `release.yml` on `main` via `workflow_dispatch`. The check for an + existing release prevents a second one. + +The three check commands are in step 3 above; what each of them checks is in +[tools/CONTRACT.md](../../tools/CONTRACT.md), where `docs verify` holds it against the actual +CLI. It is deliberately **not** repeated here: a second description of the same commands is +exactly the copy that drifts (AGENTS.md invariant 8), and this document lies outside the files +the command-record check of `docs verify` covers - so a drift here would go unnoticed. What +`pytest` expects of the developer at this point is in +[instructions/dev/testing-conventions.md](../../instructions/dev/testing-conventions.md). + +## The CI half + +`.gitea/workflows/ci.yml` runs on every push/PR against `main` (content paths excepted) and +runs the test suite, `docs verify`, `instructions verify` and a complete `setup-instance.md` +replay: a locally built release tarball, the preflight script with `--archive` in an empty +directory, then the instruction's steps - the same path a new user actually takes, just without +the download. `.gitea/workflows/nightly.yml` is the drift check against time rather than against +a commit. `.gitea/workflows/release.yml` is step 6 above. + +Three more workflows carry the live tests of the tracker adapters (Super Productivity, CalDAV): +`ci.yml` runs the CalDAV half in a step of its own against Radicale as a process, +`tracker-live.yml` runs nightly and covers both providers, and `sp-live-image.yml` builds the +`chemenu-sp-live` image daily (when there is a new version) and monthly (always) with the current +Super Productivity version. The image follows the desktop clients' update channel, not a fixed +version. Once, after the very first push, the package has to be linked to the `torben/chemenu` +repository by hand. What the suite writes, how to run it against a tracker of your own, and what +a red run means is in +[instructions/dev/tracker-testing.md](../../instructions/dev/tracker-testing.md). + +## Stack development as a session type of its own + +Three skills (`instructions/dev/`, present only in this origin repository) carry a session that +works on the stack itself rather than on wiki content through three phases: `stack-dev` works +the issue out until its body is "ready", `stack-build` builds, publishes and waits for a green CI +run, `stack-close` checks the body's final state and stale `docs/` and contract prose, and closes +the issue. The handover goes through the state in the tracker, not through a session's context: +every phase change works in the same session or after `/clear`. `stack-dev` triggers +automatically; `stack-build` and `stack-close` are started only by the operator, by slash command +(`/stack-build #N`, `/stack-close`) - exactly where the choice falls whether to continue in the +same session or a new one, and on which model. None of the three offers a model switch in the +middle of a session. Rules and the phase table are in +[instructions/dev/stack-mode.md](../../instructions/dev/stack-mode.md), the issue tracker itself +in [instructions/dev/issue-tracking.md](../../instructions/dev/issue-tracking.md). diff --git a/docs/explanation/README.md b/docs/explanation/README.md new file mode 100644 index 0000000..d95269e --- /dev/null +++ b/docs/explanation/README.md @@ -0,0 +1,18 @@ +# Explanation + +Why the stack is built the way it is. These pages are background for humans and agents alike: +read by link or on request, never as instruction, and they carry no normative sentence - a rule +lives in `AGENTS.md` or a `CONTRACT.md`, and a page here only gives its reasons. + +- [pipeline-rationale.md](pipeline-rationale.md) - why the pipeline has four stages +- [ownership-and-templates.md](ownership-and-templates.md) - why a `.template` split exists, and + why an instance comes only from a release +- [language-boundaries.md](language-boundaries.md) - why the control plane is English everywhere + and the KB language is a value +- [why-gates-are-code.md](why-gates-are-code.md) - why the gates are code rather than instruction +- [version-model.md](version-model.md) - why a version number answers a compatibility question + and a migration question separately +- [knowledge-and-commitment.md](knowledge-and-commitment.md) - why commitments live in a task + tracker rather than in `kb/` +- [model-and-effort-selection.md](model-and-effort-selection.md) - which model and effort suit + which kind of session, for the harness that has to choose diff --git a/docs/knowledge-and-commitment.md b/docs/explanation/knowledge-and-commitment.md similarity index 99% rename from docs/knowledge-and-commitment.md rename to docs/explanation/knowledge-and-commitment.md index e8da208..7f68ce7 100644 --- a/docs/knowledge-and-commitment.md +++ b/docs/explanation/knowledge-and-commitment.md @@ -4,8 +4,8 @@ Chemenu compiles knowledge into `kb/`, and it also tracks what its operator has Those look like one subject - both are "things about my projects" - and the stack deliberately keeps them apart: `kb/gtd/` holds one page per initiative, an external task tracker holds the open items, and the only thing that crosses between them is a name. This page is about why that -line was drawn there. The rules that follow from it live in [kb/CONTRACT.md](../kb/CONTRACT.md) -and the `review`, `new` and `task new` records in [tools/CONTRACT.md](../tools/CONTRACT.md). +line was drawn there. The rules that follow from it live in [kb/CONTRACT.md](../../kb/CONTRACT.md) +and the `review`, `new` and `task new` records in [tools/CONTRACT.md](../../tools/CONTRACT.md). ## Contents diff --git a/docs/language-boundaries.md b/docs/explanation/language-boundaries.md similarity index 79% rename from docs/language-boundaries.md rename to docs/explanation/language-boundaries.md index ad31ba9..2ecd6dc 100644 --- a/docs/language-boundaries.md +++ b/docs/explanation/language-boundaries.md @@ -2,10 +2,12 @@ Two languages run through this repo at once. `kb/` is written in whatever language the instance chose - German here, and the value lives in `kb/CONVENTIONS.md`'s `language:`. Everything that -tells an agent what to do - [AGENTS.md](../AGENTS.md), every `CONTRACT.md`, everything under -`instructions/` - is written in English, in every instance, whatever the first value says. +tells an agent what to do - [AGENTS.md](../../AGENTS.md), every `CONTRACT.md`, everything under +`instructions/` - is written in English, in every instance, whatever the first value says. So +is the stack's own documentation for humans, for a different reason (§ "The stack's human +documentation"). -The rule itself is in [AGENTS.md § File naming](../AGENTS.md#file-naming). This page holds the +The rule itself is in [AGENTS.md § File naming](../../AGENTS.md#file-naming). This page holds the part that is not a rule: why the line runs where it does, why the English half is not a setting, and which argument for it turned out to be wrong. @@ -17,6 +19,7 @@ and which argument for it turned out to be wrong. - [Why it is not a parameter](#why-it-is-not-a-parameter) - [What the KB language still decides](#what-the-kb-language-still-decides) - [Where the line runs around a page type](#where-the-line-runs-around-a-page-type) +- [The stack's human documentation](#the-stacks-human-documentation) - [What would put this back on the table](#what-would-put-this-back-on-the-table) @@ -35,7 +38,7 @@ guidance is still instruction addressed to an agent, and reads exactly like the four types the stack ships. Ownership says "yours"; the audience has not moved at all. So the question a line answers is not *whose file is this* but *who reads this line*, which is -the same cut [kb/CONTRACT.md](../kb/CONTRACT.md#language-and-identifiers) already makes inside a +the same cut [kb/CONTRACT.md](../../kb/CONTRACT.md#language-and-identifiers) already makes inside a single page between prose and identifiers - applied one level up, to the halves of a document. Ownership decides who may change a sentence. The reader decides what language it is in. The two questions were answered together for as long as they happened to agree. @@ -99,7 +102,8 @@ experiences: language. So an operator who reads no English gets German pages and German answers from an agent reading -English instructions. The English is what the machinery is written in, not what it says back. +English instructions. The English is what the machinery is written in, not what it says back - +with one exception, the stack's documentation for humans (§ "The stack's human documentation"). ## Where the line runs around a page type @@ -115,11 +119,28 @@ beside the template, but that split was made for ownership reasons - so an upgra the guidance without overwriting what the instance chose - and it leaves this page's argument untouched: each part is still written in the language its own reader needs, and a type-spec that declares no `guidance:` keeps both halves in one file with exactly the same rule applying inside -it. [types/type-spec.md § Who owns a type-spec](../types/type-spec.md#who-owns-a-type-spec) has +it. [types/type-spec.md § Who owns a type-spec](../../types/type-spec.md#who-owns-a-type-spec) has the split as a table, and [ownership-and-templates.md](ownership-and-templates.md) § "Where the file boundary used to strain" has what it cost to keep two audiences in one file for as long as it did. +## The stack's human documentation + +`README.md`, `INSTALL*.md`, `EVALS.md` and everything under `docs/` are read by humans, and the +reader axis alone would put them in the operator's language. They are English anyway, and the +reason is not the reader but the fact that there is no single one. This documentation belongs to +the stack, not to an instance: it ships identically to every instance, whatever each one's +`language:` says, so there is no instance value it could follow - only a translation per language +to maintain beside it, each one a second copy that drifts. And a growing part of it is generated +from the code (the command reference from `cli_contract` records, for one), whose source is +English control-plane material; a translated frame around English generated text is the +half-done version of both. + +The KB language keeps deciding what an operator reads *of their own*: pages and answers. What +the stack tells every operator about itself is the same text everywhere. `CHANGES.md` sits +outside the question - it is a log of what was true at a time, not documentation, and its older +entries are in the language they were written in. + ## What would put this back on the table A `docs/` page goes stale when the reasoning stops holding rather than when the code changes, so @@ -135,3 +156,8 @@ it is worth naming what that would look like here. Two things would: Neither is close. Both are cheaper to notice here than to rediscover in an argument about a single file. + +The human documentation has a trigger of its own: **translations the stack maintains itself, +with a check that holds each one to its English source.** Without such a check a translation is +exactly the drifting second copy the section above argues against; with one, the argument for a +single language weakens to cost alone. diff --git a/docs/model-and-effort-selection.md b/docs/explanation/model-and-effort-selection.md similarity index 99% rename from docs/model-and-effort-selection.md rename to docs/explanation/model-and-effort-selection.md index 5cc4cd6..ef9d840 100644 --- a/docs/model-and-effort-selection.md +++ b/docs/explanation/model-and-effort-selection.md @@ -2,7 +2,7 @@ Claude Code exposes three choices this repo has an opinion on: which model a session itself runs as, what model a spawned subagent gets, and which `/code-review` effort level to pick. -None of them are enforced anywhere - the gates in [instructions/gates.md](../instructions/gates.md) +None of them are enforced anywhere - the gates in [instructions/gates.md](../../instructions/gates.md) are code precisely because a model cannot be talked out of them ([why-gates-are-code.md](why-gates-are-code.md) makes that argument for gates; this page applies the same axis to who is holding the keyboard). What follows is a reference for making that choice diff --git a/docs/ownership-and-templates.md b/docs/explanation/ownership-and-templates.md similarity index 96% rename from docs/ownership-and-templates.md rename to docs/explanation/ownership-and-templates.md index 1745d7e..fd2559d 100644 --- a/docs/ownership-and-templates.md +++ b/docs/explanation/ownership-and-templates.md @@ -2,8 +2,8 @@ Chemenu ships two kinds of files side by side, and at a glance they look the same: both are plain markdown, both sit in the repo root or under `kb/`, both get read at session start. But a -stack upgrade treats them completely differently. Some - [AGENTS.md](../AGENTS.md), -[kb/CONTRACT.md](../kb/CONTRACT.md), the per-stage contracts - are identical in every instance +stack upgrade treats them completely differently. Some - [AGENTS.md](../../AGENTS.md), +[kb/CONTRACT.md](../../kb/CONTRACT.md), the per-stage contracts - are identical in every instance that runs this stack and are the next release's to replace (with one caveat about local edits, below). Others - `USER.md`, `SOUL.md`, `kb/CONVENTIONS.md`, `ENVIRONMENT.md` - describe one particular instance, and @@ -25,7 +25,7 @@ overwriting them would silently erase a choice someone made on purpose. The stack-owned files describe how the tool works. `kb/CONTRACT.md` opens by saying it holds what `tools/wikitool` enforces or what follows mechanically from how it operates - see -[kb/CONTRACT.md](../kb/CONTRACT.md), lines 10-13. That kind of statement doesn't vary by +[kb/CONTRACT.md](../../kb/CONTRACT.md), lines 10-13. That kind of statement doesn't vary by instance: the compiler behaves the same way regardless of who is running it, so the sentence describing that behavior can be copied byte-for-byte into every checkout without becoming wrong anywhere. @@ -33,7 +33,7 @@ wrong anywhere. The instance-owned files describe a choice: which language pages are written in, what tone the agent takes, who the operator is, which git remote is authoritative, which MCP servers are reachable. None of that follows from the tool's mechanics - two instances of the identical -stack can answer all of these differently and both be correct. [AGENTS.md § Personalization](../AGENTS.md#personalization) +stack can answer all of these differently and both be correct. [AGENTS.md § Personalization](../../AGENTS.md#personalization) frames the split the same way for `kb/CONTRACT.md` versus `kb/CONVENTIONS.md`: "the split is by who may change the sentence, not by what it is about." A rule about page structure could in principle have been written per-instance too, but then every instance answering "not German" to @@ -59,7 +59,7 @@ the file itself already answers that. For a while the boundary was written down as a list of paths - once in `dist_cmd.py`, once in the merge procedure a private instance was told to run by hand, and once in the check that -procedure ended with. Three copies of one fact, which is the shape [AGENTS.md](../AGENTS.md) +procedure ended with. Three copies of one fact, which is the shape [AGENTS.md](../../AGENTS.md) invariant 8 exists to forbid, and they drifted exactly as predicted: the hand-run procedure was still naming three paths after the collection contracts had moved to the instance's side of the line, so it discarded upstream changes to files it had never heard of, while its own final check @@ -81,7 +81,7 @@ The mechanism for instance-owned content is a `.template` file the distribution of the real one - `USER.md.template`, `SOUL.md.template`, `kb/CONVENTIONS.md.template`, `ENVIRONMENT.md.template`. An alternative would have been to ship nothing at all and let a brand-new instance start from a blank page. The template exists because a blank page doesn't -tell [instructions/setup-instance.md](../instructions/setup-instance.md) what shape the answer +tell [instructions/setup-instance.md](../../instructions/setup-instance.md) what shape the answer should take, and it gives nothing for a validator to check afterward. A template carries a placeholder value - a sentinel - in the fields that need a real answer. @@ -89,7 +89,7 @@ Setup interviews the operator and replaces the sentinel with what they actually gives `doctor` a mechanical way to tell "personalized" from "not yet": a file that still contains the sentinel hasn't been through setup, regardless of whether the file exists. That's also why `ENVIRONMENT.md` only warrants a WARN rather than a FAIL when absent - see -[AGENTS.md § Environment](../AGENTS.md#environment) - while a missing or unfilled +[AGENTS.md § Environment](../../AGENTS.md#environment) - while a missing or unfilled `USER.md`/`SOUL.md`/`kb/CONVENTIONS.md` is a harder failure: `ENVIRONMENT.md` describes one checkout among possibly several and is gitignored for that reason, so its absence is a normal state rather than a sign setup was skipped. diff --git a/docs/pipeline-rationale.md b/docs/explanation/pipeline-rationale.md similarity index 87% rename from docs/pipeline-rationale.md rename to docs/explanation/pipeline-rationale.md index 9454e5a..5a7a968 100644 --- a/docs/pipeline-rationale.md +++ b/docs/explanation/pipeline-rationale.md @@ -2,13 +2,13 @@ Chemenu could, in principle, be one directory: drop a file in, ask a question, get an answer computed fresh each time. It isn't built that way. The pipeline in -[AGENTS.md](../AGENTS.md#routing) - `raw/` -> `[types/ + tools/]` -> `kb/` -> `reports/`, with +[AGENTS.md](../../AGENTS.md#routing) - `raw/` -> `[types/ + tools/]` -> `kb/` -> `reports/`, with `work/` alongside rather than inside it - separates *material* from *meaning* from *byproduct*, and each seam exists because collapsing it costs something specific. ## Why raw material stays untouched -[raw/CONTRACT.md](../raw/CONTRACT.md) keeps a source exactly as it arrived. The reasoning is +[raw/CONTRACT.md](../../raw/CONTRACT.md) keeps a source exactly as it arrived. The reasoning is simple once stated: the moment someone "cleans up" or reformats a source on the way in, the thing later claims get checked against is no longer the thing that was actually said. An immutable `raw/` means a citation always resolves to the original, not to somebody's tidied @@ -31,19 +31,19 @@ promotion into `raw/` quietly doubling as the review step it cannot perform. ## Why extraction happens once, through a schema -[types/type-spec.md](../types/type-spec.md) is what stands between a raw file and a `kb/` page: +[types/type-spec.md](../../types/type-spec.md) is what stands between a raw file and a `kb/` page: a type-spec defines what a conforming instance of a page looks like, and the compiler (`tools/wikitool`) applies it. The alternative - every query re-reading and re-interpreting the source on demand - would mean paying the cost of understanding the material every single time, and getting a slightly different answer each time depending on how the question was phrased. Extracting once, against a fixed schema, turns "re-read and re-guess" into "look up what was already compiled." That is the "never re-derive, always compile" principle from -[AGENTS.md](../AGENTS.md): understanding a source is expensive and worth doing exactly once, +[AGENTS.md](../../AGENTS.md): understanding a source is expensive and worth doing exactly once, after which it becomes a cheap, stable lookup. ## Why a `kb/` page has to stand on its own -[kb/CONTRACT.md](../kb/CONTRACT.md) sets the bar for the compiled layer: a page should answer a +[kb/CONTRACT.md](../../kb/CONTRACT.md) sets the bar for the compiled layer: a page should answer a future question without sending the reader back to the source it came from. That's the payoff of compiling in the first place - if every answer still bottomed out in "go re-read the raw file," the `kb/` layer would just be a pointer with extra steps, and the cost of extraction @@ -52,7 +52,7 @@ consistent to query: the work of understanding is already sitting there, done. ## Why `reports/` doesn't need to be maintained -[reports/CONTRACT.md](../reports/CONTRACT.md) treats most of what lands in `reports/` - +[reports/CONTRACT.md](../../reports/CONTRACT.md) treats most of what lands in `reports/` - lint output, telemetry traces - as disposable. The structural content of a lint report can be recomputed from the tree at any commit, so keeping an old copy around would just be a second version of something the tool can already answer on demand, and a second copy is exactly the @@ -64,7 +64,7 @@ distinction between what's recomputable and what isn't. ## Where `work/` fits -[work/CONTRACT.md](../work/CONTRACT.md) describes a workshop, not a fifth pipeline stage: a +[work/CONTRACT.md](../../work/CONTRACT.md) describes a workshop, not a fifth pipeline stage: a place for the notes, extracts and open decisions of a task that spans more than one session, on its way toward becoming a `kb/` page. It sits beside the raw -> kb -> reports flow rather than inside it - closer in spirit to a desk than to a conveyor belt. diff --git a/docs/version-model.md b/docs/explanation/version-model.md similarity index 100% rename from docs/version-model.md rename to docs/explanation/version-model.md diff --git a/docs/why-gates-are-code.md b/docs/explanation/why-gates-are-code.md similarity index 98% rename from docs/why-gates-are-code.md rename to docs/explanation/why-gates-are-code.md index ee4e61f..e33dc68 100644 --- a/docs/why-gates-are-code.md +++ b/docs/explanation/why-gates-are-code.md @@ -4,8 +4,8 @@ Chemenu has five hard limits - the Mass-Update Gate, the Publish-Remote Gate, th Gate, the Guideline Push Gate, and the Iteration Budget Gate - and all five live inside `tools/wikitool`, not in a paragraph of instructions an agent reads and follows. The rules themselves, and what to do when -one trips, are in [AGENTS.md § Gates](../AGENTS.md#gates) and -[instructions/gates.md](../instructions/gates.md). This page is only about the design choice +one trips, are in [AGENTS.md § Gates](../../AGENTS.md#gates) and +[instructions/gates.md](../../instructions/gates.md). This page is only about the design choice underneath them: why code, and why these mechanisms in particular. diff --git a/docs/how-to/README.md b/docs/how-to/README.md new file mode 100644 index 0000000..c43ae87 --- /dev/null +++ b/docs/how-to/README.md @@ -0,0 +1,11 @@ +# How-to guides + +Task-oriented pages for the operator of an instance: one goal per page - upgrade an instance, +set up the MCP read server, run a weekly review - with the steps a human takes and the sentence +handed to the agent. A how-to guide never retells an agent's procedure: the steps an agent runs +live in `instructions/`, and a guide here says what to prepare, what to ask for and where the +agent stops. + +Nothing is here yet. Until the guides are written, the task-shaped parts of +[README.md](../../README.md), [INSTALL.md](../../INSTALL.md), +[INSTALL-MCP.md](../../INSTALL-MCP.md) and [EVALS.md](../../EVALS.md) still carry this material. diff --git a/docs/reference/README.md b/docs/reference/README.md new file mode 100644 index 0000000..7619875 --- /dev/null +++ b/docs/reference/README.md @@ -0,0 +1,10 @@ +# Reference + +Lookup pages: the command surface, exit codes and gates, the skill catalogue, the page types. A +reference page describes what is, not how to use it or why it is so - and wherever a source for +it exists in the code, it is generated from that source and checked against it rather than +written by hand. + +Nothing is generated here yet. Until it is, the command reference is +[tools/CONTRACT.md](../../tools/CONTRACT.md) (and `tools/wikitool -h`), and a page +type is described by `tools/wikitool types describe `. diff --git a/docs/tutorials/README.md b/docs/tutorials/README.md new file mode 100644 index 0000000..98cc0b8 --- /dev/null +++ b/docs/tutorials/README.md @@ -0,0 +1,8 @@ +# Tutorials + +Learning-oriented pages: a guided first walk through an instance, start to finish, for someone +who has not used one before. A tutorial shows one path that works; it does not list options or +explain why. + +Nothing is here yet. The installation tutorial is still [INSTALL.md](../../INSTALL.md) at the +repository root, which stays there; a walk from the first source to the first answer is planned. diff --git a/instructions/CONTRACT.md b/instructions/CONTRACT.md index 3c7ac71..814f949 100644 --- a/instructions/CONTRACT.md +++ b/instructions/CONTRACT.md @@ -234,7 +234,7 @@ new skill is written to match the ones around it. Three prefixes exist today, each naming the subject domain a skill operates on, not the distribution boundary it ships behind: `wiki-` for the knowledge pipeline (`wiki-ingest`, `wiki-lint`, `wiki-manage`, `wiki-query`, `wiki-status`), `gtd-` for the commitment layer -(`gtd-weekly-review` - see `kb/gtd/COLLECTION.md` and `docs/knowledge-and-commitment.md` for why +(`gtd-weekly-review` - see `kb/gtd/COLLECTION.md` and `docs/explanation/knowledge-and-commitment.md` for why that layer is named GTD rather than folded into `wiki-`), and `stack-` for the stack's own development, nested under `instructions/dev/` and therefore never present in a distributed instance (`instructions/dev/` above). diff --git a/instructions/dev/dev-setup.md b/instructions/dev/dev-setup.md index d2a3c3a..c00fbc0 100644 --- a/instructions/dev/dev-setup.md +++ b/instructions/dev/dev-setup.md @@ -39,7 +39,7 @@ development material under `instructions/dev/` and `commonplace/`, and no | No `.wikitool-release.json`: telemetry is **on** - the traces are the stack's measuring instrument (`EVALS.md` § "Whether it runs at all") | Telemetry is off until the operator turns it on | | `USER.md`/`SOUL.md` describe a demo operator and persona | Written by the operator during setup | | `tools/wikitool dist upgrade` refuses: there is no stamp to compare against. The checkout follows `main` with `tools/wikitool sync` | Updated with `dist upgrade --latest` | - | `instructions/dev/`, `commonplace/`, `DEVELOPMENT.md` and `.gitea/` are present | Never shipped | + | `instructions/dev/`, `docs/dev/`, `commonplace/` and `.gitea/` are present | Never shipped | 5. **Use `dist export` as a build and test tool.** It writes exactly the tree a release ships, so it is how a change to the shipped surface is looked at before it is released: @@ -68,6 +68,6 @@ development material under `instructions/dev/` and `commonplace/`, and no ## Scope -Not for operating an instance, and not for the release workflow itself (`DEVELOPMENT.md` for +Not for operating an instance, and not for the release workflow itself (`docs/dev/development.md` for humans, [version-parts.md](version-parts.md) for the version part). Not shipped: `dist export` prunes `instructions/dev/` wholesale. diff --git a/instructions/dev/doc-pull-through.md b/instructions/dev/doc-pull-through.md index 81dd8eb..d567ca9 100644 --- a/instructions/dev/doc-pull-through.md +++ b/instructions/dev/doc-pull-through.md @@ -36,21 +36,21 @@ touched; a row that does not apply needs no action. | A `wikitool` command's behaviour, flags, or interface | Its `cli_contract.CommandRecord` (name, synopsis, properties, exit status, retry policy - `tools/chemenu/cli_contract.py`), then `wikitool docs contract --apply` to regenerate its copy in [tools/CONTRACT.md](../../tools/CONTRACT.md) | | A stage's authoring rules (`raw/`, `kb/`, `types/`, `reports/`, `work/`, `tools/`, `instructions/`) | The touched `/CONTRACT.md` | | A rule, gate, or invariant `AGENTS.md` itself states | The relevant `AGENTS.md` section (Invariants, Gates, File naming, Routing, ...) | - | A workflow, stage, or command a human operates by hand | Whichever of `README.md`, `EVALS.md`, `tools/README.md`, `INSTALL.md`, `DEVELOPMENT.md` names it - AGENTS.md § File naming says which document is for which reader | - | The reasoning behind a gate, boundary, or design decision | The `docs/` page that carries it, if one exists (AGENTS.md § File naming lists all six reached from AGENTS.md itself, plus a seventh reached only from CLAUDE.md). **A decision with no page yet is the gap worth closing**: reasoning that lives only in a Gitea issue never ships - `dist export` carries `docs/` and no issue tracker, so a distributed instance gets the mechanism without the why | + | A workflow, stage, or command a human operates by hand | Whichever of `README.md`, `EVALS.md`, `tools/README.md`, `INSTALL.md`, `INSTALL-MCP.md` or the pages under `docs/tutorials/`, `docs/how-to/` and `docs/reference/` names it - AGENTS.md § File naming says which document is for which reader. A new page goes into the area its kind of documentation belongs to; that area's `README.md` says which | + | The reasoning behind a gate, boundary, or design decision | The `docs/explanation/` page that carries it, if one exists (AGENTS.md § File naming lists all six reached from AGENTS.md itself, plus a seventh reached only from CLAUDE.md). **A decision with no page yet is the gap worth closing**: reasoning that lives only in a Gitea issue never ships - `dist export` carries `docs/explanation/` and no issue tracker, so a distributed instance gets the mechanism without the why | | A task-tracker adapter (`tools/chemenu/tasks/`), its recorded fixtures, or the live suite | [instructions/dev/tracker-testing.md](tracker-testing.md), and `MANIFEST.json` beside the fixtures when they were re-recorded | | A skill's own step sequence or catalogue | The skill's `SKILL.md` source under `instructions//` or `instructions/dev//` | | A per-checkout configuration file an instance owns (`.wikitool-tasks.json`, `.wikitool-telemetry.json`, `.wikitool-remotes.json`, `.wikitool-upload.json`) | [INSTALL.md](../../INSTALL.md) § Konfiguration, where an operator looks the shape up; the [setup-instance.md](../setup-instance.md) decision point that offers it during setup; and `doctor`'s own row in [tools/CONTRACT.md](../../tools/CONTRACT.md), since `doctor` is what reports the file's state | | An installation instruction - [preflight.md](../preflight.md), [setup-instance.md](../setup-instance.md), [bootstrap.md](../bootstrap.md), [upgrade-instance.md](../upgrade-instance.md) - or `tools/prerequisites.txt` | [INSTALL.md](../../INSTALL.md), the human guide to the same procedure. Read it against the instruction: what to prepare, the sentence for the agent, what the agent asks, where it stops and why. `docs verify` checks only the two enumerable overlaps - the prerequisites lists, which `wikitool docs prerequisites --apply` regenerates from the manifest, and the setup questions: a question the agent asks the user carries `` where it is asked in `setup-instance.md`, and `INSTALL.md` § "Was der Agent dich fragt" names it with the same marker. Every other sentence is this session's to compare. `INSTALL.md` does not retell the steps, so a change to their order or wording alone moves nothing there | - | [dev-setup.md](dev-setup.md) | [DEVELOPMENT.md](../../DEVELOPMENT.md), read against it the same way - nothing checks this pair at all | + | [dev-setup.md](dev-setup.md), the release workflow (`version bump`/`version release`/`publish`/CI), or another developer-facing process | [docs/dev/development.md](../../docs/dev/development.md), read against it the same way - nothing checks this pair at all. A page under `docs/dev/` is never shipped, so it may link into `instructions/dev/` and cite issue numbers | | A new page type the stack requires, or a new collection | Its type-spec and `COLLECTION.md` (both as the `.template` an instance adopts), the collection table in [kb/CONTRACT.md](../../kb/CONTRACT.md), and **both adoption paths**: [setup-instance.md](../setup-instance.md) for a fresh instance and [upgrade-instance.md](../upgrade-instance.md) for an existing one, where an unadopted template is what `docs verify` refuses | 3. **A heading you changed means a table of contents to regenerate - by the tool, never by hand.** Every reference file over 100 lines carries one (`AGENTS.md`, the stage contracts, `kb/CONVENTIONS.md`, each `COLLECTION.md`, the flat `instructions/**.md` form, the - type-specs, the `docs/` pages - each with the `.template` it ships as, where one - exists, and a `SKILL.md` the one exception). Adding, renaming, - reordering or deleting a `##`/`###` heading in one of them makes its region stale, and + type-specs, the `docs/` pages in all five areas - each with the `.template` it ships + as, where one exists, and a `SKILL.md` the one exception). Adding, renaming, reordering or + deleting a `##`/`###` heading in one of them makes its region stale, and `docs verify` fails on stale exactly as it fails on missing: ```bash @@ -73,7 +73,7 @@ touched; a row that does not apply needs no action. - **The change touched no document in the table?** Nothing to do - not every stack change moves a claim. A pure bugfix with an unchanged interface is the common case. -- **Unsure whether a `docs/` page's reasoning moved?** Read it. A `docs/` page carries no +- **Unsure whether a `docs/explanation/` page's reasoning moved?** Read it. A `docs/` page carries no normative sentence and nothing verifies it by construction (AGENTS.md § File naming), so an unsure guess defaults to reading the page rather than skipping the question - [`stack-close`](stack-close/SKILL.md) step 3 asks it again in the closing phase as a diff --git a/instructions/dev/stack-close/SKILL.md b/instructions/dev/stack-close/SKILL.md index 9defb6c..aba5e7a 100644 --- a/instructions/dev/stack-close/SKILL.md +++ b/instructions/dev/stack-close/SKILL.md @@ -23,7 +23,7 @@ phases this one ends; read it first when this session started cold. The phases around the checked middle of a work package have no mechanical guard at all - `pytest`, `docs verify`, `instructions verify` and CI cover the code and tests in between, and nothing covers a changelog entry's accuracy, a `docs/` page's staleness, or an issue body's final -state (see `docs/model-and-effort-selection.md`). Asking the same session to notice it has +state (see `docs/explanation/model-and-effort-selection.md`). Asking the same session to notice it has crossed into that unchecked stretch - as a prose break inside one long skill - failed twice in a row on this stack (Gitea #42, then #30): both times the session knew the rule and skipped past it anyway, because nothing in the moment forced the question. The first split (Gitea #47) moved @@ -71,7 +71,7 @@ always will. naming) - the same is true of `tools/CONTRACT.md`'s generated command records and any touched `/CONTRACT.md`, whose prose `docs verify` checks only for presence and record membership, never for what a field or a section actually says - (`instructions/dev/doc-pull-through.md`); of `README.md`/`INSTALL.md`/`DEVELOPMENT.md` + (`instructions/dev/doc-pull-through.md`); of `README.md`/`INSTALL.md`/`docs/` prose; and of a new instruction's own wording, which `instructions verify` checks structurally but never for what it claims. If the change this package shipped moved the reasoning or the behaviour one of these documents describes, update it now; if none did, say so rather than diff --git a/instructions/dev/stack-mode.md b/instructions/dev/stack-mode.md index 0b7640c..35d873a 100644 --- a/instructions/dev/stack-mode.md +++ b/instructions/dev/stack-mode.md @@ -118,4 +118,4 @@ next phase's slash command in a session started on the right model - and the tra what carries the work across it. Which model suits which phase, and the reasoning, is -`docs/model-and-effort-selection.md`. +`docs/explanation/model-and-effort-selection.md`. diff --git a/instructions/gtd-weekly-review/SKILL.md b/instructions/gtd-weekly-review/SKILL.md index 3baa1f0..85b05ef 100644 --- a/instructions/gtd-weekly-review/SKILL.md +++ b/instructions/gtd-weekly-review/SKILL.md @@ -30,7 +30,7 @@ item outright - stays the user's own action in their tracker: that is a delibera in the command surface waiting to be filled. Do not reach for a tracker-specific tool or API to "just do it faster" for either half. The reasoning behind keeping the tracker and `kb/gtd/` on separate write paths, and behind stopping at "create" and "mark done" rather than a fuller CRUD -surface, lives in `docs/knowledge-and-commitment.md`, which this skill does not repeat. +surface, lives in `docs/explanation/knowledge-and-commitment.md`, which this skill does not repeat. ## Steps diff --git a/instructions/ingest-queue.md b/instructions/ingest-queue.md index 20ea431..8bcadab 100644 --- a/instructions/ingest-queue.md +++ b/instructions/ingest-queue.md @@ -38,7 +38,7 @@ this when a submission is already waiting and a decision is due. prints filename, size, sha256, submitter, and - as important as the submitter's name - `submitter_source`: the *header* the value came from, naming where the claim rests rather than asserting it as fact. Read - [docs/why-gates-are-code.md](../docs/why-gates-are-code.md) once for why + [docs/explanation/why-gates-are-code.md](../docs/explanation/why-gates-are-code.md) once for why this gate exists as code rather than as this paragraph alone. 2. **Read the file itself before promoting anything.** The quarantine holds diff --git a/instructions/migrations/6.0.0-type-guidance-split.md b/instructions/migrations/6.0.0-type-guidance-split.md index 2c7bb75..45f5500 100644 --- a/instructions/migrations/6.0.0-type-guidance-split.md +++ b/instructions/migrations/6.0.0-type-guidance-split.md @@ -15,7 +15,7 @@ mechanism-level advice such as citation and provenance rules - in the same file configuration and its `## Template` block. Adopting the type-spec at setup meant adopting all of it at once, and an upgrade never touched the adopted file again: the prose an instance received was frozen at the day it ran `setup-instance.md`, while every later improvement shipped only in -the `.template` beside it (`docs/ownership-and-templates.md` § "Where the file boundary used to +the `.template` beside it (`docs/explanation/ownership-and-templates.md` § "Where the file boundary used to strain"). 6.0.0 splits that prose into a separate, stack-owned `types/.guidance.md`, linked from the diff --git a/instructions/migrations/9.0.0-docs-diataxis.md b/instructions/migrations/9.0.0-docs-diataxis.md new file mode 100644 index 0000000..5cfb316 --- /dev/null +++ b/instructions/migrations/9.0.0-docs-diataxis.md @@ -0,0 +1,114 @@ +--- +type: types/instruction.md +name: 9.0.0-docs-diataxis +description: Point every file this instance owns at the explanation pages' new home under docs/explanation/ - a path rewrite with no judgment call, without which docs verify reports a dead link in kb/CONVENTIONS.md. +manual: true +migrates_to: 9.0.0 +migration_kind: mechanical +obligation: required +--- +# Point own files at `docs/explanation/` (9.0.0) + +9.0.0 sorts the stack's human documentation under `docs/` into areas - `docs/tutorials/`, +`docs/how-to/`, `docs/reference/` and `docs/explanation/` - and moves the seven pages that used +to sit directly in `docs/` into `docs/explanation/`, names unchanged. Nothing is left at the old +paths: `dist upgrade` deletes them as files the release no longer ships. + +Every file the stack ships already points at the new paths. A file **this instance owns** does +not, because an upgrade never touches it - and one such file is in every instance: +`kb/CONVENTIONS.md`, adopted from a template that linked `language-boundaries.md` at its old +place directly under `docs/`. +`tools/wikitool docs verify` checks every relative link in it, so after the upgrade it reports +that link as not resolving. This migration rewrites the old paths, and nothing else. + +It is `mechanical`: the seven names map one to one, and there is no per-file judgment beyond +which files are in scope (§ Decision points). + + +## Contents + +- [When to run](#when-to-run) +- [Steps](#steps) +- [How to tell a migrated file from an unmigrated one](#how-to-tell-a-migrated-file-from-an-unmigrated-one) +- [Decision points](#decision-points) +- [Scope](#scope) + + +## When to run + +After installing 9.0.0 machinery over an instance on 8.x - step 12 of +[upgrade-instance.md](../upgrade-instance.md), once `tools/wikitool migrate status` names this +document. Until it has run, `docs verify` reports the dead link in `kb/CONVENTIONS.md`; that +failure is expected between the machinery swap and this migration, and nothing else depends on +it. + +## Steps + +1. **Capture the baseline**, before changing anything - the list of every file this instance + tracks that names an old path, and what `docs verify` says about it now: + + ```bash + git grep -nE 'docs/(pipeline-rationale|ownership-and-templates|language-boundaries|why-gates-are-code|version-model|knowledge-and-commitment|model-and-effort-selection)\.md' -- . ':!raw' ':!CHANGES.md' ':!kb/log.md' > /tmp/9.0.0-before.txt + tools/wikitool docs verify > /tmp/9.0.0-verify-before.txt + ``` + + Expect at least `kb/CONVENTIONS.md` in the first file and its dead link in the second. + +2. **Decide which hits are in scope.** Every hit is, except a `kb/` page (a file under a + collection with a `type:` in its frontmatter) - see § Decision points. In a typical instance + the in-scope list is `kb/CONVENTIONS.md` alone; an instruction, a type-spec or a + `COLLECTION.md` the instance wrote for itself may add to it. + +3. **Rewrite the old paths in each in-scope file:** + + ```bash + sed -i -E 's#docs/(pipeline-rationale|ownership-and-templates|language-boundaries|why-gates-are-code|version-model|knowledge-and-commitment|model-and-effort-selection)\.md#docs/explanation/\1.md#g' ... + ``` + + The rewrite keeps everything before `docs/` - a link's `../` stays as it was, because only + the part below `docs/` moved. No other line of the file changes. + +4. **Check it against the baseline:** + + ```bash + git diff --stat + git grep -nE 'docs/(pipeline-rationale|ownership-and-templates|language-boundaries|why-gates-are-code|version-model|knowledge-and-commitment|model-and-effort-selection)\.md' -- . ':!raw' ':!CHANGES.md' ':!kb/log.md' + tools/wikitool docs verify + ``` + + `git diff --stat` names exactly the files step 2 put in scope, one or a few lines each. The + `git grep` prints only `kb/` pages step 2 left out - none at all in a typical instance. `docs + verify` is green; compared with `/tmp/9.0.0-verify-before.txt`, the dead-link lines are what + disappeared. + +5. **Record it:** + + ```bash + tools/wikitool migrate done 9.0.0 --pages 0 + ``` + + `--pages 0` because no `kb/` page changes. The migration is `required`, so this advances + `kb_version`. + +## How to tell a migrated file from an unmigrated one + +An unmigrated file names `docs/.md`; a migrated one names +`docs/explanation/.md`. Step 4's `git grep` lists every file still waiting, and `docs verify` +names each unmigrated reference file by line. + +## Decision points + +- **A `kb/` page mentions an old path?** Leave it out of this migration: a page records + what a source said at the time, `docs verify` does not read page bodies, and a page is changed + through `wiki-manage`, not by `sed`. Update it the next time the page is worked on, if the + mention is meant as a pointer rather than as evidence. +- **The instance links one of the pages under a different spelling** - an absolute URL into + the origin repository, say? Out of scope: such a link never resolved inside the instance, and + `docs verify` does not check it. + +## Scope + +Files this instance tracks that name one of the seven old paths, outside `raw/` (immutable), +`CHANGES.md` and `kb/log.md` (records of what was true at the time) and `kb/` pages. The +machinery itself - the moved pages, the new area READMEs, every shipped file that links them - +arrived with `dist upgrade` and needs nothing from this document. diff --git a/instructions/upgrade-instance.md b/instructions/upgrade-instance.md index 1c4fb8d..eef4596 100644 --- a/instructions/upgrade-instance.md +++ b/instructions/upgrade-instance.md @@ -209,7 +209,10 @@ a further checkout of this one ([bootstrap.md](bootstrap.md)). and filled is never touched. The `.template` files stay where they are - they are the source for the next upgrade's comparison. Any other failure is read against step 2's **Breaking Change:** line: if the release predicted it, the notes also say what fixes it; if it did not, stop and report it - rather than improvising. + rather than improvising. Where the fix they name is a required migration document - a + machinery-side file this instance owns, such as `kb/CONVENTIONS.md`, still pointing at a + path the release moved - leave the failure standing here: step 12 runs that document, and + step 13's `docs verify` is where it has to be gone. 10. **Publish the machinery swap.** A release swap is far above the Mass-Update Gate's threshold, so expect exit 42. That is not an error and not yours to clear: reproduce the file breakdown @@ -254,6 +257,7 @@ a further checkout of this one ([bootstrap.md](bootstrap.md)). ```bash tools/wikitool migrate verify --from tools/wikitool lint + tools/wikitool docs verify ``` `migrate verify` is the only check that sees a page which lost a citation, a wikilink or a diff --git a/kb/CONVENTIONS.md b/kb/CONVENTIONS.md index 997ad31..cb312f9 100644 --- a/kb/CONVENTIONS.md +++ b/kb/CONVENTIONS.md @@ -59,7 +59,7 @@ The first of those is not a setting this file withholds - it is not a setting at above is the only language value in the tree, and what it binds is page text; a control-plane document is English even when this instance wrote it for itself and never ships it. Why that is an architecture decision rather than an unset parameter: -[docs/language-boundaries.md](../docs/language-boundaries.md). +[docs/explanation/language-boundaries.md](../docs/explanation/language-boundaries.md). Which line is prose and which is an identifier - and therefore what is translated at all - is the contract's rule, not this file's: see diff --git a/kb/CONVENTIONS.md.template b/kb/CONVENTIONS.md.template index 4d3eb3b..d7ce6d5 100644 --- a/kb/CONVENTIONS.md.template +++ b/kb/CONVENTIONS.md.template @@ -59,7 +59,7 @@ The first of those is not a setting this file withholds - it is not a setting at above is the only language value in the tree, and what it binds is page text; a control-plane document is English even when this instance wrote it for itself and never ships it. Why that is an architecture decision rather than an unset parameter: -[docs/language-boundaries.md](../docs/language-boundaries.md). +[docs/explanation/language-boundaries.md](../docs/explanation/language-boundaries.md). Which line is prose and which is an identifier - and therefore what is translated at all - is the contract's rule, not this file's: see diff --git a/tools/CONTRACT.md b/tools/CONTRACT.md index 455c1fc..f12825f 100644 --- a/tools/CONTRACT.md +++ b/tools/CONTRACT.md @@ -241,7 +241,7 @@ Scaffold a new wiki page of any type. - `wikitool types describe ` - what a type requires and where it lands - `wikitool touch` - changes a page's own frontmatter afterwards - `wikitool task new` - a tracker item without a page -- `docs/knowledge-and-commitment.md` - why a project is a page and a tracker project +- `docs/explanation/knowledge-and-commitment.md` - why a project is a page and a tracker project #### `task new` @@ -307,7 +307,7 @@ Create one open item in the configured task tracker - never a kb/ page. - `wikitool task list` - a project's open items and their ids - `wikitool task close` - marks an item done - `wikitool new project` - a project page and its tracker project -- `docs/knowledge-and-commitment.md` - why commitments live in the tracker, not in `kb/` +- `docs/explanation/knowledge-and-commitment.md` - why commitments live in the tracker, not in `kb/` #### `task list` @@ -2511,6 +2511,7 @@ Check the docs that mirror the code. - 1 A subtype template `types/..md` has no type-spec with a `subtype_field:` beside it, names a value outside that field's enum, or carries frontmatter - 1 A shipped `.md`/`.template` cites an issue number - 1 A reference file's table-of-contents region is missing or stale +- 1 A file sits directly in `docs/`, or in a directory there that is not one of its five areas - 1 A reference file's relative markdown link does not resolve to an existing file - 1 An `INSTALL.md` prerequisites region is stale - 1 An `INSTALL.md` prerequisites region is missing, or names a platform no tool has @@ -2526,6 +2527,7 @@ Check the docs that mirror the code. - A subtype template `types/..md` has no type-spec with a `subtype_field:` beside it, names a value outside that field's enum, or carries frontmatter -> Rename the file to the type and value it was meant for, delete it, or remove its frontmatter block - A shipped `.md`/`.template` cites an issue number -> Say what was decided instead of pointing at where, or move the pointer behind a `` block - A reference file's table-of-contents region is missing or stale -> Run `docs toc --apply`, then re-run +- A file sits directly in `docs/`, or in a directory there that is not one of its five areas -> Move it (`git mv`) into the area its kind of documentation belongs to, and fix the links that pointed at the old path - A reference file's relative markdown link does not resolve to an existing file -> Fix the `../` count or the target's name - An `INSTALL.md` prerequisites region is stale -> Run `docs prerequisites --apply`, then re-run - An `INSTALL.md` prerequisites region is missing, or names a platform no tool has -> Add the marker pair where that list belongs (or remove the orphaned region and its introducing prose), then run `docs prerequisites --apply` @@ -2537,7 +2539,7 @@ Check the docs that mirror the code. **NOTES** -- Checks the docs that mirror the code. The name is about documentation parity, not the `docs/` directory - it neither reads nor requires one. +- Checks the docs that mirror the code. The name is about documentation parity, not the `docs/` directory - it reads that only for its layout, and never requires one. - Commands: every command has a `cli_contract` record and is listed in `cli_contract.GROUPS`, in both directions; every command's non-hidden flags appear in its record's SYNOPSIS and vice versa; `tools/CONTRACT.md`'s generated `` region matches what `docs contract` would write; no command's rendered `--help`/`-h` text cites an issue number. - Collections: every directory under `kb/` has a `COLLECTION.md` and no directory outside it does; every collection declares `profile:` and a `required_by_stack:` that agrees with the stack's own list. - Types: every type the stack lists (currently `source` and `project`) has a type-spec of that name whose schema requires the field the stack list names (`raw_files:`/`state:`); every file under `types/` declaring `type: types/type-spec.md` validates against `types/type-spec.schema.yaml`; every subtype template `types/..md` (any `` but `guidance`) sits beside a type-spec declaring `subtype_field:`, names a value that field's schema enum allows, and carries no frontmatter; no pre-migration `type: entity` blocks are left in the contracts. @@ -2546,6 +2548,7 @@ Check the docs that mirror the code. - No `.md`/`.template` file `dist export` would ship cites an issue number. A `` region is exempt: the check reads the export plan's text, from which it is already gone. - Every reference file `docs toc` covers carries the current table-of-contents region for its own headings - missing and stale are one check. - Every relative markdown link in one of those reference files resolves to an existing file. A target's `#anchor` suffix is stripped first, and code fences and inline code spans are masked before scanning, so link syntax shown as an example is not mistaken for a real reference. +- Every file under `docs/` sits in one of its areas - `tutorials/`, `how-to/`, `reference/`, `explanation/` or `dev/` - never directly in `docs/` or in another directory. - `INSTALL.md` carries one generated `` region per platform value of `tools/prerequisites.txt` (`prerequisites-` for a platform-specific one), each current; and the `` markers in `instructions/setup-instance.md` and `INSTALL.md` name the same set of keys, so a question the agent asks is never one the human guide leaves out, nor the reverse. - `VERSION` parses and the newest versioned `CHANGES.md` entry names it. An entry that crosses the compatibility boundary from the last release carries `**Breaking Change:**`, has a migration document or `**Migration:** none required`, and its `**Migration:**` line names every required migration document targeting it. - The newest versioned `CHANGES.md` entry is at most 32 000 bytes (UTF-8) as `version notes` prints it, and each of its `###` sections is exactly one paragraph of at most 1 000 bytes, heading line excluded. Older entries and a changelog with no versioned entry are not checked. @@ -2818,7 +2821,7 @@ Write a contentless, distributable copy of this repo's machinery. **NOTES** - Writes a contentless, distributable copy of this repo's machinery into an empty or new `` directory. -- Ships `AGENTS.md`/`README.md`/`EVALS.md` with any `dist:strip-start`...`dist:strip-end` marker region removed, `instructions/` (minus `instructions/dev/`), `types/` (the `root: kb` page type-specs, their schemas and their subtype templates `types/..md` re-keyed as `.template`, the stack's own verbatim), `docs/` verbatim, `tools/` (no venv/caches, and not the dev-only test suite `tools/chemenu/tests/` with its `pytest.ini`/`.coveragerc`), the `.github/hooks/`+`.vibe/` session-tracing config plus `.claude/settings.json`, `kb/CONTRACT.md` (no pages, no areas) and `VERSION`. +- Ships `AGENTS.md`/`README.md`/`EVALS.md` with any `dist:strip-start`...`dist:strip-end` marker region removed, `instructions/` (minus `instructions/dev/`), `types/` (the `root: kb` page type-specs, their schemas and their subtype templates `types/..md` re-keyed as `.template`, the stack's own verbatim), `docs/` verbatim (minus `docs/dev/`), `tools/` (no venv/caches, and not the dev-only test suite `tools/chemenu/tests/` with its `pytest.ini`/`.coveragerc`), the `.github/hooks/`+`.vibe/` session-tracing config plus `.claude/settings.json`, `kb/CONTRACT.md` (no pages, no areas) and `VERSION`. - Ships `raw/` and `incoming/` as flat roots, each with a `.gitkeep` and no subdirectories. `incoming/.gitkeep` is trackable and survives becoming a git repository, so a plain clone gets the directory without any bootstrap step. - Ships templates, never the filled files: `USER.md.template`/`SOUL.md.template`, `kb/CONVENTIONS.md.template`, and each collection's contract re-keyed as `kb//COLLECTION.md.template`. The filled `USER.md`/`SOUL.md`/`kb/CONVENTIONS.md`/`kb//COLLECTION.md`/`types/.md`/`types/..md` bind their instance; `find_leaks` refuses a plan carrying one. - Writes a generated `.wikitool-release.json` stamp: version, export date, origin, and a sha256 per exported file - the base a later upgrade compares against. diff --git a/tools/chemenu/commands/dist_cmd.py b/tools/chemenu/commands/dist_cmd.py index ae0942b..d778994 100644 --- a/tools/chemenu/commands/dist_cmd.py +++ b/tools/chemenu/commands/dist_cmd.py @@ -93,14 +93,10 @@ DIST_TEMPLATES_DIR = Path(__file__).resolve().parent.parent / "dist_templates" # optional. A distribution whose server is present but undocumented is one # whose operator finds the module by reading the source. # -# `DEVELOPMENT.md` is deliberately **absent** from this tuple, unlike every -# other root doc above. It documents the release workflow (`version bump` -> -# `version release` -> `publish` -> CI tags) and points at `instructions/dev/`, -# which this same function excludes wholesale a few lines down - a distributed -# instance has no release workflow, no CI and no issue board, so it has -# nothing for that document to describe. Do not "fix" this by adding it back: -# a root file absent from ROOT_FILES is silently skipped by every export, and -# that silence is the correct behaviour here, not a gap. +# The developer documentation (the release workflow, the development +# environment) is not a root file at all: it lives under `docs/dev/`, which +# `build_plan` excludes the way it excludes `instructions/dev/` - see +# DOCS_DEV_ONLY. ROOT_FILES = ( "AGENTS.md", "CLAUDE.md", "README.md", "EVALS.md", "INSTALL.md", "INSTALL-MCP.md", ".gitignore", ".gitattributes", "VERSION", @@ -156,6 +152,15 @@ def _is_coverage_output(filename: str) -> bool: # reconstructs it afterwards, unlike the marker-block content below. INSTRUCTIONS_EXCLUDE_DIRS = {"dev"} +# docs/dev/ is the human counterpart of instructions/dev/: the release workflow +# (`version bump` -> `version release` -> `publish` -> CI tags) and the +# development environment, linking into instructions/dev/ and citing issue +# numbers. A distributed instance has no release workflow, no CI and no issue +# board, so it has nothing for those pages to describe. Excluded by exact path, +# not by name: only the top-level `dev` area is developer material, and a +# directory of that name deeper in another area would be ordinary documentation. +DOCS_DEV_ONLY = frozenset({"dev"}) + # Stage contracts that are not collections and carry no pages: copied as a # single file each, nothing else from their directory. `kb/` is excluded here # - it is a content stage too, but it has collections underneath it, so its @@ -341,7 +346,7 @@ def instance_owned_type_stems() -> set[str]: authoring guidance around them addresses an agent and stays English like the rest of the control plane (AGENTS.md § File naming, types/type-spec.md § Who owns a type-spec). That the two halves share one file, and what it - costs, is docs/ownership-and-templates.md § Where the file boundary strains. + costs, is docs/explanation/ownership-and-templates.md § Where the file boundary strains. Read from `types/` rather than listed, so an instance adding its own page type gets the same treatment without a code change. @@ -462,11 +467,12 @@ def build_plan(origin: Optional[Origin] = None) -> dict[str, PlannedFile]: for hook_dir in HOOK_DIRS: plan.update(_copy_tree(config.ROOT / hook_dir, hook_dir, frozenset())) - # docs/ is stack background - why the stack is built the way it is - and - # ships verbatim like instructions/ and types/: it carries no page, no - # frontmatter, and (AGENTS.md § File naming) no normative sentence, so - # there is nothing instance-owned in it to split off as a .template. - plan.update(_copy_tree(config.ROOT / "docs", "docs", frozenset())) + # docs/ is the stack's human documentation - tutorials, how-to, reference + # and explanation - and ships verbatim like instructions/ and types/: it + # carries no page, no frontmatter, and (AGENTS.md § File naming) no + # normative sentence, so there is nothing instance-owned in it to split off + # as a .template. Its developer area is the one exception, see DOCS_DEV_ONLY. + plan.update(_copy_tree(config.ROOT / "docs", "docs", frozenset(), exclude_paths=DOCS_DEV_ONLY)) # `kb/CONTRACT.md` is stack-owned and ships verbatim; everything beside it # under `kb/` is the instance's own and ships only as a `.template`. That is @@ -629,9 +635,9 @@ def _write_plan(target: Path, plan: dict[str, PlannedFile]) -> None: "Ships `AGENTS.md`/`README.md`/`EVALS.md` with any `dist:strip-start`...`dist:strip-end` " "marker region removed, `instructions/` (minus `instructions/dev/`), `types/` (the " "`root: kb` page type-specs, their schemas and their subtype templates " - "`types/..md` re-keyed as `.template`, the stack's own verbatim), `docs/` verbatim, `tools/` (no " - "venv/caches, and not the dev-only test suite `tools/chemenu/tests/` with its " - "`pytest.ini`/`.coveragerc`), the " + "`types/..md` re-keyed as `.template`, the stack's own verbatim), `docs/` " + "verbatim (minus `docs/dev/`), `tools/` (no venv/caches, and not the dev-only test " + "suite `tools/chemenu/tests/` with its `pytest.ini`/`.coveragerc`), the " "`.github/hooks/`+`.vibe/` session-tracing config plus `.claude/settings.json`, " "`kb/CONTRACT.md` (no pages, no areas) and `VERSION`.", "Ships `raw/` and `incoming/` as flat roots, each with a `.gitkeep` and no " @@ -709,7 +715,7 @@ def export_command( ): """Export a contentless, distributable copy of this repo's machinery: AGENTS.md/README.md (dev-instance-only marker blocks removed), - instructions/ (no instructions/dev/), types/, docs/ verbatim, + instructions/ (no instructions/dev/), types/, docs/ verbatim (no docs/dev/), tools/ (no venv/caches), the .github/hooks/+.vibe session-tracing config plus .claude/settings.json, kb/CONTRACT.md plus a COLLECTION.md.template per collection and diff --git a/tools/chemenu/commands/docs_verify.py b/tools/chemenu/commands/docs_verify.py index 44fdadf..dba6081 100644 --- a/tools/chemenu/commands/docs_verify.py +++ b/tools/chemenu/commands/docs_verify.py @@ -56,6 +56,12 @@ contract nothing enforced - the exact "checked or absent" failure this file's opening paragraph names, just one level up, for the schema that describes the type layer instead of a copy the type layer's code produces (Gitea #105). +A ninth checks a layout rather than a copy or a reference: every file under +`docs/` sits in exactly one of its five areas - the four kinds of +documentation, plus the unshipped `docs/dev/`. A page dropped straight into +`docs/` belongs to no kind, and its reader cannot tell from the path what it +is for. + Everything here is a hard oracle: a set comparison or a regex, no judgment. Content quality of the contracts themselves stays with the LLM. """ @@ -202,16 +208,17 @@ ROOT_README = config.ROOT / "README.md" # same reason: it is human-facing prose about installing an instance, and the # command reference lives exactly once, in tools/CONTRACT.md. # -# DEVELOPMENT.md joined them after it drifted the same way (Gitea #47): it grew -# a table describing what each verify command checks, which had to be removed by -# hand because nothing compared it to anything. It is not shipped - dist_cmd -# .ROOT_FILES excludes it - and that is not an argument against listing it here: +# The developer guide joined them after it drifted the same way (Gitea #47, as +# the root `DEVELOPMENT.md` it was then): it grew a table describing what each +# verify command checks, which had to be removed by hand because nothing +# compared it to anything. It is not shipped - `dist export` excludes +# `docs/dev/` - and that is not an argument against listing it here: # `check_readmes_have_no_command_table` skips a file that does not exist, so in # a distributed instance this entry is simply inert, while in the dev checkout # (the only place the file exists, and the only place it can drift) it is # checked. The name is now narrower than the tuple - these are the human-facing # prose docs that must not re-list commands, stage README or not. -STAGE_READMES = ("tools/README.md", "INSTALL.md", "DEVELOPMENT.md") +STAGE_READMES = ("tools/README.md", "INSTALL.md", "docs/dev/development.md") # Docs that must not re-introduce the pre-migration bare-enum `type:` form. # The per-collection contracts are appended at call time, since which ones exist @@ -686,6 +693,40 @@ def check_legacy_type_blocks() -> list[str]: return issues +# The areas of `docs/`: the four kinds of documentation every instance ships, +# and the developer area `dist export` leaves out (dist_cmd.DOCS_DEV_ONLY). +# AGENTS.md § File naming defines them; this is the mechanical half of it. +DOCS_AREAS = ("tutorials", "how-to", "reference", "explanation", "dev") + + +def check_docs_layout() -> list[str]: + """Every file under `docs/` sits in exactly one of `DOCS_AREAS`. + + A file directly in `docs/`, or in a directory that is not an area, is + reported. Area membership is read off the first path segment, so a file + can be in at most one area by construction; what this catches is a file + in none. A tree without `docs/` passes - the check says where a page + goes, not that any page must exist. + """ + docs_dir = config.ROOT / "docs" + if not docs_dir.is_dir(): + return [] + issues = [] + for path in sorted(p for p in docs_dir.rglob("*") if p.is_file()): + relative = path.relative_to(docs_dir) + if len(relative.parts) == 1: + issues.append( + f"{rel_path(path)} sits directly in docs/ - move it into one of " + f"{', '.join(f'docs/{area}/' for area in DOCS_AREAS)}" + ) + elif relative.parts[0] not in DOCS_AREAS: + issues.append( + f"{rel_path(path)} is in docs/{relative.parts[0]}/, which is not a documentation " + f"area - move it into one of {', '.join(f'docs/{area}/' for area in DOCS_AREAS)}" + ) + return issues + + def check_toc_regions() -> list[str]: """Every reference file over the line threshold carries a current TOC. @@ -1167,7 +1208,7 @@ def check_breaking_change_for_boundary() -> list[str]: ), notes=( "Checks the docs that mirror the code. The name is about documentation parity, not the " - "`docs/` directory - it neither reads nor requires one.", + "`docs/` directory - it reads that only for its layout, and never requires one.", "Commands: every command has a `cli_contract` record and is listed in " "`cli_contract.GROUPS`, in both directions; every command's non-hidden flags appear in " "its record's SYNOPSIS and vice versa; `tools/CONTRACT.md`'s generated " @@ -1198,6 +1239,9 @@ def check_breaking_change_for_boundary() -> list[str]: "file. A target's `#anchor` suffix is stripped first, and code fences and inline code " "spans are masked before scanning, so link syntax shown as an example is not mistaken " "for a real reference.", + "Every file under `docs/` sits in one of its areas - `tutorials/`, `how-to/`, " + "`reference/`, `explanation/` or `dev/` - never directly in `docs/` or in another " + "directory.", "`INSTALL.md` carries one generated `` region per " "platform value of `tools/prerequisites.txt` (`prerequisites-` for a " "platform-specific one), each current; and the `` markers " @@ -1254,6 +1298,12 @@ def check_breaking_change_for_boundary() -> list[str]: cause="A reference file's table-of-contents region is missing or stale", reaction="Run `docs toc --apply`, then re-run", ), + cli_contract.Failure( + cause="A file sits directly in `docs/`, or in a directory there that is not one of " + "its five areas", + reaction="Move it (`git mv`) into the area its kind of documentation belongs to, and " + "fix the links that pointed at the old path", + ), cli_contract.Failure( cause="A reference file's relative markdown link does not resolve to an existing " "file", @@ -1308,6 +1358,7 @@ def verify(): + check_breaking_change_for_boundary() + check_changelog_budget() + check_no_issue_references() + + check_docs_layout() + check_toc_regions() + check_reference_targets() + install_doc.check_prerequisite_regions() @@ -1326,6 +1377,7 @@ def verify(): f"{len(resolver.list_type_specs())} type-spec(s) validating against their own schema, " f"{len(IGNORE_CANARIES)} ignore canaries clear, " f"no issue references in {len(shipped_prose())} shipped document(s) or command help, " + f"every docs/ file in one of its {len(DOCS_AREAS)} areas, " f"tables of contents current and every link resolving on " f"{len(toc.target_files())} reference file(s), " f"{install_doc.INSTALL_DOC} in step with tools/prerequisites.txt and " diff --git a/tools/chemenu/commands/new_page.py b/tools/chemenu/commands/new_page.py index c0efe12..7e43210 100644 --- a/tools/chemenu/commands/new_page.py +++ b/tools/chemenu/commands/new_page.py @@ -507,7 +507,7 @@ def _ensure_tracker_project(page_title: str, *, resume: bool) -> Optional[str]: "`wikitool types describe ` - what a type requires and where it lands", "`wikitool touch` - changes a page's own frontmatter afterwards", "`wikitool task new` - a tracker item without a page", - "`docs/knowledge-and-commitment.md` - why a project is a page and a tracker project", + "`docs/explanation/knowledge-and-commitment.md` - why a project is a page and a tracker project", ), )) def new_page_command( diff --git a/tools/chemenu/commands/task_cmd.py b/tools/chemenu/commands/task_cmd.py index 193aab1..ff6f4c8 100644 --- a/tools/chemenu/commands/task_cmd.py +++ b/tools/chemenu/commands/task_cmd.py @@ -4,9 +4,9 @@ read surface outside `new project` (Gitea #132, #138; #119 D1/D2/D4/D5/D6/D9). `task new` is the second write path into the task tracker, alongside `new project`'s own (`chemenu.commands.new_page._ensure_tracker_project`) - and the last creation command that pairing needed, per -`docs/knowledge-and-commitment.md`. `task close` (#138) is the one closing +`docs/explanation/knowledge-and-commitment.md`. `task close` (#138) is the one closing write: never a delete, only "mark done" (`TaskWriter.close_item`) - see that -protocol method's docstring and `docs/knowledge-and-commitment.md` for why +protocol method's docstring and `docs/explanation/knowledge-and-commitment.md` for why the surface stops there. `task list` (#138) is the read half a caller needs to get an item's id before it can close it, without first running `wikitool review`. None of the three ever touch `kb/`: an ingest that finds @@ -150,7 +150,7 @@ def _read_config(): "`wikitool task list` - a project's open items and their ids", "`wikitool task close` - marks an item done", "`wikitool new project` - a project page and its tracker project", - "`docs/knowledge-and-commitment.md` - why commitments live in the tracker, not in `kb/`", + "`docs/explanation/knowledge-and-commitment.md` - why commitments live in the tracker, not in `kb/`", ), )) def task_new_command( @@ -192,7 +192,7 @@ def task_new_command( Tracker-only by design (#132 D1): a source that carries both knowledge and a commitment gets this command for the commitment and the normal page-creation commands for the knowledge, run as two separate steps by - the calling skill - see `docs/knowledge-and-commitment.md`. + the calling skill - see `docs/explanation/knowledge-and-commitment.md`. """ if bool(project) == inbox: fail( @@ -381,7 +381,7 @@ def task_close_command( \f The only closing write this stack makes (Gitea #138); see `chemenu.tasks.protocol.TaskWriter.close_item` and - `docs/knowledge-and-commitment.md` for why.""" + `docs/explanation/knowledge-and-commitment.md` for why.""" cfg = _read_config() if cfg is None: fail( diff --git a/tools/chemenu/tasks/protocol.py b/tools/chemenu/tasks/protocol.py index 9fc4a60..cc2d340 100644 --- a/tools/chemenu/tasks/protocol.py +++ b/tools/chemenu/tasks/protocol.py @@ -21,7 +21,7 @@ cached from before that step. An item's own id (#138) is read-only data, like everything else here - it is never stored by `wikitool`, only ever passed straight back into `TaskWriter.close_item` within the same invocation. That keeps the "one name -is the only coupling" decision (`docs/knowledge-and-commitment.md` § "One +is the only coupling" decision (`docs/explanation/knowledge-and-commitment.md` § "One name, carrying the duties of an identifier") intact: no id-to-anything mapping is ever written down, so there is nothing to keep in sync. diff --git a/tools/chemenu/tests/test_dist_cmd.py b/tools/chemenu/tests/test_dist_cmd.py index 9023fa3..67ab62a 100644 --- a/tools/chemenu/tests/test_dist_cmd.py +++ b/tools/chemenu/tests/test_dist_cmd.py @@ -149,8 +149,12 @@ def repo(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Path: ) docs_dir = root / "docs" - docs_dir.mkdir() - (docs_dir / "why-gates-are-code.md").write_text("# Why gates are code\n", encoding="utf-8") + (docs_dir / "explanation").mkdir(parents=True) + (docs_dir / "explanation" / "why-gates-are-code.md").write_text( + "# Why gates are code\n", encoding="utf-8" + ) + (docs_dir / "dev").mkdir() + (docs_dir / "dev" / "development.md").write_text("# Developing this stack\n", encoding="utf-8") for relative in ("raw/CONTRACT.md", "reports/CONTRACT.md", "work/CONTRACT.md"): path = root / relative @@ -175,7 +179,16 @@ def test_plan_never_includes_commonplace(repo): def test_plan_ships_docs_verbatim(repo): plan = dist_cmd.build_plan() - assert plan["docs/why-gates-are-code.md"].content == "# Why gates are code\n" + assert plan["docs/explanation/why-gates-are-code.md"].content == "# Why gates are code\n" + + +def test_plan_never_includes_docs_dev(repo): + """docs/dev/ is the human half of instructions/dev/ - the release workflow + and the development environment - and is pruned with it, while the other + docs/ areas still ship.""" + plan = dist_cmd.build_plan() + assert not any(relative.startswith("docs/dev/") for relative in plan) + assert "docs/explanation/why-gates-are-code.md" in plan def test_plan_never_includes_instructions_dev(repo): diff --git a/tools/chemenu/tests/test_docs_verify.py b/tools/chemenu/tests/test_docs_verify.py index 24ba0d6..3e23ba8 100644 --- a/tools/chemenu/tests/test_docs_verify.py +++ b/tools/chemenu/tests/test_docs_verify.py @@ -290,12 +290,13 @@ def test_install_md_is_checked_too(tmp_path, monkeypatch): assert any("`doctor`" in issue for issue in issues) -def test_development_md_is_checked_too(tmp_path, monkeypatch): - """DEVELOPMENT.md drifted exactly this way once (Gitea #47): a table - describing what each verify command checks, removed by hand because nothing - compared it to anything.""" +def test_the_developer_guide_is_checked_too(tmp_path, monkeypatch): + """The developer guide drifted exactly this way once (Gitea #47, as the root + DEVELOPMENT.md it was then): a table describing what each verify command + checks, removed by hand because nothing compared it to anything.""" root = tmp_path - (root / "DEVELOPMENT.md").write_text( + (root / "docs" / "dev").mkdir(parents=True) + (root / "docs" / "dev" / "development.md").write_text( "| Command | Purpose |\n| `docs verify` | checks docs |\n", encoding="utf-8" ) monkeypatch.setattr(docs_verify.config, "ROOT", root) @@ -305,7 +306,7 @@ def test_development_md_is_checked_too(tmp_path, monkeypatch): def test_an_absent_listed_doc_is_skipped_not_reported(tmp_path, monkeypatch): - """The distributed-instance case: DEVELOPMENT.md is not shipped, so listing + """The distributed-instance case: docs/dev/ is not shipped, so listing it must stay inert where the file does not exist rather than failing a tree that is correct.""" root = tmp_path @@ -906,6 +907,113 @@ def test_a_target_with_neither_the_file_nor_a_template_is_still_dead(tmp_path, m assert "CONVENTIONS.md" in issues[0] +# --- docs/ layout ------------------------------------------------------------ + + +def test_this_repos_docs_are_laid_out_in_areas(): + assert docs_verify.check_docs_layout() == [] + + +def _docs_tree(tmp_path, monkeypatch, files: tuple[str, ...]): + from chemenu import config + + monkeypatch.setattr(config, "ROOT", tmp_path) + for relative in files: + path = tmp_path / relative + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text("# Page\n", encoding="utf-8") + + +def test_a_file_directly_in_docs_is_reported(tmp_path, monkeypatch): + """The layout before 9.0.0, when every page sat directly in docs/: a page + there belongs to no kind of documentation.""" + _docs_tree(tmp_path, monkeypatch, ("docs/explanation/a.md", "docs/stray.md")) + issues = docs_verify.check_docs_layout() + assert len(issues) == 1 + assert "docs/stray.md" in issues[0] and "directly in docs/" in issues[0] + + +def test_a_file_in_a_directory_that_is_no_area_is_reported(tmp_path, monkeypatch): + _docs_tree(tmp_path, monkeypatch, ("docs/guides/a.md",)) + issues = docs_verify.check_docs_layout() + assert len(issues) == 1 + assert "docs/guides/" in issues[0] + + +def test_every_area_and_nesting_inside_one_is_accepted(tmp_path, monkeypatch): + _docs_tree(tmp_path, monkeypatch, tuple( + f"docs/{area}/README.md" for area in docs_verify.DOCS_AREAS + ) + ("docs/how-to/mcp/setup.md",)) + assert docs_verify.check_docs_layout() == [] + + +def test_a_tree_without_docs_passes_the_layout_check(tmp_path, monkeypatch): + _docs_tree(tmp_path, monkeypatch, ()) + assert docs_verify.check_docs_layout() == [] + + +def test_verify_raises_when_a_docs_file_is_outside_every_area(monkeypatch): + monkeypatch.setattr(docs_verify, "check_docs_layout", lambda: ["stray"]) + with pytest.raises(typer.Exit): + docs_verify.verify() + + +# The rewrite instructions/migrations/9.0.0-docs-diataxis.md tells an instance +# to run, as the regex it spells out for `sed -E`. +_DIATAXIS_NAMES = ( + "pipeline-rationale|ownership-and-templates|language-boundaries|why-gates-are-code|" + "version-model|knowledge-and-commitment|model-and-effort-selection" +) +_DIATAXIS_SED = f"s#docs/({_DIATAXIS_NAMES})\\.md#docs/explanation/\\1.md#g" + + +def test_the_9_0_0_upgrade_breaks_an_old_conventions_link_until_the_migration_ran( + tmp_path, monkeypatch +): + """The breaking change 9.0.0 declares, end to end on the check that reports + it: an instance adopted `kb/CONVENTIONS.md` from a template linking + `../docs/language-boundaries.md`, and the upgrade never touches that + instance-owned file while it deletes the page at its old path. After the + swap `docs verify` reports the link; after the migration document's + rewrite it does not.""" + import re + + from chemenu import config + + document = ( + config.ROOT / "instructions" / "migrations" / "9.0.0-docs-diataxis.md" + ).read_text(encoding="utf-8") + monkeypatch.setattr(config, "ROOT", tmp_path) + (tmp_path / "kb").mkdir() + conventions = tmp_path / "kb" / "CONVENTIONS.md" + conventions.write_text( + "# Conventions\n\nWhy the KB language is a value: " + "[docs/language-boundaries.md](../docs/language-boundaries.md).\n", + encoding="utf-8", + ) + # The machinery as 9.0.0 installs it: the page only under its new path. + (tmp_path / "docs" / "explanation").mkdir(parents=True) + (tmp_path / "docs" / "explanation" / "language-boundaries.md").write_text( + "# Language boundaries\n", encoding="utf-8" + ) + + issues = docs_verify.check_reference_targets() + assert len(issues) == 1 + assert "kb/CONVENTIONS.md:3" in issues[0] + assert "../docs/language-boundaries.md" in issues[0] + + # The migration document's rewrite, the same expression it gives to `sed -E`. + assert _DIATAXIS_SED in document + _, pattern, replacement, _ = _DIATAXIS_SED.split("#") + conventions.write_text( + re.sub(pattern, replacement, conventions.read_text(encoding="utf-8")), + encoding="utf-8", + ) + + assert "../docs/explanation/language-boundaries.md" in conventions.read_text(encoding="utf-8") + assert docs_verify.check_reference_targets() == [] + + def test_every_reference_files_link_targets_resolve(): """Forward direction, against the real tree: every relative link in a file `toc.target_files()` covers must resolve - this is what a `../` count diff --git a/tools/chemenu/tests/test_git_publish.py b/tools/chemenu/tests/test_git_publish.py index 81a62ca..9d88ea6 100644 --- a/tools/chemenu/tests/test_git_publish.py +++ b/tools/chemenu/tests/test_git_publish.py @@ -289,7 +289,7 @@ def test_stack_machinery_excludes_ordinary_content(): """kb/ content, docs/ prose and work/ scratch carry no normative sentence and are not what the closing-phase reminder is about.""" assert not touches_stack_machinery(["kb/entities/systems/Foo.md"]) - assert not touches_stack_machinery(["docs/why-gates-are-code.md"]) + assert not touches_stack_machinery(["docs/explanation/why-gates-are-code.md"]) assert not touches_stack_machinery(["work/ingest-x/extract-0.md"]) assert not touches_stack_machinery([]) diff --git a/tools/chemenu/tests/test_toc.py b/tools/chemenu/tests/test_toc.py index 030b520..2236a4d 100644 --- a/tools/chemenu/tests/test_toc.py +++ b/tools/chemenu/tests/test_toc.py @@ -166,7 +166,7 @@ def test_target_files_matches_the_documented_scope(): assert "instructions/CONTRACT.md" in relatives assert "instructions/dev/version-parts.md" in relatives # flat, still instructions/**.md assert "types/source.md" in relatives # a page type-spec, not only the stage contract - assert "docs/version-model.md" in relatives + assert "docs/explanation/version-model.md" in relatives assert not any(rel.endswith("SKILL.md") for rel in relatives) diff --git a/types/type-guidance.md b/types/type-guidance.md index 2b6d601..97e15a6 100644 --- a/types/type-guidance.md +++ b/types/type-guidance.md @@ -55,7 +55,7 @@ that file's own `guidance:` frontmatter field. Relevant Notes: - Ships verbatim with every distribution; the type-spec it documents ships only as a `.template` - where `root: kb` - see `docs/ownership-and-templates.md` § "Where the file boundary used to + where `root: kb` - see `docs/explanation/ownership-and-templates.md` § "Where the file boundary used to strain" - No `## Template` section here: a guidance file's body is prose an author reads, not a body structure `wikitool new` would scaffold. `type-guidance` declares no `base_dir:` for the same