feat: dist upgrade - apply a stack update, not just detect one (#7)
CI / verify (push) Successful in 54s
Release / release (push) Successful in 34s

Files changed:
- CHANGES.md
- INSTALL.md
- VERSION
- tools/CONTRACT.md
- tools/chemenu/commands/dist_cmd.py
- tools/chemenu/kb_state.py
- tools/chemenu/ownership.py
- tools/chemenu/tests/test_dist_upgrade.py
This commit is contained in:
2026-09-04 11:36:21 +02:00
parent 1b5ffea854
commit cd81ba3d4f
8 changed files with 988 additions and 49 deletions
+50 -1
View File
@@ -35,7 +35,7 @@ dev-checkout concern - readable here, never shipped as something to parse.
--- ---
## 4.5.0-beta.3 - 2026-09-04 - upstream merge: keep gitignored local data under a content stage, refuse a merge git never opened, report what actually changed ## 4.5.0-beta.4 - 2026-09-04 - dist upgrade: apply a stack update, not just detect one (#7)
**Author:** Torben Nehmer **Author:** Torben Nehmer
@@ -44,6 +44,7 @@ dev-checkout concern - readable here, never shipped as something to parse.
- wikitool upstream merge/verify: code procedure for taking a stack update, ownership.py as the shared stack/instance boundary - wikitool upstream merge/verify: code procedure for taking a stack update, ownership.py as the shared stack/instance boundary
- upstream merge: combined-commit regression test (edit+add+delete+contract+template+contract-delete in one commit) - upstream merge: combined-commit regression test (edit+add+delete+contract+template+contract-delete in one commit)
- upstream merge: keep gitignored local data under a content stage, refuse a merge git never opened, report what actually changed - upstream merge: keep gitignored local data under a content stage, refuse a merge git never opened, report what actually changed
- dist upgrade: apply a stack update, not just detect one (#7)
<!-- /wikitool:bumps --> <!-- /wikitool:bumps -->
Die Prosa zu 4.4.0 - Changelog-Eintrag, `docs/version-model.md`, `instructions/dev/version-parts.md`, Die Prosa zu 4.4.0 - Changelog-Eintrag, `docs/version-model.md`, `instructions/dev/version-parts.md`,
@@ -159,6 +160,54 @@ jetzt `git diff` zwischen Vor- und Nach-Commit, kennzeichnet Löschungen, und st
bekommen, warum die Grenze ein Prädikat und keine Liste ist — die Begründung, die dieses Issue bekommen, warum die Grenze ein Prädikat und keine Liste ist — die Begründung, die dieses Issue
erarbeitet hat, gehörte in die Hintergrunddoku und nicht nur in einen Changelog-Eintrag. erarbeitet hat, gehörte in die Hintergrunddoku und nicht nur in einen Changelog-Eintrag.
**Fünfter Bump: `wikitool dist upgrade` (#7), der zweite der beiden Update-Wege.** `upstream
merge` oben bedient eine Instanz mit gemeinsamer Git-History; `dist upgrade` bedient eine
Instanz aus einem Tarball, ohne History, die bislang eine rein manuelle Prozedur in
`INSTALL.md` durchlaufen musste - Schritt 4 verlangte einen sha256-Vergleich von Hand gegen den
`files`-Block der alten `.wikitool-release.json`.
Die tragende Regel: die Schreibmenge ist genau der `files`-Block der *neuen*
`.wikitool-release.json`, minus was ein Export aus einer leeren Vorlage neu sät
(`chemenu.ownership.is_export_stub`, wie bisher schon für `kb/log.md`/`.gitkeep`) oder einmalig
sät und danach der Instanz gehört (`chemenu.ownership.is_upgrade_preserved`, neu für
`.wikitool-kb.json` und `CHANGES.md`), plus der Stamp selbst. Jeder Kandidatpfad wird gegen die
*alte* Instanz-Summe klassifiziert: unverändert wird geräuschlos überschrieben, neu im Release
wird angelegt, lokal verändert oder gelöscht wird **nie** still überschrieben - der Lauf bricht
mit der vollständigen Liste ab, außer `--keep-local` sagt ausdrücklich, dass die Dateien liegen
bleiben sollen. `--prune` entfernt zusätzlich aus dem Release entfallene Dateien, aber nur
solche, die seit der Installation unverändert sind.
Die Migrationskette nach dem Tausch wird aus den `instructions/migrations/` des *neuen* Baums
ermittelt (`kb_state.load_migrations` bekam dafür einen `directory`-Parameter) und nur
gemeldet, nie ausgeführt - es gibt bewusst kein `migrate run`. Eine bereits gegen die
*installierte* Maschinerie offene Kette lässt den Befehl abbrechen, bevor er die Quelle
überhaupt öffnet. `kb_state.divergent_files()` (bisher nur von `migrate status` gelesen) ist
jetzt eine dünne Hülle um das neue, zwei-Baum-fähige `compare_against_stamp()` - gleiches
Verhalten für den bestehenden Aufrufer, wiederverwendbar für `dist upgrade`s eigenen Vergleich.
Quelle ist immer ein bereits vorhandenes Verzeichnis oder `.tar.gz` - kein Download, das bleibt
allein `version check`s Sache. Ein Tarball muss genau ein Top-Level-Verzeichnis enthalten (die
Form, in der `release.yml` es baut) und wird gegen eine `.sha256`-Beidatei geprüft, falls eine
danebenliegt (fehlt sie: WARN, kein Abbruch). Weitere Abbruchgründe vor jedem Schreiben: fehlende
lokale `VERSION`/`.wikitool-kb.json`/Stamp mit `files`-Block, ein schmutziger Arbeitsbaum (kein
Git-Repo ist ein WARN, keine Sperre), eine Vorab-Version (`-beta.N`) ohne `--pre`, sowie ein
Downgrade; Gleichstand ist ein No-op. Ein Grenzübertritt der Kompatibilität wird laut gemeldet,
blockiert aber nicht. Committet und pusht nichts (Invariante 5).
Gegenüber dem ersten Entwurf des Issues zwei Korrekturen, die dort auch nachgetragen sind: der
`files`-Block wurde entgegen der ursprünglichen Annahme bereits vor diesem Bump gelesen
(`divergent_files`/`migrate status`), und die Migrationskette war ursprünglich falsch begründet
- sie kann nur aus dem *neuen* Baum kommen, nicht durch eine andere Abfragereihenfolge aus der
alten Instanz. 24 neue Tests unter `test_dist_upgrade.py` decken die Klassifikation, alle
Abbruchgründe, `--keep-local`, `--prune` und beide Quellformen (Verzeichnis und Tarball,
inklusive der sha256- und Top-Level-Prüfung) ab.
Bewusst nicht angetastet: `instructions/private-instance.md` (der Clone-Weg ändert sich nicht,
`INSTALL.md` benennt jetzt beide Wege nebeneinander) und die Frage, wie `dist upgrade` mit
Collection-Templates umgeht, deren Namen eine fremde Instanz gar nicht hat - es verhält sich wie
`upstream merge` und schreibt sie, was ein eigenes Issue gegen den Export wäre, keins gegen das
Upgrade.
--- ---
## 4.4.0 - 2026-09-03 - Versionskandidat statt Bump-pro-Release: VERSION traegt -beta.N, version release fixiert ## 4.4.0 - 2026-09-03 - Versionskandidat statt Bump-pro-Release: VERSION traegt -beta.N, version release fixiert
+57 -19
View File
@@ -167,10 +167,17 @@ wenn nicht). `tools/wikitool version notes` druckt den Eintrag.
### Eine Instanz aktualisieren ### Eine Instanz aktualisieren
Das Anwenden eines Updates ist ein bewusst manueller Vorgang - es schreibt in eine Instanz, die Zwei Wege, je nachdem, wie diese Instanz entstanden ist. Ein **Clone mit gemeinsamer
bereits Inhalt hat. Der Inhalt hat dabei eine **eigene Version**: `.wikitool-kb.json` sagt, in Git-History** (`upstream`-Remote auf das Ursprungs-Repo, siehe
welcher Form die Seiten vorliegen, unabhängig davon, welche Maschinerie danebensteht. Genau [instructions/private-instance.md](instructions/private-instance.md)) nimmt Stack-Updates per
dieser Unterschied ist der Zustand, in dem sich jede Instanz mitten im Upgrade befindet. echtem Drei-Wege-Merge: `tools/wikitool upstream merge`. Alles Folgende gilt für eine **Instanz
aus einem Tarball**, ohne gemeinsame History - der Weg unten unter „Eine Instanz aktualisieren"
nutzt sie.
Das Anwenden eines Updates schreibt in eine Instanz, die bereits Inhalt hat. Der Inhalt hat dabei
eine **eigene Version**: `.wikitool-kb.json` sagt, in welcher Form die Seiten vorliegen,
unabhängig davon, welche Maschinerie danebensteht. Genau dieser Unterschied ist der Zustand, in
dem sich jede Instanz mitten im Upgrade befindet.
1. **Vor dem Tausch** prüfen, was ansteht - solange `VERSION` noch die alte ist: 1. **Vor dem Tausch** prüfen, was ansteht - solange `VERSION` noch die alte ist:
@@ -178,19 +185,35 @@ dieser Unterschied ist der Zustand, in dem sich jede Instanz mitten im Upgrade b
tools/wikitool migrate status tools/wikitool migrate status
``` ```
2. Release-Tarball herunterladen und entpacken (Weg A), die Release-Notes lesen. Steht hier etwas aus, erst diese Migrationskette abschließen (Schritt 5 unten) - `dist upgrade`
3. Die **Maschinerie** aus dem Tarball über die Instanz kopieren: `tools/`, `types/`, verweigert den Tausch sonst von selbst.
`instructions/`, `AGENTS.md`, `VERSION`, `.wikitool-release.json` - **und `kb/CONTRACT.md`**.
Die letzte Datei liegt unter einem Content-Verzeichnis, ist aber Stack-Eigentum: sie hält, 2. Release-Tarball herunterladen und die Release-Notes lesen (Weg A oben).
was `wikitool` erzwingt, und ist in jeder Instanz gleich. Nicht anfassen: alles andere unter 3. **Maschinerie tauschen:**
`kb/` und `raw/`, `work/`, `.wikitool-kb.json` und `.git/` - das ist die Instanz selbst,
`kb/CONVENTIONS.md` und die `kb/*/COLLECTION.md` eingeschlossen. ```bash
4. Achtung bei lokal angepassten Stack-Dateien. Die Autorenkonventionen gehören **nicht** dazu: tools/wikitool dist upgrade <tarball-oder-verzeichnis> --dry-run
`kb/CONVENTIONS.md` und die `kb/*/COLLECTION.md` liegen unter `kb/`, werden in Schritt 3 ```
also ohnehin nicht angefasst - genau dafür ist der Schnitt da. Wer darüber hinaus etwas
unter `tools/`, `types/` oder `instructions/` verändert hat, sichert das vorher und spielt Klassifiziert jede Datei aus dem `files`-Block der neuen `.wikitool-release.json`:
es danach wieder ein. Welche Dateien das sind, verrät ein Vergleich gegen die sha256-Summen unverändert seit der Installation, lokal verändert oder gelöscht, neu im Release, oder aus dem
im `files`-Block der alten `.wikitool-release.json`. Release entfallen - und druckt die Migrationskette, die nach dem Tausch aussteht, ohne sie
auszuführen. Ohne `--dry-run` schreibt der Befehl; eine lokal veränderte oder gelöschte Datei
wird dabei **nie** stillschweigend überschrieben - der Lauf bricht mit der vollständigen Liste
ab, es sei denn `--keep-local` ist gesetzt (dann bleibt jede davon unangetastet, erneut
gemeldet). `--prune` entfernt zusätzlich Dateien, die der neue Release nicht mehr ausliefert
und die seit der Installation unverändert sind. Voraussetzungen: ein sauberer Arbeitsbaum
(kein Git-Repo ist ein WARN, keine Sperre), eine lokale `.wikitool-release.json` mit
`files`-Block (fehlt sie, siehe „Fallstricke" unten), und `.wikitool-kb.json` vorhanden.
Committet und pusht nichts (Invariante 5). Vollständiger Fehlerkontrakt:
[tools/CONTRACT.md](tools/CONTRACT.md).
Eine lokal veränderte Stack-Datei ist damit sichtbar, statt von Hand gegen die sha256-Summen
im `files`-Block geprüft werden zu müssen - genau der Schritt, der vor `4.5.0` hier stand.
4. Bei einer Kompatibilitätsgrenze (`dist upgrade` meldet sie laut) die Release-Notes vor dem
nächsten Schritt lesen: **Breaking Change:** und **Migration:** im Eintrag von
`tools/wikitool version notes` sagen, was aufhört zu funktionieren und ob der Korpus
umgeschrieben werden muss.
5. **Die Migrationskette abarbeiten.** `tools/wikitool migrate status` listet jetzt alle 5. **Die Migrationskette abarbeiten.** `tools/wikitool migrate status` listet jetzt alle
offenen Migrationen in der Reihenfolge, in der sie laufen müssen - bei einem Sprung über offenen Migrationen in der Reihenfolge, in der sie laufen müssen - bei einem Sprung über
mehrere Versionen sind das mehrere. Für jede: das genannte Dokument unter mehrere Versionen sind das mehrere. Für jede: das genannte Dokument unter
@@ -204,15 +227,30 @@ dieser Unterschied ist der Zustand, in dem sich jede Instanz mitten im Upgrade b
`done` verweigert jede Version, die nicht das nächste Glied ist - eine übersprungene `done` verweigert jede Version, die nicht das nächste Glied ist - eine übersprungene
Migration hinterlässt einen Korpus in einer Form, die keine Version beschreibt. Ein Migration hinterlässt einen Korpus in einer Form, die keine Version beschreibt. Ein
abgebrochenes Upgrade wird durch erneutes `migrate status` fortgesetzt. abgebrochenes Upgrade wird durch erneutes `migrate status` fortgesetzt.
6. Prüfen: `tools/wikitool migrate verify --from <commit vor der Migration>`, dann `doctor`, 6. Prüfen: `tools/wikitool migrate verify --from <commit vor dem Tausch>`, dann `doctor`,
`docs verify`, `instructions verify` und `lint`. Zum Schluss `docs verify`, `instructions verify` und `lint`. Zum Schluss
`tools/wikitool instructions sync` (die Skills sind Kopien) und die Agent-Session neu `tools/wikitool instructions sync` (die Skills sind Kopien) und die Agent-Session neu
starten. starten. `dist upgrade` nennt diese Reihenfolge im eigenen Abschlussbericht, führt aber keinen
der Schritte selbst aus.
`doctor` warnt, solange `kb_version` hinter `VERSION` zurückliegt und noch Migrationen offen `doctor` warnt, solange `kb_version` hinter `VERSION` zurückliegt und noch Migrationen offen
sind. Einer Instanz, die älter ist als `.wikitool-kb.json`, fehlt die Datei ganz - dann einmalig sind. Einer Instanz, die älter ist als `.wikitool-kb.json`, fehlt die Datei ganz - dann einmalig
`tools/wikitool migrate baseline <version>` aufrufen; geraten wird nichts. `tools/wikitool migrate baseline <version>` aufrufen; geraten wird nichts.
**Fallstricke.** Eine Instanz ohne lokale `.wikitool-release.json` (oder eine ohne `files`-Block,
aus der Zeit vor `4.5.0`) hat für `dist upgrade` keine Basis, gegen die es eine lokale Änderung
erkennen könnte, und verweigert den Tausch - dafür gibt es heute keine Reparatur (siehe Gitea #7).
Der Befehl lädt selbst nichts herunter: `<tarball-oder-verzeichnis>` muss vorher aus Weg A
geholt werden, und ein Tarball muss genau ein Top-Level-Verzeichnis enthalten - die Form, in der
`.gitea/workflows/release.yml` es baut.
Vor `4.5.0` stand hier ein rein manueller Ablauf (Maschinerie von Hand kopieren, `kb/CONTRACT.md`
eingeschlossen, sha256-Vergleich von Hand). `dist upgrade` ersetzt genau diesen Teil; wer ihn
dennoch von Hand nachvollziehen will oder muss (ein Werkzeug, das `wikitool` selbst nicht
ausführen kann), findet die Dateiliste im `files`-Block der `.wikitool-release.json` und die
Ausnahmen (`kb/CONVENTIONS.md`, `kb/*/COLLECTION.md`, `.wikitool-kb.json`) in
[tools/CONTRACT.md](tools/CONTRACT.md)s `dist upgrade`-Zeile.
### Sonderfall: Update von 1.x auf 2.0.0 ### Sonderfall: Update von 1.x auf 2.0.0
Mit `2.0.0` wurde das Ursprungs-Repo von `torben/llm-wiki-test1` auf `torben/chemenu` Mit `2.0.0` wurde das Ursprungs-Repo von `torben/llm-wiki-test1` auf `torben/chemenu`
+1 -1
View File
@@ -1 +1 @@
4.5.0-beta.3 4.5.0-beta.4
+2
View File
@@ -72,6 +72,7 @@ tools/wikitool <command> --help
| `eval sessions [--json]` | List the sessions that have a trace under `reports/telemetry/`, most recent first. Read-only and exempt from the Iteration Budget Gate | | `eval sessions [--json]` | List the sessions that have a trace under `reports/telemetry/`, most recent first. Read-only and exempt from the Iteration Budget Gate |
| `eval score [--session <id>] [--json] [--markdown out.md] [--save] [--fail-on-error]` | Score one traced session: structural state from `lint`'s own checks (L1) plus trajectory rules over the trace (L2) - was a refused call repeated unchanged, was a gate flag passed without that gate having refused anything, did a publish of `kb/` pages go unlogged. Defaults to the current session. `--save` writes `reports/evals/<date>/<session>.{json,md}`. Read-only over `kb/` and exempt from the budget; see [../EVALS.md](../EVALS.md) | | `eval score [--session <id>] [--json] [--markdown out.md] [--save] [--fail-on-error]` | Score one traced session: structural state from `lint`'s own checks (L1) plus trajectory rules over the trace (L2) - was a refused call repeated unchanged, was a gate flag passed without that gate having refused anything, did a publish of `kb/` pages go unlogged. Defaults to the current session. `--save` writes `reports/evals/<date>/<session>.{json,md}`. Read-only over `kb/` and exempt from the budget; see [../EVALS.md](../EVALS.md) |
| `dist export <target> [--dry-run] [--source-repo U] [--source-commit SHA] [--release-url U] [--update-url U]` | Write a contentless, distributable copy of this repo's machinery into an empty `<target>` directory: `AGENTS.md`/`README.md`/`EVALS.md` with any `<!-- dist:strip-start -->...<!-- dist:strip-end -->` region removed, `instructions/` (minus `instructions/dev/`), `types/` (the `root: kb` page type-specs and their schemas re-keyed as `.template`, the stack's own verbatim), `docs/` verbatim, `tools/` (no venv/caches), the `.github/hooks/`+`.vibe/` session-tracing config plus `.claude/settings.json`, `kb/CONTRACT.md` (no pages, no areas), empty `raw/{articles,documents,notes,assets}/`, `VERSION`, `USER.md.template`/`SOUL.md.template` plus `kb/CONVENTIONS.md.template` and each collection's contract re-keyed as `kb/<name>/COLLECTION.md.template` (the templates ship; the filled `USER.md`/`SOUL.md`/`kb/CONVENTIONS.md`/`kb/<name>/COLLECTION.md`/`types/<page-type>.md` never do - all of them bind their instance and none are the stack's to decide, and `find_leaks` refuses a plan carrying one), and a generated `.wikitool-release.json` stamp (version, export date, origin, and a sha256 per exported file - the base a later upgrade would compare against). The four origin options only fill stamp fields: `export` never calls git and cannot discover them. Refuses a non-empty target, and a tree with no `VERSION`. See `instructions/setup-instance.md`. One-way: there is no command that reconstructs a distributed instance into a dev instance - work on the stack in the origin repo (or a new dev instance exported from it) instead | | `dist export <target> [--dry-run] [--source-repo U] [--source-commit SHA] [--release-url U] [--update-url U]` | Write a contentless, distributable copy of this repo's machinery into an empty `<target>` directory: `AGENTS.md`/`README.md`/`EVALS.md` with any `<!-- dist:strip-start -->...<!-- dist:strip-end -->` region removed, `instructions/` (minus `instructions/dev/`), `types/` (the `root: kb` page type-specs and their schemas re-keyed as `.template`, the stack's own verbatim), `docs/` verbatim, `tools/` (no venv/caches), the `.github/hooks/`+`.vibe/` session-tracing config plus `.claude/settings.json`, `kb/CONTRACT.md` (no pages, no areas), empty `raw/{articles,documents,notes,assets}/`, `VERSION`, `USER.md.template`/`SOUL.md.template` plus `kb/CONVENTIONS.md.template` and each collection's contract re-keyed as `kb/<name>/COLLECTION.md.template` (the templates ship; the filled `USER.md`/`SOUL.md`/`kb/CONVENTIONS.md`/`kb/<name>/COLLECTION.md`/`types/<page-type>.md` never do - all of them bind their instance and none are the stack's to decide, and `find_leaks` refuses a plan carrying one), and a generated `.wikitool-release.json` stamp (version, export date, origin, and a sha256 per exported file - the base a later upgrade would compare against). The four origin options only fill stamp fields: `export` never calls git and cannot discover them. Refuses a non-empty target, and a tree with no `VERSION`. See `instructions/setup-instance.md`. One-way: there is no command that reconstructs a distributed instance into a dev instance - work on the stack in the origin repo (or a new dev instance exported from it) instead |
| `dist upgrade <source> [--dry-run] [--keep-local] [--prune] [--pre]` | Apply a stack update `dist export` produced - the write half of `version check`. Never downloads anything: `<source>` is an already-fetched export directory or `.tar.gz` release archive (verified against a sibling `.sha256` if one is present; WARNs, does not block, if it is absent), which must unpack to exactly one top-level directory - the shape `.gitea/workflows/release.yml` packs. The write set is exactly the *new* `.wikitool-release.json`'s `files` block, minus what an export re-seeds from a blank template every time (`kb/log.md`, `raw/*/.gitkeep` - `chemenu.ownership.is_export_stub`) or seeds once and the instance owns from then on (`.wikitool-kb.json`, `CHANGES.md` - `chemenu.ownership.is_upgrade_preserved`), plus the stamp itself, always rewritten. Every candidate path is classified against the *local* `.wikitool-release.json`'s recorded digest for it: unchanged is overwritten silently, absent from the old stamp is created, and locally modified or locally deleted is **never** silently overwritten - the run aborts with the full list unless `--keep-local` says to proceed and leave every one of them untouched. A path in the old stamp but not the new one is reported as no longer part of the release and left alone unless `--prune` is passed, which removes it only if it is still unchanged since installation. Reports the migration chain the new machinery would owe (`chemenu.kb_state.chain` over the *new* tree's `instructions/migrations/`, read via a `directory` argument to `load_migrations`) but never runs any of it - there is no `migrate run`. Refuses before touching the source at all when: `VERSION` or `.wikitool-release.json` (with a `files` block) is missing locally, `.wikitool-kb.json` is missing, a migration is already outstanding against the *installed* machinery, or the working tree is dirty (not being a git repository at all is a WARN, not a refusal). Refuses after reading the source when: it carries no `VERSION`/`.wikitool-release.json`/`files` block, its version is older than or equal to the installed one (equal is a no-op success), or it is a pre-release (`-beta.N`) without `--pre`. Reports, but does not block on, a crossed compatibility boundary. Never touches git - no commit, no push (invariant 5). See Gitea #7 and `INSTALL.md` § "Eine Instanz aktualisieren" |
| `version show [--json]` | Print this instance's stack version and where it came from (development tree, or a distribution with its export date and origin). Bare `wikitool version` is an alias for this. Read-only, offline, and **exempt from the Iteration Budget Gate** | | `version show [--json]` | Print this instance's stack version and where it came from (development tree, or a distribution with its export date and origin). Bare `wikitool version` is an alias for this. Read-only, offline, and **exempt from the Iteration Budget Gate** |
| `version check [--url U] [--timeout S] [--json]` | Ask the origin's release feed whether a newer stack exists, and whether the step crosses a compatibility boundary (`state: current\|update\|migration\|ahead`). **The only command in `wikitool` that makes a network call** - never reached implicitly from another command, needs no key, times out, and reports an unreachable feed as an error rather than as "up to date". The feed is `$WIKITOOL_UPDATE_URL`, else the release stamp's, else the built-in origin; `$WIKITOOL_UPDATE_TOKEN` is only needed if that feed is not readable anonymously. Read-only and exempt from the budget gate | | `version check [--url U] [--timeout S] [--json]` | Ask the origin's release feed whether a newer stack exists, and whether the step crosses a compatibility boundary (`state: current\|update\|migration\|ahead`). **The only command in `wikitool` that makes a network call** - never reached implicitly from another command, needs no key, times out, and reports an unreachable feed as an error rather than as "up to date". The feed is `$WIKITOOL_UPDATE_URL`, else the release stamp's, else the built-in origin; `$WIKITOOL_UPDATE_TOKEN` is only needed if that feed is not readable anonymously. Read-only and exempt from the budget gate |
| `version notes [--version X.Y.Z]` | Print one version's `CHANGES.md` entry, for use as release notes (default: this tree's `VERSION`). Read-only and exempt from the budget gate | | `version notes [--version X.Y.Z]` | Print one version's `CHANGES.md` entry, for use as release notes (default: this tree's `VERSION`). Read-only and exempt from the budget gate |
@@ -192,6 +193,7 @@ is atomic, and whether a retry is safe.
| `instructions sync` / `verify` / `list` | Nothing found under `instructions/`, a malformed instruction or `SKILL.md`, a published copy that drifted from its source, an instruction nothing references (or, for `manual: true`, one that IS linked from AGENTS.md or a skill and so risks running implicitly), something under `instructions/dev/` referenced from outside it and outside a dist:strip block, or (sync) a target directory that is not a published skill and `--force` was not passed | `sync` rewrites one directory per target (idempotent); `verify`/`list` are read-only | Fix the flagged file, then re-run. For drift, re-run `sync`: the source under `instructions/` always wins, and a published copy is never edited directly | | `instructions sync` / `verify` / `list` | Nothing found under `instructions/`, a malformed instruction or `SKILL.md`, a published copy that drifted from its source, an instruction nothing references (or, for `manual: true`, one that IS linked from AGENTS.md or a skill and so risks running implicitly), something under `instructions/dev/` referenced from outside it and outside a dist:strip block, or (sync) a target directory that is not a published skill and `--force` was not passed | `sync` rewrites one directory per target (idempotent); `verify`/`list` are read-only | Fix the flagged file, then re-run. For drift, re-run `sync`: the source under `instructions/` always wins, and a published copy is never edited directly |
| `docs verify` | A command, contract, or type-form mismatch was found | Read-only | Fix the documentation it names, then re-run | | `docs verify` | A command, contract, or type-form mismatch was found | Read-only | Fix the documentation it names, then re-run |
| `dist export` | Target exists and is not empty, is not a directory, or the tree has no readable `VERSION` | Yes - nothing is written until every file is planned | Point `<target>` at an empty (or new) directory and retry. Never merge into a non-empty one by hand | | `dist export` | Target exists and is not empty, is not a directory, or the tree has no readable `VERSION` | Yes - nothing is written until every file is planned | Point `<target>` at an empty (or new) directory and retry. Never merge into a non-empty one by hand |
| `dist upgrade` | Missing local `VERSION`/`.wikitool-release.json`(`files`)/`.wikitool-kb.json`, a migration already outstanding against the installed machinery, a dirty working tree, a source with no `VERSION`/stamp/`files` block, a source version that is older than, equal to, or (without `--pre`) a pre-release relative to the installed one, or one or more locally changed files without `--keep-local` | **Yes for the refusal cases above - nothing is written.** Once writing starts it is a plain sequential file copy with no partial-state cleanup: an interruption mid-copy (killed process, disk full) can leave the tree part-old, part-new | For every refusal above: fix the named precondition and retry - none of them are transient. For locally changed files: reconcile them by hand and retry, or re-run with `--keep-local` to proceed and leave them untouched (repeatable - it reports the same files again on every subsequent run until they stop diverging). An interrupted write is not resumed automatically; compare the tree against the printed classification and finish or revert by hand |
| `version show` / `version notes` | `VERSION` is missing or unparseable; for `notes`, no `CHANGES.md` entry names the version asked for | Read-only | Fix `VERSION`, or write the changelog entry (`version bump` writes its heading). Safe to retry | | `version show` / `version notes` | `VERSION` is missing or unparseable; for `notes`, no `CHANGES.md` entry names the version asked for | Read-only | Fix `VERSION`, or write the changelog entry (`version bump` writes its heading). Safe to retry |
| `version check` | The feed could not be reached, answered non-JSON, or carried no `tag_name`. **Never** answers "up to date" for a question it could not ask | Read-only, no local writes | A network failure is transient - retry once, then report it. HTTP 401/403 names `$WIKITOOL_UPDATE_TOKEN`; 404 means no release exists yet or the URL points at the wrong repo | | `version check` | The feed could not be reached, answered non-JSON, or carried no `tag_name`. **Never** answers "up to date" for a question it could not ask | Read-only, no local writes | A network failure is transient - retry once, then report it. HTTP 401/403 names `$WIKITOOL_UPDATE_TOKEN`; 404 means no release exists yet or the URL points at the wrong repo |
| `version bump` | More or fewer than one of `--major/--minor/--patch`, an empty `--title`, a missing `VERSION`/`CHANGES.md`, `VERSION` and the changelog's newest entry naming different versions, an escalation to a boundary crossing without `--breaking` or with neither a migration document nor `--no-migration`, or `--breaking`/`--no-migration` on a bump that crosses nothing | No - `VERSION` then `CHANGES.md` | **Not idempotent**: a second run escalates or continues the candidate again. If the outcome is uncertain, read `VERSION` and the top of `CHANGES.md` before retrying | | `version bump` | More or fewer than one of `--major/--minor/--patch`, an empty `--title`, a missing `VERSION`/`CHANGES.md`, `VERSION` and the changelog's newest entry naming different versions, an escalation to a boundary crossing without `--breaking` or with neither a migration document nor `--no-migration`, or `--breaking`/`--no-migration` on a bump that crosses nothing | No - `VERSION` then `CHANGES.md` | **Not idempotent**: a second run escalates or continues the candidate again. If the outcome is uncertain, read `VERSION` and the top of `CHANGES.md` before retrying |
+409 -1
View File
@@ -33,14 +33,20 @@ import hashlib
import json import json
import os import os
import re import re
import shutil
import stat import stat
import subprocess
import tarfile
import tempfile
from contextlib import contextmanager
from dataclasses import dataclass
from pathlib import Path from pathlib import Path
from typing import Callable, NamedTuple, Optional, Union from typing import Callable, NamedTuple, Optional, Union
import typer import typer
from chemenu import config, conventions, kb_collections, kb_state, ownership, version as version_mod from chemenu import config, conventions, kb_collections, kb_state, ownership, version as version_mod
from chemenu.commands._util import fail, rel_path, success, today_iso from chemenu.commands._util import console, fail, rel_path, success, today_iso
app = typer.Typer(help="Build a distributable copy of the wiki machinery.") app = typer.Typer(help="Build a distributable copy of the wiki machinery.")
@@ -581,3 +587,405 @@ def run_export(target: Path, dry_run: bool = False, origin: Optional[Origin] = N
_write_plan(target, plan) _write_plan(target, plan)
success(f"Exported {len(plan)} file(s) to {rel_path(target)}.") success(f"Exported {len(plan)} file(s) to {rel_path(target)}.")
# --- dist upgrade ------------------------------------------------------------
#
# Apply a release `dist export` produced, rather than merely detecting one
# (`version check`). Gitea #7 has the full design; the short version: the
# write set is exactly the *new* stamp's `files` block, minus the paths an
# export seeds once and the instance owns from then on
# (`ownership.is_export_stub`, `ownership.is_upgrade_preserved`), plus the
# stamp itself. Every candidate path is classified against the *old* stamp's
# recorded digest - unchanged, locally modified, or locally deleted - and a
# modified/deleted file is never silently overwritten. This never calls a
# release feed; the caller supplies an already-downloaded tree or archive.
@dataclass(frozen=True)
class FileClassification:
"""The four-way split of every path `dist upgrade` would touch, plus the
fifth direction (`removed`) that has no write set of its own."""
unchanged: list[str]
modified: list[str]
deleted: list[str]
new: list[str]
removed: list[str]
@property
def blocked(self) -> list[str]:
"""Locally changed paths - modified or deleted - which are never
silently overwritten."""
return sorted(self.modified + self.deleted)
def _write_candidates(new_files: dict) -> set[str]:
"""Every path `dist upgrade` may write, from the new stamp's `files`
block: everything except the paths an export re-seeds from a blank
template every time (`ownership.is_export_stub`) and the paths an export
seeds once and the instance owns afterward (`ownership.is_upgrade_preserved`).
The release stamp itself is added separately - it is never a member of its
own `files` block, see `build_stamp`."""
return {
relative
for relative in new_files
if not ownership.is_export_stub(Path(relative).name)
and not ownership.is_upgrade_preserved(relative)
}
def _classify_files(old_files: dict, new_files: dict) -> FileClassification:
candidates = _write_candidates(new_files)
recorded_for_candidates = {r: d for r, d in old_files.items() if r in candidates}
statuses = kb_state.compare_against_stamp(recorded_for_candidates)
unchanged: list[str] = []
modified: list[str] = []
deleted: list[str] = []
new: list[str] = []
for relative in sorted(candidates):
if relative not in old_files:
new.append(relative)
continue
status = statuses[relative]
if status == kb_state.UNCHANGED:
unchanged.append(relative)
elif status == kb_state.MODIFIED:
modified.append(relative)
else:
deleted.append(relative)
removed = sorted(set(old_files) - set(new_files))
return FileClassification(unchanged, modified, deleted, new, removed)
def _verify_sha256_sidecar(archive: Path) -> None:
"""WARN, never fail, on a missing sidecar - only a corrupted one that
*is* present is a reason to stop, per Gitea #7's design table."""
sidecar = archive.with_name(archive.name + ".sha256")
if not sidecar.is_file():
console.print(
f"[yellow]WARN[/yellow] No {sidecar.name} beside {archive.name} - the archive's "
"integrity is not being checked before it is extracted."
)
return
expected = sidecar.read_text(encoding="utf-8").strip().split()[0:1]
actual = hashlib.sha256(archive.read_bytes()).hexdigest()
if not expected or expected[0].lower() != actual.lower():
fail(
f"{archive.name} does not match {sidecar.name}: expected "
f"{expected[0] if expected else '(unreadable)'}, got {actual}. Re-download the "
"release archive rather than trusting one that failed its own checksum."
)
def _extract_single_top_level_dir(archive: Path, dest: Path) -> Path:
"""Extract `archive` into `dest` and return the one top-level directory it
contained - the shape `.gitea/workflows/release.yml` packs (see its
`Build the distribution tarball` step). Refuses anything else rather than
guessing which part is the machinery."""
with tarfile.open(archive) as tf:
names = [n for n in tf.getnames() if n not in ("", ".")]
top_levels = {n.split("/", 1)[0] for n in names}
if len(top_levels) != 1:
fail(
f"{archive.name} does not have exactly one top-level directory (found "
f"{len(top_levels)}: {', '.join(sorted(top_levels)) or '(empty archive)'}) - this "
"is not the shape a release tarball has, and `dist upgrade` refuses to guess "
"which part is the machinery."
)
return dest # unreachable: fail() raises typer.Exit
try:
tf.extractall(dest, filter="data") # noqa: S202 - trusted local archive, path-checked below
except TypeError:
# Python < 3.12 has no `filter=` argument. Same guard by hand:
# refuse any member whose extracted path would land outside dest.
resolved_dest = dest.resolve()
for member in tf.getmembers():
if not (resolved_dest / member.name).resolve().is_relative_to(resolved_dest):
fail(
f"{archive.name} contains a path that escapes the extraction directory: "
f"{member.name}"
)
return dest # unreachable
tf.extractall(dest) # noqa: S202 - every member path-checked above
return dest / next(iter(top_levels))
@contextmanager
def _resolved_source(source: Path):
"""Yield the directory holding a distribution export: `source` itself if
it already is one, or the single top-level directory of a `.tar.gz`
extracted into a scratch directory that is cleaned up afterward."""
if source.is_dir():
yield source
return
if not source.is_file():
fail(f"{source} does not exist.")
return
_verify_sha256_sidecar(source)
with tempfile.TemporaryDirectory(prefix="wikitool-upgrade-") as tmp:
yield _extract_single_top_level_dir(source, Path(tmp))
def _git_working_tree_status() -> Optional[str]:
"""`git status --porcelain` for `config.ROOT`, or None if it is not a git
repository at all - which is a valid, if unprotected, state for a tarball
instance, not a reason to refuse."""
result = subprocess.run(
["git", "-C", str(config.ROOT), "status", "--porcelain"],
capture_output=True,
text=True,
)
return result.stdout if result.returncode == 0 else None
def _report_plan(
classification: FileClassification,
migration_chain: list["kb_state.Migration"],
boundary_crossing: bool,
local_version: "version_mod.Version",
new_version: "version_mod.Version",
) -> None:
console.print(f"{local_version} -> {new_version}")
if boundary_crossing:
console.print(
f"[bold yellow]Crosses a compatibility boundary[/bold yellow] "
f"({local_version.compat_key} -> {new_version.compat_key}) - this is not a drop-in "
"swap; check the release notes before proceeding."
)
console.print(
f"{len(classification.unchanged)} unchanged, {len(classification.new)} new, "
f"{len(classification.blocked)} locally changed, {len(classification.removed)} removed "
"from the release."
)
if classification.modified:
console.print(f"[bold]Locally modified ({len(classification.modified)}):[/bold]")
for relative in classification.modified:
console.print(f" - {relative}")
if classification.deleted:
console.print(f"[bold]Locally deleted ({len(classification.deleted)}):[/bold]")
for relative in classification.deleted:
console.print(f" - {relative}")
if classification.removed:
console.print("[dim]No longer part of the release, not written or removed by default:[/dim]")
for relative in classification.removed:
console.print(f" [dim]- {relative}[/dim]")
if migration_chain:
console.print(
f"[cyan]{len(migration_chain)} migration(s) will be outstanding after this "
"upgrade, in this order:[/cyan]"
)
for position, migration in enumerate(migration_chain, start=1):
console.print(f" {position}. {migration.target} {migration.name} ({migration.kind})")
console.print("Report only - `dist upgrade` never runs a migration. See `wikitool migrate status`.")
@app.command("upgrade")
def upgrade_command(
source: Path = typer.Argument(
..., help="An extracted distribution directory, or a release .tar.gz archive"
),
dry_run: bool = typer.Option(
False, "--dry-run", help="Classify and report, without writing anything"
),
keep_local: bool = typer.Option(
False, "--keep-local",
help="Proceed even with locally changed files - leave each one untouched rather than aborting",
),
prune: bool = typer.Option(
False, "--prune",
help="Also delete files the new release no longer ships, if they are unchanged since install",
),
allow_pre: bool = typer.Option(
False, "--pre", help="Allow a pre-release (-beta.N) source tree - release.yml never publishes one",
),
):
"""Apply a stack update `dist export` produced - the write half of
`version check`. Never downloads anything: `source` is an already-fetched
export directory or `.tar.gz` archive. Writes exactly the new release
stamp's `files` block, minus what an export re-seeds every time
(`kb/log.md`, `raw/*/.gitkeep`) or seeds once and the instance owns from
then on (`.wikitool-kb.json`, `CHANGES.md`), classifying every candidate
against the *old* stamp's recorded digest: unchanged files are
overwritten silently, new files are created, and a locally modified or
deleted file is never silently overwritten - `dist upgrade` aborts unless
`--keep-local` says to leave it alone. Reports the migration chain the new
machinery would owe without running any of it (there is no `migrate run`).
Refuses on a missing local release stamp, a downgrade, a pre-release
source without `--pre`, or a dirty working tree. Never touches git.
See Gitea #7 and `INSTALL.md` § "Eine Instanz aktualisieren"."""
run_upgrade(
source, dry_run=dry_run, keep_local=keep_local, prune=prune, allow_pre=allow_pre
)
def run_upgrade(
source: Path,
dry_run: bool = False,
keep_local: bool = False,
prune: bool = False,
allow_pre: bool = False,
) -> None:
"""The upgrade itself, free of Typer's option objects - see `run_export`
for why this split exists."""
try:
local_version = version_mod.read_version()
except version_mod.VersionError as exc:
fail(f"{exc} - this tree has no stack version to upgrade from.")
return
old_stamp = version_mod.read_stamp()
if not old_stamp or not isinstance(old_stamp.get("files"), dict):
fail(
f"No local {version_mod.RELEASE_STAMP_FILENAME} (or it carries no `files` block). "
"Without it, `dist upgrade` cannot tell a file this instance edited from one it "
"merely received, and it refuses to guess. A checkout with shared git history takes "
"stack updates via `wikitool upstream merge` instead - it has the same information "
"as a merge base. A tarball instance that has lost its stamp has no repair path "
"today; see Gitea #7 \"Bewusst offen gelassen\"."
)
return
old_files = old_stamp["files"]
try:
kb_version = kb_state.read_kb_version()
except version_mod.VersionError as exc:
fail(str(exc))
return
if kb_version is None:
fail(
f"{kb_state.KB_STATE_FILENAME} is missing - this instance has never declared what "
"shape its content is in. Run `wikitool migrate baseline <version>` before upgrading."
)
return
outstanding = kb_state.chain(kb_state.load_migrations(), kb_version, local_version.base)
if outstanding:
fail(
f"{len(outstanding)} migration(s) are already outstanding against the installed "
f"machinery ({kb_version} -> {local_version}) - `wikitool migrate status` names them. "
"Finish them before upgrading further: a machinery swap on top of an unfinished "
"migration leaves the corpus in a shape no version describes."
)
return
tree_status = _git_working_tree_status()
if tree_status is None:
console.print(
"[yellow]WARN[/yellow] Not a git repository (or git is unavailable) - proceeding "
"without the dirty-tree check a repository would get."
)
elif tree_status.strip():
fail(
"Working tree is not clean (`git status --porcelain` printed something). "
"`dist upgrade` refuses to start on a dirty tree so a refusal never has to guess "
"which changes were already there. Commit or stash first."
)
return
with _resolved_source(source) as new_root:
version_path = new_root / version_mod.VERSION_FILENAME
if not version_path.is_file():
fail(f"{rel_path(new_root)} has no VERSION - not a distribution export.")
return
try:
new_version = version_mod.Version.parse(version_path.read_text(encoding="utf-8"))
except version_mod.VersionError as exc:
fail(str(exc))
return
if new_version.is_prerelease and not allow_pre:
fail(
f"{new_version} is a running candidate (-beta.N). `.gitea/workflows/release.yml` "
"never publishes one, so a candidate tree can only come from a dev checkout by "
"hand - pass --pre if that is deliberate."
)
return
if new_version < local_version:
fail(f"{new_version} is older than the installed {local_version} - refusing a downgrade.")
return
if new_version == local_version:
success(f"Already at {local_version}. Nothing to do.")
return
stamp_path = new_root / version_mod.RELEASE_STAMP_FILENAME
if not stamp_path.is_file():
fail(f"{rel_path(new_root)} has no {version_mod.RELEASE_STAMP_FILENAME} - not a distribution export.")
return
try:
new_stamp = json.loads(stamp_path.read_text(encoding="utf-8"))
except (json.JSONDecodeError, OSError) as exc:
fail(f"{version_mod.RELEASE_STAMP_FILENAME} in the source is not readable JSON: {exc}")
return
new_files = new_stamp.get("files") if isinstance(new_stamp, dict) else None
if not isinstance(new_files, dict):
fail(f"{version_mod.RELEASE_STAMP_FILENAME} in the source carries no `files` block.")
return
classification = _classify_files(old_files, new_files)
migration_chain = kb_state.chain(
kb_state.load_migrations(new_root / "instructions" / kb_state.MIGRATIONS_SUBDIR),
kb_version,
new_version.base,
)
boundary_crossing = local_version.compat_key != new_version.compat_key
_report_plan(classification, migration_chain, boundary_crossing, local_version, new_version)
# Dry-run's whole purpose is to preview this classification - including
# the blocked list - without raising, so it must be checked before the
# abort below rather than after: a blocked file must never turn
# `--dry-run` into a non-zero exit, or the flag stops being safe to run
# freely.
if dry_run:
success(f"Dry run: would upgrade {local_version} -> {new_version}. Nothing written.")
return
if classification.blocked and not keep_local:
fail(
f"{len(classification.blocked)} locally changed file(s) (listed above) would be "
"silently overwritten. Pass --keep-local to upgrade anyway and leave every one of "
"them untouched, or reconcile them by hand first. Nothing was written."
)
return
to_write = sorted(classification.unchanged + classification.new)
for relative in to_write:
src = new_root / relative
dst = config.ROOT / relative
dst.parent.mkdir(parents=True, exist_ok=True)
shutil.copy2(src, dst)
shutil.copy2(stamp_path, config.ROOT / version_mod.RELEASE_STAMP_FILENAME)
pruned: list[str] = []
if prune:
for relative in classification.removed:
digest = old_files.get(relative)
if digest is None:
continue
status = kb_state.compare_against_stamp({relative: digest}).get(relative)
if status != kb_state.UNCHANGED:
continue
target = config.ROOT / relative
if target.is_file():
target.unlink()
pruned.append(relative)
skipped = classification.blocked if keep_local else []
summary = (
f"Upgraded {local_version} -> {new_version}: {len(to_write)} file(s) written"
+ (f", {len(skipped)} left untouched (--keep-local)" if skipped else "")
+ (f", {len(pruned)} pruned" if pruned else "")
+ "."
)
if migration_chain:
summary += (
f" {len(migration_chain)} migration(s) now outstanding - run `wikitool migrate status`."
)
summary += (
" Nothing was committed. Now run, in order: `wikitool instructions sync`, `doctor`, "
"`docs verify`, `instructions verify`, `lint` - then restart the agent session."
)
success(summary)
+60 -27
View File
@@ -129,21 +129,27 @@ def migrations_dir() -> Path:
return config.INSTRUCTIONS_DIR / MIGRATIONS_SUBDIR return config.INSTRUCTIONS_DIR / MIGRATIONS_SUBDIR
def load_migrations() -> list[Migration]: def load_migrations(directory: Optional[Path] = None) -> list[Migration]:
"""Every migration document, sorted by target version. """Every migration document under `directory`, sorted by target version.
A malformed one is skipped rather than fatal here - `instructions verify` `directory` defaults to this instance's own `instructions/migrations/`.
is what reports it, and `migrate status` staying usable while one document `dist upgrade` (Gitea #7) passes the *new* tree's migrations directory
is broken is worth more than a second error path. instead: the migrations owed after an upgrade are documented in the
machinery being installed, not in the one still on disk - an old instance
cannot know a new version's migration chain by reading its own tree.
A malformed document is skipped rather than fatal here - `instructions
verify` is what reports it, and `migrate status` staying usable while one
document is broken is worth more than a second error path.
""" """
from chemenu.frontmatter_io import read_page from chemenu.frontmatter_io import read_page
directory = migrations_dir() base = directory if directory is not None else migrations_dir()
if not directory.is_dir(): if not base.is_dir():
return [] return []
migrations: list[Migration] = [] migrations: list[Migration] = []
for path in sorted(directory.glob("*.md")): for path in sorted(base.glob("*.md")):
try: try:
frontmatter, _ = read_page(path) frontmatter, _ = read_page(path)
except Exception: # noqa: BLE001 - a broken document is verify's finding, not ours except Exception: # noqa: BLE001 - a broken document is verify's finding, not ours
@@ -232,25 +238,60 @@ def next_link(
# --- what this instance changed about what it was given -------------------- # --- what this instance changed about what it was given --------------------
# What `compare_against_stamp` answers for one path: present and matching its
# recorded digest, present but not matching, or gone entirely. `dist upgrade`
# needs the three-way answer to tell a locally deleted file from a locally
# edited one; `divergent_files` (below) only ever needed the yes/no of
# "does this count as diverged", which both `MODIFIED` and `DELETED` answer
# the same way.
UNCHANGED = "unchanged"
MODIFIED = "modified"
DELETED = "deleted"
def compare_against_stamp(stamp_files: dict, root: Optional[Path] = None) -> dict[str, str]:
"""Classify every path in `stamp_files` (relative -> recorded sha256, the
shape of a release stamp's own `files` block) against what is actually on
disk under `root` - `UNCHANGED`, `MODIFIED`, or `DELETED`.
`root` defaults to `config.ROOT`. The general form `divergent_files` is
built on: that function only ever asks the question against this
instance's own tree, but `dist upgrade` (Gitea #7) asks it against an
already-installed tree while planning what to write, and a *second* time
against the tree it just wrote, before recording the new stamp - two trees
neither of which is necessarily `config.ROOT`.
"""
import hashlib
base = root if root is not None else config.ROOT
result: dict[str, str] = {}
for relative, digest in sorted(stamp_files.items()):
path = base / relative
if not path.is_file():
result[relative] = DELETED
continue
current = "sha256:" + hashlib.sha256(path.read_bytes()).hexdigest()
result[relative] = UNCHANGED if current == digest else MODIFIED
return result
def divergent_files() -> Optional[list[str]]: def divergent_files() -> Optional[list[str]]:
"""Files whose content no longer matches the release this instance installed. """Files whose content no longer matches the release this instance installed.
Reads the per-file sha256 in `.wikitool-release.json`, which `dist export` Reads the per-file sha256 in `.wikitool-release.json`, which `dist export`
has been writing since the stamp existed and which nothing has read until has been writing since the stamp existed. Its own docstring says why it is
now. Its own docstring says why it is there: it is the only way a later there: it is the only way a later upgrade can tell a file the instance
upgrade can tell a file the instance *edited* from one it merely *received*. *edited* from one it merely *received* - `dist upgrade` (Gitea #7) is that
later upgrade, built on the general `compare_against_stamp` above.
That distinction is what makes an `offered` migration actionable. The stack That distinction is also what makes an `offered` migration actionable. The
proposing a better `entity` template needs to know whether it may be copied stack proposing a better `entity` template needs to know whether it may be
over or whether the instance has its own version that a person has to copied over or whether the instance has its own version that a person has
reconcile - and only the recorded hash can answer that. to reconcile - and only the recorded hash can answer that.
Returns None when the question is unanswerable (a development tree, which Returns None when the question is unanswerable (a development tree, which
carries no stamp), which is different from `[]` (nothing diverged). carries no stamp), which is different from `[]` (nothing diverged).
""" """
import hashlib
from chemenu import version as version_mod from chemenu import version as version_mod
try: try:
@@ -263,13 +304,5 @@ def divergent_files() -> Optional[list[str]]:
if not isinstance(recorded, dict): if not isinstance(recorded, dict):
return None return None
divergent: list[str] = [] statuses = compare_against_stamp(recorded)
for relative, digest in sorted(recorded.items()): return [relative for relative, status in statuses.items() if status != UNCHANGED]
path = config.ROOT / relative
if not path.is_file():
divergent.append(relative)
continue
current = "sha256:" + hashlib.sha256(path.read_bytes()).hexdigest()
if current != digest:
divergent.append(relative)
return divergent
+23
View File
@@ -62,3 +62,26 @@ def is_export_stub(name: str) -> bool:
"""Whether `name` (a bare filename, not a path) is one `dist export` """Whether `name` (a bare filename, not a path) is one `dist export`
overwrites with a fresh stub of its own rather than shipping verbatim.""" overwrites with a fresh stub of its own rather than shipping verbatim."""
return name in EXPORT_STUB_NAMES return name in EXPORT_STUB_NAMES
# Root-relative paths `dist export` seeds once, from a template it owns, and
# which the instance owns exclusively from that point on. `dist upgrade`
# (Gitea #7) must never overwrite them, even though they sit in the release
# stamp's `files` block like any other planned file - the same shape as
# `EXPORT_STUB_NAMES` above, but keyed by full path rather than bare filename,
# since nothing else at the repo root gets this treatment and a bare-filename
# match would be too broad here.
#
# `.wikitool-kb.json` is `migrate done`'s state file: an upgrade that resets it
# declares a content shape nobody actually produced. `CHANGES.md` is the
# instance's own changelog, not the stack's - `dist export` seeds it from a
# blank template (`dist_templates/CHANGES.md`) the same way it seeds
# `kb/log.md`, and overwriting it on upgrade would erase every entry the
# instance ever wrote for itself.
UPGRADE_PRESERVED_PATHS = (".wikitool-kb.json", "CHANGES.md")
def is_upgrade_preserved(relative: str) -> bool:
"""Whether `relative` (a plan-relative path from the repo root, e.g.
"CHANGES.md") is one `dist upgrade` must never write."""
return relative in UPGRADE_PRESERVED_PATHS
+386
View File
@@ -0,0 +1,386 @@
"""Tests for `wikitool dist upgrade`: classification against the locally
installed release stamp, the write set, migration-chain reporting without
execution, and every refusal before anything is written. See Gitea #7."""
from __future__ import annotations
import hashlib
import json
import subprocess
import tarfile
from pathlib import Path
import pytest
import typer
from chemenu import config, kb_state, version as version_mod
from chemenu.commands import dist_cmd
def _digest(text: str) -> str:
return "sha256:" + hashlib.sha256(text.encode("utf-8")).hexdigest()
def _write_stamp(path: Path, version: str, files: dict[str, str]) -> None:
path.write_text(
json.dumps(
{
"schema": version_mod.STAMP_SCHEMA,
"version": version,
"exported_at": "2026-01-01",
"source_repo": None,
"source_commit": None,
"release_url": None,
"update_url": version_mod.DEFAULT_UPDATE_URL,
"files": files,
},
indent=2,
),
encoding="utf-8",
)
def write_migration(directory: Path, target: str, slug: str) -> None:
directory.mkdir(parents=True, exist_ok=True)
(directory / f"{target}-{slug}.md").write_text(
"---\n"
"type: types/instruction.md\n"
f"name: {target}-{slug}\n"
f"description: Migration to {target}.\n"
"manual: true\n"
f"migrates_to: {target}\n"
"migration_kind: assisted\n"
"obligation: required\n"
"---\n\n# Migration\n\nSteps.\n",
encoding="utf-8",
)
def set_kb_version(root: Path, version: str) -> None:
(root / kb_state.KB_STATE_FILENAME).write_text(
json.dumps({"schema": 1, "kb_version": version, "applied": []}), encoding="utf-8"
)
@pytest.fixture
def instance(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Path:
"""A minimal tarball instance: VERSION 1.0.0, a local release stamp
recording two tracked files exactly as installed, content stays at
1.0.0 with nothing outstanding, and no .git directory (the WARN path
for the dirty-tree check)."""
root = tmp_path / "instance"
root.mkdir()
monkeypatch.setattr(config, "ROOT", root)
(root / "VERSION").write_text("1.0.0\n", encoding="utf-8")
(root / "AGENTS.md").write_text("core\n", encoding="utf-8")
(root / "tools").mkdir()
(root / "tools" / "wikitool").write_text("#!/bin/sh\n", encoding="utf-8")
files = {
"AGENTS.md": _digest("core\n"),
"tools/wikitool": _digest("#!/bin/sh\n"),
}
_write_stamp(root / version_mod.RELEASE_STAMP_FILENAME, "1.0.0", files)
set_kb_version(root, "1.0.0")
return root
def _release(tmp_path: Path, name: str, version: str, files_content: dict[str, str]) -> Path:
"""A second, independent tree shaped like a `dist export` output: VERSION,
a release stamp whose `files` block matches `files_content` exactly, and
the files themselves."""
root = tmp_path / name
root.mkdir()
(root / "VERSION").write_text(f"{version}\n", encoding="utf-8")
files: dict[str, str] = {}
for relative, content in files_content.items():
path = root / relative
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(content, encoding="utf-8")
files[relative] = _digest(content)
_write_stamp(root / version_mod.RELEASE_STAMP_FILENAME, version, files)
return root
# --- classification / dry-run -----------------------------------------------
def test_dry_run_classifies_every_case_and_writes_nothing(instance, tmp_path):
(instance / "AGENTS.md").write_text("locally edited\n", encoding="utf-8")
(instance / "tools" / "wikitool").unlink()
release = _release(
tmp_path, "release", "1.1.0",
{
"AGENTS.md": "core\n", # locally modified
"tools/wikitool": "#!/bin/sh\n", # locally deleted
"types/entity.md": "new page type\n", # new in the release
},
)
dist_cmd.run_upgrade(release, dry_run=True)
assert (instance / "AGENTS.md").read_text(encoding="utf-8") == "locally edited\n"
assert not (instance / "tools" / "wikitool").exists()
assert not (instance / "types" / "entity.md").exists()
assert json.loads((instance / version_mod.RELEASE_STAMP_FILENAME).read_text())["version"] == "1.0.0"
def test_removed_file_is_reported_and_left_alone_without_prune(instance, tmp_path):
release = _release(tmp_path, "release", "1.1.0", {"AGENTS.md": "core\n"})
# tools/wikitool is in the old stamp but absent from the new one.
dist_cmd.run_upgrade(release, dry_run=True)
assert (instance / "tools" / "wikitool").is_file()
# --- local changes are never silently overwritten ---------------------------
def test_locally_modified_file_blocks_the_upgrade_by_default(instance, tmp_path):
(instance / "AGENTS.md").write_text("locally edited\n", encoding="utf-8")
release = _release(
tmp_path, "release", "1.1.0",
{"AGENTS.md": "core v2\n", "tools/wikitool": "#!/bin/sh\n"},
)
with pytest.raises(typer.Exit) as excinfo:
dist_cmd.run_upgrade(release)
assert excinfo.value.exit_code == 1
assert (instance / "AGENTS.md").read_text(encoding="utf-8") == "locally edited\n"
assert json.loads((instance / version_mod.RELEASE_STAMP_FILENAME).read_text())["version"] == "1.0.0"
def test_locally_deleted_file_blocks_the_upgrade_by_default(instance, tmp_path):
(instance / "tools" / "wikitool").unlink()
release = _release(
tmp_path, "release", "1.1.0",
{"AGENTS.md": "core\n", "tools/wikitool": "#!/bin/sh v2\n"},
)
with pytest.raises(typer.Exit):
dist_cmd.run_upgrade(release)
assert not (instance / "tools" / "wikitool").exists()
def test_keep_local_proceeds_and_leaves_the_changed_file_untouched(instance, tmp_path):
(instance / "AGENTS.md").write_text("locally edited\n", encoding="utf-8")
release = _release(
tmp_path, "release", "1.1.0",
{"AGENTS.md": "core v2\n", "tools/wikitool": "#!/bin/sh\n", "types/entity.md": "new\n"},
)
dist_cmd.run_upgrade(release, keep_local=True)
# The locally changed file is untouched...
assert (instance / "AGENTS.md").read_text(encoding="utf-8") == "locally edited\n"
# ...but everything unchanged/new was still written.
assert (instance / "tools" / "wikitool").read_text(encoding="utf-8") == "#!/bin/sh\n"
assert (instance / "types" / "entity.md").read_text(encoding="utf-8") == "new\n"
assert json.loads((instance / version_mod.RELEASE_STAMP_FILENAME).read_text())["version"] == "1.1.0"
# --- the write set -----------------------------------------------------------
def test_unchanged_and_new_files_are_written_silently(instance, tmp_path):
release = _release(
tmp_path, "release", "1.1.0",
{"AGENTS.md": "core\n", "tools/wikitool": "#!/bin/sh\n", "types/entity.md": "new\n"},
)
dist_cmd.run_upgrade(release)
assert (instance / "AGENTS.md").read_text(encoding="utf-8") == "core\n"
assert (instance / "types" / "entity.md").read_text(encoding="utf-8") == "new\n"
stamp = json.loads((instance / version_mod.RELEASE_STAMP_FILENAME).read_text())
assert stamp["version"] == "1.1.0"
assert stamp["files"]["types/entity.md"] == _digest("new\n")
@pytest.mark.parametrize("preserved", [".wikitool-kb.json", "CHANGES.md", "kb/log.md", "raw/notes/.gitkeep"])
def test_seeded_once_paths_are_never_written_even_if_the_release_stamp_lists_them(
instance, tmp_path, preserved
):
"""The write set is the new stamp's `files` block minus what an export
re-seeds every time or seeds once and the instance owns from then on -
this is the AGENTS.md invariant 8 test: no separate literal list here,
only `chemenu.ownership`."""
(instance / "CHANGES.md").write_text("instance's own changelog\n", encoding="utf-8")
(instance / kb_state.KB_STATE_FILENAME).write_text(
json.dumps({"schema": 1, "kb_version": "1.0.0", "applied": [{"migration": "x"}]}),
encoding="utf-8",
)
release = _release(
tmp_path, "release", "1.1.0",
{
"AGENTS.md": "core\n",
"tools/wikitool": "#!/bin/sh\n",
preserved: "a fresh stub from the new release\n",
},
)
dist_cmd.run_upgrade(release)
if preserved == "CHANGES.md":
assert (instance / preserved).read_text(encoding="utf-8") == "instance's own changelog\n"
elif preserved == ".wikitool-kb.json":
state = json.loads((instance / preserved).read_text())
assert state["applied"] == [{"migration": "x"}]
else:
assert not (instance / preserved).exists()
# --- migration chain: reported, never run -----------------------------------
def test_migration_chain_is_reported_but_never_executed(instance, tmp_path):
release = _release(
tmp_path, "release", "1.1.0",
{"AGENTS.md": "core\n", "tools/wikitool": "#!/bin/sh\n"},
)
write_migration(release / "instructions" / "migrations", "1.1.0", "some-change")
dist_cmd.run_upgrade(release)
# The chain is reported, not applied: kb_version has not moved.
assert kb_state.read_kb_version().base == version_mod.Version(1, 0, 0)
def test_outstanding_local_migration_blocks_before_touching_the_source(instance, tmp_path):
"""A migration owed against the *installed* machinery must be finished
first - the source is never even opened."""
migrations = instance / "instructions" / "migrations"
write_migration(migrations, "1.0.0", "not-yet-done")
monkey_target = instance / "does-not-exist" # never read if this check fires first
with pytest.raises(typer.Exit):
dist_cmd.run_upgrade(monkey_target)
# --- preconditions -----------------------------------------------------------
def test_missing_local_stamp_blocks(instance, tmp_path):
(instance / version_mod.RELEASE_STAMP_FILENAME).unlink()
release = _release(tmp_path, "release", "1.1.0", {"AGENTS.md": "core\n"})
with pytest.raises(typer.Exit):
dist_cmd.run_upgrade(release)
def test_missing_kb_version_blocks(instance, tmp_path):
(instance / kb_state.KB_STATE_FILENAME).unlink()
release = _release(tmp_path, "release", "1.1.0", {"AGENTS.md": "core\n"})
with pytest.raises(typer.Exit):
dist_cmd.run_upgrade(release)
def test_prerelease_source_is_refused_without_pre_flag(instance, tmp_path):
release = _release(tmp_path, "release", "1.1.0-beta.1", {"AGENTS.md": "core\n"})
with pytest.raises(typer.Exit):
dist_cmd.run_upgrade(release)
assert json.loads((instance / version_mod.RELEASE_STAMP_FILENAME).read_text())["version"] == "1.0.0"
def test_pre_flag_allows_a_prerelease_source(instance, tmp_path):
release = _release(
tmp_path, "release", "1.1.0-beta.1",
{"AGENTS.md": "core\n", "tools/wikitool": "#!/bin/sh\n"},
)
dist_cmd.run_upgrade(release, allow_pre=True)
assert json.loads((instance / version_mod.RELEASE_STAMP_FILENAME).read_text())["version"] == "1.1.0-beta.1"
def test_downgrade_is_refused(instance, tmp_path):
release = _release(tmp_path, "release", "0.9.0", {"AGENTS.md": "core\n"})
with pytest.raises(typer.Exit):
dist_cmd.run_upgrade(release)
def test_equal_version_is_a_noop(instance, tmp_path):
release = _release(tmp_path, "release", "1.0.0", {"AGENTS.md": "core\n"})
dist_cmd.run_upgrade(release) # must not raise
assert json.loads((instance / version_mod.RELEASE_STAMP_FILENAME).read_text())["version"] == "1.0.0"
def test_dirty_working_tree_blocks(instance, tmp_path):
subprocess.run(["git", "init", "-q", "-b", "main"], cwd=instance, check=True)
subprocess.run(["git", "config", "user.name", "Fixture Author"], cwd=instance, check=True)
subprocess.run(["git", "config", "user.email", "fixture@example.invalid"], cwd=instance, check=True)
(instance / "untracked.txt").write_text("dirty\n", encoding="utf-8")
release = _release(tmp_path, "release", "1.1.0", {"AGENTS.md": "core\n"})
with pytest.raises(typer.Exit):
dist_cmd.run_upgrade(release)
assert not (instance / "types").exists()
def test_source_that_does_not_exist_is_refused(instance, tmp_path):
with pytest.raises(typer.Exit):
dist_cmd.run_upgrade(tmp_path / "nowhere")
# --- prune -------------------------------------------------------------------
def test_prune_removes_only_removed_files_still_unchanged_since_install(instance, tmp_path):
(instance / "extra.txt").write_text("shipped once, edited since\n", encoding="utf-8")
old_stamp = json.loads((instance / version_mod.RELEASE_STAMP_FILENAME).read_text())
old_stamp["files"]["extra.txt"] = _digest("shipped once, unedited\n") # deliberately stale
old_stamp["files"]["gone.txt"] = _digest("also shipped once\n")
(instance / version_mod.RELEASE_STAMP_FILENAME).write_text(json.dumps(old_stamp), encoding="utf-8")
(instance / "gone.txt").write_text("also shipped once\n", encoding="utf-8")
release = _release(
tmp_path, "release", "1.1.0",
{"AGENTS.md": "core\n", "tools/wikitool": "#!/bin/sh\n"},
)
dist_cmd.run_upgrade(release, prune=True)
# gone.txt matched its recorded digest -> pruned.
assert not (instance / "gone.txt").exists()
# extra.txt was locally edited relative to its recorded digest -> kept.
assert (instance / "extra.txt").read_text(encoding="utf-8") == "shipped once, edited since\n"
# --- tarball sources ----------------------------------------------------------
def _pack(release_dir: Path, archive: Path) -> None:
with tarfile.open(archive, "w:gz") as tf:
tf.add(release_dir, arcname=release_dir.name)
def test_tarball_with_more_than_one_top_level_entry_is_refused(instance, tmp_path):
scratch = tmp_path / "scratch"
(scratch / "a").mkdir(parents=True)
(scratch / "b").mkdir(parents=True)
(scratch / "a" / "x.txt").write_text("x\n", encoding="utf-8")
(scratch / "b" / "y.txt").write_text("y\n", encoding="utf-8")
archive = tmp_path / "bad.tar.gz"
with tarfile.open(archive, "w:gz") as tf:
tf.add(scratch / "a", arcname="a")
tf.add(scratch / "b", arcname="b")
with pytest.raises(typer.Exit):
dist_cmd.run_upgrade(archive)
def test_tarball_source_is_extracted_and_applied(instance, tmp_path):
release_dir = _release(
tmp_path, "chemenu-stack-1.1.0", "1.1.0",
{"AGENTS.md": "core\n", "tools/wikitool": "#!/bin/sh\n"},
)
archive = tmp_path / "chemenu-stack-1.1.0.tar.gz"
_pack(release_dir, archive)
dist_cmd.run_upgrade(archive)
assert (instance / "AGENTS.md").read_text(encoding="utf-8") == "core\n"
assert json.loads((instance / version_mod.RELEASE_STAMP_FILENAME).read_text())["version"] == "1.1.0"
def test_tarball_sha256_sidecar_mismatch_is_refused(instance, tmp_path):
release_dir = _release(
tmp_path, "chemenu-stack-1.1.0", "1.1.0", {"AGENTS.md": "core\n"}
)
archive = tmp_path / "chemenu-stack-1.1.0.tar.gz"
_pack(release_dir, archive)
(archive.with_name(archive.name + ".sha256")).write_text(
"0" * 64 + " chemenu-stack-1.1.0.tar.gz\n", encoding="utf-8"
)
with pytest.raises(typer.Exit):
dist_cmd.run_upgrade(archive)