Compare commits

...

9 Commits

Author SHA1 Message Date
torben 87a47cc237 instructions/dev/issue-tracking.md: destructive-step invariants, comment-vs-body authority, rename sweep (#47 Block 1, #29)
CI / verify (push) Successful in 56s
Release / release (push) Successful in 35s
Files changed:
- CHANGES.md
- VERSION
- instructions/dev/issue-tracking.md
2026-09-04 14:59:30 +02:00
torben d4cbeca5a8 docs: INSTALL.md - erster Sprung auf 4.5.0 nutzt das Werkzeug aus dem neuen Tarball (#7)
CI / verify (push) Successful in 55s
Files changed:
- INSTALL.md
2026-09-04 12:52:43 +02:00
torben 0ba94c63d1 release: 4.5.0 - beide Update-Wege in Code (upstream merge, dist upgrade)
CI / verify (push) Successful in 53s
Release / release (push) Successful in 37s
Files changed:
- CHANGES.md
- VERSION
2026-09-04 12:48:05 +02:00
torben 368438e48c docs: dist upgrade - Stamp-Semantik nach --keep-local, drei Eigentumsklassen in docs/ (#7)
CI / verify (push) Successful in 54s
Release / release (push) Successful in 34s
Files changed:
- CHANGES.md
- VERSION
- docs/ownership-and-templates.md
- tools/CONTRACT.md
2026-09-04 12:41:50 +02:00
torben cd81ba3d4f feat: dist upgrade - apply a stack update, not just detect one (#7)
CI / verify (push) Successful in 54s
Release / release (push) Successful in 34s
Files changed:
- CHANGES.md
- INSTALL.md
- VERSION
- tools/CONTRACT.md
- tools/chemenu/commands/dist_cmd.py
- tools/chemenu/kb_state.py
- tools/chemenu/ownership.py
- tools/chemenu/tests/test_dist_upgrade.py
2026-09-04 11:36:21 +02:00
torben 1b5ffea854 fix: upstream merge - preserve gitignored local data, refuse a merge git never opened (4.5.0-beta.3, #30)
CI / verify (push) Successful in 57s
Release / release (push) Successful in 35s
Files changed:
- CHANGES.md
- VERSION
- docs/ownership-and-templates.md
- instructions/gates.md
- instructions/private-instance.md
- tools/CONTRACT.md
- tools/chemenu/commands/upstream_cmd.py
- tools/chemenu/tests/test_upstream_cmd.py
2026-09-04 07:35:00 +02:00
torben d2b1719a4b test: upstream merge - combined-commit regression covering the acceptance checklist (4.5.0-beta.2, #30)
CI / verify (push) Successful in 58s
Release / release (push) Successful in 35s
Files changed:
- CHANGES.md
- VERSION
- tools/chemenu/tests/test_upstream_cmd.py
2026-09-04 07:11:20 +02:00
torben 686c08bb14 feat: wikitool upstream merge/verify - code procedure for taking a stack update (4.5.0-beta.1, #30)
CI / verify (push) Successful in 57s
Release / release (push) Successful in 35s
Files changed:
- AGENTS.md
- CHANGES.md
- VERSION
- instructions/gates.md
- instructions/private-instance.md
- tools/CONTRACT.md
- tools/README.md
- tools/chemenu/cli.py
- tools/chemenu/commands/dist_cmd.py
- tools/chemenu/commands/run_budget.py
- tools/chemenu/commands/upstream_cmd.py
- tools/chemenu/ownership.py
- tools/chemenu/tests/test_upstream_cmd.py
2026-09-04 07:08:49 +02:00
torben abe5497cda docs: Faktenkorrekturen in der 4.4.0-Prosa; DEVELOPMENT.md ohne zweite Kommandobeschreibung (4.4.1-beta.1, #47)
CI / verify (push) Successful in 49s
Release / release (push) Successful in 34s
Files changed:
- CHANGES.md
- DEVELOPMENT.md
- VERSION
- docs/version-model.md
- instructions/dev/version-parts.md
2026-09-03 22:48:09 +02:00
21 changed files with 2398 additions and 169 deletions
+2 -1
View File
@@ -238,7 +238,8 @@ Every `tools/wikitool` call has exactly four outcomes:
produced. produced.
After the single allowed retry - or immediately, for the non-idempotent commands `new`, After the single allowed retry - or immediately, for the non-idempotent commands `new`,
`log append`, and `publish` - stop and report the exact command and error text to the user. `log append`, `publish`, and `upstream merge` - stop and report the exact command and error
text to the user.
Per-command detail (what exit 1 means, whether the command is atomic, whether a retry is Per-command detail (what exit 1 means, whether the command is atomic, whether a retry is
safe) is in [tools/CONTRACT.md](tools/CONTRACT.md). A gate refusal is not a validation error - safe) is in [tools/CONTRACT.md](tools/CONTRACT.md). A gate refusal is not a validation error -
+257 -6
View File
@@ -35,6 +35,255 @@ dev-checkout concern - readable here, never shipped as something to parse.
--- ---
## 4.5.1 - 2026-09-04 - issue-tracking - destructive-step invariants, comment-vs-body authority, rename sweep
**Author:** Torben Nehmer
<!-- wikitool:bumps -->
- issue-tracking - destructive-step invariants, comment-vs-body authority, rename sweep
<!-- /wikitool:bumps -->
Block 1 aus #47 (gemeinsam mit #29): drei Ergänzungen an
`instructions/dev/issue-tracking.md`, ausgelöst durch zwei Fehlerklassen, die
in derselben Sitzung am Stack aufgetreten waren.
- **Schritt 1** trägt jetzt, dass destruktive Schritte im Body die Invariante
nennen müssen, die sie nicht verletzen dürfen, und dass ein
Akzeptanzkriterium eine prüfbare Eigenschaft ist, keine Tätigkeit. Auslöser
war #30: der Body schrieb wörtlich "Arbeitsverzeichnis entfernen" für den
`upstream merge`-Ablauf, und genau das wurde zum datenvernichtenden Bug -
ein `shutil.rmtree` auf eine Stage mit gitignorierten, nicht
rekonstruierbaren Daten.
- **Schritt 2** trägt jetzt die Lesesicht auf Body und Kommentare, die es
bisher nur aus Autorensicht gab: der Body ist der Stand, Kommentare sind
Historie; ein erkennbar veralteter Body wird richtiggestellt statt
umgangen; widersprüchliche Kommentare werden nach Beleg aufgelöst, nicht
nach Datum. Auslöser war ebenfalls #30 (ein Kommentar empfahl das Gegenteil
dessen, was der Body später festlegte) und #10 (ein seit Tagen veralteter
Body gegen drei widersprechende Kommentare, zwei davon sich selbst
widersprechend).
- Neuer Abschnitt **"Renames and other decay in the tracker"** nach Schritt 7:
ein Rename ist erst fertig, wenn auch die offenen Issues nachgezogen sind,
weil `wikitool` diesen Tracker nicht kennt und nicht kennenlernen soll. Mit
der Wegweiser-vs-Beleg-Unterscheidung aus #29 und dem Hinweis, dass auch
verschwundene `kb/`-Seiten und private Infrastrukturangaben Issue-Texte
altern lassen. Ein neuer Trigger in "When to run" verweist darauf.
Keine der drei Ergänzungen verschiebt die bestehende Nummerierung der
Schritte 1-7 - die Querverweise darauf (u. a. aus
`instructions/dev/stack-dev/SKILL.md` auf Schritt 7, aus
`kb/concepts/Issue Label Scheme.md` auf Schritt 2) bleiben also gültig, ohne
angefasst zu werden.
Verifiziert: `tools/wikitool instructions verify`, `tools/wikitool docs
verify`, beide grün (Prosa-only, kein Interface geändert, PATCH).
#47 bleibt offen (Block 2: der Skill-Schnitt aus Vorschlag E; Block 3: die
beiden Nebenbefunde). #29 bleibt ebenfalls offen: dieser Block deckte nur den
Regelabsatz, nicht den noch ausstehenden Pfad-Durchgang durch #4, #5, #15,
#21, #16, #26 - der war nicht Teil des Auftrags für diesen Block.
---
## 4.5.0 - 2026-09-04 - Beide Update-Wege in Code: upstream merge fuer Clones, dist upgrade fuer Tarball-Instanzen
**Author:** Torben Nehmer
<!-- wikitool:bumps -->
- Faktenkorrekturen in der 4.4.0-Prosa; DEVELOPMENT.md ohne zweite Kommandobeschreibung
- wikitool upstream merge/verify: code procedure for taking a stack update, ownership.py as the shared stack/instance boundary
- upstream merge: combined-commit regression test (edit+add+delete+contract+template+contract-delete in one commit)
- upstream merge: keep gitignored local data under a content stage, refuse a merge git never opened, report what actually changed
- dist upgrade: apply a stack update, not just detect one (#7)
- dist upgrade: Stamp-Semantik nach --keep-local benannt, docs/ownership-and-templates.md auf drei Eigentumsklassen nachgezogen
<!-- /wikitool:bumps -->
Die Prosa zu 4.4.0 - Changelog-Eintrag, `docs/version-model.md`, `instructions/dev/version-parts.md`,
`DEVELOPMENT.md` - entstand ungeprüft: kein maschineller Check liest sie, und die Sitzung, die sie
schrieb, hat den dafür vorgesehenen Modellwechsel aus `stack-dev` Schritt 6 übersprungen. Ein
nachgeholter Durchgang hat drei Fehler gefunden, von denen der erste der teuerste war.
**Der Befund war an einer Stelle schlicht falsch.** Changelog und `docs/version-model.md`
behaupteten, unter dem alten Modell seien Nummern vergeben worden, die „nie ausgeliefert" wurden -
im Changelog sogar mit einer erfundenen Zahl („fünf Minor-Bumps ... vier nie ausgeliefert"). Das
Gegenteil stimmt: weil `release.yml` auf jede `VERSION`-Bewegung feuerte, wurde **jeder** Bump zu
einem echten, getaggten Release. Der 2026-09-03 hat vier davon in sechs Stunden erzeugt (`4.3.0`
bis `4.3.3`), zwei für reine Prosa-Änderungen. Der Schaden war nie ein Phantom-Release, sondern
dass „Release" aufhörte, etwas zu bedeuten - vier Upgrades an einem Nachmittag sind für einen
Konsumenten kein kleineres Versprechen, sondern ein unleserliches. Beide Stellen tragen jetzt den
tatsächlichen Vorgang samt Beleg.
Dazu zwei kleinere Korrekturen: `version-parts.md` nannte den Stack weiterhin `2.x` (er steht bei
`4.x`), und `DEVELOPMENT.md` trug eine Tabelle, die für jeden Verify-Befehl ein zweites Mal
beschrieb, was er prüft - eine Kopie dessen, was `tools/CONTRACT.md` hält und `docs verify` dort
gegen die CLI prüft. Die Tabelle ist raus; dass die Datei selbst außerhalb der von diesem Check
abgedeckten Dokumente liegt, steht jetzt an ihrer Stelle. Der Code-seitige Teil davon - ob
`DEVELOPMENT.md` in `docs_verify.STAGE_READMES` gehört - hängt an #47, zusammen mit der Lücke im
Skill, die den übersprungenen Modellwechsel überhaupt erst unauffällig gemacht hat.
Kein Verhalten geändert, nur Prosa - und bewusst als laufender Kandidat gelassen statt fixiert:
CIs Version-Gate verlangt die `VERSION`-Bewegung, ein Release verlangt sie nicht. Genau dafür gibt
es das Modell aus 4.4.0.
**Zweiter Bump auf demselben Kandidaten (#30):** `git merge upstream/main` behandelt einen
bewegten Korpus asymmetrisch - eine gelöschte, upstream-geänderte Seite meldet sich als Konflikt,
eine neu angelegte Seite wird still gestaged, nur eine beidseitig gelöschte Seite ist harmlos. Die
Prosa-Prozedur in `private-instance.md` § "Taking a stack update" hat das geschlossen, aber mit
vier eigenen Fehlern: der Pfadsatz stand dreifach (im Skript, im Kontroll-Grep, implizit in
`dist_cmd.py`); eine vom Upstream **gelöschte** Maschinerie-Datei wurde von `git checkout
MERGE_HEAD -- <pfad>` still ignoriert, weil das Skript kein `set -e` hatte; ein echter Konflikt in
`tools/`/`types/`/`instructions/` endete in einem von der Prosa nirgends erwähnten offenen Merge;
und ein *neuer* Maschinerie-Pfad unter einer Content-Stage hätte die Literal-Liste nie erreicht.
Die Eigentumsgrenze ist jetzt ein Prädikat statt einer Liste: `chemenu/ownership.py`,
`is_stack_owned(relative)`, wahr für `<stage>/CONTRACT.md` direkt unter einer Content-Stage
(`kb`, `raw`, `work`, `reports`) und für jeden Pfad, der dort auf `.template` endet - nicht
rekursiv, `kb/<collection>/COLLECTION.md` bleibt seit #39 instanzeigen. `dist_cmd.py` liest das
Modul jetzt statt einer eigenen `_CONTENT_ALLOWED_NAMES`-Liste zu pflegen, und
`CONTRACT_ONLY_STAGES` leitet sich aus `ownership.CONTENT_STAGES` ab statt die drei Stage-Pfade
ein zweites Mal aufzuschreiben - ein Test hält fest, dass beide Sichten nicht auseinanderlaufen
können.
Neu: `wikitool upstream merge [--remote upstream] [--branch main] [--no-fetch]` und `wikitool
upstream verify --since <rev> [--until HEAD]`. `merge` prüft Vorbedingungen (sauberer Baum, kein
laufendes Merge, Remote löst auf), warnt statt zu blockieren, wenn `.wikitool-remotes.json` fehlt,
hält den Merge offen (`--no-commit --no-ff`), zwingt jede Content-Stage auf die lokale Seite
zurück, holt dann über die Vereinigungsmenge der Bäume von `MERGE_HEAD` und `HEAD` genau die
stack-eigenen Pfade zurück - inklusive einer Löschung, falls der Upstream einen Maschinerie-Pfad
entfernt hat -, verweigert bei verbliebenen unaufgelösten Pfaden ohne zu committen, committet
sonst und verifiziert den entstandenen Commit mit derselben Logik wie `verify` - ein Fund dort
wird laut gemeldet und **nicht** automatisch zurückgerollt. Nicht idempotent (AGENTS.md § Tool
error contract), nicht budget-exempt; `verify` liest nur und ist wie `migrate verify` von der
Budget-Gate ausgenommen. Die Mass-Update-Gate greift bei einem Merge-Commit strukturell nicht -
das steht jetzt als eigener Absatz in `instructions/gates.md`, mit `upstream merge`s eigener
Nachkontrolle als der Sicherung, die hier tatsächlich trägt.
`private-instance.md` § "Taking a stack update" verweist jetzt auf den Befehl statt das Skript
auszuschreiben; die Pfadtabelle bleibt als Erklärung stehen, ist aber nicht mehr die operative
Liste. Vorschlag B (eigenes Demo-Repo) bleibt zurückgestellt, siehe #30.
23 neue Tests unter `test_upstream_cmd.py` (35 Fälle mit der parametrisierten
`is_stack_owned`-Tabelle) decken die Fälle aus der Spezifikation ab:
gelöschte vs. geänderte vs. neu angelegte Seiten, `kb/CONTRACT.md`- und Template-Änderungen,
`kb/entities/COLLECTION.md` bleibt lokal, eine gelöschte `raw/CONTRACT.md` landet, ein neuer
Template-Pfad landet, ein offener `work/`-Lauf landet nicht, ein echter `tools/`-Konflikt lässt
den Merge offen, ein schmutziger Baum wird unberührt abgewiesen, "bereits aktuell" ist ein No-op,
die Publish-Remote-Gate-Warnung, `verify` gegen einen von Hand verpfuschten Merge, und die
`dist_cmd`/`ownership`-Konsistenz.
**Dritter Bump auf demselben Kandidaten:** die im Akzeptanzkriterium geforderte Kombinationsprobe
fehlte noch - ein einzelner Upstream-Commit, der Editieren, Anlegen, Löschen einer Seite, eine
Contract-Änderung, eine Template-Änderung und eine Contract-Löschung gleichzeitig bewegt. Jetzt
als `test_one_upstream_commit_mixing_every_case_at_once` nachgetragen; alle sechs Erwartungen in
einem `upstream merge`-Aufruf verifiziert.
**Vierter Bump: zwei Fehler, die ein Review-Durchgang nach dem Publish gefunden hat.** Beide
waren in der ersten Fassung enthalten, beide hätten Daten vernichtet, und keiner der bestehenden
Tests hat sie berührt.
*Erstens: die Content-Stage wurde als Ganzes gelöscht.* `_restore_stage_to_local` hieß in der
ersten Fassung `shutil.rmtree(stage_dir)` — die wörtliche Übersetzung des `rm -rf kb raw` aus der
Prosa-Prozedur. Für `kb/` und `raw/` ist das harmlos, weil dort nichts Ignoriertes liegt. Für die
beiden Stages, die dieses Issue *neu* in den Satz aufgenommen hat, ist es das nicht: `reports/`
ist bis auf seinen Contract komplett gitignored und trägt genau die Daten, die nirgends sonst
existieren — die Telemetrie-Traces, aus denen `eval score` liest, gespeicherte Eval-Berichte,
alte Lint-Reports. In dieser Instanz standen zum Zeitpunkt des Fundes 497 Trace-Verzeichnisse
unter `reports/telemetry/`; ein einziger `upstream merge` hätte sie alle gelöscht, und zwar
stillschweigend, weil git von ignorierten Dateien nichts meldet. Die Stage wird jetzt über die
**getrackten** Pfade beider Bäume zurückgesetzt statt über das Verzeichnis; ignorierte lokale
Daten bleiben unberührt. Leergewordene Verzeichnisse werden aufgeräumt, aber nur wirklich leere.
*Zweitens: ein Merge, den git nie eröffnet hat, hätte die Maschinerie gelöscht.* Der Exit-Code
von `git merge --no-commit --no-ff` wird bewusst ignoriert (Konflikte unter den Content-Stages
sind erwartet). Nur: wenn git das Merge gar nicht erst eröffnet — unverwandte Historien, eine
ignorierte Datei im Weg —, gibt es kein `MERGE_HEAD`, `_tree_paths("MERGE_HEAD")` liefert die
leere Menge, und **jeder** stack-eigene Pfad in `HEAD` fällt damit in den Zweig „der Upstream hat
ihn gelöscht": `kb/CONTRACT.md`, `raw/CONTRACT.md` und sämtliche Templates werden entfernt. Der
Kommando-Ablauf prüft jetzt nach dem Merge-Aufruf, dass tatsächlich ein Merge offen ist, und
bricht sonst ab, ohne den Baum angefasst zu haben. Beide Fehler haben je einen Regressionstest,
und beide Tests wurden gegen die alte Fassung laufen gelassen, um zu zeigen, dass sie sie
tatsächlich fangen.
Dazu eine Ehrlichkeitskorrektur an der Erfolgsmeldung: sie zählte die *wiederhergestellten*
Pfade, nicht die geänderten — ein Merge, der eine Datei bewegt, meldete vier oder fünf. Sie fragt
jetzt `git diff` zwischen Vor- und Nach-Commit, kennzeichnet Löschungen, und stimmt damit mit dem
überein, was ein Leser nachprüfen würde. `docs/ownership-and-templates.md` hat einen Abschnitt
bekommen, warum die Grenze ein Prädikat und keine Liste ist — die Begründung, die dieses Issue
erarbeitet hat, gehörte in die Hintergrunddoku und nicht nur in einen Changelog-Eintrag.
**Fünfter Bump: `wikitool dist upgrade` (#7), der zweite der beiden Update-Wege.** `upstream
merge` oben bedient eine Instanz mit gemeinsamer Git-History; `dist upgrade` bedient eine
Instanz aus einem Tarball, ohne History, die bislang eine rein manuelle Prozedur in
`INSTALL.md` durchlaufen musste - Schritt 4 verlangte einen sha256-Vergleich von Hand gegen den
`files`-Block der alten `.wikitool-release.json`.
Die tragende Regel: die Schreibmenge ist genau der `files`-Block der *neuen*
`.wikitool-release.json`, minus was ein Export aus einer leeren Vorlage neu sät
(`chemenu.ownership.is_export_stub`, wie bisher schon für `kb/log.md`/`.gitkeep`) oder einmalig
sät und danach der Instanz gehört (`chemenu.ownership.is_upgrade_preserved`, neu für
`.wikitool-kb.json` und `CHANGES.md`), plus der Stamp selbst. Jeder Kandidatpfad wird gegen die
*alte* Instanz-Summe klassifiziert: unverändert wird geräuschlos überschrieben, neu im Release
wird angelegt, lokal verändert oder gelöscht wird **nie** still überschrieben - der Lauf bricht
mit der vollständigen Liste ab, außer `--keep-local` sagt ausdrücklich, dass die Dateien liegen
bleiben sollen. `--prune` entfernt zusätzlich aus dem Release entfallene Dateien, aber nur
solche, die seit der Installation unverändert sind.
Die Migrationskette nach dem Tausch wird aus den `instructions/migrations/` des *neuen* Baums
ermittelt (`kb_state.load_migrations` bekam dafür einen `directory`-Parameter) und nur
gemeldet, nie ausgeführt - es gibt bewusst kein `migrate run`. Eine bereits gegen die
*installierte* Maschinerie offene Kette lässt den Befehl abbrechen, bevor er die Quelle
überhaupt öffnet. `kb_state.divergent_files()` (bisher nur von `migrate status` gelesen) ist
jetzt eine dünne Hülle um das neue, zwei-Baum-fähige `compare_against_stamp()` - gleiches
Verhalten für den bestehenden Aufrufer, wiederverwendbar für `dist upgrade`s eigenen Vergleich.
Quelle ist immer ein bereits vorhandenes Verzeichnis oder `.tar.gz` - kein Download, das bleibt
allein `version check`s Sache. Ein Tarball muss genau ein Top-Level-Verzeichnis enthalten (die
Form, in der `release.yml` es baut) und wird gegen eine `.sha256`-Beidatei geprüft, falls eine
danebenliegt (fehlt sie: WARN, kein Abbruch). Weitere Abbruchgründe vor jedem Schreiben: fehlende
lokale `VERSION`/`.wikitool-kb.json`/Stamp mit `files`-Block, ein schmutziger Arbeitsbaum (kein
Git-Repo ist ein WARN, keine Sperre), eine Vorab-Version (`-beta.N`) ohne `--pre`, sowie ein
Downgrade; Gleichstand ist ein No-op. Ein Grenzübertritt der Kompatibilität wird laut gemeldet,
blockiert aber nicht. Committet und pusht nichts (Invariante 5).
Gegenüber dem ersten Entwurf des Issues zwei Korrekturen, die dort auch nachgetragen sind: der
`files`-Block wurde entgegen der ursprünglichen Annahme bereits vor diesem Bump gelesen
(`divergent_files`/`migrate status`), und die Migrationskette war ursprünglich falsch begründet
- sie kann nur aus dem *neuen* Baum kommen, nicht durch eine andere Abfragereihenfolge aus der
alten Instanz. 24 neue Tests unter `test_dist_upgrade.py` decken die Klassifikation, alle
Abbruchgründe, `--keep-local`, `--prune` und beide Quellformen (Verzeichnis und Tarball,
inklusive der sha256- und Top-Level-Prüfung) ab.
Bewusst nicht angetastet: `instructions/private-instance.md` (der Clone-Weg ändert sich nicht,
`INSTALL.md` benennt jetzt beide Wege nebeneinander) und die Frage, wie `dist upgrade` mit
Collection-Templates umgeht, deren Namen eine fremde Instanz gar nicht hat - es verhält sich wie
`upstream merge` und schreibt sie, was ein eigenes Issue gegen den Export wäre, keins gegen das
Upgrade.
**Sechster Bump: die ungeprüfte Phase nachgeholt.** Der Abschluss des vorigen Bumps lief auf
Sonnet, und `stack-dev` Schritt 6 verlangt dort genau zwei Dinge, die kein Check erzwingt:
Issue-Body und `docs/`-Veralterung. Der Body war gemacht, die `docs/`-Prüfung nicht - sie wurde
benannt statt durchgeführt. Nachgeholt auf Opus, mit einem Fund.
`docs/ownership-and-templates.md` § „The consequence in practice" beschrieb ein Upgrade als
Zweiteilung: verbatim überschreiben, `.template`-gestützte Dateien liegen lassen - und begründete
den ersten Teil damit, dass verbatim ausgelieferte Dateien „safe to replace wholesale" seien,
weil sie „never instance-specific to begin with" waren. Genau diese Annahme trifft `dist upgrade`
nicht: eine Instanz *kann* eine verbatim ausgelieferte Datei angefasst haben, und die sha256 je
Datei existiert, um das zu erkennen, statt es vorauszusetzen. Dazu fehlte die dritte Klasse ganz -
die einmalig gesäten, danach instanzeigenen Pfade (`.wikitool-kb.json`, `CHANGES.md`,
`kb/log.md`, `raw/*/.gitkeep`), die im Stamp stehen wie jede andere Datei und deshalb aktiv
ausgeschlossen werden müssen. Die Seite nennt jetzt drei Klassen und die engere praktische Regel:
überschreibe die verbatim ausgelieferten Dateien, *die diese Instanz nicht angefasst hat*.
Dazu eine Präzisierung in `tools/CONTRACT.md`, die vorher nirgends stand: nach `--keep-local`
wird der neue Stamp trotzdem vollständig geschrieben, trägt also die Release-Summe auch für
Dateien, die bewusst *nicht* geschrieben wurden. Der Stamp ist die Vergleichsbasis für den
nächsten Lauf, kein wörtliches Inventar der Platte - und genau das hält eine übersprungene Datei
bei jedem weiteren Lauf als abweichend gemeldet, statt sie nach einmaligem Überspringen still als
aktuell zu führen.
---
## 4.4.0 - 2026-09-03 - Versionskandidat statt Bump-pro-Release: VERSION traegt -beta.N, version release fixiert ## 4.4.0 - 2026-09-03 - Versionskandidat statt Bump-pro-Release: VERSION traegt -beta.N, version release fixiert
**Author:** Torben Nehmer **Author:** Torben Nehmer
@@ -43,12 +292,14 @@ dev-checkout concern - readable here, never shipped as something to parse.
- Versionskandidat statt Bump-pro-Release: VERSION traegt -beta.N, version release fixiert - Versionskandidat statt Bump-pro-Release: VERSION traegt -beta.N, version release fixiert
<!-- /wikitool:bumps --> <!-- /wikitool:bumps -->
Bisher bekam jeder `version bump` sofort eine fixierte, dauerhafte Nummer, unabhängig davon, ob Bisher bekam jeder `version bump` sofort eine fixierte Nummer, und weil `release.yml` auf jede
je ein Release dazu erschien - fünf Minor-Bumps ohne Release ergaben fünf Nummern, von denen vier `VERSION`-Bewegung feuert, wurde daraus sofort ein Release: Nummern entstanden in
nie ausgeliefert wurden. `VERSION` trägt jetzt zwischen zwei Releases **einen** laufenden Commit-Granularität statt in Release-Granularität. Der 2026-09-03 hat so vier Releases in sechs
Kandidaten (`X.Y.Z-beta.N`) statt einer neuen Nummer pro Bump: `--major/--minor/--patch` Stunden erzeugt (`4.3.0` bis `4.3.3`), zwei davon für reine Prosa-Änderungen - alle vier echt,
eskaliert diesen Kandidaten max-wins gegen den letzten Release, statt daneben eine neue Nummer zu keines davon eine Einheit, an der ein Konsument sich hätte orientieren können. `VERSION` trägt
schreiben, und schrittet dabei nie zurück. jetzt zwischen zwei Releases **einen** laufenden Kandidaten (`X.Y.Z-beta.N`):
`--major/--minor/--patch` eskaliert diesen Kandidaten max-wins gegen den letzten Release, statt
eine neue Nummer danebenzustellen, und geht dabei nie zurück.
`Version` versteht den Suffix, mit einer expliziten Ordnung `Version` versteht den Suffix, mit einer expliziten Ordnung
(`4.4.0-beta.1 < 4.4.0-beta.2 < 4.4.0`, numerisch nach `N`, nicht lexikografisch). `CHANGES.md` (`4.4.0-beta.1 < 4.4.0-beta.2 < 4.4.0`, numerisch nach `N`, nicht lexikografisch). `CHANGES.md`
+7 -9
View File
@@ -71,15 +71,13 @@ eine Sitzung ihn tatsächlich durchläuft:
lädt Tarball plus Prüfsumme hoch. **CI setzt den Tag, nie eine Sitzung** - das hält lädt Tarball plus Prüfsumme hoch. **CI setzt den Tag, nie eine Sitzung** - das hält
Invariante 5 intakt. Invariante 5 intakt.
## Verify-Befehle im Überblick Die drei Verify-Befehle stehen oben in Schritt 3; was jeder von ihnen prüft, steht in
[tools/CONTRACT.md](tools/CONTRACT.md) und wird dort von `docs verify` gegen die tatsächliche
| Befehl | Prüft | CLI gehalten. Hier steht es bewusst **nicht** noch einmal: eine zweite Beschreibung derselben
|---|---| Befehle ist genau die Kopie, die driftet (AGENTS.md Invariante 8), und dieses Dokument liegt
| `cd tools && .venv/bin/python -m pytest -q` | Die gesamte Testsuite, hermetisch gegen eine leere Maschine (siehe `instructions/dev/testing-conventions.md`) | außerhalb der Dateien, die der Kommandotabellen-Check von `docs verify` abdeckt - hier fällt eine
| `tools/wikitool docs verify` | CLI-Kommandotabelle, Contract-Präsenz, Type-Drift, `.gitignore`-Kanarienvögel, `VERSION`/`CHANGES.md`-Übereinstimmung, Grenzübertritts-Dokumentation | Drift also niemandem auf. Was `pytest` an dieser Stelle vom Entwickler erwartet, steht in
| `tools/wikitool instructions verify` | Jede Instruction und jeder Skill unter `instructions/`, verwaiste Dateien, `instructions/dev/`-Referenzen von außerhalb | [instructions/dev/testing-conventions.md](instructions/dev/testing-conventions.md).
Die volle Kommandoreferenz inklusive Fehlerkontrakt: [tools/CONTRACT.md](tools/CONTRACT.md).
## Die CI-Hälfte ## Die CI-Hälfte
+71 -19
View File
@@ -167,10 +167,17 @@ wenn nicht). `tools/wikitool version notes` druckt den Eintrag.
### Eine Instanz aktualisieren ### Eine Instanz aktualisieren
Das Anwenden eines Updates ist ein bewusst manueller Vorgang - es schreibt in eine Instanz, die Zwei Wege, je nachdem, wie diese Instanz entstanden ist. Ein **Clone mit gemeinsamer
bereits Inhalt hat. Der Inhalt hat dabei eine **eigene Version**: `.wikitool-kb.json` sagt, in Git-History** (`upstream`-Remote auf das Ursprungs-Repo, siehe
welcher Form die Seiten vorliegen, unabhängig davon, welche Maschinerie danebensteht. Genau [instructions/private-instance.md](instructions/private-instance.md)) nimmt Stack-Updates per
dieser Unterschied ist der Zustand, in dem sich jede Instanz mitten im Upgrade befindet. echtem Drei-Wege-Merge: `tools/wikitool upstream merge`. Alles Folgende gilt für eine **Instanz
aus einem Tarball**, ohne gemeinsame History - der Weg unten unter „Eine Instanz aktualisieren"
nutzt sie.
Das Anwenden eines Updates schreibt in eine Instanz, die bereits Inhalt hat. Der Inhalt hat dabei
eine **eigene Version**: `.wikitool-kb.json` sagt, in welcher Form die Seiten vorliegen,
unabhängig davon, welche Maschinerie danebensteht. Genau dieser Unterschied ist der Zustand, in
dem sich jede Instanz mitten im Upgrade befindet.
1. **Vor dem Tausch** prüfen, was ansteht - solange `VERSION` noch die alte ist: 1. **Vor dem Tausch** prüfen, was ansteht - solange `VERSION` noch die alte ist:
@@ -178,19 +185,49 @@ dieser Unterschied ist der Zustand, in dem sich jede Instanz mitten im Upgrade b
tools/wikitool migrate status tools/wikitool migrate status
``` ```
2. Release-Tarball herunterladen und entpacken (Weg A), die Release-Notes lesen. Steht hier etwas aus, erst diese Migrationskette abschließen (Schritt 5 unten) - `dist upgrade`
3. Die **Maschinerie** aus dem Tarball über die Instanz kopieren: `tools/`, `types/`, verweigert den Tausch sonst von selbst.
`instructions/`, `AGENTS.md`, `VERSION`, `.wikitool-release.json` - **und `kb/CONTRACT.md`**.
Die letzte Datei liegt unter einem Content-Verzeichnis, ist aber Stack-Eigentum: sie hält, 2. Release-Tarball herunterladen und die Release-Notes lesen (Weg A oben).
was `wikitool` erzwingt, und ist in jeder Instanz gleich. Nicht anfassen: alles andere unter 3. **Maschinerie tauschen:**
`kb/` und `raw/`, `work/`, `.wikitool-kb.json` und `.git/` - das ist die Instanz selbst,
`kb/CONVENTIONS.md` und die `kb/*/COLLECTION.md` eingeschlossen. ```bash
4. Achtung bei lokal angepassten Stack-Dateien. Die Autorenkonventionen gehören **nicht** dazu: tools/wikitool dist upgrade <tarball-oder-verzeichnis> --dry-run
`kb/CONVENTIONS.md` und die `kb/*/COLLECTION.md` liegen unter `kb/`, werden in Schritt 3 ```
also ohnehin nicht angefasst - genau dafür ist der Schnitt da. Wer darüber hinaus etwas
unter `tools/`, `types/` oder `instructions/` verändert hat, sichert das vorher und spielt **Beim ersten Sprung auf `4.5.0` oder höher gibt es dieses Kommando in der Instanz noch
es danach wieder ein. Welche Dateien das sind, verrät ein Vergleich gegen die sha256-Summen nicht** - es kam erst mit `4.5.0`. Dann das Werkzeug aus dem entpackten *neuen* Tarball
im `files`-Block der alten `.wikitool-release.json`. verwenden, gegen die alte Instanz gerichtet:
```bash
tar -xzf chemenu-stack-<version>.tar.gz
CHEMENU_ROOT="$PWD" chemenu-stack-<version>/tools/wikitool \
dist upgrade chemenu-stack-<version>.tar.gz --dry-run
```
`CHEMENU_ROOT` sagt dem Paket, auf welchen Korpus es zeigen soll (siehe § Konfiguration);
ohne die Variable würde es den entpackten Tarball selbst für die Instanz halten. Ab dem
zweiten Upgrade trägt die Instanz das Kommando selbst und die kurze Form oben genügt.
Klassifiziert jede Datei aus dem `files`-Block der neuen `.wikitool-release.json`:
unverändert seit der Installation, lokal verändert oder gelöscht, neu im Release, oder aus dem
Release entfallen - und druckt die Migrationskette, die nach dem Tausch aussteht, ohne sie
auszuführen. Ohne `--dry-run` schreibt der Befehl; eine lokal veränderte oder gelöschte Datei
wird dabei **nie** stillschweigend überschrieben - der Lauf bricht mit der vollständigen Liste
ab, es sei denn `--keep-local` ist gesetzt (dann bleibt jede davon unangetastet, erneut
gemeldet). `--prune` entfernt zusätzlich Dateien, die der neue Release nicht mehr ausliefert
und die seit der Installation unverändert sind. Voraussetzungen: ein sauberer Arbeitsbaum
(kein Git-Repo ist ein WARN, keine Sperre), eine lokale `.wikitool-release.json` mit
`files`-Block (fehlt sie, siehe „Fallstricke" unten), und `.wikitool-kb.json` vorhanden.
Committet und pusht nichts (Invariante 5). Vollständiger Fehlerkontrakt:
[tools/CONTRACT.md](tools/CONTRACT.md).
Eine lokal veränderte Stack-Datei ist damit sichtbar, statt von Hand gegen die sha256-Summen
im `files`-Block geprüft werden zu müssen - genau der Schritt, der vor `4.5.0` hier stand.
4. Bei einer Kompatibilitätsgrenze (`dist upgrade` meldet sie laut) die Release-Notes vor dem
nächsten Schritt lesen: **Breaking Change:** und **Migration:** im Eintrag von
`tools/wikitool version notes` sagen, was aufhört zu funktionieren und ob der Korpus
umgeschrieben werden muss.
5. **Die Migrationskette abarbeiten.** `tools/wikitool migrate status` listet jetzt alle 5. **Die Migrationskette abarbeiten.** `tools/wikitool migrate status` listet jetzt alle
offenen Migrationen in der Reihenfolge, in der sie laufen müssen - bei einem Sprung über offenen Migrationen in der Reihenfolge, in der sie laufen müssen - bei einem Sprung über
mehrere Versionen sind das mehrere. Für jede: das genannte Dokument unter mehrere Versionen sind das mehrere. Für jede: das genannte Dokument unter
@@ -204,15 +241,30 @@ dieser Unterschied ist der Zustand, in dem sich jede Instanz mitten im Upgrade b
`done` verweigert jede Version, die nicht das nächste Glied ist - eine übersprungene `done` verweigert jede Version, die nicht das nächste Glied ist - eine übersprungene
Migration hinterlässt einen Korpus in einer Form, die keine Version beschreibt. Ein Migration hinterlässt einen Korpus in einer Form, die keine Version beschreibt. Ein
abgebrochenes Upgrade wird durch erneutes `migrate status` fortgesetzt. abgebrochenes Upgrade wird durch erneutes `migrate status` fortgesetzt.
6. Prüfen: `tools/wikitool migrate verify --from <commit vor der Migration>`, dann `doctor`, 6. Prüfen: `tools/wikitool migrate verify --from <commit vor dem Tausch>`, dann `doctor`,
`docs verify`, `instructions verify` und `lint`. Zum Schluss `docs verify`, `instructions verify` und `lint`. Zum Schluss
`tools/wikitool instructions sync` (die Skills sind Kopien) und die Agent-Session neu `tools/wikitool instructions sync` (die Skills sind Kopien) und die Agent-Session neu
starten. starten. `dist upgrade` nennt diese Reihenfolge im eigenen Abschlussbericht, führt aber keinen
der Schritte selbst aus.
`doctor` warnt, solange `kb_version` hinter `VERSION` zurückliegt und noch Migrationen offen `doctor` warnt, solange `kb_version` hinter `VERSION` zurückliegt und noch Migrationen offen
sind. Einer Instanz, die älter ist als `.wikitool-kb.json`, fehlt die Datei ganz - dann einmalig sind. Einer Instanz, die älter ist als `.wikitool-kb.json`, fehlt die Datei ganz - dann einmalig
`tools/wikitool migrate baseline <version>` aufrufen; geraten wird nichts. `tools/wikitool migrate baseline <version>` aufrufen; geraten wird nichts.
**Fallstricke.** Eine Instanz ohne lokale `.wikitool-release.json` (oder eine ohne `files`-Block,
aus der Zeit vor `4.5.0`) hat für `dist upgrade` keine Basis, gegen die es eine lokale Änderung
erkennen könnte, und verweigert den Tausch - dafür gibt es heute keine Reparatur (siehe Gitea #7).
Der Befehl lädt selbst nichts herunter: `<tarball-oder-verzeichnis>` muss vorher aus Weg A
geholt werden, und ein Tarball muss genau ein Top-Level-Verzeichnis enthalten - die Form, in der
`.gitea/workflows/release.yml` es baut.
Vor `4.5.0` stand hier ein rein manueller Ablauf (Maschinerie von Hand kopieren, `kb/CONTRACT.md`
eingeschlossen, sha256-Vergleich von Hand). `dist upgrade` ersetzt genau diesen Teil; wer ihn
dennoch von Hand nachvollziehen will oder muss (ein Werkzeug, das `wikitool` selbst nicht
ausführen kann), findet die Dateiliste im `files`-Block der `.wikitool-release.json` und die
Ausnahmen (`kb/CONVENTIONS.md`, `kb/*/COLLECTION.md`, `.wikitool-kb.json`) in
[tools/CONTRACT.md](tools/CONTRACT.md)s `dist upgrade`-Zeile.
### Sonderfall: Update von 1.x auf 2.0.0 ### Sonderfall: Update von 1.x auf 2.0.0
Mit `2.0.0` wurde das Ursprungs-Repo von `torben/llm-wiki-test1` auf `torben/chemenu` Mit `2.0.0` wurde das Ursprungs-Repo von `torben/llm-wiki-test1` auf `torben/chemenu`
+1 -1
View File
@@ -1 +1 @@
4.4.0 4.5.1
+51 -7
View File
@@ -4,7 +4,8 @@ Chemenu ships two kinds of files side by side, and at a glance they look the sam
plain markdown, both sit in the repo root or under `kb/`, both get read at session start. But a plain markdown, both sit in the repo root or under `kb/`, both get read at session start. But a
stack upgrade treats them completely differently. Some - [AGENTS.md](../AGENTS.md), stack upgrade treats them completely differently. Some - [AGENTS.md](../AGENTS.md),
[kb/CONTRACT.md](../kb/CONTRACT.md), the per-stage contracts - are identical in every instance [kb/CONTRACT.md](../kb/CONTRACT.md), the per-stage contracts - are identical in every instance
that runs this stack and can simply be overwritten by the next release. Others - `USER.md`, that runs this stack and are the next release's to replace (with one caveat about local edits,
below). Others - `USER.md`,
`SOUL.md`, `kb/CONVENTIONS.md`, `ENVIRONMENT.md` - describe one particular instance, and `SOUL.md`, `kb/CONVENTIONS.md`, `ENVIRONMENT.md` - describe one particular instance, and
overwriting them would silently erase a choice someone made on purpose. overwriting them would silently erase a choice someone made on purpose.
@@ -42,6 +43,25 @@ Keeping the boundary at the file level, rather than trying to merge changes with
file, means an upgrade never has to guess which lines are "stack" and which are "instance" - file, means an upgrade never has to guess which lines are "stack" and which are "instance" -
the file itself already answers that. the file itself already answers that.
## Why the boundary is a predicate rather than a list
For a while the boundary was written down as a list of paths - once in `dist_cmd.py`, once in
the merge procedure a private instance was told to run by hand, and once in the check that
procedure ended with. Three copies of one fact, which is the shape [AGENTS.md](../AGENTS.md)
invariant 8 exists to forbid, and they drifted exactly as predicted: the hand-run procedure was
still naming three paths after the collection contracts had moved to the instance's side of the
line, so it discarded upstream changes to files it had never heard of, while its own final check
excluded the same three paths and therefore reported success.
`chemenu/ownership.py` replaced the lists with one question - is this path, under a content
stage, the stack's or the instance's? - answered by shape rather than by enumeration:
`<stage>/CONTRACT.md`, and anything ending `.template`. Both consumers ask it, so `dist export`
and `wikitool upstream merge` cannot disagree, and a machinery file added under a content stage
tomorrow is recognised by both without either being edited. The deeper point is not the
deduplication: a list has to be maintained by whoever remembers it exists, and the failure mode
when nobody does is silence, because a path the list has never heard of simply looks like
content.
## Why a `.template`, not just an absent file ## Why a `.template`, not just an absent file
The mechanism for instance-owned content is a `.template` file the distribution ships instead The mechanism for instance-owned content is a `.template` file the distribution ships instead
@@ -63,9 +83,33 @@ state rather than a sign setup was skipped.
## The consequence in practice ## The consequence in practice
Running a stack upgrade against an existing instance boils down to: overwrite the verbatim An upgrade sorts every shipped path into three categories, not two - and the third one only
files, leave the `.template`-sourced files alone. The verbatim files are safe to replace becomes visible once an upgrade is a command rather than a hand-run copy:
wholesale because they were never instance-specific to begin with - identical content going
back in changes nothing an instance actually decided. The template-sourced files were filled in - **Verbatim files** - `AGENTS.md`, `kb/CONTRACT.md`, the per-stage contracts, everything under
once, by a person, for a reason, and nothing about a newer release of the stack's mechanics `tools/`, `types/` and `instructions/` - are the release's to replace.
gives it standing to override that. - **`.template`-sourced files** - `USER.md`, `SOUL.md`, `kb/CONVENTIONS.md`, each
`kb/<name>/COLLECTION.md`, `ENVIRONMENT.md`, the `root: kb` type-specs - are never written by
an upgrade at all. The distribution ships only the `.template` beside them, so the filled file
is out of reach by construction rather than by a rule someone has to remember.
- **Seeded-once files** - `.wikitool-kb.json`, `CHANGES.md`, `kb/log.md`, `raw/*/.gitkeep` - are
written into a *new* instance by `dist export` and belong to the instance from then on. They
are the awkward category: they sit in the release stamp's file list like any other shipped
file, so an upgrade has to exclude them deliberately (`chemenu.ownership.is_export_stub` and
`is_upgrade_preserved`). An upgrade that re-seeded them would reset the record of which
migrations ran, or erase the changelog the instance wrote for itself.
The first category carries a caveat that the word "verbatim" hides. It says who *decides* the
content, not that overwriting is always safe: an instance can still have edited a verbatim file
- a patched `tools/`, a locally adjusted instruction - and an upgrade assuming otherwise would
destroy that silently. Avoiding that assumption is the whole reason `dist export` records a
sha256 per shipped file in `.wikitool-release.json`. `wikitool dist upgrade` compares every
candidate path against the digest recorded when it was installed, overwrites only what still
matches, and refuses rather than overwrite what does not.
So the practical rule is narrower than "overwrite the verbatim files, leave the rest alone":
overwrite the verbatim files *this instance has not touched*, never write the other two
categories, and make a locally changed file a decision someone takes deliberately instead of
one an upgrade takes for them. The template-sourced files were filled in once, by a person, for
a reason, and nothing about a newer release of the stack's mechanics gives it standing to
override that.
+19 -14
View File
@@ -83,22 +83,27 @@ matter just as much in practice: how many numbers get handed out along the way t
release. For a while the answer was "one per bump," and that turned out to be the wrong grain release. For a while the answer was "one per bump," and that turned out to be the wrong grain
entirely. entirely.
The two things that actually consume a version number are a release and CI's version gate - and Two mechanisms decide when a number gets minted, and they answer different questions. CI's
they disagree about granularity. The gate wants `VERSION` to move on every push that touches version gate asks a *commit*-level one: has this tree changed since the last push, and if so
stack-shaped paths, which is a *commit*-level question: has this tree changed since the last has `VERSION` moved with it. A release asks something else entirely: is this a state worth
push. A release wants to know something else: has *this specific number* been published, ever. handing to someone, under a number they will pin against. Tying the second to the first - every
Handing out a fresh number per bump answers the gate's question by accident and the release's `VERSION` move firing the release workflow - answers the gate correctly and the release question
question wrongly - it treats every bump as if it were about to ship, when most of them are steps by accident, because it treats every bump as if it were about to ship when most bumps are steps
toward a release that hasn't happened yet. Four bumps in one session, on the same day, for the toward a release that has not happened yet.
same eventual release, produced four numbers that a version-check feed would have reported as
four different available upgrades, three of which were never real. The failure mode is not phantom numbers; every one of those releases was real, tagged and
downloadable. It is that "real" stopped meaning anything. On 2026-09-03 this repository cut four
releases in six hours - `4.3.0` through `4.3.3` - for one continuous arc of work, two of them for
prose changes alone. Someone tracking the feed saw four upgrades and had no way to tell which, if
any, was a moment worth stopping for. A release is a promise addressed to a consumer, and a
promise made four times an afternoon is not a smaller promise, it is a less legible one.
The fix is not to slow the gate down - it still wants `VERSION` to move every time, and it still The fix is not to slow the gate down - it still wants `VERSION` to move every time, and it still
gets that. It's to stop treating every movement as a new number. Between two releases the stack gets that. It is to stop treating every movement as a number worth publishing. Between two
now carries one running candidate, escalating through `-beta.N` as bumps accumulate, and only releases the stack now carries one running candidate, escalating through `-beta.N` as bumps
`version release` spends the number for real by fixing it and closing its changelog entry. A accumulate, and only `version release` spends the number for real by fixing it and closing its
number is proposed by a bump and consumed by a release; conflating the two was the actual defect, changelog entry. A number is proposed by a bump and spent by a release; conflating the two was
not the arithmetic of any single bump. the actual defect, not the arithmetic of any single bump.
This is also why a candidate never gets to a distributed instance. The promise a released version This is also why a candidate never gets to a distributed instance. The promise a released version
makes - "install this, and it is exactly what its number says" - has no equivalent for something makes - "install this, and it is exactly what its number says" - has no equivalent for something
+69
View File
@@ -34,6 +34,8 @@ issues at that URL, which is exactly why `dist export` excludes
moved. moved.
- Closing one: the body is rewritten to its final state first, and only then - Closing one: the body is rewritten to its final state first, and only then
closed (step 7). closed (step 7).
- A rename or move ships: sweep the open issues for text that assumed the old
name or path (§ Renames and other decay in the tracker).
## Steps ## Steps
@@ -42,6 +44,26 @@ issues at that URL, which is exactly why `dist export` excludes
specific files or commands involved. An issue that only makes sense to specific files or commands involved. An issue that only makes sense to
whoever wrote it is a note, and notes were the problem. whoever wrote it is a note, and notes were the problem.
**Destructive steps carry the invariant they must not violate.** A body
that prescribes a mechanism gets built as prescribed - including its
bugs. Where a step deletes, overwrites, resets or moves, name the
property that must still hold afterwards, not only the command that gets
there. "Remove the working directory, then `git checkout HEAD --
<stage>`" is a mechanism; "the content stages must afterwards match
`HEAD` exactly, without any untracked or ignored file being touched" is
the same instruction plus its test - a build instruction and an
acceptance criterion at once, so the defect surfaces while the test is
written rather than in review afterwards. #30's `upstream merge` body
wrote the mechanism and got exactly that bug: a working-directory removal
that took a stage's gitignored, unrecoverable data with it.
**An acceptance criterion states a checkable property, not an activity.**
"Implement X" is done when someone says so; "after `upstream merge`,
`reports/` still holds every file it held before" is done when it is
true. This is not a ban on imperative steps - a numbered procedure can
still produce a correct control flow, and that is its merit - it binds
the destructive steps, and every box in the criteria list.
2. **The body is the working state, not a historical first post - keep it 2. **The body is the working state, not a historical first post - keep it
current as you go.** It is this stack's plan file: the same thing a harness's current as you go.** It is this stack's plan file: the same thing a harness's
own plan document is, and it is maintained the same way. Not written once, own plan document is, and it is maintained the same way. Not written once,
@@ -73,6 +95,32 @@ issues at that URL, which is exactly why `dist export` excludes
Body rewrites and comments are an LLM session's job. A human normally Body rewrites and comments are an LLM session's job. A human normally
touches only labels and metadata directly. touches only labels and metadata directly.
**Reading an issue, the body is the state and comments are history.** A
session picking an issue up reads the body as the spec; comments are read
for provenance - why something was decided, what was tried - never as
the current instruction. A recommendation in a comment can be older than
the body's decision and read just as convincingly: on #30 an earlier
comment recommended a smaller, `verify`-only command, while the body had
since settled on building the full `merge` command. A session trusting
the comment would have built the wrong thing, with a plausible
justification out of this repo's own tracker.
**A body that is demonstrably wrong is corrected first, not worked
around.** "Body beats comment" is a rule of precedence, not a licence to
execute a stale spec. Where a comment or the tree proves a claim in the
body false, the body is rewritten before the work starts - the rewrite
above is the fix; leaning on the comments as the "real" state is not.
#10 is the case: its body claimed coverage had never been measured while
three comments carried a percentage, a statement count and a CI run
number.
**Where two comments contradict each other, evidence decides, not
recency.** On #10, one comment showed a retrieved artifact with zero
items on a finished run - the report was not actually retrievable - and
a later comment declared the same criterion met without re-checking. The
later comment is not the newer truth, only the unchecked one. Resolve it
into the body with the evidence named, or mark the point open.
3. **Comment a changelog, never a copy.** A body rewrite gets one short comment 3. **Comment a changelog, never a copy.** A body rewrite gets one short comment
naming only what changed against the previous state - what is new, what is naming only what changed against the previous state - what is new, what is
gone, what was corrected. Do not snapshot the old body into a comment: a full gone, what was corrected. Do not snapshot the old body into a comment: a full
@@ -180,6 +228,27 @@ issues at that URL, which is exactly why `dist export` excludes
shipped. Nothing mechanical catches it (see below), which is why it is a step shipped. Nothing mechanical catches it (see below), which is why it is a step
rather than a habit. rather than a habit.
## Renames and other decay in the tracker
A rename is not finished when the tree is green. Renaming a package, a path,
a command, a flag or the repository itself moves text that lives outside the
working tree, and the open issues are the largest such text. Nothing catches
them - `wikitool` does not know this tracker exists and must not learn (see
"What no tool checks" below) - so a pass over the open issues is part of the
rename, in the session that did it, not a follow-up someone remembers.
Distinguish a wayfinder from a piece of evidence: a path meant to point at
where something *is* gets pulled through; a path quoted for what was true at
a time is left standing and dated. Note per corrected body what was pulled
through and when, so the next pass can tell a checked body from one that
merely looks right. Closed issues are out of scope - they guide nobody.
Renames are not the only thing that ages an issue text. A page a body cites
can vanish from `kb/` (`wikitool search` against the cited titles is the
second pass), and an old body can carry private infrastructure detail into
what is now a public tracker - both found in the same issue, both worth the
same look.
## What no tool checks ## What no tool checks
`wikitool` does not know this tracker exists, and should not learn. It ships to `wikitool` does not know this tracker exists, and should not learn. It ships to
+10 -8
View File
@@ -9,7 +9,7 @@ Two questions decide a version bump, and they are **not the same question**:
1. **Is the new version a drop-in replacement for the old one?** This is what the version 1. **Is the new version a drop-in replacement for the old one?** This is what the version
number itself says. Compatibility is read off the **leftmost non-zero component** - on this number itself says. Compatibility is read off the **leftmost non-zero component** - on this
stack (`2.x`) that is MAJOR, on a `0.x` stack it is MINOR. A bump that changes it is called stack (`4.x`) that is MAJOR, on a `0.x` stack it is MINOR. A bump that changes it is called
*boundary-crossing* below, because that is the term `version bump` and `docs verify` use in *boundary-crossing* below, because that is the term `version bump` and `docs verify` use in
their own messages. their own messages.
2. **Must existing content be migrated?** This is a *consequence* a boundary crossing may or 2. **Must existing content be migrated?** This is a *consequence* a boundary crossing may or
@@ -23,11 +23,13 @@ already (see the case study at the end), which is why this file exists.
## The candidate model ## The candidate model
Between two releases the stack carries **one running candidate**, not a fresh version per Between two releases the stack carries **one running candidate**, not a fresh version per
`bump`. Five bumps with no release in between used to mean five numbers, four of which nothing `bump`. Before that, every bump minted a number *and* a release: CI's version gate requires
ever consumed - the release-granularity CI's version gate wants (`VERSION` must move on every `VERSION` to move on every stack-touching push, and `release.yml` fires on every `VERSION`
stack-touching push) was being paid at bump granularity instead. A candidate closes that gap move, so releases were being cut at commit granularity. 2026-09-03 produced four of them in
without touching the gate: `VERSION` still moves on every bump, it just escalates the *same* six hours (`4.3.0` through `4.3.3`) for one arc of work - all four real, none of them a
number instead of handing out a new one. meaningful unit to anyone downstream. A candidate closes that gap without touching the gate:
`VERSION` still moves on every bump, it just escalates the *same* number instead of handing out
a new one, and only `version release` turns it into something the release workflow acts on.
- **State lives in `VERSION` itself**, as an optional `-beta.N` suffix (`4.4.0-beta.3`). No - **State lives in `VERSION` itself**, as an optional `-beta.N` suffix (`4.4.0-beta.3`). No
second state file: the last release is read back out of `CHANGES.md` (the newest entry with no second state file: the last release is read back out of `CHANGES.md` (the newest entry with no
@@ -35,8 +37,8 @@ number instead of handing out a new one.
release - derived, not stored. release - derived, not stored.
- **`--major`/`--minor`/`--patch` is max-wins escalation**, not a step you can undo. A `--patch` - **`--major`/`--minor`/`--patch` is max-wins escalation**, not a step you can undo. A `--patch`
bump on a candidate already at MINOR only advances its bump count (`N`); nothing ever steps a bump on a candidate already at MINOR only advances its bump count (`N`); nothing ever steps a
candidate back down. Declaring the part is still your judgment call, made the same way section candidate back down. Declaring the part is still your judgment call, made the same way the
below describes - `escalate()` only ever raises it further. steps below describe - `escalate()` only ever raises it further.
- **A candidate is never released.** Pre-release is a dev-checkout state; `release.yml` only acts - **A candidate is never released.** Pre-release is a dev-checkout state; `release.yml` only acts
on a suffix-free `VERSION`, so a distributed instance never sees a `-beta.` version at all, and on a suffix-free `VERSION`, so a distributed instance never sees a `-beta.` version at all, and
its parser never has to know the suffix exists. its parser never has to know the suffix exists.
+20
View File
@@ -103,6 +103,26 @@ The setup this gate exists for - a private instance that takes stack updates fro
upstream - is [private-instance.md](private-instance.md). Step 4 there arms it, deliberately upstream - is [private-instance.md](private-instance.md). Step 4 there arms it, deliberately
*before* the first `publish`: added afterwards it leaves open exactly the window it closes. *before* the first `publish`: added afterwards it leaves open exactly the window it closes.
### Mass-Update Gate blind spot: `upstream merge`
`upstream merge` (a private instance taking a stack update - see
[private-instance.md](private-instance.md)) can update or delete dozens of stack-owned paths in
one commit, and the Mass-Update Gate does not see any of it. The gate counts *working-tree*
changes before `publish` stages them; by the time `upstream merge` commits, the change is
already history, and the commit it made is not what a later `publish` would be staging - that
publish sees only whatever this session adds on top. A merge touching 200 files therefore goes
out ungated the moment it is pushed.
This is not a hole to patch by making `upstream merge` route through the gate: the gate's
question ("is this too much to publish?") does not apply to a change that only ever touches
stack-owned paths that are, by definition, not this instance's own content. The check that
actually matters here is `upstream merge`'s own postcheck - it re-verifies the merge commit
against `upstream verify`'s logic immediately after committing, and exits 1 with the offending
paths if anything landed outside a stack-owned one. **The merge commit is deliberately left in
place** rather than reverted: it exists, a human has to look at it, and a command that quietly
repaired its own mistake would hide the one event worth seeing. That postcheck is the safeguard
for this command, not the Mass-Update Gate.
## Iteration Budget Gate and loop-breaker ## Iteration Budget Gate and loop-breaker
Every `wikitool` call is counted per session. Calls are refused past **60 in a session**, or Every `wikitool` call is counted per session. Calls are refused past **60 in a session**, or
+50 -62
View File
@@ -108,69 +108,53 @@ So the merge has to be scoped. That is the procedure below, and it is not option
## Taking a stack update ## Taking a stack update
Take the machinery, never the content. The merge is held open, the content stages are forced ```bash
back to your own state, and only then does it close. tools/wikitool upstream merge --remote upstream --branch main
```
**Three files under those stages are machinery, not content**, and forcing them back is how an Take the machinery, never the content. This is the command form of the same idea a hand-rolled
upstream contract change gets silently discarded: merge would need: hold the merge open, force the content stages back to your own state, restore
only the paths that are machinery, and only then let it close. Which paths those are is not a
short literal list any more (see below) - it is `chemenu.ownership.is_stack_owned`, the same
predicate `dist_cmd.py`'s export reads, so a stack change that adds a new machinery path under a
content stage is recognised automatically rather than needing this document edited first.
| Path | Why it must take the upstream side | **What counts as machinery under a content stage**, for readers who want the shape rather than
the code:
| Path | Why it takes the upstream side |
|---|---| |---|---|
| `kb/CONTRACT.md` | The stack's own knowledge-layer contract. Every rule in it is enforced by `wikitool`; an instance never edits it | | `<stage>/CONTRACT.md` (`kb/CONTRACT.md`, `raw/CONTRACT.md`, `work/CONTRACT.md`, `reports/CONTRACT.md`) | The stack's own stage contract. Every rule in it is enforced by `wikitool`; an instance never edits it |
| `kb/CONVENTIONS.md.template` | The template your `kb/CONVENTIONS.md` was filled from. The filled file is yours; the template is the stack's | | any `*.template` under a content stage (`kb/CONVENTIONS.md.template`, each `kb/<name>/COLLECTION.md.template`, and any later one) | The template your filled file was adopted from. The filled file is yours; the template is the stack's |
| `raw/CONTRACT.md` | The raw stage's contract, for the same reason as the first row |
Everything else under `kb/` and `raw/` is yours, `kb/CONVENTIONS.md` and each Everything else under `kb/`, `raw/`, `work/` and `reports/` is yours, `kb/CONVENTIONS.md` and
`kb/<name>/COLLECTION.md` included - they bind your corpus, and they are exactly what the each `kb/<name>/COLLECTION.md` included - they bind your corpus, and they are exactly what
restore below is protecting. `upstream merge` protects.
```bash **Your local, uncommitted-by-design files under those stages survive.** Forcing a content stage
BEFORE=$(git rev-parse HEAD) back to your own state removes only what git tracks, never the directory wholesale - which
git fetch upstream matters because `reports/` is gitignored apart from its contract, so it holds data that is in no
commit and cannot be recomputed: the telemetry traces `eval score` reads, saved eval reports,
past lint reports. A merge has no business touching any of it, and does not.
# --no-commit holds the merge open; it may report conflicts under kb/ or raw/, The command itself checks its own result the same way `upstream verify` would, immediately
# which the next four lines are about to make irrelevant. after committing, and refuses loudly - without rolling the commit back - if anything landed
git merge --no-commit --no-ff upstream/main || true outside a stack-owned path. A refusal there is a bug report, not something to work around by
hand; see [tools/CONTRACT.md](../tools/CONTRACT.md) for the full error contract, including what
# Whatever the merge did to the content stages, undo it. HEAD is still your a real conflict in `tools/`/`types/`/`instructions/` leaves behind.
# pre-merge commit while the merge is open, so this restores exactly your side.
git rm -rq --cached --ignore-unmatch kb raw
rm -rf kb raw
git checkout HEAD -- kb raw
# ...then take the upstream side back for the machinery that lives among it.
# MERGE_HEAD is still resolvable while the merge is open.
git checkout MERGE_HEAD -- kb/CONTRACT.md kb/CONVENTIONS.md.template raw/CONTRACT.md
git commit --no-edit
```
Then **check that it worked**, rather than trusting that it did. The same three paths are
excluded here, spelled out rather than held in a variable so that the check can be read on its
own and copied on its own:
```bash
git diff --name-only "$BEFORE" HEAD -- kb raw \
| grep -vE '^(kb/CONTRACT\.md|kb/CONVENTIONS\.md\.template|raw/CONTRACT\.md)$'
```
Must print nothing.
An empty result is the proof that the update touched machinery only. A non-empty one means a
path slipped through - inspect it before going further.
**The exclusion is not cosmetic.** Without it the check reports *empty* for an update that just
ate a `kb/CONTRACT.md` change - it would be confirming the failure it exists to catch. If one of
the three paths does not appear in the diff at all, that is fine: it means upstream did not
touch it.
Then, as after any stack change: `doctor`, `docs verify`, `instructions verify`, `migrate status`, Then, as after any stack change: `doctor`, `docs verify`, `instructions verify`, `migrate status`,
`lint`. A `migrate status` with outstanding links means the update crossed a compatibility `lint`. A `migrate status` with outstanding links means the update crossed a compatibility
boundary - follow [migrate-corpus.md](migrate-corpus.md) before doing anything else. boundary - follow [migrate-corpus.md](migrate-corpus.md) before doing anything else.
**Why not just `git merge upstream/main`?** Because of the table above: a page the upstream **Why not just `git merge upstream/main`?** A page the upstream *adds* arrives with no conflict
*adds* arrives with no conflict and no message. You would find out when `lint` starts reporting and no message under a plain merge - measured in the table further up this document. You would
pages you never wrote - if you noticed at all. find out when `lint` starts reporting pages you never wrote, if you noticed at all. `upstream
merge` closes exactly that gap: the content stages never see the upstream's version at all.
**Checking a merge you resolved by hand instead** (or auditing a past one): `tools/wikitool
upstream verify --since <rev-before> --until <rev-after>` runs the same check `upstream merge`
runs on itself, without doing the merge.
## Where stack development happens ## Where stack development happens
@@ -186,17 +170,21 @@ merge above. Nothing is lost by the detour: the fix has to pass that CI either w
## Decision points ## Decision points
- **Merge conflict in `kb/` or `raw/`?** Expected, and already handled: the update procedure - **Merge conflict in `kb/`, `raw/`, `work/` or `reports/`?** Expected, and already handled:
above overwrites those stages with your own afterwards, so the conflict resolves itself. `upstream merge` overwrites those stages with your own afterwards, so the conflict resolves
Never resolve one by hand with `git add -A` - that is exactly how the upstream version, which itself. Never resolve one by hand with `git add -A` in a merge you are running yourself
git left sitting in your working tree, gets committed into your instance. instead - that is exactly how the upstream version, which git left sitting in your working
- **`git diff` after the merge shows something under `kb/` or `raw/`?** Stop - unless it is one tree, gets committed into your instance.
of the three machinery paths the check excludes, which is the update working as intended. For - **`upstream merge` exits 1 after committing?** Read the message: its own postcheck found
anything else the scoping step did not take: do not publish; find out which path came through content outside a stack-owned path in the commit it just made. The commit is **not** rolled
and where from. back - inspect it (`git show`, or `tools/wikitool upstream verify --since <before> --until
HEAD`) and decide by hand whether to revert it, fix forward, or report it as a stack bug. This
should not happen; if it does, `chemenu.ownership.is_stack_owned` disagreed with itself between
the restore and the check, which is exactly what the shared predicate is meant to prevent.
- **Conflict in `tools/`, `types/` or `instructions/`?** You changed the stack locally, which - **Conflict in `tools/`, `types/` or `instructions/`?** You changed the stack locally, which
step "Where stack development happens" says not to do. Take the upstream side and re-file the step "Where stack development happens" says not to do. `upstream merge` leaves the merge open
change as an issue there. rather than guessing - take the upstream side for the named paths and re-file the change as an
issue there, or resolve deliberately and finish the commit yourself.
- **...but you changed how *your pages* are written?** That is not a stack change and the rule - **...but you changed how *your pages* are written?** That is not a stack change and the rule
above does not apply to it. Language, section headings, naming forms, tone, relationship above does not apply to it. Language, section headings, naming forms, tone, relationship
labels and the confidence rubric live in `kb/CONVENTIONS.md`, and each collection's authoring labels and the confidence rubric live in `kb/CONVENTIONS.md`, and each collection's authoring
+6
View File
@@ -72,6 +72,7 @@ tools/wikitool <command> --help
| `eval sessions [--json]` | List the sessions that have a trace under `reports/telemetry/`, most recent first. Read-only and exempt from the Iteration Budget Gate | | `eval sessions [--json]` | List the sessions that have a trace under `reports/telemetry/`, most recent first. Read-only and exempt from the Iteration Budget Gate |
| `eval score [--session <id>] [--json] [--markdown out.md] [--save] [--fail-on-error]` | Score one traced session: structural state from `lint`'s own checks (L1) plus trajectory rules over the trace (L2) - was a refused call repeated unchanged, was a gate flag passed without that gate having refused anything, did a publish of `kb/` pages go unlogged. Defaults to the current session. `--save` writes `reports/evals/<date>/<session>.{json,md}`. Read-only over `kb/` and exempt from the budget; see [../EVALS.md](../EVALS.md) | | `eval score [--session <id>] [--json] [--markdown out.md] [--save] [--fail-on-error]` | Score one traced session: structural state from `lint`'s own checks (L1) plus trajectory rules over the trace (L2) - was a refused call repeated unchanged, was a gate flag passed without that gate having refused anything, did a publish of `kb/` pages go unlogged. Defaults to the current session. `--save` writes `reports/evals/<date>/<session>.{json,md}`. Read-only over `kb/` and exempt from the budget; see [../EVALS.md](../EVALS.md) |
| `dist export <target> [--dry-run] [--source-repo U] [--source-commit SHA] [--release-url U] [--update-url U]` | Write a contentless, distributable copy of this repo's machinery into an empty `<target>` directory: `AGENTS.md`/`README.md`/`EVALS.md` with any `<!-- dist:strip-start -->...<!-- dist:strip-end -->` region removed, `instructions/` (minus `instructions/dev/`), `types/` (the `root: kb` page type-specs and their schemas re-keyed as `.template`, the stack's own verbatim), `docs/` verbatim, `tools/` (no venv/caches), the `.github/hooks/`+`.vibe/` session-tracing config plus `.claude/settings.json`, `kb/CONTRACT.md` (no pages, no areas), empty `raw/{articles,documents,notes,assets}/`, `VERSION`, `USER.md.template`/`SOUL.md.template` plus `kb/CONVENTIONS.md.template` and each collection's contract re-keyed as `kb/<name>/COLLECTION.md.template` (the templates ship; the filled `USER.md`/`SOUL.md`/`kb/CONVENTIONS.md`/`kb/<name>/COLLECTION.md`/`types/<page-type>.md` never do - all of them bind their instance and none are the stack's to decide, and `find_leaks` refuses a plan carrying one), and a generated `.wikitool-release.json` stamp (version, export date, origin, and a sha256 per exported file - the base a later upgrade would compare against). The four origin options only fill stamp fields: `export` never calls git and cannot discover them. Refuses a non-empty target, and a tree with no `VERSION`. See `instructions/setup-instance.md`. One-way: there is no command that reconstructs a distributed instance into a dev instance - work on the stack in the origin repo (or a new dev instance exported from it) instead | | `dist export <target> [--dry-run] [--source-repo U] [--source-commit SHA] [--release-url U] [--update-url U]` | Write a contentless, distributable copy of this repo's machinery into an empty `<target>` directory: `AGENTS.md`/`README.md`/`EVALS.md` with any `<!-- dist:strip-start -->...<!-- dist:strip-end -->` region removed, `instructions/` (minus `instructions/dev/`), `types/` (the `root: kb` page type-specs and their schemas re-keyed as `.template`, the stack's own verbatim), `docs/` verbatim, `tools/` (no venv/caches), the `.github/hooks/`+`.vibe/` session-tracing config plus `.claude/settings.json`, `kb/CONTRACT.md` (no pages, no areas), empty `raw/{articles,documents,notes,assets}/`, `VERSION`, `USER.md.template`/`SOUL.md.template` plus `kb/CONVENTIONS.md.template` and each collection's contract re-keyed as `kb/<name>/COLLECTION.md.template` (the templates ship; the filled `USER.md`/`SOUL.md`/`kb/CONVENTIONS.md`/`kb/<name>/COLLECTION.md`/`types/<page-type>.md` never do - all of them bind their instance and none are the stack's to decide, and `find_leaks` refuses a plan carrying one), and a generated `.wikitool-release.json` stamp (version, export date, origin, and a sha256 per exported file - the base a later upgrade would compare against). The four origin options only fill stamp fields: `export` never calls git and cannot discover them. Refuses a non-empty target, and a tree with no `VERSION`. See `instructions/setup-instance.md`. One-way: there is no command that reconstructs a distributed instance into a dev instance - work on the stack in the origin repo (or a new dev instance exported from it) instead |
| `dist upgrade <source> [--dry-run] [--keep-local] [--prune] [--pre]` | Apply a stack update `dist export` produced - the write half of `version check`. Never downloads anything: `<source>` is an already-fetched export directory or `.tar.gz` release archive (verified against a sibling `.sha256` if one is present; WARNs, does not block, if it is absent), which must unpack to exactly one top-level directory - the shape `.gitea/workflows/release.yml` packs. The write set is exactly the *new* `.wikitool-release.json`'s `files` block, minus what an export re-seeds from a blank template every time (`kb/log.md`, `raw/*/.gitkeep` - `chemenu.ownership.is_export_stub`) or seeds once and the instance owns from then on (`.wikitool-kb.json`, `CHANGES.md` - `chemenu.ownership.is_upgrade_preserved`), plus the stamp itself, always rewritten. Every candidate path is classified against the *local* `.wikitool-release.json`'s recorded digest for it: unchanged is overwritten silently, absent from the old stamp is created, and locally modified or locally deleted is **never** silently overwritten - the run aborts with the full list unless `--keep-local` says to proceed and leave every one of them untouched. After a `--keep-local` run the new stamp is still written whole, so it records the release's digest for files that were deliberately *not* written: the stamp is the baseline for the next comparison, not a literal inventory of what is on disk. That is what keeps a skipped file diverging - and therefore reported - on every later run, rather than quietly reading as current once it has been skipped once. A path in the old stamp but not the new one is reported as no longer part of the release and left alone unless `--prune` is passed, which removes it only if it is still unchanged since installation. Reports the migration chain the new machinery would owe (`chemenu.kb_state.chain` over the *new* tree's `instructions/migrations/`, read via a `directory` argument to `load_migrations`) but never runs any of it - there is no `migrate run`. Refuses before touching the source at all when: `VERSION` or `.wikitool-release.json` (with a `files` block) is missing locally, `.wikitool-kb.json` is missing, a migration is already outstanding against the *installed* machinery, or the working tree is dirty (not being a git repository at all is a WARN, not a refusal). Refuses after reading the source when: it carries no `VERSION`/`.wikitool-release.json`/`files` block, its version is older than or equal to the installed one (equal is a no-op success), or it is a pre-release (`-beta.N`) without `--pre`. Reports, but does not block on, a crossed compatibility boundary. Never touches git - no commit, no push (invariant 5). See Gitea #7 and `INSTALL.md` § "Eine Instanz aktualisieren" |
| `version show [--json]` | Print this instance's stack version and where it came from (development tree, or a distribution with its export date and origin). Bare `wikitool version` is an alias for this. Read-only, offline, and **exempt from the Iteration Budget Gate** | | `version show [--json]` | Print this instance's stack version and where it came from (development tree, or a distribution with its export date and origin). Bare `wikitool version` is an alias for this. Read-only, offline, and **exempt from the Iteration Budget Gate** |
| `version check [--url U] [--timeout S] [--json]` | Ask the origin's release feed whether a newer stack exists, and whether the step crosses a compatibility boundary (`state: current\|update\|migration\|ahead`). **The only command in `wikitool` that makes a network call** - never reached implicitly from another command, needs no key, times out, and reports an unreachable feed as an error rather than as "up to date". The feed is `$WIKITOOL_UPDATE_URL`, else the release stamp's, else the built-in origin; `$WIKITOOL_UPDATE_TOKEN` is only needed if that feed is not readable anonymously. Read-only and exempt from the budget gate | | `version check [--url U] [--timeout S] [--json]` | Ask the origin's release feed whether a newer stack exists, and whether the step crosses a compatibility boundary (`state: current\|update\|migration\|ahead`). **The only command in `wikitool` that makes a network call** - never reached implicitly from another command, needs no key, times out, and reports an unreachable feed as an error rather than as "up to date". The feed is `$WIKITOOL_UPDATE_URL`, else the release stamp's, else the built-in origin; `$WIKITOOL_UPDATE_TOKEN` is only needed if that feed is not readable anonymously. Read-only and exempt from the budget gate |
| `version notes [--version X.Y.Z]` | Print one version's `CHANGES.md` entry, for use as release notes (default: this tree's `VERSION`). Read-only and exempt from the budget gate | | `version notes [--version X.Y.Z]` | Print one version's `CHANGES.md` entry, for use as release notes (default: this tree's `VERSION`). Read-only and exempt from the budget gate |
@@ -82,6 +83,8 @@ tools/wikitool <command> --help
| `migrate verify --from <rev> [--path P ...] [--expect-body-change] [--json] [--fail-on-error]` | Compare `kb/` against a git revision on the invariants a content migration must not change: wikilink and citation **counts** (not sets), footnote definitions, H1, structural frontmatter, and the **count of generated-region marker pairs** - a page that went from one links region to two has the same set of region names and a different count, and a lost marker turns a generated region into prose the next write appends a second one beside. Reports added/removed pages without failing on them. `--expect-body-change` additionally flags a page whose body did not change at all. Not migration-specific - worth running after any bulk rewrite, and the one question `lint` cannot answer, since it reads a single revision and so cannot see that something went missing. Read-only and exempt from the budget gate | | `migrate verify --from <rev> [--path P ...] [--expect-body-change] [--json] [--fail-on-error]` | Compare `kb/` against a git revision on the invariants a content migration must not change: wikilink and citation **counts** (not sets), footnote definitions, H1, structural frontmatter, and the **count of generated-region marker pairs** - a page that went from one links region to two has the same set of region names and a different count, and a lost marker turns a generated region into prose the next write appends a second one beside. Reports added/removed pages without failing on them. `--expect-body-change` additionally flags a page whose body did not change at all. Not migration-specific - worth running after any bulk rewrite, and the one question `lint` cannot answer, since it reads a single revision and so cannot see that something went missing. Read-only and exempt from the budget gate |
| `migrate done <version> [--pages N] [--dry-run]` | Record one migration as applied, advancing `kb_version` in `.wikitool-kb.json` to its target. **Refuses any version that is not the next link in the chain** - skipping one leaves the corpus in a shape no version describes, and an interrupted multi-step upgrade has to be resumable rather than guessable. An `offered` migration is recorded in the applied ledger *without* moving `kb_version` and with no ordering rule applied: it is not a link in the chain, so there is nothing to skip, and requiring the chain first would make an unrelated file upgrade wait on it. Re-recording one already in the ledger is a no-op, not an error | | `migrate done <version> [--pages N] [--dry-run]` | Record one migration as applied, advancing `kb_version` in `.wikitool-kb.json` to its target. **Refuses any version that is not the next link in the chain** - skipping one leaves the corpus in a shape no version describes, and an interrupted multi-step upgrade has to be resumable rather than guessable. An `offered` migration is recorded in the applied ledger *without* moving `kb_version` and with no ordering rule applied: it is not a link in the chain, so there is nothing to skip, and requiring the chain first would make an unrelated file upgrade wait on it. Re-recording one already in the ledger is a no-op, not an error |
| `migrate baseline <version> [--force]` | Declare `kb_version` once, for an instance predating `.wikitool-kb.json`. Refuses to overwrite an existing declaration without `--force`: advancing after a migration is `done`, which checks the chain, and this command must not become the quiet way around it | | `migrate baseline <version> [--force]` | Declare `kb_version` once, for an instance predating `.wikitool-kb.json`. Refuses to overwrite an existing declaration without `--force`: advancing after a migration is `done`, which checks the chain, and this command must not become the quiet way around it |
| `upstream merge [--remote upstream] [--branch main] [--no-fetch]` | Take a stack update into a private instance's branch, machinery only - the code procedure behind `instructions/private-instance.md` § "Taking a stack update". Refuses on a dirty working tree, a merge already in progress, or a remote that does not resolve; WARNs (does not block) when `.wikitool-remotes.json` is absent, pointing at the setup step that arms it. Fetches `<remote>/<branch>` (unless `--no-fetch`) and reports "already up to date" if nothing new exists. Otherwise opens `git merge --no-commit --no-ff <remote>/<branch>` - and stops, untouched, if git refused to open a merge at all (unrelated histories), since without a `MERGE_HEAD` every stack path would read as "the upstream deleted it". Then forces every content stage (`kb/`, `raw/`, `work/`, `reports/`) back to the local side by removing **only the paths tracked in either tree** and checking `HEAD`'s back out - never the stage directory wholesale, because `reports/` is gitignored apart from its contract and holds local, non-recomputable data (telemetry traces `eval score` reads, saved eval and lint reports) that no merge has business deleting. Then restores from the upstream side exactly the paths `chemenu.ownership.is_stack_owned` recognises as machinery (`<stage>/CONTRACT.md`, and anything ending `.template` under a content stage) - including a deletion, if the upstream removed one. A real conflict left in `tools/`, `types/` or `instructions/` after that leaves the merge open, uncommitted, and exits 1 rather than guessing. Commits with `git commit --no-edit`, then re-checks the resulting range with the same logic as `upstream verify`; a finding there is a loud, uncommitted-nothing-rolled-back error, because the merge commit already exists and needs a human's eyes, not an automatic repair. Never pushes. Not idempotent - see the tool error contract below |
| `upstream verify --since <rev> [--until HEAD]` | Compare two revisions: did anything under a content stage (`kb/`, `raw/`, `work/`, `reports/`) change except through a stack-owned path? Shares its check with `upstream merge`'s own postcheck, so a hand-resolved merge conflict, or a future `dist upgrade` (#7), can be verified the same way. Exit 1 with the offending paths if anything leaked; otherwise reports which stack-owned paths legitimately moved. Read-only and exempt from the Iteration Budget Gate, like `migrate verify` |
| `doctor [--json]` | Check that this instance is correctly configured: dependencies (Python, ripgrep), author resolution, stack version, git identity/branch/remote, published skills, kb/raw/reports/work/instructions structure, personalization (`USER.md`/`SOUL.md` present **and** filled - a file still carrying the template's sentinel is a `FAIL`, since a renamed template is not a filled one), the KB conventions (`kb/CONVENTIONS.md` present, unsentinelled, and naming all three tool-owned section headings - a `FAIL` on any of the three, because `xref`/`cite` write out of it), the environment note (`ENVIRONMENT.md` - optional, so absent is `OK`; a still-templated one is a `WARN`), generated files, and `WIKITOOL_SESSION_ID`. Read-only, exit 1 only on a `FAIL` (a missing remote, session id, or `VERSION` is a `WARN`, not a fault). Exempt from the Iteration Budget Gate | | `doctor [--json]` | Check that this instance is correctly configured: dependencies (Python, ripgrep), author resolution, stack version, git identity/branch/remote, published skills, kb/raw/reports/work/instructions structure, personalization (`USER.md`/`SOUL.md` present **and** filled - a file still carrying the template's sentinel is a `FAIL`, since a renamed template is not a filled one), the KB conventions (`kb/CONVENTIONS.md` present, unsentinelled, and naming all three tool-owned section headings - a `FAIL` on any of the three, because `xref`/`cite` write out of it), the environment note (`ENVIRONMENT.md` - optional, so absent is `OK`; a still-templated one is a `WARN`), generated files, and `WIKITOOL_SESSION_ID`. Read-only, exit 1 only on a `FAIL` (a missing remote, session id, or `VERSION` is a `WARN`, not a fault). Exempt from the Iteration Budget Gate |
## Design notes ## Design notes
@@ -190,6 +193,7 @@ is atomic, and whether a retry is safe.
| `instructions sync` / `verify` / `list` | Nothing found under `instructions/`, a malformed instruction or `SKILL.md`, a published copy that drifted from its source, an instruction nothing references (or, for `manual: true`, one that IS linked from AGENTS.md or a skill and so risks running implicitly), something under `instructions/dev/` referenced from outside it and outside a dist:strip block, or (sync) a target directory that is not a published skill and `--force` was not passed | `sync` rewrites one directory per target (idempotent); `verify`/`list` are read-only | Fix the flagged file, then re-run. For drift, re-run `sync`: the source under `instructions/` always wins, and a published copy is never edited directly | | `instructions sync` / `verify` / `list` | Nothing found under `instructions/`, a malformed instruction or `SKILL.md`, a published copy that drifted from its source, an instruction nothing references (or, for `manual: true`, one that IS linked from AGENTS.md or a skill and so risks running implicitly), something under `instructions/dev/` referenced from outside it and outside a dist:strip block, or (sync) a target directory that is not a published skill and `--force` was not passed | `sync` rewrites one directory per target (idempotent); `verify`/`list` are read-only | Fix the flagged file, then re-run. For drift, re-run `sync`: the source under `instructions/` always wins, and a published copy is never edited directly |
| `docs verify` | A command, contract, or type-form mismatch was found | Read-only | Fix the documentation it names, then re-run | | `docs verify` | A command, contract, or type-form mismatch was found | Read-only | Fix the documentation it names, then re-run |
| `dist export` | Target exists and is not empty, is not a directory, or the tree has no readable `VERSION` | Yes - nothing is written until every file is planned | Point `<target>` at an empty (or new) directory and retry. Never merge into a non-empty one by hand | | `dist export` | Target exists and is not empty, is not a directory, or the tree has no readable `VERSION` | Yes - nothing is written until every file is planned | Point `<target>` at an empty (or new) directory and retry. Never merge into a non-empty one by hand |
| `dist upgrade` | Missing local `VERSION`/`.wikitool-release.json`(`files`)/`.wikitool-kb.json`, a migration already outstanding against the installed machinery, a dirty working tree, a source with no `VERSION`/stamp/`files` block, a source version that is older than, equal to, or (without `--pre`) a pre-release relative to the installed one, or one or more locally changed files without `--keep-local` | **Yes for the refusal cases above - nothing is written.** Once writing starts it is a plain sequential file copy with no partial-state cleanup: an interruption mid-copy (killed process, disk full) can leave the tree part-old, part-new | For every refusal above: fix the named precondition and retry - none of them are transient. For locally changed files: reconcile them by hand and retry, or re-run with `--keep-local` to proceed and leave them untouched (repeatable - it reports the same files again on every subsequent run until they stop diverging). An interrupted write is not resumed automatically; compare the tree against the printed classification and finish or revert by hand |
| `version show` / `version notes` | `VERSION` is missing or unparseable; for `notes`, no `CHANGES.md` entry names the version asked for | Read-only | Fix `VERSION`, or write the changelog entry (`version bump` writes its heading). Safe to retry | | `version show` / `version notes` | `VERSION` is missing or unparseable; for `notes`, no `CHANGES.md` entry names the version asked for | Read-only | Fix `VERSION`, or write the changelog entry (`version bump` writes its heading). Safe to retry |
| `version check` | The feed could not be reached, answered non-JSON, or carried no `tag_name`. **Never** answers "up to date" for a question it could not ask | Read-only, no local writes | A network failure is transient - retry once, then report it. HTTP 401/403 names `$WIKITOOL_UPDATE_TOKEN`; 404 means no release exists yet or the URL points at the wrong repo | | `version check` | The feed could not be reached, answered non-JSON, or carried no `tag_name`. **Never** answers "up to date" for a question it could not ask | Read-only, no local writes | A network failure is transient - retry once, then report it. HTTP 401/403 names `$WIKITOOL_UPDATE_TOKEN`; 404 means no release exists yet or the URL points at the wrong repo |
| `version bump` | More or fewer than one of `--major/--minor/--patch`, an empty `--title`, a missing `VERSION`/`CHANGES.md`, `VERSION` and the changelog's newest entry naming different versions, an escalation to a boundary crossing without `--breaking` or with neither a migration document nor `--no-migration`, or `--breaking`/`--no-migration` on a bump that crosses nothing | No - `VERSION` then `CHANGES.md` | **Not idempotent**: a second run escalates or continues the candidate again. If the outcome is uncertain, read `VERSION` and the top of `CHANGES.md` before retrying | | `version bump` | More or fewer than one of `--major/--minor/--patch`, an empty `--title`, a missing `VERSION`/`CHANGES.md`, `VERSION` and the changelog's newest entry naming different versions, an escalation to a boundary crossing without `--breaking` or with neither a migration document nor `--no-migration`, or `--breaking`/`--no-migration` on a bump that crosses nothing | No - `VERSION` then `CHANGES.md` | **Not idempotent**: a second run escalates or continues the candidate again. If the outcome is uncertain, read `VERSION` and the top of `CHANGES.md` before retrying |
@@ -199,6 +203,8 @@ is atomic, and whether a retry is safe.
| `migrate verify` | Only with `--fail-on-error`: an invariant changed. Also exits 1 if `--from` is not a revision in this repository | Read-only | Exit 1 from `--fail-on-error` means "act on the findings", not "the tool is broken". A finding is never fixed by re-running - it names a page and what changed on it | | `migrate verify` | Only with `--fail-on-error`: an invariant changed. Also exits 1 if `--from` is not a revision in this repository | Read-only | Exit 1 from `--fail-on-error` means "act on the findings", not "the tool is broken". A finding is never fixed by re-running - it names a page and what changed on it |
| `migrate done` | Unknown version, no `.wikitool-kb.json`, nothing outstanding, or a *required* version that is not the next link in the chain | Yes - single file write | **Not idempotent** for a required migration: it advances the chain. For "not the next link", run `migrate status` and apply them in the order it prints - never force the order. Recording an `offered` migration *is* idempotent and safe to repeat | | `migrate done` | Unknown version, no `.wikitool-kb.json`, nothing outstanding, or a *required* version that is not the next link in the chain | Yes - single file write | **Not idempotent** for a required migration: it advances the chain. For "not the next link", run `migrate status` and apply them in the order it prints - never force the order. Recording an `offered` migration *is* idempotent and safe to repeat |
| `migrate baseline` | Unparseable version, or a declaration already exists and `--force` was not passed | Yes - single file write | Safe to re-run with the same version. If a declaration exists, it is almost always `migrate done` that was wanted | | `migrate baseline` | Unparseable version, or a declaration already exists and `--force` was not passed | Yes - single file write | Safe to re-run with the same version. If a declaration exists, it is almost always `migrate done` that was wanted |
| `upstream merge` | Dirty working tree, a merge already in progress, the remote does not resolve, git refused to open the merge at all (unrelated histories), or a real conflict remains in `tools/`/`types/`/`instructions/` after the content stages and stack-owned paths were restored | **No** - can leave an open, uncommitted merge behind on refusal after fetching | **Not idempotent, and not safe to retry unchanged.** For a dirty tree or an in-progress merge: fix the named precondition and retry once. For a real conflict: **do not retry, do not force** - resolve the named paths by hand (take the upstream side, or re-file the local change as an issue against the public repo per `instructions/private-instance.md`) and either `git commit --no-edit` yourself or `git merge --abort`. If the postcheck after commit finds a leak, the merge commit already exists and is **not** rolled back automatically - inspect it by hand; this is a bug report, not a retry |
| `upstream verify` | A leak was found (content changed under a content stage through a path that is not stack-owned), or `--since`/`--until` is not a revision in this repository | Read-only | A finding is not fixed by re-running - it names the paths that leaked. Fix the revision argument and retry for the second case |
| `doctor` | At least one check reported `FAIL` (a `WARN`, e.g. no remote or no `WIKITOOL_SESSION_ID`, does not exit 1) | Read-only | Each finding names its own fix command; re-run after applying it | | `doctor` | At least one check reported `FAIL` (a `WARN`, e.g. no remote or no `WIKITOOL_SESSION_ID`, does not exit 1) | Read-only | Each finding names its own fix command; re-run after applying it |
| `budget status` / `budget reset` | `reset` without `--yes`; `status` never fails | Read/rewrite of one JSON file | `status` is safe to retry. For `reset`: get the user's approval, then re-run with `--yes` | | `budget status` / `budget reset` | `reset` without `--yes`; `status` never fails | Read/rewrite of one JSON file | `status` is safe to retry. For `reset`: get the user's approval, then re-run with `--yes` |
| `eval sessions` | Never fails; an empty list is a valid answer | Read-only | - | | `eval sessions` | Never fails; an empty list is a valid answer | Read-only | - |
+1
View File
@@ -43,6 +43,7 @@ tools/
links.py labelled edges in `related:` - the graph's semantics as data, not prose links.py labelled edges in `related:` - the graph's semantics as data, not prose
kb_collections.py collection discovery (a directory with COLLECTION.md), and what one declares about itself kb_collections.py collection discovery (a directory with COLLECTION.md), and what one declares about itself
conventions.py kb/CONVENTIONS.md: what this instance decided about authoring, as opposed to what the stack enforces conventions.py kb/CONVENTIONS.md: what this instance decided about authoring, as opposed to what the stack enforces
ownership.py the stack-vs-instance boundary under a content stage - one predicate, read by `dist_cmd.py` and `commands/upstream_cmd.py` so the two cannot answer it differently
type_resolver.py type-spec loading and schema resolution type_resolver.py type-spec loading and schema resolution
lint_core.py the lint checks and the report, with no CLI attached lint_core.py the lint checks and the report, with no CLI attached
types_core.py type-spec listing/description, with no CLI attached types_core.py type-spec listing/description, with no CLI attached
+2
View File
@@ -31,6 +31,7 @@ try:
search as search_module, search as search_module,
touch as touch_module, touch as touch_module,
types_cmd, types_cmd,
upstream_cmd,
version_cmd, version_cmd,
work_cmd, work_cmd,
xref, xref,
@@ -69,6 +70,7 @@ app.add_typer(eval_cmd.app, name="eval")
app.add_typer(dist_cmd.app, name="dist") app.add_typer(dist_cmd.app, name="dist")
app.add_typer(version_cmd.app, name="version") app.add_typer(version_cmd.app, name="version")
app.add_typer(migrate_cmd.app, name="migrate") app.add_typer(migrate_cmd.app, name="migrate")
app.add_typer(upstream_cmd.app, name="upstream")
app.command("new")(new_page.new_page_command) app.command("new")(new_page.new_page_command)
app.command("touch")(touch_module.touch_command) app.command("touch")(touch_module.touch_command)
app.command("rename")(page_ops.rename_command) app.command("rename")(page_ops.rename_command)
+435 -15
View File
@@ -33,14 +33,20 @@ import hashlib
import json import json
import os import os
import re import re
import shutil
import stat import stat
import subprocess
import tarfile
import tempfile
from contextlib import contextmanager
from dataclasses import dataclass
from pathlib import Path from pathlib import Path
from typing import Callable, NamedTuple, Optional, Union from typing import Callable, NamedTuple, Optional, Union
import typer import typer
from chemenu import config, conventions, kb_collections, kb_state, version as version_mod from chemenu import config, conventions, kb_collections, kb_state, ownership, version as version_mod
from chemenu.commands._util import fail, rel_path, success, today_iso from chemenu.commands._util import console, fail, rel_path, success, today_iso
app = typer.Typer(help="Build a distributable copy of the wiki machinery.") app = typer.Typer(help="Build a distributable copy of the wiki machinery.")
@@ -132,8 +138,15 @@ INSTRUCTIONS_EXCLUDE_DIRS = {"dev"}
RAW_SUBDIRS = ("articles", "documents", "notes", "assets") RAW_SUBDIRS = ("articles", "documents", "notes", "assets")
# Stage contracts that are not collections and carry no pages: copied as a # Stage contracts that are not collections and carry no pages: copied as a
# single file each, nothing else from their directory. # single file each, nothing else from their directory. `kb/` is excluded here
CONTRACT_ONLY_STAGES = ("raw/CONTRACT.md", "reports/CONTRACT.md", "work/CONTRACT.md") # - it is a content stage too, but it has collections underneath it, so its
# contract is handled by `build_plan` alongside them rather than as a bare
# stage copy. Derived from `ownership.CONTENT_STAGES` rather than listed
# again, so the set this loop copies and the set `upstream merge` restores
# cannot name a different stage without one of them failing its own test.
CONTRACT_ONLY_STAGES = tuple(
f"{stage}/CONTRACT.md" for stage in ownership.CONTENT_STAGES if stage != "kb"
)
# Single tracked files copied out of an otherwise-untouched, partially-ignored # Single tracked files copied out of an otherwise-untouched, partially-ignored
# directory. `.claude/` holds the harness's own session-tracing config # directory. `.claude/` holds the harness's own session-tracing config
@@ -439,19 +452,20 @@ def build_plan(origin: Optional[Origin] = None) -> dict[str, PlannedFile]:
# appears in INSTALL.md and version.py, so such a scan would either whitelist # appears in INSTALL.md and version.py, so such a scan would either whitelist
# the very string it is looking for or cry wolf on every export. # the very string it is looking for or cry wolf on every export.
# #
# `COLLECTION.md` and `CONVENTIONS.md` are deliberately *not* on the allowed # `COLLECTION.md` and `CONVENTIONS.md` are deliberately *not* allowed through
# list any more. Both bind, and both are the instance's to write, so they cross # any more. Both bind, and both are the instance's to write, so they cross the
# the boundary as `.template` and are adopted by a rename - a plan carrying the # boundary as `.template` and are adopted by a rename - a plan carrying the
# filled name would hand a new instance this one's authoring conventions as # filled name would hand a new instance this one's authoring conventions as
# though they were the stack's. # though they were the stack's.
#
# What counts as machinery under kb/ or raw/ is no longer a second list here:
# it is `ownership.is_stack_owned`, the same predicate `upstream merge` and
# `upstream verify` restore/check against. Only the export-only stubs
# (`ownership.EXPORT_STUB_NAMES`) are allowed here without also being
# stack-owned - a merge keeps the *local* copy of those, while export writes a
# fresh one regardless of either side, so the two callers genuinely disagree
# about them and each keeps its own allowance for that one case.
_CONTENT_PREFIXES = ("kb/", "raw/") _CONTENT_PREFIXES = ("kb/", "raw/")
_CONTENT_ALLOWED_NAMES = (
"CONTRACT.md",
f"{kb_collections.CONTRACT_NAME}.template",
conventions.CONVENTIONS_TEMPLATE,
"log.md",
".gitkeep",
)
_INSTANCE_OWNED_KB_FILES = (kb_collections.CONTRACT_NAME, conventions.CONVENTIONS_FILENAME) _INSTANCE_OWNED_KB_FILES = (kb_collections.CONTRACT_NAME, conventions.CONVENTIONS_FILENAME)
@@ -473,7 +487,11 @@ def find_leaks(plan: dict[str, PlannedFile]) -> list[str]:
leaks.append(f"{relative} (this instance's page type-spec; ship the .template)") leaks.append(f"{relative} (this instance's page type-spec; ship the .template)")
elif relative.startswith("instructions/dev/"): elif relative.startswith("instructions/dev/"):
leaks.append(f"{relative} (stack-development only)") leaks.append(f"{relative} (stack-development only)")
elif relative.startswith(_CONTENT_PREFIXES) and name not in _CONTENT_ALLOWED_NAMES: elif (
relative.startswith(_CONTENT_PREFIXES)
and not ownership.is_stack_owned(relative)
and not ownership.is_export_stub(name)
):
leaks.append(f"{relative} (wiki content, not machinery)") leaks.append(f"{relative} (wiki content, not machinery)")
return leaks return leaks
@@ -569,3 +587,405 @@ def run_export(target: Path, dry_run: bool = False, origin: Optional[Origin] = N
_write_plan(target, plan) _write_plan(target, plan)
success(f"Exported {len(plan)} file(s) to {rel_path(target)}.") success(f"Exported {len(plan)} file(s) to {rel_path(target)}.")
# --- dist upgrade ------------------------------------------------------------
#
# Apply a release `dist export` produced, rather than merely detecting one
# (`version check`). Gitea #7 has the full design; the short version: the
# write set is exactly the *new* stamp's `files` block, minus the paths an
# export seeds once and the instance owns from then on
# (`ownership.is_export_stub`, `ownership.is_upgrade_preserved`), plus the
# stamp itself. Every candidate path is classified against the *old* stamp's
# recorded digest - unchanged, locally modified, or locally deleted - and a
# modified/deleted file is never silently overwritten. This never calls a
# release feed; the caller supplies an already-downloaded tree or archive.
@dataclass(frozen=True)
class FileClassification:
"""The four-way split of every path `dist upgrade` would touch, plus the
fifth direction (`removed`) that has no write set of its own."""
unchanged: list[str]
modified: list[str]
deleted: list[str]
new: list[str]
removed: list[str]
@property
def blocked(self) -> list[str]:
"""Locally changed paths - modified or deleted - which are never
silently overwritten."""
return sorted(self.modified + self.deleted)
def _write_candidates(new_files: dict) -> set[str]:
"""Every path `dist upgrade` may write, from the new stamp's `files`
block: everything except the paths an export re-seeds from a blank
template every time (`ownership.is_export_stub`) and the paths an export
seeds once and the instance owns afterward (`ownership.is_upgrade_preserved`).
The release stamp itself is added separately - it is never a member of its
own `files` block, see `build_stamp`."""
return {
relative
for relative in new_files
if not ownership.is_export_stub(Path(relative).name)
and not ownership.is_upgrade_preserved(relative)
}
def _classify_files(old_files: dict, new_files: dict) -> FileClassification:
candidates = _write_candidates(new_files)
recorded_for_candidates = {r: d for r, d in old_files.items() if r in candidates}
statuses = kb_state.compare_against_stamp(recorded_for_candidates)
unchanged: list[str] = []
modified: list[str] = []
deleted: list[str] = []
new: list[str] = []
for relative in sorted(candidates):
if relative not in old_files:
new.append(relative)
continue
status = statuses[relative]
if status == kb_state.UNCHANGED:
unchanged.append(relative)
elif status == kb_state.MODIFIED:
modified.append(relative)
else:
deleted.append(relative)
removed = sorted(set(old_files) - set(new_files))
return FileClassification(unchanged, modified, deleted, new, removed)
def _verify_sha256_sidecar(archive: Path) -> None:
"""WARN, never fail, on a missing sidecar - only a corrupted one that
*is* present is a reason to stop, per Gitea #7's design table."""
sidecar = archive.with_name(archive.name + ".sha256")
if not sidecar.is_file():
console.print(
f"[yellow]WARN[/yellow] No {sidecar.name} beside {archive.name} - the archive's "
"integrity is not being checked before it is extracted."
)
return
expected = sidecar.read_text(encoding="utf-8").strip().split()[0:1]
actual = hashlib.sha256(archive.read_bytes()).hexdigest()
if not expected or expected[0].lower() != actual.lower():
fail(
f"{archive.name} does not match {sidecar.name}: expected "
f"{expected[0] if expected else '(unreadable)'}, got {actual}. Re-download the "
"release archive rather than trusting one that failed its own checksum."
)
def _extract_single_top_level_dir(archive: Path, dest: Path) -> Path:
"""Extract `archive` into `dest` and return the one top-level directory it
contained - the shape `.gitea/workflows/release.yml` packs (see its
`Build the distribution tarball` step). Refuses anything else rather than
guessing which part is the machinery."""
with tarfile.open(archive) as tf:
names = [n for n in tf.getnames() if n not in ("", ".")]
top_levels = {n.split("/", 1)[0] for n in names}
if len(top_levels) != 1:
fail(
f"{archive.name} does not have exactly one top-level directory (found "
f"{len(top_levels)}: {', '.join(sorted(top_levels)) or '(empty archive)'}) - this "
"is not the shape a release tarball has, and `dist upgrade` refuses to guess "
"which part is the machinery."
)
return dest # unreachable: fail() raises typer.Exit
try:
tf.extractall(dest, filter="data") # noqa: S202 - trusted local archive, path-checked below
except TypeError:
# Python < 3.12 has no `filter=` argument. Same guard by hand:
# refuse any member whose extracted path would land outside dest.
resolved_dest = dest.resolve()
for member in tf.getmembers():
if not (resolved_dest / member.name).resolve().is_relative_to(resolved_dest):
fail(
f"{archive.name} contains a path that escapes the extraction directory: "
f"{member.name}"
)
return dest # unreachable
tf.extractall(dest) # noqa: S202 - every member path-checked above
return dest / next(iter(top_levels))
@contextmanager
def _resolved_source(source: Path):
"""Yield the directory holding a distribution export: `source` itself if
it already is one, or the single top-level directory of a `.tar.gz`
extracted into a scratch directory that is cleaned up afterward."""
if source.is_dir():
yield source
return
if not source.is_file():
fail(f"{source} does not exist.")
return
_verify_sha256_sidecar(source)
with tempfile.TemporaryDirectory(prefix="wikitool-upgrade-") as tmp:
yield _extract_single_top_level_dir(source, Path(tmp))
def _git_working_tree_status() -> Optional[str]:
"""`git status --porcelain` for `config.ROOT`, or None if it is not a git
repository at all - which is a valid, if unprotected, state for a tarball
instance, not a reason to refuse."""
result = subprocess.run(
["git", "-C", str(config.ROOT), "status", "--porcelain"],
capture_output=True,
text=True,
)
return result.stdout if result.returncode == 0 else None
def _report_plan(
classification: FileClassification,
migration_chain: list["kb_state.Migration"],
boundary_crossing: bool,
local_version: "version_mod.Version",
new_version: "version_mod.Version",
) -> None:
console.print(f"{local_version} -> {new_version}")
if boundary_crossing:
console.print(
f"[bold yellow]Crosses a compatibility boundary[/bold yellow] "
f"({local_version.compat_key} -> {new_version.compat_key}) - this is not a drop-in "
"swap; check the release notes before proceeding."
)
console.print(
f"{len(classification.unchanged)} unchanged, {len(classification.new)} new, "
f"{len(classification.blocked)} locally changed, {len(classification.removed)} removed "
"from the release."
)
if classification.modified:
console.print(f"[bold]Locally modified ({len(classification.modified)}):[/bold]")
for relative in classification.modified:
console.print(f" - {relative}")
if classification.deleted:
console.print(f"[bold]Locally deleted ({len(classification.deleted)}):[/bold]")
for relative in classification.deleted:
console.print(f" - {relative}")
if classification.removed:
console.print("[dim]No longer part of the release, not written or removed by default:[/dim]")
for relative in classification.removed:
console.print(f" [dim]- {relative}[/dim]")
if migration_chain:
console.print(
f"[cyan]{len(migration_chain)} migration(s) will be outstanding after this "
"upgrade, in this order:[/cyan]"
)
for position, migration in enumerate(migration_chain, start=1):
console.print(f" {position}. {migration.target} {migration.name} ({migration.kind})")
console.print("Report only - `dist upgrade` never runs a migration. See `wikitool migrate status`.")
@app.command("upgrade")
def upgrade_command(
source: Path = typer.Argument(
..., help="An extracted distribution directory, or a release .tar.gz archive"
),
dry_run: bool = typer.Option(
False, "--dry-run", help="Classify and report, without writing anything"
),
keep_local: bool = typer.Option(
False, "--keep-local",
help="Proceed even with locally changed files - leave each one untouched rather than aborting",
),
prune: bool = typer.Option(
False, "--prune",
help="Also delete files the new release no longer ships, if they are unchanged since install",
),
allow_pre: bool = typer.Option(
False, "--pre", help="Allow a pre-release (-beta.N) source tree - release.yml never publishes one",
),
):
"""Apply a stack update `dist export` produced - the write half of
`version check`. Never downloads anything: `source` is an already-fetched
export directory or `.tar.gz` archive. Writes exactly the new release
stamp's `files` block, minus what an export re-seeds every time
(`kb/log.md`, `raw/*/.gitkeep`) or seeds once and the instance owns from
then on (`.wikitool-kb.json`, `CHANGES.md`), classifying every candidate
against the *old* stamp's recorded digest: unchanged files are
overwritten silently, new files are created, and a locally modified or
deleted file is never silently overwritten - `dist upgrade` aborts unless
`--keep-local` says to leave it alone. Reports the migration chain the new
machinery would owe without running any of it (there is no `migrate run`).
Refuses on a missing local release stamp, a downgrade, a pre-release
source without `--pre`, or a dirty working tree. Never touches git.
See Gitea #7 and `INSTALL.md` § "Eine Instanz aktualisieren"."""
run_upgrade(
source, dry_run=dry_run, keep_local=keep_local, prune=prune, allow_pre=allow_pre
)
def run_upgrade(
source: Path,
dry_run: bool = False,
keep_local: bool = False,
prune: bool = False,
allow_pre: bool = False,
) -> None:
"""The upgrade itself, free of Typer's option objects - see `run_export`
for why this split exists."""
try:
local_version = version_mod.read_version()
except version_mod.VersionError as exc:
fail(f"{exc} - this tree has no stack version to upgrade from.")
return
old_stamp = version_mod.read_stamp()
if not old_stamp or not isinstance(old_stamp.get("files"), dict):
fail(
f"No local {version_mod.RELEASE_STAMP_FILENAME} (or it carries no `files` block). "
"Without it, `dist upgrade` cannot tell a file this instance edited from one it "
"merely received, and it refuses to guess. A checkout with shared git history takes "
"stack updates via `wikitool upstream merge` instead - it has the same information "
"as a merge base. A tarball instance that has lost its stamp has no repair path "
"today; see Gitea #7 \"Bewusst offen gelassen\"."
)
return
old_files = old_stamp["files"]
try:
kb_version = kb_state.read_kb_version()
except version_mod.VersionError as exc:
fail(str(exc))
return
if kb_version is None:
fail(
f"{kb_state.KB_STATE_FILENAME} is missing - this instance has never declared what "
"shape its content is in. Run `wikitool migrate baseline <version>` before upgrading."
)
return
outstanding = kb_state.chain(kb_state.load_migrations(), kb_version, local_version.base)
if outstanding:
fail(
f"{len(outstanding)} migration(s) are already outstanding against the installed "
f"machinery ({kb_version} -> {local_version}) - `wikitool migrate status` names them. "
"Finish them before upgrading further: a machinery swap on top of an unfinished "
"migration leaves the corpus in a shape no version describes."
)
return
tree_status = _git_working_tree_status()
if tree_status is None:
console.print(
"[yellow]WARN[/yellow] Not a git repository (or git is unavailable) - proceeding "
"without the dirty-tree check a repository would get."
)
elif tree_status.strip():
fail(
"Working tree is not clean (`git status --porcelain` printed something). "
"`dist upgrade` refuses to start on a dirty tree so a refusal never has to guess "
"which changes were already there. Commit or stash first."
)
return
with _resolved_source(source) as new_root:
version_path = new_root / version_mod.VERSION_FILENAME
if not version_path.is_file():
fail(f"{rel_path(new_root)} has no VERSION - not a distribution export.")
return
try:
new_version = version_mod.Version.parse(version_path.read_text(encoding="utf-8"))
except version_mod.VersionError as exc:
fail(str(exc))
return
if new_version.is_prerelease and not allow_pre:
fail(
f"{new_version} is a running candidate (-beta.N). `.gitea/workflows/release.yml` "
"never publishes one, so a candidate tree can only come from a dev checkout by "
"hand - pass --pre if that is deliberate."
)
return
if new_version < local_version:
fail(f"{new_version} is older than the installed {local_version} - refusing a downgrade.")
return
if new_version == local_version:
success(f"Already at {local_version}. Nothing to do.")
return
stamp_path = new_root / version_mod.RELEASE_STAMP_FILENAME
if not stamp_path.is_file():
fail(f"{rel_path(new_root)} has no {version_mod.RELEASE_STAMP_FILENAME} - not a distribution export.")
return
try:
new_stamp = json.loads(stamp_path.read_text(encoding="utf-8"))
except (json.JSONDecodeError, OSError) as exc:
fail(f"{version_mod.RELEASE_STAMP_FILENAME} in the source is not readable JSON: {exc}")
return
new_files = new_stamp.get("files") if isinstance(new_stamp, dict) else None
if not isinstance(new_files, dict):
fail(f"{version_mod.RELEASE_STAMP_FILENAME} in the source carries no `files` block.")
return
classification = _classify_files(old_files, new_files)
migration_chain = kb_state.chain(
kb_state.load_migrations(new_root / "instructions" / kb_state.MIGRATIONS_SUBDIR),
kb_version,
new_version.base,
)
boundary_crossing = local_version.compat_key != new_version.compat_key
_report_plan(classification, migration_chain, boundary_crossing, local_version, new_version)
# Dry-run's whole purpose is to preview this classification - including
# the blocked list - without raising, so it must be checked before the
# abort below rather than after: a blocked file must never turn
# `--dry-run` into a non-zero exit, or the flag stops being safe to run
# freely.
if dry_run:
success(f"Dry run: would upgrade {local_version} -> {new_version}. Nothing written.")
return
if classification.blocked and not keep_local:
fail(
f"{len(classification.blocked)} locally changed file(s) (listed above) would be "
"silently overwritten. Pass --keep-local to upgrade anyway and leave every one of "
"them untouched, or reconcile them by hand first. Nothing was written."
)
return
to_write = sorted(classification.unchanged + classification.new)
for relative in to_write:
src = new_root / relative
dst = config.ROOT / relative
dst.parent.mkdir(parents=True, exist_ok=True)
shutil.copy2(src, dst)
shutil.copy2(stamp_path, config.ROOT / version_mod.RELEASE_STAMP_FILENAME)
pruned: list[str] = []
if prune:
for relative in classification.removed:
digest = old_files.get(relative)
if digest is None:
continue
status = kb_state.compare_against_stamp({relative: digest}).get(relative)
if status != kb_state.UNCHANGED:
continue
target = config.ROOT / relative
if target.is_file():
target.unlink()
pruned.append(relative)
skipped = classification.blocked if keep_local else []
summary = (
f"Upgraded {local_version} -> {new_version}: {len(to_write)} file(s) written"
+ (f", {len(skipped)} left untouched (--keep-local)" if skipped else "")
+ (f", {len(pruned)} pruned" if pruned else "")
+ "."
)
if migration_chain:
summary += (
f" {len(migration_chain)} migration(s) now outstanding - run `wikitool migrate status`."
)
summary += (
" Nothing was committed. Now run, in order: `wikitool instructions sync`, `doctor`, "
"`docs verify`, `instructions verify`, `lint` - then restart the agent session."
)
success(summary)
+7
View File
@@ -101,6 +101,13 @@ SKIP_COMMAND_PATHS = {
("migrate", "list"), ("migrate", "list"),
("migrate", "status"), ("migrate", "status"),
("migrate", "verify"), ("migrate", "verify"),
# `upstream verify` only reads two git revisions and reports what changed -
# the same argument as `migrate verify`: a check that costs budget is one
# an agent starts skipping. `upstream merge` stays counted: it mutates the
# branch and can leave an open merge behind on refusal, so it belongs on
# the non-idempotent list (AGENTS.md's tool error contract) rather than
# the exempt one.
("upstream", "verify"),
} }
# Commands exempt regardless of their first argument, because that argument is # Commands exempt regardless of their first argument, because that argument is
+387
View File
@@ -0,0 +1,387 @@
"""`wikitool upstream` - take a stack update from a public upstream into a
private instance's `main` without letting the upstream's own content (a demo
corpus, a workshop run) ride along.
`git merge upstream/main` on its own treats a moved corpus dangerously
asymmetrically: a page the instance deleted and the upstream edited reports as
a conflict, a page the upstream *added* stages silently, and a page both sides
deleted is the only harmless case. `instructions/private-instance.md`'s prose
procedure closes that, by holding the merge open, forcing the content stages
(`ownership.CONTENT_STAGES`) back to the local side, and then restoring only
the paths `ownership.is_stack_owned` recognises as machinery. `upstream merge`
is that procedure in code, so the path set it acts on cannot drift from the
one `dist_cmd.py` ships - both read `chemenu.ownership` - and so a conflict in
the machinery layers, or a machinery file the upstream deleted, gets an
explained stop instead of a silently wrong commit.
`upstream verify` is the other half: given two revisions, did anything change
under a content stage except through a stack-owned path? It shares
`_content_leaks` with the postcheck `upstream merge` runs on itself, so a
hand-resolved merge or a future `dist upgrade` (Gitea #7) can be checked the
same way.
"""
from __future__ import annotations
from pathlib import Path
from typing import Optional
import typer
from chemenu import config, ownership
from chemenu.commands import git_publish
from chemenu.commands._util import console, fail, success
app = typer.Typer(help="Take a stack update from a public upstream, machinery only.")
def _run(args: list[str]):
import subprocess
return subprocess.run(args, cwd=config.ROOT, capture_output=True, text=True)
def _rev_parse(rev: str) -> Optional[str]:
result = _run(["git", "rev-parse", "--verify", "-q", rev])
return result.stdout.strip() if result.returncode == 0 else None
def _git_dir() -> Optional[Path]:
result = _run(["git", "rev-parse", "--git-dir"])
if result.returncode != 0:
return None
path = Path(result.stdout.strip())
return path if path.is_absolute() else config.ROOT / path
def _working_tree_dirty() -> bool:
result = _run(["git", "status", "--porcelain"])
return bool(result.stdout.strip())
def _merge_in_progress() -> bool:
git_dir = _git_dir()
return git_dir is not None and (git_dir / "MERGE_HEAD").exists()
def _remote_resolves(remote: str) -> bool:
return _run(["git", "remote", "get-url", remote]).returncode == 0
def _is_ancestor(ancestor: str, of: str) -> bool:
return _run(["git", "merge-base", "--is-ancestor", ancestor, of]).returncode == 0
def _tree_has_path(rev: str, path: str) -> bool:
return _run(["git", "rev-parse", "--verify", "-q", f"{rev}:{path}"]).returncode == 0
def _tree_paths(rev: str) -> set[str]:
result = _run(["git", "ls-tree", "-r", "--name-only", "-z", rev])
if result.returncode != 0:
return set()
return {p for p in result.stdout.split("\0") if p}
def _content_leaks(since: str, until: str) -> list[str]:
"""Paths under a content stage that changed between `since` and `until`
through something other than a stack-owned path. Shared by `upstream
merge`'s own postcheck and `upstream verify`, so the two cannot disagree
about what a clean update looks like."""
result = _run(["git", "diff", "--name-only", "-z", since, until, "--", *ownership.CONTENT_STAGES])
if result.returncode != 0:
fail(
f"`git diff {since} {until}` failed - is {since} a revision in this repository?\n"
f"{result.stderr}"
)
return []
changed = [p for p in result.stdout.split("\0") if p]
return sorted(p for p in changed if not ownership.is_stack_owned(p))
def _stack_paths_changed(since: str, until: str) -> list[str]:
"""The subset of the same diff that *is* a stack-owned path - the paths
that legitimately moved, for the success message."""
result = _run(["git", "diff", "--name-only", "-z", since, until, "--", *ownership.CONTENT_STAGES])
changed = [p for p in result.stdout.split("\0") if p]
return sorted(p for p in changed if ownership.is_stack_owned(p))
# --- upstream merge ---------------------------------------------------------
def _prune_empty_dirs(stage: str) -> None:
"""Remove directories left empty under `stage` after tracked files were
deleted. git tracks no directories, so an emptied one is invisible to
`git status` and would otherwise linger in the working tree as litter -
an empty `kb/<area>/` that only ever existed in the upstream's corpus.
Never touches a directory that still holds anything, ignored files
included."""
stage_dir = config.ROOT / stage
if not stage_dir.is_dir():
return
for path in sorted(stage_dir.rglob("*"), key=lambda p: len(p.parts), reverse=True):
if path.is_dir() and not any(path.iterdir()):
path.rmdir()
def _restore_stage_to_local(stage: str, tracked_paths: set[str]) -> None:
"""Force one content stage back to the local (HEAD) side, whatever the
merge did to it.
Deletes **only what git tracks on either side** - never the stage
directory wholesale. That distinction is the whole point of this function:
`reports/` is gitignored except its contract (see .gitignore), so a
content stage's working tree legitimately holds local data that is not in
any tree and not recomputable - the telemetry traces `eval score` reads,
saved eval reports, past lint reports. A blanket `rm -rf` of the stage
takes all of it out as collateral for a merge that was never about it.
Handles a stage that exists only in MERGE_HEAD too (the upstream
introduced it): what the merge wrote is removed, and there is simply
nothing to check out from HEAD afterwards.
"""
prefix = f"{stage}/"
stage_paths = [p for p in tracked_paths if p.startswith(prefix)]
if not stage_paths:
return
_run(["git", "rm", "-rq", "--cached", "--ignore-unmatch", stage])
for relative in stage_paths:
target = config.ROOT / relative
if target.is_file() or target.is_symlink():
target.unlink()
_prune_empty_dirs(stage)
if _tree_has_path("HEAD", stage):
_run(["git", "checkout", "HEAD", "--", stage])
def _remote_gate_warning() -> None:
if git_publish.read_allowed_push_urls() is not None:
return
console.print(
"[bold yellow]WARN[/bold yellow] No .wikitool-remotes.json in this checkout - the "
"Publish-Remote Gate is unarmed, so a future `publish` to the wrong remote would not "
"be caught. `upstream merge` never pushes and proceeds regardless, but a checkout that "
"takes stack updates from a public upstream should arm the gate before its next publish "
"- see instructions/private-instance.md step 4."
)
def _precondition_failure(remote: str) -> Optional[str]:
if _working_tree_dirty():
return (
"Working tree is not clean (`git status --porcelain` printed something). "
"`upstream merge` refuses to start on a dirty tree so a refusal never has to "
"guess which changes were already there. Commit or stash first."
)
if _merge_in_progress():
return (
"A merge is already in progress (.git/MERGE_HEAD exists). Resolve or abort it "
"(`git merge --abort`) before running `upstream merge`."
)
if not _remote_resolves(remote):
return f"Remote '{remote}' does not resolve (`git remote get-url {remote}` failed)."
return None
def _unresolved_conflict_message(unresolved: list[str], remote: str, branch: str) -> str:
listed = "\n".join(f" - {p}" for p in unresolved)
return (
f"A real conflict remains in the machinery layers after restoring the content stages "
f"and the stack-owned paths from {remote}/{branch}:\n{listed}\n\n"
"The merge is left open, uncommitted - nothing was written to the branch. Per "
"instructions/private-instance.md's decision points: this means the checkout changed "
"the stack locally, which private instances do not do. Take the upstream side for "
"these paths (`git checkout --theirs -- <path>` then `git add`) and re-file the local "
"change as an issue against the public repo, or resolve deliberately and "
"`git commit --no-edit` yourself. `git merge --abort` gives up the merge entirely."
)
def _postcheck_failure_message(leaks: list[str], before: str) -> str:
listed = "\n".join(f" - {p}" for p in leaks)
return (
f"The merge commit exists (content stages are not what they were before this ran), "
f"but it changed content outside of a stack-owned path:\n{listed}\n\n"
f"This was NOT rolled back - the state belongs in front of you, not behind an automatic "
f"repair the command applies to itself. Compare against the pre-merge commit ({before}) "
"and decide by hand whether to revert the merge commit, cherry-pick around it, or fix "
"forward. This is a bug in `upstream merge` or in `ownership.is_stack_owned` if it "
"reproduces - please report it rather than working around it silently."
)
def _merge_success_message(
changed: list[str], deleted: list[str], remote: str, branch: str
) -> str:
"""What the merge actually did, measured against the pre-merge commit
rather than against what was restored.
`changed` is the real diff - restoring every stack-owned path from
MERGE_HEAD touches each of them whether or not the upstream moved any, so
reporting the restore list would claim seven updates for a merge that
changed one file, and a reader who checks would find the report wrong.
"""
deleted_set = set(deleted)
lines = [
f"Merged {remote}/{branch}. Content stages "
f"({', '.join(ownership.CONTENT_STAGES)}) are unchanged."
]
if changed:
lines.append(f"Stack paths changed ({len(changed)}):")
lines += [
f" - {p}" + (" (deleted, following the upstream)" if p in deleted_set else "")
for p in changed
]
else:
lines.append("No stack-owned path changed.")
return "\n".join(lines)
@app.command("merge")
def merge_command(
remote: str = typer.Option("upstream", "--remote", help="Remote to merge from"),
branch: str = typer.Option("main", "--branch", help="Branch to merge"),
no_fetch: bool = typer.Option(
False, "--no-fetch", help="Skip `git fetch <remote>` - use whatever is already fetched"
),
):
"""Merge `<remote>/<branch>` into the current branch, machinery only:
every path under a content stage (kb/, raw/, work/, reports/) is forced
back to the local side except a stack-owned path (`<stage>/CONTRACT.md`,
or anything ending `.template` under a content stage), which is taken
from the upstream - including a deletion, if the upstream removed one. A
real conflict elsewhere (tools/, types/, instructions/) leaves the merge
open and unresolved rather than guessing. Not idempotent: it can leave an
open merge behind on refusal. See instructions/private-instance.md."""
problem = _precondition_failure(remote)
if problem:
fail(problem)
return
_remote_gate_warning()
before = _rev_parse("HEAD")
if before is None:
fail("HEAD does not resolve - is this a git repository with at least one commit?")
return
if not no_fetch:
fetch_result = _run(["git", "fetch", remote, branch])
if fetch_result.returncode != 0:
fail(f"`git fetch {remote} {branch}` failed:\n{fetch_result.stderr}")
return
remote_ref = f"{remote}/{branch}"
if _rev_parse(remote_ref) is None:
fail(f"'{remote_ref}' does not resolve - fetch it first, or check --remote/--branch.")
return
if _is_ancestor(remote_ref, "HEAD"):
success(f"Already up to date with {remote_ref}.")
return
# The exit code is deliberately not the test - conflicts under the content
# stages are expected here and are exactly what the next steps undo. What
# *is* load-bearing is that a merge actually opened: without MERGE_HEAD,
# `_tree_paths("MERGE_HEAD")` is empty, and every stack-owned path in HEAD
# would then read as "the upstream deleted it" and be removed. A merge git
# refused to start (unrelated histories, an ignored file in the way) must
# therefore stop here, with the tree untouched.
merge_result = _run(["git", "merge", "--no-commit", "--no-ff", remote_ref])
if not _merge_in_progress():
fail(
f"`git merge --no-commit --no-ff {remote_ref}` did not open a merge, so there is "
f"nothing to scope - the working tree is unchanged:\n"
f"{merge_result.stdout}{merge_result.stderr}"
)
return
merge_head_paths = _tree_paths("MERGE_HEAD")
head_paths = _tree_paths("HEAD")
tracked_paths = merge_head_paths | head_paths
for stage in ownership.CONTENT_STAGES:
_restore_stage_to_local(stage, tracked_paths)
stack_paths = sorted(
p for p in (merge_head_paths | head_paths) if ownership.is_stack_owned(p)
)
# Only the deletions are recorded: what was *restored* is every stack-owned
# path in MERGE_HEAD, which is not the same question as what changed - the
# success message asks git for that instead.
deleted: list[str] = []
for relative in stack_paths:
if relative in merge_head_paths:
checkout = _run(["git", "checkout", "MERGE_HEAD", "--", relative])
if checkout.returncode != 0:
fail(
f"`git checkout MERGE_HEAD -- {relative}` failed even though it is listed "
f"in MERGE_HEAD's own tree:\n{checkout.stderr}\nThe merge is left open."
)
return
else:
_run(["git", "rm", "-q", "--cached", "--ignore-unmatch", relative])
target = config.ROOT / relative
if target.exists():
target.unlink()
deleted.append(relative)
unresolved = [p for p in _run(["git", "diff", "--name-only", "--diff-filter=U"]).stdout.splitlines() if p]
if unresolved:
fail(_unresolved_conflict_message(unresolved, remote, branch))
return
commit_result = _run(["git", "commit", "--no-edit"])
if commit_result.returncode != 0:
fail(f"`git commit --no-edit` failed:\n{commit_result.stderr}")
return
leaks = _content_leaks(before, "HEAD")
if leaks:
fail(_postcheck_failure_message(leaks, before))
return
success(
_merge_success_message(_stack_paths_changed(before, "HEAD"), deleted, remote, branch)
)
# --- upstream verify ---------------------------------------------------------
def _verify_failure_message(leaks: list[str], since: str, until: str) -> str:
listed = "\n".join(f" - {p}" for p in leaks)
return (
f"Content under a content stage (kb/, raw/, work/, reports/) changed between {since} "
f"and {until} through a path that is not stack-owned:\n{listed}\n\n"
"That is upstream content (or an equivalent local change) that reached this range "
"outside of a stack-owned path - inspect it before trusting this range as machinery-only."
)
def _verify_success_message(stack_moved: list[str], since: str, until: str) -> str:
if not stack_moved:
return f"No content changed between {since} and {until} under kb/, raw/, work/, reports/."
listed = "\n".join(f" - {p}" for p in stack_moved)
return (
f"Clean: only stack-owned paths changed under kb/, raw/, work/, reports/ between "
f"{since} and {until}:\n{listed}"
)
@app.command("verify")
def verify_command(
since: str = typer.Option(..., "--since", help="Git revision to compare from"),
until: str = typer.Option("HEAD", "--until", help="Git revision to compare to"),
):
"""Check that nothing under a content stage changed between --since and
--until except through a stack-owned path. Read-only, and exempt from the
Iteration Budget Gate - the same treatment `migrate verify` gets, for the
same reason: a check an agent has to ration is a check that gets skipped."""
leaks = _content_leaks(since, until)
if leaks:
fail(_verify_failure_message(leaks, since, until))
return
success(_verify_success_message(_stack_paths_changed(since, until), since, until))
+60 -27
View File
@@ -129,21 +129,27 @@ def migrations_dir() -> Path:
return config.INSTRUCTIONS_DIR / MIGRATIONS_SUBDIR return config.INSTRUCTIONS_DIR / MIGRATIONS_SUBDIR
def load_migrations() -> list[Migration]: def load_migrations(directory: Optional[Path] = None) -> list[Migration]:
"""Every migration document, sorted by target version. """Every migration document under `directory`, sorted by target version.
A malformed one is skipped rather than fatal here - `instructions verify` `directory` defaults to this instance's own `instructions/migrations/`.
is what reports it, and `migrate status` staying usable while one document `dist upgrade` (Gitea #7) passes the *new* tree's migrations directory
is broken is worth more than a second error path. instead: the migrations owed after an upgrade are documented in the
machinery being installed, not in the one still on disk - an old instance
cannot know a new version's migration chain by reading its own tree.
A malformed document is skipped rather than fatal here - `instructions
verify` is what reports it, and `migrate status` staying usable while one
document is broken is worth more than a second error path.
""" """
from chemenu.frontmatter_io import read_page from chemenu.frontmatter_io import read_page
directory = migrations_dir() base = directory if directory is not None else migrations_dir()
if not directory.is_dir(): if not base.is_dir():
return [] return []
migrations: list[Migration] = [] migrations: list[Migration] = []
for path in sorted(directory.glob("*.md")): for path in sorted(base.glob("*.md")):
try: try:
frontmatter, _ = read_page(path) frontmatter, _ = read_page(path)
except Exception: # noqa: BLE001 - a broken document is verify's finding, not ours except Exception: # noqa: BLE001 - a broken document is verify's finding, not ours
@@ -232,25 +238,60 @@ def next_link(
# --- what this instance changed about what it was given -------------------- # --- what this instance changed about what it was given --------------------
# What `compare_against_stamp` answers for one path: present and matching its
# recorded digest, present but not matching, or gone entirely. `dist upgrade`
# needs the three-way answer to tell a locally deleted file from a locally
# edited one; `divergent_files` (below) only ever needed the yes/no of
# "does this count as diverged", which both `MODIFIED` and `DELETED` answer
# the same way.
UNCHANGED = "unchanged"
MODIFIED = "modified"
DELETED = "deleted"
def compare_against_stamp(stamp_files: dict, root: Optional[Path] = None) -> dict[str, str]:
"""Classify every path in `stamp_files` (relative -> recorded sha256, the
shape of a release stamp's own `files` block) against what is actually on
disk under `root` - `UNCHANGED`, `MODIFIED`, or `DELETED`.
`root` defaults to `config.ROOT`. The general form `divergent_files` is
built on: that function only ever asks the question against this
instance's own tree, but `dist upgrade` (Gitea #7) asks it against an
already-installed tree while planning what to write, and a *second* time
against the tree it just wrote, before recording the new stamp - two trees
neither of which is necessarily `config.ROOT`.
"""
import hashlib
base = root if root is not None else config.ROOT
result: dict[str, str] = {}
for relative, digest in sorted(stamp_files.items()):
path = base / relative
if not path.is_file():
result[relative] = DELETED
continue
current = "sha256:" + hashlib.sha256(path.read_bytes()).hexdigest()
result[relative] = UNCHANGED if current == digest else MODIFIED
return result
def divergent_files() -> Optional[list[str]]: def divergent_files() -> Optional[list[str]]:
"""Files whose content no longer matches the release this instance installed. """Files whose content no longer matches the release this instance installed.
Reads the per-file sha256 in `.wikitool-release.json`, which `dist export` Reads the per-file sha256 in `.wikitool-release.json`, which `dist export`
has been writing since the stamp existed and which nothing has read until has been writing since the stamp existed. Its own docstring says why it is
now. Its own docstring says why it is there: it is the only way a later there: it is the only way a later upgrade can tell a file the instance
upgrade can tell a file the instance *edited* from one it merely *received*. *edited* from one it merely *received* - `dist upgrade` (Gitea #7) is that
later upgrade, built on the general `compare_against_stamp` above.
That distinction is what makes an `offered` migration actionable. The stack That distinction is also what makes an `offered` migration actionable. The
proposing a better `entity` template needs to know whether it may be copied stack proposing a better `entity` template needs to know whether it may be
over or whether the instance has its own version that a person has to copied over or whether the instance has its own version that a person has
reconcile - and only the recorded hash can answer that. to reconcile - and only the recorded hash can answer that.
Returns None when the question is unanswerable (a development tree, which Returns None when the question is unanswerable (a development tree, which
carries no stamp), which is different from `[]` (nothing diverged). carries no stamp), which is different from `[]` (nothing diverged).
""" """
import hashlib
from chemenu import version as version_mod from chemenu import version as version_mod
try: try:
@@ -263,13 +304,5 @@ def divergent_files() -> Optional[list[str]]:
if not isinstance(recorded, dict): if not isinstance(recorded, dict):
return None return None
divergent: list[str] = [] statuses = compare_against_stamp(recorded)
for relative, digest in sorted(recorded.items()): return [relative for relative, status in statuses.items() if status != UNCHANGED]
path = config.ROOT / relative
if not path.is_file():
divergent.append(relative)
continue
current = "sha256:" + hashlib.sha256(path.read_bytes()).hexdigest()
if current != digest:
divergent.append(relative)
return divergent
+87
View File
@@ -0,0 +1,87 @@
"""The ownership boundary for a path under a content stage: does it belong to
the *stack* (ships with every distribution, wins over local content when a
private instance merges from a public upstream) or to the *instance* (never
ships filled, wins over the upstream's version)?
One predicate, so `dist_cmd.py` (export) and `upstream_cmd.py` (merge/verify)
answer the same question about the same paths instead of each keeping its own
literal list that can drift out of sync with the other - see AGENTS.md
invariant 8, and Gitea #30 for the incident that made the drift concrete
(the private-instance merge procedure hardcoded a three-path list that
`dist_cmd.py` had already outgrown).
"""
from __future__ import annotations
# The stages whose content belongs to *this instance*, not the stack. Mirrors
# the sentence .gitignore already makes about raw/, kb/ and work/ being the
# repo's content, plus reports/ - only reports/CONTRACT.md is tracked there,
# the rest is gitignored, so restoring it is a no-op today. It stays in the
# set anyway: a set that is "almost" this one is the beginning of the same
# drift this module exists to end.
CONTENT_STAGES = ("kb", "raw", "work", "reports")
# Bare filenames `dist export` overwrites with a fresh stub rather than
# shipping the stack's own copy. Not stack-owned: an upstream merge takes the
# *local* side for these (they are the instance's own log/placeholder),
# while `dist export` writes a brand-new one regardless of either side.
EXPORT_STUB_NAMES = ("log.md", ".gitkeep")
# The single machinery filename directly under a content stage's own root.
_STAGE_CONTRACT_NAME = "CONTRACT.md"
def is_stack_owned(relative: str) -> bool:
"""Whether `relative` - a path under a content stage, e.g. "kb/CONTRACT.md"
or "kb/entities/COLLECTION.md.template" - is machinery: it ships with
every distribution, and it is the side an upstream merge keeps.
True for exactly two shapes:
- `<stage>/CONTRACT.md`, directly under a content stage's own root. Not
recursive: `kb/<collection>/COLLECTION.md` sits one level deeper and is
instance-owned (see kb/CONTRACT.md's collection-ownership split).
- Any path under a content stage ending in `.template` - by construction
the stack's own copy of something the instance adopts by renaming
(`kb/CONVENTIONS.md.template` and every `kb/<name>/COLLECTION.md.template`
today; a future stack-owned template under a content stage falls under
this rule automatically, with no code change here).
False for everything else under a content stage, `EXPORT_STUB_NAMES`
included - those are handled separately by whichever caller cares about
them, because the two callers disagree about which side wins for a stub.
"""
parts = relative.split("/")
if len(parts) < 2 or parts[0] not in CONTENT_STAGES:
return False
if relative.endswith(".template"):
return True
return len(parts) == 2 and parts[1] == _STAGE_CONTRACT_NAME
def is_export_stub(name: str) -> bool:
"""Whether `name` (a bare filename, not a path) is one `dist export`
overwrites with a fresh stub of its own rather than shipping verbatim."""
return name in EXPORT_STUB_NAMES
# Root-relative paths `dist export` seeds once, from a template it owns, and
# which the instance owns exclusively from that point on. `dist upgrade`
# (Gitea #7) must never overwrite them, even though they sit in the release
# stamp's `files` block like any other planned file - the same shape as
# `EXPORT_STUB_NAMES` above, but keyed by full path rather than bare filename,
# since nothing else at the repo root gets this treatment and a bare-filename
# match would be too broad here.
#
# `.wikitool-kb.json` is `migrate done`'s state file: an upgrade that resets it
# declares a content shape nobody actually produced. `CHANGES.md` is the
# instance's own changelog, not the stack's - `dist export` seeds it from a
# blank template (`dist_templates/CHANGES.md`) the same way it seeds
# `kb/log.md`, and overwriting it on upgrade would erase every entry the
# instance ever wrote for itself.
UPGRADE_PRESERVED_PATHS = (".wikitool-kb.json", "CHANGES.md")
def is_upgrade_preserved(relative: str) -> bool:
"""Whether `relative` (a plan-relative path from the repo root, e.g.
"CHANGES.md") is one `dist upgrade` must never write."""
return relative in UPGRADE_PRESERVED_PATHS
+386
View File
@@ -0,0 +1,386 @@
"""Tests for `wikitool dist upgrade`: classification against the locally
installed release stamp, the write set, migration-chain reporting without
execution, and every refusal before anything is written. See Gitea #7."""
from __future__ import annotations
import hashlib
import json
import subprocess
import tarfile
from pathlib import Path
import pytest
import typer
from chemenu import config, kb_state, version as version_mod
from chemenu.commands import dist_cmd
def _digest(text: str) -> str:
return "sha256:" + hashlib.sha256(text.encode("utf-8")).hexdigest()
def _write_stamp(path: Path, version: str, files: dict[str, str]) -> None:
path.write_text(
json.dumps(
{
"schema": version_mod.STAMP_SCHEMA,
"version": version,
"exported_at": "2026-01-01",
"source_repo": None,
"source_commit": None,
"release_url": None,
"update_url": version_mod.DEFAULT_UPDATE_URL,
"files": files,
},
indent=2,
),
encoding="utf-8",
)
def write_migration(directory: Path, target: str, slug: str) -> None:
directory.mkdir(parents=True, exist_ok=True)
(directory / f"{target}-{slug}.md").write_text(
"---\n"
"type: types/instruction.md\n"
f"name: {target}-{slug}\n"
f"description: Migration to {target}.\n"
"manual: true\n"
f"migrates_to: {target}\n"
"migration_kind: assisted\n"
"obligation: required\n"
"---\n\n# Migration\n\nSteps.\n",
encoding="utf-8",
)
def set_kb_version(root: Path, version: str) -> None:
(root / kb_state.KB_STATE_FILENAME).write_text(
json.dumps({"schema": 1, "kb_version": version, "applied": []}), encoding="utf-8"
)
@pytest.fixture
def instance(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Path:
"""A minimal tarball instance: VERSION 1.0.0, a local release stamp
recording two tracked files exactly as installed, content stays at
1.0.0 with nothing outstanding, and no .git directory (the WARN path
for the dirty-tree check)."""
root = tmp_path / "instance"
root.mkdir()
monkeypatch.setattr(config, "ROOT", root)
(root / "VERSION").write_text("1.0.0\n", encoding="utf-8")
(root / "AGENTS.md").write_text("core\n", encoding="utf-8")
(root / "tools").mkdir()
(root / "tools" / "wikitool").write_text("#!/bin/sh\n", encoding="utf-8")
files = {
"AGENTS.md": _digest("core\n"),
"tools/wikitool": _digest("#!/bin/sh\n"),
}
_write_stamp(root / version_mod.RELEASE_STAMP_FILENAME, "1.0.0", files)
set_kb_version(root, "1.0.0")
return root
def _release(tmp_path: Path, name: str, version: str, files_content: dict[str, str]) -> Path:
"""A second, independent tree shaped like a `dist export` output: VERSION,
a release stamp whose `files` block matches `files_content` exactly, and
the files themselves."""
root = tmp_path / name
root.mkdir()
(root / "VERSION").write_text(f"{version}\n", encoding="utf-8")
files: dict[str, str] = {}
for relative, content in files_content.items():
path = root / relative
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(content, encoding="utf-8")
files[relative] = _digest(content)
_write_stamp(root / version_mod.RELEASE_STAMP_FILENAME, version, files)
return root
# --- classification / dry-run -----------------------------------------------
def test_dry_run_classifies_every_case_and_writes_nothing(instance, tmp_path):
(instance / "AGENTS.md").write_text("locally edited\n", encoding="utf-8")
(instance / "tools" / "wikitool").unlink()
release = _release(
tmp_path, "release", "1.1.0",
{
"AGENTS.md": "core\n", # locally modified
"tools/wikitool": "#!/bin/sh\n", # locally deleted
"types/entity.md": "new page type\n", # new in the release
},
)
dist_cmd.run_upgrade(release, dry_run=True)
assert (instance / "AGENTS.md").read_text(encoding="utf-8") == "locally edited\n"
assert not (instance / "tools" / "wikitool").exists()
assert not (instance / "types" / "entity.md").exists()
assert json.loads((instance / version_mod.RELEASE_STAMP_FILENAME).read_text())["version"] == "1.0.0"
def test_removed_file_is_reported_and_left_alone_without_prune(instance, tmp_path):
release = _release(tmp_path, "release", "1.1.0", {"AGENTS.md": "core\n"})
# tools/wikitool is in the old stamp but absent from the new one.
dist_cmd.run_upgrade(release, dry_run=True)
assert (instance / "tools" / "wikitool").is_file()
# --- local changes are never silently overwritten ---------------------------
def test_locally_modified_file_blocks_the_upgrade_by_default(instance, tmp_path):
(instance / "AGENTS.md").write_text("locally edited\n", encoding="utf-8")
release = _release(
tmp_path, "release", "1.1.0",
{"AGENTS.md": "core v2\n", "tools/wikitool": "#!/bin/sh\n"},
)
with pytest.raises(typer.Exit) as excinfo:
dist_cmd.run_upgrade(release)
assert excinfo.value.exit_code == 1
assert (instance / "AGENTS.md").read_text(encoding="utf-8") == "locally edited\n"
assert json.loads((instance / version_mod.RELEASE_STAMP_FILENAME).read_text())["version"] == "1.0.0"
def test_locally_deleted_file_blocks_the_upgrade_by_default(instance, tmp_path):
(instance / "tools" / "wikitool").unlink()
release = _release(
tmp_path, "release", "1.1.0",
{"AGENTS.md": "core\n", "tools/wikitool": "#!/bin/sh v2\n"},
)
with pytest.raises(typer.Exit):
dist_cmd.run_upgrade(release)
assert not (instance / "tools" / "wikitool").exists()
def test_keep_local_proceeds_and_leaves_the_changed_file_untouched(instance, tmp_path):
(instance / "AGENTS.md").write_text("locally edited\n", encoding="utf-8")
release = _release(
tmp_path, "release", "1.1.0",
{"AGENTS.md": "core v2\n", "tools/wikitool": "#!/bin/sh\n", "types/entity.md": "new\n"},
)
dist_cmd.run_upgrade(release, keep_local=True)
# The locally changed file is untouched...
assert (instance / "AGENTS.md").read_text(encoding="utf-8") == "locally edited\n"
# ...but everything unchanged/new was still written.
assert (instance / "tools" / "wikitool").read_text(encoding="utf-8") == "#!/bin/sh\n"
assert (instance / "types" / "entity.md").read_text(encoding="utf-8") == "new\n"
assert json.loads((instance / version_mod.RELEASE_STAMP_FILENAME).read_text())["version"] == "1.1.0"
# --- the write set -----------------------------------------------------------
def test_unchanged_and_new_files_are_written_silently(instance, tmp_path):
release = _release(
tmp_path, "release", "1.1.0",
{"AGENTS.md": "core\n", "tools/wikitool": "#!/bin/sh\n", "types/entity.md": "new\n"},
)
dist_cmd.run_upgrade(release)
assert (instance / "AGENTS.md").read_text(encoding="utf-8") == "core\n"
assert (instance / "types" / "entity.md").read_text(encoding="utf-8") == "new\n"
stamp = json.loads((instance / version_mod.RELEASE_STAMP_FILENAME).read_text())
assert stamp["version"] == "1.1.0"
assert stamp["files"]["types/entity.md"] == _digest("new\n")
@pytest.mark.parametrize("preserved", [".wikitool-kb.json", "CHANGES.md", "kb/log.md", "raw/notes/.gitkeep"])
def test_seeded_once_paths_are_never_written_even_if_the_release_stamp_lists_them(
instance, tmp_path, preserved
):
"""The write set is the new stamp's `files` block minus what an export
re-seeds every time or seeds once and the instance owns from then on -
this is the AGENTS.md invariant 8 test: no separate literal list here,
only `chemenu.ownership`."""
(instance / "CHANGES.md").write_text("instance's own changelog\n", encoding="utf-8")
(instance / kb_state.KB_STATE_FILENAME).write_text(
json.dumps({"schema": 1, "kb_version": "1.0.0", "applied": [{"migration": "x"}]}),
encoding="utf-8",
)
release = _release(
tmp_path, "release", "1.1.0",
{
"AGENTS.md": "core\n",
"tools/wikitool": "#!/bin/sh\n",
preserved: "a fresh stub from the new release\n",
},
)
dist_cmd.run_upgrade(release)
if preserved == "CHANGES.md":
assert (instance / preserved).read_text(encoding="utf-8") == "instance's own changelog\n"
elif preserved == ".wikitool-kb.json":
state = json.loads((instance / preserved).read_text())
assert state["applied"] == [{"migration": "x"}]
else:
assert not (instance / preserved).exists()
# --- migration chain: reported, never run -----------------------------------
def test_migration_chain_is_reported_but_never_executed(instance, tmp_path):
release = _release(
tmp_path, "release", "1.1.0",
{"AGENTS.md": "core\n", "tools/wikitool": "#!/bin/sh\n"},
)
write_migration(release / "instructions" / "migrations", "1.1.0", "some-change")
dist_cmd.run_upgrade(release)
# The chain is reported, not applied: kb_version has not moved.
assert kb_state.read_kb_version().base == version_mod.Version(1, 0, 0)
def test_outstanding_local_migration_blocks_before_touching_the_source(instance, tmp_path):
"""A migration owed against the *installed* machinery must be finished
first - the source is never even opened."""
migrations = instance / "instructions" / "migrations"
write_migration(migrations, "1.0.0", "not-yet-done")
monkey_target = instance / "does-not-exist" # never read if this check fires first
with pytest.raises(typer.Exit):
dist_cmd.run_upgrade(monkey_target)
# --- preconditions -----------------------------------------------------------
def test_missing_local_stamp_blocks(instance, tmp_path):
(instance / version_mod.RELEASE_STAMP_FILENAME).unlink()
release = _release(tmp_path, "release", "1.1.0", {"AGENTS.md": "core\n"})
with pytest.raises(typer.Exit):
dist_cmd.run_upgrade(release)
def test_missing_kb_version_blocks(instance, tmp_path):
(instance / kb_state.KB_STATE_FILENAME).unlink()
release = _release(tmp_path, "release", "1.1.0", {"AGENTS.md": "core\n"})
with pytest.raises(typer.Exit):
dist_cmd.run_upgrade(release)
def test_prerelease_source_is_refused_without_pre_flag(instance, tmp_path):
release = _release(tmp_path, "release", "1.1.0-beta.1", {"AGENTS.md": "core\n"})
with pytest.raises(typer.Exit):
dist_cmd.run_upgrade(release)
assert json.loads((instance / version_mod.RELEASE_STAMP_FILENAME).read_text())["version"] == "1.0.0"
def test_pre_flag_allows_a_prerelease_source(instance, tmp_path):
release = _release(
tmp_path, "release", "1.1.0-beta.1",
{"AGENTS.md": "core\n", "tools/wikitool": "#!/bin/sh\n"},
)
dist_cmd.run_upgrade(release, allow_pre=True)
assert json.loads((instance / version_mod.RELEASE_STAMP_FILENAME).read_text())["version"] == "1.1.0-beta.1"
def test_downgrade_is_refused(instance, tmp_path):
release = _release(tmp_path, "release", "0.9.0", {"AGENTS.md": "core\n"})
with pytest.raises(typer.Exit):
dist_cmd.run_upgrade(release)
def test_equal_version_is_a_noop(instance, tmp_path):
release = _release(tmp_path, "release", "1.0.0", {"AGENTS.md": "core\n"})
dist_cmd.run_upgrade(release) # must not raise
assert json.loads((instance / version_mod.RELEASE_STAMP_FILENAME).read_text())["version"] == "1.0.0"
def test_dirty_working_tree_blocks(instance, tmp_path):
subprocess.run(["git", "init", "-q", "-b", "main"], cwd=instance, check=True)
subprocess.run(["git", "config", "user.name", "Fixture Author"], cwd=instance, check=True)
subprocess.run(["git", "config", "user.email", "fixture@example.invalid"], cwd=instance, check=True)
(instance / "untracked.txt").write_text("dirty\n", encoding="utf-8")
release = _release(tmp_path, "release", "1.1.0", {"AGENTS.md": "core\n"})
with pytest.raises(typer.Exit):
dist_cmd.run_upgrade(release)
assert not (instance / "types").exists()
def test_source_that_does_not_exist_is_refused(instance, tmp_path):
with pytest.raises(typer.Exit):
dist_cmd.run_upgrade(tmp_path / "nowhere")
# --- prune -------------------------------------------------------------------
def test_prune_removes_only_removed_files_still_unchanged_since_install(instance, tmp_path):
(instance / "extra.txt").write_text("shipped once, edited since\n", encoding="utf-8")
old_stamp = json.loads((instance / version_mod.RELEASE_STAMP_FILENAME).read_text())
old_stamp["files"]["extra.txt"] = _digest("shipped once, unedited\n") # deliberately stale
old_stamp["files"]["gone.txt"] = _digest("also shipped once\n")
(instance / version_mod.RELEASE_STAMP_FILENAME).write_text(json.dumps(old_stamp), encoding="utf-8")
(instance / "gone.txt").write_text("also shipped once\n", encoding="utf-8")
release = _release(
tmp_path, "release", "1.1.0",
{"AGENTS.md": "core\n", "tools/wikitool": "#!/bin/sh\n"},
)
dist_cmd.run_upgrade(release, prune=True)
# gone.txt matched its recorded digest -> pruned.
assert not (instance / "gone.txt").exists()
# extra.txt was locally edited relative to its recorded digest -> kept.
assert (instance / "extra.txt").read_text(encoding="utf-8") == "shipped once, edited since\n"
# --- tarball sources ----------------------------------------------------------
def _pack(release_dir: Path, archive: Path) -> None:
with tarfile.open(archive, "w:gz") as tf:
tf.add(release_dir, arcname=release_dir.name)
def test_tarball_with_more_than_one_top_level_entry_is_refused(instance, tmp_path):
scratch = tmp_path / "scratch"
(scratch / "a").mkdir(parents=True)
(scratch / "b").mkdir(parents=True)
(scratch / "a" / "x.txt").write_text("x\n", encoding="utf-8")
(scratch / "b" / "y.txt").write_text("y\n", encoding="utf-8")
archive = tmp_path / "bad.tar.gz"
with tarfile.open(archive, "w:gz") as tf:
tf.add(scratch / "a", arcname="a")
tf.add(scratch / "b", arcname="b")
with pytest.raises(typer.Exit):
dist_cmd.run_upgrade(archive)
def test_tarball_source_is_extracted_and_applied(instance, tmp_path):
release_dir = _release(
tmp_path, "chemenu-stack-1.1.0", "1.1.0",
{"AGENTS.md": "core\n", "tools/wikitool": "#!/bin/sh\n"},
)
archive = tmp_path / "chemenu-stack-1.1.0.tar.gz"
_pack(release_dir, archive)
dist_cmd.run_upgrade(archive)
assert (instance / "AGENTS.md").read_text(encoding="utf-8") == "core\n"
assert json.loads((instance / version_mod.RELEASE_STAMP_FILENAME).read_text())["version"] == "1.1.0"
def test_tarball_sha256_sidecar_mismatch_is_refused(instance, tmp_path):
release_dir = _release(
tmp_path, "chemenu-stack-1.1.0", "1.1.0", {"AGENTS.md": "core\n"}
)
archive = tmp_path / "chemenu-stack-1.1.0.tar.gz"
_pack(release_dir, archive)
(archive.with_name(archive.name + ".sha256")).write_text(
"0" * 64 + " chemenu-stack-1.1.0.tar.gz\n", encoding="utf-8"
)
with pytest.raises(typer.Exit):
dist_cmd.run_upgrade(archive)
+470
View File
@@ -0,0 +1,470 @@
"""Tests for `wikitool upstream merge`/`upstream verify` - the code procedure
that replaces private-instance.md's prose merge script (Gitea #30).
Two real git repos stand in for a private instance (`repo`, remote name
`upstream`) and the public repo it takes updates from (`upstream`, a plain
repo committed to directly - a fetch-only remote does not need to be bare for
`git fetch` to work against it). Each scenario diverges the two by committing
independently on each side, exactly like a real fetch-only upstream would.
"""
from __future__ import annotations
import subprocess
import pytest
import typer
from chemenu import config, ownership
from chemenu.commands import git_publish, upstream_cmd
def _git(root, *args):
result = subprocess.run(["git", *args], cwd=root, capture_output=True, text=True)
assert result.returncode == 0, result.stderr
return result
def _write(root, relative, content):
path = root / relative
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(content, encoding="utf-8")
def _commit(root, message):
_git(root, "add", "-A")
_git(root, "commit", "-m", message)
@pytest.fixture
def two_repos(tmp_path, monkeypatch):
"""`repo`, a private instance, with a fetch-only `upstream` remote pointing
at a second, independent repo. Both start from the same seed commit -
kb/CONTRACT.md, kb/CONVENTIONS.md(.template), kb/entities/COLLECTION.md,
raw/CONTRACT.md, work/CONTRACT.md, reports/CONTRACT.md, and one tools/
file - which is what a private instance looks like right after the
private-instance.md setup: the tracked machinery, plus its own filled
instance files layered on top.
"""
seed = tmp_path / "seed"
seed.mkdir()
_git(seed, "init", "-b", "main")
_git(seed, "config", "user.name", "Seed")
_git(seed, "config", "user.email", "seed@example.com")
# .wikitool-remotes.json is gitignored in the real repo (it is per-checkout,
# see config.PUBLISH_REMOTES_FILENAME) - without this, dropping one into the
# fixture during a test would show up as an untracked file and trip the
# dirty-working-tree precondition for a reason that has nothing to do with
# what that test is checking.
# Mirrors the real .gitignore in the two ways that matter here:
# `.wikitool-remotes.json` is per-checkout (dropping one in during a test
# must not read as a dirty tree), and `reports/` is derived output that is
# ignored except for its contract - which is what makes a content stage
# able to hold local, non-recomputable data a merge must not touch.
_write(
seed,
".gitignore",
f"/{config.PUBLISH_REMOTES_FILENAME}\n/reports/*\n!/reports/CONTRACT.md\n",
)
_write(seed, "kb/CONTRACT.md", "stack kb contract v1\n")
_write(seed, "kb/CONVENTIONS.md.template", "template v1\n")
_write(seed, "kb/CONVENTIONS.md", "instance conventions v1\n")
_write(seed, "kb/entities/COLLECTION.md", "instance collection contract v1\n")
_write(seed, "kb/Both.md", "page both sides delete\n")
_write(seed, "kb/ToDelete.md", "page the instance will delete\n")
_write(seed, "kb/RegularPage.md", "an ordinary page neither side has touched yet\n")
_write(seed, "raw/CONTRACT.md", "raw contract v1\n")
_write(seed, "work/CONTRACT.md", "work contract v1\n")
_write(seed, "reports/CONTRACT.md", "reports contract v1\n")
_write(seed, "tools/wikitool.py", "line one\nline two\nline three\n")
_commit(seed, "seed")
upstream = tmp_path / "upstream"
subprocess.run(["git", "clone", str(seed), str(upstream)], check=True, capture_output=True)
_git(upstream, "config", "user.name", "Upstream")
_git(upstream, "config", "user.email", "upstream@example.com")
# Cloned from `upstream`, not from `seed` directly: the remote (renamed
# below) must resolve to the path this fixture actually commits new
# upstream state into, or a later `git fetch upstream main` silently
# fetches from `seed` instead and never sees anything new.
repo = tmp_path / "repo"
subprocess.run(["git", "clone", str(upstream), str(repo)], check=True, capture_output=True)
_git(repo, "config", "user.name", "Test")
_git(repo, "config", "user.email", "test@example.com")
_git(repo, "remote", "rename", "origin", "upstream")
monkeypatch.setattr(config, "ROOT", repo)
monkeypatch.setenv("WIKITOOL_SESSION_ID", "test-session")
return upstream, repo
def _merge(**overrides):
kwargs = dict(remote="upstream", branch="main", no_fetch=False)
kwargs.update(overrides)
upstream_cmd.merge_command(**kwargs)
# --- the four restbefund regressions, plus the baseline table from the issue ---
def test_upstream_edit_of_a_page_the_instance_deleted_does_not_land(two_repos):
upstream, repo = two_repos
_git(repo, "rm", "-q", "kb/ToDelete.md")
_commit(repo, "instance deletes ToDelete")
_write(upstream, "kb/ToDelete.md", "upstream edited it after the instance deleted it\n")
_commit(upstream, "upstream edits ToDelete")
_merge()
assert not (repo / "kb/ToDelete.md").exists()
def test_upstream_new_page_does_not_land(two_repos):
upstream, repo = two_repos
_write(upstream, "kb/NewPage.md", "a demo page the upstream added\n")
_commit(upstream, "upstream adds NewPage")
_merge()
assert not (repo / "kb/NewPage.md").exists()
def test_page_deleted_on_both_sides_is_a_noop(two_repos):
upstream, repo = two_repos
_git(repo, "rm", "-q", "kb/Both.md")
_commit(repo, "instance deletes Both")
_git(upstream, "rm", "-q", "kb/Both.md")
_commit(upstream, "upstream deletes Both")
_merge() # must not raise
assert not (repo / "kb/Both.md").exists()
def test_kb_contract_change_lands(two_repos):
upstream, repo = two_repos
_write(upstream, "kb/CONTRACT.md", "stack kb contract v2\n")
_commit(upstream, "upstream changes kb/CONTRACT.md")
_merge()
assert (repo / "kb/CONTRACT.md").read_text(encoding="utf-8") == "stack kb contract v2\n"
def test_conventions_template_change_lands_local_conventions_untouched(two_repos):
upstream, repo = two_repos
_write(upstream, "kb/CONVENTIONS.md.template", "template v2\n")
_commit(upstream, "upstream changes the conventions template")
_merge()
assert (repo / "kb/CONVENTIONS.md.template").read_text(encoding="utf-8") == "template v2\n"
assert (repo / "kb/CONVENTIONS.md").read_text(encoding="utf-8") == "instance conventions v1\n"
def test_collection_contract_change_does_not_land(two_repos):
"""A COLLECTION.md is instance-owned since #39 - one level deeper than
`<stage>/CONTRACT.md`, so `is_stack_owned` must say no to it."""
upstream, repo = two_repos
_write(repo, "kb/entities/COLLECTION.md", "instance collection contract v2 (local)\n")
_commit(repo, "instance rewrites its own collection contract")
_write(upstream, "kb/entities/COLLECTION.md", "upstream collection contract v2\n")
_commit(upstream, "upstream changes the default collection contract")
_merge()
assert (repo / "kb/entities/COLLECTION.md").read_text(encoding="utf-8") == (
"instance collection contract v2 (local)\n"
)
def test_upstream_deletion_of_a_contract_file_lands(two_repos):
"""Restbefund 2: a machinery file the upstream deleted must not silently
survive because `git checkout MERGE_HEAD -- <path>` has nothing to check
out."""
upstream, repo = two_repos
_git(upstream, "rm", "-q", "raw/CONTRACT.md")
_commit(upstream, "upstream drops raw/CONTRACT.md")
_merge()
assert not (repo / "raw/CONTRACT.md").exists()
def test_new_stack_template_under_a_content_stage_lands(two_repos):
"""Restbefund 4: a brand-new stack-owned path the local tree has never
seen must still be recognised by the predicate, not by a literal list."""
upstream, repo = two_repos
_write(upstream, "kb/GLOSSARY.md.template", "a stack-owned template that never existed before\n")
_commit(upstream, "upstream adds a new template")
_merge()
assert (repo / "kb/GLOSSARY.md.template").read_text(encoding="utf-8") == (
"a stack-owned template that never existed before\n"
)
def test_open_workshop_run_files_do_not_land(two_repos):
upstream, repo = two_repos
_write(upstream, "work/some-run/README.md", "an in-progress workshop run\n")
_commit(upstream, "upstream ships an open work/ run")
_merge()
assert not (repo / "work/some-run").exists()
def test_merge_keeps_ignored_local_data_under_a_content_stage(two_repos):
"""`reports/` is gitignored except its contract, so a content stage's
working tree holds local data that is in no git tree and is not
recomputable - the telemetry traces `eval score` reads, saved eval
reports, past lint reports. Forcing the stage back to the local side must
not take those out as collateral: this instance had 497 trace directories
under reports/telemetry/ when the first version of this command wiped the
stage wholesale."""
upstream, repo = two_repos
_write(repo, "reports/telemetry/session-a/trace.jsonl", '{"event": "local"}\n')
_write(repo, "reports/Lint Report 2026-09-04.md", "a local lint report\n")
assert _git(repo, "status", "--porcelain").stdout == "" # ignored, so the tree is clean
_write(upstream, "kb/CONTRACT.md", "stack kb contract v2\n")
_commit(upstream, "upstream changes kb/CONTRACT.md")
_merge()
assert (repo / "reports/telemetry/session-a/trace.jsonl").read_text(encoding="utf-8") == (
'{"event": "local"}\n'
)
assert (repo / "reports/Lint Report 2026-09-04.md").exists()
assert (repo / "reports/CONTRACT.md").read_text(encoding="utf-8") == "reports contract v1\n"
def test_upstream_content_under_a_stage_absent_from_head_does_not_land(two_repos):
"""The stage guard must not rest on the local side happening to track
something under that stage: an instance whose `work/` holds no tracked
file at all must still not receive the upstream's open run."""
upstream, repo = two_repos
_git(repo, "rm", "-q", "work/CONTRACT.md")
_commit(repo, "instance has nothing tracked under work/")
_write(upstream, "work/some-run/README.md", "an in-progress workshop run\n")
_commit(upstream, "upstream ships an open work/ run")
_merge()
assert not (repo / "work/some-run").exists()
def test_one_upstream_commit_mixing_every_case_at_once(two_repos):
"""The acceptance test from the issue: a single upstream commit that edits
a page the instance deleted, adds a new page, deletes an untouched page,
changes a stack contract, changes a template, and deletes a different
stack contract - all at once, all restored or discarded correctly by one
`upstream merge` call."""
upstream, repo = two_repos
_git(repo, "rm", "-q", "kb/ToDelete.md")
_commit(repo, "instance deletes ToDelete")
_write(upstream, "kb/ToDelete.md", "upstream edited it after the instance deleted it\n")
_write(upstream, "kb/BrandNewPage.md", "a demo page the upstream added\n")
_git(upstream, "rm", "-q", "kb/RegularPage.md")
_write(upstream, "kb/CONTRACT.md", "stack kb contract v2\n")
_write(upstream, "kb/CONVENTIONS.md.template", "template v2\n")
_git(upstream, "rm", "-q", "raw/CONTRACT.md")
_commit(upstream, "one upstream commit: edit + add + delete + contract + template + contract-delete")
_merge()
assert not (repo / "kb/ToDelete.md").exists()
assert not (repo / "kb/BrandNewPage.md").exists()
assert (repo / "kb/RegularPage.md").exists()
assert (repo / "kb/CONTRACT.md").read_text(encoding="utf-8") == "stack kb contract v2\n"
assert (repo / "kb/CONVENTIONS.md.template").read_text(encoding="utf-8") == "template v2\n"
assert (repo / "kb/CONVENTIONS.md").read_text(encoding="utf-8") == "instance conventions v1\n"
assert not (repo / "raw/CONTRACT.md").exists()
def test_real_conflict_in_tools_leaves_the_merge_open(two_repos):
upstream, repo = two_repos
_write(repo, "tools/wikitool.py", "line one\nLOCAL CHANGE\nline three\n")
_commit(repo, "local edits tools/wikitool.py")
before = _git(repo, "rev-parse", "HEAD").stdout.strip()
_write(upstream, "tools/wikitool.py", "line one\nUPSTREAM CHANGE\nline three\n")
_commit(upstream, "upstream edits the same line")
with pytest.raises(typer.Exit) as excinfo:
_merge()
assert excinfo.value.exit_code == 1
assert _git(repo, "rev-parse", "HEAD").stdout.strip() == before
assert (repo / ".git" / "MERGE_HEAD").exists()
def test_a_merge_git_refuses_to_open_deletes_nothing(two_repos, tmp_path):
"""The failure mode with the worst blast radius if it is not guarded:
without MERGE_HEAD, every stack-owned path in HEAD reads as "the upstream
deleted it", and the restore loop would remove kb/CONTRACT.md,
raw/CONTRACT.md and every template. A merge git refuses to start must stop
before that, with the tree untouched."""
upstream, repo = two_repos
unrelated = tmp_path / "unrelated"
unrelated.mkdir()
_git(unrelated, "init", "-b", "main")
_git(unrelated, "config", "user.name", "Unrelated")
_git(unrelated, "config", "user.email", "unrelated@example.com")
_write(unrelated, "somefile.md", "no shared history with the instance\n")
_commit(unrelated, "unrelated root commit")
_git(repo, "remote", "set-url", "upstream", str(unrelated))
before = _git(repo, "rev-parse", "HEAD").stdout.strip()
with pytest.raises(typer.Exit) as excinfo:
_merge()
assert excinfo.value.exit_code == 1
assert _git(repo, "rev-parse", "HEAD").stdout.strip() == before
assert (repo / "kb/CONTRACT.md").read_text(encoding="utf-8") == "stack kb contract v1\n"
assert (repo / "raw/CONTRACT.md").exists()
assert (repo / "kb/CONVENTIONS.md.template").exists()
assert _git(repo, "status", "--porcelain").stdout == ""
def test_success_message_reports_what_changed_not_what_was_restored(two_repos, capsys):
"""Restoring every stack-owned path from MERGE_HEAD touches all of them
whether or not the upstream moved any, so the report has to ask git what
changed - otherwise a one-file update is announced as five."""
upstream, repo = two_repos
_write(upstream, "kb/CONTRACT.md", "stack kb contract v2\n")
_commit(upstream, "upstream changes exactly one stack path")
_merge()
out = capsys.readouterr().out
assert "Stack paths changed (1)" in out
assert "kb/CONTRACT.md" in out
assert "kb/CONVENTIONS.md.template" not in out
def test_dirty_working_tree_is_refused_untouched(two_repos):
upstream, repo = two_repos
before = _git(repo, "rev-parse", "HEAD").stdout.strip()
(repo / "kb/CONTRACT.md").write_text("uncommitted local edit\n", encoding="utf-8")
_write(upstream, "kb/CONTRACT.md", "stack kb contract v2\n")
_commit(upstream, "upstream changes kb/CONTRACT.md")
with pytest.raises(typer.Exit) as excinfo:
_merge()
assert excinfo.value.exit_code == 1
assert _git(repo, "rev-parse", "HEAD").stdout.strip() == before
assert (repo / "kb/CONTRACT.md").read_text(encoding="utf-8") == "uncommitted local edit\n"
def test_already_up_to_date_is_a_noop(two_repos):
upstream, repo = two_repos
before = _git(repo, "rev-parse", "HEAD").stdout.strip()
_merge() # nothing new upstream at all
assert _git(repo, "rev-parse", "HEAD").stdout.strip() == before
def test_merge_warns_when_the_publish_remote_gate_is_unarmed(two_repos, capsys):
upstream, repo = two_repos
assert git_publish.read_allowed_push_urls() is None # no .wikitool-remotes.json in this repo
_write(upstream, "kb/CONTRACT.md", "stack kb contract v2\n")
_commit(upstream, "upstream changes kb/CONTRACT.md")
_merge()
captured = capsys.readouterr()
assert "WARN" in captured.out
assert ".wikitool-remotes.json" in captured.out
def test_merge_stays_silent_when_the_publish_remote_gate_is_armed(two_repos, capsys):
upstream, repo = two_repos
(repo / config.PUBLISH_REMOTES_FILENAME).write_text(
'{"schema": 1, "allowed_push_urls": ["ssh://example/test.git"]}\n', encoding="utf-8"
)
_write(upstream, "kb/CONTRACT.md", "stack kb contract v2\n")
_commit(upstream, "upstream changes kb/CONTRACT.md")
_merge()
captured = capsys.readouterr()
assert "WARN" not in captured.out
def test_dist_cmd_contract_only_stages_agree_with_ownership(two_repos):
"""Consistency guard for the ownership refactor: `dist_cmd`'s own list of
stage-contract paths and `ownership.is_stack_owned` must not be able to
name a different set of stages - both are sourced from
`ownership.CONTENT_STAGES` now, so a stage added to one and not the other
fails this rather than only surfacing in a real merge."""
from chemenu.commands import dist_cmd
assert dist_cmd.CONTRACT_ONLY_STAGES # sanity: the derivation still yields entries
for relative in dist_cmd.CONTRACT_ONLY_STAGES:
assert ownership.is_stack_owned(relative)
# --- upstream verify --------------------------------------------------------
def test_verify_is_clean_on_a_stack_owned_only_change(two_repos):
upstream, repo = two_repos
since = _git(repo, "rev-parse", "HEAD").stdout.strip()
_write(repo, "kb/CONTRACT.md", "stack kb contract v2\n")
_commit(repo, "advance kb/CONTRACT.md")
upstream_cmd.verify_command(since=since, until="HEAD") # must not raise
def test_verify_fails_on_a_hand_botched_merge(two_repos, capsys):
upstream, repo = two_repos
since = _git(repo, "rev-parse", "HEAD").stdout.strip()
_write(repo, "kb/SneakedIn.md", "content that arrived outside a stack-owned path\n")
_commit(repo, "a hand-resolved merge that let content through")
with pytest.raises(typer.Exit) as excinfo:
upstream_cmd.verify_command(since=since, until="HEAD")
assert excinfo.value.exit_code == 1
captured = capsys.readouterr()
assert "kb/SneakedIn.md" in captured.out
# --- ownership predicate, exercised directly ---------------------------------
@pytest.mark.parametrize(
"relative,expected",
[
("kb/CONTRACT.md", True),
("raw/CONTRACT.md", True),
("work/CONTRACT.md", True),
("reports/CONTRACT.md", True),
("kb/CONVENTIONS.md.template", True),
("kb/entities/COLLECTION.md.template", True),
("kb/GLOSSARY.md.template", True),
("kb/CONVENTIONS.md", False),
("kb/entities/COLLECTION.md", False),
("kb/concepts/Some Page.md", False),
("raw/notes/x.md", False),
("tools/CONTRACT.md", False), # not a content stage
("kb/log.md", False), # export stub, not stack-owned
],
)
def test_is_stack_owned(relative, expected):
assert ownership.is_stack_owned(relative) == expected