diff --git a/CHANGES.md b/CHANGES.md index bc852cc..ac2c8a8 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -35,7 +35,7 @@ dev-checkout concern - readable here, never shipped as something to parse. --- -## 4.5.0-beta.3 - 2026-09-04 - upstream merge: keep gitignored local data under a content stage, refuse a merge git never opened, report what actually changed +## 4.5.0-beta.4 - 2026-09-04 - dist upgrade: apply a stack update, not just detect one (#7) **Author:** Torben Nehmer @@ -44,6 +44,7 @@ dev-checkout concern - readable here, never shipped as something to parse. - wikitool upstream merge/verify: code procedure for taking a stack update, ownership.py as the shared stack/instance boundary - upstream merge: combined-commit regression test (edit+add+delete+contract+template+contract-delete in one commit) - upstream merge: keep gitignored local data under a content stage, refuse a merge git never opened, report what actually changed +- dist upgrade: apply a stack update, not just detect one (#7) Die Prosa zu 4.4.0 - Changelog-Eintrag, `docs/version-model.md`, `instructions/dev/version-parts.md`, @@ -159,6 +160,54 @@ jetzt `git diff` zwischen Vor- und Nach-Commit, kennzeichnet Löschungen, und st bekommen, warum die Grenze ein Prädikat und keine Liste ist — die Begründung, die dieses Issue erarbeitet hat, gehörte in die Hintergrunddoku und nicht nur in einen Changelog-Eintrag. +**Fünfter Bump: `wikitool dist upgrade` (#7), der zweite der beiden Update-Wege.** `upstream +merge` oben bedient eine Instanz mit gemeinsamer Git-History; `dist upgrade` bedient eine +Instanz aus einem Tarball, ohne History, die bislang eine rein manuelle Prozedur in +`INSTALL.md` durchlaufen musste - Schritt 4 verlangte einen sha256-Vergleich von Hand gegen den +`files`-Block der alten `.wikitool-release.json`. + +Die tragende Regel: die Schreibmenge ist genau der `files`-Block der *neuen* +`.wikitool-release.json`, minus was ein Export aus einer leeren Vorlage neu sät +(`chemenu.ownership.is_export_stub`, wie bisher schon für `kb/log.md`/`.gitkeep`) oder einmalig +sät und danach der Instanz gehört (`chemenu.ownership.is_upgrade_preserved`, neu für +`.wikitool-kb.json` und `CHANGES.md`), plus der Stamp selbst. Jeder Kandidatpfad wird gegen die +*alte* Instanz-Summe klassifiziert: unverändert wird geräuschlos überschrieben, neu im Release +wird angelegt, lokal verändert oder gelöscht wird **nie** still überschrieben - der Lauf bricht +mit der vollständigen Liste ab, außer `--keep-local` sagt ausdrücklich, dass die Dateien liegen +bleiben sollen. `--prune` entfernt zusätzlich aus dem Release entfallene Dateien, aber nur +solche, die seit der Installation unverändert sind. + +Die Migrationskette nach dem Tausch wird aus den `instructions/migrations/` des *neuen* Baums +ermittelt (`kb_state.load_migrations` bekam dafür einen `directory`-Parameter) und nur +gemeldet, nie ausgeführt - es gibt bewusst kein `migrate run`. Eine bereits gegen die +*installierte* Maschinerie offene Kette lässt den Befehl abbrechen, bevor er die Quelle +überhaupt öffnet. `kb_state.divergent_files()` (bisher nur von `migrate status` gelesen) ist +jetzt eine dünne Hülle um das neue, zwei-Baum-fähige `compare_against_stamp()` - gleiches +Verhalten für den bestehenden Aufrufer, wiederverwendbar für `dist upgrade`s eigenen Vergleich. + +Quelle ist immer ein bereits vorhandenes Verzeichnis oder `.tar.gz` - kein Download, das bleibt +allein `version check`s Sache. Ein Tarball muss genau ein Top-Level-Verzeichnis enthalten (die +Form, in der `release.yml` es baut) und wird gegen eine `.sha256`-Beidatei geprüft, falls eine +danebenliegt (fehlt sie: WARN, kein Abbruch). Weitere Abbruchgründe vor jedem Schreiben: fehlende +lokale `VERSION`/`.wikitool-kb.json`/Stamp mit `files`-Block, ein schmutziger Arbeitsbaum (kein +Git-Repo ist ein WARN, keine Sperre), eine Vorab-Version (`-beta.N`) ohne `--pre`, sowie ein +Downgrade; Gleichstand ist ein No-op. Ein Grenzübertritt der Kompatibilität wird laut gemeldet, +blockiert aber nicht. Committet und pusht nichts (Invariante 5). + +Gegenüber dem ersten Entwurf des Issues zwei Korrekturen, die dort auch nachgetragen sind: der +`files`-Block wurde entgegen der ursprünglichen Annahme bereits vor diesem Bump gelesen +(`divergent_files`/`migrate status`), und die Migrationskette war ursprünglich falsch begründet +- sie kann nur aus dem *neuen* Baum kommen, nicht durch eine andere Abfragereihenfolge aus der +alten Instanz. 24 neue Tests unter `test_dist_upgrade.py` decken die Klassifikation, alle +Abbruchgründe, `--keep-local`, `--prune` und beide Quellformen (Verzeichnis und Tarball, +inklusive der sha256- und Top-Level-Prüfung) ab. + +Bewusst nicht angetastet: `instructions/private-instance.md` (der Clone-Weg ändert sich nicht, +`INSTALL.md` benennt jetzt beide Wege nebeneinander) und die Frage, wie `dist upgrade` mit +Collection-Templates umgeht, deren Namen eine fremde Instanz gar nicht hat - es verhält sich wie +`upstream merge` und schreibt sie, was ein eigenes Issue gegen den Export wäre, keins gegen das +Upgrade. + --- ## 4.4.0 - 2026-09-03 - Versionskandidat statt Bump-pro-Release: VERSION traegt -beta.N, version release fixiert diff --git a/INSTALL.md b/INSTALL.md index aa9c9f9..125cb7d 100644 --- a/INSTALL.md +++ b/INSTALL.md @@ -167,10 +167,17 @@ wenn nicht). `tools/wikitool version notes` druckt den Eintrag. ### Eine Instanz aktualisieren -Das Anwenden eines Updates ist ein bewusst manueller Vorgang - es schreibt in eine Instanz, die -bereits Inhalt hat. Der Inhalt hat dabei eine **eigene Version**: `.wikitool-kb.json` sagt, in -welcher Form die Seiten vorliegen, unabhängig davon, welche Maschinerie danebensteht. Genau -dieser Unterschied ist der Zustand, in dem sich jede Instanz mitten im Upgrade befindet. +Zwei Wege, je nachdem, wie diese Instanz entstanden ist. Ein **Clone mit gemeinsamer +Git-History** (`upstream`-Remote auf das Ursprungs-Repo, siehe +[instructions/private-instance.md](instructions/private-instance.md)) nimmt Stack-Updates per +echtem Drei-Wege-Merge: `tools/wikitool upstream merge`. Alles Folgende gilt für eine **Instanz +aus einem Tarball**, ohne gemeinsame History - der Weg unten unter „Eine Instanz aktualisieren" +nutzt sie. + +Das Anwenden eines Updates schreibt in eine Instanz, die bereits Inhalt hat. Der Inhalt hat dabei +eine **eigene Version**: `.wikitool-kb.json` sagt, in welcher Form die Seiten vorliegen, +unabhängig davon, welche Maschinerie danebensteht. Genau dieser Unterschied ist der Zustand, in +dem sich jede Instanz mitten im Upgrade befindet. 1. **Vor dem Tausch** prüfen, was ansteht - solange `VERSION` noch die alte ist: @@ -178,19 +185,35 @@ dieser Unterschied ist der Zustand, in dem sich jede Instanz mitten im Upgrade b tools/wikitool migrate status ``` -2. Release-Tarball herunterladen und entpacken (Weg A), die Release-Notes lesen. -3. Die **Maschinerie** aus dem Tarball über die Instanz kopieren: `tools/`, `types/`, - `instructions/`, `AGENTS.md`, `VERSION`, `.wikitool-release.json` - **und `kb/CONTRACT.md`**. - Die letzte Datei liegt unter einem Content-Verzeichnis, ist aber Stack-Eigentum: sie hält, - was `wikitool` erzwingt, und ist in jeder Instanz gleich. Nicht anfassen: alles andere unter - `kb/` und `raw/`, `work/`, `.wikitool-kb.json` und `.git/` - das ist die Instanz selbst, - `kb/CONVENTIONS.md` und die `kb/*/COLLECTION.md` eingeschlossen. -4. Achtung bei lokal angepassten Stack-Dateien. Die Autorenkonventionen gehören **nicht** dazu: - `kb/CONVENTIONS.md` und die `kb/*/COLLECTION.md` liegen unter `kb/`, werden in Schritt 3 - also ohnehin nicht angefasst - genau dafür ist der Schnitt da. Wer darüber hinaus etwas - unter `tools/`, `types/` oder `instructions/` verändert hat, sichert das vorher und spielt - es danach wieder ein. Welche Dateien das sind, verrät ein Vergleich gegen die sha256-Summen - im `files`-Block der alten `.wikitool-release.json`. + Steht hier etwas aus, erst diese Migrationskette abschließen (Schritt 5 unten) - `dist upgrade` + verweigert den Tausch sonst von selbst. + +2. Release-Tarball herunterladen und die Release-Notes lesen (Weg A oben). +3. **Maschinerie tauschen:** + + ```bash + tools/wikitool dist upgrade --dry-run + ``` + + Klassifiziert jede Datei aus dem `files`-Block der neuen `.wikitool-release.json`: + unverändert seit der Installation, lokal verändert oder gelöscht, neu im Release, oder aus dem + Release entfallen - und druckt die Migrationskette, die nach dem Tausch aussteht, ohne sie + auszuführen. Ohne `--dry-run` schreibt der Befehl; eine lokal veränderte oder gelöschte Datei + wird dabei **nie** stillschweigend überschrieben - der Lauf bricht mit der vollständigen Liste + ab, es sei denn `--keep-local` ist gesetzt (dann bleibt jede davon unangetastet, erneut + gemeldet). `--prune` entfernt zusätzlich Dateien, die der neue Release nicht mehr ausliefert + und die seit der Installation unverändert sind. Voraussetzungen: ein sauberer Arbeitsbaum + (kein Git-Repo ist ein WARN, keine Sperre), eine lokale `.wikitool-release.json` mit + `files`-Block (fehlt sie, siehe „Fallstricke" unten), und `.wikitool-kb.json` vorhanden. + Committet und pusht nichts (Invariante 5). Vollständiger Fehlerkontrakt: + [tools/CONTRACT.md](tools/CONTRACT.md). + + Eine lokal veränderte Stack-Datei ist damit sichtbar, statt von Hand gegen die sha256-Summen + im `files`-Block geprüft werden zu müssen - genau der Schritt, der vor `4.5.0` hier stand. +4. Bei einer Kompatibilitätsgrenze (`dist upgrade` meldet sie laut) die Release-Notes vor dem + nächsten Schritt lesen: **Breaking Change:** und **Migration:** im Eintrag von + `tools/wikitool version notes` sagen, was aufhört zu funktionieren und ob der Korpus + umgeschrieben werden muss. 5. **Die Migrationskette abarbeiten.** `tools/wikitool migrate status` listet jetzt alle offenen Migrationen in der Reihenfolge, in der sie laufen müssen - bei einem Sprung über mehrere Versionen sind das mehrere. Für jede: das genannte Dokument unter @@ -204,15 +227,30 @@ dieser Unterschied ist der Zustand, in dem sich jede Instanz mitten im Upgrade b `done` verweigert jede Version, die nicht das nächste Glied ist - eine übersprungene Migration hinterlässt einen Korpus in einer Form, die keine Version beschreibt. Ein abgebrochenes Upgrade wird durch erneutes `migrate status` fortgesetzt. -6. Prüfen: `tools/wikitool migrate verify --from `, dann `doctor`, +6. Prüfen: `tools/wikitool migrate verify --from `, dann `doctor`, `docs verify`, `instructions verify` und `lint`. Zum Schluss `tools/wikitool instructions sync` (die Skills sind Kopien) und die Agent-Session neu - starten. + starten. `dist upgrade` nennt diese Reihenfolge im eigenen Abschlussbericht, führt aber keinen + der Schritte selbst aus. `doctor` warnt, solange `kb_version` hinter `VERSION` zurückliegt und noch Migrationen offen sind. Einer Instanz, die älter ist als `.wikitool-kb.json`, fehlt die Datei ganz - dann einmalig `tools/wikitool migrate baseline ` aufrufen; geraten wird nichts. +**Fallstricke.** Eine Instanz ohne lokale `.wikitool-release.json` (oder eine ohne `files`-Block, +aus der Zeit vor `4.5.0`) hat für `dist upgrade` keine Basis, gegen die es eine lokale Änderung +erkennen könnte, und verweigert den Tausch - dafür gibt es heute keine Reparatur (siehe Gitea #7). +Der Befehl lädt selbst nichts herunter: `` muss vorher aus Weg A +geholt werden, und ein Tarball muss genau ein Top-Level-Verzeichnis enthalten - die Form, in der +`.gitea/workflows/release.yml` es baut. + +Vor `4.5.0` stand hier ein rein manueller Ablauf (Maschinerie von Hand kopieren, `kb/CONTRACT.md` +eingeschlossen, sha256-Vergleich von Hand). `dist upgrade` ersetzt genau diesen Teil; wer ihn +dennoch von Hand nachvollziehen will oder muss (ein Werkzeug, das `wikitool` selbst nicht +ausführen kann), findet die Dateiliste im `files`-Block der `.wikitool-release.json` und die +Ausnahmen (`kb/CONVENTIONS.md`, `kb/*/COLLECTION.md`, `.wikitool-kb.json`) in +[tools/CONTRACT.md](tools/CONTRACT.md)s `dist upgrade`-Zeile. + ### Sonderfall: Update von 1.x auf 2.0.0 Mit `2.0.0` wurde das Ursprungs-Repo von `torben/llm-wiki-test1` auf `torben/chemenu` diff --git a/VERSION b/VERSION index 94cd377..04585d5 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -4.5.0-beta.3 +4.5.0-beta.4 diff --git a/tools/CONTRACT.md b/tools/CONTRACT.md index 9d4d1a5..626f81b 100644 --- a/tools/CONTRACT.md +++ b/tools/CONTRACT.md @@ -72,6 +72,7 @@ tools/wikitool --help | `eval sessions [--json]` | List the sessions that have a trace under `reports/telemetry/`, most recent first. Read-only and exempt from the Iteration Budget Gate | | `eval score [--session ] [--json] [--markdown out.md] [--save] [--fail-on-error]` | Score one traced session: structural state from `lint`'s own checks (L1) plus trajectory rules over the trace (L2) - was a refused call repeated unchanged, was a gate flag passed without that gate having refused anything, did a publish of `kb/` pages go unlogged. Defaults to the current session. `--save` writes `reports/evals//.{json,md}`. Read-only over `kb/` and exempt from the budget; see [../EVALS.md](../EVALS.md) | | `dist export [--dry-run] [--source-repo U] [--source-commit SHA] [--release-url U] [--update-url U]` | Write a contentless, distributable copy of this repo's machinery into an empty `` directory: `AGENTS.md`/`README.md`/`EVALS.md` with any `...` region removed, `instructions/` (minus `instructions/dev/`), `types/` (the `root: kb` page type-specs and their schemas re-keyed as `.template`, the stack's own verbatim), `docs/` verbatim, `tools/` (no venv/caches), the `.github/hooks/`+`.vibe/` session-tracing config plus `.claude/settings.json`, `kb/CONTRACT.md` (no pages, no areas), empty `raw/{articles,documents,notes,assets}/`, `VERSION`, `USER.md.template`/`SOUL.md.template` plus `kb/CONVENTIONS.md.template` and each collection's contract re-keyed as `kb//COLLECTION.md.template` (the templates ship; the filled `USER.md`/`SOUL.md`/`kb/CONVENTIONS.md`/`kb//COLLECTION.md`/`types/.md` never do - all of them bind their instance and none are the stack's to decide, and `find_leaks` refuses a plan carrying one), and a generated `.wikitool-release.json` stamp (version, export date, origin, and a sha256 per exported file - the base a later upgrade would compare against). The four origin options only fill stamp fields: `export` never calls git and cannot discover them. Refuses a non-empty target, and a tree with no `VERSION`. See `instructions/setup-instance.md`. One-way: there is no command that reconstructs a distributed instance into a dev instance - work on the stack in the origin repo (or a new dev instance exported from it) instead | +| `dist upgrade [--dry-run] [--keep-local] [--prune] [--pre]` | Apply a stack update `dist export` produced - the write half of `version check`. Never downloads anything: `` is an already-fetched export directory or `.tar.gz` release archive (verified against a sibling `.sha256` if one is present; WARNs, does not block, if it is absent), which must unpack to exactly one top-level directory - the shape `.gitea/workflows/release.yml` packs. The write set is exactly the *new* `.wikitool-release.json`'s `files` block, minus what an export re-seeds from a blank template every time (`kb/log.md`, `raw/*/.gitkeep` - `chemenu.ownership.is_export_stub`) or seeds once and the instance owns from then on (`.wikitool-kb.json`, `CHANGES.md` - `chemenu.ownership.is_upgrade_preserved`), plus the stamp itself, always rewritten. Every candidate path is classified against the *local* `.wikitool-release.json`'s recorded digest for it: unchanged is overwritten silently, absent from the old stamp is created, and locally modified or locally deleted is **never** silently overwritten - the run aborts with the full list unless `--keep-local` says to proceed and leave every one of them untouched. A path in the old stamp but not the new one is reported as no longer part of the release and left alone unless `--prune` is passed, which removes it only if it is still unchanged since installation. Reports the migration chain the new machinery would owe (`chemenu.kb_state.chain` over the *new* tree's `instructions/migrations/`, read via a `directory` argument to `load_migrations`) but never runs any of it - there is no `migrate run`. Refuses before touching the source at all when: `VERSION` or `.wikitool-release.json` (with a `files` block) is missing locally, `.wikitool-kb.json` is missing, a migration is already outstanding against the *installed* machinery, or the working tree is dirty (not being a git repository at all is a WARN, not a refusal). Refuses after reading the source when: it carries no `VERSION`/`.wikitool-release.json`/`files` block, its version is older than or equal to the installed one (equal is a no-op success), or it is a pre-release (`-beta.N`) without `--pre`. Reports, but does not block on, a crossed compatibility boundary. Never touches git - no commit, no push (invariant 5). See Gitea #7 and `INSTALL.md` § "Eine Instanz aktualisieren" | | `version show [--json]` | Print this instance's stack version and where it came from (development tree, or a distribution with its export date and origin). Bare `wikitool version` is an alias for this. Read-only, offline, and **exempt from the Iteration Budget Gate** | | `version check [--url U] [--timeout S] [--json]` | Ask the origin's release feed whether a newer stack exists, and whether the step crosses a compatibility boundary (`state: current\|update\|migration\|ahead`). **The only command in `wikitool` that makes a network call** - never reached implicitly from another command, needs no key, times out, and reports an unreachable feed as an error rather than as "up to date". The feed is `$WIKITOOL_UPDATE_URL`, else the release stamp's, else the built-in origin; `$WIKITOOL_UPDATE_TOKEN` is only needed if that feed is not readable anonymously. Read-only and exempt from the budget gate | | `version notes [--version X.Y.Z]` | Print one version's `CHANGES.md` entry, for use as release notes (default: this tree's `VERSION`). Read-only and exempt from the budget gate | @@ -192,6 +193,7 @@ is atomic, and whether a retry is safe. | `instructions sync` / `verify` / `list` | Nothing found under `instructions/`, a malformed instruction or `SKILL.md`, a published copy that drifted from its source, an instruction nothing references (or, for `manual: true`, one that IS linked from AGENTS.md or a skill and so risks running implicitly), something under `instructions/dev/` referenced from outside it and outside a dist:strip block, or (sync) a target directory that is not a published skill and `--force` was not passed | `sync` rewrites one directory per target (idempotent); `verify`/`list` are read-only | Fix the flagged file, then re-run. For drift, re-run `sync`: the source under `instructions/` always wins, and a published copy is never edited directly | | `docs verify` | A command, contract, or type-form mismatch was found | Read-only | Fix the documentation it names, then re-run | | `dist export` | Target exists and is not empty, is not a directory, or the tree has no readable `VERSION` | Yes - nothing is written until every file is planned | Point `` at an empty (or new) directory and retry. Never merge into a non-empty one by hand | +| `dist upgrade` | Missing local `VERSION`/`.wikitool-release.json`(`files`)/`.wikitool-kb.json`, a migration already outstanding against the installed machinery, a dirty working tree, a source with no `VERSION`/stamp/`files` block, a source version that is older than, equal to, or (without `--pre`) a pre-release relative to the installed one, or one or more locally changed files without `--keep-local` | **Yes for the refusal cases above - nothing is written.** Once writing starts it is a plain sequential file copy with no partial-state cleanup: an interruption mid-copy (killed process, disk full) can leave the tree part-old, part-new | For every refusal above: fix the named precondition and retry - none of them are transient. For locally changed files: reconcile them by hand and retry, or re-run with `--keep-local` to proceed and leave them untouched (repeatable - it reports the same files again on every subsequent run until they stop diverging). An interrupted write is not resumed automatically; compare the tree against the printed classification and finish or revert by hand | | `version show` / `version notes` | `VERSION` is missing or unparseable; for `notes`, no `CHANGES.md` entry names the version asked for | Read-only | Fix `VERSION`, or write the changelog entry (`version bump` writes its heading). Safe to retry | | `version check` | The feed could not be reached, answered non-JSON, or carried no `tag_name`. **Never** answers "up to date" for a question it could not ask | Read-only, no local writes | A network failure is transient - retry once, then report it. HTTP 401/403 names `$WIKITOOL_UPDATE_TOKEN`; 404 means no release exists yet or the URL points at the wrong repo | | `version bump` | More or fewer than one of `--major/--minor/--patch`, an empty `--title`, a missing `VERSION`/`CHANGES.md`, `VERSION` and the changelog's newest entry naming different versions, an escalation to a boundary crossing without `--breaking` or with neither a migration document nor `--no-migration`, or `--breaking`/`--no-migration` on a bump that crosses nothing | No - `VERSION` then `CHANGES.md` | **Not idempotent**: a second run escalates or continues the candidate again. If the outcome is uncertain, read `VERSION` and the top of `CHANGES.md` before retrying | diff --git a/tools/chemenu/commands/dist_cmd.py b/tools/chemenu/commands/dist_cmd.py index 1f8ebf6..5f72ace 100644 --- a/tools/chemenu/commands/dist_cmd.py +++ b/tools/chemenu/commands/dist_cmd.py @@ -33,14 +33,20 @@ import hashlib import json import os import re +import shutil import stat +import subprocess +import tarfile +import tempfile +from contextlib import contextmanager +from dataclasses import dataclass from pathlib import Path from typing import Callable, NamedTuple, Optional, Union import typer from chemenu import config, conventions, kb_collections, kb_state, ownership, version as version_mod -from chemenu.commands._util import fail, rel_path, success, today_iso +from chemenu.commands._util import console, fail, rel_path, success, today_iso app = typer.Typer(help="Build a distributable copy of the wiki machinery.") @@ -581,3 +587,405 @@ def run_export(target: Path, dry_run: bool = False, origin: Optional[Origin] = N _write_plan(target, plan) success(f"Exported {len(plan)} file(s) to {rel_path(target)}.") + + +# --- dist upgrade ------------------------------------------------------------ +# +# Apply a release `dist export` produced, rather than merely detecting one +# (`version check`). Gitea #7 has the full design; the short version: the +# write set is exactly the *new* stamp's `files` block, minus the paths an +# export seeds once and the instance owns from then on +# (`ownership.is_export_stub`, `ownership.is_upgrade_preserved`), plus the +# stamp itself. Every candidate path is classified against the *old* stamp's +# recorded digest - unchanged, locally modified, or locally deleted - and a +# modified/deleted file is never silently overwritten. This never calls a +# release feed; the caller supplies an already-downloaded tree or archive. + + +@dataclass(frozen=True) +class FileClassification: + """The four-way split of every path `dist upgrade` would touch, plus the + fifth direction (`removed`) that has no write set of its own.""" + + unchanged: list[str] + modified: list[str] + deleted: list[str] + new: list[str] + removed: list[str] + + @property + def blocked(self) -> list[str]: + """Locally changed paths - modified or deleted - which are never + silently overwritten.""" + return sorted(self.modified + self.deleted) + + +def _write_candidates(new_files: dict) -> set[str]: + """Every path `dist upgrade` may write, from the new stamp's `files` + block: everything except the paths an export re-seeds from a blank + template every time (`ownership.is_export_stub`) and the paths an export + seeds once and the instance owns afterward (`ownership.is_upgrade_preserved`). + The release stamp itself is added separately - it is never a member of its + own `files` block, see `build_stamp`.""" + return { + relative + for relative in new_files + if not ownership.is_export_stub(Path(relative).name) + and not ownership.is_upgrade_preserved(relative) + } + + +def _classify_files(old_files: dict, new_files: dict) -> FileClassification: + candidates = _write_candidates(new_files) + recorded_for_candidates = {r: d for r, d in old_files.items() if r in candidates} + statuses = kb_state.compare_against_stamp(recorded_for_candidates) + + unchanged: list[str] = [] + modified: list[str] = [] + deleted: list[str] = [] + new: list[str] = [] + for relative in sorted(candidates): + if relative not in old_files: + new.append(relative) + continue + status = statuses[relative] + if status == kb_state.UNCHANGED: + unchanged.append(relative) + elif status == kb_state.MODIFIED: + modified.append(relative) + else: + deleted.append(relative) + + removed = sorted(set(old_files) - set(new_files)) + return FileClassification(unchanged, modified, deleted, new, removed) + + +def _verify_sha256_sidecar(archive: Path) -> None: + """WARN, never fail, on a missing sidecar - only a corrupted one that + *is* present is a reason to stop, per Gitea #7's design table.""" + sidecar = archive.with_name(archive.name + ".sha256") + if not sidecar.is_file(): + console.print( + f"[yellow]WARN[/yellow] No {sidecar.name} beside {archive.name} - the archive's " + "integrity is not being checked before it is extracted." + ) + return + expected = sidecar.read_text(encoding="utf-8").strip().split()[0:1] + actual = hashlib.sha256(archive.read_bytes()).hexdigest() + if not expected or expected[0].lower() != actual.lower(): + fail( + f"{archive.name} does not match {sidecar.name}: expected " + f"{expected[0] if expected else '(unreadable)'}, got {actual}. Re-download the " + "release archive rather than trusting one that failed its own checksum." + ) + + +def _extract_single_top_level_dir(archive: Path, dest: Path) -> Path: + """Extract `archive` into `dest` and return the one top-level directory it + contained - the shape `.gitea/workflows/release.yml` packs (see its + `Build the distribution tarball` step). Refuses anything else rather than + guessing which part is the machinery.""" + with tarfile.open(archive) as tf: + names = [n for n in tf.getnames() if n not in ("", ".")] + top_levels = {n.split("/", 1)[0] for n in names} + if len(top_levels) != 1: + fail( + f"{archive.name} does not have exactly one top-level directory (found " + f"{len(top_levels)}: {', '.join(sorted(top_levels)) or '(empty archive)'}) - this " + "is not the shape a release tarball has, and `dist upgrade` refuses to guess " + "which part is the machinery." + ) + return dest # unreachable: fail() raises typer.Exit + try: + tf.extractall(dest, filter="data") # noqa: S202 - trusted local archive, path-checked below + except TypeError: + # Python < 3.12 has no `filter=` argument. Same guard by hand: + # refuse any member whose extracted path would land outside dest. + resolved_dest = dest.resolve() + for member in tf.getmembers(): + if not (resolved_dest / member.name).resolve().is_relative_to(resolved_dest): + fail( + f"{archive.name} contains a path that escapes the extraction directory: " + f"{member.name}" + ) + return dest # unreachable + tf.extractall(dest) # noqa: S202 - every member path-checked above + return dest / next(iter(top_levels)) + + +@contextmanager +def _resolved_source(source: Path): + """Yield the directory holding a distribution export: `source` itself if + it already is one, or the single top-level directory of a `.tar.gz` + extracted into a scratch directory that is cleaned up afterward.""" + if source.is_dir(): + yield source + return + if not source.is_file(): + fail(f"{source} does not exist.") + return + _verify_sha256_sidecar(source) + with tempfile.TemporaryDirectory(prefix="wikitool-upgrade-") as tmp: + yield _extract_single_top_level_dir(source, Path(tmp)) + + +def _git_working_tree_status() -> Optional[str]: + """`git status --porcelain` for `config.ROOT`, or None if it is not a git + repository at all - which is a valid, if unprotected, state for a tarball + instance, not a reason to refuse.""" + result = subprocess.run( + ["git", "-C", str(config.ROOT), "status", "--porcelain"], + capture_output=True, + text=True, + ) + return result.stdout if result.returncode == 0 else None + + +def _report_plan( + classification: FileClassification, + migration_chain: list["kb_state.Migration"], + boundary_crossing: bool, + local_version: "version_mod.Version", + new_version: "version_mod.Version", +) -> None: + console.print(f"{local_version} -> {new_version}") + if boundary_crossing: + console.print( + f"[bold yellow]Crosses a compatibility boundary[/bold yellow] " + f"({local_version.compat_key} -> {new_version.compat_key}) - this is not a drop-in " + "swap; check the release notes before proceeding." + ) + console.print( + f"{len(classification.unchanged)} unchanged, {len(classification.new)} new, " + f"{len(classification.blocked)} locally changed, {len(classification.removed)} removed " + "from the release." + ) + if classification.modified: + console.print(f"[bold]Locally modified ({len(classification.modified)}):[/bold]") + for relative in classification.modified: + console.print(f" - {relative}") + if classification.deleted: + console.print(f"[bold]Locally deleted ({len(classification.deleted)}):[/bold]") + for relative in classification.deleted: + console.print(f" - {relative}") + if classification.removed: + console.print("[dim]No longer part of the release, not written or removed by default:[/dim]") + for relative in classification.removed: + console.print(f" [dim]- {relative}[/dim]") + if migration_chain: + console.print( + f"[cyan]{len(migration_chain)} migration(s) will be outstanding after this " + "upgrade, in this order:[/cyan]" + ) + for position, migration in enumerate(migration_chain, start=1): + console.print(f" {position}. {migration.target} {migration.name} ({migration.kind})") + console.print("Report only - `dist upgrade` never runs a migration. See `wikitool migrate status`.") + + +@app.command("upgrade") +def upgrade_command( + source: Path = typer.Argument( + ..., help="An extracted distribution directory, or a release .tar.gz archive" + ), + dry_run: bool = typer.Option( + False, "--dry-run", help="Classify and report, without writing anything" + ), + keep_local: bool = typer.Option( + False, "--keep-local", + help="Proceed even with locally changed files - leave each one untouched rather than aborting", + ), + prune: bool = typer.Option( + False, "--prune", + help="Also delete files the new release no longer ships, if they are unchanged since install", + ), + allow_pre: bool = typer.Option( + False, "--pre", help="Allow a pre-release (-beta.N) source tree - release.yml never publishes one", + ), +): + """Apply a stack update `dist export` produced - the write half of + `version check`. Never downloads anything: `source` is an already-fetched + export directory or `.tar.gz` archive. Writes exactly the new release + stamp's `files` block, minus what an export re-seeds every time + (`kb/log.md`, `raw/*/.gitkeep`) or seeds once and the instance owns from + then on (`.wikitool-kb.json`, `CHANGES.md`), classifying every candidate + against the *old* stamp's recorded digest: unchanged files are + overwritten silently, new files are created, and a locally modified or + deleted file is never silently overwritten - `dist upgrade` aborts unless + `--keep-local` says to leave it alone. Reports the migration chain the new + machinery would owe without running any of it (there is no `migrate run`). + Refuses on a missing local release stamp, a downgrade, a pre-release + source without `--pre`, or a dirty working tree. Never touches git. + See Gitea #7 and `INSTALL.md` § "Eine Instanz aktualisieren".""" + run_upgrade( + source, dry_run=dry_run, keep_local=keep_local, prune=prune, allow_pre=allow_pre + ) + + +def run_upgrade( + source: Path, + dry_run: bool = False, + keep_local: bool = False, + prune: bool = False, + allow_pre: bool = False, +) -> None: + """The upgrade itself, free of Typer's option objects - see `run_export` + for why this split exists.""" + try: + local_version = version_mod.read_version() + except version_mod.VersionError as exc: + fail(f"{exc} - this tree has no stack version to upgrade from.") + return + + old_stamp = version_mod.read_stamp() + if not old_stamp or not isinstance(old_stamp.get("files"), dict): + fail( + f"No local {version_mod.RELEASE_STAMP_FILENAME} (or it carries no `files` block). " + "Without it, `dist upgrade` cannot tell a file this instance edited from one it " + "merely received, and it refuses to guess. A checkout with shared git history takes " + "stack updates via `wikitool upstream merge` instead - it has the same information " + "as a merge base. A tarball instance that has lost its stamp has no repair path " + "today; see Gitea #7 \"Bewusst offen gelassen\"." + ) + return + old_files = old_stamp["files"] + + try: + kb_version = kb_state.read_kb_version() + except version_mod.VersionError as exc: + fail(str(exc)) + return + if kb_version is None: + fail( + f"{kb_state.KB_STATE_FILENAME} is missing - this instance has never declared what " + "shape its content is in. Run `wikitool migrate baseline ` before upgrading." + ) + return + + outstanding = kb_state.chain(kb_state.load_migrations(), kb_version, local_version.base) + if outstanding: + fail( + f"{len(outstanding)} migration(s) are already outstanding against the installed " + f"machinery ({kb_version} -> {local_version}) - `wikitool migrate status` names them. " + "Finish them before upgrading further: a machinery swap on top of an unfinished " + "migration leaves the corpus in a shape no version describes." + ) + return + + tree_status = _git_working_tree_status() + if tree_status is None: + console.print( + "[yellow]WARN[/yellow] Not a git repository (or git is unavailable) - proceeding " + "without the dirty-tree check a repository would get." + ) + elif tree_status.strip(): + fail( + "Working tree is not clean (`git status --porcelain` printed something). " + "`dist upgrade` refuses to start on a dirty tree so a refusal never has to guess " + "which changes were already there. Commit or stash first." + ) + return + + with _resolved_source(source) as new_root: + version_path = new_root / version_mod.VERSION_FILENAME + if not version_path.is_file(): + fail(f"{rel_path(new_root)} has no VERSION - not a distribution export.") + return + try: + new_version = version_mod.Version.parse(version_path.read_text(encoding="utf-8")) + except version_mod.VersionError as exc: + fail(str(exc)) + return + + if new_version.is_prerelease and not allow_pre: + fail( + f"{new_version} is a running candidate (-beta.N). `.gitea/workflows/release.yml` " + "never publishes one, so a candidate tree can only come from a dev checkout by " + "hand - pass --pre if that is deliberate." + ) + return + if new_version < local_version: + fail(f"{new_version} is older than the installed {local_version} - refusing a downgrade.") + return + if new_version == local_version: + success(f"Already at {local_version}. Nothing to do.") + return + + stamp_path = new_root / version_mod.RELEASE_STAMP_FILENAME + if not stamp_path.is_file(): + fail(f"{rel_path(new_root)} has no {version_mod.RELEASE_STAMP_FILENAME} - not a distribution export.") + return + try: + new_stamp = json.loads(stamp_path.read_text(encoding="utf-8")) + except (json.JSONDecodeError, OSError) as exc: + fail(f"{version_mod.RELEASE_STAMP_FILENAME} in the source is not readable JSON: {exc}") + return + new_files = new_stamp.get("files") if isinstance(new_stamp, dict) else None + if not isinstance(new_files, dict): + fail(f"{version_mod.RELEASE_STAMP_FILENAME} in the source carries no `files` block.") + return + + classification = _classify_files(old_files, new_files) + migration_chain = kb_state.chain( + kb_state.load_migrations(new_root / "instructions" / kb_state.MIGRATIONS_SUBDIR), + kb_version, + new_version.base, + ) + boundary_crossing = local_version.compat_key != new_version.compat_key + + _report_plan(classification, migration_chain, boundary_crossing, local_version, new_version) + + # Dry-run's whole purpose is to preview this classification - including + # the blocked list - without raising, so it must be checked before the + # abort below rather than after: a blocked file must never turn + # `--dry-run` into a non-zero exit, or the flag stops being safe to run + # freely. + if dry_run: + success(f"Dry run: would upgrade {local_version} -> {new_version}. Nothing written.") + return + + if classification.blocked and not keep_local: + fail( + f"{len(classification.blocked)} locally changed file(s) (listed above) would be " + "silently overwritten. Pass --keep-local to upgrade anyway and leave every one of " + "them untouched, or reconcile them by hand first. Nothing was written." + ) + return + + to_write = sorted(classification.unchanged + classification.new) + for relative in to_write: + src = new_root / relative + dst = config.ROOT / relative + dst.parent.mkdir(parents=True, exist_ok=True) + shutil.copy2(src, dst) + shutil.copy2(stamp_path, config.ROOT / version_mod.RELEASE_STAMP_FILENAME) + + pruned: list[str] = [] + if prune: + for relative in classification.removed: + digest = old_files.get(relative) + if digest is None: + continue + status = kb_state.compare_against_stamp({relative: digest}).get(relative) + if status != kb_state.UNCHANGED: + continue + target = config.ROOT / relative + if target.is_file(): + target.unlink() + pruned.append(relative) + + skipped = classification.blocked if keep_local else [] + summary = ( + f"Upgraded {local_version} -> {new_version}: {len(to_write)} file(s) written" + + (f", {len(skipped)} left untouched (--keep-local)" if skipped else "") + + (f", {len(pruned)} pruned" if pruned else "") + + "." + ) + if migration_chain: + summary += ( + f" {len(migration_chain)} migration(s) now outstanding - run `wikitool migrate status`." + ) + summary += ( + " Nothing was committed. Now run, in order: `wikitool instructions sync`, `doctor`, " + "`docs verify`, `instructions verify`, `lint` - then restart the agent session." + ) + success(summary) diff --git a/tools/chemenu/kb_state.py b/tools/chemenu/kb_state.py index 85b57f3..dd17c4b 100644 --- a/tools/chemenu/kb_state.py +++ b/tools/chemenu/kb_state.py @@ -129,21 +129,27 @@ def migrations_dir() -> Path: return config.INSTRUCTIONS_DIR / MIGRATIONS_SUBDIR -def load_migrations() -> list[Migration]: - """Every migration document, sorted by target version. +def load_migrations(directory: Optional[Path] = None) -> list[Migration]: + """Every migration document under `directory`, sorted by target version. - A malformed one is skipped rather than fatal here - `instructions verify` - is what reports it, and `migrate status` staying usable while one document - is broken is worth more than a second error path. + `directory` defaults to this instance's own `instructions/migrations/`. + `dist upgrade` (Gitea #7) passes the *new* tree's migrations directory + instead: the migrations owed after an upgrade are documented in the + machinery being installed, not in the one still on disk - an old instance + cannot know a new version's migration chain by reading its own tree. + + A malformed document is skipped rather than fatal here - `instructions + verify` is what reports it, and `migrate status` staying usable while one + document is broken is worth more than a second error path. """ from chemenu.frontmatter_io import read_page - directory = migrations_dir() - if not directory.is_dir(): + base = directory if directory is not None else migrations_dir() + if not base.is_dir(): return [] migrations: list[Migration] = [] - for path in sorted(directory.glob("*.md")): + for path in sorted(base.glob("*.md")): try: frontmatter, _ = read_page(path) except Exception: # noqa: BLE001 - a broken document is verify's finding, not ours @@ -232,25 +238,60 @@ def next_link( # --- what this instance changed about what it was given -------------------- +# What `compare_against_stamp` answers for one path: present and matching its +# recorded digest, present but not matching, or gone entirely. `dist upgrade` +# needs the three-way answer to tell a locally deleted file from a locally +# edited one; `divergent_files` (below) only ever needed the yes/no of +# "does this count as diverged", which both `MODIFIED` and `DELETED` answer +# the same way. +UNCHANGED = "unchanged" +MODIFIED = "modified" +DELETED = "deleted" + + +def compare_against_stamp(stamp_files: dict, root: Optional[Path] = None) -> dict[str, str]: + """Classify every path in `stamp_files` (relative -> recorded sha256, the + shape of a release stamp's own `files` block) against what is actually on + disk under `root` - `UNCHANGED`, `MODIFIED`, or `DELETED`. + + `root` defaults to `config.ROOT`. The general form `divergent_files` is + built on: that function only ever asks the question against this + instance's own tree, but `dist upgrade` (Gitea #7) asks it against an + already-installed tree while planning what to write, and a *second* time + against the tree it just wrote, before recording the new stamp - two trees + neither of which is necessarily `config.ROOT`. + """ + import hashlib + + base = root if root is not None else config.ROOT + result: dict[str, str] = {} + for relative, digest in sorted(stamp_files.items()): + path = base / relative + if not path.is_file(): + result[relative] = DELETED + continue + current = "sha256:" + hashlib.sha256(path.read_bytes()).hexdigest() + result[relative] = UNCHANGED if current == digest else MODIFIED + return result + def divergent_files() -> Optional[list[str]]: """Files whose content no longer matches the release this instance installed. Reads the per-file sha256 in `.wikitool-release.json`, which `dist export` - has been writing since the stamp existed and which nothing has read until - now. Its own docstring says why it is there: it is the only way a later - upgrade can tell a file the instance *edited* from one it merely *received*. + has been writing since the stamp existed. Its own docstring says why it is + there: it is the only way a later upgrade can tell a file the instance + *edited* from one it merely *received* - `dist upgrade` (Gitea #7) is that + later upgrade, built on the general `compare_against_stamp` above. - That distinction is what makes an `offered` migration actionable. The stack - proposing a better `entity` template needs to know whether it may be copied - over or whether the instance has its own version that a person has to - reconcile - and only the recorded hash can answer that. + That distinction is also what makes an `offered` migration actionable. The + stack proposing a better `entity` template needs to know whether it may be + copied over or whether the instance has its own version that a person has + to reconcile - and only the recorded hash can answer that. Returns None when the question is unanswerable (a development tree, which carries no stamp), which is different from `[]` (nothing diverged). """ - import hashlib - from chemenu import version as version_mod try: @@ -263,13 +304,5 @@ def divergent_files() -> Optional[list[str]]: if not isinstance(recorded, dict): return None - divergent: list[str] = [] - for relative, digest in sorted(recorded.items()): - path = config.ROOT / relative - if not path.is_file(): - divergent.append(relative) - continue - current = "sha256:" + hashlib.sha256(path.read_bytes()).hexdigest() - if current != digest: - divergent.append(relative) - return divergent + statuses = compare_against_stamp(recorded) + return [relative for relative, status in statuses.items() if status != UNCHANGED] diff --git a/tools/chemenu/ownership.py b/tools/chemenu/ownership.py index 52e327c..6b14548 100644 --- a/tools/chemenu/ownership.py +++ b/tools/chemenu/ownership.py @@ -62,3 +62,26 @@ def is_export_stub(name: str) -> bool: """Whether `name` (a bare filename, not a path) is one `dist export` overwrites with a fresh stub of its own rather than shipping verbatim.""" return name in EXPORT_STUB_NAMES + + +# Root-relative paths `dist export` seeds once, from a template it owns, and +# which the instance owns exclusively from that point on. `dist upgrade` +# (Gitea #7) must never overwrite them, even though they sit in the release +# stamp's `files` block like any other planned file - the same shape as +# `EXPORT_STUB_NAMES` above, but keyed by full path rather than bare filename, +# since nothing else at the repo root gets this treatment and a bare-filename +# match would be too broad here. +# +# `.wikitool-kb.json` is `migrate done`'s state file: an upgrade that resets it +# declares a content shape nobody actually produced. `CHANGES.md` is the +# instance's own changelog, not the stack's - `dist export` seeds it from a +# blank template (`dist_templates/CHANGES.md`) the same way it seeds +# `kb/log.md`, and overwriting it on upgrade would erase every entry the +# instance ever wrote for itself. +UPGRADE_PRESERVED_PATHS = (".wikitool-kb.json", "CHANGES.md") + + +def is_upgrade_preserved(relative: str) -> bool: + """Whether `relative` (a plan-relative path from the repo root, e.g. + "CHANGES.md") is one `dist upgrade` must never write.""" + return relative in UPGRADE_PRESERVED_PATHS diff --git a/tools/chemenu/tests/test_dist_upgrade.py b/tools/chemenu/tests/test_dist_upgrade.py new file mode 100644 index 0000000..2c93bb9 --- /dev/null +++ b/tools/chemenu/tests/test_dist_upgrade.py @@ -0,0 +1,386 @@ +"""Tests for `wikitool dist upgrade`: classification against the locally +installed release stamp, the write set, migration-chain reporting without +execution, and every refusal before anything is written. See Gitea #7.""" +from __future__ import annotations + +import hashlib +import json +import subprocess +import tarfile +from pathlib import Path + +import pytest +import typer + +from chemenu import config, kb_state, version as version_mod +from chemenu.commands import dist_cmd + + +def _digest(text: str) -> str: + return "sha256:" + hashlib.sha256(text.encode("utf-8")).hexdigest() + + +def _write_stamp(path: Path, version: str, files: dict[str, str]) -> None: + path.write_text( + json.dumps( + { + "schema": version_mod.STAMP_SCHEMA, + "version": version, + "exported_at": "2026-01-01", + "source_repo": None, + "source_commit": None, + "release_url": None, + "update_url": version_mod.DEFAULT_UPDATE_URL, + "files": files, + }, + indent=2, + ), + encoding="utf-8", + ) + + +def write_migration(directory: Path, target: str, slug: str) -> None: + directory.mkdir(parents=True, exist_ok=True) + (directory / f"{target}-{slug}.md").write_text( + "---\n" + "type: types/instruction.md\n" + f"name: {target}-{slug}\n" + f"description: Migration to {target}.\n" + "manual: true\n" + f"migrates_to: {target}\n" + "migration_kind: assisted\n" + "obligation: required\n" + "---\n\n# Migration\n\nSteps.\n", + encoding="utf-8", + ) + + +def set_kb_version(root: Path, version: str) -> None: + (root / kb_state.KB_STATE_FILENAME).write_text( + json.dumps({"schema": 1, "kb_version": version, "applied": []}), encoding="utf-8" + ) + + +@pytest.fixture +def instance(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Path: + """A minimal tarball instance: VERSION 1.0.0, a local release stamp + recording two tracked files exactly as installed, content stays at + 1.0.0 with nothing outstanding, and no .git directory (the WARN path + for the dirty-tree check).""" + root = tmp_path / "instance" + root.mkdir() + monkeypatch.setattr(config, "ROOT", root) + + (root / "VERSION").write_text("1.0.0\n", encoding="utf-8") + (root / "AGENTS.md").write_text("core\n", encoding="utf-8") + (root / "tools").mkdir() + (root / "tools" / "wikitool").write_text("#!/bin/sh\n", encoding="utf-8") + + files = { + "AGENTS.md": _digest("core\n"), + "tools/wikitool": _digest("#!/bin/sh\n"), + } + _write_stamp(root / version_mod.RELEASE_STAMP_FILENAME, "1.0.0", files) + set_kb_version(root, "1.0.0") + return root + + +def _release(tmp_path: Path, name: str, version: str, files_content: dict[str, str]) -> Path: + """A second, independent tree shaped like a `dist export` output: VERSION, + a release stamp whose `files` block matches `files_content` exactly, and + the files themselves.""" + root = tmp_path / name + root.mkdir() + (root / "VERSION").write_text(f"{version}\n", encoding="utf-8") + files: dict[str, str] = {} + for relative, content in files_content.items(): + path = root / relative + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(content, encoding="utf-8") + files[relative] = _digest(content) + _write_stamp(root / version_mod.RELEASE_STAMP_FILENAME, version, files) + return root + + +# --- classification / dry-run ----------------------------------------------- + + +def test_dry_run_classifies_every_case_and_writes_nothing(instance, tmp_path): + (instance / "AGENTS.md").write_text("locally edited\n", encoding="utf-8") + (instance / "tools" / "wikitool").unlink() + + release = _release( + tmp_path, "release", "1.1.0", + { + "AGENTS.md": "core\n", # locally modified + "tools/wikitool": "#!/bin/sh\n", # locally deleted + "types/entity.md": "new page type\n", # new in the release + }, + ) + + dist_cmd.run_upgrade(release, dry_run=True) + + assert (instance / "AGENTS.md").read_text(encoding="utf-8") == "locally edited\n" + assert not (instance / "tools" / "wikitool").exists() + assert not (instance / "types" / "entity.md").exists() + assert json.loads((instance / version_mod.RELEASE_STAMP_FILENAME).read_text())["version"] == "1.0.0" + + +def test_removed_file_is_reported_and_left_alone_without_prune(instance, tmp_path): + release = _release(tmp_path, "release", "1.1.0", {"AGENTS.md": "core\n"}) + # tools/wikitool is in the old stamp but absent from the new one. + dist_cmd.run_upgrade(release, dry_run=True) + assert (instance / "tools" / "wikitool").is_file() + + +# --- local changes are never silently overwritten --------------------------- + + +def test_locally_modified_file_blocks_the_upgrade_by_default(instance, tmp_path): + (instance / "AGENTS.md").write_text("locally edited\n", encoding="utf-8") + release = _release( + tmp_path, "release", "1.1.0", + {"AGENTS.md": "core v2\n", "tools/wikitool": "#!/bin/sh\n"}, + ) + with pytest.raises(typer.Exit) as excinfo: + dist_cmd.run_upgrade(release) + assert excinfo.value.exit_code == 1 + assert (instance / "AGENTS.md").read_text(encoding="utf-8") == "locally edited\n" + assert json.loads((instance / version_mod.RELEASE_STAMP_FILENAME).read_text())["version"] == "1.0.0" + + +def test_locally_deleted_file_blocks_the_upgrade_by_default(instance, tmp_path): + (instance / "tools" / "wikitool").unlink() + release = _release( + tmp_path, "release", "1.1.0", + {"AGENTS.md": "core\n", "tools/wikitool": "#!/bin/sh v2\n"}, + ) + with pytest.raises(typer.Exit): + dist_cmd.run_upgrade(release) + assert not (instance / "tools" / "wikitool").exists() + + +def test_keep_local_proceeds_and_leaves_the_changed_file_untouched(instance, tmp_path): + (instance / "AGENTS.md").write_text("locally edited\n", encoding="utf-8") + release = _release( + tmp_path, "release", "1.1.0", + {"AGENTS.md": "core v2\n", "tools/wikitool": "#!/bin/sh\n", "types/entity.md": "new\n"}, + ) + dist_cmd.run_upgrade(release, keep_local=True) + + # The locally changed file is untouched... + assert (instance / "AGENTS.md").read_text(encoding="utf-8") == "locally edited\n" + # ...but everything unchanged/new was still written. + assert (instance / "tools" / "wikitool").read_text(encoding="utf-8") == "#!/bin/sh\n" + assert (instance / "types" / "entity.md").read_text(encoding="utf-8") == "new\n" + assert json.loads((instance / version_mod.RELEASE_STAMP_FILENAME).read_text())["version"] == "1.1.0" + + +# --- the write set ----------------------------------------------------------- + + +def test_unchanged_and_new_files_are_written_silently(instance, tmp_path): + release = _release( + tmp_path, "release", "1.1.0", + {"AGENTS.md": "core\n", "tools/wikitool": "#!/bin/sh\n", "types/entity.md": "new\n"}, + ) + dist_cmd.run_upgrade(release) + assert (instance / "AGENTS.md").read_text(encoding="utf-8") == "core\n" + assert (instance / "types" / "entity.md").read_text(encoding="utf-8") == "new\n" + stamp = json.loads((instance / version_mod.RELEASE_STAMP_FILENAME).read_text()) + assert stamp["version"] == "1.1.0" + assert stamp["files"]["types/entity.md"] == _digest("new\n") + + +@pytest.mark.parametrize("preserved", [".wikitool-kb.json", "CHANGES.md", "kb/log.md", "raw/notes/.gitkeep"]) +def test_seeded_once_paths_are_never_written_even_if_the_release_stamp_lists_them( + instance, tmp_path, preserved +): + """The write set is the new stamp's `files` block minus what an export + re-seeds every time or seeds once and the instance owns from then on - + this is the AGENTS.md invariant 8 test: no separate literal list here, + only `chemenu.ownership`.""" + (instance / "CHANGES.md").write_text("instance's own changelog\n", encoding="utf-8") + (instance / kb_state.KB_STATE_FILENAME).write_text( + json.dumps({"schema": 1, "kb_version": "1.0.0", "applied": [{"migration": "x"}]}), + encoding="utf-8", + ) + release = _release( + tmp_path, "release", "1.1.0", + { + "AGENTS.md": "core\n", + "tools/wikitool": "#!/bin/sh\n", + preserved: "a fresh stub from the new release\n", + }, + ) + dist_cmd.run_upgrade(release) + if preserved == "CHANGES.md": + assert (instance / preserved).read_text(encoding="utf-8") == "instance's own changelog\n" + elif preserved == ".wikitool-kb.json": + state = json.loads((instance / preserved).read_text()) + assert state["applied"] == [{"migration": "x"}] + else: + assert not (instance / preserved).exists() + + +# --- migration chain: reported, never run ----------------------------------- + + +def test_migration_chain_is_reported_but_never_executed(instance, tmp_path): + release = _release( + tmp_path, "release", "1.1.0", + {"AGENTS.md": "core\n", "tools/wikitool": "#!/bin/sh\n"}, + ) + write_migration(release / "instructions" / "migrations", "1.1.0", "some-change") + + dist_cmd.run_upgrade(release) + + # The chain is reported, not applied: kb_version has not moved. + assert kb_state.read_kb_version().base == version_mod.Version(1, 0, 0) + + +def test_outstanding_local_migration_blocks_before_touching_the_source(instance, tmp_path): + """A migration owed against the *installed* machinery must be finished + first - the source is never even opened.""" + migrations = instance / "instructions" / "migrations" + write_migration(migrations, "1.0.0", "not-yet-done") + monkey_target = instance / "does-not-exist" # never read if this check fires first + + with pytest.raises(typer.Exit): + dist_cmd.run_upgrade(monkey_target) + + +# --- preconditions ----------------------------------------------------------- + + +def test_missing_local_stamp_blocks(instance, tmp_path): + (instance / version_mod.RELEASE_STAMP_FILENAME).unlink() + release = _release(tmp_path, "release", "1.1.0", {"AGENTS.md": "core\n"}) + with pytest.raises(typer.Exit): + dist_cmd.run_upgrade(release) + + +def test_missing_kb_version_blocks(instance, tmp_path): + (instance / kb_state.KB_STATE_FILENAME).unlink() + release = _release(tmp_path, "release", "1.1.0", {"AGENTS.md": "core\n"}) + with pytest.raises(typer.Exit): + dist_cmd.run_upgrade(release) + + +def test_prerelease_source_is_refused_without_pre_flag(instance, tmp_path): + release = _release(tmp_path, "release", "1.1.0-beta.1", {"AGENTS.md": "core\n"}) + with pytest.raises(typer.Exit): + dist_cmd.run_upgrade(release) + assert json.loads((instance / version_mod.RELEASE_STAMP_FILENAME).read_text())["version"] == "1.0.0" + + +def test_pre_flag_allows_a_prerelease_source(instance, tmp_path): + release = _release( + tmp_path, "release", "1.1.0-beta.1", + {"AGENTS.md": "core\n", "tools/wikitool": "#!/bin/sh\n"}, + ) + dist_cmd.run_upgrade(release, allow_pre=True) + assert json.loads((instance / version_mod.RELEASE_STAMP_FILENAME).read_text())["version"] == "1.1.0-beta.1" + + +def test_downgrade_is_refused(instance, tmp_path): + release = _release(tmp_path, "release", "0.9.0", {"AGENTS.md": "core\n"}) + with pytest.raises(typer.Exit): + dist_cmd.run_upgrade(release) + + +def test_equal_version_is_a_noop(instance, tmp_path): + release = _release(tmp_path, "release", "1.0.0", {"AGENTS.md": "core\n"}) + dist_cmd.run_upgrade(release) # must not raise + assert json.loads((instance / version_mod.RELEASE_STAMP_FILENAME).read_text())["version"] == "1.0.0" + + +def test_dirty_working_tree_blocks(instance, tmp_path): + subprocess.run(["git", "init", "-q", "-b", "main"], cwd=instance, check=True) + subprocess.run(["git", "config", "user.name", "Fixture Author"], cwd=instance, check=True) + subprocess.run(["git", "config", "user.email", "fixture@example.invalid"], cwd=instance, check=True) + (instance / "untracked.txt").write_text("dirty\n", encoding="utf-8") + + release = _release(tmp_path, "release", "1.1.0", {"AGENTS.md": "core\n"}) + with pytest.raises(typer.Exit): + dist_cmd.run_upgrade(release) + assert not (instance / "types").exists() + + +def test_source_that_does_not_exist_is_refused(instance, tmp_path): + with pytest.raises(typer.Exit): + dist_cmd.run_upgrade(tmp_path / "nowhere") + + +# --- prune ------------------------------------------------------------------- + + +def test_prune_removes_only_removed_files_still_unchanged_since_install(instance, tmp_path): + (instance / "extra.txt").write_text("shipped once, edited since\n", encoding="utf-8") + old_stamp = json.loads((instance / version_mod.RELEASE_STAMP_FILENAME).read_text()) + old_stamp["files"]["extra.txt"] = _digest("shipped once, unedited\n") # deliberately stale + old_stamp["files"]["gone.txt"] = _digest("also shipped once\n") + (instance / version_mod.RELEASE_STAMP_FILENAME).write_text(json.dumps(old_stamp), encoding="utf-8") + (instance / "gone.txt").write_text("also shipped once\n", encoding="utf-8") + + release = _release( + tmp_path, "release", "1.1.0", + {"AGENTS.md": "core\n", "tools/wikitool": "#!/bin/sh\n"}, + ) + dist_cmd.run_upgrade(release, prune=True) + + # gone.txt matched its recorded digest -> pruned. + assert not (instance / "gone.txt").exists() + # extra.txt was locally edited relative to its recorded digest -> kept. + assert (instance / "extra.txt").read_text(encoding="utf-8") == "shipped once, edited since\n" + + +# --- tarball sources ---------------------------------------------------------- + + +def _pack(release_dir: Path, archive: Path) -> None: + with tarfile.open(archive, "w:gz") as tf: + tf.add(release_dir, arcname=release_dir.name) + + +def test_tarball_with_more_than_one_top_level_entry_is_refused(instance, tmp_path): + scratch = tmp_path / "scratch" + (scratch / "a").mkdir(parents=True) + (scratch / "b").mkdir(parents=True) + (scratch / "a" / "x.txt").write_text("x\n", encoding="utf-8") + (scratch / "b" / "y.txt").write_text("y\n", encoding="utf-8") + archive = tmp_path / "bad.tar.gz" + with tarfile.open(archive, "w:gz") as tf: + tf.add(scratch / "a", arcname="a") + tf.add(scratch / "b", arcname="b") + + with pytest.raises(typer.Exit): + dist_cmd.run_upgrade(archive) + + +def test_tarball_source_is_extracted_and_applied(instance, tmp_path): + release_dir = _release( + tmp_path, "chemenu-stack-1.1.0", "1.1.0", + {"AGENTS.md": "core\n", "tools/wikitool": "#!/bin/sh\n"}, + ) + archive = tmp_path / "chemenu-stack-1.1.0.tar.gz" + _pack(release_dir, archive) + + dist_cmd.run_upgrade(archive) + + assert (instance / "AGENTS.md").read_text(encoding="utf-8") == "core\n" + assert json.loads((instance / version_mod.RELEASE_STAMP_FILENAME).read_text())["version"] == "1.1.0" + + +def test_tarball_sha256_sidecar_mismatch_is_refused(instance, tmp_path): + release_dir = _release( + tmp_path, "chemenu-stack-1.1.0", "1.1.0", {"AGENTS.md": "core\n"} + ) + archive = tmp_path / "chemenu-stack-1.1.0.tar.gz" + _pack(release_dir, archive) + (archive.with_name(archive.name + ".sha256")).write_text( + "0" * 64 + " chemenu-stack-1.1.0.tar.gz\n", encoding="utf-8" + ) + + with pytest.raises(typer.Exit): + dist_cmd.run_upgrade(archive)