feat: dist upgrade --latest downloads and verifies the release from the feed, --expect pins the version (#161)
Files changed: - CHANGES.md - INSTALL.md - VERSION - instructions/upgrade-instance.md - tools/CONTRACT.md - tools/chemenu/commands/dist_cmd.py - tools/chemenu/commands/version_cmd.py - tools/chemenu/tests/test_cli.py - tools/chemenu/tests/test_dist_upgrade.py - tools/chemenu/version.py
This commit is contained in:
1 parent
dd885db625
commit
cf892315e6
10 files changed
+888
-62
No files matched your search
+46
-1
@@ -59,13 +59,14 @@ concern - readable here, never shipped as something to parse.
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 7.1.0-beta.31 - 2026-09-26 - new_page/type_resolver comments no longer claim only entities declare a layout:
|
## 7.1.0-beta.32 - 2026-09-29 - dist upgrade --latest: one-command update from the release feed
|
||||||
|
|
||||||
**Author:** Torben Nehmer
|
**Author:** Torben Nehmer
|
||||||
|
|
||||||
<!-- wikitool:bumps -->
|
<!-- wikitool:bumps -->
|
||||||
**High impact**
|
**High impact**
|
||||||
- wikitool: one data record per command - `-h`, index and CONTRACT.md render from cli_contract (Gitea #121 Phase 1)
|
- wikitool: one data record per command - `-h`, index and CONTRACT.md render from cli_contract (Gitea #121 Phase 1)
|
||||||
|
- dist upgrade --latest: one-command update from the release feed
|
||||||
|
|
||||||
**Medium impact**
|
**Medium impact**
|
||||||
- CalDAV task-tracker provider (Nextcloud Tasks, iOS Reminders); review reports unknown-value findings instead of skipping them
|
- CalDAV task-tracker provider (Nextcloud Tasks, iOS Reminders); review reports unknown-value findings instead of skipping them
|
||||||
@@ -102,6 +103,50 @@ concern - readable here, never shipped as something to parse.
|
|||||||
- new_page/type_resolver comments no longer claim only entities declare a layout:
|
- new_page/type_resolver comments no longer claim only entities declare a layout:
|
||||||
<!-- /wikitool:bumps -->
|
<!-- /wikitool:bumps -->
|
||||||
|
|
||||||
|
### dist upgrade --latest: one-command update from the release feed (Gitea #161)
|
||||||
|
|
||||||
|
Updating a tarball instance took a manual detour: `version notes`, then fetching the `.tar.gz` and
|
||||||
|
its `.sha256` from the release page by hand, then `dist upgrade <tarball>`. `dist upgrade --latest`
|
||||||
|
now does the middle part. It asks the release feed (`update_url` from the stamp,
|
||||||
|
`$WIKITOOL_UPDATE_URL` and `$WIKITOOL_UPDATE_TOKEN` as before) which release is latest, downloads
|
||||||
|
the release's archive and checksum into a scratch directory, verifies the archive against the
|
||||||
|
checksum, and hands over to the existing upgrade path unchanged - classification,
|
||||||
|
`--keep-local`/`--take-release`, migration report and every refusal are the same code. `<source>`
|
||||||
|
becomes optional; exactly one of it and `--latest` must be given.
|
||||||
|
|
||||||
|
The order is what matters. The local preconditions run first, then the feed is asked, and the
|
||||||
|
version it reports is judged **before any download**: already installed is a success no-op, an
|
||||||
|
older version is a downgrade refusal, a `-beta.N` version needs `--pre`. Only then are the two
|
||||||
|
assets looked up - by exact name, `chemenu-stack-<version>.tar.gz` and its `.sha256`, in the
|
||||||
|
`assets` of the release object the version query already returned, using the feed's own
|
||||||
|
`browser_download_url` values; no URL is composed. A release missing either asset is refused
|
||||||
|
before the first download with the assets it does have and its release page named. The checksum
|
||||||
|
is mandatory on this path (a `<source>` archive without a sibling `.sha256` is still only a
|
||||||
|
WARN), and the archive's own `VERSION` must equal the feed's version, otherwise nothing is
|
||||||
|
applied. `--dry-run` downloads and verifies too - classification needs the tree - and removes
|
||||||
|
everything again; the scratch directory goes away on every exit.
|
||||||
|
|
||||||
|
`--expect <version>` (only with `--latest`) closes the gap between reading the notes and applying
|
||||||
|
the update: the feed only offers its *latest* release, so if a newer one appeared since
|
||||||
|
`version notes` was read, the run refuses before downloading anything and names both versions.
|
||||||
|
`instructions/upgrade-instance.md` passes the version from step 2 in the dry run and in the real
|
||||||
|
run. The comparison is on parsed versions, so a `v` prefix does not matter.
|
||||||
|
|
||||||
|
`$WIKITOOL_UPDATE_TOKEN` is sent to an asset download only when the asset URL has the feed's
|
||||||
|
scheme, host and port, and as an unredirected header, so a redirect cannot carry it elsewhere.
|
||||||
|
There is no https enforcement: the checksum comes from the same host as the archive and protects
|
||||||
|
against transfer errors, not against a compromised feed - `INSTALL.md` says so.
|
||||||
|
|
||||||
|
The asset names are a Python constant (`version.ARCHIVE_NAME`) that a test ties to
|
||||||
|
`.gitea/workflows/release.yml`, so renaming them in one place fails the suite instead of the next
|
||||||
|
upgrade. `dist upgrade` is now `network: yes` in its record and in the pinned set in `test_cli.py`,
|
||||||
|
the `version check` record's "never reached implicitly" note names the one explicit exception, the
|
||||||
|
`fetch_latest` docstring says "three commands", and `tools/CONTRACT.md` is regenerated. The tests
|
||||||
|
run against a local `http.server` that logs every request and its `Authorization` header, so the
|
||||||
|
token rule and the "no asset request before the checks pass" rule are asserted on the wire. The
|
||||||
|
`upstream merge` pointers in the `dist upgrade` record and failure message stay as they are; they
|
||||||
|
belong to the separate work on the clone path.
|
||||||
|
|
||||||
### new_page/type_resolver comments no longer claim only entities declare a layout:
|
### new_page/type_resolver comments no longer claim only entities declare a layout:
|
||||||
|
|
||||||
Two code comments - `new_page.py`'s module docstring and `TypeResolver.get_layout`'s - still said
|
Two code comments - `new_page.py`'s module docstring and `TypeResolver.get_layout`'s - still said
|
||||||
|
|||||||
+37
-12
@@ -152,12 +152,14 @@ tools/wikitool version # was läuft hier, und woher kommt es
|
|||||||
tools/wikitool version check # gibt es ein neueres Release?
|
tools/wikitool version check # gibt es ein neueres Release?
|
||||||
```
|
```
|
||||||
|
|
||||||
`version check` und `version notes` sind die einzigen Befehle, die ins Netz gehen, und beide
|
`version check`, `version notes` und `dist upgrade --latest` sind die einzigen Befehle, die
|
||||||
fragen denselben Release-Feed der Ursprungs-Instanz (`$WIKITOOL_UPDATE_URL` überschreibt; sonst
|
ins Netz gehen, und alle drei fragen denselben Release-Feed der Ursprungs-Instanz
|
||||||
der Wert aus dem Stamp). `version check` ist dafür da; `version notes` greift nur dann darauf
|
(`$WIKITOOL_UPDATE_URL` überschreibt; sonst der Wert aus dem Stamp). `version check` ist dafür da;
|
||||||
zurück, wenn die lokale `CHANGES.md` den Eintrag nicht hat - auf einer Instanz also immer, siehe
|
`version notes` greift nur dann darauf zurück, wenn die lokale `CHANGES.md` den Eintrag nicht
|
||||||
unten - und sagt vorher auf stderr, welche URL es fragt. Ein nicht erreichbarer Feed wird als
|
hat - auf einer Instanz also immer, siehe unten - und sagt vorher auf stderr, welche URL es
|
||||||
Fehler gemeldet - **nie** als „aktuell" und nie als „keine Notes".
|
fragt; `dist upgrade` fragt den Feed nur, wenn `--latest` dasteht, und lädt dann auch das Release
|
||||||
|
herunter (siehe „Eine Instanz aktualisieren"). Ein nicht erreichbarer Feed wird als Fehler
|
||||||
|
gemeldet - **nie** als „aktuell" und nie als „keine Notes".
|
||||||
|
|
||||||
**Was die Versionsnummer aussagt:** kompatibel ist, was in der *linkesten von Null
|
**Was die Versionsnummer aussagt:** kompatibel ist, was in der *linkesten von Null
|
||||||
verschiedenen Stelle* übereinstimmt. `0.1.3 → 0.1.4` ist ein sicheres Update, `0.1.3 → 0.2.0`
|
verschiedenen Stelle* übereinstimmt. `0.1.3 → 0.1.4` ist ein sicheres Update, `0.1.3 → 0.2.0`
|
||||||
@@ -203,9 +205,32 @@ Agent-Sitzung ausgeführt wird; eine zweite Fassung derselben Schrittfolge an di
|
|||||||
genau die Kopie, die irgendwann auseinanderläuft. Wer den Lauf selbst fahren will, liest dieselbe
|
genau die Kopie, die irgendwann auseinanderläuft. Wer den Lauf selbst fahren will, liest dieselbe
|
||||||
Datei.
|
Datei.
|
||||||
|
|
||||||
Was dieses Dokument beiträgt, ist die Entscheidung *davor* - welches Release, ob überhaupt, woher
|
Was dieses Dokument beiträgt, ist die Entscheidung *davor* - welches Release, ob überhaupt, und
|
||||||
der Tarball kommt (§ „Version und Updates" und Weg A oben) - und der eine Sonderfall, den die
|
wem die Instanz als Quelle vertraut - und zwei Sonderfälle, die die Instruktion nicht abdecken
|
||||||
Instruktion nicht abdecken kann, weil es sie dort noch nicht gibt:
|
kann, weil es sie dort noch nicht gibt.
|
||||||
|
|
||||||
|
**Das Release kommt mit einem Befehl.** `tools/wikitool dist upgrade --latest --expect <version>`
|
||||||
|
fragt den Release-Feed, lädt Tarball und `.sha256` in ein Arbeitsverzeichnis, prüft den Tarball
|
||||||
|
gegen die Summe und wendet ihn an; das Arbeitsverzeichnis verschwindet bei jedem Ausgang wieder,
|
||||||
|
auch bei `--dry-run`. `<version>` ist die, die `version notes` gedruckt hat: der Feed kennt nur
|
||||||
|
sein *neuestes* Release, und `--expect` verweigert den Lauf **vor** dem Download, wenn inzwischen
|
||||||
|
ein neueres erschienen ist, statt es ungelesen einzuspielen. Beides zusammen entscheidet vor dem
|
||||||
|
Download über „schon aktuell", Downgrade und Vor-Release (`-beta.N` braucht `--pre`); fehlt dem
|
||||||
|
Release der Tarball oder die Summe, bricht der Befehl vor dem ersten Download ab und nennt die
|
||||||
|
Release-Seite. Wer offline arbeitet oder einen Tarball vom Betreiber bekommen hat, gibt statt
|
||||||
|
`--latest` weiter die Datei an (`dist upgrade <tarball>`).
|
||||||
|
|
||||||
|
**Was die Prüfsumme leistet - und was nicht.** Die `.sha256` liegt beim selben Feed wie der
|
||||||
|
Tarball. Sie schützt vor einer beschädigten Übertragung, nicht vor einem Feed, der selbst
|
||||||
|
kompromittiert ist: die Echtheit eines Releases beruht auf dem Vertrauen in den Host, dessen
|
||||||
|
Feed die Instanz fragt (`update_url` im Stamp). Es gibt keinen https-Zwang; wer einen
|
||||||
|
`http://`-Feed konfiguriert, tut das bewusst. `$WIKITOOL_UPDATE_TOKEN` geht nur an Downloads
|
||||||
|
auf demselben Host wie der Feed.
|
||||||
|
|
||||||
|
**Beim ersten Sprung auf ein Release, das `--latest` kennt, gibt es die Option in der Instanz
|
||||||
|
noch nicht** - die Instruktion, die dort steht, gehört zum Release, das die Instanz verlässt.
|
||||||
|
Dann den Tarball einmal von Hand holen (Weg A oben), prüfen und `dist upgrade <tarball>` geben;
|
||||||
|
ab dem Release danach trägt die Instanz `--latest` selbst.
|
||||||
|
|
||||||
**Beim ersten Sprung auf `4.5.0` oder höher gibt es `dist upgrade` in der Instanz noch nicht** -
|
**Beim ersten Sprung auf `4.5.0` oder höher gibt es `dist upgrade` in der Instanz noch nicht** -
|
||||||
es kam erst mit `4.5.0`. Dann das Werkzeug aus dem entpackten *neuen* Tarball verwenden, gegen die
|
es kam erst mit `4.5.0`. Dann das Werkzeug aus dem entpackten *neuen* Tarball verwenden, gegen die
|
||||||
@@ -233,9 +258,9 @@ sind. Einer Instanz, die älter ist als `.wikitool-kb.json`, fehlt die Datei gan
|
|||||||
**Fallstricke.** Eine Instanz ohne lokale `.wikitool-release.json` (oder eine ohne `files`-Block,
|
**Fallstricke.** Eine Instanz ohne lokale `.wikitool-release.json` (oder eine ohne `files`-Block,
|
||||||
aus der Zeit vor `4.5.0`) hat für `dist upgrade` keine Basis, gegen die es eine lokale Änderung
|
aus der Zeit vor `4.5.0`) hat für `dist upgrade` keine Basis, gegen die es eine lokale Änderung
|
||||||
erkennen könnte, und verweigert den Tausch - dafür gibt es heute keine Reparatur.
|
erkennen könnte, und verweigert den Tausch - dafür gibt es heute keine Reparatur.
|
||||||
Der Befehl lädt selbst nichts herunter: `<tarball-oder-verzeichnis>` muss vorher aus Weg A
|
Mit `<tarball-oder-verzeichnis>` lädt der Befehl selbst nichts herunter; die Datei muss vorher
|
||||||
geholt werden, und ein Tarball muss genau ein Top-Level-Verzeichnis enthalten - die Form, in der
|
aus Weg A geholt werden. Nur `--latest` lädt, und ein Tarball muss in beiden Fällen genau ein
|
||||||
`.gitea/workflows/release.yml` es baut.
|
Top-Level-Verzeichnis enthalten - die Form, in der `.gitea/workflows/release.yml` es baut.
|
||||||
|
|
||||||
Vor `4.5.0` stand hier ein rein manueller Ablauf (Maschinerie von Hand kopieren, `kb/CONTRACT.md`
|
Vor `4.5.0` stand hier ein rein manueller Ablauf (Maschinerie von Hand kopieren, `kb/CONTRACT.md`
|
||||||
eingeschlossen, sha256-Vergleich von Hand). `dist upgrade` ersetzt genau diesen Teil; wer ihn
|
eingeschlossen, sha256-Vergleich von Hand). `dist upgrade` ersetzt genau diesen Teil; wer ihn
|
||||||
|
|||||||
@@ -81,14 +81,25 @@ fresh clone ([bootstrap.md](bootstrap.md)), and not for the clone-with-upstream
|
|||||||
step 12's, run against the migration documents this instance already has. An `offered` upgrade
|
step 12's, run against the migration documents this instance already has. An `offered` upgrade
|
||||||
listed separately blocks nothing and is decided later, in step 12.
|
listed separately blocks nothing and is decided later, in step 12.
|
||||||
|
|
||||||
4. **Fetch the tarball and verify it.** `dist upgrade` downloads nothing; the file has to be
|
4. **Nothing to fetch by hand.** `dist upgrade --latest` asks the release feed for the latest
|
||||||
there already. Take the `.tar.gz` and its `.sha256` from the release page found in step 2 and
|
release, downloads its `.tar.gz` and `.sha256` into a scratch directory, checks the archive
|
||||||
check them before unpacking. A tarball must unpack to exactly one top-level directory.
|
against the checksum and removes both again - all inside the calls of steps 5 to 7. Note the
|
||||||
|
version step 2's `version notes` printed: steps 5 to 7 pass it as `--expect`, so a release that
|
||||||
|
appeared in the meantime is refused before anything is downloaded, rather than applied unread.
|
||||||
|
|
||||||
|
The offline alternative is the tarball path: with the feed unreachable, or an archive the
|
||||||
|
operator supplies, take the `.tar.gz` and its `.sha256` from the release page named in step 2,
|
||||||
|
check the archive against the checksum before unpacking, and pass the file as `<tarball>`
|
||||||
|
where the steps below say `--latest --expect <version>`. A tarball must unpack to exactly one
|
||||||
|
top-level directory. The checksum comes from the same host as the archive, so it catches a
|
||||||
|
damaged transfer, not a compromised host - who is trusted to publish releases is the
|
||||||
|
operator's decision, made before this file starts ([INSTALL.md](../INSTALL.md) § "Version und
|
||||||
|
Updates").
|
||||||
|
|
||||||
5. **Dry-run the swap and read all four counts:**
|
5. **Dry-run the swap and read all four counts:**
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
tools/wikitool dist upgrade <tarball> --dry-run
|
tools/wikitool dist upgrade --latest --expect <version from step 2> --dry-run
|
||||||
```
|
```
|
||||||
|
|
||||||
`unchanged` / `new` / `locally changed` / `removed from the release`. `unchanged` needs no
|
`unchanged` / `new` / `locally changed` / `removed from the release`. `unchanged` needs no
|
||||||
@@ -115,7 +126,7 @@ fresh clone ([bootstrap.md](bootstrap.md)), and not for the clone-with-upstream
|
|||||||
file reset and another kept. Preview it before it writes:
|
file reset and another kept. Preview it before it writes:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
tools/wikitool dist upgrade <tarball> --dry-run --take-release <path> [--take-release <path>]
|
tools/wikitool dist upgrade --latest --expect <version from step 2> --dry-run --take-release <path> [--take-release <path>]
|
||||||
```
|
```
|
||||||
|
|
||||||
The preview marks every named path as one it would overwrite from the release, and a path that
|
The preview marks every named path as one it would overwrite from the release, and a path that
|
||||||
@@ -134,7 +145,7 @@ fresh clone ([bootstrap.md](bootstrap.md)), and not for the clone-with-upstream
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
git rev-parse --short HEAD # the pre-swap commit; keep it
|
git rev-parse --short HEAD # the pre-swap commit; keep it
|
||||||
tools/wikitool dist upgrade <tarball> [--take-release <path>] [--keep-local]
|
tools/wikitool dist upgrade --latest --expect <version from step 2> [--take-release <path>] [--keep-local]
|
||||||
```
|
```
|
||||||
|
|
||||||
It writes, and commits nothing.
|
It writes, and commits nothing.
|
||||||
|
|||||||
+22
-6
@@ -2418,7 +2418,7 @@ Apply a stack update `dist export` produced - the write half of `version check`.
|
|||||||
|
|
||||||
**SYNOPSIS**
|
**SYNOPSIS**
|
||||||
|
|
||||||
- `wikitool dist upgrade <source> [--dry-run] [--keep-local] [--take-release <path>]... [--prune] [--pre]`
|
- `wikitool dist upgrade (<source> | --latest [--expect <version>]) [--dry-run] [--keep-local] [--take-release <path>]... [--prune] [--pre]`
|
||||||
|
|
||||||
**PROPERTIES**
|
**PROPERTIES**
|
||||||
|
|
||||||
@@ -2426,23 +2426,30 @@ Apply a stack update `dist export` produced - the write half of `version check`.
|
|||||||
- idempotent: no
|
- idempotent: no
|
||||||
- atomic: Yes for every refusal - nothing is written. Once writing starts it is a plain sequential file copy with no partial-state cleanup: an interruption mid-copy (killed process, disk full) can leave the tree part-old, part-new
|
- atomic: Yes for every refusal - nothing is written. Once writing starts it is a plain sequential file copy with no partial-state cleanup: an interruption mid-copy (killed process, disk full) can leave the tree part-old, part-new
|
||||||
- budget: counted
|
- budget: counted
|
||||||
- network: no
|
- network: yes
|
||||||
|
|
||||||
**EXAMPLES**
|
**EXAMPLES**
|
||||||
|
|
||||||
- `tools/wikitool dist upgrade ../chemenu-7.1.0.tar.gz --dry-run`
|
- `tools/wikitool dist upgrade ../chemenu-7.1.0.tar.gz --dry-run`
|
||||||
- `tools/wikitool dist upgrade ../chemenu-7.1.0.tar.gz`
|
- `tools/wikitool dist upgrade ../chemenu-7.1.0.tar.gz`
|
||||||
- `tools/wikitool dist upgrade ../chemenu-7.1.0.tar.gz --take-release tools/README.md`
|
- `tools/wikitool dist upgrade ../chemenu-7.1.0.tar.gz --take-release tools/README.md`
|
||||||
|
- `tools/wikitool dist upgrade --latest --expect 8.0.0 --dry-run`
|
||||||
|
|
||||||
**EXIT STATUS**
|
**EXIT STATUS**
|
||||||
|
|
||||||
- 0 success
|
- 0 success
|
||||||
- 0 The source's version equals the installed one - a no-op success
|
- 0 The source's version equals the installed one - a no-op success
|
||||||
|
- 1 Both `<source>` and `--latest`, or neither; or `--expect` without `--latest`
|
||||||
- 1 Local `VERSION` missing, or no local `.wikitool-release.json` with a `files` block
|
- 1 Local `VERSION` missing, or no local `.wikitool-release.json` with a `files` block
|
||||||
- 1 `.wikitool-kb.json` is missing
|
- 1 `.wikitool-kb.json` is missing
|
||||||
- 1 A migration is already outstanding against the *installed* machinery
|
- 1 A migration is already outstanding against the *installed* machinery
|
||||||
- 1 The working tree is dirty
|
- 1 The working tree is dirty
|
||||||
- 1 `<source>` does not exist, fails its `.sha256`, or does not unpack to exactly one top-level directory
|
- 1 `<source>` does not exist, fails its `.sha256`, or does not unpack to exactly one top-level directory
|
||||||
|
- 1 `--latest`: the release feed cannot be reached, or answers with something that is not a release
|
||||||
|
- 1 `--latest --expect`: the feed's latest release is another version than the one expected; nothing was downloaded
|
||||||
|
- 1 `--latest`: the release publishes no archive or no `.sha256` under the expected name; nothing was downloaded
|
||||||
|
- 1 `--latest`: an asset download fails, or the archive fails its `.sha256`
|
||||||
|
- 1 `--latest`: the downloaded archive's `VERSION` is not the version the feed announced
|
||||||
- 1 The source carries no `VERSION`, `.wikitool-release.json` or `files` block
|
- 1 The source carries no `VERSION`, `.wikitool-release.json` or `files` block
|
||||||
- 1 The source's version is older than the installed one, or a pre-release without `--pre`
|
- 1 The source's version is older than the installed one, or a pre-release without `--pre`
|
||||||
- 1 A `--take-release` path this run does not classify as locally changed - the one refusal a `--dry-run` also raises
|
- 1 A `--take-release` path this run does not classify as locally changed - the one refusal a `--dry-run` also raises
|
||||||
@@ -2450,11 +2457,17 @@ Apply a stack update `dist export` produced - the write half of `version check`.
|
|||||||
|
|
||||||
**ON FAILURE**
|
**ON FAILURE**
|
||||||
|
|
||||||
|
- Both `<source>` and `--latest`, or neither; or `--expect` without `--latest` -> Not transient - name exactly one source, and pass `--expect` only with `--latest`
|
||||||
- Local `VERSION` missing, or no local `.wikitool-release.json` with a `files` block -> Not transient - fix the named precondition and retry. A checkout with shared git history takes stack updates with `wikitool upstream merge` instead
|
- Local `VERSION` missing, or no local `.wikitool-release.json` with a `files` block -> Not transient - fix the named precondition and retry. A checkout with shared git history takes stack updates with `wikitool upstream merge` instead
|
||||||
- `.wikitool-kb.json` is missing -> Run `wikitool migrate baseline <version>`, then retry
|
- `.wikitool-kb.json` is missing -> Run `wikitool migrate baseline <version>`, then retry
|
||||||
- A migration is already outstanding against the *installed* machinery -> Finish it first - `wikitool migrate status` names it - then retry
|
- A migration is already outstanding against the *installed* machinery -> Finish it first - `wikitool migrate status` names it - then retry
|
||||||
- The working tree is dirty -> Commit or stash first, then retry
|
- The working tree is dirty -> Commit or stash first, then retry
|
||||||
- `<source>` does not exist, fails its `.sha256`, or does not unpack to exactly one top-level directory -> Fix the path or re-download the release archive, then retry
|
- `<source>` does not exist, fails its `.sha256`, or does not unpack to exactly one top-level directory -> Fix the path or re-download the release archive, then retry
|
||||||
|
- `--latest`: the release feed cannot be reached, or answers with something that is not a release -> Transient - retry once; then report the URL from the message, or take the release page's archive by hand and pass it as `<source>`
|
||||||
|
- `--latest --expect`: the feed's latest release is another version than the one expected; nothing was downloaded -> Not transient - read `wikitool version notes` for the version the message names, then either expect that one or stop
|
||||||
|
- `--latest`: the release publishes no archive or no `.sha256` under the expected name; nothing was downloaded -> Not transient - the message lists the assets present and the release page; report it there rather than upgrading without the checksum
|
||||||
|
- `--latest`: an asset download fails, or the archive fails its `.sha256` -> Retry once; if it fails again, report the exact message - do not fall back to an unchecked archive
|
||||||
|
- `--latest`: the downloaded archive's `VERSION` is not the version the feed announced -> Not transient - an inconsistent release; report it against the release page
|
||||||
- The source carries no `VERSION`, `.wikitool-release.json` or `files` block -> Point `<source>` at a distribution export, then retry
|
- The source carries no `VERSION`, `.wikitool-release.json` or `files` block -> Point `<source>` at a distribution export, then retry
|
||||||
- The source's version is older than the installed one, or a pre-release without `--pre` -> Not transient - choose another source, or pass `--pre` for a pre-release
|
- The source's version is older than the installed one, or a pre-release without `--pre` -> Not transient - choose another source, or pass `--pre` for a pre-release
|
||||||
- A `--take-release` path this run does not classify as locally changed - the one refusal a `--dry-run` also raises -> Correct it against the locally-changed list the refusal prints, then retry
|
- A `--take-release` path this run does not classify as locally changed - the one refusal a `--dry-run` also raises -> Correct it against the locally-changed list the refusal prints, then retry
|
||||||
@@ -2466,7 +2479,9 @@ Apply a stack update `dist export` produced - the write half of `version check`.
|
|||||||
|
|
||||||
**NOTES**
|
**NOTES**
|
||||||
|
|
||||||
- Never downloads anything: `<source>` is an already-fetched export directory or `.tar.gz` release archive. An archive is verified against a sibling `.sha256` if one is present (a missing one is a WARN, not a block) and must unpack to exactly one top-level directory - the shape `.gitea/workflows/release.yml` packs.
|
- Exactly one of `<source>` and `--latest` names the release. `<source>` is an already-fetched export directory or `.tar.gz` release archive and touches no network: the archive is verified against a sibling `.sha256` if one is present (a missing one is a WARN, not a block) and must unpack to exactly one top-level directory - the shape `.gitea/workflows/release.yml` packs.
|
||||||
|
- `--latest` asks the release feed (`update_url` of the local stamp, overridable with `$WIKITOOL_UPDATE_URL`; `$WIKITOOL_UPDATE_TOKEN` is sent along) which release is latest, checks that version - `--expect`, a downgrade, a pre-release without `--pre`, already installed - before any download, then downloads the archive and its `.sha256` into a scratch directory removed on every exit. The checksum is mandatory here (a release without one, or an archive that fails it, is an error, not a WARN) and the archive's own `VERSION` must equal the feed's version. The asset URLs are the feed's own `browser_download_url` values, and the token reaches an asset download only if it is on the feed's origin. The checksum protects against transfer errors, not against a feed that is itself compromised - authenticity is the trust in the feed's host.
|
||||||
|
- `--expect <version>` (only with `--latest`) pins the release the feed may announce: a different latest version is refused before anything is downloaded. Pass the version `wikitool version notes` was read for, in the dry run and in the real run alike.
|
||||||
- The write set is exactly the *new* `.wikitool-release.json`'s `files` block, minus what an export re-seeds from a blank template every time (`kb/log.md`, `raw/.gitkeep`) or seeds once and the instance owns from then on (`.wikitool-kb.json`, `CHANGES.md`), plus the stamp itself, always rewritten.
|
- The write set is exactly the *new* `.wikitool-release.json`'s `files` block, minus what an export re-seeds from a blank template every time (`kb/log.md`, `raw/.gitkeep`) or seeds once and the instance owns from then on (`.wikitool-kb.json`, `CHANGES.md`), plus the stamp itself, always rewritten.
|
||||||
- Every candidate path is classified against the *local* `.wikitool-release.json`'s recorded digest: unchanged is overwritten silently, absent from the old stamp is created, and locally modified or locally deleted is **never** silently overwritten.
|
- Every candidate path is classified against the *local* `.wikitool-release.json`'s recorded digest: unchanged is overwritten silently, absent from the old stamp is created, and locally modified or locally deleted is **never** silently overwritten.
|
||||||
- Locally changed files abort the run with the full list; the abort text names the three answers with the command line filled in, and none of them is the default.
|
- Locally changed files abort the run with the full list; the abort text names the three answers with the command line filled in, and none of them is the default.
|
||||||
@@ -2478,13 +2493,14 @@ Apply a stack update `dist export` produced - the write half of `version check`.
|
|||||||
- Reports, but does not block on, a crossed compatibility boundary.
|
- Reports, but does not block on, a crossed compatibility boundary.
|
||||||
- The local preconditions - `VERSION`, the release stamp, `.wikitool-kb.json`, no outstanding migration, a clean working tree - are checked before the source is read. Not being a git repository at all is a WARN, not a refusal.
|
- The local preconditions - `VERSION`, the release stamp, `.wikitool-kb.json`, no outstanding migration, a clean working tree - are checked before the source is read. Not being a git repository at all is a WARN, not a refusal.
|
||||||
- Never touches git - no commit, no push.
|
- Never touches git - no commit, no push.
|
||||||
- `--dry-run` classifies and reports without writing; a pre-release (`-beta.N`) source needs `--pre`.
|
- `--dry-run` classifies and reports without writing; a pre-release (`-beta.N`) source needs `--pre`. With `--latest` it still downloads and verifies the archive - that is the only way to classify - and removes it again.
|
||||||
- An interrupted write is not resumed automatically: compare the tree against the printed classification and finish or revert by hand.
|
- An interrupted write is not resumed automatically: compare the tree against the printed classification and finish or revert by hand.
|
||||||
- The closing report names `instructions/upgrade-instance.md`, which carries the order for everything after the swap and resumes at `instructions sync`.
|
- The closing report names `instructions/upgrade-instance.md`, which carries the order for everything after the swap and resumes at `instructions sync`.
|
||||||
|
|
||||||
**SEE ALSO**
|
**SEE ALSO**
|
||||||
|
|
||||||
- `wikitool version check` - finds out whether an update exists
|
- `wikitool version check` - finds out whether an update exists
|
||||||
|
- `wikitool version notes` - the notes of the release `--expect` should name
|
||||||
- `instructions/upgrade-instance.md` - the order after the swap
|
- `instructions/upgrade-instance.md` - the order after the swap
|
||||||
- `INSTALL.md` § "Version und Updates" - which release, whether to take it, where the tarball comes from
|
- `INSTALL.md` § "Version und Updates" - which release, whether to take it, where the tarball comes from
|
||||||
- `wikitool upstream merge` - the update path for a checkout with shared git history
|
- `wikitool upstream merge` - the update path for a checkout with shared git history
|
||||||
@@ -2571,8 +2587,8 @@ Ask the origin's release feed whether a newer stack exists.
|
|||||||
**NOTES**
|
**NOTES**
|
||||||
|
|
||||||
- Asks the release feed for its latest release and compares it with `VERSION`: `state` is `current`, `update`, `migration` (the step crosses a compatibility boundary) or `ahead`.
|
- Asks the release feed for its latest release and compares it with `VERSION`: `state` is `current`, `update`, `migration` (the step crosses a compatibility boundary) or `ahead`.
|
||||||
- The only command whose whole job is the network call - `version notes` reaches the same feed too, but only as a fallback on a distributed instance.
|
- The only command whose whole job is the network call - `version notes` reaches the same feed too, but only as a fallback on a distributed instance, and `dist upgrade --latest` asks it which release to download.
|
||||||
- Never reached implicitly from another command, needs no key, and times out after `--timeout` seconds (default 10).
|
- Never reached implicitly from another command (`dist upgrade` asks only when passed `--latest`), needs no key, and times out after `--timeout` seconds (default 10).
|
||||||
- The feed is `--url`, else `$WIKITOOL_UPDATE_URL`, else the release stamp's, else the built-in origin. `$WIKITOOL_UPDATE_TOKEN` is only needed if that feed is not readable anonymously.
|
- The feed is `--url`, else `$WIKITOOL_UPDATE_URL`, else the release stamp's, else the built-in origin. `$WIKITOOL_UPDATE_TOKEN` is only needed if that feed is not readable anonymously.
|
||||||
- For `update` or `migration` it prints that applying the release is a separate, manual step (`INSTALL.md` § "Eine Instanz aktualisieren").
|
- For `update` or `migration` it prints that applying the release is a separate, manual step (`INSTALL.md` § "Eine Instanz aktualisieren").
|
||||||
- Read-only; exempt from the Iteration Budget Gate.
|
- Read-only; exempt from the Iteration Budget Gate.
|
||||||
|
|||||||
@@ -736,8 +736,8 @@ def run_export(target: Path, dry_run: bool = False, origin: Optional[Origin] = N
|
|||||||
# modified/deleted file is never silently overwritten: the run aborts unless
|
# modified/deleted file is never silently overwritten: the run aborts unless
|
||||||
# `--keep-local` keeps it or `--take-release <path>` names it, which is the
|
# `--keep-local` keeps it or `--take-release <path>` names it, which is the
|
||||||
# difference between a file the instance means to carry and one that drifted.
|
# difference between a file the instance means to carry and one that drifted.
|
||||||
# This never calls a release feed; the caller supplies an already-downloaded
|
# The classification never touches the network: the tree or archive it works
|
||||||
# tree or archive.
|
# on is either supplied by the caller or fetched by `--latest` beforehand.
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
@@ -867,6 +867,87 @@ def _resolved_source(source: Path):
|
|||||||
yield _extract_single_top_level_dir(source, Path(tmp))
|
yield _extract_single_top_level_dir(source, Path(tmp))
|
||||||
|
|
||||||
|
|
||||||
|
def _version_gate(
|
||||||
|
new_version: "version_mod.Version",
|
||||||
|
local_version: "version_mod.Version",
|
||||||
|
allow_pre: bool,
|
||||||
|
origin: str,
|
||||||
|
) -> bool:
|
||||||
|
"""The three version refusals shared by a source tree and the feed: a
|
||||||
|
candidate without `--pre`, a downgrade, an equal version. `True` means go
|
||||||
|
on; `False` means the run ended successfully as a no-op. `origin` names
|
||||||
|
what reported the version, for the message."""
|
||||||
|
if new_version.is_prerelease and not allow_pre:
|
||||||
|
fail(
|
||||||
|
f"{origin} reports {new_version}, a running candidate (-beta.N). "
|
||||||
|
"`.gitea/workflows/release.yml` never publishes one, so it can only come from a "
|
||||||
|
"dev checkout or a hand-made release - pass --pre if that is deliberate."
|
||||||
|
)
|
||||||
|
if new_version < local_version:
|
||||||
|
fail(f"{origin} reports {new_version}, older than the installed {local_version} - refusing a downgrade.")
|
||||||
|
if new_version == local_version:
|
||||||
|
success(f"Already at {local_version}. Nothing to do.")
|
||||||
|
return False
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
def _query_feed(
|
||||||
|
stamp: dict,
|
||||||
|
local_version: "version_mod.Version",
|
||||||
|
allow_pre: bool,
|
||||||
|
expect: Optional["version_mod.Version"],
|
||||||
|
) -> Optional[tuple["version_mod.LatestRelease", str, Optional[str]]]:
|
||||||
|
"""Ask the release feed which release is latest and decide, before any
|
||||||
|
download, whether there is anything to fetch. Returns the release with the
|
||||||
|
feed URL and token, or `None` when the installed version is already it."""
|
||||||
|
url = version_mod.update_url(stamp)
|
||||||
|
token = os.environ.get(version_mod.UPDATE_TOKEN_ENV, "").strip() or None
|
||||||
|
console.print(f"Asking the release feed {url}", highlight=False)
|
||||||
|
try:
|
||||||
|
release = version_mod.fetch_latest_assets(url, token)
|
||||||
|
except version_mod.VersionError as exc:
|
||||||
|
fail(str(exc))
|
||||||
|
if expect is not None and release.version != expect:
|
||||||
|
fail(
|
||||||
|
f"The feed's latest release is {release.version}, not the expected {expect} - "
|
||||||
|
"nothing was downloaded. The feed moved on since the notes were read, or "
|
||||||
|
f"the wrong version was named: read `wikitool version notes` for {release.version} "
|
||||||
|
"before deciding to go there instead."
|
||||||
|
)
|
||||||
|
if not _version_gate(release.version, local_version, allow_pre, "the release feed"):
|
||||||
|
return None
|
||||||
|
return release, url, token
|
||||||
|
|
||||||
|
|
||||||
|
@contextmanager
|
||||||
|
def _downloaded_source(release: "version_mod.LatestRelease", feed_url: str, token: Optional[str]):
|
||||||
|
"""Yield the directory of the release the feed announced: download its
|
||||||
|
archive and checksum into a scratch directory, hold the archive to the
|
||||||
|
checksum (a missing checksum is an error, never a WARN), and unpack it.
|
||||||
|
A release without both assets is refused before the first download. The
|
||||||
|
scratch directory is removed on every exit, `--dry-run` included."""
|
||||||
|
archive_url, sum_url = _asset_urls(release)
|
||||||
|
with tempfile.TemporaryDirectory(prefix="wikitool-upgrade-") as tmp:
|
||||||
|
archive = Path(tmp) / release.archive_name
|
||||||
|
checksum = archive.with_name(archive.name + version_mod.CHECKSUM_SUFFIX)
|
||||||
|
for url, dest in ((archive_url, archive), (sum_url, checksum)):
|
||||||
|
console.print(f"Downloading {url}", highlight=False)
|
||||||
|
try:
|
||||||
|
version_mod.download_asset(url, dest, version_mod.token_for_asset(feed_url, url, token))
|
||||||
|
except version_mod.VersionError as exc:
|
||||||
|
fail(str(exc))
|
||||||
|
_verify_sha256_sidecar(archive)
|
||||||
|
yield _extract_single_top_level_dir(archive, Path(tmp) / "unpacked")
|
||||||
|
|
||||||
|
|
||||||
|
def _asset_urls(release: "version_mod.LatestRelease") -> tuple[str, str]:
|
||||||
|
try:
|
||||||
|
return release.asset_urls()
|
||||||
|
except version_mod.VersionError as exc:
|
||||||
|
fail(str(exc))
|
||||||
|
raise # unreachable: fail() raises typer.Exit
|
||||||
|
|
||||||
|
|
||||||
def _git_working_tree_status() -> Optional[str]:
|
def _git_working_tree_status() -> Optional[str]:
|
||||||
"""`git status --porcelain` for `config.ROOT`, or None if it is not a git
|
"""`git status --porcelain` for `config.ROOT`, or None if it is not a git
|
||||||
repository at all - which is a valid, if unprotected, state for a tarball
|
repository at all - which is a valid, if unprotected, state for a tarball
|
||||||
@@ -907,7 +988,7 @@ def _resolve_take_release(
|
|||||||
|
|
||||||
|
|
||||||
def _refusal_for_blocked(
|
def _refusal_for_blocked(
|
||||||
source: Path, undecided: list[str], classification: FileClassification
|
target: str, undecided: list[str], classification: FileClassification
|
||||||
) -> str:
|
) -> str:
|
||||||
"""The abort text for blocked paths no flag has answered for.
|
"""The abort text for blocked paths no flag has answered for.
|
||||||
|
|
||||||
@@ -927,9 +1008,9 @@ def _refusal_for_blocked(
|
|||||||
f"{len(undecided)} locally changed file(s) (listed above) would be silently "
|
f"{len(undecided)} locally changed file(s) (listed above) would be silently "
|
||||||
f"overwritten. Nothing was written, and none of these three is the default:\n"
|
f"overwritten. Nothing was written, and none of these three is the default:\n"
|
||||||
f" - take the release's version and discard the local change:\n"
|
f" - take the release's version and discard the local change:\n"
|
||||||
f" dist upgrade {rel_path(source)} --take-release {paths}\n"
|
f" dist upgrade {target} --take-release {paths}\n"
|
||||||
f" - keep every local change and upgrade around them ({kept_again}):\n"
|
f" - keep every local change and upgrade around them ({kept_again}):\n"
|
||||||
f" dist upgrade {rel_path(source)} --keep-local\n"
|
f" dist upgrade {target} --keep-local\n"
|
||||||
f" - reconcile them by hand first, then re-run."
|
f" - reconcile them by hand first, then re-run."
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -984,8 +1065,8 @@ def _report_plan(
|
|||||||
path="dist upgrade",
|
path="dist upgrade",
|
||||||
summary="Apply a stack update `dist export` produced - the write half of `version check`.",
|
summary="Apply a stack update `dist export` produced - the write half of `version check`.",
|
||||||
synopsis=(cli_contract.Variant(
|
synopsis=(cli_contract.Variant(
|
||||||
usage="dist upgrade <source> [--dry-run] [--keep-local] [--take-release <path>]... "
|
usage="dist upgrade (<source> | --latest [--expect <version>]) [--dry-run] "
|
||||||
"[--prune] [--pre]",
|
"[--keep-local] [--take-release <path>]... [--prune] [--pre]",
|
||||||
),),
|
),),
|
||||||
properties=cli_contract.Properties(
|
properties=cli_contract.Properties(
|
||||||
effect=cli_contract.Effect.WRITE,
|
effect=cli_contract.Effect.WRITE,
|
||||||
@@ -994,12 +1075,28 @@ def _report_plan(
|
|||||||
"sequential file copy with no partial-state cleanup: an interruption mid-copy "
|
"sequential file copy with no partial-state cleanup: an interruption mid-copy "
|
||||||
"(killed process, disk full) can leave the tree part-old, part-new",
|
"(killed process, disk full) can leave the tree part-old, part-new",
|
||||||
budget=cli_contract.Budget.COUNTED,
|
budget=cli_contract.Budget.COUNTED,
|
||||||
|
network=cli_contract.Network.YES,
|
||||||
),
|
),
|
||||||
notes=(
|
notes=(
|
||||||
"Never downloads anything: `<source>` is an already-fetched export directory or "
|
"Exactly one of `<source>` and `--latest` names the release. `<source>` is an "
|
||||||
"`.tar.gz` release archive. An archive is verified against a sibling `.sha256` if one "
|
"already-fetched export directory or `.tar.gz` release archive and touches no network: "
|
||||||
"is present (a missing one is a WARN, not a block) and must unpack to exactly one "
|
"the archive is verified against a sibling `.sha256` if one is present (a missing one "
|
||||||
"top-level directory - the shape `.gitea/workflows/release.yml` packs.",
|
"is a WARN, not a block) and must unpack to exactly one top-level directory - the "
|
||||||
|
"shape `.gitea/workflows/release.yml` packs.",
|
||||||
|
"`--latest` asks the release feed (`update_url` of the local stamp, overridable with "
|
||||||
|
"`$WIKITOOL_UPDATE_URL`; `$WIKITOOL_UPDATE_TOKEN` is sent along) which release is "
|
||||||
|
"latest, checks that version - `--expect`, a downgrade, a pre-release without `--pre`, "
|
||||||
|
"already installed - before any download, then downloads the archive and its "
|
||||||
|
"`.sha256` into a scratch directory removed on every exit. The checksum is mandatory "
|
||||||
|
"here (a release without one, or an archive that fails it, is an error, not a WARN) "
|
||||||
|
"and the archive's own `VERSION` must equal the feed's version. The asset URLs are the "
|
||||||
|
"feed's own `browser_download_url` values, and the token reaches an asset download only "
|
||||||
|
"if it is on the feed's origin. The checksum protects against transfer errors, not "
|
||||||
|
"against a feed that is itself compromised - authenticity is the trust in the feed's "
|
||||||
|
"host.",
|
||||||
|
"`--expect <version>` (only with `--latest`) pins the release the feed may announce: a "
|
||||||
|
"different latest version is refused before anything is downloaded. Pass the version "
|
||||||
|
"`wikitool version notes` was read for, in the dry run and in the real run alike.",
|
||||||
"The write set is exactly the *new* `.wikitool-release.json`'s `files` block, minus "
|
"The write set is exactly the *new* `.wikitool-release.json`'s `files` block, minus "
|
||||||
"what an export re-seeds from a blank template every time (`kb/log.md`, "
|
"what an export re-seeds from a blank template every time (`kb/log.md`, "
|
||||||
"`raw/.gitkeep`) or seeds once and the instance owns from then on "
|
"`raw/.gitkeep`) or seeds once and the instance owns from then on "
|
||||||
@@ -1029,7 +1126,8 @@ def _report_plan(
|
|||||||
"Not being a git repository at all is a WARN, not a refusal.",
|
"Not being a git repository at all is a WARN, not a refusal.",
|
||||||
"Never touches git - no commit, no push.",
|
"Never touches git - no commit, no push.",
|
||||||
"`--dry-run` classifies and reports without writing; a pre-release (`-beta.N`) source "
|
"`--dry-run` classifies and reports without writing; a pre-release (`-beta.N`) source "
|
||||||
"needs `--pre`.",
|
"needs `--pre`. With `--latest` it still downloads and verifies the archive - that is "
|
||||||
|
"the only way to classify - and removes it again.",
|
||||||
"An interrupted write is not resumed automatically: compare the tree against the "
|
"An interrupted write is not resumed automatically: compare the tree against the "
|
||||||
"printed classification and finish or revert by hand.",
|
"printed classification and finish or revert by hand.",
|
||||||
"The closing report names `instructions/upgrade-instance.md`, which carries the order "
|
"The closing report names `instructions/upgrade-instance.md`, which carries the order "
|
||||||
@@ -1041,6 +1139,11 @@ def _report_plan(
|
|||||||
reaction="",
|
reaction="",
|
||||||
code=0,
|
code=0,
|
||||||
),
|
),
|
||||||
|
cli_contract.Failure(
|
||||||
|
cause="Both `<source>` and `--latest`, or neither; or `--expect` without `--latest`",
|
||||||
|
reaction="Not transient - name exactly one source, and pass `--expect` only with "
|
||||||
|
"`--latest`",
|
||||||
|
),
|
||||||
cli_contract.Failure(
|
cli_contract.Failure(
|
||||||
cause="Local `VERSION` missing, or no local `.wikitool-release.json` with a `files` "
|
cause="Local `VERSION` missing, or no local `.wikitool-release.json` with a `files` "
|
||||||
"block",
|
"block",
|
||||||
@@ -1064,6 +1167,34 @@ def _report_plan(
|
|||||||
"one top-level directory",
|
"one top-level directory",
|
||||||
reaction="Fix the path or re-download the release archive, then retry",
|
reaction="Fix the path or re-download the release archive, then retry",
|
||||||
),
|
),
|
||||||
|
cli_contract.Failure(
|
||||||
|
cause="`--latest`: the release feed cannot be reached, or answers with something "
|
||||||
|
"that is not a release",
|
||||||
|
reaction="Transient - retry once; then report the URL from the message, or take the "
|
||||||
|
"release page's archive by hand and pass it as `<source>`",
|
||||||
|
),
|
||||||
|
cli_contract.Failure(
|
||||||
|
cause="`--latest --expect`: the feed's latest release is another version than the "
|
||||||
|
"one expected; nothing was downloaded",
|
||||||
|
reaction="Not transient - read `wikitool version notes` for the version the message "
|
||||||
|
"names, then either expect that one or stop",
|
||||||
|
),
|
||||||
|
cli_contract.Failure(
|
||||||
|
cause="`--latest`: the release publishes no archive or no `.sha256` under the "
|
||||||
|
"expected name; nothing was downloaded",
|
||||||
|
reaction="Not transient - the message lists the assets present and the release page; "
|
||||||
|
"report it there rather than upgrading without the checksum",
|
||||||
|
),
|
||||||
|
cli_contract.Failure(
|
||||||
|
cause="`--latest`: an asset download fails, or the archive fails its `.sha256`",
|
||||||
|
reaction="Retry once; if it fails again, report the exact message - do not fall back "
|
||||||
|
"to an unchecked archive",
|
||||||
|
),
|
||||||
|
cli_contract.Failure(
|
||||||
|
cause="`--latest`: the downloaded archive's `VERSION` is not the version the feed "
|
||||||
|
"announced",
|
||||||
|
reaction="Not transient - an inconsistent release; report it against the release page",
|
||||||
|
),
|
||||||
cli_contract.Failure(
|
cli_contract.Failure(
|
||||||
cause="The source carries no `VERSION`, `.wikitool-release.json` or `files` block",
|
cause="The source carries no `VERSION`, `.wikitool-release.json` or `files` block",
|
||||||
reaction="Point `<source>` at a distribution export, then retry",
|
reaction="Point `<source>` at a distribution export, then retry",
|
||||||
@@ -1091,12 +1222,14 @@ def _report_plan(
|
|||||||
"tools/wikitool dist upgrade ../chemenu-7.1.0.tar.gz --dry-run",
|
"tools/wikitool dist upgrade ../chemenu-7.1.0.tar.gz --dry-run",
|
||||||
"tools/wikitool dist upgrade ../chemenu-7.1.0.tar.gz",
|
"tools/wikitool dist upgrade ../chemenu-7.1.0.tar.gz",
|
||||||
"tools/wikitool dist upgrade ../chemenu-7.1.0.tar.gz --take-release tools/README.md",
|
"tools/wikitool dist upgrade ../chemenu-7.1.0.tar.gz --take-release tools/README.md",
|
||||||
|
"tools/wikitool dist upgrade --latest --expect 8.0.0 --dry-run",
|
||||||
),
|
),
|
||||||
never=(
|
never=(
|
||||||
"Never treat any of the three answers to locally changed files as the default.",
|
"Never treat any of the three answers to locally changed files as the default.",
|
||||||
),
|
),
|
||||||
see_also=(
|
see_also=(
|
||||||
"`wikitool version check` - finds out whether an update exists",
|
"`wikitool version check` - finds out whether an update exists",
|
||||||
|
"`wikitool version notes` - the notes of the release `--expect` should name",
|
||||||
"`instructions/upgrade-instance.md` - the order after the swap",
|
"`instructions/upgrade-instance.md` - the order after the swap",
|
||||||
"`INSTALL.md` § \"Version und Updates\" - which release, whether to take it, where "
|
"`INSTALL.md` § \"Version und Updates\" - which release, whether to take it, where "
|
||||||
"the tarball comes from",
|
"the tarball comes from",
|
||||||
@@ -1106,8 +1239,17 @@ def _report_plan(
|
|||||||
))
|
))
|
||||||
@app.command("upgrade")
|
@app.command("upgrade")
|
||||||
def upgrade_command(
|
def upgrade_command(
|
||||||
source: Path = typer.Argument(
|
source: Optional[Path] = typer.Argument(
|
||||||
..., help="An extracted distribution directory, or a release .tar.gz archive"
|
None, help="An extracted distribution directory, or a release .tar.gz archive - or use --latest"
|
||||||
|
),
|
||||||
|
latest: bool = typer.Option(
|
||||||
|
False, "--latest",
|
||||||
|
help="Download the release the feed announces as latest instead of naming a <source>",
|
||||||
|
),
|
||||||
|
expect: Optional[str] = typer.Option(
|
||||||
|
None, "--expect",
|
||||||
|
help="With --latest: refuse, before downloading anything, unless the feed's latest release "
|
||||||
|
"is this version (the one `version notes` was read for)",
|
||||||
),
|
),
|
||||||
dry_run: bool = typer.Option(
|
dry_run: bool = typer.Option(
|
||||||
False, "--dry-run", help="Classify and report, without writing anything"
|
False, "--dry-run", help="Classify and report, without writing anything"
|
||||||
@@ -1133,8 +1275,10 @@ def upgrade_command(
|
|||||||
):
|
):
|
||||||
"""Apply a stack update `dist export` produced - the write half of `version check`.
|
"""Apply a stack update `dist export` produced - the write half of `version check`.
|
||||||
\f
|
\f
|
||||||
Never downloads anything: `source` is an already-fetched
|
`source` is an already-fetched export directory or `.tar.gz` archive;
|
||||||
export directory or `.tar.gz` archive. Writes exactly the new release
|
`--latest` instead downloads the archive and its checksum from the
|
||||||
|
release feed (`--expect <version>` pins which release that may be) and
|
||||||
|
holds it to the checksum before anything else. Writes exactly the new release
|
||||||
stamp's `files` block, minus what an export re-seeds every time
|
stamp's `files` block, minus what an export re-seeds every time
|
||||||
(`kb/log.md`, `raw/.gitkeep`) or seeds once and the instance owns from
|
(`kb/log.md`, `raw/.gitkeep`) or seeds once and the instance owns from
|
||||||
then on (`.wikitool-kb.json`, `CHANGES.md`), classifying every candidate
|
then on (`.wikitool-kb.json`, `CHANGES.md`), classifying every candidate
|
||||||
@@ -1155,19 +1299,41 @@ def upgrade_command(
|
|||||||
take_release=take_release,
|
take_release=take_release,
|
||||||
prune=prune,
|
prune=prune,
|
||||||
allow_pre=allow_pre,
|
allow_pre=allow_pre,
|
||||||
|
latest=latest,
|
||||||
|
expect=expect,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
def run_upgrade(
|
def run_upgrade(
|
||||||
source: Path,
|
source: Optional[Path] = None,
|
||||||
dry_run: bool = False,
|
dry_run: bool = False,
|
||||||
keep_local: bool = False,
|
keep_local: bool = False,
|
||||||
take_release: Optional[Sequence[str]] = None,
|
take_release: Optional[Sequence[str]] = None,
|
||||||
prune: bool = False,
|
prune: bool = False,
|
||||||
allow_pre: bool = False,
|
allow_pre: bool = False,
|
||||||
|
latest: bool = False,
|
||||||
|
expect: Optional[str] = None,
|
||||||
) -> None:
|
) -> None:
|
||||||
"""The upgrade itself, free of Typer's option objects - see `run_export`
|
"""The upgrade itself, free of Typer's option objects - see `run_export`
|
||||||
for why this split exists."""
|
for why this split exists."""
|
||||||
|
if latest == (source is not None):
|
||||||
|
fail(
|
||||||
|
"Name exactly one source: `<source>` (an export directory or a release .tar.gz "
|
||||||
|
"already on disk) or `--latest` (the release the feed announces). "
|
||||||
|
+ ("Both were given." if latest else "Neither was given.")
|
||||||
|
)
|
||||||
|
return
|
||||||
|
expected_version = None
|
||||||
|
if expect is not None:
|
||||||
|
if not latest:
|
||||||
|
fail("--expect only makes sense with --latest: it pins the release the feed may announce.")
|
||||||
|
return
|
||||||
|
try:
|
||||||
|
expected_version = version_mod.Version.parse(expect)
|
||||||
|
except version_mod.VersionError as exc:
|
||||||
|
fail(f"--expect: {exc}")
|
||||||
|
return
|
||||||
|
|
||||||
try:
|
try:
|
||||||
local_version = version_mod.read_version()
|
local_version = version_mod.read_version()
|
||||||
except version_mod.VersionError as exc:
|
except version_mod.VersionError as exc:
|
||||||
@@ -1223,7 +1389,19 @@ def run_upgrade(
|
|||||||
)
|
)
|
||||||
return
|
return
|
||||||
|
|
||||||
with _resolved_source(source) as new_root:
|
feed = None
|
||||||
|
if latest:
|
||||||
|
feed = _query_feed(old_stamp, local_version, allow_pre, expected_version)
|
||||||
|
if feed is None:
|
||||||
|
return
|
||||||
|
release, feed_url, feed_token = feed
|
||||||
|
source_cm = _downloaded_source(release, feed_url, feed_token)
|
||||||
|
target = f"--latest --expect {release.version}"
|
||||||
|
else:
|
||||||
|
source_cm = _resolved_source(source)
|
||||||
|
target = rel_path(source)
|
||||||
|
|
||||||
|
with source_cm as new_root:
|
||||||
version_path = new_root / version_mod.VERSION_FILENAME
|
version_path = new_root / version_mod.VERSION_FILENAME
|
||||||
if not version_path.is_file():
|
if not version_path.is_file():
|
||||||
fail(f"{rel_path(new_root)} has no VERSION - not a distribution export.")
|
fail(f"{rel_path(new_root)} has no VERSION - not a distribution export.")
|
||||||
@@ -1234,18 +1412,16 @@ def run_upgrade(
|
|||||||
fail(str(exc))
|
fail(str(exc))
|
||||||
return
|
return
|
||||||
|
|
||||||
if new_version.is_prerelease and not allow_pre:
|
if feed is not None and new_version != feed[0].version:
|
||||||
fail(
|
fail(
|
||||||
f"{new_version} is a running candidate (-beta.N). `.gitea/workflows/release.yml` "
|
f"The downloaded archive's VERSION says {new_version}, but the feed announced "
|
||||||
"never publishes one, so a candidate tree can only come from a dev checkout by "
|
f"{feed[0].version} (tag {feed[0].tag}) - this release is inconsistent, so "
|
||||||
"hand - pass --pre if that is deliberate."
|
"nothing was applied. Report it against the release page"
|
||||||
|
+ (f" {feed[0].html_url}" if feed[0].html_url else "")
|
||||||
|
+ "."
|
||||||
)
|
)
|
||||||
return
|
return
|
||||||
if new_version < local_version:
|
if not _version_gate(new_version, local_version, allow_pre, "the source"):
|
||||||
fail(f"{new_version} is older than the installed {local_version} - refusing a downgrade.")
|
|
||||||
return
|
|
||||||
if new_version == local_version:
|
|
||||||
success(f"Already at {local_version}. Nothing to do.")
|
|
||||||
return
|
return
|
||||||
|
|
||||||
stamp_path = new_root / version_mod.RELEASE_STAMP_FILENAME
|
stamp_path = new_root / version_mod.RELEASE_STAMP_FILENAME
|
||||||
@@ -1289,7 +1465,7 @@ def run_upgrade(
|
|||||||
|
|
||||||
undecided = [path for path in classification.blocked if path not in taken]
|
undecided = [path for path in classification.blocked if path not in taken]
|
||||||
if undecided and not keep_local:
|
if undecided and not keep_local:
|
||||||
fail(_refusal_for_blocked(source, undecided, classification))
|
fail(_refusal_for_blocked(target, undecided, classification))
|
||||||
return
|
return
|
||||||
|
|
||||||
to_write = sorted(classification.unchanged + classification.new + sorted(taken))
|
to_write = sorted(classification.unchanged + classification.new + sorted(taken))
|
||||||
|
|||||||
@@ -29,6 +29,9 @@ number means, and `instructions/dev/version-parts.md` for the candidate model):
|
|||||||
stderr before asking and takes `--offline`. Both need no key, both time
|
stderr before asking and takes `--offline`. Both need no key, both time
|
||||||
out, and a feed that cannot be reached is reported as an error rather than
|
out, and a feed that cannot be reached is reported as an error rather than
|
||||||
silently answered as "up to date" or "no notes".
|
silently answered as "up to date" or "no notes".
|
||||||
|
- `dist upgrade --latest` reaches the same feed as a third, equally explicit
|
||||||
|
caller: it asks which release is latest, then downloads that release's
|
||||||
|
assets. Only `version_mod` talks to the feed; this module is not involved.
|
||||||
"""
|
"""
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
@@ -158,8 +161,10 @@ def show_command(
|
|||||||
"is `current`, `update`, `migration` (the step crosses a compatibility boundary) or "
|
"is `current`, `update`, `migration` (the step crosses a compatibility boundary) or "
|
||||||
"`ahead`.",
|
"`ahead`.",
|
||||||
"The only command whose whole job is the network call - `version notes` reaches the "
|
"The only command whose whole job is the network call - `version notes` reaches the "
|
||||||
"same feed too, but only as a fallback on a distributed instance.",
|
"same feed too, but only as a fallback on a distributed instance, and `dist upgrade "
|
||||||
"Never reached implicitly from another command, needs no key, and times out after "
|
"--latest` asks it which release to download.",
|
||||||
|
"Never reached implicitly from another command (`dist upgrade` asks only when passed "
|
||||||
|
"`--latest`), needs no key, and times out after "
|
||||||
"`--timeout` seconds (default 10).",
|
"`--timeout` seconds (default 10).",
|
||||||
"The feed is `--url`, else `$WIKITOOL_UPDATE_URL`, else the release stamp's, else the "
|
"The feed is `--url`, else `$WIKITOOL_UPDATE_URL`, else the release stamp's, else the "
|
||||||
"built-in origin. `$WIKITOOL_UPDATE_TOKEN` is only needed if that feed is not readable "
|
"built-in origin. `$WIKITOOL_UPDATE_TOKEN` is only needed if that feed is not readable "
|
||||||
|
|||||||
@@ -337,7 +337,8 @@ def test_every_gated_record_shows_its_re_run_after_exit_42():
|
|||||||
def test_network_yes_is_exactly_the_commands_that_can_reach_outside_this_checkout():
|
def test_network_yes_is_exactly_the_commands_that_can_reach_outside_this_checkout():
|
||||||
"""Gitea #144: `network:` means *any* reach outside this checkout - an HTTP call
|
"""Gitea #144: `network:` means *any* reach outside this checkout - an HTTP call
|
||||||
`wikitool` makes itself, or a git operation against a remote (fetch/ls-remote/push) -
|
`wikitool` makes itself, or a git operation against a remote (fetch/ls-remote/push) -
|
||||||
not only the two `version_cmd.py` used to claim exclusivity for. Pinned as an explicit
|
not only the two `version_cmd.py` used to claim exclusivity for. `dist upgrade` is on the list
|
||||||
|
since `--latest`, which asks the release feed and downloads from it. Pinned as an explicit
|
||||||
set so a command gaining or losing that reach is a deliberate edit here, not a silent
|
set so a command gaining or losing that reach is a deliberate edit here, not a silent
|
||||||
drift between the property and what the command actually does."""
|
drift between the property and what the command actually does."""
|
||||||
expected = {
|
expected = {
|
||||||
@@ -346,6 +347,7 @@ def test_network_yes_is_exactly_the_commands_that_can_reach_outside_this_checkou
|
|||||||
"upstream merge",
|
"upstream merge",
|
||||||
"version check",
|
"version check",
|
||||||
"version notes",
|
"version notes",
|
||||||
|
"dist upgrade",
|
||||||
"review",
|
"review",
|
||||||
"task new",
|
"task new",
|
||||||
"task list",
|
"task list",
|
||||||
|
|||||||
@@ -4,9 +4,13 @@ execution, and every refusal before anything is written. See Gitea #7."""
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import hashlib
|
import hashlib
|
||||||
|
import http.server
|
||||||
import json
|
import json
|
||||||
|
import shutil
|
||||||
import subprocess
|
import subprocess
|
||||||
import tarfile
|
import tarfile
|
||||||
|
import tempfile
|
||||||
|
import threading
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
@@ -586,3 +590,426 @@ def test_tarball_sha256_sidecar_mismatch_is_refused(instance, tmp_path):
|
|||||||
|
|
||||||
with pytest.raises(typer.Exit):
|
with pytest.raises(typer.Exit):
|
||||||
dist_cmd.run_upgrade(archive)
|
dist_cmd.run_upgrade(archive)
|
||||||
|
|
||||||
|
|
||||||
|
# --- --latest: the release feed ---------------------------------------------
|
||||||
|
#
|
||||||
|
# A real `http.server` on 127.0.0.1 rather than a fetcher stub: the download
|
||||||
|
# path is `urllib` streaming into a file, and the same-origin token rule is a
|
||||||
|
# statement about which requests carry which header - both only observable on
|
||||||
|
# the wire. Every server records `(path, Authorization)` for each request.
|
||||||
|
|
||||||
|
|
||||||
|
class FeedServer:
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self.routes: dict[str, tuple[int, bytes]] = {}
|
||||||
|
self.requests: list[tuple[str, str | None]] = []
|
||||||
|
outer = self
|
||||||
|
|
||||||
|
class Handler(http.server.BaseHTTPRequestHandler):
|
||||||
|
def do_GET(self): # noqa: N802 - http.server's name
|
||||||
|
outer.requests.append((self.path, self.headers.get("Authorization")))
|
||||||
|
status, body = outer.routes.get(self.path, (404, b"not found"))
|
||||||
|
self.send_response(status)
|
||||||
|
self.send_header("Content-Length", str(len(body)))
|
||||||
|
self.end_headers()
|
||||||
|
self.wfile.write(body)
|
||||||
|
|
||||||
|
def log_message(self, *args): # silence the test output
|
||||||
|
pass
|
||||||
|
|
||||||
|
self.httpd = http.server.HTTPServer(("127.0.0.1", 0), Handler)
|
||||||
|
self.thread = threading.Thread(target=self.httpd.serve_forever, kwargs={"poll_interval": 0.01}, daemon=True)
|
||||||
|
self.thread.start()
|
||||||
|
|
||||||
|
@property
|
||||||
|
def base(self) -> str:
|
||||||
|
return f"http://127.0.0.1:{self.httpd.server_address[1]}"
|
||||||
|
|
||||||
|
def stop(self) -> None:
|
||||||
|
self.httpd.shutdown()
|
||||||
|
self.httpd.server_close()
|
||||||
|
|
||||||
|
def paths(self) -> list[str]:
|
||||||
|
return [path for path, _ in self.requests]
|
||||||
|
|
||||||
|
def asset_paths(self) -> list[str]:
|
||||||
|
return [path for path in self.paths() if path.startswith("/assets/")]
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def feed(monkeypatch: pytest.MonkeyPatch):
|
||||||
|
server = FeedServer()
|
||||||
|
monkeypatch.setenv("WIKITOOL_UPDATE_URL", f"{server.base}/feed")
|
||||||
|
yield server
|
||||||
|
server.stop()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def other_origin():
|
||||||
|
server = FeedServer()
|
||||||
|
yield server
|
||||||
|
server.stop()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def scratch_tmp(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Path:
|
||||||
|
"""Redirect `tempfile` into a directory the test can inspect afterwards."""
|
||||||
|
scratch = tmp_path / "scratch-tmp"
|
||||||
|
scratch.mkdir()
|
||||||
|
monkeypatch.setattr(tempfile, "tempdir", str(scratch))
|
||||||
|
return scratch
|
||||||
|
|
||||||
|
|
||||||
|
def _publish(
|
||||||
|
feed: FeedServer,
|
||||||
|
tmp_path: Path,
|
||||||
|
version: str,
|
||||||
|
files: dict[str, str] | None = None,
|
||||||
|
*,
|
||||||
|
tag: str | None = None,
|
||||||
|
inner_version: str | None = None,
|
||||||
|
checksum: str | None = "ok",
|
||||||
|
archive: bool = True,
|
||||||
|
asset_server: FeedServer | None = None,
|
||||||
|
) -> str:
|
||||||
|
"""Serve `version` as the feed's latest release and return its archive name.
|
||||||
|
|
||||||
|
`inner_version` is what the archive's own VERSION says (default: the same),
|
||||||
|
`checksum` is `"ok"`, `"wrong"` or `None` for no `.sha256` asset."""
|
||||||
|
name = f"chemenu-stack-{version}"
|
||||||
|
tree = _release(
|
||||||
|
tmp_path, f"served-{name}", inner_version or version,
|
||||||
|
files or {"AGENTS.md": "core v2\n", "tools/wikitool": "#!/bin/sh\n"},
|
||||||
|
)
|
||||||
|
packed = tmp_path / f"{name}.tar.gz"
|
||||||
|
with tarfile.open(packed, "w:gz") as tf:
|
||||||
|
tf.add(tree, arcname=name)
|
||||||
|
data = packed.read_bytes()
|
||||||
|
digest = hashlib.sha256(data).hexdigest() if checksum == "ok" else "0" * 64
|
||||||
|
holder = asset_server or feed
|
||||||
|
assets = []
|
||||||
|
if archive:
|
||||||
|
holder.routes[f"/assets/{name}.tar.gz"] = (200, data)
|
||||||
|
assets.append({"name": f"{name}.tar.gz", "browser_download_url": f"{holder.base}/assets/{name}.tar.gz"})
|
||||||
|
if checksum is not None:
|
||||||
|
holder.routes[f"/assets/{name}.tar.gz.sha256"] = (200, f"{digest} {name}.tar.gz\n".encode())
|
||||||
|
assets.append({
|
||||||
|
"name": f"{name}.tar.gz.sha256",
|
||||||
|
"browser_download_url": f"{holder.base}/assets/{name}.tar.gz.sha256",
|
||||||
|
})
|
||||||
|
feed.routes["/feed"] = (
|
||||||
|
200,
|
||||||
|
json.dumps({
|
||||||
|
"tag_name": tag or f"v{version}",
|
||||||
|
"html_url": f"{feed.base}/releases/tag/v{version}",
|
||||||
|
"assets": assets,
|
||||||
|
}).encode(),
|
||||||
|
)
|
||||||
|
return name
|
||||||
|
|
||||||
|
|
||||||
|
def _stamp_version(instance: Path) -> str:
|
||||||
|
return json.loads((instance / version_mod.RELEASE_STAMP_FILENAME).read_text())["version"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_latest_downloads_verifies_and_applies_the_feeds_release(instance, tmp_path, feed, scratch_tmp):
|
||||||
|
name = _publish(feed, tmp_path, "1.1.0")
|
||||||
|
|
||||||
|
dist_cmd.run_upgrade(latest=True)
|
||||||
|
|
||||||
|
assert (instance / "AGENTS.md").read_text(encoding="utf-8") == "core v2\n"
|
||||||
|
assert _stamp_version(instance) == "1.1.0"
|
||||||
|
assert feed.paths() == ["/feed", f"/assets/{name}.tar.gz", f"/assets/{name}.tar.gz.sha256"]
|
||||||
|
assert list(scratch_tmp.iterdir()) == []
|
||||||
|
recorded = json.loads((instance / version_mod.RELEASE_STAMP_FILENAME).read_text())["files"]
|
||||||
|
assert recorded
|
||||||
|
for relative, digest in recorded.items():
|
||||||
|
assert _digest((instance / relative).read_text(encoding="utf-8")) == digest
|
||||||
|
|
||||||
|
|
||||||
|
def test_latest_yields_the_same_tree_as_the_archive_it_downloaded(instance, tmp_path, feed, monkeypatch):
|
||||||
|
name = _publish(feed, tmp_path, "1.1.0")
|
||||||
|
served = tmp_path / f"{name}.tar.gz"
|
||||||
|
snapshot = tmp_path / "instance-copy"
|
||||||
|
shutil.copytree(instance, snapshot)
|
||||||
|
|
||||||
|
dist_cmd.run_upgrade(latest=True)
|
||||||
|
via_feed = {p.relative_to(instance): p.read_bytes() for p in instance.rglob("*") if p.is_file()}
|
||||||
|
|
||||||
|
monkeypatch.setattr(config, "ROOT", snapshot)
|
||||||
|
dist_cmd.run_upgrade(served)
|
||||||
|
via_archive = {p.relative_to(snapshot): p.read_bytes() for p in snapshot.rglob("*") if p.is_file()}
|
||||||
|
|
||||||
|
assert via_feed == via_archive
|
||||||
|
|
||||||
|
|
||||||
|
def test_latest_dry_run_downloads_writes_nothing_and_cleans_up(instance, tmp_path, feed, scratch_tmp):
|
||||||
|
name = _publish(feed, tmp_path, "1.1.0")
|
||||||
|
|
||||||
|
dist_cmd.run_upgrade(latest=True, dry_run=True)
|
||||||
|
|
||||||
|
assert (instance / "AGENTS.md").read_text(encoding="utf-8") == "core\n"
|
||||||
|
assert _stamp_version(instance) == "1.0.0"
|
||||||
|
assert f"/assets/{name}.tar.gz" in feed.paths()
|
||||||
|
assert list(scratch_tmp.iterdir()) == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_latest_already_current_is_a_no_op_without_any_download(instance, tmp_path, feed):
|
||||||
|
_publish(feed, tmp_path, "1.0.0")
|
||||||
|
|
||||||
|
dist_cmd.run_upgrade(latest=True)
|
||||||
|
|
||||||
|
assert feed.paths() == ["/feed"]
|
||||||
|
assert _stamp_version(instance) == "1.0.0"
|
||||||
|
|
||||||
|
|
||||||
|
def test_latest_older_than_installed_is_a_downgrade_refusal_before_any_download(instance, tmp_path, feed, capsys):
|
||||||
|
_publish(feed, tmp_path, "0.9.0")
|
||||||
|
|
||||||
|
with pytest.raises(typer.Exit) as excinfo:
|
||||||
|
dist_cmd.run_upgrade(latest=True)
|
||||||
|
|
||||||
|
assert excinfo.value.exit_code == 1
|
||||||
|
assert "downgrade" in capsys.readouterr().out
|
||||||
|
assert feed.asset_paths() == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_latest_prerelease_needs_pre_and_is_refused_before_any_download(instance, tmp_path, feed):
|
||||||
|
_publish(feed, tmp_path, "1.1.0-beta.1")
|
||||||
|
|
||||||
|
with pytest.raises(typer.Exit):
|
||||||
|
dist_cmd.run_upgrade(latest=True)
|
||||||
|
assert feed.asset_paths() == []
|
||||||
|
|
||||||
|
dist_cmd.run_upgrade(latest=True, allow_pre=True)
|
||||||
|
assert _stamp_version(instance) == "1.1.0-beta.1"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("missing", ["checksum", "archive"])
|
||||||
|
def test_latest_release_without_an_asset_is_refused_before_any_download(
|
||||||
|
instance, tmp_path, feed, capsys, missing
|
||||||
|
):
|
||||||
|
_publish(
|
||||||
|
feed, tmp_path, "1.1.0",
|
||||||
|
checksum=None if missing == "checksum" else "ok",
|
||||||
|
archive=missing != "archive",
|
||||||
|
)
|
||||||
|
|
||||||
|
with pytest.raises(typer.Exit) as excinfo:
|
||||||
|
dist_cmd.run_upgrade(latest=True)
|
||||||
|
|
||||||
|
assert excinfo.value.exit_code == 1
|
||||||
|
out = " ".join(capsys.readouterr().out.split())
|
||||||
|
assert "chemenu-stack-1.1.0.tar.gz" in out
|
||||||
|
assert "assets present" in out
|
||||||
|
assert "/releases/tag/v1.1.0" in out
|
||||||
|
assert feed.asset_paths() == []
|
||||||
|
assert _stamp_version(instance) == "1.0.0"
|
||||||
|
|
||||||
|
|
||||||
|
def test_latest_checksum_mismatch_writes_nothing_and_cleans_up(instance, tmp_path, feed, scratch_tmp, capsys):
|
||||||
|
_publish(feed, tmp_path, "1.1.0", checksum="wrong")
|
||||||
|
|
||||||
|
with pytest.raises(typer.Exit) as excinfo:
|
||||||
|
dist_cmd.run_upgrade(latest=True)
|
||||||
|
|
||||||
|
assert excinfo.value.exit_code == 1
|
||||||
|
assert "does not match" in " ".join(capsys.readouterr().out.split())
|
||||||
|
assert (instance / "AGENTS.md").read_text(encoding="utf-8") == "core\n"
|
||||||
|
assert _stamp_version(instance) == "1.0.0"
|
||||||
|
assert list(scratch_tmp.iterdir()) == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_latest_archive_version_that_differs_from_the_feed_is_refused(instance, tmp_path, feed, scratch_tmp, capsys):
|
||||||
|
_publish(feed, tmp_path, "1.1.0", inner_version="1.2.0")
|
||||||
|
|
||||||
|
with pytest.raises(typer.Exit) as excinfo:
|
||||||
|
dist_cmd.run_upgrade(latest=True)
|
||||||
|
|
||||||
|
assert excinfo.value.exit_code == 1
|
||||||
|
out = " ".join(capsys.readouterr().out.split())
|
||||||
|
assert "1.2.0" in out and "1.1.0" in out
|
||||||
|
assert _stamp_version(instance) == "1.0.0"
|
||||||
|
assert list(scratch_tmp.iterdir()) == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_latest_asset_download_error_is_reported_with_its_url(instance, tmp_path, feed, capsys):
|
||||||
|
name = _publish(feed, tmp_path, "1.1.0")
|
||||||
|
del feed.routes[f"/assets/{name}.tar.gz"]
|
||||||
|
|
||||||
|
with pytest.raises(typer.Exit) as excinfo:
|
||||||
|
dist_cmd.run_upgrade(latest=True)
|
||||||
|
|
||||||
|
assert excinfo.value.exit_code == 1
|
||||||
|
assert f"/assets/{name}.tar.gz answered HTTP 404" in " ".join(capsys.readouterr().out.split())
|
||||||
|
assert _stamp_version(instance) == "1.0.0"
|
||||||
|
|
||||||
|
|
||||||
|
def test_latest_unreachable_feed_is_an_error_naming_the_url(instance, monkeypatch, capsys):
|
||||||
|
monkeypatch.setenv("WIKITOOL_UPDATE_URL", "http://127.0.0.1:1/feed")
|
||||||
|
|
||||||
|
with pytest.raises(typer.Exit) as excinfo:
|
||||||
|
dist_cmd.run_upgrade(latest=True)
|
||||||
|
|
||||||
|
assert excinfo.value.exit_code == 1
|
||||||
|
assert "Could not reach http://127.0.0.1:1/feed" in " ".join(capsys.readouterr().out.split())
|
||||||
|
|
||||||
|
|
||||||
|
def test_latest_respects_the_local_preconditions_before_asking_the_feed(instance, tmp_path, feed):
|
||||||
|
_publish(feed, tmp_path, "1.1.0")
|
||||||
|
(instance / "AGENTS.md").write_text("locally edited\n", encoding="utf-8")
|
||||||
|
subprocess.run(["git", "init", "-q"], cwd=instance, check=True)
|
||||||
|
|
||||||
|
with pytest.raises(typer.Exit):
|
||||||
|
dist_cmd.run_upgrade(latest=True)
|
||||||
|
|
||||||
|
assert feed.requests == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_latest_blocked_refusal_names_the_pinned_latest_invocation(instance, tmp_path, feed, capsys):
|
||||||
|
_publish(feed, tmp_path, "1.1.0")
|
||||||
|
(instance / "AGENTS.md").write_text("locally edited\n", encoding="utf-8")
|
||||||
|
|
||||||
|
with pytest.raises(typer.Exit):
|
||||||
|
dist_cmd.run_upgrade(latest=True)
|
||||||
|
|
||||||
|
out = " ".join(capsys.readouterr().out.split())
|
||||||
|
assert "dist upgrade --latest --expect 1.1.0 --take-release AGENTS.md" in out
|
||||||
|
assert "dist upgrade --latest --expect 1.1.0 --keep-local" in out
|
||||||
|
|
||||||
|
|
||||||
|
def test_source_and_latest_together_or_neither_is_refused_before_any_check(instance, tmp_path, feed):
|
||||||
|
release = _release(tmp_path, "release", "1.1.0", {"AGENTS.md": "core\n"})
|
||||||
|
|
||||||
|
with pytest.raises(typer.Exit) as both:
|
||||||
|
dist_cmd.run_upgrade(release, latest=True)
|
||||||
|
with pytest.raises(typer.Exit) as neither:
|
||||||
|
dist_cmd.run_upgrade()
|
||||||
|
|
||||||
|
assert both.value.exit_code == 1 and neither.value.exit_code == 1
|
||||||
|
assert feed.requests == []
|
||||||
|
|
||||||
|
|
||||||
|
# --- --expect ---------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("spelling", ["1.1.0", "v1.1.0"])
|
||||||
|
def test_expect_matching_the_feed_proceeds(instance, tmp_path, feed, spelling):
|
||||||
|
_publish(feed, tmp_path, "1.1.0")
|
||||||
|
|
||||||
|
dist_cmd.run_upgrade(latest=True, expect=spelling)
|
||||||
|
|
||||||
|
assert _stamp_version(instance) == "1.1.0"
|
||||||
|
|
||||||
|
|
||||||
|
def test_expect_mismatch_is_refused_before_any_download_naming_both_versions(instance, tmp_path, feed, capsys):
|
||||||
|
_publish(feed, tmp_path, "1.2.0")
|
||||||
|
|
||||||
|
with pytest.raises(typer.Exit) as excinfo:
|
||||||
|
dist_cmd.run_upgrade(latest=True, expect="1.1.0")
|
||||||
|
|
||||||
|
assert excinfo.value.exit_code == 1
|
||||||
|
out = " ".join(capsys.readouterr().out.split())
|
||||||
|
assert "1.2.0" in out and "1.1.0" in out and "version notes" in out
|
||||||
|
assert feed.asset_paths() == []
|
||||||
|
assert _stamp_version(instance) == "1.0.0"
|
||||||
|
|
||||||
|
|
||||||
|
def test_expect_mismatch_wins_over_an_already_current_install(instance, tmp_path, feed):
|
||||||
|
_publish(feed, tmp_path, "1.0.0")
|
||||||
|
|
||||||
|
with pytest.raises(typer.Exit):
|
||||||
|
dist_cmd.run_upgrade(latest=True, expect="1.1.0")
|
||||||
|
|
||||||
|
|
||||||
|
def test_expect_without_latest_is_refused(instance, tmp_path, feed):
|
||||||
|
release = _release(tmp_path, "release", "1.1.0", {"AGENTS.md": "core\n"})
|
||||||
|
|
||||||
|
with pytest.raises(typer.Exit) as excinfo:
|
||||||
|
dist_cmd.run_upgrade(release, expect="1.1.0")
|
||||||
|
|
||||||
|
assert excinfo.value.exit_code == 1
|
||||||
|
assert feed.requests == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_expect_that_is_not_a_version_is_refused(instance, feed):
|
||||||
|
with pytest.raises(typer.Exit):
|
||||||
|
dist_cmd.run_upgrade(latest=True, expect="latest")
|
||||||
|
assert feed.requests == []
|
||||||
|
|
||||||
|
|
||||||
|
# --- the token --------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_token_goes_to_the_feed_and_same_origin_assets_only(instance, tmp_path, feed, other_origin, monkeypatch):
|
||||||
|
monkeypatch.setenv("WIKITOOL_UPDATE_TOKEN", "s3cret")
|
||||||
|
name = _publish(feed, tmp_path, "1.1.0")
|
||||||
|
|
||||||
|
dist_cmd.run_upgrade(latest=True)
|
||||||
|
|
||||||
|
assert {auth for _, auth in feed.requests} == {"token s3cret"}
|
||||||
|
assert f"/assets/{name}.tar.gz" in feed.paths()
|
||||||
|
assert other_origin.requests == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_token_is_withheld_from_an_asset_on_another_origin(instance, tmp_path, feed, other_origin, monkeypatch):
|
||||||
|
monkeypatch.setenv("WIKITOOL_UPDATE_TOKEN", "s3cret")
|
||||||
|
_publish(feed, tmp_path, "1.1.0", asset_server=other_origin)
|
||||||
|
|
||||||
|
dist_cmd.run_upgrade(latest=True)
|
||||||
|
|
||||||
|
assert _stamp_version(instance) == "1.1.0"
|
||||||
|
assert feed.requests == [("/feed", "token s3cret")]
|
||||||
|
assert len(other_origin.requests) == 2
|
||||||
|
assert {auth for _, auth in other_origin.requests} == {None}
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
"feed_url, asset_url, sent",
|
||||||
|
[
|
||||||
|
("https://git.example/api/v1/releases/latest", "https://git.example/dl/a.tar.gz", True),
|
||||||
|
("https://git.example/api/v1/releases/latest", "https://git.example:443/dl/a.tar.gz", True),
|
||||||
|
("https://git.example/api/v1/releases/latest", "http://git.example/dl/a.tar.gz", False),
|
||||||
|
("https://git.example/api/v1/releases/latest", "https://cdn.example/dl/a.tar.gz", False),
|
||||||
|
("http://git.example:3000/x", "http://git.example:3001/x", False),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_token_for_asset_compares_scheme_host_and_port(feed_url, asset_url, sent):
|
||||||
|
assert version_mod.token_for_asset(feed_url, asset_url, "t") == ("t" if sent else None)
|
||||||
|
assert version_mod.token_for_asset(feed_url, asset_url, None) is None
|
||||||
|
|
||||||
|
|
||||||
|
# --- the asset names --------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_release_asset_names_match_the_workflow():
|
||||||
|
workflow = Path(__file__).resolve().parents[3] / ".gitea" / "workflows" / "release.yml"
|
||||||
|
if not workflow.is_file():
|
||||||
|
pytest.skip("a distributed instance carries no release workflow")
|
||||||
|
text = workflow.read_text(encoding="utf-8")
|
||||||
|
|
||||||
|
archive = version_mod.ARCHIVE_NAME.format(version="${VERSION}")
|
||||||
|
stem = archive.removesuffix(".tar.gz")
|
||||||
|
assert f'name="{stem}"' in text
|
||||||
|
assert archive.replace(stem, "${name}") in text
|
||||||
|
assert (archive + version_mod.CHECKSUM_SUFFIX).replace(stem, "${name}") in text
|
||||||
|
|
||||||
|
|
||||||
|
def test_fetch_latest_assets_reads_urls_off_the_release_and_skips_malformed_entries():
|
||||||
|
payload = json.dumps({
|
||||||
|
"tag_name": "v1.1.0",
|
||||||
|
"assets": [
|
||||||
|
{"name": "a.tar.gz", "browser_download_url": "https://h/a"},
|
||||||
|
{"name": "no-url"},
|
||||||
|
"not an object",
|
||||||
|
],
|
||||||
|
}).encode()
|
||||||
|
|
||||||
|
release = version_mod.fetch_latest_assets("https://h/feed", fetcher=lambda u, t, to: payload)
|
||||||
|
|
||||||
|
assert release.version == version_mod.Version.parse("1.1.0")
|
||||||
|
assert release.tag == "v1.1.0"
|
||||||
|
assert release.assets == {"a.tar.gz": "https://h/a"}
|
||||||
|
with pytest.raises(version_mod.VersionError, match="assets present: a.tar.gz"):
|
||||||
|
release.asset_urls()
|
||||||
+124
-5
@@ -37,7 +37,9 @@ import functools
|
|||||||
import json
|
import json
|
||||||
import os
|
import os
|
||||||
import re
|
import re
|
||||||
|
import shutil
|
||||||
import urllib.error
|
import urllib.error
|
||||||
|
import urllib.parse
|
||||||
import urllib.request
|
import urllib.request
|
||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
@@ -349,12 +351,15 @@ def fetch_latest(
|
|||||||
|
|
||||||
The network call sits behind `fetcher` so every caller above this line -
|
The network call sits behind `fetcher` so every caller above this line -
|
||||||
and every test - can run without a network. This is the one place that
|
and every test - can run without a network. This is the one place that
|
||||||
talks to the **release feed**, and only two commands reach it: `version
|
talks to the **release feed**, and only three commands reach it: `version
|
||||||
check`, whose whole job it is, and `version notes` on a *distributed*
|
check`, whose whole job it is, `version notes` on a *distributed*
|
||||||
instance, whose local `CHANGES.md` is a stub with no entry to print (see
|
instance, whose local `CHANGES.md` is a stub with no entry to print (see
|
||||||
`fetch_latest_notes`). Neither is implicit - `check` exists for the call,
|
`fetch_latest_notes`), and `dist upgrade --latest`, which asks the feed
|
||||||
and `notes` announces the URL it is asking before it asks, on stderr, and
|
which release to download (see `fetch_latest_assets`). None is implicit -
|
||||||
takes `--offline` for a caller that wants none of it.
|
`check` exists for the call, `notes` announces the URL it is asking before
|
||||||
|
it asks, on stderr, and takes `--offline` for a caller that wants none of
|
||||||
|
it, and `upgrade` reaches the feed only when the operator passes
|
||||||
|
`--latest`.
|
||||||
"""
|
"""
|
||||||
fetch = fetcher or _urlopen_fetch
|
fetch = fetcher or _urlopen_fetch
|
||||||
try:
|
try:
|
||||||
@@ -462,6 +467,120 @@ def fetch_latest_notes(
|
|||||||
return version, body, (str(html_url) if html_url else None)
|
return version, body, (str(html_url) if html_url else None)
|
||||||
|
|
||||||
|
|
||||||
|
# --- release assets --------------------------------------------------------
|
||||||
|
#
|
||||||
|
# `dist upgrade --latest` downloads the two assets `.gitea/workflows/release.yml`
|
||||||
|
# attaches to every release. The names are that workflow's `name=` line plus
|
||||||
|
# `.tar.gz` / `.tar.gz.sha256`; `test_release_asset_names_match_the_workflow`
|
||||||
|
# ties this constant to the workflow so one cannot move without the other.
|
||||||
|
|
||||||
|
ARCHIVE_NAME = "chemenu-stack-{version}.tar.gz"
|
||||||
|
CHECKSUM_SUFFIX = ".sha256"
|
||||||
|
DOWNLOAD_TIMEOUT = 60.0
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class LatestRelease:
|
||||||
|
"""What the feed's latest release offers: its version, its tag as the feed
|
||||||
|
spells it, its own page, and every asset as `name -> download URL`."""
|
||||||
|
|
||||||
|
version: Version
|
||||||
|
tag: str
|
||||||
|
html_url: Optional[str]
|
||||||
|
assets: dict[str, str]
|
||||||
|
|
||||||
|
@property
|
||||||
|
def archive_name(self) -> str:
|
||||||
|
return ARCHIVE_NAME.format(version=self.version)
|
||||||
|
|
||||||
|
def asset_urls(self) -> tuple[str, str]:
|
||||||
|
"""`(archive URL, checksum URL)`, read off the release object and never
|
||||||
|
composed. Raises `VersionError` naming the release page when either is
|
||||||
|
missing - an upgrade without its checksum is refused, not downgraded to
|
||||||
|
an unchecked one."""
|
||||||
|
archive = self.archive_name
|
||||||
|
wanted = (archive, archive + CHECKSUM_SUFFIX)
|
||||||
|
missing = [name for name in wanted if name not in self.assets]
|
||||||
|
if missing:
|
||||||
|
present = ", ".join(sorted(self.assets)) or "none"
|
||||||
|
page = f" - see the release page {self.html_url}" if self.html_url else ""
|
||||||
|
raise VersionError(
|
||||||
|
f"release {self.tag} does not publish {' and '.join(missing)} "
|
||||||
|
f"(assets present: {present}){page}"
|
||||||
|
)
|
||||||
|
return self.assets[wanted[0]], self.assets[wanted[1]]
|
||||||
|
|
||||||
|
|
||||||
|
def fetch_latest_assets(
|
||||||
|
url: str,
|
||||||
|
token: Optional[str] = None,
|
||||||
|
timeout: float = 10.0,
|
||||||
|
fetcher: Optional[Fetcher] = None,
|
||||||
|
) -> LatestRelease:
|
||||||
|
"""`fetch_latest`'s version plus the release's assets, from the one
|
||||||
|
response - the asset URLs are the feed's own `browser_download_url`
|
||||||
|
values, never a path this code composes (AGENTS.md invariant 7)."""
|
||||||
|
version, captured = _fetch_latest_object(url, token, timeout, fetcher)
|
||||||
|
assets: dict[str, str] = {}
|
||||||
|
raw_assets = captured.get("assets")
|
||||||
|
if isinstance(raw_assets, list):
|
||||||
|
for entry in raw_assets:
|
||||||
|
if not isinstance(entry, dict):
|
||||||
|
continue
|
||||||
|
name = str(entry.get("name") or "").strip()
|
||||||
|
download = str(entry.get("browser_download_url") or "").strip()
|
||||||
|
if name and download:
|
||||||
|
assets[name] = download
|
||||||
|
html_url = captured.get("html_url") or captured.get("url")
|
||||||
|
return LatestRelease(
|
||||||
|
version=version,
|
||||||
|
tag=str(captured.get("tag_name") or "").strip(),
|
||||||
|
html_url=str(html_url) if html_url else None,
|
||||||
|
assets=assets,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _origin(url: str) -> tuple[str, str, int]:
|
||||||
|
parts = urllib.parse.urlsplit(url)
|
||||||
|
scheme = parts.scheme.lower()
|
||||||
|
default = {"http": 80, "https": 443}.get(scheme, 0)
|
||||||
|
return scheme, (parts.hostname or "").lower(), parts.port or default
|
||||||
|
|
||||||
|
|
||||||
|
def token_for_asset(feed_url: str, asset_url: str, token: Optional[str]) -> Optional[str]:
|
||||||
|
"""The token to send with an asset download: `token` when the asset is on
|
||||||
|
the feed's own origin (scheme, host, port), `None` otherwise. An asset URL
|
||||||
|
is data from a JSON answer, so the credential does not follow it to
|
||||||
|
another host."""
|
||||||
|
return token if token and _origin(feed_url) == _origin(asset_url) else None
|
||||||
|
|
||||||
|
|
||||||
|
def download_asset(
|
||||||
|
url: str,
|
||||||
|
dest: Path,
|
||||||
|
token: Optional[str] = None,
|
||||||
|
timeout: float = DOWNLOAD_TIMEOUT,
|
||||||
|
) -> None:
|
||||||
|
"""Stream `url` into `dest`. `timeout` is a socket timeout per read, not a
|
||||||
|
deadline for the whole transfer. The token goes out as an *unredirected*
|
||||||
|
header, so a redirect to another host does not carry it along. Every
|
||||||
|
failure is a `VersionError` naming the URL."""
|
||||||
|
request = urllib.request.Request(url)
|
||||||
|
if token:
|
||||||
|
request.add_unredirected_header("Authorization", f"token {token}")
|
||||||
|
try:
|
||||||
|
with urllib.request.urlopen(request, timeout=timeout) as response: # noqa: S310 - URL from the feed
|
||||||
|
with dest.open("wb") as out:
|
||||||
|
shutil.copyfileobj(response, out)
|
||||||
|
except urllib.error.HTTPError as exc:
|
||||||
|
hint = ""
|
||||||
|
if exc.code in (401, 403):
|
||||||
|
hint = f" - the download needs authentication; set ${UPDATE_TOKEN_ENV}"
|
||||||
|
raise VersionError(f"{url} answered HTTP {exc.code}{hint}") from exc
|
||||||
|
except (urllib.error.URLError, OSError, TimeoutError) as exc:
|
||||||
|
raise VersionError(f"Could not download {url}: {exc}") from exc
|
||||||
|
|
||||||
|
|
||||||
# --- CHANGES.md ------------------------------------------------------------
|
# --- CHANGES.md ------------------------------------------------------------
|
||||||
#
|
#
|
||||||
# The changelog is prose and stays the author's job. What is mechanical is the
|
# The changelog is prose and stays the author's job. What is mechanical is the
|
||||||
|
|||||||
Reference in new issue
Block a user