feat: dist upgrade --latest downloads and verifies the release from the feed, --expect pins the version (#161)
Files changed: - CHANGES.md - INSTALL.md - VERSION - instructions/upgrade-instance.md - tools/CONTRACT.md - tools/chemenu/commands/dist_cmd.py - tools/chemenu/commands/version_cmd.py - tools/chemenu/tests/test_cli.py - tools/chemenu/tests/test_dist_upgrade.py - tools/chemenu/version.py
This commit is contained in:
1 parent
dd885db625
commit
cf892315e6
10 files changed
+888
-62
No files matched your search
+46
-1
@@ -59,13 +59,14 @@ concern - readable here, never shipped as something to parse.
|
||||
|
||||
---
|
||||
|
||||
## 7.1.0-beta.31 - 2026-09-26 - new_page/type_resolver comments no longer claim only entities declare a layout:
|
||||
## 7.1.0-beta.32 - 2026-09-29 - dist upgrade --latest: one-command update from the release feed
|
||||
|
||||
**Author:** Torben Nehmer
|
||||
|
||||
<!-- wikitool:bumps -->
|
||||
**High impact**
|
||||
- wikitool: one data record per command - `-h`, index and CONTRACT.md render from cli_contract (Gitea #121 Phase 1)
|
||||
- dist upgrade --latest: one-command update from the release feed
|
||||
|
||||
**Medium impact**
|
||||
- CalDAV task-tracker provider (Nextcloud Tasks, iOS Reminders); review reports unknown-value findings instead of skipping them
|
||||
@@ -102,6 +103,50 @@ concern - readable here, never shipped as something to parse.
|
||||
- new_page/type_resolver comments no longer claim only entities declare a layout:
|
||||
<!-- /wikitool:bumps -->
|
||||
|
||||
### dist upgrade --latest: one-command update from the release feed (Gitea #161)
|
||||
|
||||
Updating a tarball instance took a manual detour: `version notes`, then fetching the `.tar.gz` and
|
||||
its `.sha256` from the release page by hand, then `dist upgrade <tarball>`. `dist upgrade --latest`
|
||||
now does the middle part. It asks the release feed (`update_url` from the stamp,
|
||||
`$WIKITOOL_UPDATE_URL` and `$WIKITOOL_UPDATE_TOKEN` as before) which release is latest, downloads
|
||||
the release's archive and checksum into a scratch directory, verifies the archive against the
|
||||
checksum, and hands over to the existing upgrade path unchanged - classification,
|
||||
`--keep-local`/`--take-release`, migration report and every refusal are the same code. `<source>`
|
||||
becomes optional; exactly one of it and `--latest` must be given.
|
||||
|
||||
The order is what matters. The local preconditions run first, then the feed is asked, and the
|
||||
version it reports is judged **before any download**: already installed is a success no-op, an
|
||||
older version is a downgrade refusal, a `-beta.N` version needs `--pre`. Only then are the two
|
||||
assets looked up - by exact name, `chemenu-stack-<version>.tar.gz` and its `.sha256`, in the
|
||||
`assets` of the release object the version query already returned, using the feed's own
|
||||
`browser_download_url` values; no URL is composed. A release missing either asset is refused
|
||||
before the first download with the assets it does have and its release page named. The checksum
|
||||
is mandatory on this path (a `<source>` archive without a sibling `.sha256` is still only a
|
||||
WARN), and the archive's own `VERSION` must equal the feed's version, otherwise nothing is
|
||||
applied. `--dry-run` downloads and verifies too - classification needs the tree - and removes
|
||||
everything again; the scratch directory goes away on every exit.
|
||||
|
||||
`--expect <version>` (only with `--latest`) closes the gap between reading the notes and applying
|
||||
the update: the feed only offers its *latest* release, so if a newer one appeared since
|
||||
`version notes` was read, the run refuses before downloading anything and names both versions.
|
||||
`instructions/upgrade-instance.md` passes the version from step 2 in the dry run and in the real
|
||||
run. The comparison is on parsed versions, so a `v` prefix does not matter.
|
||||
|
||||
`$WIKITOOL_UPDATE_TOKEN` is sent to an asset download only when the asset URL has the feed's
|
||||
scheme, host and port, and as an unredirected header, so a redirect cannot carry it elsewhere.
|
||||
There is no https enforcement: the checksum comes from the same host as the archive and protects
|
||||
against transfer errors, not against a compromised feed - `INSTALL.md` says so.
|
||||
|
||||
The asset names are a Python constant (`version.ARCHIVE_NAME`) that a test ties to
|
||||
`.gitea/workflows/release.yml`, so renaming them in one place fails the suite instead of the next
|
||||
upgrade. `dist upgrade` is now `network: yes` in its record and in the pinned set in `test_cli.py`,
|
||||
the `version check` record's "never reached implicitly" note names the one explicit exception, the
|
||||
`fetch_latest` docstring says "three commands", and `tools/CONTRACT.md` is regenerated. The tests
|
||||
run against a local `http.server` that logs every request and its `Authorization` header, so the
|
||||
token rule and the "no asset request before the checks pass" rule are asserted on the wire. The
|
||||
`upstream merge` pointers in the `dist upgrade` record and failure message stay as they are; they
|
||||
belong to the separate work on the clone path.
|
||||
|
||||
### new_page/type_resolver comments no longer claim only entities declare a layout:
|
||||
|
||||
Two code comments - `new_page.py`'s module docstring and `TypeResolver.get_layout`'s - still said
|
||||
|
||||
+37
-12
@@ -152,12 +152,14 @@ tools/wikitool version # was läuft hier, und woher kommt es
|
||||
tools/wikitool version check # gibt es ein neueres Release?
|
||||
```
|
||||
|
||||
`version check` und `version notes` sind die einzigen Befehle, die ins Netz gehen, und beide
|
||||
fragen denselben Release-Feed der Ursprungs-Instanz (`$WIKITOOL_UPDATE_URL` überschreibt; sonst
|
||||
der Wert aus dem Stamp). `version check` ist dafür da; `version notes` greift nur dann darauf
|
||||
zurück, wenn die lokale `CHANGES.md` den Eintrag nicht hat - auf einer Instanz also immer, siehe
|
||||
unten - und sagt vorher auf stderr, welche URL es fragt. Ein nicht erreichbarer Feed wird als
|
||||
Fehler gemeldet - **nie** als „aktuell" und nie als „keine Notes".
|
||||
`version check`, `version notes` und `dist upgrade --latest` sind die einzigen Befehle, die
|
||||
ins Netz gehen, und alle drei fragen denselben Release-Feed der Ursprungs-Instanz
|
||||
(`$WIKITOOL_UPDATE_URL` überschreibt; sonst der Wert aus dem Stamp). `version check` ist dafür da;
|
||||
`version notes` greift nur dann darauf zurück, wenn die lokale `CHANGES.md` den Eintrag nicht
|
||||
hat - auf einer Instanz also immer, siehe unten - und sagt vorher auf stderr, welche URL es
|
||||
fragt; `dist upgrade` fragt den Feed nur, wenn `--latest` dasteht, und lädt dann auch das Release
|
||||
herunter (siehe „Eine Instanz aktualisieren"). Ein nicht erreichbarer Feed wird als Fehler
|
||||
gemeldet - **nie** als „aktuell" und nie als „keine Notes".
|
||||
|
||||
**Was die Versionsnummer aussagt:** kompatibel ist, was in der *linkesten von Null
|
||||
verschiedenen Stelle* übereinstimmt. `0.1.3 → 0.1.4` ist ein sicheres Update, `0.1.3 → 0.2.0`
|
||||
@@ -203,9 +205,32 @@ Agent-Sitzung ausgeführt wird; eine zweite Fassung derselben Schrittfolge an di
|
||||
genau die Kopie, die irgendwann auseinanderläuft. Wer den Lauf selbst fahren will, liest dieselbe
|
||||
Datei.
|
||||
|
||||
Was dieses Dokument beiträgt, ist die Entscheidung *davor* - welches Release, ob überhaupt, woher
|
||||
der Tarball kommt (§ „Version und Updates" und Weg A oben) - und der eine Sonderfall, den die
|
||||
Instruktion nicht abdecken kann, weil es sie dort noch nicht gibt:
|
||||
Was dieses Dokument beiträgt, ist die Entscheidung *davor* - welches Release, ob überhaupt, und
|
||||
wem die Instanz als Quelle vertraut - und zwei Sonderfälle, die die Instruktion nicht abdecken
|
||||
kann, weil es sie dort noch nicht gibt.
|
||||
|
||||
**Das Release kommt mit einem Befehl.** `tools/wikitool dist upgrade --latest --expect <version>`
|
||||
fragt den Release-Feed, lädt Tarball und `.sha256` in ein Arbeitsverzeichnis, prüft den Tarball
|
||||
gegen die Summe und wendet ihn an; das Arbeitsverzeichnis verschwindet bei jedem Ausgang wieder,
|
||||
auch bei `--dry-run`. `<version>` ist die, die `version notes` gedruckt hat: der Feed kennt nur
|
||||
sein *neuestes* Release, und `--expect` verweigert den Lauf **vor** dem Download, wenn inzwischen
|
||||
ein neueres erschienen ist, statt es ungelesen einzuspielen. Beides zusammen entscheidet vor dem
|
||||
Download über „schon aktuell", Downgrade und Vor-Release (`-beta.N` braucht `--pre`); fehlt dem
|
||||
Release der Tarball oder die Summe, bricht der Befehl vor dem ersten Download ab und nennt die
|
||||
Release-Seite. Wer offline arbeitet oder einen Tarball vom Betreiber bekommen hat, gibt statt
|
||||
`--latest` weiter die Datei an (`dist upgrade <tarball>`).
|
||||
|
||||
**Was die Prüfsumme leistet - und was nicht.** Die `.sha256` liegt beim selben Feed wie der
|
||||
Tarball. Sie schützt vor einer beschädigten Übertragung, nicht vor einem Feed, der selbst
|
||||
kompromittiert ist: die Echtheit eines Releases beruht auf dem Vertrauen in den Host, dessen
|
||||
Feed die Instanz fragt (`update_url` im Stamp). Es gibt keinen https-Zwang; wer einen
|
||||
`http://`-Feed konfiguriert, tut das bewusst. `$WIKITOOL_UPDATE_TOKEN` geht nur an Downloads
|
||||
auf demselben Host wie der Feed.
|
||||
|
||||
**Beim ersten Sprung auf ein Release, das `--latest` kennt, gibt es die Option in der Instanz
|
||||
noch nicht** - die Instruktion, die dort steht, gehört zum Release, das die Instanz verlässt.
|
||||
Dann den Tarball einmal von Hand holen (Weg A oben), prüfen und `dist upgrade <tarball>` geben;
|
||||
ab dem Release danach trägt die Instanz `--latest` selbst.
|
||||
|
||||
**Beim ersten Sprung auf `4.5.0` oder höher gibt es `dist upgrade` in der Instanz noch nicht** -
|
||||
es kam erst mit `4.5.0`. Dann das Werkzeug aus dem entpackten *neuen* Tarball verwenden, gegen die
|
||||
@@ -233,9 +258,9 @@ sind. Einer Instanz, die älter ist als `.wikitool-kb.json`, fehlt die Datei gan
|
||||
**Fallstricke.** Eine Instanz ohne lokale `.wikitool-release.json` (oder eine ohne `files`-Block,
|
||||
aus der Zeit vor `4.5.0`) hat für `dist upgrade` keine Basis, gegen die es eine lokale Änderung
|
||||
erkennen könnte, und verweigert den Tausch - dafür gibt es heute keine Reparatur.
|
||||
Der Befehl lädt selbst nichts herunter: `<tarball-oder-verzeichnis>` muss vorher aus Weg A
|
||||
geholt werden, und ein Tarball muss genau ein Top-Level-Verzeichnis enthalten - die Form, in der
|
||||
`.gitea/workflows/release.yml` es baut.
|
||||
Mit `<tarball-oder-verzeichnis>` lädt der Befehl selbst nichts herunter; die Datei muss vorher
|
||||
aus Weg A geholt werden. Nur `--latest` lädt, und ein Tarball muss in beiden Fällen genau ein
|
||||
Top-Level-Verzeichnis enthalten - die Form, in der `.gitea/workflows/release.yml` es baut.
|
||||
|
||||
Vor `4.5.0` stand hier ein rein manueller Ablauf (Maschinerie von Hand kopieren, `kb/CONTRACT.md`
|
||||
eingeschlossen, sha256-Vergleich von Hand). `dist upgrade` ersetzt genau diesen Teil; wer ihn
|
||||
|
||||
@@ -81,14 +81,25 @@ fresh clone ([bootstrap.md](bootstrap.md)), and not for the clone-with-upstream
|
||||
step 12's, run against the migration documents this instance already has. An `offered` upgrade
|
||||
listed separately blocks nothing and is decided later, in step 12.
|
||||
|
||||
4. **Fetch the tarball and verify it.** `dist upgrade` downloads nothing; the file has to be
|
||||
there already. Take the `.tar.gz` and its `.sha256` from the release page found in step 2 and
|
||||
check them before unpacking. A tarball must unpack to exactly one top-level directory.
|
||||
4. **Nothing to fetch by hand.** `dist upgrade --latest` asks the release feed for the latest
|
||||
release, downloads its `.tar.gz` and `.sha256` into a scratch directory, checks the archive
|
||||
against the checksum and removes both again - all inside the calls of steps 5 to 7. Note the
|
||||
version step 2's `version notes` printed: steps 5 to 7 pass it as `--expect`, so a release that
|
||||
appeared in the meantime is refused before anything is downloaded, rather than applied unread.
|
||||
|
||||
The offline alternative is the tarball path: with the feed unreachable, or an archive the
|
||||
operator supplies, take the `.tar.gz` and its `.sha256` from the release page named in step 2,
|
||||
check the archive against the checksum before unpacking, and pass the file as `<tarball>`
|
||||
where the steps below say `--latest --expect <version>`. A tarball must unpack to exactly one
|
||||
top-level directory. The checksum comes from the same host as the archive, so it catches a
|
||||
damaged transfer, not a compromised host - who is trusted to publish releases is the
|
||||
operator's decision, made before this file starts ([INSTALL.md](../INSTALL.md) § "Version und
|
||||
Updates").
|
||||
|
||||
5. **Dry-run the swap and read all four counts:**
|
||||
|
||||
```bash
|
||||
tools/wikitool dist upgrade <tarball> --dry-run
|
||||
tools/wikitool dist upgrade --latest --expect <version from step 2> --dry-run
|
||||
```
|
||||
|
||||
`unchanged` / `new` / `locally changed` / `removed from the release`. `unchanged` needs no
|
||||
@@ -115,7 +126,7 @@ fresh clone ([bootstrap.md](bootstrap.md)), and not for the clone-with-upstream
|
||||
file reset and another kept. Preview it before it writes:
|
||||
|
||||
```bash
|
||||
tools/wikitool dist upgrade <tarball> --dry-run --take-release <path> [--take-release <path>]
|
||||
tools/wikitool dist upgrade --latest --expect <version from step 2> --dry-run --take-release <path> [--take-release <path>]
|
||||
```
|
||||
|
||||
The preview marks every named path as one it would overwrite from the release, and a path that
|
||||
@@ -134,7 +145,7 @@ fresh clone ([bootstrap.md](bootstrap.md)), and not for the clone-with-upstream
|
||||
|
||||
```bash
|
||||
git rev-parse --short HEAD # the pre-swap commit; keep it
|
||||
tools/wikitool dist upgrade <tarball> [--take-release <path>] [--keep-local]
|
||||
tools/wikitool dist upgrade --latest --expect <version from step 2> [--take-release <path>] [--keep-local]
|
||||
```
|
||||
|
||||
It writes, and commits nothing.
|
||||
|
||||
+22
-6
@@ -2418,7 +2418,7 @@ Apply a stack update `dist export` produced - the write half of `version check`.
|
||||
|
||||
**SYNOPSIS**
|
||||
|
||||
- `wikitool dist upgrade <source> [--dry-run] [--keep-local] [--take-release <path>]... [--prune] [--pre]`
|
||||
- `wikitool dist upgrade (<source> | --latest [--expect <version>]) [--dry-run] [--keep-local] [--take-release <path>]... [--prune] [--pre]`
|
||||
|
||||
**PROPERTIES**
|
||||
|
||||
@@ -2426,23 +2426,30 @@ Apply a stack update `dist export` produced - the write half of `version check`.
|
||||
- idempotent: no
|
||||
- atomic: Yes for every refusal - nothing is written. Once writing starts it is a plain sequential file copy with no partial-state cleanup: an interruption mid-copy (killed process, disk full) can leave the tree part-old, part-new
|
||||
- budget: counted
|
||||
- network: no
|
||||
- network: yes
|
||||
|
||||
**EXAMPLES**
|
||||
|
||||
- `tools/wikitool dist upgrade ../chemenu-7.1.0.tar.gz --dry-run`
|
||||
- `tools/wikitool dist upgrade ../chemenu-7.1.0.tar.gz`
|
||||
- `tools/wikitool dist upgrade ../chemenu-7.1.0.tar.gz --take-release tools/README.md`
|
||||
- `tools/wikitool dist upgrade --latest --expect 8.0.0 --dry-run`
|
||||
|
||||
**EXIT STATUS**
|
||||
|
||||
- 0 success
|
||||
- 0 The source's version equals the installed one - a no-op success
|
||||
- 1 Both `<source>` and `--latest`, or neither; or `--expect` without `--latest`
|
||||
- 1 Local `VERSION` missing, or no local `.wikitool-release.json` with a `files` block
|
||||
- 1 `.wikitool-kb.json` is missing
|
||||
- 1 A migration is already outstanding against the *installed* machinery
|
||||
- 1 The working tree is dirty
|
||||
- 1 `<source>` does not exist, fails its `.sha256`, or does not unpack to exactly one top-level directory
|
||||
- 1 `--latest`: the release feed cannot be reached, or answers with something that is not a release
|
||||
- 1 `--latest --expect`: the feed's latest release is another version than the one expected; nothing was downloaded
|
||||
- 1 `--latest`: the release publishes no archive or no `.sha256` under the expected name; nothing was downloaded
|
||||
- 1 `--latest`: an asset download fails, or the archive fails its `.sha256`
|
||||
- 1 `--latest`: the downloaded archive's `VERSION` is not the version the feed announced
|
||||
- 1 The source carries no `VERSION`, `.wikitool-release.json` or `files` block
|
||||
- 1 The source's version is older than the installed one, or a pre-release without `--pre`
|
||||
- 1 A `--take-release` path this run does not classify as locally changed - the one refusal a `--dry-run` also raises
|
||||
@@ -2450,11 +2457,17 @@ Apply a stack update `dist export` produced - the write half of `version check`.
|
||||
|
||||
**ON FAILURE**
|
||||
|
||||
- Both `<source>` and `--latest`, or neither; or `--expect` without `--latest` -> Not transient - name exactly one source, and pass `--expect` only with `--latest`
|
||||
- Local `VERSION` missing, or no local `.wikitool-release.json` with a `files` block -> Not transient - fix the named precondition and retry. A checkout with shared git history takes stack updates with `wikitool upstream merge` instead
|
||||
- `.wikitool-kb.json` is missing -> Run `wikitool migrate baseline <version>`, then retry
|
||||
- A migration is already outstanding against the *installed* machinery -> Finish it first - `wikitool migrate status` names it - then retry
|
||||
- The working tree is dirty -> Commit or stash first, then retry
|
||||
- `<source>` does not exist, fails its `.sha256`, or does not unpack to exactly one top-level directory -> Fix the path or re-download the release archive, then retry
|
||||
- `--latest`: the release feed cannot be reached, or answers with something that is not a release -> Transient - retry once; then report the URL from the message, or take the release page's archive by hand and pass it as `<source>`
|
||||
- `--latest --expect`: the feed's latest release is another version than the one expected; nothing was downloaded -> Not transient - read `wikitool version notes` for the version the message names, then either expect that one or stop
|
||||
- `--latest`: the release publishes no archive or no `.sha256` under the expected name; nothing was downloaded -> Not transient - the message lists the assets present and the release page; report it there rather than upgrading without the checksum
|
||||
- `--latest`: an asset download fails, or the archive fails its `.sha256` -> Retry once; if it fails again, report the exact message - do not fall back to an unchecked archive
|
||||
- `--latest`: the downloaded archive's `VERSION` is not the version the feed announced -> Not transient - an inconsistent release; report it against the release page
|
||||
- The source carries no `VERSION`, `.wikitool-release.json` or `files` block -> Point `<source>` at a distribution export, then retry
|
||||
- The source's version is older than the installed one, or a pre-release without `--pre` -> Not transient - choose another source, or pass `--pre` for a pre-release
|
||||
- A `--take-release` path this run does not classify as locally changed - the one refusal a `--dry-run` also raises -> Correct it against the locally-changed list the refusal prints, then retry
|
||||
@@ -2466,7 +2479,9 @@ Apply a stack update `dist export` produced - the write half of `version check`.
|
||||
|
||||
**NOTES**
|
||||
|
||||
- Never downloads anything: `<source>` is an already-fetched export directory or `.tar.gz` release archive. An archive is verified against a sibling `.sha256` if one is present (a missing one is a WARN, not a block) and must unpack to exactly one top-level directory - the shape `.gitea/workflows/release.yml` packs.
|
||||
- Exactly one of `<source>` and `--latest` names the release. `<source>` is an already-fetched export directory or `.tar.gz` release archive and touches no network: the archive is verified against a sibling `.sha256` if one is present (a missing one is a WARN, not a block) and must unpack to exactly one top-level directory - the shape `.gitea/workflows/release.yml` packs.
|
||||
- `--latest` asks the release feed (`update_url` of the local stamp, overridable with `$WIKITOOL_UPDATE_URL`; `$WIKITOOL_UPDATE_TOKEN` is sent along) which release is latest, checks that version - `--expect`, a downgrade, a pre-release without `--pre`, already installed - before any download, then downloads the archive and its `.sha256` into a scratch directory removed on every exit. The checksum is mandatory here (a release without one, or an archive that fails it, is an error, not a WARN) and the archive's own `VERSION` must equal the feed's version. The asset URLs are the feed's own `browser_download_url` values, and the token reaches an asset download only if it is on the feed's origin. The checksum protects against transfer errors, not against a feed that is itself compromised - authenticity is the trust in the feed's host.
|
||||
- `--expect <version>` (only with `--latest`) pins the release the feed may announce: a different latest version is refused before anything is downloaded. Pass the version `wikitool version notes` was read for, in the dry run and in the real run alike.
|
||||
- The write set is exactly the *new* `.wikitool-release.json`'s `files` block, minus what an export re-seeds from a blank template every time (`kb/log.md`, `raw/.gitkeep`) or seeds once and the instance owns from then on (`.wikitool-kb.json`, `CHANGES.md`), plus the stamp itself, always rewritten.
|
||||
- Every candidate path is classified against the *local* `.wikitool-release.json`'s recorded digest: unchanged is overwritten silently, absent from the old stamp is created, and locally modified or locally deleted is **never** silently overwritten.
|
||||
- Locally changed files abort the run with the full list; the abort text names the three answers with the command line filled in, and none of them is the default.
|
||||
@@ -2478,13 +2493,14 @@ Apply a stack update `dist export` produced - the write half of `version check`.
|
||||
- Reports, but does not block on, a crossed compatibility boundary.
|
||||
- The local preconditions - `VERSION`, the release stamp, `.wikitool-kb.json`, no outstanding migration, a clean working tree - are checked before the source is read. Not being a git repository at all is a WARN, not a refusal.
|
||||
- Never touches git - no commit, no push.
|
||||
- `--dry-run` classifies and reports without writing; a pre-release (`-beta.N`) source needs `--pre`.
|
||||
- `--dry-run` classifies and reports without writing; a pre-release (`-beta.N`) source needs `--pre`. With `--latest` it still downloads and verifies the archive - that is the only way to classify - and removes it again.
|
||||
- An interrupted write is not resumed automatically: compare the tree against the printed classification and finish or revert by hand.
|
||||
- The closing report names `instructions/upgrade-instance.md`, which carries the order for everything after the swap and resumes at `instructions sync`.
|
||||
|
||||
**SEE ALSO**
|
||||
|
||||
- `wikitool version check` - finds out whether an update exists
|
||||
- `wikitool version notes` - the notes of the release `--expect` should name
|
||||
- `instructions/upgrade-instance.md` - the order after the swap
|
||||
- `INSTALL.md` § "Version und Updates" - which release, whether to take it, where the tarball comes from
|
||||
- `wikitool upstream merge` - the update path for a checkout with shared git history
|
||||
@@ -2571,8 +2587,8 @@ Ask the origin's release feed whether a newer stack exists.
|
||||
**NOTES**
|
||||
|
||||
- Asks the release feed for its latest release and compares it with `VERSION`: `state` is `current`, `update`, `migration` (the step crosses a compatibility boundary) or `ahead`.
|
||||
- The only command whose whole job is the network call - `version notes` reaches the same feed too, but only as a fallback on a distributed instance.
|
||||
- Never reached implicitly from another command, needs no key, and times out after `--timeout` seconds (default 10).
|
||||
- The only command whose whole job is the network call - `version notes` reaches the same feed too, but only as a fallback on a distributed instance, and `dist upgrade --latest` asks it which release to download.
|
||||
- Never reached implicitly from another command (`dist upgrade` asks only when passed `--latest`), needs no key, and times out after `--timeout` seconds (default 10).
|
||||
- The feed is `--url`, else `$WIKITOOL_UPDATE_URL`, else the release stamp's, else the built-in origin. `$WIKITOOL_UPDATE_TOKEN` is only needed if that feed is not readable anonymously.
|
||||
- For `update` or `migration` it prints that applying the release is a separate, manual step (`INSTALL.md` § "Eine Instanz aktualisieren").
|
||||
- Read-only; exempt from the Iteration Budget Gate.
|
||||
|
||||
@@ -736,8 +736,8 @@ def run_export(target: Path, dry_run: bool = False, origin: Optional[Origin] = N
|
||||
# modified/deleted file is never silently overwritten: the run aborts unless
|
||||
# `--keep-local` keeps it or `--take-release <path>` names it, which is the
|
||||
# difference between a file the instance means to carry and one that drifted.
|
||||
# This never calls a release feed; the caller supplies an already-downloaded
|
||||
# tree or archive.
|
||||
# The classification never touches the network: the tree or archive it works
|
||||
# on is either supplied by the caller or fetched by `--latest` beforehand.
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
@@ -867,6 +867,87 @@ def _resolved_source(source: Path):
|
||||
yield _extract_single_top_level_dir(source, Path(tmp))
|
||||
|
||||
|
||||
def _version_gate(
|
||||
new_version: "version_mod.Version",
|
||||
local_version: "version_mod.Version",
|
||||
allow_pre: bool,
|
||||
origin: str,
|
||||
) -> bool:
|
||||
"""The three version refusals shared by a source tree and the feed: a
|
||||
candidate without `--pre`, a downgrade, an equal version. `True` means go
|
||||
on; `False` means the run ended successfully as a no-op. `origin` names
|
||||
what reported the version, for the message."""
|
||||
if new_version.is_prerelease and not allow_pre:
|
||||
fail(
|
||||
f"{origin} reports {new_version}, a running candidate (-beta.N). "
|
||||
"`.gitea/workflows/release.yml` never publishes one, so it can only come from a "
|
||||
"dev checkout or a hand-made release - pass --pre if that is deliberate."
|
||||
)
|
||||
if new_version < local_version:
|
||||
fail(f"{origin} reports {new_version}, older than the installed {local_version} - refusing a downgrade.")
|
||||
if new_version == local_version:
|
||||
success(f"Already at {local_version}. Nothing to do.")
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def _query_feed(
|
||||
stamp: dict,
|
||||
local_version: "version_mod.Version",
|
||||
allow_pre: bool,
|
||||
expect: Optional["version_mod.Version"],
|
||||
) -> Optional[tuple["version_mod.LatestRelease", str, Optional[str]]]:
|
||||
"""Ask the release feed which release is latest and decide, before any
|
||||
download, whether there is anything to fetch. Returns the release with the
|
||||
feed URL and token, or `None` when the installed version is already it."""
|
||||
url = version_mod.update_url(stamp)
|
||||
token = os.environ.get(version_mod.UPDATE_TOKEN_ENV, "").strip() or None
|
||||
console.print(f"Asking the release feed {url}", highlight=False)
|
||||
try:
|
||||
release = version_mod.fetch_latest_assets(url, token)
|
||||
except version_mod.VersionError as exc:
|
||||
fail(str(exc))
|
||||
if expect is not None and release.version != expect:
|
||||
fail(
|
||||
f"The feed's latest release is {release.version}, not the expected {expect} - "
|
||||
"nothing was downloaded. The feed moved on since the notes were read, or "
|
||||
f"the wrong version was named: read `wikitool version notes` for {release.version} "
|
||||
"before deciding to go there instead."
|
||||
)
|
||||
if not _version_gate(release.version, local_version, allow_pre, "the release feed"):
|
||||
return None
|
||||
return release, url, token
|
||||
|
||||
|
||||
@contextmanager
|
||||
def _downloaded_source(release: "version_mod.LatestRelease", feed_url: str, token: Optional[str]):
|
||||
"""Yield the directory of the release the feed announced: download its
|
||||
archive and checksum into a scratch directory, hold the archive to the
|
||||
checksum (a missing checksum is an error, never a WARN), and unpack it.
|
||||
A release without both assets is refused before the first download. The
|
||||
scratch directory is removed on every exit, `--dry-run` included."""
|
||||
archive_url, sum_url = _asset_urls(release)
|
||||
with tempfile.TemporaryDirectory(prefix="wikitool-upgrade-") as tmp:
|
||||
archive = Path(tmp) / release.archive_name
|
||||
checksum = archive.with_name(archive.name + version_mod.CHECKSUM_SUFFIX)
|
||||
for url, dest in ((archive_url, archive), (sum_url, checksum)):
|
||||
console.print(f"Downloading {url}", highlight=False)
|
||||
try:
|
||||
version_mod.download_asset(url, dest, version_mod.token_for_asset(feed_url, url, token))
|
||||
except version_mod.VersionError as exc:
|
||||
fail(str(exc))
|
||||
_verify_sha256_sidecar(archive)
|
||||
yield _extract_single_top_level_dir(archive, Path(tmp) / "unpacked")
|
||||
|
||||
|
||||
def _asset_urls(release: "version_mod.LatestRelease") -> tuple[str, str]:
|
||||
try:
|
||||
return release.asset_urls()
|
||||
except version_mod.VersionError as exc:
|
||||
fail(str(exc))
|
||||
raise # unreachable: fail() raises typer.Exit
|
||||
|
||||
|
||||
def _git_working_tree_status() -> Optional[str]:
|
||||
"""`git status --porcelain` for `config.ROOT`, or None if it is not a git
|
||||
repository at all - which is a valid, if unprotected, state for a tarball
|
||||
@@ -907,7 +988,7 @@ def _resolve_take_release(
|
||||
|
||||
|
||||
def _refusal_for_blocked(
|
||||
source: Path, undecided: list[str], classification: FileClassification
|
||||
target: str, undecided: list[str], classification: FileClassification
|
||||
) -> str:
|
||||
"""The abort text for blocked paths no flag has answered for.
|
||||
|
||||
@@ -927,9 +1008,9 @@ def _refusal_for_blocked(
|
||||
f"{len(undecided)} locally changed file(s) (listed above) would be silently "
|
||||
f"overwritten. Nothing was written, and none of these three is the default:\n"
|
||||
f" - take the release's version and discard the local change:\n"
|
||||
f" dist upgrade {rel_path(source)} --take-release {paths}\n"
|
||||
f" dist upgrade {target} --take-release {paths}\n"
|
||||
f" - keep every local change and upgrade around them ({kept_again}):\n"
|
||||
f" dist upgrade {rel_path(source)} --keep-local\n"
|
||||
f" dist upgrade {target} --keep-local\n"
|
||||
f" - reconcile them by hand first, then re-run."
|
||||
)
|
||||
|
||||
@@ -984,8 +1065,8 @@ def _report_plan(
|
||||
path="dist upgrade",
|
||||
summary="Apply a stack update `dist export` produced - the write half of `version check`.",
|
||||
synopsis=(cli_contract.Variant(
|
||||
usage="dist upgrade <source> [--dry-run] [--keep-local] [--take-release <path>]... "
|
||||
"[--prune] [--pre]",
|
||||
usage="dist upgrade (<source> | --latest [--expect <version>]) [--dry-run] "
|
||||
"[--keep-local] [--take-release <path>]... [--prune] [--pre]",
|
||||
),),
|
||||
properties=cli_contract.Properties(
|
||||
effect=cli_contract.Effect.WRITE,
|
||||
@@ -994,12 +1075,28 @@ def _report_plan(
|
||||
"sequential file copy with no partial-state cleanup: an interruption mid-copy "
|
||||
"(killed process, disk full) can leave the tree part-old, part-new",
|
||||
budget=cli_contract.Budget.COUNTED,
|
||||
network=cli_contract.Network.YES,
|
||||
),
|
||||
notes=(
|
||||
"Never downloads anything: `<source>` is an already-fetched export directory or "
|
||||
"`.tar.gz` release archive. An archive is verified against a sibling `.sha256` if one "
|
||||
"is present (a missing one is a WARN, not a block) and must unpack to exactly one "
|
||||
"top-level directory - the shape `.gitea/workflows/release.yml` packs.",
|
||||
"Exactly one of `<source>` and `--latest` names the release. `<source>` is an "
|
||||
"already-fetched export directory or `.tar.gz` release archive and touches no network: "
|
||||
"the archive is verified against a sibling `.sha256` if one is present (a missing one "
|
||||
"is a WARN, not a block) and must unpack to exactly one top-level directory - the "
|
||||
"shape `.gitea/workflows/release.yml` packs.",
|
||||
"`--latest` asks the release feed (`update_url` of the local stamp, overridable with "
|
||||
"`$WIKITOOL_UPDATE_URL`; `$WIKITOOL_UPDATE_TOKEN` is sent along) which release is "
|
||||
"latest, checks that version - `--expect`, a downgrade, a pre-release without `--pre`, "
|
||||
"already installed - before any download, then downloads the archive and its "
|
||||
"`.sha256` into a scratch directory removed on every exit. The checksum is mandatory "
|
||||
"here (a release without one, or an archive that fails it, is an error, not a WARN) "
|
||||
"and the archive's own `VERSION` must equal the feed's version. The asset URLs are the "
|
||||
"feed's own `browser_download_url` values, and the token reaches an asset download only "
|
||||
"if it is on the feed's origin. The checksum protects against transfer errors, not "
|
||||
"against a feed that is itself compromised - authenticity is the trust in the feed's "
|
||||
"host.",
|
||||
"`--expect <version>` (only with `--latest`) pins the release the feed may announce: a "
|
||||
"different latest version is refused before anything is downloaded. Pass the version "
|
||||
"`wikitool version notes` was read for, in the dry run and in the real run alike.",
|
||||
"The write set is exactly the *new* `.wikitool-release.json`'s `files` block, minus "
|
||||
"what an export re-seeds from a blank template every time (`kb/log.md`, "
|
||||
"`raw/.gitkeep`) or seeds once and the instance owns from then on "
|
||||
@@ -1029,7 +1126,8 @@ def _report_plan(
|
||||
"Not being a git repository at all is a WARN, not a refusal.",
|
||||
"Never touches git - no commit, no push.",
|
||||
"`--dry-run` classifies and reports without writing; a pre-release (`-beta.N`) source "
|
||||
"needs `--pre`.",
|
||||
"needs `--pre`. With `--latest` it still downloads and verifies the archive - that is "
|
||||
"the only way to classify - and removes it again.",
|
||||
"An interrupted write is not resumed automatically: compare the tree against the "
|
||||
"printed classification and finish or revert by hand.",
|
||||
"The closing report names `instructions/upgrade-instance.md`, which carries the order "
|
||||
@@ -1041,6 +1139,11 @@ def _report_plan(
|
||||
reaction="",
|
||||
code=0,
|
||||
),
|
||||
cli_contract.Failure(
|
||||
cause="Both `<source>` and `--latest`, or neither; or `--expect` without `--latest`",
|
||||
reaction="Not transient - name exactly one source, and pass `--expect` only with "
|
||||
"`--latest`",
|
||||
),
|
||||
cli_contract.Failure(
|
||||
cause="Local `VERSION` missing, or no local `.wikitool-release.json` with a `files` "
|
||||
"block",
|
||||
@@ -1064,6 +1167,34 @@ def _report_plan(
|
||||
"one top-level directory",
|
||||
reaction="Fix the path or re-download the release archive, then retry",
|
||||
),
|
||||
cli_contract.Failure(
|
||||
cause="`--latest`: the release feed cannot be reached, or answers with something "
|
||||
"that is not a release",
|
||||
reaction="Transient - retry once; then report the URL from the message, or take the "
|
||||
"release page's archive by hand and pass it as `<source>`",
|
||||
),
|
||||
cli_contract.Failure(
|
||||
cause="`--latest --expect`: the feed's latest release is another version than the "
|
||||
"one expected; nothing was downloaded",
|
||||
reaction="Not transient - read `wikitool version notes` for the version the message "
|
||||
"names, then either expect that one or stop",
|
||||
),
|
||||
cli_contract.Failure(
|
||||
cause="`--latest`: the release publishes no archive or no `.sha256` under the "
|
||||
"expected name; nothing was downloaded",
|
||||
reaction="Not transient - the message lists the assets present and the release page; "
|
||||
"report it there rather than upgrading without the checksum",
|
||||
),
|
||||
cli_contract.Failure(
|
||||
cause="`--latest`: an asset download fails, or the archive fails its `.sha256`",
|
||||
reaction="Retry once; if it fails again, report the exact message - do not fall back "
|
||||
"to an unchecked archive",
|
||||
),
|
||||
cli_contract.Failure(
|
||||
cause="`--latest`: the downloaded archive's `VERSION` is not the version the feed "
|
||||
"announced",
|
||||
reaction="Not transient - an inconsistent release; report it against the release page",
|
||||
),
|
||||
cli_contract.Failure(
|
||||
cause="The source carries no `VERSION`, `.wikitool-release.json` or `files` block",
|
||||
reaction="Point `<source>` at a distribution export, then retry",
|
||||
@@ -1091,12 +1222,14 @@ def _report_plan(
|
||||
"tools/wikitool dist upgrade ../chemenu-7.1.0.tar.gz --dry-run",
|
||||
"tools/wikitool dist upgrade ../chemenu-7.1.0.tar.gz",
|
||||
"tools/wikitool dist upgrade ../chemenu-7.1.0.tar.gz --take-release tools/README.md",
|
||||
"tools/wikitool dist upgrade --latest --expect 8.0.0 --dry-run",
|
||||
),
|
||||
never=(
|
||||
"Never treat any of the three answers to locally changed files as the default.",
|
||||
),
|
||||
see_also=(
|
||||
"`wikitool version check` - finds out whether an update exists",
|
||||
"`wikitool version notes` - the notes of the release `--expect` should name",
|
||||
"`instructions/upgrade-instance.md` - the order after the swap",
|
||||
"`INSTALL.md` § \"Version und Updates\" - which release, whether to take it, where "
|
||||
"the tarball comes from",
|
||||
@@ -1106,8 +1239,17 @@ def _report_plan(
|
||||
))
|
||||
@app.command("upgrade")
|
||||
def upgrade_command(
|
||||
source: Path = typer.Argument(
|
||||
..., help="An extracted distribution directory, or a release .tar.gz archive"
|
||||
source: Optional[Path] = typer.Argument(
|
||||
None, help="An extracted distribution directory, or a release .tar.gz archive - or use --latest"
|
||||
),
|
||||
latest: bool = typer.Option(
|
||||
False, "--latest",
|
||||
help="Download the release the feed announces as latest instead of naming a <source>",
|
||||
),
|
||||
expect: Optional[str] = typer.Option(
|
||||
None, "--expect",
|
||||
help="With --latest: refuse, before downloading anything, unless the feed's latest release "
|
||||
"is this version (the one `version notes` was read for)",
|
||||
),
|
||||
dry_run: bool = typer.Option(
|
||||
False, "--dry-run", help="Classify and report, without writing anything"
|
||||
@@ -1133,8 +1275,10 @@ def upgrade_command(
|
||||
):
|
||||
"""Apply a stack update `dist export` produced - the write half of `version check`.
|
||||
\f
|
||||
Never downloads anything: `source` is an already-fetched
|
||||
export directory or `.tar.gz` archive. Writes exactly the new release
|
||||
`source` is an already-fetched export directory or `.tar.gz` archive;
|
||||
`--latest` instead downloads the archive and its checksum from the
|
||||
release feed (`--expect <version>` pins which release that may be) and
|
||||
holds it to the checksum before anything else. Writes exactly the new release
|
||||
stamp's `files` block, minus what an export re-seeds every time
|
||||
(`kb/log.md`, `raw/.gitkeep`) or seeds once and the instance owns from
|
||||
then on (`.wikitool-kb.json`, `CHANGES.md`), classifying every candidate
|
||||
@@ -1155,19 +1299,41 @@ def upgrade_command(
|
||||
take_release=take_release,
|
||||
prune=prune,
|
||||
allow_pre=allow_pre,
|
||||
latest=latest,
|
||||
expect=expect,
|
||||
)
|
||||
|
||||
|
||||
def run_upgrade(
|
||||
source: Path,
|
||||
source: Optional[Path] = None,
|
||||
dry_run: bool = False,
|
||||
keep_local: bool = False,
|
||||
take_release: Optional[Sequence[str]] = None,
|
||||
prune: bool = False,
|
||||
allow_pre: bool = False,
|
||||
latest: bool = False,
|
||||
expect: Optional[str] = None,
|
||||
) -> None:
|
||||
"""The upgrade itself, free of Typer's option objects - see `run_export`
|
||||
for why this split exists."""
|
||||
if latest == (source is not None):
|
||||
fail(
|
||||
"Name exactly one source: `<source>` (an export directory or a release .tar.gz "
|
||||
"already on disk) or `--latest` (the release the feed announces). "
|
||||
+ ("Both were given." if latest else "Neither was given.")
|
||||
)
|
||||
return
|
||||
expected_version = None
|
||||
if expect is not None:
|
||||
if not latest:
|
||||
fail("--expect only makes sense with --latest: it pins the release the feed may announce.")
|
||||
return
|
||||
try:
|
||||
expected_version = version_mod.Version.parse(expect)
|
||||
except version_mod.VersionError as exc:
|
||||
fail(f"--expect: {exc}")
|
||||
return
|
||||
|
||||
try:
|
||||
local_version = version_mod.read_version()
|
||||
except version_mod.VersionError as exc:
|
||||
@@ -1223,7 +1389,19 @@ def run_upgrade(
|
||||
)
|
||||
return
|
||||
|
||||
with _resolved_source(source) as new_root:
|
||||
feed = None
|
||||
if latest:
|
||||
feed = _query_feed(old_stamp, local_version, allow_pre, expected_version)
|
||||
if feed is None:
|
||||
return
|
||||
release, feed_url, feed_token = feed
|
||||
source_cm = _downloaded_source(release, feed_url, feed_token)
|
||||
target = f"--latest --expect {release.version}"
|
||||
else:
|
||||
source_cm = _resolved_source(source)
|
||||
target = rel_path(source)
|
||||
|
||||
with source_cm as new_root:
|
||||
version_path = new_root / version_mod.VERSION_FILENAME
|
||||
if not version_path.is_file():
|
||||
fail(f"{rel_path(new_root)} has no VERSION - not a distribution export.")
|
||||
@@ -1234,18 +1412,16 @@ def run_upgrade(
|
||||
fail(str(exc))
|
||||
return
|
||||
|
||||
if new_version.is_prerelease and not allow_pre:
|
||||
if feed is not None and new_version != feed[0].version:
|
||||
fail(
|
||||
f"{new_version} is a running candidate (-beta.N). `.gitea/workflows/release.yml` "
|
||||
"never publishes one, so a candidate tree can only come from a dev checkout by "
|
||||
"hand - pass --pre if that is deliberate."
|
||||
f"The downloaded archive's VERSION says {new_version}, but the feed announced "
|
||||
f"{feed[0].version} (tag {feed[0].tag}) - this release is inconsistent, so "
|
||||
"nothing was applied. Report it against the release page"
|
||||
+ (f" {feed[0].html_url}" if feed[0].html_url else "")
|
||||
+ "."
|
||||
)
|
||||
return
|
||||
if new_version < local_version:
|
||||
fail(f"{new_version} is older than the installed {local_version} - refusing a downgrade.")
|
||||
return
|
||||
if new_version == local_version:
|
||||
success(f"Already at {local_version}. Nothing to do.")
|
||||
if not _version_gate(new_version, local_version, allow_pre, "the source"):
|
||||
return
|
||||
|
||||
stamp_path = new_root / version_mod.RELEASE_STAMP_FILENAME
|
||||
@@ -1289,7 +1465,7 @@ def run_upgrade(
|
||||
|
||||
undecided = [path for path in classification.blocked if path not in taken]
|
||||
if undecided and not keep_local:
|
||||
fail(_refusal_for_blocked(source, undecided, classification))
|
||||
fail(_refusal_for_blocked(target, undecided, classification))
|
||||
return
|
||||
|
||||
to_write = sorted(classification.unchanged + classification.new + sorted(taken))
|
||||
|
||||
@@ -29,6 +29,9 @@ number means, and `instructions/dev/version-parts.md` for the candidate model):
|
||||
stderr before asking and takes `--offline`. Both need no key, both time
|
||||
out, and a feed that cannot be reached is reported as an error rather than
|
||||
silently answered as "up to date" or "no notes".
|
||||
- `dist upgrade --latest` reaches the same feed as a third, equally explicit
|
||||
caller: it asks which release is latest, then downloads that release's
|
||||
assets. Only `version_mod` talks to the feed; this module is not involved.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
@@ -158,8 +161,10 @@ def show_command(
|
||||
"is `current`, `update`, `migration` (the step crosses a compatibility boundary) or "
|
||||
"`ahead`.",
|
||||
"The only command whose whole job is the network call - `version notes` reaches the "
|
||||
"same feed too, but only as a fallback on a distributed instance.",
|
||||
"Never reached implicitly from another command, needs no key, and times out after "
|
||||
"same feed too, but only as a fallback on a distributed instance, and `dist upgrade "
|
||||
"--latest` asks it which release to download.",
|
||||
"Never reached implicitly from another command (`dist upgrade` asks only when passed "
|
||||
"`--latest`), needs no key, and times out after "
|
||||
"`--timeout` seconds (default 10).",
|
||||
"The feed is `--url`, else `$WIKITOOL_UPDATE_URL`, else the release stamp's, else the "
|
||||
"built-in origin. `$WIKITOOL_UPDATE_TOKEN` is only needed if that feed is not readable "
|
||||
|
||||
@@ -337,7 +337,8 @@ def test_every_gated_record_shows_its_re_run_after_exit_42():
|
||||
def test_network_yes_is_exactly_the_commands_that_can_reach_outside_this_checkout():
|
||||
"""Gitea #144: `network:` means *any* reach outside this checkout - an HTTP call
|
||||
`wikitool` makes itself, or a git operation against a remote (fetch/ls-remote/push) -
|
||||
not only the two `version_cmd.py` used to claim exclusivity for. Pinned as an explicit
|
||||
not only the two `version_cmd.py` used to claim exclusivity for. `dist upgrade` is on the list
|
||||
since `--latest`, which asks the release feed and downloads from it. Pinned as an explicit
|
||||
set so a command gaining or losing that reach is a deliberate edit here, not a silent
|
||||
drift between the property and what the command actually does."""
|
||||
expected = {
|
||||
@@ -346,6 +347,7 @@ def test_network_yes_is_exactly_the_commands_that_can_reach_outside_this_checkou
|
||||
"upstream merge",
|
||||
"version check",
|
||||
"version notes",
|
||||
"dist upgrade",
|
||||
"review",
|
||||
"task new",
|
||||
"task list",
|
||||
|
||||
@@ -4,9 +4,13 @@ execution, and every refusal before anything is written. See Gitea #7."""
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import http.server
|
||||
import json
|
||||
import shutil
|
||||
import subprocess
|
||||
import tarfile
|
||||
import tempfile
|
||||
import threading
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
@@ -586,3 +590,426 @@ def test_tarball_sha256_sidecar_mismatch_is_refused(instance, tmp_path):
|
||||
|
||||
with pytest.raises(typer.Exit):
|
||||
dist_cmd.run_upgrade(archive)
|
||||
|
||||
|
||||
# --- --latest: the release feed ---------------------------------------------
|
||||
#
|
||||
# A real `http.server` on 127.0.0.1 rather than a fetcher stub: the download
|
||||
# path is `urllib` streaming into a file, and the same-origin token rule is a
|
||||
# statement about which requests carry which header - both only observable on
|
||||
# the wire. Every server records `(path, Authorization)` for each request.
|
||||
|
||||
|
||||
class FeedServer:
|
||||
def __init__(self) -> None:
|
||||
self.routes: dict[str, tuple[int, bytes]] = {}
|
||||
self.requests: list[tuple[str, str | None]] = []
|
||||
outer = self
|
||||
|
||||
class Handler(http.server.BaseHTTPRequestHandler):
|
||||
def do_GET(self): # noqa: N802 - http.server's name
|
||||
outer.requests.append((self.path, self.headers.get("Authorization")))
|
||||
status, body = outer.routes.get(self.path, (404, b"not found"))
|
||||
self.send_response(status)
|
||||
self.send_header("Content-Length", str(len(body)))
|
||||
self.end_headers()
|
||||
self.wfile.write(body)
|
||||
|
||||
def log_message(self, *args): # silence the test output
|
||||
pass
|
||||
|
||||
self.httpd = http.server.HTTPServer(("127.0.0.1", 0), Handler)
|
||||
self.thread = threading.Thread(target=self.httpd.serve_forever, kwargs={"poll_interval": 0.01}, daemon=True)
|
||||
self.thread.start()
|
||||
|
||||
@property
|
||||
def base(self) -> str:
|
||||
return f"http://127.0.0.1:{self.httpd.server_address[1]}"
|
||||
|
||||
def stop(self) -> None:
|
||||
self.httpd.shutdown()
|
||||
self.httpd.server_close()
|
||||
|
||||
def paths(self) -> list[str]:
|
||||
return [path for path, _ in self.requests]
|
||||
|
||||
def asset_paths(self) -> list[str]:
|
||||
return [path for path in self.paths() if path.startswith("/assets/")]
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def feed(monkeypatch: pytest.MonkeyPatch):
|
||||
server = FeedServer()
|
||||
monkeypatch.setenv("WIKITOOL_UPDATE_URL", f"{server.base}/feed")
|
||||
yield server
|
||||
server.stop()
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def other_origin():
|
||||
server = FeedServer()
|
||||
yield server
|
||||
server.stop()
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def scratch_tmp(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Path:
|
||||
"""Redirect `tempfile` into a directory the test can inspect afterwards."""
|
||||
scratch = tmp_path / "scratch-tmp"
|
||||
scratch.mkdir()
|
||||
monkeypatch.setattr(tempfile, "tempdir", str(scratch))
|
||||
return scratch
|
||||
|
||||
|
||||
def _publish(
|
||||
feed: FeedServer,
|
||||
tmp_path: Path,
|
||||
version: str,
|
||||
files: dict[str, str] | None = None,
|
||||
*,
|
||||
tag: str | None = None,
|
||||
inner_version: str | None = None,
|
||||
checksum: str | None = "ok",
|
||||
archive: bool = True,
|
||||
asset_server: FeedServer | None = None,
|
||||
) -> str:
|
||||
"""Serve `version` as the feed's latest release and return its archive name.
|
||||
|
||||
`inner_version` is what the archive's own VERSION says (default: the same),
|
||||
`checksum` is `"ok"`, `"wrong"` or `None` for no `.sha256` asset."""
|
||||
name = f"chemenu-stack-{version}"
|
||||
tree = _release(
|
||||
tmp_path, f"served-{name}", inner_version or version,
|
||||
files or {"AGENTS.md": "core v2\n", "tools/wikitool": "#!/bin/sh\n"},
|
||||
)
|
||||
packed = tmp_path / f"{name}.tar.gz"
|
||||
with tarfile.open(packed, "w:gz") as tf:
|
||||
tf.add(tree, arcname=name)
|
||||
data = packed.read_bytes()
|
||||
digest = hashlib.sha256(data).hexdigest() if checksum == "ok" else "0" * 64
|
||||
holder = asset_server or feed
|
||||
assets = []
|
||||
if archive:
|
||||
holder.routes[f"/assets/{name}.tar.gz"] = (200, data)
|
||||
assets.append({"name": f"{name}.tar.gz", "browser_download_url": f"{holder.base}/assets/{name}.tar.gz"})
|
||||
if checksum is not None:
|
||||
holder.routes[f"/assets/{name}.tar.gz.sha256"] = (200, f"{digest} {name}.tar.gz\n".encode())
|
||||
assets.append({
|
||||
"name": f"{name}.tar.gz.sha256",
|
||||
"browser_download_url": f"{holder.base}/assets/{name}.tar.gz.sha256",
|
||||
})
|
||||
feed.routes["/feed"] = (
|
||||
200,
|
||||
json.dumps({
|
||||
"tag_name": tag or f"v{version}",
|
||||
"html_url": f"{feed.base}/releases/tag/v{version}",
|
||||
"assets": assets,
|
||||
}).encode(),
|
||||
)
|
||||
return name
|
||||
|
||||
|
||||
def _stamp_version(instance: Path) -> str:
|
||||
return json.loads((instance / version_mod.RELEASE_STAMP_FILENAME).read_text())["version"]
|
||||
|
||||
|
||||
def test_latest_downloads_verifies_and_applies_the_feeds_release(instance, tmp_path, feed, scratch_tmp):
|
||||
name = _publish(feed, tmp_path, "1.1.0")
|
||||
|
||||
dist_cmd.run_upgrade(latest=True)
|
||||
|
||||
assert (instance / "AGENTS.md").read_text(encoding="utf-8") == "core v2\n"
|
||||
assert _stamp_version(instance) == "1.1.0"
|
||||
assert feed.paths() == ["/feed", f"/assets/{name}.tar.gz", f"/assets/{name}.tar.gz.sha256"]
|
||||
assert list(scratch_tmp.iterdir()) == []
|
||||
recorded = json.loads((instance / version_mod.RELEASE_STAMP_FILENAME).read_text())["files"]
|
||||
assert recorded
|
||||
for relative, digest in recorded.items():
|
||||
assert _digest((instance / relative).read_text(encoding="utf-8")) == digest
|
||||
|
||||
|
||||
def test_latest_yields_the_same_tree_as_the_archive_it_downloaded(instance, tmp_path, feed, monkeypatch):
|
||||
name = _publish(feed, tmp_path, "1.1.0")
|
||||
served = tmp_path / f"{name}.tar.gz"
|
||||
snapshot = tmp_path / "instance-copy"
|
||||
shutil.copytree(instance, snapshot)
|
||||
|
||||
dist_cmd.run_upgrade(latest=True)
|
||||
via_feed = {p.relative_to(instance): p.read_bytes() for p in instance.rglob("*") if p.is_file()}
|
||||
|
||||
monkeypatch.setattr(config, "ROOT", snapshot)
|
||||
dist_cmd.run_upgrade(served)
|
||||
via_archive = {p.relative_to(snapshot): p.read_bytes() for p in snapshot.rglob("*") if p.is_file()}
|
||||
|
||||
assert via_feed == via_archive
|
||||
|
||||
|
||||
def test_latest_dry_run_downloads_writes_nothing_and_cleans_up(instance, tmp_path, feed, scratch_tmp):
|
||||
name = _publish(feed, tmp_path, "1.1.0")
|
||||
|
||||
dist_cmd.run_upgrade(latest=True, dry_run=True)
|
||||
|
||||
assert (instance / "AGENTS.md").read_text(encoding="utf-8") == "core\n"
|
||||
assert _stamp_version(instance) == "1.0.0"
|
||||
assert f"/assets/{name}.tar.gz" in feed.paths()
|
||||
assert list(scratch_tmp.iterdir()) == []
|
||||
|
||||
|
||||
def test_latest_already_current_is_a_no_op_without_any_download(instance, tmp_path, feed):
|
||||
_publish(feed, tmp_path, "1.0.0")
|
||||
|
||||
dist_cmd.run_upgrade(latest=True)
|
||||
|
||||
assert feed.paths() == ["/feed"]
|
||||
assert _stamp_version(instance) == "1.0.0"
|
||||
|
||||
|
||||
def test_latest_older_than_installed_is_a_downgrade_refusal_before_any_download(instance, tmp_path, feed, capsys):
|
||||
_publish(feed, tmp_path, "0.9.0")
|
||||
|
||||
with pytest.raises(typer.Exit) as excinfo:
|
||||
dist_cmd.run_upgrade(latest=True)
|
||||
|
||||
assert excinfo.value.exit_code == 1
|
||||
assert "downgrade" in capsys.readouterr().out
|
||||
assert feed.asset_paths() == []
|
||||
|
||||
|
||||
def test_latest_prerelease_needs_pre_and_is_refused_before_any_download(instance, tmp_path, feed):
|
||||
_publish(feed, tmp_path, "1.1.0-beta.1")
|
||||
|
||||
with pytest.raises(typer.Exit):
|
||||
dist_cmd.run_upgrade(latest=True)
|
||||
assert feed.asset_paths() == []
|
||||
|
||||
dist_cmd.run_upgrade(latest=True, allow_pre=True)
|
||||
assert _stamp_version(instance) == "1.1.0-beta.1"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("missing", ["checksum", "archive"])
|
||||
def test_latest_release_without_an_asset_is_refused_before_any_download(
|
||||
instance, tmp_path, feed, capsys, missing
|
||||
):
|
||||
_publish(
|
||||
feed, tmp_path, "1.1.0",
|
||||
checksum=None if missing == "checksum" else "ok",
|
||||
archive=missing != "archive",
|
||||
)
|
||||
|
||||
with pytest.raises(typer.Exit) as excinfo:
|
||||
dist_cmd.run_upgrade(latest=True)
|
||||
|
||||
assert excinfo.value.exit_code == 1
|
||||
out = " ".join(capsys.readouterr().out.split())
|
||||
assert "chemenu-stack-1.1.0.tar.gz" in out
|
||||
assert "assets present" in out
|
||||
assert "/releases/tag/v1.1.0" in out
|
||||
assert feed.asset_paths() == []
|
||||
assert _stamp_version(instance) == "1.0.0"
|
||||
|
||||
|
||||
def test_latest_checksum_mismatch_writes_nothing_and_cleans_up(instance, tmp_path, feed, scratch_tmp, capsys):
|
||||
_publish(feed, tmp_path, "1.1.0", checksum="wrong")
|
||||
|
||||
with pytest.raises(typer.Exit) as excinfo:
|
||||
dist_cmd.run_upgrade(latest=True)
|
||||
|
||||
assert excinfo.value.exit_code == 1
|
||||
assert "does not match" in " ".join(capsys.readouterr().out.split())
|
||||
assert (instance / "AGENTS.md").read_text(encoding="utf-8") == "core\n"
|
||||
assert _stamp_version(instance) == "1.0.0"
|
||||
assert list(scratch_tmp.iterdir()) == []
|
||||
|
||||
|
||||
def test_latest_archive_version_that_differs_from_the_feed_is_refused(instance, tmp_path, feed, scratch_tmp, capsys):
|
||||
_publish(feed, tmp_path, "1.1.0", inner_version="1.2.0")
|
||||
|
||||
with pytest.raises(typer.Exit) as excinfo:
|
||||
dist_cmd.run_upgrade(latest=True)
|
||||
|
||||
assert excinfo.value.exit_code == 1
|
||||
out = " ".join(capsys.readouterr().out.split())
|
||||
assert "1.2.0" in out and "1.1.0" in out
|
||||
assert _stamp_version(instance) == "1.0.0"
|
||||
assert list(scratch_tmp.iterdir()) == []
|
||||
|
||||
|
||||
def test_latest_asset_download_error_is_reported_with_its_url(instance, tmp_path, feed, capsys):
|
||||
name = _publish(feed, tmp_path, "1.1.0")
|
||||
del feed.routes[f"/assets/{name}.tar.gz"]
|
||||
|
||||
with pytest.raises(typer.Exit) as excinfo:
|
||||
dist_cmd.run_upgrade(latest=True)
|
||||
|
||||
assert excinfo.value.exit_code == 1
|
||||
assert f"/assets/{name}.tar.gz answered HTTP 404" in " ".join(capsys.readouterr().out.split())
|
||||
assert _stamp_version(instance) == "1.0.0"
|
||||
|
||||
|
||||
def test_latest_unreachable_feed_is_an_error_naming_the_url(instance, monkeypatch, capsys):
|
||||
monkeypatch.setenv("WIKITOOL_UPDATE_URL", "http://127.0.0.1:1/feed")
|
||||
|
||||
with pytest.raises(typer.Exit) as excinfo:
|
||||
dist_cmd.run_upgrade(latest=True)
|
||||
|
||||
assert excinfo.value.exit_code == 1
|
||||
assert "Could not reach http://127.0.0.1:1/feed" in " ".join(capsys.readouterr().out.split())
|
||||
|
||||
|
||||
def test_latest_respects_the_local_preconditions_before_asking_the_feed(instance, tmp_path, feed):
|
||||
_publish(feed, tmp_path, "1.1.0")
|
||||
(instance / "AGENTS.md").write_text("locally edited\n", encoding="utf-8")
|
||||
subprocess.run(["git", "init", "-q"], cwd=instance, check=True)
|
||||
|
||||
with pytest.raises(typer.Exit):
|
||||
dist_cmd.run_upgrade(latest=True)
|
||||
|
||||
assert feed.requests == []
|
||||
|
||||
|
||||
def test_latest_blocked_refusal_names_the_pinned_latest_invocation(instance, tmp_path, feed, capsys):
|
||||
_publish(feed, tmp_path, "1.1.0")
|
||||
(instance / "AGENTS.md").write_text("locally edited\n", encoding="utf-8")
|
||||
|
||||
with pytest.raises(typer.Exit):
|
||||
dist_cmd.run_upgrade(latest=True)
|
||||
|
||||
out = " ".join(capsys.readouterr().out.split())
|
||||
assert "dist upgrade --latest --expect 1.1.0 --take-release AGENTS.md" in out
|
||||
assert "dist upgrade --latest --expect 1.1.0 --keep-local" in out
|
||||
|
||||
|
||||
def test_source_and_latest_together_or_neither_is_refused_before_any_check(instance, tmp_path, feed):
|
||||
release = _release(tmp_path, "release", "1.1.0", {"AGENTS.md": "core\n"})
|
||||
|
||||
with pytest.raises(typer.Exit) as both:
|
||||
dist_cmd.run_upgrade(release, latest=True)
|
||||
with pytest.raises(typer.Exit) as neither:
|
||||
dist_cmd.run_upgrade()
|
||||
|
||||
assert both.value.exit_code == 1 and neither.value.exit_code == 1
|
||||
assert feed.requests == []
|
||||
|
||||
|
||||
# --- --expect ---------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.mark.parametrize("spelling", ["1.1.0", "v1.1.0"])
|
||||
def test_expect_matching_the_feed_proceeds(instance, tmp_path, feed, spelling):
|
||||
_publish(feed, tmp_path, "1.1.0")
|
||||
|
||||
dist_cmd.run_upgrade(latest=True, expect=spelling)
|
||||
|
||||
assert _stamp_version(instance) == "1.1.0"
|
||||
|
||||
|
||||
def test_expect_mismatch_is_refused_before_any_download_naming_both_versions(instance, tmp_path, feed, capsys):
|
||||
_publish(feed, tmp_path, "1.2.0")
|
||||
|
||||
with pytest.raises(typer.Exit) as excinfo:
|
||||
dist_cmd.run_upgrade(latest=True, expect="1.1.0")
|
||||
|
||||
assert excinfo.value.exit_code == 1
|
||||
out = " ".join(capsys.readouterr().out.split())
|
||||
assert "1.2.0" in out and "1.1.0" in out and "version notes" in out
|
||||
assert feed.asset_paths() == []
|
||||
assert _stamp_version(instance) == "1.0.0"
|
||||
|
||||
|
||||
def test_expect_mismatch_wins_over_an_already_current_install(instance, tmp_path, feed):
|
||||
_publish(feed, tmp_path, "1.0.0")
|
||||
|
||||
with pytest.raises(typer.Exit):
|
||||
dist_cmd.run_upgrade(latest=True, expect="1.1.0")
|
||||
|
||||
|
||||
def test_expect_without_latest_is_refused(instance, tmp_path, feed):
|
||||
release = _release(tmp_path, "release", "1.1.0", {"AGENTS.md": "core\n"})
|
||||
|
||||
with pytest.raises(typer.Exit) as excinfo:
|
||||
dist_cmd.run_upgrade(release, expect="1.1.0")
|
||||
|
||||
assert excinfo.value.exit_code == 1
|
||||
assert feed.requests == []
|
||||
|
||||
|
||||
def test_expect_that_is_not_a_version_is_refused(instance, feed):
|
||||
with pytest.raises(typer.Exit):
|
||||
dist_cmd.run_upgrade(latest=True, expect="latest")
|
||||
assert feed.requests == []
|
||||
|
||||
|
||||
# --- the token --------------------------------------------------------------
|
||||
|
||||
|
||||
def test_token_goes_to_the_feed_and_same_origin_assets_only(instance, tmp_path, feed, other_origin, monkeypatch):
|
||||
monkeypatch.setenv("WIKITOOL_UPDATE_TOKEN", "s3cret")
|
||||
name = _publish(feed, tmp_path, "1.1.0")
|
||||
|
||||
dist_cmd.run_upgrade(latest=True)
|
||||
|
||||
assert {auth for _, auth in feed.requests} == {"token s3cret"}
|
||||
assert f"/assets/{name}.tar.gz" in feed.paths()
|
||||
assert other_origin.requests == []
|
||||
|
||||
|
||||
def test_token_is_withheld_from_an_asset_on_another_origin(instance, tmp_path, feed, other_origin, monkeypatch):
|
||||
monkeypatch.setenv("WIKITOOL_UPDATE_TOKEN", "s3cret")
|
||||
_publish(feed, tmp_path, "1.1.0", asset_server=other_origin)
|
||||
|
||||
dist_cmd.run_upgrade(latest=True)
|
||||
|
||||
assert _stamp_version(instance) == "1.1.0"
|
||||
assert feed.requests == [("/feed", "token s3cret")]
|
||||
assert len(other_origin.requests) == 2
|
||||
assert {auth for _, auth in other_origin.requests} == {None}
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"feed_url, asset_url, sent",
|
||||
[
|
||||
("https://git.example/api/v1/releases/latest", "https://git.example/dl/a.tar.gz", True),
|
||||
("https://git.example/api/v1/releases/latest", "https://git.example:443/dl/a.tar.gz", True),
|
||||
("https://git.example/api/v1/releases/latest", "http://git.example/dl/a.tar.gz", False),
|
||||
("https://git.example/api/v1/releases/latest", "https://cdn.example/dl/a.tar.gz", False),
|
||||
("http://git.example:3000/x", "http://git.example:3001/x", False),
|
||||
],
|
||||
)
|
||||
def test_token_for_asset_compares_scheme_host_and_port(feed_url, asset_url, sent):
|
||||
assert version_mod.token_for_asset(feed_url, asset_url, "t") == ("t" if sent else None)
|
||||
assert version_mod.token_for_asset(feed_url, asset_url, None) is None
|
||||
|
||||
|
||||
# --- the asset names --------------------------------------------------------
|
||||
|
||||
|
||||
def test_release_asset_names_match_the_workflow():
|
||||
workflow = Path(__file__).resolve().parents[3] / ".gitea" / "workflows" / "release.yml"
|
||||
if not workflow.is_file():
|
||||
pytest.skip("a distributed instance carries no release workflow")
|
||||
text = workflow.read_text(encoding="utf-8")
|
||||
|
||||
archive = version_mod.ARCHIVE_NAME.format(version="${VERSION}")
|
||||
stem = archive.removesuffix(".tar.gz")
|
||||
assert f'name="{stem}"' in text
|
||||
assert archive.replace(stem, "${name}") in text
|
||||
assert (archive + version_mod.CHECKSUM_SUFFIX).replace(stem, "${name}") in text
|
||||
|
||||
|
||||
def test_fetch_latest_assets_reads_urls_off_the_release_and_skips_malformed_entries():
|
||||
payload = json.dumps({
|
||||
"tag_name": "v1.1.0",
|
||||
"assets": [
|
||||
{"name": "a.tar.gz", "browser_download_url": "https://h/a"},
|
||||
{"name": "no-url"},
|
||||
"not an object",
|
||||
],
|
||||
}).encode()
|
||||
|
||||
release = version_mod.fetch_latest_assets("https://h/feed", fetcher=lambda u, t, to: payload)
|
||||
|
||||
assert release.version == version_mod.Version.parse("1.1.0")
|
||||
assert release.tag == "v1.1.0"
|
||||
assert release.assets == {"a.tar.gz": "https://h/a"}
|
||||
with pytest.raises(version_mod.VersionError, match="assets present: a.tar.gz"):
|
||||
release.asset_urls()
|
||||
+124
-5
@@ -37,7 +37,9 @@ import functools
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import urllib.error
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
@@ -349,12 +351,15 @@ def fetch_latest(
|
||||
|
||||
The network call sits behind `fetcher` so every caller above this line -
|
||||
and every test - can run without a network. This is the one place that
|
||||
talks to the **release feed**, and only two commands reach it: `version
|
||||
check`, whose whole job it is, and `version notes` on a *distributed*
|
||||
talks to the **release feed**, and only three commands reach it: `version
|
||||
check`, whose whole job it is, `version notes` on a *distributed*
|
||||
instance, whose local `CHANGES.md` is a stub with no entry to print (see
|
||||
`fetch_latest_notes`). Neither is implicit - `check` exists for the call,
|
||||
and `notes` announces the URL it is asking before it asks, on stderr, and
|
||||
takes `--offline` for a caller that wants none of it.
|
||||
`fetch_latest_notes`), and `dist upgrade --latest`, which asks the feed
|
||||
which release to download (see `fetch_latest_assets`). None is implicit -
|
||||
`check` exists for the call, `notes` announces the URL it is asking before
|
||||
it asks, on stderr, and takes `--offline` for a caller that wants none of
|
||||
it, and `upgrade` reaches the feed only when the operator passes
|
||||
`--latest`.
|
||||
"""
|
||||
fetch = fetcher or _urlopen_fetch
|
||||
try:
|
||||
@@ -462,6 +467,120 @@ def fetch_latest_notes(
|
||||
return version, body, (str(html_url) if html_url else None)
|
||||
|
||||
|
||||
# --- release assets --------------------------------------------------------
|
||||
#
|
||||
# `dist upgrade --latest` downloads the two assets `.gitea/workflows/release.yml`
|
||||
# attaches to every release. The names are that workflow's `name=` line plus
|
||||
# `.tar.gz` / `.tar.gz.sha256`; `test_release_asset_names_match_the_workflow`
|
||||
# ties this constant to the workflow so one cannot move without the other.
|
||||
|
||||
ARCHIVE_NAME = "chemenu-stack-{version}.tar.gz"
|
||||
CHECKSUM_SUFFIX = ".sha256"
|
||||
DOWNLOAD_TIMEOUT = 60.0
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class LatestRelease:
|
||||
"""What the feed's latest release offers: its version, its tag as the feed
|
||||
spells it, its own page, and every asset as `name -> download URL`."""
|
||||
|
||||
version: Version
|
||||
tag: str
|
||||
html_url: Optional[str]
|
||||
assets: dict[str, str]
|
||||
|
||||
@property
|
||||
def archive_name(self) -> str:
|
||||
return ARCHIVE_NAME.format(version=self.version)
|
||||
|
||||
def asset_urls(self) -> tuple[str, str]:
|
||||
"""`(archive URL, checksum URL)`, read off the release object and never
|
||||
composed. Raises `VersionError` naming the release page when either is
|
||||
missing - an upgrade without its checksum is refused, not downgraded to
|
||||
an unchecked one."""
|
||||
archive = self.archive_name
|
||||
wanted = (archive, archive + CHECKSUM_SUFFIX)
|
||||
missing = [name for name in wanted if name not in self.assets]
|
||||
if missing:
|
||||
present = ", ".join(sorted(self.assets)) or "none"
|
||||
page = f" - see the release page {self.html_url}" if self.html_url else ""
|
||||
raise VersionError(
|
||||
f"release {self.tag} does not publish {' and '.join(missing)} "
|
||||
f"(assets present: {present}){page}"
|
||||
)
|
||||
return self.assets[wanted[0]], self.assets[wanted[1]]
|
||||
|
||||
|
||||
def fetch_latest_assets(
|
||||
url: str,
|
||||
token: Optional[str] = None,
|
||||
timeout: float = 10.0,
|
||||
fetcher: Optional[Fetcher] = None,
|
||||
) -> LatestRelease:
|
||||
"""`fetch_latest`'s version plus the release's assets, from the one
|
||||
response - the asset URLs are the feed's own `browser_download_url`
|
||||
values, never a path this code composes (AGENTS.md invariant 7)."""
|
||||
version, captured = _fetch_latest_object(url, token, timeout, fetcher)
|
||||
assets: dict[str, str] = {}
|
||||
raw_assets = captured.get("assets")
|
||||
if isinstance(raw_assets, list):
|
||||
for entry in raw_assets:
|
||||
if not isinstance(entry, dict):
|
||||
continue
|
||||
name = str(entry.get("name") or "").strip()
|
||||
download = str(entry.get("browser_download_url") or "").strip()
|
||||
if name and download:
|
||||
assets[name] = download
|
||||
html_url = captured.get("html_url") or captured.get("url")
|
||||
return LatestRelease(
|
||||
version=version,
|
||||
tag=str(captured.get("tag_name") or "").strip(),
|
||||
html_url=str(html_url) if html_url else None,
|
||||
assets=assets,
|
||||
)
|
||||
|
||||
|
||||
def _origin(url: str) -> tuple[str, str, int]:
|
||||
parts = urllib.parse.urlsplit(url)
|
||||
scheme = parts.scheme.lower()
|
||||
default = {"http": 80, "https": 443}.get(scheme, 0)
|
||||
return scheme, (parts.hostname or "").lower(), parts.port or default
|
||||
|
||||
|
||||
def token_for_asset(feed_url: str, asset_url: str, token: Optional[str]) -> Optional[str]:
|
||||
"""The token to send with an asset download: `token` when the asset is on
|
||||
the feed's own origin (scheme, host, port), `None` otherwise. An asset URL
|
||||
is data from a JSON answer, so the credential does not follow it to
|
||||
another host."""
|
||||
return token if token and _origin(feed_url) == _origin(asset_url) else None
|
||||
|
||||
|
||||
def download_asset(
|
||||
url: str,
|
||||
dest: Path,
|
||||
token: Optional[str] = None,
|
||||
timeout: float = DOWNLOAD_TIMEOUT,
|
||||
) -> None:
|
||||
"""Stream `url` into `dest`. `timeout` is a socket timeout per read, not a
|
||||
deadline for the whole transfer. The token goes out as an *unredirected*
|
||||
header, so a redirect to another host does not carry it along. Every
|
||||
failure is a `VersionError` naming the URL."""
|
||||
request = urllib.request.Request(url)
|
||||
if token:
|
||||
request.add_unredirected_header("Authorization", f"token {token}")
|
||||
try:
|
||||
with urllib.request.urlopen(request, timeout=timeout) as response: # noqa: S310 - URL from the feed
|
||||
with dest.open("wb") as out:
|
||||
shutil.copyfileobj(response, out)
|
||||
except urllib.error.HTTPError as exc:
|
||||
hint = ""
|
||||
if exc.code in (401, 403):
|
||||
hint = f" - the download needs authentication; set ${UPDATE_TOKEN_ENV}"
|
||||
raise VersionError(f"{url} answered HTTP {exc.code}{hint}") from exc
|
||||
except (urllib.error.URLError, OSError, TimeoutError) as exc:
|
||||
raise VersionError(f"Could not download {url}: {exc}") from exc
|
||||
|
||||
|
||||
# --- CHANGES.md ------------------------------------------------------------
|
||||
#
|
||||
# The changelog is prose and stays the author's job. What is mechanical is the
|
||||
|
||||
Reference in new issue
Block a user