From c33e8cdfb10068d81c318557d31c6533d9684363 Mon Sep 17 00:00:00 2001 From: Torben Nehmer Date: Thu, 1 Oct 2026 13:39:56 +0200 Subject: [PATCH] feat: preflight as a release asset - download, verify, unpack, then run the tree preflight (#151, C) Files changed: - .gitea/workflows/release.yml - CHANGES.md - INSTALL.md - README.md - VERSION - instructions/dev/testing-conventions.md - instructions/preflight.md - tools/CONTRACT.md - tools/README.md - tools/chemenu/tests/test_preflight.py - tools/chemenu/tests/test_preflight_pwsh.py - tools/preflight.ps1 - tools/preflight.sh --- .gitea/workflows/release.yml | 28 +- CHANGES.md | 37 ++- INSTALL.md | 6 + README.md | 4 +- VERSION | 2 +- instructions/dev/testing-conventions.md | 5 + instructions/preflight.md | 24 +- tools/CONTRACT.md | 5 + tools/README.md | 13 +- tools/chemenu/tests/test_preflight.py | 298 ++++++++++++++++++++- tools/chemenu/tests/test_preflight_pwsh.py | 172 +++++++++++- tools/preflight.ps1 | 288 ++++++++++++++++---- tools/preflight.sh | 226 ++++++++++++++-- 13 files changed, 1020 insertions(+), 88 deletions(-) diff --git a/.gitea/workflows/release.yml b/.gitea/workflows/release.yml index 6b9c825..301d540 100644 --- a/.gitea/workflows/release.yml +++ b/.gitea/workflows/release.yml @@ -3,7 +3,9 @@ # The release artifact is exactly a `dist export` tree, packed with a top-level # directory: unpack it, run instructions/setup-instance.md, and there is a # working wiki instance - no checkout of this repo required. CI already proved -# that path works before this workflow ever runs. +# that path works before this workflow ever runs. Beside it the release carries +# tools/preflight.sh and tools/preflight.ps1 as assets, which download and unpack +# that tarball themselves. # # The tag is created here, by CI, and never by an agent: AGENTS.md invariant 5 # ("never call raw git commit/push") stays intact because nothing in a session @@ -149,6 +151,28 @@ jobs: cat "${BUILD_DIR}/${name}.tar.gz.sha256" echo "name=${name}" >> "$GITHUB_OUTPUT" + # The two preflight scripts are attached to the release as well: the first + # thing a new user runs, before there is any tree to run it from. Each copy + # is the tree's script with the download address of *this* release written + # into its two placeholder lines (the tree copy keeps them empty, which is + # how a script knows it is not a release asset). The address is the public + # one, for the same reason as the URLs in `dist export` above. + download="${PUBLIC_BASE_URL}/${GITHUB_REPOSITORY}/releases/download/${TAG}" + sed \ + -e "s|^RELEASE_ARCHIVE_URL=''|RELEASE_ARCHIVE_URL='${download}/${name}.tar.gz'|" \ + -e "s|^RELEASE_CHECKSUM_URL=''|RELEASE_CHECKSUM_URL='${download}/${name}.tar.gz.sha256'|" \ + tools/preflight.sh > "${BUILD_DIR}/preflight.sh" + sed \ + -e "s|^\$ReleaseArchiveUrl = ''|\$ReleaseArchiveUrl = '${download}/${name}.tar.gz'|" \ + -e "s|^\$ReleaseChecksumUrl = ''|\$ReleaseChecksumUrl = '${download}/${name}.tar.gz.sha256'|" \ + tools/preflight.ps1 > "${BUILD_DIR}/preflight.ps1" + # A placeholder that did not match would ship a script that refuses to run. + grep -qF "RELEASE_ARCHIVE_URL='${download}/${name}.tar.gz'" "${BUILD_DIR}/preflight.sh" + grep -qF "RELEASE_CHECKSUM_URL='${download}/${name}.tar.gz.sha256'" "${BUILD_DIR}/preflight.sh" + grep -qF "ReleaseArchiveUrl = '${download}/${name}.tar.gz'" "${BUILD_DIR}/preflight.ps1" + grep -qF "ReleaseChecksumUrl = '${download}/${name}.tar.gz.sha256'" "${BUILD_DIR}/preflight.ps1" + chmod +x "${BUILD_DIR}/preflight.sh" + - name: Publish the release if: steps.version.outputs.skip != 'true' env: @@ -174,7 +198,7 @@ jobs: id="$(printf '%s' "$release" | jq -r '.id')" echo "Created release ${TAG} (id ${id})." - for asset in "${NAME}.tar.gz" "${NAME}.tar.gz.sha256"; do + for asset in "${NAME}.tar.gz" "${NAME}.tar.gz.sha256" preflight.sh preflight.ps1; do curl -sS -f -X POST "${API}/releases/${id}/assets?name=${asset}" \ -H "Authorization: token ${TOKEN}" \ -F "attachment=@${BUILD_DIR}/${asset}" > /dev/null diff --git a/CHANGES.md b/CHANGES.md index 2341fc7..746281e 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -59,7 +59,7 @@ concern - readable here, never shipped as something to parse. --- -## 8.0.0-beta.15 - 2026-10-01 - PowerShell 7 preflight and launcher: tools/preflight.ps1, tools/wikitool.ps1, doctor checks for execution policy and Mark of the Web +## 8.0.0-beta.16 - 2026-10-01 - Preflight as a release asset: download, verify and unpack the stack, then run the tree preflight **Author:** Torben Nehmer @@ -90,6 +90,7 @@ concern - readable here, never shipped as something to parse. - publish: the gate lists the staged state; a missing or unreachable remote stops before the commit - Path budget: a file's path stays at 160 characters or fewer so a Windows checkout works without long paths (#163) - PowerShell 7 preflight and launcher: tools/preflight.ps1, tools/wikitool.ps1, doctor checks for execution policy and Mark of the Web +- Preflight as a release asset: download, verify and unpack the stack, then run the tree preflight **Low impact** - version bump no longer points at version release in its output @@ -126,6 +127,40 @@ concern - readable here, never shipped as something to parse. - raw/CONTRACT.md points at the path budget for a name accepted from incoming/ +### Preflight as a release asset: download, verify and unpack the stack, then run the tree preflight + +Section C of #151, the first install. Until now a new user had to get the stack onto the machine +before any preflight could run - which is exactly the step that fails on a machine without git +or a short enough path. Each release now also attaches `preflight.sh` and `preflight.ps1` as +assets, and a script without `tools/prerequisites.txt` beside it runs in **asset mode**: it +downloads the release tarball and its `.sha256`, stops with exit 1 unless the checksum matches, +reads the folder limit from `tools/prerequisites.txt` inside the archive, and unpacks into +`chemenu/` next to itself (`--into ` for another place). It unpacks into a temporary +sibling and renames, requires exactly one top-level folder, and refuses an existing target +without touching it. Then it runs the unpacked tree's own preflight, passing `--set` and the exit +code through, so everything after the unpack is the tree mode that already existed. + +The asset copies are made by `release.yml`: it writes the same release's tarball and checksum +URLs into two placeholder lines of each script, checks that the substitution took, and uploads +the two files under exactly those names. The tree copies keep the placeholders empty; an asset +script with empty placeholders and no `--archive` exits 1 saying it does not come from a release. +`--archive ` uses a tarball already on disk (its `.sha256` must sit beside it) +for a machine that cannot download, and is the entry point the tests use. + +On POSIX asset mode needs `curl`, `tar` and `sha256sum` (or `shasum`) and stops with exit 42 and +the usual guidance block when one is missing; the PowerShell script uses what Windows ships plus +`tar`. On Windows with long paths off, the 95-character folder limit is judged at the final +target before anything is unpacked, so a too-long `--into` stops with exit 42 and the fix is a +shorter folder. Git Bash gets the target and archive converted with `cygpath -u`, because GNU +tar would read `C:` as a host. + +The tests build a release tarball with a checksum and run both scripts against it, including a +local HTTP server for the PowerShell download, the 95/96 boundary, a failed checksum, two +top-level folders, an existing target, and a test that ties the placeholder lines to the `sed` +expressions in `release.yml`. `tools/CONTRACT.md`, `tools/README.md`, `README.md`, +`instructions/preflight.md` (a new decision point for the asset script), +`instructions/dev/testing-conventions.md` and `INSTALL.md` describe the first-install route. + ### PowerShell 7 preflight and launcher: tools/preflight.ps1, tools/wikitool.ps1, doctor checks for execution policy and Mark of the Web The PowerShell half of #151. Harnesses that run in PowerShell 7 on Windows (GitHub Copilot CLI, diff --git a/INSTALL.md b/INSTALL.md index cd55bda..f06e10a 100644 --- a/INSTALL.md +++ b/INSTALL.md @@ -46,6 +46,12 @@ Die Prüfsumme ist nicht Zierde: Sie ist das Einzige, was einen unterbrochenen D einem vollständigen unterscheidet, und `sha256sum -c` muss `OK` sagen, bevor irgendetwas entpackt wird. +Statt dieser Befehle von Hand trägt jedes Release auch den Preflight selbst als Datei +(`preflight.sh`, unter Windows `preflight.ps1`). In einen leeren Ordner geladen und dort +gestartet, lädt er das Release herunter, prüft die Prüfsumme, entpackt es nach `chemenu/` neben +sich (mit `--into ` woandershin; ein vorhandenes Ziel wird nie angefasst) und führt dann +den Preflight im entpackten Baum aus, siehe [instructions/preflight.md](instructions/preflight.md). + Danach weiter mit Schritt 2 aus Weg B: den Agenten [instructions/setup-instance.md](instructions/setup-instance.md) ausführen lassen. Der entpackte Baum ist bereits eine Distribution - Schritt 1 (`dist export`) entfällt. diff --git a/README.md b/README.md index 8b81e24..7e05bb4 100644 --- a/README.md +++ b/README.md @@ -39,7 +39,9 @@ Two starting points, depending on what you're doing - full walkthrough in [INSTA ``` `tools/wikitool` refuses to start (exit 42) until the preflight has passed; if it stops - instead, its output says what to install - `instructions/preflight.md`. + instead, its output says what to install - `instructions/preflight.md`. A release carries the + same two scripts as assets that download and unpack the stack themselves, for installing + without a clone. That copies each `instructions//SKILL.md` into `.agents/skills/` (GitHub Copilot, Codex CLI, Mistral Vibe) and `.claude/skills/` (Claude Code). Re-run it after changing a skill. diff --git a/VERSION b/VERSION index 95f0f56..f2a3fb1 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -8.0.0-beta.15 +8.0.0-beta.16 diff --git a/instructions/dev/testing-conventions.md b/instructions/dev/testing-conventions.md index e54acce..5fc6b02 100644 --- a/instructions/dev/testing-conventions.md +++ b/instructions/dev/testing-conventions.md @@ -165,6 +165,11 @@ Whenever you add or change a test under `tools/chemenu/tests/`. ## Decision points +- **Testing a PowerShell script?** `test_preflight_pwsh.py` needs `pwsh` and skips silently + without it - a green run on a machine with no PowerShell 7 has not run those tests. Check + `command -v pwsh` before trusting the result; CI's `pwsh` job runs them in the image built from + `.gitea/pwsh-ci/Dockerfile`, and so can you (`docker run` that image with the checkout mounted + and `pytest tools/chemenu/tests/test_preflight_pwsh.py` as the command). - **A test genuinely needs the developer's real environment?** There is no such test, and a new one is a design problem rather than an exception: what it wants is a fixture that *builds* the state it needs inside `tmp_path`. Building it is also the only version CI can run. diff --git a/instructions/preflight.md b/instructions/preflight.md index 8c02804..e2143bc 100644 --- a/instructions/preflight.md +++ b/instructions/preflight.md @@ -68,7 +68,7 @@ It is safe to run at any time: a second run on a ready checkout changes nothing |---|---|---| | 0 | Everything is in place | Continue with the procedure that sent you here | | 42 | The user has to act | Step 3 | - | 1 | Called wrongly, or `tools/prerequisites.txt` is missing next to the script | Report the exact command and output to the user; do not retry blindly | + | 1 | Called wrongly, a download or unpack failed (asset mode), or the stack tree next to the script is incomplete | Report the exact command and output to the user; do not retry blindly | 3. **On exit 42, show the output to the user exactly as it is, then stop and wait.** It is written for someone without an IT background: each numbered block says what is missing, why @@ -99,10 +99,30 @@ It is safe to run at any time: a second run on a ready checkout changes nothing ## Decision points +- **The script is a release asset, not a tree script** - there is no `tools/prerequisites.txt` + beside it, because the user downloaded `preflight.sh` or `preflight.ps1` from a release page + into an empty folder. That is the *first* install, and the script does one more thing before + the steps above: it downloads the release tarball and its `.sha256`, refuses unless the + checksum matches, and unpacks into a `chemenu/` folder next to itself; then it runs the + preflight of the unpacked tree, passing `--set` and its exit code through. Run it exactly as + in step 1 (the path is the downloaded file, not `tools/...`), and read the exit code the same + way. After it, every later run - including the retry after an exit 42 - is the tree's own + `tools/preflight.sh` or `tools/preflight.ps1`, run from inside `chemenu/`. + - `--into ` unpacks somewhere else; an existing target is refused with exit 1 and + nothing is touched, which is the user's decision to make, not yours to resolve by deleting. + - `--archive ` uses a tarball already on disk, with its `.sha256` beside it, + when the machine cannot download. + - A checksum that does not match, a failed download, and a copy of the script that carries no + download address (it was not taken from a release) exit 1 with nothing unpacked; report the + message, and do not fetch the tarball by another route. + - On POSIX, `curl`, `tar` and `sha256sum` (or `shasum`) have to exist; when one does not, the + script stops with exit 42 like any other missing tool. The PowerShell script needs nothing + beyond what Windows ships. - **The output names a folder that is too long.** Only on Windows with long paths off: the install folder may be at most 95 characters, because every file of the wiki below it has to stay within 259. Moving the wiki to a shorter folder is the user's step; do not try to shorten - paths inside the wiki instead. + paths inside the wiki instead. In asset mode the length is judged at the folder the stack + *would* be unpacked into, before anything is unpacked; the fix is a shorter `--into`. - **The output names the PowerShell execution policy** (`Restricted` or `AllSigned`). The fix is a line the user runs in a PowerShell 7 window; it changes a setting of their account, so it is theirs to run. When a *group policy* sets it, nothing on this computer can override it: the diff --git a/tools/CONTRACT.md b/tools/CONTRACT.md index 430805e..386ef99 100644 --- a/tools/CONTRACT.md +++ b/tools/CONTRACT.md @@ -65,6 +65,11 @@ From PowerShell 7 on Windows, the twin: `pwsh -NoProfile -ExecutionPolicy Bypass Until it has passed, every `tools/wikitool` call exits 42 and names it. +A release also carries both scripts as assets (`preflight.sh`, `preflight.ps1`) for the very +first install, before there is a tree: run from a folder with no `tools/prerequisites.txt` beside +them, they download and verify the release tarball, unpack it into `chemenu/` next to themselves +(or `--into `), and run the preflight inside the unpacked tree. + ## Usage Run from the repo root: diff --git a/tools/README.md b/tools/README.md index c54633f..6536abc 100644 --- a/tools/README.md +++ b/tools/README.md @@ -32,6 +32,17 @@ it with `-m pip`. It is POSIX sh because it has to run before Python is known to exist; exit 42 means the user has to act, and its output says how. The procedure an agent follows around it is `instructions/preflight.md`. +Each release also attaches `preflight.sh` and `preflight.ps1` as assets, for the first install +before any tree exists. `release.yml` writes the download address of that same release into two +placeholder lines of the copy (`RELEASE_ARCHIVE_URL`/`RELEASE_CHECKSUM_URL` in the shell script, +`$ReleaseArchiveUrl`/`$ReleaseChecksumUrl` in the PowerShell one; the tree copy leaves them +empty). With no `prerequisites.txt` beside it, the script is in *asset mode*: it downloads the +tarball and its `.sha256`, refuses on a mismatch, reads the folder limit out of the archive, +unpacks into `