feat: preflight as a release asset - download, verify, unpack, then run the tree preflight (#151, C)
CI / verify (push) Successful in 2m26s
CI / pwsh (push) Failing after 11s
Release / release (push) Successful in 37s

Files changed:
- .gitea/workflows/release.yml
- CHANGES.md
- INSTALL.md
- README.md
- VERSION
- instructions/dev/testing-conventions.md
- instructions/preflight.md
- tools/CONTRACT.md
- tools/README.md
- tools/chemenu/tests/test_preflight.py
- tools/chemenu/tests/test_preflight_pwsh.py
- tools/preflight.ps1
- tools/preflight.sh
This commit is contained in:
torben committed 2026-10-01 13:39:56 +02:00
1 parent 210e0c8286
commit c33e8cdfb1
13 files changed
+1020 -88

No files matched your search

+5
View File
@@ -65,6 +65,11 @@ From PowerShell 7 on Windows, the twin: `pwsh -NoProfile -ExecutionPolicy Bypass
Until it has passed, every `tools/wikitool` call exits 42 and names it.
A release also carries both scripts as assets (`preflight.sh`, `preflight.ps1`) for the very
first install, before there is a tree: run from a folder with no `tools/prerequisites.txt` beside
them, they download and verify the release tarball, unpack it into `chemenu/` next to themselves
(or `--into <path>`), and run the preflight inside the unpacked tree.
## Usage
Run from the repo root:
+12 -1
View File
@@ -32,6 +32,17 @@ it with `-m pip`. It is POSIX sh because it has to run before Python is known
to exist; exit 42 means the user has to act, and its output says how. The
procedure an agent follows around it is `instructions/preflight.md`.
Each release also attaches `preflight.sh` and `preflight.ps1` as assets, for the first install
before any tree exists. `release.yml` writes the download address of that same release into two
placeholder lines of the copy (`RELEASE_ARCHIVE_URL`/`RELEASE_CHECKSUM_URL` in the shell script,
`$ReleaseArchiveUrl`/`$ReleaseChecksumUrl` in the PowerShell one; the tree copy leaves them
empty). With no `prerequisites.txt` beside it, the script is in *asset mode*: it downloads the
tarball and its `.sha256`, refuses on a mismatch, reads the folder limit out of the archive,
unpacks into `<script folder>/chemenu` (`--into <path>` to choose, an existing target is refused),
and runs the tree copy of itself, passing `--set` and its exit code through. `--archive <tarball>`
takes a local tarball instead of a download (its `.sha256` has to lie next to it); that is also
how the tests drive it.
`tools/wikitool` refuses to start (exit 42) until the preflight has written a
*complete* `.wikitool-tools.json` and the venv exists. Past that, every `git`
and `rg` the package starts goes through `chemenu/toolpaths.py`, which reads
@@ -51,7 +62,7 @@ tools/
wikitool entry point (POSIX sh): stops with exit 42 until the preflight has passed
wikitool.ps1 the same entry point for PowerShell 7, which resolves `tools/wikitool` to this file first
run_wikitool.py what the launcher runs with the venv's Python - puts chemenu on sys.path without PYTHONPATH
preflight.sh checks prerequisites.txt, records .wikitool-tools.json, creates .venv (POSIX sh)
preflight.sh checks prerequisites.txt, records .wikitool-tools.json, creates .venv (POSIX sh); as the release asset, downloads and unpacks the stack first
preflight.ps1 the same for PowerShell 7; also checks the execution policy and the Mark of the Web
prerequisites.txt what the machine needs, one `|`-separated line per tool - read by the preflight and `doctor`
trace-hook what the harness hooks call: trace_ingest.py under the venv's Python
+295 -3
View File
@@ -11,12 +11,14 @@ preflight for real.
"""
from __future__ import annotations
import hashlib
import json
import os
import shutil
import stat
import subprocess
import sys
import tarfile
from pathlib import Path
import pytest
@@ -28,7 +30,7 @@ TOOLS = config._PACKAGE_ROOT / "tools"
# What preflight.sh, the launcher and trace-hook call besides shell built-ins
# and the tools under test. `iconv` is optional in the script itself.
UTILITIES = ("dirname", "uname", "sed", "tr", "wc", "iconv", "tail", "cat", "mv", "head",
"cut", "grep", "mkdir", "cp", "rm")
"cut", "grep", "mkdir", "cp", "rm", "tar", "gzip", "sha256sum", "mktemp")
def _shells() -> list[str]:
@@ -93,6 +95,7 @@ class Machine:
self.base = base
self.root = base / root_name
self.tools = self.root / "tools"
self.script = self.tools / "preflight.sh"
self.tools.mkdir(parents=True)
for name in ("preflight.sh", "preflight.ps1", "prerequisites.txt", "wikitool", "wikitool.ps1",
"run_wikitool.py", "trace-hook"):
@@ -108,6 +111,7 @@ class Machine:
self.pip_log = base / "pip.log"
self.extra_env: dict[str, str] = {}
self.path_dirs: list[Path] = [self.stubs, self.sysbin]
self.cwd: Path | None = None
def stub(self, name: str, text: str, where: Path | None = None) -> Path:
return _executable((where or self.stubs) / name, text)
@@ -132,8 +136,8 @@ class Machine:
**self.extra_env,
}
return subprocess.run(
[shell, str(self.tools / "preflight.sh"), *args],
capture_output=True, text=True, env=env, timeout=60,
[shell, str(self.script), *args],
capture_output=True, text=True, env=env, timeout=60, cwd=self.cwd,
)
@property
@@ -354,6 +358,294 @@ def test_manifest_names_the_tools_the_stack_starts():
assert [t.name for t in manifest.tools_for("windows")] == ["python", "git", "rg", "pwsh"]
# --- asset mode (D33, D38-D41) ------------------------------------------------------
RELEASE_VERSION = "9.9.9"
DOWNLOAD_BASE = f"https://example.test/torben/chemenu/releases/download/v{RELEASE_VERSION}"
STACK_FILES = ("preflight.sh", "preflight.ps1", "prerequisites.txt", "wikitool", "wikitool.ps1",
"run_wikitool.py", "trace-hook")
# The two lines each script keeps for the release build to fill: (empty, filled).
PLACEHOLDERS = {
"preflight.sh": (
("RELEASE_ARCHIVE_URL=''", f"RELEASE_ARCHIVE_URL='{DOWNLOAD_BASE}/chemenu-stack-{RELEASE_VERSION}.tar.gz'"),
("RELEASE_CHECKSUM_URL=''", f"RELEASE_CHECKSUM_URL='{DOWNLOAD_BASE}/chemenu-stack-{RELEASE_VERSION}.tar.gz.sha256'"),
),
"preflight.ps1": (
("$ReleaseArchiveUrl = ''", f"$ReleaseArchiveUrl = '{DOWNLOAD_BASE}/chemenu-stack-{RELEASE_VERSION}.tar.gz'"),
("$ReleaseChecksumUrl = ''", f"$ReleaseChecksumUrl = '{DOWNLOAD_BASE}/chemenu-stack-{RELEASE_VERSION}.tar.gz.sha256'"),
),
}
CURL_STUB = """#!/bin/sh
# Serves the files of $RELEASE_DIR by the last part of the URL, and logs every request.
while [ $# -gt 0 ]; do
case "$1" in
-o) dest=$2; shift ;;
-*) ;;
*) url=$1 ;;
esac
shift
done
echo "$url" >> "$CURL_LOG"
src="$RELEASE_DIR/${url##*/}"
[ -f "$src" ] || exit 22
cp "$src" "$dest"
"""
def build_release(base: Path, *, limit: int | None = None, tops: tuple[str, ...] | None = None) -> Path:
"""A release tarball and its .sha256 as CI builds them; returns the tarball."""
name = f"chemenu-stack-{RELEASE_VERSION}"
stage = base / "stage"
for top in tops or (name,):
(stage / top / "tools").mkdir(parents=True)
for file in STACK_FILES:
shutil.copy2(TOOLS / file, stage / top / "tools" / file)
(stage / top / "tools" / "requirements.txt").write_text("PyYAML\n", encoding="utf-8")
if limit is not None:
manifest = stage / top / "tools" / "prerequisites.txt"
manifest.write_text(
manifest.read_text(encoding="utf-8").replace(
"limit|install_dir_max|95", f"limit|install_dir_max|{limit}"),
encoding="utf-8")
release = base / "release"
release.mkdir()
tarball = release / f"{name}.tar.gz"
with tarfile.open(tarball, "w:gz") as archive:
for top in tops or (name,):
archive.add(stage / top, arcname=top)
digest = hashlib.sha256(tarball.read_bytes()).hexdigest()
(release / f"{tarball.name}.sha256").write_text(f"{digest} {tarball.name}\n", encoding="utf-8")
return tarball
class AssetMachine(Machine):
"""The release asset on a machine with nothing unpacked yet: just the script, alone in a folder."""
def __init__(self, base: Path, *, filled: bool = False, **release):
super().__init__(base)
self.tarball = build_release(base, **release)
self.download = base / "download"
self.download.mkdir()
for name, pairs in PLACEHOLDERS.items():
text = (TOOLS / name).read_text(encoding="utf-8")
for empty, full in pairs:
assert empty in text, f"{name} lost its placeholder {empty}"
if filled:
text = text.replace(empty, full)
(self.download / name).write_text(text, encoding="utf-8")
self.script = self.download / "preflight.sh"
self.root = self.download / "chemenu"
self.tools = self.root / "tools"
self.cwd = self.download
self.extra_env.update({"RELEASE_DIR": str(self.tarball.parent), "CURL_LOG": str(base / "curl.log")})
self.standard()
def unpacked(self, root: Path | None = None) -> bool:
return (root or self.root).exists()
def leftovers(self) -> list[str]:
found = [str(path) for path in self.base.rglob(".chemenu-unpack.*")]
found += [str(path) for path in Path(os.environ.get("TMPDIR", "/tmp")).glob("chemenu-preflight.*")]
return found
@pytest.fixture
def asset(tmp_path: Path) -> AssetMachine:
return AssetMachine(tmp_path.resolve())
@pytest.mark.parametrize("shell", SHELLS)
def test_asset_unpacks_next_to_itself_and_runs_the_tree_copy(asset, shell):
result = asset.run("--archive", str(asset.tarball), shell=shell)
assert result.returncode == 0, result.stdout + result.stderr
assert "sha256 OK" in result.stdout and "Preflight passed" in result.stdout
assert (asset.tools / "preflight.sh").is_file() and (asset.tools / "prerequisites.txt").is_file()
assert asset.recorded()["complete"] is True
assert (asset.tools / ".venv").is_dir()
assert not asset.leftovers()
@pytest.mark.parametrize("shell", SHELLS)
def test_into_chooses_the_target_and_creates_missing_parents(asset, shell):
target = asset.base / "deep" / "er" / "wiki"
result = asset.run("--archive", str(asset.tarball), "--into", str(target), shell=shell)
assert result.returncode == 0, result.stdout + result.stderr
assert (target / "tools" / "preflight.sh").is_file()
assert (target / ".wikitool-tools.json").is_file()
assert not asset.unpacked()
@pytest.mark.parametrize("shell", SHELLS)
def test_a_relative_into_resolves_against_the_working_directory(asset, shell):
asset.cwd = asset.base
result = asset.run("--archive", str(asset.tarball), "--into", "here", shell=shell)
assert result.returncode == 0, result.stdout + result.stderr
assert (asset.base / "here" / "tools" / "preflight.sh").is_file()
@pytest.mark.parametrize("shell", SHELLS)
def test_an_existing_target_is_refused_and_left_alone(asset, shell):
asset.root.mkdir()
(asset.root / "mine.txt").write_text("keep", encoding="utf-8")
result = asset.run("--archive", str(asset.tarball), shell=shell)
assert result.returncode == 1
assert "already exists" in result.stderr and "--into" in result.stderr
assert [path.name for path in asset.root.iterdir()] == ["mine.txt"]
assert not asset.leftovers()
@pytest.mark.parametrize("shell", SHELLS)
def test_a_wrong_sha256_exits_1_and_unpacks_nothing(asset, shell):
checksum = asset.tarball.parent / f"{asset.tarball.name}.sha256"
checksum.write_text("0" * 64 + f" {asset.tarball.name}\n", encoding="utf-8")
result = asset.run("--archive", str(asset.tarball), shell=shell)
assert result.returncode == 1
assert "does not match" in result.stderr and "nothing was unpacked" in result.stderr
assert not asset.unpacked() and not asset.leftovers()
@pytest.mark.parametrize("shell", SHELLS)
def test_archive_without_a_checksum_file_exits_1(asset, shell):
(asset.tarball.parent / f"{asset.tarball.name}.sha256").unlink()
result = asset.run("--archive", str(asset.tarball), shell=shell)
assert result.returncode == 1
assert ".sha256 is missing" in result.stderr
assert not asset.unpacked()
@pytest.mark.parametrize("shell", SHELLS)
def test_an_archive_with_two_top_level_folders_exits_1(tmp_path, shell):
asset = AssetMachine(tmp_path.resolve(), tops=(f"chemenu-stack-{RELEASE_VERSION}", "stray"))
result = asset.run("--archive", str(asset.tarball), shell=shell)
assert result.returncode == 1
assert "exactly one top-level folder" in result.stderr
assert not asset.unpacked() and not asset.leftovers()
@pytest.mark.parametrize("shell", SHELLS)
def test_a_copy_without_download_addresses_says_it_is_no_release_asset(asset, shell):
result = asset.run(shell=shell)
assert result.returncode == 1
assert "does not come from a release" in result.stderr
assert not asset.unpacked()
@pytest.mark.parametrize("shell", SHELLS)
def test_download_fetches_both_files_of_the_same_release(tmp_path, shell):
asset = AssetMachine(tmp_path.resolve(), filled=True)
asset.stub("curl", CURL_STUB)
result = asset.run(shell=shell)
assert result.returncode == 0, result.stdout + result.stderr
requested = (asset.base / "curl.log").read_text(encoding="utf-8").split()
assert requested == [f"{DOWNLOAD_BASE}/{asset.tarball.name}", f"{DOWNLOAD_BASE}/{asset.tarball.name}.sha256"]
assert (asset.tools / "preflight.sh").is_file()
assert not asset.leftovers()
@pytest.mark.parametrize("shell", SHELLS)
def test_a_failed_download_exits_1_and_unpacks_nothing(tmp_path, shell):
asset = AssetMachine(tmp_path.resolve(), filled=True)
asset.stub("curl", CURL_STUB)
asset.extra_env["RELEASE_DIR"] = str(asset.base / "nothing-here")
result = asset.run(shell=shell)
assert result.returncode == 1
assert "the download failed" in result.stderr
assert not asset.unpacked() and not asset.leftovers()
@pytest.mark.parametrize("shell", SHELLS)
@pytest.mark.parametrize("missing", ["curl", "tar", "sha256sum"])
def test_a_missing_download_tool_is_a_stop_with_guidance(tmp_path, shell, missing):
asset = AssetMachine(tmp_path.resolve(), filled=True)
asset.stub("curl", CURL_STUB)
(asset.stubs / "curl").unlink() if missing == "curl" else (asset.sysbin / missing).unlink()
result = asset.run(shell=shell)
assert result.returncode == 42
assert_guidance(result.stdout, f"could not be found: {missing}")
assert not asset.unpacked() and not (asset.base / "curl.log").exists()
@pytest.mark.parametrize("shell", SHELLS)
def test_with_an_archive_no_download_tool_is_needed(asset, shell):
assert not (asset.stubs / "curl").exists() and not (asset.sysbin / "curl").exists()
assert asset.run("--archive", str(asset.tarball), shell=shell).returncode == 0
@pytest.mark.parametrize("shell", SHELLS)
def test_set_and_the_exit_code_pass_through_to_the_tree_copy(asset, shell):
refused = asset.run("--archive", str(asset.tarball), "--set", f"rg={asset.base / 'nowhere' / 'rg'}", shell=shell)
assert refused.returncode == 42
assert_guidance(refused.stdout, "The path given for ripgrep (rg) does not work")
assert (asset.tools / "preflight.sh").is_file() and not asset.tools_file.exists()
elsewhere = asset.stub("rg", "#!/bin/sh\necho 'ripgrep 14.1.1'\n", asset.base / "opt")
second = AssetMachine(asset.base / "second")
second.stub("rg", "#!/bin/sh\necho 'ripgrep 14.1.1'\n", asset.base / "opt")
ok = second.run("--archive", str(second.tarball), "--set", f"rg={elsewhere}", shell=shell)
assert ok.returncode == 0, ok.stdout
assert second.recorded()["tools"]["rg"] == str(elsewhere)
@pytest.mark.parametrize("shell", SHELLS)
@pytest.mark.parametrize("flag", ["--into", "--archive"])
def test_asset_options_are_a_usage_error_inside_a_tree(machine, shell, flag):
machine.standard()
result = machine.run(flag, str(machine.base), shell=shell)
assert result.returncode == 1
assert "belong to the release asset" in result.stderr
@pytest.mark.parametrize("shell", SHELLS)
@pytest.mark.parametrize("length, longpaths, limit, expected", [
(95, "0", None, 0),
(96, "0", None, 42),
(96, "1", None, 0),
(110, "0", 120, 0),
(121, "0", 120, 42),
])
def test_the_folder_limit_is_judged_at_the_final_target_from_the_archive(
tmp_path, shell, length, longpaths, limit, expected):
base = tmp_path.resolve()
name_length = length - len(str(base / "download")) - 1
assert name_length > 0, "tmp_path is too long for this test"
asset = AssetMachine(base, limit=limit)
asset.standard(pwsh=True)
asset.extra_env.update({"CHEMENU_PREFLIGHT_PLATFORM": "windows", "CHEMENU_PREFLIGHT_LONGPATHS": longpaths})
target = asset.download / ("t" * name_length)
assert len(str(target)) == length
result = asset.run("--archive", str(asset.tarball), "--into", str(target), shell=shell)
assert result.returncode == expected, result.stdout + result.stderr
if expected == 42:
assert_guidance(result.stdout, f"too long ({length} characters, at most {limit or 95})")
assert not asset.unpacked(target) and not asset.leftovers()
else:
assert (target / "tools" / "preflight.sh").is_file()
def test_the_release_workflow_fills_exactly_the_placeholders_the_scripts_keep():
workflow = Path(__file__).resolve().parents[3] / ".gitea" / "workflows" / "release.yml"
if not workflow.is_file():
pytest.skip("a distributed instance carries no release workflow")
text = workflow.read_text(encoding="utf-8")
def as_written_in_the_workflow(line: str) -> str:
return (line.replace("$Release", "\\$Release")
.replace(DOWNLOAD_BASE, "${download}")
.replace(f"chemenu-stack-{RELEASE_VERSION}", "${name}"))
for script, pairs in PLACEHOLDERS.items():
source = (TOOLS / script).read_text(encoding="utf-8")
assert f'tools/{script} > "${{BUILD_DIR}}/{script}"' in text
for empty, filled in pairs:
assert source.count(empty) == 1, f"{script} must hold {empty} exactly once"
expression = f"s|^{as_written_in_the_workflow(empty)}|{as_written_in_the_workflow(filled)}|"
assert expression in text, f"release.yml does not run {expression}"
assert 'download="${PUBLIC_BASE_URL}/${GITHUB_REPOSITORY}/releases/download/${TAG}"' in text
assert 'for asset in "${NAME}.tar.gz" "${NAME}.tar.gz.sha256" preflight.sh preflight.ps1; do' in text
# --- the launcher -----------------------------------------------------------------
+171 -1
View File
@@ -13,17 +13,21 @@ compares them byte for byte.
"""
from __future__ import annotations
import functools
import http.server
import json
import os
import shutil
import subprocess
import threading
from pathlib import Path
import pytest
from chemenu import prerequisites
from chemenu.tests.test_preflight import (
ECHO_PYTHON, SHELLS, TOOLS, Machine, _machine_with_root_of, assert_guidance,
DOWNLOAD_BASE, ECHO_PYTHON, SHELLS, TOOLS, AssetMachine, Machine, _machine_with_root_of,
assert_guidance,
)
PWSH = shutil.which("pwsh")
@@ -195,6 +199,172 @@ def test_install_folder_limit_at_the_boundary(tmp_path, length, longpaths, expec
assert not (machine.tools / ".venv").exists()
# --- asset mode (D33, D38-D41) ------------------------------------------------------
#
# The sh tests in test_preflight.py cover the same cases against preflight.sh; what is
# here is the twin's own wiring - Get-FileHash, tar.exe, Invoke-WebRequest - and the
# hand-over to the tree copy in a child pwsh.
def _asset_run(asset: AssetMachine, *args: str) -> subprocess.CompletedProcess:
env = {
"PATH": os.pathsep.join(str(d) for d in asset.path_dirs),
"HOME": str(asset.base),
"PIP_LOG": str(asset.pip_log),
"DOTNET_CLI_TELEMETRY_OPTOUT": "1",
"POWERSHELL_TELEMETRY_OPTOUT": "1",
**asset.extra_env,
}
return subprocess.run(
[PWSH, "-NoProfile", "-ExecutionPolicy", "Bypass", "-File", str(asset.download / "preflight.ps1"), *args],
capture_output=True, text=True, env=env, timeout=120, cwd=asset.cwd,
)
@pytest.fixture
def asset(tmp_path: Path) -> AssetMachine:
return AssetMachine(tmp_path.resolve())
def test_asset_unpacks_next_to_itself_and_runs_the_tree_copy(asset):
result = _asset_run(asset, "--archive", str(asset.tarball))
assert result.returncode == 0, result.stdout + result.stderr
assert "sha256 OK" in result.stdout and "Preflight passed" in result.stdout
assert (asset.tools / "preflight.ps1").is_file()
assert asset.recorded()["complete"] is True
assert not asset.leftovers()
def test_into_chooses_the_target_and_a_relative_one_follows_the_working_directory(asset):
far = asset.base / "deep" / "er" / "wiki"
assert _asset_run(asset, "--archive", str(asset.tarball), "--into", str(far)).returncode == 0
assert (far / "tools" / "preflight.ps1").is_file() and not asset.unpacked()
asset.cwd = asset.base
assert _asset_run(asset, "--archive", str(asset.tarball), "--into", "here").returncode == 0
assert (asset.base / "here" / "tools" / "preflight.ps1").is_file()
def test_an_existing_target_is_refused_and_left_alone(asset):
asset.root.mkdir()
(asset.root / "mine.txt").write_text("keep", encoding="utf-8")
result = _asset_run(asset, "--archive", str(asset.tarball))
assert result.returncode == 1
assert "already exists" in result.stderr and "--into" in result.stderr
assert [path.name for path in asset.root.iterdir()] == ["mine.txt"]
def test_a_wrong_sha256_exits_1_and_unpacks_nothing(asset):
checksum = asset.tarball.parent / f"{asset.tarball.name}.sha256"
checksum.write_text("0" * 64 + f" {asset.tarball.name}\n", encoding="utf-8")
result = _asset_run(asset, "--archive", str(asset.tarball))
assert result.returncode == 1
assert "does not match" in result.stderr and "nothing was unpacked" in result.stderr
assert not asset.unpacked() and not asset.leftovers()
def test_archive_without_a_checksum_file_exits_1(asset):
(asset.tarball.parent / f"{asset.tarball.name}.sha256").unlink()
result = _asset_run(asset, "--archive", str(asset.tarball))
assert result.returncode == 1 and ".sha256 is missing" in result.stderr
assert not asset.unpacked()
def test_an_archive_with_two_top_level_folders_exits_1(tmp_path):
asset = AssetMachine(tmp_path.resolve(), tops=("chemenu-stack-9.9.9", "stray"))
result = _asset_run(asset, "--archive", str(asset.tarball))
assert result.returncode == 1 and "exactly one top-level folder" in result.stderr
assert not asset.unpacked() and not asset.leftovers()
def test_a_copy_without_download_addresses_says_it_is_no_release_asset(asset):
result = _asset_run(asset)
assert result.returncode == 1 and "does not come from a release" in result.stderr
assert not asset.unpacked()
def test_a_missing_tar_is_a_stop_with_guidance(asset):
(asset.sysbin / "tar").unlink()
result = _asset_run(asset, "--archive", str(asset.tarball))
assert result.returncode == 42
assert_guidance(result.stdout, "(tar) could not be found")
assert not asset.unpacked()
def test_set_and_the_exit_code_pass_through_to_the_tree_copy(asset):
refused = _asset_run(asset, "--archive", str(asset.tarball), "--set", f"rg={asset.base / 'nowhere' / 'rg'}")
assert refused.returncode == 42
assert_guidance(refused.stdout, "The path given for ripgrep (rg) does not work")
assert (asset.tools / "preflight.ps1").is_file() and not asset.tools_file.exists()
second = AssetMachine(asset.base / "second")
elsewhere = second.stub("rg", "#!/bin/sh\necho 'ripgrep 14.1.1'\n", asset.base / "opt")
ok = _asset_run(second, "--archive", str(second.tarball), "--set", f"rg={elsewhere}")
assert ok.returncode == 0, ok.stdout
assert second.recorded()["tools"]["rg"] == str(elsewhere)
@pytest.mark.parametrize("flag", ["--into", "--archive"])
def test_asset_options_are_a_usage_error_inside_a_tree(machine, flag):
result = _preflight(machine, flag, str(machine.base))
assert result.returncode == 1 and "belong to the release asset" in result.stderr
@pytest.mark.parametrize("length, longpaths, limit, expected", [
(95, "0", None, 0),
(96, "0", None, 42),
(96, "1", None, 0),
(110, "0", 120, 0),
(121, "0", 120, 42),
])
def test_the_folder_limit_is_judged_at_the_final_target_from_the_archive(
tmp_path, length, longpaths, limit, expected):
base = tmp_path.resolve()
name_length = length - len(str(base / "download")) - 1
assert name_length > 0, "tmp_path is too long for this test"
asset = AssetMachine(base, limit=limit)
asset.standard(pwsh=True)
asset.extra_env.update({"CHEMENU_PREFLIGHT_PLATFORM": "windows", "CHEMENU_PREFLIGHT_LONGPATHS": longpaths})
target = asset.download / ("t" * name_length)
result = _asset_run(asset, "--archive", str(asset.tarball), "--into", str(target))
assert result.returncode == expected, result.stdout + result.stderr
if expected == 42:
assert_guidance(result.stdout, f"too long ({length} characters, at most {limit or 95})")
assert not asset.unpacked(target) and not asset.leftovers()
else:
assert (target / "tools" / "preflight.ps1").is_file()
@pytest.fixture
def release_server(asset):
handler = functools.partial(http.server.SimpleHTTPRequestHandler, directory=str(asset.tarball.parent))
handler.log_message = lambda *args: None
server = http.server.ThreadingHTTPServer(("127.0.0.1", 0), handler)
threading.Thread(target=server.serve_forever, daemon=True).start()
script = asset.download / "preflight.ps1"
script.write_text(
(TOOLS / "preflight.ps1").read_text(encoding="utf-8").replace(
"$ReleaseArchiveUrl = \'\'", f"$ReleaseArchiveUrl = \'http://127.0.0.1:{server.server_port}/{asset.tarball.name}\'").replace(
"$ReleaseChecksumUrl = \'\'", f"$ReleaseChecksumUrl = \'http://127.0.0.1:{server.server_port}/{asset.tarball.name}.sha256\'"),
encoding="utf-8")
yield asset
server.shutdown()
def test_download_fetches_both_files_and_unpacks(release_server):
result = _asset_run(release_server)
assert result.returncode == 0, result.stdout + result.stderr
assert "Downloading http://127.0.0.1" in result.stdout and "sha256 OK" in result.stdout
assert (release_server.tools / "preflight.ps1").is_file()
assert not release_server.leftovers()
def test_a_failed_download_exits_1_and_unpacks_nothing(release_server):
(release_server.tarball.parent / f"{release_server.tarball.name}.sha256").unlink()
result = _asset_run(release_server)
assert result.returncode == 1 and "the download failed" in result.stderr
assert not release_server.unpacked() and not release_server.leftovers()
# --- execution policy and Mark of the Web (D16, T6) --------------------------------
OPEN = "MachinePolicy=Undefined,UserPolicy=Undefined,Process=Undefined,CurrentUser=Undefined,LocalMachine=RemoteSigned"
+239 -49
View File
@@ -5,6 +5,17 @@
# pwsh -NoProfile -ExecutionPolicy Bypass -File tools/preflight.ps1
# pwsh -NoProfile -ExecutionPolicy Bypass -File tools/preflight.ps1 --set rg=C:\Tools\rg.exe
#
# As the release asset `preflight.ps1` - a copy with no tools/prerequisites.txt next
# to it - the script is the first install step instead: it downloads the stack
# release named in $ReleaseArchiveUrl / $ReleaseChecksumUrl, checks the sha256,
# unpacks it into <script folder>\chemenu (or --into <path>), and runs the copy of
# this script inside the unpacked tree, which does everything above.
#
# pwsh -NoProfile -ExecutionPolicy Bypass -File preflight.ps1 [--into <path>] [--archive <tarball>]
#
# --archive uses a local tarball instead of a download; <tarball>.sha256 has to lie
# next to it. Release builds fill the two URL lines below; a tree copy leaves them empty.
#
# Always start it that way (instructions/preflight.md): the bypass holds for this one
# process only and changes no setting, and it is what lets a script that carries a
# Mark of the Web start at all, so that it can report its own mark.
@@ -13,7 +24,8 @@
# Exit 42: the user has to act. The output says what, why, the command that fixes
# it and what happens next; show it verbatim and wait (AGENTS.md, Tool
# error contract). Never install anything on the user's behalf.
# Exit 1: this script was called wrongly, or the tree next to it is incomplete.
# Exit 1: this script was called wrongly, a download or unpack failed (nothing is
# unpacked then), or the tree next to it is incomplete.
#
# The counterpart of tools/preflight.sh, and the two answer the same questions from the
# same list, tools/prerequisites.txt. What only this one checks: the PowerShell execution
@@ -40,6 +52,10 @@ $Requirements = Join-Path $Dir 'requirements.txt'
$Stamp = Join-Path $Venv '.chemenu-requirements.sha256'
$Self = 'pwsh -NoProfile -ExecutionPolicy Bypass -File tools/preflight.ps1'
# Filled in by the release build, in the copy attached to the release; empty in a tree.
$ReleaseArchiveUrl = ''
$ReleaseChecksumUrl = ''
$PyProbe = "import sys; print(str(sys.version_info[0]) + '.' + str(sys.version_info[1])); print(sys.executable)"
function Write-Line {
@@ -55,6 +71,8 @@ function Write-ErrorLine {
# --- arguments ----------------------------------------------------------------
$Sets = @{}
$Into = ''
$Archive = ''
$index = 0
while ($index -lt $args.Count) {
$arg = [string]$args[$index]
@@ -68,12 +86,30 @@ while ($index -lt $args.Count) {
$given = [string]$args[$index]
} elseif ($arg.StartsWith('--set=')) {
$given = $arg.Substring(6)
} elseif ($arg -eq '--into' -or $arg -eq '--archive') {
if ($index + 1 -ge $args.Count -or -not [string]$args[$index + 1]) {
Write-ErrorLine "preflight: $arg needs a path"
exit 1
}
$index++
if ($arg -eq '--into') {
$Into = [string]$args[$index]
} else {
$Archive = [string]$args[$index]
}
$index++
continue
} elseif ($arg -eq '-h' -or $arg -eq '--help') {
Write-Line "usage: $Self [--set <tool>=<path>]..."
Write-Line ' preflight.ps1 [--into <path>] [--archive <tarball>] [--set <tool>=<path>]...'
Write-Line ''
Write-Line 'Checks the tools this stack needs (tools/prerequisites.txt), records their paths'
Write-Line 'in .wikitool-tools.json and sets up tools/.venv. Exit 0 means ready; exit 42'
Write-Line 'means the user has to act - the output says how.'
Write-Line ''
Write-Line 'The second form is the release asset: it downloads the release (or takes'
Write-Line '--archive), checks its sha256, unpacks it into <script folder>\chemenu or --into,'
Write-Line 'and runs the preflight inside it.'
exit 0
} else {
Write-ErrorLine "preflight: unknown argument: $arg"
@@ -88,8 +124,9 @@ while ($index -lt $args.Count) {
$index++
}
if (-not (Test-Path -LiteralPath $Manifest -PathType Leaf)) {
Write-ErrorLine "preflight: $Manifest is missing - this script has to run from inside an unpacked stack tree."
$AssetMode = -not (Test-Path -LiteralPath $Manifest -PathType Leaf)
if (-not $AssetMode -and ($Into -or $Archive)) {
Write-ErrorLine 'preflight: --into and --archive belong to the release asset; inside a stack tree there is nothing to unpack.'
exit 1
}
@@ -132,6 +169,205 @@ function Exit-WithGuide {
exit 42
}
# Runs a program; its standard output joined by newlines, or $null when it did not
# start or did not exit 0.
function Invoke-Native {
param([string]$Path, [string[]]$Arguments = @())
try {
$output = & $Path @Arguments 2>$null
if ($LASTEXITCODE -ne 0) {
return $null
}
return (@($output | ForEach-Object { "$_".TrimEnd("`r") }) -join "`n")
} catch {
return $null
}
}
# Same, but with the error stream merged in, for the messages the user is shown.
function Invoke-NativeVerbose {
param([string]$Path, [string[]]$Arguments = @())
try {
$output = & $Path @Arguments 2>&1
return [pscustomobject]@{
Code = $LASTEXITCODE
Output = (@($output | ForEach-Object { "$_".TrimEnd("`r") }) -join "`n")
}
} catch {
return [pscustomobject]@{ Code = -1; Output = $_.Exception.Message }
}
}
# --- asset mode: the release asset unpacks the stack, then hands over ------------------
#
# Only when no tools/prerequisites.txt lies next to this script. Nothing here reads
# the tree; the folder limit comes out of the archive itself. The tools the tree copy
# checks (Python, git, rg) are not needed until that copy runs.
function Test-LongPathsEnabled {
if ($env:CHEMENU_PREFLIGHT_LONGPATHS) {
return $env:CHEMENU_PREFLIGHT_LONGPATHS -eq '1'
}
# An unreadable key counts as "off": the limit then protects a machine it did not
# have to.
if (-not $IsWindows) {
return $false
}
try {
$value = Get-ItemPropertyValue -LiteralPath 'HKLM:\SYSTEM\CurrentControlSet\Control\FileSystem' -Name LongPathsEnabled
return $value -eq 1
} catch {
return $false
}
}
function Stop-Asset {
param([string]$Message)
Write-ErrorLine "preflight: $Message"
exit 1
}
function Resolve-AssetPath {
param([string]$Path)
if (-not [IO.Path]::IsPathRooted($Path)) {
$Path = Join-Path (Get-Location).ProviderPath $Path
}
return [IO.Path]::GetFullPath($Path).TrimEnd('\', '/')
}
function Invoke-AssetMode {
$target = if ($Into) { Resolve-AssetPath $Into } else { Join-Path $Dir 'chemenu' }
if ($null -ne (Get-Item -LiteralPath $target -Force -ErrorAction SilentlyContinue)) {
Stop-Asset "$target already exists - nothing was unpacked. Choose another folder with --into <path>, or move the existing one away."
}
if (-not $Archive -and (-not $ReleaseArchiveUrl -or -not $ReleaseChecksumUrl)) {
Stop-Asset 'this copy of the script does not come from a release (it has no download address). Download preflight.ps1 from the release page, or pass --archive <tarball>.'
}
if ($Archive) {
$Archive = Resolve-AssetPath $Archive
if (-not (Test-Path -LiteralPath $Archive -PathType Leaf)) {
Stop-Asset "--archive: $Archive is not a file."
}
if (-not (Test-Path -LiteralPath "$Archive.sha256" -PathType Leaf)) {
Stop-Asset "--archive: $Archive.sha256 is missing - the checksum file has to lie next to the tarball."
}
}
$tar = Get-Command tar -CommandType Application -ErrorAction SilentlyContinue | Select-Object -First 1
if (-not $tar) {
Add-Problem 'The tool needed to unpack the stack (tar) could not be found.' `
'the first step downloads the release, checks its checksum and unpacks it' `
'Windows 10 and 11 ship tar.exe in C:\Windows\System32 - if it is missing, repair or update Windows.'
Exit-WithGuide
}
$work = Join-Path ([IO.Path]::GetTempPath()) "chemenu-preflight.$([guid]::NewGuid().ToString('N').Substring(0, 8))"
$unpack = ''
try {
$null = New-Item -ItemType Directory -Path $work
if ($Archive) {
$archivePath = $Archive
$checksumPath = "$Archive.sha256"
} else {
$archivePath = Join-Path $work ($ReleaseArchiveUrl.Split('/')[-1])
$checksumPath = Join-Path $work ($ReleaseChecksumUrl.Split('/')[-1])
Write-Line "Downloading $ReleaseArchiveUrl"
$ProgressPreference = 'SilentlyContinue'
foreach ($pair in @(@($ReleaseArchiveUrl, $archivePath), @($ReleaseChecksumUrl, $checksumPath))) {
try {
Invoke-WebRequest -Uri $pair[0] -OutFile $pair[1]
} catch {
Stop-Asset "the download failed: $($pair[0]) ($($_.Exception.Message)) - check the internet connection, then run this again."
}
}
}
$first = Get-Content -LiteralPath $checksumPath -TotalCount 1 -Encoding utf8
$match = [regex]::Match("$first", '^([0-9A-Fa-f]{64})')
if (-not $match.Success) {
Stop-Asset 'the checksum file holds no sha256 in its first line - nothing was unpacked.'
}
$want = $match.Groups[1].Value.ToLowerInvariant()
$have = (Get-FileHash -LiteralPath $archivePath -Algorithm SHA256).Hash.ToLowerInvariant()
if ($want -ne $have) {
Stop-Asset "the sha256 of the archive does not match its checksum file (expected $want, got $have) - nothing was unpacked. Delete the download and run this again."
}
Write-Line 'sha256 OK'
$listing = Invoke-Native $tar.Source @('-tzf', $archivePath)
if ($null -eq $listing) {
Stop-Asset 'the archive could not be read - nothing was unpacked.'
}
$tops = @(
$listing -split "`n" |
ForEach-Object { ($_ -replace '\\', '/' -replace '^\./', '').Split('/')[0] } |
Where-Object { $_ -and $_ -ne '.' } |
Select-Object -Unique
)
if ($tops.Count -ne 1) {
Stop-Asset 'the archive does not hold exactly one top-level folder - nothing was unpacked.'
}
$top = $tops[0]
$manifestText = Invoke-Native $tar.Source @('-xOzf', $archivePath, "$top/tools/prerequisites.txt")
if ($null -eq $manifestText) {
Stop-Asset "the archive holds no $top/tools/prerequisites.txt - nothing was unpacked."
}
$limit = 0
foreach ($line in ($manifestText -split "`n")) {
if ($line -match '^limit\|install_dir_max\|(\d+)') {
$limit = [int]$Matches[1]
break
}
}
if ($Platform -eq 'windows' -and $limit -gt 0 -and -not (Test-LongPathsEnabled)) {
$length = $target.Length
if ($length -gt $limit) {
Add-Problem "The folder this wiki would be installed in is too long ($length characters, at most $limit): $target" `
"Windows on this computer only allows paths of up to 259 characters, and the wiki's own files need the rest" `
"Run this again with a shorter folder, for example: --into C:\Chemenu`nAlternatively, someone with administrator rights can turn on long paths in Windows."
Exit-WithGuide
}
}
$parent = Split-Path -Parent $target
$null = New-Item -ItemType Directory -Path $parent -Force
$unpack = Join-Path $parent ".chemenu-unpack.$PID"
$null = New-Item -ItemType Directory -Path $unpack
$null = Invoke-NativeVerbose $tar.Source @('-xzf', $archivePath, '-C', $unpack)
if (-not (Test-Path -LiteralPath (Join-Path $unpack $top) -PathType Container)) {
Stop-Asset "unpacking failed - the target was not created."
}
Move-Item -LiteralPath (Join-Path $unpack $top) -Destination $target
} finally {
foreach ($leftover in @($work, $unpack)) {
if ($leftover -and (Test-Path -LiteralPath $leftover)) {
Remove-Item -LiteralPath $leftover -Recurse -Force -ErrorAction SilentlyContinue
}
}
}
$treeScript = Join-Path $target 'tools/preflight.ps1'
if (-not (Test-Path -LiteralPath $treeScript -PathType Leaf)) {
Stop-Asset 'the unpacked stack has no tools/preflight.ps1.'
}
Write-Line "Unpacked into $target."
Write-Line 'If a later step stops, run tools/preflight.ps1 from inside that folder.'
Write-Line ''
$passed = @()
foreach ($name in $Sets.Keys) {
$passed += "--set=$name=$($Sets[$name])"
}
& ([Environment]::ProcessPath) -NoProfile -ExecutionPolicy Bypass -File $treeScript @passed
exit $LASTEXITCODE
}
if ($AssetMode) {
Invoke-AssetMode
}
# --- the manifest ---------------------------------------------------------------
$ManifestLines = @(
@@ -299,35 +535,6 @@ function Find-OnPath {
return $found
}
# Runs a program; its standard output joined by newlines, or $null when it did not
# start or did not exit 0.
function Invoke-Native {
param([string]$Path, [string[]]$Arguments = @())
try {
$output = & $Path @Arguments 2>$null
if ($LASTEXITCODE -ne 0) {
return $null
}
return (@($output | ForEach-Object { "$_".TrimEnd("`r") }) -join "`n")
} catch {
return $null
}
}
# Same, but with the error stream merged in, for the messages the user is shown.
function Invoke-NativeVerbose {
param([string]$Path, [string[]]$Arguments = @())
try {
$output = & $Path @Arguments 2>&1
return [pscustomobject]@{
Code = $LASTEXITCODE
Output = (@($output | ForEach-Object { "$_".TrimEnd("`r") }) -join "`n")
}
} catch {
return [pscustomobject]@{ Code = -1; Output = $_.Exception.Message }
}
}
# The value recorded for <name> in .wikitool-tools.json.
function Get-RecordedPath {
param([string]$Name)
@@ -511,23 +718,6 @@ foreach ($tool in $Tools) {
# --- install folder length (Windows, long paths off) ------------------------------
function Test-LongPathsEnabled {
if ($env:CHEMENU_PREFLIGHT_LONGPATHS) {
return $env:CHEMENU_PREFLIGHT_LONGPATHS -eq '1'
}
# An unreadable key counts as "off": the limit then protects a machine it did not
# have to.
if (-not $IsWindows) {
return $false
}
try {
$value = Get-ItemPropertyValue -LiteralPath 'HKLM:\SYSTEM\CurrentControlSet\Control\FileSystem' -Name LongPathsEnabled
return $value -eq 1
} catch {
return $false
}
}
if ($Platform -eq 'windows' -and -not (Test-LongPathsEnabled)) {
$limit = [int](Get-ManifestField 'limit' 'install_dir_max' 3)
$length = $Root.Length
+199 -27
View File
@@ -5,11 +5,23 @@
# tools/preflight.sh check, record, set up
# tools/preflight.sh --set rg=/opt/rg/rg use a path the user named
#
# As the release asset `preflight.sh` - a copy with no tools/prerequisites.txt next
# to it - the script is the first install step instead: it downloads the stack
# release named in RELEASE_ARCHIVE_URL / RELEASE_CHECKSUM_URL, checks the sha256,
# unpacks it into <script folder>/chemenu (or --into <path>), and runs the copy of
# this script inside the unpacked tree, which does everything above.
#
# preflight.sh [--into <path>] [--archive <tarball>] [--set <tool>=<path>]...
#
# --archive uses a local tarball instead of a download; <tarball>.sha256 has to lie
# next to it. Release builds fill the two URL lines below; a tree copy leaves them empty.
#
# Exit 0: everything is in place and .wikitool-tools.json is complete.
# Exit 42: the user has to act. The output says what, why, the command that fixes
# it and what happens next; show it verbatim and wait (AGENTS.md § Tool
# error contract). Never install anything on the user's behalf.
# Exit 1: this script was called wrongly, or the tree next to it is incomplete.
# Exit 1: this script was called wrongly, a download or unpack failed (nothing is
# unpacked then), or the tree next to it is incomplete.
#
# POSIX sh on purpose: it has to run before Python is known to exist, under dash,
# bash and the Git Bash that Claude Code uses on Windows. The PowerShell
@@ -31,23 +43,41 @@ REQUIREMENTS="$DIR/requirements.txt"
STAMP="$VENV/.chemenu-requirements.sha256"
SELF="tools/preflight.sh"
# Filled in by the release build, in the copy attached to the release; empty in a tree.
RELEASE_ARCHIVE_URL=''
RELEASE_CHECKSUM_URL=''
PYPROBE='import sys; print("%d.%d" % sys.version_info[:2]); print(sys.executable)'
usage() {
cat <<'EOF'
usage: tools/preflight.sh [--set <tool>=<path>]...
preflight.sh [--into <path>] [--archive <tarball>] [--set <tool>=<path>]...
Checks the tools this stack needs (tools/prerequisites.txt), records their paths
in .wikitool-tools.json and sets up tools/.venv. Exit 0 means ready; exit 42
means the user has to act - the output says how.
The second form is the release asset: it downloads the release (or takes
--archive), checks its sha256, unpacks it into <script folder>/chemenu or --into,
and runs the preflight inside it.
EOF
}
# --- arguments ----------------------------------------------------------------
SETS=""
INTO="" ARCHIVE=""
while [ $# -gt 0 ]; do
case "$1" in
--into|--archive)
if [ $# -lt 2 ] || [ -z "$2" ]; then
echo "preflight: $1 needs a path" >&2
exit 1
fi
if [ "$1" = --into ]; then INTO=$2; else ARCHIVE=$2; fi
shift
;;
--set)
if [ $# -lt 2 ]; then
echo "preflight: --set needs <tool>=<path>" >&2
@@ -65,8 +95,11 @@ ${1#--set=}" ;;
shift
done
ASSET=0
if [ ! -f "$MANIFEST" ]; then
echo "preflight: $MANIFEST is missing - this script has to run from inside an unpacked stack tree." >&2
ASSET=1
elif [ -n "$INTO$ARCHIVE" ]; then
echo "preflight: --into and --archive belong to the release asset; inside a stack tree there is nothing to unpack." >&2
exit 1
fi
@@ -124,6 +157,170 @@ stop() {
exit 42
}
longpaths_enabled() {
if [ -n "${CHEMENU_PREFLIGHT_LONGPATHS:-}" ]; then
[ "$CHEMENU_PREFLIGHT_LONGPATHS" = 1 ]
return
fi
# MSYS would rewrite the `/v` into a path without the exclusion. An unreadable
# key counts as "off": the limit then protects a machine it did not have to.
answer=$(MSYS2_ARG_CONV_EXCL='*' reg query 'HKLM\SYSTEM\CurrentControlSet\Control\FileSystem' \
/v LongPathsEnabled 2>/dev/null) || return 1
case "$answer" in *0x1*) return 0 ;; esac
return 1
}
# Length in UTF-16 code units, which is what MAX_PATH counts.
utf16_length() {
if command -v iconv >/dev/null 2>&1; then
bytes=$(printf '%s' "$1" | iconv -f UTF-8 -t UTF-16LE 2>/dev/null | wc -c | tr -d ' ')
if [ "${bytes:-0}" -gt 0 ]; then
echo $((bytes / 2))
return
fi
fi
echo ${#1}
}
# --- asset mode: the release asset unpacks the stack, then hands over ------------------
#
# Only when no tools/prerequisites.txt lies next to this script. Nothing here reads
# the tree; the folder limit comes out of the archive itself. The tools the tree copy
# checks (Python, git, rg) are not needed until that copy runs.
WORK=""
cleanup() {
[ -z "$WORK" ] || rm -rf "$WORK"
[ -z "${UNPACK:-}" ] || rm -rf "$UNPACK"
}
asset_fail() { # <message>
echo "preflight: $1" >&2
exit 1
}
sha256_of() { # <file>
if command -v sha256sum >/dev/null 2>&1; then
sha256sum "$1" | cut -d' ' -f1
else
shasum -a 256 "$1" | cut -d' ' -f1
fi
}
asset_mode() {
# GNU tar reads `C:` as a host name; Git Bash has to hand it `/c/...`.
if [ "$PLATFORM" = windows ] && command -v cygpath >/dev/null 2>&1; then
[ -z "$ARCHIVE" ] || ARCHIVE=$(cygpath -u "$ARCHIVE")
[ -z "$INTO" ] || INTO=$(cygpath -u "$INTO")
fi
if [ -n "$INTO" ]; then
case "$INTO" in /*) TARGET=$INTO ;; *) TARGET="$(pwd)/$INTO" ;; esac
else
TARGET="$DIR/chemenu"
fi
TARGET=${TARGET%/}
if [ -e "$TARGET" ] || [ -L "$TARGET" ]; then
asset_fail "$(native_path "$TARGET") already exists - nothing was unpacked. Choose another folder with --into <path>, or move the existing one away."
fi
if [ -z "$ARCHIVE" ] && { [ -z "$RELEASE_ARCHIVE_URL" ] || [ -z "$RELEASE_CHECKSUM_URL" ]; }; then
asset_fail "this copy of the script does not come from a release (it has no download address). Download preflight.sh from the release page, or pass --archive <tarball>."
fi
if [ -n "$ARCHIVE" ]; then
[ -f "$ARCHIVE" ] || asset_fail "--archive: $ARCHIVE is not a file."
[ -f "$ARCHIVE.sha256" ] || asset_fail "--archive: $ARCHIVE.sha256 is missing - the checksum file has to lie next to the tarball."
fi
# What has to be here for the download, tested before anything is fetched.
missing=""
if [ -z "$ARCHIVE" ] && ! command -v curl >/dev/null 2>&1; then missing="$missing curl"; fi
command -v tar >/dev/null 2>&1 || missing="$missing tar"
command -v sha256sum >/dev/null 2>&1 || command -v shasum >/dev/null 2>&1 || missing="$missing sha256sum"
if [ -n "$missing" ]; then
case "$PLATFORM" in
macos) hint="brew install$missing" ;;
windows) hint="Git for Windows (https://gitforwindows.org) provides all of them in Git Bash." ;;
*) hint="sudo apt install$(printf '%s' "$missing" | sed 's/ sha256sum/ coreutils/')" ;;
esac
problem "Tools needed to fetch and unpack the stack could not be found:$missing" \
"the first step downloads the release, checks its checksum and unpacks it" \
"$hint"
stop
fi
trap cleanup EXIT
WORK=$(mktemp -d "${TMPDIR:-/tmp}/chemenu-preflight.XXXXXX") || asset_fail "could not create a temporary folder."
if [ -n "$ARCHIVE" ]; then
archive=$ARCHIVE
checksum=$ARCHIVE.sha256
else
archive="$WORK/${RELEASE_ARCHIVE_URL##*/}"
checksum="$WORK/${RELEASE_CHECKSUM_URL##*/}"
echo "Downloading $RELEASE_ARCHIVE_URL"
curl -fsSL -o "$archive" "$RELEASE_ARCHIVE_URL" || asset_fail "the download failed: $RELEASE_ARCHIVE_URL (check the internet connection, then run this again)."
curl -fsSL -o "$checksum" "$RELEASE_CHECKSUM_URL" || asset_fail "the download failed: $RELEASE_CHECKSUM_URL (check the internet connection, then run this again)."
fi
want=$(sed -n '1s/^\([0-9A-Fa-f]\{64\}\).*/\1/p' "$checksum" | tr 'A-F' 'a-f')
[ -n "$want" ] || asset_fail "the checksum file holds no sha256 in its first line - nothing was unpacked."
have=$(sha256_of "$archive" | tr 'A-F' 'a-f')
if [ "$want" != "$have" ]; then
asset_fail "the sha256 of the archive does not match its checksum file (expected $want, got $have) - nothing was unpacked. Delete the download and run this again."
fi
echo "sha256 OK"
listing=$(tar -tzf "$archive" 2>/dev/null) || asset_fail "the archive could not be read - nothing was unpacked."
tops=$(printf '%s\n' "$listing" | sed 's|^\./||; s|/.*||; /^\.\{0,1\}$/d')
top=$(printf '%s\n' "$tops" | head -n 1)
other=$(printf '%s\n' "$tops" | grep -v -x -F -- "$top" | head -n 1)
if [ -z "$top" ] || [ -n "$other" ]; then
asset_fail "the archive does not hold exactly one top-level folder - nothing was unpacked."
fi
manifest=$(tar -xOzf "$archive" "$top/tools/prerequisites.txt" 2>/dev/null) \
|| asset_fail "the archive holds no $top/tools/prerequisites.txt - nothing was unpacked."
limit=$(printf '%s\n' "$manifest" | sed -n 's/^limit|install_dir_max|\([0-9][0-9]*\).*/\1/p' | head -n 1)
if [ "$PLATFORM" = windows ] && [ -n "$limit" ] && ! longpaths_enabled; then
folder=$(native_path "$TARGET")
length=$(utf16_length "$folder")
if [ "$length" -gt "$limit" ]; then
problem "The folder this wiki would be installed in is too long ($length characters, at most $limit): $folder" \
"Windows on this computer only allows paths of up to 259 characters, and the wiki's own files need the rest" \
"Run this again with a shorter folder, for example: --into C:\\\\Chemenu
Alternatively, someone with administrator rights can turn on long paths in Windows."
stop
fi
fi
parent=$(dirname -- "$TARGET")
mkdir -p "$parent" || asset_fail "could not create $parent."
UNPACK="$parent/.chemenu-unpack.$$"
mkdir "$UNPACK" || asset_fail "could not create a temporary folder next to the target."
tar -xzf "$archive" -C "$UNPACK" || asset_fail "unpacking failed - the target was not created."
[ -d "$UNPACK/$top" ] || asset_fail "unpacking produced no $top folder - the target was not created."
mv "$UNPACK/$top" "$TARGET" || asset_fail "could not move the unpacked stack to $(native_path "$TARGET")."
rm -rf "$UNPACK"
UNPACK=""
[ -f "$TARGET/tools/preflight.sh" ] || asset_fail "the unpacked stack has no tools/preflight.sh."
echo "Unpacked into $(native_path "$TARGET")."
echo "If a later step stops, run tools/preflight.sh from inside that folder."
echo ""
set --
while IFS= read -r entry; do
[ -n "$entry" ] || continue
set -- "$@" --set "$entry"
done <<EOT
$SETS
EOT
trap - EXIT
cleanup
exec /bin/sh "$TARGET/tools/preflight.sh" "$@"
}
[ "$ASSET" -eq 0 ] || asset_mode
# --- the manifest ---------------------------------------------------------------
manifest_field() { # <kind> <name> <field number, 1-based>
@@ -398,31 +595,6 @@ done
# --- install folder length (Windows, long paths off) ------------------------------
longpaths_enabled() {
if [ -n "${CHEMENU_PREFLIGHT_LONGPATHS:-}" ]; then
[ "$CHEMENU_PREFLIGHT_LONGPATHS" = 1 ]
return
fi
# MSYS would rewrite the `/v` into a path without the exclusion. An unreadable
# key counts as "off": the limit then protects a machine it did not have to.
answer=$(MSYS2_ARG_CONV_EXCL='*' reg query 'HKLM\SYSTEM\CurrentControlSet\Control\FileSystem' \
/v LongPathsEnabled 2>/dev/null) || return 1
case "$answer" in *0x1*) return 0 ;; esac
return 1
}
# Length in UTF-16 code units, which is what MAX_PATH counts.
utf16_length() {
if command -v iconv >/dev/null 2>&1; then
bytes=$(printf '%s' "$1" | iconv -f UTF-8 -t UTF-16LE 2>/dev/null | wc -c | tr -d ' ')
if [ "${bytes:-0}" -gt 0 ]; then
echo $((bytes / 2))
return
fi
fi
echo ${#1}
}
if [ "$PLATFORM" = windows ] && ! longpaths_enabled; then
limit=$(manifest_field limit install_dir_max 3)
folder=$(native_path "$ROOT")