diff --git a/CHANGES.md b/CHANGES.md index 9d103ce..9c7ae1e 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -14,7 +14,7 @@ place only: `instructions/dev/version-parts.md` § The candidate model. --- -## 9.0.0-beta.10 - 2026-10-07 - Rebase-Review Gate: Anzeigename in Hilfe- und Fehlertexten von sync und publish angeglichen +## 9.0.0-beta.11 - 2026-10-07 - Asset-Preflight: Verschieben aus dem Entpackordner wird bis ca. 30 s wiederholt, nennt die Systemmeldung und rollt nach endgültigem Scheitern zurück **Author:** Torben Nehmer @@ -34,6 +34,7 @@ place only: `instructions/dev/version-parts.md` § The candidate model. - INSTALL.md is an English installation tutorial; new tutorial from a first source to a first answer with a shipped sample; README.md a landing page - Die Formen der vier handgeschriebenen .wikitool-*.json stehen je in einem Datensatz (config_shapes), aus dem Lader, doctor und die Konfigurationsreferenz lesen - Gate-Liste: AGENTS.md § Gates ist die eine Liste (mit Rebase-Review Gate), cli_contract.GATES hält sie, docs verify vergleicht beide +- Asset-Preflight: Verschieben aus dem Entpackordner wird bis ca. 30 s wiederholt, nennt die Systemmeldung und rollt nach endgültigem Scheitern zurück **Low impact** - version bump: Abschlussmeldung nennt Stichpunkt oder Themenabsatz statt pauschal Prosa diff --git a/INSTALL.md b/INSTALL.md index c667989..f8e0955 100644 --- a/INSTALL.md +++ b/INSTALL.md @@ -133,6 +133,11 @@ are done; then it checks again. It may not work around the stop or install anyth connection and let it try again; the agent does not look for another way to download. Without a direct download you can put the tarball and its `.sha256` from the release page side by side yourself; the agent then runs the script with `--archive `. +- **A part of the stack could not be moved into place** (exit 1, "could not move ... into"). + Usually a virus scanner briefly holding a freshly unpacked file open; the script has already + waited about 30 seconds for it. If the message says "nothing was installed", the script has + taken everything back - let the agent run it again. If it names entries it could not take + back, empty the folder yourself (keep `.git`), then run it again. **In the unpacked tree** (every further run is `tools/preflight.sh`, or under PowerShell `pwsh -NoProfile -ExecutionPolicy Bypass -File tools/preflight.ps1`): diff --git a/VERSION b/VERSION index 1121bb6..9ec2163 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -9.0.0-beta.10 +9.0.0-beta.11 diff --git a/instructions/preflight.md b/instructions/preflight.md index 97f10a0..f922604 100644 --- a/instructions/preflight.md +++ b/instructions/preflight.md @@ -118,6 +118,13 @@ It is safe to run at any time: a second run on a ready checkout changes nothing - A checksum that does not match, a failed download, and a copy of the script that carries no download address (it was not taken from a release) exit 1 with nothing unpacked; report the message, and do not fetch the tarball by another route. + - A part of the stack that will not move from the unpack folder into place - on Windows a + virus scanner can hold a freshly unpacked file open for a moment - is retried for about 30 + seconds. If it still fails, the script takes back everything it moved and exits 1 with + "nothing was installed" and the system's message. Report that message; the folder is as it + was, and the user may run the script again. If the message instead names entries it could + not take back, the folder is no longer empty: whether to empty it (keeping `.git`) is the + user's decision, as above - do not delete anything yourself. - On POSIX, `curl`, `tar` and `sha256sum` (or `shasum`) have to exist; when one does not, the script stops with exit 42 like any other missing tool. The PowerShell script needs nothing beyond what Windows ships. diff --git a/tools/CONTRACT.md b/tools/CONTRACT.md index abcb337..04060c3 100644 --- a/tools/CONTRACT.md +++ b/tools/CONTRACT.md @@ -72,8 +72,10 @@ Until it has passed, every `tools/wikitool` call exits 42 and names it. A release also carries both scripts as assets (`preflight.sh`, `preflight.ps1`) for the very first install, before there is a tree: run from a folder with no `tools/prerequisites.txt` beside -them, they download and verify the release tarball, unpack it into `chemenu/` next to themselves -(or `--into `), and run the preflight inside the unpacked tree. +them, they download and verify the release tarball, unpack it into the folder they lie in (or +`--into `), and run the preflight inside the unpacked tree. A part of the stack that will +not move into place is retried for about 30 seconds; if it still fails, they take back what they +moved and exit 1, leaving the folder as it was. ## Usage diff --git a/tools/chemenu/tests/conftest.py b/tools/chemenu/tests/conftest.py index 2bd1a0a..1ee92b5 100644 --- a/tools/chemenu/tests/conftest.py +++ b/tools/chemenu/tests/conftest.py @@ -37,6 +37,7 @@ _WIKITOOL_ENV = ( "CHEMENU_PREFLIGHT_LONGPATHS", "CHEMENU_PREFLIGHT_POLICY", "CHEMENU_PREFLIGHT_MARKED", + "CHEMENU_PREFLIGHT_MOVE_FAIL", ) + tuple(var for var, _harness in HARNESS_ENV_VARS) # Environment git reads for identity or for where its repo lives. A stray diff --git a/tools/chemenu/tests/test_preflight.py b/tools/chemenu/tests/test_preflight.py index d6a4263..9e6417f 100644 --- a/tools/chemenu/tests/test_preflight.py +++ b/tools/chemenu/tests/test_preflight.py @@ -19,6 +19,7 @@ import stat import subprocess import sys import tarfile +import time from pathlib import Path import pytest @@ -607,6 +608,100 @@ def test_set_and_the_exit_code_pass_through_to_the_tree_copy(asset, shell): assert second.recorded()["tools"]["rg"] == str(elsewhere) +# --- an entry that will not move up (Gitea #197) ------------------------------------- +# +# CHEMENU_PREFLIGHT_MOVE_FAIL stands in for the file a virus scanner holds open for a +# moment; with :partial each failed attempt has already moved one file of the entry up, +# which is what a move that falls back to copying and deleting leaves behind. + +ARCHIVED_TOOLS = sorted((*STACK_FILES, "requirements.txt")) +MOVE_ATTEMPTS = 8 + + +def tools_as_unpacked(root: Path) -> list[str]: + """Every path below /tools, the tree copy's own .venv left out.""" + tools = root / "tools" + return sorted(path.relative_to(tools).as_posix() for path in tools.rglob("*") + if path.relative_to(tools).parts[0] != ".venv") + + +def folder_state(folder: Path) -> dict[str, bytes | None]: + """What a folder holds, file contents included - to compare before against after.""" + return {path.relative_to(folder).as_posix(): (path.read_bytes() if path.is_file() else None) + for path in sorted(folder.rglob("*"))} + + +@pytest.mark.parametrize("shell", SHELLS) +@pytest.mark.parametrize("partial", ["", ":partial"]) +def test_a_move_that_fails_twice_is_retried_into_a_whole_entry(asset, shell, partial): + asset.extra_env["CHEMENU_PREFLIGHT_MOVE_FAIL"] = f"tools:2{partial}" + result = asset.run("--archive", str(asset.tarball), shell=shell) + assert result.returncode == 0, result.stdout + result.stderr + assert "Preflight passed" in result.stdout + assert tools_as_unpacked(asset.root) == ARCHIVED_TOOLS + assert not asset.leftovers() + + +@pytest.mark.parametrize("shell", SHELLS) +@pytest.mark.parametrize("partial", ["", ":partial"]) +@pytest.mark.parametrize("into", [False, True]) +def test_a_move_that_never_works_is_taken_back(asset, shell, partial, into): + target = asset.base / "wiki" if into else asset.root + (target / ".git").mkdir(parents=True) + (target / ".git" / "HEAD").write_text("ref: refs/heads/main\n", encoding="utf-8") + before = folder_state(target) + asset.extra_env["CHEMENU_PREFLIGHT_MOVE_FAIL"] = f"tools:always{partial}" + args = ("--archive", str(asset.tarball), *(("--into", str(target)) if into else ())) + + started = time.monotonic() + result = asset.run(*args, shell=shell) + assert result.returncode == 1, result.stdout + result.stderr + # Eight attempts, and none of the 30 seconds of pauses between them under the hook. + assert time.monotonic() - started < 20 + assert "could not move tools into" in result.stderr + assert f"Permission denied (CHEMENU_PREFLIGHT_MOVE_FAIL, attempt {MOVE_ATTEMPTS})" in result.stderr + assert "nothing was installed" in result.stderr + assert folder_state(target) == before + assert not asset.leftovers() + + del asset.extra_env["CHEMENU_PREFLIGHT_MOVE_FAIL"] + again = asset.run(*args, shell=shell) + assert again.returncode == 0, again.stdout + again.stderr + assert tools_as_unpacked(target) == ARCHIVED_TOOLS + + +def add_to_release(asset: AssetMachine, files: dict[str, str]) -> None: + """Rebuild the asset's release with (paths below the top folder) added.""" + stage = asset.base / "restage" + with tarfile.open(asset.tarball, "r:gz") as archive: + archive.extractall(stage, filter="data") + top = stage / f"chemenu-stack-{RELEASE_VERSION}" + for name, text in files.items(): + (top / name).parent.mkdir(parents=True, exist_ok=True) + (top / name).write_text(text, encoding="utf-8") + with tarfile.open(asset.tarball, "w:gz") as archive: + archive.add(top, arcname=top.name) + digest = hashlib.sha256(asset.tarball.read_bytes()).hexdigest() + (asset.tarball.parent / f"{asset.tarball.name}.sha256").write_text( + f"{digest} {asset.tarball.name}\n", encoding="utf-8") + + +# Moved before tools/ by both scripts: the glob sorts them first, Get-ChildItem lists +# folders first and kb/ sorts before tools/. +MOVED_FIRST = {"AGENTS.md": "# agents\n", "kb/index.md": "# index\n"} + + +@pytest.mark.parametrize("shell", SHELLS) +def test_an_entry_that_moved_before_the_failure_is_taken_back_too(asset, shell): + add_to_release(asset, MOVED_FIRST) + asset.extra_env["CHEMENU_PREFLIGHT_MOVE_FAIL"] = "tools:always" + result = asset.run("--archive", str(asset.tarball), shell=shell) + assert result.returncode == 1, result.stdout + result.stderr + assert "nothing was installed" in result.stderr + assert sorted(path.name for path in asset.root.iterdir()) == [asset.script.name] + assert not asset.leftovers() + + @pytest.mark.parametrize("shell", SHELLS) @pytest.mark.parametrize("flag", ["--into", "--archive"]) def test_asset_options_are_a_usage_error_inside_a_tree(machine, shell, flag): diff --git a/tools/chemenu/tests/test_preflight_pwsh.py b/tools/chemenu/tests/test_preflight_pwsh.py index a7e6b17..93e6c54 100644 --- a/tools/chemenu/tests/test_preflight_pwsh.py +++ b/tools/chemenu/tests/test_preflight_pwsh.py @@ -20,14 +20,15 @@ import os import shutil import subprocess import threading +import time from pathlib import Path import pytest from chemenu import prerequisites from chemenu.tests.test_preflight import ( - DOWNLOAD_BASE, ECHO_PYTHON, SHELLS, TOOLS, AssetMachine, Machine, _machine_with_root_of, - assert_guidance, + ARCHIVED_TOOLS, DOWNLOAD_BASE, ECHO_PYTHON, MOVE_ATTEMPTS, MOVED_FIRST, SHELLS, TOOLS, AssetMachine, + Machine, _machine_with_root_of, add_to_release, assert_guidance, folder_state, tools_as_unpacked, ) PWSH = shutil.which("pwsh") @@ -311,6 +312,53 @@ def test_set_and_the_exit_code_pass_through_to_the_tree_copy(asset): assert second.recorded()["tools"]["rg"] == str(elsewhere) +@pytest.mark.parametrize("partial", ["", ":partial"]) +def test_a_move_that_fails_twice_is_retried_into_a_whole_entry(asset, partial): + asset.extra_env["CHEMENU_PREFLIGHT_MOVE_FAIL"] = f"tools:2{partial}" + result = _asset_run(asset, "--archive", str(asset.tarball)) + assert result.returncode == 0, result.stdout + result.stderr + assert "Preflight passed" in result.stdout + assert tools_as_unpacked(asset.root) == ARCHIVED_TOOLS + assert not asset.leftovers() + + +@pytest.mark.parametrize("partial", ["", ":partial"]) +@pytest.mark.parametrize("into", [False, True]) +def test_a_move_that_never_works_is_taken_back(asset, partial, into): + target = asset.base / "wiki" if into else asset.root + (target / ".git").mkdir(parents=True) + (target / ".git" / "HEAD").write_text("ref: refs/heads/main\n", encoding="utf-8") + before = folder_state(target) + asset.extra_env["CHEMENU_PREFLIGHT_MOVE_FAIL"] = f"tools:always{partial}" + args = ("--archive", str(asset.tarball), *(("--into", str(target)) if into else ())) + + started = time.monotonic() + result = _asset_run(asset, *args) + assert result.returncode == 1, result.stdout + result.stderr + # Eight attempts, and none of the 30 seconds of pauses between them under the hook. + assert time.monotonic() - started < 20 + assert "could not move tools into" in result.stderr + assert f"is denied. (CHEMENU_PREFLIGHT_MOVE_FAIL, attempt {MOVE_ATTEMPTS})" in result.stderr + assert "nothing was installed" in result.stderr + assert folder_state(target) == before + assert not asset.leftovers() + + del asset.extra_env["CHEMENU_PREFLIGHT_MOVE_FAIL"] + again = _asset_run(asset, *args) + assert again.returncode == 0, again.stdout + again.stderr + assert tools_as_unpacked(target) == ARCHIVED_TOOLS + + +def test_an_entry_that_moved_before_the_failure_is_taken_back_too(asset): + add_to_release(asset, MOVED_FIRST) + asset.extra_env["CHEMENU_PREFLIGHT_MOVE_FAIL"] = "tools:always" + result = _asset_run(asset, "--archive", str(asset.tarball)) + assert result.returncode == 1, result.stdout + result.stderr + assert "nothing was installed" in result.stderr + assert sorted(path.name for path in asset.root.iterdir()) == [asset.script.name] + assert not asset.leftovers() + + @pytest.mark.parametrize("flag", ["--into", "--archive"]) def test_asset_options_are_a_usage_error_inside_a_tree(machine, flag): result = _preflight(machine, flag, str(machine.base)) diff --git a/tools/preflight.ps1 b/tools/preflight.ps1 index c43cfd8..934a404 100644 --- a/tools/preflight.ps1 +++ b/tools/preflight.ps1 @@ -27,7 +27,8 @@ # it and what happens next; show it verbatim and wait (AGENTS.md, Tool # error contract). Never install anything on the user's behalf. # Exit 1: this script was called wrongly, a download or unpack failed (nothing is -# unpacked then), or the tree next to it is incomplete. +# unpacked then - or, should even that fail, the message names what is +# left), or the tree next to it is incomplete. # # The counterpart of tools/preflight.sh, and the two answer the same questions from the # same list, tools/prerequisites.txt. What only this one checks: the PowerShell execution @@ -35,11 +36,15 @@ # tools/wikitool.ps1 from starting, and that exist only here. Windows PowerShell 5.1 is # not supported; `#Requires` turns it away. # -# Four variables exist for the test suite and are read nowhere else: +# Five variables exist for the test suite and are read nowhere else: # CHEMENU_PREFLIGHT_PLATFORM (windows|macos|linux), CHEMENU_PREFLIGHT_LONGPATHS (0|1), # CHEMENU_PREFLIGHT_POLICY (`Scope=Policy,...`, as `Get-ExecutionPolicy -List` names them) # and CHEMENU_PREFLIGHT_MARKED (comma-separated script names below tools/) stand in for # the operating system, the registry, the policy and the file streams. +# CHEMENU_PREFLIGHT_MOVE_FAIL (:[:partial]) stands in for a file some other +# program holds open: moving the unpacked up fails the first times (`always`: +# every time), with :partial after moving a part of it, and the pauses between attempts +# are skipped. Set-StrictMode -Version Latest $ErrorActionPreference = 'Stop' @@ -237,23 +242,100 @@ function Resolve-AssetPath { return [IO.Path]::GetFullPath($Path).TrimEnd('\', '/') } -# Whether the install folder holds anything besides this script and a .git - the two -# things an empty folder may already carry: the script was downloaded into it, and the -# user may have cloned the instance's own, still empty repository there. -function Test-TargetOccupied { - param([string]$Target) - $selfName = Split-Path -Leaf $PSCommandPath - $isOwnFolder = [IO.Path]::GetFullPath($Target).TrimEnd('\', '/') -eq [IO.Path]::GetFullPath($Dir).TrimEnd('\', '/') - foreach ($entry in @(Get-ChildItem -LiteralPath $Target -Force)) { - if ($entry.Name -eq '.git') { - continue - } - if ($isOwnFolder -and $entry.Name -eq $selfName) { - continue - } +# This script and a .git are the two things an empty install folder may already carry: +# the script was downloaded into it, and the user may have cloned the instance's own, +# still empty repository there. Neither counts as occupying the folder, and neither is +# ever removed from it. +function Test-ProtectedName { + param([string]$Target, [string]$Name) + if ($Name -eq '.git') { return $true } - return $false + $isOwnFolder = [IO.Path]::GetFullPath($Target).TrimEnd('\', '/') -eq [IO.Path]::GetFullPath($Dir).TrimEnd('\', '/') + return $isOwnFolder -and $Name -eq (Split-Path -Leaf $PSCommandPath) +} + +# The names in the install folder besides those two. +function Get-TargetOccupant { + param([string]$Target) + return @(Get-ChildItem -LiteralPath $Target -Force | + Where-Object { -not (Test-ProtectedName $Target $_.Name) } | + ForEach-Object { $_.Name }) +} + +# How long an entry that will not move up is retried: the pauses between attempts, in +# seconds - 30 in all. On Windows a folder cannot be renamed while any file in it is +# held open, and a virus scanner or indexer looking at a freshly unpacked tree holds one +# for a moment; waiting is what gets past it. +$MovePauses = @(1, 2, 3, 4, 5, 6, 9) + +# One attempt at moving the unpacked entry up into the target; the system's +# message when it fails, $null when it worked. A failed attempt can leave part of the +# entry on either side - a move that falls back to copying and deleting stops wherever +# the lock is - so every attempt after the first starts over: what reached the target is +# removed, and the entry is unpacked afresh from the archive. Only then is the entry in +# the target, once and whole, after the attempt that works. +function Move-AssetEntry { + param([string]$Name, [int]$Attempt, [string]$Target, [string]$Unpack, [string]$Top, [string]$Tar, [string]$ArchivePath) + $source = Join-Path (Join-Path $Unpack $Top) $Name + $landed = Join-Path $Target $Name + try { + if ($Attempt -gt 1) { + if ((Test-Path -LiteralPath $landed) -and -not (Test-ProtectedName $Target $Name)) { + Remove-Item -LiteralPath $landed -Recurse -Force + } + $again = Invoke-NativeVerbose $Tar @('-xzf', $ArchivePath, '-C', $Unpack, "$Top/$Name") + if ($again.Code -ne 0) { + return "unpacking it again failed: $($again.Output)" + } + } + $hook = "$env:CHEMENU_PREFLIGHT_MOVE_FAIL".Split(':') + if ($hook[0] -eq $Name -and $hook.Count -gt 1 -and ($hook[1] -eq 'always' -or $Attempt -le [int]$hook[1])) { + if ($hook.Count -gt 2 -and $hook[2] -eq 'partial') { + if (Test-Path -LiteralPath $source -PathType Container) { + $null = New-Item -ItemType Directory -Path $landed -Force + $part = Get-ChildItem -LiteralPath $source -Force | Select-Object -First 1 + if ($part) { + Move-Item -LiteralPath $part.FullName -Destination $landed + } + } else { + $null = New-Item -ItemType File -Path $landed -Force + } + } + return "Access to the path '$source' is denied. (CHEMENU_PREFLIGHT_MOVE_FAIL, attempt $Attempt)" + } + Move-Item -LiteralPath $source -Destination $Target + return $null + } catch { + return $_.Exception.Message + } +} + +# An entry would not move even after waiting: take back everything this run moved into +# the target, so that it holds what it held before and running this again is all it +# takes. Only names that came out of the archive are removed, never a protected one. +function Undo-AssetMove { + param([string]$Target, [string]$Unpack, [string[]]$Moved, [string]$Name, [string]$Message) + foreach ($entry in $Moved + $Name) { + $path = Join-Path $Target $entry + if (-not (Test-ProtectedName $Target $entry) -and (Test-Path -LiteralPath $path)) { + try { + Remove-Item -LiteralPath $path -Recurse -Force + } catch { + $null = $_ + } + } + } + try { + Remove-Item -LiteralPath $Unpack -Recurse -Force + } catch { + $null = $_ + } + $left = @(Get-TargetOccupant $Target) + if ($left.Count -eq 0) { + Exit-Asset "could not move $Name into $Target ($Message) - nothing was installed, the folder is as it was before. Run this again." + } + Exit-Asset "could not move $Name into $Target ($Message), and could not take back what was already there: $($left -join ', ') - empty the folder (keep .git), then run this again." } function Invoke-AssetMode { @@ -262,7 +344,7 @@ function Invoke-AssetMode { if ($null -ne $existing -and -not $existing.PSIsContainer) { Exit-Asset "$target exists and is not a folder - nothing was unpacked." } - if ($null -ne $existing -and (Test-TargetOccupied $target)) { + if ($null -ne $existing -and @(Get-TargetOccupant $target).Count -gt 0) { Exit-Asset "$target is not empty - nothing was unpacked. The wiki installs into an empty folder (an empty git clone is fine): put this script into one and run it there, or pass --into ." } @@ -367,12 +449,23 @@ function Invoke-AssetMode { if (-not (Test-Path -LiteralPath $unpacked -PathType Container)) { Exit-Asset 'unpacking failed - nothing was installed.' } + $moved = @() foreach ($entry in @(Get-ChildItem -LiteralPath $unpacked -Force)) { - try { - Move-Item -LiteralPath $entry.FullName -Destination $target - } catch { - Exit-Asset "could not move $($entry.Name) into $target - the folder holds part of the stack now; empty it (keep .git) and run this again." + $attempt = 1 + while ($true) { + $failure = Move-AssetEntry $entry.Name $attempt $target $unpack $top $tar.Source $archivePath + if ($null -eq $failure) { + break + } + if ($attempt -gt $MovePauses.Count) { + Undo-AssetMove $target $unpack $moved $entry.Name $failure + } + if (-not $env:CHEMENU_PREFLIGHT_MOVE_FAIL) { + Start-Sleep -Seconds $MovePauses[$attempt - 1] + } + $attempt++ } + $moved += $entry.Name } } finally { foreach ($leftover in @($work, $unpack)) { diff --git a/tools/preflight.sh b/tools/preflight.sh index 0186471..991b758 100755 --- a/tools/preflight.sh +++ b/tools/preflight.sh @@ -23,16 +23,21 @@ # it and what happens next; show it verbatim and wait (AGENTS.md § Tool # error contract). Never install anything on the user's behalf. # Exit 1: this script was called wrongly, a download or unpack failed (nothing is -# unpacked then), or the tree next to it is incomplete. +# unpacked then - or, should even that fail, the message names what is +# left), or the tree next to it is incomplete. # # POSIX sh on purpose: it has to run before Python is known to exist, under dash, # bash and the Git Bash that Claude Code uses on Windows. The PowerShell # counterpart is tools/preflight.ps1; what both check is tools/prerequisites.txt. # The procedure around this script: instructions/preflight.md. # -# Two variables exist for the test suite and are read nowhere else: +# Three variables exist for the test suite and are read nowhere else: # CHEMENU_PREFLIGHT_PLATFORM (windows|macos|linux) and -# CHEMENU_PREFLIGHT_LONGPATHS (0|1) stand in for `uname` and the registry. +# CHEMENU_PREFLIGHT_LONGPATHS (0|1) stand in for `uname` and the registry; +# CHEMENU_PREFLIGHT_MOVE_FAIL (:[:partial]) stands in for a file some other +# program holds open: moving the unpacked up fails the first times (`always`: +# every time), with :partial after moving a part of it, and the pauses between attempts +# are skipped. set -u @@ -209,21 +214,87 @@ sha256_of() { # fi } -# Whether the install folder holds anything besides this script and a .git - the -# two things an empty folder may already carry: the script was downloaded into it, and -# the user may have cloned the instance's own, still empty repository there. -target_is_occupied() { - [ -d "$TARGET" ] || return 1 - self_name=${0##*/} - target_real=$(CDPATH='' cd -- "$TARGET" && pwd -P) - for entry in "$TARGET"/* "$TARGET"/.[!.]* "$TARGET"/..?*; do - [ -e "$entry" ] || [ -L "$entry" ] || continue - name=${entry##*/} - [ "$name" = .git ] && continue - [ "$target_real" = "$DIR" ] && [ "$name" = "$self_name" ] && continue - return 0 +# This script and a .git are the two things an empty install folder may already carry: +# the script was downloaded into it, and the user may have cloned the instance's own, +# still empty repository there. Neither counts as occupying the folder, and neither is +# ever removed from it. +is_protected() { # + [ "$1" = .git ] && return 0 + [ "$1" = "${0##*/}" ] && [ "$(CDPATH='' cd -- "$TARGET" && pwd -P)" = "$DIR" ] +} + +# The names in the install folder besides those two, one per line. +occupants() { + [ -d "$TARGET" ] || return 0 + for occupant in "$TARGET"/* "$TARGET"/.[!.]* "$TARGET"/..?*; do + [ -e "$occupant" ] || [ -L "$occupant" ] || continue + is_protected "${occupant##*/}" || printf '%s\n' "${occupant##*/}" done - return 1 +} + +# How long an entry that will not move up is retried: the pauses between attempts, in +# seconds - 30 in all. On Windows a folder cannot be renamed while any file in it is +# held open, and a virus scanner or indexer looking at a freshly unpacked tree holds one +# for a moment; waiting is what gets past it. +MOVE_PAUSES='1 2 3 4 5 6 9' + +# One attempt at moving the unpacked entry up into the target; on failure the +# system's message is in MOVE_ERROR. A failed attempt can leave part of the entry on +# either side - a move that falls back to copying and deleting stops wherever the lock +# is - so every attempt after the first starts over: what reached the target is removed, +# and the entry is unpacked afresh from the archive. Only then is the entry in the +# target, once and whole, after the attempt that works. +move_entry() { # + from="$UNPACK/$top/$1" + landed="$TARGET/$1" + if [ "$2" -gt 1 ]; then + if { [ -e "$landed" ] || [ -L "$landed" ]; } && ! is_protected "$1"; then + MOVE_ERROR=$(rm -rf -- "$landed" 2>&1) || return 1 + fi + MOVE_ERROR=$(tar -xzf "$archive" -C "$UNPACK" "$top/$1" 2>&1) \ + || { MOVE_ERROR="unpacking it again failed: $MOVE_ERROR"; return 1; } + fi + hook=${CHEMENU_PREFLIGHT_MOVE_FAIL:-} + times=${hook#*:} + case "$hook" in + "$1":always*|"$1":[0-9]*) + if [ "${times%%:*}" = always ] || [ "$2" -le "${times%%:*}" ]; then + case "$hook" in + *:partial) + if [ -d "$from" ]; then + mkdir -p "$landed" + for part in "$from"/*; do + [ -e "$part" ] && mv -- "$part" "$landed/" + break + done + else + : > "$landed" + fi ;; + esac + MOVE_ERROR="mv: cannot move '$from': Permission denied (CHEMENU_PREFLIGHT_MOVE_FAIL, attempt $2)" + return 1 + fi ;; + esac + MOVE_ERROR=$(mv -- "$from" "$TARGET/" 2>&1) +} + +# An entry would not move even after waiting: take back everything this run moved into +# the target, so that it holds what it held before and running this again is all it +# takes. Only names that came out of the archive are removed, never a protected one. +undo_move() { # + while IFS= read -r undone; do + [ -n "$undone" ] || continue + is_protected "$undone" || rm -rf -- "${TARGET:?}/$undone" 2>/dev/null + done </dev/null && UNPACK="" + left=$(occupants | tr '\n' ' ') + if [ -z "$left" ]; then + asset_fail "could not move $1 into $(native_path "$TARGET") ($MOVE_ERROR) - nothing was installed, the folder is as it was before. Run this again." + fi + asset_fail "could not move $1 into $(native_path "$TARGET") ($MOVE_ERROR), and could not take back what was already there: ${left% } - empty the folder (keep .git), then run this again." } asset_mode() { @@ -241,7 +312,7 @@ asset_mode() { if { [ -e "$TARGET" ] || [ -L "$TARGET" ]; } && [ ! -d "$TARGET" ]; then asset_fail "$(native_path "$TARGET") exists and is not a folder - nothing was unpacked." fi - if target_is_occupied; then + if [ -n "$(occupants)" ]; then asset_fail "$(native_path "$TARGET") is not empty - nothing was unpacked. The wiki installs into an empty folder (an empty git clone is fine): put this script into one and run it there, or pass --into ." fi @@ -323,9 +394,18 @@ Alternatively, someone with administrator rights can turn on long paths in Windo mkdir "$UNPACK" || asset_fail "could not create a temporary folder in the target." tar -xzf "$archive" -C "$UNPACK" || asset_fail "unpacking failed - nothing was installed." [ -d "$UNPACK/$top" ] || asset_fail "unpacking produced no $top folder - nothing was installed." + MOVED="" for entry in "$UNPACK/$top"/* "$UNPACK/$top"/.[!.]* "$UNPACK/$top"/..?*; do [ -e "$entry" ] || [ -L "$entry" ] || continue - mv "$entry" "$TARGET/" || asset_fail "could not move ${entry##*/} into $(native_path "$TARGET") - the folder holds part of the stack now; empty it (keep .git) and run this again." + attempt=1 + until move_entry "${entry##*/}" "$attempt"; do + pause=$(printf '%s\n' $MOVE_PAUSES | sed -n "${attempt}p") + [ -n "$pause" ] || undo_move "${entry##*/}" + [ -n "${CHEMENU_PREFLIGHT_MOVE_FAIL:-}" ] || sleep "$pause" + attempt=$((attempt + 1)) + done + MOVED="$MOVED +${entry##*/}" done rm -rf "$UNPACK" UNPACK=""