torben
c65d559690
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SnAJ7Z3CpVD3PRbN73QtU2 Files changed: - .gitea/workflows/ci.yml
398 lines
19 KiB
YAML
398 lines
19 KiB
YAML
# CI for the wiki stack.
|
|
#
|
|
# `verify` is the pipeline: one job, stopping at the first failure - the stack
|
|
# has no artifact to build and nothing to deploy, so its whole job is "does the
|
|
# machinery still hold together, and does the distribution it produces still
|
|
# work". `pwsh` beside it is the PowerShell half of the same question (Gitea
|
|
# #151): the same preflight and launcher, under the PowerShell 7 that Windows
|
|
# harnesses start them with, in the prebuilt `chemenu-ci-pwsh` image
|
|
# (`pwsh-ci-image.yml`). It runs on Linux, so what only a Windows machine can
|
|
# answer - the registry, the Store alias, a real Mark of the Web - is covered by
|
|
# the environment hooks `tools/preflight.ps1` documents, not by this job.
|
|
#
|
|
# Runner: `linux-docker` is one of this Gitea instance's three routing labels
|
|
# (alongside `container-builder` and `k3s-deploy`). The job image is named
|
|
# explicitly rather than inherited from the runner's label mapping, which is
|
|
# not documented anywhere: `debian:trixie-slim` is the base the instance's
|
|
# container-build workflows already use, and Trixie's python3 is 3.13, past
|
|
# the 3.11 floor `doctor` enforces.
|
|
#
|
|
# Pinning that image makes `nodejs` this workflow's own responsibility.
|
|
# `actions/checkout` is a JavaScript action, and act_runner runs it with `node`
|
|
# *inside the job container* - a slim Debian has none, and the job dies with
|
|
# exit 127 before any step of ours runs. The shape below (apt `nodejs` first,
|
|
# then `checkout@v7`) is the one proven on this instance by
|
|
# torben/gitea-mcp@ci-build, workflow `ci-build.yaml`, runs 42-45.
|
|
#
|
|
# Triggers: content commits are excluded. `publish` touches kb/, raw/ and work/
|
|
# and never the stack, so running the suite for them would be pure noise. The
|
|
# exclusions are deliberately literal rather than a `!**/CONTRACT.md` negation,
|
|
# whose support in Gitea's filter matching is unverified: every pattern here
|
|
# names content, so anything unanticipated still triggers CI. The list is
|
|
# repeated rather than shared through a YAML anchor for the same reason -
|
|
# GitHub's parser rejects anchors outright, and Gitea's is not documented to
|
|
# accept them. `kb/CONTRACT.md` is absent on purpose: it is a stack file that
|
|
# happens to live under a content directory, and it must keep its CI.
|
|
#
|
|
# That the filter works is now observed, not assumed (Gitea issue #11): commit
|
|
# f916376 published only kb/ and raw/ paths and produced no run at all, while
|
|
# the stack commits on either side of it (adfa220, 40adbb7) each produced two.
|
|
# Gitea evaluates these patterns the way GitHub does. Do not re-derive this.
|
|
|
|
name: CI
|
|
|
|
on:
|
|
push:
|
|
branches: [main]
|
|
paths-ignore:
|
|
- 'kb/*/**'
|
|
- 'kb/index.md'
|
|
- 'kb/log.md'
|
|
- 'kb/provenance.md'
|
|
- 'raw/*/**'
|
|
- 'work/*/**'
|
|
- 'reports/*/**'
|
|
pull_request:
|
|
branches: [main]
|
|
paths-ignore:
|
|
- 'kb/*/**'
|
|
- 'kb/index.md'
|
|
- 'kb/log.md'
|
|
- 'kb/provenance.md'
|
|
- 'raw/*/**'
|
|
- 'work/*/**'
|
|
- 'reports/*/**'
|
|
workflow_dispatch:
|
|
|
|
jobs:
|
|
verify:
|
|
runs-on: linux-docker
|
|
container:
|
|
image: debian:trixie-slim
|
|
env:
|
|
# Scope the Iteration Budget Gate to this run instead of letting it fall
|
|
# back to the parent PID, and keep the trace out of the checkout so the
|
|
# working tree stays clean for the ignore-rule checks.
|
|
WIKITOOL_SESSION_ID: ci-${{ github.run_id }}
|
|
WIKI_TRACE_DIR: /tmp/wikitool-trace
|
|
BUILD_DIR: /tmp/build
|
|
INSTANCE_DIR: /tmp/instance
|
|
|
|
steps:
|
|
- name: System dependencies
|
|
# `nodejs` is not for us - it is what act_runner needs to execute the
|
|
# JavaScript action in the next step. It has to be installed before the
|
|
# checkout, which is why this step comes first.
|
|
run: |
|
|
set -eu
|
|
apt-get update -qq
|
|
apt-get install -y --no-install-recommends \
|
|
python3 python3-venv git nodejs ripgrep ca-certificates
|
|
rm -rf /var/lib/apt/lists/*
|
|
|
|
- uses: actions/checkout@v7
|
|
with:
|
|
# The version gate diffs against the pushed range's base, so the
|
|
# shallow default clone is not enough.
|
|
fetch-depth: 0
|
|
|
|
- name: Tool environment
|
|
# The preflight, not a venv block of our own: it is the one way a
|
|
# checkout gets set up (instructions/preflight.md), and tools/wikitool
|
|
# refuses to start until it has passed. Run twice - the second run must
|
|
# pass without changing anything, which is what an instance relies on
|
|
# when it re-runs it after every update.
|
|
run: |
|
|
set -eu
|
|
git config --global --add safe.directory "$GITHUB_WORKSPACE"
|
|
tools/preflight.sh
|
|
cp .wikitool-tools.json /tmp/tools-first.json
|
|
tools/preflight.sh > /tmp/preflight-second.txt
|
|
cmp .wikitool-tools.json /tmp/tools-first.json
|
|
# pytest-cov is CI-only: tools/requirements.txt describes what an
|
|
# *instance* needs at runtime and ships with `dist export`, and an
|
|
# instance does not measure this suite. Installed beside pytest for
|
|
# the same reason pytest itself is.
|
|
tools/.venv/bin/python -m pip install --quiet pytest pytest-cov
|
|
# The MCP server's dependency is optional for an instance but not for
|
|
# CI: its tests skip without it, and a skipped golden test is exactly
|
|
# how the server's output and the CLI's would drift apart unnoticed.
|
|
tools/.venv/bin/python -m pip install --quiet -r tools/requirements-mcp.txt
|
|
|
|
- name: Tests
|
|
# Not run with WIKI_TRACE=0: two telemetry tests assert that a trace is
|
|
# written, and disabling the emitter globally fails them. The suite
|
|
# redirects WIKI_TRACE_DIR per test on its own.
|
|
#
|
|
# One run, not two. This job used to be the only place the suite met a
|
|
# machine with no global git configuration, which is how Gitea #8 was
|
|
# found - two tests that silently read the developer's `git config
|
|
# user.name`. That hole is now closed in the suite itself: the autouse
|
|
# `hermetic_environment` fixture gives every test an empty HOME, no
|
|
# git configuration and none of the tool's own environment, so this
|
|
# container is no longer a special environment worth a second run.
|
|
# See instructions/dev/testing-conventions.md.
|
|
#
|
|
# Coverage is measured and enforced at a floor of 85% against a measured
|
|
# 87.0% - `fail_under` in tools/.coveragerc, not a flag here, so the
|
|
# number sits next to the reasoning that produced it. It was set only
|
|
# after the number had been watched across 38 runs (Gitea #10, closed).
|
|
# A red suite from this floor means coverage actually fell; the two
|
|
# points of headroom already absorb a new thin Typer wrapper.
|
|
run: |
|
|
set -eu
|
|
cd tools
|
|
.venv/bin/python -m pytest -q \
|
|
--cov --cov-report=term --cov-report=xml --cov-report=html
|
|
|
|
- name: Start Radicale
|
|
# Its own step because the script hands the server's address to the suite through
|
|
# $GITHUB_ENV, which only reaches the steps after the one that wrote it.
|
|
run: |
|
|
set -eu
|
|
tools/.venv/bin/python -m pip install --quiet radicale
|
|
.gitea/scripts/start-radicale.sh tools/.venv/bin/python /tmp/radicale
|
|
|
|
- name: Live tracker suite (CalDAV)
|
|
# The one live tracker that needs no app and no display: a throwaway Radicale on
|
|
# loopback. `CHEMENU_LIVE_REQUIRE=caldav` turns "no server" into a failure - without
|
|
# it the suite would skip and stay green, which is exactly the outcome this step
|
|
# exists to rule out (Gitea #156). The Super Productivity half is nightly, in
|
|
# `tracker-live.yml`; see instructions/dev/tracker-testing.md.
|
|
env:
|
|
CHEMENU_LIVE_REQUIRE: caldav
|
|
run: |
|
|
set -eu
|
|
cd tools
|
|
.venv/bin/python -m pytest -q -m live_tracker -k caldav -s
|
|
|
|
- name: Coverage report
|
|
# `always()`: a red suite is exactly when the per-module numbers are
|
|
# worth reading, and the upload must not disappear with the failure.
|
|
# v3, not v4 - v4 is restricted on this Gitea instance; v3 is what is
|
|
# proven here (torben/gitea-mcp@ci-build, ci-build.yaml, runs
|
|
# 42-45).
|
|
#
|
|
# The artifact is downloadable from the run page, but the Actions
|
|
# artifact REST endpoints report `total_count: 0` for it - v3 writes
|
|
# through the older artifact API, which those endpoints do not read.
|
|
# An empty list is not a failed upload. See EVALS.md § "How much of the
|
|
# stack the suite reaches"; do not re-derive this.
|
|
if: always()
|
|
uses: actions/upload-artifact@v3
|
|
with:
|
|
name: coverage-${{ github.run_id }}
|
|
path: |
|
|
tools/coverage.xml
|
|
tools/htmlcov/
|
|
retention-days: 14
|
|
|
|
- name: Verify the development tree
|
|
run: |
|
|
set -eu
|
|
tools/wikitool docs verify
|
|
tools/wikitool instructions verify
|
|
tools/wikitool lint --fail-on-error
|
|
|
|
- name: Version gate
|
|
# A stack change with no version bump cannot be released, because the
|
|
# release would carry a change nobody named. Scoped to what
|
|
# `dist export` actually ships as behaviour - prose docs and these
|
|
# workflows are not in it, and a typo fix should not force a bump.
|
|
env:
|
|
BASE_SHA: ${{ github.event.pull_request.base.sha }}
|
|
BEFORE_SHA: ${{ github.event.before }}
|
|
run: |
|
|
set -eu
|
|
base="${BASE_SHA:-${BEFORE_SHA:-}}"
|
|
case "$base" in
|
|
""|0000000000000000000000000000000000000000)
|
|
echo "No base commit to compare against - skipping the version gate."
|
|
exit 0
|
|
;;
|
|
esac
|
|
if ! git cat-file -e "${base}^{commit}" 2>/dev/null; then
|
|
echo "Base commit $base is not in this clone - skipping the version gate."
|
|
exit 0
|
|
fi
|
|
|
|
changed="$(git diff --name-only "$base" HEAD)"
|
|
stack="$(printf '%s\n' "$changed" \
|
|
| grep -E '^(tools/|types/|instructions/|AGENTS\.md$|[^/]+/CONTRACT\.md$)' || true)"
|
|
|
|
if [ -z "$stack" ]; then
|
|
echo "No stack paths touched - no version bump required."
|
|
exit 0
|
|
fi
|
|
if printf '%s\n' "$changed" | grep -qx 'VERSION'; then
|
|
echo "Stack changed, and VERSION moved to $(cat VERSION)."
|
|
exit 0
|
|
fi
|
|
|
|
echo "Stack paths changed without a VERSION bump:"
|
|
printf '%s\n' "$stack" | sed 's/^/ /'
|
|
echo ""
|
|
echo 'Fix: tools/wikitool version bump --patch --title "<what changed>"'
|
|
echo 'Then `docs verify` holds VERSION and CHANGES.md together.'
|
|
exit 1
|
|
|
|
- name: Build a release tarball
|
|
# The same form release.yml builds - a `dist export` tree under exactly one
|
|
# top-level folder, plus its .sha256 - so the replay below starts where a
|
|
# user starts: from the archive, not from an exported tree.
|
|
run: |
|
|
set -eu
|
|
name=chemenu-stack-ci
|
|
mkdir -p "$BUILD_DIR"
|
|
tools/wikitool dist export "${BUILD_DIR}/${name}"
|
|
tar -czf "${BUILD_DIR}/${name}.tar.gz" -C "$BUILD_DIR" "$name"
|
|
( cd "$BUILD_DIR" && sha256sum "${name}.tar.gz" > "${name}.tar.gz.sha256" )
|
|
|
|
- name: The distribution works as a fresh instance
|
|
# Replays instructions/setup-instance.md end to end, minus its interactive
|
|
# decision points. What this tests is the release artifact as an
|
|
# artifact: the documented path from the preflight asset in an empty
|
|
# folder to a verified instance. Step 0 runs the asset with --archive
|
|
# instead of a download, which is the one difference from a real install.
|
|
#
|
|
# Personalization is stubbed the same way the identity is: the real
|
|
# step interviews the user, so CI substitutes a fixed answer - here,
|
|
# the template minus its sentinel line. That is deliberately the
|
|
# cheapest thing `doctor`'s personalization check accepts, because
|
|
# what is under test is that the export *carries* the templates, not
|
|
# what a person would write into them.
|
|
run: |
|
|
set -eu
|
|
mkdir -p "$INSTANCE_DIR"
|
|
cp tools/preflight.sh "$INSTANCE_DIR/preflight.sh"
|
|
cd "$INSTANCE_DIR"
|
|
sh preflight.sh --archive "${BUILD_DIR}/chemenu-stack-ci.tar.gz"
|
|
# Installed in place: the asset is gone, the tree is here.
|
|
test ! -e preflight.sh
|
|
test -f tools/preflight.sh
|
|
git init -q -b main
|
|
git config user.name "CI Instance"
|
|
git config user.email "ci@example.invalid"
|
|
for personal in USER SOUL; do
|
|
grep -v 'wikitool:template-unfilled' "$personal.md.template" > "$personal.md"
|
|
done
|
|
# The authoring conventions ride the same split one directory down,
|
|
# and are stubbed the same way: what is under test is that the export
|
|
# carries the templates and that `doctor`/`docs verify` accept an
|
|
# adopted one, not what a person would write into them. The collection
|
|
# contracts are adopted verbatim - the shipped text is a working
|
|
# default, unlike a personalization file.
|
|
grep -v 'wikitool:template-unfilled' kb/CONVENTIONS.md.template > kb/CONVENTIONS.md
|
|
tools/wikitool dist adopt
|
|
tools/wikitool instructions sync
|
|
tools/wikitool index rebuild
|
|
tools/wikitool sources rebuild-index
|
|
tools/wikitool doctor
|
|
tools/wikitool docs verify
|
|
tools/wikitool instructions verify
|
|
tools/wikitool lint --fail-on-error
|
|
tools/wikitool version show
|
|
# A fresh instance owes no migration: dist export declares its content
|
|
# version, so `status` must answer rather than ask for a baseline.
|
|
tools/wikitool migrate status
|
|
# Telemetry defaults off for a distributed instance (the
|
|
# .wikitool-release.json this export carries), so nothing above
|
|
# should have created a trace tree at all - see EVALS.md § "Whether
|
|
# it runs at all".
|
|
if [ -e reports/telemetry ]; then
|
|
echo "reports/telemetry/ exists in a fresh distributed instance - telemetry should default off"
|
|
exit 1
|
|
fi
|
|
|
|
- name: The bug-report collector works in the distribution
|
|
# tools/bugreport.py is the one part of the stack that must run when
|
|
# nothing else does, so it is exercised as shipped: from the export,
|
|
# through the tools/bugreport launcher the instructions name, which picks
|
|
# the runner's plain python3, in the instance the step above set up.
|
|
# The unit tests cover what it collects; this covers that it arrives.
|
|
run: |
|
|
set -eu
|
|
cd "$INSTANCE_DIR"
|
|
tools/bugreport --no-trace
|
|
bundle=$(ls -d reports/bugreport-*/ | head -n 1)
|
|
test -f "$bundle/MANIFEST.md"
|
|
test -f "$bundle/environment.json"
|
|
grep -q '`wikitool` started' "$bundle/MANIFEST.md"
|
|
ls reports/bugreport-*.zip
|
|
# The same collector with --pseudonymise: the mapping lies beside the
|
|
# bundle, not in it or its zip, and this container's hostname is gone.
|
|
rm -rf reports/bugreport-*
|
|
tools/bugreport --no-trace --pseudonymise
|
|
bundle=$(ls -d reports/bugreport-*/ | head -n 1)
|
|
bundle=${bundle%/}
|
|
test -f "$bundle.pseudonyms.json"
|
|
test -f "$bundle.review.txt"
|
|
grep -q 'stage 1 applied, stage 2 not yet applied' "$bundle/MANIFEST.md"
|
|
python3 - "$bundle" <<'PY'
|
|
import socket, sys, zipfile
|
|
from pathlib import Path
|
|
bundle = Path(sys.argv[1])
|
|
names = zipfile.ZipFile(str(bundle) + ".zip").namelist()
|
|
assert not [n for n in names if "pseudonyms" in n or "review" in n], names
|
|
host = socket.gethostname()
|
|
for path in bundle.rglob("*"):
|
|
if path.is_file():
|
|
assert host not in path.read_text(encoding="utf-8", errors="replace"), path
|
|
PY
|
|
|
|
pwsh:
|
|
runs-on: linux-docker
|
|
container:
|
|
image: gitea.nehmer.net/torben/chemenu-ci-pwsh:latest
|
|
env:
|
|
WIKITOOL_SESSION_ID: ci-pwsh-${{ github.run_id }}
|
|
WIKI_TRACE_DIR: /tmp/wikitool-trace
|
|
|
|
steps:
|
|
- uses: actions/checkout@v7
|
|
|
|
- name: PSScriptAnalyzer
|
|
# Positional arguments are excluded: the rule is written for cmdlets, and the two
|
|
# scripts call their own small helpers positionally throughout. Everything else the
|
|
# analyzer knows must stay silent, which includes the ASCII-only and approved-verb rules.
|
|
run: |
|
|
set -eu
|
|
pwsh -NoProfile -Command '
|
|
$found = foreach ($script in Get-ChildItem tools -Filter *.ps1) {
|
|
Invoke-ScriptAnalyzer -Path $script.FullName -ExcludeRule PSAvoidUsingPositionalParameters
|
|
}
|
|
$found | Format-List RuleName, ScriptName, Line, Message | Out-String -Width 200 | Write-Output
|
|
if (@($found).Count -gt 0) { exit 1 }
|
|
'
|
|
|
|
- name: Preflight, twice, against the POSIX one
|
|
# The two preflights answer the same questions from the same list and must write the
|
|
# same file: that is what keeps a tools/wikitool launched from either shell starting
|
|
# the same git, rg and Python. The second run must change nothing.
|
|
run: |
|
|
set -eu
|
|
git config --global --add safe.directory "$GITHUB_WORKSPACE"
|
|
pwsh -NoProfile -ExecutionPolicy Bypass -File tools/preflight.ps1
|
|
cp .wikitool-tools.json /tmp/tools-pwsh.json
|
|
pwsh -NoProfile -ExecutionPolicy Bypass -File tools/preflight.ps1 > /tmp/preflight-second.txt
|
|
cmp .wikitool-tools.json /tmp/tools-pwsh.json
|
|
rm .wikitool-tools.json
|
|
tools/preflight.sh
|
|
cmp .wikitool-tools.json /tmp/tools-pwsh.json
|
|
tools/.venv/bin/python -m pip install --quiet pytest
|
|
|
|
- name: The launcher, started from PowerShell
|
|
# `tools/wikitool` from pwsh resolves to wikitool.ps1, not the sh launcher - the one
|
|
# thing a Linux shell cannot show, so it is asked for by that exact string.
|
|
run: |
|
|
set -eu
|
|
pwsh -NoProfile -Command './tools/wikitool version show'
|
|
|
|
- name: PowerShell tests
|
|
# Skipped everywhere without pwsh, so this is the run that counts. The `verify` job
|
|
# runs the rest of the suite.
|
|
run: |
|
|
set -eu
|
|
cd tools
|
|
.venv/bin/python -m pytest -q chemenu/tests/test_preflight_pwsh.py chemenu/tests/test_preflight.py chemenu/tests/test_doctor.py chemenu/tests/test_bugreport_launcher.py
|