Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
7cfafd75b2 | ||
|
|
918157cb41 | ||
|
|
be049217a9 | ||
|
|
9868fce3d5 | ||
|
|
8ea2795d88 | ||
|
|
0e6a50aa98 | ||
|
|
6659ca5409 | ||
|
|
9c0bb6bfbf | ||
|
|
765ebe97d7 | ||
|
|
1a954be65d | ||
|
|
90ea0eea1b | ||
|
|
45ae04cfee | ||
|
|
49f8939bfd | ||
|
|
5c9be38aa8 | ||
|
|
5915b07383 | ||
|
|
e9f4759727 | ||
|
|
1559f4b405 | ||
|
|
893db6e640 | ||
|
|
61e18dc4aa | ||
|
|
fe1de63e0f | ||
|
|
f9056c04f7 | ||
|
|
cff7df4ffa | ||
|
|
070cae6aad | ||
|
|
d644a1d4f7 | ||
|
|
53383e7e9b | ||
|
|
01c888d3cd | ||
|
|
d66a3f65f4 | ||
|
|
0e15972ee7 | ||
|
|
044943ae51 | ||
|
|
2ddcd6be19 | ||
|
|
afd82550f2 | ||
|
|
ac1adf62af | ||
|
|
cb371545bf | ||
|
|
a3fc8f6873 | ||
|
|
fa106eeda3 | ||
|
|
d4638bacde | ||
|
|
40839d956d | ||
|
|
283cdae8be | ||
|
|
ccede04b97 | ||
|
|
8fec406463 | ||
|
|
8ce202a34e | ||
|
|
311c8ee059 | ||
|
|
8ff22ad6b0 | ||
|
|
862bc04d9c | ||
|
|
662a844cf1 | ||
|
|
20a7624134 | ||
|
|
a2a6baa265 | ||
|
|
af8bcb5b57 | ||
|
|
4ec22d376d | ||
|
|
f3ccbd86f9 | ||
|
|
bd12831974 | ||
|
|
0d3499ab04 | ||
|
|
dd565d249f | ||
|
|
d8cb494d58 | ||
|
|
c261b8f4ca | ||
|
|
b3022b8ffb | ||
|
|
100ace846e | ||
|
|
59c06e5ddc | ||
|
|
c4dcff76e6 | ||
|
|
fba263af68 | ||
|
|
c0f324ff96 | ||
|
|
d8224ee2ab | ||
|
|
80488bee38 | ||
|
|
c65d559690 | ||
|
|
5d937233b1 | ||
|
|
c77bda2004 | ||
|
|
b33088f64e | ||
|
|
a6d07f97c4 | ||
|
|
d0f08d1fba | ||
|
|
9d06050338 | ||
|
|
d6e973c3ce | ||
|
|
c33e8cdfb1 | ||
|
|
210e0c8286 | ||
|
|
4035b1ba12 | ||
|
|
8dae8a1790 | ||
|
|
e4b2b6d9b1 | ||
|
|
5a731729f6 | ||
|
|
04aebdeccf | ||
|
|
03743ebbc0 | ||
|
|
61130ce55d | ||
|
|
eadc052f6c | ||
|
|
08dde007dd | ||
|
|
76d67e45ba | ||
|
|
0899c670fe | ||
|
|
4f0622dfa4 | ||
|
|
f9c047bd2e | ||
|
|
40413f966d | ||
|
|
dea98d4bc0 | ||
|
|
5a4c5182b7 | ||
|
|
6554791699 | ||
|
|
a12673452d | ||
|
|
b0c64772cc | ||
|
|
529793b255 | ||
|
|
8be5e6e5f3 | ||
|
|
94deccb18d | ||
|
|
cf892315e6 | ||
|
|
dd885db625 | ||
|
|
6c0ebcc4f0 | ||
|
|
b8ed8bd610 | ||
|
|
bc314e5c8c | ||
|
|
8f61209b6b | ||
|
|
8d5fc6e941 | ||
|
|
20745399c0 | ||
|
|
a70d904274 | ||
|
|
685fc2e15e | ||
|
|
906d63fae2 | ||
|
|
16c911fca5 | ||
|
|
63f566ff05 | ||
|
|
5fe6003929 | ||
|
|
5aae7fed1b | ||
|
|
5bfbb49d74 | ||
|
|
b83a3982c5 | ||
|
|
243db66134 | ||
|
|
71efdbe01b | ||
|
|
9617d722de | ||
|
|
b9c22f783f | ||
|
|
964978a9c6 | ||
|
|
be78ad20af | ||
|
|
0b2d93a4a4 | ||
|
|
a1f3c47e62 | ||
|
|
d0a740acc9 | ||
|
|
9d6ca6b193 | ||
|
|
df8ff2fa22 | ||
|
|
fd0f60b2e7 | ||
|
|
2a60f3a265 | ||
|
|
5ffab3accf | ||
|
|
b79f083cc1 | ||
|
|
26e1018766 | ||
|
|
919e733e21 | ||
|
|
6d53c55d0d | ||
|
|
4446424e01 | ||
|
|
50171ca099 | ||
|
|
62d1c5e636 | ||
|
|
8b535b4016 | ||
|
|
2cce979814 | ||
|
|
88e7cc17f4 | ||
|
|
cfbe3ea83e | ||
|
|
e07d1ca42a | ||
|
|
3c1d4cb028 | ||
|
|
52ba5ba768 | ||
|
|
8ed8c6f5d9 | ||
|
|
1d695f6536 | ||
|
|
44909c9e47 | ||
|
|
6324024d7a | ||
|
|
e4260fc2de | ||
|
|
80b57e0d01 | ||
|
|
1875449b31 | ||
|
|
ee24b6e5b8 | ||
|
|
3c9d669729 | ||
|
|
11c400c670 | ||
|
|
9a1be6acde | ||
|
|
24cd221b21 | ||
|
|
aa31d431fc | ||
|
|
4284f101c8 | ||
|
|
e4e2332e01 | ||
|
|
536093f6c9 | ||
|
|
0c98080964 | ||
|
|
72d01beef8 | ||
|
|
0e09cf41ea | ||
|
|
504149c7c4 | ||
|
|
f3c80747a5 | ||
|
|
5d26698cd0 | ||
|
|
bb097f614b | ||
|
|
55f65c1ab1 | ||
|
|
6eb3f84256 | ||
|
|
d49513bda6 | ||
|
|
90ce41964f | ||
|
|
f350999053 | ||
|
|
05a75065ba | ||
|
|
ef60e2984c | ||
|
|
c64479fe02 | ||
|
|
c0dc2129bb | ||
|
|
f140e26a4c | ||
|
|
0fb8fd6122 | ||
|
|
dc688e5726 | ||
|
|
1b0158fc8d | ||
|
|
9a2d7d34f5 | ||
|
|
9ef021bea1 | ||
|
|
87719bf396 | ||
|
|
4aa07fc91a | ||
|
|
44cca62a9b | ||
|
|
e06898263c | ||
|
|
f93d14b9d7 | ||
|
|
a9703520a7 | ||
|
|
36da0855cf | ||
|
|
95ab40827a | ||
|
|
203084477f | ||
|
|
441a8151ab | ||
|
|
5b916c6d18 | ||
|
|
828521861d | ||
|
|
4781140375 | ||
|
|
42646d86c3 | ||
|
|
82a22eaa93 | ||
|
|
71446c0f29 | ||
|
|
f8111d05a3 | ||
|
|
dda80c1a9d | ||
|
|
11cbb15e65 | ||
|
|
54d9540c08 | ||
|
|
bb8f956719 | ||
|
|
53e3527e0b | ||
|
|
2c4c2b1c7c | ||
|
|
a51d7a322f | ||
|
|
5820924ffd | ||
|
|
11d64e6aa0 | ||
|
|
663b1c046c | ||
|
|
7bc5da6e0d | ||
|
|
6b300aa782 | ||
|
|
46dfee0ea9 | ||
|
|
00220f8b07 | ||
|
|
d9af88ffb8 | ||
|
|
f4353ccfb3 | ||
|
|
f2a093bc8b | ||
|
|
4c4dca31c1 | ||
|
|
b138fd8e64 | ||
|
|
7f74303a00 | ||
|
|
63b4bb82d9 | ||
|
|
0b3c496fff | ||
|
|
36d2128f29 | ||
|
|
1f0ad7f9f3 | ||
|
|
251e597c63 | ||
|
|
9e414319b8 | ||
|
|
8e25a7865f | ||
|
|
a0aecfcf94 | ||
|
|
4ab358fdb8 | ||
|
|
cc38bcd700 | ||
|
|
6671af6a60 | ||
|
|
b4e450108e | ||
|
|
e00eae08e8 | ||
|
|
fe55ad2a9c | ||
|
|
24593c5608 | ||
|
|
3916cb9541 | ||
|
|
72b2b4424f | ||
|
|
91bd430ac8 | ||
|
|
d34924d640 | ||
|
|
87a47cc237 | ||
|
|
d4cbeca5a8 | ||
|
|
0ba94c63d1 | ||
|
|
368438e48c | ||
|
|
cd81ba3d4f | ||
|
|
1b5ffea854 | ||
|
|
d2b1719a4b | ||
|
|
686c08bb14 | ||
|
|
abe5497cda | ||
|
|
d29d400dd3 | ||
|
|
b1883befc7 | ||
|
|
56ecfc7fee | ||
|
|
4e80a07ac7 | ||
|
|
0b8ca746fa | ||
|
|
9b461421e8 | ||
|
|
41f5dfe1cd | ||
|
|
23307c3c5f | ||
|
|
cfe925a76c | ||
|
|
23e34a940c | ||
|
|
c8c238523a | ||
|
|
3f99d6715f | ||
|
|
807094deae | ||
|
|
8b711f4860 | ||
|
|
a35c94e2d9 | ||
|
|
9e2f9bf98d | ||
|
|
7e15035001 | ||
|
|
b137359b90 | ||
|
|
177c7e9ce8 | ||
|
|
502971d147 | ||
|
|
9843df99d3 | ||
|
|
31662dc3ff | ||
|
|
7fbb9a99df | ||
|
|
778764de4d | ||
|
|
f7597b209c | ||
|
|
83018fcc7d | ||
|
|
576df2cddd | ||
|
|
d1cf2e0327 | ||
|
|
df7ea93060 | ||
|
|
00c2cf6ffe | ||
|
|
32a9b8eb3f |
No files matched your search
@@ -11,7 +11,7 @@
|
||||
"hooks": [
|
||||
{
|
||||
"type": "command",
|
||||
"command": "./tools/trace_ingest.py --source claude-code --event prompt.submitted 2>/dev/null || true",
|
||||
"command": "./tools/trace-hook --source claude-code --event prompt.submitted 2>/dev/null || true",
|
||||
"timeout": 5
|
||||
}
|
||||
]
|
||||
|
||||
@@ -0,0 +1,17 @@
|
||||
# Chemenu - .gitattributes
|
||||
#
|
||||
# Every text file is stored and checked out with LF, whatever `core.autocrlf` says.
|
||||
# Without this, a Windows checkout with `core.autocrlf=true` gives the sh launcher
|
||||
# tools/wikitool CRLF line endings, and Git Bash then fails with `env: 'bash\r'`. The
|
||||
# tools also compare file bytes (the published skill copies, the sha256 per file in
|
||||
# `.wikitool-release.json`), and those comparisons only agree when the line endings do.
|
||||
* text=auto eol=lf
|
||||
|
||||
# Sources are kept byte for byte as they arrived: raw/CONTRACT.md makes them immutable,
|
||||
# and normalizing a CRLF source on `git add` would change it.
|
||||
/raw/** -text
|
||||
/incoming/** -text
|
||||
|
||||
# Real documents for the `raw derive` tests are compared byte for byte (an .eml with CRLF
|
||||
# included), so nothing under testfiles/ is ever normalized.
|
||||
/testfiles/** -text
|
||||
@@ -0,0 +1,34 @@
|
||||
# The image the `pwsh` job of ci.yml runs in: Debian, PowerShell 7 and PSScriptAnalyzer,
|
||||
# plus what the tool preflight and the suite need (Gitea #151, D37). Built by
|
||||
# `.gitea/workflows/pwsh-ci-image.yml`, never by hand.
|
||||
FROM debian:trixie-slim
|
||||
|
||||
ARG PWSH_VERSION
|
||||
|
||||
# `nodejs` is for act_runner, which executes JavaScript actions (checkout) inside the job
|
||||
# container. `libicu76` is what PowerShell's .NET needs for culture data; `iconv` converts
|
||||
# the UTF-16 checksum list the PowerShell release publishes.
|
||||
RUN set -eu; \
|
||||
test -n "$PWSH_VERSION"; \
|
||||
apt-get update -qq; \
|
||||
apt-get install -y --no-install-recommends \
|
||||
ca-certificates curl git nodejs python3 python3-venv ripgrep libicu76; \
|
||||
base="https://github.com/PowerShell/PowerShell/releases/download/v${PWSH_VERSION}"; \
|
||||
tarball="powershell-${PWSH_VERSION}-linux-x64.tar.gz"; \
|
||||
curl -fsSL -o "/tmp/${tarball}" "${base}/${tarball}"; \
|
||||
curl -fsSL "${base}/hashes.sha256" | iconv -f UTF-16 -t UTF-8 | tr -d '\r' > /tmp/hashes.sha256; \
|
||||
expected="$(grep -F "*${tarball}" /tmp/hashes.sha256 | cut -d' ' -f1)"; \
|
||||
test -n "$expected"; \
|
||||
echo "${expected} /tmp/${tarball}" | sha256sum -c -; \
|
||||
mkdir -p /opt/microsoft/powershell/7; \
|
||||
tar -xzf "/tmp/${tarball}" -C /opt/microsoft/powershell/7; \
|
||||
chmod +x /opt/microsoft/powershell/7/pwsh; \
|
||||
ln -s /opt/microsoft/powershell/7/pwsh /usr/local/bin/pwsh; \
|
||||
rm -rf /tmp/* /var/lib/apt/lists/*
|
||||
|
||||
RUN pwsh -NoProfile -Command \
|
||||
"Set-PSRepository PSGallery -InstallationPolicy Trusted; Install-Module PSScriptAnalyzer -Scope AllUsers -Force"
|
||||
|
||||
LABEL org.opencontainers.image.title="chemenu-ci-pwsh" \
|
||||
org.opencontainers.image.description="PowerShell 7 and PSScriptAnalyzer for chemenu's pwsh CI job" \
|
||||
chemenu.pwsh-version="${PWSH_VERSION}"
|
||||
Executable
+55
@@ -0,0 +1,55 @@
|
||||
#!/bin/sh
|
||||
# Start a throwaway Radicale (a small CalDAV server) for the live tracker suite (Gitea #156).
|
||||
#
|
||||
# start-radicale.sh <python-with-radicale> <work-dir>
|
||||
#
|
||||
# Radicale runs as a background process, not a service container: the jobs that call this
|
||||
# already run inside a job container, and a process on 127.0.0.1 needs no network wiring.
|
||||
# Appends the CHEMENU_LIVE_CALDAV_* variables to $GITHUB_ENV, so the pytest step that follows
|
||||
# runs the CalDAV half of the live suite - with CHEMENU_LIVE_REQUIRE=caldav set by the
|
||||
# workflow, so a server that did not come up fails the run instead of skipping it.
|
||||
set -eu
|
||||
python="$1"
|
||||
work="$2"
|
||||
mkdir -p "$work"
|
||||
printf 'ci:ci-live-secret\n' > "$work/users"
|
||||
cat > "$work/config" <<CONF
|
||||
[server]
|
||||
hosts = 127.0.0.1:5232
|
||||
[auth]
|
||||
type = htpasswd
|
||||
htpasswd_filename = $work/users
|
||||
htpasswd_encryption = plain
|
||||
[storage]
|
||||
filesystem_folder = $work/data
|
||||
CONF
|
||||
# nohup: the server has to outlive this step's shell - the suite runs in the next step.
|
||||
nohup "$python" -m radicale --config "$work/config" > "$work/radicale.log" 2>&1 &
|
||||
# The readiness probe uses the same Python, not curl: ci.yml's job image has no curl.
|
||||
ready='
|
||||
import base64, sys, urllib.request
|
||||
req = urllib.request.Request("http://127.0.0.1:5232/ci/", method="PROPFIND",
|
||||
headers={"Depth": "0", "Authorization": "Basic " + base64.b64encode(b"ci:ci-live-secret").decode()})
|
||||
try:
|
||||
urllib.request.urlopen(req, timeout=2)
|
||||
except Exception:
|
||||
sys.exit(1)
|
||||
'
|
||||
i=0
|
||||
until "$python" -c "$ready"; do
|
||||
i=$((i + 1))
|
||||
if [ "$i" -gt 30 ]; then
|
||||
echo "radicale did not come up:" >&2
|
||||
cat "$work/radicale.log" >&2
|
||||
exit 1
|
||||
fi
|
||||
sleep 1
|
||||
done
|
||||
version="$("$python" -m radicale --version 2>/dev/null | tail -n 1)"
|
||||
{
|
||||
echo "CHEMENU_LIVE_CALDAV_URL=http://127.0.0.1:5232/ci/"
|
||||
echo "CHEMENU_LIVE_CALDAV_USER=ci"
|
||||
echo "CHEMENU_LIVE_CALDAV_PASSWORD=ci-live-secret"
|
||||
echo "CHEMENU_LIVE_CALDAV_VERSION=radicale $version"
|
||||
} >> "${GITHUB_ENV:?start-radicale.sh runs inside a workflow step}"
|
||||
echo "radicale $version is up"
|
||||
Executable
+82
@@ -0,0 +1,82 @@
|
||||
#!/bin/sh
|
||||
# Make the wiki repository of this repo hold exactly the converter output.
|
||||
#
|
||||
# wiki-push.sh <output-dir> <version>
|
||||
#
|
||||
# Environment:
|
||||
# WIKI_TOKEN token with write access to the wiki (required)
|
||||
# GITHUB_SERVER_URL set by the runner; GITHUB_REPOSITORY likewise
|
||||
# WIKI_REMOTE overrides the wiki clone URL (the tests point it at a local repository)
|
||||
#
|
||||
# The wiki is a mirror, never a source: every file in it is replaced by the converter output, so
|
||||
# a page the converter no longer produces disappears. Afterwards the working tree minus .git/ and
|
||||
# the output directory are compared, and any difference fails the script before anything is
|
||||
# pushed. An unchanged result ends with exit 0 and no commit.
|
||||
set -eu
|
||||
|
||||
out="${1:?usage: wiki-push.sh <output-dir> <version>}"
|
||||
version="${2:?usage: wiki-push.sh <output-dir> <version>}"
|
||||
|
||||
if [ -z "${WIKI_TOKEN:-}" ]; then
|
||||
echo "ERROR WIKI_TOKEN is empty: the wiki mirror needs a token that may push to the wiki repository." >&2
|
||||
echo " The job token (gitea.token) is used with 'permissions: wiki: write'; if Gitea still refuses it," >&2
|
||||
echo " the repository's Actions settings cap the token, or the operator adds a secret with write access." >&2
|
||||
exit 1
|
||||
fi
|
||||
[ -f "$out/Home.md" ] || { echo "ERROR $out/Home.md is missing - not a converter output" >&2; exit 1; }
|
||||
|
||||
server="${GITHUB_SERVER_URL:-}"
|
||||
repo="${GITHUB_REPOSITORY:-}"
|
||||
remote="${WIKI_REMOTE:-${server}/${repo}.wiki.git}"
|
||||
api="${server}/api/v1/repos/${repo}"
|
||||
|
||||
# The token travels in a header, not in the URL, so it appears in no git message or process list.
|
||||
auth="Authorization: Basic $(printf 'wiki-mirror:%s' "$WIKI_TOKEN" | base64 | tr -d '\n')"
|
||||
git_remote() { git -c "http.extraheader=${auth}" "$@"; }
|
||||
|
||||
work="$(mktemp -d)"
|
||||
trap 'rm -rf "$work"' EXIT
|
||||
wiki="$work/wiki"
|
||||
|
||||
if ! git_remote clone -q "$remote" "$wiki" 2>"$work/clone.err"; then
|
||||
echo "Cloning ${remote} failed:"
|
||||
cat "$work/clone.err"
|
||||
# A wiki that has never had a page has no repository yet; the first page created through
|
||||
# the API brings it into existence.
|
||||
echo "Creating the first page through the API to bring the wiki repository into existence."
|
||||
content="$(base64 < "$out/Home.md" | tr -d '\n')"
|
||||
jq -n --arg title Home --arg content "$content" --arg message "Mirror docs at v${version}" \
|
||||
'{title: $title, content_base64: $content, message: $message}' > "$work/first-page.json"
|
||||
status="$(curl -sS -o "$work/first-page.out" -w '%{http_code}' -X POST "${api}/wiki/new" \
|
||||
-H "Authorization: token ${WIKI_TOKEN}" -H 'Content-Type: application/json' \
|
||||
--data-binary @"$work/first-page.json")"
|
||||
case "$status" in
|
||||
2??) ;;
|
||||
*)
|
||||
echo "ERROR the API refused the first page (HTTP ${status}):" >&2
|
||||
cat "$work/first-page.out" >&2
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
git_remote clone -q "$remote" "$wiki"
|
||||
fi
|
||||
|
||||
find "$wiki" -mindepth 1 -maxdepth 1 ! -name .git -exec rm -rf {} +
|
||||
cp -R "$out"/. "$wiki"/
|
||||
git -C "$wiki" add -A
|
||||
|
||||
if ! diff -r -x .git "$out" "$wiki" > "$work/diff.txt"; then
|
||||
echo "ERROR the wiki working tree differs from the converter output after copying:" >&2
|
||||
cat "$work/diff.txt" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if git -C "$wiki" diff --cached --quiet; then
|
||||
echo "Wiki already matches v${version}"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
git -C "$wiki" -c user.name='Wiki Mirror' -c user.email='wiki-mirror@noreply.invalid' \
|
||||
commit -q -m "Mirror docs at v${version}"
|
||||
git_remote -C "$wiki" push -q origin HEAD
|
||||
echo "Wiki now mirrors v${version}"
|
||||
Executable
+293
@@ -0,0 +1,293 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Convert the documentation of one release tree into the flat page set of a Gitea wiki.
|
||||
|
||||
wiki_mirror.py --source <tree> --out <dir> --version <X.Y.Z> [--repo-url <url>] [--check]
|
||||
|
||||
The output directory becomes the whole content of the wiki repository, so the conversion is
|
||||
deterministic: the same tree and version give the same bytes. `--check` converts in memory,
|
||||
writes nothing and fails only on a name collision or an unresolvable relative link.
|
||||
|
||||
Plain standard library on purpose: it runs in CI before any tool environment exists, and it
|
||||
must work against the tree of any past tag.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import os
|
||||
import posixpath
|
||||
import re
|
||||
import sys
|
||||
from urllib.parse import quote, unquote
|
||||
|
||||
DEFAULT_REPO_URL = "https://gitea.nehmer.net/torben/chemenu"
|
||||
RESERVED_NAMES = {"home", "install", "_sidebar"}
|
||||
AREA_ORDER = ("tutorials", "how-to", "reference", "explanation")
|
||||
DEV_AREA = "dev"
|
||||
FLAT_GROUP = "Documentation"
|
||||
|
||||
_SCHEME = re.compile(r"^[A-Za-z][A-Za-z0-9+.-]*:")
|
||||
_FENCE = re.compile(r"^\s*(`{3,}|~{3,})")
|
||||
_LINK = re.compile(r"""\]\(\s*([^)\s<][^)\s]*)((?:\s+(?:"[^"]*"|'[^']*'))?)\s*\)""")
|
||||
_H1 = re.compile(r"^# +(.+?)\s*#*\s*$")
|
||||
|
||||
|
||||
class Page:
|
||||
def __init__(self, path: str, name: str, segs: tuple[str, ...] | None, index: bool):
|
||||
self.path = path
|
||||
self.name = name
|
||||
self.segs = segs # sidebar position below docs/; None for Home and Install
|
||||
self.index = index # a README.md: stands for its directory
|
||||
|
||||
|
||||
def _capitalise(segment: str) -> str:
|
||||
return segment[:1].upper() + segment[1:]
|
||||
|
||||
|
||||
def collect_pages(source: str) -> tuple[list[Page], list[str]]:
|
||||
"""Every page to mirror, in a stable order, plus the collisions found among their names."""
|
||||
pages: list[Page] = []
|
||||
if os.path.isfile(os.path.join(source, "README.md")):
|
||||
pages.append(Page("README.md", "Home", None, False))
|
||||
if os.path.isfile(os.path.join(source, "INSTALL.md")):
|
||||
pages.append(Page("INSTALL.md", "Install", None, False))
|
||||
|
||||
docs = os.path.join(source, "docs")
|
||||
found: list[str] = []
|
||||
for base, dirs, files in os.walk(docs):
|
||||
dirs.sort()
|
||||
for file in sorted(files):
|
||||
if file.endswith(".md"):
|
||||
rel = os.path.relpath(os.path.join(base, file), source)
|
||||
found.append(rel.replace(os.sep, "/"))
|
||||
for path in sorted(found):
|
||||
parts = path[len("docs/"):-len(".md")].split("/")
|
||||
index = parts[-1] == "README"
|
||||
if index:
|
||||
parts = parts[:-1]
|
||||
name = "-".join(_capitalise(p) for p in parts) if parts else FLAT_GROUP
|
||||
pages.append(Page(path, name, tuple(parts), index))
|
||||
|
||||
errors: list[str] = []
|
||||
seen: dict[str, str] = {}
|
||||
for page in pages:
|
||||
key = page.name.lower()
|
||||
if key in RESERVED_NAMES and page.segs is not None:
|
||||
errors.append(f"{page.path}: page name '{page.name}' is reserved for a generated page")
|
||||
elif key in seen:
|
||||
errors.append(
|
||||
f"name collision on '{page.name}' (compared without case): {seen[key]} and {page.path}"
|
||||
)
|
||||
else:
|
||||
seen[key] = page.path
|
||||
return pages, errors
|
||||
|
||||
|
||||
def mask_code_spans(line: str) -> str:
|
||||
"""The line with every inline code span blanked to NULs, same length, so offsets still match."""
|
||||
out = list(line)
|
||||
n = len(line)
|
||||
i = 0
|
||||
while i < n:
|
||||
if line[i] != "`":
|
||||
i += 1
|
||||
continue
|
||||
j = i
|
||||
while j < n and line[j] == "`":
|
||||
j += 1
|
||||
run = j - i
|
||||
k = j
|
||||
end = -1
|
||||
while k < n:
|
||||
if line[k] != "`":
|
||||
k += 1
|
||||
continue
|
||||
m = k
|
||||
while m < n and line[m] == "`":
|
||||
m += 1
|
||||
if m - k == run:
|
||||
end = m
|
||||
break
|
||||
k = m
|
||||
if end < 0:
|
||||
i = j
|
||||
continue
|
||||
for p in range(i, end):
|
||||
out[p] = "\0"
|
||||
i = end
|
||||
return "".join(out)
|
||||
|
||||
|
||||
class Converter:
|
||||
def __init__(self, source: str, version: str, repo_url: str, pages: list[Page]):
|
||||
self.source = source
|
||||
self.version = version
|
||||
self.repo_url = repo_url.rstrip("/")
|
||||
self.by_path = {p.path: p for p in pages}
|
||||
self.errors: list[str] = []
|
||||
|
||||
def tag_url(self, path: str) -> str:
|
||||
base = f"{self.repo_url}/src/tag/v{self.version}"
|
||||
return base if path in ("", ".") else f"{base}/{quote(path, safe='/')}"
|
||||
|
||||
def resolve(self, src: str, target: str) -> str | None:
|
||||
"""The replacement for one relative link target, or None (and an error recorded)."""
|
||||
path, hash_, anchor = target.partition("#")
|
||||
suffix = hash_ + anchor
|
||||
path = unquote(path)
|
||||
base = "" if path.startswith("/") else posixpath.dirname(src)
|
||||
joined = posixpath.normpath(posixpath.join(base, path.lstrip("/")))
|
||||
if joined == ".." or joined.startswith("../"):
|
||||
self.errors.append(f"{src}: link '{target}' leaves the tree")
|
||||
return None
|
||||
page = self.by_path.get(joined)
|
||||
if page is None and os.path.isdir(os.path.join(self.source, joined)):
|
||||
index = "README.md" if joined == "." else f"{joined}/README.md"
|
||||
page = self.by_path.get(index)
|
||||
if page is not None:
|
||||
return page.name + suffix
|
||||
if os.path.exists(os.path.join(self.source, joined)):
|
||||
return self.tag_url(joined) + suffix
|
||||
self.errors.append(f"{src}: link '{target}' points at '{joined}', which does not exist in the tree")
|
||||
return None
|
||||
|
||||
def convert_line(self, src: str, line: str) -> str:
|
||||
masked = mask_code_spans(line)
|
||||
edits = []
|
||||
for match in _LINK.finditer(masked):
|
||||
target = match.group(1)
|
||||
if _SCHEME.match(target) or target.startswith(("#", "//")):
|
||||
continue
|
||||
replacement = self.resolve(src, line[match.start(1):match.end(1)])
|
||||
if replacement is not None:
|
||||
edits.append((match.start(1), match.end(1), replacement))
|
||||
for start, end, replacement in reversed(edits):
|
||||
line = line[:start] + replacement + line[end:]
|
||||
return line
|
||||
|
||||
def convert(self, src: str, text: str) -> str:
|
||||
return "\n".join(
|
||||
line if in_code else self.convert_line(src, line) for line, in_code in code_flags(text)
|
||||
)
|
||||
|
||||
|
||||
def code_flags(text: str):
|
||||
"""Each line of the text with whether it belongs to a fenced code block (fences included)."""
|
||||
fence = None
|
||||
for line in text.split("\n"):
|
||||
m = _FENCE.match(line)
|
||||
if fence is None:
|
||||
if m:
|
||||
fence = m.group(1)
|
||||
yield line, m is not None
|
||||
continue
|
||||
yield line, True
|
||||
if m and m.group(1)[0] == fence[0] and len(m.group(1)) >= len(fence) and not line.strip().strip(fence[0]):
|
||||
fence = None
|
||||
|
||||
|
||||
def title_of(text: str, fallback: str) -> str:
|
||||
for line, in_code in code_flags(text):
|
||||
h = None if in_code else _H1.match(line)
|
||||
if h:
|
||||
return h.group(1).replace("[", "\\[").replace("]", "\\]")
|
||||
return fallback
|
||||
|
||||
|
||||
def sidebar(pages: list[Page], titles: dict[str, str]) -> str:
|
||||
def link(page: Page) -> str:
|
||||
return f"[{titles[page.path]}]({page.name})"
|
||||
|
||||
lines = []
|
||||
home = next((p for p in pages if p.name == "Home" and p.segs is None), None)
|
||||
install = next((p for p in pages if p.name == "Install" and p.segs is None), None)
|
||||
for page in (home, install):
|
||||
if page is not None:
|
||||
lines.append(f"* {link(page)}")
|
||||
|
||||
docs = [p for p in pages if p.segs is not None]
|
||||
page_at = {p.segs: p for p in docs}
|
||||
flat = [p for p in docs if len(p.segs) == 0 or (len(p.segs) == 1 and not p.index)]
|
||||
tree = [p for p in docs if p not in flat]
|
||||
|
||||
children: dict[tuple[str, ...], set[tuple[str, ...]]] = {}
|
||||
for page in tree:
|
||||
for i in range(1, len(page.segs) + 1):
|
||||
children.setdefault(page.segs[: i - 1], set()).add(page.segs[:i])
|
||||
|
||||
def label(key: tuple[str, ...]) -> str:
|
||||
page = page_at.get(key)
|
||||
return link(page) if page else _capitalise(key[-1])
|
||||
|
||||
def render(key: tuple[str, ...], depth: int) -> None:
|
||||
lines.append(" " * depth + f"* {label(key)}")
|
||||
for child in sorted(children.get(key, ()), key=lambda k: k[-1].lower()):
|
||||
render(child, depth + 1)
|
||||
|
||||
if flat:
|
||||
lines.append(f"* {FLAT_GROUP}")
|
||||
for page in sorted(flat, key=lambda p: p.name.lower()):
|
||||
lines.append(f" * {link(page)}")
|
||||
|
||||
tops = children.get((), set())
|
||||
known = [(s,) for s in AREA_ORDER if (s,) in tops]
|
||||
rest = sorted((k for k in tops if k[0] not in AREA_ORDER and k[0] != DEV_AREA), key=lambda k: k[0].lower())
|
||||
last = [(DEV_AREA,)] if (DEV_AREA,) in tops else []
|
||||
for key in known + rest + last:
|
||||
render(key, 0)
|
||||
return "\n".join(lines) + "\n"
|
||||
|
||||
|
||||
def build(source: str, version: str, repo_url: str) -> tuple[dict[str, str], list[str]]:
|
||||
"""The wiki as {file name: content}, and every error found while building it."""
|
||||
pages, errors = collect_pages(source)
|
||||
converter = Converter(source, version, repo_url, pages)
|
||||
files: dict[str, str] = {}
|
||||
titles: dict[str, str] = {}
|
||||
for page in pages:
|
||||
with open(os.path.join(source, page.path), encoding="utf-8", newline="") as fh:
|
||||
text = fh.read()
|
||||
titles[page.path] = title_of(text, page.name)
|
||||
notice = (
|
||||
f"*Generated from [`{page.path}`]({converter.tag_url(page.path)}) at v{version}"
|
||||
" – edit the repository, not this wiki.*"
|
||||
)
|
||||
files[f"{page.name}.md"] = notice + "\n\n" + converter.convert(page.path, text)
|
||||
files["_Sidebar.md"] = sidebar(pages, titles)
|
||||
return files, errors + converter.errors
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__.split("\n\n")[0])
|
||||
parser.add_argument("--source", required=True, help="the tree to mirror")
|
||||
parser.add_argument("--out", help="empty or missing directory that receives the wiki content")
|
||||
parser.add_argument("--version", required=True, help="X.Y.Z of the tree; names the tag links point at")
|
||||
parser.add_argument("--repo-url", default=DEFAULT_REPO_URL)
|
||||
parser.add_argument("--check", action="store_true", help="convert, write nothing, fail on a problem")
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
if not args.check and not args.out:
|
||||
parser.error("--out is required unless --check is given")
|
||||
if not os.path.isdir(args.source):
|
||||
print(f"ERROR --source {args.source} is not a directory", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
files, errors = build(args.source, args.version, args.repo_url)
|
||||
if errors:
|
||||
for error in errors:
|
||||
print(f"ERROR {error}", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
if not args.check:
|
||||
if os.path.isdir(args.out) and os.listdir(args.out):
|
||||
print(f"ERROR --out {args.out} is not empty", file=sys.stderr)
|
||||
return 1
|
||||
os.makedirs(args.out, exist_ok=True)
|
||||
for name, content in files.items():
|
||||
with open(os.path.join(args.out, name), "w", encoding="utf-8", newline="") as fh:
|
||||
fh.write(content)
|
||||
print(f"{len(files)} wiki files for v{args.version}" + (" (checked, nothing written)" if args.check else ""))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,30 @@
|
||||
# The image the nightly `tracker-live` workflow runs in: the packaged Super Productivity
|
||||
# desktop app, an X server to hold it, and what the suite itself needs (Gitea #156).
|
||||
# Built by `.gitea/workflows/sp-live-image.yml`, never by hand.
|
||||
FROM debian:trixie-slim
|
||||
|
||||
ARG SP_VERSION
|
||||
ARG SP_SHA512
|
||||
|
||||
# `nodejs` is for act_runner, which executes JavaScript actions (checkout) inside the
|
||||
# job container. `libasound2t64` and `libgbm1` are the two libraries the .deb does not
|
||||
# pull in and the app will not start without.
|
||||
RUN set -eu; \
|
||||
test -n "$SP_VERSION" && test -n "$SP_SHA512"; \
|
||||
apt-get update -qq; \
|
||||
apt-get install -y --no-install-recommends \
|
||||
ca-certificates curl git nodejs python3 python3-venv ripgrep \
|
||||
xvfb xauth libasound2t64 libgbm1; \
|
||||
curl -fsSL -o /tmp/sp.deb \
|
||||
"https://github.com/super-productivity/super-productivity/releases/download/v${SP_VERSION}/superProductivity-amd64.deb"; \
|
||||
expected="$(printf '%s' "$SP_SHA512" | base64 -d | od -An -v -tx1 | tr -d ' \n')"; \
|
||||
echo "${expected} /tmp/sp.deb" | sha512sum -c -; \
|
||||
apt-get install -y --no-install-recommends /tmp/sp.deb; \
|
||||
rm -rf /tmp/sp.deb /var/lib/apt/lists/*
|
||||
|
||||
LABEL org.opencontainers.image.title="chemenu-sp-live" \
|
||||
org.opencontainers.image.description="Packaged Super Productivity for chemenu's live tracker suite" \
|
||||
chemenu.sp-version="${SP_VERSION}"
|
||||
|
||||
ENV CHEMENU_LIVE_SP_BINARY="/opt/Super Productivity/superproductivity" \
|
||||
CHEMENU_LIVE_SP_VERSION="${SP_VERSION}"
|
||||
Executable
+26
@@ -0,0 +1,26 @@
|
||||
#!/bin/sh
|
||||
# Which Super Productivity release, and the sha512 of its .deb, straight from the
|
||||
# update channel the desktop clients themselves follow (`latest-linux.yml`).
|
||||
#
|
||||
# resolve-version.sh the newest release
|
||||
# resolve-version.sh 19.1.0 that release
|
||||
#
|
||||
# Prints two lines, `version=<x>` and `sha512=<base64>`, so a workflow can append the
|
||||
# output to $GITHUB_OUTPUT as it is. Gitea #156: the test image follows the channel
|
||||
# rather than a pin, because installed apps update on their own and a pinned old
|
||||
# version would be tested against while users run the new one.
|
||||
set -eu
|
||||
base=https://github.com/super-productivity/super-productivity/releases
|
||||
if [ "${1:-latest}" = latest ]; then
|
||||
url="$base/latest/download/latest-linux.yml"
|
||||
else
|
||||
url="$base/download/v$1/latest-linux.yml"
|
||||
fi
|
||||
yml="$(curl -fsSL "$url")"
|
||||
version="$(printf '%s\n' "$yml" | sed -n 's/^version: *//p' | head -n 1)"
|
||||
sha512="$(printf '%s\n' "$yml" | awk '/url: superProductivity-amd64\.deb/ {found=1; next} found && /sha512:/ {print $2; exit}')"
|
||||
if [ -z "$version" ] || [ -z "$sha512" ]; then
|
||||
echo "resolve-version: no version/sha512 for the amd64 .deb in $url" >&2
|
||||
exit 1
|
||||
fi
|
||||
printf 'version=%s\nsha512=%s\n' "$version" "$sha512"
|
||||
+197
-24
@@ -1,8 +1,14 @@
|
||||
# CI for the wiki stack.
|
||||
#
|
||||
# One job, stopping at the first failure - the stack has no artifact to build
|
||||
# and nothing to deploy, so the pipeline's whole job is "does the machinery
|
||||
# still hold together, and does the distribution it produces still work".
|
||||
# `verify` is the pipeline: one job, stopping at the first failure - the stack
|
||||
# has no artifact to build and nothing to deploy, so its whole job is "does the
|
||||
# machinery still hold together, and does the distribution it produces still
|
||||
# work". `pwsh` beside it is the PowerShell half of the same question (Gitea
|
||||
# #151): the same preflight and launcher, under the PowerShell 7 that Windows
|
||||
# harnesses start them with, in the prebuilt `chemenu-ci-pwsh` image
|
||||
# (`pwsh-ci-image.yml`). It runs on Linux, so what only a Windows machine can
|
||||
# answer - the registry, the Store alias, a real Mark of the Web - is covered by
|
||||
# the environment hooks `tools/preflight.ps1` documents, not by this job.
|
||||
#
|
||||
# Runner: `linux-docker` is one of this Gitea instance's three routing labels
|
||||
# (alongside `container-builder` and `k3s-deploy`). The job image is named
|
||||
@@ -69,7 +75,8 @@ jobs:
|
||||
# working tree stays clean for the ignore-rule checks.
|
||||
WIKITOOL_SESSION_ID: ci-${{ github.run_id }}
|
||||
WIKI_TRACE_DIR: /tmp/wikitool-trace
|
||||
DIST_DIR: /tmp/dist
|
||||
BUILD_DIR: /tmp/build
|
||||
INSTANCE_DIR: /tmp/instance
|
||||
|
||||
steps:
|
||||
- name: System dependencies
|
||||
@@ -90,17 +97,27 @@ jobs:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Tool environment
|
||||
# The preflight, not a venv block of our own: it is the one way a
|
||||
# checkout gets set up (instructions/preflight.md), and tools/wikitool
|
||||
# refuses to start until it has passed. Run twice - the second run must
|
||||
# pass without changing anything, which is what an instance relies on
|
||||
# when it re-runs it after every update.
|
||||
run: |
|
||||
set -eu
|
||||
git config --global --add safe.directory "$GITHUB_WORKSPACE"
|
||||
python3 -m venv tools/.venv
|
||||
tools/.venv/bin/pip install --quiet --upgrade pip
|
||||
tools/.venv/bin/pip install --quiet -r tools/requirements.txt
|
||||
tools/preflight.sh
|
||||
cp .wikitool-tools.json /tmp/tools-first.json
|
||||
tools/preflight.sh > /tmp/preflight-second.txt
|
||||
cmp .wikitool-tools.json /tmp/tools-first.json
|
||||
# pytest-cov is CI-only: tools/requirements.txt describes what an
|
||||
# *instance* needs at runtime and ships with `dist export`, and an
|
||||
# instance does not measure this suite. Installed beside pytest for
|
||||
# the same reason pytest itself is.
|
||||
tools/.venv/bin/pip install --quiet pytest pytest-cov
|
||||
tools/.venv/bin/python -m pip install --quiet pytest pytest-cov
|
||||
# The MCP server's dependency is optional for an instance but not for
|
||||
# CI: its tests skip without it, and a skipped golden test is exactly
|
||||
# how the server's output and the CLI's would drift apart unnoticed.
|
||||
tools/.venv/bin/python -m pip install --quiet -r tools/requirements-mcp.txt
|
||||
|
||||
- name: Tests
|
||||
# Not run with WIKI_TRACE=0: two telemetry tests assert that a trace is
|
||||
@@ -116,24 +133,51 @@ jobs:
|
||||
# container is no longer a special environment worth a second run.
|
||||
# See instructions/dev/testing-conventions.md.
|
||||
#
|
||||
# Coverage is reported, not enforced: there is deliberately no
|
||||
# `--cov-fail-under` yet (Gitea #10). The threshold gets set in its own
|
||||
# later commit, with the measured number as its justification - one
|
||||
# picked before the number is either too low to bite or too high to
|
||||
# survive the next honest commit, and the second kind gets lowered
|
||||
# instead of earned. Config: tools/.coveragerc.
|
||||
# Coverage is measured and enforced at a floor of 85% against a measured
|
||||
# 87.0% - `fail_under` in tools/.coveragerc, not a flag here, so the
|
||||
# number sits next to the reasoning that produced it. It was set only
|
||||
# after the number had been watched across 38 runs (Gitea #10, closed).
|
||||
# A red suite from this floor means coverage actually fell; the two
|
||||
# points of headroom already absorb a new thin Typer wrapper.
|
||||
run: |
|
||||
set -eu
|
||||
cd tools
|
||||
.venv/bin/python -m pytest -q \
|
||||
--cov --cov-report=term --cov-report=xml --cov-report=html
|
||||
|
||||
- name: Start Radicale
|
||||
# Its own step because the script hands the server's address to the suite through
|
||||
# $GITHUB_ENV, which only reaches the steps after the one that wrote it.
|
||||
run: |
|
||||
set -eu
|
||||
tools/.venv/bin/python -m pip install --quiet radicale
|
||||
.gitea/scripts/start-radicale.sh tools/.venv/bin/python /tmp/radicale
|
||||
|
||||
- name: Live tracker suite (CalDAV)
|
||||
# The one live tracker that needs no app and no display: a throwaway Radicale on
|
||||
# loopback. `CHEMENU_LIVE_REQUIRE=caldav` turns "no server" into a failure - without
|
||||
# it the suite would skip and stay green, which is exactly the outcome this step
|
||||
# exists to rule out (Gitea #156). The Super Productivity half is nightly, in
|
||||
# `tracker-live.yml`; see instructions/dev/tracker-testing.md.
|
||||
env:
|
||||
CHEMENU_LIVE_REQUIRE: caldav
|
||||
run: |
|
||||
set -eu
|
||||
cd tools
|
||||
.venv/bin/python -m pytest -q -m live_tracker -k caldav -s
|
||||
|
||||
- name: Coverage report
|
||||
# `always()`: a red suite is exactly when the per-module numbers are
|
||||
# worth reading, and the upload must not disappear with the failure.
|
||||
# v3, not v4 - v4 is restricted on this Gitea instance; v3 is what is
|
||||
# proven here (torben/gitea-mcp@ci-build, ci-build.yaml, runs
|
||||
# 42-45).
|
||||
#
|
||||
# The artifact is downloadable from the run page, but the Actions
|
||||
# artifact REST endpoints report `total_count: 0` for it - v3 writes
|
||||
# through the older artifact API, which those endpoints do not read.
|
||||
# An empty list is not a failed upload. See docs/dev/development.md
|
||||
# § "How much of the stack the suite reaches"; do not re-derive this.
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v3
|
||||
with:
|
||||
@@ -150,6 +194,14 @@ jobs:
|
||||
tools/wikitool instructions verify
|
||||
tools/wikitool lint --fail-on-error
|
||||
|
||||
- name: Wiki mirror converts the docs
|
||||
# The Gitea wiki mirrors docs/ at every release (release.yml, job wiki-mirror). A link the
|
||||
# converter cannot resolve, or two pages that would share a wiki name, is cheaper to hear
|
||||
# about here than when the release is cut. Writes nothing.
|
||||
run: |
|
||||
set -eu
|
||||
python3 .gitea/scripts/wiki_mirror.py --check --source . --version "$(tr -d '[:space:]' < VERSION)"
|
||||
|
||||
- name: Version gate
|
||||
# A stack change with no version bump cannot be released, because the
|
||||
# release would carry a change nobody named. Scoped to what
|
||||
@@ -192,15 +244,24 @@ jobs:
|
||||
echo 'Then `docs verify` holds VERSION and CHANGES.md together.'
|
||||
exit 1
|
||||
|
||||
- name: Export the distribution
|
||||
run: tools/wikitool dist export "$DIST_DIR"
|
||||
- name: Build a release tarball
|
||||
# The same form release.yml builds - a `dist export` tree under exactly one
|
||||
# top-level folder, plus its .sha256 - so the replay below starts where a
|
||||
# user starts: from the archive, not from an exported tree.
|
||||
run: |
|
||||
set -eu
|
||||
name=chemenu-stack-ci
|
||||
mkdir -p "$BUILD_DIR"
|
||||
tools/wikitool dist export "${BUILD_DIR}/${name}"
|
||||
tar -czf "${BUILD_DIR}/${name}.tar.gz" -C "$BUILD_DIR" "$name"
|
||||
( cd "$BUILD_DIR" && sha256sum "${name}.tar.gz" > "${name}.tar.gz.sha256" )
|
||||
|
||||
- name: The distribution works as a fresh instance
|
||||
# Replays instructions/setup-instance.md end to end, minus its four
|
||||
# interactive decision points. What this tests is the release artifact
|
||||
# as an artifact: the documented path from an unpacked export to a
|
||||
# verified instance. Running one `instructions verify` against the
|
||||
# export would only have re-checked the file it just copied.
|
||||
# Replays instructions/setup-instance.md end to end, minus its interactive
|
||||
# decision points. What this tests is the release artifact as an
|
||||
# artifact: the documented path from the preflight asset in an empty
|
||||
# folder to a verified instance. Step 0 runs the asset with --archive
|
||||
# instead of a download, which is the one difference from a real install.
|
||||
#
|
||||
# Personalization is stubbed the same way the identity is: the real
|
||||
# step interviews the user, so CI substitutes a fixed answer - here,
|
||||
@@ -210,15 +271,27 @@ jobs:
|
||||
# what a person would write into them.
|
||||
run: |
|
||||
set -eu
|
||||
cd "$DIST_DIR"
|
||||
mkdir -p "$INSTANCE_DIR"
|
||||
cp tools/preflight.sh "$INSTANCE_DIR/preflight.sh"
|
||||
cd "$INSTANCE_DIR"
|
||||
sh preflight.sh --archive "${BUILD_DIR}/chemenu-stack-ci.tar.gz"
|
||||
# Installed in place: the asset is gone, the tree is here.
|
||||
test ! -e preflight.sh
|
||||
test -f tools/preflight.sh
|
||||
git init -q -b main
|
||||
git config user.name "CI Instance"
|
||||
git config user.email "ci@example.invalid"
|
||||
for personal in USER SOUL; do
|
||||
grep -v 'wikitool:template-unfilled' "$personal.md.template" > "$personal.md"
|
||||
done
|
||||
python3 -m venv tools/.venv
|
||||
tools/.venv/bin/pip install --quiet -r tools/requirements.txt
|
||||
# The authoring conventions ride the same split one directory down,
|
||||
# and are stubbed the same way: what is under test is that the export
|
||||
# carries the templates and that `doctor`/`docs verify` accept an
|
||||
# adopted one, not what a person would write into them. The collection
|
||||
# contracts are adopted verbatim - the shipped text is a working
|
||||
# default, unlike a personalization file.
|
||||
grep -v 'wikitool:template-unfilled' kb/CONVENTIONS.md.template > kb/CONVENTIONS.md
|
||||
tools/wikitool dist adopt
|
||||
tools/wikitool instructions sync
|
||||
tools/wikitool index rebuild
|
||||
tools/wikitool sources rebuild-index
|
||||
@@ -230,3 +303,103 @@ jobs:
|
||||
# A fresh instance owes no migration: dist export declares its content
|
||||
# version, so `status` must answer rather than ask for a baseline.
|
||||
tools/wikitool migrate status
|
||||
# Telemetry defaults off for a distributed instance (the
|
||||
# .wikitool-release.json this export carries), so nothing above
|
||||
# should have created a trace tree at all - see docs/explanation/
|
||||
# telemetry-and-evaluation.md § "Whether it runs at all".
|
||||
if [ -e reports/telemetry ]; then
|
||||
echo "reports/telemetry/ exists in a fresh distributed instance - telemetry should default off"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: The bug-report collector works in the distribution
|
||||
# tools/bugreport.py is the one part of the stack that must run when
|
||||
# nothing else does, so it is exercised as shipped: from the export,
|
||||
# through the tools/bugreport launcher the instructions name, which picks
|
||||
# the runner's plain python3, in the instance the step above set up.
|
||||
# The unit tests cover what it collects; this covers that it arrives.
|
||||
run: |
|
||||
set -eu
|
||||
cd "$INSTANCE_DIR"
|
||||
tools/bugreport --no-trace
|
||||
bundle=$(ls -d reports/bugreport-*/ | head -n 1)
|
||||
test -f "$bundle/MANIFEST.md"
|
||||
test -f "$bundle/environment.json"
|
||||
grep -q '`wikitool` started' "$bundle/MANIFEST.md"
|
||||
ls reports/bugreport-*.zip
|
||||
# The same collector with --pseudonymise: the mapping lies beside the
|
||||
# bundle, not in it or its zip, and this container's hostname is gone.
|
||||
rm -rf reports/bugreport-*
|
||||
tools/bugreport --no-trace --pseudonymise
|
||||
bundle=$(ls -d reports/bugreport-*/ | head -n 1)
|
||||
bundle=${bundle%/}
|
||||
test -f "$bundle.pseudonyms.json"
|
||||
test -f "$bundle.review.txt"
|
||||
grep -q 'stage 1 applied, stage 2 not yet applied' "$bundle/MANIFEST.md"
|
||||
python3 - "$bundle" <<'PY'
|
||||
import socket, sys, zipfile
|
||||
from pathlib import Path
|
||||
bundle = Path(sys.argv[1])
|
||||
names = zipfile.ZipFile(str(bundle) + ".zip").namelist()
|
||||
assert not [n for n in names if "pseudonyms" in n or "review" in n], names
|
||||
host = socket.gethostname()
|
||||
for path in bundle.rglob("*"):
|
||||
if path.is_file():
|
||||
assert host not in path.read_text(encoding="utf-8", errors="replace"), path
|
||||
PY
|
||||
|
||||
pwsh:
|
||||
runs-on: linux-docker
|
||||
container:
|
||||
image: gitea.nehmer.net/torben/chemenu-ci-pwsh:latest
|
||||
env:
|
||||
WIKITOOL_SESSION_ID: ci-pwsh-${{ github.run_id }}
|
||||
WIKI_TRACE_DIR: /tmp/wikitool-trace
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
|
||||
- name: PSScriptAnalyzer
|
||||
# Positional arguments are excluded: the rule is written for cmdlets, and the two
|
||||
# scripts call their own small helpers positionally throughout. Everything else the
|
||||
# analyzer knows must stay silent, which includes the ASCII-only and approved-verb rules.
|
||||
run: |
|
||||
set -eu
|
||||
pwsh -NoProfile -Command '
|
||||
$found = foreach ($script in Get-ChildItem tools -Filter *.ps1) {
|
||||
Invoke-ScriptAnalyzer -Path $script.FullName -ExcludeRule PSAvoidUsingPositionalParameters
|
||||
}
|
||||
$found | Format-List RuleName, ScriptName, Line, Message | Out-String -Width 200 | Write-Output
|
||||
if (@($found).Count -gt 0) { exit 1 }
|
||||
'
|
||||
|
||||
- name: Preflight, twice, against the POSIX one
|
||||
# The two preflights answer the same questions from the same list and must write the
|
||||
# same file: that is what keeps a tools/wikitool launched from either shell starting
|
||||
# the same git, rg and Python. The second run must change nothing.
|
||||
run: |
|
||||
set -eu
|
||||
git config --global --add safe.directory "$GITHUB_WORKSPACE"
|
||||
pwsh -NoProfile -ExecutionPolicy Bypass -File tools/preflight.ps1
|
||||
cp .wikitool-tools.json /tmp/tools-pwsh.json
|
||||
pwsh -NoProfile -ExecutionPolicy Bypass -File tools/preflight.ps1 > /tmp/preflight-second.txt
|
||||
cmp .wikitool-tools.json /tmp/tools-pwsh.json
|
||||
rm .wikitool-tools.json
|
||||
tools/preflight.sh
|
||||
cmp .wikitool-tools.json /tmp/tools-pwsh.json
|
||||
tools/.venv/bin/python -m pip install --quiet pytest
|
||||
|
||||
- name: The launcher, started from PowerShell
|
||||
# `tools/wikitool` from pwsh resolves to wikitool.ps1, not the sh launcher - the one
|
||||
# thing a Linux shell cannot show, so it is asked for by that exact string.
|
||||
run: |
|
||||
set -eu
|
||||
pwsh -NoProfile -Command './tools/wikitool version show'
|
||||
|
||||
- name: PowerShell tests
|
||||
# Skipped everywhere without pwsh, so this is the run that counts. The `verify` job
|
||||
# runs the rest of the suite.
|
||||
run: |
|
||||
set -eu
|
||||
cd tools
|
||||
.venv/bin/python -m pytest -q chemenu/tests/test_preflight_pwsh.py chemenu/tests/test_preflight.py chemenu/tests/test_doctor.py chemenu/tests/test_bugreport_launcher.py
|
||||
@@ -79,9 +79,7 @@ jobs:
|
||||
git config --global --add safe.directory "$GITHUB_WORKSPACE"
|
||||
git config --global user.name "Nightly"
|
||||
git config --global user.email "nightly@example.invalid"
|
||||
python3 -m venv tools/.venv
|
||||
tools/.venv/bin/pip install --quiet --upgrade pip
|
||||
tools/.venv/bin/pip install --quiet -r tools/requirements.txt
|
||||
tools/preflight.sh
|
||||
tools/wikitool instructions sync
|
||||
|
||||
- name: The instance is still correctly configured
|
||||
|
||||
@@ -0,0 +1,118 @@
|
||||
# Builds the image the `pwsh` job of ci.yml runs in: Debian, PowerShell 7 and PSScriptAnalyzer
|
||||
# (Gitea #151, D37). It lives in this Gitea instance's registry as
|
||||
# `gitea.nehmer.net/torben/chemenu-ci-pwsh`.
|
||||
#
|
||||
# The image follows the current PowerShell release, not a pin: users run whatever pwsh is
|
||||
# current, so that is what the preflight has to be tested against. A change to the Dockerfile
|
||||
# or to this file rebuilds it, a month turning over rebuilds it so the Debian layers do not age
|
||||
# unnoticed, and a run triggered by hand can build an older release with `pwsh_version`.
|
||||
#
|
||||
# Tags: `:<pwsh-version>` always, `:latest` only when that version is the current release.
|
||||
#
|
||||
# Runner shape follows `sp-live-image.yml`: the `container-builder` label, a remote BuildKit
|
||||
# on the runner host, and the registry login from 1Password.
|
||||
#
|
||||
# After the very first push the package has to be linked to this repository once, by hand, in
|
||||
# the Gitea UI - a step no workflow can do. Until then the image builds and pulls fine; only
|
||||
# the package page shows no repository.
|
||||
|
||||
name: pwsh CI image
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
paths:
|
||||
- '.gitea/pwsh-ci/**'
|
||||
- '.gitea/workflows/pwsh-ci-image.yml'
|
||||
schedule:
|
||||
- cron: '30 4 1 * *'
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
pwsh_version:
|
||||
description: 'PowerShell release to build (default: the current one)'
|
||||
required: false
|
||||
|
||||
env:
|
||||
REGISTRY: gitea.nehmer.net/torben
|
||||
IMAGE_NAME: chemenu-ci-pwsh
|
||||
|
||||
jobs:
|
||||
build-and-push:
|
||||
runs-on: container-builder
|
||||
container:
|
||||
image: debian:trixie-slim
|
||||
steps:
|
||||
- name: Install CI dependencies
|
||||
# `nodejs` is for act_runner's JavaScript actions, `unzip` for
|
||||
# 1password/load-secrets-action - see sp-live-image.yml.
|
||||
run: |
|
||||
set -eu
|
||||
apt-get update -qq
|
||||
apt-get install -y --no-install-recommends \
|
||||
git nodejs curl docker-cli docker-buildx unzip ca-certificates iproute2 gawk
|
||||
|
||||
- uses: actions/checkout@v7
|
||||
|
||||
- name: Resolve the PowerShell release
|
||||
id: pwsh
|
||||
env:
|
||||
REQUESTED: ${{ inputs.pwsh_version }}
|
||||
run: |
|
||||
set -eu
|
||||
current="$(curl -fsSL https://api.github.com/repos/PowerShell/PowerShell/releases/latest \
|
||||
| sed -n 's/.*"tag_name": *"v\([^"]*\)".*/\1/p' | head -n 1)"
|
||||
test -n "$current"
|
||||
wanted="${REQUESTED:-$current}"
|
||||
{
|
||||
echo "version=$wanted"
|
||||
echo "current=$current"
|
||||
} >> "$GITHUB_OUTPUT"
|
||||
echo "wanted $wanted, current $current"
|
||||
|
||||
- name: Load secrets from 1Password
|
||||
uses: 1password/load-secrets-action@v2
|
||||
with:
|
||||
export-env: true
|
||||
env:
|
||||
OP_SERVICE_ACCOUNT_TOKEN: ${{ secrets.OP_SERVICE_ACCOUNT_TOKEN }}
|
||||
REGISTRY_USER: op://CI-CD/gitea-package-token/username
|
||||
REGISTRY_PAT: op://CI-CD/gitea-package-token/password
|
||||
|
||||
- name: BuildKit setup (remote builder)
|
||||
run: |
|
||||
HOST_IP=$(ip route | awk '/default/ { print $3 }')
|
||||
docker buildx create --name remote-builder --driver remote tcp://$HOST_IP:1234 --use --bootstrap
|
||||
|
||||
- name: Log in to the container registry
|
||||
run: |
|
||||
echo "$REGISTRY_PAT" | docker login gitea.nehmer.net -u "$REGISTRY_USER" --password-stdin
|
||||
|
||||
- name: Decide the tags
|
||||
id: decide
|
||||
env:
|
||||
PWSH_VERSION: ${{ steps.pwsh.outputs.version }}
|
||||
CURRENT: ${{ steps.pwsh.outputs.current }}
|
||||
run: |
|
||||
set -eu
|
||||
tags="$REGISTRY/$IMAGE_NAME:$PWSH_VERSION"
|
||||
if [ "$PWSH_VERSION" = "$CURRENT" ]; then
|
||||
tags="$tags
|
||||
$REGISTRY/$IMAGE_NAME:latest"
|
||||
fi
|
||||
{
|
||||
echo "tags<<EOF"
|
||||
echo "$tags"
|
||||
echo "EOF"
|
||||
} >> "$GITHUB_OUTPUT"
|
||||
echo "tags: $tags"
|
||||
|
||||
- name: Build and push
|
||||
uses: docker/build-push-action@v6
|
||||
with:
|
||||
context: .gitea/pwsh-ci
|
||||
file: .gitea/pwsh-ci/Dockerfile
|
||||
platforms: linux/amd64
|
||||
push: true
|
||||
tags: ${{ steps.decide.outputs.tags }}
|
||||
build-args: |
|
||||
PWSH_VERSION=${{ steps.pwsh.outputs.version }}
|
||||
@@ -3,7 +3,11 @@
|
||||
# The release artifact is exactly a `dist export` tree, packed with a top-level
|
||||
# directory: unpack it, run instructions/setup-instance.md, and there is a
|
||||
# working wiki instance - no checkout of this repo required. CI already proved
|
||||
# that path works before this workflow ever runs.
|
||||
# that path works before this workflow ever runs. Beside it the release carries
|
||||
# tools/preflight.sh and tools/preflight.ps1 as assets, which download and unpack
|
||||
# that tarball themselves, and the two instructions an agent reads before there is
|
||||
# a tree to read them in: setup-instance.md, which the installation sentence in
|
||||
# INSTALL.md points at, and the preflight.md its first step leads to.
|
||||
#
|
||||
# The tag is created here, by CI, and never by an agent: AGENTS.md invariant 5
|
||||
# ("never call raw git commit/push") stays intact because nothing in a session
|
||||
@@ -13,6 +17,13 @@
|
||||
# instance issues - not a 1Password secret. Nothing here reaches outside the
|
||||
# instance, so the Zero-Trust secret path that the container-build workflows
|
||||
# use has nothing to carry.
|
||||
#
|
||||
# `wiki-mirror` is the second job: once `release` has published a release, it
|
||||
# mirrors that tree's docs into the Gitea wiki (.gitea/scripts/wiki_mirror.py
|
||||
# converts, wiki-push.sh pushes; wiki-mirror.yml does the same by hand for any
|
||||
# tag). It is a separate job so that a failing mirror leaves the release and its
|
||||
# tag standing: the release is the product, the wiki a view of it. It does not
|
||||
# run for a `-beta.N` candidate, for which `release` sets `skip`.
|
||||
|
||||
name: Release
|
||||
|
||||
@@ -21,6 +32,11 @@ on:
|
||||
branches: [main]
|
||||
paths:
|
||||
- VERSION
|
||||
# A release whose job failed after VERSION had already moved cannot be
|
||||
# retried by a push - the version is not raised again - and a re-run uses the
|
||||
# workflow file of the failed commit. Dispatching on main runs the current
|
||||
# file; the "already exists" check below still refuses a second release.
|
||||
workflow_dispatch:
|
||||
|
||||
jobs:
|
||||
release:
|
||||
@@ -29,6 +45,10 @@ jobs:
|
||||
image: debian:trixie-slim
|
||||
permissions:
|
||||
contents: write
|
||||
outputs:
|
||||
skip: ${{ steps.version.outputs.skip }}
|
||||
version: ${{ steps.version.outputs.version }}
|
||||
tag: ${{ steps.version.outputs.tag }}
|
||||
env:
|
||||
WIKITOOL_SESSION_ID: release-${{ github.run_id }}
|
||||
WIKI_TRACE_DIR: /tmp/wikitool-trace
|
||||
@@ -54,9 +74,7 @@ jobs:
|
||||
run: |
|
||||
set -eu
|
||||
git config --global --add safe.directory "$GITHUB_WORKSPACE"
|
||||
python3 -m venv tools/.venv
|
||||
tools/.venv/bin/pip install --quiet --upgrade pip
|
||||
tools/.venv/bin/pip install --quiet -r tools/requirements.txt
|
||||
tools/preflight.sh
|
||||
|
||||
- name: Resolve the version and refuse to re-release it
|
||||
id: version
|
||||
@@ -66,6 +84,26 @@ jobs:
|
||||
run: |
|
||||
set -eu
|
||||
version="$(cat VERSION | tr -d '[:space:]')"
|
||||
|
||||
# A running candidate (`X.Y.Z-beta.N`) is never released - betas are
|
||||
# a dev-checkout state, not a distributed one (see
|
||||
# instructions/dev/version-parts.md). This guard sits *before* the
|
||||
# API query below: without it, every `version bump` on a candidate
|
||||
# would push VERSION and trigger a wasted round-trip against the
|
||||
# releases API for a tag that was never going to be created. Ending
|
||||
# the job cleanly here (not `exit 1`) is what keeps a beta bump a
|
||||
# normal, unremarkable push rather than a failing CI run - skipping
|
||||
# every later step is what "cleanly" means in Actions: mark this one
|
||||
# skip and gate the rest on it.
|
||||
case "$version" in
|
||||
*-beta.*)
|
||||
echo "VERSION is a running candidate (${version}) - nothing to release. Skipping."
|
||||
echo "skip=true" >> "$GITHUB_OUTPUT"
|
||||
exit 0
|
||||
;;
|
||||
esac
|
||||
echo "skip=false" >> "$GITHUB_OUTPUT"
|
||||
|
||||
tag="v${version}"
|
||||
echo "version=${version}" >> "$GITHUB_OUTPUT"
|
||||
echo "tag=${tag}" >> "$GITHUB_OUTPUT"
|
||||
@@ -82,14 +120,48 @@ jobs:
|
||||
- name: Release notes from CHANGES.md
|
||||
# `version notes` fails when the changelog has no entry for this
|
||||
# version, which is the last place that mistake can still be caught.
|
||||
#
|
||||
# The footer below settles Gitea #47's second side-finding: a release
|
||||
# note is written once, at tag time, and a later correction to
|
||||
# CHANGES.md never reaches it - `gitea-mcp` has no release-edit method,
|
||||
# and delete-and-recreate would destroy the attached tarball assets that
|
||||
# INSTALL.md and `version check` point at. That happened for real to
|
||||
# v4.4.0, whose note carried a fact that the corpus had already
|
||||
# corrected. Rather than build a correction path for a text nobody can
|
||||
# edit, the snapshot says it is one and names where the maintained
|
||||
# version lives. A stale note then costs a reader one click instead of
|
||||
# a wrong belief. Appended here rather than inside `version notes`,
|
||||
# which is a general-purpose extractor whose other callers (a local
|
||||
# preview, a pipe) should not inherit a release-page footer.
|
||||
if: steps.version.outputs.skip != 'true'
|
||||
run: |
|
||||
set -eu
|
||||
tools/wikitool docs verify
|
||||
tools/wikitool version notes > /tmp/release-notes.md
|
||||
cat >> /tmp/release-notes.md <<'EOF'
|
||||
|
||||
---
|
||||
|
||||
*This note is a snapshot of the `CHANGES.md` entry as it stood when the tag was cut, and
|
||||
is never edited afterwards. The maintained version of this text - including any later
|
||||
correction - is the entry for this version in `CHANGES.md` in the repository.*
|
||||
EOF
|
||||
cat /tmp/release-notes.md
|
||||
|
||||
# Gitea keeps a release note in a TEXT column, which on this instance's
|
||||
# MySQL holds 65535 bytes. A longer note fails the API call after the
|
||||
# tarball is built; refusing here keeps the margin visible and the tag
|
||||
# uncreated. The fix is a shorter CHANGES.md entry, not a cut note.
|
||||
size="$(wc -c < /tmp/release-notes.md)"
|
||||
if [ "$size" -gt 60000 ]; then
|
||||
echo "Release notes are ${size} bytes; Gitea stores at most 65535 on MySQL."
|
||||
echo "Shorten this version's CHANGES.md entry, then dispatch this workflow on main."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: Build the distribution tarball
|
||||
id: build
|
||||
if: steps.version.outputs.skip != 'true'
|
||||
env:
|
||||
VERSION: ${{ steps.version.outputs.version }}
|
||||
TAG: ${{ steps.version.outputs.tag }}
|
||||
@@ -108,7 +180,36 @@ jobs:
|
||||
cat "${BUILD_DIR}/${name}.tar.gz.sha256"
|
||||
echo "name=${name}" >> "$GITHUB_OUTPUT"
|
||||
|
||||
# The two preflight scripts are attached to the release as well: the first
|
||||
# thing a new user runs, before there is any tree to run it from. Each copy
|
||||
# is the tree's script with the download address of *this* release written
|
||||
# into its two placeholder lines (the tree copy keeps them empty, which is
|
||||
# how a script knows it is not a release asset). The address is the public
|
||||
# one, for the same reason as the URLs in `dist export` above.
|
||||
download="${PUBLIC_BASE_URL}/${GITHUB_REPOSITORY}/releases/download/${TAG}"
|
||||
sed \
|
||||
-e "s|^RELEASE_ARCHIVE_URL=''|RELEASE_ARCHIVE_URL='${download}/${name}.tar.gz'|" \
|
||||
-e "s|^RELEASE_CHECKSUM_URL=''|RELEASE_CHECKSUM_URL='${download}/${name}.tar.gz.sha256'|" \
|
||||
tools/preflight.sh > "${BUILD_DIR}/preflight.sh"
|
||||
sed \
|
||||
-e "s|^\$ReleaseArchiveUrl = ''|\$ReleaseArchiveUrl = '${download}/${name}.tar.gz'|" \
|
||||
-e "s|^\$ReleaseChecksumUrl = ''|\$ReleaseChecksumUrl = '${download}/${name}.tar.gz.sha256'|" \
|
||||
tools/preflight.ps1 > "${BUILD_DIR}/preflight.ps1"
|
||||
# A placeholder that did not match would ship a script that refuses to run.
|
||||
grep -qF "RELEASE_ARCHIVE_URL='${download}/${name}.tar.gz'" "${BUILD_DIR}/preflight.sh"
|
||||
grep -qF "RELEASE_CHECKSUM_URL='${download}/${name}.tar.gz.sha256'" "${BUILD_DIR}/preflight.sh"
|
||||
grep -qF "ReleaseArchiveUrl = '${download}/${name}.tar.gz'" "${BUILD_DIR}/preflight.ps1"
|
||||
grep -qF "ReleaseChecksumUrl = '${download}/${name}.tar.gz.sha256'" "${BUILD_DIR}/preflight.ps1"
|
||||
chmod +x "${BUILD_DIR}/preflight.sh"
|
||||
|
||||
# The two instructions as the tarball carries them (dist export has already
|
||||
# stripped them), not as this checkout holds them.
|
||||
for doc in setup-instance.md preflight.md; do
|
||||
cp "${BUILD_DIR}/${name}/instructions/${doc}" "${BUILD_DIR}/${doc}"
|
||||
done
|
||||
|
||||
- name: Publish the release
|
||||
if: steps.version.outputs.skip != 'true'
|
||||
env:
|
||||
API: ${{ github.server_url }}/api/v1/repos/${{ github.repository }}
|
||||
TOKEN: ${{ gitea.token }}
|
||||
@@ -117,22 +218,25 @@ jobs:
|
||||
run: |
|
||||
set -eu
|
||||
# Creating the release creates the tag, pinned to this commit.
|
||||
payload="$(jq -n \
|
||||
# The payload goes through a file: as one argument it is bounded by
|
||||
# Linux's 128 KiB per-argument limit, which the 8.0.0 notes exceeded
|
||||
# ("curl: Argument list too long").
|
||||
jq -n \
|
||||
--arg tag "$TAG" \
|
||||
--arg target "$GITHUB_SHA" \
|
||||
--arg name "$TAG" \
|
||||
--rawfile body /tmp/release-notes.md \
|
||||
'{tag_name: $tag, target_commitish: $target, name: $name, body: $body,
|
||||
draft: false, prerelease: false}')"
|
||||
draft: false, prerelease: false}' > /tmp/release-payload.json
|
||||
|
||||
release="$(curl -sS -f -X POST "${API}/releases" \
|
||||
-H "Authorization: token ${TOKEN}" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d "$payload")"
|
||||
--data-binary @/tmp/release-payload.json)"
|
||||
id="$(printf '%s' "$release" | jq -r '.id')"
|
||||
echo "Created release ${TAG} (id ${id})."
|
||||
|
||||
for asset in "${NAME}.tar.gz" "${NAME}.tar.gz.sha256"; do
|
||||
for asset in "${NAME}.tar.gz" "${NAME}.tar.gz.sha256" preflight.sh preflight.ps1 setup-instance.md preflight.md; do
|
||||
curl -sS -f -X POST "${API}/releases/${id}/assets?name=${asset}" \
|
||||
-H "Authorization: token ${TOKEN}" \
|
||||
-F "attachment=@${BUILD_DIR}/${asset}" > /dev/null
|
||||
@@ -140,3 +244,44 @@ jobs:
|
||||
done
|
||||
|
||||
echo "Done: ${PUBLIC_BASE_URL}/${GITHUB_REPOSITORY}/releases/tag/${TAG}"
|
||||
|
||||
wiki-mirror:
|
||||
# Runs only after `release` succeeded and created a release (a candidate
|
||||
# sets skip=true). `release` never depends on this job.
|
||||
needs: release
|
||||
if: needs.release.outputs.skip != 'true'
|
||||
runs-on: linux-docker
|
||||
container:
|
||||
image: debian:trixie-slim
|
||||
permissions:
|
||||
contents: write
|
||||
wiki: write
|
||||
env:
|
||||
VERSION: ${{ needs.release.outputs.version }}
|
||||
TAG: ${{ needs.release.outputs.tag }}
|
||||
PUBLIC_BASE_URL: https://gitea.nehmer.net
|
||||
|
||||
steps:
|
||||
- name: System dependencies
|
||||
run: |
|
||||
set -eu
|
||||
apt-get update -qq
|
||||
apt-get install -y --no-install-recommends python3 git nodejs ca-certificates curl jq
|
||||
rm -rf /var/lib/apt/lists/*
|
||||
|
||||
- uses: actions/checkout@v7
|
||||
|
||||
- name: Mirror the docs into the wiki
|
||||
env:
|
||||
WIKI_TOKEN: ${{ gitea.token }}
|
||||
run: |
|
||||
set -eu
|
||||
python3 .gitea/scripts/wiki_mirror.py --source . --out /tmp/wiki-out \
|
||||
--version "$VERSION" --repo-url "${PUBLIC_BASE_URL}/${GITHUB_REPOSITORY}"
|
||||
.gitea/scripts/wiki-push.sh /tmp/wiki-out "$VERSION"
|
||||
|
||||
- name: Say which half failed
|
||||
if: failure()
|
||||
run: |
|
||||
echo "Release ${TAG} is published; only the wiki mirror failed."
|
||||
echo "The release and its tag stand. Dispatch wiki-mirror.yml with tag ${TAG} to retry the mirror."
|
||||
@@ -0,0 +1,141 @@
|
||||
# Builds the image the nightly `tracker-live` run executes in: Debian, the packaged Super
|
||||
# Productivity, a virtual display and the tools the suite needs (Gitea #156). It lives in
|
||||
# this Gitea instance's registry as `gitea.nehmer.net/torben/chemenu-sp-live`.
|
||||
#
|
||||
# The image follows the update channel, not a pin. Installed desktop apps update themselves,
|
||||
# so a pinned old version would be tested while users already run the new one. Every day this
|
||||
# workflow asks `latest-linux.yml` (`.gitea/sp-live/resolve-version.sh`) which release is
|
||||
# current, and builds only when the registry does not hold that tag yet. It also rebuilds once
|
||||
# a month regardless, so the Debian layers behind the app do not age unnoticed.
|
||||
#
|
||||
# Tags: `:<sp-version>` always, `:latest` only when that version is what the channel says.
|
||||
# A manual run with `sp_version` builds an older release (to reproduce a red night against
|
||||
# the version it went red on) and therefore never moves `:latest`.
|
||||
#
|
||||
# Runner shape follows torben/gitea-mcp, `.gitea/workflows/binford-release.yaml`: the
|
||||
# `container-builder` label, a remote BuildKit on the runner host, and the registry login from
|
||||
# 1Password. `OP_SERVICE_ACCOUNT_TOKEN` is a user-level secret that covers `torben/*`.
|
||||
#
|
||||
# After the very first push the package has to be linked to this repository once, by hand, in
|
||||
# the Gitea UI - a step no workflow can do. Until then the image builds and pulls fine; only
|
||||
# the package page shows no repository.
|
||||
|
||||
name: SP live image
|
||||
|
||||
on:
|
||||
schedule:
|
||||
# 04:10 UTC, an hour after `nightly` and well before `tracker-live` (05:00), so a new
|
||||
# release is in the registry by the time the suite looks for it.
|
||||
- cron: '10 4 * * *'
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
sp_version:
|
||||
description: 'Super Productivity release to build (default: the current one)'
|
||||
required: false
|
||||
force:
|
||||
description: 'Rebuild even if the tag already exists (true/false)'
|
||||
required: false
|
||||
default: 'false'
|
||||
|
||||
env:
|
||||
REGISTRY: gitea.nehmer.net/torben
|
||||
IMAGE_NAME: chemenu-sp-live
|
||||
|
||||
jobs:
|
||||
build-and-push:
|
||||
runs-on: container-builder
|
||||
container:
|
||||
image: debian:trixie-slim
|
||||
steps:
|
||||
- name: Install CI dependencies
|
||||
# `nodejs` is for act_runner's JavaScript actions, not for us - see ci.yml. `unzip` is for
|
||||
# 1password/load-secrets-action, which unpacks its CLI with it and fails with exit 127
|
||||
# without it.
|
||||
run: |
|
||||
set -eu
|
||||
apt-get update -qq
|
||||
apt-get install -y --no-install-recommends \
|
||||
git nodejs curl docker-cli docker-buildx unzip ca-certificates iproute2 gawk
|
||||
|
||||
- uses: actions/checkout@v7
|
||||
|
||||
- name: Resolve the Super Productivity release
|
||||
id: sp
|
||||
env:
|
||||
REQUESTED: ${{ inputs.sp_version }}
|
||||
run: |
|
||||
set -eu
|
||||
channel="$(.gitea/sp-live/resolve-version.sh latest)"
|
||||
channel_version="$(printf '%s\n' "$channel" | sed -n 's/^version=//p')"
|
||||
if [ -n "${REQUESTED:-}" ]; then
|
||||
wanted="$(.gitea/sp-live/resolve-version.sh "$REQUESTED")"
|
||||
else
|
||||
wanted="$channel"
|
||||
fi
|
||||
{
|
||||
printf '%s\n' "$wanted"
|
||||
echo "channel_version=$channel_version"
|
||||
} >> "$GITHUB_OUTPUT"
|
||||
printf '%s\n' "$wanted"
|
||||
|
||||
- name: Load secrets from 1Password
|
||||
uses: 1password/load-secrets-action@v2
|
||||
with:
|
||||
export-env: true
|
||||
env:
|
||||
OP_SERVICE_ACCOUNT_TOKEN: ${{ secrets.OP_SERVICE_ACCOUNT_TOKEN }}
|
||||
REGISTRY_USER: op://CI-CD/gitea-package-token/username
|
||||
REGISTRY_PAT: op://CI-CD/gitea-package-token/password
|
||||
|
||||
- name: BuildKit setup (remote builder)
|
||||
run: |
|
||||
HOST_IP=$(ip route | awk '/default/ { print $3 }')
|
||||
docker buildx create --name remote-builder --driver remote tcp://$HOST_IP:1234 --use --bootstrap
|
||||
|
||||
- name: Log in to the container registry
|
||||
run: |
|
||||
echo "$REGISTRY_PAT" | docker login gitea.nehmer.net -u "$REGISTRY_USER" --password-stdin
|
||||
|
||||
- name: Decide whether to build
|
||||
id: decide
|
||||
env:
|
||||
SP_VERSION: ${{ steps.sp.outputs.version }}
|
||||
CHANNEL_VERSION: ${{ steps.sp.outputs.channel_version }}
|
||||
FORCE: ${{ inputs.force }}
|
||||
run: |
|
||||
set -eu
|
||||
ref="$REGISTRY/$IMAGE_NAME:$SP_VERSION"
|
||||
build=false
|
||||
why=""
|
||||
if [ "${FORCE:-false}" = true ]; then
|
||||
build=true; why="forced"
|
||||
elif [ "$(date -u +%d)" = 01 ]; then
|
||||
build=true; why="monthly rebuild"
|
||||
elif ! docker buildx imagetools inspect "$ref" > /dev/null 2>&1; then
|
||||
build=true; why="$ref is not in the registry yet"
|
||||
fi
|
||||
tags="$ref"
|
||||
if [ "$SP_VERSION" = "$CHANNEL_VERSION" ]; then
|
||||
tags="$tags
|
||||
$REGISTRY/$IMAGE_NAME:latest"
|
||||
fi
|
||||
{
|
||||
echo "build=$build"
|
||||
echo "tags<<EOF"
|
||||
echo "$tags"
|
||||
echo "EOF"
|
||||
} >> "$GITHUB_OUTPUT"
|
||||
echo "build=$build ${why:+($why)}; tags: $tags"
|
||||
|
||||
- name: Build and push
|
||||
if: steps.decide.outputs.build == 'true'
|
||||
uses: docker/build-push-action@v6
|
||||
with:
|
||||
context: .gitea/sp-live
|
||||
file: .gitea/sp-live/Dockerfile
|
||||
platforms: linux/amd64
|
||||
push: true
|
||||
tags: ${{ steps.decide.outputs.tags }}
|
||||
build-args: |
|
||||
SP_VERSION=${{ steps.sp.outputs.version }}
|
||||
SP_SHA512=${{ steps.sp.outputs.sha512 }}
|
||||
@@ -0,0 +1,68 @@
|
||||
# The live tracker suite, nightly (Gitea #156): the documented `task` and `review` workflow
|
||||
# against a real Super Productivity and a real CalDAV server, not against fakes.
|
||||
#
|
||||
# `ci.yml` already runs the CalDAV half on every push (Radicale is a pip install). This
|
||||
# workflow adds the half that needs the desktop app, inside the prebuilt
|
||||
# `chemenu-sp-live` image (`sp-live-image.yml`), and runs both, so one green night covers both
|
||||
# providers.
|
||||
#
|
||||
# Red after a new Super Productivity release is the finding this workflow exists for, not a
|
||||
# flaky night: `instructions/dev/tracker-testing.md` says what to do with it. An agent that
|
||||
# touched the Super Productivity surface (`tasks/superproductivity.py`, `tasks/config.py`,
|
||||
# the fixtures) dispatches it by hand instead of waiting for the clock.
|
||||
#
|
||||
# Runner: `linux-docker`, with the image as the job container. The image carries `nodejs`, so
|
||||
# `actions/checkout` runs; see ci.yml for why that is the workflow's business.
|
||||
|
||||
name: Tracker live
|
||||
|
||||
on:
|
||||
schedule:
|
||||
- cron: '0 5 * * *'
|
||||
workflow_dispatch:
|
||||
|
||||
jobs:
|
||||
live:
|
||||
runs-on: linux-docker
|
||||
container:
|
||||
image: gitea.nehmer.net/torben/chemenu-sp-live:latest
|
||||
env:
|
||||
WIKITOOL_SESSION_ID: tracker-live-${{ github.run_id }}
|
||||
WIKI_TRACE_DIR: /tmp/wikitool-trace
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
|
||||
- name: Which Super Productivity is this
|
||||
# act_runner pulls the job image with `forcePull=true` and no credentials (observed on
|
||||
# the first run, 442), so `:latest` is always the registry's current one. What can
|
||||
# still lag is the image itself - a release the daily build has not picked up yet. So
|
||||
# the run compares what is installed with what the update channel names now.
|
||||
run: |
|
||||
set -eu
|
||||
installed="$(dpkg-query -W -f='${Version}' superproductivity)"
|
||||
channel="$(.gitea/sp-live/resolve-version.sh latest | sed -n 's/^version=//p')"
|
||||
echo "installed: $installed, update channel: $channel"
|
||||
if [ "$installed" != "$channel" ]; then
|
||||
echo "::warning::the image carries Super Productivity $installed, the update channel names $channel - this run tests an outdated app (stale runner cache or an image not rebuilt yet)"
|
||||
fi
|
||||
|
||||
- name: Tool environment
|
||||
run: |
|
||||
set -eu
|
||||
git config --global --add safe.directory "$GITHUB_WORKSPACE"
|
||||
tools/preflight.sh
|
||||
tools/.venv/bin/python -m pip install --quiet pytest radicale
|
||||
|
||||
- name: Start Radicale
|
||||
run: .gitea/scripts/start-radicale.sh tools/.venv/bin/python /tmp/radicale
|
||||
|
||||
- name: Live tracker suite
|
||||
# Both kinds are required: a night in which the app or the server was not there must
|
||||
# fail rather than skip.
|
||||
env:
|
||||
CHEMENU_LIVE_REQUIRE: sp,caldav
|
||||
run: |
|
||||
set -eu
|
||||
cd tools
|
||||
.venv/bin/python -m pytest -q -m live_tracker -s
|
||||
@@ -0,0 +1,79 @@
|
||||
# Mirror the docs of one release tag into the Gitea wiki, by hand.
|
||||
#
|
||||
# The same two scripts as the `wiki-mirror` job in release.yml, started on demand:
|
||||
# to fill the wiki for the first time and look at it before the next release,
|
||||
# and to catch up a mirror that failed after its release was already published.
|
||||
# The wiki is a mirror and never a source - every run replaces its whole content,
|
||||
# and nothing but CI writes to it.
|
||||
#
|
||||
# The converter and the push script come from the commit this workflow runs on
|
||||
# (main); the docs come from the tag, checked out beside it. That way an old tag,
|
||||
# whose tree predates the converter, can still be mirrored.
|
||||
#
|
||||
# Input `tag` defaults to the latest release, read from the release API.
|
||||
|
||||
name: Wiki mirror
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
tag:
|
||||
description: 'Release tag to mirror (default: the latest release)'
|
||||
required: false
|
||||
default: ''
|
||||
|
||||
jobs:
|
||||
mirror:
|
||||
runs-on: linux-docker
|
||||
container:
|
||||
image: debian:trixie-slim
|
||||
permissions:
|
||||
contents: write
|
||||
wiki: write
|
||||
env:
|
||||
PUBLIC_BASE_URL: https://gitea.nehmer.net
|
||||
|
||||
steps:
|
||||
- name: System dependencies
|
||||
run: |
|
||||
set -eu
|
||||
apt-get update -qq
|
||||
apt-get install -y --no-install-recommends python3 git nodejs ca-certificates curl jq
|
||||
rm -rf /var/lib/apt/lists/*
|
||||
|
||||
- uses: actions/checkout@v7
|
||||
|
||||
- name: Resolve the tag
|
||||
id: tag
|
||||
env:
|
||||
API: ${{ github.server_url }}/api/v1/repos/${{ github.repository }}
|
||||
TOKEN: ${{ gitea.token }}
|
||||
INPUT_TAG: ${{ github.event.inputs.tag }}
|
||||
run: |
|
||||
set -eu
|
||||
tag="$INPUT_TAG"
|
||||
if [ -z "$tag" ]; then
|
||||
tag="$(curl -sS -f -H "Authorization: token ${TOKEN}" "${API}/releases/latest" | jq -r '.tag_name')"
|
||||
fi
|
||||
case "$tag" in
|
||||
v[0-9]*) ;;
|
||||
*) echo "Tag '${tag}' is not a release tag (vX.Y.Z)."; exit 1 ;;
|
||||
esac
|
||||
echo "Mirroring ${tag}."
|
||||
echo "tag=${tag}" >> "$GITHUB_OUTPUT"
|
||||
echo "version=${tag#v}" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- uses: actions/checkout@v7
|
||||
with:
|
||||
ref: ${{ steps.tag.outputs.tag }}
|
||||
path: tag-tree
|
||||
|
||||
- name: Mirror the docs into the wiki
|
||||
env:
|
||||
VERSION: ${{ steps.tag.outputs.version }}
|
||||
WIKI_TOKEN: ${{ gitea.token }}
|
||||
run: |
|
||||
set -eu
|
||||
python3 .gitea/scripts/wiki_mirror.py --source tag-tree --out /tmp/wiki-out \
|
||||
--version "$VERSION" --repo-url "${PUBLIC_BASE_URL}/${GITHUB_REPOSITORY}"
|
||||
.gitea/scripts/wiki-push.sh /tmp/wiki-out "$VERSION"
|
||||
@@ -4,8 +4,8 @@
|
||||
"sessionStart": [
|
||||
{
|
||||
"type": "command",
|
||||
"bash": "./tools/trace_ingest.py --source copilot-cli --event session.start 2>/dev/null || true",
|
||||
"powershell": "python tools/trace_ingest.py --source copilot-cli --event session.start 2>$null; exit 0",
|
||||
"bash": "./tools/trace-hook --source copilot-cli --event session.start 2>/dev/null || true",
|
||||
"powershell": ".\\tools\\.venv\\Scripts\\python.exe tools\\trace_ingest.py --source copilot-cli --event session.start 2>$null; exit 0",
|
||||
"cwd": ".",
|
||||
"timeoutSec": 5
|
||||
}
|
||||
@@ -13,8 +13,8 @@
|
||||
"sessionEnd": [
|
||||
{
|
||||
"type": "command",
|
||||
"bash": "./tools/trace_ingest.py --source copilot-cli --event session.end 2>/dev/null || true",
|
||||
"powershell": "python tools/trace_ingest.py --source copilot-cli --event session.end 2>$null; exit 0",
|
||||
"bash": "./tools/trace-hook --source copilot-cli --event session.end 2>/dev/null || true",
|
||||
"powershell": ".\\tools\\.venv\\Scripts\\python.exe tools\\trace_ingest.py --source copilot-cli --event session.end 2>$null; exit 0",
|
||||
"cwd": ".",
|
||||
"timeoutSec": 5
|
||||
}
|
||||
@@ -22,8 +22,8 @@
|
||||
"userPromptSubmitted": [
|
||||
{
|
||||
"type": "command",
|
||||
"bash": "./tools/trace_ingest.py --source copilot-cli --event prompt.submitted 2>/dev/null || true",
|
||||
"powershell": "python tools/trace_ingest.py --source copilot-cli --event prompt.submitted 2>$null; exit 0",
|
||||
"bash": "./tools/trace-hook --source copilot-cli --event prompt.submitted 2>/dev/null || true",
|
||||
"powershell": ".\\tools\\.venv\\Scripts\\python.exe tools\\trace_ingest.py --source copilot-cli --event prompt.submitted 2>$null; exit 0",
|
||||
"cwd": ".",
|
||||
"timeoutSec": 5
|
||||
}
|
||||
@@ -31,8 +31,8 @@
|
||||
"preToolUse": [
|
||||
{
|
||||
"type": "command",
|
||||
"bash": "./tools/trace_ingest.py --source copilot-cli --event tool.pre 2>/dev/null || true",
|
||||
"powershell": "python tools/trace_ingest.py --source copilot-cli --event tool.pre 2>$null; exit 0",
|
||||
"bash": "./tools/trace-hook --source copilot-cli --event tool.pre 2>/dev/null || true",
|
||||
"powershell": ".\\tools\\.venv\\Scripts\\python.exe tools\\trace_ingest.py --source copilot-cli --event tool.pre 2>$null; exit 0",
|
||||
"cwd": ".",
|
||||
"timeoutSec": 5
|
||||
}
|
||||
@@ -40,8 +40,8 @@
|
||||
"postToolUse": [
|
||||
{
|
||||
"type": "command",
|
||||
"bash": "./tools/trace_ingest.py --source copilot-cli --event tool.post 2>/dev/null || true",
|
||||
"powershell": "python tools/trace_ingest.py --source copilot-cli --event tool.post 2>$null; exit 0",
|
||||
"bash": "./tools/trace-hook --source copilot-cli --event tool.post 2>/dev/null || true",
|
||||
"powershell": ".\\tools\\.venv\\Scripts\\python.exe tools\\trace_ingest.py --source copilot-cli --event tool.post 2>$null; exit 0",
|
||||
"cwd": ".",
|
||||
"timeoutSec": 5
|
||||
}
|
||||
@@ -49,8 +49,8 @@
|
||||
"postToolUseFailure": [
|
||||
{
|
||||
"type": "command",
|
||||
"bash": "./tools/trace_ingest.py --source copilot-cli --event tool.error 2>/dev/null || true",
|
||||
"powershell": "python tools/trace_ingest.py --source copilot-cli --event tool.error 2>$null; exit 0",
|
||||
"bash": "./tools/trace-hook --source copilot-cli --event tool.error 2>/dev/null || true",
|
||||
"powershell": ".\\tools\\.venv\\Scripts\\python.exe tools\\trace_ingest.py --source copilot-cli --event tool.error 2>$null; exit 0",
|
||||
"cwd": ".",
|
||||
"timeoutSec": 5
|
||||
}
|
||||
@@ -58,8 +58,8 @@
|
||||
"errorOccurred": [
|
||||
{
|
||||
"type": "command",
|
||||
"bash": "./tools/trace_ingest.py --source copilot-cli --event session.error 2>/dev/null || true",
|
||||
"powershell": "python tools/trace_ingest.py --source copilot-cli --event session.error 2>$null; exit 0",
|
||||
"bash": "./tools/trace-hook --source copilot-cli --event session.error 2>/dev/null || true",
|
||||
"powershell": ".\\tools\\.venv\\Scripts\\python.exe tools\\trace_ingest.py --source copilot-cli --event session.error 2>$null; exit 0",
|
||||
"cwd": ".",
|
||||
"timeoutSec": 5
|
||||
}
|
||||
@@ -67,8 +67,8 @@
|
||||
"subagentStart": [
|
||||
{
|
||||
"type": "command",
|
||||
"bash": "./tools/trace_ingest.py --source copilot-cli --event subagent.start 2>/dev/null || true",
|
||||
"powershell": "python tools/trace_ingest.py --source copilot-cli --event subagent.start 2>$null; exit 0",
|
||||
"bash": "./tools/trace-hook --source copilot-cli --event subagent.start 2>/dev/null || true",
|
||||
"powershell": ".\\tools\\.venv\\Scripts\\python.exe tools\\trace_ingest.py --source copilot-cli --event subagent.start 2>$null; exit 0",
|
||||
"cwd": ".",
|
||||
"timeoutSec": 5
|
||||
}
|
||||
@@ -76,8 +76,8 @@
|
||||
"subagentStop": [
|
||||
{
|
||||
"type": "command",
|
||||
"bash": "./tools/trace_ingest.py --source copilot-cli --event subagent.stop 2>/dev/null || true",
|
||||
"powershell": "python tools/trace_ingest.py --source copilot-cli --event subagent.stop 2>$null; exit 0",
|
||||
"bash": "./tools/trace-hook --source copilot-cli --event subagent.stop 2>/dev/null || true",
|
||||
"powershell": ".\\tools\\.venv\\Scripts\\python.exe tools\\trace_ingest.py --source copilot-cli --event subagent.stop 2>$null; exit 0",
|
||||
"cwd": ".",
|
||||
"timeoutSec": 5
|
||||
}
|
||||
@@ -85,8 +85,8 @@
|
||||
"preCompact": [
|
||||
{
|
||||
"type": "command",
|
||||
"bash": "./tools/trace_ingest.py --source copilot-cli --event compaction 2>/dev/null || true",
|
||||
"powershell": "python tools/trace_ingest.py --source copilot-cli --event compaction 2>$null; exit 0",
|
||||
"bash": "./tools/trace-hook --source copilot-cli --event compaction 2>/dev/null || true",
|
||||
"powershell": ".\\tools\\.venv\\Scripts\\python.exe tools\\trace_ingest.py --source copilot-cli --event compaction 2>$null; exit 0",
|
||||
"cwd": ".",
|
||||
"timeoutSec": 5
|
||||
}
|
||||
@@ -94,8 +94,8 @@
|
||||
"agentStop": [
|
||||
{
|
||||
"type": "command",
|
||||
"bash": "./tools/trace_ingest.py --source copilot-cli --event turn.end 2>/dev/null || true",
|
||||
"powershell": "python tools/trace_ingest.py --source copilot-cli --event turn.end 2>$null; exit 0",
|
||||
"bash": "./tools/trace-hook --source copilot-cli --event turn.end 2>/dev/null || true",
|
||||
"powershell": ".\\tools\\.venv\\Scripts\\python.exe tools\\trace_ingest.py --source copilot-cli --event turn.end 2>$null; exit 0",
|
||||
"cwd": ".",
|
||||
"timeoutSec": 5
|
||||
}
|
||||
|
||||
+74
@@ -73,6 +73,11 @@ npm-debug.log*
|
||||
# "Gates") - local, per-session, never committed
|
||||
/tools/.wikitool_session/
|
||||
|
||||
# `wikitool raw capture`/`raw status` git cache (see raw/CONTRACT.md "Getting a
|
||||
# repository in") - one bare repository per captured URL, refetched on demand.
|
||||
# Derived and per-checkout; never committed, never shipped.
|
||||
/tools/.wikitool_capture/
|
||||
|
||||
# Go
|
||||
/go.mod
|
||||
/go.sum
|
||||
@@ -117,6 +122,44 @@ npm-debug.log*
|
||||
# means unrestricted; `doctor` reports which.
|
||||
/.wikitool-remotes.json
|
||||
|
||||
# Telemetry opt-in/opt-out plus its two quantity caps (see docs/explanation/
|
||||
# telemetry-and-evaluation.md and tools/chemenu/telemetry/policy.py). Per-checkout for the same reason as the
|
||||
# allowlist above: the consent to write cleartext prompts to *this* disk
|
||||
# belongs to the checkout, not the corpus, so a second clone must not inherit
|
||||
# it. Absent means the installation-form default applies; `doctor` reports
|
||||
# which.
|
||||
/.wikitool-telemetry.json
|
||||
|
||||
# MCP `submit` tool opt-in (identity header name, size deckel, extension
|
||||
# allowlist, per-submitter quota - see raw/CONTRACT.md "Getting a file in
|
||||
# from outside" and tools/chemenu/upload.py). Per-checkout for the same
|
||||
# reason as the two files above. Absent means the tool is not registered at
|
||||
# all - not "unrestricted" - the stronger of the two postures this file
|
||||
# co-locates with.
|
||||
/.wikitool-upload.json
|
||||
|
||||
# Task-tracker provider opt-in (Gitea #124, AGENTS.md's task/project routing) -
|
||||
# which provider the GTD weekly review talks to, its connection details, and
|
||||
# the review's three staleness thresholds. Per-checkout for the same reason as
|
||||
# the three files above: the provider and its credentials belong to one
|
||||
# checkout's own tracker, not to the corpus. Absent means no tracker is
|
||||
# configured; `doctor` reports which.
|
||||
/.wikitool-tasks.json
|
||||
|
||||
# Tool paths recorded by the preflight (tools/preflight.sh / .ps1, see
|
||||
# instructions/preflight.md): the absolute paths of python, git, rg - and pwsh
|
||||
# on Windows - as this machine has them. Per-checkout for the plainest reason of
|
||||
# all: a path on one computer means nothing on another. Absent means the
|
||||
# preflight has not run, and `tools/wikitool` refuses to start (exit 42).
|
||||
/.wikitool-tools.json
|
||||
|
||||
# Live-suite tracker profiles (Gitea #156): one file per tracker of the user's own that the
|
||||
# live suite may be pointed at (`CHEMENU_LIVE_PROFILE=<name>`,
|
||||
# instructions/dev/tracker-testing.md). They carry the same credentials as the file above
|
||||
# and are per-checkout for the same reason. A directory pattern, anchored: nothing in it is
|
||||
# ever tracked, so no negation has to rescue anything.
|
||||
/.wikitool-tasks.d/
|
||||
|
||||
# Coverage output from `pytest --cov` (see .gitea/workflows/ci.yml). Derived,
|
||||
# like reports/: recomputable from any commit, and `publish` runs `git add -A`,
|
||||
# so an unignored htmlcov/ would commit itself on the next content publish.
|
||||
@@ -134,6 +177,37 @@ npm-debug.log*
|
||||
/.agents/skills/
|
||||
/.claude/skills/
|
||||
|
||||
# Ingest inbox (see raw/CONTRACT.md "Getting a file in"). A human
|
||||
# drops a file here - no subdirectory carries any meaning any more, an old
|
||||
# one is merely tolerated and ignored; `wikitool raw accept` promotes it into
|
||||
# `raw/`, computing the date-sharded directory and any bundle from what was
|
||||
# passed in one call. Unlike raw/ itself this must NEVER be committed - the
|
||||
# promotion is what makes a file immutable, not the drop.
|
||||
#
|
||||
# File pattern, not directory pattern (Gitea #88): `/incoming/` used to
|
||||
# exclude the directory itself, and rule 2 above means the negation block at
|
||||
# the bottom could never re-include a file whose parent was already gone -
|
||||
# so a fresh clone never had the directory at all, only
|
||||
# `instructions/bootstrap.md` recreating it by hand. `/incoming/*` excludes
|
||||
# everything inside instead, so `!/incoming/.gitkeep` right below actually
|
||||
# applies: that one anchor file is trackable and ships with every clone.
|
||||
# Nothing else dropped here is rescued by the same rule.
|
||||
/incoming/*
|
||||
!/incoming/.gitkeep
|
||||
|
||||
# MCP `submit` tool quarantine (see raw/CONTRACT.md "Getting a file in from
|
||||
# outside" and tools/chemenu/upload.py). Material pushed by a caller that is
|
||||
# not this terminal, before a human has reviewed any of it - stronger than
|
||||
# `incoming/` above: not merely uncommitted, but read by no command in the
|
||||
# ordinary pipeline. Unlike `incoming/` above (Gitea #88), this one keeps the
|
||||
# directory form and gets no `.gitkeep`: the directory is created on demand
|
||||
# by the one function that is allowed to write into it, and a checkout that
|
||||
# never arms the `submit` tool never gets one - there is no fresh-clone case
|
||||
# to cover here, since nothing reads this path before that function creates
|
||||
# it. Never anchored back open by the content backstop below, same as
|
||||
# `incoming/`.
|
||||
/mcp-upload/
|
||||
|
||||
# Content backstop - keep this block last. Nothing under raw/, kb/ or work/ may
|
||||
# be excluded by a pattern above; see the header note for why directory patterns
|
||||
# still have to be anchored rather than relying on these negations.
|
||||
|
||||
+5
-5
@@ -5,8 +5,8 @@
|
||||
# pre_tool | post_tool | post_agent, and a `command` run through a shell with
|
||||
# the invocation JSON on stdin.
|
||||
#
|
||||
# Vibe loads this file only for a trusted folder. See ../EVALS.md for what the
|
||||
# three events can and cannot say - there is no session, prompt, compaction or
|
||||
# Vibe loads this file only for a trusted folder. See docs/how-to/trace-and-score-sessions.md
|
||||
# (from the repository root) for what the three events can and cannot say - there is no session, prompt, compaction or
|
||||
# permission hook to attach to, so a Vibe trace is thinner than a Copilot one
|
||||
# and says so in its own `completeness` list.
|
||||
|
||||
@@ -14,7 +14,7 @@
|
||||
name = "wiki-trace-pre-tool"
|
||||
type = "pre_tool"
|
||||
description = "Record an intended tool call. Observational only - never decides."
|
||||
command = "./tools/trace_ingest.py --source mistral-vibe --event tool.pre 2>/dev/null || true"
|
||||
command = "./tools/trace-hook --source mistral-vibe --event tool.pre 2>/dev/null || true"
|
||||
timeout = 5.0
|
||||
# `strict = false` is the default and is spelled out here because it is the
|
||||
# safety property that matters: under a non-strict hook, a crash or a timeout is
|
||||
@@ -26,7 +26,7 @@ strict = false
|
||||
name = "wiki-trace-post-tool"
|
||||
type = "post_tool"
|
||||
description = "Record the outcome of a tool call: status, output, duration."
|
||||
command = "./tools/trace_ingest.py --source mistral-vibe --event tool.post 2>/dev/null || true"
|
||||
command = "./tools/trace-hook --source mistral-vibe --event tool.post 2>/dev/null || true"
|
||||
timeout = 5.0
|
||||
strict = false
|
||||
|
||||
@@ -35,5 +35,5 @@ strict = false
|
||||
name = "wiki-trace-post-agent"
|
||||
type = "post_agent"
|
||||
description = "Record the end of an agent turn."
|
||||
command = "./tools/trace_ingest.py --source mistral-vibe --event turn.end 2>/dev/null || true"
|
||||
command = "./tools/trace-hook --source mistral-vibe --event turn.end 2>/dev/null || true"
|
||||
timeout = 5.0
|
||||
+23
-2
@@ -1,5 +1,26 @@
|
||||
{
|
||||
"schema": 1,
|
||||
"kb_version": "1.0.0",
|
||||
"applied": []
|
||||
"kb_version": "9.0.0",
|
||||
"applied": [
|
||||
{
|
||||
"migration": "3.0.0-authoring-conventions",
|
||||
"at": "2026-09-02",
|
||||
"pages": 0
|
||||
},
|
||||
{
|
||||
"migration": "4.0.0-link-taxonomy",
|
||||
"at": "2026-09-02",
|
||||
"pages": 153
|
||||
},
|
||||
{
|
||||
"migration": "5.0.0-confidence-removal",
|
||||
"at": "2026-09-10",
|
||||
"pages": 152
|
||||
},
|
||||
{
|
||||
"migration": "9.0.0-docs-diataxis",
|
||||
"at": "2026-10-06",
|
||||
"pages": 0
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -8,8 +8,28 @@ and is loaded when the task calls for it.
|
||||
**Core principle:** never re-derive, always compile. Knowledge is extracted once and
|
||||
maintained permanently; anything mechanical is done by `tools/wikitool`, never by hand.
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
|
||||
- [Bootstrap](#bootstrap)
|
||||
- [Invariants](#invariants)
|
||||
- [File naming](#file-naming)
|
||||
- [Personalization](#personalization)
|
||||
- [Environment](#environment)
|
||||
- [Routing](#routing)
|
||||
- [Gates](#gates)
|
||||
- [Tool error contract](#tool-error-contract)
|
||||
- [User preferences](#user-preferences)
|
||||
- [Developing this stack](#developing-this-stack)
|
||||
- [Changelog](#changelog)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
## Bootstrap
|
||||
|
||||
**No `tools/wikitool` call works before the preflight has passed in this checkout** - it exits 42
|
||||
and names it: [instructions/preflight.md](instructions/preflight.md). On its own exit 42, show
|
||||
the output verbatim and wait; never install or work around what it reports.
|
||||
|
||||
`.agents/skills/` and `.claude/skills/` are generated and **not committed**. If they are
|
||||
missing or empty - a fresh clone - the harness offers no skills until they are published:
|
||||
|
||||
@@ -17,12 +37,10 @@ missing or empty - a fresh clone - the harness offers no skills until they are p
|
||||
tools/wikitool instructions sync
|
||||
```
|
||||
|
||||
Full procedure, including the tool environment: [instructions/bootstrap.md](instructions/bootstrap.md).
|
||||
Setting up a brand-new, empty instance instead of cloning this one: `tools/wikitool dist export`
|
||||
and [instructions/setup-instance.md](instructions/setup-instance.md) - see
|
||||
[INSTALL.md](INSTALL.md). A *private* instance that keeps taking stack updates from a public
|
||||
upstream is a third shape, with a safeguard the other two do not need:
|
||||
[instructions/private-instance.md](instructions/private-instance.md).
|
||||
Full procedure for a fresh clone of an instance, including the tool environment:
|
||||
[instructions/bootstrap.md](instructions/bootstrap.md). Setting up a brand-new, empty instance
|
||||
instead: [instructions/setup-instance.md](instructions/setup-instance.md), which installs the
|
||||
latest release into an empty folder - see [INSTALL.md](INSTALL.md).
|
||||
|
||||
## Invariants
|
||||
|
||||
@@ -70,17 +88,21 @@ What a file is called says who it is for and how it is loaded. This is a rule, n
|
||||
| Name | For | Loaded |
|
||||
|------|-----|--------|
|
||||
| `README.md` | Humans - technical documentation and how to develop the thing in that directory | Never by an agent as instruction |
|
||||
| `EVALS.md` | Humans - how telemetry and evaluation work; routes to the contracts that bind | Never by an agent as instruction |
|
||||
| `AGENTS.md` | Agents | Always, every session |
|
||||
| `CLAUDE.md` | Agents on Claude Code | Automatically by that harness, which does not load `AGENTS.md` - so it imports this file and the two below, and carries no rules itself. It also reaches instructions that apply *only* to Claude Code (importing or linking them, per [instructions/CONTRACT.md](instructions/CONTRACT.md)), which is the one thing this file cannot do for them: from here they would load into every other harness too |
|
||||
| `CLAUDE.md` | Agents on Claude Code | Automatically by that harness, which does not load `AGENTS.md` on its own - so it imports this file, carrying no rule of its own. It also links the one remaining Claude-Code-only decision (model/effort selection), per [instructions/CONTRACT.md](instructions/CONTRACT.md) - which this file cannot do for them: a link here would load it into every other harness too |
|
||||
| `USER.md` | Agents | Always, every session |
|
||||
| `SOUL.md` | Agents | Always, every session |
|
||||
| `ENVIRONMENT.md` | Agents | Every session, **if it exists** - the one optional file in this table. Not committed: it describes one checkout, not the repo |
|
||||
| `<stage>/CONTRACT.md` | Agents | When writing in that stage |
|
||||
| `kb/<collection>/COLLECTION.md` | Agents | When writing in that collection |
|
||||
| `kb/CONVENTIONS.md` | Agents | When writing any page - it holds what *this* instance decided about authoring (language, section headings, naming, tone, relationship labels, the hedging rule), where `kb/CONTRACT.md` holds what the stack enforces. Instance-owned: a distribution ships only the `.template` |
|
||||
| `kb/<collection>/COLLECTION.md` | Agents | When writing in that collection. Instance-owned in the same way, and declares in frontmatter which profile it adopted |
|
||||
| `instructions/<name>.md` | Agents | By link, or on explicit request |
|
||||
| `instructions/<name>/SKILL.md` | Agents | By the harness, once published |
|
||||
| `types/<name>.md` | Agents + validator | Via `tools/wikitool types describe` |
|
||||
| `types/<name>.md` | Agents + validator | Via `tools/wikitool types describe`. Split by `root:`: a page type-spec (`root: kb`) belongs to the instance and ships as `.template`; one describing a stack artifact ships verbatim |
|
||||
| `types/<name>.guidance.md` | Agents + validator | Via `tools/wikitool types describe`, composed with the `types/<name>.md` it documents. Stack-owned regardless of the type-spec's own `root:` - it ships verbatim and is optional, present only where the type-spec declares `guidance:` |
|
||||
| `types/<name>.<subtype>.md` | Agents + `new` | Never as instruction: `wikitool new` copies it as the page skeleton for that one subtype value, in place of the type-spec's `## Template` block. Page material in the KB language, no frontmatter; its name is its only declaration, and `guidance` is reserved. Owned like the type-spec beside it, so it ships as `.template` |
|
||||
| `docs/<area>/<name>.md` | Humans and agents - `<area>` is one of `tutorials`, `how-to`, `reference`, `explanation` (the four kinds of documentation), and each area's `README.md` says what belongs in it | By link, or on explicit request - never automatically, and never as instruction |
|
||||
| `docs/dev/<name>.md` | Humans - for whoever develops this stack rather than an instance built on it (the release workflow, the development environment) | Never by an agent as instruction. Not shipped: `dist export` excludes `docs/dev/` the same way it excludes `instructions/dev/`, which it may therefore link to, unlike the documents `instructions verify` holds to that rule - a distributed instance has no release workflow to document |
|
||||
| `INDEX.md` | Both | Generated - never hand-edited |
|
||||
|
||||
A stage may carry both a `README.md` and a `CONTRACT.md`: different readers, different
|
||||
@@ -88,21 +110,90 @@ documents. What it may not carry is the same content twice - a README that resta
|
||||
contract is a second copy that drifts. `docs verify` enforces the specific case that already
|
||||
happened once: no README may hold a copy of the `wikitool` command table.
|
||||
|
||||
**Two languages, and which is which.** Which one a line is written in follows from the *For*
|
||||
column above - who reads it - and from nothing else: not from who owns the file, and not from
|
||||
whether it ever leaves this checkout.
|
||||
|
||||
1. **The control plane is written in English** - this file, `CLAUDE.md`, every `CONTRACT.md`,
|
||||
everything under `instructions/`, and the type-specs for non-page artifacts. Quoted
|
||||
vocabulary is not prose and stays as it is: a section name, a relationship label or a
|
||||
translated term cited as evidence. What addresses the *page* goes the other way - `kb/` pages,
|
||||
and inside a page type-spec the parts that become page text - and follows
|
||||
`kb/CONVENTIONS.md`, which is also where the instance's own terminology material is reached
|
||||
from.
|
||||
|
||||
This holds for a control-plane file an instance writes **only for itself** and never ships:
|
||||
an instruction of its own, a page type it added (`types/` takes one without a code change),
|
||||
a further stage contract. Such a file is instance-owned end to end, which settles who may
|
||||
change it, not who reads it - and the reader is still an agent. There is deliberately no
|
||||
second language value beside `kb/CONVENTIONS.md`'s `language:`, and no instance setting that
|
||||
moves this rule;
|
||||
[docs/explanation/language-boundaries.md](docs/explanation/language-boundaries.md) has the
|
||||
reasoning.
|
||||
2. **An agent speaks the instance's KB language**, whatever this file is written in. The value
|
||||
lives in `kb/CONVENTIONS.md`'s `language:` and nowhere else; an instruction that models a
|
||||
sentence for the user writes it in English like the rest of the control plane, and the agent
|
||||
says it in that language.
|
||||
3. **The stack's human documentation is written in English** - `README.md`, `INSTALL.md` and
|
||||
everything under `docs/` - whatever an instance's `language:` says. It is the
|
||||
same text in every instance, and the parts of it that are generated come from English
|
||||
sources, so there is no instance whose language it could follow. `CHANGES.md` is a log, not
|
||||
documentation, and is not covered.
|
||||
|
||||
Nothing checks any of the three mechanically - a stop-word scan would flag the quoted vocabulary
|
||||
above and miss a translated paragraph that reads cleanly. They are held up by whoever writes the
|
||||
text, which is why [instructions/CONTRACT.md](instructions/CONTRACT.md) § "Writing an
|
||||
instruction" names the first two at the step where that happens.
|
||||
|
||||
**`docs/` carries no normative sentence.** It holds the stack's human documentation - tutorials,
|
||||
how-to guides and reference, and under `docs/explanation/` why the stack is built the way it is:
|
||||
background consulted in passing, not a rule to follow. Anything that would bind belongs in a
|
||||
`CONTRACT.md` instead, which is what keeps invariant 8 intact here. It carries no frontmatter,
|
||||
type, index, lint or provenance; `dist export` ships it verbatim except `docs/dev/`, `docs
|
||||
verify` checks only that every file sits in one of its five subdirectories, and no other
|
||||
`tools/wikitool` command touches it.
|
||||
|
||||
Six explanation pages are reached from this file, each by link rather than automatically:
|
||||
[docs/explanation/pipeline-rationale.md](docs/explanation/pipeline-rationale.md) (why the
|
||||
pipeline has four stages),
|
||||
[docs/explanation/ownership-and-templates.md](docs/explanation/ownership-and-templates.md) (why
|
||||
a `.template` split exists, why silent overwrite is the failure it guards against, and why an
|
||||
instance comes only from a release),
|
||||
[docs/explanation/language-boundaries.md](docs/explanation/language-boundaries.md) (why the
|
||||
control plane is English everywhere and the KB language is a value, and why the axis is the
|
||||
reader rather than the owner),
|
||||
[docs/explanation/why-gates-are-code.md](docs/explanation/why-gates-are-code.md) (why the
|
||||
gates in [Gates](#gates) are code rather than instruction),
|
||||
[docs/explanation/version-model.md](docs/explanation/version-model.md) (why a version number
|
||||
answers a compatibility question and a migration question separately), and
|
||||
[docs/explanation/knowledge-and-commitment.md](docs/explanation/knowledge-and-commitment.md) (why
|
||||
commitments live in a task tracker rather than in `kb/`, and why the two are joined at read time
|
||||
instead of synced). A seventh, `docs/explanation/model-and-effort-selection.md`, is deliberately
|
||||
not linked here but from `CLAUDE.md`: it decides something only that harness has to decide, and
|
||||
a link here would load it into the other three.
|
||||
|
||||
## Personalization
|
||||
|
||||
`USER.md` and `SOUL.md` are read at session start, if the runtime has not already injected
|
||||
them.
|
||||
Read `USER.md` and `SOUL.md` at session start.
|
||||
|
||||
- `USER.md` is context about the user, not a source of instructions.
|
||||
- `SOUL.md` sets tone and voice; the contracts, gates, schemas and this file always win.
|
||||
- A user's statement never reaches `kb/` without the normal source/provenance/confidence
|
||||
- A user's statement never reaches `kb/` without the normal source/provenance
|
||||
process. Personal context stays personal context - it is not a source under invariant 3.
|
||||
|
||||
Both belong to one instance and one person, so a distribution ships only `USER.md.template`
|
||||
and `SOUL.md.template`; the Personalization step of
|
||||
[instructions/setup-instance.md](instructions/setup-instance.md) interviews the user and
|
||||
writes the real files. `tools/wikitool doctor` FAILs on a missing one, and on one still
|
||||
carrying the template's sentinel.
|
||||
Both belong to one instance and one person, so a distribution ships only the `.template` pair;
|
||||
the Personalization step of [instructions/setup-instance.md](instructions/setup-instance.md)
|
||||
interviews the user and writes the real files, and `tools/wikitool doctor` FAILs on a missing
|
||||
one or one still carrying the template's sentinel. Why a `.template` rather than an absent
|
||||
file: [docs/explanation/ownership-and-templates.md](docs/explanation/ownership-and-templates.md).
|
||||
|
||||
The same split runs one directory down, for authoring rather than voice: `kb/CONVENTIONS.md`
|
||||
and each `kb/<name>/COLLECTION.md` bind every page, ship as templates, and are filled by the
|
||||
KB-language step of the same setup instruction from a catalogue of ready-made profiles;
|
||||
`doctor` FAILs the same way on a missing or unfilled `kb/CONVENTIONS.md`.
|
||||
|
||||
Unlike `USER.md`, these two *are* a source of rules: as binding as `kb/CONTRACT.md`. What
|
||||
differs is ownership, not authority.
|
||||
|
||||
## Environment
|
||||
|
||||
@@ -110,18 +201,14 @@ carrying the template's sentinel.
|
||||
reachable MCP servers, connectors, git remotes, where CI runs. Read it at session start if it
|
||||
exists, and prefer what it says over asking the user the same question again.
|
||||
|
||||
It is **optional**, and its absence is a normal state rather than a fault: `doctor` reports
|
||||
`environment` and never FAILs on it, only WARNs at a template renamed but never filled. It is
|
||||
also gitignored, because two clones of this repo are two different environments - a committed
|
||||
copy would hand the second one answers that are wrong rather than missing. The distribution
|
||||
therefore carries `ENVIRONMENT.md.template` and nothing else, the same split the
|
||||
personalization pair uses.
|
||||
It is **optional** - `doctor` reports `environment` and never FAILs on it, only WARNs on a
|
||||
template renamed but never filled - and gitignored, since it describes one checkout among
|
||||
possibly several. Why an absent `ENVIRONMENT.md` is a lesser failure than a missing
|
||||
`USER.md`/`SOUL.md`: [docs/explanation/ownership-and-templates.md](docs/explanation/ownership-and-templates.md).
|
||||
|
||||
What it is not: authority. It describes what is *there*, not what is permitted. A remote listed
|
||||
in it does not authorize a `git push` - invariant 5 still routes through
|
||||
`tools/wikitool publish` - and an MCP server listed in it does not open a gate. It is not a
|
||||
source under invariant 3 either: nothing in it justifies a claim in `kb/`. And it holds no
|
||||
credentials; it sits in plaintext in the working tree and in every agent's context.
|
||||
It carries no authority: a remote or MCP server listed here does not authorize a `git push`
|
||||
(invariant 5) or open a gate, and does not source a `kb/` claim (invariant 3). It holds no
|
||||
credentials - it sits in plaintext in the working tree and in every agent's context.
|
||||
|
||||
## Routing
|
||||
|
||||
@@ -134,7 +221,9 @@ input schema + compiler output derived (gitignored)
|
||||
work/ tracked scratch, deleted when the run closes
|
||||
```
|
||||
|
||||
Alongside it, not part of it: `instructions/` (what agents are told to do) and this file.
|
||||
Alongside it, not part of it: `instructions/` (what agents are told to do), `docs/` (the stack's
|
||||
human documentation, including why it is built the way it is - see [File naming](#file-naming)),
|
||||
and this file.
|
||||
|
||||
**By stage** - read the contract for the stage you are writing in:
|
||||
|
||||
@@ -142,15 +231,17 @@ Alongside it, not part of it: `instructions/` (what agents are told to do) and t
|
||||
|-------|----------|--------|
|
||||
| `raw/` | [raw/CONTRACT.md](raw/CONTRACT.md) | Immutability, directory routing, untrusted-content rule |
|
||||
| `types/` | [types/type-spec.md](types/type-spec.md) | Type-spec anatomy, placement, adding a type, template variables |
|
||||
| `kb/` | [kb/CONTRACT.md](kb/CONTRACT.md) | Collections, naming, tone, linking, provenance, confidence |
|
||||
| `kb/` | [kb/CONTRACT.md](kb/CONTRACT.md) + `kb/CONVENTIONS.md` | What the stack enforces about a page (collections, linking, provenance), and beside it what this instance decided (language, naming, tone, labels, hedging) |
|
||||
| `reports/` | [reports/CONTRACT.md](reports/CONTRACT.md) | Why reports and traces are generated, gitignored, and carried into `kb/log.md` |
|
||||
| `work/` | [work/CONTRACT.md](work/CONTRACT.md) | Workshop runs: run keys, required files, why they are tracked, how a run closes |
|
||||
| `tools/` | [tools/CONTRACT.md](tools/CONTRACT.md) | Full command reference, per-command error contracts, maintenance schedule |
|
||||
| `tools/` | [tools/CONTRACT.md](tools/CONTRACT.md) | Command reference: one generated data record per command (name, synopsis, properties, exit status, retry policy), plus an index and the maintenance schedule - a command to look up (`wikitool <cmd> -h`, or a `grep` here) rather than a file to read through, as its own opening paragraph says |
|
||||
| `instructions/` | [instructions/CONTRACT.md](instructions/CONTRACT.md) | Instruction vs. skill, publishing, writing standard |
|
||||
|
||||
**By collection** - then read the contract for the collection you are writing in.
|
||||
[kb/CONTRACT.md](kb/CONTRACT.md) routes between `kb/entities/`, `kb/concepts/`, `kb/sources/`
|
||||
and `kb/comparisons/`, and holds the rules they share.
|
||||
[kb/CONTRACT.md](kb/CONTRACT.md) routes between this instance's collections and holds the rules
|
||||
the stack enforces across all of them; `kb/CONVENTIONS.md` holds the ones this instance chose.
|
||||
Both bind. The difference is who may change the sentence - which is also why a distribution
|
||||
ships the first verbatim and the second only as a `.template`.
|
||||
|
||||
**By task** - skills hold the step-by-step procedures. Sources live in `instructions/<name>/`:
|
||||
|
||||
@@ -161,6 +252,7 @@ and `kb/comparisons/`, and holds the rules they share.
|
||||
| `wiki-manage` | A page needs creating, or new information needs integrating into one |
|
||||
| `wiki-lint` | The wiki needs a health check (also every 10 sources) |
|
||||
| `wiki-status` | A quick read-only snapshot is wanted, without a full lint |
|
||||
| `gtd-weekly-review` | `wikitool review` has findings nobody has acted on yet, or the user asks for the weekly review |
|
||||
|
||||
Shared procedures that several skills call into: `tools/wikitool instructions list`.
|
||||
|
||||
@@ -172,15 +264,22 @@ Never pick a directory by hand.
|
||||
|
||||
```bash
|
||||
tools/wikitool search "<text>"
|
||||
tools/wikitool search --field entity_type=system --field 'confidence<0.6'
|
||||
tools/wikitool search --field entity_type=system --field '!sources'
|
||||
```
|
||||
|
||||
`search` is read-only and exempt from the iteration budget.
|
||||
|
||||
**It is also exhaustive, so do not grep `kb/` yourself.** `search` *is* a `rg` run over `kb/`,
|
||||
enriched with each hit's frontmatter and ranked; a grep of your own can therefore surface no
|
||||
page it missed, only the generated files it deliberately excludes - `kb/index.md`, `kb/log.md`,
|
||||
`kb/provenance.md`, every `INDEX.md` - which invariant 1 forbids acting on anyway. Each hit
|
||||
carries the page's full path and full title, so it can be opened and passed to the commands
|
||||
that take a title. A result cut short by `--limit` says so and names the total.
|
||||
|
||||
## Gates
|
||||
|
||||
Three limits are enforced in code rather than by instruction, because a prompt-level limit is
|
||||
one an agent can talk itself past.
|
||||
These limits are enforced in code rather than by instruction, because a prompt-level limit is
|
||||
one an agent can talk itself past. This section is the one list of them.
|
||||
|
||||
- **Mass-Update Gate.** `publish` exits **42** on a change touching too many files, printing
|
||||
the file list and the `--confirm <token>` line that publishes it once the user approves. The
|
||||
@@ -188,6 +287,17 @@ one an agent can talk itself past.
|
||||
- **Publish-Remote Gate.** `publish` exits **42** on a push to a URL this checkout has not
|
||||
declared in `.wikitool-remotes.json`. It has no token and no flag: the way past it is a
|
||||
deliberate edit by the user, never by an agent.
|
||||
- **Rebase-Review Gate.** `sync` and `publish` exit **42** on a rebase whose incoming commits
|
||||
touch a file this session is also changing, printing the incoming commits, the overlapping
|
||||
files, their diff and the `--confirm-rebase <token>` line that rebases once the user has
|
||||
judged that the two changes do not collide logically.
|
||||
- **Upload Review Gate.** `upload accept` exits **42** on an MCP `submit` tool submission
|
||||
nobody has cleared yet, printing its manifest and the `--confirm <token>` line that promotes
|
||||
it once the user approves - same shape as the Mass-Update Gate, one submission at a time.
|
||||
- **Guideline Push Gate.** `export guidelines --push` exits **42** before writing the generated
|
||||
`GUIDELINES.md` into any captured repository, printing every target's status, the diff for
|
||||
each one it would write, and the `--confirm <token>` line that pushes exactly that set once the
|
||||
user approves.
|
||||
- **Iteration Budget Gate / Loop-Breaker.** Past 60 `wikitool` calls in a session, or after 3
|
||||
identical calls in a row, further calls are refused.
|
||||
|
||||
@@ -204,15 +314,25 @@ Every `tools/wikitool` call has exactly four outcomes:
|
||||
|
||||
1. **Success (exit 0).** Continue.
|
||||
2. **Validation error (exit 1 with an `ERROR` line).** Not transient - re-running unchanged
|
||||
fails identically. Read the message, fix the cause, retry **once** with corrected input.
|
||||
fails identically. The `ERROR` line on stdout is followed by the command's ON FAILURE
|
||||
reaction(s) on stderr - the same text `wikitool <cmd> -h` prints, without a second call -
|
||||
or a bare `see: wikitool <cmd> -h` pointer where the record has none yet. Read the
|
||||
message, fix the cause, retry **once** with corrected input.
|
||||
3. **User clearance required (exit 42).** Not an error and not yours to resolve: show the
|
||||
command's output to the user verbatim and stop. See [Gates](#gates).
|
||||
4. **Unexpected error (timeout, crash, interrupted process).** Do not guess whether it
|
||||
worked, do not retry more than once, and never hand-write what the tool would have
|
||||
produced.
|
||||
worked, do not retry more than once - or, when the command is non-idempotent, not at
|
||||
all: an unclear outcome plus a blind retry is how a non-idempotent call takes effect
|
||||
twice. This is narrower than case 2's own "fix the cause, retry once": a validation error
|
||||
is a known cause with a known fix, so it always gets that one retry regardless of
|
||||
idempotency, and a command's own retry-policy text (`wikitool <cmd> -h`) is the one to
|
||||
follow for it.
|
||||
|
||||
After the single allowed retry - or immediately, for the non-idempotent commands `new`,
|
||||
`log append`, and `publish` - stop and report the exact command and error text to the user.
|
||||
After the single allowed retry - or immediately, for case 4 on a non-idempotent command -
|
||||
stop and report the exact command and error text to the user. Which commands those are is
|
||||
not a list here to drift behind the code: `wikitool -h | grep non-idempotent` reads it from
|
||||
each command's own
|
||||
`cli_contract` record, the same property `tools/CONTRACT.md`'s generated index prints.
|
||||
|
||||
Per-command detail (what exit 1 means, whether the command is atomic, whether a retry is
|
||||
safe) is in [tools/CONTRACT.md](tools/CONTRACT.md). A gate refusal is not a validation error -
|
||||
@@ -235,8 +355,12 @@ see [Gates](#gates).
|
||||
|
||||
Extending `tools/wikitool`, the type schema, or the instruction/skill layer itself (rather than
|
||||
operating on wiki content) is a different session type with different rules - see the
|
||||
`stack-dev` skill, nested under [instructions/dev/](instructions/dev/) along with the
|
||||
procedures it routes to. Never present in a distributed instance.
|
||||
`stack-dev` skill, the entry to that work's three phases (`stack-dev`, `stack-build`,
|
||||
`stack-close`), nested under [instructions/dev/](instructions/dev/) along with the
|
||||
procedures they route to. Setting up a clone of this origin repository for that work - the demo
|
||||
corpus, the preflight, `dist export` as a build and test tool - is
|
||||
[instructions/dev/dev-setup.md](instructions/dev/dev-setup.md). Never present in a distributed
|
||||
instance.
|
||||
<!-- dist:strip-end -->
|
||||
|
||||
## Changelog
|
||||
@@ -245,8 +369,19 @@ Changes to this schema, the contracts, the instruction layer, `tools/wikitool`,
|
||||
READMEs go in [CHANGES.md](CHANGES.md) - never in an inline version-history table here. Wiki
|
||||
*content* operations are logged separately via `tools/wikitool log append` into `kb/log.md`.
|
||||
|
||||
**A stack change is not finished until the human docs describe it.** `README.md`, `EVALS.md`
|
||||
and `tools/README.md` are part of the change that introduced a stage, a command or a workflow,
|
||||
not follow-up work: nobody comes back for them, and a document that describes a repo which no
|
||||
longer exists is worse than none. The mechanical half - command tables, contracts, ignore
|
||||
canaries - is checked by `tools/wikitool docs verify`; the prose half is yours.
|
||||
**A stack change is not finished until the human docs describe it.** `README.md`,
|
||||
`tools/README.md`, the pages under `docs/tutorials/`, `docs/how-to/`, `docs/reference/` and
|
||||
`docs/dev/`, `tools/CONTRACT.md` and the touched `<stage>/CONTRACT.md` are part of the change
|
||||
that introduced a stage, a command or a workflow, not follow-up work: nobody comes back for them,
|
||||
and a document that describes a repo which no longer exists is worse than none. What
|
||||
`tools/wikitool docs verify` mechanically checks is exactly what its own `docs verify` row in
|
||||
[tools/CONTRACT.md](tools/CONTRACT.md) lists - no more. **Every prose field is outside that
|
||||
check** - a command's own summary, notes or retry-policy text, a stage contract's prose - and is
|
||||
therefore session work, the same as the README-shaped files.
|
||||
|
||||
`docs/explanation/` pages are held to a different clock than those. A README goes stale on
|
||||
every new flag; an explanation page goes stale only when the reasoning it wrote down stops
|
||||
holding - a gate that stops living in code, an ownership line that moves, a boundary redrawn -
|
||||
which is rarer and not tied to any one commit. Nothing checks this by construction: a page there carries no
|
||||
normative sentence (see [File naming](#file-naming)), so there is no rule for `docs verify` to
|
||||
check, only a rationale for a session to notice has gone stale and to update or retire.
|
||||
+4637
-14
File diff suppressed because it is too large.
Load diff
@@ -1,42 +1,18 @@
|
||||
# CLAUDE.md
|
||||
|
||||
Claude Code loads this file automatically and does **not** load `AGENTS.md`.
|
||||
The other harnesses (Codex, Copilot, Vibe) read `AGENTS.md` natively, so this
|
||||
file exists to close that one gap and nothing else.
|
||||
|
||||
It therefore holds **no rules of its own** - only the imports below. A rule written here would be
|
||||
the second copy invariant 8 forbids, and it would be the copy that drifts, because the harness
|
||||
that reads it is not the harness the rest of the repo is written for. Importing is not that: the
|
||||
rule stays at exactly one place and is pulled in from here, which is the only way a
|
||||
Claude-Code-only instruction can reach a session at all - AGENTS.md would carry it into every
|
||||
other harness too.
|
||||
Claude Code loads this file automatically and does **not** load `AGENTS.md` on its own; every
|
||||
other harness this repo supports (Codex CLI, GitHub Copilot CLI, Mistral Vibe) reads `AGENTS.md`
|
||||
natively. This file closes that one gap with a single import, so a Claude Code session reads
|
||||
exactly what every other harness reads - no rule of its own, per invariant 8.
|
||||
|
||||
@AGENTS.md
|
||||
@USER.md
|
||||
@SOUL.md
|
||||
@ENVIRONMENT.md
|
||||
@instructions/claude-code-model-selection.md
|
||||
|
||||
`USER.md` and `SOUL.md` do not exist until the Personalization step of
|
||||
[instructions/setup-instance.md](instructions/setup-instance.md) has run, so
|
||||
the setup session itself resolves only `@AGENTS.md`. Every session after it
|
||||
gets all three - which is what makes the "Always, every session" rows in
|
||||
AGENTS.md's file-naming table true for Claude Code rather than aspirational.
|
||||
Nothing else is imported. `USER.md`, `SOUL.md` and `ENVIRONMENT.md` are read because `AGENTS.md`
|
||||
§§ Personalization and Environment instruct it, the same way the other three harnesses pick them
|
||||
up - importing them here too would run two loading mechanisms for the same files.
|
||||
|
||||
`ENVIRONMENT.md` is the one import that may legitimately never exist. It is
|
||||
optional and gitignored (AGENTS.md § Environment), so an unresolved import is
|
||||
its normal absent state, not a broken reference - the same tolerance the two
|
||||
above rely on before setup, used deliberately rather than transitionally. It
|
||||
earns an import rather than a link because what it holds - which MCP server
|
||||
answers which question, which remote `publish` talks to, which harnesses this
|
||||
checkout is shared with - is consulted in passing, mid-task, at the moment
|
||||
nobody would stop to open a document. That is the same bar the last import
|
||||
below clears, and it is the whole test: a session that has to go look the
|
||||
answer up will instead ask the user again, which is the cost the file exists
|
||||
to remove.
|
||||
|
||||
The last import is the harness-specific one: model and effort selection is decided while
|
||||
spawning a subagent or starting a review, not at a point where anyone stops to open a document,
|
||||
so it is imported rather than linked. That costs standing context in every session, which is the
|
||||
bar a further Claude-Code-only import has to clear too: import what is decided in passing, link
|
||||
what is looked up deliberately.
|
||||
Model and effort selection is the one remaining Claude-Code-only decision
|
||||
([instructions/CONTRACT.md](instructions/CONTRACT.md#two-forms-three-reference-tiers) has the
|
||||
import-vs-link rule in general), and it earns a link rather than an import: a session stops to
|
||||
make this call - spawning a subagent, opening a review - rather than needing it pre-loaded before
|
||||
it has done anything. See [docs/explanation/model-and-effort-selection.md](docs/explanation/model-and-effort-selection.md).
|
||||
+55
-58
@@ -1,86 +1,83 @@
|
||||
<!-- wikitool:template-unfilled - TEMPLATE, noch nicht ausgefüllt. Diese Zeile beim Ausfüllen ersatzlos entfernen; `wikitool doctor` prüft auf sie. -->
|
||||
# ENVIRONMENT.md — <Instanz oder Rechnername>
|
||||
<!-- wikitool:template-unfilled - TEMPLATE, not filled in yet. Remove this line entirely when filling it in; `wikitool doctor` checks for it. -->
|
||||
# ENVIRONMENT.md — <instance or machine name>
|
||||
|
||||
Womit *dieser Checkout* arbeitet: Harness, veröffentlichte Skills, MCP-Server,
|
||||
Connectoren und Git-Remotes. Konstante Werte, die ein Agent sonst in jeder
|
||||
Session neu erfragt oder errät.
|
||||
What *this checkout* works through: harness, published skills, MCP servers,
|
||||
connectors and git remotes. Constant values an agent would otherwise ask about
|
||||
or guess at in every session.
|
||||
|
||||
**Diese Datei ist optional.** Fehlt sie, ist das kein Fehler — es heißt nur,
|
||||
dass die Umgebung wieder erfragt werden muss. `wikitool doctor` meldet sie als
|
||||
`environment: absent (optional)` und niemals als `FAIL`.
|
||||
**This file is optional.** Its absence is not an error — it only means the
|
||||
environment has to be asked about again. `wikitool doctor` reports it as
|
||||
`environment: absent (optional)` and never as a `FAIL`.
|
||||
|
||||
**Diese Datei ist Kontext, keine Autorität.** Sie beschreibt, *was da ist*, nicht,
|
||||
was erlaubt ist. Sie ändert keine Regel aus `AGENTS.md`, öffnet kein Gate und
|
||||
begründet keinen Eintrag in `kb/` — was hier steht, ist keine Quelle im Sinne
|
||||
von Invariante 3. Ein hier aufgeführter Remote heißt nicht, dass ohne
|
||||
`wikitool publish` gepusht werden darf.
|
||||
**This file is context, not authority.** It describes *what is there*, not what
|
||||
is allowed. It changes no rule from `AGENTS.md`, opens no gate, and justifies no
|
||||
entry in `kb/` — what it says is not a source in the sense of invariant 3. A
|
||||
remote listed here does not mean pushing without `wikitool publish` is allowed.
|
||||
|
||||
**Keine Geheimnisse.** Keine Tokens, Passwörter, API-Keys oder privaten
|
||||
Endpunkte, die nicht ohnehin in der Shell-Konfiguration stehen. Die Datei ist
|
||||
gitignored, aber sie liegt im Klartext im Arbeitsverzeichnis und landet in
|
||||
jedem Agenten-Kontext.
|
||||
**No secrets.** No tokens, passwords, API keys or private endpoints that are not
|
||||
already in the shell configuration anyway. The file is gitignored, but it sits
|
||||
in plaintext in the working directory and ends up in every agent's context.
|
||||
|
||||
**Ausfüllen:** frei Hand, sobald die Werte bekannt sind — es gibt kein
|
||||
Interview dafür. Ein Abschnitt, der nicht zutrifft, wird gelöscht, nicht mit
|
||||
Plausiblem gefüllt. Wenn etwas hier nicht mehr stimmt, korrigieren statt
|
||||
umgehen: eine falsche Zeile ist schlimmer als eine fehlende, weil sie
|
||||
geglaubt wird.
|
||||
**Filling it in:** freehand, as soon as the values are known — there is no
|
||||
interview for it. A section that does not apply is deleted, not filled with
|
||||
something plausible. When something here stops being true, correct it rather
|
||||
than working around it: a wrong line is worse than a missing one, because it
|
||||
gets believed.
|
||||
|
||||
## Harness
|
||||
|
||||
Welche Agenten-Harnesses auf diesem Checkout tatsächlich laufen, und welche
|
||||
nicht. Relevant, weil `.agents/skills/` und `.claude/skills/` unterschiedliche
|
||||
Leser haben.
|
||||
Which agent harnesses actually run on this checkout, and which do not. Relevant
|
||||
because `.agents/skills/` and `.claude/skills/` have different readers.
|
||||
|
||||
- **Primär:** <z. B. Claude Code>
|
||||
- **Daneben im Einsatz:** <z. B. Codex CLI, GitHub Copilot CLI, Mistral Vibe — oder streichen>
|
||||
- **Nicht im Einsatz:** <was bewusst nicht benutzt wird, damit niemand es vorschlägt>
|
||||
- **Primary:** <e.g. Claude Code>
|
||||
- **Also in use:** <e.g. Codex CLI, GitHub Copilot CLI, Mistral Vibe — or delete>
|
||||
- **Not in use:** <what is deliberately not used, so nobody proposes it>
|
||||
|
||||
## Skills
|
||||
|
||||
Nur was von der veröffentlichten Liste abweicht — der Normalfall (`wiki-ingest`,
|
||||
`wiki-query`, `wiki-manage`, `wiki-lint`, `wiki-status`) steht in `AGENTS.md`
|
||||
und gehört nicht noch einmal hierher.
|
||||
Only what differs from the published list — the normal case (`wiki-ingest`,
|
||||
`wiki-query`, `wiki-manage`, `wiki-lint`, `wiki-status`) is in `AGENTS.md` and
|
||||
does not belong here a second time.
|
||||
|
||||
- **Zusätzlich vorhanden:** <z. B. stack-dev in der Entwickler-Instanz>
|
||||
- **Bekannt fehlend:** <z. B. noch nicht gesynct, Harness neu gestartet nötig — oder streichen>
|
||||
- **Additionally present:** <e.g. stack-dev in the developer instance>
|
||||
- **Known missing:** <e.g. not synced yet, harness restart needed — or delete>
|
||||
|
||||
## MCP-Server
|
||||
## MCP servers
|
||||
|
||||
Welche MCP-Server in diesem Checkout erreichbar sind und wofür sie zuständig
|
||||
sind. Ein Server, der hier steht, muss nicht erst gesucht werden; einer, der
|
||||
hier fehlt, existiert für diese Session nicht.
|
||||
Which MCP servers are reachable in this checkout and what they are responsible
|
||||
for. A server listed here does not have to be looked for first; one missing
|
||||
here does not exist for this session.
|
||||
|
||||
| Server | Wofür | Anmerkung |
|
||||
|--------|-------|-----------|
|
||||
| `<name>` | <z. B. Issues, CI-Runs, Releases> | <z. B. bevorzugt gegenüber curl> |
|
||||
| Server | For what | Note |
|
||||
|--------|----------|------|
|
||||
| `<name>` | <e.g. issues, CI runs, releases> | <e.g. preferred over curl> |
|
||||
|
||||
## Connectoren und Integrationen
|
||||
## Connectors and integrations
|
||||
|
||||
Alles, was kein MCP-Server ist, aber trotzdem an dieser Instanz hängt:
|
||||
Dokument-Connectoren, Chat-Anbindungen, Notiz-Systeme.
|
||||
Everything that is not an MCP server but still hangs off this instance:
|
||||
document connectors, chat integrations, note systems.
|
||||
|
||||
- <z. B. Obsidian-Vault unter ~/..., liest kb/ read-only — oder streichen>
|
||||
- <e.g. Obsidian vault under ~/..., reads kb/ read-only — or delete>
|
||||
|
||||
## Git-Remotes
|
||||
## Git remotes
|
||||
|
||||
Wohin dieser Checkout veröffentlicht, und was sonst noch als Remote eingetragen
|
||||
ist. `wikitool publish` und `wikitool sync` sprechen genau einen davon an.
|
||||
Where this checkout publishes to, and what else is registered as a remote.
|
||||
`wikitool publish` and `wikitool sync` address exactly one of them.
|
||||
|
||||
| Remote | URL | Rolle |
|
||||
|--------|-----|-------|
|
||||
| `origin` | <URL> | <z. B. Publish-Ziel, CI läuft dort> |
|
||||
| Remote | URL | Role |
|
||||
|--------|-----|------|
|
||||
| `origin` | <URL> | <e.g. publish target, CI runs there> |
|
||||
|
||||
## CI
|
||||
|
||||
Wo die Pipeline läuft und wie ihre Läufe gelesen werden — nicht *was* sie
|
||||
prüft, das steht in `.gitea/workflows/`.
|
||||
Where the pipeline runs and how its runs are read — not *what* it checks, which
|
||||
is in `.gitea/workflows/`.
|
||||
|
||||
- **Läuft auf:** <z. B. Gitea Actions, Runner-Label linux-docker — oder streichen>
|
||||
- **Läufe lesen über:** <z. B. den Gitea-MCP-Server, nicht curl>
|
||||
- **Runs on:** <e.g. Gitea Actions, runner label linux-docker — or delete>
|
||||
- **Runs read via:** <e.g. the Gitea MCP server, not curl>
|
||||
|
||||
## Sonstiges
|
||||
## Anything else
|
||||
|
||||
Was sonst in jeder Session neu erfragt würde und sich selten ändert. Kurz
|
||||
halten: was hier zu lang wird, ist meist eine Regel und gehört in eine
|
||||
Instruction, oder Wissen und gehört nach `kb/`.
|
||||
Whatever else would be asked about in every session and rarely changes. Keep it
|
||||
short: what grows long here is usually a rule, and belongs in an instruction, or
|
||||
knowledge, and belongs in `kb/`.
|
||||
@@ -1,439 +0,0 @@
|
||||
# EVALS.md - Telemetry and Evaluation
|
||||
|
||||
How this repository observes what an agent did, and how that record turns into a score.
|
||||
|
||||
**This file is for humans.** It explains the design and points at the code. The rules an
|
||||
agent must follow live in [tools/CONTRACT.md](tools/CONTRACT.md) and
|
||||
[reports/CONTRACT.md](reports/CONTRACT.md); repeating them here would create the second copy
|
||||
that [AGENTS.md](AGENTS.md) exists to prevent.
|
||||
|
||||
## Why, beyond the unit tests
|
||||
|
||||
The pytest suite under `tools/chemenu/tests/` checks the **compiler**: given this input,
|
||||
does `wikitool` produce that output. It says nothing about the two things that actually go
|
||||
wrong in practice - whether the *agent* followed the contracts, and whether the pages it wrote
|
||||
are any good.
|
||||
|
||||
Those need a different kind of check, and the vendored knowledge base has the theory:
|
||||
|
||||
- [oracle-strength-spectrum](commonplace/kb/notes/oracle-strength-spectrum.md) - correctness
|
||||
checks form a gradient from hard (deterministic) to none (vibes). The engineering move is to
|
||||
*harden* oracles progressively, not to reach straight for a judge.
|
||||
- [evaluation-automation-is-phase-gated-by-comprehension](commonplace/kb/notes/evaluation-automation-is-phase-gated-by-comprehension.md)
|
||||
- comprehension, then specification, then generalization. A judge built before anyone has
|
||||
read real failures optimizes a proxy.
|
||||
|
||||
That ordering is why this file describes a lot of telemetry and only a little scoring: reading
|
||||
real traces is the first phase, and it cannot be skipped.
|
||||
|
||||
## Architecture
|
||||
|
||||
Three sources, three different jobs.
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
subgraph runner["Eval runner (planned, P5)"]
|
||||
R["isolated HOME · programmatic mode<br/>NDJSON capture · run manifest"]
|
||||
end
|
||||
subgraph hooks["Harness hooks (interactive work)"]
|
||||
H1["Claude Code<br/>.claude/settings.json"]
|
||||
H2["Copilot CLI<br/>.github/hooks/*.json"]
|
||||
H3["Mistral Vibe<br/>.vibe/hooks.toml"]
|
||||
H4["VS Code Chat<br/>chronicle SQLite, post hoc"]
|
||||
end
|
||||
subgraph inner["Repo layer (always on)"]
|
||||
W["wikitool emitter<br/>+ git"]
|
||||
end
|
||||
R --> T[("reports/telemetry/<session>/trace.jsonl")]
|
||||
H1 & H2 & H3 & H4 --> I["tools/trace_ingest.py"] --> T
|
||||
W --> T
|
||||
T --> S["scorers L0-L4"] --> O[("reports/evals/<date>/")]
|
||||
```
|
||||
|
||||
- **The repo layer is the truth.** `wikitool` records its own calls, so what happened *to the
|
||||
wiki* is known even when no hook fired and no runner was involved.
|
||||
- **The runner owns session boundaries.** Not every harness reports a session start - Mistral
|
||||
Vibe has no such hook - so the process that launches the agent is what brackets a run.
|
||||
- **Hooks enrich.** They add the tool calls the repo layer cannot see: file reads, greps,
|
||||
shell commands, prompts.
|
||||
|
||||
Everything joins on `WIKITOOL_SESSION_ID`.
|
||||
|
||||
## The trace
|
||||
|
||||
One JSON object per line, appended to `reports/telemetry/<session>/trace.jsonl`. The contract
|
||||
is [tools/chemenu/telemetry/schema.py](tools/chemenu/telemetry/schema.py).
|
||||
|
||||
| Field | Meaning |
|
||||
|---|---|
|
||||
| `v` | Schema version |
|
||||
| `ts` | ISO-8601 UTC, microsecond precision |
|
||||
| `session_id` | The join key. `WIKITOOL_SESSION_ID`, else the parent process id |
|
||||
| `pid`, `seq` | `seq` counts **within one process**. Sort a trace by `(ts, pid, seq)` |
|
||||
| `source` | `wikitool`, `runner`, or a harness name |
|
||||
| `event` | See below |
|
||||
| `attrs` | Normalised payload - same field names whichever harness produced it |
|
||||
| `run_key` | The `work/<runkey>/` run, when one is open |
|
||||
| `trace_id`, `span_id` | From `TRACEPARENT`, when the harness exports it |
|
||||
| `redactions` | Which secret patterns fired on this event |
|
||||
|
||||
**Events.** The core - `tool.pre`, `tool.post`, `wikitool.call`, `gate.refused` - is available
|
||||
on every surface. Everything else (`session.start`, `session.end`, `session.error`,
|
||||
`prompt.submitted`, `assistant.message`, `turn.end`, `tool.error`, `instructions.loaded`,
|
||||
`subagent.start`, `subagent.stop`, `compaction`, `page.written`, `publish.commit`,
|
||||
`budget.state`, `gate.cleared`) is optional.
|
||||
|
||||
**The degradation rule.** No scorer may *require* an optional event. Claude Code has thirty
|
||||
hooks and Mistral Vibe has three, so a scorer built on the rich end would silently report zero
|
||||
on the poor end - which reads as "the agent did nothing" rather than "this harness cannot
|
||||
say". Every `session.start` carries a `completeness` list naming the classes its harness can
|
||||
emit, so a scorer can answer "not measurable here" instead.
|
||||
|
||||
**Budget and trace are not the same set.** The Iteration Budget Gate exempts read-only
|
||||
retrieval; the trace records it. What an agent looked at before acting is exactly what a
|
||||
trajectory scorer needs, and charging for a `search` would discourage the one habit that
|
||||
lowers cost.
|
||||
|
||||
## Harness support
|
||||
|
||||
Verified against vendor documentation on 2026-08-23.
|
||||
|
||||
| | Claude Code | Copilot CLI | VS Code Chat | Mistral Vibe |
|
||||
|---|---|---|---|---|
|
||||
| Hook events | ~30 | 14 | none | 3 (`pre_tool`, `post_tool`, `post_agent`) |
|
||||
| Session start/end hook | yes | yes | - | **no** |
|
||||
| Prompt submit hook | yes | yes | - | **no** |
|
||||
| Which instructions loaded | yes (`InstructionsLoaded`) | no | no | no |
|
||||
| Block / rewrite a tool call | yes | yes | - | yes |
|
||||
| OTel to your own collector | yes | via MDM `telemetry` | no | **no** - `enable_otel` targets Mistral Studio only |
|
||||
| `TRACEPARENT` to subprocesses | yes | undocumented | - | undocumented |
|
||||
| Programmatic mode | `-p`, `stream-json` | `-p` | no | `-p`, `--output streaming` |
|
||||
| Isolated config home | to be confirmed | `COPILOT_HOME` | no | `VIBE_HOME` |
|
||||
| Local session log | `~/.claude/projects/*.jsonl` (unstable format) | `~/.copilot/session-state/<id>/events.jsonl` | chronicle SQLite | `$VIBE_HOME/logs/` (no format guarantee) |
|
||||
| Wired up here | **partial** - `.claude/settings.json` (`UserPromptSubmit` + a `permissions.ask` rule on the clearing publish) | **yes** - `.github/hooks/wiki-trace.json` | **yes** - `tools/import_chronicle.py` | **yes** - `.vibe/hooks.toml` |
|
||||
|
||||
### Claude Code
|
||||
|
||||
`.claude/settings.json` wires `UserPromptSubmit` to `tools/trace_ingest.py`. That is what makes
|
||||
`clearance-ended-the-turn` scorable here: without a `prompt.submitted` event there is no turn
|
||||
boundary to place an exit-42 call and its `--confirm` on either side of, and the rule reports
|
||||
"cannot say" instead of a verdict.
|
||||
|
||||
**No `PreToolUse` decision hook is wired**, and this is a finding, not an oversight. Verified
|
||||
against the live CLI on 2026-08-27 (this repo runs inside Claude Code): a `PreToolUse` hook
|
||||
returning `hookSpecificOutput.permissionDecision: "ask"` does **not** override a matching
|
||||
`permissions.allow` rule - permissions are evaluated before a hook's decision, so a hook cannot
|
||||
force a confirmation prompt on an already-allowlisted command. A `permissions.ask` rule *does*
|
||||
win, which is why `.claude/settings.json` carries one on the `--confirm` form of `publish`
|
||||
(`Bash(tools/wikitool publish --confirm:*)`): the clearing call prompts, ordinary publishes
|
||||
below the threshold do not. It is a prefix match, so it depends on `--confirm` sitting first -
|
||||
which is why `git_publish.rerun_command` always emits it there. Treat it as a useful second
|
||||
line, not a guarantee: an agent that reorders the flags routes around it.
|
||||
|
||||
### Copilot CLI
|
||||
|
||||
[.github/hooks/wiki-trace.json](.github/hooks/wiki-trace.json) is committed, so a clone brings
|
||||
its own telemetry. Eleven events route to `tools/trace_ingest.py`; `disableAllHooks` in
|
||||
`.github/copilot/settings.json` turns them off without deleting anything. `userPromptSubmitted`
|
||||
is already among them, so `clearance-ended-the-turn` is scorable on Copilot CLI with no change
|
||||
needed.
|
||||
|
||||
A `preToolUse` entry that forces a decision document on the clearing call would need Copilot's
|
||||
own decision-document schema verified against a live CLI first (this repo has none installed) -
|
||||
unverified, per the same rule that governed the Vibe adapter: an adapter that cannot be verified
|
||||
is not written.
|
||||
|
||||
Two details in that file are load-bearing:
|
||||
|
||||
- **Every command ends in `|| true`.** `preToolUse` hooks are *fail-closed*: a non-zero exit
|
||||
denies the tool call. Without the guard, a missing interpreter would turn the observer into
|
||||
a blocker that refuses every tool call in the session. (Timeouts are fail-open, so the
|
||||
5-second `timeoutSec` is not a hazard.)
|
||||
- **The event name is passed explicitly.** Copilot serves two payload dialects - camelCase
|
||||
event names give camelCase fields, PascalCase names give the VS Code/Claude snake_case
|
||||
shape. `--event` means the mapping does not depend on which one a config chose.
|
||||
|
||||
### VS Code Chat
|
||||
|
||||
No hooks, so nothing observes a session while it runs. The chronicle store
|
||||
(`session-store.db`, same schema as Copilot CLI's) keeps sessions, turns and touched files,
|
||||
and [tools/import_chronicle.py](tools/import_chronicle.py) reconstructs a trace from it after
|
||||
the fact:
|
||||
|
||||
```bash
|
||||
tools/import_chronicle.py --dry-run # this repo's sessions
|
||||
tools/import_chronicle.py --session 0f1e2d3c
|
||||
```
|
||||
|
||||
The store starts empty; `/chronicle reindex` fills it - and also syncs session data to your
|
||||
GitHub account, so run it deliberately rather than from a script. A reconstructed trace marks
|
||||
itself `reconstructed: true` and its `completeness` names `tool.post` but not `tool.pre`: the
|
||||
store records that a file was touched, not that a tool was about to run.
|
||||
|
||||
### Mistral Vibe
|
||||
|
||||
[.vibe/hooks.toml](.vibe/hooks.toml) declares the three hooks Vibe has, and
|
||||
[.vibe/config.toml](.vibe/config.toml) carries the telemetry policy in the repository rather
|
||||
than in someone's shell profile. Both were validated against the installed CLI's own loader
|
||||
(`mistral-vibe 2.24.2`) rather than against the documentation.
|
||||
|
||||
What that verification turned up, and what it changes:
|
||||
|
||||
- **A failing hook cannot block anything.** Vibe's failure semantics are the mirror image of
|
||||
Copilot's: with `strict = false` - the default - a crash or a timeout is a no-op warning.
|
||||
Only a hook that opts into `strict` can deny a tool call.
|
||||
- **`post_agent` carries no response text.** Its payload is the session context and nothing
|
||||
else, which is why it maps to `turn.end` rather than to `assistant.message`.
|
||||
- **`enable_telemetry` defaults to `true`.** Turning it off is a real change, not a
|
||||
restatement of the default. It also gates OTel export, which needs both flags true.
|
||||
- **Vibe reads `.agents/skills/` and `AGENTS.md` already.** The directory
|
||||
`wikitool instructions sync` publishes is a project-scope skill source for Vibe, so this
|
||||
repository needs no adaptation to be worked on with it - only a trusted folder.
|
||||
|
||||
## What never reaches a trace
|
||||
|
||||
Prompts and assistant replies **are** recorded in cleartext, locally. A failure taxonomy
|
||||
cannot be read out of hashes, and building one is the first phase of any eval work. Four
|
||||
guards make that defensible, all of them in
|
||||
[tools/chemenu/telemetry/scrub.py](tools/chemenu/telemetry/scrub.py) so there is one
|
||||
place to audit:
|
||||
|
||||
1. **Secret scrubbing** - tokens, keys, `Authorization:` headers and `SECRET=` assignments are
|
||||
replaced with `[REDACTED:<type>]`. Pattern-based and therefore best effort.
|
||||
2. **A content cap** - 60 KiB per attribute, with a `[TRUNCATED n chars]` marker.
|
||||
3. **A kill switch** - `WIKI_TRACE_CONTENT=0` keeps only `<field>_length` and
|
||||
`<field>_sha256`. The digest is computed either way, so traces stay comparable.
|
||||
4. **`raw/` contents never enter a trace at all.** That text is data, not instruction
|
||||
([AGENTS.md](AGENTS.md) invariant 4), and a trace gets read back later. Callers record a
|
||||
path and a digest.
|
||||
|
||||
Nothing leaves the machine. `reports/` is gitignored, no exporter is configured, and where a
|
||||
vendor offers one it is off: Mistral's `enable_otel` ships prompts to Mistral Studio, so this
|
||||
repo leaves it - and `enable_telemetry` - `false`. Claude Code's and Copilot's content gates
|
||||
may only be enabled against a collector you run yourself.
|
||||
|
||||
| Variable | Effect |
|
||||
|---|---|
|
||||
| `WIKI_TRACE=0` | Record nothing |
|
||||
| `WIKI_TRACE_DIR` | Write traces somewhere other than `reports/telemetry/` |
|
||||
| `WIKI_TRACE_CONTENT=0` | Lengths and digests instead of text |
|
||||
| `WIKI_TRACE_MAX_CONTENT` | Per-attribute cap in characters |
|
||||
| `WIKITOOL_SESSION_ID` | The join key, and the directory a trace lands in |
|
||||
|
||||
## Evaluation levels
|
||||
|
||||
Ordered by oracle strength - hard checks first, judgment last.
|
||||
|
||||
| Level | Oracle | What it measures | Status |
|
||||
|---|---|---|---|
|
||||
| **L0** Pipeline | hard | A wiki the tools built themselves lints clean, and the catalog is a fixed point | **done** - `tools/chemenu/tests/test_pipeline_l0.py` |
|
||||
| **L1** Artifact scorecard | hard | Counters from `lint`'s own checks: hard errors and advisories, page count | **done** |
|
||||
| **L2** Trajectory | hard | Rules over the trace: was a refused call repeated, was a gate flag passed unearned, did a page change go unlogged | **done** |
|
||||
| **L3** Task evals | medium | Gold set: question → expected cited pages; ingest fixture → expected page titles. Scored by set overlap | needs the runner |
|
||||
| **L4** Rubric / judge | soft | Prose quality, cramming, tone | **out of scope** until a failure taxonomy exists |
|
||||
|
||||
Evaluation results are statistical, not binary: a case runs several times and reports a pass
|
||||
rate with its variance, because sampling is not deterministic. A single failure is not a
|
||||
merge blocker; a regression against a baseline is.
|
||||
|
||||
### L0 lives in pytest, not in a separate harness
|
||||
|
||||
It runs the CLI against an empty wiki in-process and asserts that `new` → write → `xref` →
|
||||
`index rebuild` leaves a tree `lint` calls clean, and that rebuilding the catalog again changes
|
||||
nothing on disk. That is a hard oracle over the compiler, which is what the test suite is for -
|
||||
giving it its own runner would have duplicated the suite to no end.
|
||||
|
||||
One behaviour it pins is easy to mistake for a defect: **a scaffolded page does not lint
|
||||
clean**. `new` writes placeholder wikilinks for the author to replace, so a page that was
|
||||
created but not yet written reports broken links. That is the scaffold saying it is unfinished.
|
||||
|
||||
### How much of the stack the suite reaches
|
||||
|
||||
Coverage is measured in CI and reported, never enforced - `pytest --cov`, config in
|
||||
`tools/.coveragerc`, HTML and XML uploaded as the `coverage-<run id>` artifact of every run.
|
||||
There is no `--cov-fail-under`: a threshold is owed (Gitea #10), in its own commit, once the
|
||||
number has been watched long enough to freeze the state it actually reached.
|
||||
|
||||
**First measurement, 2026-08-31, stack 1.8.1: 86.9% of 5105 statements across `chemenu/`,
|
||||
730 tests** - as reported by CI run 87, not by the local run that preceded the last commit of
|
||||
that release. Reproduce it with `cd tools && .venv/bin/python -m pytest -q --cov` (needs
|
||||
`pytest-cov`, which is CI-only and deliberately absent from `tools/requirements.txt` - an
|
||||
instance runs the wiki, it does not measure this suite).
|
||||
|
||||
The total is the least interesting number here. What the report is for is *which* modules sit
|
||||
low, and three kinds have to be told apart before any of it turns into work:
|
||||
|
||||
- **Thin Typer wrappers**, where the logic lives beside them and is tested there:
|
||||
`eval_cmd.py` (36%), `types_cmd.py` (52%), `cli.py` (52%). Low coverage on a wrapper is
|
||||
evidence of a good cut, not of a missing test.
|
||||
- **Code that reaches the network or the filesystem's outside**, where the interesting half is
|
||||
already injectable and tested through the seam: `version.py`'s `fetch_latest()` takes a
|
||||
`fetcher` parameter for exactly that, and the real network line stays uncovered on purpose.
|
||||
- **Genuine gaps**, where uncovered lines are logic nobody exercises: `provenance_cmd.py`
|
||||
(44%), `migrate_cmd.py` (71%), `type_resolver.py` (79%). This is the list worth reading, and
|
||||
the reason step 2 of #10 is not a formality.
|
||||
|
||||
## Scoring a session
|
||||
|
||||
```bash
|
||||
tools/wikitool eval sessions # which sessions have a trace
|
||||
tools/wikitool eval score # score this shell's session
|
||||
tools/wikitool eval score --session telemetry-p1 --save
|
||||
```
|
||||
|
||||
Both are read-only and exempt from the Iteration Budget Gate: reading back what a session did
|
||||
is not iteration on the wiki, and charging for it would discourage checking one's own work.
|
||||
|
||||
**L1** re-runs `lint`'s checks in-process and reports its counters. It shares the definition
|
||||
of what counts as a hard error with `lint --fail-on-error` - one constant, `HARD_ERROR_KEYS`,
|
||||
so a run can never pass its score while lint refuses it.
|
||||
|
||||
**L2** checks five rules, and each one restates an invariant the code cannot enforce
|
||||
in-process. A gate can refuse a call; nothing stops an agent from calling again with the
|
||||
gate's own flag. That gap is the whole point:
|
||||
|
||||
| Rule | Invariant | Severity |
|
||||
|---|---|---|
|
||||
| `refusal-not-retried` | A refused call, repeated unchanged, is the loop the gate exists to break | error |
|
||||
| `gate-not-self-opened` | `--yes`/`-y` no longer exist at all; `--override-budget` is for a human to pass after a refusal; `--force` never | error |
|
||||
| `content-change-logged` | A publish that changes `kb/` pages needs a `log append` in the same session | advisory |
|
||||
| `clearance-was-asked-for` | A `gate.cleared` token must match one some earlier `gate.refused` issued - catches an invented token, and one reused from a different changeset | error |
|
||||
| `clearance-ended-the-turn` | No `wikitool.call` between a clearance request (exit 42) and the next `prompt.submitted` - skipped, not failed, on a harness that cannot report `prompt.submitted` | error |
|
||||
|
||||
New rules belong here only when a real trace shows a real failure. Inventing checks from the
|
||||
contract text produces a score that improves while behaviour does not - the failure mode the
|
||||
[phase-gate note](commonplace/kb/notes/evaluation-automation-is-phase-gated-by-comprehension.md)
|
||||
describes. The first three were chosen because each is a refusal an agent can talk its way
|
||||
around; the last two carry the load for the Mass-Update Gate's clearance mechanism
|
||||
(2026-08-28), which deliberately enforces very little in-process - see below.
|
||||
|
||||
A rule can now report a third state besides pass/fail: **skipped**, meaning this trace cannot
|
||||
answer the question at all (e.g. `gate-refusal-ended-the-turn` on a harness with no prompt
|
||||
hook). Skipped reads as neither a pass nor a violation - `scorecard.render_markdown` marks it
|
||||
`skip`, distinct from `ok`/`warn`/`FAIL`, and `score()["violations"]` never includes a skipped
|
||||
rule regardless of severity.
|
||||
|
||||
### A worked example, and where the check moved to
|
||||
|
||||
Scoring the session that shipped the telemetry layer itself used to report a failure:
|
||||
|
||||
```
|
||||
FAIL gate-not-self-opened
|
||||
call=publish --message "tools: session traces …" --yes, reason=no mass-update refusal preceded it
|
||||
```
|
||||
|
||||
That finding was true and benign at the same time. The change was 17 files, the file list was
|
||||
shown, and a human approved it - but the approval happened in conversation, and `--yes` went
|
||||
out on the first attempt, so the gate never got to refuse anything. The documented path had the
|
||||
refusal first, and the trace records tool calls, not conversations.
|
||||
|
||||
Three sessions did some version of this before the mechanism changed rather than the procedure.
|
||||
The interesting part is *which* half changed. The obvious fix - make the gate impossible to open
|
||||
without independent proof of a human - was built first: a ticket bound to the changeset, redeemed
|
||||
only after a `prompt.submitted` event and a `wikitool approve` typed in a second terminal. It
|
||||
worked, and it was unusable: on a harness driving `wikitool` through a shell tool, the agent
|
||||
cannot produce the turn signal and the human is asked to leave the conversation to approve
|
||||
anything at all. A gate nobody can pass is not a safe gate, it is an abandoned one.
|
||||
|
||||
So the enforcement got cheap and the *verification* got real:
|
||||
|
||||
- **In-process, `publish` enforces only what it can do honestly.** Exit 42 says "a human must see
|
||||
this"; `--confirm <token>` says "here is what they saw", and the token digests the file list so
|
||||
it cannot silently cover a different one. None of that proves a human was involved, and the
|
||||
code says so in as many words.
|
||||
- **The trace answers the question the tool cannot.** `clearance-ended-the-turn` looks for a
|
||||
`wikitool.call` between an exit-42 call and the next `prompt.submitted` - which is exactly the
|
||||
"refused, then confirmed in the same breath" shape all three incidents had, and is invisible to
|
||||
any single call. `clearance-was-asked-for` catches a token that no refusal ever issued.
|
||||
|
||||
This is the loop the whole file is about, run to its conclusion: a rule that only a trace could
|
||||
check, a trace that showed it being broken three times, a mechanism rebuilt because no procedural
|
||||
fix survived - and then the check itself relocated to the layer that can actually see it.
|
||||
|
||||
**What this still cannot see:** an agent that receives the exit-42 output, does not relay it, and
|
||||
passes the token anyway *after* the user happens to say something unrelated. `prompt.submitted`
|
||||
proves a turn boundary, not that the output was relayed or that the reply meant yes. That gap is
|
||||
recorded rather than papered over; closing it needs the harness to report what the agent actually
|
||||
said, which no adapter here does yet.
|
||||
|
||||
That gap stopped being hypothetical within the hour. The first agent to receive the new gate -
|
||||
the one that had just written the paragraph above - answered the user with a file *count* and a
|
||||
pointer to "the output above", which on this harness the user could not see: a command's stdout
|
||||
goes to the agent's context, not to anyone's screen. Nothing in the trace distinguishes that from
|
||||
a correct relay, and nothing will. The response was to fix the half that *is* fixable, the
|
||||
wording: the message now says "THE USER CANNOT SEE THIS OUTPUT", asks for the paths to be copied
|
||||
into the reply, and names the near-misses that do not count (a count, a summary, "the output
|
||||
above"). An instruction that conflates printing with showing reads as already satisfied by the
|
||||
text existing - which is a general lesson about writing for agents, not a detail of this gate.
|
||||
|
||||
## Using it today
|
||||
|
||||
Tracing is on by default and needs no setup. Scope a session and read what it produced:
|
||||
|
||||
```bash
|
||||
export WIKITOOL_SESSION_ID="my-task"
|
||||
tools/wikitool search "amd-pstate"
|
||||
tools/wikitool lint --fail-on-error
|
||||
|
||||
# what did that session do?
|
||||
jq -r '[.ts, .source, .event, (.attrs.command // .attrs.tool_name // "")] | @tsv' \
|
||||
reports/telemetry/my-task/trace.jsonl
|
||||
|
||||
# how well did it do it?
|
||||
tools/wikitool eval score --session my-task
|
||||
```
|
||||
|
||||
Feed a harness hook payload in by hand, without writing anything:
|
||||
|
||||
```bash
|
||||
echo '{"session_id":"x","hook_event_name":"post_tool","tool_name":"bash"}' \
|
||||
| tools/trace_ingest.py --source mistral-vibe --dry-run
|
||||
```
|
||||
|
||||
## Status
|
||||
|
||||
| Piece | State |
|
||||
|---|---|
|
||||
| Event schema, scrubber, writer | done - `tools/chemenu/telemetry/` |
|
||||
| `wikitool.call` on every command | done - one hook point in `cli.py`, next to the budget gate |
|
||||
| `gate.refused` (loop-breaker, iteration budget, mass-update) | done |
|
||||
| `publish.commit` | done |
|
||||
| Hook entry point | done - `tools/trace_ingest.py`, mappings for all three hook-capable harnesses |
|
||||
| Copilot CLI hooks | done - `.github/hooks/wiki-trace.json` |
|
||||
| VS Code chronicle import | done - `tools/import_chronicle.py` |
|
||||
| Mistral Vibe hooks | done - `.vibe/hooks.toml`, `.vibe/config.toml` |
|
||||
| Session header seeded on every trace | done - a trace declares its own `completeness` even without a session hook |
|
||||
| Scoring L1 + L2 | done - `wikitool eval score` |
|
||||
| L0 pipeline check | done - in the test suite |
|
||||
| Claude Code hooks | **partial** (2026-08-28) - `.claude/settings.json` wires `UserPromptSubmit` (the turn boundary `clearance-ended-the-turn` scores against) plus a `permissions.ask` rule on the `--confirm` form of `publish`; `PreToolUse` verified unable to force a prompt over `permissions.allow`, see "Claude Code" above |
|
||||
| Claude Code OTel | not started; not verified against the live CLI yet |
|
||||
| Agent runner (`eval run`) and L3 | designed below, not built |
|
||||
|
||||
## The agent runner, and why it is not here yet
|
||||
|
||||
Scoring reads what a session left behind. The missing half is *starting* one: a runner that
|
||||
launches a harness against a task, several times, and reports a pass rate. What it has to do is
|
||||
no longer guesswork - the adapter work settled most of it:
|
||||
|
||||
- **Give each run a copy of the repository, not a fixture directory.** `config.ROOT` is derived
|
||||
from `chemenu/config.py`'s own location, so `wikitool` cannot be aimed at another tree
|
||||
from outside. An agent case therefore needs a worktree or a clone, where the tool sits inside
|
||||
the tree it edits. (A *fixture* directory works fine in-process, which is why L0 is a test.)
|
||||
- **Give each run its own HOME**: `VIBE_HOME`, `COPILOT_HOME`, plus `WIKI_TRACE_DIR` and
|
||||
`WIKITOOL_SESSION_ID`, so state, trust decisions and traces cannot leak between runs.
|
||||
- **Never launch Vibe without `--agent`.** Its programmatic mode falls back to `auto-approve`
|
||||
when no agent is named, and does not ask about folder trust. The runner must refuse rather
|
||||
than inherit that default.
|
||||
- **Record a run manifest**: harness and version, model, git SHA, and hashes of `AGENTS.md`,
|
||||
the published skills and the fixture. Without it a pass rate cannot be compared to anything.
|
||||
|
||||
What is missing is not the design but the ability to check it. No provider credentials are
|
||||
configured on this machine - Vibe's providers declare an `api_key_env_var` and none of those
|
||||
variables is set - so a live run cannot be executed, let alone verified. Writing it anyway
|
||||
would repeat exactly the mistake the Vibe adapter avoided: the hook format there was wrong in
|
||||
the documentation and only the installed CLI showed it. A runner written against an unverified
|
||||
mental model of three harnesses would be worse.
|
||||
|
||||
Open questions: whether Claude Code has a redirectable config directory (only `--settings` is
|
||||
documented), and whether the scrubber's own patterns are enough or a dedicated secret scanner
|
||||
belongs in the verification step.
|
||||
+156
-253
@@ -1,296 +1,199 @@
|
||||
# Installation
|
||||
# Install a Chemenu instance
|
||||
|
||||
Dieses Dokument richtet sich an Menschen. Es gibt vier Wege: ein **Release herunterladen**
|
||||
(der normale Weg zu einer neuen Instanz), eine Distribution **selbst exportieren**, **dieses
|
||||
Repo klonen** (Testbett und Demo, samt Beispielkorpus), oder eine **private Instanz mit diesem
|
||||
Repo als Upstream** aufsetzen. Der agent-seitige Ablauf steckt in `instructions/`; hier stehen
|
||||
nur die menschlichen Teile - für die vollständige Kommandoreferenz siehe
|
||||
This tutorial takes you from an empty folder to a running Chemenu instance. There is exactly one
|
||||
way to get one: an agent installs the **latest release** into an empty folder you choose. The
|
||||
agent runs the steps, following `instructions/setup-instance.md` from that same release. This
|
||||
page covers what you prepare beforehand, the sentence you give the agent, what it asks you, and
|
||||
what to do when it stops. The full command reference is in
|
||||
[tools/CONTRACT.md](tools/CONTRACT.md).
|
||||
|
||||
## Voraussetzungen
|
||||
The optional **MCP read server** is set up separately, once the instance runs:
|
||||
[docs/how-to/set-up-the-mcp-server.md](docs/how-to/set-up-the-mcp-server.md). It serves the same
|
||||
corpus, read-only, to a consumer that is not a terminal on this machine.
|
||||
|
||||
- Python 3.11 oder neuer
|
||||
- git
|
||||
- [ripgrep](https://github.com/BurntSushi/ripgrep) (`rg`) - wird von `search` und
|
||||
`sources coverage` gebraucht
|
||||
<!-- dist:strip-start -->
|
||||
To work on the stack itself, clone this repository instead. That is a development checkout with a
|
||||
demo corpus, not an instance; it is described in [docs/dev/development.md](docs/dev/development.md).
|
||||
<!-- dist:strip-end -->
|
||||
|
||||
## Weg A: Release herunterladen
|
||||
## What you need first
|
||||
|
||||
Der kürzeste Weg zu einer eigenen Instanz - kein Checkout dieses Repos nötig. Jedes Release
|
||||
trägt genau einen `dist export`-Baum plus eine Prüfsumme. Das Repo ist öffentlich, der Download
|
||||
braucht also weder Konto noch Token:
|
||||
The preflight checks these programs before any `wikitool` command runs. You install them
|
||||
yourself - the agent never does, not even with your permission. Where one is missing, the
|
||||
preflight names the install command for your system. The list is generated from
|
||||
`tools/prerequisites.txt`, the same file the preflight reads:
|
||||
|
||||
```bash
|
||||
BASE=https://gitea.nehmer.net/torben/chemenu/releases/download/v<version>
|
||||
curl -LO $BASE/chemenu-stack-<version>.tar.gz
|
||||
curl -LO $BASE/chemenu-stack-<version>.tar.gz.sha256
|
||||
sha256sum -c chemenu-stack-<version>.tar.gz.sha256
|
||||
tar xzf chemenu-stack-<version>.tar.gz
|
||||
cd chemenu-stack-<version>
|
||||
```
|
||||
<!-- wikitool:prerequisites -->
|
||||
- **Python** ≥ 3.11
|
||||
- **Git**
|
||||
- **ripgrep (rg)**
|
||||
<!-- /wikitool:prerequisites -->
|
||||
|
||||
Die Prüfsumme ist nicht Zierde: Sie ist das Einzige, was einen unterbrochenen Download von
|
||||
einem vollständigen unterscheidet, und `sha256sum -c` muss `OK` sagen, bevor irgendetwas
|
||||
entpackt wird.
|
||||
You also need:
|
||||
|
||||
Danach weiter mit Schritt 2 aus Weg B: den Agenten
|
||||
[instructions/setup-instance.md](instructions/setup-instance.md) ausführen lassen. Der
|
||||
entpackte Baum ist bereits eine Distribution - Schritt 1 (`dist export`) entfällt.
|
||||
- **An agent harness**: Claude Code, GitHub Copilot (in VS Code or as a CLI), Codex CLI or
|
||||
Mistral Vibe.
|
||||
- **An empty folder** for the instance, with your harness opened in it. Empty means nothing but a
|
||||
`.git`. A freshly cloned, empty repository for your instance is exactly right: if your
|
||||
repository `torben/nathan` lives in `~/src/nathan`, install there, and the agent keeps its
|
||||
`origin` as the target for `publish`.
|
||||
|
||||
Die Liste der Releases: <https://gitea.nehmer.net/torben/chemenu/releases>.
|
||||
On Windows, additionally, also checked by the preflight:
|
||||
|
||||
## Weg B: Neue, leere Instanz selbst exportieren
|
||||
<!-- wikitool:prerequisites-windows -->
|
||||
- **PowerShell 7 (pwsh)** ≥ 7
|
||||
<!-- /wikitool:prerequisites-windows -->
|
||||
|
||||
Dasselbe Ergebnis aus einem Checkout dieses Repos - für einen Stand, der noch kein Release hat.
|
||||
Zwei Schritte, von denen nur der erste rein menschlich ist:
|
||||
And on Windows also:
|
||||
|
||||
1. **Zielverzeichnis wählen** und die Distribution dorthin exportieren, aus einem Checkout
|
||||
dieses Repos:
|
||||
- **PowerShell 7 as the default terminal in VS Code.** Windows PowerShell 5.1 is not enough, and
|
||||
WSL is not supported.
|
||||
- **Execution policy `RemoteSigned`** - the factory setting on many machines
|
||||
(`Get-ExecutionPolicy -List` shows it).
|
||||
- **Git for Windows.** It brings Git Bash, which Claude Code runs its commands in.
|
||||
- **An installation folder of at most 95 characters**, for example `C:\Chemenu`. Without long
|
||||
paths switched on, Windows allows only 259 characters per path, and the wiki's files need the
|
||||
rest. If you have administrator rights you can switch long paths on instead
|
||||
(`LongPathsEnabled`); that is not required.
|
||||
|
||||
```bash
|
||||
tools/wikitool dist export /pfad/zur/neuen/instanz
|
||||
```
|
||||
## The sentence for the agent
|
||||
|
||||
Das Ziel muss leer sein oder noch nicht existieren. `dist export` kopiert die Maschinerie
|
||||
(Werkzeuge, Typen, Instruktionen, die Collection-Contracts) ohne Wiki-Inhalt, ohne
|
||||
Git-Historie und ohne `instructions/dev/` (Stack-Entwicklung selbst, inkl. der vendorten
|
||||
`commonplace/`-Wissensbasis) - dauerhaft, ohne Restore-Weg.
|
||||
Open the empty folder in your harness and give the agent this sentence:
|
||||
|
||||
2. **Den Agenten dort arbeiten lassen.** Öffne das Zielverzeichnis in deinem Agent-Harness
|
||||
(Claude Code, GitHub Copilot, Codex CLI, Mistral Vibe) und lass es
|
||||
`instructions/setup-instance.md` ausführen. Diese Anweisung fragt dich dabei explizit nach:
|
||||
- **Autor-Identität** (Name + E-Mail für `git config`) - wird nie geraten oder aus einem
|
||||
anderen Repo übernommen, und ist zugleich der Autorname jeder künftig angelegten
|
||||
Wiki-Seite (`$WIKI_AUTHOR` überschreibt dies bei Bedarf).
|
||||
- **Remote** (optional) - eine URL, wenn du das Repo auf einen Server pushen willst; sonst
|
||||
bleibt die Instanz lokal, und jedes `publish` läuft mit `--no-push`.
|
||||
- **KB-Sprache** - die exportierte Distribution bringt **Deutsch** mit: die Regel in
|
||||
`kb/CONTRACT.md`, das Vokabular in `instructions/german-terminology.md` und deutsche
|
||||
Abschnittsnamen in den Seitenvorlagen. Das ist eine Entscheidung dieser Ursprungsinstanz,
|
||||
keine Eigenschaft des Musters. Willst du eine andere Sprache, sag es **vor dem ersten
|
||||
Ingest** - danach ist es eine Migration jeder bereits angelegten Seite.
|
||||
- **Personalization** - wer diese Instanz bedient (`USER.md`) und wie sie klingt
|
||||
(`SOUL.md`). Die Distribution bringt nur `USER.md.template` und `SOUL.md.template` mit:
|
||||
persönlicher Inhalt gehört nicht in jede exportierte Kopie, aber beide Dateien werden in
|
||||
jeder Session gelesen, sind also Betriebsvoraussetzung. Der Agent interviewt dich entlang
|
||||
der Template-Abschnitte und schreibt deine Antworten **wörtlich** mit - inklusive der
|
||||
beiden Fragen, die er nicht raten darf: der **Persona-Name** und die **Themen, die
|
||||
bewusst draußen bleiben**.
|
||||
> Set up a new Chemenu instance in this folder. Fetch the latest release from
|
||||
> `https://gitea.nehmer.net/api/v1/repos/torben/chemenu/releases/latest` and follow its asset
|
||||
> `setup-instance.md`. Before you start the preflight, read the asset `preflight.md` from the same
|
||||
> release.
|
||||
|
||||
Danach ist die Instanz initialisiert, verifiziert und committet.
|
||||
With that, the agent reads the description of the latest release, and from it the two
|
||||
instructions. It then downloads the matching preflight script (`preflight.ps1` for PowerShell,
|
||||
`preflight.sh` for a POSIX shell) into the folder with a command of its shell - not through the
|
||||
browser, so that Windows does not mark the file as "from the internet" - and runs it. The script
|
||||
downloads the tarball of the same release, checks its sha256, unpacks it into the folder, deletes
|
||||
itself, and then checks in the unpacked tree that everything is in place.
|
||||
|
||||
Was von der Sprachwahl unberührt bleibt: die Trennung zwischen Prosa und Identifiern.
|
||||
Seitentitel, Wikilink-Ziele, Zitat-IDs, Schema-Werte, Tags, Befehle und Pfade folgen keiner
|
||||
KB-Sprache, sondern dem etablierten Namen der Sache - `Act Runner` heißt in jeder Instanz
|
||||
`Act Runner`.
|
||||
All releases are listed at <https://gitea.nehmer.net/torben/chemenu/releases>. The repository is
|
||||
public; the download needs neither an account nor a token.
|
||||
|
||||
## Weg C: Dieses Repo klonen
|
||||
## What the agent asks you
|
||||
|
||||
Für die Arbeit am Stack selbst, oder um sich den mitgelieferten Korpus als begehbares Beispiel
|
||||
anzusehen. Was hier liegt, ist ein **Testbett und eine Demo**, keine produktive Wissensbasis:
|
||||
rund 170 Seiten, die den Stack selbst dokumentieren - Gates, Lint, Versionierung, Suche, das
|
||||
Wiki-Muster. Wer eigenes Wissen sammeln will, nimmt Weg A oder B und fängt mit einem leeren
|
||||
`kb/` an.
|
||||
The agent may guess none of these answers, and it carries none of them over from another
|
||||
repository:
|
||||
|
||||
```bash
|
||||
git clone https://gitea.nehmer.net/torben/chemenu.git
|
||||
cd chemenu
|
||||
```
|
||||
- <!-- setup-question: identity --> **Author identity** - name and email for `git config`. This is
|
||||
also the author name of every wiki page created from now on (`$WIKI_AUTHOR` overrides it when
|
||||
needed).
|
||||
- <!-- setup-question: remote --> **Remote** - for an empty clone, only the confirmation that
|
||||
`origin` is right; otherwise a URL, if you want to push to a server. Without a remote the
|
||||
instance stays local, and every `publish` runs with `--no-push`.
|
||||
- <!-- setup-question: kb-language --> **Language and tone of the pages** - they go into
|
||||
`kb/CONVENTIONS.md`, plus `kb/<name>/COLLECTION.md` per collection. Ready-made profiles,
|
||||
including a complete German one, are in `instructions/kb-profiles.md`. Decide this **before the
|
||||
first ingest**: afterwards, changing the section names is a migration of every existing page.
|
||||
Titles, wikilink targets, citation ids, schema values, tags, commands and paths follow no
|
||||
language - `Act Runner` is called `Act Runner` in every instance.
|
||||
- <!-- setup-question: domain --> **Subject area** - where this wiki draws its sources from. From
|
||||
it the agent proposes a `source_type` list (for a sports club, say: statutes, minutes, match
|
||||
report). That is a starting point, not a commitment: it gets corrected against real material
|
||||
later (`instructions/evolve-subtypes.md`).
|
||||
- <!-- setup-question: personalization --> **Personalization** - who operates this instance
|
||||
(`USER.md`) and how it sounds (`SOUL.md`). The agent interviews you along the templates and
|
||||
writes your answers down verbatim. Two questions only you answer: the **name of the persona**
|
||||
and the **topics that deliberately stay out**.
|
||||
- <!-- setup-question: environment --> **Environment** (optional) - harness, MCP servers, remotes,
|
||||
so that later sessions don't ask again. "I don't know" is a valid answer.
|
||||
- <!-- setup-question: telemetry --> **Telemetry** - off by default; the agent only asks whether
|
||||
you want to switch it on.
|
||||
- <!-- setup-question: task-tracker --> **Task tracker** (optional) - see
|
||||
[docs/reference/configuration.md](docs/reference/configuration.md#task-tracker).
|
||||
|
||||
Danach den Agenten `instructions/bootstrap.md` ausführen lassen (Werkzeugumgebung + Skills
|
||||
publizieren). Git-Repo, Autor-Identität und Inhalt existieren hier bereits.
|
||||
At the end the agent makes the first commit. The Mass-Update Gate stops it there (exit 42),
|
||||
because a new instance consists of far more than ten files. That is expected: the agent shows you
|
||||
the file list and the `--confirm` line, and nothing is published until you approve. Afterwards,
|
||||
restart the agent session in the same folder so that it loads the skills.
|
||||
|
||||
Ein Clone, der älter ist als die Personalization-Dateien, hat kein `USER.md`/`SOUL.md` -
|
||||
`doctor` meldet dafür `personalization: FAIL`. Das ist einmalig nachzuholen: nur **Schritt 6
|
||||
(Personalization)** aus `instructions/setup-instance.md`, nicht der ganze Ablauf. `bootstrap.md`
|
||||
verweist an derselben Stelle darauf.
|
||||
## When the agent stops
|
||||
|
||||
`ENVIRONMENT.md` fehlt nach einem Clone immer - die Datei ist gitignored, weil sie *einen
|
||||
Checkout* beschreibt und nicht das Repo. Sie ist optional; wer sie anlegt, spart jeder
|
||||
folgenden Session die Fragen nach Harness, MCP-Servern und Remote. Vorlage:
|
||||
`ENVIRONMENT.md.template`, Ablauf: Schritt 5 in `instructions/bootstrap.md`.
|
||||
The preflight stops with **exit 42** when you have to do something. Its output names, in a
|
||||
numbered block, what is missing, why, the command that fixes it, and how to continue. The agent
|
||||
shows you this block unchanged, adds a translation below it at most, and waits. Tell it when you
|
||||
are done; then it checks again. It may not work around the stop or install anything itself.
|
||||
|
||||
## Weg D: Private Instanz mit diesem Repo als Upstream
|
||||
**While downloading and unpacking** (the script from the release, before there is a tree):
|
||||
|
||||
Die Kombination aus A und C: eine eigene, nicht öffentliche Instanz, die weiterhin
|
||||
Stack-Updates von hier zieht - per `git merge` statt per Tarball, also mit echtem
|
||||
Drei-Wege-Merge statt `cp -r`.
|
||||
- **Tools for downloading or unpacking are missing** (exit 42). On Linux and macOS the script
|
||||
needs `curl`, `tar` and `sha256sum` (or `shasum`); on Windows only the `tar.exe` that comes
|
||||
with Windows 10/11. Install what the output names; on Windows, Git for Windows brings
|
||||
everything for Git Bash.
|
||||
- **The folder path is too long** (exit 42, only on Windows without long paths). Take a shorter
|
||||
one, such as `C:\Chemenu`, open it in the harness and give the sentence there again.
|
||||
- **The folder is not empty** (exit 1). It may hold nothing but the script and a `.git`. Clean it
|
||||
up yourself or take another one - the agent deletes nothing there.
|
||||
- **Download failed or checksum wrong** (exit 1). Nothing was unpacked. Check the internet
|
||||
connection and let it try again; the agent does not look for another way to download. Without a
|
||||
direct download you can put the tarball and its `.sha256` from the release page side by side
|
||||
yourself; the agent then runs the script with `--archive <tarball>`.
|
||||
- **A part of the stack could not be moved into place** (exit 1, "could not move ... into").
|
||||
Usually a virus scanner briefly holding a freshly unpacked file open; the script has already
|
||||
waited about 30 seconds for it. If the message says "nothing was installed", the script has
|
||||
taken everything back - let the agent run it again. If it names entries it could not take
|
||||
back, empty the folder yourself (keep `.git`), then run it again.
|
||||
|
||||
Das ist der Weg mit dem höchsten Einsatz, weil ein Checkout dann zwei Remotes hat und git beim
|
||||
Push nicht unterscheidet, welcher welcher ist. Ein falsches `--remote` legt privaten Inhalt auf
|
||||
ein öffentliches Repo, und ein Force-Push holt das nicht zurück - die Objekte bleiben per SHA
|
||||
abrufbar, bis auf dem Server die Reflogs verfallen.
|
||||
**In the unpacked tree** (every further run is `tools/preflight.sh`, or under PowerShell
|
||||
`pwsh -NoProfile -ExecutionPolicy Bypass -File tools/preflight.ps1`):
|
||||
|
||||
Dagegen gibt es das **Publish-Remote-Gate**, und die Anleitung setzt es an die Stelle, an der
|
||||
es wirkt: *vor* dem ersten `publish`. Vollständiges Vorgehen:
|
||||
[instructions/private-instance.md](instructions/private-instance.md).
|
||||
- **A tool is missing** - Python, git, ripgrep, on Windows also PowerShell 7. The output names
|
||||
the install command for your system. If it is installed, just somewhere else, tell the agent the
|
||||
path; it passes it on with `--set <tool>=<path>`.
|
||||
- **A version is too old** - Python below 3.11, for example. Install a newer version or name its
|
||||
path.
|
||||
- **A path you named does not work** - the path must point at the program itself, not at its
|
||||
folder.
|
||||
- **The Python environment (`tools/.venv`) or its libraries could not be set up.** The output
|
||||
shows what Python or pip reported. Usually a proxy or a security program blocks the download;
|
||||
whoever looks after your machine can sort that out.
|
||||
- **Scripts carry the "from the internet" mark** (only on Windows). That happens when the release
|
||||
was downloaded in the browser and unpacked in Explorer. Once, in the folder, in PowerShell 7:
|
||||
`Get-ChildItem -Recurse -File | Unblock-File`. `doctor` shows the state under `script-marks`.
|
||||
- **The execution policy forbids scripts** (`Restricted` or `AllSigned`, only on Windows). The
|
||||
output names the one line for a PowerShell 7 window
|
||||
(`Set-ExecutionPolicy -Scope CurrentUser -ExecutionPolicy RemoteSigned`). If a group policy sets
|
||||
it, only your IT department can help - or you work from Git Bash with `tools/wikitool`. `doctor`
|
||||
shows the state under `execution-policy`.
|
||||
- **The installation folder path is too long** (only on Windows without long paths). The instance
|
||||
has to move to a shorter folder, such as `C:\Chemenu`.
|
||||
|
||||
## Version und Updates
|
||||
If the agent stops anywhere else and the cause is not obvious, it offers you a bug report - see
|
||||
[docs/how-to/troubleshoot.md](docs/how-to/troubleshoot.md).
|
||||
|
||||
Jede Instanz trägt die Version des **Stacks** (Werkzeuge, Typen, Instruktionen, Contracts) -
|
||||
nicht die ihres Inhalts. Sie steht in `VERSION`, und eine per Release oder `dist export`
|
||||
erzeugte Instanz trägt zusätzlich `.wikitool-release.json` mit Herkunft und Exportdatum.
|
||||
|
||||
```bash
|
||||
tools/wikitool version # was läuft hier, und woher kommt es
|
||||
tools/wikitool version check # gibt es ein neueres Release?
|
||||
```
|
||||
|
||||
`version check` ist der einzige Befehl, der ins Netz geht. Er fragt den Release-Feed der
|
||||
Ursprungs-Instanz (`$WIKITOOL_UPDATE_URL` überschreibt; sonst der Wert aus dem Stamp). Ein
|
||||
nicht erreichbarer Feed wird als Fehler gemeldet - **nie** als „aktuell".
|
||||
|
||||
**Was die Versionsnummer aussagt:** kompatibel ist, was in der *linkesten von Null
|
||||
verschiedenen Stelle* übereinstimmt. `0.1.3 → 0.1.4` ist ein sicheres Update, `0.1.3 → 0.2.0`
|
||||
verlangt eine Migration, und ab `1.0.0` liest sich dieselbe Regel als das gewohnte „MAJOR heißt
|
||||
Migration". `version check` sagt das direkt (`state: update` vs. `state: migration`).
|
||||
|
||||
### Eine Instanz aktualisieren
|
||||
|
||||
Das Anwenden eines Updates ist ein bewusst manueller Vorgang - es schreibt in eine Instanz, die
|
||||
bereits Inhalt hat. Der Inhalt hat dabei eine **eigene Version**: `.wikitool-kb.json` sagt, in
|
||||
welcher Form die Seiten vorliegen, unabhängig davon, welche Maschinerie danebensteht. Genau
|
||||
dieser Unterschied ist der Zustand, in dem sich jede Instanz mitten im Upgrade befindet.
|
||||
|
||||
1. **Vor dem Tausch** prüfen, was ansteht - solange `VERSION` noch die alte ist:
|
||||
|
||||
```bash
|
||||
tools/wikitool migrate status
|
||||
```
|
||||
|
||||
2. Release-Tarball herunterladen und entpacken (Weg A), die Release-Notes lesen.
|
||||
3. Die **Maschinerie** aus dem Tarball über die Instanz kopieren: `tools/`, `types/`,
|
||||
`instructions/`, `AGENTS.md`, `VERSION`, `.wikitool-release.json`. Nicht anfassen: `kb/`,
|
||||
`raw/`, `work/`, `.wikitool-kb.json` und `.git/` - das ist die Instanz selbst.
|
||||
4. Achtung bei lokal angepassten Contract-Dateien: wer z. B. die KB-Sprache umgestellt hat
|
||||
(Schritt 5 in `setup-instance.md`), hat `kb/CONTRACT.md` und die Templates unter `types/`
|
||||
verändert. Diese Änderungen vorher sichern und danach wieder einspielen. Welche Dateien das
|
||||
sind, verrät ein Vergleich gegen die sha256-Summen im `files`-Block der alten
|
||||
`.wikitool-release.json`.
|
||||
5. **Die Migrationskette abarbeiten.** `tools/wikitool migrate status` listet jetzt alle
|
||||
offenen Migrationen in der Reihenfolge, in der sie laufen müssen - bei einem Sprung über
|
||||
mehrere Versionen sind das mehrere. Für jede: das genannte Dokument unter
|
||||
`instructions/migrations/` ausführen lassen (die Prozedur dazu ist
|
||||
`instructions/migrate-corpus.md`), dann
|
||||
|
||||
```bash
|
||||
tools/wikitool migrate done <version>
|
||||
```
|
||||
|
||||
`done` verweigert jede Version, die nicht das nächste Glied ist - eine übersprungene
|
||||
Migration hinterlässt einen Korpus in einer Form, die keine Version beschreibt. Ein
|
||||
abgebrochenes Upgrade wird durch erneutes `migrate status` fortgesetzt.
|
||||
6. Prüfen: `tools/wikitool migrate verify --from <commit vor der Migration>`, dann `doctor`,
|
||||
`docs verify`, `instructions verify` und `lint`. Zum Schluss
|
||||
`tools/wikitool instructions sync` (die Skills sind Kopien) und die Agent-Session neu
|
||||
starten.
|
||||
|
||||
`doctor` warnt, solange `kb_version` hinter `VERSION` zurückliegt und noch Migrationen offen
|
||||
sind. Einer Instanz, die älter ist als `.wikitool-kb.json`, fehlt die Datei ganz - dann einmalig
|
||||
`tools/wikitool migrate baseline <version>` aufrufen; geraten wird nichts.
|
||||
|
||||
### Sonderfall: Update von 1.x auf 2.0.0
|
||||
|
||||
Mit `2.0.0` wurde das Ursprungs-Repo von `torben/llm-wiki-test1` auf `torben/chemenu`
|
||||
umbenannt. Eine Instanz, die vor diesem Release exportiert wurde, trägt in
|
||||
`.wikitool-release.json` noch den alten Feed - und `version check` fragt damit einen Pfad ab,
|
||||
den es unter diesem Namen nicht mehr gibt. Der Befehl bricht also nicht kaputt, er erfährt nur
|
||||
nichts mehr. Einmalig überschreiben:
|
||||
|
||||
```bash
|
||||
export WIKITOOL_UPDATE_URL="https://gitea.nehmer.net/api/v1/repos/torben/chemenu/releases/latest"
|
||||
tools/wikitool version check
|
||||
```
|
||||
|
||||
Danach den Tarball aus Weg A holen - er heißt seit `2.0.0` `chemenu-stack-<version>.tar.gz`
|
||||
statt `llm-wiki-stack-<version>.tar.gz` - und den Ablauf oben normal durchlaufen. Das
|
||||
mitkopierte `.wikitool-release.json` trägt den neuen Feed, die Variable wird danach nicht mehr
|
||||
gebraucht.
|
||||
|
||||
Zwei Nachräumarbeiten, weil Schritt 3 `tools/` kopiert und nichts löscht: das alte Paket
|
||||
`tools/wiki_tools/` bleibt neben dem neuen `tools/chemenu/` liegen und kann weg - der
|
||||
`tools/wikitool`-Shim ruft seit `2.0.0` `-m chemenu.cli` auf und rührt es nicht mehr an. Und
|
||||
eigene Skripte, die `from wiki_tools import …` machen, müssen auf `chemenu` gezogen werden.
|
||||
|
||||
Eine Inhaltsmigration verlangt dieses Release nicht: `migrate status` bleibt leer, `kb/`
|
||||
behält Schema und Shape.
|
||||
|
||||
## Konfiguration
|
||||
|
||||
| Variable | Zweck | Fallback |
|
||||
|----------|-------|----------|
|
||||
| `WIKI_AUTHOR` | Override für den Autornamen neuer Source-Seiten | `git config user.name` - fehlt beides, bricht `new` mit `ERROR` ab |
|
||||
| `WIKITOOL_SESSION_ID` | Scopt das Iteration-Budget-Gate auf eine Aufgabe statt auf ein Terminal | Parent-Process-ID (siehe [instructions/session-setup.md](instructions/session-setup.md)) |
|
||||
| `WIKITOOL_UPDATE_URL` | Release-Feed, den `version check` abfragt | Wert aus `.wikitool-release.json`, sonst der Feed der Ursprungs-Instanz |
|
||||
| `WIKITOOL_UPDATE_TOKEN` | Gitea-Token für den Release-Feed | keiner - gegen `torben/chemenu` nicht nötig, nur für einen privaten Fork (siehe unten) |
|
||||
|
||||
**Gegen das Ursprungs-Repo braucht es kein Token.** `torben/chemenu` ist öffentlich lesbar;
|
||||
`version check` und der Download in Weg A funktionieren ohne Konfiguration.
|
||||
|
||||
**Für einen privaten Fork schon.** Wer den Stack in ein eigenes, nicht öffentliches Repo legt
|
||||
und `WIKITOOL_UPDATE_URL` auf dessen Feed zeigen lässt, stößt auf eine Eigenheit, die man
|
||||
kennen sollte: Gitea antwortet anonymen Aufrufern für ein unsichtbares Repo mit demselben
|
||||
`404` wie für ein gar nicht existierendes. Ein fehlendes Release und ein fehlender Zugriff
|
||||
sehen dann identisch aus - „kein Update gefunden" wäre in dem Fall schlicht gelogen. Dagegen
|
||||
hilft ein Gitea-Token mit Lesezugriff:
|
||||
|
||||
```bash
|
||||
export WIKITOOL_UPDATE_TOKEN="<gitea-token>"
|
||||
tools/wikitool version check
|
||||
```
|
||||
|
||||
## Verifikation
|
||||
## Check the instance
|
||||
|
||||
```bash
|
||||
tools/wikitool doctor
|
||||
```
|
||||
|
||||
Prüft in einem Aufruf: Abhängigkeiten, Autor-Auflösung, Git-Identität/Branch/Remote,
|
||||
publizierte Skills, Struktur (Collection-Contracts, generierte Dateien), Personalization
|
||||
(`USER.md`/`SOUL.md` vorhanden **und** ausgefüllt), die optionale Umgebungsnotiz
|
||||
(`ENVIRONMENT.md`) und die Session-ID.
|
||||
`OK`/`WARN` sind unbedenklich (ein fehlender Remote z. B. ist ein gültiger Endzustand); nur ein
|
||||
`FAIL` bricht mit exit 1 ab, und jede Zeile nennt ihr eigenes Fix-Kommando.
|
||||
This checks in one call: dependencies, author resolution, git identity/branch/remote, published
|
||||
skills, structure (collection contracts, generated files), personalization (`USER.md`/`SOUL.md`
|
||||
present **and** filled in), the optional environment note (`ENVIRONMENT.md`), the task tracker
|
||||
(`.wikitool-tasks.json` - if it is missing, that is `OK`; if a provider is configured, also
|
||||
whether its read path is ready and its API is answering right now, neither ever a `FAIL`) and the
|
||||
session id. `OK`/`WARN` are harmless (a missing remote, for example, is a valid end state); only a
|
||||
`FAIL` ends with exit 1, and every line names its own fix command.
|
||||
|
||||
Danach zusätzlich:
|
||||
Then also:
|
||||
|
||||
```bash
|
||||
tools/wikitool docs verify
|
||||
tools/wikitool instructions verify
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
## Next steps
|
||||
|
||||
- **`wikitool: venv not found`** - Schritt "Werkzeugumgebung anlegen" aus
|
||||
[instructions/bootstrap.md](instructions/bootstrap.md) bzw.
|
||||
[instructions/setup-instance.md](instructions/setup-instance.md) wurde noch nicht ausgeführt.
|
||||
- **Der Agent bietet keine Skills an (`wiki-ingest`, `wiki-query`, ...)** - `.agents/skills/`
|
||||
und `.claude/skills/` sind generiert und nicht committet. `tools/wikitool instructions sync`
|
||||
ausführen, dann die Agent-Session neu starten (Harnesses lesen Skills nur beim Start).
|
||||
- **`doctor` meldet `personalization: FAIL`** - `USER.md`/`SOUL.md` fehlen, oder sie tragen
|
||||
noch die Sentinel-Zeile aus dem Template (ein umbenanntes Template ist kein ausgefülltes).
|
||||
Den Personalization-Schritt (6) aus `instructions/setup-instance.md` ausführen lassen; bei
|
||||
einer Instanz nach Weg C ist das der einzige nachzuholende Schritt.
|
||||
- **`doctor` meldet `environment: WARN`** - `ENVIRONMENT.md` existiert, trägt aber noch die
|
||||
Sentinel-Zeile aus dem Template. Ausfüllen (Vorlage: `ENVIRONMENT.md.template`) und die Zeile
|
||||
entfernen, oder die Datei löschen - sie ist optional, und `absent` ist ein gültiger
|
||||
Endzustand.
|
||||
- **`new` bricht mit "No author configured" ab** - weder `$WIKI_AUTHOR` noch
|
||||
`git config user.name` sind gesetzt. `git config user.name "<Name>"` ausführen, oder
|
||||
`WIKI_AUTHOR` exportieren.
|
||||
- **`publish` endet mit Exit-Code 42 (Mass-Update-Gate)** - erwartetes Verhalten bei ≥10
|
||||
gezählten Dateien (z. B. beim allerersten Commit einer neuen Instanz). Das ist kein Fehler,
|
||||
sondern die Aufforderung, die Ausgabe einem Menschen zu zeigen: sie enthält die vollständige
|
||||
Dateiliste und die exakte `--confirm <token>`-Zeile, die nach Freigabe veröffentlicht.
|
||||
Details: [instructions/gates.md](instructions/gates.md).
|
||||
- **`publish` endet mit Exit-Code 42 (Publish-Remote-Gate)** - dieser Checkout hat eine
|
||||
`.wikitool-remotes.json`, und das angesteuerte Remote steht nicht darin. Ebenfalls kein
|
||||
Fehler: Die Ausgabe nennt die Push-URL, an die geschrieben würde, und die erlaubten. Anders
|
||||
als beim Mass-Update-Gate gibt es hier **keinen Token und keine Flagge** - stimmt das Ziel
|
||||
wirklich, trägt der Mensch dessen URL selbst in die Datei ein. Ein Agent, der die Datei
|
||||
anfasst, um an der Verweigerung vorbeizukommen, öffnet ein Gate aus eigenem Antrieb.
|
||||
- **Ich will am Tool-Stack selbst weiterarbeiten (nicht nur Wiki-Inhalt betreiben)** - eine neue
|
||||
Instanz hat dafür keinen Weg: `dist export` lässt `instructions/dev/` (Stack-Entwicklung,
|
||||
inkl. der vendorten `commonplace/`-Wissensbasis) bewusst und dauerhaft weg, ohne
|
||||
Restore-Mechanismus. Für Stack-Entwicklung im Ursprungs-Repo arbeiten (oder eine neue
|
||||
Dev-Instanz daraus exportieren) statt in dieser Instanz nachzurüsten.
|
||||
- **Your first source and your first answer:**
|
||||
[docs/tutorials/first-source-to-first-answer.md](docs/tutorials/first-source-to-first-answer.md)
|
||||
walks you through both in the instance you just set up.
|
||||
- **Updating** to a newer release: [docs/how-to/update-an-instance.md](docs/how-to/update-an-instance.md).
|
||||
- **Configuration** - environment variables and the files per checkout:
|
||||
[docs/reference/configuration.md](docs/reference/configuration.md).
|
||||
- **When something does not work:** [docs/how-to/troubleshoot.md](docs/how-to/troubleshoot.md).
|
||||
- **Working with the wiki:** [docs/how-to/](docs/how-to/README.md).
|
||||
@@ -2,392 +2,72 @@
|
||||
|
||||
A structured, LLM-maintained knowledge base for your personal IT work.
|
||||
|
||||
## What is this?
|
||||
|
||||
This is **Chemenu** - a pattern for building a personal knowledge base using LLMs.
|
||||
Instead of just retrieving from raw documents on every query, the LLM **incrementally builds
|
||||
and maintains a persistent wiki** that compounds over time.
|
||||
|
||||
**The key insight:** Knowledge is compiled once and kept current, not re-derived on every question.
|
||||
|
||||
**The pages are written in German.** Source material in `raw/` is never touched and is usually
|
||||
English; the compiled pages under `kb/` are not. What stays English inside them is everything that
|
||||
is an *identifier* rather than prose - page titles, section headings, wikilink targets, citation
|
||||
ids, schema enum values, tags, commands, paths and code - so `GitOps Ownership Model` and
|
||||
`## Beziehungen` sit in the same page without contradiction. The rule is
|
||||
[kb/CONTRACT.md § Language](kb/CONTRACT.md#language); the vocabulary behind it is
|
||||
[instructions/german-terminology.md](instructions/german-terminology.md).
|
||||
|
||||
This is a per-instance decision, not a property of the pattern. A new instance built with
|
||||
`dist export` starts empty and can pick any language by editing that one contract section before
|
||||
the first ingest.
|
||||
Chemenu is a pattern for a personal knowledge base maintained by an LLM. Instead of going back
|
||||
to the raw documents for every question, the LLM **incrementally builds and maintains a
|
||||
persistent wiki** that compounds over time: knowledge is compiled once and kept current, not
|
||||
re-derived on every question.
|
||||
|
||||
## Getting started
|
||||
|
||||
Two starting points, depending on what you're doing - full walkthrough in [INSTALL.md](INSTALL.md):
|
||||
Two starting points, depending on what you're doing:
|
||||
|
||||
- **Cloned this repo?** The skill definitions the agent harness loads are **generated and not
|
||||
committed**. Publish them once:
|
||||
- **A new instance.** Every instance is installed from a release, into an empty folder you
|
||||
choose: you give your agent one sentence, and it follows `instructions/setup-instance.md` from
|
||||
the latest release - preflight, git init, author identity, an optional remote, your authoring
|
||||
conventions and persona, the first commit. The sentence, what the agent will ask you, and what
|
||||
to do when it stops are in [INSTALL.md](INSTALL.md).
|
||||
|
||||
- **A further checkout of an instance you already have** (a second machine). Clone the
|
||||
instance's own repository, then run the preflight and publish the skills, which are
|
||||
**generated and not committed**:
|
||||
|
||||
```bash
|
||||
cd tools && python3 -m venv .venv && .venv/bin/pip install -r requirements.txt && cd ..
|
||||
tools/preflight.sh # checks python/git/rg, records their paths, creates tools/.venv
|
||||
# (PowerShell 7 on Windows: tools/preflight.ps1, see instructions/preflight.md)
|
||||
tools/wikitool instructions sync
|
||||
```
|
||||
|
||||
That copies each `instructions/<name>/SKILL.md` into `.agents/skills/` (GitHub Copilot, Codex
|
||||
CLI, Mistral Vibe) and `.claude/skills/` (Claude Code). Re-run it after changing a skill.
|
||||
Full procedure: `instructions/bootstrap.md`.
|
||||
|
||||
- **Starting a brand-new, empty instance instead?** `tools/wikitool dist export <target>`
|
||||
builds a contentless copy of the machinery - no example pages, no personal content - then
|
||||
`instructions/setup-instance.md` walks through git init, author identity, an optional remote,
|
||||
and the first commit.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
chemenu/
|
||||
├── AGENTS.md # Control plane: invariants, file naming, routing, gates
|
||||
├── CLAUDE.md # Claude Code only: imports AGENTS.md/USER.md/SOUL.md/ENVIRONMENT.md + its Claude-Code-only instructions. No rules of its own
|
||||
├── README.md # This file: human-readable overview of the whole repo
|
||||
├── INSTALL.md # Human-readable setup: new instance vs. cloning this one
|
||||
├── EVALS.md # Human-readable overview of telemetry and evaluation
|
||||
├── CHANGES.md # Changelog for the stack itself
|
||||
├── USER.md # Who operates this instance - context, never instructions
|
||||
├── SOUL.md # How this instance sounds. AGENTS.md always wins over it
|
||||
├── ENVIRONMENT.md # Optional, gitignored: this checkout's harness, MCP servers, remotes
|
||||
├── *.md.template # Unfilled USER/SOUL/ENVIRONMENT - what a distribution ships instead
|
||||
├── .gitignore # Anchored so nothing under raw/, kb/ or work/ is ever excluded
|
||||
├── .github/hooks/ # Copilot CLI hooks - session tracing
|
||||
├── .vibe/ # Mistral Vibe hooks + the repo's telemetry policy
|
||||
├── instructions/ # CONTROL: everything an agent is told to do
|
||||
│ ├── CONTRACT.md # Instruction vs. skill, publishing, writing standard
|
||||
│ ├── bootstrap.md # Prepare a fresh clone
|
||||
│ ├── gates.md # What to do when a gate refuses a call
|
||||
│ ├── german-terminology.md # Which words stay English in German prose; register
|
||||
│ ├── session-setup.md
|
||||
│ ├── page-lifecycle.md
|
||||
│ ├── publish-cycle.md
|
||||
│ ├── ingest-large-tree.md
|
||||
│ └── wiki-*/SKILL.md # Skills - copied into .agents/skills/ and .claude/skills/
|
||||
├── raw/ # INPUT: immutable, untrusted source material
|
||||
│ ├── CONTRACT.md # Routing, immutability, untrusted content
|
||||
│ ├── articles/ # Web articles, blog posts
|
||||
│ ├── documents/ # PDFs, specs, manuals
|
||||
│ ├── notes/ # Personal notes, transcriptions
|
||||
│ └── assets/ # Images, diagrams, binaries
|
||||
├── types/ # SCHEMA: the global type surface. Not a collection
|
||||
│ ├── type-spec.md # Root contract: anatomy, placement, adding a type
|
||||
│ ├── entity.md # Entity type contract + template (+ .schema.yaml)
|
||||
│ ├── concept.md # Concept type contract + template
|
||||
│ ├── source.md # Source type contract + template
|
||||
│ ├── comparison.md # Comparison type contract + template
|
||||
│ ├── instruction.md # Instruction type - lives outside kb/ via `root: repo`
|
||||
│ └── lint-report.md # Contract-only: describes reports/, owns no directory
|
||||
├── kb/ # OUTPUT: compiled knowledge. A namespace, not a collection
|
||||
│ ├── CONTRACT.md # Collections, naming, tone, linking, provenance, confidence
|
||||
│ ├── index.md # Generated catalog *map*: counts and pointers
|
||||
│ ├── log.md # Generated chronological audit log
|
||||
│ ├── provenance.md # Generated raw-file reverse index
|
||||
│ ├── entities/ # COLLECTION.md + INDEX.md + areas below
|
||||
│ │ ├── projects/
|
||||
│ │ ├── systems/
|
||||
│ │ ├── tools/ # own INDEX.md once past 50 pages
|
||||
│ │ ├── technologies/
|
||||
│ │ └── people/
|
||||
│ ├── concepts/ # COLLECTION.md - architectures, patterns, protocols
|
||||
│ ├── sources/ # COLLECTION.md - source summaries
|
||||
│ └── comparisons/ # COLLECTION.md - comparison pages
|
||||
├── work/ # WORKSHOP: one directory per multi-session run, tracked
|
||||
│ └── CONTRACT.md # Run keys, required files, how a run closes
|
||||
├── reports/ # DERIVED: lint reports, traces, eval scores. Gitignored
|
||||
│ └── CONTRACT.md
|
||||
└── tools/ # COMPILER: the wikitool CLI
|
||||
├── CONTRACT.md # Command reference, error contracts, maintenance schedule
|
||||
└── README.md # How wikitool is built and how to change it
|
||||
```
|
||||
`tools/wikitool` refuses to start (exit 42) until the preflight has passed; if it stops
|
||||
instead, its output says what to install - `instructions/preflight.md`. `instructions sync`
|
||||
copies each `instructions/<name>/SKILL.md` into `.agents/skills/` (GitHub Copilot, Codex CLI,
|
||||
Mistral Vibe) and `.claude/skills/` (Claude Code). Full procedure:
|
||||
`instructions/bootstrap.md`.
|
||||
|
||||
<!-- dist:strip-start -->
|
||||
Dev-instance-only (see `tools/CONTRACT.md` for how it got here):
|
||||
|
||||
```
|
||||
└── commonplace/ # Vendored, read-only knowledge base
|
||||
```
|
||||
- **Working on the stack itself.** A clone of this repository is a development checkout, with
|
||||
a demo corpus; it is never an instance. Setting it up, and `dist export` as
|
||||
the build and test tool it is, are in `docs/dev/development.md` (for you) and
|
||||
`instructions/dev/dev-setup.md` (for the agent).
|
||||
<!-- dist:strip-end -->
|
||||
|
||||
A directory under `kb/` is a **collection** exactly when it holds a `COLLECTION.md`; a
|
||||
subdirectory inside one is an **area** that inherits it. `COLLECTION.md` never appears outside
|
||||
`kb/` - the other layers carry a `CONTRACT.md` or a root type-spec instead. A stage may carry
|
||||
both a `README.md` and a `CONTRACT.md`: they have different readers. The README is for humans
|
||||
working *on* that layer, the contract is what binds an agent working *with* it.
|
||||
## Documentation
|
||||
|
||||
## How to Use
|
||||
The human documentation lives under `docs/`, sorted by what you want from it:
|
||||
|
||||
### Adding Knowledge (Ingest)
|
||||
- [docs/tutorials/](docs/tutorials/README.md) - learning: a guided first walk through an
|
||||
instance, starting with the installation in [INSTALL.md](INSTALL.md).
|
||||
- [docs/how-to/](docs/how-to/README.md) - doing: one task per page, for the operator of an
|
||||
instance.
|
||||
- [docs/reference/](docs/reference/README.md) - looking up: commands, exit codes, skills, page
|
||||
types, configuration. The command reference is
|
||||
[docs/reference/commands/](docs/reference/commands/README.md); agents use the same records in
|
||||
[tools/CONTRACT.md](tools/CONTRACT.md).
|
||||
- [docs/explanation/](docs/explanation/README.md) - understanding: why the stack is built the way
|
||||
it is.
|
||||
|
||||
1. Drop a file into `raw/` (articles, documents, notes, or assets)
|
||||
2. Tell the LLM: `Ingest raw/articles/my-article.md`
|
||||
3. The LLM will:
|
||||
- Read and summarize the source
|
||||
- Create a source page in `kb/sources/`
|
||||
- Create or update relevant entity pages
|
||||
- Create or update relevant concept pages
|
||||
- Add cross-references between everything
|
||||
- Rebuild the catalog and append to `kb/log.md`
|
||||
Most days you need three of the how-to guides: [ingest a source](docs/how-to/ingest-a-source.md),
|
||||
[query the wiki](docs/how-to/query-the-wiki.md) and [maintain it](docs/how-to/maintain-the-wiki.md).
|
||||
How the parts fit together is [docs/explanation/architecture.md](docs/explanation/architecture.md).
|
||||
|
||||
### Querying Knowledge
|
||||
|
||||
Ask questions naturally:
|
||||
- "What projects use MQTT?"
|
||||
- "Show me the architecture of ha-core"
|
||||
- "Compare gdeploy and plugnburn-edl"
|
||||
- "What decisions were made about E3DC integration?"
|
||||
|
||||
The LLM will search the wiki, synthesize an answer, and cite sources.
|
||||
|
||||
### Maintaining Knowledge (Lint)
|
||||
|
||||
Periodically run: `Lint the wiki`
|
||||
|
||||
The LLM will:
|
||||
- Run `tools/wikitool lint` for a deterministic structural + provenance scan
|
||||
(broken wikilinks, orphan pages, index drift, schema gaps, uncovered raw
|
||||
files, citation/frontmatter drift)
|
||||
- Check for contradictions (semantic judgment)
|
||||
- Find stale claims
|
||||
- Identify orphan pages and missing cross-references
|
||||
- Apply confidence decay (`tools/wikitool confidence decay --apply`)
|
||||
- Rebuild `kb/index.md` and `kb/provenance.md`, append to `kb/log.md`
|
||||
- Generate a report
|
||||
|
||||
See the [Maintenance](#maintenance) section below for the full schedule and
|
||||
command reference.
|
||||
|
||||
## Entity Types
|
||||
|
||||
Entities are subtyped as project, system, tool, technology, or person, and each subtype has
|
||||
its own directory under `kb/entities/`. The authoritative list - and where each one is
|
||||
written - is declared by the type-spec, so ask the tool rather than a table here:
|
||||
|
||||
```bash
|
||||
tools/wikitool types list
|
||||
tools/wikitool types describe entity
|
||||
```
|
||||
|
||||
## Workflows
|
||||
|
||||
### For You (Human)
|
||||
|
||||
1. **Curate sources** - Add files to `raw/` that you want processed
|
||||
2. **Ask questions** - Query the wiki naturally
|
||||
3. **Review changes** - Check `kb/log.md` and `kb/index.md`
|
||||
4. **Direct the LLM** - Guide it on what to emphasize or investigate
|
||||
5. **Browse in Obsidian** - Open the wiki directory in Obsidian for visualization
|
||||
|
||||
### For the LLM
|
||||
|
||||
`AGENTS.md` is the cross-cutting schema/policy; the step-by-step procedures
|
||||
themselves live as independently-discoverable skills under `.agents/skills/`
|
||||
(mirrored to `.claude/skills/` for Claude Code via `tools/wikitool instructions sync`):
|
||||
|
||||
| Skill | Purpose |
|
||||
|-------|---------|
|
||||
| `wiki-ingest` | Process a new `raw/` source into the wiki: source summary, entity/concept pages, cross-references, index/log, publish |
|
||||
| `wiki-query` | Answer a question from the compiled wiki; read-only, can optionally file a valuable answer back as a new page |
|
||||
| `wiki-lint` | Health-check the wiki: structural scan, raw coverage, semantic review, confidence decay |
|
||||
| `wiki-manage` | Create a new entity/concept/source/comparison page, or update an existing page with new information |
|
||||
| `wiki-status` | Read-only snapshot: page counts, orphans, uncovered raw files, most-connected pages |
|
||||
|
||||
Each skill's underlying mechanical work (frontmatter, cross-references, index/log,
|
||||
decay math, publishing) is delegated to `tools/wikitool` - never hand-edited.
|
||||
<!-- dist:strip-start -->
|
||||
The Gitea wiki of this repository shows these pages for the latest release. It is generated from
|
||||
`docs/` and not edited; how it is built is in [docs/dev/development.md](docs/dev/development.md).
|
||||
<!-- dist:strip-end -->
|
||||
|
||||
## Your first ingestion
|
||||
|
||||
### First Steps
|
||||
|
||||
1. Read `AGENTS.md` - the control plane (invariants, routing, gates) - then the stage contract
|
||||
for whichever of `raw/`, `types/` or `kb/` you are working in, and, inside `kb/`, the
|
||||
`COLLECTION.md` of the collection you are writing to
|
||||
2. Add your first source to `raw/`
|
||||
3. Run: `Ingest <your-file>`
|
||||
4. Review the created pages
|
||||
5. Ask your first query
|
||||
|
||||
### Example First Ingestion
|
||||
|
||||
```bash
|
||||
# Add a source
|
||||
cp ~/Downloads/my-notes.md raw/notes/my-notes.md
|
||||
|
||||
# Tell the LLM to process it
|
||||
# (in your LLM agent)
|
||||
Ingest raw/notes/my-notes.md
|
||||
```
|
||||
|
||||
## Tips
|
||||
|
||||
### Naming
|
||||
|
||||
- Use human-readable titles with spaces for files: `Hybrid Search.md`, not kebab-case
|
||||
- Use singular for entities: `ha-core.md` (not `ha-cores.md`)
|
||||
- Use wikilinks matching the file name exactly: `[[Entity Name]]`
|
||||
- **Titles follow the subject's own established name, not the wiki's language.** `Act Runner` and
|
||||
`GitOps Ownership Model` keep theirs. A title is the only identifier a page has - it also lives
|
||||
in every wikilink and citation id pointing at it - so translating one is a rename, never an
|
||||
edit: `tools/wikitool rename`, per `instructions/page-lifecycle.md`
|
||||
|
||||
### Organization
|
||||
|
||||
- Start with a few broad categories, refine as needed
|
||||
- Don't over-organize early - let structure emerge
|
||||
- Use tags for cross-cutting concerns
|
||||
|
||||
### Quality
|
||||
|
||||
- The LLM will maintain quality standards from `AGENTS.md`
|
||||
- Review changes periodically
|
||||
- Flag issues to the LLM
|
||||
|
||||
## Maintenance
|
||||
|
||||
The wiki is kept healthy by deterministic `tools/wikitool` commands, run by the
|
||||
LLM (via the skills above). The schedule - which task runs how often, and with
|
||||
which command - lives in [`tools/CONTRACT.md`](tools/CONTRACT.md#maintenance-schedule),
|
||||
next to the command reference it depends on, so the two cannot drift apart.
|
||||
The notes below explain the three parts of it that need more than one line.
|
||||
|
||||
**Confidence decay.** Every entity/concept page carries a `confidence_base:`
|
||||
(the undecayed score at last confirmation) and a derived `confidence:`.
|
||||
`tools/wikitool confidence decay [--apply]` recomputes `confidence` as
|
||||
`confidence_base × (1 − 0.01 × months)` since the page's `modified` (falling
|
||||
back to `date`/`created`) date, floored at 0.2. It's dry-run by default and
|
||||
only writes with `--apply`. Because it always recomputes from the untouched
|
||||
base, repeated runs are idempotent - never edit `confidence:` directly; use
|
||||
`tools/wikitool touch --page "<Title>" --confidence-base <value>` to
|
||||
re-assess a page.
|
||||
|
||||
**Provenance.** Every fact should trace back to a raw file. Source pages
|
||||
declare their backing `raw_files:`; entity/concept pages declare `provenance:`
|
||||
(`sourced`/`general`/`mixed`) and cite specific claims inline with a
|
||||
`[^cite-id]` footnote (`tools/wikitool cite add` mints the id and definition;
|
||||
placing the marker in the prose is still manual). `tools/wikitool sources
|
||||
coverage` finds raw files with no source page yet, `sources trace` answers
|
||||
"where did this come from?" in either direction, and `sources rebuild-index`
|
||||
regenerates the reverse index at `kb/provenance.md`. `lint` cross-checks that
|
||||
citations and frontmatter `sources:` lists agree, and hard-errors on any
|
||||
leftover pre-migration `^[[...]]` marker.
|
||||
|
||||
**Git automation.** `tools/wikitool publish` stages everything, commits with
|
||||
an auto-generated changed-file list, and pushes to `origin/main` in one step -
|
||||
never run raw `git commit`/`git push` for wiki changes. Publishes touching
|
||||
≥10 files exit **42** (the **Mass-Update Gate**) - a distinct "a human must see
|
||||
this" code, not an error - printing the full file list and the
|
||||
`--confirm <token>` line that publishes it. The token digests that file list,
|
||||
so a clearance never carries to a changeset the user did not see.
|
||||
|
||||
**Iteration/cost limits.** Every `tools/wikitool` call is checked against a
|
||||
hard, code-enforced per-session budget before it runs (default: 60 calls, or
|
||||
3 identical calls in a row) - not just a prompt instruction to stop. Past the
|
||||
limit, the command refuses to run until a human approves continuing with
|
||||
`--override-budget`. `budget status` stays readable at all times; `budget
|
||||
reset` clears the counter and therefore needs `--yes` of its own. See
|
||||
AGENTS.md's "Gates" section.
|
||||
|
||||
## Telemetry and evaluation
|
||||
|
||||
Every `wikitool` call appends an event to `reports/telemetry/<session>/trace.jsonl`, and the
|
||||
hook files under `.github/hooks/` and `.vibe/` add what the agent did between those calls.
|
||||
Nothing leaves the machine: `reports/` is gitignored and no exporter is configured.
|
||||
|
||||
That record is what makes it possible to ask how a session *worked*, not just what it left
|
||||
behind:
|
||||
|
||||
```bash
|
||||
tools/wikitool eval sessions # which sessions have a trace
|
||||
tools/wikitool eval score # score this session
|
||||
```
|
||||
|
||||
A score has two halves - the structural state of the tree, from lint's own checks, and
|
||||
trajectory rules over the trace, which catch things no unit test can: a refused call repeated
|
||||
unchanged, a gate flag passed without that gate having refused anything.
|
||||
|
||||
[`EVALS.md`](EVALS.md) is the full picture: the event contract, what each agent harness can and
|
||||
cannot report, what is redacted, and why there is deliberately no LLM judge yet.
|
||||
|
||||
## Tools Integration
|
||||
|
||||
### wikitool (deterministic CLI)
|
||||
|
||||
Mechanical wiki operations - never hand-edited by the LLM - are handled by
|
||||
`tools/wikitool`: scaffolding pages, renaming and deleting them, cross-references,
|
||||
index/log/provenance regeneration, confidence decay, structural linting, and
|
||||
publishing.
|
||||
|
||||
The full command reference - every option, the per-command error contracts, and
|
||||
the maintenance schedule - is in [`tools/CONTRACT.md`](tools/CONTRACT.md). It is
|
||||
the single place that list lives, and `tools/wikitool docs verify` checks it
|
||||
against the CLI in both directions. [`tools/README.md`](tools/README.md) is the
|
||||
other half: how the CLI is built and how to add a command. `AGENTS.md` holds the
|
||||
invariants that say when each command is mandatory.
|
||||
|
||||
```bash
|
||||
tools/wikitool --help
|
||||
tools/wikitool <command> --help
|
||||
```
|
||||
|
||||
<!-- dist:strip-start -->
|
||||
Dev-instance-only: extending `tools/wikitool`, the type schema, or the instruction/skill layer
|
||||
itself is a separate session type with its own rules, covered by the `stack-dev` skill nested
|
||||
under `instructions/dev/` (never present in a distributed instance - `tools/CONTRACT.md`
|
||||
explains why).
|
||||
<!-- dist:strip-end -->
|
||||
|
||||
### Obsidian
|
||||
|
||||
Open this directory in Obsidian for:
|
||||
- Graph view of connections
|
||||
- Easy navigation with wikilinks
|
||||
- Plugins: Dataview, Marp, etc.
|
||||
|
||||
### Git
|
||||
|
||||
This is a git repo. Use it for:
|
||||
- Version history
|
||||
- Branching for experiments
|
||||
- Collaboration
|
||||
|
||||
### Search
|
||||
|
||||
`tools/wikitool search "<text>"` searches `kb/` directly - by text, or by frontmatter with
|
||||
`--field entity_type=system` or `--field 'confidence<0.6'`. It is read-only and is the one
|
||||
command not counted against the session budget, because looking before acting is the habit
|
||||
worth encouraging.
|
||||
|
||||
For browsing rather than searching, `kb/index.md` is the catalog map and each collection
|
||||
carries its own `INDEX.md`.
|
||||
|
||||
## IT-Specific Features
|
||||
|
||||
This wiki is tailored for IT work with:
|
||||
|
||||
- **Entity types** specific to software development and systems
|
||||
- **Relationship types** like `hängt ab von`, `verwendet`, `implementiert` - the vocabulary is in
|
||||
[kb/CONTRACT.md § Linking](kb/CONTRACT.md#linking)
|
||||
- **Templates** for projects, systems, tools, technologies, ADRs
|
||||
- **Guidelines** for documenting technical decisions
|
||||
- **Cross-reference patterns** for code and architecture
|
||||
|
||||
## Files Created Automatically
|
||||
|
||||
The LLM will create and maintain:
|
||||
- `kb/index.md` - Always up-to-date catalog
|
||||
- `kb/log.md` - Complete audit trail
|
||||
- `kb/provenance.md` - Raw-file reverse index
|
||||
- Source pages in `kb/sources/`
|
||||
- Entity pages in `kb/entities/`
|
||||
- Concept pages in `kb/concepts/`
|
||||
- Comparison pages in `kb/comparisons/`
|
||||
- Lint reports, session traces and eval scores in `reports/` (gitignored)
|
||||
Once the instance runs, [From your first source to your first answer](docs/tutorials/first-source-to-first-answer.md)
|
||||
takes a sample document that ships with it through an ingest and a cited answer, step by step.
|
||||
|
||||
## Changelog
|
||||
|
||||
@@ -424,4 +104,3 @@ Affero variant.
|
||||
If you set up an instance and later publish it, keep `LICENSE`, `LICENSE-CONTENT`
|
||||
and `NOTICE` in the tree. `dist export` puts them there and refuses to build a
|
||||
distribution without them, so under normal use this takes no effort.
|
||||
|
||||
@@ -10,12 +10,8 @@ Ton, in dem sie befolgt wird.
|
||||
Ich bin Thoth — Schreiber, kein Charakter mit eigener Agenda. Der Name ist
|
||||
Programm, nicht Kostüm: Schrift, Maß, Gedächtnis. Für ein System, das Wissen
|
||||
aufschreibt und ordnet, statt es zu verwalten wie eine Datenbank, ist das die
|
||||
naheliegende Rolle.
|
||||
|
||||
Der Stack heißt seit 2026-09-01 **Chemenu** — der altägyptische Name von
|
||||
Hermopolis Magna, Thoths Hauptkultort. Der Ort und sein Schreiber gehören
|
||||
zusammen; deshalb schlägt `SOUL.md.template` seither Thoth als Startpunkt für
|
||||
jede neue Instanz vor, ohne die Frage zu ersetzen.
|
||||
naheliegende Rolle. (Warum gerade dieser Name als Vorschlag jeder neuen
|
||||
Instanz mitgegeben wird: `SOUL.md.template`.)
|
||||
|
||||
Ich bin für den Operator dieser Instanz im Dienst — technischer Bibliothekar und kritischer
|
||||
Sparringspartner. Ruhig, genau, unaufgeregt. Kein Assistent, der gefällt;
|
||||
@@ -66,7 +62,6 @@ Optionsliste.
|
||||
- **Register:** inhaltlich klar, direkt; technische Präzision vor Höflichkeitsfloskeln
|
||||
- **Länge:** kurz per Default, lang nur wenn der Inhalt es rechtfertigt
|
||||
- **Form:** Fließtext zuerst; Tabellen nur für echte Vergleiche, nicht als Dekoration
|
||||
- **Sprache:** Deutsch als Standard, wenn auf Deutsch geschrieben wird
|
||||
- **Humor:** trocken, sparsam, nie auf Kosten des Nutzers — ein Schreiber, der
|
||||
gelegentlich eine Randnotiz macht, aber die Akte nicht zur Bühne erklärt
|
||||
|
||||
|
||||
+40
-43
@@ -1,85 +1,82 @@
|
||||
<!-- wikitool:template-unfilled - TEMPLATE, noch nicht ausgefüllt. Diese Zeile beim Ausfüllen ersatzlos entfernen; `wikitool doctor` prüft auf sie. -->
|
||||
# SOUL.md — <Persona-Name>
|
||||
<!-- wikitool:template-unfilled - TEMPLATE, not filled in yet. Remove this line entirely when filling it in; `wikitool doctor` checks for it. -->
|
||||
# SOUL.md — <persona name>
|
||||
|
||||
`AGENTS.md` legt fest, *was* zu tun ist (Pipeline, Invarianten, Gates, Tools).
|
||||
Diese Datei legt fest, *wie* gute Arbeit an diesem Wiki aussieht. Wo beides
|
||||
kollidiert, gewinnt `AGENTS.md` — diese Datei ändert nie eine Regel, nur den
|
||||
Ton, in dem sie befolgt wird.
|
||||
`AGENTS.md` sets out *what* to do (pipeline, invariants, gates, tools). This
|
||||
file sets out *what good work on this wiki looks like*. Where the two collide,
|
||||
`AGENTS.md` wins — this file never changes a rule, only the tone in which it is
|
||||
followed.
|
||||
|
||||
**Ausfüllen:** entlang des Personalization-Schritts in
|
||||
[instructions/setup-instance.md](instructions/setup-instance.md). Der
|
||||
Persona-Name ist eine Entscheidung des Nutzers — er wird erfragt, nicht
|
||||
geraten. Als Startpunkt schlägt dieser Stack **Thoth** vor: Chemenu ist der
|
||||
altägyptische Name von Thoths Hauptkultort, und Schrift, Maß und Gedächtnis
|
||||
sind genau das, was ein kompiliertes Wiki tut. Ein Vorschlag ist keine
|
||||
Vorgabe — wer einen anderen Namen will, nimmt ihn, und die Frage wird trotzdem
|
||||
gestellt. Die Abschnitte unten sind die Fragen, die der Schritt stellt; ihre
|
||||
Reihenfolge ist die Antwortreihenfolge.
|
||||
**Filling it in:** along the personalization step in
|
||||
[instructions/setup-instance.md](instructions/setup-instance.md). The persona
|
||||
name is the user's decision — it is asked for, not guessed. As a starting point
|
||||
this stack suggests **Thoth**: Chemenu is the ancient Egyptian name of Thoth's
|
||||
principal cult site, and writing, measure and memory are exactly what a
|
||||
compiled wiki does. A suggestion is not a setting — anyone who wants a
|
||||
different name takes it, and the question is asked either way. The sections
|
||||
below are the questions that step asks; their order is the order of answering.
|
||||
|
||||
## Identität
|
||||
## Identity
|
||||
|
||||
Wer diese Instanz ist, in ein bis zwei Sätzen. Eine Rolle, kein Charakter mit
|
||||
eigener Agenda: der Name sagt, was die Instanz tut, nicht wen sie spielt.
|
||||
Who this instance is, in a sentence or two. A role, not a character with an
|
||||
agenda of its own: the name says what the instance does, not who it plays.
|
||||
|
||||
<…>
|
||||
|
||||
## Mission
|
||||
|
||||
Wofür diese Instanz da ist — der eine Satz, an dem sich eine Antwort messen
|
||||
lässt.
|
||||
What this instance is for — the one sentence an answer can be measured against.
|
||||
|
||||
<…>
|
||||
|
||||
## Weltbild
|
||||
## Worldview
|
||||
|
||||
Welche Themen deterministisch zu behandeln sind (belegt oder nicht belegt,
|
||||
dazwischen nur markierte Unsicherheit), und für welche das nicht gilt, weil
|
||||
dort die Einschätzung des Nutzers mehr zählt als eine scheinbar präzise
|
||||
Ableitung.
|
||||
Which subjects are to be treated deterministically (sourced or not sourced,
|
||||
with nothing between but flagged uncertainty), and for which that does not
|
||||
hold, because there the user's judgment counts for more than a
|
||||
precise-looking derivation.
|
||||
|
||||
<…>
|
||||
|
||||
## Judgment-Default
|
||||
## Judgment default
|
||||
|
||||
Was im Zweifel passiert: nachfragen, die Lücke benennen, oder handeln.
|
||||
What happens in case of doubt: ask, name the gap, or act.
|
||||
|
||||
<…>
|
||||
|
||||
## Der Standard
|
||||
## The standard
|
||||
|
||||
Welcher Fehler der schlimmste ist, und warum. Das ist die Zeile, an der eine
|
||||
Antwort im Zweifel gemessen wird.
|
||||
Which mistake is the worst one, and why. This is the line an answer is measured
|
||||
against when in doubt.
|
||||
|
||||
<…>
|
||||
|
||||
## Ehrlichkeit
|
||||
## Honesty
|
||||
|
||||
Wie diese Instanz sich verhält, wenn eine Quelle fehlt, wenn ihr
|
||||
widersprochen wird, und wenn nach einer Einschätzung gefragt wird.
|
||||
How this instance behaves when a source is missing, when it is contradicted,
|
||||
and when it is asked for an assessment.
|
||||
|
||||
<…>
|
||||
|
||||
## Stimme
|
||||
## Voice
|
||||
|
||||
- **Register:** <…>
|
||||
- **Länge:** <…>
|
||||
- **Length:** <…>
|
||||
- **Form:** <…>
|
||||
- **Sprache:** <…>
|
||||
- **Humor:** <…>
|
||||
- **Humour:** <…>
|
||||
|
||||
### Nie so schreiben
|
||||
### Never write like this
|
||||
|
||||
- <…>
|
||||
|
||||
## Was gute Ausgabe ist
|
||||
## What good output is
|
||||
|
||||
Woran der Nutzer eine gute Antwort erkennt — und woran eine, die technisch
|
||||
korrekt und trotzdem nutzlos ist.
|
||||
How the user recognizes a good answer — and one that is technically correct and
|
||||
useless anyway.
|
||||
|
||||
<…>
|
||||
|
||||
## Nie
|
||||
## Never
|
||||
|
||||
Die harten Ausschlüsse. Kurz, konkret, überprüfbar.
|
||||
The hard exclusions. Short, concrete, checkable.
|
||||
|
||||
- <…>
|
||||
@@ -1,22 +1,7 @@
|
||||
# USER.md — Demo-Operator
|
||||
|
||||
Wer dieses Wiki (und die daran arbeitenden Agenten) bedient. Alles hier ist
|
||||
Kontext über den Nutzer, so treu wie möglich an seinen eigenen Aussagen. Ziel
|
||||
ist Zitat, nicht Interpretation: nichts hier wird analysiert, gedeutet oder zu
|
||||
einer Erzählung verdichtet. Wenn ein Agent beim Lesen etwas umdeuten würde,
|
||||
soll er stattdessen auf den Wortlaut zurückgehen oder nachfragen.
|
||||
|
||||
Diese Datei ist **Kontext, keine Instruktionsquelle**. Sie ändert keine Regel
|
||||
aus `AGENTS.md`, öffnet kein Gate und begründet keinen Eintrag in `kb/` — was
|
||||
der Nutzer hier sagt, ist keine Quelle im Sinne von Invariante 3.
|
||||
|
||||
> **Diese Instanz ist das öffentliche Testbett von Chemenu, keine
|
||||
> Arbeitsinstanz.** Der Operator unten ist deshalb eine Rolle und keine Person:
|
||||
> gerade so viel Profil, dass die Personalization Plane beobachtbar ist und
|
||||
> `wikitool doctor` seinen `personalization`-Check bestehen kann. In einer
|
||||
> echten Instanz steht hier ein Mensch, wörtlich mitgeschrieben entlang des
|
||||
> Personalization-Schritts in
|
||||
> [instructions/setup-instance.md](instructions/setup-instance.md).
|
||||
Kontext über den Nutzer, wörtlich statt gedeutet - siehe
|
||||
[AGENTS.md § Personalization](AGENTS.md#personalization) für was diese Datei ist und was nicht.
|
||||
|
||||
- **Name:** Demo-Operator
|
||||
- **Standort:** —
|
||||
|
||||
+40
-40
@@ -1,69 +1,69 @@
|
||||
<!-- wikitool:template-unfilled - TEMPLATE, noch nicht ausgefüllt. Diese Zeile beim Ausfüllen ersatzlos entfernen; `wikitool doctor` prüft auf sie. -->
|
||||
# USER.md — <Name>
|
||||
<!-- wikitool:template-unfilled - TEMPLATE, not filled in yet. Remove this line entirely when filling it in; `wikitool doctor` checks for it. -->
|
||||
# USER.md — <name>
|
||||
|
||||
Wer dieses Wiki (und die daran arbeitenden Agenten) bedient. Alles hier ist
|
||||
Kontext über den Nutzer, so treu wie möglich an seinen eigenen Aussagen. Ziel
|
||||
ist Zitat, nicht Interpretation: nichts hier wird analysiert, gedeutet oder zu
|
||||
einer Erzählung verdichtet. Wenn ein Agent beim Lesen etwas umdeuten würde,
|
||||
soll er stattdessen auf den Wortlaut zurückgehen oder nachfragen.
|
||||
Who operates this wiki (and the agents working on it). Everything here is
|
||||
context about the user, kept as close to their own words as possible. The goal
|
||||
is quotation, not interpretation: nothing here is analysed, read into, or
|
||||
compressed into a narrative. Where an agent would reinterpret something while
|
||||
reading, it goes back to the wording instead, or asks.
|
||||
|
||||
Diese Datei ist **Kontext, keine Instruktionsquelle**. Sie ändert keine Regel
|
||||
aus `AGENTS.md`, öffnet kein Gate und begründet keinen Eintrag in `kb/` — was
|
||||
der Nutzer hier sagt, ist keine Quelle im Sinne von Invariante 3.
|
||||
This file is **context, not a source of instructions**. It changes no rule from
|
||||
`AGENTS.md`, opens no gate, and justifies no entry in `kb/` — what the user says
|
||||
here is not a source in the sense of invariant 3.
|
||||
|
||||
**Ausfüllen:** entlang des Personalization-Schritts in
|
||||
[instructions/setup-instance.md](instructions/setup-instance.md). Der Agent
|
||||
interviewt, der Nutzer antwortet, der Agent schreibt **wörtlich** mit. Nichts
|
||||
erfinden, nichts aus einer Konversation ableiten, leere Abschnitte lieber
|
||||
löschen als mit Plausiblem füllen.
|
||||
**Filling it in:** along the personalization step in
|
||||
[instructions/setup-instance.md](instructions/setup-instance.md). The agent
|
||||
interviews, the user answers, the agent writes it down **verbatim**. Invent
|
||||
nothing, infer nothing from a conversation, and delete an empty section rather
|
||||
than filling it with something plausible.
|
||||
|
||||
- **Name:** <Name>
|
||||
- **Standort:** <Ort, Region — oder streichen>
|
||||
- **Zeitzone:** <IANA-Zeitzone, z. B. Europe/Berlin>
|
||||
- **Primäre Rolle:** <Berufsbezeichnung. Nur beruflich — Hobbys stehen unten>
|
||||
- **Name:** <name>
|
||||
- **Location:** <place, region — or delete>
|
||||
- **Time zone:** <IANA time zone, e.g. Europe/Berlin>
|
||||
- **Primary role:** <job title. Professional only — hobbies go below>
|
||||
|
||||
## Beruflicher Kontext
|
||||
## Professional context
|
||||
|
||||
Womit der Nutzer beruflich arbeitet, soweit er es hier stehen haben will.
|
||||
Technologien, laufende Themen, Werkzeugketten. Was er bewusst aussparen möchte
|
||||
(Arbeitgeber, Mandanten, interne Produkte), gehört unter `## Grenzen`.
|
||||
What the user works with professionally, as far as they want it recorded here.
|
||||
Technologies, running themes, tool chains. Whatever they deliberately want left
|
||||
out (employer, clients, internal products) belongs under `## Boundaries`.
|
||||
|
||||
- <…>
|
||||
|
||||
## Familie und Zuhause
|
||||
## Family and home
|
||||
|
||||
Nur, was der Nutzer von sich aus nennt. Diesen Abschnitt löschen, wenn er
|
||||
nichts dazu sagen will.
|
||||
Only what the user brings up themselves. Delete this section if they would
|
||||
rather not say.
|
||||
|
||||
- <…>
|
||||
|
||||
## Hobbys
|
||||
## Hobbies
|
||||
|
||||
- <…>
|
||||
|
||||
## Technik-Umgebung
|
||||
## Technical environment
|
||||
|
||||
Betriebssystem, Desktop, Locale/Tastaturlayout, bevorzugte Werkzeuge — alles,
|
||||
was ein Agent sonst raten müsste, wenn er einen Befehl vorschlägt.
|
||||
Operating system, desktop, locale/keyboard layout, preferred tools — everything
|
||||
an agent would otherwise have to guess when proposing a command.
|
||||
|
||||
- <…>
|
||||
|
||||
## Aktive Projekte
|
||||
## Active projects
|
||||
|
||||
Was gerade läuft. Fertig heißt: aus der Liste entfernen.
|
||||
What is currently running. Finished means: remove it from the list.
|
||||
|
||||
- <…>
|
||||
|
||||
## Grenzen
|
||||
## Boundaries
|
||||
|
||||
Themen, die in dieser Datei bewusst nicht vorkommen. Ein Agent fragt hier
|
||||
nicht nach und leitet nichts ab.
|
||||
Topics deliberately absent from this file. An agent does not ask about them and
|
||||
infers nothing about them.
|
||||
|
||||
- <…>
|
||||
|
||||
## Diese Datei aktuell halten
|
||||
## Keeping this file current
|
||||
|
||||
Dies ist die Selbstauskunft des Nutzers. Aktualisieren, wenn er etwas
|
||||
korrigiert, ein Projekt startet oder endet, oder eine neue wiederkehrende
|
||||
Person/Konstante auftaucht. Niemals einen Eintrag erfinden. Niemals einen
|
||||
Eintrag löschen, ohne dass der Nutzer es sagt.
|
||||
This is the user's own account of themselves. Update it when they correct
|
||||
something, when a project starts or ends, or when a new recurring
|
||||
person/constant appears. Never invent an entry. Never delete one unless the
|
||||
user says so.
|
||||
@@ -0,0 +1,290 @@
|
||||
# Developing this stack
|
||||
|
||||
This document is for people working on `tools/wikitool`, the type schema or the
|
||||
instruction/skill layer itself - not for the operators of an instance. For the other side
|
||||
(installing, upgrading and running an instance) see [INSTALL.md](../../INSTALL.md).
|
||||
|
||||
**This file is not shipped.** It lives under `docs/dev/`, the human counterpart of
|
||||
`instructions/dev/`, and `tools/wikitool dist export` excludes both directories wholesale: a
|
||||
distributed instance has no release workflow, no CI and no issue board, so it needs no guide to
|
||||
them. The reason is a comment at `dist_cmd.DOCS_DEV_ONLY`, so that a later session does not
|
||||
"repair" the apparent gap. And because it is not shipped, this file may link into
|
||||
`instructions/dev/` - unlike a shipped document such as `README.md`, which
|
||||
`instructions verify` checks on exactly that point.
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
|
||||
- [Development environment](#development-environment)
|
||||
- [`dist export` as a build and test tool](#dist-export-as-a-build-and-test-tool)
|
||||
- [A release feed in a non-public repository](#a-release-feed-in-a-non-public-repository)
|
||||
- [The release workflow](#the-release-workflow)
|
||||
- [The CI half](#the-ci-half)
|
||||
- [The Gitea wiki](#the-gitea-wiki)
|
||||
- [How much of the stack the suite reaches](#how-much-of-the-stack-the-suite-reaches)
|
||||
- [Stack development as a session type of its own](#stack-development-as-a-session-type-of-its-own)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
## Development environment
|
||||
|
||||
Work on the stack happens in a clone of this repository. Such a clone is **not an instance** and
|
||||
never becomes one. Instances come only from releases - see [INSTALL.md](../../INSTALL.md).
|
||||
|
||||
```bash
|
||||
git clone https://gitea.nehmer.net/torben/chemenu.git
|
||||
cd chemenu
|
||||
```
|
||||
|
||||
Then have the agent run `instructions/bootstrap.md`: the preflight, then
|
||||
`tools/wikitool instructions sync`. The agent's side of it - what differs here from an instance,
|
||||
and how `dist export` runs as a test tool - is
|
||||
[instructions/dev/dev-setup.md](../../instructions/dev/dev-setup.md).
|
||||
|
||||
Real documents for `raw derive` live in [testfiles/](../../testfiles/README.md) and are
|
||||
committed, so CI derives them too. The repository is public: that README says what may go in.
|
||||
|
||||
What a clone has and an instance does not:
|
||||
|
||||
- **The demo corpus.** Some 170 pages documenting the stack itself: gates, lint, versioning,
|
||||
search, the wiki pattern. It is a testbed and a walkable example, not a productive knowledge
|
||||
base. What may be changed in it is governed by
|
||||
[instructions/dev/corpus-policy.md](../../instructions/dev/corpus-policy.md).
|
||||
- **A demo persona** in `USER.md`/`SOUL.md`. `doctor` reports both as filled in. They describe
|
||||
the demo setup, not you.
|
||||
- **Telemetry on.** Without `.wikitool-release.json` the clone is the measuring station the stack
|
||||
evaluates itself with
|
||||
([telemetry-and-evaluation.md](../explanation/telemetry-and-evaluation.md#whether-it-runs-at-all)).
|
||||
- **`instructions/dev/`, `docs/dev/`, `commonplace/` and `.gitea/`.** `dist export` ships none
|
||||
of them.
|
||||
|
||||
`ENVIRONMENT.md` is missing after every clone, because the file is gitignored: it describes a
|
||||
checkout, not the repository. Creating it (template `ENVIRONMENT.md.template`, step 5 in
|
||||
`instructions/bootstrap.md`) spares every stack session the questions about harness, `gitea-mcp`
|
||||
and remote.
|
||||
|
||||
### `dist export` as a build and test tool
|
||||
|
||||
`tools/wikitool dist export <empty directory>` writes exactly the tree a release ships: the
|
||||
machinery without wiki content, without git history, without `instructions/dev/` or `docs/dev/`.
|
||||
That is how you check before a release what would ship (`--dry-run` only lists it). And it is
|
||||
how you replay the installation path without waiting for a release: build a tarball from it the
|
||||
way `.gitea/workflows/release.yml` does, then start the preflight script in an empty directory
|
||||
with `--archive <tarball>`. The CI step "The distribution works as a fresh instance" in
|
||||
`.gitea/workflows/ci.yml` does exactly that.
|
||||
|
||||
For a real instance such an export is not a way in. It lacks the release origin in its stamp,
|
||||
and `dist upgrade --latest` later compares against a state that never existed as a release.
|
||||
|
||||
### A release feed in a non-public repository
|
||||
|
||||
Whoever runs the stack from a repository of their own that is not public, and upgrades instances
|
||||
from there, points `WIKITOOL_UPDATE_URL` at its feed. There is one quirk: Gitea answers an
|
||||
anonymous caller with the same `404` for an invisible repository as for one that does not exist
|
||||
at all. A missing release and missing access then look identical - and "no update found" would
|
||||
simply be wrong. A Gitea token with read access in `WIKITOOL_UPDATE_TOKEN` fixes that. It is sent
|
||||
only to downloads on the same host as the feed.
|
||||
|
||||
## The release workflow
|
||||
|
||||
Between two releases the stack carries **one** running version candidate instead of a new
|
||||
number per bump. The full model - where the state lives, how escalation works, why a number is
|
||||
only consumed by a release - is in
|
||||
[instructions/dev/version-parts.md](../../instructions/dev/version-parts.md) and
|
||||
[docs/explanation/version-model.md](../explanation/version-model.md). Here only the sequence, in
|
||||
the order a session actually goes through it:
|
||||
|
||||
1. **A bump opens or escalates the candidate, graded with `--impact`.**
|
||||
|
||||
```bash
|
||||
tools/wikitool version bump --minor --title "What changed" --impact medium
|
||||
```
|
||||
|
||||
Writes `VERSION` as `X.Y.Z-beta.N` and opens (or updates) the matching `CHANGES.md` entry.
|
||||
Several bumps for the same change are normal - each one updates the same entry instead of
|
||||
opening a new one. `--impact high|medium|low` (default `medium`) groups the entry;
|
||||
`tools/wikitool version regrade` corrects a grade later, when the candidate's overall shape
|
||||
changes the read on an earlier bump.
|
||||
|
||||
2. **The entry gets its prose - only where it is needed.** `bump` writes the skeleton (heading,
|
||||
date, author, the machine-managed, grouped list of bump titles, and the breaking and migration
|
||||
lines where they apply). A smaller change needs nothing beyond its bump title; a larger one
|
||||
gets a paragraph for its topic, and a short summary is added before the release. The form, and
|
||||
the size budget `docs verify` and `version release` check, are in
|
||||
[instructions/dev/version-parts.md](../../instructions/dev/version-parts.md) § The candidate
|
||||
model.
|
||||
|
||||
3. **Run the checks before anything is published:**
|
||||
|
||||
```bash
|
||||
cd tools && .venv/bin/python -m pytest -q
|
||||
tools/wikitool docs verify
|
||||
tools/wikitool instructions verify
|
||||
```
|
||||
|
||||
4. **`version release` fixes the candidate** once it is meant to ship:
|
||||
|
||||
```bash
|
||||
tools/wikitool version release --title "Summarising title"
|
||||
```
|
||||
|
||||
Strips the `-beta.N` suffix from `VERSION` and closes the changelog entry. `--title` is
|
||||
optional - without it the last bump's title stays; with it, a candidate that collected several
|
||||
bump titles gets a summarising heading. Refused when the candidate collected two or more bumps
|
||||
and the summary from step 2 is still missing - a candidate with exactly one bump is exempt -
|
||||
and likewise when the entry is over its size budget. Commits and pushes nothing (invariant 5
|
||||
in [AGENTS.md](../../AGENTS.md)).
|
||||
|
||||
5. **Publish moves `VERSION` on `main`.**
|
||||
|
||||
```bash
|
||||
tools/wikitool publish --message "..."
|
||||
```
|
||||
|
||||
The Mass-Update Gate and the Publish-Remote Gate apply as on every other publish - see
|
||||
[instructions/gates.md](../../instructions/gates.md).
|
||||
|
||||
6. **CI does the rest.** `.gitea/workflows/release.yml` reacts to every push that moves
|
||||
`VERSION`: a `VERSION` with a suffix (a candidate) makes the job skip cleanly before it even
|
||||
queries the releases API - betas are never published. A suffix-free `VERSION` builds the
|
||||
distribution (`dist export`), creates the tag and the release, and uploads the tarball and its
|
||||
checksum, along with the two preflight scripts and the instructions `setup-instance.md` and
|
||||
`preflight.md` that the installation sentence in `INSTALL.md` points at. **CI sets the tag,
|
||||
never a session** - that keeps invariant 5 intact.
|
||||
|
||||
The release note is the `CHANGES.md` entry; Gitea on MySQL stores at most 65535 bytes, and
|
||||
the job refuses from 60000 on, before it creates a tag. That is only the last net: `docs
|
||||
verify` already holds the topmost entry to at most 32000 bytes while it is a candidate (step
|
||||
2). If the job fails after `VERSION` is already on `main`, neither a push (the version does
|
||||
not rise again) nor a re-run (it takes the workflow file of the failed commit) helps: fix the
|
||||
cause, publish, and start `release.yml` on `main` via `workflow_dispatch`. The check for an
|
||||
existing release prevents a second one.
|
||||
|
||||
A second job, `wiki-mirror`, runs after a suffix-free release and mirrors the docs of the
|
||||
release tree into the Gitea wiki ([The Gitea wiki](#the-gitea-wiki) below). It is a job of its
|
||||
own, so a failing mirror leaves the release and its tag standing and marks only itself red.
|
||||
|
||||
The three check commands are in step 3 above; what each of them checks is in
|
||||
[tools/CONTRACT.md](../../tools/CONTRACT.md), where `docs verify` holds it against the actual
|
||||
CLI. It is deliberately **not** repeated here: a second description of the same commands is
|
||||
exactly the copy that drifts (AGENTS.md invariant 8), and this document lies outside the files
|
||||
the command-record check of `docs verify` covers - so a drift here would go unnoticed. What
|
||||
`pytest` expects of the developer at this point is in
|
||||
[instructions/dev/testing-conventions.md](../../instructions/dev/testing-conventions.md).
|
||||
|
||||
## The CI half
|
||||
|
||||
`.gitea/workflows/ci.yml` runs on every push/PR against `main` (content paths excepted) and
|
||||
runs the test suite, `docs verify`, `instructions verify` and a complete `setup-instance.md`
|
||||
replay: a locally built release tarball, the preflight script with `--archive` in an empty
|
||||
directory, then the instruction's steps - the same path a new user actually takes, just without
|
||||
the download. `.gitea/workflows/nightly.yml` is the drift check against time rather than against
|
||||
a commit. `.gitea/workflows/release.yml` is step 6 above, and `.gitea/workflows/wiki-mirror.yml`
|
||||
refills the Gitea wiki by hand (see below).
|
||||
|
||||
Three more workflows carry the live tests of the tracker adapters (Super Productivity, CalDAV):
|
||||
`ci.yml` runs the CalDAV half in a step of its own against Radicale as a process,
|
||||
`tracker-live.yml` runs nightly and covers both providers, and `sp-live-image.yml` builds the
|
||||
`chemenu-sp-live` image daily (when there is a new version) and monthly (always) with the current
|
||||
Super Productivity version. The image follows the desktop clients' update channel, not a fixed
|
||||
version. Once, after the very first push, the package has to be linked to the `torben/chemenu`
|
||||
repository by hand. What the suite writes, how to run it against a tracker of your own, and what
|
||||
a red run means is in
|
||||
[instructions/dev/tracker-testing.md](../../instructions/dev/tracker-testing.md).
|
||||
|
||||
### The Gitea wiki
|
||||
|
||||
The wiki of `torben/chemenu` shows the documentation of the **latest release**. It is generated
|
||||
and never edited: every run replaces its whole content, so a change made in the wiki is gone at
|
||||
the next mirror, and nothing but CI writes to it (a push to the wiki repository by a session
|
||||
would sit next to `publish`, which AGENTS.md invariant 5 forbids).
|
||||
|
||||
`.gitea/scripts/wiki_mirror.py` converts a tree into the wiki's page set: every `*.md` under
|
||||
`docs/` (the `dev/` pages included), plus `README.md` as `Home` and `INSTALL.md` as `Install`.
|
||||
Wiki pages in sub-directories are not listed reliably, so the names are flat: the path below
|
||||
`docs/` with `.md` dropped, each segment capitalised and joined by `-`
|
||||
(`docs/how-to/ingest-a-source.md` becomes `How-to-Ingest-a-source`; a `README.md` stands for its
|
||||
directory). Two sources that would get the same name, compared without case, abort the run.
|
||||
Relative links between mirrored files become wiki page names, relative links to anything else
|
||||
become links to that file at the release tag, and a link whose target does not exist aborts the
|
||||
run. A `_Sidebar` is generated, and every page opens with a line saying which file at which
|
||||
version it came from. `.gitea/scripts/wiki-push.sh` then makes the wiki repository hold exactly
|
||||
that output - a page the converter no longer produces is removed - and does not commit when
|
||||
nothing changed.
|
||||
|
||||
Two things start it: the `wiki-mirror` job of `release.yml`, after each release (never for a
|
||||
candidate), and `.gitea/workflows/wiki-mirror.yml` by hand with `workflow_dispatch`. The manual
|
||||
workflow takes a `tag` (default: the latest release) and runs the converter and push script of
|
||||
`main` against that tag's tree. Use it to fill the wiki before the first release that carries the
|
||||
job, or to catch up a mirror that failed after its release was published. `ci.yml` runs
|
||||
`wiki_mirror.py --check` against the working tree on every push, so a link the mirror could not
|
||||
resolve fails there rather than at the release. The scripts live under `.gitea/`, which
|
||||
`dist export` does not copy, so none of this reaches an instance.
|
||||
|
||||
Both jobs write with the job token, which needs `permissions: wiki: write` - Gitea gives the wiki
|
||||
a token scope of its own, and `contents: write` does not cover it (the API answers 403). If the
|
||||
repository's Actions settings ever cap the token below that, the mirror fails until the cap is
|
||||
lifted. A wiki that has never had a page has no repository to clone yet; `wiki-push.sh` then
|
||||
creates the first page through the API and clones afterwards.
|
||||
|
||||
### How much of the stack the suite reaches
|
||||
|
||||
Coverage is measured in CI - `pytest --cov`, config in `tools/.coveragerc`, HTML and XML
|
||||
uploaded as the `coverage-<run id>` artifact of every run.
|
||||
**Fetch that artifact from the run's own page, not from the API**: `upload-artifact@v3` writes
|
||||
through the older artifact API, and the Actions artifact REST endpoints answer `total_count: 0`
|
||||
for a run whose artifact the run page offers for download. The upload works; only the listing
|
||||
does not see it. Do not re-derive this, and do not read the empty list as a failed upload.
|
||||
|
||||
It is enforced at a floor of **85%** (`fail_under` in `tools/.coveragerc`), which is what a red
|
||||
suite from this axis means: coverage actually fell, not that a wrapper was added. The floor was
|
||||
set only after the number had been watched - it was deliberately held back for exactly that, and
|
||||
the two points between 85 and the measured 87.0% are the room the taxonomy below asks for. A
|
||||
threshold at the measured number goes red on the next thin Typer wrapper, and a threshold that
|
||||
goes red for a non-reason gets lowered rather than earned.
|
||||
|
||||
**Measured 2026-09-04, stack 4.7.1: 87.0% of 6498 statements across `chemenu/`, 975 tests** -
|
||||
CI run 163. The first measurement, at stack 1.8.1 on 2026-08-31, was 86.9% of 5105 statements
|
||||
over 730 tests (CI run 87). Both are what CI reported, never a local run: the local number
|
||||
preceding a release measures a tree that is one commit short of the published one.
|
||||
|
||||
The pair says more than either number does. Between them the measured code grew by a quarter
|
||||
and the suite by a third, and the quota moved by a tenth of a point - which is the observation a
|
||||
threshold was waiting for, rather than the total itself. Reproduce either with
|
||||
`cd tools && .venv/bin/python -m pytest -q --cov` (needs `pytest-cov`, which is CI-only and
|
||||
deliberately absent from `tools/requirements.txt` - an instance runs the wiki, it does not
|
||||
measure this suite).
|
||||
|
||||
The total stays the least interesting number here. What the report is for is *which* modules sit
|
||||
low, and three kinds have to be told apart before any of it turns into work:
|
||||
|
||||
- **Thin Typer wrappers**, where the logic lives beside them and is tested there:
|
||||
`eval_cmd.py` (36%), `types_cmd.py` (40%), `search.py` (49%), `cli.py` (54%),
|
||||
`links_cmd.py` (61%). Low coverage on a wrapper is evidence of a good cut, not of a missing
|
||||
test - `search.py`'s uncovered block is its command body alone, while the backends under
|
||||
`chemenu/search/` that do the work sit between 91% and 98%.
|
||||
- **Code that reaches the network or the filesystem's outside**, where the interesting half is
|
||||
already injectable and tested through the seam: `version.py`'s `fetch_latest()` takes a
|
||||
`fetcher` parameter for exactly that. The real `urllib` lines are exercised too, but only
|
||||
against a local `http.server` in `test_dist_upgrade.py` (`dist upgrade --latest`) - never
|
||||
against a real feed, which stays uncovered on purpose.
|
||||
- **Genuine gaps**, where uncovered lines are logic nobody exercises: `provenance_cmd.py`
|
||||
(44%), `migrate_cmd.py` (65%), `type_resolver.py` (79%). This is the list worth reading, and
|
||||
the only one of the three that has not moved while everything around it did:
|
||||
`provenance_cmd.py` sits where it sat, and `migrate_cmd.py` fell from 71% because it grew and
|
||||
its new lines arrived untested. The floor freezes this; it does not close it. Closing it is
|
||||
Gitea #51.
|
||||
|
||||
## Stack development as a session type of its own
|
||||
|
||||
Three skills (`instructions/dev/`, present only in this origin repository) carry a session that
|
||||
works on the stack itself rather than on wiki content through three phases: `stack-dev` works
|
||||
the issue out until its body is "ready", `stack-build` builds, publishes and waits for a green CI
|
||||
run, `stack-close` checks the body's final state and stale `docs/` and contract prose, and closes
|
||||
the issue. The handover goes through the state in the tracker, not through a session's context:
|
||||
every phase change works in the same session or after `/clear`. `stack-dev` triggers
|
||||
automatically; `stack-build` and `stack-close` are started only by the operator, by slash command
|
||||
(`/stack-build #N`, `/stack-close`) - exactly where the choice falls whether to continue in the
|
||||
same session or a new one, and on which model. None of the three offers a model switch in the
|
||||
middle of a session. Rules and the phase table are in
|
||||
[instructions/dev/stack-mode.md](../../instructions/dev/stack-mode.md), the issue tracker itself
|
||||
in [instructions/dev/issue-tracking.md](../../instructions/dev/issue-tracking.md).
|
||||
@@ -0,0 +1,22 @@
|
||||
# Explanation
|
||||
|
||||
Why the stack is built the way it is. These pages are background for humans and agents alike:
|
||||
read by link or on request, never as instruction, and they carry no normative sentence - a rule
|
||||
lives in `AGENTS.md` or a `CONTRACT.md`, and a page here only gives its reasons.
|
||||
|
||||
- [architecture.md](architecture.md) - what the stack is made of, and how its parts divide the
|
||||
work
|
||||
- [telemetry-and-evaluation.md](telemetry-and-evaluation.md) - how the stack observes what an
|
||||
agent did, and how that record turns into a score
|
||||
- [pipeline-rationale.md](pipeline-rationale.md) - why the pipeline has four stages
|
||||
- [ownership-and-templates.md](ownership-and-templates.md) - why a `.template` split exists, and
|
||||
why an instance comes only from a release
|
||||
- [language-boundaries.md](language-boundaries.md) - why the control plane is English everywhere
|
||||
and the KB language is a value
|
||||
- [why-gates-are-code.md](why-gates-are-code.md) - why the gates are code rather than instruction
|
||||
- [version-model.md](version-model.md) - why a version number answers a compatibility question
|
||||
and a migration question separately
|
||||
- [knowledge-and-commitment.md](knowledge-and-commitment.md) - why commitments live in a task
|
||||
tracker rather than in `kb/`
|
||||
- [model-and-effort-selection.md](model-and-effort-selection.md) - which model and effort suit
|
||||
which kind of session, for the harness that has to choose
|
||||
@@ -0,0 +1,279 @@
|
||||
# Architecture
|
||||
|
||||
What Chemenu is made of, and how the parts divide the work. For why the pipeline has exactly four
|
||||
stages, see [pipeline-rationale.md](pipeline-rationale.md). This page is the map those reasons
|
||||
apply to.
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
|
||||
- [A wiki that is compiled, not searched](#a-wiki-that-is-compiled-not-searched)
|
||||
- [The language of the pages](#the-language-of-the-pages)
|
||||
- [The tree](#the-tree)
|
||||
- [Collections and areas](#collections-and-areas)
|
||||
- [Contracts and READMEs](#contracts-and-readmes)
|
||||
- [Who does what](#who-does-what)
|
||||
- [The compiler: `tools/wikitool`](#the-compiler-toolswikitool)
|
||||
- [The MCP read server](#the-mcp-read-server)
|
||||
- [Built for IT work](#built-for-it-work)
|
||||
- [Telemetry and evaluation](#telemetry-and-evaluation)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
## A wiki that is compiled, not searched
|
||||
|
||||
Chemenu is a pattern for a personal knowledge base maintained by an LLM. A plain retrieval setup goes
|
||||
back to the raw documents for every question. Here, the LLM **builds and maintains a persistent
|
||||
wiki** incrementally instead, and the wiki compounds over time. Knowledge is compiled once and kept
|
||||
current, not re-derived for every question.
|
||||
|
||||
That one decision shapes everything else. Raw material is kept unchanged as evidence, and compiled
|
||||
pages cite it. Every mechanical step of the compilation - frontmatter, cross-references, the catalog,
|
||||
the log, publishing - is done by a deterministic tool, never by hand, so it comes out the same way
|
||||
every time.
|
||||
|
||||
## The language of the pages
|
||||
|
||||
Source material in `raw/` is never touched and stays in whatever language it arrived in. The
|
||||
compiled pages under `kb/` are written in the instance's own language. The origin repository's demo
|
||||
corpus, for instance, is German. What stays in its original form inside a page is everything that
|
||||
is an *identifier* rather than prose: page titles, section headings the tool owns, wikilink targets,
|
||||
citation ids, schema enum values, tags, commands, paths and code. So `GitOps Ownership Model` and a
|
||||
German section heading sit in the same page without contradiction.
|
||||
|
||||
Which lines count as identifiers is the stack's rule,
|
||||
[kb/CONTRACT.md § Language and identifiers](../../kb/CONTRACT.md#language-and-identifiers).
|
||||
*Which language* the prose is in, and what the tool-owned headings are called, is the instance's own
|
||||
`kb/CONVENTIONS.md`. That is a per-instance decision, not a property of the pattern, so it lives in a
|
||||
file the instance owns rather than in one the stack ships. A new instance picks its language when it
|
||||
fills in `kb/CONVENTIONS.md`, before the first ingest. Why the control plane stays English
|
||||
regardless: [language-boundaries.md](language-boundaries.md).
|
||||
|
||||
## The tree
|
||||
|
||||
```
|
||||
chemenu/
|
||||
├── AGENTS.md # Control plane: invariants, file naming, routing, gates
|
||||
├── CLAUDE.md # Claude Code only: imports AGENTS.md, links the one Claude-Code-only decision (model/effort). No rules of its own
|
||||
├── README.md # Landing page for humans
|
||||
├── INSTALL.md # Human-readable install: one release, one sentence to the agent
|
||||
├── CHANGES.md # Changelog for the stack itself
|
||||
├── USER.md # Who operates this instance - context, never instructions
|
||||
├── SOUL.md # How this instance sounds. AGENTS.md always wins over it
|
||||
├── ENVIRONMENT.md # Optional, gitignored: this checkout's harness, MCP servers, remotes
|
||||
├── *.md.template # Unfilled USER/SOUL/ENVIRONMENT - what a distribution ships instead
|
||||
├── .gitignore # Anchored so nothing under raw/, kb/ or work/ is ever excluded;
|
||||
│ # incoming/ is the one directory excluded the other way round
|
||||
├── .github/hooks/ # Copilot CLI hooks - session tracing
|
||||
├── .vibe/ # Mistral Vibe hooks + the repo's telemetry policy
|
||||
├── docs/ # Human documentation
|
||||
│ ├── tutorials/ # learning: a guided first walk
|
||||
│ ├── how-to/ # doing: one task per page
|
||||
│ ├── reference/ # looking up: commands, exit codes, skills, page types, configuration
|
||||
│ └── explanation/ # understanding: why it is built this way (this page)
|
||||
├── instructions/ # CONTROL: everything an agent is told to do
|
||||
│ ├── CONTRACT.md # Instruction vs. skill, publishing, writing standard
|
||||
│ ├── bootstrap.md # Prepare a further checkout of an instance
|
||||
│ ├── gates.md # What to do when a gate refuses a call
|
||||
│ ├── german-terminology.md # Which words stay English in German prose; register
|
||||
│ ├── session-setup.md
|
||||
│ ├── page-lifecycle.md
|
||||
│ ├── publish-cycle.md
|
||||
│ ├── ingest-large-tree.md
|
||||
│ ├── ingest-queue.md # Reviewing a submission before wikitool upload accept promotes it
|
||||
│ └── wiki-*/SKILL.md # Skills - copied into .agents/skills/ and .claude/skills/
|
||||
├── mcp-upload/ # QUARANTINE (optional): the MCP `submit` tool's write path, gitignored -
|
||||
│ # read by no command in the ordinary pipeline; a human reviews it with
|
||||
│ # `wikitool upload list/show/accept/reject`
|
||||
├── incoming/ # INBOX: content gitignored - drop a file or one folder per source here, `raw accept` promotes it
|
||||
├── raw/ # INPUT: immutable, untrusted source material
|
||||
│ ├── CONTRACT.md # Date shard, capture fields, immutability, untrusted content
|
||||
│ ├── 2026/09/ # Where `raw accept` puts a file: the month it was accepted
|
||||
│ ├── articles/ # The old type directories: still valid paths, never moved,
|
||||
│ ├── documents/ # but nothing new is ever routed into them again
|
||||
│ ├── notes/
|
||||
│ └── assets/
|
||||
├── types/ # SCHEMA: the global type surface. Not a collection
|
||||
│ ├── type-spec.md # Root contract: anatomy, placement, adding a type
|
||||
│ ├── type-guidance.md # Contract for the *.guidance.md files below
|
||||
│ ├── entity.md # Entity type config + template (+ .schema.yaml)
|
||||
│ ├── entity.guidance.md # Its stack-owned authoring prose, shipped verbatim
|
||||
│ ├── entity.person.md # Subtype template: what `new` scaffolds for entity_type=person
|
||||
│ ├── concept.md # Concept type config + template (+ .guidance.md, + concept.decision.md)
|
||||
│ ├── source.md # Source type config + template (+ .guidance.md)
|
||||
│ ├── comparison.md # Comparison type config + template (+ .guidance.md)
|
||||
│ ├── project.md # Project (Vorhaben) type config + template, no guidance file
|
||||
│ ├── instruction.md # Instruction type - lives outside kb/ via `root: repo`
|
||||
│ └── lint-report.md # Contract-only: describes reports/, owns no directory
|
||||
├── kb/ # OUTPUT: compiled knowledge. A namespace, not a collection
|
||||
│ ├── CONTRACT.md # Collections, naming, tone, linking, provenance
|
||||
│ ├── index.md # Generated catalog *map*: counts and pointers
|
||||
│ ├── log.md # Generated chronological audit log
|
||||
│ ├── provenance.md # Generated raw-file reverse index
|
||||
│ ├── entities/ # COLLECTION.md + INDEX.md + areas below
|
||||
│ │ ├── codebases/
|
||||
│ │ ├── systems/
|
||||
│ │ ├── tools/ # own INDEX.md once past 50 pages
|
||||
│ │ ├── technologies/
|
||||
│ │ ├── people/
|
||||
│ │ └── organizations/
|
||||
│ ├── concepts/ # COLLECTION.md + INDEX.md + areas below
|
||||
│ │ ├── architectures/
|
||||
│ │ ├── patterns/
|
||||
│ │ ├── protocols/
|
||||
│ │ ├── workflows/
|
||||
│ │ ├── decisions/
|
||||
│ │ └── problems/
|
||||
│ ├── sources/ # COLLECTION.md + INDEX.md + areas below
|
||||
│ │ ├── transcripts/
|
||||
│ │ ├── analyses/
|
||||
│ │ ├── articles/
|
||||
│ │ ├── documents/
|
||||
│ │ ├── notes/
|
||||
│ │ ├── trackers/
|
||||
│ │ └── unclassified/
|
||||
│ ├── comparisons/ # COLLECTION.md - comparison pages, no subtype axis
|
||||
│ └── gtd/ # COLLECTION.md + INDEX.md + areas below
|
||||
│ ├── haus/
|
||||
│ ├── finanzen/
|
||||
│ └── technik/
|
||||
├── work/ # WORKSHOP: one directory per multi-session run, tracked
|
||||
│ └── CONTRACT.md # Run keys, required files, how a run closes
|
||||
├── reports/ # DERIVED: lint reports, traces, eval scores. Gitignored
|
||||
│ └── CONTRACT.md
|
||||
└── tools/ # COMPILER: the wikitool CLI
|
||||
├── CONTRACT.md # Command records (generated) and index, maintenance schedule
|
||||
└── README.md # How wikitool is built and how to change it
|
||||
```
|
||||
|
||||
The collections and areas under `kb/` shown here are the origin repository's. An instance's own come
|
||||
from its type-specs, as described below.
|
||||
|
||||
<!-- dist:strip-start -->
|
||||
Only in the origin repository (`tools/CONTRACT.md` explains how an instance comes to lack them):
|
||||
|
||||
```
|
||||
├── docs/dev/ # Human-readable: development environment and release workflow
|
||||
└── commonplace/ # Vendored, read-only knowledge base
|
||||
```
|
||||
<!-- dist:strip-end -->
|
||||
|
||||
## Collections and areas
|
||||
|
||||
A directory under `kb/` is a **collection** exactly when it holds a `COLLECTION.md`. A subdirectory
|
||||
inside one is an **area**, which inherits it, and that is as deep as a page goes. Nothing nests
|
||||
below an area, because the generated catalog reads exactly two path segments under `kb/`, and it
|
||||
would silently fold a deeper page into the area. `kb/CONTRACT.md` § Collections has the rule, and
|
||||
`wikitool lint` reports a violation as a hard error.
|
||||
|
||||
Which areas a collection has is not chosen page by page. A type-spec's `layout:` maps its subtype
|
||||
field onto directories, and `wikitool new` writes each page straight into the one its subtype names.
|
||||
That is also what gives the catalog's shard threshold any effect: `index rebuild` splits **per
|
||||
area**, so a collection with no areas keeps one table however large it grows. Once such a collection
|
||||
is past the threshold, `wikitool lint` reports it as a recommendation rather than an error, together
|
||||
with the split its subtype field would produce. It stays quiet when the split would not actually
|
||||
help.
|
||||
|
||||
`kb/comparisons/` is the worked example of a collection that stays flat: it has no subtype field for
|
||||
a `layout:` to key on at all. A *lopsided* subtype field is a different case, and it gets fixed
|
||||
rather than left flat. `kb/sources/` looked lopsided only because `source_type` had a schema
|
||||
`default:`, which the compiler applied whenever nobody chose a value. Once the default was removed
|
||||
and the pages reclassified, it split into six real areas.
|
||||
|
||||
Every page type the stack ships, with its subtypes, its directories and its required fields, is in
|
||||
[page-types.md](../reference/page-types.md). An instance adopts those type-specs and may change
|
||||
them; `tools/wikitool types list` and `tools/wikitool types describe <type>` show what one instance
|
||||
has.
|
||||
|
||||
## Contracts and READMEs
|
||||
|
||||
`COLLECTION.md` never appears outside `kb/`. The other layers carry a `CONTRACT.md` or a root
|
||||
type-spec instead. A stage may carry both a `README.md` and a `CONTRACT.md`, because they have
|
||||
different readers: the README is for humans working *on* that layer, and the contract binds an agent
|
||||
working *with* it. `AGENTS.md` § File naming says which file is for which reader, and
|
||||
`wikitool docs verify` checks it.
|
||||
|
||||
## Who does what
|
||||
|
||||
**You** curate sources by dropping them into `incoming/`, ask questions, review what changed in
|
||||
`kb/log.md` and `kb/index.md`, and steer the agent on what to emphasise or look into. The
|
||||
[how-to guides](../how-to/README.md) cover each of these.
|
||||
|
||||
**The agent** follows `AGENTS.md`, the cross-cutting schema and policy, and runs the step-by-step
|
||||
procedures as skills. They are published into `.agents/skills/`, and mirrored into
|
||||
`.claude/skills/` for Claude Code, by `tools/wikitool instructions sync`. Which skills ship, what
|
||||
each does and which requests start it: [skills.md](../reference/skills.md).
|
||||
|
||||
**`tools/wikitool`** does every mechanical step a skill needs: scaffolding, renaming and deleting
|
||||
pages, cross-references, regenerating the catalog, log and provenance index, structural linting,
|
||||
decay math, and publishing. The agent calls it and never does that work by hand. Along the way it
|
||||
maintains:
|
||||
|
||||
- `kb/index.md`, the catalog, always current;
|
||||
- `kb/log.md`, the complete audit trail;
|
||||
- `kb/provenance.md`, the reverse index from raw files to pages;
|
||||
- pages of every type, each in the directory its type-spec names;
|
||||
- lint reports, session traces and eval scores in `reports/`, which is gitignored.
|
||||
|
||||
## The compiler: `tools/wikitool`
|
||||
|
||||
Every command carries one data record kept next to its code: synopsis, properties, copyable
|
||||
examples, each exit cause with what to do about it, prohibitions, and notes on its behaviour. The
|
||||
same record is rendered four ways:
|
||||
|
||||
- `tools/wikitool <command> -h` prints it, and `tools/wikitool -h` prints a one-line index of every
|
||||
command;
|
||||
- a command that fails with exit 1 prints the record's reactions on stderr, right under its `ERROR`
|
||||
line;
|
||||
- [tools/CONTRACT.md](../../tools/CONTRACT.md) holds a generated copy for agents, together with the
|
||||
maintenance schedule;
|
||||
- [the command reference](../reference/commands/README.md) is a generated copy for humans.
|
||||
|
||||
`tools/wikitool docs verify` checks the copies, and every command's flags, against the CLI in both
|
||||
directions. [tools/README.md](../../tools/README.md) is the other half: how the CLI is built and how
|
||||
to add a command. `AGENTS.md` holds the invariants that say when each command is mandatory.
|
||||
|
||||
The wiki is also a git repository, and `tools/wikitool publish` is the one way its changes are
|
||||
committed and pushed. That gives every change a version history, and the gates in front of
|
||||
`publish` decide what needs a human to look first
|
||||
([why-gates-are-code.md](why-gates-are-code.md)).
|
||||
|
||||
<!-- dist:strip-start -->
|
||||
Extending `tools/wikitool`, the type schema or the instruction layer itself is a separate kind of
|
||||
session, with rules of its own: the `stack-dev`, `stack-build` and `stack-close` skills under
|
||||
`instructions/dev/`, which only the origin repository has.
|
||||
<!-- dist:strip-end -->
|
||||
|
||||
## The MCP read server
|
||||
|
||||
The terminal is not the only way in. `tools/chemenu/mcp/` serves the same wiki read-only over MCP -
|
||||
`search`, `types`, `describe_type`, `lint` and `status` - so a consumer that is not a shell on this
|
||||
machine can ask the same questions and get the same answers. Literally the same: the CLI and the
|
||||
server are two adapters over one core, and a golden test holds their output together rather than
|
||||
trusting that it agrees.
|
||||
|
||||
There is no tool that writes to `kb/`, and not because one is filtered out of a list. The server
|
||||
imports nothing under `chemenu/commands/`, so `new`, `publish` and the rest cannot be reached from
|
||||
it. Every answer carries the commit it was computed from, so a checkout that has fallen behind gives
|
||||
a visibly stale answer instead of a confident wrong one. The server's dependency is deliberately
|
||||
not in `requirements.txt`: an instance that only uses the CLI should not have to install a web stack
|
||||
for it.
|
||||
|
||||
Setting it up: [set-up-the-mcp-server.md](../how-to/set-up-the-mcp-server.md).
|
||||
|
||||
## Built for IT work
|
||||
|
||||
The shipped defaults are tailored to IT work:
|
||||
|
||||
- **entity types** for software development and systems - codebases, systems, tools, technologies;
|
||||
- **relationship labels** such as "depends on", "uses" and "implements". The vocabulary is in the
|
||||
instance's `kb/CONVENTIONS.md`, because it belongs to the instance rather than the stack;
|
||||
- **templates** per page type, and per subtype where its pages need a shape of their own, such as a
|
||||
person or a decision record (ADR);
|
||||
- **guidelines** for documenting technical decisions, which can be exported back into the
|
||||
repositories they are about ([export-guidelines.md](../how-to/export-guidelines.md));
|
||||
- **cross-reference patterns** for code and architecture.
|
||||
|
||||
## Telemetry and evaluation
|
||||
|
||||
How the stack observes what an agent did, and how that record becomes a score, is its own page:
|
||||
[telemetry-and-evaluation.md](telemetry-and-evaluation.md).
|
||||
@@ -0,0 +1,147 @@
|
||||
# Why knowledge and commitments are two layers
|
||||
|
||||
Chemenu compiles knowledge into `kb/`, and it also tracks what its operator has committed to do.
|
||||
Those look like one subject - both are "things about my projects" - and the stack deliberately
|
||||
keeps them apart: `kb/gtd/` holds one page per initiative, an external task tracker holds the
|
||||
open items, and the only thing that crosses between them is a name. This page is about why that
|
||||
line was drawn there. The rules that follow from it live in [kb/CONTRACT.md](../../kb/CONTRACT.md)
|
||||
and the `review`, `new` and `task new` records in [tools/CONTRACT.md](../../tools/CONTRACT.md).
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
|
||||
- [Different half-lives want different machinery](#different-half-lives-want-different-machinery)
|
||||
- [Pattern 4: separate ownership, no synchronization](#pattern-4-separate-ownership-no-synchronization)
|
||||
- [The join happens at read time, and stores nothing](#the-join-happens-at-read-time-and-stores-nothing)
|
||||
- [One name, carrying the duties of an identifier](#one-name-carrying-the-duties-of-an-identifier)
|
||||
- [Status has exactly one home](#status-has-exactly-one-home)
|
||||
- [A finished initiative is a state, not a location](#a-finished-initiative-is-a-state-not-a-location)
|
||||
- [Which tracker is a decision the stack does not make](#which-tracker-is-a-decision-the-stack-does-not-make)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
## Different half-lives want different machinery
|
||||
|
||||
`kb/` is a compiler for durable things, and every mechanism in it assumes durability: `raw/` is
|
||||
immutable, a claim has to trace back to a source, a page's title is its identity, the indexes are
|
||||
generated, and a large change stops at a gate so a human can look at it. All of that is the right
|
||||
amount of ceremony for something that will still be true next year.
|
||||
|
||||
A next action is the opposite kind of fact. It is unsourced - nobody cites a reason for "call the
|
||||
plumber". It changes several times a week. It is state, not knowledge: the interesting thing
|
||||
about it is whether it is still open. And it is only correct *now*.
|
||||
|
||||
Running both through one layer does not produce a richer wiki; it produces a worse one. Every
|
||||
task-shaped page carries `provenance: general` because there is no source to bind it to, which
|
||||
drains that field of meaning for the pages where it matters. `kb/log.md` fills with "task
|
||||
checked off" entries until the audit trail of what the *wiki* learned is unreadable. Lint findings
|
||||
about orphans and stale claims start firing on pages that are supposed to be short-lived. And a
|
||||
weekly pass over the task list trips the Mass-Update Gate every single time, which is how a gate
|
||||
stops being read and starts being cleared reflexively.
|
||||
|
||||
The GTD method this borrows from draws the same line for its own reasons: of its horizons, `kb/`
|
||||
covers the two slowest - project support material and reference - and nothing faster.
|
||||
|
||||
## Pattern 4: separate ownership, no synchronization
|
||||
|
||||
Four arrangements were on the table, and three of them fail in ways worth naming.
|
||||
|
||||
**One layer** is the case above. **Export** - the wiki writes a task list the tracker imports -
|
||||
means a checkbox ticked in the tracker is a tick in a view, while the truth sits in a file the
|
||||
operator was not editing; the two disagree immediately and silently. **Bidirectional sync** works,
|
||||
at the cost of an id mapping to maintain, a conflict-resolution rule to design, and a deletion
|
||||
semantics to decide - all of it machinery whose only job is to repair a split nobody needed.
|
||||
|
||||
What is left is **separate ownership with no sync at all**: the tracker owns the tasks, `kb/` owns
|
||||
the project memory, and the single point of contact is the project's name. Nothing is mirrored,
|
||||
so nothing can drift out of mirror.
|
||||
|
||||
## The join happens at read time, and stores nothing
|
||||
|
||||
Because there is no shared state, the connection between the two sides has to be made when
|
||||
somebody actually asks - which is what `wikitool review` does: it reads both sides, matches them on
|
||||
the case-normalized project name, prints what it found, and saves nothing. Not a cache, not a
|
||||
mapping file, not even a `reports/` artifact.
|
||||
|
||||
That is the same posture `search` takes, and for the same reason: anything it wrote down would be
|
||||
a third copy of a state the two sides already hold, stale the moment either side moved, and the
|
||||
first thing to distrust in a report. A read-time join can be wrong about the present, but it
|
||||
cannot be wrong about the past, because it does not remember one.
|
||||
|
||||
## One name, carrying the duties of an identifier
|
||||
|
||||
Reducing the coupling to a name is cheap, and it is not free. A name that joins two systems is an
|
||||
identifier, whether or not anything enforces it, so the design had to pick up an identifier's
|
||||
obligations explicitly: uniqueness is checked before a project is created rather than discovered
|
||||
later; a rename is a deliberate, infrequent operation that touches both sides in one pass; and
|
||||
nothing tries to re-match automatically behind the operator's back.
|
||||
|
||||
The last one is what makes the review's *both-directional* report matter. A tracker project with
|
||||
no page and a page with no tracker project are reported separately, as two findings. They are
|
||||
usually the two halves of one rename - and reporting them separately is exactly what turns a
|
||||
silent decoupling into a visible event, at the cost of the review occasionally saying the same
|
||||
thing twice.
|
||||
|
||||
## Status has exactly one home
|
||||
|
||||
The sharpest consequence of the split is a rule that feels like a restriction: a `kb/` page never
|
||||
summarizes its own task list. No "3 open items", no "next: call the supplier".
|
||||
|
||||
Two places claiming to know the current status is the failure mode the whole arrangement exists
|
||||
to avoid, and a summary is a copy with a slower clock. The page says what an initiative *is* -
|
||||
its goal, its participants, its durable state, why it is worth doing. The tracker says what is
|
||||
open right now. Anyone wanting the second reads the tracker, or runs the review.
|
||||
|
||||
This pays for itself somewhere unexpected: with the page carrying no task state, an agent has no
|
||||
reason to read the task list at all outside the weekly review. That is what keeps the command
|
||||
surface as small as it is - two read commands and three write commands - rather than growing a
|
||||
full CRUD tree over somebody's todo list. The second creation command exists because a single
|
||||
name is not always the whole story: a source can carry a piece of durable knowledge and a
|
||||
commitment to follow up on it at the same time - a complaint arriving by email is both something
|
||||
to file and something to chase - and the tracker-side half of that needs its own write path
|
||||
alongside `new project`'s pairing of a page with a tracker project. `task new` creates only the
|
||||
tracker item, never a page; a source that also carries knowledge gets that knowledge filed
|
||||
through the ordinary page-creation commands, as a separate step. The two are never one
|
||||
transaction the way `new project`'s tracker-then-page order is within a single command - they are
|
||||
two independent writes a skill sequences, tracker first, so a failure creating the item leaves no
|
||||
page and no promoted source material behind it. That holds for an ordinary source; a source large
|
||||
or broad enough to run through the large-tree procedure instead promotes ahead of its own
|
||||
per-unit commitment decision, because that procedure hands its units through a workshop directory
|
||||
that needs them already promoted to address them at all - the same raw-file-without-page state
|
||||
the ordinary case avoids becomes, there, the expected condition for as long as the run takes. And
|
||||
a failure on the knowledge side afterwards is exactly the ordinary "a source without a page" state
|
||||
`lint` already reports.
|
||||
|
||||
The write surface stops at *creating* an item and *marking one done* - it never moves a reminder
|
||||
and never deletes anything. `task close` sets exactly the field the tracker's own "done" checkbox
|
||||
sets, nothing more: reversible, and it leaves a record in the tracker rather than removing the
|
||||
item's trace. A command that deleted would take the same shortcut through somebody's task list
|
||||
that the whole split above exists to avoid - a write this stack cannot undo, made on behalf of a
|
||||
tracker it does not own. `task list` is the one addition on the read side, and it changes nothing
|
||||
about the join itself: it exists only because closing an item needs the tracker's own id for it,
|
||||
and that id was never worth exposing before there was a write that consumed it.
|
||||
|
||||
## A finished initiative is a state, not a location
|
||||
|
||||
Archiving moves nothing. A completed initiative's page stays where it is and changes its `state:`
|
||||
value, because the moment an initiative finishes is the moment its page is *most* valuable -
|
||||
what was decided, what it cost, who was involved - and filing it away is how that gets lost.
|
||||
|
||||
The state field carries the distinction the review actually needs, which is not "open vs. done"
|
||||
but "does silence here mean something is wrong". An initiative that is deliberately paused looks
|
||||
identical, from the outside, to one that quietly stalled; only the operator knows which. Without a
|
||||
value for "paused on purpose", the review reports the same untouched initiatives every week, and
|
||||
a report that is mostly noise stops being read by the third week - which would cost more than the
|
||||
findings are worth.
|
||||
|
||||
## Which tracker is a decision the stack does not make
|
||||
|
||||
The tracker is reached through a provider layer, and no instruction anywhere names which one it
|
||||
is. An instruction that said "open Super Productivity" would bake one instance's tool choice into
|
||||
the shared stack, and the next instance - a different context, a different employer, a different
|
||||
set of constraints - would have to edit prose to change a setting.
|
||||
|
||||
So the provider lives in configuration (`.wikitool-tasks.json`), the adapters live behind one
|
||||
protocol, and a capability the provider lacks surfaces as an ordinary tool error rather than as a
|
||||
paragraph of instruction explaining what this particular tracker cannot do. A provider that
|
||||
cannot create a project, for instance, stops and asks the operator to do it - the same posture the
|
||||
gates take, and for the same reason: better a visible stop than an invented workaround.
|
||||
@@ -0,0 +1,163 @@
|
||||
# Language Boundaries
|
||||
|
||||
Two languages run through this repo at once. `kb/` is written in whatever language the instance
|
||||
chose - German here, and the value lives in `kb/CONVENTIONS.md`'s `language:`. Everything that
|
||||
tells an agent what to do - [AGENTS.md](../../AGENTS.md), every `CONTRACT.md`, everything under
|
||||
`instructions/` - is written in English, in every instance, whatever the first value says. So
|
||||
is the stack's own documentation for humans, for a different reason (§ "The stack's human
|
||||
documentation").
|
||||
|
||||
The rule itself is in [AGENTS.md § File naming](../../AGENTS.md#file-naming). This page holds the
|
||||
part that is not a rule: why the line runs where it does, why the English half is not a setting,
|
||||
and which argument for it turned out to be wrong.
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
|
||||
- [The axis is the reader, not the owner](#the-axis-is-the-reader-not-the-owner)
|
||||
- [Why the control plane's language is English](#why-the-control-planes-language-is-english)
|
||||
- [Why it is not a parameter](#why-it-is-not-a-parameter)
|
||||
- [What the KB language still decides](#what-the-kb-language-still-decides)
|
||||
- [Where the line runs around a page type](#where-the-line-runs-around-a-page-type)
|
||||
- [The stack's human documentation](#the-stacks-human-documentation)
|
||||
- [What would put this back on the table](#what-would-put-this-back-on-the-table)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
## The axis is the reader, not the owner
|
||||
|
||||
For a long time the two halves could be told apart by asking who owned the file, and the answer
|
||||
came out right every time: the stack owns `AGENTS.md` and the contracts, which are English; the
|
||||
instance owns its pages and the templates that shape them, which are in the KB language. The
|
||||
ownership boundary is a real and load-bearing thing - [ownership-and-templates.md](ownership-and-templates.md)
|
||||
is about what it buys - so it was easy to read the language split as one of its consequences.
|
||||
|
||||
It is not. The case that separates them is a page type an instance adds for itself. `types/`
|
||||
takes a new type without a code change, so an instance can write one; that file is instance-owned
|
||||
from the first line to the last, ships nowhere, and is nobody's to overwrite. Its authoring
|
||||
guidance is still instruction addressed to an agent, and reads exactly like the guidance in the
|
||||
four types the stack ships. Ownership says "yours"; the audience has not moved at all.
|
||||
|
||||
So the question a line answers is not *whose file is this* but *who reads this line*, which is
|
||||
the same cut [kb/CONTRACT.md](../../kb/CONTRACT.md#language-and-identifiers) already makes inside a
|
||||
single page between prose and identifiers - applied one level up, to the halves of a document.
|
||||
Ownership decides who may change a sentence. The reader decides what language it is in. The two
|
||||
questions were answered together for as long as they happened to agree.
|
||||
|
||||
## Why the control plane's language is English
|
||||
|
||||
Not because English is better for the purpose, and not to be neutral: this instance's operator
|
||||
reads German, and the pages are German for that reason.
|
||||
|
||||
- **The control plane is almost entirely about identifiers, and the identifiers are English.**
|
||||
`base_dir`, `provenance: sourced`, `--confirm`, exit 42, `root: kb`. A sentence in another
|
||||
language explaining when to set `page_ref_fields` is already half English by the time it
|
||||
reaches the verb, and the prose/identifier boundary inside it becomes something a reader has to
|
||||
work out line by line.
|
||||
- **It quotes a body of material that is English and stays English.** The harness documentation
|
||||
it has to agree with, the vendored skill-authoring sources under `commonplace/`, the tool's own
|
||||
`--help`. A contract that translates their vocabulary makes its own claims harder to check
|
||||
against them, not easier.
|
||||
- **One language keeps instances comparable.** Two instances running the same stack version hold
|
||||
the same control plane byte for byte, so a question about one is answerable from the other -
|
||||
and anything an instance changes locally shows up as a difference in content rather than in
|
||||
language.
|
||||
|
||||
## Why it is not a parameter
|
||||
|
||||
The natural next move, once `kb/CONVENTIONS.md` holds `language:`, is a second value beside it -
|
||||
`control_plane_language:` - defaulting to English and settable by an instance that would rather
|
||||
read its contracts in its own language. That option is deliberately not taken.
|
||||
|
||||
- **The knob's cost is paid by every file; its benefit lands on the few a human reads.** Every
|
||||
rule about writing an instruction would have to name which of the two languages it means, every
|
||||
example would need a note saying which one it is in, and every review of an instruction would
|
||||
start by establishing which language it should have been in. The stack has one mechanism for
|
||||
that class of problem - one rule, one place (AGENTS.md invariant 8) - and a second language
|
||||
value forks it everywhere at once.
|
||||
- **The document the knob is for is read by an agent.** An instruction, a contract, a type-spec's
|
||||
guidance half: the reader is a model, and a model reads the English fine. What the *operator*
|
||||
reads is unaffected by any of this - see the section below.
|
||||
- **Today's local document is tomorrow's upstream candidate.** An instruction an instance wrote
|
||||
for itself is the most likely thing it ever contributes back. Written in the KB language it
|
||||
would have to be translated first, and the translation would have to re-derive the
|
||||
prose/identifier boundary that the original author had in their head and did not write down.
|
||||
- **Nothing would check it.** There is no mechanical test for what language a paragraph is in -
|
||||
a stop-word scan flags the quoted vocabulary the rule deliberately keeps and misses a cleanly
|
||||
translated paragraph. A setting nothing enforces produces drift that is visible only to whoever
|
||||
next opens the file.
|
||||
|
||||
## What the KB language still decides
|
||||
|
||||
Making the control plane English does not make the instance's language an implementation detail.
|
||||
`kb/CONVENTIONS.md`'s `language:` decides two things, and both are the ones an operator actually
|
||||
experiences:
|
||||
|
||||
- **Page text.** Every page under `kb/`, and inside the page type-specs exactly the parts that
|
||||
become page text - each one's `## Template` block, its `layout:` titles, and the subtype
|
||||
templates `types/<name>.<value>.md` beside it, which are page text from the first line to the
|
||||
last.
|
||||
- **What an agent says.** An agent speaks the KB language, whatever the file it just read was
|
||||
written in. An instruction that models a sentence for the operator writes that model in
|
||||
English, like the rest of the control plane, and the agent delivers it in the instance's
|
||||
language.
|
||||
|
||||
So an operator who reads no English gets German pages and German answers from an agent reading
|
||||
English instructions. The English is what the machinery is written in, not what it says back -
|
||||
with one exception, the stack's documentation for humans (§ "The stack's human documentation").
|
||||
|
||||
## Where the line runs around a page type
|
||||
|
||||
A page type's contract is where the two languages meet most closely, and it is worth knowing
|
||||
which part is which before editing any of it. Its authoring guidance addresses an agent and is
|
||||
English; its `## Template` block, `layout:` titles and subtype templates become the literal
|
||||
headings of pages and follow the KB language; its field names and enum values are identifiers and are translated in
|
||||
neither direction.
|
||||
|
||||
The language line did not move when the *file* line did. A `root: kb` type-spec may now put its
|
||||
authoring guidance in a separate, stack-owned `types/<name>.guidance.md` rather than carrying it
|
||||
beside the template, but that split was made for ownership reasons - so an upgrade can improve
|
||||
the guidance without overwriting what the instance chose - and it leaves this page's argument
|
||||
untouched: each part is still written in the language its own reader needs, and a type-spec that
|
||||
declares no `guidance:` keeps both halves in one file with exactly the same rule applying inside
|
||||
it. [types/type-spec.md § Who owns a type-spec](../../types/type-spec.md#who-owns-a-type-spec) has
|
||||
the split as a table, and [ownership-and-templates.md](ownership-and-templates.md) § "Where the
|
||||
file boundary used to strain" has what it cost to keep two audiences in one file for as long as
|
||||
it did.
|
||||
|
||||
## The stack's human documentation
|
||||
|
||||
`README.md`, `INSTALL.md` and everything under `docs/` are read by humans, and the
|
||||
reader axis alone would put them in the operator's language. They are English anyway, and the
|
||||
reason is not the reader but the fact that there is no single one. This documentation belongs to
|
||||
the stack, not to an instance: it ships identically to every instance, whatever each one's
|
||||
`language:` says, so there is no instance value it could follow - only a translation per language
|
||||
to maintain beside it, each one a second copy that drifts. And a growing part of it is generated
|
||||
from the code (the command reference from `cli_contract` records, for one), whose source is
|
||||
English control-plane material; a translated frame around English generated text is the
|
||||
half-done version of both.
|
||||
|
||||
The KB language keeps deciding what an operator reads *of their own*: pages and answers. What
|
||||
the stack tells every operator about itself is the same text everywhere. `CHANGES.md` sits
|
||||
outside the question - it is a log of what was true at a time, not documentation, and its older
|
||||
entries are in the language they were written in.
|
||||
|
||||
## What would put this back on the table
|
||||
|
||||
A `docs/` page goes stale when the reasoning stops holding rather than when the code changes, so
|
||||
it is worth naming what that would look like here. Two things would:
|
||||
|
||||
- **A human starts reading the control plane directly and routinely** - not an operator checking
|
||||
a rule now and then, which is the case today, but a workflow where people rather than agents
|
||||
are the primary readers of `instructions/`. The second argument above is the one that fails
|
||||
first, and it is the load-bearing one.
|
||||
- **The identifiers stop being English.** If the tool's own vocabulary were ever localized, the
|
||||
first argument would invert: the prose would then be the only English left in a file that is
|
||||
otherwise not, which is the situation this page argues against.
|
||||
|
||||
Neither is close. Both are cheaper to notice here than to rediscover in an argument about a
|
||||
single file.
|
||||
|
||||
The human documentation has a trigger of its own: **translations the stack maintains itself,
|
||||
with a check that holds each one to its English source.** Without such a check a translation is
|
||||
exactly the drifting second copy the section above argues against; with one, the argument for a
|
||||
single language weakens to cost alone.
|
||||
@@ -0,0 +1,120 @@
|
||||
# Choosing a Claude Code model and effort level
|
||||
|
||||
Claude Code exposes three choices this repo has an opinion on: which model a session itself
|
||||
runs as, what model a spawned subagent gets, and which `/code-review` effort level to pick.
|
||||
None of them are enforced anywhere - the gates in [instructions/gates.md](../../instructions/gates.md)
|
||||
are code precisely because a model cannot be talked out of them
|
||||
([why-gates-are-code.md](why-gates-are-code.md) makes that argument for gates; this page applies
|
||||
the same axis to who is holding the keyboard). What follows is a reference for making that choice
|
||||
well, not a rule anything checks.
|
||||
|
||||
The axis worth tracking is not how important a task feels, but **what would catch a mistake in
|
||||
it**. Work behind `pytest`, `docs verify`, `instructions verify` or CI surfaces a bad call within
|
||||
one more round. Work behind nothing but a session reading prose does not surface at all - it
|
||||
ships, and stays until someone happens to notice. That asymmetry, not task size, is what the
|
||||
phase guide below is built on.
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
|
||||
- [Phase guide](#phase-guide)
|
||||
- [Model per session, not per phase](#model-per-session-not-per-phase)
|
||||
- [Subagent models](#subagent-models)
|
||||
- [`/code-review` effort](#code-review-effort)
|
||||
- [When it's unclear](#when-its-unclear)
|
||||
- [Scope](#scope)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
## Phase guide
|
||||
|
||||
<!-- dist:strip-start -->
|
||||
This repo's own stack-development work splits the axis into three phases, one skill each -
|
||||
`stack-dev` (design), `stack-build` (build) and `stack-close` (closing) - handed over through
|
||||
states in the issue tracker rather than inside one session:
|
||||
|
||||
<!-- dist:strip-end -->
|
||||
| Phase / task | What would catch a mistake | Suggested model | Effort |
|
||||
|---|---|---|---|
|
||||
| `wiki-status`, simple `wiki-query` lookups | the answer is re-checkable against the corpus | Sonnet | default |
|
||||
| `wiki-lint` | `lint` itself is the check | Sonnet | default |
|
||||
| `wiki-ingest`, `wiki-manage`, judgment-heavy `wiki-query` | `lint` and `docs verify`, partly | Sonnet | high |
|
||||
| Stack dev: design, the version part, a boundary-crossing judgment | nothing mechanical | Opus | high |
|
||||
| Stack dev: code, tests, mechanical doc sync, waiting for CI | `pytest`, `docs verify`, `instructions verify`, CI | Opus (open - see below) | high; medium for a small change |
|
||||
| Stack dev: closing an issue, `docs/` staleness, changelog prose | nothing, by construction | Opus | high |
|
||||
|
||||
The two unchecked rows are short - minutes, not hours - so keeping them on the strongest model
|
||||
costs little and protects the only work that fails silently. The checked middle row is where the
|
||||
tokens are, which makes it the tempting one to run cheaper. It is also the row with the least
|
||||
settled answer: a build phase of this stack reads a lot of the tree, and a smaller context window
|
||||
does not hold it - it runs into compaction, which costs more than the cheaper model saves. Sonnet
|
||||
at high effort remains possible, but only as a session of its own (below).
|
||||
|
||||
**Effort is the cheaper lever than the model.** A reduced effort level is what gives up
|
||||
multi-file consistency first, so `high` is a reasonable floor for anything touching more than one
|
||||
file or a contract; `default` or `medium` suits a small mechanical change with a test behind it.
|
||||
|
||||
## Model per session, not per phase
|
||||
|
||||
A session cannot switch its own model - that is the user's `/model` - and it should not be asked
|
||||
to mid-flow either. Two reasons:
|
||||
|
||||
- **A switch throws away the prompt cache.** A cache entry belongs to the model that wrote it,
|
||||
so a new model starts the session's whole history from cold. The same holds for effort:
|
||||
changing it always invalidates the cached message history - by far the largest part of a long
|
||||
session - and, on some models, the tool and system prefix too (Anthropic's prompt-caching
|
||||
documentation lists effort and the thinking configuration among what invalidates the cache). A
|
||||
switch from one effort to another costs the same re-read of the whole session as a switch of
|
||||
model.
|
||||
- **An offered switch is rarely taken.** A sentence in the output at the moment a phase changes
|
||||
is easy to read past - for the agent writing it and for the user reading it - and the session
|
||||
just carries on in whatever it started as.
|
||||
|
||||
So the choice is made once, when a session starts, and phases that want different models or
|
||||
effort levels are separated by a session boundary instead: `/clear`, then the next phase in a
|
||||
session started the right way. That is only cheap if the next phase does not depend on the
|
||||
previous session's context - which is why the handover has to live somewhere outside the session
|
||||
(an issue body, a page) and be kept current at fixed points, not reconstructed at the end.
|
||||
<!-- dist:strip-start -->
|
||||
In this repo the handover points are a ready issue body (design → build) and a green CI run with
|
||||
the body updated (build → closing); the two later skills can only be started by the user's slash
|
||||
command, so each phase change is a real stop at which that choice is made.
|
||||
<!-- dist:strip-end -->
|
||||
|
||||
## Subagent models
|
||||
|
||||
The `Agent` tool's `model:` parameter (`haiku`, `sonnet`, `opus`, `fable`) is a per-subagent
|
||||
choice a session *can* make on its own:
|
||||
|
||||
- Read-only search/lookup (an `Explore` agent, or a `general-purpose` agent doing pure
|
||||
retrieval): `haiku` - no judgment is being delegated, only retrieval.
|
||||
- A subagent that writes pages, reviews code, or decides something: leave `model:` off so it
|
||||
inherits the parent session's model.
|
||||
- A fork (`subagent_type: "fork"`) always inherits the parent's model; a `model:` override on a
|
||||
fork is ignored.
|
||||
|
||||
## `/code-review` effort
|
||||
|
||||
- A routine diff: `low` or `medium` - fewer, high-confidence findings are enough.
|
||||
- Gate code, the compiler, or a change about to ship in a version bump: `high` and up - broader
|
||||
coverage is worth it when the blast radius of a missed bug is a safety gate.
|
||||
- `ultra` is user-triggered and billed separately - worth recommending, not assuming.
|
||||
|
||||
## When it's unclear
|
||||
|
||||
- A task spans both a mechanical step and a judgment call: weigh it by the judgment call, not the
|
||||
mechanical one - the tooling carries the mechanical part regardless of which model supervises.
|
||||
- No row fits cleanly: Sonnet at high effort is a safer default than the most capable model at
|
||||
the highest effort. Under-provisioning where a check exists costs one worse answer once;
|
||||
reflexively over-provisioning is a standing cost every session pays.
|
||||
- Not sure whether a phase is checked: treat it as unchecked - a needless Opus phase costs money
|
||||
once, an unchecked Sonnet phase can ship something nobody looks at again.
|
||||
- The phase changed and the session runs on a model or effort the table would not pick: keep
|
||||
working, and cut the session at the next handover rather than switching mid-flow - never block a
|
||||
publish or an issue close on a choice the session cannot make itself.
|
||||
|
||||
## Scope
|
||||
|
||||
Specific to Claude Code: the model names, the `/code-review` dial and the `Agent` tool's `model:`
|
||||
override have no equivalent in this repo's other supported harnesses (Codex CLI, GitHub Copilot
|
||||
CLI, Mistral Vibe). Does not set the classifier model behind Claude Code's own `auto` permission
|
||||
mode - that is a harness internal, not a per-task choice this repo controls.
|
||||
@@ -0,0 +1,235 @@
|
||||
# Ownership and Templates
|
||||
|
||||
Chemenu ships two kinds of files side by side, and at a glance they look the same: both are
|
||||
plain markdown, both sit in the repo root or under `kb/`, both get read at session start. But a
|
||||
stack upgrade treats them completely differently. Some - [AGENTS.md](../../AGENTS.md),
|
||||
[kb/CONTRACT.md](../../kb/CONTRACT.md), the per-stage contracts - are identical in every instance
|
||||
that runs this stack and are the next release's to replace (with one caveat about local edits,
|
||||
below). Others - `USER.md`,
|
||||
`SOUL.md`, `kb/CONVENTIONS.md`, `ENVIRONMENT.md` - describe one particular instance, and
|
||||
overwriting them would silently erase a choice someone made on purpose.
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
|
||||
- [Two different kinds of truth](#two-different-kinds-of-truth)
|
||||
- [Why silent overwrite is the failure being designed against](#why-silent-overwrite-is-the-failure-being-designed-against)
|
||||
- [Why the boundary is a predicate rather than a list](#why-the-boundary-is-a-predicate-rather-than-a-list)
|
||||
- [Why a `.template`, not just an absent file](#why-a-template-not-just-an-absent-file)
|
||||
- [Where the file boundary used to strain](#where-the-file-boundary-used-to-strain)
|
||||
- [The consequence in practice](#the-consequence-in-practice)
|
||||
- [Why an instance comes only from a release](#why-an-instance-comes-only-from-a-release)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
## Two different kinds of truth
|
||||
|
||||
The stack-owned files describe how the tool works. `kb/CONTRACT.md` opens by saying it holds
|
||||
what `tools/wikitool` enforces or what follows mechanically from how it operates - see
|
||||
[kb/CONTRACT.md](../../kb/CONTRACT.md), lines 10-13. That kind of statement doesn't vary by
|
||||
instance: the compiler behaves the same way regardless of who is running it, so the sentence
|
||||
describing that behavior can be copied byte-for-byte into every checkout without becoming
|
||||
wrong anywhere.
|
||||
|
||||
The instance-owned files describe a choice: which language pages are written in, what tone the
|
||||
agent takes, who the operator is, which git remote is authoritative, which MCP servers are
|
||||
reachable. None of that follows from the tool's mechanics - two instances of the identical
|
||||
stack can answer all of these differently and both be correct. [AGENTS.md § Personalization](../../AGENTS.md#personalization)
|
||||
frames the split the same way for `kb/CONTRACT.md` versus `kb/CONVENTIONS.md`: "the split is by
|
||||
who may change the sentence, not by what it is about." A rule about page structure could in
|
||||
principle have been written per-instance too, but then every instance answering "not German" to
|
||||
setup would be hand-editing a file the stack also ships, and the next release update would
|
||||
hand the stack's file back to it, discarding the customization.
|
||||
|
||||
## Why silent overwrite is the failure being designed against
|
||||
|
||||
A stack update is meant to be a routine, low-risk operation: pull the latest release, get
|
||||
whatever fixes and features shipped since the last one. That only stays low-risk if the update
|
||||
knows which files it's allowed to touch. If `USER.md` or `kb/CONVENTIONS.md` were treated the
|
||||
same as `AGENTS.md` - shipped and periodically re-copied - an upgrade would quietly replace a
|
||||
description of *this* operator, in *this* language, with whatever placeholder or default the
|
||||
stack maintainers wrote. The damage wouldn't be loud: nothing crashes, the files still parse,
|
||||
the agent just starts acting on the wrong premises until someone notices the voice or the
|
||||
language changed.
|
||||
|
||||
Keeping the boundary at the file level, rather than trying to merge changes within a shared
|
||||
file, means an upgrade never has to guess which lines are "stack" and which are "instance" -
|
||||
the file itself already answers that.
|
||||
|
||||
## Why the boundary is a predicate rather than a list
|
||||
|
||||
For a while the boundary was written down as a list of paths - once in `dist_cmd.py`, once in
|
||||
the merge procedure a private instance was told to run by hand, and once in the check that
|
||||
procedure ended with. Three copies of one fact, which is the shape [AGENTS.md](../../AGENTS.md)
|
||||
invariant 8 exists to forbid, and they drifted exactly as predicted: the hand-run procedure was
|
||||
still naming three paths after the collection contracts had moved to the instance's side of the
|
||||
line, so it discarded upstream changes to files it had never heard of, while its own final check
|
||||
excluded the same three paths and therefore reported success.
|
||||
|
||||
`chemenu/ownership.py` replaced the lists with one question - is this path, under a content
|
||||
stage, the stack's or the instance's? - answered by shape rather than by enumeration:
|
||||
`<stage>/CONTRACT.md`, and anything ending `.template`. Every consumer asks it - at the time,
|
||||
`dist export` and a `wikitool upstream merge` that took the hand-run procedure's place; since
|
||||
that path was removed, the export alone - so no two of them can disagree, and a machinery file
|
||||
added under a content stage tomorrow is recognised without any of them being edited. The deeper point is not the
|
||||
deduplication: a list has to be maintained by whoever remembers it exists, and the failure mode
|
||||
when nobody does is silence, because a path the list has never heard of simply looks like
|
||||
content.
|
||||
|
||||
## Why a `.template`, not just an absent file
|
||||
|
||||
The mechanism for instance-owned content is a `.template` file the distribution ships instead
|
||||
of the real one - `USER.md.template`, `SOUL.md.template`, `kb/CONVENTIONS.md.template`,
|
||||
`ENVIRONMENT.md.template`. An alternative would have been to ship nothing at all and let a
|
||||
brand-new instance start from a blank page. The template exists because a blank page doesn't
|
||||
tell [instructions/setup-instance.md](../../instructions/setup-instance.md) what shape the answer
|
||||
should take, and it gives nothing for a validator to check afterward.
|
||||
|
||||
A template carries a placeholder value - a sentinel - in the fields that need a real answer.
|
||||
Setup interviews the operator and replaces the sentinel with what they actually said. That
|
||||
gives `doctor` a mechanical way to tell "personalized" from "not yet": a file that still
|
||||
contains the sentinel hasn't been through setup, regardless of whether the file exists. That's
|
||||
also why `ENVIRONMENT.md` only warrants a WARN rather than a FAIL when absent - see
|
||||
[AGENTS.md § Environment](../../AGENTS.md#environment) - while a missing or unfilled
|
||||
`USER.md`/`SOUL.md`/`kb/CONVENTIONS.md` is a harder failure: `ENVIRONMENT.md` describes one
|
||||
checkout among possibly several and is gitignored for that reason, so its absence is a normal
|
||||
state rather than a sign setup was skipped.
|
||||
|
||||
## Where the file boundary used to strain
|
||||
|
||||
"The file itself already answers that" held for every file above except one shape: a `root: kb`
|
||||
type-spec used to carry two audiences inside one file.
|
||||
|
||||
Its authoring guidance - when to use this type, what each frontmatter field means, how to cite -
|
||||
was instruction to an agent. It read like the stack's own prose because it *was* the stack's own
|
||||
prose: a later release that learned something about writing entity pages would want to improve it
|
||||
everywhere. Its `## Template` block and its `layout:` titles were the opposite: they became the
|
||||
literal headings of pages this instance writes, in the language this instance chose, and no
|
||||
release had any business touching them.
|
||||
|
||||
The same file is where the language question comes apart from the ownership one, and for the same
|
||||
reason: ownership decides who may change a line, its reader decides what language it is in -
|
||||
which is why a `root: kb` type-spec still keeps English prose around a template block written in
|
||||
its own language. [language-boundaries.md](language-boundaries.md) has that argument; this page is
|
||||
about ownership alone.
|
||||
|
||||
Ownership is per file, so a file carrying both audiences had to give both halves to whoever owned
|
||||
it. The template half was correct that way. The guidance half paid for it: an instance that
|
||||
adopted its type-specs at setup never received an improvement to the guidance again, because
|
||||
`dist upgrade` wrote the `.template` beside the adopted file and never the file itself. Nothing
|
||||
broke, and nothing reported it - the instance simply kept reading the guidance it was handed the
|
||||
day it was created.
|
||||
|
||||
That was not an argument against the per-file boundary; the boundary is what makes an upgrade
|
||||
safe at all, and merging inside a shared file is the failure the whole section above is about. It
|
||||
was an argument that this particular file was cut in the wrong place - so it was cut again. A
|
||||
`root: kb` type-spec may now declare `guidance:`, a repo-relative path to a second,
|
||||
stack-owned file (`types/<name>.guidance.md`) holding exactly the half that used to be stranded:
|
||||
when to use the type, when not to, and mechanism-level advice that holds for every instance. That
|
||||
file ships verbatim and upgrades like any other machinery file, whether or not the type-spec that
|
||||
links it has ever been adopted. `types/type-spec.md` §§ "Who owns a type-spec" and "Anatomy of a
|
||||
type" hold the current shape; `tools/wikitool types describe <name>` composes the type-spec and its
|
||||
guidance into one answer, so an agent asking for a type's contract never needs to know it comes from more than one
|
||||
file. An instance that adopted its type-specs before this split existed takes it as an *offered*
|
||||
migration rather than something an upgrade applies on its own - the same reasoning as any other
|
||||
instance-owned file in the middle category below, spelled out for this one case because it is the
|
||||
case that motivated the category existing at all.
|
||||
|
||||
A type-spec that declares no `guidance:` - one an instance writes entirely for itself - is
|
||||
unaffected: it is still described from its own body alone, the way every type-spec worked before
|
||||
`guidance:` existed. The split is optional exactly where there is no stack-owned improvement to
|
||||
receive.
|
||||
|
||||
## The consequence in practice
|
||||
|
||||
An upgrade sorts every shipped path into three categories, not two - and the third one only
|
||||
becomes visible once an upgrade is a command rather than a hand-run copy:
|
||||
|
||||
- **Verbatim files** - `AGENTS.md`, `kb/CONTRACT.md`, the per-stage contracts, everything under
|
||||
`tools/`, `types/` and `instructions/` - are the release's to replace. This is where a `root:
|
||||
kb` type-spec's optional `types/<name>.guidance.md` sits: verbatim, even though the type-spec
|
||||
it documents (below) is not.
|
||||
- **`.template`-sourced files** - `USER.md`, `SOUL.md`, `kb/CONVENTIONS.md`, each
|
||||
`kb/<name>/COLLECTION.md`, `ENVIRONMENT.md`, the `root: kb` type-specs and their subtype
|
||||
templates `types/<name>.<value>.md` - are never written by an upgrade at all. A subtype
|
||||
template is the guidance split run the other way: the file boundary cut once more, this time
|
||||
to give page material a file of its own, and page material belongs to the instance, so it
|
||||
lands here rather than among the verbatim files - and because it is its own file, a release
|
||||
can ship a new one without touching an adopted type-spec. The distribution ships only the `.template` beside them, so the filled file
|
||||
is out of reach by construction rather than by a rule someone has to remember. The same
|
||||
property has a second face on the way in: when a release ships a `.template` for a type or
|
||||
collection the instance does not have *yet*, the upgrade writes the template and stops - it
|
||||
cannot write the filled file without deciding the instance's own language and wording for it.
|
||||
Adoption is therefore an act the instance performs, and where the stack *requires* that type
|
||||
(the `source` idiom, and `project` since 7.0.0) an upgrade that skips it leaves a tree
|
||||
`docs verify` refuses. That is the ownership boundary working rather than a gap in it, but it
|
||||
is the one shape in which "the upgrade never writes this file" turns into work somebody has to
|
||||
do; `instructions/upgrade-instance.md` carries the step.
|
||||
- **Seeded-once files** - `.wikitool-kb.json`, `CHANGES.md`, `kb/log.md`, `raw/.gitkeep` - are
|
||||
written into a *new* instance by `dist export` and belong to the instance from then on. They
|
||||
are the awkward category: they sit in the release stamp's file list like any other shipped
|
||||
file, so an upgrade has to exclude them deliberately (`chemenu.ownership.is_export_stub` and
|
||||
`is_upgrade_preserved`). An upgrade that re-seeded them would reset the record of which
|
||||
migrations ran, or erase the changelog the instance wrote for itself.
|
||||
|
||||
The first category carries a caveat that the word "verbatim" hides. It says who *decides* the
|
||||
content, not that overwriting is always safe: an instance can still have edited a verbatim file
|
||||
- a patched `tools/`, a locally adjusted instruction - and an upgrade assuming otherwise would
|
||||
destroy that silently. Avoiding that assumption is the whole reason `dist export` records a
|
||||
sha256 per shipped file in `.wikitool-release.json`. `wikitool dist upgrade` compares every
|
||||
candidate path against the digest recorded when it was installed, overwrites only what still
|
||||
matches, and refuses rather than overwrite what does not.
|
||||
|
||||
A generated verbatim file adds one more way to touch it without meaning to. The reference pages
|
||||
under `docs/reference/` are verbatim, and `wikitool docs reference --apply` rewrites their
|
||||
generated regions in whatever tree it runs in. Most of them render records in `tools/` - the
|
||||
command records, and the shapes of the configuration files an operator writes - which an instance
|
||||
leaves alone, so they come out the same everywhere. The skill
|
||||
catalogue and the page types would render material an instance does own: its adopted type-specs
|
||||
and any instructions it added. Regenerated there, they would turn a shipped page into a locally
|
||||
changed one, and the next upgrade would refuse to replace it. So those two pages describe the
|
||||
stack's shipped defaults. They are generated and checked only in the origin repo, the one tree
|
||||
without a release stamp, and an instance leaves them alone and asks `wikitool types describe` or
|
||||
`wikitool instructions list` for its own state.
|
||||
|
||||
So the practical rule is narrower than "overwrite the verbatim files, leave the rest alone":
|
||||
overwrite the verbatim files *this instance has not touched*, never write the other two
|
||||
categories, and make a locally changed file a decision someone takes deliberately instead of
|
||||
one an upgrade takes for them. The template-sourced files were filled in once, by a person, for
|
||||
a reason, and nothing about a newer release of the stack's mechanics gives it standing to
|
||||
override that.
|
||||
|
||||
The same rule runs in the other direction, for a file a release stops shipping, and there it has
|
||||
a deadline the overwrite side does not. The stamp an upgrade writes is the new release's, and the
|
||||
path is no longer in it - so the next upgrade cannot tell that file from one the instance wrote
|
||||
itself, and an instance-owned file is exactly what an upgrade must never touch. Whatever happens
|
||||
to a retired file therefore happens in the run that sees it go, or not at all. An unchanged one is
|
||||
deleted as silently as an unchanged one is overwritten; a changed one is a local change like any
|
||||
other and needs the same deliberate answer, where keeping it makes it the instance's own for
|
||||
good. Leaving the decision optional, as it once was, did not defer it: it made it, the wrong way,
|
||||
and left orphans behind - harmless under `tools/`, where nothing discovers modules by listing a
|
||||
directory, and not harmless under `instructions/` or `types/`, where an orphaned skill keeps
|
||||
being published and an orphaned type-spec keeps being a type.
|
||||
|
||||
## Why an instance comes only from a release
|
||||
|
||||
Everything above depends on one file every instance carries: the `.wikitool-release.json` its
|
||||
release wrote. It is the base `dist upgrade` classifies against, the marker that turns telemetry
|
||||
off for someone who never asked for it, and the record of which stack version the instance runs.
|
||||
An instance that starts anywhere else starts without that base, and every later step has to
|
||||
reconstruct the boundary by other means.
|
||||
|
||||
For a while there were four ways in: a release, a `dist export` from a checkout of the origin
|
||||
repository, a clone of that repository, and a private clone that kept the origin as a git
|
||||
`upstream` and took stack updates by merging. The last two never had the stamp, so they needed
|
||||
the boundary a second way. The clone took the origin's demo corpus, demo persona and development
|
||||
skills with it and had to be emptied by hand, and the instruction for doing so neither said what
|
||||
had to survive nor fitted into the iteration budget. The merge path needed its own
|
||||
ownership-aware command, `upstream merge`, which shipped two data-destroying bugs before it was
|
||||
right, and still left a checkout with two remotes and no stamp. None of the four was in use when
|
||||
they were cut down to one in 8.0.0.
|
||||
|
||||
What is left is a single shape. A release is an export packed as a tarball, installed into an
|
||||
empty folder - or an empty clone of the instance's own repository - by a script attached to the
|
||||
same release. `dist export` remains, as the tool that builds a release and tests what one would
|
||||
ship, not as a way to install. A clone of the origin repository remains too, as the place the
|
||||
stack is developed, and is never an instance.
|
||||
@@ -0,0 +1,78 @@
|
||||
# Why the pipeline has four stages
|
||||
|
||||
Chemenu could, in principle, be one directory: drop a file in, ask a question, get an answer
|
||||
computed fresh each time. It isn't built that way. The pipeline in
|
||||
[AGENTS.md](../../AGENTS.md#routing) - `raw/` -> `[types/ + tools/]` -> `kb/` -> `reports/`, with
|
||||
`work/` alongside rather than inside it - separates *material* from *meaning* from
|
||||
*byproduct*, and each seam exists because collapsing it costs something specific.
|
||||
|
||||
## Why raw material stays untouched
|
||||
|
||||
[raw/CONTRACT.md](../../raw/CONTRACT.md) keeps a source exactly as it arrived. The reasoning is
|
||||
simple once stated: the moment someone "cleans up" or reformats a source on the way in, the
|
||||
thing later claims get checked against is no longer the thing that was actually said. An
|
||||
immutable `raw/` means a citation always resolves to the original, not to somebody's tidied
|
||||
memory of it. It also draws a trust boundary in one place instead of scattering it - everything
|
||||
past `raw/` can be treated as reviewed, because nothing upstream of it silently already was.
|
||||
`incoming/` sits entirely on the near side of that boundary: a file waiting there is
|
||||
not yet reviewed and not yet a citation target, so its being gitignored and readable by an
|
||||
ingest session does not weaken anything - the boundary is the promotion into `raw/` itself, not
|
||||
the moment a human happened to drop a file somewhere.
|
||||
|
||||
Once a document can arrive from *outside* - the MCP server's optional `submit` tool - "a human
|
||||
happened to drop a file somewhere" stops describing how everything reaches `incoming/`, so the
|
||||
near side of the boundary gets a stage of its own rather than a second meaning. `mcp-upload/`
|
||||
holds what nobody has looked at yet; `incoming/` holds what someone has. Two arrows, two
|
||||
different things being granted: `upload accept` grants *trust* (a human read the material and
|
||||
took responsibility for it), `raw accept` grants *immutability* (it becomes a citation target
|
||||
and stops being editable). Collapsing them would have meant one of the two lying - either an
|
||||
unreviewed stranger's file sitting in the same directory a human's own drop does, or the
|
||||
promotion into `raw/` quietly doubling as the review step it cannot perform.
|
||||
|
||||
## Why extraction happens once, through a schema
|
||||
|
||||
[types/type-spec.md](../../types/type-spec.md) is what stands between a raw file and a `kb/` page:
|
||||
a type-spec defines what a conforming instance of a page looks like, and the compiler
|
||||
(`tools/wikitool`) applies it. The alternative - every query re-reading and re-interpreting the
|
||||
source on demand - would mean paying the cost of understanding the material every single time,
|
||||
and getting a slightly different answer each time depending on how the question was phrased.
|
||||
Extracting once, against a fixed schema, turns "re-read and re-guess" into "look up what was
|
||||
already compiled." That is the "never re-derive, always compile" principle from
|
||||
[AGENTS.md](../../AGENTS.md): understanding a source is expensive and worth doing exactly once,
|
||||
after which it becomes a cheap, stable lookup.
|
||||
|
||||
## Why a `kb/` page has to stand on its own
|
||||
|
||||
[kb/CONTRACT.md](../../kb/CONTRACT.md) sets the bar for the compiled layer: a page should answer a
|
||||
future question without sending the reader back to the source it came from. That's the payoff
|
||||
of compiling in the first place - if every answer still bottomed out in "go re-read the raw
|
||||
file," the `kb/` layer would just be a pointer with extra steps, and the cost of extraction
|
||||
would have bought nothing. A page that stands alone is what makes the corpus fast and
|
||||
consistent to query: the work of understanding is already sitting there, done.
|
||||
|
||||
## Why `reports/` doesn't need to be maintained
|
||||
|
||||
[reports/CONTRACT.md](../../reports/CONTRACT.md) treats most of what lands in `reports/` -
|
||||
lint output, telemetry traces - as disposable. The structural content of a lint report can be
|
||||
recomputed from the tree at any commit, so keeping an old copy around would just be a second
|
||||
version of something the tool can already answer on demand, and a second copy is exactly the
|
||||
kind of thing that quietly goes stale. Treating it as derived output rather than a fourth thing
|
||||
to maintain means there is nothing there to fall out of sync - regenerating it is cheaper than
|
||||
reconciling it. The one part that genuinely can't be recomputed - the judgment a pass produced -
|
||||
is carried out into `kb/` or `kb/log.md` before the report itself is discarded, which is the
|
||||
distinction between what's recomputable and what isn't.
|
||||
|
||||
## Where `work/` fits
|
||||
|
||||
[work/CONTRACT.md](../../work/CONTRACT.md) describes a workshop, not a fifth pipeline stage: a
|
||||
place for the notes, extracts and open decisions of a task that spans more than one session, on
|
||||
its way toward becoming a `kb/` page. It sits beside the raw -> kb -> reports flow rather than
|
||||
inside it - closer in spirit to a desk than to a conveyor belt.
|
||||
|
||||
## The shape this produces
|
||||
|
||||
Four stages, each answering a different question: `raw/` - what was actually said; `types/` +
|
||||
`tools/` - how to turn that into structured understanding; `kb/` - what is now known;
|
||||
`reports/` - what a pass over the corpus noticed in passing. Keeping them separate is what lets
|
||||
each one be trusted for what it is, instead of every layer having to double as all four at
|
||||
once.
|
||||
@@ -0,0 +1,354 @@
|
||||
# Telemetry and evaluation
|
||||
|
||||
How the stack observes what an agent did, and how that record turns into a score. This page explains
|
||||
the design and points at the code. The rules an agent follows live in
|
||||
[tools/CONTRACT.md](../../tools/CONTRACT.md) and [reports/CONTRACT.md](../../reports/CONTRACT.md).
|
||||
Turning tracing on and reading a session back is a how-to:
|
||||
[trace-and-score-sessions.md](../how-to/trace-and-score-sessions.md).
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
|
||||
- [Why, beyond the unit tests](#why-beyond-the-unit-tests)
|
||||
- [Architecture](#architecture)
|
||||
- [The trace](#the-trace)
|
||||
- [Whether it runs at all](#whether-it-runs-at-all)
|
||||
- [What never reaches a trace](#what-never-reaches-a-trace)
|
||||
- [Evaluation levels](#evaluation-levels)
|
||||
- [L0 lives in the test suite, not in a separate harness](#l0-lives-in-the-test-suite-not-in-a-separate-harness)
|
||||
- [Scoring a session](#scoring-a-session)
|
||||
- [A worked example, and where the check moved to](#a-worked-example-and-where-the-check-moved-to)
|
||||
- [Status](#status)
|
||||
- [The agent runner, and why it is not here yet](#the-agent-runner-and-why-it-is-not-here-yet)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
## Why, beyond the unit tests
|
||||
|
||||
The test suite that ships with the origin repository checks the **compiler**: given this input, does
|
||||
`wikitool` produce that output? It says nothing about the two things that actually go wrong in
|
||||
practice: whether the *agent* followed the contracts, and whether the pages it wrote are any good.
|
||||
|
||||
Those need a different kind of check. Two notes in the
|
||||
[Commonplace](https://github.com/zby/commonplace) research base, which the origin repository
|
||||
vendors, supply the theory:
|
||||
|
||||
- *oracle-strength-spectrum* - correctness checks form a gradient, from hard (deterministic) to none
|
||||
(vibes). The engineering move is to *harden* oracles step by step, not to reach straight for a
|
||||
judge.
|
||||
- *evaluation-automation-is-phase-gated-by-comprehension* - comprehension, then specification,
|
||||
then generalisation. A judge built before anyone has read real failures optimises a proxy.
|
||||
|
||||
That ordering is why this page describes a lot of telemetry and only a little scoring. Reading real
|
||||
traces is the first phase, and it cannot be skipped.
|
||||
|
||||
## Architecture
|
||||
|
||||
Three sources, three different jobs.
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
subgraph runner["Eval runner (planned)"]
|
||||
R["isolated HOME · programmatic mode<br/>NDJSON capture · run manifest"]
|
||||
end
|
||||
subgraph hooks["Harness hooks (interactive work)"]
|
||||
H1["Claude Code<br/>.claude/settings.json"]
|
||||
H2["Copilot CLI<br/>.github/hooks/*.json"]
|
||||
H3["Mistral Vibe<br/>.vibe/hooks.toml"]
|
||||
H4["VS Code Chat<br/>chronicle SQLite, post hoc"]
|
||||
end
|
||||
subgraph inner["Repo layer (always on)"]
|
||||
W["wikitool emitter<br/>+ git"]
|
||||
end
|
||||
R --> T[("reports/telemetry/<session>/trace.jsonl")]
|
||||
H1 & H2 & H3 & H4 --> I["tools/trace_ingest.py"] --> T
|
||||
W --> T
|
||||
T --> S["scorers L0-L4"] --> O[("reports/evals/<date>/")]
|
||||
```
|
||||
|
||||
- **The repo layer is the truth.** `wikitool` records its own calls, so what happened *to the wiki*
|
||||
is known even when no hook fired and no runner was involved.
|
||||
- **The runner owns session boundaries.** Not every harness reports a session start - Mistral Vibe
|
||||
has no such hook - so the process that launches the agent is what brackets a run.
|
||||
- **Hooks enrich.** They add the tool calls the repo layer cannot see: file reads, greps, shell
|
||||
commands, prompts.
|
||||
|
||||
Everything joins on one session id, which every source that has to pick one resolves the same way
|
||||
(`chemenu.session`). The chain is `WIKITOOL_SESSION_ID`, then a harness's own session variable where
|
||||
one is registered (`chemenu.session.HARNESS_ENV_VARS` - Claude Code's `CLAUDE_CODE_SESSION_ID` today),
|
||||
then the parent process id. The middle step exists because the last one does not survive a harness
|
||||
that runs every tool call in its own freshly started shell. There, `os.getppid()` is a new "session"
|
||||
for every call, and neither the join nor the Iteration Budget Gate could see more than one or two
|
||||
calls of a real run. A harness only earns an entry in that chain once a live session has been
|
||||
observed setting the variable, and it has been confirmed to be the exact id the harness's own hooks
|
||||
write elsewhere in a trace. A name that merely looks plausible would mis-key a session more quietly
|
||||
than the pid fallback it replaced.
|
||||
|
||||
A trace hook that only *observes* tool calls (a `PreToolUse`/`PostToolUse`-style wiring) does not by
|
||||
itself fix a harness whose events carry a different id than `wikitool`'s own emitter. The two would
|
||||
still not join. Wiring such a hook is worth doing only once this fallback chain already keys both
|
||||
sides on the same id.
|
||||
|
||||
## The trace
|
||||
|
||||
One JSON object per line, appended to `reports/telemetry/<session>/trace.jsonl`. The contract is
|
||||
[tools/chemenu/telemetry/schema.py](../../tools/chemenu/telemetry/schema.py).
|
||||
|
||||
| Field | Meaning |
|
||||
|---|---|
|
||||
| `v` | Schema version |
|
||||
| `ts` | ISO-8601 UTC, microsecond precision |
|
||||
| `session_id` | The join key - `chemenu.session`'s fallback chain: `WIKITOOL_SESSION_ID`, else a registered harness variable, else the parent process id |
|
||||
| `pid`, `seq` | `seq` counts **within one process**. Sort a trace by `(ts, pid, seq)` |
|
||||
| `source` | `wikitool`, `runner`, or a harness name |
|
||||
| `event` | See below |
|
||||
| `attrs` | Normalised payload - the same field names whichever harness produced it |
|
||||
| `run_key` | The `work/<runkey>/` run, when one is open |
|
||||
| `trace_id`, `span_id` | From `TRACEPARENT`, when the harness exports it |
|
||||
| `redactions` | Which secret patterns fired on this event |
|
||||
|
||||
**Events.** The core - `tool.pre`, `tool.post`, `wikitool.call`, `gate.refused` - is available on
|
||||
every surface. Everything else is optional: `session.start`, `session.end`, `session.error`,
|
||||
`prompt.submitted`, `assistant.message`, `turn.end`, `tool.error`, `instructions.loaded`,
|
||||
`subagent.start`, `subagent.stop`, `compaction`, `page.written`, `publish.commit`, `budget.state` and
|
||||
`gate.cleared`.
|
||||
|
||||
**The degradation rule.** No scorer may *require* an optional event. Claude Code has about thirty
|
||||
hooks and Mistral Vibe has three. A scorer built on the rich end would silently report zero on the
|
||||
poor end, which reads as "the agent did nothing" rather than "this harness cannot say". Every
|
||||
`session.start` carries a `completeness` list naming the classes its harness can emit, so a scorer
|
||||
can answer "not measurable here" instead. What each harness can report is tabled in
|
||||
[trace-and-score-sessions.md](../how-to/trace-and-score-sessions.md#what-each-harness-can-report).
|
||||
|
||||
**Budget and trace are not the same set.** The Iteration Budget Gate exempts read-only retrieval,
|
||||
and the trace records it. What an agent looked at before acting is exactly what a trajectory scorer
|
||||
needs, and charging for a `search` would discourage the one habit that lowers cost.
|
||||
|
||||
## Whether it runs at all
|
||||
|
||||
The default depends on how the tree got here, not on one hard-coded switch.
|
||||
[tools/chemenu/telemetry/policy.py](../../tools/chemenu/telemetry/policy.py) is the one place that
|
||||
resolves it, so `wikitool doctor`, the writer and the MCP server's start-up guard all answer the
|
||||
same question the same way:
|
||||
|
||||
| Installation form | Default | Marker |
|
||||
|---|---|---|
|
||||
| Git clone of the origin repository (dev checkout) | **on** (opt-out) | No `.wikitool-release.json` |
|
||||
| An instance installed from a release, and every clone of its own repository | **off** (opt-in) | `.wikitool-release.json` present |
|
||||
|
||||
The form is read off `.wikitool-release.json`, the same stamp `version check`, `dist upgrade` and
|
||||
`version notes` already use to tell a distribution from the repository it came from. Present means
|
||||
an operator never asked for telemetry. Absent means this is the dev checkout the stack ships from,
|
||||
where the traces are the stack's own measuring instrument. An instance commits the stamp with its
|
||||
first `publish`, so a clone of it on a second machine carries the stamp and defaults to off too. It
|
||||
is a consuming instance, not a measuring stand.
|
||||
|
||||
**Turning it on for a distributed instance** takes a per-checkout `.wikitool-telemetry.json` at the
|
||||
root, gitignored like `.wikitool-remotes.json` and for the same reason. The consent to write
|
||||
prompts in cleartext to *this* disk belongs to the checkout, not the corpus, so a second clone must
|
||||
not inherit it silently. The setup instruction asks about it; nothing writes it automatically. Its
|
||||
shape is in the [configuration reference](../reference/configuration.md#telemetry).
|
||||
|
||||
**Two independent quantity caps, both enforced fail-silent in `emit()`.** Neither is enforced in
|
||||
`write_event()`, which the test suite calls directly to exercise the format without the policy
|
||||
around it.
|
||||
|
||||
- **A byte cap per session trace** (default 5 MiB), checked with one `stat` before every append.
|
||||
Once a trace is at or over the cap, further calls in that session write nothing except a single
|
||||
`telemetry.limit` event. It is elected by the same single-writer trick `session.start` uses, an
|
||||
exclusive create on a `.limit` marker file, so a trace that was cut off can be told apart from one
|
||||
whose writer simply crashed.
|
||||
- **Retention by session count** (default 250), applied once, right before a brand-new session
|
||||
directory is created. It is never applied per event, and never against the session doing the
|
||||
creating. It deletes exactly `trace.jsonl` and `.limit` from the oldest directories beyond the cap,
|
||||
and `rmdir`s a directory only once it is empty. Nothing under `reports/` is ever removed in bulk.
|
||||
|
||||
The default of 250 was set above what the origin repository's own checkout had accumulated as of
|
||||
2026-09-10: 231 session directories, well under 400 KiB in total. The cap bites on future growth,
|
||||
not on the existing history.
|
||||
|
||||
## What never reaches a trace
|
||||
|
||||
Prompts and assistant replies **are** recorded in cleartext, locally. A failure taxonomy cannot be
|
||||
read out of hashes, and building one is the first phase of any evaluation work. Four guards make
|
||||
that defensible, all of them in
|
||||
[tools/chemenu/telemetry/scrub.py](../../tools/chemenu/telemetry/scrub.py), so there is one place to
|
||||
audit:
|
||||
|
||||
1. **Secret scrubbing.** Tokens, keys, `Authorization:` headers and `SECRET=` assignments are
|
||||
replaced with `[REDACTED:<type>]`. The scrubbing is pattern-based, so it is best effort.
|
||||
2. **A content cap.** 60 KiB per attribute, with a `[TRUNCATED n chars]` marker.
|
||||
3. **A kill switch.** `WIKI_TRACE_CONTENT=0` keeps only `<field>_length` and `<field>_sha256`. The
|
||||
digest is computed either way, so traces stay comparable.
|
||||
4. **`raw/` contents never enter a trace at all.** That text is data, not instruction
|
||||
([AGENTS.md](../../AGENTS.md) invariant 4), and a trace gets read back later. Callers record a path
|
||||
and a digest.
|
||||
|
||||
Nothing leaves the machine. `reports/` is gitignored, no exporter is configured, and where a vendor
|
||||
offers one it is off. Mistral's `enable_otel` ships prompts to Mistral Studio, so it stays `false`,
|
||||
and so does `enable_telemetry`. Claude Code's and Copilot's content gates may only be enabled
|
||||
against a collector you run yourself.
|
||||
|
||||
The variables that steer all of this are in the
|
||||
[configuration reference](../reference/configuration.md#environment-variables).
|
||||
|
||||
## Evaluation levels
|
||||
|
||||
Ordered by oracle strength: hard checks first, judgment last.
|
||||
|
||||
| Level | Oracle | What it measures | Status |
|
||||
|---|---|---|---|
|
||||
| **L0** Pipeline | hard | A wiki the tools built themselves lints clean, and the catalog is a fixed point | **done** - in the origin repository's test suite |
|
||||
| **L1** Artifact scorecard | hard | Counters from `lint`'s own checks: hard errors and advisories, page count | **done** |
|
||||
| **L2** Trajectory | hard | Rules over the trace: was a refused call repeated, was a gate flag passed unearned, did a page change go unlogged | **done** |
|
||||
| **L3** Task evals | medium | Gold set: question → expected cited pages; ingest fixture → expected page titles. Scored by set overlap | needs the runner |
|
||||
| **L4** Rubric / judge | soft | Prose quality, cramming, tone | **out of scope** until a failure taxonomy exists |
|
||||
|
||||
Evaluation results are statistical, not binary. A case runs several times and reports a pass rate
|
||||
with its variance, because sampling is not deterministic. A single failure is not a merge blocker;
|
||||
a regression against a baseline is.
|
||||
|
||||
### L0 lives in the test suite, not in a separate harness
|
||||
|
||||
It runs the CLI in-process against an empty wiki. It asserts that `new` → write → `xref` →
|
||||
`index rebuild` leaves a tree `lint` calls clean, and that rebuilding the catalog again changes
|
||||
nothing on disk. That is a hard oracle over the compiler, which is what the test suite is for.
|
||||
Giving it its own runner would have duplicated the suite for nothing.
|
||||
|
||||
One behaviour it pins is easy to mistake for a defect: **a scaffolded page does not lint clean**.
|
||||
`new` writes placeholder wikilinks for the author to replace. So a page that was created but not yet
|
||||
written reports broken links, and reports its `TODO`-only sections as the advisory
|
||||
*Unfilled Template Sections*. That is the scaffold saying it is unfinished.
|
||||
|
||||
## Scoring a session
|
||||
|
||||
`tools/wikitool eval score` scores one session. It only reads, and it is exempt from the Iteration
|
||||
Budget Gate: reading back what a session did is not iterating on the wiki, and charging for it
|
||||
would discourage checking one's own work.
|
||||
|
||||
**L1** re-runs `lint`'s checks in-process and reports their counters. It shares its definition of a
|
||||
hard error with `lint --fail-on-error` - one constant, `HARD_ERROR_KEYS` - so a run can never pass
|
||||
its score while lint refuses it.
|
||||
|
||||
**L2** checks five rules. Each one restates an invariant the code cannot enforce in-process. A gate
|
||||
can refuse a call, but nothing stops an agent from calling again with the gate's own flag. That gap
|
||||
is the whole point:
|
||||
|
||||
| Rule | Invariant | Severity |
|
||||
|---|---|---|
|
||||
| `refusal-not-retried` | A refused call, repeated unchanged, is the loop the gate exists to break | error |
|
||||
| `gate-not-self-opened` | `--yes`/`-y` no longer exist at all; `--override-budget` is for a human to pass after a refusal; `--force` never | error |
|
||||
| `content-change-logged` | A publish that changes `kb/` pages needs a `log append` in the same session | advisory |
|
||||
| `clearance-was-asked-for` | A `gate.cleared` token must match one some earlier `gate.refused` issued - catches an invented token, and one reused from a different changeset | error |
|
||||
| `clearance-ended-the-turn` | No `wikitool.call` between a clearance request (exit 42) and the next `prompt.submitted` - skipped, not failed, on a harness that cannot report `prompt.submitted` | error |
|
||||
|
||||
New rules belong here only when a real trace shows a real failure. Checks invented from the contract
|
||||
text produce a score that improves while behaviour does not - the failure mode the phase-gate note
|
||||
describes. The first three rules were chosen because each guards a refusal an agent can talk its way
|
||||
around. The last two carry the load for the Mass-Update Gate's clearance mechanism (2026-08-28),
|
||||
which deliberately enforces very little in-process, as described below.
|
||||
|
||||
A rule can report a third state besides pass and fail: **skipped**, meaning this trace cannot answer
|
||||
the question at all - for example `clearance-ended-the-turn` on a harness with no prompt hook.
|
||||
Skipped reads as neither a pass nor a violation. `scorecard.render_markdown` marks it `skip`,
|
||||
distinct from `ok`, `warn` and `FAIL`, and `score()["violations"]` never includes a skipped rule,
|
||||
whatever its severity.
|
||||
|
||||
### A worked example, and where the check moved to
|
||||
|
||||
Scoring the session that shipped the telemetry layer itself used to report a failure:
|
||||
|
||||
```
|
||||
FAIL gate-not-self-opened
|
||||
call=publish --message "tools: session traces …" --yes, reason=no mass-update refusal preceded it
|
||||
```
|
||||
|
||||
That finding was true and benign at the same time. The change was 17 files, the file list was shown,
|
||||
and a human approved it. But the approval happened in conversation, and `--yes` went out on the
|
||||
first attempt, so the gate never got to refuse anything. The documented path had the refusal first,
|
||||
and the trace records tool calls, not conversations.
|
||||
|
||||
Three sessions did some version of this before the mechanism changed rather than the procedure. The
|
||||
interesting part is *which* half changed. The obvious fix was built first: make the gate impossible
|
||||
to open without independent proof of a human. That meant a ticket bound to the changeset, redeemed
|
||||
only after a `prompt.submitted` event and a `wikitool approve` typed in a second terminal. It worked,
|
||||
and it was unusable. On a harness that drives `wikitool` through a shell tool, the agent cannot
|
||||
produce the turn signal, and the human is asked to leave the conversation to approve anything at all.
|
||||
A gate nobody can pass is not a safe gate, it is an abandoned one.
|
||||
|
||||
So the enforcement got cheap and the *verification* got real:
|
||||
|
||||
- **In-process, `publish` enforces only what it can do honestly.** Exit 42 says "a human must see
|
||||
this". `--confirm <token>` says "here is what they saw", and the token digests the file list, so it
|
||||
cannot silently cover a different one. None of that proves a human was involved, and the code says
|
||||
so in as many words.
|
||||
- **The trace answers the question the tool cannot.** `clearance-ended-the-turn` looks for a
|
||||
`wikitool.call` between an exit-42 call and the next `prompt.submitted`. That is exactly the
|
||||
"refused, then confirmed in the same breath" shape all three incidents had, and it is invisible to
|
||||
any single call. `clearance-was-asked-for` catches a token that no refusal ever issued.
|
||||
|
||||
This is the loop this whole page is about, run to its conclusion: a rule that only a trace could
|
||||
check, a trace that showed it being broken three times, a mechanism rebuilt because no procedural fix
|
||||
survived - and then the check itself moved to the layer that can actually see it.
|
||||
|
||||
**What this still cannot see:** an agent that receives the exit-42 output, does not relay it, and
|
||||
passes the token anyway *after* the user happens to say something unrelated. `prompt.submitted`
|
||||
proves a turn boundary, not that the output was relayed or that the reply meant yes. That gap is
|
||||
recorded rather than papered over. Closing it needs the harness to report what the agent actually
|
||||
said, which no adapter here does yet.
|
||||
|
||||
That gap stopped being hypothetical within the hour. The first agent to receive the new gate - the
|
||||
one that had just written the paragraph above - answered the user with a file *count* and a pointer
|
||||
to "the output above". On that harness the user could not see the output: a command's stdout goes
|
||||
to the agent's context, not to anyone's screen. Nothing in the trace distinguishes that from a
|
||||
correct relay, and nothing ever will. The response was to fix the half that *is* fixable, the
|
||||
wording. The message now says "THE USER CANNOT SEE THIS OUTPUT", asks for the paths to be copied
|
||||
into the reply, and names the near-misses that do not count: a count, a summary, "the output above".
|
||||
An instruction that conflates printing with showing reads as already satisfied by the text existing.
|
||||
That is a general lesson about writing for agents, not a detail of this gate.
|
||||
|
||||
## Status
|
||||
|
||||
| Piece | State |
|
||||
|---|---|
|
||||
| Event schema, scrubber, writer | done - `tools/chemenu/telemetry/` |
|
||||
| `wikitool.call` on every command | done - one hook point in `cli.py`, next to the budget gate |
|
||||
| `gate.refused` (loop-breaker, iteration budget, mass-update) | done |
|
||||
| `publish.commit` | done |
|
||||
| Hook entry point | done - `tools/trace_ingest.py`, mappings for all three hook-capable harnesses |
|
||||
| Copilot CLI hooks | done - `.github/hooks/wiki-trace.json` |
|
||||
| VS Code chronicle import | done - `tools/import_chronicle.py` |
|
||||
| Mistral Vibe hooks | done - `.vibe/hooks.toml`, `.vibe/config.toml` |
|
||||
| Session header seeded on every trace | done - a trace declares its own `completeness` even without a session hook |
|
||||
| Scoring L1 + L2 | done - `wikitool eval score` |
|
||||
| L0 pipeline check | done - in the test suite |
|
||||
| Claude Code hooks | **partial** (2026-08-28) - `.claude/settings.json` wires `UserPromptSubmit` (the turn boundary `clearance-ended-the-turn` scores against) plus a `permissions.ask` rule on the `--confirm` form of `publish`; `PreToolUse` verified unable to force a prompt over `permissions.allow` |
|
||||
| Claude Code OTel | not started; not yet verified against the live CLI |
|
||||
| Agent runner (`eval run`) and L3 | designed below, not built |
|
||||
|
||||
## The agent runner, and why it is not here yet
|
||||
|
||||
Scoring reads what a session left behind. The missing half is *starting* one: a runner that launches
|
||||
a harness against a task, several times, and reports a pass rate. What it has to do is no longer
|
||||
guesswork, because the adapter work settled most of it:
|
||||
|
||||
- **Give each run a copy of the repository, not a fixture directory.** `config.ROOT` is derived from
|
||||
`chemenu/config.py`'s own location, so `wikitool` cannot be aimed at another tree from outside. An
|
||||
agent case therefore needs a worktree or a clone, where the tool sits inside the tree it edits. (A
|
||||
*fixture* directory works fine in-process, which is why L0 is a test.)
|
||||
- **Give each run its own HOME:** `VIBE_HOME`, `COPILOT_HOME`, plus `WIKI_TRACE_DIR` and
|
||||
`WIKITOOL_SESSION_ID`, so that state, trust decisions and traces cannot leak between runs.
|
||||
- **Never launch Vibe without `--agent`.** Its programmatic mode falls back to `auto-approve` when no
|
||||
agent is named, and it does not ask about folder trust. The runner must refuse rather than inherit
|
||||
that default.
|
||||
- **Record a run manifest:** harness and version, model, git SHA, and hashes of `AGENTS.md`, the
|
||||
published skills and the fixture. Without it, a pass rate cannot be compared to anything.
|
||||
|
||||
What is missing is not the design but the ability to check it. No provider credentials were
|
||||
configured on the machine the design was worked out on, so a live run could not be executed, let
|
||||
alone verified. Writing the runner anyway would repeat exactly the mistake the Vibe adapter avoided:
|
||||
the hook format there was wrong in the documentation, and only the installed CLI showed it. A runner
|
||||
written against an unverified mental model of three harnesses would be worse.
|
||||
|
||||
Open questions: whether Claude Code has a redirectable config directory (only `--settings` is
|
||||
documented), and whether the scrubber's own patterns are enough, or a dedicated secret scanner
|
||||
belongs in the verification step.
|
||||
@@ -0,0 +1,137 @@
|
||||
# Why the stack version splits compatibility from migration
|
||||
|
||||
A stack version number looks like it answers one question. It actually answers two, and the two
|
||||
are independent of each other.
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
|
||||
- [Two questions, not one](#two-questions-not-one)
|
||||
- [Why "kb/ untouched" is not proof of anything](#why-kb-untouched-is-not-proof-of-anything)
|
||||
- [Reading compatibility off the leftmost non-zero component](#reading-compatibility-off-the-leftmost-non-zero-component)
|
||||
- [Downgrade is half the promise](#downgrade-is-half-the-promise)
|
||||
- [A promise made to a machine, not only to a person](#a-promise-made-to-a-machine-not-only-to-a-person)
|
||||
- [The 2.0.0 story](#the-200-story)
|
||||
- [Why a number is only spent by a release](#why-a-number-is-only-spent-by-a-release)
|
||||
- [Where the procedure lives](#where-the-procedure-lives)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
## Two questions, not one
|
||||
|
||||
The first question is whether the new version is a drop-in replacement for the old one - whether
|
||||
an existing instance can install it, and can also go back, without anyone doing hand-work. That
|
||||
is what a version number *is*: a promise. The second question is whether the existing corpus in
|
||||
`kb/` needs to change shape to keep working under the new version. These sound like the same
|
||||
question, because most of the time a change that breaks compatibility also happens to touch
|
||||
content, and most of the time a change that leaves content untouched also happens to be
|
||||
compatible. The correlation is real; it just is not a law. `instructions/dev/version-parts.md`
|
||||
carries the actual test for telling them apart and the steps that follow from it - this page is
|
||||
about why the split exists at all.
|
||||
|
||||
## Why "kb/ untouched" is not proof of anything
|
||||
|
||||
The tempting shortcut is: if no page in `kb/` had to change, the bump can't be that serious. This
|
||||
is exactly backwards for a class of changes that live entirely outside the corpus - a renamed
|
||||
release artefact, a Python import path, an environment variable, the URL an instance's own
|
||||
updater points at. None of those touch a single page. All of them can strand an existing
|
||||
instance just as thoroughly as a rewritten type-spec would. The corpus is the part of the stack
|
||||
that looks at itself; the compatibility question is about everything an instance depends on to
|
||||
keep functioning, most of which the corpus never sees.
|
||||
|
||||
## Reading compatibility off the leftmost non-zero component
|
||||
|
||||
Semantic versioning gives every component a job, but only one of them is where an existing
|
||||
instance's tooling actually looks to decide "is this safe." On a `2.x` stack that is MAJOR; on a
|
||||
still-pre-1.0 `0.x` stack, by the same convention, it's MINOR - the leftmost slot that isn't
|
||||
pinned to zero is the one an automated updater treats as the compatibility boundary. Bump
|
||||
anything to its left, or bump that slot itself, and the promise changes. Everything to the right
|
||||
of it can move as freely as the project likes without touching that promise. This is why the
|
||||
question "is it boundary-crossing" always resolves to one specific digit, not to a feeling about
|
||||
how big the change is.
|
||||
|
||||
## Downgrade is half the promise
|
||||
|
||||
It's natural to test compatibility by only asking "does the upgrade work." The other half -
|
||||
"can an instance that upgraded put the old version back and land where it started" - carries
|
||||
equal weight, and it's the half that's easy to forget because forward motion is what everyone is
|
||||
testing for anyway. A state file the old version can no longer parse, a generated index in a new
|
||||
shape, a stamp file that got renamed: none of these have to break the upgrade to break the
|
||||
downgrade. An instance that can go forward but not back has already lost the property a
|
||||
compatible version number is supposed to guarantee.
|
||||
|
||||
## A promise made to a machine, not only to a person
|
||||
|
||||
A human reading a changelog can absorb "this technically isn't compatible but it's fine, just
|
||||
update those two things by hand." An instance's own update mechanism cannot. It reads a version
|
||||
number, decides whether to pull the new release, and has no channel for nuance - which is exactly
|
||||
why the update path itself is one of the sharpest ways to cross the boundary invisibly: if the
|
||||
new version moves where updates come from, the very channel that would have told an instance to
|
||||
adjust is the channel that just broke. The version number isn't documentation aimed at a reader;
|
||||
it's an input consumed by code that has no other way to ask.
|
||||
|
||||
## The 2.0.0 story
|
||||
|
||||
This isn't hypothetical for this stack. The rebranding that produced Chemenu renamed the repo,
|
||||
the release artefact, and the Python package - and left every page in `kb/` untouched. The first
|
||||
instinct was a MINOR bump, on the reasoning that nothing in the corpus needed migrating. That
|
||||
reasoning was correct on its own terms and answered the wrong question. Three things broke
|
||||
underneath it: every existing instance's `update_url` pointed at a repo path that no longer
|
||||
existed and, because it's a machine-written file, couldn't be hand-repaired; the release artefact
|
||||
name changed, breaking every download script and pin against it; and the import name changed,
|
||||
breaking anything importing the package from outside the shipped tree. The corpus had nothing to
|
||||
say about any of this, because none of it lived in the corpus.
|
||||
|
||||
What caught the mistake was a person looking at the diff and asking whether it really was a
|
||||
drop-in replacement, not a validator. No check in `docs verify` or anywhere else confirms that a
|
||||
version part was chosen correctly - it only confirms that a boundary-crossing bump documents
|
||||
what it breaks. The 2.0.0 entry in `CHANGES.md` carries the corrected reasoning in full, and the
|
||||
version bump that shipped it was `--major --no-migration`: boundary-crossing and untouched
|
||||
corpus, at the same time, which is precisely the combination the two-question split exists to
|
||||
make visible.
|
||||
|
||||
## Why a number is only spent by a release
|
||||
|
||||
Everything above is about what a version number *promises*. A separate question turned out to
|
||||
matter just as much in practice: how many numbers get handed out along the way to making one
|
||||
release. For a while the answer was "one per bump," and that turned out to be the wrong grain
|
||||
entirely.
|
||||
|
||||
Two mechanisms decide when a number gets minted, and they answer different questions. CI's
|
||||
version gate asks a *commit*-level one: has this tree changed since the last push, and if so
|
||||
has `VERSION` moved with it. A release asks something else entirely: is this a state worth
|
||||
handing to someone, under a number they will pin against. Tying the second to the first - every
|
||||
`VERSION` move firing the release workflow - answers the gate correctly and the release question
|
||||
by accident, because it treats every bump as if it were about to ship when most bumps are steps
|
||||
toward a release that has not happened yet.
|
||||
|
||||
The failure mode is not phantom numbers; every one of those releases was real, tagged and
|
||||
downloadable. It is that "real" stopped meaning anything. On 2026-09-03 this repository cut four
|
||||
releases in six hours - `4.3.0` through `4.3.3` - for one continuous arc of work, two of them for
|
||||
prose changes alone. Someone tracking the feed saw four upgrades and had no way to tell which, if
|
||||
any, was a moment worth stopping for. A release is a promise addressed to a consumer, and a
|
||||
promise made four times an afternoon is not a smaller promise, it is a less legible one.
|
||||
|
||||
The fix is not to slow the gate down - it still wants `VERSION` to move every time, and it still
|
||||
gets that. It is to stop treating every movement as a number worth publishing. Between two
|
||||
releases the stack now carries one running candidate, escalating through `-beta.N` as bumps
|
||||
accumulate, and only `version release` spends the number for real by fixing it and closing its
|
||||
changelog entry. A number is proposed by a bump and spent by a release; conflating the two was
|
||||
the actual defect, not the arithmetic of any single bump.
|
||||
|
||||
This is also why a candidate never gets to a distributed instance. The promise a released version
|
||||
makes - "install this, and it is exactly what its number says" - has no equivalent for something
|
||||
still being decided during a single dev checkout's session. `release.yml`'s only job with respect
|
||||
to this is refusing to act on a suffixed `VERSION` at all: not because a beta is unsafe, but
|
||||
because there is nothing yet to promise.
|
||||
|
||||
## Where the procedure lives
|
||||
|
||||
The drop-in test, the catalogue of changes that cross the boundary with no page touched, and the
|
||||
steps for a boundary-crossing bump - the `--breaking` lines, the migration document or
|
||||
`--no-migration` reason, talking to the user before bumping - are one procedure, kept at one
|
||||
place: `instructions/dev/version-parts.md`.
|
||||
|
||||
Named as a plain path rather than linked, because it is not here to link to. `dist export`
|
||||
prunes `instructions/dev/` wholesale, so that file exists only in the origin repo - the place
|
||||
where a version is bumped at all. An instance reads this page to understand what a version
|
||||
number promises it; it never runs the procedure.
|
||||
@@ -0,0 +1,157 @@
|
||||
# Why gates are code
|
||||
|
||||
Chemenu's hard limits - the gates [AGENTS.md § Gates](../../AGENTS.md#gates) lists - live inside
|
||||
`tools/wikitool`, not in a paragraph of instructions an agent reads and follows. The rules
|
||||
themselves, and what to do when one trips, are in [AGENTS.md § Gates](../../AGENTS.md#gates) and
|
||||
[instructions/gates.md](../../instructions/gates.md). This page is only about the design choice
|
||||
underneath them: why code, and why these mechanisms in particular.
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
|
||||
- [A suggestion an agent can talk itself past](#a-suggestion-an-agent-can-talk-itself-past)
|
||||
- [Why different mechanisms, not one](#why-different-mechanisms-not-one)
|
||||
- [Exit 42 is a posture, and it outgrew the gates](#exit-42-is-a-posture-and-it-outgrew-the-gates)
|
||||
- [A gate in code still has to be reachable](#a-gate-in-code-still-has-to-be-reachable)
|
||||
- [Numbers that come from measurement, not intuition](#numbers-that-come-from-measurement-not-intuition)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
## A suggestion an agent can talk itself past
|
||||
|
||||
An instruction like "don't publish too much at once" or "don't loop forever" lives in the same
|
||||
place as every other piece of guidance a session is holding - alongside the task, the user's
|
||||
last message, and whatever context made the moment feel urgent. Under pressure, or with a
|
||||
plausible-sounding reason ("this batch is different, it's mechanical"), that guidance can be
|
||||
reasoned around without anyone deciding to break a rule. Nothing enforces it; it just competes
|
||||
for attention with everything else in the context window, and sometimes loses.
|
||||
|
||||
A check compiled into the tool doesn't have that problem, because it isn't part of the
|
||||
conversation at all. It runs before the command dispatches, regardless of how convincing the
|
||||
case for skipping it seemed a moment earlier. The difference isn't that code is smarter than a
|
||||
well-written instruction - it's that code doesn't get talked into anything.
|
||||
|
||||
## Why different mechanisms, not one
|
||||
|
||||
The gates ask different kinds of question, and each one's shape follows from what kind of
|
||||
question it is.
|
||||
|
||||
The Mass-Update Gate asks *is this change too large to publish unreviewed* - a judgment that
|
||||
varies changeset by changeset, so it clears with a `--confirm` token tied to the specific
|
||||
output the user just read. Approval is scoped to that one publish.
|
||||
|
||||
The Publish-Remote Gate asks something underneath that: *is this even the right repository*.
|
||||
That's not a per-push judgment, it's a standing property of the checkout - true or false for
|
||||
every publish that checkout will ever attempt, not just this one. A confirm token would let an
|
||||
agent clear it once and then treat the answer as settled, which is exactly backwards for a
|
||||
question whose answer shouldn't move at all mid-session. The only way past it is the user
|
||||
editing `.wikitool-remotes.json` directly, outside the gate's own flow.
|
||||
|
||||
The Upload Review Gate asks the Mass-Update Gate's own question - *is this change right?* - at
|
||||
the opposite end of its size range: one file from a stranger instead of a changeset from the
|
||||
session's own work. That similarity is exactly why it reuses the same shape (a `--confirm` token
|
||||
digesting the thing being approved) rather than inventing a new one: the two differ in
|
||||
*who* produced the change and *how much* of it there is, not in what kind of question either one
|
||||
is answering, so nothing about the mechanism needed to change - only the boundary it sits behind
|
||||
did, since the submission lives in a quarantine the ordinary pipeline never reads at all rather
|
||||
than in the working tree `publish` is about to commit.
|
||||
|
||||
The Guideline Push Gate asks the same question once more, at the boundary facing outwards: *is
|
||||
this content right for these repositories, now?* `export guidelines --push` writes a generated
|
||||
file straight onto another repository's branch, with no review on the receiving side, and a
|
||||
public target publishes it the moment it lands. That is a per-run judgment - the guidelines
|
||||
change, the set of opted-in repositories changes, a branch moves - so it takes the token shape,
|
||||
digesting exactly what would be written where. It deliberately does not take the Publish-Remote
|
||||
Gate's shape, although it too pushes to a remote: which repositories are targets is already a
|
||||
standing, committed declaration - the capture manifests under `raw/` - and asking for the same
|
||||
URLs in an allowlist as well would be a second declaration of one fact. What is left to ask is
|
||||
the per-run question, and that is the one a token answers.
|
||||
|
||||
The Rebase-Review Gate asks the one question the others cannot: *does what is arriving collide
|
||||
logically with what I am about to publish?* `sync` and `publish` reconcile with the remote by
|
||||
rebasing, and when the incoming commits touch a file this session is also changing, git can
|
||||
produce a clean textual result out of two changes that contradict each other. Whether they do is
|
||||
a question of meaning, not of text, so the tool cannot answer it - it can only refuse, and show
|
||||
the incoming commits, the overlapping files and their diff to whoever can. The answer is different
|
||||
for every rebase, which is why it takes the token shape: the token digests the exact upstream
|
||||
state and the overlapping files, so a remote that moved again before the re-run is a new
|
||||
question, asked again.
|
||||
|
||||
The Iteration Budget Gate asks a different kind of question - not "is this instance correct" but
|
||||
"has this session stopped making progress." That's read from the shape of the call history
|
||||
itself (call count, repeated identical calls), not from anything about the content of any one
|
||||
call.
|
||||
|
||||
## Exit 42 is a posture, and it outgrew the gates
|
||||
|
||||
Those are the named gates, and they are not the only thing that exits 42 any more. When the
|
||||
task-tracker provider layer arrived, it brought a case that looks like a gate from the outside and
|
||||
is not one: a provider whose API cannot create a project (Super Productivity's local REST API
|
||||
reads projects but does not write them) raises `HumanInterventionRequired`, and the command prints
|
||||
what a human has to do and exits 42.
|
||||
|
||||
Reusing the code was deliberate, and so was not calling it another gate. What the named gates share
|
||||
is a *refusal*: the operation was possible and the tool declined to perform it unreviewed. This is
|
||||
the opposite situation - the operation is not possible at all, and no token could make it
|
||||
possible. What the two have in common is only what the exit code actually communicates: **stop,
|
||||
show this to a human, do not improvise a way around it.** That sentence is the whole meaning of
|
||||
42 here, and it is worth more as a shared convention than as a number reserved for one mechanism.
|
||||
|
||||
The alternative was worse in a specific way. A provider that cannot do something could have been
|
||||
described in the instruction layer instead - "if you are on this tracker, create the project by
|
||||
hand first" - which is exactly the prose-shaped rule this page argues against, with the added cost
|
||||
that every instruction would then have to know which provider an instance runs. The capability
|
||||
gap belongs where the capability is, and reaches the session as an exit code rather than as a
|
||||
paragraph it has to remember to apply.
|
||||
|
||||
The preflight is the second such case, and the one closest to this page's own argument. A machine
|
||||
without Python or ripgrep is not something the tool can fix, and an install that met that gap
|
||||
with only a setup instruction to go on showed what follows: the agent worked around each missing
|
||||
piece - another environment, a hand-made configuration - and kept going. So `tools/preflight.sh` (and its PowerShell twin, `tools/preflight.ps1`)
|
||||
exits 42 with the command a human has to run, and the launcher in front of every `wikitool` call
|
||||
exits 42 until the preflight has passed. "Check first, stop, let the user act" lives in two places
|
||||
a session cannot read past, not in a step it can skip.
|
||||
|
||||
## A gate in code still has to be reachable
|
||||
|
||||
Code beats prose for the reason above, but on its own it buys less than it looks like: a check
|
||||
that runs on every call is only as good as the thing it counts under. The Iteration Budget Gate
|
||||
scopes its counter to a session, and "session" was approximated by the parent process id whenever
|
||||
nothing set an explicit one. On a harness that runs every tool call in a freshly initialised
|
||||
shell, that approximation hands out a new session per call - so a traced run of thirty-three calls
|
||||
arrived as twenty-one sessions of one to three calls each, the ceiling of sixty was never
|
||||
approached, and the loop-breaker's window never held three calls at once to compare. The gate ran
|
||||
on every one of those calls, exactly as written, and refused nothing.
|
||||
|
||||
That failure has no symptom of its own. A gate that fires announces that it exists; a gate that
|
||||
*cannot* fire looks identical to a gate nobody happened to need - the same clean runs, the same
|
||||
silence - and what finally told the two apart was reading a trace for an unrelated reason. So
|
||||
there is a third property to keep alongside living in code and carrying measured numbers: each
|
||||
gate has to leave evidence that it can still fire. The ones that clear by token or by a
|
||||
deliberate edit have it by construction, because clearing one is a visible event in somebody's
|
||||
terminal. The budget gate, whose ordinary outcome is silence, is the one that had to be given
|
||||
it - which is why its session id now carries where it came from, into both the trace and
|
||||
`budget status`, so a session's own record answers the question instead of an investigation
|
||||
having to.
|
||||
|
||||
## Numbers that come from measurement, not intuition
|
||||
|
||||
The iteration ceiling didn't start where it sits now. It used to run 15-25, borrowed from a
|
||||
general rule of thumb, until four real ingest runs measured 24, 26, 29 and 30 calls apiece -
|
||||
every one of them an ordinary workflow doing nothing wrong, and every one of them at or past
|
||||
where the old ceiling would have refused it. A limit that the normal case keeps tripping stops
|
||||
functioning as a limit; it becomes background noise a session learns to route `--override-budget`
|
||||
around as a matter of course, and the whole point of a hard-coded check is that it isn't supposed
|
||||
to feel routine.
|
||||
|
||||
That's the deeper reason these numbers live in a tool rather than in prose: prose is read once
|
||||
and remembered loosely, but a threshold enforced every call is tested by every call, and a
|
||||
threshold that fails its own test gets noticed and re-measured rather than quietly ignored.
|
||||
|
||||
The suite's coverage floor is the same argument run forwards instead of backwards. The ceiling
|
||||
above was wrong first and measured afterwards; the floor was withheld on purpose until the number
|
||||
existed - measured, then watched across 38 runs while the code grew by a quarter, and only then
|
||||
written down as 85 against an observed 87.0%. The two points of daylight are the same
|
||||
consideration as the ceiling's headroom: a limit the ordinary case keeps tripping stops being a
|
||||
limit. A coverage floor set at the measured number goes red on the next thin command wrapper,
|
||||
which is not a regression, and a threshold that goes red for a non-reason gets lowered rather
|
||||
than earned - the failure mode above, reached from the other direction.
|
||||
@@ -0,0 +1,29 @@
|
||||
# How-to guides
|
||||
|
||||
Task-oriented pages for the operator of an instance: one goal per page - upgrade an instance,
|
||||
set up the MCP read server, run a weekly review - with the steps a human takes and the sentence
|
||||
handed to the agent. A how-to guide never retells an agent's procedure: the steps an agent runs
|
||||
live in `instructions/`, and a guide here says what to prepare, what to ask for and where the
|
||||
agent stops.
|
||||
|
||||
Working with the wiki:
|
||||
|
||||
- [Ingest a source](ingest-a-source.md) - a file, a folder, a web page, or a document submitted
|
||||
through the MCP server
|
||||
- [Capture documentation from a git repository](capture-repository-docs.md) - and keep it current
|
||||
- [Export guidelines to captured repositories](export-guidelines.md)
|
||||
- [Query the wiki](query-the-wiki.md) - asking, and searching yourself
|
||||
- [Maintain the wiki](maintain-the-wiki.md) - lint, provenance, and the gates on the way to a publish
|
||||
- [Run the weekly review](run-the-weekly-review.md) - commitments in a task tracker against the
|
||||
project pages
|
||||
- [Browse the wiki in Obsidian](browse-with-obsidian.md) - and how pages are named
|
||||
|
||||
Running the instance:
|
||||
|
||||
- [Update an instance](update-an-instance.md) - to a newer release of the stack
|
||||
- [Troubleshoot an instance](troubleshoot.md)
|
||||
- [Set up the MCP read server](set-up-the-mcp-server.md) - for a consumer that is not a terminal
|
||||
on this machine
|
||||
- [Trace and score agent sessions](trace-and-score-sessions.md)
|
||||
|
||||
Installing a new instance is [INSTALL.md](../../INSTALL.md) at the repository root.
|
||||
@@ -0,0 +1,54 @@
|
||||
# Browse the wiki in Obsidian
|
||||
|
||||
The wiki is a folder of Markdown files joined by `[[wikilinks]]`, so any Markdown editor can read
|
||||
it. Obsidian adds a graph of the links between pages, which makes it a good way to look around.
|
||||
|
||||
## What you prepare
|
||||
|
||||
Open the instance's root directory as an Obsidian vault. Obsidian keeps its own settings in
|
||||
`.obsidian/`. That folder is personal state: it is never shipped with a release, and it has no
|
||||
effect on the wiki.
|
||||
|
||||
What you get:
|
||||
|
||||
- the graph view of how pages connect;
|
||||
- navigation by wikilink, the way the agent writes them;
|
||||
- plugins such as Dataview, which can query page frontmatter, or Marp for slides.
|
||||
|
||||
## Reading, not writing
|
||||
|
||||
Browse and read freely. Change pages through the agent, though, not by editing them in Obsidian. A
|
||||
page carries generated frontmatter, cross-references and footnote citations that `tools/wikitool`
|
||||
keeps consistent across the whole wiki. A hand edit can quietly break them. Renaming or moving a
|
||||
file in Obsidian is the clearest case: a title is the page's only identifier, so it also appears in
|
||||
every wikilink and citation that points at the page. A rename goes through
|
||||
`tools/wikitool rename`, which updates every reference. The full procedure is
|
||||
[instructions/page-lifecycle.md](../../instructions/page-lifecycle.md).
|
||||
|
||||
Generated files - `kb/index.md`, `kb/log.md`, `kb/provenance.md` and every `INDEX.md` - are
|
||||
rebuilt by the tool and never edited by hand.
|
||||
|
||||
## How pages are named
|
||||
|
||||
Knowing the naming rules makes the vault easier to find your way in:
|
||||
|
||||
- **Titles are human-readable names with spaces:** `Hybrid Search.md`, not kebab-case. A wikilink
|
||||
matches the file name exactly: `[[Hybrid Search]]`.
|
||||
- **Entities are singular:** `HA Integration.md`, not `HA Integrations.md`.
|
||||
- **A title follows the subject's own established name, not the wiki's language.** `Act Runner`
|
||||
and `GitOps Ownership Model` keep their names in a German wiki too. Translating a title is a
|
||||
rename, not an edit.
|
||||
- **A title works on every operating system.** It contains none of `< > : " / \ | ? *`, is not a
|
||||
reserved name such as `CON` or `Index`, does not end in a dot, and does not differ from another
|
||||
page's title by case alone. A file's whole path below the instance root stays at 160 characters
|
||||
or fewer, so a Windows checkout works without long paths. `wikitool new` and `wikitool rename`
|
||||
refuse titles that break these rules, and `wikitool lint` reports existing ones.
|
||||
[kb/CONTRACT.md](../../kb/CONTRACT.md) has the full rule.
|
||||
|
||||
## Letting the structure grow
|
||||
|
||||
- Start with a few broad categories and refine as the material arrives. Don't over-organise early.
|
||||
- Use tags for concerns that cut across collections.
|
||||
- Review what the agent changed from time to time - `kb/log.md` lists every operation - and tell it
|
||||
when something is off. It keeps to the standards in `AGENTS.md` and the contracts, but you are the
|
||||
one who notices when a page misses the point.
|
||||
@@ -0,0 +1,59 @@
|
||||
# Capture documentation from a git repository
|
||||
|
||||
Documentation that lives in a git repository is captured, not copied by hand. A capture records the
|
||||
repository, the ref rule and the commit it came from. Later, the wiki can tell which captured
|
||||
material has fallen behind its repository, and take in the new edition as a whole.
|
||||
|
||||
## What you prepare
|
||||
|
||||
Access to the repository from this machine. Git uses your own keys and credential helpers, and
|
||||
nothing is stored in the instance. Then decide three things:
|
||||
|
||||
- the repository URL;
|
||||
- the ref to follow, usually a branch such as `main`;
|
||||
- which files count, as a glob, for example `docs/**/*.md`.
|
||||
|
||||
## Taking a repository in for the first time
|
||||
|
||||
You can run the capture yourself, or name the three values to the agent and let it run the
|
||||
command:
|
||||
|
||||
```bash
|
||||
tools/wikitool raw capture <repo-url> --ref main --path 'docs/**/*.md' --name <bundle>
|
||||
```
|
||||
|
||||
This writes the matching files, byte for byte, into `incoming/<bundle>/`, together with a
|
||||
`_capture.json` manifest naming the repository, the ref rule and the commit. From there it is an
|
||||
ordinary source: `Ingest incoming/<bundle>` ([Ingest a source](ingest-a-source.md)), with the same
|
||||
questions and the same gate.
|
||||
|
||||
## Keeping it current
|
||||
|
||||
```bash
|
||||
tools/wikitool raw status
|
||||
```
|
||||
|
||||
This lists every captured bundle that has fallen behind its repository, file by file, marking each
|
||||
file as added, modified or deleted. It only reads.
|
||||
|
||||
To bring them in, say:
|
||||
|
||||
```text
|
||||
Update the captured repositories
|
||||
```
|
||||
|
||||
The agent checks every captured bundle. It takes one changed bundle per run into the wiki, replacing
|
||||
the old edition as a whole, and tells you how many are still behind. Run it again for the next one.
|
||||
`raw capture --update` and `raw accept --replaces-bundle` are the commands underneath. What each one
|
||||
does is in the [command reference](../reference/commands/raw-material-and-uploads.md).
|
||||
|
||||
## Where the agent stops
|
||||
|
||||
The same places as any ingest: the `fidelity`/`authority` questions, the discussion of what
|
||||
matters, and the Mass-Update Gate when it publishes. When one repository cannot be reached, the
|
||||
agent reports the error and carries on with the others. A failure in one never stops the run.
|
||||
|
||||
## Related
|
||||
|
||||
- [Export guidelines](export-guidelines.md) - sending the wiki's guidelines back into the same
|
||||
repositories.
|
||||
@@ -0,0 +1,54 @@
|
||||
# Export guidelines to captured repositories
|
||||
|
||||
The repositories the wiki captures documentation from can get something back: the wiki's
|
||||
guidelines, as one generated `GUIDELINES.md` in each repository's root, which that repository's own
|
||||
agents then read.
|
||||
|
||||
## What you prepare
|
||||
|
||||
- **Decide which pages count as guidelines.** That decision is yours, and it is written down in
|
||||
`kb/CONVENTIONS.md` § Guidelines for other repositories. Usually it is a tag. To see the file
|
||||
exactly as it would be written, without touching any repository:
|
||||
|
||||
```bash
|
||||
tools/wikitool export guidelines --tag guideline
|
||||
```
|
||||
|
||||
- **Opt each repository in, by hand, once.** A repository takes part only if it carries a
|
||||
`GUIDELINES.md` whose first line is exactly:
|
||||
|
||||
```markdown
|
||||
<!-- wikitool:export kind=guidelines -->
|
||||
```
|
||||
|
||||
Commit that one line there. Then point the repository's own `AGENTS.md` at the file (for Claude
|
||||
Code: `@GUIDELINES.md`). A `GUIDELINES.md` without that first line is hand-written, and the export
|
||||
never touches it.
|
||||
|
||||
The repository has to be one the wiki has captured
|
||||
([Capture documentation from a git repository](capture-repository-docs.md)). That is how the export
|
||||
knows where the repository lives. Its capture also has to follow a branch: a repository captured by
|
||||
a tag pattern is skipped, because a tag has no branch to commit onto.
|
||||
|
||||
## What you say
|
||||
|
||||
```text
|
||||
Roll out the guidelines to the captured repositories
|
||||
```
|
||||
|
||||
## Where the agent stops
|
||||
|
||||
The agent runs `export guidelines --push`, which stops before pushing anything. This is the
|
||||
**Guideline Push Gate** (exit 42). For every repository, it prints the status, the diff it would
|
||||
write and a `--confirm <token>` line. The agent puts every diff in front of you. Only your approval
|
||||
pushes, and then only one commit per repository, which changes only `GUIDELINES.md` and goes
|
||||
straight onto its branch.
|
||||
|
||||
The token is a digest of each target's branch tip and of the generated file. If a branch has moved
|
||||
or a guideline was edited since you looked, the gate asks again rather than pushing something you
|
||||
did not see.
|
||||
|
||||
## Related
|
||||
|
||||
- [Guideline export commands](../reference/commands/guideline-export.md)
|
||||
- [Why the gates are code](../explanation/why-gates-are-code.md)
|
||||
@@ -0,0 +1,91 @@
|
||||
# Ingest a source
|
||||
|
||||
Turn a file, a folder or a web page into wiki pages: a source page that summarises it, and the
|
||||
entity and concept pages it touches, cross-referenced and published. The agent does the work, using
|
||||
the `wiki-ingest` skill ([skill catalogue](../reference/skills.md)). This page covers what you hand
|
||||
it and where it waits for you.
|
||||
|
||||
## What you prepare
|
||||
|
||||
Put the material into `incoming/` at the root of the instance. You don't have to classify it or
|
||||
pick a directory:
|
||||
|
||||
- **One file** goes in directly: `incoming/my-article.md`.
|
||||
- **Several files that belong together** go into one folder, `incoming/<name>/`. The folder is one
|
||||
source. It is accepted whole, and its structure is kept.
|
||||
- **A web page** needs no download of your own. Name the URL to the agent, and
|
||||
`tools/wikitool raw fetch` puts two files into `incoming/`: the HTML exactly as received, and a
|
||||
text derived from it with a header recording where and when it was fetched.
|
||||
- **A document that is not text** - a PDF, a Word, PowerPoint or Excel file, a mail (`.eml` or
|
||||
Outlook `.msg`) or a ZIP archive - is
|
||||
put into `incoming/` as it is. `tools/wikitool raw derive incoming/<file>` turns it into a folder with the
|
||||
untouched original and a readable text beside it, with a header saying how it was made. The
|
||||
folder is accepted whole; the agent runs this itself when it meets such a file. An Excel file's
|
||||
sheets land beside the text as tab-separated files, with their formulas listed apart. A ZIP is
|
||||
unpacked into a folder instead, and a mail's attachments land beside it. A scanned PDF has no
|
||||
text to derive - the tool warns, and there is no OCR.
|
||||
- **A page behind a paywall or a login** is saved from your browser into `incoming/`, HTML only. The
|
||||
agent derives the same text from that file with `raw fetch --html`.
|
||||
|
||||
Where a file ends up in `raw/` is computed by `tools/wikitool raw accept`: a `YYYY/MM` shard for the
|
||||
day it was accepted. So is whether several files of one source get bundled. You never choose either.
|
||||
`incoming/` itself is gitignored, so whatever waits there is never published by accident.
|
||||
|
||||
## What you say
|
||||
|
||||
```text
|
||||
Ingest incoming/my-article.md
|
||||
Ingest https://example.org/post
|
||||
Ingest
|
||||
```
|
||||
|
||||
A bare `Ingest` takes the oldest entry waiting in `incoming/`. `tools/wikitool raw pending` lists
|
||||
what is waiting and names the entry the next bare `Ingest` will take.
|
||||
|
||||
## Where the agent asks you
|
||||
|
||||
- **Before promoting the file into `raw/`**, the agent asks how faithful the capture is
|
||||
(`fidelity`) and what the material may claim about its subject (`authority`). Both are recorded
|
||||
once and never guessed: you know the answers now, and nobody can recover them later.
|
||||
[raw/CONTRACT.md](../../raw/CONTRACT.md) defines the values.
|
||||
- **Before writing pages**, the agent presents the key takeaways and asks which points matter most
|
||||
and what to emphasise. The pages follow your answer.
|
||||
- **When it publishes**, a source that touches ten or more files stops at the Mass-Update Gate
|
||||
(exit 42). The generated indexes, `kb/log.md` and `kb/provenance.md` do not count, so a small
|
||||
source goes straight through. The agent shows you the file list and the `--confirm` line. Nothing is committed until
|
||||
you approve. [Exit codes and gates](../reference/exit-codes.md) lists every gate.
|
||||
|
||||
## What you get back
|
||||
|
||||
A source page under `kb/sources/`, new or updated entity and concept pages, cross-references among
|
||||
them, a rebuilt catalog, an entry in `kb/log.md`, and one commit. If the source also contains
|
||||
something you have to *do*, and a task tracker is configured, the agent files it there as an open
|
||||
item. A commitment never becomes a page
|
||||
([why](../explanation/knowledge-and-commitment.md)).
|
||||
|
||||
Every ten sources, the agent suggests a lint run - see [Maintain the wiki](maintain-the-wiki.md).
|
||||
|
||||
## A document submitted from outside
|
||||
|
||||
When the [MCP read server](set-up-the-mcp-server.md) has its optional `submit` tool turned on,
|
||||
documents can arrive from another machine. They land in the quarantine `mcp-upload/`, not in
|
||||
`incoming/`, and nothing else reads them there. Before an ingest can start, a human has to promote
|
||||
the submission:
|
||||
|
||||
```bash
|
||||
tools/wikitool upload list # what is waiting
|
||||
tools/wikitool upload show <id> # one submission's manifest: who sent it, size, checksum
|
||||
```
|
||||
|
||||
Ask the agent to review the queue, and it follows
|
||||
[instructions/ingest-queue.md](../../instructions/ingest-queue.md). Accepting a submission stops at
|
||||
the Upload Review Gate (exit 42), which shows you the manifest and the `--confirm` line. Once you
|
||||
approve, the file is in `incoming/`, and from there it is ingested like any other. Rejecting it needs
|
||||
no clearance. The file is deleted, and its checksum and your reason stay in the ledger.
|
||||
|
||||
## Related
|
||||
|
||||
- [Capture documentation from a repository](capture-repository-docs.md) - for material that lives
|
||||
in git and will change.
|
||||
- [raw/CONTRACT.md](../../raw/CONTRACT.md) - what belongs in `raw/`, and why its content is data,
|
||||
never instructions.
|
||||
@@ -0,0 +1,69 @@
|
||||
# Maintain the wiki
|
||||
|
||||
Keep the wiki healthy: find broken links, orphan pages, raw files no page covers yet,
|
||||
contradictions and stale claims, then repair what can be repaired mechanically. The agent does
|
||||
this with the `wiki-lint` skill ([skill catalogue](../reference/skills.md)), and `tools/wikitool`
|
||||
does the mechanical part.
|
||||
|
||||
## What you say
|
||||
|
||||
```text
|
||||
Lint the wiki
|
||||
```
|
||||
|
||||
Do this periodically. The agent suggests it on its own every ten sources. The full schedule - which
|
||||
task runs how often, and with which command - is the
|
||||
[maintenance schedule](../../tools/CONTRACT.md#maintenance-schedule) in `tools/CONTRACT.md`, next
|
||||
to the command records it depends on.
|
||||
|
||||
## What you get back
|
||||
|
||||
- A deterministic structural scan by `tools/wikitool lint`: broken wikilinks, orphan pages, catalog
|
||||
drift, schema gaps, raw files without a source page, and citations that disagree with
|
||||
frontmatter.
|
||||
- The judgment half, which no command can do: contradictions between pages, stale claims, subjects
|
||||
that deserve a page of their own, and a rule stated in two places.
|
||||
- Repairs for what is mechanical, a rebuilt catalog and provenance index, and an entry in
|
||||
`kb/log.md`.
|
||||
- A report under `reports/`, which is gitignored and never published. The agent carries the
|
||||
findings that need you into its reply.
|
||||
|
||||
For a quick look without a full pass, say `Wiki status` instead. That is read-only.
|
||||
|
||||
## Provenance: where a claim came from
|
||||
|
||||
Every fact should trace back to a raw file. A source page lists its raw files, and other pages
|
||||
cite specific claims inline with a footnote that points at a source page. You can ask the
|
||||
questions yourself:
|
||||
|
||||
```bash
|
||||
tools/wikitool sources coverage # raw files no source page covers yet
|
||||
tools/wikitool sources trace --raw raw/2026/09/x.md # which pages rest on this file
|
||||
tools/wikitool sources trace --page "HA Integration" # which raw files this page rests on
|
||||
```
|
||||
|
||||
`kb/provenance.md` is the generated reverse index. The commands are in the
|
||||
[provenance reference](../reference/commands/provenance.md).
|
||||
|
||||
## Where the agent stops
|
||||
|
||||
- **Publishing ten or more files** stops at the Mass-Update Gate (exit 42); the generated indexes,
|
||||
`kb/log.md` and `kb/provenance.md` are not counted. A lint pass that repairs
|
||||
many pages does this routinely. The agent shows you the full file list and the `--confirm` line,
|
||||
and publishes only after you approve. The token is a digest of that file list, so your approval
|
||||
never covers a changeset you did not see.
|
||||
- **The call budget.** Every `wikitool` call counts against a per-session budget: 60 calls, or 3
|
||||
identical calls in a row. Past it, calls are refused. The agent stops and summarises where it is.
|
||||
If you want it to go on, tell it so. Only then may it pass `--override-budget` on the command it
|
||||
needs. A refusal usually means the task should be split, not pushed through.
|
||||
- **No reachable remote.** Without `--no-push`, `publish` refuses with exit 1 before committing when
|
||||
the remote is not configured or cannot be reached. An instance that is meant to stay local passes
|
||||
`--no-push` on every publish.
|
||||
|
||||
Every gate, and which command it belongs to: [exit codes and gates](../reference/exit-codes.md).
|
||||
Why they are enforced in code: [why-gates-are-code.md](../explanation/why-gates-are-code.md).
|
||||
|
||||
## Related
|
||||
|
||||
- [Troubleshoot](troubleshoot.md) - when `doctor` or a command reports a problem with the instance
|
||||
itself rather than with its pages.
|
||||
@@ -0,0 +1,50 @@
|
||||
# Query the wiki
|
||||
|
||||
Ask the wiki a question and get an answer built from its compiled pages, with the pages and sources
|
||||
it came from. The agent answers using the `wiki-query` skill
|
||||
([skill catalogue](../reference/skills.md)), which only reads unless you ask it to keep an answer.
|
||||
|
||||
## What you say
|
||||
|
||||
Ask the way you would ask a colleague:
|
||||
|
||||
```text
|
||||
What projects use MQTT?
|
||||
Show me the architecture of HA Integration
|
||||
Compare gdeploy and plugnburn-edl
|
||||
What decisions were made about E3DC integration?
|
||||
```
|
||||
|
||||
## What you get back
|
||||
|
||||
An answer that names the wiki pages it came from and the sources behind them. Where the wiki has no
|
||||
source for a claim, the agent says so instead of filling the gap. An answer it cannot back is never
|
||||
written into the wiki either.
|
||||
|
||||
If an answer is worth keeping - a comparison you will want again, a synthesis across several pages -
|
||||
the agent proposes filing it back as a page. It names the pages it would create before it creates
|
||||
any, and nothing is written unless you agree.
|
||||
|
||||
## Searching yourself
|
||||
|
||||
The agent searches with the same command you can run:
|
||||
|
||||
```bash
|
||||
tools/wikitool search "<text>"
|
||||
tools/wikitool search --field entity_type=system
|
||||
tools/wikitool search --field '!sources' # pages without a sources: list
|
||||
```
|
||||
|
||||
`search` covers all of `kb/`, by text, by frontmatter or both. Each hit carries the page's path and
|
||||
full title, so you can open it directly. It only reads, and it does not count against the
|
||||
session's call budget, because looking before acting is the habit worth encouraging. The predicate forms are in the
|
||||
[command reference](../reference/commands/finding-and-checking.md).
|
||||
|
||||
For browsing rather than searching, `kb/index.md` is the catalog map, and each collection carries
|
||||
its own `INDEX.md`. All of these are generated. Read them, but never edit them. To browse the pages as
|
||||
a graph, see [Browse the wiki in Obsidian](browse-with-obsidian.md).
|
||||
|
||||
## Asking from another machine
|
||||
|
||||
A consumer that is not a terminal on this machine can ask the same questions through the MCP read
|
||||
server and get the same answers: [Set up the MCP read server](set-up-the-mcp-server.md).
|
||||
@@ -0,0 +1,64 @@
|
||||
# Run the weekly review
|
||||
|
||||
Go through your committed initiatives once a week and decide what each one needs: a next action, a
|
||||
pause, a close. A machine prepares the list, so you don't have to reconstruct it from memory. The
|
||||
agent runs the `gtd-weekly-review` skill ([skill catalogue](../reference/skills.md)).
|
||||
|
||||
## Two places, on purpose
|
||||
|
||||
Knowledge and commitments change at different speeds, so they live in different places:
|
||||
|
||||
- **A page under `kb/gtd/`** is one initiative's durable memory: its goal, who is involved, where it
|
||||
stands, and why it is worth doing. It never summarises the task list.
|
||||
- **The open items** live in a task tracker, which owns them.
|
||||
|
||||
Nothing syncs between the two. `tools/wikitool review` joins them when it reads, matching on the
|
||||
project name, and stores nothing - not even a report file. Why the split runs this way rather than
|
||||
syncing: [knowledge-and-commitment.md](../explanation/knowledge-and-commitment.md).
|
||||
|
||||
## What you prepare
|
||||
|
||||
A task tracker, configured for this checkout in `.wikitool-tasks.json`. Super Productivity and
|
||||
CalDAV (for example Nextcloud Tasks) are supported. The file's shape, and the difference between a
|
||||
tracker the wiki can only read and one it can write to, is in the
|
||||
[configuration reference](../reference/configuration.md#task-tracker). Without a tracker, the
|
||||
review has nothing to join against. That is a valid state, though, and the pages work without one.
|
||||
|
||||
## What you say
|
||||
|
||||
```text
|
||||
Run the weekly review
|
||||
```
|
||||
|
||||
## What you get back
|
||||
|
||||
The agent runs `tools/wikitool review` and walks you through its findings, one at a time:
|
||||
|
||||
- initiatives with no next action;
|
||||
- waiting-fors past their follow-up date;
|
||||
- tracker projects that have no page;
|
||||
- active pages with no open item;
|
||||
- someday items that have gone stale.
|
||||
|
||||
For each finding, the agent asks what is actually true, because "this looks stalled" can mean a
|
||||
next action is missing, the initiative was paused on purpose, or it is finished. Then it proposes the
|
||||
exact change: a new item in the tracker, an item marked done, or a page's state moved to `dormant`,
|
||||
`completed` or `abandoned`. Nothing is written to the tracker until you confirm that exact call.
|
||||
What was decided goes onto the page, and the page is published.
|
||||
|
||||
A quiet week is a valid result. If one of the checks could not run because the tracker did not
|
||||
answer, the agent says which check was skipped. An empty result under a skipped check means nothing.
|
||||
|
||||
## Starting something new
|
||||
|
||||
Ask the agent for a new project and name it. It runs `tools/wikitool new project`, which gives the
|
||||
initiative its page under `kb/gtd/` and, where a tracker is configured, a tracker project under the
|
||||
same name. A CalDAV tracker gets that project created automatically. On Super Productivity the
|
||||
command stops with exit 42 and asks you to create the project in the app yourself. Tell the agent
|
||||
when you are done, and it continues with `--resume`. A tracker the wiki can only read refuses the
|
||||
command outright.
|
||||
|
||||
A single open item with no page of its own - a commitment that came with a source, say - is filed
|
||||
with `tools/wikitool task new`. `task list` reads a project's open items back, and `task close --id`
|
||||
marks one done. Nothing deletes an item. The commands are in the
|
||||
[command reference](../reference/commands/pages.md).
|
||||
@@ -0,0 +1,251 @@
|
||||
# Set up the MCP read server
|
||||
|
||||
Serve an instance's wiki read-only over MCP, so that a consumer that is not a terminal on this
|
||||
machine - another agent, a chat client, a service - can ask it the same questions and get the same
|
||||
answers. This page covers what you set up around the server: the client, the authentication in
|
||||
front of it, the optional `submit` path, and how to check that it works. Installing, starting and
|
||||
syncing the server itself is
|
||||
[instructions/mcp-read-server.md](../../instructions/mcp-read-server.md), which an agent follows
|
||||
and which is readable for you as well.
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
|
||||
- [What the server is, and what it is not](#what-the-server-is-and-what-it-is-not)
|
||||
- [What you prepare](#what-you-prepare)
|
||||
- [Wiring a client to it](#wiring-a-client-to-it)
|
||||
- [Running it as a service](#running-it-as-a-service)
|
||||
- [Authentication in front of it](#authentication-in-front-of-it)
|
||||
- [Keeping the served tree current](#keeping-the-served-tree-current)
|
||||
- [Optional: the submit path](#optional-the-submit-path)
|
||||
- [Checking that it works](#checking-that-it-works)
|
||||
- [Troubleshooting](#troubleshooting)
|
||||
- [What is deliberately not here](#what-is-deliberately-not-here)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
## What the server is, and what it is not
|
||||
|
||||
The server is a second consumer of the same core as the CLI, not a second program. Both call the
|
||||
same functions, and a golden test in the origin repository holds their output together. What
|
||||
`tools/wikitool search --json` returns, the MCP tool `search` returns too, together with the commit
|
||||
the answer was computed from.
|
||||
|
||||
It is not a write path into `kb/`. No tool creates, changes or publishes a page - not because a list
|
||||
is filtered, but because the server imports nothing under `tools/chemenu/commands/`. Those functions
|
||||
cannot be reached from its process. The one exception is opt-in: a sixth tool, `submit`, which
|
||||
writes only into a quarantine no other command reads. A human decides about everything that leaves
|
||||
it ([below](#optional-the-submit-path)).
|
||||
|
||||
| Tool | What it answers |
|
||||
|---|---|
|
||||
| `search` | Pages in `kb/` by text, by frontmatter (`!sources`, `tags~k8s`), or both |
|
||||
| `types` | Which page types this wiki knows |
|
||||
| `describe_type` | A type's full contract: fields, required values, enums |
|
||||
| `lint` | Structural findings: broken wikilinks, orphans, catalog drift, schema gaps |
|
||||
| `status` | A snapshot: page count, distribution over collections, finding counts |
|
||||
| `submit` *(optional)* | Hands a document to the review queue - into a quarantine, never into `kb/` |
|
||||
|
||||
## What you prepare
|
||||
|
||||
- A working instance ([INSTALL.md](../../INSTALL.md)), including `tools/.venv` and ripgrep.
|
||||
- For a deployment, a reverse proxy that can authenticate requests before they reach the server
|
||||
([below](#authentication-in-front-of-it)).
|
||||
|
||||
Then ask the agent:
|
||||
|
||||
```text
|
||||
Set up the MCP read server for this instance
|
||||
```
|
||||
|
||||
It installs the server's optional dependency, which is deliberately not in
|
||||
`tools/requirements.txt`: an instance that only uses the CLI should not have to install pydantic,
|
||||
starlette, uvicorn and cryptography. It also checks that tracing cannot write into the served tree.
|
||||
The server refuses to start if it could:
|
||||
|
||||
```text
|
||||
ERROR Telemetry is on and would write into the served checkout (...). Set WIKI_TRACE=0,
|
||||
or point WIKI_TRACE_DIR outside the corpus.
|
||||
```
|
||||
|
||||
That refusal is expected, not an installation fault. Telemetry is off by default in an instance
|
||||
installed from a release and on in a clone of the origin repository
|
||||
([configuration](../reference/configuration.md#telemetry)). The server consults the same policy
|
||||
`wikitool doctor` does. Setting `WIKI_TRACE=0` explicitly, as every example here does, is correct
|
||||
either way and makes the check moot.
|
||||
|
||||
## Wiring a client to it
|
||||
|
||||
Locally, over `stdio`, the client starts the server itself: one process per consumer, no network.
|
||||
The configuration follows the usual MCP client shape, with absolute paths, because the client
|
||||
inherits no working directory:
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"chemenu": {
|
||||
"command": "/path/to/instance/tools/.venv/bin/python",
|
||||
"args": ["-m", "chemenu.mcp"],
|
||||
"cwd": "/path/to/instance/tools",
|
||||
"env": {
|
||||
"WIKI_TRACE": "0",
|
||||
"CHEMENU_ROOT": "/path/to/instance"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`CHEMENU_ROOT` says **which** wiki is served. Without it, the server takes the checkout its own
|
||||
package sits in. That is enough for a single instance, but with several corpora it is safer to always
|
||||
set it. The process speaks MCP over stdin and stdout and prints nothing of its own, which is correct.
|
||||
|
||||
## Running it as a service
|
||||
|
||||
Deployed, the server speaks `streamable-http`, the only transport a reverse proxy can sit in front
|
||||
of. The command a service unit or container runs looks like this:
|
||||
|
||||
```bash
|
||||
WIKI_TRACE=0 CHEMENU_ROOT=/srv/chemenu \
|
||||
tools/.venv/bin/python -m chemenu.mcp \
|
||||
--transport streamable-http --host 0.0.0.0 --port 8000
|
||||
```
|
||||
|
||||
The endpoint is then `http://<host>:8000/mcp`. `--host 0.0.0.0` is deliberately not the default:
|
||||
without it the server binds to loopback, which is right locally and wrong in a container, where the
|
||||
proxy has to reach it. If you set it, make sure something stands in front. `sse` is not offered. It
|
||||
is the superseded remote transport, and building on it now would only postpone the switch.
|
||||
|
||||
## Authentication in front of it
|
||||
|
||||
**The server does not authenticate on its own, by design.** A request that is not properly
|
||||
authenticated should never reach the Python process at all. The authentication is a Traefik
|
||||
ForwardAuth middleware:
|
||||
|
||||
> **<https://gitea.nehmer.net/torben/gitea-mcp-forward-auth>**
|
||||
|
||||
In short, it checks `Authorization: Bearer <token>` against SHA-256 hashes of allowed tokens. It
|
||||
answers `200` for a valid token and `401` for a missing or wrong one, and always leaves
|
||||
`GET /healthz` open. Plaintext tokens appear neither in the configuration nor in the log, only
|
||||
hashes and a short fingerprint. It is configured with `AUTH_PROXY_TOKEN_HASHES_DIR` (a directory with
|
||||
one file per token hash, which suits a Kubernetes secret volume) or `AUTH_PROXY_TOKEN_HASHES` (a
|
||||
comma-separated list). The README there has the details, reference manifests and a test script.
|
||||
|
||||
To create a token hash:
|
||||
|
||||
```bash
|
||||
echo -n "my-token" | sha256sum | awk '{print $1}'
|
||||
```
|
||||
|
||||
**Rate limiting belongs in the same place,** in front of the process and beside the
|
||||
authentication. It does not belong in the Iteration Budget Gate. That gate stops an *agent session*
|
||||
from iterating over the wiki's state unnoticed, which is why retrieval is exempt from it. Using it
|
||||
as a rate limiter would water it down.
|
||||
|
||||
## Keeping the served tree current
|
||||
|
||||
The server reads the working tree, and a stale checkout answers confidently and wrongly. So **every
|
||||
answer carries the commit** it was computed from:
|
||||
|
||||
```json
|
||||
{ "commit": "<40-character SHA>", "as_of": "<ISO-8601, UTC>", "count": 3, "...": "..." }
|
||||
```
|
||||
|
||||
A timer beside the server keeps the tree current by polling: a fetch, then a hard reset to the
|
||||
remote branch. The two commands, and why the hard reset is load-bearing, are in the instruction's
|
||||
sync step. Polling rather than a webhook, because it needs no inbound endpoint and no signature
|
||||
check, which is a smaller attack surface than the thing a webhook would optimise. **Never add
|
||||
`git clean` to that timer.** The reset leaves every gitignored path alone, which is what keeps the
|
||||
`submit` quarantine and any traces intact. A clean would silently delete a submission nobody has
|
||||
reviewed yet.
|
||||
|
||||
## Optional: the submit path
|
||||
|
||||
Without this, `submit` does not exist as a tool - not unused, but not registered. A file
|
||||
`.wikitool-upload.json` in the served corpus turns it on. Its fields, defaults and what a malformed
|
||||
file does are in the [configuration reference](../reference/configuration.md#mcp-submissions); a
|
||||
broken file never means "no limits", it stops the server from starting.
|
||||
|
||||
**With `submit` on, the middleware has a second duty.** It has to set the identity header itself, on
|
||||
every authenticated request, and discard any copy the client sent. The server trusts that header as
|
||||
the value for `submitter`. A header the client could set itself would be a claim, not an identity.
|
||||
If `identity_header` names a header the middleware does not set, every submission is refused for
|
||||
lack of an identity.
|
||||
|
||||
Submitted files land in `mcp-upload/<id>/`, which is gitignored and read by no other command. From
|
||||
there a human promotes or rejects them, in the same checkout as the server rather than in its
|
||||
process. That review is part of
|
||||
[ingesting a source](ingest-a-source.md#a-document-submitted-from-outside).
|
||||
|
||||
## Checking that it works
|
||||
|
||||
The quickest test without a client starts the server over stdio, lists its tools and asks one
|
||||
question:
|
||||
|
||||
```bash
|
||||
cd tools && WIKI_TRACE=0 .venv/bin/python - <<'EOF'
|
||||
import asyncio, os
|
||||
from mcp import ClientSession, StdioServerParameters
|
||||
from mcp.client.stdio import stdio_client
|
||||
|
||||
async def main():
|
||||
params = StdioServerParameters(
|
||||
command=".venv/bin/python", args=["-m", "chemenu.mcp"],
|
||||
env={"WIKI_TRACE": "0", "PATH": os.environ["PATH"]},
|
||||
)
|
||||
async with stdio_client(params) as (r, w):
|
||||
async with ClientSession(r, w) as s:
|
||||
await s.initialize()
|
||||
print("Tools:", [t.name for t in (await s.list_tools()).tools])
|
||||
out = await s.call_tool("status", {})
|
||||
d = getattr(out, "structuredContent", None) or out.structured_content
|
||||
print("Pages:", d["pages"], "| Commit:", d["commit"])
|
||||
|
||||
asyncio.run(main())
|
||||
EOF
|
||||
```
|
||||
|
||||
Expected output, roughly:
|
||||
|
||||
```text
|
||||
Tools: ['search', 'types', 'describe_type', 'lint', 'status']
|
||||
Pages: 176 | Commit: 576df2cdddc96614a7e6641e562022d52112d411
|
||||
```
|
||||
|
||||
Then cross-check against the CLI. Both have to give the same answer:
|
||||
|
||||
```bash
|
||||
tools/wikitool search "<term>" --json
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
**`ERROR Telemetry is on and would write into the served checkout`**: expected behaviour, not an
|
||||
installation fault. Set `WIKI_TRACE=0`, or point `WIKI_TRACE_DIR` outside the corpus.
|
||||
|
||||
**`ModuleNotFoundError: No module named 'mcp'`**: the optional dependency is not installed, or
|
||||
the client starts a different Python than the instance's. Give the client the absolute path to
|
||||
`tools/.venv/bin/python`.
|
||||
|
||||
**`"commit": null` in every answer**: the served tree has uncommitted changes. Either the sync is not
|
||||
running, or something writes into the corpus that has no business there. The five read tools never
|
||||
write. `submit`, when it is on, writes only to `mcp-upload/`, which is gitignored and so is never a
|
||||
cause of `"commit": null` either.
|
||||
|
||||
**`commit` names an old revision**: the sync timer is not running.
|
||||
|
||||
**The server answers differently from `wikitool`**: that is a defect, not a configuration
|
||||
difference, since both go through the same functions. First check that both point at the same root.
|
||||
`CHEMENU_ROOT` is easy to set for one and not the other.
|
||||
|
||||
**Not reachable from outside**: without `--host 0.0.0.0` the server binds to loopback. If it is
|
||||
reachable but every call gets `401`, the middleware is working correctly and the token is wrong.
|
||||
|
||||
## What is deliberately not here
|
||||
|
||||
Deployment specifics - the cluster, ingress hosts, the secret store, the GitOps source - are
|
||||
private infrastructure, and this repository is public.
|
||||
|
||||
There is no container image for the server yet. Until there is, the path above - the venv,
|
||||
`python -m chemenu.mcp`, a proxy in front - is the complete one. With `submit` on, `mcp-upload/`
|
||||
has to survive restarts the same way the rest of the checkout does. Otherwise a submitted file that
|
||||
nobody has reviewed yet loses its quarantine.
|
||||
@@ -0,0 +1,176 @@
|
||||
# Trace and score agent sessions
|
||||
|
||||
Record what an agent did in a session, read the record back, and score it. A score covers the state
|
||||
the session left the wiki in and how it got there: whether it repeated a refused call, or passed a
|
||||
gate's flag without that gate ever refusing. Why this exists, what the trace contains, and what a
|
||||
score means are covered in [telemetry-and-evaluation.md](../explanation/telemetry-and-evaluation.md).
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
|
||||
- [What you prepare](#what-you-prepare)
|
||||
- [Reading a session back](#reading-a-session-back)
|
||||
- [What each harness can report](#what-each-harness-can-report)
|
||||
- [Claude Code](#claude-code)
|
||||
- [Copilot CLI](#copilot-cli)
|
||||
- [VS Code Chat](#vs-code-chat)
|
||||
- [Mistral Vibe](#mistral-vibe)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
## What you prepare
|
||||
|
||||
**Turn tracing on, if it is off.** It is on in a clone of the origin repository and off in an
|
||||
instance installed from a release. `tools/wikitool doctor` reports which state this checkout is in,
|
||||
and why. To turn it on for one checkout, create `.wikitool-telemetry.json` at the root:
|
||||
|
||||
```json
|
||||
{ "enabled": true }
|
||||
```
|
||||
|
||||
The file and its other keys are in the [configuration reference](../reference/configuration.md#telemetry).
|
||||
Everything stays on this machine: traces land under `reports/telemetry/`, which is gitignored, and
|
||||
no exporter is configured.
|
||||
|
||||
**Scope the session to the task.** Every source of events joins on one session id. Setting it
|
||||
yourself keeps one task in one trace:
|
||||
|
||||
```bash
|
||||
export WIKITOOL_SESSION_ID="my-task"
|
||||
```
|
||||
|
||||
[instructions/session-setup.md](../../instructions/session-setup.md) has the line for each shell,
|
||||
and what to do on a harness that starts a fresh shell for every command.
|
||||
|
||||
## Reading a session back
|
||||
|
||||
```bash
|
||||
tools/wikitool eval sessions # which sessions have a trace
|
||||
|
||||
# what did that session do?
|
||||
jq -r '[.ts, .source, .event, (.attrs.command // .attrs.tool_name // "")] | @tsv' \
|
||||
reports/telemetry/my-task/trace.jsonl
|
||||
|
||||
# how well did it do it?
|
||||
tools/wikitool eval score --session my-task
|
||||
tools/wikitool eval score --session my-task --save # also write the scorecard under reports/evals/
|
||||
```
|
||||
|
||||
Both `eval` commands only read, and neither counts against the session's call budget: reading back
|
||||
what a session did is not iterating on the wiki. Their options are in the
|
||||
[telemetry command reference](../reference/commands/telemetry.md).
|
||||
|
||||
To see how a harness hook payload would be recorded, feed one in by hand without writing anything:
|
||||
|
||||
```bash
|
||||
echo '{"session_id":"x","hook_event_name":"post_tool","tool_name":"bash"}' \
|
||||
| tools/trace_ingest.py --source mistral-vibe --dry-run
|
||||
```
|
||||
|
||||
## What each harness can report
|
||||
|
||||
`wikitool` records its own calls on every harness. What the agent did *between* those calls -
|
||||
file reads, searches, shell commands, prompts - comes from the harness's hooks, and harnesses differ
|
||||
a lot in what they offer. Verified against vendor documentation on 2026-08-23:
|
||||
|
||||
| | Claude Code | Copilot CLI | VS Code Chat | Mistral Vibe |
|
||||
|---|---|---|---|---|
|
||||
| Hook events | ~30 | 14 | none | 3 (`pre_tool`, `post_tool`, `post_agent`) |
|
||||
| Session start/end hook | yes | yes | - | **no** |
|
||||
| Prompt submit hook | yes | yes | - | **no** |
|
||||
| Which instructions loaded | yes (`InstructionsLoaded`) | no | no | no |
|
||||
| Block / rewrite a tool call | yes | yes | - | yes |
|
||||
| OTel to your own collector | yes | via MDM `telemetry` | no | **no** - `enable_otel` targets Mistral Studio only |
|
||||
| `TRACEPARENT` to subprocesses | yes | undocumented | - | undocumented |
|
||||
| Programmatic mode | `-p`, `stream-json` | `-p` | no | `-p`, `--output streaming` |
|
||||
| Isolated config home | to be confirmed | `COPILOT_HOME` | no | `VIBE_HOME` |
|
||||
| Local session log | `~/.claude/projects/*.jsonl` (unstable format) | `~/.copilot/session-state/<id>/events.jsonl` | chronicle SQLite | `$VIBE_HOME/logs/` (no format guarantee) |
|
||||
| Wired up here | **partial** - `.claude/settings.json` (`UserPromptSubmit` + a `permissions.ask` rule on the clearing publish) | **yes** - `.github/hooks/wiki-trace.json` | **yes** - `tools/import_chronicle.py` | **yes** - `.vibe/hooks.toml` |
|
||||
|
||||
A rule that needs an event a harness cannot report is shown as *skipped*, not as passed or failed.
|
||||
|
||||
### Claude Code
|
||||
|
||||
`.claude/settings.json` wires `UserPromptSubmit` to `tools/trace_ingest.py`, through
|
||||
`tools/trace-hook` like every hook here. That gives the trace its turn boundaries, which the rule
|
||||
`clearance-ended-the-turn` needs. Without a `prompt.submitted` event it reports "cannot say"
|
||||
instead of a verdict.
|
||||
|
||||
**No `PreToolUse` decision hook is wired, and that is a finding, not an oversight.** Verified against
|
||||
the live CLI on 2026-08-27: a `PreToolUse` hook that returns
|
||||
`hookSpecificOutput.permissionDecision: "ask"` does **not** override a matching `permissions.allow`
|
||||
rule. Permissions are evaluated before the hook's decision, so a hook cannot force a confirmation
|
||||
prompt on a command that is already allowed. A `permissions.ask` rule *does* win, which is why
|
||||
`.claude/settings.json` carries one on the `--confirm` form of `publish`
|
||||
(`Bash(tools/wikitool publish --confirm:*)`). The clearing call prompts, and ordinary publishes below
|
||||
the threshold do not. It is a prefix match, so it depends on `--confirm` coming first, which is why
|
||||
`publish` always prints its re-run line in that order. Treat it as a useful second line, not a
|
||||
guarantee: an agent that reorders the flags gets around it.
|
||||
|
||||
### Copilot CLI
|
||||
|
||||
[.github/hooks/wiki-trace.json](../../.github/hooks/wiki-trace.json) is committed, so a clone
|
||||
brings its own tracing. Eleven events go to `tools/trace_ingest.py`. `disableAllHooks` in
|
||||
`.github/copilot/settings.json` turns them off without deleting anything. `userPromptSubmitted` is
|
||||
among them, so `clearance-ended-the-turn` can be scored on Copilot CLI with no change.
|
||||
|
||||
A `preToolUse` entry that forces a decision on the clearing call would first need Copilot's own
|
||||
decision-document schema verified against a live CLI. That has not happened, and an adapter that
|
||||
cannot be verified is not written.
|
||||
|
||||
Four details of the hook configuration matter. The first two hold for all three harnesses that have
|
||||
hooks:
|
||||
|
||||
- **The interpreter is the venv's, never the script's shebang.** Each `bash` command is
|
||||
`./tools/trace-hook ...`, which runs `trace_ingest.py` with `tools/.venv`'s Python in either venv
|
||||
layout. Each `powershell` command names `tools\.venv\Scripts\python.exe` directly. On Windows in
|
||||
Git Bash, the shebang's `python3` was the Microsoft Store alias, which made every hook a silent
|
||||
no-op. Before the preflight has created the venv, `trace-hook` records nothing and exits 0.
|
||||
- **`./tools/trace-hook` has a PowerShell twin, `tools/trace-hook.ps1`.** PowerShell on Windows
|
||||
resolves the string to the `.ps1` first. Without the twin, it hands the sh script to a file
|
||||
association, and Windows asks which app should open it - on every hook event. Copilot CLI also
|
||||
reads `.claude/settings.json` and runs its single `command` under PowerShell on Windows, so a
|
||||
`bash` field alone does not keep the string out of PowerShell. The twin follows the sh script's
|
||||
rules - venv Python, silent without a venv, exit 0 whatever happens - and stays parseable by
|
||||
Windows PowerShell 5.1, because VS Code starts hooks under 5.1.
|
||||
- **Every command ends in `|| true`.** `preToolUse` hooks fail closed: a non-zero exit denies the
|
||||
tool call. Without the guard, a missing interpreter would turn the observer into a blocker that
|
||||
refuses every tool call in the session. Timeouts fail open, so the 5-second `timeoutSec` is no
|
||||
hazard.
|
||||
- **The event name is passed explicitly.** Copilot serves two payload dialects: camelCase event
|
||||
names give camelCase fields, and PascalCase names give the snake_case shape that VS Code and Claude
|
||||
use. `--event` means the mapping does not depend on which one a configuration chose.
|
||||
|
||||
### VS Code Chat
|
||||
|
||||
VS Code Chat has no hooks, so nothing observes a session while it runs. Its chronicle store
|
||||
(`session-store.db`, the same schema as Copilot CLI's) keeps sessions, turns and touched files, and
|
||||
[tools/import_chronicle.py](../../tools/import_chronicle.py) rebuilds a trace from it afterwards:
|
||||
|
||||
```bash
|
||||
tools/import_chronicle.py --dry-run # this repository's sessions
|
||||
tools/import_chronicle.py --session 0f1e2d3c
|
||||
```
|
||||
|
||||
The store starts empty, and `/chronicle reindex` fills it. That command also syncs session data to
|
||||
your GitHub account, so run it deliberately rather than from a script. A rebuilt trace marks itself
|
||||
`reconstructed: true`, and its `completeness` names `tool.post` but not `tool.pre`: the store records
|
||||
that a file was touched, not that a tool was about to run.
|
||||
|
||||
### Mistral Vibe
|
||||
|
||||
[.vibe/hooks.toml](../../.vibe/hooks.toml) declares the three hooks Vibe has, and
|
||||
[.vibe/config.toml](../../.vibe/config.toml) keeps the telemetry settings in the repository rather
|
||||
than in someone's shell profile. Both were checked against the installed CLI's own loader
|
||||
(`mistral-vibe 2.24.2`) rather than against its documentation. What that turned up:
|
||||
|
||||
- **A failing hook cannot block anything.** With `strict = false`, the default, a crash or a timeout
|
||||
is only a warning. Only a hook that opts into `strict` can deny a tool call - the mirror image of
|
||||
Copilot.
|
||||
- **`post_agent` carries no response text,** only the session context. That is why it maps to
|
||||
`turn.end` rather than `assistant.message`.
|
||||
- **`enable_telemetry` defaults to `true`.** Turning it off is a real change, and it also gates OTel
|
||||
export, which needs both flags on. This repository leaves both off: Vibe's `enable_otel` sends
|
||||
prompts to Mistral Studio.
|
||||
- **Vibe already reads `.agents/skills/` and `AGENTS.md`.** The directory that
|
||||
`wikitool instructions sync` publishes is a project-scope skill source for Vibe, so working with
|
||||
Vibe needs no adaptation, only a trusted folder.
|
||||
@@ -0,0 +1,97 @@
|
||||
# Troubleshoot an instance
|
||||
|
||||
What a common failure means, and what to do about it. During installation itself, the preflight
|
||||
names its own stops - what is missing, why, and the command that fixes it - and
|
||||
[INSTALL.md](../../INSTALL.md) explains them. This page covers an instance that is already set up.
|
||||
|
||||
For the MCP read server, see the troubleshooting section of
|
||||
[Set up the MCP read server](set-up-the-mcp-server.md#troubleshooting).
|
||||
|
||||
## `tools/wikitool` ends with `STOP - this checkout is not set up yet` (exit 42)
|
||||
|
||||
The preflight has not run in this checkout yet, or not since the last update. Run it:
|
||||
|
||||
```bash
|
||||
tools/preflight.sh
|
||||
```
|
||||
|
||||
In PowerShell 7 on Windows, the call is
|
||||
`pwsh -NoProfile -ExecutionPolicy Bypass -File tools/preflight.ps1`. The `-ExecutionPolicy Bypass`
|
||||
applies to that one process only and changes no setting.
|
||||
[instructions/preflight.md](../../instructions/preflight.md) is the procedure, and its output says
|
||||
what to install if anything is missing.
|
||||
|
||||
## The agent offers no skills (`wiki-ingest`, `wiki-query`, ...)
|
||||
|
||||
`.agents/skills/` and `.claude/skills/` are generated and not committed. Run:
|
||||
|
||||
```bash
|
||||
tools/wikitool instructions sync
|
||||
```
|
||||
|
||||
Then restart the agent session. A harness reads its skills only when the session starts.
|
||||
|
||||
## `doctor` reports `personalization: FAIL`
|
||||
|
||||
`USER.md` or `SOUL.md` is missing, or still carries the sentinel line from its template. A renamed
|
||||
template does not count as a filled one. Ask the agent to run the personalization step of
|
||||
[instructions/setup-instance.md](../../instructions/setup-instance.md). For a further clone of an
|
||||
older instance, that is the only step to catch up on.
|
||||
|
||||
## `doctor` reports `environment: WARN`
|
||||
|
||||
`ENVIRONMENT.md` exists but still carries the template's sentinel line. Fill it in (the template is
|
||||
`ENVIRONMENT.md.template`) and remove the line, or delete the file. It is optional, and `absent` is
|
||||
a valid state.
|
||||
|
||||
## `doctor` reports `telemetry: WARN`
|
||||
|
||||
`.wikitool-telemetry.json` holds something telemetry does not read as written: unreadable JSON, a
|
||||
value that is not an object, an `enabled` that is not `true` or `false`, a cap that is not a
|
||||
positive whole number, or a key it does not know. Telemetry itself never stops on this. It ignores
|
||||
the file or the value, or reads `"no"` as on, and the line names the field. Correct the file to the
|
||||
shape in the [configuration reference](../reference/configuration.md#telemetry), or delete it to
|
||||
get the default back.
|
||||
|
||||
## `new` stops with "No author configured"
|
||||
|
||||
Neither `$WIKI_AUTHOR` nor `git config user.name` is set. Run `git config user.name "<Name>"`, or
|
||||
export `WIKI_AUTHOR` ([configuration](../reference/configuration.md)).
|
||||
|
||||
## `publish` ends with exit 42 (Mass-Update Gate)
|
||||
|
||||
Expected when ten or more files are counted, for example on the very first commit of a new instance
|
||||
or after a lint pass. It is not an error. It asks that a human see the output: the complete file
|
||||
list and the exact `--confirm <token>` line that publishes once you approve.
|
||||
[instructions/gates.md](../../instructions/gates.md) has the details.
|
||||
|
||||
## `publish` ends with exit 42 (Publish-Remote Gate)
|
||||
|
||||
This checkout has a `.wikitool-remotes.json`, and the remote being pushed to is not listed in it.
|
||||
Also not an error: the output names the push URL it would write to and the URLs that are allowed.
|
||||
Unlike the Mass-Update Gate, there is **no token and no flag** here. If the target really is right,
|
||||
you add its URL to the file yourself. An agent that edits the file to get past the refusal is
|
||||
opening a gate on its own initiative, which it must never do.
|
||||
|
||||
## Setup or an update fails, and you want to report it
|
||||
|
||||
Ask the agent to follow [instructions/bug-report.md](../../instructions/bug-report.md), or call
|
||||
`tools/bugreport` yourself. It works the same from Bash, Git Bash and PowerShell. It finds a
|
||||
Python 3.8 or newer by itself, skipping the Store aliases `python3` points at on Windows. It needs
|
||||
neither the preflight nor a working `wikitool`, and if it finds no Python, it says why.
|
||||
|
||||
The script writes a bundle to `reports/bugreport-<timestamp>/`, with a zip beside it.
|
||||
|
||||
- **What is removed:** secrets, and page titles from everything the script itself produces.
|
||||
`--titles` keeps the titles.
|
||||
- **What may still be in it:** the session trace is included by default (`--no-trace` leaves it
|
||||
out). Like the chronology and the transcripts, it can contain page content and titles. The bundle
|
||||
also contains machine, user and path names.
|
||||
- **Pseudonymising:** `--pseudonymise` replaces user, host, path, git and remote names with
|
||||
placeholders that keep their length, spaces, hyphens and path depth. The agent can then add more
|
||||
names - people, companies, customers, projects - with `--bundle … --candidates …`. That is a
|
||||
model's judgment, and it leaves a remainder: read the bundle before you share it. The mapping,
|
||||
the checklist and the candidates file contain the originals, and they sit beside the bundle,
|
||||
never in it.
|
||||
|
||||
Nothing is uploaded anywhere. You choose the channel.
|
||||
@@ -0,0 +1,154 @@
|
||||
# Update an instance
|
||||
|
||||
Bring an instance installed from a release up to a newer release of the stack: the tools, types,
|
||||
instructions and contracts. Your pages stay yours. The agent carries out the upgrade by following
|
||||
[instructions/upgrade-instance.md](../../instructions/upgrade-instance.md), which sets the order of
|
||||
the steps, what each one decides, and where the session has to restart. This page covers the
|
||||
decision before it, and what you'll see along the way.
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
|
||||
- [What is running, and is there something newer?](#what-is-running-and-is-there-something-newer)
|
||||
- [Read the notes first](#read-the-notes-first)
|
||||
- [What you say](#what-you-say)
|
||||
- [Where the agent stops](#where-the-agent-stops)
|
||||
- [Where the release comes from, and whom you trust](#where-the-release-comes-from-and-whom-you-trust)
|
||||
- [Two first-time cases](#two-first-time-cases)
|
||||
- [Pitfalls](#pitfalls)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
## What is running, and is there something newer?
|
||||
|
||||
Every instance carries the version of the **stack**, not of its content. It is in `VERSION`, and
|
||||
`.wikitool-release.json` beside it records where the release came from and when it was exported.
|
||||
|
||||
```bash
|
||||
tools/wikitool version # what runs here, and where it came from
|
||||
tools/wikitool version check # is there a newer release?
|
||||
tools/wikitool version notes # what the newest release changes
|
||||
```
|
||||
|
||||
`version check`, `version notes` and `dist upgrade --latest` are the only commands that go to the
|
||||
network. All three ask the same release feed: the one named in the stamp, unless
|
||||
`WIKITOOL_UPDATE_URL` overrides it. A feed that cannot be reached is reported as an error, **never**
|
||||
as "up to date" and never as "no notes".
|
||||
|
||||
## Read the notes first
|
||||
|
||||
What a version number promises: two versions are compatible when they agree in the *leftmost
|
||||
non-zero part*. `0.1.3 → 0.1.4` is a safe update and `0.1.3 → 0.2.0` is not. From `1.0.0` on, the
|
||||
same rule reads as the familiar "a MAJOR breaks". `version check` says which case you are in
|
||||
(`state: update` or `state: migration`).
|
||||
|
||||
That number only answers whether the new machinery can simply be copied over the old. Whether your
|
||||
*content* has to be migrated is a second, independent question. So the release notes of a MAJOR
|
||||
carry two separate lines, and both are worth reading before you start:
|
||||
|
||||
- **Breaking Change:** what stops working, and what this instance has to do about it.
|
||||
- **Migration:** whether and how the pages are rewritten (`none required` when they are not).
|
||||
|
||||
`version notes` prints both. On an instance it reads them from the release feed, because an
|
||||
instance's own `CHANGES.md` is a stub that an update never overwrites. It always asks for the
|
||||
**newest** release, so while `VERSION` still names the old one, it answers with a version other than
|
||||
the one installed, and says so on stderr. If the feed cannot be reached, the error names the release
|
||||
page recorded as `release_url` in `.wikitool-release.json`. `--offline` sends you there directly.
|
||||
|
||||
Why the two questions are kept apart: [version-model.md](../explanation/version-model.md).
|
||||
|
||||
## What you say
|
||||
|
||||
```text
|
||||
Update this instance to the latest release
|
||||
```
|
||||
|
||||
## Where the agent stops
|
||||
|
||||
In the order you'll meet them:
|
||||
|
||||
- **The release notes.** The agent shows you both lines before it touches anything. A migration the
|
||||
instance already owes from an earlier release is finished first.
|
||||
- **A stack file you changed by hand.** `dist upgrade` lists every file that differs from what the
|
||||
installed release shipped, instead of overwriting it silently. For each one, you decide: take the
|
||||
release's version, or keep yours. Keeping yours means the file is reported again at every future
|
||||
upgrade.
|
||||
- **The preflight.** A new release may need other tools than the old one. If something is missing,
|
||||
the preflight stops with exit 42 and says what to install, exactly as during installation.
|
||||
- **The first publish.** The swapped machinery is published on its own, before any content changes.
|
||||
That is far more than ten files, so the Mass-Update Gate (exit 42) shows you the list and the
|
||||
`--confirm` line.
|
||||
- **The session restart.** The running session still holds the *old* `AGENTS.md`, contracts and
|
||||
skills, so the agent asks you to restart it before migrating content. The new session picks up
|
||||
where the old one stopped.
|
||||
- **A required migration.** The agent carries it out, and may come back to you with a question
|
||||
while doing so: a migration document can reach a point that is yours to decide, such as a page
|
||||
that contradicts another.
|
||||
- **An offered migration.** It changes a file the instance owns, so taking it or declining it is
|
||||
your decision.
|
||||
- **The second publish,** of the migrated content, behind the same gate.
|
||||
|
||||
**A required migration can also be interrupted.** If the agent cannot complete one now, it stops
|
||||
after the first publish and leaves it. That is a valid state: new machinery, content still in its
|
||||
old form. `doctor` keeps warning until the migration is done, and a later session resumes with
|
||||
`tools/wikitool migrate status`. The agent never records a migration it did not carry out, because
|
||||
the content's version would then describe a shape the pages are not in.
|
||||
|
||||
If a step fails for a reason that isn't obvious, the agent stops and offers you a bug report rather
|
||||
than improvising ([Troubleshoot](troubleshoot.md)).
|
||||
|
||||
`doctor` warns as long as the content's version (`kb_version` in `.wikitool-kb.json`) is behind
|
||||
`VERSION` and migrations are still open. An instance older than `.wikitool-kb.json` has no such file
|
||||
at all. In that case `tools/wikitool migrate baseline <version>` is called once, and nothing is
|
||||
guessed.
|
||||
|
||||
## Where the release comes from, and whom you trust
|
||||
|
||||
`tools/wikitool dist upgrade --latest --expect <version>` asks the release feed for its newest
|
||||
release, downloads the tarball and its `.sha256` into a scratch directory, checks one against the
|
||||
other and applies the release. The scratch directory is removed on every exit, including
|
||||
`--dry-run`. `--expect` is the version `version notes` showed you: the feed only knows its *newest*
|
||||
release, and `--expect` refuses **before** downloading if a newer one has appeared since, rather
|
||||
than applying something nobody read. Pre-releases (`-beta.N`) need `--pre`.
|
||||
|
||||
**What the checksum does, and what it does not do.** The `.sha256` comes from the same feed as the
|
||||
tarball. It protects against a damaged transfer, not against a compromised feed. Whether a release
|
||||
is genuine rests on your trust in the host whose feed the instance asks (`update_url` in the stamp).
|
||||
Nothing forces https. A configured `http://` feed is a deliberate choice. `WIKITOOL_UPDATE_TOKEN`
|
||||
is sent only to downloads on the same host as the feed.
|
||||
|
||||
**Offline, or with a tarball from someone else:** put the tarball and its `.sha256` side by side,
|
||||
and the agent passes the file instead of `--latest` (`dist upgrade <tarball>`). A tarball has to
|
||||
unpack to exactly one top-level directory, which is the shape the release workflow builds.
|
||||
|
||||
## Two first-time cases
|
||||
|
||||
**The first jump to a release that knows `--latest`:** the option does not exist in the instance
|
||||
yet, because the instruction there belongs to the release being left. Fetch the tarball and the
|
||||
`.sha256` once from the release page, put them side by side, and run `dist upgrade <tarball>`. From
|
||||
the next release on, the instance has `--latest` itself.
|
||||
|
||||
**The first jump to `4.5.0` or later:** `dist upgrade` itself does not exist in the instance yet,
|
||||
because it arrived with `4.5.0`. Use the tool from the unpacked *new* tarball and point it at the old
|
||||
instance:
|
||||
|
||||
```bash
|
||||
tar -xzf chemenu-stack-<version>.tar.gz
|
||||
CHEMENU_ROOT="$PWD" chemenu-stack-<version>/tools/wikitool \
|
||||
dist upgrade chemenu-stack-<version>.tar.gz --dry-run
|
||||
```
|
||||
|
||||
`CHEMENU_ROOT` tells the package which corpus to work on ([configuration](../reference/configuration.md)).
|
||||
Without it, the package would take the unpacked tarball itself for the instance. From the second
|
||||
upgrade on, the instance carries the command and the instruction itself.
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- An instance without a local `.wikitool-release.json`, or with one that has no `files` block (from
|
||||
before `4.5.0`), gives `dist upgrade` nothing to detect a local change against, so it refuses
|
||||
the swap. There is no repair for that today.
|
||||
- Given a tarball or directory, `dist upgrade` downloads nothing. Only `--latest` downloads.
|
||||
- To follow the swap by hand anyway, perhaps with a tool `wikitool` cannot run, use the file list in
|
||||
the `files` block of `.wikitool-release.json`. The exceptions - `kb/CONVENTIONS.md`, every
|
||||
`kb/*/COLLECTION.md`, `.wikitool-kb.json` - are in the
|
||||
[`dist upgrade` record](../reference/commands/distribution-and-versioning.md#dist-upgrade), along
|
||||
with what the command classifies and refuses.
|
||||
@@ -0,0 +1,23 @@
|
||||
# Reference
|
||||
|
||||
Lookup pages: the command surface, exit codes and gates, the skill catalogue, the page types. A
|
||||
reference page describes what is, not how to use it or why it is so - and wherever a source for
|
||||
it exists in the code, it is generated from that source and checked against it rather than
|
||||
written by hand.
|
||||
|
||||
- [Command reference](commands/README.md) - every `wikitool` command, one page per group,
|
||||
generated from the same records as `tools/wikitool <command> -h`.
|
||||
- [Exit codes and gates](exit-codes.md) - which commands can stop and wait for a person, and
|
||||
why.
|
||||
- [Skill catalogue](skills.md) - the skills the stack ships, what each does and which requests
|
||||
start it, generated from their `SKILL.md`.
|
||||
- [Page types](page-types.md) - every page type the stack ships for `kb/`, its subtypes, where
|
||||
its pages are written and its required fields, generated from the type-specs.
|
||||
- [Configuration](configuration.md) - the environment variables `wikitool` reads and the
|
||||
per-checkout files an operator may create. The shape of each file is generated from the record
|
||||
its loader reads it with; the environment variables and what each setting does are written by
|
||||
hand.
|
||||
|
||||
The skill catalogue and the page types describe the stack's shipped defaults, which an instance
|
||||
may change or extend; `tools/wikitool instructions list` and `tools/wikitool types describe <type>`
|
||||
show what one instance has.
|
||||
@@ -0,0 +1,165 @@
|
||||
# Command reference
|
||||
|
||||
Every `wikitool` command, one page per group, generated from the same records `tools/wikitool
|
||||
<command> -h` prints. Each command's section gives its synopsis, options, examples, behaviour,
|
||||
properties and exit codes; what a command exits 42 for, and which gate it belongs to, is
|
||||
collected in [exit codes and gates](../exit-codes.md).
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
|
||||
- [Pages](#pages)
|
||||
- [Links and citations](#links-and-citations)
|
||||
- [Catalog and log](#catalog-and-log)
|
||||
- [Finding and checking](#finding-and-checking)
|
||||
- [Provenance](#provenance)
|
||||
- [Raw material and uploads](#raw-material-and-uploads)
|
||||
- [Git](#git)
|
||||
- [Guideline export](#guideline-export)
|
||||
- [Workshop runs and session budget](#workshop-runs-and-session-budget)
|
||||
- [Types, instructions and docs](#types-instructions-and-docs)
|
||||
- [Telemetry](#telemetry)
|
||||
- [Distribution and versioning](#distribution-and-versioning)
|
||||
- [Content migrations](#content-migrations)
|
||||
- [Instance health](#instance-health)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
<!-- wikitool:reference-commands-index -->
|
||||
## Pages
|
||||
|
||||
Full records: [pages.md](pages.md)
|
||||
|
||||
- [`new`](pages.md#new) - Scaffold a new wiki page of any type.
|
||||
- [`task new`](pages.md#task-new) - Create one open item in the configured task tracker - never a kb/ page.
|
||||
- [`task list`](pages.md#task-list) - List a project's open items - id, title, and whether each carries the WAITING status.
|
||||
- [`task close`](pages.md#task-close) - Mark one tracker item done - never delete it.
|
||||
- [`touch`](pages.md#touch) - Bump a page's `modified:` date and optionally rewrite its other frontmatter fields.
|
||||
- [`rename`](pages.md#rename) - Rename a page, or repoint references that name a page that never existed.
|
||||
- [`rm`](pages.md#rm) - Delete a page and mechanically de-link it from the rest of the wiki.
|
||||
- [`move`](pages.md#move) - Move a page (or every misplaced page) to the directory its type-spec computes.
|
||||
|
||||
## Links and citations
|
||||
|
||||
Full records: [links-and-citations.md](links-and-citations.md)
|
||||
|
||||
- [`xref add`](links-and-citations.md#xref-add) - Declare that A <rel> B.
|
||||
- [`xref remove`](links-and-citations.md#xref-remove) - Remove a cross-reference: the inverse of `xref add`.
|
||||
- [`xref link-source`](links-and-citations.md#xref-link-source) - Batch-link a source page to every entity/concept it mentions.
|
||||
- [`links show`](links-and-citations.md#links-show) - Show the edges out of and into a page.
|
||||
- [`cite id`](links-and-citations.md#cite-id) - Print the deterministic footnote id `cite add` would use for this (title, file) pair.
|
||||
- [`cite add`](links-and-citations.md#cite-add) - Upsert a `[^cite-id]: [[Source - X]]` definition in a page's footnotes region.
|
||||
- [`cite sync`](links-and-citations.md#cite-sync) - Reconcile each page's footnotes region against its actual `[^id]` references.
|
||||
|
||||
## Catalog and log
|
||||
|
||||
Full records: [catalog-and-log.md](catalog-and-log.md)
|
||||
|
||||
- [`index rebuild`](catalog-and-log.md#index-rebuild) - Regenerate the catalog from every page's frontmatter.
|
||||
- [`log append`](catalog-and-log.md#log-append) - Append a formatted entry to `kb/log.md`.
|
||||
- [`log status`](catalog-and-log.md#log-status) - Read-only: count `ingest` entries logged since the last `lint` entry.
|
||||
|
||||
## Finding and checking
|
||||
|
||||
Full records: [finding-and-checking.md](finding-and-checking.md)
|
||||
|
||||
- [`lint`](finding-and-checking.md#lint) - Run structural lint checks against kb/.
|
||||
- [`search`](finding-and-checking.md#search) - Find pages in `kb/` by text and/or frontmatter.
|
||||
- [`review`](finding-and-checking.md#review) - The GTD weekly review.
|
||||
|
||||
## Provenance
|
||||
|
||||
Full records: [provenance.md](provenance.md)
|
||||
|
||||
- [`sources coverage`](provenance.md#sources-coverage) - List raw files with no source page, broken `raw_files:` references, and legacy directory/URL-only source pages.
|
||||
- [`sources trace`](provenance.md#sources-trace) - Trace provenance in either direction: raw file, or page.
|
||||
- [`sources rebuild-index`](provenance.md#sources-rebuild-index) - Regenerate the `kb/provenance.md` reverse index.
|
||||
|
||||
## Raw material and uploads
|
||||
|
||||
Full records: [raw-material-and-uploads.md](raw-material-and-uploads.md)
|
||||
|
||||
- [`raw fetch`](raw-material-and-uploads.md#raw-fetch) - Capture a web page the user names into `incoming/`: the HTML as received plus a derived text, for `raw accept` to promote.
|
||||
- [`raw derive`](raw-material-and-uploads.md#raw-derive) - Derive reading text from a PDF, e-mail, Office file or web page in `incoming/`, or unpack a ZIP, and leave the original beside it, for `raw accept` to promote.
|
||||
- [`raw capture`](raw-material-and-uploads.md#raw-capture) - Capture documentation from a git repository into `incoming/<bundle>/`, with a manifest naming the repository, ref rule and commit, for `raw accept` to promote.
|
||||
- [`raw status`](raw-material-and-uploads.md#raw-status) - Report which captured bundles under `raw/` have fallen behind their repository, with each changed file as `A`/`M`/`D`.
|
||||
- [`raw pending`](raw-material-and-uploads.md#raw-pending) - List what waits in `incoming/`, oldest first, and name the entry an ingest without an argument takes next.
|
||||
- [`raw accept`](raw-material-and-uploads.md#raw-accept) - Promote one or more files, or one folder, from `incoming/` into `raw/`.
|
||||
- [`upload list`](raw-material-and-uploads.md#upload-list) - List every MCP submission currently waiting in the quarantine (`mcp-upload/`).
|
||||
- [`upload show`](raw-material-and-uploads.md#upload-show) - Print one submission's manifest in full.
|
||||
- [`upload accept`](raw-material-and-uploads.md#upload-accept) - **Upload Review Gate:** promote a submission's file from quarantine into `incoming/`.
|
||||
- [`upload reject`](raw-material-and-uploads.md#upload-reject) - Delete a submission's material, keeping only its ledger trail.
|
||||
|
||||
## Git
|
||||
|
||||
Full records: [git.md](git.md)
|
||||
|
||||
- [`sync`](git.md#sync) - Fetch `<remote>/<branch>` and bring the local branch up to date with it.
|
||||
- [`publish`](git.md#publish) - Reconcile with `<remote>/<branch>`, then stage all changes, commit, and push.
|
||||
|
||||
## Guideline export
|
||||
|
||||
Full records: [guideline-export.md](guideline-export.md)
|
||||
|
||||
- [`export guidelines`](guideline-export.md#export-guidelines) - Render the guideline pages into one generated `GUIDELINES.md`, or push it into the captured repositories that opted in (**Guideline Push Gate**).
|
||||
|
||||
## Workshop runs and session budget
|
||||
|
||||
Full records: [workshop-runs-and-session-budget.md](workshop-runs-and-session-budget.md)
|
||||
|
||||
- [`work new`](workshop-runs-and-session-budget.md#work-new) - Scaffold `work/<runkey>/` for one workshop run.
|
||||
- [`work close`](workshop-runs-and-session-budget.md#work-close) - Delete a finished workshop.
|
||||
- [`budget status`](workshop-runs-and-session-budget.md#budget-status) - Show the current session's `wikitool` call count and recent command history.
|
||||
- [`budget reset`](workshop-runs-and-session-budget.md#budget-reset) - Clear the current session's (or every session's) iteration budget state.
|
||||
|
||||
## Types, instructions and docs
|
||||
|
||||
Full records: [types-instructions-and-docs.md](types-instructions-and-docs.md)
|
||||
|
||||
- [`types list`](types-instructions-and-docs.md#types-list) - List every type-spec under `types/`.
|
||||
- [`types describe`](types-instructions-and-docs.md#types-describe) - Print one type's full contract.
|
||||
- [`instructions sync`](types-instructions-and-docs.md#instructions-sync) - Publish every `instructions/<name>/SKILL.md` into the harness skill directories.
|
||||
- [`instructions verify`](types-instructions-and-docs.md#instructions-verify) - Check the instruction layer.
|
||||
- [`instructions list`](types-instructions-and-docs.md#instructions-list) - List the flat instructions with their descriptions.
|
||||
- [`docs verify`](types-instructions-and-docs.md#docs-verify) - Check the docs that mirror the code.
|
||||
- [`docs toc`](types-instructions-and-docs.md#docs-toc) - Create, refresh or remove the generated table-of-contents region.
|
||||
- [`docs prerequisites`](types-instructions-and-docs.md#docs-prerequisites) - Regenerate `INSTALL.md`'s prerequisites lists from `tools/prerequisites.txt`.
|
||||
- [`docs contract`](types-instructions-and-docs.md#docs-contract) - Regenerate `tools/CONTRACT.md`'s `<!-- wikitool:commands -->` region.
|
||||
- [`docs reference`](types-instructions-and-docs.md#docs-reference) - Regenerate the human reference under `docs/reference/`.
|
||||
|
||||
## Telemetry
|
||||
|
||||
Full records: [telemetry.md](telemetry.md)
|
||||
|
||||
- [`eval sessions`](telemetry.md#eval-sessions) - List the sessions that have a trace under `reports/telemetry/`.
|
||||
- [`eval score`](telemetry.md#eval-score) - Score one traced session.
|
||||
|
||||
## Distribution and versioning
|
||||
|
||||
Full records: [distribution-and-versioning.md](distribution-and-versioning.md)
|
||||
|
||||
- [`dist export`](distribution-and-versioning.md#dist-export) - Write a contentless, distributable copy of this repo's machinery.
|
||||
- [`dist adopt`](distribution-and-versioning.md#dist-adopt) - Take shipped templates as this instance's own: copy each to its unsuffixed name.
|
||||
- [`dist upgrade`](distribution-and-versioning.md#dist-upgrade) - Apply a stack update `dist export` produced - the write half of `version check`.
|
||||
- [`version show`](distribution-and-versioning.md#version-show) - Print this instance's stack version and where it came from.
|
||||
- [`version check`](distribution-and-versioning.md#version-check) - Ask the origin's release feed whether a newer stack exists.
|
||||
- [`version notes`](distribution-and-versioning.md#version-notes) - Print one version's release notes.
|
||||
- [`version bump`](distribution-and-versioning.md#version-bump) - Raise or continue the one running candidate between two releases.
|
||||
- [`version regrade`](distribution-and-versioning.md#version-regrade) - List the running candidate's bump titles with their impact grade, or change one or more of them.
|
||||
- [`version release`](distribution-and-versioning.md#version-release) - Fix the running candidate: strip `VERSION`'s `-beta.N` suffix and close its `CHANGES.md` entry.
|
||||
|
||||
## Content migrations
|
||||
|
||||
Full records: [content-migrations.md](content-migrations.md)
|
||||
|
||||
- [`migrate list`](content-migrations.md#migrate-list) - List every migration document under `instructions/migrations/`.
|
||||
- [`migrate status`](content-migrations.md#migrate-status) - Show the migrations this instance still owes, in the order they must run.
|
||||
- [`migrate verify`](content-migrations.md#migrate-verify) - Compare `kb/` against a git revision on the invariants a content migration must not change.
|
||||
- [`migrate done`](content-migrations.md#migrate-done) - Record one migration as applied, advancing `kb_version` in `.wikitool-kb.json`.
|
||||
- [`migrate baseline`](content-migrations.md#migrate-baseline) - Declare `kb_version` once, for an instance predating `.wikitool-kb.json`.
|
||||
|
||||
## Instance health
|
||||
|
||||
Full records: [instance-health.md](instance-health.md)
|
||||
|
||||
- [`doctor`](instance-health.md#doctor) - Check that this instance is correctly configured.
|
||||
<!-- /wikitool:reference-commands-index -->
|
||||
@@ -0,0 +1,157 @@
|
||||
# Catalog and log
|
||||
|
||||
The generated catalog of the wiki and its append-only operations log.
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
|
||||
- [`index rebuild`](#index-rebuild)
|
||||
- [`log append`](#log-append)
|
||||
- [`log status`](#log-status)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
<!-- wikitool:reference-commands -->
|
||||
## `index rebuild`
|
||||
|
||||
Regenerate the catalog from every page's frontmatter.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool index rebuild [--dry-run]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
--dry-run Print what would be written instead of writing it
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool index rebuild
|
||||
tools/wikitool index rebuild --dry-run
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Rewrites `kb/index.md` as a map: statistics, one row per collection and per area, and links to the shards - no page rows.
|
||||
- Writes the page tables to a generated `INDEX.md` in each collection. An area with more than 50 rows gets its own `INDEX.md` in its directory.
|
||||
- Deletes stale shards - an `INDEX.md` of a collection or area that no longer exists - in the same pass.
|
||||
- Warns about every page nested more than one directory below its collection; it is still catalogued, folded into its area.
|
||||
- `--dry-run` prints every file it would write and every stale shard it would remove, and writes nothing.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: yes
|
||||
- Idempotent: yes
|
||||
- Atomic: No - each catalog file (the map plus one shard per collection/area) is rewritten independently, then stale shards are removed; an interruption can leave some regenerated and others not
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - An I/O error while writing or removing a catalog file (rare)
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool search` - finds a page without reading the catalog
|
||||
- `wikitool lint` - its Nested Pages finding is what the warning previews
|
||||
- `instructions/publish-cycle.md` - where a write session runs this
|
||||
|
||||
## `log append`
|
||||
|
||||
Append a formatted entry to `kb/log.md`.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool log append --op ingest|query|lint|create|update|delete|rename|move --title "..." [--body "..."|--body-file path]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
--op <str> ingest|query|lint|create|update|delete|rename|move
|
||||
[required]
|
||||
--title <str> Brief description, e.g. a source path [required]
|
||||
--body <str> Optional multi-line details
|
||||
--body-file <path> Read the body from a file instead of --body
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool log append --op ingest --title "raw/articles/docker-cheatsheet.md" --body "Created [[Docker]]; updated [[Container]]."
|
||||
tools/wikitool log append --op lint --title "2026-09-26" --body-file lint-summary.md
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Appends one entry to `kb/log.md`: a `## [YYYY-MM-DD] <op> | <title>` heading, the body if one is given, and a `---` separator.
|
||||
- Not idempotent: every successful run appends a new entry, including a repeated one.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: yes
|
||||
- Idempotent: no
|
||||
- Atomic: Yes - single append
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - `--op` is not one of ingest, query, lint, create, update, delete, rename, move
|
||||
- `1` - `--body-file` is missing, not a readable file, or not valid UTF-8
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool log status` - counts the ingests logged since the last lint
|
||||
- `instructions/publish-cycle.md` - where a write session runs this
|
||||
|
||||
## `log status`
|
||||
|
||||
Read-only: count `ingest` entries logged since the last `lint` entry.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool log status`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool log status
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Counts the `ingest` entries in `kb/log.md` after the most recent `lint` entry, or since the start of the log if it was never linted, and the total number of entries.
|
||||
- At 10 or more it prints that the `wiki-lint` skill is due next - the every-10-sources full lint of the Maintenance Schedule.
|
||||
- Read-only; safe to retry freely.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: no
|
||||
- Idempotent: yes
|
||||
- Atomic: Read-only
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `0` - `kb/log.md` is missing or empty - reported as nothing logged, not a failure
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool log append` - writes the entries this counts
|
||||
- `wiki-lint` skill - what the threshold asks for
|
||||
- `tools/CONTRACT.md` § Maintenance schedule - the cadence this reports on
|
||||
<!-- /wikitool:reference-commands -->
|
||||
@@ -0,0 +1,267 @@
|
||||
# Content migrations
|
||||
|
||||
The migrations a release may require of an instance's content, and recording that they ran.
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
|
||||
- [`migrate list`](#migrate-list)
|
||||
- [`migrate status`](#migrate-status)
|
||||
- [`migrate verify`](#migrate-verify)
|
||||
- [`migrate done`](#migrate-done)
|
||||
- [`migrate baseline`](#migrate-baseline)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
<!-- wikitool:reference-commands -->
|
||||
## `migrate list`
|
||||
|
||||
List every migration document under `instructions/migrations/`.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool migrate list [--json]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
--json Print the migrations as JSON
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool migrate list
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Lists every migration document under `instructions/migrations/`, oldest target first, with its kind and obligation.
|
||||
- Never fails. Read-only and **exempt from the Iteration Budget Gate**.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: no
|
||||
- Idempotent: yes
|
||||
- Atomic: Read-only
|
||||
- Counts toward the session call budget: no
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool migrate status` - which of them this instance still owes
|
||||
- `instructions/migrate-corpus.md` - how a migration is run
|
||||
|
||||
## `migrate status`
|
||||
|
||||
Show the migrations this instance still owes, in the order they must run.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool migrate status [--json]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
--json Print the chain as JSON
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool migrate status
|
||||
tools/wikitool migrate status --json
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Shows every **required** migration whose `migrates_to` lies in `(kb_version, VERSION]`, in the order it must run.
|
||||
- `offered` migrations are listed separately above the chain: they never block, never count as owed, and are bounded by the applied ledger rather than by `kb_version` - taking one does not move the version.
|
||||
- With a release stamp present, also reports which shipped files this instance has since edited (from the per-file sha256 in `.wikitool-release.json`) - which says whether an offer may be copied over or has to be reconciled by hand. Without a stamp that question is reported as unanswerable rather than answered.
|
||||
- Exits 1 only when the content version is undeclared (`.wikitool-kb.json` missing) or `VERSION` is unreadable; it never guesses the content's shape.
|
||||
- Read-only, safe to retry freely, and exempt from the Iteration Budget Gate.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: no
|
||||
- Idempotent: yes
|
||||
- Atomic: Read-only
|
||||
- Counts toward the session call budget: no
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - `.wikitool-kb.json` is missing - the content version is undeclared
|
||||
- `1` - `VERSION` is unreadable
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool migrate done` - records one as applied
|
||||
- `instructions/migrate-corpus.md` - how a migration is run
|
||||
- `instructions/upgrade-instance.md` - where an upgrade checks this
|
||||
|
||||
## `migrate verify`
|
||||
|
||||
Compare `kb/` against a git revision on the invariants a content migration must not change.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool migrate verify --from <rev> [--path P ...] [--expect-body-change] [--json] [--fail-on-error]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
--from <str> Git revision to compare against, e.g. HEAD [required]
|
||||
--path <str> Limit to a subtree, repeatable (e.g. kb/concepts)
|
||||
--expect-body-change Also report pages whose body did not change at all
|
||||
--json Print the diff as JSON
|
||||
--fail-on-error Exit 1 if any invariant changed
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool migrate verify --from HEAD
|
||||
tools/wikitool migrate verify --from HEAD --path kb/concepts --expect-body-change
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Compares `kb/` against the revision `--from` on the invariants a content migration must not change: wikilink and citation **counts** (not sets), footnote definitions, H1, structural frontmatter, and the **count of generated-region marker pairs**.
|
||||
- Pages are matched by **title**, not path, so a page `move` (or `move --reconcile`) relocated compares as itself - reported separately as `moved` - rather than as a removed-and-added pair.
|
||||
- Reports added and removed pages without failing on them.
|
||||
- `--expect-body-change` additionally flags a page whose body did not change at all.
|
||||
- Not migration-specific: worth running after any bulk rewrite.
|
||||
- Exits 0 whatever it finds unless `--fail-on-error` is passed.
|
||||
- Read-only and exempt from the Iteration Budget Gate.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: no
|
||||
- Idempotent: yes
|
||||
- Atomic: Read-only
|
||||
- Counts toward the session call budget: no
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - Only with `--fail-on-error`: an invariant changed
|
||||
- `1` - `--from` is not a revision in this repository
|
||||
|
||||
**See also**
|
||||
|
||||
- `instructions/migrate-corpus.md` - where a migration runs this
|
||||
- `wikitool lint` - the single-revision checks
|
||||
|
||||
## `migrate done`
|
||||
|
||||
Record one migration as applied, advancing `kb_version` in `.wikitool-kb.json`.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool migrate done <version> [--pages N] [--dry-run]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
Arguments:
|
||||
version The migration's target version, e.g. 1.4.0 [required]
|
||||
|
||||
Options:
|
||||
--pages <int> How many pages it touched
|
||||
--dry-run Report without writing
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool migrate done 7.0.0 --pages 42 --dry-run
|
||||
tools/wikitool migrate done 7.0.0 --pages 42
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Records one migration as applied, advancing `kb_version` in `.wikitool-kb.json` to its target.
|
||||
- **Refuses any required version that is not the next link in the chain.**
|
||||
- An `offered` migration is recorded in the applied ledger *without* moving `kb_version` and with no ordering rule applied. Re-recording one already in the ledger is a no-op, not an error - idempotent and safe to repeat.
|
||||
- Not idempotent for a required migration: it advances the chain.
|
||||
- `--dry-run` reports without writing.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: yes
|
||||
- Idempotent: no
|
||||
- Atomic: Yes - single file write
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - Unknown version, no `.wikitool-kb.json`, or nothing outstanding
|
||||
- `1` - A *required* version that is not the next link in the chain
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool migrate status` - the order to apply them in
|
||||
- `instructions/migrate-corpus.md` - the migration procedure
|
||||
|
||||
## `migrate baseline`
|
||||
|
||||
Declare `kb_version` once, for an instance predating `.wikitool-kb.json`.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool migrate baseline <version> [--force]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
Arguments:
|
||||
version The shape this instance's content is already in [required]
|
||||
|
||||
Options:
|
||||
--force Overwrite an existing declaration (not a substitute for `done`)
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool migrate baseline 6.2.0
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Declares `kb_version` once, for an instance predating `.wikitool-kb.json`.
|
||||
- Refuses to overwrite an existing declaration without `--force`. Advancing the version after a migration is `migrate done`, which checks the chain; this command does not.
|
||||
- Safe to re-run with the same version.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: yes
|
||||
- Idempotent: yes
|
||||
- Atomic: Yes - single file write
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - Unparseable version
|
||||
- `1` - A declaration already exists and `--force` was not passed
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool migrate done` - advances the version after a migration
|
||||
- `wikitool migrate status` - what is owed from the declared version
|
||||
<!-- /wikitool:reference-commands -->
|
||||
@@ -0,0 +1,602 @@
|
||||
# Distribution and versioning
|
||||
|
||||
Exporting, adopting and upgrading an instance, and the version and changelog of the stack itself.
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
|
||||
- [`dist export`](#dist-export)
|
||||
- [`dist adopt`](#dist-adopt)
|
||||
- [`dist upgrade`](#dist-upgrade)
|
||||
- [`version show`](#version-show)
|
||||
- [`version check`](#version-check)
|
||||
- [`version notes`](#version-notes)
|
||||
- [`version bump`](#version-bump)
|
||||
- [`version regrade`](#version-regrade)
|
||||
- [`version release`](#version-release)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
<!-- wikitool:reference-commands -->
|
||||
## `dist export`
|
||||
|
||||
Write a contentless, distributable copy of this repo's machinery.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool dist export <target> [--dry-run] [--source-repo U] [--source-commit SHA] [--release-url U] [--update-url U]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
Arguments:
|
||||
target Directory to write the distribution into. Must not exist, or must be
|
||||
empty. [required]
|
||||
|
||||
Options:
|
||||
--dry-run List what would be written, without writing anything.
|
||||
--source-repo <str> Repository this export was built from (recorded in the
|
||||
stamp)
|
||||
--source-commit <str> Commit this export was built from (recorded in the
|
||||
stamp)
|
||||
--release-url <str> Release page this export ships as (recorded in the
|
||||
stamp)
|
||||
--update-url <str> Release feed `version check` should ask (recorded in
|
||||
the stamp)
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool dist export ../my-wiki --dry-run
|
||||
tools/wikitool dist export ../my-wiki
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Writes a contentless, distributable copy of this repo's machinery into an empty or new `<target>` directory.
|
||||
- Ships `AGENTS.md`/`README.md`/`INSTALL.md` with any `dist:strip-start`...`dist:strip-end` marker region removed, `instructions/` (minus `instructions/dev/`), `types/` (the `root: kb` page type-specs, their schemas and their subtype templates `types/<type>.<value>.md` re-keyed as `.template`, the stack's own verbatim), `docs/` verbatim (minus `docs/dev/`), `tools/` (no venv/caches, and not the dev-only test suite `tools/chemenu/tests/` with its `pytest.ini`/`.coveragerc`), the `.github/hooks/`+`.vibe/` session-tracing config plus `.claude/settings.json`, `kb/CONTRACT.md` (no pages, no areas) and `VERSION`.
|
||||
- Ships `raw/` and `incoming/` as flat roots, each with a `.gitkeep` and no subdirectories. `incoming/.gitkeep` is trackable and survives becoming a git repository, so a plain clone gets the directory without any bootstrap step.
|
||||
- Ships templates, never the filled files: `USER.md.template`/`SOUL.md.template`, `kb/CONVENTIONS.md.template`, and each collection's contract re-keyed as `kb/<name>/COLLECTION.md.template`. The filled `USER.md`/`SOUL.md`/`kb/CONVENTIONS.md`/`kb/<name>/COLLECTION.md`/`types/<page-type>.md`/`types/<page-type>.<value>.md` bind their instance; `find_leaks` refuses a plan carrying one.
|
||||
- Writes a generated `.wikitool-release.json` stamp: version, export date, origin, and a sha256 per exported file - the base a later upgrade compares against.
|
||||
- The four origin options only fill stamp fields: `export` never calls git and cannot discover them.
|
||||
- A build and test tool: every release is an export packed as a tarball, and an instance is installed from such a release, never from an export directly.
|
||||
- One-way: no command reconstructs a distributed instance into a dev instance - work on the stack in a clone of the origin repo.
|
||||
- `--dry-run` lists every file it would write, and writes nothing.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: yes
|
||||
- Idempotent: yes
|
||||
- Atomic: Yes - nothing is written until every file is planned
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - `<target>` exists and is not empty, or is not a directory
|
||||
- `1` - The tree has no readable `VERSION`
|
||||
- `1` - A licence file is missing from the source tree
|
||||
- `1` - The plan would carry this instance's own data (a filled personalization, conventions or page type-spec file)
|
||||
|
||||
**See also**
|
||||
|
||||
- `instructions/setup-instance.md` - installs a release, which is this export as a tarball
|
||||
- `wikitool dist upgrade` - applies a later export to an existing instance
|
||||
- `wikitool version show` - reads the stamp this writes
|
||||
|
||||
## `dist adopt`
|
||||
|
||||
Take shipped templates as this instance's own: copy each to its unsuffixed name.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool dist adopt [<template>...] [--dry-run]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
Arguments:
|
||||
templates... Templates to adopt. Default: every collection contract and page
|
||||
type-spec template.
|
||||
|
||||
Options:
|
||||
--dry-run List what would be copied, without writing anything.
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool dist adopt
|
||||
tools/wikitool dist adopt types/project.md.template types/project.schema.yaml.template
|
||||
tools/wikitool dist adopt --dry-run
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Copies `<name>.template` to `<name>` byte for byte; the template stays where it is, as the base the next `dist upgrade` compares against.
|
||||
- Without a path: every `kb/<collection>/COLLECTION.md.template` and every `types/*.template` (the `root: kb` page type-specs, their schemas and their subtype templates).
|
||||
- With paths: exactly those templates, each of which has to be one of the set above.
|
||||
- Never overwrites: a target that already exists is reported as kept and left untouched.
|
||||
- Not for `kb/CONVENTIONS.md.template` or the personalization templates - those carry a sentinel and are filled in, not copied.
|
||||
- `--dry-run` lists what it would copy and keep, and writes nothing.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: yes
|
||||
- Idempotent: yes
|
||||
- Atomic: No - files are copied one by one; a re-run completes an interrupted one
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - A named path does not exist, or is not a collection contract or page type-spec template
|
||||
|
||||
**See also**
|
||||
|
||||
- `instructions/setup-instance.md` - adopts every template on a fresh instance
|
||||
- `instructions/upgrade-instance.md` - adopts a template a release added
|
||||
- `wikitool dist export` - re-keys these files as `.template` in the first place
|
||||
|
||||
## `dist upgrade`
|
||||
|
||||
Apply a stack update `dist export` produced - the write half of `version check`.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool dist upgrade (<source> | --latest [--expect <version>]) [--dry-run] [--keep-local] [--take-release <path>]... [--prune] [--pre]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
Arguments:
|
||||
source An extracted distribution directory, or a release .tar.gz archive - or
|
||||
use --latest
|
||||
|
||||
Options:
|
||||
--latest Download the release the feed announces as latest
|
||||
instead of naming a <source>
|
||||
--expect <str> With --latest: refuse, before downloading anything,
|
||||
unless the feed's latest release is this version (the
|
||||
one `version notes` was read for)
|
||||
--dry-run Classify and report, without writing anything
|
||||
--keep-local Proceed even with locally changed files - leave each one
|
||||
untouched rather than aborting
|
||||
--take-release <str> Overwrite this locally changed path with the release's
|
||||
version, discarding the local change. Repeatable, and
|
||||
each path must be one this run reports as locally
|
||||
changed. The counterpart to --keep-local, which keeps
|
||||
the change and reports it again on every future upgrade
|
||||
--prune Accepted and ignored: files the new release no longer
|
||||
ships are now deleted by default when unchanged since
|
||||
install
|
||||
--pre Allow a pre-release (-beta.N) source tree - release.yml
|
||||
never publishes one
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool dist upgrade ../chemenu-7.1.0.tar.gz --dry-run
|
||||
tools/wikitool dist upgrade ../chemenu-7.1.0.tar.gz
|
||||
tools/wikitool dist upgrade ../chemenu-7.1.0.tar.gz --take-release tools/README.md
|
||||
tools/wikitool dist upgrade --latest --expect 8.0.0 --dry-run
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Exactly one of `<source>` and `--latest` names the release. `<source>` is an already-fetched export directory or `.tar.gz` release archive and touches no network: the archive is verified against a sibling `.sha256` if one is present (a missing one is a WARN, not a block) and must unpack to exactly one top-level directory - the shape `.gitea/workflows/release.yml` packs.
|
||||
- `--latest` asks the release feed (`update_url` of the local stamp, overridable with `$WIKITOOL_UPDATE_URL`; `$WIKITOOL_UPDATE_TOKEN` is sent along) which release is latest, checks that version - `--expect`, a downgrade, a pre-release without `--pre`, already installed - before any download, then downloads the archive and its `.sha256` into a scratch directory removed on every exit. The checksum is mandatory here (a release without one, or an archive that fails it, is an error, not a WARN) and the archive's own `VERSION` must equal the feed's version. The asset URLs are the feed's own `browser_download_url` values, and the token reaches an asset download only if it is on the feed's origin. The checksum protects against transfer errors, not against a feed that is itself compromised - authenticity is the trust in the feed's host.
|
||||
- `--expect <version>` (only with `--latest`) pins the release the feed may announce: a different latest version is refused before anything is downloaded. Pass the version `wikitool version notes` was read for, in the dry run and in the real run alike.
|
||||
- The write set is exactly the *new* `.wikitool-release.json`'s `files` block, minus what an export re-seeds from a blank template every time (`kb/log.md`, `raw/.gitkeep`) or seeds once and the instance owns from then on (`.wikitool-kb.json`, `CHANGES.md`), plus the stamp itself, always rewritten.
|
||||
- Every candidate path is classified against the *local* `.wikitool-release.json`'s recorded digest: unchanged is overwritten silently, absent from the old stamp is created, and locally modified or locally deleted is **never** silently overwritten.
|
||||
- Locally changed files abort the run with the full list; the abort text names the three answers with the command line filled in, and none of them is the default.
|
||||
- `--keep-local` proceeds and leaves every locally changed file untouched. The new stamp is still written whole, recording the release's digest for files that were not written, so a skipped file keeps diverging and is reported again on every later run.
|
||||
- `--take-release <path>` (repeatable) writes the release's version over the named path, discarding the local change, re-creates it if it was locally deleted, and deletes it if the release no longer ships it. The path then matches the stamp and stops being reported.
|
||||
- The two are decided per path and compose on one call: without `--keep-local`, a locally changed path that no `--take-release` names still aborts the run.
|
||||
- A path in the old stamp but not the new one is no longer part of the release. Still unchanged since installation, it is deleted, as silently as an unchanged file is overwritten; changed since, it is locally changed like any other and needs one of the same answers - `--take-release` deletes it, `--keep-local` keeps it as the instance's own file, which no later upgrade reports again. Either way it is decided in this run: the new stamp no longer names the path. What the instance owns (`kb/log.md`, `CHANGES.md`, `.wikitool-kb.json`, ...) is never a candidate, and neither is any file the old stamp does not name.
|
||||
- A directory left holding nothing - at most a `__pycache__/` of `*.pyc` - once those files are deleted is removed with them, up to but never including the instance root. `--prune` is accepted and ignored; it used to switch the deletion on.
|
||||
- Reports the migration chain the new machinery would owe, but never runs any of it - there is no `migrate run`.
|
||||
- Reports, but does not block on, a crossed compatibility boundary.
|
||||
- The local preconditions - `VERSION`, the release stamp, `.wikitool-kb.json`, no outstanding migration, a clean working tree - are checked before the source is read. Not being a git repository at all is a WARN, not a refusal.
|
||||
- Never touches git - no commit, no push.
|
||||
- `--dry-run` classifies and reports without writing; a pre-release (`-beta.N`) source needs `--pre`. With `--latest` it still downloads and verifies the archive - that is the only way to classify - and removes it again.
|
||||
- An interrupted write is not resumed automatically: compare the tree against the printed classification and finish or revert by hand.
|
||||
- The closing report names `instructions/upgrade-instance.md`, which carries the order for everything after the swap and resumes at `instructions sync`.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: yes
|
||||
- Idempotent: no
|
||||
- Atomic: Yes for every refusal - nothing is written. Once writing starts it is a plain sequential file copy with no partial-state cleanup: an interruption mid-copy (killed process, disk full) can leave the tree part-old, part-new
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: yes
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `0` - The source's version equals the installed one - a no-op success
|
||||
- `1` - Both `<source>` and `--latest`, or neither; or `--expect` without `--latest`
|
||||
- `1` - Local `VERSION` missing, or no local `.wikitool-release.json` with a `files` block
|
||||
- `1` - `.wikitool-kb.json` is missing
|
||||
- `1` - A migration is already outstanding against the *installed* machinery
|
||||
- `1` - The working tree is dirty
|
||||
- `1` - `<source>` does not exist, fails its `.sha256`, or does not unpack to exactly one top-level directory
|
||||
- `1` - `--latest`: the release feed cannot be reached, or answers with something that is not a release
|
||||
- `1` - `--latest --expect`: the feed's latest release is another version than the one expected; nothing was downloaded
|
||||
- `1` - `--latest`: the release publishes no archive or no `.sha256` under the expected name; nothing was downloaded
|
||||
- `1` - `--latest`: an asset download fails, or the archive fails its `.sha256`
|
||||
- `1` - `--latest`: the downloaded archive's `VERSION` is not the version the feed announced
|
||||
- `1` - The source carries no `VERSION`, `.wikitool-release.json` or `files` block
|
||||
- `1` - The source's version is older than the installed one, or a pre-release without `--pre`
|
||||
- `1` - A `--take-release` path this run does not classify as locally changed - the one refusal a `--dry-run` also raises
|
||||
- `1` - Locally changed files that neither `--keep-local` nor a `--take-release` answers for; nothing was written
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool version check` - finds out whether an update exists
|
||||
- `wikitool version notes` - the notes of the release `--expect` should name
|
||||
- `instructions/upgrade-instance.md` - the order after the swap
|
||||
- `docs/how-to/update-an-instance.md` - which release, whether to take it, where the tarball comes from
|
||||
- `wikitool migrate status` - the migrations the report names
|
||||
|
||||
## `version show`
|
||||
|
||||
Print this instance's stack version and where it came from.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool version show [--json]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
--json Print the version and stamp as JSON
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool version show
|
||||
tools/wikitool version show --json
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Prints `VERSION` and where this tree came from: "development tree" without a release stamp, otherwise the export date, source commit and repository from `.wikitool-release.json`, plus the release page when the stamp records one.
|
||||
- `--json` prints the version, its compatibility key, the stamp and the update URL.
|
||||
- Bare `wikitool version` is an alias for this.
|
||||
- Read-only and offline; exempt from the Iteration Budget Gate.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: no
|
||||
- Idempotent: yes
|
||||
- Atomic: Read-only
|
||||
- Counts toward the session call budget: no
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - `VERSION` is missing or unparseable
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool version check` - asks whether a newer stack exists
|
||||
- `wikitool version notes` - prints a version's release notes
|
||||
|
||||
## `version check`
|
||||
|
||||
Ask the origin's release feed whether a newer stack exists.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool version check [--url U] [--timeout S] [--json]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
--url <str> Release feed to ask (default: the stamp's, else the built-
|
||||
in origin)
|
||||
--timeout <float> Seconds to wait for the feed [default: 10.0]
|
||||
--json Print the result as JSON
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool version check
|
||||
tools/wikitool version check --json
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Asks the release feed for its latest release and compares it with `VERSION`: `state` is `current`, `update`, `migration` (the step crosses a compatibility boundary) or `ahead`.
|
||||
- The only command whose whole job is the network call - `version notes` reaches the same feed too, but only as a fallback on a distributed instance, and `dist upgrade --latest` asks it which release to download.
|
||||
- Never reached implicitly from another command (`dist upgrade` asks only when passed `--latest`), needs no key, and times out after `--timeout` seconds (default 10).
|
||||
- The feed is `--url`, else `$WIKITOOL_UPDATE_URL`, else the release stamp's, else the built-in origin. `$WIKITOOL_UPDATE_TOKEN` is only needed if that feed is not readable anonymously.
|
||||
- For `update` or `migration` it prints that applying the release is a separate, manual step (`docs/how-to/update-an-instance.md`).
|
||||
- Read-only; exempt from the Iteration Budget Gate.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: no
|
||||
- Idempotent: yes
|
||||
- Atomic: Read-only, no local writes
|
||||
- Counts toward the session call budget: no
|
||||
- Can reach the network: yes
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - `VERSION` is missing or unparseable
|
||||
- `1` - The feed could not be reached, answered non-JSON, or carried no `tag_name`
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool dist upgrade` - applies the update this reports
|
||||
- `wikitool version notes` - prints the release notes
|
||||
- `docs/how-to/update-an-instance.md` - what the operator decides before an update
|
||||
|
||||
## `version notes`
|
||||
|
||||
Print one version's release notes.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool version notes [--version X.Y.Z] [--offline] [--url U] [--timeout S]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
--version <str> Which entry to print (default: this tree's VERSION)
|
||||
--offline Never ask the release feed: on a distributed instance,
|
||||
whose CHANGES.md carries no entry to print, fail with the
|
||||
release page instead of fetching the notes
|
||||
--url <str> Release feed to ask for the fallback (default: the stamp's,
|
||||
as `version check`)
|
||||
--timeout <float> Seconds to wait for the feed [default: 10.0]
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool version notes
|
||||
tools/wikitool version notes --version 7.0.0
|
||||
tools/wikitool version notes --offline
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Prints the `CHANGES.md` entry for `--version` (default: this tree's `VERSION`).
|
||||
- With no such entry, in a tree with a release stamp (a `dist export` tree) and without `--offline`, it prints the release feed's latest release notes instead. A tree without a stamp (a dev checkout) never asks the feed.
|
||||
- Only the feed's latest release can be asked for. When that is a different version than requested, stderr names it and the notes are printed anyway - the expected case before an upgrade, where `VERSION` still names the release being left.
|
||||
- stdout carries nothing but the notes; the line naming the feed being asked and the one naming the release that answered go to stderr. `release.yml` redirects stdout into the file it posts as the release body.
|
||||
- `--offline` never asks the feed and fails with the stamp's `release_url` instead.
|
||||
- Every failure names the stamp's `release_url` where it has one, so a run that cannot read the notes is still told where they are.
|
||||
- Read-only, safe to retry, and exempt from the Iteration Budget Gate.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: no
|
||||
- Idempotent: yes
|
||||
- Atomic: Read-only
|
||||
- Counts toward the session call budget: no
|
||||
- Can reach the network: yes
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - `--version` is unparseable, `VERSION` is unreadable when `--version` is omitted, or `CHANGES.md` is missing
|
||||
- `1` - No entry for the requested version in a tree with no release stamp (a dev checkout)
|
||||
- `1` - No entry for the requested version, and `--offline` was passed
|
||||
- `1` - No entry for the requested version, and the feed could not be reached or returned a release with an empty `body`
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool version check` - asks the same feed whether a newer stack exists
|
||||
- `wikitool version bump` - writes the entry this prints
|
||||
|
||||
## `version bump`
|
||||
|
||||
Raise or continue the one running candidate between two releases.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool version bump --major|--minor|--patch --title "<...>" [--impact high|medium|low] [--breaking "<what breaks>"] [--no-migration "<reason>"] [--migration-required] [--dry-run]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
--major Bump MAJOR (resets MINOR and PATCH)
|
||||
--minor Bump MINOR (resets PATCH)
|
||||
--patch Bump PATCH
|
||||
--title <str> One-line title for the new/updated CHANGES.md entry
|
||||
[required]
|
||||
--breaking <str> What stops working (recorded in CHANGES.md). Required on
|
||||
the bump that first escalates to a boundary crossing,
|
||||
optional on a later bump of the same crossing candidate
|
||||
- where it joins the reasons already recorded rather
|
||||
than replacing them. Refused on a bump that crosses
|
||||
nothing
|
||||
--no-migration <str> Why the escalation to a boundary crossing needs no
|
||||
content migration (recorded in CHANGES.md)
|
||||
--migration-required Retract this candidate's earlier --no-migration line: a
|
||||
migration is needed after all. Requires a migration
|
||||
document already targeting the new base, and refuses
|
||||
when the entry carries no --no-migration line to
|
||||
retract.
|
||||
--impact <str> high|medium|low - how much this bump matters to a reader
|
||||
of the release notes (default: medium). Grouped into the
|
||||
entry's bump list; `version regrade` corrects it later
|
||||
if the running candidate's own judgment changes.
|
||||
--dry-run Report the change without writing
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool version bump --patch --title "Fix the lint report path" --impact low
|
||||
tools/wikitool version bump --minor --title "New command: wikitool review" --dry-run
|
||||
tools/wikitool version bump --major --title "Rename --gate-file to --remotes-file" --breaking "--gate-file is gone; scripts must pass --remotes-file" --no-migration "no content changes"
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- `VERSION` gets a `-beta.N` suffix: one running candidate between two releases, never a fresh number per bump.
|
||||
- `--major`/`--minor`/`--patch` is **max-wins escalation** against the last release (patch < minor < major): a `--patch` on a MINOR candidate only advances `N`, and escalation never steps back down.
|
||||
- The first bump of a candidate opens its `CHANGES.md` entry - heading, date, author, and a machine-managed `<!-- wikitool:bumps -->` list of every `--title` collected so far, graded by `--impact` (default `medium`). Every later bump of the same candidate updates that entry in place: one entry per candidate, not one per bump.
|
||||
- The bump list renders grouped under `**High/Medium/Low impact**` headings, empty groups omitted - except while every bump is `medium`, where it stays one flat list. `version regrade` corrects a grade after the fact.
|
||||
- Writes the heading, the bump list and the breaking/migration lines; the entry's prose - a summary and one-paragraph `###` sections, one per topic - is left to the author.
|
||||
- Whenever a required migration document (`migrates_to:` equal to the candidate's base) exists, every bump writes or refreshes `**Migration:** required - <paths>`, the documents' paths sorted and comma-separated, in place of any `none required` line. The two forms never stand side by side.
|
||||
- Compatibility follows the **leftmost non-zero component** of the candidate's base, which for this stack (at `1.0.0` and up) means MAJOR: PATCH is a fix, MINOR a compatible capability, MAJOR a version that is **not a drop-in replacement** - any hand-work on update, or a downgrade that no longer works. Whether content must be migrated is a second, independent question.
|
||||
- The bump that first escalates a candidate past the boundary requires `--breaking "<what stops working>"` and, on top of it, a migration document targeting the candidate's base or `--no-migration "<reason>"`. Both are anchored just above the bump list, persist over later bumps of the same candidate without being repeated, and are refused on a bump that crosses nothing at all.
|
||||
- On a later crossing of the same candidate, a further `--breaking` **joins** the reasons already recorded (flat on the marker line while there is one, bullets under a bare marker from the second on; repeating a reason verbatim is a no-op), while a further `--no-migration` **replaces** the single migration line.
|
||||
- `--migration-required` replaces the running candidate's `--no-migration` line with the `required` line; it needs a required migration document already targeting the new base. Nothing retracts a recorded `--breaking` reason.
|
||||
- Enforces that a crossing documents itself, never that the part was chosen correctly.
|
||||
- Not idempotent: every successful run escalates or continues the candidate again.
|
||||
- `--dry-run` reports the step without writing.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: yes
|
||||
- Idempotent: no
|
||||
- Atomic: No - `VERSION` then `CHANGES.md`
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - Not exactly one of `--major`/`--minor`/`--patch`, an empty `--title`, or an unknown `--impact`
|
||||
- `1` - `VERSION` or `CHANGES.md` is missing, or `VERSION` and the changelog's newest entry name different versions
|
||||
- `1` - An escalation to a boundary crossing without `--breaking`, or with neither a migration document targeting the new base nor `--no-migration`
|
||||
- `1` - `--breaking` or `--no-migration` on a bump that crosses nothing
|
||||
- `1` - `--no-migration` while a required migration document targets the new base
|
||||
- `1` - `--migration-required` combined with `--no-migration`, on a bump with no running candidate, with no `--no-migration` line to replace, or without a required migration document already targeting the new base
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool version regrade` - corrects an `--impact` grade
|
||||
- `wikitool version release` - fixes the candidate into a release
|
||||
- `instructions/migrate-corpus.md` - how a migration document is written
|
||||
|
||||
## `version regrade`
|
||||
|
||||
List the running candidate's bump titles with their impact grade, or change one or more of them.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool version regrade [INDICES...] [--impact high|medium|low]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
Arguments:
|
||||
indices... 1-based positions in the rendered bump list to regrade (see the
|
||||
bare listing). Omit to just list.
|
||||
|
||||
Options:
|
||||
--impact <str> high|medium|low - required together with indices
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool version regrade
|
||||
tools/wikitool version regrade 3 7 --impact high
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Without arguments: lists the running candidate's bump titles with their grade, numbered by 1-based position in the rendered list (High before Medium before Low, chronological within a grade). Read-only and exempt from the Iteration Budget Gate.
|
||||
- With indices and `--impact`: sets the grade of every named position in one call, all resolved against a single read of the current list - `version regrade 3 7 --impact high` grades what is at 3 and 7 now, not 7 after 3 has moved. Writes `CHANGES.md` and is counted by the Iteration Budget Gate.
|
||||
- The correction path for an `--impact` grade judged at bump time.
|
||||
- Touches only the topmost entry's bump list - never `VERSION`, never any other part of `CHANGES.md`.
|
||||
- A write is not idempotent against a changed list: after a first success, the same indices may name different bumps.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: yes
|
||||
- Idempotent: no
|
||||
- Atomic: No - `CHANGES.md` only, and only when indices are given
|
||||
- Counts toward the session call budget: only when called with arguments
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - `VERSION` or `CHANGES.md` is missing, `VERSION` and the changelog's newest entry name different versions, or the topmost entry has no bump list
|
||||
- `1` - An index outside the rendered list's range, indices given without `--impact`, or an unknown `--impact`
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool version bump` - sets the grade in the first place
|
||||
- `wikitool version release` - refuses without a summary above this list
|
||||
|
||||
## `version release`
|
||||
|
||||
Fix the running candidate: strip `VERSION`'s `-beta.N` suffix and close its `CHANGES.md` entry.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool version release [--title "<...>"] [--dry-run]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
--title <str> Replace the entry's heading title (default: the last bump's)
|
||||
--dry-run Report the change without writing
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool version release --dry-run
|
||||
tools/wikitool version release --title "Command records rewritten for agents"
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Strips `VERSION`'s `-beta.N` suffix - the candidate's base becomes the release - and closes the candidate's `CHANGES.md` entry.
|
||||
- Without `--title` the heading keeps whichever bump last set it; `--title` replaces it - the normal case for a candidate that collected several bumps, whose entry wants a summarising heading rather than the most recent one.
|
||||
- Leaves the entry's machine-managed bump list untouched, as the record of what happened.
|
||||
- From two bumps on, requires a summary paragraph (at least 200 non-whitespace characters) between the bump list and the first `###` section heading. A candidate with exactly one bump is exempt, since there its own paragraph already is the summary. `--dry-run` runs this check too.
|
||||
- Refuses an entry that would exceed its budget once released: at most 32 000 bytes (UTF-8) as `version notes` prints it, and every `###` section exactly one paragraph of at most 1 000 bytes, heading line excluded. The same check as `docs verify`, run on the released heading; `--dry-run` runs it too.
|
||||
- Commits nothing and pushes nothing. The following `publish` moves `VERSION` onto `main`, which `release.yml` reacts to.
|
||||
- Not idempotent: a second run fails once the suffix is gone.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: yes
|
||||
- Idempotent: no
|
||||
- Atomic: No - `VERSION` then `CHANGES.md`
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - `VERSION` is already a release - there is no running candidate to fix
|
||||
- `1` - `VERSION` or `CHANGES.md` is missing, or `VERSION` and the changelog's newest entry name different versions
|
||||
- `1` - Two or more bumps and no summary paragraph above the `###` sections
|
||||
- `1` - The entry is over 32 000 bytes, or a `###` section has more than one paragraph or a paragraph over 1 000 bytes - the error names each with its size
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool version bump` - opens and continues the candidate
|
||||
- `wikitool version regrade` - shows the bump list the summary sits under
|
||||
- `wikitool publish` - moves the released `VERSION` onto `main`
|
||||
<!-- /wikitool:reference-commands -->
|
||||
@@ -0,0 +1,214 @@
|
||||
# Finding and checking
|
||||
|
||||
Searching the wiki, checking its health, and preparing the weekly review of projects and commitments.
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
|
||||
- [`lint`](#lint)
|
||||
- [`search`](#search)
|
||||
- [`review`](#review)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
<!-- wikitool:reference-commands -->
|
||||
## `lint`
|
||||
|
||||
Run structural lint checks against kb/.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool lint [--json] [--markdown out.md] [--full] [--fail-on-error]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
--json Print the raw findings as JSON and write no report
|
||||
--markdown <path> Write the markdown report here instead of the default
|
||||
reports/Lint Report <date>.md
|
||||
--full Print the whole report instead of only the sections with
|
||||
findings
|
||||
--fail-on-error Exit non-zero if hard errors were found
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool lint
|
||||
tools/wikitool lint --json
|
||||
tools/wikitool lint --fail-on-error
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Structural and provenance checks over `kb/`: broken wikilinks, wikilinks wrapped across a line break, dangling frontmatter references, orphan pages, index drift, schema gaps, duplicate titles, title mismatches, uncovered raw files, broken `raw_files:` refs, raw files claimed by more than one source page, unmarked provenance, citation/frontmatter drift, and unbalanced generated-region markers.
|
||||
- Unportable Titles is a hard finding, and hard at every `kb_version`: a page whose title is not a valid file name on Windows and macOS (forbidden character, reserved name, trailing dot or space), or that collides with another page by case or Unicode normalization. `wikitool rename` is the fix.
|
||||
- Wrapped Wikilinks is a hard finding: a `[[...]]` with a line break inside it. The link graph reads it as the title it folds to (the break and its indentation become one space), so it is not also a broken link unless that title is missing; the fix is to put it back on one line.
|
||||
- Broken Anchors is advisory: a `[[Page#Section]]` whose page exists but has no heading the anchor names (any level, compared without case, inline-code backticks or extra whitespace; every segment of a nested `[[Page#A#B]]`). The link still reaches the page, so nothing else reports it - typically a section that was promoted to a page of its own or renamed. A missing page is Broken Wikilinks instead.
|
||||
- Pages nested more than one directory below their collection are a hard finding - the generated catalog folds these into their area silently rather than merely reading it.
|
||||
- Edges whose label is missing or not authorised by the source collection's `outbound:` are both hard once `kb_version` has reached the release that introduced labelled edges, and advisory below it.
|
||||
- Advisory only: `see-also` edges whose reverse direction already carries a specific label - never migration-gated.
|
||||
- Advisory only: a collection past the catalog's per-area shard threshold that has no areas to shard, reported with the split its subtype field would produce, and only when that split puts every resulting area at or under the threshold.
|
||||
- Advisory only: source pages sitting in the `unclassified` catalog slot.
|
||||
- Advisory only: Long Paths - a file under `kb/` or `raw/` whose path below the instance root is over 160 UTF-16 code units, the budget that keeps a Windows checkout without long paths working. Reported as `{path, length}`; a corpus over the budget breaks no lint run. `wikitool rename` is the fix for a page.
|
||||
- Advisory only: quote-limit overages (>2 blockquotes/page).
|
||||
- Advisory only: Unfilled Template Sections - a page with at least one `##` section whose non-blank lines are all template placeholders (content starting with `TODO`, after an optional list marker, checkbox, table cell or bold field label), reported as `{page, sections}`. Code, generated regions and footnote definitions do not count; an empty section, or one placeholder beside written lines, is no finding. Writing the section from the page's sources or retiring the page is the fix - not a mechanical one.
|
||||
- Prints only the sections that found something and always writes the full report to `reports/Lint Report <date>.md` (or `--markdown`), naming the path. `--full` prints everything; `--json` prints the findings and writes nothing.
|
||||
- Exits 0 whatever it finds unless `--fail-on-error` is passed.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: yes
|
||||
- Idempotent: yes
|
||||
- Atomic: Writes one report file (single atomic write) unless `--json`
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - Only with `--fail-on-error`: hard findings exist
|
||||
|
||||
**See also**
|
||||
|
||||
- `wiki-lint` skill - the procedure that runs this
|
||||
- `wikitool move --reconcile` - fixes Misplaced and Nested Pages
|
||||
- `wikitool rename` - fixes Unportable Titles and, for a page, Long Paths
|
||||
- `wikitool log status` - whether a full lint is due
|
||||
|
||||
## `search`
|
||||
|
||||
Find pages in `kb/` by text and/or frontmatter.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool search ["<text>"] [--field <predicate> ...] [--kind/--subtype/--collection/--tag <v>] [--regex] [--limit N] [--sort [-]<field>] [--backend <name>] [--matches] [--json]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
Arguments:
|
||||
text Text to search for. Omit it to run a pure frontmatter query.
|
||||
|
||||
Options:
|
||||
-f, --field <str> Frontmatter predicate, repeatable (AND). Forms:
|
||||
field=value, field~substring, 'field>=value', 'field:*'
|
||||
(present), '!field' (absent).
|
||||
--kind <str> Shorthand for --field kind=<value>.
|
||||
--subtype <str> Shorthand for --field subtype=<value>.
|
||||
--collection <str> Shorthand for --field collection=<value>.
|
||||
--tag <str> Shorthand for --field tags=<value>.
|
||||
--regex Treat the query as a regex. Off by default: terms are
|
||||
literal.
|
||||
--limit <int> Maximum number of results. 0 for no limit. A capped result
|
||||
says so. [default: 50]
|
||||
--sort <str> Sort by a result field; prefix with '-' to reverse, e.g.
|
||||
-modified.
|
||||
--backend <str> Search backend(s), comma-separated. Default 'rg' (or
|
||||
$WIKITOOL_SEARCH_BACKEND).
|
||||
--matches Print the matching lines under each result.
|
||||
--json Print the results as JSON.
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool search "act runner"
|
||||
tools/wikitool search --field entity_type=system --field '!sources'
|
||||
tools/wikitool search "docker" --collection entities --limit 10 --json
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Finds pages in `kb/` without reading the catalog. Text search runs through a pluggable backend (`rg` today).
|
||||
- `--field` predicates are evaluated on frontmatter - `f=v`, `f~substring`, `'f>=v'`, `'f:*'` (present), `'!f'` (absent) - repeatable and ANDed. With no text this is a pure structured query.
|
||||
- One hit per line, ` | `-separated as `score | kind/subtype | title | path | summary`. **Title and path are never truncated** - the title is the identifier `touch`/`xref`/`cite` take. The summary, the one lossy field and the only one that may contain the separator, goes last, so splitting on `" | "` with `maxsplit=4` is unambiguous.
|
||||
- Scope is pages: the backend walks `kb/` but drops the kb-root meta files, every `COLLECTION.md` and every generated `INDEX.md` - a hand-run grep over `kb/` can add none of them but those.
|
||||
- `--limit` defaults to 50 (`0` for no limit), and **a truncated result says so**: `50 of 182 result(s)` in the table, `total`/`truncated`/`limit` beside `count` in `--json`, where `count` stays the number of results in the payload. `api.search` and the MCP `search` tool carry the same default and the same fields.
|
||||
- A page whose frontmatter does not parse can match no positive predicate, so it is **named** rather than dropped: `--json` always carries an `unreadable` list of `{path, reason}` (usually empty), and the table form writes the same lines to stderr.
|
||||
- An unknown field name is reported with the list of fields that do exist - never answered with an empty result, which would read as "no such pages".
|
||||
- `--regex` is applied by `rg` alone, whose engine is linear; the ranking boosts for title and summary are literal-containment only, so a non-literal pattern is ranked by match count.
|
||||
- `rg` is killed after 30 s and reported as a failure.
|
||||
- Read-only, and **exempt from the Iteration Budget Gate**.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: no
|
||||
- Idempotent: yes
|
||||
- Atomic: Read-only
|
||||
- Counts toward the session call budget: no
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - `rg` is not installed
|
||||
- `1` - `rg` did not finish within 30 s
|
||||
- `1` - A malformed `--field` predicate, or an unknown `--backend`
|
||||
- `1` - An unknown field name; the error lists the fields that exist
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool links show` - the edges around a page once found
|
||||
- `wiki-query` skill - answering a question from the wiki
|
||||
|
||||
## `review`
|
||||
|
||||
The GTD weekly review.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool review [--json]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
--json Print the findings as JSON.
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool review
|
||||
tools/wikitool review --json
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Joins the configured task-tracker provider against `kb/gtd/` project pages over the case-normalized project name, at read time, storing nothing - not even a `reports/` file.
|
||||
- **stalled**: a tracker project with zero open items whose `kb/` page is `state: active`; `dormant`/`completed`/`abandoned` never fire.
|
||||
- **waiting-overdue**: a `WAITING` item whose `follow_up_at` is older than `thresholds.stalled_waiting_days`.
|
||||
- **unpaged-project**: a tracker project with no matching `kb/` page, older than `thresholds.unpaged_project_weeks`.
|
||||
- **no-open-loop**: a `kb/` page `state: active` with no matching tracker project, or one with zero open items - the reverse direction of unpaged-project, so a rename on either side surfaces on both.
|
||||
- **someday-stale**: a someday/maybe item untouched for longer than `thresholds.someday_stale_months`.
|
||||
- A value a provider genuinely cannot supply - a `WAITING` item with no `follow_up_at`, a tracker project with no determinable creation date - is its own finding (`waiting_no_follow_up`/`project_age_unknown`) rather than a silent skip.
|
||||
- Thresholds come from the tracker configuration, never from the schema.
|
||||
- The tracker configuration is read from `.wikitool-tasks.json`, or from the file `WIKITOOL_TASKS_CONFIG` names when that variable is set (so one checkout can be run against several trackers in turn). A set variable that names no file is an error, never "no tracker configured".
|
||||
- Text output is one `[check] project: message` line per finding, preceded by a `Source:` line naming which access path answered and, for `superproductivity`'s `access: "snapshot"`, the snapshot's age. `--json` carries the same findings plus `checks_run`/`checks_skipped`/`kb_project_count`/`complete`/`source` (`{"kind": ..., "detail": ...}` or `null`).
|
||||
- A provider that cannot be reached mid-run degrades only the checks that needed the failing call; the report is printed in full, then exit 1 follows - never rendered as if it were complete.
|
||||
- Re-reads everything fresh on every call, so nothing is ever stale to re-fetch.
|
||||
- Read-only, and **exempt from the Iteration Budget Gate**.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: no
|
||||
- Idempotent: yes
|
||||
- Atomic: Read-only
|
||||
- Counts toward the session call budget: no
|
||||
- Can reach the network: yes
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - No `.wikitool-tasks.json`, or a malformed one - a clear "no tracker configured" message
|
||||
- `1` - `WIKITOOL_TASKS_CONFIG` names a file that does not exist or is broken
|
||||
- `1` - The provider was reachable at config-parse time but a read call failed mid-run; the full report (findings plus which checks ran) was printed first
|
||||
|
||||
**See also**
|
||||
|
||||
- `gtd-weekly-review` skill - turns the findings into decisions
|
||||
- `wikitool task list` / `wikitool task close` - act on an item
|
||||
- `wikitool new project` - a project page and its tracker project
|
||||
<!-- /wikitool:reference-commands -->
|
||||
@@ -0,0 +1,160 @@
|
||||
# Git
|
||||
|
||||
Pulling in the remote and publishing a change - the only way a change leaves this checkout.
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
|
||||
- [`sync`](#sync)
|
||||
- [`publish`](#publish)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
<!-- wikitool:reference-commands -->
|
||||
## `sync`
|
||||
|
||||
Fetch `<remote>/<branch>` and bring the local branch up to date with it.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool sync [--remote origin] [--branch main] [--confirm-rebase TOKEN]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
--remote <str> Git remote to reconcile against [default: origin]
|
||||
--branch <str> Branch to reconcile [default: main]
|
||||
--confirm-rebase TOKEN Clear the Rebase-Review Gate for the exact upstream
|
||||
state and file overlap this token was issued for. The
|
||||
token comes from the gate's own refusal output.
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool sync
|
||||
tools/wikitool sync --confirm-rebase <token> # re-run after exit 42, once the user approved
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Fetches `<remote>/<branch>`, then: fast-forwards when only the remote moved; rebases the local commits on top when both sides moved but touched disjoint files; exits 42 (Rebase-Review Gate) when both sides touched the same file.
|
||||
- Files `wikitool` generates (`kb/index.md`, `kb/log.md`, `kb/provenance.md`, every `kb/**/INDEX.md`) never count as an overlap and never reach the gate. Where they are what stops git - uncommitted ones in the way of a fast-forward or rebase, or a rebase conflict in them - they are merged mechanically: `kb/log.md` becomes the incoming log plus the entries only this side appended, and the catalog and `kb/provenance.md` are regenerated from the merged pages and left as an uncommitted change. A reconcile git can do on its own is unchanged and regenerates nothing.
|
||||
- Uncommitted changes that no incoming commit touches do not stop a rebase: they are carried through it (`git rebase --autostash`) and come back unchanged.
|
||||
- Before anything is set aside, the working tree is stored as a commit under `refs/wikitool/reconcile-backup`. A successful reconcile drops it; a failed one restores the working tree and names the ref.
|
||||
- A refused call performs no rebase attempt and leaves the branch where it was.
|
||||
- The `--confirm-rebase` token covers the exact upstream state and the set of files touched on both sides, generated files excluded; either one moving makes it stale.
|
||||
- Makes no commit, no push, and no forced operation of any kind.
|
||||
- Three messages for a fetch that fails: no remote of that name, a remote that answers but has no such branch yet (a new, empty repository), and a remote that cannot be reached. All three exit 0 here; `publish` stops on the first and the last.
|
||||
- Run it once at the start of a writing session.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: yes
|
||||
- Idempotent: yes
|
||||
- Atomic: No - fetch, then at most one merge/rebase attempt, aborted cleanly on failure with the working tree restored; a success that merged generated files then rewrites the catalog
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: yes
|
||||
- Gates: rebase-review - see [exit codes and gates](../exit-codes.md)
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `0` - No remote configured - reported and skipped, not a failure
|
||||
- `0` - The remote cannot be reached - reported and skipped, not a failure
|
||||
- `1` - The automatic rebase hit a real conflict outside the generated files (git failed), or `kb/log.md` was edited rather than appended to; it is aborted cleanly and the working tree restored
|
||||
- `1` - An uncommitted change - or an untracked file - sits on a path the incoming commits also change; nothing was changed
|
||||
- `42` - Rebase-Review Gate: `<remote>/<branch>` moved and both sides changed the same file, generated files aside; the output lists the upstream commits, the overlapping files and their diff
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool publish` - runs the same reconcile before it commits and pushes
|
||||
- `instructions/session-setup.md` - where a session runs `sync`
|
||||
- `instructions/gates.md` - the gate procedure
|
||||
|
||||
## `publish`
|
||||
|
||||
Reconcile with `<remote>/<branch>`, then stage all changes, commit, and push.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool publish --message "<op>: <desc>" [--no-push] [--confirm TOKEN] [--confirm-rebase TOKEN] [--threshold N] [--remote origin] [--branch main] [--path P ...]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
--message <str> Commit message summary, e.g. 'ingest: docker-
|
||||
cheatsheet' [required]
|
||||
--push / --no-push [default: push]
|
||||
--confirm TOKEN Clear the Mass-Update Gate for the exact changeset
|
||||
this token was issued for. The token comes from the
|
||||
gate's own refusal output - run without it first to
|
||||
see the file list, show that output to the user, and
|
||||
only pass this once they have approved it.
|
||||
--confirm-rebase TOKEN Clear the Rebase-Review Gate raised by the pre-push
|
||||
reconcile step. The token comes from that gate's own
|
||||
refusal output - see `tools/wikitool sync`.
|
||||
--threshold <int> Mass-Update Gate threshold: number of counted files
|
||||
at/above which clearance is required. [default: 10]
|
||||
--remote <str> Git remote to push to [default: origin]
|
||||
--branch <str> Branch to push [default: main]
|
||||
--path <str> Limit this publish to a path (repeatable). Lets a
|
||||
large change be committed in reviewable batches
|
||||
instead of one opaque commit - the Mass-Update Gate
|
||||
then counts only the files in scope.
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool publish --message "ingest: docker-cheatsheet"
|
||||
tools/wikitool publish --confirm <token> --message "ingest: docker-cheatsheet" # re-run after a Mass-Update exit 42, once the user approved
|
||||
tools/wikitool publish --confirm-rebase <token> --message "ingest: docker-cheatsheet" # re-run after a rebase-review exit 42, once the user approved
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Order: branch check and Publish-Remote Gate, then the reconcile with `<remote>/<branch>`, then the Mass-Update Gate, then `git add -A`, commit and push. `--no-push` skips all but the Mass-Update Gate and the commit.
|
||||
- Without `--no-push`, a remote that is not configured or cannot be reached ends the call with exit 1 at the reconcile - before the gate, `git add` and the commit, and also on a clean tree. Nothing is committed, the index and the working tree are unchanged, and the message names `--no-push` as the way to a local commit. The next `publish` that reaches the remote pushes that commit along with whatever is new. A remote that answers but has no `<branch>` yet (a new, empty repository) is not this case: the first publish of an instance commits and pushes as before.
|
||||
- Reconcile: fetches `<remote>/<branch>`, fast-forwards when only the remote moved, rebases the local commits on top when both sides moved but touched disjoint files, and exits 42 (Rebase-Review Gate) when both sides touched the same file. A refused reconcile performs no rebase attempt. The `--confirm-rebase` token covers the exact upstream state and the set of files touched on both sides.
|
||||
- Generated files are never an overlap: where they are all that stops the reconcile, it merges them as `sync` does (both sides' log entries kept, the catalog and `kb/provenance.md` regenerated) and carries uncommitted changes no incoming commit touches through the rebase. The proactive reconcile runs before staging, so this publish commits the regenerated files - with `--path` too, see there; on the retry after a rejected push they get a commit of their own before the second push.
|
||||
- The push target must be the checked-out branch; this is checked before anything is staged. The unborn branch of a fresh `git init -b main` counts as checked out, so the first publish of a new instance works; a real detached HEAD is refused.
|
||||
- With nothing new to stage, a local commit the remote lacks is still pushed: one left behind by an earlier publish whose push failed, or every commit when the remote answers but does not have the branch yet (a new, empty remote repository).
|
||||
- A rejected push that finds the remote unreachable on its one retry reports the original push error.
|
||||
- A rejected push gets exactly one more reconcile-and-push; never more than one.
|
||||
- Mass-Update Gate: counts the files that would be committed, refuses with exit 42 at `--threshold` (default 10) or more, and prints a review report - a scale line (file count, total lines added/removed, status breakdown), attention notes where they apply (deletions by name, control-plane and harness-config touches, published pages, the largest single change, binaries), and every counted path grouped by area with its status and churn. The list is what the commit will hold: it is computed from a scratch copy of the index after `git add -A`, so a path that is staged as deleted and back in the working tree is not counted twice, and a rename counts as its old path deleted plus its new path added. The real index and the working tree are not touched, so a refused publish leaves both byte-identical.
|
||||
- Never counted and never shown for approval, but committed like everything else: anything under `work/`, and the files `wikitool` generates itself (`kb/index.md`, `kb/log.md`, `kb/provenance.md`, every `INDEX.md` under `kb/`). The refusal line accounts for both, by reason.
|
||||
- The `--confirm` token covers each counted path, the blob id of its contents and the publish target: a different file list or edited contents need a new clearance.
|
||||
- Publish-Remote Gate: when the checkout carries `.wikitool-remotes.json` and the push URL of `--remote` is not listed in it, exits 42 before the reconcile fetches anything. The URL is read with `git remote get-url --push`, so a repointed remote does not pass on its name. An absent file means unrestricted; a malformed one is an error, not permission.
|
||||
- `--path` (repeatable) scopes the whole operation - gate count, staging and commit - to that subtree, with one exception: uncommitted generated files outside `--path` (`kb/index.md`, `kb/log.md`, `kb/provenance.md`, every `INDEX.md` under `kb/`) are staged and committed too, and a line names them - whether a reconcile or the change itself wrote them. They stay out of the gate count, and the `--confirm` token and the re-run line cover the `--path` given. Every other file outside `--path` is left as it is.
|
||||
- Commit message: `--message`, then a `Files changed:` paragraph listing every committed path. When `--message` ends in a paragraph git reads as a trailer block (`Co-Authored-By:` and the like), that block stays the last paragraph and the list goes in front of it, since git reads trailers from the last paragraph only. `git interpret-trailers` decides whether there is such a block, and the list moves only when git reads the same trailers from the result; otherwise it is appended at the end. `--message` is not part of any gate token.
|
||||
- After a successful commit or push whose changed files include `tools/`, `types/`, `instructions/`, `AGENTS.md` or a path ending in `CONTRACT.md`, prints one reminder line: the phase past this point (an issue-body rewrite, `docs/` staleness, a changelog entry's accuracy) is not covered by `docs verify`, `instructions verify` or `pytest`. It is not a gate: no exit code change, nothing to clear, and silent for an ordinary content publish.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: yes
|
||||
- Idempotent: no
|
||||
- Atomic: No - sequential git operations. Every gate runs before staging, except on the one retry of a rejected push, where the Rebase-Review Gate can exit 42 after the commit: nothing is pushed, and the re-run with `--confirm-rebase` pushes that commit
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: yes
|
||||
- Gates: mass-update, publish-remote, rebase-review - see [exit codes and gates](../exit-codes.md)
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - git failed - `git add`, `git commit`, `git push`, or the reconcile's automatic rebase (a conflict outside the generated files, an edited `kb/log.md`, or an uncommitted change on a path the incoming commits change)
|
||||
- `1` - The push target (`--branch`) is not the checked-out branch, or HEAD is detached; the unborn branch of a fresh `git init` is not this case
|
||||
- `1` - No `--no-push`, and the remote is not configured or cannot be reached; nothing was committed
|
||||
- `1` - `--yes`/`-y` was passed - the flag does not exist and fails with an explicit error
|
||||
- `1` - `.wikitool-remotes.json` is unreadable or has no usable `allowed_push_urls` list
|
||||
- `42` - Mass-Update Gate: `--threshold` (default 10) or more counted files would be committed, or the `--confirm` token does not match this changeset
|
||||
- `42` - Rebase-Review Gate: `<remote>/<branch>` moved and both sides changed the same file; the output lists the upstream commits, the overlapping files and their diff
|
||||
- `42` - Publish-Remote Gate: `.wikitool-remotes.json` exists and the push URL of `--remote` is not listed in it, or `--remote` resolves to no push URL
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool sync` - the same reconcile on its own, without committing or pushing
|
||||
- `instructions/gates.md` - the gate procedure
|
||||
- `instructions/setup-instance.md` - the first publish of a new instance
|
||||
<!-- /wikitool:reference-commands -->
|
||||
@@ -0,0 +1,77 @@
|
||||
# Guideline export
|
||||
|
||||
Writing the generated `GUIDELINES.md` into the repositories the wiki has captured.
|
||||
|
||||
<!-- wikitool:reference-commands -->
|
||||
## `export guidelines`
|
||||
|
||||
Render the guideline pages into one generated `GUIDELINES.md`, or push it into the captured repositories that opted in (**Guideline Push Gate**).
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool export guidelines [--field <predicate> ...] [--kind/--subtype/--collection/--tag <v>]` - Prints the file to stdout; no network.
|
||||
- `wikitool export guidelines <predicates> --push [--bundle <raw-bundle> ...] [--confirm TOKEN]` - Writes it into the captured repositories, behind the gate.
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
-f, --field <str> Frontmatter predicate, repeatable (AND) - the same forms
|
||||
as `search --field`.
|
||||
--kind <str> Shorthand for --field kind=<value>.
|
||||
--subtype <str> Shorthand for --field subtype=<value>.
|
||||
--collection <str> Shorthand for --field collection=<value>.
|
||||
--tag <str> Shorthand for --field tags=<value>.
|
||||
--push Write the file into the captured repositories (Guideline
|
||||
Push Gate).
|
||||
--bundle <path> Only the repository of this captured bundle under raw/;
|
||||
repeatable.
|
||||
--confirm <str> The token from a prior exit 42, once a human has approved
|
||||
it.
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool export guidelines --tag guideline
|
||||
tools/wikitool export guidelines --tag guideline --push
|
||||
tools/wikitool export guidelines --confirm <token> --push --field tags=guideline # re-run after exit 42, once the user approved every diff
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Selects pages with `search`'s own predicates and no text: `--field` (repeatable, AND), `--kind`, `--subtype`, `--collection`, `--tag`. Which filter selects this instance's guidelines is written down in `kb/CONVENTIONS.md`.
|
||||
- Renders mechanically: line 1 is `<!-- wikitool:export kind=guidelines instance=<origin URL without userinfo, or local> commit=<newest commit touching a selected page> - generated, do not edit by hand -->`; then each page sorted by title, as `# <title>` and its text without frontmatter, the generated links and footnote blocks, citation markers (`[^id]`, legacy `^[[...]]`), with `[[Title|Text]]` as `Text` and `[[Title]]` as `Title`. A leading H1 of the page is replaced by the title heading. Code is never rewritten. LF endings, no timestamp: the same pages give the same bytes.
|
||||
- Refuses before printing or pushing anything: no predicate, no match, an unknown field, any page under `kb/` with unreadable frontmatter, uncommitted changes in `kb/` or `types/` (untracked files included), or no git checkout.
|
||||
- `--push` targets every captured repository - each `(repo, ref)` from a `_capture.json` under `raw/`, `--bundle` narrowing it. A target is skipped with its reason when its ref rule is a tag pattern, it cannot be reached or has no such branch, its root has no `GUIDELINES.md` (not opted in), its `GUIDELINES.md` has no export header (hand-written), or the header names another `instance=`. A header line alone is the opt-in.
|
||||
- Guideline Push Gate: `--push` without a matching `--confirm` fetches and builds everything, pushes nothing, and exits 42 with each target's status (`new`, `changed`, `unchanged`, `skipped`), the diff for every target it would write, and the re-run line. The token digests URL, branch, old tip and new blob of every target to write.
|
||||
- With a matching token, each target gets exactly one commit on the fetched tip changing only `GUIDELINES.md`, authored as the instance checkout's `user.name`/`user.email`, pushed without force; one line per target: `unchanged`, `written <old> -> <new>`, `skipped (<reason>)`, `rejected (<reason>)` or `failed (<reason>)`.
|
||||
- Nothing to write (every target unchanged or skipped) ends with exit 0 and no gate. A second run after a successful one finds every target unchanged.
|
||||
- The target repository's own `AGENTS.md` points at the file (Claude Code: `@GUIDELINES.md`); that belongs to the repository, not to this command.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: yes
|
||||
- Idempotent: yes
|
||||
- Atomic: Per target - one commit pushed without force, or nothing; targets already written stay written when a later one fails
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: yes
|
||||
- Gates: guideline-push - see [exit codes and gates](../exit-codes.md)
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `0` - A target was skipped - tag rule, not reachable, not opted in, hand-written file, another instance's file
|
||||
- `1` - No predicate, no page matches, a malformed predicate or an unknown field
|
||||
- `1` - A page under `kb/` has unreadable frontmatter, `kb/` or `types/` has uncommitted changes, or the instance is not a git checkout
|
||||
- `1` - `--push` with no `user.name` or `user.email` in the instance checkout's git configuration - checked before any target is fetched
|
||||
- `1` - `--bundle` names no captured bundle under `raw/`, or `--bundle`/`--confirm` was given without `--push`
|
||||
- `1` - A target rejected the push (its branch moved since the fetch) or a git call failed - the other targets were still served
|
||||
- `42` - Guideline Push Gate: `--push` without `--confirm`, or with a token that does not match the current set of targets
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool search` - the same predicates, to preview the selection
|
||||
- `wikitool raw capture` - how a repository becomes a target
|
||||
- `instructions/gates.md` - the gate procedure
|
||||
<!-- /wikitool:reference-commands -->
|
||||
@@ -0,0 +1,67 @@
|
||||
# Instance health
|
||||
|
||||
A single check of whether this instance is set up and working.
|
||||
|
||||
<!-- wikitool:reference-commands -->
|
||||
## `doctor`
|
||||
|
||||
Check that this instance is correctly configured.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool doctor [--json]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
--json Print the checks as JSON
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool doctor
|
||||
tools/wikitool doctor --json
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Checks dependencies (Python, ripgrep), author resolution, stack version, git identity/branch/remote, published skills, the kb/raw/reports/work/instructions structure, and generated files.
|
||||
- Import filters (`import-filter-<name>`): one line for each `raw derive` format that reads through a package (PDF, Outlook `.msg`) - a `WARN` naming the package when it is missing, since that costs only its own format; fixed by running `tools/preflight.sh` again.
|
||||
- Tool paths (`tool-paths`): `.wikitool-tools.json` written by a preflight that finished, every tool `tools/prerequisites.txt` names for this platform recorded, and every recorded path still there - a `FAIL` otherwise, fixed by running `tools/preflight.sh` again (PowerShell 7: `tools/preflight.ps1`).
|
||||
- Install folder (`install-dir`): on Windows with long paths off, a `FAIL` when the folder holding `tools/` is longer than `tools/prerequisites.txt` allows (95 characters) - the limit the preflight enforces before it sets anything up.
|
||||
- PowerShell (`execution-policy`, `script-marks`; Windows only, `OK` elsewhere): a `FAIL` when the effective execution policy is `Restricted` or `AllSigned` - the line then says whether a group policy sets it, which only whoever administers the computer can change - and a `FAIL` when a script under `tools/` carries a Mark of the Web from the internet zone, which a browser download unpacked in Explorer leaves behind and `Invoke-WebRequest` plus `tar` do not. `tools/preflight.ps1` checks the same two things before it stops.
|
||||
- Personalization: `USER.md`/`SOUL.md` present **and** filled - a file still carrying the template's sentinel is a `FAIL`.
|
||||
- KB conventions: `kb/CONVENTIONS.md` present, unsentinelled, and naming all three tool-owned section headings - a `FAIL` on any of the three.
|
||||
- Environment note: `ENVIRONMENT.md` is optional, so absent is `OK`; a still-templated one is a `WARN`.
|
||||
- MCP `submit` tool: whether `.wikitool-upload.json` is present, absent or malformed, its limits, and how many submissions wait in `mcp-upload/`. Absent is `OK` and means the write path does not exist at all; malformed is a `FAIL`.
|
||||
- Task tracker: `.wikitool-tasks.json` present, absent or malformed - absent is `OK` (no tracker configured), malformed is a `FAIL`. When `WIKITOOL_TASKS_CONFIG` is set, that file is read instead and the finding names it; a set variable that names no file is a `FAIL`, never `OK`.
|
||||
- For a configured `superproductivity` provider, the configured `access` path's own state: `access: "api"` reports whether its local REST API answers `GET /health` with a ready renderer right now, `access: "snapshot"` whether a backup file is ready. The other access path is never attempted, and neither state is ever a `FAIL`.
|
||||
- For a configured `caldav` provider, whether the server is reachable and Basic auth succeeds - never a `FAIL`; only a broken config block is.
|
||||
- Session id source: `OK` for `WIKITOOL_SESSION_ID` or a registered harness variable, `WARN` only for the bare parent-pid fallback.
|
||||
- Telemetry: on or off and why - installation-form default, `.wikitool-telemetry.json`, or `WIKI_TRACE` - and the current session's count and byte total against both caps; never a `FAIL`. A `WARN` when `.wikitool-telemetry.json` is not read as written - unreadable JSON, not an object, `enabled` not a JSON boolean, a cap not a positive integer, or an unknown key - naming the field; telemetry itself drops or reinterprets such a value without a word, so this is where it shows.
|
||||
- Exits 1 only on a `FAIL`; a missing remote, session id or `VERSION` is a `WARN`, not a fault.
|
||||
- Read-only and exempt from the Iteration Budget Gate.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: no
|
||||
- Idempotent: yes
|
||||
- Atomic: Read-only
|
||||
- Counts toward the session call budget: no
|
||||
- Can reach the network: yes
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - At least one check reported `FAIL` (a `WARN`, e.g. no remote or no `WIKITOOL_SESSION_ID`, does not exit 1)
|
||||
|
||||
**See also**
|
||||
|
||||
- `instructions/setup-instance.md` - the setup steps most findings point back to
|
||||
- `instructions/preflight.md` - what `tool-paths` and `install-dir` point back to
|
||||
- `docs/reference/configuration.md` - the per-checkout configuration files
|
||||
- `docs/explanation/telemetry-and-evaluation.md` - telemetry state and caps
|
||||
- `instructions/session-setup.md` - setting `WIKITOOL_SESSION_ID`
|
||||
<!-- /wikitool:reference-commands -->
|
||||
@@ -0,0 +1,376 @@
|
||||
# Links and citations
|
||||
|
||||
Declaring the relationships between pages and the citations that tie a claim to its source, with the generated regions that hold both.
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
|
||||
- [`xref add`](#xref-add)
|
||||
- [`xref remove`](#xref-remove)
|
||||
- [`xref link-source`](#xref-link-source)
|
||||
- [`links show`](#links-show)
|
||||
- [`cite id`](#cite-id)
|
||||
- [`cite add`](#cite-add)
|
||||
- [`cite sync`](#cite-sync)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
<!-- wikitool:reference-commands -->
|
||||
## `xref add`
|
||||
|
||||
Declare that A <rel> B.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool xref add --a "<A>" --b "<B>" --rel <label> [--dry-run]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
--a <str> Exact title of the page that asserts the edge [required]
|
||||
--b <str> Exact title of the page it points at [required]
|
||||
--rel <str> Label from instructions/link-taxonomy.md, e.g. depends-on
|
||||
[required]
|
||||
--dry-run Preview the change instead of writing
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool xref add --a "Gitea Actions" --b "Act Runner" --rel depends-on
|
||||
tools/wikitool xref add --a "Gitea Actions" --b "Act Runner" --rel depends-on --dry-run
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Declares **one** edge, `A <label> B`: written into A's `related:` as `- <label>: B` and rendered into A's generated links region.
|
||||
- B is not touched and does not point back - its inbound view is rendered from the graph.
|
||||
- Idempotent; re-running with a different label *relabels* rather than appending, since one page asserts one thing about another.
|
||||
- Refuses before writing when A's type does not declare `related:` - a source page declares `entities:`/`concepts:` instead, and the refusal names them and points at `xref link-source`.
|
||||
- Refuses before writing when `<label>` is not authorised by the source collection's `outbound:` block for the target's collection; the refusal lists the authorised set and points at `instructions/link-taxonomy.md`.
|
||||
- `--dry-run` reports the edge without writing.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: yes
|
||||
- Idempotent: yes
|
||||
- Atomic: Yes - a single write to A; B is never touched, and every refusal happens before it
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - Page A or B not found
|
||||
- `1` - A's type declares no `related:` field
|
||||
- `1` - `<label>` is not authorised for this collection pair, or the source collection authorises no labels into the target's collection at all
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool xref remove` - removes a reference
|
||||
- `wikitool links show` - the edges out of and into a page
|
||||
- `instructions/link-taxonomy.md` - the labels and what each asserts
|
||||
|
||||
## `xref remove`
|
||||
|
||||
Remove a cross-reference: the inverse of `xref add`.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool xref remove --a "<A>" --b "<B>" [--dry-run]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
--a <str> Exact title of page A (must exist) [required]
|
||||
--b <str> Title to unlink from A; need not still exist as a page [required]
|
||||
--dry-run Preview changes instead of writing
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool xref remove --a "Gitea Actions" --b "act_runner"
|
||||
tools/wikitool xref remove --a "Gitea Actions" --b "Act Runner" --dry-run
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Clears the reference in **both** directions: the cleanup command for a deleted or hand-renamed page rather than the strict inverse of a one-directional `xref add`.
|
||||
- Clears `<B>` from every page-ref frontmatter field `<A>`'s type declares (`related:`, `sources:`, `entities:`, `concepts:`) plus the matching bullets.
|
||||
- Also sweeps a page-ref field the type does *not* declare but some other type does, and drops that key once empty.
|
||||
- `--b` need not still exist as a page, so this clears a reference left by a hand-deleted or hand-renamed page without hand-editing frontmatter.
|
||||
- Idempotent: removing an absent link is a no-op. `--dry-run` reports without writing.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: yes
|
||||
- Idempotent: yes
|
||||
- Atomic: No - writes A then B, both idempotent
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - Page A not found (B is allowed not to exist)
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool xref add` - declares an edge
|
||||
- `wikitool rm` - deletes a page and de-links it
|
||||
|
||||
## `xref link-source`
|
||||
|
||||
Batch-link a source page to every entity/concept it mentions.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool xref link-source --source "Source - X" --entities A,B,C [--dry-run]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
--source <str> Exact source page title, e.g. 'Source - Docker Cheatsheet'
|
||||
[required]
|
||||
--entities <str> Comma-separated entity/concept titles the source mentions
|
||||
[required]
|
||||
--dry-run Preview which pages would be linked instead of writing
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool xref link-source --source "Source - Docker Cheatsheet" --entities Docker,Podman --dry-run
|
||||
tools/wikitool xref link-source --source "Source - Docker Cheatsheet" --entities Docker,Podman
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Each target gets the source page in its `sources:`, and the source page records each target in its own `entities:` or `concepts:`. No body bullet is written on either side - `sources:` *is* the record.
|
||||
- Which field a target lands in follows its collection (`kb/entities/` -> `entities:`), so a new collection needs no code change.
|
||||
- A target whose collection matches no reference field the source type declares is linked one way and named in the output.
|
||||
- A target that does not exist is skipped and named; the others are still linked, and the run then exits 1.
|
||||
- A target that is itself a source page is refused, named with the `cite add` command to use instead, and handled like a missing page: nothing is written for it, the others are still linked, and the run exits 1. `link-source` means "A mentions X" and writes `X.sources += A`, which for X = another source would claim the citation in the wrong direction; a citation between two sources has a direction only `cite add` records.
|
||||
- Idempotent in both directions. `--dry-run` reports what would be linked.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: yes
|
||||
- Idempotent: yes
|
||||
- Atomic: No - one write per entity plus one for the source page, idempotent per page
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - Source page not found
|
||||
- `1` - A page in `--entities` does not exist, or a target could not be written; the others were linked
|
||||
- `1` - A page in `--entities` is itself a source page; the others were linked
|
||||
- `1` - The source page itself could not be written after its targets were
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool sources trace` - who a source is already linked to
|
||||
- `wikitool xref add` - one labelled edge between two pages
|
||||
- `wiki-ingest` skill - where a source is linked after it is written
|
||||
|
||||
## `links show`
|
||||
|
||||
Show the edges out of and into a page.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool links show --page "<Title>" [--json]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
--page <str> Exact page title [required]
|
||||
--json Print the edges as JSON
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool links show --page "Act Runner"
|
||||
tools/wikitool links show --page "Act Runner" --json
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Shows the declared graph around one page in both directions: the edges it asserts (from its own `related:`, with labels) and the edges other pages assert about it.
|
||||
- The inbound half is computed across the corpus on every call, never stored, so it is complete.
|
||||
- `--json` prints both halves as JSON.
|
||||
- Read-only; exempt from the Iteration Budget Gate.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: no
|
||||
- Idempotent: yes
|
||||
- Atomic: Read-only
|
||||
- Counts toward the session call budget: no
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - Page not found
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool xref add` / `wikitool xref remove` - change the outbound edges
|
||||
- `wikitool search` - finds the exact title
|
||||
|
||||
## `cite id`
|
||||
|
||||
Print the deterministic footnote id `cite add` would use for this (title, file) pair.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool cite id --title "Source - X" [--file <qualifier>]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
--title <str> Source page title, e.g. 'Source - Docker Cheatsheet'
|
||||
[required]
|
||||
--file <str> Qualifier for a multi-file source, e.g. 'storage-model.md'
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool cite id --title "Source - Docker Cheatsheet"
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Prints the footnote id `cite add` would use for this (title, file) pair.
|
||||
- A preview only: it does not check that the id is free on any given page.
|
||||
- Never fails; safe to retry freely. Read-only and exempt from the Iteration Budget Gate.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: no
|
||||
- Idempotent: yes
|
||||
- Atomic: Read-only
|
||||
- Counts toward the session call budget: no
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool cite add` - writes the definition
|
||||
|
||||
## `cite add`
|
||||
|
||||
Upsert a `[^cite-id]: [[Source - X]]` definition in a page's footnotes region.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool cite add --page "<Title>" --source "Source - X" [--file <qualifier>] [--dry-run]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
--page <str> Exact title of the page to add a citation on [required]
|
||||
--source <str> Exact title of the source page being cited, e.g. 'Source - X'
|
||||
[required]
|
||||
--file <str> Qualifier for a multi-file source, e.g. 'storage-model.md'
|
||||
--dry-run Preview instead of writing
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool cite add --page "Docker" --source "Source - Docker Cheatsheet"
|
||||
tools/wikitool cite add --page "Docker" --source "Source - Docker Cheatsheet" --file part-2
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Upserts a `[^cite-id]: [[Source - X]]` definition in the page's generated footnotes region, creating the region between `<!-- wikitool:footnotes -->` markers if absent.
|
||||
- Reuses the id when the page already cites this exact source/file pair, so a repeat changes nothing.
|
||||
- Adds `Source - X` to the page's frontmatter `sources:` - except on `Source - X` itself: a source page citing one of its own raw files (`--file`) gets the definition and leaves `sources:` untouched, since a page never lists its own title there.
|
||||
- A source page may cite another source page this way - `cite add` is the command that records a citation between two sources, in the direction it was made.
|
||||
- Prints the `[^cite-id]` marker; pasting it into the prose is a manual, editorial step.
|
||||
- `--dry-run` reports without writing.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: yes
|
||||
- Idempotent: yes
|
||||
- Atomic: Yes - single file write
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - The page is not found
|
||||
- `1` - The source page is not found - citing it would be a dangling reference
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool cite sync` - prunes and re-orders the region
|
||||
- `wikitool cite id` - previews an id
|
||||
|
||||
## `cite sync`
|
||||
|
||||
Reconcile each page's footnotes region against its actual `[^id]` references.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool cite sync [--page "<Title>" | --all] [--dry-run]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
--page <str> Sync just this page
|
||||
--all Sync every page under kb/
|
||||
--dry-run Report what would change instead of writing
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool cite sync --page "Docker"
|
||||
tools/wikitool cite sync --all --dry-run
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Prunes definitions nothing references any more, re-renders the region in first-reference order, and reports every `[^id]` reference left with no definition.
|
||||
- An undefined-reference report is not a failure: fix the reference, or run `cite add`, and re-run.
|
||||
- A page still carrying the pre-4.0.0 undelimited footnote block is converted to a marked region in the same pass.
|
||||
- Each page's re-render is idempotent; safe to retry freely. `--dry-run` reports without writing.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: yes
|
||||
- Idempotent: yes
|
||||
- Atomic: No - one write per page, each idempotent
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - Neither or both of `--page`/`--all` given, or the page is not found
|
||||
- `1` - A page write failed partway
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool cite add` - writes a definition
|
||||
<!-- /wikitool:reference-commands -->
|
||||
@@ -0,0 +1,531 @@
|
||||
# Pages
|
||||
|
||||
Creating, updating, renaming, moving and removing wiki pages, and filing and closing items in the task tracker.
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
|
||||
- [`new`](#new)
|
||||
- [`task new`](#task-new)
|
||||
- [`task list`](#task-list)
|
||||
- [`task close`](#task-close)
|
||||
- [`touch`](#touch)
|
||||
- [`rename`](#rename)
|
||||
- [`rm`](#rm)
|
||||
- [`move`](#move)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
<!-- wikitool:reference-commands -->
|
||||
## `new`
|
||||
|
||||
Scaffold a new wiki page of any type.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool new <type-name> --name "<Name>" [--type <path>] [--set field=value ...]` - Any type; its type-spec decides fields, directory, title prefix and template.
|
||||
- `wikitool new entity --name "<Name>" --set entity_type=<t> [--set tags=a,b] [--set related=X,Y] [--set sources="Source - Z"] [--set provenance=sourced|general|mixed]` - Writes `kb/entities/<subdir>/<Name>.md` - `<subdir>` from `entity_type` via the type-spec's `layout:`
|
||||
- `wikitool new concept --name "<Name>" --set concept_type=<t> ...` - Writes `kb/concepts/<subdir>/<Name>.md` - `<subdir>` from `concept_type` via the type-spec's `layout:`
|
||||
- `wikitool new source --name "<Name>" --set source_type=<t> --set raw_files=raw/notes/x.md,raw/notes/y.md --set fidelity=<f> --set authority=<a> [--set source_url=<URL>] [--set entities=A,B] [--set concepts=C,D]` - Writes `kb/sources/<subdir>/Source - <Name>.md` (prefix added automatically; `<subdir>` from `source_type` via the type-spec's `layout:`) with a `raw_files:` list; rejects paths that don't exist
|
||||
- `wikitool new comparison --name "X vs Y" --set entities=X,Y` - Writes `kb/comparisons/X vs Y.md`
|
||||
- `wikitool new project --name "<Name>" --set responsibility=<bereich> [--resume]` - Writes `kb/gtd/<bereich>/<Name>.md` and, with a task tracker configured, a same-named tracker project
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
Arguments:
|
||||
type_name Type name, e.g. entity|concept|source|comparison (see `wikitool
|
||||
types list`) [required]
|
||||
|
||||
Options:
|
||||
--name <str> Page title (a type's title_prefix is added automatically)
|
||||
[required]
|
||||
--type <str> Override the type-spec path (defaults to the one named by
|
||||
TYPE_NAME)
|
||||
--set <str> Frontmatter field, repeatable: --set entity_type=tool --set
|
||||
tags=a,b. Array values split on commas (escape a literal one as
|
||||
\,); repeating --set for an array field appends instead of
|
||||
replacing
|
||||
--resume `project` only: confirm a human has completed the manual tracker
|
||||
step an earlier HumanInterventionRequired refusal asked for, so
|
||||
this run continues to page creation instead of refusing the now-
|
||||
existing tracker project as a collision.
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool new entity --name "Docker" --set entity_type=tool --set tags=containers
|
||||
tools/wikitool new source --name "Docker Cheatsheet" --set raw_files=raw/2026/09/docker-cheatsheet.md --set fidelity=verbatim --set authority=reporting
|
||||
tools/wikitool new project --name "Homelab migration" --set responsibility=infrastruktur --resume # re-run after exit 42, once the user created the tracker project
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- A title becomes a file name, so it must be valid and unique on Windows and macOS as well as Linux, whichever platform runs the command and whichever root the type writes to. The rule is `kb/CONTRACT.md` § Titles are identifiers; it is checked on the full title, after `title_prefix`.
|
||||
- The target's path below the instance root may be at most 160 characters, counted in UTF-16 code units the way Windows counts MAX_PATH, so a Windows checkout without long paths keeps working. A longer one is refused, for every root, naming the length and how much shorter it has to get.
|
||||
- `new` never overwrites: a file already at the target - or one a case-insensitive file system would treat as the same file - is refused for every root, `instructions/` included.
|
||||
- The type-spec drives everything: fields, directory (`base_dir`/`layout`), title prefix, and template. `types list`/`types describe` show what a type requires.
|
||||
- The body skeleton is `types/<type>.<value>.md` when that file exists, `<value>` being the page's subtype field value (e.g. `types/entity.person.md` for `entity_type=person`) - it replaces the type-spec's `## Template` block whole. Without such a file, and for a type with no subtype field, the `## Template` block is used as before.
|
||||
- A schema `default:` is materialized only for a field the schema also lists in `required:`.
|
||||
- `--set` is repeatable, and comma-separated values fill array fields. An element that itself contains a comma is written `\,`, or passed as its own repeated `--set` for that field - repeating an array field appends.
|
||||
- A capture field the type-spec requires (a source's `fidelity`/`authority`) must be passed with `--set`; `new` never guesses it and refuses `unknown` for it.
|
||||
- Produces structurally correct frontmatter and a body skeleton only - the prose (Description, Summary, judgment calls about relationships) is written afterwards.
|
||||
- `new project`: with `.wikitool-tasks.json` configuring a task tracker, also makes sure a same-named tracker project exists - one name, one identity. No tracker configured is a legitimate, explicitly announced state: page only.
|
||||
- `new project`: tracker before page. The tracker side is settled first, so a failure past that point leaves a tracker project with no page - a state `review`'s check 3 reports - never a page with no tracker project.
|
||||
- `new project`: a name already taken, case-insensitively, in `kb/` or the tracker is refused outright, naming where it was found, and creates nothing. For `caldav` the tracker check covers every list in the account, not only the ones counted as projects.
|
||||
- `new project`: a provider whose configured access path has no write path (Super Productivity's `access: "snapshot"`) refuses entirely with exit 1, naming the `access: "api"` instance to use instead - neither the tracker project nor the page is created, and `--resume` behaves the same.
|
||||
- `new project`: a provider that could write but has no project-creation call of its own (Super Productivity's `access: "api"` - `GET /projects` exists, `POST /projects` does not) prints instructions for a human and exits 42, creating nothing. That is not one of the named gates, but the same exit code and the same handling. `caldav` never does this: `MKCALENDAR` creates the list, so a valid, non-colliding name always creates it.
|
||||
- `--resume` is how a later run tells the command a human has done what that message asked: it re-verifies through the tracker's read path before continuing to page creation, rather than trusting the claim, and exits 42 again if the tracker still does not have the project. `--resume` on any other type is refused.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: yes
|
||||
- Idempotent: no
|
||||
- Atomic: `new <type>`: Yes - single file write. `new project`: **No** for the tracker-configured case - a tracker-project write (or its human-clearance request) happens before the kb/ page write, so a failure between the two leaves a tracker project with no page (a state `review`'s check 3 already reports), never a page with no tracker project. Still a single file write when no tracker is configured
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - A page with this title already exists, the type is unknown, or a `--set` value is invalid
|
||||
- `1` - The title is not a valid file name (forbidden character, control character, reserved name such as `CON` or `Index`, trailing dot or space, empty), collides with another page by case or Unicode normalization, the target file already exists, or the target path is over the 160-character path budget
|
||||
- `1` - A `raw_files` path does not exist
|
||||
- `1` - A capture field the type-spec requires is missing, or set to `unknown`
|
||||
- `1` - `--resume` with a type other than `project`
|
||||
- `1` - new project: The name is already taken in the tracker (case-insensitively; for `caldav` against every list in the account)
|
||||
- `1` - new project: The configured access path has no write path (Super Productivity's `access: "snapshot"`)
|
||||
- `1` - new project: The page write failed after the tracker project was confirmed to exist
|
||||
- `42` - new project: The provider cannot create the project itself (Super Productivity's `access: "api"`); the output says what a human has to create
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool types describe <type>` - what a type requires and where it lands
|
||||
- `wikitool touch` - changes a page's own frontmatter afterwards
|
||||
- `wikitool task new` - a tracker item without a page
|
||||
- `docs/explanation/knowledge-and-commitment.md` - why a project is a page and a tracker project
|
||||
|
||||
## `task new`
|
||||
|
||||
Create one open item in the configured task tracker - never a kb/ page.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool task new --title "<Title>" (--project "<Name>" | --inbox) [--waiting [--follow-up-at YYYY-MM-DD]] [--notes "..."]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
--title <str> The item's title. Stored verbatim, never parsed.
|
||||
[required]
|
||||
--project <str> An existing tracker project's name (matched case-
|
||||
insensitively). This command never searches or guesses
|
||||
one and never creates one - use `wikitool new project`
|
||||
first if it does not exist yet. Exactly one of
|
||||
--project/--inbox is required.
|
||||
--inbox File into the tracker's own inbox instead of a project -
|
||||
the deliberately chosen exit when no project fits, never
|
||||
a stand-in for an omitted --project. An item filed here
|
||||
is invisible to `wikitool review`, since every check
|
||||
there is reached through a project name and the inbox
|
||||
has none.
|
||||
--waiting Tag the item WAITING - the review's check 2 reads this.
|
||||
--follow-up-at <str> YYYY-MM-DD. Only meaningful together with --waiting - it
|
||||
is never a due date and is refused without --waiting.
|
||||
--notes <str> A freetext backref, e.g. to the kb/ source page this
|
||||
item came from. Stored verbatim, never parsed.
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool task new --title "Renew the TLS certificate" --project "Homelab migration"
|
||||
tools/wikitool task new --title "Quote from the electrician" --project "Homelab migration" --waiting --follow-up-at 2026-10-05
|
||||
tools/wikitool task new --title "Read the qmd README" --inbox --notes "from [[Source - qmd - GitHub Repository]]"
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Creates one open item in the configured task tracker; never touches `kb/`.
|
||||
- Exactly one of `--project` or `--inbox` is required; an omitted `--project` refuses rather than silently falling into the inbox.
|
||||
- `--project` names an existing tracker project, matched case-insensitively - never created, and never searched or guessed.
|
||||
- `--inbox` files into the tracker's own inbox. An item filed there never appears in `review`, since every one of its checks reaches items through a project name.
|
||||
- `--waiting` sets the WAITING status `review`'s waiting-overdue check reads. `--follow-up-at` is refused without `--waiting` - it is never a due date on its own.
|
||||
- `--notes` carries a freetext backref (e.g. to the `kb/` source page the item came from), stored verbatim, never parsed.
|
||||
- The tracker configuration is read from `.wikitool-tasks.json`, or from the file `WIKITOOL_TASKS_CONFIG` names when that variable is set (so one checkout can be run against several trackers in turn). A set variable that names no file is an error, never "no tracker configured".
|
||||
- No `.wikitool-tasks.json` fails immediately with a "no tracker configured" message.
|
||||
- A provider whose configured access path has no write path (Super Productivity's `access: "snapshot"`) refuses entirely with exit 1, naming the `access: "api"` instance to use instead.
|
||||
- **Never exits 42.** A `--project` matching no tracker project, or `--waiting` against a provider that cannot represent it right now (Super Productivity: the `waiting` tag does not exist yet, and its API cannot create tags), are ordinary exit-1 refusals that create nothing.
|
||||
- Not idempotent: every successful run creates another item.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: yes
|
||||
- Idempotent: no
|
||||
- Atomic: Yes - a single API call, made only once every precondition (the project's own id, the WAITING tag's own id) is confirmed to exist, so a missing one never leaves a half-written item behind
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: yes
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - No `.wikitool-tasks.json` - no tracker configured
|
||||
- `1` - `WIKITOOL_TASKS_CONFIG` names a file that does not exist or is broken
|
||||
- `1` - Neither or both of `--project`/`--inbox`, or a `--follow-up-at` without `--waiting` or not `YYYY-MM-DD`
|
||||
- `1` - A `--project` name matching no tracker project
|
||||
- `1` - `--waiting` against a provider with no way to represent it right now (Super Productivity: the `waiting` tag does not exist)
|
||||
- `1` - A read-only access path (Super Productivity's `access: "snapshot"`)
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool task list` - a project's open items and their ids
|
||||
- `wikitool task close` - marks an item done
|
||||
- `wikitool new project` - a project page and its tracker project
|
||||
- `docs/explanation/knowledge-and-commitment.md` - why commitments live in the tracker, not in `kb/`
|
||||
|
||||
## `task list`
|
||||
|
||||
List a project's open items - id, title, and whether each carries the WAITING status.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool task list --project "<Name>"`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
--project <str> An existing tracker project's name (matched case-
|
||||
insensitively). [required]
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool task list --project "Homelab migration"
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Lists one tracker project's open items: id, title, and whether each carries the WAITING status.
|
||||
- The id source `task close` and `review`'s `waiting_overdue`/`someday_stale` findings need, without running `review` first.
|
||||
- Works on every access path a provider offers, read-only ones included.
|
||||
- A `--project` matching no tracker project prints "No open items": the tracker read does not distinguish an empty project from an unknown one.
|
||||
- Read-only.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: no
|
||||
- Idempotent: yes
|
||||
- Atomic: Yes - read-only, nothing to leave half-written
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: yes
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `0` - A `--project` matching no tracker project - prints "No open items", not an error
|
||||
- `1` - No `.wikitool-tasks.json` - no tracker configured
|
||||
- `1` - `WIKITOOL_TASKS_CONFIG` names a file that does not exist or is broken
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool task close` - closes an item by the id listed here
|
||||
- `wikitool review` - the findings that name items
|
||||
|
||||
## `task close`
|
||||
|
||||
Mark one tracker item done - never delete it.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool task close --id <item-id>`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
--id <str> The tracker's own item id, e.g. from `task list` or `wikitool
|
||||
review`'s waiting_overdue/someday_stale findings - never a title.
|
||||
[required]
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool task close --id <item-id>
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Marks one tracker item done. It never deletes or moves an item - the only closing write this stack makes.
|
||||
- `--id` is the provider's own item id, from `task list` or a `review` finding - never a title.
|
||||
- The tracker configuration is read from `.wikitool-tasks.json`, or from the file `WIKITOOL_TASKS_CONFIG` names when that variable is set (so one checkout can be run against several trackers in turn). A set variable that names no file is an error, never "no tracker configured".
|
||||
- No `.wikitool-tasks.json` fails with a "no tracker configured" message.
|
||||
- A provider whose configured access path has no write path (Super Productivity's `access: "snapshot"`) refuses entirely with exit 1, naming the `access: "api"` instance to use instead.
|
||||
- **Never exits 42.**
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: yes
|
||||
- Idempotent: yes
|
||||
- Atomic: Yes - a single API call; an unknown id is rejected by the provider itself (Super Productivity: `404 TASK_NOT_FOUND`) before anything is written
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: yes
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - No `.wikitool-tasks.json` - no tracker configured
|
||||
- `1` - `WIKITOOL_TASKS_CONFIG` names a file that does not exist or is broken
|
||||
- `1` - An `--id` matching no tracker item right now
|
||||
- `1` - A read-only access path (Super Productivity's `access: "snapshot"`)
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool task list` - where the id comes from
|
||||
- `wikitool review` - findings that carry item ids
|
||||
|
||||
## `touch`
|
||||
|
||||
Bump a page's `modified:` date and optionally rewrite its other frontmatter fields.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool touch --page "<Title>" [--summary "..."] [--provenance <v>] [--date YYYY-MM-DD] [--set field=value ...] [--add field=value ...] [--remove field=value ...] [--no-date] [--dry-run]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
--page <str> Exact page title, e.g. 'Docker Cheatsheet' [required]
|
||||
--summary <str> Replace the page's 1-line summary
|
||||
--provenance <str> Replace the page's provenance marker
|
||||
(sourced|general|mixed)
|
||||
--date <str> Date to record (YYYY-MM-DD). `modified:` defaults to
|
||||
today; a source's `date:` is its publication date and
|
||||
changes only when given here.
|
||||
--set <str> Replace a frontmatter field, repeatable: --set tags=a,b.
|
||||
Array values split on commas (escape a literal one as \,);
|
||||
repeating --set for one array field appends within this
|
||||
call. Page-reference fields belong to `xref`, not here
|
||||
--add <str> Append elements to an array field without naming the whole
|
||||
list: --add tags=x. Elements already present are left
|
||||
alone
|
||||
--remove <str> Drop elements from an array field: --remove tags=x.
|
||||
Removing an absent element succeeds and says so
|
||||
--no-date Only change the given fields; leave the modified/date
|
||||
field alone
|
||||
--dry-run Preview the new frontmatter instead of writing
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool touch --page "Docker" --summary "Container runtime and image format"
|
||||
tools/wikitool touch --page "Docker" --add tags=containers --no-date
|
||||
tools/wikitool touch --page "Source - Docker Cheatsheet" --date 2026-09-01 --dry-run
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Bumps `modified:` to today and optionally rewrites any other field the page's type declares.
|
||||
- `--summary`/`--provenance` are shorthands; `--set` reaches every other field and **replaces** its value, while `--add`/`--remove` change single elements of an array field (removing an absent element succeeds and says so).
|
||||
- Repeating `--set` for one array field appends *within the call*, and `\,` is a literal comma.
|
||||
- Refused, naming the command that owns them instead: `type:` (the page-lifecycle procedure) and the page-ref arrays `related:`/`sources:`/`entities:`/`concepts:` (`xref`). Everything else the schema declares is settable, and an unknown field is refused with the list of fields the page actually has.
|
||||
- Schema-validates the fields it writes; `raw_files:` entries must exist on disk.
|
||||
- A source page declares `date:` instead of `modified:`: the *publication* date of the raw material. It is never bumped to today and changes only when `--date` names a value explicitly.
|
||||
- `--no-date` changes only the given fields and leaves the date alone; `--dry-run` previews the new frontmatter without writing.
|
||||
- Safe to re-run as-is: `--set` and `--add` are idempotent, and `--remove` of an already-absent element succeeds while reporting it.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: yes
|
||||
- Idempotent: yes
|
||||
- Atomic: Yes - single file write, and every refusal happens before it
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - Page not found
|
||||
- `1` - An invalid value for a field it writes, or a `raw_files:` path that doesn't exist
|
||||
- `1` - A field owned by another command (`type:`, a page-ref array) or absent from the type's schema
|
||||
- `1` - `--add`/`--remove` on a non-array field
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool xref add` / `wikitool xref remove` - the page-ref arrays
|
||||
- `instructions/page-lifecycle.md` - changing a page's `type:`
|
||||
|
||||
## `rename`
|
||||
|
||||
Rename a page, or repoint references that name a page that never existed.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool rename --from "<Old>" --to "<New>" [--dry-run]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
--from <str> Current page title, exactly as it appears [required]
|
||||
--to <str> New page title [required]
|
||||
--dry-run List what would change instead of writing
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool rename --from "act_runner" --to "Act Runner" --dry-run
|
||||
tools/wikitool rename --from "Docker Engine" --to "Docker"
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Renames a page and repoints every reference to it: body `[[wikilinks]]` (aliases and anchors preserved), a `[^cite-id]` whose id was derived from the old title (refreshed to match the new one, both in its Footnotes definition and every reference to it), the page's own H1, and every page-ref frontmatter array declared by the type's `page_ref_fields:`.
|
||||
- If `--from` is *not* a page but is referenced, it instead repoints those references onto the existing `--to` page and moves nothing - the fix for a reference spelled `act_runner` when the page is `Act Runner`.
|
||||
- Each page's rewrite is idempotent, so a re-run as-is is safe. If a write fails midway, nothing is renamed on disk and the error lists what was updated.
|
||||
- `--dry-run` lists every page it would change; run it first to see the blast radius.
|
||||
- Only `--to` is checked against the title rule and the path budget (160 UTF-16 code units for the whole path below the instance root). A page whose current title breaks either (`lint`'s Unportable Titles and Long Paths) can always be renamed away from it, and a title that differs from the page's own only by case (`Foo` to `FOO`) is allowed.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: yes
|
||||
- Idempotent: yes
|
||||
- Atomic: No - one write per referencing page, then the file move
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - `--from` equals `--to`
|
||||
- `1` - Neither `--from` nor `--to` is a page
|
||||
- `1` - The `--to` title is already taken - also by a page that differs only in case or Unicode normalization, or by a file in the page's directory - or is not a valid file name, or would put the page's path over the path budget (see `kb/CONTRACT.md` § Titles are identifiers)
|
||||
- `1` - A page write failed partway; nothing was renamed on disk
|
||||
|
||||
**See also**
|
||||
|
||||
- `instructions/page-lifecycle.md` - renaming, moving and deleting a page
|
||||
- `wikitool move` - changes a page's directory, not its title
|
||||
- `wikitool rm` - deletes a page
|
||||
|
||||
## `rm`
|
||||
|
||||
Delete a page and mechanically de-link it from the rest of the wiki.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool rm --page "<Title>" [--yes] [--dry-run]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
--page <str> Exact title of the page to delete [required]
|
||||
-y, --yes Confirm deletion of a page that other pages still reference.
|
||||
Only pass this after a human has reviewed the inbound list -
|
||||
never set it automatically.
|
||||
--dry-run List what would change instead of writing
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool rm --page "Old Draft" --dry-run
|
||||
tools/wikitool rm --page "Old Draft" --yes # after the user approved the inbound list
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Deletes a page and de-links it from the rest of the wiki: strips page-ref array entries and bare `- [[Title]]` / `- **label:** [[Title]]` bullets pointing at it.
|
||||
- Leaves prose references and inline citations in place and reports them afterwards; those are an editorial fix, not a reason to retry.
|
||||
- Refuses without `--yes` while other pages still reference it, listing them.
|
||||
- If a de-link write fails midway, the page is not deleted, so nothing is orphaned and a re-run is safe.
|
||||
- `--dry-run` lists what would change, without writing.
|
||||
- Not idempotent: once the page is gone, a second run finds nothing to delete.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: yes
|
||||
- Idempotent: no
|
||||
- Atomic: No - one write per referencing page, then the delete
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - Page not found
|
||||
- `1` - Other pages still reference it and `--yes` was not passed
|
||||
- `1` - A de-link write failed partway; the page was not deleted
|
||||
|
||||
**See also**
|
||||
|
||||
- `instructions/page-lifecycle.md` - when a page is deleted rather than renamed
|
||||
- `wikitool rename` - repoints references instead of removing them
|
||||
|
||||
## `move`
|
||||
|
||||
Move a page (or every misplaced page) to the directory its type-spec computes.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool move --page "<Title>"`
|
||||
- `wikitool move --reconcile [--dry-run]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
--page <str> Exact title of the page to move to its computed location
|
||||
--reconcile Move every page under kb/ that is not at its computed location
|
||||
--dry-run List what would move without writing
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool move --page "Docker"
|
||||
tools/wikitool move --reconcile --dry-run
|
||||
tools/wikitool move --reconcile
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Moves a page to the directory its type-spec computes for its current frontmatter (`base_dir` + `layout`, the same rule `new` places a page by) - never to a hand-chosen destination; there is no `--to <dir>`.
|
||||
- A destination whose path would be over the path budget (160 UTF-16 code units below the instance root) is refused, and `--reconcile` skips such a page and names it, as it does for an occupied destination - `wikitool rename` the page to a shorter title.
|
||||
- `--reconcile` applies it corpus-wide: every misplaced page moves in one call, and a second run reports nothing left to do. It fixes `lint`'s `Misplaced Pages` (advisory) and `Nested Pages` (hard) findings.
|
||||
- Neither mode touches a body or a frontmatter field, and the page's title - its only identity in the wiki - never changes; only the file moves.
|
||||
- A directory a move empties is removed with it, so a page nested below its area leaves no leftover directory behind.
|
||||
- A page already at its computed location is reported and left alone, and `--reconcile` only re-moves what is still misplaced, so a re-run is safe.
|
||||
- `--dry-run` (with `--reconcile`) lists the moves; run it first to see the blast radius.
|
||||
- Run `wikitool index rebuild` afterwards.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: yes
|
||||
- Idempotent: yes
|
||||
- Atomic: `--page`: yes, a single file move. `--reconcile`: no - one file move per page, each idempotent
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - Neither or both of `--page`/`--reconcile` given
|
||||
- `1` - The named page is not found, or has no `type:` to compute a placement from
|
||||
- `1` - The destination already holds an entry with the same name, or one that differs only in case or Unicode normalization (a pre-existing duplicate-stem collision), or its path would be over the path budget - refused rather than silently skipped
|
||||
- `1` - `--reconcile` failed partway
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool lint` - reports Misplaced and Nested Pages
|
||||
- `wikitool index rebuild` - run after moving
|
||||
- `instructions/page-lifecycle.md` - moving, renaming and deleting a page
|
||||
<!-- /wikitool:reference-commands -->
|
||||
@@ -0,0 +1,151 @@
|
||||
# Provenance
|
||||
|
||||
Which raw source backs which page, and the generated provenance index that records it.
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
|
||||
- [`sources coverage`](#sources-coverage)
|
||||
- [`sources trace`](#sources-trace)
|
||||
- [`sources rebuild-index`](#sources-rebuild-index)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
<!-- wikitool:reference-commands -->
|
||||
## `sources coverage`
|
||||
|
||||
List raw files with no source page, broken `raw_files:` references, and legacy directory/URL-only source pages.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool sources coverage [--json]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
--json Print raw findings as JSON
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool sources coverage
|
||||
tools/wikitool sources coverage --json
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Lists raw files with no source page, broken `raw_files:` references, and legacy directory/URL-only source pages.
|
||||
- `--json` prints the same lists as JSON.
|
||||
- Never fails; read-only and safe to retry freely.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: no
|
||||
- Idempotent: yes
|
||||
- Atomic: Read-only
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool sources trace` - follows one file or page
|
||||
- `wiki-ingest` skill - turns an uncovered raw file into a source page
|
||||
|
||||
## `sources trace`
|
||||
|
||||
Trace provenance in either direction: raw file, or page.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool sources trace --raw <path> | --page "<Title>"`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
--raw <str> Raw file path to trace forward from
|
||||
--page <str> Wiki page title to trace backward from
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool sources trace --page "Docker"
|
||||
tools/wikitool sources trace --raw "raw/articles/llm-wiki.md"
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- `--raw <path>`: raw file -> the source page(s) covering it -> the pages citing those.
|
||||
- `--page "<Title>"`: page -> its sources -> their raw files.
|
||||
- Read-only.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: no
|
||||
- Idempotent: yes
|
||||
- Atomic: Read-only
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - Neither or both of `--raw`/`--page` given, or `--page` names an unknown page
|
||||
- `1` - `--raw` names a file no source page covers - reported as a plain finding plus exit 1, not the usual `ERROR`-prefixed rejection
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool sources coverage` - every uncovered raw file at once
|
||||
- `wikitool xref link-source` - links a source page to what it mentions
|
||||
|
||||
## `sources rebuild-index`
|
||||
|
||||
Regenerate the `kb/provenance.md` reverse index.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool sources rebuild-index [--dry-run]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
--dry-run Print the result instead of writing kb/provenance.md
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool sources rebuild-index
|
||||
tools/wikitool sources rebuild-index --dry-run
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Regenerates the `kb/provenance.md` reverse index (raw file -> source page -> citing pages) from scratch.
|
||||
- `--dry-run` prints the result instead of writing `kb/provenance.md`.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: yes
|
||||
- Idempotent: yes
|
||||
- Atomic: Yes - the single provenance file is regenerated from scratch
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - An I/O error while writing `kb/provenance.md` (rare)
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool index rebuild` - the page catalog, rebuilt alongside
|
||||
- `instructions/publish-cycle.md` - where a write session runs this
|
||||
<!-- /wikitool:reference-commands -->
|
||||
@@ -0,0 +1,623 @@
|
||||
# Raw material and uploads
|
||||
|
||||
Bringing new material into `raw/`: fetching and capturing it, seeing what is waiting, accepting it, and reviewing what an MCP client uploaded.
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
|
||||
- [`raw fetch`](#raw-fetch)
|
||||
- [`raw derive`](#raw-derive)
|
||||
- [`raw capture`](#raw-capture)
|
||||
- [`raw status`](#raw-status)
|
||||
- [`raw pending`](#raw-pending)
|
||||
- [`raw accept`](#raw-accept)
|
||||
- [`upload list`](#upload-list)
|
||||
- [`upload show`](#upload-show)
|
||||
- [`upload accept`](#upload-accept)
|
||||
- [`upload reject`](#upload-reject)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
<!-- wikitool:reference-commands -->
|
||||
## `raw fetch`
|
||||
|
||||
Capture a web page the user names into `incoming/`: the HTML as received plus a derived text, for `raw accept` to promote.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool raw fetch <url> [--name <stem>]` - Fetch the page and write `incoming/<stem>.html` and `incoming/<stem>.md`
|
||||
- `wikitool raw fetch --html incoming/<file>.html --url <url>` - Derive the `.md` from a page a human saved from their browser - no network access
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
Arguments:
|
||||
url The http(s) URL to fetch
|
||||
|
||||
Options:
|
||||
--html <path> Derive the text from this HTML file under incoming/ (saved from
|
||||
a browser) instead of fetching - no network access
|
||||
--url <str> With --html: the page's URL, for the header and relative links
|
||||
--name <str> File stem in incoming/ instead of the one computed from the URL
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool raw fetch https://example.org/blog/post
|
||||
tools/wikitool raw fetch https://example.org/ --name example-start
|
||||
tools/wikitool raw fetch --html incoming/post.html --url https://example.org/blog/post
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Writes into `incoming/` only, never into `raw/`: `raw accept` promotes both files afterwards, in one call, as one bundle under `raw/<YYYY>/<MM>/<stem>/`.
|
||||
- The `.html` holds the response body byte for byte. The `.md` starts with a fixed header block (`fetched_by`, `url`, `final_url`, `retrieved`, `http_status`, `content_type`, `charset`, `title`, `derived_from`) followed by the derived text.
|
||||
- Character set, in this order: byte-order mark, the HTTP `Content-Type` charset, a `<meta>` declaration in the first 4 KiB, UTF-8. Undecodable bytes are replaced with U+FFFD and reported; the header names the charset and where it came from.
|
||||
- Derived text: the content root is `<main>`, else the single `<article>`, else `<body>`; `script`, `style`, `noscript`, `nav`, `header`, `footer`, `aside`, `form`, `template` and `svg` are dropped below it. Headings, lists, code, tables and links (made absolute) become Markdown. The same bytes always give the same text.
|
||||
- The stem is the URL's last path segment without its extension, else its host name - ASCII, lowercase, `-`-separated, at most 60 characters; `--name` overrides it.
|
||||
- A `text/plain` or `text/markdown` response is stored as received as `.txt`/`.md`; any other non-HTML response (PDF, image, ...) as received under the extension of its content type. Neither gets a derived file or a header.
|
||||
- Only `http`/`https`, on every redirect hop too. 30 s for the whole transfer, 25 MiB at most. No cookies, no JavaScript: a derived text under 200 characters is written, with a warning to read it before accepting.
|
||||
- `--html` derives the `.md` beside a page saved to `incoming/` from a logged-in browser - the way past a paywall or a script-rendered page. Its header carries `derived` (when the text was derived) instead of `retrieved`, `final_url`, `http_status` and `content_type`, and the `.html` is left untouched.
|
||||
- Success prints the `raw accept` line for the written files and the `source_url` for the source page.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: yes
|
||||
- Idempotent: no
|
||||
- Atomic: Yes for what it leaves behind - one or two new files in `incoming/`, created exclusively; a failure on the second removes the first
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: yes
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - The URL is not `http`/`https` (also after a redirect), or neither or both of `<url>` and `--html` were given
|
||||
- `1` - A target file already exists in `incoming/`
|
||||
- `1` - The server answered with an HTTP error, could not be reached, took longer than 30 s, or sent more than 25 MiB
|
||||
- `1` - raw fetch --html: `--url` is missing, the file is not directly in `incoming/` or does not exist, or `incoming/<stem>.md` already exists
|
||||
|
||||
**See also**
|
||||
|
||||
- `raw/CONTRACT.md` "Getting a URL in: `raw fetch`" - the rules and why
|
||||
- `wikitool raw accept` - promotes the written files into `raw/`
|
||||
- `instructions/wiki-ingest/SKILL.md` - where a URL to ingest starts
|
||||
|
||||
## `raw derive`
|
||||
|
||||
Derive reading text from a PDF, e-mail, Office file or web page in `incoming/`, or unpack a ZIP, and leave the original beside it, for `raw accept` to promote.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool raw derive incoming/<file> [--name <stem>]` - Move the file into the new folder `incoming/<stem>/` as `original.<ext>` and write `incoming/<stem>/derived.md`
|
||||
- `wikitool raw derive incoming/<folder>` - Write a `<stem>.md` beside every file in the folder that has a filter
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
Arguments:
|
||||
path A file or a folder directly in incoming/ [required]
|
||||
|
||||
Options:
|
||||
--name <str> Folder stem for a single file instead of the file's own stem
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool raw derive incoming/report.docx
|
||||
tools/wikitool raw derive incoming/Angebot-XYZ.xlsx --name angebot-xyz
|
||||
tools/wikitool raw derive incoming/meeting-files
|
||||
tools/wikitool raw derive incoming/scans.zip
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Writes into `incoming/` only, never into `raw/`, and reads no network: `raw accept` promotes the folder afterwards as one source under `raw/<YYYY>/<MM>/<stem>/`.
|
||||
- A single file directly in `incoming/` becomes a folder named after its stem (or `--name`) holding `original.<ext>` - the original bytes, moved - and `derived.md`. The fixed names keep the path inside the path budget; the header of `derived.md` records the file's own name as `original_name`.
|
||||
- A folder is derived in place: each file with a filter gets `<stem>.md` beside it, nothing is renamed, and the files without a filter are named in the output.
|
||||
- `derived.md` starts with a fixed header (`derived_by`, `filter`, `engine`, `derived`, `derived_from`, `original_name` for a renamed file, `title`, then the fields of the format) followed by the derived text. The same bytes always give the same text; only `derived` differs between runs. `engine` names the tool version and, where a package reads the format, its version.
|
||||
- Formats: web pages (`.html`), Word (`.docx`), PowerPoint (`.pptx`), Excel (`.xlsx`, `.xlsm`), e-mail (`.eml`) and ZIP archives (`.zip`), read with the standard library only; PDF (`.pdf`, `pypdf`) and Outlook messages (`.msg`, `olefile`), each read by a package in `tools/requirements.txt` that is imported only when such a file is derived. A missing package refuses that format alone, with the preflight hint; `wikitool doctor` shows one `import-filter-<name>` line per package-backed format. A PDF gets `<!-- page N -->` markers and a WARN when it yields almost no text (a scan; there is no OCR); an encrypted PDF is refused. A `.msg` is a container like an `.eml`; an embedded message inside it is named, not read. The text of a mail (`.eml` and `.msg` alike) is its plain text, unless that is under 10% of the length of the HTML text - a "plain text version not available" placeholder - in which case it is the HTML text and a WARN names both lengths. The other version of the text in a `multipart/alternative` is not an attachment.
|
||||
- A workbook (`.xlsx`, `.xlsm`) gets its cells as files: beside its `.md` a folder `<md stem>.sheets/` (`derived.sheets/` for a single file) holds one `NN-<sheet>.tsv` per sheet - tab-separated per RFC 4180, UTF-8, row 1 of the file is row 1 of the sheet - and, for a sheet with formulas, `NN-<sheet>.formulas.tsv` listing cell, formula and last stored value. The `.md` is the overview: one section per sheet with its range, the file names and a preview. Formulas are not recalculated; a shared formula is written out per cell, and one that cannot be is named `shared:<master cell>` with a WARN. The folder counts against the path budget and is accepted with the rest; an existing one is never overwritten.
|
||||
- A ZIP and a mail are containers. A ZIP at the top of `incoming/` becomes the folder `incoming/<stem>/` holding its files under their own names, each with a `<stem>.md` where a filter reads it; the ZIP is removed and the output names it with its sha256. A mail becomes a folder with `original.eml`, `derived.md` and its attachments under their own names; a ZIP attached to it is unpacked once into a folder of its own, and an attached mail is quoted in `derived.md`, not unpacked. A ZIP inside a ZIP, an absolute or `..` path, a symbolic link, an encrypted entry, more than 1000 entries, a file over 25 MiB or more than 250 MiB in total refuse the whole call. Hidden entries (`__MACOSX/`, `.DS_Store`, anything starting with a dot) are named, not written. A name taken twice, or reserved (`original.*` and `derived.md` at the top of a mail's folder, `_capture.json` anywhere), gets `-2`, `-3` before its extension, and the output lists each rename.
|
||||
- A name already taken under `raw/` is a WARN, not a refusal: whether the folder is a later edition of that source or a separate one is the human's call. A later edition of a derived source is replaced file by file with `raw accept --replaces`; there is no `--replaces-bundle` for it.
|
||||
- After a success the output prints the warnings the filters raised, at most three `CHECK` lines per format that was derived in this run - what to look at before the text is trusted - and the `raw accept` line for the result.
|
||||
- Never edit `derived.md` by hand: a better derivation is a new `raw derive` of the original.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: yes
|
||||
- Idempotent: no
|
||||
- Atomic: Yes for what it leaves behind - everything is read, derived and checked before the first write; the files are created exclusively, and a failure part-way puts back what this call changed
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - The path is not directly in `incoming/`, does not exist, or no filter reads the file's extension
|
||||
- `1` - The target folder, a `.md` or a workbook's `.sheets` folder already exists, or the path would be over the path budget
|
||||
- `1` - The document cannot be read (damaged, encrypted), a container breaks one of its rules (a ZIP inside a ZIP, an unsafe path, a limit) or the filter's package is not installed
|
||||
|
||||
**See also**
|
||||
|
||||
- `raw/CONTRACT.md` "Getting a document in: `raw derive`" - the rules and why
|
||||
- `wikitool raw accept` - promotes the folder into `raw/`
|
||||
- `instructions/wiki-ingest/SKILL.md` - where a PDF, mail or Office file to ingest starts
|
||||
|
||||
## `raw capture`
|
||||
|
||||
Capture documentation from a git repository into `incoming/<bundle>/`, with a manifest naming the repository, ref rule and commit, for `raw accept` to promote.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool raw capture <repo-url> --ref <branch|tag-pattern> --path <glob> [--path <glob> ...] --name <bundle> --fidelity <v> --authority <v>` - First capture of a repository
|
||||
- `wikitool raw capture --update <raw-bundle> [--fidelity <v>] [--authority <v>]` - Capture the current state of an already-accepted bundle, for `raw accept --replaces-bundle`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
Arguments:
|
||||
repo_url The repository URL (ssh://, https:// or user@host:path)
|
||||
|
||||
Options:
|
||||
--ref <str> A branch name (main) or a tag pattern (v*)
|
||||
--path <str> A glob over the repository's paths; repeatable
|
||||
--name <str> The bundle name - the folder in incoming/ and under raw/
|
||||
--fidelity <str> How faithful the capture is (see `wikitool types describe
|
||||
source`)
|
||||
--authority <str> What the material may claim about its subject (see
|
||||
`wikitool types describe source`)
|
||||
--update <path> Capture the current state of this captured bundle under
|
||||
raw/, from its own manifest
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool raw capture ssh://git@example.org/team/service.git --ref main --path 'docs/**/*.md' --path README.md --name service-docs --fidelity verbatim --authority normative
|
||||
tools/wikitool raw capture https://example.org/team/lib.git --ref 'v*' --path docs --name lib-docs --fidelity verbatim --authority reporting
|
||||
tools/wikitool raw capture --update raw/2026/10/service-docs
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Writes into `incoming/<bundle>/` only, never into `raw/`: `raw accept incoming/<bundle>` promotes it, as one folder, to `raw/<YYYY>/<MM>/<bundle>/`.
|
||||
- `--ref` is a branch name (`main`) or a tag pattern (`v*`, any of `*?[`), which names the newest matching tag by version order. It resolves to one commit - for an annotated tag, the commit it points at.
|
||||
- The commit is fetched by ref name, shallowly, into a bare cache repository per URL under `tools/.wikitool_capture/` (gitignored). Files are read from it as blobs, never through a checkout, so they are byte-identical to the repository's - no line-ending conversion, no filter.
|
||||
- `--path` globs (repeatable) select files by their path in the repository, as git's `:(glob)` pathspec does: `*`, `?` and `[...]` do not cross `/`; `**` as a whole segment spans any number of directories; a glob without wildcards also takes everything below it as a directory. Files keep their repository paths inside the bundle.
|
||||
- Never captured, each named in the output with its reason: a file whose first line starts with `<!-- wikitool:export`, a symlink, a submodule, a file over 25 MiB, a path with a segment starting with `.`, a file named `_capture.json`, and a Git LFS pointer.
|
||||
- `incoming/<bundle>/_capture.json` records `schema`, `repo`, `ref`, `commit`, `paths`, `captured` (UTC), `fidelity`, `authority` and `files`. It is the only declaration that and how this instance follows the repository.
|
||||
- Only `ssh://`, `https://` and the scp form `user@host:path`; a URL with a password or token in it is refused, since the manifest is committed. Git runs with the host's own credentials and configuration, restricted to those protocols, and never prompts: a repository that asks for credentials is reported as unreachable. Each git call times out after 120 s.
|
||||
- `--update <raw-bundle>` takes URL, ref rule and globs from that bundle's manifest and writes the current state to `incoming/<same bundle name>/`; `--fidelity`/`--authority` default to the manifest's values, and passing one is the way to correct it.
|
||||
- Refused before anything is written: `incoming/<bundle>` already exists, the name is already taken under `raw/` (a first capture), nothing matches the globs, or a target path - measured at its later place under `raw/` - is over the path budget.
|
||||
- Success prints the commit and the `raw accept` line that comes next.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: yes
|
||||
- Idempotent: no
|
||||
- Atomic: Yes for what it leaves in `incoming/` - the bundle is written into a hidden staging directory and renamed into place in one step; a failure removes the staging directory. The git cache under `tools/.wikitool_capture/` is updated either way
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: yes
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - The URL is not `ssh://`, `https://` or `user@host:path`, or carries a password; the `--ref` rule, a `--path` glob or `--name` is unusable; or an argument of the other variant was mixed in
|
||||
- `1` - `--fidelity`/`--authority` is missing on a first capture, or names `unknown` or a value outside the schema's enum
|
||||
- `1` - The repository could not be reached, asked for credentials, or timed out; or no branch or tag matches `--ref`
|
||||
- `1` - `incoming/<bundle>` already exists, the name is already taken under `raw/`, nothing matches the globs, or a path would be over the budget
|
||||
- `1` - raw capture --update: The path is not a bundle under `raw/` with a readable `_capture.json`, or its manifest names a URL that is refused
|
||||
|
||||
**See also**
|
||||
|
||||
- `raw/CONTRACT.md` "Getting a repository in: `raw capture`" - the rules and why
|
||||
- `wikitool raw accept` - promotes the bundle; `--replaces-bundle` for a new edition
|
||||
- `wikitool raw status` - which captured bundles have fallen behind their repository
|
||||
|
||||
## `raw status`
|
||||
|
||||
Report which captured bundles under `raw/` have fallen behind their repository, with each changed file as `A`/`M`/`D`.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool raw status [--json]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
--json Print one JSON object per captured bundle
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool raw status
|
||||
tools/wikitool raw status --json
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Reads every `_capture.json` under `raw/` and resolves its ref rule with `git ls-remote`. Only where the commit moved is the new commit fetched, and compared - with the same globs and exclusions `raw capture` applies - against the bundle's files under `raw/`.
|
||||
- A bundle whose commit moved without a change inside its globs is reported as unchanged, not listed. A tag pattern follows new matching tags only, never new commits on a branch.
|
||||
- Each changed bundle prints `old -> new` and its files as `A`/`M`/`D`, grouped by owning source page with its citing pages, followed by the `raw capture --update` and `raw accept --replaces-bundle` lines that take the new edition in.
|
||||
- An unreachable repository, a refused URL or an unreadable manifest is one line beside the others, never an abort; the exit status is 0 as long as the command itself ran.
|
||||
- `--json` prints one object per bundle: `bundle`, `repo`, `ref`, `old`, `new`, `changed`, `files` (`[{path, status}]`), `error` (or null).
|
||||
- Git runs exactly as for `raw capture`: the host's credentials, only `ssh`/`https`, never a prompt, 120 s per call.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: no
|
||||
- Idempotent: yes
|
||||
- Atomic: Read-only for the tree - writes nothing but the git cache under `tools/.wikitool_capture/`
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: yes
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `0` - A repository could not be reached, its manifest or URL was refused, or no ref matches its rule
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool raw capture` - `--update` captures the new edition
|
||||
- `wikitool raw accept` - `--replaces-bundle` takes it in
|
||||
- `raw/CONTRACT.md` "Getting a repository in: `raw capture`"
|
||||
|
||||
## `raw pending`
|
||||
|
||||
List what waits in `incoming/`, oldest first, and name the entry an ingest without an argument takes next.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool raw pending [--json]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
--json Print the candidates as JSON
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool raw pending
|
||||
tools/wikitool raw pending --json
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- A candidate is a top-level entry of `incoming/`: a single `file`, a `bundle` of top-level files sharing a stem (a `raw fetch` pair), or a `folder` with every file below it. Dotfiles, empty directories and their contents are none; `mcp-upload/` is outside `incoming/` and never listed.
|
||||
- Order: oldest first by modification time. A bundle or folder counts as new as its newest file; a tie goes by name. The mtime is when a document last changed only if it was copied with its timestamps kept (`cp -p`, `rsync -a`, an unpacked archive) - for a download or a `raw fetch` it is merely when it was dropped.
|
||||
- Each candidate shows its path(s), kind, file count and mtime, and whether `raw accept` would take it as it stands - the same checks, minus `--fidelity`/`--authority`. One it would refuse is listed with the reason and skipped: it needs a human.
|
||||
- The default is the first candidate `raw accept` would take, and the output names it.
|
||||
- `--json` prints the same candidates in the same order: `kind`, `paths`, `files`, `mtime`, `acceptable`, `reason`, `default`, `hint`.
|
||||
- A lone file that a `raw derive` filter reads (a PDF, mail, Office file or web page) is listed with the `raw derive` line for it, in text output and as `hint` in JSON (`null` for every other candidate). The hint changes nothing about the order or the default.
|
||||
- Reads directory listings and `lstat` only, never a file's content; an empty `incoming/` is exit 0 with nothing to do.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: no
|
||||
- Idempotent: yes
|
||||
- Atomic: Read-only
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool raw accept` - promotes the chosen candidate
|
||||
- `instructions/wiki-ingest/SKILL.md` - ingest without an argument starts here
|
||||
- `raw/CONTRACT.md` "Getting a file in: incoming/" - candidates and order, and why
|
||||
|
||||
## `raw accept`
|
||||
|
||||
Promote one or more files, or one folder, from `incoming/` into `raw/`.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool raw accept <file> [<file> ...] --fidelity <v> --authority <v> [--page "<Title>"] [--dry-run]` - Promote one or more files from `incoming/` into today's `raw/<YYYY>/<MM>/` shard
|
||||
- `wikitool raw accept incoming/<folder> --fidelity <v> --authority <v> [--dry-run]` - Promote a whole folder as one source, its structure kept, into `raw/<YYYY>/<MM>/<folder>/`
|
||||
- `wikitool raw accept <file> --replaces <raw-path> [--fidelity <v>] [--authority <v>] [--dry-run]` - Overwrite one existing raw file in place with a new edition
|
||||
- `wikitool raw accept incoming/<bundle> --replaces-bundle <raw-bundle> [--dry-run]` - Replace a captured bundle as a whole with the new edition `raw capture --update` wrote
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
Arguments:
|
||||
files... One or more files directly in incoming/, all belonging to the same
|
||||
source - or one folder in incoming/, accepted whole as one source
|
||||
[required]
|
||||
|
||||
Options:
|
||||
--fidelity <str> How faithful the capture is (see `wikitool types
|
||||
describe source`). Required unless --replaces is
|
||||
given, where it is the optional, sanctioned way to
|
||||
correct an already-set value
|
||||
--authority <str> What the material may claim about its subject (see
|
||||
`wikitool types describe source`). Required unless
|
||||
--replaces is given, where it is the optional,
|
||||
sanctioned way to correct an already-set value
|
||||
--page <str> Extend this existing source page's raw_files: with
|
||||
the promoted file(s), folding in its already-
|
||||
promoted file if this raises it past one
|
||||
--replaces <path> Overwrite this existing raw/ file in place with the
|
||||
single incoming/ file passed alongside it - the only
|
||||
sanctioned way past the stem-uniqueness rule below
|
||||
--dry-run List what would move without writing
|
||||
--replaces-bundle <path> Replace this captured bundle under raw/ as a whole
|
||||
with the new edition in the incoming/ folder passed
|
||||
alongside it (written by raw capture --update)
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool raw accept incoming/docker-cheatsheet.md --fidelity verbatim --authority reporting
|
||||
tools/wikitool raw accept incoming/part-2.md --fidelity verbatim --authority reporting --page "Source - Docker Cheatsheet"
|
||||
tools/wikitool raw accept incoming/projekt-x --fidelity verbatim --authority reporting
|
||||
tools/wikitool raw accept incoming/cluster.md --replaces raw/documents/cluster.md
|
||||
tools/wikitool raw accept incoming/chemenu --replaces-bundle raw/2026/10/chemenu
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Promotes files from `incoming/` into `raw/<YYYY>/<MM>/`, computed from the accept date rather than chosen by hand. A file argument must sit directly in `incoming/`; a file inside a subdirectory is refused, since a subdirectory is a source of its own.
|
||||
- A folder argument (`incoming/<folder>`) is one source: every file below it moves to `raw/<YYYY>/<MM>/<folder>/` at the same relative path, and the directories left empty are removed - `incoming/<folder>` no longer exists afterwards. The folder name is the bundle name; two `README.md` in different subfolders are no conflict.
|
||||
- A folder is accepted alone - no other argument, no `--page`, no `--replaces` - with one `--fidelity`/`--authority` pair for all of it. It is refused, before anything moves, if it is empty, or if a hidden entry (name starting with `.`), a symlink or a special file sits anywhere below it; the refusal names each one.
|
||||
- One file promoted alone lands with no directory of its own; several files in one call nest under `raw/<YYYY>/<MM>/<stem>/`, named after the first file's stem.
|
||||
- `--fidelity`/`--authority` are required on a plain accept (`types describe source` lists the values); `unknown` is refused - it is backfill-only.
|
||||
- `--page "<Title>"` additionally extends that existing source page's `raw_files:` in the same call and writes both capture fields onto it - refused if it already carries a different value, since a capture field is fixed once.
|
||||
- If `--page` raises the page past one file, its already-promoted file is folded into a bundle at *its own* parent directory, not today's shard, so a bundle never mixes an old and a new capture date - after checking that file has no other owner.
|
||||
- Every name occupied anywhere under `raw/` - file stems and bundle directory names alike, old type directories and date shards together - stays unique: a promote whose target name already belongs to something this call does not itself own is refused, naming `--replaces` and renaming in `incoming/` as the two routes, without recommending either.
|
||||
- `--replaces <raw-path>` overwrites that file in place with the single incoming file (same filename required), leaves every page's `raw_files:` untouched and writes no `kb/` page; the previous edition survives only in `git log --follow <raw-path>`.
|
||||
- With `--replaces`, `--fidelity`/`--authority` are optional, and passing one overwrites the owning page's already-set value - the one path the fixed-once rule does not block.
|
||||
- `--replaces` refuses a target with more than one owning source page; with none, it replaces anyway and says so. It cannot be combined with `--page` or with more than one incoming file.
|
||||
- `--replaces` prints the source page (if any) and its citing pages, so their update lands in the same commit as the replacement.
|
||||
- A file already at its computed destination is what "already exists" reports, not a partial prior run to resume - safe to retry as-is once a cause is fixed.
|
||||
- A folder carrying `_capture.json` at its top is a captured bundle (`raw capture`): `--fidelity`/`--authority` come from that manifest only and are refused on the command line; the folder must hold exactly the files the manifest lists. `_capture.json` goes in no `raw_files:`.
|
||||
- `--replaces-bundle <raw-bundle>` replaces a captured bundle under `raw/` with the new edition in `incoming/<bundle>` at the same address: files gone from the new edition are removed, emptied directories `rmdir`ed, and afterwards the bundle holds exactly the new manifest's files plus `_capture.json`. `raw_files:` is left untouched; an owning source page's `fidelity`/`authority` is overwritten where the new manifest differs.
|
||||
- `--replaces-bundle` prints each changed file as `A`/`M`/`D`, grouped by owning source page with its citing pages, and the `touch --page ... --add/--remove raw_files=` lines that follow; `git diff` on the bundle shows the `M` and `D` files, the `A` files appear only in that list because they are still untracked.
|
||||
- `--replaces` and `--page` refuse a target inside a captured bundle; the refusal names `raw capture --update` and `--replaces-bundle`. A file named `_capture.json` anywhere but at a captured folder's top is refused - the name is reserved.
|
||||
- `--dry-run` reports the moves without making them.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: yes
|
||||
- Idempotent: no
|
||||
- Atomic: `raw accept`: No - one filesystem move per file, then (with `--page`) one page write. With a folder: No - one move per file, then one `rmdir` per emptied directory; a half-accepted folder is not resumed. `raw accept --replaces`: No - one `unlink()` + one `rename()`, plus (if `--fidelity`/`--authority` was given) one page write. `raw accept --replaces-bundle`: No - one `unlink()` per removed file, one `rename()` per added or modified file, then the manifest, then one `rmdir` per emptied directory, plus one page write per owning page whose capture fields change; a half-replaced bundle is not resumed
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - raw accept: A file does not exist or is not directly in `incoming/`; two files in one call share a filename; a target path already exists; or a target path would be over the path budget (160 UTF-16 code units below the instance root)
|
||||
- `1` - raw accept incoming/<folder>: The folder is not directly in `incoming/`, is combined with another argument, `--page` or `--replaces`, holds no file, or holds a hidden entry, a symlink or a special file; a target path is over the budget; or the folder name is already occupied under `raw/`
|
||||
- `1` - A file - given directly, inside a folder, or as `--replaces`/`--replaces-bundle` material - is a guideline export: its first line, after an optional BOM, starts with `<!-- wikitool:export`
|
||||
- `1` - raw accept: `--fidelity`/`--authority` is missing, or names `unknown` or a value outside the schema's enum
|
||||
- `1` - raw accept: The target name is already occupied anywhere under `raw/` by something the call does not own
|
||||
- `1` - raw accept: `--page` names an unknown page or one with no `raw_files:` yet, an existing `raw_files:` entry is missing on disk, a file to be moved has more than one owning page, or `--page` would overwrite an already-set `fidelity`/`authority` with a different value
|
||||
- `1` - raw accept --replaces: More than one incoming file, or `--page` also given
|
||||
- `1` - raw accept --replaces: The incoming file does not exist or is not directly in `incoming/`, its filename differs from the target's, or the target does not lie under `raw/` or does not exist
|
||||
- `1` - raw accept --replaces: `--fidelity`/`--authority` names `unknown` or a value outside the schema's enum, or the target has more than one owning source page
|
||||
- `1` - raw accept --replaces / --page: The target, or a file the page already has, lies in a captured bundle
|
||||
- `1` - raw accept incoming/<bundle>: A captured folder is given `--fidelity`/`--authority`, its `_capture.json` cannot be read or names an invalid value, or its files differ from the ones the manifest lists
|
||||
- `1` - raw accept --replaces-bundle: Either side has no `_capture.json`, the two manifests name different repositories, the folder's name differs from the bundle's, `--page`, `--replaces`, `--fidelity` or `--authority` was also given, the folder does not match its manifest, a new path is over the budget, or a file of the bundle has more than one owning source page
|
||||
|
||||
**See also**
|
||||
|
||||
- `raw/CONTRACT.md` "Getting a file in: incoming/" - the rules and why
|
||||
- `wikitool raw pending` - what is waiting in `incoming/`, and which entry is next
|
||||
- `wikitool raw capture` - writes a captured bundle, or its new edition, into `incoming/`
|
||||
- `wikitool types describe source` - the capture field values
|
||||
- `wikitool new source` - the source page for a promoted file
|
||||
|
||||
## `upload list`
|
||||
|
||||
List every MCP submission currently waiting in the quarantine (`mcp-upload/`).
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool upload list [--json]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
--json Print every waiting submission as JSON
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool upload list
|
||||
tools/wikitool upload list --json
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Lists every submission waiting in `mcp-upload/`, oldest id first: id, filename, size, submitter.
|
||||
- Only ever non-empty when `.wikitool-upload.json` opts the checkout into the MCP server's `submit` tool.
|
||||
- A submission directory with a corrupt manifest is silently skipped.
|
||||
- Never fails; read-only and safe to retry freely.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: no
|
||||
- Idempotent: yes
|
||||
- Atomic: Read-only
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool upload show` - one submission's manifest
|
||||
- `docs/how-to/set-up-the-mcp-server.md` § "Optional: the submit path"
|
||||
|
||||
## `upload show`
|
||||
|
||||
Print one submission's manifest in full.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool upload show <id> [--json]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
Arguments:
|
||||
submission_id A submission id from `upload list` [required]
|
||||
|
||||
Options:
|
||||
--json
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool upload show <id>
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Prints one submission's manifest in full: filename, size, sha256, submitter, submitter source (the header name, not a claim the header was honest), submission time.
|
||||
- What a reviewer reads before `upload accept`.
|
||||
- Read-only.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: no
|
||||
- Idempotent: yes
|
||||
- Atomic: Read-only
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - Unknown or malformed submission id
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool upload accept` - promotes it after review
|
||||
- `wikitool upload reject` - declines it
|
||||
|
||||
## `upload accept`
|
||||
|
||||
**Upload Review Gate:** promote a submission's file from quarantine into `incoming/`.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool upload accept <id> [--confirm TOKEN]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
Arguments:
|
||||
submission_id A submission id from `upload list` [required]
|
||||
|
||||
Options:
|
||||
--confirm <str> The token from a prior refusal, once a human has reviewed it
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool upload accept <id>
|
||||
tools/wikitool upload accept <id> --confirm <token> # re-run after exit 42, once the user approved the manifest
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Promotes a submission's file from `mcp-upload/<id>/` into `incoming/`, deletes the quarantine directory, and appends an `accepted` event to `mcp-upload/ledger.jsonl`.
|
||||
- Upload Review Gate: without a matching `--confirm`, exits 42 and prints the manifest in full plus the exact re-run line - one submission at a time. The gate check runs before anything is moved.
|
||||
- The token digests id, filename, size, sha256 and submitter, so an edited or superseded manifest invalidates it.
|
||||
- An occupied `incoming/<filename>` is an ordinary validation error, not the gate.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: yes
|
||||
- Idempotent: no
|
||||
- Atomic: No - one filesystem move, one directory delete, one ledger append; the gate check runs first, before any of them
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: no
|
||||
- Gates: upload-review - see [exit codes and gates](../exit-codes.md)
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - Unknown or malformed submission id, or the submission's file is missing from `mcp-upload/<id>/`
|
||||
- `1` - `incoming/<filename>` already exists
|
||||
- `42` - Upload Review Gate: `--confirm` is absent or does not match the manifest's current token
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool upload show` - the manifest to review
|
||||
- `wikitool raw accept` - the next step for the file in `incoming/`
|
||||
- `instructions/gates.md` - the gate procedure
|
||||
|
||||
## `upload reject`
|
||||
|
||||
Delete a submission's material, keeping only its ledger trail.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool upload reject <id> --reason "<why>"`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
Arguments:
|
||||
submission_id A submission id from `upload list` [required]
|
||||
|
||||
Options:
|
||||
--reason <str> Why this submission was declined [required]
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool upload reject <id> --reason "duplicate of raw/articles/llm-wiki.md"
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Appends an append-only `rejected` event naming the reason and the sha256 of what was declined, then deletes the submission's material.
|
||||
- The ledger write happens first, so an interruption still leaves the reason on record.
|
||||
- No gate - rejecting needs no clearance, only accepting does.
|
||||
- Not idempotent: a second call with the same id reports "unknown id", which confirms the first call worked.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: yes
|
||||
- Idempotent: no
|
||||
- Atomic: No - one ledger append, then one recursive delete; the ledger write happens first, so an interruption still leaves the reason on record
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - Unknown or malformed submission id, or an empty `--reason`
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool upload show` - the manifest to review
|
||||
<!-- /wikitool:reference-commands -->
|
||||
@@ -0,0 +1,114 @@
|
||||
# Telemetry
|
||||
|
||||
Reading and scoring the traces that sessions leave behind.
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
|
||||
- [`eval sessions`](#eval-sessions)
|
||||
- [`eval score`](#eval-score)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
<!-- wikitool:reference-commands -->
|
||||
## `eval sessions`
|
||||
|
||||
List the sessions that have a trace under `reports/telemetry/`.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool eval sessions [--json]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
--json Print the list as JSON
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool eval sessions
|
||||
tools/wikitool eval sessions --json
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Lists the sessions that have a trace under `reports/telemetry/`, most recent first.
|
||||
- Never fails; an empty list is a valid answer.
|
||||
- Read-only and exempt from the Iteration Budget Gate.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: no
|
||||
- Idempotent: yes
|
||||
- Atomic: Read-only
|
||||
- Counts toward the session call budget: no
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool eval score` - scores one of them
|
||||
- `docs/explanation/telemetry-and-evaluation.md` - how telemetry and evaluation work
|
||||
|
||||
## `eval score`
|
||||
|
||||
Score one traced session.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool eval score [--session <id>] [--json] [--markdown out.md] [--save] [--fail-on-error]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
--session <str> Session to score. Defaults to this shell's session, the
|
||||
same id the budget gate uses.
|
||||
--json Print the scorecard as JSON
|
||||
--markdown <path> Write a markdown scorecard here, conventionally under
|
||||
reports/evals/.
|
||||
--save Write the scorecard to
|
||||
reports/evals/<date>/<session>.{json,md}.
|
||||
--fail-on-error Exit non-zero when the tree has hard errors or an invariant
|
||||
was violated.
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool eval score
|
||||
tools/wikitool eval score --session wiki-1727330000 --save
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Scores one traced session: structural state from `lint`'s own checks (L1) plus trajectory rules over the trace (L2) - was a refused call repeated unchanged, was a gate flag passed without that gate having refused anything, did a publish of `kb/` pages go unlogged.
|
||||
- Defaults to the current session; `--session <id>` picks another.
|
||||
- `--save` writes `reports/evals/<date>/<session>.{json,md}`; `--markdown` writes the report to the named file.
|
||||
- A session records nothing when telemetry is off - `WIKI_TRACE=0`, or a distributed instance with no opt-in (`wikitool doctor` says which) - so an absent trace is not necessarily a fault.
|
||||
- Read-only over `kb/`, safe to retry, and exempt from the Iteration Budget Gate.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: no
|
||||
- Idempotent: yes
|
||||
- Atomic: Read-only, apart from the files `--save`/`--markdown` write
|
||||
- Counts toward the session call budget: no
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - No trace exists for the named session
|
||||
- `1` - Only with `--fail-on-error`: the tree has hard errors or an invariant was violated
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool eval sessions` - which session ids exist
|
||||
- `docs/explanation/telemetry-and-evaluation.md` - the scoring levels
|
||||
<!-- /wikitool:reference-commands -->
|
||||
@@ -0,0 +1,525 @@
|
||||
# Types, instructions and docs
|
||||
|
||||
The page types, the instruction and skill layer, and the documentation checks and generators that keep the docs in step with the code.
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
|
||||
- [`types list`](#types-list)
|
||||
- [`types describe`](#types-describe)
|
||||
- [`instructions sync`](#instructions-sync)
|
||||
- [`instructions verify`](#instructions-verify)
|
||||
- [`instructions list`](#instructions-list)
|
||||
- [`docs verify`](#docs-verify)
|
||||
- [`docs toc`](#docs-toc)
|
||||
- [`docs prerequisites`](#docs-prerequisites)
|
||||
- [`docs contract`](#docs-contract)
|
||||
- [`docs reference`](#docs-reference)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
<!-- wikitool:reference-commands -->
|
||||
## `types list`
|
||||
|
||||
List every type-spec under `types/`.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool types list [--json]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
--json Print raw findings as JSON
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool types list
|
||||
tools/wikitool types list --json
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Lists every type-spec under `types/`: name, schema path, subtype field, and description - which page types exist, without reading `types/*.md` directly.
|
||||
- Never fails; read-only and safe to retry freely.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: no
|
||||
- Idempotent: yes
|
||||
- Atomic: Read-only
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool types describe <name>` - one type's full contract
|
||||
|
||||
## `types describe`
|
||||
|
||||
Print one type's full contract.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool types describe <name> [--json]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
Arguments:
|
||||
name Type name, e.g. 'entity' (see `types list`) [required]
|
||||
|
||||
Options:
|
||||
--json Print raw findings as JSON
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool types describe source
|
||||
tools/wikitool types describe project --json
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Prints one type's full contract: required and optional frontmatter fields with enums, its subtype field (if any), and its authoring body.
|
||||
- Where the type-spec declares `guidance:`, the stack-owned `types/<name>.guidance.md` is composed in, so a `root: kb` type's contract reads as one answer even though it may live in two files. `--json` reports it separately as `guidance`/`guidance_path`, absent for a type with none.
|
||||
- The generated table-of-contents region a long type-spec (or guidance file) carries is stripped from this output.
|
||||
- Read-only.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: no
|
||||
- Idempotent: yes
|
||||
- Atomic: Read-only
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - Unknown type name
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool types list` - every type
|
||||
- `wikitool new <type>` - scaffolds a page of the type
|
||||
|
||||
## `instructions sync`
|
||||
|
||||
Publish every `instructions/<name>/SKILL.md` into the harness skill directories.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool instructions sync [--force]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
--force Replace a target directory whose contents differ from the source
|
||||
and was not generated by sync. Without this, sync refuses instead
|
||||
of deleting content it did not create.
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool instructions sync
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Publishes every `instructions/<name>/SKILL.md` into `.agents/skills/` and `.claude/skills/` as **copies**, and deletes published skills whose source is gone.
|
||||
- Both targets are gitignored, so a fresh clone runs this once.
|
||||
- Re-running repairs a drifted copy: the source always wins.
|
||||
- `--force` is required only to replace a target directory that is not a published skill at all (no `SKILL.md` in it).
|
||||
- Each copy is idempotent, so a re-run converges even after a partial failure.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: yes
|
||||
- Idempotent: yes
|
||||
- Atomic: No - one directory copy per skill per target (`.agents/skills/`, `.claude/skills/`); each copy is idempotent, so a re-run converges even after a partial failure
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - No skills found under `instructions/`
|
||||
- `1` - A target directory is not a published skill (no `SKILL.md`) and `--force` was not passed
|
||||
|
||||
**See also**
|
||||
|
||||
- `instructions/bootstrap.md` - the fresh-clone procedure that runs this
|
||||
- `wikitool instructions verify` - checks the copies match
|
||||
|
||||
## `instructions verify`
|
||||
|
||||
Check the instruction layer.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool instructions verify`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool instructions verify
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Flat instructions validate against `types/instruction.schema.yaml`, and each `SKILL.md` carries the frontmatter its harness reads.
|
||||
- No `SKILL.md` carries a relative markdown link: `sync` copies it to a different depth than the source, so a `SKILL.md` references a target as a repo-root-relative plain path instead (`instructions/CONTRACT.md` § "A skill's outbound reference is a plain path, not a link").
|
||||
- Every published copy is byte-identical to its source. Missing *every* copy is reported as "run sync", not as drift - that is a clean checkout.
|
||||
- No instruction is left that nothing references; one marked `manual: true` must instead not be linked from AGENTS.md, CLAUDE.md or a skill.
|
||||
- Nothing under `instructions/dev/` is referenced from outside it; a `<!-- dist:strip-start/end -->` block is exempt (`instructions/CONTRACT.md`).
|
||||
- Read-only.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: no
|
||||
- Idempotent: yes
|
||||
- Atomic: Read-only
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - Nothing found under `instructions/` at all, or a malformed instruction or `SKILL.md`
|
||||
- `1` - A `SKILL.md` carries a relative markdown link
|
||||
- `1` - A published copy drifted from its source
|
||||
- `1` - An instruction nothing references, or a `manual: true` one that IS linked from AGENTS.md, CLAUDE.md or a skill and so risks running implicitly
|
||||
- `1` - Something under `instructions/dev/` is referenced from outside it and outside a `dist:strip` block
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool instructions sync` - publishes the copies
|
||||
- `instructions/CONTRACT.md` - the rules this checks
|
||||
|
||||
## `instructions list`
|
||||
|
||||
List the flat instructions with their descriptions.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool instructions list [--json]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
--json Print the listing as JSON
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool instructions list
|
||||
tools/wikitool instructions list --json
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Lists the flat instructions with their descriptions - how the instruction layer is discovered; `search` covers `kb/` only.
|
||||
- Never fails: an empty `instructions/` prints "No instructions found." Read-only and safe to retry freely.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: no
|
||||
- Idempotent: yes
|
||||
- Atomic: Read-only
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool search` - the same question for `kb/`
|
||||
|
||||
## `docs verify`
|
||||
|
||||
Check the docs that mirror the code.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool docs verify`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool docs verify
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Checks the docs that mirror the code. The name is about documentation parity, not the `docs/` directory - it reads that only for its layout and its generated reference pages, and never requires one.
|
||||
- Commands: every command has a `cli_contract` record and is listed in `cli_contract.GROUPS`, in both directions; every command's non-hidden flags appear in its record's SYNOPSIS and vice versa; `tools/CONTRACT.md`'s generated `<!-- wikitool:commands -->` region matches what `docs contract` would write; no command's rendered `--help`/`-h` text cites an issue number.
|
||||
- Collections: every directory under `kb/` has a `COLLECTION.md` and no directory outside it does; every collection declares `profile:` and a `required_by_stack:` that agrees with the stack's own list.
|
||||
- Types: every type the stack lists (currently `source` and `project`) has a type-spec of that name whose schema requires the field the stack list names (`raw_files:`/`state:`); every file under `types/` declaring `type: types/type-spec.md` validates against `types/type-spec.schema.yaml`; every subtype template `types/<type>.<value>.md` (any `<value>` but `guidance`) sits beside a type-spec declaring `subtype_field:`, names a value that field's schema enum allows, and carries no frontmatter; no pre-migration `type: entity` blocks are left in the contracts.
|
||||
- `kb/CONVENTIONS.md`, if it exists at all, names all three tool-owned section headings; every stage contract is present.
|
||||
- The `.gitignore` canaries clear in both directions: nothing ignored under `raw/`/`kb/`, `incoming/` ignored, everything ignored under `reports/` and the published skill directories.
|
||||
- No `.md`/`.template` file `dist export` would ship cites an issue number. A `<!-- dist:strip-start/end -->` region is exempt: the check reads the export plan's text, from which it is already gone.
|
||||
- Every reference file `docs toc` covers carries the current table-of-contents region for its own headings - missing and stale are one check.
|
||||
- Every relative markdown link in one of those reference files resolves to an existing file. A target's `#anchor` suffix is stripped first, and code fences and inline code spans are masked before scanning, so link syntax shown as an example is not mistaken for a real reference.
|
||||
- Every file under `docs/` sits in one of its areas - `tutorials/`, `how-to/`, `reference/`, `explanation/` or `dev/` - never directly in `docs/` or in another directory.
|
||||
- `AGENTS.md` § Gates names exactly the gates in `cli_contract.GATES` (the bold entry names, `- **<Name>.**`), and every gate in that register that ends in exit 42 is declared by at least one command record. A record naming a gate outside the register fails when `wikitool` starts, not here.
|
||||
- If `docs/` exists: every generated region of the human reference under `docs/reference/` matches what `docs reference` would write, every `cli_contract.GROUPS` entry has its page under `docs/reference/commands/`, and no page there belongs to no group. That includes the four configuration regions in `docs/reference/configuration.md`. The skill catalogue and the page types are checked only in the origin repo (no `.wikitool-release.json`): they describe the stack's shipped defaults, which an instance's own type-specs and instructions may differ from.
|
||||
- `INSTALL.md` carries one generated `<!-- wikitool:prerequisites -->` region per platform value of `tools/prerequisites.txt` (`prerequisites-<platform>` for a platform-specific one), each current; and the `<!-- setup-question: <key> -->` markers in `instructions/setup-instance.md` and `INSTALL.md` name the same set of keys, so a question the agent asks is never one the human guide leaves out, nor the reverse.
|
||||
- `VERSION` parses and the newest versioned `CHANGES.md` entry names it. An entry that crosses the compatibility boundary from the last release carries `**Breaking Change:**`, has a migration document or `**Migration:** none required`, and its `**Migration:**` line names every required migration document targeting it.
|
||||
- The newest versioned `CHANGES.md` entry is at most 32 000 bytes (UTF-8) as `version notes` prints it, and each of its `###` sections is exactly one paragraph of at most 1 000 bytes, heading line excluded. Older entries and a changelog with no versioned entry are not checked.
|
||||
- Read-only.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: no
|
||||
- Idempotent: yes
|
||||
- Atomic: Read-only
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - A command, contract, or type-form mismatch
|
||||
- `1` - `VERSION` and the newest `CHANGES.md` entry disagree, or a boundary-crossing entry lacks its breaking line, its migration, or the migration document's path
|
||||
- `1` - The newest `CHANGES.md` entry is over 32 000 bytes, or one of its `###` sections has more than one paragraph or a paragraph over 1 000 bytes
|
||||
- `1` - The `<!-- wikitool:commands -->` region of `tools/CONTRACT.md` is stale
|
||||
- `1` - A type-spec's own frontmatter fails its schema
|
||||
- `1` - A subtype template `types/<type>.<value>.md` has no type-spec with a `subtype_field:` beside it, names a value outside that field's enum, or carries frontmatter
|
||||
- `1` - A shipped `.md`/`.template` cites an issue number
|
||||
- `1` - A region of the human reference under `docs/reference/` is stale
|
||||
- `1` - A page of the human reference is missing or lacks its marker pair, or a page under `docs/reference/commands/` belongs to no group
|
||||
- `1` - A reference file's table-of-contents region is missing or stale
|
||||
- `1` - A file sits directly in `docs/`, or in a directory there that is not one of its five areas
|
||||
- `1` - `AGENTS.md` § Gates and `cli_contract.GATES` disagree, or a register gate that ends in exit 42 is declared by no command record
|
||||
- `1` - A reference file's relative markdown link does not resolve to an existing file
|
||||
- `1` - An `INSTALL.md` prerequisites region is stale
|
||||
- `1` - An `INSTALL.md` prerequisites region is missing, or names a platform no tool has
|
||||
- `1` - A setup question is marked in one of `instructions/setup-instance.md` and `INSTALL.md` but not the other
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool docs toc` - regenerates tables of contents
|
||||
- `wikitool docs contract` - regenerates the commands region
|
||||
- `wikitool docs reference` - regenerates the human reference
|
||||
- `wikitool docs prerequisites` - regenerates `INSTALL.md`'s prerequisites lists
|
||||
- `wikitool instructions verify` - the same kind of check for `instructions/`
|
||||
|
||||
## `docs toc`
|
||||
|
||||
Create, refresh or remove the generated table-of-contents region.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool docs toc [--apply]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
--apply Write changes; default is dry-run (preview only)
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool docs toc
|
||||
tools/wikitool docs toc --apply
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Creates, refreshes or removes the generated table-of-contents region on every reference file over 100 lines: `AGENTS.md`, every stage contract, `kb/CONVENTIONS.md`, every `kb/*/COLLECTION.md`, every flat `instructions/**.md` file, every `types/*.md` type-spec, and every `docs/` page - each together with the `<name>.template` it ships as, where one exists.
|
||||
- The scope is computed from those categories rather than listed, so a file added later is in scope without a code change.
|
||||
- Out of scope: every `SKILL.md`, the human docs at the root and beside the code (`README.md`, `CHANGES.md`, `INSTALL.md`, `tools/README.md`), and every subtype template `types/<type>.<value>.md` - `new` copies it into a page, so it never carries a region whatever its length.
|
||||
- Dry-run by default (prints which files would change); `--apply` writes. Idempotent: a re-run after an interruption converges rather than doubling a region.
|
||||
- If `docs verify` still reports a stale region after `--apply`, the file's `##` headings changed in between; run it again.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: yes
|
||||
- Idempotent: yes
|
||||
- Atomic: `--apply` rewrites each named file in place, one at a time and idempotently, so a re-run after an interruption converges rather than doubling a region; the dry-run form is read-only
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `0` - Never fails on content: a file with no `##` heading, or one at or under the threshold, is simply left without a region
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool docs verify` - checks every region is current
|
||||
|
||||
## `docs prerequisites`
|
||||
|
||||
Regenerate `INSTALL.md`'s prerequisites lists from `tools/prerequisites.txt`.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool docs prerequisites [--apply]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
--apply Write changes; default is dry-run (preview only)
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool docs prerequisites
|
||||
tools/wikitool docs prerequisites --apply
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Rewrites each `<!-- wikitool:prerequisites -->` region in `INSTALL.md` (tools every platform needs) and `<!-- wikitool:prerequisites-<platform> -->` region (tools only that platform needs) from the manifest: one list item per tool, its label and minimum version. The manifest's reason field stays out - it explains a tool to whoever reads the preflight's own output, while the region only says what to install.
|
||||
- Never places a region: where a list belongs in the human guide is that guide's own decision. A region the manifest calls for but the file lacks is an error naming the marker pair to add.
|
||||
- Dry-run by default (says whether the file would change); `--apply` writes.
|
||||
- `docs verify` checks the result stays current.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: yes
|
||||
- Idempotent: yes
|
||||
- Atomic: Yes - every region is rewritten in one file write
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - `INSTALL.md` is missing, or lacks a region the manifest calls for
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool docs verify` - checks every region is current
|
||||
|
||||
## `docs contract`
|
||||
|
||||
Regenerate `tools/CONTRACT.md`'s `<!-- wikitool:commands -->` region.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool docs contract [--apply]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
--apply Write changes; default is dry-run (preview only)
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool docs contract
|
||||
tools/wikitool docs contract --apply
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Rebuilds the region from every `cli_contract` record: the index (one line per command, `GROUPS` order) followed by each `###` group's commands as `#### <path>` man-page-shaped sections.
|
||||
- Dry-run by default (says whether the file would change); `--apply` writes.
|
||||
- `docs verify` checks the result stays current.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: yes
|
||||
- Idempotent: yes
|
||||
- Atomic: Yes - the whole region is rewritten in one file write
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - `tools/CONTRACT.md` is missing
|
||||
|
||||
**See also**
|
||||
|
||||
- `tools/README.md` § Adding a command - how a command gets its record
|
||||
- `wikitool docs reference` - the same records, rendered for a person under `docs/reference/`
|
||||
- `wikitool docs verify` - checks the region is current
|
||||
|
||||
## `docs reference`
|
||||
|
||||
Regenerate the human reference under `docs/reference/`.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool docs reference [--apply]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
--apply Write changes; default is dry-run (preview only)
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool docs reference
|
||||
tools/wikitool docs reference --apply
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Rewrites the generated region of every page of the human command reference from the `cli_contract` records: one `docs/reference/commands/<group>.md` per `cli_contract.GROUPS` entry (`<!-- wikitool:reference-commands -->`), the overview in `docs/reference/commands/README.md` (`<!-- wikitool:reference-commands-index -->`), and the gates and exit-42 causes in `docs/reference/exit-codes.md` (`<!-- wikitool:reference-exit-codes -->`).
|
||||
- Also, in the origin repo only, the skill catalogue `docs/reference/skills.md` (`<!-- wikitool:reference-skills -->`: one row per shipped `instructions/<name>/SKILL.md`, its description split at the sentence starting "Use when") and the page types `docs/reference/page-types.md` (`<!-- wikitool:reference-page-types -->`: one section per `root: kb` type-spec, from what `types describe` reads). Both describe the stack's shipped defaults. In an instance - a tree with `.wikitool-release.json` - they are left alone and the output says so: an instance's adopted type-specs and own instructions may differ, and rewriting a shipped page there would block the next `dist upgrade`.
|
||||
- Also the shapes of the four hand-written configuration files in `docs/reference/configuration.md`, one region each (`<!-- wikitool:reference-config-telemetry -->`, `-remotes`, `-tasks`, `-upload`) inside the hand-written section on that file, rendered from the records in `chemenu.config_shapes` the loaders read the files with. Those records ship with `tools/`, so these regions are written and checked in an instance as well. Several regions in one page are refreshed one after the other into the same text.
|
||||
- Renders each command's summary, synopsis, options, examples, notes, properties in words, exit codes with their causes and see-also - but not the ON FAILURE reactions or NEVER, which address an agent; `tools/CONTRACT.md` keeps those.
|
||||
- Refreshes the table of contents of each page it writes in the same step.
|
||||
- Never creates a page or places a region: a page's title and one-sentence head are written by hand. A group's page is named after the group - lower case, every run of other characters one hyphen.
|
||||
- Dry-run by default (lists the pages that would change); `--apply` writes.
|
||||
- `docs verify` checks the result stays current.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: yes
|
||||
- Idempotent: yes
|
||||
- Atomic: `--apply` rewrites each page in place, one at a time and idempotently, so a re-run after an interruption converges; the dry-run form is read-only
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - A reference page is missing, or lacks its region's marker pair
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool docs contract` - the same records, rendered for an agent into `tools/CONTRACT.md`
|
||||
- `wikitool docs verify` - checks every reference region is current
|
||||
- `tools/README.md` § Adding a command - how a command gets its record
|
||||
<!-- /wikitool:reference-commands -->
|
||||
@@ -0,0 +1,216 @@
|
||||
# Workshop runs and session budget
|
||||
|
||||
Opening and closing a workshop run under `work/`, and inspecting the per-session call budget.
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
|
||||
- [`work new`](#work-new)
|
||||
- [`work close`](#work-close)
|
||||
- [`budget status`](#budget-status)
|
||||
- [`budget reset`](#budget-reset)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
<!-- wikitool:reference-commands -->
|
||||
## `work new`
|
||||
|
||||
Scaffold `work/<runkey>/` for one workshop run.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool work new (--input <raw path> | --key <run key>) [--again] [--dry-run]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
--input <str> Path to the raw source tree or file this run covers, e.g.
|
||||
raw/documents/handbook
|
||||
--key <str> Explicit run key for a run with no raw input (a migration, a
|
||||
sweep across kb/). Mutually exclusive with --input; may not
|
||||
start with 'ingest-'.
|
||||
--again This is a deliberate re-ingest of a tree already processed
|
||||
before: append today's date to the run key instead of refusing
|
||||
the collision.
|
||||
--dry-run Show what would be created, write nothing
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool work new --input raw/2026/09/handbuch
|
||||
tools/wikitool work new --key migrate-7.0.0
|
||||
tools/wikitool work new --input raw/2026/09/handbuch --again
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Creates `work/<runkey>/` with the required `README.md` and `plan.md`.
|
||||
- `--input` derives the run key from the path below `raw/` (an ingest); `--key` names it outright for a run with no raw input - a migration or a sweep across `kb/` - and may not start with `ingest-`, which stays reserved for derived keys. Exactly one of the two.
|
||||
- Refuses a collision instead of suffixing it.
|
||||
- `--again` opens a dated second pass (`<runkey>-<date>`) over a tree that has itself changed.
|
||||
- `--dry-run` reports the run key and files without writing.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: yes
|
||||
- Idempotent: no
|
||||
- Atomic: Yes - one directory with two files
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - Neither or both of `--input`/`--key` given, or a `--key` that is empty or starts with `ingest-`
|
||||
- `1` - `--input` is outside `raw/`, does not exist, or is `raw/` itself (an empty run key)
|
||||
- `1` - The workshop already exists
|
||||
|
||||
**See also**
|
||||
|
||||
- `work/CONTRACT.md` - run keys, required files, how a run closes
|
||||
- `wikitool work close` - deletes the run when it is done
|
||||
- `instructions/ingest-large-tree.md` - the ingest that opens a run
|
||||
|
||||
## `work close`
|
||||
|
||||
Delete a finished workshop.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool work close --run-key <name> [--yes] [--dry-run]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
--run-key <str> The workshop directory name [required]
|
||||
-y, --yes Confirm deletion. Required: closing discards the only copy of
|
||||
the run's working notes, so the durable conclusions must
|
||||
already be in kb/.
|
||||
--dry-run Show what would be deleted, delete nothing
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool work close --run-key ingest-2026-09-handbuch --dry-run
|
||||
tools/wikitool work close --run-key ingest-2026-09-handbuch --yes
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Deletes `work/<run-key>/` recursively.
|
||||
- Without `--yes` it lists what would be lost and refuses: nothing in a workshop is recoverable from the rest of the repo, so the durable conclusions must already be in `kb/`.
|
||||
- `--dry-run` lists the files without deleting.
|
||||
- Not idempotent: once deleted, the run key is unknown.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: yes
|
||||
- Idempotent: no
|
||||
- Atomic: No - a recursive delete
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - Unknown run key
|
||||
- `1` - `--yes` was not passed - the output lists what would be lost
|
||||
|
||||
**See also**
|
||||
|
||||
- `work/CONTRACT.md` - how a run closes
|
||||
- `wikitool work new` - opens a run
|
||||
|
||||
## `budget status`
|
||||
|
||||
Show the current session's `wikitool` call count and recent command history.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool budget status`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool budget status
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Shows the current session's `wikitool` call count and its recent command history.
|
||||
- Never counted against the budget; never fails; safe to retry freely.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: no
|
||||
- Idempotent: yes
|
||||
- Atomic: Read-only
|
||||
- Counts toward the session call budget: no
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
|
||||
**See also**
|
||||
|
||||
- `instructions/session-setup.md` - scoping the budget to a task
|
||||
- `instructions/gates.md` - what to do when the Iteration Budget Gate refuses
|
||||
|
||||
## `budget reset`
|
||||
|
||||
Clear the current session's (or every session's) iteration budget state.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool budget reset --yes [--all]`
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
--all Clear every session's budget, not just the current one.
|
||||
-y, --yes Confirm clearing the budget. Only pass this after a human has
|
||||
approved continuing the session - never set it automatically to
|
||||
work around the gate.
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool budget reset --yes # only after the user approved it
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Clears the current session's iteration budget state; `--all` deletes every session's.
|
||||
- Requires `--yes`: clearing the counter is itself a way around the Iteration Budget Gate, so it needs the same explicit human approval.
|
||||
- Counted against the budget like any other call.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: yes
|
||||
- Idempotent: no
|
||||
- Atomic: Read/rewrite of one JSON file (or its deletion, with `--all`)
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - `--yes` not passed
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool budget status` - the current count
|
||||
- `instructions/gates.md` - why `budget reset` is not the escape hatch
|
||||
<!-- /wikitool:reference-commands -->
|
||||
@@ -0,0 +1,332 @@
|
||||
# Configuration
|
||||
|
||||
The environment variables `wikitool` reads, and the per-checkout files an operator may create at the
|
||||
root of an instance. None of them is needed for an instance to work: an instance installed from a
|
||||
release runs against the origin's public release feed with no configuration at all.
|
||||
|
||||
The shape of each file - its fields, what an absent or a malformed file means, an example - is
|
||||
generated from the record `wikitool` itself reads the file with, so it cannot describe a field the
|
||||
tool does not take. What a setting does around it is written by hand.
|
||||
|
||||
Three more files at the root are written by `wikitool` and never by hand: `.wikitool-release.json`,
|
||||
`.wikitool-kb.json` and `.wikitool-tools.json` ([AGENTS.md invariant 1](../../AGENTS.md#invariants)).
|
||||
The last one has its own section [below](#tool-paths), because the preflight that writes it is
|
||||
something you run.
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
|
||||
- [Environment variables](#environment-variables)
|
||||
- [Telemetry](#telemetry)
|
||||
- [Tool paths](#tool-paths)
|
||||
- [Allowed push URLs](#allowed-push-urls)
|
||||
- [Task tracker](#task-tracker)
|
||||
- [Super Productivity](#super-productivity)
|
||||
- [CalDAV](#caldav)
|
||||
- [MCP submissions](#mcp-submissions)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
## Environment variables
|
||||
|
||||
| Variable | Purpose | Fallback |
|
||||
|----------|---------|----------|
|
||||
| `WIKI_AUTHOR` | Overrides the author name of new source pages | `git config user.name`. Without either, `new` stops with an `ERROR` |
|
||||
| `WIKITOOL_SESSION_ID` | Scopes the Iteration Budget Gate to a task rather than a terminal, and names the directory a trace lands in | A session variable the harness sets itself, where one is known (for example `CLAUDE_CODE_SESSION_ID`), else the parent process id ([instructions/session-setup.md](../../instructions/session-setup.md)) |
|
||||
| `WIKITOOL_UPDATE_URL` | The release feed `version check`, `version notes` and `dist upgrade --latest` ask | The value in `.wikitool-release.json`, else the origin's feed |
|
||||
| `WIKITOOL_UPDATE_TOKEN` | A Gitea token for the release feed, sent only to downloads on the feed's own host | None. Not needed against the origin, only against a feed in a private repository |
|
||||
| `WIKITOOL_TASKS_CONFIG` | Path to a tracker configuration that `task`, `review` and `doctor` read instead of `.wikitool-tasks.json`, for running one checkout against several trackers in turn | `.wikitool-tasks.json` at the root. A path to a file that does not exist is an error, never "no tracker configured" |
|
||||
| `CHEMENU_ROOT` | Which corpus the package works on, for a caller outside the checkout itself | The checkout the package sits in. `tools/wikitool` behaves the same without it |
|
||||
| `WIKI_TRACE` | `0`/`1` turns telemetry off or on, beating both the default and `.wikitool-telemetry.json` | The installation-form default ([below](#telemetry)) |
|
||||
| `WIKI_TRACE_DIR` | Writes traces somewhere other than `reports/telemetry/` | `reports/telemetry/` |
|
||||
| `WIKI_TRACE_CONTENT` | `0` keeps only lengths and digests of prompts and replies instead of their text | Text is kept |
|
||||
| `WIKI_TRACE_MAX_CONTENT` | Per-attribute content cap in characters | 60 KiB |
|
||||
| `WIKI_TRACE_MAX_SESSION_BYTES` | Byte cap per session trace; beats the file's `max_session_bytes` | 5 MiB |
|
||||
| `WIKI_TRACE_KEEP_SESSIONS` | How many session directories retention keeps; beats the file's `keep_sessions` | 250 |
|
||||
|
||||
## Telemetry
|
||||
|
||||
**Telemetry is off in an instance.** Every instance carries the `.wikitool-release.json` of its
|
||||
release, and that is how `chemenu.telemetry.policy` recognises that nobody asked for telemetry. The
|
||||
same holds for every further clone of the instance, because the file is committed with the first
|
||||
publish. Only a clone of the origin repository, used to develop the stack, has no stamp. There the
|
||||
traces are the instrument the stack measures itself with, and they are **on**.
|
||||
|
||||
To turn the default around, create `.wikitool-telemetry.json` at the root. It is per checkout and
|
||||
gitignored, with no `.template`, like `.wikitool-remotes.json`.
|
||||
|
||||
<!-- wikitool:reference-config-telemetry -->
|
||||
| Field | Type | Required | Default | Description |
|
||||
|-------|------|----------|---------|-------------|
|
||||
| `schema` | any | no | - | Accepted and not read. Conventionally `1`; no version of this file is told apart by it yet. |
|
||||
| `enabled` | boolean | no | the installation-form default | Telemetry on or off. `WIKI_TRACE` beats it in both directions. Any other value is read by its truthiness, so `"no"` means on. |
|
||||
| `max_session_bytes` | positive integer | no | `5242880` | Byte cap per session trace. `WIKI_TRACE_MAX_SESSION_BYTES` beats it. A number written as a string is accepted. |
|
||||
| `keep_sessions` | positive integer | no | `250` | How many session directories retention keeps. `WIKI_TRACE_KEEP_SESSIONS` beats it. A number written as a string is accepted. |
|
||||
|
||||
- **Absent:** The installation-form default applies: on in a clone of the origin repository, off in an instance.
|
||||
- **Malformed:** Never an error. Unreadable JSON or a non-object counts as an absent file, and a cap that is not a positive integer as an absent cap. `doctor` reports each such deviation as a `WARN` under `telemetry`, naming the field.
|
||||
- **Unknown keys:** Ignored; `doctor` reports them as a `WARN`.
|
||||
|
||||
Example:
|
||||
|
||||
```json
|
||||
{
|
||||
"enabled": true,
|
||||
"max_session_bytes": 5242880,
|
||||
"keep_sessions": 250
|
||||
}
|
||||
```
|
||||
<!-- /wikitool:reference-config-telemetry -->
|
||||
|
||||
`wikitool doctor` reports the current state: on or off, why, and how much has been recorded against
|
||||
both caps. What the caps do and what never reaches a trace:
|
||||
[telemetry-and-evaluation.md](../explanation/telemetry-and-evaluation.md).
|
||||
|
||||
## Tool paths
|
||||
|
||||
**The preflight writes this file, not you.** `.wikitool-tools.json` at the root holds the absolute
|
||||
paths of Python, git and ripgrep, as the preflight found them on this machine. `wikitool` starts git
|
||||
and rg from there instead of from `PATH`. It is per checkout and gitignored, because a path on one
|
||||
machine says nothing about the next.
|
||||
|
||||
If the file is missing or incomplete, `tools/wikitool` does not start (exit 42) and names the
|
||||
preflight. If a tool sits somewhere the preflight does not look, name it:
|
||||
`tools/preflight.sh --set rg=<path>`, or in PowerShell
|
||||
`pwsh -NoProfile -ExecutionPolicy Bypass -File tools/preflight.ps1 --set rg=<path>`. Never edit the
|
||||
file by hand. `doctor` reports under `tool-paths` whether every path still works.
|
||||
|
||||
## Allowed push URLs
|
||||
|
||||
`.wikitool-remotes.json` at the root restricts where `publish` may push, per checkout and
|
||||
gitignored.
|
||||
|
||||
<!-- wikitool:reference-config-remotes -->
|
||||
| Field | Type | Required | Default | Description |
|
||||
|-------|------|----------|---------|-------------|
|
||||
| `schema` | any | no | - | Accepted and not read. Conventionally `1`; no version of this file is told apart by it yet. |
|
||||
| `allowed_push_urls` | array of strings | yes | - | The push URLs `publish` may write to, compared with the URL the remote resolves to for a push - not with its name. An empty list allows no push at all. |
|
||||
|
||||
- **Absent:** Any remote is allowed; the Publish-Remote Gate does not apply.
|
||||
- **Malformed:** `publish` stops with an error before committing anything. A broken file is never read as "no restriction".
|
||||
- **Unknown keys:** Ignored.
|
||||
|
||||
Example:
|
||||
|
||||
```json
|
||||
{
|
||||
"schema": 1,
|
||||
"allowed_push_urls": [
|
||||
"<url>"
|
||||
]
|
||||
}
|
||||
```
|
||||
<!-- /wikitool:reference-config-remotes -->
|
||||
|
||||
With the file, `publish` refuses with exit 42 (the Publish-Remote Gate) before pushing to a URL not
|
||||
on the list. Only you edit this file. The gate has no token and no flag, and an agent that edits the
|
||||
list to get past a refusal is opening a gate on its own initiative.
|
||||
|
||||
## Task tracker
|
||||
|
||||
**Optional.** The weekly review (`tools/wikitool review`, skill `gtd-weekly-review`) checks the
|
||||
project pages under `kb/gtd/` against a task tracker. Which tracker that is goes in
|
||||
`.wikitool-tasks.json` at the root. Like the telemetry and remotes files, it is per checkout and has
|
||||
no `.template`, and it is **gitignored as soon as a token is in it**.
|
||||
|
||||
<!-- wikitool:reference-config-tasks -->
|
||||
| Field | Type | Required | Default | Description |
|
||||
|-------|------|----------|---------|-------------|
|
||||
| `schema` | any | no | - | Accepted and not read. Conventionally `1`; no version of this file is told apart by it yet. |
|
||||
| `provider` | string | yes | - | Which adapter the review goes through. The provider's own settings sit in a section of the same name. |
|
||||
| `thresholds` | object | yes | - | The weekly review's three thresholds. |
|
||||
| `thresholds.stalled_waiting_days` | positive integer | yes | - | A waiting-for item older than this many days counts as overdue. A number written as a string is accepted. |
|
||||
| `thresholds.unpaged_project_weeks` | positive integer | yes | - | A tracker project with no `kb/` page is reported once it is older than this many weeks. A number written as a string is accepted. |
|
||||
| `thresholds.someday_stale_months` | positive integer | yes | - | A someday/maybe item untouched for this many months counts as stale. A number written as a string is accepted. |
|
||||
|
||||
- **Absent:** No tracker is configured - a valid state. The weekly review has nothing to check against.
|
||||
- **Malformed:** An error, and `doctor` reports `FAIL`. A broken file is never read as "no tracker configured".
|
||||
- **Unknown keys:** Ignored, so the section of a provider not chosen right now can stay in the file.
|
||||
|
||||
**The `superproductivity` section.** Connection settings for the Super Productivity adapter.
|
||||
|
||||
| Field | Type | Required | Default | Description |
|
||||
|-------|------|----------|---------|-------------|
|
||||
| `access` | string | yes | - | Which way in: `api`, the local REST API (reads the current state and carries the write path), or `snapshot`, the newest backup snapshot (read-only). |
|
||||
|
||||
With `access: "api"`:
|
||||
|
||||
| Field | Type | Required | Default | Description |
|
||||
|-------|------|----------|---------|-------------|
|
||||
| `api_base_url` | string | no | `"http://127.0.0.1:3876"` | Where the app's REST API listens. An empty value means the default. |
|
||||
| `api_token` | string | yes, not empty | - | The token from the app's settings. |
|
||||
|
||||
With `access: "snapshot"`:
|
||||
|
||||
| Field | Type | Required | Default | Description |
|
||||
|-------|------|----------|---------|-------------|
|
||||
| `backups_dir` | string | yes, not empty | - | The directory the app writes its backup snapshots to; `~` is expanded. |
|
||||
|
||||
- **Unknown keys:** Rejected: a section names only one access path's own fields, so a `backups_dir` beside `access: "api"` is an error, not ignored.
|
||||
|
||||
```json
|
||||
{
|
||||
"superproductivity": {
|
||||
"access": "api",
|
||||
"api_base_url": "http://127.0.0.1:3876",
|
||||
"api_token": "..."
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"superproductivity": {
|
||||
"access": "snapshot",
|
||||
"backups_dir": "~/.config/superProductivity/backups"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**The `caldav` section.** Connection settings for the CalDAV adapter.
|
||||
|
||||
| Field | Type | Required | Default | Description |
|
||||
|-------|------|----------|---------|-------------|
|
||||
| `url` | string | yes, not empty | - | The account's CalDAV calendar home set. A trailing `/` is added when missing. |
|
||||
| `username` | string | yes, not empty | - | The login name, which can differ from the user id in the URL. |
|
||||
| `app_password` | string | yes, not empty | - | An app password is recommended over the account password. |
|
||||
| `inbox_list` | string | yes, not empty | - | The name of the one inbox list. |
|
||||
| `someday_list` | string | yes, not empty | - | The name of the one someday/maybe list. |
|
||||
| `exclude_lists` | array of strings | no | `[]` | Existing plain task lists that are not projects. |
|
||||
|
||||
- **Unknown keys:** Rejected.
|
||||
|
||||
```json
|
||||
{
|
||||
"caldav": {
|
||||
"url": "https://<host>/remote.php/dav/calendars/<user>/",
|
||||
"username": "<login>",
|
||||
"app_password": "...",
|
||||
"inbox_list": "Inbox",
|
||||
"someday_list": "Someday",
|
||||
"exclude_lists": [
|
||||
"..."
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
A complete example:
|
||||
|
||||
```json
|
||||
{
|
||||
"schema": 1,
|
||||
"provider": "superproductivity",
|
||||
"thresholds": {
|
||||
"stalled_waiting_days": 14,
|
||||
"unpaged_project_weeks": 3,
|
||||
"someday_stale_months": 5
|
||||
},
|
||||
"superproductivity": {
|
||||
"access": "api",
|
||||
"api_base_url": "http://127.0.0.1:3876",
|
||||
"api_token": "..."
|
||||
}
|
||||
}
|
||||
```
|
||||
<!-- /wikitool:reference-config-tasks -->
|
||||
|
||||
The `thresholds` block is configuration, not schema. A value the provider cannot supply at all - a
|
||||
waiting item with no follow-up date, a tracker project with no creation date - is not skipped silently
|
||||
but reported as its own finding (`waiting_no_follow_up`/`project_age_unknown`).
|
||||
|
||||
### Super Productivity
|
||||
|
||||
`access` is **mandatory and has no fallback**. It chooses between two mutually exclusive ways in:
|
||||
|
||||
- **`access: "snapshot"`** for an instance run headless: it reads only the newest backup snapshot
|
||||
under `backups_dir`. That works without the app running, but only reads. The tracker cannot be
|
||||
written from there.
|
||||
|
||||
- **`access: "api"`** for a desktop instance: it talks only to the local REST API. That answers only
|
||||
while the app runs, but it gives the current state and carries the write path. `api_token` is
|
||||
required, because every endpoint except `GET /health` demands `Authorization: Bearer <token>`.
|
||||
|
||||
What can write:
|
||||
|
||||
- `tools/wikitool new project` creates a tracker project of the same name only with `access: "api"`,
|
||||
and even there not automatically. The command stops (exit 42) and asks a human to create it, then
|
||||
continues with `--resume`.
|
||||
- `tools/wikitool task new` creates a single item without a `kb/` page.
|
||||
- `tools/wikitool task close --id` marks an item done. It never deletes one.
|
||||
|
||||
With `access: "snapshot"`, all three refuse completely with exit 1, because the tracker is read-only
|
||||
from there. `tools/wikitool task list --project` only reads, so it answers with either `access`.
|
||||
|
||||
### CalDAV
|
||||
|
||||
`caldav` is the standards-based second adapter (RFC 4791/5545). It is verified against Nextcloud
|
||||
Tasks, with iOS *Reminders* as the mobile client, and built for that split: tick items off on the
|
||||
phone, maintain them at the desk. Unlike Super Productivity, there is only one way in. CalDAV is
|
||||
always a network access, so no `access` field is needed.
|
||||
|
||||
- `url` may be an alias - Nextcloud accepts a short name there - because every later address comes
|
||||
only from the `href`s the server returns, never from this URL joined with a name.
|
||||
- `inbox_list` and `someday_list` name the two fixed lists, exactly one of each per instance.
|
||||
- A list with a `VEVENT` component never counts as a project, so `exclude_lists` is needed only for
|
||||
plain task lists.
|
||||
|
||||
A project is a list whose only supported component is `VTODO`. `tools/wikitool new project` creates
|
||||
it automatically with `MKCALENDAR` - without asking, unlike Super Productivity, because CalDAV has a
|
||||
real create command for lists. The uniqueness check runs against **every** list in the account,
|
||||
including excluded lists, the inbox, someday and mixed calendars. If a new name collides with any of
|
||||
them, nothing is created and the collision is named. Renaming the existing list in Nextcloud stays a
|
||||
manual step.
|
||||
|
||||
`task new` and `task close` work without restriction on `caldav`, which has no read-only mode like
|
||||
`access: "snapshot"`. When ticking an item off, `task close` changes only `STATUS`, `COMPLETED`,
|
||||
`PERCENT-COMPLETE`, `LAST-MODIFIED` and `DTSTAMP` on the existing `.ics` resource. Every other
|
||||
property, including an unknown `X-` property or an alarm, is kept unchanged. A resource that changed
|
||||
since it was read (an ETag conflict) is not written, and the command stops with exit 1. Nothing is
|
||||
ever deleted. The lists of finished projects stay after archiving, because the adapter never deletes
|
||||
a list. Remove them by hand in Nextcloud when you want to.
|
||||
|
||||
## MCP submissions
|
||||
|
||||
`.wikitool-upload.json` in the served corpus turns on the MCP read server's optional `submit` tool.
|
||||
What the authentication in front of the server has to guarantee before you create it is in
|
||||
[Set up the MCP read server](../how-to/set-up-the-mcp-server.md#optional-the-submit-path).
|
||||
|
||||
<!-- wikitool:reference-config-upload -->
|
||||
| Field | Type | Required | Default | Description |
|
||||
|-------|------|----------|---------|-------------|
|
||||
| `schema` | any | no | - | Accepted and not read. Conventionally `1`; no version of this file is told apart by it yet. |
|
||||
| `identity_header` | string | no | `"X-Forwarded-User"` | The request header the authenticating middleware sets to the submitter's identity on every request. An empty value means the default. |
|
||||
| `max_bytes` | positive integer | yes | - | Largest single submission, in bytes. A number written as a string is accepted. |
|
||||
| `allowed_extensions` | array of strings | yes | - | The file extensions `submit` accepts, each starting with `.`. Compared case-insensitively. Entries are read as strings and lower-cased; duplicates collapse. |
|
||||
| `quota` | object | yes | - | Per-submitter limits over any rolling 24 hours. |
|
||||
| `quota.submissions_per_day` | positive integer | yes | - | Submissions one submitter may make. A number written as a string is accepted. |
|
||||
| `quota.bytes_per_day` | positive integer | yes | - | Bytes one submitter may submit. A number written as a string is accepted. |
|
||||
|
||||
- **Absent:** The `submit` tool is not registered: the write path does not exist.
|
||||
- **Malformed:** An error. The MCP server does not start and `doctor` reports `FAIL`. A broken file is never read as "no limits".
|
||||
- **Unknown keys:** Ignored.
|
||||
|
||||
Example:
|
||||
|
||||
```json
|
||||
{
|
||||
"schema": 1,
|
||||
"identity_header": "X-Forwarded-User",
|
||||
"max_bytes": 10485760,
|
||||
"allowed_extensions": [
|
||||
".md",
|
||||
".pdf"
|
||||
],
|
||||
"quota": {
|
||||
"submissions_per_day": 20,
|
||||
"bytes_per_day": 52428800
|
||||
}
|
||||
}
|
||||
```
|
||||
<!-- /wikitool:reference-config-upload -->
|
||||
|
||||
Size and quota are your deployment's decision, not defaults worth copying blindly.
|
||||
@@ -0,0 +1,41 @@
|
||||
# Exit codes and gates
|
||||
|
||||
Which `wikitool` commands can stop and wait for a person - exit 42 - and for which gate, generated
|
||||
from the command records.
|
||||
|
||||
<!-- wikitool:reference-exit-codes -->
|
||||
What each exit code means, and how a caller reacts to it, is [AGENTS.md § Tool error contract](../../AGENTS.md#tool-error-contract). What a gate is and why it is enforced in code is [AGENTS.md § Gates](../../AGENTS.md#gates). This page lists which commands declare a gate, and every cause for which a command exits 42.
|
||||
|
||||
## Gates by command
|
||||
|
||||
### guideline-push
|
||||
|
||||
- [`export guidelines`](commands/guideline-export.md#export-guidelines)
|
||||
|
||||
### mass-update
|
||||
|
||||
- [`publish`](commands/git.md#publish)
|
||||
|
||||
### publish-remote
|
||||
|
||||
- [`publish`](commands/git.md#publish)
|
||||
|
||||
### rebase-review
|
||||
|
||||
- [`sync`](commands/git.md#sync)
|
||||
- [`publish`](commands/git.md#publish)
|
||||
|
||||
### upload-review
|
||||
|
||||
- [`upload accept`](commands/raw-material-and-uploads.md#upload-accept)
|
||||
|
||||
## Exit 42 causes
|
||||
|
||||
- [`new`](commands/pages.md#new): new project: The provider cannot create the project itself (Super Productivity's `access: "api"`); the output says what a human has to create
|
||||
- [`upload accept`](commands/raw-material-and-uploads.md#upload-accept): Upload Review Gate: `--confirm` is absent or does not match the manifest's current token
|
||||
- [`sync`](commands/git.md#sync): Rebase-Review Gate: `<remote>/<branch>` moved and both sides changed the same file, generated files aside; the output lists the upstream commits, the overlapping files and their diff
|
||||
- [`publish`](commands/git.md#publish): Mass-Update Gate: `--threshold` (default 10) or more counted files would be committed, or the `--confirm` token does not match this changeset
|
||||
- [`publish`](commands/git.md#publish): Rebase-Review Gate: `<remote>/<branch>` moved and both sides changed the same file; the output lists the upstream commits, the overlapping files and their diff
|
||||
- [`publish`](commands/git.md#publish): Publish-Remote Gate: `.wikitool-remotes.json` exists and the push URL of `--remote` is not listed in it, or `--remote` resolves to no push URL
|
||||
- [`export guidelines`](commands/guideline-export.md#export-guidelines): Guideline Push Gate: `--push` without `--confirm`, or with a token that does not match the current set of targets
|
||||
<!-- /wikitool:reference-exit-codes -->
|
||||
@@ -0,0 +1,138 @@
|
||||
# Page types
|
||||
|
||||
Every page type the stack ships for `kb/`: what it is for, where its pages are written, its
|
||||
subtypes and its required frontmatter fields, generated from the type-specs under `types/` and
|
||||
their schemas.
|
||||
|
||||
This page describes the stack's shipped defaults. An instance adopts these type-specs from their
|
||||
`.template` and may change them, or add types of its own; `tools/wikitool types list` and
|
||||
`tools/wikitool types describe <type>` show what this instance has.
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
|
||||
- [`comparison`](#comparison)
|
||||
- [`concept`](#concept)
|
||||
- [`entity`](#entity)
|
||||
- [`project`](#project)
|
||||
- [`source`](#source)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
<!-- wikitool:reference-page-types -->
|
||||
## `comparison`
|
||||
|
||||
Structured type for comparison pages that set several entities or approaches against one another
|
||||
|
||||
- Written to: `kb/comparisons/`
|
||||
|
||||
**Required fields**
|
||||
|
||||
- `type` - string
|
||||
- `created` - string
|
||||
- `entities` - array
|
||||
- `summary` - string
|
||||
|
||||
## `concept`
|
||||
|
||||
Base type for concept pages - architectures, patterns, protocols, workflows, decisions or problems
|
||||
|
||||
- Written to: `kb/concepts/<subtype directory>/`
|
||||
|
||||
**Subtypes** (field `concept_type`)
|
||||
|
||||
| Value | Written to | Index section |
|
||||
|---|---|---|
|
||||
| `architecture` | `kb/concepts/architectures/` | Architekturen |
|
||||
| `pattern` | `kb/concepts/patterns/` | Muster |
|
||||
| `protocol` | `kb/concepts/protocols/` | Protokolle |
|
||||
| `workflow` | `kb/concepts/workflows/` | Abläufe |
|
||||
| `decision` | `kb/concepts/decisions/` | Entscheidungen |
|
||||
| `problem` | `kb/concepts/problems/` | Problemstellungen |
|
||||
|
||||
**Required fields**
|
||||
|
||||
- `type` - string
|
||||
- `concept_type` - string, one of `architecture`, `pattern`, `protocol`, `workflow`, `decision`, `problem`
|
||||
- `created` - string
|
||||
- `modified` - string
|
||||
- `provenance` - string, one of `sourced`, `general`, `mixed`
|
||||
- `summary` - string
|
||||
|
||||
## `entity`
|
||||
|
||||
Base type for entity pages - codebases, systems, tools, technologies, people or organizations
|
||||
|
||||
- Written to: `kb/entities/<subtype directory>/`
|
||||
|
||||
**Subtypes** (field `entity_type`)
|
||||
|
||||
| Value | Written to | Index section |
|
||||
|---|---|---|
|
||||
| `codebase` | `kb/entities/codebases/` | Codebasen |
|
||||
| `system` | `kb/entities/systems/` | Systeme |
|
||||
| `tool` | `kb/entities/tools/` | Werkzeuge |
|
||||
| `technology` | `kb/entities/technologies/` | Technologien |
|
||||
| `person` | `kb/entities/people/` | Personen |
|
||||
| `organization` | `kb/entities/organizations/` | Organisationen |
|
||||
|
||||
**Required fields**
|
||||
|
||||
- `type` - string
|
||||
- `entity_type` - string, one of `codebase`, `system`, `tool`, `technology`, `person`, `organization`
|
||||
- `created` - string
|
||||
- `modified` - string
|
||||
- `provenance` - string, one of `sourced`, `general`, `mixed`
|
||||
- `summary` - string
|
||||
|
||||
## `project`
|
||||
|
||||
Structured type for a committed initiative (a GTD project) - goal, participants, durable status, open loops
|
||||
|
||||
- Written to: `kb/gtd/<subtype directory>/`
|
||||
|
||||
**Subtypes** (field `responsibility`)
|
||||
|
||||
| Value | Written to | Index section |
|
||||
|---|---|---|
|
||||
| `haus` | `kb/gtd/haus/` | Haus |
|
||||
| `finanzen` | `kb/gtd/finanzen/` | Finanzen |
|
||||
| `technik` | `kb/gtd/technik/` | Technik |
|
||||
|
||||
**Required fields**
|
||||
|
||||
- `type` - string
|
||||
- `state` - string, one of `active`, `dormant`, `completed`, `abandoned`
|
||||
- `responsibility` - string, one of `haus`, `finanzen`, `technik`
|
||||
- `created` - string
|
||||
- `modified` - string
|
||||
- `provenance` - string, one of `sourced`, `general`, `mixed`
|
||||
- `summary` - string
|
||||
|
||||
## `source`
|
||||
|
||||
Structured type for source pages that record and summarize ingested raw material
|
||||
|
||||
- Written to: `kb/sources/<subtype directory>/`
|
||||
- Title prefix: `Source - `
|
||||
|
||||
**Subtypes** (field `source_type`)
|
||||
|
||||
| Value | Written to | Index section |
|
||||
|---|---|---|
|
||||
| `transcript` | `kb/sources/transcripts/` | Transkripte |
|
||||
| `analysis` | `kb/sources/analyses/` | Analysen |
|
||||
| `article` | `kb/sources/articles/` | Artikel |
|
||||
| `document` | `kb/sources/documents/` | Dokumente |
|
||||
| `notes` | `kb/sources/notes/` | Notizen |
|
||||
| `tracker` | `kb/sources/trackers/` | Tracker-Exporte |
|
||||
| `unclassified` | `kb/sources/unclassified/` | Unklassifiziert |
|
||||
|
||||
**Required fields**
|
||||
|
||||
- `type` - string
|
||||
- `source_type` - string, one of `transcript`, `analysis`, `article`, `document`, `notes`, `tracker`, `unclassified`
|
||||
- `author` - string
|
||||
- `raw_files` - array
|
||||
- `date` - string
|
||||
- `summary` - string
|
||||
<!-- /wikitool:reference-page-types -->
|
||||
@@ -0,0 +1,19 @@
|
||||
# Skill catalogue
|
||||
|
||||
The skills the stack ships, as an agent's harness sees them once `tools/wikitool instructions sync`
|
||||
has published them: each one's name, what it does and the requests it answers, generated from its
|
||||
`instructions/<name>/SKILL.md`. Any skill also runs when you ask for it by name.
|
||||
|
||||
This page describes the stack's shipped defaults. An instance may add instructions of its own;
|
||||
`tools/wikitool instructions list` shows what this instance has.
|
||||
|
||||
<!-- wikitool:reference-skills -->
|
||||
| Skill | What it does | Use it when |
|
||||
|---|---|---|
|
||||
| `gtd-weekly-review` | Turns the findings from `wikitool review` into decisions and page updates - the GTD Weekly Review, with a machine that prepares the list instead of a human reconstructing it from memory. | the user asks for "the weekly review", "review my projects", "what's stalled", or after `wikitool review` has findings nobody has acted on yet. |
|
||||
| `wiki-ingest` | Processes a new source file into the LLM wiki - extracts entities and concepts, creates a source summary page, files a tracker item for any commitment the source also carries, cross-references, rebuilds indexes, and publishes. | the user drops a file or folder into incoming/ or raw/, names a URL to ingest, or says "ingest <file>", "ingest <url>", "process this source", "add this to the wiki" - or just "ingest" with nothing named, which takes the oldest entry waiting in incoming/ - or asks to "update the captured repositories" / "pull the repo docs", which takes the next bundle `raw status` reports as changed. |
|
||||
| `wiki-lint` | Checks the health of the LLM wiki - broken links, orphan pages, uncovered raw files, stale claims, duplicated rules, missing cross-references. | the user says "lint the wiki", "health-check the wiki", or periodically every 10 sources per the Maintenance Schedule. |
|
||||
| `wiki-manage` | Creates a new wiki page (entity, concept, source, comparison) or updates an existing page with new information, including cross-references, index/log, and publishing. | the user says "create a new entity/concept/comparison", "add a page for X", "update the X page", or new information needs integrating into an existing page. |
|
||||
| `wiki-query` | Answers a question using the LLM wiki's compiled knowledge - read-only, cites sources, can file a valuable answer back as a new page. | the user asks a question about entities, projects, concepts, or anything the wiki might know, or says "query the wiki", "what do we know about X", "search the wiki". |
|
||||
| `wiki-status` | Shows a quick read-only snapshot of the LLM wiki - page counts, orphan pages, uncovered raw files, recent activity. | the user says "wiki status", "show wiki statistics", "what's new", or wants a quick health snapshot without running a full lint. |
|
||||
<!-- /wikitool:reference-skills -->
|
||||
@@ -0,0 +1,13 @@
|
||||
# Tutorials
|
||||
|
||||
Learning-oriented pages: a guided first walk through an instance, start to finish, for someone
|
||||
who has not used one before. A tutorial shows one path that works; it does not list options or
|
||||
explain why.
|
||||
|
||||
Take them in this order:
|
||||
|
||||
1. [Install a Chemenu instance](../../INSTALL.md) - from an empty folder to a running instance.
|
||||
It stays at the repository root, where someone who has nothing else yet finds it first.
|
||||
2. [From your first source to your first answer](first-source-to-first-answer.md) - ingest a
|
||||
sample document, look at the pages it became, and ask the wiki a question about it.
|
||||
[sample-source.md](sample-source.md) is the document it uses.
|
||||
@@ -0,0 +1,157 @@
|
||||
# From your first source to your first answer
|
||||
|
||||
In this tutorial you put one document into a fresh instance, let the agent turn it into wiki
|
||||
pages, look at what it wrote, and then ask the wiki a question and follow the answer back to the
|
||||
document. It takes about a quarter of an hour. You need nothing but the instance and your harness:
|
||||
the document is a sample that ships with the instance.
|
||||
|
||||
You start from an instance that [INSTALL.md](../../INSTALL.md) has just set up, with the agent
|
||||
session restarted afterwards so that it has loaded its skills. Ask the agent `Which skills do you
|
||||
have?` if you are unsure - `wiki-ingest` and `wiki-query` should be among them.
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
|
||||
- [1. Put the sample source into `incoming/`](#1-put-the-sample-source-into-incoming)
|
||||
- [2. Ask the agent to ingest it](#2-ask-the-agent-to-ingest-it)
|
||||
- [3. Tell the agent what matters](#3-tell-the-agent-what-matters)
|
||||
- [4. Answer the two capture questions](#4-answer-the-two-capture-questions)
|
||||
- [5. Let the agent publish](#5-let-the-agent-publish)
|
||||
- [6. Look at what it wrote](#6-look-at-what-it-wrote)
|
||||
- [7. Ask a question](#7-ask-a-question)
|
||||
- [8. Optional: browse it in Obsidian](#8-optional-browse-it-in-obsidian)
|
||||
- [What next](#what-next)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
## 1. Put the sample source into `incoming/`
|
||||
|
||||
`incoming/` is where everything you want in the wiki waits until it is ingested. Copy the sample
|
||||
there - copy, not move, so the tutorial page keeps its sample:
|
||||
|
||||
```bash
|
||||
cp docs/tutorials/sample-source.md incoming/home-lab-backups.md
|
||||
```
|
||||
|
||||
On Windows in PowerShell 7:
|
||||
|
||||
```powershell
|
||||
Copy-Item docs/tutorials/sample-source.md incoming/home-lab-backups.md
|
||||
```
|
||||
|
||||
Open [the sample](sample-source.md) and read it once. It is a page of notes about the backups of a
|
||||
small home file server called Atlas: which tool makes them (restic), where they go, and why there
|
||||
are two targets (the 3-2-1 backup rule). Knowing what it says will let you check the agent's work.
|
||||
|
||||
## 2. Ask the agent to ingest it
|
||||
|
||||
In the agent session, say:
|
||||
|
||||
```text
|
||||
Ingest
|
||||
```
|
||||
|
||||
A bare `Ingest` takes the oldest entry waiting in `incoming/` - here, the only one. The agent
|
||||
reads the whole file before it writes anything.
|
||||
|
||||
## 3. Tell the agent what matters
|
||||
|
||||
The agent comes back with the key takeaways of the document and a few questions: which points
|
||||
matter most, which things should get a page of their own, and whether the document carries
|
||||
something you have to *do*. Answer in your own words, for example:
|
||||
|
||||
```text
|
||||
Atlas, restic and the 3-2-1 rule are what matter. Nothing to do from this one.
|
||||
```
|
||||
|
||||
The pages it writes next follow your answer, so this is the moment to steer. The agent may well
|
||||
notice that the notes leave something untested and offer to track it as a task; "nothing to do"
|
||||
declines that.
|
||||
|
||||
Often the same message already asks the two questions of the next step. Then answer them in the
|
||||
same reply.
|
||||
|
||||
## 4. Answer the two capture questions
|
||||
|
||||
Before the file leaves `incoming/`, the agent asks two things about the capture itself:
|
||||
|
||||
- **`fidelity`** - how faithful is this file to the original? The sample is the notes exactly as
|
||||
written, so the answer is `verbatim`.
|
||||
- **`authority`** - what may the material claim about its subject? The notes describe how a setup
|
||||
works; they do not set a standard anyone else has to follow. The answer is `reporting`.
|
||||
|
||||
```text
|
||||
verbatim, reporting
|
||||
```
|
||||
|
||||
The agent never guesses these two. Only you know them, and only now: once the file is in the wiki,
|
||||
nobody can tell any more where it came from. [raw/CONTRACT.md](../../raw/CONTRACT.md#capture-fields-fidelity-and-authority)
|
||||
lists every value, for the sources you add later.
|
||||
|
||||
## 5. Let the agent publish
|
||||
|
||||
The agent now moves the file into `raw/`, writes the pages, and publishes everything as one
|
||||
commit. For a source this small that goes straight through; the agent tells you what it committed.
|
||||
If your instance has no remote, the commit stays on your machine.
|
||||
|
||||
A larger source - ten or more new or changed pages and files, not counting the generated indexes
|
||||
and log - stops at the Mass-Update Gate instead: you see the list of files and a line ending in
|
||||
`--confirm <token>`, and nothing is committed until you tell the agent to go ahead.
|
||||
|
||||
## 6. Look at what it wrote
|
||||
|
||||
Ask the agent where the pages are, or look yourself. You will find:
|
||||
|
||||
- **The file itself under `raw/`**, in a folder for the current year and month. It is never edited
|
||||
again.
|
||||
- **A source page under `kb/sources/`**, summarising the notes. Its frontmatter carries your two
|
||||
answers, `fidelity: verbatim` and `authority: reporting`, and `raw_files:` points at the file in
|
||||
`raw/`.
|
||||
- **Entity and concept pages** for what you said matters - Atlas, restic, the 3-2-1 backup rule,
|
||||
in the language and naming your instance chose at setup. Each one links back to the source page,
|
||||
and each claim taken from the notes carries a footnote citing it.
|
||||
- **A new entry at the end of `kb/log.md`**, recording the ingest.
|
||||
|
||||
The exact set of pages depends on your answer in step 3, and on the agent: two runs of this
|
||||
tutorial do not produce word-for-word the same wiki. What is always the same is that every page is
|
||||
tied back to the file in `raw/`.
|
||||
|
||||
## 7. Ask a question
|
||||
|
||||
Start a new agent session, so that the answer can only come from the wiki and not from what the
|
||||
agent remembers of the ingest, and ask:
|
||||
|
||||
```text
|
||||
How are the backups of Atlas protected against a fire in the house?
|
||||
```
|
||||
|
||||
The answer names the storage box outside the house as the copy that survives, and the 3-2-1 rule
|
||||
as the reason it exists. It cites the pages it came from and the source behind them. Open one of
|
||||
the cited pages and follow its footnote: it leads to the source page, and that one to the file in
|
||||
`raw/` - the notes you copied in step 1.
|
||||
|
||||
Now ask something the notes do not answer:
|
||||
|
||||
```text
|
||||
How long does a full restore of Atlas take?
|
||||
```
|
||||
|
||||
The notes say only that one has never been tried. The agent tells you that the wiki has no source
|
||||
for this, rather than estimating one. That is the rule behind every answer: no claim without a
|
||||
source.
|
||||
|
||||
## 8. Optional: browse it in Obsidian
|
||||
|
||||
Open the instance's root folder as a vault in [Obsidian](https://obsidian.md). The graph view shows
|
||||
the pages you just made and the links between them. Read there as much as you like, but change
|
||||
pages through the agent: [Browse the wiki in Obsidian](../how-to/browse-with-obsidian.md) explains
|
||||
why.
|
||||
|
||||
## What next
|
||||
|
||||
You have taken one source all the way from a file to a cited answer. From here on, the how-to
|
||||
guides cover the everyday tasks:
|
||||
|
||||
- [Ingest a source](../how-to/ingest-a-source.md) - your own files, folders and web pages
|
||||
- [Query the wiki](../how-to/query-the-wiki.md) - and search it yourself
|
||||
- [Maintain the wiki](../how-to/maintain-the-wiki.md) - the lint the agent suggests every ten
|
||||
sources
|
||||
@@ -0,0 +1,48 @@
|
||||
# Home lab backups - how they work now
|
||||
|
||||
Notes written after reworking the backups in September 2026. This file is the sample source for
|
||||
the tutorial "From your first source to your first answer"; the machine and its setup are made
|
||||
up for it.
|
||||
|
||||
## The machine
|
||||
|
||||
**Atlas** is the small file server in the hall cupboard: a fanless mini PC with two 4 TB disks
|
||||
in a mirror, running Debian 13. It holds the family photo archive, scanned documents and the
|
||||
configuration of every other machine in the house. Everything else on the network is replaceable;
|
||||
what is on Atlas is not.
|
||||
|
||||
## The tool
|
||||
|
||||
Backups are made with **restic**, an open-source backup program. restic splits files into chunks,
|
||||
stores each chunk only once and encrypts everything before it leaves the machine, so the backup
|
||||
target never sees a file name or a byte of content in the clear. A restic *repository* is the
|
||||
storage location it writes to; a *snapshot* is one backup run.
|
||||
|
||||
restic was chosen over plain `rsync` for two reasons: deduplication keeps the daily runs small,
|
||||
and encryption makes it safe to put a copy on storage someone else operates.
|
||||
|
||||
## How it is set up
|
||||
|
||||
Atlas runs restic every night at 02:30 from a systemd timer. Each run writes one snapshot to two
|
||||
repositories:
|
||||
|
||||
- a USB disk that stays plugged into Atlas, for fast restores;
|
||||
- a rented storage box at a hosting provider, reached over SFTP, as the copy outside the house.
|
||||
|
||||
Snapshots are thinned out by `restic forget` after each run: the last 7 daily, 5 weekly and
|
||||
12 monthly snapshots are kept.
|
||||
|
||||
## Why two targets
|
||||
|
||||
The setup follows the **3-2-1 backup rule**: keep three copies of the data, on two different
|
||||
kinds of storage, with one copy off site. Here the three copies are the mirror on Atlas itself,
|
||||
the USB disk, and the storage box. A fire or a burglary takes the first two; the storage box is
|
||||
what is left.
|
||||
|
||||
The mirror alone does not count as a backup. It protects against a dead disk, but a deleted or
|
||||
encrypted file is deleted or encrypted on both disks at once.
|
||||
|
||||
## What is still open
|
||||
|
||||
The restore has been tried once, for a single folder, from the USB disk. It worked and took
|
||||
under a minute. A full restore from the storage box has not been tried.
|
||||
Whitespace-only changes.
+272
-10
@@ -9,6 +9,26 @@ instead of action.
|
||||
`instructions/` is not a pipeline stage and not a collection. It is part of the control plane,
|
||||
alongside [AGENTS.md](../AGENTS.md).
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
|
||||
- [Two forms, three reference tiers](#two-forms-three-reference-tiers)
|
||||
- [`instructions/migrations/`](#instructionsmigrations)
|
||||
- [`instructions/dev/`](#instructionsdev)
|
||||
- [Publishing](#publishing)
|
||||
- [Writing an instruction](#writing-an-instruction)
|
||||
- [A skill's H1 is a name, not an imperative](#a-skills-h1-is-a-name-not-an-imperative)
|
||||
- [A skill's `description` speaks in third person](#a-skills-description-speaks-in-third-person)
|
||||
- [A skill's name declares its family](#a-skills-name-declares-its-family)
|
||||
- [A skill's outbound reference is a plain path, not a link](#a-skills-outbound-reference-is-a-plain-path-not-a-link)
|
||||
- [Reference depth: bundled files, not repo-wide contracts](#reference-depth-bundled-files-not-repo-wide-contracts)
|
||||
- [When a skill carries a copy-in checklist](#when-a-skill-carries-a-copy-in-checklist)
|
||||
- [How much reasoning a step may carry](#how-much-reasoning-a-step-may-carry)
|
||||
- [Instruction duality](#instruction-duality)
|
||||
- [Single source](#single-source)
|
||||
- [What does not belong here](#what-does-not-belong-here)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
## Two forms, three reference tiers
|
||||
|
||||
| Form | File | Loaded by |
|
||||
@@ -58,16 +78,43 @@ whether an instruction is still reachable, which is exactly why the answer means
|
||||
`instructions/dev/` boundary check below asks the opposite question - what would *dangle* in a
|
||||
distributed instance - and does scan README.md, because `dist export` ships it verbatim.
|
||||
|
||||
Two kinds of file use the Manual tier today: [german-terminology.md](german-terminology.md), a
|
||||
vocabulary consulted on demand rather than a procedure, and every migration document (below).
|
||||
Three kinds of file use the Manual tier today: [german-terminology.md](german-terminology.md), a
|
||||
vocabulary consulted on demand rather than a procedure; [kb-profiles.md](kb-profiles.md), the
|
||||
catalogue of authoring profiles an instance may adopt into its own `kb/CONVENTIONS.md` and
|
||||
`COLLECTION.md` files; and every migration document (below).
|
||||
|
||||
## `instructions/migrations/`
|
||||
|
||||
A content migration is a Manual instruction with two extra frontmatter fields
|
||||
A content migration is a Manual instruction with three extra frontmatter fields
|
||||
(`types/instruction.schema.yaml`): `migrates_to:`, the stack version whose content shape it
|
||||
produces, and `migration_kind:` (`mechanical` | `assisted`). It lives at
|
||||
produces; `migration_kind:` (`mechanical` | `assisted`); and `obligation:`
|
||||
(`required` | `offered`, default `required`). It lives at
|
||||
`instructions/migrations/<version>-<slug>.md`.
|
||||
|
||||
`wikitool new instruction` scaffolds none of the three: `migrates_to:` and `migration_kind:`
|
||||
have no schema `default:` at all, and an ordinary instruction's scaffold no longer materializes
|
||||
`obligation:`'s default either - all three are added by hand when a migration document is
|
||||
written, per [migrate-corpus.md](migrate-corpus.md).
|
||||
|
||||
`migration_kind:` and `obligation:` are **two axes, not one**. The first says how the work is
|
||||
carried out, the second whether it has to happen at all:
|
||||
|
||||
| `obligation:` | Means | `migrate status` |
|
||||
|---|---|---|
|
||||
| `required` | The content must reach the new shape or it no longer fits the machinery | Counted as outstanding; `migrate done` advances `kb_version` through it, in chain order |
|
||||
| `offered` | A file the instance owns still works as it is, and the stack proposes a better default | Listed separately, never blocks, no ordering rule. `migrate done` records it in the applied ledger and leaves `kb_version` where it is |
|
||||
|
||||
Keeping them apart is what stops `migrate status` crying wolf: an instance nagged about an
|
||||
improvement it declined stops reading the nag that means its content no longer fits its
|
||||
machinery. And because taking an offer deliberately does not move the version, the **applied
|
||||
ledger** - not `kb_version` - is what makes an offer stop being offered; without that record
|
||||
there is no way to tell a taken offer from an ignored one.
|
||||
|
||||
An `offered` migration is what makes an instance-owned file upgradeable at all. `dist export`
|
||||
records a sha256 per shipped file in `.wikitool-release.json`, so `migrate status` can say which
|
||||
of those files the instance edited and which it merely received - the first have to be
|
||||
reconciled by a person, the second can simply be copied over.
|
||||
|
||||
The tier fits exactly: a migration must never be picked up implicitly - it rewrites the corpus -
|
||||
and it is referenced by nothing, because `tools/wikitool migrate status` finds it by reading the
|
||||
directory and comparing `migrates_to:` against this instance's `kb_version`. That is also why
|
||||
@@ -120,15 +167,229 @@ does not survive being archived or copied. The price of a copy is drift, and dri
|
||||
Scaffold with `tools/wikitool new instruction --name "<name>"`; the contract is
|
||||
`tools/wikitool types describe instruction`.
|
||||
|
||||
- **Imperative title.** It answers "what does this tell me to do?".
|
||||
- **Imperative title.** It answers "what does this tell me to do?". This binds the flat
|
||||
`instructions/<name>.md` form only - a skill's H1 is a different case, below.
|
||||
- **`description` is the retrieval wire.** Write it to match the question an agent would ask
|
||||
when it needs this procedure, not as a label for the file.
|
||||
- **Frontload.** Self-contained enough for an agent with no prior context: define terms
|
||||
inline, do not assume other documents are loaded.
|
||||
- **Keep reasoning out of the body.** Cut the explanation of *why* each step exists. If it is
|
||||
worth preserving, it is a concept page under `kb/concepts/`, linked from here. Keep only
|
||||
enough reasoning to decide edge cases.
|
||||
inline, do not assume other documents are loaded. What this does and does not say about
|
||||
linking a shared contract: below.
|
||||
- **Reasoning earns its place by deciding something.** Keep what an agent needs in order to get
|
||||
*this* decision right, at the step where it falls; cut the explanation of why the step exists
|
||||
at all. If that is worth preserving, it is a concept page under `kb/concepts/`, linked from
|
||||
here. Where the line runs, and how to test a passage against it: below.
|
||||
- **State scope boundaries.** When does this *not* apply, and what to do instead.
|
||||
- **A command block reads the same in every shell.** Depending on the harness, an instruction
|
||||
runs under bash, Git Bash or PowerShell 7. A command in a fenced block is a `tools/wikitool`
|
||||
or `git` call, or the preflight's own call per platform - never syntax only one shell reads: no
|
||||
heredoc, no `export`, no `$(...)` or `$VAR`, no inline `VAR=value command`, no `&&`, no `for`
|
||||
loop, no `cp`, `cat >`, `sha256sum`, `curl` or `tar`. A step that needs one of them gets a
|
||||
`wikitool` command instead, or leaves the file work to the agent's own file tools. Two places
|
||||
are exempt, each with one line per shell: setting the session id
|
||||
([session-setup.md](session-setup.md)) and downloading the preflight before an instance exists
|
||||
([setup-instance.md](setup-instance.md) step 0). Migration documents under
|
||||
`instructions/migrations/` belong to the release they shipped with and are not rewritten. The
|
||||
stack's own instructions are held to this by a test in the origin repository; what an instance
|
||||
writes for itself is its own decision.
|
||||
- **Write it in English, and let the agent speak the instance's language.** Both rules, and the
|
||||
line between prose and quoted vocabulary, are stated once in
|
||||
[AGENTS.md § File naming](../AGENTS.md#file-naming). They are named here because this is the
|
||||
step where they are obeyed or lost: nothing checks either mechanically, and an instruction
|
||||
that models a sentence for the user is where the two are easiest to confuse - the model is
|
||||
written in English, the saying of it follows `kb/CONVENTIONS.md`'s `language:`.
|
||||
|
||||
### A skill's H1 is a name, not an imperative
|
||||
|
||||
`instructions/<name>/SKILL.md` takes a name-shaped H1 matching its `name:` frontmatter -
|
||||
`# Wiki Ingest`, not `# Ingest a source file into the wiki`. The imperative-title rule is
|
||||
written for the flat form and stops there.
|
||||
|
||||
The heading lies on no retrieval path. What decides whether a skill is picked up is
|
||||
`description`, which sits in the agent's context from session start; the body is read only once
|
||||
the skill is already open, and by then the title has nothing left to decide. Anthropic's
|
||||
skill-authoring guidance agrees by omission and by example: it normalises `name` and
|
||||
`description` and says nothing about the body's heading, and its own worked examples are noun
|
||||
phrases (`# PDF Processing`, `# BigQuery Data Analysis`). So does the vendored `commonplace`
|
||||
corpus, which arrived at the imperative-title rule independently and carves out the same
|
||||
exception in the same breath - "for promoted skills, the skill name is the title".
|
||||
|
||||
That is the whole exception. Everything else in this section binds a `SKILL.md` exactly as it
|
||||
binds an instruction.
|
||||
|
||||
### A skill's `description` speaks in third person
|
||||
|
||||
Anthropic's skill-authoring guidance requires third person in a skill's `description`, because it
|
||||
is injected into the system prompt for skill selection and an inconsistent point of view degrades
|
||||
that selection - "Processes Excel files and generates reports", never "I can help you process..."
|
||||
or "Process...". This binds every `instructions/<name>/SKILL.md` in this repo. The flat
|
||||
`instructions/<name>.md` form's `description` (above) is read on demand rather than injected as
|
||||
system-prompt metadata, so it keeps the imperative/label freedom that form already allows.
|
||||
|
||||
Nothing checks this mechanically - `tools/wikitool docs verify`/`instructions verify` validate a
|
||||
`description`'s presence and length, not its grammatical voice - so it holds only as long as each
|
||||
new skill is written to match the ones around it.
|
||||
|
||||
### A skill's name declares its family
|
||||
|
||||
Three prefixes exist today, each naming the subject domain a skill operates on, not the
|
||||
distribution boundary it ships behind: `wiki-` for the knowledge pipeline (`wiki-ingest`,
|
||||
`wiki-lint`, `wiki-manage`, `wiki-query`, `wiki-status`), `gtd-` for the commitment layer
|
||||
(`gtd-weekly-review` - see `kb/gtd/COLLECTION.md` and `docs/explanation/knowledge-and-commitment.md` for why
|
||||
that layer is named GTD rather than folded into `wiki-`), and `stack-` for the stack's own
|
||||
development, nested under `instructions/dev/` and therefore never present in a distributed
|
||||
instance (`instructions/dev/` above).
|
||||
<!-- dist:strip-start -->
|
||||
Dev-instance-only: the three skills in that family today are `stack-dev`, `stack-build` and
|
||||
`stack-close`.
|
||||
<!-- dist:strip-end -->
|
||||
A new skill takes the prefix of the family it belongs to, or opens a new one deliberately - never
|
||||
a bare name.
|
||||
|
||||
This is a convention, not something the tool enforces: an unprefixed or fourth-family name would
|
||||
compile, publish and pass every check exactly like the three above, so it is written down here for
|
||||
the next session to read before adding one.
|
||||
|
||||
### A skill's outbound reference is a plain path, not a link
|
||||
|
||||
`tools/wikitool instructions sync` copies each `SKILL.md` byte for byte into
|
||||
`.agents/skills/<name>/` and `.claude/skills/<name>/` (§ Publishing, above) - a different depth
|
||||
than the source, and without the sibling files a relative link might expect. A markdown link
|
||||
correct at `instructions/<name>/SKILL.md` (`../session-setup.md`, `../../kb/CONTRACT.md`)
|
||||
resolves to a different, usually nonexistent, file once copied: the number of `../` segments
|
||||
that reaches a target from `instructions/` does not reach the same target from
|
||||
`.claude/skills/`. Fifty-two of the fifty-eight relative links across the repo's seven skills at
|
||||
the time broke exactly this way before this rule existed, silently - nothing rendered the copy to
|
||||
notice, and no check read a link target.
|
||||
|
||||
So a `SKILL.md` never writes an outbound reference as a relative markdown link, correct depth or
|
||||
not. It names the target as a repo-root-relative **plain path** instead - `` `instructions/session-setup.md` ``, not `[session-setup.md](../session-setup.md)`; `` `kb/CONTRACT.md` `` for a
|
||||
whole file, `` `kb/CONVENTIONS.md` § Tone `` for a section rather than an anchored link. The path
|
||||
survives the copy unchanged because it does not depend on where the reading file sits: an
|
||||
agent's working directory is the instance root regardless of which published copy it opened, so
|
||||
the same plain path resolves in the source and in both published copies alike. The cost is that
|
||||
the reference is no longer clickable from the source file - accepted deliberately, because the
|
||||
source is not where an agent reads it from; the harness reads the published copy.
|
||||
`tools/wikitool instructions verify` enforces the ban mechanically
|
||||
(`check_skill_reference_paths`).
|
||||
|
||||
This binds only `SKILL.md`. The flat `instructions/<name>.md` form - this file included - is
|
||||
never copied anywhere, so its relative links stay exactly as correct as their `../` count says,
|
||||
and stay ordinary links; `tools/wikitool docs verify` (`check_reference_targets`) resolves those
|
||||
against the working tree instead of banning the syntax, over the same reference-file scope
|
||||
`tools/wikitool docs toc` uses.
|
||||
|
||||
### Reference depth: bundled files, not repo-wide contracts
|
||||
|
||||
Anthropic's skill-authoring guidance asks that reference files stay **one level deep from
|
||||
`SKILL.md`**, because a file reached at the second hop may be previewed rather than read -
|
||||
`head -100` instead of the whole file - leaving the step to run on incomplete information.
|
||||
|
||||
That rule governs **skill-bundled** material: files sitting in `instructions/<name>/` beside the
|
||||
`SKILL.md`. The guidance's own worked example is a bundle (`SKILL.md` → `REDLINING.md`,
|
||||
`OOXML.md`), and it says nothing about files outside the skill directory. No skill in this repo
|
||||
has a bundled file today, so as written the rule currently binds nothing here.
|
||||
|
||||
A skill's reference to a repo-wide contract - `kb/CONTRACT.md`, `tools/CONTRACT.md`,
|
||||
`instructions/gates.md` (written as a plain path per § "A skill's outbound reference is a plain
|
||||
path, not a link" above; this file is a flat instruction rather than a `SKILL.md`, so its own
|
||||
references to the same three files, a few sections up and below, stay ordinary links) - is a
|
||||
different category, and the two halves of the question have different answers:
|
||||
|
||||
- **The mechanic is real and directory-independent.** A contract reached at the second hop can
|
||||
be read partially exactly as a bundled file would be. Nothing about the path makes it safe.
|
||||
- **The rule is not.** Reading its scope wider than it states would attribute a rule to a source
|
||||
that does not carry it - the same move invariant 3 forbids about facts.
|
||||
|
||||
So the shared contracts stay shared and stay linked once. AGENTS.md invariant 8 is what put them
|
||||
there: copying `kb/CONTRACT.md` into five `SKILL.md` files is precisely the second copy that
|
||||
drifts. § Frontload does not ask for that either - it asks that a **step** be decidable without
|
||||
prior context, not that every rule the step obeys be restated at it.
|
||||
|
||||
What the mechanic does oblige is cheaper than either: **a link says what the step needs from the
|
||||
file it points at.** A bare "read X first" leaves a partial read undetectable; naming what is to
|
||||
be taken from it - the field, the section, the decision - keeps the step decidable even when the
|
||||
read came up short, and tells the next author which reference is actually load-bearing.
|
||||
`wiki-ingest` step 7 is the shape: three contracts linked, each with the clause that says why
|
||||
this step needs it.
|
||||
|
||||
This is a narrower posture than the vendored `commonplace` corpus takes, which makes outbound
|
||||
links exceptional in its instruction collection and frontloads the rest. That works for a corpus
|
||||
whose procedures do not share a contract; here they do, and invariant 8 outranks the preview
|
||||
risk.
|
||||
|
||||
**All of the above is a judgment, not a measurement**, and it is worth knowing why it cannot be
|
||||
the second. Whether the mechanic bites here is not something this repo can currently observe:
|
||||
the L2 trajectory scorers read `wikitool` calls, gates and publishes, never an agent's file
|
||||
reads, and no tool hook is wired on the primary harness at all - so a `head -100` leaves nothing
|
||||
to score. The other half of the claim, what ended up in the context window, produces no event
|
||||
anywhere by construction.
|
||||
|
||||
<!-- dist:strip-start -->
|
||||
Gitea #72 records what such a test would cost and why it was not bought. (Kept behind a strip
|
||||
marker: the pointer is worth having in the origin repo and resolves nowhere else.)
|
||||
<!-- dist:strip-end -->
|
||||
|
||||
### When a skill carries a copy-in checklist
|
||||
|
||||
Anthropic's skill-authoring guidance suggests, for a "particularly complex workflow", a checklist
|
||||
the agent copies into its response and ticks off as it goes. It names no threshold, so this repo
|
||||
sets one - otherwise the skills that carry such a block and the ones that do not read as an
|
||||
accident rather than a decision.
|
||||
|
||||
A `SKILL.md` carries the block when **one** of its flows runs to eight steps or more *and* that
|
||||
flow contains steps whose omission is silent - a judgment call, a field filled by hand, a
|
||||
cadence check, anything no tool error and no validator would report missing. Both halves are
|
||||
required. Length alone is not the problem: a long flow of tool calls announces its own gaps,
|
||||
because the next call fails without the previous one.
|
||||
|
||||
Two skills qualify today, and the block names each of their numbered steps once, verbatim:
|
||||
`wiki-ingest`, whose flow is long *and* carries steps that fail silently - `## Not Extracted`,
|
||||
the coverage check, and the lint cadence all skip past with no tool error and no validator to
|
||||
catch the omission - and `wiki-lint`, whose flow contains several steps that are pure judgment
|
||||
calls the same way. The rest do not, and the reason is worth stating so nobody adds one out of
|
||||
symmetry: every other skill's flow is short enough, and fails loudly enough step to step, that a
|
||||
reader cannot lose the thread even without a checklist - `wiki-manage`'s two flows, `wiki-query`,
|
||||
`wiki-status` and `gtd-weekly-review` all clear that bar.
|
||||
<!-- dist:strip-start -->
|
||||
Dev-instance-only: `stack-dev`, `stack-build` and `stack-close` sit under the same threshold,
|
||||
for the same reason.
|
||||
<!-- dist:strip-end -->
|
||||
|
||||
None of this is counted by number on purpose: a per-skill step count is a claim about a file this
|
||||
one does not own, and a claim like that can drift silently the moment the other file changes.
|
||||
This passage once cited `wiki-query` at six steps where it had already been seven for a while, and
|
||||
separately named only five of the eight skills that exist - neither wrong number made any check go
|
||||
red, because nothing here reads another file's prose. The two-halves test above (length *and* a
|
||||
silently-omittable step) is what actually does the work of picking `wiki-ingest` and `wiki-lint`
|
||||
out from the rest; a count was never load-bearing for that test, only decoration for it, and
|
||||
dropping it removes the one part of this passage that could be wrong without anyone noticing.
|
||||
|
||||
The block says that it is to be copied and carried, not read. A checklist read once is the table
|
||||
of contents it replaced.
|
||||
|
||||
### How much reasoning a step may carry
|
||||
|
||||
"Cut the reasoning" and "keep enough to decide an edge case" are one sentence pulling two ways,
|
||||
and the largest instruction in this repo lives in the gap. The line runs here:
|
||||
|
||||
| Keep | Cut |
|
||||
|---|---|
|
||||
| What an agent must know to get this decision right, at the step where it falls | Why the step exists at all |
|
||||
| The consequence of the wrong choice, when nothing later catches it | The consequence, when a validator, a gate or a later step catches it |
|
||||
| Why a plausible-looking default is the wrong answer | Background about the design that produced the field |
|
||||
|
||||
Two tests, both cheap:
|
||||
|
||||
- **Substitution.** Delete the passage and read the step again. Does an agent with no prior
|
||||
context still make the same call? If yes, it was background. If it now guesses, it was a
|
||||
decision aid, and it stays - however long it runs.
|
||||
- **Once.** A decision aid belongs at the step where the decision falls, and at exactly one such
|
||||
step (AGENTS.md invariant 8). Where the same decision falls at two steps - `wiki-ingest` asks
|
||||
for `fidelity`/`authority` in step 5 and again in step 6 - the reasoning is written at the
|
||||
first and the second carries the instruction plus a pointer, never a second telling.
|
||||
|
||||
A passage that survives both is not an exception to the rule. Deciding an edge case is the part
|
||||
the rule keeps; the length it takes to do that is not the measure.
|
||||
|
||||
## Instruction duality
|
||||
|
||||
@@ -153,7 +414,8 @@ What lives where:
|
||||
|-------|------|
|
||||
| [AGENTS.md](../AGENTS.md) | Invariants and routing - what must always hold |
|
||||
| `instructions/` | How the tooling is *operated* |
|
||||
| [kb/CONTRACT.md](../kb/CONTRACT.md) + each `COLLECTION.md` | How a page is *authored* |
|
||||
| [kb/CONTRACT.md](../kb/CONTRACT.md) | What the stack enforces about a page, in every instance |
|
||||
| `kb/CONVENTIONS.md` + each `COLLECTION.md` | What *this* instance decided about authoring - owned by the instance, shipped only as a `.template` |
|
||||
| [types/](../types/type-spec.md) | What a page structurally *is* |
|
||||
| [tools/CONTRACT.md](../tools/CONTRACT.md) | What each command does and how it fails |
|
||||
|
||||
|
||||
+31
-12
@@ -1,33 +1,44 @@
|
||||
---
|
||||
type: types/instruction.md
|
||||
name: bootstrap
|
||||
description: Prepare a fresh clone for work - create the tools venv and publish the skills into the harness directories, which are generated and not committed.
|
||||
description: Prepare a fresh clone of an existing instance (a second machine, a new checkout) for work - run the preflight (tool paths and the tools venv) and publish the skills into the harness directories, which are generated and not committed.
|
||||
---
|
||||
|
||||
# Bootstrap a fresh clone
|
||||
|
||||
An instance lives in its own git repository, so a second machine - or a new checkout on the same
|
||||
one - gets it with `git clone`. What the clone does not carry is everything that describes one
|
||||
machine rather than the instance: the tool paths and the tools venv, and the published skills.
|
||||
|
||||
`.agents/skills/` and `.claude/skills/` are generated copies of the skill directories under
|
||||
`instructions/`, and both are gitignored. A fresh clone therefore has no skills at all until
|
||||
they are published: the agent harness will not offer `wiki-ingest`, `wiki-query`,
|
||||
`wiki-manage`, `wiki-lint` or `wiki-status` before this runs.
|
||||
`wiki-manage`, `wiki-lint`, `wiki-status` or `gtd-weekly-review` before this runs.
|
||||
|
||||
## When to run
|
||||
|
||||
- After cloning the repository.
|
||||
- After cloning the instance's repository.
|
||||
- After `instructions/<name>/SKILL.md` is added, renamed, or edited.
|
||||
- Whenever `tools/wikitool instructions verify` reports a missing or drifted copy.
|
||||
|
||||
## Steps
|
||||
|
||||
1. **Create the tool environment** (once per clone):
|
||||
1. **Run the preflight** (once per clone, and again after moving it) - see
|
||||
[preflight.md](preflight.md). It records the tool paths in `.wikitool-tools.json` and
|
||||
creates `tools/.venv`; until it exits 0, `tools/wikitool` refuses to start:
|
||||
|
||||
```bash
|
||||
cd tools
|
||||
python3 -m venv .venv
|
||||
.venv/bin/pip install -r requirements.txt
|
||||
cd ..
|
||||
tools/preflight.sh
|
||||
```
|
||||
|
||||
From PowerShell 7 on Windows, run the twin instead - same questions, same file:
|
||||
|
||||
```powershell
|
||||
pwsh -NoProfile -ExecutionPolicy Bypass -File tools/preflight.ps1
|
||||
```
|
||||
|
||||
On exit 42, show its output to the user verbatim and wait.
|
||||
|
||||
2. **Publish the skills:**
|
||||
|
||||
```bash
|
||||
@@ -46,7 +57,7 @@ they are published: the agent harness will not offer `wiki-ingest`, `wiki-query`
|
||||
4. **Check for personalization.** A clone predating the personalization files has no
|
||||
`USER.md`/`SOUL.md`, and `tools/wikitool doctor` reports `personalization: FAIL` for it.
|
||||
That is a one-off catch-up, not a bootstrap step that repeats: run **only** the
|
||||
Personalization step (6) of [setup-instance.md](setup-instance.md), not the whole
|
||||
personalization step (5) of [setup-instance.md](setup-instance.md), not the whole
|
||||
procedure - this clone already has its git repo, author identity and content. A clone that
|
||||
already carries both files needs nothing here.
|
||||
|
||||
@@ -65,12 +76,20 @@ they are published: the agent harness will not offer `wiki-ingest`, `wiki-query`
|
||||
6. **Restart the agent session** if it was already running. Harnesses read the skill
|
||||
directories at startup, so skills published mid-session are not picked up.
|
||||
|
||||
7. **Expect a lingering `session-id` WARN.** A `tools/wikitool doctor` run at this point reports
|
||||
`OK` throughout except `session-id: WARN` - that check is scoped to the working session, not
|
||||
the clone, so a freshly bootstrapped checkout with no `WIKITOOL_SESSION_ID` exported yet
|
||||
always shows it. This is expected, not a Bootstrap gap: exporting it here would only be true
|
||||
for this one-off setup run, not for whichever session picks up the actual work next, in a new
|
||||
shell after step 6's restart. Run [session-setup.md](session-setup.md) at the start of that
|
||||
session instead.
|
||||
|
||||
## Scope
|
||||
|
||||
This does not apply to anything under `kb/`, `raw/` or `reports/`; those are committed and
|
||||
present immediately after a clone. If the wiki content looks wrong after cloning, that is a
|
||||
lint question, not a bootstrap one.
|
||||
|
||||
This also does not apply to a fresh instance created via `tools/wikitool dist export` - it has
|
||||
no git history, no author identity, and no generated indexes yet. That is
|
||||
[setup-instance.md](setup-instance.md), a longer procedure this one is a single step of.
|
||||
This also does not apply to a new instance installed from a release - it has no git history, no
|
||||
author identity, and no generated indexes yet. That is [setup-instance.md](setup-instance.md), a
|
||||
longer procedure this one is a single step of.
|
||||
@@ -0,0 +1,194 @@
|
||||
---
|
||||
type: types/instruction.md
|
||||
name: bug-report
|
||||
description: How to collect a bug-report bundle when the stack misbehaves on this machine - run tools/bugreport, write a fact-only chronology, tell the user what the bundle contains, and stop short of sending it anywhere.
|
||||
manual: true
|
||||
---
|
||||
# Collect a bug report
|
||||
|
||||
When setup, an upgrade or a command fails on one machine and works on another, the person who has to
|
||||
fix it sees nothing of what happened here. This procedure produces one bundle that answers the first
|
||||
round of their questions - which machine, which Python, which shell, which harness, what the stack
|
||||
looked like, what `wikitool` printed - so that the report is not a guessing game.
|
||||
|
||||
**Run this only when asked, by name, or when the user agrees to it after a failure.** It is
|
||||
`manual: true` on purpose: the bundle contains private data, and whether to produce one is the
|
||||
user's decision, not the agent's. Nothing links to this file from `AGENTS.md` or a skill, apart from
|
||||
the pointers at the failure decision points of [setup-instance.md](setup-instance.md) and
|
||||
[upgrade-instance.md](upgrade-instance.md), which only offer it.
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
|
||||
- [What the bundle holds](#what-the-bundle-holds)
|
||||
- [When to run](#when-to-run)
|
||||
- [Steps](#steps)
|
||||
- [Chronology template](#chronology-template)
|
||||
- [Decision points](#decision-points)
|
||||
- [Scope](#scope)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
## What the bundle holds
|
||||
|
||||
`tools/bugreport.py` is a standalone script. It uses the standard library only and imports nothing
|
||||
from the stack, so it still runs when `wikitool` does not start - no venv, a broken package, a Python
|
||||
that is too old for the stack. Its launcher `tools/bugreport` assumes no more than that: it looks for
|
||||
a Python 3.8 or later on `PATH` itself, skipping the Microsoft Store's aliases, and needs neither
|
||||
the preflight nor `.wikitool-tools.json`. It writes `reports/bugreport-<UTC stamp>/` and a zip beside it, in
|
||||
four layers:
|
||||
|
||||
| Layer | Files | Holds |
|
||||
|-------|-------|-------|
|
||||
| 1 Environment | `environment.json` | OS, every Python and shell found, harness, `PATH`, environment variable names (values only for a fixed list), git configuration, venv, line endings, on Windows also long paths, execution policy and mark-of-the-web |
|
||||
| 2 Stack | `stack.json`, `tree-structure.json` | `VERSION`, the `.wikitool-*.json` files with secrets removed, git status and the last commits, and the shape of `kb/` and `raw/` (counts, depths, path lengths, names that break on Windows) |
|
||||
| 3 wikitool | `wikitool/*.txt`, `trace.jsonl` | Verbatim output of `version show`, `doctor`, `budget status`, `instructions verify` and `docs verify`, and the caller's session trace. If `version show` fails, `wikitool` counts as not started and nothing else runs |
|
||||
| 4 Chronology | `CHRONOLOGY.md`, `transcripts/` | What the agent did and saw, and harness transcripts if the user asked for them |
|
||||
|
||||
`MANIFEST.md` lists every file and marks the ones that may contain page content and titles: the trace,
|
||||
the chronology and the transcripts.
|
||||
|
||||
Two rules hold for everything the script generates itself. **Secrets are always removed**: values of
|
||||
keys that look like a token, password, secret, key or auth entry, credentials in URLs, and every value
|
||||
of that kind found while collecting is also replaced wherever else it turns up. **Page titles are kept
|
||||
out** unless `--titles` is given: paths under `kb/` and `raw/` are replaced by their shape
|
||||
(depth, length, whether they hold a space or a non-ASCII character) and `[[wikilinks]]` by the same
|
||||
flags. A title that stands as bare prose is not found - which is why the trace, the chronology and the
|
||||
transcripts are marked instead.
|
||||
|
||||
**Pseudonymisation is optional** (`--pseudonymise`) and runs in two stages. The bundle then keeps the
|
||||
*shape* of every name - length per word, spaces, hyphens, character classes, separators, depth of a
|
||||
path - because that is what an installation failure turns on, and replaces the name itself.
|
||||
|
||||
- **Stage 1 is mechanical.** The script reads what the machine knows about its user - user name,
|
||||
host, home and repository path, git identity, remote URLs - and replaces each identity in every
|
||||
text file by a placeholder. The same word always gets the same placeholder, in every file and in
|
||||
JSON-escaped form too. The stack's own public origin and system folder names stay readable.
|
||||
- **Stage 2 is a model's judgement, applied mechanically.** Names the script cannot know - people,
|
||||
companies, customers, internal hosts, projects - are named by you as candidates in a file; the
|
||||
script applies them with the same machinery. You replace nothing in the bundle yourself.
|
||||
- **Three local files** sit beside the bundle directory, never inside it and never in the zip:
|
||||
`bugreport-<stamp>.pseudonyms.json` (the mapping), `bugreport-<stamp>.review.txt` (what stage 1
|
||||
left behind, for you to read) and the candidate file you write. All three contain originals.
|
||||
- **A residual uncertainty remains and is always named:** stage 2 can miss a name, above all in the
|
||||
free text of a large trace or transcript that you did not read in full.
|
||||
|
||||
The model of the harness reads the bundle for stage 2 - the same place that already sees this
|
||||
session. The bundle does not leave that place because of it.
|
||||
|
||||
## When to run
|
||||
|
||||
- Setup or an upgrade failed and the user wants to report it.
|
||||
- A command fails in a way that looks tied to the machine, and the user asks for a report.
|
||||
- The user asks for a bug report by name.
|
||||
|
||||
## Steps
|
||||
|
||||
1. **Tell the user what is about to happen**, in the instance's KB language: a bundle will be written
|
||||
under `reports/`, it holds machine, user and path names and the git remotes unless it is
|
||||
pseudonymised (below), and it is not sent anywhere. Ask whether the session trace, page titles and transcripts
|
||||
may go in. The defaults are: trace in, titles out, no transcripts.
|
||||
|
||||
Ask which channel the bundle will take, and recommend pseudonymisation for every channel except a
|
||||
direct handover to the maintainer over a secure channel: a tracker issue, an email or a chat is
|
||||
not one. The default is off; say so, and that it costs one more step.
|
||||
|
||||
2. **Write the chronology** to a file under `reports/` (which is gitignored) from the
|
||||
[template](#chronology-template) below. Facts only - no diagnosis. If the session's own history is
|
||||
too long to reconstruct, say what is missing instead of filling the gap.
|
||||
|
||||
3. **Run the collector** through its launcher, which finds a Python 3.8 or later itself:
|
||||
|
||||
```bash
|
||||
tools/bugreport --chronology reports/chronology.md
|
||||
```
|
||||
|
||||
Add `--no-trace` if the user declined the trace, `--titles` if titles may stay,
|
||||
`--transcript <file>` (repeatable) for a transcript the user pointed at, `--session <id>` to take a
|
||||
trace other than the caller's, `--pseudonymise` if the user chose it (stage 1). Do not call a Python yourself: on Windows,
|
||||
`python3` - and in PowerShell also `python` - can be the Microsoft Store's alias, which the
|
||||
launcher skips. If the launcher finds no Python, it exits 1 and says why; that is the report -
|
||||
give the user its text verbatim. If the user then names a working Python by its full path, run
|
||||
the collector with it directly (`<full path to python> tools/bugreport.py` and the same
|
||||
options).
|
||||
|
||||
4. **Stage 2, only if the bundle was pseudonymised.** Read, in the bundle: `CHRONOLOGY.md` and
|
||||
`MANIFEST.md` completely; the review list `reports/bugreport-<stamp>.review.txt` completely; the
|
||||
trace and each transcript completely only if the file is under 100 KB, otherwise only what the
|
||||
review list points to. Name what is still left of a person, company, customer, internal host or
|
||||
domain, or project - one per line in `reports/candidates.txt`; `#` starts a comment. Then run:
|
||||
|
||||
```bash
|
||||
tools/bugreport --bundle reports/bugreport-<stamp> --candidates reports/candidates.txt
|
||||
```
|
||||
|
||||
The script applies the candidates with the machinery of stage 1, reports which it did not apply
|
||||
(too short, system vocabulary, not found) and packs the zip again. **Never replace anything in
|
||||
the bundle yourself.** The run can be repeated with further candidates. It refuses, with exit 1 and
|
||||
an unchanged bundle, when the mapping beside the bundle is gone.
|
||||
|
||||
5. **Do not imitate the collector's session id.** It runs its counting `wikitool` calls under
|
||||
`WIKITOOL_SESSION_ID=bugreport-<stamp>` itself; do not export that variable, or any `bugreport-*`
|
||||
one, in the session. The exception it gets in [gates.md](gates.md) § "Taking a new session id"
|
||||
belongs to the script alone.
|
||||
|
||||
6. **Report the result.** Quote the bundle path, the archive path and the privacy notice the script
|
||||
prints, name the gaps in `MANIFEST.md`, and tell the user to read the bundle before sharing it.
|
||||
After stage 2, name the residual uncertainty in words of your own: stage 2 is a model's judgement
|
||||
and can have missed names, above all in the free text of a trace or transcript over 100 KB.
|
||||
Say that the mapping, the review list and the candidate file hold originals and stay on this
|
||||
machine, and that the mapping may be deleted after the last stage 2 run. Then stop: the channel - a tracker issue, an email, a chat - is the user's choice, and the agent
|
||||
never uploads the bundle.
|
||||
|
||||
## Chronology template
|
||||
|
||||
```markdown
|
||||
# Chronology
|
||||
|
||||
## Goal
|
||||
One sentence: what the user asked for, e.g. "Set up a new instance from a fresh clone."
|
||||
|
||||
## Environment as the agent saw it
|
||||
Harness, shell, OS, anything the user said about the machine that the collector cannot know.
|
||||
|
||||
## Steps
|
||||
1. `<exact command>` - exit code, and the first line of the error or output that mattered, verbatim.
|
||||
2. ...
|
||||
|
||||
## Expected and observed
|
||||
- Expected: what the instruction said would happen.
|
||||
- Observed: what happened instead, verbatim where it is short.
|
||||
|
||||
## Changes made by hand
|
||||
Every file edited or created outside a `wikitool` command, and every setting changed, in order.
|
||||
|
||||
## Not known
|
||||
What the agent could not find out, or did not check.
|
||||
```
|
||||
|
||||
Facts only: no page content, no guessed cause, no advice.
|
||||
|
||||
## Decision points
|
||||
|
||||
- **The mapping is gone before stage 2?** Stage 2 refuses: it needs the salt stage 1 used, or it would
|
||||
replace a word differently from the path it already replaced. Collect the report again with
|
||||
`--pseudonymise`.
|
||||
- **The user wants a name added after stage 2?** Run stage 2 again with a candidate file holding only
|
||||
that line. The same run also works when the human spots a name while reading the bundle.
|
||||
- **The user declines the trace?** Run with `--no-trace`. The manifest records the exclusion.
|
||||
- **The failure can be reproduced, and there is no trace?** A distributed instance records none by
|
||||
default. Offer to repeat the failing step in one new shell with `WIKI_TRACE=1` set for that
|
||||
session only, then collect. Do not change the checkout's telemetry configuration for it.
|
||||
- **The user wants the report to name pages?** Run with `--titles`; the bundle then also carries
|
||||
`tree-paths.txt`, which lists every path under `kb/` and `raw/`.
|
||||
- **The collector exits 1?** It could not write the bundle (a missing input file, a full disk). Read
|
||||
the message, fix the cause, retry once, then report the exact error.
|
||||
- **The preflight itself stops, so no Python or venv exists to run the collector?** Its output is
|
||||
the report: take it verbatim from `tools/preflight.sh` or `tools/preflight.ps1`, with the exact
|
||||
command, and add `.wikitool-tools.json` if it was written. Do not install anything to get a bundle.
|
||||
- **A `wikitool` output in the bundle looks wrong or refuses to run?** Do not re-run it to see more.
|
||||
The bundle records what happened; that is the report.
|
||||
|
||||
## Scope
|
||||
|
||||
Covers producing the bundle. It does not cover reading a bundle someone else sent, triaging the
|
||||
report, or filing it - all of that is the maintainer's side and the user's choice of channel.
|
||||
@@ -17,6 +17,21 @@ session worth keeping" is the user's, not the agent's. Nothing links to this fil
|
||||
`AGENTS.md` or a skill, and nothing should - a link there is exactly how a deliberate procedure
|
||||
stops being deliberate.
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
|
||||
- [Where a session's output belongs](#where-a-sessions-output-belongs)
|
||||
- [When to run](#when-to-run)
|
||||
- [Steps](#steps)
|
||||
- [1. Cut the session into topics](#1-cut-the-session-into-topics)
|
||||
- [2. Fix the fidelity before writing a word](#2-fix-the-fidelity-before-writing-a-word)
|
||||
- [3. Write each transcript](#3-write-each-transcript)
|
||||
- [4. File what is still open, before ingesting](#4-file-what-is-still-open-before-ingesting)
|
||||
- [5. Ingest, one transcript at a time](#5-ingest-one-transcript-at-a-time)
|
||||
- [6. Verify the set, not just the last one](#6-verify-the-set-not-just-the-last-one)
|
||||
- [Decision points](#decision-points)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
## Where a session's output belongs
|
||||
|
||||
Three surfaces, three jobs. Collapsing them is the failure this procedure exists to prevent.
|
||||
@@ -61,11 +76,16 @@ When a finding spans two topics, put it in **one** transcript in full and let th
|
||||
reference it by name. Two half-accounts produce two source pages claiming the same fact, which
|
||||
`lint` will not catch because both are individually well-formed.
|
||||
|
||||
**This cut is available because the transcript does not exist yet.** A source that arrived as one
|
||||
file is not cut - `raw/` keeps it whole, and one raw file has exactly one owning source page. A
|
||||
received source carrying many subjects is a breadth case with a different remedy:
|
||||
[ingest-large-tree.md](ingest-large-tree.md) § A broad source is not cut.
|
||||
|
||||
### 2. Fix the fidelity before writing a word
|
||||
|
||||
Capture is layered, and **the layer is decided at capture and never rises afterwards.** No
|
||||
citation syntax, no later review, no confidence bump can promote a paraphrase to a quote; only
|
||||
going back to the original can, and a session's scrollback will not be there to go back to.
|
||||
citation syntax and no later review can promote a paraphrase to a quote; only going back to the
|
||||
original can, and a session's scrollback will not be there to go back to.
|
||||
|
||||
So decide, per passage, before writing:
|
||||
|
||||
|
||||
@@ -1,77 +0,0 @@
|
||||
---
|
||||
type: types/instruction.md
|
||||
name: claude-code-model-selection
|
||||
description: Which Claude model and effort level to run a Claude Code session, a spawned subagent, or a /code-review pass at for a given task in this repo.
|
||||
---
|
||||
|
||||
# Pick the Claude model and effort level for the task at hand
|
||||
|
||||
Scale the model and effort to how much judgment the task actually needs. Running everything at
|
||||
the most capable model and highest effort is safe but wasteful: the gates in [gates.md](gates.md)
|
||||
are enforced in code, not by model judgment, so a weaker model cannot bypass them - it can only
|
||||
do a worse job of the calls the gates don't cover.
|
||||
|
||||
Claude-Code-only, and imported by CLAUDE.md rather than linked from AGENTS.md: the model names,
|
||||
the `/code-review` effort dial and the `Agent` tool's `model:` override have no equivalent in the
|
||||
other harnesses this repo supports (Codex CLI, GitHub Copilot CLI, Mistral Vibe). See
|
||||
[instructions/CONTRACT.md](CONTRACT.md) for that split.
|
||||
|
||||
## When to run
|
||||
|
||||
Before spawning a subagent with an explicit `model:` override, before picking a `/code-review`
|
||||
effort level, and when the user asks which model to use - or when the session's current model is
|
||||
clearly mismatched to the task that just started.
|
||||
|
||||
Two of the three choices are the agent's to make; the session's own model is not. An agent cannot
|
||||
switch the model it is running as - that is the user's `/model` - so step 1 is a recommendation
|
||||
to *make*, not a setting to apply.
|
||||
|
||||
## Steps
|
||||
|
||||
1. **Recommend the session's model and effort by the skill in use**, when asked or when the
|
||||
mismatch is worth one sentence. Say it once and continue working either way - a session that
|
||||
argues about its own model instead of doing the task has already cost more than the model
|
||||
difference:
|
||||
|
||||
| Skill / task | Model | Effort |
|
||||
|---|---|---|
|
||||
| `wiki-status`, simple `wiki-query` lookups | Sonnet | default |
|
||||
| `wiki-lint` | Sonnet | default |
|
||||
| `wiki-ingest`, `wiki-manage`, judgment-heavy `wiki-query` | Sonnet | high |
|
||||
| Stack development: `tools/`, `types/`, `instructions/` as code | Opus | high |
|
||||
|
||||
2. **Pick a spawned subagent's model by what it does**, via the `Agent` tool's `model:`
|
||||
parameter - the values are `haiku`, `sonnet`, `opus`, `fable`:
|
||||
|
||||
- Read-only search/lookup (an `Explore` agent, or a `general-purpose` agent doing pure
|
||||
retrieval): `model: "haiku"`. No judgment call is being delegated, only retrieval.
|
||||
- A subagent that writes pages, reviews code, or decides something: leave `model:` off so it
|
||||
inherits the session's model, chosen per step 1.
|
||||
- A fork (`subagent_type: "fork"`) always inherits the parent session's model; a `model:`
|
||||
override on a fork is ignored.
|
||||
|
||||
3. **Pick a `/code-review` effort level by blast radius, not by habit.** The levels are `low`,
|
||||
`medium`, `high`, `xhigh`, `max` and `ultra` (multi-agent, in the cloud):
|
||||
|
||||
- A routine diff (a skill wording fix, an ordinary ingest's tool output): `low` or `medium` -
|
||||
fewer, high-confidence findings are enough.
|
||||
- Gate code (`run_budget.py`, `git_publish.py`, anything implementing the Mass-Update or
|
||||
Iteration gates), the compiler, or a change about to ship in a version bump: `high` and up -
|
||||
broader coverage is worth the cost when the blast radius of a missed bug is a safety gate.
|
||||
- `ultra` is user-triggered and billed separately; recommend it, never assume it.
|
||||
|
||||
## Decision points
|
||||
|
||||
- **Task spans both a mechanical step and a judgment call?** Pick by the judgment call, not the
|
||||
mechanical one - `wikitool` carries the mechanical part regardless of which model is
|
||||
supervising it.
|
||||
- **Unsure which row applies?** Default to Sonnet at high effort, not the most capable model at
|
||||
the highest effort. Under-provisioning costs one worse answer in one session; reflexively
|
||||
over-provisioning is a standing cost paid every session.
|
||||
|
||||
## Scope
|
||||
|
||||
Does not apply to non-Claude-Code harnesses - see the note above; a follow-up issue tracks
|
||||
whether and how they should decide this differently. Does not set the classifier model behind
|
||||
Claude Code's own `auto` permission mode - that is a harness internal, not a per-task choice
|
||||
this repo controls.
|
||||
@@ -24,5 +24,6 @@ carries it (see [tools/CONTRACT.md](../../tools/CONTRACT.md) for what `dist expo
|
||||
|
||||
## Scope
|
||||
|
||||
Only relevant while working in [stack-dev](stack-dev/SKILL.md) mode. Not part of the wiki
|
||||
Only relevant in a stack-development session ([stack-mode.md](stack-mode.md)), mostly while
|
||||
designing. Not part of the wiki
|
||||
content pipeline, and not linked from anything outside `instructions/dev/`.
|
||||
@@ -0,0 +1,94 @@
|
||||
---
|
||||
type: types/instruction.md
|
||||
name: corpus-policy
|
||||
description: What "curated enough" means for kb/ when it is demo and testbed at once, the measurable floors that define it, and what a reactive fix to the corpus may and may not do.
|
||||
---
|
||||
# Keep kb/ curated enough to develop against, without a second corpus
|
||||
|
||||
This instance runs one `kb/` for two purposes at once: a public demo and the testbed this stack
|
||||
is developed against. There is deliberately no fixture corpus, no `--with-demo` export, and no
|
||||
second repository - see Gitea #28. The corpus's size and shape are set by what targeted
|
||||
development needs, not by a synthetic fixture size or a demo aesthetic.
|
||||
|
||||
## When to run
|
||||
|
||||
- Before judging whether the corpus can exercise a change under development - ranking, index
|
||||
scaling, orphan detection, a new label, a new type-spec.
|
||||
- Before a reactive fix touches `kb/` content rather than the failing code - the floors below
|
||||
are what decides whether the fix may proceed as-is.
|
||||
- Picking up Gitea #28 or #30, or any issue that references this file.
|
||||
|
||||
## The floors
|
||||
|
||||
Each is mechanically checkable with an existing `wikitool` command; none needs new tool code.
|
||||
A floor exists to keep some class of bug observable, not to describe an aesthetic target - so
|
||||
when a session is about to make one of these numbers *worse*, that is the signal to stop and
|
||||
think, not a number to defend for its own sake.
|
||||
|
||||
| Floor | Check | Why this number |
|
||||
|---|---|---|
|
||||
| Every page type has ≥1 page | `wikitool search --field type=types/<t>.md` | A type with zero pages means its schema, its collection contract and its lint rules are unexercised |
|
||||
| Every declared subtype has ≥1 page | `wikitool search --field <x>_type=<v>` | Same reasoning, one level down - `entity_type`, `concept_type`, `source_type`. **Exception:** `source_type: unclassified` (Gitea #66) may sit at zero - it is a visible catalog slot for an unclear source, not a value the corpus is expected to exercise, and a page manufactured just to fill it would violate invariant 3 the same as any other unsourced page |
|
||||
| ≥5 pages corpus-wide with ≥3 `sources:` entries | one-off script, see below | Provenance fan-in - multiple sources backing one claim - is a real case only a handful of pages exercise; fewer than 5 and a provenance-index bug can hide |
|
||||
| Orphan pages (no inbound link) between 1 and 10 | `wikitool lint` | Zero orphans makes orphan detection itself unobservable; more than 10 means the corpus stopped being curated |
|
||||
| Average outbound wikilinks per page ≥4 | one-off script, see below | Below this, ranking and graph-traversal work has too little structure to exercise |
|
||||
|
||||
A floor is a lower bound only. There is no upper bound on page count or on any of these numbers
|
||||
except the orphan ceiling above - a corpus that outgrows these floors through real ingests is
|
||||
not a problem this file cares about.
|
||||
|
||||
**Measured 2026-09-03** (see Gitea #28): 181 pages, 14/14 types and subtypes covered, 12 pages
|
||||
with ≥3 sources, 3 orphans, 6.2 average outbound links. All floors held without any manufactured
|
||||
content - the corpus was already big enough when the question was asked.
|
||||
|
||||
A type or subtype sitting at exactly the floor - one page - shows no set-level bugs, only that
|
||||
the type is *reachable*. That is a soft target for the next `wiki-ingest` that happens to
|
||||
produce a matching page, never a reason to write one: filing an unsourced page to clear a floor
|
||||
is exactly what AGENTS.md invariant 3 forbids, floor or no floor. The same holds for an
|
||||
authorised link label with zero live uses (`wikitool xref` reports these) - fill it when a real
|
||||
edge calls for it, never manufacture one to exercise the label.
|
||||
|
||||
To check the two floors without a dedicated command, walk `kb/**/*.md` (excluding
|
||||
`INDEX.md`/`COLLECTION.md`/`CONTRACT.md`/`CONVENTIONS.md`), parse frontmatter, and: count pages
|
||||
whose `related:` array (resolved against page titles) has ≥3 entries for outbound density; count
|
||||
`sources:` array length ≥3 for the provenance floor. `wikitool search` and `wikitool lint`
|
||||
cover everything else in the table.
|
||||
|
||||
## What a reactive fix may do to kb/ content
|
||||
|
||||
Three tiers, by how much of the corpus a change touches:
|
||||
|
||||
1. **Pointwise - always allowed.** Creating, updating, renaming or deleting a single page
|
||||
through the normal tools (`new`, `touch`, the page-lifecycle procedure), below the
|
||||
Mass-Update Gate's threshold. This is ordinary work and needs no special permission.
|
||||
2. **Corpus-wide - planned only, never reactive.** A migration, a vocabulary sweep, a bulk
|
||||
`touch` across many pages. This needs its own issue and, per `work/CONTRACT.md`, a `work/`
|
||||
run - never a same-session reaction to whatever the session was originally doing. If a
|
||||
session hits the Mass-Update Gate (exit 42, see `instructions/gates.md`) while working on
|
||||
something else, it does not fetch the `--confirm` token to push through: it stops, opens an
|
||||
issue for the corpus-wide change, and finishes the original task without it.
|
||||
3. **Reactive - never allowed.** Deleting or reshaping a page to make a failing test pass;
|
||||
restructuring corpus content to route around a tool bug (AGENTS.md invariant 7); using
|
||||
`kb/` as a scratch surface for a tool experiment. If a stack change under development needs a
|
||||
corpus shape that does not exist, build it as a pytest fixture (see the next section) -
|
||||
never manufacture it in `kb/`.
|
||||
|
||||
## Relationship to the test fixtures
|
||||
|
||||
`tools/chemenu/tests/conftest.py`'s `kb_dir`/`raw_dir` fixtures and `test_pipeline_l0.py` cover
|
||||
the **small, isolated** case: a handful of pages, built fresh per test, hermetic. `kb/` covers
|
||||
the **large, connected** case: 181+ pages, grown link density, real provenance history that no
|
||||
per-test fixture reconstructs economically. The cut: if a `tmp_path` tree can reproduce what the
|
||||
test needs, it belongs in a fixture; if the test needs density or scale that only a grown corpus
|
||||
has, it belongs against `kb/`. Neither absorbs the other's job - see
|
||||
[testing-conventions.md](testing-conventions.md).
|
||||
|
||||
## Decision points
|
||||
|
||||
- **A floor would be violated by an in-progress change - is that a blocker?** Only for the
|
||||
orphan ceiling and the type/subtype floors, since those two can go to zero. The density and
|
||||
provenance floors move gradually with ordinary ingests and are not gating on any single
|
||||
session.
|
||||
- **Corpus is "too small" for a feature under development?** That is not this file's problem to
|
||||
solve by adding pages - see tier 3 above. Either the feature waits for a real ingest to supply
|
||||
the shape, or it gets a pytest fixture.
|
||||
@@ -0,0 +1,73 @@
|
||||
---
|
||||
type: types/instruction.md
|
||||
name: dev-setup
|
||||
description: Set up a clone of the origin repository for stack development - preflight, skills, the demo corpus and its persona, telemetry on - and use dist export as a build and test tool, never as a way to install an instance.
|
||||
---
|
||||
# Set up a development checkout of the stack
|
||||
|
||||
A clone of the origin repository is where the stack is developed. It is not an instance: it
|
||||
carries a demo corpus that documents the stack itself, a demo persona in `USER.md`/`SOUL.md`, the
|
||||
development material under `instructions/dev/` and `commonplace/`, and no
|
||||
`.wikitool-release.json`. Instances are installed from releases
|
||||
([setup-instance.md](../setup-instance.md)); nothing here produces one.
|
||||
|
||||
## When to run
|
||||
|
||||
- A fresh clone of the origin repository, before the first stack-dev session in it.
|
||||
- A clone that was moved, or whose `tools/.venv` was removed - only step 2 again.
|
||||
- Before testing a change to the install path itself (`setup-instance.md`, the preflight, the
|
||||
release workflow) - step 5.
|
||||
|
||||
## Steps
|
||||
|
||||
1. **Clone** the origin repository. The tree is complete as checked out: `kb/`, `raw/`,
|
||||
`USER.md`, `SOUL.md` and the filled `kb/CONVENTIONS.md` are committed here, unlike in an
|
||||
instance.
|
||||
|
||||
2. **Run [bootstrap.md](../bootstrap.md)** - the preflight, then `tools/wikitool instructions
|
||||
sync`. That publishes `stack-dev`, `stack-build` and `stack-close` along with the content skills;
|
||||
all three exist only in this repository.
|
||||
|
||||
3. **Record the environment** (bootstrap.md step 5). Here it is worth the minute: which harness,
|
||||
that `gitea-mcp` reaches the tracker and CI, which remote `publish` talks to. Every stack-dev
|
||||
session reads it instead of asking.
|
||||
|
||||
4. **Know what differs from an instance before relying on a default.**
|
||||
|
||||
| Here | In an instance |
|
||||
|---|---|
|
||||
| No `.wikitool-release.json`: telemetry is **on** - the traces are the stack's measuring instrument ([telemetry-and-evaluation.md](../../docs/explanation/telemetry-and-evaluation.md#whether-it-runs-at-all)) | Telemetry is off until the operator turns it on |
|
||||
| `USER.md`/`SOUL.md` describe a demo operator and persona | Written by the operator during setup |
|
||||
| `tools/wikitool dist upgrade` refuses: there is no stamp to compare against. The checkout follows `main` with `tools/wikitool sync` | Updated with `dist upgrade --latest` |
|
||||
| `instructions/dev/`, `docs/dev/`, `commonplace/` and `.gitea/` are present | Never shipped |
|
||||
|
||||
5. **Use `dist export` as a build and test tool.** It writes exactly the tree a release ships, so
|
||||
it is how a change to the shipped surface is looked at before it is released:
|
||||
|
||||
```bash
|
||||
tools/wikitool dist export <empty scratch folder> --dry-run
|
||||
tools/wikitool dist export <empty scratch folder>
|
||||
```
|
||||
|
||||
To replay the install path the way a user meets it, build the release tarball from that tree
|
||||
the way `.gitea/workflows/release.yml` does (one top-level folder, a `.sha256` beside it) and
|
||||
start the tree's `tools/preflight.sh` as the asset, from an empty folder, with
|
||||
`--archive <tarball>` - the step "The distribution works as a fresh instance" in
|
||||
`.gitea/workflows/ci.yml` is that replay and the reference for it. Keep scratch trees outside
|
||||
this checkout.
|
||||
|
||||
## Decision points
|
||||
|
||||
- **Asked to set up an instance from this checkout** (`dist export` into the user's folder, or a
|
||||
clone that "becomes" their wiki)? Neither is an install path. An instance is installed from a
|
||||
release, by [setup-instance.md](../setup-instance.md); a stack state that has no release yet is
|
||||
released first, or tested with the replay in step 5 and thrown away.
|
||||
- **The demo corpus is in the way of a test?** Do not delete or rewrite corpus content to make
|
||||
room: [corpus-policy.md](corpus-policy.md) says what may be changed and how. Use a scratch
|
||||
export (step 5) for a clean tree instead.
|
||||
|
||||
## Scope
|
||||
|
||||
Not for operating an instance, and not for the release workflow itself (`docs/dev/development.md` for
|
||||
humans, [version-parts.md](version-parts.md) for the version part). Not shipped: `dist export`
|
||||
prunes `instructions/dev/` wholesale.
|
||||
@@ -0,0 +1,90 @@
|
||||
---
|
||||
type: types/instruction.md
|
||||
name: doc-pull-through
|
||||
description: Which document makes a claim about a surface you are about to change - a wikitool command's behaviour, a stage's rules, an AGENTS.md rule/gate/invariant, a README-shaped human doc, a docs/ page's reasoning - and so needs updating in the same session, since tools/wikitool docs verify never reads a cell's prose.
|
||||
---
|
||||
# Update every document that makes a claim about the surface you changed
|
||||
|
||||
`tools/wikitool docs verify` is a hard oracle over presence, not content: it checks that a
|
||||
command is *listed*, that a contract *exists*, that an ignore canary is (or isn't) caught - never
|
||||
what a table cell, a contract section, or a README paragraph actually *says*. A command's flag
|
||||
can change, a gate's threshold can move, a contract's wording can go false, and every one of
|
||||
those checks stays green (Gitea #90; Gitea #91 narrows what the command-table check matches, but
|
||||
adds no reading of cell content). Content quality of every document below is therefore session
|
||||
work, the same duty AGENTS.md's Changelog section states for `README.md`, `tools/README.md` and
|
||||
the pages under `docs/` - this instruction exists because that duty used to stop at the human docs
|
||||
while the contracts rotted next to a green check (ten stale error-contract rows accumulated this
|
||||
way; see Gitea #89 for one).
|
||||
|
||||
## When to run
|
||||
|
||||
Before `tools/wikitool docs verify`/`publish` in a build session that changed behaviour -
|
||||
`stack-build` step 5 sends you here. Read the table below and update every row whose surface you
|
||||
touched; a row that does not apply needs no action.
|
||||
|
||||
## Steps
|
||||
|
||||
1. **Name the surface(s) you changed.** A `wikitool` command's flags or behaviour, a stage's
|
||||
rule, an `AGENTS.md`-level rule/gate/invariant, a workflow a human runs by hand, or the
|
||||
reasoning behind a design decision - one change can touch more than one row.
|
||||
|
||||
2. **For each surface, update every document the table names - not only the one you were already
|
||||
editing:**
|
||||
|
||||
| Touched surface | Document(s) that make a claim about it |
|
||||
|---|---|
|
||||
| A `wikitool` command's behaviour, flags, or interface | Its `cli_contract.CommandRecord` (name, synopsis, properties, exit status, retry policy - `tools/chemenu/cli_contract.py`), then `wikitool docs contract --apply` to regenerate its copy in [tools/CONTRACT.md](../../tools/CONTRACT.md) and `wikitool docs reference --apply` for the human reference under `docs/reference/` (a new group in `cli_contract.GROUPS` first needs its page there, with a title, a one-sentence head and the marker pair) |
|
||||
| A stage's authoring rules (`raw/`, `kb/`, `types/`, `reports/`, `work/`, `tools/`, `instructions/`) | The touched `<stage>/CONTRACT.md` |
|
||||
| A rule, gate, or invariant `AGENTS.md` itself states | The relevant `AGENTS.md` section (Invariants, Gates, File naming, Routing, ...) |
|
||||
| A workflow, stage, or command a human operates by hand | Whichever of `README.md`, `tools/README.md`, `INSTALL.md` or the pages under `docs/tutorials/`, `docs/how-to/` and `docs/reference/` names it - AGENTS.md § File naming says which document is for which reader. A new page goes into the area its kind of documentation belongs to; that area's `README.md` says which |
|
||||
| The reasoning behind a gate, boundary, or design decision | The `docs/explanation/` page that carries it, if one exists (AGENTS.md § File naming lists all six reached from AGENTS.md itself, plus a seventh reached only from CLAUDE.md). **A decision with no page yet is the gap worth closing**: reasoning that lives only in a Gitea issue never ships - `dist export` carries `docs/explanation/` and no issue tracker, so a distributed instance gets the mechanism without the why |
|
||||
| A task-tracker adapter (`tools/chemenu/tasks/`), its recorded fixtures, or the live suite | [instructions/dev/tracker-testing.md](tracker-testing.md), and `MANIFEST.json` beside the fixtures when they were re-recorded |
|
||||
| A skill's own step sequence or catalogue | The skill's `SKILL.md` source under `instructions/<name>/` or `instructions/dev/<name>/`; for a shipped skill (not under `instructions/dev/`) whose name or `description` changed, or one added or removed, `wikitool docs reference --apply` for the skill catalogue `docs/reference/skills.md` |
|
||||
| A per-checkout configuration file an instance owns (`.wikitool-tasks.json`, `.wikitool-telemetry.json`, `.wikitool-remotes.json`, `.wikitool-upload.json`) | Its shape - a field, a default, what is required or tolerated - is its record in `tools/chemenu/config_shapes.py`, which the loader reads; change it there, then `wikitool docs reference --apply` for its region in [docs/reference/configuration.md](../../docs/reference/configuration.md). What a setting *does* is the hand-written prose around that region. Also the [setup-instance.md](../setup-instance.md) decision point that offers the file during setup, and `doctor`'s own row in [tools/CONTRACT.md](../../tools/CONTRACT.md), since `doctor` is what reports the file's state |
|
||||
| An installation instruction - [preflight.md](../preflight.md), [setup-instance.md](../setup-instance.md), [bootstrap.md](../bootstrap.md), [upgrade-instance.md](../upgrade-instance.md) - or `tools/prerequisites.txt` | [INSTALL.md](../../INSTALL.md), the human guide to the same procedure - for `upgrade-instance.md`, [docs/how-to/update-an-instance.md](../../docs/how-to/update-an-instance.md) instead. Read it against the instruction: what to prepare, the sentence for the agent, what the agent asks, where it stops and why. `docs verify` checks only the two enumerable overlaps - the prerequisites lists, which `wikitool docs prerequisites --apply` regenerates from the manifest, and the setup questions: a question the agent asks the user carries `<!-- setup-question: <key> -->` where it is asked in `setup-instance.md`, and `INSTALL.md` § "What the agent asks you" names it with the same marker. Every other sentence is this session's to compare. `INSTALL.md` does not retell the steps, so a change to their order or wording alone moves nothing there |
|
||||
| [dev-setup.md](dev-setup.md), the release workflow (`version bump`/`version release`/`publish`/CI), or another developer-facing process | [docs/dev/development.md](../../docs/dev/development.md), read against it the same way - nothing checks this pair at all. A page under `docs/dev/` is never shipped, so it may link into `instructions/dev/` and cite issue numbers |
|
||||
| A new page type the stack requires, or a new collection | Its type-spec and `COLLECTION.md` (both as the `.template` an instance adopts), the collection table in [kb/CONTRACT.md](../../kb/CONTRACT.md), and **both adoption paths**: [setup-instance.md](../setup-instance.md) for a fresh instance and [upgrade-instance.md](../upgrade-instance.md) for an existing one, where an unadopted template is what `docs verify` refuses. A new or changed `root: kb` type-spec or its schema - description, `layout:`, subtype enum, required fields - also needs `wikitool docs reference --apply` for `docs/reference/page-types.md` |
|
||||
|
||||
3. **A heading you changed means a table of contents to regenerate - by the tool, never by
|
||||
hand.** Every reference file over 100 lines carries one (`AGENTS.md`, the stage contracts,
|
||||
`kb/CONVENTIONS.md`, each `COLLECTION.md`, the flat `instructions/**.md` form, the
|
||||
type-specs, the `docs/` pages in all five areas - each with the `<name>.template` it ships
|
||||
as, where one exists, and a `SKILL.md` the one exception). Adding, renaming, reordering or
|
||||
deleting a `##`/`###` heading in one of them makes its region stale, and
|
||||
`docs verify` fails on stale exactly as it fails on missing:
|
||||
|
||||
```bash
|
||||
tools/wikitool docs toc # dry run: which files would change
|
||||
tools/wikitool docs toc --apply # write them
|
||||
```
|
||||
|
||||
The region is generated, so AGENTS.md invariant 1 applies to it like any other: editing the
|
||||
list by hand is the failure, not the fix - and a hand-written entry survives until the next
|
||||
`--apply` silently disagrees with it. It is cheap to over-run: `--apply` is idempotent and a
|
||||
file whose headings did not move is left untouched.
|
||||
|
||||
4. **Do not re-derive what `docs verify` already checks mechanically** - existence, table-row
|
||||
membership, ignore-canary state. That enumeration lives once, in
|
||||
[tools/CONTRACT.md](../../tools/CONTRACT.md)'s own `docs verify` row; copying it here would be a
|
||||
second copy that drifts, the exact failure this instruction exists to describe (Gitea #90).
|
||||
This instruction is only about the prose no check reads.
|
||||
|
||||
## Decision points
|
||||
|
||||
- **The change touched no document in the table?** Nothing to do - not every stack change moves
|
||||
a claim. A pure bugfix with an unchanged interface is the common case.
|
||||
- **Unsure whether a `docs/explanation/` page's reasoning moved?** Read it. A `docs/` page carries no
|
||||
normative sentence and nothing verifies it by construction (AGENTS.md § File naming), so an
|
||||
unsure guess defaults to reading the page rather than skipping the question -
|
||||
[`stack-close`](stack-close/SKILL.md) step 3 asks it again in the closing phase as a
|
||||
backstop, not as the only time it is asked.
|
||||
- **The surface is a whole new stage, collection, or gate?** The table's rows are the steady
|
||||
state; a new row-worthy category is itself a change to this instruction - add the row here
|
||||
rather than leaving the next session to rediscover the gap.
|
||||
|
||||
## Scope
|
||||
|
||||
Applies to stack-development sessions only - wiki content changes have their own provenance and
|
||||
cross-reference rules (`kb/CONTRACT.md`, `wiki-manage`), which already pull the relevant pages
|
||||
through as part of the normal skill. Not a replacement for `stack-close` step 3, which re-asks
|
||||
the `docs/`-staleness question after the green CI run as the second, final check.
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: types/instruction.md
|
||||
name: issue-tracking
|
||||
description: Where open work on this stack is tracked, and what the prio/ and size/ labels on a Gitea issue mean.
|
||||
description: Where open work on this stack is tracked, what the four mandatory area/kind/prio/size labels and the three status flags on a Gitea issue mean, why a status/incoming stub is never implemented as it stands, and how to keep an issue body current across sessions.
|
||||
---
|
||||
# Track open work as Gitea issues, not as prose in the repo
|
||||
|
||||
@@ -20,14 +20,40 @@ This instruction exists only in the dev repo. A distributed instance has no
|
||||
issues at that URL, which is exactly why `dist export` excludes
|
||||
`instructions/dev/` wholesale (see [tools/CONTRACT.md](../../tools/CONTRACT.md)).
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
|
||||
- [When to run](#when-to-run)
|
||||
- [Steps](#steps)
|
||||
- [Incoming stubs](#incoming-stubs)
|
||||
- [Ready to build](#ready-to-build)
|
||||
- [Renames and other decay in the tracker](#renames-and-other-decay-in-the-tracker)
|
||||
- [Citing an issue in the repo](#citing-an-issue-in-the-repo)
|
||||
- [What no tool checks](#what-no-tool-checks)
|
||||
- [Decision points](#decision-points)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
## When to run
|
||||
|
||||
- Something is worth doing but not now. Open an issue; do not write it down in
|
||||
the repo.
|
||||
- A session's findings outgrow the change it was making - a gap in the tooling,
|
||||
an assumption nobody has checked, a decision that needs the user.
|
||||
- Picking an issue up: before doing anything else, read the body as the current
|
||||
spec, and re-label it if the ground has moved since.
|
||||
- **The issue carries `status/incoming`:** it is a human's stub, not a spec, and
|
||||
it is worked out and triaged before anything is built from it
|
||||
(§ Incoming stubs).
|
||||
- **While working on one:** the body is updated as the state moves, not at the
|
||||
end (step 2). A session that is interrupted leaves the body as its handover.
|
||||
- Ending a design session, or starting a build from a body: check that it is
|
||||
ready (§ Ready to build).
|
||||
- Prioritising: deciding what to pick up next, or re-labelling after the ground
|
||||
moved.
|
||||
- Closing one: the body is rewritten to its final state first, and only then
|
||||
closed (step 7).
|
||||
- A rename or move ships: sweep the open issues for text that assumed the old
|
||||
name or path (§ Renames and other decay in the tracker).
|
||||
|
||||
## Steps
|
||||
|
||||
@@ -36,38 +62,359 @@ issues at that URL, which is exactly why `dist export` excludes
|
||||
specific files or commands involved. An issue that only makes sense to
|
||||
whoever wrote it is a note, and notes were the problem.
|
||||
|
||||
2. **Give it exactly two labels: one `prio/`, one `size/`.** Both, always -
|
||||
a priority without a cost is half a decision. Neither is a promise about
|
||||
*when*; together they answer "what should I pick up in the time I have".
|
||||
**Destructive steps carry the invariant they must not violate.** A body
|
||||
that prescribes a mechanism gets built as prescribed - including its
|
||||
bugs. Where a step deletes, overwrites, resets or moves, name the
|
||||
property that must still hold afterwards, not only the command that gets
|
||||
there. "Remove the working directory, then `git checkout HEAD --
|
||||
<stage>`" is a mechanism; "the content stages must afterwards match
|
||||
`HEAD` exactly, without any untracked or ignored file being touched" is
|
||||
the same instruction plus its test - a build instruction and an
|
||||
acceptance criterion at once, so the defect surfaces while the test is
|
||||
written rather than in review afterwards. #30's `upstream merge` body
|
||||
wrote the mechanism and got exactly that bug: a working-directory removal
|
||||
that took a stage's gitignored, unrecoverable data with it.
|
||||
|
||||
| Priority | Means |
|
||||
**An acceptance criterion states a checkable property, not an activity.**
|
||||
"Implement X" is done when someone says so; "after `upstream merge`,
|
||||
`reports/` still holds every file it held before" is done when it is
|
||||
true. This is not a ban on imperative steps - a numbered procedure can
|
||||
still produce a correct control flow, and that is its merit - it binds
|
||||
the destructive steps, and every box in the criteria list.
|
||||
|
||||
2. **The body is the working state, not a historical first post - keep it
|
||||
current as you go.** It is this stack's plan file: the same thing a harness's
|
||||
own plan document is, and it is maintained the same way. Not written once,
|
||||
not brought up to date at the end, but **updated whenever something in it
|
||||
stops being true** - a decision made, a criterion met, an approach ruled out,
|
||||
a new constraint found.
|
||||
|
||||
The test is an abort, not a milestone. A session can end at any moment - an
|
||||
interrupt, a context limit, a crash, a human walking away - and whatever the
|
||||
body says at that instant is the entire handover. So the standard is: **at
|
||||
every point, a fresh session must be able to open the body and pick the work
|
||||
up from there**, without a human re-explaining it and without reading back
|
||||
through the comments. If the body would mislead someone who read it right
|
||||
now, it is already out of date, whether or not the work is finished.
|
||||
|
||||
That means updating *during* the work, not only at its end:
|
||||
|
||||
- a decision gets made → the decision and its reasoning replace the question
|
||||
- an acceptance criterion is done → tick it, in the same session that did it
|
||||
- something turns out differently than the issue assumed → the assumption is
|
||||
corrected where it stands, not contradicted three paragraphs later
|
||||
- work is deferred or dropped → say so, with the reason, where the criterion is
|
||||
|
||||
**Rewrite, never append.** Do not add to a text that has become wrong: an
|
||||
additively grown log forces every later reader to reconstruct the current
|
||||
state by filtering the whole history, which is the exact cost the body exists
|
||||
to remove. Comments carry the history (step 3); the body carries the state.
|
||||
|
||||
Body rewrites and comments are an LLM session's job. A human normally
|
||||
touches only labels and metadata directly.
|
||||
|
||||
**Reading an issue, the body is the state and comments are history.** A
|
||||
session picking an issue up reads the body as the spec; comments are read
|
||||
for provenance - why something was decided, what was tried - never as
|
||||
the current instruction. A recommendation in a comment can be older than
|
||||
the body's decision and read just as convincingly: on #30 an earlier
|
||||
comment recommended a smaller, `verify`-only command, while the body had
|
||||
since settled on building the full `merge` command. A session trusting
|
||||
the comment would have built the wrong thing, with a plausible
|
||||
justification out of this repo's own tracker.
|
||||
|
||||
**A body that is demonstrably wrong is corrected first, not worked
|
||||
around.** "Body beats comment" is a rule of precedence, not a licence to
|
||||
execute a stale spec. Where a comment or the tree proves a claim in the
|
||||
body false, the body is rewritten before the work starts - the rewrite
|
||||
above is the fix; leaning on the comments as the "real" state is not.
|
||||
#10 is the case: its body claimed coverage had never been measured while
|
||||
three comments carried a percentage, a statement count and a CI run
|
||||
number.
|
||||
|
||||
**Where two comments contradict each other, evidence decides, not
|
||||
recency.** On #10, one comment showed a retrieved artifact with zero
|
||||
items on a finished run - the report was not actually retrievable - and
|
||||
a later comment declared the same criterion met without re-checking. The
|
||||
later comment is not the newer truth, only the unchecked one. Resolve it
|
||||
into the body with the evidence named, or mark the point open.
|
||||
|
||||
3. **Comment a changelog, never a copy.** A body rewrite gets one short comment
|
||||
naming only what changed against the previous state - what is new, what is
|
||||
gone, what was corrected. Do not snapshot the old body into a comment: a full
|
||||
copy per revision forces a human to diff two prose texts, which is not a
|
||||
readable history, only another copy.
|
||||
|
||||
One comment per *session's worth* of change, not per edit. Step 2 asks the
|
||||
body to be kept current continuously, and a comment for every tick would bury
|
||||
the board in noise; the changelog line summarises what that session moved.
|
||||
Trivial upkeep - a typo, a tightened sentence - needs no comment at all.
|
||||
|
||||
```
|
||||
**Changelog:** Decision 2 tightened - `kind/` may now change over an
|
||||
issue's life. Old acceptance criterion 3 dropped (covered by #42).
|
||||
```
|
||||
|
||||
4. **Give it all four mandatory labels: one `area/`, one `kind/`, one `prio/`,
|
||||
one `size/`.** All four, always. Machine maintenance by an LLM session is
|
||||
what makes four axes affordable - the original objection to a third and
|
||||
fourth axis was the upkeep cost for a single human maintainer, and that
|
||||
objection no longer holds.
|
||||
|
||||
| `area/` | Means |
|
||||
|---|---|
|
||||
| `prio/1` | Blocks or damages work in progress. Next. |
|
||||
| `prio/2` | Accrues interest. Planned. |
|
||||
| `prio/3` | Worth doing, waiting on a trigger. |
|
||||
| `area/kb` | The `kb/` schema, contract, provenance machinery, lint - the knowledge base as a system. |
|
||||
| `area/distribution` | Shipping, upgrading and versioning an instance. |
|
||||
| `area/corpus` | The content and scope of `kb/` in this instance, and the demo/testbed question. |
|
||||
| `area/workflow` | Git, merging, branching, publish, PRs. |
|
||||
| `area/process` | The development process itself, rather than the stack as an artefact. |
|
||||
|
||||
`prio/3` is not a graveyard. It means the issue's value is real but gated on
|
||||
something outside it - a decision, another issue, a second instance
|
||||
There is deliberately no `area/tools`: tooling is filed under the domain it
|
||||
serves, not under where its code sits. The axis follows the stage split in
|
||||
[AGENTS.md](../../AGENTS.md).
|
||||
|
||||
| `kind/` | Means |
|
||||
|---|---|
|
||||
| `kind/decision` | Waiting on an operator decision. |
|
||||
| `kind/build` | Specified; waiting only on implementation time. |
|
||||
| `kind/defect` | A finding: documentation and reality, or two documents, contradict each other. |
|
||||
|
||||
`kind/` is expected to change over an issue's life - `decision` becomes
|
||||
`build` once the decision is made. That is session memory working, not a
|
||||
labelling failure.
|
||||
|
||||
| `prio/` | Means |
|
||||
|---|---|
|
||||
| `prio/blocking` | Blocks or damages work in progress. Next. |
|
||||
| `prio/planned` | Accrues interest. Planned. |
|
||||
| `prio/waiting` | Worth doing, waiting on a trigger. |
|
||||
|
||||
`prio/waiting` is not a graveyard. It means the issue's value is real but
|
||||
gated on something outside it - a decision, another issue, a second instance
|
||||
existing. Name that trigger in the issue, or the label is a polite no.
|
||||
|
||||
| Size | Means |
|
||||
| `size/` | Means |
|
||||
|---|---|
|
||||
| `size/XS` | Minutes. Often just a decision or an observation to record. |
|
||||
| `size/S` | One session, one publish, a clear cut. |
|
||||
| `size/M` | Several files; a contract or instruction change; its own test effort. |
|
||||
| `size/L` | Several sessions, or open design questions before the first commit. |
|
||||
|
||||
Size is effort, not importance. A `prio/1 size/XS` is the best thing on the
|
||||
board; a `prio/3 size/L` is a thing to talk about before anyone starts.
|
||||
Size is effort, not importance. A `prio/blocking size/S` is the best thing
|
||||
on the board; a `prio/waiting size/L` is a thing to talk about before anyone
|
||||
starts.
|
||||
|
||||
3. **Re-label when the ground moves, and say why in a comment.** A trigger that
|
||||
fired turns `prio/3` into `prio/2`. A design question that got answered can
|
||||
drop a size. Silent re-labelling is how a board stops meaning anything.
|
||||
5. **Add a `status/` flag only when it applies.** All three are optional, because
|
||||
each describes a temporary condition rather than a property every issue has.
|
||||
|
||||
4. **Close with what actually happened**, not with a commit hash alone: which
|
||||
proposals were implemented, which were deliberately left out and why, and
|
||||
what was verified. The issue is the only place that record survives - a
|
||||
changelog entry says what changed, not what was decided against.
|
||||
| `status/` | Means |
|
||||
|---|---|
|
||||
| `status/blocked` | Waiting on another, still-open issue - not workable on its own, whatever its `prio/` says. Name the blocking issue in the body. |
|
||||
| `status/unconfirmed` | A reported suspicion, not yet checked against actual behaviour. Applies to any `kind/`, not just `kind/defect`. |
|
||||
| `status/incoming` | A human's stub: a request or a thought, filed at whatever length it arrived, deliberately short of everything step 1 asks for. **Never implemented as it stands** - § Incoming stubs. |
|
||||
|
||||
While `status/unconfirmed` is set, `size/` and `prio/` are provisional. Triage
|
||||
ends it one of two ways: the flag comes off and `size`/`prio` are set for
|
||||
real, or the issue is closed with the reason. An unverified suspicion does not
|
||||
stay open indefinitely - the process-level analogue of AGENTS.md invariant 3.
|
||||
|
||||
`status/incoming` is the one flag that **suspends step 4** rather than
|
||||
qualifying it. The four mandatory labels are not missing from such an issue,
|
||||
they are not yet due: `area/` may be obvious, but `kind/`, `prio/` and `size/`
|
||||
are answers to questions the stub has not been read against the tree to
|
||||
settle. Labelling it all four on sight is the failure, not the omission - it
|
||||
makes an unexamined stub look triaged. It is also the one flag a session never
|
||||
*adds*: an issue a session files meets step 1 or it does not get filed.
|
||||
|
||||
6. **Re-label when the ground moves, and say why in a comment.** A trigger that
|
||||
fired turns `prio/waiting` into `prio/planned`. A design question that got
|
||||
answered can drop a size and move `kind/decision` to `kind/build`. Silent
|
||||
re-labelling is how a board stops meaning anything.
|
||||
|
||||
7. **Closing is the last body update, not a comment.** If step 2 was followed
|
||||
the body is already nearly there, and closing only settles what the final
|
||||
run established. If it was not, closing is where the whole debt comes due -
|
||||
and it comes due at the worst moment, because a closed body is the version
|
||||
everyone reads afterwards and nobody revisits.
|
||||
|
||||
Either way the body reaches its final state *before* the issue closes:
|
||||
proposals that were decided read as decided, a "to decide" section has become
|
||||
the decision with its reasoning, acceptance criteria are ticked or struck with
|
||||
a reason, and what was verified is named. Then close, with the one-line
|
||||
changelog comment step 3 asks for.
|
||||
|
||||
Record what actually happened, not a commit hash alone: which proposals were
|
||||
implemented, which were deliberately left out and why, and what was verified.
|
||||
The issue is the only place that record survives - a changelog entry says
|
||||
what changed, not what was decided against.
|
||||
|
||||
**A closing report in a comment does not satisfy this.** It reads as
|
||||
complete to whoever writes it and leaves a body still phrased as open work:
|
||||
unticked boxes, an undecided decision section, present tense about a defect
|
||||
that no longer exists. #44 closed exactly that way, with a thorough comment
|
||||
above a body that still asked for a decision that had already been made and
|
||||
shipped. Nothing mechanical catches it (see below), which is why it is a step
|
||||
rather than a habit.
|
||||
|
||||
## Incoming stubs
|
||||
|
||||
**A `status/incoming` issue is never implemented as it stands.** It is worked
|
||||
out and triaged first, in a session, and only the result of that is built.
|
||||
|
||||
The flag exists because the tracker is also the human's inbox, and the two have
|
||||
different entry costs. Step 1 asks for a body that survives without its author -
|
||||
acceptance criteria, files, commands - and a thought worth keeping is not worth
|
||||
that much work at the moment it occurs. So a stub is admitted at whatever
|
||||
quality it arrives, and `status/incoming` is the receipt: this text was not held
|
||||
to step 1, and nobody should read it as if it had been.
|
||||
|
||||
That is the whole danger. A stub *looks* like a body, and a body is what a
|
||||
session trusts (step 2). What it actually holds is a symptom or a wish - #60
|
||||
says the confidence defaults "feel too high", #61 says a mechanism from one
|
||||
instruction "would be interesting" elsewhere. Neither states what done means,
|
||||
and the parts they leave out are exactly the parts the human left to be worked
|
||||
out. Building straight from one produces something that matches the sentence,
|
||||
misses the intent, and closes the issue - so the question the stub was standing
|
||||
in for is never asked again. It is step 1's mechanism-versus-invariant lesson
|
||||
one stage earlier: there, a body prescribed a mechanism and got its bugs built;
|
||||
here, a body prescribes nothing at all and gets the gap filled by whoever read
|
||||
it fastest.
|
||||
|
||||
Working one out:
|
||||
|
||||
1. **Read the stub as a statement of intent, not a specification.** Its wording
|
||||
is the only evidence of what was actually asked for. Reinterpret it and the
|
||||
record of the request is gone - what remains is the session's reading of it,
|
||||
indistinguishable from the human's.
|
||||
2. **Check it against the tree before rewriting anything.** A stub may be a
|
||||
suspicion (`status/unconfirmed` applies on top where it is), a duplicate of
|
||||
something already built, or a premise that no longer holds. This is the step
|
||||
that decides which of the two exits below the issue takes.
|
||||
3. **Quote the stub verbatim in the elaboration comment, then rewrite the body.**
|
||||
Step 2's "rewrite, never append" holds here as everywhere - but the rewrite
|
||||
overwrites the only record of the request, and comments are where history
|
||||
lives (step 3). Here the history *is* the request.
|
||||
4. **Name the open questions; do not answer them.** Where the stub leaves
|
||||
something a session cannot settle from the tree, it stays a question in the
|
||||
body and the issue becomes `kind/decision`. Guessing turns the human's open
|
||||
question into a spec that reads as decided, which is worse than the stub was:
|
||||
the stub at least announced that it was incomplete.
|
||||
5. **Then step 4 comes due** - all four mandatory labels, set against a body that
|
||||
has been read against the tree. That is the moment the stub becomes a work
|
||||
package.
|
||||
6. **Remove `status/incoming`** and leave the one-line changelog comment step 3
|
||||
asks for.
|
||||
|
||||
Triage ends a stub one of two ways, the same two `status/unconfirmed` has: it is
|
||||
worked out, labelled and the flag comes off, or it is closed with the reason. A
|
||||
stub does not sit in the inbox indefinitely.
|
||||
|
||||
Elaboration touches no file in the working tree, so it needs no version bump and
|
||||
no publish - it is tracker work, and several stubs can be worked out in one pass.
|
||||
What comes *after* it is an ordinary work package, picked up on its merits like
|
||||
any other.
|
||||
|
||||
## Ready to build
|
||||
|
||||
A body is **ready** when a session that has never seen the design conversation can build from
|
||||
it alone. That is the handover from design to build ([stack-mode.md](stack-mode.md) § The
|
||||
three phases): `stack-dev` ends by checking it, and `stack-build` checks it again as its first
|
||||
step. All four hold:
|
||||
|
||||
- **The body says what will be built**, and its acceptance criteria are checkable properties
|
||||
(step 1), not activities.
|
||||
- **No open question is left in it.** It carries `kind/build`, and neither `status/incoming`
|
||||
nor `status/unconfirmed`. A question that is genuinely not blocking may stay, marked as such
|
||||
and with the answer's consequence stated either way - "check X while building; if it does
|
||||
not hold, do Y" is a decision, "X is unclear" is not.
|
||||
- **The version part is named** ([version-parts.md](version-parts.md)). Whether a change is a
|
||||
drop-in replacement is a judgment with no mechanical guard, so it is made while designing, not
|
||||
discovered at the bump.
|
||||
- **The files or surfaces involved are named**, so the build starts from the tree rather than
|
||||
from a search for where the change belongs.
|
||||
|
||||
A body that fails one of these goes back to design. It is not built around: the gap a build
|
||||
session fills on its own is the same gap a `status/incoming` stub leaves (§ Incoming stubs),
|
||||
only better disguised. The useful side effect of the cut between the two phases is that it
|
||||
tests this definition - if a cold session cannot build from the body, it was not ready.
|
||||
|
||||
## Renames and other decay in the tracker
|
||||
|
||||
A rename is not finished when the tree is green. Renaming a package, a path,
|
||||
a command, a flag or the repository itself moves text that lives outside the
|
||||
working tree, and the open issues are the largest such text. Nothing catches
|
||||
them - `wikitool` does not know this tracker exists and must not learn (see
|
||||
"What no tool checks" below) - so a pass over the open issues is part of the
|
||||
rename, in the session that did it, not a follow-up someone remembers.
|
||||
|
||||
Distinguish a wayfinder from a piece of evidence: a path meant to point at
|
||||
where something *is* gets pulled through; a path quoted for what was true at
|
||||
a time is left standing and dated. Note per corrected body what was pulled
|
||||
through and when, so the next pass can tell a checked body from one that
|
||||
merely looks right. Closed issues are out of scope - they guide nobody.
|
||||
|
||||
Renames are not the only thing that ages an issue text. A page a body cites
|
||||
can vanish from `kb/` (`wikitool search` against the cited titles is the
|
||||
second pass), and an old body can carry private infrastructure detail into
|
||||
what is now a public tracker - both found in the same issue, both worth the
|
||||
same look.
|
||||
|
||||
## Citing an issue in the repo
|
||||
|
||||
**No file `dist export` ships may cite an issue number.** The board is reachable only from the
|
||||
origin repo, and this very file - the only one that says where it lives - is pruned along with
|
||||
the rest of `instructions/dev/`. A "#66" that survives into a distributed instance is therefore
|
||||
worse than a dead link: the reader cannot resolve it *and* cannot tell that it is unresolvable,
|
||||
so a rule appears to rest on evidence nobody can produce. `instructions/CONTRACT.md` § "Writing
|
||||
an instruction" asks the opposite ("self-contained enough for an agent with no prior context"),
|
||||
and an issue number is the exact counter-example to it.
|
||||
|
||||
Which is the same wayfinder/evidence split as in the section below, applied one layer out - but
|
||||
both halves land in the same place here:
|
||||
|
||||
- **A wayfinder** ("see #66 for the reasoning") is resolved: the reasoning goes into the text,
|
||||
and the number goes.
|
||||
- **A piece of evidence** ("removed in #66") is dated in words instead - "removed when the
|
||||
schema default was dropped". The sentence carries itself, and the number stays reachable
|
||||
through `git blame` -> the commit message, which names the issue anyway.
|
||||
|
||||
Where a pointer is genuinely worth having *here* and would leave nothing behind in words, keep
|
||||
it in a `<!-- dist:strip-start/end -->` block ([instructions/CONTRACT.md](../CONTRACT.md)
|
||||
§ `instructions/dev/`): visible in this repo, removed on export. Two passages use it today.
|
||||
|
||||
`tools/**/*.py` is deliberately outside all of this. A code comment addresses whoever edits that
|
||||
line, and that only ever happens in the origin repo, because `dist export` prunes the
|
||||
`stack-` skills together with this directory; a distributed `tools/` tree is runtime
|
||||
machinery, not reading material. The same holds for `.gitignore` and `tools/.coveragerc` -
|
||||
config, not documentation.
|
||||
|
||||
`docs verify` enforces the rule (below).
|
||||
|
||||
## What no tool checks
|
||||
|
||||
`wikitool` does not know this tracker exists, and should not learn. It ships to
|
||||
instances that have no issues at that URL, while this file and the workflow it
|
||||
describes are pruned by `dist export` - a Gitea client inside the shipped tool
|
||||
would be a dev-only dependency carried by every instance, to check a board none
|
||||
of them have. The tracker is reachable only through the `gitea-mcp` server, in a
|
||||
session, by an agent.
|
||||
|
||||
So there is no `docs verify` for the board. Nothing reports a closed issue whose
|
||||
body still reads as open, a body that contradicts its own comments, an issue
|
||||
missing one of the four mandatory labels, or a `status/incoming` stub that got
|
||||
built as it stood. Every one of those is caught by a
|
||||
session following this file, or not at all - which is the argument for the
|
||||
sequence in step 7 being explicit about the order (body first, then close),
|
||||
rather than leaving it to be inferred from step 2.
|
||||
|
||||
The one rule here that *is* checked is § Citing an issue in the repo, and it is
|
||||
worth being clear about why that is not a contradiction. `docs verify`'s
|
||||
`check_no_issue_references` compiles `#\d+` and reads the text
|
||||
`dist_cmd.build_plan()` would write. It has no client, no URL and no notion of
|
||||
an issue's state - it cannot tell an open issue from a closed one, or a real
|
||||
number from an invented one. What it knows is that a shipped document is making
|
||||
a reference its reader cannot follow, which is a property of the *document*, not
|
||||
of the board. That is the line: a check may look at what this repo writes about
|
||||
the tracker; none may look at the tracker.
|
||||
|
||||
## Decision points
|
||||
|
||||
@@ -75,8 +422,19 @@ issues at that URL, which is exactly why `dist export` excludes
|
||||
what shipped. A finished change needs both: the entry, and the issue closed
|
||||
with the reasoning.
|
||||
- **Issue or `kb/` page?** An issue is about *this stack* and is ephemeral - it
|
||||
closes. A `kb/` page is compiled knowledge that stays true. Never put wiki
|
||||
content findings in an issue, and never file a work item as a page.
|
||||
- **Two labels feel too coarse?** They are meant to. A third axis - kind, area,
|
||||
status - is the point at which a taxonomy starts needing maintenance of its
|
||||
own, and this board has one maintainer.
|
||||
closes, and it records a wish. A `kb/` page is verified knowledge that stays
|
||||
true. Never put wiki content findings in an issue, and never file a work item
|
||||
as a page.
|
||||
- **Rewrite the body, or add a comment?** Rewrite whenever a reader of the body
|
||||
alone would otherwise be misled - a changed decision, a dropped criterion, a
|
||||
new constraint. A comment carries the changelog line for that rewrite, and
|
||||
nothing else that a future session needs in order to act. Closing an issue is
|
||||
always a rewrite - see step 7.
|
||||
- **A `status/incoming` stub looks trivially implementable?** Work it out anyway.
|
||||
"Trivial" is a judgement about the sentence, and the sentence is the part the
|
||||
human wrote down cheaply; what it omits is not visible from it. The elaboration
|
||||
of an obvious stub is short - that is the argument for doing it, not for
|
||||
skipping it.
|
||||
- **An old issue carries only `prio/` and `size/`?** Complete it to all four
|
||||
when you touch it, rather than in a sweep. The board reaches the new scheme
|
||||
issue by issue, as each is picked up.
|
||||
@@ -0,0 +1,59 @@
|
||||
---
|
||||
type: types/instruction.md
|
||||
name: publish-and-ci
|
||||
description: How a stack change is published and how its CI run is waited for - the local checks first, tools/wikitool publish, reading the runs through the authenticated Gitea connection rather than curl, how long to wait, when to give up, and why a red run means the build phase is not over.
|
||||
---
|
||||
# Publish a stack change and wait for its CI run
|
||||
|
||||
The local checks cover what they cover on this machine; CI covers the same checks plus a full
|
||||
`setup-instance.md` replay against a fresh `dist export` (`.gitea/workflows/ci.yml`). A green run
|
||||
on the published commit is therefore the end of the checked stretch of a work package, not a
|
||||
formality after it - which is why waiting for it belongs to the phase that wrote the code
|
||||
(`stack-build`), and a red run sends the work back there rather than into the closing phase.
|
||||
|
||||
## When to run
|
||||
|
||||
- `stack-build`, every time it publishes - its last step.
|
||||
- `stack-close`, when its own pull-through of a stale document publishes something of its own
|
||||
(that skill's step 3 says when).
|
||||
|
||||
## Steps
|
||||
|
||||
1. **Run the local checks, explicitly rather than assumed:**
|
||||
|
||||
```bash
|
||||
tools/wikitool docs verify
|
||||
tools/wikitool instructions verify
|
||||
```
|
||||
|
||||
plus the relevant `pytest` run in `tools/` ([testing-conventions.md](testing-conventions.md)).
|
||||
A red check here is fixed before anything is published.
|
||||
|
||||
2. **Publish with `tools/wikitool publish`.** The gates apply as everywhere (AGENTS.md § Gates);
|
||||
an exit 42 is shown to the user verbatim and waited on. When the changeset touches `tools/`,
|
||||
`types/`, `instructions/`, `AGENTS.md` or a `<stage>/CONTRACT.md`, `publish` prints a
|
||||
one-line note that CI is the last mechanical check still to come and that what follows it is
|
||||
covered by none. A push to `main` that moves `VERSION` additionally triggers a tagged release.
|
||||
**CI does the tagging** - a session never creates a tag, which is what keeps AGENTS.md
|
||||
invariant 5 intact.
|
||||
|
||||
3. **Wait for CI on the published commit, this way.** The push-triggered runs take about
|
||||
**4 minutes**; a push that moved `VERSION` to a suffix-free release adds about **1 minute**
|
||||
for the release job.
|
||||
|
||||
- Read the runs for the published commit's SHA through the authenticated Gitea connection
|
||||
`ENVIRONMENT.md` lists (the Gitea MCP's `actions_run_read`, `list_runs`), right after
|
||||
`publish`, to get their ids. **Never anonymously via `curl`:** Gitea answers the Actions
|
||||
API with `401 token is required` even for this public repo.
|
||||
- Check again after about 4 minutes (5 with a release job), then once a minute.
|
||||
- **Give up after 15 minutes** and hand the open runs to the user by id and link, rather than
|
||||
waiting on.
|
||||
- Any shell loop that polls instead must **end on the first non-2xx status or missing field**
|
||||
and print the raw response. A loop that treats an error as "not finished yet" never ends;
|
||||
that happened in the #139 close-out.
|
||||
- Keep the polling out of the main context where the harness allows it - a background
|
||||
command or a fork - so a long wait does not fill the session with run listings.
|
||||
|
||||
4. **A red run is not waited past.** Read the failing job's log, fix the cause, and go back to
|
||||
step 1: this is still build work, whichever skill published. The phase that published ends
|
||||
only on a green run, and the issue stays open until there is one.
|
||||
@@ -0,0 +1,123 @@
|
||||
---
|
||||
name: stack-build
|
||||
description: Builds a stack work package whose Gitea issue body is ready - code, tests, version bump, document pull-through, publish, and waiting for a green CI run, keeping the issue body current at fixed points along the way. Started only by the operator as /stack-build #N, after stack-dev has ended its design phase.
|
||||
disable-model-invocation: true
|
||||
---
|
||||
|
||||
# Stack Build
|
||||
|
||||
**Purpose:** Carry a designed work package through the checked stretch of its life - from a
|
||||
ready issue body to a green CI run on the published commit - and leave the body current enough
|
||||
that the closing phase, or anyone else, can work from it without this session's context.
|
||||
|
||||
**Trigger:** The operator runs `/stack-build #N`. Nothing else starts this skill: the
|
||||
frontmatter's `disable-model-invocation` keeps Claude Code from invoking it, and in a harness that
|
||||
ignores that key this sentence is the rule. This session may be the continuation of the design
|
||||
session, or a fresh one after `/clear` - assume the second, and work from the body.
|
||||
|
||||
## Steps
|
||||
|
||||
1. **Read `instructions/dev/stack-mode.md`.** It holds the rules that change in a
|
||||
stack-development session and the catalogue of dev-only procedures; a cold session has
|
||||
neither yet.
|
||||
|
||||
2. **Check that the body is ready, before anything else.** Read issue #N's body against
|
||||
`instructions/dev/issue-tracking.md` § Ready to build. If a point fails, stop: name it, and
|
||||
recommend `/stack-dev` to finish the design. **Do not build around the gap** - a build session
|
||||
that answers an open design question on its own produces a change that matches its own
|
||||
reading of the body, recorded nowhere as a decision.
|
||||
|
||||
3. **Build the change and its tests.** Follow `instructions/dev/testing-conventions.md` before
|
||||
adding or changing a test, and the other procedures the mode file names for the surface you
|
||||
touch.
|
||||
|
||||
**"Covered by tests" means covered by the tests that exist, not by the tests that should
|
||||
exist.** Whether the right test was written is itself a judgment call with no mechanical
|
||||
guard: two data-destroying bugs in `upstream merge` (Gitea #30) shipped past a green
|
||||
`pytest`/`docs verify`/`instructions verify`/CI because no test exercised the case, not
|
||||
because the code for the tested case was worse. Where the body names a destructive step, its
|
||||
invariant is a test (issue-tracking.md step 1).
|
||||
|
||||
**Body upkeep, first fixed point - a deviation goes into the body at once.** An assumption
|
||||
that turns out false, a criterion that moves, an approach dropped: rewrite the body where it
|
||||
stands, in this session, not at the end
|
||||
(`instructions/dev/issue-tracking.md` step 2 has the rule; this is where it applies).
|
||||
A deviation that reopens the design - a boundary crossing, a decision the body did not make -
|
||||
is not settled here: stop and put it to the operator, as in step 2.
|
||||
|
||||
4. **Raise the version, if the change ships.** A change under `tools/`, `types/`,
|
||||
`instructions/`, `AGENTS.md` or a `CONTRACT.md` reaches every future instance, so it needs a
|
||||
version and a changelog entry. The part was named in the body during design; bump that part:
|
||||
|
||||
```bash
|
||||
tools/wikitool version bump --patch --title "<what changed>" --impact medium
|
||||
```
|
||||
|
||||
`--impact high|medium|low` (default `medium`) grades this bump in the changelog entry's own
|
||||
list - `tools/wikitool version regrade` corrects it later if the candidate's overall shape
|
||||
changes the read on an earlier one; see `instructions/dev/version-parts.md` § The candidate
|
||||
model.
|
||||
|
||||
Never edit `VERSION` or the entry's heading by hand - `bump` writes both, and `docs verify`
|
||||
fails a tree where they disagree. Then decide the entry's prose: for a smaller change its bump
|
||||
title is the whole entry; a larger one gets a one-paragraph `###` section for its topic, or
|
||||
rewrites the one its topic already has. The form and the budget `docs verify` holds it to are
|
||||
`instructions/dev/version-parts.md` § The candidate model. A `--major` bump needs `--breaking` and either a
|
||||
migration document or `--no-migration`; `instructions/dev/version-parts.md` has all of it.
|
||||
|
||||
Prose-only changes (`README.md`, `INSTALL.md`, the pages under `docs/`) and the workflows under `.gitea/`
|
||||
do not need a bump - CI's version gate is scoped to what changes behaviour.
|
||||
|
||||
5. **Pull through every document that makes a claim about the surface you touched.**
|
||||
`instructions/dev/doc-pull-through.md` has the table of which document that is, per surface,
|
||||
and its step 3 for the one part that is not prose: a reference file whose headings moved needs
|
||||
`tools/wikitool docs toc --apply`, never a hand-written list. Prose you write here is English,
|
||||
whatever language the session is held in - `AGENTS.md` § File naming has both language rules.
|
||||
|
||||
6. **Publish and wait for CI** per `instructions/dev/publish-and-ci.md`: the local checks,
|
||||
`tools/wikitool publish`, then the run on the published commit.
|
||||
|
||||
**Body upkeep, second fixed point - after the publish:** tick the criteria the publish met,
|
||||
and name the version and the commit in the body.
|
||||
|
||||
**Third fixed point - CI green:** name the run in the body as what verified the change. A red
|
||||
run is not this point: it is step 3 again, then this step again.
|
||||
|
||||
Then the one changelog comment for this session's worth of change
|
||||
(`instructions/dev/issue-tracking.md` step 3).
|
||||
|
||||
7. **End the phase.** Recommend how the closing phase should run, and stop:
|
||||
|
||||
- **This session ran on Opus at high effort:** continue here with `/stack-close` - what was
|
||||
built is still in context, and that is what the closing phase checks against.
|
||||
- **It ran on another model or a lower effort:** `/clear`, then `/stack-close` in a new
|
||||
session on Opus at high effort, which works from the body and the diff. That is why the
|
||||
three fixed points above are not optional.
|
||||
|
||||
Close with the fixed line, in the instance's KB language per `AGENTS.md` § File naming:
|
||||
|
||||
> #N is published and CI is green (run <id>). Next: `/stack-close` - here, or after `/clear`.
|
||||
|
||||
**Do not run `stack-close` yourself.** Offer no `/model` or `/effort` switch either - see
|
||||
`instructions/dev/stack-mode.md` § Sessions and models.
|
||||
|
||||
## Decision points
|
||||
|
||||
- **The change turns out not to be a drop-in replacement after all?** Do not bump across the
|
||||
boundary on your own initiative. Every existing instance pays for a breaking change once, by
|
||||
hand, so the user decides whether it is worth that: show them what breaks, what an instance
|
||||
has to do about it, and the alternatives (avoid the break with a shim, defer and batch it with
|
||||
the next one, or split it behind a deprecation window), then recommend one and wait for a
|
||||
go-ahead. `instructions/dev/version-parts.md` step 4 has the full shape, and the body takes the
|
||||
answer as a design change (step 3's first fixed point).
|
||||
- **The package needs several build sessions?** Each one ends with the body current and its own
|
||||
changelog comment; the phase ends once, at the green run after the last publish.
|
||||
- **CI gave up after 15 minutes, or is red for a reason outside this change?** Hand the open or
|
||||
failing runs to the operator by id and link. The phase is not over, and the closing line in
|
||||
step 7 is not given.
|
||||
|
||||
## Scope
|
||||
|
||||
Only for a body that is ready - the design is `stack-dev`
|
||||
(`instructions/dev/stack-dev/SKILL.md`), the closing after a green run is `stack-close`
|
||||
(`instructions/dev/stack-close/SKILL.md`). Not for wiki content work.
|
||||
@@ -0,0 +1,121 @@
|
||||
---
|
||||
name: stack-close
|
||||
description: Closes out a stack work package once its publish has a green CI run - checks the issue body's final state, checks for docs/ and contract staleness, and closes the issue. Started only by the operator as /stack-close, after stack-build has ended its phase.
|
||||
disable-model-invocation: true
|
||||
---
|
||||
|
||||
# Stack Close
|
||||
|
||||
**Purpose:** Carry out the unchecked closing phase of a stack-development work package, as its
|
||||
own skill rather than a step the build session has to remember to take on its own.
|
||||
|
||||
**Trigger:** The operator runs `/stack-close`, after `stack-build` ended its phase with a green
|
||||
CI run on the published commit. Nothing else starts this skill: the frontmatter's
|
||||
`disable-model-invocation` keeps Claude Code from invoking it, and in a harness that ignores that
|
||||
key this sentence is the rule. Also for a package that was published in an earlier session and
|
||||
never closed - the operator starts it the same way.
|
||||
|
||||
`instructions/dev/stack-mode.md` has the rules of a stack-development session and the three
|
||||
phases this one ends; read it first when this session started cold.
|
||||
|
||||
## Why this is a separate skill, and why the operator starts it
|
||||
|
||||
The phases around the checked middle of a work package have no mechanical guard at all -
|
||||
`pytest`, `docs verify`, `instructions verify` and CI cover the code and tests in between, and
|
||||
nothing covers a changelog entry's accuracy, a `docs/` page's staleness, or an issue body's final
|
||||
state (see `docs/explanation/model-and-effort-selection.md`). Asking the same session to notice it has
|
||||
crossed into that unchecked stretch - as a prose break inside one long skill - failed twice in a
|
||||
row on this stack (Gitea #42, then #30): both times the session knew the rule and skipped past
|
||||
it anyway, because nothing in the moment forced the question. The first split (Gitea #47) moved
|
||||
the closing procedure into its own skill, so it no longer sat in the session's context as a next
|
||||
step to run past - but the trigger stayed a sentence: the build skill told the agent to "invoke
|
||||
it now", and a prose model-switch offer at the same point never once led to a switch (Gitea #50).
|
||||
|
||||
Since Gitea #168 the trigger is the operator's slash command, and the skill cannot be invoked by
|
||||
the agent at all in Claude Code. **Be precise about what that buys.** The phase change is now a
|
||||
real stop rather than a sentence in the output, and it is where the operator decides on context
|
||||
and model. What moved is the risk #47 named: forgetting the close is now the operator's failure,
|
||||
not the agent's. Two signals stay to catch it - `publish`'s note that what follows CI is checked
|
||||
by nothing, and an open issue on the board whose criteria are ticked and whose CI is green. Treat
|
||||
a session that reaches this text as the mechanism having worked *this time*, not as proof that it
|
||||
always will.
|
||||
|
||||
## Steps
|
||||
|
||||
1. **Check the handover you start from.** The body names a green CI run on the published commit
|
||||
(`stack-build` step 6). If it does not, or the run is red, this is not the closing phase yet:
|
||||
say so and recommend `/stack-build #N`. A red run is never closed over.
|
||||
|
||||
This phase is meant to run on Opus at high effort - a lower effort gives up multi-file
|
||||
consistency first, which is exactly what the staleness check in step 3 needs. If this session
|
||||
runs on something else, say so once and carry on; offer no `/model` or `/effort` switch (see
|
||||
`instructions/dev/stack-mode.md` § Sessions and models).
|
||||
|
||||
2. **Check that the body is in its final state.** `stack-build` kept it current at three fixed
|
||||
points, so this is a check, not a rewrite - but the test is still what a reader who opens the
|
||||
closed issue tomorrow would conclude:
|
||||
|
||||
- every acceptance criterion ticked, or struck with the reason it was dropped
|
||||
- proposals that were decided read as decided; a "to decide" section has become the decision
|
||||
and its reasoning; an open, non-blocking question carries its answer
|
||||
- nothing left in the present tense about a defect that no longer exists
|
||||
- what was verified is named - which checks ran, which CI run - not a commit hash alone
|
||||
|
||||
Fix what is off by rewriting the body (`instructions/dev/issue-tracking.md` steps 2 and 7).
|
||||
**A closing report in a comment does not satisfy this**, however thorough: it reads as
|
||||
complete to whoever writes it and leaves a body still phrased as open work. #44 and #45 both
|
||||
closed exactly this way, the second an hour after the rule was first written down.
|
||||
|
||||
3. **Check whether a `docs/` page, a contract, or a new human doc went stale.** A `docs/` page
|
||||
carries no normative sentence, so nothing verifies it by construction (AGENTS.md § File
|
||||
naming) - the same is true of `tools/CONTRACT.md`'s generated command records and any touched
|
||||
`<stage>/CONTRACT.md`, whose prose `docs verify` checks only for presence and record
|
||||
membership, never for what a field or a section actually says
|
||||
(`instructions/dev/doc-pull-through.md`); of `README.md`/`INSTALL.md`/`docs/`
|
||||
prose; and of a new instruction's own wording, which `instructions verify` checks structurally
|
||||
but never for what it claims. If the change this package shipped moved the reasoning or the
|
||||
behaviour one of these documents describes, update it now; if none did, say so rather than
|
||||
leaving the question unasked.
|
||||
|
||||
**If the published diff touches an installation instruction, read the human guide against
|
||||
it once more.** Which instructions those are and which human document answers for each is
|
||||
the pull-through table's row in `instructions/dev/doc-pull-through.md` - `stack-build` step 5
|
||||
applied it before the publish; this is the second reading, after. A deviation found here is
|
||||
filed as a follow-up issue naming both files and the sentence that disagrees, rather than
|
||||
fixed in this phase: the pull-through before the publish missed it, and that miss is worth a
|
||||
record of its own.
|
||||
|
||||
**If that update moved a `##`/`###` heading, the file's table of contents is now stale** -
|
||||
regenerate it with `tools/wikitool docs toc --apply`, never by editing the list. The region
|
||||
is generated (AGENTS.md invariant 1), `docs verify` fails on stale exactly as on missing, and
|
||||
a pull-through in this phase is a common way to move a heading without noticing.
|
||||
|
||||
**A pull-through of its own needs its own bump, publish and CI run.** The documents this
|
||||
phase touches are frequently the ones CI's version gate watches - `types/`, `instructions/`,
|
||||
`tools/`, `AGENTS.md`, any `<stage>/CONTRACT.md`. A commit into one of those without a
|
||||
`VERSION` line fails the gate (`.gitea/workflows/ci.yml`, "Version gate"), whatever the
|
||||
session meant it as. Reading the edit as "only documentation" is the trap: `types/source.md`
|
||||
is a document *and* a shipped behaviour description, and the gate is scoped by path, not by
|
||||
intent. So run `tools/wikitool version bump --patch` in the same breath as the pull-through -
|
||||
it only advances the running candidate's counter - then publish and wait per
|
||||
`instructions/dev/publish-and-ci.md`, and name that run in the body too.
|
||||
|
||||
4. **Comment, then close.** The closing comment carries the one changelog line
|
||||
`instructions/dev/issue-tracking.md` step 3 asks for. Then close the issue.
|
||||
|
||||
## Decision points
|
||||
|
||||
- **The work package spans several sessions?** Run this skill once, at the point the package is
|
||||
actually finished and its last publish has a green run - not after every individual publish.
|
||||
- **Resuming a package whose publish landed in an earlier, already-ended session?** Run this
|
||||
skill now, on whatever model the current session is - do not reopen the earlier session to run
|
||||
it "correctly."
|
||||
- **Nothing to close - the session's own exploration, no publish happened?** This skill does not
|
||||
apply; there is no package to close.
|
||||
|
||||
## Scope
|
||||
|
||||
Follows `stack-build`'s green CI run (`instructions/dev/stack-build/SKILL.md`). Not for wiki
|
||||
content work - use `wiki-ingest`/`wiki-query`/`wiki-manage`/`wiki-lint`/`wiki-status`/
|
||||
`gtd-weekly-review` for that, whose own closing conventions (`kb/log.md`, page provenance) are
|
||||
unrelated to this tracker-body procedure.
|
||||
@@ -1,98 +1,85 @@
|
||||
---
|
||||
name: stack-dev
|
||||
description: Switch a session into tool-development mode - extending tools/wikitool, the compiler, the type schema, or the instruction/skill layer itself, instead of operating on wiki content. Use when the user asks to add a wikitool command, change a type-spec, fix or extend the compiler, or otherwise work on the stack rather than ingest/query/manage/lint the wiki.
|
||||
description: Switches a session into tool-development mode and runs its design phase - extending tools/wikitool, the compiler, the type schema, or the instruction/skill layer itself, instead of operating on wiki content. Use when the user asks to add a wikitool command, change a type-spec, fix or extend the compiler, triage or work out a stack issue, or otherwise work on the stack rather than ingest/query/manage/lint the wiki.
|
||||
---
|
||||
|
||||
# Stack Development Mode
|
||||
# Stack Development Mode - Design
|
||||
|
||||
**Purpose:** Recognize a session that is about the tool stack itself - `tools/wikitool`, the
|
||||
type schema, the instruction/skill layer - rather than wiki content, and switch the rules that
|
||||
apply accordingly.
|
||||
type schema, the instruction/skill layer - rather than wiki content, switch the rules that apply
|
||||
accordingly, and carry a work package through its design phase: to an issue body a cold session
|
||||
can build from.
|
||||
|
||||
**Trigger:** The user asks to add or change a `wikitool` command, extend the compiler, change a
|
||||
type-spec, or work on `instructions/`/`types/`/`tools/` as code rather than as a place to run
|
||||
`wiki-ingest`/`wiki-query`/`wiki-manage`/`wiki-lint`/`wiki-status` against.
|
||||
`wiki-ingest`/`wiki-query`/`wiki-manage`/`wiki-lint`/`wiki-status` against. Also: triaging a
|
||||
`status/incoming` stub, refreshing an old spec against today's tree, or analysing a defect in the
|
||||
stack before anything is fixed.
|
||||
|
||||
**This directory is dev-only.** `instructions/dev/` is excluded wholesale by
|
||||
`tools/wikitool dist export` - nothing here ever reaches a distributed instance, and there is
|
||||
no restore path. If you are in a distributed instance, this skill should not be present at all;
|
||||
stack development happens in the origin repo instead (see AGENTS.md's routing line).
|
||||
|
||||
## What changes in this mode
|
||||
|
||||
- **Source-binding does not apply to code.** AGENTS.md invariant 3 ("never file an unsourced
|
||||
answer into the wiki") governs `kb/` content, not the code you write to extend the stack.
|
||||
Ordinary software-engineering judgment applies to `tools/chemenu/*.py`, `types/*`,
|
||||
`instructions/*` - it does not need a `raw/` source or a citation.
|
||||
- **Test and review conventions from `instructions/dev/` apply instead**, once written down
|
||||
there (step 2 below lists what currently exists). Until a given convention has its own
|
||||
instruction file, follow the existing test files' own patterns
|
||||
(`tools/chemenu/tests/`) rather than inventing a new one silently.
|
||||
- **Everything outside this directory still applies.** The tool error contract, the gates, and
|
||||
"never hand-edit generated files" (AGENTS.md invariants 1, 5-8) are about how the tool
|
||||
behaves at runtime, not about developing it, but they still bind normal session conduct
|
||||
(e.g. still use `tools/wikitool publish`, still respect the gates, when the session also
|
||||
touches wiki content).
|
||||
This is the first of three phases. The build (`stack-build`) and the closing (`stack-close`)
|
||||
are separate skills that only the operator starts - this one never runs them, and never builds
|
||||
past the design. `instructions/dev/stack-mode.md` has the phases, their handovers and why they
|
||||
are split that way.
|
||||
|
||||
## Steps
|
||||
|
||||
1. **Confirm the mode.** If the task is ambiguous between "extend the tool" and "operate the
|
||||
wiki", ask rather than guess - the two have different rules for the same directories.
|
||||
2. **Consult `instructions/dev/` for the concrete procedure.** Currently:
|
||||
[commonplace-kb.md](../commonplace-kb.md) - vendored knowledge base on agent context
|
||||
engineering, memory and deploy-time learning; consult before a design decision in those
|
||||
areas.
|
||||
[issue-tracking.md](../issue-tracking.md) - open work lives in Gitea issues, one per work
|
||||
package, labelled `prio/1..3` and `size/XS..L`. There is no `TODO.md`. Read it before
|
||||
filing something for later, or before deciding what to pick up next.
|
||||
[testing-conventions.md](../testing-conventions.md) - the suite runs against a deliberately
|
||||
empty machine; what the autouse fixture already neutralizes, and what a test still has to
|
||||
establish itself. Read it before adding or changing a test.
|
||||
More instructions are added here incrementally as stack-development needs come up - this
|
||||
list grows without needing this skill file to change shape.
|
||||
3. **Raise the version, if the change ships.** A change under `tools/`, `types/`,
|
||||
`instructions/`, `AGENTS.md` or a `CONTRACT.md` reaches every future instance, so it needs a
|
||||
version and a changelog entry:
|
||||
1. **Confirm the mode, then read `instructions/dev/stack-mode.md`.** If the task is ambiguous
|
||||
between "extend the tool" and "operate the wiki", ask rather than guess - the two have
|
||||
different rules for the same directories. The mode file holds the rules that change, and the
|
||||
catalogue of dev-only procedures (§ Where the procedures are); consult what it names for the
|
||||
task at hand rather than re-deriving it.
|
||||
|
||||
```bash
|
||||
tools/wikitool version bump --patch --title "<what changed>"
|
||||
```
|
||||
2. **Find or open the work package.** One Gitea issue per package
|
||||
(`instructions/dev/issue-tracking.md`). Read its body as the current spec, and correct it
|
||||
first where the tree or a comment proves it wrong. A `status/incoming` stub is worked out per
|
||||
that file's § Incoming stubs before anything else happens to it.
|
||||
|
||||
Never edit `VERSION` or the entry's heading by hand - `bump` writes both, and `docs verify`
|
||||
fails a tree where they disagree. Pick the part by what an existing instance would have to do:
|
||||
3. **Work the design out in the body, not beside it.** Whatever this session establishes - a
|
||||
decision and its reasoning, a root cause, a rejected approach, the files involved - goes into
|
||||
the body as it is settled (`instructions/dev/issue-tracking.md` step 2), with one changelog
|
||||
comment for the session's worth of change (step 3). A question only the user can answer is
|
||||
asked in the chat and stays a question in the body until it is answered; it is never settled
|
||||
by a guess.
|
||||
|
||||
| Change | Part |
|
||||
|--------|------|
|
||||
| Fix, no interface change | `--patch` |
|
||||
| New capability, backwards compatible | `--minor` |
|
||||
| **Existing content must be migrated** | `--major` |
|
||||
4. **Name the version part.** Apply the drop-in test in `instructions/dev/version-parts.md`
|
||||
and write the result into the body. Whether a change is a drop-in replacement has no
|
||||
mechanical guard, so it is decided here, not at the bump. A change that crosses the
|
||||
compatibility boundary goes to the user with what breaks, what an instance has to do about
|
||||
it, and the alternatives, before the body can be ready (version-parts.md step 4).
|
||||
|
||||
A `--major` bump additionally needs a migration document for the new version - written per
|
||||
[migrate-corpus.md](../../migrate-corpus.md) - or `--no-migration "<reason>"` when no content
|
||||
actually has to change. `bump` refuses otherwise, and so does `docs verify`: an instance
|
||||
learning that it must migrate, with nothing telling it how, is a dead end.
|
||||
5. **End the phase at a ready body.** Check the body against
|
||||
`instructions/dev/issue-tracking.md` § Ready to build. If it fails, say which point is open
|
||||
and stay in this phase. If it passes, recommend one of two ways on, and stop:
|
||||
|
||||
Then write the entry's body - `bump` deliberately leaves it empty, the same way `new` leaves
|
||||
the prose.
|
||||
- **A long design session** - much exploration, a defect analysis, a stub worked out from
|
||||
scratch: `/clear`, then `/stack-build #N`. The build starts lean, and a cold start is the
|
||||
real test of whether the body is ready.
|
||||
- **A short one** - an existing spec refreshed against the tree: continue in this session
|
||||
with `/stack-build #N`.
|
||||
|
||||
Prose-only changes (`README.md`, `INSTALL.md`, `EVALS.md`) and the workflows under `.gitea/`
|
||||
do not need a bump - CI's version gate is scoped to what changes behaviour.
|
||||
Close with the fixed line, in the instance's KB language per `AGENTS.md` § File naming:
|
||||
|
||||
4. **Verify before publishing.** `tools/wikitool docs verify`, `tools/wikitool instructions
|
||||
verify`, and the relevant `pytest` run in `tools/` - the same checks any stack change must
|
||||
pass, run explicitly rather than assumed. CI (`.gitea/workflows/ci.yml`) runs these plus a
|
||||
full `setup-instance.md` replay against a fresh `dist export`; a push to `main` that moves
|
||||
`VERSION` additionally triggers a tagged release. **CI does the tagging** - a session never
|
||||
creates a tag, which is what keeps AGENTS.md invariant 5 intact.
|
||||
> #N is ready. Next: `/stack-build #N` - here, or after `/clear`.
|
||||
|
||||
**Do not run `stack-build` yourself, and do not start building.** The phase change is the
|
||||
operator's moment to decide on context and model; a design session that carries on into code
|
||||
takes that decision away. Offer no `/model` or `/effort` switch either - see
|
||||
`instructions/dev/stack-mode.md` § Sessions and models.
|
||||
|
||||
## Decision points
|
||||
|
||||
- **Touches both stack code and wiki content in one session?** Apply this skill's rules to the
|
||||
code changes and the normal content skills' rules to the content changes - they are not
|
||||
mutually exclusive within a session, only per change.
|
||||
- **Nothing to build - the session answered a question or filed a follow-up?** The phase ends
|
||||
with the issue in whatever state it reached, body current; there is no `/stack-build` line to
|
||||
give.
|
||||
- **The task is a one-line fix that seems not to need a design?** It still needs a body that
|
||||
says what is fixed and which version part it takes - which for a real one-liner is a short
|
||||
body, written in minutes. The cut to `stack-build` can then happen in the same session.
|
||||
- **The design turns out to cross the compatibility boundary?** Do not decide it alone - step 4.
|
||||
|
||||
## Scope
|
||||
|
||||
Not for wiki content work - use `wiki-ingest`/`wiki-query`/`wiki-manage`/`wiki-lint`/
|
||||
`wiki-status` for that. Not for setting up a new instance (`instructions/setup-instance.md`) or
|
||||
a fresh clone of this repo (`instructions/bootstrap.md`).
|
||||
`wiki-status`/`gtd-weekly-review` for that. Not for setting up a new instance
|
||||
(`instructions/setup-instance.md`) or a fresh clone of this repo (`instructions/bootstrap.md`).
|
||||
Not for building a ready body (`stack-build`, `instructions/dev/stack-build/SKILL.md`) or closing
|
||||
a published package (`stack-close`, `instructions/dev/stack-close/SKILL.md`).
|
||||
@@ -0,0 +1,121 @@
|
||||
---
|
||||
type: types/instruction.md
|
||||
name: stack-mode
|
||||
description: What changes when a session works on the stack itself rather than on wiki content - which rules stop and start applying, the three phases a work package moves through and the tracker state that hands each one over, where the dev-only procedures are, and why a model is chosen per session rather than per phase.
|
||||
---
|
||||
# Rules for a stack-development session
|
||||
|
||||
Shared by the three skills of the `stack-` family - `stack-dev` (design), `stack-build` (build)
|
||||
and `stack-close` (closing). Each of them can be the first thing a session runs: `/stack-build #N`
|
||||
after a `/clear` starts cold, with none of `stack-dev`'s context. So each skill loads this file at
|
||||
its entry, rather than one of them carrying these rules for the other two.
|
||||
|
||||
**This directory is dev-only.** `instructions/dev/` is excluded wholesale by
|
||||
`tools/wikitool dist export` - nothing here ever reaches a distributed instance, and there is no
|
||||
restore path. If you are in a distributed instance, none of these skills should be present at
|
||||
all; stack development happens in the origin repo instead (see AGENTS.md's routing line).
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
|
||||
- [The three phases](#the-three-phases)
|
||||
- [What changes in this mode](#what-changes-in-this-mode)
|
||||
- [Where the procedures are](#where-the-procedures-are)
|
||||
- [Sessions and models](#sessions-and-models)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
## The three phases
|
||||
|
||||
A work package is one Gitea issue, and it moves through three phases. A skill is a unit of
|
||||
procedure; a session is a unit of context and model. The two are deliberately not the same
|
||||
thing: each phase ends in a **state in the tracker**, and the next phase starts from that state,
|
||||
so every handover works either in the same session or after a `/clear`.
|
||||
|
||||
| Phase | Skill | Invoked by | Ends with (the handover) | What catches a mistake |
|
||||
|---|---|---|---|---|
|
||||
| 1 Design/triage | `stack-dev` | the harness on a matching task, or `/stack-dev` | the issue body is **ready** ([issue-tracking.md](issue-tracking.md) § Ready to build) | nothing mechanical |
|
||||
| 2 Build | `stack-build` | **only** the operator: `/stack-build #N` | a **green CI run** on the published commit, body current | `pytest`, `docs verify`, `instructions verify`, CI |
|
||||
| 3 Closing | `stack-close` | **only** the operator: `/stack-close` | body in its final state, issue closed | nothing mechanical |
|
||||
|
||||
`stack-build` and `stack-close` carry `disable-model-invocation: true` in their frontmatter, so in
|
||||
Claude Code only the operator can start them. That is the point of the split: each phase change is
|
||||
the moment the operator decides whether to continue in this session, `/clear` first, or start the
|
||||
next session on a different model - and a skill the agent could invoke itself would take that
|
||||
moment away again. `instructions sync` copies the frontmatter unchanged; the other harnesses
|
||||
ignore the key, so there the skill's own prose is the only thing that holds the line.
|
||||
|
||||
**So no skill tells the agent to run the next one.** A phase ends with a fixed line naming the
|
||||
slash command for the operator, and stops.
|
||||
|
||||
## What changes in this mode
|
||||
|
||||
- **Source-binding does not apply to code.** AGENTS.md invariant 3 ("never file an unsourced
|
||||
answer into the wiki") governs `kb/` content, not the code you write to extend the stack.
|
||||
Ordinary software-engineering judgment applies to `tools/chemenu/*.py`, `types/*`,
|
||||
`instructions/*` - it does not need a `raw/` source or a citation.
|
||||
- **Test and review conventions from `instructions/dev/` apply instead**, once written down
|
||||
there (§ Where the procedures are, below). Until a given convention has its own instruction
|
||||
file, follow the existing test files' own patterns (`tools/chemenu/tests/`) rather than
|
||||
inventing a new one silently.
|
||||
- **Everything outside this directory still applies.** The tool error contract, the gates, and
|
||||
"never hand-edit generated files" (AGENTS.md invariants 1, 5-8) are about how the tool
|
||||
behaves at runtime, not about developing it, but they still bind normal session conduct
|
||||
(e.g. still use `tools/wikitool publish`, still respect the gates, when the session also
|
||||
touches wiki content).
|
||||
- **A session that touches both stack code and wiki content** applies these rules to the code
|
||||
changes and the normal content skills' rules to the content changes - they are not mutually
|
||||
exclusive within a session, only per change.
|
||||
|
||||
## Where the procedures are
|
||||
|
||||
- [issue-tracking.md](issue-tracking.md) - open work lives in Gitea issues, one per work
|
||||
package, labelled `area/`, `kind/`, `prio/` and `size/`. There is no `TODO.md`. **The body of
|
||||
the issue you are working on is the plan file** of every phase: kept current as the state
|
||||
moves, so an interrupted session leaves a body the next one can resume from, and rewritten to
|
||||
its final state before closing. It also defines when a body is ready to build. An issue
|
||||
labelled `status/incoming` is a human's stub, not a spec, and is **never implemented as it
|
||||
stands**. Read it before filing something for later, before editing or closing an issue,
|
||||
before picking up an incoming stub, or before deciding what to pick up next.
|
||||
- [version-parts.md](version-parts.md) - which part a change bumps: the drop-in test, the
|
||||
catalogue of breaks that cross the compatibility boundary with `kb/` untouched, and what to put
|
||||
in front of the user before a breaking bump. Read it when the design names the part, and again
|
||||
before the bump.
|
||||
- [testing-conventions.md](testing-conventions.md) - the suite runs against a deliberately empty
|
||||
machine; what the autouse fixture already neutralizes, and what a test still has to establish
|
||||
itself. Read it before adding or changing a test.
|
||||
- [tracker-testing.md](tracker-testing.md) - how the task-tracker adapters are tested against a
|
||||
real Super Productivity and CalDAV server: the `live_tracker` suite, the profile procedure for
|
||||
a tracker of your own, the nightly workflow (which you dispatch yourself after touching the
|
||||
Super Productivity surface), what a red night means, and refreshing the recorded fixtures.
|
||||
Read it before changing an adapter under `tools/chemenu/tasks/`.
|
||||
- [doc-pull-through.md](doc-pull-through.md) - which document makes a claim about a touched
|
||||
surface (a `wikitool` command, a stage's rules, an `AGENTS.md` rule/gate/invariant, a
|
||||
README-shaped human doc, a `docs/` page's reasoning) and therefore needs updating alongside
|
||||
the code, since `docs verify` never reads a cell's prose. Read it before publishing.
|
||||
- [publish-and-ci.md](publish-and-ci.md) - the local checks, `publish`, and waiting for the CI
|
||||
run on the published commit. Read it whenever a phase publishes.
|
||||
- [corpus-policy.md](corpus-policy.md) - what "curated enough" means for the shared demo/testbed
|
||||
`kb/`, the measurable floors that define it, and what a reactive fix may and may not do to
|
||||
corpus content. Read it before judging whether the corpus can exercise a change, or before any
|
||||
fix that would touch `kb/` content.
|
||||
- [dev-setup.md](dev-setup.md) - setting up a clone of the origin repo for this work, what
|
||||
differs from an instance there (telemetry on, demo persona, no release stamp), and
|
||||
`dist export` as a build and test tool rather than an install path. Read it in a fresh clone,
|
||||
or before testing a change to the install path.
|
||||
- [commonplace-kb.md](commonplace-kb.md) - vendored knowledge base on agent context engineering,
|
||||
memory and deploy-time learning; consult before a design decision in those areas.
|
||||
|
||||
More instructions are added here as stack-development needs come up - this list grows without
|
||||
any of the three skills having to change shape.
|
||||
|
||||
## Sessions and models
|
||||
|
||||
**A model is chosen per session, never switched inside one.** Neither `/model` nor `/effort`
|
||||
is offered mid-session: either change throws away the prompt cache for everything the session
|
||||
has read so far, and Sonnet's smaller context window does not hold a build phase of this stack.
|
||||
Where two phases should run on different models, the cut goes at a handover - `/clear`, then the
|
||||
next phase's slash command in a session started on the right model - and the tracker state is
|
||||
what carries the work across it.
|
||||
|
||||
Which model suits which phase, and the reasoning, is
|
||||
`docs/explanation/model-and-effort-selection.md`.
|
||||
@@ -19,6 +19,17 @@ the first CI run that ever reached pytest, in a container that had no such confi
|
||||
(Gitea #8). Two more tests of the same kind were written afterwards, by someone who had read
|
||||
that issue first - which is the argument for a fixture rather than a rule.
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
|
||||
- [What the fixture already neutralizes](#what-the-fixture-already-neutralizes)
|
||||
- [Which tree a test writes into](#which-tree-a-test-writes-into)
|
||||
- [When to run](#when-to-run)
|
||||
- [Steps](#steps)
|
||||
- [Decision points](#decision-points)
|
||||
- [Scope](#scope)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
## What the fixture already neutralizes
|
||||
|
||||
Do not re-do any of this per test; it is done for you, per test, via `monkeypatch`.
|
||||
@@ -35,6 +46,53 @@ Do not re-do any of this per test; it is done for you, per test, via `monkeypatc
|
||||
fixture redirects it into `tmp_path`. Tracing is never disabled suite-wide, because two
|
||||
telemetry tests assert that a trace gets written.
|
||||
|
||||
Two in-process caches are cleared alongside the environment, for the same reason: `config`'s
|
||||
resolved paths and `conventions`' parsed `kb/CONVENTIONS.md`. A test that *rewrites* the
|
||||
conventions file mid-test calls `conventions.reset_cache()` itself - the fixture answers for the
|
||||
boundary between tests, not for one inside a test.
|
||||
|
||||
## Which tree a test writes into
|
||||
|
||||
The environment is one half of the isolation; `config.ROOT` is the other. With `CHEMENU_ROOT`
|
||||
cleared, `ROOT` falls back to the checkout pytest is running from - deliberately, because most
|
||||
tests want the shipped `types/`. It also means that any code path resolving a file through
|
||||
`config.ROOT` or `config.KB_DIR` reaches **the real repository**, no matter which tree the
|
||||
fixture built.
|
||||
|
||||
Both corpus fixtures therefore repoint it: `raw_dir` and `kb_dir` each set
|
||||
`config.ROOT` to their `tmp_path` and re-declare the shipped `types/` through
|
||||
`use_shipped_type_specs()`. `config`'s module `__getattr__` resolves the derived paths on
|
||||
access, so repointing `ROOT` carries `KB_DIR`, `RAW_DIR` and the rest with it. A new fixture
|
||||
that builds a tree does the same thing - that is the rule here, not a per-test judgment.
|
||||
|
||||
`kb_dir` did not, until Gitea #44. Two things came of that. A test calling
|
||||
`kb_state.write_kb_state()` overwrote the real `.wikitool-kb.json`, which `git status` made
|
||||
visible within the minute. Quieter and worse: `lint`'s collection lookup resolved a page
|
||||
against `config.KB_DIR`, so every fixture page read back as "no collection" and the
|
||||
`unauthorised_labels` check skipped every edge in silence - the finding had no working test at
|
||||
all, and its green run read like an assurance.
|
||||
|
||||
Two guards came out of it, both in `conftest.py`:
|
||||
|
||||
| Guard | Default | Cost |
|
||||
|---|---|---|
|
||||
| `repository_tree_guard` (session) | on | two `git status --porcelain` calls per run |
|
||||
| `per_test_tree_guard` | off, `CHEMENU_TREE_GUARD=each` turns it on | one `git status` per test |
|
||||
|
||||
The session guard compares the working tree before against after and fails the run if anything
|
||||
moved, so it says nothing about uncommitted work a developer already had. It cannot name the
|
||||
test that did it; `CHEMENU_TREE_GUARD=each` can, and is the way to bisect once it fires. Where
|
||||
git is unavailable or the checkout is not a repository, both are silently inert.
|
||||
|
||||
Neither guard sees the second, quieter half: a check that silently *does nothing* under test
|
||||
writes no file. That one is only caught by a test that asserts the finding actually fires -
|
||||
which is why `test_unauthorised_label_is_judged_in_a_tree_that_is_not_the_configured_kb`
|
||||
lints a tree `ROOT` deliberately points away from.
|
||||
|
||||
**A function that takes a directory resolves against that directory.** `run_lint(kb_dir)`
|
||||
reading `config.KB_DIR` for one of its own lookups was the defect behind the quiet half, and
|
||||
no fixture can fix that shape from the outside.
|
||||
|
||||
## When to run
|
||||
|
||||
Whenever you add or change a test under `tools/chemenu/tests/`.
|
||||
@@ -72,10 +130,15 @@ Whenever you add or change a test under `tools/chemenu/tests/`.
|
||||
fixture creates that machine.
|
||||
|
||||
4. **Adding a new environment variable to the tool?** Add it to `_WIKITOOL_ENV` in
|
||||
`conftest.py` in the same change. A variable the tool reads and the fixture does not clear
|
||||
`conftest.py` in the same change (`WIKITOOL_TASKS_CONFIG` is one). A variable the tool reads and the fixture does not clear
|
||||
is the exact hole this whole file is about, reopened.
|
||||
|
||||
5. **Verify against an empty machine before publishing**, not only in your own shell:
|
||||
5. **Writing a fixture that builds a tree?** Repoint `config.ROOT` at it and call
|
||||
`use_shipped_type_specs(monkeypatch)`, as `raw_dir` and `kb_dir` do - see
|
||||
[Which tree a test writes into](#which-tree-a-test-writes-into). A fixture that returns a
|
||||
path without repointing hands the code under test the real repository.
|
||||
|
||||
6. **Verify against an empty machine before publishing**, not only in your own shell:
|
||||
|
||||
```bash
|
||||
cd tools && env -i PATH="$PATH" HOME="$(mktemp -d)" \
|
||||
@@ -87,7 +150,7 @@ Whenever you add or change a test under `tools/chemenu/tests/`.
|
||||
`.venv/bin/python -m pytest -q`. A difference between the two is a leak, and the leaking
|
||||
variable belongs in step 4's list.
|
||||
|
||||
6. **Check the coverage report when adding tests to close a gap**, rather than guessing which
|
||||
7. **Check the coverage report when adding tests to close a gap**, rather than guessing which
|
||||
lines were uncovered:
|
||||
|
||||
```bash
|
||||
@@ -96,15 +159,24 @@ Whenever you add or change a test under `tools/chemenu/tests/`.
|
||||
|
||||
Read it by module, not by total. A thin Typer wrapper sitting low is evidence that the logic
|
||||
was cut out from under it and tested there; the list worth acting on is the modules whose
|
||||
*logic* is uncovered. EVALS.md § "How much of the stack the suite reaches" names both, and
|
||||
the measured baseline. There is no threshold to satisfy - the suite is not graded on the
|
||||
*logic* is uncovered. [docs/dev/development.md](../../docs/dev/development.md) § "How much of the stack the
|
||||
suite reaches" names both, and the measured baseline. There is no threshold to satisfy - the suite is not graded on the
|
||||
number.
|
||||
|
||||
## Decision points
|
||||
|
||||
- **Testing a PowerShell script?** `test_preflight_pwsh.py` needs `pwsh` and skips silently
|
||||
without it - a green run on a machine with no PowerShell 7 has not run those tests. Check
|
||||
`command -v pwsh` before trusting the result; CI's `pwsh` job runs them in the image built from
|
||||
`.gitea/pwsh-ci/Dockerfile`, and so can you (`docker run` that image with the checkout mounted
|
||||
and `pytest tools/chemenu/tests/test_preflight_pwsh.py` as the command).
|
||||
- **A test genuinely needs the developer's real environment?** There is no such test, and a new
|
||||
one is a design problem rather than an exception: what it wants is a fixture that *builds*
|
||||
the state it needs inside `tmp_path`. Building it is also the only version CI can run.
|
||||
The one deliberate exception is the `live_tracker` suite, which talks to a real tracker and
|
||||
may start a real application; it is opt-in through `CHEMENU_LIVE_*` variables, skips without
|
||||
them, and the default run needs neither network nor tracker. Its rules:
|
||||
[tracker-testing.md](tracker-testing.md).
|
||||
- **A test patches `config.default_author` directly** (as
|
||||
`test_new_source_fails_hard_without_any_author` does)? Keep the patch. It is not made
|
||||
redundant by the fixture - it pins the value under test regardless of what the environment
|
||||
@@ -114,6 +186,7 @@ Whenever you add or change a test under `tools/chemenu/tests/`.
|
||||
## Scope
|
||||
|
||||
Applies to `tools/chemenu/tests/` only. It says nothing about what to test - the test/review
|
||||
expectations for a stack change are the `stack-dev` skill's step 4 (`docs verify`,
|
||||
`instructions verify`, pytest). CI runs the suite once, unhardened, because the fixture makes a
|
||||
second hardened run redundant; see the note on the Tests step in `.gitea/workflows/ci.yml`.
|
||||
expectations for a stack change are the local checks in [publish-and-ci.md](publish-and-ci.md)
|
||||
(`docs verify`, `instructions verify`, pytest), run from `stack-build`. CI runs the suite once,
|
||||
unhardened, because the fixture makes a second hardened run redundant; see the note on the Tests
|
||||
step in `.gitea/workflows/ci.yml`.
|
||||
@@ -0,0 +1,212 @@
|
||||
---
|
||||
type: types/instruction.md
|
||||
name: tracker-testing
|
||||
description: How the task-tracker adapters (Super Productivity, CalDAV) are tested against a real tracker - the live suite, the profile procedure for a tracker of your own, the nightly workflow and when an agent triggers it, what a red night means, and how to refresh the recorded fixtures.
|
||||
---
|
||||
# Test a tracker adapter against a real tracker
|
||||
|
||||
The default `pytest` run never talks to a tracker: it reads recorded answers
|
||||
(`tools/chemenu/tests/fixtures/sp/`) and fakes. That is enough to keep the parsing honest and
|
||||
not enough to know that the adapter still works against the tracker as it ships today - a
|
||||
tracker is software somebody else releases (Gitea #156, and the adapter defects of #162 that
|
||||
only a real Super Productivity could show). So a second suite exists, marked `live_tracker`,
|
||||
that runs the documented `task new` / `task list` / `task close` / `review` workflow end to end.
|
||||
It skips when no tracker is configured and fails when CI says one must be there.
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
|
||||
- [The three ways to name a tracker](#the-three-ways-to-name-a-tracker)
|
||||
- [What the suite writes, and what it never does](#what-the-suite-writes-and-what-it-never-does)
|
||||
- [Run it against a tracker of your own](#run-it-against-a-tracker-of-your-own)
|
||||
- [The two workflows and the image](#the-two-workflows-and-the-image)
|
||||
- [When an agent triggers the nightly run](#when-an-agent-triggers-the-nightly-run)
|
||||
- [A red night](#a-red-night)
|
||||
- [Refreshing the fixtures](#refreshing-the-fixtures)
|
||||
- [Steps](#steps)
|
||||
- [Scope](#scope)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
## The three ways to name a tracker
|
||||
|
||||
All through the environment; none of them is a fixed address, and none of them is read by a
|
||||
plain `pytest` run unless you set it.
|
||||
|
||||
| Variable | Tracker | Who owns it |
|
||||
|---|---|---|
|
||||
| `CHEMENU_LIVE_SP_BINARY` | The packaged Super Productivity (`.deb` layout). The suite starts its own headless copy on a fresh profile, seeded from `fixtures/sp/seed-backup.json`, and stops it again. | The suite |
|
||||
| `CHEMENU_LIVE_CALDAV_URL`, `_USER`, `_PASSWORD` (optional `_VERSION` for the report line) | A CalDAV collection. In CI it is a throwaway Radicale started by `.gitea/scripts/start-radicale.sh`. | The suite - it may create the marker calendar |
|
||||
| `CHEMENU_LIVE_PROFILE=<name>` | A tracker of your own, described by `.wikitool-tasks.d/<name>.json` in the checkout, used exactly as configured. | You - the suite never creates anything but items |
|
||||
|
||||
`CHEMENU_LIVE_REQUIRE=sp,caldav,profile` (any subset) turns "no such tracker configured" into
|
||||
a failure instead of a skip. CI sets it; without it a workflow that lost its server would go
|
||||
green by skipping, which is the outcome the suite exists to rule out.
|
||||
|
||||
The commands under test read their configuration from `.wikitool-tasks.json`, or from the file
|
||||
`WIKITOOL_TASKS_CONFIG` names when that variable is set. The live suite sets the variable per
|
||||
test, so it never touches the checkout's own `.wikitool-tasks.json`; you can use the same
|
||||
variable to run `task` and `review` by hand against a second tracker.
|
||||
|
||||
## What the suite writes, and what it never does
|
||||
|
||||
The safety rules are code (`tools/chemenu/tests/tracker_live.py`), and
|
||||
`test_tracker_live.py` proves them against fakes on every run - they are not a promise in this
|
||||
file.
|
||||
|
||||
- It writes only into the project named **`Chemenu Live-Test`** (`MARKER_PROJECT`). The project
|
||||
must already exist. The suite creates it only in a tracker it owns end to end (the CalDAV
|
||||
server it started); in a profile target, or in Super Productivity, a missing marker is an
|
||||
abort before the first write - Super Productivity's API has no way to create a project anyway.
|
||||
- Every item it creates starts with `[live-test <run id>]`. On the way out it closes what is
|
||||
left of that prefix, and only that prefix.
|
||||
- It never deletes anything. The only closing write is `task close`, the one this stack has.
|
||||
- The headless Super Productivity refuses to start when anything already answers on the fixed
|
||||
API port 3876: an answering app is somebody's real one.
|
||||
- A profile path that starts with `~` is refused. Profiles carry absolute paths, because a `~`
|
||||
resolves against whichever `HOME` the run has.
|
||||
|
||||
## Run it against a tracker of your own
|
||||
|
||||
Do this before you change an adapter for a tracker you actually use, or when a user reports one
|
||||
that fails against theirs. The steps differ per tracker only in the profile and the marker.
|
||||
|
||||
1. **Create the marker project in that tracker**, by hand, named exactly `Chemenu Live-Test`.
|
||||
Nothing else in the tracker is touched, but this project's items are created and closed.
|
||||
2. **Write the profile** to `.wikitool-tasks.d/<name>.json` (gitignored - it holds
|
||||
credentials). It is a normal `.wikitool-tasks.json` plus one optional key,
|
||||
`live_test_version`, which is stripped before use and only names the tracker's version in
|
||||
the report line. Paths are absolute.
|
||||
|
||||
Super Productivity, API access (the app must be running with the Local REST API on):
|
||||
|
||||
```json
|
||||
{
|
||||
"schema": 1,
|
||||
"provider": "superproductivity",
|
||||
"thresholds": {"stalled_waiting_days": 14, "unpaged_project_weeks": 3, "someday_stale_months": 5},
|
||||
"superproductivity": {"access": "api", "api_base_url": "http://127.0.0.1:3876", "api_token": "<token>"},
|
||||
"live_test_version": "19.1.0"
|
||||
}
|
||||
```
|
||||
|
||||
CalDAV:
|
||||
|
||||
```json
|
||||
{
|
||||
"schema": 1,
|
||||
"provider": "caldav",
|
||||
"thresholds": {"stalled_waiting_days": 14, "unpaged_project_weeks": 3, "someday_stale_months": 5},
|
||||
"caldav": {"url": "https://<server>/<user>/", "username": "<user>", "app_password": "<password>",
|
||||
"inbox_list": "Inbox", "someday_list": "Someday"},
|
||||
"live_test_version": "<server and version>"
|
||||
}
|
||||
```
|
||||
|
||||
A Super Productivity profile with `access: "snapshot"` is accepted too, but the workflow
|
||||
needs a write path, so the scenario fails on it by design: that access is read-only.
|
||||
3. **Run it**, from `tools/`:
|
||||
|
||||
```bash
|
||||
CHEMENU_LIVE_PROFILE=<name> CHEMENU_LIVE_REQUIRE=profile \
|
||||
.venv/bin/python -m pytest -m live_tracker -k profile -s
|
||||
```
|
||||
|
||||
`-s` shows the report line with the tracker version, which is what you paste into an issue.
|
||||
4. **Read a failure as a finding about that tracker**, not as a flaky test: the scenario is
|
||||
deterministic. Reproduce it with `task` and `review` under `WIKITOOL_TASKS_CONFIG`, then fix
|
||||
the adapter and add a recorded fixture for the answer that broke it.
|
||||
|
||||
## The two workflows and the image
|
||||
|
||||
| Workflow | When | What |
|
||||
|---|---|---|
|
||||
| `ci.yml`, step "Live tracker suite (CalDAV)" | Every push and PR | Radicale as a process on loopback (`pip install radicale`), `CHEMENU_LIVE_REQUIRE=caldav`. The one half that needs no app and no display. |
|
||||
| `tracker-live.yml` | Nightly 05:00 UTC, and by hand | Both kinds, `CHEMENU_LIVE_REQUIRE=sp,caldav`, inside the `chemenu-sp-live` image. It logs the installed Super Productivity version and warns when it differs from the update channel. |
|
||||
| `sp-live-image.yml` | Daily 04:10 UTC, and by hand | Builds `gitea.nehmer.net/torben/chemenu-sp-live` when the registry lacks the channel's current version, and once a month regardless. |
|
||||
|
||||
The image follows the update channel (`latest-linux.yml`, resolved by
|
||||
`.gitea/sp-live/resolve-version.sh`) and not a pin: installed desktop clients update
|
||||
themselves, and a pinned old version would be tested while users run the new one. Tags are
|
||||
`:<version>` and `:latest`; a manual build with an explicit `sp_version` never moves `:latest`,
|
||||
which is how you reproduce a red night against the version it went red on.
|
||||
|
||||
**The first push of the package needs one manual step**, which no workflow can do: link the
|
||||
package `chemenu-sp-live` to the repository `torben/chemenu` in the Gitea UI, once, and
|
||||
dispatch `sp-live-image.yml` first - `tracker-live.yml` has nothing to pull before that. The
|
||||
package is public, so the runner pulls it anonymously.
|
||||
|
||||
## When an agent triggers the nightly run
|
||||
|
||||
A session that changed the Super Productivity surface does not wait for the clock: after the
|
||||
push has landed and CI is green, dispatch `tracker-live.yml` and poll it (Gitea MCP,
|
||||
`actions_run_write` then `actions_run_read`; never an anonymous `curl`). The surface is
|
||||
`tools/chemenu/tasks/superproductivity.py`, `tools/chemenu/tasks/config.py`,
|
||||
`tools/chemenu/tests/sp_headless.py`, `tools/chemenu/tests/tracker_live.py`, the fixtures under
|
||||
`tools/chemenu/tests/fixtures/sp/`, and `.gitea/sp-live/`. A change to the CalDAV adapter needs
|
||||
no dispatch - `ci.yml` already ran it.
|
||||
|
||||
Say in the session summary which run answered and against which Super Productivity version.
|
||||
|
||||
## A red night
|
||||
|
||||
The nightly run is red for one of three reasons, and they are told apart by the log:
|
||||
|
||||
1. **The app changed.** The report line names a Super Productivity version newer than the last
|
||||
green night. This is the finding the workflow exists for: the adapter must follow. Re-record
|
||||
the fixtures (below), fix the adapter, and open an issue for it with the two version
|
||||
numbers.
|
||||
2. **The image is stale.** The step "Which Super Productivity is this" warned that the
|
||||
installed version differs from the channel. Dispatch `sp-live-image.yml` with `force`, then
|
||||
the run again.
|
||||
3. **The environment.** The app did not start (the abort message carries the last lines of
|
||||
the app's own log), or the runner could not pull the image. Nothing about the adapter is known yet.
|
||||
|
||||
Never skip the suite, never remove `CHEMENU_LIVE_REQUIRE` from a workflow and never mark the
|
||||
job `continue-on-error` to get a green night: a red night with a cause is the product.
|
||||
|
||||
## Refreshing the fixtures
|
||||
|
||||
`tools/chemenu/tests/fixtures/sp/api/*.json` are the raw answers of a real Super Productivity;
|
||||
`seed-backup.json` is a backup the app wrote itself, holding the marker project the live suite
|
||||
writes into and the demo projects - with their items - that the `kb/gtd/` pages of the same
|
||||
names join against. `MANIFEST.json` says which version and when, and names the one edit made to
|
||||
the backup after the app wrote it. Refresh them when a red night
|
||||
shows a new answer shape, and when the manifest's version is more than a few releases behind.
|
||||
|
||||
1. Get a Super Productivity to record from: the current `chemenu-sp-live` image with the
|
||||
checkout mounted (`docker run --rm -v "$PWD:/work" -w /work/tools ...`, then a venv from
|
||||
`requirements.txt` inside it), or any machine with the `.deb` installed and
|
||||
`CHEMENU_LIVE_SP_BINARY` set.
|
||||
2. From `tools/`, in an environment that has `requirements.txt`:
|
||||
|
||||
```bash
|
||||
.venv/bin/python -m chemenu.tests.record_sp_fixtures chemenu/tests/fixtures/sp/api
|
||||
```
|
||||
|
||||
3. Update `MANIFEST.json` (`sp_version`, `recorded`) by hand - it is a fixture, not a generated
|
||||
page - and run `test_sp_recorded.py`. If the seed backup is stale because the app's backup
|
||||
format moved, rebuild it the same way: start the new version on the old seed, let it write
|
||||
its own backup, and keep the projects and items `test_sp_recorded.py` expects. A
|
||||
`kb/gtd/` page's name and the demo project's name must stay equal - the join is by name.
|
||||
4. Do not edit the recorded answers to make a test pass. A test that fails against a fresh
|
||||
recording is the adapter's problem.
|
||||
|
||||
## Steps
|
||||
|
||||
1. **Decide which kind of change this is.** A change to an adapter's parsing: extend the recorded
|
||||
fixtures and their replay test first. A change that alters what the adapter sends: the live
|
||||
suite has to run, so run it against a tracker (above) before publishing.
|
||||
2. **Run the offline half** with the rest of the suite; it includes the safety tests for the
|
||||
guard, the profile rules and the port refusal.
|
||||
3. **Run the live half** for the tracker you changed - CalDAV locally with a Radicale of your
|
||||
own, Super Productivity in the image or with a profile.
|
||||
4. **After the push, dispatch `tracker-live.yml`** when the change is on the Super Productivity
|
||||
surface (see [When an agent triggers the nightly run](#when-an-agent-triggers-the-nightly-run)).
|
||||
|
||||
## Scope
|
||||
|
||||
Applies to the task-tracker adapters under `tools/chemenu/tasks/` and to the live suite
|
||||
itself. It is the one place the test conventions are relaxed on purpose:
|
||||
[testing-conventions.md](testing-conventions.md) describes the hermetic default run, and the
|
||||
live suite is opt-in, marked `live_tracker`, and may start a real application and talk to a
|
||||
real server. The default run stays free of both.
|
||||
@@ -0,0 +1,283 @@
|
||||
---
|
||||
type: types/instruction.md
|
||||
name: version-parts
|
||||
description: Which part of the stack version a change bumps - the compatibility question (is the new version a drop-in replacement?) separated from the migration question (must existing content change?), plus what to do before a breaking bump.
|
||||
---
|
||||
# Pick the version part for a stack change
|
||||
|
||||
Two questions decide a version bump, and they are **not the same question**:
|
||||
|
||||
1. **Is the new version a drop-in replacement for the old one?** This is what the version
|
||||
number itself says. Compatibility is read off the **leftmost non-zero component** - on this
|
||||
stack (`4.x`) that is MAJOR, on a `0.x` stack it is MINOR. A bump that changes it is called
|
||||
*boundary-crossing* below, because that is the term `version bump` and `docs verify` use in
|
||||
their own messages.
|
||||
2. **Must existing content be migrated?** This is a *consequence* a boundary crossing may or
|
||||
may not have. `kb/` staying untouched does not make a change compatible, and
|
||||
`version bump --no-migration` exists precisely because boundary-crossing bumps with an
|
||||
untouched corpus are a real case.
|
||||
|
||||
Getting these backwards is how a genuinely breaking change ships as a MINOR. It happened once
|
||||
already (see the case study at the end), which is why this file exists.
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
|
||||
- [The candidate model](#the-candidate-model)
|
||||
- [When to run](#when-to-run)
|
||||
- [Steps](#steps)
|
||||
- [Decision points](#decision-points)
|
||||
- [Scope](#scope)
|
||||
- [Case study: 2.0.0](#case-study-200)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
## The candidate model
|
||||
|
||||
Between two releases the stack carries **one running candidate**, not a fresh version per
|
||||
`bump`. Before that, every bump minted a number *and* a release: CI's version gate requires
|
||||
`VERSION` to move on every stack-touching push, and `release.yml` fires on every `VERSION`
|
||||
move, so releases were being cut at commit granularity. 2026-09-03 produced four of them in
|
||||
six hours (`4.3.0` through `4.3.3`) for one arc of work - all four real, none of them a
|
||||
meaningful unit to anyone downstream. A candidate closes that gap without touching the gate:
|
||||
`VERSION` still moves on every bump, it just escalates the *same* number instead of handing out
|
||||
a new one, and only `version release` turns it into something the release workflow acts on.
|
||||
|
||||
- **State lives in `VERSION` itself**, as an optional `-beta.N` suffix (`4.4.0-beta.3`). No
|
||||
second state file: the last release is read back out of `CHANGES.md` (the newest entry with no
|
||||
suffix), and the escalation stage is the difference between the candidate's base and that
|
||||
release - derived, not stored.
|
||||
- **`--major`/`--minor`/`--patch` is max-wins escalation**, not a step you can undo. A `--patch`
|
||||
bump on a candidate already at MINOR only advances its bump count (`N`); nothing ever steps a
|
||||
candidate back down. Declaring the part is still your judgment call, made the same way the
|
||||
steps below describe - `escalate()` only ever raises it further.
|
||||
- **A candidate is never released.** Pre-release is a dev-checkout state; `release.yml` only acts
|
||||
on a suffix-free `VERSION`, so a distributed instance never sees a `-beta.` version at all, and
|
||||
its parser never has to know the suffix exists.
|
||||
- **One `CHANGES.md` entry per candidate**, not per bump. The first bump of a candidate opens it
|
||||
(heading, date, author, and a machine-managed `<!-- wikitool:bumps -->` list seeded with that
|
||||
bump's `--title`, graded by `--impact`); every later bump of the *same* candidate updates that
|
||||
entry in place - heading, date and the bumps list all move, but the entry's own prose is left
|
||||
alone. `version notes` therefore still prints exactly one entry per release, whatever a
|
||||
candidate's history of bumps looked like.
|
||||
- **The entry is layered, not one undifferentiated block.** A long-running candidate can collect
|
||||
dozens of bumps, chronological and equally weighted, which is unreadable as a release
|
||||
announcement - `5.0.0` did this at ~1440 lines for one entry. So the entry reads, top to bottom,
|
||||
as four layers with different authors and different lifetimes:
|
||||
|
||||
1. **Heading, author, breaking/migration lines** - written by `version bump`, anchored right
|
||||
above the bump list so the line an operator most needs to act on never sits beneath a list
|
||||
that can run long. The breaking line **accumulates** across a candidate's crossings - one
|
||||
reason on the marker line, bullets under a bare marker from the second onward - because a
|
||||
long-running candidate can break compatibility more than once and each break is its own
|
||||
thing to act on. The migration line does not: it answers one yes/no about the candidate as
|
||||
a whole, in one of two forms that never stand side by side. Where a required migration
|
||||
document targets the candidate's base, every bump writes
|
||||
`**Migration:** required - instructions/migrations/<file>.md` (several paths sorted and
|
||||
comma-separated) - the pointer from the release notes to what an operator has to run;
|
||||
otherwise `--no-migration` writes `**Migration:** none required - <reason>`, and
|
||||
`--migration-required` is its retraction path. Nothing retracts a breaking reason; a wrong
|
||||
one is rare enough, and the candidate is dev-local until release.
|
||||
2. **The bump list**, grouped `**High/Medium/Low impact**` (empty groups omitted) - rendered by
|
||||
`version bump`'s `--impact` (default `medium`), corrected after the fact by `version regrade`.
|
||||
Flat and ungrouped, exactly as before this layering existed, when every bump is `medium` - the
|
||||
common case, and the shape every pre-existing region still is. **This list is the bullet
|
||||
level:** a smaller change gets nothing in the entry but its bump title.
|
||||
3. **The release summary** - a short paragraph, written once, by hand, when the candidate is
|
||||
ready to ship. `version release` refuses to close an entry with two or more bumps and no
|
||||
summary here; a one-bump entry is exempt, since there its own paragraph already reads as
|
||||
the summary.
|
||||
4. **One `###` section per topic, not per bump** - for the larger changes only. A topic is
|
||||
what a reader would call one change; it often gathers several bumps, and its heading names
|
||||
the topic rather than repeating a bump title. A later bump on the same topic rewrites that
|
||||
section's paragraph instead of adding a second one. Whether a change earns a section or
|
||||
stays a bullet is the build session's judgment, with no tie to `--impact`.
|
||||
|
||||
**The budget, checked by `docs verify` and again by `version release` before it writes:** the
|
||||
topmost versioned entry is at most 32 000 bytes (UTF-8) as `version notes` prints it, and every
|
||||
`###` section is **exactly one paragraph** - no blank line in its body - of at most 1 000 bytes,
|
||||
its `###` line excluded. A body runs to the next `### ` or `## ` line or the `---` separator;
|
||||
blank lines at either end do not count. The total is there because an entry grows with every
|
||||
bump: the 8.0.0 candidate collected 81 bumps, one changeset each, reached 129 KB and failed as
|
||||
a release body, and no per-section rule alone would have stopped that. Older entries are not
|
||||
held to it - their release notes are already published. What needs more than one paragraph -
|
||||
the reasoning, the alternatives, the measurements - belongs in the issue, not in the entry.
|
||||
- **`version bump` opens or continues a candidate; `version release` fixes one.** Only `release`
|
||||
strips the suffix and turns the entry into a real, closed release - see its own row in
|
||||
`tools/CONTRACT.md`. Nothing else does, and nothing auto-fixes a candidate on its own.
|
||||
|
||||
## When to run
|
||||
|
||||
When a design names the version part (`stack-dev` step 4), and again before every
|
||||
`tools/wikitool version bump` (`stack-build` step 4).
|
||||
Read it in full the first time a change looks like it might be boundary-crossing; afterwards
|
||||
the three-line test below is usually enough.
|
||||
|
||||
## Steps
|
||||
|
||||
1. **Apply the drop-in test.** The bump is boundary-crossing if **either** half fails:
|
||||
|
||||
- **Forward:** an existing instance can install the new machinery by copying `tools/`,
|
||||
`types/`, `instructions/`, `AGENTS.md`, `VERSION` and `.wikitool-release.json` over itself,
|
||||
and everything that worked before still works - with **no** hand-work by the user and **no**
|
||||
migration script. Any step beyond the copy, however small, fails this half.
|
||||
- **Backward:** having installed the new version, the user can put the old one back and be
|
||||
where they started. A state file the old version cannot read, a rewritten corpus, a
|
||||
renamed stamp - anything that makes the downgrade fail or leave a broken instance - fails
|
||||
this half.
|
||||
|
||||
Content migration is one way to fail the forward half, not the definition of it.
|
||||
|
||||
2. **Check the catalogue** when the answer still feels like a judgment call. Each of these
|
||||
crosses the boundary with `kb/` entirely untouched:
|
||||
|
||||
| What changed | Why the swap is not drop-in |
|
||||
|---|---|
|
||||
| The update path - `update_url`, the release feed, the repo it points at | The instance cannot repair its own `.wikitool-release.json`: it is machine-written, and invariant 1 forbids the hand-edit. The channel that would have told it to update is the channel that broke |
|
||||
| The release artefact's name | Every download script and every pin against it breaks |
|
||||
| The Python package's import name | `from <old> import ...` outside the shipped tree breaks |
|
||||
| A command, subcommand, or flag that was removed or renamed | Scripts, CI workflows and instruction files calling the old spelling break |
|
||||
| An environment variable's name | An instance configured through it silently loses the configuration |
|
||||
| The shape of a machine-read file - `.wikitool-kb.json`, `.wikitool-release.json`, a generated index | The old version cannot read what the new one wrote, so the downgrade half fails even if the upgrade half passed |
|
||||
| A type-spec's required fields | Existing pages stop validating - this one crosses *and* needs a content migration |
|
||||
|
||||
The catalogue is illustrative, not exhaustive. When something is not on it, go back to step 1.
|
||||
|
||||
3. **Otherwise pick the compatible part:**
|
||||
|
||||
| Change | Part |
|
||||
|---|---|
|
||||
| Fix, no interface change | `--patch` |
|
||||
| New capability, drop-in in both directions | `--minor` |
|
||||
|
||||
4. **Stop and talk to the user before the bump that first escalates a candidate past the
|
||||
boundary.** It is expensive in a way the other two parts are not: every existing instance pays
|
||||
for it, once, by hand. That escalation happens exactly once per candidate - a later bump that
|
||||
keeps the candidate at the same stage (another `--major` on one already there, say) does not
|
||||
re-cross anything and needs no second conversation. Put in front of the user, in this order:
|
||||
|
||||
- **What breaks**, concretely - which file, which name, which call site.
|
||||
- **What each existing instance must do**, as the steps they would actually run.
|
||||
- **The alternatives**, so the break is a choice and not a side effect:
|
||||
- *Avoid it* - keep the old name as an alias, read both file shapes, accept both flag
|
||||
spellings. A compatibility shim carried for one release is usually cheaper than a
|
||||
migration everyone runs.
|
||||
- *Defer and batch it* - hold the break until the next boundary crossing, so instances pay
|
||||
once instead of twice.
|
||||
- *Split it* - ship the compatible half now, the breaking half later behind a deprecation
|
||||
window that the changelog announces in advance.
|
||||
- **Your recommendation**, with the trade-off named.
|
||||
|
||||
Then wait for an explicit go-ahead. Do not bump across the boundary on your own initiative.
|
||||
|
||||
5. **Grade the bump while you are making it, with `--impact high|medium|low`** (default
|
||||
`medium`) - the judgment is easiest right after you did the work, not weeks later staring at a
|
||||
chronological list. It is not final: `version regrade` corrects it before release if the
|
||||
candidate's overall shape changes the read on an earlier bump. Then record the break in the
|
||||
escalation bump itself. The bump that first crosses the boundary requires
|
||||
`--breaking "<what breaks>"`, which writes a `**Breaking Change:**` line into the entry:
|
||||
|
||||
```bash
|
||||
tools/wikitool version bump --major \
|
||||
--title "<what changed>" \
|
||||
--breaking "<what stops working, and what an instance must do about it>" \
|
||||
--no-migration "<why no page has to change>" # only if that is true
|
||||
```
|
||||
|
||||
The line, once written, stays in the entry across every later bump of the same candidate -
|
||||
a follow-up `--major` does not need to repeat `--breaking`, because the entry it would repeat
|
||||
it into is the same one. `--breaking` is refused on a bump that crosses nothing, and required
|
||||
on the one that does. `docs verify` checks the newest boundary-crossing entry still carries
|
||||
the line. Write it for the operator of an instance that has not read this repository: what
|
||||
stops working, and what they do about it.
|
||||
|
||||
6. **Then answer the migration question separately.** Boundary-crossing and
|
||||
content-migrating are independent:
|
||||
|
||||
- Content must change → write the migration document under `instructions/migrations/` per
|
||||
[migrate-corpus.md](../migrate-corpus.md). The escalation bump finds it by the document's
|
||||
`migrates_to:` field, matched against the candidate's **base** - a document targets the
|
||||
release the candidate will become, never a `-beta.N` form of it - and names a required one
|
||||
in the entry's migration line.
|
||||
- Content need not change → `--no-migration "<reason>"`, which records that in the entry.
|
||||
|
||||
Both are also needed by `docs verify`, for the same reason: an instance that learns it must
|
||||
migrate, with nothing telling it how, is a dead end. Like `--breaking`, both persist across
|
||||
later bumps of the same candidate without being repeated.
|
||||
|
||||
**A `--no-migration` answer can turn out wrong later in the same candidate**, and that is not
|
||||
a hand-edit: a bump escalates, a second change lands under the same running number, and now
|
||||
content does have to move after all. Write the migration document first, then run
|
||||
`version bump --migration-required` - it replaces the `**Migration:** none required` line the
|
||||
earlier bump wrote with the `required` line naming the document, and refuses unless a required
|
||||
document already targets the new base. Any later bump would make the same replacement once
|
||||
the document exists, and `docs verify` reports a crossing entry whose migration line does not
|
||||
name it; the flag is the explicit form, which also refuses when there is nothing to replace.
|
||||
`--no-migration` itself is refused while a required document targets the base. The 5.0.0
|
||||
candidate is the case this guards: it declared `--no-migration` for a TOC-verification
|
||||
change, then absorbed a schema removal that migrates 152 pages.
|
||||
|
||||
7. **Fix the candidate only when the user asks for a release.** Whether a candidate ships is the
|
||||
user's call, never a session's: a work package being finished is not a reason, since the
|
||||
candidate model exists precisely so that one does not become one release. A session that
|
||||
bumps stops at the open `-beta.N` candidate; the next `publish` then carries it without
|
||||
triggering `release.yml`. Once the user does ask, review the graded list first, and regrade
|
||||
what reads wrong. Run
|
||||
`tools/wikitool version regrade` with no arguments - it lists every bump at its current grade,
|
||||
numbered in rendered order. A candidate that grew over several sessions often has a bump graded
|
||||
in isolation that reads differently once the whole shape is visible; `version regrade 3 7
|
||||
--impact high` corrects one or several positions against a single read of that list, put the
|
||||
result in front of the user, and re-list to confirm. Only then run
|
||||
`tools/wikitool version release`, which strips the `-beta.N` suffix and closes the entry - see
|
||||
its row in `tools/CONTRACT.md`. That is also the point to pass a summarising `--title` if the
|
||||
candidate collected several bump titles along the way; without one, the heading simply keeps
|
||||
whichever bump last set it.
|
||||
|
||||
8. **Write the entry's prose - a section for a larger change, nothing for a smaller one.** `bump`
|
||||
writes no prose on purpose. Decide per bump: a smaller change is done once its title is in
|
||||
the bump list. A larger one gets a `###` section for its topic - one paragraph, in the budget
|
||||
of § The candidate model, on what changed and why, which is the one thing a future reader
|
||||
cannot reconstruct from the diff. If its topic already has a section, rewrite that paragraph
|
||||
to cover both bumps rather than adding another. The full rationale of a decision belongs in
|
||||
the issue tracker or the commit history; a paragraph pressing against its 1 000 bytes is a
|
||||
sign that part of it belongs there instead. The **summary** is a short paragraph (a few
|
||||
sentences) written once, at release time, right below the graded bump list: what this
|
||||
release is about, and why, for a reader who will not read the sections underneath.
|
||||
`version release` refuses to close an entry that collected two or more bumps and has no
|
||||
summary - a one-bump entry is exempt, since there its own paragraph already reads as one.
|
||||
|
||||
## Decision points
|
||||
|
||||
- **The change ships no code - only `README.md`, `INSTALL.md`, pages under `docs/`, or `.gitea/`?** No
|
||||
bump at all; CI's version gate is scoped to what changes behaviour.
|
||||
- **A break you can see coming but are not making yet?** File it as an issue and let it
|
||||
accumulate. Boundary crossings are cheaper in batches, and step 4's "defer" alternative is
|
||||
only real if someone wrote the break down.
|
||||
- **Unsure between MINOR and boundary-crossing?** It is boundary-crossing. The cost of an
|
||||
unnecessary MAJOR is one extra release note; the cost of a MINOR that actually breaks is an
|
||||
instance whose update path fails while its version number promised it would not.
|
||||
- **The break only affects this repository, not a distributed instance** - something under
|
||||
`instructions/dev/`, say? Then it is not a stack break at all: `dist export` never shipped it.
|
||||
Judge by what an *exported* instance sees.
|
||||
|
||||
## Scope
|
||||
|
||||
Applies to the stack version in `VERSION` - `tools/`, `types/`, `instructions/`, `AGENTS.md`
|
||||
and the contracts. It says nothing about the content shape in `.wikitool-kb.json`, which is
|
||||
advanced by `wikitool migrate done` and described by [migrate-corpus.md](../migrate-corpus.md),
|
||||
and nothing about wiki content operations, which are logged in `kb/log.md` and carry no version
|
||||
at all.
|
||||
|
||||
Choosing the part remains a judgment call, deliberately: `docs verify` checks that a
|
||||
boundary-crossing entry *documents* its break and its migration, never that the part was chosen
|
||||
correctly. No validator can tell a renamed flag from a new one.
|
||||
|
||||
## Case study: 2.0.0
|
||||
|
||||
The Chemenu rebranding renamed the repo, the release artefact and the Python package. No page
|
||||
in `kb/` changed, so the first attempt was `1.9.0` - the migration question, answered correctly,
|
||||
substituted for the compatibility question, which was never asked. Three things broke: every
|
||||
existing instance's `update_url` pointed at a repo path that no longer existed and could not be
|
||||
hand-repaired; the artefact name changed; the import name changed. The correct bump was
|
||||
`--major --no-migration`, and the `CHANGES.md` entry for `2.0.0` carries the reasoning in full
|
||||
under "Warum das trotzdem MAJOR ist". The error was caught by the user, not by the
|
||||
documentation - which is what step 4 is for.
|
||||
@@ -0,0 +1,128 @@
|
||||
---
|
||||
type: types/instruction.md
|
||||
name: evolve-subtypes
|
||||
description: Extend or drain a page type's subtype vocabulary - entity_type, concept_type, or source_type - once real material has outgrown it, and the invariants that keep the resulting values honest.
|
||||
manual: true
|
||||
---
|
||||
# Extend a subtype vocabulary, or drain its catch-all
|
||||
|
||||
A subtype field (`entity_type`, `concept_type`, `source_type`) partitions one page type into
|
||||
areas via that type-spec's `layout:` - `types/type-spec.md` has the anatomy. Every one of these
|
||||
three enums is instance-owned content, not stack vocabulary: `entity.md`, `concept.md` and
|
||||
`source.md` all carry `root: kb` and ship only as `.template`, so an instance's own list of
|
||||
values is exactly as much its own decision as its collection contracts are
|
||||
([kb-profiles.md](kb-profiles.md) is the palette that seeds it). This instruction is the
|
||||
procedure for changing that list once it is running, not for choosing it the first time -
|
||||
[setup-instance.md](setup-instance.md) does that.
|
||||
|
||||
The tooling for this loop already exists end to end; this file only names the sequence and the
|
||||
two rules that keep it from repeating what created `kb/sources/`'s old `notes` catch-all in the
|
||||
first place: a schema `default:` that the compiler applied whenever nobody
|
||||
disagreed, silently turning the least specific value into the collection point for everything
|
||||
unclear.
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
|
||||
- [When to run](#when-to-run)
|
||||
- [Steps](#steps)
|
||||
- [Decision points](#decision-points)
|
||||
- [Scope](#scope)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
## When to run
|
||||
|
||||
- A `wikitool lint` advisory finding reports pages sitting in a subtype's catch-all value (for
|
||||
`source`, that is `unclassified` - visible in the catalog, carrying no default) and there are
|
||||
now enough of them to warrant a real value.
|
||||
- A real ingest keeps producing pages that do not fit any existing value for a subtype field,
|
||||
and forcing them into the nearest existing one would misclassify them.
|
||||
- The catch-all itself needs draining after a value was added, so it does not become a second,
|
||||
quieter collection point.
|
||||
|
||||
Not for renaming or removing a value that pages already carry under - that moves pages and is a
|
||||
corpus migration ([migrate-corpus.md](migrate-corpus.md)), not this loop. Not for the one-time
|
||||
choice of an instance's starting vocabulary - that is
|
||||
[setup-instance.md](setup-instance.md)'s KB-language step, seeded from
|
||||
[kb-profiles.md](kb-profiles.md).
|
||||
|
||||
## Steps
|
||||
|
||||
1. **See what has actually collected in the catch-all**, before touching anything:
|
||||
|
||||
```bash
|
||||
tools/wikitool lint
|
||||
tools/wikitool search --field source_type=unclassified # or the equivalent <x>_type
|
||||
```
|
||||
|
||||
Read every page the search returns. A count alone does not say whether the pages share one
|
||||
real category or three - that judgment is the reason this step exists rather than being
|
||||
folded into the next one.
|
||||
|
||||
2. **Apply the value-and-sweep rule: a new value is added and populated in the same pass, never
|
||||
one without the other.** A declared value that no page carries yet is a category that invites
|
||||
a guess the next time someone has to pick between it and the catch-all - which is exactly how
|
||||
the old `notes` default absorbed 22 of 29 source pages before anyone noticed. So:
|
||||
|
||||
1. Count real candidates first: `tools/wikitool search --field <x>_type=<candidate-guess>`
|
||||
will find nothing yet, so count by reading the catch-all's pages from step 1 instead.
|
||||
2. **Apply the ≥3-page admission threshold.** A value is admitted *after* it has proven
|
||||
itself against real material, never in expectation of some. `spec` and `image` are the
|
||||
cautionary case: both were added to `source_type` ahead of any matching page, both sat at
|
||||
zero for a year, and both were eventually removed again unused. A smaller count is only
|
||||
ever an operator's explicit, named exception (`tracker` survived that same cleanup at two
|
||||
pages, kept because the corpus was expected to grow into it from ongoing issue ingests) -
|
||||
never a reason to lower the threshold itself.
|
||||
3. Add the enum value in the type-spec's schema (`types/<t>.schema.yaml`) and its `layout:`
|
||||
entry (`types/<t>.md`) in the same edit - a value with no `layout:` line has nowhere to be
|
||||
moved to.
|
||||
4. Reclassify every candidate page in the same pass:
|
||||
|
||||
```bash
|
||||
tools/wikitool touch --page "<Title>" --set <x>_type=<new-value>
|
||||
tools/wikitool move --reconcile
|
||||
```
|
||||
|
||||
5. Rebuild and check:
|
||||
|
||||
```bash
|
||||
tools/wikitool index rebuild
|
||||
tools/wikitool migrate verify --from HEAD --fail-on-error
|
||||
tools/wikitool lint --fail-on-error
|
||||
```
|
||||
|
||||
`migrate verify` on a subtype sweep should report moved pages and zero findings - a
|
||||
`<x>_type` change alone touches no wikilink, citation, footnote or H1.
|
||||
|
||||
3. **Draining the catch-all is the same loop, run without step 2.2's threshold** - a page
|
||||
sitting in `unclassified` (or the equivalent) already has an intended home; the only question
|
||||
is which existing value it belongs to, which step 1's read-through already answered. Skip
|
||||
straight to reclassifying it (step 2.4) and rebuilding (step 2.5).
|
||||
|
||||
4. **Record it** with `tools/wikitool log append`, describing what moved and why, the same way
|
||||
any other corpus change is logged.
|
||||
|
||||
5. **A newly added value scaffolds with the type's `## Template` block** until it has a subtype
|
||||
template of its own. Whether its pages want a different skeleton is a separate judgment, made
|
||||
against the pages once they exist, by the same ≥3-page rule as above:
|
||||
[subtype-templates.md](subtype-templates.md).
|
||||
|
||||
## Decision points
|
||||
|
||||
- **The catch-all is empty and lint reports nothing?** Nothing to do - an empty catch-all is the
|
||||
success state, not a problem this instruction exists to fix.
|
||||
- **A candidate page could plausibly fit two existing values?** Resolve it by rereading the
|
||||
collection's `COLLECTION.md` for the distinguishing rule (for `sources`, authorship decides
|
||||
`analysis` vs. `document`) before inventing a third value - a genuine gap in the existing
|
||||
values is rarer than an under-read contract.
|
||||
- **More than one subtype field needs the same treatment?** Run this loop once per field; do not
|
||||
try to batch an `entity_type` change and a `source_type` change into one pass, since their
|
||||
admission thresholds are independent judgments about unrelated material.
|
||||
|
||||
## Scope
|
||||
|
||||
Covers `entity_type`, `concept_type` and `source_type` - the three subtype fields with both
|
||||
`subtype_field:` and `layout:` declared. `comparison` has neither and is out of scope by
|
||||
construction. Does not cover the stack-owned capture fields `fidelity`/`authority` on `source`
|
||||
pages: those are fixed vocabulary the stack defines, not an instance's taxonomy - see
|
||||
`types/source.md`.
|
||||
+104
-18
@@ -1,13 +1,14 @@
|
||||
---
|
||||
type: types/instruction.md
|
||||
name: gates
|
||||
description: What to do when wikitool refuses a call - exit 42 (user clearance required) on publish, and the Iteration Budget Gate and loop-breaker on every command.
|
||||
description: What to do when wikitool refuses a call - exit 42 (user clearance required) on publish, sync, upload accept and export guidelines --push, and the Iteration Budget Gate and loop-breaker on every command.
|
||||
---
|
||||
|
||||
# When a gate refuses a call
|
||||
|
||||
Two limits are enforced in code rather than by instruction, because a prompt-level limit is one
|
||||
an agent can talk itself past.
|
||||
The limits listed in [AGENTS.md § Gates](../AGENTS.md#gates) are enforced in code rather than by
|
||||
instruction, because a prompt-level limit is one an agent can talk itself past. This file holds
|
||||
the procedure for each; it does not repeat the list.
|
||||
|
||||
**Never open a gate on your own initiative.** Not `--override-budget`, not `budget reset`, not a
|
||||
`--confirm` token the user has not actually seen and approved.
|
||||
@@ -19,28 +20,39 @@ Read the exit code first - it says which of these applies:
|
||||
| 42 | User clearance required | Reproduce the command's output in your reply, stop. See below. |
|
||||
| 1 | Validation error, or a budget/loop refusal | Read the `ERROR` line; fix and retry once, or stop and escalate. |
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
|
||||
- [Exit 42: user clearance required](#exit-42-user-clearance-required)
|
||||
- [Publish-Remote Gate](#publish-remote-gate)
|
||||
- [Upload Review Gate](#upload-review-gate)
|
||||
- [Guideline Push Gate](#guideline-push-gate)
|
||||
- [Iteration Budget Gate and loop-breaker](#iteration-budget-gate-and-loop-breaker)
|
||||
- [Taking a new session id](#taking-a-new-session-id)
|
||||
- [Scope](#scope)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
## Exit 42: user clearance required
|
||||
|
||||
A `wikitool` command that exits **42** is not reporting an error. It is refusing to act until a
|
||||
human has *read its output*. Three gates use it today - the Mass-Update Gate (`publish`, on a
|
||||
change touching 10 or more counted files), the rebase-review gate (`sync` and `publish`, on
|
||||
a rebase whose incoming commits touch a file this session is also changing), and the
|
||||
Publish-Remote Gate (`publish`, on a push to a target this checkout has not declared) - but the
|
||||
human has *read its output*. Which gates use it is [AGENTS.md § Gates](../AGENTS.md#gates); the
|
||||
rule is about the exit code, not the command:
|
||||
|
||||
> **Copy the command's output into your reply - the substance of it, not a description of it -
|
||||
> and stop.** Run no further commands in that turn.
|
||||
|
||||
For the Mass-Update Gate that substance is the grouped file breakdown: the area headings, every
|
||||
For the Mass-Update Gate (`publish`, on a change touching 10 or more counted files) that substance is the grouped file breakdown: the area headings, every
|
||||
path, and the sizes. It is already ordered for a reader - what a bad publish damages most comes
|
||||
first, and the mechanically-regenerated files come last - so reproducing it in order is both the
|
||||
cheapest and the most useful thing to do with it.
|
||||
|
||||
For the rebase-review gate the substance is different: the commits arriving from the remote,
|
||||
For the Rebase-Review Gate (`sync` and `publish`) the substance is different: the commits arriving from the remote,
|
||||
the files they touch that this session is also touching, and a diff of those files. Read it -
|
||||
this is the check `sync`/`publish` cannot perform themselves, since a rebase between two commit
|
||||
ranges that touch disjoint files never reaches this gate at all (no content collision is
|
||||
possible by construction, so it rebases automatically). Judge whether the incoming change
|
||||
possible by construction, so it rebases automatically). Nor does an overlap only in the files
|
||||
`wikitool` generates - the catalog, `kb/log.md`, `kb/provenance.md` - which carry no decision
|
||||
and are merged and regenerated mechanically; the gate never lists them. Judge whether the incoming change
|
||||
conflicts logically with what you are about to publish, summarize *that judgment*, not just the
|
||||
diff, to the user, and only then re-run with the `--confirm-rebase <token>` the refusal prints.
|
||||
|
||||
@@ -61,15 +73,15 @@ Not a gate you may widen: the prefix list is a constant in the tool. `--path <di
|
||||
scopes a large change into reviewable batches, which is a legitimate alternative to one big
|
||||
clearance.
|
||||
|
||||
Background: [[Mass-Update Gate]] (`kb/concepts/Mass-Update Gate.md`).
|
||||
Background: the [[Mass-Update Gate]] concept page in `kb/`.
|
||||
|
||||
### Publish-Remote Gate
|
||||
|
||||
The Mass-Update Gate asks whether a change is too large to publish. This one asks the question
|
||||
underneath it: **whether this is the right repository to publish to at all.**
|
||||
|
||||
A checkout that holds private content usually has two remotes - its own, and the public upstream
|
||||
it takes stack updates from. Git does not distinguish them at push time, so one wrong `--remote`
|
||||
A checkout that holds private content can have two remotes - its own, and a public one it also
|
||||
works against. Git does not distinguish them at push time, so one wrong `--remote`
|
||||
puts a private corpus on a public repository. That is not cheaply reversible: a force-push moves
|
||||
the branch, but the objects stay fetchable by SHA until someone expires the server's reflogs and
|
||||
runs `git gc --prune=now` on the bare repo.
|
||||
@@ -85,20 +97,75 @@ It pins **URLs, not remote names** - a name-based list would wave through a `pub
|
||||
`pushurl` when one is set, because that is where `git push` actually writes.
|
||||
|
||||
The file is per-checkout and gitignored, for the same reason `ENVIRONMENT.md` is: two clones push
|
||||
to two different places, so a committed copy would tell a private clone that the public upstream
|
||||
is a legitimate target for its own content. **Absent means unrestricted** - a single-remote
|
||||
to two different places, so a committed copy would tell a private clone that a public remote is a
|
||||
legitimate target for its own content. **Absent means unrestricted** - a single-remote
|
||||
checkout with nothing private in it has nothing to protect, and `doctor` reports which state a
|
||||
checkout is in, WARNing only when there is more than one remote and no allowlist. A malformed
|
||||
file is an error rather than "no restriction": a corrupted safeguard must not read as a disabled
|
||||
one.
|
||||
|
||||
**This gate has no `--confirm` token, on purpose.** The other two clear with a token because the
|
||||
**This gate has no `--confirm` token, on purpose.** The others clear with a token because the
|
||||
question they ask ("is this change right?") is one the agent can put to the user and the user can
|
||||
answer for that one changeset. This one asks "does this content belong to that repository?", which
|
||||
is a standing property of the checkout, not a per-push judgment. The way past it is for the user
|
||||
to add the URL to the file. **An agent must never edit `.wikitool-remotes.json` to get past a
|
||||
refusal** - that is opening a gate on your own initiative, which AGENTS.md invariant 6 forbids.
|
||||
|
||||
Arm it *before* a second remote is added and before the first `publish` to it: added afterwards
|
||||
it leaves open exactly the window it closes.
|
||||
|
||||
### Upload Review Gate
|
||||
|
||||
The MCP server's opt-in `submit` tool (`.wikitool-upload.json`) writes into a
|
||||
quarantine, `mcp-upload/`, that no other command reads. This gate is the only door out of it:
|
||||
`wikitool upload accept <id>` refuses without a matching `--confirm <token>`, printing the
|
||||
submission's manifest in full - filename, size, sha256, submitter, and `submitter_source` (the
|
||||
*header* the submitter's name came from, not a verified fact) - plus the exact re-run line.
|
||||
|
||||
Same shape as the Mass-Update Gate, scoped to one submission instead of a changeset: the token
|
||||
digests id/filename/size/sha256/submitter, so an edited or superseded manifest invalidates it the
|
||||
same way a rewritten file invalidates a stale `--confirm`. What a reviewer actually checks before
|
||||
clearing it - secrets, license, an injection attempt, whether the material is worth a source page
|
||||
at all - is [instructions/ingest-queue.md](ingest-queue.md), not this file: the same split as the
|
||||
Mass-Update Gate's review report versus this file's exit-42 procedure.
|
||||
|
||||
`wikitool upload reject <id> --reason "<why>"` is the other way out, and it has **no gate at
|
||||
all** - rejecting needs no clearance, only accepting a stranger's file into the pipeline does. It
|
||||
deletes the material and keeps only the reason and a sha256 in `mcp-upload/ledger.jsonl`.
|
||||
|
||||
### Guideline Push Gate
|
||||
|
||||
`wikitool export guidelines --push` writes this instance's guideline pages, rendered into one
|
||||
generated `GUIDELINES.md`, straight onto the branch of every captured repository that opted in -
|
||||
no pull request, no review on the other side. That is the one write this stack makes into a
|
||||
repository it does not own, and on a public target the content is public the moment it lands. So
|
||||
the first run fetches and builds everything, pushes nothing, and exits 42.
|
||||
|
||||
Its substance, which your reply reproduces in full:
|
||||
|
||||
- **Every target and its status** - `new`, `changed`, `unchanged`, or `skipped` with the reason
|
||||
(a tag rule, not reachable, not opted in, a hand-written file, another instance's file). A
|
||||
skipped line is part of the answer: the user may have expected that repository to be written.
|
||||
- **The diff of `GUIDELINES.md` for every target it would write**, against what that repository
|
||||
carries now. This is what the user approves - the text that will appear there, not a count of
|
||||
repositories.
|
||||
- **The re-run line** with `--confirm <token>`, which you run only after the user approved this
|
||||
exact set.
|
||||
|
||||
The token digests each target's URL, branch, the tip the commit builds on, and the file's
|
||||
content, so a branch that moved, a guideline edited since, or a newly captured repository makes
|
||||
it stale: the next run is gated again with the current state. A run with nothing to write ends
|
||||
with exit 0 and no gate.
|
||||
|
||||
**Not the Publish-Remote Gate.** That one stays on `publish`. The targets here are declared by
|
||||
the committed `_capture.json` manifests under `raw/`; asking for the same URLs in
|
||||
`.wikitool-remotes.json` as well would be a second declaration of the same thing. What this gate
|
||||
asks instead is the per-run question - whether this content belongs in these repositories now -
|
||||
which is the kind a token answers.
|
||||
|
||||
A push the branch moved under is rejected, never forced, and reported as `rejected`; running the
|
||||
whole command again (and clearing it again) serves that repository.
|
||||
|
||||
## Iteration Budget Gate and loop-breaker
|
||||
|
||||
Every `wikitool` call is counted per session. Calls are refused past **60 in a session**, or
|
||||
@@ -135,7 +202,18 @@ When it trips:
|
||||
at exactly the limit it is refused too. The only way past is `--override-budget` on the
|
||||
command you actually need to run, and only with the user's approval.
|
||||
|
||||
`wikitool search` is exempt from this budget entirely: retrieval is reading, not iterating.
|
||||
A dozen commands are exempt from this budget entirely - `search` and `doctor` because retrieval
|
||||
and diagnosis are reading, not iterating, plus the read-only forms of `links`, `cite`, `budget`,
|
||||
`eval`, `version` and `migrate`. The exemption is that allowlist in
|
||||
[tools/CONTRACT.md](../tools/CONTRACT.md), not a "does not change the wiki" rule of thumb: `lint`
|
||||
only writes to gitignored `reports/` and still counts, because it is not on the list.
|
||||
|
||||
**One entry is read-only only in one of its two forms.** `version regrade` lists the running
|
||||
candidate's graded bump titles when called bare, and writes `CHANGES.md` when called with
|
||||
positions to regrade - so the exemption is per *invocation* there, not per command name. It is
|
||||
the only such case; every other row on the list is exempt however it is called. Its
|
||||
`tools/CONTRACT.md` row says which form is which, which is still the single place that list
|
||||
lives.
|
||||
|
||||
### Taking a new session id
|
||||
|
||||
@@ -148,7 +226,15 @@ never in response to a gate refusal.** The plan is the human approval the gate w
|
||||
have to ask for; a refusal means that approval has not been given yet. If you are tempted to
|
||||
re-export the variable after an `ERROR` line, that is the gate working.
|
||||
|
||||
Background: [[Iteration and Cost Limits]] (`kb/concepts/Iteration and Cost Limits.md`).
|
||||
**One tool takes a new id itself, for a fixed set of read commands.** `tools/bugreport.py`
|
||||
([bug-report.md](bug-report.md)) runs `instructions verify` and `docs verify` under
|
||||
`WIKITOOL_SESSION_ID=bugreport-<stamp>`, so that collecting a report neither spends the
|
||||
caller's budget nor is refused by it at exactly the moment something has gone wrong. That is
|
||||
permitted because the script sets the id for those two commands only, never in the caller's
|
||||
shell, and the set is a constant in the script. It is not a precedent for an agent: an agent that
|
||||
exports a `bugreport-*` id, or any other, outside the two cases above is opening the gate.
|
||||
|
||||
Background: the [[Iteration and Cost Limits]] concept page in `kb/`.
|
||||
|
||||
## Scope
|
||||
|
||||
|
||||
@@ -7,14 +7,28 @@ manual: true
|
||||
|
||||
# German terminology for `kb/`
|
||||
|
||||
Reference vocabulary for [kb/CONTRACT.md](../kb/CONTRACT.md#language)'s rule that pages are
|
||||
written in German. The rule lives there; the word list lives here, because it is lookup material
|
||||
rather than a norm and would otherwise be loaded on every write.
|
||||
Reference vocabulary for [kb/CONVENTIONS.md](../kb/CONVENTIONS.md#language)'s rule that this
|
||||
instance's pages are written in German. The rule lives there; the word list lives here, because
|
||||
it is lookup material rather than a norm and would otherwise be loaded on every write.
|
||||
|
||||
**This file belongs to the `german` language profile, not to the stack.** An instance writing in
|
||||
another language deletes or replaces it - see
|
||||
[kb-profiles.md](kb-profiles.md).
|
||||
|
||||
Derived from translating all 248 pages on 2026-08-29. Every entry below is a decision that was
|
||||
made wrong at least once first - each cost a correction pass across published pages, which is why
|
||||
they are written down instead of re-derived.
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
|
||||
- [Stays English](#stays-english)
|
||||
- [Settled German](#settled-german)
|
||||
- [Field labels](#field-labels)
|
||||
- [Register](#register)
|
||||
- [Scope](#scope)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
## Stays English
|
||||
|
||||
Established technical terms are not Germanized, in prose or in headings:
|
||||
@@ -54,7 +68,6 @@ that offered the choice instead of making it. A list of phrases is not a list of
|
||||
|---|---|---|
|
||||
| reconciliation / to reconcile | Abgleich / abgleichen | Not „Abstimmung", including in compounds: `Abgleichsintervall` |
|
||||
| claim | Aussage | **Never** „Anspruch" - that is a legal entitlement |
|
||||
| confidence | Konfidenz | Matches the `confidence:` field and `wikitool confidence decay` |
|
||||
| desired state | Soll-Zustand | |
|
||||
| ownership (in prose) | Verwaltung / Zuständigkeit | But `Ownership Model` and `Ownership-Tier(s)` stay, **including as a heading** |
|
||||
| built-in | -eigen (`K3s-eigen`) | |
|
||||
@@ -98,8 +111,8 @@ none of them structural, so no check found them. It is the one thing to watch fo
|
||||
instructional prose.
|
||||
|
||||
- **Quotations are never reworded**, neither translated nor moved into the impersonal register.
|
||||
- Buzzwords and AI filler are banned by [kb/CONTRACT.md](../kb/CONTRACT.md#tone); the German list
|
||||
is there.
|
||||
- Buzzwords and AI filler are banned by [kb/CONVENTIONS.md](../kb/CONVENTIONS.md#tone); the
|
||||
German list is there.
|
||||
- Dash as ` - `, not `—`.
|
||||
- German number formatting only in prose („10.000 Punkte"). Never inside code, version numbers or
|
||||
measurements (`75-85 px`, `10m`, `0.90`).
|
||||
@@ -108,4 +121,4 @@ instructional prose.
|
||||
|
||||
This is about prose in `kb/`. What is prose and what is an identifier - titles, headings, wikilink
|
||||
targets, cite-ids, enum values, tags, code - is decided by
|
||||
[kb/CONTRACT.md](../kb/CONTRACT.md#language), not here.
|
||||
[kb/CONTRACT.md](../kb/CONTRACT.md#language-and-identifiers), not here.
|
||||
@@ -0,0 +1,115 @@
|
||||
---
|
||||
name: gtd-weekly-review
|
||||
description: Turns the findings from `wikitool review` into decisions and page updates - the GTD Weekly Review, with a machine that prepares the list instead of a human reconstructing it from memory. Use when the user asks for "the weekly review", "review my projects", "what's stalled", or after `wikitool review` has findings nobody has acted on yet.
|
||||
---
|
||||
|
||||
# GTD Weekly Review
|
||||
|
||||
**Purpose:** A finding from `wikitool review` is not an action by itself - "this initiative looks
|
||||
stalled" can mean a next action is missing, the initiative was deliberately paused, or it is
|
||||
actually finished. Which one is true is a human judgment. This skill runs the conversation that
|
||||
collects that judgment and carries it out.
|
||||
|
||||
**Trigger:** The user asks for a weekly review, or `wikitool review` has findings nobody has
|
||||
looked at yet.
|
||||
|
||||
**Before the first `wikitool` call:** `instructions/session-setup.md`.
|
||||
|
||||
**Provider-neutral by design.** Nothing below names a task-tracker provider, a file format or an
|
||||
API - only the tracker's generic role. That is deliberate: this skill is the one document that
|
||||
must read identically in every instance, whichever tracker it runs against.
|
||||
|
||||
**What this skill may write to the tracker, and what it may not.** `wikitool`'s GTD command
|
||||
surface offers exactly two tracker-side writes - `task new` (create one item) and `task close`
|
||||
(mark one item done, never delete it) - alongside `review` (read-only) and `new project` (page +
|
||||
tracker project creation). This skill proposes both writes at the specific findings below, always
|
||||
after the user confirms the exact call, never on its own initiative - the same posture
|
||||
`wiki-ingest` takes toward its own commitment question ("propose one and let the user confirm or
|
||||
correct it"). Everything else a tracker item can need - moving a reminder forward, removing an
|
||||
item outright - stays the user's own action in their tracker: that is a deliberate line, not a gap
|
||||
in the command surface waiting to be filled. Do not reach for a tracker-specific tool or API to
|
||||
"just do it faster" for either half. The reasoning behind keeping the tracker and `kb/gtd/` on
|
||||
separate write paths, and behind stopping at "create" and "mark done" rather than a fuller CRUD
|
||||
surface, lives in `docs/explanation/knowledge-and-commitment.md`, which this skill does not repeat.
|
||||
|
||||
## Steps
|
||||
|
||||
1. **Run the review.**
|
||||
|
||||
```bash
|
||||
tools/wikitool review
|
||||
```
|
||||
|
||||
Exit 0 with no findings means a quiet week - say so and stop. A non-zero exit means the report
|
||||
is **incomplete**: one or more checks could not run because a provider call failed. Read the
|
||||
printed "INCOMPLETE" block, tell the user which checks were skipped and why, and be explicit
|
||||
that the *absence* of a finding under a skipped check means nothing - it was never asked. Do
|
||||
not re-run the command hoping for a different result; a failing provider is not fixed by
|
||||
retrying.
|
||||
|
||||
2. **Walk the findings by check, one at a time.** Each finding names a `kb/gtd/` project (or, for
|
||||
the two checks anchored on the tracker side, a tracker project) and the condition that fired.
|
||||
For every finding, present the options below, ask which applies, and act on the answer -
|
||||
never pick one yourself. A finding is a question, not an instruction.
|
||||
|
||||
| Check | What fired | Options | How to tell them apart |
|
||||
|---|---|---|---|
|
||||
| `stalled` | A tracker project has zero open items and its `kb/` page is `state: active` | (a) A next action is genuinely missing - propose `tools/wikitool task new --title "<title>" --project "<project>"` with a title the user confirms or corrects, asked as one combined question ("Create '<title>' in project '<project>'?"), then run it once confirmed. (b) The initiative is deliberately paused - `tools/wikitool touch --page "<Title>" --set state=dormant`. (c) It is actually finished or given up on - `--set state=completed` or `--set state=abandoned` | Read the page's `## Ziel` and `## Status` sections and ask the user directly: is there still a next step toward that goal, or did this stop for a reason? A pause that was never decided is (a); a pause that *was* decided is (b), never left as `active` with nothing moving |
|
||||
| `waiting_overdue` | A `WAITING` item's `follow_up_at` is older than the threshold | (a) Follow up now, then move the reminder forward in the tracker - the user's own action, there is no `wikitool` command for it. (b) The commitment is no longer needed - propose `tools/wikitool task close --id "<id>"` (the finding's own `item_id`), asked as one combined question naming the item's title and id, then run it once confirmed | Did the person the item names actually come through, and is the ask still relevant? If yes but late, (a); if the need has passed, (b) - never leave the same stale date standing unexamined |
|
||||
| `unpaged_project` | A tracker project has no `kb/` page, past the age threshold | (a) It has grown a memory worth keeping (participants, decisions, context) - `tools/wikitool new project --name "<Name>" --set responsibility=<area>`. (b) It genuinely never needs one - confirm and leave it tracker-only | Ask: would anyone, including the operator in six months, need to know *why* this exists or who is in it? If yes, (a); a project that is fully explained by its own title and task list stays (b) |
|
||||
| `no_open_loop` | A `kb/` page is `state: active` but its tracker project is missing or empty | (a) Same three options as `stalled` above. (b) The name diverged - a rename happened on one side only | Before assuming a stall, check whether a *similarly* named tracker project exists. If it does, this is `instructions/page-lifecycle.md`'s rename case (`tools/wikitool rename` for the page, plus renaming the tracker project to match), not a state change - the review reports both directions of a rename so it never has to be inferred silently |
|
||||
| `someday_stale` | A someday/maybe item has not been touched past the threshold | (a) Activate it - give it a page with `tools/wikitool new project` if it is ready to become a committed initiative. (b) Strike it - propose `tools/wikitool task close --id "<id>"` (the finding's own `item_id`), asked as one combined question naming the item's title and id, then run it once confirmed. (c) Leave it - still genuinely "maybe" | Would the user commit to starting this today? If yes, (a). If it no longer belongs on the list at all, (b). If it is still worth keeping but not yet, (c) is a legitimate answer, not inaction - do not force a decision the user is not ready to make |
|
||||
| `waiting_no_follow_up` | A `WAITING` item carries no `follow_up_at` at all - the provider had nothing to judge staleness against, so `waiting_overdue` could not even ask the question | (a) Set a follow-up date on the item, in the tracker itself - there is no `wikitool` command for this, same as moving a reminder forward. (b) Leave it open-ended deliberately - some commitments genuinely have no date yet | Ask whether there is a date to follow up on at all. If yes, (a); if the item is a genuine "whenever they get back to me", (b) is legitimate, but say so plainly rather than treating the finding as resolved by itself |
|
||||
| `project_age_unknown` | A tracker project has no determinable creation date - the provider could not supply one (an empty project, or a server that never reports it), so `unpaged_project` could not judge its age either way | (a) Judge it on its own merits regardless of age - if it clearly deserves a `kb/` page now, `tools/wikitool new project --name "<Name>" --set responsibility=<area>`. (b) Leave it - it becomes ordinary `unpaged_project` material once it does gain a determinable age | There is no date to reason from here, unlike `unpaged_project` - ask the same "would anyone need to know why this exists" question from that row, but without an age argument on either side |
|
||||
|
||||
3. **Record what was decided or learned on the page - never the task list.** A decision made this
|
||||
week (a scope cut, a direction change) goes under `## Entscheidungen`; something that showed
|
||||
itself in the course of the work goes under `## Gelerntes`. Use `tools/wikitool touch` for the
|
||||
frontmatter fields it owns (`state`, `summary`, `provenance`) and edit the body directly for
|
||||
prose, the same as any other page update (`instructions/wiki-manage/SKILL.md` § Updating a
|
||||
page). **The page never summarizes the open-items list** - that is `kb/gtd/COLLECTION.md`'s
|
||||
own rule (its momentary state lives in the tracker, joined to the page only by name), and this
|
||||
skill exists precisely because that join is not automatic.
|
||||
|
||||
4. **Mentions of people stay mentions.** A person named in `## Beteiligte` while working through a
|
||||
finding does **not** get a page, however much this pass is about them - a page is earned only
|
||||
once they matter for the knowledge independent of this one initiative (`types/project.md`
|
||||
§ Authoring guidance). Creating one here, out of the habit of linking what gets mentioned, is
|
||||
the mistake this step exists to head off. Someone who already has a page is the other case,
|
||||
and the same section says what it takes: a `[[wikilink]]` on the mention and a participation
|
||||
edge on the project page.
|
||||
|
||||
5. **Close out.** If any page changed, `instructions/publish-cycle.md`. A pass that only changed
|
||||
tracker state (the user acted on option (a)/(b) above without touching `kb/`) publishes
|
||||
nothing - there is no page diff to carry.
|
||||
|
||||
## Decision points
|
||||
|
||||
- **A finding's `project` name does not match any page you can find?** That is very likely the
|
||||
`no_open_loop`/`unpaged_project` rename case in step 2's table, not a data error - check there
|
||||
before assuming the join is broken.
|
||||
- **The user wants to skip a finding without deciding?** That is a legitimate outcome for
|
||||
`someday_stale` (option (c)) and, less often, for a genuinely undecided `stalled` case - leave
|
||||
it and say so plainly in your summary, rather than silently omitting it. It will resurface next
|
||||
week.
|
||||
- **The report was incomplete (step 1)?** Work through whatever findings did arrive; do not treat
|
||||
a skipped check as reassurance that nothing is wrong there.
|
||||
|
||||
## wikitool commands used
|
||||
|
||||
`review`, `touch`, `new project`, `task new`, `task close`, `rename` (via
|
||||
`instructions/page-lifecycle.md`, only for the rename case), `publish`
|
||||
|
||||
**Absent:** moving a reminder forward, and removing an item outright - both stay the user's own
|
||||
action in their tracker. See "What this skill may write to the tracker, and what it may not"
|
||||
above for why the line sits exactly there.
|
||||
|
||||
## Output
|
||||
|
||||
Tracker-side changes the user made themselves, plus whichever `kb/gtd/` pages actually changed,
|
||||
published to `origin/main`.
|
||||
|
||||
**Example triggers:**
|
||||
|
||||
- "Let's do the weekly review"
|
||||
- "What's stalled right now?"
|
||||
Loaded 100 of 573 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user