diff --git a/.gitea/workflows/release.yml b/.gitea/workflows/release.yml index 34e6639..ba1b27d 100644 --- a/.gitea/workflows/release.yml +++ b/.gitea/workflows/release.yml @@ -25,6 +25,11 @@ on: branches: [main] paths: - VERSION + # A release whose job failed after VERSION had already moved cannot be + # retried by a push - the version is not raised again - and a re-run uses the + # workflow file of the failed commit. Dispatching on main runs the current + # file; the "already exists" check below still refuses a second release. + workflow_dispatch: jobs: release: @@ -132,6 +137,17 @@ jobs: EOF cat /tmp/release-notes.md + # Gitea keeps a release note in a TEXT column, which on this instance's + # MySQL holds 65535 bytes. A longer note fails the API call after the + # tarball is built; refusing here keeps the margin visible and the tag + # uncreated. The fix is a shorter CHANGES.md entry, not a cut note. + size="$(wc -c < /tmp/release-notes.md)" + if [ "$size" -gt 60000 ]; then + echo "Release notes are ${size} bytes; Gitea stores at most 65535 on MySQL." + echo "Shorten this version's CHANGES.md entry, then dispatch this workflow on main." + exit 1 + fi + - name: Build the distribution tarball id: build if: steps.version.outputs.skip != 'true' @@ -191,18 +207,21 @@ jobs: run: | set -eu # Creating the release creates the tag, pinned to this commit. - payload="$(jq -n \ + # The payload goes through a file: as one argument it is bounded by + # Linux's 128 KiB per-argument limit, which the 8.0.0 notes exceeded + # ("curl: Argument list too long"). + jq -n \ --arg tag "$TAG" \ --arg target "$GITHUB_SHA" \ --arg name "$TAG" \ --rawfile body /tmp/release-notes.md \ '{tag_name: $tag, target_commitish: $target, name: $name, body: $body, - draft: false, prerelease: false}')" + draft: false, prerelease: false}' > /tmp/release-payload.json release="$(curl -sS -f -X POST "${API}/releases" \ -H "Authorization: token ${TOKEN}" \ -H "Content-Type: application/json" \ - -d "$payload")" + --data-binary @/tmp/release-payload.json)" id="$(printf '%s' "$release" | jq -r '.id')" echo "Created release ${TAG} (id ${id})." diff --git a/CHANGES.md b/CHANGES.md index 23c1ee7..4a6d3ff 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -187,1586 +187,153 @@ Nach dem Update läuft der Preflight einmal, betroffene Seiten werden mit `tools umbenannt, eine Offline-Sitzung braucht für `publish` `--no-push`, und eine Instanz aus einem der entfallenen Installationswege wird einmal aus einem Release neu aufgesetzt. -### raw accept --replaces-bundle: Erfolgsmeldung und Record nennen die A-Liste neben git diff - -`raw accept --replaces-bundle` closed with "`git diff` on the bundle shows the edition's -changes", and its command record said the same. Files the new edition added are still untracked -at that point, so `git diff` never shows them - a session that read only the diff missed every -new file. The success line and the record now say that `git diff` shows the `M` and `D` files -and that the `A` files are only in the list the accept prints, matching what `raw/CONTRACT.md` -and `wiki-ingest` already said. Drop-in: wording only, no behaviour changes (Gitea #181). - -### publish --path nimmt ungespeicherte generierte Dateien außerhalb des Pfads mit - -`publish --path ` staged and committed only ``. When the catalog and log it had written - -or the ones a reconcile had just merged and regenerated - lay outside ``, the push carried -the new page with a catalog that did not list it and a log without its entries, and left the -generated files behind uncommitted until a `publish` without `--path` took them along. Nothing was -lost, but the remote stayed inconsistent in between, and another session's `sync` worked against -that state. - -- **Uncommitted generated files outside `--path` now join the commit** (`kb/index.md`, - `kb/log.md`, `kb/provenance.md`, every `kb/**/INDEX.md`), and a line names them: - `Including N generated file(s) outside --path: ...`. Git decides what lies under ``, by the - same pathspec the staging uses; files already under it are neither added nor named. -- **Regardless of whether this run's reconcile regenerated anything.** A run the Mass-Update - Gate refuses has already reconciled; the confirmed re-run finds nothing left to regenerate, - and a condition on the regeneration would have pushed the inconsistent state on exactly that - path - the same goes for a `sync` followed by `publish --path`. -- **The gate is unchanged.** The added files stay out of its count, and the `--confirm` token - and the re-run line cover the `--path` given, so a clearance issued for `` stays valid. -- **Every other file outside `--path`** is left alone, as before. - -Where uncommitted pages also lie outside ``, the catalog that goes out with `` already -lists them until the next `publish` sends them - the reverse of before, where the page went out -and the catalog did not know it. Drop-in: no command, flag, file format or state file changes. -`publish`'s record follows (Gitea #182). - -### sync record: the autostash note states the behaviour, not the change - -One note in `sync`'s command record, written with 8.0.0-beta.46, said that uncommitted changes no -incoming commit touches "no longer" stop a rebase - a sentence about the change, which a reader of -the record has nothing to compare against. It now states what `sync` does (Gitea #180). - -### publish/sync merge generated files mechanically and carry non-overlapping uncommitted work through a rebase - -Every ingest touches `kb/index.md` and `kb/log.md`, so two ingests on two checkouts - or two -sessions - always "overlapped", in files that carry no decision. `reconcile` (behind `sync` and -both of `publish`'s reconciles) failed on all three shapes of that: an uncommitted ingest stopped -the fast-forward with exit 1; two committed ingests went to the rebase-review gate and, once -cleared, failed on the two appends at the end of the log; and any uncommitted file at all made -`git rebase` refuse, so a stranded commit plus a new ingest could not be published. - -- **Generated files never count as an overlap.** The rebase-review gate, its token and its diff - leave out `kb/index.md`, `kb/log.md`, `kb/provenance.md` and every `kb/**/INDEX.md`; an - overlapping *non*-generated file still goes to review. -- **Where they are what stops git, they are merged mechanically.** Uncommitted generated files - are set aside (the log's appended bytes kept, the rest discarded); rebase conflicts in them are - resolved step by step - the catalog and provenance take the side being rebased onto, the log - takes that side plus what the replayed commit appended. Afterwards the log gets the set-aside - entries back at its end and the catalog and `kb/provenance.md` are regenerated in-process, left - as an uncommitted change. `index rebuild` and `sources rebuild-index` lend their writing half - for that as `index_build.write_index` and `provenance_cmd.write_provenance_index`. -- **Uncommitted work no incoming commit touches rides through a rebase** (`git rebase - --autostash`) and comes back byte-identical. -- **Before the first write, the working tree is stored** as a commit under - `refs/wikitool/reconcile-backup`; a success drops it, a failure restores the working tree and - names the ref. -- **`publish`'s retry after a rejected push** commits the regenerated files on their own before - the second push, since staging is long past by then. -- **Unchanged where git manages on its own:** a reconcile that succeeded before runs the same git - command and regenerates nothing - `index rebuild` stamps today's date into `kb/index.md`, and a - rebuild there would leave every `sync` with a modified file. A log edited rather than appended - to, or an uncommitted change on a path the incoming commits change, stops exactly as before. -- **`is_generated` is narrowed to `kb/`**, as AGENTS.md invariant 1 already defines it: an - `INDEX.md` in a captured repository under `raw/` is source material, counted by the Mass-Update - Gate and never discarded. - -Drop-in: no command, flag, file format or state file changes, and the previous version ignores a -leftover backup ref. Cases that ended in exit 1 or 42 now go through. A `--confirm-rebase` token -issued before the update may read as stale afterwards, because the overlap it covers no longer -lists generated files - the gate then asks again, as for any stale token. `sync`'s and -`publish`'s records, `instructions/session-setup.md` and `instructions/gates.md` follow (Gitea -#180). - -### kb/CONVENTIONS.md.template: the guideline filter is a default, not a setup question - -The § Guidelines for other repositories section shipped with a `{...}` placeholder, and -`setup-instance.md` reads every placeholder in that template as a question to put to the user - -so the export's filter had quietly become a new setup question, one a fresh instance with no -pages cannot sensibly answer and `INSTALL.md` does not list. The section now states a default, -`--tag guideline`, to be changed in place once the instance decides otherwise. This instance's -own `kb/CONVENTIONS.md` points at `search --tag guideline` for the pages that carry the tag; four -concept pages now do (Gitea #179). - -### export guidelines: the guideline pages as a generated GUIDELINES.md, pushed into the captured repositories behind the Guideline Push Gate - -The other direction of `raw capture`. Project repositories each carried their own copy of the -same agent rules, drifting apart; the instance now holds them once and delivers them as a -generated, committed file, so each repository works on its own - in CI, without MCP. - -- **`export guidelines `** selects pages with `search`'s own predicates and no text - (`--field`, `--kind`, `--subtype`, `--collection`, `--tag`), and prints one `GUIDELINES.md`. - Line 1 is ``; - then each page by title as `# ` and its text without frontmatter, the generated links and - footnotes regions and citation markers, with wikilinks turned into their text - code left - byte for byte. No timestamp and not `HEAD`: the same pages give the same bytes, so an instance - commit that touches no guideline changes no target repository. It refuses with no predicate, no - match, an unknown field, any unreadable frontmatter under `kb/`, or uncommitted changes in `kb/` - or `types/`. -- **`--push`** writes that file into every captured repository - the `(repo, ref)` of each - `_capture.json`, `--bundle` narrowing it - that opted in by carrying a `GUIDELINES.md` whose - first line is the export header; a header line alone is the opt-in. A hand-written file, one - from another `instance=`, a tag rule, and an unreachable repository are skipped with the reason. - Each written target gets one commit on the fetched tip changing only `GUIDELINES.md`, built with - plumbing in the bare capture cache (no working tree), authored as the instance checkout's - `user.name`/`user.email`, and pushed without force; a branch that moved since the fetch is - rejected, read from `push --porcelain`'s status flag rather than git's translated messages. -- **The Guideline Push Gate** - a fifth named gate. `--push` without a matching `--confirm` - pushes nothing and exits 42 with each target's status, the diff of every file it would write, - and the re-run line; the token digests URL, branch, old tip and new blob of every target to - write. Nothing to write means no gate. The Publish-Remote Gate does not apply - the targets are - declared by the committed manifests already. -- **No way back in:** `raw accept` refuses a file whose first line starts with - `<!-- wikitool:export`, given on its own, inside a folder, or as `--replaces`/`--replaces-bundle` - material - `raw capture` already left such files out. `EXPORT_MARKER` now lives once, in the new - core module `guideline_export.py`. - -Which pages are guidelines is an instance decision: `kb/CONVENTIONS.md` (and its template) has a -new section § Guidelines for other repositories; this instance's filter is `--tag guideline`. -`kb/CONTRACT.md` says what leaving the wiki does to a page; `raw/CONTRACT.md`, `AGENTS.md` -§ Gates, `instructions/gates.md`, `docs/why-gates-are-code.md`, `README.md` and -`tools/README.md` follow. Internals: `kb_scan.LINK_RE` (moved from `commands/page_ops`, which -re-exports it), `filters.raw_predicates` shared with `search`, `repo_capture.run_git_result` and -`captured_manifests` shared with `raw status`. - -A new command and a new refusal for files no tool produced before - drop-in in both directions, -no page or manifest changes (Gitea #179). - -### wiki-ingest: updating the captured repositories is a step-1 branch over raw status - -`raw status`, `raw capture --update` and `raw accept --replaces-bundle` existed, but nothing told -a session what to do when asked to update the captured repositories: the decision point for a new -edition began with "`raw status` reported it" and nobody ran `raw status`. Step 1 of `wiki-ingest` -now has that entry, built like the existing branch over `raw pending`. It runs `raw status`, ends -the run when nothing changed (naming every unreachable or refused repository with its reason), -otherwise takes one changed bundle per run (the one the user named, else the first in the output), -announces it with `old -> new` and how many others changed, and runs the `raw capture --update` -line `raw status` printed. From step 5 the existing decision point carries the run. The user never -names a commit or a path. The skill's description and example triggers name the request ("update -the captured repositories", "pull the repo docs"). - -The decision point now says what the edition diff is: `git diff -- <raw-bundle>` for changed and -removed files, plus the `A` list `raw accept` prints, because added files are still untracked and -never show up in `git diff`. `raw/CONTRACT.md` § "Getting a repository in" and `README.md` say how -an update is started. - -Instruction text only, no interface changed - drop-in in both directions; an older version simply -lacks the branch (Gitea #178). - -### raw accept: an occupied folder name held by a captured bundle points at --replaces-bundle - -The ON FAILURE reaction `raw accept incoming/<folder>` prints for an occupied name still said -"rename the folder - there is no `--replaces` for a folder". For the new edition of a captured -bundle that is the wrong route: renaming makes it a second, separate source. The reaction now -names `--replaces-bundle` for a name held by a captured bundle and keeps renaming for every other -folder. `raw/CONTRACT.md` § "Getting a file in" says the same about the capture fields of a -captured folder: they come from its manifest, not from one flag pair (Gitea #177). - -### raw capture / raw status / --replaces-bundle: documentation from git repositories as a bundle, with drift reporting - -Documentation from a git repository used to reach an instance only by hand: copy the files into -`incoming/`, rename them around the global name rule, and remember nowhere which repository and -commit they came from. Three commands replace that: - -- **`raw capture <repo-url> --ref <rule> --path <glob>... --name <bundle> --fidelity <v> - --authority <v>`** resolves the ref rule - a branch, or a tag pattern such as `v*` that takes the - newest matching tag by version order - to one commit. It fetches that commit by ref name, - shallowly, into a bare cache per URL under `tools/.wikitool_capture/` (gitignored, never - exported), and writes the files the globs select into `incoming/<bundle>/` at their repository - paths. Files are read as blobs, never through a checkout, so they are byte-identical to the - repository even under `core.autocrlf`. Globs follow git's `:(glob)` pathspec, checked against - git itself in the tests. `_capture.json` beside the files records repository, ref rule, commit, - globs, capture time, `fidelity`, `authority` and the file list. `--update <raw-bundle>` captures - the current state from that manifest. -- **`raw status`** reports every captured bundle whose files changed in the repository, as - `A`/`M`/`D` grouped by owning source page, and stays quiet about a commit that moved without a - change inside the globs. An unreachable repository is one line, never an abort; `--json` prints - the same report as one object per bundle. -- **`raw accept --replaces-bundle <raw-bundle> incoming/<bundle>`** replaces a captured bundle as - a whole at its existing address. Afterwards it holds exactly the new manifest's files plus - `_capture.json`; files the repository dropped are removed, and emptied directories `rmdir`ed. - `raw_files:` is left alone, as with `--replaces`, and the command prints the `touch --add/--remove - raw_files=` lines that follow. - -Mechanical exclusions, each named in the output: a file whose first line starts with -`<!-- wikitool:export`, symlinks, submodules, files over 25 MiB, hidden path segments, files -named `_capture.json`, and Git LFS pointers. Git runs with the host's credentials, limited to -`ssh`/`https` through `GIT_ALLOW_PROTOCOL`, with no terminal prompt, no askpass and a timeout. A -repository asking for a password is reported as unreachable rather than hanging an unattended -run. A URL carrying a password is refused, because the manifest is committed. - -`raw accept` takes a captured folder's capture fields from its manifest only, and refuses them on -the command line. `--replaces` and `--page` refuse a target inside a captured bundle. - -**Fixed along the way:** `sources coverage` and `lint` ignored every `CONTRACT.md` under `raw/` -at any depth, not just the stage's own `raw/CONTRACT.md`. A captured repository's contract -file would have been invisible. The exclusion is now anchored to `raw/CONTRACT.md`, and -`_capture.json` is excluded instead. An instance with a `CONTRACT.md` somewhere below `raw/` may -see it reported as uncovered for the first time - a finding about a file that was always there, -nothing to migrate. - -Drop-in in both directions: without a `_capture.json` under `raw/` no existing command behaves -differently. An older version would report a manifest as an uncovered raw file. `raw/CONTRACT.md` -§ "Getting a repository in" and `wiki-ingest` describe the workflow (Gitea #177). - -### Comparison and source pages accept the sources: that cite add writes; sources may cite sources - -`cite add` writes the cited source into the page's `sources:`, but neither the `comparison` nor -the `source` type declared that field, and both schemas set `additionalProperties: false`. So a -comparison page cited the way `kb/comparisons/COLLECTION.md` asks for, or a source page citing one -of its own raw files with `--file`, failed `lint` with `'sources' was unexpected`. Both types now -declare `sources` as an optional array, in the schema and in `page_ref_fields:`. The second part -matters: `rename`/`rm` carry only declared reference fields, and `lint` resolves only declared -fields. A fix to the schema alone would leave dead `sources:` entries behind after a source is -renamed, and nothing would report them. - -That also makes a citation between two sources legal: `cite add --page "Source - A" --source -"Source - B"` records `Source - B` on A and leaves B alone. Two changes keep that edge directed: - -- `cite add` no longer writes a page's own title into its `sources:`. A source page citing its own - raw file gets the footnote definition and leaves `sources:` untouched, not even as an empty - list. `lint` and `citing_pages()` already ignored the self-edge. -- `xref link-source` refuses a target that is itself a source page. It writes nothing for that - target, names it with the `cite add` command to use instead, links the others and exits 1. It - used to write `B.sources += A`, claiming the citation in the wrong direction, onto a page whose - schema then rejected it. - -`kb/CONTRACT.md` § Provenance and citation and `wiki-ingest` step 9 describe both. - -**What an existing instance has to do by hand.** The schema and type-spec of `comparison` and -`source` belong to the instance once adopted. `dist upgrade` only renews the `.template` beside -them, so the type half of this change does not arrive on its own. For each of the two types: - -1. Copy the `sources` property from `types/<type>.schema.yaml.template` into - `types/<type>.schema.yaml`. -2. Add `sources` to `page_ref_fields:` in `types/<type>.md`. Without it a rename leaves the - entries pointing at the old title. -3. Copy the `sources` row of the frontmatter table into the same file. - -An instance that has already loosened only the schema still needs step 2. The `cite add` and -`link-source` changes are under `tools/` and arrive with the normal upgrade, so a source page -citing one of its own raw files validates without the steps. Until they are done, `lint` keeps -reporting a comparison page that carries a citation, or a source page citing another source, as -a schema error, exactly as before. Nothing that worked before breaks. Found while writing the first comparison page of an -instance on 7.0.0 (Gitea #173). - -### wiki-ingest and wiki-manage call xref add with --rel, not the --rel-a/--rel-b removed in 4.0.0 - -Since 4.0.0 `xref add` declares one directed edge and takes a single `--rel`, but the -cross-reference step of both skills still showed the mirrored form with `--rel-a`/`--rel-b`, so -every session that followed it failed with `No such option` and had to recover the syntax from -`-h`. Both now show `xref add --a --b --rel` with the reading `[A] <label> [B]` as a comment, and -point at `kb/CONTRACT.md` § Linking for direction and reverse edges instead of restating the rule. -A scan of every documented `wikitool` call against the command tree found no other stale flag. -Found during an ingest in an instance on 7.0.0 (Gitea #174). - -### kb/CONTRACT.md names unwritten scaffold sections and the Unfilled Template Sections finding - -The stage contract pairs each page property with the `lint` finding that checks it, and its -"Every page should" list did not yet name the one the previous bump added. It now says that a -page leaves no section of its scaffold unwritten, that `lint` reports a section still made of -nothing but `TODO` placeholders as advisory *Unfilled Template Sections*, that the fix is writing -it from a source or retiring the page, and that a field the source does not give may keep its -`TODO` beside written lines. Found in the closing pass of Gitea #94. - -### lint: Unfilled Template Sections - a section still holding only its template's TODO placeholders (advisory) - -`lint` had no finding for a page without substance: every check measured structure, links, -provenance or schema, so a page carrying only its `wikitool new` scaffold and one line read the -same as a complete one - and a later session took its subject as covered and stopped looking at -the source. The new advisory finding `unfilled_sections` names each page with a `##` section -whose non-blank lines are all template placeholders, with those sections' headings. A -placeholder line is one whose content starts with the token `TODO` (after an optional list -marker, checkbox, table cell or bold field label); code, generated regions and footnote -definitions are out of view, and neither an empty section nor a single open field beside written -ones is reported. Word count and "page carries a placeholder" were measured against two corpora -and rejected - both misjudge short complete pages or pages with one honest gap marker; the -section rule separated stubs from complete pages in both. On the demo corpus it names 34 pages. -`types/type-spec.md` now makes `TODO` the contract for placeholder text in a template, an -instance-owned one included; `wiki-lint` keeps the finding out of its mechanical repair step; -`eval score` counts it under `advisories`. Not a hard error, so no instance's -`lint --fail-on-error` turns red on this upgrade (Gitea #94). - -### kb/CONTRACT.md names section anchors in wikilinks and the Broken Anchors finding - -The stage contract's rules for writing a wikilink (§ Titles are identifiers) covered wrapped links -but not `[[Title#Section]]`, which the previous bump made a checked form. It now says that the -title part is the reference (the graph counts it, `rename` carries the anchor), that a stale -anchor is reported as advisory *Broken Anchors*, and that a section is never an edge target. -Found in the closing pass of Gitea #172. - -### Organisationsseiten: Personen als Abschnitt mit Aufstieg, entity_type organization, member-of, Lint-Befund broken_anchors - -Personen, von denen eine Quelle nur Namen, Rolle und Tätigkeitsbereich hergibt, waren bisher -Stub-Seiten - weit unter dem `Stub Threshold` und gegen die Regel aus `wiki-ingest` Schritt 7, nach -der ein Thema nur mit Material eine Seite bekommt. Nach Kunde gruppieren ließen sie sich auch -nicht, weil `entity_type` genau eine Area wählt. Jetzt stehen sie als `###`-Abschnitt unter -`## Personen` auf der Seite ihrer Organisation und steigen erst zur eigenen Seite auf, wenn eine -Quelle Material dafür hergibt; dann trägt die Personenseite `member-of` auf die Organisation. -Dazu kommt ein eigener `entity_type: organization` mit Area `organizations/` und eigener -Seitenvorlage (`types/entity.organization.md`). `person` meint nur noch Menschen, und -`E3DC GmbH` ist im Korpus umgezogen. Der Aufstieg ist eine Prozedur in -`instructions/page-lifecycle.md` und bewusst kein Kommando, weil zwei seiner Schritte Urteile sind. -Was er stillschweigend kaputt machen kann, meldet `lint` jetzt als **Broken Anchors**: ein -`[[Seite#Abschnitt]]`, dessen Seite existiert, aber keinen solchen Abschnitt mehr hat. Der Befund -ist nicht hart, damit der Lint einer bestehenden Instanz nach dem Upgrade nicht rot wird, wo er -vorher grün war (Gitea #172). - -### Link-Katalog: Beteiligungs- und RACI-Label von der Projektseite aus - -Der Katalog hatte für Beteiligung kein Label - nur `owns`, `maintains` und `authored`, alle von der -Person aus, und wer mitwirkt, ohne einzustehen, landete bei `owns` (zu stark) oder `see-also` -(nichtssagend). Neu in `instructions/link-taxonomy.md` § Operational: `involves` (beteiligt, Rolle -offen) und die RACI-Label `staffed-by`, `owned-by`, `consults`, `informs`. Sie werden **auf der -Projekt- bzw. Codebase-Seite** geschrieben und zeigen auf Person oder Organisation, weil dort die -Frage „wer ist beteiligt?" gestellt wird und eine Seite nur ihre eigenen Kanten im Links-Block -zeigt. `owned-by` ist die Gegenrichtung von `owns` - die vierte Inversen-Paarung - statt eines -zweiten Wortes für dieselbe Aussage. `kb/gtd/COLLECTION.md` und `kb/entities/COLLECTION.md` -autorisieren `involves` und `owned-by`; die RACI-Label nimmt eine Instanz für Kundenprojekte in -ihre eigene `COLLECTION.md` auf. `types/project.md`, `kb/gtd/COLLECTION.md` und der Skill -`gtd-weekly-review` sagen jetzt, was mit einer Person in `## Beteiligte` passiert, sobald sie eine -eigene Seite hat: Wikilink auf die Erwähnung, Kante auf der Projektseite. Kein Code kennt die -Label; eine bestehende Instanz merkt nichts, bis sie eines autorisiert (Gitea #118). - -### The test suite no longer ships, and dist upgrade deletes what a release stops shipping - -`dist export` used to ship all of `tools/chemenu/tests/` with `tools/pytest.ini` and -`tools/.coveragerc` - about a quarter of every release's files. The suite tests the origin -repository: it reads that repository's own type-specs, conventions and collections, where an -instance carries templates or its own decisions, so in a freshly exported tree 257 of its tests -failed. All three now stay behind, matched by exact path, so a directory that only happens to be -called `tests` elsewhere under `tools/` still ships. `tools/README.md` drops its Tests section on -export; `tools/CONTRACT.md`, `EVALS.md`, `instructions/mcp-read-server.md` and -`tools/requirements-mcp.txt` say the suite lives in the origin repository. - -`dist upgrade` used to keep a file the new release no longer ships unless `--prune` was passed. -After that run the new stamp no longer named the path, so no later upgrade could find the file -again. Now the upgrade settles it in the run that sees the file go. If the file is unchanged -since install, it is deleted, just as an unchanged file is overwritten without asking. Any -directory that this leaves holding nothing (at most a `__pycache__/` of `*.pyc`) goes with it. -If the file was changed since install, it is listed as locally changed ("no longer shipped") and -needs the usual answer: `--take-release` deletes it, and `--keep-local` keeps it as the -instance's own file. The upgrade never deletes a file that neither stamp names, and never an -instance-owned seed (`CHANGES.md`, `kb/log.md`, `.wikitool-kb.json`). `--prune` is still -accepted and does nothing (Gitea #113). - -**For an existing instance:** the upgrade *into* this release still runs the installed -`dist upgrade`, which keeps no-longer-shipped files unless told otherwise. Pass `--prune` on that -one run. It then removes the test suite, `tools/pytest.ini` and `tools/.coveragerc`, and also -`instructions/private-instance.md` and `tools/chemenu/commands/upstream_cmd.py`, which this -release drops as well. If they are already left over, remove them by hand, together with any of -these three, which earlier releases dropped: -`tools/chemenu/sections.py`, `instructions/claude-code-model-selection.md`, -`tools/chemenu/commands/confidence_decay.py`. - -```bash -git rm -r --ignore-unmatch tools/chemenu/tests tools/pytest.ini tools/.coveragerc \ - instructions/private-instance.md tools/chemenu/commands/upstream_cmd.py \ - tools/chemenu/sections.py instructions/claude-code-model-selection.md \ - tools/chemenu/commands/confidence_decay.py -``` - -From the next upgrade on, this happens on its own. - -### Page-material passages in type-spec.md, type-guidance.md and language-boundaries.md name subtype templates - -Found in the closing check of the change below: three documents listed what in `types/` is page -material - and therefore in the KB language and the instance's to rewrite - as the `## Template` -block and the `layout:` titles alone. `types/type-spec.md` (§ Who owns a type-spec, its prose and -its language table), `types/type-guidance.md` (§ Conventions) and -`docs/language-boundaries.md` now name the subtype templates `types/<name>.<value>.md` among -them, and a comment in `dist_cmd` does the same for `dist adopt`'s scope. No behaviour change -(Gitea #117). - -### new: a subtype gets its own page skeleton from types/<type>.<value>.md - -A type-spec carried one `## Template` block for every value of its subtype field, so `wikitool -new` scaffolded a person with `Version`, `Sprache/Technik` and `Repository`, and a decision with -"Wann zu verwenden" instead of context and consequences - and authors rebuilt those pages by -hand. Now a file `types/<type>.<value>.md` beside the type-spec is the skeleton for pages whose -subtype field holds `<value>`: plain markdown in the KB language, no frontmatter, its name the -only declaration. It replaces the block whole; every subtype without such a file, and every type -without `subtype_field:`, scaffolds byte-identically to before. `guidance` is reserved for -`<type>.guidance.md`. The stack ships two: `types/entity.person.md` (Rolle, Zugehörigkeit, -Wirkungszeitraum, Beiträge) and `types/concept.decision.md` (Kontext, Entscheidung, Alternativen, -Konsequenzen, Status). - -`docs verify` reports a subtype template whose `types/<type>.md` is no type-spec with -`subtype_field:`, whose value the field's enum does not allow, or which carries frontmatter - -each would otherwise be a file nothing reads. Subtype templates never get a table-of-contents -region, since `new` copies them into every page. They are page material of an instance-owned type, -so `dist export` ships them as `.template`, `find_leaks` refuses one under its own name, and -`dist adopt` takes them; no export carries one unsuffixed, so no `dist upgrade` writes over an -adopted one. A new manual instruction, `instructions/subtype-templates.md`, is the interview that -finds subtypes whose pages depart from their template (at least three pages, the same way) and -writes the template the user accepts. `types/entity.md` and its guidance no longer speak of -"projects" since the subtype became `codebase`. - -**For an existing instance:** nothing changes until it adopts a template - `dist upgrade` lays -`types/entity.person.md.template` and `types/concept.decision.md.template` beside the adopted -type-specs; `tools/wikitool dist adopt <path>` takes one, leaving it takes none. Both are valid. -No page changes (Gitea #117). - -### lint: a wikilink wrapped across a line break is its own finding; rename and rm see it - -A `[[...]]` written across a line break - prose wrapped at a fixed column, with the break -landing inside the link - was read with the break as part of its title. `lint` reported the -target as a missing page, with the line break in the printed name; `rm`'s inbound check did not -count it, so a page referenced only that way was deleted without the `--yes` confirmation; and -`rename` left it pointing at the old title. A link target is now read through one function, -`kb_scan.normalize_link_target()`, which folds a line break and its indentation to one space - -used by the link graph, the index check, `rename` and `rm` alike. That such a link is wrapped -is still a defect: `lint` reports it in a new hard section, *Wrapped Wikilinks*, and reports it -as broken as well only if the folded title is missing. No instance's lint gets redder: every -link the new section names was a hard broken-link finding before. `kb/CONTRACT.md` § Titles are -identifiers states the rule (Gitea #115). - -`stack-close`'s closing comment no longer carries a per-phase table of model, effort, session -shape and compaction, nor the `size/` label beside it; the comment is the changelog line, then -the issue closes. The operator decided the models used need not be documented (Gitea #170), so -the record is dropped without a replacement, together with the sentences that leaned on it: `stack-mode.md` § Sessions -and models, and two in `docs/model-and-effort-selection.md`. Which model suits which phase is -unchanged. Dev-only apart from one `docs/` sentence; no behaviour change. - -### raw fetch --html record and kb/CONTRACT.md path budget follow the folder accept - -Two sentences the previous change left behind, found in its closing check. `raw fetch --html`'s -command record still named a file "not under `incoming/`" as its refusal, though since the -previous change a saved page inside a subdirectory of `incoming/` is refused too; it now says -"not directly in `incoming/`" and names moving the file up as the fix. `kb/CONTRACT.md`'s path -budget paragraph named only renaming a file in `incoming/` as the remedy for `raw accept`; for a -folder the remedy is a shorter folder name or shorter names inside it, and it now says so. No -behaviour change. - -### incoming/ as a queue: raw pending picks the next entry, raw accept takes a whole folder - -**Breaking.** A subdirectory of `incoming/` was tolerated and ignored since the date shard -replaced the type directories, so an old `incoming/documents/` habit kept working. It protected -nothing - the kind of source comes from its content now, as `source_type:` - and it left a folder -of files that belong together with no way in: `raw accept` took files at most one level down, -bundled them flat under the first file's stem, lost the folder's name and left the emptied -directory behind, and a folder with subfolders could not be accepted at all. Now a file argument -of `raw accept` (also with `--replaces`) and of `raw fetch --html` must sit directly in -`incoming/`; a file inside a subdirectory is refused, and the message names both ways out. - -**A folder is one source.** `raw accept incoming/<folder> --fidelity ... --authority ...` moves -every file below it to `raw/<YYYY>/<MM>/<folder>/` at the same relative path and removes the -directories left empty. The folder name is the bundle name and falls under the existing name -rule; the files inside do not, so several `README.md` in one tree are no conflict. It is accepted -alone (no other argument, no `--page`, no `--replaces`), and every check runs before anything -moves: an empty folder, or a hidden entry, a symlink or a special file anywhere below it, is -refused with each entry named - which is what guarantees that the clean-up, an `rmdir` per -directory and never a recursive delete, cannot take a file with it. A folder that passes the -large-tree thresholds continues with `work new --input raw/<YYYY>/<MM>/<folder>`, which the -success message prints. - -**`tools/wikitool raw pending`** reads `incoming/` as a queue without changing it: the -top-level entries, with top-level files sharing a stem as one bundle (a `raw fetch` pair) and a -folder as one candidate, oldest first by modification time - a bundle or folder as new as its -newest file - each marked with whether `raw accept` would take it, by the same checks, and why -not. The default is the first acceptable one. `wiki-ingest` uses it when the user names nothing: -it announces the entry it took, ingests exactly that one, and says at the end how many still -wait. `raw/CONTRACT.md` § Getting a file in describes candidates, order and the limit of an -mtime, which is a document's last change only when it was copied with its timestamps kept. - -No migration: no page changes and `raw/` is untouched. An instance whose scripts write to -`incoming/<type>/` changes them to write directly into `incoming/`. - -### kb/CONTRACT.md: an external article's raw_files point under raw/, a raw fetch capture names both files - -`kb/CONTRACT.md` still told a source page for an external article to point `raw_files:` at "the -local copy under `raw/articles/`" - wrong since promotions moved to the date shard, and wrong in -exactly the path `raw fetch` now makes common. It now says the local copy under `raw/`, and that -a page captured with `raw fetch` lists both the received `.html` and the derived `.md`. -`wiki-ingest` gains a URL as a second example trigger. Found in the closing check of #120. - -### raw fetch: a sanctioned intake for a URL into incoming/ (#120) - -A URL the user wanted ingested had no tool and no procedure: every session built its own chain -of `curl`, a guessed character set, boilerplate cut by line number and a header written from -memory, so two sessions turned the same article into two different, permanent `raw/` files. The -new `tools/wikitool raw fetch <url>` fetches the page and writes two files into `incoming/` - the -HTML exactly as received, and a `.md` with a fixed header (`url`, `final_url`, `retrieved`, -`http_status`, `content_type`, `charset` and where it came from, `title`, `derived_from`) above a -Markdown-like text derived from it. `raw accept` then promotes both in one call as one bundle, -unchanged, so the capture fields are still asked once, at the same point of `wiki-ingest`. The -HTML is kept because it is what was received: a claim stays checkable against it even where the -derivation lost something. - -The derivation is deterministic and needs no new dependency (`urllib`, `html.parser`): charset -from the byte-order mark, then the HTTP header, then a `<meta>` in the first 4 KiB, then UTF-8, -with undecodable bytes replaced and reported; content root `<main>`, else a single `<article>`, -else `<body>`, with navigation, header, footer, aside, forms and scripts dropped and links made -absolute. Only `http`/`https` on every redirect hop, 30 s, 25 MiB, no cookies; a non-HTML answer -(plain text, PDF, image) is stored as received with no derivation. For a paywall, a login or a -script-rendered page, `raw fetch --html incoming/<file>.html --url <url>` derives the same `.md` -from a page the human saved from their own browser, without network access - the tool holds no -credentials. The header's values are YAML-quoted where needed (`charset: "utf-8 (from: header)"`), -so the block parses as the frontmatter its `---` fences suggest. - -`raw/CONTRACT.md` has a new section "Getting a URL in: `raw fetch`" (the bundle rule, the header, -`--html`, and that only a URL the user named is fetched), and says why the MCP server gets no -fetch tool. `wiki-ingest` starts a URL with `raw fetch` and stops at a teaser instead of -ingesting it. - -### Stack-Entwicklung in drei Phasen: `stack-dev` (Design), `stack-build`, `stack-close` (#168) - -Bisher hatte die Stack-Entwicklung zwei Skills für drei Phasen und koppelte jeden Phasenwechsel an -einen Modellwechsel mitten in der Sitzung: `stack-dev` umfasste Design und Bau und bot am Übergang -`/model sonnet` an, `stack-close` bot `/model opus` an und wartete auch noch auf CI. Der Wechsel -fand in der Praxis nie statt (#50), er hätte den Prompt-Cache verworfen, und die Bauphase passt -nicht in Sonnets Kontext (#151 B lief in die Kompaktierung). Ein roter CI-Lauf machte außerdem die -Abschlussphase wieder zur Bauphase. - -Jetzt hat jede Phase ihren Skill, und übergeben wird über einen Zustand im Tracker statt über den -Kontext einer Sitzung: - -- `stack-dev` ist der Design-Skill und bleibt der automatische Einstieg. Er endet an einem Body, - der **ready** ist (`instructions/dev/issue-tracking.md` § Ready to build), und nennt dem - Betreiber `/stack-build #N`. -- `stack-build` (neu) prüft zuerst, ob der Body ready ist, baut, bumpt, zieht die Dokumente nach, - publiziert und wartet auf grünes CI. Den Body pflegt er an drei festen Stellen: bei einer - Abweichung, nach dem Publish, bei grünem CI. -- `stack-close` prüft den Endzustand des Bodys und veraltete `docs/`- und Contract-Prosa, dann - schließt er das Issue. Der Schließkommentar nennt pro Phase Modell, Effort, Sitzungsgrenze und - Kontextüberlauf, dazu `size/`. - -`stack-build` und `stack-close` tragen `disable-model-invocation: true`. In Claude Code kann sie -deshalb nur der Betreiber starten, und jeder Phasenwechsel ist ein echter Halt, an dem er über -Weitermachen, `/clear` oder ein anderes Modell entscheidet. Keiner der drei Skills bietet noch -einen `/model`- oder `/effort`-Wechsel in der Sitzung an. Laut Anthropics Doku zum Prompt-Caching -invalidiert auch eine Effort-Änderung den gecachten Gesprächsverlauf, daher wird auch dort an -einer Übergabe geschnitten statt umgeschaltet. - -Gemeinsames steht je einmal in eigenen Instruktionen: die Mode-Regeln, die Phasentabelle und der -Katalog der Dev-Verfahren in `instructions/dev/stack-mode.md`, die lokalen Checks, `publish` und -das CI-Warten in `instructions/dev/publish-and-ci.md`. Die Notiz, die `publish` nach einem -Stack-Publish druckt, sagt jetzt, dass CI noch kommt und erst danach die ungeprüfte Strecke -beginnt. `docs/model-and-effort-selection.md` beschreibt die Modellwahl pro Sitzung und nennt -Sonnet nicht mehr als Standard für die Bauphase. - -### `publish` keeps a closing trailer block of `--message` last (#149) - -`publish` appended its `Files changed:` list to the end of `--message`. git reads trailers only -from a message's last paragraph, so a message ending in `Co-Authored-By:`/`Claude-Session:` -lines ended up with the file list as its last paragraph, and `git log --format=%(trailers)`, -`git interpret-trailers --parse` and the co-author display on Gitea and GitHub read nothing. -That held for every attributed commit `publish` made. - -The list now goes in front of a closing trailer block. Whether `--message` ends in one is -decided by `git interpret-trailers --parse --no-divider`, not by a pattern of our own, so -continuation lines, `(cherry picked from ...)`, the title rule and the share of trailer lines a -block needs are judged the way git judges them. `--no-divider` matches how `git log` reads a -commit: past a `---` line. The list moves only when git reads exactly the same trailers from the -result as from `--message`. A message without a trailer block is committed byte-identically to -before. Neither gate token covers `--message`, so a `--confirm` or `--confirm-rebase` token -clears exactly what it cleared before. - -Commits already on `main` keep their unreadable trailers: rewriting them would need a force-push. - -### `tools/bugreport`: the collector finds its own Python (#166) - -The bug-report collector is the one part of the stack that has to run when nothing else does, -and it was started as `python3 tools/bugreport.py`. On the Windows target that name is the -Microsoft Store's alias in Git Bash, and in PowerShell `python` can be one too - so the report -failed exactly where it was needed. - -It now has a launcher pair after the pattern of `tools/wikitool`: `tools/bugreport` (sh, for -Linux, macOS and Git Bash) and `tools/bugreport.ps1`, which PowerShell resolves the same string -to first. They look for a Python the way the preflight does - `python3`, `python` on `PATH`; on -Windows `python`, `py -3`, `python3`, with the registry's `PATH` added under PowerShell - and then -try the venv's. Each candidate is probed for 3.8 or newer before it runs anything, and nothing -under `WindowsApps` is ever started. One that is too old, or that the machine refuses (a venv -`python.exe` blocked by Defender), is passed over. Finding none, the launcher exits 1 and says -why, naming any Python it found too old and the direct call with a full path as the way out. -Neither launcher assumes the preflight, `.wikitool-tools.json` or the venv, and the PowerShell -one keeps to what Windows PowerShell 5.1 understands, so a report never fails on the shell's -version first. - -`instructions/bug-report.md`, `INSTALL.md` § Troubleshooting, the stage-2 line the collector -prints and the CI step that runs it in the distribution all use `tools/bugreport`. The shell test -for shipped instructions now refuses a bare `python3`/`python`/`py` call in a command block. -Starting `bugreport.py` directly still works. - -### INSTALL.md held to the installation instructions (#154) - -The installation procedure has one source, the instructions under `instructions/`; `INSTALL.md` -is the human guide beside it, in the instance's language, and since #153 it no longer retells -the steps. One shared file was rejected (D12): an agent reads every sentence as an instruction, -and the two readers need different things. What the two still share is two enumerable lists, -and both are now checked by `docs verify`: - -- **Prerequisites.** `INSTALL.md` carries one generated region per platform value of - `tools/prerequisites.txt` - `<!-- wikitool:prerequisites -->` for every platform, - `<!-- wikitool:prerequisites-windows -->` for Windows only - rendered as label and minimum - version, without the manifest's English reason field. The new `wikitool docs prerequisites - [--apply]` rewrites them; it never places a missing region, since where a list belongs is the - human guide's decision, and reports it instead. A tool added to the manifest fails `docs - verify` until the region is regenerated. -- **Setup questions.** Every place `instructions/setup-instance.md` asks the user something - carries `<!-- setup-question: <key> -->` (eight today: `identity`, `remote`, `kb-language`, - `domain`, `personalization`, `environment`, `telemetry`, `task-tracker`), and the matching - bullet in `INSTALL.md` § "Was der Agent dich fragt" carries the same marker. `docs verify` - compares the two sets in both directions. Markers rather than a frontmatter list: they sit - where the question is asked, visible to whoever adds the next one, and the instruction schema - stays closed. - -The prose that no check reads is session work. `instructions/dev/doc-pull-through.md` gains rows -mapping the installation instructions to `INSTALL.md` and `dev-setup.md` to `DEVELOPMENT.md`, -applied before the publish; `stack-close` step 3 reads the pair again after it and files a -deviation as a follow-up issue. - -### Installation only from a release, into an empty folder; upstream merge/verify and the other install paths removed (#153) - -The install run analysed in Gitea #140 failed on an instruction that contradicted itself, and the -path it took (Weg D, a private clone with this repo as `upstream`) was one of four. All four were -cut down to one (D2, D3): an instance is installed from the latest release into an empty folder, -`dist export` is a build tool, and a clone of this repository is development. None of the removed -paths was in use, so the break has no transition. - -- **Removed.** `wikitool upstream merge` and `upstream verify` (`commands/upstream_cmd.py` and its - tests, the "Private instances" group of command records) and `instructions/private-instance.md`. - Every reference in the shipped tree goes with them; `ownership.is_stack_owned` stays, because - `dist export` uses it. The 3.0.0 migration document dates its reference in words. The - Publish-Remote Gate stays: any checkout with two remotes needs it, and only its rationale lost - the reference to the removed path. `gates.md` loses its section on `upstream merge` bypassing - the Mass-Update Gate. -- **The install path.** `instructions/setup-instance.md` starts at the release: it is attached - to every release as an asset, together with `instructions/preflight.md` (E1), and the - installation sentence in `INSTALL.md` points the agent at both through the API's - `releases/latest` - Gitea 1.26 has no stable "latest" download link. Step 0 downloads the - preflight asset with the shell's own command (E2, so no Mark of the Web) and runs it with the - bypass prefix. The export step is gone, `git init` runs only where there is no repository yet, - and an empty clone keeps its `origin`. The steps are renumbered; references to the - personalization step name it rather than its number. -- **The preflight asset installs in place (E6, changes D40).** It installs into its own folder, - which has to be empty apart from the script and a `.git`, unpacks into a temporary folder - inside it, moves the stack up and removes itself, so the first commit carries the stack and - nothing else. `--into` installs elsewhere under the same rule. A non-empty target is refused - with exit 1. Tests cover both shells, an empty clone, and the self-removal. Under bash the - folder-too-long guidance printed `C:\\Chemenu` with a doubled backslash; it now prints one. -- **`wikitool dist adopt` (E3).** Copies `kb/*/COLLECTION.md.template` and `types/*.template` to - their unsuffixed names, never over an existing file. It replaces the `for … cp` loop in - `setup-instance.md`, the two `cp` lines in `upgrade-instance.md` and the loop in the CI replay. -- **Shell-neutral instructions (E4).** A command block in a shipped instruction is a - `tools/wikitool` or `git` call or the preflight's own call; `instructions/CONTRACT.md` states - it, and `tests/test_instructions_shell.py` holds it, with two exceptions of one line per shell: - the session id (D26) and the preflight download (E2). Migration documents are out of scope. - The same test checks that every PowerShell preflight call in a shipped file carries - `pwsh -NoProfile -ExecutionPolicy Bypass -File`. `session-setup.md` loses the bash-only inline - form and the `$(date +%s)` id; `upgrade-instance.md`, `ingest-large-tree.md` and - `mcp-read-server.md` follow, as does the hint `work new` prints. -- **Session id under Copilot (E5).** Neither Copilot CLI nor Copilot's agent mode in VS Code sets - a session variable (checked against their documentation), so `session.HARNESS_ENV_VARS` is - unchanged. `setup-instance.md` has the agent set `WIKITOOL_SESSION_ID` with the line for its - shell before `doctor`, and `session-setup.md` says why. -- **CI.** The replay builds a release tarball the way `release.yml` does and starts the preflight - asset with `--archive` in an empty folder (D41). It no longer asserts the launcher's exit 42 - before the preflight - the asset runs the tree preflight itself, and the launcher's refusal is - tested in `test_preflight.py`. `release.yml` attaches `setup-instance.md` and `preflight.md` - from the exported tree. -- **Docs.** `INSTALL.md` describes one path: what has to be there first (Windows: PowerShell 7, - `RemoteSigned`, Git for Windows, a folder of at most 95 characters), the sentence for the - agent, the questions it asks, and what to do at every stop of the preflight in both modes. - `DEVELOPMENT.md` gains the development checkout, `dist export` as a build and test tool, and the - private release feed that used to sit in `INSTALL.md`; `instructions/dev/dev-setup.md` is its - agent-side counterpart. `bootstrap.md` is for a further checkout of an existing instance (D11). - `docs/ownership-and-templates.md` records why an instance comes only from a release. - -### Windows-Portabilität: Pfadtrenner, Zeilenenden, Encoding und Locks - -The Python package assumed POSIX in several places that nothing on Linux would ever reveal -(Gitea #152, part of #140). In the run analysed in #140, `instructions verify` failed on all 23 -instructions on Windows. This changeset makes the package behave the same on Windows, and holds -it there with guards that run in the ordinary Linux CI. - -- **Path separators.** Every `str(<path>.relative_to(...))` is now `.as_posix()`, as are the - error messages that printed a relative path. On Windows these strings came out as - `kb\x.md`. They were then compared with POSIX keys or stored. The type-spec - self-reference check in `type_resolver.py` is one of them, and it failed every validation. -- **ripgrep paths.** `rg --json` writes `\` on Windows, and `--path-separator /` does not - reach its JSON output (measured on the target system, T3). `search/ripgrep.py` converts the - separator where it parses a match, and does so only when `os.sep` is `\`. Nothing is lost: - no Windows path component can contain `\`, and since #155 no page title can either. -- **Line endings.** A new `.gitattributes` (`* text=auto eol=lf`) keeps every text file LF in - a checkout with `core.autocrlf=true`. Without it the sh launcher gets CRLF and Git Bash fails - with `env: 'bash\r'`. `raw/` and `incoming/` are `-text`, so a source is stored byte for - byte as it arrived. `dist export` ships the file. The index was LF throughout already, so - renormalizing changes nothing. Every text write now passes `newline="\n"`, so the byte - comparison of published skill copies and the per-file sha256 in `dist upgrade` agree on - Windows too. -- **Decoding.** Every `subprocess` call with `text=True` names `encoding="utf-8"`. Without it, - Windows decodes `git` and `rg` output in the locale's code page (cp1252). -- **wikitool's own output.** `tools/run_wikitool.py`, the file both launchers run, sets - stdout and stderr to UTF-8. Python on Windows writes into a pipe in cp1252. Measured on - the target system, PowerShell decodes the output of a child process with - `[Console]::OutputEncoding`. Under Copilot that is UTF-8, and Git Bash passes bytes through - unchanged. Before this change `doctor` showed `Fu�noten` under both harnesses. A console is - unaffected either way. -- **Hook payloads.** `trace_ingest.py` reads its stdin as UTF-8 bytes. A locale-decoded read - failed outright on Windows when a payload contained a character such as `Ł`, whose UTF-8 - form holds a byte that cp1252 leaves undefined. -- **Locks.** The budget counter and the telemetry writer locked with `fcntl` and silently - skipped the lock where it does not exist. Parallel calls on Windows could then lose a - budget increment. The new module `chemenu/filelock.py` is the only one allowed to import - `fcntl` or `msvcrt`. On Windows it locks one byte far past the file's data with - `msvcrt.locking`, because a Windows lock is mandatory and a lock on the data would block - `trace_ingest.py` from reading a trace. It waits for a contended lock the way `flock` does. - -New tests: `tests/test_portability.py` reads the source of `tools/chemenu` and the scripts -beside it. It fails on a stringified `relative_to`, on a text open/read/write without -`encoding=`, on a text write without `newline=`, on `text=True` without `encoding=`, and on an -`fcntl`/`msvcrt` import outside `filelock.py`. Each detector also gets the defect it exists -for, so a guard that matches nothing cannot pass. The `fcntl` fallback is tested with the -import hidden and a fake `msvcrt`. A real Windows lock is not exercised, because CI runs only -Linux. The UTF-8 output and stdin are tested under `PYTHONIOENCODING=cp1252`, which simulates -the Windows pipe. Further tests feed the T3 JSON line through `search/ripgrep.py` with a -simulated Windows separator and check `.gitattributes` with `git check-attr`. Whether `doctor` -shows `Fußnoten` under Copilot and Claude Code is checked by hand on the target machine. - -### trace-hook.ps1: Copilot hooks no longer open Windows' choose-an-app dialog - -On the Windows target machine, Copilot opened Windows' "choose an app" dialog for `trace-hook` -on hook events (Gitea #164). Some PowerShell had run `./tools/trace-hook ...`, and an -extensionless sh script has no program associated with it. The `bash` field of -`.github/hooks/wiki-trace.json` was not the path in: both Copilot clients take the `powershell` -field on Windows. Copilot CLI, however, also reads `.claude/settings.json`, whose -`UserPromptSubmit` hook has only a `command`, and it runs that `command` under PowerShell on -Windows. - -`tools/trace-hook.ps1` is the PowerShell twin of `tools/trace-hook`, after the -`wikitool`/`wikitool.ps1` pattern: PowerShell on Windows resolves `./tools/trace-hook` to the -`.ps1` first. It does what the sh script does: it runs `trace_ingest.py` with the venv's Python -in either layout, records nothing without a venv, and exits 0 whatever happens. It has no -`#Requires -Version 7`, because VS Code starts hooks under Windows PowerShell 5.1. No hook -command string changed, so Linux, macOS and Claude Code under Git Bash behave exactly as -before. The preflight's and `doctor`'s Mark of the Web check and CI's PSScriptAnalyzer step -already cover every `.ps1` under `tools/`. - -New tests: one guards that every extensionless hook target has a `.ps1` twin. The pwsh tests -cover arguments and stdin reaching the venv Python, silence without a venv, and exit 0 when -the Python fails. Whether the dialog is really gone in both clients is checked by hand on the -target machine. - -### preflight.ps1: the asset-mode error helper is Exit-Asset, so PSScriptAnalyzer passes - -The helper that ends asset mode with exit 1 was called `Stop-Asset`. `Stop` is one of the verbs -PSScriptAnalyzer reads as changing system state, so `PSUseShouldProcessForStateChangingFunctions` -failed the CI `pwsh` job on the previous bump. It is now `Exit-Asset`, beside `Exit-WithGuide`; -behaviour is unchanged. - -### Preflight as a release asset: download, verify and unpack the stack, then run the tree preflight - -Section C of #151, the first install. Until now a new user had to get the stack onto the machine -before any preflight could run - which is exactly the step that fails on a machine without git -or a short enough path. Each release now also attaches `preflight.sh` and `preflight.ps1` as -assets, and a script without `tools/prerequisites.txt` beside it runs in **asset mode**: it -downloads the release tarball and its `.sha256`, stops with exit 1 unless the checksum matches, -reads the folder limit from `tools/prerequisites.txt` inside the archive, and unpacks into -`chemenu/` next to itself (`--into <path>` for another place). It unpacks into a temporary -sibling and renames, requires exactly one top-level folder, and refuses an existing target -without touching it. Then it runs the unpacked tree's own preflight, passing `--set` and the exit -code through, so everything after the unpack is the tree mode that already existed. - -The asset copies are made by `release.yml`: it writes the same release's tarball and checksum -URLs into two placeholder lines of each script, checks that the substitution took, and uploads -the two files under exactly those names. The tree copies keep the placeholders empty; an asset -script with empty placeholders and no `--archive` exits 1 saying it does not come from a release. -`--archive <tarball>` uses a tarball already on disk (its `<tarball>.sha256` must sit beside it) -for a machine that cannot download, and is the entry point the tests use. - -On POSIX asset mode needs `curl`, `tar` and `sha256sum` (or `shasum`) and stops with exit 42 and -the usual guidance block when one is missing; the PowerShell script uses what Windows 10 and 11 -ship (`Invoke-WebRequest`, `Get-FileHash`, `tar.exe`) and stops the same way only when `tar` is -gone. On Windows with long paths off, the 95-character folder limit is judged at the final -target before anything is unpacked, so a too-long `--into` stops with exit 42 and the fix is a -shorter folder. Git Bash gets the target and archive converted with `cygpath -u`, because GNU -tar would read `C:` as a host. - -The tests build a release tarball with a checksum and run both scripts against it, including a -local HTTP server for the PowerShell download, the 95/96 boundary, a failed checksum, two -top-level folders, an existing target, and a test that ties the placeholder lines to the `sed` -expressions in `release.yml`. `tools/CONTRACT.md`, `tools/README.md`, `README.md`, -`instructions/preflight.md` (a new decision point for the asset script), -`instructions/dev/testing-conventions.md` and `INSTALL.md` describe the first-install route. - -### PowerShell 7 preflight and launcher: tools/preflight.ps1, tools/wikitool.ps1, doctor checks for execution policy and Mark of the Web - -The PowerShell half of #151. Harnesses that run in PowerShell 7 on Windows (GitHub Copilot CLI, -for one) resolve `tools/wikitool` to `tools/wikitool.ps1` before the sh launcher, so without it -the call ended silently. `tools/wikitool.ps1` does what the sh launcher does: it stops with -exit 42 until the preflight has written a complete `.wikitool-tools.json`, accepts either venv -layout, and passes the CLI's exit code through. There is deliberately no `.cmd`. - -`tools/preflight.ps1` (`#Requires -Version 7`) answers the same questions as `preflight.sh` -from the same `tools/prerequisites.txt` and writes the same file, byte for byte - CI compares -the two. It is always started as -`pwsh -NoProfile -ExecutionPolicy Bypass -File tools/preflight.ps1`: the bypass holds for that -one process, changes no setting, and lets the script report a Mark of the Web on itself. On -Windows it also reads the machine's `PATH` from the registry, so a session that inherited an -old one still finds a tool installed since, and it never runs or records the Microsoft Store -alias. What only it checks: the effective execution policy (`Restricted` or `AllSigned` is a -stop; when a group policy sets it, the output says that only the administrator can change it -and points at Git Bash) and any `*.ps1` under `tools/` carrying a Mark of the Web from the -internet zone, with the `Unblock-File` line that fixes it. - -`doctor` gains `execution-policy` and `script-marks` (Windows only, `OK` elsewhere). The -launcher's STOP text, `toolpaths.PREFLIGHT`, `doctor`'s fix line, the missing-dependency message -and `dist export`'s summary name the PowerShell call beside the sh one. `bugreport.py` starts -`wikitool.ps1` with the same bypass, so a blocking policy shows up as a `doctor` finding instead -of stopping the report. `instructions/preflight.md` carries both calls, the `--set` form and the -two new decision points; `INSTALL.md` has the Windows prerequisite and troubleshooting for the -policy and the Mark of the Web. - -The tests run against the same stub machine as the sh ones and skip without `pwsh`. CI gets a -`pwsh` job in the new image `chemenu-ci-pwsh` (`.gitea/pwsh-ci/`, built by -`pwsh-ci-image.yml`, monthly and on change): PSScriptAnalyzer over `tools/*.ps1`, both -preflights compared, `tools/wikitool` started from pwsh, and the PowerShell tests. - -### Preflight: prerequisites checked and tool paths recorded before wikitool runs (#151, POSIX half) - -The Windows install that prompted this found Python missing, then `rg`, and the agent worked -around each gap instead of stopping. `tools/preflight.sh` is now the one way a checkout gets -set up. It checks the tools `tools/prerequisites.txt` lists (Python 3.11+, git, ripgrep, and -PowerShell 7 on Windows) and records each one's absolute path in `.wikitool-tools.json`, which -is gitignored and per checkout. It then creates `tools/.venv` from the recorded Python with -`-m venv` and `-m pip`. Anything missing, too old or unusable ends in exit 42 with a numbered -block for the user: what, why, the command that fixes it, what next. `--set <tool>=<path>` -takes a path the user names. It is POSIX sh because it has to run before Python is known to -exist, under dash, bash and Git Bash. On Windows it tries `python`, `py -3`, `python3` in that -order, never runs a Microsoft Store alias, and records `sys.executable`. With long paths off it -refuses an install folder over 95 characters, the other half of the 160-character path budget. - -`tools/wikitool` is POSIX sh now as well. It stops with exit 42 until the preflight has written -a complete file, reads both venv layouts, and starts Python through `tools/run_wikitool.py` -instead of `PYTHONPATH`. Every `git` and `rg` the package starts goes through -`chemenu.toolpaths`, so it comes from the recorded path rather than the session's `PATH`. A -path that has gone is an `ERROR` line naming the preflight, not a traceback. `doctor` gains -`tool-paths` and `install-dir`. The harness hooks start `trace_ingest.py` through -`tools/trace-hook`, under the venv's Python, because the script's `python3` shebang is the -Store alias in Git Bash. The CI workflows, `bootstrap.md`, `setup-instance.md` and -`upgrade-instance.md` run the preflight instead of their own venv steps. The rules an agent -follows around it are in `instructions/preflight.md`. The PowerShell half and the download -mode for a first install follow in the same candidate. - -### raw/CONTRACT.md points at the path budget for a name accepted from incoming/ - -The path budget's close-out review found the raw stage contract silent on it, although `raw -accept` now refuses a name that would put the file over it. `raw/CONTRACT.md` § Getting a file in -says so in one sentence and links the rule in `kb/CONTRACT.md` rather than restating it. - -### Path budget: a file's path stays at 160 characters or fewer so a Windows checkout works without long paths (#163) - -Windows counts 259 characters for a whole path, the install folder included, and the target system -has long paths off. A page title long enough to be a sentence made a checkout fail there, in a -place nobody would look. The path of any file below the instance root now has a budget of 160 -characters, counted in UTF-16 code units the way Windows counts (an emoji takes two); the folder -limit that `doctor` and the install preflight enforce is the other half of the same sum. - -`new` (every root), `rename` (`--to` only, also under `--dry-run`), `move` (a single page, and -`--reconcile`, which skips and names the target as it does for an occupied one) and `raw accept` -(the remedy is renaming the file in `incoming/`; `incoming/` and `raw/` stay unchanged) refuse a -longer path with exit 1 before writing anything. `lint` reports existing files over the budget -under a new advisory finding, Long Paths, for `kb/` and `raw/`; it is not a hard error, so -`--fail-on-error` does not start failing a corpus that predates the rule. The fix for an existing -page is `wikitool rename`, and `--from` is never checked, so renaming away from a long title works. - -The measurement is `titles.path_budget_problem`, a pure function beside the title rules, and -`kb/CONTRACT.md` § Titles are identifiers carries the normative statement. - -### setup-instance step 15 names --no-push for a local-only first publish; gates.md and a docstring follow #159 - -The close-out review of #159 found three places the change had not reached. `setup-instance.md` -step 15 shows the first `publish` without `--no-push` and announces exit 42; a local-only instance -now gets exit 1 there, before the gate, so the step says so and names the flag. `gates.md` said -the Mass-Update Gate counts "working-tree changes before `publish` stages them", which no longer -describes a gate that stages into a scratch index; it now says the uncommitted changes `publish` is -about to stage. The `_reconcile_summary` docstring still named "no remote" as an outcome with no -message. - -### publish: the gate lists the staged state; a missing or unreachable remote stops before the commit (Gitea #159) - -**The Mass-Update Gate counted a path twice.** `collect_changes` read `git status --porcelain`, which -reports the index and the working tree separately. A path staged as deleted that sits in the working -tree again (`git rm -r raw`, then `git restore --source=HEAD -- raw/CONTRACT.md`) appears as `D ` and -`??`; the gate counted both, while `git add -A` cancels them out and the commit held neither. The list -a human approved therefore named a deletion and a new file that were never committed. `collect_changes` -now stages into a scratch copy of the index (`GIT_INDEX_FILE`) and reads `git diff --cached --no-renames` -from it, so the list is the state the commit will hold. The real index and the working tree stay -byte-identical, also when the computation fails; `git add` writes the new blobs into the object store, -where `gc` collects the unreferenced ones. - -- The digest in the `--confirm` token is now the blob id of the staged content instead of a sha256 over - the working-tree file, so the token binds to exactly what is committed. A deletion still has none. -- A rename is listed as its old path deleted plus its new path added, which is what the commit holds and - what the "deletions by name" note has to see. The `renamed` status is gone from the scale line. -- `_numstat`, `_untracked_stat`, `_changed_files`, `parse_porcelain_entries` and `parse_porcelain_z` are - removed; nothing else called them. The "Files changed:" list in the commit message comes from the same - list and is correct for the same reason. -- With `--path`, the list is restricted to that subtree even when more is staged, as the commit is. - -**One message for two states became three.** A failed fetch was reported as "No remote configured, or -origin could not be reached". It is now `no-remote`, `remote-lacks-branch` (the remote answers and has no -such branch yet - the first publish of an instance) or `unreachable`, told apart by `git remote get-url` -and the exit code of `git ls-remote --exit-code`, each with its own message. `sync` exits 0 in all three. - -**`publish` stops before the commit when it cannot publish.** Without `--no-push`, `unreachable` and -`no-remote` end the call with exit 1 at the reconcile - before the gate, `git add` and the commit, and -also on a clean tree, where it used to say "Nothing to commit". It used to commit and fail at the push, -which left a commit that only a hand-made `git push` could send. The messages name `--no-push` as the -way to a local commit; the next `publish` that reaches the remote sends that commit along. `remote-lacks-branch` -is unaffected, so the first publish of an instance still commits and pushes. The Publish-Remote Gate, -which runs first when `.wikitool-remotes.json` exists, is unchanged, and so is the retry after a rejected -push: a remote that has become unreachable by then reports the original push error. - -This is a **breaking** change in the sense of the version model: an offline session, or an instance that -stays local, has to pass `--no-push` on every `publish`. No content changes, so `**Migration:**` stays -"none required". `instructions/publish-cycle.md` has the new decision point, `setup-instance.md` step 4 -and `INSTALL.md` say what a local-only instance now sees without the flag. - -### bug-report instruction: step 1 no longer calls every bundle unpseudonymised - -Step 1 told the agent to announce that the bundle "is not pseudonymised" and then, one paragraph -later, to offer `--pseudonymise`. It now says the bundle holds machine, user and path names and the -git remotes unless it is pseudonymised. - -### Bug-report collector can pseudonymise identities, in two stages (Gitea #158) - -A bundle carries machine, user and path names, and an installation failure usually turns on the -*shape* of those names, not on the names. `tools/bugreport.py --pseudonymise` therefore replaces -each identity by a placeholder of the same shape and leaves the structure alone. It is opt-in, -costs one more step, and the instruction recommends it for every channel except a direct handover -to the maintainer over a secure channel. - -- **Stage 1 is mechanical.** The script reads user, `USERDOMAIN`/`COMPUTERNAME`, hostname, home and - repository path, `git config user.name`/`user.email` and the remote URLs, and replaces each word - of them in every text file, in raw and JSON-escaped form. A placeholder is an HMAC-SHA256 of the - lowercased word under a per-bundle random salt, so it keeps length, digit/ASCII/non-ASCII class - and per-occurrence case, is injective through a counter, and differs between two bundles. Words of - the stack's own vocabulary, top-level domains and the public origin stay readable. Matching is - whole-identity, longest first, on word boundaries. -- **Stage 2 is a model's judgement, applied by the script.** The agent reads the review list plus - `CHRONOLOGY.md` and `MANIFEST.md` in full, and trace and transcripts in full only under 100 KB - per file, and names further people, companies, customers, internal hosts and projects in a - candidate file. `--bundle DIR --candidates FILE` applies them with the same machinery and packs - the zip again; the model replaces nothing itself. It refuses with exit 1 and an unchanged bundle - when the mapping is gone, and can be repeated. -- **Three local files** - the mapping with its salt, the review list and the candidate file - sit - beside the bundle directory, never inside it and never in the zip. All three hold originals. -- **`MANIFEST.md` names three privacy states** (none, stage 1, stage 1 and 2) and lists the - placeholders, never an original. The closing output and the instruction name the residual - uncertainty: stage 2 can miss a name in free text it did not read in full. -- `instructions/bug-report.md`, `reports/CONTRACT.md`, `tools/README.md` and `INSTALL.md` describe - both stages and the three files; CI runs the collector with `--pseudonymise` from the exported - distribution and checks that the mapping is beside, not in, the bundle and that the host name is - gone. - -### reports/CONTRACT.md: only the collector's two counting calls take the bugreport session id - -The contract said all of the collector's `wikitool` calls run under `bugreport-<stamp>`. Only -`instructions verify` and `docs verify` do; `version show`, `doctor` and `budget status` inherit -the caller's id, as `gates.md` and the script already said. `INSTALL.md`'s troubleshooting entry -also no longer claims page titles stay out of the whole bundle: they stay out of what the script -generates, while the trace - included by default - may carry them. - -### bug-report instruction: offer WIKI_TRACE=1 for a reproduction - -A distributed instance records no trace by default, so the default "trace in" often had nothing to -include. `instructions/bug-report.md` now offers, when the failure can be reproduced, to repeat the -failing step in one new shell with `WIKI_TRACE=1` set for that session only - never by changing -the checkout's telemetry configuration. - -### Bug-report collector: tools/bugreport.py and instructions/bug-report.md (Gitea #157) - -A failure on one machine used to reach the maintainer as a description. `tools/bugreport.py` now -collects the first round of answers into one bundle, and it does so when `wikitool` itself does -not start: it uses the standard library only, imports nothing from `chemenu`, and keeps to -Python 3.8 syntax. - -- **Four layers.** The environment (OS, every Python and shell found, harness, `PATH`, git - configuration, venv, line endings, on Windows also long paths, execution policy and - mark-of-the-web), the stack (`VERSION`, the `.wikitool-*.json` files, git status and log, the - shape of `kb/` and `raw/` with path lengths and names that break on Windows), verbatim output of - `version show`, `doctor`, `budget status`, `instructions verify` and `docs verify` plus the - caller's session trace, and the agent's chronology with any transcripts. The result is - `reports/bugreport-<UTC stamp>/` and a zip beside it. -- **Secrets are always removed; page titles are kept out of everything the script generates** - unless `--titles` is given. Environment variable names are all recorded, values only for a - fixed list. The trace, the chronology and the transcripts are marked in the manifest as - possibly containing page content and titles. Nothing is uploaded. -- **Its own session id.** The two counting calls run under `WIKITOOL_SESSION_ID=bugreport-<stamp>`, - so collecting a report neither spends nor is refused by the caller's budget. `gates.md` § - "Taking a new session id" names this as the second permitted case, for that script alone. -- **`instructions/bug-report.md`** (`manual: true`) is the agent's side: what to tell the user, - a fact-only chronology template, and the rule to stop before sending anything. `setup-instance.md` - and `upgrade-instance.md` offer it at a failure that has no obvious cause. -- `reports/CONTRACT.md` names the bundle as a third kind of output, `tools/README.md` explains why the - script sits beside the package, `INSTALL.md` has a troubleshooting entry, and CI runs the script - from the exported distribution. - -### Demo corpus follows the decided project pages: three states, seed with their items (Gitea #156) - -The first build of the live tracker suite shipped two demo project pages whose names came from a -draft seed rather than from the decision the operator had made. This replaces them with the -decided three, one per `state:` the weekly review treats differently: -`Chemenu 8.0.0 - Installation und Windows` (active), `Aufgabenverwaltung mit Tracker-Anbindung` -(completed) and `Reproduktionslauf des Korpus` (dormant). - -- **Seed backup.** Rebuilt by starting Super Productivity on the three projects, filing the - items through wikitool's own writer and keeping the backup the app then wrote itself: eight - open items in the active project, one of them `waiting` with an overdue follow-up, and one - someday item. `MANIFEST.json` names the one edit made after the app wrote it - the REST API - cannot file a backlog item, so that item was moved there in the backup. -- **Recorded answers** re-recorded against 19.1.0 from the new seed. `test_sp_recorded.py` now - checks the demo projects too - counts, the waiting item and its date, the someday item - on - both read paths. -- `review` against the seed reports `waiting_overdue` for the waiting item and stays quiet on the - completed and the dormant project, which is the join the demo exists to show. - -### Live tracker suite: WIKITOOL_TASKS_CONFIG override, real-tracker tests for Super Productivity and CalDAV, nightly workflow and test image (Gitea #156) - -The task-tracker adapters were only ever tested against fakes, which is how #162 stayed hidden. -This adds a suite that runs the documented `task new` / `task list` / `task close` / `review` -workflow against a real tracker, and the machinery to run it on a clock. - -- **`WIKITOOL_TASKS_CONFIG`** names the tracker configuration `task`, `review` and `doctor` - read instead of `.wikitool-tasks.json`, so one checkout can be run against several trackers in - turn. A set variable that names no file is an error naming the path - never "no tracker - configured". Listed in INSTALL.md's variable table and in the affected command records. -- **The `live_tracker` suite** (`tests/test_tracker_live.py`, `tests/tracker_live.py`) writes only - into a project named `Chemenu Live-Test`, prefixes every item with the run id, deletes - nothing, and aborts before the first write when the marker project is missing. It skips - without a tracker; `CHEMENU_LIVE_REQUIRE` turns a missing one into a failure. A tracker of - your own is named by `.wikitool-tasks.d/<name>.json` (gitignored, absolute paths only) and - `CHEMENU_LIVE_PROFILE`. -- **Headless Super Productivity** (`tests/sp_headless.py`) starts the packaged app on a seeded - profile, accepts its startup restore dialog over the DevTools protocol and refuses to start - when something already answers on the fixed API port. -- **Recorded fixtures** (`tests/fixtures/sp/`) hold real answers of v19.1.0 and a backup the app - wrote itself; `test_sp_recorded.py` replays them in the default run, and - `tests/record_sp_fixtures.py` records them again. -- **CI.** `ci.yml` runs the CalDAV half against a Radicale process on every push. - `tracker-live.yml` runs both providers nightly inside the `chemenu-sp-live` image, which - `sp-live-image.yml` builds daily when the update channel has a new Super Productivity version - and monthly regardless. The image follows the channel rather than a pin, because installed - desktop clients update themselves. -- **Docs.** `instructions/dev/tracker-testing.md` has the profile procedure per tracker, when an - agent dispatches the nightly run, what a red night means and how to refresh the fixtures; - `testing-conventions.md` names the live suite as the one deliberate exception to the hermetic - default. `docs verify` gained an ignore canary for `.wikitool-tasks.d/`. -- **Demo corpus.** `kb/gtd/technik/` gains three project pages taken from this repository's own - work, one per `state:` the weekly review treats differently (active, completed, dormant); the - seed backup carries matching projects with their items, so `review` has something to join. The - first build shipped two differently named pages; the entry below corrects them. - -The Docker package `chemenu-sp-live` has to be linked to this repository once, by hand, after the -first image build. - -### Super Productivity API path: unwrap the {ok, data} envelope, exclude the inbox project, ready-aware health (Gitea #162) - -Preparing the live tracker tests (#156) ran the real Super Productivity v19.1.0 headless for the -first time, and four assumptions about its local REST API did not hold. The test fakes had been -written from the same assumptions, so the suite stayed green while `access: "api"` could not read -anything. - -- **The envelope.** The app answers `{"ok": true, "data": ...}` or `{"ok": false, "error": {"code", - "message"}}`; the adapter expected a bare list. Every read - `review`, `task list`, `task new`, - `task close` - failed with "did not return a list of objects". `_ApiClient` now unwraps `data`, - raises with the API's own code and message on `ok: false`, and refuses a body without the - envelope instead of guessing. -- **The inbox.** `GET /projects` and the backup snapshot both list `INBOX_PROJECT`, so `review` - would have reported "Inbox" as a tracker project without a `kb/` page. Both readers and the - writer's name-to-id lookup now drop it. -- **`health()`** requires `data.rendererReady`; an app whose renderer is still loading no longer - counts as healthy, and `doctor` says so. -- **Closing a task** sets `doneOn` and `modified`; the module docstring claimed it does not. The - behaviour is harmless - the store fills them exactly as for a tick in the UI - the reasoning is - corrected. - -The fakes in `test_superproductivity.py`, `test_task_cmd.py` and `test_new_page.py` now answer in the real envelope, -and a test pins that a bare list is rejected. Drop-in in both directions; no `.wikitool-tasks.json` -change. - -### Page titles must form valid, unique file names on Windows and macOS (Gitea #155) - -A title is the wiki's only identifier for a page and becomes the file name one to one, but nothing -checked that the name was usable outside Linux. A corpus written on Linux could not be checked out -on Windows (`CON.md`, `A: B.md`, a trailing dot) or collapsed two pages into one on macOS and -Windows (`Foo.md` and `FOO.md`, or the same accented title in NFC and NFD). The rule is now stated -once, in `kb/CONTRACT.md` § "Titles are identifiers", implemented as pure functions in -`chemenu/titles.py`, and enforced on every platform - a corpus written on Linux is read on the -others. - -A title is refused when it is empty, contains one of `< > : " / \ | ? *` or a control character, -ends with a dot or a space, or starts - before its first dot, ignoring case and trailing spaces - -with a Windows device name (`CON`, `PRN`, `AUX`, `NUL`, `COM0`-`COM9`, `LPT0`-`LPT9`, including the -superscript digits) or with `INDEX` or `COLLECTION`, the two names the stack owns next to a page. -Two titles collide when their NFC-normalized, case-folded forms are equal. The full title, -including a type's `title_prefix`, is what is checked. - -- `new` checks the title for every type and every root, then the collision against the corpus for - `kb/` pages, then that the target file does not exist - all before anything is created, the - tracker project included. This last check also fixes a data-loss bug found on the way: for the - `root: repo` types, `new instruction --name gates` silently overwrote `instructions/gates.md`. - `new` never overwrites an existing file now. -- `rename --to` is checked the same way, also under `--dry-run`, with the page itself excluded so - a case-only rename (`Foo` to `FOO`) still works. `rename --from` is deliberately never checked: - it is how a page that is already invalid gets fixed. -- `move` refuses a target that an existing entry claims under another case or normalization, for - a single move and in `--reconcile` alike. -- `lint` reports existing violations under **Unportable Titles**, with the colliding paths named - and a `wikitool rename` remedy. The finding is a hard error at every `kb_version` and is - deliberately not migration-gated: there is no migration for it, so `kb_version` never advances - on its account, and each affected page is renamed individually. - -The bump is `--major` because a corpus that carries such a title stops passing `lint --fail-on-error` -after the upgrade; a demo/testbed corpus and the shipped instructions are clean. Uncertain and -refused conservatively: whether `COM0`, `LPT0` and the superscript forms are device names on every -Windows version differs, so all of them are refused. - -`tools/CONTRACT.md` is regenerated, and `kb/CONTRACT.md`, `kb/CONVENTIONS.md` and its template, -`instructions/page-lifecycle.md`, `instructions/wiki-lint/SKILL.md` and `README.md` carry the -rule. - -### dist upgrade --latest: one-command update from the release feed (Gitea #161) - -Updating a tarball instance took a manual detour: `version notes`, then fetching the `.tar.gz` and -its `.sha256` from the release page by hand, then `dist upgrade <tarball>`. `dist upgrade --latest` -now does the middle part. It asks the release feed (`update_url` from the stamp, -`$WIKITOOL_UPDATE_URL` and `$WIKITOOL_UPDATE_TOKEN` as before) which release is latest, downloads -the release's archive and checksum into a scratch directory, verifies the archive against the -checksum, and hands over to the existing upgrade path unchanged - classification, -`--keep-local`/`--take-release`, migration report and every refusal are the same code. `<source>` -becomes optional; exactly one of it and `--latest` must be given. - -The order is what matters. The local preconditions run first, then the feed is asked, and the -version it reports is judged **before any download**: already installed is a success no-op, an -older version is a downgrade refusal, a `-beta.N` version needs `--pre`. Only then are the two -assets looked up - by exact name, `chemenu-stack-<version>.tar.gz` and its `.sha256`, in the -`assets` of the release object the version query already returned, using the feed's own -`browser_download_url` values; no URL is composed. A release missing either asset is refused -before the first download with the assets it does have and its release page named. The checksum -is mandatory on this path (a `<source>` archive without a sibling `.sha256` is still only a -WARN), and the archive's own `VERSION` must equal the feed's version, otherwise nothing is -applied. `--dry-run` downloads and verifies too - classification needs the tree - and removes -everything again; the scratch directory goes away on every exit. - -`--expect <version>` (only with `--latest`) closes the gap between reading the notes and applying -the update: the feed only offers its *latest* release, so if a newer one appeared since -`version notes` was read, the run refuses before downloading anything and names both versions. -`instructions/upgrade-instance.md` passes the version from step 2 in the dry run and in the real -run. The comparison is on parsed versions, so a `v` prefix does not matter. - -`$WIKITOOL_UPDATE_TOKEN` is sent to an asset download only when the asset URL has the feed's -scheme, host and port, and as an unredirected header, so a redirect cannot carry it elsewhere. -There is no https enforcement: the checksum comes from the same host as the archive and protects -against transfer errors, not against a compromised feed - `INSTALL.md` says so. - -The asset names are a Python constant (`version.ARCHIVE_NAME`) that a test ties to -`.gitea/workflows/release.yml`, so renaming them in one place fails the suite instead of the next -upgrade. `dist upgrade` is now `network: yes` in its record and in the pinned set in `test_cli.py`, -the `version check` record's "never reached implicitly" note names the one explicit exception, the -`fetch_latest` docstring says "three commands", and `tools/CONTRACT.md` is regenerated. The tests -run against a local `http.server` that logs every request and its `Authorization` header, so the -token rule and the "no asset request before the checks pass" rule are asserted on the wire. The -`upstream merge` pointers in the `dist upgrade` record and failure message stay as they are; they -belong to the separate work on the clone path. - -### new_page/type_resolver comments no longer claim only entities declare a layout: - -Two code comments - `new_page.py`'s module docstring and `TypeResolver.get_layout`'s - still said -subtype-driven placement applied to "currently just entities", the same stale assumption behind -the `new` record's flat `concept`/`source` paths. They now name the four shipped type-specs that -declare a `layout:`. A test docstring that claimed to run `new source`'s `usage` line verbatim -now says what the test does: it passes the fields that line names. Comments only. - -### new: concept/source record notes name their layout-computed subdirectory; source usage names its required fields (Gitea #150) - -`new`'s `cli_contract` record promised `kb/concepts/<Name>.md` and -`kb/sources/Source - <Name>.md` for the `concept` and `source` variants - flat, with no -subdirectory. Both type-specs have carried a `layout:` for a while (`concept_type`/`source_type` -picks the area, same rule an entity or a project already follows), so the actual path is -`kb/concepts/<subdir>/<Name>.md` and `kb/sources/<subdir>/Source - <Name>.md`. Nothing wrote a -page to the wrong place - `new` computes the path itself - but the record is exactly what an -agent reads to find one afterwards, and it was wrong: `instructions/gates.md` named two concept -pages by their old flat path (fixed in the gates.md entry further down) with the record itself as the plausible source of that -assumption. Both notes now name the subdirectory and where it comes from; `comparison`'s note was -checked against `types/comparison.md` and left alone; it genuinely has no `layout:`. - -Found in the same pass: `new source`'s example line omitted `source_type` (required, no default -since #66) and `fidelity`/`authority` (enforced by `new` itself since #67), so copying it verbatim -always failed. It now names all three. - -The new test in `tools/chemenu/tests/test_new_page.py` reads each variant's note out of the -record itself and turns it into the path pattern it promises, then checks what `new` actually -wrote against that pattern - coupled to the note text, not to a path re-typed into the test, which -is what let the existing per-type path tests stay green through this exact drift. A companion test -asserts every `Writes \`kb/...\`` variant has a case, so a future variant without one is caught -here instead of silently going unchecked. `tools/CONTRACT.md` regenerated via -`wikitool docs contract --apply`. - -### cli.py: removed a stale duplicate help-patch line outside the typer._click fallback's try (Gitea #148) - -The `typer._click.core.Command.format_help` patch was applied twice: once inside a -`try/except (ImportError, AttributeError)` meant to let a future typer without `typer._click` -degrade to Click's own plain help instead of crashing every invocation, and once more on the next -module-level line, unconditionally. Whenever the `try` actually failed, that second line referenced -two names the failed import never defined and raised `NameError` at import time - the exact crash -the fallback exists to prevent, on every single `wikitool` call. Typer 0.27.2 still has the module, -so nothing showed it in practice; the line was pure dead weight until the day it wasn't. Removed, -so the patch is applied exactly where the `try` already applies it. - -The new regression test drives `chemenu.cli` in a subprocess with a `builtins.__import__` hook -that raises only for `typer._click.core` imported from `chemenu.cli`/`__main__`, then checks that -`wikitool search -h` still exits 0 with Click's own plain help. Two more direct ways to simulate -the missing module were tried and rejected: `sys.modules['typer._click.core'] = None` also breaks -typer's own lazy import of `typer._click.decorators` inside `get_help_option`, so `-h` fails -regardless of what `cli.py` does; `importlib.reload(chemenu.cli)` reruns the module in the same -`__dict__`, so the names from the first, real import survive and the broken line runs -successfully - a test built that way would stay green against the exact bug it exists to catch. - -### gates.md and a run_budget comment name kb pages by title, not by a path that moved - -`instructions/gates.md` pointed at `kb/concepts/Mass-Update Gate.md` and -`kb/concepts/Iteration and Cost Limits.md`, and a comment in `run_budget.py` at the second - both -pages live under `kb/concepts/workflows/`, where `types/concept.md` places a workflow concept. A -page's title is its only stable identifier, and its directory is whatever the type-spec computes, -so the three references now name the page by title and no longer carry a path. The comment also -says why the page still quotes the old 15-25 band: deliberately, as a sourced claim about the -field, next to the band measured here. Text only. - -### Budget gate and loop-breaker refusals exit without a traceback - -`cli.main()` calls `run_budget.record_and_check()` before Typer ever dispatches to a subcommand, -so a refusal leaves through `_util.fail()`'s `typer.Exit` outside any Click context - nothing -caught it there, so the process exited 1 correctly but printed a Python traceback right after the -`ERROR` line (Gitea #147). That traceback sat exactly in the output AGENTS.md § Gates asks a -session to show a human and stop on; it reads as a crash rather than a gate and invites the retry -the message forbids (Gitea #52). `main()` now catches `typer.Exit` around that one call and exits -with its code directly, and two new subprocess tests - through the real `python -m chemenu.cli` -entry point, not the direct `record_and_check()` call the existing cross-process tests use - -pin that a tripped gate leaves neither stream carrying a traceback, still emits `gate.refused`, -and neither records a `wikitool.call` nor advances the session's counter for the refused call -itself. Both refusal messages, the `budget` command group's own help text, and one code comment -also pointed at "the tooling contract's 'Iteration and Cost Limits'" or "...Tool Error -Contracts" - neither section exists by that name - and now name AGENTS.md § Tool error contract -and `instructions/gates.md` § Iteration Budget Gate and loop-breaker, the sections that do. - -### CalDAV task-tracker provider (Nextcloud Tasks, iOS Reminders); review reports unknown-value findings instead of skipping them - -A second task-tracker adapter, `caldav` (RFC 4791/5545), so an instance is not bound to Super -Productivity - built against Nextcloud Tasks with iOS *Erinnerungen* as the mobile client and -verified against a real account. `TaskReader`/`TaskWriter` are implemented in full without -`tasks/protocol.py` changing at all, proving the provider layer is genuinely exchangeable. The -mapping: a project is a calendar collection whose only supported component is `VTODO`; `WAITING` -is a `waiting` category; `follow_up_at` is `DTSTART`, never `DUE`; a project's `created` falls -back to the earliest item's own `CREATED` (no server in the test account returns -`DAV:creationdate` on a calendar); `create_project` uses a real `MKCALENDAR` call and so never -needs a human-clearance step the way Super Productivity's does; `close_item` changes only -`STATUS`/`COMPLETED`/`PERCENT-COMPLETE`/`LAST-MODIFIED`/`DTSTAMP` on the existing resource and -refuses on an ETag conflict; nothing is ever deleted. Every address used - a list's or an item's -- comes from the server's own `href`, never built from the configured URL and a name, since the -configured URL may be an alias for a different canonical path. - -Alongside it, `wikitool review`'s checks 2 and 3 no longer silently skip a value a provider -cannot supply - a `WAITING` item with no `follow_up_at`, or a tracker project with no -determinable creation date - and instead report it as its own finding -(`waiting_no_follow_up`/`project_age_unknown`), for every provider. `icalendar` is now a required -dependency (`tools/requirements.txt`), imported only when `caldav` is actually configured. - -### version bump no longer points at version release in its output - -`version bump` used to end with "and `version release` once the candidate is ready to ship", and -an agent read that as its own next step and fixed a candidate into a release unasked. The hint -is gone: whether a candidate ships is the user's decision, as `DEVELOPMENT.md` already says for -humans, and `instructions/dev/version-parts.md` step 7 now says so for agents. - -### stack-close: wait for CI through the authenticated Gitea connection, with timings and a give-up point - -Sessions kept improvising how to wait for CI before closing an issue, and one of those -improvisations - an anonymous `curl` loop against the Actions API, which answers `401` even for -this public repo - treated every error as "not finished yet" and never ended. `stack-close` step 2 -now names the one way to do it: read the runs through the authenticated Gitea connection, first -check after about 4 minutes (5 with a release job), then once a minute, and hand back to the user -after 15. Any shell loop that polls instead must end on the first unexpected response. Dev-only; -nothing here ships to an instance. - -### wikitool: one data record per command - `-h`, index and CONTRACT.md render from cli_contract (Gitea #121 Phase 1) - -`--help`/`-h`, the `tools/CONTRACT.md` command reference, and the run-time budget exemption list -used to be four hand-maintained copies of the same facts about a command - a docstring, two -tables in `tools/CONTRACT.md` (§ Commands, § Error contracts), and `run_budget.py`'s own -`SKIP_COMMANDS`/`SKIP_COMMAND_PATHS` sets - and they had already drifted (`xref add`/ -`xref link-source` were missing `--dry-run` from their documented synopsis; the tool error -contract's non-idempotent list disagreed with the per-command retry-policy cells it stood next -to). All 61 commands (60 existing, plus the new `docs contract`) now carry one -`cli_contract.CommandRecord` - name, synopsis, properties (effect, idempotency, atomicity, -budget, network, gates), exit status and retry policy - attached to the command function by a -`@cli_contract.record(...)` decorator in its own module. Three views render from that one -source: `wikitool <cmd> -h` (the full record, plain text), `wikitool -h` (an index line per -command, fixed-width and `grep`-stable), and `tools/CONTRACT.md`'s generated -`<!-- wikitool:commands -->` region (`wikitool docs contract [--apply]`), which replaces the two -old tables. `run_budget.is_exempt` now reads a command's `budget:` property directly instead of -carrying its own list, which is what `wikitool -h | grep non-idempotent`/`budget:exempt` now -answers for `AGENTS.md`'s tool error contract instead of a hand-written enumeration. - -Help itself changed shape: Rich's boxed panels are off (`typer.core.HAS_RICH = False`) for both -`--help` and a usage error, so the output is the same plain, GNU-style text with or without a -TTY - byte-identical, which a new test pins by comparing a real run against one with `isatty` -patched. `-h` is now a recognised alias for `--help` on every command (no command used the flag -for anything else). `wikitool docs verify` grew three checks to hold the new machinery to the -same "checked or absent" rule as everything else it enforces: every registered command has -exactly one `cli_contract` record and appears exactly once in `cli_contract.GROUPS`, every -command's non-hidden flags match its record's SYNOPSIS in both directions, and no command's -rendered help - a docstring above its `\f` marker, or an option's own `help=` text - cites an -issue number (a distributed instance has no tracker to resolve one against, the same reasoning -`check_no_issue_references` already applied to shipped `.md` files). - -Redactional work - examples, an explicit "never do this" section per command, and pulling the -"why" out of a record's NOTES into its own section - is Phase 2 (Gitea #142), deliberately not -part of this change: this pass moved the existing table cells' text into records mechanically, -without editing it beyond the one documented fix (the `--dry-run` synopsis gap above) and the -docstring/option-text rewording needed to stop citing issue numbers in rendered help. - -### wikitool: usage lines name wikitool, and the -h acceptance checks become tests - -Under `tools/wikitool` - which runs `python -m chemenu.cli` - every usage line and "Try ... -h" -hint named `python -m chemenu.cli`, a command nobody should copy; the CLI now passes -`prog_name="wikitool"` explicitly. The properties the previous change promised for help output - -byte-identical with and without a TTY, `-h` and `--help` identical for every command, no Rich -frame characters in help or in a usage error, the top-level index listing at least the seven -non-idempotent commands - had only been checked by hand; they are now pinned in -`test_cli.py`. - -### dist export no longer cuts the dist export record out of the shipped tools/CONTRACT.md - -`dist export` removes every `dist:strip-start`/`dist:strip-end` marker span from the files it -ships, and the `dist export` record's own notes quoted that marker pair literally - so the export -cut a piece out of the middle of the generated command region, and the fresh instance's own -`docs verify` then reported that region stale. CI's fresh-instance replay caught it on the first -push. The old table row carried the same quote and was mangled the same way; nothing compared the -shipped copy against its source until the region became generated. The record now names the -markers without spelling out the pair, and a new test pins that the export plan ships -`tools/CONTRACT.md` byte-identical to the working tree. - -### Command records, Git group: NOTES as bullets, one exit line per cause, examples and prohibitions - -First pass of the editorial rewrite of the command records, starting with `sync` and `publish`. -Text only - no command behaves differently. The record model grew what the rewrite needs: -`notes` takes a tuple of present-tense bullets (a plain string, the old form, still renders as -one paragraph until every group is done), and `Failure` is now one cause with its own reaction -and exit code (`cause`, `reaction`, `code` 0/1/42) instead of one lumped "exit 1 means"/"retry -policy" pair per command. EXIT STATUS lists one line per cause; ON FAILURE repeats the cause -next to its reaction so each line reads on its own; an explicit exit-42 cause replaces the -generic gate line, and a record declaring one without a gate is refused at import. The field -rename is mechanical across all records, so every not-yet-rewritten command's ON FAILURE line now -reads `<old exit-1 text> -> <old retry text>`. - -`sync` and `publish` now carry copyable EXAMPLES (including the re-run after exit 42), a NEVER -section (do not retry or force a failed git step, do not pass an unapproved token, do not edit -`.wikitool-remotes.json` past a refusal), SEE ALSO, and self-contained NOTES in place of "exactly -like `sync`". Sentences that only explained *why* left the records; each was already a comment -at the code that implements it. `publish`'s `atomic` property says "every gate" instead of -"both gates" - it has three, and all run before staging. How a record's prose is written is now -stated once, in the `CommandRecord` docstring, and `tools/README.md` points there. - -### Command records, Catalog and log group: bullets, examples, prohibitions - -`index rebuild`, `log append` and `log status` rewritten the same way; text only. `log append` -now lists its two exit-1 causes separately and states in NEVER what its retry policy said in -prose: check the tail of `kb/log.md` before re-running after an uncertain outcome. -`index rebuild`'s NOTES name the nested-page warning and what `--dry-run` prints, both of which -the command already did. `log status` no longer claims it "reports 0" for a missing log - it -reports that nothing is logged yet, which is what it always printed. - -### Command records, Distribution and versioning group: one line per cause, examples, prohibitions - -`dist export`, `dist upgrade` and the six `version` commands rewritten the same way; text only. -The long single-paragraph records - `dist upgrade` and `version bump` above all - are now one -bullet per behaviour, and each exit-1 cause carries its own reaction, taken from what the -command's own error message already tells the caller (`migrate baseline`, `migrate status`, -commit or stash, `upstream merge`). Three corrections to what the records claimed: `dist upgrade` -listed an equal source version both as a no-op success and as an exit-1 refusal - it is a -success, and EXIT STATUS now says so; `dist export` and `dist upgrade` gained the exit-1 causes -their code already had but their records omitted (a missing licence, an export plan leaking -instance data, a bad archive checksum); and `version regrade`'s NOTES, garbled when they were -moved over mechanically, read as sentences again. Two code comments changed with them: the one -above `dist export`'s `raw/`/`incoming/` anchors had claimed `.gitignore` drops `incoming/`, -which stopped being true when `!/incoming/.gitkeep` was added, and `dist upgrade`'s stamp write -now carries the reasoning for writing the stamp whole after `--keep-local`, which used to live -in the record. - -### Command records, Pages group: one line per cause, examples, prohibitions - -`new`, `task new`/`list`/`close`, `touch`, `rename`, `rm` and `move` rewritten the same way; -text only. `new`'s long per-variant notes moved into NOTES bullets, leaving each variant one line -that says where the page lands, and `new project`'s tracker cases - name taken, read-only access -path, a provider that cannot create projects (exit 42, cleared with `--resume`) - each got their -own exit line and reaction. Every "same posture as `new project`" and "same reasoning as -`task new`" in the three `task` records is replaced by the fact it pointed at. The records also -name failures the code already had and the old text left out: `new`'s missing capture field and -its page-write failure after the tracker project was confirmed, and the partial-write failures of -`rename`, `rm` and `move --reconcile`. The reasoning behind `task new`/`task close` never exiting -42, and behind `task close` taking an id rather than a title, moved into `task_cmd.py`'s module -docstring. - -### Command records, Links and citations group: one line per cause, examples, prohibitions - -`xref add`/`remove`/`link-source`, `links show` and `cite id`/`add`/`sync` rewritten the same -way; text only. `xref add`'s label refusal, previously described only in NOTES, is now an exit -line of its own, and its exit text says which page's type is checked for `related:` (A's - the -only one the code looks at). `xref link-source` states that a missing target is skipped while the -rest are still linked, before the run exits 1, and loses the history of the See Also bullet it -no longer writes. The two `cite` write commands and `cite id` carry AGENTS.md invariant 1's rule -on citation ids as NEVER, where an agent looking up one command finds it. - -### Command records, Finding and checking group: one line per cause, examples, prohibitions - -`lint`, `search` and `review` rewritten the same way; text only. `lint`'s single sentence -listing every check is now one bullet per category, with which findings are hard, advisory or -migration-gated stated per bullet; `review`'s five checks are one bullet each. `search` splits -its four exit-1 causes, and carries AGENTS.md's "do not grep `kb/` yourself" as NEVER next to -the scope rule that justifies it. One claim is deliberately left as it stood: `lint`'s record -still describes the quote limit in blockquoted lines while the code counts quotes - which of -the two is meant is not this change's call to make. - -### Command records, Provenance group: examples, exit lines per cause - -`sources coverage`, `sources trace` and `sources rebuild-index` rewritten the same way; text -only. `sources trace` separates an argument error from the uncovered-file finding it also exits -1 on, since the second is a result to act on rather than an argument to fix; -`sources rebuild-index` names `kb/provenance.md` as generated in NEVER. - -### Command records, Raw material and uploads group: one line per cause, examples, prohibitions - -`raw accept` and the four `upload` commands rewritten the same way; text only. `raw accept`'s -two paragraph-long variant notes became one line each, with the behaviour in NOTES bullets and -its two exit-1 lists split into seven causes. The name-occupied refusal now carries, in its own -reaction and in NEVER, what `raw/CONTRACT.md` already asks of an agent: show the message and -wait, since only the user can tell a new edition from a second source. `upload accept` states its -gate's shape itself instead of pointing at the Mass-Update Gate's. - -### Command records, Workshop runs and session budget group: examples, prohibitions - -`work new`, `work close`, `budget status` and `budget reset` rewritten the same way; text only. -`budget reset` now carries AGENTS.md invariant 6's rule as NEVER - it is never run on an agent's -own initiative to get past a budget refusal - and `work close` states the caller's side of its -`--yes`: the run's conclusions are in `kb/` first. - -### Command records, Types, instructions and docs group: one line per cause, examples, prohibitions - -`types list`/`describe`, `instructions sync`/`verify`/`list` and `docs verify`/`toc`/`contract` -rewritten the same way; text only. `docs verify`'s single sentence naming some twenty checks is -now grouped by what it checks (commands, collections, types, ignore canaries, shipped issue -references, tables of contents, links), with six exit-1 causes and a reaction each. `docs toc`'s -"never fails on content" had been carried over as an exit-1 cause, so the index listed it as -`exit:0,1`; it is now a success line and the index says `exit:0`, which is what the command has -always done. The reasoning `docs toc`'s record carried about its scope already lived in -`toc.py`'s module docstring and now lives only there. - -### Command records, Telemetry group: examples, the missing --fail-on-error exit line - -`eval sessions` and `eval score` rewritten the same way; text only. `eval score --fail-on-error` -has always exited 1 on a failed scorecard, but its record only listed the missing-trace case; it -now names both. - -### Command records, Content migrations group: one line per cause, examples, prohibitions - -The five `migrate` commands rewritten the same way; text only. `migrate done` and -`migrate baseline` state as NEVER what their prose implied - never force the chain's order, -never advance the version with `baseline --force` or by editing `.wikitool-kb.json` - and -`migrate verify` separates a bad `--from` revision from the findings it exits 1 on. The reasoning -behind counting marker pairs rather than comparing their names, and behind an offered migration -ignoring the chain, moved into the `verify` and `done` docstrings. - -### Command records, Private instances group: one line per cause, examples, prohibitions - -`upstream merge` and `upstream verify` rewritten the same way; text only. `upstream merge`'s -single paragraph is now one bullet per step of the merge, and its exit-1 causes - including the -fetch failure, a failing git step inside the open merge, and the post-commit leak, which its old -record mentioned only in passing - each carry their own reaction. - -### Command records, Instance health group: one bullet per check, examples - -`doctor` rewritten the same way; text only. Its one-sentence inventory of every check is now one -bullet per area, each stating which outcome is `OK`, `WARN` or `FAIL`. One stale reason was -dropped rather than moved: the record justified the conventions `FAIL` by `xref`/`cite` writing -out of the section headings, while the check's own docstring calls those headings cosmetic and -grounds the `FAIL` in the file binding every page. - -### Command records: NOTES is always a tuple of bullets; every record's examples are tested - -With every group rewritten, `CommandRecord.notes` no longer accepts the single-paragraph string -it carried over from the first pass; a record that passes one is refused at import. Two tests -over the real registry hold what the rewrite established: every command has at least one -example, and every command with a gate shows how its clearance is passed back in (`--confirm`, -`--confirm-rebase` or `--resume`). - -### fail() prints the command's ON FAILURE lines on stderr - -`_util.fail()` used to print only its `ERROR` line; the reaction a caller needs the moment a -command declines lived one lookup away, in `wikitool <cmd> -h`'s ON FAILURE section - exactly -the lookup AGENTS.md's own tool error contract already warned is the one most likely to be -skipped in the heat of a failure. `fail()` now prints that section's exit-1 causes (each with a -reaction) right after the `ERROR` line, on stderr and as plain text rather than through Rich - a -reaction can carry a literal `[--flag]`, which Rich would otherwise read as markup. A record -with no exit-1 cause of its own falls back to a bare `see: wikitool <cmd> -h` pointer; -`docs contract` is the one real command that hits it today. Nothing about stdout changes: a -command's output on success, or up to and including its `ERROR` line on failure, is -byte-identical to before. `cli.py`'s and `_util.py`'s lookup of the running command's -`cli_contract` path is now one shared function, `cli_contract.path_of`, in place of a private -copy that used to live only in `cli.py`. - -### network: property defined; sync, publish and upstream merge marked networked - -The `network:` property had no written meaning, and the records disagreed with the code and with -each other. `sync`, `publish` and `upstream merge` talk to a git remote (`fetch`, `ls-remote`, -`push`) but said `network: no`, while `version check`'s record called itself one of only two -networked commands - next to `review`, `task new`/`list`/`close` and `doctor`, all of which -already said `yes`. `cli_contract.Network` now carries the meaning: `yes` when at least one path -through the command can reach an endpoint outside the checkout, by an HTTP call of `wikitool`'s -own or by a git operation against a remote; what is possible counts, not what is usual, so -`--offline`, `--no-fetch` or a local-path remote does not turn it back to `no`, and a git call -that only reads refs already on disk is not network access. `sync`, `publish` and `upstream -merge` now say `yes`; `upstream verify` only reads fetched refs and stays `no`. The count claim -is gone from `version check`'s record, from two docstrings in `version_cmd.py`, and from -`version.fetch_latest`, which claimed to be the only place talking to a remote host although the -CalDAV and Super Productivity providers do too. A test over the real registry pins the set of -`network: yes` commands, so a change to it is an edit to that test rather than silent drift. - -### log append: unreadable --body-file is an ERROR line, not a traceback - -`log append` read `--body-file` with an unguarded `Path.read_text()`. A missing file, a -directory or a file that is not valid UTF-8 ended the call in a Python traceback, while the -command's record promised an `ERROR` line with "fix the path and retry once" - so an agent was -told to treat the same failure as case 2 of the tool error contract (validation error, retry -once) by the record and as case 4 (unexpected error, do not retry a non-idempotent command) by -the output. The read is now guarded against `OSError` and `UnicodeDecodeError` and leaves -through `fail()`, which also gives the budget slot back, before `kb/log.md` is opened: the log -stays byte-identical in every failure case, and a parametrised test pins all three. The -record's cause now names the cases ("missing, not a readable file, or not valid UTF-8"). - -### Command records: three more mismatches from #142 aligned to code - -Three more text-vs-code disagreements from #142's collection issue (Gitea #146), plus a -correction of one of #142's own text fixes - all decided in the code's favour and fixed in the -text, no behaviour change. `lint`'s quote-limit cap counts blockquotes, not lines, since #22 -changed the unit; `kb/CONTRACT.md`'s Quotation cap section had kept the old wording and now -names the same unit, with code masked out first. `docs contract`'s record carried no exit-1 -cause at all although `contract_command` fails when `tools/CONTRACT.md` is missing, so it now -names that cause; the merged-stream test that used to exercise the `see:` fallback through this -very gap now pins the command's own ON FAILURE line literally, and the fallback itself stays -covered by its own fixture-based test. `xref add`'s `atomic` property still described a two-write -shape, though the code and the record's own NOTES say an edge is written into A only and B is -never touched; it now says so. `publish`'s `atomic` property, tightened earlier in this -candidate to "every gate runs before staging", still missed the one case that breaks it: on the -single retry of a rejected push, the rebase-review gate can exit 42 after the local commit -exists - nothing is pushed, and the `--confirm-rebase` re-run pushes that commit. - -The five other text fixes #142 made in the code's direction were checked against the code and -stand as they are. - -### docs contract: the merged-stream test pins its own ON FAILURE line - -The previous change's test compared `docs contract`'s output only against -`render_failure_hint()` of the same record - which would still pass if the record lost its -exit-1 cause again, because both sides would then fall back to the same `see:` line. It now also -pins the `ON FAILURE (wikitool docs contract -h):` header and the missing-file cause literally; -removing the cause from the record fails it. - ---- +Die größeren Änderungen folgen je mit einem Absatz. Alle übrigen stehen als Stichpunkt in der Liste +oben; die ausführliche Begründung steht im jeweils genannten Gitea-Issue. + +### Installation nur aus einem Release, in einen leeren Ordner (#153, #154) + +**Breaking.** Von vier Installationswegen bleibt einer: Eine Instanz entsteht aus dem neuesten +Release in einem leeren Ordner (`instructions/setup-instance.md`). `dist export` ist ein +Build-Werkzeug, ein Klon dieses Repositorys ist Entwicklung. `wikitool upstream merge`, +`upstream verify` und `instructions/private-instance.md` entfallen; das Publish-Remote-Gate bleibt. +`INSTALL.md` erzählt die Schritte nicht mehr nach, und `docs verify` hält seine +Voraussetzungsliste und die Setup-Fragen mit den Instruktionen im Gleichschritt. Eine Instanz aus +einem der entfallenen Wege wird einmal aus einem Release neu aufgesetzt und übernimmt `kb/`, +`raw/` und die persönlichen Dateien. + +### Preflight: Voraussetzungen prüfen, Werkzeugpfade festhalten, anhalten statt ausweichen (#151) + +**Breaking.** `tools/preflight.sh` (POSIX sh) und `tools/preflight.ps1` (PowerShell 7) prüfen +Python 3.11+, git und ripgrep, halten deren absolute Pfade in `.wikitool-tools.json` fest und legen +`tools/.venv` an. Fehlt etwas, endet der Lauf mit Exit 42 und einem nummerierten Block für den +Menschen: was fehlt, warum, welcher Befehl es behebt. Bis der Preflight bestanden ist, verweigert +`tools/wikitool` jeden Aufruf mit Exit 42 - ein Agent meldet eine Lücke, statt sie zu umgehen. +`tools/wikitool.ps1` ist der Launcher für Harnesses unter PowerShell 7, `doctor` prüft Execution +Policy und Mark of the Web. Jedes Release trägt beide Skripte zusätzlich als Asset: Ohne Baum +daneben laden sie Tarball und Prüfsumme, prüfen, entpacken und starten dann den Preflight des +Baums. + +### Windows-Portabilität: Pfadtrenner, Zeilenenden, Encoding und Locks (#152, #164) + +Das Python-Paket nahm an mehreren Stellen POSIX an; unter Windows scheiterte etwa +`instructions verify` an allen Instruktionen. Relative Pfade werden jetzt als POSIX-Strings +verglichen und gespeichert, `rg`-Pfade normalisiert, `.gitattributes` hält Textdateien auf LF +(`raw/` und `incoming/` bleiben byte-genau), jeder `subprocess`-Aufruf und die Ausgabe von +`wikitool` selbst laufen in UTF-8, und die Sperren funktionieren auch unter Windows. Guards in der +normalen Linux-CI halten das fest. Copilot öffnet unter Windows für `trace-hook` keinen +„App auswählen"-Dialog mehr. + +### Seitentitel als gültige, eindeutige Dateinamen auf Windows und macOS (#155) + +**Breaking.** Ein Titel wird eins zu eins zum Dateinamen, aber nichts prüfte, ob der Name außerhalb +von Linux taugt (`CON.md`, `A: B.md`, ein Punkt am Ende; `Foo.md` neben `FOO.md`, NFC neben NFD). +Die Regel steht jetzt einmal in `kb/CONTRACT.md` § "Titles are identifiers" und gilt auf jeder +Plattform: `new` und `rename` verweigern verbotene Zeichen, reservierte Namen (auch `INDEX` und +`COLLECTION`), einen Punkt oder ein Leerzeichen am Ende und Kollisionen über +Groß-/Kleinschreibung oder Unicode-Normalisierung. `lint` meldet bestehende Verstöße als harte +Fehler; Abhilfe ist `tools/wikitool rename`. + +### Pfadbudget von 160 Zeichen (#163) + +**Breaking.** Windows zählt 259 Zeichen für den ganzen Pfad samt Installationsordner, und auf dem +Zielsystem sind lange Pfade aus. Unterhalb der Instanzwurzel hat jeder Pfad jetzt ein Budget von +160 Zeichen in UTF-16-Codeeinheiten; die andere Hälfte der Summe ist das Ordnerlimit, das `doctor` +und der Preflight prüfen. `new`, `rename`, `move` und `raw accept` verweigern ein längeres Ziel, +`lint` meldet bestehende Dateien als *Long Paths* (advisory). + +### incoming/ als Warteschlange: raw pending und Ordner-Accept + +**Breaking.** Eine Datei in einem Unterordner von `incoming/` nehmen `raw accept` (auch mit +`--replaces`) und `raw fetch --html` nicht mehr an. Ein Unterordner ist jetzt eine Quelle: +`raw accept incoming/<ordner>` verschiebt ihn als Ganzes mit seinen relativen Pfaden nach +`raw/<JJJJ>/<MM>/<ordner>/`. `raw pending` nennt den ältesten wartenden Eintrag, und ein +`wiki-ingest` ohne Angabe nimmt genau diesen. Wer noch nach `incoming/<typ>/` schreibt, legt +Dateien direkt in `incoming/` ab. + +### publish und sync: Gate auf dem gestagten Stand, Remote vor dem Commit, generierte Dateien mechanisch zusammengeführt (#159, #180, #182, #149) + +**Breaking.** `publish` ohne `--no-push` bricht mit Exit 1 vor dem Commit ab, wenn der Remote fehlt +oder nicht erreichbar ist, statt lokal zu committen und am Push zu scheitern; eine +Offline-Sitzung oder eine rein lokale Instanz übergibt `--no-push`. Das Mass-Update-Gate listet +genau den Stand, den der Commit enthalten wird, statt Pfade doppelt zu zählen. `reconcile` hinter +`sync` und `publish` wertet `kb/index.md`, `kb/log.md`, `kb/provenance.md` und die +`INDEX.md`-Dateien nicht mehr als Überschneidung, führt sie mechanisch zusammen und trägt nicht +überlappende ungespeicherte Arbeit per Autostash durch einen Rebase; vorher sichert es den +Arbeitsbaum unter `refs/wikitool/reconcile-backup`. `publish --path` nimmt die generierten Dateien +außerhalb des Pfads mit, und ein Trailer-Block am Ende von `--message` bleibt dessen letzter +Absatz, sodass git `Co-Authored-By` wieder liest. + +### dist upgrade --latest: Aktualisierung aus dem Release-Feed in einem Befehl (#161) + +`dist upgrade --latest` fragt den Release-Feed nach dem neuesten Release und beurteilt die Version +vor jedem Download: bereits installiert ist ein Erfolg ohne Wirkung, eine ältere eine +Verweigerung, eine Beta braucht `--pre`. Dann lädt es Archiv und Prüfsumme, prüft sie und übergibt +an den unveränderten Upgrade-Pfad. `dist upgrade` löscht außerdem, was ein Release nicht mehr +ausliefert - etwa die Testsuite, die jetzt im Origin-Repository bleibt, weil sie dessen eigene +Type-Specs und Konventionen testet. + +### Ein Datensatz pro Kommando (#121) + +Hilfetext, Kommandoreferenz in `tools/CONTRACT.md` und Budget-Ausnahmen waren vier handgepflegte +Kopien derselben Fakten und schon auseinandergelaufen. Jedes Kommando trägt jetzt einen +`cli_contract`-Datensatz - Synopsis, Eigenschaften (Wirkung, Idempotenz, Atomarität, Budget, +Netzwerk, Gates), Exit-Status je Ursache mit Reaktion, Beispiele und Verbote -, aus dem +`wikitool <cmd> -h`, der Index in `wikitool -h` und `tools/CONTRACT.md` erzeugt werden. Alle +Datensätze wurden dabei gruppenweise überarbeitet und gegen den Code abgeglichen. Scheitert ein +Kommando, druckt `fail()` die passenden ON-FAILURE-Zeilen gleich nach der `ERROR`-Zeile auf +stderr; Budget-Gate und Loop-Breaker verweigern ohne Traceback. + +### CalDAV-Tracker und Live-Tests gegen echte Tracker (#156, #162) + +Ein zweiter Task-Tracker-Adapter, `caldav` (Nextcloud Tasks, iOS Erinnerungen), implementiert +`TaskReader`/`TaskWriter` vollständig, ohne dass sich das Provider-Protokoll ändert; `review` +meldet unbekannte Werte als Befund, statt sie zu überspringen. Der erste Lauf gegen ein echtes +Super Productivity zeigte falsche Annahmen über dessen REST-API (`{ok, data}`-Hülle, +Inbox-Projekt, Health-Check), die auch die Test-Fakes trugen - behoben. Die neue +`live_tracker`-Suite fährt `task new`/`list`/`close` und `review` gegen echte Tracker, nächtlich und +mit eigenem Test-Image; `WIKITOOL_TASKS_CONFIG` wählt die Tracker-Konfiguration. + +### Bugreport-Sammler (#157, #158, #166) + +`tools/bugreport.py` sammelt die erste Runde Antworten zu einem Fehler in ein Bündel, auch wenn +`wikitool` selbst nicht startet: nur Standardbibliothek, Python-3.8-Syntax, und ein eigener +Starter findet sein Python und überspringt die Store-Aliase unter Windows. `--pseudonymise` +ersetzt Maschinen-, Benutzer- und Pfadnamen durch Platzhalter gleicher Form. Die Prozedur für den +Agenten steht in `instructions/bug-report.md`. + +### raw fetch: eine URL geregelt nach incoming/ (#120) + +Bisher baute jede Sitzung ihre eigene `curl`-Kette, und zwei Sitzungen machten aus demselben +Artikel zwei verschiedene `raw/`-Dateien. `raw fetch <url>` schreibt das HTML wie empfangen und +eine daraus abgeleitete `.md` mit festem Kopf (URL, Abrufzeit, Status, Zeichensatz, Titel) nach +`incoming/`; `raw accept` übernimmt beide als ein Bündel. + +### raw capture, raw status, export guidelines: Dokumentation aus Git-Repositories und zurück + +`raw capture <repo-url> --ref <regel> --path <glob>...` holt die gewählten Dateien eines Commits +byte-genau als Bündel nach `incoming/<name>/`, mit einem `_capture.json`, das Repository, +Ref-Regel und Commit festhält. `raw status` meldet, welche erfassten Bündel sich im Repository +geändert haben, `raw capture --update` holt den neuen Stand, und `raw accept --replaces-bundle` +ersetzt ein Bündel als Ganzes; `wiki-ingest` hat dafür einen eigenen Einstieg. Die Gegenrichtung +ist `export guidelines`: Es erzeugt aus ausgewählten Seiten (Standard `--tag guideline`) ein +`GUIDELINES.md`, und `--push` schreibt es in jedes erfasste Repository, das sich mit einer solchen +Datei dafür entschieden hat - hinter dem neuen Guideline Push Gate, dem fünften Gate. + +### Seitenvorlagen pro Subtyp, Organisationsseiten und Beteiligungs-Label (#117, #172) + +`wikitool new` nimmt für einen Subtyp die Vorlage `types/<typ>.<wert>.md`, wenn es sie gibt, statt +eines Templates für alle Werte. Personen, von denen eine Quelle nur Namen und Rolle hergibt, +stehen als Abschnitt auf der Seite ihrer Organisation (neuer `entity_type: organization`) und +steigen erst mit Material zur eigenen Seite auf. Der Link-Katalog bekommt `involves` und die +RACI-Label `staffed-by`, `owned-by`, `consults` und `informs`, geschrieben auf der Projektseite. + +### lint: neue Befunde (#94, #172) + +`lint` meldet *Unfilled Template Sections* - Abschnitte, die nur aus den `TODO`-Platzhaltern +ihrer Vorlage bestehen -, *Broken Anchors* für ein `[[Seite#Abschnitt]]` ohne diesen Abschnitt und +Wikilinks, die über einen Zeilenumbruch laufen; `rename` und `rm` erkennen solche Links jetzt. +Die ersten beiden sind advisory, damit der Lint einer bestehenden Instanz nach dem Upgrade nicht +rot wird. ## 7.0.0 - 2026-09-22 - Task-Tracker-Anbindung: Vorhaben als Seitenart, Verpflichtungsschicht, Weekly Review als Read-Time-Join diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md index 9992795..5f78467 100644 --- a/DEVELOPMENT.md +++ b/DEVELOPMENT.md @@ -135,6 +135,13 @@ eine Sitzung ihn tatsächlich durchläuft: `setup-instance.md` und `preflight.md`, auf die der Installationssatz in `INSTALL.md` zeigt. **CI setzt den Tag, nie eine Sitzung** - das hält Invariante 5 intakt. + Die Release-Notiz ist der `CHANGES.md`-Eintrag; Gitea speichert auf MySQL höchstens 65535 + Bytes, und der Job verweigert ab 60000, bevor er einen Tag anlegt. Scheitert der Job, nachdem + `VERSION` schon auf `main` steht, hilft weder ein Push (die Version steigt nicht noch einmal) + noch ein Re-run (er nimmt die Workflow-Datei des gescheiterten Commits): Ursache beheben, + publishen und `release.yml` per `workflow_dispatch` auf `main` starten. Die Prüfung auf ein + schon vorhandenes Release verhindert ein zweites. + Die drei Verify-Befehle stehen oben in Schritt 3; was jeder von ihnen prüft, steht in [tools/CONTRACT.md](tools/CONTRACT.md) und wird dort von `docs verify` gegen die tatsächliche CLI gehalten. Hier steht es bewusst **nicht** noch einmal: eine zweite Beschreibung derselben