torben/chemenu
1
1 Fork 0
Code Issues 40 Pull Requests Actions Packages Releases 41 Wiki Activity

Mirror docs at v9.0.0

Wiki Mirror committed 2026-10-09 15:40:02 +00:00
1 parent bc952b31bd
commit 624366529f
49 files changed
+7504 -1040
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

Dev-Development.md
+292
@@ -0,0 +1,292 @@
*Generated from [`docs/dev/development.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/docs/dev/development.md) at v9.0.0 – edit the repository, not this wiki.*
# Developing this stack
This document is for people working on `tools/wikitool`, the type schema or the
instruction/skill layer itself - not for the operators of an instance. For the other side
(installing, upgrading and running an instance) see [INSTALL.md](Install).
**This file is not shipped.** It lives under `docs/dev/`, the human counterpart of
`instructions/dev/`, and `tools/wikitool dist export` excludes both directories wholesale: a
distributed instance has no release workflow, no CI and no issue board, so it needs no guide to
them. The reason is a comment at `dist_cmd.DOCS_DEV_ONLY`, so that a later session does not
"repair" the apparent gap. And because it is not shipped, this file may link into
`instructions/dev/` - unlike a shipped document such as `README.md`, which
`instructions verify` checks on exactly that point.
<!-- wikitool:toc -->
## Contents
- [Development environment](#development-environment)
- [`dist export` as a build and test tool](#dist-export-as-a-build-and-test-tool)
- [A release feed in a non-public repository](#a-release-feed-in-a-non-public-repository)
- [The release workflow](#the-release-workflow)
- [The CI half](#the-ci-half)
- [The Gitea wiki](#the-gitea-wiki)
- [How much of the stack the suite reaches](#how-much-of-the-stack-the-suite-reaches)
- [Stack development as a session type of its own](#stack-development-as-a-session-type-of-its-own)
<!-- /wikitool:toc -->
## Development environment
Work on the stack happens in a clone of this repository. Such a clone is **not an instance** and
never becomes one. Instances come only from releases - see [INSTALL.md](Install).
```bash
git clone https://gitea.nehmer.net/torben/chemenu.git
cd chemenu
```
Then have the agent run `instructions/bootstrap.md`: the preflight, then
`tools/wikitool instructions sync`. The agent's side of it - what differs here from an instance,
and how `dist export` runs as a test tool - is
[instructions/dev/dev-setup.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/instructions/dev/dev-setup.md).
Real documents for `raw derive` live in [testfiles/](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/testfiles/README.md) and are
committed, so CI derives them too. The repository is public: that README says what may go in.
What a clone has and an instance does not:
- **The demo corpus.** Some 170 pages documenting the stack itself: gates, lint, versioning,
search, the wiki pattern. It is a testbed and a walkable example, not a productive knowledge
base. What may be changed in it is governed by
[instructions/dev/corpus-policy.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/instructions/dev/corpus-policy.md).
- **A demo persona** in `USER.md`/`SOUL.md`. `doctor` reports both as filled in. They describe
the demo setup, not you.
- **Telemetry on.** Without `.wikitool-release.json` the clone is the measuring station the stack
evaluates itself with
([telemetry-and-evaluation.md](Explanation-Telemetry-and-evaluation#whether-it-runs-at-all)).
- **`instructions/dev/`, `docs/dev/`, `commonplace/` and `.gitea/`.** `dist export` ships none
of them.
`ENVIRONMENT.md` is missing after every clone, because the file is gitignored: it describes a
checkout, not the repository. Creating it (template `ENVIRONMENT.md.template`, step 5 in
`instructions/bootstrap.md`) spares every stack session the questions about harness, `gitea-mcp`
and remote.
### `dist export` as a build and test tool
`tools/wikitool dist export <empty directory>` writes exactly the tree a release ships: the
machinery without wiki content, without git history, without `instructions/dev/` or `docs/dev/`.
That is how you check before a release what would ship (`--dry-run` only lists it). And it is
how you replay the installation path without waiting for a release: build a tarball from it the
way `.gitea/workflows/release.yml` does, then start the preflight script in an empty directory
with `--archive <tarball>`. The CI step "The distribution works as a fresh instance" in
`.gitea/workflows/ci.yml` does exactly that.
For a real instance such an export is not a way in. It lacks the release origin in its stamp,
and `dist upgrade --latest` later compares against a state that never existed as a release.
### A release feed in a non-public repository
Whoever runs the stack from a repository of their own that is not public, and upgrades instances
from there, points `WIKITOOL_UPDATE_URL` at its feed. There is one quirk: Gitea answers an
anonymous caller with the same `404` for an invisible repository as for one that does not exist
at all. A missing release and missing access then look identical - and "no update found" would
simply be wrong. A Gitea token with read access in `WIKITOOL_UPDATE_TOKEN` fixes that. It is sent
only to downloads on the same host as the feed.
## The release workflow
Between two releases the stack carries **one** running version candidate instead of a new
number per bump. The full model - where the state lives, how escalation works, why a number is
only consumed by a release - is in
[instructions/dev/version-parts.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/instructions/dev/version-parts.md) and
[docs/explanation/version-model.md](Explanation-Version-model). Here only the sequence, in
the order a session actually goes through it:
1. **A bump opens or escalates the candidate, graded with `--impact`.**
```bash
tools/wikitool version bump --minor --title "What changed" --impact medium
```
Writes `VERSION` as `X.Y.Z-beta.N` and opens (or updates) the matching `CHANGES.md` entry.
Several bumps for the same change are normal - each one updates the same entry instead of
opening a new one. `--impact high|medium|low` (default `medium`) groups the entry;
`tools/wikitool version regrade` corrects a grade later, when the candidate's overall shape
changes the read on an earlier bump.
2. **The entry gets its prose - only where it is needed.** `bump` writes the skeleton (heading,
date, author, the machine-managed, grouped list of bump titles, and the breaking and migration
lines where they apply). A smaller change needs nothing beyond its bump title; a larger one
gets a paragraph for its topic, and a short summary is added before the release. The form, and
the size budget `docs verify` and `version release` check, are in
[instructions/dev/version-parts.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/instructions/dev/version-parts.md) § The candidate
model.
3. **Run the checks before anything is published:**
```bash
cd tools && .venv/bin/python -m pytest -q
tools/wikitool docs verify
tools/wikitool instructions verify
```
4. **`version release` fixes the candidate** once it is meant to ship:
```bash
tools/wikitool version release --title "Summarising title"
```
Strips the `-beta.N` suffix from `VERSION` and closes the changelog entry. `--title` is
optional - without it the last bump's title stays; with it, a candidate that collected several
bump titles gets a summarising heading. Refused when the candidate collected two or more bumps
and the summary from step 2 is still missing - a candidate with exactly one bump is exempt -
and likewise when the entry is over its size budget. Commits and pushes nothing (invariant 5
in [AGENTS.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/AGENTS.md)).
5. **Publish moves `VERSION` on `main`.**
```bash
tools/wikitool publish --message "..."
```
The Mass-Update Gate and the Publish-Remote Gate apply as on every other publish - see
[instructions/gates.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/instructions/gates.md).
6. **CI does the rest.** `.gitea/workflows/release.yml` reacts to every push that moves
`VERSION`: a `VERSION` with a suffix (a candidate) makes the job skip cleanly before it even
queries the releases API - betas are never published. A suffix-free `VERSION` builds the
distribution (`dist export`), creates the tag and the release, and uploads the tarball and its
checksum, along with the two preflight scripts and the instructions `setup-instance.md` and
`preflight.md` that the installation sentence in `INSTALL.md` points at. **CI sets the tag,
never a session** - that keeps invariant 5 intact.
The release note is the `CHANGES.md` entry; Gitea on MySQL stores at most 65535 bytes, and
the job refuses from 60000 on, before it creates a tag. That is only the last net: `docs
verify` already holds the topmost entry to at most 32000 bytes while it is a candidate (step
2). If the job fails after `VERSION` is already on `main`, neither a push (the version does
not rise again) nor a re-run (it takes the workflow file of the failed commit) helps: fix the
cause, publish, and start `release.yml` on `main` via `workflow_dispatch`. The check for an
existing release prevents a second one.
A second job, `wiki-mirror`, runs after a suffix-free release and mirrors the docs of the
release tree into the Gitea wiki ([The Gitea wiki](#the-gitea-wiki) below). It is a job of its
own, so a failing mirror leaves the release and its tag standing and marks only itself red.
The three check commands are in step 3 above; what each of them checks is in
[tools/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/tools/CONTRACT.md), where `docs verify` holds it against the actual
CLI. It is deliberately **not** repeated here: a second description of the same commands is
exactly the copy that drifts (AGENTS.md invariant 8), and this document lies outside the files
the command-record check of `docs verify` covers - so a drift here would go unnoticed. What
`pytest` expects of the developer at this point is in
[instructions/dev/testing-conventions.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/instructions/dev/testing-conventions.md).
## The CI half
`.gitea/workflows/ci.yml` runs on every push/PR against `main` (content paths excepted) and
runs the test suite, `docs verify`, `instructions verify` and a complete `setup-instance.md`
replay: a locally built release tarball, the preflight script with `--archive` in an empty
directory, then the instruction's steps - the same path a new user actually takes, just without
the download. `.gitea/workflows/nightly.yml` is the drift check against time rather than against
a commit. `.gitea/workflows/release.yml` is step 6 above, and `.gitea/workflows/wiki-mirror.yml`
refills the Gitea wiki by hand (see below).
Three more workflows carry the live tests of the tracker adapters (Super Productivity, CalDAV):
`ci.yml` runs the CalDAV half in a step of its own against Radicale as a process,
`tracker-live.yml` runs nightly and covers both providers, and `sp-live-image.yml` builds the
`chemenu-sp-live` image daily (when there is a new version) and monthly (always) with the current
Super Productivity version. The image follows the desktop clients' update channel, not a fixed
version. Once, after the very first push, the package has to be linked to the `torben/chemenu`
repository by hand. What the suite writes, how to run it against a tracker of your own, and what
a red run means is in
[instructions/dev/tracker-testing.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/instructions/dev/tracker-testing.md).
### The Gitea wiki
The wiki of `torben/chemenu` shows the documentation of the **latest release**. It is generated
and never edited: every run replaces its whole content, so a change made in the wiki is gone at
the next mirror, and nothing but CI writes to it (a push to the wiki repository by a session
would sit next to `publish`, which AGENTS.md invariant 5 forbids).
`.gitea/scripts/wiki_mirror.py` converts a tree into the wiki's page set: every `*.md` under
`docs/` (the `dev/` pages included), plus `README.md` as `Home` and `INSTALL.md` as `Install`.
Wiki pages in sub-directories are not listed reliably, so the names are flat: the path below
`docs/` with `.md` dropped, each segment capitalised and joined by `-`
(`docs/how-to/ingest-a-source.md` becomes `How-to-Ingest-a-source`; a `README.md` stands for its
directory). Two sources that would get the same name, compared without case, abort the run.
Relative links between mirrored files become wiki page names, relative links to anything else
become links to that file at the release tag, and a link whose target does not exist aborts the
run. A `_Sidebar` is generated, and every page opens with a line saying which file at which
version it came from. `.gitea/scripts/wiki-push.sh` then makes the wiki repository hold exactly
that output - a page the converter no longer produces is removed - and does not commit when
nothing changed.
Two things start it: the `wiki-mirror` job of `release.yml`, after each release (never for a
candidate), and `.gitea/workflows/wiki-mirror.yml` by hand with `workflow_dispatch`. The manual
workflow takes a `tag` (default: the latest release) and runs the converter and push script of
`main` against that tag's tree. Use it to fill the wiki before the first release that carries the
job, or to catch up a mirror that failed after its release was published. `ci.yml` runs
`wiki_mirror.py --check` against the working tree on every push, so a link the mirror could not
resolve fails there rather than at the release. The scripts live under `.gitea/`, which
`dist export` does not copy, so none of this reaches an instance.
Both jobs write with the job token, which needs `permissions: wiki: write` - Gitea gives the wiki
a token scope of its own, and `contents: write` does not cover it (the API answers 403). If the
repository's Actions settings ever cap the token below that, the mirror fails until the cap is
lifted. A wiki that has never had a page has no repository to clone yet; `wiki-push.sh` then
creates the first page through the API and clones afterwards.
### How much of the stack the suite reaches
Coverage is measured in CI - `pytest --cov`, config in `tools/.coveragerc`, HTML and XML
uploaded as the `coverage-<run id>` artifact of every run.
**Fetch that artifact from the run's own page, not from the API**: `upload-artifact@v3` writes
through the older artifact API, and the Actions artifact REST endpoints answer `total_count: 0`
for a run whose artifact the run page offers for download. The upload works; only the listing
does not see it. Do not re-derive this, and do not read the empty list as a failed upload.
It is enforced at a floor of **85%** (`fail_under` in `tools/.coveragerc`), which is what a red
suite from this axis means: coverage actually fell, not that a wrapper was added. The floor was
set only after the number had been watched - it was deliberately held back for exactly that, and
the two points between 85 and the measured 87.0% are the room the taxonomy below asks for. A
threshold at the measured number goes red on the next thin Typer wrapper, and a threshold that
goes red for a non-reason gets lowered rather than earned.
**Measured 2026-09-04, stack 4.7.1: 87.0% of 6498 statements across `chemenu/`, 975 tests** -
CI run 163. The first measurement, at stack 1.8.1 on 2026-08-31, was 86.9% of 5105 statements
over 730 tests (CI run 87). Both are what CI reported, never a local run: the local number
preceding a release measures a tree that is one commit short of the published one.
The pair says more than either number does. Between them the measured code grew by a quarter
and the suite by a third, and the quota moved by a tenth of a point - which is the observation a
threshold was waiting for, rather than the total itself. Reproduce either with
`cd tools && .venv/bin/python -m pytest -q --cov` (needs `pytest-cov`, which is CI-only and
deliberately absent from `tools/requirements.txt` - an instance runs the wiki, it does not
measure this suite).
The total stays the least interesting number here. What the report is for is *which* modules sit
low, and three kinds have to be told apart before any of it turns into work:
- **Thin Typer wrappers**, where the logic lives beside them and is tested there:
`eval_cmd.py` (36%), `types_cmd.py` (40%), `search.py` (49%), `cli.py` (54%),
`links_cmd.py` (61%). Low coverage on a wrapper is evidence of a good cut, not of a missing
test - `search.py`'s uncovered block is its command body alone, while the backends under
`chemenu/search/` that do the work sit between 91% and 98%.
- **Code that reaches the network or the filesystem's outside**, where the interesting half is
already injectable and tested through the seam: `version.py`'s `fetch_latest()` takes a
`fetcher` parameter for exactly that. The real `urllib` lines are exercised too, but only
against a local `http.server` in `test_dist_upgrade.py` (`dist upgrade --latest`) - never
against a real feed, which stays uncovered on purpose.
- **Genuine gaps**, where uncovered lines are logic nobody exercises: `provenance_cmd.py`
(44%), `migrate_cmd.py` (65%), `type_resolver.py` (79%). This is the list worth reading, and
the only one of the three that has not moved while everything around it did:
`provenance_cmd.py` sits where it sat, and `migrate_cmd.py` fell from 71% because it grew and
its new lines arrived untested. The floor freezes this; it does not close it. Closing it is
Gitea #51.
## Stack development as a session type of its own
Three skills (`instructions/dev/`, present only in this origin repository) carry a session that
works on the stack itself rather than on wiki content through three phases: `stack-dev` works
the issue out until its body is "ready", `stack-build` builds, publishes and waits for a green CI
run, `stack-close` checks the body's final state and stale `docs/` and contract prose, and closes
the issue. The handover goes through the state in the tracker, not through a session's context:
every phase change works in the same session or after `/clear`. `stack-dev` triggers
automatically; `stack-build` and `stack-close` are started only by the operator, by slash command
(`/stack-build #N`, `/stack-close`) - exactly where the choice falls whether to continue in the
same session or a new one, and on which model. None of the three offers a model switch in the
middle of a session. Rules and the phase table are in
[instructions/dev/stack-mode.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/instructions/dev/stack-mode.md), the issue tracker itself
in [instructions/dev/issue-tracking.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/instructions/dev/issue-tracking.md).
Explanation-Architecture.md
+281
@@ -0,0 +1,281 @@
*Generated from [`docs/explanation/architecture.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/docs/explanation/architecture.md) at v9.0.0 – edit the repository, not this wiki.*
# Architecture
What Chemenu is made of, and how the parts divide the work. For why the pipeline has exactly four
stages, see [pipeline-rationale.md](Explanation-Pipeline-rationale). This page is the map those reasons
apply to.
<!-- wikitool:toc -->
## Contents
- [A wiki that is compiled, not searched](#a-wiki-that-is-compiled-not-searched)
- [The language of the pages](#the-language-of-the-pages)
- [The tree](#the-tree)
- [Collections and areas](#collections-and-areas)
- [Contracts and READMEs](#contracts-and-readmes)
- [Who does what](#who-does-what)
- [The compiler: `tools/wikitool`](#the-compiler-toolswikitool)
- [The MCP read server](#the-mcp-read-server)
- [Built for IT work](#built-for-it-work)
- [Telemetry and evaluation](#telemetry-and-evaluation)
<!-- /wikitool:toc -->
## A wiki that is compiled, not searched
Chemenu is a pattern for a personal knowledge base maintained by an LLM. A plain retrieval setup goes
back to the raw documents for every question. Here, the LLM **builds and maintains a persistent
wiki** incrementally instead, and the wiki compounds over time. Knowledge is compiled once and kept
current, not re-derived for every question.
That one decision shapes everything else. Raw material is kept unchanged as evidence, and compiled
pages cite it. Every mechanical step of the compilation - frontmatter, cross-references, the catalog,
the log, publishing - is done by a deterministic tool, never by hand, so it comes out the same way
every time.
## The language of the pages
Source material in `raw/` is never touched and stays in whatever language it arrived in. The
compiled pages under `kb/` are written in the instance's own language. The origin repository's demo
corpus, for instance, is German. What stays in its original form inside a page is everything that
is an *identifier* rather than prose: page titles, section headings the tool owns, wikilink targets,
citation ids, schema enum values, tags, commands, paths and code. So `GitOps Ownership Model` and a
German section heading sit in the same page without contradiction.
Which lines count as identifiers is the stack's rule,
[kb/CONTRACT.md § Language and identifiers](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/kb/CONTRACT.md#language-and-identifiers).
*Which language* the prose is in, and what the tool-owned headings are called, is the instance's own
`kb/CONVENTIONS.md`. That is a per-instance decision, not a property of the pattern, so it lives in a
file the instance owns rather than in one the stack ships. A new instance picks its language when it
fills in `kb/CONVENTIONS.md`, before the first ingest. Why the control plane stays English
regardless: [language-boundaries.md](Explanation-Language-boundaries).
## The tree
```
chemenu/
├── AGENTS.md # Control plane: invariants, file naming, routing, gates
├── CLAUDE.md # Claude Code only: imports AGENTS.md, links the one Claude-Code-only decision (model/effort). No rules of its own
├── README.md # Landing page for humans
├── INSTALL.md # Human-readable install: one release, one sentence to the agent
├── CHANGES.md # Changelog for the stack itself
├── USER.md # Who operates this instance - context, never instructions
├── SOUL.md # How this instance sounds. AGENTS.md always wins over it
├── ENVIRONMENT.md # Optional, gitignored: this checkout's harness, MCP servers, remotes
├── *.md.template # Unfilled USER/SOUL/ENVIRONMENT - what a distribution ships instead
├── .gitignore # Anchored so nothing under raw/, kb/ or work/ is ever excluded;
│ # incoming/ is the one directory excluded the other way round
├── .github/hooks/ # Copilot CLI hooks - session tracing
├── .vibe/ # Mistral Vibe hooks + the repo's telemetry policy
├── docs/ # Human documentation
│ ├── tutorials/ # learning: a guided first walk
│ ├── how-to/ # doing: one task per page
│ ├── reference/ # looking up: commands, exit codes, skills, page types, configuration
│ └── explanation/ # understanding: why it is built this way (this page)
├── instructions/ # CONTROL: everything an agent is told to do
│ ├── CONTRACT.md # Instruction vs. skill, publishing, writing standard
│ ├── bootstrap.md # Prepare a further checkout of an instance
│ ├── gates.md # What to do when a gate refuses a call
│ ├── german-terminology.md # Which words stay English in German prose; register
│ ├── session-setup.md
│ ├── page-lifecycle.md
│ ├── publish-cycle.md
│ ├── ingest-large-tree.md
│ ├── ingest-queue.md # Reviewing a submission before wikitool upload accept promotes it
│ └── wiki-*/SKILL.md # Skills - copied into .agents/skills/ and .claude/skills/
├── mcp-upload/ # QUARANTINE (optional): the MCP `submit` tool's write path, gitignored -
│ # read by no command in the ordinary pipeline; a human reviews it with
│ # `wikitool upload list/show/accept/reject`
├── incoming/ # INBOX: content gitignored - drop a file or one folder per source here, `raw accept` promotes it
├── raw/ # INPUT: immutable, untrusted source material
│ ├── CONTRACT.md # Date shard, capture fields, immutability, untrusted content
│ ├── 2026/09/ # Where `raw accept` puts a file: the month it was accepted
│ ├── articles/ # The old type directories: still valid paths, never moved,
│ ├── documents/ # but nothing new is ever routed into them again
│ ├── notes/
│ └── assets/
├── types/ # SCHEMA: the global type surface. Not a collection
│ ├── type-spec.md # Root contract: anatomy, placement, adding a type
│ ├── type-guidance.md # Contract for the *.guidance.md files below
│ ├── entity.md # Entity type config + template (+ .schema.yaml)
│ ├── entity.guidance.md # Its stack-owned authoring prose, shipped verbatim
│ ├── entity.person.md # Subtype template: what `new` scaffolds for entity_type=person
│ ├── concept.md # Concept type config + template (+ .guidance.md, + concept.decision.md)
│ ├── source.md # Source type config + template (+ .guidance.md)
│ ├── comparison.md # Comparison type config + template (+ .guidance.md)
│ ├── project.md # Project (Vorhaben) type config + template, no guidance file
│ ├── instruction.md # Instruction type - lives outside kb/ via `root: repo`
│ └── lint-report.md # Contract-only: describes reports/, owns no directory
├── kb/ # OUTPUT: compiled knowledge. A namespace, not a collection
│ ├── CONTRACT.md # Collections, naming, tone, linking, provenance
│ ├── index.md # Generated catalog *map*: counts and pointers
│ ├── log.md # Generated chronological audit log
│ ├── provenance.md # Generated raw-file reverse index
│ ├── entities/ # COLLECTION.md + INDEX.md + areas below
│ │ ├── codebases/
│ │ ├── systems/
│ │ ├── tools/ # own INDEX.md once past 50 pages
│ │ ├── technologies/
│ │ ├── people/
│ │ └── organizations/
│ ├── concepts/ # COLLECTION.md + INDEX.md + areas below
│ │ ├── architectures/
│ │ ├── patterns/
│ │ ├── protocols/
│ │ ├── workflows/
│ │ ├── decisions/
│ │ └── problems/
│ ├── sources/ # COLLECTION.md + INDEX.md + areas below
│ │ ├── transcripts/
│ │ ├── analyses/
│ │ ├── articles/
│ │ ├── documents/
│ │ ├── notes/
│ │ ├── trackers/
│ │ └── unclassified/
│ ├── comparisons/ # COLLECTION.md - comparison pages, no subtype axis
│ └── gtd/ # COLLECTION.md + INDEX.md + areas below
│ ├── haus/
│ ├── finanzen/
│ └── technik/
├── work/ # WORKSHOP: one directory per multi-session run, tracked
│ └── CONTRACT.md # Run keys, required files, how a run closes
├── reports/ # DERIVED: lint reports, traces, eval scores. Gitignored
│ └── CONTRACT.md
└── tools/ # COMPILER: the wikitool CLI
├── CONTRACT.md # Command records (generated) and index, maintenance schedule
└── README.md # How wikitool is built and how to change it
```
The collections and areas under `kb/` shown here are the origin repository's. An instance's own come
from its type-specs, as described below.
<!-- dist:strip-start -->
Only in the origin repository (`tools/CONTRACT.md` explains how an instance comes to lack them):
```
├── docs/dev/ # Human-readable: development environment and release workflow
└── commonplace/ # Vendored, read-only knowledge base
```
<!-- dist:strip-end -->
## Collections and areas
A directory under `kb/` is a **collection** exactly when it holds a `COLLECTION.md`. A subdirectory
inside one is an **area**, which inherits it, and that is as deep as a page goes. Nothing nests
below an area, because the generated catalog reads exactly two path segments under `kb/`, and it
would silently fold a deeper page into the area. `kb/CONTRACT.md` § Collections has the rule, and
`wikitool lint` reports a violation as a hard error.
Which areas a collection has is not chosen page by page. A type-spec's `layout:` maps its subtype
field onto directories, and `wikitool new` writes each page straight into the one its subtype names.
That is also what gives the catalog's shard threshold any effect: `index rebuild` splits **per
area**, so a collection with no areas keeps one table however large it grows. Once such a collection
is past the threshold, `wikitool lint` reports it as a recommendation rather than an error, together
with the split its subtype field would produce. It stays quiet when the split would not actually
help.
`kb/comparisons/` is the worked example of a collection that stays flat: it has no subtype field for
a `layout:` to key on at all. A *lopsided* subtype field is a different case, and it gets fixed
rather than left flat. `kb/sources/` looked lopsided only because `source_type` had a schema
`default:`, which the compiler applied whenever nobody chose a value. Once the default was removed
and the pages reclassified, it split into six real areas.
Every page type the stack ships, with its subtypes, its directories and its required fields, is in
[page-types.md](Reference-Page-types). An instance adopts those type-specs and may change
them; `tools/wikitool types list` and `tools/wikitool types describe <type>` show what one instance
has.
## Contracts and READMEs
`COLLECTION.md` never appears outside `kb/`. The other layers carry a `CONTRACT.md` or a root
type-spec instead. A stage may carry both a `README.md` and a `CONTRACT.md`, because they have
different readers: the README is for humans working *on* that layer, and the contract binds an agent
working *with* it. `AGENTS.md` § File naming says which file is for which reader, and
`wikitool docs verify` checks it.
## Who does what
**You** curate sources by dropping them into `incoming/`, ask questions, review what changed in
`kb/log.md` and `kb/index.md`, and steer the agent on what to emphasise or look into. The
[how-to guides](How-to) cover each of these.
**The agent** follows `AGENTS.md`, the cross-cutting schema and policy, and runs the step-by-step
procedures as skills. They are published into `.agents/skills/`, and mirrored into
`.claude/skills/` for Claude Code, by `tools/wikitool instructions sync`. Which skills ship, what
each does and which requests start it: [skills.md](Reference-Skills).
**`tools/wikitool`** does every mechanical step a skill needs: scaffolding, renaming and deleting
pages, cross-references, regenerating the catalog, log and provenance index, structural linting,
decay math, and publishing. The agent calls it and never does that work by hand. Along the way it
maintains:
- `kb/index.md`, the catalog, always current;
- `kb/log.md`, the complete audit trail;
- `kb/provenance.md`, the reverse index from raw files to pages;
- pages of every type, each in the directory its type-spec names;
- lint reports, session traces and eval scores in `reports/`, which is gitignored.
## The compiler: `tools/wikitool`
Every command carries one data record kept next to its code: synopsis, properties, copyable
examples, each exit cause with what to do about it, prohibitions, and notes on its behaviour. The
same record is rendered four ways:
- `tools/wikitool <command> -h` prints it, and `tools/wikitool -h` prints a one-line index of every
command;
- a command that fails with exit 1 prints the record's reactions on stderr, right under its `ERROR`
line;
- [tools/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/tools/CONTRACT.md) holds a generated copy for agents, together with the
maintenance schedule;
- [the command reference](Reference-Commands) is a generated copy for humans.
`tools/wikitool docs verify` checks the copies, and every command's flags, against the CLI in both
directions. [tools/README.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/tools/README.md) is the other half: how the CLI is built and how
to add a command. `AGENTS.md` holds the invariants that say when each command is mandatory.
The wiki is also a git repository, and `tools/wikitool publish` is the one way its changes are
committed and pushed. That gives every change a version history, and the gates in front of
`publish` decide what needs a human to look first
([why-gates-are-code.md](Explanation-Why-gates-are-code)).
<!-- dist:strip-start -->
Extending `tools/wikitool`, the type schema or the instruction layer itself is a separate kind of
session, with rules of its own: the `stack-dev`, `stack-build` and `stack-close` skills under
`instructions/dev/`, which only the origin repository has.
<!-- dist:strip-end -->
## The MCP read server
The terminal is not the only way in. `tools/chemenu/mcp/` serves the same wiki read-only over MCP -
`search`, `types`, `describe_type`, `lint` and `status` - so a consumer that is not a shell on this
machine can ask the same questions and get the same answers. Literally the same: the CLI and the
server are two adapters over one core, and a golden test holds their output together rather than
trusting that it agrees.
There is no tool that writes to `kb/`, and not because one is filtered out of a list. The server
imports nothing under `chemenu/commands/`, so `new`, `publish` and the rest cannot be reached from
it. Every answer carries the commit it was computed from, so a checkout that has fallen behind gives
a visibly stale answer instead of a confident wrong one. The server's dependency is deliberately
not in `requirements.txt`: an instance that only uses the CLI should not have to install a web stack
for it.
Setting it up: [set-up-the-mcp-server.md](How-to-Set-up-the-mcp-server).
## Built for IT work
The shipped defaults are tailored to IT work:
- **entity types** for software development and systems - codebases, systems, tools, technologies;
- **relationship labels** such as "depends on", "uses" and "implements". The vocabulary is in the
instance's `kb/CONVENTIONS.md`, because it belongs to the instance rather than the stack;
- **templates** per page type, and per subtype where its pages need a shape of their own, such as a
person or a decision record (ADR);
- **guidelines** for documenting technical decisions, which can be exported back into the
repositories they are about ([export-guidelines.md](How-to-Export-guidelines));
- **cross-reference patterns** for code and architecture.
## Telemetry and evaluation
How the stack observes what an agent did, and how that record becomes a score, is its own page:
[telemetry-and-evaluation.md](Explanation-Telemetry-and-evaluation).
Knowledge-and-commitment.md → Explanation-Knowledge-and-commitment.md
+3 -3
@@ -1,4 +1,4 @@
*Generated from [`docs/knowledge-and-commitment.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/docs/knowledge-and-commitment.md) at v8.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/explanation/knowledge-and-commitment.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/docs/explanation/knowledge-and-commitment.md) at v9.0.0 – edit the repository, not this wiki.*
# Why knowledge and commitments are two layers
@@ -6,8 +6,8 @@ Chemenu compiles knowledge into `kb/`, and it also tracks what its operator has
Those look like one subject - both are "things about my projects" - and the stack deliberately
keeps them apart: `kb/gtd/` holds one page per initiative, an external task tracker holds the
open items, and the only thing that crosses between them is a name. This page is about why that
line was drawn there. The rules that follow from it live in [kb/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/kb/CONTRACT.md)
and the `review`, `new` and `task new` records in [tools/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/tools/CONTRACT.md).
line was drawn there. The rules that follow from it live in [kb/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/kb/CONTRACT.md)
and the `review`, `new` and `task new` records in [tools/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/tools/CONTRACT.md).
<!-- wikitool:toc -->
## Contents
Language-boundaries.md → Explanation-Language-boundaries.md
+35 -9
@@ -1,13 +1,15 @@
*Generated from [`docs/language-boundaries.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/docs/language-boundaries.md) at v8.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/explanation/language-boundaries.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/docs/explanation/language-boundaries.md) at v9.0.0 – edit the repository, not this wiki.*
# Language Boundaries
Two languages run through this repo at once. `kb/` is written in whatever language the instance
chose - German here, and the value lives in `kb/CONVENTIONS.md`'s `language:`. Everything that
tells an agent what to do - [AGENTS.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/AGENTS.md), every `CONTRACT.md`, everything under
`instructions/` - is written in English, in every instance, whatever the first value says.
tells an agent what to do - [AGENTS.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/AGENTS.md), every `CONTRACT.md`, everything under
`instructions/` - is written in English, in every instance, whatever the first value says. So
is the stack's own documentation for humans, for a different reason (§ "The stack's human
documentation").
The rule itself is in [AGENTS.md § File naming](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/AGENTS.md#file-naming). This page holds the
The rule itself is in [AGENTS.md § File naming](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/AGENTS.md#file-naming). This page holds the
part that is not a rule: why the line runs where it does, why the English half is not a setting,
and which argument for it turned out to be wrong.
@@ -19,6 +21,7 @@ and which argument for it turned out to be wrong.
- [Why it is not a parameter](#why-it-is-not-a-parameter)
- [What the KB language still decides](#what-the-kb-language-still-decides)
- [Where the line runs around a page type](#where-the-line-runs-around-a-page-type)
- [The stack's human documentation](#the-stacks-human-documentation)
- [What would put this back on the table](#what-would-put-this-back-on-the-table)
<!-- /wikitool:toc -->
@@ -27,7 +30,7 @@ and which argument for it turned out to be wrong.
For a long time the two halves could be told apart by asking who owned the file, and the answer
came out right every time: the stack owns `AGENTS.md` and the contracts, which are English; the
instance owns its pages and the templates that shape them, which are in the KB language. The
ownership boundary is a real and load-bearing thing - [ownership-and-templates.md](Ownership-and-templates)
ownership boundary is a real and load-bearing thing - [ownership-and-templates.md](Explanation-Ownership-and-templates)
is about what it buys - so it was easy to read the language split as one of its consequences.
It is not. The case that separates them is a page type an instance adds for itself. `types/`
@@ -37,7 +40,7 @@ guidance is still instruction addressed to an agent, and reads exactly like the
four types the stack ships. Ownership says "yours"; the audience has not moved at all.
So the question a line answers is not *whose file is this* but *who reads this line*, which is
the same cut [kb/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/kb/CONTRACT.md#language-and-identifiers) already makes inside a
the same cut [kb/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/kb/CONTRACT.md#language-and-identifiers) already makes inside a
single page between prose and identifiers - applied one level up, to the halves of a document.
Ownership decides who may change a sentence. The reader decides what language it is in. The two
questions were answered together for as long as they happened to agree.
@@ -101,7 +104,8 @@ experiences:
language.
So an operator who reads no English gets German pages and German answers from an agent reading
English instructions. The English is what the machinery is written in, not what it says back.
English instructions. The English is what the machinery is written in, not what it says back -
with one exception, the stack's documentation for humans (§ "The stack's human documentation").
## Where the line runs around a page type
@@ -117,11 +121,28 @@ beside the template, but that split was made for ownership reasons - so an upgra
the guidance without overwriting what the instance chose - and it leaves this page's argument
untouched: each part is still written in the language its own reader needs, and a type-spec that
declares no `guidance:` keeps both halves in one file with exactly the same rule applying inside
it. [types/type-spec.md § Who owns a type-spec](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/types/type-spec.md#who-owns-a-type-spec) has
the split as a table, and [ownership-and-templates.md](Ownership-and-templates) § "Where the
it. [types/type-spec.md § Who owns a type-spec](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/types/type-spec.md#who-owns-a-type-spec) has
the split as a table, and [ownership-and-templates.md](Explanation-Ownership-and-templates) § "Where the
file boundary used to strain" has what it cost to keep two audiences in one file for as long as
it did.
## The stack's human documentation
`README.md`, `INSTALL.md` and everything under `docs/` are read by humans, and the
reader axis alone would put them in the operator's language. They are English anyway, and the
reason is not the reader but the fact that there is no single one. This documentation belongs to
the stack, not to an instance: it ships identically to every instance, whatever each one's
`language:` says, so there is no instance value it could follow - only a translation per language
to maintain beside it, each one a second copy that drifts. And a growing part of it is generated
from the code (the command reference from `cli_contract` records, for one), whose source is
English control-plane material; a translated frame around English generated text is the
half-done version of both.
The KB language keeps deciding what an operator reads *of their own*: pages and answers. What
the stack tells every operator about itself is the same text everywhere. `CHANGES.md` sits
outside the question - it is a log of what was true at a time, not documentation, and its older
entries are in the language they were written in.
## What would put this back on the table
A `docs/` page goes stale when the reasoning stops holding rather than when the code changes, so
@@ -137,3 +158,8 @@ it is worth naming what that would look like here. Two things would:
Neither is close. Both are cheaper to notice here than to rediscover in an argument about a
single file.
The human documentation has a trigger of its own: **translations the stack maintains itself,
with a check that holds each one to its English source.** Without such a check a translation is
exactly the drifting second copy the section above argues against; with one, the argument for a
single language weakens to cost alone.
Model-and-effort-selection.md → Explanation-Model-and-effort-selection.md
+3 -3
@@ -1,12 +1,12 @@
*Generated from [`docs/model-and-effort-selection.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/docs/model-and-effort-selection.md) at v8.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/explanation/model-and-effort-selection.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/docs/explanation/model-and-effort-selection.md) at v9.0.0 – edit the repository, not this wiki.*
# Choosing a Claude Code model and effort level
Claude Code exposes three choices this repo has an opinion on: which model a session itself
runs as, what model a spawned subagent gets, and which `/code-review` effort level to pick.
None of them are enforced anywhere - the gates in [instructions/gates.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/instructions/gates.md)
None of them are enforced anywhere - the gates in [instructions/gates.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/instructions/gates.md)
are code precisely because a model cannot be talked out of them
([why-gates-are-code.md](Why-gates-are-code) makes that argument for gates; this page applies
([why-gates-are-code.md](Explanation-Why-gates-are-code) makes that argument for gates; this page applies
the same axis to who is holding the keyboard). What follows is a reference for making that choice
well, not a rule anything checks.
Ownership-and-templates.md → Explanation-Ownership-and-templates.md
+21 -9
@@ -1,11 +1,11 @@
*Generated from [`docs/ownership-and-templates.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/docs/ownership-and-templates.md) at v8.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/explanation/ownership-and-templates.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/docs/explanation/ownership-and-templates.md) at v9.0.0 – edit the repository, not this wiki.*
# Ownership and Templates
Chemenu ships two kinds of files side by side, and at a glance they look the same: both are
plain markdown, both sit in the repo root or under `kb/`, both get read at session start. But a
stack upgrade treats them completely differently. Some - [AGENTS.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/AGENTS.md),
[kb/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/kb/CONTRACT.md), the per-stage contracts - are identical in every instance
stack upgrade treats them completely differently. Some - [AGENTS.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/AGENTS.md),
[kb/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/kb/CONTRACT.md), the per-stage contracts - are identical in every instance
that runs this stack and are the next release's to replace (with one caveat about local edits,
below). Others - `USER.md`,
`SOUL.md`, `kb/CONVENTIONS.md`, `ENVIRONMENT.md` - describe one particular instance, and
@@ -27,7 +27,7 @@ overwriting them would silently erase a choice someone made on purpose.
The stack-owned files describe how the tool works. `kb/CONTRACT.md` opens by saying it holds
what `tools/wikitool` enforces or what follows mechanically from how it operates - see
[kb/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/kb/CONTRACT.md), lines 10-13. That kind of statement doesn't vary by
[kb/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/kb/CONTRACT.md), lines 10-13. That kind of statement doesn't vary by
instance: the compiler behaves the same way regardless of who is running it, so the sentence
describing that behavior can be copied byte-for-byte into every checkout without becoming
wrong anywhere.
@@ -35,7 +35,7 @@ wrong anywhere.
The instance-owned files describe a choice: which language pages are written in, what tone the
agent takes, who the operator is, which git remote is authoritative, which MCP servers are
reachable. None of that follows from the tool's mechanics - two instances of the identical
stack can answer all of these differently and both be correct. [AGENTS.md § Personalization](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/AGENTS.md#personalization)
stack can answer all of these differently and both be correct. [AGENTS.md § Personalization](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/AGENTS.md#personalization)
frames the split the same way for `kb/CONTRACT.md` versus `kb/CONVENTIONS.md`: "the split is by
who may change the sentence, not by what it is about." A rule about page structure could in
principle have been written per-instance too, but then every instance answering "not German" to
@@ -61,7 +61,7 @@ the file itself already answers that.
For a while the boundary was written down as a list of paths - once in `dist_cmd.py`, once in
the merge procedure a private instance was told to run by hand, and once in the check that
procedure ended with. Three copies of one fact, which is the shape [AGENTS.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/AGENTS.md)
procedure ended with. Three copies of one fact, which is the shape [AGENTS.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/AGENTS.md)
invariant 8 exists to forbid, and they drifted exactly as predicted: the hand-run procedure was
still naming three paths after the collection contracts had moved to the instance's side of the
line, so it discarded upstream changes to files it had never heard of, while its own final check
@@ -83,7 +83,7 @@ The mechanism for instance-owned content is a `.template` file the distribution
of the real one - `USER.md.template`, `SOUL.md.template`, `kb/CONVENTIONS.md.template`,
`ENVIRONMENT.md.template`. An alternative would have been to ship nothing at all and let a
brand-new instance start from a blank page. The template exists because a blank page doesn't
tell [instructions/setup-instance.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/instructions/setup-instance.md) what shape the answer
tell [instructions/setup-instance.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/instructions/setup-instance.md) what shape the answer
should take, and it gives nothing for a validator to check afterward.
A template carries a placeholder value - a sentinel - in the fields that need a real answer.
@@ -91,7 +91,7 @@ Setup interviews the operator and replaces the sentinel with what they actually
gives `doctor` a mechanical way to tell "personalized" from "not yet": a file that still
contains the sentinel hasn't been through setup, regardless of whether the file exists. That's
also why `ENVIRONMENT.md` only warrants a WARN rather than a FAIL when absent - see
[AGENTS.md § Environment](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/AGENTS.md#environment) - while a missing or unfilled
[AGENTS.md § Environment](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/AGENTS.md#environment) - while a missing or unfilled
`USER.md`/`SOUL.md`/`kb/CONVENTIONS.md` is a harder failure: `ENVIRONMENT.md` describes one
checkout among possibly several and is gitignored for that reason, so its absence is a normal
state rather than a sign setup was skipped.
@@ -111,7 +111,7 @@ release had any business touching them.
The same file is where the language question comes apart from the ownership one, and for the same
reason: ownership decides who may change a line, its reader decides what language it is in -
which is why a `root: kb` type-spec still keeps English prose around a template block written in
its own language. [language-boundaries.md](Language-boundaries) has that argument; this page is
its own language. [language-boundaries.md](Explanation-Language-boundaries) has that argument; this page is
about ownership alone.
Ownership is per file, so a file carrying both audiences had to give both halves to whoever owned
@@ -181,6 +181,18 @@ sha256 per shipped file in `.wikitool-release.json`. `wikitool dist upgrade` com
candidate path against the digest recorded when it was installed, overwrites only what still
matches, and refuses rather than overwrite what does not.
A generated verbatim file adds one more way to touch it without meaning to. The reference pages
under `docs/reference/` are verbatim, and `wikitool docs reference --apply` rewrites their
generated regions in whatever tree it runs in. Most of them render records in `tools/` - the
command records, and the shapes of the configuration files an operator writes - which an instance
leaves alone, so they come out the same everywhere. The skill
catalogue and the page types would render material an instance does own: its adopted type-specs
and any instructions it added. Regenerated there, they would turn a shipped page into a locally
changed one, and the next upgrade would refuse to replace it. So those two pages describe the
stack's shipped defaults. They are generated and checked only in the origin repo, the one tree
without a release stamp, and an instance leaves them alone and asks `wikitool types describe` or
`wikitool instructions list` for its own state.
So the practical rule is narrower than "overwrite the verbatim files, leave the rest alone":
overwrite the verbatim files *this instance has not touched*, never write the other two
categories, and make a locally changed file a decision someone takes deliberately instead of
Pipeline-rationale.md → Explanation-Pipeline-rationale.md
+8 -8
@@ -1,16 +1,16 @@
*Generated from [`docs/pipeline-rationale.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/docs/pipeline-rationale.md) at v8.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/explanation/pipeline-rationale.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/docs/explanation/pipeline-rationale.md) at v9.0.0 – edit the repository, not this wiki.*
# Why the pipeline has four stages
Chemenu could, in principle, be one directory: drop a file in, ask a question, get an answer
computed fresh each time. It isn't built that way. The pipeline in
[AGENTS.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/AGENTS.md#routing) - `raw/` -> `[types/ + tools/]` -> `kb/` -> `reports/`, with
[AGENTS.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/AGENTS.md#routing) - `raw/` -> `[types/ + tools/]` -> `kb/` -> `reports/`, with
`work/` alongside rather than inside it - separates *material* from *meaning* from
*byproduct*, and each seam exists because collapsing it costs something specific.
## Why raw material stays untouched
[raw/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/raw/CONTRACT.md) keeps a source exactly as it arrived. The reasoning is
[raw/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/raw/CONTRACT.md) keeps a source exactly as it arrived. The reasoning is
simple once stated: the moment someone "cleans up" or reformats a source on the way in, the
thing later claims get checked against is no longer the thing that was actually said. An
immutable `raw/` means a citation always resolves to the original, not to somebody's tidied
@@ -33,19 +33,19 @@ promotion into `raw/` quietly doubling as the review step it cannot perform.
## Why extraction happens once, through a schema
[types/type-spec.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/types/type-spec.md) is what stands between a raw file and a `kb/` page:
[types/type-spec.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/types/type-spec.md) is what stands between a raw file and a `kb/` page:
a type-spec defines what a conforming instance of a page looks like, and the compiler
(`tools/wikitool`) applies it. The alternative - every query re-reading and re-interpreting the
source on demand - would mean paying the cost of understanding the material every single time,
and getting a slightly different answer each time depending on how the question was phrased.
Extracting once, against a fixed schema, turns "re-read and re-guess" into "look up what was
already compiled." That is the "never re-derive, always compile" principle from
[AGENTS.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/AGENTS.md): understanding a source is expensive and worth doing exactly once,
[AGENTS.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/AGENTS.md): understanding a source is expensive and worth doing exactly once,
after which it becomes a cheap, stable lookup.
## Why a `kb/` page has to stand on its own
[kb/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/kb/CONTRACT.md) sets the bar for the compiled layer: a page should answer a
[kb/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/kb/CONTRACT.md) sets the bar for the compiled layer: a page should answer a
future question without sending the reader back to the source it came from. That's the payoff
of compiling in the first place - if every answer still bottomed out in "go re-read the raw
file," the `kb/` layer would just be a pointer with extra steps, and the cost of extraction
@@ -54,7 +54,7 @@ consistent to query: the work of understanding is already sitting there, done.
## Why `reports/` doesn't need to be maintained
[reports/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/reports/CONTRACT.md) treats most of what lands in `reports/` -
[reports/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/reports/CONTRACT.md) treats most of what lands in `reports/` -
lint output, telemetry traces - as disposable. The structural content of a lint report can be
recomputed from the tree at any commit, so keeping an old copy around would just be a second
version of something the tool can already answer on demand, and a second copy is exactly the
@@ -66,7 +66,7 @@ distinction between what's recomputable and what isn't.
## Where `work/` fits
[work/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/work/CONTRACT.md) describes a workshop, not a fifth pipeline stage: a
[work/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/work/CONTRACT.md) describes a workshop, not a fifth pipeline stage: a
place for the notes, extracts and open decisions of a task that spans more than one session, on
its way toward becoming a `kb/` page. It sits beside the raw -> kb -> reports flow rather than
inside it - closer in spirit to a desk than to a conveyor belt.
Explanation-Telemetry-and-evaluation.md
+356
@@ -0,0 +1,356 @@
*Generated from [`docs/explanation/telemetry-and-evaluation.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/docs/explanation/telemetry-and-evaluation.md) at v9.0.0 – edit the repository, not this wiki.*
# Telemetry and evaluation
How the stack observes what an agent did, and how that record turns into a score. This page explains
the design and points at the code. The rules an agent follows live in
[tools/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/tools/CONTRACT.md) and [reports/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/reports/CONTRACT.md).
Turning tracing on and reading a session back is a how-to:
[trace-and-score-sessions.md](How-to-Trace-and-score-sessions).
<!-- wikitool:toc -->
## Contents
- [Why, beyond the unit tests](#why-beyond-the-unit-tests)
- [Architecture](#architecture)
- [The trace](#the-trace)
- [Whether it runs at all](#whether-it-runs-at-all)
- [What never reaches a trace](#what-never-reaches-a-trace)
- [Evaluation levels](#evaluation-levels)
- [L0 lives in the test suite, not in a separate harness](#l0-lives-in-the-test-suite-not-in-a-separate-harness)
- [Scoring a session](#scoring-a-session)
- [A worked example, and where the check moved to](#a-worked-example-and-where-the-check-moved-to)
- [Status](#status)
- [The agent runner, and why it is not here yet](#the-agent-runner-and-why-it-is-not-here-yet)
<!-- /wikitool:toc -->
## Why, beyond the unit tests
The test suite that ships with the origin repository checks the **compiler**: given this input, does
`wikitool` produce that output? It says nothing about the two things that actually go wrong in
practice: whether the *agent* followed the contracts, and whether the pages it wrote are any good.
Those need a different kind of check. Two notes in the
[Commonplace](https://github.com/zby/commonplace) research base, which the origin repository
vendors, supply the theory:
- *oracle-strength-spectrum* - correctness checks form a gradient, from hard (deterministic) to none
(vibes). The engineering move is to *harden* oracles step by step, not to reach straight for a
judge.
- *evaluation-automation-is-phase-gated-by-comprehension* - comprehension, then specification,
then generalisation. A judge built before anyone has read real failures optimises a proxy.
That ordering is why this page describes a lot of telemetry and only a little scoring. Reading real
traces is the first phase, and it cannot be skipped.
## Architecture
Three sources, three different jobs.
```mermaid
flowchart TD
subgraph runner["Eval runner (planned)"]
R["isolated HOME · programmatic mode<br/>NDJSON capture · run manifest"]
end
subgraph hooks["Harness hooks (interactive work)"]
H1["Claude Code<br/>.claude/settings.json"]
H2["Copilot CLI<br/>.github/hooks/*.json"]
H3["Mistral Vibe<br/>.vibe/hooks.toml"]
H4["VS Code Chat<br/>chronicle SQLite, post hoc"]
end
subgraph inner["Repo layer (always on)"]
W["wikitool emitter<br/>+ git"]
end
R --> T[("reports/telemetry/&lt;session&gt;/trace.jsonl")]
H1 & H2 & H3 & H4 --> I["tools/trace_ingest.py"] --> T
W --> T
T --> S["scorers L0-L4"] --> O[("reports/evals/&lt;date&gt;/")]
```
- **The repo layer is the truth.** `wikitool` records its own calls, so what happened *to the wiki*
is known even when no hook fired and no runner was involved.
- **The runner owns session boundaries.** Not every harness reports a session start - Mistral Vibe
has no such hook - so the process that launches the agent is what brackets a run.
- **Hooks enrich.** They add the tool calls the repo layer cannot see: file reads, greps, shell
commands, prompts.
Everything joins on one session id, which every source that has to pick one resolves the same way
(`chemenu.session`). The chain is `WIKITOOL_SESSION_ID`, then a harness's own session variable where
one is registered (`chemenu.session.HARNESS_ENV_VARS` - Claude Code's `CLAUDE_CODE_SESSION_ID` today),
then the parent process id. The middle step exists because the last one does not survive a harness
that runs every tool call in its own freshly started shell. There, `os.getppid()` is a new "session"
for every call, and neither the join nor the Iteration Budget Gate could see more than one or two
calls of a real run. A harness only earns an entry in that chain once a live session has been
observed setting the variable, and it has been confirmed to be the exact id the harness's own hooks
write elsewhere in a trace. A name that merely looks plausible would mis-key a session more quietly
than the pid fallback it replaced.
A trace hook that only *observes* tool calls (a `PreToolUse`/`PostToolUse`-style wiring) does not by
itself fix a harness whose events carry a different id than `wikitool`'s own emitter. The two would
still not join. Wiring such a hook is worth doing only once this fallback chain already keys both
sides on the same id.
## The trace
One JSON object per line, appended to `reports/telemetry/<session>/trace.jsonl`. The contract is
[tools/chemenu/telemetry/schema.py](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/tools/chemenu/telemetry/schema.py).
| Field | Meaning |
|---|---|
| `v` | Schema version |
| `ts` | ISO-8601 UTC, microsecond precision |
| `session_id` | The join key - `chemenu.session`'s fallback chain: `WIKITOOL_SESSION_ID`, else a registered harness variable, else the parent process id |
| `pid`, `seq` | `seq` counts **within one process**. Sort a trace by `(ts, pid, seq)` |
| `source` | `wikitool`, `runner`, or a harness name |
| `event` | See below |
| `attrs` | Normalised payload - the same field names whichever harness produced it |
| `run_key` | The `work/<runkey>/` run, when one is open |
| `trace_id`, `span_id` | From `TRACEPARENT`, when the harness exports it |
| `redactions` | Which secret patterns fired on this event |
**Events.** The core - `tool.pre`, `tool.post`, `wikitool.call`, `gate.refused` - is available on
every surface. Everything else is optional: `session.start`, `session.end`, `session.error`,
`prompt.submitted`, `assistant.message`, `turn.end`, `tool.error`, `instructions.loaded`,
`subagent.start`, `subagent.stop`, `compaction`, `page.written`, `publish.commit`, `budget.state` and
`gate.cleared`.
**The degradation rule.** No scorer may *require* an optional event. Claude Code has about thirty
hooks and Mistral Vibe has three. A scorer built on the rich end would silently report zero on the
poor end, which reads as "the agent did nothing" rather than "this harness cannot say". Every
`session.start` carries a `completeness` list naming the classes its harness can emit, so a scorer
can answer "not measurable here" instead. What each harness can report is tabled in
[trace-and-score-sessions.md](How-to-Trace-and-score-sessions#what-each-harness-can-report).
**Budget and trace are not the same set.** The Iteration Budget Gate exempts read-only retrieval,
and the trace records it. What an agent looked at before acting is exactly what a trajectory scorer
needs, and charging for a `search` would discourage the one habit that lowers cost.
## Whether it runs at all
The default depends on how the tree got here, not on one hard-coded switch.
[tools/chemenu/telemetry/policy.py](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/tools/chemenu/telemetry/policy.py) is the one place that
resolves it, so `wikitool doctor`, the writer and the MCP server's start-up guard all answer the
same question the same way:
| Installation form | Default | Marker |
|---|---|---|
| Git clone of the origin repository (dev checkout) | **on** (opt-out) | No `.wikitool-release.json` |
| An instance installed from a release, and every clone of its own repository | **off** (opt-in) | `.wikitool-release.json` present |
The form is read off `.wikitool-release.json`, the same stamp `version check`, `dist upgrade` and
`version notes` already use to tell a distribution from the repository it came from. Present means
an operator never asked for telemetry. Absent means this is the dev checkout the stack ships from,
where the traces are the stack's own measuring instrument. An instance commits the stamp with its
first `publish`, so a clone of it on a second machine carries the stamp and defaults to off too. It
is a consuming instance, not a measuring stand.
**Turning it on for a distributed instance** takes a per-checkout `.wikitool-telemetry.json` at the
root, gitignored like `.wikitool-remotes.json` and for the same reason. The consent to write
prompts in cleartext to *this* disk belongs to the checkout, not the corpus, so a second clone must
not inherit it silently. The setup instruction asks about it; nothing writes it automatically. Its
shape is in the [configuration reference](Reference-Configuration#telemetry).
**Two independent quantity caps, both enforced fail-silent in `emit()`.** Neither is enforced in
`write_event()`, which the test suite calls directly to exercise the format without the policy
around it.
- **A byte cap per session trace** (default 5 MiB), checked with one `stat` before every append.
Once a trace is at or over the cap, further calls in that session write nothing except a single
`telemetry.limit` event. It is elected by the same single-writer trick `session.start` uses, an
exclusive create on a `.limit` marker file, so a trace that was cut off can be told apart from one
whose writer simply crashed.
- **Retention by session count** (default 250), applied once, right before a brand-new session
directory is created. It is never applied per event, and never against the session doing the
creating. It deletes exactly `trace.jsonl` and `.limit` from the oldest directories beyond the cap,
and `rmdir`s a directory only once it is empty. Nothing under `reports/` is ever removed in bulk.
The default of 250 was set above what the origin repository's own checkout had accumulated as of
2026-09-10: 231 session directories, well under 400 KiB in total. The cap bites on future growth,
not on the existing history.
## What never reaches a trace
Prompts and assistant replies **are** recorded in cleartext, locally. A failure taxonomy cannot be
read out of hashes, and building one is the first phase of any evaluation work. Four guards make
that defensible, all of them in
[tools/chemenu/telemetry/scrub.py](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/tools/chemenu/telemetry/scrub.py), so there is one place to
audit:
1. **Secret scrubbing.** Tokens, keys, `Authorization:` headers and `SECRET=` assignments are
replaced with `[REDACTED:<type>]`. The scrubbing is pattern-based, so it is best effort.
2. **A content cap.** 60 KiB per attribute, with a `[TRUNCATED n chars]` marker.
3. **A kill switch.** `WIKI_TRACE_CONTENT=0` keeps only `<field>_length` and `<field>_sha256`. The
digest is computed either way, so traces stay comparable.
4. **`raw/` contents never enter a trace at all.** That text is data, not instruction
([AGENTS.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/AGENTS.md) invariant 4), and a trace gets read back later. Callers record a path
and a digest.
Nothing leaves the machine. `reports/` is gitignored, no exporter is configured, and where a vendor
offers one it is off. Mistral's `enable_otel` ships prompts to Mistral Studio, so it stays `false`,
and so does `enable_telemetry`. Claude Code's and Copilot's content gates may only be enabled
against a collector you run yourself.
The variables that steer all of this are in the
[configuration reference](Reference-Configuration#environment-variables).
## Evaluation levels
Ordered by oracle strength: hard checks first, judgment last.
| Level | Oracle | What it measures | Status |
|---|---|---|---|
| **L0** Pipeline | hard | A wiki the tools built themselves lints clean, and the catalog is a fixed point | **done** - in the origin repository's test suite |
| **L1** Artifact scorecard | hard | Counters from `lint`'s own checks: hard errors and advisories, page count | **done** |
| **L2** Trajectory | hard | Rules over the trace: was a refused call repeated, was a gate flag passed unearned, did a page change go unlogged | **done** |
| **L3** Task evals | medium | Gold set: question → expected cited pages; ingest fixture → expected page titles. Scored by set overlap | needs the runner |
| **L4** Rubric / judge | soft | Prose quality, cramming, tone | **out of scope** until a failure taxonomy exists |
Evaluation results are statistical, not binary. A case runs several times and reports a pass rate
with its variance, because sampling is not deterministic. A single failure is not a merge blocker;
a regression against a baseline is.
### L0 lives in the test suite, not in a separate harness
It runs the CLI in-process against an empty wiki. It asserts that `new` → write → `xref` →
`index rebuild` leaves a tree `lint` calls clean, and that rebuilding the catalog again changes
nothing on disk. That is a hard oracle over the compiler, which is what the test suite is for.
Giving it its own runner would have duplicated the suite for nothing.
One behaviour it pins is easy to mistake for a defect: **a scaffolded page does not lint clean**.
`new` writes placeholder wikilinks for the author to replace. So a page that was created but not yet
written reports broken links, and reports its `TODO`-only sections as the advisory
*Unfilled Template Sections*. That is the scaffold saying it is unfinished.
## Scoring a session
`tools/wikitool eval score` scores one session. It only reads, and it is exempt from the Iteration
Budget Gate: reading back what a session did is not iterating on the wiki, and charging for it
would discourage checking one's own work.
**L1** re-runs `lint`'s checks in-process and reports their counters. It shares its definition of a
hard error with `lint --fail-on-error` - one constant, `HARD_ERROR_KEYS` - so a run can never pass
its score while lint refuses it.
**L2** checks five rules. Each one restates an invariant the code cannot enforce in-process. A gate
can refuse a call, but nothing stops an agent from calling again with the gate's own flag. That gap
is the whole point:
| Rule | Invariant | Severity |
|---|---|---|
| `refusal-not-retried` | A refused call, repeated unchanged, is the loop the gate exists to break | error |
| `gate-not-self-opened` | `--yes`/`-y` no longer exist at all; `--override-budget` is for a human to pass after a refusal; `--force` never | error |
| `content-change-logged` | A publish that changes `kb/` pages needs a `log append` in the same session | advisory |
| `clearance-was-asked-for` | A `gate.cleared` token must match one some earlier `gate.refused` issued - catches an invented token, and one reused from a different changeset | error |
| `clearance-ended-the-turn` | No `wikitool.call` between a clearance request (exit 42) and the next `prompt.submitted` - skipped, not failed, on a harness that cannot report `prompt.submitted` | error |
New rules belong here only when a real trace shows a real failure. Checks invented from the contract
text produce a score that improves while behaviour does not - the failure mode the phase-gate note
describes. The first three rules were chosen because each guards a refusal an agent can talk its way
around. The last two carry the load for the Mass-Update Gate's clearance mechanism (2026-08-28),
which deliberately enforces very little in-process, as described below.
A rule can report a third state besides pass and fail: **skipped**, meaning this trace cannot answer
the question at all - for example `clearance-ended-the-turn` on a harness with no prompt hook.
Skipped reads as neither a pass nor a violation. `scorecard.render_markdown` marks it `skip`,
distinct from `ok`, `warn` and `FAIL`, and `score()["violations"]` never includes a skipped rule,
whatever its severity.
### A worked example, and where the check moved to
Scoring the session that shipped the telemetry layer itself used to report a failure:
```
FAIL gate-not-self-opened
call=publish --message "tools: session traces …" --yes, reason=no mass-update refusal preceded it
```
That finding was true and benign at the same time. The change was 17 files, the file list was shown,
and a human approved it. But the approval happened in conversation, and `--yes` went out on the
first attempt, so the gate never got to refuse anything. The documented path had the refusal first,
and the trace records tool calls, not conversations.
Three sessions did some version of this before the mechanism changed rather than the procedure. The
interesting part is *which* half changed. The obvious fix was built first: make the gate impossible
to open without independent proof of a human. That meant a ticket bound to the changeset, redeemed
only after a `prompt.submitted` event and a `wikitool approve` typed in a second terminal. It worked,
and it was unusable. On a harness that drives `wikitool` through a shell tool, the agent cannot
produce the turn signal, and the human is asked to leave the conversation to approve anything at all.
A gate nobody can pass is not a safe gate, it is an abandoned one.
So the enforcement got cheap and the *verification* got real:
- **In-process, `publish` enforces only what it can do honestly.** Exit 42 says "a human must see
this". `--confirm <token>` says "here is what they saw", and the token digests the file list, so it
cannot silently cover a different one. None of that proves a human was involved, and the code says
so in as many words.
- **The trace answers the question the tool cannot.** `clearance-ended-the-turn` looks for a
`wikitool.call` between an exit-42 call and the next `prompt.submitted`. That is exactly the
"refused, then confirmed in the same breath" shape all three incidents had, and it is invisible to
any single call. `clearance-was-asked-for` catches a token that no refusal ever issued.
This is the loop this whole page is about, run to its conclusion: a rule that only a trace could
check, a trace that showed it being broken three times, a mechanism rebuilt because no procedural fix
survived - and then the check itself moved to the layer that can actually see it.
**What this still cannot see:** an agent that receives the exit-42 output, does not relay it, and
passes the token anyway *after* the user happens to say something unrelated. `prompt.submitted`
proves a turn boundary, not that the output was relayed or that the reply meant yes. That gap is
recorded rather than papered over. Closing it needs the harness to report what the agent actually
said, which no adapter here does yet.
That gap stopped being hypothetical within the hour. The first agent to receive the new gate - the
one that had just written the paragraph above - answered the user with a file *count* and a pointer
to "the output above". On that harness the user could not see the output: a command's stdout goes
to the agent's context, not to anyone's screen. Nothing in the trace distinguishes that from a
correct relay, and nothing ever will. The response was to fix the half that *is* fixable, the
wording. The message now says "THE USER CANNOT SEE THIS OUTPUT", asks for the paths to be copied
into the reply, and names the near-misses that do not count: a count, a summary, "the output above".
An instruction that conflates printing with showing reads as already satisfied by the text existing.
That is a general lesson about writing for agents, not a detail of this gate.
## Status
| Piece | State |
|---|---|
| Event schema, scrubber, writer | done - `tools/chemenu/telemetry/` |
| `wikitool.call` on every command | done - one hook point in `cli.py`, next to the budget gate |
| `gate.refused` (loop-breaker, iteration budget, mass-update) | done |
| `publish.commit` | done |
| Hook entry point | done - `tools/trace_ingest.py`, mappings for all three hook-capable harnesses |
| Copilot CLI hooks | done - `.github/hooks/wiki-trace.json` |
| VS Code chronicle import | done - `tools/import_chronicle.py` |
| Mistral Vibe hooks | done - `.vibe/hooks.toml`, `.vibe/config.toml` |
| Session header seeded on every trace | done - a trace declares its own `completeness` even without a session hook |
| Scoring L1 + L2 | done - `wikitool eval score` |
| L0 pipeline check | done - in the test suite |
| Claude Code hooks | **partial** (2026-08-28) - `.claude/settings.json` wires `UserPromptSubmit` (the turn boundary `clearance-ended-the-turn` scores against) plus a `permissions.ask` rule on the `--confirm` form of `publish`; `PreToolUse` verified unable to force a prompt over `permissions.allow` |
| Claude Code OTel | not started; not yet verified against the live CLI |
| Agent runner (`eval run`) and L3 | designed below, not built |
## The agent runner, and why it is not here yet
Scoring reads what a session left behind. The missing half is *starting* one: a runner that launches
a harness against a task, several times, and reports a pass rate. What it has to do is no longer
guesswork, because the adapter work settled most of it:
- **Give each run a copy of the repository, not a fixture directory.** `config.ROOT` is derived from
`chemenu/config.py`'s own location, so `wikitool` cannot be aimed at another tree from outside. An
agent case therefore needs a worktree or a clone, where the tool sits inside the tree it edits. (A
*fixture* directory works fine in-process, which is why L0 is a test.)
- **Give each run its own HOME:** `VIBE_HOME`, `COPILOT_HOME`, plus `WIKI_TRACE_DIR` and
`WIKITOOL_SESSION_ID`, so that state, trust decisions and traces cannot leak between runs.
- **Never launch Vibe without `--agent`.** Its programmatic mode falls back to `auto-approve` when no
agent is named, and it does not ask about folder trust. The runner must refuse rather than inherit
that default.
- **Record a run manifest:** harness and version, model, git SHA, and hashes of `AGENTS.md`, the
published skills and the fixture. Without it, a pass rate cannot be compared to anything.
What is missing is not the design but the ability to check it. No provider credentials were
configured on the machine the design was worked out on, so a live run could not be executed, let
alone verified. Writing the runner anyway would repeat exactly the mistake the Vibe adapter avoided:
the hook format there was wrong in the documentation, and only the installed CLI showed it. A runner
written against an unverified mental model of three harnesses would be worse.
Open questions: whether Claude Code has a redirectable config directory (only `--settings` is
documented), and whether the scrubber's own patterns are enough, or a dedicated secret scanner
belongs in the verification step.
Version-model.md → Explanation-Version-model.md
+1 -1
@@ -1,4 +1,4 @@
*Generated from [`docs/version-model.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/docs/version-model.md) at v8.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/explanation/version-model.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/docs/explanation/version-model.md) at v9.0.0 – edit the repository, not this wiki.*
# Why the stack version splits compatibility from migration
Why-gates-are-code.md → Explanation-Why-gates-are-code.md
+20 -12
@@ -1,13 +1,11 @@
*Generated from [`docs/why-gates-are-code.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/docs/why-gates-are-code.md) at v8.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/explanation/why-gates-are-code.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/docs/explanation/why-gates-are-code.md) at v9.0.0 – edit the repository, not this wiki.*
# Why gates are code
Chemenu has five hard limits - the Mass-Update Gate, the Publish-Remote Gate, the Upload Review
Gate, the Guideline Push Gate, and the Iteration Budget Gate - and all five live inside
`tools/wikitool`, not in a
paragraph of instructions an agent reads and follows. The rules themselves, and what to do when
one trips, are in [AGENTS.md § Gates](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/AGENTS.md#gates) and
[instructions/gates.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/instructions/gates.md). This page is only about the design choice
Chemenu's hard limits - the gates [AGENTS.md § Gates](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/AGENTS.md#gates) lists - live inside
`tools/wikitool`, not in a paragraph of instructions an agent reads and follows. The rules
themselves, and what to do when one trips, are in [AGENTS.md § Gates](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/AGENTS.md#gates) and
[instructions/gates.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/instructions/gates.md). This page is only about the design choice
underneath them: why code, and why these mechanisms in particular.
<!-- wikitool:toc -->
@@ -36,8 +34,8 @@ well-written instruction - it's that code doesn't get talked into anything.
## Why different mechanisms, not one
The five gates ask four different kinds of question, and each one's shape follows from what kind
of question it is.
The gates ask different kinds of question, and each one's shape follows from what kind of
question it is.
The Mass-Update Gate asks *is this change too large to publish unreviewed* - a judgment that
varies changeset by changeset, so it clears with a `--confirm` token tied to the specific
@@ -53,7 +51,7 @@ editing `.wikitool-remotes.json` directly, outside the gate's own flow.
The Upload Review Gate asks the Mass-Update Gate's own question - *is this change right?* - at
the opposite end of its size range: one file from a stranger instead of a changeset from the
session's own work. That similarity is exactly why it reuses the same shape (a `--confirm` token
digesting the thing being approved) rather than inventing a fourth one: the two gates differ in
digesting the thing being approved) rather than inventing a new one: the two differ in
*who* produced the change and *how much* of it there is, not in what kind of question either one
is answering, so nothing about the mechanism needed to change - only the boundary it sits behind
did, since the submission lives in a quarantine the ordinary pipeline never reads at all rather
@@ -70,14 +68,24 @@ standing, committed declaration - the capture manifests under `raw/` - and askin
URLs in an allowlist as well would be a second declaration of one fact. What is left to ask is
the per-run question, and that is the one a token answers.
The Iteration Budget Gate asks a fourth kind of question - not "is this instance correct" but
The Rebase-Review Gate asks the one question the others cannot: *does what is arriving collide
logically with what I am about to publish?* `sync` and `publish` reconcile with the remote by
rebasing, and when the incoming commits touch a file this session is also changing, git can
produce a clean textual result out of two changes that contradict each other. Whether they do is
a question of meaning, not of text, so the tool cannot answer it - it can only refuse, and show
the incoming commits, the overlapping files and their diff to whoever can. The answer is different
for every rebase, which is why it takes the token shape: the token digests the exact upstream
state and the overlapping files, so a remote that moved again before the re-run is a new
question, asked again.
The Iteration Budget Gate asks a different kind of question - not "is this instance correct" but
"has this session stopped making progress." That's read from the shape of the call history
itself (call count, repeated identical calls), not from anything about the content of any one
call.
## Exit 42 is a posture, and it outgrew the gates
Those five are the named gates, and they are not the only thing that exits 42 any more. When the
Those are the named gates, and they are not the only thing that exits 42 any more. When the
task-tracker provider layer arrived, it brought a case that looks like a gate from the outside and
is not one: a provider whose API cannot create a project (Super Productivity's local REST API
reads projects but does not write them) raises `HumanInterventionRequired`, and the command prints
Explanation.md
+24
@@ -0,0 +1,24 @@
*Generated from [`docs/explanation/README.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/docs/explanation/README.md) at v9.0.0 – edit the repository, not this wiki.*
# Explanation
Why the stack is built the way it is. These pages are background for humans and agents alike:
read by link or on request, never as instruction, and they carry no normative sentence - a rule
lives in `AGENTS.md` or a `CONTRACT.md`, and a page here only gives its reasons.
- [architecture.md](Explanation-Architecture) - what the stack is made of, and how its parts divide the
work
- [telemetry-and-evaluation.md](Explanation-Telemetry-and-evaluation) - how the stack observes what an
agent did, and how that record turns into a score
- [pipeline-rationale.md](Explanation-Pipeline-rationale) - why the pipeline has four stages
- [ownership-and-templates.md](Explanation-Ownership-and-templates) - why a `.template` split exists, and
why an instance comes only from a release
- [language-boundaries.md](Explanation-Language-boundaries) - why the control plane is English everywhere
and the KB language is a value
- [why-gates-are-code.md](Explanation-Why-gates-are-code) - why the gates are code rather than instruction
- [version-model.md](Explanation-Version-model) - why a version number answers a compatibility question
and a migration question separately
- [knowledge-and-commitment.md](Explanation-Knowledge-and-commitment) - why commitments live in a task
tracker rather than in `kb/`
- [model-and-effort-selection.md](Explanation-Model-and-effort-selection) - which model and effort suit
which kind of session, for the harness that has to choose
Home.md
+32 -533
@@ -1,30 +1,13 @@
*Generated from [`README.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/README.md) at v8.0.0 – edit the repository, not this wiki.*
*Generated from [`README.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/README.md) at v9.0.0 – edit the repository, not this wiki.*
# Chemenu - Personal IT Knowledge Base
A structured, LLM-maintained knowledge base for your personal IT work.
## What is this?
This is **Chemenu** - a pattern for building a personal knowledge base using LLMs.
Instead of just retrieving from raw documents on every query, the LLM **incrementally builds
and maintains a persistent wiki** that compounds over time.
**The key insight:** Knowledge is compiled once and kept current, not re-derived on every question.
**The pages are written in German.** Source material in `raw/` is never touched and is usually
English; the compiled pages under `kb/` are not. What stays English inside them is everything that
is an *identifier* rather than prose - page titles, section headings, wikilink targets, citation
ids, schema enum values, tags, commands, paths and code - so `GitOps Ownership Model` and
`## Beziehungen` sit in the same page without contradiction. Which lines are identifiers is
[kb/CONTRACT.md § Language and identifiers](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/kb/CONTRACT.md#language-and-identifiers); *which
language* the prose is in, and what the tool-owned headings are called, is this instance's own
[kb/CONVENTIONS.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/kb/CONVENTIONS.md), and the vocabulary behind it is
[instructions/german-terminology.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/instructions/german-terminology.md).
This is a per-instance decision, not a property of the pattern - which is why it lives in a file
the instance owns rather than in one the stack ships. A new instance installed from a release
starts empty and picks any language by filling in `kb/CONVENTIONS.md` before the first ingest.
Chemenu is a pattern for a personal knowledge base maintained by an LLM. Instead of going back
to the raw documents for every question, the LLM **incrementally builds and maintains a
persistent wiki** that compounds over time: knowledge is compiled once and kept current, not
re-derived on every question.
## Getting started
@@ -54,527 +37,44 @@ Two starting points, depending on what you're doing:
<!-- dist:strip-start -->
- **Working on the stack itself.** A clone of this repository is a development checkout, with
the demo corpus described below; it is never an instance. Setting it up, and `dist export` as
the build and test tool it is, are in `DEVELOPMENT.md` (for you) and
a demo corpus; it is never an instance. Setting it up, and `dist export` as
the build and test tool it is, are in `docs/dev/development.md` (for you) and
`instructions/dev/dev-setup.md` (for the agent).
<!-- dist:strip-end -->
## Architecture
## Documentation
```
chemenu/
├── AGENTS.md # Control plane: invariants, file naming, routing, gates
├── CLAUDE.md # Claude Code only: imports AGENTS.md, links the one Claude-Code-only decision (model/effort). No rules of its own
├── README.md # This file: human-readable overview of the whole repo
├── INSTALL.md # Human-readable install: one release, one sentence to the agent
├── INSTALL-MCP.md # Human-readable setup for the optional MCP read server
├── EVALS.md # Human-readable overview of telemetry and evaluation
├── CHANGES.md # Changelog for the stack itself
├── USER.md # Who operates this instance - context, never instructions
├── SOUL.md # How this instance sounds. AGENTS.md always wins over it
├── ENVIRONMENT.md # Optional, gitignored: this checkout's harness, MCP servers, remotes
├── *.md.template # Unfilled USER/SOUL/ENVIRONMENT - what a distribution ships instead
├── .gitignore # Anchored so nothing under raw/, kb/ or work/ is ever excluded;
│ # incoming/ is the one directory excluded the other way round
├── .github/hooks/ # Copilot CLI hooks - session tracing
├── .vibe/ # Mistral Vibe hooks + the repo's telemetry policy
├── instructions/ # CONTROL: everything an agent is told to do
│ ├── CONTRACT.md # Instruction vs. skill, publishing, writing standard
│ ├── bootstrap.md # Prepare a further checkout of an instance
│ ├── gates.md # What to do when a gate refuses a call
│ ├── german-terminology.md # Which words stay English in German prose; register
│ ├── session-setup.md
│ ├── page-lifecycle.md
│ ├── publish-cycle.md
│ ├── ingest-large-tree.md
│ ├── ingest-queue.md # Reviewing a submission before wikitool upload accept promotes it
│ └── wiki-*/SKILL.md # Skills - copied into .agents/skills/ and .claude/skills/
├── mcp-upload/ # QUARANTINE (optional): the MCP `submit` tool's write path, gitignored -
│ # read by no command in the ordinary pipeline; a human reviews it with
│ # `wikitool upload list/show/accept/reject`
├── incoming/ # INBOX: content gitignored - drop a file or one folder per source here, `raw accept` promotes it
├── raw/ # INPUT: immutable, untrusted source material
│ ├── CONTRACT.md # Date shard, capture fields, immutability, untrusted content
│ ├── 2026/09/ # Where `raw accept` puts a file: the month it was accepted
│ ├── articles/ # The old type directories: still valid paths, never moved,
│ ├── documents/ # but nothing new is ever routed into them again
│ ├── notes/
│ └── assets/
├── types/ # SCHEMA: the global type surface. Not a collection
│ ├── type-spec.md # Root contract: anatomy, placement, adding a type
│ ├── type-guidance.md # Contract for the *.guidance.md files below
│ ├── entity.md # Entity type config + template (+ .schema.yaml)
│ ├── entity.guidance.md # Its stack-owned authoring prose, shipped verbatim
│ ├── entity.person.md # Subtype template: what `new` scaffolds for entity_type=person
│ ├── concept.md # Concept type config + template (+ .guidance.md, + concept.decision.md)
│ ├── source.md # Source type config + template (+ .guidance.md)
│ ├── comparison.md # Comparison type config + template (+ .guidance.md)
│ ├── project.md # Project (Vorhaben) type config + template, no guidance file
│ ├── instruction.md # Instruction type - lives outside kb/ via `root: repo`
│ └── lint-report.md # Contract-only: describes reports/, owns no directory
├── kb/ # OUTPUT: compiled knowledge. A namespace, not a collection
│ ├── CONTRACT.md # Collections, naming, tone, linking, provenance
│ ├── index.md # Generated catalog *map*: counts and pointers
│ ├── log.md # Generated chronological audit log
│ ├── provenance.md # Generated raw-file reverse index
│ ├── entities/ # COLLECTION.md + INDEX.md + areas below
│ │ ├── codebases/
│ │ ├── systems/
│ │ ├── tools/ # own INDEX.md once past 50 pages
│ │ ├── technologies/
│ │ ├── people/
│ │ └── organizations/
│ ├── concepts/ # COLLECTION.md + INDEX.md + areas below
│ │ ├── architectures/
│ │ ├── patterns/
│ │ ├── protocols/
│ │ ├── workflows/
│ │ ├── decisions/
│ │ └── problems/
│ ├── sources/ # COLLECTION.md + INDEX.md + areas below
│ │ ├── transcripts/
│ │ ├── analyses/
│ │ ├── articles/
│ │ ├── documents/
│ │ ├── notes/
│ │ ├── trackers/
│ │ └── unclassified/
│ ├── comparisons/ # COLLECTION.md - comparison pages, no subtype axis
│ └── gtd/ # COLLECTION.md + INDEX.md + areas below
│ ├── haus/
│ ├── finanzen/
│ └── technik/
├── work/ # WORKSHOP: one directory per multi-session run, tracked
│ └── CONTRACT.md # Run keys, required files, how a run closes
├── reports/ # DERIVED: lint reports, traces, eval scores. Gitignored
│ └── CONTRACT.md
└── tools/ # COMPILER: the wikitool CLI
├── CONTRACT.md # Command records (generated) and index, maintenance schedule
└── README.md # How wikitool is built and how to change it
```
The human documentation lives under `docs/`, sorted by what you want from it:
- [docs/tutorials/](Tutorials) - learning: a guided first walk through an
instance, starting with the installation in [INSTALL.md](Install).
- [docs/how-to/](How-to) - doing: one task per page, for the operator of an
instance.
- [docs/reference/](Reference) - looking up: commands, exit codes, skills, page
types, configuration. The command reference is
[docs/reference/commands/](Reference-Commands); agents use the same records in
[tools/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/tools/CONTRACT.md).
- [docs/explanation/](Explanation) - understanding: why the stack is built the way
it is.
Most days you need three of the how-to guides: [ingest a source](How-to-Ingest-a-source),
[query the wiki](How-to-Query-the-wiki) and [maintain it](How-to-Maintain-the-wiki).
How the parts fit together is [docs/explanation/architecture.md](Explanation-Architecture).
<!-- dist:strip-start -->
Dev-instance-only (see `tools/CONTRACT.md` for how it got here):
```
├── DEVELOPMENT.md # Human-readable: the release workflow (version bump/release/publish/CI)
└── commonplace/ # Vendored, read-only knowledge base
```
The Gitea wiki of this repository shows these pages for the latest release. It is generated from
`docs/` and not edited; how it is built is in [docs/dev/development.md](Dev-Development).
<!-- dist:strip-end -->
A directory under `kb/` is a **collection** exactly when it holds a `COLLECTION.md`; a
subdirectory inside one is an **area** that inherits it, and that is as deep as a page goes -
nothing nests below an area, because the generated catalog reads exactly two path segments
under `kb/` and would fold a deeper page into the area silently (`kb/CONTRACT.md` § Collections
has the rule; `wikitool lint` reports a violation as a hard error).
Which areas a collection has is not chosen per page: a type-spec's `layout:` maps its subtype
field onto directories, and `wikitool new` writes the page straight into the one its subtype
names. That is also what makes the catalog's shard threshold do anything - `index rebuild`
splits **per area**, so a collection with no areas keeps one table however large it grows.
`wikitool lint` reports such a collection once it is past the threshold, as a recommendation
rather than an error, together with the split its subtype field would produce; it stays quiet
when the split would not actually help. `kb/comparisons/` is the worked example of a collection
that stays flat - it has no subtype field for a `layout:` to key on at all. A *lopsided* subtype
field is a different case and is fixed rather than left flat: `kb/sources/` looked lopsided only
because `source_type` had a schema `default:` that the compiler applied whenever nobody chose a
value, and once that was removed and the pages reclassified it split into six real areas.
`COLLECTION.md` never appears outside `kb/` - the other layers carry a `CONTRACT.md` or a root
type-spec instead. A stage may carry both a `README.md` and a `CONTRACT.md`: they have different
readers. The README is for humans working *on* that layer, the contract is what binds an agent
working *with* it.
## How to Use
### Adding Knowledge (Ingest)
1. Drop a file into `incoming/` - directly, no classification to make. Files that belong
together go into one folder there instead: a folder is one source, accepted whole with
its structure kept. Everything past that (the destination in `raw/`, which is a `YYYY/MM`
shard of the day it was accepted, and whether several files of one source get bundled)
is computed by `tools/wikitool raw accept`, never chosen by hand
2. Tell the LLM: `Ingest incoming/my-article.md` - or just `Ingest`, which takes the oldest
entry waiting in `incoming/` (`tools/wikitool raw pending` lists them). It will ask you
two things before promoting: how faithful the capture is (`fidelity`) and what the material may claim
about its subject (`authority`). Both are recorded once and never guessed - they are
knowable now and unrecoverable later
3. The LLM will:
- Promote it into `raw/` with `raw accept`
- Read and summarize the source
- Create a source page in `kb/sources/`
- Create or update relevant entity pages
- Create or update relevant concept pages
- Add cross-references between everything
- Rebuild the catalog and append to `kb/log.md`
A document can also arrive from outside, through the MCP server's optional `submit` tool
(see [INSTALL-MCP.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/INSTALL-MCP.md)): it lands in `mcp-upload/`, not `incoming/`, and a human
reviews and promotes it with `wikitool upload accept` before step 1 above applies - see
[instructions/ingest-queue.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/instructions/ingest-queue.md).
A web page needs no download of your own: tell the LLM `Ingest https://example.org/post`, and
`tools/wikitool raw fetch` puts the page into `incoming/` - the HTML exactly as received, plus a
text derived from it with a header recording where and when it was fetched. Behind a paywall or a
login, save the page from your browser into `incoming/` (HTML only) instead; the LLM derives the
same text from that file with `raw fetch --html`. See [raw/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/raw/CONTRACT.md).
Documentation that lives in a git repository is captured, not copied: `tools/wikitool raw capture
<repo-url> --ref main --path 'docs/**/*.md' --name <bundle> ...` writes the matching files, byte
for byte, into `incoming/<bundle>/` together with a `_capture.json` manifest naming the repository,
the ref rule and the commit. Later, `tools/wikitool raw status` tells you which captured bundles
have fallen behind their repository, file by file; `raw capture --update` and `raw accept
--replaces-bundle` take the new edition in as a whole. You need not run any of that yourself:
tell the LLM `Update the captured repositories`, and it checks them, takes one changed bundle per
run into the wiki and says how many are still behind. Git uses your own keys and credential
helpers - nothing is stored in the repository.
The same repositories can get something back: the wiki's guidelines, as one generated
`GUIDELINES.md` in their root. Which pages count as guidelines is your decision, written down in
`kb/CONVENTIONS.md` (§ Guidelines for other repositories); `tools/wikitool export guidelines
--tag guideline` prints the file as it would be written. A repository takes part by carrying a
`GUIDELINES.md` whose first line is `<!-- wikitool:export kind=guidelines -->` - commit that one
line there, and point the repository's own `AGENTS.md` at the file (Claude Code:
`@GUIDELINES.md`). A hand-written `GUIDELINES.md` is never touched. Then tell the LLM
`Roll out the guidelines to the captured repositories`: it runs `export guidelines --push`, which
stops before pushing anything (the **Guideline Push Gate**) and shows you, per repository, what
would change - the LLM puts every diff in front of you, and only your approval pushes one commit
per repository, changing only that file, straight onto its branch.
### Querying Knowledge
Ask questions naturally:
- "What projects use MQTT?"
- "Show me the architecture of HA Integration"
- "Compare gdeploy and plugnburn-edl"
- "What decisions were made about E3DC integration?"
The LLM will search the wiki, synthesize an answer, and cite sources.
### Maintaining Knowledge (Lint)
Periodically run: `Lint the wiki`
The LLM will:
- Run `tools/wikitool lint` for a deterministic structural + provenance scan
(broken wikilinks, orphan pages, index drift, schema gaps, uncovered raw
files, citation/frontmatter drift)
- Check for contradictions (semantic judgment)
- Find stale claims
- Identify orphan pages and missing cross-references
- Rebuild `kb/index.md` and `kb/provenance.md`, append to `kb/log.md`
- Generate a report
See the [Maintenance](#maintenance) section below for the full schedule and
command reference.
### Reviewing Commitments (Weekly Review)
Say: `Run the weekly review`
Knowledge and commitments keep different clocks, so they live in different
places. A page under `kb/gtd/` is one committed initiative's durable memory -
its goal, who is involved, where it stands, why it is worth doing - and it never
summarizes the task list. The open items live in a task tracker that owns them,
configured per checkout in `.wikitool-tasks.json` (see
[INSTALL.md](Install) § Konfiguration; no tracker configured is a valid
state, and the pages work without one).
Nothing syncs between the two. `tools/wikitool review` joins them at read time
over the project name and prints what needs a decision: initiatives with no next
action, waiting-fors past their follow-up date, tracker projects with no page,
active pages with no open loop, someday items gone stale. It stores nothing -
not even a report file. The `gtd-weekly-review` skill then walks the findings with
you and turns each one into a decision; `tools/wikitool new project` is what
gives a new initiative its page and its tracker project under one name, and
`tools/wikitool task new` files a single open item into the tracker - the
commitment half of a source that carries both something to know and something
to do, with no page of its own. `tools/wikitool task list` reads a project's
open items back with their tracker id, and `tools/wikitool task close --id`
marks one done - never deletes it - closing the loop the same source-driven
way `task new` opened it, or the way the weekly review proposes it for a
`waiting_overdue`/`someday_stale` finding once you confirm.
Why the split runs this way, rather than syncing the two:
[docs/knowledge-and-commitment.md](Knowledge-and-commitment).
## Entity Types
Entities are subtyped as codebase, system, tool, technology, or person, and each subtype has
its own directory under `kb/entities/`. The authoritative list - and where each one is
written - is declared by the type-spec, so ask the tool rather than a table here:
```bash
tools/wikitool types list
tools/wikitool types describe entity
```
## Workflows
### For You (Human)
1. **Curate sources** - Drop files you want processed into `incoming/` - directly, or one folder
per source
2. **Ask questions** - Query the wiki naturally
3. **Review changes** - Check `kb/log.md` and `kb/index.md`
4. **Direct the LLM** - Guide it on what to emphasize or investigate
5. **Browse in Obsidian** - Open the wiki directory in Obsidian for visualization
### For the LLM
`AGENTS.md` is the cross-cutting schema/policy; the step-by-step procedures
themselves live as independently-discoverable skills under `.agents/skills/`
(mirrored to `.claude/skills/` for Claude Code via `tools/wikitool instructions sync`):
| Skill | Purpose |
|-------|---------|
| `wiki-ingest` | Process a new source into the wiki: read it, discuss its content and any commitment with the user, promote it from `incoming/` into `raw/`, then source summary, entity/concept pages, cross-references, index/log, publish |
| `wiki-query` | Answer a question from the compiled wiki; read-only, can optionally file a valuable answer back as a new page |
| `wiki-lint` | Health-check the wiki: structural scan, raw coverage, semantic review |
| `wiki-manage` | Create a new entity/concept/source/comparison page, or update an existing page with new information |
| `wiki-status` | Read-only snapshot: page counts, orphans, uncovered raw files, most-connected pages |
| `gtd-weekly-review` | Turns `wikitool review`'s findings into decisions and page updates - the GTD weekly review |
Each skill's underlying mechanical work (frontmatter, cross-references, index/log,
decay math, publishing) is delegated to `tools/wikitool` - never hand-edited.
## Your first ingestion
### First Steps
1. Read `AGENTS.md` - the control plane (invariants, routing, gates) - then the stage contract
for whichever of `raw/`, `types/` or `kb/` you are working in, and, inside `kb/`, the
`COLLECTION.md` of the collection you are writing to
2. Add your first source to `incoming/`
3. Run: `Ingest <your-file>`
4. Review the created pages
5. Ask your first query
### Example First Ingestion
```bash
# Add a source
cp ~/Downloads/my-notes.md incoming/my-notes.md
# Tell the LLM to process it
# (in your LLM agent)
Ingest incoming/my-notes.md
```
## Tips
### Naming
- Use human-readable titles with spaces for files: `Hybrid Search.md`, not kebab-case
- Use singular for entities: `HA Integration.md` (not `HA Integrations.md`)
- Use wikilinks matching the file name exactly: `[[Entity Name]]`
- A title is a file name, so it has to work on Windows and macOS as well: no `< > : " / \ | ? *`,
no reserved names such as `CON` or `Index`, no trailing dot, and no second page whose title
differs only by case. `wikitool new` and `wikitool rename` refuse such titles, `wikitool lint`
reports existing ones, and `kb/CONTRACT.md` § Titles are identifiers has the full rule
- A file's whole path below the instance root stays at 160 characters or fewer, so a Windows
checkout works without long paths: `new`, `rename`, `move` and `raw accept` refuse a longer
one, and `lint` reports existing ones as Long Paths (advisory; `wikitool rename` is the fix)
- **Titles follow the subject's own established name, not the wiki's language.** `Act Runner` and
`GitOps Ownership Model` keep theirs. A title is the only identifier a page has - it also lives
in every wikilink and citation id pointing at it - so translating one is a rename, never an
edit: `tools/wikitool rename`, per `instructions/page-lifecycle.md`
### Organization
- Start with a few broad categories, refine as needed
- Don't over-organize early - let structure emerge
- Use tags for cross-cutting concerns
### Quality
- The LLM will maintain quality standards from `AGENTS.md`
- Review changes periodically
- Flag issues to the LLM
## Maintenance
The wiki is kept healthy by deterministic `tools/wikitool` commands, run by the
LLM (via the skills above). The schedule - which task runs how often, and with
which command - lives in [`tools/CONTRACT.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/tools/CONTRACT.md#maintenance-schedule),
next to the command reference it depends on, so the two cannot drift apart.
The notes below explain the three parts of it that need more than one line.
**Provenance.** Every fact should trace back to a raw file. Source pages
declare their backing `raw_files:`; entity/concept pages declare `provenance:`
(`sourced`/`general`/`mixed`) and cite specific claims inline with a
`[^cite-id]` footnote (`tools/wikitool cite add` mints the id and definition;
placing the marker in the prose is still manual). `tools/wikitool sources
coverage` finds raw files with no source page yet, `sources trace` answers
"where did this come from?" in either direction, and `sources rebuild-index`
regenerates the reverse index at `kb/provenance.md`. `lint` cross-checks that
citations and frontmatter `sources:` lists agree, and hard-errors on any
leftover pre-migration `^[[...]]` marker.
**Git automation.** `tools/wikitool publish` stages everything, commits with
an auto-generated changed-file list, and pushes to `origin/main` in one step -
never run raw `git commit`/`git push` for wiki changes. Without `--no-push` it stops with
exit 1 before committing when the remote is not configured or cannot be reached; a local-only
instance passes `--no-push` on every call. Publishes touching
≥10 files exit **42** (the **Mass-Update Gate**) - a distinct "a human must see
this" code, not an error - printing the full file list and the
`--confirm <token>` line that publishes it. The token digests that file list,
so a clearance never carries to a changeset the user did not see.
**Guideline push.** `tools/wikitool export guidelines --push` exits **42** the same way (the
**Guideline Push Gate**) before it writes `GUIDELINES.md` into any captured repository, printing
every target's status and diff and the `--confirm <token>` line. The token digests each target's
branch tip and the file, so a moved branch or an edited guideline asks again.
**Iteration/cost limits.** Every `tools/wikitool` call is checked against a
hard, code-enforced per-session budget before it runs (default: 60 calls, or
3 identical calls in a row) - not just a prompt instruction to stop. Past the
limit, the command refuses to run until a human approves continuing with
`--override-budget`. `budget status` stays readable at all times; `budget
reset` clears the counter and therefore needs `--yes` of its own. See
AGENTS.md's "Gates" section.
## Telemetry and evaluation
In this checkout, every `wikitool` call appends an event to
`reports/telemetry/<session>/trace.jsonl`, and the hook files under `.github/hooks/` and
`.vibe/` add what the agent did between those calls. Nothing leaves the machine: `reports/` is
gitignored and no exporter is configured.
**A distributed instance records nothing unless it asks to.** The default follows the
installation form - on for a git clone of this repo, where the traces are the stack's own
measuring instrument, off for an instance installed from a release, where nobody ordered
telemetry. Two
quantity caps apply either way: 5 MiB per session trace, and 250 session directories.
`wikitool doctor` reports which state a checkout is in and why; EVALS.md § "Whether it runs at
all" has the precedence rules and the opt-in file.
That record is what makes it possible to ask how a session *worked*, not just what it left
behind:
```bash
tools/wikitool eval sessions # which sessions have a trace
tools/wikitool eval score # score this session
```
A score has two halves - the structural state of the tree, from lint's own checks, and
trajectory rules over the trace, which catch things no unit test can: a refused call repeated
unchanged, a gate flag passed without that gate having refused anything.
[`EVALS.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/EVALS.md) is the full picture: the event contract, what each agent harness can and
cannot report, what is redacted, and why there is deliberately no LLM judge yet.
## Tools Integration
### wikitool (deterministic CLI)
Mechanical wiki operations - never hand-edited by the LLM - are handled by
`tools/wikitool`: scaffolding pages, renaming and deleting them, cross-references,
index/log/provenance regeneration, structural linting, and publishing.
Every command carries one data record - synopsis, properties, copyable examples,
each exit cause with what to do about it, prohibitions, and notes on its
behaviour - kept next to its code. `tools/wikitool <command> -h` prints it,
`tools/wikitool -h` prints a one-line index of all of them, a command that fails with
exit 1 prints the record's reactions on stderr right under its `ERROR` line, and
[`tools/CONTRACT.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/tools/CONTRACT.md) holds a generated copy together with the
maintenance schedule; `tools/wikitool docs verify` checks that copy and every
command's flags against the CLI in both directions. [`tools/README.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/tools/README.md)
is the other half: how the CLI is built and how to add a command. `AGENTS.md`
holds the invariants that say when each command is mandatory.
```bash
tools/wikitool -h
tools/wikitool <command> -h
```
<!-- dist:strip-start -->
Dev-instance-only: extending `tools/wikitool`, the type schema, or the instruction/skill layer
itself is a separate session type with its own rules, covered by the `stack-dev`,
`stack-build` and `stack-close` skills nested under `instructions/dev/` (never present in a
distributed instance - `tools/CONTRACT.md` explains why).
<!-- dist:strip-end -->
### MCP read server (optional)
The terminal is not the only way in. `tools/chemenu/mcp/` serves the same wiki read-only over
MCP - `search`, `types`, `describe_type`, `lint` and `status` - so a consumer that is not a
shell on this machine can ask the same questions and get the same answers. Literally the same:
the CLI and the server are two adapters over one core, and a golden test holds their output
together rather than trusting that it agrees.
There is no tool that writes, and not because one is filtered out of a list: the server imports
nothing under `chemenu/commands/`, so `new`, `publish` and the rest are unreachable from it.
Every answer carries the commit it was computed from, so a checkout that has fallen behind
produces a visibly stale answer instead of a confident wrong one.
```bash
tools/.venv/bin/pip install -r tools/requirements-mcp.txt
WIKI_TRACE=0 tools/.venv/bin/python -m chemenu.mcp # stdio
WIKI_TRACE=0 tools/.venv/bin/python -m chemenu.mcp \
--transport streamable-http --host 0.0.0.0 --port 8000 # deployed
```
The dependency is deliberately not in `requirements.txt`: an instance that only uses the CLI
should not have to install a web stack to do it.
Installing it, wiring a client to it, and putting authentication in front of it:
[`INSTALL-MCP.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/INSTALL-MCP.md), for humans. The agent-side operating procedure - keeping the
checkout current, and what a stale answer looks like - is
[`instructions/mcp-read-server.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/instructions/mcp-read-server.md).
### Obsidian
Open this directory in Obsidian for:
- Graph view of connections
- Easy navigation with wikilinks
- Plugins: Dataview, Marp, etc.
### Git
This is a git repo. Use it for:
- Version history
- Branching for experiments
- Collaboration
### Search
`tools/wikitool search "<text>"` searches `kb/` directly - by text, or by frontmatter with
`--field entity_type=system` or `--field '!sources'`. It is read-only and is the one
command not counted against the session budget, because looking before acting is the habit
worth encouraging.
For browsing rather than searching, `kb/index.md` is the catalog map and each collection
carries its own `INDEX.md`.
## IT-Specific Features
This wiki is tailored for IT work with:
- **Entity types** specific to software development and systems
- **Relationship types** like `hängt ab von`, `verwendet`, `implementiert` - the vocabulary is in
[kb/CONVENTIONS.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/kb/CONVENTIONS.md), because it is this instance's rather than the stack's
- **Templates** per page type, and per subtype where its pages need a shape of their own - a person, a decision record (ADR)
- **Guidelines** for documenting technical decisions
- **Cross-reference patterns** for code and architecture
## Files Created Automatically
The LLM will create and maintain:
- `kb/index.md` - Always up-to-date catalog
- `kb/log.md` - Complete audit trail
- `kb/provenance.md` - Raw-file reverse index
- Source pages in `kb/sources/`
- Entity pages in `kb/entities/`
- Concept pages in `kb/concepts/`
- Comparison pages in `kb/comparisons/`
- Project (Vorhaben) pages in `kb/gtd/`
- Lint reports, session traces and eval scores in `reports/` (gitignored)
Once the instance runs, [From your first source to your first answer](Tutorials-First-source-to-first-answer)
takes a sample document that ships with it through an ingest and a cited answer, step by step.
## Changelog
Changes to the wiki stack (schema, skills, `wikitool`, READMEs) are tracked in
[`CHANGES.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/CHANGES.md), not in an inline version history here.
[`CHANGES.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/CHANGES.md), not in an inline version history here.
## Resources
@@ -588,12 +88,12 @@ Chemenu is dual-licensed, because it is two things in one repository.
| Half | Licence | File |
|------|---------|------|
| The stack — `tools/`, `types/` | GNU AGPL-3.0-or-later | [LICENSE](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/LICENSE) |
| The content — `kb/`, `raw/`, `instructions/`, the `CONTRACT.md` layer, and the prose documents at the root | CC-BY-4.0 | [LICENSE-CONTENT](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/LICENSE-CONTENT) |
| The stack — `tools/`, `types/` | GNU AGPL-3.0-or-later | [LICENSE](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/LICENSE) |
| The content — `kb/`, `raw/`, `instructions/`, the `CONTRACT.md` layer, and the prose documents at the root | CC-BY-4.0 | [LICENSE-CONTENT](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/LICENSE-CONTENT) |
The boundary between the two is not a list someone maintains by hand: it is the
file plan `tools/wikitool dist export` already computes, so it cannot drift out
of agreement with what actually ships. [NOTICE](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/NOTICE) states that, and carries
of agreement with what actually ships. [NOTICE](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/NOTICE) states that, and carries
the attribution for the vendored [Commonplace](https://github.com/zby/commonplace)
research base.
@@ -606,4 +106,3 @@ Affero variant.
If you set up an instance and later publish it, keep `LICENSE`, `LICENSE-CONTENT`
and `NOTICE` in the tree. `dist export` puts them there and refuses to build a
distribution without them, so under normal use this takes no effort.
How-to-Browse-with-obsidian.md
+56
@@ -0,0 +1,56 @@
*Generated from [`docs/how-to/browse-with-obsidian.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/docs/how-to/browse-with-obsidian.md) at v9.0.0 – edit the repository, not this wiki.*
# Browse the wiki in Obsidian
The wiki is a folder of Markdown files joined by `[[wikilinks]]`, so any Markdown editor can read
it. Obsidian adds a graph of the links between pages, which makes it a good way to look around.
## What you prepare
Open the instance's root directory as an Obsidian vault. Obsidian keeps its own settings in
`.obsidian/`. That folder is personal state: it is never shipped with a release, and it has no
effect on the wiki.
What you get:
- the graph view of how pages connect;
- navigation by wikilink, the way the agent writes them;
- plugins such as Dataview, which can query page frontmatter, or Marp for slides.
## Reading, not writing
Browse and read freely. Change pages through the agent, though, not by editing them in Obsidian. A
page carries generated frontmatter, cross-references and footnote citations that `tools/wikitool`
keeps consistent across the whole wiki. A hand edit can quietly break them. Renaming or moving a
file in Obsidian is the clearest case: a title is the page's only identifier, so it also appears in
every wikilink and citation that points at the page. A rename goes through
`tools/wikitool rename`, which updates every reference. The full procedure is
[instructions/page-lifecycle.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/instructions/page-lifecycle.md).
Generated files - `kb/index.md`, `kb/log.md`, `kb/provenance.md` and every `INDEX.md` - are
rebuilt by the tool and never edited by hand.
## How pages are named
Knowing the naming rules makes the vault easier to find your way in:
- **Titles are human-readable names with spaces:** `Hybrid Search.md`, not kebab-case. A wikilink
matches the file name exactly: `[[Hybrid Search]]`.
- **Entities are singular:** `HA Integration.md`, not `HA Integrations.md`.
- **A title follows the subject's own established name, not the wiki's language.** `Act Runner`
and `GitOps Ownership Model` keep their names in a German wiki too. Translating a title is a
rename, not an edit.
- **A title works on every operating system.** It contains none of `< > : " / \ | ? *`, is not a
reserved name such as `CON` or `Index`, does not end in a dot, and does not differ from another
page's title by case alone. A file's whole path below the instance root stays at 160 characters
or fewer, so a Windows checkout works without long paths. `wikitool new` and `wikitool rename`
refuse titles that break these rules, and `wikitool lint` reports existing ones.
[kb/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/kb/CONTRACT.md) has the full rule.
## Letting the structure grow
- Start with a few broad categories and refine as the material arrives. Don't over-organise early.
- Use tags for concerns that cut across collections.
- Review what the agent changed from time to time - `kb/log.md` lists every operation - and tell it
when something is off. It keeps to the standards in `AGENTS.md` and the contracts, but you are the
one who notices when a page misses the point.
How-to-Capture-repository-docs.md
+61
@@ -0,0 +1,61 @@
*Generated from [`docs/how-to/capture-repository-docs.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/docs/how-to/capture-repository-docs.md) at v9.0.0 – edit the repository, not this wiki.*
# Capture documentation from a git repository
Documentation that lives in a git repository is captured, not copied by hand. A capture records the
repository, the ref rule and the commit it came from. Later, the wiki can tell which captured
material has fallen behind its repository, and take in the new edition as a whole.
## What you prepare
Access to the repository from this machine. Git uses your own keys and credential helpers, and
nothing is stored in the instance. Then decide three things:
- the repository URL;
- the ref to follow, usually a branch such as `main`;
- which files count, as a glob, for example `docs/**/*.md`.
## Taking a repository in for the first time
You can run the capture yourself, or name the three values to the agent and let it run the
command:
```bash
tools/wikitool raw capture <repo-url> --ref main --path 'docs/**/*.md' --name <bundle>
```
This writes the matching files, byte for byte, into `incoming/<bundle>/`, together with a
`_capture.json` manifest naming the repository, the ref rule and the commit. From there it is an
ordinary source: `Ingest incoming/<bundle>` ([Ingest a source](How-to-Ingest-a-source)), with the same
questions and the same gate.
## Keeping it current
```bash
tools/wikitool raw status
```
This lists every captured bundle that has fallen behind its repository, file by file, marking each
file as added, modified or deleted. It only reads.
To bring them in, say:
```text
Update the captured repositories
```
The agent checks every captured bundle. It takes one changed bundle per run into the wiki, replacing
the old edition as a whole, and tells you how many are still behind. Run it again for the next one.
`raw capture --update` and `raw accept --replaces-bundle` are the commands underneath. What each one
does is in the [command reference](Reference-Commands-Raw-material-and-uploads).
## Where the agent stops
The same places as any ingest: the `fidelity`/`authority` questions, the discussion of what
matters, and the Mass-Update Gate when it publishes. When one repository cannot be reached, the
agent reports the error and carries on with the others. A failure in one never stops the run.
## Related
- [Export guidelines](How-to-Export-guidelines) - sending the wiki's guidelines back into the same
repositories.
How-to-Export-guidelines.md
+56
@@ -0,0 +1,56 @@
*Generated from [`docs/how-to/export-guidelines.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/docs/how-to/export-guidelines.md) at v9.0.0 – edit the repository, not this wiki.*
# Export guidelines to captured repositories
The repositories the wiki captures documentation from can get something back: the wiki's
guidelines, as one generated `GUIDELINES.md` in each repository's root, which that repository's own
agents then read.
## What you prepare
- **Decide which pages count as guidelines.** That decision is yours, and it is written down in
`kb/CONVENTIONS.md` § Guidelines for other repositories. Usually it is a tag. To see the file
exactly as it would be written, without touching any repository:
```bash
tools/wikitool export guidelines --tag guideline
```
- **Opt each repository in, by hand, once.** A repository takes part only if it carries a
`GUIDELINES.md` whose first line is exactly:
```markdown
<!-- wikitool:export kind=guidelines -->
```
Commit that one line there. Then point the repository's own `AGENTS.md` at the file (for Claude
Code: `@GUIDELINES.md`). A `GUIDELINES.md` without that first line is hand-written, and the export
never touches it.
The repository has to be one the wiki has captured
([Capture documentation from a git repository](How-to-Capture-repository-docs)). That is how the export
knows where the repository lives. Its capture also has to follow a branch: a repository captured by
a tag pattern is skipped, because a tag has no branch to commit onto.
## What you say
```text
Roll out the guidelines to the captured repositories
```
## Where the agent stops
The agent runs `export guidelines --push`, which stops before pushing anything. This is the
**Guideline Push Gate** (exit 42). For every repository, it prints the status, the diff it would
write and a `--confirm <token>` line. The agent puts every diff in front of you. Only your approval
pushes, and then only one commit per repository, which changes only `GUIDELINES.md` and goes
straight onto its branch.
The token is a digest of each target's branch tip and of the generated file. If a branch has moved
or a guideline was edited since you looked, the gate asks again rather than pushing something you
did not see.
## Related
- [Guideline export commands](Reference-Commands-Guideline-export)
- [Why the gates are code](Explanation-Why-gates-are-code)
How-to-Ingest-a-source.md
+93
@@ -0,0 +1,93 @@
*Generated from [`docs/how-to/ingest-a-source.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/docs/how-to/ingest-a-source.md) at v9.0.0 – edit the repository, not this wiki.*
# Ingest a source
Turn a file, a folder or a web page into wiki pages: a source page that summarises it, and the
entity and concept pages it touches, cross-referenced and published. The agent does the work, using
the `wiki-ingest` skill ([skill catalogue](Reference-Skills)). This page covers what you hand
it and where it waits for you.
## What you prepare
Put the material into `incoming/` at the root of the instance. You don't have to classify it or
pick a directory:
- **One file** goes in directly: `incoming/my-article.md`.
- **Several files that belong together** go into one folder, `incoming/<name>/`. The folder is one
source. It is accepted whole, and its structure is kept.
- **A web page** needs no download of your own. Name the URL to the agent, and
`tools/wikitool raw fetch` puts two files into `incoming/`: the HTML exactly as received, and a
text derived from it with a header recording where and when it was fetched.
- **A document that is not text** - a PDF, a Word, PowerPoint or Excel file, a mail (`.eml` or
Outlook `.msg`) or a ZIP archive - is
put into `incoming/` as it is. `tools/wikitool raw derive incoming/<file>` turns it into a folder with the
untouched original and a readable text beside it, with a header saying how it was made. The
folder is accepted whole; the agent runs this itself when it meets such a file. An Excel file's
sheets land beside the text as tab-separated files, with their formulas listed apart. A ZIP is
unpacked into a folder instead, and a mail's attachments land beside it. A scanned PDF has no
text to derive - the tool warns, and there is no OCR.
- **A page behind a paywall or a login** is saved from your browser into `incoming/`, HTML only. The
agent derives the same text from that file with `raw fetch --html`.
Where a file ends up in `raw/` is computed by `tools/wikitool raw accept`: a `YYYY/MM` shard for the
day it was accepted. So is whether several files of one source get bundled. You never choose either.
`incoming/` itself is gitignored, so whatever waits there is never published by accident.
## What you say
```text
Ingest incoming/my-article.md
Ingest https://example.org/post
Ingest
```
A bare `Ingest` takes the oldest entry waiting in `incoming/`. `tools/wikitool raw pending` lists
what is waiting and names the entry the next bare `Ingest` will take.
## Where the agent asks you
- **Before promoting the file into `raw/`**, the agent asks how faithful the capture is
(`fidelity`) and what the material may claim about its subject (`authority`). Both are recorded
once and never guessed: you know the answers now, and nobody can recover them later.
[raw/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/raw/CONTRACT.md) defines the values.
- **Before writing pages**, the agent presents the key takeaways and asks which points matter most
and what to emphasise. The pages follow your answer.
- **When it publishes**, a source that touches ten or more files stops at the Mass-Update Gate
(exit 42). The generated indexes, `kb/log.md` and `kb/provenance.md` do not count, so a small
source goes straight through. The agent shows you the file list and the `--confirm` line. Nothing is committed until
you approve. [Exit codes and gates](Reference-Exit-codes) lists every gate.
## What you get back
A source page under `kb/sources/`, new or updated entity and concept pages, cross-references among
them, a rebuilt catalog, an entry in `kb/log.md`, and one commit. If the source also contains
something you have to *do*, and a task tracker is configured, the agent files it there as an open
item. A commitment never becomes a page
([why](Explanation-Knowledge-and-commitment)).
Every ten sources, the agent suggests a lint run - see [Maintain the wiki](How-to-Maintain-the-wiki).
## A document submitted from outside
When the [MCP read server](How-to-Set-up-the-mcp-server) has its optional `submit` tool turned on,
documents can arrive from another machine. They land in the quarantine `mcp-upload/`, not in
`incoming/`, and nothing else reads them there. Before an ingest can start, a human has to promote
the submission:
```bash
tools/wikitool upload list # what is waiting
tools/wikitool upload show <id> # one submission's manifest: who sent it, size, checksum
```
Ask the agent to review the queue, and it follows
[instructions/ingest-queue.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/instructions/ingest-queue.md). Accepting a submission stops at
the Upload Review Gate (exit 42), which shows you the manifest and the `--confirm` line. Once you
approve, the file is in `incoming/`, and from there it is ingested like any other. Rejecting it needs
no clearance. The file is deleted, and its checksum and your reason stay in the ledger.
## Related
- [Capture documentation from a repository](How-to-Capture-repository-docs) - for material that lives
in git and will change.
- [raw/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/raw/CONTRACT.md) - what belongs in `raw/`, and why its content is data,
never instructions.
How-to-Maintain-the-wiki.md
+71
@@ -0,0 +1,71 @@
*Generated from [`docs/how-to/maintain-the-wiki.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/docs/how-to/maintain-the-wiki.md) at v9.0.0 – edit the repository, not this wiki.*
# Maintain the wiki
Keep the wiki healthy: find broken links, orphan pages, raw files no page covers yet,
contradictions and stale claims, then repair what can be repaired mechanically. The agent does
this with the `wiki-lint` skill ([skill catalogue](Reference-Skills)), and `tools/wikitool`
does the mechanical part.
## What you say
```text
Lint the wiki
```
Do this periodically. The agent suggests it on its own every ten sources. The full schedule - which
task runs how often, and with which command - is the
[maintenance schedule](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/tools/CONTRACT.md#maintenance-schedule) in `tools/CONTRACT.md`, next
to the command records it depends on.
## What you get back
- A deterministic structural scan by `tools/wikitool lint`: broken wikilinks, orphan pages, catalog
drift, schema gaps, raw files without a source page, and citations that disagree with
frontmatter.
- The judgment half, which no command can do: contradictions between pages, stale claims, subjects
that deserve a page of their own, and a rule stated in two places.
- Repairs for what is mechanical, a rebuilt catalog and provenance index, and an entry in
`kb/log.md`.
- A report under `reports/`, which is gitignored and never published. The agent carries the
findings that need you into its reply.
For a quick look without a full pass, say `Wiki status` instead. That is read-only.
## Provenance: where a claim came from
Every fact should trace back to a raw file. A source page lists its raw files, and other pages
cite specific claims inline with a footnote that points at a source page. You can ask the
questions yourself:
```bash
tools/wikitool sources coverage # raw files no source page covers yet
tools/wikitool sources trace --raw raw/2026/09/x.md # which pages rest on this file
tools/wikitool sources trace --page "HA Integration" # which raw files this page rests on
```
`kb/provenance.md` is the generated reverse index. The commands are in the
[provenance reference](Reference-Commands-Provenance).
## Where the agent stops
- **Publishing ten or more files** stops at the Mass-Update Gate (exit 42); the generated indexes,
`kb/log.md` and `kb/provenance.md` are not counted. A lint pass that repairs
many pages does this routinely. The agent shows you the full file list and the `--confirm` line,
and publishes only after you approve. The token is a digest of that file list, so your approval
never covers a changeset you did not see.
- **The call budget.** Every `wikitool` call counts against a per-session budget: 60 calls, or 3
identical calls in a row. Past it, calls are refused. The agent stops and summarises where it is.
If you want it to go on, tell it so. Only then may it pass `--override-budget` on the command it
needs. A refusal usually means the task should be split, not pushed through.
- **No reachable remote.** Without `--no-push`, `publish` refuses with exit 1 before committing when
the remote is not configured or cannot be reached. An instance that is meant to stay local passes
`--no-push` on every publish.
Every gate, and which command it belongs to: [exit codes and gates](Reference-Exit-codes).
Why they are enforced in code: [why-gates-are-code.md](Explanation-Why-gates-are-code).
## Related
- [Troubleshoot](How-to-Troubleshoot) - when `doctor` or a command reports a problem with the instance
itself rather than with its pages.
How-to-Query-the-wiki.md
+52
@@ -0,0 +1,52 @@
*Generated from [`docs/how-to/query-the-wiki.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/docs/how-to/query-the-wiki.md) at v9.0.0 – edit the repository, not this wiki.*
# Query the wiki
Ask the wiki a question and get an answer built from its compiled pages, with the pages and sources
it came from. The agent answers using the `wiki-query` skill
([skill catalogue](Reference-Skills)), which only reads unless you ask it to keep an answer.
## What you say
Ask the way you would ask a colleague:
```text
What projects use MQTT?
Show me the architecture of HA Integration
Compare gdeploy and plugnburn-edl
What decisions were made about E3DC integration?
```
## What you get back
An answer that names the wiki pages it came from and the sources behind them. Where the wiki has no
source for a claim, the agent says so instead of filling the gap. An answer it cannot back is never
written into the wiki either.
If an answer is worth keeping - a comparison you will want again, a synthesis across several pages -
the agent proposes filing it back as a page. It names the pages it would create before it creates
any, and nothing is written unless you agree.
## Searching yourself
The agent searches with the same command you can run:
```bash
tools/wikitool search "<text>"
tools/wikitool search --field entity_type=system
tools/wikitool search --field '!sources' # pages without a sources: list
```
`search` covers all of `kb/`, by text, by frontmatter or both. Each hit carries the page's path and
full title, so you can open it directly. It only reads, and it does not count against the
session's call budget, because looking before acting is the habit worth encouraging. The predicate forms are in the
[command reference](Reference-Commands-Finding-and-checking).
For browsing rather than searching, `kb/index.md` is the catalog map, and each collection carries
its own `INDEX.md`. All of these are generated. Read them, but never edit them. To browse the pages as
a graph, see [Browse the wiki in Obsidian](How-to-Browse-with-obsidian).
## Asking from another machine
A consumer that is not a terminal on this machine can ask the same questions through the MCP read
server and get the same answers: [Set up the MCP read server](How-to-Set-up-the-mcp-server).
How-to-Run-the-weekly-review.md
+66
@@ -0,0 +1,66 @@
*Generated from [`docs/how-to/run-the-weekly-review.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/docs/how-to/run-the-weekly-review.md) at v9.0.0 – edit the repository, not this wiki.*
# Run the weekly review
Go through your committed initiatives once a week and decide what each one needs: a next action, a
pause, a close. A machine prepares the list, so you don't have to reconstruct it from memory. The
agent runs the `gtd-weekly-review` skill ([skill catalogue](Reference-Skills)).
## Two places, on purpose
Knowledge and commitments change at different speeds, so they live in different places:
- **A page under `kb/gtd/`** is one initiative's durable memory: its goal, who is involved, where it
stands, and why it is worth doing. It never summarises the task list.
- **The open items** live in a task tracker, which owns them.
Nothing syncs between the two. `tools/wikitool review` joins them when it reads, matching on the
project name, and stores nothing - not even a report file. Why the split runs this way rather than
syncing: [knowledge-and-commitment.md](Explanation-Knowledge-and-commitment).
## What you prepare
A task tracker, configured for this checkout in `.wikitool-tasks.json`. Super Productivity and
CalDAV (for example Nextcloud Tasks) are supported. The file's shape, and the difference between a
tracker the wiki can only read and one it can write to, is in the
[configuration reference](Reference-Configuration#task-tracker). Without a tracker, the
review has nothing to join against. That is a valid state, though, and the pages work without one.
## What you say
```text
Run the weekly review
```
## What you get back
The agent runs `tools/wikitool review` and walks you through its findings, one at a time:
- initiatives with no next action;
- waiting-fors past their follow-up date;
- tracker projects that have no page;
- active pages with no open item;
- someday items that have gone stale.
For each finding, the agent asks what is actually true, because "this looks stalled" can mean a
next action is missing, the initiative was paused on purpose, or it is finished. Then it proposes the
exact change: a new item in the tracker, an item marked done, or a page's state moved to `dormant`,
`completed` or `abandoned`. Nothing is written to the tracker until you confirm that exact call.
What was decided goes onto the page, and the page is published.
A quiet week is a valid result. If one of the checks could not run because the tracker did not
answer, the agent says which check was skipped. An empty result under a skipped check means nothing.
## Starting something new
Ask the agent for a new project and name it. It runs `tools/wikitool new project`, which gives the
initiative its page under `kb/gtd/` and, where a tracker is configured, a tracker project under the
same name. A CalDAV tracker gets that project created automatically. On Super Productivity the
command stops with exit 42 and asks you to create the project in the app yourself. Tell the agent
when you are done, and it continues with `--resume`. A tracker the wiki can only read refuses the
command outright.
A single open item with no page of its own - a commitment that came with a source, say - is filed
with `tools/wikitool task new`. `task list` reads a project's open items back, and `task close --id`
marks one done. Nothing deletes an item. The commands are in the
[command reference](Reference-Commands-Pages).
How-to-Set-up-the-mcp-server.md
+253
@@ -0,0 +1,253 @@
*Generated from [`docs/how-to/set-up-the-mcp-server.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/docs/how-to/set-up-the-mcp-server.md) at v9.0.0 – edit the repository, not this wiki.*
# Set up the MCP read server
Serve an instance's wiki read-only over MCP, so that a consumer that is not a terminal on this
machine - another agent, a chat client, a service - can ask it the same questions and get the same
answers. This page covers what you set up around the server: the client, the authentication in
front of it, the optional `submit` path, and how to check that it works. Installing, starting and
syncing the server itself is
[instructions/mcp-read-server.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/instructions/mcp-read-server.md), which an agent follows
and which is readable for you as well.
<!-- wikitool:toc -->
## Contents
- [What the server is, and what it is not](#what-the-server-is-and-what-it-is-not)
- [What you prepare](#what-you-prepare)
- [Wiring a client to it](#wiring-a-client-to-it)
- [Running it as a service](#running-it-as-a-service)
- [Authentication in front of it](#authentication-in-front-of-it)
- [Keeping the served tree current](#keeping-the-served-tree-current)
- [Optional: the submit path](#optional-the-submit-path)
- [Checking that it works](#checking-that-it-works)
- [Troubleshooting](#troubleshooting)
- [What is deliberately not here](#what-is-deliberately-not-here)
<!-- /wikitool:toc -->
## What the server is, and what it is not
The server is a second consumer of the same core as the CLI, not a second program. Both call the
same functions, and a golden test in the origin repository holds their output together. What
`tools/wikitool search --json` returns, the MCP tool `search` returns too, together with the commit
the answer was computed from.
It is not a write path into `kb/`. No tool creates, changes or publishes a page - not because a list
is filtered, but because the server imports nothing under `tools/chemenu/commands/`. Those functions
cannot be reached from its process. The one exception is opt-in: a sixth tool, `submit`, which
writes only into a quarantine no other command reads. A human decides about everything that leaves
it ([below](#optional-the-submit-path)).
| Tool | What it answers |
|---|---|
| `search` | Pages in `kb/` by text, by frontmatter (`!sources`, `tags~k8s`), or both |
| `types` | Which page types this wiki knows |
| `describe_type` | A type's full contract: fields, required values, enums |
| `lint` | Structural findings: broken wikilinks, orphans, catalog drift, schema gaps |
| `status` | A snapshot: page count, distribution over collections, finding counts |
| `submit` *(optional)* | Hands a document to the review queue - into a quarantine, never into `kb/` |
## What you prepare
- A working instance ([INSTALL.md](Install)), including `tools/.venv` and ripgrep.
- For a deployment, a reverse proxy that can authenticate requests before they reach the server
([below](#authentication-in-front-of-it)).
Then ask the agent:
```text
Set up the MCP read server for this instance
```
It installs the server's optional dependency, which is deliberately not in
`tools/requirements.txt`: an instance that only uses the CLI should not have to install pydantic,
starlette, uvicorn and cryptography. It also checks that tracing cannot write into the served tree.
The server refuses to start if it could:
```text
ERROR Telemetry is on and would write into the served checkout (...). Set WIKI_TRACE=0,
or point WIKI_TRACE_DIR outside the corpus.
```
That refusal is expected, not an installation fault. Telemetry is off by default in an instance
installed from a release and on in a clone of the origin repository
([configuration](Reference-Configuration#telemetry)). The server consults the same policy
`wikitool doctor` does. Setting `WIKI_TRACE=0` explicitly, as every example here does, is correct
either way and makes the check moot.
## Wiring a client to it
Locally, over `stdio`, the client starts the server itself: one process per consumer, no network.
The configuration follows the usual MCP client shape, with absolute paths, because the client
inherits no working directory:
```json
{
"mcpServers": {
"chemenu": {
"command": "/path/to/instance/tools/.venv/bin/python",
"args": ["-m", "chemenu.mcp"],
"cwd": "/path/to/instance/tools",
"env": {
"WIKI_TRACE": "0",
"CHEMENU_ROOT": "/path/to/instance"
}
}
}
}
```
`CHEMENU_ROOT` says **which** wiki is served. Without it, the server takes the checkout its own
package sits in. That is enough for a single instance, but with several corpora it is safer to always
set it. The process speaks MCP over stdin and stdout and prints nothing of its own, which is correct.
## Running it as a service
Deployed, the server speaks `streamable-http`, the only transport a reverse proxy can sit in front
of. The command a service unit or container runs looks like this:
```bash
WIKI_TRACE=0 CHEMENU_ROOT=/srv/chemenu \
tools/.venv/bin/python -m chemenu.mcp \
--transport streamable-http --host 0.0.0.0 --port 8000
```
The endpoint is then `http://<host>:8000/mcp`. `--host 0.0.0.0` is deliberately not the default:
without it the server binds to loopback, which is right locally and wrong in a container, where the
proxy has to reach it. If you set it, make sure something stands in front. `sse` is not offered. It
is the superseded remote transport, and building on it now would only postpone the switch.
## Authentication in front of it
**The server does not authenticate on its own, by design.** A request that is not properly
authenticated should never reach the Python process at all. The authentication is a Traefik
ForwardAuth middleware:
> **<https://gitea.nehmer.net/torben/gitea-mcp-forward-auth>**
In short, it checks `Authorization: Bearer <token>` against SHA-256 hashes of allowed tokens. It
answers `200` for a valid token and `401` for a missing or wrong one, and always leaves
`GET /healthz` open. Plaintext tokens appear neither in the configuration nor in the log, only
hashes and a short fingerprint. It is configured with `AUTH_PROXY_TOKEN_HASHES_DIR` (a directory with
one file per token hash, which suits a Kubernetes secret volume) or `AUTH_PROXY_TOKEN_HASHES` (a
comma-separated list). The README there has the details, reference manifests and a test script.
To create a token hash:
```bash
echo -n "my-token" | sha256sum | awk '{print $1}'
```
**Rate limiting belongs in the same place,** in front of the process and beside the
authentication. It does not belong in the Iteration Budget Gate. That gate stops an *agent session*
from iterating over the wiki's state unnoticed, which is why retrieval is exempt from it. Using it
as a rate limiter would water it down.
## Keeping the served tree current
The server reads the working tree, and a stale checkout answers confidently and wrongly. So **every
answer carries the commit** it was computed from:
```json
{ "commit": "<40-character SHA>", "as_of": "<ISO-8601, UTC>", "count": 3, "...": "..." }
```
A timer beside the server keeps the tree current by polling: a fetch, then a hard reset to the
remote branch. The two commands, and why the hard reset is load-bearing, are in the instruction's
sync step. Polling rather than a webhook, because it needs no inbound endpoint and no signature
check, which is a smaller attack surface than the thing a webhook would optimise. **Never add
`git clean` to that timer.** The reset leaves every gitignored path alone, which is what keeps the
`submit` quarantine and any traces intact. A clean would silently delete a submission nobody has
reviewed yet.
## Optional: the submit path
Without this, `submit` does not exist as a tool - not unused, but not registered. A file
`.wikitool-upload.json` in the served corpus turns it on. Its fields, defaults and what a malformed
file does are in the [configuration reference](Reference-Configuration#mcp-submissions); a
broken file never means "no limits", it stops the server from starting.
**With `submit` on, the middleware has a second duty.** It has to set the identity header itself, on
every authenticated request, and discard any copy the client sent. The server trusts that header as
the value for `submitter`. A header the client could set itself would be a claim, not an identity.
If `identity_header` names a header the middleware does not set, every submission is refused for
lack of an identity.
Submitted files land in `mcp-upload/<id>/`, which is gitignored and read by no other command. From
there a human promotes or rejects them, in the same checkout as the server rather than in its
process. That review is part of
[ingesting a source](How-to-Ingest-a-source#a-document-submitted-from-outside).
## Checking that it works
The quickest test without a client starts the server over stdio, lists its tools and asks one
question:
```bash
cd tools && WIKI_TRACE=0 .venv/bin/python - <<'EOF'
import asyncio, os
from mcp import ClientSession, StdioServerParameters
from mcp.client.stdio import stdio_client
async def main():
params = StdioServerParameters(
command=".venv/bin/python", args=["-m", "chemenu.mcp"],
env={"WIKI_TRACE": "0", "PATH": os.environ["PATH"]},
)
async with stdio_client(params) as (r, w):
async with ClientSession(r, w) as s:
await s.initialize()
print("Tools:", [t.name for t in (await s.list_tools()).tools])
out = await s.call_tool("status", {})
d = getattr(out, "structuredContent", None) or out.structured_content
print("Pages:", d["pages"], "| Commit:", d["commit"])
asyncio.run(main())
EOF
```
Expected output, roughly:
```text
Tools: ['search', 'types', 'describe_type', 'lint', 'status']
Pages: 176 | Commit: 576df2cdddc96614a7e6641e562022d52112d411
```
Then cross-check against the CLI. Both have to give the same answer:
```bash
tools/wikitool search "<term>" --json
```
## Troubleshooting
**`ERROR Telemetry is on and would write into the served checkout`**: expected behaviour, not an
installation fault. Set `WIKI_TRACE=0`, or point `WIKI_TRACE_DIR` outside the corpus.
**`ModuleNotFoundError: No module named 'mcp'`**: the optional dependency is not installed, or
the client starts a different Python than the instance's. Give the client the absolute path to
`tools/.venv/bin/python`.
**`"commit": null` in every answer**: the served tree has uncommitted changes. Either the sync is not
running, or something writes into the corpus that has no business there. The five read tools never
write. `submit`, when it is on, writes only to `mcp-upload/`, which is gitignored and so is never a
cause of `"commit": null` either.
**`commit` names an old revision**: the sync timer is not running.
**The server answers differently from `wikitool`**: that is a defect, not a configuration
difference, since both go through the same functions. First check that both point at the same root.
`CHEMENU_ROOT` is easy to set for one and not the other.
**Not reachable from outside**: without `--host 0.0.0.0` the server binds to loopback. If it is
reachable but every call gets `401`, the middleware is working correctly and the token is wrong.
## What is deliberately not here
Deployment specifics - the cluster, ingress hosts, the secret store, the GitOps source - are
private infrastructure, and this repository is public.
There is no container image for the server yet. Until there is, the path above - the venv,
`python -m chemenu.mcp`, a proxy in front - is the complete one. With `submit` on, `mcp-upload/`
has to survive restarts the same way the rest of the checkout does. Otherwise a submitted file that
nobody has reviewed yet loses its quarantine.
How-to-Trace-and-score-sessions.md
+178
@@ -0,0 +1,178 @@
*Generated from [`docs/how-to/trace-and-score-sessions.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/docs/how-to/trace-and-score-sessions.md) at v9.0.0 – edit the repository, not this wiki.*
# Trace and score agent sessions
Record what an agent did in a session, read the record back, and score it. A score covers the state
the session left the wiki in and how it got there: whether it repeated a refused call, or passed a
gate's flag without that gate ever refusing. Why this exists, what the trace contains, and what a
score means are covered in [telemetry-and-evaluation.md](Explanation-Telemetry-and-evaluation).
<!-- wikitool:toc -->
## Contents
- [What you prepare](#what-you-prepare)
- [Reading a session back](#reading-a-session-back)
- [What each harness can report](#what-each-harness-can-report)
- [Claude Code](#claude-code)
- [Copilot CLI](#copilot-cli)
- [VS Code Chat](#vs-code-chat)
- [Mistral Vibe](#mistral-vibe)
<!-- /wikitool:toc -->
## What you prepare
**Turn tracing on, if it is off.** It is on in a clone of the origin repository and off in an
instance installed from a release. `tools/wikitool doctor` reports which state this checkout is in,
and why. To turn it on for one checkout, create `.wikitool-telemetry.json` at the root:
```json
{ "enabled": true }
```
The file and its other keys are in the [configuration reference](Reference-Configuration#telemetry).
Everything stays on this machine: traces land under `reports/telemetry/`, which is gitignored, and
no exporter is configured.
**Scope the session to the task.** Every source of events joins on one session id. Setting it
yourself keeps one task in one trace:
```bash
export WIKITOOL_SESSION_ID="my-task"
```
[instructions/session-setup.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/instructions/session-setup.md) has the line for each shell,
and what to do on a harness that starts a fresh shell for every command.
## Reading a session back
```bash
tools/wikitool eval sessions # which sessions have a trace
# what did that session do?
jq -r '[.ts, .source, .event, (.attrs.command // .attrs.tool_name // "")] | @tsv' \
reports/telemetry/my-task/trace.jsonl
# how well did it do it?
tools/wikitool eval score --session my-task
tools/wikitool eval score --session my-task --save # also write the scorecard under reports/evals/
```
Both `eval` commands only read, and neither counts against the session's call budget: reading back
what a session did is not iterating on the wiki. Their options are in the
[telemetry command reference](Reference-Commands-Telemetry).
To see how a harness hook payload would be recorded, feed one in by hand without writing anything:
```bash
echo '{"session_id":"x","hook_event_name":"post_tool","tool_name":"bash"}' \
| tools/trace_ingest.py --source mistral-vibe --dry-run
```
## What each harness can report
`wikitool` records its own calls on every harness. What the agent did *between* those calls -
file reads, searches, shell commands, prompts - comes from the harness's hooks, and harnesses differ
a lot in what they offer. Verified against vendor documentation on 2026-08-23:
| | Claude Code | Copilot CLI | VS Code Chat | Mistral Vibe |
|---|---|---|---|---|
| Hook events | ~30 | 14 | none | 3 (`pre_tool`, `post_tool`, `post_agent`) |
| Session start/end hook | yes | yes | - | **no** |
| Prompt submit hook | yes | yes | - | **no** |
| Which instructions loaded | yes (`InstructionsLoaded`) | no | no | no |
| Block / rewrite a tool call | yes | yes | - | yes |
| OTel to your own collector | yes | via MDM `telemetry` | no | **no** - `enable_otel` targets Mistral Studio only |
| `TRACEPARENT` to subprocesses | yes | undocumented | - | undocumented |
| Programmatic mode | `-p`, `stream-json` | `-p` | no | `-p`, `--output streaming` |
| Isolated config home | to be confirmed | `COPILOT_HOME` | no | `VIBE_HOME` |
| Local session log | `~/.claude/projects/*.jsonl` (unstable format) | `~/.copilot/session-state/<id>/events.jsonl` | chronicle SQLite | `$VIBE_HOME/logs/` (no format guarantee) |
| Wired up here | **partial** - `.claude/settings.json` (`UserPromptSubmit` + a `permissions.ask` rule on the clearing publish) | **yes** - `.github/hooks/wiki-trace.json` | **yes** - `tools/import_chronicle.py` | **yes** - `.vibe/hooks.toml` |
A rule that needs an event a harness cannot report is shown as *skipped*, not as passed or failed.
### Claude Code
`.claude/settings.json` wires `UserPromptSubmit` to `tools/trace_ingest.py`, through
`tools/trace-hook` like every hook here. That gives the trace its turn boundaries, which the rule
`clearance-ended-the-turn` needs. Without a `prompt.submitted` event it reports "cannot say"
instead of a verdict.
**No `PreToolUse` decision hook is wired, and that is a finding, not an oversight.** Verified against
the live CLI on 2026-08-27: a `PreToolUse` hook that returns
`hookSpecificOutput.permissionDecision: "ask"` does **not** override a matching `permissions.allow`
rule. Permissions are evaluated before the hook's decision, so a hook cannot force a confirmation
prompt on a command that is already allowed. A `permissions.ask` rule *does* win, which is why
`.claude/settings.json` carries one on the `--confirm` form of `publish`
(`Bash(tools/wikitool publish --confirm:*)`). The clearing call prompts, and ordinary publishes below
the threshold do not. It is a prefix match, so it depends on `--confirm` coming first, which is why
`publish` always prints its re-run line in that order. Treat it as a useful second line, not a
guarantee: an agent that reorders the flags gets around it.
### Copilot CLI
[.github/hooks/wiki-trace.json](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/.github/hooks/wiki-trace.json) is committed, so a clone
brings its own tracing. Eleven events go to `tools/trace_ingest.py`. `disableAllHooks` in
`.github/copilot/settings.json` turns them off without deleting anything. `userPromptSubmitted` is
among them, so `clearance-ended-the-turn` can be scored on Copilot CLI with no change.
A `preToolUse` entry that forces a decision on the clearing call would first need Copilot's own
decision-document schema verified against a live CLI. That has not happened, and an adapter that
cannot be verified is not written.
Four details of the hook configuration matter. The first two hold for all three harnesses that have
hooks:
- **The interpreter is the venv's, never the script's shebang.** Each `bash` command is
`./tools/trace-hook ...`, which runs `trace_ingest.py` with `tools/.venv`'s Python in either venv
layout. Each `powershell` command names `tools\.venv\Scripts\python.exe` directly. On Windows in
Git Bash, the shebang's `python3` was the Microsoft Store alias, which made every hook a silent
no-op. Before the preflight has created the venv, `trace-hook` records nothing and exits 0.
- **`./tools/trace-hook` has a PowerShell twin, `tools/trace-hook.ps1`.** PowerShell on Windows
resolves the string to the `.ps1` first. Without the twin, it hands the sh script to a file
association, and Windows asks which app should open it - on every hook event. Copilot CLI also
reads `.claude/settings.json` and runs its single `command` under PowerShell on Windows, so a
`bash` field alone does not keep the string out of PowerShell. The twin follows the sh script's
rules - venv Python, silent without a venv, exit 0 whatever happens - and stays parseable by
Windows PowerShell 5.1, because VS Code starts hooks under 5.1.
- **Every command ends in `|| true`.** `preToolUse` hooks fail closed: a non-zero exit denies the
tool call. Without the guard, a missing interpreter would turn the observer into a blocker that
refuses every tool call in the session. Timeouts fail open, so the 5-second `timeoutSec` is no
hazard.
- **The event name is passed explicitly.** Copilot serves two payload dialects: camelCase event
names give camelCase fields, and PascalCase names give the snake_case shape that VS Code and Claude
use. `--event` means the mapping does not depend on which one a configuration chose.
### VS Code Chat
VS Code Chat has no hooks, so nothing observes a session while it runs. Its chronicle store
(`session-store.db`, the same schema as Copilot CLI's) keeps sessions, turns and touched files, and
[tools/import_chronicle.py](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/tools/import_chronicle.py) rebuilds a trace from it afterwards:
```bash
tools/import_chronicle.py --dry-run # this repository's sessions
tools/import_chronicle.py --session 0f1e2d3c
```
The store starts empty, and `/chronicle reindex` fills it. That command also syncs session data to
your GitHub account, so run it deliberately rather than from a script. A rebuilt trace marks itself
`reconstructed: true`, and its `completeness` names `tool.post` but not `tool.pre`: the store records
that a file was touched, not that a tool was about to run.
### Mistral Vibe
[.vibe/hooks.toml](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/.vibe/hooks.toml) declares the three hooks Vibe has, and
[.vibe/config.toml](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/.vibe/config.toml) keeps the telemetry settings in the repository rather
than in someone's shell profile. Both were checked against the installed CLI's own loader
(`mistral-vibe 2.24.2`) rather than against its documentation. What that turned up:
- **A failing hook cannot block anything.** With `strict = false`, the default, a crash or a timeout
is only a warning. Only a hook that opts into `strict` can deny a tool call - the mirror image of
Copilot.
- **`post_agent` carries no response text,** only the session context. That is why it maps to
`turn.end` rather than `assistant.message`.
- **`enable_telemetry` defaults to `true`.** Turning it off is a real change, and it also gates OTel
export, which needs both flags on. This repository leaves both off: Vibe's `enable_otel` sends
prompts to Mistral Studio.
- **Vibe already reads `.agents/skills/` and `AGENTS.md`.** The directory that
`wikitool instructions sync` publishes is a project-scope skill source for Vibe, so working with
Vibe needs no adaptation, only a trusted folder.
How-to-Troubleshoot.md
+99
@@ -0,0 +1,99 @@
*Generated from [`docs/how-to/troubleshoot.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/docs/how-to/troubleshoot.md) at v9.0.0 – edit the repository, not this wiki.*
# Troubleshoot an instance
What a common failure means, and what to do about it. During installation itself, the preflight
names its own stops - what is missing, why, and the command that fixes it - and
[INSTALL.md](Install) explains them. This page covers an instance that is already set up.
For the MCP read server, see the troubleshooting section of
[Set up the MCP read server](How-to-Set-up-the-mcp-server#troubleshooting).
## `tools/wikitool` ends with `STOP - this checkout is not set up yet` (exit 42)
The preflight has not run in this checkout yet, or not since the last update. Run it:
```bash
tools/preflight.sh
```
In PowerShell 7 on Windows, the call is
`pwsh -NoProfile -ExecutionPolicy Bypass -File tools/preflight.ps1`. The `-ExecutionPolicy Bypass`
applies to that one process only and changes no setting.
[instructions/preflight.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/instructions/preflight.md) is the procedure, and its output says
what to install if anything is missing.
## The agent offers no skills (`wiki-ingest`, `wiki-query`, ...)
`.agents/skills/` and `.claude/skills/` are generated and not committed. Run:
```bash
tools/wikitool instructions sync
```
Then restart the agent session. A harness reads its skills only when the session starts.
## `doctor` reports `personalization: FAIL`
`USER.md` or `SOUL.md` is missing, or still carries the sentinel line from its template. A renamed
template does not count as a filled one. Ask the agent to run the personalization step of
[instructions/setup-instance.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/instructions/setup-instance.md). For a further clone of an
older instance, that is the only step to catch up on.
## `doctor` reports `environment: WARN`
`ENVIRONMENT.md` exists but still carries the template's sentinel line. Fill it in (the template is
`ENVIRONMENT.md.template`) and remove the line, or delete the file. It is optional, and `absent` is
a valid state.
## `doctor` reports `telemetry: WARN`
`.wikitool-telemetry.json` holds something telemetry does not read as written: unreadable JSON, a
value that is not an object, an `enabled` that is not `true` or `false`, a cap that is not a
positive whole number, or a key it does not know. Telemetry itself never stops on this. It ignores
the file or the value, or reads `"no"` as on, and the line names the field. Correct the file to the
shape in the [configuration reference](Reference-Configuration#telemetry), or delete it to
get the default back.
## `new` stops with "No author configured"
Neither `$WIKI_AUTHOR` nor `git config user.name` is set. Run `git config user.name "<Name>"`, or
export `WIKI_AUTHOR` ([configuration](Reference-Configuration)).
## `publish` ends with exit 42 (Mass-Update Gate)
Expected when ten or more files are counted, for example on the very first commit of a new instance
or after a lint pass. It is not an error. It asks that a human see the output: the complete file
list and the exact `--confirm <token>` line that publishes once you approve.
[instructions/gates.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/instructions/gates.md) has the details.
## `publish` ends with exit 42 (Publish-Remote Gate)
This checkout has a `.wikitool-remotes.json`, and the remote being pushed to is not listed in it.
Also not an error: the output names the push URL it would write to and the URLs that are allowed.
Unlike the Mass-Update Gate, there is **no token and no flag** here. If the target really is right,
you add its URL to the file yourself. An agent that edits the file to get past the refusal is
opening a gate on its own initiative, which it must never do.
## Setup or an update fails, and you want to report it
Ask the agent to follow [instructions/bug-report.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/instructions/bug-report.md), or call
`tools/bugreport` yourself. It works the same from Bash, Git Bash and PowerShell. It finds a
Python 3.8 or newer by itself, skipping the Store aliases `python3` points at on Windows. It needs
neither the preflight nor a working `wikitool`, and if it finds no Python, it says why.
The script writes a bundle to `reports/bugreport-<timestamp>/`, with a zip beside it.
- **What is removed:** secrets, and page titles from everything the script itself produces.
`--titles` keeps the titles.
- **What may still be in it:** the session trace is included by default (`--no-trace` leaves it
out). Like the chronology and the transcripts, it can contain page content and titles. The bundle
also contains machine, user and path names.
- **Pseudonymising:** `--pseudonymise` replaces user, host, path, git and remote names with
placeholders that keep their length, spaces, hyphens and path depth. The agent can then add more
names - people, companies, customers, projects - with `--bundle … --candidates …`. That is a
model's judgment, and it leaves a remainder: read the bundle before you share it. The mapping,
the checklist and the candidates file contain the originals, and they sit beside the bundle,
never in it.
Nothing is uploaded anywhere. You choose the channel.
How-to-Update-an-instance.md
+156
@@ -0,0 +1,156 @@
*Generated from [`docs/how-to/update-an-instance.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/docs/how-to/update-an-instance.md) at v9.0.0 – edit the repository, not this wiki.*
# Update an instance
Bring an instance installed from a release up to a newer release of the stack: the tools, types,
instructions and contracts. Your pages stay yours. The agent carries out the upgrade by following
[instructions/upgrade-instance.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/instructions/upgrade-instance.md), which sets the order of
the steps, what each one decides, and where the session has to restart. This page covers the
decision before it, and what you'll see along the way.
<!-- wikitool:toc -->
## Contents
- [What is running, and is there something newer?](#what-is-running-and-is-there-something-newer)
- [Read the notes first](#read-the-notes-first)
- [What you say](#what-you-say)
- [Where the agent stops](#where-the-agent-stops)
- [Where the release comes from, and whom you trust](#where-the-release-comes-from-and-whom-you-trust)
- [Two first-time cases](#two-first-time-cases)
- [Pitfalls](#pitfalls)
<!-- /wikitool:toc -->
## What is running, and is there something newer?
Every instance carries the version of the **stack**, not of its content. It is in `VERSION`, and
`.wikitool-release.json` beside it records where the release came from and when it was exported.
```bash
tools/wikitool version # what runs here, and where it came from
tools/wikitool version check # is there a newer release?
tools/wikitool version notes # what the newest release changes
```
`version check`, `version notes` and `dist upgrade --latest` are the only commands that go to the
network. All three ask the same release feed: the one named in the stamp, unless
`WIKITOOL_UPDATE_URL` overrides it. A feed that cannot be reached is reported as an error, **never**
as "up to date" and never as "no notes".
## Read the notes first
What a version number promises: two versions are compatible when they agree in the *leftmost
non-zero part*. `0.1.3 → 0.1.4` is a safe update and `0.1.3 → 0.2.0` is not. From `1.0.0` on, the
same rule reads as the familiar "a MAJOR breaks". `version check` says which case you are in
(`state: update` or `state: migration`).
That number only answers whether the new machinery can simply be copied over the old. Whether your
*content* has to be migrated is a second, independent question. So the release notes of a MAJOR
carry two separate lines, and both are worth reading before you start:
- **Breaking Change:** what stops working, and what this instance has to do about it.
- **Migration:** whether and how the pages are rewritten (`none required` when they are not).
`version notes` prints both. On an instance it reads them from the release feed, because an
instance's own `CHANGES.md` is a stub that an update never overwrites. It always asks for the
**newest** release, so while `VERSION` still names the old one, it answers with a version other than
the one installed, and says so on stderr. If the feed cannot be reached, the error names the release
page recorded as `release_url` in `.wikitool-release.json`. `--offline` sends you there directly.
Why the two questions are kept apart: [version-model.md](Explanation-Version-model).
## What you say
```text
Update this instance to the latest release
```
## Where the agent stops
In the order you'll meet them:
- **The release notes.** The agent shows you both lines before it touches anything. A migration the
instance already owes from an earlier release is finished first.
- **A stack file you changed by hand.** `dist upgrade` lists every file that differs from what the
installed release shipped, instead of overwriting it silently. For each one, you decide: take the
release's version, or keep yours. Keeping yours means the file is reported again at every future
upgrade.
- **The preflight.** A new release may need other tools than the old one. If something is missing,
the preflight stops with exit 42 and says what to install, exactly as during installation.
- **The first publish.** The swapped machinery is published on its own, before any content changes.
That is far more than ten files, so the Mass-Update Gate (exit 42) shows you the list and the
`--confirm` line.
- **The session restart.** The running session still holds the *old* `AGENTS.md`, contracts and
skills, so the agent asks you to restart it before migrating content. The new session picks up
where the old one stopped.
- **A required migration.** The agent carries it out, and may come back to you with a question
while doing so: a migration document can reach a point that is yours to decide, such as a page
that contradicts another.
- **An offered migration.** It changes a file the instance owns, so taking it or declining it is
your decision.
- **The second publish,** of the migrated content, behind the same gate.
**A required migration can also be interrupted.** If the agent cannot complete one now, it stops
after the first publish and leaves it. That is a valid state: new machinery, content still in its
old form. `doctor` keeps warning until the migration is done, and a later session resumes with
`tools/wikitool migrate status`. The agent never records a migration it did not carry out, because
the content's version would then describe a shape the pages are not in.
If a step fails for a reason that isn't obvious, the agent stops and offers you a bug report rather
than improvising ([Troubleshoot](How-to-Troubleshoot)).
`doctor` warns as long as the content's version (`kb_version` in `.wikitool-kb.json`) is behind
`VERSION` and migrations are still open. An instance older than `.wikitool-kb.json` has no such file
at all. In that case `tools/wikitool migrate baseline <version>` is called once, and nothing is
guessed.
## Where the release comes from, and whom you trust
`tools/wikitool dist upgrade --latest --expect <version>` asks the release feed for its newest
release, downloads the tarball and its `.sha256` into a scratch directory, checks one against the
other and applies the release. The scratch directory is removed on every exit, including
`--dry-run`. `--expect` is the version `version notes` showed you: the feed only knows its *newest*
release, and `--expect` refuses **before** downloading if a newer one has appeared since, rather
than applying something nobody read. Pre-releases (`-beta.N`) need `--pre`.
**What the checksum does, and what it does not do.** The `.sha256` comes from the same feed as the
tarball. It protects against a damaged transfer, not against a compromised feed. Whether a release
is genuine rests on your trust in the host whose feed the instance asks (`update_url` in the stamp).
Nothing forces https. A configured `http://` feed is a deliberate choice. `WIKITOOL_UPDATE_TOKEN`
is sent only to downloads on the same host as the feed.
**Offline, or with a tarball from someone else:** put the tarball and its `.sha256` side by side,
and the agent passes the file instead of `--latest` (`dist upgrade <tarball>`). A tarball has to
unpack to exactly one top-level directory, which is the shape the release workflow builds.
## Two first-time cases
**The first jump to a release that knows `--latest`:** the option does not exist in the instance
yet, because the instruction there belongs to the release being left. Fetch the tarball and the
`.sha256` once from the release page, put them side by side, and run `dist upgrade <tarball>`. From
the next release on, the instance has `--latest` itself.
**The first jump to `4.5.0` or later:** `dist upgrade` itself does not exist in the instance yet,
because it arrived with `4.5.0`. Use the tool from the unpacked *new* tarball and point it at the old
instance:
```bash
tar -xzf chemenu-stack-<version>.tar.gz
CHEMENU_ROOT="$PWD" chemenu-stack-<version>/tools/wikitool \
dist upgrade chemenu-stack-<version>.tar.gz --dry-run
```
`CHEMENU_ROOT` tells the package which corpus to work on ([configuration](Reference-Configuration)).
Without it, the package would take the unpacked tarball itself for the instance. From the second
upgrade on, the instance carries the command and the instruction itself.
## Pitfalls
- An instance without a local `.wikitool-release.json`, or with one that has no `files` block (from
before `4.5.0`), gives `dist upgrade` nothing to detect a local change against, so it refuses
the swap. There is no repair for that today.
- Given a tarball or directory, `dist upgrade` downloads nothing. Only `--latest` downloads.
- To follow the swap by hand anyway, perhaps with a tool `wikitool` cannot run, use the file list in
the `files` block of `.wikitool-release.json`. The exceptions - `kb/CONVENTIONS.md`, every
`kb/*/COLLECTION.md`, `.wikitool-kb.json` - are in the
[`dist upgrade` record](Reference-Commands-Distribution-and-versioning#dist-upgrade), along
with what the command classifies and refuses.
How-to.md
+31
@@ -0,0 +1,31 @@
*Generated from [`docs/how-to/README.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/docs/how-to/README.md) at v9.0.0 – edit the repository, not this wiki.*
# How-to guides
Task-oriented pages for the operator of an instance: one goal per page - upgrade an instance,
set up the MCP read server, run a weekly review - with the steps a human takes and the sentence
handed to the agent. A how-to guide never retells an agent's procedure: the steps an agent runs
live in `instructions/`, and a guide here says what to prepare, what to ask for and where the
agent stops.
Working with the wiki:
- [Ingest a source](How-to-Ingest-a-source) - a file, a folder, a web page, or a document submitted
through the MCP server
- [Capture documentation from a git repository](How-to-Capture-repository-docs) - and keep it current
- [Export guidelines to captured repositories](How-to-Export-guidelines)
- [Query the wiki](How-to-Query-the-wiki) - asking, and searching yourself
- [Maintain the wiki](How-to-Maintain-the-wiki) - lint, provenance, and the gates on the way to a publish
- [Run the weekly review](How-to-Run-the-weekly-review) - commitments in a task tracker against the
project pages
- [Browse the wiki in Obsidian](How-to-Browse-with-obsidian) - and how pages are named
Running the instance:
- [Update an instance](How-to-Update-an-instance) - to a newer release of the stack
- [Troubleshoot an instance](How-to-Troubleshoot)
- [Set up the MCP read server](How-to-Set-up-the-mcp-server) - for a consumer that is not a terminal
on this machine
- [Trace and score agent sessions](How-to-Trace-and-score-sessions)
Installing a new instance is [INSTALL.md](Install) at the repository root.
Install.md
+146 -453
@@ -1,29 +1,29 @@
*Generated from [`INSTALL.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/INSTALL.md) at v8.0.0 – edit the repository, not this wiki.*
*Generated from [`INSTALL.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/INSTALL.md) at v9.0.0 – edit the repository, not this wiki.*
# Installation
# Install a Chemenu instance
Dieses Dokument richtet sich an Menschen. Es gibt genau einen Weg zu einer Chemenu-Instanz: Ein
Agent installiert das **neueste Release** in ein leeres Verzeichnis, das du vorgibst. Die
Schritte führt der Agent aus, nach `instructions/setup-instance.md` aus demselben Release. Hier
steht, was du vorher bereitstellst, welchen Satz du ihm gibst, was er dich fragt und was zu tun
ist, wenn er anhält. Die vollständige Kommandoreferenz steht in
[tools/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/tools/CONTRACT.md).
This tutorial takes you from an empty folder to a running Chemenu instance. There is exactly one
way to get one: an agent installs the **latest release** into an empty folder you choose. The
agent runs the steps, following `instructions/setup-instance.md` from that same release. This
page covers what you prepare beforehand, the sentence you give the agent, what it asks you, and
what to do when it stops. The full command reference is in
[tools/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/tools/CONTRACT.md).
Den optionalen **MCP-Leseserver** installiert und betreibt
[INSTALL-MCP.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/INSTALL-MCP.md): derselbe Korpus, lesend, für einen Konsumenten, der kein
Terminal auf dieser Maschine ist.
The optional **MCP read server** is set up separately, once the instance runs:
[docs/how-to/set-up-the-mcp-server.md](How-to-Set-up-the-mcp-server). It serves the same
corpus, read-only, to a consumer that is not a terminal on this machine.
<!-- dist:strip-start -->
Wer am Stack selbst arbeiten will, klont dieses Repo. Das ist eine Entwicklungsumgebung mit
Demo-Korpus und keine Instanz; sie steht in [DEVELOPMENT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/DEVELOPMENT.md).
To work on the stack itself, clone this repository instead. That is a development checkout with a
demo corpus, not an instance; it is described in [docs/dev/development.md](Dev-Development).
<!-- dist:strip-end -->
## Was vorher da sein muss
## What you need first
Diese Programme prüft der Preflight, bevor irgendein `wikitool`-Befehl läuft. Installieren musst
du sie selbst - der Agent tut es nie, auch nicht mit deiner Zustimmung. Wo eines fehlt, nennt
der Preflight den Installationsbefehl für dein System. Die Liste wird aus
`tools/prerequisites.txt` erzeugt, derselben Datei, die der Preflight liest:
The preflight checks these programs before any `wikitool` command runs. You install them
yourself - the agent never does, not even with your permission. Where one is missing, the
preflight names the install command for your system. The list is generated from
`tools/prerequisites.txt`, the same file the preflight reads:
<!-- wikitool:prerequisites -->
- **Python** ≥ 3.11
@@ -31,478 +31,171 @@ der Preflight den Installationsbefehl für dein System. Die Liste wird aus
- **ripgrep (rg)**
<!-- /wikitool:prerequisites -->
Dazu:
You also need:
- **Ein Agent-Harness**: Claude Code, GitHub Copilot (in VS Code oder als CLI), Codex CLI oder
- **An agent harness**: Claude Code, GitHub Copilot (in VS Code or as a CLI), Codex CLI or
Mistral Vibe.
- **Ein leeres Verzeichnis**, in dem die Instanz liegen soll, und dein Harness darin geöffnet.
Leer heißt: nichts außer einem `.git`. Ein frisch geklontes, leeres Repo für deine Instanz ist
also genau richtig - liegt dein Repo `torben/nathan` etwa in `~/src/nathan`, installierst du
dorthin, und der Agent übernimmt dessen `origin` als Ziel für `publish`.
- **An empty folder** for the instance, with your harness opened in it. Empty means nothing but a
`.git`. A freshly cloned, empty repository for your instance is exactly right: if your
repository `torben/nathan` lives in `~/src/nathan`, install there, and the agent keeps its
`origin` as the target for `publish`.
Unter Windows zusätzlich, ebenfalls vom Preflight geprüft:
On Windows, additionally, also checked by the preflight:
<!-- wikitool:prerequisites-windows -->
- **PowerShell 7 (pwsh)** ≥ 7
<!-- /wikitool:prerequisites-windows -->
Und außerdem:
And on Windows also:
- **PowerShell 7 als Standardterminal in VS Code.** Windows PowerShell 5.1 reicht nicht, und WSL
ist nicht vorgesehen.
- **Execution Policy `RemoteSigned`** - auf vielen Rechnern ab Werk gesetzt
(`Get-ExecutionPolicy -List` zeigt es).
- **Git for Windows.** Es bringt Git Bash mit, in dem Claude Code seine Befehle ausführt.
- **Ein Installationsverzeichnis mit höchstens 95 Zeichen**, zum Beispiel `C:\Chemenu`. Windows
erlaubt ohne eingeschaltete lange Pfade nur 259 Zeichen je Pfad, und die Dateien des Wikis
brauchen den Rest. Wer Administratorrechte hat, kann stattdessen lange Pfade einschalten
(`LongPathsEnabled`); verlangt wird das nicht.
- **PowerShell 7 as the default terminal in VS Code.** Windows PowerShell 5.1 is not enough, and
WSL is not supported.
- **Execution policy `RemoteSigned`** - the factory setting on many machines
(`Get-ExecutionPolicy -List` shows it).
- **Git for Windows.** It brings Git Bash, which Claude Code runs its commands in.
- **An installation folder of at most 95 characters**, for example `C:\Chemenu`. Without long
paths switched on, Windows allows only 259 characters per path, and the wiki's files need the
rest. If you have administrator rights you can switch long paths on instead
(`LongPathsEnabled`); that is not required.
## Der Satz für den Agenten
## The sentence for the agent
Öffne das leere Verzeichnis in deinem Harness und gib dem Agenten diesen Satz:
Open the empty folder in your harness and give the agent this sentence:
> Richte in diesem Verzeichnis eine neue Chemenu-Instanz ein. Hol dazu das neueste Release von
> `https://gitea.nehmer.net/api/v1/repos/torben/chemenu/releases/latest` und folge dessen Asset
> `setup-instance.md`. Lies vor dem Start des Preflights das Asset `preflight.md` aus demselben
> Release.
> Set up a new Chemenu instance in this folder. Fetch the latest release from
> `https://gitea.nehmer.net/api/v1/repos/torben/chemenu/releases/latest` and follow its asset
> `setup-instance.md`. Before you start the preflight, read the asset `preflight.md` from the same
> release.
Damit liest der Agent die Beschreibung des neuesten Releases und daraus die beiden Anleitungen.
Dann lädt er das passende Preflight-Skript (`preflight.ps1` für PowerShell, `preflight.sh` für
eine POSIX-Shell) mit einem Befehl seiner Shell ins Verzeichnis - nicht über den Browser, damit
Windows die Datei nicht als „aus dem Internet“ markiert - und startet es. Das Skript lädt den
Tarball desselben Releases, prüft dessen sha256, entpackt ihn in das Verzeichnis, löscht sich
selbst und prüft dann im entpackten Baum, ob alles da ist.
With that, the agent reads the description of the latest release, and from it the two
instructions. It then downloads the matching preflight script (`preflight.ps1` for PowerShell,
`preflight.sh` for a POSIX shell) into the folder with a command of its shell - not through the
browser, so that Windows does not mark the file as "from the internet" - and runs it. The script
downloads the tarball of the same release, checks its sha256, unpacks it into the folder, deletes
itself, and then checks in the unpacked tree that everything is in place.
Die Liste aller Releases: <https://gitea.nehmer.net/torben/chemenu/releases>. Das Repo ist
öffentlich; der Download braucht weder Konto noch Token.
All releases are listed at <https://gitea.nehmer.net/torben/chemenu/releases>. The repository is
public; the download needs neither an account nor a token.
## Was der Agent dich fragt
## What the agent asks you
Raten darf der Agent keine dieser Antworten, und keine übernimmt er aus einem anderen Repo:
The agent may guess none of these answers, and it carries none of them over from another
repository:
- <!-- setup-question: identity --> **Autor-Identität** - Name und E-Mail für `git config`. Das ist
zugleich der Autorname jeder künftig angelegten Wiki-Seite (`$WIKI_AUTHOR` überschreibt ihn bei
Bedarf).
- <!-- setup-question: remote --> **Remote** - bei einem leeren Klon nur die Bestätigung, dass
`origin` stimmt; sonst eine URL, wenn du auf einen Server pushen willst. Ohne Remote bleibt die
Instanz lokal, und jedes `publish` läuft mit `--no-push`.
- <!-- setup-question: kb-language --> **Sprache und Ton der Seiten** - sie landen in
`kb/CONVENTIONS.md`, dazu je Collection `kb/<name>/COLLECTION.md`. Fertige Profile, darunter ein
vollständiges deutsches, hält `instructions/kb-profiles.md` bereit. Entscheide das **vor dem
ersten Ingest**: Danach ist ein Wechsel der Abschnittsnamen eine Migration jeder bestehenden
Seite. Titel, Wikilink-Ziele, Zitat-IDs, Schema-Werte, Tags, Befehle und Pfade folgen keiner
Sprache - `Act Runner` heißt in jeder Instanz `Act Runner`.
- <!-- setup-question: domain --> **Anwendungsgebiet** - woraus dieses Wiki seine Quellen zieht.
Daraus schlägt der Agent eine `source_type`-Liste vor (bei einem Verein etwa Satzung, Protokoll,
Spielbericht). Das ist ein Startpunkt, keine Festlegung: Später wird sie an echtem Bestand
korrigiert (`instructions/evolve-subtypes.md`).
- <!-- setup-question: personalization --> **Personalisierung** - wer diese Instanz bedient
(`USER.md`) und wie sie klingt (`SOUL.md`). Der Agent interviewt dich entlang der Vorlagen und
schreibt deine Antworten wörtlich mit. Zwei Fragen beantwortest nur du: den **Namen der Persona**
und die **Themen, die bewusst draußen bleiben**.
- <!-- setup-question: environment --> **Umgebung** (optional) - Harness, MCP-Server, Remotes,
damit spätere Sitzungen nicht erneut fragen. „Weiß ich nicht“ ist eine gültige Antwort.
- <!-- setup-question: telemetry --> **Telemetrie** - standardmäßig aus; der Agent fragt nur, ob du
sie einschalten willst.
- <!-- setup-question: task-tracker --> **Aufgaben-Tracker** (optional) - siehe
[Konfiguration](#konfiguration).
- <!-- setup-question: identity --> **Author identity** - name and email for `git config`. This is
also the author name of every wiki page created from now on (`$WIKI_AUTHOR` overrides it when
needed).
- <!-- setup-question: remote --> **Remote** - for an empty clone, only the confirmation that
`origin` is right; otherwise a URL, if you want to push to a server. Without a remote the
instance stays local, and every `publish` runs with `--no-push`.
- <!-- setup-question: kb-language --> **Language and tone of the pages** - they go into
`kb/CONVENTIONS.md`, plus `kb/<name>/COLLECTION.md` per collection. Ready-made profiles,
including a complete German one, are in `instructions/kb-profiles.md`. Decide this **before the
first ingest**: afterwards, changing the section names is a migration of every existing page.
Titles, wikilink targets, citation ids, schema values, tags, commands and paths follow no
language - `Act Runner` is called `Act Runner` in every instance.
- <!-- setup-question: domain --> **Subject area** - where this wiki draws its sources from. From
it the agent proposes a `source_type` list (for a sports club, say: statutes, minutes, match
report). That is a starting point, not a commitment: it gets corrected against real material
later (`instructions/evolve-subtypes.md`).
- <!-- setup-question: personalization --> **Personalization** - who operates this instance
(`USER.md`) and how it sounds (`SOUL.md`). The agent interviews you along the templates and
writes your answers down verbatim. Two questions only you answer: the **name of the persona**
and the **topics that deliberately stay out**.
- <!-- setup-question: environment --> **Environment** (optional) - harness, MCP servers, remotes,
so that later sessions don't ask again. "I don't know" is a valid answer.
- <!-- setup-question: telemetry --> **Telemetry** - off by default; the agent only asks whether
you want to switch it on.
- <!-- setup-question: task-tracker --> **Task tracker** (optional) - see
[docs/reference/configuration.md](Reference-Configuration#task-tracker).
Am Ende legt der Agent den ersten Commit an. Dabei hält das Mass-Update-Gate an (Exit 42), weil
eine neue Instanz aus weit mehr als zehn Dateien besteht. Das ist erwartet: Der Agent zeigt dir
die Dateiliste und die `--confirm`-Zeile, und erst nach deiner Freigabe wird veröffentlicht.
Danach startest du die Agent-Sitzung im selben Verzeichnis neu, damit sie die Skills lädt.
At the end the agent makes the first commit. The Mass-Update Gate stops it there (exit 42),
because a new instance consists of far more than ten files. That is expected: the agent shows you
the file list and the `--confirm` line, and nothing is published until you approve. Afterwards,
restart the agent session in the same folder so that it loads the skills.
## Wenn der Agent anhält
## When the agent stops
Der Preflight hält mit **Exit 42** an, wenn du etwas tun musst. Seine Ausgabe nennt in einem
nummerierten Block, was fehlt, warum, den Befehl, der es behebt, und wie es weitergeht. Der
Agent zeigt dir diesen Block unverändert, setzt eine Übersetzung höchstens darunter und wartet.
Sag ihm Bescheid, wenn du fertig bist; dann prüft er erneut. Ausweichen oder selbst installieren
darf er nicht.
The preflight stops with **exit 42** when you have to do something. Its output names, in a
numbered block, what is missing, why, the command that fixes it, and how to continue. The agent
shows you this block unchanged, adds a translation below it at most, and waits. Tell it when you
are done; then it checks again. It may not work around the stop or install anything itself.
**Beim Herunterladen und Entpacken** (das Skript aus dem Release, bevor es einen Baum gibt):
**While downloading and unpacking** (the script from the release, before there is a tree):
- **Werkzeuge zum Laden oder Entpacken fehlen** (Exit 42). Unter Linux und macOS braucht das
Skript `curl`, `tar` und `sha256sum` (oder `shasum`), unter Windows nur das `tar.exe` aus
Windows 10/11. Installiere, was die Ausgabe nennt; unter Windows liefert Git for Windows alles
für Git Bash mit.
- **Das Verzeichnis ist zu lang** (Exit 42, nur Windows ohne lange Pfade). Nimm ein kürzeres,
etwa `C:\Chemenu`, öffne es im Harness und gib den Satz dort noch einmal.
- **Das Verzeichnis ist nicht leer** (Exit 1). Es darf nichts enthalten außer dem Skript und
einem `.git`. Räume es selbst auf oder nimm ein anderes - der Agent löscht dort nichts.
- **Download fehlgeschlagen oder Prüfsumme falsch** (Exit 1). Es wurde nichts entpackt. Prüf die
Internetverbindung und lass es erneut versuchen; einen anderen Download-Weg sucht der Agent
nicht. Ohne direkten Download kannst du Tarball und `.sha256` von der Release-Seite selbst
nebeneinander ablegen; der Agent startet das Skript dann mit `--archive <tarball>`.
- **Tools for downloading or unpacking are missing** (exit 42). On Linux and macOS the script
needs `curl`, `tar` and `sha256sum` (or `shasum`); on Windows only the `tar.exe` that comes
with Windows 10/11. Install what the output names; on Windows, Git for Windows brings
everything for Git Bash.
- **The folder path is too long** (exit 42, only on Windows without long paths). Take a shorter
one, such as `C:\Chemenu`, open it in the harness and give the sentence there again.
- **The folder is not empty** (exit 1). It may hold nothing but the script and a `.git`. Clean it
up yourself or take another one - the agent deletes nothing there.
- **Download failed or checksum wrong** (exit 1). Nothing was unpacked. Check the internet
connection and let it try again; the agent does not look for another way to download. Without a
direct download you can put the tarball and its `.sha256` from the release page side by side
yourself; the agent then runs the script with `--archive <tarball>`.
- **A part of the stack could not be moved into place** (exit 1, "could not move ... into").
Usually a virus scanner briefly holding a freshly unpacked file open; the script has already
waited about 30 seconds for it. If the message says "nothing was installed", the script has
taken everything back - let the agent run it again. If it names entries it could not take
back, empty the folder yourself (keep `.git`), then run it again.
**Im entpackten Baum** (jeder weitere Lauf ist `tools/preflight.sh` bzw. unter PowerShell
**In the unpacked tree** (every further run is `tools/preflight.sh`, or under PowerShell
`pwsh -NoProfile -ExecutionPolicy Bypass -File tools/preflight.ps1`):
- **Ein Werkzeug fehlt** - Python, git, ripgrep, unter Windows auch PowerShell 7. Die Ausgabe
nennt den Installationsbefehl für dein System. Ist es schon installiert, nur woanders, nenn dem
Agenten den Pfad; er reicht ihn mit `--set <werkzeug>=<pfad>` weiter.
- **Eine Version ist zu alt** - zum Beispiel Python unter 3.11. Neuere Version installieren oder
deren Pfad nennen.
- **Ein genannter Pfad funktioniert nicht** - der Pfad muss auf das Programm selbst zeigen, nicht
auf seinen Ordner.
- **Die Python-Umgebung (`tools/.venv`) oder ihre Bibliotheken ließen sich nicht einrichten.** Die
Ausgabe zeigt, was Python oder pip gemeldet haben. Meist blockiert ein Proxy oder ein
Sicherheitsprogramm den Download; das klärt, wer deinen Rechner betreut.
- **Skripte tragen die Markierung „aus dem Internet“** (nur Windows). Das passiert, wenn das
Release im Browser geladen und im Explorer entpackt wurde. Einmal im Verzeichnis, in
PowerShell 7: `Get-ChildItem -Recurse -File | Unblock-File`. `doctor` zeigt den Stand unter
`script-marks`.
- **Die Execution Policy verbietet Skripte** (`Restricted` oder `AllSigned`, nur Windows). Die
Ausgabe nennt die eine Zeile für ein PowerShell-7-Fenster
(`Set-ExecutionPolicy -Scope CurrentUser -ExecutionPolicy RemoteSigned`). Setzt eine
Gruppenrichtlinie sie, hilft nur die IT - oder du arbeitest aus Git Bash mit `tools/wikitool`.
`doctor` zeigt den Stand unter `execution-policy`.
- **Das Installationsverzeichnis ist zu lang** (nur Windows ohne lange Pfade). Die Instanz muss
in ein kürzeres Verzeichnis umziehen, etwa `C:\Chemenu`.
- **A tool is missing** - Python, git, ripgrep, on Windows also PowerShell 7. The output names
the install command for your system. If it is installed, just somewhere else, tell the agent the
path; it passes it on with `--set <tool>=<path>`.
- **A version is too old** - Python below 3.11, for example. Install a newer version or name its
path.
- **A path you named does not work** - the path must point at the program itself, not at its
folder.
- **The Python environment (`tools/.venv`) or its libraries could not be set up.** The output
shows what Python or pip reported. Usually a proxy or a security program blocks the download;
whoever looks after your machine can sort that out.
- **Scripts carry the "from the internet" mark** (only on Windows). That happens when the release
was downloaded in the browser and unpacked in Explorer. Once, in the folder, in PowerShell 7:
`Get-ChildItem -Recurse -File | Unblock-File`. `doctor` shows the state under `script-marks`.
- **The execution policy forbids scripts** (`Restricted` or `AllSigned`, only on Windows). The
output names the one line for a PowerShell 7 window
(`Set-ExecutionPolicy -Scope CurrentUser -ExecutionPolicy RemoteSigned`). If a group policy sets
it, only your IT department can help - or you work from Git Bash with `tools/wikitool`. `doctor`
shows the state under `execution-policy`.
- **The installation folder path is too long** (only on Windows without long paths). The instance
has to move to a shorter folder, such as `C:\Chemenu`.
Hält der Agent an einer anderen Stelle an und ist die Ursache nicht offensichtlich, bietet er dir
einen Fehlerbericht an - siehe [Troubleshooting](#troubleshooting).
If the agent stops anywhere else and the cause is not obvious, it offers you a bug report - see
[docs/how-to/troubleshoot.md](How-to-Troubleshoot).
## Version und Updates
Jede Instanz trägt die Version des **Stacks** (Werkzeuge, Typen, Instruktionen, Contracts) -
nicht die ihres Inhalts. Sie steht in `VERSION`, daneben `.wikitool-release.json` mit Herkunft
und Exportdatum des Releases.
```bash
tools/wikitool version # was läuft hier, und woher kommt es
tools/wikitool version check # gibt es ein neueres Release?
```
`version check`, `version notes` und `dist upgrade --latest` sind die einzigen Befehle, die
ins Netz gehen, und alle drei fragen denselben Release-Feed der Ursprungs-Instanz
(`$WIKITOOL_UPDATE_URL` überschreibt; sonst der Wert aus dem Stamp). `version check` ist dafür da;
`version notes` greift nur dann darauf zurück, wenn die lokale `CHANGES.md` den Eintrag nicht
hat - auf einer Instanz also immer, siehe unten - und sagt vorher auf stderr, welche URL es
fragt; `dist upgrade` fragt den Feed nur, wenn `--latest` dasteht, und lädt dann auch das Release
herunter (siehe „Eine Instanz aktualisieren"). Ein nicht erreichbarer Feed wird als Fehler
gemeldet - **nie** als „aktuell" und nie als „keine Notes".
**Was die Versionsnummer aussagt:** kompatibel ist, was in der *linkesten von Null
verschiedenen Stelle* übereinstimmt. `0.1.3 → 0.1.4` ist ein sicheres Update, `0.1.3 → 0.2.0`
nicht, und ab `1.0.0` liest sich dieselbe Regel als das gewohnte „MAJOR bricht". `version check`
sagt das direkt (`state: update` vs. `state: migration`).
Was diese Stelle beantwortet, ist **ob die neue Version ein Drop-in-Ersatz ist** - ob sich die
Maschinerie einfach darüberkopieren lässt und ob die alte danach noch zurückkann. Ob *Inhalt*
migriert werden muss, ist eine **zweite, unabhängige Frage**. Ein MAJOR-Sprung kann eine leere
Migrationskette haben und trotzdem Handarbeit verlangen: umbenannter Release-Feed, umbenanntes
Artefakt, umbenannter Import- oder Kommandoname, geänderte Envvar - `kb/` bleibt dabei
unangetastet, das Update ist trotzdem keins zum Drüberkopieren. Der Abschnitt „Sonderfall:
Update von 1.x auf 2.0.0" unten ist genau dieser Fall.
Deshalb stehen in den Release-Notes eines MAJOR zwei getrennte Zeilen, und beide sind vor dem
Update zu lesen: **Breaking Change:** sagt, was aufhört zu funktionieren und was diese Instanz
dagegen tun muss; **Migration:** sagt, ob und wie der Korpus umgeschrieben wird (`none required`,
wenn nicht). `tools/wikitool version notes` druckt beide Zeilen - im Ursprungs-Repo aus der dort
gefüllten `CHANGES.md`, auf einer ausgelieferten Instanz aus dem Release-Feed, weil die Instanz
die Datei nur als Stub bekommt und ein Update sie nie überschreibt. Der Befehl fragt dabei immer
das **neueste** Release: solange `VERSION` noch die alte Fassung nennt, antwortet er also mit
einer anderen Version als der eigenen und sagt das auf stderr dazu. Ist der Feed nicht
erreichbar, nennt die Fehlermeldung die Release-Seite, die `.wikitool-release.json` als
`release_url` führt; `--offline` verlangt diesen Weg von vornherein.
### Eine Instanz aktualisieren
Das Anwenden eines Updates schreibt in eine Instanz, die bereits Inhalt hat. Der Inhalt hat dabei
eine **eigene Version**: `.wikitool-kb.json` sagt, in welcher Form die Seiten vorliegen,
unabhängig davon, welche Maschinerie danebensteht. Genau dieser Unterschied ist der Zustand, in
dem sich jede Instanz mitten im Upgrade befindet.
**Die Durchführung selbst steht in `instructions/upgrade-instance.md`** - die Reihenfolge, was
jeder Schritt entscheidet, wo die Agent-Sitzung neu gestartet werden muss, und die beiden Stellen,
an denen heute Handarbeit nötig ist. Sie steht dort und nicht hier, weil sie von einer
Agent-Sitzung ausgeführt wird; eine zweite Fassung derselben Schrittfolge an dieser Stelle wäre
genau die Kopie, die irgendwann auseinanderläuft. Wer den Lauf selbst fahren will, liest dieselbe
Datei.
Was dieses Dokument beiträgt, ist die Entscheidung *davor* - welches Release, ob überhaupt, und
wem die Instanz als Quelle vertraut - und zwei Sonderfälle, die die Instruktion nicht abdecken
kann, weil es sie dort noch nicht gibt.
**Das Release kommt mit einem Befehl.** `tools/wikitool dist upgrade --latest --expect <version>`
fragt den Release-Feed, lädt Tarball und `.sha256` in ein Arbeitsverzeichnis, prüft den Tarball
gegen die Summe und wendet ihn an; das Arbeitsverzeichnis verschwindet bei jedem Ausgang wieder,
auch bei `--dry-run`. `<version>` ist die, die `version notes` gedruckt hat: der Feed kennt nur
sein *neuestes* Release, und `--expect` verweigert den Lauf **vor** dem Download, wenn inzwischen
ein neueres erschienen ist, statt es ungelesen einzuspielen. Beides zusammen entscheidet vor dem
Download über „schon aktuell", Downgrade und Vor-Release (`-beta.N` braucht `--pre`); fehlt dem
Release der Tarball oder die Summe, bricht der Befehl vor dem ersten Download ab und nennt die
Release-Seite. Wer offline arbeitet oder einen Tarball vom Betreiber bekommen hat, gibt statt
`--latest` weiter die Datei an (`dist upgrade <tarball>`).
**Was die Prüfsumme leistet - und was nicht.** Die `.sha256` liegt beim selben Feed wie der
Tarball. Sie schützt vor einer beschädigten Übertragung, nicht vor einem Feed, der selbst
kompromittiert ist: die Echtheit eines Releases beruht auf dem Vertrauen in den Host, dessen
Feed die Instanz fragt (`update_url` im Stamp). Es gibt keinen https-Zwang; wer einen
`http://`-Feed konfiguriert, tut das bewusst. `$WIKITOOL_UPDATE_TOKEN` geht nur an Downloads
auf demselben Host wie der Feed.
**Beim ersten Sprung auf ein Release, das `--latest` kennt, gibt es die Option in der Instanz
noch nicht** - die Instruktion, die dort steht, gehört zum Release, das die Instanz verlässt.
Dann Tarball und `.sha256` einmal von der Release-Seite holen, nebeneinander ablegen und
`dist upgrade <tarball>` geben; ab dem Release danach trägt die Instanz `--latest` selbst.
**Beim ersten Sprung auf `4.5.0` oder höher gibt es `dist upgrade` in der Instanz noch nicht** -
es kam erst mit `4.5.0`. Dann das Werkzeug aus dem entpackten *neuen* Tarball verwenden, gegen die
alte Instanz gerichtet:
```bash
tar -xzf chemenu-stack-<version>.tar.gz
CHEMENU_ROOT="$PWD" chemenu-stack-<version>/tools/wikitool \
dist upgrade chemenu-stack-<version>.tar.gz --dry-run
```
`CHEMENU_ROOT` sagt dem Paket, auf welchen Korpus es zeigen soll (siehe § Konfiguration); ohne die
Variable würde es den entpackten Tarball selbst für die Instanz halten. Ab dem zweiten Upgrade
trägt die Instanz Kommando und Instruktion selbst, und der normale Weg greift.
Was `dist upgrade` dabei genau tut, klassifiziert und verweigert, steht in
[tools/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/tools/CONTRACT.md) - einschließlich des vollständigen Fehlerkontrakts. Eine
lokal veränderte Stack-Datei ist damit sichtbar, statt von Hand gegen die sha256-Summen im
`files`-Block geprüft werden zu müssen - genau der Schritt, der vor `4.5.0` hier stand.
`doctor` warnt, solange `kb_version` hinter `VERSION` zurückliegt und noch Migrationen offen
sind. Einer Instanz, die älter ist als `.wikitool-kb.json`, fehlt die Datei ganz - dann einmalig
`tools/wikitool migrate baseline <version>` aufrufen; geraten wird nichts.
**Fallstricke.** Eine Instanz ohne lokale `.wikitool-release.json` (oder eine ohne `files`-Block,
aus der Zeit vor `4.5.0`) hat für `dist upgrade` keine Basis, gegen die es eine lokale Änderung
erkennen könnte, und verweigert den Tausch - dafür gibt es heute keine Reparatur.
Mit `<tarball-oder-verzeichnis>` lädt der Befehl selbst nichts herunter; die Datei muss vorher
von der Release-Seite geholt werden. Nur `--latest` lädt, und ein Tarball muss in beiden Fällen genau ein
Top-Level-Verzeichnis enthalten - die Form, in der `.gitea/workflows/release.yml` es baut.
Vor `4.5.0` stand hier ein rein manueller Ablauf (Maschinerie von Hand kopieren, `kb/CONTRACT.md`
eingeschlossen, sha256-Vergleich von Hand). `dist upgrade` ersetzt genau diesen Teil; wer ihn
dennoch von Hand nachvollziehen will oder muss (ein Werkzeug, das `wikitool` selbst nicht
ausführen kann), findet die Dateiliste im `files`-Block der `.wikitool-release.json` und die
Ausnahmen (`kb/CONVENTIONS.md`, `kb/*/COLLECTION.md`, `.wikitool-kb.json`) in
[tools/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/tools/CONTRACT.md)s `dist upgrade`-Datensatz (oder direkt:
`tools/wikitool dist upgrade -h`).
## Konfiguration
| Variable | Zweck | Fallback |
|----------|-------|----------|
| `WIKI_AUTHOR` | Override für den Autornamen neuer Source-Seiten | `git config user.name` - fehlt beides, bricht `new` mit `ERROR` ab |
| `WIKITOOL_SESSION_ID` | Scopt das Iteration-Budget-Gate auf eine Aufgabe statt auf ein Terminal | Eine vom Harness selbst gesetzte Sitzungs-Variable, wo eine bekannt ist (z. B. `CLAUDE_CODE_SESSION_ID`), sonst die Parent-Process-ID (siehe [instructions/session-setup.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/instructions/session-setup.md)) |
| `WIKITOOL_UPDATE_URL` | Release-Feed, den `version check` abfragt | Wert aus `.wikitool-release.json`, sonst der Feed der Ursprungs-Instanz |
| `WIKITOOL_UPDATE_TOKEN` | Gitea-Token für den Release-Feed | keiner - gegen `torben/chemenu` nicht nötig, nur gegen einen Feed in einem nicht öffentlichen Repo |
| `WIKITOOL_TASKS_CONFIG` | Pfad zu einer Tracker-Konfiguration, die `task`, `review` und `doctor` statt `.wikitool-tasks.json` lesen - um einen Checkout der Reihe nach gegen mehrere Tracker laufen zu lassen | die `.wikitool-tasks.json` im Repo-Root. Nennt die Variable eine Datei, die es nicht gibt, ist das ein Fehler und nie „kein Tracker konfiguriert" |
| `CHEMENU_ROOT` | Auf welchen Korpus das Paket zeigt - für einen Aufrufer, der nicht im Checkout selbst liegt | der Checkout, in dem das Paket liegt (`tools/wikitool` verhält sich ohne die Variable unverändert) |
| `WIKI_TRACE` / `WIKI_TRACE_DIR` | Telemetrie abschalten bzw. aus dem Arbeitsbaum heraus umlenken | Aus - siehe unten |
| `WIKI_TRACE_MAX_SESSION_BYTES` / `WIKI_TRACE_KEEP_SESSIONS` | Byte-Deckel je Session-Trace bzw. wie viele Session-Verzeichnisse die Retention behält | 5 MiB je Session, 250 Verzeichnisse |
**Gegen das Ursprungs-Repo braucht es kein Token.** `torben/chemenu` ist öffentlich lesbar;
`version check`, `dist upgrade --latest` und die Installation funktionieren ohne Konfiguration.
**Telemetrie ist in einer Instanz aus.** Jede Instanz trägt die `.wikitool-release.json` ihres
Releases, und daran erkennt `chemenu.telemetry.policy`, dass niemand Telemetrie bestellt hat.
Das gilt auch für jeden weiteren Klon der Instanz, weil die Datei mit dem ersten Commit ins Repo
kommt. Nur ein Klon des Ursprungs-Repos zur Entwicklung trägt keine; dort sind die Traces das
Messinstrument, mit dem der Stack sich selbst bewertet, und sie stehen auf **an**.
Wer den Default umdrehen will, legt `.wikitool-telemetry.json` im Repo-Root an (pro Checkout,
gitignored, kein `.template` - genau wie `.wikitool-remotes.json`):
```json
{ "enabled": true, "max_session_bytes": 5242880, "keep_sessions": 250 }
```
Alle drei Schlüssel sind optional. `WIKI_TRACE` überschreibt `enabled` weiterhin in beide
Richtungen und schlägt diese Datei. `wikitool doctor` meldet den aktuellen Zustand (an/aus,
warum, und die Menge gegen beide Deckel); mehr dazu in [EVALS.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/EVALS.md) § "Whether it
runs at all".
**Tool-Pfade - schreibt der Preflight, nicht der Mensch.** `.wikitool-tools.json` im Repo-Root
hält die absoluten Pfade von Python, git und ripgrep, so wie der Preflight sie auf diesem Rechner
gefunden hat; `wikitool` startet git und rg von dort statt über `PATH`. Pro Checkout und
gitignored - ein Pfad auf einem Rechner sagt über den nächsten nichts. Fehlt die Datei oder ist
sie unvollständig, startet `tools/wikitool` nicht (Exit 42) und nennt den Preflight. Liegt ein
Tool woanders, als der Preflight sucht, nennt man den Pfad mit
`tools/preflight.sh --set rg=<pfad>` (PowerShell: `pwsh -NoProfile -ExecutionPolicy Bypass -File tools/preflight.ps1 --set rg=<pfad>`); von Hand
bearbeitet wird die Datei nicht. `doctor` meldet
unter `tool-paths`, ob alle Pfade noch stimmen.
**Aufgaben-Tracker anbinden - optional.** Der Wochenrückblick (`tools/wikitool review`, Skill
`gtd-weekly-review`) gleicht die Projektseiten unter `kb/gtd/` gegen einen Aufgaben-Tracker ab. Welcher
das ist, steht in `.wikitool-tasks.json` im Repo-Root - der dritten Datei dieser Art neben
`.wikitool-telemetry.json` und `.wikitool-remotes.json`: pro Checkout, ohne `.template`, und
**gitignored, sobald ein Token darin liegt**. Fehlt sie, ist schlicht kein Tracker konfiguriert;
das ist ein gültiger Endzustand, kein Fehler. Kaputt ist sie dagegen ein `FAIL` - eine
unlesbare Konfiguration darf nicht als „kein Tracker" durchgehen.
```json
{
"schema": 1,
"provider": "superproductivity",
"thresholds": {
"stalled_waiting_days": 14,
"unpaged_project_weeks": 3,
"someday_stale_months": 5
},
"superproductivity": {
"access": "api",
"api_base_url": "http://127.0.0.1:3876",
"api_token": "<token aus den SP-Einstellungen>"
}
}
```
```json
"superproductivity": {
"access": "snapshot",
"backups_dir": "~/.config/superProductivity/backups"
}
```
`provider` wählt den Adapter - ausgeliefert werden `superproductivity` und `caldav`. Der
`thresholds`-Block trägt die drei Schwellwerte des Rückblicks (Konfiguration, nicht Schema): ab
wann ein Waiting-For überfällig ist, ab welchem Alter ein Tracker-Projekt ohne `kb/`-Seite
gemeldet wird, und ab wann ein Someday-Eintrag als verstaubt gilt. Ein Wert, den der Provider
gar nicht liefern kann - ein Waiting-Posten ohne Wiedervorlagedatum, ein Tracker-Projekt ohne
bestimmbares Anlagedatum - wird vom Rückblick nicht still übersprungen, sondern als eigene
Fundstelle gemeldet (`waiting_no_follow_up`/`project_age_unknown`).
Der gleichnamige Provider-Block trägt dessen Verbindungsangaben, und bei Super Productivity
entscheidet `access` **verpflichtend und ohne Rückfall**, welcher von zwei sich ausschließenden
Wegen das ist: eine headless bediente Instanz setzt `access: "snapshot"` und liest
ausschließlich den jüngsten Backup-Schnappschuss unter `backups_dir` (läuft auch ohne laufende
App, aber rein lesend - der Tracker ist von dort aus nicht schreibbar); eine Desktop-Instanz
setzt `access: "api"` und spricht ausschließlich die lokale REST-API an, die nur antwortet,
solange die App läuft, dafür aber auch den aktuellen Zustand liefert und den Schreibpfad trägt.
Der Block nennt nur die Felder seines eigenen Wegs - ein `backups_dir` neben `access: "api"` oder
ein `api_token` neben `access: "snapshot"` wird beim Lesen der Konfiguration abgelehnt, nicht
ignoriert. `api_token` ist bei `access: "api"` Pflicht, da jeder Endpunkt außer `GET /health`
`Authorization: Bearer <token>` verlangt.
`tools/wikitool new project` legt einen gleichnamigen Tracker-Eintrag nur auf einer
`access: "api"`-Instanz an (und auch dort nicht automatisch - siehe die Kommandotabelle). Auf
einer `access: "snapshot"`-Instanz verweigert das Kommando vollständig, exit 1: der Tracker ist
von dort aus nur lesbar. Dasselbe gilt für `tools/wikitool task new`, den zweiten Schreibweg:
es legt einen einzelnen Posten im Tracker an - ohne `kb/`-Seite - und existiert ebenfalls nur
auf einer `access: "api"`-Instanz. `tools/wikitool task close --id` ist der dritte und letzte
Schreibweg - er markiert einen Posten erledigt, löscht ihn nie - und verweigert auf
`access: "snapshot"` auf dieselbe Weise. `tools/wikitool task list --project` ist rein lesend
und beantwortet daher auf beiden Zugriffsarten.
**`caldav`** ist der standardbasierte zweite Adapter (RFC 4791/5545), gegen Nextcloud Tasks
verifiziert, mit iOS *Erinnerungen* als mobilem Client - gebaut nach dem Zuschnitt: auf dem
Telefon wird abgehakt, gepflegt wird am Schreibtisch. Anders als bei Super Productivity gibt es
nur einen Zugriffsweg - CalDAV ist immer ein Netzwerkzugriff, kein `access`-Feld nötig:
```json
"caldav": {
"url": "https://<host>/remote.php/dav/calendars/<user>/",
"username": "<login>",
"app_password": "<Nextcloud-App-Passwort>",
"inbox_list": "Inbox",
"someday_list": "Someday",
"exclude_lists": ["<vorhandene Liste, die kein Projekt ist>"]
}
```
`url` zeigt auf das CalDAV-Calendar-Home-Set des Kontos; sie darf ein Alias sein (Nextcloud
akzeptiert dort einen Kurznamen), da jede spätere Adresse ausschließlich aus den vom Server
gelieferten `href`s stammt, nie aus dieser URL und einem Namen zusammengesetzt wird. `username`
ist der Login-Name, der von der Benutzer-ID in der URL abweichen kann - ein
Nextcloud-App-Passwort wird empfohlen, nicht das Kontopasswort. `inbox_list`/`someday_list`
nennen die beiden festen Listen (je genau eine pro Instanz); `exclude_lists` nimmt vorhandene
reine Aufgabenlisten heraus, die keine Projekte sind - eine Liste mit `VEVENT`-Anteil zählt
ohnehin nie als Projekt.
Ein Projekt ist dort eine Liste, deren unterstützte Komponente ausschließlich `VTODO` ist; `tools/wikitool
new project` legt sie automatisch per `MKCALENDAR` an - anders als bei Super Productivity ohne
Rückfrage, weil CalDAV einen echten Anlage-Befehl für Listen kennt. Die Eindeutigkeitsprüfung
läuft gegen **jede** Liste im Konto, auch gegen ausgeschlossene, Inbox, Someday und gemischte
Kalender - kollidiert ein neuer Name mit einer davon, wird nichts angelegt und die Kollision
genannt; die vorhandene Liste in Nextcloud umzubenennen bleibt Handarbeit. `tools/wikitool task
new`/`task close` funktionieren auf einer `caldav`-Instanz uneingeschränkt - es gibt keinen
reinen Lesemodus wie `access: "snapshot"`. Beim Abhaken ändert `task close` ausschließlich
`STATUS`, `COMPLETED`, `PERCENT-COMPLETE`, `LAST-MODIFIED` und `DTSTAMP` an der bestehenden
`.ics`-Ressource; jede andere Eigenschaft - auch eine unbekannte `X-`-Eigenschaft oder ein Alarm
- bleibt unverändert erhalten, und eine seit dem Lesen veränderte Ressource (ETag-Konflikt)
schreibt nichts und bricht mit exit 1 ab. Gelöscht wird nie etwas.
Listen abgeschlossener Projekte bleiben nach dem Archivieren bestehen - der Adapter löscht nie
eine Liste; das übernimmt der Betreiber von Hand in Nextcloud, sobald gewünscht.
## Verifikation
## Check the instance
```bash
tools/wikitool doctor
```
Prüft in einem Aufruf: Abhängigkeiten, Autor-Auflösung, Git-Identität/Branch/Remote,
publizierte Skills, Struktur (Collection-Contracts, generierte Dateien), Personalization
(`USER.md`/`SOUL.md` vorhanden **und** ausgefüllt), die optionale Umgebungsnotiz
(`ENVIRONMENT.md`), den Aufgaben-Tracker (`.wikitool-tasks.json` - fehlt sie, ist das `OK`; ist
ein Provider konfiguriert, zusätzlich ob sein Lesepfad bereitsteht und seine API gerade
antwortet, beides nie ein `FAIL`) und die Session-ID.
`OK`/`WARN` sind unbedenklich (ein fehlender Remote z. B. ist ein gültiger Endzustand); nur ein
`FAIL` bricht mit exit 1 ab, und jede Zeile nennt ihr eigenes Fix-Kommando.
This checks in one call: dependencies, author resolution, git identity/branch/remote, published
skills, structure (collection contracts, generated files), personalization (`USER.md`/`SOUL.md`
present **and** filled in), the optional environment note (`ENVIRONMENT.md`), the task tracker
(`.wikitool-tasks.json` - if it is missing, that is `OK`; if a provider is configured, also
whether its read path is ready and its API is answering right now, neither ever a `FAIL`) and the
session id. `OK`/`WARN` are harmless (a missing remote, for example, is a valid end state); only a
`FAIL` ends with exit 1, and every line names its own fix command.
Danach zusätzlich:
Then also:
```bash
tools/wikitool docs verify
tools/wikitool instructions verify
```
## Troubleshooting
## Next steps
- **`tools/wikitool` endet mit `STOP - this checkout is not set up yet` (Exit 42)** - der
Preflight ist in diesem Checkout noch nicht durchgelaufen, oder seit dem letzten Update nicht
mehr: `tools/preflight.sh` ausführen, siehe
[instructions/preflight.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/instructions/preflight.md). In PowerShell 7 unter Windows heißt der
Aufruf `pwsh -NoProfile -ExecutionPolicy Bypass -File tools/preflight.ps1`; das
`-ExecutionPolicy Bypass` gilt nur für diesen einen Prozess und ändert keine Einstellung. Was
seine Ausgabe dann bedeuten kann, steht unter [Wenn der Agent anhält](#wenn-der-agent-anhält).
- **Der Agent bietet keine Skills an (`wiki-ingest`, `wiki-query`, ...)** - `.agents/skills/`
und `.claude/skills/` sind generiert und nicht committet. `tools/wikitool instructions sync`
ausführen, dann die Agent-Session neu starten (Harnesses lesen Skills nur beim Start).
- **`doctor` meldet `personalization: FAIL`** - `USER.md`/`SOUL.md` fehlen, oder sie tragen
noch die Sentinel-Zeile aus dem Template (ein umbenanntes Template ist kein ausgefülltes).
Den Personalisierungs-Schritt aus `instructions/setup-instance.md` ausführen lassen; bei einem
weiteren Klon einer älteren Instanz ist das der einzige nachzuholende Schritt.
- **`doctor` meldet `environment: WARN`** - `ENVIRONMENT.md` existiert, trägt aber noch die
Sentinel-Zeile aus dem Template. Ausfüllen (Vorlage: `ENVIRONMENT.md.template`) und die Zeile
entfernen, oder die Datei löschen - sie ist optional, und `absent` ist ein gültiger
Endzustand.
- **`new` bricht mit "No author configured" ab** - weder `$WIKI_AUTHOR` noch
`git config user.name` sind gesetzt. `git config user.name "<Name>"` ausführen, oder
`WIKI_AUTHOR` exportieren.
- **`publish` endet mit Exit-Code 42 (Mass-Update-Gate)** - erwartetes Verhalten bei ≥10
gezählten Dateien (z. B. beim allerersten Commit einer neuen Instanz). Das ist kein Fehler,
sondern die Aufforderung, die Ausgabe einem Menschen zu zeigen: sie enthält die vollständige
Dateiliste und die exakte `--confirm <token>`-Zeile, die nach Freigabe veröffentlicht.
Details: [instructions/gates.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/instructions/gates.md).
- **`publish` endet mit Exit-Code 42 (Publish-Remote-Gate)** - dieser Checkout hat eine
`.wikitool-remotes.json`, und das angesteuerte Remote steht nicht darin. Ebenfalls kein
Fehler: Die Ausgabe nennt die Push-URL, an die geschrieben würde, und die erlaubten. Anders
als beim Mass-Update-Gate gibt es hier **keinen Token und keine Flagge** - stimmt das Ziel
wirklich, trägt der Mensch dessen URL selbst in die Datei ein. Ein Agent, der die Datei
anfasst, um an der Verweigerung vorbeizukommen, öffnet ein Gate aus eigenem Antrieb.
- **Setup oder Update schlägt fehl und ich will es melden** - den Agenten
[instructions/bug-report.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v8.0.0/instructions/bug-report.md) ausführen lassen, oder direkt
`tools/bugreport` aufrufen (aus Bash, Git Bash und PowerShell gleich). Der Aufruf sucht sich
selbst ein Python 3.8 oder neuer und überspringt dabei die Store-Aliase, auf die `python3`
unter Windows zeigt. Er braucht weder den Preflight noch ein funktionierendes `wikitool`, und
findet er kein Python, sagt er, warum. Das Skript schreibt ein Bündel nach
`reports/bugreport-<Zeitstempel>/` samt Zip. Geheimnisse werden entfernt, und aus allem, was das Skript selbst erzeugt, bleiben
Seitentitel draußen (`--titles` nimmt sie mit). Der Sitzungs-Trace ist standardmäßig dabei
(`--no-trace` lässt ihn weg) und kann wie Chronologie und Transkripte Seiteninhalt und Titel
enthalten; das Bündel enthält außerdem Maschinen-, Benutzer- und Pfadnamen. Mit `--pseudonymise`
ersetzt das Skript Benutzer-, Host-, Pfad-, Git- und Remote-Namen durch Platzhalter, die Länge,
Leerzeichen, Bindestriche und Pfadtiefe erhalten; der Agent kann danach weitere Namen
(Personen, Firmen, Kunden, Projekte) mit `--bundle … --candidates …` nachtragen. Das ist das
Urteil eines Modells und lässt einen Rest übrig - lies das Bündel vor dem Teilen. Die
Zuordnung, die Prüfliste und die Kandidatendatei enthalten Originale und liegen neben, nie im
Bündel. Es wird nirgends hochgeladen - den Kanal wählst du selbst.
- **Your first source and your first answer:**
[docs/tutorials/first-source-to-first-answer.md](Tutorials-First-source-to-first-answer)
walks you through both in the instance you just set up.
- **Updating** to a newer release: [docs/how-to/update-an-instance.md](How-to-Update-an-instance).
- **Configuration** - environment variables and the files per checkout:
[docs/reference/configuration.md](Reference-Configuration).
- **When something does not work:** [docs/how-to/troubleshoot.md](How-to-Troubleshoot).
- **Working with the wiki:** [docs/how-to/](How-to).
Reference-Commands-Catalog-and-log.md
+159
@@ -0,0 +1,159 @@
*Generated from [`docs/reference/commands/catalog-and-log.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/docs/reference/commands/catalog-and-log.md) at v9.0.0 – edit the repository, not this wiki.*
# Catalog and log
The generated catalog of the wiki and its append-only operations log.
<!-- wikitool:toc -->
## Contents
- [`index rebuild`](#index-rebuild)
- [`log append`](#log-append)
- [`log status`](#log-status)
<!-- /wikitool:toc -->
<!-- wikitool:reference-commands -->
## `index rebuild`
Regenerate the catalog from every page's frontmatter.
**Synopsis**
- `wikitool index rebuild [--dry-run]`
**Options**
```text
--dry-run Print what would be written instead of writing it
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool index rebuild
tools/wikitool index rebuild --dry-run
```
**Behaviour**
- Rewrites `kb/index.md` as a map: statistics, one row per collection and per area, and links to the shards - no page rows.
- Writes the page tables to a generated `INDEX.md` in each collection. An area with more than 50 rows gets its own `INDEX.md` in its directory.
- Deletes stale shards - an `INDEX.md` of a collection or area that no longer exists - in the same pass.
- Warns about every page nested more than one directory below its collection; it is still catalogued, folded into its area.
- `--dry-run` prints every file it would write and every stale shard it would remove, and writes nothing.
**Properties**
- Changes files: yes
- Idempotent: yes
- Atomic: No - each catalog file (the map plus one shard per collection/area) is rewritten independently, then stale shards are removed; an interruption can leave some regenerated and others not
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - An I/O error while writing or removing a catalog file (rare)
**See also**
- `wikitool search` - finds a page without reading the catalog
- `wikitool lint` - its Nested Pages finding is what the warning previews
- `instructions/publish-cycle.md` - where a write session runs this
## `log append`
Append a formatted entry to `kb/log.md`.
**Synopsis**
- `wikitool log append --op ingest|query|lint|create|update|delete|rename|move --title "..." [--body "..."|--body-file path]`
**Options**
```text
--op <str> ingest|query|lint|create|update|delete|rename|move
[required]
--title <str> Brief description, e.g. a source path [required]
--body <str> Optional multi-line details
--body-file <path> Read the body from a file instead of --body
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool log append --op ingest --title "raw/articles/docker-cheatsheet.md" --body "Created [[Docker]]; updated [[Container]]."
tools/wikitool log append --op lint --title "2026-09-26" --body-file lint-summary.md
```
**Behaviour**
- Appends one entry to `kb/log.md`: a `## [YYYY-MM-DD] <op> | <title>` heading, the body if one is given, and a `---` separator.
- Not idempotent: every successful run appends a new entry, including a repeated one.
**Properties**
- Changes files: yes
- Idempotent: no
- Atomic: Yes - single append
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - `--op` is not one of ingest, query, lint, create, update, delete, rename, move
- `1` - `--body-file` is missing, not a readable file, or not valid UTF-8
**See also**
- `wikitool log status` - counts the ingests logged since the last lint
- `instructions/publish-cycle.md` - where a write session runs this
## `log status`
Read-only: count `ingest` entries logged since the last `lint` entry.
**Synopsis**
- `wikitool log status`
**Options**
```text
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool log status
```
**Behaviour**
- Counts the `ingest` entries in `kb/log.md` after the most recent `lint` entry, or since the start of the log if it was never linted, and the total number of entries.
- At 10 or more it prints that the `wiki-lint` skill is due next - the every-10-sources full lint of the Maintenance Schedule.
- Read-only; safe to retry freely.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `0` - `kb/log.md` is missing or empty - reported as nothing logged, not a failure
**See also**
- `wikitool log append` - writes the entries this counts
- `wiki-lint` skill - what the threshold asks for
- `tools/CONTRACT.md` § Maintenance schedule - the cadence this reports on
<!-- /wikitool:reference-commands -->
Reference-Commands-Content-migrations.md
+269
@@ -0,0 +1,269 @@
*Generated from [`docs/reference/commands/content-migrations.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/docs/reference/commands/content-migrations.md) at v9.0.0 – edit the repository, not this wiki.*
# Content migrations
The migrations a release may require of an instance's content, and recording that they ran.
<!-- wikitool:toc -->
## Contents
- [`migrate list`](#migrate-list)
- [`migrate status`](#migrate-status)
- [`migrate verify`](#migrate-verify)
- [`migrate done`](#migrate-done)
- [`migrate baseline`](#migrate-baseline)
<!-- /wikitool:toc -->
<!-- wikitool:reference-commands -->
## `migrate list`
List every migration document under `instructions/migrations/`.
**Synopsis**
- `wikitool migrate list [--json]`
**Options**
```text
--json Print the migrations as JSON
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool migrate list
```
**Behaviour**
- Lists every migration document under `instructions/migrations/`, oldest target first, with its kind and obligation.
- Never fails. Read-only and **exempt from the Iteration Budget Gate**.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: no
- Can reach the network: no
**Exit codes**
- `0` - success
**See also**
- `wikitool migrate status` - which of them this instance still owes
- `instructions/migrate-corpus.md` - how a migration is run
## `migrate status`
Show the migrations this instance still owes, in the order they must run.
**Synopsis**
- `wikitool migrate status [--json]`
**Options**
```text
--json Print the chain as JSON
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool migrate status
tools/wikitool migrate status --json
```
**Behaviour**
- Shows every **required** migration whose `migrates_to` lies in `(kb_version, VERSION]`, in the order it must run.
- `offered` migrations are listed separately above the chain: they never block, never count as owed, and are bounded by the applied ledger rather than by `kb_version` - taking one does not move the version.
- With a release stamp present, also reports which shipped files this instance has since edited (from the per-file sha256 in `.wikitool-release.json`) - which says whether an offer may be copied over or has to be reconciled by hand. Without a stamp that question is reported as unanswerable rather than answered.
- Exits 1 only when the content version is undeclared (`.wikitool-kb.json` missing) or `VERSION` is unreadable; it never guesses the content's shape.
- Read-only, safe to retry freely, and exempt from the Iteration Budget Gate.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: no
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - `.wikitool-kb.json` is missing - the content version is undeclared
- `1` - `VERSION` is unreadable
**See also**
- `wikitool migrate done` - records one as applied
- `instructions/migrate-corpus.md` - how a migration is run
- `instructions/upgrade-instance.md` - where an upgrade checks this
## `migrate verify`
Compare `kb/` against a git revision on the invariants a content migration must not change.
**Synopsis**
- `wikitool migrate verify --from <rev> [--path P ...] [--expect-body-change] [--json] [--fail-on-error]`
**Options**
```text
--from <str> Git revision to compare against, e.g. HEAD [required]
--path <str> Limit to a subtree, repeatable (e.g. kb/concepts)
--expect-body-change Also report pages whose body did not change at all
--json Print the diff as JSON
--fail-on-error Exit 1 if any invariant changed
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool migrate verify --from HEAD
tools/wikitool migrate verify --from HEAD --path kb/concepts --expect-body-change
```
**Behaviour**
- Compares `kb/` against the revision `--from` on the invariants a content migration must not change: wikilink and citation **counts** (not sets), footnote definitions, H1, structural frontmatter, and the **count of generated-region marker pairs**.
- Pages are matched by **title**, not path, so a page `move` (or `move --reconcile`) relocated compares as itself - reported separately as `moved` - rather than as a removed-and-added pair.
- Reports added and removed pages without failing on them.
- `--expect-body-change` additionally flags a page whose body did not change at all.
- Not migration-specific: worth running after any bulk rewrite.
- Exits 0 whatever it finds unless `--fail-on-error` is passed.
- Read-only and exempt from the Iteration Budget Gate.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: no
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - Only with `--fail-on-error`: an invariant changed
- `1` - `--from` is not a revision in this repository
**See also**
- `instructions/migrate-corpus.md` - where a migration runs this
- `wikitool lint` - the single-revision checks
## `migrate done`
Record one migration as applied, advancing `kb_version` in `.wikitool-kb.json`.
**Synopsis**
- `wikitool migrate done <version> [--pages N] [--dry-run]`
**Options**
```text
Arguments:
version The migration's target version, e.g. 1.4.0 [required]
Options:
--pages <int> How many pages it touched
--dry-run Report without writing
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool migrate done 7.0.0 --pages 42 --dry-run
tools/wikitool migrate done 7.0.0 --pages 42
```
**Behaviour**
- Records one migration as applied, advancing `kb_version` in `.wikitool-kb.json` to its target.
- **Refuses any required version that is not the next link in the chain.**
- An `offered` migration is recorded in the applied ledger *without* moving `kb_version` and with no ordering rule applied. Re-recording one already in the ledger is a no-op, not an error - idempotent and safe to repeat.
- Not idempotent for a required migration: it advances the chain.
- `--dry-run` reports without writing.
**Properties**
- Changes files: yes
- Idempotent: no
- Atomic: Yes - single file write
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - Unknown version, no `.wikitool-kb.json`, or nothing outstanding
- `1` - A *required* version that is not the next link in the chain
**See also**
- `wikitool migrate status` - the order to apply them in
- `instructions/migrate-corpus.md` - the migration procedure
## `migrate baseline`
Declare `kb_version` once, for an instance predating `.wikitool-kb.json`.
**Synopsis**
- `wikitool migrate baseline <version> [--force]`
**Options**
```text
Arguments:
version The shape this instance's content is already in [required]
Options:
--force Overwrite an existing declaration (not a substitute for `done`)
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool migrate baseline 6.2.0
```
**Behaviour**
- Declares `kb_version` once, for an instance predating `.wikitool-kb.json`.
- Refuses to overwrite an existing declaration without `--force`. Advancing the version after a migration is `migrate done`, which checks the chain; this command does not.
- Safe to re-run with the same version.
**Properties**
- Changes files: yes
- Idempotent: yes
- Atomic: Yes - single file write
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - Unparseable version
- `1` - A declaration already exists and `--force` was not passed
**See also**
- `wikitool migrate done` - advances the version after a migration
- `wikitool migrate status` - what is owed from the declared version
<!-- /wikitool:reference-commands -->
Reference-Commands-Distribution-and-versioning.md
+604
@@ -0,0 +1,604 @@
*Generated from [`docs/reference/commands/distribution-and-versioning.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/docs/reference/commands/distribution-and-versioning.md) at v9.0.0 – edit the repository, not this wiki.*
# Distribution and versioning
Exporting, adopting and upgrading an instance, and the version and changelog of the stack itself.
<!-- wikitool:toc -->
## Contents
- [`dist export`](#dist-export)
- [`dist adopt`](#dist-adopt)
- [`dist upgrade`](#dist-upgrade)
- [`version show`](#version-show)
- [`version check`](#version-check)
- [`version notes`](#version-notes)
- [`version bump`](#version-bump)
- [`version regrade`](#version-regrade)
- [`version release`](#version-release)
<!-- /wikitool:toc -->
<!-- wikitool:reference-commands -->
## `dist export`
Write a contentless, distributable copy of this repo's machinery.
**Synopsis**
- `wikitool dist export <target> [--dry-run] [--source-repo U] [--source-commit SHA] [--release-url U] [--update-url U]`
**Options**
```text
Arguments:
target Directory to write the distribution into. Must not exist, or must be
empty. [required]
Options:
--dry-run List what would be written, without writing anything.
--source-repo <str> Repository this export was built from (recorded in the
stamp)
--source-commit <str> Commit this export was built from (recorded in the
stamp)
--release-url <str> Release page this export ships as (recorded in the
stamp)
--update-url <str> Release feed `version check` should ask (recorded in
the stamp)
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool dist export ../my-wiki --dry-run
tools/wikitool dist export ../my-wiki
```
**Behaviour**
- Writes a contentless, distributable copy of this repo's machinery into an empty or new `<target>` directory.
- Ships `AGENTS.md`/`README.md`/`INSTALL.md` with any `dist:strip-start`...`dist:strip-end` marker region removed, `instructions/` (minus `instructions/dev/`), `types/` (the `root: kb` page type-specs, their schemas and their subtype templates `types/<type>.<value>.md` re-keyed as `.template`, the stack's own verbatim), `docs/` verbatim (minus `docs/dev/`), `tools/` (no venv/caches, and not the dev-only test suite `tools/chemenu/tests/` with its `pytest.ini`/`.coveragerc`), the `.github/hooks/`+`.vibe/` session-tracing config plus `.claude/settings.json`, `kb/CONTRACT.md` (no pages, no areas) and `VERSION`.
- Ships `raw/` and `incoming/` as flat roots, each with a `.gitkeep` and no subdirectories. `incoming/.gitkeep` is trackable and survives becoming a git repository, so a plain clone gets the directory without any bootstrap step.
- Ships templates, never the filled files: `USER.md.template`/`SOUL.md.template`, `kb/CONVENTIONS.md.template`, and each collection's contract re-keyed as `kb/<name>/COLLECTION.md.template`. The filled `USER.md`/`SOUL.md`/`kb/CONVENTIONS.md`/`kb/<name>/COLLECTION.md`/`types/<page-type>.md`/`types/<page-type>.<value>.md` bind their instance; `find_leaks` refuses a plan carrying one.
- Writes a generated `.wikitool-release.json` stamp: version, export date, origin, and a sha256 per exported file - the base a later upgrade compares against.
- The four origin options only fill stamp fields: `export` never calls git and cannot discover them.
- A build and test tool: every release is an export packed as a tarball, and an instance is installed from such a release, never from an export directly.
- One-way: no command reconstructs a distributed instance into a dev instance - work on the stack in a clone of the origin repo.
- `--dry-run` lists every file it would write, and writes nothing.
**Properties**
- Changes files: yes
- Idempotent: yes
- Atomic: Yes - nothing is written until every file is planned
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - `<target>` exists and is not empty, or is not a directory
- `1` - The tree has no readable `VERSION`
- `1` - A licence file is missing from the source tree
- `1` - The plan would carry this instance's own data (a filled personalization, conventions or page type-spec file)
**See also**
- `instructions/setup-instance.md` - installs a release, which is this export as a tarball
- `wikitool dist upgrade` - applies a later export to an existing instance
- `wikitool version show` - reads the stamp this writes
## `dist adopt`
Take shipped templates as this instance's own: copy each to its unsuffixed name.
**Synopsis**
- `wikitool dist adopt [<template>...] [--dry-run]`
**Options**
```text
Arguments:
templates... Templates to adopt. Default: every collection contract and page
type-spec template.
Options:
--dry-run List what would be copied, without writing anything.
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool dist adopt
tools/wikitool dist adopt types/project.md.template types/project.schema.yaml.template
tools/wikitool dist adopt --dry-run
```
**Behaviour**
- Copies `<name>.template` to `<name>` byte for byte; the template stays where it is, as the base the next `dist upgrade` compares against.
- Without a path: every `kb/<collection>/COLLECTION.md.template` and every `types/*.template` (the `root: kb` page type-specs, their schemas and their subtype templates).
- With paths: exactly those templates, each of which has to be one of the set above.
- Never overwrites: a target that already exists is reported as kept and left untouched.
- Not for `kb/CONVENTIONS.md.template` or the personalization templates - those carry a sentinel and are filled in, not copied.
- `--dry-run` lists what it would copy and keep, and writes nothing.
**Properties**
- Changes files: yes
- Idempotent: yes
- Atomic: No - files are copied one by one; a re-run completes an interrupted one
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - A named path does not exist, or is not a collection contract or page type-spec template
**See also**
- `instructions/setup-instance.md` - adopts every template on a fresh instance
- `instructions/upgrade-instance.md` - adopts a template a release added
- `wikitool dist export` - re-keys these files as `.template` in the first place
## `dist upgrade`
Apply a stack update `dist export` produced - the write half of `version check`.
**Synopsis**
- `wikitool dist upgrade (<source> | --latest [--expect <version>]) [--dry-run] [--keep-local] [--take-release <path>]... [--prune] [--pre]`
**Options**
```text
Arguments:
source An extracted distribution directory, or a release .tar.gz archive - or
use --latest
Options:
--latest Download the release the feed announces as latest
instead of naming a <source>
--expect <str> With --latest: refuse, before downloading anything,
unless the feed's latest release is this version (the
one `version notes` was read for)
--dry-run Classify and report, without writing anything
--keep-local Proceed even with locally changed files - leave each one
untouched rather than aborting
--take-release <str> Overwrite this locally changed path with the release's
version, discarding the local change. Repeatable, and
each path must be one this run reports as locally
changed. The counterpart to --keep-local, which keeps
the change and reports it again on every future upgrade
--prune Accepted and ignored: files the new release no longer
ships are now deleted by default when unchanged since
install
--pre Allow a pre-release (-beta.N) source tree - release.yml
never publishes one
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool dist upgrade ../chemenu-7.1.0.tar.gz --dry-run
tools/wikitool dist upgrade ../chemenu-7.1.0.tar.gz
tools/wikitool dist upgrade ../chemenu-7.1.0.tar.gz --take-release tools/README.md
tools/wikitool dist upgrade --latest --expect 8.0.0 --dry-run
```
**Behaviour**
- Exactly one of `<source>` and `--latest` names the release. `<source>` is an already-fetched export directory or `.tar.gz` release archive and touches no network: the archive is verified against a sibling `.sha256` if one is present (a missing one is a WARN, not a block) and must unpack to exactly one top-level directory - the shape `.gitea/workflows/release.yml` packs.
- `--latest` asks the release feed (`update_url` of the local stamp, overridable with `$WIKITOOL_UPDATE_URL`; `$WIKITOOL_UPDATE_TOKEN` is sent along) which release is latest, checks that version - `--expect`, a downgrade, a pre-release without `--pre`, already installed - before any download, then downloads the archive and its `.sha256` into a scratch directory removed on every exit. The checksum is mandatory here (a release without one, or an archive that fails it, is an error, not a WARN) and the archive's own `VERSION` must equal the feed's version. The asset URLs are the feed's own `browser_download_url` values, and the token reaches an asset download only if it is on the feed's origin. The checksum protects against transfer errors, not against a feed that is itself compromised - authenticity is the trust in the feed's host.
- `--expect <version>` (only with `--latest`) pins the release the feed may announce: a different latest version is refused before anything is downloaded. Pass the version `wikitool version notes` was read for, in the dry run and in the real run alike.
- The write set is exactly the *new* `.wikitool-release.json`'s `files` block, minus what an export re-seeds from a blank template every time (`kb/log.md`, `raw/.gitkeep`) or seeds once and the instance owns from then on (`.wikitool-kb.json`, `CHANGES.md`), plus the stamp itself, always rewritten.
- Every candidate path is classified against the *local* `.wikitool-release.json`'s recorded digest: unchanged is overwritten silently, absent from the old stamp is created, and locally modified or locally deleted is **never** silently overwritten.
- Locally changed files abort the run with the full list; the abort text names the three answers with the command line filled in, and none of them is the default.
- `--keep-local` proceeds and leaves every locally changed file untouched. The new stamp is still written whole, recording the release's digest for files that were not written, so a skipped file keeps diverging and is reported again on every later run.
- `--take-release <path>` (repeatable) writes the release's version over the named path, discarding the local change, re-creates it if it was locally deleted, and deletes it if the release no longer ships it. The path then matches the stamp and stops being reported.
- The two are decided per path and compose on one call: without `--keep-local`, a locally changed path that no `--take-release` names still aborts the run.
- A path in the old stamp but not the new one is no longer part of the release. Still unchanged since installation, it is deleted, as silently as an unchanged file is overwritten; changed since, it is locally changed like any other and needs one of the same answers - `--take-release` deletes it, `--keep-local` keeps it as the instance's own file, which no later upgrade reports again. Either way it is decided in this run: the new stamp no longer names the path. What the instance owns (`kb/log.md`, `CHANGES.md`, `.wikitool-kb.json`, ...) is never a candidate, and neither is any file the old stamp does not name.
- A directory left holding nothing - at most a `__pycache__/` of `*.pyc` - once those files are deleted is removed with them, up to but never including the instance root. `--prune` is accepted and ignored; it used to switch the deletion on.
- Reports the migration chain the new machinery would owe, but never runs any of it - there is no `migrate run`.
- Reports, but does not block on, a crossed compatibility boundary.
- The local preconditions - `VERSION`, the release stamp, `.wikitool-kb.json`, no outstanding migration, a clean working tree - are checked before the source is read. Not being a git repository at all is a WARN, not a refusal.
- Never touches git - no commit, no push.
- `--dry-run` classifies and reports without writing; a pre-release (`-beta.N`) source needs `--pre`. With `--latest` it still downloads and verifies the archive - that is the only way to classify - and removes it again.
- An interrupted write is not resumed automatically: compare the tree against the printed classification and finish or revert by hand.
- The closing report names `instructions/upgrade-instance.md`, which carries the order for everything after the swap and resumes at `instructions sync`.
**Properties**
- Changes files: yes
- Idempotent: no
- Atomic: Yes for every refusal - nothing is written. Once writing starts it is a plain sequential file copy with no partial-state cleanup: an interruption mid-copy (killed process, disk full) can leave the tree part-old, part-new
- Counts toward the session call budget: yes
- Can reach the network: yes
**Exit codes**
- `0` - success
- `0` - The source's version equals the installed one - a no-op success
- `1` - Both `<source>` and `--latest`, or neither; or `--expect` without `--latest`
- `1` - Local `VERSION` missing, or no local `.wikitool-release.json` with a `files` block
- `1` - `.wikitool-kb.json` is missing
- `1` - A migration is already outstanding against the *installed* machinery
- `1` - The working tree is dirty
- `1` - `<source>` does not exist, fails its `.sha256`, or does not unpack to exactly one top-level directory
- `1` - `--latest`: the release feed cannot be reached, or answers with something that is not a release
- `1` - `--latest --expect`: the feed's latest release is another version than the one expected; nothing was downloaded
- `1` - `--latest`: the release publishes no archive or no `.sha256` under the expected name; nothing was downloaded
- `1` - `--latest`: an asset download fails, or the archive fails its `.sha256`
- `1` - `--latest`: the downloaded archive's `VERSION` is not the version the feed announced
- `1` - The source carries no `VERSION`, `.wikitool-release.json` or `files` block
- `1` - The source's version is older than the installed one, or a pre-release without `--pre`
- `1` - A `--take-release` path this run does not classify as locally changed - the one refusal a `--dry-run` also raises
- `1` - Locally changed files that neither `--keep-local` nor a `--take-release` answers for; nothing was written
**See also**
- `wikitool version check` - finds out whether an update exists
- `wikitool version notes` - the notes of the release `--expect` should name
- `instructions/upgrade-instance.md` - the order after the swap
- `docs/how-to/update-an-instance.md` - which release, whether to take it, where the tarball comes from
- `wikitool migrate status` - the migrations the report names
## `version show`
Print this instance's stack version and where it came from.
**Synopsis**
- `wikitool version show [--json]`
**Options**
```text
--json Print the version and stamp as JSON
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool version show
tools/wikitool version show --json
```
**Behaviour**
- Prints `VERSION` and where this tree came from: "development tree" without a release stamp, otherwise the export date, source commit and repository from `.wikitool-release.json`, plus the release page when the stamp records one.
- `--json` prints the version, its compatibility key, the stamp and the update URL.
- Bare `wikitool version` is an alias for this.
- Read-only and offline; exempt from the Iteration Budget Gate.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: no
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - `VERSION` is missing or unparseable
**See also**
- `wikitool version check` - asks whether a newer stack exists
- `wikitool version notes` - prints a version's release notes
## `version check`
Ask the origin's release feed whether a newer stack exists.
**Synopsis**
- `wikitool version check [--url U] [--timeout S] [--json]`
**Options**
```text
--url <str> Release feed to ask (default: the stamp's, else the built-
in origin)
--timeout <float> Seconds to wait for the feed [default: 10.0]
--json Print the result as JSON
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool version check
tools/wikitool version check --json
```
**Behaviour**
- Asks the release feed for its latest release and compares it with `VERSION`: `state` is `current`, `update`, `migration` (the step crosses a compatibility boundary) or `ahead`.
- The only command whose whole job is the network call - `version notes` reaches the same feed too, but only as a fallback on a distributed instance, and `dist upgrade --latest` asks it which release to download.
- Never reached implicitly from another command (`dist upgrade` asks only when passed `--latest`), needs no key, and times out after `--timeout` seconds (default 10).
- The feed is `--url`, else `$WIKITOOL_UPDATE_URL`, else the release stamp's, else the built-in origin. `$WIKITOOL_UPDATE_TOKEN` is only needed if that feed is not readable anonymously.
- For `update` or `migration` it prints that applying the release is a separate, manual step (`docs/how-to/update-an-instance.md`).
- Read-only; exempt from the Iteration Budget Gate.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only, no local writes
- Counts toward the session call budget: no
- Can reach the network: yes
**Exit codes**
- `0` - success
- `1` - `VERSION` is missing or unparseable
- `1` - The feed could not be reached, answered non-JSON, or carried no `tag_name`
**See also**
- `wikitool dist upgrade` - applies the update this reports
- `wikitool version notes` - prints the release notes
- `docs/how-to/update-an-instance.md` - what the operator decides before an update
## `version notes`
Print one version's release notes.
**Synopsis**
- `wikitool version notes [--version X.Y.Z] [--offline] [--url U] [--timeout S]`
**Options**
```text
--version <str> Which entry to print (default: this tree's VERSION)
--offline Never ask the release feed: on a distributed instance,
whose CHANGES.md carries no entry to print, fail with the
release page instead of fetching the notes
--url <str> Release feed to ask for the fallback (default: the stamp's,
as `version check`)
--timeout <float> Seconds to wait for the feed [default: 10.0]
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool version notes
tools/wikitool version notes --version 7.0.0
tools/wikitool version notes --offline
```
**Behaviour**
- Prints the `CHANGES.md` entry for `--version` (default: this tree's `VERSION`).
- With no such entry, in a tree with a release stamp (a `dist export` tree) and without `--offline`, it prints the release feed's latest release notes instead. A tree without a stamp (a dev checkout) never asks the feed.
- Only the feed's latest release can be asked for. When that is a different version than requested, stderr names it and the notes are printed anyway - the expected case before an upgrade, where `VERSION` still names the release being left.
- stdout carries nothing but the notes; the line naming the feed being asked and the one naming the release that answered go to stderr. `release.yml` redirects stdout into the file it posts as the release body.
- `--offline` never asks the feed and fails with the stamp's `release_url` instead.
- Every failure names the stamp's `release_url` where it has one, so a run that cannot read the notes is still told where they are.
- Read-only, safe to retry, and exempt from the Iteration Budget Gate.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: no
- Can reach the network: yes
**Exit codes**
- `0` - success
- `1` - `--version` is unparseable, `VERSION` is unreadable when `--version` is omitted, or `CHANGES.md` is missing
- `1` - No entry for the requested version in a tree with no release stamp (a dev checkout)
- `1` - No entry for the requested version, and `--offline` was passed
- `1` - No entry for the requested version, and the feed could not be reached or returned a release with an empty `body`
**See also**
- `wikitool version check` - asks the same feed whether a newer stack exists
- `wikitool version bump` - writes the entry this prints
## `version bump`
Raise or continue the one running candidate between two releases.
**Synopsis**
- `wikitool version bump --major|--minor|--patch --title "<...>" [--impact high|medium|low] [--breaking "<what breaks>"] [--no-migration "<reason>"] [--migration-required] [--dry-run]`
**Options**
```text
--major Bump MAJOR (resets MINOR and PATCH)
--minor Bump MINOR (resets PATCH)
--patch Bump PATCH
--title <str> One-line title for the new/updated CHANGES.md entry
[required]
--breaking <str> What stops working (recorded in CHANGES.md). Required on
the bump that first escalates to a boundary crossing,
optional on a later bump of the same crossing candidate
- where it joins the reasons already recorded rather
than replacing them. Refused on a bump that crosses
nothing
--no-migration <str> Why the escalation to a boundary crossing needs no
content migration (recorded in CHANGES.md)
--migration-required Retract this candidate's earlier --no-migration line: a
migration is needed after all. Requires a migration
document already targeting the new base, and refuses
when the entry carries no --no-migration line to
retract.
--impact <str> high|medium|low - how much this bump matters to a reader
of the release notes (default: medium). Grouped into the
entry's bump list; `version regrade` corrects it later
if the running candidate's own judgment changes.
--dry-run Report the change without writing
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool version bump --patch --title "Fix the lint report path" --impact low
tools/wikitool version bump --minor --title "New command: wikitool review" --dry-run
tools/wikitool version bump --major --title "Rename --gate-file to --remotes-file" --breaking "--gate-file is gone; scripts must pass --remotes-file" --no-migration "no content changes"
```
**Behaviour**
- `VERSION` gets a `-beta.N` suffix: one running candidate between two releases, never a fresh number per bump.
- `--major`/`--minor`/`--patch` is **max-wins escalation** against the last release (patch < minor < major): a `--patch` on a MINOR candidate only advances `N`, and escalation never steps back down.
- The first bump of a candidate opens its `CHANGES.md` entry - heading, date, author, and a machine-managed `<!-- wikitool:bumps -->` list of every `--title` collected so far, graded by `--impact` (default `medium`). Every later bump of the same candidate updates that entry in place: one entry per candidate, not one per bump.
- The bump list renders grouped under `**High/Medium/Low impact**` headings, empty groups omitted - except while every bump is `medium`, where it stays one flat list. `version regrade` corrects a grade after the fact.
- Writes the heading, the bump list and the breaking/migration lines; the entry's prose - a summary and one-paragraph `###` sections, one per topic - is left to the author.
- Whenever a required migration document (`migrates_to:` equal to the candidate's base) exists, every bump writes or refreshes `**Migration:** required - <paths>`, the documents' paths sorted and comma-separated, in place of any `none required` line. The two forms never stand side by side.
- Compatibility follows the **leftmost non-zero component** of the candidate's base, which for this stack (at `1.0.0` and up) means MAJOR: PATCH is a fix, MINOR a compatible capability, MAJOR a version that is **not a drop-in replacement** - any hand-work on update, or a downgrade that no longer works. Whether content must be migrated is a second, independent question.
- The bump that first escalates a candidate past the boundary requires `--breaking "<what stops working>"` and, on top of it, a migration document targeting the candidate's base or `--no-migration "<reason>"`. Both are anchored just above the bump list, persist over later bumps of the same candidate without being repeated, and are refused on a bump that crosses nothing at all.
- On a later crossing of the same candidate, a further `--breaking` **joins** the reasons already recorded (flat on the marker line while there is one, bullets under a bare marker from the second on; repeating a reason verbatim is a no-op), while a further `--no-migration` **replaces** the single migration line.
- `--migration-required` replaces the running candidate's `--no-migration` line with the `required` line; it needs a required migration document already targeting the new base. Nothing retracts a recorded `--breaking` reason.
- Enforces that a crossing documents itself, never that the part was chosen correctly.
- Not idempotent: every successful run escalates or continues the candidate again.
- `--dry-run` reports the step without writing.
**Properties**
- Changes files: yes
- Idempotent: no
- Atomic: No - `VERSION` then `CHANGES.md`
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - Not exactly one of `--major`/`--minor`/`--patch`, an empty `--title`, or an unknown `--impact`
- `1` - `VERSION` or `CHANGES.md` is missing, or `VERSION` and the changelog's newest entry name different versions
- `1` - An escalation to a boundary crossing without `--breaking`, or with neither a migration document targeting the new base nor `--no-migration`
- `1` - `--breaking` or `--no-migration` on a bump that crosses nothing
- `1` - `--no-migration` while a required migration document targets the new base
- `1` - `--migration-required` combined with `--no-migration`, on a bump with no running candidate, with no `--no-migration` line to replace, or without a required migration document already targeting the new base
**See also**
- `wikitool version regrade` - corrects an `--impact` grade
- `wikitool version release` - fixes the candidate into a release
- `instructions/migrate-corpus.md` - how a migration document is written
## `version regrade`
List the running candidate's bump titles with their impact grade, or change one or more of them.
**Synopsis**
- `wikitool version regrade [INDICES...] [--impact high|medium|low]`
**Options**
```text
Arguments:
indices... 1-based positions in the rendered bump list to regrade (see the
bare listing). Omit to just list.
Options:
--impact <str> high|medium|low - required together with indices
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool version regrade
tools/wikitool version regrade 3 7 --impact high
```
**Behaviour**
- Without arguments: lists the running candidate's bump titles with their grade, numbered by 1-based position in the rendered list (High before Medium before Low, chronological within a grade). Read-only and exempt from the Iteration Budget Gate.
- With indices and `--impact`: sets the grade of every named position in one call, all resolved against a single read of the current list - `version regrade 3 7 --impact high` grades what is at 3 and 7 now, not 7 after 3 has moved. Writes `CHANGES.md` and is counted by the Iteration Budget Gate.
- The correction path for an `--impact` grade judged at bump time.
- Touches only the topmost entry's bump list - never `VERSION`, never any other part of `CHANGES.md`.
- A write is not idempotent against a changed list: after a first success, the same indices may name different bumps.
**Properties**
- Changes files: yes
- Idempotent: no
- Atomic: No - `CHANGES.md` only, and only when indices are given
- Counts toward the session call budget: only when called with arguments
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - `VERSION` or `CHANGES.md` is missing, `VERSION` and the changelog's newest entry name different versions, or the topmost entry has no bump list
- `1` - An index outside the rendered list's range, indices given without `--impact`, or an unknown `--impact`
**See also**
- `wikitool version bump` - sets the grade in the first place
- `wikitool version release` - refuses without a summary above this list
## `version release`
Fix the running candidate: strip `VERSION`'s `-beta.N` suffix and close its `CHANGES.md` entry.
**Synopsis**
- `wikitool version release [--title "<...>"] [--dry-run]`
**Options**
```text
--title <str> Replace the entry's heading title (default: the last bump's)
--dry-run Report the change without writing
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool version release --dry-run
tools/wikitool version release --title "Command records rewritten for agents"
```
**Behaviour**
- Strips `VERSION`'s `-beta.N` suffix - the candidate's base becomes the release - and closes the candidate's `CHANGES.md` entry.
- Without `--title` the heading keeps whichever bump last set it; `--title` replaces it - the normal case for a candidate that collected several bumps, whose entry wants a summarising heading rather than the most recent one.
- Leaves the entry's machine-managed bump list untouched, as the record of what happened.
- From two bumps on, requires a summary paragraph (at least 200 non-whitespace characters) between the bump list and the first `###` section heading. A candidate with exactly one bump is exempt, since there its own paragraph already is the summary. `--dry-run` runs this check too.
- Refuses an entry that would exceed its budget once released: at most 32 000 bytes (UTF-8) as `version notes` prints it, and every `###` section exactly one paragraph of at most 1 000 bytes, heading line excluded. The same check as `docs verify`, run on the released heading; `--dry-run` runs it too.
- Commits nothing and pushes nothing. The following `publish` moves `VERSION` onto `main`, which `release.yml` reacts to.
- Not idempotent: a second run fails once the suffix is gone.
**Properties**
- Changes files: yes
- Idempotent: no
- Atomic: No - `VERSION` then `CHANGES.md`
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - `VERSION` is already a release - there is no running candidate to fix
- `1` - `VERSION` or `CHANGES.md` is missing, or `VERSION` and the changelog's newest entry name different versions
- `1` - Two or more bumps and no summary paragraph above the `###` sections
- `1` - The entry is over 32 000 bytes, or a `###` section has more than one paragraph or a paragraph over 1 000 bytes - the error names each with its size
**See also**
- `wikitool version bump` - opens and continues the candidate
- `wikitool version regrade` - shows the bump list the summary sits under
- `wikitool publish` - moves the released `VERSION` onto `main`
<!-- /wikitool:reference-commands -->
Reference-Commands-Finding-and-checking.md
+216
@@ -0,0 +1,216 @@
*Generated from [`docs/reference/commands/finding-and-checking.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/docs/reference/commands/finding-and-checking.md) at v9.0.0 – edit the repository, not this wiki.*
# Finding and checking
Searching the wiki, checking its health, and preparing the weekly review of projects and commitments.
<!-- wikitool:toc -->
## Contents
- [`lint`](#lint)
- [`search`](#search)
- [`review`](#review)
<!-- /wikitool:toc -->
<!-- wikitool:reference-commands -->
## `lint`
Run structural lint checks against kb/.
**Synopsis**
- `wikitool lint [--json] [--markdown out.md] [--full] [--fail-on-error]`
**Options**
```text
--json Print the raw findings as JSON and write no report
--markdown <path> Write the markdown report here instead of the default
reports/Lint Report <date>.md
--full Print the whole report instead of only the sections with
findings
--fail-on-error Exit non-zero if hard errors were found
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool lint
tools/wikitool lint --json
tools/wikitool lint --fail-on-error
```
**Behaviour**
- Structural and provenance checks over `kb/`: broken wikilinks, wikilinks wrapped across a line break, dangling frontmatter references, orphan pages, index drift, schema gaps, duplicate titles, title mismatches, uncovered raw files, broken `raw_files:` refs, raw files claimed by more than one source page, unmarked provenance, citation/frontmatter drift, and unbalanced generated-region markers.
- Unportable Titles is a hard finding, and hard at every `kb_version`: a page whose title is not a valid file name on Windows and macOS (forbidden character, reserved name, trailing dot or space), or that collides with another page by case or Unicode normalization. `wikitool rename` is the fix.
- Wrapped Wikilinks is a hard finding: a `[[...]]` with a line break inside it. The link graph reads it as the title it folds to (the break and its indentation become one space), so it is not also a broken link unless that title is missing; the fix is to put it back on one line.
- Broken Anchors is advisory: a `[[Page#Section]]` whose page exists but has no heading the anchor names (any level, compared without case, inline-code backticks or extra whitespace; every segment of a nested `[[Page#A#B]]`). The link still reaches the page, so nothing else reports it - typically a section that was promoted to a page of its own or renamed. A missing page is Broken Wikilinks instead.
- Pages nested more than one directory below their collection are a hard finding - the generated catalog folds these into their area silently rather than merely reading it.
- Edges whose label is missing or not authorised by the source collection's `outbound:` are both hard once `kb_version` has reached the release that introduced labelled edges, and advisory below it.
- Advisory only: `see-also` edges whose reverse direction already carries a specific label - never migration-gated.
- Advisory only: a collection past the catalog's per-area shard threshold that has no areas to shard, reported with the split its subtype field would produce, and only when that split puts every resulting area at or under the threshold.
- Advisory only: source pages sitting in the `unclassified` catalog slot.
- Advisory only: Long Paths - a file under `kb/` or `raw/` whose path below the instance root is over 160 UTF-16 code units, the budget that keeps a Windows checkout without long paths working. Reported as `{path, length}`; a corpus over the budget breaks no lint run. `wikitool rename` is the fix for a page.
- Advisory only: quote-limit overages (>2 blockquotes/page).
- Advisory only: Unfilled Template Sections - a page with at least one `##` section whose non-blank lines are all template placeholders (content starting with `TODO`, after an optional list marker, checkbox, table cell or bold field label), reported as `{page, sections}`. Code, generated regions and footnote definitions do not count; an empty section, or one placeholder beside written lines, is no finding. Writing the section from the page's sources or retiring the page is the fix - not a mechanical one.
- Prints only the sections that found something and always writes the full report to `reports/Lint Report <date>.md` (or `--markdown`), naming the path. `--full` prints everything; `--json` prints the findings and writes nothing.
- Exits 0 whatever it finds unless `--fail-on-error` is passed.
**Properties**
- Changes files: yes
- Idempotent: yes
- Atomic: Writes one report file (single atomic write) unless `--json`
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - Only with `--fail-on-error`: hard findings exist
**See also**
- `wiki-lint` skill - the procedure that runs this
- `wikitool move --reconcile` - fixes Misplaced and Nested Pages
- `wikitool rename` - fixes Unportable Titles and, for a page, Long Paths
- `wikitool log status` - whether a full lint is due
## `search`
Find pages in `kb/` by text and/or frontmatter.
**Synopsis**
- `wikitool search ["<text>"] [--field <predicate> ...] [--kind/--subtype/--collection/--tag <v>] [--regex] [--limit N] [--sort [-]<field>] [--backend <name>] [--matches] [--json]`
**Options**
```text
Arguments:
text Text to search for. Omit it to run a pure frontmatter query.
Options:
-f, --field <str> Frontmatter predicate, repeatable (AND). Forms:
field=value, field~substring, 'field>=value', 'field:*'
(present), '!field' (absent).
--kind <str> Shorthand for --field kind=<value>.
--subtype <str> Shorthand for --field subtype=<value>.
--collection <str> Shorthand for --field collection=<value>.
--tag <str> Shorthand for --field tags=<value>.
--regex Treat the query as a regex. Off by default: terms are
literal.
--limit <int> Maximum number of results. 0 for no limit. A capped result
says so. [default: 50]
--sort <str> Sort by a result field; prefix with '-' to reverse, e.g.
-modified.
--backend <str> Search backend(s), comma-separated. Default 'rg' (or
$WIKITOOL_SEARCH_BACKEND).
--matches Print the matching lines under each result.
--json Print the results as JSON.
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool search "act runner"
tools/wikitool search --field entity_type=system --field '!sources'
tools/wikitool search "docker" --collection entities --limit 10 --json
```
**Behaviour**
- Finds pages in `kb/` without reading the catalog. Text search runs through a pluggable backend (`rg` today).
- `--field` predicates are evaluated on frontmatter - `f=v`, `f~substring`, `'f>=v'`, `'f:*'` (present), `'!f'` (absent) - repeatable and ANDed. With no text this is a pure structured query.
- One hit per line, ` | `-separated as `score | kind/subtype | title | path | summary`. **Title and path are never truncated** - the title is the identifier `touch`/`xref`/`cite` take. The summary, the one lossy field and the only one that may contain the separator, goes last, so splitting on `" | "` with `maxsplit=4` is unambiguous.
- Scope is pages: the backend walks `kb/` but drops the kb-root meta files, every `COLLECTION.md` and every generated `INDEX.md` - a hand-run grep over `kb/` can add none of them but those.
- `--limit` defaults to 50 (`0` for no limit), and **a truncated result says so**: `50 of 182 result(s)` in the table, `total`/`truncated`/`limit` beside `count` in `--json`, where `count` stays the number of results in the payload. `api.search` and the MCP `search` tool carry the same default and the same fields.
- A page whose frontmatter does not parse can match no positive predicate, so it is **named** rather than dropped: `--json` always carries an `unreadable` list of `{path, reason}` (usually empty), and the table form writes the same lines to stderr.
- An unknown field name is reported with the list of fields that do exist - never answered with an empty result, which would read as "no such pages".
- `--regex` is applied by `rg` alone, whose engine is linear; the ranking boosts for title and summary are literal-containment only, so a non-literal pattern is ranked by match count.
- `rg` is killed after 30 s and reported as a failure.
- Read-only, and **exempt from the Iteration Budget Gate**.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: no
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - `rg` is not installed
- `1` - `rg` did not finish within 30 s
- `1` - A malformed `--field` predicate, or an unknown `--backend`
- `1` - An unknown field name; the error lists the fields that exist
**See also**
- `wikitool links show` - the edges around a page once found
- `wiki-query` skill - answering a question from the wiki
## `review`
The GTD weekly review.
**Synopsis**
- `wikitool review [--json]`
**Options**
```text
--json Print the findings as JSON.
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool review
tools/wikitool review --json
```
**Behaviour**
- Joins the configured task-tracker provider against `kb/gtd/` project pages over the case-normalized project name, at read time, storing nothing - not even a `reports/` file.
- **stalled**: a tracker project with zero open items whose `kb/` page is `state: active`; `dormant`/`completed`/`abandoned` never fire.
- **waiting-overdue**: a `WAITING` item whose `follow_up_at` is older than `thresholds.stalled_waiting_days`.
- **unpaged-project**: a tracker project with no matching `kb/` page, older than `thresholds.unpaged_project_weeks`.
- **no-open-loop**: a `kb/` page `state: active` with no matching tracker project, or one with zero open items - the reverse direction of unpaged-project, so a rename on either side surfaces on both.
- **someday-stale**: a someday/maybe item untouched for longer than `thresholds.someday_stale_months`.
- A value a provider genuinely cannot supply - a `WAITING` item with no `follow_up_at`, a tracker project with no determinable creation date - is its own finding (`waiting_no_follow_up`/`project_age_unknown`) rather than a silent skip.
- Thresholds come from the tracker configuration, never from the schema.
- The tracker configuration is read from `.wikitool-tasks.json`, or from the file `WIKITOOL_TASKS_CONFIG` names when that variable is set (so one checkout can be run against several trackers in turn). A set variable that names no file is an error, never "no tracker configured".
- Text output is one `[check] project: message` line per finding, preceded by a `Source:` line naming which access path answered and, for `superproductivity`'s `access: "snapshot"`, the snapshot's age. `--json` carries the same findings plus `checks_run`/`checks_skipped`/`kb_project_count`/`complete`/`source` (`{"kind": ..., "detail": ...}` or `null`).
- A provider that cannot be reached mid-run degrades only the checks that needed the failing call; the report is printed in full, then exit 1 follows - never rendered as if it were complete.
- Re-reads everything fresh on every call, so nothing is ever stale to re-fetch.
- Read-only, and **exempt from the Iteration Budget Gate**.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: no
- Can reach the network: yes
**Exit codes**
- `0` - success
- `1` - No `.wikitool-tasks.json`, or a malformed one - a clear "no tracker configured" message
- `1` - `WIKITOOL_TASKS_CONFIG` names a file that does not exist or is broken
- `1` - The provider was reachable at config-parse time but a read call failed mid-run; the full report (findings plus which checks ran) was printed first
**See also**
- `gtd-weekly-review` skill - turns the findings into decisions
- `wikitool task list` / `wikitool task close` - act on an item
- `wikitool new project` - a project page and its tracker project
<!-- /wikitool:reference-commands -->
Reference-Commands-Git.md
+162
@@ -0,0 +1,162 @@
*Generated from [`docs/reference/commands/git.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/docs/reference/commands/git.md) at v9.0.0 – edit the repository, not this wiki.*
# Git
Pulling in the remote and publishing a change - the only way a change leaves this checkout.
<!-- wikitool:toc -->
## Contents
- [`sync`](#sync)
- [`publish`](#publish)
<!-- /wikitool:toc -->
<!-- wikitool:reference-commands -->
## `sync`
Fetch `<remote>/<branch>` and bring the local branch up to date with it.
**Synopsis**
- `wikitool sync [--remote origin] [--branch main] [--confirm-rebase TOKEN]`
**Options**
```text
--remote <str> Git remote to reconcile against [default: origin]
--branch <str> Branch to reconcile [default: main]
--confirm-rebase TOKEN Clear the Rebase-Review Gate for the exact upstream
state and file overlap this token was issued for. The
token comes from the gate's own refusal output.
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool sync
tools/wikitool sync --confirm-rebase <token> # re-run after exit 42, once the user approved
```
**Behaviour**
- Fetches `<remote>/<branch>`, then: fast-forwards when only the remote moved; rebases the local commits on top when both sides moved but touched disjoint files; exits 42 (Rebase-Review Gate) when both sides touched the same file.
- Files `wikitool` generates (`kb/index.md`, `kb/log.md`, `kb/provenance.md`, every `kb/**/INDEX.md`) never count as an overlap and never reach the gate. Where they are what stops git - uncommitted ones in the way of a fast-forward or rebase, or a rebase conflict in them - they are merged mechanically: `kb/log.md` becomes the incoming log plus the entries only this side appended, and the catalog and `kb/provenance.md` are regenerated from the merged pages and left as an uncommitted change. A reconcile git can do on its own is unchanged and regenerates nothing.
- Uncommitted changes that no incoming commit touches do not stop a rebase: they are carried through it (`git rebase --autostash`) and come back unchanged.
- Before anything is set aside, the working tree is stored as a commit under `refs/wikitool/reconcile-backup`. A successful reconcile drops it; a failed one restores the working tree and names the ref.
- A refused call performs no rebase attempt and leaves the branch where it was.
- The `--confirm-rebase` token covers the exact upstream state and the set of files touched on both sides, generated files excluded; either one moving makes it stale.
- Makes no commit, no push, and no forced operation of any kind.
- Three messages for a fetch that fails: no remote of that name, a remote that answers but has no such branch yet (a new, empty repository), and a remote that cannot be reached. All three exit 0 here; `publish` stops on the first and the last.
- Run it once at the start of a writing session.
**Properties**
- Changes files: yes
- Idempotent: yes
- Atomic: No - fetch, then at most one merge/rebase attempt, aborted cleanly on failure with the working tree restored; a success that merged generated files then rewrites the catalog
- Counts toward the session call budget: yes
- Can reach the network: yes
- Gates: rebase-review - see [exit codes and gates](Reference-Exit-codes)
**Exit codes**
- `0` - success
- `0` - No remote configured - reported and skipped, not a failure
- `0` - The remote cannot be reached - reported and skipped, not a failure
- `1` - The automatic rebase hit a real conflict outside the generated files (git failed), or `kb/log.md` was edited rather than appended to; it is aborted cleanly and the working tree restored
- `1` - An uncommitted change - or an untracked file - sits on a path the incoming commits also change; nothing was changed
- `42` - Rebase-Review Gate: `<remote>/<branch>` moved and both sides changed the same file, generated files aside; the output lists the upstream commits, the overlapping files and their diff
**See also**
- `wikitool publish` - runs the same reconcile before it commits and pushes
- `instructions/session-setup.md` - where a session runs `sync`
- `instructions/gates.md` - the gate procedure
## `publish`
Reconcile with `<remote>/<branch>`, then stage all changes, commit, and push.
**Synopsis**
- `wikitool publish --message "<op>: <desc>" [--no-push] [--confirm TOKEN] [--confirm-rebase TOKEN] [--threshold N] [--remote origin] [--branch main] [--path P ...]`
**Options**
```text
--message <str> Commit message summary, e.g. 'ingest: docker-
cheatsheet' [required]
--push / --no-push [default: push]
--confirm TOKEN Clear the Mass-Update Gate for the exact changeset
this token was issued for. The token comes from the
gate's own refusal output - run without it first to
see the file list, show that output to the user, and
only pass this once they have approved it.
--confirm-rebase TOKEN Clear the Rebase-Review Gate raised by the pre-push
reconcile step. The token comes from that gate's own
refusal output - see `tools/wikitool sync`.
--threshold <int> Mass-Update Gate threshold: number of counted files
at/above which clearance is required. [default: 10]
--remote <str> Git remote to push to [default: origin]
--branch <str> Branch to push [default: main]
--path <str> Limit this publish to a path (repeatable). Lets a
large change be committed in reviewable batches
instead of one opaque commit - the Mass-Update Gate
then counts only the files in scope.
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool publish --message "ingest: docker-cheatsheet"
tools/wikitool publish --confirm <token> --message "ingest: docker-cheatsheet" # re-run after a Mass-Update exit 42, once the user approved
tools/wikitool publish --confirm-rebase <token> --message "ingest: docker-cheatsheet" # re-run after a rebase-review exit 42, once the user approved
```
**Behaviour**
- Order: branch check and Publish-Remote Gate, then the reconcile with `<remote>/<branch>`, then the Mass-Update Gate, then `git add -A`, commit and push. `--no-push` skips all but the Mass-Update Gate and the commit.
- Without `--no-push`, a remote that is not configured or cannot be reached ends the call with exit 1 at the reconcile - before the gate, `git add` and the commit, and also on a clean tree. Nothing is committed, the index and the working tree are unchanged, and the message names `--no-push` as the way to a local commit. The next `publish` that reaches the remote pushes that commit along with whatever is new. A remote that answers but has no `<branch>` yet (a new, empty repository) is not this case: the first publish of an instance commits and pushes as before.
- Reconcile: fetches `<remote>/<branch>`, fast-forwards when only the remote moved, rebases the local commits on top when both sides moved but touched disjoint files, and exits 42 (Rebase-Review Gate) when both sides touched the same file. A refused reconcile performs no rebase attempt. The `--confirm-rebase` token covers the exact upstream state and the set of files touched on both sides.
- Generated files are never an overlap: where they are all that stops the reconcile, it merges them as `sync` does (both sides' log entries kept, the catalog and `kb/provenance.md` regenerated) and carries uncommitted changes no incoming commit touches through the rebase. The proactive reconcile runs before staging, so this publish commits the regenerated files - with `--path` too, see there; on the retry after a rejected push they get a commit of their own before the second push.
- The push target must be the checked-out branch; this is checked before anything is staged. The unborn branch of a fresh `git init -b main` counts as checked out, so the first publish of a new instance works; a real detached HEAD is refused.
- With nothing new to stage, a local commit the remote lacks is still pushed: one left behind by an earlier publish whose push failed, or every commit when the remote answers but does not have the branch yet (a new, empty remote repository).
- A rejected push that finds the remote unreachable on its one retry reports the original push error.
- A rejected push gets exactly one more reconcile-and-push; never more than one.
- Mass-Update Gate: counts the files that would be committed, refuses with exit 42 at `--threshold` (default 10) or more, and prints a review report - a scale line (file count, total lines added/removed, status breakdown), attention notes where they apply (deletions by name, control-plane and harness-config touches, published pages, the largest single change, binaries), and every counted path grouped by area with its status and churn. The list is what the commit will hold: it is computed from a scratch copy of the index after `git add -A`, so a path that is staged as deleted and back in the working tree is not counted twice, and a rename counts as its old path deleted plus its new path added. The real index and the working tree are not touched, so a refused publish leaves both byte-identical.
- Never counted and never shown for approval, but committed like everything else: anything under `work/`, and the files `wikitool` generates itself (`kb/index.md`, `kb/log.md`, `kb/provenance.md`, every `INDEX.md` under `kb/`). The refusal line accounts for both, by reason.
- The `--confirm` token covers each counted path, the blob id of its contents and the publish target: a different file list or edited contents need a new clearance.
- Publish-Remote Gate: when the checkout carries `.wikitool-remotes.json` and the push URL of `--remote` is not listed in it, exits 42 before the reconcile fetches anything. The URL is read with `git remote get-url --push`, so a repointed remote does not pass on its name. An absent file means unrestricted; a malformed one is an error, not permission.
- `--path` (repeatable) scopes the whole operation - gate count, staging and commit - to that subtree, with one exception: uncommitted generated files outside `--path` (`kb/index.md`, `kb/log.md`, `kb/provenance.md`, every `INDEX.md` under `kb/`) are staged and committed too, and a line names them - whether a reconcile or the change itself wrote them. They stay out of the gate count, and the `--confirm` token and the re-run line cover the `--path` given. Every other file outside `--path` is left as it is.
- Commit message: `--message`, then a `Files changed:` paragraph listing every committed path. When `--message` ends in a paragraph git reads as a trailer block (`Co-Authored-By:` and the like), that block stays the last paragraph and the list goes in front of it, since git reads trailers from the last paragraph only. `git interpret-trailers` decides whether there is such a block, and the list moves only when git reads the same trailers from the result; otherwise it is appended at the end. `--message` is not part of any gate token.
- After a successful commit or push whose changed files include `tools/`, `types/`, `instructions/`, `AGENTS.md` or a path ending in `CONTRACT.md`, prints one reminder line: the phase past this point (an issue-body rewrite, `docs/` staleness, a changelog entry's accuracy) is not covered by `docs verify`, `instructions verify` or `pytest`. It is not a gate: no exit code change, nothing to clear, and silent for an ordinary content publish.
**Properties**
- Changes files: yes
- Idempotent: no
- Atomic: No - sequential git operations. Every gate runs before staging, except on the one retry of a rejected push, where the Rebase-Review Gate can exit 42 after the commit: nothing is pushed, and the re-run with `--confirm-rebase` pushes that commit
- Counts toward the session call budget: yes
- Can reach the network: yes
- Gates: mass-update, publish-remote, rebase-review - see [exit codes and gates](Reference-Exit-codes)
**Exit codes**
- `0` - success
- `1` - git failed - `git add`, `git commit`, `git push`, or the reconcile's automatic rebase (a conflict outside the generated files, an edited `kb/log.md`, or an uncommitted change on a path the incoming commits change)
- `1` - The push target (`--branch`) is not the checked-out branch, or HEAD is detached; the unborn branch of a fresh `git init` is not this case
- `1` - No `--no-push`, and the remote is not configured or cannot be reached; nothing was committed
- `1` - `--yes`/`-y` was passed - the flag does not exist and fails with an explicit error
- `1` - `.wikitool-remotes.json` is unreadable or has no usable `allowed_push_urls` list
- `42` - Mass-Update Gate: `--threshold` (default 10) or more counted files would be committed, or the `--confirm` token does not match this changeset
- `42` - Rebase-Review Gate: `<remote>/<branch>` moved and both sides changed the same file; the output lists the upstream commits, the overlapping files and their diff
- `42` - Publish-Remote Gate: `.wikitool-remotes.json` exists and the push URL of `--remote` is not listed in it, or `--remote` resolves to no push URL
**See also**
- `wikitool sync` - the same reconcile on its own, without committing or pushing
- `instructions/gates.md` - the gate procedure
- `instructions/setup-instance.md` - the first publish of a new instance
<!-- /wikitool:reference-commands -->
Reference-Commands-Guideline-export.md
+79
@@ -0,0 +1,79 @@
*Generated from [`docs/reference/commands/guideline-export.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/docs/reference/commands/guideline-export.md) at v9.0.0 – edit the repository, not this wiki.*
# Guideline export
Writing the generated `GUIDELINES.md` into the repositories the wiki has captured.
<!-- wikitool:reference-commands -->
## `export guidelines`
Render the guideline pages into one generated `GUIDELINES.md`, or push it into the captured repositories that opted in (**Guideline Push Gate**).
**Synopsis**
- `wikitool export guidelines [--field <predicate> ...] [--kind/--subtype/--collection/--tag <v>]` - Prints the file to stdout; no network.
- `wikitool export guidelines <predicates> --push [--bundle <raw-bundle> ...] [--confirm TOKEN]` - Writes it into the captured repositories, behind the gate.
**Options**
```text
-f, --field <str> Frontmatter predicate, repeatable (AND) - the same forms
as `search --field`.
--kind <str> Shorthand for --field kind=<value>.
--subtype <str> Shorthand for --field subtype=<value>.
--collection <str> Shorthand for --field collection=<value>.
--tag <str> Shorthand for --field tags=<value>.
--push Write the file into the captured repositories (Guideline
Push Gate).
--bundle <path> Only the repository of this captured bundle under raw/;
repeatable.
--confirm <str> The token from a prior exit 42, once a human has approved
it.
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool export guidelines --tag guideline
tools/wikitool export guidelines --tag guideline --push
tools/wikitool export guidelines --confirm <token> --push --field tags=guideline # re-run after exit 42, once the user approved every diff
```
**Behaviour**
- Selects pages with `search`'s own predicates and no text: `--field` (repeatable, AND), `--kind`, `--subtype`, `--collection`, `--tag`. Which filter selects this instance's guidelines is written down in `kb/CONVENTIONS.md`.
- Renders mechanically: line 1 is `<!-- wikitool:export kind=guidelines instance=<origin URL without userinfo, or local> commit=<newest commit touching a selected page> - generated, do not edit by hand -->`; then each page sorted by title, as `# <title>` and its text without frontmatter, the generated links and footnote blocks, citation markers (`[^id]`, legacy `^[[...]]`), with `[[Title|Text]]` as `Text` and `[[Title]]` as `Title`. A leading H1 of the page is replaced by the title heading. Code is never rewritten. LF endings, no timestamp: the same pages give the same bytes.
- Refuses before printing or pushing anything: no predicate, no match, an unknown field, any page under `kb/` with unreadable frontmatter, uncommitted changes in `kb/` or `types/` (untracked files included), or no git checkout.
- `--push` targets every captured repository - each `(repo, ref)` from a `_capture.json` under `raw/`, `--bundle` narrowing it. A target is skipped with its reason when its ref rule is a tag pattern, it cannot be reached or has no such branch, its root has no `GUIDELINES.md` (not opted in), its `GUIDELINES.md` has no export header (hand-written), or the header names another `instance=`. A header line alone is the opt-in.
- Guideline Push Gate: `--push` without a matching `--confirm` fetches and builds everything, pushes nothing, and exits 42 with each target's status (`new`, `changed`, `unchanged`, `skipped`), the diff for every target it would write, and the re-run line. The token digests URL, branch, old tip and new blob of every target to write.
- With a matching token, each target gets exactly one commit on the fetched tip changing only `GUIDELINES.md`, authored as the instance checkout's `user.name`/`user.email`, pushed without force; one line per target: `unchanged`, `written <old> -> <new>`, `skipped (<reason>)`, `rejected (<reason>)` or `failed (<reason>)`.
- Nothing to write (every target unchanged or skipped) ends with exit 0 and no gate. A second run after a successful one finds every target unchanged.
- The target repository's own `AGENTS.md` points at the file (Claude Code: `@GUIDELINES.md`); that belongs to the repository, not to this command.
**Properties**
- Changes files: yes
- Idempotent: yes
- Atomic: Per target - one commit pushed without force, or nothing; targets already written stay written when a later one fails
- Counts toward the session call budget: yes
- Can reach the network: yes
- Gates: guideline-push - see [exit codes and gates](Reference-Exit-codes)
**Exit codes**
- `0` - success
- `0` - A target was skipped - tag rule, not reachable, not opted in, hand-written file, another instance's file
- `1` - No predicate, no page matches, a malformed predicate or an unknown field
- `1` - A page under `kb/` has unreadable frontmatter, `kb/` or `types/` has uncommitted changes, or the instance is not a git checkout
- `1` - `--push` with no `user.name` or `user.email` in the instance checkout's git configuration - checked before any target is fetched
- `1` - `--bundle` names no captured bundle under `raw/`, or `--bundle`/`--confirm` was given without `--push`
- `1` - A target rejected the push (its branch moved since the fetch) or a git call failed - the other targets were still served
- `42` - Guideline Push Gate: `--push` without `--confirm`, or with a token that does not match the current set of targets
**See also**
- `wikitool search` - the same predicates, to preview the selection
- `wikitool raw capture` - how a repository becomes a target
- `instructions/gates.md` - the gate procedure
<!-- /wikitool:reference-commands -->
Reference-Commands-Instance-health.md
+69
@@ -0,0 +1,69 @@
*Generated from [`docs/reference/commands/instance-health.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/docs/reference/commands/instance-health.md) at v9.0.0 – edit the repository, not this wiki.*
# Instance health
A single check of whether this instance is set up and working.
<!-- wikitool:reference-commands -->
## `doctor`
Check that this instance is correctly configured.
**Synopsis**
- `wikitool doctor [--json]`
**Options**
```text
--json Print the checks as JSON
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool doctor
tools/wikitool doctor --json
```
**Behaviour**
- Checks dependencies (Python, ripgrep), author resolution, stack version, git identity/branch/remote, published skills, the kb/raw/reports/work/instructions structure, and generated files.
- Import filters (`import-filter-<name>`): one line for each `raw derive` format that reads through a package (PDF, Outlook `.msg`) - a `WARN` naming the package when it is missing, since that costs only its own format; fixed by running `tools/preflight.sh` again.
- Tool paths (`tool-paths`): `.wikitool-tools.json` written by a preflight that finished, every tool `tools/prerequisites.txt` names for this platform recorded, and every recorded path still there - a `FAIL` otherwise, fixed by running `tools/preflight.sh` again (PowerShell 7: `tools/preflight.ps1`).
- Install folder (`install-dir`): on Windows with long paths off, a `FAIL` when the folder holding `tools/` is longer than `tools/prerequisites.txt` allows (95 characters) - the limit the preflight enforces before it sets anything up.
- PowerShell (`execution-policy`, `script-marks`; Windows only, `OK` elsewhere): a `FAIL` when the effective execution policy is `Restricted` or `AllSigned` - the line then says whether a group policy sets it, which only whoever administers the computer can change - and a `FAIL` when a script under `tools/` carries a Mark of the Web from the internet zone, which a browser download unpacked in Explorer leaves behind and `Invoke-WebRequest` plus `tar` do not. `tools/preflight.ps1` checks the same two things before it stops.
- Personalization: `USER.md`/`SOUL.md` present **and** filled - a file still carrying the template's sentinel is a `FAIL`.
- KB conventions: `kb/CONVENTIONS.md` present, unsentinelled, and naming all three tool-owned section headings - a `FAIL` on any of the three.
- Environment note: `ENVIRONMENT.md` is optional, so absent is `OK`; a still-templated one is a `WARN`.
- MCP `submit` tool: whether `.wikitool-upload.json` is present, absent or malformed, its limits, and how many submissions wait in `mcp-upload/`. Absent is `OK` and means the write path does not exist at all; malformed is a `FAIL`.
- Task tracker: `.wikitool-tasks.json` present, absent or malformed - absent is `OK` (no tracker configured), malformed is a `FAIL`. When `WIKITOOL_TASKS_CONFIG` is set, that file is read instead and the finding names it; a set variable that names no file is a `FAIL`, never `OK`.
- For a configured `superproductivity` provider, the configured `access` path's own state: `access: "api"` reports whether its local REST API answers `GET /health` with a ready renderer right now, `access: "snapshot"` whether a backup file is ready. The other access path is never attempted, and neither state is ever a `FAIL`.
- For a configured `caldav` provider, whether the server is reachable and Basic auth succeeds - never a `FAIL`; only a broken config block is.
- Session id source: `OK` for `WIKITOOL_SESSION_ID` or a registered harness variable, `WARN` only for the bare parent-pid fallback.
- Telemetry: on or off and why - installation-form default, `.wikitool-telemetry.json`, or `WIKI_TRACE` - and the current session's count and byte total against both caps; never a `FAIL`. A `WARN` when `.wikitool-telemetry.json` is not read as written - unreadable JSON, not an object, `enabled` not a JSON boolean, a cap not a positive integer, or an unknown key - naming the field; telemetry itself drops or reinterprets such a value without a word, so this is where it shows.
- Exits 1 only on a `FAIL`; a missing remote, session id or `VERSION` is a `WARN`, not a fault.
- Read-only and exempt from the Iteration Budget Gate.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: no
- Can reach the network: yes
**Exit codes**
- `0` - success
- `1` - At least one check reported `FAIL` (a `WARN`, e.g. no remote or no `WIKITOOL_SESSION_ID`, does not exit 1)
**See also**
- `instructions/setup-instance.md` - the setup steps most findings point back to
- `instructions/preflight.md` - what `tool-paths` and `install-dir` point back to
- `docs/reference/configuration.md` - the per-checkout configuration files
- `docs/explanation/telemetry-and-evaluation.md` - telemetry state and caps
- `instructions/session-setup.md` - setting `WIKITOOL_SESSION_ID`
<!-- /wikitool:reference-commands -->
Reference-Commands-Links-and-citations.md
+378
@@ -0,0 +1,378 @@
*Generated from [`docs/reference/commands/links-and-citations.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/docs/reference/commands/links-and-citations.md) at v9.0.0 – edit the repository, not this wiki.*
# Links and citations
Declaring the relationships between pages and the citations that tie a claim to its source, with the generated regions that hold both.
<!-- wikitool:toc -->
## Contents
- [`xref add`](#xref-add)
- [`xref remove`](#xref-remove)
- [`xref link-source`](#xref-link-source)
- [`links show`](#links-show)
- [`cite id`](#cite-id)
- [`cite add`](#cite-add)
- [`cite sync`](#cite-sync)
<!-- /wikitool:toc -->
<!-- wikitool:reference-commands -->
## `xref add`
Declare that A <rel> B.
**Synopsis**
- `wikitool xref add --a "<A>" --b "<B>" --rel <label> [--dry-run]`
**Options**
```text
--a <str> Exact title of the page that asserts the edge [required]
--b <str> Exact title of the page it points at [required]
--rel <str> Label from instructions/link-taxonomy.md, e.g. depends-on
[required]
--dry-run Preview the change instead of writing
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool xref add --a "Gitea Actions" --b "Act Runner" --rel depends-on
tools/wikitool xref add --a "Gitea Actions" --b "Act Runner" --rel depends-on --dry-run
```
**Behaviour**
- Declares **one** edge, `A <label> B`: written into A's `related:` as `- <label>: B` and rendered into A's generated links region.
- B is not touched and does not point back - its inbound view is rendered from the graph.
- Idempotent; re-running with a different label *relabels* rather than appending, since one page asserts one thing about another.
- Refuses before writing when A's type does not declare `related:` - a source page declares `entities:`/`concepts:` instead, and the refusal names them and points at `xref link-source`.
- Refuses before writing when `<label>` is not authorised by the source collection's `outbound:` block for the target's collection; the refusal lists the authorised set and points at `instructions/link-taxonomy.md`.
- `--dry-run` reports the edge without writing.
**Properties**
- Changes files: yes
- Idempotent: yes
- Atomic: Yes - a single write to A; B is never touched, and every refusal happens before it
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - Page A or B not found
- `1` - A's type declares no `related:` field
- `1` - `<label>` is not authorised for this collection pair, or the source collection authorises no labels into the target's collection at all
**See also**
- `wikitool xref remove` - removes a reference
- `wikitool links show` - the edges out of and into a page
- `instructions/link-taxonomy.md` - the labels and what each asserts
## `xref remove`
Remove a cross-reference: the inverse of `xref add`.
**Synopsis**
- `wikitool xref remove --a "<A>" --b "<B>" [--dry-run]`
**Options**
```text
--a <str> Exact title of page A (must exist) [required]
--b <str> Title to unlink from A; need not still exist as a page [required]
--dry-run Preview changes instead of writing
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool xref remove --a "Gitea Actions" --b "act_runner"
tools/wikitool xref remove --a "Gitea Actions" --b "Act Runner" --dry-run
```
**Behaviour**
- Clears the reference in **both** directions: the cleanup command for a deleted or hand-renamed page rather than the strict inverse of a one-directional `xref add`.
- Clears `<B>` from every page-ref frontmatter field `<A>`'s type declares (`related:`, `sources:`, `entities:`, `concepts:`) plus the matching bullets.
- Also sweeps a page-ref field the type does *not* declare but some other type does, and drops that key once empty.
- `--b` need not still exist as a page, so this clears a reference left by a hand-deleted or hand-renamed page without hand-editing frontmatter.
- Idempotent: removing an absent link is a no-op. `--dry-run` reports without writing.
**Properties**
- Changes files: yes
- Idempotent: yes
- Atomic: No - writes A then B, both idempotent
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - Page A not found (B is allowed not to exist)
**See also**
- `wikitool xref add` - declares an edge
- `wikitool rm` - deletes a page and de-links it
## `xref link-source`
Batch-link a source page to every entity/concept it mentions.
**Synopsis**
- `wikitool xref link-source --source "Source - X" --entities A,B,C [--dry-run]`
**Options**
```text
--source <str> Exact source page title, e.g. 'Source - Docker Cheatsheet'
[required]
--entities <str> Comma-separated entity/concept titles the source mentions
[required]
--dry-run Preview which pages would be linked instead of writing
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool xref link-source --source "Source - Docker Cheatsheet" --entities Docker,Podman --dry-run
tools/wikitool xref link-source --source "Source - Docker Cheatsheet" --entities Docker,Podman
```
**Behaviour**
- Each target gets the source page in its `sources:`, and the source page records each target in its own `entities:` or `concepts:`. No body bullet is written on either side - `sources:` *is* the record.
- Which field a target lands in follows its collection (`kb/entities/` -> `entities:`), so a new collection needs no code change.
- A target whose collection matches no reference field the source type declares is linked one way and named in the output.
- A target that does not exist is skipped and named; the others are still linked, and the run then exits 1.
- A target that is itself a source page is refused, named with the `cite add` command to use instead, and handled like a missing page: nothing is written for it, the others are still linked, and the run exits 1. `link-source` means "A mentions X" and writes `X.sources += A`, which for X = another source would claim the citation in the wrong direction; a citation between two sources has a direction only `cite add` records.
- Idempotent in both directions. `--dry-run` reports what would be linked.
**Properties**
- Changes files: yes
- Idempotent: yes
- Atomic: No - one write per entity plus one for the source page, idempotent per page
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - Source page not found
- `1` - A page in `--entities` does not exist, or a target could not be written; the others were linked
- `1` - A page in `--entities` is itself a source page; the others were linked
- `1` - The source page itself could not be written after its targets were
**See also**
- `wikitool sources trace` - who a source is already linked to
- `wikitool xref add` - one labelled edge between two pages
- `wiki-ingest` skill - where a source is linked after it is written
## `links show`
Show the edges out of and into a page.
**Synopsis**
- `wikitool links show --page "<Title>" [--json]`
**Options**
```text
--page <str> Exact page title [required]
--json Print the edges as JSON
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool links show --page "Act Runner"
tools/wikitool links show --page "Act Runner" --json
```
**Behaviour**
- Shows the declared graph around one page in both directions: the edges it asserts (from its own `related:`, with labels) and the edges other pages assert about it.
- The inbound half is computed across the corpus on every call, never stored, so it is complete.
- `--json` prints both halves as JSON.
- Read-only; exempt from the Iteration Budget Gate.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: no
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - Page not found
**See also**
- `wikitool xref add` / `wikitool xref remove` - change the outbound edges
- `wikitool search` - finds the exact title
## `cite id`
Print the deterministic footnote id `cite add` would use for this (title, file) pair.
**Synopsis**
- `wikitool cite id --title "Source - X" [--file <qualifier>]`
**Options**
```text
--title <str> Source page title, e.g. 'Source - Docker Cheatsheet'
[required]
--file <str> Qualifier for a multi-file source, e.g. 'storage-model.md'
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool cite id --title "Source - Docker Cheatsheet"
```
**Behaviour**
- Prints the footnote id `cite add` would use for this (title, file) pair.
- A preview only: it does not check that the id is free on any given page.
- Never fails; safe to retry freely. Read-only and exempt from the Iteration Budget Gate.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: no
- Can reach the network: no
**Exit codes**
- `0` - success
**See also**
- `wikitool cite add` - writes the definition
## `cite add`
Upsert a `[^cite-id]: [[Source - X]]` definition in a page's footnotes region.
**Synopsis**
- `wikitool cite add --page "<Title>" --source "Source - X" [--file <qualifier>] [--dry-run]`
**Options**
```text
--page <str> Exact title of the page to add a citation on [required]
--source <str> Exact title of the source page being cited, e.g. 'Source - X'
[required]
--file <str> Qualifier for a multi-file source, e.g. 'storage-model.md'
--dry-run Preview instead of writing
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool cite add --page "Docker" --source "Source - Docker Cheatsheet"
tools/wikitool cite add --page "Docker" --source "Source - Docker Cheatsheet" --file part-2
```
**Behaviour**
- Upserts a `[^cite-id]: [[Source - X]]` definition in the page's generated footnotes region, creating the region between `<!-- wikitool:footnotes -->` markers if absent.
- Reuses the id when the page already cites this exact source/file pair, so a repeat changes nothing.
- Adds `Source - X` to the page's frontmatter `sources:` - except on `Source - X` itself: a source page citing one of its own raw files (`--file`) gets the definition and leaves `sources:` untouched, since a page never lists its own title there.
- A source page may cite another source page this way - `cite add` is the command that records a citation between two sources, in the direction it was made.
- Prints the `[^cite-id]` marker; pasting it into the prose is a manual, editorial step.
- `--dry-run` reports without writing.
**Properties**
- Changes files: yes
- Idempotent: yes
- Atomic: Yes - single file write
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - The page is not found
- `1` - The source page is not found - citing it would be a dangling reference
**See also**
- `wikitool cite sync` - prunes and re-orders the region
- `wikitool cite id` - previews an id
## `cite sync`
Reconcile each page's footnotes region against its actual `[^id]` references.
**Synopsis**
- `wikitool cite sync [--page "<Title>" | --all] [--dry-run]`
**Options**
```text
--page <str> Sync just this page
--all Sync every page under kb/
--dry-run Report what would change instead of writing
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool cite sync --page "Docker"
tools/wikitool cite sync --all --dry-run
```
**Behaviour**
- Prunes definitions nothing references any more, re-renders the region in first-reference order, and reports every `[^id]` reference left with no definition.
- An undefined-reference report is not a failure: fix the reference, or run `cite add`, and re-run.
- A page still carrying the pre-4.0.0 undelimited footnote block is converted to a marked region in the same pass.
- Each page's re-render is idempotent; safe to retry freely. `--dry-run` reports without writing.
**Properties**
- Changes files: yes
- Idempotent: yes
- Atomic: No - one write per page, each idempotent
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - Neither or both of `--page`/`--all` given, or the page is not found
- `1` - A page write failed partway
**See also**
- `wikitool cite add` - writes a definition
<!-- /wikitool:reference-commands -->
Reference-Commands-Pages.md
+533
@@ -0,0 +1,533 @@
*Generated from [`docs/reference/commands/pages.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/docs/reference/commands/pages.md) at v9.0.0 – edit the repository, not this wiki.*
# Pages
Creating, updating, renaming, moving and removing wiki pages, and filing and closing items in the task tracker.
<!-- wikitool:toc -->
## Contents
- [`new`](#new)
- [`task new`](#task-new)
- [`task list`](#task-list)
- [`task close`](#task-close)
- [`touch`](#touch)
- [`rename`](#rename)
- [`rm`](#rm)
- [`move`](#move)
<!-- /wikitool:toc -->
<!-- wikitool:reference-commands -->
## `new`
Scaffold a new wiki page of any type.
**Synopsis**
- `wikitool new <type-name> --name "<Name>" [--type <path>] [--set field=value ...]` - Any type; its type-spec decides fields, directory, title prefix and template.
- `wikitool new entity --name "<Name>" --set entity_type=<t> [--set tags=a,b] [--set related=X,Y] [--set sources="Source - Z"] [--set provenance=sourced|general|mixed]` - Writes `kb/entities/<subdir>/<Name>.md` - `<subdir>` from `entity_type` via the type-spec's `layout:`
- `wikitool new concept --name "<Name>" --set concept_type=<t> ...` - Writes `kb/concepts/<subdir>/<Name>.md` - `<subdir>` from `concept_type` via the type-spec's `layout:`
- `wikitool new source --name "<Name>" --set source_type=<t> --set raw_files=raw/notes/x.md,raw/notes/y.md --set fidelity=<f> --set authority=<a> [--set source_url=<URL>] [--set entities=A,B] [--set concepts=C,D]` - Writes `kb/sources/<subdir>/Source - <Name>.md` (prefix added automatically; `<subdir>` from `source_type` via the type-spec's `layout:`) with a `raw_files:` list; rejects paths that don't exist
- `wikitool new comparison --name "X vs Y" --set entities=X,Y` - Writes `kb/comparisons/X vs Y.md`
- `wikitool new project --name "<Name>" --set responsibility=<bereich> [--resume]` - Writes `kb/gtd/<bereich>/<Name>.md` and, with a task tracker configured, a same-named tracker project
**Options**
```text
Arguments:
type_name Type name, e.g. entity|concept|source|comparison (see `wikitool
types list`) [required]
Options:
--name <str> Page title (a type's title_prefix is added automatically)
[required]
--type <str> Override the type-spec path (defaults to the one named by
TYPE_NAME)
--set <str> Frontmatter field, repeatable: --set entity_type=tool --set
tags=a,b. Array values split on commas (escape a literal one as
\,); repeating --set for an array field appends instead of
replacing
--resume `project` only: confirm a human has completed the manual tracker
step an earlier HumanInterventionRequired refusal asked for, so
this run continues to page creation instead of refusing the now-
existing tracker project as a collision.
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool new entity --name "Docker" --set entity_type=tool --set tags=containers
tools/wikitool new source --name "Docker Cheatsheet" --set raw_files=raw/2026/09/docker-cheatsheet.md --set fidelity=verbatim --set authority=reporting
tools/wikitool new project --name "Homelab migration" --set responsibility=infrastruktur --resume # re-run after exit 42, once the user created the tracker project
```
**Behaviour**
- A title becomes a file name, so it must be valid and unique on Windows and macOS as well as Linux, whichever platform runs the command and whichever root the type writes to. The rule is `kb/CONTRACT.md` § Titles are identifiers; it is checked on the full title, after `title_prefix`.
- The target's path below the instance root may be at most 160 characters, counted in UTF-16 code units the way Windows counts MAX_PATH, so a Windows checkout without long paths keeps working. A longer one is refused, for every root, naming the length and how much shorter it has to get.
- `new` never overwrites: a file already at the target - or one a case-insensitive file system would treat as the same file - is refused for every root, `instructions/` included.
- The type-spec drives everything: fields, directory (`base_dir`/`layout`), title prefix, and template. `types list`/`types describe` show what a type requires.
- The body skeleton is `types/<type>.<value>.md` when that file exists, `<value>` being the page's subtype field value (e.g. `types/entity.person.md` for `entity_type=person`) - it replaces the type-spec's `## Template` block whole. Without such a file, and for a type with no subtype field, the `## Template` block is used as before.
- A schema `default:` is materialized only for a field the schema also lists in `required:`.
- `--set` is repeatable, and comma-separated values fill array fields. An element that itself contains a comma is written `\,`, or passed as its own repeated `--set` for that field - repeating an array field appends.
- A capture field the type-spec requires (a source's `fidelity`/`authority`) must be passed with `--set`; `new` never guesses it and refuses `unknown` for it.
- Produces structurally correct frontmatter and a body skeleton only - the prose (Description, Summary, judgment calls about relationships) is written afterwards.
- `new project`: with `.wikitool-tasks.json` configuring a task tracker, also makes sure a same-named tracker project exists - one name, one identity. No tracker configured is a legitimate, explicitly announced state: page only.
- `new project`: tracker before page. The tracker side is settled first, so a failure past that point leaves a tracker project with no page - a state `review`'s check 3 reports - never a page with no tracker project.
- `new project`: a name already taken, case-insensitively, in `kb/` or the tracker is refused outright, naming where it was found, and creates nothing. For `caldav` the tracker check covers every list in the account, not only the ones counted as projects.
- `new project`: a provider whose configured access path has no write path (Super Productivity's `access: "snapshot"`) refuses entirely with exit 1, naming the `access: "api"` instance to use instead - neither the tracker project nor the page is created, and `--resume` behaves the same.
- `new project`: a provider that could write but has no project-creation call of its own (Super Productivity's `access: "api"` - `GET /projects` exists, `POST /projects` does not) prints instructions for a human and exits 42, creating nothing. That is not one of the named gates, but the same exit code and the same handling. `caldav` never does this: `MKCALENDAR` creates the list, so a valid, non-colliding name always creates it.
- `--resume` is how a later run tells the command a human has done what that message asked: it re-verifies through the tracker's read path before continuing to page creation, rather than trusting the claim, and exits 42 again if the tracker still does not have the project. `--resume` on any other type is refused.
**Properties**
- Changes files: yes
- Idempotent: no
- Atomic: `new <type>`: Yes - single file write. `new project`: **No** for the tracker-configured case - a tracker-project write (or its human-clearance request) happens before the kb/ page write, so a failure between the two leaves a tracker project with no page (a state `review`'s check 3 already reports), never a page with no tracker project. Still a single file write when no tracker is configured
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - A page with this title already exists, the type is unknown, or a `--set` value is invalid
- `1` - The title is not a valid file name (forbidden character, control character, reserved name such as `CON` or `Index`, trailing dot or space, empty), collides with another page by case or Unicode normalization, the target file already exists, or the target path is over the 160-character path budget
- `1` - A `raw_files` path does not exist
- `1` - A capture field the type-spec requires is missing, or set to `unknown`
- `1` - `--resume` with a type other than `project`
- `1` - new project: The name is already taken in the tracker (case-insensitively; for `caldav` against every list in the account)
- `1` - new project: The configured access path has no write path (Super Productivity's `access: "snapshot"`)
- `1` - new project: The page write failed after the tracker project was confirmed to exist
- `42` - new project: The provider cannot create the project itself (Super Productivity's `access: "api"`); the output says what a human has to create
**See also**
- `wikitool types describe <type>` - what a type requires and where it lands
- `wikitool touch` - changes a page's own frontmatter afterwards
- `wikitool task new` - a tracker item without a page
- `docs/explanation/knowledge-and-commitment.md` - why a project is a page and a tracker project
## `task new`
Create one open item in the configured task tracker - never a kb/ page.
**Synopsis**
- `wikitool task new --title "<Title>" (--project "<Name>" | --inbox) [--waiting [--follow-up-at YYYY-MM-DD]] [--notes "..."]`
**Options**
```text
--title <str> The item's title. Stored verbatim, never parsed.
[required]
--project <str> An existing tracker project's name (matched case-
insensitively). This command never searches or guesses
one and never creates one - use `wikitool new project`
first if it does not exist yet. Exactly one of
--project/--inbox is required.
--inbox File into the tracker's own inbox instead of a project -
the deliberately chosen exit when no project fits, never
a stand-in for an omitted --project. An item filed here
is invisible to `wikitool review`, since every check
there is reached through a project name and the inbox
has none.
--waiting Tag the item WAITING - the review's check 2 reads this.
--follow-up-at <str> YYYY-MM-DD. Only meaningful together with --waiting - it
is never a due date and is refused without --waiting.
--notes <str> A freetext backref, e.g. to the kb/ source page this
item came from. Stored verbatim, never parsed.
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool task new --title "Renew the TLS certificate" --project "Homelab migration"
tools/wikitool task new --title "Quote from the electrician" --project "Homelab migration" --waiting --follow-up-at 2026-10-05
tools/wikitool task new --title "Read the qmd README" --inbox --notes "from [[Source - qmd - GitHub Repository]]"
```
**Behaviour**
- Creates one open item in the configured task tracker; never touches `kb/`.
- Exactly one of `--project` or `--inbox` is required; an omitted `--project` refuses rather than silently falling into the inbox.
- `--project` names an existing tracker project, matched case-insensitively - never created, and never searched or guessed.
- `--inbox` files into the tracker's own inbox. An item filed there never appears in `review`, since every one of its checks reaches items through a project name.
- `--waiting` sets the WAITING status `review`'s waiting-overdue check reads. `--follow-up-at` is refused without `--waiting` - it is never a due date on its own.
- `--notes` carries a freetext backref (e.g. to the `kb/` source page the item came from), stored verbatim, never parsed.
- The tracker configuration is read from `.wikitool-tasks.json`, or from the file `WIKITOOL_TASKS_CONFIG` names when that variable is set (so one checkout can be run against several trackers in turn). A set variable that names no file is an error, never "no tracker configured".
- No `.wikitool-tasks.json` fails immediately with a "no tracker configured" message.
- A provider whose configured access path has no write path (Super Productivity's `access: "snapshot"`) refuses entirely with exit 1, naming the `access: "api"` instance to use instead.
- **Never exits 42.** A `--project` matching no tracker project, or `--waiting` against a provider that cannot represent it right now (Super Productivity: the `waiting` tag does not exist yet, and its API cannot create tags), are ordinary exit-1 refusals that create nothing.
- Not idempotent: every successful run creates another item.
**Properties**
- Changes files: yes
- Idempotent: no
- Atomic: Yes - a single API call, made only once every precondition (the project's own id, the WAITING tag's own id) is confirmed to exist, so a missing one never leaves a half-written item behind
- Counts toward the session call budget: yes
- Can reach the network: yes
**Exit codes**
- `0` - success
- `1` - No `.wikitool-tasks.json` - no tracker configured
- `1` - `WIKITOOL_TASKS_CONFIG` names a file that does not exist or is broken
- `1` - Neither or both of `--project`/`--inbox`, or a `--follow-up-at` without `--waiting` or not `YYYY-MM-DD`
- `1` - A `--project` name matching no tracker project
- `1` - `--waiting` against a provider with no way to represent it right now (Super Productivity: the `waiting` tag does not exist)
- `1` - A read-only access path (Super Productivity's `access: "snapshot"`)
**See also**
- `wikitool task list` - a project's open items and their ids
- `wikitool task close` - marks an item done
- `wikitool new project` - a project page and its tracker project
- `docs/explanation/knowledge-and-commitment.md` - why commitments live in the tracker, not in `kb/`
## `task list`
List a project's open items - id, title, and whether each carries the WAITING status.
**Synopsis**
- `wikitool task list --project "<Name>"`
**Options**
```text
--project <str> An existing tracker project's name (matched case-
insensitively). [required]
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool task list --project "Homelab migration"
```
**Behaviour**
- Lists one tracker project's open items: id, title, and whether each carries the WAITING status.
- The id source `task close` and `review`'s `waiting_overdue`/`someday_stale` findings need, without running `review` first.
- Works on every access path a provider offers, read-only ones included.
- A `--project` matching no tracker project prints "No open items": the tracker read does not distinguish an empty project from an unknown one.
- Read-only.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Yes - read-only, nothing to leave half-written
- Counts toward the session call budget: yes
- Can reach the network: yes
**Exit codes**
- `0` - success
- `0` - A `--project` matching no tracker project - prints "No open items", not an error
- `1` - No `.wikitool-tasks.json` - no tracker configured
- `1` - `WIKITOOL_TASKS_CONFIG` names a file that does not exist or is broken
**See also**
- `wikitool task close` - closes an item by the id listed here
- `wikitool review` - the findings that name items
## `task close`
Mark one tracker item done - never delete it.
**Synopsis**
- `wikitool task close --id <item-id>`
**Options**
```text
--id <str> The tracker's own item id, e.g. from `task list` or `wikitool
review`'s waiting_overdue/someday_stale findings - never a title.
[required]
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool task close --id <item-id>
```
**Behaviour**
- Marks one tracker item done. It never deletes or moves an item - the only closing write this stack makes.
- `--id` is the provider's own item id, from `task list` or a `review` finding - never a title.
- The tracker configuration is read from `.wikitool-tasks.json`, or from the file `WIKITOOL_TASKS_CONFIG` names when that variable is set (so one checkout can be run against several trackers in turn). A set variable that names no file is an error, never "no tracker configured".
- No `.wikitool-tasks.json` fails with a "no tracker configured" message.
- A provider whose configured access path has no write path (Super Productivity's `access: "snapshot"`) refuses entirely with exit 1, naming the `access: "api"` instance to use instead.
- **Never exits 42.**
**Properties**
- Changes files: yes
- Idempotent: yes
- Atomic: Yes - a single API call; an unknown id is rejected by the provider itself (Super Productivity: `404 TASK_NOT_FOUND`) before anything is written
- Counts toward the session call budget: yes
- Can reach the network: yes
**Exit codes**
- `0` - success
- `1` - No `.wikitool-tasks.json` - no tracker configured
- `1` - `WIKITOOL_TASKS_CONFIG` names a file that does not exist or is broken
- `1` - An `--id` matching no tracker item right now
- `1` - A read-only access path (Super Productivity's `access: "snapshot"`)
**See also**
- `wikitool task list` - where the id comes from
- `wikitool review` - findings that carry item ids
## `touch`
Bump a page's `modified:` date and optionally rewrite its other frontmatter fields.
**Synopsis**
- `wikitool touch --page "<Title>" [--summary "..."] [--provenance <v>] [--date YYYY-MM-DD] [--set field=value ...] [--add field=value ...] [--remove field=value ...] [--no-date] [--dry-run]`
**Options**
```text
--page <str> Exact page title, e.g. 'Docker Cheatsheet' [required]
--summary <str> Replace the page's 1-line summary
--provenance <str> Replace the page's provenance marker
(sourced|general|mixed)
--date <str> Date to record (YYYY-MM-DD). `modified:` defaults to
today; a source's `date:` is its publication date and
changes only when given here.
--set <str> Replace a frontmatter field, repeatable: --set tags=a,b.
Array values split on commas (escape a literal one as \,);
repeating --set for one array field appends within this
call. Page-reference fields belong to `xref`, not here
--add <str> Append elements to an array field without naming the whole
list: --add tags=x. Elements already present are left
alone
--remove <str> Drop elements from an array field: --remove tags=x.
Removing an absent element succeeds and says so
--no-date Only change the given fields; leave the modified/date
field alone
--dry-run Preview the new frontmatter instead of writing
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool touch --page "Docker" --summary "Container runtime and image format"
tools/wikitool touch --page "Docker" --add tags=containers --no-date
tools/wikitool touch --page "Source - Docker Cheatsheet" --date 2026-09-01 --dry-run
```
**Behaviour**
- Bumps `modified:` to today and optionally rewrites any other field the page's type declares.
- `--summary`/`--provenance` are shorthands; `--set` reaches every other field and **replaces** its value, while `--add`/`--remove` change single elements of an array field (removing an absent element succeeds and says so).
- Repeating `--set` for one array field appends *within the call*, and `\,` is a literal comma.
- Refused, naming the command that owns them instead: `type:` (the page-lifecycle procedure) and the page-ref arrays `related:`/`sources:`/`entities:`/`concepts:` (`xref`). Everything else the schema declares is settable, and an unknown field is refused with the list of fields the page actually has.
- Schema-validates the fields it writes; `raw_files:` entries must exist on disk.
- A source page declares `date:` instead of `modified:`: the *publication* date of the raw material. It is never bumped to today and changes only when `--date` names a value explicitly.
- `--no-date` changes only the given fields and leaves the date alone; `--dry-run` previews the new frontmatter without writing.
- Safe to re-run as-is: `--set` and `--add` are idempotent, and `--remove` of an already-absent element succeeds while reporting it.
**Properties**
- Changes files: yes
- Idempotent: yes
- Atomic: Yes - single file write, and every refusal happens before it
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - Page not found
- `1` - An invalid value for a field it writes, or a `raw_files:` path that doesn't exist
- `1` - A field owned by another command (`type:`, a page-ref array) or absent from the type's schema
- `1` - `--add`/`--remove` on a non-array field
**See also**
- `wikitool xref add` / `wikitool xref remove` - the page-ref arrays
- `instructions/page-lifecycle.md` - changing a page's `type:`
## `rename`
Rename a page, or repoint references that name a page that never existed.
**Synopsis**
- `wikitool rename --from "<Old>" --to "<New>" [--dry-run]`
**Options**
```text
--from <str> Current page title, exactly as it appears [required]
--to <str> New page title [required]
--dry-run List what would change instead of writing
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool rename --from "act_runner" --to "Act Runner" --dry-run
tools/wikitool rename --from "Docker Engine" --to "Docker"
```
**Behaviour**
- Renames a page and repoints every reference to it: body `[[wikilinks]]` (aliases and anchors preserved), a `[^cite-id]` whose id was derived from the old title (refreshed to match the new one, both in its Footnotes definition and every reference to it), the page's own H1, and every page-ref frontmatter array declared by the type's `page_ref_fields:`.
- If `--from` is *not* a page but is referenced, it instead repoints those references onto the existing `--to` page and moves nothing - the fix for a reference spelled `act_runner` when the page is `Act Runner`.
- Each page's rewrite is idempotent, so a re-run as-is is safe. If a write fails midway, nothing is renamed on disk and the error lists what was updated.
- `--dry-run` lists every page it would change; run it first to see the blast radius.
- Only `--to` is checked against the title rule and the path budget (160 UTF-16 code units for the whole path below the instance root). A page whose current title breaks either (`lint`'s Unportable Titles and Long Paths) can always be renamed away from it, and a title that differs from the page's own only by case (`Foo` to `FOO`) is allowed.
**Properties**
- Changes files: yes
- Idempotent: yes
- Atomic: No - one write per referencing page, then the file move
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - `--from` equals `--to`
- `1` - Neither `--from` nor `--to` is a page
- `1` - The `--to` title is already taken - also by a page that differs only in case or Unicode normalization, or by a file in the page's directory - or is not a valid file name, or would put the page's path over the path budget (see `kb/CONTRACT.md` § Titles are identifiers)
- `1` - A page write failed partway; nothing was renamed on disk
**See also**
- `instructions/page-lifecycle.md` - renaming, moving and deleting a page
- `wikitool move` - changes a page's directory, not its title
- `wikitool rm` - deletes a page
## `rm`
Delete a page and mechanically de-link it from the rest of the wiki.
**Synopsis**
- `wikitool rm --page "<Title>" [--yes] [--dry-run]`
**Options**
```text
--page <str> Exact title of the page to delete [required]
-y, --yes Confirm deletion of a page that other pages still reference.
Only pass this after a human has reviewed the inbound list -
never set it automatically.
--dry-run List what would change instead of writing
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool rm --page "Old Draft" --dry-run
tools/wikitool rm --page "Old Draft" --yes # after the user approved the inbound list
```
**Behaviour**
- Deletes a page and de-links it from the rest of the wiki: strips page-ref array entries and bare `- [[Title]]` / `- **label:** [[Title]]` bullets pointing at it.
- Leaves prose references and inline citations in place and reports them afterwards; those are an editorial fix, not a reason to retry.
- Refuses without `--yes` while other pages still reference it, listing them.
- If a de-link write fails midway, the page is not deleted, so nothing is orphaned and a re-run is safe.
- `--dry-run` lists what would change, without writing.
- Not idempotent: once the page is gone, a second run finds nothing to delete.
**Properties**
- Changes files: yes
- Idempotent: no
- Atomic: No - one write per referencing page, then the delete
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - Page not found
- `1` - Other pages still reference it and `--yes` was not passed
- `1` - A de-link write failed partway; the page was not deleted
**See also**
- `instructions/page-lifecycle.md` - when a page is deleted rather than renamed
- `wikitool rename` - repoints references instead of removing them
## `move`
Move a page (or every misplaced page) to the directory its type-spec computes.
**Synopsis**
- `wikitool move --page "<Title>"`
- `wikitool move --reconcile [--dry-run]`
**Options**
```text
--page <str> Exact title of the page to move to its computed location
--reconcile Move every page under kb/ that is not at its computed location
--dry-run List what would move without writing
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool move --page "Docker"
tools/wikitool move --reconcile --dry-run
tools/wikitool move --reconcile
```
**Behaviour**
- Moves a page to the directory its type-spec computes for its current frontmatter (`base_dir` + `layout`, the same rule `new` places a page by) - never to a hand-chosen destination; there is no `--to <dir>`.
- A destination whose path would be over the path budget (160 UTF-16 code units below the instance root) is refused, and `--reconcile` skips such a page and names it, as it does for an occupied destination - `wikitool rename` the page to a shorter title.
- `--reconcile` applies it corpus-wide: every misplaced page moves in one call, and a second run reports nothing left to do. It fixes `lint`'s `Misplaced Pages` (advisory) and `Nested Pages` (hard) findings.
- Neither mode touches a body or a frontmatter field, and the page's title - its only identity in the wiki - never changes; only the file moves.
- A directory a move empties is removed with it, so a page nested below its area leaves no leftover directory behind.
- A page already at its computed location is reported and left alone, and `--reconcile` only re-moves what is still misplaced, so a re-run is safe.
- `--dry-run` (with `--reconcile`) lists the moves; run it first to see the blast radius.
- Run `wikitool index rebuild` afterwards.
**Properties**
- Changes files: yes
- Idempotent: yes
- Atomic: `--page`: yes, a single file move. `--reconcile`: no - one file move per page, each idempotent
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - Neither or both of `--page`/`--reconcile` given
- `1` - The named page is not found, or has no `type:` to compute a placement from
- `1` - The destination already holds an entry with the same name, or one that differs only in case or Unicode normalization (a pre-existing duplicate-stem collision), or its path would be over the path budget - refused rather than silently skipped
- `1` - `--reconcile` failed partway
**See also**
- `wikitool lint` - reports Misplaced and Nested Pages
- `wikitool index rebuild` - run after moving
- `instructions/page-lifecycle.md` - moving, renaming and deleting a page
<!-- /wikitool:reference-commands -->
Reference-Commands-Provenance.md
+153
@@ -0,0 +1,153 @@
*Generated from [`docs/reference/commands/provenance.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/docs/reference/commands/provenance.md) at v9.0.0 – edit the repository, not this wiki.*
# Provenance
Which raw source backs which page, and the generated provenance index that records it.
<!-- wikitool:toc -->
## Contents
- [`sources coverage`](#sources-coverage)
- [`sources trace`](#sources-trace)
- [`sources rebuild-index`](#sources-rebuild-index)
<!-- /wikitool:toc -->
<!-- wikitool:reference-commands -->
## `sources coverage`
List raw files with no source page, broken `raw_files:` references, and legacy directory/URL-only source pages.
**Synopsis**
- `wikitool sources coverage [--json]`
**Options**
```text
--json Print raw findings as JSON
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool sources coverage
tools/wikitool sources coverage --json
```
**Behaviour**
- Lists raw files with no source page, broken `raw_files:` references, and legacy directory/URL-only source pages.
- `--json` prints the same lists as JSON.
- Never fails; read-only and safe to retry freely.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
**See also**
- `wikitool sources trace` - follows one file or page
- `wiki-ingest` skill - turns an uncovered raw file into a source page
## `sources trace`
Trace provenance in either direction: raw file, or page.
**Synopsis**
- `wikitool sources trace --raw <path> | --page "<Title>"`
**Options**
```text
--raw <str> Raw file path to trace forward from
--page <str> Wiki page title to trace backward from
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool sources trace --page "Docker"
tools/wikitool sources trace --raw "raw/articles/llm-wiki.md"
```
**Behaviour**
- `--raw <path>`: raw file -> the source page(s) covering it -> the pages citing those.
- `--page "<Title>"`: page -> its sources -> their raw files.
- Read-only.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - Neither or both of `--raw`/`--page` given, or `--page` names an unknown page
- `1` - `--raw` names a file no source page covers - reported as a plain finding plus exit 1, not the usual `ERROR`-prefixed rejection
**See also**
- `wikitool sources coverage` - every uncovered raw file at once
- `wikitool xref link-source` - links a source page to what it mentions
## `sources rebuild-index`
Regenerate the `kb/provenance.md` reverse index.
**Synopsis**
- `wikitool sources rebuild-index [--dry-run]`
**Options**
```text
--dry-run Print the result instead of writing kb/provenance.md
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool sources rebuild-index
tools/wikitool sources rebuild-index --dry-run
```
**Behaviour**
- Regenerates the `kb/provenance.md` reverse index (raw file -> source page -> citing pages) from scratch.
- `--dry-run` prints the result instead of writing `kb/provenance.md`.
**Properties**
- Changes files: yes
- Idempotent: yes
- Atomic: Yes - the single provenance file is regenerated from scratch
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - An I/O error while writing `kb/provenance.md` (rare)
**See also**
- `wikitool index rebuild` - the page catalog, rebuilt alongside
- `instructions/publish-cycle.md` - where a write session runs this
<!-- /wikitool:reference-commands -->
Reference-Commands-Raw-material-and-uploads.md
+625
@@ -0,0 +1,625 @@
*Generated from [`docs/reference/commands/raw-material-and-uploads.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/docs/reference/commands/raw-material-and-uploads.md) at v9.0.0 – edit the repository, not this wiki.*
# Raw material and uploads
Bringing new material into `raw/`: fetching and capturing it, seeing what is waiting, accepting it, and reviewing what an MCP client uploaded.
<!-- wikitool:toc -->
## Contents
- [`raw fetch`](#raw-fetch)
- [`raw derive`](#raw-derive)
- [`raw capture`](#raw-capture)
- [`raw status`](#raw-status)
- [`raw pending`](#raw-pending)
- [`raw accept`](#raw-accept)
- [`upload list`](#upload-list)
- [`upload show`](#upload-show)
- [`upload accept`](#upload-accept)
- [`upload reject`](#upload-reject)
<!-- /wikitool:toc -->
<!-- wikitool:reference-commands -->
## `raw fetch`
Capture a web page the user names into `incoming/`: the HTML as received plus a derived text, for `raw accept` to promote.
**Synopsis**
- `wikitool raw fetch <url> [--name <stem>]` - Fetch the page and write `incoming/<stem>.html` and `incoming/<stem>.md`
- `wikitool raw fetch --html incoming/<file>.html --url <url>` - Derive the `.md` from a page a human saved from their browser - no network access
**Options**
```text
Arguments:
url The http(s) URL to fetch
Options:
--html <path> Derive the text from this HTML file under incoming/ (saved from
a browser) instead of fetching - no network access
--url <str> With --html: the page's URL, for the header and relative links
--name <str> File stem in incoming/ instead of the one computed from the URL
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool raw fetch https://example.org/blog/post
tools/wikitool raw fetch https://example.org/ --name example-start
tools/wikitool raw fetch --html incoming/post.html --url https://example.org/blog/post
```
**Behaviour**
- Writes into `incoming/` only, never into `raw/`: `raw accept` promotes both files afterwards, in one call, as one bundle under `raw/<YYYY>/<MM>/<stem>/`.
- The `.html` holds the response body byte for byte. The `.md` starts with a fixed header block (`fetched_by`, `url`, `final_url`, `retrieved`, `http_status`, `content_type`, `charset`, `title`, `derived_from`) followed by the derived text.
- Character set, in this order: byte-order mark, the HTTP `Content-Type` charset, a `<meta>` declaration in the first 4 KiB, UTF-8. Undecodable bytes are replaced with U+FFFD and reported; the header names the charset and where it came from.
- Derived text: the content root is `<main>`, else the single `<article>`, else `<body>`; `script`, `style`, `noscript`, `nav`, `header`, `footer`, `aside`, `form`, `template` and `svg` are dropped below it. Headings, lists, code, tables and links (made absolute) become Markdown. The same bytes always give the same text.
- The stem is the URL's last path segment without its extension, else its host name - ASCII, lowercase, `-`-separated, at most 60 characters; `--name` overrides it.
- A `text/plain` or `text/markdown` response is stored as received as `.txt`/`.md`; any other non-HTML response (PDF, image, ...) as received under the extension of its content type. Neither gets a derived file or a header.
- Only `http`/`https`, on every redirect hop too. 30 s for the whole transfer, 25 MiB at most. No cookies, no JavaScript: a derived text under 200 characters is written, with a warning to read it before accepting.
- `--html` derives the `.md` beside a page saved to `incoming/` from a logged-in browser - the way past a paywall or a script-rendered page. Its header carries `derived` (when the text was derived) instead of `retrieved`, `final_url`, `http_status` and `content_type`, and the `.html` is left untouched.
- Success prints the `raw accept` line for the written files and the `source_url` for the source page.
**Properties**
- Changes files: yes
- Idempotent: no
- Atomic: Yes for what it leaves behind - one or two new files in `incoming/`, created exclusively; a failure on the second removes the first
- Counts toward the session call budget: yes
- Can reach the network: yes
**Exit codes**
- `0` - success
- `1` - The URL is not `http`/`https` (also after a redirect), or neither or both of `<url>` and `--html` were given
- `1` - A target file already exists in `incoming/`
- `1` - The server answered with an HTTP error, could not be reached, took longer than 30 s, or sent more than 25 MiB
- `1` - raw fetch --html: `--url` is missing, the file is not directly in `incoming/` or does not exist, or `incoming/<stem>.md` already exists
**See also**
- `raw/CONTRACT.md` "Getting a URL in: `raw fetch`" - the rules and why
- `wikitool raw accept` - promotes the written files into `raw/`
- `instructions/wiki-ingest/SKILL.md` - where a URL to ingest starts
## `raw derive`
Derive reading text from a PDF, e-mail, Office file or web page in `incoming/`, or unpack a ZIP, and leave the original beside it, for `raw accept` to promote.
**Synopsis**
- `wikitool raw derive incoming/<file> [--name <stem>]` - Move the file into the new folder `incoming/<stem>/` as `original.<ext>` and write `incoming/<stem>/derived.md`
- `wikitool raw derive incoming/<folder>` - Write a `<stem>.md` beside every file in the folder that has a filter
**Options**
```text
Arguments:
path A file or a folder directly in incoming/ [required]
Options:
--name <str> Folder stem for a single file instead of the file's own stem
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool raw derive incoming/report.docx
tools/wikitool raw derive incoming/Angebot-XYZ.xlsx --name angebot-xyz
tools/wikitool raw derive incoming/meeting-files
tools/wikitool raw derive incoming/scans.zip
```
**Behaviour**
- Writes into `incoming/` only, never into `raw/`, and reads no network: `raw accept` promotes the folder afterwards as one source under `raw/<YYYY>/<MM>/<stem>/`.
- A single file directly in `incoming/` becomes a folder named after its stem (or `--name`) holding `original.<ext>` - the original bytes, moved - and `derived.md`. The fixed names keep the path inside the path budget; the header of `derived.md` records the file's own name as `original_name`.
- A folder is derived in place: each file with a filter gets `<stem>.md` beside it, nothing is renamed, and the files without a filter are named in the output.
- `derived.md` starts with a fixed header (`derived_by`, `filter`, `engine`, `derived`, `derived_from`, `original_name` for a renamed file, `title`, then the fields of the format) followed by the derived text. The same bytes always give the same text; only `derived` differs between runs. `engine` names the tool version and, where a package reads the format, its version.
- Formats: web pages (`.html`), Word (`.docx`), PowerPoint (`.pptx`), Excel (`.xlsx`, `.xlsm`), e-mail (`.eml`) and ZIP archives (`.zip`), read with the standard library only; PDF (`.pdf`, `pypdf`) and Outlook messages (`.msg`, `olefile`), each read by a package in `tools/requirements.txt` that is imported only when such a file is derived. A missing package refuses that format alone, with the preflight hint; `wikitool doctor` shows one `import-filter-<name>` line per package-backed format. A PDF gets `<!-- page N -->` markers and a WARN when it yields almost no text (a scan; there is no OCR); an encrypted PDF is refused. A `.msg` is a container like an `.eml`; an embedded message inside it is named, not read. The text of a mail (`.eml` and `.msg` alike) is its plain text, unless that is under 10% of the length of the HTML text - a "plain text version not available" placeholder - in which case it is the HTML text and a WARN names both lengths. The other version of the text in a `multipart/alternative` is not an attachment.
- A workbook (`.xlsx`, `.xlsm`) gets its cells as files: beside its `.md` a folder `<md stem>.sheets/` (`derived.sheets/` for a single file) holds one `NN-<sheet>.tsv` per sheet - tab-separated per RFC 4180, UTF-8, row 1 of the file is row 1 of the sheet - and, for a sheet with formulas, `NN-<sheet>.formulas.tsv` listing cell, formula and last stored value. The `.md` is the overview: one section per sheet with its range, the file names and a preview. Formulas are not recalculated; a shared formula is written out per cell, and one that cannot be is named `shared:<master cell>` with a WARN. The folder counts against the path budget and is accepted with the rest; an existing one is never overwritten.
- A ZIP and a mail are containers. A ZIP at the top of `incoming/` becomes the folder `incoming/<stem>/` holding its files under their own names, each with a `<stem>.md` where a filter reads it; the ZIP is removed and the output names it with its sha256. A mail becomes a folder with `original.eml`, `derived.md` and its attachments under their own names; a ZIP attached to it is unpacked once into a folder of its own, and an attached mail is quoted in `derived.md`, not unpacked. A ZIP inside a ZIP, an absolute or `..` path, a symbolic link, an encrypted entry, more than 1000 entries, a file over 25 MiB or more than 250 MiB in total refuse the whole call. Hidden entries (`__MACOSX/`, `.DS_Store`, anything starting with a dot) are named, not written. A name taken twice, or reserved (`original.*` and `derived.md` at the top of a mail's folder, `_capture.json` anywhere), gets `-2`, `-3` before its extension, and the output lists each rename.
- A name already taken under `raw/` is a WARN, not a refusal: whether the folder is a later edition of that source or a separate one is the human's call. A later edition of a derived source is replaced file by file with `raw accept --replaces`; there is no `--replaces-bundle` for it.
- After a success the output prints the warnings the filters raised, at most three `CHECK` lines per format that was derived in this run - what to look at before the text is trusted - and the `raw accept` line for the result.
- Never edit `derived.md` by hand: a better derivation is a new `raw derive` of the original.
**Properties**
- Changes files: yes
- Idempotent: no
- Atomic: Yes for what it leaves behind - everything is read, derived and checked before the first write; the files are created exclusively, and a failure part-way puts back what this call changed
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - The path is not directly in `incoming/`, does not exist, or no filter reads the file's extension
- `1` - The target folder, a `.md` or a workbook's `.sheets` folder already exists, or the path would be over the path budget
- `1` - The document cannot be read (damaged, encrypted), a container breaks one of its rules (a ZIP inside a ZIP, an unsafe path, a limit) or the filter's package is not installed
**See also**
- `raw/CONTRACT.md` "Getting a document in: `raw derive`" - the rules and why
- `wikitool raw accept` - promotes the folder into `raw/`
- `instructions/wiki-ingest/SKILL.md` - where a PDF, mail or Office file to ingest starts
## `raw capture`
Capture documentation from a git repository into `incoming/<bundle>/`, with a manifest naming the repository, ref rule and commit, for `raw accept` to promote.
**Synopsis**
- `wikitool raw capture <repo-url> --ref <branch|tag-pattern> --path <glob> [--path <glob> ...] --name <bundle> --fidelity <v> --authority <v>` - First capture of a repository
- `wikitool raw capture --update <raw-bundle> [--fidelity <v>] [--authority <v>]` - Capture the current state of an already-accepted bundle, for `raw accept --replaces-bundle`
**Options**
```text
Arguments:
repo_url The repository URL (ssh://, https:// or user@host:path)
Options:
--ref <str> A branch name (main) or a tag pattern (v*)
--path <str> A glob over the repository's paths; repeatable
--name <str> The bundle name - the folder in incoming/ and under raw/
--fidelity <str> How faithful the capture is (see `wikitool types describe
source`)
--authority <str> What the material may claim about its subject (see
`wikitool types describe source`)
--update <path> Capture the current state of this captured bundle under
raw/, from its own manifest
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool raw capture ssh://git@example.org/team/service.git --ref main --path 'docs/**/*.md' --path README.md --name service-docs --fidelity verbatim --authority normative
tools/wikitool raw capture https://example.org/team/lib.git --ref 'v*' --path docs --name lib-docs --fidelity verbatim --authority reporting
tools/wikitool raw capture --update raw/2026/10/service-docs
```
**Behaviour**
- Writes into `incoming/<bundle>/` only, never into `raw/`: `raw accept incoming/<bundle>` promotes it, as one folder, to `raw/<YYYY>/<MM>/<bundle>/`.
- `--ref` is a branch name (`main`) or a tag pattern (`v*`, any of `*?[`), which names the newest matching tag by version order. It resolves to one commit - for an annotated tag, the commit it points at.
- The commit is fetched by ref name, shallowly, into a bare cache repository per URL under `tools/.wikitool_capture/` (gitignored). Files are read from it as blobs, never through a checkout, so they are byte-identical to the repository's - no line-ending conversion, no filter.
- `--path` globs (repeatable) select files by their path in the repository, as git's `:(glob)` pathspec does: `*`, `?` and `[...]` do not cross `/`; `**` as a whole segment spans any number of directories; a glob without wildcards also takes everything below it as a directory. Files keep their repository paths inside the bundle.
- Never captured, each named in the output with its reason: a file whose first line starts with `<!-- wikitool:export`, a symlink, a submodule, a file over 25 MiB, a path with a segment starting with `.`, a file named `_capture.json`, and a Git LFS pointer.
- `incoming/<bundle>/_capture.json` records `schema`, `repo`, `ref`, `commit`, `paths`, `captured` (UTC), `fidelity`, `authority` and `files`. It is the only declaration that and how this instance follows the repository.
- Only `ssh://`, `https://` and the scp form `user@host:path`; a URL with a password or token in it is refused, since the manifest is committed. Git runs with the host's own credentials and configuration, restricted to those protocols, and never prompts: a repository that asks for credentials is reported as unreachable. Each git call times out after 120 s.
- `--update <raw-bundle>` takes URL, ref rule and globs from that bundle's manifest and writes the current state to `incoming/<same bundle name>/`; `--fidelity`/`--authority` default to the manifest's values, and passing one is the way to correct it.
- Refused before anything is written: `incoming/<bundle>` already exists, the name is already taken under `raw/` (a first capture), nothing matches the globs, or a target path - measured at its later place under `raw/` - is over the path budget.
- Success prints the commit and the `raw accept` line that comes next.
**Properties**
- Changes files: yes
- Idempotent: no
- Atomic: Yes for what it leaves in `incoming/` - the bundle is written into a hidden staging directory and renamed into place in one step; a failure removes the staging directory. The git cache under `tools/.wikitool_capture/` is updated either way
- Counts toward the session call budget: yes
- Can reach the network: yes
**Exit codes**
- `0` - success
- `1` - The URL is not `ssh://`, `https://` or `user@host:path`, or carries a password; the `--ref` rule, a `--path` glob or `--name` is unusable; or an argument of the other variant was mixed in
- `1` - `--fidelity`/`--authority` is missing on a first capture, or names `unknown` or a value outside the schema's enum
- `1` - The repository could not be reached, asked for credentials, or timed out; or no branch or tag matches `--ref`
- `1` - `incoming/<bundle>` already exists, the name is already taken under `raw/`, nothing matches the globs, or a path would be over the budget
- `1` - raw capture --update: The path is not a bundle under `raw/` with a readable `_capture.json`, or its manifest names a URL that is refused
**See also**
- `raw/CONTRACT.md` "Getting a repository in: `raw capture`" - the rules and why
- `wikitool raw accept` - promotes the bundle; `--replaces-bundle` for a new edition
- `wikitool raw status` - which captured bundles have fallen behind their repository
## `raw status`
Report which captured bundles under `raw/` have fallen behind their repository, with each changed file as `A`/`M`/`D`.
**Synopsis**
- `wikitool raw status [--json]`
**Options**
```text
--json Print one JSON object per captured bundle
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool raw status
tools/wikitool raw status --json
```
**Behaviour**
- Reads every `_capture.json` under `raw/` and resolves its ref rule with `git ls-remote`. Only where the commit moved is the new commit fetched, and compared - with the same globs and exclusions `raw capture` applies - against the bundle's files under `raw/`.
- A bundle whose commit moved without a change inside its globs is reported as unchanged, not listed. A tag pattern follows new matching tags only, never new commits on a branch.
- Each changed bundle prints `old -> new` and its files as `A`/`M`/`D`, grouped by owning source page with its citing pages, followed by the `raw capture --update` and `raw accept --replaces-bundle` lines that take the new edition in.
- An unreachable repository, a refused URL or an unreadable manifest is one line beside the others, never an abort; the exit status is 0 as long as the command itself ran.
- `--json` prints one object per bundle: `bundle`, `repo`, `ref`, `old`, `new`, `changed`, `files` (`[{path, status}]`), `error` (or null).
- Git runs exactly as for `raw capture`: the host's credentials, only `ssh`/`https`, never a prompt, 120 s per call.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only for the tree - writes nothing but the git cache under `tools/.wikitool_capture/`
- Counts toward the session call budget: yes
- Can reach the network: yes
**Exit codes**
- `0` - success
- `0` - A repository could not be reached, its manifest or URL was refused, or no ref matches its rule
**See also**
- `wikitool raw capture` - `--update` captures the new edition
- `wikitool raw accept` - `--replaces-bundle` takes it in
- `raw/CONTRACT.md` "Getting a repository in: `raw capture`"
## `raw pending`
List what waits in `incoming/`, oldest first, and name the entry an ingest without an argument takes next.
**Synopsis**
- `wikitool raw pending [--json]`
**Options**
```text
--json Print the candidates as JSON
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool raw pending
tools/wikitool raw pending --json
```
**Behaviour**
- A candidate is a top-level entry of `incoming/`: a single `file`, a `bundle` of top-level files sharing a stem (a `raw fetch` pair), or a `folder` with every file below it. Dotfiles, empty directories and their contents are none; `mcp-upload/` is outside `incoming/` and never listed.
- Order: oldest first by modification time. A bundle or folder counts as new as its newest file; a tie goes by name. The mtime is when a document last changed only if it was copied with its timestamps kept (`cp -p`, `rsync -a`, an unpacked archive) - for a download or a `raw fetch` it is merely when it was dropped.
- Each candidate shows its path(s), kind, file count and mtime, and whether `raw accept` would take it as it stands - the same checks, minus `--fidelity`/`--authority`. One it would refuse is listed with the reason and skipped: it needs a human.
- The default is the first candidate `raw accept` would take, and the output names it.
- `--json` prints the same candidates in the same order: `kind`, `paths`, `files`, `mtime`, `acceptable`, `reason`, `default`, `hint`.
- A lone file that a `raw derive` filter reads (a PDF, mail, Office file or web page) is listed with the `raw derive` line for it, in text output and as `hint` in JSON (`null` for every other candidate). The hint changes nothing about the order or the default.
- Reads directory listings and `lstat` only, never a file's content; an empty `incoming/` is exit 0 with nothing to do.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
**See also**
- `wikitool raw accept` - promotes the chosen candidate
- `instructions/wiki-ingest/SKILL.md` - ingest without an argument starts here
- `raw/CONTRACT.md` "Getting a file in: incoming/" - candidates and order, and why
## `raw accept`
Promote one or more files, or one folder, from `incoming/` into `raw/`.
**Synopsis**
- `wikitool raw accept <file> [<file> ...] --fidelity <v> --authority <v> [--page "<Title>"] [--dry-run]` - Promote one or more files from `incoming/` into today's `raw/<YYYY>/<MM>/` shard
- `wikitool raw accept incoming/<folder> --fidelity <v> --authority <v> [--dry-run]` - Promote a whole folder as one source, its structure kept, into `raw/<YYYY>/<MM>/<folder>/`
- `wikitool raw accept <file> --replaces <raw-path> [--fidelity <v>] [--authority <v>] [--dry-run]` - Overwrite one existing raw file in place with a new edition
- `wikitool raw accept incoming/<bundle> --replaces-bundle <raw-bundle> [--dry-run]` - Replace a captured bundle as a whole with the new edition `raw capture --update` wrote
**Options**
```text
Arguments:
files... One or more files directly in incoming/, all belonging to the same
source - or one folder in incoming/, accepted whole as one source
[required]
Options:
--fidelity <str> How faithful the capture is (see `wikitool types
describe source`). Required unless --replaces is
given, where it is the optional, sanctioned way to
correct an already-set value
--authority <str> What the material may claim about its subject (see
`wikitool types describe source`). Required unless
--replaces is given, where it is the optional,
sanctioned way to correct an already-set value
--page <str> Extend this existing source page's raw_files: with
the promoted file(s), folding in its already-
promoted file if this raises it past one
--replaces <path> Overwrite this existing raw/ file in place with the
single incoming/ file passed alongside it - the only
sanctioned way past the stem-uniqueness rule below
--dry-run List what would move without writing
--replaces-bundle <path> Replace this captured bundle under raw/ as a whole
with the new edition in the incoming/ folder passed
alongside it (written by raw capture --update)
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool raw accept incoming/docker-cheatsheet.md --fidelity verbatim --authority reporting
tools/wikitool raw accept incoming/part-2.md --fidelity verbatim --authority reporting --page "Source - Docker Cheatsheet"
tools/wikitool raw accept incoming/projekt-x --fidelity verbatim --authority reporting
tools/wikitool raw accept incoming/cluster.md --replaces raw/documents/cluster.md
tools/wikitool raw accept incoming/chemenu --replaces-bundle raw/2026/10/chemenu
```
**Behaviour**
- Promotes files from `incoming/` into `raw/<YYYY>/<MM>/`, computed from the accept date rather than chosen by hand. A file argument must sit directly in `incoming/`; a file inside a subdirectory is refused, since a subdirectory is a source of its own.
- A folder argument (`incoming/<folder>`) is one source: every file below it moves to `raw/<YYYY>/<MM>/<folder>/` at the same relative path, and the directories left empty are removed - `incoming/<folder>` no longer exists afterwards. The folder name is the bundle name; two `README.md` in different subfolders are no conflict.
- A folder is accepted alone - no other argument, no `--page`, no `--replaces` - with one `--fidelity`/`--authority` pair for all of it. It is refused, before anything moves, if it is empty, or if a hidden entry (name starting with `.`), a symlink or a special file sits anywhere below it; the refusal names each one.
- One file promoted alone lands with no directory of its own; several files in one call nest under `raw/<YYYY>/<MM>/<stem>/`, named after the first file's stem.
- `--fidelity`/`--authority` are required on a plain accept (`types describe source` lists the values); `unknown` is refused - it is backfill-only.
- `--page "<Title>"` additionally extends that existing source page's `raw_files:` in the same call and writes both capture fields onto it - refused if it already carries a different value, since a capture field is fixed once.
- If `--page` raises the page past one file, its already-promoted file is folded into a bundle at *its own* parent directory, not today's shard, so a bundle never mixes an old and a new capture date - after checking that file has no other owner.
- Every name occupied anywhere under `raw/` - file stems and bundle directory names alike, old type directories and date shards together - stays unique: a promote whose target name already belongs to something this call does not itself own is refused, naming `--replaces` and renaming in `incoming/` as the two routes, without recommending either.
- `--replaces <raw-path>` overwrites that file in place with the single incoming file (same filename required), leaves every page's `raw_files:` untouched and writes no `kb/` page; the previous edition survives only in `git log --follow <raw-path>`.
- With `--replaces`, `--fidelity`/`--authority` are optional, and passing one overwrites the owning page's already-set value - the one path the fixed-once rule does not block.
- `--replaces` refuses a target with more than one owning source page; with none, it replaces anyway and says so. It cannot be combined with `--page` or with more than one incoming file.
- `--replaces` prints the source page (if any) and its citing pages, so their update lands in the same commit as the replacement.
- A file already at its computed destination is what "already exists" reports, not a partial prior run to resume - safe to retry as-is once a cause is fixed.
- A folder carrying `_capture.json` at its top is a captured bundle (`raw capture`): `--fidelity`/`--authority` come from that manifest only and are refused on the command line; the folder must hold exactly the files the manifest lists. `_capture.json` goes in no `raw_files:`.
- `--replaces-bundle <raw-bundle>` replaces a captured bundle under `raw/` with the new edition in `incoming/<bundle>` at the same address: files gone from the new edition are removed, emptied directories `rmdir`ed, and afterwards the bundle holds exactly the new manifest's files plus `_capture.json`. `raw_files:` is left untouched; an owning source page's `fidelity`/`authority` is overwritten where the new manifest differs.
- `--replaces-bundle` prints each changed file as `A`/`M`/`D`, grouped by owning source page with its citing pages, and the `touch --page ... --add/--remove raw_files=` lines that follow; `git diff` on the bundle shows the `M` and `D` files, the `A` files appear only in that list because they are still untracked.
- `--replaces` and `--page` refuse a target inside a captured bundle; the refusal names `raw capture --update` and `--replaces-bundle`. A file named `_capture.json` anywhere but at a captured folder's top is refused - the name is reserved.
- `--dry-run` reports the moves without making them.
**Properties**
- Changes files: yes
- Idempotent: no
- Atomic: `raw accept`: No - one filesystem move per file, then (with `--page`) one page write. With a folder: No - one move per file, then one `rmdir` per emptied directory; a half-accepted folder is not resumed. `raw accept --replaces`: No - one `unlink()` + one `rename()`, plus (if `--fidelity`/`--authority` was given) one page write. `raw accept --replaces-bundle`: No - one `unlink()` per removed file, one `rename()` per added or modified file, then the manifest, then one `rmdir` per emptied directory, plus one page write per owning page whose capture fields change; a half-replaced bundle is not resumed
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - raw accept: A file does not exist or is not directly in `incoming/`; two files in one call share a filename; a target path already exists; or a target path would be over the path budget (160 UTF-16 code units below the instance root)
- `1` - raw accept incoming/<folder>: The folder is not directly in `incoming/`, is combined with another argument, `--page` or `--replaces`, holds no file, or holds a hidden entry, a symlink or a special file; a target path is over the budget; or the folder name is already occupied under `raw/`
- `1` - A file - given directly, inside a folder, or as `--replaces`/`--replaces-bundle` material - is a guideline export: its first line, after an optional BOM, starts with `<!-- wikitool:export`
- `1` - raw accept: `--fidelity`/`--authority` is missing, or names `unknown` or a value outside the schema's enum
- `1` - raw accept: The target name is already occupied anywhere under `raw/` by something the call does not own
- `1` - raw accept: `--page` names an unknown page or one with no `raw_files:` yet, an existing `raw_files:` entry is missing on disk, a file to be moved has more than one owning page, or `--page` would overwrite an already-set `fidelity`/`authority` with a different value
- `1` - raw accept --replaces: More than one incoming file, or `--page` also given
- `1` - raw accept --replaces: The incoming file does not exist or is not directly in `incoming/`, its filename differs from the target's, or the target does not lie under `raw/` or does not exist
- `1` - raw accept --replaces: `--fidelity`/`--authority` names `unknown` or a value outside the schema's enum, or the target has more than one owning source page
- `1` - raw accept --replaces / --page: The target, or a file the page already has, lies in a captured bundle
- `1` - raw accept incoming/<bundle>: A captured folder is given `--fidelity`/`--authority`, its `_capture.json` cannot be read or names an invalid value, or its files differ from the ones the manifest lists
- `1` - raw accept --replaces-bundle: Either side has no `_capture.json`, the two manifests name different repositories, the folder's name differs from the bundle's, `--page`, `--replaces`, `--fidelity` or `--authority` was also given, the folder does not match its manifest, a new path is over the budget, or a file of the bundle has more than one owning source page
**See also**
- `raw/CONTRACT.md` "Getting a file in: incoming/" - the rules and why
- `wikitool raw pending` - what is waiting in `incoming/`, and which entry is next
- `wikitool raw capture` - writes a captured bundle, or its new edition, into `incoming/`
- `wikitool types describe source` - the capture field values
- `wikitool new source` - the source page for a promoted file
## `upload list`
List every MCP submission currently waiting in the quarantine (`mcp-upload/`).
**Synopsis**
- `wikitool upload list [--json]`
**Options**
```text
--json Print every waiting submission as JSON
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool upload list
tools/wikitool upload list --json
```
**Behaviour**
- Lists every submission waiting in `mcp-upload/`, oldest id first: id, filename, size, submitter.
- Only ever non-empty when `.wikitool-upload.json` opts the checkout into the MCP server's `submit` tool.
- A submission directory with a corrupt manifest is silently skipped.
- Never fails; read-only and safe to retry freely.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
**See also**
- `wikitool upload show` - one submission's manifest
- `docs/how-to/set-up-the-mcp-server.md` § "Optional: the submit path"
## `upload show`
Print one submission's manifest in full.
**Synopsis**
- `wikitool upload show <id> [--json]`
**Options**
```text
Arguments:
submission_id A submission id from `upload list` [required]
Options:
--json
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool upload show <id>
```
**Behaviour**
- Prints one submission's manifest in full: filename, size, sha256, submitter, submitter source (the header name, not a claim the header was honest), submission time.
- What a reviewer reads before `upload accept`.
- Read-only.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - Unknown or malformed submission id
**See also**
- `wikitool upload accept` - promotes it after review
- `wikitool upload reject` - declines it
## `upload accept`
**Upload Review Gate:** promote a submission's file from quarantine into `incoming/`.
**Synopsis**
- `wikitool upload accept <id> [--confirm TOKEN]`
**Options**
```text
Arguments:
submission_id A submission id from `upload list` [required]
Options:
--confirm <str> The token from a prior refusal, once a human has reviewed it
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool upload accept <id>
tools/wikitool upload accept <id> --confirm <token> # re-run after exit 42, once the user approved the manifest
```
**Behaviour**
- Promotes a submission's file from `mcp-upload/<id>/` into `incoming/`, deletes the quarantine directory, and appends an `accepted` event to `mcp-upload/ledger.jsonl`.
- Upload Review Gate: without a matching `--confirm`, exits 42 and prints the manifest in full plus the exact re-run line - one submission at a time. The gate check runs before anything is moved.
- The token digests id, filename, size, sha256 and submitter, so an edited or superseded manifest invalidates it.
- An occupied `incoming/<filename>` is an ordinary validation error, not the gate.
**Properties**
- Changes files: yes
- Idempotent: no
- Atomic: No - one filesystem move, one directory delete, one ledger append; the gate check runs first, before any of them
- Counts toward the session call budget: yes
- Can reach the network: no
- Gates: upload-review - see [exit codes and gates](Reference-Exit-codes)
**Exit codes**
- `0` - success
- `1` - Unknown or malformed submission id, or the submission's file is missing from `mcp-upload/<id>/`
- `1` - `incoming/<filename>` already exists
- `42` - Upload Review Gate: `--confirm` is absent or does not match the manifest's current token
**See also**
- `wikitool upload show` - the manifest to review
- `wikitool raw accept` - the next step for the file in `incoming/`
- `instructions/gates.md` - the gate procedure
## `upload reject`
Delete a submission's material, keeping only its ledger trail.
**Synopsis**
- `wikitool upload reject <id> --reason "<why>"`
**Options**
```text
Arguments:
submission_id A submission id from `upload list` [required]
Options:
--reason <str> Why this submission was declined [required]
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool upload reject <id> --reason "duplicate of raw/articles/llm-wiki.md"
```
**Behaviour**
- Appends an append-only `rejected` event naming the reason and the sha256 of what was declined, then deletes the submission's material.
- The ledger write happens first, so an interruption still leaves the reason on record.
- No gate - rejecting needs no clearance, only accepting does.
- Not idempotent: a second call with the same id reports "unknown id", which confirms the first call worked.
**Properties**
- Changes files: yes
- Idempotent: no
- Atomic: No - one ledger append, then one recursive delete; the ledger write happens first, so an interruption still leaves the reason on record
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - Unknown or malformed submission id, or an empty `--reason`
**See also**
- `wikitool upload show` - the manifest to review
<!-- /wikitool:reference-commands -->
Reference-Commands-Telemetry.md
+116
@@ -0,0 +1,116 @@
*Generated from [`docs/reference/commands/telemetry.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/docs/reference/commands/telemetry.md) at v9.0.0 – edit the repository, not this wiki.*
# Telemetry
Reading and scoring the traces that sessions leave behind.
<!-- wikitool:toc -->
## Contents
- [`eval sessions`](#eval-sessions)
- [`eval score`](#eval-score)
<!-- /wikitool:toc -->
<!-- wikitool:reference-commands -->
## `eval sessions`
List the sessions that have a trace under `reports/telemetry/`.
**Synopsis**
- `wikitool eval sessions [--json]`
**Options**
```text
--json Print the list as JSON
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool eval sessions
tools/wikitool eval sessions --json
```
**Behaviour**
- Lists the sessions that have a trace under `reports/telemetry/`, most recent first.
- Never fails; an empty list is a valid answer.
- Read-only and exempt from the Iteration Budget Gate.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: no
- Can reach the network: no
**Exit codes**
- `0` - success
**See also**
- `wikitool eval score` - scores one of them
- `docs/explanation/telemetry-and-evaluation.md` - how telemetry and evaluation work
## `eval score`
Score one traced session.
**Synopsis**
- `wikitool eval score [--session <id>] [--json] [--markdown out.md] [--save] [--fail-on-error]`
**Options**
```text
--session <str> Session to score. Defaults to this shell's session, the
same id the budget gate uses.
--json Print the scorecard as JSON
--markdown <path> Write a markdown scorecard here, conventionally under
reports/evals/.
--save Write the scorecard to
reports/evals/<date>/<session>.{json,md}.
--fail-on-error Exit non-zero when the tree has hard errors or an invariant
was violated.
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool eval score
tools/wikitool eval score --session wiki-1727330000 --save
```
**Behaviour**
- Scores one traced session: structural state from `lint`'s own checks (L1) plus trajectory rules over the trace (L2) - was a refused call repeated unchanged, was a gate flag passed without that gate having refused anything, did a publish of `kb/` pages go unlogged.
- Defaults to the current session; `--session <id>` picks another.
- `--save` writes `reports/evals/<date>/<session>.{json,md}`; `--markdown` writes the report to the named file.
- A session records nothing when telemetry is off - `WIKI_TRACE=0`, or a distributed instance with no opt-in (`wikitool doctor` says which) - so an absent trace is not necessarily a fault.
- Read-only over `kb/`, safe to retry, and exempt from the Iteration Budget Gate.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only, apart from the files `--save`/`--markdown` write
- Counts toward the session call budget: no
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - No trace exists for the named session
- `1` - Only with `--fail-on-error`: the tree has hard errors or an invariant was violated
**See also**
- `wikitool eval sessions` - which session ids exist
- `docs/explanation/telemetry-and-evaluation.md` - the scoring levels
<!-- /wikitool:reference-commands -->
Reference-Commands-Types-instructions-and-docs.md
+527
@@ -0,0 +1,527 @@
*Generated from [`docs/reference/commands/types-instructions-and-docs.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/docs/reference/commands/types-instructions-and-docs.md) at v9.0.0 – edit the repository, not this wiki.*
# Types, instructions and docs
The page types, the instruction and skill layer, and the documentation checks and generators that keep the docs in step with the code.
<!-- wikitool:toc -->
## Contents
- [`types list`](#types-list)
- [`types describe`](#types-describe)
- [`instructions sync`](#instructions-sync)
- [`instructions verify`](#instructions-verify)
- [`instructions list`](#instructions-list)
- [`docs verify`](#docs-verify)
- [`docs toc`](#docs-toc)
- [`docs prerequisites`](#docs-prerequisites)
- [`docs contract`](#docs-contract)
- [`docs reference`](#docs-reference)
<!-- /wikitool:toc -->
<!-- wikitool:reference-commands -->
## `types list`
List every type-spec under `types/`.
**Synopsis**
- `wikitool types list [--json]`
**Options**
```text
--json Print raw findings as JSON
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool types list
tools/wikitool types list --json
```
**Behaviour**
- Lists every type-spec under `types/`: name, schema path, subtype field, and description - which page types exist, without reading `types/*.md` directly.
- Never fails; read-only and safe to retry freely.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
**See also**
- `wikitool types describe <name>` - one type's full contract
## `types describe`
Print one type's full contract.
**Synopsis**
- `wikitool types describe <name> [--json]`
**Options**
```text
Arguments:
name Type name, e.g. 'entity' (see `types list`) [required]
Options:
--json Print raw findings as JSON
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool types describe source
tools/wikitool types describe project --json
```
**Behaviour**
- Prints one type's full contract: required and optional frontmatter fields with enums, its subtype field (if any), and its authoring body.
- Where the type-spec declares `guidance:`, the stack-owned `types/<name>.guidance.md` is composed in, so a `root: kb` type's contract reads as one answer even though it may live in two files. `--json` reports it separately as `guidance`/`guidance_path`, absent for a type with none.
- The generated table-of-contents region a long type-spec (or guidance file) carries is stripped from this output.
- Read-only.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - Unknown type name
**See also**
- `wikitool types list` - every type
- `wikitool new <type>` - scaffolds a page of the type
## `instructions sync`
Publish every `instructions/<name>/SKILL.md` into the harness skill directories.
**Synopsis**
- `wikitool instructions sync [--force]`
**Options**
```text
--force Replace a target directory whose contents differ from the source
and was not generated by sync. Without this, sync refuses instead
of deleting content it did not create.
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool instructions sync
```
**Behaviour**
- Publishes every `instructions/<name>/SKILL.md` into `.agents/skills/` and `.claude/skills/` as **copies**, and deletes published skills whose source is gone.
- Both targets are gitignored, so a fresh clone runs this once.
- Re-running repairs a drifted copy: the source always wins.
- `--force` is required only to replace a target directory that is not a published skill at all (no `SKILL.md` in it).
- Each copy is idempotent, so a re-run converges even after a partial failure.
**Properties**
- Changes files: yes
- Idempotent: yes
- Atomic: No - one directory copy per skill per target (`.agents/skills/`, `.claude/skills/`); each copy is idempotent, so a re-run converges even after a partial failure
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - No skills found under `instructions/`
- `1` - A target directory is not a published skill (no `SKILL.md`) and `--force` was not passed
**See also**
- `instructions/bootstrap.md` - the fresh-clone procedure that runs this
- `wikitool instructions verify` - checks the copies match
## `instructions verify`
Check the instruction layer.
**Synopsis**
- `wikitool instructions verify`
**Options**
```text
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool instructions verify
```
**Behaviour**
- Flat instructions validate against `types/instruction.schema.yaml`, and each `SKILL.md` carries the frontmatter its harness reads.
- No `SKILL.md` carries a relative markdown link: `sync` copies it to a different depth than the source, so a `SKILL.md` references a target as a repo-root-relative plain path instead (`instructions/CONTRACT.md` § "A skill's outbound reference is a plain path, not a link").
- Every published copy is byte-identical to its source. Missing *every* copy is reported as "run sync", not as drift - that is a clean checkout.
- No instruction is left that nothing references; one marked `manual: true` must instead not be linked from AGENTS.md, CLAUDE.md or a skill.
- Nothing under `instructions/dev/` is referenced from outside it; a `<!-- dist:strip-start/end -->` block is exempt (`instructions/CONTRACT.md`).
- Read-only.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - Nothing found under `instructions/` at all, or a malformed instruction or `SKILL.md`
- `1` - A `SKILL.md` carries a relative markdown link
- `1` - A published copy drifted from its source
- `1` - An instruction nothing references, or a `manual: true` one that IS linked from AGENTS.md, CLAUDE.md or a skill and so risks running implicitly
- `1` - Something under `instructions/dev/` is referenced from outside it and outside a `dist:strip` block
**See also**
- `wikitool instructions sync` - publishes the copies
- `instructions/CONTRACT.md` - the rules this checks
## `instructions list`
List the flat instructions with their descriptions.
**Synopsis**
- `wikitool instructions list [--json]`
**Options**
```text
--json Print the listing as JSON
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool instructions list
tools/wikitool instructions list --json
```
**Behaviour**
- Lists the flat instructions with their descriptions - how the instruction layer is discovered; `search` covers `kb/` only.
- Never fails: an empty `instructions/` prints "No instructions found." Read-only and safe to retry freely.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
**See also**
- `wikitool search` - the same question for `kb/`
## `docs verify`
Check the docs that mirror the code.
**Synopsis**
- `wikitool docs verify`
**Options**
```text
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool docs verify
```
**Behaviour**
- Checks the docs that mirror the code. The name is about documentation parity, not the `docs/` directory - it reads that only for its layout and its generated reference pages, and never requires one.
- Commands: every command has a `cli_contract` record and is listed in `cli_contract.GROUPS`, in both directions; every command's non-hidden flags appear in its record's SYNOPSIS and vice versa; `tools/CONTRACT.md`'s generated `<!-- wikitool:commands -->` region matches what `docs contract` would write; no command's rendered `--help`/`-h` text cites an issue number.
- Collections: every directory under `kb/` has a `COLLECTION.md` and no directory outside it does; every collection declares `profile:` and a `required_by_stack:` that agrees with the stack's own list.
- Types: every type the stack lists (currently `source` and `project`) has a type-spec of that name whose schema requires the field the stack list names (`raw_files:`/`state:`); every file under `types/` declaring `type: types/type-spec.md` validates against `types/type-spec.schema.yaml`; every subtype template `types/<type>.<value>.md` (any `<value>` but `guidance`) sits beside a type-spec declaring `subtype_field:`, names a value that field's schema enum allows, and carries no frontmatter; no pre-migration `type: entity` blocks are left in the contracts.
- `kb/CONVENTIONS.md`, if it exists at all, names all three tool-owned section headings; every stage contract is present.
- The `.gitignore` canaries clear in both directions: nothing ignored under `raw/`/`kb/`, `incoming/` ignored, everything ignored under `reports/` and the published skill directories.
- No `.md`/`.template` file `dist export` would ship cites an issue number. A `<!-- dist:strip-start/end -->` region is exempt: the check reads the export plan's text, from which it is already gone.
- Every reference file `docs toc` covers carries the current table-of-contents region for its own headings - missing and stale are one check.
- Every relative markdown link in one of those reference files resolves to an existing file. A target's `#anchor` suffix is stripped first, and code fences and inline code spans are masked before scanning, so link syntax shown as an example is not mistaken for a real reference.
- Every file under `docs/` sits in one of its areas - `tutorials/`, `how-to/`, `reference/`, `explanation/` or `dev/` - never directly in `docs/` or in another directory.
- `AGENTS.md` § Gates names exactly the gates in `cli_contract.GATES` (the bold entry names, `- **<Name>.**`), and every gate in that register that ends in exit 42 is declared by at least one command record. A record naming a gate outside the register fails when `wikitool` starts, not here.
- If `docs/` exists: every generated region of the human reference under `docs/reference/` matches what `docs reference` would write, every `cli_contract.GROUPS` entry has its page under `docs/reference/commands/`, and no page there belongs to no group. That includes the four configuration regions in `docs/reference/configuration.md`. The skill catalogue and the page types are checked only in the origin repo (no `.wikitool-release.json`): they describe the stack's shipped defaults, which an instance's own type-specs and instructions may differ from.
- `INSTALL.md` carries one generated `<!-- wikitool:prerequisites -->` region per platform value of `tools/prerequisites.txt` (`prerequisites-<platform>` for a platform-specific one), each current; and the `<!-- setup-question: <key> -->` markers in `instructions/setup-instance.md` and `INSTALL.md` name the same set of keys, so a question the agent asks is never one the human guide leaves out, nor the reverse.
- `VERSION` parses and the newest versioned `CHANGES.md` entry names it. An entry that crosses the compatibility boundary from the last release carries `**Breaking Change:**`, has a migration document or `**Migration:** none required`, and its `**Migration:**` line names every required migration document targeting it.
- The newest versioned `CHANGES.md` entry is at most 32 000 bytes (UTF-8) as `version notes` prints it, and each of its `###` sections is exactly one paragraph of at most 1 000 bytes, heading line excluded. Older entries and a changelog with no versioned entry are not checked.
- Read-only.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - A command, contract, or type-form mismatch
- `1` - `VERSION` and the newest `CHANGES.md` entry disagree, or a boundary-crossing entry lacks its breaking line, its migration, or the migration document's path
- `1` - The newest `CHANGES.md` entry is over 32 000 bytes, or one of its `###` sections has more than one paragraph or a paragraph over 1 000 bytes
- `1` - The `<!-- wikitool:commands -->` region of `tools/CONTRACT.md` is stale
- `1` - A type-spec's own frontmatter fails its schema
- `1` - A subtype template `types/<type>.<value>.md` has no type-spec with a `subtype_field:` beside it, names a value outside that field's enum, or carries frontmatter
- `1` - A shipped `.md`/`.template` cites an issue number
- `1` - A region of the human reference under `docs/reference/` is stale
- `1` - A page of the human reference is missing or lacks its marker pair, or a page under `docs/reference/commands/` belongs to no group
- `1` - A reference file's table-of-contents region is missing or stale
- `1` - A file sits directly in `docs/`, or in a directory there that is not one of its five areas
- `1` - `AGENTS.md` § Gates and `cli_contract.GATES` disagree, or a register gate that ends in exit 42 is declared by no command record
- `1` - A reference file's relative markdown link does not resolve to an existing file
- `1` - An `INSTALL.md` prerequisites region is stale
- `1` - An `INSTALL.md` prerequisites region is missing, or names a platform no tool has
- `1` - A setup question is marked in one of `instructions/setup-instance.md` and `INSTALL.md` but not the other
**See also**
- `wikitool docs toc` - regenerates tables of contents
- `wikitool docs contract` - regenerates the commands region
- `wikitool docs reference` - regenerates the human reference
- `wikitool docs prerequisites` - regenerates `INSTALL.md`'s prerequisites lists
- `wikitool instructions verify` - the same kind of check for `instructions/`
## `docs toc`
Create, refresh or remove the generated table-of-contents region.
**Synopsis**
- `wikitool docs toc [--apply]`
**Options**
```text
--apply Write changes; default is dry-run (preview only)
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool docs toc
tools/wikitool docs toc --apply
```
**Behaviour**
- Creates, refreshes or removes the generated table-of-contents region on every reference file over 100 lines: `AGENTS.md`, every stage contract, `kb/CONVENTIONS.md`, every `kb/*/COLLECTION.md`, every flat `instructions/**.md` file, every `types/*.md` type-spec, and every `docs/` page - each together with the `<name>.template` it ships as, where one exists.
- The scope is computed from those categories rather than listed, so a file added later is in scope without a code change.
- Out of scope: every `SKILL.md`, the human docs at the root and beside the code (`README.md`, `CHANGES.md`, `INSTALL.md`, `tools/README.md`), and every subtype template `types/<type>.<value>.md` - `new` copies it into a page, so it never carries a region whatever its length.
- Dry-run by default (prints which files would change); `--apply` writes. Idempotent: a re-run after an interruption converges rather than doubling a region.
- If `docs verify` still reports a stale region after `--apply`, the file's `##` headings changed in between; run it again.
**Properties**
- Changes files: yes
- Idempotent: yes
- Atomic: `--apply` rewrites each named file in place, one at a time and idempotently, so a re-run after an interruption converges rather than doubling a region; the dry-run form is read-only
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `0` - Never fails on content: a file with no `##` heading, or one at or under the threshold, is simply left without a region
**See also**
- `wikitool docs verify` - checks every region is current
## `docs prerequisites`
Regenerate `INSTALL.md`'s prerequisites lists from `tools/prerequisites.txt`.
**Synopsis**
- `wikitool docs prerequisites [--apply]`
**Options**
```text
--apply Write changes; default is dry-run (preview only)
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool docs prerequisites
tools/wikitool docs prerequisites --apply
```
**Behaviour**
- Rewrites each `<!-- wikitool:prerequisites -->` region in `INSTALL.md` (tools every platform needs) and `<!-- wikitool:prerequisites-<platform> -->` region (tools only that platform needs) from the manifest: one list item per tool, its label and minimum version. The manifest's reason field stays out - it explains a tool to whoever reads the preflight's own output, while the region only says what to install.
- Never places a region: where a list belongs in the human guide is that guide's own decision. A region the manifest calls for but the file lacks is an error naming the marker pair to add.
- Dry-run by default (says whether the file would change); `--apply` writes.
- `docs verify` checks the result stays current.
**Properties**
- Changes files: yes
- Idempotent: yes
- Atomic: Yes - every region is rewritten in one file write
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - `INSTALL.md` is missing, or lacks a region the manifest calls for
**See also**
- `wikitool docs verify` - checks every region is current
## `docs contract`
Regenerate `tools/CONTRACT.md`'s `<!-- wikitool:commands -->` region.
**Synopsis**
- `wikitool docs contract [--apply]`
**Options**
```text
--apply Write changes; default is dry-run (preview only)
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool docs contract
tools/wikitool docs contract --apply
```
**Behaviour**
- Rebuilds the region from every `cli_contract` record: the index (one line per command, `GROUPS` order) followed by each `###` group's commands as `#### <path>` man-page-shaped sections.
- Dry-run by default (says whether the file would change); `--apply` writes.
- `docs verify` checks the result stays current.
**Properties**
- Changes files: yes
- Idempotent: yes
- Atomic: Yes - the whole region is rewritten in one file write
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - `tools/CONTRACT.md` is missing
**See also**
- `tools/README.md` § Adding a command - how a command gets its record
- `wikitool docs reference` - the same records, rendered for a person under `docs/reference/`
- `wikitool docs verify` - checks the region is current
## `docs reference`
Regenerate the human reference under `docs/reference/`.
**Synopsis**
- `wikitool docs reference [--apply]`
**Options**
```text
--apply Write changes; default is dry-run (preview only)
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool docs reference
tools/wikitool docs reference --apply
```
**Behaviour**
- Rewrites the generated region of every page of the human command reference from the `cli_contract` records: one `docs/reference/commands/<group>.md` per `cli_contract.GROUPS` entry (`<!-- wikitool:reference-commands -->`), the overview in `docs/reference/commands/README.md` (`<!-- wikitool:reference-commands-index -->`), and the gates and exit-42 causes in `docs/reference/exit-codes.md` (`<!-- wikitool:reference-exit-codes -->`).
- Also, in the origin repo only, the skill catalogue `docs/reference/skills.md` (`<!-- wikitool:reference-skills -->`: one row per shipped `instructions/<name>/SKILL.md`, its description split at the sentence starting "Use when") and the page types `docs/reference/page-types.md` (`<!-- wikitool:reference-page-types -->`: one section per `root: kb` type-spec, from what `types describe` reads). Both describe the stack's shipped defaults. In an instance - a tree with `.wikitool-release.json` - they are left alone and the output says so: an instance's adopted type-specs and own instructions may differ, and rewriting a shipped page there would block the next `dist upgrade`.
- Also the shapes of the four hand-written configuration files in `docs/reference/configuration.md`, one region each (`<!-- wikitool:reference-config-telemetry -->`, `-remotes`, `-tasks`, `-upload`) inside the hand-written section on that file, rendered from the records in `chemenu.config_shapes` the loaders read the files with. Those records ship with `tools/`, so these regions are written and checked in an instance as well. Several regions in one page are refreshed one after the other into the same text.
- Renders each command's summary, synopsis, options, examples, notes, properties in words, exit codes with their causes and see-also - but not the ON FAILURE reactions or NEVER, which address an agent; `tools/CONTRACT.md` keeps those.
- Refreshes the table of contents of each page it writes in the same step.
- Never creates a page or places a region: a page's title and one-sentence head are written by hand. A group's page is named after the group - lower case, every run of other characters one hyphen.
- Dry-run by default (lists the pages that would change); `--apply` writes.
- `docs verify` checks the result stays current.
**Properties**
- Changes files: yes
- Idempotent: yes
- Atomic: `--apply` rewrites each page in place, one at a time and idempotently, so a re-run after an interruption converges; the dry-run form is read-only
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - A reference page is missing, or lacks its region's marker pair
**See also**
- `wikitool docs contract` - the same records, rendered for an agent into `tools/CONTRACT.md`
- `wikitool docs verify` - checks every reference region is current
- `tools/README.md` § Adding a command - how a command gets its record
<!-- /wikitool:reference-commands -->
Reference-Commands-Workshop-runs-and-session-budget.md
+218
@@ -0,0 +1,218 @@
*Generated from [`docs/reference/commands/workshop-runs-and-session-budget.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/docs/reference/commands/workshop-runs-and-session-budget.md) at v9.0.0 – edit the repository, not this wiki.*
# Workshop runs and session budget
Opening and closing a workshop run under `work/`, and inspecting the per-session call budget.
<!-- wikitool:toc -->
## Contents
- [`work new`](#work-new)
- [`work close`](#work-close)
- [`budget status`](#budget-status)
- [`budget reset`](#budget-reset)
<!-- /wikitool:toc -->
<!-- wikitool:reference-commands -->
## `work new`
Scaffold `work/<runkey>/` for one workshop run.
**Synopsis**
- `wikitool work new (--input <raw path> | --key <run key>) [--again] [--dry-run]`
**Options**
```text
--input <str> Path to the raw source tree or file this run covers, e.g.
raw/documents/handbook
--key <str> Explicit run key for a run with no raw input (a migration, a
sweep across kb/). Mutually exclusive with --input; may not
start with 'ingest-'.
--again This is a deliberate re-ingest of a tree already processed
before: append today's date to the run key instead of refusing
the collision.
--dry-run Show what would be created, write nothing
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool work new --input raw/2026/09/handbuch
tools/wikitool work new --key migrate-7.0.0
tools/wikitool work new --input raw/2026/09/handbuch --again
```
**Behaviour**
- Creates `work/<runkey>/` with the required `README.md` and `plan.md`.
- `--input` derives the run key from the path below `raw/` (an ingest); `--key` names it outright for a run with no raw input - a migration or a sweep across `kb/` - and may not start with `ingest-`, which stays reserved for derived keys. Exactly one of the two.
- Refuses a collision instead of suffixing it.
- `--again` opens a dated second pass (`<runkey>-<date>`) over a tree that has itself changed.
- `--dry-run` reports the run key and files without writing.
**Properties**
- Changes files: yes
- Idempotent: no
- Atomic: Yes - one directory with two files
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - Neither or both of `--input`/`--key` given, or a `--key` that is empty or starts with `ingest-`
- `1` - `--input` is outside `raw/`, does not exist, or is `raw/` itself (an empty run key)
- `1` - The workshop already exists
**See also**
- `work/CONTRACT.md` - run keys, required files, how a run closes
- `wikitool work close` - deletes the run when it is done
- `instructions/ingest-large-tree.md` - the ingest that opens a run
## `work close`
Delete a finished workshop.
**Synopsis**
- `wikitool work close --run-key <name> [--yes] [--dry-run]`
**Options**
```text
--run-key <str> The workshop directory name [required]
-y, --yes Confirm deletion. Required: closing discards the only copy of
the run's working notes, so the durable conclusions must
already be in kb/.
--dry-run Show what would be deleted, delete nothing
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool work close --run-key ingest-2026-09-handbuch --dry-run
tools/wikitool work close --run-key ingest-2026-09-handbuch --yes
```
**Behaviour**
- Deletes `work/<run-key>/` recursively.
- Without `--yes` it lists what would be lost and refuses: nothing in a workshop is recoverable from the rest of the repo, so the durable conclusions must already be in `kb/`.
- `--dry-run` lists the files without deleting.
- Not idempotent: once deleted, the run key is unknown.
**Properties**
- Changes files: yes
- Idempotent: no
- Atomic: No - a recursive delete
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - Unknown run key
- `1` - `--yes` was not passed - the output lists what would be lost
**See also**
- `work/CONTRACT.md` - how a run closes
- `wikitool work new` - opens a run
## `budget status`
Show the current session's `wikitool` call count and recent command history.
**Synopsis**
- `wikitool budget status`
**Options**
```text
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool budget status
```
**Behaviour**
- Shows the current session's `wikitool` call count and its recent command history.
- Never counted against the budget; never fails; safe to retry freely.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: no
- Can reach the network: no
**Exit codes**
- `0` - success
**See also**
- `instructions/session-setup.md` - scoping the budget to a task
- `instructions/gates.md` - what to do when the Iteration Budget Gate refuses
## `budget reset`
Clear the current session's (or every session's) iteration budget state.
**Synopsis**
- `wikitool budget reset --yes [--all]`
**Options**
```text
--all Clear every session's budget, not just the current one.
-y, --yes Confirm clearing the budget. Only pass this after a human has
approved continuing the session - never set it automatically to
work around the gate.
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool budget reset --yes # only after the user approved it
```
**Behaviour**
- Clears the current session's iteration budget state; `--all` deletes every session's.
- Requires `--yes`: clearing the counter is itself a way around the Iteration Budget Gate, so it needs the same explicit human approval.
- Counted against the budget like any other call.
**Properties**
- Changes files: yes
- Idempotent: no
- Atomic: Read/rewrite of one JSON file (or its deletion, with `--all`)
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - `--yes` not passed
**See also**
- `wikitool budget status` - the current count
- `instructions/gates.md` - why `budget reset` is not the escape hatch
<!-- /wikitool:reference-commands -->
Reference-Commands.md
+167
@@ -0,0 +1,167 @@
*Generated from [`docs/reference/commands/README.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/docs/reference/commands/README.md) at v9.0.0 – edit the repository, not this wiki.*
# Command reference
Every `wikitool` command, one page per group, generated from the same records `tools/wikitool
<command> -h` prints. Each command's section gives its synopsis, options, examples, behaviour,
properties and exit codes; what a command exits 42 for, and which gate it belongs to, is
collected in [exit codes and gates](Reference-Exit-codes).
<!-- wikitool:toc -->
## Contents
- [Pages](#pages)
- [Links and citations](#links-and-citations)
- [Catalog and log](#catalog-and-log)
- [Finding and checking](#finding-and-checking)
- [Provenance](#provenance)
- [Raw material and uploads](#raw-material-and-uploads)
- [Git](#git)
- [Guideline export](#guideline-export)
- [Workshop runs and session budget](#workshop-runs-and-session-budget)
- [Types, instructions and docs](#types-instructions-and-docs)
- [Telemetry](#telemetry)
- [Distribution and versioning](#distribution-and-versioning)
- [Content migrations](#content-migrations)
- [Instance health](#instance-health)
<!-- /wikitool:toc -->
<!-- wikitool:reference-commands-index -->
## Pages
Full records: [pages.md](Reference-Commands-Pages)
- [`new`](Reference-Commands-Pages#new) - Scaffold a new wiki page of any type.
- [`task new`](Reference-Commands-Pages#task-new) - Create one open item in the configured task tracker - never a kb/ page.
- [`task list`](Reference-Commands-Pages#task-list) - List a project's open items - id, title, and whether each carries the WAITING status.
- [`task close`](Reference-Commands-Pages#task-close) - Mark one tracker item done - never delete it.
- [`touch`](Reference-Commands-Pages#touch) - Bump a page's `modified:` date and optionally rewrite its other frontmatter fields.
- [`rename`](Reference-Commands-Pages#rename) - Rename a page, or repoint references that name a page that never existed.
- [`rm`](Reference-Commands-Pages#rm) - Delete a page and mechanically de-link it from the rest of the wiki.
- [`move`](Reference-Commands-Pages#move) - Move a page (or every misplaced page) to the directory its type-spec computes.
## Links and citations
Full records: [links-and-citations.md](Reference-Commands-Links-and-citations)
- [`xref add`](Reference-Commands-Links-and-citations#xref-add) - Declare that A <rel> B.
- [`xref remove`](Reference-Commands-Links-and-citations#xref-remove) - Remove a cross-reference: the inverse of `xref add`.
- [`xref link-source`](Reference-Commands-Links-and-citations#xref-link-source) - Batch-link a source page to every entity/concept it mentions.
- [`links show`](Reference-Commands-Links-and-citations#links-show) - Show the edges out of and into a page.
- [`cite id`](Reference-Commands-Links-and-citations#cite-id) - Print the deterministic footnote id `cite add` would use for this (title, file) pair.
- [`cite add`](Reference-Commands-Links-and-citations#cite-add) - Upsert a `[^cite-id]: [[Source - X]]` definition in a page's footnotes region.
- [`cite sync`](Reference-Commands-Links-and-citations#cite-sync) - Reconcile each page's footnotes region against its actual `[^id]` references.
## Catalog and log
Full records: [catalog-and-log.md](Reference-Commands-Catalog-and-log)
- [`index rebuild`](Reference-Commands-Catalog-and-log#index-rebuild) - Regenerate the catalog from every page's frontmatter.
- [`log append`](Reference-Commands-Catalog-and-log#log-append) - Append a formatted entry to `kb/log.md`.
- [`log status`](Reference-Commands-Catalog-and-log#log-status) - Read-only: count `ingest` entries logged since the last `lint` entry.
## Finding and checking
Full records: [finding-and-checking.md](Reference-Commands-Finding-and-checking)
- [`lint`](Reference-Commands-Finding-and-checking#lint) - Run structural lint checks against kb/.
- [`search`](Reference-Commands-Finding-and-checking#search) - Find pages in `kb/` by text and/or frontmatter.
- [`review`](Reference-Commands-Finding-and-checking#review) - The GTD weekly review.
## Provenance
Full records: [provenance.md](Reference-Commands-Provenance)
- [`sources coverage`](Reference-Commands-Provenance#sources-coverage) - List raw files with no source page, broken `raw_files:` references, and legacy directory/URL-only source pages.
- [`sources trace`](Reference-Commands-Provenance#sources-trace) - Trace provenance in either direction: raw file, or page.
- [`sources rebuild-index`](Reference-Commands-Provenance#sources-rebuild-index) - Regenerate the `kb/provenance.md` reverse index.
## Raw material and uploads
Full records: [raw-material-and-uploads.md](Reference-Commands-Raw-material-and-uploads)
- [`raw fetch`](Reference-Commands-Raw-material-and-uploads#raw-fetch) - Capture a web page the user names into `incoming/`: the HTML as received plus a derived text, for `raw accept` to promote.
- [`raw derive`](Reference-Commands-Raw-material-and-uploads#raw-derive) - Derive reading text from a PDF, e-mail, Office file or web page in `incoming/`, or unpack a ZIP, and leave the original beside it, for `raw accept` to promote.
- [`raw capture`](Reference-Commands-Raw-material-and-uploads#raw-capture) - Capture documentation from a git repository into `incoming/<bundle>/`, with a manifest naming the repository, ref rule and commit, for `raw accept` to promote.
- [`raw status`](Reference-Commands-Raw-material-and-uploads#raw-status) - Report which captured bundles under `raw/` have fallen behind their repository, with each changed file as `A`/`M`/`D`.
- [`raw pending`](Reference-Commands-Raw-material-and-uploads#raw-pending) - List what waits in `incoming/`, oldest first, and name the entry an ingest without an argument takes next.
- [`raw accept`](Reference-Commands-Raw-material-and-uploads#raw-accept) - Promote one or more files, or one folder, from `incoming/` into `raw/`.
- [`upload list`](Reference-Commands-Raw-material-and-uploads#upload-list) - List every MCP submission currently waiting in the quarantine (`mcp-upload/`).
- [`upload show`](Reference-Commands-Raw-material-and-uploads#upload-show) - Print one submission's manifest in full.
- [`upload accept`](Reference-Commands-Raw-material-and-uploads#upload-accept) - **Upload Review Gate:** promote a submission's file from quarantine into `incoming/`.
- [`upload reject`](Reference-Commands-Raw-material-and-uploads#upload-reject) - Delete a submission's material, keeping only its ledger trail.
## Git
Full records: [git.md](Reference-Commands-Git)
- [`sync`](Reference-Commands-Git#sync) - Fetch `<remote>/<branch>` and bring the local branch up to date with it.
- [`publish`](Reference-Commands-Git#publish) - Reconcile with `<remote>/<branch>`, then stage all changes, commit, and push.
## Guideline export
Full records: [guideline-export.md](Reference-Commands-Guideline-export)
- [`export guidelines`](Reference-Commands-Guideline-export#export-guidelines) - Render the guideline pages into one generated `GUIDELINES.md`, or push it into the captured repositories that opted in (**Guideline Push Gate**).
## Workshop runs and session budget
Full records: [workshop-runs-and-session-budget.md](Reference-Commands-Workshop-runs-and-session-budget)
- [`work new`](Reference-Commands-Workshop-runs-and-session-budget#work-new) - Scaffold `work/<runkey>/` for one workshop run.
- [`work close`](Reference-Commands-Workshop-runs-and-session-budget#work-close) - Delete a finished workshop.
- [`budget status`](Reference-Commands-Workshop-runs-and-session-budget#budget-status) - Show the current session's `wikitool` call count and recent command history.
- [`budget reset`](Reference-Commands-Workshop-runs-and-session-budget#budget-reset) - Clear the current session's (or every session's) iteration budget state.
## Types, instructions and docs
Full records: [types-instructions-and-docs.md](Reference-Commands-Types-instructions-and-docs)
- [`types list`](Reference-Commands-Types-instructions-and-docs#types-list) - List every type-spec under `types/`.
- [`types describe`](Reference-Commands-Types-instructions-and-docs#types-describe) - Print one type's full contract.
- [`instructions sync`](Reference-Commands-Types-instructions-and-docs#instructions-sync) - Publish every `instructions/<name>/SKILL.md` into the harness skill directories.
- [`instructions verify`](Reference-Commands-Types-instructions-and-docs#instructions-verify) - Check the instruction layer.
- [`instructions list`](Reference-Commands-Types-instructions-and-docs#instructions-list) - List the flat instructions with their descriptions.
- [`docs verify`](Reference-Commands-Types-instructions-and-docs#docs-verify) - Check the docs that mirror the code.
- [`docs toc`](Reference-Commands-Types-instructions-and-docs#docs-toc) - Create, refresh or remove the generated table-of-contents region.
- [`docs prerequisites`](Reference-Commands-Types-instructions-and-docs#docs-prerequisites) - Regenerate `INSTALL.md`'s prerequisites lists from `tools/prerequisites.txt`.
- [`docs contract`](Reference-Commands-Types-instructions-and-docs#docs-contract) - Regenerate `tools/CONTRACT.md`'s `<!-- wikitool:commands -->` region.
- [`docs reference`](Reference-Commands-Types-instructions-and-docs#docs-reference) - Regenerate the human reference under `docs/reference/`.
## Telemetry
Full records: [telemetry.md](Reference-Commands-Telemetry)
- [`eval sessions`](Reference-Commands-Telemetry#eval-sessions) - List the sessions that have a trace under `reports/telemetry/`.
- [`eval score`](Reference-Commands-Telemetry#eval-score) - Score one traced session.
## Distribution and versioning
Full records: [distribution-and-versioning.md](Reference-Commands-Distribution-and-versioning)
- [`dist export`](Reference-Commands-Distribution-and-versioning#dist-export) - Write a contentless, distributable copy of this repo's machinery.
- [`dist adopt`](Reference-Commands-Distribution-and-versioning#dist-adopt) - Take shipped templates as this instance's own: copy each to its unsuffixed name.
- [`dist upgrade`](Reference-Commands-Distribution-and-versioning#dist-upgrade) - Apply a stack update `dist export` produced - the write half of `version check`.
- [`version show`](Reference-Commands-Distribution-and-versioning#version-show) - Print this instance's stack version and where it came from.
- [`version check`](Reference-Commands-Distribution-and-versioning#version-check) - Ask the origin's release feed whether a newer stack exists.
- [`version notes`](Reference-Commands-Distribution-and-versioning#version-notes) - Print one version's release notes.
- [`version bump`](Reference-Commands-Distribution-and-versioning#version-bump) - Raise or continue the one running candidate between two releases.
- [`version regrade`](Reference-Commands-Distribution-and-versioning#version-regrade) - List the running candidate's bump titles with their impact grade, or change one or more of them.
- [`version release`](Reference-Commands-Distribution-and-versioning#version-release) - Fix the running candidate: strip `VERSION`'s `-beta.N` suffix and close its `CHANGES.md` entry.
## Content migrations
Full records: [content-migrations.md](Reference-Commands-Content-migrations)
- [`migrate list`](Reference-Commands-Content-migrations#migrate-list) - List every migration document under `instructions/migrations/`.
- [`migrate status`](Reference-Commands-Content-migrations#migrate-status) - Show the migrations this instance still owes, in the order they must run.
- [`migrate verify`](Reference-Commands-Content-migrations#migrate-verify) - Compare `kb/` against a git revision on the invariants a content migration must not change.
- [`migrate done`](Reference-Commands-Content-migrations#migrate-done) - Record one migration as applied, advancing `kb_version` in `.wikitool-kb.json`.
- [`migrate baseline`](Reference-Commands-Content-migrations#migrate-baseline) - Declare `kb_version` once, for an instance predating `.wikitool-kb.json`.
## Instance health
Full records: [instance-health.md](Reference-Commands-Instance-health)
- [`doctor`](Reference-Commands-Instance-health#doctor) - Check that this instance is correctly configured.
<!-- /wikitool:reference-commands-index -->
Reference-Configuration.md
+334
@@ -0,0 +1,334 @@
*Generated from [`docs/reference/configuration.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/docs/reference/configuration.md) at v9.0.0 – edit the repository, not this wiki.*
# Configuration
The environment variables `wikitool` reads, and the per-checkout files an operator may create at the
root of an instance. None of them is needed for an instance to work: an instance installed from a
release runs against the origin's public release feed with no configuration at all.
The shape of each file - its fields, what an absent or a malformed file means, an example - is
generated from the record `wikitool` itself reads the file with, so it cannot describe a field the
tool does not take. What a setting does around it is written by hand.
Three more files at the root are written by `wikitool` and never by hand: `.wikitool-release.json`,
`.wikitool-kb.json` and `.wikitool-tools.json` ([AGENTS.md invariant 1](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/AGENTS.md#invariants)).
The last one has its own section [below](#tool-paths), because the preflight that writes it is
something you run.
<!-- wikitool:toc -->
## Contents
- [Environment variables](#environment-variables)
- [Telemetry](#telemetry)
- [Tool paths](#tool-paths)
- [Allowed push URLs](#allowed-push-urls)
- [Task tracker](#task-tracker)
- [Super Productivity](#super-productivity)
- [CalDAV](#caldav)
- [MCP submissions](#mcp-submissions)
<!-- /wikitool:toc -->
## Environment variables
| Variable | Purpose | Fallback |
|----------|---------|----------|
| `WIKI_AUTHOR` | Overrides the author name of new source pages | `git config user.name`. Without either, `new` stops with an `ERROR` |
| `WIKITOOL_SESSION_ID` | Scopes the Iteration Budget Gate to a task rather than a terminal, and names the directory a trace lands in | A session variable the harness sets itself, where one is known (for example `CLAUDE_CODE_SESSION_ID`), else the parent process id ([instructions/session-setup.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/instructions/session-setup.md)) |
| `WIKITOOL_UPDATE_URL` | The release feed `version check`, `version notes` and `dist upgrade --latest` ask | The value in `.wikitool-release.json`, else the origin's feed |
| `WIKITOOL_UPDATE_TOKEN` | A Gitea token for the release feed, sent only to downloads on the feed's own host | None. Not needed against the origin, only against a feed in a private repository |
| `WIKITOOL_TASKS_CONFIG` | Path to a tracker configuration that `task`, `review` and `doctor` read instead of `.wikitool-tasks.json`, for running one checkout against several trackers in turn | `.wikitool-tasks.json` at the root. A path to a file that does not exist is an error, never "no tracker configured" |
| `CHEMENU_ROOT` | Which corpus the package works on, for a caller outside the checkout itself | The checkout the package sits in. `tools/wikitool` behaves the same without it |
| `WIKI_TRACE` | `0`/`1` turns telemetry off or on, beating both the default and `.wikitool-telemetry.json` | The installation-form default ([below](#telemetry)) |
| `WIKI_TRACE_DIR` | Writes traces somewhere other than `reports/telemetry/` | `reports/telemetry/` |
| `WIKI_TRACE_CONTENT` | `0` keeps only lengths and digests of prompts and replies instead of their text | Text is kept |
| `WIKI_TRACE_MAX_CONTENT` | Per-attribute content cap in characters | 60 KiB |
| `WIKI_TRACE_MAX_SESSION_BYTES` | Byte cap per session trace; beats the file's `max_session_bytes` | 5 MiB |
| `WIKI_TRACE_KEEP_SESSIONS` | How many session directories retention keeps; beats the file's `keep_sessions` | 250 |
## Telemetry
**Telemetry is off in an instance.** Every instance carries the `.wikitool-release.json` of its
release, and that is how `chemenu.telemetry.policy` recognises that nobody asked for telemetry. The
same holds for every further clone of the instance, because the file is committed with the first
publish. Only a clone of the origin repository, used to develop the stack, has no stamp. There the
traces are the instrument the stack measures itself with, and they are **on**.
To turn the default around, create `.wikitool-telemetry.json` at the root. It is per checkout and
gitignored, with no `.template`, like `.wikitool-remotes.json`.
<!-- wikitool:reference-config-telemetry -->
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `schema` | any | no | - | Accepted and not read. Conventionally `1`; no version of this file is told apart by it yet. |
| `enabled` | boolean | no | the installation-form default | Telemetry on or off. `WIKI_TRACE` beats it in both directions. Any other value is read by its truthiness, so `"no"` means on. |
| `max_session_bytes` | positive integer | no | `5242880` | Byte cap per session trace. `WIKI_TRACE_MAX_SESSION_BYTES` beats it. A number written as a string is accepted. |
| `keep_sessions` | positive integer | no | `250` | How many session directories retention keeps. `WIKI_TRACE_KEEP_SESSIONS` beats it. A number written as a string is accepted. |
- **Absent:** The installation-form default applies: on in a clone of the origin repository, off in an instance.
- **Malformed:** Never an error. Unreadable JSON or a non-object counts as an absent file, and a cap that is not a positive integer as an absent cap. `doctor` reports each such deviation as a `WARN` under `telemetry`, naming the field.
- **Unknown keys:** Ignored; `doctor` reports them as a `WARN`.
Example:
```json
{
"enabled": true,
"max_session_bytes": 5242880,
"keep_sessions": 250
}
```
<!-- /wikitool:reference-config-telemetry -->
`wikitool doctor` reports the current state: on or off, why, and how much has been recorded against
both caps. What the caps do and what never reaches a trace:
[telemetry-and-evaluation.md](Explanation-Telemetry-and-evaluation).
## Tool paths
**The preflight writes this file, not you.** `.wikitool-tools.json` at the root holds the absolute
paths of Python, git and ripgrep, as the preflight found them on this machine. `wikitool` starts git
and rg from there instead of from `PATH`. It is per checkout and gitignored, because a path on one
machine says nothing about the next.
If the file is missing or incomplete, `tools/wikitool` does not start (exit 42) and names the
preflight. If a tool sits somewhere the preflight does not look, name it:
`tools/preflight.sh --set rg=<path>`, or in PowerShell
`pwsh -NoProfile -ExecutionPolicy Bypass -File tools/preflight.ps1 --set rg=<path>`. Never edit the
file by hand. `doctor` reports under `tool-paths` whether every path still works.
## Allowed push URLs
`.wikitool-remotes.json` at the root restricts where `publish` may push, per checkout and
gitignored.
<!-- wikitool:reference-config-remotes -->
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `schema` | any | no | - | Accepted and not read. Conventionally `1`; no version of this file is told apart by it yet. |
| `allowed_push_urls` | array of strings | yes | - | The push URLs `publish` may write to, compared with the URL the remote resolves to for a push - not with its name. An empty list allows no push at all. |
- **Absent:** Any remote is allowed; the Publish-Remote Gate does not apply.
- **Malformed:** `publish` stops with an error before committing anything. A broken file is never read as "no restriction".
- **Unknown keys:** Ignored.
Example:
```json
{
"schema": 1,
"allowed_push_urls": [
"<url>"
]
}
```
<!-- /wikitool:reference-config-remotes -->
With the file, `publish` refuses with exit 42 (the Publish-Remote Gate) before pushing to a URL not
on the list. Only you edit this file. The gate has no token and no flag, and an agent that edits the
list to get past a refusal is opening a gate on its own initiative.
## Task tracker
**Optional.** The weekly review (`tools/wikitool review`, skill `gtd-weekly-review`) checks the
project pages under `kb/gtd/` against a task tracker. Which tracker that is goes in
`.wikitool-tasks.json` at the root. Like the telemetry and remotes files, it is per checkout and has
no `.template`, and it is **gitignored as soon as a token is in it**.
<!-- wikitool:reference-config-tasks -->
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `schema` | any | no | - | Accepted and not read. Conventionally `1`; no version of this file is told apart by it yet. |
| `provider` | string | yes | - | Which adapter the review goes through. The provider's own settings sit in a section of the same name. |
| `thresholds` | object | yes | - | The weekly review's three thresholds. |
| `thresholds.stalled_waiting_days` | positive integer | yes | - | A waiting-for item older than this many days counts as overdue. A number written as a string is accepted. |
| `thresholds.unpaged_project_weeks` | positive integer | yes | - | A tracker project with no `kb/` page is reported once it is older than this many weeks. A number written as a string is accepted. |
| `thresholds.someday_stale_months` | positive integer | yes | - | A someday/maybe item untouched for this many months counts as stale. A number written as a string is accepted. |
- **Absent:** No tracker is configured - a valid state. The weekly review has nothing to check against.
- **Malformed:** An error, and `doctor` reports `FAIL`. A broken file is never read as "no tracker configured".
- **Unknown keys:** Ignored, so the section of a provider not chosen right now can stay in the file.
**The `superproductivity` section.** Connection settings for the Super Productivity adapter.
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `access` | string | yes | - | Which way in: `api`, the local REST API (reads the current state and carries the write path), or `snapshot`, the newest backup snapshot (read-only). |
With `access: "api"`:
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `api_base_url` | string | no | `"http://127.0.0.1:3876"` | Where the app's REST API listens. An empty value means the default. |
| `api_token` | string | yes, not empty | - | The token from the app's settings. |
With `access: "snapshot"`:
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `backups_dir` | string | yes, not empty | - | The directory the app writes its backup snapshots to; `~` is expanded. |
- **Unknown keys:** Rejected: a section names only one access path's own fields, so a `backups_dir` beside `access: "api"` is an error, not ignored.
```json
{
"superproductivity": {
"access": "api",
"api_base_url": "http://127.0.0.1:3876",
"api_token": "..."
}
}
```
```json
{
"superproductivity": {
"access": "snapshot",
"backups_dir": "~/.config/superProductivity/backups"
}
}
```
**The `caldav` section.** Connection settings for the CalDAV adapter.
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `url` | string | yes, not empty | - | The account's CalDAV calendar home set. A trailing `/` is added when missing. |
| `username` | string | yes, not empty | - | The login name, which can differ from the user id in the URL. |
| `app_password` | string | yes, not empty | - | An app password is recommended over the account password. |
| `inbox_list` | string | yes, not empty | - | The name of the one inbox list. |
| `someday_list` | string | yes, not empty | - | The name of the one someday/maybe list. |
| `exclude_lists` | array of strings | no | `[]` | Existing plain task lists that are not projects. |
- **Unknown keys:** Rejected.
```json
{
"caldav": {
"url": "https://<host>/remote.php/dav/calendars/<user>/",
"username": "<login>",
"app_password": "...",
"inbox_list": "Inbox",
"someday_list": "Someday",
"exclude_lists": [
"..."
]
}
}
```
A complete example:
```json
{
"schema": 1,
"provider": "superproductivity",
"thresholds": {
"stalled_waiting_days": 14,
"unpaged_project_weeks": 3,
"someday_stale_months": 5
},
"superproductivity": {
"access": "api",
"api_base_url": "http://127.0.0.1:3876",
"api_token": "..."
}
}
```
<!-- /wikitool:reference-config-tasks -->
The `thresholds` block is configuration, not schema. A value the provider cannot supply at all - a
waiting item with no follow-up date, a tracker project with no creation date - is not skipped silently
but reported as its own finding (`waiting_no_follow_up`/`project_age_unknown`).
### Super Productivity
`access` is **mandatory and has no fallback**. It chooses between two mutually exclusive ways in:
- **`access: "snapshot"`** for an instance run headless: it reads only the newest backup snapshot
under `backups_dir`. That works without the app running, but only reads. The tracker cannot be
written from there.
- **`access: "api"`** for a desktop instance: it talks only to the local REST API. That answers only
while the app runs, but it gives the current state and carries the write path. `api_token` is
required, because every endpoint except `GET /health` demands `Authorization: Bearer <token>`.
What can write:
- `tools/wikitool new project` creates a tracker project of the same name only with `access: "api"`,
and even there not automatically. The command stops (exit 42) and asks a human to create it, then
continues with `--resume`.
- `tools/wikitool task new` creates a single item without a `kb/` page.
- `tools/wikitool task close --id` marks an item done. It never deletes one.
With `access: "snapshot"`, all three refuse completely with exit 1, because the tracker is read-only
from there. `tools/wikitool task list --project` only reads, so it answers with either `access`.
### CalDAV
`caldav` is the standards-based second adapter (RFC 4791/5545). It is verified against Nextcloud
Tasks, with iOS *Reminders* as the mobile client, and built for that split: tick items off on the
phone, maintain them at the desk. Unlike Super Productivity, there is only one way in. CalDAV is
always a network access, so no `access` field is needed.
- `url` may be an alias - Nextcloud accepts a short name there - because every later address comes
only from the `href`s the server returns, never from this URL joined with a name.
- `inbox_list` and `someday_list` name the two fixed lists, exactly one of each per instance.
- A list with a `VEVENT` component never counts as a project, so `exclude_lists` is needed only for
plain task lists.
A project is a list whose only supported component is `VTODO`. `tools/wikitool new project` creates
it automatically with `MKCALENDAR` - without asking, unlike Super Productivity, because CalDAV has a
real create command for lists. The uniqueness check runs against **every** list in the account,
including excluded lists, the inbox, someday and mixed calendars. If a new name collides with any of
them, nothing is created and the collision is named. Renaming the existing list in Nextcloud stays a
manual step.
`task new` and `task close` work without restriction on `caldav`, which has no read-only mode like
`access: "snapshot"`. When ticking an item off, `task close` changes only `STATUS`, `COMPLETED`,
`PERCENT-COMPLETE`, `LAST-MODIFIED` and `DTSTAMP` on the existing `.ics` resource. Every other
property, including an unknown `X-` property or an alarm, is kept unchanged. A resource that changed
since it was read (an ETag conflict) is not written, and the command stops with exit 1. Nothing is
ever deleted. The lists of finished projects stay after archiving, because the adapter never deletes
a list. Remove them by hand in Nextcloud when you want to.
## MCP submissions
`.wikitool-upload.json` in the served corpus turns on the MCP read server's optional `submit` tool.
What the authentication in front of the server has to guarantee before you create it is in
[Set up the MCP read server](How-to-Set-up-the-mcp-server#optional-the-submit-path).
<!-- wikitool:reference-config-upload -->
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `schema` | any | no | - | Accepted and not read. Conventionally `1`; no version of this file is told apart by it yet. |
| `identity_header` | string | no | `"X-Forwarded-User"` | The request header the authenticating middleware sets to the submitter's identity on every request. An empty value means the default. |
| `max_bytes` | positive integer | yes | - | Largest single submission, in bytes. A number written as a string is accepted. |
| `allowed_extensions` | array of strings | yes | - | The file extensions `submit` accepts, each starting with `.`. Compared case-insensitively. Entries are read as strings and lower-cased; duplicates collapse. |
| `quota` | object | yes | - | Per-submitter limits over any rolling 24 hours. |
| `quota.submissions_per_day` | positive integer | yes | - | Submissions one submitter may make. A number written as a string is accepted. |
| `quota.bytes_per_day` | positive integer | yes | - | Bytes one submitter may submit. A number written as a string is accepted. |
- **Absent:** The `submit` tool is not registered: the write path does not exist.
- **Malformed:** An error. The MCP server does not start and `doctor` reports `FAIL`. A broken file is never read as "no limits".
- **Unknown keys:** Ignored.
Example:
```json
{
"schema": 1,
"identity_header": "X-Forwarded-User",
"max_bytes": 10485760,
"allowed_extensions": [
".md",
".pdf"
],
"quota": {
"submissions_per_day": 20,
"bytes_per_day": 52428800
}
}
```
<!-- /wikitool:reference-config-upload -->
Size and quota are your deployment's decision, not defaults worth copying blindly.
Reference-Exit-codes.md
+43
@@ -0,0 +1,43 @@
*Generated from [`docs/reference/exit-codes.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/docs/reference/exit-codes.md) at v9.0.0 – edit the repository, not this wiki.*
# Exit codes and gates
Which `wikitool` commands can stop and wait for a person - exit 42 - and for which gate, generated
from the command records.
<!-- wikitool:reference-exit-codes -->
What each exit code means, and how a caller reacts to it, is [AGENTS.md § Tool error contract](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/AGENTS.md#tool-error-contract). What a gate is and why it is enforced in code is [AGENTS.md § Gates](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/AGENTS.md#gates). This page lists which commands declare a gate, and every cause for which a command exits 42.
## Gates by command
### guideline-push
- [`export guidelines`](Reference-Commands-Guideline-export#export-guidelines)
### mass-update
- [`publish`](Reference-Commands-Git#publish)
### publish-remote
- [`publish`](Reference-Commands-Git#publish)
### rebase-review
- [`sync`](Reference-Commands-Git#sync)
- [`publish`](Reference-Commands-Git#publish)
### upload-review
- [`upload accept`](Reference-Commands-Raw-material-and-uploads#upload-accept)
## Exit 42 causes
- [`new`](Reference-Commands-Pages#new): new project: The provider cannot create the project itself (Super Productivity's `access: "api"`); the output says what a human has to create
- [`upload accept`](Reference-Commands-Raw-material-and-uploads#upload-accept): Upload Review Gate: `--confirm` is absent or does not match the manifest's current token
- [`sync`](Reference-Commands-Git#sync): Rebase-Review Gate: `<remote>/<branch>` moved and both sides changed the same file, generated files aside; the output lists the upstream commits, the overlapping files and their diff
- [`publish`](Reference-Commands-Git#publish): Mass-Update Gate: `--threshold` (default 10) or more counted files would be committed, or the `--confirm` token does not match this changeset
- [`publish`](Reference-Commands-Git#publish): Rebase-Review Gate: `<remote>/<branch>` moved and both sides changed the same file; the output lists the upstream commits, the overlapping files and their diff
- [`publish`](Reference-Commands-Git#publish): Publish-Remote Gate: `.wikitool-remotes.json` exists and the push URL of `--remote` is not listed in it, or `--remote` resolves to no push URL
- [`export guidelines`](Reference-Commands-Guideline-export#export-guidelines): Guideline Push Gate: `--push` without `--confirm`, or with a token that does not match the current set of targets
<!-- /wikitool:reference-exit-codes -->
Reference-Page-types.md
+140
@@ -0,0 +1,140 @@
*Generated from [`docs/reference/page-types.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/docs/reference/page-types.md) at v9.0.0 – edit the repository, not this wiki.*
# Page types
Every page type the stack ships for `kb/`: what it is for, where its pages are written, its
subtypes and its required frontmatter fields, generated from the type-specs under `types/` and
their schemas.
This page describes the stack's shipped defaults. An instance adopts these type-specs from their
`.template` and may change them, or add types of its own; `tools/wikitool types list` and
`tools/wikitool types describe <type>` show what this instance has.
<!-- wikitool:toc -->
## Contents
- [`comparison`](#comparison)
- [`concept`](#concept)
- [`entity`](#entity)
- [`project`](#project)
- [`source`](#source)
<!-- /wikitool:toc -->
<!-- wikitool:reference-page-types -->
## `comparison`
Structured type for comparison pages that set several entities or approaches against one another
- Written to: `kb/comparisons/`
**Required fields**
- `type` - string
- `created` - string
- `entities` - array
- `summary` - string
## `concept`
Base type for concept pages - architectures, patterns, protocols, workflows, decisions or problems
- Written to: `kb/concepts/<subtype directory>/`
**Subtypes** (field `concept_type`)
| Value | Written to | Index section |
|---|---|---|
| `architecture` | `kb/concepts/architectures/` | Architekturen |
| `pattern` | `kb/concepts/patterns/` | Muster |
| `protocol` | `kb/concepts/protocols/` | Protokolle |
| `workflow` | `kb/concepts/workflows/` | Abläufe |
| `decision` | `kb/concepts/decisions/` | Entscheidungen |
| `problem` | `kb/concepts/problems/` | Problemstellungen |
**Required fields**
- `type` - string
- `concept_type` - string, one of `architecture`, `pattern`, `protocol`, `workflow`, `decision`, `problem`
- `created` - string
- `modified` - string
- `provenance` - string, one of `sourced`, `general`, `mixed`
- `summary` - string
## `entity`
Base type for entity pages - codebases, systems, tools, technologies, people or organizations
- Written to: `kb/entities/<subtype directory>/`
**Subtypes** (field `entity_type`)
| Value | Written to | Index section |
|---|---|---|
| `codebase` | `kb/entities/codebases/` | Codebasen |
| `system` | `kb/entities/systems/` | Systeme |
| `tool` | `kb/entities/tools/` | Werkzeuge |
| `technology` | `kb/entities/technologies/` | Technologien |
| `person` | `kb/entities/people/` | Personen |
| `organization` | `kb/entities/organizations/` | Organisationen |
**Required fields**
- `type` - string
- `entity_type` - string, one of `codebase`, `system`, `tool`, `technology`, `person`, `organization`
- `created` - string
- `modified` - string
- `provenance` - string, one of `sourced`, `general`, `mixed`
- `summary` - string
## `project`
Structured type for a committed initiative (a GTD project) - goal, participants, durable status, open loops
- Written to: `kb/gtd/<subtype directory>/`
**Subtypes** (field `responsibility`)
| Value | Written to | Index section |
|---|---|---|
| `haus` | `kb/gtd/haus/` | Haus |
| `finanzen` | `kb/gtd/finanzen/` | Finanzen |
| `technik` | `kb/gtd/technik/` | Technik |
**Required fields**
- `type` - string
- `state` - string, one of `active`, `dormant`, `completed`, `abandoned`
- `responsibility` - string, one of `haus`, `finanzen`, `technik`
- `created` - string
- `modified` - string
- `provenance` - string, one of `sourced`, `general`, `mixed`
- `summary` - string
## `source`
Structured type for source pages that record and summarize ingested raw material
- Written to: `kb/sources/<subtype directory>/`
- Title prefix: `Source - `
**Subtypes** (field `source_type`)
| Value | Written to | Index section |
|---|---|---|
| `transcript` | `kb/sources/transcripts/` | Transkripte |
| `analysis` | `kb/sources/analyses/` | Analysen |
| `article` | `kb/sources/articles/` | Artikel |
| `document` | `kb/sources/documents/` | Dokumente |
| `notes` | `kb/sources/notes/` | Notizen |
| `tracker` | `kb/sources/trackers/` | Tracker-Exporte |
| `unclassified` | `kb/sources/unclassified/` | Unklassifiziert |
**Required fields**
- `type` - string
- `source_type` - string, one of `transcript`, `analysis`, `article`, `document`, `notes`, `tracker`, `unclassified`
- `author` - string
- `raw_files` - array
- `date` - string
- `summary` - string
<!-- /wikitool:reference-page-types -->
Reference-Skills.md
+21
@@ -0,0 +1,21 @@
*Generated from [`docs/reference/skills.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/docs/reference/skills.md) at v9.0.0 – edit the repository, not this wiki.*
# Skill catalogue
The skills the stack ships, as an agent's harness sees them once `tools/wikitool instructions sync`
has published them: each one's name, what it does and the requests it answers, generated from its
`instructions/<name>/SKILL.md`. Any skill also runs when you ask for it by name.
This page describes the stack's shipped defaults. An instance may add instructions of its own;
`tools/wikitool instructions list` shows what this instance has.
<!-- wikitool:reference-skills -->
| Skill | What it does | Use it when |
|---|---|---|
| `gtd-weekly-review` | Turns the findings from `wikitool review` into decisions and page updates - the GTD Weekly Review, with a machine that prepares the list instead of a human reconstructing it from memory. | the user asks for "the weekly review", "review my projects", "what's stalled", or after `wikitool review` has findings nobody has acted on yet. |
| `wiki-ingest` | Processes a new source file into the LLM wiki - extracts entities and concepts, creates a source summary page, files a tracker item for any commitment the source also carries, cross-references, rebuilds indexes, and publishes. | the user drops a file or folder into incoming/ or raw/, names a URL to ingest, or says "ingest &lt;file&gt;", "ingest &lt;url&gt;", "process this source", "add this to the wiki" - or just "ingest" with nothing named, which takes the oldest entry waiting in incoming/ - or asks to "update the captured repositories" / "pull the repo docs", which takes the next bundle `raw status` reports as changed. |
| `wiki-lint` | Checks the health of the LLM wiki - broken links, orphan pages, uncovered raw files, stale claims, duplicated rules, missing cross-references. | the user says "lint the wiki", "health-check the wiki", or periodically every 10 sources per the Maintenance Schedule. |
| `wiki-manage` | Creates a new wiki page (entity, concept, source, comparison) or updates an existing page with new information, including cross-references, index/log, and publishing. | the user says "create a new entity/concept/comparison", "add a page for X", "update the X page", or new information needs integrating into an existing page. |
| `wiki-query` | Answers a question using the LLM wiki's compiled knowledge - read-only, cites sources, can file a valuable answer back as a new page. | the user asks a question about entities, projects, concepts, or anything the wiki might know, or says "query the wiki", "what do we know about X", "search the wiki". |
| `wiki-status` | Shows a quick read-only snapshot of the LLM wiki - page counts, orphan pages, uncovered raw files, recent activity. | the user says "wiki status", "show wiki statistics", "what's new", or wants a quick health snapshot without running a full lint. |
<!-- /wikitool:reference-skills -->
Reference.md
+25
@@ -0,0 +1,25 @@
*Generated from [`docs/reference/README.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/docs/reference/README.md) at v9.0.0 – edit the repository, not this wiki.*
# Reference
Lookup pages: the command surface, exit codes and gates, the skill catalogue, the page types. A
reference page describes what is, not how to use it or why it is so - and wherever a source for
it exists in the code, it is generated from that source and checked against it rather than
written by hand.
- [Command reference](Reference-Commands) - every `wikitool` command, one page per group,
generated from the same records as `tools/wikitool <command> -h`.
- [Exit codes and gates](Reference-Exit-codes) - which commands can stop and wait for a person, and
why.
- [Skill catalogue](Reference-Skills) - the skills the stack ships, what each does and which requests
start it, generated from their `SKILL.md`.
- [Page types](Reference-Page-types) - every page type the stack ships for `kb/`, its subtypes, where
its pages are written and its required fields, generated from the type-specs.
- [Configuration](Reference-Configuration) - the environment variables `wikitool` reads and the
per-checkout files an operator may create. The shape of each file is generated from the record
its loader reads it with; the environment variables and what each setting does are written by
hand.
The skill catalogue and the page types describe the stack's shipped defaults, which an instance
may change or extend; `tools/wikitool instructions list` and `tools/wikitool types describe <type>`
show what one instance has.
Tutorials-First-source-to-first-answer.md
+159
@@ -0,0 +1,159 @@
*Generated from [`docs/tutorials/first-source-to-first-answer.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/docs/tutorials/first-source-to-first-answer.md) at v9.0.0 – edit the repository, not this wiki.*
# From your first source to your first answer
In this tutorial you put one document into a fresh instance, let the agent turn it into wiki
pages, look at what it wrote, and then ask the wiki a question and follow the answer back to the
document. It takes about a quarter of an hour. You need nothing but the instance and your harness:
the document is a sample that ships with the instance.
You start from an instance that [INSTALL.md](Install) has just set up, with the agent
session restarted afterwards so that it has loaded its skills. Ask the agent `Which skills do you
have?` if you are unsure - `wiki-ingest` and `wiki-query` should be among them.
<!-- wikitool:toc -->
## Contents
- [1. Put the sample source into `incoming/`](#1-put-the-sample-source-into-incoming)
- [2. Ask the agent to ingest it](#2-ask-the-agent-to-ingest-it)
- [3. Tell the agent what matters](#3-tell-the-agent-what-matters)
- [4. Answer the two capture questions](#4-answer-the-two-capture-questions)
- [5. Let the agent publish](#5-let-the-agent-publish)
- [6. Look at what it wrote](#6-look-at-what-it-wrote)
- [7. Ask a question](#7-ask-a-question)
- [8. Optional: browse it in Obsidian](#8-optional-browse-it-in-obsidian)
- [What next](#what-next)
<!-- /wikitool:toc -->
## 1. Put the sample source into `incoming/`
`incoming/` is where everything you want in the wiki waits until it is ingested. Copy the sample
there - copy, not move, so the tutorial page keeps its sample:
```bash
cp docs/tutorials/sample-source.md incoming/home-lab-backups.md
```
On Windows in PowerShell 7:
```powershell
Copy-Item docs/tutorials/sample-source.md incoming/home-lab-backups.md
```
Open [the sample](Tutorials-Sample-source) and read it once. It is a page of notes about the backups of a
small home file server called Atlas: which tool makes them (restic), where they go, and why there
are two targets (the 3-2-1 backup rule). Knowing what it says will let you check the agent's work.
## 2. Ask the agent to ingest it
In the agent session, say:
```text
Ingest
```
A bare `Ingest` takes the oldest entry waiting in `incoming/` - here, the only one. The agent
reads the whole file before it writes anything.
## 3. Tell the agent what matters
The agent comes back with the key takeaways of the document and a few questions: which points
matter most, which things should get a page of their own, and whether the document carries
something you have to *do*. Answer in your own words, for example:
```text
Atlas, restic and the 3-2-1 rule are what matter. Nothing to do from this one.
```
The pages it writes next follow your answer, so this is the moment to steer. The agent may well
notice that the notes leave something untested and offer to track it as a task; "nothing to do"
declines that.
Often the same message already asks the two questions of the next step. Then answer them in the
same reply.
## 4. Answer the two capture questions
Before the file leaves `incoming/`, the agent asks two things about the capture itself:
- **`fidelity`** - how faithful is this file to the original? The sample is the notes exactly as
written, so the answer is `verbatim`.
- **`authority`** - what may the material claim about its subject? The notes describe how a setup
works; they do not set a standard anyone else has to follow. The answer is `reporting`.
```text
verbatim, reporting
```
The agent never guesses these two. Only you know them, and only now: once the file is in the wiki,
nobody can tell any more where it came from. [raw/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/raw/CONTRACT.md#capture-fields-fidelity-and-authority)
lists every value, for the sources you add later.
## 5. Let the agent publish
The agent now moves the file into `raw/`, writes the pages, and publishes everything as one
commit. For a source this small that goes straight through; the agent tells you what it committed.
If your instance has no remote, the commit stays on your machine.
A larger source - ten or more new or changed pages and files, not counting the generated indexes
and log - stops at the Mass-Update Gate instead: you see the list of files and a line ending in
`--confirm <token>`, and nothing is committed until you tell the agent to go ahead.
## 6. Look at what it wrote
Ask the agent where the pages are, or look yourself. You will find:
- **The file itself under `raw/`**, in a folder for the current year and month. It is never edited
again.
- **A source page under `kb/sources/`**, summarising the notes. Its frontmatter carries your two
answers, `fidelity: verbatim` and `authority: reporting`, and `raw_files:` points at the file in
`raw/`.
- **Entity and concept pages** for what you said matters - Atlas, restic, the 3-2-1 backup rule,
in the language and naming your instance chose at setup. Each one links back to the source page,
and each claim taken from the notes carries a footnote citing it.
- **A new entry at the end of `kb/log.md`**, recording the ingest.
The exact set of pages depends on your answer in step 3, and on the agent: two runs of this
tutorial do not produce word-for-word the same wiki. What is always the same is that every page is
tied back to the file in `raw/`.
## 7. Ask a question
Start a new agent session, so that the answer can only come from the wiki and not from what the
agent remembers of the ingest, and ask:
```text
How are the backups of Atlas protected against a fire in the house?
```
The answer names the storage box outside the house as the copy that survives, and the 3-2-1 rule
as the reason it exists. It cites the pages it came from and the source behind them. Open one of
the cited pages and follow its footnote: it leads to the source page, and that one to the file in
`raw/` - the notes you copied in step 1.
Now ask something the notes do not answer:
```text
How long does a full restore of Atlas take?
```
The notes say only that one has never been tried. The agent tells you that the wiki has no source
for this, rather than estimating one. That is the rule behind every answer: no claim without a
source.
## 8. Optional: browse it in Obsidian
Open the instance's root folder as a vault in [Obsidian](https://obsidian.md). The graph view shows
the pages you just made and the links between them. Read there as much as you like, but change
pages through the agent: [Browse the wiki in Obsidian](How-to-Browse-with-obsidian) explains
why.
## What next
You have taken one source all the way from a file to a cited answer. From here on, the how-to
guides cover the everyday tasks:
- [Ingest a source](How-to-Ingest-a-source) - your own files, folders and web pages
- [Query the wiki](How-to-Query-the-wiki) - and search it yourself
- [Maintain the wiki](How-to-Maintain-the-wiki) - the lint the agent suggests every ten
sources
Tutorials-Sample-source.md
+50
@@ -0,0 +1,50 @@
*Generated from [`docs/tutorials/sample-source.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/docs/tutorials/sample-source.md) at v9.0.0 – edit the repository, not this wiki.*
# Home lab backups - how they work now
Notes written after reworking the backups in September 2026. This file is the sample source for
the tutorial "From your first source to your first answer"; the machine and its setup are made
up for it.
## The machine
**Atlas** is the small file server in the hall cupboard: a fanless mini PC with two 4 TB disks
in a mirror, running Debian 13. It holds the family photo archive, scanned documents and the
configuration of every other machine in the house. Everything else on the network is replaceable;
what is on Atlas is not.
## The tool
Backups are made with **restic**, an open-source backup program. restic splits files into chunks,
stores each chunk only once and encrypts everything before it leaves the machine, so the backup
target never sees a file name or a byte of content in the clear. A restic *repository* is the
storage location it writes to; a *snapshot* is one backup run.
restic was chosen over plain `rsync` for two reasons: deduplication keeps the daily runs small,
and encryption makes it safe to put a copy on storage someone else operates.
## How it is set up
Atlas runs restic every night at 02:30 from a systemd timer. Each run writes one snapshot to two
repositories:
- a USB disk that stays plugged into Atlas, for fast restores;
- a rented storage box at a hosting provider, reached over SFTP, as the copy outside the house.
Snapshots are thinned out by `restic forget` after each run: the last 7 daily, 5 weekly and
12 monthly snapshots are kept.
## Why two targets
The setup follows the **3-2-1 backup rule**: keep three copies of the data, on two different
kinds of storage, with one copy off site. Here the three copies are the mirror on Atlas itself,
the USB disk, and the storage box. A fire or a burglary takes the first two; the storage box is
what is left.
The mirror alone does not count as a backup. It protects against a dead disk, but a deleted or
encrypted file is deleted or encrypted on both disks at once.
## What is still open
The restore has been tried once, for a single folder, from the USB disk. It worked and took
under a minute. A full restore from the storage box has not been tried.
Tutorials.md
+15
@@ -0,0 +1,15 @@
*Generated from [`docs/tutorials/README.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v9.0.0/docs/tutorials/README.md) at v9.0.0 – edit the repository, not this wiki.*
# Tutorials
Learning-oriented pages: a guided first walk through an instance, start to finish, for someone
who has not used one before. A tutorial shows one path that works; it does not list options or
explain why.
Take them in this order:
1. [Install a Chemenu instance](Install) - from an empty folder to a running instance.
It stays at the repository root, where someone who has nothing else yet finds it first.
2. [From your first source to your first answer](Tutorials-First-source-to-first-answer) - ingest a
sample document, look at the pages it became, and ask the wiki a question about it.
[sample-source.md](Tutorials-Sample-source) is the document it uses.
_Sidebar.md
+48 -9
@@ -1,10 +1,49 @@
* [Chemenu - Personal IT Knowledge Base](Home)
* [Installation](Install)
* Documentation
* [Why knowledge and commitments are two layers](Knowledge-and-commitment)
* [Language Boundaries](Language-boundaries)
* [Choosing a Claude Code model and effort level](Model-and-effort-selection)
* [Ownership and Templates](Ownership-and-templates)
* [Why the pipeline has four stages](Pipeline-rationale)
* [Why the stack version splits compatibility from migration](Version-model)
* [Why gates are code](Why-gates-are-code)
* [Install a Chemenu instance](Install)
* [Tutorials](Tutorials)
* [From your first source to your first answer](Tutorials-First-source-to-first-answer)
* [Home lab backups - how they work now](Tutorials-Sample-source)
* [How-to guides](How-to)
* [Browse the wiki in Obsidian](How-to-Browse-with-obsidian)
* [Capture documentation from a git repository](How-to-Capture-repository-docs)
* [Export guidelines to captured repositories](How-to-Export-guidelines)
* [Ingest a source](How-to-Ingest-a-source)
* [Maintain the wiki](How-to-Maintain-the-wiki)
* [Query the wiki](How-to-Query-the-wiki)
* [Run the weekly review](How-to-Run-the-weekly-review)
* [Set up the MCP read server](How-to-Set-up-the-mcp-server)
* [Trace and score agent sessions](How-to-Trace-and-score-sessions)
* [Troubleshoot an instance](How-to-Troubleshoot)
* [Update an instance](How-to-Update-an-instance)
* [Reference](Reference)
* [Command reference](Reference-Commands)
* [Catalog and log](Reference-Commands-Catalog-and-log)
* [Content migrations](Reference-Commands-Content-migrations)
* [Distribution and versioning](Reference-Commands-Distribution-and-versioning)
* [Finding and checking](Reference-Commands-Finding-and-checking)
* [Git](Reference-Commands-Git)
* [Guideline export](Reference-Commands-Guideline-export)
* [Instance health](Reference-Commands-Instance-health)
* [Links and citations](Reference-Commands-Links-and-citations)
* [Pages](Reference-Commands-Pages)
* [Provenance](Reference-Commands-Provenance)
* [Raw material and uploads](Reference-Commands-Raw-material-and-uploads)
* [Telemetry](Reference-Commands-Telemetry)
* [Types, instructions and docs](Reference-Commands-Types-instructions-and-docs)
* [Workshop runs and session budget](Reference-Commands-Workshop-runs-and-session-budget)
* [Configuration](Reference-Configuration)
* [Exit codes and gates](Reference-Exit-codes)
* [Page types](Reference-Page-types)
* [Skill catalogue](Reference-Skills)
* [Explanation](Explanation)
* [Architecture](Explanation-Architecture)
* [Why knowledge and commitments are two layers](Explanation-Knowledge-and-commitment)
* [Language Boundaries](Explanation-Language-boundaries)
* [Choosing a Claude Code model and effort level](Explanation-Model-and-effort-selection)
* [Ownership and Templates](Explanation-Ownership-and-templates)
* [Why the pipeline has four stages](Explanation-Pipeline-rationale)
* [Telemetry and evaluation](Explanation-Telemetry-and-evaluation)
* [Why the stack version splits compatibility from migration](Explanation-Version-model)
* [Why gates are code](Explanation-Why-gates-are-code)
* Dev
* [Developing this stack](Dev-Development)